Redesign of The Linux Driver For The Mesa Anything IO Cards.
Redesign of The Linux Driver For The Mesa Anything IO Cards.
Redesign of The Linux Driver For The Mesa Anything IO Cards.
Jeffry Molanus
Pre-doctoral assignment
Supervisors:
prof.dr.ir. J. van Amerongen
dr.ir. J.F. Broenink
ir. M.A. Groothuis
October 2007
This report contains the results of the investigation to the relevant hardware differences as
well as software differences. Based up on these results, and certain requirements a new driver
is written since updating the existing drivers will cost just as much, or maybe more effort then
writing a new one. Besides the effort involved the control laboratory wishes to have just one
driver that supports all of these cards on the latest stable kernel version. Since the 4I68 card
has some relevant differences regarding resources, internal data structures of both old drivers
have to be changed, justifying the rewriting even more.
As a test, the newly developed driver replaced one of the old drivers in a working control
setup; the Paper Path. The card is tested for any abnormal behaviour. The new card, the 4I68
is also tested in conjunction with the 4I65 card to test if the new card works, and to see if having
multiple different cards works as well.
The recommendation is to remove the control logic off the driver from kernel space and to
use user space I/O. This will enable a tighter integration with for instance, the 4C tool chain
and improve performance of the Anything I/O card.
Er zijn verschillende versies van de kaart beschikbaar bij de control vakgroep, de Anything
I/O 4I65, 5I20 en de 4I68. Voor laatst genoemde is geen Linux driver beschikbaar. De andere 2
kaarten, de 4I65 en de 5I20 worden ondersteund door twee verschillende drivers. Elke individu-
ele driver heeft zo zijn voordelen maar beiden drivers hebben geen ondersteuning voor memory
mapped I/O.
In dit verslag staan de resultaten van het onderzoek naar de relevante hardware verschillen
als mede naar de software verschillen. Op basis van deze resultaten en bepaalde vereisten is er
een nieuwe driver geschreven omdat het updaten van de bestaande drivers net zoveel of miss-
chien wel meer in spanning vergt dan het schrijven van een nieuwe. Naast de in spanning die er
mee gepaard gaat, wenst de control groep één driver die alle kaarten ondersteund op de nieuwste
stabiele kernel versie. Omdat de 4I68 relevante verschillen heeft met betrekking tot de gebruikte
bronnen, moeten de interne data structuren van beide drivers worden aangepast die het her-
schrijven nog meer rechtvaardigen.
Als een test, wordt de nieuw ontwikkelde driver vervangen voor de oude driver in een werk-
ende opstelling; De Paper Path. De kaart is getest op vertoon van abnormaal gedrag. De nieuwe
kaart, the 4I68 is ook getest is samenwerking met de 4I65 kaart, om zo te testen of de nieuwe
kaart werkt, en om te kijken of het hebben van meerdere en verschillende kaarten in één op-
stelling ook werkt.
De aanbeveling is om de controle logica van de driver uit kernel space te verwijderen en user space
I/O te gebruiken. Dit maakt het mogelijk om betere intergratie te krijgen met, bijvoorbeeld de
4C tool chain en om prestatie verbeteringen te verkrijgen.
1 Introduction 1
1.1 Approach of the project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Report outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
6 Conclusions 16
6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.2 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
A Template Changes 17
A.1 Function changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
A.2 Initialising the card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
A.3 The registers structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Bibliography 19
Figure 1.1: Example of the Anything I/O card in a real setup (JIWY)
The goal of this project is to compare both drivers and determine their best features, and merge
these features in to one of the existing drivers or maybe to write a new Linux driver from scratch.
Furthermore support must be added for the 4I68 Anything I/O card.
Figure 2.1: Overview of the Anything I/O 4I65 en 5I20 card (Groothuis, 2004)
When looking in table 2.1 the most relevant difference is the PLX chip used on the 4I68, in
comparison to the PLX used on the 4I65 and the 5I20 it has some power management capabilities.
Also the 4I68 has less LEDs available for programming. Since the 4I68 uses a different type of
FPGA in comparison with the two other boards the voltage level is lower. The 5I20 uses the
normal PCI bus, and has a different form factor. This however does not make any difference
for its functionality nor does it require any extra measures for the Linux driver. The pin out
of the cards differ all together. Please refer to the PIN files provided by Mesa Electronics. The
interconnect of the buses are depicted in figure 2.1.
The PLX PCI bridge local configuration registers can be accessed via I/O or memory.
Table 2.2: Default PLX settings for PLX configuration and FPGA control register manipulation
The PLX PCI bridge allows for separate memory and I/O regions to be mapped to the local
bus that connects to the FPGA.
Through these BARs the driver has provide the application developer full control over all FPGA
registers. Another relevant difference is that the 4I68 cannot be reseted by means of software.
The reason for this is that there are only 2 I/O pins available for controlling the programming
procedure of the FPGA. This leads to a requirement for resetting the FPGA by means of soft-
ware, namely that the DISABLECONF pin is set to float in the FPGA configuration. However,
a design error was made during the development of the 468I board by Mesa, the wrong resistor
values where used on the board which leads to not being able to reset the board by any means
of software regardless of the DISABLECONF state.
Since the architecture used with the Anything I/O cards is x86 based, reaching the whole
I/O space with register based I/O is impossible. More detail about Memory Mapped I/O
will be given in chapter 3.
• Support for multiple and different cards
Some physical setups may require multiple and different Anything I/O cards on one
PC/104-plus stack.
• Support for kernel versions range
The Control engineering laboratory makes use of different kernel versions. Support has to
be added from Linux 2.6.10 till 2.6.20 (currently the latest).
Besides these requirements some changes need to be made to the card detection routines that
are part of the driver since the functions currently used in both drivers are deprecated. Since the
2.6 Linux kernel, a new and more structured method was introduced to scan and detected the
hardware. The new driver should use these functions, and doing so, enable the use of different
cards with just one driver. Other changes should be made on how the card is programmed. Since
programming and actually using the I/O card can be seen as two different devices, the driver
should make this difference visible in user space by means of two separate device files. This makes
programming the FPGA in principle the same as the first driver developed by Marcel Groothuis
but with a different implementation. This is the preferred approach since programming the
FPGA wonŠt require any user space tools.
2.5 Strategy
Looking at the requirements and changes to the detection routines, the choice for starting from
scratch becomes apparent. The best arguments for this choice are the fact that the API of the
2.6 kernel has changed a lot. Furthermore the internal data structures of the anyio proc and
the anyio old driver would need a redesign in order to support the 4I68 card. Besides that, the
Control engineering laboratory wishes to have just one driver that supports all of Anything I/O
card. So a new driver will be written from scratch as opposed to updating both of the old drivers.
A new user space library will be written that will accommodate the control application de-
veloper with a predefined set of functions needed to get full functionality of the Anything I/O
cards. The library will also provide a structured fashion to use several and different cards all
together.
Once a driver resides in the kernel, and thus is part of the kernel space, there is a need to
communicate with this piece of hardware by means of the earlier explained system calls. Using
the ioctl() system call, which stands for Input Ouput ConTroL, the user space application is
able to send a structure of data in to the kernel. The driver has to implement the ioctl() call
itself. It has to make a clear distinction between the different arguments passed and has to
determine if the request made is sane. In case of the Anything I/O driver the ioctl() call, will
call yet another function which performs the actual I/O on the hardware. In this particular case
the inX() and outX() functions, where X stands for byte, word or dword, see figure 3.2.
The coverage of memory mapped I/O will not go in to great detail since this involves memory
management in general and that is the hardest and most complex part of the Linux kernel.
However, when using memory mapped I/O, the CPU uses the same address bus and thus in-
structions to write into memory and I/O registers as opposed to direct I/O where the data is
passed directly in to the registers of the device. Direct I/O has a limited address space. What
basically happens is that the hardware registers that cannot be reached by the conventional
function calls inX and outX, are mapped to (virtual) memory. The reason why this is being
used at all is that the CPU needs to reserve address for I/O operations. A trade off is there for
chosen between directly addressable I/O registers and physical ram, when there is need for more
I/O addresses memory mapped I/O is the solution. Also, this makes writing device drivers in
C is a lot easier. Reason for this is that registers can be manipulated like if they were normal
variables.
Figure 3.3: Data path difference between direct I/O and memory mapped I/O
In figure 3.3 two different data paths are illustrated to give insight on what the fundamental
difference is between direct I/O and memory mapped I/O. The arrow (1A) and (1B) depict the
path for memory mapped I/O, the data goes in to main system memory and is passed from
there on to the device. The second path (2) depicts the direct I/O path.
With the right (root) permissions, a user space program is able to directly write to the first
256 addresses omitting the driver all together, this delivers a speedup of about 66% (Groothuis,
2004) The data flow is depicted in figure 4.1. The direct flow is given by (B). However the first
addresses can also be reached utilizing the kernel driver, this path is depicted by (B), (C) and
(D). Depending on the address where the data is written the next step is either to the FPGA of
the LEDs. When using memory mapped I/O there is only one path available namely path (A)
to (D), and again depending on the address path data is send or read from the FPGA or the
LEDs.
char anyio read byte(struct registers *card, int offset) read one byte at offset
short anyio read word(struct registers *card, int offset) read two bytes at offset
int anyio read dword(struct registers *card, int offset) read four bytes at offset
void anyio write byte(struct registers *card, char byte, int offset) write byte at offset
void anyio write word(struct registers *card, short word, int offset) write two bytes at offset
void anyio write dword(struct registers *card, int dword, int offset) write four bytes at offset
int anyio status(struct registers *card) returns the status of the card
int anyio open(char *devname, struct registers *card, int directio) open the card for usage
8 struct r e g i s t e r s c a r d 1 ; /∗ card 1 ∗/
9 struct r e g i s t e r s c a r d 2 ; /∗ dummy card f o r example r e a s o n s ∗/
10
11 a n y i o o p e n ( ”/ dev / a n y i o 0 ” , &card1 , 0 ) ;
12 a n y i o s t a t u s (& c a r d 1 ) ;
13 a n y i o w r i t e b y t e (&card1 , 0 x00 , 1 6 ) ; /∗ c o n f i g u r e as o u t p u t ∗/
14 a n y i o w r i t e b y t e (&card1 , 0 x f f , 1 6 ) ; /∗ c o n f i g u r e as o u t p u t ∗/
15 a n y i o r e a d b y t e (&card1 , 1 6 ) ;
16 p r i n t f ( ”r e a d %c \n ” , c a r d 1 . datab . v a l u e ) ;
17 /∗ same r e s u l t as l i n e a bo ve ∗/
18 p r i n t f ( ‘ ‘ r e a d %c \n ’ ’ , a n y i o r e a d b y t e (&card1 , 1 6 ) ;
19
20 while ( 1 )
21 {
22 a n y i o w r i t e b y t e (&card1 , 0 , 0 ) ;
23 sleep (1);
24 a n y i o w r i t e b y t e (&card1 , 0 xFF , 0 ) ;
25 sleep (1);
26 }
27 }
Listing 4.1: Example program that oscillates output pins
To make use of the Anything I/O cards, a structure of the type register has to be declared. This
structure holds the addresses of the registers used by the Anything I/O card, for more details
about this structure please refer to Appendix A, or refer to the appropriate header file(s). To
use multiple cards of different types, another declaration of the type registers is needed, in the
example program this is illustrated by the declaration of the dummy card. This dummy card
could be used as a real card and be manipulated in the same manner as card1.
4.3 Conclusion
Based upon the requirements and the proposed improvements, a new driver is written. The
driver supports all of the Anything I/O cards used at the control engineering laboratory, the
5I20, 4I65 and the new 4I68. All of these cards can mixed together and are usable with just
one new driver. All of these cards now support memory mapped I/O enlarging the maximum
address space of the FPGA configuration.The programming method of the anyio old driver has
been used and, a new user space library has been written to provide the application developer
with a set of functions to make use of the card. Testing of the driver with a real setup is done
in Chapter 5.
The Paper Path setup currently uses the anyio proc driver, which was replaced by the anyio v3
driver. The control software generated by 20-sim and the 4C tool chain is adapted to use the
newly developed anyio library. In order to achieve this, only the template used by the 4C tool
chain is changed. The changes can be found in Appendix A. Section 5.1 will cover the steps and
requirements in order to load the driver. Section 5.2 is about programming the FPGA. Finally
in section 5.3 the tests preformed are described and section 5.4 concludes this chapter.
After this final step, the paper path is ready to use in combination with the new driver. Sheets
of paper are placed in the paper path input module. And the setup is monitored for any odd
behavior. The 4I68 card is also tested. There is no working setup like the paper path to test the
card in. So in order to test the new card, a simple program that toggles the output pins of the
Anything I/O card is developed. The 4I68 card is then stacked on to the paper path PC/104
and both Anything I/O cards are tested simultaneously. At the output pins an oscilloscope is
connected. The complete setup is depicted in figure 5.2 the results of the scope are shown in
5.3.
5.4 Conclusion
The anyio v3 driver works well on both of the Anything I/O cards. No odd behavior is seen
during the test of the Paper path. Using different cards at the same time also is a success.
Conclusion of the test, therefore, is that the new driver supports all of the boards used at the
Control engineering laboratory and is able to replace the anyio proc and the anyio old driver
without breaking the setup. Small changes are required to the 4C templates of setups that
currently make use of the old drivers.
6.2 Recommendations
Doing I/O in user space would improve the overall performance of the Anything I/O cards.
If the driver can be controlled outside kernel space, more higher level libraries are available
to control the card and give it a tighter integration with, for instance the 4C tool chain. UIO
(User space I/O) is now part of the main kernel branch and should be used in future development.
The new driver uses the ioctl() system call interface which allows passing data structures to
the driver. Each time a system call is called, the kernel switches to kernel space and serves the
request. Each kernel switch itself costs time and performing the services costs time. Looking at
the dominant behavior of the Anything I/O driver, all it does is read and write to registers with
a fixed address. Doing I/O in user space would eliminate the need for these switches. During
development of the new Anything I/O driver kernel facilities to accommodate for this where
being developed. Currently programming the card is initiated in user space however, the actual
implementation is in kernel space, handling files in kernel space in general is discouraged. The
whole implementation of programming the card could become part of the 4C tool chain instead.
These 2 function calls have to be changed in order to work with the new driver and the new
Anyio library in to:
1 a n y i o r e a d d w o r d (&card1 , ADPDenBase ) ;
2 a n y i o w r i t e d w o r d (&card1 ,&RegVal ,ADMCR) ;
Listing A.2: new functions
3 struct r e g i s t e r s
4 {
5
10 int f d ; /∗ f i l e d i s c r i p t o r t o w r i t e t o ∗/
11 u32 f p g a 1 6 b a s e ;
12 u32 f p g a 3 2 b a s e ;
13 char d i r e c t i o ; /∗ e n a b l e d i r e c t i /o ∗/
14 };
15
16 struct a n y i o d a t a b
17 {
18 u8 v a l u e ; // v a l u e t o w r i t e or r e t u r n
19 u32 o f f s e t ; // p o r t o f f s e t from b a s e a d d r e s s
20 u8 i o t y p e ; // s e e d e f i n e s ( p l x , f p g a 1 6 o f f p g a 3 2 i o p o r t s )
21 };
22
37 struct a n y i o s t a t u s
38 {
39 u32 f p g a 1 6 b a s e ;
40 u32 f p g a 3 2 b a s e ;
41 int is programmed ;
42 char c o n f i g n a m e [NAME LEN] ;
43 int minor ;
44
45
46 };
Listing A.3: Anyio data structures
Kuppeveld, T.G.M.G.M. van, Sprik, E. (2006), “Design and Realization of the ViewCorrect
plotter”, Pre-doctoral assignment-Report 033CE2006, Control Laboratory, University of Twente,
October 2006
Otto, M. , ”Mechatronic setup for Boderc project”, MSc.-Report 038CE2005, Control Labo-
ratory, University of Twente, October 2005
Groothuis, M.A. (2004), “Distributed HIL simulation for BodeRC”, MSc.-Report 020CE2004,
Control Laboratory, University of Twente, August 2004
Buit, E. (2005), “PC104 stack mechatronic control platform”, MSc.-Report 009CE2005, Con-
trol Laboratory, University of Twente, April 2005
Visser, P. M., M. A. Groothuis, et al. (2006), ”Multi-purpose toolchain for embedded con-
trol system code on a variety of targets.”
Corbet, J., Rubini, A., Kroah-Hartman G. (2005), “Linux Device Drivers, Third Edition”,
O’Reilly Media, Inc ISBN: 0-596-00590-3