%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% STREET ADDRESS (REQUIRED)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-... Praha, Czech Republic\\
+Technick\'{a} 2, 166 27 Praha 6, Czech Republic FIXME\\
+%Karlovo náměstí 13, 121 35 Praha 2
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% E-MAIL (REQUIRED)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% AFFILIATION (REQUIRED)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-Czech Technical University in Prague\\
+Czech Technical University in Prague, Faculty of Electrical Engineering\\
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% STREET ADDRESS (REQUIRED)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-... Praha, Czech Republic\\
+Technick\'{a} 2, 166 27 Praha 6, Czech Republic\\
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% E-MAIL (REQUIRED)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
When teaching development of Linux drivers one of the approaches is to explain kernel API and programming paradigms by creating driver which doesn't require some special hardware -- e.g. character driver which returns upper case ASCII text when receiving lower case. Although this approach can be useful, the issues associated with dealing with hardware should be practices as well.
-The approach we took in this work eliminates need of physical access to hardware whereas it provides full feature set of PCI device in form of virtual hardware.
+The approach we took in this work eliminates the need of physical access to hardware whereas it provides full feature set of PCI device in form of virtual hardware. This was possible by implementing virtual PCI device into Qemu emulator.
-Main reason of choosing DAQ cards for this project was ease of interfacing from programmers view. This can be very helpful for beginners who are not familiar with hardware related topics.
+Main reason of choosing DAQ cards for this project was ease of interfacing from programmers view and straightforward testing of proper function of the driver. This can be very helpful for beginners who are not familiar with hardware related topics.
\section{Humusoft MF614, MF624}
Humusoft MF614 and MF624 are data acquisition (DAQ) cards. Both of this cards use PCI interface to connect to computer. The main features this cards provide are \textit{digital inputs}, \textit{digital outputs}, \textit{ADCs}, \textit{DACs}, \textit{timers}, \textit{encoder inputs}. Humusoft MF614 is predecessor of MF624 -- available functions are quite similar. There is main difference in driver programming -- MF614 has only 8-bit wide registers, whereas MF624 16- or 32-bit wide.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{UIO driver}
-Each UIO driver consists of two parts -- small kernel module (its main purpose is interrupt handling/disabling) and user-space driver logic (as shown in figure 1). This approaches main advantage is that the most of the development happens in user-space, thus during prototyping the driver (or when using bad one) it won't withdraw kernels integrity and stability.
+Each UIO driver consists of two parts -- small kernel module (the need for it is mostly because of device-specific interrupt handling/disabling) and user-space driver logic (as shown in figure 1). This approaches main advantage is that the most of the development happens in user-space, thus during prototyping the driver (or when using bad one) it won't withdraw kernels integrity and stability.
\epsin{img/uio}{80}{fig1:uio}{UIO driver structure}
-\subsection*{Implementing kernel part}
+\subsection*{Implementing the kernel part}
In case of writing UIO driver for PCI device, initialization function of the module registers \texttt{struct pci\_driver} in standard way, where the probe function handles initialization of UIO-related structures. The main structure holding all data of UIO driver is \texttt{struct uio\_info}. Its simple initialization (including registration) is shown below:
\begin{verbatim}
\end{verbatim}
Structure \texttt{uio\_mem} is used for enabling memory-mapped I/O regions (MMIO), whereas structure \texttt{uio\_port} is used for port-mapped I/O (PMIO).
+\subsection*{Driver \texttt{uio\_pci\_generic}}
+When dealing with any device compliant to PCI 2.3, it is also possible to use \texttt{uio\_pci\_generic} driver in kernel instead of programming specific one. This driver makes all memory regions of the device available to user-space.
+
+Binding to the device is done by writing Vendor and Device ID into \texttt{/sys/bus/pci/drivers/ uio\_pci\_generic/new\_id} file.
+
+Interrupt handler uses Interrupt Disable bit in the PCI command register and Interrupt Status bit in the PCI status register. Because none (FIXME any?) of MF614 or MF624 are PCI 2.3 compliant it is not possible to use this driver for them.
+
\subsection*{Interface to user-space}
Communication with kernel part of the UIO driver is possible through \texttt{/dev/uioX} file (where X is number of instance of driver). There are several syscalls possible to use when interfacing with this file:
\begin{description}
\item[\texttt{open()}] opens the device, returns file descriptor used for another syscalls.
\item[\texttt{read()}] blocks until an interrupt occurs (the value read is number of interrupts seen by device).
\item[\texttt{mmap()}] is used to map devices memory to user-space. The offset value passed to \texttt{mmap()} de\-ter\-mines the memory area of a device to map -- for \textit{n-th} area offset should be \textit{n*\texttt{sysconf( \_SC\_PAGESIZE)}}.
-\item[FIXME] disabling interrupts
+\item[\texttt{irqcontrol()}] is used for enabling (called with parameter \texttt{(int) 1}) or disabling (parameter \texttt{(int) 0}) interrupts.
\end{description}
+It is possible to define your own \texttt{mmap()}, \texttt{open()}, \texttt{release()} functions as an option. When there is need to use \texttt{irqcontrol()}, it is necessary to implement this function per device.
+
+When using UIO and \texttt{mmap()} with MF624 card (which has 32 or 128 bytes long memory regions) there is an issue with the return value of this syscall -- the pointer to the memory seems to be page-size-aligned, so it was necessary to add low bits of physical address of each memory region to it. FIXME
Another way how to gain information from some UIO driver is by reading \texttt{/sys/class/uio/uioX}. Most of the files are read-only. The subdirectory \texttt{maps} contains information about MMIO regions mapped by the driver, subdirectory \texttt{portio} is for PMIO.
\item[Kcomedilib] is also part of Linux kernel. It provides the same API as Comedili, whereas this is used for real-time applications.
\end{description}
-\subsection*{Implementing driver}
+\subsection*{Implementing the driver}
Each Comedi driver should register to the list of active Comedi drivers. This is done by invoking \texttt{comedi\_driver\_register()} function. The only parameter passed to this function is pointer to \texttt{struct comedi\_driver} structure. The most important fields of this structure are:
\begin{verbatim}
const char *driver_name; /* "my_driver" */
There are already applications using Comedi API\footnote{For basic list of available applications see http://www.comedi.org/applications.html} -- in some cases there is no need for implementing user-space application from scratch.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Qemu virtual hardware}
-Qemu is open-source processor emulator. Unlike common virtualization solutions it is able of emulating x86, x86-64, ARM and other widespread processor architectures. For the purposes of this work it was used for implementing the emulation of Humusoft MF624 DAQ card.
-
-\epsin{img/qemu}{80}{fig2:qemu}{Qemu implementing virtual MF624 device}
+Qemu is open-source processor emulator. Unlike common virtualization solutions it is able of emulating x86, x86-64, ARM and other widespread processor architectures. For the purposes of this work it was used for implementing virtual Humusoft MF624 DAQ card.
+\subsection*{Usage}
When running any guest operating system in Qemu (with support for MF624 activated) the virtual MF624 device is available in the same way as if it was real hardware -- there are no issues with interfacing between guest operating system and virtual device. Interfacing between virtual hardware and \textit{real world} is handled by TCP/IP connection from MF624 module in Qemu to \textit{host} operating system. It is used for reading/setting output/input values (as shown in figure 2). The most fundamental way of communication through this channel is by using \texttt{telnet} application. Example of real communication:
\begin{verbatim}
$ telnet localhost 55555
telnet> Connection closed.
\end{verbatim}
-As much more easier way of interfacing, there is also graphical application created just for purposes of communicating with virtual MF624 card (see figure 3). It was created using Qt4 graphical toolkit.
+As a much more easier way of interfacing, there is also graphical application created just for purposes of communicating with virtual MF624 card (see figure 3). It was created using Qt4 graphical toolkit.
+
+\epsin{img/qemu}{80}{fig2:qemu}{Qemu implementing virtual MF624 device}
+
+
+\subsection*{Implementation of virtual MF624}
+When creating new virtual device in Qemu, main hook into Qemu infrastructure is done by invoking \texttt{device\_init()} with parameter of pointer to initialization function with prototype of \texttt{static void (*)(void)}. For registering new PCI device, it is necessary to call \texttt{pci\_qdev\_register()} passing parameter of pointer to \texttt{PCIDeviceInfo}. The most important fields of this Qemu-specific data type are pointers to \textit{init} and \textit{exit} functions with prototype of \texttt{int (*)(PCIDevice *)}.
+
+The PCI device specific initialization consists of:
+\begin{itemize}
+\item Initializing configuration space of PCI device -- e.g. setting Vendor and Device IDs, device class, interrupt pin.
+\item Registration of I/O memory used by the device.
+\item Initialization of physical memory and mapping it to particular \textit{BARs} (Base Address Registers) of PCI device.
+\end{itemize}
+
+The very basic (non-compilable) example:
+
+\end{multicols}
+\begin{verbatim}
+ 1 | static CPUReadMemoryFunc * const mf624_BAR0_read[3] = { NULL, NULL, mf624_BAR0_read32 };
+ 2 | static CPUWriteMemoryFunc * const mf624_BAR0_write[3] = { NULL, NULL, mf624_BAR0_write32 };
+ 3 |
+ 4 | static void mf624_map(PCIDevice *pci_dev, int region, pcibus_t addr, pcibus_t sz, int tp)
+ 5 | {
+ 6 | mf624_state_t *s = DO_UPCAST(mf624_state_t, dev, pci_dev);
+ 7 | cpu_register_physical_memory(addr + 0x0, BAR0_size, s->BAR0_mem_table_index);
+ 8 | }
+ 9 |
+10 | static int pci_mf624_init(PCIDevice *pci_dev)
+11 | {
+12 | mf624_state_t *s = DO_UPCAST(mf624_state_t, dev, pci_dev); /* i.e. container_of() */
+13 | uint8_t *pci_conf;
+14 |
+15 | pci_conf = s->dev.config;
+16 | pci_config_set_vendor_id(pci_conf, PCI_VENDOR_ID_HUMUSOFT);
+17 | /* ... */
+18 | pci_conf[PCI_INTERRUPT_PIN] = 0x1;
+19 |
+20 | s->BAR0_mem_table_index = cpu_register_io_memory(mf624_BAR0_read, mf624_BAR0_write,
+21 | s, DEVICE_NATIVE_ENDIAN); /* returns unsigned int */
+22 | pci_register_bar(&s->dev, 0, BAR0_size, PCI_BASE_ADDRESS_SPACE_MEMORY, mf624_map);
+23 | return 0;
+24 | }
+25 |
+26 | static PCIDeviceInfo mf624_info = {
+27 | .qdev.name = "mf624", .qdev.size = sizeof(mf624_state_t),
+28 | .init = mf624_init, .exit = mf624_exit,
+29 | };
+30 |
+31 | static void reg_dev(void) { pci_qdev_register(&mf624_info); }
+32 | device_init(reg_dev)
+\end{verbatim}
+\begin{multicols}{2}
+
\epsin{img/qt_gui}{70}{fig3:qemu2}{Graphical application used for interfacing between virtual MF624 and \textit{real world}}
-\section{Conclusions}
+\section{Conclusion}
The outcome of this work creates basic integrated tool for teaching PCI driver development under GNU/Linux operating system. Its main advantage is possibility to train driver development on \textit{real hardware} without the necessity of having expensive DAQ device. The other advantage is safe environment for driver prototyping -- where no mistake can damage host operating system.
All the information (including source code) related to topic covered in this article are publicly available on web page \texttt{rtime.felk.cvut.cz/ hw/index.php/Humusoft\_MF6xx}
projects / mf6xx.git / commitdiff