2 % Typographic conventions (already applied)
3 % * Each \caption{} should end with a dot
4 % * "Linux traffic control" will be written with lowercase T and C
6 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
8 \documentclass[11pt,abstract]{scrreprt}
9 \usepackage[utf8]{inputenc}
10 % \usepackage[T1]{fontenc}
11 \usepackage[right]{lineno}
14 \usepackage{longtable}
28 \usepackage[draft]{fixme}
36 %\usepackage{multicol}
38 breaklinks = true, %allow links to break over lines
39 pdffitwindow = true, %resize document window to fit document size
41 % linkcolor = darkblue,
42 % citecolor = darkblue,
43 % urlcolor = darkblue,
51 \RecustomVerbatimEnvironment{Verbatim}{Verbatim}{frame=single,framesep=4pt}
54 \setlength{\columnsep}{8mm}
55 \providecommand{\alert}[1]{\textbf{#1}}
57 \title{Linux-Based CAN-Ethernet Gateway}
58 \author{R. Matějka, M. Sojka\\
59 Czech Technical University in Prague}
60 \providecommand{\version}{???}
61 \date{\today\\Version \version}
63 \newcommand{\superscript}[1]{\ensuremath{^{\mathrm{#1}}}}
64 \newcommand{\subscript}[1]{\ensuremath{_{\mathrm{#1}}}}
65 \renewcommand{\th}[0]{\superscript{th}}
66 \renewcommand{\st}[0]{\superscript{st}}
67 \newcommand{\nd}[0]{\superscript{nd}}
68 \newcommand{\rd}[0]{\superscript{rd}}
71 {\begin{vertbar}\begin{itshape}}
72 {\end{itshape}\end{vertbar}}
75 \lstset{language=sh,frame=single,basicstyle=\tt,commentstyle=\itshape\color{gray},columns=flexible}
86 \label{cha:assignment}
88 The goal is to implement CAN-Ethernet gateway based on Linux's AF\_CAN
89 subsystem with the following features:
91 \item Both user- and kernel-space implementations will be developed.
92 User-space one will contain only the most basic functionality.
93 \item Ethernet side will use UDP datagrams to carry the CAN messages.
94 Later it will be extended to support also TCP.
95 \item Initial version will route all CAN frames to Ethernet side and
96 all received UDP datagrams to CAN messages. Later, filtering
97 capabilities will be added.
98 \item UDP frames will contain timestamps of the time when the CAN
100 \item For kernel-based gateway, implement a user-space configuration
101 tool, similar to \texttt{cangw} tool from \texttt{can-utils}.
105 \chapter{Project overview}
106 \section{Gateway architecture}
108 The software architecture of canethgw is illustrated on figure \ref{fig:swarch}.
109 This section describes what can be seen on this figure.
111 The can-ethernet gateway (canethgw) is comprised of two kernel threads.
112 One thread is serving the can to ethernet routing (can $\rightarrow$ eth)
113 and the other thread is serving the oposite direction (eth $\rightarrow$ can).
115 The gateway uses berkley sockets as network interaces. There are two sockets
116 in the program. One socket is binded to can network and the other one to ethernet. These
117 two sockets are shared by kernel threads.
119 To get a better notion about canethgw operation a working cycle description is given.
120 The working cycle of the can $\rightarrow$ eth kernel thread is as follows. The thread is
121 waiting for incoming data on \emph{can} socket. When data are received, they are
122 put to udp packet and are sent to ethernet. There can be more than one recipient in ethernet
123 network. The program holds list of recipients and sends copy of incoming data
124 to all of them. This working cycle description can be analogically applied to eth $\rightarrow$ can
127 The gateway is configured over netlink with cegw tool. More on cegw is in
128 section \ref{sec:assoc}
130 \includegraphics[scale=0.5]{img/swarch.pdf}
131 \captionof{figure}{Software architecture of canethgw.}\label{fig:swarch}
133 % \bibliography{can-eth-gw}
134 % \bibliographystyle{IEEEtran}
136 %% ASSOCIATED PROGRAMS %%
137 \section{Associated programs}
139 The project consists not only from canethgw, but also from configuration
140 utility cegw, userspace implementation of the gateway, benchmarking program
141 and other utils. These associated programs are presented below.
145 Cegw is a configuration utility for canethgw. Its syntax is similar to
146 cangw tool from can-utils. The difference from cangw is that every
147 interface has to be defined with type-prefix. This can be explained well
148 using an example. Figure \ref{fig:cegwex} displays commands to do a simple
149 setup of canethgw. This setup sends all can frames received on
150 vcan0 to 127.0.0.1:10502 and all can frames received from 127.0.0.1:10501
153 # listen for incomping udp at 127.0.0.1 port 10501
154 cegw --listen udp@127.0.0.1:10501
155 # all from vcan0 route to 127.0.0.1 port 10502
156 cegw --add -s can@vcan0 -d udp@127.0.0.1:10502
157 # all from source address .. route to vcan0
158 cegw --add -s udp@127.0.0.1:10502 -d can@vcan0
160 \captionof{figure}{Simple setup using cegw.}\label{fig:cegwex}
162 \item[userspace implementation]
163 To measure a performance of canethgw, another implementation of
164 gateway was done in userspace. This implementation resembles
165 the kernel implementation but \emph{poll} is used instead of threads.
166 How to start a simple setup is in figure \ref{fig:userspace}.
168 canethgw -s can@vcan0 -d udp@127.0.0.1:10502 -l udp@127.0.0.1:10501
170 \captionof{figure}{Command to run userspace canethgw.}\label{fig:userspace}
173 This is a benchmarking program. It sends can frames to defined interface and
174 listens on another interface for reception. Then it prints the transmission
175 duration to stdout. How to start simple cegwbench session is in figure
176 \ref{fig:runbench}. This program was used for benchmark described in
177 section \ref{sec:benchmark}.
179 cegwbench -s udp@127.0.0.1:10501 -d can@vcan0 -n 100 -m oneattime
181 \captionof{figure}{cegwbench example.}\label{fig:runbench}
184 Sends a can frame over udp to localhost:10501. This is based on cansend
185 from can-utils and the can frame specification string is the same.
189 \chapter{Design issues}
190 This chapter encompasses problems which came up during development.
191 This knowledge may be helpful in further research.
193 The original cangw was designed in modular way, so the first attempt to
194 design canethgw was done on cangw basis. The original cangw works with
195 sk\_buff structure and in softirq. However, there are some issues,
196 which complicate applicability of this approach for canethgw. These
197 problems are listed below:
200 \item[sending udp packet in softirq]
201 The original cangw is receiving can frames in softirq. If one wanted
202 to send such a can frame to ethernet with udp/ip packet, he have to
203 avoid sleeping which is not easy. I have not found a way how to send a
204 udp packet using the protocol stack without sleeping. I also think
205 that bypassing protocol stack and reimplementing udp and ip is not a
206 good idea, because it will induce bugs.
208 \item[receiving udp packet]
209 The gateway have to hook somewhere to receive udp or tcp packets.
210 I have not found any convenient method to do this.
213 These problems led me to search for a different approach. The found solution
214 is based on berkley sockets. Even this approach has some issues, which cause
215 difficuties. These difficulties are listed below. For more complete
216 information about this solution see section \ref{sec:arch}.
219 \item[socket creation]
220 \label{itm:socket_creation}
221 It is not possible to create a \emph{can} socket in a netlink callback. Such an
222 attempt will cause rtnl\_lock deadlock. rtnl\_lock is being
223 held when netlink callback
224 is processing, but the same lock is needed by register\_netdevice\_notifier
225 \cite{register_netdevice_notifier} function when \emph{can} socket is
227 This problem can be solved by postponing the creation process.
229 \item[socket rebinding]
230 Once the socket is binded to a particular address it cannot be
231 changed. Rebinding is not possible. The overcome for this is to release the
232 old socket and create a new one.
234 \item[netlink feedback]
235 This problem is tightly linked to the socket creation issue described above.
236 If some operation cannot be done in netlink handler and have to be
237 postponed, the netlink handler can hardly return any feedback about an
238 operation result. That is why the configuration tool (cegw) doesn't return
239 information about an operation result.
240 \item[terminating thread]
241 The termination of kernel thread is not straightforward in this case,
242 because the needs are slightly different than kernel api. The standard kernel
243 thread is working in a cycle waiting for kthread\_should\_stop(..) to return
244 true. This approach is not possible because the thread can be sleeping in
245 kernel\_sendmsg function and therefore not responding to kthread\_stop. Waking
246 using wake\_process won't work, because no incoming data will cause sleep
247 again. The socket shutdown is used for waking from receiving.
251 \label{sec:benchmark}
252 To evaluate canethgw a benchmark was conducted. The objective was
253 to compare performance of the gateway implementated in
254 userspace and in the kernel.
256 The benchmark was designed as a set of four measurements. There is a separete
257 measurement for each of both directions, that is eth$\rightarrow$can and
258 can$\rightarrow$eth. These both directions are measured with userspace and
259 kernel implementation. In every of four measurements 100 can frames
262 The benchmark was performed on dedicated computer running very bare,
263 single-purpose distribution to avoid external effects. The results show
264 that kernel implementation is approximately 10\% faster. Figure \ref{fig:bench}
265 shows transmission duration of every can frame sent.
267 Furher information, including benchmark source code, can be found in
268 \emph{bench} folder in repository\cite{repo}.
271 \includegraphics[scale=0.35]{img/bench.png}
272 \captionof{figure}{canethgw performance.}\label{fig:bench}
276 \chapter{Repository contents}
277 Repository can be found at \cite{repo}. Below is a list of folders and
280 \begin{tabular}{l p{11cm}}
282 Scripts which prepare and perform benchmark on dedicated computer are
285 Linux distribution used for testing and benchmarking.
286 Busybox and kernel configuration files can be found here. \\
288 Folder with documentation. \\
290 Can-ethernet gateway kernel implementation. \\
292 Contatins scripts used in debuging to test functionality. \\
294 Can-ethernet gateway userspace implementation. \\
295 \textbf{utils/cegw/} &
296 Configuration utility for kernel canethgw. \\
297 \textbf{utils/cegwbench/} &
298 Program used for benchmark. \\
299 \textbf{utils/cesend/} &
300 Sends can frame over udp. \\
304 \begin{thebibliography}{9}
306 \href{https://rtime.felk.cvut.cz/gitweb/can-eth-gw.git}{project repository}
307 \bibitem{register_netdevice_notifier}
308 \href{http://lxr.linux.no/linux+v3.5.3/net/core/dev.c#L1351}{register\_netdevice\_notifier(..)}
309 \end{thebibliography}
316 %%% ispell-local-dictionary: american