1 [[!meta title="Embedded Linux"]]
8 The aim of this exercise is to practice building of the base
9 system from open-source components presented during 2-nd lecture
10 and motivate and practically introduce students for topics presented
11 during 4-th lecture "*Linux kernel - beginning, development, components
12 and device drivers; GNU libc and user space*". If there are some
13 steps or relations clear to you, we recommend you try Google to find
14 information first and prepare to ask questions at the 4-th lecture.
16 Open Source Software (OSS) projects are often not used alone but
17 in combination with other OSS projects, resulting in the so-called OSS stacks
18 or even distributions. Perhaps the best known is the LAMP stack - Linux,
19 Apache, MySQL, PHP. In today's exercise you will learn and experiment
20 with another, very often used, a stack of [Linux][kenrel] + [BusyBox][bb] (+[Dropbear][dropbear] SSH server).
22 [BusyBox][bb] is a set of basic/standard UNIX utilities (shell, editor
23 and such utilities as ls, mkdir, …) compiled into one binary.
24 In combination with the Linux kernel it forms a complete minimal
25 operating system with a relatively small footprint. Thanks to this
26 combination is often used in embedded applications such
27 as [WiFi routers and ADSL modems][owrt].
29 We try to create a complete open source operating system from the
30 Linux kernel and user space environment consisting of just busybox
31 during this exercise. Next, you try to program and load a simple
32 module into the kernel.
34 [kenrel]: http://kernel.org/
35 [bb]:http://busybox.net/
36 [owrt]:http://www.openwrt.org/
37 [dropbear]:http://matt.ucc.asn.au/dropbear/dropbear.html
42 1. Download/clone [source code][bbgit] of [BusyBox][bb] project:
44 git clone git://busybox.net/busybox.git --reference /usr/src/busybox.git
47 Because expanded source tree is a current development snapshot, it is
48 quite possible that some problem is encountered during compilation, installing
49 or use of the project. Our significant advantage is that cloned project
50 repository contains whole project development history
54 and we can choose an older/stable version which is not affected by such problems.
57 git checkout -f 1_18_3
59 [bbgit]:http://git.busybox.net/busybox/
61 2. Next step is to configure BusyBox such way as intended.
65 There is possibility to control build of many components but default
66 configuration is suitable for our task. Selection of `Exit` and confirm
67 by `Yes` to save configuration is enough for our case then.
69 3. Project is compiled by standard command
77 installs busybox into local directory `./_install`. Notice, please,
78 that there is only single binary executable file `bin/busybox` and all
79 other directory contents are symbolic links to that binary file.
81 Because we have not used *cross compilation*, which is quite often in case
82 of build for embedded devices, it is possible to test functionality of
83 our result of *native build* directly on build machine. The shell
84 `./_install/bin/sh` can be run for example (it can be terminated by `exit` command).
86 In the case of a real embedded system would have to go on and test the BusyBox
87 after booting on the target hardware.
89 5. If you can use supervisor (root) rights on build system, then it is possible
90 to test BusyBox in [chroot environment][chroot], i.e. with actual running kernel
91 instance, but with minimal filesystem populated during BusyBox install:
93 # chroot _install /bin/sh
95 But you find that the attempt to execute BusyBox would fail, because there are
96 necessary system libraries for default/non-static build of BusyBox and these
97 libraries are not available in `_install` substree after chroot.
99 [chroot]:http://en.wikipedia.org/wiki/Chroot
101 6. Next command can be used to determine missing libraries and their paths
103 ldd _install/bin/busybox
105 These libraries must be copied to the directory `_install`. The command
106 to copy libraries can look like line below on 32-bit system:
109 cp /lib/i686/cmov/libm.so.6 /lib/i686/cmov/libc.so.6 /lib/ld-linux.so.2 _install/lib
111 The ELF file "interpreter" (`/lib64/ld-linux-x86-64.so.2`) is searched
112 in directory `/lib64` on 64-bit system by kernel.
113 The `/lib64` can be substituted by symbolic link to directory `/lib`
115 ( cd _install && ln -s lib lib64 )
117 The BusyBox can be executed in chroot environment after libraries setup.
119 5. The easiest way how to boot into the user environment you just created
120 is to store pack directory tree in the Linux kernel initial RAM-disk
121 format and boot Linux with this RAM-disk.
123 For everything worked as planned, there are some minimal set of named
124 entries required in `/dev` directory of `_install` tree in addition
125 to already setup binary and libraries. The device nodes in the `/dev`
126 directory are required for correct access virtual terminals foe example.
128 1. If the root access is available, then the device nodes can be
129 prepared and RAM-disk archive prepared by next commands:
131 mkdir _install/{dev,etc,proc,sys}
132 sudo cp -a /dev/tty? _install/dev
133 ln -s bin/busybox _install/init
134 (cd _install; find . | cpio -o -H newc | gzip) > ramdisk
136 2. If the root access is not available on system used for RAM-disk
137 preparation then the [gen_init_cpio][gic] from kernel build
138 can be used for preparation of RAM-disk archive with set
139 of special files and directories injected.
144 nod /dev/tty0 644 0 0 c 4 0
145 nod /dev/tty1 644 0 0 c 4 1
146 nod /dev/tty2 644 0 0 c 4 2
147 nod /dev/tty3 644 0 0 c 4 3
148 nod /dev/tty4 644 0 0 c 4 4
149 slink /init bin/busybox 700 0 0
154 find _install -mindepth 1 -type d -printf "dir /%P %m 0 0\n"
155 find _install -type f -printf "file /%P %p %m 0 0\n"
156 find _install -type l -printf "slink /%P %l %m 0 0\n"
159 gen_init_cpio filelist | gzip > ramdisk
161 [gic]:http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=usr/gen_init_cpio.c;hb=HEAD
163 5. **The Linux Kernel**. Preparation of the kernel is quite similar as steps used for BusyBox:
164 the kernel source code is downloaded, configured (i.e. make menuconfig)
165 and compiled (make). Given the size of the kernel and required time
166 (the build tree takes about 1 GB of disk space and compilation
167 takes about 20 minutes), we skip these steps and use the already kernel
168 already installed from a distribution.
170 If you choose to use self build kernel then the path to that
171 kernel image would be specified in after *-kernel* option
172 in next step QEMU command.
174 6. Booting kernel with the prepared filesystem (in emulator)
176 The next command is used on 32-bit system to start emulator:
178 qemu -kernel /boot/vmlinuz-2.6.26-2-686 -initrd ramdisk
180 The 64-bit enabled variant of QEMU has to be used on 64-bit Debian
181 system to emulate 64-bit target system:
183 qemu-system-x86_64 -kernel /boot/vmlinuz-2.6.32-5-amd64 -initrd ramdisk
185 If the host system CPU provides hardware virtualization support
186 then it is advantageous to use [KVM][kvm] enhanced emulator which is
187 significantly faster. Next command can be used in such case:
189 kvm -kernel /boot/vmlinuz-2.6.32-5-amd64 -initrd ramdisk
191 [kvm]:http://www.linux-kvm.org/
193 7. If all goes well, next message is displayed
195 Please press Enter to activate this console.
197 When you press Enter then a shell is started and you can test work
198 inside a system which you have just created.
200 Possible Enhancements
201 =====================
203 Additionally, you can work out minor improvements to the system that
204 can simplify your further work within a booted system.
206 1. You can mount a `/proc` virtual filesystem to allow commands such as `ps`
207 (listing running processes) to function correctly. Use next command in
208 the emulator, not on your workstation.
210 mount -t proc none /proc
212 2. The commands intended to be executed during each system start
213 should be writtent into newly created file `/etc/init.d/rcS`.
215 mkdir -p _install/etc/init.d
216 cat <<EOF > _install/etc/init.d/rcS
218 mount -t proc none /proc
221 chmod +x _install/etc/init.d/rcS # nastavenà spustitelnosti
223 The RAM-disk has to be build again and the system should be booted
224 again to test added functionality.
226 3. Messages print by kernel messages running in QEMU emulator can be
227 can be redirected to a virtual serial port and captured to a file.
228 This can be achieved by next QEMU options
230 qemu -serial file:/tmp/virtual_guest.log ...
232 and kernel is run with parameter `console=ttyS0` for console redirection
234 qemu -serial file:/tmp/virtual_guest.log -append console=ttyS0 ...
236 4. The network connection can be provided to the virtual system as well.
237 The most simple option is to use NAT on the user level and emulate
238 standard NIC card hardware for virtualized system
240 qemu -net nic,vlan=0,model=ne2k_pci -net user,vlan=0 ...
242 5. If QEMU or KVM supports Plan9 virtio network then it is it is possible
243 to promote part of the host system directory tree to the virtualized guest
246 qemu -virtfs local,path=shared_dir_name,security_model=none,mount_tag=shared_tag ...
248 The directory tree is then attached (mounted) from guest system through next command
255 modprobe 9pnet_virtio
257 mkdir -p /mnt/shareddir
258 mount -t 9p -o trans=virtio shared_tag /mnt/shareddir
261 Kernel Loadable Modules
262 =======================
264 Kernel modules are separate compiled pieces of code that can be loaded
265 into running Linux kernel. If we wanted to find an analogy in the user
266 environment, then it would be shared *shared/dynamically linked library*
267 or program *plugins*. Kernel module can provide device driver functionality,
268 can add support for a file system, can add new features to the core
269 (i.e., firewalls), or serve as a library of utility functions for other
270 modules (i.e., libata).
272 The example of source code of a simple kernel module follows:
274 #include <linux/init.h>
275 #include <linux/module.h>
276 MODULE_LICENSE("Dual BSD/GPL");
278 static int hello_init(void)
280 printk(KERN_ALERT "Hello, world\n");
284 static void hello_exit(void)
286 printk(KERN_ALERT "Goodbye, cruel world\n");
289 module_init(hello_init);
290 module_exit(hello_exit);
291 *Example taken from [LDD3][LDD3].*
293 The compilation can be controlled by a simple Makefile,
294 which will contain a single line (here we assume that the
295 above file is named **khello.c**):
299 Now a `make` program has to be invoked with right parameters:
301 make -C /lib/modules/$(uname -r)/build M=$(pwd) modules
303 The parameter `-C` direct a `make` build control tool
304 to read kernel installation provided `Makefile` from actual
305 running kernel sources/build directory directory (where
306 are actual directory located can be check by link examination
307 `readlink -f /lib/modules/$(uname -r)/build`). The parameter
308 `M` specifies that your module is located in the current
309 directory and kernel make-system should build only in that directory.
310 The word `modules` on the end means that you want to compiled modules only.
312 If everything goes well, the file `khello.ko` is created as a result
313 of compilation, which is module which can be load into kernel by command
321 Create a simple kernel module, which after logs your name after insertion
322 into the kernel. The kernel log can be examined by `dmesg` command.
323 No other functionality is required from the module.
324 Demonstrate functionality of result of your work in a emulator
325 environment with root filesystem created according above described steps.
327 Who will get bored, may try to extend the module such way that his/her
328 name appeared in the file `/proc/myname` or a simple driver
329 can be created which would return your name when read from `/dev/myname`
330 is issued. Instructions can be found in [this article][henson_drivers] (page 2 and 3).
335 * The shell history search by Ctrl-`R` followed by search text
336 can be used to repeat a command already issued. For example:
340 * To quickly copy text between programs (i.e., commands from the shell of this page)
341 use a middle mouse button. It works so that the text is selected by mouse
342 left button (the Ctrl-C is not pressed/used) and then position mouse
343 to desired target terminal/window/location and press the middle mouse button.
344 It is not necessary to touch keyboard at all and you get forward faster.
346 * If the QEMU is executed over a remote connection (i.e. on server postel)
347 then tunneling of X protocol (`ssh -X`) is required to allow
348 QEMU emulated screen of target system to be launched. Or the QEMU
349 and target system can be run with text mode console (`qemu -curses`).
350 Yet another option is to run QEMU without console and graphic card
351 emulation at all (`qemu -nographic`) and setup emulated system system
352 console to be redirected to serial port.
357 * [ramfs, rootfs and initramfs](http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/ramfs-rootfs-initramfs.txt;hb=HEAD)
358 * [Early userspace support](http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/early-userspace/README;hb=HEAD)
359 * [Inside the Linux boot process](http://www.ibm.com/developerworks/linux/library/l-linuxboot/)
360 * [The Linux Bootdisk HOWTO](http://www.tldp.org/HOWTO/Bootdisk-HOWTO/)
361 * [Linux Loadable Kernel Module HOWTO](http://tldp.org/HOWTO/Module-HOWTO/)
362 * [Linux Device Drivers, Third Edition][LDD3]
363 * [Kbuild & modules](http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/kbuild/modules.txt;hb=HEAD)
364 * [/dev/hello_world: A Simple Introduction to Device Drivers under Linux](henson_drivers)
366 [LDD3]:http://lwn.net/Kernel/LDD3/
367 [henson_drivers]:http://linuxdevcenter.com/pub/a/linux/2007/07/05/devhelloworld-a-simple-introduction-to-device-drivers-under-linux.html