[[!meta title="Embedded Linux"]] [[!toc]] Aim of the Exercises ==================== The aim of this exercise is to practice building of the base system from open-source components presented during 2-nd lecture and motivate and practically introduce students for topics presented during 4-th lecture "*Linux kernel - beginning, development, components and device drivers; GNU libc and user space*". If there are some steps or relations clear to you, we recommend you try Google to find information first and prepare to ask questions at the 4-th lecture. Open Source Software (OSS) projects are often not used alone but in combination with other OSS projects, resulting in the so-called OSS stacks or even distributions. Perhaps the best known is the LAMP stack - Linux, Apache, MySQL, PHP. In today's exercise you will learn and experiment with another, very often used, a stack of [Linux][kenrel] + [BusyBox][bb] (+[Dropbear][dropbear] SSH server). [BusyBox][bb] is a set of basic/standard UNIX utilities (shell, editor and such utilities as ls, mkdir, …) compiled into one binary. In combination with the Linux kernel it forms a complete minimal operating system with a relatively small footprint. Thanks to this combination is often used in embedded applications such as [WiFi routers and ADSL modems][owrt]. We try to create a complete open source operating system from the Linux kernel and user space environment consisting of just busybox during this exercise. Next, you try to program and load a simple module into the kernel. [kenrel]: http://kernel.org/ [bb]:http://busybox.net/ [owrt]:http://www.openwrt.org/ [dropbear]:http://matt.ucc.asn.au/dropbear/dropbear.html The Steps ========= 1. Download/clone [source code][bbgit] of [BusyBox][bb] project: git clone git://busybox.net/busybox.git --reference /usr/src/busybox.git cd busybox Because expanded source tree is a current development snapshot, it is quite possible that some problem is encountered during compilation, installing or use of the project. Our significant advantage is that cloned project repository contains whole project development history qgit and we can choose an older/stable version which is not affected by such problems. For example git checkout -f 1_18_3 [bbgit]:http://git.busybox.net/busybox/ 2. Next step is to configure BusyBox such way as intended. make menuconfig There is possibility to control build of many components but default configuration is suitable for our task. Selection of `Exit` and confirm by `Yes` to save configuration is enough for our case then. 3. Project is compiled by standard command make 4. Command make install installs busybox into local directory `./_install`. Notice, please, that there is only single binary executable file `bin/busybox` and all other directory contents are symbolic links to that binary file. Because we have not used *cross compilation*, which is quite often in case of build for embedded devices, it is possible to test functionality of our result of *native build* directly on build machine. The shell `./_install/bin/sh` can be run for example (it can be terminated by `exit` command). In the case of a real embedded system would have to go on and test the BusyBox after booting on the target hardware. 5. If you can use supervisor (root) rights on build system, then it is possible to test BusyBox in [chroot environment][chroot], i.e. with actual running kernel instance, but with minimal filesystem populated during BusyBox install: # chroot _install /bin/sh But you find that the attempt to execute BusyBox would fail, because there are necessary system libraries for default/non-static build of BusyBox and these libraries are not available in `_install` substree after chroot. [chroot]:http://en.wikipedia.org/wiki/Chroot 6. Next command can be used to determine missing libraries and their paths ldd _install/bin/busybox These libraries must be copied to the directory `_install`. The command to copy libraries can look like line below on 32-bit system: mkdir _install/lib cp /lib/i686/cmov/libm.so.6 /lib/i686/cmov/libc.so.6 /lib/ld-linux.so.2 _install/lib The ELF file "interpreter" (`/lib64/ld-linux-x86-64.so.2`) is searched in directory `/lib64` on 64-bit system by kernel. The `/lib64` can be substituted by symbolic link to directory `/lib` ( cd _install && ln -s lib lib64 ) The BusyBox can be executed in chroot environment after libraries setup. 5. The easiest way how to boot into the user environment you just created is to store pack directory tree in the Linux kernel initial RAM-disk format and boot Linux with this RAM-disk. For everything worked as planned, there are some minimal set of named entries required in `/dev` directory of `_install` tree in addition to already setup binary and libraries. The device nodes in the `/dev` directory are required for correct access virtual terminals foe example. 1. If the root access is available, then the device nodes can be prepared and RAM-disk archive prepared by next commands: mkdir _install/{dev,etc,proc,sys} sudo cp -a /dev/tty? _install/dev ln -s bin/busybox _install/init (cd _install; find . | cpio -o -H newc | gzip) > ramdisk 2. If the root access is not available on system used for RAM-disk preparation then the [gen_init_cpio][gic] from kernel build can be used for preparation of RAM-disk archive with set of special files and directories injected. ( cat < filelist gen_init_cpio filelist | gzip > ramdisk [gic]:http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=usr/gen_init_cpio.c;hb=HEAD 5. **The Linux Kernel**. Preparation of the kernel is quite similar as steps used for BusyBox: the kernel source code is downloaded, configured (i.e. make menuconfig) and compiled (make). Given the size of the kernel and required time (the build tree takes about 1 GB of disk space and compilation takes about 20 minutes), we skip these steps and use the already kernel already installed from a distribution. If you choose to use self build kernel then the path to that kernel image would be specified in after *-kernel* option in next step QEMU command. 6. Booting kernel with the prepared filesystem (in emulator) The next command is used on 32-bit system to start emulator: qemu -kernel /boot/vmlinuz-2.6.26-2-686 -initrd ramdisk The 64-bit enabled variant of QEMU has to be used on 64-bit Debian system to emulate 64-bit target system: qemu-system-x86_64 -kernel /boot/vmlinuz-2.6.32-5-amd64 -initrd ramdisk If the host system CPU provides hardware virtualization support then it is advantageous to use [KVM][kvm] enhanced emulator which is significantly faster. Next command can be used in such case: kvm -kernel /boot/vmlinuz-2.6.32-5-amd64 -initrd ramdisk [kvm]:http://www.linux-kvm.org/ 7. If all goes well, next message is displayed Please press Enter to activate this console. When you press Enter then a shell is started and you can test work inside a system which you have just created. Possible Enhancements ===================== Additionally, you can work out minor improvements to the system that can simplify your further work within a booted system. 1. You can mount a `/proc` virtual filesystem to allow commands such as `ps` (listing running processes) to function correctly. Use next command in the emulator, not on your workstation. mount -t proc none /proc 2. The commands intended to be executed during each system start should be writtent into newly created file `/etc/init.d/rcS`. mkdir -p _install/etc/init.d cat < _install/etc/init.d/rcS #!/bin/sh mount -t proc none /proc echo Nazdar!!!! EOF chmod +x _install/etc/init.d/rcS # nastavení spustitelnosti The RAM-disk has to be build again and the system should be booted again to test added functionality. 3. Messages print by kernel messages running in QEMU emulator can be can be redirected to a virtual serial port and captured to a file. This can be achieved by next QEMU options qemu -serial file:/tmp/virtual_guest.log ... and kernel is run with parameter `console=ttyS0` for console redirection qemu -serial file:/tmp/virtual_guest.log -append console=ttyS0 ... 4. The network connection can be provided to the virtual system as well. The most simple option is to use NAT on the user level and emulate standard NIC card hardware for virtualized system qemu -net nic,vlan=0,model=ne2k_pci -net user,vlan=0 ... 5. If QEMU or KVM supports Plan9 virtio network then it is it is possible to promote part of the host system directory tree to the virtualized guest system qemu -virtfs local,path=shared_dir_name,security_model=none,mount_tag=shared_tag ... The directory tree is then attached (mounted) from guest system through next command sequence modprobe virtio modprobe virtio_ring modprobe virtio_pci modprobe 9pnet modprobe 9pnet_virtio modprobe 9p mkdir -p /mnt/shareddir mount -t 9p -o trans=virtio shared_tag /mnt/shareddir Kernel Loadable Modules ======================= Kernel modules are separate compiled pieces of code that can be loaded into running Linux kernel. If we wanted to find an analogy in the user environment, then it would be shared *shared/dynamically linked library* or program *plugins*. Kernel module can provide device driver functionality, can add support for a file system, can add new features to the core (i.e., firewalls), or serve as a library of utility functions for other modules (i.e., libata). The example of source code of a simple kernel module follows: #include #include MODULE_LICENSE("Dual BSD/GPL"); static int hello_init(void) { printk(KERN_ALERT "Hello, world\n"); return 0; } static void hello_exit(void) { printk(KERN_ALERT "Goodbye, cruel world\n"); } module_init(hello_init); module_exit(hello_exit); *Example taken from [LDD3][LDD3].* The compilation can be controlled by a simple Makefile, which will contain a single line (here we assume that the above file is named **khello.c**): obj-m = khello.o Now a `make` program has to be invoked with right parameters: make -C /lib/modules/$(uname -r)/build M=$(pwd) modules The parameter `-C` direct a `make` build control tool to read kernel installation provided `Makefile` from actual running kernel sources/build directory directory (where are actual directory located can be check by link examination `readlink -f /lib/modules/$(uname -r)/build`). The parameter `M` specifies that your module is located in the current directory and kernel make-system should build only in that directory. The word `modules` on the end means that you want to compiled modules only. If everything goes well, the file `khello.ko` is created as a result of compilation, which is module which can be load into kernel by command insmod khello.ko Assignment ========== Create a simple kernel module, which after logs your name after insertion into the kernel. The kernel log can be examined by `dmesg` command. No other functionality is required from the module. Demonstrate functionality of result of your work in a emulator environment with root filesystem created according above described steps. Who will get bored, may try to extend the module such way that his/her name appeared in the file `/proc/myname` or a simple driver can be created which would return your name when read from `/dev/myname` is issued. Instructions can be found in [this article][henson_drivers] (page 2 and 3). Tips and Tricks =============== * The shell history search by Ctrl-`R` followed by search text can be used to repeat a command already issued. For example: cpio * To quickly copy text between programs (i.e., commands from the shell of this page) use a middle mouse button. It works so that the text is selected by mouse left button (the Ctrl-C is not pressed/used) and then position mouse to desired target terminal/window/location and press the middle mouse button. It is not necessary to touch keyboard at all and you get forward faster. * If the QEMU is executed over a remote connection (i.e. on server postel) then tunneling of X protocol (`ssh -X`) is required to allow QEMU emulated screen of target system to be launched. Or the QEMU and target system can be run with text mode console (`qemu -curses`). Yet another option is to run QEMU without console and graphic card emulation at all (`qemu -nographic`) and setup emulated system system console to be redirected to serial port. References =========== * [ramfs, rootfs and initramfs](http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/ramfs-rootfs-initramfs.txt;hb=HEAD) * [Early userspace support](http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/early-userspace/README;hb=HEAD) * [Inside the Linux boot process](http://www.ibm.com/developerworks/linux/library/l-linuxboot/) * [The Linux Bootdisk HOWTO](http://www.tldp.org/HOWTO/Bootdisk-HOWTO/) * [Linux Loadable Kernel Module HOWTO](http://tldp.org/HOWTO/Module-HOWTO/) * [Linux Device Drivers, Third Edition][LDD3] * [Kbuild & modules](http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/kbuild/modules.txt;hb=HEAD) * [/dev/hello_world: A Simple Introduction to Device Drivers under Linux](henson_drivers) [LDD3]:http://lwn.net/Kernel/LDD3/ [henson_drivers]:http://linuxdevcenter.com/pub/a/linux/2007/07/05/devhelloworld-a-simple-introduction-to-device-drivers-under-linux.html