Bootlin

Finally, the last day of the 2012 edition of the Embedded Linux Conference has arrived. Including the Android Builders Summit, it was a very busy week with five full days of presentations, a very intensive learning session, but also highly motivating and refreshing. Here is, with a little bit of delay, the report of this last day.

Thanks to the kind help of Benjamin Zores (from Alcatel/Lucent, the GeeXboX and OpenBricks projects) who kindly accepted to record the Userland Tools and Techniques For Linux Board Bring-Up and Systems Integration, both Grégory and myself could attend the talk from Greg Ungerer titled M68K: Life in the Old Architecture. Greg started with a very nice and clear explanation of the history of the 68k architecture from a hardware perspective, and detailed its evolution into the Coldfire architecture. The history is quite complicated: the first 68k processors had no MMU, and then MMU was added starting at the 68030 family. However, when Freescale started with Coldfire, which uses a subset of the 68k instruction set, they removed the MMU, until Coldfire V4e, on which an MMU is available. Originally, the Linux port in arch/m68k only supported the classic 68k with MMU, and support for non-MMU Coldfires was added in uClinux. Later, support for non-MMU Coldfires was added into the mainline kernel in arch/m68knommu, with unfortunately a lot of duplication from arch/m68k. The two directories have been merged again some time ago: the merge had already been done in a mechanic fashion (merging identical files, renaming different files that had similar names), and a huge cleanup effort has taken place since then. The cleanup effort is not completely done yet, but it’s getting close, according to Greg Ungerer. At the end of the session, there has been a question on how m68k/coldfire developers typically generate their userspace, and Greg said he uses something similar to Buildroot, which in fact is uClinux-dist. I jumped in, and said that we would definitely like to have Coldfire support, especially since the activity on uClinux-dist isn’t very strong. I also asked what were the remaining differences between the uClinux kernel and the mainline kernel, and according to Greg, there is almost no difference now except maybe support for a few boards. Greg only uses the mainline Linux kernel now for his m68k and Coldfire developments.

The next conference I attended was the talk from Gary Bisson (Adeneo Embedded) titled Useful USB Gadgets on Linux. I rescued the speaker by lending my laptop because his laptop had no VGA output. Fortunately, the speaker was French, so he could adapt quickly to our bizarre azerty keyboard layout. Gary gave quite a bit of context on what USB is, and explained the USB terminology such as interfaces, end-points, configurations, etc. He then quickly described the Linux USB Gadget stack and gadgetfs for the implementation of USB gadget drivers in userspace. He then presented the existing USB gadget drivers in the kernel, mainly the zero gadget driver (for testing purposes), the mass storage gadget driver, the serial gadget driver and the Ethernet gadget driver. At the end of the presentation, he made a demonstration on a BeagleBoard-XM with the gadget multi driver, which allows to expose multiple gadget interfaces at the same time. So he showed that he could expose the Ethernet interface, the Mass Storage interface and the Serial interface, and demonstrated their usage from the host machine. Overall the talk was good, but I was personally expecting a more in-depth look at USB Gadget driver development, and not only usage: I have already been using gadget drivers for some time now, and I was more interested in having details on developing custom gadget drivers rather than simply on using the existing ones.

After a quick break, Grégory and I attended the Getting the First Open Source GSM Stack in Linux talk by Marcin Mielczarczyk from Tieto. It was an absolutely excellent talk. Marcin described the work he and one of his colleague did to reverse engineer a cheap Chinese phone and port U-Boot and Linux on it. Marcin started by giving details about the landscape of those cheap Chinese phones, and it was quite funny: there are brands like Nokla, Sany Eracsson or SciPhone that create phones that are similar in shape and design to phones from the original brands, but with completely different hardware, and usually completely different software. Marcin said that the great thing about those phones is that they are really cheap (which is nice when you need to do some hardware modifications on them for reverse engineering purposes), can easily be bought from auction sites like eBay, and usually do not use any sort of encryption or signature mechanism to prevent the execution of a different operating system or bootloader. The motivation of Marcin in getting Linux to run on such a phone was to ultimately be able to run the complete OsmocomBB software GSM stack inside the phone. OsmocomBB is a free software implementation of a GSM communication stack, lead by Harald Welte. For the moment, the OsmocomBB project uses phones based on the Calypso based-band processor, and only use the phone for the layer 1 (physical layer) of the communication, while the above layers (layer 2 and 3) are implemented in a PC that communicates with the phone over a serial port. Marcin would like to integrate everything inside the phone itself, in order to make the free software GSM stack completely autonomous and fully usable directly on the phone. Marcin decided to pick the SciPhoneDreamG2, a phone that uses the Mediatek 6235 processor, which has the great advantage of being an ARM9 processor, allowing to run a full-blown Linux, and having a datasheet available on the Web. The original operating system of the phone is Nucleus, on top of which the Chinese brand has added an interface that completely mimics Android but is not Android at all. Marcin described the work he did to understand where the UART port and JTAG port was connected (for this work, he mentioned the usage of the JTAG finder project, a software one can run on a micro-controller and that automatically finds which pins are the JTAG pins of a processor). Once he had access to a serial console and the JTAG, he dumped the memory, and started understanding how the boot process was working, and how the existing boot loader was initializing the DRAM. This work was completely done by disassembling the code, which required quite some effort, according to Marcin. Once this was done, he said that porting U-Boot only required creating a basic UART driver and a timer driver, and porting a basic Linux only required a similar UART driver and timer driver, but also an interrupt driver. Marcin and his colleague then went one in developing the other drivers, such as SD, USB, GPIOs and more, and they detailed some of the issues they faced and the time required for these different tasks. In the end, the project is not yet finished, since OsmocomBB does not run on the phone yet, but this is the next goal for Marcin and his colleague. In the end, it was a very interesting goal, detailing in an informative and amusing way an absolutely excellent reverse-engineering effort conducted by Marcin. I would strongly recommend watching the video of this talk.

The last afternoon of ELC started with a talk from Linus Walleij from Linaro, Pin Control Subsystem Overview. Linus Walleij started by describing with lots of details how I/O pins are implemented from a hardware perspective. He first described a basic I/O pin, on which the software can just control the level. On top of this, he explained the hardware logic used to generate interrupts and wake-up events from I/O pins. And finally, he added that those I/O pins are nowadays commonly multiplexed since the SoC do not have enough pins to expose all their possible features, so a given pin can be used either for one function (say, one pin of a I2C bus) or another function (say, one pin of a parallel LCD interface) or as a general purpose I/O. Since this multiplexing is controlled by software, the code for the various ARM sub-architectures in the Linux kernel have each implemented their own little framework and API to solve that problem, and it’s up to each board file to set their I/O multiplexing settings. Unfortunately, since each ARM sub-architecture has its own implementation, there is no coherent API, and there is code duplication. Linus Walleij’s pin mux subsystem intends to solve that. It has already been merged in mainline, in the drivers/pinctrl directory, and a few ARM sub-architectures have started using it, with more to come in the near future, said Linus. Basically, the pinmux subsystem allows to describe which pins are available on the SoC, how they are grouped together in functions, and how drivers can select which function should be activated at an I/O multiplexing level. Of course, the pinmux subsystem detects conflicting usage of I/O, for example if two different drivers want to use the same pin with a different function. Linus also clarified how drivers for I/O pins block should be implemented in the kernel now. If what you have is a simple GPIO expander, then the driver for it should lie in drivers/gpio and it should use the gpio_chip structure. If this simple GPIO expander is also capable of generating interrupts, then the driver should still be in drivers/gpio, but in addition to the gpio_chip structure, it should also register an irq_chip structure. And finally, if instead this I/O pin controller supports multiplexing, then the driver for it should be implemented in drivers/pinctrl, and it should register into the GPIO subsystem (through the gpio_chip structure), into the IRQ subsystem (through the irq_chip structure) and into the pinmux subsystem (through the pinctrl_desc and other related structures). All in all, Linus’s presentation was a great talk, but I wished he would have put more details on the actual API and data structures: his description of the data structures through UML diagrams were a bit hard to follow.

For the last session of the day, I initially planned to attend Pintu Kummar’s talk on Controlling Linux Memory Fragmentation and Higher Order Allocation Failure: Analysis, Observations and Results, but this session was unfortunately canceled. Therefore, I joined my colleague Maxime Ripard and attended Lucas de Marchi talk about Managing Kernel Modules With kmod. Basically, about a year ago, Lennart Poettering, developer of the systemd new userspace init implementation for Linux, listed a set of topics that he wanted to see improved in Linux to make the initialization sequence perform better. Amongst them was the development of a userspace library to manage kernel modules (query information, insert and remove modules). The problem is that until now, the only way to load and remove modules was to call the modprobe, insmod or rmmod programs, which for each module load operation, required a costly sequence of fork/exec. Since udev tries to load up to 200-300 modules at startup (sometimes just to discover that the module is already loaded), this takes a significant amount of time. So Lucas de Marchi, who works at ProFUSION, decided to step up, and did the implementation of kmod. kmod is composed of a C library which implements the core logic of the module information query, module loading and module removal operation, supporting all the fine details that modprobe was supporting (such as dependency handling, module aliases and the configuration files in /etc/modprobe.d/ with options for modules, blacklisted modules). kmod also contains replacement programs for the insmod, lsmod, rmmod and modprobe programs, directly inside a single kmod binary, with symlinks pointing to it for the various commands. kmod is now a full replacement for the old module-init-tools, which has been marked as obsolete by his former maintainer, Jon Masters (who has joined the kmod project). Desktop distributions have started to pick up kmod (Arch Linux, Fedora, and Debian in experimental), as well as embedded Linux build systems. Lucas mentioned that Buildroot had the latest version of kmod, while OpenEmbedded had a slighly older version, and that he didn’t know about other build systems. In the end, this kmod project does not bring a lot of new features or innovations, but is a well-appreciated initiative to make module management better in Linux. What’s very impressive in the time frame in which the project was done: in about a year, the project got started, the development was done, and it is now a full replacement of the old solution, which has been marked deprecated. Great job!

Finally, as every ELC, the conference was closed with a game involving all the attendees, and allowing to win nice prizes such as development boards, USB scopes, audio/video portable players (PMPs), and more. The game started with a set of geek questions (such as “Will the Linux kernel in version 3.3 have more or less than 15 millions lines of code ?”, or “Is the distance from the Earth to the Moon smaller or higher than 150.000 miles ?”), and then a rock/paper/scissors game, and finally a raffle. This closing game is always a nice way of ending ELC.

This year’s edition of the Android Builders Summit and the Embedded Linux Conference have been great, with lots of interesting technical talks, and lots of side discussions with various developers. Many thanks to the conference organizers and speakers!

We hope that those five blog posts reporting some details about those conferences have been interesting to those who didn’t have the chance to attend, and we are definitely looking forward the next edition of the Embedded Linux Conference Europe, which will take place in Barcelona from November 5th to November 7th. Note that the call for papers has already been published. It’s time to think about what you’re doing in the embedded Linux world, and to propose a corresponding talk!