Tag: kernel

Bootlin contributions to the 3.9 kernel

A few months ago, we published a blog post showing our contributions to the 3.8 Linux kernel. With 128 commits merged in 3.8, Bootlin was ranked as the 17th company in terms of kernel contributions.

The 3.9 kernel has been released a few weeks ago, with again a significant number of contributions from Bootlin. According to these statistics, Bootlin contributed 92 patches during the 3.9 cycle, making the company the 26th most important contributor to the Linux kernel for this release, and this time, five engineers from Bootlin contributed patches.

Among the contributions that we made:

  • Added a basic infrastructure for irqchip drivers in the drivers/irqchip directory. This directory is now used to store the drivers for the IRQ controllers of various processors.
  • Made a number of improvements to the Marvell SDIO driver, including the addition of a Device Tree binding for it, and enabled its usage on Marvell Armada 370 and Armada XP platforms, as well as converting the Marvell Kirkwood platforms to use Device Tree probing instead of legacy probing for their SDIO interface.
  • Contributed a number of improvements to support Crystalfontz i.MX28 based modules, including the Device Tree for the CFA10037 expansion board, various improvements for the CFA10049 expansion boards, and a driver for the Himax HX8357B LCD controller.
  • A large number of improvements to the support of the Allwinner ARM SoCs, most notably a pinctrl driver for those SoCs, which allows to configure the muxing of I/O pins, and a gpio driver, to use the pins as general-purpose I/Os. We also contributed the support for the Miniand Hackberry platform, based on an Allwinner SoC. This work is all done by our engineer Maxime Ripard, who is the maintainer of the Allwinner SoC support in the Linux kernel.
  • Improvements to the PCA953x driver (for I2C GPIO expanders) in order to support the PCA9505 chip, that has 40 GPIOs. This required quite some work, as the PCA953x was originally limited to chips having at most 32 GPIOs. This improvement was done in order to support the GPIO expander box provided by Globalscale for the Armada 370-based Mirabox platform.
  • We added support for the Real Time Clock on Armada 370 and Armada XP based platforms, added support for local timers on Armada XP, added support for the new Armada XP GP evaluation board.
  • We enabled support for the SPI controllers and the USB controllers on Armada 370 and Armada XP based platforms.

Our high rate of contributions is going to continue, as we already have 95 patches merged for the upcoming 3.10 kernel and have already submitted a number of patches for the 3.11 kernel.

Here are details about our contributions to the 3.9 kernel:

Linux kernel 3.8 released, Bootlin top #17 contributor

Thomas Petazzoni and Grégory Clement, Bootlin kernel engineers
Thomas Petazzoni (front) and Grégory Clement (back) at the Embedded Linux Conference 2013 in San Francisco, discussing ARM Linux kernel issues.
Early last week, version 3.8 of the Linux kernel has been released by Linus Torvalds. The KernelNewbies web site, has, as usual, a great summary of what’s new in this release, together with lots of links to the relevant LWN articles. With 12394 commits, 3.8 has been the busiest ever kernel release cycle, the previous record being held by 2.6.25 with 12243 commits.

Despite this huge activity, Bootlin has been the 17th most active employer during the 3.8 cycle, with 128 commits merged into the mainline Linux kernel, representing a bit more than 1% of the total number of commits. See the statistics by employer at http://www.remword.com/kps_result/3.8_whole.html and in the traditional LWN article. This puts Bootlin before Nvidia, Qualcomm, ARM or Oracle in number of commits, and just a few commits behind Freescale. See the Git repository for the list of our contributions.

In detail, Bootlin contributions for 3.8 have been:

  • A large number of contributions related to the support of the Marvell Armada 370 and Armada XP SoCs, done by Grégory Clement and Thomas Petazzoni. Contributions included: a new network driver for the Armada 370 and Armada XP, support for the Armada XP-based OpenBlocks AX3 platform, support for the Armada 370-based Globalscale Mirabox platform, a big number of improvements and Device Tree support for the Marvell XOR engine driver, beginning of Device Tree support for the older Marvell Orion5x SoC family, support for the L2 cache found in Armada 370/XP, clock drivers for Armada 370/XP, SMP support for Armada XP, enabling of SATA on Armada 370/XP platforms.
  • The contribution of the initial support for a new SoC family in the mainline Linux kernel: the Allwinner A10 and Allwinner A13 ARM SoCs. This support has been contributed by Maxime Ripard, who has become the maintainer for this new ARM sub-architecture.
  • A driver for the I2C-based SSD1304 OLED display, a nice 128×32 pixels monochrome OLED display, contributed by Maxime Ripard.
  • A number of improvements in the support for the Crystalfontz i.MX28-based platforms, the CFA10036 and its expansion board the CFA10049. These contributions have also been made by Maxime Ripard.

Through these contributions, Bootlin have gained a good expertise in support for ARM SoCs and boards inside the Linux kernel. If you are interested in having us help you bring the support of your ARM board or ARM SoC into the mainline Linux kernel, do not hesitate to contact us, you will be directly answered by our engineers doing Linux kernel development!

Bootlin customer project on Kickstarter!

For about 6 months, we’ve been working with Crystalfontz America on an imx28-based board, targeted at the hackers and DIYers. We’ve been working on the BSP, adding support to Linux and in Buildroot for this board. Support in the mainline Linux kernel is also in pretty good shape, and we continue to post patches to improve it.

The CFA-10036 is actually a computer-on-module with a small OLED display, and comes with two (for now) breakout boards, the CFA-10037, which adds USB and Ethernet connectivity, and an awful lot of exposed GPIOs, and the soon-to-be announced CFA-10049, which is more targeted to industrial or robotic uses, with additional ADCs, fan controller, 1-wire, LCD, rotary encoder, and so on. See more details.

The project is getting close to completion, since Crystalfontz started its funding campaign on Kickstarter.

For those who are not familiar with Kickstarter, it’s a way for creators to get funding and sense customer interest in their projects. If you find the device interesting you can either make a small pledge to show that you like the project, or make a bigger one and will receive board(s) and accessories corresponding to how much you pledged. If the project doesn’t meet its funding goals, you won’t be charged at all. I advise you to read the Kickstarter FAQ to understand Kickstarter better.

ELCE 2012 slides: porting Linux to new ARM SoC

We are just returning from Barcelona, Spain, after participating to the 2012 edition of the Embedded Linux Conference Europe. My colleague Thomas Petazzoni has delivered the below presentation:

Your New ARM SoC Linux Support Check-List

Since Linus Torvalds raised warnings about the state of the ARM architecture support in the Linux kernel, a huge amount of effort and reorganization has happened in the way Linux supports ARM SoCs. From the addition of the device tree to the pinctrl subsystem, from the new clock framework to the new rules in code organization and design, the changes have been significant over the last one and half year in theARM Linux kernel world.

Based on the speaker’s experience on getting the new Marvell Armada 370 and Armada XP SoC supported in the mainline Linux kernel, we will give an overview of those changes and summarize the new rules for ARM Linux support. We aim at helping developers willing to add suppot for new ARM SoCs in the Linux kernel by providing a check-list of things to do.

Thomas Petazzoni is an embedded Linux engineer and trainer at Bootlin since 2008. He has been involved with multiple projects around the Linux kernel, especially the mainlining of Marvell Armada 370/XP SoCs support. He is also a major contributor to the Buildroot embedded Linux build system with more than 1100 patches merged.

The presentation slides and their sources are now available here. We have also shot a video of Thomas’ talk and it should be available in the next weeks. Stay tuned!

Do not hesitate to contact us if you are looking for engineers to port Linux to new hardware.

Embedded Linux, kernel and Android engineer job openings (2012)

Home based jobs in Europe or at one of our offices in France

Penguin worksTo meet increasing demand for its Embedded Linux, kernel and Android engineering services, Bootlin is looking for developers:

  • With experience developing embedded Linux systems
  • With experience developing device drivers for the Linux kernel, and porting Linux on new hardware
  • With visible contributions to Free Software used in embedded systems, such as the Linux kernel, BusyBox, build systems, compilers…
  • With technical writing skills and an interest for training

Experience with Android low-level development, allowing to teach our Android System Development course would also be a strong advantage, though not mandatory.

A first possibility is be hired in France. Being able to join one of our offices in France (Toulouse or Orange) will be an advantage, but working from home in other parts of France will be possible too. We are also open to people living in a country with the Euro currency, working from home, and able to work as full time contractors.

We have a first opening that we would like to fill between September and December 2012. If demand continues to grow, we expect to hire more engineers with the same profile in the following months. We also hope to expand the home based jobs to countries outside Europe in the next years, but it will take a bit more time.

See our careers page for a full description.

Bootlin at the Libre Software Meeting

In a previous post, we detailed all the talks of the Embedded Systems and Open Hardware track of the Libre Software Meeting, taking place in Geneva in early July.

Bootlin will have a quite important presence at this event, with three talks and one tutorial given by Bootlin engineers. You’ll find below the descriptions of the talks given by Bootlin. Both my colleague Maxime Ripard and myself will be present at Libre Software Meeting, and we will be happy to meet you there to discuss Embedded Linux and Android topics!

A look through the Android Stack

Android has established itself in the past years as a major player in the mobile market, outperforming any other mobile systems.

To do so, Google relied both on well established open-source components, such as the Linux Kernel, and munching them together in a brand new userspace environment. This talk will detail the most important components of Android userspace and the interactions between them that allow developers to face a consistent API for their applications.

This talk will be given on Tuesday 9th July 2012, at 14:00, by Maxime Ripard, embedded Linux and Android engineer at Bootlin. Maxime is also teaching our newest training course on Android system development.

Buildroot: a nice, simple and efficient embedded Linux build system

Started in late 2001 by uClibc developers, Buildroot has grown over its 10 years history from a testing tool for the uClibc C library to a complete, vendor-neutral, embedded Linux build system. Until early 2009, the project was mostly unmaintained and the quality slowly decreased, frustrating many Buildroot users. Fortunately, since early 2009, Peter Korsgaard took over the maintainership of Buildroot, and the project has considerably evolved since then: stable releases are published every three months, the user and developer community has grown significantly, the existing features have been cleaned up, many other new features have been added, the project is no longer uClibc-specific and the quality has been vastly improved. Buildroot now offers a nice, simple and efficient mechanism to build small to medium sized embedded Linux systems, such as the ones found in many industrial systems or highly dedicated systems. Many users are amazed about how easy it is to get started with Buildroot, especially compared to other build systems. This presentation will show how Buildroot can be used to build embedded Linux systems, highlighting the new features and improvements made over the last few years, and detailing how the simplicity of Buildroot allows you to focus on developing the applications for your system. A quick overview of the future Buildroot developments will also be provided.

This talk will take place on Wednesday 10th July at 17:00 and will be given by Thomas Petazzoni, embedded Linux engineer at Bootlin, and long time Buildroot contributor.

Linux kernel on ARM: consolidation work

In Spring 2011, Linus Torvalds asked the ARM Linux maintainers to clean up the contents of arch/arm/ in the Linux kernel code by doing more consolidation between ARM sub-architectures.

More than a year later, a lot of work has been accomplished in this area, especially thanks to the introduction of the device tree for the ARM architecture, the pinctrl subsystem and the clock framework into the Linux kernel.

Through this talk, we will present the challenges the ARM architecture creates in terms of Linux kernel support, and then describe from a technical point of view how the device tree, the pinctrl subsystem and the clock subsystem work and how they can improve the consolidation between different ARM sub-architectures.

The talk will be designed to be accessible to an audience having only a moderate knowledge of kernel programming and internals, and will therefore provide enough context for such audience to understand the issues that those different mechanisms are striving to solve.

This talk will take place on Thursday 11th July at 10:00 and will be given by Thomas Petazzoni, embedded Linux engineer at Bootlin.

Tutorial on using Buildroot, a nice, simple and efficient embedded Linux build system

Started in late 2001 by uClibc developers, Buildroot has grown over its 10 years history from a testing tool for the uClibc C library to a complete, vendor-neutral, embedded Linux build system. Until early 2009, the project was mostly unmaintained and the quality slowly decreased, frustrating many Buildroot users. Fortunately, since early 2009, Peter Korsgaard took over the maintainership of Buildroot, and the project has considerably evolved since then: stable releases are published every three months, the user and developer community has grown significantly, the existing features have been cleaned up, many other new features have been added, the project is no longer uClibc-specific and the quality has been vastly improved. Buildroot now offers a nice, simple and efficient mechanism to build small to medium sized embedded Linux systems, such as the ones found in many industrial systems or highly dedicated systems. Many users are amazed about how easy it is to get started with Buildroot, especially compared to other build systems.

This workshop follows the Buildroot presentation proposed in the same topic. During one half-day participants will be introduced on how to efficiently use Buildroot for their own projects:

  • Basic usage of Buildroot: generate the first system, boot it on a hardware platform
  • Add packages to Buildroot
  • Customize Buildroot for real-life projects: how to integrate project specific patches, configuration and customization

Participants are invited to come with their own laptop, installed with a sufficiently recent GNU/Linux distribution. Participants are recommended to attend the Buildroot talk by the same speaker before attending the workshop, as the talk will give an overall introduction on Buildroot.

This tutorial will take place on Thursday 11th July from 14:00 to 17:00 and will be given by Thomas Petazzoni, embedded Linux engineer at Bootlin, and long time Buildroot contributor.

Linux kernel engineer job in Nice

Penguin worksBootlin is looking for a embedded Linux and kernel engineer in the area of Nice in France (on the French Riviera). The contract will be home based, but will also involve working at customer locations in the same area, possibly for long periods of time.

A detailed job description is available on our careers page.

For this particular job opening, we absolutely need someone with prior experience with kernel and driver development, and contributions to the official Linux kernel sources will be a strong advantage. This is because a customer of ours is looking for an engineer to develop new drivers and port the Linux kernel to pre-silicon and silicon platforms.

Once we find a candidate with the expected skills and profile, and once the customer agrees to contract this person for this initial project, the engineer will be hired by Bootlin under a permanent contract, and will work on the customer site for at least 6 months.

Once the initial assignment is over, our engineer will continue to work on projects for other Bootlin customers, and will also give embedded Linux and kernel training sessions to customers throughout the world.

Note that this position is open to people who do not speak French, but are ready to settle in the French Riviera and to be hired through a French contract.

If you are interested in this position, see our job description for details about how to apply.

A new tool to ease kernel maintainer life

When you are involved in mainlining or maintaining some kernel code, a non negligible part of your time is spent checking patches or patchsets themselves . It is not the most interesting part but it is truly necessary to help merging in kernel code, or to make sure you don’t break anything, for example building with an incompatible configuration.

Aiaiai developed by Artem Bityutskiy is a tool to do most of this task for you! It uses other checking tools and scripts such as sparse, coccinelle and checkpatch.pl, and comes with its own set of tools and scripts. I don’t know what does “aiaiai” stands for, but in French it sounds like “Ouch Ouch Ouch”, the sound you could make if you forget to use this tool 😉

PS: On my G+ post, Yegor Yefremov pointed that “aiaiai” means something like “tsk tsk!” (shame on you!) in Russian.

How to boot an uncompressed Linux kernel on ARM

This is a quick post to share my experience booting uncompressed Linux kernel images, during the benchmarks of kernel compression options, and no compression at all was one of these options.

It is sometimes useful to boot a kernel image with no compression. Though the kernel image is bigger, and takes more time to copy from storage to RAM, the kernel image no longer has to be decompressed to RAM. This is useful for systems with a very slow CPU, or very little RAM to store both the compressed and uncompressed images during the boot phase. The typical case is booting CPUs emulated by FPGA, during processor development, before the final silicon is out. For example, I saw a Cortex A15 chip boot at 11 MHz during Linaro Connect Q2.11 in Budapest. At this clock frequency, booting a kernel image with no compression saves several minutes of boot time, reducing development and test time. Note that with such hardware emulators, copying the kernel image to RAM is cheap, as it is done by the emulator from a file given by the user, before starting to emulate the system.

Building a kernel image with no compression on ARM is easy, but only once you know where the uncompressed image is and what to do! For people who have never done that before, I’m sharing quick instructions here.

To generate your uncompressed kernel image, all you have to do is run the usual make command. The file that you need is arch/arm/boot/Image.

Depending on the bootloader that you use, this could be sufficient. However, if you use U-boot, you still need to put this image in a uImage container, to let U-boot know about details such as how big the image is, what its entry point is, whether it is compressed or not… The problem is you can’t run make uImage any more to produce this container. That’s because Linux on ARM has no configuration option to keep the kernel uncompressed, and the uImage file would contain a compressed kernel.

Therefore, you have to create the uImage by invoking the mkimage command manually. To do this without having to guess the right mkimage parameters, I recommend to run make V=1 uImage once:

$ make V=1 uImage
...
  Kernel: arch/arm/boot/zImage is ready
  /bin/bash /home/mike/linux/scripts/mkuboot.sh -A arm -O linux -T kernel -C none -a 0x80008000 -e 0x80008000 -n 'Linux-3.3.0-rc6-00164-g4f262ac' -d arch/arm/boot/zImage arch/arm/boot/uImage
Image Name:   Linux-3.3.0-rc6-00164-g4f262ac
Created:      Thu Mar  8 13:54:00 2012
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    3351272 Bytes = 3272.73 kB = 3.20 MB
Load Address: 80008000
Entry Point:  80008000
  Image arch/arm/boot/uImage is ready

Don’t be surprised if the above message says that the kernel is uncompressed (corresponding to -C none). If we told U-boot that the image is already compressed, it would take care of uncompressing it to RAM before starting the kernel image.

Now, you know what mkimage command you need to run. Just invoke this command on the Image file instead of zImage (you can directly replace mkuboot.sh by mkimage):

$ mkimage -A arm -O linux -T kernel -C none -a 0x80008000 -e 0x80008000 -n 'Linux-3.3.0-rc6-00164-g4f262ac' -d arch/arm/boot/Image arch/arm/boot/uImage
Image Name:   Linux-3.3.0-rc6-00164-g4f262ac
Created:      Thu Mar  8 14:02:27 2012
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    6958068 Bytes = 6794.99 kB = 6.64 MB
Load Address: 80008000
Entry Point:  80008000

Now, you can use your uImage file as usual.

Linux on ARM: xz kernel decompression benchmarks

I recently managed to find time to clean up and submit my patches for xz kernel compression support on ARM, which I started working on back in November, during my flight to Linaro Connect. However, it was too late as Russell King, the ARM Linux maintainer, alreadyaccepted a similar patch, about 3 weeks before my submission. The lesson I learned was that checking a git tree is not always sufficient. I should have checked the mailing list archives too.

The good news is that xz kernel compression support should be available in Linux 3.4 in a few months from now. xz is a compression format based on the LZMA2 compression algorithm. It can be considered as the successor of lzma, and achieves even better compression ratios!

Before submitting my patches, I ran a few benchmarks on my own implementation. As the decompressing code is the same, the results should be the same as if I had used the patches that are going upstream.

Benchmark methodology

For both boards I tested, I used the same pre 3.3 Linux kernel from Linus Torvalds’ mainline git tree. I also used the U-boot bootloader in both cases.

I used the very useful grabserial script from Tim Bird. This utility reads messages coming out of the serial line, and adds timestamps to each line it receives. This allow to measure time from the earliest power on stages, and doesn’t slow down the target system by adding instrumentation to it.

Our benchmarks just measure the time for the bootloader to copy the kernel to RAM, and then the time taken by the kernel to uncompress itself.

  • Loading time is measured between “reading uImage” and “OK” (right before “Starting kernel”) in the bootloader messages.
  • Compression time measured between “Uncompressing Linux” and “done”: 
    ~/bin/grabserial -v -d /dev/ttyUSB0 -e 15 -t -m "Uncompressing Linux" -i "done," > booting-lzo.log

Benchmarks on OMAP4 Panda

The Panda board has a fast dual Cortex A9 CPU (OMAP 4430) running at 1 GHz. The standard way to boot this board is from an MMC/SD card. Unfortunately, the MMC/SD interface of the board is rather slow.

In this case, we have a fast CPU, but with rather slow storage. Therefore, the time taken to copy the kernel from storage to RAM is expected to have a significant impact on boot time.

This case typically represents todays multimedia and mobile devices such as phones, media players and tablets.

Compression Size Loading time Uncompressing time Total time
gzip 3355768 2.213376 0.501500 2.714876
lzma 2488144 1.647410 1.399552 3.046962
xz 2366192 1.566978 1.299516 2.866494
lzo 3697840 2.471497 0.160596 2.632093
None 6965644 4.626749 0 4.626749

Results on Calao Systems USB-A9263 (AT91)

The USB-A9263 board from Calao Systems has a cheaper and much slower AT91SAM9263 CPU running at 200 MHz.

Here we are booting from NAND flash, which is the fastest way to boot a kernel on this board. Note that we are using the nboot command from U-boot, which guarantees that we just copy the number of bytes specified in the uImage header.

In this case, we have a slow CPU with slow storage. Therefore, we expect both the kernel size and the decompression algorithm to have a major impact on boot time.

This case is a typical example of industrial systems (AT91SAM9263 is still very popular in such applications, as we can see from customer requests), booting from NAND storage operating with a 200 to 400 MHz CPU.

Compression Size Loading time Uncompressing time Total time
gzip 2386936 5.843289 0.935495 6.778784
lzma 1794344 4.465542 6.513644 10.979186
xz 1725360 4.308605 4.816191 9.124796
lzo 2608624 6.351539 0.447336 6.798875
None 4647908 11.080560 0 11.080560

Lessons learned

Here’s what we learned from these benchmarks:

  • lzo is still the best solution for minimum boot time. Remember, lzo kernel compression was merged by Bootlin.
  • xz is always better than lzma, both in terms of image size. Therefore, there’s no reason to stick to lzma compression if you used it.
  • Because of their heavy CPU usage, lzma and xz remain pretty bad in terms of boot time, on most types of storage devices. On systems with a fast CPU, and very slow storage though, xz should be the best solution
  • On systems with a fast CPU, like the Panda board, boot time with xz is actually pretty close to lzo, and therefore can be a very interesting compromise between kernel size and boot time.
  • Using a kernel image without compression is rarely a worthy solution, except in systems with a very slow CPU. This is the case of CPUs emulated on an FPGA (typically during chip development, before silicon is available). In this particular case, copying to memory is directly done by the emulator, and we just need CPU cycles to start the kernel.