Author: Thomas Petazzoni

Thomas Petazzoni is Bootlin's co-owner and CEO. Thomas joined Bootlin in 2008 as a kernel and embedded Linux engineer, became CTO in 2013, and co-owner/CEO in 2021. More details...

Videos from Libre Software Meeting 2010 in France

RMLL 2010The Libre Software Meeting (Rencontres Mondiales du Logiciel Libre in French) is a community-driven event that takes place every year in France, and covers a wide range of topics in the free and open source software domain. Each year, an Embedded systems and free hardware topic is proposed, which in 2010 was lead by Florian Fainelli, Pierre Ficheux and myself.

While most of the talks took place in French, a few talks were given in English and as we recoded videos from those talks, we thought it’d be a good idea to highlight them to the english readers of our blog. We found it especially important since amongst those videos, there are two particularly interesting presentations from Sarah Sharp, a kernel developer from Intel, about USB3 and its support in Linux. As usual, all our videos are published under the Creative Commons Attribution – ShareAlike Licence version 3.0 license.

Sarah Sharp presenting USB3 at LSM2010

The talks given in English were :

Our french readers will find all other videos in this blog post.

Bootlin at the Embedded Linux Conference and Android Builders Summit

In just two weeks from now, the Embedded Linux Conference will start in San Francisco, followed by the Android Builders Summit, at the usual Hotel Kabuki location, where the conference is taking place for the third consecutive year.

Embedded Linux Conference 2011

The program of the Embedded Linux Conference has been announced recently, and as usual, features a wide set of technical embedded Linux talks:

  • Filesystem/storage: Power Fail Safe FAT File System, Optimizations For Cheap Flash Media, from Arnd Bergmann, who has also recently published a very interesting article on the same topic.
  • Power management: Faster Resume For More Energy Savings on MeeGo, Powerdebug(ging): A Linaro Perspective, How to Power Tune a Device Running on a Linux Kernel for Better Suspend Battery Life, The Evolution of Tracing and Profiling for Power Management and Accelerators, Runtime PM: Upstream I/O Device Power Management
  • Real-time: Solving Real-Time Scheduling Problems with RT_PREEMPT and Deadline-Based Scheduler, Real-time Audio on Embedded Devices, Identifying Embedded Real-Time Latency Issues: I-Cache and Locks
  • Build system, with a huge number of Yocto-related talks, but no other build systems represented: State of OpenEmbedded Internal Toolchain and SDKs, Yocto Project: Practical Kernel Development Tutorial, Building Custom Embedded Images with Yocto, The Yocto Project and its Application Development Toolkit (ADT) – The Answer to Effective Embedded Application Development, Yocto Project Community BoFs, Delivering Predictability: The Yocto Project Autobuilder, Automated Sanity Testing, License Collection, and Build Statistics Tracking
  • Multimedia: Fun with QML and JavaScript, Integrating a Hardware Video Codec into Android Stagefright using OpenMAX IL, Media Controller Framework (MCF) For OMAP2+ Display Subsystem, Video4linux: Progress, New videobuf2 Framework and the Media Controller, Bringing up HDMI Display for OMAP4 Panda Board – Design, Challenges and Lessons Learned, Linux Graphics Meets the ARM Ecosystem
  • FPGA: Dynamic Co-simulation of FPGA-based Linux Systems-on-Chip, A High Performance Interface Between the OMAP3 and an FPGA
  • Networking: What Embedded Linux Developers Should Know About IPv6, Zigbee Networking & Linux
  • Debugging: Kernel Shark Tutorial and Tools and Techniques for Debugging Embedded Systems
  • Optimization: Snapshot Booting on Embedded Linux, ARM Neon Instruction Set and Why You Should Care, Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libraries and Toolchain, High-Performance Computing using GPUs, What Are and How to Find a Program’s Unused DSOs
  • Low-level: Board Bringup: Open Source Hardware and Software Tools, Working with HardIRQs: Life Beyond Static IRQ Assignments, Genie in the Bottle: Linux Drivers for the AM1808 PRU
  • And many other talks on various topics: LLVM, Clang and Embedded Linux Systems, Linaro: A Year of Change, Control, Recover and Debug Your Embedded Product with PCD, Developer’s Diary: Helping the Process, High-Level Web Interface to Low-Level Linux I/O on the Beagleboard, Linaro Automated Validation on ARM, Crowd Sourcing and Protecting the Open Source Community, Android for Servers?, Hot Multi-OS Switch: How to run Ubuntu, ChromiumOS, Android at the Same Time on an Embedded Device.

This edition will be the first one organized since the merge between the CE Linux Forum into the Linux Foundation, and will therefore be a great opportunity to see if this merge had any impact on the technical quality of the conference.

My colleagues Maxime Ripard (who joined Bootlin just a week ago) and Gregory Clement as well as myself will be present at the Embedded Linux Conference and the Android Builders Summit, and we will as usual record all talks of both of these conferences and will put them online, as we have done recently for the talks that took place during the Embedded Linux Conference Europe 2010 in Cambridge. Do not hesitate to meet us in San Francisco!

Barebox 2011.03 released, with contributions from Bootlin

BareboxBarebox is a bootloader started about two years ago for embedded systems of various architectures. It plays the same role as U-Boot, which is the best known project in this area, but has several advantages over U-Boot. First, it has a much better configuration and compilation system, based on the one used by the Linux kernel: instead of the rusty include/configs/myboard.h configuration headers in U-Boot, Barebox provides a nice menuconfig/xconfig/defconfig based configuration system, that everyone is familiar with. Second, Barebox has a source code organization very similar to the one of the Linux kernel and has replicated the device/driver model of the kernel. This allows to have a nice separation between device drivers and their instantiation, and a source code that looks familiar to anyone that already does kernel development.

Of course, as Barebox is newer than U-Boot, the number of architectures and platforms is more limited, but it is growing rapidly. It already supports ARM, PPC, Blackfin, x86 and a testing sandbox architecture. On ARM, the supported platforms are AT91, EP93xx, iMX, Nomadik, OMAP, S3C24xx and Versatile. On PPC, a single mpc5xxx platform is supported. Patches to add support for the NIOS architecture have also been posted recently (NIOS is a soft-core architecture from Altera).

As a young but fast-growing project, Barebox has chosen a quick development cycle: new releases are made each month, and Barebox 2011.03 has been released a few days ago. It has many ARM and generic improvements, but is also the first release with contributions from Bootlin :

Gregory CLEMENT (3):
      BMP: Add support for 32bpp video frame buffer
      ARM STM/i.MX: Add possibility to choose the bit per pixel for STM video driver
      fb i.MX23/28: Add the reset control of LCD

My colleague Gregory Clement has contributed several improvements to framebuffer support on the i.MX platform. Those improvements were made in the context of a customer project, for which Barebox was used as a way of showing immediately after the device start-up a nice logo on the screen, while the system continues to boot in the background. Initially, the user had to wait 20+ seconds to see a logo on the screen showing that the system was booting. With our Barebox based solution, a logo is now visible on the screen less than 2 seconds after the power on button is pushed.

Snowball, a new community Linux development platform

Snowball platformThe success of the BeagleBoard platform, a low-cost development platform, that has greatly contributed to the success of Texas Instruments OMAP3 processor in the embedded Linux industry, seems to have inspired another processor manufacturer: ST Ericsson. They have recently unveiled Snowball, a low-cost development platform for their AP9500 processor, which features a dual Cortex A-9 ARM core and a Mali 400 GPU.

The development board is designed and produced by our partner Calao Systems, and offers the following features:

  • The AP9500 processor, dual Cortex-A9 and Mali 400 GPU
  • 4 to 8 GB of e-MMC storage
  • 1 GB of LP-DDR2 RAM
  • Micro-SD slot
  • Ethernet connector, Wifi and Bluetooth
  • HDMI output, composite video output
  • Audio in/out
  • USB On The Go
  • Battery charger
  • On-board battery to keep time
  • Serial port connector, JTAG connector, MiPi 34 debug connector
  • Builtin GPS
  • 3-axis accelerometer, magnetometer and gyrometer, one pressure sensor
  • Expansion connectors to access SPI, I2C, LCD, MiPi devices, GPIO, UART, etc.
  • Last but not least, the board can be powered via USB (through a regular cable or through a Y one if power hungry devices like Wifi are used.)

The technical documentation page has a few more details, but at this time, they isn’t a lot of public information available about the AP9500 processor. I hope that ST Ericsson will fully understand how open source works and will soon release datasheets for the AP9500 in an open way. Interestingly, the AP9500 does not use the traditional PowerVR SGX 3D graphics core designed by Imagination Technologies and found in many other ARM processors, but instead uses the Mali graphics core, which is designed directly by ARM. It seems ARM has already open-sourced the kernel side bits of their graphic drivers, but it looks like a proprietary binary blob in userspace is still present.

The board will be available in two variants:

  • A Product Development Kit variant for 241 Euros.
  • A Software Development Kit variant for 165 Euros. My understanding is that the only difference between the two are the expansion connectors, present on the PDK variant but not on the SDK variant.

The board should be widely available at the end of Q2 2011, i.e around June, though at Bootlin, we will receive our first samples by the end of March thanks to our partnership with Calao Systems. The Snowball platform is supported by the Igloo Community, which hosts mailing-lists, an IRC channel, documentation and will also provide Meego and Android builds for the Snowball in the future.

Stay tuned on this blog. As soon as we get our own boards, we will write about our experiments with them.

Buildroot 2011.02 released, with many interesting updates and commercial support!

Buildroot logoAs usual, the latest Buildroot version has been released just in time on the last day of the month: Buildroot 2011.02 is available for download!

This release of the increasingly popular embedded Linux build system provides new interesting features and updates:

  • Support of external toolchains has been improved with support for toolchain profiles. Those are predefined configurations for well-known toolchains such as the CodeSourcery ones for ARM, PowerPC, MIPS and SuperH. Buildroot is now capable of automatically downloading and installing those external toolchains, which is much easier than downloading them manually. It’s now easy to provide users with Buildroot configurations that use well known toolchains, without requiring them to pre-install anything specific.
  • Support for board configurations has been completely rewritten and largely simplified. All board-specific Makefile and configuration options have been removed, and instead, each board is represented by a single, simple (less than 20 lines) defconfig file, in the configs/ directory. In addition to the existing configuration, we have added support for the Mini2440 platform but also for many Qemu emulated platforms: Qemu ARM Versatile, Qemu MIPSel Malta, Qemu PowerPC G3 Beige, Qemu SH4 r2d and Qemu x86. Those configurations allow to easily produce a known-to-work system for the Qemu emulator, making Buildroot even easier to start working on your embedded Linux system. See the documentation for more details on how to add your own board support.
  • Support for the Blackfin architecture has been added, thanks to Mike Frysinger. This support came along with a lot of fixes to make Buildroot work better for non-MMU architectures, since Blackfin is the first actively supported non-MMU architecture in Buildroot. There will certainly be further improvements to support non-MMU architectures, and hopefully additional non-MMU platforms will be added. For those platforms, Buildroot is generally a very good embedded Linux build system, as those architectures are typically used for small to medium sized systems, with a relatively limited number of components.
  • The Crosstool-NG back-end has been improved and extended to support more Buildroot options, and has been upgraded to a newer Crosstool-NG version. This back-end is the third mode for Buildroot toolchain: it allows Buildroot to use Crosstool-NG as the toolchain generator.
  • Ccache support has been reworked and it now works properly. Since Buildroot often requires complete rebuilds from scratch, having the ccache compiler cache is very nice. On my laptop, compiling from scratch a sample Buildroot system was taking 5 minutes and 29 seconds without ccache, and now only takes 3 minutes and 40 seconds with ccache enabled and the cache already filled by a previous build.
  • A new CMake infrastructure has been added for packages, next to the existing generic and autotools infrastructure. For the moment, only two packages are using the infrastructure (cdrkit and libcuefile), but CMake is an increasingly popular build system and we will definitely see more packages using it in the future. Moreover, Buildroot generates a CMake toolchain file that describes the toolchain used by Buildroot, and which makes it very easy to cross-compile external libraries/applications for the Buildroot system using cmake -DCMAKE_TOOLCHAIN_FILE=/path/to/buildroot/output/toolchainfile.cmake.
  • A very nice cleanup job of the internal toolchain build process has been started by Gustavo Zacarias. The build process of binutils, gmp, mpfr and mpc has been migrated to proper packages, and this will also be done for gcc and gdb in the future.
  • As preliminary steps towards the generation of a standalone SDK from Buildroot, two important changes have been made. First, the staging directory is now inside $(O)/host/usr/PLATFORM-TUPLE/sysroot/, but a symbolic link from $(O)/staging has been kept for compatibility. This change will allow the $(O)/host directory to be the standalone SDK in the future. The second change is on pkg-config: its configuration has been adjusted so that it behaves properly to compile target packages without needing any environment variables or options. It makes the Buildroot pkg-config much easier to use to compile external applications.
  • The Python package has been upgraded to the latest Python version, 2.7.1. This was needed since a long time, since the version of Python we had in Buildroot was only 2.4. Moreover, the package has been completely rewritten, with more options, and has been tested on several platforms. Two external Python modules, python-mad and python-serial have also been added as packages, to show how such modules can be integrated into Buildroot.
  • A set of packages to add support in GStreamer for the TI DSP codecs has been added: gst-dsp, gst-omapfb, tidsp-binaries, dsp-tools, thanks to Felipe Contreras.
  • We have a bunch of new packages as well: mpd, the Music Player Daemon with many audio codecs and libraries, the dhrystone and whetstone benchmarks and other tools such as xmlstarlet, fbgrab, irda-utils, lsuio, etc.
  • Many other packages have been upgraded or fixed, and the results of our random configuration builds are much, much better than they were in the past.

As a Bootlin engineer, I have again contributed significantly to this release: Peter Korsgaard, the Buildroot maintainer, has done 171 commits, Gustavo Zacarias has done 119 commits and I have done 103 commits. The next committer is Mike Frysinger (for the great Blackfin support) with 22 commits.

It is with this great Buildroot knowledge and experience that Bootlin has launched a few weeks ago an official offering of Buildroot commercial support. If you are using Buildroot for your embedded product, or want the buyers of your hardware platform to have a simple but efficient embedded Linux build system and you need help, development or consulting, do not hesitate to contact us.

Buildroot 2010.11 release and roadmap

Buildroot logoThe 2010.11 release of Buildroot has been published on November, 30th. Buildroot is a tool that eases the process of building an embedded Linux system: cross-compiling toolchain, root filesystem with dozens or hundreds of libraries and applications, bootloader and kernel.

Release 2010.11

Releases are made every three months, and the latest 2010.11 release has a number of improvements :

  • Experimental crosstool-NG back-end for handling the toolchain. Before, Buildroot could either compile a toolchain for you, or use an already existing external toolchain. However, the internal Buildroot process for building a toolchain is limited to uClibc and its maintenance is a duplication of the work done by the Crosstool-NG community. While this back-end is only experimental at the moment, the intention is to make it the default back-end in some future release, if everything works fine. Thanks to Yann E. Morin for implementing this back-end, and making related changes to Crosstool-NG.
  • The Kconfig infrastructure has been updated to the one of 2.6.36 and has been cleaned up (we now have a nice quilt patch series). This brings savedefconfig (for minimal Buildroot configurations) and nconfig support. We now also create a convenience Makefile wrapper in the output directory for out-of-tree builds, like the kernel has. So once you have done a first make O=/some/path menuconfig, you can go into /some/path and directly use make something without having to specify the output directory.
  • Old-style package hooks (*_HOOK_POST_*) have been removed. This was just a clean up process, because we have a more modern way of allowing packages to hook specific commands at various steps of the build process.
  • Download handling has been reworked and support for git/svn downloads was added, so that packages only available through version control systems can be added easily in Buildroot. The make source (to download all needed tarballs and files so that the build can be done completely offline) and make external-deps (to show all files that would need to be downloaded) commands have been improved to fix several issues
  • On the architecture side, support for ARM Cortex A9 and Sparc LEON variants was added. Support for Alpha, Cris, IA64 and Sparc64 (deprecated in 2010.08) was removed.
  • We also added a few more packages: argp-standalone, gdk-pixbuf, gpsd, gst-ffmpeg, libmpeg2, kbd, librsvg, nuttcp, rng-tools, rrdtool and xz. And removed some deprecated packages: dillo, libglib12, libgtk12, microwin and pcmcia.
  • Many, many packages have been updated. In particular, the Gtk+ library has been updated from 2.12 to 2.20, keeping the DirectFB support functional thanks to the work of Lionel Landwerlin (the DirectFB support in Gtk+ was broken since 2.12). So, finally, this brings us a recent and fresh Gtk+ library in Buildroot.
  • Many packages were converted to the autotargets or gentargets infrastructures (that we use to describe how a particular package should be downloaded, extracted, configured, built and installed), in particular thanks to the work of Martin Banky.

I have again contributed quite a bit to this release, but not as much as I wanted to. Here are the numbers:

git shortlog -s -n 2010.08..
   168  Peter Korsgaard
   115  Thomas Petazzoni
    55  Gustavo Zacarias
    37  Martin Banky
    26  Yann E. MORIN
    15  Lionel Landwerlin
    13  Mike Frysinger
    10  Paulius Zaleckas
     8  Maxime Petazzoni
     4  Konrad Eisele
     3  Chih-Min Chao
     2  Yegor Yefremov
     1  Andy Gibbs
     1  Felipe Contreras
     1  Frederik Pasch
     1  Heiko Zuerker
     1  Javier Viguera
     1  Luca Ceresoli
     1  Marcelo Roberto Jimenez
     1  Marcus Osdoba
     1  Matt Johnson
     1  Paul Burton
     1  Paul Jones
     1  Stanislav Bogatyrev
     1  Thomas Rudin
     1  Will Newton

The things I worked on are: cleanup of the kconfig patches and switch to the 2.6.36 one, removal of old-style hooks and many other small package cleanups, Gtk+ upgrade and documentation improvements. I’m also very happy to see that the number of contributors is increasing, as is the number of questions asked on the list, on IRC and on the bug tracker.

Buildroot developer day

A Buildroot Developer Day, meeting of some of the Buildroot developers, took place on October, 29th in Cambridge, UK, just after the Embedded Linux Conference Europe. Peter Korsgaard (Buildroot maintainer), Lionel Landwerlin (contributor), Yann E. Morin (contributor, Crosstool-NG developer), Nicolas Ferre (Atmel), Patrice Vilchez (Atmel) and Thomas Petazzoni (Bootlin) attended the meeting.

There has been discussions about cleaning up board support, libtool issues, Crosstool-NG integration, toolchain cleanup, top-level parallel make and package management. A report has been posted to the Buildroot mailing list.

The next Buildroot Developer Day will take place on Monday, 7th February, just after FOSDEM, in Brussels.

Roadmap for 2011.02

Here are some of the things that are in the pipeline, for 2011.02 if everything goes well.

  • Conversion of more (all remaining?) packages to the gentargets and autotargets infrastructures.
  • Cleanup of board support in Buildroot. We will now use minimal defconfigs for both Buildroot itself and for kernel configuration. The messy target/device/ directory will be cleaned-up. And we will add support for some emulated boards based on Qemu (x86, ARM, MIPS, PowerPC, SH4). This work is already done, so it’s very likely to be in 2011.02
  • Better support of devtmpfs, mdev and udev. An option will allow the user to select between static device creation and those three techniques for having a dynamic /dev. This is also ready.
  • Improvements in the external toolchain configuration and setup. Buildroot will be able to automatically download some well-known external toolchains, making them easier to use.
  • The Buildroot internal toolchain build process will be reworked, with conversion of most of the code to the package infrastructures, and the toolchain binaries and sysroot move to $(HOST_DIR). This will make it much, much easier to use the toolchain produced by Buildroot outside of Buildroot, and will allow us to generate a SDK to be shared with application developers, without requiring them to run Buildroot. See the report from the Buildroot Developers Day for details. This work has already been started by Gustavo Zacarias, and at least part of it will be in 2011.02.
  • Package management will be improved. Buildroot will know which package installed what in the different directories, and will therefore be able to cleanly remove a package from the system if it has been disabled in the configuration, without requiring a completely clean rebuild. Later on, this work could be used to generate .ipk packages, but for the moment, the focus is on being able to remove libraries and applications from the system when they are unselected from the configuration. This work has already been started by Lionel Landwerlin, and we will do our best to merge it into 2011.02.
  • Support for non-MMU architectures, and particularly Blackfin. Mike Frysinger has already posted patches to add Blackfin and support non-MMU for several packages. This is really a good news, as Buildroot is a good fit for non-MMU systems.
  • Public access to regression tests results

Toolchain cleanup work and board cleanup work are really interesting pieces, because they are the two remaining parts of Buildroot that haven’t been cleaned up and/or rewritten since the Buildroot project came back to life late 2008/early 2009. There are of course still some areas that need improvements of course, but for me, those two pieces are really closing the large cleanup work that has been started almost two years ago.

Of course, don’t hesitate to contact the Buildroot mailing-list, the IRC channel or the bug tracker if you have any questions or issues in using Buildroot.

Busybox 1.18.0, a few new applets

BusyBox logoBusybox, the embedded Linux swiss-army knife, has seen a new release recently: 1.18.0. As usual, it brings dozens of bug fixes, but it also includes a few new applets that are worth noting:

  • pmap, which shows the memory mappings for a particular process. It is just a formatted version of what you can find in /proc/PID/maps, but still nice to have. Knowing the mappings in a particular process is often useful for debugging purposes.
  • add-shell and remove-shell, that respectively add and remove a shell from /etc/shells. This file lists the shells that are acceptable for the chsh command, for example.
  • nandwrite and nanddump, that respectively write to NAND with bad block management, and dumps NAND flash contents. Those two commands expand the set of MTD-related commands in Busybox and are replacements for the version available in mtd-utils. There were already other MTD related utilities in Busybox: flashcp (write to a NOR flash), flash_eraseall (erase a NOR or NAND flash), flash_lock and flash_unlock to protect/unprotect sectors on flash storage. The main advantage is that these utilities relieve you from having to cross-compile mtd-utils, which is never as simple as cross-compiling BusyBox.
  • base64, a base64 encoder and decoder.
  • mpstat, iostat and powertop, three new statistics programs. mpstat (as in multiprocessor stat) reports interrupts and CPU usage on multiprocessor systems, on a per-CPU basis. iostat reports CPU and block device statistics. powertop is a lightweight variant of the popular tool used to diagnose sources of wake-ups, but is only limited to Intel-based machines at the moment.
  • nbd-client, a client for the Network Block Device protocol, so that an embedded system can mount volumes shared by NBD
  • blockdev, allows to perform some ioctl() on block devices such as get sector size, get/set block size, flush buffers, etc.

Have fun with Busybox!

Linux Kernel Development, third edition

Linux Kernel Development, by Robert Love, 3rd edition

Linux Kernel Development is a book authored by Robert Love, a famous kernel developer. Contrary to the very famous Linux Device Drivers book, Linux Kernel Development is not oriented towards driver development, but instead covers how the core Linux kernel works.

Having this knowledge is not absolutely necessary to write Linux device drivers, but having a good overall understanding of the kernel always help to understand what’s going on in your system, even at the application level. In July this year, the third edition of Linux Kernel Development has been published, which upgrades the book contents to kernel version 2.6.34, a good opportunity to have a new look at the book that Bootlin received a few weeks ago.

After a quick introduction to the kernel sources (configuring, building, organization of the source tree), the book immediately dives into kernel internals:

  • Process management: how the kernel represents processes and their state, how processes are created inside the kernel, how threads are handled, are processes are terminated.
  • Process scheduling: a full chapter dedicated to the Linux kernel process scheduler. The new CFS scheduler is of course covered in great detail, with large portion of commented source code, for those who want to understand the fine details of the scheduler. Topics such as process sleep/wake-up, preemption, real-time scheduling policies are also covered.
  • System calls are then covered: how they are implemented, how parameters are passed from userspace to the kernel, etc. The call path from your user-space application down to the kernel is well explained in this chapter.
  • Kernel data structures: a generic chapter which details the kernel API for linked lists, queues, maps, and binary trees. Those APIs are omni-present inside the kernel, and it’s therefore a good idea to know how they work, both for understand existing code and for writing new code.
  • Interrupts: how interrupts are handled and how one can write an interrupt handler. Unfortunately, the newly introduced threaded interrupt handlers are not covered, but it’s true that their usage is not yet very widespread inside the mainline kernel.
  • Bottom halves and deferring work, a topic closely related to interrupt handling. It covers bottom halves, softirqs, tasklets and workqueues.
  • Kernel synchronization: two chapters are dedicated to this topic. First a chapter detailing why synchronization is needed, what are the sources of concurrency and what should be protected against concurrent access. And then a chapter detailing the mechanisms provided by the kernel to implement proper synchronization: atomic operations, spin locks, reader-writer spin locks, semaphores, reader-writer semaphores, mutexes, completion variables, sequential locks, preemption disabling, ordering and barriers
  • Timers and time management details how the kernel manages time: ticks, jiffies counter, timers, delaying execution of code are covered in this chapter. There are unfortunately no details about the clocksource and clockevents infrastructure, and no details about how timers and high-resolution timers are implemented. Contrary to other chapters that go fairly deep into the implementation details, this one mostly only covers the API to time management rather than the internals.
  • Memory management is the topic of the following chapter: physical memory management with the page allocator and the physical zones, then the kmalloc, vmalloc and SLAB allocators are covered. High-memory mappings, a topic specific to 32 bits architectures having more than a gigabyte of RAM is also covered in detail. The per-cpu interface is also covered, and will help those who want to understand parts of the kernel that have been optimized for scalability on multiple CPUs.
  • The Virtual Filesystem, with its different objects: superblock, inode, dentry and file is covered in good detail.
  • The block layer is then covered, with a description of the role of the bio structure, the request queues, and the I/O schedulers.
  • Then, the book goes back to more details about the internals of memory management: the mm_struct memory descriptor, the virtual memory areas (so called VMAs) and how they relate with the mmap/munmap system calls. The next chapter continues with a detailed description of the page cache implementation.
  • A fairly strange chapter called “Devices and modules” gives some information about kernel modules (how to build them, how to use module parameters, how dependencies are handled), then covers the internal of the device model (kobjects, ktypes, ksets, krefs) and finally sysfs. Just like the chapter covering the device model in the third edition of Linux Device Drivers, I think it totally misses the point. All the kobject, ktypes, ksets and krefs stuff is very low-level plumbing used by the Linux device model, but it is not exactly what the driver developer needs to interact with in the first place. In my opinion, a good description of the device model should rather explain what struct bus_type, struct driver and struct device are, how they are specialized for the different bus types in pci_driver, pci_device, usb_driver, usb_device, platform_driver, platform_device, and how the registration/probing of drivers and devices is done. I’ve recently given a talk about this topic, the video is in French, but the slides are in English.
  • Debugging is then covered: printk of course, but also oopses, debugging-related kernel options, the magic SysRq key, kernel debuggers, etc.
  • A rather generic chapter about Portability is then proposed, and finally a chapter about Patches, Hacking and the community that details the kernel community, the kernel coding style, how to generate and submit patches, etc.

All in all, Linux Kernel Development remains very good reading. I particularly appreciate the writing style of Robert Love, who manages to make a deeply-technical book interesting and easy to read. Of course, there are some topics in the kernel in which I had to dive myself and for which I’d expect to see more details in this book, but giving every possible detail about a huge beast such as the Linux kernel in just 400 pages is not possible!

Videos from FOSDEM 2010

Peter Korsgaard presenting Buildroot in the Cross build systems workshop at FOSDEM 2010

Like every year, the Free and Open Source Developer European Meeting took place early February in Brussels, and Thomas Petazzoni, from Bootlin, attended and recorded a few talks from the embedded session. However, contrary to previous years, I haven’t been able to record all talks from the embedded session, since I attended talks from other sessions which were already being recorded by others.

Gian-Carlo Pascutto presenting Embedded software development best practices at FOSDEM 2010

We also attended talks from the X.org and Coreboot developer roooms : videos for the X.org developer room can be found at http://video.fosdem.org/2010/devrooms/xorg/ and videos for the Coreboot developer room can be found at http://video.fosdem.org/2010/devrooms/coreboot/.

Buildroot 2010.08 released!

Buildroot logoOn the last day of August, just in time, the 2010.08 version of Buildroot has been released. For the record, Buildroot is an easy-to-use embedded Linux build system: it can build your toolchain, your root filesystem with all its components (Busybox, libraries, applications, etc.), your kernel and your bootloaders, or any combination of these components.

Amongst the interesting changes in this version :

  • Complete rewrite of the bootloader build code. It contained a lot of legacy, unused and unclear stuff, it is now much easier to use and extend. We’ve removed support for Yaboot and added support for the new Barebox bootloader, and all the code to support AT91Bootstrap, AT91DataFlashBoot, U-Boot, Grub and Grub 2 has been rewritten.
  • Complete rewrite of the Linux kernel build code. It was also complicated to use, with an horribly complicated kernel version selection mechanism, the new code is much easier to configure and use.
  • The configuration file .config is now located in the out-of-tree directory when the O= option is used. So typically, for an out-of-tree build (which are very convenient when using the same Buildroot source tree for different projects/tests), you could do : mkdir ~/myoutput ; make O=~/myoutput menuconfig ; make O=~/myoutput
  • Support for building NPTL toolchains with uClibc, using the latest uClibc snapshots.
  • Support for the gconfig Gtk-based configurator, in addition to the already available menuconfig and xconfig
  • A particular effort has been put on fixing many of the bugs in our Bugzilla, improving robustness thanks to automated random builds, and converting even more packages to the generic and autotools infrastructure
  • Various things have also been deprecated: support for the CRIS, IA64, Sparc64 and Alpha architectures, support for Gtk over DirectFB (which is at the moment not supported upstream), Java support (no maintainer has volunteered to maintain this in Buildroot)
  • Many components have been bumped to newer versions
  • The shared configuration cache, which allowed to speed up the configuration of different packages, has been disabled by default, since it was causing a lot of problems with certain package configurations

I’ve again contributed to a significant portion of this release, being the author of the bootloader build code cleanup, the Linux kernel build code rewrite, leading an effort to reduce the number of outstanding bugs in our Bugzilla and many other little things. The contributors for this release are shown below :

   175  Peter Korsgaard
   168  Thomas Petazzoni
    38  Gustavo Zacarias
    18  cmchao
     8  Luca Ceresoli
     7  Paul Jones
     6  Lionel Landwerlin
     6  Malte Starostik
     5  Yann E. MORIN
     3  Julien Boibessot
     3  Khem Raj
     2  Dmytro Milinevskyy
     2  Francois Perrad
     2  Nick Leverton
     2  Peter Huewe
     2  Stanislav Bogatyrev
     1  Baruch Siach
     1  Bjørn Forsman
     1  Daniel Hobi
     1  Darcy Watkins
     1  Darius Augulis
     1  H Hartley Sweeten
     1  Karl Krach
     1  Kelvin Cheung
     1  Ossy
     1  Sagaert Johan
     1  Simon Pasch
     1  Slava Zanko
     1  Thiago A. Correa
     1  Will Wagner
     1  Yegor Yefremov

For the next release, there are already a few things in the pipeline :

  • Cleanup of all the board support code in Buildroot, in order to cleanly add support for more boards like BeagleBoard, Qemu boards, Calao boards, etc. We’ll use the new minimal defconfig mechanism used by the kernel. I’ve already started working on this
  • Cleanup of the package download process, to support Git and SVN download. The code has already been written by Maxime Petazzoni, reviewed on the list, so I expect it to be included fairly soon
  • Rewrite of libtool handling code, to remove some of our ugly libtool hacks. The code is currently being worked on by Lionel Landwerlin
  • Support for compiling toolchain using Crosstool-NG as a backend. The code is currently being finalized by Yann E. Morin, the author of Crosstool-NG
  • Further work on package uninstallation, clean partial rebuild. Some work has been started by Lionel Landwerlin, but it needs some discussion
  • Continue the conversion of packages to the generic and autotools infrastructures
  • I have also a ton of other things on my TODO-list : rework gdb/gdbserver support with external toolchains, rework the configuration of IPv6/RPC/locale/etc. with external toolchains, set up a Wiki-based Buildroot website with tutorials and better documentation, clean up the toolchain build process, reduce the number of “enhancement” bugs waiting in our Bugzilla, etc.

As Peter Korsgaard, Buildroot maintainer, said in the 2010.08 announcement: The next release is going to be 2010.11. Expect the first release candidate in late October and the final release at the end of November..

It is worth noting that we will be having a Buildroot Developer Day, on Friday 29th October, right after Embedded Linux Conference Europe. At least Peter Korsgaard, Lionel Landwerlin, Yann E. Morin and myself should be there.