Bootlin at the Embedded Linux Conference and Android Builders Summit

In just two weeks from now, the Embedded Linux Conference will start in San Francisco, followed by the Android Builders Summit, at the usual Hotel Kabuki location, where the conference is taking place for the third consecutive year.

Embedded Linux Conference 2011

The program of the Embedded Linux Conference has been announced recently, and as usual, features a wide set of technical embedded Linux talks:

  • Filesystem/storage: Power Fail Safe FAT File System, Optimizations For Cheap Flash Media, from Arnd Bergmann, who has also recently published a very interesting article on the same topic.
  • Power management: Faster Resume For More Energy Savings on MeeGo, Powerdebug(ging): A Linaro Perspective, How to Power Tune a Device Running on a Linux Kernel for Better Suspend Battery Life, The Evolution of Tracing and Profiling for Power Management and Accelerators, Runtime PM: Upstream I/O Device Power Management
  • Real-time: Solving Real-Time Scheduling Problems with RT_PREEMPT and Deadline-Based Scheduler, Real-time Audio on Embedded Devices, Identifying Embedded Real-Time Latency Issues: I-Cache and Locks
  • Build system, with a huge number of Yocto-related talks, but no other build systems represented: State of OpenEmbedded Internal Toolchain and SDKs, Yocto Project: Practical Kernel Development Tutorial, Building Custom Embedded Images with Yocto, The Yocto Project and its Application Development Toolkit (ADT) – The Answer to Effective Embedded Application Development, Yocto Project Community BoFs, Delivering Predictability: The Yocto Project Autobuilder, Automated Sanity Testing, License Collection, and Build Statistics Tracking
  • Multimedia: Fun with QML and JavaScript, Integrating a Hardware Video Codec into Android Stagefright using OpenMAX IL, Media Controller Framework (MCF) For OMAP2+ Display Subsystem, Video4linux: Progress, New videobuf2 Framework and the Media Controller, Bringing up HDMI Display for OMAP4 Panda Board – Design, Challenges and Lessons Learned, Linux Graphics Meets the ARM Ecosystem
  • FPGA: Dynamic Co-simulation of FPGA-based Linux Systems-on-Chip, A High Performance Interface Between the OMAP3 and an FPGA
  • Networking: What Embedded Linux Developers Should Know About IPv6, Zigbee Networking & Linux
  • Debugging: Kernel Shark Tutorial and Tools and Techniques for Debugging Embedded Systems
  • Optimization: Snapshot Booting on Embedded Linux, ARM Neon Instruction Set and Why You Should Care, Controlling Memory Footprint at All Layers: Linux Kernel, Applications, Libraries and Toolchain, High-Performance Computing using GPUs, What Are and How to Find a Program’s Unused DSOs
  • Low-level: Board Bringup: Open Source Hardware and Software Tools, Working with HardIRQs: Life Beyond Static IRQ Assignments, Genie in the Bottle: Linux Drivers for the AM1808 PRU
  • And many other talks on various topics: LLVM, Clang and Embedded Linux Systems, Linaro: A Year of Change, Control, Recover and Debug Your Embedded Product with PCD, Developer’s Diary: Helping the Process, High-Level Web Interface to Low-Level Linux I/O on the Beagleboard, Linaro Automated Validation on ARM, Crowd Sourcing and Protecting the Open Source Community, Android for Servers?, Hot Multi-OS Switch: How to run Ubuntu, ChromiumOS, Android at the Same Time on an Embedded Device.

This edition will be the first one organized since the merge between the CE Linux Forum into the Linux Foundation, and will therefore be a great opportunity to see if this merge had any impact on the technical quality of the conference.

My colleagues Maxime Ripard (who joined Bootlin just a week ago) and Gregory Clement as well as myself will be present at the Embedded Linux Conference and the Android Builders Summit, and we will as usual record all talks of both of these conferences and will put them online, as we have done recently for the talks that took place during the Embedded Linux Conference Europe 2010 in Cambridge. Do not hesitate to meet us in San Francisco!

ELC-E 2010 tutorial videos

Videos from the embedded Linux and Android tutorials at ELC-E 2010, by Chris Simmonds

As releasing ELC-E 2010 videos, here are recordings of the embedded Linux and Android tutorials, performed by Chris Simmonds on October 26, 2010.

Chris SimmondsVideo capture
2net
The Embedded Linux Quick Start Guide – Part 1
Slides
Video (52 minutes, 397M)
Chris SimmondsVideo capture
2net
The Embedded Linux Quick Start Guide – Part 2
Slides
Video (79 minutes, 660M)
Chris SimmondsVideo capture
2net
The Embedded Linux Quick Start Guide – Part 3
Slides
Video (67 minutes, 501M)
Chris SimmondsVideo capture
2net
What else can you do with Android? – Part 1
Slides
Video (49 minutes, 432M)
Chris SimmondsVideo capture
2net
What else can you do with Android? – Part 2
Slides
Video (31 minutes, 293M)
Chris SimmondsVideo capture
2net
What else can you do with Android? – Part 3
Slides
Video (59 minutes, 545M)

GStreamer 2010 conference videos

Videos from the 2010 edition of the GStreamer conference, Cambridge, UK, Oct. 26, 2010.

After releasing ELC-E videos, here are videos from the 2010 edition of the GStreamer conference. As usual, these videos are released under the terms of the Creative Commons Attribution – ShareAlike Licence version 3.0.

Jonas HolmbergVideo capture
Axis
Case study – GStreamer on Axis devices
Slides
Video (26 minutes):
full HD (223M), 450×800 (72M)

Sebastian DrögeVideo capture
Collabora Multimedia
WebM and GStreamer
Slides
Video (8 minutes):
full HD (75M), 450×800 (24M)

Florent ThieryVideo capture
UbiCast
Using gstreamer for building automated webcasting systems
Slides
Video (31 minutes):
full HD (361M), 450×800 (107M)

Zeeshan AliVideo capture
Nokia
Implementing DLNA using GStreamer
Slides
Video (29 minutes):
full HD (282M), 450×800 (88M)

Olivier CrêteVideo capture
Collabora
Integrating VideoConferencing into Everyday Applications
Slides
Video (29 minutes):
full HD (334M), 450×800 (103M)

Håvard GraffVideo capture
Tandberg
Case study – Tandberg and GStreamer
Video (61 minutes):
full HD (734M), 450×800 (212M)

Wim TaymansVideo capture
Collabora Multimedia
Keynote – GStreamer – Current and future development
Video (47 minutes):
full HD (310M), 450×800 (136M)

Rob ClarkVideo capture
Texas Instruments
GStreamer and OMAP4
Video (37 minutes):
full HD (336M), 450×800 (128M)

Martin BissonVideo capture
3D Stereoscopic and GStreamer
Video (13 minutes):
full HD (124M), 450×800 (39M)

Jan SchmidtVideo capture
Oracle Corporation
Interactivity in GStreamer pipelines
Slides
Video (20 minutes):
full HD (265M), 450×800 (76M)

Mike SmithVideo capture
Songbird
Cross platform development with GStreamer
Video (51 minutes):
full HD (542M), 450×800 (165M)

Emanuele QuacchioVideo capture
ST Microelectronics
A GStreamer based framework for adaptive streaming applications
Video (54 minutes):
full HD (592M), 450×800 (174M)

Josep TorraVideo capture
Fluendo
Intel SMD elements in GStreamer
Slides
Video (50 minutes):
full HD (371M), 450×800 (151M)

Philippe NormandVideo capture
Igalia
Webkit, HTML5 and GStreamer
Slides
Video (38 minutes):
full HD (229M), 450×800 (106M)

Edward HerveyVideo capture
Collabora Multimedia
Challenges of video editing in your pocket
Slides
Video (53 minutes):
full HD (416M), 450×800 (174M)

David SchleefVideo capture
Entropy Wave
Optimizing multimedia with Orc
Slides
Video (58 minutes):
full HD (493M), 450×800 (175M)

Luciana FujiiVideo capture
Holoscopio
Landell – live streaming for the masses
Slides
Video (21 minutes):
full HD (110M), 450×800 (58M)

Zaheer MeraliVideo capture
Flumotion and GStreamer
Video (35 minutes):
full HD (202M)

Andrey Nechypurenko and Maksym ParkachovVideo capture
Adaptive video streaming with Ice and GStreamer
Slides
Video (35 minutes):
full HD (279M), 450×800 (107M)

ELCE 2010 videos – Now in full HD

Videos from the Embedded Linux Conference Europe, Cambridge, UK, October 2010

Just a few weeks before the next edition of the Embedded Linux Conference in San Francisco, here are the videos from the previous edition in Europe a few months ago.

These videos took more time to process than expected, because of intense months on our side, but also because of the switch to the VP8 video codec. VP8 is the new Open Source and royalty free video codec, and is a successor to the codec that Theora was derived from. Unlike Theora which is now lagging behind, it is a very close competitor to H264, both in terms of quality and video size.

The switch to VP8 allowed us to release the videos in their original full HD resolution (we now have three full HD camcorders to shoot conference videos), with video files of about the same size.

You will probably need a recent GNU/Linux distribution (such as Ubuntu 10.10) to watch these videos. As this codec released by Google is taking off quickly, you should also find solutions to watch videos on Windows and MacOS X. Don’t hesitate to post comments here about your experience playing these files. You can even watch them on the Panda board, which can decode VP8 with its hardware video decoder.

As usual, these videos are released under the terms of the Creative Commons Attribution – ShareAlike Licence version 3.0.

As often in conferences, the videos were unfortunately shot is tough lighting conditions. The organizers usually turn off the lights to make it easier for the audience to read the screen. The consequence is a high contrast between the speaker and the screen, causing the speaker to appear very dark when we film her or him together with the screen. In a number of videos, we tried to solve this by using a beach mode provided by our camcorders. While the speaker now looks great, this unfortunately blurred the screen, causing more inconvenience than benefits. We gave up this mode in the last videos and will shoot ELC 2011 is the standard way, even if the speaker looks dark again. At least, with full HD videos, you will be able to read the slides directly on the screen.

The videos from the 2010 GStreamer conference will also be available in the next days, and to help you produce your own videos, we will release our new video processing scripts soon too.

Ruud DerwigVideo capture
Welcome speech
Video (15 minutes):
full HD (228M), 450×800 (71M)

Wolfram SangVideo capture
Pengutronix
Developer’s Diary: Supporting Maintainers
Slides
Video (46 minutes):
full HD (888M)
Rekha Kumar and Nipuna GunasekeraVideo capture
Texas Instruments
Panda board demonstration
Video (14 minutes):
full HD (373M), 450×800 (85M)

Robert SchusterVideo capture
OpenJDK
OpenJDK for Embedded Linux Devices
Slides
Video (39 minutes):
full HD (947M), 450×800 (225M)
Philippe RobinVideo capture
Linaro
Facilitating Open Source Development and Collaboration
Slides
Video (46 minutes):
full HD (548M), 450×800 (160M)
Tim BirdVideo capture
Sony
Android System Programming – Tips and Tricks
Slides
Video (40 minutes):
full HD (471M), 450×800 (142M)
Mischa Jonker and Ruud DerwigVideo capture
Synopsys
Portability is for People Who Cannot Write New Programs – GNU/Linux/OS on ARC
Slides
Video (34 minutes):
full HD (517M), 450×800 (148M)
Leif LindholmVideo capture
ARM
Software Considerations When Using High-Performance Memory Systems
Slides
Video (46 minutes):
full HD (511M), 450×800 (153M)
Ravi Sankar GunturVideo capture
Samsung
A Simple Method to Detect Memory Leaks and Buffer Overruns
Slides
Video (17 minutes):
full HD (171M), 450×800 (55M)
Will NewtonVideo capture
Imagination Technologies
Exploiting On-chip Memories in Embedded Linux Applications
Slides
Video (20 minutes):
full HD (255M), 450×800 (68M)
Andrey FedotovVideo capture
AFSoft
Linux Application in Safety-Critical Environment: A Real-Life Example
Video (39 minutes):
full HD (304M), 450×800 (111M)

Anna DushistovaVideo capture
Mentor Graphics
Eclipse and Embedded Linux Developers: What It Can and What It Cannot Do For You
Slides
Video (31 minutes):
full HD (338M), 450×800 (112M)
Yoshitake KobayashiVideo capture
Toshiba
Linux Kernel Acceleration for Long-term Testing
Slides
Video (30 minutes):
full HD (249M), 450×800 (89M)
Ralf BaechleVideo capture
Wind River
Embedded Linux – The State of the Nation
Video (40 minutes):
full HD (375M), 450×800 (128M)

Jim ZemlinVideo capture
Linux Foundation
The Linux Foundation and CELF
Video (21 minutes):
full HD (282M), 450×800 (71M)

Andrew MurrayVideo capture
MPC Data
The Right Approach to Minimal Boot Times
Slides
Video (41 minutes):
full HD (472M), 450×800 (149M)
Robert Schwebel and Sascha HauerVideo capture
Pengutronix
Barebox: Booting Linux Fast and Fancy
Slides
Video (45 minutes):
full HD (779M), 450×800 (192M)
Kevin HilmanVideo capture
Deep Root Systems
Runtime Power Management
Slides
Video (45 minutes):
full HD (780M), 450×800 (195M)
Michael OpdenackerVideo capture
Bootlin
Flash Filesystem Benchmarks
Slides
Video (47 minutes):
full HD (937M), 450×800 (229M)
Ari RauchVideo capture
Texas Instruments
The Dynamic Role of Open Linux Architectures in Today’s Mobile Landscape
Slides
Video (34 minutes):
full HD (789M), 450×800 (203M)
Benjamin GaignardVideo capture
ST-Ericsson
Android and GStreamer
Slides
Video (42 minutes):
full HD (432M), 450×800 (133M)
Hans VerkuilVideo capture
Tandberg
Supporting SoC Video Subsystems in Video4linux
Slides
Video (45 minutes):
full HD (424M), 450×800 (139M)
Benjamin ZoresVideo capture
Alcatel-Lucent
State of Multimedia in 2010’s Embedded Linux Devices
Slides
Video (45 minutes):
full HD (784M), 450×800 (219M)
Iago Toral QuirogaVideo capture
Igalia/Grilo
Grilo: Integrating Multimedia Content in Applications
Slides
Video (32 minutes):
full HD (515M), 450×800 (149M)
Jean-Paul SamanVideo capture
M2X BV
Porting VLC to TI DaVinci
Slides
Video (46 minutes):
full HD (516M), 450×800 (167M)
Stefan KostVideo capture
Nokia
Meego Multimedia
Slides
Video (37 minutes):
full HD (316M), 450×800 (105M)
Vitaly WoolVideo capture
Porting Legacy Code to Linux Userspace Driver Framework
Video (26 minutes):
full HD (400M), 450×800 (108M)

Martin MichlmayrVideo capture
Debian
Adapting Debian Installer to NAS and Other Consumer Devices
Slides
Video (21 minutes):
full HD (196M), 450×800 (62M)
Frank ScholzVideo capture
Android and Its Impact On Home Entertainment and Home Automation
Video (28 minutes):
full HD (347M), 450×800 (101M)

WookeyVideo capture
Yaffs
Yaffs updates
Slides
Video (27 minutes):
full HD (312M), 450×800 (95M)
Yann E. MorinVideo capture
Crosstool-ng
Crosstool-NG, A Cross-Toolchain Generator
Slides
Video (41 minutes):
full HD (1.1G), 450×800 (185M)
Armijn HemelVideo capture
Loohuis Consulting
Introducing the Binary Analysis Tool
Slides
Video (47 minutes):
full HD (507M), 450×800 (155M)
Kees-Jan DijkzeulVideo capture
Sioux Embedded Systems
A Gentle Introduction to Autotools
Slides
Video (41 minutes):
full HD (371M), 450×800 (124M)
Klaas Van GendVideo capture
Montavista
Deflating the Virtualization Hype in 3 Simple Steps
Slides
Video (38 minutes):
full HD (507M), 450×800 (131M)
Peter KorsgaardVideo capture
Buildroot
Do More With Less – On Driver-less Interfacing with Embedded Devices
Slides
Video (48 minutes):
full HD (529M), 450×800 (173M)
Ray KinsellaVideo capture
Intel
Xen in Embedded Systems
Slides
Video (34 minutes):
full HD (380M), 450×800 (135M)
Arnout VandecappelleVideo capture
Mind
Practical Testing of Open Source Embedded Systems
Slides
Video (51 minutes):
full HD (364M), 450×800 (160M)
Carmelo Amoroso and Rosario ContarinoVideo capture
STMicroelectronics
Lightweight Prelinker for Kernel Modules
Slides
Video (45 minutes):
full HD (302M), 450×800 (129M)
Frank RowandVideo capture
Sony
Identifying Embedded Real-Time Latency Issues: I-Cache and Locks
Slides
Video (43 minutes):
full HD (272M), 450×800 (120M)
David AndersVideo capture
Texas Instruments
Board Bringup: Methods and Utilities
Slides
Video (34 minutes):
full HD (248M), 450×800 (99M)
John OgnessVideo capture
Linutronix
IPL+UBI: Flexible and Reliable with Linux as the Bootloader
Slides
Video (27 minutes):
full HD (232M), 450×800 (89M)
Vitaly WoolVideo capture
WLAN Chips in Embedded Linux Systems
Video (23 minutes):
full HD (264M), 450×800 (82M)

Grant LikelyVideo capture
Secret Lab Technologies
ARM Flattened Device Tree Status Report
Slides
Video (40 minutes):
full HD (542M), 450×800 (173M)
Koen KooiVideo capture
OpenEmbedded
The State of OpenEmbedded and Tooling to Make Life Easier
Slides
Video (44 minutes):
full HD (308M), 450×800 (122M)
Harald WelteVideo capture
OpenBSC
Running your own GSM+GPRS network using OpenBSC, OsmoSGSN and OpenGGSN
Slides
Video (49 minutes):
full HD (402M), 450×800 (163M)
Arun RaghavanVideo capture
Collabora
PulseAudio In The Embedded World
Slides
Video (30 minutes):
full HD (204M), 450×800 (88M)
Jake EdgeVideo capture
LWN.net
Understanding Threat Models for Embedded Devices
Slides
Video (29 minutes):
full HD (186M), 450×800 (80M)
Gustavo F. PadovanVideo capture
Profusion
The Linux Bluetooth Stack
Slides
Video (30 minutes):
full HD (213M), 450×800 (87M)
Klaas van GendVideo capture
Closing session
Video (62 minutes):
full HD (1.2G), 450×800 (285M)

Chris SimmondsVideo capture
2net
The Embedded Linux Quick Start Guide – Part 1
Slides
Video (52 minutes):
full HD (397M)
Chris SimmondsVideo capture
2net
The Embedded Linux Quick Start Guide – Part 2
Slides
Video (79 minutes):
full HD (660M)
Chris SimmondsVideo capture
2net
The Embedded Linux Quick Start Guide – Part 3
Slides
Video (67 minutes):
full HD (501M)
Chris SimmondsVideo capture
2net
What else can you do with Android? – Part 1
Slides
Video (49 minutes):
full HD (432M), 450×800 (144M)
Chris SimmondsVideo capture
2net
What else can you do with Android? – Part 2
Slides
Video (31 minutes):
full HD (293M), 450×800 (94M)
Chris SimmondsVideo capture
2net
What else can you do with Android? – Part 3
Slides
Video (59 minutes):
full HD (545M), 450×800 (180M)

Here are also videos of the Embedded Linux and Android tutorials by Chris Simmonds.

Barebox 2011.03 released, with contributions from Bootlin

BareboxBarebox is a bootloader started about two years ago for embedded systems of various architectures. It plays the same role as U-Boot, which is the best known project in this area, but has several advantages over U-Boot. First, it has a much better configuration and compilation system, based on the one used by the Linux kernel: instead of the rusty include/configs/myboard.h configuration headers in U-Boot, Barebox provides a nice menuconfig/xconfig/defconfig based configuration system, that everyone is familiar with. Second, Barebox has a source code organization very similar to the one of the Linux kernel and has replicated the device/driver model of the kernel. This allows to have a nice separation between device drivers and their instantiation, and a source code that looks familiar to anyone that already does kernel development.

Of course, as Barebox is newer than U-Boot, the number of architectures and platforms is more limited, but it is growing rapidly. It already supports ARM, PPC, Blackfin, x86 and a testing sandbox architecture. On ARM, the supported platforms are AT91, EP93xx, iMX, Nomadik, OMAP, S3C24xx and Versatile. On PPC, a single mpc5xxx platform is supported. Patches to add support for the NIOS architecture have also been posted recently (NIOS is a soft-core architecture from Altera).

As a young but fast-growing project, Barebox has chosen a quick development cycle: new releases are made each month, and Barebox 2011.03 has been released a few days ago. It has many ARM and generic improvements, but is also the first release with contributions from Bootlin :

Gregory CLEMENT (3):
      BMP: Add support for 32bpp video frame buffer
      ARM STM/i.MX: Add possibility to choose the bit per pixel for STM video driver
      fb i.MX23/28: Add the reset control of LCD

My colleague Gregory Clement has contributed several improvements to framebuffer support on the i.MX platform. Those improvements were made in the context of a customer project, for which Barebox was used as a way of showing immediately after the device start-up a nice logo on the screen, while the system continues to boot in the background. Initially, the user had to wait 20+ seconds to see a logo on the screen showing that the system was booting. With our Barebox based solution, a logo is now visible on the screen less than 2 seconds after the power on button is pushed.

Snowball, a new community Linux development platform

Snowball platformThe success of the BeagleBoard platform, a low-cost development platform, that has greatly contributed to the success of Texas Instruments OMAP3 processor in the embedded Linux industry, seems to have inspired another processor manufacturer: ST Ericsson. They have recently unveiled Snowball, a low-cost development platform for their AP9500 processor, which features a dual Cortex A-9 ARM core and a Mali 400 GPU.

The development board is designed and produced by our partner Calao Systems, and offers the following features:

  • The AP9500 processor, dual Cortex-A9 and Mali 400 GPU
  • 4 to 8 GB of e-MMC storage
  • 1 GB of LP-DDR2 RAM
  • Micro-SD slot
  • Ethernet connector, Wifi and Bluetooth
  • HDMI output, composite video output
  • Audio in/out
  • USB On The Go
  • Battery charger
  • On-board battery to keep time
  • Serial port connector, JTAG connector, MiPi 34 debug connector
  • Builtin GPS
  • 3-axis accelerometer, magnetometer and gyrometer, one pressure sensor
  • Expansion connectors to access SPI, I2C, LCD, MiPi devices, GPIO, UART, etc.
  • Last but not least, the board can be powered via USB (through a regular cable or through a Y one if power hungry devices like Wifi are used.)

The technical documentation page has a few more details, but at this time, they isn’t a lot of public information available about the AP9500 processor. I hope that ST Ericsson will fully understand how open source works and will soon release datasheets for the AP9500 in an open way. Interestingly, the AP9500 does not use the traditional PowerVR SGX 3D graphics core designed by Imagination Technologies and found in many other ARM processors, but instead uses the Mali graphics core, which is designed directly by ARM. It seems ARM has already open-sourced the kernel side bits of their graphic drivers, but it looks like a proprietary binary blob in userspace is still present.

The board will be available in two variants:

  • A Product Development Kit variant for 241 Euros.
  • A Software Development Kit variant for 165 Euros. My understanding is that the only difference between the two are the expansion connectors, present on the PDK variant but not on the SDK variant.

The board should be widely available at the end of Q2 2011, i.e around June, though at Bootlin, we will receive our first samples by the end of March thanks to our partnership with Calao Systems. The Snowball platform is supported by the Igloo Community, which hosts mailing-lists, an IRC channel, documentation and will also provide Meego and Android builds for the Snowball in the future.

Stay tuned on this blog. As soon as we get our own boards, we will write about our experiments with them.

Buildroot 2011.02 released, with many interesting updates and commercial support!

Buildroot logoAs usual, the latest Buildroot version has been released just in time on the last day of the month: Buildroot 2011.02 is available for download!

This release of the increasingly popular embedded Linux build system provides new interesting features and updates:

  • Support of external toolchains has been improved with support for toolchain profiles. Those are predefined configurations for well-known toolchains such as the CodeSourcery ones for ARM, PowerPC, MIPS and SuperH. Buildroot is now capable of automatically downloading and installing those external toolchains, which is much easier than downloading them manually. It’s now easy to provide users with Buildroot configurations that use well known toolchains, without requiring them to pre-install anything specific.
  • Support for board configurations has been completely rewritten and largely simplified. All board-specific Makefile and configuration options have been removed, and instead, each board is represented by a single, simple (less than 20 lines) defconfig file, in the configs/ directory. In addition to the existing configuration, we have added support for the Mini2440 platform but also for many Qemu emulated platforms: Qemu ARM Versatile, Qemu MIPSel Malta, Qemu PowerPC G3 Beige, Qemu SH4 r2d and Qemu x86. Those configurations allow to easily produce a known-to-work system for the Qemu emulator, making Buildroot even easier to start working on your embedded Linux system. See the documentation for more details on how to add your own board support.
  • Support for the Blackfin architecture has been added, thanks to Mike Frysinger. This support came along with a lot of fixes to make Buildroot work better for non-MMU architectures, since Blackfin is the first actively supported non-MMU architecture in Buildroot. There will certainly be further improvements to support non-MMU architectures, and hopefully additional non-MMU platforms will be added. For those platforms, Buildroot is generally a very good embedded Linux build system, as those architectures are typically used for small to medium sized systems, with a relatively limited number of components.
  • The Crosstool-NG back-end has been improved and extended to support more Buildroot options, and has been upgraded to a newer Crosstool-NG version. This back-end is the third mode for Buildroot toolchain: it allows Buildroot to use Crosstool-NG as the toolchain generator.
  • Ccache support has been reworked and it now works properly. Since Buildroot often requires complete rebuilds from scratch, having the ccache compiler cache is very nice. On my laptop, compiling from scratch a sample Buildroot system was taking 5 minutes and 29 seconds without ccache, and now only takes 3 minutes and 40 seconds with ccache enabled and the cache already filled by a previous build.
  • A new CMake infrastructure has been added for packages, next to the existing generic and autotools infrastructure. For the moment, only two packages are using the infrastructure (cdrkit and libcuefile), but CMake is an increasingly popular build system and we will definitely see more packages using it in the future. Moreover, Buildroot generates a CMake toolchain file that describes the toolchain used by Buildroot, and which makes it very easy to cross-compile external libraries/applications for the Buildroot system using cmake -DCMAKE_TOOLCHAIN_FILE=/path/to/buildroot/output/toolchainfile.cmake.
  • A very nice cleanup job of the internal toolchain build process has been started by Gustavo Zacarias. The build process of binutils, gmp, mpfr and mpc has been migrated to proper packages, and this will also be done for gcc and gdb in the future.
  • As preliminary steps towards the generation of a standalone SDK from Buildroot, two important changes have been made. First, the staging directory is now inside $(O)/host/usr/PLATFORM-TUPLE/sysroot/, but a symbolic link from $(O)/staging has been kept for compatibility. This change will allow the $(O)/host directory to be the standalone SDK in the future. The second change is on pkg-config: its configuration has been adjusted so that it behaves properly to compile target packages without needing any environment variables or options. It makes the Buildroot pkg-config much easier to use to compile external applications.
  • The Python package has been upgraded to the latest Python version, 2.7.1. This was needed since a long time, since the version of Python we had in Buildroot was only 2.4. Moreover, the package has been completely rewritten, with more options, and has been tested on several platforms. Two external Python modules, python-mad and python-serial have also been added as packages, to show how such modules can be integrated into Buildroot.
  • A set of packages to add support in GStreamer for the TI DSP codecs has been added: gst-dsp, gst-omapfb, tidsp-binaries, dsp-tools, thanks to Felipe Contreras.
  • We have a bunch of new packages as well: mpd, the Music Player Daemon with many audio codecs and libraries, the dhrystone and whetstone benchmarks and other tools such as xmlstarlet, fbgrab, irda-utils, lsuio, etc.
  • Many other packages have been upgraded or fixed, and the results of our random configuration builds are much, much better than they were in the past.

As a Bootlin engineer, I have again contributed significantly to this release: Peter Korsgaard, the Buildroot maintainer, has done 171 commits, Gustavo Zacarias has done 119 commits and I have done 103 commits. The next committer is Mike Frysinger (for the great Blackfin support) with 22 commits.

It is with this great Buildroot knowledge and experience that Bootlin has launched a few weeks ago an official offering of Buildroot commercial support. If you are using Buildroot for your embedded product, or want the buyers of your hardware platform to have a simple but efficient embedded Linux build system and you need help, development or consulting, do not hesitate to contact us.

Buildroot 2010.11 release and roadmap

Buildroot logoThe 2010.11 release of Buildroot has been published on November, 30th. Buildroot is a tool that eases the process of building an embedded Linux system: cross-compiling toolchain, root filesystem with dozens or hundreds of libraries and applications, bootloader and kernel.

Release 2010.11

Releases are made every three months, and the latest 2010.11 release has a number of improvements :

  • Experimental crosstool-NG back-end for handling the toolchain. Before, Buildroot could either compile a toolchain for you, or use an already existing external toolchain. However, the internal Buildroot process for building a toolchain is limited to uClibc and its maintenance is a duplication of the work done by the Crosstool-NG community. While this back-end is only experimental at the moment, the intention is to make it the default back-end in some future release, if everything works fine. Thanks to Yann E. Morin for implementing this back-end, and making related changes to Crosstool-NG.
  • The Kconfig infrastructure has been updated to the one of 2.6.36 and has been cleaned up (we now have a nice quilt patch series). This brings savedefconfig (for minimal Buildroot configurations) and nconfig support. We now also create a convenience Makefile wrapper in the output directory for out-of-tree builds, like the kernel has. So once you have done a first make O=/some/path menuconfig, you can go into /some/path and directly use make something without having to specify the output directory.
  • Old-style package hooks (*_HOOK_POST_*) have been removed. This was just a clean up process, because we have a more modern way of allowing packages to hook specific commands at various steps of the build process.
  • Download handling has been reworked and support for git/svn downloads was added, so that packages only available through version control systems can be added easily in Buildroot. The make source (to download all needed tarballs and files so that the build can be done completely offline) and make external-deps (to show all files that would need to be downloaded) commands have been improved to fix several issues
  • On the architecture side, support for ARM Cortex A9 and Sparc LEON variants was added. Support for Alpha, Cris, IA64 and Sparc64 (deprecated in 2010.08) was removed.
  • We also added a few more packages: argp-standalone, gdk-pixbuf, gpsd, gst-ffmpeg, libmpeg2, kbd, librsvg, nuttcp, rng-tools, rrdtool and xz. And removed some deprecated packages: dillo, libglib12, libgtk12, microwin and pcmcia.
  • Many, many packages have been updated. In particular, the Gtk+ library has been updated from 2.12 to 2.20, keeping the DirectFB support functional thanks to the work of Lionel Landwerlin (the DirectFB support in Gtk+ was broken since 2.12). So, finally, this brings us a recent and fresh Gtk+ library in Buildroot.
  • Many packages were converted to the autotargets or gentargets infrastructures (that we use to describe how a particular package should be downloaded, extracted, configured, built and installed), in particular thanks to the work of Martin Banky.

I have again contributed quite a bit to this release, but not as much as I wanted to. Here are the numbers:

git shortlog -s -n 2010.08..
   168  Peter Korsgaard
   115  Thomas Petazzoni
    55  Gustavo Zacarias
    37  Martin Banky
    26  Yann E. MORIN
    15  Lionel Landwerlin
    13  Mike Frysinger
    10  Paulius Zaleckas
     8  Maxime Petazzoni
     4  Konrad Eisele
     3  Chih-Min Chao
     2  Yegor Yefremov
     1  Andy Gibbs
     1  Felipe Contreras
     1  Frederik Pasch
     1  Heiko Zuerker
     1  Javier Viguera
     1  Luca Ceresoli
     1  Marcelo Roberto Jimenez
     1  Marcus Osdoba
     1  Matt Johnson
     1  Paul Burton
     1  Paul Jones
     1  Stanislav Bogatyrev
     1  Thomas Rudin
     1  Will Newton

The things I worked on are: cleanup of the kconfig patches and switch to the 2.6.36 one, removal of old-style hooks and many other small package cleanups, Gtk+ upgrade and documentation improvements. I’m also very happy to see that the number of contributors is increasing, as is the number of questions asked on the list, on IRC and on the bug tracker.

Buildroot developer day

A Buildroot Developer Day, meeting of some of the Buildroot developers, took place on October, 29th in Cambridge, UK, just after the Embedded Linux Conference Europe. Peter Korsgaard (Buildroot maintainer), Lionel Landwerlin (contributor), Yann E. Morin (contributor, Crosstool-NG developer), Nicolas Ferre (Atmel), Patrice Vilchez (Atmel) and Thomas Petazzoni (Bootlin) attended the meeting.

There has been discussions about cleaning up board support, libtool issues, Crosstool-NG integration, toolchain cleanup, top-level parallel make and package management. A report has been posted to the Buildroot mailing list.

The next Buildroot Developer Day will take place on Monday, 7th February, just after FOSDEM, in Brussels.

Roadmap for 2011.02

Here are some of the things that are in the pipeline, for 2011.02 if everything goes well.

  • Conversion of more (all remaining?) packages to the gentargets and autotargets infrastructures.
  • Cleanup of board support in Buildroot. We will now use minimal defconfigs for both Buildroot itself and for kernel configuration. The messy target/device/ directory will be cleaned-up. And we will add support for some emulated boards based on Qemu (x86, ARM, MIPS, PowerPC, SH4). This work is already done, so it’s very likely to be in 2011.02
  • Better support of devtmpfs, mdev and udev. An option will allow the user to select between static device creation and those three techniques for having a dynamic /dev. This is also ready.
  • Improvements in the external toolchain configuration and setup. Buildroot will be able to automatically download some well-known external toolchains, making them easier to use.
  • The Buildroot internal toolchain build process will be reworked, with conversion of most of the code to the package infrastructures, and the toolchain binaries and sysroot move to $(HOST_DIR). This will make it much, much easier to use the toolchain produced by Buildroot outside of Buildroot, and will allow us to generate a SDK to be shared with application developers, without requiring them to run Buildroot. See the report from the Buildroot Developers Day for details. This work has already been started by Gustavo Zacarias, and at least part of it will be in 2011.02.
  • Package management will be improved. Buildroot will know which package installed what in the different directories, and will therefore be able to cleanly remove a package from the system if it has been disabled in the configuration, without requiring a completely clean rebuild. Later on, this work could be used to generate .ipk packages, but for the moment, the focus is on being able to remove libraries and applications from the system when they are unselected from the configuration. This work has already been started by Lionel Landwerlin, and we will do our best to merge it into 2011.02.
  • Support for non-MMU architectures, and particularly Blackfin. Mike Frysinger has already posted patches to add Blackfin and support non-MMU for several packages. This is really a good news, as Buildroot is a good fit for non-MMU systems.
  • Public access to regression tests results

Toolchain cleanup work and board cleanup work are really interesting pieces, because they are the two remaining parts of Buildroot that haven’t been cleaned up and/or rewritten since the Buildroot project came back to life late 2008/early 2009. There are of course still some areas that need improvements of course, but for me, those two pieces are really closing the large cleanup work that has been started almost two years ago.

Of course, don’t hesitate to contact the Buildroot mailing-list, the IRC channel or the bug tracker if you have any questions or issues in using Buildroot.

Busybox 1.18.0, a few new applets

BusyBox logoBusybox, the embedded Linux swiss-army knife, has seen a new release recently: 1.18.0. As usual, it brings dozens of bug fixes, but it also includes a few new applets that are worth noting:

  • pmap, which shows the memory mappings for a particular process. It is just a formatted version of what you can find in /proc/PID/maps, but still nice to have. Knowing the mappings in a particular process is often useful for debugging purposes.
  • add-shell and remove-shell, that respectively add and remove a shell from /etc/shells. This file lists the shells that are acceptable for the chsh command, for example.
  • nandwrite and nanddump, that respectively write to NAND with bad block management, and dumps NAND flash contents. Those two commands expand the set of MTD-related commands in Busybox and are replacements for the version available in mtd-utils. There were already other MTD related utilities in Busybox: flashcp (write to a NOR flash), flash_eraseall (erase a NOR or NAND flash), flash_lock and flash_unlock to protect/unprotect sectors on flash storage. The main advantage is that these utilities relieve you from having to cross-compile mtd-utils, which is never as simple as cross-compiling BusyBox.
  • base64, a base64 encoder and decoder.
  • mpstat, iostat and powertop, three new statistics programs. mpstat (as in multiprocessor stat) reports interrupts and CPU usage on multiprocessor systems, on a per-CPU basis. iostat reports CPU and block device statistics. powertop is a lightweight variant of the popular tool used to diagnose sources of wake-ups, but is only limited to Intel-based machines at the moment.
  • nbd-client, a client for the Network Block Device protocol, so that an embedded system can mount volumes shared by NBD
  • blockdev, allows to perform some ioctl() on block devices such as get sector size, get/set block size, flush buffers, etc.

Have fun with Busybox!

Linux Kernel Development, third edition

Linux Kernel Development, by Robert Love, 3rd edition

Linux Kernel Development is a book authored by Robert Love, a famous kernel developer. Contrary to the very famous Linux Device Drivers book, Linux Kernel Development is not oriented towards driver development, but instead covers how the core Linux kernel works.

Having this knowledge is not absolutely necessary to write Linux device drivers, but having a good overall understanding of the kernel always help to understand what’s going on in your system, even at the application level. In July this year, the third edition of Linux Kernel Development has been published, which upgrades the book contents to kernel version 2.6.34, a good opportunity to have a new look at the book that Bootlin received a few weeks ago.

After a quick introduction to the kernel sources (configuring, building, organization of the source tree), the book immediately dives into kernel internals:

  • Process management: how the kernel represents processes and their state, how processes are created inside the kernel, how threads are handled, are processes are terminated.
  • Process scheduling: a full chapter dedicated to the Linux kernel process scheduler. The new CFS scheduler is of course covered in great detail, with large portion of commented source code, for those who want to understand the fine details of the scheduler. Topics such as process sleep/wake-up, preemption, real-time scheduling policies are also covered.
  • System calls are then covered: how they are implemented, how parameters are passed from userspace to the kernel, etc. The call path from your user-space application down to the kernel is well explained in this chapter.
  • Kernel data structures: a generic chapter which details the kernel API for linked lists, queues, maps, and binary trees. Those APIs are omni-present inside the kernel, and it’s therefore a good idea to know how they work, both for understand existing code and for writing new code.
  • Interrupts: how interrupts are handled and how one can write an interrupt handler. Unfortunately, the newly introduced threaded interrupt handlers are not covered, but it’s true that their usage is not yet very widespread inside the mainline kernel.
  • Bottom halves and deferring work, a topic closely related to interrupt handling. It covers bottom halves, softirqs, tasklets and workqueues.
  • Kernel synchronization: two chapters are dedicated to this topic. First a chapter detailing why synchronization is needed, what are the sources of concurrency and what should be protected against concurrent access. And then a chapter detailing the mechanisms provided by the kernel to implement proper synchronization: atomic operations, spin locks, reader-writer spin locks, semaphores, reader-writer semaphores, mutexes, completion variables, sequential locks, preemption disabling, ordering and barriers
  • Timers and time management details how the kernel manages time: ticks, jiffies counter, timers, delaying execution of code are covered in this chapter. There are unfortunately no details about the clocksource and clockevents infrastructure, and no details about how timers and high-resolution timers are implemented. Contrary to other chapters that go fairly deep into the implementation details, this one mostly only covers the API to time management rather than the internals.
  • Memory management is the topic of the following chapter: physical memory management with the page allocator and the physical zones, then the kmalloc, vmalloc and SLAB allocators are covered. High-memory mappings, a topic specific to 32 bits architectures having more than a gigabyte of RAM is also covered in detail. The per-cpu interface is also covered, and will help those who want to understand parts of the kernel that have been optimized for scalability on multiple CPUs.
  • The Virtual Filesystem, with its different objects: superblock, inode, dentry and file is covered in good detail.
  • The block layer is then covered, with a description of the role of the bio structure, the request queues, and the I/O schedulers.
  • Then, the book goes back to more details about the internals of memory management: the mm_struct memory descriptor, the virtual memory areas (so called VMAs) and how they relate with the mmap/munmap system calls. The next chapter continues with a detailed description of the page cache implementation.
  • A fairly strange chapter called “Devices and modules” gives some information about kernel modules (how to build them, how to use module parameters, how dependencies are handled), then covers the internal of the device model (kobjects, ktypes, ksets, krefs) and finally sysfs. Just like the chapter covering the device model in the third edition of Linux Device Drivers, I think it totally misses the point. All the kobject, ktypes, ksets and krefs stuff is very low-level plumbing used by the Linux device model, but it is not exactly what the driver developer needs to interact with in the first place. In my opinion, a good description of the device model should rather explain what struct bus_type, struct driver and struct device are, how they are specialized for the different bus types in pci_driver, pci_device, usb_driver, usb_device, platform_driver, platform_device, and how the registration/probing of drivers and devices is done. I’ve recently given a talk about this topic, the video is in French, but the slides are in English.
  • Debugging is then covered: printk of course, but also oopses, debugging-related kernel options, the magic SysRq key, kernel debuggers, etc.
  • A rather generic chapter about Portability is then proposed, and finally a chapter about Patches, Hacking and the community that details the kernel community, the kernel coding style, how to generate and submit patches, etc.

All in all, Linux Kernel Development remains very good reading. I particularly appreciate the writing style of Robert Love, who manages to make a deeply-technical book interesting and easy to read. Of course, there are some topics in the kernel in which I had to dive myself and for which I’d expect to see more details in this book, but giving every possible detail about a huge beast such as the Linux kernel in just 400 pages is not possible!