Category: Technical

Snowball, a new community Linux development platform

Snowball platformThe success of the BeagleBoard platform, a low-cost development platform, that has greatly contributed to the success of Texas Instruments OMAP3 processor in the embedded Linux industry, seems to have inspired another processor manufacturer: ST Ericsson. They have recently unveiled Snowball, a low-cost development platform for their AP9500 processor, which features a dual Cortex A-9 ARM core and a Mali 400 GPU.

The development board is designed and produced by our partner Calao Systems, and offers the following features:

  • The AP9500 processor, dual Cortex-A9 and Mali 400 GPU
  • 4 to 8 GB of e-MMC storage
  • 1 GB of LP-DDR2 RAM
  • Micro-SD slot
  • Ethernet connector, Wifi and Bluetooth
  • HDMI output, composite video output
  • Audio in/out
  • USB On The Go
  • Battery charger
  • On-board battery to keep time
  • Serial port connector, JTAG connector, MiPi 34 debug connector
  • Builtin GPS
  • 3-axis accelerometer, magnetometer and gyrometer, one pressure sensor
  • Expansion connectors to access SPI, I2C, LCD, MiPi devices, GPIO, UART, etc.
  • Last but not least, the board can be powered via USB (through a regular cable or through a Y one if power hungry devices like Wifi are used.)

The technical documentation page has a few more details, but at this time, they isn’t a lot of public information available about the AP9500 processor. I hope that ST Ericsson will fully understand how open source works and will soon release datasheets for the AP9500 in an open way. Interestingly, the AP9500 does not use the traditional PowerVR SGX 3D graphics core designed by Imagination Technologies and found in many other ARM processors, but instead uses the Mali graphics core, which is designed directly by ARM. It seems ARM has already open-sourced the kernel side bits of their graphic drivers, but it looks like a proprietary binary blob in userspace is still present.

The board will be available in two variants:

  • A Product Development Kit variant for 241 Euros.
  • A Software Development Kit variant for 165 Euros. My understanding is that the only difference between the two are the expansion connectors, present on the PDK variant but not on the SDK variant.

The board should be widely available at the end of Q2 2011, i.e around June, though at Bootlin, we will receive our first samples by the end of March thanks to our partnership with Calao Systems. The Snowball platform is supported by the Igloo Community, which hosts mailing-lists, an IRC channel, documentation and will also provide Meego and Android builds for the Snowball in the future.

Stay tuned on this blog. As soon as we get our own boards, we will write about our experiments with them.

Buildroot 2011.02 released, with many interesting updates and commercial support!

Buildroot logoAs usual, the latest Buildroot version has been released just in time on the last day of the month: Buildroot 2011.02 is available for download!

This release of the increasingly popular embedded Linux build system provides new interesting features and updates:

  • Support of external toolchains has been improved with support for toolchain profiles. Those are predefined configurations for well-known toolchains such as the CodeSourcery ones for ARM, PowerPC, MIPS and SuperH. Buildroot is now capable of automatically downloading and installing those external toolchains, which is much easier than downloading them manually. It’s now easy to provide users with Buildroot configurations that use well known toolchains, without requiring them to pre-install anything specific.
  • Support for board configurations has been completely rewritten and largely simplified. All board-specific Makefile and configuration options have been removed, and instead, each board is represented by a single, simple (less than 20 lines) defconfig file, in the configs/ directory. In addition to the existing configuration, we have added support for the Mini2440 platform but also for many Qemu emulated platforms: Qemu ARM Versatile, Qemu MIPSel Malta, Qemu PowerPC G3 Beige, Qemu SH4 r2d and Qemu x86. Those configurations allow to easily produce a known-to-work system for the Qemu emulator, making Buildroot even easier to start working on your embedded Linux system. See the documentation for more details on how to add your own board support.
  • Support for the Blackfin architecture has been added, thanks to Mike Frysinger. This support came along with a lot of fixes to make Buildroot work better for non-MMU architectures, since Blackfin is the first actively supported non-MMU architecture in Buildroot. There will certainly be further improvements to support non-MMU architectures, and hopefully additional non-MMU platforms will be added. For those platforms, Buildroot is generally a very good embedded Linux build system, as those architectures are typically used for small to medium sized systems, with a relatively limited number of components.
  • The Crosstool-NG back-end has been improved and extended to support more Buildroot options, and has been upgraded to a newer Crosstool-NG version. This back-end is the third mode for Buildroot toolchain: it allows Buildroot to use Crosstool-NG as the toolchain generator.
  • Ccache support has been reworked and it now works properly. Since Buildroot often requires complete rebuilds from scratch, having the ccache compiler cache is very nice. On my laptop, compiling from scratch a sample Buildroot system was taking 5 minutes and 29 seconds without ccache, and now only takes 3 minutes and 40 seconds with ccache enabled and the cache already filled by a previous build.
  • A new CMake infrastructure has been added for packages, next to the existing generic and autotools infrastructure. For the moment, only two packages are using the infrastructure (cdrkit and libcuefile), but CMake is an increasingly popular build system and we will definitely see more packages using it in the future. Moreover, Buildroot generates a CMake toolchain file that describes the toolchain used by Buildroot, and which makes it very easy to cross-compile external libraries/applications for the Buildroot system using cmake -DCMAKE_TOOLCHAIN_FILE=/path/to/buildroot/output/toolchainfile.cmake.
  • A very nice cleanup job of the internal toolchain build process has been started by Gustavo Zacarias. The build process of binutils, gmp, mpfr and mpc has been migrated to proper packages, and this will also be done for gcc and gdb in the future.
  • As preliminary steps towards the generation of a standalone SDK from Buildroot, two important changes have been made. First, the staging directory is now inside $(O)/host/usr/PLATFORM-TUPLE/sysroot/, but a symbolic link from $(O)/staging has been kept for compatibility. This change will allow the $(O)/host directory to be the standalone SDK in the future. The second change is on pkg-config: its configuration has been adjusted so that it behaves properly to compile target packages without needing any environment variables or options. It makes the Buildroot pkg-config much easier to use to compile external applications.
  • The Python package has been upgraded to the latest Python version, 2.7.1. This was needed since a long time, since the version of Python we had in Buildroot was only 2.4. Moreover, the package has been completely rewritten, with more options, and has been tested on several platforms. Two external Python modules, python-mad and python-serial have also been added as packages, to show how such modules can be integrated into Buildroot.
  • A set of packages to add support in GStreamer for the TI DSP codecs has been added: gst-dsp, gst-omapfb, tidsp-binaries, dsp-tools, thanks to Felipe Contreras.
  • We have a bunch of new packages as well: mpd, the Music Player Daemon with many audio codecs and libraries, the dhrystone and whetstone benchmarks and other tools such as xmlstarlet, fbgrab, irda-utils, lsuio, etc.
  • Many other packages have been upgraded or fixed, and the results of our random configuration builds are much, much better than they were in the past.

As a Bootlin engineer, I have again contributed significantly to this release: Peter Korsgaard, the Buildroot maintainer, has done 171 commits, Gustavo Zacarias has done 119 commits and I have done 103 commits. The next committer is Mike Frysinger (for the great Blackfin support) with 22 commits.

It is with this great Buildroot knowledge and experience that Bootlin has launched a few weeks ago an official offering of Buildroot commercial support. If you are using Buildroot for your embedded product, or want the buyers of your hardware platform to have a simple but efficient embedded Linux build system and you need help, development or consulting, do not hesitate to contact us.

Buildroot 2010.11 release and roadmap

Buildroot logoThe 2010.11 release of Buildroot has been published on November, 30th. Buildroot is a tool that eases the process of building an embedded Linux system: cross-compiling toolchain, root filesystem with dozens or hundreds of libraries and applications, bootloader and kernel.

Release 2010.11

Releases are made every three months, and the latest 2010.11 release has a number of improvements :

  • Experimental crosstool-NG back-end for handling the toolchain. Before, Buildroot could either compile a toolchain for you, or use an already existing external toolchain. However, the internal Buildroot process for building a toolchain is limited to uClibc and its maintenance is a duplication of the work done by the Crosstool-NG community. While this back-end is only experimental at the moment, the intention is to make it the default back-end in some future release, if everything works fine. Thanks to Yann E. Morin for implementing this back-end, and making related changes to Crosstool-NG.
  • The Kconfig infrastructure has been updated to the one of 2.6.36 and has been cleaned up (we now have a nice quilt patch series). This brings savedefconfig (for minimal Buildroot configurations) and nconfig support. We now also create a convenience Makefile wrapper in the output directory for out-of-tree builds, like the kernel has. So once you have done a first make O=/some/path menuconfig, you can go into /some/path and directly use make something without having to specify the output directory.
  • Old-style package hooks (*_HOOK_POST_*) have been removed. This was just a clean up process, because we have a more modern way of allowing packages to hook specific commands at various steps of the build process.
  • Download handling has been reworked and support for git/svn downloads was added, so that packages only available through version control systems can be added easily in Buildroot. The make source (to download all needed tarballs and files so that the build can be done completely offline) and make external-deps (to show all files that would need to be downloaded) commands have been improved to fix several issues
  • On the architecture side, support for ARM Cortex A9 and Sparc LEON variants was added. Support for Alpha, Cris, IA64 and Sparc64 (deprecated in 2010.08) was removed.
  • We also added a few more packages: argp-standalone, gdk-pixbuf, gpsd, gst-ffmpeg, libmpeg2, kbd, librsvg, nuttcp, rng-tools, rrdtool and xz. And removed some deprecated packages: dillo, libglib12, libgtk12, microwin and pcmcia.
  • Many, many packages have been updated. In particular, the Gtk+ library has been updated from 2.12 to 2.20, keeping the DirectFB support functional thanks to the work of Lionel Landwerlin (the DirectFB support in Gtk+ was broken since 2.12). So, finally, this brings us a recent and fresh Gtk+ library in Buildroot.
  • Many packages were converted to the autotargets or gentargets infrastructures (that we use to describe how a particular package should be downloaded, extracted, configured, built and installed), in particular thanks to the work of Martin Banky.

I have again contributed quite a bit to this release, but not as much as I wanted to. Here are the numbers:

git shortlog -s -n 2010.08..
   168  Peter Korsgaard
   115  Thomas Petazzoni
    55  Gustavo Zacarias
    37  Martin Banky
    26  Yann E. MORIN
    15  Lionel Landwerlin
    13  Mike Frysinger
    10  Paulius Zaleckas
     8  Maxime Petazzoni
     4  Konrad Eisele
     3  Chih-Min Chao
     2  Yegor Yefremov
     1  Andy Gibbs
     1  Felipe Contreras
     1  Frederik Pasch
     1  Heiko Zuerker
     1  Javier Viguera
     1  Luca Ceresoli
     1  Marcelo Roberto Jimenez
     1  Marcus Osdoba
     1  Matt Johnson
     1  Paul Burton
     1  Paul Jones
     1  Stanislav Bogatyrev
     1  Thomas Rudin
     1  Will Newton

The things I worked on are: cleanup of the kconfig patches and switch to the 2.6.36 one, removal of old-style hooks and many other small package cleanups, Gtk+ upgrade and documentation improvements. I’m also very happy to see that the number of contributors is increasing, as is the number of questions asked on the list, on IRC and on the bug tracker.

Buildroot developer day

A Buildroot Developer Day, meeting of some of the Buildroot developers, took place on October, 29th in Cambridge, UK, just after the Embedded Linux Conference Europe. Peter Korsgaard (Buildroot maintainer), Lionel Landwerlin (contributor), Yann E. Morin (contributor, Crosstool-NG developer), Nicolas Ferre (Atmel), Patrice Vilchez (Atmel) and Thomas Petazzoni (Bootlin) attended the meeting.

There has been discussions about cleaning up board support, libtool issues, Crosstool-NG integration, toolchain cleanup, top-level parallel make and package management. A report has been posted to the Buildroot mailing list.

The next Buildroot Developer Day will take place on Monday, 7th February, just after FOSDEM, in Brussels.

Roadmap for 2011.02

Here are some of the things that are in the pipeline, for 2011.02 if everything goes well.

  • Conversion of more (all remaining?) packages to the gentargets and autotargets infrastructures.
  • Cleanup of board support in Buildroot. We will now use minimal defconfigs for both Buildroot itself and for kernel configuration. The messy target/device/ directory will be cleaned-up. And we will add support for some emulated boards based on Qemu (x86, ARM, MIPS, PowerPC, SH4). This work is already done, so it’s very likely to be in 2011.02
  • Better support of devtmpfs, mdev and udev. An option will allow the user to select between static device creation and those three techniques for having a dynamic /dev. This is also ready.
  • Improvements in the external toolchain configuration and setup. Buildroot will be able to automatically download some well-known external toolchains, making them easier to use.
  • The Buildroot internal toolchain build process will be reworked, with conversion of most of the code to the package infrastructures, and the toolchain binaries and sysroot move to $(HOST_DIR). This will make it much, much easier to use the toolchain produced by Buildroot outside of Buildroot, and will allow us to generate a SDK to be shared with application developers, without requiring them to run Buildroot. See the report from the Buildroot Developers Day for details. This work has already been started by Gustavo Zacarias, and at least part of it will be in 2011.02.
  • Package management will be improved. Buildroot will know which package installed what in the different directories, and will therefore be able to cleanly remove a package from the system if it has been disabled in the configuration, without requiring a completely clean rebuild. Later on, this work could be used to generate .ipk packages, but for the moment, the focus is on being able to remove libraries and applications from the system when they are unselected from the configuration. This work has already been started by Lionel Landwerlin, and we will do our best to merge it into 2011.02.
  • Support for non-MMU architectures, and particularly Blackfin. Mike Frysinger has already posted patches to add Blackfin and support non-MMU for several packages. This is really a good news, as Buildroot is a good fit for non-MMU systems.
  • Public access to regression tests results

Toolchain cleanup work and board cleanup work are really interesting pieces, because they are the two remaining parts of Buildroot that haven’t been cleaned up and/or rewritten since the Buildroot project came back to life late 2008/early 2009. There are of course still some areas that need improvements of course, but for me, those two pieces are really closing the large cleanup work that has been started almost two years ago.

Of course, don’t hesitate to contact the Buildroot mailing-list, the IRC channel or the bug tracker if you have any questions or issues in using Buildroot.

Busybox 1.18.0, a few new applets

BusyBox logoBusybox, the embedded Linux swiss-army knife, has seen a new release recently: 1.18.0. As usual, it brings dozens of bug fixes, but it also includes a few new applets that are worth noting:

  • pmap, which shows the memory mappings for a particular process. It is just a formatted version of what you can find in /proc/PID/maps, but still nice to have. Knowing the mappings in a particular process is often useful for debugging purposes.
  • add-shell and remove-shell, that respectively add and remove a shell from /etc/shells. This file lists the shells that are acceptable for the chsh command, for example.
  • nandwrite and nanddump, that respectively write to NAND with bad block management, and dumps NAND flash contents. Those two commands expand the set of MTD-related commands in Busybox and are replacements for the version available in mtd-utils. There were already other MTD related utilities in Busybox: flashcp (write to a NOR flash), flash_eraseall (erase a NOR or NAND flash), flash_lock and flash_unlock to protect/unprotect sectors on flash storage. The main advantage is that these utilities relieve you from having to cross-compile mtd-utils, which is never as simple as cross-compiling BusyBox.
  • base64, a base64 encoder and decoder.
  • mpstat, iostat and powertop, three new statistics programs. mpstat (as in multiprocessor stat) reports interrupts and CPU usage on multiprocessor systems, on a per-CPU basis. iostat reports CPU and block device statistics. powertop is a lightweight variant of the popular tool used to diagnose sources of wake-ups, but is only limited to Intel-based machines at the moment.
  • nbd-client, a client for the Network Block Device protocol, so that an embedded system can mount volumes shared by NBD
  • blockdev, allows to perform some ioctl() on block devices such as get sector size, get/set block size, flush buffers, etc.

Have fun with Busybox!

Linux Kernel Development, third edition

Linux Kernel Development, by Robert Love, 3rd edition

Linux Kernel Development is a book authored by Robert Love, a famous kernel developer. Contrary to the very famous Linux Device Drivers book, Linux Kernel Development is not oriented towards driver development, but instead covers how the core Linux kernel works.

Having this knowledge is not absolutely necessary to write Linux device drivers, but having a good overall understanding of the kernel always help to understand what’s going on in your system, even at the application level. In July this year, the third edition of Linux Kernel Development has been published, which upgrades the book contents to kernel version 2.6.34, a good opportunity to have a new look at the book that Bootlin received a few weeks ago.

After a quick introduction to the kernel sources (configuring, building, organization of the source tree), the book immediately dives into kernel internals:

  • Process management: how the kernel represents processes and their state, how processes are created inside the kernel, how threads are handled, are processes are terminated.
  • Process scheduling: a full chapter dedicated to the Linux kernel process scheduler. The new CFS scheduler is of course covered in great detail, with large portion of commented source code, for those who want to understand the fine details of the scheduler. Topics such as process sleep/wake-up, preemption, real-time scheduling policies are also covered.
  • System calls are then covered: how they are implemented, how parameters are passed from userspace to the kernel, etc. The call path from your user-space application down to the kernel is well explained in this chapter.
  • Kernel data structures: a generic chapter which details the kernel API for linked lists, queues, maps, and binary trees. Those APIs are omni-present inside the kernel, and it’s therefore a good idea to know how they work, both for understand existing code and for writing new code.
  • Interrupts: how interrupts are handled and how one can write an interrupt handler. Unfortunately, the newly introduced threaded interrupt handlers are not covered, but it’s true that their usage is not yet very widespread inside the mainline kernel.
  • Bottom halves and deferring work, a topic closely related to interrupt handling. It covers bottom halves, softirqs, tasklets and workqueues.
  • Kernel synchronization: two chapters are dedicated to this topic. First a chapter detailing why synchronization is needed, what are the sources of concurrency and what should be protected against concurrent access. And then a chapter detailing the mechanisms provided by the kernel to implement proper synchronization: atomic operations, spin locks, reader-writer spin locks, semaphores, reader-writer semaphores, mutexes, completion variables, sequential locks, preemption disabling, ordering and barriers
  • Timers and time management details how the kernel manages time: ticks, jiffies counter, timers, delaying execution of code are covered in this chapter. There are unfortunately no details about the clocksource and clockevents infrastructure, and no details about how timers and high-resolution timers are implemented. Contrary to other chapters that go fairly deep into the implementation details, this one mostly only covers the API to time management rather than the internals.
  • Memory management is the topic of the following chapter: physical memory management with the page allocator and the physical zones, then the kmalloc, vmalloc and SLAB allocators are covered. High-memory mappings, a topic specific to 32 bits architectures having more than a gigabyte of RAM is also covered in detail. The per-cpu interface is also covered, and will help those who want to understand parts of the kernel that have been optimized for scalability on multiple CPUs.
  • The Virtual Filesystem, with its different objects: superblock, inode, dentry and file is covered in good detail.
  • The block layer is then covered, with a description of the role of the bio structure, the request queues, and the I/O schedulers.
  • Then, the book goes back to more details about the internals of memory management: the mm_struct memory descriptor, the virtual memory areas (so called VMAs) and how they relate with the mmap/munmap system calls. The next chapter continues with a detailed description of the page cache implementation.
  • A fairly strange chapter called “Devices and modules” gives some information about kernel modules (how to build them, how to use module parameters, how dependencies are handled), then covers the internal of the device model (kobjects, ktypes, ksets, krefs) and finally sysfs. Just like the chapter covering the device model in the third edition of Linux Device Drivers, I think it totally misses the point. All the kobject, ktypes, ksets and krefs stuff is very low-level plumbing used by the Linux device model, but it is not exactly what the driver developer needs to interact with in the first place. In my opinion, a good description of the device model should rather explain what struct bus_type, struct driver and struct device are, how they are specialized for the different bus types in pci_driver, pci_device, usb_driver, usb_device, platform_driver, platform_device, and how the registration/probing of drivers and devices is done. I’ve recently given a talk about this topic, the video is in French, but the slides are in English.
  • Debugging is then covered: printk of course, but also oopses, debugging-related kernel options, the magic SysRq key, kernel debuggers, etc.
  • A rather generic chapter about Portability is then proposed, and finally a chapter about Patches, Hacking and the community that details the kernel community, the kernel coding style, how to generate and submit patches, etc.

All in all, Linux Kernel Development remains very good reading. I particularly appreciate the writing style of Robert Love, who manages to make a deeply-technical book interesting and easy to read. Of course, there are some topics in the kernel in which I had to dive myself and for which I’d expect to see more details in this book, but giving every possible detail about a huge beast such as the Linux kernel in just 400 pages is not possible!

Videos from FOSDEM 2010

Peter Korsgaard presenting Buildroot in the Cross build systems workshop at FOSDEM 2010

Like every year, the Free and Open Source Developer European Meeting took place early February in Brussels, and Thomas Petazzoni, from Bootlin, attended and recorded a few talks from the embedded session. However, contrary to previous years, I haven’t been able to record all talks from the embedded session, since I attended talks from other sessions which were already being recorded by others.

Gian-Carlo Pascutto presenting Embedded software development best practices at FOSDEM 2010

We also attended talks from the X.org and Coreboot developer roooms : videos for the X.org developer room can be found at http://video.fosdem.org/2010/devrooms/xorg/ and videos for the Coreboot developer room can be found at http://video.fosdem.org/2010/devrooms/coreboot/.

ELC 2010 videos

Videos from the Embedded Linux Conference in San Francisco, April 12-14, 2010.

The 2010 edition of the Embedded Linux Conference was once again a very interesting event. For embedded Linux developers, the Embedded Linux Conference is a perfect place to learn about new technologies, profit from the experience of other developers, and to meet key software developers.

For people who couldn’t attend this conference, and for single core people who didn’t manage to attend two or three talks at the same time, here are the videos that we managed to shoot. As usual, the videos are released with a Creative Commons Attribution – ShareAlike 3.0 license.

We hope it makes you feel like joining the next edition of the conference. If you can’t wait, what about going to ELC Europe in Cambridge (UK) in late October? It has a very interesting program too. Of course, the sessions will also be recorded. I hope to see you there!

Buildroot 2010.08 released!

Buildroot logoOn the last day of August, just in time, the 2010.08 version of Buildroot has been released. For the record, Buildroot is an easy-to-use embedded Linux build system: it can build your toolchain, your root filesystem with all its components (Busybox, libraries, applications, etc.), your kernel and your bootloaders, or any combination of these components.

Amongst the interesting changes in this version :

  • Complete rewrite of the bootloader build code. It contained a lot of legacy, unused and unclear stuff, it is now much easier to use and extend. We’ve removed support for Yaboot and added support for the new Barebox bootloader, and all the code to support AT91Bootstrap, AT91DataFlashBoot, U-Boot, Grub and Grub 2 has been rewritten.
  • Complete rewrite of the Linux kernel build code. It was also complicated to use, with an horribly complicated kernel version selection mechanism, the new code is much easier to configure and use.
  • The configuration file .config is now located in the out-of-tree directory when the O= option is used. So typically, for an out-of-tree build (which are very convenient when using the same Buildroot source tree for different projects/tests), you could do : mkdir ~/myoutput ; make O=~/myoutput menuconfig ; make O=~/myoutput
  • Support for building NPTL toolchains with uClibc, using the latest uClibc snapshots.
  • Support for the gconfig Gtk-based configurator, in addition to the already available menuconfig and xconfig
  • A particular effort has been put on fixing many of the bugs in our Bugzilla, improving robustness thanks to automated random builds, and converting even more packages to the generic and autotools infrastructure
  • Various things have also been deprecated: support for the CRIS, IA64, Sparc64 and Alpha architectures, support for Gtk over DirectFB (which is at the moment not supported upstream), Java support (no maintainer has volunteered to maintain this in Buildroot)
  • Many components have been bumped to newer versions
  • The shared configuration cache, which allowed to speed up the configuration of different packages, has been disabled by default, since it was causing a lot of problems with certain package configurations

I’ve again contributed to a significant portion of this release, being the author of the bootloader build code cleanup, the Linux kernel build code rewrite, leading an effort to reduce the number of outstanding bugs in our Bugzilla and many other little things. The contributors for this release are shown below :

   175  Peter Korsgaard
   168  Thomas Petazzoni
    38  Gustavo Zacarias
    18  cmchao
     8  Luca Ceresoli
     7  Paul Jones
     6  Lionel Landwerlin
     6  Malte Starostik
     5  Yann E. MORIN
     3  Julien Boibessot
     3  Khem Raj
     2  Dmytro Milinevskyy
     2  Francois Perrad
     2  Nick Leverton
     2  Peter Huewe
     2  Stanislav Bogatyrev
     1  Baruch Siach
     1  Bjørn Forsman
     1  Daniel Hobi
     1  Darcy Watkins
     1  Darius Augulis
     1  H Hartley Sweeten
     1  Karl Krach
     1  Kelvin Cheung
     1  Ossy
     1  Sagaert Johan
     1  Simon Pasch
     1  Slava Zanko
     1  Thiago A. Correa
     1  Will Wagner
     1  Yegor Yefremov

For the next release, there are already a few things in the pipeline :

  • Cleanup of all the board support code in Buildroot, in order to cleanly add support for more boards like BeagleBoard, Qemu boards, Calao boards, etc. We’ll use the new minimal defconfig mechanism used by the kernel. I’ve already started working on this
  • Cleanup of the package download process, to support Git and SVN download. The code has already been written by Maxime Petazzoni, reviewed on the list, so I expect it to be included fairly soon
  • Rewrite of libtool handling code, to remove some of our ugly libtool hacks. The code is currently being worked on by Lionel Landwerlin
  • Support for compiling toolchain using Crosstool-NG as a backend. The code is currently being finalized by Yann E. Morin, the author of Crosstool-NG
  • Further work on package uninstallation, clean partial rebuild. Some work has been started by Lionel Landwerlin, but it needs some discussion
  • Continue the conversion of packages to the generic and autotools infrastructures
  • I have also a ton of other things on my TODO-list : rework gdb/gdbserver support with external toolchains, rework the configuration of IPv6/RPC/locale/etc. with external toolchains, set up a Wiki-based Buildroot website with tutorials and better documentation, clean up the toolchain build process, reduce the number of “enhancement” bugs waiting in our Bugzilla, etc.

As Peter Korsgaard, Buildroot maintainer, said in the 2010.08 announcement: The next release is going to be 2010.11. Expect the first release candidate in late October and the final release at the end of November..

It is worth noting that we will be having a Buildroot Developer Day, on Friday 29th October, right after Embedded Linux Conference Europe. At least Peter Korsgaard, Lionel Landwerlin, Yann E. Morin and myself should be there.

Update your WordPress site from scripts

An example Python script, which can be re-used with other website engines.

WordPress logoThe Bootlin website is proudly powered by WordPress. It is a mix of static pages and blog posts, and we are very satisfied of the way we can manage and post content.

Being the webmaster, I had an issue though. Several of our pages share the same bits of content, in particular the descriptions of our public training sessions. To avoid discrepancies between pages, I ended up writing scripts to generate the contents of these pages from common parts. However, updating those pages on the website was done with manual copying and pasting, which was time consuming and error prone.

Fortunately, after some on-line research and practical experiments, I found a simple way of automating the process of login to our WordPress site, open a page for editing, and submitting a new version of the contents.

Since this wasn’t really straightforward, I’m happy to share the update-wordpress-page Python script I came up with, hoping that it be useful to you too.

This script can only be used to modify an existing page. It just changes the contents, and doesn’t touch other attributes. Of course, you could also extend it to create new pages and posts, but this would represent much more work, having lots of input fields to fill.

Another approach would have been to open a direct database connection to the server running WordPress, and then to perform your updates directly with SQL commands. However, this requires a knowledge about WordPress databases (making the script much less generic), and the open database port also makes your website less secure.

To use my script, you will first have to find the edit URL for your page, which reveals the WordPress post id.

I suggest you to create a special WordPress user with Editor privileges. The page history will then show which changes were automated, and which were manual.

The last thing you will have to do is create a $HOME/.update-wordpress-page configuration file as follows:

[bootlin.com/fr]
user=bot
password=ERrdrsdGp8

[bootlin.com]
user=bot
password=Hgdeedxx55

You can easily tune this script to support other web content engines. You first need to identify the login page (WordPress uses cookies to authenticate a session, instead of simple http authentication). Then, you will find the names of the input forms by reading the login page HTML code.

The second step is to open the page editing URL, and find out the name of the input form used for the page contents.

We don’t offer official support for our script, but I hope that this working code example will help you to make your own scripts, and to get you started faster. Python’s urllib2 and ClientForm really make this easy to do. What I especially like with ClientForm is its ability to modify the value of a given form, without having to read and fill any other input forms in the page, to keep their default content.

How to find the root device?

How to find which device corresponds to your root filesystem

I recently found something I was looking for for quite a long time. If you use the mount command in Linux, you can see that the root device is not listed like the other mounted filesystems:

/dev/root on / type ext3 (rw)
/dev/mmcblk0p1 on /mmcboot type vfat (rw)
proc on /proc type proc (rw)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /dev type tmpfs (rw,mode=0755)
...

For the / mount point, you are just told that it corresponds to /dev/root, which is not the real device you are looking for.

Of course, you can look at the kernel command line and see on which initial root filesystem Linux was instructed to boot (root parameter):

$ cat /proc/cmdline
mem=512M console=ttyS2,115200n8 root=/dev/mmcblk0p2 rw rootwait

However, this doesn’t mean that what you see is the current root device. Many Linux systems boot on intermediate root filesystems (like initramdisks and initramfs), which are just used to access the final one.

I explored the contents of /proc, but didn’t find any file revealing what the root device is.

Fortunately, I eventually found a command to find the root device:

$ rdev
/dev/mmcblk0p2 /

But how does this work? How could we find such information by ourselves? Use the Source, Luke!

When you ask yourself questions like this one, the best is to look at the BusyBox sources which implement this command. These sources are usually simpler than the ones for the same GNU command.

Here is what BusyBox rdev does… It first runs the stat system call on the / directory. Let’s run the stat command that corresponds to it:

$ stat /
  File: `/'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
Device: b302h/45826d	Inode: 2           Links: 23
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2010-07-21 22:00:01.000000000 +0200
Modify: 2010-06-13 15:04:37.000000000 +0200
Change: 2010-06-13 15:04:37.000000000 +0200

What’s interesting is the Device field. It means that the device corresponding to / has the major number b3 in hexadecimal (179 in decimal), and minor number 02. Bingo, this corresponds to /dev/mmcblk0p2:

$ ls -l /dev/mmcblk0p2 
brw-rw---- 1 root disk 179, 2 Jan  1  1970 /dev/mmcblk0p2

Therefore, what BusyBox rdev does is walk through /dev and its subdirectories to find a device file matching the major and minor numbers.

This is not a completely generic solution though. On some very simple embedded systems, you don’t even need to create device files for all existing devices. In particular, the device file for the root filesystem doesn’t have to exist. In such a case, rdev wouldn’t be able to find the root device.

A more generic solution could be to walk through /sys/block which enumerates all the block devices present on a system (even if not all of them have an entry in /dev/. This would allow to find the device with the matching major and minor numbers:

$ cat /sys/block/mmcblk0/mmcblk0p1/dev
179:1

Through this example, you can see how useful it can be to study the sources of system commands to understand how the system works. BusyBox sources, implementing simplified versions of GNU utilities, make this even easier.