In our effort to continuously improve and update our training materials, we have recently updated our Buildroot training course to Buildroot 2016.05, which was released at the end of May. In addition to adapting our practical labs to use this new Buildroot version, we have also improved the training materials to cover some of the new features that have been added over the last year in Buildroot. The most important changes are:
Cover the graph-size functionality, which allows to generate a pie chart of the filesystem size, per package. This is a very nice feature to analyze the size of your root filesystem and see how to reduce it.
Improve the description about the local site method and the override source directory functionalities, that are very useful when doing active application/library development in Buildroot, or to package custom application/library code.
Add explanations about using genimage to create complete SD card images that are ready to be flashed.
Add explanations about the hash file that can be added to packages to verify the integrity of the source code that is downloaded before it gets built.
For one of our customers building a product based on i.MX6 with a fairly low-volume, we had to design a mechanism to perform the factory flashing of each product. The goal is to be able to take a freshly produced device from the state of a brick to a state where it has a working embedded Linux system flashed on it. This specific product is using an eMMC as its main storage, and our solution only needs a USB connection with the platform, which makes it a lot simpler than solutions based on network (TFTP, NFS, etc.).
In order to achieve this goal, we have combined the imx-usb-loader tool with the fastboot support in U-Boot and some scripting. Thanks to this combination of a tool, running a single script is sufficient to perform the factory flashing, or even restore an already flashed device back to a known state.
The overall flow of our solution, executed by a shell script, is:
imx-usb-loader pushes over USB a U-Boot bootloader into the i.MX6 RAM, and runs it;
This U-Boot automatically enters fastboot mode;
Using the fastboot protocol and its support in U-Boot, we send and flash each part of the system: partition table, bootloader, bootloader environment and root filesystem (which contains the kernel image).
The SECO uQ7 i.MX6 platform used for our project.
imx-usb-loader
imx-usb-loader is a tool written by Boundary Devices that leverages the Serial Download Procotol (SDP) available in Freescale i.MX5/i.MX6 processors. Implemented in the ROM code of the Freescale SoCs, this protocol allows to send some code over USB or UART to a Freescale processor, even on a platform that has nothing flashed (no bootloader, no operating system). It is therefore a very handy tool to recover i.MX6 platforms, or as an initial step for factory flashing: you can send a U-Boot image over USB and have it run on your platform.
This tool already existed, we only created a package for it in the Buildroot build system, since Buildroot is used for this particular project.
Fastboot
Fastboot is a protocol originally created for Android, which is used primarily to modify the flash filesystem via a USB connection from a host computer. Most Android systems run a bootloader that implements the fastboot protocol, and therefore can be reflashed from a host computer running the corresponding fastboot tool. It sounded like a good candidate for the second step of our factory flashing process, to actually flash the different parts of our system.
Setting up fastboot on the device side
The well known U-Boot bootloader has limited support for this protocol:
The fastboot documentation in U-Boot can be found in the source code, in the doc/README.android-fastboot file. A description of the available fastboot options in U-Boot can be found in this documentation as well as examples. This gives us the device side of the protocol.
In order to make fastboot work in U-Boot, we modified the board configuration file to add the following configuration options:
Other options have to be selected, depending on the platform to fullfil the fastboot dependencies, such as USB Gadget support, GPT partition support, partitions UUID support or the USB download gadget. They aren’t explicitly defined anywhere, but have to be enabled for the build to succeed.
U-Boot enters the fastboot mode on demand: it has to be explicitly started from the U-Boot command line:
U-Boot> fastboot
From now on, U-Boot waits over USB for the host computer to send fastboot commands.
Using fastboot on the host computer side
Fastboot needs a user-space program on the host computer side to talk to the board. This tool can be found in the Android SDK and is often available through packages in many Linux distributions. However, to make things easier and like we did for imx-usb-loader, we sent a patch to add the Android tools such as fastboot and adb to the Buildroot build system. As of this writing, our patch is still waiting to be applied by the Buildroot maintainers.
Thanks to this, we can use the fastboot tool to list the available fastboot devices connected:
# fastboot devices
Flashing eMMC partitions
For its flashing feature, fastboot identifies the different parts of the system by names. U-Boot maps those names to the name of GPT partitions, so your eMMC normally requires to be partitioned using a GPT partition table and not an old MBR partition table. For example, provided your eMMC has a GPT partition called rootfs, you can do:
# fastboot flash rootfs rootfs.ext4
To reflash the contents of the rootfs partition with the rootfs.ext4 image.
However, while using GPT partitioning is fine in most cases, i.MX6 has a constraint that the bootloader needs to be at a specific location on the eMMC that conflicts with the location of the GPT partition table.
To work around this problem, we patched U-Boot to allow the fastboot flash command to use an absolute offset in the eMMC instead of a partition name. Instead of displaying an error if a partition does not exists, fastboot tries to use the name as an absolute offset. This allowed us to use MBR partitions and to flash at defined offset our images, including U-Boot. For example, to flash U-Boot, we use:
The fastboot command must be explicitly called from the U-Boot prompt in order to enter fastboot mode. This is an issue for our use case, because the flashing process can’t be fully automated and required a human interaction. Using imx-usb-loader, we want to send a U-Boot image that automatically enters fastmode mode.
To achieve this, we modified the U-Boot configuration, to start the fastboot command at boot time:
Of course, this configuration is only used for the U-Boot sent using imx-usb-loader. The final U-Boot flashed on the device will not have the same configuration. To distinguish the two images, we named the U-Boot image dedicated to fastboot uboot_DO_NOT_TOUCH.
Putting it all together
We wrote a shell script to automatically launch the modified U-Boot image on the board, and then flash the different images on the eMMC (U-Boot and the root filesystem). We also added an option to flash an MBR partition table as well as flashing a zeroed file to wipe the U-Boot environment. In our project, Buildroot is being used, so our tool makes some assumptions about the location of the tools and image files.
Our script can be found here: flash.sh. To flash the entire system:
# ./flash.sh -a
To flash only certain parts, like the bootloader:
# ./flash.sh -b
By default, our script expects the Buildroot output directory to be in buildroot/output, but this can be overridden using the BUILDROOT environment variable.
Conclusion
By assembling existing tools and mechanisms, we have been able to quickly create a factory flashing process for i.MX6 platforms that is really simple and efficient. It is worth mentioning that we have re-used the same idea for the factory flashing process of the C.H.I.P computer. On the C.H.I.P, instead of using imx-usb-loader, we have used FEL based booting: the C.H.I.P indeed uses an Allwinner ARM processor, providing a different recovery mechanism than the one available on i.MX6.
The FOSDEM conference will take place on January 30-31 in Brussels, Belgium. Like every year, there are lots of interesting talks for embedded developers, starting from the Embedded, Mobile and Automotive Devroom, but also the Hardware track, the Graphics track. Some talks of the IoT and Security devrooms may also be interesting to embedded developers.
Thomas Petazzoni, embedded Linux engineer and CTO at Bootlin, will be present during the FOSDEM conference. Thomas will also participate to the Buildroot Developers Meeting that will take place on February 1-2 in Brussels, hosted by Google.
The Buildroot project has recently released a new version, 2015.05. With exactly 1800 patches, it’s the largest release cycle ever, with patches from more than 100 different contributors. It’s an impressive number, showing the growing popularity of Buildroot as an embedded Linux build system.
The CHANGES file summarizes the most important improvements of this release.
Amongst those 1800 patches, 143 patches were contributed by Bootlin. Our most significant contributions for this release have been:
Addition of a package for the wf111 WiFi drivers. They allow to use a WiFi chip from Bluegiga, which is being used in one of our customer projects.
Addition of the support for using uClibc-ng. uClibc-ng is a “collaborative” fork of the uClibc project, which aims at doing more regular releases and have better testing. Maintained by Waldemar Brodkorb, the project has already seen several releases since its initial 1.0 release. Waldemar is merging patches from the original uClibc regularly, and adding more fixes. It allows Buildroot and other uClibc users to have well-identified uClibc stable versions instead of a 3 years old 0.9.33.2 version with dozens of patches on top of it. uClibc-ng is not currently used as the default uClibc version as of 2015.05, but it might very well be the case in 2015.08.
Important internal changes to the core infrastructure. Until this release, the make legal-info, make source, make external-deps and make source-check logic was relying only on the Buildroot configuration file. This was giving correct results for target packages which all have a corresponding Buildroot configuration option, but not for host packages (which for most of them don’t have Buildroot configuration options). Only a manual two-level dependency handling was done for the host packages for the above mentioned commands. With our work, the handling of those features has been moved to be part of the package infrastructure itself, so it’s using proper make recursivity to resolve the entire dependency tree. Due to this, the results of make legal-info or make external-deps may be longer following this release, but it’s because it’s now actually correct and complete. You can look at the patches for more details, but these changes are very deep into the core Buildroot infrastructure.
Large number of build fixes. We contributed 52 patches fixing issues detected by the autobuild infrastructure.
Addition of the imx-usb-loader package, which can be used to load over USB a new bootloader on i.MX6 platforms, even if the platform has no bootloader or a broken bootloader. We also use it as part of one of our customer projects.
With 142 patches, Bootlin engineer Thomas Petazzoni is the third contributor to this release by number of patches:
But by far, our most important contribution by far for this release is Thomas acting as the interim maintainer: on the total of 1800 patches merged for this release, Thomas has been the committer of 1446 patches. He has therefore been very active in merging the patches contributed by the Buildroot community.
There are already some very interesting goals set for the Buildroot 2015.08 release, as you can see on the Buildroot release goals page.
Also, if you want to learn Buildroot in details, do not hesitate to look at our Buildroot training course!
Last year, Bootlin launched a new training course on using the Yocto Project and OpenEmbedded to develop embedded Linux systems. In the selection of build system tools available in the embedded Linux ecosystem, another very popular choice is Buildroot, and we are happy to announce today that we are releasing a new 3 days training course on Buildroot!
Bootlin is a major contributor to the Buildroot upstream project, with more than 2800 patches merged as of May 2015 (including the ones contributed with our previous name). Our engineer Thomas Petazzoni alone has contributed more than 2700 patches. He has gathered an extensive knowledge of Buildroot and its internals, being one of the primary authors of the core infrastructures of Buildroot. He is a major participant to the Buildroot community, organizing the regular Buildroot Developer Days, supporting users through the mailing list and on IRC. Last but not least, Thomas acts as an interim maintainer when the main Buildroot maintainer is not available, an indication of Thomas strong involvement in the Buildroot project.
In addition, Bootlin has used and is using Buildroot in a significant number of customer projects, giving us an excellent view of Buildroot usage for real projects. This feedback has been driving some of our Buildroot contributions over the last years.
The 3 days training we have developed covers all the aspects of Buildroot: basic usage and configuration, understanding the source and build trees, creating new packages including advanced aspects, analyzing the build, tips for organizing your Buildroot work, using Buildroot for application development and more. See the detailed agenda.
We can deliver this training course anywhere in the world, at your location (see our rates and related details). We have also scheduled a first public session in English in Toulouse, France, on November 30 to December 2. Contact us at training@bootlin.com if you are interested.
And finally, last but not least, like we do for all our training sessions, we are making the training materials freely available under a Creative Commons BY-SA license, at the time of the training announcement: the first session of this course is being given this week. For the Buildroot training, the available materials are:
Our materials have already been reviewed by some of the most prominent contributors to Buildroot: Peter Korsgaard (Buildroot maintainer), Yann E. Morin, Thomas De Schampheleire, Gustavo Zacarias and Arnout Vandecappelle. We would like to take this opportunity to thank them for their useful comments and suggestions in the development of this new training course.
For many open-source developers based in Europe, the FOSDEM is probably the most useful, interesting and exciting conference. Once again this year, several Bootlin engineers will attend the conference:
Maxime Ripard, mainly involved in Allwinner related kernel development, as well as more recently OpenWRT support for Marvell platforms
Antoine Ténart, involved in Marvell Berlin related kernel development, and one of the developers of our Yocto Project and OpenEmbedded training course
Alexandre Belloni, involved in Atmel processors related kernel development, and also one of our Yocto expert.
Thomas Petazzoni, involved in Marvell EBU processors related kernel development, and doing a lot of Buildroot contributions.
If you are attending, and want to know more about Bootlin, or discuss career or project opportunities, do not hesitate to contact us prior to the conference. Many of us will probably attend a significant number of talks from the Embedded track, so it should be easy to find us.
Last but not least, Alexandre Belloni will be giving a talk about Starting with the Yocto Project, which will take place on Sunday, at 3 PM in room Lameere.
Finally, Thomas Petazzoni has organized and will participate to the Buildroot Developers Meeting organized right after FOSDEM, and sponsored by Google and Mind.
The Bootlin team wishes you a Happy New Year for 2015, with plenty of optimism and energy!
Bootlin is happy to take this opportunity to share some news about the latest training and contribution activities of the company.
Kernel contributions
We continue to work significantly on support for various ARM processors in the Linux kernel. Our contributions to the latest kernel releases:
147 patches from Bootlin merged in Linux 3.17, making Bootlin the 14th contributing company for this release by number of patches. See our blog post about this release.
155 patches from Bootlin merged in Linux 3.18, making Bootlin the 14th contributing company. See our blog post for more details.
For the upcoming 3.19 release, we already have 196 patches merged.
One of the highlights was that we added support for the Atmel SAMA5D4 SoC to the Linux kernel even before the new chip was announced by Atmel! That’s a very positive sign for customers when an SoC is supported in the mainline Linux kernel sources right at product launch, instead of having to wait for months or years before the community developers can catch up.
Note that we also added Atmel SAMA5D3 SoC support to Xenomai, a hard real-time extension for the Linux kernel. Thanks to this, the Atmel SAMA5D3 Xplained board can now run with the 2.6.x release of Xenomai.
Besides those highlights, most of our kernel contributions were as usual centered around support for specific families of ARM processors: CPUs from Marvell EBU and Marvell Berlin, from Atmel and from Allwinner. We added a new network driver for some Marvell EBU processors, added SMP support for Marvell Berlin processors, added a DMA controller driver for Allwinner processors, and did a lot of maintenance work to support these processors in the mainline kernel.
Buildroot contributions
Our involvement into the Buildroot project, a popular embedded Linux build system, is going on. Our engineer Thomas Petazzoni contributed 136 patches to the 2014.11 release, making him the second contributor by number of patches. Thomas is also taking care of the maintenance of the project on a more regular basis, reviewing and merging patches from contributors.
OpenWRT contributions
We have recently started contributing to the OpenWRT project: improve the kernel support to use defconfig, introduce a notion of board to support different NAND configurations for each platform. We will soon to be pushing support for the Marvell Armada 385 platform, and improved support for the Marvell Armada 370 and XP platforms.
Recent projects
Besides our publicly visible kernel contributions, we do also work on customer-specific projects. Among the latest projects we have done:
Develop a complete Board Support Package for a custom TI AM335x based platform: U-Boot porting, Linux kernel porting, and development of a Yocto-generated system. Qt5 and OpenGL are used for the graphical application, a fairly complex audio setup had to be supported, and many traditional interfaces as well (USB Host and Device, CAN, display, etc.)
Develop a Board Support Package for a custom Marvell Armada 375 based platform for a telephony system. Not only did we port a Linux kernel on this platform, but we also wrote several DAHDI drivers to interface the telephony hardware of the platform with Asterisk.
NAND and UBI stress-testing for a customer-specific Freescale i.MX28 based platform. We improved the NAND controller driver, added a new MTD tool to generate bitflips, and did some long term power-cut stress-testing of the UBIFS setup to ensure the reliability of the platform. See our kernel driver improvements and the new nandflipbits tool.
Adapt an existing ADC driver for a customer-specific platform to the modern Industrial Input Output (IIO) subsystem of the kernel.
Conferences: FOSDEM, Embedded World and Embedded Linux Conference
Several Bootlin engineers will participate to the FOSDEM conference, taking place on January 30 and February 1 in Brussels. In addition, Thomas Petazzoni will be participating to the Buildroot Developers Meeting that takes place right after FOSDEM in the Google offices in Brussels.
Bootlin will participate to the Embedded World trade show on February 24-26 in Nuremberg, Germany. We will be present at Atmel’s booth and visiting exhibitor booths too. For people in Europe, this will be a good opportunity to ask your questions about our embedded Linux training and engineering services. In particular, you will be able meet our engineers Alexandre Belloni, Thomas Petazzoni (CTO), Michael Opdenacker (CEO) and Anja Roubin as well, the new person in charge of our training services.
This year again, most of the Bootlin engineering team (7 engineers) will participate to the 2015 edition of the Embedded Linux Conference on March 23-25 in San Jose, California. We submitted several talk proposals, but our presence won’t depend on the number of talks that are eventually accepted. Participating to this conference, and to its European edition in the fall too, is very important for us to make sure we do not miss any of the interesting developments in the technical community, and above all to strengthen our ties with the community developers. This helps us to be good technical trainers with valuable experience and information to share. The strong relationships with other community developers (and in particular with project maintainers) also help us when our customers contract us to add hardware support or features to official versions of community projects such as the Linux kernel.
Free technical documentation resources
Since the latest edition of this newsletter, we started running our new Yocto Project and OpenEmbedded course, and we released all training materials for this course. As usual, such materials are meant to be used by people learning by themselves too. All you have to do is get your hands on a Beaglebone Black board, read the slides and try to do the labs!
The news is that we will run new public sessions in Paris, in addition to the ones we usually organize in Toulouse, Avignon and Lyon in France. We are starting with our embedded Linux and our Yocto courses, but other topics will follow too.
Android system development training, June 8-11, Toulouse, France (in English)
See sessions and dates for more details. Of course, we can also deliver our training courses at your location, anywhere in the world. Feel free to contact us for a quote.
If you are interested in more frequent news about Bootlin, you can follow us on Twitter, Google+ and LinkedIn.
Bootlin is happy to share some news about the latest training and contribution activities of the company.
Bootlin continued to contribute significantly to the Linux kernel, and specifically improvements to support for several ARM processors. We have continued to work significantly on the Marvell Armada ARM processors, we have increased our contributions to support for Allwinner ARM processors, and have started contributing to improving or developing support for Atmel AT91 and Marvell Berlin processors.
Bootlin engineers merged 121 patches in the 3.13 release of the Linux kernel, making Bootlin the 17th contributing company. In the recently released 3.14, we have also merged 121 patches, making Bootlin the 21st contributing company.
Thanks to these contributions, we have gained significant expertise in supporting ARM platforms in the Linux kernel, allowing us to efficiently help our customers for kernel porting and device driver development activities.
We expect these contributions to significantly increase in the future, and we have already more than 210 patches lined-up for the upcoming 3.15 kernel, making Bootlin the 14th contributing company for 3.15.
Another project that Bootlin has contributed to in a significant way is the Buildroot build system. We have pushed 133 patches in the 2014.02 release and more than 130 patches in the upcoming 2014.05 release, making Bootlin in the top five of the most active contributors.
Some of our contributions to this release are quite important: introduction of an infrastructure to better support Python packages, and introduction of a mechanism to more easily allow companies using Buildroot to keep their own package recipes separated from the Buildroot core, addition of the support for the musl C library.
We are launching a new training course about the Yocto Project. Yocto is one of the leading embedded Linux build system, used by more and more hardware vendors and companies around the world to automate and simplify the build process of embedded systems.
In order to help companies interested in using Yocto for their products, our new 3-day training course mixing lectures and practical labs, gives all the necessary details to build one’s own system, customize its image, add more packages, and generally understand how to use Yocto.
This training course can be delivered on-site upon request, and we will also be offering public sessions starting this fall.
As usual, our training materials will be made freely available under a Creative Commons license.
Bootlin has recently welcomed two new engineers in its engineering team: Antoine Ténart and Boris Brezillon. Antoine and Boris bring some additional experience in Linux kernel development, Android porting and development, and generally embedded Linux knowledge to Bootlin.
As demand for our Linux kernel mainlining services is growing in a strong way, we plan to announce new job openings for Q4 2014.
If you are interested in joining Bootlin, don’t hesitate to contact us early. This way, we will already know you and your interest in working with us when the job openings are confirmed.
Last month, Bootlin participated to the Embedded Linux Conference: the entire team of 7 engineers was present in San Jose, for the most important conference of the embedded Linux community.
In addition to attending the conference, Bootlin engineers gave a total of seven talks during this conference:
One of the most important conference of the Embedded Linux community will take place at the end of this month in California: the Embedded Linux Conference will be held in San Jose from April, 29th to May, 1st, co-located with the Android Builders Summit. The schedule for both of these events has been published, and it is full of interesting talks on a wide range of embedded topics.
As usual, Bootlin will participate to this conference, but this participation will be the most important ever:
No less than seven engineers from Bootlin will participate to the event, which is almost our entire engineering team: Alexandre Belloni, Maxime Ripard, Boris Brezillon, Antoine Ténart, Grégory Clement, Michael Opdenacker and Thomas Petazzoni. Only Ezequiel Garcia will be missing.
Seven talks or BOFs from Bootlin engineers are part of the conference schedule:
If you are interested in embedded Linux, we highly advise you to attend this conference. And if you are interested in business or recruiting opportunities with Bootlin, it will also be the perfect time to meet us!
Thomas Petazzoni also participated to the two days Buildroot Developers Meeting after the FOSDEM conference. This meeting gathered 10 contributors to the Buildroot project physically present and two additional remote participants. The event was sponsored by Google and Mind, thanks a lot to them! During those two days, the participants were able to discuss a very large number of topics that are often difficult to discuss over mailing lists or IRC, and a significant work to clean up the oldest pending patches was done. In addition to this, these meetings are also very important to allow the contributors to know each other, as it makes future online discussions and collaborations much easier and fruitful. For more details, see the complete report of the event.
Buildroot Developers Meeting in Brussels
Also, if you’re interested in Buildroot, the project has applied to participate to the next edition of the Google Summer of Code. Two project ideas are already listed on the project wiki, feel free to contact Thomas Petazzoni if you are a student interested in these topics, or if you have other proposals to make for Buildroot.