Bootlin – Page 20 – Embedded Linux and kernel engineering

Videos and slides of Bootlin presentations at FOSDEM 2021

The videos from Bootlin’s presentations earlier this month at FOSDEM 2021 are now publicly available. Once again, FOSDEM was a busy event, even if it was online for once. As in most technical conferences, Bootlin engineers volunteered to share their experience and research by giving two talks.

Maxime Chevallier – Network Performance in the Linux Kernel, Getting the most out of the Hardware

Abstract: The networking stack is one of the most complex and optimized subsystems in the Linux kernel, and for a good reason. Between the wild range of applications, the complexity and variety of the networking hardware, getting good performance while keeping the stack easily usable from userspace has been a long-standing challenge.

Nowadays, complex Network Interface Controllers (NICs) can be found even on small embedded systems, bringing powerful features that were previously found only in the server world closest to day to day users.

This is a good opportunity to dive into the Linux Networking stack, to discover what is used to make networking as fast as possible, both by using all the features of the hardware and by implementing some clever software tricks.

In this talk, we cover these various techniques, ranging from simple batch processing with NAPI, queue management with RSS, RPS, XPS and so on, flow steering and filtering with ethool and TC, to finish with the newest big change that is XDP.

We dive into these various techniques and see how to configure them to squeeze the most out of your hardware, and discover that what was previously in the realm of datacenters and huge computers can now also be applied to embedded linux development.

Here are PDF slides for this presentation.

Michael Opdenacker – Embedded Linux from Scratch in 45 minutes, on RISC-V

Abstract: Discover how to build your own embedded Linux system completely from scratch. In this presentation and tutorial, we show how to build a custom toolchain (Buildroot), bootloader (opensbi / U-Boot) and kernel (Linux), that one can run on a system with the new RISV-V open Instruction Set Architecture emulated by QEMU. We also show how one can build a minimal root filesystem by oneself thanks to the BusyBox project. The presentation ends by showing how to control the system remotely through a tiny webserver. The approach is to provide only the files that are strictly necessary. That’s all the interest of embedded Linux: one can really control and understand everything that runs on the system, and see how simple the system can be. That’s much easier than trying to understand how a GNU/Linux system works from a distribution as complex as Debian!

The presentation also shares details about what’s specific to the RISC-V architecture, in particular about the various stages of the boot process. This presentation shares all the hardware (!), source code build instructions and demo binaries needed to reproduce everything at home, and add specific improvements. Most of the details are also useful to people using other hardware architectures (in particular arm and arm64).

It’s probably the first time a tutorial manages to show so many aspects of embedded Linux in less than an hour. See by yourself! At least, that’s for sure the first one demonstrating how to boot Linux from U-Boot in a RISC-V system emulated by QEMU.

Here are PDF slides for this presentation.

Bootlin acquired by Bootlin CTO Thomas Petazzoni and engineer Alexandre Belloni

We are happy to announce that Bootlin (formerly Free Electrons) has been acquired by two of its employees, Thomas Petazzoni and Alexandre Belloni.

Bootlin was founded in 2004 by Michael Opdenacker, with the goal of promoting the use of Linux and Free Software in embedded systems worldwide.

Thomas Petazzoni joined Bootlin in 2008, as the first employee. Thomas expanded the company offering by starting an engineering services activity, contributed to the growth of the company and took a CTO position. Thomas has a strong technical, open-source and embedded Linux background: he is the co-maintainer of the Buildroot project, has contributed to the Linux kernel, spoke at multiple international conferences and is the member of several embedded Linux conferences program committees. As Bootlin CTO, Thomas has been in charge of the complete engineering services activity: communication, sales, customer interaction, project management, scheduling and review.

Alexandre Belloni joined Bootlin in 2013, as an embedded Linux engineer. Alexandre has a deep open-source and technical background as well: he is the maintainer of multiple subsystems in the Linux kernel to which he has made significant contributions, and is an expert of the Yocto Project. He has been working closely with Thomas for many years in expanding and managing the engineering services activity.

As part of this acquisition, Thomas Petazzoni will become Bootlin’s CEO, while Alexandre Belloni will take the role of Bootlin’s COO. Michael Opdenacker will stay within Bootlin as an embedded Linux engineer and trainer.

“This acquisition is a logical continuation of my involvement in Bootlin and in the broader embedded Linux community” said Bootlin’s CEO Thomas Petazzoni, who added “I am proud to be leading the excellent engineering team at Bootlin, who will continue to offer the same training and engineering expertise to its customers worldwide”.

Alexandre Belloni, Bootlin’s COO, continued: “with two owners having both a strong technical background and a deep involvement in the open-source community, we intend to continue driving Bootlin with the same core values: technical excellence, open-source contribution and knowledge sharing”

Michael Opdenacker, Bootlin’s founder and former CEO, concluded: “Maria Llavata and I, after more than ten years of dedication to Bootlin’s customers and to the worldwide community of embedded Linux users and developers, are really happy to hand over the baton to Alexandre and Thomas, who have all the energy and enthusiasm needed to continue, expand and renew this beautiful adventure. I believe that Bootlin still has many new things to offer to the world, the fact that I am still on board is a proof of our trust in its new leadership.”

Device Tree 101 webinar slides and videos

As we announced back in January, we have offered in partnership with ST on February 9 a free webinar titled Device Tree 101, which gives a detailed introduction to the Device Tree, an important mechanism used in the embedded Linux ecosystem to describe hardware platforms. We were happy to see the interest around this topic and webinar.

Bootlin has always shared freely and openly all its technical contents, including our training materials, and this webinar is no exception. We are therefore sharing the slides and video recording of both sessions of this webinar:

Thanks to everyone who participated and thanks to ST for the support in organizing this event! Do not hesitate to share and/or like our video, and to suggest us other topics that would be useful to cover in future webinars!

New training course: embedded Linux boot time optimization

For many embedded products, the issue of how much time it takes from power-on to the application being fully usable by the end-user is an important challenge. Bootlin has been providing its expertise and experience in this area to its customers for many years through numerous boot time optimization projects, and we have shared this knowledge through a number of talks at several conferences over the past years.

We are now happy to announce that we have a new training course Embedded Linux boot time optimization, open for public registration. This training course was already given to selected Bootlin customers and is now available for everyone.

The training course will be lead by Michael Opdenacker, Bootlin’s founder, and author of several publications on the topic of boot time optimization. The course is organized over 4 sessions of 4 hours, with a significant fraction of time spent on practical demonstrations showing on a real-life example the techniques to measure and reduce the boot time of an embedded Linux system.

As usual with Bootlin, the training materials are fully available: Agenda, Slides and Practical lab instructions.

Our first course open for public registration will take place from April 6th to April 9th, 2021, from 14:00 to 18:00 UTC+2 (Paris time) on each day. The session cost is 519 EUR if you take advantage of the early bird price available until March 9th. Otherwise, the regular rate is 619 EUR. You can register now for this course on Eventbrite.

Also, if you’re interested in organizing a dedicated session for your company, do not hesitate to contact us.

Online training courses in March/April 2021

Our online training courses of January are now all completed, and were again successful. So we’re happy to announce the next dates for our public training courses in March and April, for all our courses: Embedded Linux system development, Linux kernel driver development, Yocto and OpenEmbedded system development, Buildroot system development and Linux graphics stack.

It is worth mentioning that we now have an Early Bird price, which is valid up to 1 month before the course, so register early if you’re interested!

Type	Dates	Time	Duration	Cost and registration
Embedded Linux (agenda)	Mar. 8, 9, 10, 11, 12, 15, 16, 2021 + extra session on Mar. 17 if needed	14:00 – 18:00 (Paris, UTC+1)	28 h	Early: 829 EUR^* Regular: 929 EUR^* Register
Linux kernel (agenda)	Mar. 9, 10, 11, 12, 16, 17, 18, 2021	13:30 – 17:30 (Paris time, UTC+1)	28 h	Early: 829 EUR^* Regular: 929 EUR^* register
Buildroot (agenda)	Mar. 8, 9, 10, 11, 2021	14:00 – 18:00 (Paris time, UTC+1)	16 h	Early: 519 EUR^* Regular: 619 EUR^* Register
Yocto Project (agenda)	Mar. 22, 23, 24, 25, 2021 + extra session on Mar 26 if needed	14:00 – 18:00 (Paris time, UTC+1)	16 h	Early: 519 EUR^* Regular: 619 EUR^* Register
Linux Graphics (agenda)	Apr. 6, 7, 8, 9, 2021	14:00 – 18:00 (Paris time, UTC+2)	16 h	Early: 519 EUR^* Regular: 619 EUR^* Register

Free “Device Tree 101” webinar, on February 9, 2021

In partnership with ST, we are organizing on February 9, 2021, a free webinar entitled “Device Tree 101”.

The Device Tree has been adopted for the ARM 32-bit Linux kernel support almost a decade ago, and since then, its usage has expanded to many other CPU architectures in Linux, as well as bootloaders such as U-Boot or Barebox. Even though Device Tree is no longer a new mechanism, developers coming into the embedded Linux world often struggle to understand what Device Trees are, what is their syntax, how they interact with the Linux kernel device drivers, what Device Tree bindings are, and more. This webinar will offer a deep dive into the Device Tree, to jump start new developers in using this description language that is now ubiquitous in the vast majority of embedded Linux projects. This webinar will be illustrated with numerous examples applicable to the STM32MP1 MPU platforms, which make extensive usage of the Device Tree.

This webinar will take place on February 9, 2021, and is proposed at two different times during the day: at 10 AM CET (UTC+1) and 5 PM CET (UTC+1). The duration of the webinar is 1 hours and 30 minutes. Registration is free at https://www.eventbrite.com/e/135964923747. The webinar itself will be hosted as a Youtube Live stream, which will allow participants to ask questions in the chat during the webinar.

The trainer for this webinar is Thomas Petazzoni, Bootlin’s CTO. Thomas is the author of the popular « Device Tree for Dummies » talk given in 2014 and which helped numerous embedded Linux developers get started with the Device Tree. Thomas has contributed over 900 patches to the official Linux kernel, mainly around ARM hardware platform support. He is also the co-maintainer of the Buildroot open-source project.

Bootlin at FOSDEM 2021: two talks, member of Embedded program committee

Like all conferences in these times, FOSDEM will take place as an online, virtual event. For all the FOSDEM regular attendees, it will certainly be a very different experience, and for sure, we will all miss the chocolate, waffles, beer, mussels as well as the rainy, muddy, snowy, foggy and cold weather that characterize Brussels in early February. But nevertheless, knowledge sharing and discussions must go on, and FOSDEM will take place! As usual, FOSDEM takes place the first week-end of February, on February 6-7, and the event is completely free, with no registration required.

This time around, Bootlin is once again contributing to FOSDEM:

Thomas Petazzoni is a member of the program committee for the Embedded, Mobile and Automotive Devroom. As such, he has worked with the other members of this program committee to select the talks, define the schedule, and will help organize the talks during FOSDEM itself.
Michael Opdenacker will give a talk on Embedded Linux “from scratch” in 45 minutes… on Risc-V, an updated version of his talk given at Live Embedded Event. This talk will take place on February 6, at 14:00 UTC+1.
Maxime Chevallier will give a talk on Networking Performances in the Linux Kernel, Getting the most out of the Hardware, sharing his experience working on embedded networking hardware. This talk will take place on February 7, at 13:00 UTC+1.

Make sure to check out the rest of the Embedded Devroom schedule, as well as the overall FOSDEM schedule.

Bootlin toolchains integration in Buildroot

Since 2017, Bootlin is freely providing ready-to-use pre-built cross-compilation toolchains at https://toolchains.bootlin.com/. We are now providing over 150 toolchains, for a wide range of CPU architectures, covering the glibc, uClibc-ng and musl C libraries, with up-to-date gcc, binutils, gdb and C library support.

We recently contributed an improvement to Buildroot that allows those toolchains to very easily be used in Buildroot configurations: the Bootlin toolchains are now all known by Buildroot as existing external toolchains, next to toolchains from other vendors such as ARM, Synopsys and others.

If you are building a Buildroot system for a CPU architecture variant that has a matching toolchain available from bootlin.toolchains.com, then Bootlin toolchains will naturally show up in the Toolchain sub-menu, when the selected Toolchain type is External toolchain. For example, if the selected CPU architecture is ARM little endian Cortex-A9, with VFP you will see:

Once Bootlin toolchains is selected, a new sub-option Bootlin toolchain variant appears, which allows to choose between the different toolchains applicable to the selected CPU architecture:

This hopefully should make Bootlin toolchains easier to use for Buildroot users.

Internally, this support for Bootlin toolchains in Buildroot is generated and updated using the support/scripts/gen-bootlin-toolchains script. In addition to making the toolchains available to the user, it allows generates some Buildroot test cases for each toolchain, so that each of those configuration is tested by Buildroot continuous integration, see support/testing/tests/toolchain/test_external_bootlin.py.

Linux 5.10, Bootlin contributions

Linux 5.10 was released a few weeks ago, and while 5.11-rc2 is already out, it’s still time to look at what Bootlin contributed to the 5.10 kernel. As usual, for a broad overview of the major changes in 5.10, we recommend reading the LWN articles: 5.10 merge window part 1, the rest of the 5.10 merge window, or the 5.10 KernelNewbies page.

Overall, Bootlin contributed 78 patches to this kernel release, in the following areas:

Alexandre Belloni did a number of improvements in the support of Microchip ARM platforms: device tree updates, code cleanups, etc.
Alexandre Belloni added a new rv3032 RTC driver and did some improvements to the r9701 RTC driver.
Miquèl Raynal implemented a significant rework of how ECC engines are handled in the MTD subsystems, so that ECC engines can be used not just for parallel NANDs but also for SPI NANDs. See also the talk that Miquèl gave at the Embedded Linux Conference Europe on this topic: slides and video.
Miquèl Raynal contributed a few improvements to the tlv320aic32x4 audio codec driver.
Paul Kocialkowski made some small improvements, one in the OV5640 camera sensor driver, and one in the Rockchip DRM driver.
Thomas Petazzoni implemented a performance improvement in the max310x driver, used for SPI-connected UART controllers.

In addition to these code contributions, we also contribute by having several of our engineers be maintainers of a few subsystems of the Linux kernel. As part of this:

Miquèl Raynal reviewed and merged 47 patches touching the MTD subsystem he co-maintains with other kernel developers.
Alexandre Belloni reviewed and merged 42 patches touching either the Microchip ARM or MIPS platforms, or the RTC subsystem.
Grégory Clement reviewed and merged 2 patches touching the Marvell ARM/ARM64 platform support.

Here is the complete list of our commits to 5.10.

Bootlin welcomes Thomas Perrot in its team

Since December 1st, 2020, we’re happy to have in our team an additional engineer, Thomas Perrot, who joined our office in Toulouse, France.

Thomas brings 6+ years of experience working on embedded Linux systems, during which he worked at Intel on Android platforms, and then at Sigfox on the base stations for Sigfox’s radio network. Thomas has experience working with Linux on x86-64, ARM and ARM64 platforms, with a wide range of skills: bootloader development, Linux kernel and driver development, Yocto integration, OTA updates. Thomas was also deeply involved in the strong security aspects of Sigfox base stations, with secure boot and measured boot, TPM, integrity measurement, etc. At Sigfox, Thomas was involved in all steps of the product life-cycle, from the design phase all the way to the in-field deployment, update and maintenance. Last but not least, Thomas is a Linux technologist and a free software enthusiast, who hacks some open source hardware projects on his free time. See also Thomas page on our website and his LinkedIn profile.

Thomas Perrot is joining our growing team of engineers in Toulouse, which already included Paul Kocialkowski, Miquèl Raynal, Köry Maincent, Maxime Chevallier and Thomas Petazzoni.