2021 at Bootlin, a year in review

2021 has come to an end, a year that everyone will most likely consider as somewhat complicated and unusual, even though the current situation seems to now becoming the new normal. The switch to a new year is generally a good moment to take a step back, and review what happened in the past year, and draw some directions for the coming year.

In this blog post, we’d like to do exactly this for Bootlin, which has seen a number of significant changes this year, as well as a continuation of its usual activities.

We also take this opportunity to wish you all an happy year 2022, and send you our best wishes. May 2022 be full of interesting projects and also be a safe year for everyone.

Here are the main topics that we cover in this lengthy blog post:

Acquisition and recruiting

One major change for Bootlin in 2021 is that the company was acquired by Thomas Petazzoni, former CTO, and Alexandre Belloni. So it’s an internal acquisition, by former employees, meaning that Bootlin has kept the same offering and spirit. Further, Bootlin’s original founder Michael Opdenacker is still in the team, but now as an employee. See our blog post regarding the acquisition, back in February.

In 2021, we also recruited several engineers, bringing significant additional expertise to our team:

  • Thomas Perrot, who joined just before the start of 2021, bringing 10 years of experience in embedded Linux BSP development, with a strong Yocto expertise.
  • Hervé Codina, who joined in March 2021, bringing 20+ years of experience in bare-metal and Linux embedded development.
  • Clément Léger who joined in June 2021, bringing 10 years of experience in Linux kernel development, including the expertise on porting the Linux kernel to a brand new CPU architecture.

We continue to have open positions for embedded Linux engineers, and we plan to hire 3 to 4 engineers in 2022, hiring for the first time engineers located outside of France.

Engineering projects

As is the case every year, our engineering team has been kept busy this year mostly by our engineering projects, all focused on our core expertise of low-level embedded Linux development.

We have selected below a few highlights of our work in 2021, in various areas.

Build systems

Apart from delivering numerous BSPs based on Yocto or Buildroot to our customers, we have also directly contributed to both the Yocto and Buildroot open-source projects.

On the Yocto Project side:

  • Bootlin engineer Michael Opdenacker has become one of the co-maintainers of the official Yocto Project documentation, making numerous improvements and contributions to this documentation. See our blog post on this topic. We will continue this involvement in the Yocto Project documentation in 2022.
  • Bootlin engineer and COO Alexandre Belloni has been active in the build and release engineering effort of the Yocto Project, as a member of the Yocto Project SWAT team. Alexandre has been working directly with the main Yocto Project architect, Richard Purdie, on the review and validation of new contributions. Other Bootlin engineers have also helped in investigating and resolving specific bugs. We will also continue this involvement in the Yocto Project build engineering in 2022.

On the Buildroot side:

  • We have continued our work towards the implementation of top-level parallel build support in Buildroot. Bootlin engineer Hervé Codina has posted several iterations of a patch series bringing a mechanism preventing file overwrites between packages, a requirement for proper top-level parallel build.
  • We implemented and published, in partnership with ST, a Buildroot BR2_EXTERNAL called buildroot-external-st which contains example configurations for the STM32MP1 platforms from ST.
  • We contributed support for hybrid ISO9660 images, supporting in a single image PC platforms based on legacy BIOS, 32-bit UEFI BIOS and 64-bit UEFI BIOS, using the grub2 bootloader.
  • We helped one of our customers reduce their out-of-tree Buildroot patches by upstreaming a number of new Buildroot packages.

Outside of our the Yocto Project and Buildroot, we also ventured into the world of Debian/Ubuntu for embedded systems, by using the ELBE tool, to which we contributed support for building Ubuntu-based images. See our blog post that describes how to use ELBE to automatically build Ubuntu-based images for the Raspberry Pi.

Audio support in Linux

Audio support in the Linux kernel is another area where Bootlin engineers have specific expertise with. Here are some examples of audio related projects we worked on in 2021:

  • Integration of audio support for an i.MX6 platform with a complex dual AD1978 audio codec configuration, with a TDM8 audio interface between the SoC and the audio codecs.
  • Development of a brand new and complete ALSA driver for a new PCIe sound card, based on a FPGA.
  • Integration of audio support for an i.MX8 platform used in the automotive space with a complex audio configuration that involves multiple A2B transceivers through the Analog Devices AD2428 A2B master, Bluetooth audio, and more.

OP-TEE

In 2021, we started working on a major project: adding support for the Microchip SAMA5D2 processor to the OP-TEE project. This project is already well underway, as we have a functional port of OP-TEE, which is now being upstreamed.

As part of this, Bootlin engineer Clément Léger has implemented and contributed a generic clock framework for OP-TEE, which has already been accepted upstream. See our blog post for more details.

Clément also gave a talk at the Embedded Linux Conference 2021 on the topic of OP-TEE, titled OP-TEE: When Linux Loses Control. Slides and video are available.

We expect to continue the upstreaming of the SAMA5D2 support in 2022, and also develop support for additional SAM5D2 hardware capabilities in OP-TEE.

U-Boot extension board manager

In collaboration with the BeagleBoard.org community, we developed and contributed to U-Boot a generic extension board manager. This mechanism allows hardware platforms such as the BeagleBone to automatically detect extension boards that are connected, and apply the Device Tree overlays that provide the hardware description for those extension boards.

This work was covered in detail in the talk given by Bootlin engineer Köry Maincent at the Live Embedded Event in June 2021, see the slides and video.

Modems

In 2021, we had the opportunity to work on several projects that involved 3G/4G/5G modems, strengthening our knowledge of the modem stack in Linux, especially around modem-manager, libmbim and libqmi.

For example, we worked on the support of a Quectel BG95 modem interfaced with a RaspberryPi 4, or the support of a Sierra Wireless EM9190 modem interfaced over PCIe to an NXP i.MX6. The latter was particularly challenging and is still on-going, as the upstream Linux support for PCIe 5G modems is still very recent. We contributed a few fixes in this area.

Secure boot

Secure boot remains an important topic for a growing number of projects, and in 2021, we helped customers with secure boot on several i.MX6 platforms, and on one i.MX8 platform. The Yocto integration of secure boot was also a key aspects in those projects, to get a proper process for signing and verifying all states of the boot process. We also have a few U-Boot contributions in our contribution pipeline related to improving secure boot support. See for example the talk from Thomas Perrot at Live Embedded Event, relating the secure boot setup on i.MX8.

Camera, ISP and video support

We finished 2020 with a lot of on-going contributions to the Linux Video4Linux subsystem, which we had summarized in a blog post early 2021. In particular, our drivers for the OV5568 and OV8865 camera sensors were merged upstream.

In 2021, Bootlin engineer and multimedia expert Paul Kocialkowski continued to work on several multimedia topics. Some of the key projects included:

  • Optimizing the H264 decode → rescale → H264 encode pipeline of one of our customers, on an Allwinner H3 platform. We were able to significantly optimize the pipeline by leveraging the H264 decoder built into the Allwinner processor, for which Paul had written a Linux kernel driver several years ago, and by developing a custom ffmpeg plugin that offloaded the rescaling to the GPU, using the open-source lima support. A fully open-source solution!
  • Developing a proper Linux kernel driver for the Allwinner ISP, with support for debayering and noise filtering. This was a significant challenge as the Allwinner ISP was so far only supported through closed-source binary blobs. We have already submitted a first iteration to the upstream Linux kernel community. See our blog post for more details, as well as the talk given by Paul at the Embedded Linux Conference: slides and video.
  • Improving the Allwinner Linux kernel camera driver to support single buffer capture. This is useful in scenarios where camera sensors are not used to capture a sequence of frames, but just single frames, and the available memory is limited. This is particularly true on platforms such as the Allwinner V3s, which may be limited to just 64MB of RAM. This is going to be submitted upstream soon, as part of our on-going work on the Allwinner camera driver.

NAND and flash support

With Bootlin engineer Miquèl Raynal being a maintainer of the NAND subsystem in Linux and a co-maintainer of the MTD subsystem, it should be no surprise that we have continued to deal with a number of flash memory related projects in 2021:

  • We have brought in mainline the support for several NAND flash controllers:
    • For the Arasan NAND controller, used in some Xilinx processors. See our blog post on this topic.
    • For the ARM Primecell PL35x NAND controller, used in some other Xilinx processors. See our blog post on this topic.
    • For the NAND controller found at least in the the Renesas R-Car Gen3 and RZ/N1 processors.
  • Contributed support for the NV-DDR interface, which is used by some NAND flash chips and controllers to increase the throughput. See our blog post.
  • Continued our work on generalizing ECC support in the MTD subsystem, so that all ECC setups (software ECC, on-die ECC, ECC in the NAND controller, or external ECC engine) can be supported regardless of the NAND interface (serial or parallel). See the talk from Miquèl Raynal, ECC engines given at last year’s ELCE.
  • Improved the TI GPMC NAND controller driver to properly support NAND chip with large pages (larger than 4KB).

ADC/IIO support

We’ve helped a few customers with Linux kernel support for ADC devices, namely:

  • Extend the support for the MAX1027 family of ADCs, with support for external triggers
  • Bring support for the ADC found in TI AM437x processors

All of these were contributed to the upstream Linux kernel, Miquèl Raynal also wrote a extensive blog post on various aspects of the IIO subsystem and has taken the opportunity of these projects to also improve/clarify various aspects of the IIO core in a recent patch series.

Networking

Networking support in the Linux kernel is also one area where Bootlin is very active. Here are some key projects we worked on in 2021, some of them being on-going with additional work expected in 2022:

  • We implemented support for PTP offloading for the Qualcomm AR803X PHY.
  • We started working on QUSGMII support in Linux, a new standard that allows PTP time-stamps to be included directly on the preamble of QSGMII frames.
  • We started working on the 802.15.4 stack in the Linux kernel, with our initial step being support for passive scanning.
  • We implemented mqprio support in the mvneta Linux kernel driver, used for some Marvell platforms. mqprio is a queuing discipline that allows mapping traffic flows to hardware queue ranges using priorities and a configurable priority to traffic class mapping.
  • We implemented Frame DMA support in the Ocelot Ethernet switch driver, significantly improving the performance of frame injection/extraction by the CPU into/from the switch traffic.
  • We developed proper SFP support in Linux and U-Boot, with dynamic reconfiguration (in Linux only), for a customer using the Zynq 7000 platform and a complex network setup.

BSP development

Many of our projects are obviously related to the development of update of complete Linux BSP for our customers (bootloader, Linux kernel, custom embedded Linux distribution). Here are a few examples:

  • Development of a brand new U-Boot, Linux and Yocto based BSP to migrate an existing product running Windows CE on a ST Spear320 processor to Linux. This sort of development proved to be challenging as the Spear320 support in both U-Boot and Linux is close to be abandoned (and in fact has since then been removed from U-Boot upstream). We hope to be able to contribute to improve the upstream Spear320 support in 2022.
  • Migration of an AM335x/Buildroot based BSP to newer Linux kernel and Buildroot releases, for a customer in the healthcare industry
  • On-going migration of an i.MX6 BSP from an old kernel release and a custom build-system to recent versions of U-Boot and Linux as well as a proper standardized Yocto Project based Linux distribution. This project presents some interesting challenges as it uses only one of the two Cortex-A9 cores to run Linux, the other one runs a bare-metal application, and we will also migrate this to a proper usage of the remoteproc and rpmsg mechanisms.
  • We worked with several customers on STM32MP1 platforms, helping with porting on new platforms, extending the Device Tree and device drivers, developing custom Yocto-based or Buildroot-based distributions. STM32MP1 is definitely becoming a popular platform for a number of new projects.
  • We also worked with several customers on custom platforms based on the RasberryPi Compute Module, doing Device Tree configuration/tweaking and Yocto integration.

Over-the-Air update integration

We continued to help our customers with the integration of Over-The-Air update solutions in their embedded Linux systems. A few examples:

  • We integrated RAUC for an i.MX6 platform, using the Barebox bootloader and Yocto Project based distribution. Bootlin engineer Kamel Bouhara wrote a detailed blog post on this topic, as well as the talk from Kamel at Live Embedded Event
  • We integrated the Azure Device Update for IoT Hub, using the U-Boot bootloader on a RaspberryPi CM4 platform, also with a Yocto Project based distribution. Internally, the Device Update for IoT Hub is based on swupdate
  • As part of a migration of an existing embedded product based on Debian to a Yocto-based distribution, we are integrating Mender.

Crédit Impôt Recherche

For our French customers, another important milestone achieved by Bootlin in 2021 is the delivery of our Crédit Impôt Recherche agreement, which allows our French customers to benefit from tax incentives on research and development activities done by Bootlin for their projects. See our blog post for more details.

Training

With the COVID19 still making travel conditions difficult and uncertain, most of our training activity in 2021 was dedicated to on-line training courses. Indeed, 69 out of our 72 training sessions were delivered online this year. In total, we delivered our training courses to 906 engineers in 2021.

In 2021, we published two new training courses:

Just like all our other training courses, the complete training materials for those new courses are freely available, distributed under the CC-BY-SA license.

Another major event in 2021 was our work to get the French Qualiopi certification, which proves the quality of our training organization and processes. As part of this, we have improved several aspects of our training courses, mainly regarding feedback collection and handling as well as the evaluation of the participants.

In 2022, we expect to at least:

  • Improve how our online course are delivered, by ensuring more participants can do the practical labs by themselves, which is an important part of the learning process
  • Publish at least one more training course. We already have plans for a course on a topic that we think will be very relevant to many embedded Linux engineers. Stay tuned for updates on this!
  • Have additional capacity to deliver our training courses, which are seeing significant demand.

For more details about our training offering, see our training page, which details our dedicated on-line and on-site sessions as well as our public on-line sessions.

Contributions

In Bootlin continued and strong open-source focus, we once again contributed to different open-source projects in 2021:

  • Linux kernel. The best summaries are our blog posts about our contributions to Linux 5.10, Linux 5.11, Linux 5.12, Linux 5.13, as well as Linux 5.14 and 5.15. Some of our significant contributions: huge effort on ECC engine support in the MTD subsystem, rv3032 RTC driver, support for new MIPS platforms from Microchip, a Simple Audio Mux driver, major work on timer/TCB support on Microchip ARM platforms, new I3C master controller driver for the Silvaco I3C IP, new drivers for the OV5568 and OV8865 camera sensors, NAND controller drivers for the Arasan, PL35x and Renesas IPs, enabling of ADC support on TI AM437x.
  • U-Boot. Our biggest contribution has been the extension board manager, which is described earlier in this blog post. We also contributed various small fixes and improvements.
  • OP-TEE. We contributed a generic clock framework, and the support for the SAMA5D2 clock tree and TRNG driver. So far 55 commits from Bootlin have been integrated in OP-TEE.
  • Buildroot. We contributed a total of 257 patches. The main contributions are in the area of tooling to list security vulnerabilities in Buildroot packages, support for Bootlin external toolchains, improved SELinux support, support for the Beagle-V RISC-V platform, a good number of new packages, support for hybrid ISO9660 images. In addition, Thomas Petazzoni continued his work as a co-maintainer of the project: Thomas has reviewed and merged close to 2000 patches from Buildroot contributors throughout 2021.
  • Yocto Project. As outlined above in this blog, we did major contributions to the Yocto Project, with Michael Opdenacker becoming co-maintainer of the Yocto Project documentation, and Alexandre Belloni being involved in the build and release engineering effort.
  • Linux Test Project. We fixed a number of issues in LTP tests that prevented from using LTP in embedded-oriented systems, such as the ones that can be generated using the Yocto Project. See our blog post for more details.
  • Our training materials, which are all freely available, have seen no less than 872 commits. These includes updates to our existing training courses, but also our new Real-Time Linux with PREEMPT_RT training course.
  • Our freely available toolchains have also been improved and updated. See our blog post.
  • We continued to maintain our very popular Elixir code indexing tool.

Conferences

Here are the talks that we presented this year, at various virtual events:

  • FOSDEM 2021
    • Networking Performance in the Linux Kernel, Getting the most out of the Hardware, by Maxime Chevallier. Slides and video.
    • Embedded Linux from scratch in 45 minutes, on RISC-V, by Michael Opdenacker. slides and video.
  • Live Embedded Event
    • Security vulnerability tracking tools in Buildroot, by Thomas Petazzoni. Slides and video.
    • Secure boot in embedded Linux systems, by Thomas Perrot. Slides and video.
    • Understanding U-Boot Falcon Mode, by Michael Opdenacker. Slides and video.
    • Device Tree overlays and U-Boot extension board management, by Köry Maincent. Slides and video.
    • Getting started with RAUC, by Kamel Bouhara. Slides and video.
  • Embedded Linux Conference
    • I3C in tomorrow’s design, by Miquèl Raynal. Slides and video.
    • Embedded Linux nuggets found in Buildroot package Eldorado, by Michael Opdenacker. Slides and video.
    • OP-TEE: When Linux Loses Control, by Clément Léger. Slides and video.
    • Advanced Camera Support on Allwinner SoCs with Mainline Linux, by Paul Kocialkowski. Slides and video.
  • A webinar organized in partnership with ST, Device Tree 101, by Thomas Petazzoni. Slides and video.
  • A webinar organized in partnership with Microchip, Improving Linux Boot Time in Your Embedded Application, with the participation of Thomas Petazzoni. Link.

Bootlin toolchains updated, 2021.11 release

Bootlin has been offering since 2017 a large set of ready to use pre-compiled cross-compilation toolchains at toolchains.bootlin.com. These toolchains are available for a wide range of CPU architectures and CPU variants, and support either the glibc, uClibc-ng or musl C libraries, where applicable.

It’s been quite some time since the last release of those toolchains, so we took the opportunity of this quiet period between Christmas and New Year to finally update the toolchains. We’re happy to announce that we have now published a total of 187 toolchains targeting 46 different CPU architecture variants. As the toolchain release name suggests, they are now built with Buildroot 2021.11.

The most important changes are the following ones:

  • The so-called stable toolchains are now based on gcc 10.3.0, binutils 2.36.1, Linux headers 4.9, gdb 10.2, glibc 2.34, uClibc 1.0.39 and musl 1.2.2
  • The so-called bleeding-edge toolchains are now based on gcc 11.2, binutils 2.37, Linux headers 5.4, gdb 11.1, glibc 2.34, uClibc 1.0.39 and musl 1.2.2
  • The riscv64 toolchains (targeting the LP64 ABI) have been replaced by riscv64-lp64d toolchains (targeting the LP64D ABI), making them more generally useful.
  • Runtime testing using Qemu has been added for RISC-V 64-bit, m68k-68xxx and OpenRISC.
  • The gdb cross-debugger is now compiled with both Python and TUI support.
  • Internally, there’s been some significant refactoring of the scripts and Gitlab CI pipelines that control the build and testing of those toolchains. Many thanks to Romain Naour for providing the ground work that enabled this refactoring.

The toolchains can be directly downloaded from toolchains.bootlin.com. We have also submitted a patch to Buildroot that updates those toolchains, which are directly usable in Buildroot as external toolchains.

Bootlin is now a Qualiopi certified training provider

Certificat Qualiopi BootlinBootlin has been delivering training courses in the field of Embedded Linux since its creation in 2004, delivering over 430 courses to more than 4500 engineers just since 2009, in over 40 countries, with a high-level of quality and a full transparency, with fully open training materials and publicly available training evaluations.

In France, a new quality certification brand for training and skills development service providers, called Qualiopi, has been created and is going to be needed starting January 1, 2022 for French training providers who want to deliver services to French customers who use public funding for training courses. For more details about Qualiopi, see this page in English, or the official page from the French Ministry, in French.

Bootlin has gone through the Qualiopi certification process, making a number of improvements to how we deliver and manage our training courses along the way, and we are happy to announce that Bootlin, at the first attempt, was successfully certified as a Qualiopi compliant training service provider. See our certificate, which can also be verified on the AFNOR certification authority website.

We are proud of this result, which shows our commitment to delivering high quality training courses to all our customers. Do not hesitate to look at our training courses portfolio and contact us if you are interested.

Announcing buildroot-external-st, Buildroot support for STM32MP1 platforms

STM32MP1 Discovery Kit 2Back in 2019, ST released a brand new processor family, the STM32MP1, whose members are currently based on a dual Cortex-A7 to run Linux combined with one Cortex-M4 to run bare-metal applications, together with a wide range of peripherals.

Following the release of this new platform, Bootlin ported its Embedded Linux and Yocto training courses to be available on STM32MP1, and also published a long series of tutorials showing how to use Buildroot to build an embedded Linux system on STM32MP1: part 1, part 2, part 3, part 4, part 5, part 6 and part 7.

We are happy to announce that we have partnered with ST to develop an improved support of Buildroot on STM32MP1, which is materialized by a Buildroot BR2_EXTERNAL available on Github at https://github.com/bootlin/buildroot-external-st. In Buildroot, a BR2_EXTERNAL is an extension of the core Buildroot, with additional configurations and/or packages.

This BR2_EXTERNAL tree is an extension of Buildroot 2021.02, which provides four example Buildroot configuration to easily get started on STM32MP1 platforms:

  • st_stm32mp157a_dk1, building a basic Linux system for the STM32MP1 Discovery Kit 1 platform, with a minimal Linux root filesystem
  • st_stm32mp157c_dk2, building a basic Linux system for the STM32MP1 Discovery Kit 2 platform, with a minimal Linux root filesystem
  • st_stm32mp157a_dk1_demo, building a much more featureful Linux system for the STM32MP1 Discovery Kit 1 platform, with Linux root filesystem that allows to run Qt5 applications with OpenGL acceleration on the HDMI output, that supports audio, demonstrates the usage of the Cortex-M4, uses OP-TEE, and more.
  • st_stm32mp157c_dk2_demo, building a much more featureful Linux system for the STM32MP1 Discovery Kit 2 platform, with Linux root filesystem that allows to run Qt5 applications with OpenGL acceleration on both the integrated DSI display and HDMI output, that supports audio, WiFi and Bluetooth, demonstrates the usage of the Cortex-M4, uses OP-TEE, and more.

This BR2_EXTERNAL is designed to work with Buildroot 2021.02, with only a small set of modifications, which since then have been integrated in upstream Buildroot, ensuring that STM32MP1 users can directly use upstream Buildroot for their projects.

There is extensive documentation on how to use this BR2_EXTERNAL tree as well as how to test the various features of the STM32MP1 platform: using STM32CubeProgrammer, using Device Tree generated from STM32 CubeMX, using the Cortex-M4, testing display support, using Qt5, using WiFi, using Bluetooth, using audio and using OP-TEE. We have also documented the internals of the BR2_EXTERNAL components.

This Buildroot support is using the latest software components from the recently released 3.1 BSP from ST (see release notes), so it is based on Linux 5.10, U-Boot 2020.10, TF-A 2.4 and OP-TEE 3.12. We will keep this BR2_EXTERNAL updated with newer releases of the ST BSP.

If you face any issue while using this Buildroot support for STM32MP1, you can use the issue tracker of the Github project, or use the ST community forums. Bootlin can also provide commercial support on Linux on STM32MP1 platforms, as well as training courses.

Bootlin eligible to French “Crédit Impôt Recherche” tax incentive

Bootlin CIR agreementA number of years ago, the French tax system has created a tax incentive mechanism called Crédit Impôt Recherche (Research Tax Credit) that allows startups and innovative companies to get tax deductions corresponding to a fraction of their research and development costs. This allows French companies to more easily invest in research and development activities.

In 2021, Bootlin has initiated the process to be eligible to this tax incentive mechanism, and we are happy to announce that after studying Bootlin’s expertise, engineering experience and achievements, the French tax administration has confirmed that Bootlin can deliver research and development activities fulfilling the Crédit Impôt Recherche criteria to its customers. This means that Bootlin customers in France can now integrate the cost of Bootlin engineering services that correspond to research and development activities into their Crédit Impôt Recherche and receive a tax incentive corresponding to up to 30% of the cost of our engineering services.

For our customers outside of France, this tax incentive is obviously not available, but the certification of Bootlin by the French tax administration as a company able to deliver research and development activities is another testimonial of our strong technical expertise in our field of Embedded Linux and Linux kernel development.

Bootlin contributions to Linux 5.14 and 5.15

It’s been a while we haven’t posted about Bootlin contributions to the Linux kernel, and in fact missed both the Linux 5.14 and Linux 5.15 releases, which we will cover in this blog post.

Linux 5.14 was released on August 29, 2021. The usual KernelNewbies.org page and the LWN articles on the merge window (part 1 and part 2) provide the best summaries of the new features and hardware support offered by this release.

Linux 5.15 on the other hand was released on November 1, 2021. Here as well, we have a great KernelNewbies.org article and LWN articles on the merge window (part 1 and part 2).

In total for those two releases, Bootlin contributed 79 commits, in various areas:

  • Alexandre Belloni, as the RTC subsystem maintainer, contributed 9 patches improving various aspects of RTC drivers, the RTC subsystem or Device Tree bindings for RTC
  • Clément Léger contributed a small improvement to the at_xdmac driver used on Microchip ARM platforms
  • Hervé Codina enabled Ethernet support on the old ST Spear320 SoC, by leveraging the existing stmmac Ethernet controller driver
  • Maxime Chevallier fixed a small issue with the Ethernet PHY on the i.MX6 Solidrun system-on-module
  • Miquèl Raynal added support for NV-DDR timings in the MTD susbsystem. This allows to improve performance with NAND flash memories that support those timings. Their usage is specifically implemented in the Arasan NAND controller driver, which Miquèl contributed back in Linux 5.8. See our previous blog post on this topic for more details
  • Miquèl Raynal added support for yet another NAND controller driver, the ARM PL35x, which is used for example on Xilinx Zynq 7000. See our previous blog post on this topic.
  • Miquèl Raynal added support for NAND chips with large pages (larger than 4 KB) to the OMAP GPMC driver.
  • Miquèl Raynal made a few fixes to the IIO driver for the max1027 ADC.
  • Paul Kocialkowski contributed a few patches to enable usage of the Hantro video decoder driver on the Rockchip PX30 processor.
  • Thomas Perrot contributed one patch to enable usage of the Flex Timers on i.MX7, and one to fix an issue in the PL022 SPI controller driver.

And now, as usual the complete list of our contributions to Linux 5.14 and 5.15:

The backbone of a Linux Industrial I/O driver

As part of recent projects, we had to dig into the Linux kernel Industrial I/O (IIO) subsystem with the goals of supporting a new ADC and adding new features to an existing driver. These tasks involved quite a few discussions between our engineering team and the IIO maintainers and reviewers. The aim of this blog post is to summarize the substance of these explanations to help others understand how an IIO kernel driver works and interacts with the core IIO subsystem.

Disclaimer: The IIO core is huge and keeps evolving. The aim of this article is not to cover it entirely, but at least explain our knowledge of how to use its basic features for common situations.

What is IIO?

The Industrial I/O subsystem covers any type of device that is commonly called as a “sensor”: ADCs, IMUs, temperature sensors, accelerometers, pressure sensors, potentiometers, light sensors, proximity sensors, etc (as well as few actuators, which I will on purpose disregard in this blog post). All these devices, besides measuring truly different physical components of our three dimensional world, end-up sharing quite a few properties. Any of these sensors must first be configured in order to know what must be measured and possibly how. When adequate, the device must be triggered in order to start converting. When the requested samples are ready, there must be some kind of signaling involved in order for the user to retrieve and process the data.

When thinking about the generic interfaces which could be needed by all these devices, it is quite straightforward to list:

  • The configuration before sampling
  • The triggering mechanism
  • The signaling for an end of conversion situation
  • The reading of the samples
  • The advertisement of the data

Registering an IIO device

The IIO core manipulates struct iio_dev * objects which inherits from struct device. This object should be allocated by the device driver with devm_iio_device_alloc(), providing the size of the driver’s internal structure as second argument. The allocated area dedicated for this internal pointer can be then retrieved with iio_priv().

This iio_dev structure must then be filled with a number of information:

  • The name of the device
  • A set of struct iio_info operations, typically a hook to read one or multiple samples on demand, optionally be able to write to the device, etc.
  • A set of supported modes, such as INDIO_DIRECT_MODE, which is used when samples can be retrieved at any time by the user from sysfs.
  • A scan mask, namely available_scan_masks which defines what are the possible/impossible scan combinations when requesting a read. Typically, a device might be configured to scan all of its internal channels from 1 to N. This can be described with a list of GENMASK(X, 0), with X ranging from 0 to the maximum number of channels. When the user will request a given set of channels, the IIO core will go through all the available masks registered by the driver and pick the first one that contains the desired channels. The selected mask will be available to the driver through the active_scan_mask entry of the iio_dev structure. If ‘anything goes’ and the devices has no restriction regarding which channel(s) can be scanned, this field should be skipped.
  • A definition of all the possible channels, including the type of physical measurements the device is able to perform (IIO_VOLTAGE, IIO_CURRENT, IIO_TEMPERATURE, IIO_STEPS, IIO_ROT, etc), the channel index and the data format.

Here is an example of channel description and below the meaning of these fields.

struct iio_chan_spec chan1 = {
	.type = IIO_VOLTAGE,
	.indexed = 0,
	.channel = index,
	.info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
	.info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE),
	.scan_index = 1,
	.scan_type = {
		.sign = 'u',
		.realbits = 10,
		.storagebits = 16,
		.shift = 2,
		.endianness = IIO_BE,
	},
}
  • .info_mask_separate indicates an entry in sysfs that will be present for this the channel. In this case, IIO_CHAN_INFO_RAW indicates it is going to be the raw value of the sample.
  • .info_mask_shared_by_type indicates an entry in sysfs that will be shared by all channels of the same type. IIO_CHAN_INFO_SCALE means that there will be a common voltage scale sysfs entry shared by all the voltage raw entries. If the device was also able to read a temperature, we would also have had a single file indicating the scale for all the temperature samples.
  • The .scan_type field in the example indicates that values are provided as 16-bit big-endian samples that must be shifted by two bits. The full scale range is 0-1023. This conversion only applies to the buffer reading path: raw values directly read from sysfs and returned by the ->read_raw() hook (see below) should be converted by the driver itself.

Once the device fully described (and initialized, of course), the driver must register it with devm_iio_device_register().

Scaling factors

The int (*read_raw)(struct iio_dev *indio_dev, struct iio_chan_spec const *chan, int *val, int *val2, long mask) callback will be executed when reading either of the ‘raw value’ or ‘scale’ files from sysfs.

The type of data that must be returned is provided in the mask parameter: IIO_CHAN_INFO_RAW to retrieve the raw measurement or IIO_CHAN_INFO_SCALE to retrieve the scaling parameters, based on the scale information available in the iio_chan_spec structure that describes the channel.

For the IIO_CHAN_INFO_RAW case, most drivers return an IIO_VAL_INT type which can be simply “returned” into the *val argument. It is however possible to return a fixed point number, in this case the logic explained right after applies.

For the IIO_CHAN_INFO_SCALE case, the return value indicates what type of scaling should be done. In most cases here a fixed point value will be used so *val and *val2 will carry the scaling parameters. Here are two examples:

  1. IIO_TEMP example: 
    *val = 1;
*val2 = 8;
ret = IIO_VAL_FRACTIONAL;

    The full scale sample value should be multiplied by 1/8 in order to get Celcius degrees.

  2. IIO_VOLTAGE example:

    *val = 2500;
*val2 = 10;
ret = IIO_VAL_FRACTIONAL_LOG2;

    The full scale range is a 10-bit value mapped to a 0-2500mV input level, said otherwise the scaling factor should be 2500 / 1024. The core will automatically do the computation of this factor and return 2,44140625 to the user in order to get milli-Volts.

Sampling

There are two basic common cases here.

In simple situations, a “single” on-demand read was issued by user-space directly by reading /sys/bus/devices/iio:device/in_<type><index>_raw. In this case the ->read_raw() callback should handle basically all the steps necessary to get a measurement, as detailed in our introduction.

However, user-space can also pick a more advanced way of interacting with the measurement device, called triggers.

A trigger is a specific configuration of the device which will sample a number of channels upon a specific event. This event might be the user requesting it from userspace with a so called software trigger, it might also be an external hardware event, or a periodic signal, or an internal continuous read mode… There are many ways of triggering a sensor and they are all covered by the subsystem.

Many devices cannot handle both modes at the same time. The only situation where this might work smoothly is when a device provides a hardware FIFO where you can read from (or a ‘latest value’ register) while not disrupting the FIFO read back. Otherwise, it will be needed, in order to avoid collisions between these two modes, to verify that exclusive access to the device is granted with a call to iio_device_claim_direct_mode() when starting a direct mode operation. As this helper grabs a mutex, it should be only called from process context and always be balanced with a call to iio_device_release_direct_mode().

IIO interoperability model

In the IIO core these four concepts are used:

  • IIO device: the hardware part which produces samples
  • IIO trigger: the signaling capability to request a conversion start
  • IIO buffers: where to store the samples
  • IIO events: threshold detectors

Even though a single hardware device might have hardware support for all these features, they must be described and handled separately so that, when applicable, other IIO devices might use them as well, eg. IIO device 2 could start a conversion upon IIO device 1 trigger state change. In practice it is not always possible but the way the API is built should lead us to keep things well separated anyway.

Registering a trigger

A struct iio_trigger must be allocated by devm_iio_trigger_alloc(), giving the new trigger a name.

The trigger should then receive a set of operations (struct iio_trigger_ops) with at least ->set_trigger_state() implemented, in order to switch on and off the trigger. One can use iio_trigger_set_drvdata() in order to link private data with the trigger and get this pointer back from the trigger callbacks.

Once initialized, devm_iio_trigger_register() will register the IIO trigger. This trigger will appear as a dedicated IIO device in sysfs.

It is likely that an IRQ will need to be registered as part of the trigger initialization step: the driver must be notified somehow that the trigger was toggled. If the asynchronous signaling is tied to a “trigger change” condition, which is the easiest situation, then it is advised to provide iio_trigger_generic_data_rdy_poll() as hard IRQ handler. This helper will just call iio_trigger_poll() and return.

You may of course want to handle more than this but in any case the rule is clear, triggers, buffers and devices should be fully separated. Hence, do not directly handle any data from this handler: an IIO trigger is only supposed to indicate a hardware transition, no more.

The call to iio_trigger_poll() will effectively go through the IIO internal interrupt tree, find the device that is connected to the trigger which fired and call the relevant handler in order to request the waiting device to process the data (which may be identical or different than the triggering device).

In the case of the device being limited to, for instance, an End Of Conversion (EOC) interrupt, you should still consider this signal as being suitable for being registered as a trigger. Yes, this might imply an additional delay between the hardware toggling and the IRQ being fired which is not ideal, but from a software point of view, the split between driver code and core logic will let other IIO devices use this IRQ as a trigger with no additional change needed to your code.

Note: There is one exception here. When a device does not provide any visible per-scan interrupt and the software has only access to some kind of FIFO watermark events, the whole trigger + buffer representation is swapped with a pure buffer-only implementation.

Registering triggered buffers

If the device itself is able to provide fast samples, the driver should also register a buffer, with iio_triggered_buffer_setup(). Both a hard IRQ handler and a threaded IRQ handler can be registered, as well as additional callbacks called before and after enabling and disabling the buffers in order to eg. configure the requested channels based on the current ->active_scan_mask.

Upon a trigger condition, these are the handlers that might be chosen by the core if the trigger is connected to your device!

The hard IRQ handler might be used to eg. save timestamps. The threaded IRQ handler is dedicated to the data processing. Depending of the type of trigger (iio_trigger_using_own()) the driver must decide whether it should start a conversion manually or if the data is waiting somewhere in a hardware FIFO, ready to be retrieved.

The final step is to push the samples into the core’s buffers. This should not be done manually. Let’s say that the user requested channels 0, 1 and 3 while the selected scan mask was including all channels from 0 to 4. Just calling iio_push_to_buffers() is the solution: the core knows that it will receive five samples of 16 bits, it also knows that the user only requested three of them and will automatically pick the right ones.

With all these IIO objects registered, you should be able to properly interact with the core and the other drivers, providing trigger capabilities to third party devices, or benefiting from other’s triggers.

What if my design lacks trigger capabilities?

You can still use triggers by enabling IIO_CONFIGFS (enables the configuration interface) and IIO_SW_TRIGGER. Then, you can either choose to trigger your scans from userspace with a simple file write, thanks to CONFIG_IIO_SYSFS_TRIGGER, or leverage timers to get periodic scans with CONFIG_IIO_HRTIMER_TRIGGER.

As an example, here is how to create a sysfs trigger:

# echo 0 > /sys/bus/iio/devices/iio_sysfs_trigger/add_trigger
# cat /sys/bus/iio/devices/iio_sysfs_trigger/trigger0/name
sysfstrig0

And here is how to create a timer based software trigger:

# mkdir -p /config
# mount -t configfs none /config
# mkdir /config/iio/triggers/hrtimer/my_5ms_trigger
# cat /sys/bus/iio/devices/trigger0/name
my_5ms_trigger
# echo 200 > /sys/bus/iio/devices/trigger0/sampling_frequency

How to use triggers and buffers from userspace

Just for the reference, linking an IIO trigger to an IIO device is as simple as:

# cat /sys/bus/iio/devices/trigger0/name > /sys/bus/iio/devices/iio:device0/trigger/current_trigger

The next step is to configure the channels that should be scanned:

# echo 1 > /sys/bus/iio/devices/iio:device0/scan_elements/in_voltage0_en
# echo 1 > /sys/bus/iio/devices/iio:device0/scan_elements/in_voltage1_en
# echo 1 > /sys/bus/iio/devices/iio:device0/scan_elements/in_voltage3_en

Starting the sampling process is managed with:

# echo 1 > /sys/bus/iio/devices/iio:device0/buffer/enable

In the case of a sysfs software trigger, it is the user’s responsibility to timely run:

# echo 1 > /sys/bus/iio/devices/trigger0/trigger_now

The samples are available to be read in /dev/iio\:device0.

The decoding process before the scaling operation must be performed by the userspace, following the content of:

# cat /sys/bus/iio/devices/iio:device0/scan_elements/in_voltage0_type
be:u12/16>>2

Which in this case would mean that each sample is 16 bits wide, values should be considered big-endian, shifted twice before being considered as an unsigned 12-bit value.

# od -t x1 /dev/iio\:device0
0000000 08 06

Should be interpreted as 0x806 >> 2 = 0x201, which should be then multipled by the scaling factor in order to get the final mV value.

Conclusion

While contributing to this (relatively new) subsystem, we discovered a number of interesting features and design choices which would really benefit from a much tougher in-kernel documentation as most of the available information explains how to use IIO (with libiio or configfs) more than how to write a decent and properly shaped IIO device driver. As the subsystem is still pretty recent, it is valid to look at existing drivers to make design choices, but that is not a magic solution as no device never fully matches any software API anyway, and sensors unfortunately do not escape from that sticky rule.

We want to warmly thank Jonathan Cameron, IIO founder and maintainer, for his precious feedback on the mailing list, as well as his valuable review and contribution to this blog post.

We hope this article will help you go through this API and if it does, please mind letting us know by dropping a comment in the section down below!

OP-TEE gains a clock framework contributed by Bootlin

Introduction

OP-TEE logoOP-TEE is a popular open-source reference implementation of a Trusted Execution Environment that relis on the Arm Trustzone technology. While working on the OP-TEE port for an ARM 32-bit system-on-chip, the Microchip SAMA5D2, we needed to add support for the complete clock tree of this SoC. OP-TEE did not have any generic clock support at all and we felt the need to add such a framework. Thanks to this framework, support the 10+ clocks of the Microchip SAMA5D2 was easily imported from Linux with less work than a complete rewrite of the clock tree. Using generic subsystems allows to lower the maintenance cost and easily add new clocks.

In this blog post, we will describe in more details this clock framework, and the contributions we are doing to the OP-TEE project.

Clock framework

The clock framework that we contributed to OP-TEE allows to register clocks and represent a full clock tree with parents. Device Tree support has been added to allows parsing the clocks and their relationships from Device Tree. It provides a consumer API that allows device drivers to query clocks from their Device Tree node, enable or disable them, and get or set the needed clock rates.

assigned-clock-parents and assigned-clock-rates Device Tree properties are also supported and will apply the clock parents and rates described in these properties. A fixed-clock driver matching the "fixed-clock" compatible string has also been added since this one is often present in SoC Device Trees.

Peripheral drivers in OP-TEE can now use the functions provided by the clock framework to get clocks from the Device Tree using clk_dt_get_by_name() and then enable/disable them at will with clk_enable() and clk_disable() . Rates can also be set and retrieved using clk_set_rate() and clk_get_rate().

The pull request was made on OP-TEE github and contained the following commits, which have now been merged in the official upstream OP-TEE project:

Future work

With this clock framework in place, we are soon going to contribute support for the Microchip SAMA5D2, which will make use of the new clock framework. Some other platforms will also gain cleaner clock support thanks to this framework: for example, the existing STM32MP1 clock support is expected to be migrated to this clock framework.

In addition, based on this clock framework, SCMI (System Control and Management Interface) clock support has also been added. While OP-TEE already has support for exposing SCMI clocks to clients, the actual callbacks have to be implemented by platform-specific code. This additional support will allow exposing clocks registered within the clock framework to a SCMI client without any custom platform code. A Device Tree description will allow matching SCMI clock identifiers with clocks provided by clock drivers.

We have already submitted a pull request for this support, which is currently under review: Provide plat_scmi_clock_* using clock framework.

New training course: Real-Time Linux with PREEMPT_RT

In the field of embedded systems, a number of applications need real-time guarantees, and the Linux ecosystem has been offering for a long time a number of solutions to address those needs, either by improving the Linux kernel itself using the PREEMPT_RT approach, or by using a co-kernel approach such as the one offered by Xenomai. Bootlin training’s portfolio already has an initial coverage of these topics in our Embedded Linux system development course.

Today, we are happy to announce a brand new Real-Time Linux with PREEMPT_RT, which is specifically focused on the PREEMPT_RT solution. This solution made a vast amount of progress in recent times in terms of integration into the official Linux kernel, which makes it even more relevant for a number of projects which need real-time guarantees.

The main topics covered by the course are:

  • What is a real-time and deterministic operating system
  • How to configure, build and setup a PREEMPT_RT enabled Linux kernel
  • How to identify and benchmark the hardware platform in terms of real-time characteristics
  • How to configure and tune the Linux kernel and the system for deterministic behavior
  • How to develop and debug real-time user-space Linux applications as well as analyze latencies

The course is illustrated by practical labs or demonstrations made on the BeagleBone Black platform. It has been developed and is taught by Bootlin engineer Maxime Chevallier, who is an experienced embedded Linux and Linux kernel engineer and trainer.

As usual our training materials are fully open-source, including the ones for this brand new session. You can read the Slides and Practical lab instructions. Bootlin is one of the very few companies delivering training courses to make its training materials open-source: by choosing to work with Bootlin for your trainings, you support our work on developing and publishing freely available training materials.

If you’re interested in getting this new Real-Time Linux with PREEMPT_RT training course, you have three options:

  • Public on-line sessions, opened to individual registration. The course lasts 3 sessions of 4 hours. We have scheduled a first session on January 19-21, 2022, and registration is open, for 399 EUR at the Early bird rate, of 449 EUR at the Regular rate.
  • Dedicated on-line sessions, which we organize at the date/time of your choice. The course also lasts 3 sessions of 4 hours. Contact us to request a quote.
  • Dedicated on-site sessions, where our trainer travels to your location to deliver the training course. In this case, the course lasts two full days. Contact us to request a quote.

Maintaining Yocto Project Documentation

For many years, Bootlin has been a strong user and a contributor to the Yocto Project, delivering numerous customized embedded Linux distributions and Board Support Packages based on Yocto Project and OpenEmbedded to its customers, for a wide range of hardware platforms and architectures.

In 2021, we have been able to bring this engagement further, as Bootlin engineer Michael Opdenacker has been given the opportunity to work as a maintainer for Yocto Project’s documentation, thanks to funding from the Linux Foundation. Since the mourning of Scott Rifenbark, the former maintainer, in early 2020, the project was in need for someone to fill this role.

Yocto Project developers and contributors did their best in the meantime though, in particular by migrating the documentation sources from the DocBook format to reStructuredText, to generate documentation using Sphinx. This migration was also done by the Linux kernel community, as reStructuredText seems to make it easier for developers to contribute to documentation.

For Michael Opdenacker, the interest of getting back to Yocto Project and OpenEmbedded was strong: he was one of the early users, over 15 years ago, of BitBake and OpenEmbedded. Many things have changed since Michael was generating his own OPIE and GPE Palmtop Environment images for handheld devices back in 2004 and 2005.

Indeed, since that time, under the Yocto Project umbrella, BitBake and OpenEmbedded have formed what’s probably the smartest, most versatile and most powerful embedded Linux build system. This build system takes a pretty steep learning curve though, and that’s why maintaining high quality documentation was a key focus from the start.

In addition to acting as a maintainer (reviewing and merging patches from contributors, encouraging contributions, managing documentation bugs, keeping the documentation current with the evolution of the code), we have so far also managed to:

  • Create brand new documentation, thanks to studying the source code and to knowledge from core developers. In particular, we documented vulnerability (CVE) management and drafted a first description of Hash Equivalence.
  • Implement a new infrastructure to generate diagrams from SVG sources. Currently, most graphics are bitmaps and do not render well in PDF and EPUB output. The next step is to migrate all diagrams to SVG.
  • Implement styling improvements, trying to eliminate unnecessarily complex sentences and make the documents easier to read.
  • Propose a “Documentation standards” document in markdown format, together with a diagram template with reusable shapes, text boxes and clipart, so that new documentation and diagrams are created in a way that’s consistent with existing documents.
  • Find bugs and improvement opportunities, and implement some of them, thanks to testing the software and reading the code.

Here are our contributions to the project so far, as of Oct. 20, 2021:

Yocto Project is a very welcoming open-source project, in which core developers are keen to help users and contributors. This makes it a real pleasure to participate to this project, in addition to the satisfaction to contribute to the progress of Embedded Linux.

If you are interested in studying Yocto Project and Bitbake’s documentation and contributing to it, you will be most welcome! The best is to get in touch with our community through the docs mailing list or chat with us on IRC. See our mailing lists and chat page for details.

If you’re interested in using Yocto Project for your embedded Linux projects, we also recommend you to check out our Yocto Project and OpenEmbedded system development training course, and its freely available training materials. We recently announced new dates for our upcoming online training classes on the Yocto Project: