Linux 5.16 released: Bootlin contributions

Linux 5.16 has been released on January 9. As usual, our recommended reading to learn more about this release is the corresponding Kernelnewbies.org page and the two articles from LWN covering the 5.16 merge window: part 1 and part 2.

As usual, Bootlin contributed a number of patches to this release, with a total of 117 commits, making Bootlin the 22th contributing company according to statistics (Unknown and Hobbyists not counting as companies).

Here are the main highlights of our contributions to 5.16:

  • Alexandre Belloni, as the maintainer of the RTC subsystem, continued to improve the core subsystem and RTC drivers. He added a new user-space interface to RTC to allow getting/settings RTC parameters. It is used to get the features of an RTC from user-space, to get/set the RTC correction or to configure Backup Switch Mode. In addition, Alexandre made various improvements to several RTC drivers, such as adding Backup Switch Mode, and general refactoring.
  • Clément Léger did a small fix in the clock drivers for Microchip ARM platforms, fixing an issue discovered as part of his work porting OP-TEE to Microchip ARM platforms.
  • Hervé Codina made some fixes to the fsmc NAND controller driver, which is used on a number of old platforms from ST. They fix support of certain NAND chips on those platforms. These issues were discovered as part of the development of a Linux BSP for an old ST Spear320 platform.
  • Maxime Chevallier fixed a deadlock in the network stack, that was causing the kernel to stop booting when using a root filesystem over NFS combined with the network interface using a SFP module.
  • Miquèl Raynal contributed many improvements to the max1027 ADC driver in the IIO subsystem, supporting hardware triggers.
  • Miquèl Raynal contributed support for the ADC found on Texas Instruments AM437x platforms. This required significant changes in the MFD driver that is used to support the multiple features of this ADC, as well as improvements in the existing IIO driver for TI ADCs.
  • Paul Kocialkowski contributed a small addition enabling the Rockchip VPU on Rockchip PX30 platforms, and merged the Device Tree bindings for the logiCVC display controller (but not yet the driver itself).

And now, the complete list of commits:

2021 at Bootlin, a year in review

2021 has come to an end, a year that everyone will most likely consider as somewhat complicated and unusual, even though the current situation seems to now becoming the new normal. The switch to a new year is generally a good moment to take a step back, and review what happened in the past year, and draw some directions for the coming year.

In this blog post, we’d like to do exactly this for Bootlin, which has seen a number of significant changes this year, as well as a continuation of its usual activities.

We also take this opportunity to wish you all an happy year 2022, and send you our best wishes. May 2022 be full of interesting projects and also be a safe year for everyone.

Here are the main topics that we cover in this lengthy blog post:

Acquisition and recruiting

One major change for Bootlin in 2021 is that the company was acquired by Thomas Petazzoni, former CTO, and Alexandre Belloni. So it’s an internal acquisition, by former employees, meaning that Bootlin has kept the same offering and spirit. Further, Bootlin’s original founder Michael Opdenacker is still in the team, but now as an employee. See our blog post regarding the acquisition, back in February.

In 2021, we also recruited several engineers, bringing significant additional expertise to our team:

  • Thomas Perrot, who joined just before the start of 2021, bringing 10 years of experience in embedded Linux BSP development, with a strong Yocto expertise.
  • Hervé Codina, who joined in March 2021, bringing 20+ years of experience in bare-metal and Linux embedded development.
  • Clément Léger who joined in June 2021, bringing 10 years of experience in Linux kernel development, including the expertise on porting the Linux kernel to a brand new CPU architecture.

We continue to have open positions for embedded Linux engineers, and we plan to hire 3 to 4 engineers in 2022, hiring for the first time engineers located outside of France.

Engineering projects

As is the case every year, our engineering team has been kept busy this year mostly by our engineering projects, all focused on our core expertise of low-level embedded Linux development.

We have selected below a few highlights of our work in 2021, in various areas.

Build systems

Apart from delivering numerous BSPs based on Yocto or Buildroot to our customers, we have also directly contributed to both the Yocto and Buildroot open-source projects.

On the Yocto Project side:

  • Bootlin engineer Michael Opdenacker has become one of the co-maintainers of the official Yocto Project documentation, making numerous improvements and contributions to this documentation. See our blog post on this topic. We will continue this involvement in the Yocto Project documentation in 2022.
  • Bootlin engineer and COO Alexandre Belloni has been active in the build and release engineering effort of the Yocto Project, as a member of the Yocto Project SWAT team. Alexandre has been working directly with the main Yocto Project architect, Richard Purdie, on the review and validation of new contributions. Other Bootlin engineers have also helped in investigating and resolving specific bugs. We will also continue this involvement in the Yocto Project build engineering in 2022.

On the Buildroot side:

  • We have continued our work towards the implementation of top-level parallel build support in Buildroot. Bootlin engineer Hervé Codina has posted several iterations of a patch series bringing a mechanism preventing file overwrites between packages, a requirement for proper top-level parallel build.
  • We implemented and published, in partnership with ST, a Buildroot BR2_EXTERNAL called buildroot-external-st which contains example configurations for the STM32MP1 platforms from ST.
  • We contributed support for hybrid ISO9660 images, supporting in a single image PC platforms based on legacy BIOS, 32-bit UEFI BIOS and 64-bit UEFI BIOS, using the grub2 bootloader.
  • We helped one of our customers reduce their out-of-tree Buildroot patches by upstreaming a number of new Buildroot packages.

Outside of our the Yocto Project and Buildroot, we also ventured into the world of Debian/Ubuntu for embedded systems, by using the ELBE tool, to which we contributed support for building Ubuntu-based images. See our blog post that describes how to use ELBE to automatically build Ubuntu-based images for the Raspberry Pi.

Audio support in Linux

Audio support in the Linux kernel is another area where Bootlin engineers have specific expertise with. Here are some examples of audio related projects we worked on in 2021:

  • Integration of audio support for an i.MX6 platform with a complex dual AD1978 audio codec configuration, with a TDM8 audio interface between the SoC and the audio codecs.
  • Development of a brand new and complete ALSA driver for a new PCIe sound card, based on a FPGA.
  • Integration of audio support for an i.MX8 platform used in the automotive space with a complex audio configuration that involves multiple A2B transceivers through the Analog Devices AD2428 A2B master, Bluetooth audio, and more.

OP-TEE

In 2021, we started working on a major project: adding support for the Microchip SAMA5D2 processor to the OP-TEE project. This project is already well underway, as we have a functional port of OP-TEE, which is now being upstreamed.

As part of this, Bootlin engineer Clément Léger has implemented and contributed a generic clock framework for OP-TEE, which has already been accepted upstream. See our blog post for more details.

Clément also gave a talk at the Embedded Linux Conference 2021 on the topic of OP-TEE, titled OP-TEE: When Linux Loses Control. Slides and video are available.

We expect to continue the upstreaming of the SAMA5D2 support in 2022, and also develop support for additional SAM5D2 hardware capabilities in OP-TEE.

U-Boot extension board manager

In collaboration with the BeagleBoard.org community, we developed and contributed to U-Boot a generic extension board manager. This mechanism allows hardware platforms such as the BeagleBone to automatically detect extension boards that are connected, and apply the Device Tree overlays that provide the hardware description for those extension boards.

This work was covered in detail in the talk given by Bootlin engineer Köry Maincent at the Live Embedded Event in June 2021, see the slides and video.

Modems

In 2021, we had the opportunity to work on several projects that involved 3G/4G/5G modems, strengthening our knowledge of the modem stack in Linux, especially around modem-manager, libmbim and libqmi.

For example, we worked on the support of a Quectel BG95 modem interfaced with a RaspberryPi 4, or the support of a Sierra Wireless EM9190 modem interfaced over PCIe to an NXP i.MX6. The latter was particularly challenging and is still on-going, as the upstream Linux support for PCIe 5G modems is still very recent. We contributed a few fixes in this area.

Secure boot

Secure boot remains an important topic for a growing number of projects, and in 2021, we helped customers with secure boot on several i.MX6 platforms, and on one i.MX8 platform. The Yocto integration of secure boot was also a key aspects in those projects, to get a proper process for signing and verifying all states of the boot process. We also have a few U-Boot contributions in our contribution pipeline related to improving secure boot support. See for example the talk from Thomas Perrot at Live Embedded Event, relating the secure boot setup on i.MX8.

Camera, ISP and video support

We finished 2020 with a lot of on-going contributions to the Linux Video4Linux subsystem, which we had summarized in a blog post early 2021. In particular, our drivers for the OV5568 and OV8865 camera sensors were merged upstream.

In 2021, Bootlin engineer and multimedia expert Paul Kocialkowski continued to work on several multimedia topics. Some of the key projects included:

  • Optimizing the H264 decode → rescale → H264 encode pipeline of one of our customers, on an Allwinner H3 platform. We were able to significantly optimize the pipeline by leveraging the H264 decoder built into the Allwinner processor, for which Paul had written a Linux kernel driver several years ago, and by developing a custom ffmpeg plugin that offloaded the rescaling to the GPU, using the open-source lima support. A fully open-source solution!
  • Developing a proper Linux kernel driver for the Allwinner ISP, with support for debayering and noise filtering. This was a significant challenge as the Allwinner ISP was so far only supported through closed-source binary blobs. We have already submitted a first iteration to the upstream Linux kernel community. See our blog post for more details, as well as the talk given by Paul at the Embedded Linux Conference: slides and video.
  • Improving the Allwinner Linux kernel camera driver to support single buffer capture. This is useful in scenarios where camera sensors are not used to capture a sequence of frames, but just single frames, and the available memory is limited. This is particularly true on platforms such as the Allwinner V3s, which may be limited to just 64MB of RAM. This is going to be submitted upstream soon, as part of our on-going work on the Allwinner camera driver.

NAND and flash support

With Bootlin engineer Miquèl Raynal being a maintainer of the NAND subsystem in Linux and a co-maintainer of the MTD subsystem, it should be no surprise that we have continued to deal with a number of flash memory related projects in 2021:

  • We have brought in mainline the support for several NAND flash controllers:
    • For the Arasan NAND controller, used in some Xilinx processors. See our blog post on this topic.
    • For the ARM Primecell PL35x NAND controller, used in some other Xilinx processors. See our blog post on this topic.
    • For the NAND controller found at least in the the Renesas R-Car Gen3 and RZ/N1 processors.
  • Contributed support for the NV-DDR interface, which is used by some NAND flash chips and controllers to increase the throughput. See our blog post.
  • Continued our work on generalizing ECC support in the MTD subsystem, so that all ECC setups (software ECC, on-die ECC, ECC in the NAND controller, or external ECC engine) can be supported regardless of the NAND interface (serial or parallel). See the talk from Miquèl Raynal, ECC engines given at last year’s ELCE.
  • Improved the TI GPMC NAND controller driver to properly support NAND chip with large pages (larger than 4KB).

ADC/IIO support

We’ve helped a few customers with Linux kernel support for ADC devices, namely:

  • Extend the support for the MAX1027 family of ADCs, with support for external triggers
  • Bring support for the ADC found in TI AM437x processors

All of these were contributed to the upstream Linux kernel, Miquèl Raynal also wrote a extensive blog post on various aspects of the IIO subsystem and has taken the opportunity of these projects to also improve/clarify various aspects of the IIO core in a recent patch series.

Networking

Networking support in the Linux kernel is also one area where Bootlin is very active. Here are some key projects we worked on in 2021, some of them being on-going with additional work expected in 2022:

  • We implemented support for PTP offloading for the Qualcomm AR803X PHY.
  • We started working on QUSGMII support in Linux, a new standard that allows PTP time-stamps to be included directly on the preamble of QSGMII frames.
  • We started working on the 802.15.4 stack in the Linux kernel, with our initial step being support for passive scanning.
  • We implemented mqprio support in the mvneta Linux kernel driver, used for some Marvell platforms. mqprio is a queuing discipline that allows mapping traffic flows to hardware queue ranges using priorities and a configurable priority to traffic class mapping.
  • We implemented Frame DMA support in the Ocelot Ethernet switch driver, significantly improving the performance of frame injection/extraction by the CPU into/from the switch traffic.
  • We developed proper SFP support in Linux and U-Boot, with dynamic reconfiguration (in Linux only), for a customer using the Zynq 7000 platform and a complex network setup.

BSP development

Many of our projects are obviously related to the development of update of complete Linux BSP for our customers (bootloader, Linux kernel, custom embedded Linux distribution). Here are a few examples:

  • Development of a brand new U-Boot, Linux and Yocto based BSP to migrate an existing product running Windows CE on a ST Spear320 processor to Linux. This sort of development proved to be challenging as the Spear320 support in both U-Boot and Linux is close to be abandoned (and in fact has since then been removed from U-Boot upstream). We hope to be able to contribute to improve the upstream Spear320 support in 2022.
  • Migration of an AM335x/Buildroot based BSP to newer Linux kernel and Buildroot releases, for a customer in the healthcare industry
  • On-going migration of an i.MX6 BSP from an old kernel release and a custom build-system to recent versions of U-Boot and Linux as well as a proper standardized Yocto Project based Linux distribution. This project presents some interesting challenges as it uses only one of the two Cortex-A9 cores to run Linux, the other one runs a bare-metal application, and we will also migrate this to a proper usage of the remoteproc and rpmsg mechanisms.
  • We worked with several customers on STM32MP1 platforms, helping with porting on new platforms, extending the Device Tree and device drivers, developing custom Yocto-based or Buildroot-based distributions. STM32MP1 is definitely becoming a popular platform for a number of new projects.
  • We also worked with several customers on custom platforms based on the RasberryPi Compute Module, doing Device Tree configuration/tweaking and Yocto integration.

Over-the-Air update integration

We continued to help our customers with the integration of Over-The-Air update solutions in their embedded Linux systems. A few examples:

  • We integrated RAUC for an i.MX6 platform, using the Barebox bootloader and Yocto Project based distribution. Bootlin engineer Kamel Bouhara wrote a detailed blog post on this topic, as well as the talk from Kamel at Live Embedded Event
  • We integrated the Azure Device Update for IoT Hub, using the U-Boot bootloader on a RaspberryPi CM4 platform, also with a Yocto Project based distribution. Internally, the Device Update for IoT Hub is based on swupdate
  • As part of a migration of an existing embedded product based on Debian to a Yocto-based distribution, we are integrating Mender.

Crédit Impôt Recherche

For our French customers, another important milestone achieved by Bootlin in 2021 is the delivery of our Crédit Impôt Recherche agreement, which allows our French customers to benefit from tax incentives on research and development activities done by Bootlin for their projects. See our blog post for more details.

Training

With the COVID19 still making travel conditions difficult and uncertain, most of our training activity in 2021 was dedicated to on-line training courses. Indeed, 69 out of our 72 training sessions were delivered online this year. In total, we delivered our training courses to 906 engineers in 2021.

In 2021, we published two new training courses:

Just like all our other training courses, the complete training materials for those new courses are freely available, distributed under the CC-BY-SA license.

Another major event in 2021 was our work to get the French Qualiopi certification, which proves the quality of our training organization and processes. As part of this, we have improved several aspects of our training courses, mainly regarding feedback collection and handling as well as the evaluation of the participants.

In 2022, we expect to at least:

  • Improve how our online course are delivered, by ensuring more participants can do the practical labs by themselves, which is an important part of the learning process
  • Publish at least one more training course. We already have plans for a course on a topic that we think will be very relevant to many embedded Linux engineers. Stay tuned for updates on this!
  • Have additional capacity to deliver our training courses, which are seeing significant demand.

For more details about our training offering, see our training page, which details our dedicated on-line and on-site sessions as well as our public on-line sessions.

Contributions

In Bootlin continued and strong open-source focus, we once again contributed to different open-source projects in 2021:

  • Linux kernel. The best summaries are our blog posts about our contributions to Linux 5.10, Linux 5.11, Linux 5.12, Linux 5.13, as well as Linux 5.14 and 5.15. Some of our significant contributions: huge effort on ECC engine support in the MTD subsystem, rv3032 RTC driver, support for new MIPS platforms from Microchip, a Simple Audio Mux driver, major work on timer/TCB support on Microchip ARM platforms, new I3C master controller driver for the Silvaco I3C IP, new drivers for the OV5568 and OV8865 camera sensors, NAND controller drivers for the Arasan, PL35x and Renesas IPs, enabling of ADC support on TI AM437x.
  • U-Boot. Our biggest contribution has been the extension board manager, which is described earlier in this blog post. We also contributed various small fixes and improvements.
  • OP-TEE. We contributed a generic clock framework, and the support for the SAMA5D2 clock tree and TRNG driver. So far 55 commits from Bootlin have been integrated in OP-TEE.
  • Buildroot. We contributed a total of 257 patches. The main contributions are in the area of tooling to list security vulnerabilities in Buildroot packages, support for Bootlin external toolchains, improved SELinux support, support for the Beagle-V RISC-V platform, a good number of new packages, support for hybrid ISO9660 images. In addition, Thomas Petazzoni continued his work as a co-maintainer of the project: Thomas has reviewed and merged close to 2000 patches from Buildroot contributors throughout 2021.
  • Yocto Project. As outlined above in this blog, we did major contributions to the Yocto Project, with Michael Opdenacker becoming co-maintainer of the Yocto Project documentation, and Alexandre Belloni being involved in the build and release engineering effort.
  • Linux Test Project. We fixed a number of issues in LTP tests that prevented from using LTP in embedded-oriented systems, such as the ones that can be generated using the Yocto Project. See our blog post for more details.
  • Our training materials, which are all freely available, have seen no less than 872 commits. These includes updates to our existing training courses, but also our new Real-Time Linux with PREEMPT_RT training course.
  • Our freely available toolchains have also been improved and updated. See our blog post.
  • We continued to maintain our very popular Elixir code indexing tool.

Conferences

Here are the talks that we presented this year, at various virtual events:

  • FOSDEM 2021
    • Networking Performance in the Linux Kernel, Getting the most out of the Hardware, by Maxime Chevallier. Slides and video.
    • Embedded Linux from scratch in 45 minutes, on RISC-V, by Michael Opdenacker. slides and video.
  • Live Embedded Event
    • Security vulnerability tracking tools in Buildroot, by Thomas Petazzoni. Slides and video.
    • Secure boot in embedded Linux systems, by Thomas Perrot. Slides and video.
    • Understanding U-Boot Falcon Mode, by Michael Opdenacker. Slides and video.
    • Device Tree overlays and U-Boot extension board management, by Köry Maincent. Slides and video.
    • Getting started with RAUC, by Kamel Bouhara. Slides and video.
  • Embedded Linux Conference
    • I3C in tomorrow’s design, by Miquèl Raynal. Slides and video.
    • Embedded Linux nuggets found in Buildroot package Eldorado, by Michael Opdenacker. Slides and video.
    • OP-TEE: When Linux Loses Control, by Clément Léger. Slides and video.
    • Advanced Camera Support on Allwinner SoCs with Mainline Linux, by Paul Kocialkowski. Slides and video.
  • A webinar organized in partnership with ST, Device Tree 101, by Thomas Petazzoni. Slides and video.
  • A webinar organized in partnership with Microchip, Improving Linux Boot Time in Your Embedded Application, with the participation of Thomas Petazzoni. Link.

Bootlin toolchains updated, 2021.11 release

Bootlin has been offering since 2017 a large set of ready to use pre-compiled cross-compilation toolchains at toolchains.bootlin.com. These toolchains are available for a wide range of CPU architectures and CPU variants, and support either the glibc, uClibc-ng or musl C libraries, where applicable.

It’s been quite some time since the last release of those toolchains, so we took the opportunity of this quiet period between Christmas and New Year to finally update the toolchains. We’re happy to announce that we have now published a total of 187 toolchains targeting 46 different CPU architecture variants. As the toolchain release name suggests, they are now built with Buildroot 2021.11.

The most important changes are the following ones:

  • The so-called stable toolchains are now based on gcc 10.3.0, binutils 2.36.1, Linux headers 4.9, gdb 10.2, glibc 2.34, uClibc 1.0.39 and musl 1.2.2
  • The so-called bleeding-edge toolchains are now based on gcc 11.2, binutils 2.37, Linux headers 5.4, gdb 11.1, glibc 2.34, uClibc 1.0.39 and musl 1.2.2
  • The riscv64 toolchains (targeting the LP64 ABI) have been replaced by riscv64-lp64d toolchains (targeting the LP64D ABI), making them more generally useful.
  • Runtime testing using Qemu has been added for RISC-V 64-bit, m68k-68xxx and OpenRISC.
  • The gdb cross-debugger is now compiled with both Python and TUI support.
  • Internally, there’s been some significant refactoring of the scripts and Gitlab CI pipelines that control the build and testing of those toolchains. Many thanks to Romain Naour for providing the ground work that enabled this refactoring.

The toolchains can be directly downloaded from toolchains.bootlin.com. We have also submitted a patch to Buildroot that updates those toolchains, which are directly usable in Buildroot as external toolchains.

Bootlin is now a Qualiopi certified training provider

Certificat Qualiopi BootlinBootlin has been delivering training courses in the field of Embedded Linux since its creation in 2004, delivering over 430 courses to more than 4500 engineers just since 2009, in over 40 countries, with a high-level of quality and a full transparency, with fully open training materials and publicly available training evaluations.

In France, a new quality certification brand for training and skills development service providers, called Qualiopi, has been created and is going to be needed starting January 1, 2022 for French training providers who want to deliver services to French customers who use public funding for training courses. For more details about Qualiopi, see this page in English, or the official page from the French Ministry, in French.

Bootlin has gone through the Qualiopi certification process, making a number of improvements to how we deliver and manage our training courses along the way, and we are happy to announce that Bootlin, at the first attempt, was successfully certified as a Qualiopi compliant training service provider. See our certificate, which can also be verified on the AFNOR certification authority website.

We are proud of this result, which shows our commitment to delivering high quality training courses to all our customers. Do not hesitate to look at our training courses portfolio and contact us if you are interested.

Announcing buildroot-external-st, Buildroot support for STM32MP1 platforms

STM32MP1 Discovery Kit 2Back in 2019, ST released a brand new processor family, the STM32MP1, whose members are currently based on a dual Cortex-A7 to run Linux combined with one Cortex-M4 to run bare-metal applications, together with a wide range of peripherals.

Following the release of this new platform, Bootlin ported its Embedded Linux and Yocto training courses to be available on STM32MP1, and also published a long series of tutorials showing how to use Buildroot to build an embedded Linux system on STM32MP1: part 1, part 2, part 3, part 4, part 5, part 6 and part 7.

We are happy to announce that we have partnered with ST to develop an improved support of Buildroot on STM32MP1, which is materialized by a Buildroot BR2_EXTERNAL available on Github at https://github.com/bootlin/buildroot-external-st. In Buildroot, a BR2_EXTERNAL is an extension of the core Buildroot, with additional configurations and/or packages.

This BR2_EXTERNAL tree is an extension of Buildroot 2021.02, which provides four example Buildroot configuration to easily get started on STM32MP1 platforms:

  • st_stm32mp157a_dk1, building a basic Linux system for the STM32MP1 Discovery Kit 1 platform, with a minimal Linux root filesystem
  • st_stm32mp157c_dk2, building a basic Linux system for the STM32MP1 Discovery Kit 2 platform, with a minimal Linux root filesystem
  • st_stm32mp157a_dk1_demo, building a much more featureful Linux system for the STM32MP1 Discovery Kit 1 platform, with Linux root filesystem that allows to run Qt5 applications with OpenGL acceleration on the HDMI output, that supports audio, demonstrates the usage of the Cortex-M4, uses OP-TEE, and more.
  • st_stm32mp157c_dk2_demo, building a much more featureful Linux system for the STM32MP1 Discovery Kit 2 platform, with Linux root filesystem that allows to run Qt5 applications with OpenGL acceleration on both the integrated DSI display and HDMI output, that supports audio, WiFi and Bluetooth, demonstrates the usage of the Cortex-M4, uses OP-TEE, and more.

This BR2_EXTERNAL is designed to work with Buildroot 2021.02, with only a small set of modifications, which since then have been integrated in upstream Buildroot, ensuring that STM32MP1 users can directly use upstream Buildroot for their projects.

There is extensive documentation on how to use this BR2_EXTERNAL tree as well as how to test the various features of the STM32MP1 platform: using STM32CubeProgrammer, using Device Tree generated from STM32 CubeMX, using the Cortex-M4, testing display support, using Qt5, using WiFi, using Bluetooth, using audio and using OP-TEE. We have also documented the internals of the BR2_EXTERNAL components.

This Buildroot support is using the latest software components from the recently released 3.1 BSP from ST (see release notes), so it is based on Linux 5.10, U-Boot 2020.10, TF-A 2.4 and OP-TEE 3.12. We will keep this BR2_EXTERNAL updated with newer releases of the ST BSP.

If you face any issue while using this Buildroot support for STM32MP1, you can use the issue tracker of the Github project, or use the ST community forums. Bootlin can also provide commercial support on Linux on STM32MP1 platforms, as well as training courses.