Thomas Petazzoni is Bootlin's co-owner and CEO. Thomas joined Bootlin in 2008 as a kernel and embedded Linux engineer, became CTO in 2013, and co-owner/CEO in 2021. More details...
Since December 1st, 2020, we’re happy to have in our team an additional engineer, Thomas Perrot, who joined our office in Toulouse, France.
Thomas brings 6+ years of experience working on embedded Linux systems, during which he worked at Intel on Android platforms, and then at Sigfox on the base stations for Sigfox’s radio network. Thomas has experience working with Linux on x86-64, ARM and ARM64 platforms, with a wide range of skills: bootloader development, Linux kernel and driver development, Yocto integration, OTA updates. Thomas was also deeply involved in the strong security aspects of Sigfox base stations, with secure boot and measured boot, TPM, integrity measurement, etc. At Sigfox, Thomas was involved in all steps of the product life-cycle, from the design phase all the way to the in-field deployment, update and maintenance. Last but not least, Thomas is a Linux technologist and a free software enthusiast, who hacks some open source hardware projects on his free time. See also Thomas page on our website and his LinkedIn profile.
Over the past years, we have been more and more involved in projects that have significant multimedia requirements. As part of this trend, 2020 has lead us to work on a number of contributions to the Video4Linux subsystem of the Linux kernel, with new drivers for camera interfaces, camera sensors, video decoders, and even HW-accelerated video encoding. In this blog post, we propose to summarize our contributions and their status on the following topics:
Rockchip PX30, RK1808, RK3128 and RK3288 camera interface driver
Allwinner A31, V3s/V3/S3 and A83T MIPI CSI-2 support for the camera interface driver
Omnivision OV8865 camera sensor driver
Omnivision OV5648 camera sensor driver
TW9900 PAL/NTSC video decoder driver
Rockchip HW-accelerated H264 video encoding
Rockchip camera interface
The Rockchip ARM processors are known to have very good support in the upstream Linux kernel. However, one area where the support was lacking is in the support of the camera interface used by those SoCs. And it turns out that Bootlin engineer Maxime Chevallier has worked precisely on this topic throughout 2020: the development and upstreaming of the rkvip driver, a Video4Linux driver for the Rockchip camera interface. While the work was done and tested on a Rockchip PX30 platform, the same camera interface is used on RK1808, RK3128 and RK3288.
Several iterations of the driver have been posted on the linux-media mailing list, with the latest iteration, version 5, posted on December 29, 2020:
We’re hoping to get this driver merged soon, as we have now addressed the feedback that was received through the 5 iterations the patch series as gone through. It should be noted that for now it only supports the parallel BT656 interface as this is what we needed for our current project, we are definitely able to extend it to support MIPI CSI2 as well if you’re interested!
It should be noted that as a result of this work, Maxime Chevallier also prepared and delivered a talk From a video sensor to your display which was given at the Embedded Linux Conference Europe 2020. See the slides and video.
Allwinner MIPI CSI2 camera interface
As part of an internship in 2020 and then a customer project, Bootlin intern Kévin L’Hôpital and Bootlin engineer Paul Kocialkowski worked on extending the Allwinnera camera interface support with support for MIPI CSI2 cameras. In fact, this addition was done to two Allwinner camera interface drivers: the sun6i driver which is used on Allwinner A31 and V3s/V3/S3, and the sun8i-a83t, which is used on the Allwinner A83T.
Through a fairly long 15 patches patch series, support for MIPI CSI2 is added to both camera interface controllers. We have tested both with Omnivision sensors, which are described below.
The series is currently in its third iteration, which was posted by Paul Kocialkowski on December 11, 2020 on the linux-media mailing list:
Paul Kocialkowski (15):
docs: phy: Add a part about PHY mode and submode
phy: Distinguish between Rx and Tx for MIPI D-PHY with submodes
phy: allwinner: phy-sun6i-mipi-dphy: Support D-PHY Rx mode for MIPI
CSI-2
media: sun6i-csi: Use common V4L2 format info for storage bpp
media: sun6i-csi: Only configure the interface data width for parallel
dt-bindings: media: sun6i-a31-csi: Add MIPI CSI-2 input port
media: sun6i-csi: Add support for MIPI CSI-2 bridge input
dt-bindings: media: Add A31 MIPI CSI-2 bindings documentation
media: sunxi: Add support for the A31 MIPI CSI-2 controller
ARM: dts: sun8i: v3s: Add nodes for MIPI CSI-2 support
MAINTAINERS: Add entry for the Allwinner A31 MIPI CSI-2 bridge
dt-bindings: media: Add A83T MIPI CSI-2 bindings documentation
media: sunxi: Add support for the A83T MIPI CSI-2 controller
ARM: dts: sun8i: a83t: Add MIPI CSI-2 controller node
MAINTAINERS: Add entry for the Allwinner A83T MIPI CSI-2 bridge
.../media/allwinner,sun6i-a31-csi.yaml | 88 ++-
.../media/allwinner,sun6i-a31-mipi-csi2.yaml | 149 ++++
.../media/allwinner,sun8i-a83t-mipi-csi2.yaml | 147 ++++
Documentation/driver-api/phy/phy.rst | 18 +
MAINTAINERS | 16 +
arch/arm/boot/dts/sun8i-a83t-bananapi-m3.dts | 2 +-
arch/arm/boot/dts/sun8i-a83t.dtsi | 26 +
arch/arm/boot/dts/sun8i-v3s.dtsi | 67 ++
drivers/media/platform/sunxi/Kconfig | 2 +
drivers/media/platform/sunxi/Makefile | 2 +
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 165 +++--
.../platform/sunxi/sun6i-csi/sun6i_csi.h | 58 +-
.../platform/sunxi/sun6i-csi/sun6i_video.c | 53 +-
.../platform/sunxi/sun6i-csi/sun6i_video.h | 7 +-
.../platform/sunxi/sun6i-mipi-csi2/Kconfig | 12 +
.../platform/sunxi/sun6i-mipi-csi2/Makefile | 4 +
.../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c | 590 ++++++++++++++++
.../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h | 117 ++++
.../sunxi/sun8i-a83t-mipi-csi2/Kconfig | 11 +
.../sunxi/sun8i-a83t-mipi-csi2/Makefile | 4 +
.../sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.c | 92 +++
.../sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.h | 39 ++
.../sun8i_a83t_mipi_csi2.c | 657 ++++++++++++++++++
.../sun8i_a83t_mipi_csi2.h | 197 ++++++
drivers/phy/allwinner/phy-sun6i-mipi-dphy.c | 164 ++++-
drivers/staging/media/rkisp1/rkisp1-isp.c | 3 +-
include/linux/phy/phy-mipi-dphy.h | 13 +
27 files changed, 2581 insertions(+), 122 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun6i-a31-mipi-csi2.yaml
create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun8i-a83t-mipi-csi2.yaml
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Kconfig
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/Makefile
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
create mode 100644 drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.h
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/Kconfig
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/Makefile
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.c
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_dphy.h
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.c
create mode 100644 drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.h
Here as well, the patch series has gone through a number of iterations, with significant reshaping to take into account the comments and feedback of other kernel developers and maintainers, so we hope to be near the point where it can be merged.
Omnivision OV8865 camera sensor driver
As part of his internship at Bootlin in 2020, Kévin L’Hôpital implemented a driver for the OV8865 camera sensor, connected over MIPI CSI2 to an Allwinner A83T platform. This OV8865 was then taken by Bootlin engineer Paul Kocialkowski, who did additional rework and polishing.
We are currently at the 4th iteration of this driver, which has been posted on December 11, 2020, and it has now been accepted and submitted to the V4L maintainer in a pull request.
In addition to the work done by Bootlin intern Kévin L’Hôpital on OV8865 with Allwinner A83T, Paul Kocialkowski worked on OV5648 with Allwinner V3s, also connected over MIPI CSI2. This work results in a driver for the OV5648 camera sensor, which Paul has submitted to the linux-media mailing list.
This driver is now in is 5th iteration, posted on December 11, 2020, and it has now been accepted and submitted to the V4L maintainer in a pull request.
In addition to working on the Rockchip camera interface driver, Maxime Chevallier has also worked on a driver for the TW9900 PAL/NTSC video decoder. This chip from Renesas, takes as input an analog PAL or NTSC signal, digitizes it and outputs it on a parallel BT656 interface, which in our case was connected to a Rockchip PX30 platform.
Maxime posted the third iteration of the patch series adding this driver on December 22, 2020 on the linux-media mailing list.
In 2018 and thanks to success of the crowd-funding campaign we ran back then, Bootlin engineer Paul Kocialkowski pioneered support for stateless video decoders in the Linux kernel, with a first driver supporting MPEG2, H264 and H265 HW-accelerated video decoding on Allwinner platforms.
In 2020, Paul was tasked to work on HW-accelerated H264 video encoding for Rockchip platforms, which also use a stateless video encoder. Of course, Paul took the same approach of going towards an upstream-acceptable solution rather than relying on out-of-tree and vendor-specific solutions provided by Rockchip.
As explained in Paul’s talk, this is not fully ready for upstream, as lots of discussions are needed on the user-space APIs, especially around the topic of rate control.
If you are interested in having this work fully available in the upstream Linux kernel, please contact us. We are looking for additional funding and support to push this completely upstream.
Conclusion
As can be seen from the numerous topics covered in this blog post, Bootlin has significant experience with the Video4Linux subsystem, and is able to both implement support for new hardware, extend the Video4Linux subsystem if needed, and contribute drivers and changes to the official Linux kernel.
It’s the beginning of 2021, and Bootlin’s offering of online training courses continues. We have dates available for our 5 training courses, at an affordable cost, and with the same quality characteristics of all Bootlin courses: trainers with proven in-field experience, fully open-source training materials and worldwide recognized training contents.
Linux kernel driver development course, taught by Bootlin engineer Grégory Clement, who has 19 years of embedded development experience, is a Linux kernel maintainer, and has contributed over 400 patches to the official Linux kernel. This course takes place from January 11 to January 19, register for 829 EUR
Buildroot system development, taught by Bootlin CTO Thomas Petazzoni, who is one of the co-maintainers of Buildroot, and has contributed over 4000 patches to the project over the past 12 years. The course will take place from March 8 to March 11, register for 519 EUR
See our training page for more details about all our training courses!
Yesterday, Bootlin co-organized and participated to the first edition of Live Embedded Event, a new online conference dedicated to embedded systems topics. In addition to co-organizing the event, we also gave four different talks at this conference, and we are happy to share the slides and videos of our talks.
Introduction video
The following video was presented during the introduction keynote of the event, where each sponsor and co-organizer could say a few words.
An overview of the Linux kernel and user-space graphics stack
Talk given by Bootlin engineer Paul Kocialkowski, slides available in PDF.
ASoC: Supporting Audio on an Embedded Board
Talk given by Bootlin engineer Alexandre Belloni, slides available in PDF.
Embedded Linux “from scratch” in 45 minutes…on RISC-V
Talk given by Bootlin founder and CEO Michael Opdenacker, slides available in PDF.
Device Tree: hardware description for everybody!
Talk given by Bootlin CTO Thomas Petazzoni, slides available in PDF.
As we announced back in October, Bootlin has participated to the organization of a new online conference around embedded systems: Live Embedded Event, which will take place on December 3. The registration is totally free, and the event will propose 4 tracks throughout the day, covering a wide range of topics. We encourage you to register and participate to the event!
Bootlin is not only co-organizing the event, but also has 4 talks that have been accepted:
An overview of the Linux and user-space graphics stack, by Paul Kocialkowski, Bootlin’s multimedia expert and author of our Linux Graphics training course. Talk abstract: Graphics with the Linux kernel is often perceived as a haystack, composed of many components that have complex interactions with eachother. While this is not always far from the truth, this talk will try to provide a clear overview of the different pieces in the stack and what their individual role is. Throughout the presentation, various APIs, protocols and software implementations will be demystified, covering both display and render aspects. This includes topics such as DRM, Wayland, Xorg, OpenGL, Mesa and more!
Embedded Linux from scratch in 45 minutes… on RISC-V, by Michael Opdenacker, Bootlin’s founder and CEO. Talk abstract: Join and discover how to build your own embedded Linux system completely from scratch. You will build your own toolchain, bootloader and kernel, this time on the Risc-V open Instruction Set Architecture, that you will run through a system emulated by QEMU. You will also build a minimal root filesystem by yourself thanks to the BusyBox project. You will finish by controlling the system through a tiny webserver. The approach will be to provide only the files that are strictly necessary. That’s all the interest of embedded Linux: you can really control and understand everything that runs on your system, and see how simple the system can be. That’s much easier than trying to understand how a GNU/Linux system works from a distribution as complex as Debian!
ASoC: audio on an embedded board, by Alexandre Belloni, Bootlin’s audio expert, Linux kernel and Yocto trainer, as well as kernel maintainer. Talk abstract: ASoC, which stands for ALSA System on Chip, is a Linux kernel subsystem created to provide better ALSA support for system-on-chip and portable audio codecs. It allows to reuse codec drivers across multiple architectures and provides an API to integrate them with the SoC audio interface. This talk will present the typical hardware architecture of audio devices on embedded platforms, how to integrate the audio codec and processor audio interface using device tree. Examples, common issues and debugging tips will also be discussed.
Device Tree: hardware description for everybody!, by Thomas Petazzoni, Bootlin’s CTO and co-maintainer of Buildroot. Talk abstract: The Device Tree has been adopted for the ARM 32-bit Linux kernel support almost a decade ago, and since then, its usage has expanded to many other CPU architectures in Linux, as well as bootloaders such as U-Boot and Barebox. Even though Device Tree is no longer a new thing, developers coming into the embedded Linux world often struggle to understand what Device Trees are, what is their syntax, how they interact with the Linux kernel device drivers, what Device Tree bindings are, and more. This talk will provide an introduction to the Device Tree, to jump start new developers in using this description language that is now ubiquitous in the vast majority of embedded Linux projects.
Note that we are aware of limitations of the platform that was chosen for the online conference: the details of the talks, and details about sponsors are only visible for people who registered. This is unfortunate, but we discovered this when it was too late to change the platform, together with a few other limitations. Let’s say this is a first edition, and regardless of those limitations, the selection of talks and contents is very promising!
The Embedded Linux Conference Europe took place online last week. While we definitely missed the experience of an in-person event, we strongly participated to this conference with no less than 7 talks on various topics showing Bootlin expertise in different fields: Linux kernel development in networking, multimedia and storage, but also build systems and tooling. We’re happy to be publishing now the slides and videos of our talks.
From the camera sensor to the user: the journey of a video frame, Maxime Chevallier
In these times of COVID19, pretty much all of the existing conferences have moved to an online format. For example, the Embedded Linux Conference Europe is going to take place next week, online, and Bootlin will significantly contribute to the event with no less than 7 talks on a wide range of topics.
But this trend for online conferences has also spurred the creation of new events. And specifically, we’re happy to announce the creation of a new conference oriented towards our favorite topic of embedded systems: Live Embedded Event. It will take place online on December 3 and will have a broader range of topics covered than ELC typically has, as Live Embedded Event is open to non-Linux embedded topics, hardware platform and interfaces discussions, and more.
The event is mainly organized by four French companies specialized in embedded systems, and we expect the contents to be a mix of talks in English and in French. Speaking of contents, the Call For Papers is open until November 10, so you can have the chance to share your knowledge about embedded topics, show your expertise, and meet new users and customers. And of course, you can also simply register as an attendee: the event is free !
The online training courses that we started earlier this year continue to have a good success, and as our sessions scheduled for September/October are now completed or full, we are happy to announce our next set of dates for November/December:
All courses are taught by experienced Bootlin engineers, who are not just trainers: they in fact spend most of their time working on engineering projects for our customers, and share their experience through these training courses. The above courses will be taught by Michael Opdenacker, Alexandre Belloni, Maxime Chevallier, Paul Kocialkowski and Thomas Petazzoni.
Note that in addition to those public training courses, we can upon request organize dedicated online training courses, to match the needs, schedule and availability of our customers. Do not hesitate to contact us for any question.
Boot testing in Qemu was added for PowerPC64 E5500, NIOSII and m68k MCF5208.
In addition, it is worth mentioning that all those Bootlin toolchains are now directly accessible from Buildroot: make menuconfig shows the Bootlin toolchains available for the current selected CPU architecture, and Buildroot is able to automatically download and use the toolchain. This feature will be available starting from Buildroot 2020.11:
Thanks again to the entire Buildroot community, and especially Romain Naour, for all the fixes and improvements related to toolchain support that make this project possible. In the next weeks, we hope to be able to deliver further updated bleeding-edge toolchains, with glibc 2.32 and binutils 2.35. Stay tuned!
If you face any issue, or need additional features in those toolchains, do not hesitate to report an issue in our issue tracker.
The schedule for the next Embedded Linux Conference Europe has been recently published, and Bootlin will once again be strongly present at this (virtual) event by giving a number of presentations. The registration for ELC-E is open, and due to the virtual nature of the event, the registration cost is only $50, which makes is accessible to pretty much everybody.
From the Camera Sensor to the User, the Journey of a Video Frame. In this talk, Bootlin engineer Maxime Chevallier will share his experience working in the Video4Linux subsystem, implementing support for the Rockchip camera interface controller, and a PAL/NTSC decoder used as the input source. Talk on Monday October 26 at 18:30 GMT.
Yocto Project and OpenEmbedded: A Collection of Best Practices. In this talk, Bootlin engineer, and Yocto Project expert and trainer Alexandre Belloni will share his experience of using Yocto Project and OpenEmbedded through a collection of best practices. There are indeed numerous ways of using OpenEmbedded and the Yocto Project, but some ways and solutions are better than others! Talk on Tuesday October 27 at 13:00 GMT.
Building Embedded Debian and Ubuntu Systems with ELBE. In this talk, Köry Maincent will share his experience using the ELBE build system, which can be used to automate the process of creating embedded Linux systems based on Debian or Ubuntu. Köry has contributed to ELBE the support for building Ubuntu systems, and has used ELBE on two different projects. This is an interesting alternative to the traditional cross-compilation approach taken by Yocto Project, OpenEmbedded or Buildroot. Talk on Tuesday October 27 at 15:15 GMT.
Using Visual Studio Code for Embedded Development. In this talk, Michael Opdenacker will share his experiments of using VS Code for embedded Linux development, which extensions are available to help navigate in the code, build and debug code, from kernel-space and user-space. Talk on Tuesday October 27 at 17:15 GMT.
Precision Time Protocol (PTP) and Packet Timestamping in Linux. Antoine Ténart has implemented PTP support in both an Ethernet switch driver, and an Ethernet PHY driver, both in the upstream Linux kernel. He will share his experience about PTP, its support in Linux, and its offloading at the MAC and PHY level. Talk on Tuesday October 27 at 19:30 GMT.
Supporting Hardware-Accelerated Video Encoding with Mainline. After working on the HW-accelerated video decoding on Allwinner platforms as part of our crowdfunded effort, Paul Kocialkowski recently worked on HW-accelerated video encoding on Rockchip platforms. In this talk, he will share the issues encountered, and what needs to be resolved to create a useful kernel to userspace interface to properly support stateless video encoders. Talk on Wednesday October 28 at 16:15 GMT.
Understand ECC Support for NAND Flash Devices in Linux. Miquèl Raynal, the Linux kernel NAND subsystem maintainer, has recently worked on improving support for various strategies to handle ECC for NAND flash devices. He will share some background information on ECC, why they are needed, how and where ECC are typically handled, and how the Linux kernel deals with the different possibilities. Talk on Wednesday October 28 at 18:30 GMT.
In addition to contributing talks, Bootlin CTO Thomas Petazzoni is also a member of the ELC-E program committee: he reviewed, ranked all talk submitted for the conference and participated with the rest of the committee to the selection of the talks that are now scheduled for the event.
Even though we once again won’t have the chance to meet our fellow members of the embedded Linux community in person, we look forward to attending a set of great talks, and have interesting discussions during the Q&A and through the instant messaging platform that will be available around the conference.
Since December 1st, 2020, we’re happy to have in our team an additional engineer, Thomas Perrot, who joined our office in Toulouse, France.
The