Back from ELC North-America: selection of talks from the Bootlin team

As discussed in our previous blog post, Bootlin had again a strong presence at the Embedded Linux Conference North-America, with 8 attendees, 5 talks, one BoF and two E-ALE tutorial sessions.

In this blog post, we would like to highlight a number of talks from the conference that we found interesting. Each Bootlin engineer who attended the conference has selected one talk, and gives his/her feedback about this talk.

Device Tree BoF – Frank Rowand

Talk selected by Michael Opdenacker

The Device Tree BoF (Birds of a Feather session, which means an informal session about a technical topic, allowing participants to openly share questions and information) has been part of Embedded Linux Conferences for at least 2 or 3 years. For me, it has always been a good source of updates about the topic.

Frank started by sharing details about the Device Tree Workshop held in October in Prague, a one day meet-up and workshop for Device Tree contributors (like Maxime Ripard and Thomas Petazzoni from Bootlin who were invited), to address issues and plan for the next months. Slides and notes can be found on elinux.org.

Frank then went on by mentioning utilities, such as:

  • scripts/dtc/dt_to_config. It is not very new in the mainline kernel, but useful to generate a kernel configuration suitable for the devices present on your platform, in case you didn’t know this tool exists.
  • There’s an upcoming patch adding options to dtc (--annotate --full) to keep track of the line numbers in the device tree sources. This helps to locate in which DT source file a given property value comes from. The patch was idle for some time but Julia Lawall volunteered to take care of it. Thanks to her updates, this feature should be accepted in mainline soon.
  • The device tree compiler in mainline has also been augmented with further build checks. You can now use them by adding W=1 to make dtb. Here is an example for the Beagle Bone Black dtb:
make W=1 am335x-boneblack.dtb
  CHK     scripts/mod/devicetable-offsets.h
  DTC     arch/arm/boot/dts/am335x-boneblack.dtb
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/i2c@44e0b000/tda19988 has a reg or ranges property, but no unit name
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/i2c@44e0b000/tda19988/ports/port@0 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/i2c@44e0b000/tda19988/ports/port@0/endpoint@0 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/ethernet@4a100000/slave@4a100200 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/ethernet@4a100000/slave@4a100300 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (unit_address_vs_reg): Node /ocp/lcdc@4830e000/port/endpoint@0 has a unit name, but no reg property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/prcm@200000/clocks missing or empty reg/ranges property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/prcm@200000/clockdomains missing or empty reg/ranges property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/scm@210000/scm_conf@0/clocks missing or empty reg/ranges property
arch/arm/boot/dts/am335x-boneblack.dtb: Warning (simple_bus_reg): Node /ocp/l4_wkup@44c00000/scm@210000/clockdomains missing or empty reg/ranges property

As far as I am concerned, the most interesting news remained the one that since Linux 4.15, device tree overlays are now easier to code. You no longer have to define weird “fragment” elements. You can now directly write normal nodes and use labels. The syntax is now exactly the same as for regular device tree sources!

For more details, grab the slides and if you event want to follow the discussions that happened that day, watch the video.

Tutorial: Introduction to Reverse Engineering – Mike Anderson

Talk selected by Quentin Schulz

Mike presented in an unusual 2-hour-long slot the different techniques to reverse-engineer things and the different reasons why you’d do so. After a mandatory disclaimer that reverse engineering may be illegal in some regions of the world, he introduced the different tools that anyone aspiring to reverse engineer should possess: from the obvious logic analyzer, multimeter, screwdrivers to the surprising heat gun. He then gave the first and very important step of the reverse engineering process: gathering information about the product by looking for patents, the FCC registration, manufacturer as well as carefully opening its case to examine the different components (maybe with the help of a microscope).

Later, Mike gave the multiple ways to retrieve the firmwares from the product, from the soldering of a JTAG interface to the downloading from the official website. He then offered some tools that can be used to dive into binaries and start the guessing game, and he finished his talk with an example of a reverse engineering of a protocol which required a lot of guessing and social reverse engineering.

Mike’s talk was pleasant to attend because of the high-level presentation of how to do reverse engineering while giving a quick real-life example.

For more details, watch the video and grab the slides.

Graphics Performance Analysis with FrameRetrace: A Responsive UI for ApiTrace – Mark Janes, Intel

Talk selected by Boris Brezillon

I first heard of FrameRetrace when Eric Anholt asked us to add support for the VC4 GPU to this tool, and my experience with it had been rather frustrating in that I was mainly struggling to make it work on a not yet supported architecture instead of being a simple user. So, when I saw that Mark was giving a talk on FrameRetrace usefulness and how to use it, I figured I couldn’t miss it. Well, those who looked carefuly at the schedule know I couldn’t attend it because I was giving my talk at the same time, but I did see it at FOSDEM a month before, and I’m pretty sure not much has changed since then.

Mark first described the GPU debugging/perf-anlysis tools ecosystem, saying that each GPU vendor has its own proprietary tools which most of the time are only supported on Windows. FrameRetrace is an attempt at providing a tool that exposes similar features while being open-source, cross-platform, and easily extensible to new hardware. This project is actually based on an exisiting project called ApiTrace, which it uses to capture OpenGL traces. The new feature that is interesting in FrameRetrace is that you can select the frame you want to replay, get all the hardware perf counters exposed by the GPU for this specific rendering job in order to figure out what is going wrong and then play with the OpenGL code to see how you can make things better and replay the rendering job with your local modification to see if it actually solves the problem.

I must admit I was really impressed by the demo, and now I understand why Eric (and others in the community) would like to have their GPU properly supported in FrameRetrace. It really looks like the kind of tool you don’t know you need until you’ve tested it, but once you do, you can’t do without it.

For more details, watch the video and grab the slides.

The Salmon Diet: Up-Streaming Drivers as a Form of Optimization – Gilad Ben-Yossef

Talk selected by Miquèl Raynal

Gilad was hired about a year ago to become the maintainer of the ARM® TrustZone® CryptoCell® device driver. Until now this driver was out of tree until it has been decided to upstream it. Here starts Gilad’s story.

It appeared that the right way to make it upstream was to go through the staging tree and the whole process around it. It was the first time for him to do it that way, that is why he felt it was interesting to share his experience.

At the beginning of his talk, he recalls that the driver was actually working, people already relied on it. Plus, it was released under the GPL. While all of this could make you think it was clean enough, Gilad realized that people who wrote it actually did not think about upstreaming and almost every patch to clean that driver removed more lines than it added, shrinking step by step the driver until 30% of the lines were removed!

Of course, removing the existing hardware abstraction macros was something to do, as well as running and correcting the whole checkpatch.pl output, but there are plenty of other good habits that one can adapt to his own situation, explained and well illustrated all along this talk.

For more details, watch the video and grab the slides.

Measuring and Summarizing Latencies using the Trace Event Subsystem – Tom Zanussi

Talk selected by Maxime Chevallier

Having some experience dealing with RT topics on Linux, I was looking forward to seeing Tom’s talk about these tracing features.

He gave really good examples on how to use the existing ‘latency histogram’ traces to get a cyclictest-like metric of wakeup latencies by measuring the time between sched_waking and sched_switch, and explaining how this could be re-used for other measurements such as network latencies.

What he presented was more than just having a trace in a buffer when a function is called. The tracing subsystem allows the use of handlers to perform actions when an event occurs. As an example, he demonstrated how to use the onmax handler to accumulate the maximum wakeup latency observed, each time saving crucial pieces of information on the execution context.

He then went on to describe the next-level features that are being merged, namely function events by Steven Rostedt, and Inter events by Tom himself. They allow the user to use any of the kernel functions as tracepoints, and build complex events and traces to pinpoint really specific sequences.

I recommend to read this LWN article on inter-event tracing, and of course have a look at Tom’s talk and slides.

Steering Xenomai into the Real-Time Linux Future – Jan Kiszka

Talk selected by Thomas Petazzoni

In this talk, long-term Xenomai developer and Siemens engineer Jan Kiszka gave a very interesting status of the Xenomai project and its roadmap. He started by refreshing the audience about what Xenomai is: an RTOS-to-Linux portability framework. It comes in two flavors: a co-kernel extension for a patched Linux kernel, and as libraries running for native Linux (including PREEMPT_RT). He then went on to compare the respective advantages and drawbacks of the two flavors, citing accurate modeling of legacy RTOS behavior and strong separation of real-time vs. non real-time code as the key advantages of the co-kernel approach.

Jan then summarized the history of Xenomai, from the early days as a sub-project of RTAI to the current status of Xenomai 3.0, released in 2015 after more than 5 years of development. However, even though Xenomai is widely used in the industry, its development relies on just a few individuals. Siemens is a heavy user of Xenomai, and in 2017, they started a discussion: should they migrate away from Xenomai or invest into it. Siemens made the decision to invest in the project. The same year, Xenomai main developer Philippe Gerum published an e-mail RTnet, Analogy and the elephant in the room also calling for help to maintain some parts of Xenomai.

Following those discussions, some changes were decided in the Xenomai project: Philippe Gerum will step back from the project lead, and Jan Kiszka will take over his role.

Regarding the I-Pipe kernel patch (which allows to support the co-kernel approach), the Xenomai project will discontinue a number of architectures (nios2, SH, Blackfin, PPC64, ARM < v7) and will only maintain patches for the latest Linux kernel LTS, in order to reduce the maintenance effort. Jan announced that Xenomai 3.0.7 is soon to be released, that Xenomai 3.1 will introduce ARM64 support, and that Xenomai 2 is unmaintained and therefore users should migrate to Xenomai 3. He also gave a status on the driver stacks, citing that RTnet needs more love, and that Analogy is orphaned and needs a new maintainer. Towards the end of his talk, Jan then started discussing the more distant future of Xenomai. The future version of Xenomai has the goal of improving the integration of the co-kernel approach, to simplify maintenance and possibly provide a chance to be upstreamed in Linux. This new approach will be split in two elements, called Dovetail (interrupt routing, co-kernel hooks) and Steely (co-kernel implementation). He made it clear that this is currently not production-ready at all. He noted that this new implementation allows a significant reduction of the code base, about 50% smaller than the current implementation. The code is already available in two Git repositories: Dovetail and Steely.

All in all, Jan’s talk was a very interesting one, providing a good coverage of Xenomai’s status and future. The video of his talk is definitely worth watching, and the slides are also available.

An Introduction to Asymmetric Multiprocessing: When this Architecture can be a Game Changer and How to Survive It – Nicola La Gloria & Laura Nao

Talk selected by Mylène Josserand

In this talk, Nicola La Gloria and Laura Nao from Kynetics presented how to handle communication between a micro-controller running bare metal code and a CPU with a full OS (such as GNU/Linux or Android).

They showed the different approaches for communication (supervised or not supervised: i.e. CPU and MCU can communicate using an hypervisor or directly) and presented the Inter-Processing Communication.

After this introduction, Laura told us about their use-case, running on an NXP i.MX7, which comprises a Cortex-M4 micro-controller and a Cortex-A7 processor: the MCU retrieves data from a sensor, which will be displayed by the CPU.
She gave feedback on how they implemented this communication and the different mechanisms used (Message Unit, RPMsg, RDC, kexec/kdump, etc). The explanation of the different mechanisms was really interesting, and particularly relevant for those who had never heard about them.

They did a short tutorial and gave some tips that would definitely be appreciated by people who start this kind of project. And finally, they did a demonstration of all the work they have done.

So if you are interested in the subject or even only for your general knowledge, have a look at their talk (video and slides).

System-in-Package Technology: Making it Easier to Build Your Own Linux Computer – Erik Welsh & Jason Kridner

Talk selected by Alexandre Belloni

Eric Welsh started to talk about how software influences hardware design and why open source hardware matters. He then presented the System-in-Package technology and in particular the Octavo OSD3358. It includes a TI AAM335x SoC, DDR3 SDRAM, a PMIC and all the related power circuitry, components which are always required. This allows the hardware engineers to concentrate on the added value of the final product.

Great pictures and videos of the SiP internals and its manufacturing process were shown.

Jason then came on stage to present the OSD3358 integration on the PocketBeagle.

Eric finally explained how easy it is to assemble the OSD3358 on a PCB, even by hand with a video to prove it. He finally concluded by summing up the benefits of using a SiP: easy bring-up, lower cost of PCB, easy manufacturing and migration from SBC prototyping to custom PCB.

It was quite enlightening for software engineers as it showed the hardware internals with some great details.
Have a look at the video and slides.

In conclusion, this was again a really nice opportunity to share and acquire knowledge from other engineers deeply involved in the Open Source community, as well as meeting people that we sometimes know only by their name on a mailing list. Next year this event will happen in Monterey Bay, California (March 19 – 21, 2019). See you there!

Allwinner VPU support in mainline Linux status update (week 13)

While this week’s goal was set to supporting dmabuf in our Sunxi-Cedrus VAAPI implementation with GStreamer, progress was made on GStreamer support alone. The first milestone was displaying the video from videotestsrc, which produces a sample test output, to kmssink, which handles video output directly with DRM planes.

The Gstreamer pipeline for the test
The working result on display

This required specific adaptation for GStreamer to cope with the Allwinner DRM driver. More specifically, changes that allow the Allwinner DRM driver in kmspipe as well as devices that don’t sit on a PCI bus in gtreamer-vaapi were submitted. Patches to fix both these issues were sent for review earlier today and the first one was already accepted and merged. Thanks to Nicolas Dufresne for his help in the process!

On the VAAPI side, things are more complicated and a number of different areas of the sunxi-cedrus VAAPI backend had to be reworked. For instance, the slices constituting MPEG2 frames are not submitted in the same way by VLC and GStreamer. Buffers management is also done slightly differently between these two users of libVA. The net result is that a segmentation fault caused by a memory management mishaps is occurring with GStreamer and is still being investigated.

This week was also the occasion to dig into the partial reference code that Allwinner published in 2015 (to grasp a better understanding of the MPEG2 decoding process with the VPU) and start updating the register documentation on the linux-sunxi wiki, where some fields documented by the reference code were still marked as unknown. The Sunxi-Cedrus page was also updated to reflect the current status and effort. These resources will get updated as development happens. More precisely, we’d like to provide instructions to deploy this work, once it has reached a decent level of usability. Stay tuned for our next update!

Kickstarter for the Allwinner VPU Linux kernel support successfully funded!

Early February, we announced the launch of our first Kickstarter campaign, whose goal was to add support for the video decoding unit found in most Allwinner processors to the official Linux kernel.

We were very pleased to see that in just one day, enough companies and individuals participated to fund the main goal of the campaign, collecting more than 17,600 EUR. And now that the campaign is over since March 18, we are even more pleased, as we have reached a funding level sufficient to cover not only our main goal, but also our two first stretch goals.

Thanks to the participation of all those companies and individuals, we will be able to:

  1. Deliver our main goal, which we expect to deliver by the end of June 2018, which includes:
    • Making sure that the codec works on the older Allwinner SoCs that are still widely used: A10 (Cubieboard), A13 (A13-Olinuxino), A20 (Cubieboard 2, A20-Olinuxino), A33 (A33-Olinuxino, BananaPi M2-Magic), R8 (CHIP) and R16 (NES and Super NES classic). Support for the newer SoCs (H3, H5 and A64) requires more work, and is part of our first stretch goal below.
    • Polishing the existing MPEG2 decoding support to make it fully production ready.
    • Implementing H264 video decoding, since H264 is by far one of the most popular video codec.
    • Modifying the Allwinner display driver in order to be able to directly display the decoded frames instead of converting and copying those frames, which is very inefficient from a CPU consumption point of view
    • Providing a user-space library easy to integrate in the popular open-source video players
    • Upstreaming those changes to the official Linux kernel
  2. Deliver our two first stretch goals, which we expect to deliver by the end of 2018, which includes:
    • Supporting the newer Allwinner SoCs, such as the H3 (Most of the Orange Pis, Nano Pi M1, ..), H5 (Orange Pi PC2, NanoPi NEO2, …) and A64 (Pine64, BananaPi M64, …).
    • H265 video decoding support

Unfortunately, our third stretch goal, which consisted in adding support for H264 encoding was not reached, so we don’t know yet if we will have enough time to look into this topic.

Maxime Ripard and Paul Kocialkowski
Maxime Ripard and Paul Kocialkowski working on Allwinner VPU support

However, the work on the other topics has already started, with our intern Paul Kocialkowski has started to work since March 1st on this Allwinner VPU effort, and Maxime Ripard has started this week. We have already published, will continue to publish a report every week: week 10, week 11 and week 12. You can follow the progress of this project by reading our blog, our Twitter feed or the Kickstarter updates. You can also read the Sunxi-cedrus Wiki page to get all the details about this project and its progress.

Beyond the specific topic of the Allwinner VPU support, we are very happy to see that the model of funding upstream Linux kernel work through crowd-funding has worked. Most Kickstarter projects, in exchange for the participants contribution, provide to the participant a specific product (a book, a device) that only benefits to the contributor. Here, the result of this campaign will be shared freely with everybody, both Kickstarter contributors and non-contributors, and we’re proud to see that our experience has convinced numerous companies and individuals to support our project. Of course, we will be organizing in the near future the shipment of the t-shirts as well as the beer drinking sessions with Bootlin engineer Maxime Ripard.

To conclude, we would like to thank all our participants (we’re naming only the ones who baked at a level above 16 EUR, as above this level contributors are going to be mentioned in the CREDITS file, which clarifies their intent to be publicly named). First a number of companies supported our work: OrangePi, Libre Computer, neutis.io, FriendlyArm, Pine64, Olimex and of course a huge number of individuals: Abe Lacker, Adam Morris, Adam Oberbeck, Alerino Reis, Alex, Alexander A. Istomin, Alexander Kamm, Alexandru Nedel, Alex Kaplan, Amarpreet Minhas, André, Andreas, Andreas Färber, Andreas Rozek, Andre Przywara, Andrew Langley, Angel Rua Amo, Anssi Kolehmainen, Antony, Appreciation of Efforts, Aron Somodi, Artur Huhtaniemi, Atsushi Sasaki, Bastien Nocera, Bavay, Benjamin Glass, Benjamin Larsson, Ben Young, Bernard D’Havé, Bert Lindner, Bert Vermeulen, BESSIERES MARC, Biji-san, Bob Black, brot, Bruce Shipman, Butterkeks, Carla Sella, Carl Wall, Carsten Tolkmit, cbrocas, chae, Charlie Bruce, Christian Gnägi, Christian Pellegrin, Christian Stalp, Christophe Vaillot, Christoph Kröppl, Conan Kudo (ニール・ゴンパ), D1don, Dale Cousins, Daniel, Daniel Hrynczenko, Daniel Kulesz, Daniel Mühlbachler, David Pottage, David Willmore, defsy, Denis Bodor, DESSARD Guy, Dimitrios Bogiatzoules, Dominique Dumont, Doyle Young, Dubouil, Eelco Wesemann, eineki, Emil Karlson, Emmanuel Fusté, Erdem MEYDANLI, Eric des Courtis, Eric Jensen, Eric Koorn, Éric Périé, Erik, erikf, Evaryont, Fabian Korak, Felix Eickeler, Flo, Florian Beier, Florian Kempf, Frank, Frank van Kesteren, Frederir, G40, Gabor, Gabriel Ortiz, Garrett Gee, Georg Ottinger, Gerald Hochegger, Geralt, ghostpatch, Gianpaolo Macario, Giulio Benetti, Guenther Gassner, Guilhem, Guilhem Saurel, hackman, Hamish, Hanno Helge, Hans-Frieder Vogt, Heinz Thölecke, Henrik Kuhn, hook, Hugh Reynolds, Ian Daniher, iav, Inapplicable, Ingo Strauch, Ioan Rogers, Irvel Nduva, James, James Cloos, James Valleroy, jan koopmanschap, Jared Smith, Jari-Matti, Jarkko Pöyry, Jasper Horn, jean, Jean-Pierre Rivière, Jeffrey Sites, Jernej, Jerome Hanoteau, JK, John Kelley, Johnny Sorocil, Juanjo Marin, Jussi Pakkanen, Justin Ross, Karl Palsson, Kazım Rıfat Özyılmaz, Kean, Kevin Fowlks, Kevin Read, kicklix, Kiesel, Koen Kooi, Korbinian Probst, kratz00, Laurent GUERBY, Lee Donaghy, liushuyu, Logicite, luigi, Lukas Schauer, lzrmzz, Maksims Matjakubovs, Manuel, Marcel Sarge, Marcus Cooper, Mario Villarreal, Mark Dietzer, Markus Härer, Martijn Bosgraaf, mateuszkj, Mathias Brossard, mathieu, Matsumoto Kenichi, Matthew Zhang, Matthias, Matthias Lamm, Matt Mets, Maxime Brousse, Me, MESNIL Mikaël, Michael Gregorowicz, Michael Thalmeier, Michal Zatloukal, Mirko Vogt, mouren, N/A, naguirre, Neil Davenport, Nick Crasci, Nick Richards, Oleksij Rempel, oliver, Oliver Heyme, OSAKANA TARO, othiman, ozcoder, Pablo, Patrick, Paul Philippov, Paul Sykes, Per Larsson, Peter Gnodde, Peter Robinson, Philip-Dylan Gleonec, Phipli, Phoenix Chen, Pierce Lopez, Priit Laes, Prisma, Rainer Stober, Reignier, René Kliment, Reto Haeberli, Ricardo Salveti de Araujo, Richard Cote, Richard Ferlazzo, Riku Voipio, Robert Lukierski, Robert McQueen, roens, Rui Gu, Ryan Casey, Salvatore Bognanni, Samuel Frederick, Scott Devaney, Sebastian Krzyszkowiak, Sébastien DA ROCHA, Sergey Kopalkin, Sertac Tulluk, Shelby Cruver, Shervin Emami, SIMANCAS, Simon Josefsson, Spas Kyuchukov, ssam, Stan, Stanislav Bogatyrev, Stas, Stefan Bethke, Stefan Monnier, Steffen Elste, Stephan, Stephan Bärwolf, Stephen Kelly, Steven Seifried, Stokes Gresh, Sven Kasemann, SvOlli, Tarjei Solvang Tjønn, Tetsuyuki Kobayashi, Texier Pierre-jean, Thomas Monjalon, Thomas Samson, Tim Symossek, Todd Zebert, Tomas Virgl, tpc010, Tyler Style, Valentin Hăloiu, valhalla, Vasily Evseenko, Vitaly Shukela, Xavier Duret, Yannick Allard, Yves Serrano, Zoltan Herpai, ZotoPatate, zym060050.

Thanks everybody!

Bootlin toolchains updated, 2018.02 version

Back in June last year, we launched a new service that provides pre-compiled and ready-to-use cross compilation toolchains for a large number of CPU architectures and C library configurations. They are available from toolchains.bootlin.com.

We have recently updated those toolchains, with the following improvements:

  • Our stable family of toolchains has been updated in terms of components versions: we’re using gcc 6.3.0, binutils 2.29.1, gdb 7.11.1, kernel headers 4.1, glibc 2.26, musl 1.1.18 and uClibc 1.0.28
  • Our bleeding-edge family of toolchains has also been updated in terms of components versions: we’re using gcc 7.3.0, binutils 2.30, gdb 8.0.1, kernel headers 4.9.80, glibc 2.27, musl 1.1.8 and uClibc 1.0.28
  • The tarballs now have a more nice-looking version number, and the version number is also included in the directory after extracting the tarball
  • Qemu testing of the PowerPC64 little-endian configuration was added

Toolchains.bootlin.com

Next in our TODO list are:

  • Adding an ARMv7 Thumb-2 configuration
  • Adding more PowerPC variants, as suggested by this issue and this pull request
  • Enabling Python support in gdb, as suggested by this issue

We remain interested in getting your feedback about those toolchains, so do not hesitate to contact us at info@bootlin.com, or through the Github issue tracker.

Back from ELC North America: slides and videos of Bootlin talks

Mid-March of this year, 8 engineers from Bootlin attended the Embedded Linux Conference North-America in Portland, Oregon. We had a strong presence at this conference with 5 talks, one BoF and two E-ALE tutorial sessions.

Bootlin team at ELC 2018 from left to right: Quentin Schulz, Alexandre Belloni, Boris Brezillon, Mylène Josserand, Miquèl Raynal, Maxime Chevallier, Thomas Petazzoni and Michael Opdenacker

In this first blog post about ELC 2018, we want to share the slides and videos of the talks we gave during the conference.

Buildroot: What’s new? – Thomas Petazzoni

Buildroot is a popular and easy to use embedded Linux build system. Within minutes, it is capable of generating lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs.

After a short introduction about Buildroot, this talk will go through the numerous new features and improvements that have appeared in the last few years, and show how they can be useful for developers, users and contributors.

This talk is an updated version of the one given at ELCE 2017.

Slides [PDF], Slides [LaTeX source]

Drive your NAND with Linux – Miquèl Raynal

NAND flash chips are almost everywhere, sometimes hidden in eMMCs, sometimes they are just parallel NAND chips under the orders of your favorite NAND controller. Each NAND vendor follows its own rules. Each SoC vendor creates his preferred abstraction for interacting with these chips.

Handling all of that requires some abstraction, and that is currently being enhanced in Linux! A new interface, called exec_op is showing up. It has been designed to match the most diverse situations. It should ease the support of advanced controllers as well as the implementation of vendor-specific NAND flash features.

This talk will start with some basics about NAND memories, especially their weaknesses and how we get rid of them. It will also show how the interaction between NAND chips and controllers has been standardized over the years and how it is planned to drive NAND controllers within Linux.

Slides [PDF], Slides [LaTeX source]

Secure Boot from A to Z – Quentin Schulz & Mylène Josserand

Based on our complementary experience on building a secure system on an i.MX6 custom board, we’ll present how to build a complete chain-of-trust for a platform.

This talk will introduce each and every link of the chain-of-trust from the boot ROM to filesystem, as well as the bootloader and kernel with real life examples.

We’ll go through everything needed from the signing of binaries (U-Boot and kernel) to the secured automation of kernel booting within the bootloader, the use of dm-verity and switchroot for securing the filesystem, and more.

Slides [PDF], Slides [LaTeX source]

I + I2C = I3C: What’s in this Additional ‘I’? – Boris Brezillon

The MIPI Alliance recently released version 1 of the I3C (pronounce ‘eye-three-see’) bus specification, which is supposed to be an improvement over the long-standing I2C and SPI protocols. Compared to I2C/SPI, I3C provides a higher data rate, lower power consumption and additional features such as dynamic address assignment, host join, in-band interrupts. For the last year or so, Free Electrons has been working with Cadence Design Systems on supporting this new kind of bus in Linux.

With this talk we would like to introduce this new bus and the concepts it brings to the table. We will also detail how we plan to expose the new features exposed by the I3C protocol in Linux and go through future possible improvements of the I3C framework that has already been submitted for review on the Linux kernel mailing list.

Slides [PDF], Slides [LaTeX source]

Introduction to Linux Kernel Driver Programming: i2c drivers – Michael Opdenacker

For people new to Linux kernel driver programming, writing a driver for an I2C device is a relatively easy way to start. This presentation will start by explaining the Device Model, the mechanism that the Linux kernel offers to bind drivers to devices. Even though the way to detect or describe devices can depend on the bus or CPU architecture, the infrastructure binding devices with drivers is universal and therefore applies to all types of device drivers in the Linux kernel. You will see how the driver uses one of the frameworks offered by the Linux kernel to expose device data to user space in a generic way. Once again, this type of mechanism is used everywhere in the Linux kernel.

Michael presented this topic as part of the E-ALE track, we’ll update this blog article once the recording is available to embed the video.

Slides [PDF], Slides [OpenOffice]

BoF: Embedded Linux Size – Michael Opdenacker

This “Birds of a Feather” session will start by a quick update on available resources, patches and recent work to reduce the size of user-space and of the Linux kernel (in particular the efforts from Nicolas Pitre).

An ARM based system running the mainline kernel with about 3 MB of RAM will also be demonstrated.

If you are interested in the size topic, please join this BoF and share your experience, the resources you have found and your ideas for further size reduction techniques! This BoF will build upon the one run at the latest Embedded Linux Conference in Europe.

We’ll update this blog article once the recording is available to embed the video.

Getting Started with Buildroot – Thomas Petazzoni

Need to create simple and optimized Linux systems for your embedded devices? Tired of complicated tools? You should try Buildroot!

In this tutorial, we will first introduce Buildroot, a popular embedded Linux build system, that allows you to build your own cross-compilation toolchain, Linux kernel and bootloader images, as well as root filesystem with your selection of user-space libraries and applications, all from an easy-to-use “menuconfig” interface.

Slides [PDF], Lab [PDF], Lab & slides [LaTeX source]

Thomas presented this topic as part of the E-ALE track, we’ll update this blog article once the recording is available to embed the video.

Ethernet Switch Support in the Linux Kernel – Alexandre Belloni

Hardware Ethernet switches are appearing on more SoC families and can take care of many network functionalities like VLAN tagging, IGMP snooping, link aggregation,… Linux is able to offload network processing to those switches using the switchdev and the DSA APIs.

This talk will introduce the Ethernet switches and their typical features, the Linux switchdev and DSA APIs and their differences. It will also give an overview of sample implementations and how to use the features from userspace.

Slides [PDF], Slides [LaTeX source]

Allwinner VPU support in mainline Linux status update (week 12)

Following up on the work started last week, I finished implementing initial support for displaying the NV12-based tiled format (that we shall call MB32-tiled NV12). The frame, that was dumped from the VPU, is now correctly displayed on the screen (after adapting scaling coefficients that needed specific tweaking for this use case).

The result can be shown in the following picture, where our Big Buck Bunny has the right coloring:

Scaling is also supported for the tiled format, so the frame can be shown in full screen without resorting to software scaling.

A series of patches supporting these features was sent for review on the dri-devel mailing list, where it already got some feedback from Maxime Ripard (who maintains the sun4i DRM driver impacted by these patches) as well as other members of the community! There is already enough material to craft a second version and send it again for review.

Significant time was spent figuring out the DRM, KMS, DRI and X11 graphics pipeline (as well as specific details of the inner workings of display hardware) and how to properly integrate the overlay DRM plane with all this. We are evaluating all our options here before spending time on a specific implementation. Of course, we are trying to keep things as generic as possible and avoid introducing platform-specific code in userspace, but there are also challenges to overcome in this regard. On the Wayland side, things are looking much brighter as compositors such as Weston have support for managing hardware planes directly, so there should be less work required.

Finally, I started working on dmabuf support, that I am testing with gstreamer‘s kmssink, that allows outputting directly in a hardware plane. Once this work is ready, we’ll be able to get an idea of the performance of the VPU when it is not limited by software-based untiling and compositing. Stay tuned for updates in this direction!

Allwinner VPU support in mainline Linux status update (week 11)

After the initial submission of the Sunxi-Cedrus driver last week, I spent most of this week looking into the sun4i DRM (Direct Rendering Manager) driver. The driver is in charge of handling the display pipeline on Allwinner SoCs. Tight integration of the VPU and the display pipeline is required in order to achieve decent video playback performance. That is because the output format of the VPU is a 32×32 tiled format based on NV12, a YUV420 semi-planar format, with one plane for the Y component (luminance) and one plane for the interleaved UV components (chrominance). While NV12 is a standard format for video output, the tiling is rather specific to the VPU, so the frames have to be untiled before they can be used. This operation, when done in software, is rather slow. Moreover, software-based compositing of the decoded frames is also a bottleneck that impacts the overall performance.

In order to circumvent these issues, we will be using the display engine itself to untile the VPU output frames and show the untiled frames directly in a dedicated hardware plane, that is then composed with the primary plane. This requires several features and especially support for the display engine’s frontend, that has the required components to untile and decode the frames. Partial support for the frontend was recently contributed by Maxime Ripard and is on its way to landing in the mainline Linux kernel, providing a base for my VPU-related work. Maxime’s patches allow scaling hardware planes (among other things), a feature that will be very useful for scaling videos to the screen size in hardware rather than software (which is another major bottleneck for performance).

Support for untiling the VPU frames is approaching completion (luminance is correctly decoded while chrominance is not yet correctly handled).

Decoding the MB32 tiled format with sun4i-drm

Once the frames are properly shown on screen, it’ll be time to make sure that dmabuf works as expected, which will allow us to send buffers from the VPU to the display engine without any copy, thus improving performance.

We should be making good progress on this topic over the upcoming week and start contributing patches to the sun4i DRM driver, so stay tuned for our next status update!

Linux 4.15 released, Bootlin contributions

Penguin from Mylène Josserand
Drawing from Mylène Josserand,
based on a picture from Samuel Blanc under CC-BY-SA

After a month of February busier than usual, with the renaming of our company from Free Electrons to Bootlin, our participation to FOSDEM and the welcoming of Maxime Chevallier, the latest addition to our engineering team, our article on the latest release of the Linux kernel arrives a bit late, more than a month after Linux 4.15 has been released by Linus Torvalds.

As usual, LWN.net did an interesting coverage of this release cycle merge window, highlighting the most important changes: The first half of the 4.15 merge window and The rest of the 4.15 merge window. Due to the now well-known Spectre and Meltdown vulnerabilities and the resulting effort to try to mitigate them, 4.15 required a -rc9, which happened the last time back in 2011 with the 3.1, Torvalds said.

According to Linux Kernel Patch statistics, Free Electrons (now Bootlin) contributed 150 patches to this release, making it the 16th contributing company by number of commits.

The main highlights of our contributions are:

  • In the RTC subsystem, Alexandre Belloni made a number of improvements to various drivers, mainly making them use the nvmem subsystem where appropriate, and use the recently introduced rtc_register_device() API.
  • In the MTD subsystem, both Boris Brezillon and Miquèl Raynal made a number of contributions, mainly fixes.
  • For Marvell platforms
    • Antoine Ténart contributed a few fixes to the >inside-secure crypto accelerator driver, used on Marvell Armada 3700 and Armada 7K/8K
    • Antoine Ténart also contributed fixes and improvements to the mvpp2 network driver, used for the Ethernet controller on the Marvell Armada 7K/8K. His improvements include preparation work to support Receive Side Scaling (RSS).
    • Antoine Ténart enabled more networking ports and features in some Armada 7K/8K boards, especially SFP ports on Armada 7040 DB and Armada 7040 DB.
    • Boris Brezillon contributed a few fixes to the Marvell CESA crypto accelerator driver, used on the older Orion, Kirkwood, Armada 370/XP/38x processors. He migrated the driver to use the skcipher interface of the Linux kernel crypto framework.
    • Grégory Clement enabled NAND support on Armada 7K, and contributed a number of fixes around MMC support for some Marvell boards.
    • Thomas Petazzoni contributed a few minor Device Tree enhancements for Marvell platforms: fixing MPP muxing on an older Kirkwood platform, enabling more PCIe ports on Armada 8040 DB, etc.
    • Miquèl Raynal contributed support for more advanced statistics in the mvpp2 network driver.
    • Miquèl Raynal added support for the extended UART for the Marvell Armada 3720 processor, both in the UART driver and in the Device Tree.
  • For the RaspberryPi platform, Boris Brezillon contributed a few fixes to the vc4 display driver, and added support for the new DRM_IOCTL_VC4_GEM_MADVISE ioctl, which can be used to ask the userspace applications to purge inactive buffers when allocations start to fail in the kernel.
  • For Allwinner platforms
    • Mylène Josserand contributed a fix for the Allwinner A83 clock driver, fixing I2C bus clocks.
    • Quentin Schulz contributed a few fixes to the sun4i-gpadc-iio.c driver, which is used for the ADCs on several Allwinner processors.
    • Maxime Ripard made a number of fixes to the sun8i-codec driver, fixing clock issues, left/right channels inversion, etc.
    • Maxime Ripard made a number of improvements to the sun4i DRM display driver.
    • Maxime Ripard improved the support for the A83 processor (described the UART1 controller, the MMC1 controller, added support for display clocks) and added the Device Tree for a new A83 device.
    • Maxime Ripard also did a number of cleanups and misc improvements in a significant number of Device Tree files for Allwinner platforms.
  • Thomas Petazzoni made a few fixes to the sh_eth network driver, used on several Renesas SuperH platform, as part of a recent project Bootlin did on SuperH 4.

Bootlin engineers are not only contributors, but also maintainers of various subsystems in the Linux kernel, which means they are involved in the process of reviewing, discussing and merging patches contributed to those subsystems:

  • Maxime Ripard, as the Allwinner platform co-maintainer, merged 108 patches from other contributors
  • Boris Brezillon, as the MTD/NAND maintainer, merged 34 patches from other contributors
  • Alexandre Belloni, as the RTC maintainer and Atmel platform co-maintainer, merged 50 patches from other contributors
  • Grégory Clement, as the Marvell EBU co-maintainer, merged 24 patches from other contributors

Here is the commit by commit detail of our contributons to 4.15:

Allwinner VPU support in mainline Linux status update (week 10)

Just over a week ago, I started my internship focused on adding upstream Linux kernel support for the Allwinner VPU at Bootlin’s Toulouse office. The team has been super-friendly and very helpful to help me get settled and I’m definitely happy about moving to Toulouse for the occasion!

This first week of work was focused on studying and rebasing the work done by Florent Revest a year and a half ago. As a main development target, I went for an A33-based board, the SinA33 from Sinlinx. Florent’s patches for the sunxi-cedrus driver were rebased against the latest release candidate version of Linus’ tree, v4.16-rc4.

VPU decoding with Cedrus on the Sinlinx A33

The driver was then adapted to use the latest version of the V4L2 request API, a crucial piece of plumbing needed to provide coherency between setting specific controls for the media stream and the input/output buffers that these controls are related to. A few bugs needed fixing along the way, in order to avoid memory corruptions (use-after-free) and to properly schedule the VPU to run when a request is submitted. With these fixes the driver was ready, so it was sent for review on the linux-media mailing list. On the userspace side, the cedrus-specific libva was also updated to use the latest version of the request API.

The next step in the pipeline is to use a common buffer for the VPU’s decoded frame and the display controller’s plane, using dmabuf. This should bring a significant performance improvement and eventually allow for hardware-based scaling when decoding videos through the standard DRM/KMS interfaces. However, this requires adding support for the specific format used by the VPU (a multiplanar NV12 format with 32×32 tiles) into the display controller code.

Bootlin contributes a new interface to the Linux NAND subsystem

MTD stack

Over the last months, Bootlin engineers Boris Brezillon and Miquèl Raynal have been working on rewriting the NAND controller driver used on a large number of Marvell SoCs. This NAND controller driver had grown very complicated, and Miquèl’s adventure in this rework led him to contribute a new interface to the NAND framework, in order to simplify implementing NAND controller drivers for complex NAND controllers. In this blog post, Miquèl summarizes the original issue, and how it is solved by the ->exec_op() interface he has contributed.

Introduction

The NAND framework is the layer between the generic MTD layer and the NAND controller drivers. Its purpose is to handle MTD requests and transform them into understandable NAND operations the controller will have to send to the NAND chip.

For general information about NANDs, the reader is invited to read the ONFI specification (Open NAND Flash Interface) which defines the most common NAND operations.

Interacting with a NAND chip

Raw NANDs (so-called “parallel NANDs”) are slave devices waiting for instructions from the controller. An operation is a sequence of instructions usually referred as “command” (CMD), “addresses” (ADDR), and “data” cycles (DATA_IN/DATA_OUT) and sometimes wait periods (WAITRDY). Some everyday operations any NAND enthusiast should know by heart are, for instance:

NAND operation example

How it was handled in the Linux kernel

Today, a majority of NAND controlller drivers implement the ->cmd_ctrl() hook. It aimed to be a very small function, designed to just send command and address cycles independently, usually embedding some very controller-specific logic. This hook was supposed to be called by a function of higher level from the NAND core, ->cmdfunc(). In addition to calling ->cmd_ctrl() to send command and address cycles, the core would also call ->read|write_byte|word|buf() hooks to actually move data from the NAND controller and the memory (the DATA parts in the diagram above).

This approach worked very well with simple NAND controllers, which are just able to send command and address cycles one at a time to the NAND chip, without any extra intelligence. However, NAND controllers have become more and more complex and now can handle higher-level operations, usually to provide higher performance. For example, a NAND controller may provide an operation that would do all of the command and address cycles of a read-page operation in one-go. Some controllers even support only those higher-level operations, and are not able to simply do the basic operation of sending one command cycle or one data cycle. To handle such controllers, their drivers were overloading the ->cmdfunc() hook directly, circumventing the generic NAND core implementation of ->cmdfunc(). This is a first drawback: it is no longer possible to easily add logic to the NAND core to support new NAND operations, because some drivers overload the ->cmdfunc() logic. Worse, ->cmdfunc() doesn’t provide some information such as the length of the data transfer, which some controllers actually need in order to run the desired operation. NAND controller drivers started to have complicated state machines just to work around the NAND framework limitations.

NAND stack before exec_op

Some driver-specific implementations of this hook started diverging from the original one, giving maintainers a lot of pain to maintain the whole subsystem, specifically when they needed to introduce additional vendor-specific operations support. These implementations were not only diverse but also incomplete, sometimes buggy and most importantly, developers had to guess the data that would probably be moved by the core after that, which is clearly a symptom that the framework was not fitting the user needs anymore.

The ->exec_op() era

The NAND subsystem maintainers decided to switch to a new approach, based on a new hook called ->exec_op(), implemented by NAND controller drivers and called by the generic NAND core. The logic behind that name is to provide to every controller a generic interface that can easily be extended and exposes the overall NAND operation to be performed. This way, the driver can optimize depending on the controller capabilities without the need of a complex state machine as ->cmdfunc() was.

All major NAND generic raw operations like reset, reading the NAND ID, selecting a set of timings, reading/writing data and so on found their place into small internal functions named nand_[operation]_op().

From the NAND controller driver point of view, an array of instructions is received for each operation. The controller then needs to parse these instructions, decides if it can handle the overall operation, splits the operation if needed, and executes what is requested.

Using the ->exec_op() interface is as simple as declaring a list with the controller capabilities, each entry of this array having a callback function knowing the overall operation that will actually handle all the logic. The NAND core was enhanced with a proper parser that one may use in his driver to handle the callback selection logic.

NAND stack with exec_op

For a more complete overview, one can check the slides and the video of Miquèl’s presentation at FOSDEM about NAND flash memories and the introduction of ->exec_op() in the Linux kernel.

Current status

The ->exec_op() interface in the NAND core has been accepted and merged upstream, and will be part of Linux 4.16. The first driver converted to this new interface was obviously the NAND controller driver used on Marvell platforms, pxa3xx_nand. It has been rewritten as marvell_nand, and will also be part of Linux 4.16. Even though the new driver is longer (by lines of code) than the previous one, it supports additional features (such as raw read and write operations), allows the NAND core to pass custom commands to the NAND chip, and has a logic that is a lot less complicated.

Miquèl has also worked on converting the fsmc_nand driver to ->exec_op(), but this work hasn’t been merged yet. In the community, Stefan Agner has taken on the task to convert the vf610_nfc driver to this new approach.

Bootlin is proud to have contributed such enhancements to the Linux kernel, and hopes to see other developers contribute to this subsystem in the near future, by migrating their favorite NAND controller driver to ->exec_op()!