Wrapping up the Allwinner VPU crowdfunded Linux driver work

Back in early 2018, Bootlin started a crowd-funding campaign to fund the development of an upstream Linux kernel driver for the VPU found in Allwinner processors. Thanks to the support from over 400 contributors, companies and individuals, we have been able to bring support for hardware-accelerated video decoding in the mainline Linux kernel for Allwinner platforms.

From April 2018 to end of 2019, Paul Kocialkowski and Maxime Ripard at Bootlin worked hard on developing the driver and getting it accepted upstream, as well as developing the corresponding user-space components. We regularly published the progress of our work on this blog.

As of the end of 2019, we can say that all the goals defined in the Kickstarter have been completed:

  • We have an upstream Linux kernel for the Allwinner VPU, in drivers/staging/media/sunxi/cedrus, which supports MPEG2 decoding (since Linux 4.19), H264 decoding (since Linux 5.2) and H265 decoding (will be in the upcoming Linux 5.5)
  • We have a user-space VA-API implementation called libva-v4l2-request, and which allows to use any Linux kernel video codec based on the request API.
  • We have enabled the Linux kernel driver on all platforms we listed in our Kickstarter campaign: A13/A10S/A20/A33/H3 (since Linux 4.19), A64/H5 (since Linux 4.20), A10 (since Linux 5.0) and H6 (since Linux 5.1, contributed by Jernej Skrabec)

This means that the effort that was funded by the Kickstarter campaign is now over, and from now on, we are operating in maintenance mode regarding the cedrus driver: we are currently not actively working on developing new features for the driver anymore.

The Cedrus Linux driver in action

Of course, there are plenty of additional features that can be added to the driver: support for H264 encoding, support for high-profile H264 decoding, support for other video codecs. Bootlin is obviously available to develop those additional features for customers, do not hesitate to contact us if you are interested.

Overall, we found this experience of funding upstream Linux kernel development through crowd-funding very interesting and we’re happy to have been successful at delivering what was promised in our campaign. Looking at the bigger picture, the Linux userspace API for video decoding with stateless hardware codecs in V4L2 has been maturing for a while and is getting closer and closer to being finalized and declared a stable kernel API: this project has been key in the introduction of this API, as cedrus was the first driver merged to require and use it. Additional drivers are appearing for other stateless decoding engines, such as the Hantro G1 (found in Rockchip, i.MX and Microchip platforms) or the rkvdec engine. We are of course also interested in working on support for these VPUs, as we have gained significant familiarity with all things related to hardware video decoding during the cedrus adventure.

Back from ELCE 2019: our talks videos, slides, and more!

With 8 engineers participating to the Embedded Linux Conference Europe, almost the entire Bootlin engineering team took part to the conference. As usual, we not only attended the event, but also contributed by giving a total of 5 talks and 2 tutorials, for which we’re happy to share below the videos and slides. Also, as part of this conference, Bootlin CTO Thomas Petazzoni received an award for his contribution to the conference.

Buildroot, what’s new ?

Talk given by Thomas Petazzoni, slides in PDF and slides source code.

Timing boot time reduction techniques

Talk given by Michael Opdenacker, slides in PDF, slides source code.

Integrating hardware-accelerated video decoding with the display stack

Talk given by Paul Kocialkowski, slides in PDF, slides source code.

RTC subsystem, recent changes and where it is heading

Talk given by Alexandre Belloni, slides in PDF, slides source code.

Flash subsystems status update

Talk given by Miquèl Raynal (from Bootlin) and Richard Weinberger (from sigma star gmbh), slides in PDF, slides source code.

Offloading network traffic classification to hardware

Talk given by Maxime Chevallier, slides in PDF, slides source code.

Introduction to Linux Kernel driver programming

Tutorial given by Michael Opdenacker, slides in PDF, slides source document. The video is not yet available, but should be published in the future.

Introduction to the Buildroot embedded Linux build system

Tutorial given by Thomas Petazzoni, slides in PDF, slides source code. The video is not yet available, but should be published in the future.

Award to Thomas Petazzoni

During the traditional closing game of the conference, we were really happy to have Bootlin’s CTO Thomas Petazzoni called on stage, to receive from the hands of Tim Bird, an award for his continuous 11 year participation to the conference, with 24 presentations given, one keynote and for the past two years, participation to the conference program committee. We are honored and proud by this recognition of Thomas contribution to the conference.

Thomas Petazzoni receives ELCE conference award

Thomas Petazzoni receives ELCE conference award

Thomas Petazzoni receives ELCE conference award

Slides and videos from the Embedded Linux Conference Europe 2016

Last month, the entire Bootlin engineering team attended the Embedded Linux Conference Europe in Berlin. The slides and videos of the talks have been posted, including the ones from the seven talks given by Bootlin engineers:

  • Alexandre Belloni presented on ASoC: Supporting Audio on an Embedded Board, slides and video.
  • Boris Brezillon presented on Modernizing the NAND framework, the big picture, slides and video.
  • Boris Brezillon, together with Richard Weinberger from sigma star, presented on Running UBI/UBIFS on MLC NAND, slides and video.
  • Grégory Clement presented on Your newer ARM64 SoC Linux check list, slides and video.
  • Thomas Petazzoni presented on Anatomy of cross-compilation toolchains, slides and video.
  • Maxime Ripard presented on Supporting the camera interface on the C.H.I.P, slides and video.
  • Quentin Schulz and Antoine Ténart presented on Building a board farm: continuous integration and remote control, slides and video.

Support for the Allwinner VPU in the mainline Linux kernel

Over the last few years, and most recently with the support for the C.H.I.P platform, Bootlin has been heavily involved in initiating and improving the support in the mainline Linux kernel for the Allwinner ARM processors. As of today, a large number of hardware features of the Allwinner processors, especially the older ones such as the A10 or the A13 used in the CHIP, are usable with the mainline Linux kernel, including complex functionality such as display support and 3D acceleration. However, one feature that was still lacking is proper support for the Video Processing Unit (VPU) that allows to accelerate in hardware the decoding and encoding of popular video formats.

During the past two months, Florent Revest, a 19 year old intern at Bootlin worked on a mainline solution for this Video Processing Unit. His work followed the reverse engineering effort of the Cedrus project, and this topic was also listed as a High Priority Reverse Engineering Project by the FSF.

The internship resulted in a new sunxi-cedrus driver, a Video4Linux memory-to-memory decoder kernel driver and a corresponding VA-API backend, which allows numerous userspace applications to use the decoding capabilities. Both projects have both been published on Github:

Currently, the combination of the kernel driver and VA-API backend supports MPEG2 and MPEG4 decoding only. There is for the moment no support for encoding, and no support for H264, though we believe support for both aspects can be added within the architecture of the existing driver and VA-API backend.

A first RFC patchset of the kernel driver has been sent to the linux-media mailing list, and a complete documentation providing installation information and architecture details has been written on the linux-sunxi’s wiki.

Here is a video of VLC playing a MPEG2 demo video on top of this stack on the Next Thing’s C.H.I.P:

Videos of XDC2014 and Kernel Recipes 2014

Recently, two interesting conferences took place in France: the X.org developer conference (in Bordeaux, October 8th-10th) and the Kernel Recipes conference (in Paris, September 25th-26th).

X.org Foundation logo

Kernel Recipes logo

Both conferences have now published videos and slides of the different talks:

  • for the XDC 2014 conference, they are available in the program page
  • for the Kernel Recipes conference, they are available from the schedule page

It also means that the video of the talk given by Bootlin engineer Maxime Ripard about the support for Allwinner processors in the kernel is now available: video, slides.

https://www.youtube.com/watch?v=yJmZkzq651Q