Crowdfunding Campaign
It has been over a year since we launched a crowdfunding campaign to fund the development of an upstream Linux kernel driver and userspace support for the Allwinner VPU. The funding campaign was a frank success, with over 400 backers contributing a total of over 30 k€ out of the 17,6 k€ set as the initial goal. This enabled us to work on additional stretch goals, namely support for new Allwinner SoCs and H.265 decoding support.
Initial Development
Work on the Allwinner VPU started back in March 2018, being the main topic of my 6-month internship at our office in Toulouse. Bootlin engineer and long-time Allwinner Linux kernel maintainer Maxime Ripard rapidly joined the effort hands-on, to bring up support for H.264 decoding. Aspects covered by our effort include the kernel driver (cedrus), VAAPI userspace library (libva-v4l2-request) as well as testing tools (v4l2-request-test, libva-dump) and various upstream projects such as VLC and GStreamer.
We worked hard to deliver the campaign’s goals and submitted numerous revisions of the base cedrus driver along the way. By July, we announced the delivery of the campaign’s main goals (although some goals were not fully met, as explained in the associated blog post) and accompanied it with a release (tagged release-2018-07 in our git repositories).
By the end of August, we had added support for MPEG-2, H.264 and H.265 for first-generation Allwinner SoCs (and the H3 in addition), including support for accelerated display of decoded frames in the DRM driver. See our detailed blog post presenting the status at that point. Still, our changes had yet to be included in the Linux kernel.
End of the Year Status
We kept working intermittently on VPU support over the following months and manged to get the Cedrus driver accepted in Linux at the same time as the media request API. We also continued to work on submitting new versions of the series adding H.264 and H.265 support to our driver. Last but not least, we worked on adding support for the H5, A64 and A10 platforms, which were missing from the initial delivery. A dedicated blog post presents the status at the end of 2018.
Recent Developments
In 2019, Bootlin has been continuing the effort to maintain the driver and get the remaining patch series integrated in the mainline Linux kernel. We managed to get the remaining patches for DRM support merged and they will be included in Linux 5.1!
Regarding codecs support, there are still discussions happening around the H.264 and H.265 series which are now at their sixth and third revisions respectively. We are hoping that the situation will settle and that these series will be merged (in staging) as soon as possible.
March 2019 Release
With modifications taking place in the (unstable) kernel interface and userspace being updated accordingly, it became quite hard for users to properly pick the kernel and userspace components that work together. Because of that, we decided to make a new release (tagged release-2019-03 in our git repositories). It packs an updated kernel tree (based on the next media tree with our ongoing patch series applied atop) and matching versions of libva-v4l2-request and v4l2-request-test.
External Contributions
We received a few contributions along the way, such as support for the H6 SoC in the cedrus driver (that should make it to Linux 5.2) and a few minor fixes for the driver. We also received and reviewed improvement to our v4l2-request-test testing tool.
Even though not a backer (sorry) thanks to Jernej im an enthusiastic user of your efforts in the form of Jernej’s LE for my (til then abandoned) Beelink X2. It’s already now the best working player for zattoo. I’m really impressed. This effort demonstrates once again what an advantage an upstream kernel is for an SOC.
I heard that tiled support is still a problem and the modificeren needs patching, as well as FFMPEG.
What si the status of that?
When using our VAAPI backend with ffmpeg, it should be quite transparent: our backend will either do software untiling or do direct buffer sharing (DRM Prime/dma-buf), where the tiled modifier is properly passed. For the native ffmpeg implementation from https://github.com/Kwiboo/FFmpeg/tree/v4l2-request-hwaccel, it is quite likely that the tiled format is not properly handled indeed. Note that this native port is work-in-progress and will change in the future. Still, it would be great to have support for our tiled format by the time the implementation hits upstream ffmpeg!
Some work on supporting tiled formats was already done. But biggest problem is actually lack of hardware as neither me nor Kwiboo have any board with DE1.
Nice work!
Are you also working on the encoding part of the VPU?
That would be very nice.