Allwinner VPU support in mainline Linux status update (week 27)

This week, significant time was dedicated to preparing a new revision of the Sunxi-Cedrus VPU kernel driver. This new version (that was started last week) based on version 15 of the media requests API brought about a number of challenges. First off, integrating the recently-tested VPU-side untiling of the destination buffers required a significant rewrite of the part in charge of managing formats and buffers. The part of our driver handling V4L2 controls (that are used to submit the frame metadata) was also significantly reworked to allow validating that the frame metadata has indeed been submitted by userspace before launching a decode run. An initial implementation of this was brought up and discussed with V4L2 maintainer Hans Verkuil, who is backing (and baking) the requests API series. He came up with a specific patch that should allow properly implementing this detection at the right time (when checking the media request’s validity, instead of at the start of the run). Hans also solved various reliability issues that we were experiencing when using the requests API with our driver. As a result, he posted version 16 of the requests API series with these fixes. We are hoping that this version will be one of the final iterations of this long-awaited series!

While rebasing H264 support, we experienced a strange issue where the destination buffers were sometimes corrupted and sometimes not. All the hardware configuration (register writes) were exactly the same, except for the buffer addresses (that naturally tend to change depending on allocations order in the related CMA memory pool). After some investigation, we discovered that when the gap between the luma and chroma planes of the destination buffer are too distant, a corruption happens. It may be that some offset is used in the hardware at some point and that it is not coded on enough bits to represent a large gap. The way to work around this is to make sure that all the planes of our destination buffer are allocated contiguously. In practice, this means that we need a single allocation for the each whole destination buffer (with the size of its two planes), ensuring that there is no gap between the planes.

A random bug in the development of Sunxi-Cedrus with funky colors

The work has continued on H264, and especially to add support for the High Profile decoding. My test video showed a limitation in our current code however, due to what seems to be a limitation of the libva API. Indeed, the H264 codec relies on a decoded picture buffer (DPB) that holds the previous decoded pictures that might be used as reference frames to decode the current frame. The kernel interface needs that DPB, and our driver will also need it to perform some ID assignation for the current frame. However, libva only gives the list of frames needed to decode the current frame, and not the whole DPB. That leads to a situation where subsequent frames, using the same reference frames set, will be assigned the same ID, which obviously doesn’t work very well. Most of the week has been spent trying to evaluate how we can address that issue, and to start implementing a solution that would be based on a cache of the reference frames passed to our libva driver.

Author: Paul Kocialkowski

Paul is a kernel and embedded Linux engineer at Bootlin, which he joined in 2018. See More details...

One thought on “Allwinner VPU support in mainline Linux status update (week 27)”

  1. If the decoded picture buffer contains the reference frame, then this frame is never used by the application as it still needs to go through the deblocking filter. It could live entirely in the kernel as is done in the coda driver.

Leave a Reply