The Linux Display Next Hackfest, graciously hosted by Igalia in the beautiful city of A Coruña, was an event that brought together a diverse group of individuals from various projects and companies, all with a common interest in the world of Linux graphics.
The event was a buzzing hive of discussions, with the primary focus being on Direct Rendering Manager (DRM), compositors and an array of topics related to graphics in Linux. Representatives from Fedora, Mutter, KWin, AMD, Nvidia, Google, and many others were present, lending their expertise and insights to the conversations.
From May 14 to May 16, Igalia is organizing the 2024 Display Next Hackfest, an event where talented developers will gather to explore the latest technologies and trends in the Linux Display Stack.
As explained on the event website:
It has an unconference format where participants propose topics for presenting, roadmapping, discussing and examining together. It aims to unblock bottlenecks, design solutions, raise pitfalls and accommodate the needs of each layer of the display stack. Participants should feel free to propose any topic which interests them. Some topics from the previous edition include: HDR and color management, frame timing and variable refresh rate (VRR), atomic flips, testing and CI, etc.
Bootlin engineer Louis Chauvet, who has started contributing to the Linux kernel VKMS driver, and is starting to work on IGT and the latest version of the Chamelium CI testing hardware, will participate to this hackfest, together with many developers from Igalia, Redhat, Intel, Google, RaspberryPi, AMD, ARM, Collabora and more. This will allow us to discuss current developments and topics, and meet the relevant developers of the Linux graphics/display community.
LogicBricks is a vendor of numerous IP blocks, ranging from display controllers, audio controllers, 3D accelerators and many other specialized IP blocks. Most of these IP blocks are designed to work with the Xilinx Zynq 7000 system-on-chip, which includes an FPGA area. And indeed, because the Zynq 7000 does not have a display controller, one of Bootlin customers has selected the LogicBricks logiCVC-ML IP to provide display support for their Zynq 7000 design.
LogiBricks provide one driver based on the framebuffer subsystem and another one based on the DRM subsystem, but none of these drivers are in the upstream Linux kernel. Bootlin engineer Paul Kocialkowski worked on a clean DRM driver for this IP block, and submitted the first version to the upstream Linux kernel. We already received some useful comments on the Device Tree binding for this IP block, which is pretty elaborate due to the number of aspects/features that can be tuned at IP synthesis time, and we will of course take into account those comments and send new iterations of the patch series until it gets merged.
In the e-mail containing the driver patch itself, Paul gives a summary of the IP features that are supported and tested, and those that re either untested or unsupported:
Introduces a driver for the LogiCVC display controller, a programmable
logic controller optimized for use in Xilinx Zynq-7000 SoCs and other
Xilinx FPGAs. The controller is mostly configured at logic synthesis
time so only a subset of configuration is left for the driver to
handle.
The following features are implemented and tested:
- LVDS 4-bit interface;
- RGB565 pixel formats;
- Multiple layers and hardware composition;
- Layer-wide alpha mode;
The following features are implemented but untested:
- Other RGB pixel formats;
- Layer framebuffer configuration for version 4;
- Lowest-layer used as background color;
- Per-pixel alpha mode.
The following features are not implemented:
- YUV pixel formats;
- DVI, LVDS 3-bit, ITU656 and camera link interfaces;
- External parallel input for layer;
- Color-keying;
- LUT-based alpha modes.
Additional implementation-specific notes:
- Panels are only enabled after the first page flip to avoid flashing a
white screen.
- Depth used in context of the LogiCVC driver only counts color components
to match the definition of the synthesis parameters.
Support is implemented for both version 3 and 4 of the controller.
With version 3, framebuffers are stored in a dedicated contiguous
memory area, with a base address hardcoded for each layer. This requires
using a dedicated CMA pool registered at the base address and tweaking a
few offset-related registers to try to use any buffer allocated from
the pool. This is done on a best-effort basis to have the hardware cope
with the DRM framebuffer allocation model and there is no guarantee
that each buffer allocated by GEM CMA can be used for any layer.
In particular, buffers allocated below the base address for a layer are
guaranteed not to be configurable for that layer. See the implementation of
logicvc_layer_buffer_find_setup for specifics.
Version 4 allows configuring each buffer address directly, which
guarantees that any buffer can be configured.
We have been working with Raspberry Pi for quite some time, especially on areas related to the display side. Our work is part of a larger ongoing effort to move away from using the VC4 firmware for display operations and use native Linux drivers instead, which interact with the hardware directly. This transition is a long process, which requires bringing the native drivers to a point where they are efficient and reliable enough to cover most use cases of Raspberry Pi users.
Continuous Integration (CI) plays an important role in that process, since it allows detecting regressions early in the development cycle. This is why we have been tasked with improving testing in IGT GPU Tools, the test suite for the DRM subsystem of the kernel (which handles display). We already presented the work we conducted for testing various pixel formats with IGT on the Raspberry Pi’s VC4 last year. Since then, we have continued the work on IGT and brought it even further.
Improving YUV and Adding Tiled Pixel Formats Support
We continued the work on pixel formats by generalizing support for YUV buffers and reworking the format conversion helpers to support most of the common YUV formats instead of a reduced number of them. This lead to numerous commits that were merged in IGT:
In the meantime, we have also added support for testing specific tiling modes for display buffers. Tiling modes indicate that the pixel data is laid out in a different fashion than the usual line-after-line linear raster order. It provides more efficient data access to the hardware and yields better performance. They are used by the GPU (T tiling) or the VPU (SAND tiling). This required introducing a few changes to IGT as well as adding helpers for converting to the tiling modes, which was done in the following commits:
The display engine hardware used on the Raspberry Pi allows displaying multiple framebuffers on-screen, in addition to the primary one (where the user interface lives). This feature is especially useful to display video streams directly, without having to perform the composition step with the CPU or GPU. The display engine offers features such as colorspace conversion (for converting YUV to RGB) and scaling, which are usually quite intensive tasks. In the Linux kernel’s DRM subsystem, this ability of the display engine hardware is exposed through DRM planes.
Displaying multiple DRM planes
We worked on adding support for testing DRM planes with the Chamelium board, with a fuzzing test that selects randomized attributes for the planes. Our work lead to the introduction of a new test in IGT:
With the Chamelium, there are two major ways of finding out whether the captured display is correct or not:
Comparing the captured frame’s CRC with a CRC calculated from the reference frame;
Comparing the pixels in the captured and reference frames.
While the first method is the fastest one (because the captured frame’s CRC is calculated by the Chamelium board directly), it can only work if the framebuffer and the reference are guaranteed to be pixel-perfect. Since HDMI is a digital interface, this is generally the case. But as soon as scaling or colorspace conversion is involved, the algorithms used by the hardware do not result in the exact same pixels as performing the operation on the reference with the CPU.
Because of this issue, we had to come up with a specific checking method that excludes areas where there are such differences. Since our display pattern resembles a colorful checkerboard with solid-filled areas, most of the differences are only noticeable at the edges of each color block. As a result, we introduced a checking method that excludes the checkerboard edges from the comparison.
Detecting the edges (in blue) of a multi-plane pattern
This method turned out to provide good results and very few incorrect results after some tweaking. It was contributed to IGT with commit:
We also worked on implementing display pipeline underrun detection in the kernel’s VC4 DRM driver. Underruns occur when too much pixel data is provided (e.g. because of too many DRM planes enabled) and the hardware can’t keep up. In addition, a bandwidth filter was also added to reject configurations that would likely lead to an underrun. This lead to a few commits that were already merged upstream:
We prepared tests in IGT to ensure that the underruns are correctly reported, that the bandwidth protection does its job and that both are consistent. This test was submitted for review with patch:
From May 14 to May 16, 
