Announcing buildroot-external-st, Buildroot support for STM32MP1 platforms

STM32MP1 Discovery Kit 2Back in 2019, ST released a brand new processor family, the STM32MP1, whose members are currently based on a dual Cortex-A7 to run Linux combined with one Cortex-M4 to run bare-metal applications, together with a wide range of peripherals.

Following the release of this new platform, Bootlin ported its Embedded Linux and Yocto training courses to be available on STM32MP1, and also published a long series of tutorials showing how to use Buildroot to build an embedded Linux system on STM32MP1: part 1, part 2, part 3, part 4, part 5, part 6 and part 7.

We are happy to announce that we have partnered with ST to develop an improved support of Buildroot on STM32MP1, which is materialized by a Buildroot BR2_EXTERNAL available on Github at https://github.com/bootlin/buildroot-external-st. In Buildroot, a BR2_EXTERNAL is an extension of the core Buildroot, with additional configurations and/or packages.

This BR2_EXTERNAL tree is an extension of Buildroot 2021.02, which provides four example Buildroot configuration to easily get started on STM32MP1 platforms:

  • st_stm32mp157a_dk1, building a basic Linux system for the STM32MP1 Discovery Kit 1 platform, with a minimal Linux root filesystem
  • st_stm32mp157c_dk2, building a basic Linux system for the STM32MP1 Discovery Kit 2 platform, with a minimal Linux root filesystem
  • st_stm32mp157a_dk1_demo, building a much more featureful Linux system for the STM32MP1 Discovery Kit 1 platform, with Linux root filesystem that allows to run Qt5 applications with OpenGL acceleration on the HDMI output, that supports audio, demonstrates the usage of the Cortex-M4, uses OP-TEE, and more.
  • st_stm32mp157c_dk2_demo, building a much more featureful Linux system for the STM32MP1 Discovery Kit 2 platform, with Linux root filesystem that allows to run Qt5 applications with OpenGL acceleration on both the integrated DSI display and HDMI output, that supports audio, WiFi and Bluetooth, demonstrates the usage of the Cortex-M4, uses OP-TEE, and more.

This BR2_EXTERNAL is designed to work with Buildroot 2021.02, with only a small set of modifications, which since then have been integrated in upstream Buildroot, ensuring that STM32MP1 users can directly use upstream Buildroot for their projects.

There is extensive documentation on how to use this BR2_EXTERNAL tree as well as how to test the various features of the STM32MP1 platform: using STM32CubeProgrammer, using Device Tree generated from STM32 CubeMX, using the Cortex-M4, testing display support, using Qt5, using WiFi, using Bluetooth, using audio and using OP-TEE. We have also documented the internals of the BR2_EXTERNAL components.

This Buildroot support is using the latest software components from the recently released 3.1 BSP from ST (see release notes), so it is based on Linux 5.10, U-Boot 2020.10, TF-A 2.4 and OP-TEE 3.12. We will keep this BR2_EXTERNAL updated with newer releases of the ST BSP.

If you face any issue while using this Buildroot support for STM32MP1, you can use the issue tracker of the Github project, or use the ST community forums. Bootlin can also provide commercial support on Linux on STM32MP1 platforms, as well as training courses.

Embedded Linux Conference 2021 schedule published, 4 talks from Bootlin

The schedule for the Embedded Linux Conference 2021 has been published and features 4 talks proposed by Bootlin !

This year, the ELC will take place in Seatle but will be organised as a hybrid virtual/physical event  due to the pandemic.  As usual the ELC will have a really interesting schedule with 46 talks covering a wide range of topics: build system, kernel graphics, boot process, security, etc.

See below the details of Bootlin talks that will be presented as virtual talks.

Advanced Camera Support on Allwinner SoCs with Mainline Linux – Paul Kocialkowski, Bootlin

Capturing pixels with a camera involves a number of steps, from the ADC reading the photosites in the image sensor to the final pixel values that are ready for encode/display, with various processing and transmission taking place along the way. While simple cases put most of the heavy lifting on the image sensor’s side (through its embedded processor) and use a simple parallel bus for transmission, advanced cases require more work to be done outside of the sensor. In addition, modern high-speed transmission buses also bring-in more complexity. This talk will present how support for such an advanced use case was integrated into the mainline Linux kernel, using the Media and V4L2 APIs. It involves supporting a sensor using the raw Bayer RGB format, transmission over the MIPI CSI-2 bus as well as support for the Image Signal Processor (ISP) found on Allwinner platforms. A specific focus will be set on this ISP, with details about the features it implements as well as the internal and userspace APIs that are used to support it. The integration between all of the involved components will also be highlighted.

Talk given by Paul Kocialkowski, at 4:50 PM PDT on September 27, 2021. See this talk in the schedule.

Embedded Linux Nuggets found in Buildroot Package Eldorado

To this date, Buildroot supports more than 2,500 packages, selected for the ability to run them on embedded Linux systems. We’ve gone exploring this Eldorado, and came back with multiple nuggets of all shapes and colors. Join this playful presentation and as if you were still a new comer to the embedded Linux community, discover lesser known tools and resources that can add to the functionality of your systems or make your life as a developer easier and more fun. Whenever possible, each resource will be shown through a quick demonstration or video capture. During this talk, I’ll also open an Etherpad for all participants to share their favorite solutions with the rest of the audience, especially the ones that deserve to be better known, and could be worth supporting in Buildroot too. We will close the session by an open review and discussion based on the nuggets shared by the audience.

Michael Opdenacker

Talk given by Michael Opdenacker, at 12:00 PM PDT on September 28, 2021. See this talk in the schedule.

I3C in Tomorrow’s Design

I3C is the new bus specification by the MIPI Alliance. While being compatible with I2C devices, this bus brings a colorful set of new features such as dynamic address assignment, in-band interrupts, hot-join, master handover and many others. It was improved once again recently with the 1.1 version of the specification which brought timer based sampling synchronization and targeted reset. All this make the I3C bus a good candidate for a number of new situations compared to its I2C cousin. It is then more and more being included in new hardware designs. With this talk we would like to propose a reminder of the various components and concepts of this relatively new bus. We will then detail how it is implemented in the Linux kernel with a short guided tour in the I3C core. Since the previous talk on I3C in 2018 by Boris Brezillon, I3C has now become a reality and starts to become available in real hardware designs. This talk will recap the basics of I3C as well as add details of the 1.1 specification and improvements in the Linux support.

Miquèl Raynal

Talk given by Miquèl Raynal, at 4:00 PM PDT on September 28, 2021. See this talk in the schedule.

OP-TEE: When Linux Loses Control

OP-TEE is an open-source Trusted Execution Environment designed to be executed in a secure context as a companion to a non secure Linux system. But what happens to the peripherals control since OP-TEE can forbid the non-secure OS to access them ? When running with a TEE, Linux isn’t in charge anymore of some critical peripherals and relies on the TEE to access and configure them. There are multiple protocols and methods to access these peripherals that are supported by Linux (SCMI, PSCI, SMC). Supporting them for a SoC requires understanding the various interactions between the systems and how to modify them to fit that new control scheme. Additionally, the configuration must be passed from OP-TEE to Linux to allow a seamless integration. This talk will cover the boot process to start a secure system and the modifications needed to run Linux when OP-TEE is in charge of some peripherals. The work that has been done for a specific SoC will be described to have a tangible real-world use-case.

Clément Léger

Talk given by Clément Léger, at 12:00 PM PDT on September 29, 2021. See this talk in the schedule.

Slides and videos of Bootlin talks at Live Embedded Event #2

The second edition of Live Embedded Event took place on June 3rd, exactly 6 months after the first edition. Even though there were a few issues with the online platform, it was once again great to learn new things about embedded, and share some of the work we’ve been doing at Bootlin on various topics. For the next edition, we plan to switch to a different online platform, hopefully providing a better experience.

But in the mean time, all videos of the event have been posted on the Youtube Channel of the event. The talks from Bootlin have been posted on Bootlin’s Youtube Channel.

Indeed, in addition to being part of the organization committee, Bootlin prepared and delivered 5 talks as part of Live Embedded Event, covering different topics we have worked on in the recent months for our customers.

Understanding U-Boot Falcon Mode and adding support for new boards, Michael Opdenacker

Slides [PDF]

Introduction to RAUC, Kamel Bouhara

Slides [PDF]

Security vulnerability tracking tools in Buildroot, Thomas Petazzoni

Slides [PDF]

Secure boot in embedded Linux systems, Thomas Perrot

Slides [PDF]

Device Tree overlays and U-boot extension board management, Köry Maincent

Slides [PDF]

Bootlin “Buildroot system development” course updated to Buildroot 2021.02

Bootlin has been offering for several years a Buildroot system development course, which allows engineers interested in learning and understanding the Buildroot embedded Linux build system to get up to speed very quickly.

In preparation for our public Buildroot system development course next week, we updated our training materials, both slides and labs to Buildroot 2021.02, which is the latest stable Buildroot release as of today, and is also a Long Term Support release.

Buildroot slide

In addition to updating to a newer Buildroot version, we also use newer U-Boot and Linux versions for the practical labs on BeagleBone Black Wireless. The slides were also updated to document some new features that appeared between 2020.02 and 2021.02. If you’re interested, check out the materials on the training page.

We have one seat left for this training course next week, which will be taught by long-time Buildroot contributor and developer Thomas Petazzoni. Register now and take the last seat!

Bootlin toolchains integration in Buildroot

Since 2017, Bootlin is freely providing ready-to-use pre-built cross-compilation toolchains at https://toolchains.bootlin.com/. We are now providing over 150 toolchains, for a wide range of CPU architectures, covering the glibc, uClibc-ng and musl C libraries, with up-to-date gcc, binutils, gdb and C library support.

We recently contributed an improvement to Buildroot that allows those toolchains to very easily be used in Buildroot configurations: the Bootlin toolchains are now all known by Buildroot as existing external toolchains, next to toolchains from other vendors such as ARM, Synopsys and others.

If you are building a Buildroot system for a CPU architecture variant that has a matching toolchain available from bootlin.toolchains.com, then Bootlin toolchains will naturally show up in the Toolchain sub-menu, when the selected Toolchain type is External toolchain. For example, if the selected CPU architecture is ARM little endian Cortex-A9, with VFP you will see:

Bootlin toolchain selection

Once Bootlin toolchains is selected, a new sub-option Bootlin toolchain variant appears, which allows to choose between the different toolchains applicable to the selected CPU architecture:

Bootlin toolchain choice

This hopefully should make Bootlin toolchains easier to use for Buildroot users.

Internally, this support for Bootlin toolchains in Buildroot is generated and updated using the support/scripts/gen-bootlin-toolchains script. In addition to making the toolchains available to the user, it allows generates some Buildroot test cases for each toolchain, so that each of those configuration is tested by Buildroot continuous integration, see support/testing/tests/toolchain/test_external_bootlin.py.

Bootlin toolchains 2020.08 released

Bootlin toolchainsWe are happy to announce a new release of the freely available cross-compilation toolchains we provide at toolchains.bootlin.com, version 2020.08-1.

Here are the main changes compared to our previous 2020.02 release:

  • Bleeding edge toolchains are now using: gcc 10.2, binutils 2.34, gdb 9.2, kernel headers 5.4, glibc 2.31, musl 1.2.0, uclibc-ng 1.0.34
  • Stable toolchains are using: gcc 9.3, binutils 2.33, gdb 8.3, kernel headers 4.9, glibc 2.31, musl 1.2.0, uclibc-ng 1.0.34
  • Fortran support has been enabled in all tolchains
  • Several new CPU architecture variants are supported, each with a new toolchain
  • Boot testing in Qemu was added for PowerPC64 E5500, NIOSII and m68k MCF5208.

In addition, it is worth mentioning that all those Bootlin toolchains are now directly accessible from Buildroot: make menuconfig shows the Bootlin toolchains available for the current selected CPU architecture, and Buildroot is able to automatically download and use the toolchain. This feature will be available starting from Buildroot 2020.11:

Thanks again to the entire Buildroot community, and especially Romain Naour, for all the fixes and improvements related to toolchain support that make this project possible. In the next weeks, we hope to be able to deliver further updated bleeding-edge toolchains, with glibc 2.32 and binutils 2.35. Stay tuned!

If you face any issue, or need additional features in those toolchains, do not hesitate to report an issue in our issue tracker.

Covid-19: Bootlin proposes online sessions for all its courses

Tux working on embedded Linux on a couchLike most of us, due to the Covid-19 epidemic, you may be forced to work from home. To take advantage from this time confined at home, we are now proposing all our training courses as online seminars. You can then benefit from the contents and quality of Bootlin training sessions, without leaving the comfort and safety of your home. During our online seminars, our instructors will alternate between presentations and practical demonstrations, executing the instructions of our practical labs.

At any time, participants will be able to ask questions.

We can propose such remote training both through public online sessions, open to individual registration, as well as dedicated online sessions, for participants from the same company.

Public online sessions

We’re trying to propose time slots that should be manageable from Europe, Middle East, Africa and at least for the East Coast of North America. All such sessions will be taught in English. As usual with all our sessions, all our training materials (lectures and lab instructions) are freely available from the pages describing our courses.

Our Embedded Linux and Linux kernel courses are delivered over 7 half days of 4 hours each, while our Yocto Project, Buildroot and Linux Graphics courses are delivered over 4 half days. For our embedded Linux and Yocto Project courses, we propose an additional date in case some extra time is needed to complete the agenda.

Here are all the available sessions. If the situation lasts longer, we will create new sessions as needed:

Type Dates Time Duration Expected trainer Cost and registration
Embedded Linux (agenda) Sep. 28, 29, 30, Oct. 1, 2, 5, 6 2020. 17:00 – 21:00 (Paris), 8:00 – 12:00 (San Francisco) 28 h Michael Opdenacker 829 EUR + VAT* (register)
Embedded Linux (agenda) Nov. 2, 3, 4, 5, 6, 9, 10, 12, 2020. 14:00 – 18:00 (Paris), 8:00 – 12:00 (New York) 28 h Michael Opdenacker 829 EUR + VAT* (register)
Linux kernel (agenda) Nov. 16, 17, 18, 19, 23, 24, 25, 26 14:00 – 18:00 (Paris time) 28 h Alexandre Belloni 829 EUR + VAT* (register)
Yocto Project (agenda) Nov. 30, Dec. 1, 2, 3, 4, 2020 14:00 – 18:00 (Paris time) 16 h Maxime Chevallier 519 EUR + VAT* (register)
Buildroot (agenda) Dec. 7, 8, 9, 10, and 11, 2020 14:00 – 18:00 (Paris time) 16 h Thomas Petazzoni 519 EUR + VAT* (register)
Linux Graphics (agenda) Dec. 1, 2, 3, 4, 2020 14:00 – 18:00 (Paris time) 16 h Paul Kocialkowski 519 EUR + VAT* (register

* VAT: applies to businesses in France and to individuals from all countries. Businesses in the European Union won’t be charged VAT only if they provide valid company information and VAT number to Evenbrite at registration time. For businesses in other countries, we should be able to grant them a VAT refund, provided they send us a proof of company incorporation before the end of the session.

Each public session will be confirmed once there are at least 6 participants. If the minimum number of participants is not reached, Bootlin will propose new dates or a full refund (including Eventbrite fees) if no new date works for the participant.

We guarantee that the maximum number of participants will be 12.

Dedicated online sessions

If you have enough people to train, such dedicated sessions can be a worthy alternative to public ones:

  • Flexible dates and daily durations, corresponding to the availability of your teams.
  • Confidentiality: freedom to ask questions that are related to your company’s projects and plans.
  • If time left, possibility to have knowledge sharing time with the instructor, that could go beyond the scope of the training course.
  • Language: possibility to have a session in French instead of in English.

Online seminar details

Each session will be given through Jitsi Meet, a Free Software solution that we are trying to promote. As a backup solution, we will also be able to Google Hangouts Meet. Each participant should have her or his own connection and computer (with webcam and microphone) and if possible headsets, to avoid echo issues between audio input and output. This is probably the best solution to allow each participant to ask questions and write comments in the chat window. We also support people connecting from the same conference room with suitable equipment.

Each participant is asked to connect 15 minutes before the session starts, to make sure her or his setup works (instructions will be sent before the event).

How to register

For online public sessions, use the EventBrite links in the above list of sessions to register one or several individuals.

To register an entire group (for dedicated sessions), please contact training@bootlin.com and tell us the type of session you are interested in. We will then send you a registration form to collect all the details we need to send you a quote.

You can also ask all your questions by calling +33 484 258 097.

Questions and answers

Q : Should I order hardware in advance, our hardware included in the training cost?
R : No, practical labs are replaced by technical demonstrations, so you will be able to follow the course without any hardware. However, you can still order the hardware by checking the “Shopping list” pages of presentation materials for each session. This way, between each session, you will be able to replay by yourself the labs demonstrated by your trainer, ask all your questions, and get help between sessions through our dedicated Matrix channel to reach your goals.

Q: Why just demos instead of practicing with real hardware?
A: We are not ready to support a sufficient number of participants doing practical labs remotely with real hardware. This is more complicated and time consuming than in real life. Hence, what we we’re proposing is to replace practical labs with practical demonstrations shown by the instructor. The instructor will go through the normal practical labs with the standard hardware that we’re using.

Q: Would it be possible to run practical labs on the QEMU emulator?
R: Yes, it’s coming. In the embedded Linux course, we are already offering instructions to run most practical labs on QEMU between the sessions, before the practical demos performed by the trainer. We should also be able to propose such instructions for our Yocto Project and Buildroot training courses in the next months. Such work is likely to take more time for our Linux kernel course, practical labs being closer to the hardware that we use.

Q: Why proposing half days instead of full days?
A: From our experience, it’s very difficult to stay focused on a new technical topic for an entire day without having periodic moments when you are active (which happens in our public and on-site sessions, in which we interleave lectures and practical labs). Hence, we believe that daily slots of 4 hours (with a small break in the middle) is a good solution, also leaving extra time for following up your normal work.

Bootlin at FOSDEM and Buildroot Developers Meeting

FOSDEM 2020This week-end takes place one of the biggest and most important free and open-source software conference in Europe: FOSDEM. It will once again feature a very large number of talks, organized in several main tracks and developer rooms.

Bootlin CTO Thomas Petazzoni will participate to the FOSDEM conference, of course attending many of the talks from the Embedded, Mobile and Automative Devroom, to which he participated to the talk review and selection. Do not hesitate to get in touch with Thomas if you want to discuss career or business opportunities with Bootlin.

In addition, Thomas will also participate to the 3-day Buildroot Developers meeting which takes place in Brussels right after the FOSDEM conference, kindly hosted by Google. During 3 days, some of the core Buildroot developers will work together to discuss the future of Buildroot, as well as review and discuss pending patches and proposals.

Building a Linux system for the STM32MP1: developing a Qt5 graphical application

After showing how to build a minimal Linux system for the STM32MP157 platform, how to connect and use an I2C based pressure/temperature/humidity sensor and how to integrate Qt5 in our system, how to set up a development environment to write our own Qt5 application, we are finally going to write our Qt5 application.

List of articles in this series:

  1. Building a Linux system for the STM32MP1: basic system
  2. Building a Linux system for the STM32MP1: connecting an I2C sensor
  3. Building a Linux system for the STM32MP1: enabling Qt5 for graphical applications
  4. Building a Linux system for the STM32MP1: setting up a Qt5 application development environment
  5. Building a Linux system for the STM32MP1: developing a Qt5 graphical application
  6. Building a Linux system for the STM32MP1: implementing factory flashing
  7. Building a Linux system for the STM32MP1: remote firmware updates

Disclaimer

Before we get started in this blog post, it is important to mention that it is not meant to be a full introduction to programming applications with Qt5. This would require much more than a blog post, and the Qt web site has extensive documentation.

Also, we want to make it clear that Bootlin’s core expertise is in low-level embedded Linux development, not in Qt application development. Therefore, our example application may not show the best practices in terms of Qt development. We welcome comments and suggestions from our readers to improve the example used in this blog post.

Reading sensor data

As we’ve seen in a previous article, the sensor data is available by reading the following files:

  • /sys/bus/iio/devices/iio:device2/in_temp_input for the temperature
  • /sys/bus/iio/devices/iio:device2/in_pressure_input for the pressure
  • /sys/bus/iio/devices/iio:device2/in_humidityrelative_input for the humidity

So what we will do is writing a new class called DataProvider, which will read those files once per second, and emit a signal with the 3 values every second. Slots and signals is a fundamental mechanism in Qt, which allows to connect emitters of events to receivers for those events. In our case, the DataProvider class will emit a signal when new sensor values are read, while another class in charge of the graphical UI will receive those signals.

At this step, we don’t yet have a graphical UI, so we’ll simply add a few debugging messages in the DataProvider to make sure it works as expected.

Let’s start by adding a data-provider.h file to our project:

#ifndef DATA_PROVIDER_H
#define DATA_PROVIDER_H

#include <QtCore/QTimer>

class DataProvider: public QObject
{
    Q_OBJECT

public:
    DataProvider();

private slots:
    void handleTimer();

signals:
    void valueChanged(float temp, float pressure, float humidity);

private:
    QTimer timer;
};

#endif /* DATA_PROVIDER_H */

It creates a very simple class than inherits from QObject, with:

  • A constructor
  • A private slot handleTimer which will be used internally by the class QTimer’s instance to notify that a timer has expired. This is what will allow us to poll the sensor values every second.
  • A valueChanged signal, which will be emitted by the class every time new sensor values are available.

Then, the implementation of this class in data-provider.cpp is fairly straight-forward:

#include <QtCore/QFile>
#include <QDebug>
#include "data-provider.h"

DataProvider::DataProvider()
{
    QObject::connect(&timer, &QTimer::timeout,
		     this, &DataProvider::handleTimer);
    timer.setInterval(1000);
    timer.start();
}

void DataProvider::handleTimer()
{
    QFile temp_f("/sys/bus/iio/devices/iio:device2/in_temp_input");
    QFile pressure_f("/sys/bus/iio/devices/iio:device2/in_pressure_input");
    QFile humidity_f("/sys/bus/iio/devices/iio:device2/in_humidityrelative_input");

    if (!temp_f.open(QIODevice::ReadOnly | QIODevice::Text))
        return;
    if (!pressure_f.open(QIODevice::ReadOnly | QIODevice::Text))
        return;
    if (!humidity_f.open(QIODevice::ReadOnly | QIODevice::Text))
        return;

    float temp = QString(temp_f.readAll()).toDouble() / 1000;
    float pressure = QString(pressure_f.readAll()).toDouble() * 10;
    float humidity = QString(humidity_f.readAll()).toDouble() / 1000;

    qDebug() << "Temperature: " << temp << "Pressure: " << pressure << "Humidity: " << humidity;

    emit valueChanged(temp, pressure, humidity);
}

The constructor of the class connects the QTimer::timeout signal of the QTimer to this class handlerTimer slot, sets the timer interval to 1000 milliseconds, and starts the timer. This is what will ensure the handleTimer method gets called every second.

In the handleTimer method, we open the 3 files in sysfs, read their value and convert them to meaningful units: the temperature in Celcius, the pressure in hPA, and the humidity in percent. We then print a debugging message and emit the signal with the three values.

With this in place, we need to make sure those two files are properly taken into account by our project, by changing the .pro file as follows:

QT += widgets
SOURCES = main.cpp data-provider.cpp
HEADERS = data-provider.h
INSTALLS += target
target.path = /usr/bin

The data-provider.cpp file was added to SOURCES, while data-provider.h was added to the new HEADERS.

Now, we just need to change main.cpp to instantiate one DataProvider object:

#include <QApplication>
#include <QPushButton&ht;
#include "data-provider.h"

int main(int argc, char* argv[])
{
    QApplication app(argc, argv);
    QPushButton hello("Hello world!");
    DataProvider dp;
    hello.resize(100,30);
    hello.show();
    return app.exec();
}

With this, you can now build and run the application, and you should see every second the debugging message showing the temperature, pressure and humidity values:

# qt-sensor-demo -platform linuxfb
Temperature:  28.12  Pressure:  1003.08  Humidity:  32.235
Temperature:  28.12  Pressure:  1003.07  Humidity:  32.246
Temperature:  28.12  Pressure:  1003.06  Humidity:  32.256
Temperature:  28.12  Pressure:  1003.08  Humidity:  32.267

Displaying sensor data

We now want to display the sensor data. For this, we'll create a UI with two panels, one to display the numeric value of the temperature, humidity and pressure, and another panel with a chart of the temperature. At the bottom of the screen, two buttons Values and Chart will allow to switch between both panels.

So, we'll create a Window class to encapsulate the overall window layout and behavior, and a Values class providing the widget showing the 3 values. We'll leave the chart implementation to the next section. To help you follow the code in this section, here is a diagram that shows the different widgets and how they will be grouped together in our user interface:

Qt Sensor demo UI

Let's start by implementing the Values widget, which will be used to show the 3 numeric values, one below each other. The values.h file will look like this:

#ifndef VALUES_H
#define VALUES_H

#include <QWidget>

class QLabel;

class Values : public QWidget
{
    Q_OBJECT

public:
    Values();

public slots:
    void handleValueChanged(float temp, float pressure, float humidity);

private:
    QLabel *temperature_v;
    QLabel *pressure_v;
    QLabel *humidity_v;
};

#endif /* VALUES_H */

So it has a simple constructor, a slot to be notified of new values available, and 3 text labels to display the 3 values. The implementation in values.cpp is:

// SPDX-License-Identifier: MIT
#include <QtWidgets>
#include "values.h"

Values::Values()
{
    QVBoxLayout *layout = new QVBoxLayout;

    QLabel *temperature_l = new QLabel(tr("Temperature (°C)"));
    QLabel *pressure_l = new QLabel(tr("Pressure (hPa)"));
    QLabel *humidity_l = new QLabel(tr("Humidity (%)"));

    temperature_v = new QLabel();
    pressure_v = new QLabel();
    humidity_v = new QLabel();

    QFont f = temperature_v->font();
    f.setPointSize(28);
    f.setBold(true);
    temperature_v->setFont(f);
    pressure_v->setFont(f);
    humidity_v->setFont(f);
    temperature_v->setAlignment(Qt::AlignRight | Qt::AlignVCenter);
    pressure_v->setAlignment(Qt::AlignRight | Qt::AlignVCenter);
    humidity_v->setAlignment(Qt::AlignRight | Qt::AlignVCenter);

    layout->addWidget(temperature_l);
    layout->addWidget(temperature_v);
    layout->addWidget(pressure_l);
    layout->addWidget(pressure_v);
    layout->addWidget(humidity_l);
    layout->addWidget(humidity_v);

    setLayout(layout);
}

void Values::handleValueChanged(float temp, float pressure, float humidity)
{
    temperature_v->setText(QString::number(temp, 'f', 2));
    pressure_v->setText(QString::number(pressure, 'f', 1));
    humidity_v->setText(QString::number(humidity, 'f', 1));
}

The constructor creates 3 text labels for the legends ("Temperature (°C)", "Pressure (hPA)" and "Humidity (%)"), then instantiates the 3 text labels for the values themselves. It sets up the font and text alignment properties for those labels, and then adds all widgets in a QVBoxLayout so that they all appear vertically below each other.

The handleValueChanged slot simply updates the text labels contents with the new sensor values, doing the proper text formatting on the way.

With the Values class implemented, we can now implement the main Window class. The window.h will contain:

#ifndef WINDOW_H
#define WINDOW_H

#include <QWidget>

class Values;

class Window : public QWidget
{
    Q_OBJECT

public slots:
    void handleValueChanged(float temp, float pressure, float humidity);

public:
    Window();

private:
    Values *values;
};

#endif

Beyond a simple constructor, it has a slot to receive new sensor values, and a reference to a Values widget instance.

The implementation in window.cpp is as follows:

#include <QtWidgets>

#include "window.h"
#include "values.h"

Window::Window()
{
    values = new Values;
    QVBoxLayout *layout = new QVBoxLayout;
    QHBoxLayout *buttons = new QHBoxLayout;

    QPushButton *values_button = new QPushButton("Values");
    QPushButton *chart_button = new QPushButton("Chart");

    buttons->addWidget(values_button);
    buttons->addWidget(chart_button);

    layout->addWidget(values);
    layout->addLayout(buttons);

    setLayout(layout);

    setWindowTitle(tr("Sensors"));
}

void Window::handleValueChanged(float temp, float pressure, float humidity)
{
    values->handleValueChanged(temp, pressure, humidity);
}

The constructor creates a horizontal layout QHBoxLayout with two buttons: Values and Chart. Those will be used in the next section to switch between the Values panel and the Chart panel. For now, they don't do anything.

Then, the constructor adds the Value widget, and the horizontal layout box with the buttons into a vertical box layout, assigns the main window layout and defines the window title.

The handleValueChanged slot implementation just forwards the call to the Values::handleValueChanged method.

Now, obviously main.cpp needs to be changed: instead of creating a button, we'll create our window, and do a bit of additional setup:

#include <QApplication>
#include "window.h"
#include "data-provider.h"

int main(int argc, char* argv[])
{
    QApplication app(argc, argv);
    DataProvider dp;
    Window window;

    QObject::connect(&dp, &DataProvider::valueChanged,
		     &window, &Window::handleValueChanged);

    window.setFixedSize(480, 800);
    window.setStyleSheet("background-color: white;");
    window.show();
    return app.exec();
}

So, not only we create the Window, but more importantly, we connect the valueChanged signal of DataProvider to the handleValueChanged slot of Window. We define the window size (which is fixed, to match the STM32MP15 Discovery board panel) and set the background color of the application.

Obviously, the qt-sensor-demo.pro file needs to be adjusted to build our new files. It now looks like this:

QT += widgets
SOURCES = main.cpp data-provider.cpp window.cpp values.cpp
HEADERS = data-provider.h window.h values.h
INSTALLS += target
target.path = /usr/bin

With this done, we can run the Qt5 application on our target, and see:

Qt5 application showing the I2C sensor data

Graphing the temperature

The final part of developing our application is to implement a graph showing the evolution of temperature over time. For this, we are going to use the very convenient Qt Charts module, which is available in a separate Qt module from the base of Qt.

To implement the graph widget itself, we'll create a new Chart class:

#ifndef CHART_H
#define CHART_H

#include <QtCharts/QChart>

QT_CHARTS_BEGIN_NAMESPACE
class QSplineSeries;
class QValueAxis;
QT_CHARTS_END_NAMESPACE

QT_CHARTS_USE_NAMESPACE

class Chart: public QChart
{
    Q_OBJECT

public:
    Chart(QGraphicsItem *parent = 0, Qt::WindowFlags wFlags = 0);

public slots:
    void handleValueChanged(float temp, float pressure, float humidity);

private:
    QSplineSeries *m_series;
    QStringList m_titles;
    QValueAxis *m_axisX;
    QValueAxis *m_axisY;
    int xpos;
};

#endif /* CHART_H */

This class inherits from the QChart class provided by Qt. It provides a constructor and destructor, a slot that allows to receive notification of new sensor values, and it has a number of private variables to manage the chart itself.

Let's go through the implementation of this class now:

#include "chart.h"
#include <QtCharts/QAbstractAxis>
#include <QtCharts/QSplineSeries>
#include <QtCharts/QValueAxis>

Chart::Chart(QGraphicsItem *parent, Qt::WindowFlags wFlags):
    QChart(QChart::ChartTypeCartesian, parent, wFlags),
    m_series(0),
    m_axisX(new QValueAxis()),
    m_axisY(new QValueAxis()),
    xpos(0)
{
    m_series = new QSplineSeries(this);
    QPen pen(Qt::red);
    pen.setWidth(2);
    m_series->setPen(pen);
    m_series->append(xpos, 30);

    addSeries(m_series);

    addAxis(m_axisX,Qt::AlignBottom);
    addAxis(m_axisY,Qt::AlignLeft);
    m_series->attachAxis(m_axisX);
    m_series->attachAxis(m_axisY);
    m_axisX->setTickCount(5);
    m_axisX->setRange(0, 60);
    m_axisY->setRange(0, 50);

    QFont f = m_axisX->labelsFont();
    f.setPointSize(8);
    m_axisX->setLabelsFont(f);
    m_axisY->setLabelsFont(f);

    setMargins(QMargins(0,0,0,0));

    setTitle("Temperature (°C)");
    legend()->hide();
}

void Chart::handleValueChanged(float temp, float pressure, float humidity)
{
    Q_UNUSED(pressure);
    Q_UNUSED(humidity);
    m_series->append(xpos, temp);
    xpos++;
    if (xpos >= 60)
      scroll(plotArea().width() / 60, 0);
}

The constructor simply sets up the QChart we inherit from: defining the axis, their range, the pen width and color, etc. On the X axis (time), we are going to show 60 measurements, and since our handleValueChanged slot is going to be called every second, it means our graph will show the last 60 seconds of temperature measurement. On the Y axis (temperature), we can show temperatures from 0°C to 50°C. Of course, this is all very hardcoded in this example, for simplicity.

The handleValueChanged slot appends the new temperature value to the graph, and then updates the area displayed by the graph so that always the last 60 seconds are visible.

Now, we need to integrate this to our existing Window class, so that we can display the chart, and switch between the numeric values and the chart. First, we need to do some changes in window.h, and below we'll show only the diff to make the differences very clear:

diff --git a/window.h b/window.h
index 3d63d38..05d1f39 100644
--- a/window.h
+++ b/window.h
@@ -3,8 +3,12 @@
 #define WINDOW_H
 
 #include <QWidget>
+#include <QtCharts/QChartView>
+
+QT_CHARTS_USE_NAMESPACE
 
 class Values;
+class Chart;
 
 class Window : public QWidget
 {
@@ -13,11 +17,17 @@ class Window : public QWidget
 public slots:
     void handleValueChanged(float temp, float pressure, float humidity);
 
+private slots:
+    void chartButtonClicked();
+    void valuesButtonClicked();
+
 public:
     Window();
 
 private:
     Values *values;
+    QChartView *chartView;
+    Chart *chart;
 };
 
 #endif

So, we're defining two private slots that will be used for the two buttons that allow to switch between the numeric values and the chart, and then we add two variables, one for the chart itself, and one for the QChartView (which basically renders the graph into a widget).

Then, in window.cpp, we do the following changes:

diff --git a/window.cpp b/window.cpp
index aba2862..d654964 100644
--- a/window.cpp
+++ b/window.cpp
@@ -3,28 +3,54 @@
 
 #include "window.h"
 #include "values.h"
+#include "chart.h"
 
 Window::Window()
 {
     values = new Values;
+    chart = new Chart;
     QVBoxLayout *layout = new QVBoxLayout;
     QHBoxLayout *buttons = new QHBoxLayout;
 
     QPushButton *values_button = new QPushButton("Values");
     QPushButton *chart_button = new QPushButton("Chart");
 
+    QObject::connect(chart_button, &QPushButton::clicked,
+                    this, &Window::chartButtonClicked);
+    QObject::connect(values_button, &QPushButton::clicked,
+                    this, &Window::valuesButtonClicked);
+
     buttons->addWidget(values_button);
     buttons->addWidget(chart_button);
 
+    chartView = new QChartView(chart);
+    chartView->setRenderHint(QPainter::Antialiasing);
+
     layout->addWidget(values);
+    layout->addWidget(chartView);
     layout->addLayout(buttons);
 
     setLayout(layout);
 
+    chartView->hide();
+
     setWindowTitle(tr("Sensors"));
 }
 
 void Window::handleValueChanged(float temp, float pressure, float humidity)
 {
     values->handleValueChanged(temp, pressure, humidity);
+    chart->handleValueChanged(temp, pressure, humidity);
+}
+
+void Window::chartButtonClicked()
+{
+    values->hide();
+    chartView->show();
+}
+
+void Window::valuesButtonClicked()
+{
+    values->show();
+    chartView->hide();
 }

So, in the constructor we are connecting the clicked signals of the two buttons to their respective slots. We create the Chart object, and then the QChartView to render the graph. We add the latter as an additional widget in the QVBoxLayout, and we hide it.

The existing handleValueChanged slot is modified to also update the Chart object with the new sensor values.

Finally, the new chartButtonClicked and valuesButtonClicked slots implement the logic that is executed when the buttons are pressed. We simply hide or show the appropriate widget to display either the numeric values or the chart. There is probably a nicer way to achieve this in Qt, but this was good enough for our example.

Now that the source code is in place, we of course need to adjust the build logic in qt-sensor-demo.pro:

--- a/qt-sensor-demo.pro
+++ b/qt-sensor-demo.pro
@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: MIT
-QT += widgets
-SOURCES = main.cpp data-provider.cpp window.cpp values.cpp
-HEADERS = data-provider.h window.h values.h
+QT += widgets charts
+SOURCES = main.cpp data-provider.cpp window.cpp values.cpp chart.cpp
+HEADERS = data-provider.h window.h values.h chart.h
 INSTALLS += target
 target.path = /usr/bin

Besides the obvious addition of the chart.cpp and chart.h file, the other important addition is charts to the QT variable. This tells qmake that our application is using the Qt Charts, and that we therefore need to link against the appropriate libraries.

Building the application

At this point, if you try to build the application, it will fail because QtCharts has not been built as part of our Buildroot configuration. In order to address this, run Buildroot's make menuconfig, enable the BR2_PACKAGE_QT5CHARTS option (in Target packages -> Graphic libraries and applications -> Qt5 -> qt5charts).

Then, run the Buildroot build with make, and reflash the resulting SD card image.

Now, you can build again your application, either with Qt Creator if you've been using Qt Creator, or manually. If you build it manually, you'll have to run qmake again to regenerate the Makefile, and then build with make.

When you run the application on the target, the GUI will display the same numeric values as before, but now if you press the Chart button, it will show something like:

Qt5 application with chart

Adjusting the Buildroot package

We have for now been building this application manually, but as explained in our previous blog post, we really want Buildroot to be able to build our complete system, including our application. For this reason, we had created a qt-sensor-demo package, which gets our application source code, configures it with qmake, builds it and installs it.

However, with the new use of Qt Charts, our qt-sensor-demo package needs a few adjustements:

  • The Config.in file needs an additional select BR2_PACKAGE_QT5CHARTS, to make sure Qt Charts are enabled in the Buildroot configuration
  • The qt-sensor-demo.mk file needs an additional qt5charts in the QT_SENSOR_DEMO_DEPENDENCIES variable to make sure the qt5charts package gets built before qt-sensor-demo

With this in place, you can run:

make qt-sensor-demo-rebuild
make

And you have an SD card image that includes our application!

Starting the application automatically at boot time

The next and almost final step for this blog post is to get our application automatically started at boot time. We can simply add a small shell script on the target in /etc/init.d/: the default Buildroot configuration for the init system will execute all scripts named Ssomething in /etc/init.d/. We'll add a file named package/qt-sensor-demo/S99qt-sensor-demo with these contents:

#!/bin/sh

DAEMON="qt-sensor-demo"
DAEMON_ARGS="-platform linuxfb"
PIDFILE="/var/run/qt-sensor-demo.pid"

start() {
	printf 'Starting %s: ' "$DAEMON"
	start-stop-daemon -b -m -S -q -p "$PIDFILE" -x "/usr/bin/$DAEMON" -- $DAEMON_ARGS
	status=$?
	if [ "$status" -eq 0 ]; then
		echo "OK"
	else
		echo "FAIL"
	fi
	return "$status"
}

stop () {
	printf 'Stopping %s: ' "$DAEMON"
	start-stop-daemon -K -q -p "$PIDFILE"
	status=$?
	if [ "$status" -eq 0 ]; then
		rm -f "$PIDFILE"
		echo "OK"
	else
		echo "FAIL"
	fi
	return "$status"
}

restart () {
	stop
	sleep 1
	start
}

case "$1" in
        start|stop|restart)
		"$1";;
	reload)
		# Restart, since there is no true "reload" feature.
		restart;;
        *)
                echo "Usage: $0 {start|stop|restart|reload}"
                exit 1
esac

This is the canonical init script used in Buildroot to start system daemons and services, and is modeled after the one in package/busybox/S01syslogd. It uses the start-stop-daemon program to start our application in the background.

Then, to get this init script installed, we need to adjust package/qt-sensor-demo/qt-sensor-demo.mk with the following additional lines:

define QT_SENSOR_DEMO_INSTALL_INIT_SYSV
        $(INSTALL) -D -m 755 package/qt-sensor-demo/S99qt-sensor-demo \
                $(TARGET_DIR)/etc/init.d/S99qt-sensor-demo
endef

This ensures that the init script gets installed in /etc/init.d/S99qt-sensor-demo as part of the build process of our qt-sensor-demo package. Note that an init script works fine if you're using the Busybox init implementation or the sysvinit init implementation (we're using the default Buildroot setup here, which uses the Busybox init implementation). If you want to use systemd as an init implementation, then a different setup is necessary.

With this done, you simply need to reinstall the application and regenerate the SD card image

$ make qt-sensor-demo-reinstall
$ make

You can now test your SD card image on you board, and you should see the application being started automatically, with the following messages at boot time

Starting dropbear sshd: OK
Starting qt-sensor-demo: OK

Welcome to Buildroot

Avoid unnecessary logging on the display panel

In our current setup, the kernel messages are being sent to both the serial port and the framebuffer console, which means they appear on the display panel. This is not very pretty, and we would like the display to remain black until the application starts, while keeping the kernel messages on the serial port for debugging purposes. Also, we would like the framebuffer console text cursor to not be displayed, to really have a fully black screen. To achieve this we will add two arguments on the Linux kernel command line:

  • console=ttySTM0,115200, which will tell the Linux kernel to only use the serial port as the console, and not all registered consoles, which would include the framebuffer console. This option will make sure the kernel messages are not displayed on the screen.
  • vt.global_cursor_default=0, which will tell the Linux kernel to not display any cursor on the framebuffer console.

So, to add those options, we simply modify board/stmicroelectronics/stm32mp157-dk/overlay/boot/extlinux/extlinux.conf in Buildroot as follows:

label stm32mp15-buildroot
  kernel /boot/zImage
  devicetree /boot/stm32mp157c-dk2.dtb
  append root=/dev/mmcblk0p4 rootwait console=ttySTM0,115200 vt.global_cursor_default=0

Of course, rebuild the SD card image with make, reflash and test the result on your STM32MP1 platform.

Conclusion

In this blog post, we have seen how to write a real (but admittedly very simple) Qt application, how to make it read and display sensor data, and how to integrate this application so that it gets started at boot time.

You can find the Buildroot changes corresponding to this blog post in the 2019.02/stm32mp157-dk-blog-5 branch of our repository. The qt-sensor-demo application code can be found in the blog-5 branch of this application Git repository.

Stay tuned for our next blog post about factory flashing and OTA update!

Improvements to Buildroot maintenance tooling

From mid-April to end of August, Victor Huesca, a student from the University of Toulouse joined Bootlin’s team for a 3.5 months internship. His internship was focused on the Buildroot project, and Victor’s mission was to improve various aspect of the tooling around Buildroot to help in the maintenance of this build system. In this blog post, we will present the different improvements and features implemented by Victor during his internship. This internship was funded by Bootlin, and entirely focused on contributing to the Buildroot project.

Notifications of new upstream versions of packages

Buildroot has over 2400 packages for a wide variety of software components, and it is a challenge to keep all of those packages updated with the latest releases from the upstream developers. Buildroot has a nice statistics page with all its packages, and in early 2019, your author added support for querying the release-monitoring.org service to find the latest upstream version of each package. This allowed our statistics page to show the current version in Buildroot and the latest version available upstream for all Builroot packages.

Victor improved this by implementing e-mail notifications to Buildroot developers about their package having new upstream releases available. Indeed, Buildroot has a DEVELOPERS file which associates the name and e-mail of Buildroot contributors/developers with the packages they take care of. So, what Victor did is:

  • Extend the pkg-stats script, which generates the statistics page, to not only generate a HTML output, but also a JSON output. A JSON output is obviously a lot more usable by other tools. Victor also improved the efficiency of this script in several ways, especially by parallelizing the requests made to release-monitoring.org.
  • Extend the daily-mail script, which until now was only sending autobuild results, to also send notifications about packages that are not up to date with their latest upstream version

The notifications are sent only once a week, both individually to the developers, and globally on the mailing list. They are grouped in a single e-mail with the existing autobuild results notifications, to minimize the amount of e-mails received by developers. You can see an example of such a notification in this e-mail, with a small excerpt below:

Packages having a newer version
===============================

             name              | found by |        link to release-monitoring.org        |   version    |   upstream   | orph?
-------------------------------+----------+----------------------------------------------+--------------+--------------+-------
                           4th |  DISTRO  | https://release-monitoring.org/project/20471 | 3.62.5       | 3625         |     
                        acpica |  DISTRO  | https://release-monitoring.org/project/00018 | 20190703     | 20190816     |     
                         acpid |  DISTRO  | https://release-monitoring.org/project/00019 | 2.0.30       | 2.0.32       | ORPH
                       acsccid |  DISTRO  | https://release-monitoring.org/project/15661 | 1.1.4        | 1.1.7        |     
            adwaita-icon-theme |  DISTRO  | https://release-monitoring.org/project/13117 | 3.22.0       | 3.33.92      |     
                       aespipe |  GUESS   | https://release-monitoring.org/project/21320 | 2.4d         | 2.4e         | ORPH
                 android-tools |  GUESS   | https://release-monitoring.org/project/13989 | 4.2.2+git... | 10.0.0_r2    |     
                      argparse |  GUESS   | https://release-monitoring.org/project/02100 | 0.7.0-1      | 1.0.10       |     
                         argus |  GUESS   | https://release-monitoring.org/project/00107 | 3.0.8        | 3.0.8.2      | ORPH
                     armadillo |  GUESS   | https://release-monitoring.org/project/07006 | 7.900.1      | 9.700.2      |     
                   at-spi2-atk |  GUESS   | https://release-monitoring.org/project/07840 | 2.26.2       | 2.33.92      |     
                  at-spi2-core |  GUESS   | https://release-monitoring.org/project/07841 | 2.28.0       | 2.33.92      |     
                         atkmm |  GUESS   | https://release-monitoring.org/project/07962 | 2.24.2       | 2.29.1       |     
                      automake |  DISTRO  | https://release-monitoring.org/project/00144 | 1.15.1       | 1.16.1       | ORPH
[...]

As part of this work, Victor also improved the matching of versions between the Buildroot package versions and the upstream versions. Indeed, for many packages, Buildroot used to use the full Git tag name as the version (for example v1.3), while release-monitoring.org removes any prefix and keeps only 1.3.

As of today, not all Buildroot packages match with a project known by release-monitoring.org, either because release-monitoring.org doesn’t know the project, or because the name is slightly different, but we are improving this progressively (the name mismatch can be handled by creating a mapping on release-monitoring.org, thanks to the concept of distribution they have).

The work of Victor has already proven to be very useful: a number of infrequent contributors suddenly started taking care of the packages they had contributed a long time ago and perhaps forgotten since then, which is very good.

Notifications of defconfig and runtime test failures

Buildroot provides a number of defconfig files, which are example Buildroot configuration for a wide range of hardware platforms (Raspberry Pi, BeagleBone, Qemu emulated machines, NXP or Microchip evaluation boards, and more). These defconfigs offers a very simple way for users to get a minimal Buildroot system up and running on those hardware platforms, making them a great starting point. Of course, to make them useful, they have to build properly, and we regularly build them using Gitlab CI to ensure they continue to build.

Buildroot also has runtime tests, which were initially introduced in the project by your author back in 2017. Those runtime tests are test cases that will each build a specific well-defined Buildroot configuration, boot it under Qemu, and verify that everything works properly. For example, the filesystem test cases will each make a Buildroot build with a specific filesystem image format selected, and boot the result under Qemu, to make sure that the filesystem image is correct and working. We also have a significant number of test cases for Perl or Python modules, which simply build the Perl or Python interpreter with a collection of modules, boot under Qemu, and verify that those modules can be loaded/imported. Just like the defconfigs, these runtime tests are already tested on a regular basis using Gitlab CI, to detect and fix any regression.

However, the results of those tests in Gitlab CI (and especially failures) were not notified to the Buildroot community in a meaningful way. This is where Victor filled in the gap, by adding the appropriate notifications.

He further extended the daily-mail script so that using the Gitlab CI API, the latest Gitlab CI pipelines for the Buildroot project are retrieved, the defconfig and runtime test failures are identified, and the appropriate Buildroot developers and contributors are notified. Indeed, just like packages are referenced in Buildroot’s DEVELOPERS file, the defconfigs and runtime tests are also referenced. The daily-mail script will notify individual developers about the defconfig and runtime tests they take care of, and it will also globally notify the mailing list about all defconfig and runtime test failures.

An example output, visible in this notification e-mail is:


Detail of defconfig failures
----------------------------

                        defconfig |                        link to the job                        | orph?
----------------------------------+---------------------------------------------------------------+------
                amarula_a64_relic |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126211  |      
               arcturus_ucls1012a |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126214  |      
                        bananapro |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126246  |      
                   beaglebone_qt5 |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126249  | ORPH 
        engicam_imx6qdl_icore_qt5 |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126255  |      
                 imx6-sabresd_qt5 |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126282  |      
                       imx6ulpico |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126286  |      
                        imx7dpico |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126288  |      
                    licheepi_zero |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126292  |      
               linksprite_pcduino |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126293  |      
                    orangepi_lite |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126321  |      
                   orangepi_lite2 |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126320  |      
                 orangepi_pc_plus |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126326  |      
                    orangepi_zero |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126331  |      
              orangepi_zero_plus2 |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126332  |      
                   pc_x86_64_bios |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126334  |      
                    pc_x86_64_efi |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126335  |      
               raspberrypi3_qt5we |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126382  | ORPH 
                            warp7 |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126413  |      
                        warpboard |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126414  | ORPH 


Detail of runtime-test failures
-------------------------------

             runtime-test |                        link to the job                        | orph?
--------------------------+---------------------------------------------------------------+------
...ystemSystemdRoIfupdown |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126471  | ORPH 
...ystemSystemdRoNetworkd |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126473  | ORPH 
...nitSystemSystemdRwFull |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126475  | ORPH 
...ystemSystemdRwIfupdown |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126476  | ORPH 
...ystemSystemdRwNetworkd |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126477  | ORPH 
             TestSyslogNg |  https://gitlab.com/buildroot.org/buildroot/-/jobs/289126581  | ORPH 

Overall, thanks to Victor’s work, a single e-mail now reports autobuilder failures, the need to update packages to a newer upstream versions, defconfig build failures and runtime tests failures. This is a really good improvement in the tooling of the Buildroot community!

Buildroot autobuilder search capabilities

Buildroot provides over 2400 packages, and many of them have configurable features and optional dependencies. This creates a massive amount of possible configuration combinations, making it impossible to test all of them. To make sure as many Buildroot configurations build properly, the project has been running for many years the Buildroot autobuilders. A number of build machines build random Buildroot configurations 24/7, and report their results to autobuild.buildroot.org. This helps tremendously the Buildroot developers and maintainers to detect the problematic packages and configurations.

For a long time, the autobuild.buildroot.org allowed to filter build results by architecture, C library, failing package, and a few other criterias. Such filtering is very often useful to understand when a package started failing to build, and in which situations it fails to build.

autobuild.buildroot.org was also collecting in a database all the configuration symbols (the BR2_something symbols) for every Buildroot configuration that was built. However, the size of this database made any query excessively long, so we were not able to make use of it so far. This was annoying because it would sometimes be useful to ask could you tell me which configuration had BR2_PACKAGE_STRACE=y and built successfully ?.

That’s where Victor jumped in:

  • He improved the database by adding the appropriate indexes and found a reasonably efficient way to query the database when configuration symbols are involved
  • He added filtering per configuration symbol, which can be done using GET arguments on the main autobuild.buildroot.org page: http://autobuild.buildroot.org/?status=OK&symbols[BR2_PACKAGE_STRACE]=y will show the builds that had BR2_PACKAGE_STRACE=y and that ended successfully. Multiple symbols[name]=value arguments can be passed.
  • Since writing such queries by hand was a bit cumbersome, Victor also added a new search page.
Autobuild search page
Autobuild search page

This work will be very useful in the future to analyze build failures and understand better in which situations they are happening.

Conclusion

Victor’s internship has been very productive in improving the tooling used by the Buildroot community to maintain the project. All the work done by Victor has been merged, is in production, and is already showing some useful results.