Continuous integration in Yocto: improving the regressions detection

The Yocto Project is an open source umbrella project which gathers all needed tools to build full Linux distributions for a wide variety of devices. As the interest for Yocto grew since its first steps, its size and number of use cases increased consequentially. This growth quickly introduced the need of automated testing so that developers can keep introducing new features to the project while making sure not to break any existing part. Bootlin engineer Alexis Lothoré has recently been involved in the Continuous Integration infrastructure of the Yocto Project and has brought improvements to allow Yocto maintainers to detect regressions earlier.

Continue reading “Continuous integration in Yocto: improving the regressions detection”

Embedded Linux and Yocto training courses now available in Italian

Flag of Italy Buongiorno!

For many years, we have been offering our training courses in both English and French. We are happy to announce that we are now expanding our offering towards our Italian customers: we are now able to deliver our very popular Embedded Linux system development and Yocto Project and OpenEmbedded development training courses in Italian. Those courses are delivered by a native Italian speaker, Bootlin engineer Luca Ceresoli. Luca has 20 years of experience working with embedded systems, including 15 years on embedded Linux.

Thanks to this offering, our Italian customers can enjoy a training course delivered in their native language, which obviously facilitates communication, understanding and interaction. We can offer our Embedded Linux course and Yocto course in Italian through either private on-site sessions or private on-line sessions. Contact us at training@bootlin.com for details and pricing!

Ti aspettiamo!

Updated Yocto support for SiFive RISC-V platforms

A few months ago, we started supporting SiFive in their effort to maintain and improve the Yocto support for their RISC-V processors and platforms. The Yocto support for SiFive platforms is divided into two layers:

A BSP layer: meta-sifive
A demo/SDK layer: freedom-u-sdk

These layers allow to build ready-to-use Yocto images for the Qemu RISC-V 64-bit emulation, the HiFive Unleashed and HiFive Unmatched development boards.

As part of our work on the Yocto support for SiFive platofrms, we have already published three new releases of these layers: 2022.06, 2022.08 and 2022.10. In this blog post, we review the main highlights of those releases.

Continue reading “Updated Yocto support for SiFive RISC-V platforms”

Upcoming Yocto Project Summit 2022.11 – One talk from Bootlin

As every six months for the last two years, a new virtual edition of the Yocto Project Summit is coming, and its schedule has been announced.

This summit will be over 3 days:

Tuesday, November 29
Two tracks in parallel, a beginner track and a “hands-on” track for people already familiar with the concepts.
Wednesday, November 30
Only one track, with intermediate level talks on all kinds of topics.
Thursday, December 1
Only one track, starting with “product showcase” talks and going on with intermediate level talks on various topics too.

Last but not least, at the end of each day, you will get a chance to hangout with other contributors and users, and ask all the questions that you may have.

Bootlin is proud to contribute one talk to this summit: Bitbaking SPDX SBoM which we will prepare and present. This talk will cover one of the topics we explored to document features of the Yocto Project which had no documentation yet. SPDX and SBoM are related to software supply chain security, vulnerability management and license compliance. These are hot topics these days, and there will be another presentation about SBoMs (SBoMs and Supply Chain with the Yocto Project by Joshua Watt), and two about security (Detecting and fixing CVE security issues in yocto based embedded Linux distribution by Mikko Rapeli, and Maintenance and Security of a Yocto Project-based Distribution: A Year of Experiences by Marta Rybczynska).

Bootlin is indeed involved in the Yocto Project by maintaining its documentation (see the active contributors through the git repository), and also a participant to the Yocto SWAT team, keeping track of all the issues encountered by the autobuilder machines and runs.

Though the Yocto Project and OpenEmbedded are Open Source projects, registration for this conference is not free but just costs 40 USD, to cover infrastructure and staffing costs, the event being hosted by the Linux Foundation.

Yocto training course updated to Kirkstone release

Back in May 2022, the Yocto Project published the Kirkstone release, the latest Long Term Support version of the popular embedded Linux build system. See the release notes of this 4.0 release.

For many years, Bootlin has helped plenty of engineers around the world get started with the Yocto Project and OpenEmbedded thanks to our Yocto Project and OpenEmbedded development training course, whose training materials are freely accessible, to everyone.

We are happy to announce that we have just published the update of these materials to cover this new Kirstone release. Our practical lab instructions, available for both the STM32MP1 Discovery Kit and the BeagleBoneBlack, have been correspondingly updated:

This update was done by Bootlin engineer Luca Ceresoli, who is teaching this updated version of our course this week. The next public session, open to individual registration, will take place on September 26-30, and will be taught by Maxime Chevallier. If you are interested, you can register directly online. We also offer this course in private sessions, organized on demand for your team, either on-line or on-site, you can contact us for more details.

Using Flutter on NVidia Jetson to build graphical applications

Introduction

Flutter is an open source UI framework, released in 2017 by Google, that allows the creation of multi-platform applications, without having to worry about constraints related to supported platforms.

Flutter applications are written in a programming language called Dart, then compiled and run as a native applications, to be efficiently executed on Linux, Android, iOS or Windows platforms, but also as Web applications.

On Linux platforms, Flutter can be used on top of several graphic back-ends:

DRM
Wayland
X11

Flutter is composed of four main parts:

the embedder (C++, Java…): the glue for specific platforms that provides surface rendering, vsync.
the engine (C/C++): the graphic engine based on Skia, that provides asset resolution, graphics shell, Dart VM…
the framework (Dart): to create UI by using widgets, animation…
the applications (Dart)

In this article, we show how to build a custom Linux distribution that includes a Flutter Embedder that uses the DRM/EGLStream backend, in order to run the Flutter Gallery application on the NVIDIA Tegra Xavier NX platform.

In addition, we will also extend a Yocto SDK to embed the Flutter toolchain, to be able to build Flutter applications directly with the SDK.

Configure Yocto and build an image

To build our Flutter-enabled Linux distribution, we have chosen to use OpenEmbedded, driven through the Kas utility. Kas is a tool developed by Siemens to facilitate the setup of projects based on Bitbake, such as OpenEmbedded or Yocto.

Kas relies on a YAML file that indicates the information required to:

clone bitbake and required layers
configure the build environment
launch bitbake process

Below is the Kas YAML file that we created for this example:

header:
  version: 11

build_system: oe

machine: jetson-xavier-nx-devkit

distro: nodistro

target:
 - core-image-minimal

repos:

  bitbake:
    url: https://git.openembedded.org/bitbake
    refspec: "2.0"
    layers:
      .: excluded

  openembedded-core:
    url: https://git.openembedded.org/openembedded-core
    refspec: kirkstone
    layers:
      meta:

  meta-clang:
    url: https://github.com/kraj/meta-clang.git
    refspec: kirkstone
    layers:
      .:

  meta-flutter:
    url: https://github.com/meta-flutter/meta-flutter.git
    refspec: kirkstone
    layers:
      .:

  meta-tegra:
    url: https://github.com/OE4T/meta-tegra.git
    refspec: kirkstone
    layers:
      .:

local_conf_header:
  standard: |
    FLUTTER_RUNTIME = "release"
    FLUTTER_SDK_TAG = "3.0.1"

    DISTRO_FEATURES:append = " opengl wayland"
    REQUIRED_DISTRO_FEATURES:append = " opengl wayland"

    IMAGE_INSTALL:append = " flutter-drm-eglstream-backend flutter-gallery-release tegra-udrm-probeconf"

    TOOLCHAIN_HOST_TASK:append = " nativesdk-flutter-sdk"
    TOOLCHAIN_TARGET_TASK:append = " gtk+3-dev"

    CLANGSDK = "1"
    INIT_MANAGER ?= "systemd"

Several OpenEmbedded layers are used:

Obviously the openembedded-core layer, the base
The meta-flutter layer, which contains all the Flutter related recipes
The meta-tegra layer, which is our BSP layer containing all the bootloader/kernel and machine-specific recipes for the Nvidia Jetson Xavier NX platform
The meta-clang layer, which is needed by the meta-flutter layer, as Flutter is built using the Clang compiler

As we chose to use a DRM/EGLStream backend of Flutter we extend the DISTRO_FEATURES to enable the support of OpenGL and Wayland:

DISTRO_FEATURES:append = " opengl wayland"

In addition, we extend the list of packages that will be installed in the image with the ones that provide the Flutter embedder and the Gallery application:

IMAGE_INSTALL:append = " flutter-drm-eglstream-backend flutter-gallery-release"

Moreover, as we want to use the release 2.10.5 of Flutter, and without debug support:

FLUTTER_RUNTIME = "release"
FLUTTER_SDK_TAG = "3.0.1"

Finally, we also install the package that provides a configuration to set the correct modeset when the Tegra direct rendering module is probed.

IMAGE_INSTALL:append = " tegra-udrm-probeconf"

Using this YAML file, we can instruct Kas to launch the build:

kas build kas-flutter-example.yml

Storage flash process

Images and Nvidia tools to flash Jetson platforms are packaged into a tarball built and deployed as a target image into the folder build/tmp-glibc/deploy/images.

The SD card image for the Jetson Xavier NX can be flashed in two different ways:

from the target board,
from the host.

Here, we explain how to flash it from the target board.

Note: Each Jetson model has its own particular storage layout.

First, we need to extract the tegraflash archive:

mkdir tegraflash                                                                                                                                                                                                                          
cd tegraflash                                                                                                                                                                                                                             
tar -xvzf ../build/tmp-glibc/deploy/images/jetson-xavier-nx-devkit/core-image-minimal-jetson-xavier-nx-devkit.tegraflash.tar.gz

Moreover, to be able to flash the Jetson Xavier NX, it is required to switch it in recovery mode. For that it is necessary to connect a jumper between the 3rd and 4th pins from the right hand side of the “button header” underneath the back of the module (FRC and GND; see the labeling on the underside of the carrier board).

With this done, the module will power up in recovery mode automatically and will be visible from the host PC as an additional USB device:

lsusb |egrep 0955
Bus 003 Device 047: ID 0955:7e19 NVIDIA Corp.

Overall, here are the steps to follow to flash the SD card and the SPI Nand:

Start with your Jetson powered off.
Enable the recovery mode, as indicated above.
Connect the USB cable from your Jetson to your development host.
Insert an SD card into the slot on the module.
Power on the Jetson and put it into recovery mode.
Execute ./doflash.sh from the extracted tegraflash archive.

Finally, the target will boot.

Launch the Flutter application

Now, it is possible to start the Flutter Gallery application which is part of the core-image-minimal image, together with the Flutter stack, with the following command:

flutter-drm-eglstream-backend -b /usr/share/gallery

Customize a Yocto SDK

The OpenEmbedded build system can also be used to generate an application development SDK, that is a self-extracting tarball containing a cross-development toolchain, libraries and headers. This allows application developers to build, deploy and debug applications without having to do the OpenEmbedded build themselves.

It is possible to enrich the SDK’s sysroots with additional packages, through the variables TOOLCHAIN_HOST_TASK and TOOLCHAIN_TARGET_TASK.
That allows for example to extend the SDK with for example profiling tools, debug tools, symbols to be able to debug offline.

So, we used these variables to append the Flutter SDK and required dependencies to the Yocto SDK, to be able to cross-build Flutter applications with it.

TOOLCHAIN_HOST_TASK:append = " nativesdk-flutter-sdk"
TOOLCHAIN_TARGET_TASK:append = " gtk+3-dev"

To build the SDK with the same setup as the image previously built, we invoke Kas as follows:

kas shell kas-flutter-example.yml -c "bitbake -fc populate_sdk core-image-minimal"

Deploy the SDK

The SDKs built by OpenEmbedded are deployed in the folder build/tmp-glibc/deploy/sdk, so they can be extracted as follows to a folder:

build/tmp-glibc/deploy/sdk/oecore-x86_64-armv8a-toolchain-nodistro.0.sh -y -d ${destination}

To use the SDK, it is required to source the environment setup script that will set some cross-compile variables, like CC, LD, GDB, in the shell environment to develop or debug applications with SDK’s sysroots:

source <destination>/environment-setup-armv8a-oe-linux

Cross-build the Flutter gallery application

To illustrate how to use the SDK, let’s see how to build the Gallery Flutter application with the Yocto SDK. Before calling the flutter command, the SDK environment-setup script has been sourced and the following environment variables have been set:

FLUTTER_SDK: the path to the Flutter SDK into the Yocto SDK,
ENGINE_SDK: where the Flutter engine shall be built,
PATH: to extend the shell environment with Flutter tools provided by the SDK.

We can then retrieve the application source code:

git clone git@github.com:flutter/gallery.git
cd gallery
git checkout 9eb785cb997ff56c46e933c1c591f0a6f31454f6

Here, it is a workaround, that allows the flutter command line to correctly find the version of Flutter SDK:

export SDK_ROOT=/sysroots/x86_64-oesdk-linux/usr/share/flutter/sdk
git config --global --add safe.directory $SDK_ROOT
chmod a+rw $SDK_ROOT -R
rm -rf ${SDK_ROOT}/bin/cache/pkg/sky_engine/

Without the workaround above, the following error is raised:

The current Flutter SDK version is 0.0.0-unknown.
[...]
Failed to find the latest git commit date: VersionCheckError: Command exited with code 128: git -c log.showSignature=false log -n 1 --pretty=format:%ad --date=iso
Standard out:
Standard error: error: object directory build/downloads/git2/github.com.flutter.flutter.git/objects does not exist; check .git/objects/info/alternates
fatal: bad object HEAD
Returning 1970-01-01 01:00:00.000 instead.
[...]

Set the required environment variables and build the application for Linux:

export FLUTTER_SDK="${destination}/sysroots/x86_64-oesdk-linux/usr/share/flutter/sdk"
export PATH=${FLUTTER_SDK}/bin:$PATH
export ENGINE_SDK="./engine_sdk/sdk"

flutter config --enable-linux-desktop
flutter doctor -v
flutter build linux --release
flutter build bundle

This gives you the Flutter application, ready to run on the target!

Conclusion

In this blog post, we have shown that deploying Flutter on an OpenEmbedded distribution was a relatively easy process, and that the SDK can be extended to allow building Flutter applications.

Bootlin at the Embedded Linux Conference North America 2022

Bootlin CEO Thomas Petazzoni and COO Alexandre Belloni will both be attending the next Embedded Linux Conference North America, on June 21-24 in Austin, Texas.

In addition, both Thomas and Alexandre will be speaking at the event:

Thomas Petazzoni will give a talk Buildroot: what’s new?, providing an update on the improvements and new features in the Buildroot build system that have been integrated over the past two years
Alexandre Belloni will give a talk Yocto Project Autobuilders and the SWAT Team, during which he will explain what’s happening behind in the scenes in the Yocto Project to review and validate contributions before they are integrated.

Thomas and Alexandre will also naturally be available during the event to discuss business or career opportunities, so do not hesitate to get in touch if you’re interested.

Finally, prior to the event, Thomas Petazzoni will be in the Bay Area on June 13-15, also available for meetings or discussions.

Maintaining Yocto Project Documentation

For many years, Bootlin has been a strong user and a contributor to the Yocto Project, delivering numerous customized embedded Linux distributions and Board Support Packages based on Yocto Project and OpenEmbedded to its customers, for a wide range of hardware platforms and architectures.

In 2021, we have been able to bring this engagement further, as Bootlin engineer Michael Opdenacker has been given the opportunity to work as a maintainer for Yocto Project’s documentation, thanks to funding from the Linux Foundation. Since the mourning of Scott Rifenbark, the former maintainer, in early 2020, the project was in need for someone to fill this role.

Yocto Project developers and contributors did their best in the meantime though, in particular by migrating the documentation sources from the DocBook format to reStructuredText, to generate documentation using Sphinx. This migration was also done by the Linux kernel community, as reStructuredText seems to make it easier for developers to contribute to documentation.

For Michael Opdenacker, the interest of getting back to Yocto Project and OpenEmbedded was strong: he was one of the early users, over 15 years ago, of BitBake and OpenEmbedded. Many things have changed since Michael was generating his own OPIE and GPE Palmtop Environment images for handheld devices back in 2004 and 2005.

Indeed, since that time, under the Yocto Project umbrella, BitBake and OpenEmbedded have formed what’s probably the smartest, most versatile and most powerful embedded Linux build system. This build system takes a pretty steep learning curve though, and that’s why maintaining high quality documentation was a key focus from the start.

In addition to acting as a maintainer (reviewing and merging patches from contributors, encouraging contributions, managing documentation bugs, keeping the documentation current with the evolution of the code), we have so far also managed to:

Create brand new documentation, thanks to studying the source code and to knowledge from core developers. In particular, we documented vulnerability (CVE) management and drafted a first description of Hash Equivalence.
Implement a new infrastructure to generate diagrams from SVG sources. Currently, most graphics are bitmaps and do not render well in PDF and EPUB output. The next step is to migrate all diagrams to SVG.
Implement styling improvements, trying to eliminate unnecessarily complex sentences and make the documents easier to read.
Propose a “Documentation standards” document in markdown format, together with a diagram template with reusable shapes, text boxes and clipart, so that new documentation and diagrams are created in a way that’s consistent with existing documents.
Find bugs and improvement opportunities, and implement some of them, thanks to testing the software and reading the code.

Here are our contributions to the project so far, as of Oct. 20, 2021:

Yocto Project documentation: 71 commits
Bitbake documentation: 10 commits
openembedded-core: 11 commits
meta-openembedded: 5 commits

Yocto Project is a very welcoming open-source project, in which core developers are keen to help users and contributors. This makes it a real pleasure to participate to this project, in addition to the satisfaction to contribute to the progress of Embedded Linux.

If you are interested in studying Yocto Project and Bitbake’s documentation and contributing to it, you will be most welcome! The best is to get in touch with our community through the docs mailing list or chat with us on IRC. See our mailing lists and chat page for details.

If you’re interested in using Yocto Project for your embedded Linux projects, we also recommend you to check out our Yocto Project and OpenEmbedded system development training course, and its freely available training materials. We recently announced new dates for our upcoming online training classes on the Yocto Project:

January 31, February 1, 2, 3, 2022 from 09:00 to 13:00 UTC+1, which is a time zone suitable for participants in Europe, Middle East and Asia.
March 7, 8, 9, 10, 2022 from 18:00 to 22:00 UTC+1, which is a time zone for participants in America.

Videos and slides from Bootlin talks at Embedded Linux Conference Europe 2020

The Embedded Linux Conference Europe took place online last week. While we definitely missed the experience of an in-person event, we strongly participated to this conference with no less than 7 talks on various topics showing Bootlin expertise in different fields: Linux kernel development in networking, multimedia and storage, but also build systems and tooling. We’re happy to be publishing now the slides and videos of our talks.

From the camera sensor to the user: the journey of a video frame, Maxime Chevallier