Support for Yocto buildtools added to upstream Kas

Yocto Buildtools in Kas Bootlin has recently contributed to the Kas project by adding support for Yocto Project’s Buildtools. In this blog post, we will give an overview of what Kas is, what Buildtools are, why an integration of Buildtools was deemed useful and relevant, and how to use it.

Kas and buildtools: why?

Kas is an open-source tool developed by Siemens, which simplifies the setup of Bitbake-based projects. It allows to clone and checkout bitbake layers, create default bitbake settings (machine, arch, …), launch a minimal build environment and initiate the bitbake build process. It has become very popular in the Yocto ecosystem, and we use it extensively at Bootlin for the Yocto-based projects that we develop and deliver to our customers.

On the other hand, the OpenEmbedded build system requires certain tools on the host machine, such as Git, GCC, Python, and Make. If your system does not provide the minimum required versions, you can download an archive provided by the Yocto Project, called buildtools, that includes them. This archive also provides an environment setup script, similar to a Yocto SDK. For more details, see the documentation.

It turned out that one of our customers was exactly in this situation: their build environment does not allow containerized solutions or system updates. As a result, they are limited to a set of outdated host tools, which restricts development in the long run, especially when new Yocto versions are released and migrations are required. Using buildtools allows to solve this problem, as it provides sufficiently recent versions of the important host tools needed for an OpenEmbedded build to proceed successfully.

There are multiple ways of obtaining Buildtools: one can rely on the install-buildtools script from OpenEmbedded, manually downloading it (e.g., with wget), and even building it from scratch.

However, all of these approaches add extra steps to the build process, while Kas is precisely designed to automate the complete build process. Therefore, the idea behind supporting Buildtools in Kas is to make sure that the build process stays as simple as invoking kas build, with no extra manual steps (which could potentially affect reproducibility) needed.

Our contribution to Kas

We contributed this integration of buildtools in Kas through a number of commits:

All of those commits are part of the recently released Kas 5.0.

Using Buildtools in Kas

The Kas documentation has been updated to explain how to use the buildtools integration. But essentially, it is all about adding an extra buildtools dict in the YAML file, with the following properties:

version: the Buildtools version tag (mandatory)
sha256sum: the expected checksum of the downloaded tarball (mandatory)
base_url and filename: used to download a custom buildtools archive, not coming from the official Yocto Project servers (optional)

Based on this information, Kas will download the specified Buildtools tarball, verify its integrity using the hash, then extracts it. Finally, Kas runs the environment setup script, which updates the PATH variable (along with other relevant environment variables) so that the programs provided by Buildtools take precedence over those from the host system.

Here is an example YAML file:

repos:
  poky:
    url: https://git.yoctoproject.org/poky.git
    # when specifying a tag, optionally provide a commit hash
    tag: yocto-5.2
    commit: 9b96fdbb0cab02f4a6180e812b02bc9d4c41b1a5
    signed: true
    allowed_signers:
      - YoctoBuildandRelease
    layers:
      meta:
      meta-poky:

buildtools:
  version: "5.2"
  sha256sum: "6f69fba75c8f3142bb49558afa3ed5dd0723a3beda169b057a5238013623462d"

Conclusion

Thanks to this contribution, Kas users can run their builds with even less concern about the host machine. Containers are already supported through kas-container, and users can now combine them with Buildtools, or rely on Buildtools alone.