ELBE: automated building of Ubuntu images for a Raspberry Pi 3B

Building embedded Linux systems

ELBETypical embedded Linux systems include a wide number of software components, which all need to be compiled and integrated together. Two main approaches are used in the industry to integrate such embedded Linux systems: build systems such as Yocto/OpenEmbedded, Buildroot or OpenWrt, and binary distributions such as Debian, Ubuntu or Fedora. Of course, both options have their own advantages and drawbacks.

One of the benefits of using standard binary distributions such as Debian or Ubuntu is their widespread use, their serious and long-term security maintenance and their large number of packages. However, they often lack appropriate tools to automate the process of creating a complete Linux system image that combines existing binary packages and custom packages.

In this blog post, we introduce ELBE (Embedded Linux Build Environment), which is a build system designed to build Debian distributions and images for the embedded world. While ELBE was initially focused on Debian only, Bootlin contributed support for building Ubuntu images with ELBE, and this blog post will show as an example how to build an Ubuntu image with ELBE for a Raspberry Pi 3B.

ELBE base principle

When you first run ELBE, it creates a Virtual Machine (VM) for building root filesystems. This VM is called initvm. The process of building the root filesystem for your image is to submit and XML file to the initvm, which triggers the building of an image.

The ELBE XML file can contain an archive, which can contain configuration files, and additional software. It uses pre-built software in the form of Debian/Ubuntu packages (.deb). It is also possible to use custom repositories to get special packages into the root filesystem. The resulting root file system (a customized Debian or Ubuntu distribution) can still be upgraded and maintained through Debian’s tools such as APT (Advanced Package Tool). This is the biggest difference between ELBE and other build systems like the Yocto Project and Buildroot.

Bootlin contributions

As mentioned in this blog post introduction, Bootlin contributed support for building Ubuntu images to ELBE, which led to the following upstream commits:

Build an Ubuntu image for the Raspberry Pi 3B

We are now going to illustrate how to use ELBE by showing how to build an image for the popular RaspberryPi 3B platform.

Add required packages

This was tested on Ubuntu 20.04. Install the below packages if needed, and make sure you are in the libvirt, libvirt-qemu and kvm groups:

$ sudo apt install python3 python3-debian python3-mako \
  python3-lxml python3-apt python3-gpg python3-suds \
  python3-libvirt qemu-utils qemu-kvm p7zip-full \
  make libvirt-daemon libvirt-daemon-system \
  libvirt-clients python3-urwid
$ sudo adduser youruser libvirt 
$ sudo adduser youruser libvirt-qemu
$ sudo adduser youruser kvm
$ newgrp libvirt
$ newgrp libvirt-qemu
$ newgrp kvm

Prepare ELBE initvm

First, you need to clone ELBE’s git reposority:

git clone https://github.com/Linutronix/elbe.git

We need to use the v13.2 version because our latest contributions for Ubuntu support made it to 13.2:

$ cd elbe
$ git checkout v13.2

To create the initvm:

$ PATH=$PATH:$(pwd)
$ elbe initvm create --devel

The --devel parameter allows to use ELBE from the current working directory into the initvm.

If the command fails with the Signature with unknown key: message you need to add these keys to apt. Use the following command where XXX is the key to be added:

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys XXX

Creating your initvm should take at least 10 to 20 minutes.

In case you rebooted your computer or stopped the VM, you will need to start it:

$ elbe initvm start

Create an ELBE project for our Ubuntu image.

To begin with, we will base our image on the armhf-ubuntu example. We create an ELBE pbuilder project and not a simple ELBE project because we later want to build our own Linux kernel package for our board:

$ elbe pbuilder create --xmlfile=examples/armhf-ubuntu.xml \
  --writeproject rpi.prj --cross

The project identifier is written to rpi.prj. We save the identifier to a shell variable to simplify the next ELBE commands:

$ PRJ=$(cat rpi.prj)

Build the Linux package

As explained earlier we want to use ELBE to build our package for the Linux kernel. ELBE uses the standard Debian tool pbuilder to build packages. Therefore, we need to have debianized sources (i.e sources with the appropriate Debian metadata in a debian/ subfolder) to build a package with pbuilder.

First clone the Linux repositories:

$ git clone -b rpi-5.10.y https://github.com/raspberrypi/linux.git
$ cd linux

Debianize the Linux repositories. We use the elbe debianize command to simplify the generation of the debian folder:

$ elbe debianize

Fill the settings in the UI as follows (make sure you reduce the font size if you don’t see the Confirm button):

Make sure you set Name to rpi. Otherwise, you won’t get the output file names we use in the upcoming instructions.

The debianize command helps to create the skeleton of the debian folder in the sources. It has been pre-configured for a few packages like bootloaders or the Linux kernel, to create the rules to build these packages. It may need further modifications to finish the packaging process. Take a look a the manual to have more information on debianization. In our case, we need to tweak the debian/ folder with the two following steps to cross-build the Raspberry Linux kernel without error.

Append the below lines to the debian/rules file (use tabs instead of spaces):

override_dh_strip:
	dh_strip -Xscripts

override_dh_shlibdeps:
	dh_shlibdeps -Xscripts

Remove the following line from the debian/linux-image-5.10-rpi.install file:

./lib/firmware/*

Update the source format:

$ echo "1.0" > debian/source/format

The Linux kernel sources are now ready, we can run elbe pbuilder to compile them:

$ mkdir ../out
$ elbe pbuilder build --project $PRJ --cross --out ../out

According to how fast your system is, this can run for hours!

If everything ends well without error the out/ directory has been filled with output files:

$ ls ../out
linux-5.10-rpi_1.0_armhf.buildinfo
linux-5.10-rpi_1.0_armhf.changes
linux-5.10-rpi_1.0.dsc
linux-5.10-rpi_1.0.tar.gz
linux-headers-5.10-rpi_1.0_armhf.deb
linux-image-5.10-rpi_1.0_armhf.deb
linux-libc-dev-5.10-rpi_1.0_armhf.deb

Update the Ubuntu XML image description

Now we have our Linux kernel packaged we can move on to the image generation. Since we started from examples/armhf-ubuntu.xml, we will modify this file to fit our needs.

We begin by adding the Linux kernel package to the XML image description in the pkg-list node:


<pkg-list>
...
	<pkg>linux-image-5.10-rpi</pkg>
...
</pkg-list>

We also have to add the Device Tree to the boot/ directory because the Linux kernel package installs all the Device Trees into the /usr/lib directory.

This change is part of the rootfs modifications, therefore it is described under the finetuning XML node. We also rename the kernel image to kernel.img:


<finetuning>
...
	<cp path="/usr/lib/linux-image-5.10-rpi/bcm2710-rpi-3-b.dtb">/boot/bcm2710-rpi-3-b.dtb</cp>
	<cp path="/usr/lib/linux-image-5.10-rpi/overlays">/boot/overlays</cp>
	<mv path="/boot/vmlinuz-5.10-rpi">/boot/kernel.img</mv>
...
</finetuning>

We want to use an SD card on our Raspberry Pi, so we have to describe the partitioning of our image. For this purpose, we add the images and the fstab XML nodes to the target XML node:


<target>
...
	<images>
		<msdoshd>
			<name>sdcard.img</name>
			<size>1500MiB</size>
				<partition>
					<size>50MiB</size>
					<label>boot</label>
					<bootable/>
				</partition>
				<partition>
					<size>remain</size>
					<label>rfs</label>
				</partition>
		</msdoshd>
	</images>
	<fstab>
		<bylabel>
			<label>rfs</label>
			<mountpoint>/</mountpoint>
			<fs>
				<type>ext2</type>
			</fs>
		</bylabel>
		<bylabel>
			<label>boot</label>
			<mountpoint>/boot</mountpoint>
			<fs>
				<type>vfat</type>
			</fs>
		</bylabel>
	</fstab>
...
</target>

The Raspberry Pi board also needs firmware binaries and configurations file to boot properly. We will use the overlay directory to add these Raspberry firmware files to the image:

$ mkdir -p overlay/boot
$ cd overlay/boot
$ wget https://github.com/raspberrypi/firmware/raw/1.20210201/boot/bootcode.bin
$ wget https://github.com/raspberrypi/firmware/raw/1.20210201/boot/start.elf
$ wget https://github.com/raspberrypi/firmware/raw/1.20210201/boot/fixup.dat
$ echo "console=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootwait" > cmdline.txt
$ echo "dtoverlay=miniuart-bt" > config.txt

ELBE stores the overlay uuencoded in the XML file using the chg_archive command:

$ elbe chg_archive examples/armhf-ubuntu.xml overlay

The archive node got created in the XML file.

To tell ELBE that the XML file has changed, you need to send it to the initvm:

$ elbe control set_xml $PRJ examples/armhf-ubuntu.xml

Then build the image with ELBE:

$ elbe control build $PRJ
$ elbe control wait_busy $PRJ

Finally, if the build completes successfully, you can retrieve the image file from the initvm:

$ elbe control get_files $PRJ
$ elbe control get_file $PRJ sdcard.img.tar.gz

Now you can flash the SD card image:

$ tar xf sdcard.img.tar.gz
$ dd if=sdcard.img of=/dev/sdX bs=1M

And boot the board with root and foo as login and password:

Ubuntu 18.04.1 LTS myUbuntu ttyAMA0

myUbuntu login: root
Password: 
Welcome to Ubuntu 18.04.1 LTS (GNU/Linux 5.10-rpi armv7l)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

root@myUbuntu:~# 

Note: Ubuntu cannot be built for Raspberry A, B, B+, 0 and 0W according to https://wiki.ubuntu.com/ARM/RaspberryPi, as Ubuntu targets the ARMv7-A architecture, while the older RaspberryPi use an ARMv6 processor.

Further details

Author: Köry Maincent

Köry Maincent is an embedded Linux and kernel engineer at Bootlin, which he joined in 2020.

12 thoughts on “ELBE: automated building of Ubuntu images for a Raspberry Pi 3B”

  1. Hello,

    It’s a bit sad, there are raspi3-firmware (renamed to raspi-firmware on new versions) on debian but no on ubuntu, but on ubuntu linux-firmware-raspi2 should do the same things for firmware files, avoid downloading with wget.
    Why you didn’t use u-boot with flash-kernel ? It’s already packaged.

    With debian you can use this file /etc/default/raspi-firmware to generate the cmdline.txt and use it with u-boot and flash-kernel, that’s make the system easy to upgrade.

    Thanks.

    1. Hello,
      I did not know about these packages.
      They may indeed ease the creation of the boot directory.
      Thanks for this information.

  2. Hi,
    Running through your example I get a 403 error when trying to create the vm. I’m on Ubuntu 20.04.
    elbe initvm create –devel

    –2021-06-17 11:48:33– http://ftp.de.debian.org//debian/dists/buster/main/installer-amd64/current/images/cdrom/vmlinuz
    Resolving http://ftp.de.debian.org (ftp.de.debian.org)… 141.76.2.4
    Connecting to http://ftp.de.debian.org (ftp.de.debian.org)|141.76.2.4|:80… connected.
    HTTP request sent, awaiting response… 403 Forbidden
    2021-06-17 11:48:34 ERROR 403: Forbidden.

    [ERROR] Failure to download kernel/initrd debian Package:
    [ERROR]
    [ERROR] HashValidationFailed Failed to download http://ftp.de.debian.org//debian/dists/buster/main/installer-amd64/current/images/./cdrom/vmlinuz
    [ERROR]
    [ERROR] Check Mirror configuration
    ‘elbe init’ Failed
    Giving up

    Any suggestions welcome! – thanks.

    1. The error is related to the connection of the Debian FTP server.
      The issue comes either from your network, or from an instability of the Debian server.

  3. Great job! Thanks for sharing!
    As to your work related to Rockchip RK3399, can you share some details?
    And a silly question, what if I want to build a Ubuntu rootfs only, ignoring the kernel part (which is provided by others)?

    1. Thanks for your comment!
      About my work on RK3399, the board was using a GR3399 SOC. A devicetree and a defconfig were written to follow the hardware description and the needs of the project.
      I was not in charge of the Kernel part but if you have questions I can ping the guy who was in charge of this. I have only packaged the software components like Kernel or U-boot and build the Ubuntu image through ELBE.
      In Debian world, all is managed with packages, therefore if you want a rootfs without the Kernel you just need to not add the linux-image package to your image.

  4. Thanks for this – it was easy to get things running on rpi3.

    Do you have any words of wisdom for rpi4? I’ve modified the obvious bits like kernel config in debianize menu and updated the dtb name, but the rpi 4 doesn’t show any signs of life at boot.

    Thanks!

  5. Hi,

    thanks for the blog. I have an issue when to try your steps. The initvm went well but the next steps fail.

    schwm@pluto:~/build/elbe$ elbe pbuilder create –xmlfile=examples/armhf-ubuntu.xml –writeproject rpi.prj –cross
    Creating pbuilder
    [INFO] Building pbuilder started
    [CMD] rm -rf “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder” “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder_cross”
    [CMD] mkdir -p “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder_cross/hooks.d”
    [CMD] mkdir -p “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder_cross/aptcache”
    [CMD] mkdir -p “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/aptconfdir/apt.conf.d”
    [CMD] mkdir -p “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/ccache”
    [CMD] chmod a+w “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/ccache”
    [CMD] chmod -R 755 “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder_cross/hooks.d”
    [CMD] pbuilder –create –buildplace “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder_cross” –configfile “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/cross_pbuilderrc” –aptconfdir “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/aptconfdir” –debootstrapopts –include=”git,gnupg” ;
    W: /.pbuilderrc does not exist
    W: cgroups are not available on the host, not using them.
    I: Distribution is bionic.
    I: Current time: Thu Apr 21 13:58:06 UTC 2022
    I: pbuilder-time-stamp: 1650549486
    I: Building the build environment
    I: running debootstrap
    /usr/sbin/debootstrap
    W: Cannot check Release signature; keyring file not available /usr/share/keyrings/ubuntu-archive-keyring.gpg
    E: Keyring-based check was requested; aborting accordingly
    E: debootstrap failed
    E: Tail of debootstrap.log:
    E: End of debootstrap.log
    W: Aborting with an error
    I: cleaning the build env
    I: removing directory /var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder_cross/872 and its subdirectories
    [ERROR] Pbuilder failed
    Traceback (most recent call last):
    File “/var/cache/elbe/devel/elbepack/asyncworker.py”, line 262, in execute
    self.project.create_pbuilder(self.cross, self.noccache,
    File “/var/cache/elbe/devel/elbepack/elbeproject.py”, line 879, in create_pbuilder
    do(‘pbuilder –create –buildplace “%s” ‘
    File “/var/cache/elbe/devel/elbepack/shellhelper.py”, line 258, in do
    raise CommandError(cmd, p.returncode)
    elbepack.shellhelper.CommandError: Error: 1 returned from Command pbuilder –create –buildplace “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/pbuilder_cross” –configfile “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/cross_pbuilderrc” –aptconfdir “/var/cache/elbe/2a1dcebe-b480-4e0d-ae7b-d538ab4bb0d4/aptconfdir” –debootstrapopts –include=”git,gnupg” ;
    Project build was not successful, current status: build_failed
    elbe control wait_busy Failed
    Giving up

    My distro is:

    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=20.04
    DISTRIB_CODENAME=focal
    DISTRIB_DESCRIPTION=”Ubuntu 20.04.4 LTS”

    thanks micky

    1. Hello, what version of ELBE are you using?

      You may need to update the initvm XML if you are using the last ELBE version. I had received this mail about Ubuntu keyring issue from the maintainer. I hope this can give you some hint of debugging.

      Debian bullseye is planned for release on August 14th. Currently, there is one problem with a bullseye-based initvm: https://bugs.debian.org/929165 prevents ubuntu-keyring from migrating to bullseye. I do not expect it to happen, so we should remove that package from the default initvm. Users who want to build Ubuntu-based systems are still able to build the initvm from a custom XML.

Leave a Reply