Building a Linux system for the STM32MP1: remote firmware updates

After another long break, here is our new article in the series of blog posts about building a Linux system for the STM32MP1 platform. After showing how to build a minimal Linux system for the STM32MP157 platform, how to connect and use an I2C based pressure/temperature/humidity sensor and how to integrate Qt5 in our system, how to set up a development environment to write our own Qt5 application, how to develop a Qt5 application, and how to setup factory flashing, we are now going to discuss the topic of in-field firmware update.

List of articles in this series:

  1. Building a Linux system for the STM32MP1: basic system
  2. Building a Linux system for the STM32MP1: connecting an I2C sensor
  3. Building a Linux system for the STM32MP1: enabling Qt5 for graphical applications
  4. Building a Linux system for the STM32MP1: setting up a Qt5 application development environment
  5. Building a Linux system for the STM32MP1: developing a Qt5 graphical application
  6. Building a Linux system for the STM32MP1: implementing factory flashing
  7. Building a Linux system for the STM32MP1: remote firmware updates

Why remote firmware updates?

The days and age when it was possible to build and flash an embedded system firmware, ship the device and forget it, are long behind us. Systems have gotten more complicated, and we therefore have to fix bugs and security issues after the device has been shipped, and we often want to deploy new features in the field into existing devices. For all those reasons, the ability to remotely update the firmware of embedded devices is now a must-have.

Open-source firmware update tools

There are different possibilities to update your system:

  • If you’re using a binary distribution, use the package manager of this distribution to update individual components
  • Do complete system image updates, at the block-level, replacing the entire system image with an updated one. Three main open-source solutions are available: swupdate, Mender.io and RAUC.
  • Do file-based updates, with solutions such as OSTree.

In this blog post, we are going to show how to set up the swupdate solution.

swupdate is a tool installed on the target that can receive an update image (.swu file), either from a local media or from a remote server, and use it to update various parts of the system. Typically, it will be used to update the Linux kernel and the root filesystem, but it can also be used to update additional partitions, FPGA bitstreams, etc.

swupdate implements two possible update strategies:

  • A dual copy strategy, where the storage has enough space to store two copies of the entire filesystem. This allows to run the system from copy A, update copy B, and reboot it into copy B. The next update will of course update copy A.
  • A single copy strategy, where the upgrade process consists in rebooting into a minimal system that runs entirely from RAM, and that will be responsible for updating the system on storage.

For this blog post, we will implement the dual copy strategy, but the single copy strategy is also supported for systems with tighter storage restrictions.

We are going to setup swupdate step by step: first by triggering updates locally, and then seeing how to trigger updates remotely.

Local usage of swupdate

Add USB storage support

As a first step, in order to transfer the update image to the target, we will use a USB stick. This requires having USB mass storage support in the Linux kernel. So let’s adjust our Linux kernel configuration by running make linux-menuconfig. Within the Linux kernel configuration:

  • Enable the CONFIG_SCSI option. This is a requirement for USB mass storage support
  • Enable the CONFIG_BLK_DEV_SD option, needed for SCSI disk support, which is another requirement for USB mass storage.
  • Enable the CONFIG_USB_STORAGE option.
  • The CONFIG_VFAT_FS option, to support the FAT filesystem, is already enabled.
  • Enable the CONFIG_NLS_CODEPAGE_437 and CONFIG_NLS_ISO8859_1 options, to have the necessary support to decode filenames in the FAT filesystem.

Then, run make linux-update-defconfig to preserve these kernel configurations changes in your kernel configuration file at board/stmicroelectronics/stm32mp157-dk/linux.config.

swupdate setup

In Target packages, System tools, enable swupdate. You can disable the install default website setting since we are not going to use the internal swupdate web server.

Take this opportunity to also enable the gptfdisk tool and its sgdisk sub-option in the Hardware handling submenu. We will need this tool later to update the partition table at the end of the update process.

Now that we have both both USB storage support and the swupdate package enabled, let’s build a new version of our system by running make. Flash the resulting image on your SD card, and boot your target. You should have swupdate available:

# swupdate -h
Swupdate v2018.11.0

Licensed under GPLv2. See source distribution for detailed copyright notices.

swupdate (compiled Mar  4 2020)
Usage swupdate [OPTION]
 -f, --file           : configuration file to use
 -p, --postupdate               : execute post-update command
 -e, --select , : Select software images set and source
                                  Ex.: stable,main
 -i, --image          : Software to be installed
 -l, --loglevel          : logging level
 -L, --syslog                   : enable syslog logger
 -n, --dry-run                  : run SWUpdate without installing the software
 -N, --no-downgrading  : not install a release older as 
 -o, --output      : saves the incoming stream
 -v, --verbose                  : be verbose, set maximum loglevel
     --version                  : print SWUpdate version and exit
 -c, --check                    : check image and exit, use with -i 
 -h, --help                     : print this help and exit
 -w, --webserver [OPTIONS]      : Parameters to be passed to webserver
	mongoose arguments:
	  -l, --listing                  : enable directory listing
	  -p, --port               : server port number  (default: 8080)
	  -r, --document-root      : path to document root directory (default: .)
	  -a, --api-version [1|2]        : set Web protocol API to v1 (legacy) or v2 (default v2)
	  --auth-domain                  : set authentication domain if any (default: none)
	  --global-auth-file             : set authentication file if any (default: none)

Take a USB stick with a FAT filesystem on it, which you can mount:

# mount /dev/sda1 /mnt

If that works, we’re now ready to move on to the next step of actually getting a firmware update image.

Generate the swupdate image

swupdate has its own update image format, and you need to generate an image that complies with this format so that swupdate can use it to upgrade your system. The format is simple: it’s a CPIO archive, which contains one file named sw-description describing the contents of the update image, and one or several additional files that are the images to update.

First, let’s create our sw-description file in board/stmicroelectronics/stm32mp157-dk/sw-description. The tags and properties available are described in the swupdate documentation.

software = {
	version = "0.1.0";
	rootfs = {
		rootfs-1: {
			images: (
			{
				filename = "rootfs.ext4.gz";
				compressed = true;
				device = "/dev/mmcblk0p4";
			});
		}
		rootfs-2: {
			images: (
			{
				filename = "rootfs.ext4.gz";
				compressed = true;
				device = "/dev/mmcblk0p5";
			});
		}
	}
}

This describes a single software component rootfs, which is available as two software collections, to implement the dual copy mechanism. The root filesystem will have one copy in /dev/mmcblk0p4 and another copy in /dev/mmcblk0p5. They will be updated from a compressed image called rootfs.ext4.gz.

Once this sw-description file is written, we can write a small script that generates the swupdate image. We’ll put this script in board/stmicroelectronics/stm32mp157-dk/gen-swupdate-image.sh:

#!/bin/sh

BOARD_DIR=$(dirname $0)

cp ${BOARD_DIR}/sw-description ${BINARIES_DIR}

IMG_FILES="sw-description rootfs.ext4.gz"

pushd ${BINARIES_DIR}
for f in ${IMG_FILES} ; do
	echo ${f}
done | cpio -ovL -H crc > buildroot.swu
popd

It simply copies the sw-description file to BINARIES_DIR (which is output/images), and then creates a buildroot.swu CPIO archive that contains the sw-description and rootfs.ext4.gz files.

Of course, make sure this script has executable permissions.

Then, we need to slightly adjust our Buildroot configuration, so run make menuconfig, and:

  • In System configuration, in the option Custom scripts to run after creating filesystem images, add board/stmicroelectronics/stm32mp157-dk/gen-swupdate-image.sh after the existing value support/scripts/genimage.sh. This will make sure our new script generating the swupdate image is executed as a post-image script, at the end of the build.
  • In Filesystem images, enable the gzip compression method for the ext2/3/4 root filesystem, so that a rootfs.ext4.gz image is generated.

With that in place, we are now able to generate our firmware image, by simply running make in Buildroot. At the end of the build, the output/images/ folder should contain the sw-description and rootfs.ext4.gz files. You can look at the contents of buildroot.swu:

$ cat output/images/buildroot.swu | cpio -it
sw-description
rootfs.ext4.gz
58225 blocks

Partioning scheme and booting logic

We now need to adjust the partitioning scheme of our SD card so that it has two partitions for the root filesystem, one for each copy. This partitioning scheme is defined in board/stmicroelectronics/stm32mp157-dk/genimage.cfg, which we change to:

image sdcard.img {
        hdimage {
                gpt = "true"
        }

        partition fsbl1 {
                image = "tf-a-stm32mp157c-dk2.stm32"
        }

        partition fsbl2 {
                image = "tf-a-stm32mp157c-dk2.stm32"
        }

        partition ssbl {
                image = "u-boot.stm32"
        }

        partition rootfs1 {
                image = "rootfs.ext4"
                partition-type = 0x83
                bootable = "yes"
                size = 256M
        }

        partition rootfs2 {
                partition-type = 0x83
                size = 256M
        }
}

As explained in the first blog post of this series, the /boot/extlinux/extlinux.conf file is read by the bootloader to know how to boot the system. Among other things, this file defines the Linux kernel command line, which contains root=/dev/mmcblk0p4 to tell the kernel where the root filesystem is. But with our dual copy upgrade scheme, the root filesystem will sometimes be on /dev/mmcblk0p4, sometimes on /dev/mmcblk0p5. To achieve that without constantly updating the extlinux.conf file, we will use /dev/mmcblk0p${devplist} instead. devplist is a U-Boot variable that indicates from which partition the extlinux.conf file was read, which turns out to be the partition of our root filesystem. So, your board/stmicroelectronics/stm32mp157-dk/overlay/boot/extlinux/extlinux.conf file should look like this:

label stm32mp15-buildroot
  kernel /boot/zImage
  devicetree /boot/stm32mp157c-dk2.dtb
  append root=/dev/mmcblk0p${devplist} rootwait console=ttySTM0,115200 vt.global_cursor_default=0

For the dual copy strategy to work, we need to tell the bootloader to boot either from the root filesystem in the rootfs1 partition or the rootfs2 partition. This will be done using the bootable flag of each GPT partition, and this is what this script does: it toggles the bootable flag of 4th and 5th partition of the SD card. Thanks to this, at the next reboot, U-Boot will consider the system located in the other SD card partition. This work will be done by a /etc/swupdate/postupdate.sh script, that you will store in board/stmicroelectronics/stm32mp157-dk/overlay/etc/swupdate/postupdate.sh, which contains:

#!/bin/sh
sgdisk -A 4:toggle:2 -A 5:toggle:2 /dev/mmcblk0
reboot

Make sure this script is executable.

With all these changes in place, let’s restart the Buildroot build by running make. The sdcard.img should contain the new partioning scheme:

$ sgdisk -p output/images/sdcard.img
[...]
Number  Start (sector)    End (sector)  Size       Code  Name
   1              34             497   232.0 KiB   8300  fsbl1
   2             498             961   232.0 KiB   8300  fsbl2
   3             962            2423   731.0 KiB   8300  ssbl
   4            2424          526711   256.0 MiB   8300  rootfs1
   5          526712         1050999   256.0 MiB   8300  rootfs2

Reflash your SD card with the new sdcard.img, and boot this new system. Transfer the buildroot.swu update image to your USB stick.

Testing the firmware update locally

After booting the system, mount the USB stick, which contains the buildroot.swu file:

# mount /dev/sda1 /mnt/
# ls /mnt/
buildroot.swu

Let’s trigger the system upgrade with swupdate:

# swupdate -i /mnt/buildroot.swu -e rootfs,rootfs-2 -p /etc/swupdate/postupdate.sh

Swupdate v2018.11.0

Licensed under GPLv2. See source distribution for detailed copyright notices.

Registered handlers:
	dummy
	raw
	rawfile
software set: rootfs mode: rootfs-2
Software updated successfully
Please reboot the device to start the new software
[INFO ] : SWUPDATE successful ! 
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
# Stopping qt-sensor-demo: OK
Stopping dropbear sshd: OK
Stopping network: OK
Saving random seed... done.
Stopping klogd: OK
Stopping syslogd: OK
umount: devtmpfs busy - remounted read-only
[  761.949576] EXT4-fs (mmcblk0p4): re-mounted. Opts: (null)
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[  763.965243] reboot: ResNOTICE:  CPU: STM32MP157CAC Rev.B
NOTICE:  Model: STMicroelectronics STM32MP157C-DK2 Discovery Board

The -i option indicates the firmware update file, while the -e option indicates which software component should be updated. Here we update the rootfs in its slot 2, rootfs-2, which is in /dev/mmcblk0p5. The -p option tells to run our post-update script when the update is successful. In the above log, we see that the system is being rebooted right after the update.

At the next boot, you should see:

U-Boot 2018.11-stm32mp-r2.1 (Mar 04 2020 - 15:28:34 +0100)
[...]
mmc0 is current device
Scanning mmc 0:5...
Found /boot/extlinux/extlinux.conf
[...]
append: root=/dev/mmcblk0p5 rootwait console=ttySTM0,115200 vt.global_cursor_default=0

during the U-Boot part. So we see it is loading extlinux.conf from the MMC partition 5, and has properly set root=/dev/mmcblk0p5. So the kernel and Device Tree will be loaded from MMC partition 5, and this partition will also be used by Linux as the root filesystem.

With all this logic, we could now potentially have some script that gets triggered when a USB stick is inserted, mount it, check if an update image is available on the USB stick, and if so, launch swupdate and reboot. This would be perfectly fine for local updates, for example with an operator in charge of doing the update of the device.

However, we can do better, and support over-the-air updates, a topic that we will discuss in the next section.

Over-the-air updates

To support over-the-air updates with swupdate, we will have to:

  1. Install on a server a Web interface that allows the swupdate program to retrieve firmware update files, and the user to trigger the updates.
  2. Run swupdate in daemon mode on the target.

Set up the web server: hawkBit

swupdate is capable of interfacing with a management interface provided by the Eclipse hawkBit project. Using this web interface, one can manage its fleet of embedded devices, and rollout updates to these devices remotely.

hawkBit has plenty of capabilities, and we are here going to set it up in a very minimal way, with no authentication and a very simple configuration.

As suggested in the project getting started page, we’ll use a pre-existing Docker container image to run hawkBit:

sudo docker run -p 8080:8080 hawkbit/hawkbit-update-server:latest \
     --hawkbit.dmf.rabbitmq.enabled=false \
     --hawkbit.server.ddi.security.authentication.anonymous.enabled=true

After a short while, it should show:

2020-03-06 09:15:46.492  ... Started ServerConnector@3728a578{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-03-06 09:15:46.507  ... Jetty started on port(s) 8080 (http/1.1) with context path '/'
2020-03-06 09:15:46.514  ... Started Start in 21.312 seconds (JVM running for 22.108)

From this point, you can connect with your web browser to http://localhost:8080 to access the hawkBit interface. Login with the admin login and admin password.

hawkBit login

Once in the main hawkBit interface, go to the System Config tab, and enable the option Allow targets to download artifacts without security credentials. Of course, for a real deployment, you will want to set up proper credentials and authentification.

hawkBit System Config

In the Distribution tab, create a new Distribution by clicking on the plus sign in the Distributions panel:

hawkBit New Distribution

Then in the same tab, but in the Software Modules panel, create a new software module:

hawkBit New Software Module

Once done, assign the newly added software module to the Buildroot distribution by dragging-drop it into the Buildroot distribution. Things should then look like this:

hawkBit Distribution

Things are now pretty much ready on the hawkBit side now. Let’s move on with the embedded device side.

Configure swupdate

We need to adjust the configuration of swupdate to enable its Suricatta functionality which is what allows to connect to an hawkBit server.

In Buildroot’s menuconfig, enable the libcurl (BR2_PACKAGE_LIBCURL) and json-c (BR2_PACKAGE_JSON_C) packages, both of which are needed for swupdate’s Suricatta. While at it, since we will adjust the swupdate configuration and we’ll want to preserve our custom configuration, change the BR2_PACKAGE_SWUPDATE_CONFIG option to point to board/stmicroelectronics/stm32mp157-dk/swupdate.config.

Then, run:

$ make swupdate-menuconfig

to enter the swupdate configuration interface. Enable the Suricatta option, and inside this menu, in the Server submenu, verify that the Server Type is hawkBit support. You can now exit the swupdate menuconfig.

Save our custom swupdate configuration permanently:

$ make swupdate-update-defconfig

WIth this proper swupdate configuration in place, we now need to create a runtime configuration file for swupdate, and an init script to start swupdate at boot time. Let’s start with the runtime configuration file, which we’ll store in board/stmicroelectronics/stm32mp157-dk/overlay/etc/swupdate/swupdate.cfg, containing:

globals :
{
	postupdatecmd = "/etc/swupdate/postupdate.sh";
};

suricatta :
{
	tenant = "default";
	id = "DEV001";
	url = "http://192.168.42.1:8080";
};

We specify the path to our post-update script so that it doesn’t have to be specified on the command line, and then we specify the Suricatta configuration details: id is the unique identifier of our board, the URL is the URL to connect to the hawkBit instance (make sure to replace that with the IP address of where you’re running hawkBit). tenant should be default, unless you’re using your hawkBit instance in complex setups to for example serve multiple customers.

Our post-update script also needs to be slightly adjusted. Indeed, we will need a marker that tells us upon reboot that an update has been done, in order to confirm to the server that the update has been successfully applied. So we change board/stmicroelectronics/stm32mp157-dk/overlay/etc/swupdate/postupdate.sh to:

#!/bin/sh

PART_STATUS=$(sgdisk -A 4:get:2 /dev/mmcblk0)
if test "${PART_STATUS}" = "4:2:1" ; then
        NEXT_ROOTFS=/dev/mmcblk0p5
else
        NEXT_ROOTFS=/dev/mmcblk0p4
fi

# Add update marker
mount ${NEXT_ROOTFS} /mnt
touch /mnt/update-ok
umount /mnt

sgdisk -A 4:toggle:2 -A 5:toggle:2 /dev/mmcblk0
reboot

What we do is that we simply mount the next root filesystem, and create a file /update-ok. This file will be checked by our swupdate init script, see below.

Then, our init script will be in board/stmicroelectronics/stm32mp157-dk/overlay/etc/init.d/S98swupdate, with executable permissions, and contain:

#!/bin/sh

DAEMON="swupdate"
PIDFILE="/var/run/$DAEMON.pid"

PART_STATUS=$(sgdisk -A 4:get:2 /dev/mmcblk0)
if test "${PART_STATUS}" = "4:2:1" ; then
	ROOTFS=rootfs-2
else
	ROOTFS=rootfs-1
fi

if test -f /update-ok ; then
	SURICATTA_ARGS="-c 2"
	rm -f /update-ok
fi

start() {
	printf 'Starting %s: ' "$DAEMON"
	# shellcheck disable=SC2086 # we need the word splitting
	start-stop-daemon -b -q -m -S -p "$PIDFILE" -x "/usr/bin/$DAEMON" \
		-- -f /etc/swupdate/swupdate.cfg -L -e rootfs,${ROOTFS} -u "${SURICATTA_ARGS}"
	status=$?
	if [ "$status" -eq 0 ]; then
		echo "OK"
	else
		echo "FAIL"
	fi
	return "$status"
}

stop() {
	printf 'Stopping %s: ' "$DAEMON"
	start-stop-daemon -K -q -p "$PIDFILE"
	status=$?
	if [ "$status" -eq 0 ]; then
		rm -f "$PIDFILE"
		echo "OK"
	else
		echo "FAIL"
	fi
	return "$status"
}

restart() {
	stop
	sleep 1
	start
}

case "$1" in
        start|stop|restart)
		"$1";;
	reload)
		# Restart, since there is no true "reload" feature.
		restart;;
        *)
                echo "Usage: $0 {start|stop|restart|reload}"
                exit 1
esac

This is modeled after typical Buildroot init scripts. A few points worth mentioning:

  • At the beginning of the script, we determine which copy of the root filesystem needs to be updated by looking at which partition currently is marked “bootable”. This is used to fill in the ROOTFS variable.
  • We also determine if we are just finishing an update, by looking at the presence of a /update-ok file.
  • When starting swupdate, we pass a few options: -f with the path to the swupdate configuration file, -L to enable syslog logging, -e to indicate which copy of the root filesystem should be updated, and -u '${SURICATTA_ARGS}' to run in Suricatta mode, with SURICATTA_ARGS containing -c 2 to confirm the completion of an update.

Generate a new image with the updated swupdate, its configuration file and init script, and reboot your system.

Deploying an update

When booting, your system starts swupdate automatically:

Starting swupdate: OK
[...]
# ps aux | grep swupdate
  125 root     /usr/bin/swupdate -f /etc/swupdate/swupdate.cfg -L -e rootfs,rootfs-1 -u
  132 root     /usr/bin/swupdate -f /etc/swupdate/swupdate.cfg -L -e rootfs,rootfs-1 -u

Back to the hawkBit administration interface, the Deployment tab should show one notification:

hawkBit new device notification

and when clicking on it, you should see our DEV001 device:

hawkBit new device

Now, go to the Upload tab, select the Buildroot software module, and click on Upload File. Upload the buildroot.swu file here:

hawkBit Upload

Back into the Deployment tab, drag and drop the Buildroot distribution into the DEV001 device. A pending update should appear in the Action history for DEV001:

hawkBit upgrade pending

The swupdate on your target will poll regularly the server (by default every 300 seconds, can be customized in the System config tab of the hawkBit interface) to know if an update is available. When that happens, the update will be downloaded and applied, the system will reboot, and at the next boot the update will be confirmed as successful, showing this status in the hawkBit interface:

hawkBit upgrade confirmed

If you’ve reached this step, your system has been successfully updated, congratulations! Of course, there are many more things to do to get a proper swupdate/hawkBit deployment: assign unique device IDs (for example based on MAC addresses or SoC serial number), implement proper authentication between the swupdate client and the server, implement image encryption if necessary, improve the upgrade validation mechanism to make sure it detects if the new image doesn’t boot properly, etc.

Conclusion

In this blog post, we have learned about firmware upgrade solutions, and specifically about swupdate. We’ve seen how to set up swupdate in the context of Buildroot, first for local updates, and then for remote updates using the hawkBit management interface. Hopefully this will be useful for your future embedded projects!

As usual, the complete Buildroot code to reproduce the same setup is available in our branch 2019.02/stm32mp157-dk-blog-7, in two commits: one for the first step implementing support just for local updates, and another one for remote update support.

Bootlin at Embedded World 2020

STMicro at Embedded World 2020Bootlin will be preent at Embedded World 2020, in Nuremberg on February 25-27. We will be present on STMicroelectronics booth in hall 4A, stand 138. We will have two demos of the STM32MP1 platform running Linux, and of course details about Bootlin services around embedded Linux and Linux kernel development and training.

Three people from Bootlin will be present: Michael Opdenacker (CEO), Thomas Petazzoni (CTO) and Alexandre Belloni (embedded Linux engineer and trainer).

Embedded World 2020

Do not hesitate to get in touch with us prior to the event if you would like to schedule a meeting to discuss business, project or career opportunities.

Linux 5.5 released, Bootlin contributions inside

Linux 5.5 was recently released, as usual bringing a large number of new features and improvements, which are nicely detailed in the LWN articles on merge window part 1 and merge window part 2, but also on the Kernelnewbies wiki.

According to the statistics, a total of 14350 changes were made to this kernel release, to which Bootlin contributed 124 patches, making us the 19th contributing company by number of commits. Here are the highlights of our contributions:

  • By far and large our most important achievement in Linux 5.5 is the merge of the H265 decoding support in the Allwinner VPU driver, developed by Paul Kocialkowski. This was the last missing feature to complete the effort funded by the Kickstarter campaign we launched in February 2018. See our blog post wrapping up the Allwinner VPU work.
  • Alexandre Belloni as the RTC subsystem maintainer, as usual contributed a large number of RTC driver improvements and fixes.
  • Antoine Ténart contributed some improvements to the Cadence MACB driver, most notably used as the Ethernet controller driver on Microchip/Atmel platforms. The main improvement is the conversion to the phylink subsystem for the interaction with the Ethernet PHY.
  • Grégory Clement contributed numerous enhancements to the Microchip/Atmel SPI controller driver, mainly aimed at fixing and improving the support for Chip Select, both native Chip Selects and GPIO-based Chip Selects.
  • Kamel Bouhara contributed a few additional Device Tree files to describe home automation hardware platforms from Overkiz, based on the Microchip SAMA5D2 processor, as well as an improvement to a Microchip SoC driver that allows to expose the platform serial number to user-space.
  • Miquèl Raynal added support for a new Marvell system-on-chip called Marvell CN9130. Despite the name, this chip is from a hardware point of view in the same family as the Marvell Armada 7K and 8K, which were already supported upstream.
  • Miquèl Raynal contributed a number of fixes and improvements to the Macronix SPI controller driver.
  • Miquèl Raynal added support for the ADC converters over SPI MAX1227, MAX1229 and MAX1231 to the existing max1027 IIO driver in the kernel. These ADCs have a 12-bit resolution.
  • Miquèl Raynal improved the SPI controller driver for the Zynq system-on-chips to correctly support multiple Chip Selects.

In addition to these direct contributions, some of the Bootlin engineers are also Linux kernel maintainers and therefore review and merge patches from other contributors: Alexandre Belloni as the RTC subsystem maintainer and Microchip platform co-maintainer reviewed and merged 45 patches from other contributors, Miquèl Raynal as the MTD subsystem co-maintainer reviewed and merged 39 patches from other contributors and Grégory Clement as the Marvell platform co-maintainer reviewed and merged 29 patches from other contributors.

Here is the detail of our contributions:

Bootlin at FOSDEM and Buildroot Developers Meeting

FOSDEM 2020This week-end takes place one of the biggest and most important free and open-source software conference in Europe: FOSDEM. It will once again feature a very large number of talks, organized in several main tracks and developer rooms.

Bootlin CTO Thomas Petazzoni will participate to the FOSDEM conference, of course attending many of the talks from the Embedded, Mobile and Automative Devroom, to which he participated to the talk review and selection. Do not hesitate to get in touch with Thomas if you want to discuss career or business opportunities with Bootlin.

In addition, Thomas will also participate to the 3-day Buildroot Developers meeting which takes place in Brussels right after the FOSDEM conference, kindly hosted by Google. During 3 days, some of the core Buildroot developers will work together to discuss the future of Buildroot, as well as review and discuss pending patches and proposals.

Building a Linux system for the STM32MP1: implementing factory flashing

After several months, it’s time to resume our series of blog posts about building a Linux system for the STM32MP1 platform. After showing how to build a minimal Linux system for the STM32MP157 platform, how to connect and use an I2C based pressure/temperature/humidity sensor and how to integrate Qt5 in our system, how to set up a development environment to write our own Qt5 application and how to develop a Qt5 application, we will now cover the topic of factory flashing.

List of articles in this series:

  1. Building a Linux system for the STM32MP1: basic system
  2. Building a Linux system for the STM32MP1: connecting an I2C sensor
  3. Building a Linux system for the STM32MP1: enabling Qt5 for graphical applications
  4. Building a Linux system for the STM32MP1: setting up a Qt5 application development environment
  5. Building a Linux system for the STM32MP1: developing a Qt5 graphical application
  6. Building a Linux system for the STM32MP1: implementing factory flashing
  7. Building a Linux system for the STM32MP1: remote firmware updates

What is factory flashing ?

So far, we have used a microSD card as storage for the Linux system running on the STM32MP1 platform. Since this media is removable, we can easily switch the microSD card back and forth between the STM32MP1 platform and our development workstation, which is nice during development and debugging.

However, an actual product will most likely use some form of non-removable persistent storage, typically an eMMC or a NAND flash. While not available on the STM32MP1-DK1 board probably for cost reasons, these storage devices are very common in most embedded systems. For example, the STM32MP157A-EV1 board provides three non-removable persistent storage devices: a 4 GB eMMC, a 1 GB NAND flash, and a 64 MB QSPI NOR flash.

When such storage devices are shipped by their manufacturer, they are typically empty. Therefore, as part of the manufacturing process of your embedded systems, you will have to load the relevant storage device with your Linux system, applications and data, so that the embedded system is fully operational: this is the process referred to as factory flashing in this blog post.

If you are doing a very high volume product, you can ask your eMMC or NAND flash vendor to pre-load a system image on the storage before it is shipped to you and assembled on your board. However, many companies do products with volumes that are not large enough to make such a strategy possible: in this case, you really receive an empty storage device, and have to flash it.

A first possibility to flash the non-removable storage is to use a removable storage device, boot a Linux system on the device, and use it to flash the non-removable storage. This can definitely be a possible option in some situations, but it is not always possible (if there’s no removable storage device interface at all) or not always practical.

However, most system-on-chips, including the STM32MP1 include some ROM code that the processor executes at boot time, even before it loads the first stage bootloader. This ROM code is primarly responsible for loading the first stage bootloader into memory, but it also very often offers a communication channel with the outside world, which can be used to gain control of a platform that has nothing at all on its storage. This communication channel is typically over USB or UART, and most often uses a custom, vendor-specific protocol, which is understood by vendor-specific tools. This protocol generally allows to send some code to the target and get it executed, which is sufficient to be able to reflash the target device.

Here are a few examples with system-on-chips from various vendors:

  • The ROM code of the NXP i.MX processors implements a USB-based protocol, which can be interfaced either using the NXP-provided mfgtools, or using the community-developed imx_usb_loader. The latter was presented in one of our earlier blog posts about i.MX6 factory flashing.
  • The ROM code of Microchip SAMA5 processors implements a USB-based protocol, which can be interfaced using a tool called SAM-BA
  • The ROM code of Rockchip processors implements a USB-based protocol, which can be interfaced either using a Rockchip-specific tool called rkdeveloptool
  • The ROM code of the ST STM32MP15 processors also implement a USB-based protocol, which can be interfaced using the STM32 Cube Programmer

Obviously, in this blog post, we are going to use the latter, STM32 Cube Programmer, to flash our STM32MP1 platform. Since the DK2 board only has a removable device, we will use the tool to flash the SD card, but the process and logic would be the same for any other (non-removable) storage device.

Getting and installing STM32 Cube Programmer

While ST generally has very good upstream and open-source support for its products, the STM32 Cube Programmer unfortunately doesn’t follow this strategy: you need to be registered on the ST web site to download it, and its source code is not available. Due to this registration process, we for example cannot create a Buildroot package that would automatically download and install this tool for you.

So, follow the process to create an account on the ST web site, and then go to the STM32 Cube Programmer page. At the time of this writing, the latest version is 2.2.1, but according this Wiki page, this version doesn’t work for the STM32MP1 platform. Instead, select to download the 2.2.0 version, which is known to work. You will then download a file called en.stm32cubeprog.zip (it doesn’t have the version in its name, which isn’t great) weighting 187 MB, and which has the SHA256 hash 91107b4d605d126f5c32977247d7419d42abb2655848d2d1a16d52f5f7633d2d.

Extract this ZIP file somewhere in your system, and then run the SetupSTM32CubeProgrammer-2.2.0.linux executable:

$ ./SetupSTM32CubeProgrammer-2.2.0.linux

Got through the installation steps. On our system, we customized the installation path to be just $HOME/stm32cube, and the remainder of this blog post will assume this is where you installed the STM32 Cube Programmer.

In this blog post, we are only going to use the command line interface (CLI) of STM32 Cube Programmer, so just make sure you can run the corresponding tool:

$ ~/stm32cube/bin/STM32_Programmer_CLI
      -------------------------------------------------------------------
                        STM32CubeProgrammer v2.2.0
      -------------------------------------------------------------------


Usage :
STM32_Programmer_CLI.exe [command_1] [Arguments_1][[command_2] [Arguments_2]...]
[...]

Testing the communication with the board

On the back of the board, there is a two-way DIP switch labeled SW1, which is used to configure the boot mode. When both are “ON”, the board boots from the SD card. When both are “OFF”, the board enters the “USB boot for flashing mode”, which is what we are going to use. So switch both switches to OFF, and reset the board.

Plug an additional USB-C cable from the board CN7 connector (which is located between the HDMI port and the 4 USB host ports).

USB-C connection for factory flashing

Then, reset the board. If you run lsusb on your Linux workstation, you should see a new device:

Bus 003 Device 011: ID 0483:df11 STMicroelectronics STM Device in DFU Mode

Then, you can ask STM32_Programmer_CLI to list the devices it sees over USB. This needs root permissions (unless appropriate udev rules are created):

$ sudo ~/stm32cube/bin/STM32_Programmer_CLI -l usb
      -------------------------------------------------------------------
                        STM32CubeProgrammer v2.2.0
      -------------------------------------------------------------------

=====  DFU Interface   =====

Total number of available STM32 device in DFU mode: 1

  Device Index           : USB1
  USB Bus Number         : 003
  USB Address Number     : 003
  Product ID             : DFU in HS Mode @Device ID /0x500, @Revision ID /0x0000
  Serial number          : 004200343338510534383330
  Firmware version       : 0x0110
  Device ID              : 0x0500

Good, the STM32CubeProgrammer tool is seeing our board, and we see that the Device Index is USB1. Keep that in mind for the next steps.

Change the Linux system boot chain

STM32CubeProgrammer works by sending a U-Boot bootloader over USB, and then talking to this U-Boot to make it erase the MMC or NAND flash, and make it write some data to those storage devices. However, for some reason, STM32CubeProgrammer doesn’t work with the boot flow we have used so far, which uses the U-Boot SPL as the first-stage bootloader, and U-Boot itself as the second stage bootloader. It only works when the first stage bootloader is the Arm Trusted Firmware, also called TF-A. You can get more details about the different possible boot chains on STM32MP1 on this Wiki page.

Due to this constraint, we are going to switch our Buildroot configuration to use TF-A instead of U-Boot SPL as the first stage bootloader.

First of all, we need to backport two Buildroot commits, which did not exist in the Buildroot 2019.02 we are using, but have been integrated later. The first commit, 9dbc934217e170578d4cbfdf524bc1b3988d0b9e allows to build TF-A for ARM 32-bit platforms, while the second commit, e4d276c357fdf9f19f99f826cab63f373687f902 allows to provide a custom name for the TF-A image name.

In Buildroot, do:

$ git cherry-pick 9dbc934217e170578d4cbfdf524bc1b3988d0b9e
$ git cherry-pick e4d276c357fdf9f19f99f826cab63f373687f902

The second one will cause some minor conflict in boot/arm-trusted-firmware/Config.in. Resolve the conflict by removing the BR2_TARGET_ARM_TRUSTED_FIRMWARE_DEBUG option from this file, remove the conflict markers, then run:

git add boot/arm-trusted-firmware/Config.in
git commit

If you’re not sure about this, you can check our 2019.02/stm32mp157-dk-blog-6 branch on Github, which has these changes already integrated.

Once done, we can run make menuconfig and start modifying the Buildroot configuration. Here are the changes that we need:

  • In the Bootloaders menu, enable ARM Trusted Firmware (ATF), and then:
    • Set ATF Version to Custom Git repository
    • Set URL of custom repository to https://github.com/STMicroelectronics/arm-trusted-firmware.git
    • Set Custom repository version to v2.0-stm32mp-r2
    • Set ATF platform to stm32mp1
    • Set Additional ATF build variables to DTB_FILE_NAME=stm32mp157c-dk2.dtb AARCH32_SP=sp_min. The DTB_FILE_NAME selects the correct Device Tree file for the DK2 board, while the AARCH32_SP indicates that we are using the “minimal” secure payload, and not a complete Trusted Execution Environment such as OP-TEE.
    • Set Binary boot images to *.stm32. This makes sure the final image gets copied to output/images.
  • Still in the Bootloaders menu, inside the U-Boot option, make the following changes:
    • Change Board defconfig to stm32mp15_trusted. This is the most important change, which makes U-Boot build only the second stage, and in a format that gets loaded by TF-A as the first stage.
    • In U-Boot binary format, disable u-boot.img, and instead enable Custom (specify below) and indicate u-boot.stm32 as the value for U-Boot binary format: custom names.
    • Disable the Install U-Boot SPL binary image option.

Overall, the diff of the changes in the configuration looks like this:

@@ -30,16 +30,22 @@ BR2_TARGET_ROOTFS_EXT2=y
 BR2_TARGET_ROOTFS_EXT2_4=y
 BR2_TARGET_ROOTFS_EXT2_SIZE="120M"
 # BR2_TARGET_ROOTFS_TAR is not set
+BR2_TARGET_ARM_TRUSTED_FIRMWARE=y
+BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_GIT=y
+BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_REPO_URL="https://github.com/STMicroelectronics/arm-trusted-firmware.git"
+BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_REPO_VERSION="69cc28c5a1b877cf67def7f94dece087f3917b1c"
+BR2_TARGET_ARM_TRUSTED_FIRMWARE_PLATFORM="stm32mp1"
+BR2_TARGET_ARM_TRUSTED_FIRMWARE_ADDITIONAL_VARIABLES="DTB_FILE_NAME=stm32mp157c-dk2.dtb AARCH32_SP=sp_min"
+BR2_TARGET_ARM_TRUSTED_FIRMWARE_IMAGES="*.stm32"
 BR2_TARGET_UBOOT=y
 BR2_TARGET_UBOOT_BUILD_SYSTEM_KCONFIG=y
 BR2_TARGET_UBOOT_CUSTOM_GIT=y
 BR2_TARGET_UBOOT_CUSTOM_REPO_URL="https://github.com/STMicroelectronics/u-boot.git"
 BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION="v2018.11-stm32mp-r2.1"
-BR2_TARGET_UBOOT_BOARD_DEFCONFIG="stm32mp15_basic"
+BR2_TARGET_UBOOT_BOARD_DEFCONFIG="stm32mp15_trusted"
 BR2_TARGET_UBOOT_CONFIG_FRAGMENT_FILES="board/stmicroelectronics/stm32mp157-dk/uboot-fragment.config"
 # BR2_TARGET_UBOOT_FORMAT_BIN is not set
-BR2_TARGET_UBOOT_FORMAT_IMG=y
-BR2_TARGET_UBOOT_SPL=y
-BR2_TARGET_UBOOT_SPL_NAME="spl/u-boot-spl.stm32"
+BR2_TARGET_UBOOT_FORMAT_CUSTOM=y
+BR2_TARGET_UBOOT_FORMAT_CUSTOM_NAME="u-boot.stm32"
 BR2_TARGET_UBOOT_CUSTOM_MAKEOPTS="DEVICE_TREE=stm32mp157c-dk2"
 BR2_PACKAGE_HOST_GENIMAGE=y

Before we can restart the build, we need to adjust the genimage.cfg file that describes the layout of the SD card. Indeed, the file name of the first stage bootloader is now tf-a-stm32mp157c-dk2.stm32 instead of u-boot-spl.stm32 and the file name of the second stage bootloader is now u-boot.stm32 instead of u-boot.img. All in all, your genimage.cfg file in board/stmicroelectronics/stm32mp157-dk/genimage.cfg should now look like this:

image sdcard.img {
	hdimage {
		gpt = "true"
	}

	partition fsbl1 {
		image = "tf-a-stm32mp157c-dk2.stm32"
	}

	partition fsbl2 {
		image = "tf-a-stm32mp157c-dk2.stm32"
	}

	partition ssbl {
		image = "u-boot.stm32"
	}

	partition rootfs {
		image = "rootfs.ext4"
		partition-type = 0x83
		bootable = "yes"
		size = 256M
	}
}

With this in place, it’s time to restart the build. You can do a complete rebuild with make clean all, or you can just clean up U-Boot, and restart the build:

$ make uboot-dirclean
$ make

You should now have in output/images the new TF-A image tf-a-stm32mp157c-dk2.stm32 and the new U-Boot image u-boot.stm32. Of course the sdcard.img file has been updated.

Rather than updating our SD card on our workstation, we’ll directly use the STM32CubeProgrammer tool to do that, in the next section.

Flashing the board

The STM32CubeProgrammer tool takes as input a flash layout file, which has a .tsv extension. The format of this file is extensively documented on this Wiki page. It is essentially a text file that says what should be flashed in each partition.

In our case, we are going to simply flash the entire sdcard.img instead of flashing partition by partition. To achieve this, we are going to use the RawImage image type, also described on the Wiki page.

Let’s create a file board/stmicroelectronics/stm32mp157-dk/flash.tsv, with the following contents:

#Opt	Id	Name	Type	IP	Offset	Binary
-	0x01	fsbl1-boot	Binary	none	0x0	tf-a-stm32mp157c-dk2.stm32
-	0x03	ssbl-boot	Binary	none	0x0	u-boot.stm32
P	0x10	sdcard	RawImage	mmc0		0x0	sdcard.img

The first line is a comment, just to help remember what each field is about. The second and third lines tell STM32CubeProgrammer which bootloader images should be used as part of the flashing process. Finally, the last line says we want to flash sdcard.img as a raw image on the mmc0 device.

Then, go do output/images, and run STM32CubeProgrammer. We use the -c port=usb1 argument, because our board was detected as device USB1 when we enumerated all detected devices using the -l usb previously.

$ cd output/images/
$ sudo ~/stm32cube/bin/STM32_Programmer_CLI -c port=usb1 -w ../../board/stmicroelectronics/stm32mp157-dk/flash.tsv

The output will look like this:

      -------------------------------------------------------------------
                        STM32CubeProgrammer v2.2.0                  
      -------------------------------------------------------------------



USB speed   : High Speed (480MBit/s)
Manuf. ID   : STMicroelectronics
Product ID  : DFU in HS Mode @Device ID /0x500, @Revision ID /0x0000
SN          : 004200343338510534383330
FW version  : 0x0110
Device ID   : 0x0500
Device name : STM32MPxxx
Device type : MPU
Device CPU  : Cortex-A7


Start Embedded Flashing service



Memory Programming ...
Opening and parsing file: tf-a-stm32mp157c-dk2.stm32
  File          : tf-a-stm32mp157c-dk2.stm32
  Size          : 237161 Bytes
  Partition ID  : 0x01 

Download in Progress:
[==================================================] 100% 

File download complete
Time elapsed during download operation: 00:00:00.444

RUNNING Program ... 
  PartID:      :0x01 
Start operation done successfully at partition 0x01

Flashlayout Programming ...
[==================================================] 100% 
Running Flashlayout Partition ...
Flashlayout partition started successfully


Memory Programming ...
Opening and parsing file: u-boot.stm32
  File          : u-boot.stm32
  Size          : 748042 Bytes
  Partition ID  : 0x03 

Download in Progress:
[==================================================] 100% 

File download complete
Time elapsed during download operation: 00:00:00.791

RUNNING Program ... 
  PartID:      :0x03 

reconnecting the device ...

USB speed   : High Speed (480MBit/s)
Manuf. ID   : STMicroelectronics
Product ID  : USB download gadget@Device ID /0x500, @Revision ID /0x0000
SN          : 004200343338510534383330
FW version  : 0x0110
Device ID   : 0x0500
Start operation done successfully at partition 0x03


Memory Programming ...
Opening and parsing file: sdcard.img
  File          : sdcard.img
  Size          : 539002368 Bytes
  Partition ID  : 0x10 

Download in Progress:
[==================================================] 100% 

File download complete
Time elapsed during download operation: 00:04:31.583

RUNNING Program ... 
  PartID:      :0x10 
Start operation done successfully at partition 0x10
Flashing service completed successfully

Finally, we can toggle back the SW1 DIP switches to their ON position, to boot again from SD card, and hit the reset button. The board should boot, but this time with our new image, which uses TF-A instead of U-Boot SPL, so the first lines of the boot process should look like this:

NOTICE:  CPU: STM32MP157CAC Rev.B
NOTICE:  Model: STMicroelectronics STM32MP157C-DK2 Discovery Board
NOTICE:  Board: MB1272 Var2 Rev.C-01
NOTICE:  BL2: v2.0-r2.0(release):
NOTICE:  BL2: Built : 16:10:53, Jan  7 2020
NOTICE:  BL2: Booting BL32
NOTICE:  SP_MIN: v2.0-r2.0(release):
NOTICE:  SP_MIN: Built : 16:10:53, Jan  7 2020

U-Boot 2018.11-stm32mp-r2.1 (Jan 07 2020 - 16:13:55 +0100)

Conclusion

In this article, we have discussed the concept of factory flashing, understood better the different boot chains available for the STM32MP1, switched to a boot chain using TF-A, and presented how to use STM32CubeProgrammer to reflash the entire SD card.

As usual, we have a branch on Github with the Buildroot changes corresponding to this blog post, see the branch 2019.02/stm32mp157-dk-blog-6.

Stay tuned for the next article in this series of blog post, in which we will cover the topic of Over-The-Air firmware update.