Building a Linux system for the STM32MP1: remote firmware updates

After another long break, here is our new article in the series of blog posts about building a Linux system for the STM32MP1 platform. After showing how to build a minimal Linux system for the STM32MP157 platform, how to connect and use an I2C based pressure/temperature/humidity sensor and how to integrate Qt5 in our system, how to set up a development environment to write our own Qt5 application, how to develop a Qt5 application, and how to setup factory flashing, we are now going to discuss the topic of in-field firmware update.

List of articles in this series:

Why remote firmware updates?

The days and age when it was possible to build and flash an embedded system firmware, ship the device and forget it, are long behind us. Systems have gotten more complicated, and we therefore have to fix bugs and security issues after the device has been shipped, and we often want to deploy new features in the field into existing devices. For all those reasons, the ability to remotely update the firmware of embedded devices is now a must-have.

Open-source firmware update tools

There are different possibilities to update your system:

If you’re using a binary distribution, use the package manager of this distribution to update individual components
Do complete system image updates, at the block-level, replacing the entire system image with an updated one. Three main open-source solutions are available: swupdate, Mender.io and RAUC.
Do file-based updates, with solutions such as OSTree.

In this blog post, we are going to show how to set up the swupdate solution.

swupdate is a tool installed on the target that can receive an update image (.swu file), either from a local media or from a remote server, and use it to update various parts of the system. Typically, it will be used to update the Linux kernel and the root filesystem, but it can also be used to update additional partitions, FPGA bitstreams, etc.

swupdate implements two possible update strategies:

A dual copy strategy, where the storage has enough space to store two copies of the entire filesystem. This allows to run the system from copy A, update copy B, and reboot it into copy B. The next update will of course update copy A.
A single copy strategy, where the upgrade process consists in rebooting into a minimal system that runs entirely from RAM, and that will be responsible for updating the system on storage.

For this blog post, we will implement the dual copy strategy, but the single copy strategy is also supported for systems with tighter storage restrictions.

We are going to setup swupdate step by step: first by triggering updates locally, and then seeing how to trigger updates remotely.

Local usage of swupdate

Add USB storage support

As a first step, in order to transfer the update image to the target, we will use a USB stick. This requires having USB mass storage support in the Linux kernel. So let’s adjust our Linux kernel configuration by running make linux-menuconfig. Within the Linux kernel configuration:

Enable the CONFIG_SCSI option. This is a requirement for USB mass storage support
Enable the CONFIG_BLK_DEV_SD option, needed for SCSI disk support, which is another requirement for USB mass storage.
Enable the CONFIG_USB_STORAGE option.
The CONFIG_VFAT_FS option, to support the FAT filesystem, is already enabled.
Enable the CONFIG_NLS_CODEPAGE_437 and CONFIG_NLS_ISO8859_1 options, to have the necessary support to decode filenames in the FAT filesystem.

Then, run make linux-update-defconfig to preserve these kernel configurations changes in your kernel configuration file at board/stmicroelectronics/stm32mp157-dk/linux.config.

swupdate setup

In Target packages, System tools, enable swupdate. You can disable the install default website setting since we are not going to use the internal swupdate web server.

Take this opportunity to also enable the gptfdisk tool and its sgdisk sub-option in the Hardware handling submenu. We will need this tool later to update the partition table at the end of the update process.

Now that we have both both USB storage support and the swupdate package enabled, let’s build a new version of our system by running make. Flash the resulting image on your SD card, and boot your target. You should have swupdate available:

# swupdate -h
Swupdate v2018.11.0

Licensed under GPLv2. See source distribution for detailed copyright notices.

swupdate (compiled Mar  4 2020)
Usage swupdate [OPTION]
 -f, --file           : configuration file to use
 -p, --postupdate               : execute post-update command
 -e, --select , : Select software images set and source
                                  Ex.: stable,main
 -i, --image          : Software to be installed
 -l, --loglevel          : logging level
 -L, --syslog                   : enable syslog logger
 -n, --dry-run                  : run SWUpdate without installing the software
 -N, --no-downgrading  : not install a release older as 
 -o, --output      : saves the incoming stream
 -v, --verbose                  : be verbose, set maximum loglevel
     --version                  : print SWUpdate version and exit
 -c, --check                    : check image and exit, use with -i 
 -h, --help                     : print this help and exit
 -w, --webserver [OPTIONS]      : Parameters to be passed to webserver
	mongoose arguments:
	  -l, --listing                  : enable directory listing
	  -p, --port               : server port number  (default: 8080)
	  -r, --document-root      : path to document root directory (default: .)
	  -a, --api-version [1|2]        : set Web protocol API to v1 (legacy) or v2 (default v2)
	  --auth-domain                  : set authentication domain if any (default: none)
	  --global-auth-file             : set authentication file if any (default: none)

Take a USB stick with a FAT filesystem on it, which you can mount:

# mount /dev/sda1 /mnt

If that works, we’re now ready to move on to the next step of actually getting a firmware update image.

Generate the swupdate image

swupdate has its own update image format, and you need to generate an image that complies with this format so that swupdate can use it to upgrade your system. The format is simple: it’s a CPIO archive, which contains one file named sw-description describing the contents of the update image, and one or several additional files that are the images to update.

First, let’s create our sw-description file in board/stmicroelectronics/stm32mp157-dk/sw-description. The tags and properties available are described in the swupdate documentation.

software = {
	version = "0.1.0";
	rootfs = {
		rootfs-1: {
			images: (
			{
				filename = "rootfs.ext4.gz";
				compressed = true;
				device = "/dev/mmcblk0p4";
			});
		}
		rootfs-2: {
			images: (
			{
				filename = "rootfs.ext4.gz";
				compressed = true;
				device = "/dev/mmcblk0p5";
			});
		}
	}
}

This describes a single software component rootfs, which is available as two software collections, to implement the dual copy mechanism. The root filesystem will have one copy in /dev/mmcblk0p4 and another copy in /dev/mmcblk0p5. They will be updated from a compressed image called rootfs.ext4.gz.

Once this sw-description file is written, we can write a small script that generates the swupdate image. We’ll put this script in board/stmicroelectronics/stm32mp157-dk/gen-swupdate-image.sh:

#!/bin/sh

BOARD_DIR=$(dirname $0)

cp ${BOARD_DIR}/sw-description ${BINARIES_DIR}

IMG_FILES="sw-description rootfs.ext4.gz"

pushd ${BINARIES_DIR}
for f in ${IMG_FILES} ; do
	echo ${f}
done | cpio -ovL -H crc > buildroot.swu
popd

It simply copies the sw-description file to BINARIES_DIR (which is output/images), and then creates a buildroot.swu CPIO archive that contains the sw-description and rootfs.ext4.gz files.

Of course, make sure this script has executable permissions.

Then, we need to slightly adjust our Buildroot configuration, so run make menuconfig, and:

In System configuration, in the option Custom scripts to run after creating filesystem images, add board/stmicroelectronics/stm32mp157-dk/gen-swupdate-image.sh after the existing value support/scripts/genimage.sh. This will make sure our new script generating the swupdate image is executed as a post-image script, at the end of the build.
In Filesystem images, enable the gzip compression method for the ext2/3/4 root filesystem, so that a rootfs.ext4.gz image is generated.

With that in place, we are now able to generate our firmware image, by simply running make in Buildroot. At the end of the build, the output/images/ folder should contain the sw-description and rootfs.ext4.gz files. You can look at the contents of buildroot.swu:

$ cat output/images/buildroot.swu | cpio -it
sw-description
rootfs.ext4.gz
58225 blocks

Partioning scheme and booting logic

We now need to adjust the partitioning scheme of our SD card so that it has two partitions for the root filesystem, one for each copy. This partitioning scheme is defined in board/stmicroelectronics/stm32mp157-dk/genimage.cfg, which we change to:

image sdcard.img {
        hdimage {
                gpt = "true"
        }

        partition fsbl1 {
                image = "tf-a-stm32mp157c-dk2.stm32"
        }

        partition fsbl2 {
                image = "tf-a-stm32mp157c-dk2.stm32"
        }

        partition ssbl {
                image = "u-boot.stm32"
        }

        partition rootfs1 {
                image = "rootfs.ext4"
                partition-type = 0x83
                bootable = "yes"
                size = 256M
        }

        partition rootfs2 {
                partition-type = 0x83
                size = 256M
        }
}

As explained in the first blog post of this series, the /boot/extlinux/extlinux.conf file is read by the bootloader to know how to boot the system. Among other things, this file defines the Linux kernel command line, which contains root=/dev/mmcblk0p4 to tell the kernel where the root filesystem is. But with our dual copy upgrade scheme, the root filesystem will sometimes be on /dev/mmcblk0p4, sometimes on /dev/mmcblk0p5. To achieve that without constantly updating the extlinux.conf file, we will use /dev/mmcblk0p${devplist} instead. devplist is a U-Boot variable that indicates from which partition the extlinux.conf file was read, which turns out to be the partition of our root filesystem. So, your board/stmicroelectronics/stm32mp157-dk/overlay/boot/extlinux/extlinux.conf file should look like this:

label stm32mp15-buildroot
  kernel /boot/zImage
  devicetree /boot/stm32mp157c-dk2.dtb
  append root=/dev/mmcblk0p${devplist} rootwait console=ttySTM0,115200 vt.global_cursor_default=0

For the dual copy strategy to work, we need to tell the bootloader to boot either from the root filesystem in the rootfs1 partition or the rootfs2 partition. This will be done using the bootable flag of each GPT partition, and this is what this script does: it toggles the bootable flag of 4th and 5th partition of the SD card. This way, the partition with the bootable flag will lose it, and the other partition will gain it. Thanks to this, at the next reboot, U-Boot will consider the system located in the other SD card partition. This work will be done by a /etc/swupdate/postupdate.sh script, that you will store in board/stmicroelectronics/stm32mp157-dk/overlay/etc/swupdate/postupdate.sh, which contains:

#!/bin/sh
sgdisk -A 4:toggle:2 -A 5:toggle:2 /dev/mmcblk0
reboot

Make sure this script is executable.

With all these changes in place, let’s restart the Buildroot build by running make. The sdcard.img should contain the new partioning scheme:

$ sgdisk -p output/images/sdcard.img
[...]
Number  Start (sector)    End (sector)  Size       Code  Name
   1              34             497   232.0 KiB   8300  fsbl1
   2             498             961   232.0 KiB   8300  fsbl2
   3             962            2423   731.0 KiB   8300  ssbl
   4            2424          526711   256.0 MiB   8300  rootfs1
   5          526712         1050999   256.0 MiB   8300  rootfs2

Reflash your SD card with the new sdcard.img, and boot this new system. Transfer the buildroot.swu update image to your USB stick.

Testing the firmware update locally

After booting the system, mount the USB stick, which contains the buildroot.swu file:

# mount /dev/sda1 /mnt/
# ls /mnt/
buildroot.swu

Let’s trigger the system upgrade with swupdate:

# swupdate -i /mnt/buildroot.swu -e rootfs,rootfs-2 -p /etc/swupdate/postupdate.sh

Swupdate v2018.11.0

Licensed under GPLv2. See source distribution for detailed copyright notices.

Registered handlers:
	dummy
	raw
	rawfile
software set: rootfs mode: rootfs-2
Software updated successfully
Please reboot the device to start the new software
[INFO ] : SWUPDATE successful ! 
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
# Stopping qt-sensor-demo: OK
Stopping dropbear sshd: OK
Stopping network: OK
Saving random seed... done.
Stopping klogd: OK
Stopping syslogd: OK
umount: devtmpfs busy - remounted read-only
[  761.949576] EXT4-fs (mmcblk0p4): re-mounted. Opts: (null)
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[  763.965243] reboot: ResNOTICE:  CPU: STM32MP157CAC Rev.B
NOTICE:  Model: STMicroelectronics STM32MP157C-DK2 Discovery Board

The -i option indicates the firmware update file, while the -e option indicates which software component should be updated. Here we update the rootfs in its slot 2, rootfs-2, which is in /dev/mmcblk0p5. The -p option tells to run our post-update script when the update is successful. In the above log, we see that the system is being rebooted right after the update.

At the next boot, you should see:

U-Boot 2018.11-stm32mp-r2.1 (Mar 04 2020 - 15:28:34 +0100)
[...]
mmc0 is current device
Scanning mmc 0:5...
Found /boot/extlinux/extlinux.conf
[...]
append: root=/dev/mmcblk0p5 rootwait console=ttySTM0,115200 vt.global_cursor_default=0

during the U-Boot part. So we see it is loading extlinux.conf from the MMC partition 5, and has properly set root=/dev/mmcblk0p5. So the kernel and Device Tree will be loaded from MMC partition 5, and this partition will also be used by Linux as the root filesystem.

With all this logic, we could now potentially have some script that gets triggered when a USB stick is inserted, mount it, check if an update image is available on the USB stick, and if so, launch swupdate and reboot. This would be perfectly fine for local updates, for example with an operator in charge of doing the update of the device.

However, we can do better, and support over-the-air updates, a topic that we will discuss in the next section.

Over-the-air updates

To support over-the-air updates with swupdate, we will have to:

Install on a server a Web interface that allows the swupdate program to retrieve firmware update files, and the user to trigger the updates.
Run swupdate in daemon mode on the target.

Set up the web server: hawkBit

swupdate is capable of interfacing with a management interface provided by the Eclipse hawkBit project. Using this web interface, one can manage its fleet of embedded devices, and rollout updates to these devices remotely.

hawkBit has plenty of capabilities, and we are here going to set it up in a very minimal way, with no authentication and a very simple configuration.

As suggested in the project getting started page, we’ll use a pre-existing Docker container image to run hawkBit:

sudo docker run -p 8080:8080 hawkbit/hawkbit-update-server:latest \
     --hawkbit.dmf.rabbitmq.enabled=false \
     --hawkbit.server.ddi.security.authentication.anonymous.enabled=true

After a short while, it should show:

2020-03-06 09:15:46.492  ... Started ServerConnector@3728a578{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-03-06 09:15:46.507  ... Jetty started on port(s) 8080 (http/1.1) with context path '/'
2020-03-06 09:15:46.514  ... Started Start in 21.312 seconds (JVM running for 22.108)

From this point, you can connect with your web browser to http://localhost:8080 to access the hawkBit interface. Login with the admin login and admin password.

hawkBit login

Once in the main hawkBit interface, go to the System Config tab, and enable the option Allow targets to download artifacts without security credentials. Of course, for a real deployment, you will want to set up proper credentials and authentification.

hawkBit System Config

In the Distribution tab, create a new Distribution by clicking on the plus sign in the Distributions panel:

hawkBit New Distribution

Then in the same tab, but in the Software Modules panel, create a new software module:

hawkBit New Software Module

Once done, assign the newly added software module to the Buildroot distribution by dragging-drop it into the Buildroot distribution. Things should then look like this:

hawkBit Distribution

Things are now pretty much ready on the hawkBit side now. Let’s move on with the embedded device side.

Configure swupdate

We need to adjust the configuration of swupdate to enable its Suricatta functionality which is what allows to connect to an hawkBit server.

In Buildroot’s menuconfig, enable the libcurl (BR2_PACKAGE_LIBCURL) and json-c (BR2_PACKAGE_JSON_C) packages, both of which are needed for swupdate’s Suricatta. While at it, since we will adjust the swupdate configuration and we’ll want to preserve our custom configuration, change the BR2_PACKAGE_SWUPDATE_CONFIG option to point to board/stmicroelectronics/stm32mp157-dk/swupdate.config.

Then, run:

$ make swupdate-menuconfig

to enter the swupdate configuration interface. Enable the Suricatta option, and inside this menu, in the Server submenu, verify that the Server Type is hawkBit support. You can now exit the swupdate menuconfig.

Save our custom swupdate configuration permanently:

$ make swupdate-update-defconfig

With this proper swupdate configuration in place, we now need to create a runtime configuration file for swupdate, and an init script to start swupdate at boot time. Let’s start with the runtime configuration file, which we’ll store in board/stmicroelectronics/stm32mp157-dk/overlay/etc/swupdate/swupdate.cfg, containing:

globals :
{
	postupdatecmd = "/etc/swupdate/postupdate.sh";
};

suricatta :
{
	tenant = "default";
	id = "DEV001";
	url = "http://192.168.42.1:8080";
};

We specify the path to our post-update script so that it doesn’t have to be specified on the command line, and then we specify the Suricatta configuration details: id is the unique identifier of our board, the URL is the URL to connect to the hawkBit instance (make sure to replace that with the IP address of where you’re running hawkBit). tenant should be default, unless you’re using your hawkBit instance in complex setups to for example serve multiple customers.

Our post-update script also needs to be slightly adjusted. Indeed, we will need a marker that tells us upon reboot that an update has been done, in order to confirm to the server that the update has been successfully applied. So we change board/stmicroelectronics/stm32mp157-dk/overlay/etc/swupdate/postupdate.sh to:

#!/bin/sh

PART_STATUS=$(sgdisk -A 4:get:2 /dev/mmcblk0)
if test "${PART_STATUS}" = "4:2:1" ; then
        NEXT_ROOTFS=/dev/mmcblk0p5
else
        NEXT_ROOTFS=/dev/mmcblk0p4
fi

# Add update marker
mount ${NEXT_ROOTFS} /mnt
touch /mnt/update-ok
umount /mnt

sgdisk -A 4:toggle:2 -A 5:toggle:2 /dev/mmcblk0
reboot

What we do is that we simply mount the next root filesystem, and create a file /update-ok. This file will be checked by our swupdate init script, see below.

Then, our init script will be in board/stmicroelectronics/stm32mp157-dk/overlay/etc/init.d/S98swupdate, with executable permissions, and contain:

#!/bin/sh

DAEMON="swupdate"
PIDFILE="/var/run/$DAEMON.pid"

PART_STATUS=$(sgdisk -A 4:get:2 /dev/mmcblk0)
if test "${PART_STATUS}" = "4:2:1" ; then
	ROOTFS=rootfs-2
else
	ROOTFS=rootfs-1
fi

if test -f /update-ok ; then
	SURICATTA_ARGS="-c 2"
	rm -f /update-ok
fi

start() {
	printf 'Starting %s: ' "$DAEMON"
	# shellcheck disable=SC2086 # we need the word splitting
	start-stop-daemon -b -q -m -S -p "$PIDFILE" -x "/usr/bin/$DAEMON" \
		-- -f /etc/swupdate/swupdate.cfg -L -e rootfs,${ROOTFS} -u "${SURICATTA_ARGS}"
	status=$?
	if [ "$status" -eq 0 ]; then
		echo "OK"
	else
		echo "FAIL"
	fi
	return "$status"
}

stop() {
	printf 'Stopping %s: ' "$DAEMON"
	start-stop-daemon -K -q -p "$PIDFILE"
	status=$?
	if [ "$status" -eq 0 ]; then
		rm -f "$PIDFILE"
		echo "OK"
	else
		echo "FAIL"
	fi
	return "$status"
}

restart() {
	stop
	sleep 1
	start
}

case "$1" in
        start|stop|restart)
		"$1";;
	reload)
		# Restart, since there is no true "reload" feature.
		restart;;
        *)
                echo "Usage: $0 {start|stop|restart|reload}"
                exit 1
esac

This is modeled after typical Buildroot init scripts. A few points worth mentioning:

At the beginning of the script, we determine which copy of the root filesystem needs to be updated by looking at which partition currently is marked “bootable”. This is used to fill in the ROOTFS variable.
We also determine if we are just finishing an update, by looking at the presence of a /update-ok file.
When starting swupdate, we pass a few options: -f with the path to the swupdate configuration file, -L to enable syslog logging, -e to indicate which copy of the root filesystem should be updated, and -u '${SURICATTA_ARGS}' to run in Suricatta mode, with SURICATTA_ARGS containing -c 2 to confirm the completion of an update.

Generate a new image with the updated swupdate, its configuration file and init script, and reboot your system.

Deploying an update

When booting, your system starts swupdate automatically:

Starting swupdate: OK
[...]
# ps aux | grep swupdate
  125 root     /usr/bin/swupdate -f /etc/swupdate/swupdate.cfg -L -e rootfs,rootfs-1 -u
  132 root     /usr/bin/swupdate -f /etc/swupdate/swupdate.cfg -L -e rootfs,rootfs-1 -u

Back to the hawkBit administration interface, the Deployment tab should show one notification:

hawkBit new device notification

and when clicking on it, you should see our DEV001 device:

hawkBit new device

Now, go to the Upload tab, select the Buildroot software module, and click on Upload File. Upload the buildroot.swu file here:

hawkBit Upload

Back into the Deployment tab, drag and drop the Buildroot distribution into the DEV001 device. A pending update should appear in the Action history for DEV001:

hawkBit upgrade pending

The swupdate on your target will poll regularly the server (by default every 300 seconds, can be customized in the System config tab of the hawkBit interface) to know if an update is available. When that happens, the update will be downloaded and applied, the system will reboot, and at the next boot the update will be confirmed as successful, showing this status in the hawkBit interface:

hawkBit upgrade confirmed

If you’ve reached this step, your system has been successfully updated, congratulations! Of course, there are many more things to do to get a proper swupdate/hawkBit deployment: assign unique device IDs (for example based on MAC addresses or SoC serial number), implement proper authentication between the swupdate client and the server, implement image encryption if necessary, improve the upgrade validation mechanism to make sure it detects if the new image doesn’t boot properly, etc.

Conclusion

In this blog post, we have learned about firmware upgrade solutions, and specifically about swupdate. We’ve seen how to set up swupdate in the context of Buildroot, first for local updates, and then for remote updates using the hawkBit management interface. Hopefully this will be useful for your future embedded projects!

As usual, the complete Buildroot code to reproduce the same setup is available in our branch 2019.02/stm32mp157-dk-blog-7, in two commits: one for the first step implementing support just for local updates, and another one for remote update support.

Author: Thomas Petazzoni

Thomas Petazzoni is Bootlin's co-owner and CEO. Thomas joined Bootlin in 2008 as a kernel and embedded Linux engineer, became CTO in 2013, and co-owner/CEO in 2021. More details... View all posts by Thomas Petazzoni

46 thoughts on “Building a Linux system for the STM32MP1: remote firmware updates”

David says:

March 16, 2020 at 8:38 pm

Is there a way to subscribe to some mailing list that when a new post is released then the mailing list will be notified?

Reply
1. Michael Opdenacker says:
  
  March 17, 2020 at 7:14 am
  
  There is no mailing list because we don’t want to have to manage private information such as the e-mail addresses of subscribers. However, you can use our RSS feed and an RSS feed reader to get notifications of new articles.
  Michael.
  
  Reply
Anthony.Harivel says:

March 26, 2020 at 5:58 pm

Hello!

Just to add my two cents from this awesome blog post:

– When building swupdate from Buildroot, one should make sure that “CONFIG_GUNZIP=y” is activated in package/swupdate/swupdate.config if you have a *.gz rootfs. Otherwise you get the following error when adding verbose to the command line:
[TRACE] : SWUPDATE running : [copyfile] : Requested decompression method (2) is not configured!
because in swupdate/core/cpio_utils.c, you have the following line:
if (compressed) {
if (compressed == COMPRESSED_TRUE) {
WARN(“compressed argument: boolean form is deprecated, use the string form”);
}
#ifdef CONFIG_GUNZIP
…
#endif
…
{
TRACE(“Requested decompression method (%d) is not configured!”, compressed);
ret = -EINVAL;
goto copyfile_exit;
}

– Talking about compression, in the sw-description file, writing “compressed=true;” is deprecated (see the code from swupdate core above”. Instead write “compressed=zlib” or “compressed=zstd” if your package is compressed by either one of those.

Thank you !

Anthony Harivel

Reply
1. Thomas Petazzoni says:
  
  March 27, 2020 at 8:22 am
  Hello Anthony, and thanks for your comments!
  
  Regarding the enabling of CONFIG_GUNZIP=y, this is done automatically in the context of this blog post series. Indeed, in previous blog posts, we have enabled Qt5, which selects BR2_PACKAGE_ZLIB=y. And when BR2_PACKAGE_ZLIB=y, the Buildroot swupdate Makefile passes HAVE_ZLIB=y to swupdate’s configuration, and as a consequence, CONFIG_GUNZIP is automatically enabled:
  
  $ make stm32mp157_dk_defconfig
  […]
  $ make swupdate-menuconfig
  And see, CONFIG_GUNZIP is enabled!
  
  Regarding your second question: our blog post series is based on Buildroot 2019.02, which uses swupdate 2018.11. As of swupdate 2018.11, compressed is really a boolean, as explained in the documentation:
```
   +-------------+----------+------------+---------------------------------------+
   | compressed  | bool     | images     | flag to indicate that "filename" is   |
   |             |          | files      | zlib-compressed and must be           |
   |             |          |            | decompressed before being installed   |
   +-------------+----------+------------+---------------------------------------+
```
  It is swupdate upstream commit 7da867e6e602415f8daafb0b4ec5700fc847f2ca that changed the “compressed” property from a boolean to a string. This commit was released as part of swupdate 2019.11, which was released *after* Buildroot 2019.02.
  Reply
Dan says:

March 27, 2020 at 8:41 pm

Did you miss a command. before running make swupdate-menuconfig I was running into the following error:
for f in board/stmicroelectronics/stm32mp157-dk/swupdate.config ; do if [ ! -f “${f}” ]; then printf “Kconfig fragment ‘%s’ for ‘%s’ does not exist\n” “${f}” “swupdate”; exit 1; fi; done
Kconfig fragment ‘board/stmicroelectronics/stm32mp157-dk/swupdate.config’ for ‘swupdate’ does not exist
package/swupdate/swupdate.mk:145: recipe for target ‘board/stmicroelectronics/stm32mp157-dk/swupdate.config’ failed
make[1]: *** [board/stmicroelectronics/stm32mp157-dk/swupdate.config] Error 1

I fixed with:
cp package/swupdate/swupdate.config board/stmicroelectronics/stm32mp157-dk/swupdate.config

Reply
1. Thomas Petazzoni says:
  
  March 30, 2020 at 9:08 am
  
  Hello Dan. I am not sure how you get to this build failure: board/stmicroelectronics/stm32mp157-dk/swupdate.config already exists in the Github tree. However, I’ve spotted some issues in configs/stm32mp157_dk_defconfig, which was not pointing to the right configuration file for swupdate, and missing a few options. I.e the blog post was correct, but not what was pushed on Github. Could you retry, and if it doesn’t work, provide the complete sequence of steps leading to the build failure you’re seeing ?
  
  Reply
Richard says:

March 31, 2020 at 8:30 am

Hi Thomas,
Great post ! This works very well.
Just a small thing :
In the script board/stmicroelectronics/stm32mp157-dk/gen-swupdate-image.sh
the pushd / popd commands are not POSIX, so it fails under Debian 10 (sh is dash on Debian)
So, the shebang should be #!/bin/bash, or the popd/pushd should be cd ${BINARIES_DIR}/cd –
(To be honest, I’m surprised those commands are not in dash…)

And thanks for your work, it saves a lot of time !

Reply
1. Thomas Petazzoni says:
  
  April 6, 2020 at 7:03 am
  
  Hello Richard. Thanks for your bug report, I have fixed than by using /bin/bash in the shebang. It is now pushed on Github, fixed for all other people who will try this tutorial on Debian.
  
  Reply
2. Bigroot says:
  
  February 1, 2021 at 9:08 pm
  
  thanks for your comment, had the same issue
  
  Reply
Aaron says:

April 3, 2020 at 9:29 pm

Hey! thanks for these!

I noticed that buildroot now includes the GCNano binaries required to use the GPU in the STM32MP1; however, I can not for the life of me get it to build. Do you plan on doing a tutorial on using these binaries or have any tips!

Greatly appreciated! Thanks

Reply
1. Thomas Petazzoni says:
  
  April 6, 2020 at 7:01 am
  
  Hello Aaron. I was at this point not planning on doing any tutorial on enabling the GPU. What issue are you facing ? Could you perhaps test with Buildroot 2020.02, and if you face any issue, report a bug to the Buildroot bug tracker or mailing list ?
  
  Reply
  1. Anonymous says:
    
    April 6, 2020 at 3:54 pm
    
    Thanks for the response. I will report a bug in the tracker and see what I get.
    
    This is the error I get with 2020.02:
    
    Pastebin link: https://pastebin.com/9VHWLK4D
    
    Raw Paste Data:
    
    CC [M] /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.o
    /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.c: In function ‘_DmaAlloc’:
    /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.c:167:11: error: implicit declaration of function ‘dma_alloc_writecombine’; did you mean ‘pgprot_writecombine’? [-Werror=implicit-function-declaration]
    167 | = dma_alloc_writecombine(galcore_device, NumPages * PAGE_SIZE, &mdlPriv->dmaHandle, gfp);
    | ^~~~~~~~~~~~~~~~~~~~~~
    | pgprot_writecombine
    /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.c:167:9: error: assignment to ‘gctPOINTER’ {aka ‘void *’} from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
    167 | = dma_alloc_writecombine(galcore_device, NumPages * PAGE_SIZE, &mdlPriv->dmaHandle, gfp);
    | ^
    /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.c: In function ‘_DmaFree’:
    /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.c:288:5: error: implicit declaration of function ‘dma_free_writecombine’; did you mean ‘pgprot_writecombine’? [-Werror=implicit-function-declaration]
    288 | dma_free_writecombine(galcore_device, Mdl->numPages * PAGE_SIZE, mdlPriv->kvaddr, mdlPriv->dmaHandle);
    | ^~~~~~~~~~~~~~~~~~~~~
    | pgprot_writecombine
    /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.c: In function ‘_DmaMmap’:
    /home/ajdecker/projects/buildroot/output/build/gcnano-binaries-c01642ed5e18cf09ecd905af193e935cb3be95ed/./hal/os/linux/kernel/allocator/default/gc_hal_kernel_allocator_dma.c:324:9: error: implicit declaration of function ‘dma_mmap_writecombine’ [-Werror=implicit-function-declaration]
    324 | if (dma_mmap_writecombine(gcvNULL,
    | ^~~~~~~~~~~~~~~~~~~~~
    
    Reply
    1. Thomas Petazzoni says:
      
      April 6, 2020 at 5:13 pm
      
      Buildroot 2020.02 uses the mainline Linux kernel for the STM32MP1-DK2 board. However it is quite likely that their out of tree gcnano-binaries only compiles properly with the 4.19-based fork of the kernel provided by ST. It should however be possible to fix it to work with an upstream Linux kernel, with a bit of effort.
      
      Reply
Ajava says:

April 27, 2020 at 3:13 pm

Hi, and thanks for these helpful article series. Unfortunately i get following error at the end of build process. Beause there is actually just one rootfs.ext2-file there and no rootfs.ext4. Is it a BUG in Buildroot, or am i doing something wrong? I just used you repo without actually changing anything.

>>> Executing post-image script support/scripts/genimage.sh
….
ERROR: file(rootfs.ext4): stat(/home/ajava/Projekte/BR-STM32MP1/tpetazzoni/buildroot/output/images/rootfs.ext4) failed: No such file or directory
ERROR: hdimage(sdcard.img): could not setup rootfs.ext4
Makefile:813: recipe for target ‘target-post-image’ failed
make: *** [target-post-image] Error 1

Thanks!
Ajava

Reply
1. Thomas Petazzoni says:
  
  April 28, 2020 at 2:55 pm
  
  Hello Ajava. If you don’t have rootfs.ext4, then it means you have not enabled generating an ext4 filesystem. This was done in the very first blog post of this series, at https://bootlin.com/blog/building-a-linux-system-for-the-stm32mp1-basic-system/. In this blog post, we suggest to use a defconfig we provide, and it does enable generating an ext4 filesystem image, see https://github.com/tpetazzoni/buildroot/commit/0dbb719bf0eee2efad1f00b80506474ab8265072#diff-90c83f390fd24d67a8e1579c7a3000d9
  
  Could you double check your Buildroot configuration ?
  
  Reply
ajava says:

April 29, 2020 at 7:29 am

Thanks for the response, well i actually checked it again.
The config in branch ‘stm32mp157-dk-blog-7’ is as follow:

“`
BR2_TARGET_ROOTFS_EXT2=y
BR2_TARGET_ROOTFS_EXT2_4=y
BR2_TARGET_ROOTFS_EXT2_SIZE=”120M”
BR2_TARGET_ROOTFS_EXT2_GZIP=y
“`

I think the problem is, that as soon as “GZIP” is activated buildroot does not create ‘rootfs.ext4’ anymore, but just ‘rootfs.ext2.gz’ and a softlink ‘rootfs.ext4.’gz’. Why?

Thanks

Reply
1. Thomas Petazzoni says:
  
  April 29, 2020 at 8:51 pm
  
  Hello. You are absolutely right, there is an issue here! On one hand the genimage configuration file expect an uncompressed rootfs.ext4 image, and on the other hand the swupdate image generation logic expects a compressed ext4 image. I will have a look at that and fix it. Thanks for the report!
  
  Reply
  1. Jose says:
    
    June 25, 2023 at 5:46 pm
    
    Hi there, has this issue been fix? Thanks!
    
    Reply
Kingsong says:

May 15, 2020 at 1:54 pm

Will there be a tutorial for remote process management and updating to a newer version of buildroot and also some serial communication example ?

Reply
1. Thomas Petazzoni says:
  
  May 18, 2020 at 6:39 am
  
  Hello Kingsong. I am not sure what you mean by “remote process management” exactly. Are you talking about using the Cortex-M4 of the STM32MP1 ?
  
  In any case, this blog post series if for now complete, we don’t have specific plans at the moment to extend it with more articles. It might happen, but it’s not on our short term TODO-list.
  
  Reply
Geoffrey Van Landeghem says:

July 7, 2020 at 10:08 am

Hi,
you mentioned a few image update solutions, but picked swupdate.
Is there any particular reason why you selected this over the others?

Reply
1. Thomas Petazzoni says:
  
  July 8, 2020 at 10:43 am
  
  Thanks Geoffrey for your comment. There is no strong reason. Compared to Mender, swupdate has the advantage of having a lighter client (Mender is written in Go, so the mender client is quite large, while swupdate is plain simple C code). However, Mender has (I believe) a better Web UI to manage the fleet of devices, while swupdate doesn’t provide any, and simply suggests to use Hawkbit, which isn’t entirely great. There is also RAUC from Pengutronix, which is a bit similar to swupdate: written in C, light client, but the Web UI to manage the fleet is also not provided, and they “simply” point to using Hawkbit. Really, each solution has its own advantages/drawbacks, we have used Mender, swupdate and RAUC at Bootlin depending on the projects. For this blog post, I had to pick one 🙂
  
  Reply
Matthew says:

January 29, 2021 at 2:44 am

Does the docker image run on the STM32MP1 itself? I didn’t realize such a small device was capable of running docker.

Reply
1. Thomas Petazzoni says:
  
  January 29, 2021 at 9:39 pm
  
  No, the Docker image described in this blog post is used to run the server side of the OTA update mechanism, called Hawkbit.
  
  However, the STM32MP1 is perfectly capable of running Docker, but that is not what is covered in this blog post.
  
  Reply
lcd says:

March 12, 2021 at 4:15 pm

After making the update from the tutorial,
in order to make another update,
what should I do in hawkBit?
I am searching for a different way than
restarting hawkBit docker image and redo the hawkBit tutorial part.
Thanks

Reply
1. lcd says:
  
  March 29, 2021 at 10:37 am
  
  I have found it: in hawkBit update menu, just delete the old swu file and upload a new .swu, and, after that – back into the Deployment tab, drag and drop the Buildroot distribution into the DEV001 device.
  Thanks
  
  Reply
  1. Dawood says:
    
    March 24, 2023 at 3:40 am
    
    What do you mean by Buildroot distribution, do I need to add the whole buildroot directory?
    Also, I couldn’t understand that distribution part. I have .swu file in upload section. I can see on Deployment menu Target device but there is nothing in distribution. I have created both module distribution and software. but I do not understand how I need to drag software module into distribution and so on. It would be nice if you help me out.
    
    Reply
npal says:

November 26, 2021 at 4:31 pm

Hi ,
I am exactly following the steps mentioned but getting this issue:
# swupdate -i /mnt/buildroot.swu -e rootfs,rootfs-2 -p /etc/swupdate/postupdate.
sh
Swupdate v2018.11.0

Licensed under GPLv2. See source distribution for detailed copyright notices.

Registered handlers:
dummy
raw
rawfile
software set: rootfs mode: rootfs-2
Software updated failed
#
Been scratching head for days but unable to figure it out.

Reply
1. Thomas Petazzoni says:
  
  November 28, 2021 at 3:10 pm
  
  Sorry to hear that this isn’t working for you. Unfortunately, there’s not enough detail there to help. Could you run swupdate with more logging, by passing “-l 2” as an option ?
  
  Did you make sure that the partionning scheme of your SD card has been updated to have two partitions for the rootfs ?
  
  Reply
Sourabh says:

December 8, 2021 at 7:05 pm

Hello Thomas,

Thank you for the wonderful series of articles.

I am using Yocto Project to integrate SWUpdate (dual bank strategy) on Avenger96 (STM32MP157AAC) board. Do you have similar article series for Yocto and SWUpdate? If so, that would be really helpful for me to get started.

Thanks in advance!

Reply
1. Thomas Petazzoni says:
  
  December 9, 2021 at 12:04 pm
  
  No, we don’t have a similar article about Yocto and swupdate. But you can look at https://sbabic.github.io/swupdate/building-with-yocto.html which has a lot of details.
  
  We have articles on Yocto+RAUC: https://bootlin.com/blog/another-system-update-adventure-with-rauc-barebox-yocto-project/ and on Yocto+Mender: https://bootlin.com/blog/mender-how-to-integrate-an-ota-updater/
  
  Reply
  1. Sourabh says:
    
    December 9, 2021 at 1:57 pm
    
    Hi Thomas,
    
    Thank you for the update. Yes, I looked into Yocto+RAUC artcile and will try it in the coming days.
    
    And also how about delta updates using SWUpdate? Can we do binary delta updates in SWUpdate? Do you have any working reference or article on it?
    
    Reply
  2. Sourabh says:
    
    December 10, 2021 at 10:33 am
    
    Hi Thomas,
    
    Thanks for the update.
    
    I will go though the Yocto+RAUC article in the coming days.
    
    And I would also like to know whether you have tried binary delta updates with SWUpdate? Can you please let me know how we can do delta updates using SWUpdate?
    
    Thanks.
    
    Reply
Sourabh says:

December 8, 2021 at 7:10 pm

Hello Thomas,

Thanks for the wonderful series of articles.

Currently, I am trying to integrate SWUpdate on Avenger96(STM32MP157AAC) board using Yocto Project. Do you also have similar article series for SWUpdate and Yocto? If so, this would be very helpful for me as a starting point.

Reply
Sourabh says:

January 10, 2022 at 12:18 pm

Hello Thomas,

In the init script “S98swupdate”, I have a doubt regarding PID file. Will this file be already available in “var/run/” location? Or is it created as part of this script?

And if we want to test HW-SW compatibility feature of SWUpdate then do we need to send corresponding argument using -H option in this file? Can you please let me know how should we take care of this?

Your help will be much appreciated.

Thanks in advance.

Reply
1. Thomas Petazzoni says:
  
  January 19, 2022 at 1:51 pm
  
  The point of the PID file is that it gets created when the service starts, and allows to know which process should be stopped when stopping the service. So indeed, it will only appear when the service gets started.
  
  And no, the HW-SW compatibility feature of SWupdate has nothing to do with the PID file.
  
  Reply
Kip says:

March 31, 2022 at 8:26 am

Hello Thomas,

Thank you for your very usefull articles !

I got the same issue that npal when running SWupdate. After using -l 5 option, I saw that the issue come from the compression of the images. The “compressed” attribut in sw-description file is now used with string value instead of boolean. With the gzip compression method for file system, I used “zlib” as value for compresed attribut (the other value possible is “zstd”) and I still can’t update my system (decompression method not found). If I remove compression for file system, swupdate work perfectly but then, the swu file is too heavy.

Do you have any idea how to solve this issue and use compression ?

Thank you again !

Reply
1. Thomas Petazzoni says:
  
  April 1, 2022 at 9:37 pm
  
  Your swupdate is probably configured without the appropriate compression support. Make sure to enable CONFIG_GUNZIP in the swupdate configuration (make swupdate-menuconfig).
  
  Reply
  1. Kip says:
    
    April 5, 2022 at 1:33 pm
    
    Thank you for your help, it work fine now !
    
    Reply
zz3 says:

January 13, 2023 at 3:57 pm

Hello Thomas,

Thank you for these awesome tutorials!

I am now testing the local update using SWUpdate. But when I mount the USB key on my Raspberry Pi 3, then launch the command :
$ swupdate -i /mnt/buildroot.swu -e rootfs,rootfs-2 -p /etc/swupdate/postupdate.sh
The terminal shows the message “Swupate was successful! ”
But then I get the errors :
ERROR : Caution: invalid main GPT header, but valid backup; regenerating main header from backup!
ERROR : Warning : invalid CRC on main header data; loaded backup partition table.

Then the system reboots on the same partition and not on the one that is supposed to be updated.

I tried to change the genimage.cfg file by adding in the hdimage section : “gpt=true”. But the compiler indicates that it is deprecated and suggests using “partition-table-type = gpt”.
However, I still get the ERROR : ‘partition-type is only valid for mbr and hybrid partition-table-type’.

Do you have an idea about what may be causing this?

Reply
1. Köry Maincent says:
  
  January 13, 2023 at 5:03 pm
  
  Indeed this may come from the GPT partition table.
  Could you try with partition-table-type = "gpt"
  and replace partition-type = "0x83" by partition-type-uuid = U
  
  Reply
  1. zz3 says:
    
    January 15, 2023 at 2:46 pm
    
    Thank you very much for your reply!
    I tried to do what you suggested but the system doesn’t boot.
    I also tried replacing partition-type = 0xC with partition-type-uuid = F in the boot section of the genimage.cfg file but the system doesn’t boot either.
    
    Reply
    1. Köry Maincent says:
      
      January 27, 2023 at 11:36 am
      
      ok, keep the old partition type description and boot the board.
      In fact, I was wrong the GUID is not related to the bootable flag.
      Then could you run the sgdisk command of the postupdate script and show me the result:
      sgdisk -p /dev/mmcblk0 sgdisk -A 4:get:2 /dev/mmcblk0 sgdisk -A 5:get:2 /dev/mmcblk0
      You seems to have partition table damaged.
      
      Reply
      1. zz3 says:
        
        January 30, 2023 at 8:45 pm
        
        Thank you for your reply!
        
        So, I removed the line “partition-table-type = “gpt”” in the genimage.cfg and I kept the partition-type lines.
        
        Then, I tried the three commands you gave me but I kept getting the same errors. Here is the error log :
        
        Caution: invalid main GPT header, but valid backup; regenerating main header from backup!
        Warning : Invalid CRC on main header data; loaded backup partition table.
        Warning! Main and backup partition tables differ! Use the ‘c’ and ‘e’ options on the recovery & transformation menu to examine the two tables.
        Warning! Main partition table CRC mismatch! Loaded backup partition table instead of main partition table!
        Warning! One or more CRCs don’ t match. You should repair the disk!
        Main header : ERROR
        Backup header : OK
        Main partition table: ERROR
        Backup partition table: OK
        Invalid partition data!
        
        Do you know how I can solve this ?
        
        Reply
      2. zz3 says:
        
        February 10, 2023 at 4:26 pm
        
        Hello !
        Like you said earlier, it is probably a problem due to the partition table. So, I rebuild an OS from scratch using Buildroot and I only installed the sgdisk package to get the command you gave me. So this is what I get when I use the command sgdisk –p /dev/mmcblk0:
        
        Found invalid GPT and valid MBR; converting MBR to GPT format in memory.
        Warning! Main partition overlaps the first partition by 33 blocks!
        You will need to delete this partition or resize it in another utility.
        Disk /dev/mmcblk0: 31422464 sectors, 15.0 GiB
        Sector size (logical/physical): 512/512 bytes
        Disk identifier (GUID): 88BFFBDD-6042-4609-AD9A-B15B6781D806
        Partition table holds up to 128 entries
        Main partition table begins at sector 2 and ends at sector 33
        First usable sector is 34, last usable sector is 31422430
        Partitions will be aligned on 1-sector boundaries
        Total free space is 31111134 sectors (14.8 GiB)
        Number Start (sector) End (sector) Size Code Name
        1 1 65536 32.0 MiB 0700 Microsoft basic data
        2 65537 311296 120.0 MiB 8300 Linux Filesystem
        
        I used the tool gparted to put unallocated space between the partitions to solve the problem of partition overlapping the other one. And this is what I get when I use the same command:
        
        Found invalid GPT and valid MBR; converting MBR to GPT format in memory.
        Disk /dev/mmcblk0: 31422464 sectors, 15.0 GiB
        Sector size (logical/physical): 512/512 bytes
        Disk identifier (GUID): AD528525-4829-4588-9FA8-1146268AE73C
        Partition table holds up to 128 entries
        Main partition table begins at sector 2 and ends at sector 33
        First usable sector is 34, last usable sector is 31422430
        Partitions will be aligned on 2048-sector boundaries
        Total free space is 31111101 sectors (14.8 GiB)
        Number Start (sector) End (sector) Size Code Name
        1 5132288 5197823 32.0 MiB 0700 Microsoft basic data
        2 25151488 25397247 120.0 MiB 8300 Linux Filesystem
        
        Do you know if it is normal or what I could do to solve the problem?
        
        Reply
        
        Köry Maincent says:
        
        March 3, 2023 at 4:30 pm
        
        It might be due to U-boot environment overwriting some partitions. I vaguely remember something about it.
        Could you try to set the ssbl partition size to 2M?
        
        Also it is weird that sgdisk sees an MBR partition table. Keep the partition-table-type = "gpt" to have GPT partition table.
        
        Reply