I recently managed to find time to clean up and submit my patches for xz kernel compression support on ARM, which I started working on back in November, during my flight to Linaro Connect. However, it was too late as Russell King, the ARM Linux maintainer, alreadyaccepted a similar patch, about 3 weeks before my submission. The lesson I learned was that checking a git tree is not always sufficient. I should have checked the mailing list archives too.
The good news is that xz kernel compression support should be available in Linux 3.4 in a few months from now. xz is a compression format based on the LZMA2 compression algorithm. It can be considered as the successor of lzma, and achieves even better compression ratios!
Before submitting my patches, I ran a few benchmarks on my own implementation. As the decompressing code is the same, the results should be the same as if I had used the patches that are going upstream.
Benchmark methodology
For both boards I tested, I used the same pre 3.3 Linux kernel from Linus Torvalds’ mainline git tree. I also used the U-boot bootloader in both cases.
I used the very useful grabserial script from Tim Bird. This utility reads messages coming out of the serial line, and adds timestamps to each line it receives. This allow to measure time from the earliest power on stages, and doesn’t slow down the target system by adding instrumentation to it.
Our benchmarks just measure the time for the bootloader to copy the kernel to RAM, and then the time taken by the kernel to uncompress itself.
- Loading time is measured between “reading uImage” and “OK” (right before “Starting kernel”) in the bootloader messages.
- Compression time measured between “Uncompressing Linux” and “done”:
~/bin/grabserial -v -d /dev/ttyUSB0 -e 15 -t -m "Uncompressing Linux" -i "done," > booting-lzo.log
Benchmarks on OMAP4 Panda
The Panda board has a fast dual Cortex A9 CPU (OMAP 4430) running at 1 GHz. The standard way to boot this board is from an MMC/SD card. Unfortunately, the MMC/SD interface of the board is rather slow.
In this case, we have a fast CPU, but with rather slow storage. Therefore, the time taken to copy the kernel from storage to RAM is expected to have a significant impact on boot time.
This case typically represents todays multimedia and mobile devices such as phones, media players and tablets.
| Compression | Size | Loading time | Uncompressing time | Total time |
|---|---|---|---|---|
| gzip | 3355768 | 2.213376 | 0.501500 | 2.714876 |
| lzma | 2488144 | 1.647410 | 1.399552 | 3.046962 |
| xz | 2366192 | 1.566978 | 1.299516 | 2.866494 |
| lzo | 3697840 | 2.471497 | 0.160596 | 2.632093 |
| None | 6965644 | 4.626749 | 0 | 4.626749 |
Results on Calao Systems USB-A9263 (AT91)
The USB-A9263 board from Calao Systems has a cheaper and much slower AT91SAM9263 CPU running at 200 MHz.
Here we are booting from NAND flash, which is the fastest way to boot a kernel on this board. Note that we are using the nboot command from U-boot, which guarantees that we just copy the number of bytes specified in the uImage header.
In this case, we have a slow CPU with slow storage. Therefore, we expect both the kernel size and the decompression algorithm to have a major impact on boot time.
This case is a typical example of industrial systems (AT91SAM9263 is still very popular in such applications, as we can see from customer requests), booting from NAND storage operating with a 200 to 400 MHz CPU.
| Compression | Size | Loading time | Uncompressing time | Total time |
|---|---|---|---|---|
| gzip | 2386936 | 5.843289 | 0.935495 | 6.778784 |
| lzma | 1794344 | 4.465542 | 6.513644 | 10.979186 |
| xz | 1725360 | 4.308605 | 4.816191 | 9.124796 |
| lzo | 2608624 | 6.351539 | 0.447336 | 6.798875 |
| None | 4647908 | 11.080560 | 0 | 11.080560 |
Lessons learned
Here’s what we learned from these benchmarks:
- lzo is still the best solution for minimum boot time. Remember, lzo kernel compression was merged by Bootlin.
- xz is always better than lzma, both in terms of image size. Therefore, there’s no reason to stick to lzma compression if you used it.
- Because of their heavy CPU usage, lzma and xz remain pretty bad in terms of boot time, on most types of storage devices. On systems with a fast CPU, and very slow storage though, xz should be the best solution
- On systems with a fast CPU, like the Panda board, boot time with xz is actually pretty close to lzo, and therefore can be a very interesting compromise between kernel size and boot time.
- Using a kernel image without compression is rarely a worthy solution, except in systems with a very slow CPU. This is the case of CPUs emulated on an FPGA (typically during chip development, before silicon is available). In this particular case, copying to memory is directly done by the emulator, and we just need CPU cycles to start the kernel.
