Bootlin at Kernel Recipes, September 2013, Paris

Kernel Recipes conference logoFor the second year, a company called Hupstream, located in Paris, is organizing in France a conference fully dedicated to kernel development: Kernel Recipes (French), on September 24th and 25th.

While we at Bootlin couldn’t participate to last year edition, it looked like a very interesting conference, with both useful talks and an audience grouping a significant number of kernel developers and companies interested in kernel development in France. So this year, we are very happy to participate to this conference:

We are definitely looking forward to meeting some of the French kernel developers we have been in contact with over the last months (see the list of participants). It is also worth mentioning that Bootlin has an open position for a kernel or embedded Linux engineer, and that this conference is a great opportunity to meet us!

Bootlin at Linux Plumbers 2013, September 2013, New Orleans

Linux Plumbers conference logoThe Linux Plumbers conference has been running since quite a few years now, and has established itself as an important event for the Linux community that takes care of the low-level aspects of the Linux system: the kernel of course, but also the related userspace low-level components. It was originally the primary goal of this conference: get the userspace developers of the low-level components and the kernel developers to meet together. Nowadays, Linux Plumbers is organized as both a set of regular talks and a set of half-day mini-conferences.

The 2013 edition will be held from September 18th to September 20th in New Orleans, United States, and will cover a wide range of topics: ACPI, power management, PCI, Android, Graphics, Automotive, Boot and Core OS, File and Storage systems, LLVM, Network virtualization, Scaling, Secure Boot, Virtualization and more.

As part of Bootlin strong engagement in the community, and continuous effort to ensure its engineers are up-to-date with the latest Linux developments, our engineer Maxime Ripard will attend this conference. For those attending the conference, do not hesitate to meet Maxime, who has an interesting Android experience to share thanks to the Android training course he created at Bootlin, as well has a nice ARM kernel development experience as the maintainer of the Linux kernel support for the Allwinner ARM processors.

Contributions to Barebox: initial Marvell SoC support

Barebox is a bootloader that strives to be a modern alternative to U-Boot. It currently supports ARM, Blackfin, MIPS, NIOS2, OpenRISC, PowerPC and x86 as CPU architectures, and while it doesn’t have as much hardware support as U-Boot yet, it does have a number of very significant advantages over U-Boot: a proper device model very similar to the one used in the Linux kernel, which makes the code very clean and nice, and a configuration system that uses kconfig, like the Linux kernel, which is a lot better than the per-board header files used by U-Boot with lots of cryptic macros.

Bootlin had already contributed to Barebox in the past, as our engineer Maxime Ripard added the support for the Crystalfontz i.MX28 boards.

More recently, we contributed basic support for the Marvell Kirkwood, Marvell Armada 370 and Marvell Armada XP ARM processors. This work was released as part of the 2013.07.0 release. For now, the support is fairly minimal, as it only allows to boot a Barebox bootloader that has serial port support. The most important part of the work was to write a kwbimage tool (see kwbimage.c), which allows to generate bootable images for Marvell processors. Our work contained minimal support for the Armada XP-based OpenBlocks AX3 board, the Armada XP-based GP development board, the Armada 370-based Mirabox from Globalscale and the Kirkwood-based Guruplug from Globalscale. Our work was quickly extended by Sebastian Hesselbarth, who added basic support for the Marvell Dove processor, and the Cubox platform from SolidRun, which uses the Dove processor.

Of course, such support is far from being complete, we are hoping in the future to add support for network, NAND and SD, in order to make Barebox really useful and usable on Marvell platforms.

The details of our contributions are:

Starting Linux directly from AT91bootstrap3

Here is an update for our previous article on booting linux directly from AT91bootstrap. On newer ATMEL platforms, you will have to use AT91bootstrap 3. It now has a convenient way to be configured to boot directly to Linux.

You can check it out from github:

git clone git://github.com/linux4sam/at91bootstrap.git

That version of AT91bootstrap is using the same configuration mechanism as the Linux kernel. You will find default configurations, named in the form:
<board_name><storage>_<boot_strategy>_defconfig

  • board_name can be: at91sam9260ek, at91sam9261ek, at91sam9263ek, at91sam9g10ek, at91sam9g20ek, at91sam9m10g45ek, at91sam9n12ek, at91sam9rlek, at91sam9x5ek, at91sam9xeek or at91sama5d3xek
  • storage can be:
    • df for DataFlash
    • nf for NAND flash
    • sd for SD card
  • our main interest will be in boot_strategy which can be:
    • uboot: start u-boot or any other bootloader
    • linux: boot Linux directly, passing a kernel command line
    • linux_dt: boot Linux directly, using a Device Tree
    • android: boot Linux directly, in an Android configuration

Let’s take for example the latest evaluation boards from ATMEL, the SAMA5D3x-EK. If you are booting from NAND flash:

make at91sama5d3xeknf_linux_dt_defconfig
make

You’ll end up with a file named at91sama5d3xek-nandflashboot-linux-dt-3.5.4.bin in the binaries/ folder. This is your first stage bootloader. It has the same storage layout as used in the u-boot strategy so you can flash it and it will work.

As a last note, I’ll had that less is not always faster. On our benchmarks, booting the SAMA5D31-EK using AT91bootstrap, then Barebox was faster than just using AT91bootstrap. The main reason is that barebox is actually enabling the caches and decompresses the kernel(see below, the kernel is also enaling the caches before decompressing itself) before booting.

Bootlin contributions to the 3.10 kernel

The 3.10 Linux kernel has been released a few days ago. According to LWN, with almost 13.500 non-merge commits, the 3.10 has been the busiest ever, and also the fastest. Bootlin engineers again contributed to this release, with 99 patches integrated, making Bootlin the 28th most active company contributing, right between ST-Ericsson (103 patches) and ARM (97 patches). See http://www.remword.com/kps_result/3.10_whole.html for the complete statistics.

This time, Bootlin contributions include:

  • LPAE support for the Marvell Armada XP SoC, done by Grégory Clement.
  • Fix for errata 4742 of the PJ4B CPU core (used in Armada 370/XP), which prevented booting Armada 370 platforms after ARM optimized some TLB operations. Done by Grégory Clement.
  • Support for NOR flash on Marvell Armada 370 and Armada XP SoC, done by Ezequiel Garcia
  • Addition of a mvebu-mbus driver to handle the address decoding mechanism and configurable memory windows of Marvell SoC. The mach-kirkwood, mach-orion5x, mach-dove, mach-mv78xx0 and mach-mvebu Marvell platforms are all converted to use it. Developping this driver was a requirement to enable PCIe in a Device Tree compatible way on these platforms. Done by Thomas Petazzoni.
  • Addition of Device Tree information for the PCIe controllers of the Armada 370 and Armada XP, but unfortunately not the PCIe driver itself (which will arrive in 3.11). Done by Thomas Petazzoni.
  • Support for the thermal sensor on Marvell Armada 370 and Armada XP SoC, done by Ezequiel Garcia
  • A lot of reorganization of the Device Tree compatible strings for the Allwinner ARM SoC support, to prepare for the addition of additional SoCs in the future. Done by Maxime Ripard.
  • Improvements to the Allwinner pinctrl driver, with support for the A10 and A13 SoC. Done by Maxime Ripard.
  • Enabling of the I2C GPIO expander of the Armada 370 based Mirabox platform. Done by Grégory Clement.
  • A few updates to the support for the i.MX28 Crystalfontz boards: touchscreen and one-wire support on CFA10049. Done by Alexandre Belloni.
  • Various cleanups and improvements to the OMAP GMPC driver, done by Ezequiel Garcia.
  • Various cleanups and improvements to the Marvell Armada 370/XP IRQ controller driver, done by Thomas Petazzoni.

In detail, the contributions are:

pdf-link-checker – find broken hyperlinks in PDF documents

Python powered logoTired of seeing your documents out of date? Don’t want to manually review them?

pdf-link-checker is a simple tool that parses a PDF document and checks for broken hyperlinks. It does this by sending simple HTTP requests to each link found in a given document.

  • External references can be a very valuable part of your documents. Broken links reduce their usefulness as well as the impression they make. They also give the feeling that your documents are outdated and older than they are.
  • Web sites evolve frequently. Having an automated way of detecting obsolete links is essential to keeping your documents up to date.

Of course, pdf-link-checker is free software (GNU GPLv2 license).

Why?

We are using pdf-link-checker to make sure that our Android, embedded Linux, and kernel training materials are always up to date. They contain references to useful resources on the Internet, but such resources can disappear or be moved to other places. Our training materials are created from LaTeX source code, but instead of implementing a broken link checker for LaTeX, we preferred to develop a checker for the exported PDF file. This is a much more generic solution, which could interest billions of users!

pdf-link-checker can be used to check hyperlinks in most document formats. All you need is a utility to convert your document format to PDF, with the ability to preserve hyperlinks. We recommend to open your documents with the excellent and free software LibreOffice office software (supporting GNU/Linux, MacOS X and Windows), offering a very easy to export to the PDF format. This way, you can use pdf-link-checker to find broken links in any text and presentation document, such as LibreOffice presentations and slides, Microsoft Word (doc / docx) and PowerPoint (ppt / pptx), RTF documents and HTML pages.

Installing

On GNU/Linux

Installing pdf-link-checker is very easy. First, we recommend to install the pip python package installer if you don’t have it yet.

  • sudo apt-get install python-pip on Debian based systems (such as Ubuntu)
  • sudo yum install python-pip on RPM based systems (Red Hat, Fedora, Suse, Mandriva…)

Then, installing pdf-link-checker along with its dependencies is easy:

$ pip install pdf-link-checker

On Windows

First, see instructions for installing Python and pip on Windows.

Then, you can install pdf-link-checker as follows:

$ pip install pdf-link-checker

Running pdf-link-checker

Using pdf-link-checker is even easier:

$ pdf-link-checker my-awesome-doc.pdf

Usage

./pdf-link-checker --help
Usage: pdf-link-checker [options] [PDF document files]

Reports broken hyperlinks in PDF documents

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         display progress information
  -s, --status          store check status information in a .checked file
  -d, --debug           display debug information
  -t MAX_THREADS, --max-threads=MAX_THREADS
                        set the maximum number of parallel threads to create
  -r MAX_REQUESTS_PER_HOST, --max-requests-per-host=MAX_REQUESTS_PER_HOST
                        set the maximum number of parallel requests per host
  -x EXCLUDE_HOSTS, --exclude-hosts=EXCLUDE_HOSTS
                        ignore urls which host name belongs to the given list
  -m TIMEOUT, --timeout=TIMEOUT
                        set the timeout for the requests
  --check-url=CHECK_URL
                        checks given url instead of checking PDF (debug)

Option details

  • --max-threads
    Specifies the maximum number of allowed threads (default: 100). To speed up the run, pdf-link-checker will launch several threads in order to check several links in parallel. This option allows to set a limit to the number of threads.

  • --max-requests-per-host
    Specifies the maximum number of allowed requests per host. Some URLs may belong to the same host, and since pdf-link-checker can check many URLs at the same time, we may want to set a limit to the number of requests per host. Otherwise, some hosts may confuse the check with a DoS attack.

  • --status
    Allows to create a .input-file.checked in case no broken hyperlink was found. This can allow scripts to skip the execution of pdf-link-checker for documents which have already been validated.

Limitations

  • pdf-link-checker won’t detect and check URLs which are not properly declared as hyperlinks.
  • It doesn’t support checking internal links yet. This feature is on our todo list though.
  • It doesn’t support checking links that require authentication yet. The plan is to ignore such URLs.

Getting help and helping out

Please use GitHub’s ressources for reporting issues, asking questions, etc.

Patches and pull requests are welcome of course! Browse our Git repository and feel free to contribute!

Workshop on the Altera SoC FPGA platform

On June 25, 2013, Altera and some partner companies where organizing a workshop in Toulouse (where some of the Bootlin offices are located) about the Altera SoC FPGA platform. For $99, I participated to this full-day workshop, with a small training of the hardware and software tools available for this platform, and went away with a the development kit that was used for the practical labs, the Arrow SoCKit evaluation board.

sockit-1

The Altera SoC FPGA platform is a single chip that combines a dual Cortex-A9 processor and a Cyclone V FPGA (other variants are available, such as single core ARM, or bigger FPGAs). Since both are integrated on the same chip, it removes the need for a complex bus between the application processor and a separate FPGA, and it provides a very high bandwidth between the processor and the FPGA to exchange data. As can be seen on the following diagram, the Arrow SocKit evaluation board has a good number of peripherals, some of them connected to the HPS side of the processor (HPS stands for Hard Processor System, which is dual Cortex-A9 and all the hard peripherals), and some others connected to the FPGA side of the processor. Specifically, the workshop was centered around playing with the LEDs and the buttons connected to the FPGA side.

Block diagram of the SoCKit evaluation board
Block diagram of the SoCKit evaluation board

The morning of the workshop was dedicated to the hardware part, using two tools: Qsys (Altera’s System Integration Tool) and Quartus II (FPGA design). The Qsys tool allows to graphically build the hardware architecture of the SoC FPGA: you can bring the Hard Processor System, and then some additional FPGA IPs and connect them together. In the examples, two PIO IPs were added to the FPGA side, one to control LEDs and the other to control buttons. All the pin muxing and the DRAM configuration also takes place in Qsys, which in the end will automatically generate numerous elements:

  • FPGA code to be used by Quartus (for the FPGA side)
  • pre-loader source code (actually modifications to U-Boot to take into account the pin muxing and the DRAM configuration)
  • a Device Tree for the Linux kernel (that takes into account all the peripherals enabled in the system configuration)
  • etc.
Qsys allows to design the overall architecture of the SoC FPGA components, including all the FPGA IPs. On this picture, we see the two PIO IPs used to control LEDs and buttons.
Qsys also to design the overall architecture of the SoC FPGA components, including all the FPGA IPs. On this picture, we see the two PIO IPs used to control LEDs and buttons.

Once this system-level configuration is done with Qsys, Quartus is used to synthesize the FPGA side. Finally, the FPGA bitstream can be loaded into the FPGA, and using a JTAG connection and a System Console tool, one can directly read/write the registers of the FPGA IPs that have been integrated, and easily test them without having any software running. With the SOCKit evaluation board, the JTAG connection takes place through a USB to micro-USB cable, so no special hardware is needed.

The afternoon of the workshop was dedicated to the software part, obviously mostly centered on U-Boot and Linux. The pre-loader source code (actually the U-Boot SPL) is generated by the Qsys tool to take into account all the details of the system configuration, so after doing a build of U-Boot, the labs explained how to push the U-Boot binary to the board using the JTAG connection, under the DS-5 development environment (which is the development environment provided by ARM, based on Eclipse, with a special integration of Altera tools). The lab showed that you can set breakpoints, and do step by step execution of U-Boot, as one would expect. One nice thing is that the “Peripheral Registers” view in DS-5 is automatically updated according to the hardware components in the system: all the registers of the FPGA IPs we had integrated were immediately visible, and one was able to play with the LEDs and buttons directly from DS-5 through the JTAG connection. The remainder of the labs were dedicated to booting a pre-built Linux kernel, using a Device Tree generated by Qsys previously and loaded by U-Boot. Once Linux was booted, the labs were demonstrating how to play with the LEDs using /sys/class/leds/.

Registers of FPGA IPs are automatically visible in DS5
Registers of FPGA IPs are automatically visible in DS5

All in all, it was a great workshop, giving a good overview of the tools they offer and the capabilities of such platforms, I definitely recommend others to look for the other dates of this workshop and attend, it was well worth the price (dates in US, dates in Europe). I also appreciated that the amount of marketing/commercial was really reduced to the minimum. After the lunch break, there was an additional presentation by a person from Linear Technology about the power supply architecture of two boards based on this SoC FPGA processor, and it was very technical and highly informative.

As an embedded Linux developer, I’ve however found the tools to be too much based on graphical interfaces, with millions of windows and buttons to fill in, and a sensation of not really controlling which tool was doing what exactly. For example, it is a bit unclear at first which files are really your “source” files and which other files are generated. Having graphical interfaces also makes me wonder how all the build steps can be automated. It seems like version control and automated builds are not necessarily taken into account when designing those tools, but more investigation is certainly needed to get a good understanding of what those tools are doing.

On the Linux kernel side, it is worth noting that Altera has engaged into an upstreaming process for this architecture, and one can find some mainline support for this platform in arch/arm/mach-socfpga in the kernel tree. It is worth mentioning that Xilinx also has a similar architecture combining a dual Cortex-A9 and FPGA, called Zynq, which has mainline kernel support in arch/arm/mach-zynq.

And below, the front and bottom of the SoC Kit evaluation board:

SoC Kit evaluation board back side

SoC Kit evaluation board front side

GSoC: Buildroot gaining improved ARM multimedia support

Google Summer of Code 2013 logo

The Buildroot project is participating for the first time to the famous Google Summer of Code. This program, operated by Google, allows open-source projects to have students working on specific tasks for the summer, and the students get paid for their work, get mentored by open-source developers, learn about software development, open-source communities and more.

For its first participation to the GSoC, the Buildroot community has chosen one project: improving support for multimedia features of popular ARM SoCs. This consists in packaging in Buildroot all the necessary libraries and software components to support OpenGL, OpenVG, EGL, OpenMAX and similar technologies for the major ARM processors. The selected student for this project is Spenser Gilliland and Bootlin engineer Thomas Petazzoni is mentoring Spenser for this project.

The focus of the project is to add support for the multimedia features of the OMAP3, OMAP4 and AM33xx processors from Texas Instruments, the Broadcom processor found on the RasberryPi, the i.MX6 processor from Freescale, the Exynos 4 from Samsung and the Allwinner A1x processors. Throughout the next three months, support for the multimedia capabilities of those processors in Buildroot should become easier to use.

Spenser has already contributed support for GStreamer 1.x in Buildroot (which required upgrading the entire GLib/Gtk/Webkit stack) and OpenMAX support for the RasberryPi, and he is currently working on OpenGL support for the OMAP3/OMAP4/AM33xx platforms. The initial part of Spenser’s work will be in the next 2013.08 Buildroot release, while the remainder will have to wait the 2013.11 release.

For more details about the project, see its description on the eLinux.org wiki, and you can also keep track of Spenser’s progress.

Buildroot 2013.05 released, Bootlin contributions inside!

Buildroot logoAs planned by the release schedule, the Buildroot 2013.05 version landed at the end of May. Peter Korsgaard, the project’s maintainer, highlighted the most important additions in his release email. With more than 900 commits, it has been the busiest ever development cycle, showing that the Buildroot project is more and more active.

With 175 commits in this release, Bootlin has again participated significantly to the development of Buildroot:

   217  Gustavo Zacarias
   167  Thomas Petazzoni (Bootlin)
   109  Will Wagner
    86  Peter Korsgaard
    44  Simon Dawson
    27  Yann E. MORIN
    25  gilles.talis@gmail.com
[...]
     6  Maxime Ripard (Bootlin)
[...]
     1  Alexandre Belloni (Bootlin)
     1  Ezequiel Garcia (Bootlin)
[...]

Amongst the features and improvements contributed by Bootlin:

  • Support for the next generation Wayland display server has been added. For now, only Wayland over the framebuffer is supported, but additional improvements are expected to come in the future.
  • Integration of packages to build all the Qt5 components: qt5base, qt5declarative, qt5graphicaleffects, qt5imageformats, qt5jsbackends, qt5multimedia, qt5quick1, qt5script, qt5svg, qt5webkit and qt5xmlpatterns.
  • A mechanism of virtual packages to expose the OpenGL, OpenVG and EGL implementations has been put in place, with for now the RasberryPi providing such implementations. Those virtual packages are for example used in the Qt5 packages mentionned above, for those that require OpenGL.
  • A cleanup of Buildroot core dependencies: flex and bison are no longer mandatory to use Buildroot, they are automatically built when needed. This apparently simple move required a number of fixes and updated to a significant number of packages.
  • Many external toolchains were updated, especially the Linaro toolchains.
  • The build process of gdb was converted to the package infrastructure, instead of being a hand-written Makefile. This is part of an effort to progressively convert the toolchain building process to the package infrastructure.
  • A default configuration was added for the Atmel AT91SAM9G45M10-EK evaluation board, which allows Buildroot users to easily build a minimal working system for this platform.
  • A number of build issues were fixed by Maxime Ripard, thanks to the daily automated builds done by the Bootlin Jenkins system that Maxime has set up.
  • A huge number of build issues trigerred by the autobuilders have also been fixed thanks to Bootlin engineers contributions.

In addition to this, Thomas Petazzoni has done some major improvements to the automated build system that the Buildroot project uses, which he detailed in an e-mail sent to the project mailing list. These improvements make the autobuilder infrastructure more scalable, and allows to provide statistics, and a much better daily report sent to the project’s mailing list.

In detail, the contributions of Bootlin were:

Bootlin contributions to the 3.9 kernel

A few months ago, we published a blog post showing our contributions to the 3.8 Linux kernel. With 128 commits merged in 3.8, Bootlin was ranked as the 17th company in terms of kernel contributions.

The 3.9 kernel has been released a few weeks ago, with again a significant number of contributions from Bootlin. According to these statistics, Bootlin contributed 92 patches during the 3.9 cycle, making the company the 26th most important contributor to the Linux kernel for this release, and this time, five engineers from Bootlin contributed patches.

Among the contributions that we made:

  • Added a basic infrastructure for irqchip drivers in the drivers/irqchip directory. This directory is now used to store the drivers for the IRQ controllers of various processors.
  • Made a number of improvements to the Marvell SDIO driver, including the addition of a Device Tree binding for it, and enabled its usage on Marvell Armada 370 and Armada XP platforms, as well as converting the Marvell Kirkwood platforms to use Device Tree probing instead of legacy probing for their SDIO interface.
  • Contributed a number of improvements to support Crystalfontz i.MX28 based modules, including the Device Tree for the CFA10037 expansion board, various improvements for the CFA10049 expansion boards, and a driver for the Himax HX8357B LCD controller.
  • A large number of improvements to the support of the Allwinner ARM SoCs, most notably a pinctrl driver for those SoCs, which allows to configure the muxing of I/O pins, and a gpio driver, to use the pins as general-purpose I/Os. We also contributed the support for the Miniand Hackberry platform, based on an Allwinner SoC. This work is all done by our engineer Maxime Ripard, who is the maintainer of the Allwinner SoC support in the Linux kernel.
  • Improvements to the PCA953x driver (for I2C GPIO expanders) in order to support the PCA9505 chip, that has 40 GPIOs. This required quite some work, as the PCA953x was originally limited to chips having at most 32 GPIOs. This improvement was done in order to support the GPIO expander box provided by Globalscale for the Armada 370-based Mirabox platform.
  • We added support for the Real Time Clock on Armada 370 and Armada XP based platforms, added support for local timers on Armada XP, added support for the new Armada XP GP evaluation board.
  • We enabled support for the SPI controllers and the USB controllers on Armada 370 and Armada XP based platforms.

Our high rate of contributions is going to continue, as we already have 95 patches merged for the upcoming 3.10 kernel and have already submitted a number of patches for the 3.11 kernel.

Here are details about our contributions to the 3.9 kernel: