Ubuntu 10.04 on the IGEPv2 board

Installing Ubuntu 10.04 on the IGEPv2 board. Using the board as a small server.

IGEPv2 boardLast year, folks at Texas Instruments told me about the IGEP v2 board. This board is similar to the Beagle board, but also features 512 MB of RAM and NAND flash (instead of 256 for the Beagle), on board Ethernet (RJ45), Wi-Fi and Bluetooth, all this for only 145 €! Its fast ARM CPU (TI OMAP 3530 running at 720 MHz) and graphical capabilities allow it to be used in for services usually performed by desktop or server CPUs.

At the Bootlin main office, we needed a server to share files, create backups and upload these backups to our servers on the Internet. I decided to use the Ubuntu 10.04 distribution on ARM, based on Debian GNU/Linux. As I didn’t find all the details I needed on the IGEP community website, here are the steps that I took. Several details were found on the http://labs.igep.es/index.php/How_to_get_the_Ubuntu_distribution page though.

This page assumes that you are familiar with building the Linux kernel, controlling an embedded board from a serial line and booting it, and using the GNU/Linux system in general (see the training materials from our embedded Linux course). Beginners may be lost because we don’t give all the details, but more experienced developers should just find the board specific details that they need.

First, get an SD card (at least 2 GB), and prepare its partitions with the mkcard.sh utility.

To compile your kernel, get a CodeSourcery toolchain for ARM. I used the 2010q1 release. Install it in /usr/local/CodeSourcery/arm-2010q1/ (for example)

Get the kernel sources:

$ mkdir $HOME/igep
$ cd $HOME/igep
$ git clone git://git.igep.es/pub/scm/linux-omap-2.6.git
$ cd linux-omap-2.6/

Let’s switch to the latest stable version:

$ git tag
v2.6.28.10-3
v2.6.28.10-igep0020b-0
v2.6.28.10-igep0020b-1
v2.6.28.10-igep0020b-2
v2.6.33.2-0
v2.6.33.4-0
$ git checkout -b v2.6.33.4-0 v2.6.33.4-0
Checking out files: 100% (13116/13116), done.
Switched to a new branch 'v2.6.33.4-0'

Now, set the environment variables for cross-compiling the kernel sources to the arm architecture:

$ export PATH=/usr/local/CodeSourcery/arm-2010q1/bin:$PATH
$ export ARCH=arm
$ export CROSS_COMPILE=arm-none-linux-gnueabi-

Now, take the default configuration for the board and build your kernel:

$ make help | grep igep
$ make igep0020_defconfig
$ make -j 4
$ make uImage

It’s time to build your Ubuntu filesystem, using the Rootstock utility:

$ tar zxvf rootstock-0.1.99.3.tar.gz
$ cd rootstock-0.1.99.3
$ sudo ./rootstock --fqdn igepv2 --login mike --password letmein \
  --imagesize 2G --seed build-essential,openssh-server --dist lucid

Copy the kernel to the first partition of your SD card:

cp arch/arm/boot/uImage /media/boot/
cp .config /media/boot/config-2.6.33.4

Install the root filesystem on the second partition of your SD card:

$ cd /media/rootfs/
$ sudo tar zxvf $HOME/igep/rootstock-0.1.99.3/armel-rootfs-201006102239.tgz

Configure the rootfs to let you log in on the serial console (ttyS2 with OMAP). Do this by copying etc/init/tty1.conf to etc/init/ttyS2.conf and replacing tty1 by ttyS2 in this file.

Install kernel modules manually for the first time:

$ mkdir -p /lib/modules
$ cd $HOME/igep/linux-omap-2.6/
$ make INSTALL_MOD_PATH=/media/rootfs modules_install

In the Rootstock version I tested, the specified user didn’t get created (bug report). To be able to log in, I had to disable the root password by removing the first * character in the root entry in etc/shadow:

We are now ready to boot our new system. First, unmount your SD card partitions:

$ sudo umount /media/boot
$ sudo umount /media/rootfs

Insert your SD card in the slot on your board, connect your serial cable and in the U-boot prompt on the serial line, configure the kernel boot parameters:

$ setenv bootargs mem=512M console=ttyS2,115200n8 root=/dev/mmcblk0p2 rw rootwait
$ setenv bootcmd 'mmc init 0 ; fatload mmc 0 80000000 uImage ; bootm 80000000'
$ setenv autostart yes
$ saveenv
$boot

You should see your Linux kernel boot and get to a login shell. Log in as root with no password.

It is now time for the final tweaks. First, create a non root user (remember the Rootstock bug), allow it to run the sudo command, and choose a root password too:

adduser mike
adduser mike sudo
passwd

Let’s cope with a last Rootstock bug. Add the updates and security repositories to /etc/apt/sources.list:

deb http://ports.ubuntu.com/ubuntu-ports lucid-updates main
deb http://ports.ubuntu.com/ubuntu-ports lucid-security main

Without this, you would miss package updates and security releases, and your packages would never change!

If you use the IGEP board as a server as I do, you may need your server to have a fixed MAC address. The trouble is the e2prom storing the MAC address is not populated by default, and every time you boot, the kernel gives you a random MAC address.

The easiest fix I found was to choose an arbitrary MAC address (you can take the first random one that you get), and force it in /etc/network/interfaces:

auto eth0
iface eth0 inet dhcp
hwaddress ether 00:01:04:1b:2C:1F

As the IGEPv2 board doesn’t have a battery by default, it won’t be able to keep the correct time. You can use the ntp daemon to address this:

sudo apt-get install ntp

Your configuration should now be complete. You can now use your IGEPv2 board as a tiny, ultra low power server with Ubuntu server. All the rest is ordinary Debian / Ubuntu server administration. Of course, you can also install desktop packages and use your board as a desktop replacement (you may need to add kernel command line settings for graphics). Have fun!

By the way, the IGEPv2 board is not the best solution if you all you need is a server. The amazing graphical capabilities of the OMAP chip would just be useless. For a server, better, cheaper and more powerful alternatives are the SheevaPlug and GuruPlug. Don’t miss these very nice devices!

Linux device drivers architecture talk at Libre Software Meeting

recursive device modelThomas Petazzoni gave a talk on the Linux kernel architecture for device drivers at the Libre Software Meeting in Bordeaux, France. While the talk was given in French, the materials are in English and can therefore benefit a larger audience. The talk seems to have been well-received, especially from people already having a basic Linux kernel development experience. The topics covered are part of our Linux Kernel development training, and are also usually very appreciated from the trainees already having Linux kernel experience.

The idea of the talk is to give an overview of how device drivers fit into the kernel, both to expose their functionality to upper layers (such as a network device driver exposes itself to the kernel network infrastructure) and to detect/access the hardware using the device/driver model, which is quite hard to understand from the source code only.

The talk went through the following sections :

  • First a basic introduction to device drivers: how devices are seen from userspace applications, and how a simple, raw, character driver can be implemented. It allowed to expose the principle of operations and their similarity with methods in object-oriented programming, and the principle of registration to an upper-layer infrastructure
  • Then, an introduction to what I call « kernel frameworks », i.e kernel subsystems that specialize a general device type (i.e character device) into a particular device type (i.e serial port device, framebuffer device, etc.). The talk illustrates this with the framebuffer core and the serial port core.
  • Finally, an explanation about the device model: bus drivers, adapter drivers and device drivers. I started with the example of the USB bus: being a dynamically-enumerated bus, it provides a good illustration of the device model principles. At the end, I explained how the device model works for the devices embedded into a SoC using the platform drivers/devices mechanism

The slides for this talk are no longer available, but their updates are now integrated in our Linux kernel and driver development training materials which are freely available.

Embedded Linux Opportunities – Keynote in Eindhoven

Fontys UniversityIf you live near Eindhoven in the Netherlands, or if you can easily get there, you may be interested in the seminar organized by Fontys University on June 25, 2010.

I will be in Eindhoven on that week to give our Embedded Linux System Development course. The session is organized by Fontys University and Alten PTS, and Fontys invited me to give a keynote at their seminar on Friday.

The seminar topic is: “Embedded Linux for SMEs” – A guide to professional development:

  • Ruud Ermers (Fontys) – A Guide to professional development
  • Gerben Blom (Alten PTS) – Android for SMEs
  • Michael Opdenacker (Bootlin) – Embedded Linux Opportunities. Ideas and advise for the Open Minded.
  • The grand race, between arm9 controlled vehicles.
  • Technical demos

Here is the abstract for my speech:

Michael Opdenacker created Bootlin in 2004, a company that is best known worldwide for the technical resources it shares with the embedded Linux community. The more this company shares (in the true sense of sharing: giving and receiving), the more successful it gets.

That’s why Michael will come to exchange ideas and experience on building successful embeddded Linux systems and businesses. Just make sure you send him a SIGSTOP signal at the end of his timeslot.

To water your mouth, this talk will show a few things which are possible with embedded Linux and free software building blocks. You will see how you can turn an idea into something that could keep you busy during your whole life if you are not careful.

So, if you are in the Eindhoven area, this could be a nice opportunity to meet. I hope to see you there!

Buildroot 2010.05 released: again many Bootlin contributions

Buildroot logoJust one day before the end of May, Buildroot 2010.05 has been released by Peter Korsgaard, as predicted by the fixed release schedule used by the project. It can be downloaded at http://buildroot.org/downloads/buildroot-2010.05.tar.bz2. For the record, Buildroot is a simple and efficient tool to build embedded Linux systems: cross-compiling toolchain, root filesystem, kernel image and bootloader.

Major changes

The major user-visible changes are:

  • Re-organization of the menuconfig layout for packages. All packages are now organized in categories, making them easier to find
  • Our X.org package set has been upgraded to X.org 7.5.
  • Several new packages have been added: cdrkit, cramfs, genext2fs, genromfs, libatomic_ops, librsync, libusb-compat, lmbench, netperf, squashfs, squashfs3, squid. Many of them have been added as the result of a filesystem code generation cleanup
  • On the internal toolchain side (i.e toolchains generated by Buildroot), we have added support for uClibc 0.9.31, GCC 4.4.4, GDB 7.x and binutils 2.20.1.
  • On the external toolchain side (i.e. re-using existing toolchains), we have improved support for multilib toolchains (such as CodeSourcery toolchains)

In addition to these changes, 41 bugs of our bug tracker have been fixed, and dozens of packages have been upgraded or fixed.

Bootlin contributions

Bootlin has again made significant contributions to this release:

git shortlog -s -n 2010.02..
   224  Paulius Zaleckas
   182  Thomas Petazzoni, from Bootlin
   148  Peter Korsgaard
    28  Gustavo Zacarias
    26  Will Wagner
    14  Lionel Landwerlin
     6  Yann E. MORIN
[...]

The things we have contributed include:

  • A big cleanup in the Buildroot code that generates the root filesystem images. It has been moved from various directories in target/ to a single, central location: fs/. The code that handled the compilation of host utilities to generate the filesystem images (genext2fs, cdrkit, mtd-utils, cramfs, squashfs, etc.) has been moved to normal packages, and an infrastructure has been added to factorize the common code of the various filesystem generation makefiles.
  • Better support for multilib external toolchains
  • A new script that generates nice dependency graph (see below)
  • A clarification of the gettext integration, to make it work properly with glibc toolchains.
  • Fixes to bug #75, bug #1789.
  • Dozens of build fixes found by testing random configurations.
  • Various code cleanups, that lead to the removal of several configuration options, which makes the usage of Buildroot a little bit easier.

Dependency graph generation

Thanks to the new package infrastructure that we have included in Buildroot a few releases ago, it is now easier to retrieve the list of dependencies of each package in a generic way. Using this, I recently implemented a dependency graph generation tool. It allows to generate nice graphs of the dependencies for a given package, like libgtk2 in the following example (click for the full sized version):

Note that packages in red are packages that do not use the generic or autotools infrastructure, so we couldn’t determine what their dependencies are.

We can also generate the dependency graph for a complete Buildroot configuration, with all packages:

Using this tool is fairly easy. You must first install the graphviz package on your distribution. For a single package dependency graph:

./scripts/graph-depends libgtk2 > libgtk2.dot
dot -Tpdf libgtk2.dot -o libgtk2.pdf

For a full dependency graph:

./scripts/graph-depends > full.dot
dot -Tpdf full.dot -o full.pdf

Note that the dependency graph always depends on the selected set of packages. It is not the absolute dependency graph, which would contain all existing dependencies. It only show the dependencies as they are in your current configuration.

Plans for the next release

The next release is scheduled for August (2010.08). I have in mind various things:

  • Cleanup of the bootloader compilation code and integration of support for Barebox. This is already implemented in one of my branch, so I should be able to push this fairly soon to Buildroot
  • Cleanup of the Linux kernel compilation code, with a much-wanted simplification of this. Again, this is already implemented on my side, but requires a little bit more work before being able to push this
  • Continue the effort to convert packages to the generic or autotools infrastructure. I have already sent a status update on this topic to the project mailing-list. We have 56 packages to convert to the generic infrastructure and 77 to convert to the autotools infrastructure.
  • Integration with Crosstool-NG, which is a job that Yann Morin, developer of Crosstool-NG has already started.
  • More improvements of the external toolchain integration
  • If some time is left, a cleanup and reorganization of the board support mechanism, so that we can add more boards in a sane way

Issues with changing Ethernet device names

Beagle boardNo eth0 interface in your system!

If you are using Ubuntu or Debian on ARM, as I am doing at the moment, you may be surprised to see that your system has an eth1 interface (or even eth5), but no eth0!.

This also happens on x86, of course, and probably with other distributions. I faced and understood this problem when I migrated a KVM virtual machine from a regular network interface card to a virtio one. eth0 was gone, and because my network settings were bound to this interface, I couldn’t connect to my virtual machine anymore.

In recent releases of Ubuntu, this happens because of some udev rules, which try to make sure that each network card always gets bound to the same ethx device. So, every time the system see a new network card, the /etc/udev/rules.d/70-persistent-net.rules file gets updated as follows:

# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x14e4:0x167a (tg3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:18:8b:87:52:12", ATTR{type}=="1",
KERNEL=="eth*", NAME="eth0"

# USB device 0x:0x (asix)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:25:00:eb:eb:26", ATTR{type}=="1", KERNEL=="eth*",  NAME="eth1"

This can happen if you replace a USB to Ethernet dongle by another one with a different MAC address, or if you reuse a filesystem built on another board. Even if the board and Ethernet device are the same, you will get eth1 instead of eth0. This can be annoying if you configured your network through /etc/network/interfaces or if you bring up your network with your own scripts.

A clean solution for this problem is simply to remove the /etc/udev/rules.d/70-persistent-net.rules file. The next time you boot, you will get back to a normal eth0 network device.

I checked that these persistent net naming rules are not part of udev releases. In other distributions, you could have udev without such rules.

Location of Embedded Linux Conference Europe 2010 announced

Cambridge, UKWhile the American Embedded Linux Conference always takes place in the San Francisco area in California, the European Embedded Linux Conference Europe travels each year in a different country across Europe: it took place in Austria in 2007, in the Netherlands in 2008 and in France in 2009. The location for the 2010 edition of ELCE has been announced recently: it will take place on October 27th and 28th in Cambridge, United Kingdom. As usual there will be many talks, Bird-of-a-Feather sessions, technical demonstrations and more. If you’re an embedded Linux developer in Europe, you should definitely ask your employer to send you to this conference!

Presentations on the following topics are encouraged: audio, video, and graphics systems for embedded products, security, system size, boot-up time, meeting real-time constraints, power management, streaming media, flash memory devices and filesystems, technologies related to cell phones, digital set top boxes, handheld devices, or other CE products, development tools for embedded users, use of Linux in actual products, practical experience and war stories, standards for CE products. The proposal must be received by June, 30th. My colleague Michael Opdenacker, founder of Bootlin, is a member of the program committee.

On the day before, note that there will also be an interesting event: GStreamer Conference 2010. As GStreamer is a key component of today’s multimedia systems based on embedded Linux. Staying in Cambridge for three days instead of two could then make a lot of sense.

Crosstool-NG 1.7.0, Bootlin contributed basic Blackfin support

Yann E. Morin, maintainer of Crosstool-NG, has just announced version 1.7.0 of his toolchain building tool we like so much at Bootlin. For the record, Crosstool-NG allows to build cross-compiling toolchains for a wide-range of architectures, using either glibc, uClibc, eglibc or newlib, and everything is configured through a menuconfig interface, familiar to all embedded Linux developers.

Blackfin architectureBootlin has contributed to this release, by adding some support for the Blackfin architecture, on which we have been working recently. Crosstool-NG is now capable of generating a FDPIC cross-compiling toolchain for Blackfin. FDPIC allows to use shared libraries even on no-MMU architectures like the Blackfin. This support of Blackfin in Crosstool-NG is only the beginning, a lot of work remains to be done to make sure that the toolchain works really well, and to support FLAT binaries and bare-metal as well.

Here are the patches that Bootlin has contributed for this release :

The other major changes for this release are :

  • two new architectures have been added: Blackfin and mips64
  • building canadian-crosses has been enhanced
  • experimental support for gcc-4.5 has been added
  • eglibc can be optimized for size (instead of speed)
  • companion libraries can be built statically, removing the need for a wrapper
  • many components versions have been added, and associated patchsets updated
  • some less important new features, or enhancements for stability and usability
  • a lot of bug-fixes, of which some to help on Darwin & *BSD

The 1.7.0 release can be downloaded at http://ymorin.is-a-geek.org/download/crosstool-ng/crosstool-ng-1.7.0.tar.bz2 and the homepage of the Crosstool-NG project is http://ymorin.is-a-geek.org/projects/crosstool. Enjoy !

New jobs at Bootlin

Penguin worksLooking for kernel and embedded Linux experts

Bootlin is looking for experienced members of the Free Software community to satisfy increasing demand for development, consulting and training on embedded Linux and on the Linux kernel.

One thing that distinguishes our positions from others is that contributing to the community will be part of your objectives.

All the details can be found on our careers page.

ELCE 2009 videos

Videos from the Embedded Linux Conference Europe, Grenoble, October 2009

ELCE 2009Just a few weeks before the next edition of the Embedded Linux Conference in San Francisco, here are the videos from the previous edition in Europe a few months ago.

These videos were shot by Satoru Ueda and Tim Bird (Sony), Ruud Derwig (NXP) and by Thomas Petazzoni and Michael Opdenacker (Bootlin). As usual, they are released under the terms of the Creative Commons Attribution – ShareAlike Licence version 3.0.

Ruud DerwigIf you have never been to an Embedded Linux Conference yet, these videos should show you how useful this conference is for embedded Linux system developers. This is the place where you can discover new development tools and technologies that will change your working life, benefit from the experience from your peers, get the opportunity to talk to the fantastic people who implement the Free and Open Source software that makes your system run, and win cool penguin goodies. So, don’t miss next next edition in San Francisco.

ELC 2010 program announced

Japantown, San FranciscoThe program of talks and BOFs of the 2010 edition of the Embedded Linux Conference has been published a few days ago, an opportunity to look at the most important and interesting conference for embedded Linux developers. For the record, ELC 2010 will take place from April, 12th to April, 14th in San Francisco, CA, USA, in the same place as the 2009 edition.

A nice set of talks

  • A set of real-time related talks: Real-Time Linux Failure, by Frank Rowand (works for Sony, well known for his preempt-rt related talks at various ELC conferences), Effective Use of RT-Preempt, by Kevin Dankwardt, Using Interrupt Threads to Prioritize Interrupts, by Mike Anderson (also well known for his very interactive talks, he will also be giving his traditional Using JTAG to debug Linux device drivers tutorial), Measuring Responsiveness of Linux Kernel on Embedded System, by YungJoon Jung and DongHyouk Lim.
  • A talk by Grant Likely about Flattened Device Tree ARM support update, an effort to convert the ARM architecture to the same organization used in PowerPC, with a device tree file describing the hardware details instead of platform_device definitions in plain C. An important change for anyone doing ARM kernel development.
  • Several power-management related talks: Runtime Power Management: Overview and Platform Implementation, by Kevin Hillman (who works for Deep Root Systems and has done a huge amount of work in the OMAP power management area). Runtime Power Management is probably the most important change done recently to the power management infrastructure of the Linux kernel, so this talk is certainly worth a look, all the more as Kevin is a very good speaker. On power manegement, there will also be other talks : DVFS for the Embedded Linux, by Yong Bon Koo and Youngbin Seo, Wake-ups effect on idle power for Intel’s Moorestown MID and smartphone platform, by German Monroy (Intel), Workload based aggressive Power Management on the Intel Moorestown MID and future Intel MID/Smartphone Platforms, by Sujith Thomas (from Intel).
  • Japan Town, San FranciscoThe usual tracing-related talks, with Using the LTTng tracer for system-wide performance analysis and debugging by Mathieu Desnoyers and Ftrace – embedded edition, by Steven Rostedt. A talk on debugging Linux toolchain overview with advanced debugging and tracing features, by Dominique Toupin.
  • Talks about platforms: a keynote by Greg Kroah Hartmann on Android: a case study of an embedded Linux project (during which Greg will probably explain why the Android kernel modifications are not mainlined), Experiences in Android Porting, Lessons learned, tips and tricks, by Mark Gross and Understanding and Developing Applications for the Maemo Platform, by Leandro Melo de Sales, even though the recent merge of Maemo and Moblin to create MeeGo is likely to change some technical aspects of application development for this platform.
  • The question of multi-core now also seems to be present in embedded conferences: Strategies for Migrating Uniprocessor Code to Multi-Core, by Mike Anderson, Embedded Multi-core with Adeos, Dan Malek, Lock-free algorithm for Multi-core architecture, Hiromasa Kanda. Multi-core Scheduling optimizations for soft real-time multi-threaded applications – A cooperation aware approach, Lucas Martins De Marchi.
  • Some security talks, with Mike Anderson (again !) talking about Creating a Secure Router Using SELinux and Jake Edge about Understanding threat models for embedded devices
  • Some more-or-less multimedia-oriented talks: Supporting SoC video subsystems in video4linux, by Hans Verkuil, An Introduction to the Qt Development Framework, by Jeremy Katz, GeeXboX Enna: embedded Media Center, by Benjamin Zores, Case Study – Embedded Linux in a digital television STB, by Melanie Rhianna Lewis
  • In the other talks, I’ve noted the Small Business Owners BOF by Grant Likely, Evaluation of Data Reliability on Linux File Systems by Yoshitake Kobayashi, Porting the Linux Kernel to x86 MID platforms, by Jacob Pan, Linux without a bootloader? by Greg Ungerer, Kexec – Ready for Embedded Linux by Magnus Damn, Custom hardware modeling for FPGAs and Embedded Linux Platforms with QEMU, by John Williams, Edgar Iglesias.

Both Michael Opdenacker and I will be there at ELC. We hope to meet you during this conference!