Free Electrons becomes Bootlin

Bootlin logo

Free Electrons is changing to a new name, in the context of a trademark dispute.

Reasons for changing

On July 25, 2017, the company FREE SAS, a French telecom operator, known as the owner of the free.fr website, filed a complaint before the District Court of Paris against Free Electrons and its founder Michael Opdenacker for infringing upon 3 trademarks which include the word “free” and on FREE SAS’s rights on its domain name and its company name.

In this complaint, FREE SAS asked, among others, the French judges to order Free Electrons and its founder Michael Opdenacker to pay the total sum of 107,000 euros on various grounds, to order Free Electrons to change name, to delete the domain name “free-electrons.com” within 15 days and to cease all use of the sign “FREE ELECTRONS” but also of the term “free” alone or with any other terms in any field in which FREE SAS is active or for any goods and services covered by its prior trademarks.

Michael Opdenacker and Free Electrons’ management consider that these claims are unfounded as both companies were coexisting peacefully since 2005.

The services we offer are different, we target a different audience (professionals instead of individuals), and most of our communication efforts are in English, to reach an international audience. Therefore Michael Opdenacker and Free Electrons’ management believe that there is no risk of confusion between Free Electrons and FREE SAS.

However, FREE SAS has filed in excess of 100 oppositions and District Court actions against trademarks or names containing “free”. In view of the resources needed to fight this case, Free Electrons has decided to change name without waiting for the decision of the District Court.

This will allow us to stay focused on our projects rather than exhausting ourselves fighting a long legal battle.

The new name

Amongst all the new names we considered, “Bootlin” came out as our favorite option. It can’t express all our values but it corresponds to what we’ve been working on since the beginning and hope to continue to do for many years: booting Linux on new hardware.

Of course, “booting” here shouldn’t be limited to getting a first shell prompt on new hardware. It means doing whatever is needed to run Linux by taking the best advantage of software and hardware capabilities.

Same team, same passion

Nothing else changes in the company. We are the same engineers, the same Linux kernel contributors and maintainers (now 6 of us have their names in the Linux MAINTAINERS file), with the same technical skills and appetite for new technical challenges.

More than ever, we remain united by the passion we all share in the company since the beginning: working with hardware and low-level software, working together with the free software community, and sharing the experience with others so that they can at least get the best of what the community offers and hopefully one day become active contributors too. “Get the best of the community” is effectively one of our slogans.

Practical details

The only thing we’re changing is the name (“Bootlin” instead of “Free Electrons”), the domain name (bootlin.com instead of free-electrons.com) and the logo. The two penguins, our mascots which have been the key identification of Free Electrons for many years will stay the same. Except for the domain name change, all URLs should stay the same, and all e-mail addresses too.

For the moment, we’ve just migrated the mail and main web servers. The other services will be updated progressively.

For practical reasons, the name of the company running Bootlin will remain “Free Electrons” for a few more months. Until then, there won’t be any impact on the way we interact with our customers. We will let our ongoing customers know when the legal name changes.

What about links to free-electrons.com resources, made by community websites but also in mailing lists archives and in public forums? Of course, we redirected the old URLs to the new ones, and will continue to do so as long as we can. However, depending on the outcome of the legal procedure, we may not be able to keep the free-electrons.com domain forever. Therefore, we would be grateful if you could update all your links to our site whenever feasible, to avoid the risk of broken links in the future.

Bootlin at FOSDEM and Buildroot Developers meeting

The FOSDEM conference will take place next week-end in Brussels, Belgium. As the biggest open-source conference event in Europe, featuring a number of talks related to embedded systems and generally low-level development, Bootlin never misses this event!

Fosdem 2018 logo

This year, Bootlin engineer Miquèl Raynal will be giving a talk Drive your NAND within Linux – Forget the word “nightmare”, sharing details on the enhancements he has contributed to the Linux kernel MTD subsystem, and which are scheduled to be merged in the 4.16 Linux kernel release.

In addition to Miquèl’s talk, a number of other Bootlin engineers will be attending the event: Mylène Josserand, Quentin Schulz, Antoine Ténart, Boris Brezillon and Thomas Petazzoni.

Buildroot logoFinally, Bootlin is also sponsoring the participation of Thomas Petazzoni to the Buildroot Developers Meeting, which is a 2-day event dedicated to the development of the Buildroot embedded Linux build system. With 14 attendees, this event will have the largest number of participants it ever had. We take this opportunity to thank Google and Mind, who are sponsoring the event by providing the meeting room, lunch and social event for the attendees.

Tourist suggestions in Avignon, France

We very regularly organize public training sessions about embedded Linux and kernel and drivers training sessions in Avignon in France. Of course, we can also organize such sessions anywhere in the world.

AvignonThese tips only reflect our own interests and tastes. There are certainly others other worthy things to do and see in the area. We would recommend to buy a tourist guide book about Provence, which will have a much more exhaustive and objective coverage. See also the Avignon tourism website for updates and information about local events.

Sightseeing in Avignon

Popes Palace, AvignonThe historic city centre of Avignon, the Popes’ Palace, all the episcopal buildings and the Saint Benezet Bridge are listed as world heritage sites by UNESCO.  The Avignon city walls constitute the 2nd longest continuous wall in the world, after the Great Wall of China. The annual Festival d’Avignon (around July) is one of the biggest art festivals in the world.

The Popes Palace one of the 10 most visited monuments in France. It is the biggest Gothic edifice in all of Europe and awe-inspiring monument to the importance of Avignon in the Christian world of the Middle Ages.

You can go up to Rocher des Doms, the cradle of the city, for a refreshing break among the peacocks and swans by the pond. From this spot, you have panoramic views over the entire city, its tiled roofs and bell towers and over to Villeneuve-les-Avignon and all the surrounding areas (stairs behind the main building of the palace).

Saint Benezet Bridge, AvignonThe Saint Bénézet Bridge was built around 1180 – miraculously, according to legend, by a simple shepherd – to link the city to Villeneuve-les-Avignon. Over the years, wars and successive flooding by the Rhone partially destroyed the bridge. Today, the 12th century Saint Nicholas Chapel remains, as well as four arches of which the span constitutes, according to a prestigious civil engineer, an amazing feat for the period.

Museums

Hieronymus Bosch, Calvet Museum, Avignon

The Petit Palais Museum: exceptional collection of paintings from Italy and Provence (end 13th – beg. 14th century) and collection of Romanesque and Gothic sculptures from Avignon.

The Calvet Museum: in a splendid 18th century town house. Fine arts, painting and sculpture collection (15th – 20th century).

The Angladon Museum: ancient town house in the center of Avignon’s old town: masterpieces of the 19th and 20th century and 18th century living rooms, signed furniture, works of art, paintings and drawings.

Collection Lambert en Avignon: amazing showcase of modern art, spanning from paintings and land art to photography and video.

Shopping

Shopping Street, Avignon

At the bottom of the Place de l’Horloge, the main avenue, Rue de la Republique, divides the city into two. On one side, the Rue Saint Agricol leads to the chic and bourgeois neighborhood with the Rue Joseph Vernet, the 5th Avenue of Avignon where designer clothing and luxury goods shops abound, and the Place Crillon, with the prestigious Hotel d’Europe in a former private town mansion built in the 16th century for the Marquis de Graveson.

The Les Halles market of Avignon is a treasure trove for all kinds of products from the Provence area. Every day from 6:30 am to 1:30 pm vendors sell local vegetables, herbs, meats, olives and oil, and anything lavender-related. The market is housed in a big, modern building in the city center which facade is overgrown with plants and moss, making it stand out from the traditional architecture around it.

The Rue de la Republique, the central axis of the city, and the pedestrian areas are rather dedicated to big shopping brands: Fnac (books, music and technology), Zara, H&M, Eram, André, Naf-Naf, Promod, Kookai, etc.

You can also find many little designer shops in the very typical Rue des teinturiers.

Restaurants

To eat at the restaurants in town, it is recommended to book a table in advance. The best idea for up-to-date advice is probably to use sites that collect tourist reviews (here’s the TripAdvisor page for restaurants in Avignon).

Here are a few that we like and at least gave us a good impression the last time we went:

  • Le 26, 26 rue des Trois Faucons, 84000 Avignon
  • D’ici et d’ailleurs 4 Rue Galante, 84000 Avignon, 04 90 14 63 65

Here are more good restaurants worth visiting outside of Avignon:

  • Domaine de la Camarette 439 Chemin des Brunettes, 84210 Pernes les Fontaines, 04 90 616 078 (booking is compulsory)
  • Mas de Grès 1651 RD 901 – Four à Chaux-Isle sur Sorgue, 84800 Lagnes 04 90 20 32 85

More sightseeing in Provence

Culture and heritage

L'Isle sur la Sorgue
L’Isle sur la Sorgue:
“Comtadine Venice” with its
numerous canals and 300 antique sellers

Orange
Orange:
The Roman Theater and Triumphal Arch

Gordes
Gordes:
See the village, lavender fields and the Sénanque Abbey

Venasque
Venasque:
The medieval Baptistery and the old town

Vaison la Romaine
Vaison la Romaine:
Roman ruins and the old town

Carpentras
Carpentras:
The oldest synagogue in France

Pernes les Fontaines
Pernes les Fontaines:
The 40 fountains

Aix en Provence
Aix en Provence:
The mountain “Sainte Victoire” – subject of a number of Cézanne’s paintings.

Aix en Provence
Aix en Provence:
The Cours Mirabeau and the old town

Uzès
Uzès:
The famous Roman “Pont du Gard” aqueduct and the old town.

Arles
Arles:
The Roman amphitheatre and the old town.

Les Baux-de-Provence
Les Baux-de-Provence:
Cathédrale d’Images: sound and lights show in ancient and spectacular stone-pits.

Nîmes
Nîmes:
The Roman amphitheatre and the arenas.

Museums

Pernes les Fontaines Costumes comtadines
The museum of the Comtadin costume: in an ancient shop of the 19th century, exhibition of ancient Comtadin costumes and traditions linked to the materials.
The Maison Fléchier: reconstruction of the last santonnier’s workshop of Pernes les Fontaines, traditions of the Comtat Venaissin and of Provençal Christmas.

Fontaine de Vaucluse
Speleology museum: in a recreated life-size scenery, presentation of the researches and explorations of the Fontaine de Vaucluse and other speleological sites.
Library Pétrarque museum

RoussillonOchres in Roussillon

Conservatoire des ocres et pigments appliqués: ancient ochre factory created in 1871.

Carpentras
Duplessis museum: collection of fine arts of the town and art works by artists of the Comtat Venaissin
Pharmacy Museum: preserved in order from the 18th century with its jars of Italian, Montpellier and Moustier earthenware.

Cavaillon
Provençal nativity scene museum: one of the nicest santon collections of the Luberon and the Alpilles.

Gordes
500px photo (217546431)

Pol Mara museum: 200 works to discover in the castle with the biggest Renaissance chimney of Europe.

Nature

The régional Parc of the Luberon: it provides a habitat to a exceptional variety of flora and fauna, as for an architectural and natural heritage of high value.

The Mont Ventoux: The giant of the Provence (1912 m).

Fontaine de Vaucluse: natural resurgence of the Sorgue that gushes forth at the foot of a 230 m cliff.

The Provençal Colorado: Remains of an ancient ochre quarry on more than 30 hectares. Colorado provençal 2

The Alpilles: exceptional massif with famous towns and villages such as les Baux de Provence, Saint Rémy en Provence

The Thouzon cave in le Thor: discovered the 23rd of January 1902, it is the only natural cave laid out for tourism in the Vaucluse. It lies in the centre of the Pays des Sorgues.

The Dentelles de Montmirail: A small chain of mountain with a dramatically jagged shape of their peaks. A perfect place for hiking.

The gorges of the Nesque: imposing wild canyon, perfect for cycling and walking.

The Orgnacpothole in Bollène: Cave and regional prehistoric museum

The villages

Our region is famous for its picturesque villages, with their Mediterranean flair. One can, while walking, make discoveries and plunge in the history of the region.
Oppède le Vieux 2013 table d orientation 02

The villages with character in:
The Ventoux: Venasque, Le Beaucet, Séguret, Le Crestet, Brantes, Méthamis, Mazan
The Luberon: Gordes, Roussillon, Murs, Joucas, Bonnieux, Lacoste, Ménerbes, Lauris,
Oppède le Vieux, Lourmarin, Cabrières d’Avignon, Saignon, Curcurron
The Alpilles: Les Baux de Provence, Saint Rémy de Provence, Eygalières

Visit their markets – between rosemary and thyme, melon or figs the beautiful stallholders with their singsong accents sale peaches, apricots or a nice bunch of lavender.

You’ll find the planning for markets around Avignon here.

Do not miss: the farmer markets in Coustellet (Sundays from 9 a.m. until 1 p.m.) and in Velleron (in the summer, every day as from 6 p.m., except Sundays and bank holidays).
Market in Aix-en-Provence, France (6053041300)

The Wine road

Luberon red wine and food pairing

Between the Luberon, the Ventoux and the Côtes du Rhône, the Vaucluse can be proud of being one of the nicest wine region of France. The clayey chalky soil, the different grape varieties, and the wine grower’s know-how produce world famous wines. Here under, a list of the best wine growers of the Côtes du Rhône, Côtes du Luberon and Côtes du Ventoux.
To taste with moderation.

The Côtes du Rhône

Sablet:
Domaine de Piaugier

Rasteau:
Domaine de la Soumade

Vacqueyras:
Domaine des Amouriers

Gigondas:
Domaine de la Présidente

Uchaux:
SCEA Château d’Hugues

Châteauneuf du Pape:
Domaine de Beaurenard
La Cave du Verger des Papes
Château la Nerthe

Tavel:
Domaine de la Mordorée

Saint Gervais:
Domaine St Anne

Saint Victor la Coste:
Domaine Pélaquié

Beaumes de Venise:
Domaine de la Pigeade
Domaine de Durban
Vignes beaumes de venise

The Côtes du Luberon

Oppède:
Domaine de la Citadelle

Apt:
Domaine de Mayol

La Tour d’Aigues:
Château de la Dorgonne

La Motte d’Aigues:
Bastide du Claux

The Côtes du Ventoux

Mormoiron:
Château Pesquié

Mazan:
Domaine de Fondrèche
Terre de Solence

Flassan:
La Ferme Saint Pierre

The country products

Truffel
The famous truffle markets, with their special atmosphere, take place in winter (Fridays in Carpentras and Saturdays in Richerenches). Once in the season (the last week of December) takes place the big Truffle Market of Ménerbes, where is also to see the Maison de la Truffe et du Vin du Luberon (the House of truffle and wine of the Luberon), bringing lots of information on those subjects.

Olive oil
A list of oil mills in the Luberon area.Old Olive Oil Mill in Mouriès
The mill Moulin de la Balméenne in Beaume de Venise

Goats cheese
Family Morard in Buoux

Tapenade (olive paste)
”Les Délices du Luberon” in Isle sur la Sorgue

Berlingots (boiled sweet)
Confectionner’s shop ”Confiserie du Mont Ventoux” in Carpentras
Calissons (sweets made of ground almond)
Confectionner’s shop ”Confiserie du Roy René” in Aix en Provence

Croquettes (almond croquettes)
Traditional biscuit factory ”Aujoras” in Morières-les-Avignon

Nougat
The nougat factory ”Silvain Frères” in Saint Didier Nougat de Montélimar - 20090704

Activities for the young and the not-so-young

All year round

The Jarditrain in Saint Didier
miniature world dedicated to the train passion

Karting in Monteux

The Crocodile Farm in Pierrelatte

La Barben Zoo
The biggest animal parc of south-eastern France

Haribo Sweets museum in Uzès
Come and discover the great story of the sweets and try them.

Hot-air balloon
One of the most original and beautiful ways of discovering the Luberon, gliding with the birds over the beautiful landscapes. One can do that just next to ours, in Joucas.

During the summer

Kayak Vert in Fontaine de Vaucluse (March – November)
From Fontaine du Vaucluse down to l’Isle sur la Sorgue, discover the joys of canoeing during the 2 hours of the descent of the Sorgue. https://www.canoefrance.comKayak à l'Isle-sur-la-Sorgue

Passerelle des Cimes (bridges in the trees) in Lagnes (March – November)
Young and not-so-young will go for an adventure on differents secure itinaries: https://www.parcours-aerien.com

Mini golf de la Peupleraie in Isle sur la Sorgue (Mai – September)
Outdoor activity Espace for the whole family

Circus castle Alexis Gruss in Piolenc (Mai – September)
Visit of the Circus Gruss museum, make up, ride in a barouche, circus workshop, trapeze, horses. https://www.alexis-gruss.com/piolenc/accueil.htm©Jacques Gavard - Alexis Gruss

Colorado Aventure in Rustrel (March – November)
3 hour adventure for the whole family, courses for children and juniors.
https://www.colorado-aventures.com

Bambouseraie en Cevennes (March – November)
A unique journey in the heart of giant bambou forest and hundred-year-old trees, approximately 100 km away from Avignon.

Back from ELCE: participation to the Device Tree workshop

After publishing our slides and videos from the Embedded Linux Conference Europe (ELCE), reporting on talks selected by Bootlin engineers, and mentioning the award given to Michael Opdenacker, here comes the last blog post giving feedback from our participation to the 2017 edition of this conference.

On Thursday after ELCE, Bootlin engineers Maxime Ripard and Thomas Petazzoni participated to the Device Tree workshop, a day-long meeting to discuss the status and future of Device Tree support, especially in the context of the Linux kernel.

Device Tree Workshoup group photo 2017

Beyond participating to the event, Maxime and Thomas also presented briefly on two topics:

  • Maxime Ripard brought up the topic of handling foreign DT bindings (see slides). Currently, the Device Tree bindings documentation is stored in the Linux kernel source tree, in Documentation/devicetree/bindings/. However, in theory, bindings are not operating-system specific, and indeed the same bindings are used in other projects: U-Boot, Barebox, FreeBSD, Zephyr, and probably more. Maxime raised the question of what these projects should do when they create new bindings or extend existing ones? Should they contribute a patch to Linux? Should we have a separate repository for DT bindings? A bit of discussion followed, but without getting to a real conclusion.
  • Thomas Petazzoni presented on the topic of avoiding duplication in Device Tree representations (see slides). Recent Marvell Armada processors have a hardware layout where a block containing multiple IPs is duplicated several times in the SoC. In the currently available Armada 8040 there are two copies of the CP110 hardware block, and the Linux kernel carries a separate description for each. While very similar, those descriptions have subtle differences that make it non-trivial to de-duplicate. However, future SoCs will not have just 2 copies of the same hardware block, 4 copies or potentially more. In such a situation, duplicating the Device Tree description is no longer reasonable. Thomas presented a solution based on the C pre-processor, and commented on other options, such as a script to generate DTs, or improvements in the DT compiler itself. A discussion around those options followed, and while tooling improvements were considered as being the long-term solution, in the short term the solution based on the C pre-processor was acceptable upstream.

For Bootlin, participating to such events is very important, as it allows to expose to kernel developers the issue we are facing in some of our projects, and to get direct feedback from the developers on how to move forward on those topics. We definitely intend to continue participating in similar events in the future, for topics of interest to Bootlin.

MIPI I3C specification published, and new iteration of Linux I3C subsystem

MIPI I3C specification publishedBack in August 2017, Bootlin contributed to the Linux kernel a patch series adding support for the new MIPI I3C bus, a bus that aims at replacing busses like I2C and SPI, by offering better performance, lower power consumption, and new features like discovery, in-band interrupts and hot join.

At the time of our submission, the I3C specification was closed, but a few days ago, the MIPI Alliance announced that the I3C specification was now publicly available. This is of course very good news as it will allow a much easier and wider adoption of I3C, and it was a somewhat unexpected move since the MIPI Alliance had traditionally kept its specifications only for its members. Hopefully the I3C experience will encourage the MIPI Alliance to follow the same direction for existing or future protocols.

With this announcement from the MIPI Alliance, it was time for us to submit an updated version of our I3C support for the Linux kernel, which Bootlin engineer Boris Brezillon did on Thursday: [PATCH v2 0/7] Add the I3C subsystem. Compared to the previous version submitted in August, this new version has interesting improvements:

  • A generic infrastructure to support IBIs (in-band interrupts) was added
  • Helpers to support hot-join were added to the core I3C subsystem
  • The Cadence I3C controller driver was improved to support IBIs and hot-join
  • And of course, many of the comments received on the first iteration have been addressed

With the specification now public, we hope to receive useful comments and feedback from the Linux kernel community to improve, and hopefully in the near future, merge the support for the MIPI I3C bus.

Bootlin contributes Linux support for Microsemi MIPS SoC

VSC7513 Block Diagram
Microsemi VSC7513 Block Diagram
Earlier this month, Bootlin engineer Alexandre Belloni posted a patch series (in its second version) adding initial support for Microsemi Ocelot SoCs, the VSC7513 and VSC7514. These SoCs are used for switches, so the biggest part of the chip is a switch fabric, built around a MIPS core and a few basic peripherals. While Bootlin generally works on ARM platforms and has contributed support for numerous ARM processors in the Linux kernel, for this project we are contributing the support for a MIPS processor.

Alexandre’s initial patch series contains the basic support for the SoC:

All in all, this patch series only adds support to boot the platform up to a shell, with interrupts, pin-muxing, GPIOs and UARTs enabled. Additional features will be contributed later, especially support for the switch fabric in the form of a switchdev driver.

We are happy to be working on Microsemi platforms, and to bring the support for yet another hardware platform to the official Linux kernel.

Feedback from the Netdev 2.2 conference

The Netdev 2.2 conference took place in Seoul, South Korea. As we work on a diversity of networking topics at Bootlin as part of our Linux kernel contributions, Bootlin engineers Alexandre Belloni and Antoine Ténart went to Seoul to attend lots of interesting sessions and to meet with the Linux networking community. Below, they report on what they learned from this conference, by highlighting two talks they particularly liked.

Linux Networking Dietary Restrictions — slides — video

David S. Miller gave a keynote about reducing the size of core structures in the Linux kernel networking core. The idea behind his work is to use smaller structures which has many benefits in terms of performance as less cache misses will occur and less memory resources are needed. This is especially true in the networking core as small changes may have enormous impacts and improve performance a lot. Another argument from his maintainer hat perspective is the maintainability, where smaller structures usually means less complexity.

He presented five techniques he used to shrink the networking core data structures. The first one was to identify members of common base structures that are only used in sub-classes, as these members can easily be moved out and not impact all the data paths.

The second one makes use of what David calls “state compression”, aka. understanding the real width of the information stored in data structures and to pack flags together to save space. In his mind a boolean should take a single bit whereas in the kernel it requires way more space than that. While this is fine for many uses it makes sense to compress all these data in critical structures.

Then David S. Miller spoke about unused bits in pointers where in the kernel all pointers have 3 bits never used. He argued these bits are 3 boolean values that should be used to reduce core data structure sizes. This technique and the state compression one can be used by introducing helpers to safely access the data.

Another technique he used was to unionize members that aren’t used at the same time. This helps reducing even more the structure size by not having areas of memory never used during identified steps in the networking stack.

Finally he showed us the last technique he used, which was using lookup keys instead of pointers when the objects can be found cheaply based on their index. While this cannot be used for every object, it helped reducing some data structures.

While going through all these techniques he gave many examples to help understanding what can be saved and how it was effective. This was overall a great talk showing a critical aspect we do not always think of when writing drivers, which can lead to big performance improvements.

David S. Miller at Nedev 2.2

WireGuard: Next-generation Secure Kernel Network Tunnel — slides — video

Jason A. Donenfeld presented his new and shiny L3 network tunneling mechanism, in Linux. After two years of development this in-kernel formally proven cryptographic protocol is ready to be submitted upstream to get the first rounds of review.

The idea behind Wireguard is to provide, with a small code base, a simple interface to establish and maintain encrypted tunnels. Jason made a demo which was impressive by its simplicity when securely connecting two machines, while it can be a real pain when working with OpenVPN or IPsec. Under the hood this mechanism uses UDP packets on top of either IPv4 and IPv6 to transport encrypted packets using modern cryptographic principles. The authentication is similar to what SSH is using: static private/public key pairs. One particularly nice design choice is the fact that Wireguard is exposed as a stateless interface to the administrator whereas the protocol is stateful and timer based, which allow to put devices into sleep mode and not to care about it.

One of the difficulty to get Wireguard accepted upstream is its cryptographic needs, which do not match what can provide the kernel cryptographic framework. Jason knows this and plan to first send patches to rework the cryptographic framework so that his module nicely integrates with in-kernel APIs. First RFC patches for Wireguard should be sent at the end of 2017, or at the beginning of 2018.

We look forward to seeing Wireguard hit the mainline kernel, to allow everybody to establish secure tunnels in an easy way!

Jason A. Donenfeld at Netdev 2.2

Conclusion

Netdev 2.2 was again an excellent experience for us. It was an (almost) single track format, running alongside the workshops, allowing to not miss any session. The technical content let us dive deeply in the inner working of the network stack and stay up-to-date with the current developments.

Thanks for organizing this and for the impressive job, we had an amazing time!

Back from ELCE: award to Bootlin CEO Michael Opdenacker

The Embedded Linux Conference Europe 2017 took place at the end of October in Prague. We already posted about this event by sharing the slides and videos of Bootlin talks and later by sharing our selection of talks given by other speakers.

During the closing session of this conference, Bootlin CEO Michael Opdenacker has received from the hands of Tim Bird, on behalf of the ELCE committee, an award for its continuous participation to the Embedded Linux Conference Europe. Indeed, Michael has participated to all 11 editions of ELCE, with no interruption. He has been very active in promoting the event, especially through the video recording effort that Bootlin did in the early years of the conference, as well as through the numerous talks given by Bootlin.

Michael Opdenacker receives an award at the Embedded Linux Conference Europe

Bootlin is proud to see its continuous commitment to knowledge sharing and community participation be recognized by this award!

Linux 4.14 released, Bootlin contributions

Penguin from Mylène Josserand
Drawing from Mylène Josserand,
based on a picture from Samuel Blanc under CC-BY-SA
Linux 4.14, which is going to become the next Long Term Supported version, has been released a week ago by Linus Torvalds. As usual, LWN.net did an interesting coverage of this release cycle merge window, highlighting the most important changes: The first half of the 4.14 merge window and The rest of the 4.14 merge window.

According to Linux Kernel Patch statistics, Bootlin contributed 111 patches to this release, making it the 24th contributing company by number of commits: a somewhat lower than usual contribution level from our side. At least, Bootlin cannot be blamed for trying to push more code into 4.14 because of its Long Term Support nature! 🙂

The main highlights of our contributions are:

  • On the RTC subsystem, Alexandre Belloni made as usual a number of fixes and improvements to various drivers, especially the ds1307 driver.
  • On the NAND subsystem, Boris Brezillon did a number of small improvements in various areas.
  • On the support for Marvell platforms
    • Antoine Ténart improved the ppv2 network driver used by the Marvell Armada 7K/8K SoCs: support for 10G speed and TSO support are the main highlights. In order to support 10G speed, Antoine added a driver in drivers/phy/ to configure the common PHYs in the Armada 7K/8K SoCs.
    • Thomas Petazzoni also improved the ppv2 network driver by adding support for TX interrupts and per-CPU RX interrupts.
    • Grégory Clement contributed some patches to enable NAND support on Armada 7K/8K, as well as a number of fixes in different areas (GPIO fix, clock handling fixes, etc.)
    • Miquèl Raynal contributed a fix for the Armada 3700 SPI controller driver.
  • On the support for Allwinner platforms
    • Maxime Ripard contributed the support for a new board, the BananaPI M2-Magic. Maxime also contributed a few fixes to the Allwinner DRM driver, and a few other misc fixes (clock, MMC, RTC, etc.).
    • Quentin Schulz contributed the support for the power button functionality of the AXP221 (PMIC used in several Allwinner platforms)
  • On the support for Atmel platforms, Quentin Schulz improved the clock drivers for this platform to properly support the Audio PLL, which allowed to fix the Atmel audio drivers. He also fixed suspend/resume support in the Atmel MMC driver to support the deep sleep mode of the SAMA5D2 processor.

In addition to making direct contributions, Bootlin is also involved in the Linux kernel development by having a number of its engineers act as Linux kernel maintainers. As part of this effort, Bootlin engineers have reviewed, merged and sent pull requests for a large number of contributions from other developers:

  • Boris Brezillon, as the NAND subsystem maintainer and MTD subsystem co-maintainer, merged 68 patches from other developers.
  • Alexandre Belloni, as the RTC subsystem maintainer and Atmel ARM platform co-maintainer, merged 32 patches from other developers.
  • Grégory Clement, as the Marvell ARM platform co-maintainer, merged 29 patches from other developers.
  • Maxime Ripard, as the Allwinner ARM platform co-maintainer, merged 18 patches from other developers.

This flow of patches from kernel maintainers to other kernel maintainers is also nicely described for the 4.14 release by the Patch flow into the mainline for 4.14 LWN.net article.

The detailed list of our contributions:

Mender: How to integrate an OTA updater

Recently, our customer Senic asked us to integrate an Over-The-Air (OTA) mechanism in their embedded Linux system, and after some discussion, they ended up chosing Mender. This article will detail an example of Mender’s integration and how to use it.

What is Mender?

Mender is an open source remote updater for embedded devices. It is composed of a client installed on the embedded device, and a management server installed on a remote server. However, the server is not mandatory as Mender can be used standalone, with updates triggered directly on the embedded device.

Image taken from Mender’s website

In order to offer a fallback in case of failure, Mender uses the double partition layout: the device will have at least 2 rootfs partitions, one active and one inactive. Mender will deploy an update on the inactive partition, so that in case of an error during the update process, it will still have the active partition intact. If the update succeeds, it will switch to the updated partition: the active partition becomes inactive and the inactive one becomes the new active. As the kernel and the device tree are stored in the /boot folder of the root filesystem, it is possible to easily update an entire system. Note that Mender needs at least 4 partitions:

  • bootloader partition
  • data persistent partition
  • rootfs + kernel active partition
  • rootfs + kernel inactive partition

It is, of course, customizable if you need more partitions.

Two reference devices are supported: the BeagleBone Black and a virtual device. In our case, the board was a Nanopi-Neo, which is based on an Allwinner H3.

Mender provides a Yocto Project layer containing all the necessary classes and recipes to make it work. The most important thing to know is that it will produce an image ready to be written to an SD card to flash empty boards. It will also produce “artifacts” (files with .mender extension) that will be used to update an existing system.

Installation and setup

In this section, we will see how to setup the Mender client and server for your project. Most of the instructions are taken from the Mender documentation that we found well detailed and really pleasant to read. We’ll simply summarize the most important steps.

Server side

The Mender server will allow you to remotely update devices. The server can be installed in two modes:

  • demo mode: Used to test a demo server. It can be nice to test it if you just want to quickly deploy a Mender solution, for testing purpose only. It includes a demo layer that simplify and configure for you a default Mender server on localhost of your workstation.
  • production mode: Used for production. We will focus on this mode as we wanted to use Mender in a production context. This mode allows to customize the server configuration: IP address, certificates, etc. Because of that, some configuration will be necessary (which is not the case in the demo mode).

In order to install the Mender server, you should first install Docker CE and Docker Compose. Have a look at the corresponding Docker instructions.

Setup

  • Download the integration repository from Mender:
  • $ git clone https://github.com/mendersoftware/integration mender-server
    
  • Checkout 1.1.0 tag (latest version at the moment of the test)
  • $ cd mender-server
    $ git checkout 1.1.0 -b my-production-setup
    
  • Copy the template folder and update all the references to “template”
  • $ cp -a template production
    $ cd production
    $ sed -i -e 's#/template/#/production/#g' prod.yml
    
  • Download Docker images
  • $ ./run pull
    
  • Use the keygen script to create certificates for domain names (e.g. mender.foobar.com and s3.foobar.com)
  • $ CERT_API_CN=mender.foobar.com CERT_STORAGE_CN=s3.foobar.com ../keygen
    
  • Some persistent storage will be needed by Mender so create a few Docker volumes:
  • $ docker volume create --name=mender-artifacts
    $ docker volume create --name=mender-deployments-db
    $ docker volume create --name=mender-useradm-db
    $ docker volume create --name=mender-inventory-db
    $ docker volume create --name=mender-deviceadm-db
    $ docker volume create --name=mender-deviceauth-db
    

Final configuration

This final configuration will link the generated keys with the Mender server. All the modifications will be in the prod.yml file.

  • Locate the storage-proxy service in prod.yml and set it to your domain name. In our case s3.foobar.com under the networks.mender.aliases
  • Locate the minio service. Set MINIO_ACCESS_KEY to “mender-deployments” and the MINIO_SECRET_KEY to a generated password (with e.g.: $ apg -n1 -a0 -m32)
  • Locate the mender-deployments service. Set DEPLOYMENTS_AWS_AUTH_KEY and DEPLOYMENTS_AWS_AUTH_SECRET to respectively the value of MINIO_ACCESS_KEY and MINIO_SECRET_KEY. Set DEPLOYMENTS_AWS_URI to point to your domain such as https://s3.foobar.com:9000

Start the server

Make sure that the domain names you have defined (mender.foobar.com and s3.foobar.com) are accessible, potentially by adding them to /etc/hosts if you’re just testing.

  • Start the server
  • $ ./run up -d
    
  • If it is a new installation, request initial user login:
  • $ curl -X POST  -D - --cacert keys-generated/certs/api-gateway/cert.crt https://mender.foobar.com:443/api/management/v1/useradm/auth/login
    
  • Check that you can create a user and login to mender UI:
  •  $ firefox http://mender.foobar.com:443 

Client side – Yocto Project

Mender has a Yocto Project layer to easily interface with your own layer.
We will see how to customize your layer and image components (U-Boot, Linux kernel) to correctly configure it for Mender use.

In this section, we will assume that you have your own U-Boot and your own kernel repositories (and thus, recipes) and that you retrieved the correct branch of this layer.

Machine and distro configurations

  • Make sure that the kernel image and Device Tree files are installed in the root filesystem image
  • RDEPENDS_kernel-base += "kernel-image kernel-devicetree"
    
  • Update the distro to inherit the mender-full class and add systemd as the init manager (we only tested Mender’s integration with systemd)
  • # Enable systemd for Mender
    DISTRO_FEATURES_append = " systemd"
    VIRTUAL-RUNTIME_init_manager = "systemd"
    DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
    VIRTUAL-RUNTIME_initscripts = ""
    
    INHERIT += "mender-full"
    
  • By default, Mender assumes that your storage device is /dev/mmcblk0, that mmcblk0p1 is your boot partition (containing the bootloader), that mmcblk0p2 and mmcblk0p3 are your two root filesystem partitions, and that mmcblk0p5 is your data partition. If that’s the case for you, then everything is fine! However, if you need a different layout, you need to update your machine configuration. Mender’s client will retrieve which storage device to use by using the MENDER_STORAGE_DEVICE variable (which defaults to mmcblk0). The partitions themselves should be specified using MENDER_BOOT_PART, MENDER_ROOTFS_PART_A, MENDER_ROOTFS_PART_B and ROOTFS_DATA_PART. If you need to change the default storage or the partitions’ layout, edit in your machine configuration the different variables according to your need. Here is an example for /dev/sda:
  • MENDER_STORAGE_DEVICE = "/dev/sda"
    MENDER_STORAGE_DEVICE_BASE = "${MENDER_STORAGE_DEVICE}"
    MENDER_BOOT_PART = "${MENDER_STORAGE_DEVICE_BASE}1"
    MENDER_ROOTFS_PART_A = "${MENDER_STORAGE_DEVICE_BASE}2"
    MENDER_ROOTFS_PART_B = "${MENDER_STORAGE_DEVICE_BASE}3"
    MENDER_DATA_PART = "${MENDER_STORAGE_DEVICE_BASE}5"
    
  • Do not forget to update the artifact name in your local.conf, for example:

    MENDER_ARTIFACT_NAME = "release-1"
    

As described in Mender’s documentation, Mender will store the artifact name in its artifact image. It must be unique which is what we expect because an artifact will represent a release tag or a delivery. Note that if you forgot to update it and upload an artifact with the same name as an existing in the web UI, it will not be taken into account.

U-Boot configuration tuning

Some modifications in U-Boot are necessary to be able to perform the rollback (use a different partition after an unsuccessful update)

  • Mender needs BOOTCOUNT support in U-Boot. It creates a bootcount variable that will be incremented each time a reboot appears (or reset to 1 after a power-on reset). Mender will use this variable in its rollback mechanism.
    Make sure to enable it in your U-Boot configuration. This will most likely require a patch to your board .h configuration file, enabling:
  • #define CONFIG_BOOTCOUNT_LIMIT
    #define CONFIG_BOOTCOUNT_ENV
    
  • Remove environment variables that will be redefined by Mender. They are defined in Mender’s documentation.
  • Update your U-Boot recipe to inherit Mender’s one and make sure to provide U-Boot virtual package (using PROVIDES)
  • # Mender integration
    require recipes-bsp/u-boot/u-boot-mender.inc
    PROVIDES += "u-boot"
    RPROVIDES_${PN} += "u-boot"
    BOOTENV_SIZE = "0x20000"
    

    The BOOTENV_SIZE must be set the same content as the U-Boot CONFIG_ENV_SIZE variable. It will be used by the u-boot-fw-utils tool to retrieve the U-Boot environment variables.

    Mender is using u-boot-fw-utils so make sure that you have a recipe for it and that Mender include’s file is included. To do that, you can create a bbappend file on the default recipe or create your own recipe if you need a specific version. Have a look at Mender’s documentation example.

  • Tune your U-Boot environment to use Mender’s variables. Here are some examples of the modifications to be done. Set the root= kernel argument to use ${mender_kernel_root}, set the bootcmd to load the kernel image and Device Tree from ${mender_uboot_root} and to run mender_setup. Make sure that you are loading the Linux kernel image and Device Tree file from the root filesystem /boot directory.
    setenv bootargs 'console=${console} root=${mender_kernel_root} rootwait'
    setenv mmcboot 'load ${mender_uboot_root} ${fdt_addr_r} boot/my-device-tree.dtb; load ${mender_uboot_root} ${kernel_addr_r} boot/zImage; bootz ${kernel_addr_r} - ${fdt_addr_r}'
    setenv bootcmd 'run mender_setup; run mmcboot'
    

Mender’s client recipe

As stated in the introduction, Mender has a client, in the form of a userspace application, that will be used on the target. Mender’s layer has a Yocto recipe for it but it does not have our server certificates. To establish a connection between the client and the server, the certificates have to be installed in the image. For that, a bbappend recipe will be created. It will also allow to perform additional Mender configuration, such as defining the server URL.

  • Create a bbappend for the Mender recipe
  • FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
    SRC_URI_append = " file://server.crt"
    MENDER_SERVER_URL = "https://mender.senic.com"
    
  • Copy your server certificates in the bbappend recipe folder

Recompile an image and now, we should have everything we need to be able to update an image. Do not hesitate to run the integration checklist, it is really a convenient way to know if everything is correctly configured (or not).

If you want to be more robust and secure, you can sign your artifacts to be sure that they come from a trusted source. If you want this feature, have a look at this documentation.

Usage

Standalone mode

To update an artifact using the standalone mode (i.e. without server), here are the commands to use. You will need to update them according to your needs.

  • On your work station, create a simple HTTP server in your Yocto deploy folder:
  • $ python -m SimpleHTTPServer
  • On the target, start mender in standalone mode
  • $ mender -log-level info -rootfs http://192.168.42.251:8000/foobar.mender

    You can also use the mender command to start an update from a local .mender file, provided by a USB key or SD card.

  • Once finished, you will have to reboot the target manually
  • $ reboot

    After the first reboot, you will be on the the new active partition (if the previous one was /dev/mmcblk0p2, you should be on /dev/mmcblk0p3). Check the kernel version, artifact name or command line:

    $ uname -a
    $ cat /etc/mender/artifact_info
    $ cat /proc/cmdline
    

    If you are okay with this update, you will have to commit the modification otherwise the update will not be persistent and once you will reboot the board, Mender will rollback to the previous partition:

    $ mender -commit

Using Mender’s server UI

The Mender server UI provides a management interface to deploy updates on all your devices. It knows about all your devices, their current software version, and you can plan deployments on all or a subset of your devices. Here are the basic steps to trigger a deployment:

  • Login (or create an account) into the mender server UI: https://mender.foobar.com:443
  • Power-up your device
  • The first time, you will have to authorize the device. You will find it in your “dashboard” or in the “devices” section.
  • After authorizing it, it will retrieve device information such as current software version, MAC address, network interface, and so on
  • To update a partition, you will have to create a deployment using an artifact.
  • Upload the new artifact in the server UI using the “Artifacts” section
  • Deploy the new artifact using the “deployment” or the “devices” section. You will retrieve the status of the deployment in the “status” field. It will be in “installing”, “rebooting”, etc. The board will reboot and the partition should be updated.

Troubleshooting

Here are some issues we faced when we integrated Mender for our device. The Mender documentation also has a troubleshooting section so have a look at it if you are facing issues. Otherwise, the community seems to be active even if we did not need to interact with it as it worked like a charm when we tried it.

Update systemd’s service starting

By default, the Mender systemd service will start after the service “resolved” the domain name. On our target device, the network was available only via WiFi. We had to wait for the wlan0 interface to be up and configured to automatically connect a network before starting Mender’s service. Otherwise, it leads to an error due to the network being unreachable. To solve this issue which is specific to our platform, we set the systemd dependencies to “network-online.target” to be sure that a network is available:

-After=systemd-resolved.service
+After=network-online.target
+Wants=network-online.target

It now matches our use case because the Mender service will start only if the wlan0 connection is available and working.

Certificate expired

The certificates generated and used by Mender have a validity period. In case your board does not have a RTC set, Mender can fail with the error:

systemctl status mender
[...]
... level=error msg="authorize failed: transient error: authorization request failed: failed to execute authorization request:
Post https:///api/devices/v1/authentication/auth_requests: x509: certificate has expired or is not yet valid" module=state

To solve this issue, update the date on your board and make sure your RTC is correctly set.

Device deletion

While testing Mender’s server (version 1.0), we always used the same board and got into the issue that the board was already registered in the Server UI but had a different Device ID (which is used by Mender to identify devices). Because of that, the server was always rejecting the authentication. The next release of the Mender server offers the possibility to remove a device so we updated the Mender’s server to the last version.

Deployments not taken into account

Note that the Mender’s client is checking by default every 30 minutes if a deployment is available for this device. During testing, you may want to reduce this period, which you can in the Mender’s configuration file using its UpdatePollIntervalSeconds variable.

Conclusion

Mender is an OTA updater for Embedded devices. It has a great documentation in the form of tutorials which makes the integration easy. While testing it, the only issues we got were related to our custom platform or were already indicated in the documentation. Deploying it on a board was not difficult, only some U-Boot/kernel and Yocto Project modifications were necessary. All in all, Mender worked perfectly fine for our project!