Bootlin participated to the Embedded Linux Conference Europe last week in Prague. With 7 engineers attending, 4 talks, one BoF and a poster at the technical showcase, we had a strong presence to this major conference of the embedded Linux ecosystem. All of us had a great time at this event, attending interesting talks and meeting numerous open-source developers.
Bootlin team at the Embedded Linux Conference Europe 2017. Top, from left to right: Maxime Ripard, Grégory Clement, Boris Brezillon, Quentin Schulz. Bottom, from left to right: Miquèl Raynal, Thomas Petazzoni, Michael Opdenacker.
In this first blog post about ELCE, we want to share the slides and videos of the talks we have given during the conference.
SD/eMMC: New Speed Modes and Their Support in Linux – Gregory Clement
Since the introduction of the original “default”(DS) and “high speed”(HS) modes, the SD card standard has evolved by introducing new speed modes, such as SDR12, SDR25, SDR50, SDR104, etc. The same happened to the eMMC standard, with the introduction of new high speed modes named DDR52, HS200, HS400, etc. The Linux kernel has obviously evolved to support these new speed modes, both in the MMC core and through the addition of new drivers.
This talk will start by introducing the SD and eMMC standards and how they work at the hardware level, with a specific focus on the new speed modes. With this hardware background in place, we will then detail how these standards are supported by Linux, see what is still missing, and what we can expect to see in the future.
An Overview of the Linux Kernel Crypto Subsystem – Boris Brezillon
The Linux kernel has long provided cryptographic support for in-kernel users (like the network or storage stacks) and has been pushed to open these cryptographic capabilities to user-space along the way.
But what is exactly inside this subsystem, and how can it be used by kernel users? What is the official userspace interface exposing these features and what are non-upstream alternatives? When should we use a HW engine compared to a purely software based implementation? What’s inside a crypto engine driver and what precautions should be taken when developing one?
These are some of the questions we’ll answer throughout this talk, after having given a short introduction to cryptographic algorithms.
Buildroot is a popular and easy to use embedded Linux build system. Within minutes, it is capable of generating lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs.
Since our last “What’s new” talk at ELC 2014, three and half years have passed, and Buildroot has continued to evolve significantly.
After a short introduction about Buildroot, this talk will go through the numerous new features and improvements that have appeared over the last years, and show how they can be useful for developers, users and contributors.
Porting U-Boot and Linux on New ARM Boards: A Step-by-Step Guide – Quentin Schulz
May it be because of a lack of documentation or because we don’t know where to look or where to start, it is not always easy to get started with U-Boot or Linux, and know how to port them to a new ARM platform.
Based on experience porting modern versions of U-Boot and Linux on a custom Freescale/NXP i.MX6 platform, this talk will offer a step-by-step guide through the porting process. From board files to Device Trees, through Kconfig, device model, defconfigs, and tips and tricks, join this talk to discover how to get U-Boot and Linux up and running on your brand new ARM platform!
This “Birds of a Feather” session will start by a quick update on available resources and recent efforts to reduce the size of the Linux kernel and the filesystem it uses.
An ARM based system running the mainline kernel with about 3 MB of RAM will also be demonstrated.
If you are interested in the size topic, please join this BoF and share your experience, the resources you have found and your ideas for further size reduction techniques!
Back in April 2017, Bootlin engineer Antoine Ténart participated to NetDev 2.1, the most important conference discussing Linux networking support. After the conference, Antoine published a summary of it, reporting on the most interesting talks and topics that have been discussed.
Next week, NetDev 2.2 takes place in Seoul, South Korea, and this time around, two Bootlin engineers will be attending the event: Alexandre Belloni and Antoine Ténart. We are getting more and more projects with networking related topics, and therefore the wide range of talks proposed at NetDev 2.2 will definitely help grow our expertise in this field.
Do not hesitate to get in touch with Alexandre or Antoine if you are also attending this event!
Bootlin engineers Quentin Schulz, Maxime Ripard and Miquèl Raynal will also be attending the event, which means that 7 people from Bootlin will participate to ELCE!
In addition to the main ELCE conference, Thomas Petazzoni will participate to the Buildroot Developers Days, a 2-day hackaton organized on Saturday and Sunday prior to ELCE, and will participate to the Device Tree workshop organized on Thursday afternoon.
Once again, we’re really happy to participate to this conference, and looking forward to meeting again with a large number of Linux kernel and embedded Linux developers!
The Kernel Recipes conference has become over the last few years a very interesting conference, with an original single track format and a limited number of attendees, which fosters communication and networking. Held in Paris, France, it is obviously a conference of choice for Bootlin engineers to attend and speak at. We participated to multiple editions, Bootlin engineer Maxime Ripard gave a talk at the 2014 edition, while Thomas Petazzoni gave a talk at the 2013 edition.
In 2017, the organizers decided to complement the 3-day Kernel Recipes conference with a 1-day Embedded Recipes event, and Bootlin will participate by having two engineers attend those events and give talks:
The Linux Plumbers conference has established itself as a major conference in the Linux ecosystem, discussing numerous aspects of the low-level layers of the Linux software stack. Linux Plumbers is organized around a number of micro-conferences, plus a number of more regular talks.
Bootlin already participated to several previous editions of Linux Plumbers, and will again participate to this year’s edition that takes place in Los Angeles on September 13-15. Bootlin engineers Boris Brezillon, Alexandre Belloni, Grégory Clement and Thomas Petazzoni will attend the conference.
If you’re attending this conference, or are located in the Los Angeles area, and want to meet us, do not hesitate to drop us a line at info@bootlin.com. You can also follow Bootlin Twitter feed for updates during the conference.
At Bootlin, we regularly work on networking topics as part of our Linux kernel contributions and thus we decided to attend our very first Netdev conference this year in Montreal. With the recent evolution of the network subsystem and its drivers capabilities, the conference was a very good opportunity to stay up-to-date, thanks to lots of interesting sessions.
Eric Dumazet presenting “Busypolling next generation”
The speakers and the Netdev committee did an impressive job by offering such a great schedule and the recorded talks are already available on the Netdev Youtube channel. We particularly liked a few of those talks.
Andrew Lunn, Viven Didelot and Florian Fainelli presented DSA, the Distributed Switch Architecture, by giving an overview of what DSA is and by then presenting its design. They completed their talk by discussing the future of this subsystem.
DSA in one slide
The goal of the DSA subsystem is to support Ethernet switches connected to the CPU through an Ethernet controller. The distributed part comes from the possibility to have multiple switches connected together through dedicated ports. DSA was introduced nearly 10 years ago but was mostly quiet and only recently came back to life thanks to contributions made by the authors of this talk, its maintainers.
The main idea of DSA is to reuse the available internal representations and tools to describe and configure the switches. Ports are represented as Linux network interfaces to allow the userspace to configure them using common tools, the Linux bridging concept is used for interface bridging and the Linux bonding concept for port trunks. A switch handled by DSA is not seen as a special device with its own control interface but rather as an hardware accelerator for specific networking capabilities.
DSA has its own data plane where the switch ports are slave interfaces and the Ethernet controller connected to the SoC a master one. Tagging protocols are used to direct the frames to a specific port when coming from the SoC, as well as when received by the switch. For example, the RX path has an extra check after netif_receive_skb() so that if DSA is used, the frame can be tagged and reinjected into the network stack RX flow.
Finally, they talked about the relationship between DSA and Switchdev, and cross-chip configuration for interconnected switches. They also exposed the upcoming changes in DSA as well as long term goals.
As part of the network performances workshop, Jesper Dangaard Brouer presented memory bottlenecks in the allocators caused by specific network workloads, and how to deal with them. The SLAB/SLUB baseline performances are found to be too slow, particularly when using XDP. A way from a driver to solve this issue is to implement a custom page recycling mechanism and that’s what all high-speed drivers do. He then displayed some data to show why this mechanism is needed when targeting the 10G network budget.
Jesper is working on a generic solution called page pool and sent a first RFC at the end of 2016. As mentioned in the cover letter, it’s still not ready for inclusion and was only sent for early reviews. He also made a small overview of his implementation.
These two talks were given by Gilberto Bertin from Cloudflare and Martin Lau from Facebook. While they were not talking about device driver implementation or improvements in the network stack directly related to what we do at Bootlin, it was nice to see how XDP is used in production.
XDP, the eXpress Data Path, provides a programmable data path at the lowest point of the network stack by processing RX packets directly out of the drivers’ RX ring queues. It’s quite new and is an answer to lots of userspace based solutions such as DPDK. Gilberto andMartin showed excellent results, confirming the usefulness of XDP.
From a driver point of view, some changes are required to support it. RX hooks must be added as well as some API changes and the driver’s memory model often needs to be updated. So far, in v4.10, only a few drivers are supporting XDP.
David S. Miller, the maintainer of the Linux networking stack and drivers, did an interesting keynote about XDP and eBPF. The eXpress Data Path clearly was the hot topic of this Netdev 2.1 conference with lots of talks related to the concept and David did a good overview of what XDP is, its purposes, advantages and limitations. He also quickly covered eBPF, the extended Berkeley Packet Filters, which is used in XDP to filter packets.
This presentation was a comprehensive introduction to the concepts introduced by XDP and its different use cases.
Conclusion
Netdev 2.1 was an excellent experience for us. The conference was well organized, the single track format allowed us to see every session on the schedule, and meeting with attendees and speakers was easy. The content was highly technical and an excellent opportunity to stay up-to-date with the latest changes of the networking subsystem in the kernel. The conference hosted both talks about in-kernel topics and their use in userspace, which we think is a very good approach to not focus only on the kernel side but also to be aware of the users needs and their use cases.
Netdev 2.1 is the fourth edition of the technical conference on Linux networking. This conference is driven by the community and focus on both the kernel networking subsystems (device drivers, net stack, protocols) and their use in user-space.
This edition will be held in Montreal, Canada, April 6 to 8, and the schedule has been posted recently, featuring amongst other things a talk giving an overview and the current status display of the Distributed Switch Architecture (DSA) or a workshop about how to enable drivers to cope with heavy workloads, to improve performances.
At Bootlin, we regularly work on networking related topics, especially as part of our Linux kernel contribution for the support of Marvell or Annapurna Labs ARM SoCs. Therefore, we decided to attend our first Netdev conference to stay up-to-date with the network subsystem and network drivers capabilities, and to learn from the community latest developments.
Our engineer Antoine Ténart will be representing Bootlin at this event. We’re looking forward to being there!
Last month, five engineers from Bootlin participated to the Embedded Linux Conference in Portlan, Oregon. It was once again a great conference to learn new things about embedded Linux and the Linux kernel, and to meet developers from the open-source community.
Bootlin team at work at ELC 2017, with Maxime Ripard, Antoine Ténart, Mylène Josserand and Quentin Schulz
Of course, the slides from many other talks are progressively being uploaded, and the Linux Foundation published the video recordings in a record time: they are all already available on Youtube!
Below, each Bootlin engineer who attended the conference has selected one talk he/she has liked, and gives a quick summary of the talk, hopefully to encourage you watch the corresponding video recording.
Using SWupdate to Upgrade your system, Gabriel Huau
Gabriel Huau from Witekio did a great talk at ELC about SWUpdate, a tool created by Denx to update your system. The talk gives an overview of this tool, how it is working and how to use it. Updating your system is very important for embedded devices to fix some bugs/security fixes or add new features, but in an industrial context, it is sometimes difficult to perform an update: devices not easily accessible, large number of devices and variants, etc. A tool that can update the system automatically or even Over The Air (OTA) can be very useful. SWUpdate is one of them.
SWUpdate allows to update different parts of an embedded system such as the bootloader, the kernel, the device tree, the root file system and also the application data.
It handles different image types: UBI, MTD, Raw, Custom LUA, u-boot environment and even your custom one. It includes a notifier to be able to receive feedback about the updating process which can be useful in some cases. SWUPdate uses different local and OTA/remote interfaces such as USB, SD card, HTTP, etc. It is based on a simple update image format to indicate which images must be updated.
Many customizations can be done with this tool as it is provided with the classic menuconfig configuration tool. One great thing is that this tool is supported by Yocto Project and Buildroot so it can be easily tested.
Do not hesitate to have a look to his slides, the video of his talk or directly test SWUpdate!
GCC/Clang Optimizations for embedded Linux, Khem Raj
Khem Raj from Comcast is a frequent speaker at the Embedded Linux Conference, and one of his strong fields of expertise is C compilers, especially LLVM/Clang and Gcc. His talk at this conference can interest anyone developing code in the C language, to know about optimizations that the compilers can use to improve the performance or size of generated binaries. See the video and slides.
One noteworthy optimization is Clang’s -Oz (Gcc doesn’t have it), which goes even beyond -Os, by disabling loop vectorization. Note that Clang already performs better than Gcc in terms of code size (according to our own measurements). On the topic of bundle optimizations such as -O2 or -Os, Khem added that specific optimizations can be disabled in both compilers through the -fno- command line option preceding the name of a given optimization. The name of each optimization in a given bundle can be found through the -fverbose-asm command line option.
Another new optimization option is -Og, which is different from the traditional -g option. It still allows to produce code that can be debugged, but in a way that provides a reasonable level of runtime performance.
On the performance side, he also recalled the Feedback-Directed Optimizations (FDO), already covered in earlier Embedded Linux Conferences, which can be used to feed the compiler with profiler statistics about code branches. The compiler can use such information to optimize branches which are the more frequent at run-time.
Khem’s last advise was not to optimize too early, and first make sure you do your debugging and profiling work first, as heavily optimized code can be very difficult to debug. Therefore, optimizations are for well-proven code only.
Note that Khem also gave a similar talk in the IoT track for the conference, which was more focused on bare-metal code optimization code and portability: “Optimizing C for microcontrollers” (slides, video).
A Journey through Upstream Atomic KMS to Achieve DP Compliance, Manasi Navare
This talk was about the journey of a new comer in the mainline kernel community to fix the DisplayPort support in Intel i915 DRM driver. It first presented what happens from the moment we plug a cable in a monitor until we actually see an image, then where the driver is in the kernel: in the DRM subsystem, between the hardware (an Intel Integrated Graphics device) and the libdrm userspace library on which userspace applications such as the X server rely.
The bug to fix was that case when the driver would fail after updating to the requested resolution for a DP link. The other existing drivers usually fail before updating the resolution, so Manasi had to add a way to tell the userspace the DP link failed after updating the resolution. Such addition would be useless without applications using this new information, therefore she had to work with their developers to make the applications behave correctly when reading this important information.
With a working set of patches, she thought she had done most of the work with only the upstreaming left and didn’t know it would take her many versions to make it upstream. She wished to have sent a first version of a driver for review earlier to save time over the whole development plus upstreaming process. She also had to make sure the changes in the userspace applications will be ready when the driver will be upstreamed.
The talk was a good introduction on how DisplayPort works and an excellent example on why involving the community even in early stages of the development process may be a good idea to quicken the overall driver development process by avoiding complete rewriting of some code parts when upstreaming is under way.
Stephen did a great talk about one thing that is often overlooked, and really shouldn’t: Timekeeping. He started by explaining the various timekeeping mechanisms, both in hardware and how Linux use them. That meant covering the counters, timers, the tick, the jiffies, and the various POSIX clocks, and detailing the various frameworks using them. He also explained the various bugs that might be encountered when having a too naive counter implementation for example, or using the wrong POSIX clock from an application.
Karim did a very good introduction to Android Things. His talk was a great overview of what this new OS from Google targeting embedded devices is, and where it comes from. He started by showing the history of Android, and he explained what this system brought to the embedded market. He then switched to the birth of Android Things; a reboot of Google’s strategy for connected devices. He finally gave an in depth explanation of the internals of this new OS, by comparing Android Things and Android, with lots of examples and demos.
Android Things replaces Brillo / Weave, and unlike its predecessor is built reusing available tools and services. It’s in fact a lightweight version of Android, with many services removed and a few additions like the PIO API to drive GPIO, I2C, PWM or UART controllers. A few services were replaced as well, most notably the launcher. The result is a not so big, but not so small, system that can run on headless devices to control various sensors; with an Android API for application developers.
Linux.conf.au, which takes place every year in January in Australia or New Zealand, is a major event of the Linux community. Bootlin already participated to this event three years ago, and will participate again to this year’s edition, which will take place from January 16 to January 20 2017 in Hobart, Tasmania.
This time, Bootlin CTO Thomas Petazzoni will give a talk titled A tour of the ARM architecture and its Linux support, in which he will share with LCA attendees what is the ARM architecture, how its Linux support is working, what the numerous variants of ARM processors and boards mean, what is the Device Tree, the ARM specific bootloaders, and more.
If some of our readers located in Australia, New Zealand or neighboring countries plan on attending the conference, do not hesitate to drop us a mail so that we can meet during the event!
Last month, the entire Bootlin engineering team attended the Embedded Linux Conference Europe in Berlin. The slides and videos of the talks have been posted, including the ones from the seven talks given by Bootlin engineers:
Alexandre Belloni presented on ASoC: Supporting Audio on an Embedded Board, slides and video.
Boris Brezillon presented on Modernizing the NAND framework, the big picture, slides and video.
Boris Brezillon, together with Richard Weinberger from sigma star, presented on Running UBI/UBIFS on MLC NAND, slides and video.
Grégory Clement presented on Your newer ARM64 SoC Linux check list, slides and video.
Thomas Petazzoni presented on Anatomy of cross-compilation toolchains, slides and video.
Maxime Ripard presented on Supporting the camera interface on the C.H.I.P, slides and video.
Quentin Schulz and Antoine Ténart presented on Building a board farm: continuous integration and remote control, slides and video.
Since the introduction of the original “default”(DS) and “high speed”(HS) modes, the SD card standard has evolved by introducing new speed modes, such as SDR12, SDR25, SDR50, SDR104, etc. The same happened to the eMMC standard, with the introduction of new high speed modes named DDR52, HS200, HS400, etc. The Linux kernel has obviously evolved to support these new speed modes, both in the MMC core and through the addition of new drivers.
The Linux kernel has long provided cryptographic support for in-kernel users (like the network or storage stacks) and has been pushed to open these cryptographic capabilities to user-space along the way.
Buildroot is a popular and easy to use embedded Linux build system. Within minutes, it is capable of generating lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs.
