New LXR website

I am pleased to announce that our https://elixir.bootlin.com website is back on line.

As some of you probably noticed, our service had been down for several months. When it was working, it was based on LXR 0.9.5. This version had nice improvements over the stable release (0.3.1), like better display of the kernel sources, but on the other hand, it also had ugly drawbacks. In particular, it stored data in an SQL database. This made the server consume more CPU resources, and made it very long to index a new kernel version (about 10 hours instead of just a few minutes). Disk space was also multiplied by 3 or 4, if I recall correctly. Anyway, the major problem was that that version didn’t scale: the service was getting slower each time a new version was added. Apparently, the bigger the database, the slower the server got.

Eventually, that 0.9.5 based server just died. I didn’t change anything before this happened and everything looked all right. I checked configuration files and packages, but there seemed to be no way to make it run again. The only solution left was a brand new install from scratch.

I first evaluated the LXRng, a new fork of the software on the http://lxr.linux.no website. This new version looks nice. In particular, though it’s still using a database, this new branch seems to withstand a greater number of kernel source versions, as http://lxr.linux.no answers pretty fast now and indexes a pretty long list of versions. However, I found its interface confusing and not as convenient as it was in the original branch, especially for identifier search. It could be because this new version is not mature enough yet, or just because I was too familiar with the original interface. The best is to try by yourself!

I also tried to make a new installation of the latest CVS sources of the main branch. However, this didn’t work as expected, as I wanted to run the new service on a recent distro with long term support (Ubuntu Gutsy Gibbon). Gutsy Gibbon only supports MySQL 5.0, but LXR-CVS proved to be only compatible with MySQL 4.0. I did apply some patches, but still got SQL query errors with MySQL 5.0.

I eventually decided to go back to the good old 0.3.1 stable version, and I don’t regret it:

• This version is extremely simple. You just need a web server with CGI scripts. No trouble with Apache2, no need to make modperl work. No need to install database software.
• This version is extremely fast too. It just takes a few minutes to add a new version, and serving identifier searches is very fast. Just try with a widely used function, like outb. With LXR 0.9.5, it could take up to 1 minute to display all the files in which the symbol was found.
• This version scales by design. Each supported source version has its own index files, and there is no central blob getting bigger and bigger. This simple design also makes it very easy to remove or to update source versions (like upgrading 2.6.28 sources to 2.6.28.1). With LXR 0.9.5, you had to make your own SQL queries to remove a version from the database!
• This version lacks a few features (like direct links to C include files, or like file descriptions), but hey, the main features are there: source navigation and identifier search. The only significant feature that is kind of missing in our site is freetext search. Version 0.3.1 only supported a proprietary searching tool, so we decided to rely on Google’s search instead. This is not perfect as we won’t have version-specific search, but freetext search is a secondary feature for us anyway. We wanted to have this service back on line, with at least its main features.
• Note that I had to make minor changes to make the website XHTML 1.0 Transitional compliant and pass the W3C Markup Validation Service checks I also fixed a bug in the diff markup script. Here is an archive of our install. Don’t hesitate to compare it with the original code and templates, and reuse our modified templates if you like them.

Thanks to this, the code hyperlinks in our kernel training slides work again at last! Every time we mention the name of a kernel source file or quote example code, you can click on the file name or on each function or structure type name, and you will be taken to the corresponding page on our LXR site!

Don’t forget that other valuable LXR websites exist for the Linux kernel. See our LXR websites list. Don’t hesitate to post a comment if you know other useful ones.

Author: Michael Opdenacker

Michael Opdenacker is the founder of Bootlin, and was its CEO until 2021. He is best known for all the free embedded Linux and kernel training materials that he created together with Thomas Petazzoni. He is always looking for ways to increase performance, reduce size and boot time, and to maximize Linux' world domination. More details... View all posts by Michael Opdenacker