Introduction
The Linux Test Project is a project that develops and maintains a large test suite that helps validating the reliability, robustness and stability of the Linux kernel and related features. LTP has been mainly developed by companies such as IBM, Cisco, Fujitsu, SUSE, RedHat, with a focus on desktop distributions.
On the embedded side, both the openembedded-core Yocto layer and Buildroot have packages that allow to use LTP on embedded targets. However, for a recent project, we practically tried to run the full LTP test suite on an i.MX8 based platform running a Linux system built with Yocto. It turned out that LTP was apparently not very often tested on Busybox-based embedded systems, and we faced a number of issues. In addition to reporting various bugs/issues to the upstream LTP project, we also contributed a number of fixes and improvements:
- testcases: ar: Add TST_NEEDS_CMDS on “ar”
- testcases: file: Add TST_NEEDS_CMDS on readelf
- testcases: tar: Add TST_NEEDS_CMDS on gzip and bzip
- testcases: logrotate: Check if “file” command exists
- testcases: wc: Add “unrecognized option” as TCONF
- syscalls: ioctl03: Handle ENOENT error
- testcase: gethostid: Check if hostid is available
- testcase: lib: Create tst_getconf to replace “getconf”
- testcases: Replace “getconf” by “tst_getconf”
- tcp_cmds/ping/ping02: Make it compatible with Busybox
- network/traceroute01: make it compatible with Busybox
- net/host: update to new API
- net/ip_tests: Return TCONF if driver dummy not present
- net: netstat: clean and update to the new API
- fastopen: add require to sch_netem driver
- ipsec_lib: make tst_net_run “tst_check_driver” quiet
- containers/netns: update netns shell test to new API
- sctp/func_test: add check on the sctp driver
- quota_remount_test01: update to new API
- cmdlib: Update exists function to exit with TCONF
Our contributions received a very warm welcome in the LTP community, which turned out to be very open and responsive. We hope that these contributions will encourage others to use LTP, and hopefully to make sure it continues to work on embedded platforms.
Quick start guide
At the time of this writing, LTP has more than 3800 tests written by the community, including about 1000 network-related tests. The tests are grouped together in categories described by files in the runtest/ folder. Based on this, two scenarios of tests are defined: default and network which are described by two files in the scenario_groups/ folder. These two scenarios simply list the categories of tests that need to be executed.
Here are the contents of the default and network:
$ cat scenario_groups/default syscalls fs fs_perms_simple fsx dio io mm ipc sched math nptl pty containers fs_bind controllers filecaps cap_bounds fcntl-locktests connectors power_management_tests hugetlb commands hyperthreading can cpuhotplug net.ipv6_lib input cve crypto kernel_misc uevent
$ cat scenario_groups/network can net.features net.ipv6 net.ipv6_lib net.tcp_cmds net.multicast net.rpc net.nfs net.rpc_tests net.tirpc_tests net.sctp net_stress.appl net_stress.broken_ip net_stress.interface net_stress.ipsec_dccp net_stress.ipsec_icmp net_stress.ipsec_sctp net_stress.ipsec_tcp net_stress.ipsec_udp net_stress.multicast net_stress.route
Once you have LTP built and installed on your board thanks to the appropriate OpenEmbedded or Buildroot package, you can run these two scenarios of test with the following commands (-n specify the network one):
$ cd /opt/ltp $ ./runltp $ ./runltp -n
Then take a look at the content of the result and the output directories.
For more information on building or running LTP please read this readme.
LTP is good to run to identify the bugs in embedded linux systems
What about running LTP-DDT?
Have you guys tried running LTP- DDT?
No, we didn’t know about LTP-DDT. Is this the project at http://arago-project.org/git/projects/test-automation/ltp-ddt.git ? There is version on Github at https://github.com/rogerq/ltp-ddt which hasn’t seen any update since 2013. Unless I missed it, I didn’t find any real “home page” for this project. Was it contributed back to upstream LTP ?
Thomas, it was’nt – but there are discussions ongoing on how to.. http://arago-project.org/git/projects/test-automation/ltp-ddt.git (it will move to git.ti.com later this year) is a fork of LTP (synced every year to upstream) – so not too far off.. The github is one of our ex-developer’s copy..
Hey Nishanth, nice to see you here! It would probably be worth making LTP-DDT a bit more visible. Ideally, by bringing it upstream, or at the very least by having a proper homepage somewhere, taking offline the unmaintained Github fork that shows up as the first Google search result.