Running large numbers of testimage tasks in parallel

[Richard Purdie <richard.purdie@linuxfoundation.org>]

In order to try and attempt to reproduce the autobuilder qemuarm*
hangs, I wanted to share a quick guide on how to hack things to make
running multiple testimage tasks in parallel easy.

Firstly, I added:

BBCLASSEXTEND = "${@' '.join(['mcextend:'+str(x) for x in range(0,100)])}"

to core-image-sato.bb

I then deleted the test case files listed at the end of the email to
make the tests run faster. The hangs are generally in the boot process
before login so if we reach that point, it isn't a hang scenario. I
could have probably set TEST_SUITES instead.

I then ran "bitbake core-image-sato" to get a base image to run the
tests with. After that, I used the following script fragment:

cd tmp/deploy/images/qemuarm64
for i in `seq 1 100`
do
    ln -s core-image-sato{,-$i}-qemuarm64.rootfs.manifest
    ln -s core-image-sato{,-$i}-qemuarm64.rootfs.ext4.zst
    ln -s core-image-sato{,-$i}-qemuarm64.rootfs.testdata.json 
done

I could the run 100 testimages in parallel with:

bitbake `for ((i=1; i<=80; i++)); do echo -n "core-image-sato-$i "; done` -c testimage

(which will run BB_NUMBER_THREADS in parallel)

I'm sure there are neater ways to do it but this basically works.

I haven't had any luck reproducing hangs with qemuarm64 with this yet.
I did have to switch from using a DISPLAY over ssh to unsetting DISPLAY
and using qemu's local vnc as otherwise the display traffic over ssh
was limiting the CPU load I could trigger on my server.

Hope someone finds this useful!

Cheers,

Richard


	deleted:    meta-
selftest/lib/oeqa/runtime/cases/dnf_runtime.py
	deleted:    meta/lib/oeqa/runtime/cases/_qemutiny.py
	deleted:    meta/lib/oeqa/runtime/cases/apt.py
	deleted:    meta/lib/oeqa/runtime/cases/buildcpio.py
	deleted:    meta/lib/oeqa/runtime/cases/buildgalculator.py
	deleted:    meta/lib/oeqa/runtime/cases/buildlzip.py
	deleted:    meta/lib/oeqa/runtime/cases/connman.py
	deleted:    meta/lib/oeqa/runtime/cases/dnf.py
	deleted:    meta/lib/oeqa/runtime/cases/ethernet_ip_connman.py
	deleted:    meta/lib/oeqa/runtime/cases/gcc.py
	deleted:    meta/lib/oeqa/runtime/cases/gi.py
	deleted:    meta/lib/oeqa/runtime/cases/go.py
	deleted:    meta/lib/oeqa/runtime/cases/gstreamer.py
	deleted:    meta/lib/oeqa/runtime/cases/kernelmodule.py
	deleted:    meta/lib/oeqa/runtime/cases/ksample.py
	deleted:    meta/lib/oeqa/runtime/cases/ldd.py
	deleted:    meta/lib/oeqa/runtime/cases/logrotate.py
	deleted:    meta/lib/oeqa/runtime/cases/ltp.py
	deleted:    meta/lib/oeqa/runtime/cases/ltp_compliance.py
	deleted:    meta/lib/oeqa/runtime/cases/ltp_stress.py
	deleted:    meta/lib/oeqa/runtime/cases/maturin.py
	deleted:    meta/lib/oeqa/runtime/cases/multilib.py
	deleted:    meta/lib/oeqa/runtime/cases/oe_syslog.py
	deleted:    meta/lib/oeqa/runtime/cases/opkg.py
	deleted:    meta/lib/oeqa/runtime/cases/pam.py
	deleted:    meta/lib/oeqa/runtime/cases/ptest.py
	deleted:    meta/lib/oeqa/runtime/cases/python.py
	deleted:    meta/lib/oeqa/runtime/cases/rpm.py
	deleted:    meta/lib/oeqa/runtime/cases/rt.py
	deleted:    meta/lib/oeqa/runtime/cases/rtc.py
	deleted:    meta/lib/oeqa/runtime/cases/rust.py
	deleted:    meta/lib/oeqa/runtime/cases/scons.py
	deleted:    meta/lib/oeqa/runtime/cases/stap.py
	deleted:    meta/lib/oeqa/runtime/cases/storage.py
	deleted:    meta/lib/oeqa/runtime/cases/suspend.py
	deleted:    meta/lib/oeqa/runtime/cases/systemd.py
	deleted:    meta/lib/oeqa/runtime/cases/terminal.py
	deleted:    meta/lib/oeqa/runtime/cases/uki.py
	deleted:    meta/lib/oeqa/runtime/cases/usb_hid.py
	deleted:    meta/lib/oeqa/runtime/cases/weston.py
	deleted:    meta/lib/oeqa/runtime/cases/x32lib.py