[PATCH v3 00/12] selftests/nolibc: add minimal kernel config support - part1

Zhangjin Wu
2023-07-27T20:21:54Z

Hi, Willy

Most of the suggestions of v2 [1] have been applied in this v3 revision,
except the local menuconfig and mrproper targets, as explained in [2].

A fresh run with tinyconfig for ppc, ppc64 and ppc64le:

$ for arch in ppc ppc64 ppc64le; do \
mkdir -p $PWD/kernel-$arch;
time make defconfig run DEFCONFIG=tinyconfig ARCH=$arch O=$PWD/kernel-$arch RUN_OUT=$PWD/run.$arch.out;
done

rerun for ppc, ppc64 and ppc64le:

$ for arch in ppc ppc64 ppc64le; do \
make rerun ARCH=$arch O=$PWD/kernel-$arch RUN_OUT=$PWD/run.$arch.out;
done
Running /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/kernel-ppc/vmlinux on qemu-system-ppc
>> [ppc] Kernel command line: console=ttyS0 panic=-1
printk: console [ttyS0] enabled
Run /init as init process
Running test 'startup'
Running test 'syscall'
Running test 'stdlib'
Running test 'vfprintf'
Running test 'protection'
Leaving init with final status: 0
reboot: Power down
powered off, test finish
qemu-system-ppc: terminating on signal 15 from pid 190248 ()

165 test(s): 156 passed, 9 skipped, 0 failed => status: warning

See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.ppc.out
Running /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/kernel-ppc64/vmlinux on qemu-system-ppc64
Linux version 6.4.0+ (ubuntu@linux-lab) (powerpc64le-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #2 SMP Fri Jul 28 01:40:55 CST 2023
Kernel command line: console=hvc0 panic=-1
printk: console [hvc0] enabled
printk: console [hvc0] enabled
Run /init as init process
Running test 'startup'
Running test 'syscall'
Running test 'stdlib'
Running test 'vfprintf'
Running test 'protection'
Leaving init with final status: 0
reboot: Power down
powered off, test finish

165 test(s): 156 passed, 9 skipped, 0 failed => status: warning

See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.ppc64.out
Running /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/kernel-ppc64le/arch/powerpc/boot/zImage on qemu-system-ppc64le
Linux version 6.4.0+ (ubuntu@linux-lab) (powerpc64le-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #2 SMP Fri Jul 28 01:41:12 CST 2023
Kernel command line: console=hvc0 panic=-1
Run /init as init process
Running test 'startup'
Running test 'syscall'
Running test 'stdlib'
Running test 'vfprintf'
Running test 'protection'
Leaving init with final status: 0
reboot: Power down
powered off, test finish

165 test(s): 156 passed, 9 skipped, 0 failed => status: warning

See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.ppc64le.out

A fast report on existing test logs:

$ for arch in ppc ppc64 ppc64le; do \
make report ARCH=$arch RUN_OUT=$PWD/run.$arch.out | grep status; \
done
165 test(s): 156 passed, 9 skipped, 0 failed => status: warning
165 test(s): 156 passed, 9 skipped, 0 failed => status: warning
165 test(s): 156 passed, 9 skipped, 0 failed => status: warning

Changes from v2 --> v3:

* selftests/nolibc: allow report with existing test log
selftests/nolibc: fix up O= option support
selftests/nolibc: allow customize CROSS_COMPILE by architecture
selftests/nolibc: customize CROSS_COMPILE for 32/64-bit powerpc
selftests/nolibc: tinyconfig: add extra common options

No Change.

* selftests/nolibc: add macros to reduce duplicated changes

Remove REPORT_RUN_OUT and LOG_OUT.

* selftests/nolibc: string the core targets

Removed extconfig target from our v3 powerpc patchset [3], the
operations have been merged into the defconfig target.

Let kernel depends on $(KERNEL_CONFIG) instead of the removed
extconfig.

* selftests/nolibc: add menuconfig and mrproper for development

like the other local nolibc targets, still require local menuconfig
and mrproper targets for consistent usage with the same ARCH and no -C
/path/to/srctree

Merge them together to reduce duplicated entries.

* selftests/nolibc: allow quit qemu-system when poweroff fails

Enhance timeout logic with more expected strings print and detection
about the booting of bios, kernel, init and test.

Add a default 10 seconds of QEMU_TIMEOUT for every architecture to
detect all of the potential boog hang or failed poweroff.

* selftests/nolibc: customize QEMU_TIMEOUT for ppc64/ppc64le

Reduce QEMU_TIMEOUT from 60 seconds to a more normal 15 and 20
seconds for ppc64 and ppc64le respectively. the main time cost is
the slow bios used.

* selftests/nolibc: tinyconfig: add support for 32/64-bit powerpc

Rename the file names to shorter ones as suggestions from the powerpc
patchset.

* selftests/nolibc: speed up some targets with multiple jobs

New to speed up with -j<N> by default.

Best regards,
Zhangjin
---
[1]: https://lore.kernel.org/lkml/cover.1689759351.git.falcon@tinylab.org/
[2]: https://lore.kernel.org/lkml/20230727132418.117924-1-falcon@tinylab.org/
[3]: https://lore.kernel.org/lkml/8e9e5ac6283c6ec2ecf10a70ce55b219028497c1.1690468707.git.falcon@tinylab.org/

Zhangjin Wu (12):
selftests/nolibc: allow report with existing test log
selftests/nolibc: add macros to reduce duplicated changes
selftests/nolibc: fix up O= option support
selftests/nolibc: string the core targets
selftests/nolibc: allow customize CROSS_COMPILE by architecture
selftests/nolibc: customize CROSS_COMPILE for 32/64-bit powerpc
selftests/nolibc: add menuconfig and mrproper for development
selftests/nolibc: allow quit qemu-system when poweroff fails
selftests/nolibc: customize QEMU_TIMEOUT for ppc64/ppc64le
selftests/nolibc: tinyconfig: add extra common options
selftests/nolibc: tinyconfig: add support for 32/64-bit powerpc
selftests/nolibc: speed up some targets with multiple jobs

tools/testing/selftests/nolibc/Makefile | 102 ++++++++++++++----
.../selftests/nolibc/configs/common.config | 4 +
.../selftests/nolibc/configs/ppc.config | 3 +
.../selftests/nolibc/configs/ppc64.config | 3 +
.../selftests/nolibc/configs/ppc64le.config | 4 +
5 files changed, 98 insertions(+), 18 deletions(-)
create mode 100644 tools/testing/selftests/nolibc/configs/common.config
create mode 100644 tools/testing/selftests/nolibc/configs/ppc64.config
create mode 100644 tools/testing/selftests/nolibc/configs/ppc64le.config

--
2.25.1