[PATCH v4 00/14] kselftest harness and nolibc compatibility

[PATCH v4 00/14] kselftest harness and nolibc compatibility

[Thomas Weißschuh]

Nolibc is useful for selftests as the test programs can be very small,
and compiled with just a kernel crosscompiler, without userspace support.
Currently nolibc is only usable with kselftest.h, not the more
convenient to use kselftest_harness.h
This series provides this compatibility by removing the usage of problematic
libc features from the harness.

Based on nolibc/for-next.
The series is meant to be merged through the nolibc tree.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
Changes in v4:
- Drop patches for nolibc which where already applied
- Preserve signatures of test functions for tests making assumptions about them
  drop 'selftests: harness: Always provide "self" and "variant"'
  add 'selftests: harness: Add "variant" and "self" to test metadata'
  adapt 'selftests: harness: Stop using setjmp()/longjmp()'
- Validate test function signatures in harness selftest
- Link to v3: https://lore.kernel.org/r/20250411-nolibc-kselftest-harness-v3-0-4d9c0295893f@linutronix.de

Changes in v3:
- Send patches to correct kselftest harness maintainers
- Move harness selftest to dedicated directory
- Add harness selftest to MAINTAINERS
- Integrate harness selftest cleanup with the selftest framework
- Consistently use "kselftest harness" in commit messages
- Properly propagate kselftest harness failure
- Link to v2: https://lore.kernel.org/r/20250407-nolibc-kselftest-harness-v2-0-f8812f76e930@linutronix.de

Changes in v2:
- Rebase unto v6.15-rc1
- Rename internal nolibc symbols
- Handle edge case of waitpid(INT_MIN) == ESRCH
- Fix arm configurations for final testing patch
- Clean up global getopt.h variable declarations
- Add Acks from Willy
- Link to v1: https://lore.kernel.org/r/20250304-nolibc-kselftest-harness-v1-0-adca7cd231e2@linutronix.de

---
Thomas Weißschuh (14):
      selftests: harness: Add kselftest harness selftest
      selftests: harness: Use C89 comment style
      selftests: harness: Ignore unused variant argument warning
      selftests: harness: Mark functions without prototypes static
      selftests: harness: Remove inline qualifier for wrappers
      selftests: harness: Remove dependency on libatomic
      selftests: harness: Implement test timeouts through pidfd
      selftests: harness: Don't set setup_completed for fixtureless tests
      selftests: harness: Move teardown conditional into test metadata
      selftests: harness: Add teardown callback to test metadata
      selftests: harness: Add "variant" and "self" to test metadata
      selftests: harness: Stop using setjmp()/longjmp()
      selftests: harness: Guard includes on nolibc
      HACK: selftests/nolibc: demonstrate usage of the kselftest harness

 MAINTAINERS                                        |    1 +
 tools/testing/selftests/Makefile                   |    1 +
 tools/testing/selftests/kselftest_harness.h        |  175 +-
 .../testing/selftests/kselftest_harness/.gitignore |    2 +
 tools/testing/selftests/kselftest_harness/Makefile |    7 +
 .../selftests/kselftest_harness/harness-selftest.c |  138 ++
 .../kselftest_harness/harness-selftest.expected    |   64 +
 .../kselftest_harness/harness-selftest.sh          |   13 +
 tools/testing/selftests/nolibc/Makefile            |   15 +-
 tools/testing/selftests/nolibc/harness-selftest.c  |    1 +
 tools/testing/selftests/nolibc/nolibc-test.c       | 1715 +-------------------
 tools/testing/selftests/nolibc/run-tests.sh        |    2 +-
 12 files changed, 313 insertions(+), 1821 deletions(-)
---
base-commit: 2051d3b830c0889ae55e37e9e8ff0d43a4acd482
change-id: 20250130-nolibc-kselftest-harness-8b2c8cac43bf

Best regards,
-- 
Thomas Weißschuh <thomas.weissschuh@linutronix.de>

[PATCH v4 01/14] selftests: harness: Add kselftest harness selftest

[Thomas Weißschuh]

Add a selftest for the kselftest harness itself so any changes can be
validated.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 MAINTAINERS                                        |   1 +
 tools/testing/selftests/Makefile                   |   1 +
 .../testing/selftests/kselftest_harness/.gitignore |   2 +
 tools/testing/selftests/kselftest_harness/Makefile |   7 ++
 .../selftests/kselftest_harness/harness-selftest.c | 138 +++++++++++++++++++++
 .../kselftest_harness/harness-selftest.expected    |  64 ++++++++++
 .../kselftest_harness/harness-selftest.sh          |  13 ++
 7 files changed, 226 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 96b82704950184bd71623ff41fc4df31e4c7fe87..9d5278df33c8b63b3b08155991b789b3a998f80e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -21742,6 +21742,7 @@ F:	include/linux/seccomp.h
 F:	include/uapi/linux/seccomp.h
 F:	kernel/seccomp.c
 F:	tools/testing/selftests/kselftest_harness.h
+F:	tools/testing/selftests/kselftest_harness/
 F:	tools/testing/selftests/seccomp/*
 K:	\bsecure_computing
 K:	\bTIF_SECCOMP\b
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index c77c8c8e3d9bdd8047c9cb7722c3830447e504e5..27592909a5969da009d71be6c8330fe6779e7354 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -48,6 +48,7 @@ TARGETS += ipc
 TARGETS += ir
 TARGETS += kcmp
 TARGETS += kexec
+TARGETS += kselftest_harness
 TARGETS += kvm
 TARGETS += landlock
 TARGETS += lib
diff --git a/tools/testing/selftests/kselftest_harness/.gitignore b/tools/testing/selftests/kselftest_harness/.gitignore
new file mode 100644
index 0000000000000000000000000000000000000000..e4e476a333c912602161448bc61787732a6fa2e2
--- /dev/null
+++ b/tools/testing/selftests/kselftest_harness/.gitignore
@@ -0,0 +1,2 @@
+/harness-selftest
+/harness-selftest.seen
diff --git a/tools/testing/selftests/kselftest_harness/Makefile b/tools/testing/selftests/kselftest_harness/Makefile
new file mode 100644
index 0000000000000000000000000000000000000000..0617535a6ce424ff977e033b0a3a01c3117aefcf
--- /dev/null
+++ b/tools/testing/selftests/kselftest_harness/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+
+TEST_GEN_PROGS_EXTENDED := harness-selftest
+TEST_PROGS := harness-selftest.sh
+EXTRA_CLEAN := harness-selftest.seen
+
+include ../lib.mk
diff --git a/tools/testing/selftests/kselftest_harness/harness-selftest.c b/tools/testing/selftests/kselftest_harness/harness-selftest.c
new file mode 100644
index 0000000000000000000000000000000000000000..a0e3e89ae291f8d9f4f2ab9746a838fd1bfaf6c1
--- /dev/null
+++ b/tools/testing/selftests/kselftest_harness/harness-selftest.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <stdio.h>
+
+#ifndef NOLIBC
+#include <sys/resource.h>
+#include <sys/prctl.h>
+#endif
+
+/* Avoid any inconsistencies */
+#define TH_LOG_STREAM stdout
+
+#include "../kselftest_harness.h"
+
+static void test_helper(struct __test_metadata *_metadata)
+{
+	ASSERT_EQ(0, 0);
+}
+
+TEST(standalone_pass) {
+	TH_LOG("before");
+	ASSERT_EQ(0, 0);
+	EXPECT_EQ(0, 0);
+	test_helper(_metadata);
+	TH_LOG("after");
+}
+
+TEST(standalone_fail) {
+	TH_LOG("before");
+	EXPECT_EQ(0, 0);
+	EXPECT_EQ(0, 1);
+	ASSERT_EQ(0, 1);
+	TH_LOG("after");
+}
+
+TEST_SIGNAL(signal_pass, SIGUSR1) {
+	TH_LOG("before");
+	ASSERT_EQ(0, 0);
+	TH_LOG("after");
+	kill(getpid(), SIGUSR1);
+}
+
+TEST_SIGNAL(signal_fail, SIGUSR1) {
+	TH_LOG("before");
+	ASSERT_EQ(0, 1);
+	TH_LOG("after");
+	kill(getpid(), SIGUSR1);
+}
+
+FIXTURE(fixture) {
+	pid_t testpid;
+};
+
+FIXTURE_SETUP(fixture) {
+	TH_LOG("setup");
+	self->testpid = getpid();
+}
+
+FIXTURE_TEARDOWN(fixture) {
+	TH_LOG("teardown same-process=%d", self->testpid == getpid());
+}
+
+TEST_F(fixture, pass) {
+	TH_LOG("before");
+	ASSERT_EQ(0, 0);
+	test_helper(_metadata);
+	standalone_pass(_metadata);
+	TH_LOG("after");
+}
+
+TEST_F(fixture, fail) {
+	TH_LOG("before");
+	ASSERT_EQ(0, 1);
+	fixture_pass(_metadata, self, variant);
+	TH_LOG("after");
+}
+
+TEST_F_TIMEOUT(fixture, timeout, 1) {
+	TH_LOG("before");
+	sleep(2);
+	TH_LOG("after");
+}
+
+FIXTURE(fixture_parent) {
+	pid_t testpid;
+};
+
+FIXTURE_SETUP(fixture_parent) {
+	TH_LOG("setup");
+	self->testpid = getpid();
+}
+
+FIXTURE_TEARDOWN_PARENT(fixture_parent) {
+	TH_LOG("teardown same-process=%d", self->testpid == getpid());
+}
+
+TEST_F(fixture_parent, pass) {
+	TH_LOG("before");
+	ASSERT_EQ(0, 0);
+	TH_LOG("after");
+}
+
+FIXTURE(fixture_setup_failure) {
+	pid_t testpid;
+};
+
+FIXTURE_SETUP(fixture_setup_failure) {
+	TH_LOG("setup");
+	self->testpid = getpid();
+	ASSERT_EQ(0, 1);
+}
+
+FIXTURE_TEARDOWN(fixture_setup_failure) {
+	TH_LOG("teardown same-process=%d", self->testpid == getpid());
+}
+
+TEST_F(fixture_setup_failure, pass) {
+	TH_LOG("before");
+	ASSERT_EQ(0, 0);
+	TH_LOG("after");
+}
+
+int main(int argc, char **argv)
+{
+	/*
+	 * The harness uses abort() to signal assertion failures, which triggers coredumps.
+	 * This may be useful to debug real failures but not for this selftest, disable them.
+	 */
+	struct rlimit rlimit = {
+		.rlim_cur = 0,
+		.rlim_max = 0,
+	};
+
+	prctl(PR_SET_DUMPABLE, 0, 0, 0, 0);
+	setrlimit(RLIMIT_CORE, &rlimit);
+
+	return test_harness_run(argc, argv);
+}
diff --git a/tools/testing/selftests/kselftest_harness/harness-selftest.expected b/tools/testing/selftests/kselftest_harness/harness-selftest.expected
new file mode 100644
index 0000000000000000000000000000000000000000..61ba88d233f91d21b1e26a427c04092b8c37747d
--- /dev/null
+++ b/tools/testing/selftests/kselftest_harness/harness-selftest.expected
@@ -0,0 +1,64 @@
+TAP version 13
+1..9
+# Starting 9 tests from 4 test cases.
+#  RUN           global.standalone_pass ...
+# harness-selftest.c:21:standalone_pass:before
+# harness-selftest.c:25:standalone_pass:after
+#            OK  global.standalone_pass
+ok 1 global.standalone_pass
+#  RUN           global.standalone_fail ...
+# harness-selftest.c:29:standalone_fail:before
+# harness-selftest.c:31:standalone_fail:Expected 0 (0) == 1 (1)
+# harness-selftest.c:32:standalone_fail:Expected 0 (0) == 1 (1)
+# standalone_fail: Test terminated by assertion
+#          FAIL  global.standalone_fail
+not ok 2 global.standalone_fail
+#  RUN           global.signal_pass ...
+# harness-selftest.c:37:signal_pass:before
+# harness-selftest.c:39:signal_pass:after
+#            OK  global.signal_pass
+ok 3 global.signal_pass
+#  RUN           global.signal_fail ...
+# harness-selftest.c:44:signal_fail:before
+# harness-selftest.c:45:signal_fail:Expected 0 (0) == 1 (1)
+# signal_fail: Test terminated by assertion
+#          FAIL  global.signal_fail
+not ok 4 global.signal_fail
+#  RUN           fixture.pass ...
+# harness-selftest.c:55:pass:setup
+# harness-selftest.c:64:pass:before
+# harness-selftest.c:21:pass:before
+# harness-selftest.c:25:pass:after
+# harness-selftest.c:68:pass:after
+# harness-selftest.c:60:pass:teardown same-process=1
+#            OK  fixture.pass
+ok 5 fixture.pass
+#  RUN           fixture.fail ...
+# harness-selftest.c:55:fail:setup
+# harness-selftest.c:72:fail:before
+# harness-selftest.c:73:fail:Expected 0 (0) == 1 (1)
+# harness-selftest.c:60:fail:teardown same-process=1
+# fail: Test terminated by assertion
+#          FAIL  fixture.fail
+not ok 6 fixture.fail
+#  RUN           fixture.timeout ...
+# harness-selftest.c:55:timeout:setup
+# harness-selftest.c:79:timeout:before
+# timeout: Test terminated by timeout
+#          FAIL  fixture.timeout
+not ok 7 fixture.timeout
+#  RUN           fixture_parent.pass ...
+# harness-selftest.c:89:pass:setup
+# harness-selftest.c:98:pass:before
+# harness-selftest.c:100:pass:after
+# harness-selftest.c:94:pass:teardown same-process=0
+#            OK  fixture_parent.pass
+ok 8 fixture_parent.pass
+#  RUN           fixture_setup_failure.pass ...
+# harness-selftest.c:108:pass:setup
+# harness-selftest.c:110:pass:Expected 0 (0) == 1 (1)
+# pass: Test terminated by assertion
+#          FAIL  fixture_setup_failure.pass
+not ok 9 fixture_setup_failure.pass
+# FAILED: 4 / 9 tests passed.
+# Totals: pass:4 fail:5 xfail:0 xpass:0 skip:0 error:0
diff --git a/tools/testing/selftests/kselftest_harness/harness-selftest.sh b/tools/testing/selftests/kselftest_harness/harness-selftest.sh
new file mode 100755
index 0000000000000000000000000000000000000000..fe72d16370fe5bc16706289ff4e1ff44db180017
--- /dev/null
+++ b/tools/testing/selftests/kselftest_harness/harness-selftest.sh
@@ -0,0 +1,13 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Selftest for kselftest_harness.h
+#
+
+set -e
+
+DIR="$(dirname $(readlink -f "$0"))"
+
+"$DIR"/harness-selftest > harness-selftest.seen || true
+
+diff -u "$DIR"/harness-selftest.expected harness-selftest.seen

-- 
2.49.0

[PATCH v4 02/14] selftests: harness: Use C89 comment style

[Thomas Weißschuh]

All comments in this file use C89 comment style.
Except for this one. Change it to get one step closer to C89
compatibility.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 666c9fde76da9d25fe6e248a7f2143c113473fe1..bac4327775ea65dbe977e9b22ee548bedcbd33ff 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -983,7 +983,7 @@ static void __timeout_handler(int sig, siginfo_t *info, void *ucontext)
 	}
 
 	t->timed_out = true;
-	// signal process group
+	/* signal process group */
 	kill(-(t->pid), SIGKILL);
 }
 

-- 
2.49.0

[PATCH v4 03/14] selftests: harness: Ignore unused variant argument warning

[Thomas Weißschuh]

For tests without fixtures the variant argument is unused.
This is intentional, prevent to compiler from complaining.

Example warning:

    harness-selftest.c: In function 'wrapper_standalone_pass':
    ../kselftest_harness.h:181:52: error: unused parameter 'variant' [-Werror=unused-parameter]
      181 |                 struct __fixture_variant_metadata *variant) \
          |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
    ../kselftest_harness.h:156:25: note: in expansion of macro '__TEST_IMPL'
      156 | #define TEST(test_name) __TEST_IMPL(test_name, -1)
          |                         ^~~~~~~~~~~
    harness-selftest.c:15:1: note: in expansion of macro 'TEST'
       15 | TEST(standalone_pass) {
          | ^~~~

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index bac4327775ea65dbe977e9b22ee548bedcbd33ff..2b350ed60b2bf1cbede8e3a9b4ac5fe716900144 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -174,7 +174,7 @@
 	static void test_name(struct __test_metadata *_metadata); \
 	static inline void wrapper_##test_name( \
 		struct __test_metadata *_metadata, \
-		struct __fixture_variant_metadata *variant) \
+		struct __fixture_variant_metadata __attribute__((unused)) *variant) \
 	{ \
 		_metadata->setup_completed = true; \
 		if (setjmp(_metadata->env) == 0) \

-- 
2.49.0

[PATCH v4 04/14] selftests: harness: Mark functions without prototypes static

[Thomas Weißschuh]

With -Wmissing-prototypes the compiler will warn about non-static
functions which don't have a prototype defined.
As they are not used from a different compilation unit they don't need to
be defined globally.

Avoid the issue by marking the functions static.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 2b350ed60b2bf1cbede8e3a9b4ac5fe716900144..5822bc0b86a3c623fd34830fb8b541b27672a00b 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -258,7 +258,7 @@
  * A bare "return;" statement may be used to return early.
  */
 #define FIXTURE_SETUP(fixture_name) \
-	void fixture_name##_setup( \
+	static void fixture_name##_setup( \
 		struct __test_metadata __attribute__((unused)) *_metadata, \
 		FIXTURE_DATA(fixture_name) __attribute__((unused)) *self, \
 		const FIXTURE_VARIANT(fixture_name) \
@@ -307,7 +307,7 @@
 	__FIXTURE_TEARDOWN(fixture_name)
 
 #define __FIXTURE_TEARDOWN(fixture_name) \
-	void fixture_name##_teardown( \
+	static void fixture_name##_teardown( \
 		struct __test_metadata __attribute__((unused)) *_metadata, \
 		FIXTURE_DATA(fixture_name) __attribute__((unused)) *self, \
 		const FIXTURE_VARIANT(fixture_name) \
@@ -987,7 +987,7 @@ static void __timeout_handler(int sig, siginfo_t *info, void *ucontext)
 	kill(-(t->pid), SIGKILL);
 }
 
-void __wait_for_test(struct __test_metadata *t)
+static void __wait_for_test(struct __test_metadata *t)
 {
 	struct sigaction action = {
 		.sa_sigaction = __timeout_handler,
@@ -1205,9 +1205,9 @@ static bool test_enabled(int argc, char **argv,
 	return !has_positive;
 }
 
-void __run_test(struct __fixture_metadata *f,
-		struct __fixture_variant_metadata *variant,
-		struct __test_metadata *t)
+static void __run_test(struct __fixture_metadata *f,
+		       struct __fixture_variant_metadata *variant,
+		       struct __test_metadata *t)
 {
 	struct __test_xfail *xfail;
 	char test_name[1024];

-- 
2.49.0

[PATCH v4 05/14] selftests: harness: Remove inline qualifier for wrappers

[Thomas Weißschuh]

The pointers to the wrappers are stored in function pointers,
preventing them from actually being inlined.
Remove the inline qualifier, aligning these wrappers with the other
functions defined through macros.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 5822bc0b86a3c623fd34830fb8b541b27672a00b..222a4f51a8d704c41597e09a241ad887ef787139 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -172,7 +172,7 @@
 
 #define __TEST_IMPL(test_name, _signal) \
 	static void test_name(struct __test_metadata *_metadata); \
-	static inline void wrapper_##test_name( \
+	static void wrapper_##test_name( \
 		struct __test_metadata *_metadata, \
 		struct __fixture_variant_metadata __attribute__((unused)) *variant) \
 	{ \
@@ -401,7 +401,7 @@
 		struct __test_metadata *_metadata, \
 		FIXTURE_DATA(fixture_name) *self, \
 		const FIXTURE_VARIANT(fixture_name) *variant); \
-	static inline void wrapper_##fixture_name##_##test_name( \
+	static void wrapper_##fixture_name##_##test_name( \
 		struct __test_metadata *_metadata, \
 		struct __fixture_variant_metadata *variant) \
 	{ \

-- 
2.49.0

[PATCH v4 06/14] selftests: harness: Remove dependency on libatomic

[Thomas Weißschuh]

__sync_bool_compare_and_swap() is deprecated and requires libatomic on
GCC. Compiler toolchains don't necessarily have libatomic available, so
avoid this requirement by using atomics that don't need libatomic.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 222a4f51a8d704c41597e09a241ad887ef787139..7ec4f66d0e3d7f129f6c2a45ff58310dabe5d03f 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -439,12 +439,12 @@
 		} \
 		if (child == 0) { \
 			if (_metadata->setup_completed && !fixture_name##_teardown_parent && \
-					__sync_bool_compare_and_swap(teardown, false, true)) \
+					!__atomic_test_and_set(teardown, __ATOMIC_RELAXED)) \
 				fixture_name##_teardown(_metadata, self, variant->data); \
 			_exit(0); \
 		} \
 		if (_metadata->setup_completed && fixture_name##_teardown_parent && \
-				__sync_bool_compare_and_swap(teardown, false, true)) \
+				!__atomic_test_and_set(teardown, __ATOMIC_RELAXED)) \
 			fixture_name##_teardown(_metadata, self, variant->data); \
 		munmap(teardown, sizeof(*teardown)); \
 		if (self && fixture_name##_teardown_parent) \

-- 
2.49.0

[PATCH v4 07/14] selftests: harness: Implement test timeouts through pidfd

[Thomas Weißschuh]

Make the kselftest harness compatible with nolibc which does not implement
signals by replacing the signal logic with pidfds.
The code also becomes simpler.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 72 ++++++++++-------------------
 1 file changed, 25 insertions(+), 47 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 7ec4f66d0e3d7f129f6c2a45ff58310dabe5d03f..1e459619fe8657d7d213a7b16d7bcbc58e76e892 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -56,6 +56,8 @@
 #include <asm/types.h>
 #include <ctype.h>
 #include <errno.h>
+#include <linux/unistd.h>
+#include <poll.h>
 #include <stdbool.h>
 #include <stdint.h>
 #include <stdio.h>
@@ -914,7 +916,6 @@ struct __test_metadata {
 	int exit_code;
 	int trigger; /* extra handler after the evaluation */
 	int timeout;	/* seconds to wait for test timeout */
-	bool timed_out;	/* did this test timeout instead of exiting? */
 	bool aborted;	/* stopped test due to failed ASSERT */
 	bool setup_completed; /* did setup finish? */
 	jmp_buf env;	/* for exiting out of test early */
@@ -964,75 +965,52 @@ static inline void __test_check_assert(struct __test_metadata *t)
 		abort();
 }
 
-struct __test_metadata *__active_test;
-static void __timeout_handler(int sig, siginfo_t *info, void *ucontext)
-{
-	struct __test_metadata *t = __active_test;
-
-	/* Sanity check handler execution environment. */
-	if (!t) {
-		fprintf(TH_LOG_STREAM,
-			"# no active test in SIGALRM handler!?\n");
-		abort();
-	}
-	if (sig != SIGALRM || sig != info->si_signo) {
-		fprintf(TH_LOG_STREAM,
-			"# %s: SIGALRM handler caught signal %d!?\n",
-			t->name, sig != SIGALRM ? sig : info->si_signo);
-		abort();
-	}
-
-	t->timed_out = true;
-	/* signal process group */
-	kill(-(t->pid), SIGKILL);
-}
-
 static void __wait_for_test(struct __test_metadata *t)
 {
-	struct sigaction action = {
-		.sa_sigaction = __timeout_handler,
-		.sa_flags = SA_SIGINFO,
-	};
-	struct sigaction saved_action;
 	/*
 	 * Sets status so that WIFEXITED(status) returns true and
 	 * WEXITSTATUS(status) returns KSFT_FAIL.  This safe default value
 	 * should never be evaluated because of the waitpid(2) check and
-	 * SIGALRM handling.
+	 * timeout handling.
 	 */
 	int status = KSFT_FAIL << 8;
-	int child;
+	struct pollfd poll_child;
+	int ret, child, childfd;
+	bool timed_out = false;
 
-	if (sigaction(SIGALRM, &action, &saved_action)) {
+	childfd = syscall(__NR_pidfd_open, t->pid, 0);
+	if (childfd == -1) {
 		t->exit_code = KSFT_FAIL;
 		fprintf(TH_LOG_STREAM,
-			"# %s: unable to install SIGALRM handler\n",
+			"# %s: unable to open pidfd\n",
 			t->name);
 		return;
 	}
-	__active_test = t;
-	t->timed_out = false;
-	alarm(t->timeout);
-	child = waitpid(t->pid, &status, 0);
-	if (child == -1 && errno != EINTR) {
+
+	poll_child.fd = childfd;
+	poll_child.events = POLLIN;
+	ret = poll(&poll_child, 1, t->timeout * 1000);
+	if (ret == -1) {
 		t->exit_code = KSFT_FAIL;
 		fprintf(TH_LOG_STREAM,
-			"# %s: Failed to wait for PID %d (errno: %d)\n",
-			t->name, t->pid, errno);
+			"# %s: unable to wait on child pidfd\n",
+			t->name);
 		return;
+	} else if (ret == 0) {
+		timed_out = true;
+		/* signal process group */
+		kill(-(t->pid), SIGKILL);
 	}
-
-	alarm(0);
-	if (sigaction(SIGALRM, &saved_action, NULL)) {
+	child = waitpid(t->pid, &status, WNOHANG);
+	if (child == -1 && errno != EINTR) {
 		t->exit_code = KSFT_FAIL;
 		fprintf(TH_LOG_STREAM,
-			"# %s: unable to uninstall SIGALRM handler\n",
-			t->name);
+			"# %s: Failed to wait for PID %d (errno: %d)\n",
+			t->name, t->pid, errno);
 		return;
 	}
-	__active_test = NULL;
 
-	if (t->timed_out) {
+	if (timed_out) {
 		t->exit_code = KSFT_FAIL;
 		fprintf(TH_LOG_STREAM,
 			"# %s: Test terminated by timeout\n", t->name);

-- 
2.49.0

[PATCH v4 08/14] selftests: harness: Don't set setup_completed for fixtureless tests

[Thomas Weißschuh]

This field is unused and has no meaning for tests without fixtures.
Don't set it for them.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 1e459619fe8657d7d213a7b16d7bcbc58e76e892..905986debbfb0ce8c9659dbd52b6c67c6759cae7 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -178,7 +178,6 @@
 		struct __test_metadata *_metadata, \
 		struct __fixture_variant_metadata __attribute__((unused)) *variant) \
 	{ \
-		_metadata->setup_completed = true; \
 		if (setjmp(_metadata->env) == 0) \
 			test_name(_metadata); \
 		__test_check_assert(_metadata); \

-- 
2.49.0

[PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Thomas Weißschuh]

To get rid of setjmp()/longjmp(), the teardown logic needs to be usable
from __bail(). To access the atomic teardown conditional from there,
move it into the test metadata.
This also allows the removal of "setup_completed".

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 905986debbfb0ce8c9659dbd52b6c67c6759cae7..895821af3e5c5752065561d0a108210d79e9eeee 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -411,9 +411,9 @@
 		pid_t child = 1; \
 		int status = 0; \
 		/* Makes sure there is only one teardown, even when child forks again. */ \
-		bool *teardown = mmap(NULL, sizeof(*teardown), \
+		_metadata->no_teardown = mmap(NULL, sizeof(*_metadata->no_teardown), \
 			PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); \
-		*teardown = false; \
+		*_metadata->no_teardown = true; \
 		if (sizeof(*self) > 0) { \
 			if (fixture_name##_teardown_parent) { \
 				self = mmap(NULL, sizeof(*self), PROT_READ | PROT_WRITE, \
@@ -431,7 +431,7 @@
 				/* Let setup failure terminate early. */ \
 				if (_metadata->exit_code) \
 					_exit(0); \
-				_metadata->setup_completed = true; \
+				*_metadata->no_teardown = false; \
 				fixture_name##_##test_name(_metadata, self, variant->data); \
 			} else if (child < 0 || child != waitpid(child, &status, 0)) { \
 				ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \
@@ -439,15 +439,16 @@
 			} \
 		} \
 		if (child == 0) { \
-			if (_metadata->setup_completed && !fixture_name##_teardown_parent && \
-					!__atomic_test_and_set(teardown, __ATOMIC_RELAXED)) \
+			if (!fixture_name##_teardown_parent && \
+					!__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
 				fixture_name##_teardown(_metadata, self, variant->data); \
 			_exit(0); \
 		} \
-		if (_metadata->setup_completed && fixture_name##_teardown_parent && \
-				!__atomic_test_and_set(teardown, __ATOMIC_RELAXED)) \
+		if (fixture_name##_teardown_parent && \
+				!__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
 			fixture_name##_teardown(_metadata, self, variant->data); \
-		munmap(teardown, sizeof(*teardown)); \
+		munmap(_metadata->no_teardown, sizeof(*_metadata->no_teardown)); \
+		_metadata->no_teardown = NULL; \
 		if (self && fixture_name##_teardown_parent) \
 			munmap(self, sizeof(*self)); \
 		if (WIFEXITED(status)) { \
@@ -916,7 +917,7 @@ struct __test_metadata {
 	int trigger; /* extra handler after the evaluation */
 	int timeout;	/* seconds to wait for test timeout */
 	bool aborted;	/* stopped test due to failed ASSERT */
-	bool setup_completed; /* did setup finish? */
+	bool *no_teardown; /* fixture needs teardown */
 	jmp_buf env;	/* for exiting out of test early */
 	struct __test_results *results;
 	struct __test_metadata *prev, *next;
@@ -1195,7 +1196,7 @@ static void __run_test(struct __fixture_metadata *f,
 	t->exit_code = KSFT_PASS;
 	t->trigger = 0;
 	t->aborted = false;
-	t->setup_completed = false;
+	t->no_teardown = NULL;
 	memset(t->env, 0, sizeof(t->env));
 	memset(t->results->reason, 0, sizeof(t->results->reason));
 

-- 
2.49.0

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

Hi Thomas,

CC += Jason

On Mon, May 05, 2025 at 05:15:27PM +0200, Thomas Weißschuh wrote:
> To get rid of setjmp()/longjmp(), the teardown logic needs to be usable
> from __bail(). To access the atomic teardown conditional from there,
> move it into the test metadata.
> This also allows the removal of "setup_completed".
> 
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> Acked-by: Shuah Khan <skhan@linuxfoundation.org>

Certain hugepage tests in iommufd selftest (CONFIG_IOMMUFD_TEST)
start to fail since v6.16-rc1, though the test functions weren't
changed during last cycle:

------------------------------------------------------------------
#  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
# enforce_dirty: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
not ok 193 iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
#  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.set_dirty_tracking ...
# set_dirty_tracking: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.set_dirty_tracking
not ok 194 iommufd_dirty_tracking.domain_dirty128M_huge.set_dirty_tracking
#  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.device_dirty_capability ...
# device_dirty_capability: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.device_dirty_capability
not ok 195 iommufd_dirty_tracking.domain_dirty128M_huge.device_dirty_capability
#  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap ...
# get_dirty_bitmap: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap
not ok 196 iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap
#  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear ...
# get_dirty_bitmap_no_clear: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear
not ok 197 iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear
------------------------------------------------------------------

Git bisect points to this patch, and reverting it and its following
patches fixes these.

I haven't debugged it, hoping you might have a quick thought. Lemme
know if you need some details to figure out what's going on.

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Thomas Weißschuh]

Hi Nicolin,

On Mon, Jun 09, 2025 at 11:49:05PM -0700, Nicolin Chen wrote:
> CC += Jason
> 
> On Mon, May 05, 2025 at 05:15:27PM +0200, Thomas Weißschuh wrote:
> > To get rid of setjmp()/longjmp(), the teardown logic needs to be usable
> > from __bail(). To access the atomic teardown conditional from there,
> > move it into the test metadata.
> > This also allows the removal of "setup_completed".
> > 
> > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> > Acked-by: Shuah Khan <skhan@linuxfoundation.org>
> 
> Certain hugepage tests in iommufd selftest (CONFIG_IOMMUFD_TEST)
> start to fail since v6.16-rc1, though the test functions weren't
> changed during last cycle:

Thanks for the report.

> ------------------------------------------------------------------
> #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> # enforce_dirty: Test terminated unexpectedly by signal 11
> #          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
> not ok 193 iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
> #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.set_dirty_tracking ...
> # set_dirty_tracking: Test terminated unexpectedly by signal 11
> #          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.set_dirty_tracking
> not ok 194 iommufd_dirty_tracking.domain_dirty128M_huge.set_dirty_tracking
> #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.device_dirty_capability ...
> # device_dirty_capability: Test terminated unexpectedly by signal 11
> #          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.device_dirty_capability
> not ok 195 iommufd_dirty_tracking.domain_dirty128M_huge.device_dirty_capability
> #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap ...
> # get_dirty_bitmap: Test terminated unexpectedly by signal 11
> #          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap
> not ok 196 iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap
> #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear ...
> # get_dirty_bitmap_no_clear: Test terminated unexpectedly by signal 11
> #          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear
> not ok 197 iommufd_dirty_tracking.domain_dirty128M_huge.get_dirty_bitmap_no_clear
> ------------------------------------------------------------------
> 
> Git bisect points to this patch, and reverting it and its following
> patches fixes these.
> 
> I haven't debugged it, hoping you might have a quick thought. Lemme
> know if you need some details to figure out what's going on.

I can't reproduce this report.
On my development machine or a virtme-ng VM I get the following failure:

	# ./iommufd -r iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
	TAP version 13
	1..1
	# Starting 1 tests from 1 test cases.
	#  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
	iommufd: iommufd.c:2042: iommufd_dirty_tracking_setup: Assertion `vrc == self->buffer' failed.
	# enforce_dirty: Test terminated by assertion
	#          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
	not ok 1 iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
	# FAILED: 0 / 1 tests passed.
	# Totals: pass:0 fail:1 xfail:0 xpass:0 skip:0 error:0

Specifically the mmap() fails with ENOMEM.

When booting the VM with "hugepages=100" the test succeeds.
The same result happens when running all the subtests.

Could you give more specific reproduction steps?

On another note, the selftest should use the kselftest_harness' ASSERT_*()
macros instead of plain assert().


Thomas

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Tue, Jun 10, 2025 at 01:38:22PM +0200, Thomas Weißschuh wrote:
> Hi Nicolin,
> 
> On Mon, Jun 09, 2025 at 11:49:05PM -0700, Nicolin Chen wrote:
> > CC += Jason
> > 
> > On Mon, May 05, 2025 at 05:15:27PM +0200, Thomas Weißschuh wrote:
> > > To get rid of setjmp()/longjmp(), the teardown logic needs to be usable
> > > from __bail(). To access the atomic teardown conditional from there,
> > > move it into the test metadata.
> > > This also allows the removal of "setup_completed".
> > > 
> > > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> > > Acked-by: Shuah Khan <skhan@linuxfoundation.org>
> > 
> > Certain hugepage tests in iommufd selftest (CONFIG_IOMMUFD_TEST)
> > start to fail since v6.16-rc1, though the test functions weren't
> > changed during last cycle:
> 
> Thanks for the report.
> 
> > ------------------------------------------------------------------
> > #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> > # enforce_dirty: Test terminated unexpectedly by signal 11

Sig 11 is weird..

> I can't reproduce this report.
> On my development machine or a virtme-ng VM I get the following failure:
> 
> 	# ./iommufd -r iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
> 	TAP version 13
> 	1..1
> 	# Starting 1 tests from 1 test cases.
> 	#  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> 	iommufd: iommufd.c:2042: iommufd_dirty_tracking_setup: Assertion `vrc == self->buffer' failed.
> 	# enforce_dirty: Test terminated by assertion
> 	#          FAIL  iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
> 	not ok 1 iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty
> 	# FAILED: 0 / 1 tests passed.
> 	# Totals: pass:0 fail:1 xfail:0 xpass:0 skip:0 error:0
> 
> Specifically the mmap() fails with ENOMEM.
> 
> When booting the VM with "hugepages=100" the test succeeds.

Yes, that is required

> On another note, the selftest should use the kselftest_harness' ASSERT_*()
> macros instead of plain assert().

IIRC the kselftest stuff explodes if you try to use it's assert
functions within a fixture setup/teardown context.

I also wasn't able to reproduce this (x86 ubuntu 24 LTS OS) Maybe
it is ARM specific, I think Nicolin is running on ARM..

I did see the other warnings that Nicolin reported.

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Tue, Jun 10, 2025 at 09:09:02AM -0300, Jason Gunthorpe wrote:
> On Tue, Jun 10, 2025 at 01:38:22PM +0200, Thomas Weißschuh wrote:
> > > ------------------------------------------------------------------
> > > #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> > > # enforce_dirty: Test terminated unexpectedly by signal 11
> 
> Sig 11 is weird..

> > On another note, the selftest should use the kselftest_harness' ASSERT_*()
> > macros instead of plain assert().
> 
> IIRC the kselftest stuff explodes if you try to use it's assert
> functions within a fixture setup/teardown context.
> 
> I also wasn't able to reproduce this (x86 ubuntu 24 LTS OS) Maybe
> it is ARM specific, I think Nicolin is running on ARM..

Yes. And I was running with 64KB page size. I just quickly retried
with 4KB page size (matching x86), and all failed tests pass now.

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Tue, Jun 10, 2025 at 11:48:44AM -0700, Nicolin Chen wrote:
> On Tue, Jun 10, 2025 at 09:09:02AM -0300, Jason Gunthorpe wrote:
> > On Tue, Jun 10, 2025 at 01:38:22PM +0200, Thomas Weißschuh wrote:
> > > > ------------------------------------------------------------------
> > > > #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> > > > # enforce_dirty: Test terminated unexpectedly by signal 11
> > 
> > Sig 11 is weird..
> 
> > > On another note, the selftest should use the kselftest_harness' ASSERT_*()
> > > macros instead of plain assert().
> > 
> > IIRC the kselftest stuff explodes if you try to use it's assert
> > functions within a fixture setup/teardown context.
> > 
> > I also wasn't able to reproduce this (x86 ubuntu 24 LTS OS) Maybe
> > it is ARM specific, I think Nicolin is running on ARM..
> 
> Yes. And I was running with 64KB page size. I just quickly retried
> with 4KB page size (matching x86), and all failed tests pass now.

That's a weird thing to be sensitive too. Can you get a backtrace from
the crash, what function/line is crashing?

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Tue, Jun 10, 2025 at 08:46:57PM -0300, Jason Gunthorpe wrote:
> On Tue, Jun 10, 2025 at 11:48:44AM -0700, Nicolin Chen wrote:
> > On Tue, Jun 10, 2025 at 09:09:02AM -0300, Jason Gunthorpe wrote:
> > > On Tue, Jun 10, 2025 at 01:38:22PM +0200, Thomas Weißschuh wrote:
> > > > > ------------------------------------------------------------------
> > > > > #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> > > > > # enforce_dirty: Test terminated unexpectedly by signal 11
> > > 
> > > Sig 11 is weird..
> > 
> > > > On another note, the selftest should use the kselftest_harness' ASSERT_*()
> > > > macros instead of plain assert().
> > > 
> > > IIRC the kselftest stuff explodes if you try to use it's assert
> > > functions within a fixture setup/teardown context.
> > > 
> > > I also wasn't able to reproduce this (x86 ubuntu 24 LTS OS) Maybe
> > > it is ARM specific, I think Nicolin is running on ARM..
> > 
> > Yes. And I was running with 64KB page size. I just quickly retried
> > with 4KB page size (matching x86), and all failed tests pass now.
> 
> That's a weird thing to be sensitive too. Can you get a backtrace from
> the crash, what function/line is crashing?

I think I am getting what's going on. Here the harness code has a
parent process and a child process:

--------------------------------------------------------------
428-            /* _metadata and potentially self are shared with all forks. */ \
429:            child = fork(); \
430:            if (child == 0) { \
431-                    fixture_name##_setup(_metadata, self, variant->data); \
432-                    /* Let setup failure terminate early. */ \
433-                    if (_metadata->exit_code) \
434-                            _exit(0); \
435-                    *_metadata->no_teardown = false; \
436-                    fixture_name##_##test_name(_metadata, self, variant->data); \
437-                    _metadata->teardown_fn(false, _metadata, self, variant->data); \
438-                    _exit(0); \
439:            } else if (child < 0 || child != waitpid(child, &status, 0)) { \
440-                    ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \
441-                    _metadata->exit_code = KSFT_FAIL; \
442-            } \
443-            _metadata->teardown_fn(true, _metadata, self, variant->data); \
444-            munmap(_metadata->no_teardown, sizeof(*_metadata->no_teardown)); \
445-            _metadata->no_teardown = NULL; \
446-            if (self && fixture_name##_teardown_parent) \
447-                    munmap(self, sizeof(*self)); \
448-            if (WIFEXITED(status)) { \
449-                    if (WEXITSTATUS(status)) \
450-                            _metadata->exit_code = WEXITSTATUS(status); \
451-            } else if (WIFSIGNALED(status)) { \
452-                    /* Forward signal to __wait_for_test(). */ \
453-                    kill(getpid(), WTERMSIG(status)); \
454-            } \
....
456-    static void wrapper_##fixture_name##_##test_name##_teardown( \
457-            bool in_parent, struct __test_metadata *_metadata, \
458-            void *self, const void *variant) \
459-    { \
460-            if (fixture_name##_teardown_parent == in_parent && \
461-                            !__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
462-                    fixture_name##_teardown(_metadata, self, variant); \
463-    } \
--------------------------------------------------------------

I found there is a race between those two processes, resulting in
the teardown() not getting invoked: I added some ksft_print_msg()
calls in-between the lines to debug, those tests can pass mostly,
as teardown() got invoked.

I think the reason why those huge page cases fail is just because 
the huge version of setup() takes longer time..

I haven't figured out a proper fix yet, but something smells bad:
1) *no_teardown is set non-atomically, while both processes calls
   __atomic_test_and_set()
2) parent doesn't seem to wait for the setup() to complete..
3) when parent runs faster than the child that is still running
   setup(), the parent unmaps the no_teardown and set it to NULL,
   then UAF in the child, i.e. signal 11?

I think Thomas should have an idea with these info :)

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Thomas Weißschuh]

On Wed, Jun 11, 2025 at 12:05:25AM -0700, Nicolin Chen wrote:
> On Tue, Jun 10, 2025 at 08:46:57PM -0300, Jason Gunthorpe wrote:
> > On Tue, Jun 10, 2025 at 11:48:44AM -0700, Nicolin Chen wrote:
> > > On Tue, Jun 10, 2025 at 09:09:02AM -0300, Jason Gunthorpe wrote:
> > > > On Tue, Jun 10, 2025 at 01:38:22PM +0200, Thomas Weißschuh wrote:
> > > > > > ------------------------------------------------------------------
> > > > > > #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> > > > > > # enforce_dirty: Test terminated unexpectedly by signal 11
> > > > 
> > > > Sig 11 is weird..
> > > 
> > > > > On another note, the selftest should use the kselftest_harness' ASSERT_*()
> > > > > macros instead of plain assert().
> > > > 
> > > > IIRC the kselftest stuff explodes if you try to use it's assert
> > > > functions within a fixture setup/teardown context.
> > > > 
> > > > I also wasn't able to reproduce this (x86 ubuntu 24 LTS OS) Maybe
> > > > it is ARM specific, I think Nicolin is running on ARM..
> > > 
> > > Yes. And I was running with 64KB page size. I just quickly retried
> > > with 4KB page size (matching x86), and all failed tests pass now.
> > 
> > That's a weird thing to be sensitive too. Can you get a backtrace from
> > the crash, what function/line is crashing?
> 
> I think I am getting what's going on. Here the harness code has a
> parent process and a child process:
> 
> --------------------------------------------------------------
> 428-            /* _metadata and potentially self are shared with all forks. */ \
> 429:            child = fork(); \
> 430:            if (child == 0) { \
> 431-                    fixture_name##_setup(_metadata, self, variant->data); \
> 432-                    /* Let setup failure terminate early. */ \
> 433-                    if (_metadata->exit_code) \
> 434-                            _exit(0); \
> 435-                    *_metadata->no_teardown = false; \
> 436-                    fixture_name##_##test_name(_metadata, self, variant->data); \
> 437-                    _metadata->teardown_fn(false, _metadata, self, variant->data); \
> 438-                    _exit(0); \
> 439:            } else if (child < 0 || child != waitpid(child, &status, 0)) { \
> 440-                    ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \
> 441-                    _metadata->exit_code = KSFT_FAIL; \
> 442-            } \
> 443-            _metadata->teardown_fn(true, _metadata, self, variant->data); \
> 444-            munmap(_metadata->no_teardown, sizeof(*_metadata->no_teardown)); \
> 445-            _metadata->no_teardown = NULL; \
> 446-            if (self && fixture_name##_teardown_parent) \
> 447-                    munmap(self, sizeof(*self)); \
> 448-            if (WIFEXITED(status)) { \
> 449-                    if (WEXITSTATUS(status)) \
> 450-                            _metadata->exit_code = WEXITSTATUS(status); \
> 451-            } else if (WIFSIGNALED(status)) { \
> 452-                    /* Forward signal to __wait_for_test(). */ \
> 453-                    kill(getpid(), WTERMSIG(status)); \
> 454-            } \
> ....
> 456-    static void wrapper_##fixture_name##_##test_name##_teardown( \
> 457-            bool in_parent, struct __test_metadata *_metadata, \
> 458-            void *self, const void *variant) \
> 459-    { \
> 460-            if (fixture_name##_teardown_parent == in_parent && \
> 461-                            !__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
> 462-                    fixture_name##_teardown(_metadata, self, variant); \
> 463-    } \
> --------------------------------------------------------------
> 
> I found there is a race between those two processes, resulting in
> the teardown() not getting invoked: I added some ksft_print_msg()
> calls in-between the lines to debug, those tests can pass mostly,
> as teardown() got invoked.
> 
> I think the reason why those huge page cases fail is just because 
> the huge version of setup() takes longer time..

Can you try to recreate this issue with changes to
tools/testing/selftests/kselftest_harness/harness-selftest.c ?

> I haven't figured out a proper fix yet, but something smells bad:
> 1) *no_teardown is set non-atomically, while both processes calls
>    __atomic_test_and_set()

Does this make a difference?

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 2925e47db995..89fb37a21d9d 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -410,7 +410,7 @@
                /* Makes sure there is only one teardown, even when child forks again. */ \
                _metadata->no_teardown = mmap(NULL, sizeof(*_metadata->no_teardown), \
                        PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); \
-               *_metadata->no_teardown = true; \
+               __atomic_store_n(_metadata->no_teardown, true, __ATOMIC_SEQ_CST); \
                if (sizeof(*self) > 0) { \
                        if (fixture_name##_teardown_parent) { \
                                self = mmap(NULL, sizeof(*self), PROT_READ | PROT_WRITE, \
@@ -429,7 +429,7 @@
                        /* Let setup failure terminate early. */ \
                        if (_metadata->exit_code) \
                                _exit(0); \
-                       *_metadata->no_teardown = false; \
+                       __atomic_store_n(_metadata->no_teardown, false, __ATOMIC_SEQ_CST); \
                        fixture_name##_##test_name(_metadata, self, variant->data); \
                        _metadata->teardown_fn(false, _metadata, self, variant->data); \
                        _exit(0); \
@@ -455,7 +455,7 @@
                void *self, const void *variant) \
        { \
                if (fixture_name##_teardown_parent == in_parent && \
-                               !__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
+                               !__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_SEQ_CST)) \
                        fixture_name##_teardown(_metadata, self, variant); \
        } \
        static struct __test_metadata *_##fixture_name##_##test_name##_object; \

> 2) parent doesn't seem to wait for the setup() to complete..

setup() is called in the child (L431) right before the testcase itself is
called (L436). The parent waits for the child to exit (L439) before unmapping.

> 3) when parent runs faster than the child that is still running
>    setup(), the parent unmaps the no_teardown and set it to NULL,
>    then UAF in the child, i.e. signal 11?

That should never happen as the waitpid() will block until the child running
setup() and the testcase itself have exited.

Does the issue also happen when you only execute a single testcase?
$ ./iommufd -r iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty


Thomas

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Wed, Jun 11, 2025 at 10:04:35AM +0200, Thomas Weißschuh wrote:
> On Wed, Jun 11, 2025 at 12:05:25AM -0700, Nicolin Chen wrote:
> > On Tue, Jun 10, 2025 at 08:46:57PM -0300, Jason Gunthorpe wrote:
> > > On Tue, Jun 10, 2025 at 11:48:44AM -0700, Nicolin Chen wrote:
> > > > On Tue, Jun 10, 2025 at 09:09:02AM -0300, Jason Gunthorpe wrote:
> > > > > On Tue, Jun 10, 2025 at 01:38:22PM +0200, Thomas Weißschuh wrote:
> > > > > > > ------------------------------------------------------------------
> > > > > > > #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> > > > > > > # enforce_dirty: Test terminated unexpectedly by signal 11
> > > > > 
> > > > > Sig 11 is weird..
> > > > 
> > > > > > On another note, the selftest should use the kselftest_harness' ASSERT_*()
> > > > > > macros instead of plain assert().
> > > > > 
> > > > > IIRC the kselftest stuff explodes if you try to use it's assert
> > > > > functions within a fixture setup/teardown context.
> > > > > 
> > > > > I also wasn't able to reproduce this (x86 ubuntu 24 LTS OS) Maybe
> > > > > it is ARM specific, I think Nicolin is running on ARM..
> > > > 
> > > > Yes. And I was running with 64KB page size. I just quickly retried
> > > > with 4KB page size (matching x86), and all failed tests pass now.
> > > 
> > > That's a weird thing to be sensitive too. Can you get a backtrace from
> > > the crash, what function/line is crashing?
> > 
> > I think I am getting what's going on. Here the harness code has a
> > parent process and a child process:
> > 
> > --------------------------------------------------------------
> > 428-            /* _metadata and potentially self are shared with all forks. */ \
> > 429:            child = fork(); \
> > 430:            if (child == 0) { \
> > 431-                    fixture_name##_setup(_metadata, self, variant->data); \
> > 432-                    /* Let setup failure terminate early. */ \
> > 433-                    if (_metadata->exit_code) \
> > 434-                            _exit(0); \
> > 435-                    *_metadata->no_teardown = false; \
> > 436-                    fixture_name##_##test_name(_metadata, self, variant->data); \
> > 437-                    _metadata->teardown_fn(false, _metadata, self, variant->data); \
> > 438-                    _exit(0); \
> > 439:            } else if (child < 0 || child != waitpid(child, &status, 0)) { \
> > 440-                    ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \
> > 441-                    _metadata->exit_code = KSFT_FAIL; \
> > 442-            } \
> > 443-            _metadata->teardown_fn(true, _metadata, self, variant->data); \
> > 444-            munmap(_metadata->no_teardown, sizeof(*_metadata->no_teardown)); \
> > 445-            _metadata->no_teardown = NULL; \
> > 446-            if (self && fixture_name##_teardown_parent) \
> > 447-                    munmap(self, sizeof(*self)); \
> > 448-            if (WIFEXITED(status)) { \
> > 449-                    if (WEXITSTATUS(status)) \
> > 450-                            _metadata->exit_code = WEXITSTATUS(status); \
> > 451-            } else if (WIFSIGNALED(status)) { \
> > 452-                    /* Forward signal to __wait_for_test(). */ \
> > 453-                    kill(getpid(), WTERMSIG(status)); \
> > 454-            } \
> > ....
> > 456-    static void wrapper_##fixture_name##_##test_name##_teardown( \
> > 457-            bool in_parent, struct __test_metadata *_metadata, \
> > 458-            void *self, const void *variant) \
> > 459-    { \
> > 460-            if (fixture_name##_teardown_parent == in_parent && \
> > 461-                            !__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
> > 462-                    fixture_name##_teardown(_metadata, self, variant); \
> > 463-    } \
> > --------------------------------------------------------------
> > 
> > I found there is a race between those two processes, resulting in
> > the teardown() not getting invoked: I added some ksft_print_msg()
> > calls in-between the lines to debug, those tests can pass mostly,
> > as teardown() got invoked.
> > 
> > I think the reason why those huge page cases fail is just because 
> > the huge version of setup() takes longer time..
> 
> Can you try to recreate this issue with changes to
> tools/testing/selftests/kselftest_harness/harness-selftest.c ?

Hmm, I assume all 9 cases should pass? Mine only passes 4 on rc1,
without any change (perhaps we should start from here?):

TAP version 13
1..9
# Starting 9 tests from 4 test cases.
#  RUN           global.standalone_pass ...
# harness-selftest.c:19:standalone_pass:before
# harness-selftest.c:23:standalone_pass:after
#            OK  global.standalone_pass
ok 1 global.standalone_pass
#  RUN           global.standalone_fail ...
# harness-selftest.c:27:standalone_fail:before
# harness-selftest.c:29:standalone_fail:Expected 0 (0) == 1 (1)
# harness-selftest.c:30:standalone_fail:Expected 0 (0) == 1 (1)
# standalone_fail: Test terminated by assertion
#          FAIL  global.standalone_fail
not ok 2 global.standalone_fail
#  RUN           global.signal_pass ...
# harness-selftest.c:35:signal_pass:before
# harness-selftest.c:37:signal_pass:after
#            OK  global.signal_pass
ok 3 global.signal_pass
#  RUN           global.signal_fail ...
# harness-selftest.c:42:signal_fail:before
# harness-selftest.c:43:signal_fail:Expected 0 (0) == 1 (1)
# signal_fail: Test terminated by assertion
#          FAIL  global.signal_fail
not ok 4 global.signal_fail
#  RUN           fixture.pass ...
# harness-selftest.c:53:pass:setup
# harness-selftest.c:62:pass:before
# harness-selftest.c:19:pass:before
# harness-selftest.c:23:pass:after
# harness-selftest.c:66:pass:after
# harness-selftest.c:58:pass:teardown same-process=1
#            OK  fixture.pass
ok 5 fixture.pass
#  RUN           fixture.fail ...
# harness-selftest.c:53:fail:setup
# harness-selftest.c:70:fail:before
# harness-selftest.c:71:fail:Expected 0 (0) == 1 (1)
# harness-selftest.c:58:fail:teardown same-process=1
# fail: Test terminated by assertion
#          FAIL  fixture.fail
not ok 6 fixture.fail
#  RUN           fixture.timeout ...
# harness-selftest.c:53:timeout:setup
# harness-selftest.c:77:timeout:before
# timeout: Test terminated by timeout
#          FAIL  fixture.timeout
not ok 7 fixture.timeout
#  RUN           fixture_parent.pass ...
# harness-selftest.c:87:pass:setup
# harness-selftest.c:96:pass:before
# harness-selftest.c:98:pass:after
# harness-selftest.c:92:pass:teardown same-process=0
#            OK  fixture_parent.pass
ok 8 fixture_parent.pass
#  RUN           fixture_setup_failure.pass ...
# harness-selftest.c:106:pass:setup
# harness-selftest.c:108:pass:Expected 0 (0) == 1 (1)
# pass: Test terminated by assertion
#          FAIL  fixture_setup_failure.pass
not ok 9 fixture_setup_failure.pass
# FAILED: 4 / 9 tests passed.
# Totals: pass:4 fail:5 xfail:0 xpass:0 skip:0 error:0

> > I haven't figured out a proper fix yet, but something smells bad:
> > 1) *no_teardown is set non-atomically, while both processes calls
> >    __atomic_test_and_set()
> 
> Does this make a difference?
> 
> diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
> index 2925e47db995..89fb37a21d9d 100644
> --- a/tools/testing/selftests/kselftest_harness.h
> +++ b/tools/testing/selftests/kselftest_harness.h
> @@ -410,7 +410,7 @@
>                 /* Makes sure there is only one teardown, even when child forks again. */ \
>                 _metadata->no_teardown = mmap(NULL, sizeof(*_metadata->no_teardown), \
>                         PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); \
> -               *_metadata->no_teardown = true; \
> +               __atomic_store_n(_metadata->no_teardown, true, __ATOMIC_SEQ_CST); \
>                 if (sizeof(*self) > 0) { \
>                         if (fixture_name##_teardown_parent) { \
>                                 self = mmap(NULL, sizeof(*self), PROT_READ | PROT_WRITE, \
> @@ -429,7 +429,7 @@
>                         /* Let setup failure terminate early. */ \
>                         if (_metadata->exit_code) \
>                                 _exit(0); \
> -                       *_metadata->no_teardown = false; \
> +                       __atomic_store_n(_metadata->no_teardown, false, __ATOMIC_SEQ_CST); \
>                         fixture_name##_##test_name(_metadata, self, variant->data); \
>                         _metadata->teardown_fn(false, _metadata, self, variant->data); \
>                         _exit(0); \
> @@ -455,7 +455,7 @@
>                 void *self, const void *variant) \
>         { \
>                 if (fixture_name##_teardown_parent == in_parent && \
> -                               !__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
> +                               !__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_SEQ_CST)) \
>                         fixture_name##_teardown(_metadata, self, variant); \
>         } \
>         static struct __test_metadata *_##fixture_name##_##test_name##_object; \

Unfortunately, no :(

> 
> > 2) parent doesn't seem to wait for the setup() to complete..
> 
> setup() is called in the child (L431) right before the testcase itself is
> called (L436). The parent waits for the child to exit (L439) before unmapping.
> 
> > 3) when parent runs faster than the child that is still running
> >    setup(), the parent unmaps the no_teardown and set it to NULL,
> >    then UAF in the child, i.e. signal 11?
> 
> That should never happen as the waitpid() will block until the child running
> setup() and the testcase itself have exited.

Ah, maybe I was wrong about these narratives. But the results show
that iommufd_dirty_tracking_teardown() was not called in the failed
cases:

// I added a huge.c file to run only 4 cases on one variant
// And I added two fprintf to its FIXTURE_SETUP/TEARDOWN().

TAP version 13
1..4
# Starting 4 tests from 1 test cases.
#  RUN           iommufd_dirty_tracking.domain_dirty64M_huge.set_dirty_tracking ...
---------iommufd_dirty_tracking_setup
---------iommufd_dirty_tracking_teardown
#            OK  iommufd_dirty_tracking.domain_dirty64M_huge.set_dirty_tracking
ok 1 iommufd_dirty_tracking.domain_dirty64M_huge.set_dirty_tracking
#  RUN           iommufd_dirty_tracking.domain_dirty64M_huge.device_dirty_capability ...
---------iommufd_dirty_tracking_setup
# device_dirty_capability: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty64M_huge.device_dirty_capability
not ok 2 iommufd_dirty_tracking.domain_dirty64M_huge.device_dirty_capability
#  RUN           iommufd_dirty_tracking.domain_dirty64M_huge.get_dirty_bitmap ...
---------iommufd_dirty_tracking_setup
# get_dirty_bitmap: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty64M_huge.get_dirty_bitmap
not ok 3 iommufd_dirty_tracking.domain_dirty64M_huge.get_dirty_bitmap
#  RUN           iommufd_dirty_tracking.domain_dirty64M_huge.get_dirty_bitmap_no_clear ...
---------iommufd_dirty_tracking_setup
# get_dirty_bitmap_no_clear: Test terminated unexpectedly by signal 11
#          FAIL  iommufd_dirty_tracking.domain_dirty64M_huge.get_dirty_bitmap_no_clear
not ok 4 iommufd_dirty_tracking.domain_dirty64M_huge.get_dirty_bitmap_no_clear
# FAILED: 1 / 4 tests passed.

> 
> Does the issue also happen when you only execute a single testcase?
> $ ./iommufd -r iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty

$ sudo tools/testing/selftests/iommu/iommufd -r iommufd_dirty_tracking.domain_dirty128M.enforce_dirty
TAP version 13
1..1
# Starting 1 tests from 1 test cases.
#  RUN           iommufd_dirty_tracking.domain_dirty128M.enforce_dirty ...
#            OK  iommufd_dirty_tracking.domain_dirty128M.enforce_dirty
ok 1 iommufd_dirty_tracking.domain_dirty128M.enforce_dirty
# PASSED: 1 / 1 tests passed.
# Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0

This one passes. Looks like the first hugepage case would pass but
the following ones would fail if running them sequentially..

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Wed, Jun 11, 2025 at 10:19:56AM -0700, Nicolin Chen wrote:
> On Wed, Jun 11, 2025 at 10:04:35AM +0200, Thomas Weißschuh wrote:
> > On Wed, Jun 11, 2025 at 12:05:25AM -0700, Nicolin Chen wrote:
> > > 2) parent doesn't seem to wait for the setup() to complete..
> > 
> > setup() is called in the child (L431) right before the testcase itself is
> > called (L436). The parent waits for the child to exit (L439) before unmapping.
> > 
> > > 3) when parent runs faster than the child that is still running
> > >    setup(), the parent unmaps the no_teardown and set it to NULL,
> > >    then UAF in the child, i.e. signal 11?
> > 
> > That should never happen as the waitpid() will block until the child running
> > setup() and the testcase itself have exited.
> 
> Ah, maybe I was wrong about these narratives. But the results show
> that iommufd_dirty_tracking_teardown() was not called in the failed
> cases:

Here is a new finding...

As you replied that I was wrong about the race between the parent
and the child processes, the parent does wait for the completion
of the child. But the child exited with status=139 i.e. signal 11
due to UAF, which however is resulted from the iommufd test code:

FIXTURE_SETUP(iommufd_dirty_tracking)
{
	....
	vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
	^
	|
        after this line, the _metadata->no_teardown is set to NULL.

So, the child process accessing this NULL pointer crashed with the
signal 11..

And I did a further experiment by turning "bool *no_teardown" to a
"bool no_teardown". Then, the mmap() in iommufd_dirty_tracking will
set _metadata->teardown_fn function pointer to NULL..

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Wed, Jun 11, 2025 at 02:48:16PM -0700, Nicolin Chen wrote:
> On Wed, Jun 11, 2025 at 10:19:56AM -0700, Nicolin Chen wrote:
> > On Wed, Jun 11, 2025 at 10:04:35AM +0200, Thomas Weißschuh wrote:
> > > On Wed, Jun 11, 2025 at 12:05:25AM -0700, Nicolin Chen wrote:
> > > > 2) parent doesn't seem to wait for the setup() to complete..
> > > 
> > > setup() is called in the child (L431) right before the testcase itself is
> > > called (L436). The parent waits for the child to exit (L439) before unmapping.
> > > 
> > > > 3) when parent runs faster than the child that is still running
> > > >    setup(), the parent unmaps the no_teardown and set it to NULL,
> > > >    then UAF in the child, i.e. signal 11?
> > > 
> > > That should never happen as the waitpid() will block until the child running
> > > setup() and the testcase itself have exited.
> > 
> > Ah, maybe I was wrong about these narratives. But the results show
> > that iommufd_dirty_tracking_teardown() was not called in the failed
> > cases:
> 
> Here is a new finding...
> 
> As you replied that I was wrong about the race between the parent
> and the child processes, the parent does wait for the completion
> of the child. But the child exited with status=139 i.e. signal 11
> due to UAF, which however is resulted from the iommufd test code:
> 
> FIXTURE_SETUP(iommufd_dirty_tracking)
> {
> 	....
> 	vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
> 	^
> 	|
>         after this line, the _metadata->no_teardown is set to NULL.
> 
> So, the child process accessing this NULL pointer crashed with the
> signal 11..
> 
> And I did a further experiment by turning "bool *no_teardown" to a
> "bool no_teardown". Then, the mmap() in iommufd_dirty_tracking will
> set _metadata->teardown_fn function pointer to NULL..

So, the test case sets an alignment with HUGEPAGE_SIZE=512MB while
allocating buffer_size=64MB:
	rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
	vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
this gives the self->buffer a location that is 512MB aligned, but
only mmap part of one 512MB huge page.

On the other hand, _metadata->no_teardown was mmap() outside the
range of the [self->buffer, self->buffer + 64MB), but within the
range of [self->buffer, self->buffer + 512MB).

E.g.
   _metadata->no_teardown = 0xfffbfc610000 // inside range2 below
   buffer=[0xfffbe0000000, fffbe4000000) // range1
   buffer=[0xfffbe0000000, fffc00000000) // range2

Then ,the "vrc = mmap(..." overwrites the _metadata->no_teardown
location to NULL..

The following change can fix, though it feels odd that the buffer
has to be preserved with the entire huge page:
---------------------------------------------------------------
@@ -2024,3 +2027,4 @@ FIXTURE_SETUP(iommufd_dirty_tracking)

-       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
+       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE,
+                           __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE));
        if (rc || !self->buffer) {
---------------------------------------------------------------

Any thought?

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Wed, Jun 11, 2025 at 04:43:00PM -0700, Nicolin Chen wrote:
> So, the test case sets an alignment with HUGEPAGE_SIZE=512MB while
> allocating buffer_size=64MB:
> 	rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> 	vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
> this gives the self->buffer a location that is 512MB aligned, but
> only mmap part of one 512MB huge page.
> 
> On the other hand, _metadata->no_teardown was mmap() outside the
> range of the [self->buffer, self->buffer + 64MB), but within the
> range of [self->buffer, self->buffer + 512MB).
> 
> E.g.
>    _metadata->no_teardown = 0xfffbfc610000 // inside range2 below
>    buffer=[0xfffbe0000000, fffbe4000000) // range1
>    buffer=[0xfffbe0000000, fffc00000000) // range2
> 
> Then ,the "vrc = mmap(..." overwrites the _metadata->no_teardown
> location to NULL..
> 
> The following change can fix, though it feels odd that the buffer
> has to be preserved with the entire huge page:
> ---------------------------------------------------------------
> @@ -2024,3 +2027,4 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
> 
> -       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> +       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE,
> +                           __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE));
>         if (rc || !self->buffer) {
> ---------------------------------------------------------------
> 
> Any thought?

This seems like something, variant->buffer_size should not
be less than HUGEPAGE_SIZE I guess that is possible on 64K ARM64

But I still don't quite get it..

        rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);

Should allocate buffer_size

        mmap_flags = MAP_SHARED | MAP_ANONYMOUS | MAP_FIXED;
                mmap_flags |= MAP_HUGETLB | MAP_POPULATE;
        vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
                   mmap_flags, -1, 0);

Should fail if buffer_size is not a multiple of HUGEPAGE_SIZE? 

It certainly shouldn't mmap past the provided buffer_size!!!

Are you seeing the above mmap succeed and also map beyond buffer -> buffer + buffer_size?

I think that would be a kernel bug in MAP_HUGETLB!

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Wed, Jun 11, 2025 at 08:51:17PM -0300, Jason Gunthorpe wrote:
> On Wed, Jun 11, 2025 at 04:43:00PM -0700, Nicolin Chen wrote:
> > So, the test case sets an alignment with HUGEPAGE_SIZE=512MB while
> > allocating buffer_size=64MB:
> > 	rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> > 	vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
> > this gives the self->buffer a location that is 512MB aligned, but
> > only mmap part of one 512MB huge page.
> > 
> > On the other hand, _metadata->no_teardown was mmap() outside the
> > range of the [self->buffer, self->buffer + 64MB), but within the
> > range of [self->buffer, self->buffer + 512MB).
> > 
> > E.g.
> >    _metadata->no_teardown = 0xfffbfc610000 // inside range2 below
> >    buffer=[0xfffbe0000000, fffbe4000000) // range1
> >    buffer=[0xfffbe0000000, fffc00000000) // range2
> > 
> > Then ,the "vrc = mmap(..." overwrites the _metadata->no_teardown
> > location to NULL..
> > 
> > The following change can fix, though it feels odd that the buffer
> > has to be preserved with the entire huge page:
> > ---------------------------------------------------------------
> > @@ -2024,3 +2027,4 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
> > 
> > -       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> > +       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE,
> > +                           __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE));
> >         if (rc || !self->buffer) {
> > ---------------------------------------------------------------
> > 
> > Any thought?
> 
> This seems like something, variant->buffer_size should not
> be less than HUGEPAGE_SIZE I guess that is possible on 64K ARM64
> 
> But I still don't quite get it..
> 
>         rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> 
> Should allocate buffer_size
> 
>         mmap_flags = MAP_SHARED | MAP_ANONYMOUS | MAP_FIXED;
>                 mmap_flags |= MAP_HUGETLB | MAP_POPULATE;
>         vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE,
>                    mmap_flags, -1, 0);
> 
> Should fail if buffer_size is not a multiple of HUGEPAGE_SIZE? 

Yea, I think you are right. But..

> It certainly shouldn't mmap past the provided buffer_size!!!
> 
> Are you seeing the above mmap succeed and also map beyond buffer -> buffer + buffer_size?
> 
> I think that would be a kernel bug in MAP_HUGETLB!

..I did some bpftrace:

ksys_mmap_pgoff() addr=ffff80000000, len=4000000
    hugetlb_file_setup(): size=0x20000000
        hugetlb_reserve_pages() from=0, to=1
        hugetlb_reserve_pages() returned: ret=1
    hugetlb_file_setup() returned: size=0x20000000 ret=-281471746619776
    vm_mmap_pgoff() addr=ffff80000000, len=20000000
        do_mmap() addr=ffff80000000, len=20000000
            hugetlb_reserve_pages() from=0, to=1
            hugetlb_reserve_pages() returned: ret=1
        do_mmap() returned: addr=0xffff80000000 ret=ffff80000000, pop=20000000
    vm_mmap_pgoff() returned: addr=0xffff80000000 ret=ffff80000000
ksys_mmap_pgoff() returned: addr=0xffff80000000 ret=ffff80000000

We can see the 64MB was rounded up to 512MB by ksys_mmap_pgoff()
when being passed in to hugetlb_file_setup() at:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/mmap.c?h=v6.16-rc1#n594
"		len = ALIGN(len, huge_page_size(hs));  "

By looking at the comments here..:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/hugetlbfs/inode.c#n1521
"
/*
 * Note that size should be aligned to proper hugepage size in caller side,
 * otherwise hugetlb_reserve_pages reserves one less hugepages than intended.
 */
struct file *hugetlb_file_setup(const char *name, size_t size,
"

..I guess this function was supposed to fail the not-a-multiple
case as you remarked? But it certainly can't do that, when that
size passed in is already hugepage-aligned..

It feels like a kernel bug as you suspect :-/


And I just found one more weird thing...

In iommufd.c selftest code, we have:
"static __attribute__((constructor)) void setup_sizes(void)"
where it does another pair of posix_memalign/mmap, although this
one doesn't flag MAP_HUGETLB and shouldn't impact what is coming
to the next...

If I keep this code, the first hugepage test case can pass (64MB
buffer_size; 512MB THP), but all the following cases will fail,
as I reported here:
https://lore.kernel.org/all/aEm6tuzy7WK12sMh@nvidia.com/

If I remove this code, the hugepage test case will fail from the
first case with signal 11. But this time, it is not because the
mmap() overwrites the _metadata->no_teardown, it's because mmap()
call itself crashed...

And, in either a failed case (crashed) or a passed case, the top
kernel function ksys_mmap_pgoff() returned successfully, which
means it seemingly crashed inside the libc?

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Wed, Jun 11, 2025 at 11:59:00PM -0700, Nicolin Chen wrote:

> We can see the 64MB was rounded up to 512MB by ksys_mmap_pgoff()
> when being passed in to hugetlb_file_setup() at:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/mmap.c?h=v6.16-rc1#n594
> "		len = ALIGN(len, huge_page_size(hs));  "
> 
> By looking at the comments here..:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/hugetlbfs/inode.c#n1521
> "
> /*
>  * Note that size should be aligned to proper hugepage size in caller side,
>  * otherwise hugetlb_reserve_pages reserves one less hugepages than intended.
>  */
> struct file *hugetlb_file_setup(const char *name, size_t size,
> "
> 
> ..I guess this function was supposed to fail the not-a-multiple
> case as you remarked? But it certainly can't do that, when that
> size passed in is already hugepage-aligned..
> 
> It feels like a kernel bug as you suspect :-/

Certainly is
 
> And I just found one more weird thing...
> 
> In iommufd.c selftest code, we have:
> "static __attribute__((constructor)) void setup_sizes(void)"
> where it does another pair of posix_memalign/mmap, although this
> one doesn't flag MAP_HUGETLB and shouldn't impact what is coming
> to the next...

This could all just be more weirdness from the above, it doesn't
really make alot of sense.

I think change things so the MAP_HUGETLB test all skip if
HUGEPAGE_SIZE < buffer_size and move on..

Can't run those tests on ARM64 64k which is unfortunate.. I thought
there were patches to give that config a 2M huge page size option
based on the new contiguous page support though? Maybe it was only THPS..

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Thomas Weißschuh]

On Thu, Jun 12, 2025 at 10:58:02AM -0300, Jason Gunthorpe wrote:
> On Wed, Jun 11, 2025 at 11:59:00PM -0700, Nicolin Chen wrote:
> 
> > We can see the 64MB was rounded up to 512MB by ksys_mmap_pgoff()
> > when being passed in to hugetlb_file_setup() at:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/mmap.c?h=v6.16-rc1#n594
> > "		len = ALIGN(len, huge_page_size(hs));  "
> > 
> > By looking at the comments here..:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/hugetlbfs/inode.c#n1521
> > "
> > /*
> >  * Note that size should be aligned to proper hugepage size in caller side,
> >  * otherwise hugetlb_reserve_pages reserves one less hugepages than intended.
> >  */
> > struct file *hugetlb_file_setup(const char *name, size_t size,
> > "
> > 
> > ..I guess this function was supposed to fail the not-a-multiple
> > case as you remarked? But it certainly can't do that, when that
> > size passed in is already hugepage-aligned..
> > 
> > It feels like a kernel bug as you suspect :-/
> 
> Certainly is
>  
> > And I just found one more weird thing...
> > 
> > In iommufd.c selftest code, we have:
> > "static __attribute__((constructor)) void setup_sizes(void)"
> > where it does another pair of posix_memalign/mmap, although this
> > one doesn't flag MAP_HUGETLB and shouldn't impact what is coming
> > to the next...
> 
> This could all just be more weirdness from the above, it doesn't
> really make alot of sense.
> 
> I think change things so the MAP_HUGETLB test all skip if
> HUGEPAGE_SIZE < buffer_size and move on..
> 
> Can't run those tests on ARM64 64k which is unfortunate.. I thought
> there were patches to give that config a 2M huge page size option
> based on the new contiguous page support though? Maybe it was only THPS..

If the assumption is that this is most likely a kernel bug,
shouldn't it be fixed properly rather than worked around?
After all the job of a selftest is to detect bugs to be fixed.

But I wasn't able to follow all of your discussions,
so I may be missing something.

If the test is broken on ARM64 64k in general then I am also wondering how
it didn't fail before my change to the selftest harness.


Thomas

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Thu, Jun 12, 2025 at 04:27:41PM +0200, Thomas Weißschuh wrote:

> If the assumption is that this is most likely a kernel bug,
> shouldn't it be fixed properly rather than worked around?
> After all the job of a selftest is to detect bugs to be fixed.

I investigated the history for a bit and it seems likely we cannot
change the kernel here. Call it an undocumented "feature".

MAP_HUGETLBFS rounds up the length to some value, userspace has to
figure that out and not pass incorrect lengths.  The selftest is doing
that wrong.

> If the test is broken on ARM64 64k in general then I am also wondering how
> it didn't fail before my change to the selftest harness.

It got lucky and didn't overmap something important.

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Thomas Weißschuh]

On Thu, Jun 12, 2025 at 11:58:01AM -0300, Jason Gunthorpe wrote:
> On Thu, Jun 12, 2025 at 04:27:41PM +0200, Thomas Weißschuh wrote:
> 
> > If the assumption is that this is most likely a kernel bug,
> > shouldn't it be fixed properly rather than worked around?
> > After all the job of a selftest is to detect bugs to be fixed.
> 
> I investigated the history for a bit and it seems likely we cannot
> change the kernel here. Call it an undocumented "feature".

I looked a bit and it seems to be mentioned in mmap(2):

	For mmap(), offset must be a multiple of the underlying huge page size.
	The system automatically aligns length to be a multiple of the underlying huge page size.

And MAP_FIXED is documented to wipe away whichever mapping was there before.

> MAP_HUGETLBFS rounds up the length to some value, userspace has to
> figure that out and not pass incorrect lengths.  The selftest is doing
> that wrong.

The selftest would be more robust if MAP_FIXED is replaced by
MAP_FIXED_NOREPLACE. Even with the new explicit skip logic it should
make debugging easier if something goes wrong.

> > If the test is broken on ARM64 64k in general then I am also wondering how
> > it didn't fail before my change to the selftest harness.
> 
> It got lucky and didn't overmap something important.

Oh, okay.

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Thu, Jun 12, 2025 at 05:23:01PM +0200, Thomas Weißschuh wrote:
> On Thu, Jun 12, 2025 at 11:58:01AM -0300, Jason Gunthorpe wrote:
> > On Thu, Jun 12, 2025 at 04:27:41PM +0200, Thomas Weißschuh wrote:
> > 
> > > If the assumption is that this is most likely a kernel bug,
> > > shouldn't it be fixed properly rather than worked around?
> > > After all the job of a selftest is to detect bugs to be fixed.
> > 
> > I investigated the history for a bit and it seems likely we cannot
> > change the kernel here. Call it an undocumented "feature".
> 
> I looked a bit and it seems to be mentioned in mmap(2):
> 
> 	For mmap(), offset must be a multiple of the underlying huge page size.
> 	The system automatically aligns length to be a multiple of the underlying huge page size.

Oh there you go then :) Horrible design. No way for userspace to know
what the rounded up length actually was and thus no way for
userspace to unmap it.

> > MAP_HUGETLBFS rounds up the length to some value, userspace has to
> > figure that out and not pass incorrect lengths.  The selftest is doing
> > that wrong.
> 
> The selftest would be more robust if MAP_FIXED is replaced by
> MAP_FIXED_NOREPLACE. Even with the new explicit skip logic it should
> make debugging easier if something goes wrong.

The point is to replace something that is already mapped there, though
I no longer remember why it is working like this.

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Thu, Jun 12, 2025 at 12:42:42PM -0300, Jason Gunthorpe wrote:
> On Thu, Jun 12, 2025 at 05:23:01PM +0200, Thomas Weißschuh wrote:
> > On Thu, Jun 12, 2025 at 11:58:01AM -0300, Jason Gunthorpe wrote:
> > > On Thu, Jun 12, 2025 at 04:27:41PM +0200, Thomas Weißschuh wrote:
> > > 
> > > > If the assumption is that this is most likely a kernel bug,
> > > > shouldn't it be fixed properly rather than worked around?
> > > > After all the job of a selftest is to detect bugs to be fixed.
> > > 
> > > I investigated the history for a bit and it seems likely we cannot
> > > change the kernel here. Call it an undocumented "feature".
> > 
> > I looked a bit and it seems to be mentioned in mmap(2):
> > 
> > 	For mmap(), offset must be a multiple of the underlying huge page size.
> > 	The system automatically aligns length to be a multiple of the underlying huge page size.
> 
> Oh there you go then :) Horrible design. No way for userspace to know
> what the rounded up length actually was and thus no way for
> userspace to unmap it.

OK. I think we would have to skip those cases then.

> > > MAP_HUGETLBFS rounds up the length to some value, userspace has to
> > > figure that out and not pass incorrect lengths.  The selftest is doing
> > > that wrong.
> > 
> > The selftest would be more robust if MAP_FIXED is replaced by
> > MAP_FIXED_NOREPLACE. Even with the new explicit skip logic it should
> > make debugging easier if something goes wrong.
> 
> The point is to replace something that is already mapped there, though
> I no longer remember why it is working like this.

By replacing MAP_FIXED with MAP_FIXED_NOREPLACE, at the existing
two places, the selftest crashed at early setup_sizes...:

iommufd: iommufd.c:53: setup_sizes: Assertion `vrc == buffer' failed.
/nicolinc/iommufd_selftest.sh: line 19: 21487 Aborted
		(core dumped) tools/testing/selftests/iommu/iommufd
strace:
mmap(0xffff80000000, 1048576, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS|MAP_FIXED_NOREPLACE, -1, 0) = -1 EEXIST (File exists)

This one doesn't MAP_HUGETLBFS btw...

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Thu, Jun 12, 2025 at 10:53:34AM -0700, Nicolin Chen wrote:
> On Thu, Jun 12, 2025 at 12:42:42PM -0300, Jason Gunthorpe wrote:
> > On Thu, Jun 12, 2025 at 05:23:01PM +0200, Thomas Weißschuh wrote:
> > > On Thu, Jun 12, 2025 at 11:58:01AM -0300, Jason Gunthorpe wrote:
> > > > On Thu, Jun 12, 2025 at 04:27:41PM +0200, Thomas Weißschuh wrote:
> > > > 
> > > > > If the assumption is that this is most likely a kernel bug,
> > > > > shouldn't it be fixed properly rather than worked around?
> > > > > After all the job of a selftest is to detect bugs to be fixed.
> > > > 
> > > > I investigated the history for a bit and it seems likely we cannot
> > > > change the kernel here. Call it an undocumented "feature".
> > > 
> > > I looked a bit and it seems to be mentioned in mmap(2):
> > > 
> > > 	For mmap(), offset must be a multiple of the underlying huge page size.
> > > 	The system automatically aligns length to be a multiple of the underlying huge page size.
> > 
> > Oh there you go then :) Horrible design. No way for userspace to know
> > what the rounded up length actually was and thus no way for
> > userspace to unmap it.
> 
> OK. I think we would have to skip those cases then.

Or.. maybe we could just allocate a huge page:

@@ -2022,7 +2023,19 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
        self->fd = open("/dev/iommu", O_RDWR);
        ASSERT_NE(-1, self->fd);

-       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
+       if (variant->hugepages) {
+               /*
+                * Allocation must be aligned to the HUGEPAGE_SIZE, because the
+                * following mmap() will automatically align the length to be a
+                * multiple of the underlying huge page size. Failing to do the
+                * same at this allocation will result in a memory overwrite by
+                * the mmap().
+                */
+               size = __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE);
+       } else {
+               size = variant->buffer_size;
+       }
+       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, size);
        if (rc || !self->buffer) {
                SKIP(return, "Skipping buffer_size=%lu due to errno=%d",
                           variant->buffer_size, rc);

It can just upsize the allocation, i.e. the test case will only
use the first 64M or 128MB out of the reserved 512MB huge page.

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Thu, Jun 12, 2025 at 11:53:24AM -0700, Nicolin Chen wrote:
> @@ -2022,7 +2023,19 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
>         self->fd = open("/dev/iommu", O_RDWR);
>         ASSERT_NE(-1, self->fd);
> 
> -       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> +       if (variant->hugepages) {
> +               /*
> +                * Allocation must be aligned to the HUGEPAGE_SIZE, because the
> +                * following mmap() will automatically align the length to be a
> +                * multiple of the underlying huge page size. Failing to do the
> +                * same at this allocation will result in a memory overwrite by
> +                * the mmap().
> +                */
> +               size = __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE);
> +       } else {
> +               size = variant->buffer_size;
> +       }
> +       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, size);
>         if (rc || !self->buffer) {
>                 SKIP(return, "Skipping buffer_size=%lu due to errno=%d",
>                            variant->buffer_size, rc);
> 
> It can just upsize the allocation, i.e. the test case will only
> use the first 64M or 128MB out of the reserved 512MB huge page.

The MAP_HUGETLBFS is required that is the whole point of what it is
doing..

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Thu, Jun 12, 2025 at 03:56:13PM -0300, Jason Gunthorpe wrote:
> On Thu, Jun 12, 2025 at 11:53:24AM -0700, Nicolin Chen wrote:
> > @@ -2022,7 +2023,19 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
> >         self->fd = open("/dev/iommu", O_RDWR);
> >         ASSERT_NE(-1, self->fd);
> > 
> > -       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> > +       if (variant->hugepages) {
> > +               /*
> > +                * Allocation must be aligned to the HUGEPAGE_SIZE, because the
> > +                * following mmap() will automatically align the length to be a
> > +                * multiple of the underlying huge page size. Failing to do the
> > +                * same at this allocation will result in a memory overwrite by
> > +                * the mmap().
> > +                */
> > +               size = __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE);
> > +       } else {
> > +               size = variant->buffer_size;
> > +       }
> > +       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, size);
> >         if (rc || !self->buffer) {
> >                 SKIP(return, "Skipping buffer_size=%lu due to errno=%d",
> >                            variant->buffer_size, rc);
> > 
> > It can just upsize the allocation, i.e. the test case will only
> > use the first 64M or 128MB out of the reserved 512MB huge page.
> 
> The MAP_HUGETLBFS is required that is the whole point of what it is
> doing..

I am not quite following this.. MAP_HUGETLB will be still set.

And the underlying selftest case is using:
	MOCK_HUGE_PAGE_SIZE = 512 * MOCK_IO_PAGE_SIZE

Does it matter if the underlying allocation has an overshot?

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Jason Gunthorpe]

On Thu, Jun 12, 2025 at 12:03:14PM -0700, Nicolin Chen wrote:
> On Thu, Jun 12, 2025 at 03:56:13PM -0300, Jason Gunthorpe wrote:
> > On Thu, Jun 12, 2025 at 11:53:24AM -0700, Nicolin Chen wrote:
> > > @@ -2022,7 +2023,19 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
> > >         self->fd = open("/dev/iommu", O_RDWR);
> > >         ASSERT_NE(-1, self->fd);
> > > 
> > > -       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> > > +       if (variant->hugepages) {
> > > +               /*
> > > +                * Allocation must be aligned to the HUGEPAGE_SIZE, because the
> > > +                * following mmap() will automatically align the length to be a
> > > +                * multiple of the underlying huge page size. Failing to do the
> > > +                * same at this allocation will result in a memory overwrite by
> > > +                * the mmap().
> > > +                */
> > > +               size = __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE);
> > > +       } else {
> > > +               size = variant->buffer_size;
> > > +       }
> > > +       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, size);
> > >         if (rc || !self->buffer) {
> > >                 SKIP(return, "Skipping buffer_size=%lu due to errno=%d",
> > >                            variant->buffer_size, rc);
> > > 
> > > It can just upsize the allocation, i.e. the test case will only
> > > use the first 64M or 128MB out of the reserved 512MB huge page.
> > 
> > The MAP_HUGETLBFS is required that is the whole point of what it is
> > doing..
> 
> I am not quite following this.. MAP_HUGETLB will be still set.
> 
> And the underlying selftest case is using:
> 	MOCK_HUGE_PAGE_SIZE = 512 * MOCK_IO_PAGE_SIZE
> 
> Does it matter if the underlying allocation has an overshot?

I expect munmap won't work with the wrong size and the test will OOM?

You'd be better to correct the actual variant->buffer_size..

Jason

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Nicolin Chen]

On Thu, Jun 12, 2025 at 08:31:38PM -0300, Jason Gunthorpe wrote:
> On Thu, Jun 12, 2025 at 12:03:14PM -0700, Nicolin Chen wrote:
> > On Thu, Jun 12, 2025 at 03:56:13PM -0300, Jason Gunthorpe wrote:
> > > On Thu, Jun 12, 2025 at 11:53:24AM -0700, Nicolin Chen wrote:
> > > > @@ -2022,7 +2023,19 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
> > > >         self->fd = open("/dev/iommu", O_RDWR);
> > > >         ASSERT_NE(-1, self->fd);
> > > > 
> > > > -       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size);
> > > > +       if (variant->hugepages) {
> > > > +               /*
> > > > +                * Allocation must be aligned to the HUGEPAGE_SIZE, because the
> > > > +                * following mmap() will automatically align the length to be a
> > > > +                * multiple of the underlying huge page size. Failing to do the
> > > > +                * same at this allocation will result in a memory overwrite by
> > > > +                * the mmap().
> > > > +                */
> > > > +               size = __ALIGN_KERNEL(variant->buffer_size, HUGEPAGE_SIZE);
> > > > +       } else {
> > > > +               size = variant->buffer_size;
> > > > +       }
> > > > +       rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, size);
> > > >         if (rc || !self->buffer) {
> > > >                 SKIP(return, "Skipping buffer_size=%lu due to errno=%d",
> > > >                            variant->buffer_size, rc);
> > > > 
> > > > It can just upsize the allocation, i.e. the test case will only
> > > > use the first 64M or 128MB out of the reserved 512MB huge page.
> > > 
> > > The MAP_HUGETLBFS is required that is the whole point of what it is
> > > doing..
> > 
> > I am not quite following this.. MAP_HUGETLB will be still set.
> > 
> > And the underlying selftest case is using:
> > 	MOCK_HUGE_PAGE_SIZE = 512 * MOCK_IO_PAGE_SIZE
> > 
> > Does it matter if the underlying allocation has an overshot?
> 
> I expect munmap won't work with the wrong size and the test will OOM?
> 
> You'd be better to correct the actual variant->buffer_size..

I saw test passing, before I posted that.

But you are certainly right: while mmap() handling MAP_HUGETLB will
align up the size, the munmap() doesn't. So, passing in to them the
same variant->buffer_size will result in a size mismatch.

I don't think we should change the variant->buffer_size, because it
affects the bitmap sizes in those dirty_tracking test cases. And if
we align up every single variant->buffer_size, the variants of 64MB
and 128Mb will be two duplicated 512MB cases, right?

I think we can just add this on top of that:

 FIXTURE_TEARDOWN(iommufd_dirty_tracking)
 {
-       munmap(self->buffer, variant->buffer_size);
-       munmap(self->bitmap, DIV_ROUND_UP(self->bitmap_size, BITS_PER_BYTE));
+       unsigned long size = variant->buffer_size;
+
+       if (variant->hugepages)
+               size = __ALIGN_KERNEL(size, HUGEPAGE_SIZE);
+       munmap(self->buffer, size);
+       free(self->buffer);
+       free(self->bitmap);
        teardown_iommufd(self->fd, _metadata);
 }

This FIXTURE_TEARDOWN() didn't free the memory allocated by the two
posix_memalign calls in the FIXTURE_SETUP()..

Thanks
Nicolin

Re: [PATCH v4 09/14] selftests: harness: Move teardown conditional into test metadata

[Thomas Weißschuh]

On Tue, Jun 10, 2025 at 11:48:44AM -0700, Nicolin Chen wrote:
> On Tue, Jun 10, 2025 at 09:09:02AM -0300, Jason Gunthorpe wrote:
> > On Tue, Jun 10, 2025 at 01:38:22PM +0200, Thomas Weißschuh wrote:
> > > > ------------------------------------------------------------------
> > > > #  RUN           iommufd_dirty_tracking.domain_dirty128M_huge.enforce_dirty ...
> > > > # enforce_dirty: Test terminated unexpectedly by signal 11
> > 
> > Sig 11 is weird..
> 
> > > On another note, the selftest should use the kselftest_harness' ASSERT_*()
> > > macros instead of plain assert().
> > 
> > IIRC the kselftest stuff explodes if you try to use it's assert
> > functions within a fixture setup/teardown context.
> > 
> > I also wasn't able to reproduce this (x86 ubuntu 24 LTS OS) Maybe
> > it is ARM specific, I think Nicolin is running on ARM..
> 
> Yes. And I was running with 64KB page size. I just quickly retried
> with 4KB page size (matching x86), and all failed tests pass now.

That is very important to know. It should be mentioned in the report.

So I tried to reproduce it. To get even the mmap() in the test to succeed I
needed to also pass default_hugepagesz=2MiB.
Also 1GiB of memory was not enough. 30GiB was however.
But then the tests succeeds fine for me.

So I'll need reproduction steps.


Thomas

[PATCH v4 10/14] selftests: harness: Add teardown callback to test metadata

[Thomas Weißschuh]

To get rid of setjmp()/longjmp(), the teardown logic needs to be usable
from __bail(). Introduce a new callback for it.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 895821af3e5c5752065561d0a108210d79e9eeee..acb476093b74968ecb71180ade9e5852b22da170 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -439,14 +439,10 @@
 			} \
 		} \
 		if (child == 0) { \
-			if (!fixture_name##_teardown_parent && \
-					!__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
-				fixture_name##_teardown(_metadata, self, variant->data); \
+			_metadata->teardown_fn(false, _metadata, self, variant->data); \
 			_exit(0); \
 		} \
-		if (fixture_name##_teardown_parent && \
-				!__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
-			fixture_name##_teardown(_metadata, self, variant->data); \
+		_metadata->teardown_fn(true, _metadata, self, variant->data); \
 		munmap(_metadata->no_teardown, sizeof(*_metadata->no_teardown)); \
 		_metadata->no_teardown = NULL; \
 		if (self && fixture_name##_teardown_parent) \
@@ -460,6 +456,14 @@
 		} \
 		__test_check_assert(_metadata); \
 	} \
+	static void wrapper_##fixture_name##_##test_name##_teardown( \
+		bool in_parent, struct __test_metadata *_metadata, \
+		void *self, const void *variant) \
+	{ \
+		if (fixture_name##_teardown_parent == in_parent && \
+				!__atomic_test_and_set(_metadata->no_teardown, __ATOMIC_RELAXED)) \
+			fixture_name##_teardown(_metadata, self, variant); \
+	} \
 	static struct __test_metadata *_##fixture_name##_##test_name##_object; \
 	static void __attribute__((constructor)) \
 			_register_##fixture_name##_##test_name(void) \
@@ -469,6 +473,7 @@
 		object->name = #test_name; \
 		object->fn = &wrapper_##fixture_name##_##test_name; \
 		object->fixture = &_##fixture_name##_fixture_object; \
+		object->teardown_fn = &wrapper_##fixture_name##_##test_name##_teardown; \
 		object->termsig = signal; \
 		object->timeout = tmout; \
 		_##fixture_name##_##test_name##_object = object; \
@@ -912,6 +917,8 @@ struct __test_metadata {
 		   struct __fixture_variant_metadata *);
 	pid_t pid;	/* pid of test when being run */
 	struct __fixture_metadata *fixture;
+	void (*teardown_fn)(bool in_parent, struct __test_metadata *_metadata,
+			    void *self, const void *variant);
 	int termsig;
 	int exit_code;
 	int trigger; /* extra handler after the evaluation */

-- 
2.49.0

[PATCH v4 11/14] selftests: harness: Add "variant" and "self" to test metadata

[Thomas Weißschuh]

To get rid of setjmp()/longjmp(), the variant and self need to be usable
from __bail().

Make them available from the test metadata.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
 tools/testing/selftests/kselftest_harness.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index acb476093b74968ecb71180ade9e5852b22da170..088c875df91a58f8760749b6047b246fb2a7891f 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -423,6 +423,8 @@
 				self = &self_private; \
 			} \
 		} \
+		_metadata->variant = variant->data; \
+		_metadata->self = self; \
 		if (setjmp(_metadata->env) == 0) { \
 			/* _metadata and potentially self are shared with all forks. */ \
 			child = fork(); \
@@ -926,6 +928,8 @@ struct __test_metadata {
 	bool aborted;	/* stopped test due to failed ASSERT */
 	bool *no_teardown; /* fixture needs teardown */
 	jmp_buf env;	/* for exiting out of test early */
+	void *self;
+	const void *variant;
 	struct __test_results *results;
 	struct __test_metadata *prev, *next;
 };

-- 
2.49.0

[PATCH v4 12/14] selftests: harness: Stop using setjmp()/longjmp()

[Thomas Weißschuh]

Usage of longjmp() was added to ensure that teardown is always run in
commit 63e6b2a42342 ("selftests/harness: Run TEARDOWN for ASSERT failures")
However instead of calling longjmp() to the teardown handler it is easier to
just call the teardown handler directly from __bail().
Any potential duplicate teardown invocations are harmless as the actual
handler will only ever be executed once since
commit fff37bd32c76 ("selftests/harness: Fix fixture teardown").

Additionally this removes a incompatibility with nolibc,
which does not support setjmp()/longjmp().

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
 tools/testing/selftests/kselftest_harness.h | 45 ++++++++++-------------------
 1 file changed, 15 insertions(+), 30 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 088c875df91a58f8760749b6047b246fb2a7891f..2925e47db995d7197ed1f55bd9cc657669df9bd3 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -67,7 +67,6 @@
 #include <sys/types.h>
 #include <sys/wait.h>
 #include <unistd.h>
-#include <setjmp.h>
 
 #include "kselftest.h"
 
@@ -178,9 +177,7 @@
 		struct __test_metadata *_metadata, \
 		struct __fixture_variant_metadata __attribute__((unused)) *variant) \
 	{ \
-		if (setjmp(_metadata->env) == 0) \
-			test_name(_metadata); \
-		__test_check_assert(_metadata); \
+		test_name(_metadata); \
 	} \
 	static struct __test_metadata _##test_name##_object = \
 		{ .name = #test_name, \
@@ -425,24 +422,20 @@
 		} \
 		_metadata->variant = variant->data; \
 		_metadata->self = self; \
-		if (setjmp(_metadata->env) == 0) { \
-			/* _metadata and potentially self are shared with all forks. */ \
-			child = fork(); \
-			if (child == 0) { \
-				fixture_name##_setup(_metadata, self, variant->data); \
-				/* Let setup failure terminate early. */ \
-				if (_metadata->exit_code) \
-					_exit(0); \
-				*_metadata->no_teardown = false; \
-				fixture_name##_##test_name(_metadata, self, variant->data); \
-			} else if (child < 0 || child != waitpid(child, &status, 0)) { \
-				ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \
-				_metadata->exit_code = KSFT_FAIL; \
-			} \
-		} \
+		/* _metadata and potentially self are shared with all forks. */ \
+		child = fork(); \
 		if (child == 0) { \
+			fixture_name##_setup(_metadata, self, variant->data); \
+			/* Let setup failure terminate early. */ \
+			if (_metadata->exit_code) \
+				_exit(0); \
+			*_metadata->no_teardown = false; \
+			fixture_name##_##test_name(_metadata, self, variant->data); \
 			_metadata->teardown_fn(false, _metadata, self, variant->data); \
 			_exit(0); \
+		} else if (child < 0 || child != waitpid(child, &status, 0)) { \
+			ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \
+			_metadata->exit_code = KSFT_FAIL; \
 		} \
 		_metadata->teardown_fn(true, _metadata, self, variant->data); \
 		munmap(_metadata->no_teardown, sizeof(*_metadata->no_teardown)); \
@@ -456,7 +449,6 @@
 			/* Forward signal to __wait_for_test(). */ \
 			kill(getpid(), WTERMSIG(status)); \
 		} \
-		__test_check_assert(_metadata); \
 	} \
 	static void wrapper_##fixture_name##_##test_name##_teardown( \
 		bool in_parent, struct __test_metadata *_metadata, \
@@ -927,7 +919,6 @@ struct __test_metadata {
 	int timeout;	/* seconds to wait for test timeout */
 	bool aborted;	/* stopped test due to failed ASSERT */
 	bool *no_teardown; /* fixture needs teardown */
-	jmp_buf env;	/* for exiting out of test early */
 	void *self;
 	const void *variant;
 	struct __test_results *results;
@@ -963,19 +954,14 @@ static inline int __bail(int for_realz, struct __test_metadata *t)
 {
 	/* if this is ASSERT, return immediately. */
 	if (for_realz) {
-		t->aborted = true;
-		longjmp(t->env, 1);
+		if (t->teardown_fn)
+			t->teardown_fn(false, t, t->self, t->variant);
+		abort();
 	}
 	/* otherwise, end the for loop and continue. */
 	return 0;
 }
 
-static inline void __test_check_assert(struct __test_metadata *t)
-{
-	if (t->aborted)
-		abort();
-}
-
 static void __wait_for_test(struct __test_metadata *t)
 {
 	/*
@@ -1208,7 +1194,6 @@ static void __run_test(struct __fixture_metadata *f,
 	t->trigger = 0;
 	t->aborted = false;
 	t->no_teardown = NULL;
-	memset(t->env, 0, sizeof(t->env));
 	memset(t->results->reason, 0, sizeof(t->results->reason));
 
 	snprintf(test_name, sizeof(test_name), "%s%s%s.%s",

-- 
2.49.0

Re: [PATCH v4 12/14] selftests: harness: Stop using setjmp()/longjmp()

[Nicolin Chen]

Hi Thomas,

CC += Jason

On Mon, May 05, 2025 at 05:15:30PM +0200, Thomas Weißschuh wrote:
> Usage of longjmp() was added to ensure that teardown is always run in
> commit 63e6b2a42342 ("selftests/harness: Run TEARDOWN for ASSERT failures")
> However instead of calling longjmp() to the teardown handler it is easier to
> just call the teardown handler directly from __bail().
> Any potential duplicate teardown invocations are harmless as the actual
> handler will only ever be executed once since
> commit fff37bd32c76 ("selftests/harness: Fix fixture teardown").
> 
> Additionally this removes a incompatibility with nolibc,
> which does not support setjmp()/longjmp().
> 
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>

The iommufd selftest (CONFIG_IOMMUFD_TEST) starts to give warnings
when building with v6.16-rc1, though the test code wasn't changed
at these two functions:
------------------------------------------------------------------
make: Entering directory '/nicolinc/linux-stable/tools/testing/selftests/iommu'
  CC       iommufd
iommufd.c: In function ‘wrapper_iommufd_mock_domain_all_aligns’:
iommufd.c:1806:17: warning: ‘mfd’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 1806 |                 close(mfd);
      |                 ^~~~~~~~~~
iommufd.c:1766:13: note: ‘mfd’ was declared here
 1766 |         int mfd;
      |             ^~~
iommufd.c: In function ‘wrapper_iommufd_mock_domain_all_aligns_copy’:
iommufd.c:1869:17: warning: ‘mfd’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 1869 |                 close(mfd);
      |                 ^~~~~~~~~~
iommufd.c:1818:13: note: ‘mfd’ was declared here
 1818 |         int mfd;
      |             ^~~
  CC       iommufd_fail_nth
make: Leaving directory '/nicolinc/linux-stable/tools/testing/selftests/iommu'
------------------------------------------------------------------

Git bisect points to this patch, and reverting it fixes these.

Both mfds are under the same "if (variant->driver)" check, so the
warnings don't seem legit to me.

Do you have any idea why this happens?

Thanks
Nicolin

Re: [PATCH v4 12/14] selftests: harness: Stop using setjmp()/longjmp()

[Thomas Weißschuh]

Hi Nicolin,

On Mon, Jun 09, 2025 at 11:40:34PM -0700, Nicolin Chen wrote:
> Hi Thomas,
> 
> CC += Jason
> 
> On Mon, May 05, 2025 at 05:15:30PM +0200, Thomas Weißschuh wrote:
> > Usage of longjmp() was added to ensure that teardown is always run in
> > commit 63e6b2a42342 ("selftests/harness: Run TEARDOWN for ASSERT failures")
> > However instead of calling longjmp() to the teardown handler it is easier to
> > just call the teardown handler directly from __bail().
> > Any potential duplicate teardown invocations are harmless as the actual
> > handler will only ever be executed once since
> > commit fff37bd32c76 ("selftests/harness: Fix fixture teardown").
> > 
> > Additionally this removes a incompatibility with nolibc,
> > which does not support setjmp()/longjmp().
> > 
> > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> 
> The iommufd selftest (CONFIG_IOMMUFD_TEST) starts to give warnings
> when building with v6.16-rc1, though the test code wasn't changed
> at these two functions:

Thanks for the report.

> ------------------------------------------------------------------
> make: Entering directory '/nicolinc/linux-stable/tools/testing/selftests/iommu'
>   CC       iommufd
> iommufd.c: In function ‘wrapper_iommufd_mock_domain_all_aligns’:
> iommufd.c:1806:17: warning: ‘mfd’ may be used uninitialized in this function [-Wmaybe-uninitialized]
>  1806 |                 close(mfd);
>       |                 ^~~~~~~~~~
> iommufd.c:1766:13: note: ‘mfd’ was declared here
>  1766 |         int mfd;
>       |             ^~~
> iommufd.c: In function ‘wrapper_iommufd_mock_domain_all_aligns_copy’:
> iommufd.c:1869:17: warning: ‘mfd’ may be used uninitialized in this function [-Wmaybe-uninitialized]
>  1869 |                 close(mfd);
>       |                 ^~~~~~~~~~
> iommufd.c:1818:13: note: ‘mfd’ was declared here
>  1818 |         int mfd;
>       |             ^~~
>   CC       iommufd_fail_nth
> make: Leaving directory '/nicolinc/linux-stable/tools/testing/selftests/iommu'
> ------------------------------------------------------------------
> 
> Git bisect points to this patch, and reverting it fixes these.
> 
> Both mfds are under the same "if (variant->driver)" check, so the
> warnings don't seem legit to me.
> 
> Do you have any idea why this happens?

It does look very weird. I could understand if the compiler assumes that
variant->file changes during the runtime of the function.
But even if I work around this, by introducing a local variable "bool file =
variant->file" the issue persists.
However as soon as the value of of "bool file" is fixed to either "true" or
"false" it goes away.


The following diff *alone* also prevents the warning, but that doesn't
make any sense either:

--- a/tools/testing/selftests/iommu/iommufd_utils.h
+++ b/tools/testing/selftests/iommu/iommufd_utils.h
@@ -65,7 +65,7 @@ static inline void *memfd_mmap(size_t length, int prot, int flags, int *mfd_p)
                return MAP_FAILED;
        if (ftruncate(mfd, length))
                return MAP_FAILED;
-       *mfd_p = mfd;
+       *mfd_p = 0;
        return mmap(0, length, prot, flags, mfd, 0);
 }


Maybe the logic became too complex for GCC?
Case in point, when trying with an older GCC 13.2, the following warning appeared:

In file included from iommufd_utils.h:14,
                 from iommufd.c:12:
In function 'iommufd_viommu_vdevice_alloc',
    inlined from 'wrapper_iommufd_viommu_vdevice_alloc' at iommufd.c:2731:1:
../kselftest_harness.h:760:12: warning: 'ret' may be used uninitialized [-Wmaybe-uninitialized]
  760 |         if (!(__exp _t __seen)) { \
      |            ^
../kselftest_harness.h:513:9: note: in expansion of macro '__EXPECT'
  513 |         __EXPECT(expected, #expected, seen, #seen, ==, 1)
      |         ^~~~~~~~
iommufd_utils.h:1005:9: note: in expansion of macro 'ASSERT_EQ'
 1005 |         ASSERT_EQ(0, _test_cmd_trigger_vevents(self->fd, dev_id, nvevents))
      |         ^~~~~~~~~
iommufd.c:2766:17: note: in expansion of macro 'test_cmd_trigger_vevents'
 2766 |                 test_cmd_trigger_vevents(dev_id, 3);
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~
iommufd_utils.h: In function 'wrapper_iommufd_viommu_vdevice_alloc':
iommufd_utils.h:993:13: note: 'ret' was declared here
  993 |         int ret;
      |             ^~~

Which is also just a false-positive and fixed with newer GCCs.


Thomas

Re: [PATCH v4 12/14] selftests: harness: Stop using setjmp()/longjmp()

[Nicolin Chen]

On Tue, Jun 10, 2025 at 02:21:25PM +0200, Thomas Weißschuh wrote:
> The following diff *alone* also prevents the warning, but that doesn't
> make any sense either:
> 
> --- a/tools/testing/selftests/iommu/iommufd_utils.h
> +++ b/tools/testing/selftests/iommu/iommufd_utils.h
> @@ -65,7 +65,7 @@ static inline void *memfd_mmap(size_t length, int prot, int flags, int *mfd_p)
>                 return MAP_FAILED;
>         if (ftruncate(mfd, length))
>                 return MAP_FAILED;
> -       *mfd_p = mfd;
> +       *mfd_p = 0;
>         return mmap(0, length, prot, flags, mfd, 0);
>  }
> 
> 
> Maybe the logic became too complex for GCC?

Maybe. Those warnings are gone using a dummy setjmp() without a
longjmp() :-/

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 2925e47db995..2dc288413fc7 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -67,6 +67,7 @@
 #include <sys/types.h>
 #include <sys/wait.h>
 #include <unistd.h>
+#include <setjmp.h>

 #include "kselftest.h"

@@ -407,6 +408,7 @@
                FIXTURE_DATA(fixture_name) self_private, *self = NULL; \
                pid_t child = 1; \
                int status = 0; \
+               jmp_buf test = {}; \
                /* Makes sure there is only one teardown, even when child forks again. */ \
                _metadata->no_teardown = mmap(NULL, sizeof(*_metadata->no_teardown), \
                        PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); \
@@ -422,6 +424,7 @@
                } \
                _metadata->variant = variant->data; \
                _metadata->self = self; \
+               setjmp(test); \
                /* _metadata and potentially self are shared with all forks. */ \
                child = fork(); \
                if (child == 0) { \

[PATCH v4 13/14] selftests: harness: Guard includes on nolibc

[Thomas Weißschuh]

Nolibc doesn't provide all normal header files.

Don't try to include these non-existent header files, as the symbols are
available unconditionally anyways.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 tools/testing/selftests/kselftest_harness.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index 2925e47db995d7197ed1f55bd9cc657669df9bd3..3fa92860cb25f399e27b25385d9de5b54359b93b 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -57,16 +57,19 @@
 #include <ctype.h>
 #include <errno.h>
 #include <linux/unistd.h>
-#include <poll.h>
 #include <stdbool.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <unistd.h>
+
+#ifndef NOLIBC
+#include <poll.h>
 #include <sys/mman.h>
 #include <sys/types.h>
 #include <sys/wait.h>
-#include <unistd.h>
+#endif
 
 #include "kselftest.h"
 

-- 
2.49.0

[PATCH v4 14/14] HACK: selftests/nolibc: demonstrate usage of the kselftest harness

[Thomas Weißschuh]

Show how to use the kselftest harness together with nolibc.
This just runs the existing harness selftest by crudely replacing the
regular nolibc-test.c with the harness-selftest.c to get that wired up easily.
To use it:
	$ cd tools/testing/selftests/nolibc/
	$ ./run-tests -m user

In the future nolibc-test can use the harness for itself.

Not-Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
---
 .../selftests/kselftest_harness/harness-selftest.c |    2 +-
 tools/testing/selftests/nolibc/Makefile            |   15 +-
 tools/testing/selftests/nolibc/harness-selftest.c  |    1 +
 tools/testing/selftests/nolibc/nolibc-test.c       | 1715 +-------------------
 tools/testing/selftests/nolibc/run-tests.sh        |    2 +-
 5 files changed, 12 insertions(+), 1723 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness/harness-selftest.c b/tools/testing/selftests/kselftest_harness/harness-selftest.c
index a0e3e89ae291f8d9f4f2ab9746a838fd1bfaf6c1..c522a18b95cf070b9c0323db2b4fb4f79e2d8788 100644
--- a/tools/testing/selftests/kselftest_harness/harness-selftest.c
+++ b/tools/testing/selftests/kselftest_harness/harness-selftest.c
@@ -1,4 +1,4 @@
-// SPDX-License-Identifier: GPL-2.0
+/* SPDX-License-Identifier: GPL-2.0 */
 
 #include <stdio.h>
 
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index 2671383045dbb51682fd62ac8c8df9c317157317..af350f1dd584eb917a2e20f7caa6748f0ce12b67 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -180,8 +180,8 @@ Q=@
 endif
 
 CFLAGS_i386 = $(call cc-option,-m32)
-CFLAGS_arm = -marm
-CFLAGS_armthumb = -mthumb -march=armv6t2
+CFLAGS_arm = -marm -march=armv7-a
+CFLAGS_armthumb = -mthumb -march=armv7
 CFLAGS_ppc = -m32 -mbig-endian -mno-vsx $(call cc-option,-mmultiple)
 CFLAGS_ppc64 = -m64 -mbig-endian -mno-vsx $(call cc-option,-mmultiple)
 CFLAGS_ppc64le = -m64 -mlittle-endian -mno-vsx $(call cc-option,-mabi=elfv2)
@@ -195,9 +195,11 @@ CFLAGS_XARCH = $(CFLAGS_$(XARCH))
 endif
 CFLAGS_STACKPROTECTOR ?= $(call cc-option,-mstack-protector-guard=global $(call cc-option,-fstack-protector-all))
 CFLAGS_SANITIZER ?= $(call cc-option,-fsanitize=undefined -fsanitize-trap=all)
-CFLAGS  ?= -Os -fno-ident -fno-asynchronous-unwind-tables -std=c89 -W -Wall -Wextra \
+CFLAGS  ?= -Os -fno-ident -fno-asynchronous-unwind-tables -std=c89 -W -Wall -Wextra -ggdb -ffreestanding \
+	        -ffile-prefix-map=./= \
 		$(call cc-option,-fno-stack-protector) $(call cc-option,-Wmissing-prototypes) \
-		$(CFLAGS_XARCH) $(CFLAGS_STACKPROTECTOR) $(CFLAGS_SANITIZER) $(CFLAGS_EXTRA)
+		$(call cc-option,-mno-outline-atomics) \
+		$(CFLAGS_$(XARCH)) $(CFLAGS_STACKPROTECTOR) $(CFLAGS_SANITIZER) $(CFLAGS_EXTRA)
 LDFLAGS :=
 
 LIBGCC := -lgcc
@@ -213,10 +215,7 @@ include $(srctree)/tools/scripts/Makefile.include
 # GCC uses "s390", clang "systemz"
 CLANG_CROSS_FLAGS := $(subst --target=s390-linux,--target=systemz-linux,$(CLANG_CROSS_FLAGS))
 
-REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{if (!f) printf("\n"); f++; print;} /\[SKIPPED\][\r]*$$/{s++} \
-		END{ printf("\n%3d test(s): %3d passed, %3d skipped, %3d failed => status: ", p+s+f, p, s, f); \
-		if (f || !p) printf("failure\n"); else if (s) printf("warning\n"); else printf("success\n");; \
-		printf("\nSee all results in %s\n", ARGV[1]); }'
+REPORT  = sed -i -e '/^\[/d' -e 's/\x0d//' run.out; cmp ../kselftest_harness/harness-selftest.expected run.out && echo ok; true
 
 help:
 	@echo "Supported targets under selftests/nolibc:"
diff --git a/tools/testing/selftests/nolibc/harness-selftest.c b/tools/testing/selftests/nolibc/harness-selftest.c
new file mode 120000
index 0000000000000000000000000000000000000000..a086839bc322efb070b8f03363ef2874d9f44c17
--- /dev/null
+++ b/tools/testing/selftests/nolibc/harness-selftest.c
@@ -0,0 +1 @@
+../kselftest_harness/harness-selftest.c
\ No newline at end of file
diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 1ad0db92f0ed47f708363b2e558717fa0e686b8f..9a074d2b24c99d86bf27f8399f2e7dc719dbcd24 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -1,1716 +1,5 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 
-#define _GNU_SOURCE
-#define _LARGEFILE64_SOURCE
+#define inline __inline__
 
-/* libc-specific include files
- * The program may be built in 3 ways:
- *   $(CC) -nostdlib -include /path/to/nolibc.h => NOLIBC already defined
- *   $(CC) -nostdlib -I/path/to/nolibc/sysroot  => _NOLIBC_* guards are present
- *   $(CC) with default libc                    => NOLIBC* never defined
- */
-#ifndef NOLIBC
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#ifndef _NOLIBC_STDIO_H
-/* standard libcs need more includes */
-#include <sys/auxv.h>
-#include <sys/ioctl.h>
-#include <sys/mman.h>
-#include <sys/mount.h>
-#include <sys/prctl.h>
-#include <sys/reboot.h>
-#include <sys/resource.h>
-#include <sys/stat.h>
-#include <sys/syscall.h>
-#include <sys/sysmacros.h>
-#include <sys/time.h>
-#include <sys/utsname.h>
-#include <sys/wait.h>
-#include <dirent.h>
-#include <errno.h>
-#include <fcntl.h>
-#include <poll.h>
-#include <sched.h>
-#include <signal.h>
-#include <stdarg.h>
-#include <stddef.h>
-#include <stdint.h>
-#include <unistd.h>
-#include <limits.h>
-#include <ctype.h>
-#endif
-#endif
-
-#pragma GCC diagnostic ignored "-Wmissing-prototypes"
-
-#include "nolibc-test-linkage.h"
-
-/* for the type of int_fast16_t and int_fast32_t, musl differs from glibc and nolibc */
-#define SINT_MAX_OF_TYPE(type) (((type)1 << (sizeof(type) * 8 - 2)) - (type)1 + ((type)1 << (sizeof(type) * 8 - 2)))
-#define SINT_MIN_OF_TYPE(type) (-SINT_MAX_OF_TYPE(type) - 1)
-
-/* will be used to test initialization of environ */
-static char **test_envp;
-
-/* will be used to test initialization of argv */
-static char **test_argv;
-
-/* will be used to test initialization of argc */
-static int test_argc;
-
-/* will be used by some test cases as readable file, please don't write it */
-static const char *argv0;
-
-/* will be used by constructor tests */
-static int constructor_test_value;
-
-static const int is_nolibc =
-#ifdef NOLIBC
-	1
-#else
-	0
-#endif
-;
-
-/* definition of a series of tests */
-struct test {
-	const char *name;              /* test name */
-	int (*func)(int min, int max); /* handler */
-};
-
-#ifndef _NOLIBC_STDLIB_H
-char *itoa(int i)
-{
-	static char buf[12];
-	int ret;
-
-	ret = snprintf(buf, sizeof(buf), "%d", i);
-	return (ret >= 0 && ret < sizeof(buf)) ? buf : "#err";
-}
-#endif
-
-#define CASE_ERR(err) \
-	case err: return #err
-
-/* returns the error name (e.g. "ENOENT") for common errors, "SUCCESS" for 0,
- * or the decimal value for less common ones.
- */
-static const char *errorname(int err)
-{
-	switch (err) {
-	case 0: return "SUCCESS";
-	CASE_ERR(EPERM);
-	CASE_ERR(ENOENT);
-	CASE_ERR(ESRCH);
-	CASE_ERR(EINTR);
-	CASE_ERR(EIO);
-	CASE_ERR(ENXIO);
-	CASE_ERR(E2BIG);
-	CASE_ERR(ENOEXEC);
-	CASE_ERR(EBADF);
-	CASE_ERR(ECHILD);
-	CASE_ERR(EAGAIN);
-	CASE_ERR(ENOMEM);
-	CASE_ERR(EACCES);
-	CASE_ERR(EFAULT);
-	CASE_ERR(ENOTBLK);
-	CASE_ERR(EBUSY);
-	CASE_ERR(EEXIST);
-	CASE_ERR(EXDEV);
-	CASE_ERR(ENODEV);
-	CASE_ERR(ENOTDIR);
-	CASE_ERR(EISDIR);
-	CASE_ERR(EINVAL);
-	CASE_ERR(ENFILE);
-	CASE_ERR(EMFILE);
-	CASE_ERR(ENOTTY);
-	CASE_ERR(ETXTBSY);
-	CASE_ERR(EFBIG);
-	CASE_ERR(ENOSPC);
-	CASE_ERR(ESPIPE);
-	CASE_ERR(EROFS);
-	CASE_ERR(EMLINK);
-	CASE_ERR(EPIPE);
-	CASE_ERR(EDOM);
-	CASE_ERR(ERANGE);
-	CASE_ERR(ENOSYS);
-	CASE_ERR(EOVERFLOW);
-	default:
-		return itoa(err);
-	}
-}
-
-static void align_result(size_t llen)
-{
-	const size_t align = 64;
-	char buf[align];
-	size_t n;
-
-	if (llen >= align)
-		return;
-
-	n = align - llen;
-	memset(buf, ' ', n);
-	buf[n] = '\0';
-	fputs(buf, stdout);
-}
-
-enum RESULT {
-	OK,
-	FAIL,
-	SKIPPED,
-};
-
-static void result(int llen, enum RESULT r)
-{
-	const char *msg;
-
-	if (r == OK)
-		msg = "  [OK]";
-	else if (r == SKIPPED)
-		msg = "[SKIPPED]";
-	else
-		msg = " [FAIL]";
-
-	align_result(llen);
-	puts(msg);
-}
-
-/* The tests below are intended to be used by the macroes, which evaluate
- * expression <expr>, print the status to stdout, and update the "ret"
- * variable to count failures. The functions themselves return the number
- * of failures, thus either 0 or 1.
- */
-
-#define EXPECT_ZR(cond, expr)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_zr(expr, llen); } while (0)
-
-static __attribute__((unused))
-int expect_zr(int expr, int llen)
-{
-	int ret = !(expr == 0);
-
-	llen += printf(" = %d ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_NZ(cond, expr, val)			\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_nz(expr, llen; } while (0)
-
-static __attribute__((unused))
-int expect_nz(int expr, int llen)
-{
-	int ret = !(expr != 0);
-
-	llen += printf(" = %d ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_EQ(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_eq(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_eq(uint64_t expr, int llen, uint64_t val)
-{
-	int ret = !(expr == val);
-
-	llen += printf(" = %lld ", (long long)expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_NE(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ne(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_ne(int expr, int llen, int val)
-{
-	int ret = !(expr != val);
-
-	llen += printf(" = %d ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_GE(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ge(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_ge(int expr, int llen, int val)
-{
-	int ret = !(expr >= val);
-
-	llen += printf(" = %d ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_GT(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_gt(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_gt(int expr, int llen, int val)
-{
-	int ret = !(expr > val);
-
-	llen += printf(" = %d ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_LE(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_le(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_le(int expr, int llen, int val)
-{
-	int ret = !(expr <= val);
-
-	llen += printf(" = %d ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_LT(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_lt(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_lt(int expr, int llen, int val)
-{
-	int ret = !(expr < val);
-
-	llen += printf(" = %d ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_SYSZR(cond, expr)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_syszr(expr, llen); } while (0)
-
-static __attribute__((unused))
-int expect_syszr(int expr, int llen)
-{
-	int ret = 0;
-
-	if (errno == ENOSYS) {
-		llen += printf(" = ENOSYS");
-		result(llen, SKIPPED);
-	} else if (expr) {
-		ret = 1;
-		llen += printf(" = %d %s ", expr, errorname(errno));
-		result(llen, FAIL);
-	} else {
-		llen += printf(" = %d ", expr);
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_SYSEQ(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_syseq(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_syseq(int expr, int llen, int val)
-{
-	int ret = 0;
-
-	if (expr != val) {
-		ret = 1;
-		llen += printf(" = %d %s ", expr, errorname(errno));
-		result(llen, FAIL);
-	} else {
-		llen += printf(" = %d ", expr);
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_SYSNE(cond, expr, val)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_sysne(expr, llen, val); } while (0)
-
-static __attribute__((unused))
-int expect_sysne(int expr, int llen, int val)
-{
-	int ret = 0;
-
-	if (errno == ENOSYS) {
-		llen += printf(" = ENOSYS");
-		result(llen, SKIPPED);
-	} else if (expr == val) {
-		ret = 1;
-		llen += printf(" = %d %s ", expr, errorname(errno));
-		result(llen, FAIL);
-	} else {
-		llen += printf(" = %d ", expr);
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_SYSER2(cond, expr, expret, experr1, experr2)		\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_syserr2(expr, expret, experr1, experr2, llen); } while (0)
-
-#define EXPECT_SYSER(cond, expr, expret, experr)			\
-	EXPECT_SYSER2(cond, expr, expret, experr, 0)
-
-static __attribute__((unused))
-int expect_syserr2(int expr, int expret, int experr1, int experr2, int llen)
-{
-	int ret = 0;
-	int _errno = errno;
-
-	llen += printf(" = %d %s ", expr, errorname(_errno));
-	if (errno == ENOSYS) {
-		result(llen, SKIPPED);
-	} else if (expr != expret || (_errno != experr1 && _errno != experr2)) {
-		ret = 1;
-		if (experr2 == 0)
-			llen += printf(" != (%d %s) ", expret, errorname(experr1));
-		else
-			llen += printf(" != (%d %s %s) ", expret, errorname(experr1), errorname(experr2));
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_PTRZR(cond, expr)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrzr(expr, llen); } while (0)
-
-static __attribute__((unused))
-int expect_ptrzr(const void *expr, int llen)
-{
-	int ret = 0;
-
-	llen += printf(" = <%p> ", expr);
-	if (expr) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_PTRNZ(cond, expr)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrnz(expr, llen); } while (0)
-
-static __attribute__((unused))
-int expect_ptrnz(const void *expr, int llen)
-{
-	int ret = 0;
-
-	llen += printf(" = <%p> ", expr);
-	if (!expr) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-#define EXPECT_PTREQ(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptreq(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_ptreq(const void *expr, int llen, const void *cmp)
-{
-	int ret = 0;
-
-	llen += printf(" = <%p> ", expr);
-	if (expr != cmp) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-#define EXPECT_PTRNE(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrne(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_ptrne(const void *expr, int llen, const void *cmp)
-{
-	int ret = 0;
-
-	llen += printf(" = <%p> ", expr);
-	if (expr == cmp) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-#define EXPECT_PTRGE(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrge(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_ptrge(const void *expr, int llen, const void *cmp)
-{
-	int ret = !(expr >= cmp);
-
-	llen += printf(" = <%p> ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-#define EXPECT_PTRGT(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrgt(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_ptrgt(const void *expr, int llen, const void *cmp)
-{
-	int ret = !(expr > cmp);
-
-	llen += printf(" = <%p> ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_PTRLE(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrle(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_ptrle(const void *expr, int llen, const void *cmp)
-{
-	int ret = !(expr <= cmp);
-
-	llen += printf(" = <%p> ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-
-#define EXPECT_PTRLT(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrlt(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_ptrlt(const void *expr, int llen, const void *cmp)
-{
-	int ret = !(expr < cmp);
-
-	llen += printf(" = <%p> ", expr);
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-#define EXPECT_PTRER2(cond, expr, expret, experr1, experr2)		\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_ptrerr2(expr, expret, experr1, experr2, llen); } while (0)
-
-#define EXPECT_PTRER(cond, expr, expret, experr)			\
-	EXPECT_PTRER2(cond, expr, expret, experr, 0)
-
-static __attribute__((unused))
-int expect_ptrerr2(const void *expr, const void *expret, int experr1, int experr2, int llen)
-{
-	int ret = 0;
-	int _errno = errno;
-
-	llen += printf(" = <%p> %s ", expr, errorname(_errno));
-	if (expr != expret || (_errno != experr1 && _errno != experr2)) {
-		ret = 1;
-		if (experr2 == 0)
-			llen += printf(" != (<%p> %s) ", expret, errorname(experr1));
-		else
-			llen += printf(" != (<%p> %s %s) ", expret, errorname(experr1), errorname(experr2));
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-#define EXPECT_STRZR(cond, expr)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_strzr(expr, llen); } while (0)
-
-static __attribute__((unused))
-int expect_strzr(const char *expr, int llen)
-{
-	int ret = 0;
-
-	llen += printf(" = <%s> ", expr ? expr : "(null)");
-	if (expr) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_STRNZ(cond, expr)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_strnz(expr, llen); } while (0)
-
-static __attribute__((unused))
-int expect_strnz(const char *expr, int llen)
-{
-	int ret = 0;
-
-	llen += printf(" = <%s> ", expr ? expr : "(null)");
-	if (!expr) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_STREQ(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_streq(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_streq(const char *expr, int llen, const char *cmp)
-{
-	int ret = 0;
-
-	llen += printf(" = <%s> ", expr);
-	if (strcmp(expr, cmp) != 0) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-
-#define EXPECT_STRNE(cond, expr, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_strne(expr, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_strne(const char *expr, int llen, const char *cmp)
-{
-	int ret = 0;
-
-	llen += printf(" = <%s> ", expr);
-	if (strcmp(expr, cmp) == 0) {
-		ret = 1;
-		result(llen, FAIL);
-	} else {
-		result(llen, OK);
-	}
-	return ret;
-}
-
-#define EXPECT_STRBUFEQ(cond, expr, buf, val, cmp)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_str_buf_eq(expr, buf, val, llen, cmp); } while (0)
-
-static __attribute__((unused))
-int expect_str_buf_eq(size_t expr, const char *buf, size_t val, int llen, const char *cmp)
-{
-	llen += printf(" = %lu <%s> ", (unsigned long)expr, buf);
-	if (strcmp(buf, cmp) != 0) {
-		result(llen, FAIL);
-		return 1;
-	}
-	if (expr != val) {
-		result(llen, FAIL);
-		return 1;
-	}
-
-	result(llen, OK);
-	return 0;
-}
-
-#define EXPECT_STRTOX(cond, func, input, base, expected, chars, expected_errno)				\
-	do { if (!(cond)) result(llen, SKIPPED); else ret += expect_strtox(llen, func, input, base, expected, chars, expected_errno); } while (0)
-
-static __attribute__((unused))
-int expect_strtox(int llen, void *func, const char *input, int base, intmax_t expected, int expected_chars, int expected_errno)
-{
-	char *endptr;
-	int actual_errno, actual_chars;
-	intmax_t r;
-
-	errno = 0;
-	if (func == strtol) {
-		r = strtol(input, &endptr, base);
-	} else if (func == strtoul) {
-		r = strtoul(input, &endptr, base);
-	} else {
-		result(llen, FAIL);
-		return 1;
-	}
-	actual_errno = errno;
-	actual_chars = endptr - input;
-
-	llen += printf(" %lld = %lld", (long long)expected, (long long)r);
-	if (r != expected) {
-		result(llen, FAIL);
-		return 1;
-	}
-	if (expected_chars == -1) {
-		if (*endptr != '\0') {
-			result(llen, FAIL);
-			return 1;
-		}
-	} else if (expected_chars != actual_chars) {
-		result(llen, FAIL);
-		return 1;
-	}
-	if (actual_errno != expected_errno) {
-		result(llen, FAIL);
-		return 1;
-	}
-
-	result(llen, OK);
-	return 0;
-}
-
-/* declare tests based on line numbers. There must be exactly one test per line. */
-#define CASE_TEST(name) \
-	case __LINE__: llen += printf("%d %s", test, #name);
-
-/* constructors validate that they are executed in definition order */
-__attribute__((constructor))
-static void constructor1(void)
-{
-	constructor_test_value |= 1 << 0;
-}
-
-__attribute__((constructor))
-static void constructor2(int argc, char **argv, char **envp)
-{
-	if (argc && argv && envp)
-		constructor_test_value |= 1 << 1;
-}
-
-int run_startup(int min, int max)
-{
-	int test;
-	int ret = 0;
-	/* kernel at least passes HOME and TERM, shell passes more */
-	int env_total = 2;
-	/* checking NULL for argv/argv0, environ and _auxv is not enough, let's compare with sbrk(0) or &end */
-	extern char end;
-	char *brk = sbrk(0) != (void *)-1 ? sbrk(0) : &end;
-	/* differ from nolibc, both glibc and musl have no global _auxv */
-	const unsigned long *test_auxv = (void *)-1;
-#ifdef NOLIBC
-	test_auxv = _auxv;
-#endif
-
-	for (test = min; test >= 0 && test <= max; test++) {
-		int llen = 0; /* line length */
-
-		/* avoid leaving empty lines below, this will insert holes into
-		 * test numbers.
-		 */
-		switch (test + __LINE__ + 1) {
-		CASE_TEST(argc);             EXPECT_GE(1, test_argc, 1); break;
-		CASE_TEST(argv_addr);        EXPECT_PTRGT(1, test_argv, brk); break;
-		CASE_TEST(argv_environ);     EXPECT_PTRLT(1, test_argv, environ); break;
-		CASE_TEST(argv_total);       EXPECT_EQ(1, environ - test_argv - 1, test_argc ?: 1); break;
-		CASE_TEST(argv0_addr);       EXPECT_PTRGT(1, argv0, brk); break;
-		CASE_TEST(argv0_str);        EXPECT_STRNZ(1, argv0 > brk ? argv0 : NULL); break;
-		CASE_TEST(argv0_len);        EXPECT_GE(1,  argv0 > brk ? strlen(argv0) : 0, 1); break;
-		CASE_TEST(environ_addr);     EXPECT_PTRGT(1, environ, brk); break;
-		CASE_TEST(environ_envp);     EXPECT_PTREQ(1, environ, test_envp); break;
-		CASE_TEST(environ_auxv);     EXPECT_PTRLT(test_auxv != (void *)-1, environ, test_auxv); break;
-		CASE_TEST(environ_total);    EXPECT_GE(test_auxv != (void *)-1, (void *)test_auxv - (void *)environ - 1, env_total); break;
-		CASE_TEST(environ_HOME);     EXPECT_PTRNZ(1, getenv("HOME")); break;
-		CASE_TEST(auxv_addr);        EXPECT_PTRGT(test_auxv != (void *)-1, test_auxv, brk); break;
-		CASE_TEST(auxv_AT_UID);      EXPECT_EQ(1, getauxval(AT_UID), getuid()); break;
-		CASE_TEST(constructor);      EXPECT_EQ(is_nolibc, constructor_test_value, 0x3); break;
-		CASE_TEST(linkage_errno);    EXPECT_PTREQ(1, linkage_test_errno_addr(), &errno); break;
-		CASE_TEST(linkage_constr);   EXPECT_EQ(1, linkage_test_constructor_test_value, 0x3); break;
-		case __LINE__:
-			return ret; /* must be last */
-		/* note: do not set any defaults so as to permit holes above */
-		}
-	}
-	return ret;
-}
-
-
-/* used by some syscall tests below */
-int test_getdents64(const char *dir)
-{
-	char buffer[4096];
-	int fd, ret;
-	int err;
-
-	ret = fd = open(dir, O_RDONLY | O_DIRECTORY, 0);
-	if (ret < 0)
-		return ret;
-
-	ret = getdents64(fd, (void *)buffer, sizeof(buffer));
-	err = errno;
-	close(fd);
-
-	errno = err;
-	return ret;
-}
-
-static int test_dirent(void)
-{
-	int comm = 0, cmdline = 0;
-	struct dirent dirent, *result;
-	DIR *dir;
-	int ret;
-
-	dir = opendir("/proc/self");
-	if (!dir)
-		return 1;
-
-	while (1) {
-		errno = 0;
-		ret = readdir_r(dir, &dirent, &result);
-		if (ret != 0)
-			return 1;
-		if (!result)
-			break;
-
-		if (strcmp(dirent.d_name, "comm") == 0)
-			comm++;
-		else if (strcmp(dirent.d_name, "cmdline") == 0)
-			cmdline++;
-	}
-
-	if (errno)
-		return 1;
-
-	ret = closedir(dir);
-	if (ret)
-		return 1;
-
-	if (comm != 1 || cmdline != 1)
-		return 1;
-
-	return 0;
-}
-
-int test_getpagesize(void)
-{
-	int x = getpagesize();
-	int c;
-
-	if (x < 0)
-		return x;
-
-#if defined(__x86_64__) || defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__)
-	/*
-	 * x86 family is always 4K page.
-	 */
-	c = (x == 4096);
-#elif defined(__aarch64__)
-	/*
-	 * Linux aarch64 supports three values of page size: 4K, 16K, and 64K
-	 * which are selected at kernel compilation time.
-	 */
-	c = (x == 4096 || x == (16 * 1024) || x == (64 * 1024));
-#else
-	/*
-	 * Assuming other architectures must have at least 4K page.
-	 */
-	c = (x >= 4096);
-#endif
-
-	return !c;
-}
-
-int test_fork(void)
-{
-	int status;
-	pid_t pid;
-
-	/* flush the printf buffer to avoid child flush it */
-	fflush(stdout);
-	fflush(stderr);
-
-	pid = fork();
-
-	switch (pid) {
-	case -1:
-		return 1;
-
-	case 0:
-		exit(123);
-
-	default:
-		pid = waitpid(pid, &status, 0);
-
-		return pid == -1 || !WIFEXITED(status) || WEXITSTATUS(status) != 123;
-	}
-}
-
-int test_stat_timestamps(void)
-{
-	struct stat st;
-
-	if (sizeof(st.st_atim.tv_sec) != sizeof(st.st_atime))
-		return 1;
-
-	if (stat("/proc/self/", &st) && stat(argv0, &st) && stat("/", &st))
-		return 1;
-
-	if (st.st_atim.tv_sec != st.st_atime || st.st_atim.tv_nsec > 1000000000)
-		return 1;
-
-	if (st.st_mtim.tv_sec != st.st_mtime || st.st_mtim.tv_nsec > 1000000000)
-		return 1;
-
-	if (st.st_ctim.tv_sec != st.st_ctime || st.st_ctim.tv_nsec > 1000000000)
-		return 1;
-
-	return 0;
-}
-
-int test_uname(void)
-{
-	struct utsname buf;
-	char osrelease[sizeof(buf.release)];
-	ssize_t r;
-	int fd;
-
-	memset(&buf.domainname, 'P', sizeof(buf.domainname));
-
-	if (uname(&buf))
-		return 1;
-
-	if (strncmp("Linux", buf.sysname, sizeof(buf.sysname)))
-		return 1;
-
-	fd = open("/proc/sys/kernel/osrelease", O_RDONLY);
-	if (fd == -1)
-		return 1;
-
-	r = read(fd, osrelease, sizeof(osrelease));
-	if (r == -1)
-		return 1;
-
-	close(fd);
-
-	if (osrelease[r - 1] == '\n')
-		r--;
-
-	/* Validate one of the later fields to ensure field sizes are correct */
-	if (strncmp(osrelease, buf.release, r))
-		return 1;
-
-	/* Ensure the field domainname is set, it is missing from struct old_utsname */
-	if (strnlen(buf.domainname, sizeof(buf.domainname)) == sizeof(buf.domainname))
-		return 1;
-
-	return 0;
-}
-
-int test_mmap_munmap(void)
-{
-	int ret, fd, i, page_size;
-	void *mem;
-	size_t file_size, length;
-	off_t offset, pa_offset;
-	struct stat stat_buf;
-	const char * const files[] = {
-		"/dev/zero",
-		"/proc/1/exe", "/proc/self/exe",
-		argv0,
-		NULL
-	};
-
-	page_size = getpagesize();
-	if (page_size < 0)
-		return 1;
-
-	/* find a right file to mmap, existed and accessible */
-	for (i = 0; files[i] != NULL; i++) {
-		ret = fd = open(files[i], O_RDONLY);
-		if (ret == -1)
-			continue;
-		else
-			break;
-	}
-	if (ret == -1)
-		return 1;
-
-	ret = stat(files[i], &stat_buf);
-	if (ret == -1)
-		goto end;
-
-	/* file size of the special /dev/zero is 0, let's assign one manually */
-	if (i == 0)
-		file_size = 3*page_size;
-	else
-		file_size = stat_buf.st_size;
-
-	offset = file_size - 1;
-	if (offset < 0)
-		offset = 0;
-	length = file_size - offset;
-	pa_offset = offset & ~(page_size - 1);
-
-	mem = mmap(NULL, length + offset - pa_offset, PROT_READ, MAP_SHARED, fd, pa_offset);
-	if (mem == MAP_FAILED) {
-		ret = 1;
-		goto end;
-	}
-
-	ret = munmap(mem, length + offset - pa_offset);
-
-end:
-	close(fd);
-	return !!ret;
-}
-
-int test_pipe(void)
-{
-	const char *const msg = "hello, nolibc";
-	int pipefd[2];
-	char buf[32];
-	size_t len;
-
-	if (pipe(pipefd) == -1)
-		return 1;
-
-	write(pipefd[1], msg, strlen(msg));
-	close(pipefd[1]);
-	len = read(pipefd[0], buf, sizeof(buf));
-	close(pipefd[0]);
-
-	if (len != strlen(msg))
-		return 1;
-
-	return !!memcmp(buf, msg, len);
-}
-
-int test_rlimit(void)
-{
-	struct rlimit rlim = {
-		.rlim_cur = 1 << 20,
-		.rlim_max = 1 << 21,
-	};
-	int ret;
-
-	ret = setrlimit(RLIMIT_CORE, &rlim);
-	if (ret)
-		return -1;
-
-	rlim.rlim_cur = 0;
-	rlim.rlim_max = 0;
-
-	ret = getrlimit(RLIMIT_CORE, &rlim);
-	if (ret)
-		return -1;
-
-	if (rlim.rlim_cur != 1 << 20)
-		return -1;
-	if (rlim.rlim_max != 1 << 21)
-		return -1;
-
-	return 0;
-}
-
-int test_openat(void)
-{
-	int dev, null;
-
-	dev = openat(AT_FDCWD, "/dev", O_DIRECTORY);
-	if (dev < 0)
-		return -1;
-
-	null = openat(dev, "null", O_RDONLY);
-	close(dev);
-	if (null < 0)
-		return -1;
-
-	close(null);
-	return 0;
-}
-
-/* Run syscall tests between IDs <min> and <max>.
- * Return 0 on success, non-zero on failure.
- */
-int run_syscall(int min, int max)
-{
-	struct timeval tv;
-	struct timezone tz;
-	struct stat stat_buf;
-	int euid0;
-	int proc;
-	int test;
-	int tmp;
-	int ret = 0;
-	void *p1, *p2;
-	int has_gettid = 1;
-	int has_brk;
-
-	/* <proc> indicates whether or not /proc is mounted */
-	proc = stat("/proc", &stat_buf) == 0;
-
-	/* this will be used to skip certain tests that can't be run unprivileged */
-	euid0 = geteuid() == 0;
-
-	/* from 2.30, glibc provides gettid() */
-#if defined(__GLIBC_MINOR__) && defined(__GLIBC__)
-	has_gettid = __GLIBC__ > 2 || (__GLIBC__ == 2 && __GLIBC_MINOR__ >= 30);
-#endif
-
-	/* on musl setting brk()/sbrk() always fails */
-	has_brk = brk(0) == 0;
-
-	for (test = min; test >= 0 && test <= max; test++) {
-		int llen = 0; /* line length */
-
-		/* avoid leaving empty lines below, this will insert holes into
-		 * test numbers.
-		 */
-		switch (test + __LINE__ + 1) {
-		CASE_TEST(getpid);            EXPECT_SYSNE(1, getpid(), -1); break;
-		CASE_TEST(getppid);           EXPECT_SYSNE(1, getppid(), -1); break;
-		CASE_TEST(gettid);            EXPECT_SYSNE(has_gettid, gettid(), -1); break;
-		CASE_TEST(getpgid_self);      EXPECT_SYSNE(1, getpgid(0), -1); break;
-		CASE_TEST(getpgid_bad);       EXPECT_SYSER(1, getpgid(-1), -1, ESRCH); break;
-		CASE_TEST(kill_0);            EXPECT_SYSZR(1, kill(getpid(), 0)); break;
-		CASE_TEST(kill_CONT);         EXPECT_SYSZR(1, kill(getpid(), 0)); break;
-		CASE_TEST(kill_BADPID);       EXPECT_SYSER(1, kill(INT_MAX, 0), -1, ESRCH); break;
-		CASE_TEST(sbrk_0);            EXPECT_PTRNE(has_brk, sbrk(0), (void *)-1); break;
-		CASE_TEST(sbrk);              if ((p1 = p2 = sbrk(4096)) != (void *)-1) p2 = sbrk(-4096); EXPECT_SYSZR(has_brk, (p2 == (void *)-1) || p2 == p1); break;
-		CASE_TEST(brk);               EXPECT_SYSZR(has_brk, brk(sbrk(0))); break;
-		CASE_TEST(chdir_root);        EXPECT_SYSZR(1, chdir("/")); chdir(getenv("PWD")); break;
-		CASE_TEST(chdir_dot);         EXPECT_SYSZR(1, chdir(".")); break;
-		CASE_TEST(chdir_blah);        EXPECT_SYSER(1, chdir("/blah"), -1, ENOENT); break;
-		CASE_TEST(chmod_argv0);       EXPECT_SYSZR(1, chmod(argv0, 0555)); break;
-		CASE_TEST(chmod_self);        EXPECT_SYSER(proc, chmod("/proc/self", 0555), -1, EPERM); break;
-		CASE_TEST(chown_self);        EXPECT_SYSER(proc, chown("/proc/self", 0, 0), -1, EPERM); break;
-		CASE_TEST(chroot_root);       EXPECT_SYSZR(euid0, chroot("/")); break;
-		CASE_TEST(chroot_blah);       EXPECT_SYSER(1, chroot("/proc/self/blah"), -1, ENOENT); break;
-		CASE_TEST(chroot_exe);        EXPECT_SYSER(1, chroot(argv0), -1, ENOTDIR); break;
-		CASE_TEST(close_m1);          EXPECT_SYSER(1, close(-1), -1, EBADF); break;
-		CASE_TEST(close_dup);         EXPECT_SYSZR(1, close(dup(0))); break;
-		CASE_TEST(dup_0);             tmp = dup(0);  EXPECT_SYSNE(1, tmp, -1); close(tmp); break;
-		CASE_TEST(dup_m1);            tmp = dup(-1); EXPECT_SYSER(1, tmp, -1, EBADF); if (tmp != -1) close(tmp); break;
-		CASE_TEST(dup2_0);            tmp = dup2(0, 100);  EXPECT_SYSNE(1, tmp, -1); close(tmp); break;
-		CASE_TEST(dup2_m1);           tmp = dup2(-1, 100); EXPECT_SYSER(1, tmp, -1, EBADF); if (tmp != -1) close(tmp); break;
-		CASE_TEST(dup3_0);            tmp = dup3(0, 100, 0);  EXPECT_SYSNE(1, tmp, -1); close(tmp); break;
-		CASE_TEST(dup3_m1);           tmp = dup3(-1, 100, 0); EXPECT_SYSER(1, tmp, -1, EBADF); if (tmp != -1) close(tmp); break;
-		CASE_TEST(execve_root);       EXPECT_SYSER(1, execve("/", (char*[]){ [0] = "/", [1] = NULL }, NULL), -1, EACCES); break;
-		CASE_TEST(fork);              EXPECT_SYSZR(1, test_fork()); break;
-		CASE_TEST(getdents64_root);   EXPECT_SYSNE(1, test_getdents64("/"), -1); break;
-		CASE_TEST(getdents64_null);   EXPECT_SYSER(1, test_getdents64("/dev/null"), -1, ENOTDIR); break;
-		CASE_TEST(directories);       EXPECT_SYSZR(proc, test_dirent()); break;
-		CASE_TEST(gettimeofday_tv);   EXPECT_SYSZR(1, gettimeofday(&tv, NULL)); break;
-		CASE_TEST(gettimeofday_tv_tz);EXPECT_SYSZR(1, gettimeofday(&tv, &tz)); break;
-		CASE_TEST(getpagesize);       EXPECT_SYSZR(1, test_getpagesize()); break;
-		CASE_TEST(ioctl_tiocinq);     EXPECT_SYSZR(1, ioctl(0, TIOCINQ, &tmp)); break;
-		CASE_TEST(link_root1);        EXPECT_SYSER(1, link("/", "/"), -1, EEXIST); break;
-		CASE_TEST(link_blah);         EXPECT_SYSER(1, link("/proc/self/blah", "/blah"), -1, ENOENT); break;
-		CASE_TEST(link_dir);          EXPECT_SYSER(euid0, link("/", "/blah"), -1, EPERM); break;
-		CASE_TEST(link_cross);        EXPECT_SYSER(proc, link("/proc/self/cmdline", "/blah"), -1, EXDEV); break;
-		CASE_TEST(lseek_m1);          EXPECT_SYSER(1, lseek(-1, 0, SEEK_SET), -1, EBADF); break;
-		CASE_TEST(lseek_0);           EXPECT_SYSER(1, lseek(0, 0, SEEK_SET), -1, ESPIPE); break;
-		CASE_TEST(mkdir_root);        EXPECT_SYSER(1, mkdir("/", 0755), -1, EEXIST); break;
-		CASE_TEST(mmap_bad);          EXPECT_PTRER(1, mmap(NULL, 0, PROT_READ, MAP_PRIVATE, 0, 0), MAP_FAILED, EINVAL); break;
-		CASE_TEST(munmap_bad);        EXPECT_SYSER(1, munmap(NULL, 0), -1, EINVAL); break;
-		CASE_TEST(mmap_munmap_good);  EXPECT_SYSZR(1, test_mmap_munmap()); break;
-		CASE_TEST(open_tty);          EXPECT_SYSNE(1, tmp = open("/dev/null", O_RDONLY), -1); if (tmp != -1) close(tmp); break;
-		CASE_TEST(open_blah);         EXPECT_SYSER(1, tmp = open("/proc/self/blah", O_RDONLY), -1, ENOENT); if (tmp != -1) close(tmp); break;
-		CASE_TEST(openat_dir);        EXPECT_SYSZR(1, test_openat()); break;
-		CASE_TEST(pipe);              EXPECT_SYSZR(1, test_pipe()); break;
-		CASE_TEST(poll_null);         EXPECT_SYSZR(1, poll(NULL, 0, 0)); break;
-		CASE_TEST(poll_stdout);       EXPECT_SYSNE(1, ({ struct pollfd fds = { 1, POLLOUT, 0}; poll(&fds, 1, 0); }), -1); break;
-		CASE_TEST(poll_fault);        EXPECT_SYSER(1, poll(NULL, 1, 0), -1, EFAULT); break;
-		CASE_TEST(prctl);             EXPECT_SYSER(1, prctl(PR_SET_NAME, (unsigned long)NULL, 0, 0, 0), -1, EFAULT); break;
-		CASE_TEST(read_badf);         EXPECT_SYSER(1, read(-1, &tmp, 1), -1, EBADF); break;
-		CASE_TEST(rlimit);            EXPECT_SYSZR(1, test_rlimit()); break;
-		CASE_TEST(rmdir_blah);        EXPECT_SYSER(1, rmdir("/blah"), -1, ENOENT); break;
-		CASE_TEST(sched_yield);       EXPECT_SYSZR(1, sched_yield()); break;
-		CASE_TEST(select_null);       EXPECT_SYSZR(1, ({ struct timeval tv = { 0 }; select(0, NULL, NULL, NULL, &tv); })); break;
-		CASE_TEST(select_stdout);     EXPECT_SYSNE(1, ({ fd_set fds; FD_ZERO(&fds); FD_SET(1, &fds); select(2, NULL, &fds, NULL, NULL); }), -1); break;
-		CASE_TEST(select_fault);      EXPECT_SYSER(1, select(1, (void *)1, NULL, NULL, 0), -1, EFAULT); break;
-		CASE_TEST(stat_blah);         EXPECT_SYSER(1, stat("/proc/self/blah", &stat_buf), -1, ENOENT); break;
-		CASE_TEST(stat_fault);        EXPECT_SYSER(1, stat(NULL, &stat_buf), -1, EFAULT); break;
-		CASE_TEST(stat_timestamps);   EXPECT_SYSZR(1, test_stat_timestamps()); break;
-		CASE_TEST(symlink_root);      EXPECT_SYSER(1, symlink("/", "/"), -1, EEXIST); break;
-		CASE_TEST(uname);             EXPECT_SYSZR(proc, test_uname()); break;
-		CASE_TEST(uname_fault);       EXPECT_SYSER(1, uname(NULL), -1, EFAULT); break;
-		CASE_TEST(unlink_root);       EXPECT_SYSER(1, unlink("/"), -1, EISDIR); break;
-		CASE_TEST(unlink_blah);       EXPECT_SYSER(1, unlink("/proc/self/blah"), -1, ENOENT); break;
-		CASE_TEST(wait_child);        EXPECT_SYSER(1, wait(&tmp), -1, ECHILD); break;
-		CASE_TEST(waitpid_min);       EXPECT_SYSER(1, waitpid(INT_MIN, &tmp, WNOHANG), -1, ESRCH); break;
-		CASE_TEST(waitpid_child);     EXPECT_SYSER(1, waitpid(getpid(), &tmp, WNOHANG), -1, ECHILD); break;
-		CASE_TEST(write_badf);        EXPECT_SYSER(1, write(-1, &tmp, 1), -1, EBADF); break;
-		CASE_TEST(write_zero);        EXPECT_SYSZR(1, write(1, &tmp, 0)); break;
-		CASE_TEST(syscall_noargs);    EXPECT_SYSEQ(1, syscall(__NR_getpid), getpid()); break;
-		CASE_TEST(syscall_args);      EXPECT_SYSER(1, syscall(__NR_statx, 0, NULL, 0, 0, NULL), -1, EFAULT); break;
-		case __LINE__:
-			return ret; /* must be last */
-		/* note: do not set any defaults so as to permit holes above */
-		}
-	}
-	return ret;
-}
-
-int run_stdlib(int min, int max)
-{
-	int test;
-	int ret = 0;
-
-	for (test = min; test >= 0 && test <= max; test++) {
-		int llen = 0; /* line length */
-
-		/* For functions that take a long buffer, like strlcat()
-		 * Add some more chars after the \0, to test functions that overwrite the buffer set
-		 * the \0 at the exact right position.
-		 */
-		char buf[10] = "test123456";
-		buf[4] = '\0';
-
-
-		/* avoid leaving empty lines below, this will insert holes into
-		 * test numbers.
-		 */
-		switch (test + __LINE__ + 1) {
-		CASE_TEST(getenv_TERM);        EXPECT_STRNZ(1, getenv("TERM")); break;
-		CASE_TEST(getenv_blah);        EXPECT_STRZR(1, getenv("blah")); break;
-		CASE_TEST(setcmp_blah_blah);   EXPECT_EQ(1, strcmp("blah", "blah"), 0); break;
-		CASE_TEST(setcmp_blah_blah2);  EXPECT_NE(1, strcmp("blah", "blah2"), 0); break;
-		CASE_TEST(setncmp_blah_blah);  EXPECT_EQ(1, strncmp("blah", "blah", 10), 0); break;
-		CASE_TEST(setncmp_blah_blah4); EXPECT_EQ(1, strncmp("blah", "blah4", 4), 0); break;
-		CASE_TEST(setncmp_blah_blah5); EXPECT_NE(1, strncmp("blah", "blah5", 5), 0); break;
-		CASE_TEST(setncmp_blah_blah6); EXPECT_NE(1, strncmp("blah", "blah6", 6), 0); break;
-		CASE_TEST(strchr_foobar_o);    EXPECT_STREQ(1, strchr("foobar", 'o'), "oobar"); break;
-		CASE_TEST(strchr_foobar_z);    EXPECT_STRZR(1, strchr("foobar", 'z')); break;
-		CASE_TEST(strrchr_foobar_o);   EXPECT_STREQ(1, strrchr("foobar", 'o'), "obar"); break;
-		CASE_TEST(strrchr_foobar_z);   EXPECT_STRZR(1, strrchr("foobar", 'z')); break;
-		CASE_TEST(strlcat_0);          EXPECT_STRBUFEQ(is_nolibc, strlcat(buf, "bar", 0), buf, 3, "test"); break;
-		CASE_TEST(strlcat_1);          EXPECT_STRBUFEQ(is_nolibc, strlcat(buf, "bar", 1), buf, 4, "test"); break;
-		CASE_TEST(strlcat_5);          EXPECT_STRBUFEQ(is_nolibc, strlcat(buf, "bar", 5), buf, 7, "test"); break;
-		CASE_TEST(strlcat_6);          EXPECT_STRBUFEQ(is_nolibc, strlcat(buf, "bar", 6), buf, 7, "testb"); break;
-		CASE_TEST(strlcat_7);          EXPECT_STRBUFEQ(is_nolibc, strlcat(buf, "bar", 7), buf, 7, "testba"); break;
-		CASE_TEST(strlcat_8);          EXPECT_STRBUFEQ(is_nolibc, strlcat(buf, "bar", 8), buf, 7, "testbar"); break;
-		CASE_TEST(strlcpy_0);          EXPECT_STRBUFEQ(is_nolibc, strlcpy(buf, "bar", 0), buf, 3, "test"); break;
-		CASE_TEST(strlcpy_1);          EXPECT_STRBUFEQ(is_nolibc, strlcpy(buf, "bar", 1), buf, 3, ""); break;
-		CASE_TEST(strlcpy_2);          EXPECT_STRBUFEQ(is_nolibc, strlcpy(buf, "bar", 2), buf, 3, "b"); break;
-		CASE_TEST(strlcpy_3);          EXPECT_STRBUFEQ(is_nolibc, strlcpy(buf, "bar", 3), buf, 3, "ba"); break;
-		CASE_TEST(strlcpy_4);          EXPECT_STRBUFEQ(is_nolibc, strlcpy(buf, "bar", 4), buf, 3, "bar"); break;
-		CASE_TEST(memcmp_20_20);       EXPECT_EQ(1, memcmp("aaa\x20", "aaa\x20", 4), 0); break;
-		CASE_TEST(memcmp_20_60);       EXPECT_LT(1, memcmp("aaa\x20", "aaa\x60", 4), 0); break;
-		CASE_TEST(memcmp_60_20);       EXPECT_GT(1, memcmp("aaa\x60", "aaa\x20", 4), 0); break;
-		CASE_TEST(memcmp_20_e0);       EXPECT_LT(1, memcmp("aaa\x20", "aaa\xe0", 4), 0); break;
-		CASE_TEST(memcmp_e0_20);       EXPECT_GT(1, memcmp("aaa\xe0", "aaa\x20", 4), 0); break;
-		CASE_TEST(memcmp_80_e0);       EXPECT_LT(1, memcmp("aaa\x80", "aaa\xe0", 4), 0); break;
-		CASE_TEST(memcmp_e0_80);       EXPECT_GT(1, memcmp("aaa\xe0", "aaa\x80", 4), 0); break;
-		CASE_TEST(limit_int8_max);          EXPECT_EQ(1, INT8_MAX,         (int8_t)          0x7f); break;
-		CASE_TEST(limit_int8_min);          EXPECT_EQ(1, INT8_MIN,         (int8_t)          0x80); break;
-		CASE_TEST(limit_uint8_max);         EXPECT_EQ(1, UINT8_MAX,        (uint8_t)         0xff); break;
-		CASE_TEST(limit_int16_max);         EXPECT_EQ(1, INT16_MAX,        (int16_t)         0x7fff); break;
-		CASE_TEST(limit_int16_min);         EXPECT_EQ(1, INT16_MIN,        (int16_t)         0x8000); break;
-		CASE_TEST(limit_uint16_max);        EXPECT_EQ(1, UINT16_MAX,       (uint16_t)        0xffff); break;
-		CASE_TEST(limit_int32_max);         EXPECT_EQ(1, INT32_MAX,        (int32_t)         0x7fffffff); break;
-		CASE_TEST(limit_int32_min);         EXPECT_EQ(1, INT32_MIN,        (int32_t)         0x80000000); break;
-		CASE_TEST(limit_uint32_max);        EXPECT_EQ(1, UINT32_MAX,       (uint32_t)        0xffffffff); break;
-		CASE_TEST(limit_int64_max);         EXPECT_EQ(1, INT64_MAX,        (int64_t)         0x7fffffffffffffff); break;
-		CASE_TEST(limit_int64_min);         EXPECT_EQ(1, INT64_MIN,        (int64_t)         0x8000000000000000); break;
-		CASE_TEST(limit_uint64_max);        EXPECT_EQ(1, UINT64_MAX,       (uint64_t)        0xffffffffffffffff); break;
-		CASE_TEST(limit_int_least8_max);    EXPECT_EQ(1, INT_LEAST8_MAX,   (int_least8_t)    0x7f); break;
-		CASE_TEST(limit_int_least8_min);    EXPECT_EQ(1, INT_LEAST8_MIN,   (int_least8_t)    0x80); break;
-		CASE_TEST(limit_uint_least8_max);   EXPECT_EQ(1, UINT_LEAST8_MAX,  (uint_least8_t)   0xff); break;
-		CASE_TEST(limit_int_least16_max);   EXPECT_EQ(1, INT_LEAST16_MAX,  (int_least16_t)   0x7fff); break;
-		CASE_TEST(limit_int_least16_min);   EXPECT_EQ(1, INT_LEAST16_MIN,  (int_least16_t)   0x8000); break;
-		CASE_TEST(limit_uint_least16_max);  EXPECT_EQ(1, UINT_LEAST16_MAX, (uint_least16_t)  0xffff); break;
-		CASE_TEST(limit_int_least32_max);   EXPECT_EQ(1, INT_LEAST32_MAX,  (int_least32_t)   0x7fffffff); break;
-		CASE_TEST(limit_int_least32_min);   EXPECT_EQ(1, INT_LEAST32_MIN,  (int_least32_t)   0x80000000); break;
-		CASE_TEST(limit_uint_least32_max);  EXPECT_EQ(1, UINT_LEAST32_MAX, (uint_least32_t)  0xffffffffU); break;
-		CASE_TEST(limit_int_least64_min);   EXPECT_EQ(1, INT_LEAST64_MIN,  (int_least64_t)   0x8000000000000000LL); break;
-		CASE_TEST(limit_int_least64_max);   EXPECT_EQ(1, INT_LEAST64_MAX,  (int_least64_t)   0x7fffffffffffffffLL); break;
-		CASE_TEST(limit_uint_least64_max);  EXPECT_EQ(1, UINT_LEAST64_MAX, (uint_least64_t)  0xffffffffffffffffULL); break;
-		CASE_TEST(limit_int_fast8_max);     EXPECT_EQ(1, INT_FAST8_MAX,    (int_fast8_t)     0x7f); break;
-		CASE_TEST(limit_int_fast8_min);     EXPECT_EQ(1, INT_FAST8_MIN,    (int_fast8_t)     0x80); break;
-		CASE_TEST(limit_uint_fast8_max);    EXPECT_EQ(1, UINT_FAST8_MAX,   (uint_fast8_t)    0xff); break;
-		CASE_TEST(limit_int_fast16_min);    EXPECT_EQ(1, INT_FAST16_MIN,   (int_fast16_t)    SINT_MIN_OF_TYPE(int_fast16_t)); break;
-		CASE_TEST(limit_int_fast16_max);    EXPECT_EQ(1, INT_FAST16_MAX,   (int_fast16_t)    SINT_MAX_OF_TYPE(int_fast16_t)); break;
-		CASE_TEST(limit_uint_fast16_max);   EXPECT_EQ(1, UINT_FAST16_MAX,  (uint_fast16_t)   UINTPTR_MAX); break;
-		CASE_TEST(limit_int_fast32_min);    EXPECT_EQ(1, INT_FAST32_MIN,   (int_fast32_t)    SINT_MIN_OF_TYPE(int_fast32_t)); break;
-		CASE_TEST(limit_int_fast32_max);    EXPECT_EQ(1, INT_FAST32_MAX,   (int_fast32_t)    SINT_MAX_OF_TYPE(int_fast32_t)); break;
-		CASE_TEST(limit_uint_fast32_max);   EXPECT_EQ(1, UINT_FAST32_MAX,  (uint_fast32_t)   UINTPTR_MAX); break;
-		CASE_TEST(limit_int_fast64_min);    EXPECT_EQ(1, INT_FAST64_MIN,   (int_fast64_t)    INT64_MIN); break;
-		CASE_TEST(limit_int_fast64_max);    EXPECT_EQ(1, INT_FAST64_MAX,   (int_fast64_t)    INT64_MAX); break;
-		CASE_TEST(limit_uint_fast64_max);   EXPECT_EQ(1, UINT_FAST64_MAX,  (uint_fast64_t)   UINT64_MAX); break;
-		CASE_TEST(sizeof_long_sane);        EXPECT_EQ(1, sizeof(long) == 8 || sizeof(long) == 4, 1); break;
-		CASE_TEST(limit_intptr_min);        EXPECT_EQ(1, INTPTR_MIN,  sizeof(long) == 8 ? (intptr_t)  0x8000000000000000LL  : (intptr_t)  0x80000000); break;
-		CASE_TEST(limit_intptr_max);        EXPECT_EQ(1, INTPTR_MAX,  sizeof(long) == 8 ? (intptr_t)  0x7fffffffffffffffLL  : (intptr_t)  0x7fffffff); break;
-		CASE_TEST(limit_uintptr_max);       EXPECT_EQ(1, UINTPTR_MAX, sizeof(long) == 8 ? (uintptr_t) 0xffffffffffffffffULL : (uintptr_t) 0xffffffffU); break;
-		CASE_TEST(limit_ptrdiff_min);       EXPECT_EQ(1, PTRDIFF_MIN, sizeof(long) == 8 ? (ptrdiff_t) 0x8000000000000000LL  : (ptrdiff_t) 0x80000000); break;
-		CASE_TEST(limit_ptrdiff_max);       EXPECT_EQ(1, PTRDIFF_MAX, sizeof(long) == 8 ? (ptrdiff_t) 0x7fffffffffffffffLL  : (ptrdiff_t) 0x7fffffff); break;
-		CASE_TEST(limit_size_max);          EXPECT_EQ(1, SIZE_MAX,    sizeof(long) == 8 ? (size_t)    0xffffffffffffffffULL : (size_t)    0xffffffffU); break;
-		CASE_TEST(strtol_simple);           EXPECT_STRTOX(1, strtol, "35", 10, 35, -1, 0); break;
-		CASE_TEST(strtol_positive);         EXPECT_STRTOX(1, strtol, "+35", 10, 35, -1, 0); break;
-		CASE_TEST(strtol_negative);         EXPECT_STRTOX(1, strtol, "-35", 10, -35, -1, 0); break;
-		CASE_TEST(strtol_hex_auto);         EXPECT_STRTOX(1, strtol, "0xFF", 0, 255, -1, 0); break;
-		CASE_TEST(strtol_base36);           EXPECT_STRTOX(1, strtol, "12yZ", 36, 50507, -1, 0); break;
-		CASE_TEST(strtol_cutoff);           EXPECT_STRTOX(1, strtol, "1234567890", 8, 342391, 7, 0); break;
-		CASE_TEST(strtol_octal_auto);       EXPECT_STRTOX(1, strtol, "011", 0, 9, -1, 0); break;
-		CASE_TEST(strtol_hex_00);           EXPECT_STRTOX(1, strtol, "0x00", 16, 0, -1, 0); break;
-		CASE_TEST(strtol_hex_FF);           EXPECT_STRTOX(1, strtol, "FF", 16, 255, -1, 0); break;
-		CASE_TEST(strtol_hex_ff);           EXPECT_STRTOX(1, strtol, "ff", 16, 255, -1, 0); break;
-		CASE_TEST(strtol_hex_prefix);       EXPECT_STRTOX(1, strtol, "0xFF", 16, 255, -1, 0); break;
-		CASE_TEST(strtol_trailer);          EXPECT_STRTOX(1, strtol, "35foo", 10, 35, 2, 0); break;
-		CASE_TEST(strtol_overflow);         EXPECT_STRTOX(1, strtol, "0x8000000000000000", 16, LONG_MAX, -1, ERANGE); break;
-		CASE_TEST(strtol_underflow);        EXPECT_STRTOX(1, strtol, "-0x8000000000000001", 16, LONG_MIN, -1, ERANGE); break;
-		CASE_TEST(strtoul_negative);        EXPECT_STRTOX(1, strtoul, "-0x1", 16, ULONG_MAX, 4, 0); break;
-		CASE_TEST(strtoul_overflow);        EXPECT_STRTOX(1, strtoul, "0x10000000000000000", 16, ULONG_MAX, -1, ERANGE); break;
-		CASE_TEST(strerror_success);        EXPECT_STREQ(is_nolibc, strerror(0), "errno=0"); break;
-		CASE_TEST(strerror_EINVAL);         EXPECT_STREQ(is_nolibc, strerror(EINVAL), "errno=22"); break;
-		CASE_TEST(strerror_int_max);        EXPECT_STREQ(is_nolibc, strerror(INT_MAX), "errno=2147483647"); break;
-		CASE_TEST(strerror_int_min);        EXPECT_STREQ(is_nolibc, strerror(INT_MIN), "errno=-2147483648"); break;
-		CASE_TEST(tolower);                 EXPECT_EQ(1, tolower('A'), 'a'); break;
-		CASE_TEST(tolower_noop);            EXPECT_EQ(1, tolower('a'), 'a'); break;
-		CASE_TEST(toupper);                 EXPECT_EQ(1, toupper('a'), 'A'); break;
-		CASE_TEST(toupper_noop);            EXPECT_EQ(1, toupper('A'), 'A'); break;
-
-		case __LINE__:
-			return ret; /* must be last */
-		/* note: do not set any defaults so as to permit holes above */
-		}
-	}
-	return ret;
-}
-
-#define EXPECT_VFPRINTF(c, expected, fmt, ...)				\
-	ret += expect_vfprintf(llen, c, expected, fmt, ##__VA_ARGS__)
-
-static int expect_vfprintf(int llen, int c, const char *expected, const char *fmt, ...)
-{
-	char buf[100];
-	va_list args;
-	ssize_t w;
-	int ret;
-
-
-	va_start(args, fmt);
-	/* Only allow writing 21 bytes, to test truncation */
-	w = vsnprintf(buf, 21, fmt, args);
-	va_end(args);
-
-	if (w != c) {
-		llen += printf(" written(%d) != %d", (int)w, c);
-		result(llen, FAIL);
-		return 1;
-	}
-
-	llen += printf(" \"%s\" = \"%s\"", expected, buf);
-	ret = strncmp(expected, buf, c);
-
-	result(llen, ret ? FAIL : OK);
-	return ret;
-}
-
-static int test_scanf(void)
-{
-	unsigned long long ull;
-	unsigned long ul;
-	unsigned int u;
-	long long ll;
-	long l;
-	void *p;
-	int i;
-
-	/* return __LINE__ to point to the specific failure */
-
-	/* test EOF */
-	if (sscanf("", "foo") != EOF)
-		return __LINE__;
-
-	/* test simple literal without placeholder */
-	if (sscanf("foo", "foo") != 0)
-		return __LINE__;
-
-	/* test single placeholder */
-	if (sscanf("123", "%d", &i) != 1)
-		return __LINE__;
-
-	if (i != 123)
-		return __LINE__;
-
-	/* test multiple place holders and separators */
-	if (sscanf("a123b456c0x90", "a%db%uc%p", &i, &u, &p) != 3)
-		return __LINE__;
-
-	if (i != 123)
-		return __LINE__;
-
-	if (u != 456)
-		return __LINE__;
-
-	if (p != (void *)0x90)
-		return __LINE__;
-
-	/* test space handling */
-	if (sscanf("a    b1", "a b%d", &i) != 1)
-		return __LINE__;
-
-	if (i != 1)
-		return __LINE__;
-
-	/* test literal percent */
-	if (sscanf("a%1", "a%%%d", &i) != 1)
-		return __LINE__;
-
-	if (i != 1)
-		return __LINE__;
-
-	/* test stdint.h types */
-	if (sscanf("1|2|3|4|5|6",
-		   "%d|%ld|%lld|%u|%lu|%llu",
-		   &i, &l, &ll, &u, &ul, &ull) != 6)
-		return __LINE__;
-
-	if (i != 1 || l != 2 || ll != 3 ||
-	    u != 4 || ul != 5 || ull != 6)
-		return __LINE__;
-
-	return 0;
-}
-
-static int run_printf(int min, int max)
-{
-	int test;
-	int ret = 0;
-
-	for (test = min; test >= 0 && test <= max; test++) {
-		int llen = 0; /* line length */
-
-		/* avoid leaving empty lines below, this will insert holes into
-		 * test numbers.
-		 */
-		switch (test + __LINE__ + 1) {
-		CASE_TEST(empty);        EXPECT_VFPRINTF(0, "", ""); break;
-		CASE_TEST(simple);       EXPECT_VFPRINTF(3, "foo", "foo"); break;
-		CASE_TEST(string);       EXPECT_VFPRINTF(3, "foo", "%s", "foo"); break;
-		CASE_TEST(number);       EXPECT_VFPRINTF(4, "1234", "%d", 1234); break;
-		CASE_TEST(negnumber);    EXPECT_VFPRINTF(5, "-1234", "%d", -1234); break;
-		CASE_TEST(unsigned);     EXPECT_VFPRINTF(5, "12345", "%u", 12345); break;
-		CASE_TEST(char);         EXPECT_VFPRINTF(1, "c", "%c", 'c'); break;
-		CASE_TEST(hex);          EXPECT_VFPRINTF(1, "f", "%x", 0xf); break;
-		CASE_TEST(pointer);      EXPECT_VFPRINTF(3, "0x1", "%p", (void *) 0x1); break;
-		CASE_TEST(uintmax_t);    EXPECT_VFPRINTF(20, "18446744073709551615", "%ju", 0xffffffffffffffffULL); break;
-		CASE_TEST(intmax_t);     EXPECT_VFPRINTF(20, "-9223372036854775807", "%jd", 0x8000000000000001LL); break;
-		CASE_TEST(truncation);   EXPECT_VFPRINTF(25, "01234567890123456789", "%s", "0123456789012345678901234"); break;
-		CASE_TEST(string_width); EXPECT_VFPRINTF(10, "         1", "%10s", "1"); break;
-		CASE_TEST(number_width); EXPECT_VFPRINTF(10, "         1", "%10d", 1); break;
-		CASE_TEST(width_trunc);  EXPECT_VFPRINTF(25, "                    ", "%25d", 1); break;
-		CASE_TEST(scanf);        EXPECT_ZR(1, test_scanf()); break;
-		case __LINE__:
-			return ret; /* must be last */
-		/* note: do not set any defaults so as to permit holes above */
-		}
-	}
-	return ret;
-}
-
-__attribute__((no_sanitize("undefined")))
-static int smash_stack(void)
-{
-	char buf[100];
-	volatile char *ptr = buf;
-	size_t i;
-
-	for (i = 0; i < 200; i++)
-		ptr[i] = 'P';
-
-	return 1;
-}
-
-static int run_protection(int min __attribute__((unused)),
-			  int max __attribute__((unused)))
-{
-	pid_t pid;
-	int llen = 0, status;
-	struct rlimit rlimit = { 0, 0 };
-
-	llen += printf("0 -fstackprotector ");
-
-#if !defined(_NOLIBC_STACKPROTECTOR)
-	llen += printf("not supported");
-	result(llen, SKIPPED);
-	return 0;
-#endif
-
-#if defined(_NOLIBC_STACKPROTECTOR)
-	if (!__stack_chk_guard) {
-		llen += printf("__stack_chk_guard not initialized");
-		result(llen, FAIL);
-		return 1;
-	}
-#endif
-
-	pid = -1;
-	pid = fork();
-
-	switch (pid) {
-	case -1:
-		llen += printf("fork()");
-		result(llen, FAIL);
-		return 1;
-
-	case 0:
-		close(STDOUT_FILENO);
-		close(STDERR_FILENO);
-
-		prctl(PR_SET_DUMPABLE, 0, 0, 0, 0);
-		setrlimit(RLIMIT_CORE, &rlimit);
-		smash_stack();
-		return 1;
-
-	default:
-		pid = waitpid(pid, &status, 0);
-
-		if (pid == -1 || !WIFSIGNALED(status) || WTERMSIG(status) != SIGABRT) {
-			llen += printf("waitpid()");
-			result(llen, FAIL);
-			return 1;
-		}
-		result(llen, OK);
-		return 0;
-	}
-}
-
-/* prepare what needs to be prepared for pid 1 (stdio, /dev, /proc, etc) */
-int prepare(void)
-{
-	struct stat stat_buf;
-
-	/* It's possible that /dev doesn't even exist or was not mounted, so
-	 * we'll try to create it, mount it, or create minimal entries into it.
-	 * We want at least /dev/null and /dev/console.
-	 */
-	if (stat("/dev/.", &stat_buf) == 0 || mkdir("/dev", 0755) == 0) {
-		if (stat("/dev/console", &stat_buf) != 0 ||
-		    stat("/dev/null", &stat_buf) != 0 ||
-		    stat("/dev/zero", &stat_buf) != 0) {
-			/* try devtmpfs first, otherwise fall back to manual creation */
-			if (mount("/dev", "/dev", "devtmpfs", 0, 0) != 0) {
-				mknod("/dev/console", 0600 | S_IFCHR, makedev(5, 1));
-				mknod("/dev/null",    0666 | S_IFCHR, makedev(1, 3));
-				mknod("/dev/zero",    0666 | S_IFCHR, makedev(1, 5));
-			}
-		}
-	}
-
-	/* If no /dev/console was found before calling init, stdio is closed so
-	 * we need to reopen it from /dev/console. If it failed above, it will
-	 * still fail here and we cannot emit a message anyway.
-	 */
-	if (close(dup(1)) == -1) {
-		int fd = open("/dev/console", O_RDWR);
-
-		if (fd >= 0) {
-			if (fd != 0)
-				dup2(fd, 0);
-			if (fd != 1)
-				dup2(fd, 1);
-			if (fd != 2)
-				dup2(fd, 2);
-			if (fd > 2)
-				close(fd);
-			puts("\nSuccessfully reopened /dev/console.");
-		}
-	}
-
-	/* try to mount /proc if not mounted. Silently fail otherwise */
-	if (stat("/proc/.", &stat_buf) == 0 || mkdir("/proc", 0755) == 0) {
-		if (stat("/proc/self", &stat_buf) != 0) {
-			/* If not mountable, remove /proc completely to avoid misuse */
-			if (mount("none", "/proc", "proc", 0, 0) != 0)
-				rmdir("/proc");
-		}
-	}
-
-	/* some tests rely on a writable /tmp */
-	mkdir("/tmp", 0755);
-
-	return 0;
-}
-
-/* This is the definition of known test names, with their functions */
-static const struct test test_names[] = {
-	/* add new tests here */
-	{ .name = "startup",    .func = run_startup    },
-	{ .name = "syscall",    .func = run_syscall    },
-	{ .name = "stdlib",     .func = run_stdlib     },
-	{ .name = "printf",     .func = run_printf     },
-	{ .name = "protection", .func = run_protection },
-	{ 0 }
-};
-
-static int is_setting_valid(char *test)
-{
-	int idx, len, test_len, valid = 0;
-	char delimiter;
-
-	if (!test)
-		return valid;
-
-	test_len = strlen(test);
-
-	for (idx = 0; test_names[idx].name; idx++) {
-		len = strlen(test_names[idx].name);
-		if (test_len < len)
-			continue;
-
-		if (strncmp(test, test_names[idx].name, len) != 0)
-			continue;
-
-		delimiter = test[len];
-		if (delimiter != ':' && delimiter != ',' && delimiter != '\0')
-			continue;
-
-		valid = 1;
-		break;
-	}
-
-	return valid;
-}
-
-int main(int argc, char **argv, char **envp)
-{
-	int min = 0;
-	int max = INT_MAX;
-	int ret = 0;
-	int err;
-	int idx;
-	char *test;
-
-	argv0 = argv[0];
-	test_argc = argc;
-	test_argv = argv;
-	test_envp = envp;
-
-	/* when called as init, it's possible that no console was opened, for
-	 * example if no /dev file system was provided. We'll check that fd#1
-	 * was opened, and if not we'll attempt to create and open /dev/console
-	 * and /dev/null that we'll use for later tests.
-	 */
-	if (getpid() == 1)
-		prepare();
-
-	/* the definition of a series of tests comes from either argv[1] or the
-	 * "NOLIBC_TEST" environment variable. It's made of a comma-delimited
-	 * series of test names and optional ranges:
-	 *    syscall:5-15[:.*],stdlib:8-10
-	 */
-	test = argv[1];
-	if (!is_setting_valid(test))
-		test = getenv("NOLIBC_TEST");
-
-	if (is_setting_valid(test)) {
-		char *comma, *colon, *dash, *value;
-
-		do {
-			comma = strchr(test, ',');
-			if (comma)
-				*(comma++) = '\0';
-
-			colon = strchr(test, ':');
-			if (colon)
-				*(colon++) = '\0';
-
-			for (idx = 0; test_names[idx].name; idx++) {
-				if (strcmp(test, test_names[idx].name) == 0)
-					break;
-			}
-
-			if (test_names[idx].name) {
-				/* The test was named, it will be called at least
-				 * once. We may have an optional range at <colon>
-				 * here, which defaults to the full range.
-				 */
-				do {
-					min = 0; max = INT_MAX;
-					value = colon;
-					if (value && *value) {
-						colon = strchr(value, ':');
-						if (colon)
-							*(colon++) = '\0';
-
-						dash = strchr(value, '-');
-						if (dash)
-							*(dash++) = '\0';
-
-						/* support :val: :min-max: :min-: :-max: */
-						if (*value)
-							min = atoi(value);
-						if (!dash)
-							max = min;
-						else if (*dash)
-							max = atoi(dash);
-
-						value = colon;
-					}
-
-					/* now's time to call the test */
-					printf("Running test '%s'\n", test_names[idx].name);
-					err = test_names[idx].func(min, max);
-					ret += err;
-					printf("Errors during this test: %d\n\n", err);
-				} while (colon && *colon);
-			} else
-				printf("Ignoring unknown test name '%s'\n", test);
-
-			test = comma;
-		} while (test && *test);
-	} else {
-		/* no test mentioned, run everything */
-		for (idx = 0; test_names[idx].name; idx++) {
-			printf("Running test '%s'\n", test_names[idx].name);
-			err = test_names[idx].func(min, max);
-			ret += err;
-			printf("Errors during this test: %d\n\n", err);
-		}
-	}
-
-	printf("Total number of errors: %d\n", ret);
-
-	if (getpid() == 1) {
-		/* we're running as init, there's no other process on the
-		 * system, thus likely started from a VM for a quick check.
-		 * Exiting will provoke a kernel panic that may be reported
-		 * as an error by Qemu or the hypervisor, while stopping
-		 * cleanly will often be reported as a success. This allows
-		 * to use the output of this program for bisecting kernels.
-		 */
-		printf("Leaving init with final status: %d\n", !!ret);
-		if (ret == 0)
-			reboot(RB_POWER_OFF);
-#if defined(__x86_64__)
-		/* QEMU started with "-device isa-debug-exit -no-reboot" will
-		 * exit with status code 2N+1 when N is written to 0x501. We
-		 * hard-code the syscall here as it's arch-dependent.
-		 */
-		else if (syscall(__NR_ioperm, 0x501, 1, 1) == 0)
-			__asm__ volatile ("outb %%al, %%dx" :: "d"(0x501), "a"(0));
-		/* if it does nothing, fall back to the regular panic */
-#endif
-	}
-
-	printf("Exiting with status %d\n", !!ret);
-	return !!ret;
-}
+#include "harness-selftest.c"
diff --git a/tools/testing/selftests/nolibc/run-tests.sh b/tools/testing/selftests/nolibc/run-tests.sh
index 8277599e6441a933d9c1ec5003acf49b06df226f..8ac447e8547c2aa67615a4dc8a37fa4d853ce92a 100755
--- a/tools/testing/selftests/nolibc/run-tests.sh
+++ b/tools/testing/selftests/nolibc/run-tests.sh
@@ -196,7 +196,7 @@ test_arch() {
 	swallow_output "${MAKE[@]}" defconfig
 	swallow_output "${MAKE[@]}" CFLAGS_EXTRA="$CFLAGS_EXTRA" "$test_target" V=1
 	cp run.out run.out."${arch}"
-	"${MAKE[@]}" report | grep passed
+	"${MAKE[@]}" report
 }
 
 if [ "$perform_download" -ne 0 ]; then

-- 
2.49.0

Re: [PATCH v4 14/14] HACK: selftests/nolibc: demonstrate usage of the kselftest harness

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 668 bytes --]

On Mon, May 05, 2025 at 05:15:32PM +0200, Thomas Weißschuh wrote:

> Show how to use the kselftest harness together with nolibc.
> This just runs the existing harness selftest by crudely replacing the
> regular nolibc-test.c with the harness-selftest.c to get that wired up easily.
> To use it:
> 	$ cd tools/testing/selftests/nolibc/
> 	$ ./run-tests -m user
> 
> In the future nolibc-test can use the harness for itself.

FWIW there's also several arm64 tests which are written using nolibc
which could usefully be converted to this, they're using nolibc because
they test interfaces intended to be used by libc and so want to avoid
conflicting uses.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v4 00/14] kselftest harness and nolibc compatibility

[Thomas Weißschuh]

Hi Shuah and Kees,

On 2025-05-05 17:15:18+0200, Thomas Weißschuh wrote:
> Nolibc is useful for selftests as the test programs can be very small,
> and compiled with just a kernel crosscompiler, without userspace support.
> Currently nolibc is only usable with kselftest.h, not the more
> convenient to use kselftest_harness.h
> This series provides this compatibility by removing the usage of problematic
> libc features from the harness.

I'd like to get this series into the next merge window.
For that I'd like to expose it to linux-next through the nolibc tree.
If you don't have the time for a review or issues crop up, I will drop
the patches again.

Are you fine with that?

The issues reported by Mark have been fixed and tests have been written
for them.

> Based on nolibc/for-next.
> The series is meant to be merged through the nolibc tree.
> 
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> ---
> Changes in v4:
> - Drop patches for nolibc which where already applied
> - Preserve signatures of test functions for tests making assumptions about them
>   drop 'selftests: harness: Always provide "self" and "variant"'
>   add 'selftests: harness: Add "variant" and "self" to test metadata'
>   adapt 'selftests: harness: Stop using setjmp()/longjmp()'
> - Validate test function signatures in harness selftest
> - Link to v3: https://lore.kernel.org/r/20250411-nolibc-kselftest-harness-v3-0-4d9c0295893f@linutronix.de
> 
> Changes in v3:
> - Send patches to correct kselftest harness maintainers
> - Move harness selftest to dedicated directory
> - Add harness selftest to MAINTAINERS
> - Integrate harness selftest cleanup with the selftest framework
> - Consistently use "kselftest harness" in commit messages
> - Properly propagate kselftest harness failure
> - Link to v2: https://lore.kernel.org/r/20250407-nolibc-kselftest-harness-v2-0-f8812f76e930@linutronix.de
> 
> Changes in v2:
> - Rebase unto v6.15-rc1
> - Rename internal nolibc symbols
> - Handle edge case of waitpid(INT_MIN) == ESRCH
> - Fix arm configurations for final testing patch
> - Clean up global getopt.h variable declarations
> - Add Acks from Willy
> - Link to v1: https://lore.kernel.org/r/20250304-nolibc-kselftest-harness-v1-0-adca7cd231e2@linutronix.de
> 
> ---
> Thomas Weißschuh (14):
>       selftests: harness: Add kselftest harness selftest
>       selftests: harness: Use C89 comment style
>       selftests: harness: Ignore unused variant argument warning
>       selftests: harness: Mark functions without prototypes static
>       selftests: harness: Remove inline qualifier for wrappers
>       selftests: harness: Remove dependency on libatomic
>       selftests: harness: Implement test timeouts through pidfd
>       selftests: harness: Don't set setup_completed for fixtureless tests
>       selftests: harness: Move teardown conditional into test metadata
>       selftests: harness: Add teardown callback to test metadata
>       selftests: harness: Add "variant" and "self" to test metadata
>       selftests: harness: Stop using setjmp()/longjmp()
>       selftests: harness: Guard includes on nolibc
>       HACK: selftests/nolibc: demonstrate usage of the kselftest harness
> 
>  MAINTAINERS                                        |    1 +
>  tools/testing/selftests/Makefile                   |    1 +
>  tools/testing/selftests/kselftest_harness.h        |  175 +-
>  .../testing/selftests/kselftest_harness/.gitignore |    2 +
>  tools/testing/selftests/kselftest_harness/Makefile |    7 +
>  .../selftests/kselftest_harness/harness-selftest.c |  138 ++
>  .../kselftest_harness/harness-selftest.expected    |   64 +
>  .../kselftest_harness/harness-selftest.sh          |   13 +
>  tools/testing/selftests/nolibc/Makefile            |   15 +-
>  tools/testing/selftests/nolibc/harness-selftest.c  |    1 +
>  tools/testing/selftests/nolibc/nolibc-test.c       | 1715 +-------------------
>  tools/testing/selftests/nolibc/run-tests.sh        |    2 +-
>  12 files changed, 313 insertions(+), 1821 deletions(-)
> ---
> base-commit: 2051d3b830c0889ae55e37e9e8ff0d43a4acd482
> change-id: 20250130-nolibc-kselftest-harness-8b2c8cac43bf
> 
> Best regards,
> -- 
> Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> 

Re: [PATCH v4 00/14] kselftest harness and nolibc compatibility

[Shuah Khan]

On 5/10/25 00:54, Thomas Weißschuh wrote:
> Hi Shuah and Kees,
> 
> On 2025-05-05 17:15:18+0200, Thomas Weißschuh wrote:
>> Nolibc is useful for selftests as the test programs can be very small,
>> and compiled with just a kernel crosscompiler, without userspace support.
>> Currently nolibc is only usable with kselftest.h, not the more
>> convenient to use kselftest_harness.h
>> This series provides this compatibility by removing the usage of problematic
>> libc features from the harness.
> 
> I'd like to get this series into the next merge window.
> For that I'd like to expose it to linux-next through the nolibc tree.
> If you don't have the time for a review or issues crop up, I will drop
> the patches again.
> 
> Are you fine with that?

Didn't I respond to v13 saying you can include in your nolibc PR?
If I didn't here is my Reviewed-by.

> 
> The issues reported by Mark have been fixed and tests have been written
> for them.

Reviewed-by: Shuah Khan <skhan@linuxfoundation.org>

thanks,
-- Shuah

Re: [PATCH v4 00/14] kselftest harness and nolibc compatibility

[Thomas Weißschuh]

On 2025-05-12 15:32:40-0600, Shuah Khan wrote:
> On 5/10/25 00:54, Thomas Weißschuh wrote:
> > Hi Shuah and Kees,
> > 
> > On 2025-05-05 17:15:18+0200, Thomas Weißschuh wrote:
> > > Nolibc is useful for selftests as the test programs can be very small,
> > > and compiled with just a kernel crosscompiler, without userspace support.
> > > Currently nolibc is only usable with kselftest.h, not the more
> > > convenient to use kselftest_harness.h
> > > This series provides this compatibility by removing the usage of problematic
> > > libc features from the harness.
> > 
> > I'd like to get this series into the next merge window.
> > For that I'd like to expose it to linux-next through the nolibc tree.
> > If you don't have the time for a review or issues crop up, I will drop
> > the patches again.
> > 
> > Are you fine with that?
> 
> Didn't I respond to v13 saying you can include in your nolibc PR?
> If I didn't here is my Reviewed-by.

You did ack v3. But as some patches changed, I wanted to reconfirm the
new contents.

> > 
> > The issues reported by Mark have been fixed and tests have been written
> > for them.
> 
> Reviewed-by: Shuah Khan <skhan@linuxfoundation.org>

Thanks!
As most patches already carry your Ack, I transformed this Reviewed-by
into an Acked-by for the two new patches for consistency.


Thomas