[PATCH] selftests/harness: harness-selftest: cover XFAIL_ADD ctor ordering

[PATCH] selftests/harness: harness-selftest: cover XFAIL_ADD ctor ordering

[Sun Jian]

Add a fixture-based test with a single variant and mark it with
XFAIL_ADD(). The test intentionally fails and is expected to be
accounted as XFAIL, covering the constructor ordering dependency
between TEST_F() registration and XFAIL_ADD() registration.

Suggested-by: Thomas Weißschuh <thomas@t-8ch.de>
Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
---
 .../kselftest_harness/harness-selftest.c      | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/tools/testing/selftests/kselftest_harness/harness-selftest.c b/tools/testing/selftests/kselftest_harness/harness-selftest.c
index 7820bb5d0e6d..9920ef7e4385 100644
--- a/tools/testing/selftests/kselftest_harness/harness-selftest.c
+++ b/tools/testing/selftests/kselftest_harness/harness-selftest.c
@@ -118,6 +118,29 @@ TEST_F(fixture_setup_failure, pass) {
 	TH_LOG("after");
 }
 
+FIXTURE(xfail_ctor_ordering) {
+};
+
+FIXTURE_SETUP(xfail_ctor_ordering) {
+}
+
+FIXTURE_TEARDOWN(xfail_ctor_ordering) {
+}
+
+FIXTURE_VARIANT(xfail_ctor_ordering) {
+	int dummy;
+};
+
+FIXTURE_VARIANT_ADD(xfail_ctor_ordering, only) {
+	.dummy = 0,
+};
+
+TEST_F(xfail_ctor_ordering, expected_fail) {
+	ASSERT_EQ(1, 0);
+}
+
+XFAIL_ADD(xfail_ctor_ordering, only, expected_fail);
+
 int main(int argc, char **argv)
 {
 	/*

base-commit: 1f318b96cc84d7c2ab792fcc0bfd42a7ca890681
-- 
2.43.0

Re: [PATCH] selftests/harness: harness-selftest: cover XFAIL_ADD ctor ordering

[Thomas Weißschuh]

Hi!

On 2026-03-09 17:37:52+0800, Sun Jian wrote:
> Add a fixture-based test with a single variant and mark it with
> XFAIL_ADD(). The test intentionally fails and is expected to be
> accounted as XFAIL, covering the constructor ordering dependency
> between TEST_F() registration and XFAIL_ADD() registration.
> 
> Suggested-by: Thomas Weißschuh <thomas@t-8ch.de>

I used my wrong email address for this.
Could you switch it to:
Thomas Weißschuh <linux@weissschuh.net>

> Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
> ---
>  .../kselftest_harness/harness-selftest.c      | 23 +++++++++++++++++++

Also update harness-selftest.expected with the new output. To verify
that everything works, run 'make run_tests' from the test directory.

>  1 file changed, 23 insertions(+)

(...)

[PATCH v2] selftests/harness: harness-selftest: cover XFAIL_ADD ctor ordering

[Sun Jian]

Add a fixture-based test with a single variant and mark it with
XFAIL_ADD(). The test intentionally fails and is expected to be
accounted as XFAIL, covering the constructor ordering dependency
between TEST_F() registration and XFAIL_ADD() registration.

Suggested-by: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
---
v2:
- update Suggested-by address
- update harness-selftest.expected
- verify with 'make run_tests'

 .../kselftest_harness/harness-selftest.c      | 23 ++++++++++
 .../harness-selftest.expected                 | 43 +++++++++++--------
 2 files changed, 47 insertions(+), 19 deletions(-)

diff --git a/tools/testing/selftests/kselftest_harness/harness-selftest.c b/tools/testing/selftests/kselftest_harness/harness-selftest.c
index 7820bb5d0e6d..9920ef7e4385 100644
--- a/tools/testing/selftests/kselftest_harness/harness-selftest.c
+++ b/tools/testing/selftests/kselftest_harness/harness-selftest.c
@@ -118,6 +118,29 @@ TEST_F(fixture_setup_failure, pass) {
 	TH_LOG("after");
 }
 
+FIXTURE(xfail_ctor_ordering) {
+};
+
+FIXTURE_SETUP(xfail_ctor_ordering) {
+}
+
+FIXTURE_TEARDOWN(xfail_ctor_ordering) {
+}
+
+FIXTURE_VARIANT(xfail_ctor_ordering) {
+	int dummy;
+};
+
+FIXTURE_VARIANT_ADD(xfail_ctor_ordering, only) {
+	.dummy = 0,
+};
+
+TEST_F(xfail_ctor_ordering, expected_fail) {
+	ASSERT_EQ(1, 0);
+}
+
+XFAIL_ADD(xfail_ctor_ordering, only, expected_fail);
+
 int main(int argc, char **argv)
 {
 	/*
diff --git a/tools/testing/selftests/kselftest_harness/harness-selftest.expected b/tools/testing/selftests/kselftest_harness/harness-selftest.expected
index 97e1418c1c7e..e0088c9e07b2 100644
--- a/tools/testing/selftests/kselftest_harness/harness-selftest.expected
+++ b/tools/testing/selftests/kselftest_harness/harness-selftest.expected
@@ -1,6 +1,6 @@
 TAP version 13
-1..9
-# Starting 9 tests from 4 test cases.
+1..10
+# Starting 10 tests from 5 test cases.
 #  RUN           global.standalone_pass ...
 # harness-selftest.c:19:standalone_pass:before
 # harness-selftest.c:23:standalone_pass:after
@@ -24,15 +24,12 @@ ok 3 global.signal_pass
 # signal_fail: Test terminated by assertion
 #          FAIL  global.signal_fail
 not ok 4 global.signal_fail
-#  RUN           fixture.pass ...
-# harness-selftest.c:53:pass:setup
-# harness-selftest.c:62:pass:before
-# harness-selftest.c:19:pass:before
-# harness-selftest.c:23:pass:after
-# harness-selftest.c:66:pass:after
-# harness-selftest.c:58:pass:teardown same-process=1
-#            OK  fixture.pass
-ok 5 fixture.pass
+#  RUN           fixture.timeout ...
+# harness-selftest.c:53:timeout:setup
+# harness-selftest.c:77:timeout:before
+# timeout: Test terminated by timeout
+#          FAIL  fixture.timeout
+not ok 5 fixture.timeout
 #  RUN           fixture.fail ...
 # harness-selftest.c:53:fail:setup
 # harness-selftest.c:70:fail:before
@@ -41,12 +38,15 @@ ok 5 fixture.pass
 # fail: Test terminated by assertion
 #          FAIL  fixture.fail
 not ok 6 fixture.fail
-#  RUN           fixture.timeout ...
-# harness-selftest.c:53:timeout:setup
-# harness-selftest.c:77:timeout:before
-# timeout: Test terminated by timeout
-#          FAIL  fixture.timeout
-not ok 7 fixture.timeout
+#  RUN           fixture.pass ...
+# harness-selftest.c:53:pass:setup
+# harness-selftest.c:62:pass:before
+# harness-selftest.c:19:pass:before
+# harness-selftest.c:23:pass:after
+# harness-selftest.c:66:pass:after
+# harness-selftest.c:58:pass:teardown same-process=1
+#            OK  fixture.pass
+ok 7 fixture.pass
 #  RUN           fixture_parent.pass ...
 # harness-selftest.c:87:pass:setup
 # harness-selftest.c:96:pass:before
@@ -60,5 +60,10 @@ ok 8 fixture_parent.pass
 # pass: Test terminated by assertion
 #          FAIL  fixture_setup_failure.pass
 not ok 9 fixture_setup_failure.pass
-# FAILED: 4 / 9 tests passed.
-# Totals: pass:4 fail:5 xfail:0 xpass:0 skip:0 error:0
+#  RUN           xfail_ctor_ordering.only.expected_fail ...
+# harness-selftest.c:139:expected_fail:Expected 1 (1) == 0 (0)
+# expected_fail: Test terminated by assertion
+#          FAIL  xfail_ctor_ordering.only.expected_fail
+ok 10 xfail_ctor_ordering.only.expected_fail # XFAIL unknown
+# FAILED: 5 / 10 tests passed.
+# Totals: pass:4 fail:5 xfail:1 xpass:0 skip:0 error:0

base-commit: 1f318b96cc84d7c2ab792fcc0bfd42a7ca890681
-- 
2.43.0

Re: [PATCH v2] selftests/harness: harness-selftest: cover XFAIL_ADD ctor ordering

[Bjoern Doebel]

On Mon, Mar 09, 2026 at 06:10:44PM +0800, Sun Jian wrote:
> Add a fixture-based test with a single variant and mark it with
> XFAIL_ADD(). The test intentionally fails and is expected to be
> accounted as XFAIL, covering the constructor ordering dependency
> between TEST_F() registration and XFAIL_ADD() registration.
> 
> Suggested-by: Thomas Weißschuh <linux@weissschuh.net>
> Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>

Tested-by: Bjoern Doebel <doebel@amazon.de>

Tested on Amazon Linux 2023 kernel 6.18.20 on both x86_64 and arm64.

Best,
Bjoern





Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

Re: [PATCH v2] selftests/harness: harness-selftest: cover XFAIL_ADD ctor ordering

[Thomas Weißschuh]

Hi!

On 2026-03-09 18:10:44+0800, Sun Jian wrote:
> Add a fixture-based test with a single variant and mark it with
> XFAIL_ADD(). The test intentionally fails and is expected to be
> accounted as XFAIL, covering the constructor ordering dependency
> between TEST_F() registration and XFAIL_ADD() registration.
> 
> Suggested-by: Thomas Weißschuh <linux@weissschuh.net>
> Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
> ---
> v2:
> - update Suggested-by address
> - update harness-selftest.expected
> - verify with 'make run_tests'
> 
>  .../kselftest_harness/harness-selftest.c      | 23 ++++++++++
>  .../harness-selftest.expected                 | 43 +++++++++++--------
>  2 files changed, 47 insertions(+), 19 deletions(-)

Can you rebase this on kselftest/fixes?
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/log/?h=fixes

This doesn't apply anymore and I suspect the proposed ordering suffers
from the issue fixed by commit 465b05bae5ac ("selftests: harness:
Restore order of test functions")


Thomas

Re: [PATCH v2] selftests/harness: harness-selftest: cover XFAIL_ADD ctor ordering

[sun jian]

On Thu, Apr 30, 2026 at 12:21 AM Thomas Weißschuh <linux@weissschuh.net> wrote:
>
> Hi!
>
> On 2026-03-09 18:10:44+0800, Sun Jian wrote:
> > Add a fixture-based test with a single variant and mark it with
> > XFAIL_ADD(). The test intentionally fails and is expected to be
> > accounted as XFAIL, covering the constructor ordering dependency
> > between TEST_F() registration and XFAIL_ADD() registration.
> >
> > Suggested-by: Thomas Weißschuh <linux@weissschuh.net>
> > Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
> > ---
> > v2:
> > - update Suggested-by address
> > - update harness-selftest.expected
> > - verify with 'make run_tests'
> >
> >  .../kselftest_harness/harness-selftest.c      | 23 ++++++++++
> >  .../harness-selftest.expected                 | 43 +++++++++++--------
> >  2 files changed, 47 insertions(+), 19 deletions(-)
>
> Can you rebase this on kselftest/fixes?
> https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/log/?h=fixes
>
> This doesn't apply anymore and I suspect the proposed ordering suffers
> from the issue fixed by commit 465b05bae5ac ("selftests: harness:
> Restore order of test functions")
>
>
> Thomas

Hi Thomas,

Thanks, I will send a v3 shortly.

Sun Jian