LinuxPPC-Dev Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/2] kunit: Add support for skipping entire test suites
@ 2026-06-04 16:27 Vaibhav Jain
  2026-06-04 16:28 ` [PATCH 1/2] kunit: Add ability to skip " Vaibhav Jain
  2026-06-04 16:28 ` [PATCH 2/2] kunit: Add example of test suite that can be skipped at runtime Vaibhav Jain
  0 siblings, 2 replies; 7+ messages in thread
From: Vaibhav Jain @ 2026-06-04 16:27 UTC (permalink / raw)
  To: linuxppc-dev, kvm, kvm-ppc, linux-kselftest, kunit-dev,
	linux-kernel
  Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
	Brendan Higgins, David Gow, Rae Moar

This patch series introduces the ability to skip entire 'kunit_suite'
based on runtime conditions, addressing a limitation where test suites
could only skip individual test cases or fail when prerequisites were not
met.

The motivation for this feature comes from test suites that depend on
specific hardware features, kernel capabilities, or runtime conditions.
Currently, such suites must either:
* Fail when prerequisites are missing
* Skip each test case individually with redundant checks
* Implement workarounds to avoid running tests

An example of such a requirement came from [1] where the patch author
wanted to skip the entire 'kunit_suite' but then had to resort marking all
struct 'kunit_case' as skipped by accessing 'kunit_case.status' private
struct member. This usecase being addressed in the patch[1] can be better
implemented with the changes proposed in this patch series.

Structure of the patch series
=============================
PATCH 1:
* Add a 'status' field to struct kunit_suite that allows 'suite_init'
  callbacks to mark the entire suite as KUNIT_SKIPPED.
* Modify the KUnit core to check this newly introduced 'status' field
  and bypass all test cases when a suite is marked as skipped.

Patch 2:
* Providing an example in kunit-example-test.c demonstrating the usage
  pattern.

The implementation is minimal and non-intrusive, adding only a status field
to kunit_suite and checks in two key functions. Test suites that don't use
this proposed feature should be unaffected.

References
==========
[1] https://lore.kernel.org/all/20260604092931.344101-1-vaibhav@linux.ibm.com

Vaibhav Jain (2):
  kunit: Add ability to skip entire test suites
  kunit: Add example of test suite that can be skipped at runtime

 include/kunit/test.h           |  1 +
 lib/kunit/kunit-example-test.c | 29 +++++++++++++++++++++++++++++
 lib/kunit/test.c               | 11 +++++++++++
 3 files changed, 41 insertions(+)

-- 
2.54.0



^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 1/2] kunit: Add ability to skip entire test suites
  2026-06-04 16:27 [PATCH 0/2] kunit: Add support for skipping entire test suites Vaibhav Jain
@ 2026-06-04 16:28 ` Vaibhav Jain
  2026-06-05  5:09   ` David Gow
  2026-06-04 16:28 ` [PATCH 2/2] kunit: Add example of test suite that can be skipped at runtime Vaibhav Jain
  1 sibling, 1 reply; 7+ messages in thread
From: Vaibhav Jain @ 2026-06-04 16:28 UTC (permalink / raw)
  To: linuxppc-dev, kvm, kvm-ppc, linux-kselftest, kunit-dev,
	linux-kernel
  Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
	Brendan Higgins, David Gow, Rae Moar

Currently, KUnit provides mechanisms to skip individual test cases, but
there is no way to skip an entire test suite based on runtime conditions
checked during suite initialization. This limitation forces test suites
to either fail or skip tests individually when certain prerequisites are
not available.

To address this limitation, the patch adds a 'status' field to struct
kunit_suite that allows suite_init callbacks to mark the entire suite as
KUNIT_SKIPPED. When a suite is marked as skipped, all test cases within
that suite are bypassed without execution.

The patch proposes changes to kunit_suite_has_succeeded() to check suite
status before evaluating individual test case results. Also
kunit_run_tests() is updated to skip suite execution if 'kunit_suite.status'
is set to KUNIT_SKIPPED, thats either set before suite_init or by the
suite_init callback itself.

This enables test suites to perform runtime capability checks in their
'suite_init' callback and gracefully skip all tests when prerequisites are
not met, rather than reporting failures or requiring each test case to
perform redundant checks.

Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
 include/kunit/test.h |  1 +
 lib/kunit/test.c     | 11 +++++++++++
 2 files changed, 12 insertions(+)

diff --git a/include/kunit/test.h b/include/kunit/test.h
index ce0573e196ce..395221d623f7 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -285,6 +285,7 @@ struct kunit_suite {
 	struct string_stream *log;
 	int suite_init_err;
 	bool is_init;
+	enum kunit_status status;
 };
 
 /* Stores an array of suites, end points one past the end */
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 99773e000e1b..989acc770265 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -214,6 +214,9 @@ enum kunit_status kunit_suite_has_succeeded(struct kunit_suite *suite)
 	const struct kunit_case *test_case;
 	enum kunit_status status = KUNIT_SKIPPED;
 
+	if (suite->status == KUNIT_SKIPPED)
+		return KUNIT_SKIPPED;
+
 	if (suite->suite_init_err)
 		return KUNIT_FAILURE;
 
@@ -795,12 +798,20 @@ int kunit_run_tests(struct kunit_suite *suite)
 	/* Taint the kernel so we know we've run tests. */
 	add_taint(TAINT_TEST, LOCKDEP_STILL_OK);
 
+	if (suite->status == KUNIT_SKIPPED)
+		goto suite_end;
+
 	if (suite->suite_init) {
 		suite->suite_init_err = suite->suite_init(suite);
 		if (suite->suite_init_err) {
+			suite->status = KUNIT_FAILURE;
 			kunit_err(suite, KUNIT_SUBTEST_INDENT
 				  "# failed to initialize (%d)", suite->suite_init_err);
 			goto suite_end;
+
+		} else if (suite->status == KUNIT_SKIPPED) {
+			/* Skip this kunit suite */
+			goto suite_end;
 		}
 	}
 
-- 
2.54.0



^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 2/2] kunit: Add example of test suite that can be skipped at runtime
  2026-06-04 16:27 [PATCH 0/2] kunit: Add support for skipping entire test suites Vaibhav Jain
  2026-06-04 16:28 ` [PATCH 1/2] kunit: Add ability to skip " Vaibhav Jain
@ 2026-06-04 16:28 ` Vaibhav Jain
  2026-06-05  5:10   ` David Gow
  1 sibling, 1 reply; 7+ messages in thread
From: Vaibhav Jain @ 2026-06-04 16:28 UTC (permalink / raw)
  To: linuxppc-dev, kvm, kvm-ppc, linux-kselftest, kunit-dev,
	linux-kernel
  Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
	Brendan Higgins, David Gow, Rae Moar

Add an example test suite name 'example_test_skip_suite' to
'kunit-example-test.c' that shows how to skip an entire test suite based on
runtime conditions.

The example suite 'example_skip_suite' provides a 'suite_init' callback
named example_skip_suite_init() which marks the entire suite as skipped
using kunit_mark_skipped().

This demonstrates a way for conditionally skipping test suites when any
prerequisites for kunit_suite execution are not met. The 'suite_init'
callback can perform any necessary checks and mark the suite as skipped,
preventing all test cases from executing while also indicating why the
suite was skipped.

Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
 lib/kunit/kunit-example-test.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
index 0bae7b7ca0b0..b8ded54fa46d 100644
--- a/lib/kunit/kunit-example-test.c
+++ b/lib/kunit/kunit-example-test.c
@@ -591,5 +591,34 @@ static struct kunit_suite example_init_test_suite = {
  */
 kunit_test_init_section_suites(&example_init_test_suite);
 
+/*
+ * This test should always be skipped.
+ */
+static void example_skip_suite_test(struct kunit *test)
+{
+	/* This line should never be seen */
+	KUNIT_FAIL(test, "You should not see a this.");
+}
+
+static struct kunit_case  example_skip_suite_test_cases[] = {
+	KUNIT_CASE(example_skip_suite_test),
+	{}
+};
+
+static int example_skip_suite_init(struct kunit_suite *suite)
+{
+	kunit_mark_skipped(suite, "Test suite expected to be skipped");
+	return 0;
+}
+
+static struct kunit_suite example_test_skip_suite = {
+	.name = "example_skip_suite",
+	.suite_init = example_skip_suite_init,
+	.test_cases = example_skip_suite_test_cases,
+};
+
+/* This registers a test suite that will be skipped */
+kunit_test_suite(example_test_skip_suite);
+
 MODULE_DESCRIPTION("Example KUnit test suite");
 MODULE_LICENSE("GPL v2");
-- 
2.54.0



^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 1/2] kunit: Add ability to skip entire test suites
  2026-06-04 16:28 ` [PATCH 1/2] kunit: Add ability to skip " Vaibhav Jain
@ 2026-06-05  5:09   ` David Gow
  2026-06-08  9:04     ` Vaibhav Jain
  0 siblings, 1 reply; 7+ messages in thread
From: David Gow @ 2026-06-05  5:09 UTC (permalink / raw)
  To: Vaibhav Jain, linuxppc-dev, kvm, kvm-ppc, linux-kselftest,
	kunit-dev, linux-kernel
  Cc: Madhavan Srinivasan, Michael Ellerman, Brendan Higgins, Rae Moar

Le 05/06/2026 à 12:28 AM, Vaibhav Jain a écrit :
> Currently, KUnit provides mechanisms to skip individual test cases, but
> there is no way to skip an entire test suite based on runtime conditions
> checked during suite initialization. This limitation forces test suites
> to either fail or skip tests individually when certain prerequisites are
> not available.
> 
> To address this limitation, the patch adds a 'status' field to struct
> kunit_suite that allows suite_init callbacks to mark the entire suite as
> KUNIT_SKIPPED. When a suite is marked as skipped, all test cases within
> that suite are bypassed without execution.
> 
> The patch proposes changes to kunit_suite_has_succeeded() to check suite
> status before evaluating individual test case results. Also
> kunit_run_tests() is updated to skip suite execution if 'kunit_suite.status'
> is set to KUNIT_SKIPPED, thats either set before suite_init or by the
> suite_init callback itself.
> 
> This enables test suites to perform runtime capability checks in their
> 'suite_init' callback and gracefully skip all tests when prerequisites are
> not met, rather than reporting failures or requiring each test case to
> perform redundant checks.
> 
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---

Thanks — this is great! There are a few cases it's not handling
properly, though, particularly with respect to the debugfs support.

I think we need to:
- Reset suite->status to KUNIT_SUCCESS in kunit_init_suite(), so that a
suite which is re-run via debugfs isn't automatically skipped again.
- Fix the result handling in debugfs to handle skipped suites.
- (Optional) Maybe we could get rid of kunit_suite::suite_init_err now
that there's a specific status value. That'd have to be done carefully
to preserve all of the semantics, though.

Here's a quick (and slightly hacky) patch to fix the first couple of issues:
---
diff --git a/lib/kunit/debugfs.c b/lib/kunit/debugfs.c
index 9c326f1837bd..23d34bfdba95 100644
--- a/lib/kunit/debugfs.c
+++ b/lib/kunit/debugfs.c
@@ -58,6 +58,22 @@ static void debugfs_print_result(struct seq_file
*seq, struct string_stream *log
 	spin_unlock(&log->lock);
 }

+/* Print the result line for a suite. */
+static void debugfs_print_ok_not_ok(struct seq_file *seq,
+				    enum kunit_status status,
+				    size_t test_number,
+				    const char *description,
+				    const char *directive)
+{
+	const char *directive_header = (status == KUNIT_SKIPPED) ? " # SKIP "
: "";
+	const char *directive_body = (status == KUNIT_SKIPPED) ? directive : "";
+
+	seq_printf(seq, "%s %zd %s%s%s\n",
+		   kunit_status_to_ok_not_ok(status),
+		   test_number, description, directive_header,
+		   directive_body);
+}
+
 /*
  * /sys/kernel/debug/kunit/<testsuite>/results shows all results for
testsuite.
  */
@@ -77,17 +93,17 @@ static int debugfs_print_results(struct seq_file
*seq, void *v)
 	seq_puts(seq, "1..1\n");

 	/* Print suite header because it is not stored in the test logs. */
-	seq_puts(seq, KUNIT_SUBTEST_INDENT "KTAP version 1\n");
-	seq_printf(seq, KUNIT_SUBTEST_INDENT "# Subtest: %s\n", suite->name);
-	seq_printf(seq, KUNIT_SUBTEST_INDENT "1..%zd\n",
kunit_suite_num_test_cases(suite));
-
-	kunit_suite_for_each_test_case(suite, test_case)
-		debugfs_print_result(seq, test_case->log);
+	if (suite->status != KUNIT_SKIPPED) {
+		seq_puts(seq, KUNIT_SUBTEST_INDENT "KTAP version 1\n");
+		seq_printf(seq, KUNIT_SUBTEST_INDENT "# Subtest: %s\n", suite->name);
+		seq_printf(seq, KUNIT_SUBTEST_INDENT "1..%zd\n",
kunit_suite_num_test_cases(suite));

+		kunit_suite_for_each_test_case(suite, test_case)
+			debugfs_print_result(seq, test_case->log);
+	}
 	debugfs_print_result(seq, suite->log);

-	seq_printf(seq, "%s %d %s\n",
-		   kunit_status_to_ok_not_ok(success), 1, suite->name);
+	debugfs_print_ok_not_ok(seq, success, 1, suite->name,
suite->status_comment);
 	return 0;
 }

diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index c0ae45a22b2c..2ff145796450 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -798,9 +798,6 @@ int kunit_run_tests(struct kunit_suite *suite)
 	/* Taint the kernel so we know we've run tests. */
 	add_taint(TAINT_TEST, LOCKDEP_STILL_OK);

-	if (suite->status == KUNIT_SKIPPED)
-		goto suite_end;
-
 	if (suite->suite_init) {
 		suite->suite_init_err = suite->suite_init(suite);
 		if (suite->suite_init_err) {
@@ -836,6 +833,7 @@ static void kunit_init_suite(struct kunit_suite *suite)
 	kunit_debugfs_create_suite(suite);
 	suite->status_comment[0] = '\0';
 	suite->suite_init_err = 0;
+	suite->status = KUNIT_SUCCESS;

 	if (suite->log)
 		string_stream_clear(suite->log);
---


>  include/kunit/test.h |  1 +
>  lib/kunit/test.c     | 11 +++++++++++
>  2 files changed, 12 insertions(+)
> 
> diff --git a/include/kunit/test.h b/include/kunit/test.h
> index ce0573e196ce..395221d623f7 100644
> --- a/include/kunit/test.h
> +++ b/include/kunit/test.h
> @@ -285,6 +285,7 @@ struct kunit_suite {
>  	struct string_stream *log;
>  	int suite_init_err;
>  	bool is_init;
> +	enum kunit_status status;
>  };
>  
>  /* Stores an array of suites, end points one past the end */
> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> index 99773e000e1b..989acc770265 100644
> --- a/lib/kunit/test.c
> +++ b/lib/kunit/test.c
> @@ -214,6 +214,9 @@ enum kunit_status kunit_suite_has_succeeded(struct kunit_suite *suite)
>  	const struct kunit_case *test_case;
>  	enum kunit_status status = KUNIT_SKIPPED;
>  
> +	if (suite->status == KUNIT_SKIPPED)
> +		return KUNIT_SKIPPED;
> +
>  	if (suite->suite_init_err)
>  		return KUNIT_FAILURE;
>  
> @@ -795,12 +798,20 @@ int kunit_run_tests(struct kunit_suite *suite)
>  	/* Taint the kernel so we know we've run tests. */
>  	add_taint(TAINT_TEST, LOCKDEP_STILL_OK);
>  
> +	if (suite->status == KUNIT_SKIPPED)
> +		goto suite_end;
> +

Do we want this? If a test is run more than once, we probably want to
re-run suite_init so that we can tell if we should still skip it.

While I don't think it's likely that a test which was previously skipped
will suddenly become available, it's not impossible with, e.g., CPU hotplug.

>  	if (suite->suite_init) {
>  		suite->suite_init_err = suite->suite_init(suite);
>  		if (suite->suite_init_err) {
> +			suite->status = KUNIT_FAILURE;
>  			kunit_err(suite, KUNIT_SUBTEST_INDENT
>  				  "# failed to initialize (%d)", suite->suite_init_err);
>  			goto suite_end;
> +
> +		} else if (suite->status == KUNIT_SKIPPED) {
> +			/* Skip this kunit suite */
> +			goto suite_end;
>  		}
>  	}
>  

Cheers,
-- David


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] kunit: Add example of test suite that can be skipped at runtime
  2026-06-04 16:28 ` [PATCH 2/2] kunit: Add example of test suite that can be skipped at runtime Vaibhav Jain
@ 2026-06-05  5:10   ` David Gow
  2026-06-08  8:41     ` Vaibhav Jain
  0 siblings, 1 reply; 7+ messages in thread
From: David Gow @ 2026-06-05  5:10 UTC (permalink / raw)
  To: Vaibhav Jain, linuxppc-dev, kvm, kvm-ppc, linux-kselftest,
	kunit-dev, linux-kernel
  Cc: Madhavan Srinivasan, Michael Ellerman, Brendan Higgins, Rae Moar

Le 05/06/2026 à 12:28 AM, Vaibhav Jain a écrit :
> Add an example test suite name 'example_test_skip_suite' to
> 'kunit-example-test.c' that shows how to skip an entire test suite based on
> runtime conditions.
> 
> The example suite 'example_skip_suite' provides a 'suite_init' callback
> named example_skip_suite_init() which marks the entire suite as skipped
> using kunit_mark_skipped().
> 
> This demonstrates a way for conditionally skipping test suites when any
> prerequisites for kunit_suite execution are not met. The 'suite_init'
> callback can perform any necessary checks and mark the suite as skipped,
> preventing all test cases from executing while also indicating why the
> suite was skipped.
> 
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---

Thanks!

Reviewed-by: David Gow <david@davidgow.net>

Cheers,
-- David

>  lib/kunit/kunit-example-test.c | 29 +++++++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
> 
> diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
> index 0bae7b7ca0b0..b8ded54fa46d 100644
> --- a/lib/kunit/kunit-example-test.c
> +++ b/lib/kunit/kunit-example-test.c
> @@ -591,5 +591,34 @@ static struct kunit_suite example_init_test_suite = {
>   */
>  kunit_test_init_section_suites(&example_init_test_suite);
>  
> +/*
> + * This test should always be skipped.
> + */
> +static void example_skip_suite_test(struct kunit *test)
> +{
> +	/* This line should never be seen */
> +	KUNIT_FAIL(test, "You should not see a this.");
> +}
> +
> +static struct kunit_case  example_skip_suite_test_cases[] = {
> +	KUNIT_CASE(example_skip_suite_test),
> +	{}
> +};
> +
> +static int example_skip_suite_init(struct kunit_suite *suite)
> +{
> +	kunit_mark_skipped(suite, "Test suite expected to be skipped");
> +	return 0;
> +}
> +
> +static struct kunit_suite example_test_skip_suite = {
> +	.name = "example_skip_suite",
> +	.suite_init = example_skip_suite_init,
> +	.test_cases = example_skip_suite_test_cases,
> +};
> +
> +/* This registers a test suite that will be skipped */
> +kunit_test_suite(example_test_skip_suite);
> +
>  MODULE_DESCRIPTION("Example KUnit test suite");
>  MODULE_LICENSE("GPL v2");



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] kunit: Add example of test suite that can be skipped at runtime
  2026-06-05  5:10   ` David Gow
@ 2026-06-08  8:41     ` Vaibhav Jain
  0 siblings, 0 replies; 7+ messages in thread
From: Vaibhav Jain @ 2026-06-08  8:41 UTC (permalink / raw)
  To: David Gow, linuxppc-dev, kvm, kvm-ppc, linux-kselftest, kunit-dev,
	linux-kernel
  Cc: Madhavan Srinivasan, Michael Ellerman, Brendan Higgins, Rae Moar

David Gow <david@davidgow.net> writes:

> Le 05/06/2026 à 12:28 AM, Vaibhav Jain a écrit :
>> Add an example test suite name 'example_test_skip_suite' to
>> 'kunit-example-test.c' that shows how to skip an entire test suite based on
>> runtime conditions.
>> 
>> The example suite 'example_skip_suite' provides a 'suite_init' callback
>> named example_skip_suite_init() which marks the entire suite as skipped
>> using kunit_mark_skipped().
>> 
>> This demonstrates a way for conditionally skipping test suites when any
>> prerequisites for kunit_suite execution are not met. The 'suite_init'
>> callback can perform any necessary checks and mark the suite as skipped,
>> preventing all test cases from executing while also indicating why the
>> suite was skipped.
>> 
>> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
>> ---
>
> Thanks!
>
> Reviewed-by: David Gow <david@davidgow.net>
Awesome, thanks
Will add your RB to v2 of the patch series
>
> Cheers,
> -- David
>
>>  lib/kunit/kunit-example-test.c | 29 +++++++++++++++++++++++++++++
>>  1 file changed, 29 insertions(+)
>> 
>> diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
>> index 0bae7b7ca0b0..b8ded54fa46d 100644
>> --- a/lib/kunit/kunit-example-test.c
>> +++ b/lib/kunit/kunit-example-test.c
>> @@ -591,5 +591,34 @@ static struct kunit_suite example_init_test_suite = {
>>   */
>>  kunit_test_init_section_suites(&example_init_test_suite);
>>  
>> +/*
>> + * This test should always be skipped.
>> + */
>> +static void example_skip_suite_test(struct kunit *test)
>> +{
>> +	/* This line should never be seen */
>> +	KUNIT_FAIL(test, "You should not see a this.");
>> +}
>> +
>> +static struct kunit_case  example_skip_suite_test_cases[] = {
>> +	KUNIT_CASE(example_skip_suite_test),
>> +	{}
>> +};
>> +
>> +static int example_skip_suite_init(struct kunit_suite *suite)
>> +{
>> +	kunit_mark_skipped(suite, "Test suite expected to be skipped");
>> +	return 0;
>> +}
>> +
>> +static struct kunit_suite example_test_skip_suite = {
>> +	.name = "example_skip_suite",
>> +	.suite_init = example_skip_suite_init,
>> +	.test_cases = example_skip_suite_test_cases,
>> +};
>> +
>> +/* This registers a test suite that will be skipped */
>> +kunit_test_suite(example_test_skip_suite);
>> +
>>  MODULE_DESCRIPTION("Example KUnit test suite");
>>  MODULE_LICENSE("GPL v2");
>

-- 
Cheers
~ Vaibhav


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 1/2] kunit: Add ability to skip entire test suites
  2026-06-05  5:09   ` David Gow
@ 2026-06-08  9:04     ` Vaibhav Jain
  0 siblings, 0 replies; 7+ messages in thread
From: Vaibhav Jain @ 2026-06-08  9:04 UTC (permalink / raw)
  To: David Gow, linuxppc-dev, kvm, kvm-ppc, linux-kselftest, kunit-dev,
	linux-kernel
  Cc: Madhavan Srinivasan, Michael Ellerman, Brendan Higgins, Rae Moar

Hi David,

Thanks for looking into this patch. My responses inline below:


David Gow <david@davidgow.net> writes:

<snip>

>
> Thanks — this is great! There are a few cases it's not handling
> properly, though, particularly with respect to the debugfs support.
>
> I think we need to:
> - Reset suite->status to KUNIT_SUCCESS in kunit_init_suite(), so that a
> suite which is re-run via debugfs isn't automatically skipped again.
> - Fix the result handling in debugfs to handle skipped suites.
Agree, and I have addressed this in v2 of the patch using the
code-snippets you have shared.

> - (Optional) Maybe we could get rid of kunit_suite::suite_init_err now
> that there's a specific status value. That'd have to be done carefully
> to preserve all of the semantics, though.
Yes fully agree that kunit_suite::suite_init_err will be redundant after
introduction of kunit_suite::status. I have started looking into this
and will post a different patch{series} for this.

>
> Here's a quick (and slightly hacky) patch to fix the first couple of issues:
> ---
> diff --git a/lib/kunit/debugfs.c b/lib/kunit/debugfs.c
> index 9c326f1837bd..23d34bfdba95 100644
> --- a/lib/kunit/debugfs.c
> +++ b/lib/kunit/debugfs.c
> @@ -58,6 +58,22 @@ static void debugfs_print_result(struct seq_file
> *seq, struct string_stream *log
>  	spin_unlock(&log->lock);
>  }
>
> +/* Print the result line for a suite. */
> +static void debugfs_print_ok_not_ok(struct seq_file *seq,
> +				    enum kunit_status status,
> +				    size_t test_number,
> +				    const char *description,
> +				    const char *directive)
> +{
> +	const char *directive_header = (status == KUNIT_SKIPPED) ? " # SKIP "
> : "";
> +	const char *directive_body = (status == KUNIT_SKIPPED) ? directive : "";
> +
> +	seq_printf(seq, "%s %zd %s%s%s\n",
> +		   kunit_status_to_ok_not_ok(status),
> +		   test_number, description, directive_header,
> +		   directive_body);
> +}
> +
>  /*
>   * /sys/kernel/debug/kunit/<testsuite>/results shows all results for
> testsuite.
>   */
> @@ -77,17 +93,17 @@ static int debugfs_print_results(struct seq_file
> *seq, void *v)
>  	seq_puts(seq, "1..1\n");
>
>  	/* Print suite header because it is not stored in the test logs. */
> -	seq_puts(seq, KUNIT_SUBTEST_INDENT "KTAP version 1\n");
> -	seq_printf(seq, KUNIT_SUBTEST_INDENT "# Subtest: %s\n", suite->name);
> -	seq_printf(seq, KUNIT_SUBTEST_INDENT "1..%zd\n",
> kunit_suite_num_test_cases(suite));
> -
> -	kunit_suite_for_each_test_case(suite, test_case)
> -		debugfs_print_result(seq, test_case->log);
> +	if (suite->status != KUNIT_SKIPPED) {
> +		seq_puts(seq, KUNIT_SUBTEST_INDENT "KTAP version 1\n");
> +		seq_printf(seq, KUNIT_SUBTEST_INDENT "# Subtest: %s\n", suite->name);
> +		seq_printf(seq, KUNIT_SUBTEST_INDENT "1..%zd\n",
> kunit_suite_num_test_cases(suite));
>
> +		kunit_suite_for_each_test_case(suite, test_case)
> +			debugfs_print_result(seq, test_case->log);
> +	}
>  	debugfs_print_result(seq, suite->log);
>
> -	seq_printf(seq, "%s %d %s\n",
> -		   kunit_status_to_ok_not_ok(success), 1, suite->name);
> +	debugfs_print_ok_not_ok(seq, success, 1, suite->name,
> suite->status_comment);
>  	return 0;
>  }
Thanks, have modified these proposed changes a bit for v2

>
> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> index c0ae45a22b2c..2ff145796450 100644
> --- a/lib/kunit/test.c
> +++ b/lib/kunit/test.c
> @@ -798,9 +798,6 @@ int kunit_run_tests(struct kunit_suite *suite)
>  	/* Taint the kernel so we know we've run tests. */
>  	add_taint(TAINT_TEST, LOCKDEP_STILL_OK);
>
> -	if (suite->status == KUNIT_SKIPPED)
> -		goto suite_end;
> -
>  	if (suite->suite_init) {
>  		suite->suite_init_err = suite->suite_init(suite);
>  		if (suite->suite_init_err) {
> @@ -836,6 +833,7 @@ static void kunit_init_suite(struct kunit_suite *suite)
>  	kunit_debugfs_create_suite(suite);
>  	suite->status_comment[0] = '\0';
>  	suite->suite_init_err = 0;
> +	suite->status = KUNIT_SUCCESS;
Thanks, taken this change for v2.

>
>  	if (suite->log)
>  		string_stream_clear(suite->log);
> ---
>
>
>>  include/kunit/test.h |  1 +
>>  lib/kunit/test.c     | 11 +++++++++++
>>  2 files changed, 12 insertions(+)
>> 
>> diff --git a/include/kunit/test.h b/include/kunit/test.h
>> index ce0573e196ce..395221d623f7 100644
>> --- a/include/kunit/test.h
>> +++ b/include/kunit/test.h
>> @@ -285,6 +285,7 @@ struct kunit_suite {
>>  	struct string_stream *log;
>>  	int suite_init_err;
>>  	bool is_init;
>> +	enum kunit_status status;
>>  };
>>  
>>  /* Stores an array of suites, end points one past the end */
>> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
>> index 99773e000e1b..989acc770265 100644
>> --- a/lib/kunit/test.c
>> +++ b/lib/kunit/test.c
>> @@ -214,6 +214,9 @@ enum kunit_status kunit_suite_has_succeeded(struct kunit_suite *suite)
>>  	const struct kunit_case *test_case;
>>  	enum kunit_status status = KUNIT_SKIPPED;
>>  
>> +	if (suite->status == KUNIT_SKIPPED)
>> +		return KUNIT_SKIPPED;
>> +
>>  	if (suite->suite_init_err)
>>  		return KUNIT_FAILURE;
>>  
>> @@ -795,12 +798,20 @@ int kunit_run_tests(struct kunit_suite *suite)
>>  	/* Taint the kernel so we know we've run tests. */
>>  	add_taint(TAINT_TEST, LOCKDEP_STILL_OK);
>>  
>> +	if (suite->status == KUNIT_SKIPPED)
>> +		goto suite_end;
>> +
>
> Do we want this? If a test is run more than once, we probably want to
> re-run suite_init so that we can tell if we should still skip it.
This condition should prevent a 'kunit_suite' from running if its marked
skipped before suite_init() is called. 

>
> While I don't think it's likely that a test which was previously skipped
> will suddenly become available, it's not impossible with, e.g., CPU hotplug.
Agree, I have updated kunit_init_suite() to set 'kunit_suite.status' ==
KUNIT_SUCCESS. This should force the suite_init() to be calledback again
when the suite is re-run.


-- 
Cheers
~ Vaibhav


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-06-08  9:04 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-04 16:27 [PATCH 0/2] kunit: Add support for skipping entire test suites Vaibhav Jain
2026-06-04 16:28 ` [PATCH 1/2] kunit: Add ability to skip " Vaibhav Jain
2026-06-05  5:09   ` David Gow
2026-06-08  9:04     ` Vaibhav Jain
2026-06-04 16:28 ` [PATCH 2/2] kunit: Add example of test suite that can be skipped at runtime Vaibhav Jain
2026-06-05  5:10   ` David Gow
2026-06-08  8:41     ` Vaibhav Jain

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox