* [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework
@ 2025-08-26 9:13 David Gow
2025-08-26 9:13 ` [PATCH v4 1/7] kunit: Add parent kunit for parameterized test context David Gow
` (7 more replies)
0 siblings, 8 replies; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: David Gow, mark.rutland, elver, dvyukov, lucas.demarchi,
thomas.hellstrom, rodrigo.vivi, linux-kselftest, kunit-dev,
kasan-dev, intel-xe, dri-devel, linux-kernel, Stephen Rothwell
Hi all,
This is a new version of Marie's patch series, with a couple of extra
fixes squashed in, notably:
- drm/xe/tests: Fix some additional gen_params signatures
https://lore.kernel.org/linux-kselftest/20250821135447.1618942-1-davidgow@google.com/
- kunit: Only output a test plan if we're using kunit_array_gen_params
https://lore.kernel.org/linux-kselftest/20250821135447.1618942-2-davidgow@google.com/
These should fix the issues found in linux-next here:
https://lore.kernel.org/linux-next/20250818120846.347d64b1@canb.auug.org.au/
These changes only affect patches 3 and 4 of the series, the others are
unchanged from v3.
Thanks, everyone, and sorry for the inconvenience!
Cheers,
-- David
---
Hello!
KUnit offers a parameterized testing framework, where tests can be
run multiple times with different inputs. However, the current
implementation uses the same `struct kunit` for each parameter run.
After each run, the test context gets cleaned up, which creates
the following limitations:
a. There is no way to store resources that are accessible across
the individual parameter runs.
b. It's not possible to pass additional context, besides the previous
parameter (and potentially anything else that is stored in the current
test context), to the parameter generator function.
c. Test users are restricted to using pre-defined static arrays
of parameter objects or generate_params() to define their
parameters. There is no flexibility to make a custom dynamic
array without using generate_params(), which can be complex if
generating the next parameter depends on more than just the single
previous parameter.
This patch series resolves these limitations by:
1. [P 1] Giving each parameterized run its own `struct kunit`. It will
remove the need to manage state, such as resetting the `test->priv`
field or the `test->status_comment` after every parameter run.
2. [P 1] Introducing parameterized test context available to all
parameter runs through the parent pointer of type `struct kunit`.
This context won't be used to execute any test logic, but will
instead be used for storing shared resources. Each parameter run
context will have a reference to that parent instance and thus,
have access to those resources.
3. [P 2] Introducing param_init() and param_exit() functions that can
initialize and exit the parameterized test context. They will run once
before and after the parameterized test. param_init() can be used to add
resources to share between parameter runs, pass parameter arrays, and
any other setup logic. While param_exit() can be used to clean up
resources that were not managed by the parameterized test, and
any other teardown logic.
4. [P 3] Passing the parameterized test context as an additional argument
to generate_params(). This provides generate_params() with more context,
making parameter generation much more flexible. The generate_params()
implementations in the KCSAN and drm/xe tests have been adapted to match
the new function pointer signature.
5. [P 4] Introducing a `params_array` field in `struct kunit`. This will
allow the parameterized test context to have direct storage of the
parameter array, enabling features like using dynamic parameter arrays
or using context beyond just the previous parameter. This will also
enable outputting the KTAP test plan for a parameterized test when the
parameter count is available.
Patches 5 and 6 add examples tests to lib/kunit/kunit-example-test.c to
showcase the new features and patch 7 updates the KUnit documentation
to reflect all the framework changes.
Thank you!
-Marie
---
Changes in v4:
Link to v3 of this patch series:
https://lore.kernel.org/linux-kselftest/20250815103604.3857930-1-marievic@google.com/
- Fixup the signatures of some more gen_params functions in the drm/xe
driver.
- Only print a KTAP test plan if a parameterised test is using the
built-in kunit_array_gen_params generating function, fixing the issues
with generator functions which skip array elements.
Changes in v3:
Link to v2 of this patch series:
https://lore.kernel.org/all/20250811221739.2694336-1-marievic@google.com/
- Added logic for skipping the parameter runs and updating the test statistics
when parameterized test initialization fails.
- Minor changes to the documentation.
- Commit message formatting.
Changes in v2:
Link to v1 of this patch series:
https://lore.kernel.org/all/20250729193647.3410634-1-marievic@google.com/
- Establish parameterized testing terminology:
- "parameterized test" will refer to the group of all runs of a single test
function with different parameters.
- "parameter run" will refer to the execution of the test case function with
a single parameter.
- "parameterized test context" is the `struct kunit` that holds the context
for the entire parameterized test.
- "parameter run context" is the `struct kunit` that holds the context of the
individual parameter run.
- A test is defined to be a parameterized tests if it was registered with a
generator function.
- Make comment edits to reflect the established terminology.
- Require users to manually pass kunit_array_gen_params() to
KUNIT_CASE_PARAM_WITH_INIT() as the generator function, unless they want to
provide their own generator function, if the parameter array was registered
in param_init(). This is to be consistent with the definition of a
parameterized test, i.e. generate_params() is never NULL if it's
a parameterized test.
- Change name of kunit_get_next_param_and_desc() to
kunit_array_gen_params().
- Other minor function name changes such as removing the "__" prefix in front
of internal functions.
- Change signature of get_description() in `struct params_array` to accept
the parameterized test context, as well.
- Output the KTAP test plan for a parameterized test when the parameter count
is available.
- Cover letter was made more concise.
- Edits to the example tests.
- Fix bug of parameterized test init/exit logic being done outside of the
parameterized test check.
- Fix bugs identified by the kernel test robot.
---
Marie Zhussupova (7):
kunit: Add parent kunit for parameterized test context
kunit: Introduce param_init/exit for parameterized test context
management
kunit: Pass parameterized test context to generate_params()
kunit: Enable direct registration of parameter arrays to a KUnit test
kunit: Add example parameterized test with shared resource management
using the Resource API
kunit: Add example parameterized test with direct dynamic parameter
array setup
Documentation: kunit: Document new parameterized test features
Documentation/dev-tools/kunit/usage.rst | 342 +++++++++++++++++++++++-
drivers/gpu/drm/xe/tests/xe_pci.c | 14 +-
drivers/gpu/drm/xe/tests/xe_pci_test.h | 9 +-
include/kunit/test.h | 95 ++++++-
kernel/kcsan/kcsan_test.c | 2 +-
lib/kunit/kunit-example-test.c | 217 +++++++++++++++
lib/kunit/test.c | 94 +++++--
rust/kernel/kunit.rs | 4 +
8 files changed, 740 insertions(+), 37 deletions(-)
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 1/7] kunit: Add parent kunit for parameterized test context
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
@ 2025-08-26 9:13 ` David Gow
2025-08-26 9:13 ` [PATCH v4 2/7] kunit: Introduce param_init/exit for parameterized test context management David Gow
` (6 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: mark.rutland, elver, dvyukov, lucas.demarchi, thomas.hellstrom,
rodrigo.vivi, linux-kselftest, kunit-dev, kasan-dev, intel-xe,
dri-devel, linux-kernel, Stephen Rothwell, David Gow
From: Marie Zhussupova <marievic@google.com>
Currently, KUnit parameterized tests lack a mechanism to share
resources across parameter runs because the same `struct kunit`
instance is cleaned up and reused for each run.
This patch introduces parameterized test context, enabling test
users to share resources between parameter runs. It also allows
setting up resources that need to be available for all parameter
runs only once, which is helpful in cases where setup is expensive.
To establish a parameterized test context, this patch adds a
parent pointer field to `struct kunit`. This allows resources added
to the parent `struct kunit` to be shared and accessible across all
parameter runs.
In kunit_run_tests(), the default `struct kunit` created is now
designated to act as the parameterized test context whenever a test
is parameterized.
Subsequently, a new `struct kunit` is made for each parameter run, and
its parent pointer is set to the `struct kunit` that holds the
parameterized test context.
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Signed-off-by: Marie Zhussupova <marievic@google.com>
Signed-off-by: David Gow <davidgow@google.com>
---
No changes in v4:
v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-2-marievic@google.com/
Changes in v3:
v2: https://lore.kernel.org/all/20250811221739.2694336-2-marievic@google.com/
- Commit message formatting.
Changes in v2:
v1: https://lore.kernel.org/all/20250729193647.3410634-2-marievic@google.com/
- Descriptions of the parent pointer in `struct kunit` were changed to
be more general, as it could be used to share resources not only
between parameter runs but also between test cases in the future.
- When printing parameter descriptions using test.param_index was changed
to param_test.param_index.
- kunit_cleanup(&test) in kunit_run_tests() was moved inside the
parameterized test check.
- The comments and the commit message were changed to reflect the
parameterized testing terminology. See the patch series cover letter
change log for the definitions.
---
include/kunit/test.h | 8 ++++++--
lib/kunit/test.c | 34 ++++++++++++++++++++--------------
2 files changed, 26 insertions(+), 16 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index d958ee53050e..9766403afd56 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -268,14 +268,18 @@ struct kunit_suite_set {
*
* @priv: for user to store arbitrary data. Commonly used to pass data
* created in the init function (see &struct kunit_suite).
+ * @parent: reference to the parent context of type struct kunit that can
+ * be used for storing shared resources.
*
* Used to store information about the current context under which the test
* is running. Most of this data is private and should only be accessed
- * indirectly via public functions; the one exception is @priv which can be
- * used by the test writer to store arbitrary data.
+ * indirectly via public functions; the two exceptions are @priv and @parent
+ * which can be used by the test writer to store arbitrary data and access the
+ * parent context, respectively.
*/
struct kunit {
void *priv;
+ struct kunit *parent;
/* private: internal use only. */
const char *name; /* Read only after initialization! */
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index d2bfa331a2b1..587b5c51db58 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -647,6 +647,7 @@ int kunit_run_tests(struct kunit_suite *suite)
struct kunit_case *test_case;
struct kunit_result_stats suite_stats = { 0 };
struct kunit_result_stats total_stats = { 0 };
+ const void *curr_param;
/* Taint the kernel so we know we've run tests. */
add_taint(TAINT_TEST, LOCKDEP_STILL_OK);
@@ -679,37 +680,42 @@ int kunit_run_tests(struct kunit_suite *suite)
} else {
/* Get initial param. */
param_desc[0] = '\0';
- test.param_value = test_case->generate_params(NULL, param_desc);
+ /* TODO: Make generate_params try-catch */
+ curr_param = test_case->generate_params(NULL, param_desc);
test_case->status = KUNIT_SKIPPED;
kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
"KTAP version 1\n");
kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
"# Subtest: %s", test_case->name);
- while (test.param_value) {
- kunit_run_case_catch_errors(suite, test_case, &test);
+ while (curr_param) {
+ struct kunit param_test = {
+ .param_value = curr_param,
+ .param_index = ++test.param_index,
+ .parent = &test,
+ };
+ kunit_init_test(¶m_test, test_case->name, test_case->log);
+ kunit_run_case_catch_errors(suite, test_case, ¶m_test);
if (param_desc[0] == '\0') {
snprintf(param_desc, sizeof(param_desc),
- "param-%d", test.param_index);
+ "param-%d", param_test.param_index);
}
- kunit_print_ok_not_ok(&test, KUNIT_LEVEL_CASE_PARAM,
- test.status,
- test.param_index + 1,
+ kunit_print_ok_not_ok(¶m_test, KUNIT_LEVEL_CASE_PARAM,
+ param_test.status,
+ param_test.param_index,
param_desc,
- test.status_comment);
+ param_test.status_comment);
- kunit_update_stats(¶m_stats, test.status);
+ kunit_update_stats(¶m_stats, param_test.status);
/* Get next param. */
param_desc[0] = '\0';
- test.param_value = test_case->generate_params(test.param_value, param_desc);
- test.param_index++;
- test.status = KUNIT_SUCCESS;
- test.status_comment[0] = '\0';
- test.priv = NULL;
+ curr_param = test_case->generate_params(curr_param, param_desc);
}
+ /* TODO: Put this kunit_cleanup into a try-catch. */
+ kunit_cleanup(&test);
}
kunit_print_attr((void *)test_case, true, KUNIT_LEVEL_CASE);
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 2/7] kunit: Introduce param_init/exit for parameterized test context management
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
2025-08-26 9:13 ` [PATCH v4 1/7] kunit: Add parent kunit for parameterized test context David Gow
@ 2025-08-26 9:13 ` David Gow
2025-08-26 9:13 ` [PATCH v4 3/7] kunit: Pass parameterized test context to generate_params() David Gow
` (5 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: mark.rutland, elver, dvyukov, lucas.demarchi, thomas.hellstrom,
rodrigo.vivi, linux-kselftest, kunit-dev, kasan-dev, intel-xe,
dri-devel, linux-kernel, Stephen Rothwell, David Gow
From: Marie Zhussupova <marievic@google.com>
Add (*param_init) and (*param_exit) function pointers to
`struct kunit_case`. Users will be able to set them via the new
KUNIT_CASE_PARAM_WITH_INIT() macro.
param_init/exit will be invoked by kunit_run_tests() once before and once
after the parameterized test, respectively. They will receive the
`struct kunit` that holds the parameterized test context; facilitating
init and exit for shared state.
This patch also sets param_init/exit to None in rust/kernel/kunit.rs.
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Marie Zhussupova <marievic@google.com>
Signed-off-by: David Gow <davidgow@google.com>
---
No changes in v4:
v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-3-marievic@google.com/
Changes in v3:
v2: https://lore.kernel.org/all/20250811221739.2694336-3-marievic@google.com/
- kunit_init_parent_param_test() now sets both the `struct kunit_case`
and the `struct kunit` statuses as failed if the parameterized test
init failed. The failure message was also changed to include the failure
code, mirroring the kunit_suite init failure message.
- A check for parameter init failure was added in kunit_run_tests(). So,
if the init failed, the framework will skip the parameter runs and
update the param_test statistics to count that failure.
- Commit message formatting.
Changes in v2:
v1: https://lore.kernel.org/all/20250729193647.3410634-3-marievic@google.com/
- param init/exit were set to None in rust/kernel/kunit.rs to fix the
Rust breakage.
- The name of __kunit_init_parent_test was changed to
kunit_init_parent_param_test and its call was changed to happen only
if the test is parameterized.
- The param_exit call was also moved inside the check for if the test is
parameterized.
- KUNIT_CASE_PARAM_WITH_INIT() macro logic was change to not automatically
set generate_params() to KUnit's built-in generator function. Instead,
the test user will be asked to provide it themselves.
- The comments and the commit message were changed to reflect the
parameterized testing terminology. See the patch series cover letter
change log for the definitions.
---
include/kunit/test.h | 25 +++++++++++++++++++++++++
lib/kunit/test.c | 27 ++++++++++++++++++++++++++-
rust/kernel/kunit.rs | 4 ++++
3 files changed, 55 insertions(+), 1 deletion(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index 9766403afd56..fc8fd55b2dfb 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -92,6 +92,8 @@ struct kunit_attributes {
* @name: the name of the test case.
* @generate_params: the generator function for parameterized tests.
* @attr: the attributes associated with the test
+ * @param_init: The init function to run before a parameterized test.
+ * @param_exit: The exit function to run after a parameterized test.
*
* A test case is a function with the signature,
* ``void (*)(struct kunit *)``
@@ -128,6 +130,8 @@ struct kunit_case {
const char *name;
const void* (*generate_params)(const void *prev, char *desc);
struct kunit_attributes attr;
+ int (*param_init)(struct kunit *test);
+ void (*param_exit)(struct kunit *test);
/* private: internal use only. */
enum kunit_status status;
@@ -218,6 +222,27 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status)
.generate_params = gen_params, \
.attr = attributes, .module_name = KBUILD_MODNAME}
+/**
+ * KUNIT_CASE_PARAM_WITH_INIT - Define a parameterized KUnit test case with custom
+ * param_init() and param_exit() functions.
+ * @test_name: The function implementing the test case.
+ * @gen_params: The function to generate parameters for the test case.
+ * @init: A reference to the param_init() function to run before a parameterized test.
+ * @exit: A reference to the param_exit() function to run after a parameterized test.
+ *
+ * Provides the option to register param_init() and param_exit() functions.
+ * param_init/exit will be passed the parameterized test context and run once
+ * before and once after the parameterized test. The init function can be used
+ * to add resources to share between parameter runs, and any other setup logic.
+ * The exit function can be used to clean up resources that were not managed by
+ * the parameterized test, and any other teardown logic.
+ */
+#define KUNIT_CASE_PARAM_WITH_INIT(test_name, gen_params, init, exit) \
+ { .run_case = test_name, .name = #test_name, \
+ .generate_params = gen_params, \
+ .param_init = init, .param_exit = exit, \
+ .module_name = KBUILD_MODNAME}
+
/**
* struct kunit_suite - describes a related collection of &struct kunit_case
*
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 587b5c51db58..0fe61dec5a96 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -641,6 +641,20 @@ static void kunit_accumulate_stats(struct kunit_result_stats *total,
total->total += add.total;
}
+static void kunit_init_parent_param_test(struct kunit_case *test_case, struct kunit *test)
+{
+ if (test_case->param_init) {
+ int err = test_case->param_init(test);
+
+ if (err) {
+ kunit_err(test_case, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
+ "# failed to initialize parent parameter test (%d)", err);
+ test->status = KUNIT_FAILURE;
+ test_case->status = KUNIT_FAILURE;
+ }
+ }
+}
+
int kunit_run_tests(struct kunit_suite *suite)
{
char param_desc[KUNIT_PARAM_DESC_SIZE];
@@ -678,6 +692,11 @@ int kunit_run_tests(struct kunit_suite *suite)
kunit_run_case_catch_errors(suite, test_case, &test);
kunit_update_stats(¶m_stats, test.status);
} else {
+ kunit_init_parent_param_test(test_case, &test);
+ if (test_case->status == KUNIT_FAILURE) {
+ kunit_update_stats(¶m_stats, test.status);
+ goto test_case_end;
+ }
/* Get initial param. */
param_desc[0] = '\0';
/* TODO: Make generate_params try-catch */
@@ -714,10 +733,16 @@ int kunit_run_tests(struct kunit_suite *suite)
param_desc[0] = '\0';
curr_param = test_case->generate_params(curr_param, param_desc);
}
+ /*
+ * TODO: Put into a try catch. Since we don't need suite->exit
+ * for it we can't reuse kunit_try_run_cleanup for this yet.
+ */
+ if (test_case->param_exit)
+ test_case->param_exit(&test);
/* TODO: Put this kunit_cleanup into a try-catch. */
kunit_cleanup(&test);
}
-
+test_case_end:
kunit_print_attr((void *)test_case, true, KUNIT_LEVEL_CASE);
kunit_print_test_stats(&test, param_stats);
diff --git a/rust/kernel/kunit.rs b/rust/kernel/kunit.rs
index 41efd87595d6..b1c97f8029c7 100644
--- a/rust/kernel/kunit.rs
+++ b/rust/kernel/kunit.rs
@@ -210,6 +210,8 @@ pub const fn kunit_case(
status: kernel::bindings::kunit_status_KUNIT_SUCCESS,
module_name: core::ptr::null_mut(),
log: core::ptr::null_mut(),
+ param_init: None,
+ param_exit: None,
}
}
@@ -229,6 +231,8 @@ pub const fn kunit_case_null() -> kernel::bindings::kunit_case {
status: kernel::bindings::kunit_status_KUNIT_SUCCESS,
module_name: core::ptr::null_mut(),
log: core::ptr::null_mut(),
+ param_init: None,
+ param_exit: None,
}
}
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 3/7] kunit: Pass parameterized test context to generate_params()
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
2025-08-26 9:13 ` [PATCH v4 1/7] kunit: Add parent kunit for parameterized test context David Gow
2025-08-26 9:13 ` [PATCH v4 2/7] kunit: Introduce param_init/exit for parameterized test context management David Gow
@ 2025-08-26 9:13 ` David Gow
2025-08-28 22:19 ` Lucas De Marchi
2025-08-26 9:13 ` [PATCH v4 4/7] kunit: Enable direct registration of parameter arrays to a KUnit test David Gow
` (4 subsequent siblings)
7 siblings, 1 reply; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: mark.rutland, elver, dvyukov, lucas.demarchi, thomas.hellstrom,
rodrigo.vivi, linux-kselftest, kunit-dev, kasan-dev, intel-xe,
dri-devel, linux-kernel, Stephen Rothwell, David Gow
From: Marie Zhussupova <marievic@google.com>
To enable more complex parameterized testing scenarios, the
generate_params() function needs additional context beyond just
the previously generated parameter. This patch modifies the
generate_params() function signature to include an extra
`struct kunit *test` argument, giving test users access to the
parameterized test context when generating parameters.
The `struct kunit *test` argument was added as the first parameter
to the function signature as it aligns with the convention of other
KUnit functions that accept `struct kunit *test` first. This also
mirrors the "this" or "self" reference found in object-oriented
programming languages.
This patch also modifies xe_pci_live_device_gen_param() in xe_pci.c
and nthreads_gen_params() in kcsan_test.c to reflect this signature
change.
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Acked-by: Marco Elver <elver@google.com>
Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Marie Zhussupova <marievic@google.com>
[Catch some additional gen_params signatures in drm/xe/tests --David]
Signed-off-by: David Gow <davidgow@google.com>
---
Changes in v4:
v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-4-marievic@google.com/
- Fixup some additional generate_params signature changes in xe_pci.
- These are also available as a separate patch here:
https://lore.kernel.org/linux-kselftest/20250821135447.1618942-1-davidgow@google.com/
Changes in v3:
v2: https://lore.kernel.org/all/20250811221739.2694336-4-marievic@google.com/
- Commit message formatting.
Changes in v2:
v1: https://lore.kernel.org/all/20250729193647.3410634-4-marievic@google.com/
https://lore.kernel.org/all/20250729193647.3410634-5-marievic@google.com/
https://lore.kernel.org/all/20250729193647.3410634-6-marievic@google.com/
- generate_params signature changes in xe_pci.c and kcsan_test.c were
squashed into a single patch to avoid in-between breakages in the series.
- The comments and the commit message were changed to reflect the
parameterized testing terminology. See the patch series cover letter
change log for the definitions.
---
drivers/gpu/drm/xe/tests/xe_pci.c | 14 +++++++-------
drivers/gpu/drm/xe/tests/xe_pci_test.h | 9 +++++----
include/kunit/test.h | 9 ++++++---
kernel/kcsan/kcsan_test.c | 2 +-
lib/kunit/test.c | 5 +++--
5 files changed, 22 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/drm/xe/tests/xe_pci.c b/drivers/gpu/drm/xe/tests/xe_pci.c
index 9c715e59f030..f707e0a54295 100644
--- a/drivers/gpu/drm/xe/tests/xe_pci.c
+++ b/drivers/gpu/drm/xe/tests/xe_pci.c
@@ -44,9 +44,9 @@ KUNIT_ARRAY_PARAM(pci_id, pciidlist, xe_pci_id_kunit_desc);
*
* Return: pointer to the next parameter or NULL if no more parameters
*/
-const void *xe_pci_graphics_ip_gen_param(const void *prev, char *desc)
+const void *xe_pci_graphics_ip_gen_param(struct kunit *test, const void *prev, char *desc)
{
- return graphics_ip_gen_params(prev, desc);
+ return graphics_ip_gen_params(test, prev, desc);
}
EXPORT_SYMBOL_IF_KUNIT(xe_pci_graphics_ip_gen_param);
@@ -61,9 +61,9 @@ EXPORT_SYMBOL_IF_KUNIT(xe_pci_graphics_ip_gen_param);
*
* Return: pointer to the next parameter or NULL if no more parameters
*/
-const void *xe_pci_media_ip_gen_param(const void *prev, char *desc)
+const void *xe_pci_media_ip_gen_param(struct kunit *test, const void *prev, char *desc)
{
- return media_ip_gen_params(prev, desc);
+ return media_ip_gen_params(test, prev, desc);
}
EXPORT_SYMBOL_IF_KUNIT(xe_pci_media_ip_gen_param);
@@ -78,9 +78,9 @@ EXPORT_SYMBOL_IF_KUNIT(xe_pci_media_ip_gen_param);
*
* Return: pointer to the next parameter or NULL if no more parameters
*/
-const void *xe_pci_id_gen_param(const void *prev, char *desc)
+const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc)
{
- const struct pci_device_id *pci = pci_id_gen_params(prev, desc);
+ const struct pci_device_id *pci = pci_id_gen_params(test, prev, desc);
return pci->driver_data ? pci : NULL;
}
@@ -159,7 +159,7 @@ EXPORT_SYMBOL_IF_KUNIT(xe_pci_fake_device_init);
* Return: pointer to the next &struct xe_device ready to be used as a parameter
* or NULL if there are no more Xe devices on the system.
*/
-const void *xe_pci_live_device_gen_param(const void *prev, char *desc)
+const void *xe_pci_live_device_gen_param(struct kunit *test, const void *prev, char *desc)
{
const struct xe_device *xe = prev;
struct device *dev = xe ? xe->drm.dev : NULL;
diff --git a/drivers/gpu/drm/xe/tests/xe_pci_test.h b/drivers/gpu/drm/xe/tests/xe_pci_test.h
index ce4d2b86b778..6d8bc56f7bde 100644
--- a/drivers/gpu/drm/xe/tests/xe_pci_test.h
+++ b/drivers/gpu/drm/xe/tests/xe_pci_test.h
@@ -7,6 +7,7 @@
#define _XE_PCI_TEST_H_
#include <linux/types.h>
+#include <kunit/test.h>
#include "xe_platform_types.h"
#include "xe_sriov_types.h"
@@ -25,9 +26,9 @@ struct xe_pci_fake_data {
int xe_pci_fake_device_init(struct xe_device *xe);
-const void *xe_pci_graphics_ip_gen_param(const void *prev, char *desc);
-const void *xe_pci_media_ip_gen_param(const void *prev, char *desc);
-const void *xe_pci_id_gen_param(const void *prev, char *desc);
-const void *xe_pci_live_device_gen_param(const void *prev, char *desc);
+const void *xe_pci_graphics_ip_gen_param(struct kunit *test, const void *prev, char *desc);
+const void *xe_pci_media_ip_gen_param(struct kunit *test, const void *prev, char *desc);
+const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc);
+const void *xe_pci_live_device_gen_param(struct kunit *test, const void *prev, char *desc);
#endif
diff --git a/include/kunit/test.h b/include/kunit/test.h
index fc8fd55b2dfb..8eba1b03c3e3 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -128,7 +128,8 @@ struct kunit_attributes {
struct kunit_case {
void (*run_case)(struct kunit *test);
const char *name;
- const void* (*generate_params)(const void *prev, char *desc);
+ const void* (*generate_params)(struct kunit *test,
+ const void *prev, char *desc);
struct kunit_attributes attr;
int (*param_init)(struct kunit *test);
void (*param_exit)(struct kunit *test);
@@ -1703,7 +1704,8 @@ do { \
* Define function @name_gen_params which uses @array to generate parameters.
*/
#define KUNIT_ARRAY_PARAM(name, array, get_desc) \
- static const void *name##_gen_params(const void *prev, char *desc) \
+ static const void *name##_gen_params(struct kunit *test, \
+ const void *prev, char *desc) \
{ \
typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
if (__next - (array) < ARRAY_SIZE((array))) { \
@@ -1724,7 +1726,8 @@ do { \
* Define function @name_gen_params which uses @array to generate parameters.
*/
#define KUNIT_ARRAY_PARAM_DESC(name, array, desc_member) \
- static const void *name##_gen_params(const void *prev, char *desc) \
+ static const void *name##_gen_params(struct kunit *test, \
+ const void *prev, char *desc) \
{ \
typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
if (__next - (array) < ARRAY_SIZE((array))) { \
diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c
index 49ab81faaed9..a13a090bb2a7 100644
--- a/kernel/kcsan/kcsan_test.c
+++ b/kernel/kcsan/kcsan_test.c
@@ -1383,7 +1383,7 @@ static void test_atomic_builtins_missing_barrier(struct kunit *test)
* The thread counts are chosen to cover potentially interesting boundaries and
* corner cases (2 to 5), and then stress the system with larger counts.
*/
-static const void *nthreads_gen_params(const void *prev, char *desc)
+static const void *nthreads_gen_params(struct kunit *test, const void *prev, char *desc)
{
long nthreads = (long)prev;
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 0fe61dec5a96..50705248abad 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -700,7 +700,7 @@ int kunit_run_tests(struct kunit_suite *suite)
/* Get initial param. */
param_desc[0] = '\0';
/* TODO: Make generate_params try-catch */
- curr_param = test_case->generate_params(NULL, param_desc);
+ curr_param = test_case->generate_params(&test, NULL, param_desc);
test_case->status = KUNIT_SKIPPED;
kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
"KTAP version 1\n");
@@ -731,7 +731,8 @@ int kunit_run_tests(struct kunit_suite *suite)
/* Get next param. */
param_desc[0] = '\0';
- curr_param = test_case->generate_params(curr_param, param_desc);
+ curr_param = test_case->generate_params(&test, curr_param,
+ param_desc);
}
/*
* TODO: Put into a try catch. Since we don't need suite->exit
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 4/7] kunit: Enable direct registration of parameter arrays to a KUnit test
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
` (2 preceding siblings ...)
2025-08-26 9:13 ` [PATCH v4 3/7] kunit: Pass parameterized test context to generate_params() David Gow
@ 2025-08-26 9:13 ` David Gow
2025-08-26 9:13 ` [PATCH v4 5/7] kunit: Add example parameterized test with shared resource management using the Resource API David Gow
` (3 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: mark.rutland, elver, dvyukov, lucas.demarchi, thomas.hellstrom,
rodrigo.vivi, linux-kselftest, kunit-dev, kasan-dev, intel-xe,
dri-devel, linux-kernel, Stephen Rothwell, David Gow
From: Marie Zhussupova <marievic@google.com>
KUnit parameterized tests currently support two primary methods f
or getting parameters:
1. Defining custom logic within a generate_params() function.
2. Using the KUNIT_ARRAY_PARAM() and KUNIT_ARRAY_PARAM_DESC()
macros with a pre-defined static array and passing
the created *_gen_params() to KUNIT_CASE_PARAM().
These methods present limitations when dealing with dynamically
generated parameter arrays, or in scenarios where populating parameters
sequentially via generate_params() is inefficient or overly complex.
This patch addresses these limitations by adding a new `params_array`
field to `struct kunit`, of the type `kunit_params`. The
`struct kunit_params` is designed to store the parameter array itself,
along with essential metadata including the parameter count, parameter
size, and a get_description() function for providing custom descriptions
for individual parameters.
The `params_array` field can be populated by calling the new
kunit_register_params_array() macro from within a param_init() function.
This will register the array as part of the parameterized test context.
The user will then need to pass kunit_array_gen_params() to the
KUNIT_CASE_PARAM_WITH_INIT() macro as the generator function, if not
providing their own. kunit_array_gen_params() is a KUnit helper that will
use the registered array to generate parameters.
The arrays passed to KUNIT_ARRAY_PARAM(,DESC) will also be registered to
the parameterized test context for consistency as well as for higher
availability of the parameter count that will be used for outputting a KTAP
test plan for a parameterized test.
This modification provides greater flexibility to the KUnit framework,
allowing testers to easily register and utilize both dynamic and static
parameter arrays.
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Signed-off-by: Marie Zhussupova <marievic@google.com>
[Only output the test plan if using kunit_array_gen_params --David]
Signed-off-by: David Gow <davidgow@google.com>
---
Changes in v4:
v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-5-marievic@google.com/
- Only output a KTAP test plan if the generate_params function is
kunit_array_gen_params.
- This fixes an issue with generate_params functions which use an array,
but skip some elements.
- This change is also available as a separate patch here:
https://lore.kernel.org/linux-kselftest/20250821135447.1618942-2-davidgow@google.com/
Changes in v3:
v2: https://lore.kernel.org/all/20250811221739.2694336-5-marievic@google.com/
- Commit message formatting.
Changes in v2:
v1: https://lore.kernel.org/all/20250729193647.3410634-7-marievic@google.com/
- If the parameter count is available for a parameterized test, the
kunit_run_tests() function will now output the KTAP test plan for it.
- The name of the struct kunit_params field in struct kunit was changed
from params_data to params_array. This name change better reflects its
purpose, which is to encapsulate both the parameter array and its
associated metadata.
- The name of `kunit_get_next_param_and_desc` was changed to
`kunit_array_gen_params` to make it simpler and to better fit its purpose
of being KUnit's built-in generator function that uses arrays to generate
parameters.
- The signature of get_description() in `struct params_array` was changed to
accept the parameterized test context, as well. This way test users can
potentially use information available in the parameterized test context,
such as the parameterized test name for setting the parameter descriptions.
- The type of `num_params` in `struct params_array` was changed from int to
size_t for better handling of the array size.
- The name of __kunit_init_params() was changed to be kunit_init_params().
Logic that sets the get_description() function pointer to NULL was also
added in there.
- `kunit_array_gen_params` is now exported to make it available to use
with modules.
- Instead of allowing NULL to be passed in as the parameter generator
function in the KUNIT_CASE_PARAM_WITH_INIT macro, users will now be asked
to provide `kunit_array_gen_params` as the generator function. This will
ensure that a parameterized test remains defined by the existence of a
parameter generation function.
- KUNIT_ARRAY_PARAM(,DESC) will now additionally register the passed in array
in struct kunit_params. This will make things more consistent i.e. if a
parameter array is available then the struct kunit_params field in parent
struct kunit is populated. Additionally, this will increase the
availability of the KTAP test plan.
- The comments and the commit message were changed to reflect the
parameterized testing terminology. See the patch series cover letter
change log for the definitions.
---
include/kunit/test.h | 65 ++++++++++++++++++++++++++++++++++++++++----
lib/kunit/test.c | 32 ++++++++++++++++++++++
2 files changed, 91 insertions(+), 6 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index 8eba1b03c3e3..5ec5182b5e57 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -234,9 +234,13 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status)
* Provides the option to register param_init() and param_exit() functions.
* param_init/exit will be passed the parameterized test context and run once
* before and once after the parameterized test. The init function can be used
- * to add resources to share between parameter runs, and any other setup logic.
- * The exit function can be used to clean up resources that were not managed by
- * the parameterized test, and any other teardown logic.
+ * to add resources to share between parameter runs, pass parameter arrays,
+ * and any other setup logic. The exit function can be used to clean up resources
+ * that were not managed by the parameterized test, and any other teardown logic.
+ *
+ * Note: If you are registering a parameter array in param_init() with
+ * kunit_register_param_array() then you need to pass kunit_array_gen_params()
+ * to this as the generator function.
*/
#define KUNIT_CASE_PARAM_WITH_INIT(test_name, gen_params, init, exit) \
{ .run_case = test_name, .name = #test_name, \
@@ -289,6 +293,20 @@ struct kunit_suite_set {
struct kunit_suite * const *end;
};
+/* Stores the pointer to the parameter array and its metadata. */
+struct kunit_params {
+ /*
+ * Reference to the parameter array for a parameterized test. This
+ * is NULL if a parameter array wasn't directly passed to the
+ * parameterized test context struct kunit via kunit_register_params_array().
+ */
+ const void *params;
+ /* Reference to a function that gets the description of a parameter. */
+ void (*get_description)(struct kunit *test, const void *param, char *desc);
+ size_t num_params;
+ size_t elem_size;
+};
+
/**
* struct kunit - represents a running instance of a test.
*
@@ -296,16 +314,18 @@ struct kunit_suite_set {
* created in the init function (see &struct kunit_suite).
* @parent: reference to the parent context of type struct kunit that can
* be used for storing shared resources.
+ * @params_array: for storing the parameter array.
*
* Used to store information about the current context under which the test
* is running. Most of this data is private and should only be accessed
- * indirectly via public functions; the two exceptions are @priv and @parent
- * which can be used by the test writer to store arbitrary data and access the
- * parent context, respectively.
+ * indirectly via public functions; the exceptions are @priv, @parent and
+ * @params_array which can be used by the test writer to store arbitrary data,
+ * access the parent context, and to store the parameter array, respectively.
*/
struct kunit {
void *priv;
struct kunit *parent;
+ struct kunit_params params_array;
/* private: internal use only. */
const char *name; /* Read only after initialization! */
@@ -376,6 +396,8 @@ void kunit_exec_list_tests(struct kunit_suite_set *suite_set, bool include_attr)
struct kunit_suite_set kunit_merge_suite_sets(struct kunit_suite_set init_suite_set,
struct kunit_suite_set suite_set);
+const void *kunit_array_gen_params(struct kunit *test, const void *prev, char *desc);
+
#if IS_BUILTIN(CONFIG_KUNIT)
int kunit_run_all_tests(void);
#else
@@ -1708,6 +1730,8 @@ do { \
const void *prev, char *desc) \
{ \
typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
+ if (!prev) \
+ kunit_register_params_array(test, array, ARRAY_SIZE(array), NULL); \
if (__next - (array) < ARRAY_SIZE((array))) { \
void (*__get_desc)(typeof(__next), char *) = get_desc; \
if (__get_desc) \
@@ -1730,6 +1754,8 @@ do { \
const void *prev, char *desc) \
{ \
typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
+ if (!prev) \
+ kunit_register_params_array(test, array, ARRAY_SIZE(array), NULL); \
if (__next - (array) < ARRAY_SIZE((array))) { \
strscpy(desc, __next->desc_member, KUNIT_PARAM_DESC_SIZE); \
return __next; \
@@ -1737,6 +1763,33 @@ do { \
return NULL; \
}
+/**
+ * kunit_register_params_array() - Register parameter array for a KUnit test.
+ * @test: The KUnit test structure to which parameters will be added.
+ * @array: An array of test parameters.
+ * @param_count: Number of parameters.
+ * @get_desc: Function that generates a string description for a given parameter
+ * element.
+ *
+ * This macro initializes the @test's parameter array data, storing information
+ * including the parameter array, its count, the element size, and the parameter
+ * description function within `test->params_array`.
+ *
+ * Note: If using this macro in param_init(), kunit_array_gen_params()
+ * will then need to be manually provided as the parameter generator function to
+ * KUNIT_CASE_PARAM_WITH_INIT(). kunit_array_gen_params() is a KUnit
+ * function that uses the registered array to generate parameters
+ */
+#define kunit_register_params_array(test, array, param_count, get_desc) \
+ do { \
+ struct kunit *_test = (test); \
+ const typeof((array)[0]) * _params_ptr = &(array)[0]; \
+ _test->params_array.params = _params_ptr; \
+ _test->params_array.num_params = (param_count); \
+ _test->params_array.elem_size = sizeof(*_params_ptr); \
+ _test->params_array.get_description = (get_desc); \
+ } while (0)
+
// TODO(dlatypov@google.com): consider eventually migrating users to explicitly
// include resource.h themselves if they need it.
#include <kunit/resource.h>
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 50705248abad..bb66ea1a3eac 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -337,6 +337,14 @@ void __kunit_do_failed_assertion(struct kunit *test,
}
EXPORT_SYMBOL_GPL(__kunit_do_failed_assertion);
+static void kunit_init_params(struct kunit *test)
+{
+ test->params_array.params = NULL;
+ test->params_array.get_description = NULL;
+ test->params_array.num_params = 0;
+ test->params_array.elem_size = 0;
+}
+
void kunit_init_test(struct kunit *test, const char *name, struct string_stream *log)
{
spin_lock_init(&test->lock);
@@ -347,6 +355,7 @@ void kunit_init_test(struct kunit *test, const char *name, struct string_stream
string_stream_clear(log);
test->status = KUNIT_SUCCESS;
test->status_comment[0] = '\0';
+ kunit_init_params(test);
}
EXPORT_SYMBOL_GPL(kunit_init_test);
@@ -641,6 +650,23 @@ static void kunit_accumulate_stats(struct kunit_result_stats *total,
total->total += add.total;
}
+const void *kunit_array_gen_params(struct kunit *test, const void *prev, char *desc)
+{
+ struct kunit_params *params_arr = &test->params_array;
+ const void *param;
+
+ if (test->param_index < params_arr->num_params) {
+ param = (char *)params_arr->params
+ + test->param_index * params_arr->elem_size;
+
+ if (params_arr->get_description)
+ params_arr->get_description(test, param, desc);
+ return param;
+ }
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(kunit_array_gen_params);
+
static void kunit_init_parent_param_test(struct kunit_case *test_case, struct kunit *test)
{
if (test_case->param_init) {
@@ -706,6 +732,12 @@ int kunit_run_tests(struct kunit_suite *suite)
"KTAP version 1\n");
kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
"# Subtest: %s", test_case->name);
+ if (test.params_array.params &&
+ test_case->generate_params == kunit_array_gen_params) {
+ kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT
+ KUNIT_SUBTEST_INDENT "1..%zd\n",
+ test.params_array.num_params);
+ }
while (curr_param) {
struct kunit param_test = {
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 5/7] kunit: Add example parameterized test with shared resource management using the Resource API
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
` (3 preceding siblings ...)
2025-08-26 9:13 ` [PATCH v4 4/7] kunit: Enable direct registration of parameter arrays to a KUnit test David Gow
@ 2025-08-26 9:13 ` David Gow
2025-08-26 9:13 ` [PATCH v4 6/7] kunit: Add example parameterized test with direct dynamic parameter array setup David Gow
` (2 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: mark.rutland, elver, dvyukov, lucas.demarchi, thomas.hellstrom,
rodrigo.vivi, linux-kselftest, kunit-dev, kasan-dev, intel-xe,
dri-devel, linux-kernel, Stephen Rothwell, David Gow
From: Marie Zhussupova <marievic@google.com>
Add example_params_test_with_init() to illustrate how to manage
shared resources across a parameterized KUnit test. This example
showcases the use of the new param_init() function and its registration
to a test using the KUNIT_CASE_PARAM_WITH_INIT() macro.
Additionally, the test demonstrates how to directly pass a parameter array
to the parameterized test context via kunit_register_params_array()
and leveraging the Resource API for shared resource management.
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Marie Zhussupova <marievic@google.com>
Signed-off-by: David Gow <davidgow@google.com>
---
No changes in v4:
v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-6-marievic@google.com/
Changes in v3:
v2: https://lore.kernel.org/all/20250811221739.2694336-6-marievic@google.com/
- Code comment edits.
Changes in v2:
v1: https://lore.kernel.org/all/20250729193647.3410634-8-marievic@google.com/
- kunit_array_gen_params() is now explicitly passed to
KUNIT_CASE_PARAM_WITH_INIT() to be consistent with a parameterized test
being defined by the existence of the generate_params() function.
- The comments were edited to be more concise.
- The patch header was changed to reflect that this example test's intent
is more aligned with showcasing using the Resource API for shared
resource management.
- The comments and the commit message were changed to reflect the
parameterized testing terminology. See the patch series cover letter
change log for the definitions.
---
lib/kunit/kunit-example-test.c | 113 +++++++++++++++++++++++++++++++++
1 file changed, 113 insertions(+)
diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
index 3056d6bc705d..3e858367be01 100644
--- a/lib/kunit/kunit-example-test.c
+++ b/lib/kunit/kunit-example-test.c
@@ -277,6 +277,117 @@ static void example_slow_test(struct kunit *test)
KUNIT_EXPECT_EQ(test, 1 + 1, 2);
}
+/*
+ * This custom function allocates memory and sets the information we want
+ * stored in the kunit_resource->data field.
+ */
+static int example_resource_init(struct kunit_resource *res, void *context)
+{
+ int *info = kmalloc(sizeof(*info), GFP_KERNEL);
+
+ if (!info)
+ return -ENOMEM;
+ *info = *(int *)context;
+ res->data = info;
+ return 0;
+}
+
+/*
+ * This function deallocates memory for the kunit_resource->data field.
+ */
+static void example_resource_free(struct kunit_resource *res)
+{
+ kfree(res->data);
+}
+
+/*
+ * This match function is invoked by kunit_find_resource() to locate
+ * a test resource based on certain criteria.
+ */
+static bool example_resource_alloc_match(struct kunit *test,
+ struct kunit_resource *res,
+ void *match_data)
+{
+ return res->data && res->free == example_resource_free;
+}
+
+/*
+ * This is an example of a function that provides a description for each of the
+ * parameters in a parameterized test.
+ */
+static void example_param_array_get_desc(struct kunit *test, const void *p, char *desc)
+{
+ const struct example_param *param = p;
+
+ snprintf(desc, KUNIT_PARAM_DESC_SIZE,
+ "example check if %d is less than or equal to 3", param->value);
+}
+
+/*
+ * This function gets passed in the parameterized test context i.e. the
+ * struct kunit belonging to the parameterized test. You can use this function
+ * to add resources you want shared across the whole parameterized test or
+ * for additional setup.
+ */
+static int example_param_init(struct kunit *test)
+{
+ int ctx = 3; /* Data to be stored. */
+ size_t arr_size = ARRAY_SIZE(example_params_array);
+
+ /*
+ * This allocates a struct kunit_resource, sets its data field to
+ * ctx, and adds it to the struct kunit's resources list. Note that
+ * this is parameterized test managed. So, it doesn't need to have
+ * a custom exit function to deallocation as it will get cleaned up at
+ * the end of the parameterized test.
+ */
+ void *data = kunit_alloc_resource(test, example_resource_init, example_resource_free,
+ GFP_KERNEL, &ctx);
+
+ if (!data)
+ return -ENOMEM;
+ /*
+ * Pass the parameter array information to the parameterized test context
+ * struct kunit. Note that you will need to provide kunit_array_gen_params()
+ * as the generator function to KUNIT_CASE_PARAM_WITH_INIT() when registering
+ * a parameter array this route.
+ */
+ kunit_register_params_array(test, example_params_array, arr_size,
+ example_param_array_get_desc);
+ return 0;
+}
+
+/*
+ * This is an example of a test that uses shared resources available in the
+ * parameterized test context.
+ */
+static void example_params_test_with_init(struct kunit *test)
+{
+ int threshold;
+ struct kunit_resource *res;
+ const struct example_param *param = test->param_value;
+
+ /* By design, param pointer will not be NULL. */
+ KUNIT_ASSERT_NOT_NULL(test, param);
+
+ /*
+ * Here we pass test->parent to search for shared resources in the
+ * parameterized test context.
+ */
+ res = kunit_find_resource(test->parent, example_resource_alloc_match, NULL);
+
+ KUNIT_ASSERT_NOT_NULL(test, res);
+
+ /* Since kunit_resource->data is a void pointer we need to typecast it. */
+ threshold = *((int *)res->data);
+
+ /* Assert that the parameter is less than or equal to a certain threshold. */
+ KUNIT_ASSERT_LE(test, param->value, threshold);
+
+ /* This decreases the reference count after calling kunit_find_resource(). */
+ kunit_put_resource(res);
+}
+
/*
* Here we make a list of all the test cases we want to add to the test suite
* below.
@@ -296,6 +407,8 @@ static struct kunit_case example_test_cases[] = {
KUNIT_CASE(example_static_stub_using_fn_ptr_test),
KUNIT_CASE(example_priv_test),
KUNIT_CASE_PARAM(example_params_test, example_gen_params),
+ KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init, kunit_array_gen_params,
+ example_param_init, NULL),
KUNIT_CASE_SLOW(example_slow_test),
{}
};
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 6/7] kunit: Add example parameterized test with direct dynamic parameter array setup
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
` (4 preceding siblings ...)
2025-08-26 9:13 ` [PATCH v4 5/7] kunit: Add example parameterized test with shared resource management using the Resource API David Gow
@ 2025-08-26 9:13 ` David Gow
2025-08-26 9:13 ` [PATCH v4 7/7] Documentation: kunit: Document new parameterized test features David Gow
2025-08-28 16:56 ` [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework Mark Rutland
7 siblings, 0 replies; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: mark.rutland, elver, dvyukov, lucas.demarchi, thomas.hellstrom,
rodrigo.vivi, linux-kselftest, kunit-dev, kasan-dev, intel-xe,
dri-devel, linux-kernel, Stephen Rothwell, David Gow
From: Marie Zhussupova <marievic@google.com>
Introduce example_params_test_with_init_dynamic_arr(). This new
KUnit test demonstrates directly assigning a dynamic parameter
array, using the kunit_register_params_array() macro, to a
parameterized test context.
It highlights the use of param_init() and param_exit() for
initialization and exit of a parameterized test, and their
registration to the test case with KUNIT_CASE_PARAM_WITH_INIT().
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Marie Zhussupova <marievic@google.com>
Signed-off-by: David Gow <davidgow@google.com>
---
Changes in v4:
v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-7-marievic@google.com/
- No changes.
Changes in v3:
v2: https://lore.kernel.org/all/20250811221739.2694336-7-marievic@google.com/
- No changes.
Changes in v2:
v1: https://lore.kernel.org/all/20250729193647.3410634-9-marievic@google.com/
- kunit_array_gen_params() is now explicitly passed to
KUNIT_CASE_PARAM_WITH_INIT() to be consistent with the parameterized test
being defined by the existence of the generate_params() function.
- param_init() was changed to output a log at the start of a parameterized
test.
- The parameter array was changed to be allocated using kunit_kmalloc_array(),
a KUnit memory allocation API, as that would be the preferred/easier method.
To still demonstrate a use of param_exit(), it now outputs a log at the end
of the parameterized test.
- The comments and the commit message were changed to reflect the
parameterized testing terminology. See the patch series cover letter
change log for the definitions.
---
lib/kunit/kunit-example-test.c | 104 +++++++++++++++++++++++++++++++++
1 file changed, 104 insertions(+)
diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
index 3e858367be01..9452b163956f 100644
--- a/lib/kunit/kunit-example-test.c
+++ b/lib/kunit/kunit-example-test.c
@@ -388,6 +388,107 @@ static void example_params_test_with_init(struct kunit *test)
kunit_put_resource(res);
}
+/*
+ * Helper function to create a parameter array of Fibonacci numbers. This example
+ * highlights a parameter generation scenario that is:
+ * 1. Not feasible to fully pre-generate at compile time.
+ * 2. Challenging to implement with a standard generate_params() function,
+ * as it only provides the previous parameter, while Fibonacci requires
+ * access to two preceding values for calculation.
+ */
+static void *make_fibonacci_params(struct kunit *test, size_t seq_size)
+{
+ int *seq;
+
+ if (seq_size <= 0)
+ return NULL;
+ /*
+ * Using kunit_kmalloc_array here ties the lifetime of the array to
+ * the parameterized test i.e. it will get automatically cleaned up
+ * by KUnit after the parameterized test finishes.
+ */
+ seq = kunit_kmalloc_array(test, seq_size, sizeof(int), GFP_KERNEL);
+
+ if (!seq)
+ return NULL;
+ if (seq_size >= 1)
+ seq[0] = 0;
+ if (seq_size >= 2)
+ seq[1] = 1;
+ for (int i = 2; i < seq_size; i++)
+ seq[i] = seq[i - 1] + seq[i - 2];
+ return seq;
+}
+
+/*
+ * This is an example of a function that provides a description for each of the
+ * parameters.
+ */
+static void example_param_dynamic_arr_get_desc(struct kunit *test, const void *p, char *desc)
+{
+ const int *fib_num = p;
+
+ snprintf(desc, KUNIT_PARAM_DESC_SIZE, "fibonacci param: %d", *fib_num);
+}
+
+/*
+ * Example of a parameterized test param_init() function that registers a dynamic
+ * array of parameters.
+ */
+static int example_param_init_dynamic_arr(struct kunit *test)
+{
+ size_t seq_size;
+ int *fibonacci_params;
+
+ kunit_info(test, "initializing parameterized test\n");
+
+ seq_size = 6;
+ fibonacci_params = make_fibonacci_params(test, seq_size);
+
+ if (!fibonacci_params)
+ return -ENOMEM;
+
+ /*
+ * Passes the dynamic parameter array information to the parameterized test
+ * context struct kunit. The array and its metadata will be stored in
+ * test->parent->params_array. The array itself will be located in
+ * params_data.params.
+ *
+ * Note that you will need to pass kunit_array_gen_params() as the
+ * generator function to KUNIT_CASE_PARAM_WITH_INIT() when registering
+ * a parameter array this route.
+ */
+ kunit_register_params_array(test, fibonacci_params, seq_size,
+ example_param_dynamic_arr_get_desc);
+ return 0;
+}
+
+/*
+ * Example of a parameterized test param_exit() function that outputs a log
+ * at the end of the parameterized test. It could also be used for any other
+ * teardown logic.
+ */
+static void example_param_exit_dynamic_arr(struct kunit *test)
+{
+ kunit_info(test, "exiting parameterized test\n");
+}
+
+/*
+ * Example of test that uses the registered dynamic array to perform assertions
+ * and expectations.
+ */
+static void example_params_test_with_init_dynamic_arr(struct kunit *test)
+{
+ const int *param = test->param_value;
+ int param_val;
+
+ /* By design, param pointer will not be NULL. */
+ KUNIT_ASSERT_NOT_NULL(test, param);
+
+ param_val = *param;
+ KUNIT_EXPECT_EQ(test, param_val - param_val, 0);
+}
+
/*
* Here we make a list of all the test cases we want to add to the test suite
* below.
@@ -409,6 +510,9 @@ static struct kunit_case example_test_cases[] = {
KUNIT_CASE_PARAM(example_params_test, example_gen_params),
KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init, kunit_array_gen_params,
example_param_init, NULL),
+ KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init_dynamic_arr,
+ kunit_array_gen_params, example_param_init_dynamic_arr,
+ example_param_exit_dynamic_arr),
KUNIT_CASE_SLOW(example_slow_test),
{}
};
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 7/7] Documentation: kunit: Document new parameterized test features
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
` (5 preceding siblings ...)
2025-08-26 9:13 ` [PATCH v4 6/7] kunit: Add example parameterized test with direct dynamic parameter array setup David Gow
@ 2025-08-26 9:13 ` David Gow
2025-08-28 16:56 ` [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework Mark Rutland
7 siblings, 0 replies; 10+ messages in thread
From: David Gow @ 2025-08-26 9:13 UTC (permalink / raw)
To: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins
Cc: mark.rutland, elver, dvyukov, lucas.demarchi, thomas.hellstrom,
rodrigo.vivi, linux-kselftest, kunit-dev, kasan-dev, intel-xe,
dri-devel, linux-kernel, Stephen Rothwell, David Gow
From: Marie Zhussupova <marievic@google.com>
This patch updates the KUnit docs to show how to use the new
parameterized test context to share resources between parameter runs.
It documents and show examples of different ways the test user can
pass parameter arrays to a parameterized test. Finally, it specifies the
parameterized testing terminology.
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Marie Zhussupova <marievic@google.com>
Signed-off-by: David Gow <davidgow@google.com>
---
Changes in v4:
v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-8-marievic@google.com/
- No changes.
Changes in v3:
v2: https://lore.kernel.org/all/20250811221739.2694336-8-marievic@google.com/
- Parameterized test terminology was made more concise.
- Introduction now includes more background information on the
generate_params() function.
- Minor wording edits for conciseness.
- Code line number references were removed as they could quickly go
out of date.
Changes in v2:
v1: https://lore.kernel.org/all/20250729193647.3410634-10-marievic@google.com/
- The documentation was updated to establish the parameterized
testing terminology and reflect all the patch series changes.
- The references to other parts of the KUnit Documentation were
not changed from being "Documentation/dev-tools/kunit/api/test.rst"
to ":ref:`kunit-resource`" links as originally planned. This is
because the existing way shows up as a link to a webpage and it
would be hard for people reading the documentation as an .rst
file to find the referred section without having the file path.
- The code examples were made more concise.
- Minor edits to titles and formatting.
---
Documentation/dev-tools/kunit/usage.rst | 342 +++++++++++++++++++++++-
1 file changed, 337 insertions(+), 5 deletions(-)
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index 066ecda1dd98..ebd06f5ea455 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -542,11 +542,31 @@ There is more boilerplate code involved, but it can:
Parameterized Testing
~~~~~~~~~~~~~~~~~~~~~
-The table-driven testing pattern is common enough that KUnit has special
-support for it.
+To run a test case against multiple inputs, KUnit provides a parameterized
+testing framework. This feature formalizes and extends the concept of
+table-driven tests discussed previously.
-By reusing the same ``cases`` array from above, we can write the test as a
-"parameterized test" with the following.
+A KUnit test is determined to be parameterized if a parameter generator function
+is provided when registering the test case. A test user can either write their
+own generator function or use one that is provided by KUnit. The generator
+function is stored in ``kunit_case->generate_params`` and can be set using the
+macros described in the section below.
+
+To establish the terminology, a "parameterized test" is a test which is run
+multiple times (once per "parameter" or "parameter run"). Each parameter run has
+both its own independent ``struct kunit`` (the "parameter run context") and
+access to a shared parent ``struct kunit`` (the "parameterized test context").
+
+Passing Parameters to a Test
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+There are three ways to provide the parameters to a test:
+
+Array Parameter Macros:
+
+ KUnit provides special support for the common table-driven testing pattern.
+ By applying either ``KUNIT_ARRAY_PARAM`` or ``KUNIT_ARRAY_PARAM_DESC`` to the
+ ``cases`` array from the previous section, we can create a parameterized test
+ as shown below:
.. code-block:: c
@@ -555,7 +575,7 @@ By reusing the same ``cases`` array from above, we can write the test as a
const char *str;
const char *sha1;
};
- const struct sha1_test_case cases[] = {
+ static const struct sha1_test_case cases[] = {
{
.str = "hello world",
.sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
@@ -590,6 +610,318 @@ By reusing the same ``cases`` array from above, we can write the test as a
{}
};
+Custom Parameter Generator Function:
+
+ The generator function is responsible for generating parameters one-by-one
+ and has the following signature:
+ ``const void* (*)(struct kunit *test, const void *prev, char *desc)``.
+ You can pass the generator function to the ``KUNIT_CASE_PARAM``
+ or ``KUNIT_CASE_PARAM_WITH_INIT`` macros.
+
+ The function receives the previously generated parameter as the ``prev`` argument
+ (which is ``NULL`` on the first call) and can also access the parameterized
+ test context passed as the ``test`` argument. KUnit calls this function
+ repeatedly until it returns ``NULL``, which signifies that a parameterized
+ test ended.
+
+ Below is an example of how it works:
+
+.. code-block:: c
+
+ #define MAX_TEST_BUFFER_SIZE 8
+
+ // Example generator function. It produces a sequence of buffer sizes that
+ // are powers of two, starting at 1 (e.g., 1, 2, 4, 8).
+ static const void *buffer_size_gen_params(struct kunit *test, const void *prev, char *desc)
+ {
+ long prev_buffer_size = (long)prev;
+ long next_buffer_size = 1; // Start with an initial size of 1.
+
+ // Stop generating parameters if the limit is reached or exceeded.
+ if (prev_buffer_size >= MAX_TEST_BUFFER_SIZE)
+ return NULL;
+
+ // For subsequent calls, calculate the next size by doubling the previous one.
+ if (prev)
+ next_buffer_size = prev_buffer_size << 1;
+
+ return (void *)next_buffer_size;
+ }
+
+ // Simple test to validate that kunit_kzalloc provides zeroed memory.
+ static void buffer_zero_test(struct kunit *test)
+ {
+ long buffer_size = (long)test->param_value;
+ // Use kunit_kzalloc to allocate a zero-initialized buffer. This makes the
+ // memory "parameter run managed," meaning it's automatically cleaned up at
+ // the end of each parameter run.
+ int *buf = kunit_kzalloc(test, buffer_size * sizeof(int), GFP_KERNEL);
+
+ // Ensure the allocation was successful.
+ KUNIT_ASSERT_NOT_NULL(test, buf);
+
+ // Loop through the buffer and confirm every element is zero.
+ for (int i = 0; i < buffer_size; i++)
+ KUNIT_EXPECT_EQ(test, buf[i], 0);
+ }
+
+ static struct kunit_case buffer_test_cases[] = {
+ KUNIT_CASE_PARAM(buffer_zero_test, buffer_size_gen_params),
+ {}
+ };
+
+Runtime Parameter Array Registration in the Init Function:
+
+ For scenarios where you might need to initialize a parameterized test, you
+ can directly register a parameter array to the parameterized test context.
+
+ To do this, you must pass the parameterized test context, the array itself,
+ the array size, and a ``get_description()`` function to the
+ ``kunit_register_params_array()`` macro. This macro populates
+ ``struct kunit_params`` within the parameterized test context, effectively
+ storing a parameter array object. The ``get_description()`` function will
+ be used for populating parameter descriptions and has the following signature:
+ ``void (*)(struct kunit *test, const void *param, char *desc)``. Note that it
+ also has access to the parameterized test context.
+
+ .. important::
+ When using this way to register a parameter array, you will need to
+ manually pass ``kunit_array_gen_params()`` as the generator function to
+ ``KUNIT_CASE_PARAM_WITH_INIT``. ``kunit_array_gen_params()`` is a KUnit
+ helper that will use the registered array to generate the parameters.
+
+ If needed, instead of passing the KUnit helper, you can also pass your
+ own custom generator function that utilizes the parameter array. To
+ access the parameter array from within the parameter generator
+ function use ``test->params_array.params``.
+
+ The ``kunit_register_params_array()`` macro should be called within a
+ ``param_init()`` function that initializes the parameterized test and has
+ the following signature ``int (*)(struct kunit *test)``. For a detailed
+ explanation of this mechanism please refer to the "Adding Shared Resources"
+ section that is after this one. This method supports registering both
+ dynamically built and static parameter arrays.
+
+ The code snippet below shows the ``example_param_init_dynamic_arr`` test that
+ utilizes ``make_fibonacci_params()`` to create a dynamic array, which is then
+ registered using ``kunit_register_params_array()``. To see the full code
+ please refer to lib/kunit/kunit-example-test.c.
+
+.. code-block:: c
+
+ /*
+ * Example of a parameterized test param_init() function that registers a dynamic
+ * array of parameters.
+ */
+ static int example_param_init_dynamic_arr(struct kunit *test)
+ {
+ size_t seq_size;
+ int *fibonacci_params;
+
+ kunit_info(test, "initializing parameterized test\n");
+
+ seq_size = 6;
+ fibonacci_params = make_fibonacci_params(test, seq_size);
+ if (!fibonacci_params)
+ return -ENOMEM;
+ /*
+ * Passes the dynamic parameter array information to the parameterized test
+ * context struct kunit. The array and its metadata will be stored in
+ * test->parent->params_array. The array itself will be located in
+ * params_data.params.
+ */
+ kunit_register_params_array(test, fibonacci_params, seq_size,
+ example_param_dynamic_arr_get_desc);
+ return 0;
+ }
+
+ static struct kunit_case example_test_cases[] = {
+ /*
+ * Note how we pass kunit_array_gen_params() to use the array we
+ * registered in example_param_init_dynamic_arr() to generate
+ * parameters.
+ */
+ KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init_dynamic_arr,
+ kunit_array_gen_params,
+ example_param_init_dynamic_arr,
+ example_param_exit_dynamic_arr),
+ {}
+ };
+
+Adding Shared Resources
+^^^^^^^^^^^^^^^^^^^^^^^
+All parameter runs in this framework hold a reference to the parameterized test
+context, which can be accessed using the parent ``struct kunit`` pointer. The
+parameterized test context is not used to execute any test logic itself; instead,
+it serves as a container for shared resources.
+
+It's possible to add resources to share between parameter runs within a
+parameterized test by using ``KUNIT_CASE_PARAM_WITH_INIT``, to which you pass
+custom ``param_init()`` and ``param_exit()`` functions. These functions run once
+before and once after the parameterized test, respectively.
+
+The ``param_init()`` function, with the signature ``int (*)(struct kunit *test)``,
+can be used for adding resources to the ``resources`` or ``priv`` fields of
+the parameterized test context, registering the parameter array, and any other
+initialization logic.
+
+The ``param_exit()`` function, with the signature ``void (*)(struct kunit *test)``,
+can be used to release any resources that were not parameterized test managed (i.e.
+not automatically cleaned up after the parameterized test ends) and for any other
+exit logic.
+
+Both ``param_init()`` and ``param_exit()`` are passed the parameterized test
+context behind the scenes. However, the test case function receives the parameter
+run context. Therefore, to manage and access shared resources from within a test
+case function, you must use ``test->parent``.
+
+For instance, finding a shared resource allocated by the Resource API requires
+passing ``test->parent`` to ``kunit_find_resource()``. This principle extends to
+all other APIs that might be used in the test case function, including
+``kunit_kzalloc()``, ``kunit_kmalloc_array()``, and others (see
+Documentation/dev-tools/kunit/api/test.rst and the
+Documentation/dev-tools/kunit/api/resource.rst).
+
+.. note::
+ The ``suite->init()`` function, which executes before each parameter run,
+ receives the parameter run context. Therefore, any resources set up in
+ ``suite->init()`` are cleaned up after each parameter run.
+
+The code below shows how you can add the shared resources. Note that this code
+utilizes the Resource API, which you can read more about here:
+Documentation/dev-tools/kunit/api/resource.rst. To see the full version of this
+code please refer to lib/kunit/kunit-example-test.c.
+
+.. code-block:: c
+
+ static int example_resource_init(struct kunit_resource *res, void *context)
+ {
+ ... /* Code that allocates memory and stores context in res->data. */
+ }
+
+ /* This function deallocates memory for the kunit_resource->data field. */
+ static void example_resource_free(struct kunit_resource *res)
+ {
+ kfree(res->data);
+ }
+
+ /* This match function locates a test resource based on defined criteria. */
+ static bool example_resource_alloc_match(struct kunit *test, struct kunit_resource *res,
+ void *match_data)
+ {
+ return res->data && res->free == example_resource_free;
+ }
+
+ /* Function to initialize the parameterized test. */
+ static int example_param_init(struct kunit *test)
+ {
+ int ctx = 3; /* Data to be stored. */
+ void *data = kunit_alloc_resource(test, example_resource_init,
+ example_resource_free,
+ GFP_KERNEL, &ctx);
+ if (!data)
+ return -ENOMEM;
+ kunit_register_params_array(test, example_params_array,
+ ARRAY_SIZE(example_params_array));
+ return 0;
+ }
+
+ /* Example test that uses shared resources in test->resources. */
+ static void example_params_test_with_init(struct kunit *test)
+ {
+ int threshold;
+ const struct example_param *param = test->param_value;
+ /* Here we pass test->parent to access the parameterized test context. */
+ struct kunit_resource *res = kunit_find_resource(test->parent,
+ example_resource_alloc_match,
+ NULL);
+
+ threshold = *((int *)res->data);
+ KUNIT_ASSERT_LE(test, param->value, threshold);
+ kunit_put_resource(res);
+ }
+
+ static struct kunit_case example_test_cases[] = {
+ KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init, kunit_array_gen_params,
+ example_param_init, NULL),
+ {}
+ };
+
+As an alternative to using the KUnit Resource API for sharing resources, you can
+place them in ``test->parent->priv``. This serves as a more lightweight method
+for resource storage, best for scenarios where complex resource management is
+not required.
+
+As stated previously ``param_init()`` and ``param_exit()`` get the parameterized
+test context. So, you can directly use ``test->priv`` within ``param_init/exit``
+to manage shared resources. However, from within the test case function, you must
+navigate up to the parent ``struct kunit`` i.e. the parameterized test context.
+Therefore, you need to use ``test->parent->priv`` to access those same
+resources.
+
+The resources placed in ``test->parent->priv`` will need to be allocated in
+memory to persist across the parameter runs. If memory is allocated using the
+KUnit memory allocation APIs (described more in the "Allocating Memory" section
+below), you won't need to worry about deallocation. The APIs will make the memory
+parameterized test 'managed', ensuring that it will automatically get cleaned up
+after the parameterized test concludes.
+
+The code below demonstrates example usage of the ``priv`` field for shared
+resources:
+
+.. code-block:: c
+
+ static const struct example_param {
+ int value;
+ } example_params_array[] = {
+ { .value = 3, },
+ { .value = 2, },
+ { .value = 1, },
+ { .value = 0, },
+ };
+
+ /* Initialize the parameterized test context. */
+ static int example_param_init_priv(struct kunit *test)
+ {
+ int ctx = 3; /* Data to be stored. */
+ int arr_size = ARRAY_SIZE(example_params_array);
+
+ /*
+ * Allocate memory using kunit_kzalloc(). Since the `param_init`
+ * function receives the parameterized test context, this memory
+ * allocation will be scoped to the lifetime of the parameterized test.
+ */
+ test->priv = kunit_kzalloc(test, sizeof(int), GFP_KERNEL);
+
+ /* Assign the context value to test->priv.*/
+ *((int *)test->priv) = ctx;
+
+ /* Register the parameter array. */
+ kunit_register_params_array(test, example_params_array, arr_size, NULL);
+ return 0;
+ }
+
+ static void example_params_test_with_init_priv(struct kunit *test)
+ {
+ int threshold;
+ const struct example_param *param = test->param_value;
+
+ /* By design, test->parent will not be NULL. */
+ KUNIT_ASSERT_NOT_NULL(test, test->parent);
+
+ /* Here we use test->parent->priv to access the shared resource. */
+ threshold = *(int *)test->parent->priv;
+
+ KUNIT_ASSERT_LE(test, param->value, threshold);
+ }
+
+ static struct kunit_case example_tests[] = {
+ KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init_priv,
+ kunit_array_gen_params,
+ example_param_init_priv, NULL),
+ {}
+ };
+
Allocating Memory
-----------------
--
2.51.0.261.g7ce5a0a67e-goog
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
` (6 preceding siblings ...)
2025-08-26 9:13 ` [PATCH v4 7/7] Documentation: kunit: Document new parameterized test features David Gow
@ 2025-08-28 16:56 ` Mark Rutland
7 siblings, 0 replies; 10+ messages in thread
From: Mark Rutland @ 2025-08-28 16:56 UTC (permalink / raw)
To: David Gow
Cc: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins,
elver, dvyukov, lucas.demarchi, thomas.hellstrom, rodrigo.vivi,
linux-kselftest, kunit-dev, kasan-dev, intel-xe, dri-devel,
linux-kernel, Stephen Rothwell
On Tue, Aug 26, 2025 at 05:13:30PM +0800, David Gow wrote:
> Hi all,
>
> This is a new version of Marie's patch series, with a couple of extra
> fixes squashed in, notably:
> - drm/xe/tests: Fix some additional gen_params signatures
> https://lore.kernel.org/linux-kselftest/20250821135447.1618942-1-davidgow@google.com/
> - kunit: Only output a test plan if we're using kunit_array_gen_params
> https://lore.kernel.org/linux-kselftest/20250821135447.1618942-2-davidgow@google.com/
>
> These should fix the issues found in linux-next here:
> https://lore.kernel.org/linux-next/20250818120846.347d64b1@canb.auug.org.au/
>
> These changes only affect patches 3 and 4 of the series, the others are
> unchanged from v3.
>
> Thanks, everyone, and sorry for the inconvenience!
Thanks for this!
I had a go at converting some of my aarch64 instruction encoding tests
over to this, and having the ability to dynamically generate the params
array before iterating over the case makes that much easier to handle.
FWIW, for the series:
Acked-by: Mark Rutland <mark.rutland@arm.com>
I'll see about getting those converted over and posted once this is in.
Mark.
>
> Cheers,
> -- David
>
> ---
>
> Hello!
>
> KUnit offers a parameterized testing framework, where tests can be
> run multiple times with different inputs. However, the current
> implementation uses the same `struct kunit` for each parameter run.
> After each run, the test context gets cleaned up, which creates
> the following limitations:
>
> a. There is no way to store resources that are accessible across
> the individual parameter runs.
> b. It's not possible to pass additional context, besides the previous
> parameter (and potentially anything else that is stored in the current
> test context), to the parameter generator function.
> c. Test users are restricted to using pre-defined static arrays
> of parameter objects or generate_params() to define their
> parameters. There is no flexibility to make a custom dynamic
> array without using generate_params(), which can be complex if
> generating the next parameter depends on more than just the single
> previous parameter.
>
> This patch series resolves these limitations by:
>
> 1. [P 1] Giving each parameterized run its own `struct kunit`. It will
> remove the need to manage state, such as resetting the `test->priv`
> field or the `test->status_comment` after every parameter run.
>
> 2. [P 1] Introducing parameterized test context available to all
> parameter runs through the parent pointer of type `struct kunit`.
> This context won't be used to execute any test logic, but will
> instead be used for storing shared resources. Each parameter run
> context will have a reference to that parent instance and thus,
> have access to those resources.
>
> 3. [P 2] Introducing param_init() and param_exit() functions that can
> initialize and exit the parameterized test context. They will run once
> before and after the parameterized test. param_init() can be used to add
> resources to share between parameter runs, pass parameter arrays, and
> any other setup logic. While param_exit() can be used to clean up
> resources that were not managed by the parameterized test, and
> any other teardown logic.
>
> 4. [P 3] Passing the parameterized test context as an additional argument
> to generate_params(). This provides generate_params() with more context,
> making parameter generation much more flexible. The generate_params()
> implementations in the KCSAN and drm/xe tests have been adapted to match
> the new function pointer signature.
>
> 5. [P 4] Introducing a `params_array` field in `struct kunit`. This will
> allow the parameterized test context to have direct storage of the
> parameter array, enabling features like using dynamic parameter arrays
> or using context beyond just the previous parameter. This will also
> enable outputting the KTAP test plan for a parameterized test when the
> parameter count is available.
>
> Patches 5 and 6 add examples tests to lib/kunit/kunit-example-test.c to
> showcase the new features and patch 7 updates the KUnit documentation
> to reflect all the framework changes.
>
> Thank you!
> -Marie
>
> ---
>
> Changes in v4:
>
> Link to v3 of this patch series:
> https://lore.kernel.org/linux-kselftest/20250815103604.3857930-1-marievic@google.com/
>
> - Fixup the signatures of some more gen_params functions in the drm/xe
> driver.
> - Only print a KTAP test plan if a parameterised test is using the
> built-in kunit_array_gen_params generating function, fixing the issues
> with generator functions which skip array elements.
>
> Changes in v3:
>
> Link to v2 of this patch series:
> https://lore.kernel.org/all/20250811221739.2694336-1-marievic@google.com/
>
> - Added logic for skipping the parameter runs and updating the test statistics
> when parameterized test initialization fails.
> - Minor changes to the documentation.
> - Commit message formatting.
>
> Changes in v2:
>
> Link to v1 of this patch series:
> https://lore.kernel.org/all/20250729193647.3410634-1-marievic@google.com/
>
> - Establish parameterized testing terminology:
> - "parameterized test" will refer to the group of all runs of a single test
> function with different parameters.
> - "parameter run" will refer to the execution of the test case function with
> a single parameter.
> - "parameterized test context" is the `struct kunit` that holds the context
> for the entire parameterized test.
> - "parameter run context" is the `struct kunit` that holds the context of the
> individual parameter run.
> - A test is defined to be a parameterized tests if it was registered with a
> generator function.
> - Make comment edits to reflect the established terminology.
> - Require users to manually pass kunit_array_gen_params() to
> KUNIT_CASE_PARAM_WITH_INIT() as the generator function, unless they want to
> provide their own generator function, if the parameter array was registered
> in param_init(). This is to be consistent with the definition of a
> parameterized test, i.e. generate_params() is never NULL if it's
> a parameterized test.
> - Change name of kunit_get_next_param_and_desc() to
> kunit_array_gen_params().
> - Other minor function name changes such as removing the "__" prefix in front
> of internal functions.
> - Change signature of get_description() in `struct params_array` to accept
> the parameterized test context, as well.
> - Output the KTAP test plan for a parameterized test when the parameter count
> is available.
> - Cover letter was made more concise.
> - Edits to the example tests.
> - Fix bug of parameterized test init/exit logic being done outside of the
> parameterized test check.
> - Fix bugs identified by the kernel test robot.
>
> ---
>
> Marie Zhussupova (7):
> kunit: Add parent kunit for parameterized test context
> kunit: Introduce param_init/exit for parameterized test context
> management
> kunit: Pass parameterized test context to generate_params()
> kunit: Enable direct registration of parameter arrays to a KUnit test
> kunit: Add example parameterized test with shared resource management
> using the Resource API
> kunit: Add example parameterized test with direct dynamic parameter
> array setup
> Documentation: kunit: Document new parameterized test features
>
> Documentation/dev-tools/kunit/usage.rst | 342 +++++++++++++++++++++++-
> drivers/gpu/drm/xe/tests/xe_pci.c | 14 +-
> drivers/gpu/drm/xe/tests/xe_pci_test.h | 9 +-
> include/kunit/test.h | 95 ++++++-
> kernel/kcsan/kcsan_test.c | 2 +-
> lib/kunit/kunit-example-test.c | 217 +++++++++++++++
> lib/kunit/test.c | 94 +++++--
> rust/kernel/kunit.rs | 4 +
> 8 files changed, 740 insertions(+), 37 deletions(-)
>
> --
> 2.51.0.261.g7ce5a0a67e-goog
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v4 3/7] kunit: Pass parameterized test context to generate_params()
2025-08-26 9:13 ` [PATCH v4 3/7] kunit: Pass parameterized test context to generate_params() David Gow
@ 2025-08-28 22:19 ` Lucas De Marchi
0 siblings, 0 replies; 10+ messages in thread
From: Lucas De Marchi @ 2025-08-28 22:19 UTC (permalink / raw)
To: David Gow
Cc: Marie Zhussupova, marievictoria875, rmoar, shuah, brendan.higgins,
mark.rutland, elver, dvyukov, thomas.hellstrom, rodrigo.vivi,
linux-kselftest, kunit-dev, kasan-dev, intel-xe, dri-devel,
linux-kernel, Stephen Rothwell
On Tue, Aug 26, 2025 at 05:13:33PM +0800, David Gow wrote:
>From: Marie Zhussupova <marievic@google.com>
>
>To enable more complex parameterized testing scenarios, the
>generate_params() function needs additional context beyond just
>the previously generated parameter. This patch modifies the
>generate_params() function signature to include an extra
>`struct kunit *test` argument, giving test users access to the
>parameterized test context when generating parameters.
>
>The `struct kunit *test` argument was added as the first parameter
>to the function signature as it aligns with the convention of other
>KUnit functions that accept `struct kunit *test` first. This also
>mirrors the "this" or "self" reference found in object-oriented
>programming languages.
>
>This patch also modifies xe_pci_live_device_gen_param() in xe_pci.c
>and nthreads_gen_params() in kcsan_test.c to reflect this signature
>change.
>
>Reviewed-by: David Gow <davidgow@google.com>
>Reviewed-by: Rae Moar <rmoar@google.com>
>Acked-by: Marco Elver <elver@google.com>
>Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
>Signed-off-by: Marie Zhussupova <marievic@google.com>
>[Catch some additional gen_params signatures in drm/xe/tests --David]
>Signed-off-by: David Gow <davidgow@google.com>
>---
>
>Changes in v4:
>v3: https://lore.kernel.org/linux-kselftest/20250815103604.3857930-4-marievic@google.com/
>- Fixup some additional generate_params signature changes in xe_pci.
>- These are also available as a separate patch here:
> https://lore.kernel.org/linux-kselftest/20250821135447.1618942-1-davidgow@google.com/
>
>Changes in v3:
>v2: https://lore.kernel.org/all/20250811221739.2694336-4-marievic@google.com/
>- Commit message formatting.
>
>Changes in v2:
>v1: https://lore.kernel.org/all/20250729193647.3410634-4-marievic@google.com/
> https://lore.kernel.org/all/20250729193647.3410634-5-marievic@google.com/
> https://lore.kernel.org/all/20250729193647.3410634-6-marievic@google.com/
>- generate_params signature changes in xe_pci.c and kcsan_test.c were
> squashed into a single patch to avoid in-between breakages in the series.
>- The comments and the commit message were changed to reflect the
> parameterized testing terminology. See the patch series cover letter
> change log for the definitions.
>
>---
> drivers/gpu/drm/xe/tests/xe_pci.c | 14 +++++++-------
> drivers/gpu/drm/xe/tests/xe_pci_test.h | 9 +++++----
Acked-by: Lucas De Marchi <lucas.demarchi@intel.com>
thanks
Lucas De Marchi
> include/kunit/test.h | 9 ++++++---
> kernel/kcsan/kcsan_test.c | 2 +-
> lib/kunit/test.c | 5 +++--
> 5 files changed, 22 insertions(+), 17 deletions(-)
>
>diff --git a/drivers/gpu/drm/xe/tests/xe_pci.c b/drivers/gpu/drm/xe/tests/xe_pci.c
>index 9c715e59f030..f707e0a54295 100644
>--- a/drivers/gpu/drm/xe/tests/xe_pci.c
>+++ b/drivers/gpu/drm/xe/tests/xe_pci.c
>@@ -44,9 +44,9 @@ KUNIT_ARRAY_PARAM(pci_id, pciidlist, xe_pci_id_kunit_desc);
> *
> * Return: pointer to the next parameter or NULL if no more parameters
> */
>-const void *xe_pci_graphics_ip_gen_param(const void *prev, char *desc)
>+const void *xe_pci_graphics_ip_gen_param(struct kunit *test, const void *prev, char *desc)
> {
>- return graphics_ip_gen_params(prev, desc);
>+ return graphics_ip_gen_params(test, prev, desc);
> }
> EXPORT_SYMBOL_IF_KUNIT(xe_pci_graphics_ip_gen_param);
>
>@@ -61,9 +61,9 @@ EXPORT_SYMBOL_IF_KUNIT(xe_pci_graphics_ip_gen_param);
> *
> * Return: pointer to the next parameter or NULL if no more parameters
> */
>-const void *xe_pci_media_ip_gen_param(const void *prev, char *desc)
>+const void *xe_pci_media_ip_gen_param(struct kunit *test, const void *prev, char *desc)
> {
>- return media_ip_gen_params(prev, desc);
>+ return media_ip_gen_params(test, prev, desc);
> }
> EXPORT_SYMBOL_IF_KUNIT(xe_pci_media_ip_gen_param);
>
>@@ -78,9 +78,9 @@ EXPORT_SYMBOL_IF_KUNIT(xe_pci_media_ip_gen_param);
> *
> * Return: pointer to the next parameter or NULL if no more parameters
> */
>-const void *xe_pci_id_gen_param(const void *prev, char *desc)
>+const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc)
> {
>- const struct pci_device_id *pci = pci_id_gen_params(prev, desc);
>+ const struct pci_device_id *pci = pci_id_gen_params(test, prev, desc);
>
> return pci->driver_data ? pci : NULL;
> }
>@@ -159,7 +159,7 @@ EXPORT_SYMBOL_IF_KUNIT(xe_pci_fake_device_init);
> * Return: pointer to the next &struct xe_device ready to be used as a parameter
> * or NULL if there are no more Xe devices on the system.
> */
>-const void *xe_pci_live_device_gen_param(const void *prev, char *desc)
>+const void *xe_pci_live_device_gen_param(struct kunit *test, const void *prev, char *desc)
> {
> const struct xe_device *xe = prev;
> struct device *dev = xe ? xe->drm.dev : NULL;
>diff --git a/drivers/gpu/drm/xe/tests/xe_pci_test.h b/drivers/gpu/drm/xe/tests/xe_pci_test.h
>index ce4d2b86b778..6d8bc56f7bde 100644
>--- a/drivers/gpu/drm/xe/tests/xe_pci_test.h
>+++ b/drivers/gpu/drm/xe/tests/xe_pci_test.h
>@@ -7,6 +7,7 @@
> #define _XE_PCI_TEST_H_
>
> #include <linux/types.h>
>+#include <kunit/test.h>
>
> #include "xe_platform_types.h"
> #include "xe_sriov_types.h"
>@@ -25,9 +26,9 @@ struct xe_pci_fake_data {
>
> int xe_pci_fake_device_init(struct xe_device *xe);
>
>-const void *xe_pci_graphics_ip_gen_param(const void *prev, char *desc);
>-const void *xe_pci_media_ip_gen_param(const void *prev, char *desc);
>-const void *xe_pci_id_gen_param(const void *prev, char *desc);
>-const void *xe_pci_live_device_gen_param(const void *prev, char *desc);
>+const void *xe_pci_graphics_ip_gen_param(struct kunit *test, const void *prev, char *desc);
>+const void *xe_pci_media_ip_gen_param(struct kunit *test, const void *prev, char *desc);
>+const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc);
>+const void *xe_pci_live_device_gen_param(struct kunit *test, const void *prev, char *desc);
>
> #endif
>diff --git a/include/kunit/test.h b/include/kunit/test.h
>index fc8fd55b2dfb..8eba1b03c3e3 100644
>--- a/include/kunit/test.h
>+++ b/include/kunit/test.h
>@@ -128,7 +128,8 @@ struct kunit_attributes {
> struct kunit_case {
> void (*run_case)(struct kunit *test);
> const char *name;
>- const void* (*generate_params)(const void *prev, char *desc);
>+ const void* (*generate_params)(struct kunit *test,
>+ const void *prev, char *desc);
> struct kunit_attributes attr;
> int (*param_init)(struct kunit *test);
> void (*param_exit)(struct kunit *test);
>@@ -1703,7 +1704,8 @@ do { \
> * Define function @name_gen_params which uses @array to generate parameters.
> */
> #define KUNIT_ARRAY_PARAM(name, array, get_desc) \
>- static const void *name##_gen_params(const void *prev, char *desc) \
>+ static const void *name##_gen_params(struct kunit *test, \
>+ const void *prev, char *desc) \
> { \
> typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
> if (__next - (array) < ARRAY_SIZE((array))) { \
>@@ -1724,7 +1726,8 @@ do { \
> * Define function @name_gen_params which uses @array to generate parameters.
> */
> #define KUNIT_ARRAY_PARAM_DESC(name, array, desc_member) \
>- static const void *name##_gen_params(const void *prev, char *desc) \
>+ static const void *name##_gen_params(struct kunit *test, \
>+ const void *prev, char *desc) \
> { \
> typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
> if (__next - (array) < ARRAY_SIZE((array))) { \
>diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c
>index 49ab81faaed9..a13a090bb2a7 100644
>--- a/kernel/kcsan/kcsan_test.c
>+++ b/kernel/kcsan/kcsan_test.c
>@@ -1383,7 +1383,7 @@ static void test_atomic_builtins_missing_barrier(struct kunit *test)
> * The thread counts are chosen to cover potentially interesting boundaries and
> * corner cases (2 to 5), and then stress the system with larger counts.
> */
>-static const void *nthreads_gen_params(const void *prev, char *desc)
>+static const void *nthreads_gen_params(struct kunit *test, const void *prev, char *desc)
> {
> long nthreads = (long)prev;
>
>diff --git a/lib/kunit/test.c b/lib/kunit/test.c
>index 0fe61dec5a96..50705248abad 100644
>--- a/lib/kunit/test.c
>+++ b/lib/kunit/test.c
>@@ -700,7 +700,7 @@ int kunit_run_tests(struct kunit_suite *suite)
> /* Get initial param. */
> param_desc[0] = '\0';
> /* TODO: Make generate_params try-catch */
>- curr_param = test_case->generate_params(NULL, param_desc);
>+ curr_param = test_case->generate_params(&test, NULL, param_desc);
> test_case->status = KUNIT_SKIPPED;
> kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
> "KTAP version 1\n");
>@@ -731,7 +731,8 @@ int kunit_run_tests(struct kunit_suite *suite)
>
> /* Get next param. */
> param_desc[0] = '\0';
>- curr_param = test_case->generate_params(curr_param, param_desc);
>+ curr_param = test_case->generate_params(&test, curr_param,
>+ param_desc);
> }
> /*
> * TODO: Put into a try catch. Since we don't need suite->exit
>--
>2.51.0.261.g7ce5a0a67e-goog
>
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2025-08-28 22:19 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-26 9:13 [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework David Gow
2025-08-26 9:13 ` [PATCH v4 1/7] kunit: Add parent kunit for parameterized test context David Gow
2025-08-26 9:13 ` [PATCH v4 2/7] kunit: Introduce param_init/exit for parameterized test context management David Gow
2025-08-26 9:13 ` [PATCH v4 3/7] kunit: Pass parameterized test context to generate_params() David Gow
2025-08-28 22:19 ` Lucas De Marchi
2025-08-26 9:13 ` [PATCH v4 4/7] kunit: Enable direct registration of parameter arrays to a KUnit test David Gow
2025-08-26 9:13 ` [PATCH v4 5/7] kunit: Add example parameterized test with shared resource management using the Resource API David Gow
2025-08-26 9:13 ` [PATCH v4 6/7] kunit: Add example parameterized test with direct dynamic parameter array setup David Gow
2025-08-26 9:13 ` [PATCH v4 7/7] Documentation: kunit: Document new parameterized test features David Gow
2025-08-28 16:56 ` [PATCH v4 0/7] kunit: Refactor and extend KUnit's parameterized testing framework Mark Rutland
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).