* [PATCH v2 01/12] test: Add ut_asserteq_regex() for regex pattern matching
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 02/12] test: Add a helper to check the next line against a regex Simon Glass
` (12 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass
Add a new assertion macro ut_asserteq_regex() that checks if a string
matches a regular expression pattern using the SLRE library.
This is useful for tests where exact string-matching is difficult, such
as when output contains line numbers or other variable content.
Use a helper function ut_check_regex() to avoid including slre.h in the
header.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
include/test/ut.h | 36 +++++++++++++++++++++++++++++-------
test/ut.c | 30 ++++++++++++++++++++++++++++++
2 files changed, 59 insertions(+), 7 deletions(-)
diff --git a/include/test/ut.h b/include/test/ut.h
index 6510c35997f..cda15f3a7a9 100644
--- a/include/test/ut.h
+++ b/include/test/ut.h
@@ -13,6 +13,9 @@
#include <linux/err.h>
#include <test/test.h>
+/* Size of error buffer for ut_check_regex() */
+#define UT_REGEX_ERR_SIZE 256
+
struct unit_test_state;
/**
@@ -41,6 +44,16 @@ void ut_failf(struct unit_test_state *uts, const char *fname, int line,
const char *func, const char *cond, const char *fmt, ...)
__attribute__ ((format (__printf__, 6, 7)));
+/**
+ * ut_check_regex() - Check if a string matches a regex pattern
+ *
+ * @pattern: Regular expression pattern
+ * @str: String to match against
+ * @err: Buffer to hold error message on failure (UT_REGEX_ERR_SIZE bytes)
+ * Return: 0 if match, -EINVAL if pattern is invalid, -ENOENT if no match
+ */
+int ut_check_regex(const char *pattern, const char *str, char *err);
+
/**
* ut_check_console_line() - Check the next console line against expectations
*
@@ -264,13 +277,22 @@ int ut_check_console_dump(struct unit_test_state *uts, int total_bytes);
__ret; \
})
-/**
- * ut_asserteq_mem() - Assert that two memory areas are equal
- *
- * @expr1: expected value
- * @expr2: actual value
- * @len: length of the memory areas
- */
+/* Assert that a string matches a regex pattern */
+#define ut_asserteq_regex(pattern, str) ({ \
+ const char *_pattern = (pattern), *_str = (str); \
+ char _err[UT_REGEX_ERR_SIZE]; \
+ int __ret = 0; \
+ \
+ __ret = ut_check_regex(_pattern, _str, _err); \
+ if (__ret) { \
+ ut_failf(uts, __FILE__, __LINE__, __func__, \
+ #pattern " matches " #str, "%s", _err); \
+ return CMD_RET_FAILURE; \
+ } \
+ __ret; \
+})
+
+/* Assert that two memory areas are equal */
#define ut_asserteq_mem(expr1, expr2, len) ({ \
const u8 *_val1 = (u8 *)(expr1), *_val2 = (u8 *)(expr2); \
const uint __len = len; \
diff --git a/test/ut.c b/test/ut.c
index a16fdfb3a93..b4f2a8bf40f 100644
--- a/test/ut.c
+++ b/test/ut.c
@@ -6,7 +6,10 @@
*/
#include <console.h>
+#include <errno.h>
#include <malloc.h>
+#include <slre.h>
+#include <vsprintf.h>
#ifdef CONFIG_SANDBOX
#include <asm/state.h>
#endif
@@ -38,6 +41,33 @@ void ut_failf(struct unit_test_state *uts, const char *fname, int line,
uts->cur.fail_count++;
}
+int ut_check_regex(const char *pattern, const char *str, char *err)
+{
+ struct slre slre;
+
+ if (!pattern || !str) {
+ snprintf(err, UT_REGEX_ERR_SIZE,
+ "NULL value: pattern=%s, str=%s",
+ pattern ? pattern : "(null)",
+ str ? str : "(null)");
+ return -EINVAL;
+ }
+
+ if (!slre_compile(&slre, pattern)) {
+ snprintf(err, UT_REGEX_ERR_SIZE,
+ "Invalid regex '%s': %s", pattern, slre.err_str);
+ return -EINVAL;
+ }
+
+ if (!slre_match(&slre, str, strlen(str), NULL)) {
+ snprintf(err, UT_REGEX_ERR_SIZE,
+ "No match: pattern '%s', str '%s'", pattern, str);
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
ulong ut_check_free(void)
{
struct mallinfo info = mallinfo();
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 02/12] test: Add a helper to check the next line against a regex
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
2026-04-12 11:19 ` [PATCH v2 01/12] test: Add ut_asserteq_regex() for regex pattern matching Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 03/12] test: Add a private buffer for tests Simon Glass
` (11 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass
Add a new ut_assert_nextline_regex() macro and
ut_check_console_line_regex() helper to check console output against a
regex pattern. This is useful when the exact output varies (e.g., file
paths or line numbers in error messages).
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
include/test/ut.h | 33 +++++++++++++++++++++++++++++----
test/ut.c | 21 +++++++++++++++++++++
2 files changed, 50 insertions(+), 4 deletions(-)
diff --git a/include/test/ut.h b/include/test/ut.h
index cda15f3a7a9..8f0e69a7734 100644
--- a/include/test/ut.h
+++ b/include/test/ut.h
@@ -87,6 +87,20 @@ int ut_check_console_line(struct unit_test_state *uts, const char *fmt, ...)
int ut_check_console_linen(struct unit_test_state *uts, const char *fmt, ...)
__attribute__ ((format (__printf__, 2, 3)));
+/**
+ * ut_check_console_line_regex() - Check the next console line against a regex
+ *
+ * This checks the next line of console output against a regex pattern.
+ *
+ * After the function returns, uts->expect_str holds the regex pattern and
+ * uts->actual_str holds the actual string read from the console.
+ *
+ * @uts: Test state
+ * @regex: Regular expression pattern to match against
+ * Return: 0 if OK, other value on error
+ */
+int ut_check_console_line_regex(struct unit_test_state *uts, const char *regex);
+
/**
* ut_check_skipline() - Check that the next console line exists and skip it
*
@@ -451,10 +465,21 @@ int ut_check_console_dump(struct unit_test_state *uts, int total_bytes);
__ret; \
})
-/**
- * ut_assert_skipline() - Assert that there is a 'next' console output line,
- * and skip it
- */
+/* Assert that the next console output line matches a regex pattern */
+#define ut_assert_nextline_regex(pattern) ({ \
+ int __ret = 0; \
+ \
+ if (ut_check_console_line_regex(uts, pattern)) { \
+ ut_failf(uts, __FILE__, __LINE__, __func__, \
+ "console regex", \
+ "\nExpected regex '%s',\n got '%s'", \
+ uts->expect_str, uts->actual_str); \
+ return CMD_RET_FAILURE; \
+ } \
+ __ret; \
+})
+
+/* Assert that there is a 'next' console output line, and skip it */
#define ut_assert_skipline() ({ \
int __ret = 0; \
\
diff --git a/test/ut.c b/test/ut.c
index b4f2a8bf40f..b0cc0f3e8ff 100644
--- a/test/ut.c
+++ b/test/ut.c
@@ -139,6 +139,27 @@ int ut_check_console_linen(struct unit_test_state *uts, const char *fmt, ...)
strlen(uts->expect_str));
}
+int ut_check_console_line_regex(struct unit_test_state *uts, const char *regex)
+{
+ char err[UT_REGEX_ERR_SIZE];
+ int len;
+ int ret;
+
+ len = strlcpy(uts->expect_str, regex, sizeof(uts->expect_str));
+ if (len >= sizeof(uts->expect_str)) {
+ ut_fail(uts, __FILE__, __LINE__, __func__,
+ "unit_test_state->expect_str too small");
+ return -EOVERFLOW;
+ }
+ ret = readline_check(uts);
+ if (ret == -ENOENT)
+ return 1;
+
+ ret = ut_check_regex(regex, uts->actual_str, err);
+
+ return ret;
+}
+
int ut_check_skipline(struct unit_test_state *uts)
{
int ret;
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 03/12] test: Add a private buffer for tests
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
2026-04-12 11:19 ` [PATCH v2 01/12] test: Add ut_asserteq_regex() for regex pattern matching Simon Glass
2026-04-12 11:19 ` [PATCH v2 02/12] test: Add a helper to check the next line against a regex Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 04/12] test: Add argument-type definitions Simon Glass
` (10 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass, Simon Glass
From: Simon Glass <simon.glass@canonical.com>
Add a priv[] buffer to struct unit_test_state that tests can use for
their own data. This avoids the need to allocate memory or use global
variables for test-specific state.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
---
(no changes since v1)
include/test/test.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/test/test.h b/include/test/test.h
index 0f2b68a5dee..f8a80a88959 100644
--- a/include/test/test.h
+++ b/include/test/test.h
@@ -9,6 +9,8 @@
#include <malloc.h>
#include <linux/bitops.h>
+#define UT_PRIV_SIZE 256
+
/**
* struct ut_stats - Statistics about tests run
*
@@ -53,6 +55,7 @@ struct ut_stats {
* @old_bloblist: stores the old gd->bloblist pointer
* @expect_str: Temporary string used to hold expected string value
* @actual_str: Temporary string used to hold actual string value
+ * @priv: Private data for tests to use as needed
*/
struct unit_test_state {
struct ut_stats cur;
@@ -78,6 +81,7 @@ struct unit_test_state {
void *old_bloblist;
char expect_str[512];
char actual_str[512];
+ char priv[UT_PRIV_SIZE];
};
/* Test flags for each test */
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 04/12] test: Add argument-type definitions
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (2 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 03/12] test: Add a private buffer for tests Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 05/12] test: Add a macro to declare unit tests with arguments Simon Glass
` (9 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass
Add types for declaring and storing unit test arguments:
- enum ut_arg_type: INT, BOOL, STR types
- enum ut_arg_flags: OPTIONAL flag for non-required args
- struct ut_arg_def: declares expected args with defaults
- struct ut_arg: holds parsed argument values
This prepares for passing key=value arguments to tests via the 'ut'
command instead of needing to use environment variables.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
include/test/test.h | 65 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)
diff --git a/include/test/test.h b/include/test/test.h
index f8a80a88959..c9b56c66eae 100644
--- a/include/test/test.h
+++ b/include/test/test.h
@@ -29,6 +29,40 @@ struct ut_stats {
ulong duration_ms;
};
+/**
+ * enum ut_arg_type - Type of a unit test argument
+ *
+ * @UT_ARG_INT: Integer argument (hex with 0x prefix, or decimal) -> vint
+ * @UT_ARG_BOOL: Boolean argument (0 or 1) -> vbool
+ * @UT_ARG_STR: String argument -> vstr
+ */
+enum ut_arg_type {
+ UT_ARG_INT,
+ UT_ARG_BOOL,
+ UT_ARG_STR,
+};
+
+/**
+ * struct ut_arg - Parsed unit test argument value
+ *
+ * Holds the parsed value of an argument after command-line processing.
+ *
+ * @name: Name of the argument (points to ut_arg_def.name)
+ * @type: Type of the argument
+ * @vint: Integer value (when type is UT_ARG_INT)
+ * @vbool: Boolean value (when type is UT_ARG_BOOL)
+ * @vstr: String value (when type is UT_ARG_STR, points into argv)
+ */
+struct ut_arg {
+ const char *name;
+ enum ut_arg_type type;
+ union {
+ long vint;
+ bool vbool;
+ const char *vstr;
+ };
+};
+
/*
* struct unit_test_state - Entire state of test system
*
@@ -108,6 +142,37 @@ enum ut_flags {
UTF_UNINIT = BIT(13), /* test uninits a suite */
};
+/**
+ * enum ut_arg_flags - Flags for unit test arguments
+ *
+ * @UT_ARGF_OPTIONAL: Argument is optional; use default value if not provided
+ */
+enum ut_arg_flags {
+ UT_ARGF_OPTIONAL = BIT(0),
+};
+
+/**
+ * struct ut_arg_def - Definition of a unit test argument
+ *
+ * Declares an expected argument for a test, including its name, type,
+ * whether it is optional, and its default value.
+ *
+ * @name: Name of the argument (used in key=value matching)
+ * @type: Type of the argument (int, bool, or string)
+ * @flags: Argument flags (e.g., UT_ARGF_OPTIONAL)
+ * @def: Default value (used when argument is optional and not provided)
+ */
+struct ut_arg_def {
+ const char *name;
+ enum ut_arg_type type;
+ int flags;
+ union {
+ long vint;
+ bool vbool;
+ const char *vstr;
+ } def;
+};
+
/**
* struct unit_test - Information about a unit test
*
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 05/12] test: Add a macro to declare unit tests with arguments
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (3 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 04/12] test: Add argument-type definitions Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 06/12] test: Add support for passing arguments to C tests Simon Glass
` (8 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass
Add a UNIT_TEST_ARGS() macro to declare and register a test with inline
argument definitions. The variadic arguments use struct ut_arg_def and
a NULL terminator is added automatically to the list.
Example usage:
static int my_test(struct unit_test_state *uts) { ... }
UNIT_TEST_ARGS(my_test, UTF_CONSOLE, my_suite,
{ "path", UT_ARG_STR },
{ "count", UT_ARG_INT, UT_ARGF_OPTIONAL, { .vint = 10 } });
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
include/test/test.h | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/include/test/test.h b/include/test/test.h
index c9b56c66eae..cdd19392844 100644
--- a/include/test/test.h
+++ b/include/test/test.h
@@ -49,6 +49,7 @@ enum ut_arg_type {
*
* @name: Name of the argument (points to ut_arg_def.name)
* @type: Type of the argument
+ * @provided: true if value was provided on command line
* @vint: Integer value (when type is UT_ARG_INT)
* @vbool: Boolean value (when type is UT_ARG_BOOL)
* @vstr: String value (when type is UT_ARG_STR, points into argv)
@@ -56,6 +57,7 @@ enum ut_arg_type {
struct ut_arg {
const char *name;
enum ut_arg_type type;
+ bool provided;
union {
long vint;
bool vbool;
@@ -179,12 +181,14 @@ struct ut_arg_def {
* @name: Name of test
* @func: Function to call to perform test
* @flags: Flags indicated pre-conditions for test
+ * @arg_defs: Argument definitions (NULL-terminated array), or NULL
*/
struct unit_test {
const char *file;
const char *name;
int (*func)(struct unit_test_state *state);
int flags;
+ const struct ut_arg_def *arg_defs;
};
/**
@@ -236,6 +240,33 @@ struct unit_test {
.func = _name, \
}
+/**
+ * UNIT_TEST_ARGS() - create unit test entry with inline argument definitions
+ *
+ * Like UNIT_TEST() but allows specifying argument definitions inline.
+ * The variadic arguments are struct ut_arg_def initializers. The NULL
+ * terminator is added automatically by the macro.
+ *
+ * Example:
+ * UNIT_TEST_ARGS(my_test, UTF_CONSOLE, my_suite,
+ * { "path", UT_ARG_STR },
+ * { "count", UT_ARG_INT, UT_ARGF_OPTIONAL, { .vint = 10 } });
+ *
+ * @_name: Test function name
+ * @_flags: Test flags (see enum ut_flags)
+ * @_suite: Test suite name
+ * @...: Argument definitions (struct ut_arg_def initializers)
+ */
+#define UNIT_TEST_ARGS(_name, _flags, _suite, ...) \
+ static const struct ut_arg_def _name##_args[] = { __VA_ARGS__, { NULL } }; \
+ ll_entry_declare(struct unit_test, _name, ut_ ## _suite) = { \
+ .file = __FILE__, \
+ .name = #_name, \
+ .flags = _flags, \
+ .func = _name, \
+ .arg_defs = _name##_args, \
+ }
+
/* Get the start of a list of unit tests for a particular suite */
#define UNIT_TEST_SUITE_START(_suite) \
ll_entry_start(struct unit_test, ut_ ## _suite)
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 06/12] test: Add support for passing arguments to C tests
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (4 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 05/12] test: Add a macro to declare unit tests with arguments Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 07/12] test: Enhance the ut command to pass test arguments Simon Glass
` (7 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass, Andrew Goodbody
Add support for passing key=value arguments to unit tests. The test
framework parses arguments based on definitions provided by each test
and makes them available via uts->args[]
For now the 'ut' command does not support this new feature.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
arch/sandbox/cpu/spl.c | 3 +-
include/test/test.h | 5 ++
include/test/ut.h | 4 +-
test/cmd_ut.c | 3 +-
test/test-main.c | 169 ++++++++++++++++++++++++++++++++++++++++-
5 files changed, 177 insertions(+), 7 deletions(-)
diff --git a/arch/sandbox/cpu/spl.c b/arch/sandbox/cpu/spl.c
index 7ee4975523e..162b678314e 100644
--- a/arch/sandbox/cpu/spl.c
+++ b/arch/sandbox/cpu/spl.c
@@ -152,7 +152,8 @@ void spl_board_init(void)
ut_init_state(&uts);
ret = ut_run_list(&uts, "spl", NULL, tests, count,
- state->select_unittests, 1, false, NULL);
+ state->select_unittests, 1, false, NULL,
+ 0, NULL);
ut_report(&uts.cur, 1);
ut_uninit_state(&uts);
/* continue execution into U-Boot */
diff --git a/include/test/test.h b/include/test/test.h
index cdd19392844..bd47025ce50 100644
--- a/include/test/test.h
+++ b/include/test/test.h
@@ -9,6 +9,7 @@
#include <malloc.h>
#include <linux/bitops.h>
+#define UT_MAX_ARGS 8
#define UT_PRIV_SIZE 256
/**
@@ -92,6 +93,8 @@ struct ut_arg {
* @expect_str: Temporary string used to hold expected string value
* @actual_str: Temporary string used to hold actual string value
* @priv: Private data for tests to use as needed
+ * @args: Parsed argument values for current test
+ * @arg_count: Number of parsed arguments
*/
struct unit_test_state {
struct ut_stats cur;
@@ -118,6 +121,8 @@ struct unit_test_state {
char expect_str[512];
char actual_str[512];
char priv[UT_PRIV_SIZE];
+ struct ut_arg args[UT_MAX_ARGS];
+ int arg_count;
};
/* Test flags for each test */
diff --git a/include/test/ut.h b/include/test/ut.h
index 8f0e69a7734..1c4ee390fd8 100644
--- a/include/test/ut.h
+++ b/include/test/ut.h
@@ -675,12 +675,14 @@ void ut_uninit_state(struct unit_test_state *uts);
* name is the name of the test to run. This is used to find which test causes
* another test to fail. If the one test fails, testing stops immediately.
* Pass NULL to disable this
+ * @argc: Number of test arguments (key=value pairs), 0 if none
+ * @argv: Test argument array, NULL if none
* Return: 0 if all tests passed, -1 if any failed
*/
int ut_run_list(struct unit_test_state *uts, const char *category,
const char *prefix, struct unit_test *tests, int count,
const char *select_name, int runs_per_test, bool force_run,
- const char *test_insert);
+ const char *test_insert, int argc, char *const argv[]);
/**
* ut_report() - Report stats on a test run
diff --git a/test/cmd_ut.c b/test/cmd_ut.c
index 44e5fdfdaa6..7304c32bec5 100644
--- a/test/cmd_ut.c
+++ b/test/cmd_ut.c
@@ -130,7 +130,8 @@ static int run_suite(struct unit_test_state *uts, struct suite *ste,
snprintf(prefix, sizeof(prefix), "%s_test_", ste->name);
ret = ut_run_list(uts, ste->name, prefix, ste->start, n_ents,
- select_name, runs_per_test, force_run, test_insert);
+ select_name, runs_per_test, force_run, test_insert,
+ 0, NULL);
return ret;
}
diff --git a/test/test-main.c b/test/test-main.c
index 3254325e8b1..9d8ab794db8 100644
--- a/test/test-main.c
+++ b/test/test-main.c
@@ -87,6 +87,154 @@ void ut_uninit_state(struct unit_test_state *uts)
}
}
+/**
+ * ut_count_args() - Count the number of arguments in a NULL-terminated array
+ *
+ * @defs: Argument definitions array (NULL-terminated)
+ * Return: Number of arguments
+ */
+static int ut_count_args(const struct ut_arg_def *defs)
+{
+ int count = 0;
+
+ if (defs) {
+ while (defs[count].name)
+ count++;
+ }
+
+ return count;
+}
+
+/**
+ * ut_set_arg() - Find and set an argument value
+ *
+ * Search through argument definitions to find a matching key and set its value.
+ *
+ * @defs: Argument definitions array
+ * @args: Argument values array to update
+ * @count: Number of argument definitions
+ * @key: Key name to search for
+ * @key_len: Length of key name
+ * @val: Value string to parse
+ * Return: true if argument was found and set, false otherwise
+ */
+static bool ut_set_arg(const struct ut_arg_def *defs, struct ut_arg *args,
+ int count, const char *key, int key_len, const char *val)
+{
+ int j;
+
+ for (j = 0; j < count; j++) {
+ if (strlen(defs[j].name) == key_len &&
+ !strncmp(defs[j].name, key, key_len)) {
+ switch (defs[j].type) {
+ case UT_ARG_INT:
+ args[j].vint = simple_strtol(val, NULL, 0);
+ break;
+ case UT_ARG_BOOL:
+ args[j].vbool = *val == '1';
+ break;
+ case UT_ARG_STR:
+ args[j].vstr = val;
+ break;
+ }
+ args[j].provided = true;
+ return true;
+ }
+ }
+
+ return false;
+}
+
+/**
+ * ut_parse_args() - Parse command-line arguments for a test
+ *
+ * Parse key=value arguments from the command line and set up uts->args based on
+ * the test's argument definitions.
+ *
+ * @uts: Unit test state (args and arg_count will be set)
+ * @test: Test being run (provides arg_defs)
+ * @argc: Number of arguments
+ * @argv: Argument array (key=value strings)
+ * Return: 0 on success, -EINVAL on parse error
+ */
+static int ut_parse_args(struct unit_test_state *uts, struct unit_test *test,
+ int argc, char *const argv[])
+{
+ const struct ut_arg_def *defs = test->arg_defs;
+ struct ut_arg *args = uts->args;
+ int count = ut_count_args(defs);
+ int i;
+
+ uts->arg_count = 0;
+
+ /* No arguments expected */
+ if (!count) {
+ if (argc > 0) {
+ printf("Test '%s' does not accept arguments\n",
+ test->name);
+ return -EINVAL;
+ }
+ return 0;
+ }
+
+ if (count > UT_MAX_ARGS) {
+ printf("Test '%s' has too many arguments (%d > %d)\n",
+ test->name, count, UT_MAX_ARGS);
+ return -EINVAL;
+ }
+
+ /* Initialise from defaults */
+ for (i = 0; i < count; i++) {
+ args[i].name = defs[i].name;
+ args[i].type = defs[i].type;
+ args[i].provided = false;
+ switch (defs[i].type) {
+ case UT_ARG_INT:
+ args[i].vint = defs[i].def.vint;
+ break;
+ case UT_ARG_BOOL:
+ args[i].vbool = defs[i].def.vbool;
+ break;
+ case UT_ARG_STR:
+ args[i].vstr = defs[i].def.vstr;
+ break;
+ }
+ }
+
+ /* Parse command-line key=value pairs */
+ for (i = 0; i < argc; i++) {
+ const char *arg = argv[i];
+ const char *eq = strchr(arg, '=');
+ int key_len;
+
+ if (!eq) {
+ printf("Invalid argument '%s' (expected key=value)\n",
+ arg);
+ return -EINVAL;
+ }
+ key_len = eq - arg;
+
+ if (!ut_set_arg(defs, args, count, arg, key_len, eq + 1)) {
+ printf("Unknown argument '%.*s' for test '%s'\n",
+ key_len, arg, test->name);
+ return -EINVAL;
+ }
+ }
+
+ /* Check required arguments are provided */
+ for (i = 0; i < count; i++) {
+ if (!args[i].provided && !(defs[i].flags & UT_ARGF_OPTIONAL)) {
+ printf("Missing required argument '%s' for test '%s'\n",
+ defs[i].name, test->name);
+ return -EINVAL;
+ }
+ }
+
+ uts->arg_count = count;
+
+ return 0;
+}
+
/**
* dm_test_pre_run() - Get ready to run a driver model test
*
@@ -588,12 +736,15 @@ static int ut_run_test_live_flat(struct unit_test_state *uts,
* @test_insert: String describing a test to run after n other tests run, in the
* format n:name where n is the number of tests to run before this one and
* name is the name of the test to run
+ * @argc: Number of test arguments (key=value pairs)
+ * @argv: Test argument array
* Return: 0 if all tests passed, -ENOENT if test @select_name was not found,
* -EBADF if any failed
*/
static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
struct unit_test *tests, int count,
- const char *select_name, const char *test_insert)
+ const char *select_name, const char *test_insert,
+ int argc, char *const argv[])
{
int prefix_len = prefix ? strlen(prefix) : 0;
struct unit_test *test, *one;
@@ -649,6 +800,11 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
uts->cur.test_count++;
if (one && upto == pos) {
+ ret = ut_parse_args(uts, one, argc, argv);
+ if (ret) {
+ uts->cur.fail_count++;
+ return ret;
+ }
ret = ut_run_test_live_flat(uts, one, NULL);
if (uts->cur.fail_count != old_fail_count) {
printf("Test '%s' failed %d times (position %d)\n",
@@ -662,6 +818,12 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix,
if (prefix_len && !strncmp(test_name, prefix, prefix_len))
test_name = test_name + prefix_len;
+ ret = ut_parse_args(uts, test, argc, argv);
+ if (ret) {
+ found++;
+ uts->cur.fail_count++;
+ continue;
+ }
for (i = 0; i < uts->runs_per_test; i++)
ret = ut_run_test_live_flat(uts, test, test_name);
if (uts->cur.fail_count != old_fail_count) {
@@ -701,9 +863,8 @@ void ut_report(struct ut_stats *stats, int run_count)
int ut_run_list(struct unit_test_state *uts, const char *category,
const char *prefix, struct unit_test *tests, int count,
const char *select_name, int runs_per_test, bool force_run,
- const char *test_insert)
+ const char *test_insert, int argc, char *const argv[])
{
- ;
bool has_dm_tests = false;
ulong start_offset = 0;
ulong test_offset = 0;
@@ -745,7 +906,7 @@ int ut_run_list(struct unit_test_state *uts, const char *category,
}
uts->force_run = force_run;
ret = ut_run_tests(uts, prefix, tests, count, select_name,
- test_insert);
+ test_insert, argc, argv);
/* Best efforts only...ignore errors */
if (has_dm_tests)
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 07/12] test: Enhance the ut command to pass test arguments
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (5 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 06/12] test: Add support for passing arguments to C tests Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 08/12] test: Add type-checked argument accessor functions Simon Glass
` (6 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass
Update the ut command to permit passing arguments to tests.
Usage: ut -f fs test_name key1=value1 key2=value2
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
doc/usage/cmd/ut.rst | 11 ++++++++++-
test/cmd_ut.c | 29 ++++++++++++++++++++---------
2 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst
index 1acf3126680..d8c3cbf496c 100644
--- a/doc/usage/cmd/ut.rst
+++ b/doc/usage/cmd/ut.rst
@@ -11,7 +11,7 @@ Synopsis
::
- ut [-r<runs>] [-f] [-I<n>:<one_test>] [-r<n>] [<suite> | 'all' [<test>]]
+ ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
ut [-s] info
Description
@@ -37,6 +37,11 @@ test
causes another test to fail. If the one test fails, testing stops
immediately.
+args
+ Optional arguments to pass to the test, in `name=value` format. These are
+ used by tests declared with `UNIT_TEST_ARGS()` which define expected
+ argument names and types.
+
Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it
includes a near-complete set of emulators, no code-size limits, many CONFIG
options enabled and runs easily in CI without needing QEMU. It is also possible
@@ -201,3 +206,7 @@ Run a selection of three suites::
Tests run: 10, 12 ms, average: 1 ms, failures: 0
Suites run: 3, total tests run: 37, 26 ms, average: 0 ms, failures: 0
Average test time: 0 ms, worst case 'mem' took 1 ms
+
+Run a test with arguments (used by tests declared with UNIT_TEST_ARGS)::
+
+ => ut -f fs fs_test_ls_norun fs_type=ext4 fs_image=/tmp/img small=1MB.file
diff --git a/test/cmd_ut.c b/test/cmd_ut.c
index 7304c32bec5..9d36bd5dc87 100644
--- a/test/cmd_ut.c
+++ b/test/cmd_ut.c
@@ -30,7 +30,7 @@ struct suite {
static int do_ut_all(struct unit_test_state *uts, const char *select_name,
int runs_per_test, bool force_run,
- const char *test_insert);
+ const char *test_insert, int argc, char *const argv[]);
static int do_ut_info(bool show_suites);
@@ -120,7 +120,7 @@ static bool has_tests(struct suite *ste)
/** run_suite() - Run a suite of tests */
static int run_suite(struct unit_test_state *uts, struct suite *ste,
const char *select_name, int runs_per_test, bool force_run,
- const char *test_insert)
+ const char *test_insert, int argc, char *const argv[])
{
int n_ents = ste->end - ste->start;
char prefix[30];
@@ -131,7 +131,7 @@ static int run_suite(struct unit_test_state *uts, struct suite *ste,
ret = ut_run_list(uts, ste->name, prefix, ste->start, n_ents,
select_name, runs_per_test, force_run, test_insert,
- 0, NULL);
+ argc, argv);
return ret;
}
@@ -167,7 +167,8 @@ static void update_stats(struct unit_test_state *uts, const struct suite *ste)
}
static int do_ut_all(struct unit_test_state *uts, const char *select_name,
- int runs_per_test, bool force_run, const char *test_insert)
+ int runs_per_test, bool force_run, const char *test_insert,
+ int argc, char *const argv[])
{
int i;
int retval;
@@ -179,7 +180,7 @@ static int do_ut_all(struct unit_test_state *uts, const char *select_name,
if (has_tests(ste)) {
printf("----Running %s tests----\n", ste->name);
retval = run_suite(uts, ste, select_name, runs_per_test,
- force_run, test_insert);
+ force_run, test_insert, argc, argv);
if (!any_fail)
any_fail = retval;
update_stats(uts, ste);
@@ -244,6 +245,8 @@ static struct suite *find_suite(const char *name)
static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
{
const char *test_insert = NULL, *select_name;
+ int test_argc;
+ char *const *test_argv;
struct unit_test_state uts;
bool show_suites = false;
bool force_run = false;
@@ -285,9 +288,14 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
ut_init_state(&uts);
name = argv[0];
select_name = cmd_arg1(argc, argv);
+
+ /* Test arguments are after suite name and test name */
+ test_argc = argc > 2 ? argc - 2 : 0;
+ test_argv = argc > 2 ? argv + 2 : NULL;
+
if (!strcmp(name, "all")) {
ret = do_ut_all(&uts, select_name, runs_per_text, force_run,
- test_insert);
+ test_insert, test_argc, test_argv);
} else if (!strcmp(name, "info")) {
ret = do_ut_info(show_suites);
} else {
@@ -306,7 +314,8 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
}
ret = run_suite(&uts, ste, select_name, runs_per_text,
- force_run, test_insert);
+ force_run, test_insert, test_argc,
+ test_argv);
if (!any_fail)
any_fail = ret;
update_stats(&uts, ste);
@@ -322,12 +331,14 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[])
}
U_BOOT_LONGHELP(ut,
- "[-rs] [-f] [-I<n>:<one_test>][<suites>] - run unit tests\n"
+ "[-rs] [-f] [-I<n>:<one_test>] <suite> [<test> [<args>...]] - run unit tests\n"
" -r<runs> Number of times to run each test\n"
" -f Force 'manual' tests to run as well\n"
" -I Test to run after <n> other tests have run\n"
" -s Show all suites with ut info\n"
- " <suites> Comma-separated list of suites to run\n"
+ " <suite> Test suite to run (or comma-separated list)\n"
+ " <test> Specific test to run (optional)\n"
+ " <args> Test arguments as key=value pairs (optional)\n"
"\n"
"Options for <suite>:\n"
"all - execute all enabled tests\n"
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 08/12] test: Add type-checked argument accessor functions
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (6 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 07/12] test: Enhance the ut command to pass test arguments Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 09/12] test: Add tests for unit-test arguments Simon Glass
` (5 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass, Andrew Goodbody
Add ut_get_str(), ut_get_int(), and ut_get_bool() functions with
corresponding ut_str(), ut_int(), and ut_bool() macros for accessing
test arguments with type checking.
These functions check that the argument index is within bounds and the
type matches what was requested.
The first failure for a test is reported via ut_failf() which should
make it fairly easy to debug the test.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
include/test/test.h | 2 ++
include/test/ut.h | 50 ++++++++++++++++++++++++++++++++++
test/test-main.c | 1 +
test/ut.c | 66 +++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 119 insertions(+)
diff --git a/include/test/test.h b/include/test/test.h
index bd47025ce50..f435d6f8c6a 100644
--- a/include/test/test.h
+++ b/include/test/test.h
@@ -95,6 +95,7 @@ struct ut_arg {
* @priv: Private data for tests to use as needed
* @args: Parsed argument values for current test
* @arg_count: Number of parsed arguments
+ * @arg_error: Set if ut_str/int/bool() detects a type mismatch
*/
struct unit_test_state {
struct ut_stats cur;
@@ -123,6 +124,7 @@ struct unit_test_state {
char priv[UT_PRIV_SIZE];
struct ut_arg args[UT_MAX_ARGS];
int arg_count;
+ bool arg_error;
};
/* Test flags for each test */
diff --git a/include/test/ut.h b/include/test/ut.h
index 1c4ee390fd8..6115711574c 100644
--- a/include/test/ut.h
+++ b/include/test/ut.h
@@ -692,4 +692,54 @@ int ut_run_list(struct unit_test_state *uts, const char *category,
*/
void ut_report(struct ut_stats *stats, int run_count);
+/**
+ * ut_get_str() - Get a string test argument
+ *
+ * Fails the test if the argument type is not UT_ARG_STR.
+ *
+ * @uts: Test state
+ * @n: Argument index
+ * @file: Filename of caller
+ * @line: Line number of caller
+ * @func: Function name of caller
+ * Return: String value, or NULL if type mismatch
+ */
+const char *ut_get_str(struct unit_test_state *uts, int n, const char *file,
+ int line, const char *func);
+
+/**
+ * ut_get_int() - Get an integer test argument
+ *
+ * Fails the test if the argument type is not UT_ARG_INT.
+ *
+ * @uts: Test state
+ * @n: Argument index
+ * @file: Filename of caller
+ * @line: Line number of caller
+ * @func: Function name of caller
+ * Return: Integer value, or 0 if type mismatch
+ */
+long ut_get_int(struct unit_test_state *uts, int n, const char *file,
+ int line, const char *func);
+
+/**
+ * ut_get_bool() - Get a boolean test argument
+ *
+ * Fails the test if the argument type is not UT_ARG_BOOL.
+ *
+ * @uts: Test state
+ * @n: Argument index
+ * @file: Filename of caller
+ * @line: Line number of caller
+ * @func: Function name of caller
+ * Return: Boolean value, or false if type mismatch
+ */
+bool ut_get_bool(struct unit_test_state *uts, int n, const char *file,
+ int line, const char *func);
+
+/* Helpers for accessing test arguments with type checking */
+#define ut_str(n) ut_get_str(uts, n, __FILE__, __LINE__, __func__)
+#define ut_int(n) ut_get_int(uts, n, __FILE__, __LINE__, __func__)
+#define ut_bool(n) ut_get_bool(uts, n, __FILE__, __LINE__, __func__)
+
#endif
diff --git a/test/test-main.c b/test/test-main.c
index 9d8ab794db8..9fc4bca8e20 100644
--- a/test/test-main.c
+++ b/test/test-main.c
@@ -636,6 +636,7 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test,
if (ret)
return ret;
+ uts->arg_error = false;
ret = test->func(uts);
if (ret == -EAGAIN)
skip_test(uts);
diff --git a/test/ut.c b/test/ut.c
index b0cc0f3e8ff..a13e4825eac 100644
--- a/test/ut.c
+++ b/test/ut.c
@@ -284,3 +284,69 @@ void ut_set_skip_delays(struct unit_test_state *uts, bool skip_delays)
state_set_skip_delays(skip_delays);
#endif
}
+
+const char *ut_get_str(struct unit_test_state *uts, int n, const char *file,
+ int line, const char *func)
+{
+ if (n < 0 || n >= uts->arg_count) {
+ if (!uts->arg_error)
+ ut_failf(uts, file, line, func, "ut_str() arg check",
+ "arg %d is invalid (arg_count=%d)", n,
+ uts->arg_count);
+ uts->arg_error = true;
+ return NULL;
+ }
+ if (uts->args[n].type != UT_ARG_STR) {
+ if (!uts->arg_error)
+ ut_failf(uts, file, line, func, "ut_str() type check",
+ "arg %d is not a string", n);
+ uts->arg_error = true;
+ return NULL;
+ }
+
+ return uts->args[n].vstr;
+}
+
+long ut_get_int(struct unit_test_state *uts, int n, const char *file,
+ int line, const char *func)
+{
+ if (n < 0 || n >= uts->arg_count) {
+ if (!uts->arg_error)
+ ut_failf(uts, file, line, func, "ut_int() arg check",
+ "arg %d is invalid (arg_count=%d)", n,
+ uts->arg_count);
+ uts->arg_error = true;
+ return 0;
+ }
+ if (uts->args[n].type != UT_ARG_INT) {
+ if (!uts->arg_error)
+ ut_failf(uts, file, line, func, "ut_int() type check",
+ "arg %d is not an int", n);
+ uts->arg_error = true;
+ return 0;
+ }
+
+ return uts->args[n].vint;
+}
+
+bool ut_get_bool(struct unit_test_state *uts, int n, const char *file,
+ int line, const char *func)
+{
+ if (n < 0 || n >= uts->arg_count) {
+ if (!uts->arg_error)
+ ut_failf(uts, file, line, func, "ut_bool() arg check",
+ "arg %d is invalid (arg_count=%d)", n,
+ uts->arg_count);
+ uts->arg_error = true;
+ return false;
+ }
+ if (uts->args[n].type != UT_ARG_BOOL) {
+ if (!uts->arg_error)
+ ut_failf(uts, file, line, func, "ut_bool() type check",
+ "arg %d is not a bool", n);
+ uts->arg_error = true;
+ return false;
+ }
+
+ return uts->args[n].vbool;
+}
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 09/12] test: Add tests for unit-test arguments
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (7 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 08/12] test: Add type-checked argument accessor functions Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 10/12] test: Add documentation for the test framework Simon Glass
` (4 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass
Add a set of tests to check the behaviour of test arguments and the ut
command. This includes failure cases, where the wrong type or a
non-existent argument is requested.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
test/common/Makefile | 1 +
test/common/test_args.c | 176 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 177 insertions(+)
create mode 100644 test/common/test_args.c
diff --git a/test/common/Makefile b/test/common/Makefile
index baefc7b3622..bfeb29346f5 100644
--- a/test/common/Makefile
+++ b/test/common/Makefile
@@ -11,3 +11,4 @@ obj-$(CONFIG_CYCLIC) += cyclic.o
obj-$(CONFIG_EVENT_DYNAMIC) += event.o
obj-y += cread.o
obj-$(CONFIG_$(PHASE_)CMDLINE) += print.o
+obj-$(CONFIG_$(PHASE_)CMDLINE) += test_args.o
diff --git a/test/common/test_args.c b/test/common/test_args.c
new file mode 100644
index 00000000000..d81039e75ac
--- /dev/null
+++ b/test/common/test_args.c
@@ -0,0 +1,176 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Test for unit test arguments
+ *
+ * Copyright 2026 Canonical Ltd
+ * Written by Simon Glass <simon.glass@canonical.com>
+ */
+
+#include <string.h>
+#include <test/common.h>
+#include <test/test.h>
+#include <test/ut.h>
+
+/* Test that string arguments work correctly */
+static int test_args_str_norun(struct unit_test_state *uts)
+{
+ ut_asserteq_str("hello", ut_str(0));
+
+ return 0;
+}
+UNIT_TEST_ARGS(test_args_str_norun, UTF_CONSOLE | UTF_MANUAL, common,
+ { "strval", UT_ARG_STR });
+
+/* Test that integer arguments work correctly */
+static int test_args_int_norun(struct unit_test_state *uts)
+{
+ ut_asserteq(1234, ut_int(0));
+
+ return 0;
+}
+UNIT_TEST_ARGS(test_args_int_norun, UTF_CONSOLE | UTF_MANUAL, common,
+ { "intval", UT_ARG_INT });
+
+/* Test that boolean arguments work correctly */
+static int test_args_bool_norun(struct unit_test_state *uts)
+{
+ ut_asserteq(true, ut_bool(0));
+
+ return 0;
+}
+UNIT_TEST_ARGS(test_args_bool_norun, UTF_CONSOLE | UTF_MANUAL, common,
+ { "boolval", UT_ARG_BOOL });
+
+/* Test multiple arguments of different types */
+static int test_args_multi_norun(struct unit_test_state *uts)
+{
+ ut_asserteq_str("test", ut_str(0));
+ ut_asserteq(42, ut_int(1));
+ ut_asserteq(true, ut_bool(2));
+
+ return 0;
+}
+UNIT_TEST_ARGS(test_args_multi_norun, UTF_CONSOLE | UTF_MANUAL, common,
+ { "str", UT_ARG_STR },
+ { "num", UT_ARG_INT },
+ { "flag", UT_ARG_BOOL });
+
+/* Test optional arguments with defaults */
+static int test_args_optional_norun(struct unit_test_state *uts)
+{
+ /* Required arg should match what was passed */
+ ut_asserteq_str("required", ut_str(0));
+
+ /* Optional args should have default values if not provided */
+ ut_asserteq(99, ut_int(1));
+ ut_asserteq(false, ut_bool(2));
+
+ return 0;
+}
+UNIT_TEST_ARGS(test_args_optional_norun, UTF_CONSOLE | UTF_MANUAL, common,
+ { "req", UT_ARG_STR },
+ { "opt_int", UT_ARG_INT, UT_ARGF_OPTIONAL, { .vint = 99 } },
+ { "opt_bool", UT_ARG_BOOL, UT_ARGF_OPTIONAL, { .vbool = false } });
+
+/* Test requesting wrong type - ut_int() on a string arg should fail */
+static int test_args_wrongtype_norun(struct unit_test_state *uts)
+{
+ /* This should fail - asking for int but arg is string */
+ ut_asserteq(0, ut_int(0));
+ ut_asserteq(true, uts->arg_error);
+
+ return 0;
+}
+UNIT_TEST_ARGS(test_args_wrongtype_norun, UTF_MANUAL, common,
+ { "strval", UT_ARG_STR });
+
+/* Test requesting invalid arg number - ut_str(1) when only arg 0 exists */
+static int test_args_badnum_norun(struct unit_test_state *uts)
+{
+ /* This should fail - asking for arg 1 but only arg 0 exists */
+ ut_asserteq_ptr(NULL, ut_str(1));
+ ut_asserteq(true, uts->arg_error);
+
+ return 0;
+}
+UNIT_TEST_ARGS(test_args_badnum_norun, UTF_MANUAL, common,
+ { "strval", UT_ARG_STR });
+
+/* Wrapper test that runs the manual tests with proper arguments */
+static int test_args(struct unit_test_state *uts)
+{
+ ut_assertok(run_command("ut -f common test_args_str_norun strval=hello",
+ 0));
+ ut_assertok(run_command("ut -f common test_args_int_norun intval=1234",
+ 0));
+ ut_assertok(run_command("ut -f common test_args_bool_norun boolval=1",
+ 0));
+ ut_assertok(run_command("ut -f common test_args_multi_norun str=test num=42 flag=1",
+ 0));
+ ut_assertok(run_command("ut -f common test_args_optional_norun req=required",
+ 0));
+
+ return 0;
+}
+COMMON_TEST(test_args, UTF_CONSOLE);
+
+/* Test argument-parsing failure cases - these should all fail */
+static int test_args_fail(struct unit_test_state *uts)
+{
+ /* Missing required argument - should fail */
+ ut_asserteq(1, run_command("ut -f common test_args_str_norun", 0));
+ ut_assert_nextline("Missing required argument 'strval' for test 'test_args_str_norun'");
+ ut_assert_nextline_regex("Tests run: 1,.*failures: 1");
+ ut_assert_console_end();
+
+ /* Unknown argument name - should fail */
+ ut_asserteq(1, run_command("ut -f common test_args_str_norun badarg=x",
+ 0));
+ ut_assert_nextline("Unknown argument 'badarg' for test 'test_args_str_norun'");
+ ut_assert_nextline_regex("Tests run: 1,.*failures: 1");
+ ut_assert_console_end();
+
+ /* Invalid format (no = sign) - should fail */
+ ut_asserteq(1, run_command("ut -f common test_args_str_norun strval",
+ 0));
+ ut_assert_nextline("Invalid argument 'strval' (expected key=value)");
+ ut_assert_nextline_regex("Tests run: 1,.*failures: 1");
+ ut_assert_console_end();
+
+ return 0;
+}
+COMMON_TEST(test_args_fail, UTF_CONSOLE);
+
+/* Test that requesting wrong type fails - ut_int() on string arg */
+static int test_args_wrongtype(struct unit_test_state *uts)
+{
+ ut_asserteq(1,
+ run_command("ut -R -f common test_args_wrongtype_norun strval=hello",
+ 0));
+ ut_assert_nextline("Test: test_args_wrongtype_norun: test_args.c");
+ ut_assert_nextline_regex(
+ "test/common/test_args.c:.*, test_args_wrongtype_norun\\(\\): ut_int\\(\\) type check: arg 0 is not an int");
+ ut_assert_nextline("Test 'test_args_wrongtype_norun' failed 1 times");
+ ut_assert_nextline_regex("Tests run: 1,.*failures: 1");
+ ut_assert_console_end();
+
+ return 0;
+}
+COMMON_TEST(test_args_wrongtype, UTF_CONSOLE);
+
+/* Test that requesting invalid arg number fails */
+static int test_args_badnum(struct unit_test_state *uts)
+{
+ ut_asserteq(1,
+ run_command("ut -R -f common test_args_badnum_norun strval=hello",
+ 0));
+ ut_assert_nextline("Test: test_args_badnum_norun: test_args.c");
+ ut_assert_nextline_regex(
+ "test/common/test_args.c:.*, test_args_badnum_norun\\(\\): ut_str\\(\\) arg check: arg 1 is invalid \\(arg_count=1\\)");
+ ut_assert_nextline("Test 'test_args_badnum_norun' failed 1 times");
+ ut_assert_nextline_regex("Tests run: 1,.*failures: 1");
+ ut_assert_console_end();
+
+ return 0;
+}
+COMMON_TEST(test_args_badnum, UTF_CONSOLE);
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 10/12] test: Add documentation for the test framework
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (8 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 09/12] test: Add tests for unit-test arguments Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 11/12] test: fs: add C-based filesystem tests Simon Glass
` (3 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass, Ilias Apalodimas
Add documentation for test assertions, the private buffer, and test
parameters. This covers the available ut_assert*() macros, console
output checks, memory helpers, and how to pass typed arguments from
Python to C tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
doc/develop/tests_writing.rst | 169 ++++++++++++++++++++++++++++++++++
doc/usage/cmd/ut.rst | 11 ++-
2 files changed, 178 insertions(+), 2 deletions(-)
diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst
index 1a020caa411..40192f9db30 100644
--- a/doc/develop/tests_writing.rst
+++ b/doc/develop/tests_writing.rst
@@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the
Python test.
+Passing arguments to C tests
+----------------------------
+
+Sometimes a C test needs parameters from Python, such as filenames or expected
+values that are generated at runtime. The test-argument feature allows this.
+
+Use the `UNIT_TEST_ARGS` macro to declare a test with arguments::
+
+ static int my_test_norun(struct unit_test_state *uts)
+ {
+ const char *filename = ut_str(0);
+ int count = ut_int(1);
+
+ /* test code using filename and count */
+
+ return 0;
+ }
+ UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
+ { "filename", UT_ARG_STR },
+ { "count", UT_ARG_INT });
+
+Each argument definition specifies a name and type:
+
+- `UT_ARG_STR` - string argument, accessed via `ut_str(n)`
+- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)`
+- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)`
+ (use `1` for true, any other value for false)
+
+Arguments are passed on the command line in `name=value` format::
+
+ ut -f my_suite my_test_norun filename=/path/to/file count=42
+
+From Python, you can call the test like this::
+
+ cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}'
+ ubman.run_command(cmd)
+
+This approach combines Python's flexibility for setup (creating files,
+generating values) with C's speed and debuggability for the actual test logic.
+
+
How slow are Python tests?
--------------------------
@@ -384,6 +425,134 @@ existing suite or creating a new one.
An example SPL test is spl_test_load().
+.. _tests_writing_assertions:
+
+Assertions
+----------
+
+The test framework provides various assertion macros, defined in
+``include/test/ut.h``. All of these return from the test function on failure,
+unless ``uts->soft_fail`` is set.
+
+Basic assertions
+~~~~~~~~~~~~~~~~
+
+ut_assert(cond)
+ Assert that a condition is non-zero (true)
+
+ut_assertf(cond, fmt, args...)
+ Assert that a condition is non-zero, with a printf() message on failure
+
+ut_assertok(cond)
+ Assert that an operation succeeds (returns 0)
+
+ut_reportf(fmt, args...)
+ Report a failure with a printf() message (always fails)
+
+Value comparisons
+~~~~~~~~~~~~~~~~~
+
+ut_asserteq(expr1, expr2)
+ Assert that two int expressions are equal
+
+ut_asserteq_64(expr1, expr2)
+ Assert that two 64-bit expressions are equal
+
+ut_asserteq_str(expr1, expr2)
+ Assert that two strings are equal
+
+ut_asserteq_strn(expr1, expr2)
+ Assert that two strings are equal, up to the length of the first
+
+ut_asserteq_mem(expr1, expr2, len)
+ Assert that two memory areas are equal
+
+ut_asserteq_ptr(expr1, expr2)
+ Assert that two pointers are equal
+
+ut_asserteq_addr(expr1, expr2)
+ Assert that two addresses (converted from pointers via map_to_sysmem())
+ are equal
+
+ut_asserteq_regex(pattern, str)
+ Assert that a string matches a regular expression pattern. Uses the SLRE
+ library for regex matching. Useful when exact matching is fragile, e.g.
+ when output contains line numbers or variable content.
+
+Pointer assertions
+~~~~~~~~~~~~~~~~~~
+
+ut_assertnull(expr)
+ Assert that a pointer is NULL
+
+ut_assertnonnull(expr)
+ Assert that a pointer is not NULL
+
+ut_assertok_ptr(expr)
+ Assert that a pointer is not an error pointer (checked with IS_ERR())
+
+Console output assertions
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+These are used to check console output when ``UTF_CONSOLE`` flag is set.
+
+ut_assert_nextline(fmt, args...)
+ Assert that the next console output line matches the format string
+
+ut_assert_nextlinen(fmt, args...)
+ Assert that the next console output line matches up to the format
+ string length
+
+ut_assert_nextline_regex(pattern)
+ Assert that the next console output line matches a regex pattern
+
+ut_assert_nextline_empty()
+ Assert that the next console output line is empty
+
+ut_assert_skipline()
+ Assert that there is a next console output line, and skip it
+
+ut_assert_skip_to_line(fmt, args...)
+ Skip console output until a matching line is found
+
+ut_assert_skip_to_linen(fmt, args...)
+ Skip console output until a partial match is found (compares up to the
+ format-string length)
+
+ut_assert_console_end()
+ Assert that there is no more console output
+
+ut_assert_nextlines_are_dump(total_bytes)
+ Assert that the next lines are a print_buffer() hex dump of the specified
+ size
+
+Memory helpers
+~~~~~~~~~~~~~~
+
+These help check for memory leaks:
+
+ut_check_free()
+ Return the number of bytes free in the malloc() pool
+
+ut_check_delta(last)
+ Return the change in free memory since ``last`` was obtained from
+ ``ut_check_free()``. A positive value means more memory has been allocated.
+
+Private buffer
+~~~~~~~~~~~~~~
+
+Each test has access to a private buffer ``uts->priv`` (256 bytes) for temporary
+data. This avoids the need to allocate memory or use global variables::
+
+ static int my_test(struct unit_test_state *uts)
+ {
+ snprintf(uts->priv, sizeof(uts->priv), "/%s", filename);
+ /* use uts->priv as a path string */
+
+ return 0;
+ }
+
+
Writing Python tests
--------------------
diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst
index d8c3cbf496c..a26ee6ad7de 100644
--- a/doc/usage/cmd/ut.rst
+++ b/doc/usage/cmd/ut.rst
@@ -11,7 +11,7 @@ Synopsis
::
- ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
+ ut [-r<runs>] [-f] [-R] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
ut [-s] info
Description
@@ -37,10 +37,17 @@ test
causes another test to fail. If the one test fails, testing stops
immediately.
+-R
+ Preserve console recording on test failure. Normally when a test fails,
+ console recording is disabled so error messages go directly to output.
+ This flag keeps recording enabled, which is useful when testing the test
+ framework itself.
+
args
Optional arguments to pass to the test, in `name=value` format. These are
used by tests declared with `UNIT_TEST_ARGS()` which define expected
- argument names and types.
+ argument names and types. See :ref:`develop/tests_writing:passing arguments
+ to c tests` for details.
Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it
includes a near-complete set of emulators, no code-size limits, many CONFIG
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 11/12] test: fs: add C-based filesystem tests
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (9 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 10/12] test: Add documentation for the test framework Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-12 11:19 ` [PATCH v2 12/12] test: fs: Update Python tests to call C implementations Simon Glass
` (2 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass, Ilias Apalodimas
Add C implementations of filesystem tests that can be called via
the 'ut fs' command. These tests use UTF_MANUAL flag since they require
external setup, i.e. creation of filesystem images.
This covers the existing TestFsBasic tests.
The tests use typed arguments (fs_type, fs_image, md5 values) passed
via the command line.
Add a few helpers to make the code easier to read.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
include/test/fs.h | 39 +++++
test/Makefile | 1 +
test/cmd_ut.c | 2 +
test/fs/Makefile | 3 +
test/fs/fs_basic.c | 407 +++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 452 insertions(+)
create mode 100644 include/test/fs.h
create mode 100644 test/fs/Makefile
create mode 100644 test/fs/fs_basic.c
diff --git a/include/test/fs.h b/include/test/fs.h
new file mode 100644
index 00000000000..7fdb8d70451
--- /dev/null
+++ b/include/test/fs.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2026 Canonical Ltd
+ */
+
+#ifndef __TEST_FS_H
+#define __TEST_FS_H
+
+#include <test/test.h>
+#include <test/ut.h>
+
+/**
+ * FS_TEST() - Define a new filesystem test
+ *
+ * @name: Name of test function
+ * @flags: Flags for the test (see enum ut_flags)
+ */
+#define FS_TEST(_name, _flags) UNIT_TEST(_name, UTF_DM | (_flags), fs)
+
+/**
+ * FS_TEST_ARGS() - Define a filesystem test with inline arguments
+ *
+ * Like FS_TEST() but for tests that take arguments.
+ * The test can access arguments via uts->args[].
+ * The NULL terminator is added automatically.
+ *
+ * Example:
+ * FS_TEST_ARGS(my_test, UTF_MANUAL,
+ * { "fs_type", UT_ARG_STR },
+ * { "fs_image", UT_ARG_STR });
+ *
+ * @name: Name of test function
+ * @flags: Flags for the test (see enum ut_flags)
+ * @...: Argument definitions (struct ut_arg_def initializers)
+ */
+#define FS_TEST_ARGS(_name, _flags, ...) \
+ UNIT_TEST_ARGS(_name, UTF_DM | (_flags), fs, __VA_ARGS__)
+
+#endif /* __TEST_FS_H */
diff --git a/test/Makefile b/test/Makefile
index 5676bd35963..02ded045995 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -22,6 +22,7 @@ obj-y += boot/
obj-y += common/
obj-$(CONFIG_UT_ENV) += env/
obj-$(CONFIG_UT_FDT_OVERLAY) += fdt_overlay/
+obj-$(CONFIG_SANDBOX) += fs/
obj-y += log/
else
obj-$(CONFIG_SPL_UT_LOAD) += image/
diff --git a/test/cmd_ut.c b/test/cmd_ut.c
index 9d36bd5dc87..827fcbc9fcb 100644
--- a/test/cmd_ut.c
+++ b/test/cmd_ut.c
@@ -60,6 +60,7 @@ SUITE_DECL(exit);
SUITE_DECL(fdt);
SUITE_DECL(fdt_overlay);
SUITE_DECL(font);
+SUITE_DECL(fs);
SUITE_DECL(hush);
SUITE_DECL(lib);
SUITE_DECL(loadm);
@@ -87,6 +88,7 @@ static struct suite suites[] = {
SUITE(fdt, "fdt command"),
SUITE(fdt_overlay, "device tree overlays"),
SUITE(font, "font command"),
+ SUITE(fs, "filesystem tests"),
SUITE(hush, "hush behaviour"),
SUITE(lib, "library functions"),
SUITE(loadm, "loadm command parameters and loading memory blob"),
diff --git a/test/fs/Makefile b/test/fs/Makefile
new file mode 100644
index 00000000000..5899be8e667
--- /dev/null
+++ b/test/fs/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0+
+
+obj-y += fs_basic.o
diff --git a/test/fs/fs_basic.c b/test/fs/fs_basic.c
new file mode 100644
index 00000000000..a6f1e056f5b
--- /dev/null
+++ b/test/fs/fs_basic.c
@@ -0,0 +1,407 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Basic filesystem tests - C implementation for Python wrapper
+ *
+ * These tests are marked UTF_MANUAL and are intended to be called from
+ * test_basic.py which sets up the filesystem image and expected values.
+ *
+ * Copyright 2026 Canonical Ltd
+ */
+
+#include <command.h>
+#include <dm.h>
+#include <env.h>
+#include <fs.h>
+#include <hexdump.h>
+#include <image.h>
+#include <linux/sizes.h>
+#include <mapmem.h>
+#include <test/fs.h>
+#include <test/test.h>
+#include <test/ut.h>
+#include <u-boot/md5.h>
+
+/* Test constants matching fstest_defs.py */
+#define ADDR 0x01000008
+
+/*
+ * Common argument indices. Each test declares only the arguments it needs,
+ * so indices 2+ vary per test - see comments in each test.
+ */
+#define FS_ARG_TYPE 0 /* fs_type: ext4, fat, exfat, fs_generic */
+#define FS_ARG_IMAGE 1 /* fs_image: path to filesystem image */
+
+/* Common arguments for all filesystem tests (indices 0 and 1) */
+#define COMMON_ARGS \
+ { "fs_type", UT_ARG_STR }, \
+ { "fs_image", UT_ARG_STR }
+
+/**
+ * get_fs_type(uts) - Get filesystem type enum from test argument
+ *
+ * Reads the fs_type argument and returns the appropriate FS_TYPE_* enum value.
+ *
+ * Return: filesystem type enum
+ */
+static int get_fs_type(struct unit_test_state *uts)
+{
+ const char *fs_type = ut_str(FS_ARG_TYPE);
+
+ if (!fs_type)
+ return FS_TYPE_ANY;
+
+ if (!strcmp(fs_type, "ext4"))
+ return FS_TYPE_EXT;
+ if (!strcmp(fs_type, "fat"))
+ return FS_TYPE_FAT;
+ if (!strcmp(fs_type, "exfat"))
+ return FS_TYPE_EXFAT;
+
+ /* fs_generic uses FS_TYPE_ANY */
+ return FS_TYPE_ANY;
+}
+
+/* Set up the host filesystem block device */
+static int set_fs(struct unit_test_state *uts)
+{
+ return fs_set_blk_dev("host", "0:0", get_fs_type(uts));
+}
+
+/* Build a path by prepending "/" to the leaf filename, with optional suffix */
+static const char *getpath(struct unit_test_state *uts, const char *leaf,
+ const char *suffix)
+{
+ snprintf(uts->priv, sizeof(uts->priv), "/%s%s", leaf, suffix ?: "");
+
+ return uts->priv;
+}
+
+/**
+ * prep_fs() - Prepare filesystem for testing
+ *
+ * Binds the fs_image argument as host device 0, sets up the block device,
+ * and optionally returns a zeroed buffer.
+ *
+ * @uts: Unit test state
+ * @len: Length of buffer to allocate and zero, or 0 for none
+ * @bufp: Returns pointer to zeroed buffer, or NULL if @len is 0
+ * Return: 0 on success, negative on error
+ */
+static int prep_fs(struct unit_test_state *uts, uint len, void **bufp)
+{
+ const char *fs_image = ut_str(FS_ARG_IMAGE);
+
+ ut_assertnonnull(fs_image);
+ ut_assertok(run_commandf("host bind 0 %s", fs_image));
+ ut_assertok(set_fs(uts));
+
+ if (len) {
+ *bufp = map_sysmem(ADDR, len);
+ memset(*bufp, '\0', len);
+ }
+
+ return 0;
+}
+
+/**
+ * fs_write_supported(uts) - Check if write is supported for current fs type
+ *
+ * Reads the fs_type argument and checks if write support is enabled
+ * for that filesystem type.
+ *
+ * Return: true if write is supported, false otherwise
+ */
+static bool fs_write_supported(struct unit_test_state *uts)
+{
+ const char *fs_type = ut_str(FS_ARG_TYPE);
+
+ if (!fs_type)
+ return false;
+
+ if (!strcmp(fs_type, "ext4"))
+ return IS_ENABLED(CONFIG_EXT4_WRITE);
+ if (!strcmp(fs_type, "fat"))
+ return IS_ENABLED(CONFIG_CMD_FAT_WRITE);
+
+ /* fs_generic and exfat use generic write which is always available */
+ return true;
+}
+
+/**
+ * verify_md5() - Calculate MD5 of buffer and verify against expected
+ *
+ * Uses arg 3 (md5val) as the expected MD5 hex string.
+ *
+ * @uts: Unit test state
+ * @buf: Buffer to calculate MD5 of
+ * @len: Length of buffer
+ *
+ * Return: 0 if MD5 matches, -EINVAL otherwise
+ */
+static int verify_md5(struct unit_test_state *uts, const void *buf, size_t len)
+{
+ u8 digest[MD5_SUM_LEN], expected[MD5_SUM_LEN];
+ const char *expected_hex = ut_str(3);
+
+ ut_assertok(hex2bin(expected, expected_hex, MD5_SUM_LEN));
+
+ md5_wd(buf, len, digest, CHUNKSZ_MD5);
+ ut_asserteq_mem(expected, digest, MD5_SUM_LEN);
+
+ return 0;
+}
+
+/* Test Case 1 - ls command, listing root directory and invalid directory */
+static int fs_test_ls_norun(struct unit_test_state *uts)
+{
+ const char *small = ut_str(2);
+ const char *big = ut_str(3);
+ struct fs_dir_stream *dirs;
+ struct fs_dirent *dent;
+ bool found_big = false, found_small = false, found_subdir = false;
+
+ ut_assertok(prep_fs(uts, 0, NULL));
+
+ /* Test listing root directory */
+ dirs = fs_opendir("/");
+ ut_assertnonnull(dirs);
+
+ while ((dent = fs_readdir(dirs))) {
+ if (!strcmp(dent->name, big)) {
+ found_big = true;
+ ut_asserteq(FS_DT_REG, dent->type);
+ } else if (!strcmp(dent->name, small)) {
+ found_small = true;
+ ut_asserteq(FS_DT_REG, dent->type);
+ } else if (!strcmp(dent->name, "SUBDIR")) {
+ found_subdir = true;
+ ut_asserteq(FS_DT_DIR, dent->type);
+ }
+ }
+ fs_closedir(dirs);
+
+ ut_assert(found_big);
+ ut_assert(found_small);
+ ut_assert(found_subdir);
+
+ /* Test invalid directory returns error */
+ ut_assertok(set_fs(uts));
+ dirs = fs_opendir("/invalid_d");
+ ut_assertnull(dirs);
+
+ /* Test file exists */
+ ut_assertok(set_fs(uts));
+ ut_assert(fs_exists(small));
+
+ /* Test non-existent file */
+ ut_assertok(set_fs(uts));
+ ut_assert(!fs_exists("nonexistent.file"));
+
+ return 0;
+}
+FS_TEST_ARGS(fs_test_ls_norun, UTF_SCAN_FDT | UTF_CONSOLE | UTF_MANUAL,
+ COMMON_ARGS, { "small", UT_ARG_STR }, { "big", UT_ARG_STR });
+
+/* Test Case 2 - size command for small file (1MB) */
+static int fs_test_size_small_norun(struct unit_test_state *uts)
+{
+ const char *small = ut_str(2);
+ loff_t size;
+
+ ut_assertok(prep_fs(uts, 0, NULL));
+ ut_assertok(fs_size(getpath(uts, small, NULL), &size));
+ ut_asserteq(SZ_1M, size);
+
+ /* Test size via path with '..' */
+ ut_assertok(set_fs(uts));
+ snprintf(uts->priv, sizeof(uts->priv), "/SUBDIR/../%s", small);
+ ut_assertok(fs_size(uts->priv, &size));
+ ut_asserteq(SZ_1M, size);
+
+ return 0;
+}
+FS_TEST_ARGS(fs_test_size_small_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "small", UT_ARG_STR });
+
+/* Test Case 3 - size command for large file (2500 MiB) */
+static int fs_test_size_big_norun(struct unit_test_state *uts)
+{
+ const char *big = ut_str(2);
+ loff_t size;
+
+ ut_assertok(prep_fs(uts, 0, NULL));
+ ut_assertok(fs_size(getpath(uts, big, NULL), &size));
+ ut_asserteq_64((loff_t)SZ_1M * 2500, size); /* 2500 MiB = 0x9c400000 */
+
+ return 0;
+}
+FS_TEST_ARGS(fs_test_size_big_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "big", UT_ARG_STR });
+
+/* Load a file at a given offset and verify the size and MD5 */
+static int check_load(struct unit_test_state *uts, const char *fname,
+ ulong offset, ulong read_len, ulong expect_len,
+ bool check_md5)
+{
+ loff_t actual;
+ void *buf;
+
+ ut_assertok(prep_fs(uts, expect_len, &buf));
+ ut_assertok(fs_read(getpath(uts, fname, NULL), ADDR, offset, read_len,
+ &actual));
+ ut_asserteq(expect_len, actual);
+ if (check_md5)
+ ut_assertok(verify_md5(uts, buf, expect_len));
+
+ return 0;
+}
+
+/* Test Case 4 - load small file, verify MD5 */
+static int fs_test_load_small_norun(struct unit_test_state *uts)
+{
+ return check_load(uts, ut_str(2), 0, 0, SZ_1M, true);
+}
+FS_TEST_ARGS(fs_test_load_small_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "small", UT_ARG_STR }, { "md5val", UT_ARG_STR });
+
+/* Test Case 5 - load first 1MB of big file */
+static int fs_test_load_big_first_norun(struct unit_test_state *uts)
+{
+ return check_load(uts, ut_str(2), 0, SZ_1M, SZ_1M, true);
+}
+FS_TEST_ARGS(fs_test_load_big_first_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR });
+
+/* Test Case 6 - load last 1MB of big file (offset 0x9c300000) */
+static int fs_test_load_big_last_norun(struct unit_test_state *uts)
+{
+ return check_load(uts, ut_str(2), 0x9c300000, SZ_1M, SZ_1M, true);
+}
+FS_TEST_ARGS(fs_test_load_big_last_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR });
+
+/* Test Case 7 - load 1MB from last 1MB chunk of 2GB (offset 0x7ff00000) */
+static int fs_test_load_big_2g_last_norun(struct unit_test_state *uts)
+{
+ return check_load(uts, ut_str(2), 0x7ff00000, SZ_1M, SZ_1M, true);
+}
+FS_TEST_ARGS(fs_test_load_big_2g_last_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR });
+
+/* Test Case 8 - load first 1MB in 2GB region (offset 0x80000000) */
+static int fs_test_load_big_2g_first_norun(struct unit_test_state *uts)
+{
+ return check_load(uts, ut_str(2), 0x80000000, SZ_1M, SZ_1M, true);
+}
+FS_TEST_ARGS(fs_test_load_big_2g_first_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR });
+
+/* Test Case 9 - load 1MB crossing 2GB boundary (offset 0x7ff80000) */
+static int fs_test_load_big_2g_cross_norun(struct unit_test_state *uts)
+{
+ return check_load(uts, ut_str(2), 0x7ff80000, SZ_1M, SZ_1M, true);
+}
+FS_TEST_ARGS(fs_test_load_big_2g_cross_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR });
+
+/* Test Case 10 - load beyond file end (2MB from offset, only 1MB remains) */
+static int fs_test_load_beyond_norun(struct unit_test_state *uts)
+{
+ return check_load(uts, ut_str(2), 0x9c300000, SZ_2M, SZ_1M, false);
+}
+FS_TEST_ARGS(fs_test_load_beyond_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "big", UT_ARG_STR });
+
+/* Test Case 11 - write file */
+static int fs_test_write_norun(struct unit_test_state *uts)
+{
+ const char *small = ut_str(2);
+ loff_t actread, actwrite;
+ void *buf;
+
+ if (!fs_write_supported(uts))
+ return -EAGAIN;
+
+ ut_assertok(prep_fs(uts, SZ_1M, &buf));
+
+ /* Read small file */
+ ut_assertok(fs_read(getpath(uts, small, NULL), ADDR, 0, 0, &actread));
+ ut_asserteq(SZ_1M, actread);
+
+ /* Write it back with new name */
+ ut_assertok(set_fs(uts));
+ ut_assertok(fs_write(getpath(uts, small, ".w"), ADDR, 0, SZ_1M,
+ &actwrite));
+ ut_asserteq(SZ_1M, actwrite);
+
+ /* Read back and verify MD5 */
+ ut_assertok(set_fs(uts));
+ memset(buf, '\0', SZ_1M);
+ ut_assertok(fs_read(getpath(uts, small, ".w"), ADDR, 0, 0, &actread));
+ ut_asserteq(SZ_1M, actread);
+
+ ut_assertok(verify_md5(uts, buf, SZ_1M));
+
+ return 0;
+}
+FS_TEST_ARGS(fs_test_write_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "small", UT_ARG_STR }, { "md5val", UT_ARG_STR });
+
+/* Test Case 12 - write to "." directory (should fail) */
+static int fs_test_write_dot_norun(struct unit_test_state *uts)
+{
+ loff_t actwrite;
+
+ if (!fs_write_supported(uts))
+ return -EAGAIN;
+
+ ut_assertok(prep_fs(uts, 0, NULL));
+
+ /* Writing to "." should fail */
+ ut_assert(fs_write("/.", ADDR, 0, 0x10, &actwrite));
+
+ return 0;
+}
+FS_TEST_ARGS(fs_test_write_dot_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS);
+
+/* Test Case 13 - write via "./" path */
+static int fs_test_write_dotpath_norun(struct unit_test_state *uts)
+{
+ const char *small = ut_str(2);
+ loff_t actread, actwrite;
+ void *buf;
+
+ if (!fs_write_supported(uts))
+ return -EAGAIN;
+
+ ut_assertok(prep_fs(uts, SZ_1M, &buf));
+
+ /* Read small file */
+ ut_assertok(fs_read(getpath(uts, small, NULL), ADDR, 0, 0, &actread));
+ ut_asserteq(SZ_1M, actread);
+
+ /* Write via "./" path */
+ ut_assertok(set_fs(uts));
+ snprintf(uts->priv, sizeof(uts->priv), "/./%s2", small);
+ ut_assertok(fs_write(uts->priv, ADDR, 0, SZ_1M, &actwrite));
+ ut_asserteq(SZ_1M, actwrite);
+
+ /* Read back via "./" path and verify */
+ ut_assertok(set_fs(uts));
+ memset(buf, '\0', SZ_1M);
+ ut_assertok(fs_read(uts->priv, ADDR, 0, 0, &actread));
+ ut_asserteq(SZ_1M, actread);
+ ut_assertok(verify_md5(uts, buf, SZ_1M));
+
+ /* Also verify via normal path */
+ ut_assertok(set_fs(uts));
+ memset(buf, '\0', SZ_1M);
+ ut_assertok(fs_read(getpath(uts, small, "2"), ADDR, 0, 0, &actread));
+ ut_asserteq(SZ_1M, actread);
+ ut_assertok(verify_md5(uts, buf, SZ_1M));
+
+ return 0;
+}
+FS_TEST_ARGS(fs_test_write_dotpath_norun, UTF_SCAN_FDT | UTF_MANUAL,
+ COMMON_ARGS, { "small", UT_ARG_STR }, { "md5val", UT_ARG_STR });
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 12/12] test: fs: Update Python tests to call C implementations
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (10 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 11/12] test: fs: add C-based filesystem tests Simon Glass
@ 2026-04-12 11:19 ` Simon Glass
2026-04-13 15:29 ` [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Tom Rini
2026-04-13 20:13 ` Tom Rini
13 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-12 11:19 UTC (permalink / raw)
To: u-boot; +Cc: Tom Rini, Heinrich Schuchardt, Simon Glass, Marek Vasut
Update test_basic.py to call the new C-based filesystem tests via the
'ut' command. This allows tests to run more directly, calling the actual
FS layer, without having to go through the command interface and
worrying about which filesystem command to use.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
test/py/tests/test_fs/conftest.py | 4 +-
test/py/tests/test_fs/test_basic.py | 346 ++++++++--------------------
2 files changed, 98 insertions(+), 252 deletions(-)
diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
index 0205048e73a..5fc796097e3 100644
--- a/test/py/tests/test_fs/conftest.py
+++ b/test/py/tests/test_fs/conftest.py
@@ -198,8 +198,6 @@ def fs_obj_basic(request, u_boot_config):
volume file name and a list of MD5 hashes.
"""
fs_type = request.param
- fs_cmd_prefix = fstype_to_prefix(fs_type)
- fs_cmd_write = 'save' if fs_type == 'fs_generic' or fs_type == 'exfat' else 'write'
fs_img = ''
fs_ubtype = fstype_to_ubname(fs_type)
@@ -289,7 +287,7 @@ def fs_obj_basic(request, u_boot_config):
pytest.skip('Setup failed for filesystem: ' + fs_type + '. {}'.format(err))
return
else:
- yield [fs_ubtype, fs_cmd_prefix, fs_cmd_write, fs_img, md5val]
+ yield [fs_ubtype, fs_img, md5val]
finally:
call('rm -rf %s' % scratch_dir, shell=True)
call('rm -f %s' % fs_img, shell=True)
diff --git a/test/py/tests/test_fs/test_basic.py b/test/py/tests/test_fs/test_basic.py
index 88b163ce305..7f805d04dd5 100644
--- a/test/py/tests/test_fs/test_basic.py
+++ b/test/py/tests/test_fs/test_basic.py
@@ -6,298 +6,146 @@
"""
This test verifies basic read/write operation on file system.
+
+Tests are implemented in C (test/fs/fs_basic.c) and called from here.
+Python handles filesystem image setup and environment variable configuration.
"""
import pytest
-import re
-from fstest_defs import *
+from fstest_defs import SMALL_FILE, BIG_FILE
from fstest_helpers import assert_fs_integrity
-@pytest.mark.boardspec('sandbox')
-@pytest.mark.slow
-class TestFsBasic(object):
- def test_fs1(self, ubman, fs_obj_basic):
- """
- Test Case 1 - ls command, listing a root directory and invalid directory
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
- with ubman.log.section('Test Case 1a - ls'):
- # Test Case 1 - ls
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sls host 0:0' % fs_cmd_prefix])
- assert(re.search('2621440000 *%s' % BIG_FILE, ''.join(output)))
- assert(re.search('1048576 *%s' % SMALL_FILE, ''.join(output)))
- with ubman.log.section('Test Case 1b - ls (invalid dir)'):
- # In addition, test with a nonexistent directory to see if we crash.
- output = ubman.run_command(
- '%sls host 0:0 invalid_d' % fs_cmd_prefix)
- assert('' == output)
+def run_c_test(ubman, fs_type, fs_img, test_name, small=None, big=None,
+ md5val=None):
+ """Run a C unit test with proper setup.
- with ubman.log.section('Test Case 1c - test -e'):
- # Test Case 1 - test -e
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- 'test -e host 0:0 1MB.file && echo PASS'])
- assert('PASS' in ''.join(output))
+ Args:
+ ubman (ConsoleBase): U-Boot console manager
+ fs_type (str): Filesystem type (ext4, fat, fs_generic, exfat)
+ fs_img (str): Path to filesystem image
+ test_name (str): Name of C test function (without _norun suffix)
+ small (str): Filename of small test file (optional)
+ big (str): Filename of big test file (optional)
+ md5val (str): Expected MD5 value for verification (optional)
- with ubman.log.section('Test Case 1d - test -e (invalid file)'):
- # In addition, test with a nonexistent file to see if we crash.
- output = ubman.run_command(
- 'test -e host 0:0 2MB.file || echo PASS')
- assert('PASS' in ''.join(output))
+ Returns:
+ bool: True if test passed, False otherwise
+ """
+ # Build the command with arguments
+ cmd = f'ut -f fs {test_name}_norun fs_type={fs_type} fs_image={fs_img}'
+ if small:
+ cmd += f' small={small}'
+ if big:
+ cmd += f' big={big}'
+ if md5val:
+ cmd += f' md5val={md5val}'
- def test_fs2(self, ubman, fs_obj_basic):
- """
- Test Case 2 - size command for a small file
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
- with ubman.log.section('Test Case 2a - size (small)'):
- # 1MB is 0x0010 0000
- # Test Case 2a - size of small file
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%ssize host 0:0 /%s' % (fs_cmd_prefix, SMALL_FILE),
- 'printenv filesize',
- 'setenv filesize'])
- assert('filesize=100000' in ''.join(output))
+ # Run the C test
+ ubman.run_command(cmd)
+
+ # Check result
+ result = ubman.run_command('echo $?')
+ return result.strip() == '0'
+
+
+@pytest.mark.boardspec('sandbox')
+@pytest.mark.slow
+class TestFsBasic:
+ """Test basic filesystem operations via C unit tests."""
+
+ def test_fs1(self, ubman, fs_obj_basic):
+ """Test Case 1 - ls command, listing root and invalid directories"""
+ fs_type, fs_img, _ = fs_obj_basic
+ with ubman.log.section('Test Case 1 - ls'):
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_ls',
+ small=SMALL_FILE, big=BIG_FILE)
- with ubman.log.section('Test Case 2b - size (/../<file>)'):
- # Test Case 2b - size of small file via a path using '..'
- output = ubman.run_command_list([
- '%ssize host 0:0 /SUBDIR/../%s' % (fs_cmd_prefix, SMALL_FILE),
- 'printenv filesize',
- 'setenv filesize'])
- assert('filesize=100000' in ''.join(output))
+ def test_fs2(self, ubman, fs_obj_basic):
+ """Test Case 2 - size command for a small file"""
+ fs_type, fs_img, _ = fs_obj_basic
+ with ubman.log.section('Test Case 2 - size (small)'):
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_small',
+ small=SMALL_FILE)
def test_fs3(self, ubman, fs_obj_basic):
- """
- Test Case 3 - size command for a large file
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 3 - size command for a large file"""
+ fs_type, fs_img, _ = fs_obj_basic
with ubman.log.section('Test Case 3 - size (large)'):
- # 2.5GB (1024*1024*2500) is 0x9C40 0000
- # Test Case 3 - size of big file
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%ssize host 0:0 /%s' % (fs_cmd_prefix, BIG_FILE),
- 'printenv filesize',
- 'setenv filesize'])
- assert('filesize=9c400000' in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_big',
+ big=BIG_FILE)
def test_fs4(self, ubman, fs_obj_basic):
- """
- Test Case 4 - load a small file, 1MB
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 4 - load a small file, 1MB"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 4 - load (small)'):
- # Test Case 4a - Read full 1MB of small file
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s' % (fs_cmd_prefix, ADDR, SMALL_FILE),
- 'printenv filesize'])
- assert('filesize=100000' in ''.join(output))
-
- # Test Case 4b - Read full 1MB of small file
- output = ubman.run_command_list([
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[0] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_small',
+ small=SMALL_FILE, md5val=md5val[0])
def test_fs5(self, ubman, fs_obj_basic):
- """
- Test Case 5 - load, reading first 1MB of 3GB file
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 5 - load, reading first 1MB of 3GB file"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 5 - load (first 1MB)'):
- # Test Case 5a - First 1MB of big file
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s %x 0x0' % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH),
- 'printenv filesize'])
- assert('filesize=100000' in ''.join(output))
-
- # Test Case 5b - First 1MB of big file
- output = ubman.run_command_list([
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[1] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_big_first',
+ big=BIG_FILE, md5val=md5val[1])
def test_fs6(self, ubman, fs_obj_basic):
- """
- Test Case 6 - load, reading last 1MB of 3GB file
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 6 - load, reading last 1MB of 3GB file"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 6 - load (last 1MB)'):
- # fails for ext as no offset support
- # Test Case 6a - Last 1MB of big file
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s %x 0x9c300000'
- % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH),
- 'printenv filesize'])
- assert('filesize=100000' in ''.join(output))
-
- # Test Case 6b - Last 1MB of big file
- output = ubman.run_command_list([
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[2] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_big_last',
+ big=BIG_FILE, md5val=md5val[2])
def test_fs7(self, ubman, fs_obj_basic):
- """
- Test Case 7 - load, 1MB from the last 1MB in 2GB
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 7 - load, 1MB from the last 1MB in 2GB"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 7 - load (last 1MB in 2GB)'):
- # fails for ext as no offset support
- # Test Case 7a - One from the last 1MB chunk of 2GB
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s %x 0x7ff00000'
- % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH),
- 'printenv filesize'])
- assert('filesize=100000' in ''.join(output))
-
- # Test Case 7b - One from the last 1MB chunk of 2GB
- output = ubman.run_command_list([
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[3] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img,
+ 'fs_test_load_big_2g_last',
+ big=BIG_FILE, md5val=md5val[3])
def test_fs8(self, ubman, fs_obj_basic):
- """
- Test Case 8 - load, reading first 1MB in 2GB
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 8 - load, reading first 1MB in 2GB"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 8 - load (first 1MB in 2GB)'):
- # fails for ext as no offset support
- # Test Case 8a - One from the start 1MB chunk from 2GB
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s %x 0x80000000'
- % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH),
- 'printenv filesize'])
- assert('filesize=100000' in ''.join(output))
-
- # Test Case 8b - One from the start 1MB chunk from 2GB
- output = ubman.run_command_list([
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[4] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img,
+ 'fs_test_load_big_2g_first',
+ big=BIG_FILE, md5val=md5val[4])
def test_fs9(self, ubman, fs_obj_basic):
- """
- Test Case 9 - load, 1MB crossing 2GB boundary
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 9 - load, 1MB crossing 2GB boundary"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 9 - load (crossing 2GB boundary)'):
- # fails for ext as no offset support
- # Test Case 9a - One 1MB chunk crossing the 2GB boundary
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s %x 0x7ff80000'
- % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH),
- 'printenv filesize'])
- assert('filesize=100000' in ''.join(output))
-
- # Test Case 9b - One 1MB chunk crossing the 2GB boundary
- output = ubman.run_command_list([
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[5] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img,
+ 'fs_test_load_big_2g_cross',
+ big=BIG_FILE, md5val=md5val[5])
def test_fs10(self, ubman, fs_obj_basic):
- """
- Test Case 10 - load, reading beyond file end'):
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 10 - load, reading beyond file end"""
+ fs_type, fs_img, _ = fs_obj_basic
with ubman.log.section('Test Case 10 - load (beyond file end)'):
- # Generic failure case
- # Test Case 10 - 2MB chunk from the last 1MB of big file
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s 0x00200000 0x9c300000'
- % (fs_cmd_prefix, ADDR, BIG_FILE),
- 'printenv filesize',
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert('filesize=100000' in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_beyond',
+ big=BIG_FILE)
def test_fs11(self, ubman, fs_obj_basic):
- """
- Test Case 11 - write'
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 11 - write"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 11 - write'):
- # Read 1MB from small file
- # Write it back to test the writes
- # Test Case 11a - Check that the write succeeded
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s' % (fs_cmd_prefix, ADDR, SMALL_FILE),
- '%s%s host 0:0 %x /%s.w $filesize'
- % (fs_cmd_prefix, fs_cmd_write, ADDR, SMALL_FILE)])
- assert('1048576 bytes written' in ''.join(output))
-
- # Test Case 11b - Check md5 of written to is same
- # as the one read from
- output = ubman.run_command_list([
- '%sload host 0:0 %x /%s.w' % (fs_cmd_prefix, ADDR, SMALL_FILE),
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[0] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write',
+ small=SMALL_FILE, md5val=md5val[0])
assert_fs_integrity(fs_type, fs_img)
def test_fs12(self, ubman, fs_obj_basic):
- """
- Test Case 12 - write to "." directory
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 12 - write to "." directory"""
+ fs_type, fs_img, _ = fs_obj_basic
with ubman.log.section('Test Case 12 - write (".")'):
- # Next test case checks writing a file whose dirent
- # is the first in the block, which is always true for "."
- # The write should fail, but the lookup should work
- # Test Case 12 - Check directory traversal
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%s%s host 0:0 %x /. 0x10'
- % (fs_cmd_prefix, fs_cmd_write, ADDR)])
- assert('Unable to write' in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write_dot')
assert_fs_integrity(fs_type, fs_img)
def test_fs13(self, ubman, fs_obj_basic):
- """
- Test Case 13 - write to a file with "/./<filename>"
- """
- fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic
+ """Test Case 13 - write to a file with '/./<filename>'"""
+ fs_type, fs_img, md5val = fs_obj_basic
with ubman.log.section('Test Case 13 - write ("./<file>")'):
- # Read 1MB from small file
- # Write it via "same directory", i.e. "." dirent
- # Test Case 13a - Check directory traversal
- output = ubman.run_command_list([
- 'host bind 0 %s' % fs_img,
- '%sload host 0:0 %x /%s' % (fs_cmd_prefix, ADDR, SMALL_FILE),
- '%s%s host 0:0 %x /./%s2 $filesize'
- % (fs_cmd_prefix, fs_cmd_write, ADDR, SMALL_FILE)])
- assert('1048576 bytes written' in ''.join(output))
-
- # Test Case 13b - Check md5 of written to is same
- # as the one read from
- output = ubman.run_command_list([
- 'mw.b %x 00 100' % ADDR,
- '%sload host 0:0 %x /./%s2' % (fs_cmd_prefix, ADDR, SMALL_FILE),
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[0] in ''.join(output))
-
- # Test Case 13c - Check md5 of written to is same
- # as the one read from
- output = ubman.run_command_list([
- 'mw.b %x 00 100' % ADDR,
- '%sload host 0:0 %x /%s2' % (fs_cmd_prefix, ADDR, SMALL_FILE),
- 'md5sum %x $filesize' % ADDR,
- 'setenv filesize'])
- assert(md5val[0] in ''.join(output))
+ assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write_dotpath',
+ small=SMALL_FILE, md5val=md5val[0])
assert_fs_integrity(fs_type, fs_img)
--
2.43.0
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH v2 00/12] test: Add support for passing arguments to C unit tests
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (11 preceding siblings ...)
2026-04-12 11:19 ` [PATCH v2 12/12] test: fs: Update Python tests to call C implementations Simon Glass
@ 2026-04-13 15:29 ` Tom Rini
2026-04-13 15:40 ` Simon Glass
2026-04-13 20:13 ` Tom Rini
13 siblings, 1 reply; 17+ messages in thread
From: Tom Rini @ 2026-04-13 15:29 UTC (permalink / raw)
To: Simon Glass
Cc: u-boot, Heinrich Schuchardt, Andrew Goodbody, Ilias Apalodimas,
Marek Vasut
[-- Attachment #1: Type: text/plain, Size: 2663 bytes --]
On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:
> Some tests have two parts: a Python test which handles setup, such as
> creating filesystem images or configuring the environment, and a C
> test which does the bulk of the work including assertions. At present
> there is no way to pass information between these two parts, so
> things like paths and device names must be hard-coded in the C test.
>
> This tends to make people write tests in Python, but this is not ideal
> for several reasons:
>
> - Python lacks access to things like return codes and has to scan the
> console output to figure out what happened
> - Python lacks access to internal state, so it cannot directly check
> the result of an operation
> - Python is ~100x slower due to console interaction, etc.
> - C tests can be stepped through in gdb, which is much harder with
> Python driving the console
> - C tests can exercise internal APIs directly (e.g. fs_read(),
> fs_size()) rather than going through the command layer, so they
> test more precisely what they intend to
>
> This series adds infrastructure for passing typed, runtime arguments
> from Python to C unit tests, using a name=value format on the ut
> command line.
>
> The series includes:
>
> - Regex assertion helpers for matching variable output in tests
>
> - A private buffer in unit_test_state for test-local temporary data
>
> - Argument-type definitions (string, integer, boolean) with optional
> flags and default values
>
> - A new UNIT_TEST_ARGS() macro for declaring tests with typed
> arguments
>
> - Argument parsing in the test framework, accepting name=value format
>
> - Updates to the ut command to pass arguments through to tests
>
> - Type-checked accessor macros ut_str(), ut_int(), and ut_bool()
> with bounds validation
>
> - Tests for the argument feature covering type checking, optional
> arguments, and argument-parsing failures
>
> - Documentation for the test-parameter feature
>
> - C-based filesystem tests as an example of the hybrid approach,
> with Python wrappers that pass filesystem type, image path, and
> expected MD5 values to the C tests
>
> Note: This series depends on:
>
> https://patchwork.ozlabs.org/project/uboot/list/?series=496972
>
> Changes in v2:
> - Correct double signoff
Reposting a series this large, on the weekend, the day after v1, because
you made a trivial mistake is bad form. You should just acknowledge that
in the cover letter and wait for feedback.
And since I'm seeing over 100 emails from you this weekend, this
matters.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v2 00/12] test: Add support for passing arguments to C unit tests
2026-04-13 15:29 ` [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Tom Rini
@ 2026-04-13 15:40 ` Simon Glass
0 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-13 15:40 UTC (permalink / raw)
To: Tom Rini
Cc: u-boot, Heinrich Schuchardt, Andrew Goodbody, Ilias Apalodimas,
Marek Vasut
Hi Tom,
On Mon, 13 Apr 2026 at 09:29, Tom Rini <trini@konsulko.com> wrote:
>
> On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:
>
> > Some tests have two parts: a Python test which handles setup, such as
> > creating filesystem images or configuring the environment, and a C
> > test which does the bulk of the work including assertions. At present
> > there is no way to pass information between these two parts, so
> > things like paths and device names must be hard-coded in the C test.
> >
> > This tends to make people write tests in Python, but this is not ideal
> > for several reasons:
> >
> > - Python lacks access to things like return codes and has to scan the
> > console output to figure out what happened
> > - Python lacks access to internal state, so it cannot directly check
> > the result of an operation
> > - Python is ~100x slower due to console interaction, etc.
> > - C tests can be stepped through in gdb, which is much harder with
> > Python driving the console
> > - C tests can exercise internal APIs directly (e.g. fs_read(),
> > fs_size()) rather than going through the command layer, so they
> > test more precisely what they intend to
> >
> > This series adds infrastructure for passing typed, runtime arguments
> > from Python to C unit tests, using a name=value format on the ut
> > command line.
> >
> > The series includes:
> >
> > - Regex assertion helpers for matching variable output in tests
> >
> > - A private buffer in unit_test_state for test-local temporary data
> >
> > - Argument-type definitions (string, integer, boolean) with optional
> > flags and default values
> >
> > - A new UNIT_TEST_ARGS() macro for declaring tests with typed
> > arguments
> >
> > - Argument parsing in the test framework, accepting name=value format
> >
> > - Updates to the ut command to pass arguments through to tests
> >
> > - Type-checked accessor macros ut_str(), ut_int(), and ut_bool()
> > with bounds validation
> >
> > - Tests for the argument feature covering type checking, optional
> > arguments, and argument-parsing failures
> >
> > - Documentation for the test-parameter feature
> >
> > - C-based filesystem tests as an example of the hybrid approach,
> > with Python wrappers that pass filesystem type, image path, and
> > expected MD5 values to the C tests
> >
> > Note: This series depends on:
> >
> > https://patchwork.ozlabs.org/project/uboot/list/?series=496972
> >
> > Changes in v2:
> > - Correct double signoff
>
> Reposting a series this large, on the weekend, the day after v1, because
> you made a trivial mistake is bad form. You should just acknowledge that
> in the cover letter and wait for feedback.
OK I'll do that if I make this mistake again.
> And since I'm seeing over 100 emails from you this weekend, this
> matters.
Yes, I was trying to catch up on various things.
Regards,
Simon
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 00/12] test: Add support for passing arguments to C unit tests
2026-04-12 11:19 [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Simon Glass
` (12 preceding siblings ...)
2026-04-13 15:29 ` [PATCH v2 00/12] test: Add support for passing arguments to C unit tests Tom Rini
@ 2026-04-13 20:13 ` Tom Rini
2026-04-16 17:42 ` Simon Glass
13 siblings, 1 reply; 17+ messages in thread
From: Tom Rini @ 2026-04-13 20:13 UTC (permalink / raw)
To: Simon Glass
Cc: u-boot, Heinrich Schuchardt, Andrew Goodbody, Ilias Apalodimas,
Marek Vasut
[-- Attachment #1: Type: text/plain, Size: 2514 bytes --]
On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:
> Some tests have two parts: a Python test which handles setup, such as
> creating filesystem images or configuring the environment, and a C
> test which does the bulk of the work including assertions. At present
> there is no way to pass information between these two parts, so
> things like paths and device names must be hard-coded in the C test.
>
> This tends to make people write tests in Python, but this is not ideal
> for several reasons:
You still need to work on explaining your changes in a positive manner.
> - Python lacks access to things like return codes and has to scan the
> console output to figure out what happened
Pro: Python exercises things the way users exercise them and so exposes
deficiencies in our user interface as well, when applicable.
> - Python lacks access to internal state, so it cannot directly check
> the result of an operation
It's unclear when that is, or is not a problem and when tests that do
check intermediate state are providing value.
> - Python is ~100x slower due to console interaction, etc.
Again, maybe you should go back to your suggestion of several years ago
to see about addressing that, as the tests themselves are not slow, it's
the pipe? Our brave new AI powered world runs on Python handing stuff
off and getting it back super quickly.
> - C tests can be stepped through in gdb, which is much harder with
> Python driving the console
> - C tests can exercise internal APIs directly (e.g. fs_read(),
> fs_size()) rather than going through the command layer, so they
> test more precisely what they intend to
All things that may or may not be problems for other people, or are
intentional choices as they again test the system the way it's used.
Using gdb attached to u-boot in the other terminal where you're running
breaking things sounds like a reasonable debug path to me. And having
unit tests calling in to functions is why today we have a lot of things
not marked as "static" and so not being optimized as well as they could
by the compiler, which is its own set of trade-offs and not an automatic
good thing and design win.
[snip]
> 16 files changed, 1437 insertions(+), 279 deletions(-)
All of which is to say, you need to explain yourself in a positive, not
negative or oppositional way. Otherwise this reads like "I don't like
what we do now, here's a bunch of change to what I like instead".
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v2 00/12] test: Add support for passing arguments to C unit tests
2026-04-13 20:13 ` Tom Rini
@ 2026-04-16 17:42 ` Simon Glass
0 siblings, 0 replies; 17+ messages in thread
From: Simon Glass @ 2026-04-16 17:42 UTC (permalink / raw)
To: Tom Rini
Cc: u-boot, Heinrich Schuchardt, Andrew Goodbody, Ilias Apalodimas,
Marek Vasut
Hi Tom,
On Mon, 13 Apr 2026 at 14:13, Tom Rini <trini@konsulko.com> wrote:
>
> On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:
> > Some tests have two parts: a Python test which handles setup, such as
> > creating filesystem images or configuring the environment, and a C
> > test which does the bulk of the work including assertions. At present
> > there is no way to pass information between these two parts, so
> > things like paths and device names must be hard-coded in the C test.
> >
> > This tends to make people write tests in Python, but this is not ideal
> > for several reasons:
>
> You still need to work on explaining your changes in a positive manner.
>
> > - Python lacks access to things like return codes and has to scan the
> > console output to figure out what happened
>
> Pro: Python exercises things the way users exercise them and so exposes
> deficiencies in our user interface as well, when applicable.
>
> > - Python lacks access to internal state, so it cannot directly check
> > the result of an operation
>
> It's unclear when that is, or is not a problem and when tests that do
> check intermediate state are providing value.
>
> > - Python is ~100x slower due to console interaction, etc.
>
> Again, maybe you should go back to your suggestion of several years ago
> to see about addressing that, as the tests themselves are not slow, it's
> the pipe? Our brave new AI powered world runs on Python handing stuff
> off and getting it back super quickly.
>
> > - C tests can be stepped through in gdb, which is much harder with
> > Python driving the console
> > - C tests can exercise internal APIs directly (e.g. fs_read(),
> > fs_size()) rather than going through the command layer, so they
> > test more precisely what they intend to
>
> All things that may or may not be problems for other people, or are
> intentional choices as they again test the system the way it's used.
> Using gdb attached to u-boot in the other terminal where you're running
> breaking things sounds like a reasonable debug path to me. And having
> unit tests calling in to functions is why today we have a lot of things
> not marked as "static" and so not being optimized as well as they could
> by the compiler, which is its own set of trade-offs and not an automatic
> good thing and design win.
>
> [snip]
> > 16 files changed, 1437 insertions(+), 279 deletions(-)
>
> All of which is to say, you need to explain yourself in a positive, not
> negative or oppositional way. Otherwise this reads like "I don't like
> what we do now, here's a bunch of change to what I like instead".
Just to check, is this a discussion and you would like me to reply
with more details about the benefits of C over Python in particular
circumstances? Or are you wanting the cover-letter rewritten. Or is
this just a NAK?
I did some work on using a socket to talk to sandbox (it is faster and
also avoids some of the confusion as to what state sandbox is in
without having to rely on looking at console output). But I never
quite finished it. The other option (which we briefly discussed years
ago) would be to build U-Boot as a Python module. I haven't tried that
at all, but it's an idea.
I am well aware of the tradeoffs between Python and C, so I am not
advocating one over the other in all cases. I use gdb a lot with the
pytests - e.g. hunting down memory leaks recent. Really I just want
them to coexist better - i.e. improving the sandbox pattern where
Python sets something up and then C runs it.
- Simon
^ permalink raw reply [flat|nested] 17+ messages in thread