[LTP] [PATCH v3 0/4] Shell test library v3

[LTP] [PATCH v3 0/4] Shell test library v3

[Cyril Hrubis]

Changes in v4:

- Added fix for MUSL build failures (new patch)
  (as reported by Peter Vorel)

- Fixes requested by Ritchie

  - Constified the first tst_run_shell() parameter

  - Added GP_JSON_ATTR_IDX() macro so that we can make sure
    the order is not broken when parsing JSON environment metadata

  - Changed tst_run_shell() to tst_run_script()

  - Changed the format of the metadata comment and added support
    for documentation support, now it looks like:

    # ---
    # doc
    #
    # [Description]
    #
    # Test description ...
    # ---
    #
    # ---
    # env
    #
    # {
    #  "some_key": "some_value"
    #  ...
    # }
    # ---

Cyril Hrubis (4):
  include: tst_clone.h: Fix possible MUSL build failures
  Add support for mixing C and shell code
  libs: Vendor ujson library
  testcaes/lib: Add shell loader

 include/old/test.h                            |    1 -
 include/tst_clone.h                           |    2 +
 include/tst_test.h                            |   39 +-
 include/ujson.h                               |   13 +
 include/ujson_common.h                        |   69 ++
 include/ujson_reader.h                        |  543 +++++++++
 include/ujson_utf.h                           |  168 +++
 include/ujson_writer.h                        |  224 ++++
 lib/tst_test.c                                |   51 +
 libs/ujson/Makefile                           |   12 +
 libs/ujson/ujson_common.c                     |   38 +
 libs/ujson/ujson_reader.c                     | 1081 +++++++++++++++++
 libs/ujson/ujson_utf.c                        |  105 ++
 libs/ujson/ujson_writer.c                     |  491 ++++++++
 testcases/kernel/syscalls/clone/clone02.c     |    1 +
 testcases/lib/.gitignore                      |    2 +
 testcases/lib/Makefile                        |    8 +-
 testcases/lib/run_tests.sh                    |   32 +
 testcases/lib/tests/.gitignore                |    6 +
 testcases/lib/tests/Makefile                  |   11 +
 testcases/lib/tests/shell_loader.sh           |   26 +
 .../lib/tests/shell_loader_all_filesystems.sh |   27 +
 .../lib/tests/shell_loader_filesystems.sh     |   33 +
 .../lib/tests/shell_loader_invalid_block.sh   |   26 +
 .../tests/shell_loader_invalid_metadata.sh    |   15 +
 testcases/lib/tests/shell_loader_kconfigs.sh  |   12 +
 .../lib/tests/shell_loader_no_metadata.sh     |    8 +
 .../lib/tests/shell_loader_supported_archs.sh |   12 +
 testcases/lib/tests/shell_loader_tags.sh      |   15 +
 testcases/lib/tests/shell_loader_tcnt.sh      |   15 +
 .../lib/tests/shell_loader_wrong_metadata.sh  |   15 +
 testcases/lib/tests/shell_test01.c            |   17 +
 testcases/lib/tests/shell_test02.c            |   18 +
 testcases/lib/tests/shell_test03.c            |   25 +
 testcases/lib/tests/shell_test04.c            |   18 +
 testcases/lib/tests/shell_test05.c            |   27 +
 testcases/lib/tests/shell_test06.c            |   16 +
 testcases/lib/tests/shell_test_brk.sh         |    6 +
 testcases/lib/tests/shell_test_check_argv.sh  |   23 +
 testcases/lib/tests/shell_test_checkpoint.sh  |    7 +
 testcases/lib/tests/shell_test_pass.sh        |    6 +
 testcases/lib/tst_env.sh                      |   25 +
 testcases/lib/tst_loader.sh                   |   11 +
 testcases/lib/tst_res_.c                      |   58 +
 testcases/lib/tst_run_shell.c                 |  491 ++++++++
 45 files changed, 3845 insertions(+), 4 deletions(-)
 create mode 100644 include/ujson.h
 create mode 100644 include/ujson_common.h
 create mode 100644 include/ujson_reader.h
 create mode 100644 include/ujson_utf.h
 create mode 100644 include/ujson_writer.h
 create mode 100644 libs/ujson/Makefile
 create mode 100644 libs/ujson/ujson_common.c
 create mode 100644 libs/ujson/ujson_reader.c
 create mode 100644 libs/ujson/ujson_utf.c
 create mode 100644 libs/ujson/ujson_writer.c
 create mode 100755 testcases/lib/run_tests.sh
 create mode 100644 testcases/lib/tests/.gitignore
 create mode 100644 testcases/lib/tests/Makefile
 create mode 100755 testcases/lib/tests/shell_loader.sh
 create mode 100755 testcases/lib/tests/shell_loader_all_filesystems.sh
 create mode 100755 testcases/lib/tests/shell_loader_filesystems.sh
 create mode 100755 testcases/lib/tests/shell_loader_invalid_block.sh
 create mode 100755 testcases/lib/tests/shell_loader_invalid_metadata.sh
 create mode 100755 testcases/lib/tests/shell_loader_kconfigs.sh
 create mode 100755 testcases/lib/tests/shell_loader_no_metadata.sh
 create mode 100755 testcases/lib/tests/shell_loader_supported_archs.sh
 create mode 100755 testcases/lib/tests/shell_loader_tags.sh
 create mode 100755 testcases/lib/tests/shell_loader_tcnt.sh
 create mode 100755 testcases/lib/tests/shell_loader_wrong_metadata.sh
 create mode 100644 testcases/lib/tests/shell_test01.c
 create mode 100644 testcases/lib/tests/shell_test02.c
 create mode 100644 testcases/lib/tests/shell_test03.c
 create mode 100644 testcases/lib/tests/shell_test04.c
 create mode 100644 testcases/lib/tests/shell_test05.c
 create mode 100644 testcases/lib/tests/shell_test06.c
 create mode 100755 testcases/lib/tests/shell_test_brk.sh
 create mode 100755 testcases/lib/tests/shell_test_check_argv.sh
 create mode 100755 testcases/lib/tests/shell_test_checkpoint.sh
 create mode 100755 testcases/lib/tests/shell_test_pass.sh
 create mode 100644 testcases/lib/tst_env.sh
 create mode 100644 testcases/lib/tst_loader.sh
 create mode 100644 testcases/lib/tst_res_.c
 create mode 100644 testcases/lib/tst_run_shell.c

-- 
2.44.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 1/4] include: tst_clone.h: Fix possible MUSL build failures

[Cyril Hrubis]

The problem is that on musl sched.h exposes clone() when _GNU_SOURCE is
defined and at the same time sched.h does not get pulled before
tst_clone.h gets included, which means that the macro from tst_clone.h
that rewrites clone() functions actually rewrites the function
declaration in the system header.

We remove the tst_clone.h from the old library because the newly
included sched.h causes conflicts in the cpuset/cpuset_lib/ directory
and the tst_clone.h header is used only in a single old library test
i.e. clone02.c.

This commit is needed in order to avoid build failures in the next
commits that add support for the shell test library.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 include/old/test.h                        | 1 -
 include/tst_clone.h                       | 2 ++
 testcases/kernel/syscalls/clone/clone02.c | 1 +
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/include/old/test.h b/include/old/test.h
index 0e210e4ef..306868fb5 100644
--- a/include/old/test.h
+++ b/include/old/test.h
@@ -31,7 +31,6 @@
 #include "tst_pid.h"
 #include "tst_cmd.h"
 #include "tst_cpu.h"
-#include "tst_clone.h"
 #include "old_device.h"
 #include "old_tmpdir.h"
 #include "tst_minmax.h"
diff --git a/include/tst_clone.h b/include/tst_clone.h
index 56f23142d..a57d761ca 100644
--- a/include/tst_clone.h
+++ b/include/tst_clone.h
@@ -5,6 +5,8 @@
 #ifndef TST_CLONE_H__
 #define TST_CLONE_H__
 
+#include <sched.h>
+
 #ifdef TST_TEST_H__
 
 /* The parts of clone3's clone_args we support */
diff --git a/testcases/kernel/syscalls/clone/clone02.c b/testcases/kernel/syscalls/clone/clone02.c
index 821adc2d9..fd3ee1aed 100644
--- a/testcases/kernel/syscalls/clone/clone02.c
+++ b/testcases/kernel/syscalls/clone/clone02.c
@@ -59,6 +59,7 @@
 #include <sched.h>
 #include "test.h"
 #include "safe_macros.h"
+#include "tst_clone.h"
 
 #define FLAG_ALL (CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|SIGCHLD)
 #define FLAG_NONE SIGCHLD
-- 
2.44.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 1/4] include: tst_clone.h: Fix possible MUSL build failures

[Petr Vorel]

Hi Cyril,

> The problem is that on musl sched.h exposes clone() when _GNU_SOURCE is
> defined and at the same time sched.h does not get pulled before
> tst_clone.h gets included, which means that the macro from tst_clone.h
> that rewrites clone() functions actually rewrites the function
> declaration in the system header.

BTW glibc guards clone() with #ifdef __USE_GNU, which should have the same
effect as _GNU_SOURCE. I wonder why only musl got broken.

> We remove the tst_clone.h from the old library because the newly
> included sched.h causes conflicts in the cpuset/cpuset_lib/ directory
> and the tst_clone.h header is used only in a single old library test
> i.e. clone02.c.

> This commit is needed in order to avoid build failures in the next
> commits that add support for the shell test library.

Thanks for fixing this!

Reported-by: Petr Vorel <pvorel@suse.cz>
Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 1/4] include: tst_clone.h: Fix possible MUSL build failures

[Cyril Hrubis]

Hi!
> BTW glibc guards clone() with #ifdef __USE_GNU, which should have the same
> effect as _GNU_SOURCE. I wonder why only musl got broken.

I guess that the definitions gets pulled from other header indirectly in
glibc which happens before we define the macro.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Cyril Hrubis]

This is a proof of a concept of a seamless C and shell integration. The
idea is that with this you can mix shell and C code as much as as you
wish to get the best of the two worlds.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
Reviewed-by: Richard Palethorpe <io@richiejp.com>
---
 include/tst_test.h                           | 37 +++++++++++++
 lib/tst_test.c                               | 51 +++++++++++++++++
 testcases/lib/.gitignore                     |  1 +
 testcases/lib/Makefile                       |  4 +-
 testcases/lib/run_tests.sh                   | 11 ++++
 testcases/lib/tests/.gitignore               |  6 ++
 testcases/lib/tests/Makefile                 | 11 ++++
 testcases/lib/tests/shell_test01.c           | 17 ++++++
 testcases/lib/tests/shell_test02.c           | 18 ++++++
 testcases/lib/tests/shell_test03.c           | 25 +++++++++
 testcases/lib/tests/shell_test04.c           | 18 ++++++
 testcases/lib/tests/shell_test05.c           | 27 +++++++++
 testcases/lib/tests/shell_test06.c           | 16 ++++++
 testcases/lib/tests/shell_test_brk.sh        |  6 ++
 testcases/lib/tests/shell_test_check_argv.sh | 23 ++++++++
 testcases/lib/tests/shell_test_checkpoint.sh |  7 +++
 testcases/lib/tests/shell_test_pass.sh       |  6 ++
 testcases/lib/tst_env.sh                     | 21 +++++++
 testcases/lib/tst_res_.c                     | 58 ++++++++++++++++++++
 19 files changed, 361 insertions(+), 2 deletions(-)
 create mode 100755 testcases/lib/run_tests.sh
 create mode 100644 testcases/lib/tests/.gitignore
 create mode 100644 testcases/lib/tests/Makefile
 create mode 100644 testcases/lib/tests/shell_test01.c
 create mode 100644 testcases/lib/tests/shell_test02.c
 create mode 100644 testcases/lib/tests/shell_test03.c
 create mode 100644 testcases/lib/tests/shell_test04.c
 create mode 100644 testcases/lib/tests/shell_test05.c
 create mode 100644 testcases/lib/tests/shell_test06.c
 create mode 100755 testcases/lib/tests/shell_test_brk.sh
 create mode 100755 testcases/lib/tests/shell_test_check_argv.sh
 create mode 100755 testcases/lib/tests/shell_test_checkpoint.sh
 create mode 100755 testcases/lib/tests/shell_test_pass.sh
 create mode 100644 testcases/lib/tst_env.sh
 create mode 100644 testcases/lib/tst_res_.c

diff --git a/include/tst_test.h b/include/tst_test.h
index afc6a5714..9871676a5 100644
--- a/include/tst_test.h
+++ b/include/tst_test.h
@@ -331,6 +331,8 @@ struct tst_fs {
  * @child_needs_reinit: Has to be set if the test needs to call tst_reinit()
  *                      from a process started by exec().
  *
+ * @runs_script: Implies child_needs_reinit and forks_child at the moment.
+ *
  * @needs_devfs: If set the devfs is mounted at tst_test.mntpoint. This is
  *               needed for tests that need to create device files since tmpfs
  *               at /tmp is usually mounted with 'nodev' option.
@@ -521,6 +523,7 @@ struct tst_fs {
 	unsigned int mount_device:1;
 	unsigned int needs_rofs:1;
 	unsigned int child_needs_reinit:1;
+	unsigned int runs_script:1;
 	unsigned int needs_devfs:1;
 	unsigned int restore_wallclock:1;
 
@@ -529,6 +532,8 @@ struct tst_fs {
 	unsigned int skip_in_lockdown:1;
 	unsigned int skip_in_secureboot:1;
 	unsigned int skip_in_compat:1;
+
+
 	int needs_abi_bits;
 
 	unsigned int needs_hugetlbfs:1;
@@ -616,6 +621,38 @@ void tst_run_tcases(int argc, char *argv[], struct tst_test *self)
  */
 void tst_reinit(void);
 
+/**
+ * tst_run_script() - Prepare the environment and execute a (shell) script.
+ *
+ * @script_name: A filename of the script.
+ * @params: A NULL terminated array of (shell) script parameters, pass NULL if
+ *          none are needed. This what is passed starting from argv[1].
+ *
+ * The (shell) script is executed with LTP_IPC_PATH in environment so that the
+ * binary helpers such as tst_res_ or tst_checkpoint work properly when executed
+ * from the script. This also means that the tst_test.runs_script flag needs to
+ * be set.
+ *
+ * A shell script has to source the tst_env.sh shell script at the start and
+ * after that it's free to use tst_res in the same way C code would use.
+ *
+ * Example shell script that reports success::
+ *
+ *   #!/bin/sh
+ *   . tst_env.sh
+ *
+ *   tst_res TPASS "Example test works"
+ *
+ * The call returns a pid in a case that you want to examine the return value
+ * of the script yourself. If you do not need to check the return value
+ * yourself you can use tst_reap_children() to wait for the completion. Or let
+ * the test library collect the child automatically, just be wary that the
+ * script and the test both runs concurently at the same time in this case.
+ *
+ * Return: A pid of the (shell) script process.
+ */
+int tst_run_script(const char *script_name, char *const params[]);
+
 unsigned int tst_multiply_timeout(unsigned int timeout);
 
 /*
diff --git a/lib/tst_test.c b/lib/tst_test.c
index 201b81e14..918bee2a1 100644
--- a/lib/tst_test.c
+++ b/lib/tst_test.c
@@ -4,6 +4,8 @@
  * Copyright (c) Linux Test Project, 2016-2024
  */
 
+#define _GNU_SOURCE
+
 #include <limits.h>
 #include <stdio.h>
 #include <stdarg.h>
@@ -174,6 +176,50 @@ void tst_reinit(void)
 	SAFE_CLOSE(fd);
 }
 
+extern char **environ;
+
+static unsigned int params_array_len(char *const array[])
+{
+	unsigned int ret = 0;
+
+	if (!array)
+		return 0;
+
+	while (*(array++))
+		ret++;
+
+	return ret;
+}
+
+int tst_run_script(const char *script_name, char *const params[])
+{
+	int pid;
+	unsigned int i, params_len = params_array_len(params);
+	char *argv[params_len + 2];
+
+	if (!tst_test->runs_script)
+		tst_brk(TBROK, "runs_script flag must be set!");
+
+	argv[0] = (char*)script_name;
+
+	if (params) {
+		for (i = 0; i < params_len; i++)
+			argv[i+1] = params[i];
+	}
+
+	argv[params_len+1] = NULL;
+
+	pid = SAFE_FORK();
+	if (pid)
+		return pid;
+
+	execvpe(script_name, argv, environ);
+
+	tst_brk(TBROK | TERRNO, "execvpe(%s, ...) failed!", script_name);
+
+	return -1;
+}
+
 static void update_results(int ttype)
 {
 	if (!results)
@@ -1226,6 +1272,11 @@ static void do_setup(int argc, char *argv[])
 		tdebug = 1;
 	}
 
+	if (tst_test->runs_script) {
+		tst_test->child_needs_reinit = 1;
+		tst_test->forks_child = 1;
+	}
+
 	if (tst_test->needs_kconfigs && tst_kconfig_check(tst_test->needs_kconfigs))
 		tst_brk(TCONF, "Aborting due to unsuitable kernel config, see above!");
 
diff --git a/testcases/lib/.gitignore b/testcases/lib/.gitignore
index e8afd06f3..d0dacf62a 100644
--- a/testcases/lib/.gitignore
+++ b/testcases/lib/.gitignore
@@ -23,3 +23,4 @@
 /tst_sleep
 /tst_supported_fs
 /tst_timeout_kill
+/tst_res_
diff --git a/testcases/lib/Makefile b/testcases/lib/Makefile
index 990b46089..928d76d62 100644
--- a/testcases/lib/Makefile
+++ b/testcases/lib/Makefile
@@ -13,6 +13,6 @@ MAKE_TARGETS		:= tst_sleep tst_random tst_checkpoint tst_rod tst_kvcmp\
 			   tst_getconf tst_supported_fs tst_check_drivers tst_get_unused_port\
 			   tst_get_median tst_hexdump tst_get_free_pids tst_timeout_kill\
 			   tst_check_kconfigs tst_cgctl tst_fsfreeze tst_ns_create tst_ns_exec\
-			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled
+			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_
 
-include $(top_srcdir)/include/mk/generic_leaf_target.mk
+include $(top_srcdir)/include/mk/generic_trunk_target.mk
diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh
new file mode 100755
index 000000000..60e7d1bcf
--- /dev/null
+++ b/testcases/lib/run_tests.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+
+testdir=$(realpath $(dirname $0))
+export PATH=$PATH:$testdir:$testdir/tests/
+
+for i in `seq -w 01 06`; do
+	echo
+	echo "*** Running shell_test$i ***"
+	echo
+	./tests/shell_test$i
+done
diff --git a/testcases/lib/tests/.gitignore b/testcases/lib/tests/.gitignore
new file mode 100644
index 000000000..da967c4d6
--- /dev/null
+++ b/testcases/lib/tests/.gitignore
@@ -0,0 +1,6 @@
+shell_test01
+shell_test02
+shell_test03
+shell_test04
+shell_test05
+shell_test06
diff --git a/testcases/lib/tests/Makefile b/testcases/lib/tests/Makefile
new file mode 100644
index 000000000..5a5cf5310
--- /dev/null
+++ b/testcases/lib/tests/Makefile
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (C) 2009, Cisco Systems Inc.
+# Ngie Cooper, August 2009
+
+top_srcdir		?= ../../..
+
+include $(top_srcdir)/include/mk/testcases.mk
+
+INSTALL_TARGETS=
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
diff --git a/testcases/lib/tests/shell_test01.c b/testcases/lib/tests/shell_test01.c
new file mode 100644
index 000000000..b9f07308e
--- /dev/null
+++ b/testcases/lib/tests/shell_test01.c
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Shell test example.
+ */
+
+#include "tst_test.h"
+
+static void run_test(void)
+{
+	tst_run_script("shell_test_pass.sh", NULL);
+	tst_res(TINFO, "C test exits now");
+}
+
+static struct tst_test test = {
+	.runs_script = 1,
+	.test_all = run_test,
+};
diff --git a/testcases/lib/tests/shell_test02.c b/testcases/lib/tests/shell_test02.c
new file mode 100644
index 000000000..087055794
--- /dev/null
+++ b/testcases/lib/tests/shell_test02.c
@@ -0,0 +1,18 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Shell test example.
+ */
+
+#include "tst_test.h"
+
+static void run_test(void)
+{
+	tst_run_script("shell_test_pass.sh", NULL);
+	tst_reap_children();
+	tst_res(TINFO, "Shell test has finished at this point!");
+}
+
+static struct tst_test test = {
+	.runs_script = 1,
+	.test_all = run_test,
+};
diff --git a/testcases/lib/tests/shell_test03.c b/testcases/lib/tests/shell_test03.c
new file mode 100644
index 000000000..61436891e
--- /dev/null
+++ b/testcases/lib/tests/shell_test03.c
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Shell test example.
+ */
+
+#include <sys/wait.h>
+#include "tst_test.h"
+
+static void run_test(void)
+{
+	int pid, status;
+
+	pid = tst_run_script("shell_test_pass.sh", NULL);
+
+	tst_res(TINFO, "Waiting for the pid %i", pid);
+
+	waitpid(pid, &status, 0);
+
+	tst_res(TINFO, "Shell test has %s", tst_strstatus(status));
+}
+
+static struct tst_test test = {
+	.runs_script = 1,
+	.test_all = run_test,
+};
diff --git a/testcases/lib/tests/shell_test04.c b/testcases/lib/tests/shell_test04.c
new file mode 100644
index 000000000..a32dd1e9f
--- /dev/null
+++ b/testcases/lib/tests/shell_test04.c
@@ -0,0 +1,18 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Shell test example.
+ */
+
+#include "tst_test.h"
+
+static void run_test(void)
+{
+	char *const params[] = {"param1", "param2", NULL};
+
+	tst_run_script("shell_test_check_argv.sh", params);
+}
+
+static struct tst_test test = {
+	.runs_script = 1,
+	.test_all = run_test,
+};
diff --git a/testcases/lib/tests/shell_test05.c b/testcases/lib/tests/shell_test05.c
new file mode 100644
index 000000000..771af8fc3
--- /dev/null
+++ b/testcases/lib/tests/shell_test05.c
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Shell test example.
+ */
+
+#include "tst_test.h"
+
+static void run_test(void)
+{
+	int pid;
+
+	pid = tst_run_script("shell_test_checkpoint.sh", NULL);
+
+	tst_res(TINFO, "Waiting for shell to sleep on checkpoint!");
+
+	TST_PROCESS_STATE_WAIT(pid, 'S', 10000);
+
+	tst_res(TINFO, "Waking shell child!");
+
+	TST_CHECKPOINT_WAKE(0);
+}
+
+static struct tst_test test = {
+	.runs_script = 1,
+	.needs_checkpoints = 1,
+	.test_all = run_test,
+};
diff --git a/testcases/lib/tests/shell_test06.c b/testcases/lib/tests/shell_test06.c
new file mode 100644
index 000000000..89d66bab0
--- /dev/null
+++ b/testcases/lib/tests/shell_test06.c
@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Shell test example.
+ */
+
+#include "tst_test.h"
+
+static void run_test(void)
+{
+	tst_run_script("shell_test_brk.sh", NULL);
+}
+
+static struct tst_test test = {
+	.runs_script = 1,
+	.test_all = run_test,
+};
diff --git a/testcases/lib/tests/shell_test_brk.sh b/testcases/lib/tests/shell_test_brk.sh
new file mode 100755
index 000000000..f266dc3fe
--- /dev/null
+++ b/testcases/lib/tests/shell_test_brk.sh
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+. tst_env.sh
+
+tst_brk TCONF "This exits test and the next message should not be reached"
+tst_res TFAIL "If you see this the test failed"
diff --git a/testcases/lib/tests/shell_test_check_argv.sh b/testcases/lib/tests/shell_test_check_argv.sh
new file mode 100755
index 000000000..ce357027d
--- /dev/null
+++ b/testcases/lib/tests/shell_test_check_argv.sh
@@ -0,0 +1,23 @@
+#!/bin/sh
+
+. tst_env.sh
+
+tst_res TINFO "argv = $@"
+
+if [ $# -ne 2 ]; then
+	tst_res TFAIL "Wrong number of parameters got $# expected 2"
+else
+	tst_res TPASS "Got 2 parameters"
+fi
+
+if [ "$1" != "param1" ]; then
+	tst_res TFAIL "First parameter is $1 expected param1"
+else
+	tst_res TPASS "First parameter is $1"
+fi
+
+if [ "$2" != "param2" ]; then
+	tst_res TFAIL "Second parameter is $2 expected param2"
+else
+	tst_res TPASS "Second parameter is $2"
+fi
diff --git a/testcases/lib/tests/shell_test_checkpoint.sh b/testcases/lib/tests/shell_test_checkpoint.sh
new file mode 100755
index 000000000..0ceb7cf66
--- /dev/null
+++ b/testcases/lib/tests/shell_test_checkpoint.sh
@@ -0,0 +1,7 @@
+#!/bin/sh
+
+. tst_env.sh
+
+tst_res TINFO "Waiting for a checkpoint 0"
+tst_checkpoint wait 10000 0
+tst_res TPASS "Continuing after checkpoint"
diff --git a/testcases/lib/tests/shell_test_pass.sh b/testcases/lib/tests/shell_test_pass.sh
new file mode 100755
index 000000000..fd0684eb2
--- /dev/null
+++ b/testcases/lib/tests/shell_test_pass.sh
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+. tst_env.sh
+
+tst_res TPASS "This is called from the shell script!"
+tst_sleep 100ms
diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
new file mode 100644
index 000000000..948bc5024
--- /dev/null
+++ b/testcases/lib/tst_env.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+tst_script_name=$(basename $0)
+
+if [ -z "$LTP_IPC_PATH" ]; then
+	echo "This script has to be executed from a LTP loader!"
+	exit 1
+fi
+
+tst_brk_()
+{
+	tst_res_ "$@"
+
+	case "$3" in
+		"TBROK") exit 2;;
+		*) exit 0;;
+	esac
+}
+
+alias tst_res="tst_res_ $tst_script_name \$LINENO"
+alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
diff --git a/testcases/lib/tst_res_.c b/testcases/lib/tst_res_.c
new file mode 100644
index 000000000..a43920f36
--- /dev/null
+++ b/testcases/lib/tst_res_.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2024 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+#define TST_NO_DEFAULT_MAIN
+#include "tst_test.h"
+
+static void print_help(void)
+{
+	printf("Usage: tst_res_ filename lineno [TPASS|TFAIL|TCONF|TINFO|TDEBUG] 'A short description'\n");
+}
+
+int main(int argc, char *argv[])
+{
+	int type, i;
+
+	if (argc < 5)
+		goto help;
+
+	if (!strcmp(argv[3], "TPASS"))
+		type = TPASS;
+	else if (!strcmp(argv[3], "TFAIL"))
+		type = TFAIL;
+	else if (!strcmp(argv[3], "TCONF"))
+		type = TCONF;
+	else if (!strcmp(argv[3], "TINFO"))
+		type = TINFO;
+	else if (!strcmp(argv[3], "TDEBUG"))
+		type = TDEBUG;
+	else
+		goto help;
+
+	size_t len = 0;
+
+	for (i = 4; i < argc; i++)
+		len += strlen(argv[i]) + 1;
+
+	char *msg = SAFE_MALLOC(len);
+	char *msgp = msg;
+
+	for (i = 4; i < argc; i++) {
+		msgp = strcpy(msgp, argv[i]);
+		msgp += strlen(argv[i]);
+		*(msgp++) = ' ';
+	}
+
+	*(msgp - 1) = 0;
+
+	tst_reinit();
+
+	tst_res_(argv[1], atoi(argv[2]), type, "%s", msg);
+
+	return 0;
+help:
+	print_help();
+	return 1;
+}
-- 
2.44.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Andrea Cervesato via ltp]

Hi!

I like this minimal approach.

Reviewed-by: Andrea Cervesato <andrea.cervesato@suse.com>

On 8/27/24 14:02, Cyril Hrubis wrote:
> This is a proof of a concept of a seamless C and shell integration. The
> idea is that with this you can mix shell and C code as much as as you
> wish to get the best of the two worlds.
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> Reviewed-by: Richard Palethorpe <io@richiejp.com>
> ---
>   include/tst_test.h                           | 37 +++++++++++++
>   lib/tst_test.c                               | 51 +++++++++++++++++
>   testcases/lib/.gitignore                     |  1 +
>   testcases/lib/Makefile                       |  4 +-
>   testcases/lib/run_tests.sh                   | 11 ++++
>   testcases/lib/tests/.gitignore               |  6 ++
>   testcases/lib/tests/Makefile                 | 11 ++++
>   testcases/lib/tests/shell_test01.c           | 17 ++++++
>   testcases/lib/tests/shell_test02.c           | 18 ++++++
>   testcases/lib/tests/shell_test03.c           | 25 +++++++++
>   testcases/lib/tests/shell_test04.c           | 18 ++++++
>   testcases/lib/tests/shell_test05.c           | 27 +++++++++
>   testcases/lib/tests/shell_test06.c           | 16 ++++++
>   testcases/lib/tests/shell_test_brk.sh        |  6 ++
>   testcases/lib/tests/shell_test_check_argv.sh | 23 ++++++++
>   testcases/lib/tests/shell_test_checkpoint.sh |  7 +++
>   testcases/lib/tests/shell_test_pass.sh       |  6 ++
>   testcases/lib/tst_env.sh                     | 21 +++++++
>   testcases/lib/tst_res_.c                     | 58 ++++++++++++++++++++
>   19 files changed, 361 insertions(+), 2 deletions(-)
>   create mode 100755 testcases/lib/run_tests.sh
>   create mode 100644 testcases/lib/tests/.gitignore
>   create mode 100644 testcases/lib/tests/Makefile
>   create mode 100644 testcases/lib/tests/shell_test01.c
>   create mode 100644 testcases/lib/tests/shell_test02.c
>   create mode 100644 testcases/lib/tests/shell_test03.c
>   create mode 100644 testcases/lib/tests/shell_test04.c
>   create mode 100644 testcases/lib/tests/shell_test05.c
>   create mode 100644 testcases/lib/tests/shell_test06.c
>   create mode 100755 testcases/lib/tests/shell_test_brk.sh
>   create mode 100755 testcases/lib/tests/shell_test_check_argv.sh
>   create mode 100755 testcases/lib/tests/shell_test_checkpoint.sh
>   create mode 100755 testcases/lib/tests/shell_test_pass.sh
>   create mode 100644 testcases/lib/tst_env.sh
>   create mode 100644 testcases/lib/tst_res_.c
>
> diff --git a/include/tst_test.h b/include/tst_test.h
> index afc6a5714..9871676a5 100644
> --- a/include/tst_test.h
> +++ b/include/tst_test.h
> @@ -331,6 +331,8 @@ struct tst_fs {
>    * @child_needs_reinit: Has to be set if the test needs to call tst_reinit()
>    *                      from a process started by exec().
>    *
> + * @runs_script: Implies child_needs_reinit and forks_child at the moment.
> + *
>    * @needs_devfs: If set the devfs is mounted at tst_test.mntpoint. This is
>    *               needed for tests that need to create device files since tmpfs
>    *               at /tmp is usually mounted with 'nodev' option.
> @@ -521,6 +523,7 @@ struct tst_fs {
>   	unsigned int mount_device:1;
>   	unsigned int needs_rofs:1;
>   	unsigned int child_needs_reinit:1;
> +	unsigned int runs_script:1;
>   	unsigned int needs_devfs:1;
>   	unsigned int restore_wallclock:1;
>   
> @@ -529,6 +532,8 @@ struct tst_fs {
>   	unsigned int skip_in_lockdown:1;
>   	unsigned int skip_in_secureboot:1;
>   	unsigned int skip_in_compat:1;
> +
> +
>   	int needs_abi_bits;
>   
>   	unsigned int needs_hugetlbfs:1;
> @@ -616,6 +621,38 @@ void tst_run_tcases(int argc, char *argv[], struct tst_test *self)
>    */
>   void tst_reinit(void);
>   
> +/**
> + * tst_run_script() - Prepare the environment and execute a (shell) script.
> + *
> + * @script_name: A filename of the script.
> + * @params: A NULL terminated array of (shell) script parameters, pass NULL if
> + *          none are needed. This what is passed starting from argv[1].
> + *
> + * The (shell) script is executed with LTP_IPC_PATH in environment so that the
> + * binary helpers such as tst_res_ or tst_checkpoint work properly when executed
> + * from the script. This also means that the tst_test.runs_script flag needs to
> + * be set.
> + *
> + * A shell script has to source the tst_env.sh shell script at the start and
> + * after that it's free to use tst_res in the same way C code would use.
> + *
> + * Example shell script that reports success::
> + *
> + *   #!/bin/sh
> + *   . tst_env.sh
> + *
> + *   tst_res TPASS "Example test works"
> + *
> + * The call returns a pid in a case that you want to examine the return value
> + * of the script yourself. If you do not need to check the return value
> + * yourself you can use tst_reap_children() to wait for the completion. Or let
> + * the test library collect the child automatically, just be wary that the
> + * script and the test both runs concurently at the same time in this case.
> + *
> + * Return: A pid of the (shell) script process.
> + */
> +int tst_run_script(const char *script_name, char *const params[]);
> +
>   unsigned int tst_multiply_timeout(unsigned int timeout);
>   
>   /*
> diff --git a/lib/tst_test.c b/lib/tst_test.c
> index 201b81e14..918bee2a1 100644
> --- a/lib/tst_test.c
> +++ b/lib/tst_test.c
> @@ -4,6 +4,8 @@
>    * Copyright (c) Linux Test Project, 2016-2024
>    */
>   
> +#define _GNU_SOURCE
> +
>   #include <limits.h>
>   #include <stdio.h>
>   #include <stdarg.h>
> @@ -174,6 +176,50 @@ void tst_reinit(void)
>   	SAFE_CLOSE(fd);
>   }
>   
> +extern char **environ;
> +
> +static unsigned int params_array_len(char *const array[])
> +{
> +	unsigned int ret = 0;
> +
> +	if (!array)
> +		return 0;
> +
> +	while (*(array++))
> +		ret++;
> +
> +	return ret;
> +}
> +
> +int tst_run_script(const char *script_name, char *const params[])
> +{
> +	int pid;
> +	unsigned int i, params_len = params_array_len(params);
> +	char *argv[params_len + 2];
> +
> +	if (!tst_test->runs_script)
> +		tst_brk(TBROK, "runs_script flag must be set!");
> +
> +	argv[0] = (char*)script_name;
> +
> +	if (params) {
> +		for (i = 0; i < params_len; i++)
> +			argv[i+1] = params[i];
> +	}
> +
> +	argv[params_len+1] = NULL;
> +
> +	pid = SAFE_FORK();
> +	if (pid)
> +		return pid;
> +
> +	execvpe(script_name, argv, environ);
> +
> +	tst_brk(TBROK | TERRNO, "execvpe(%s, ...) failed!", script_name);
> +
> +	return -1;
> +}
> +
>   static void update_results(int ttype)
>   {
>   	if (!results)
> @@ -1226,6 +1272,11 @@ static void do_setup(int argc, char *argv[])
>   		tdebug = 1;
>   	}
>   
> +	if (tst_test->runs_script) {
> +		tst_test->child_needs_reinit = 1;
> +		tst_test->forks_child = 1;
> +	}
> +
>   	if (tst_test->needs_kconfigs && tst_kconfig_check(tst_test->needs_kconfigs))
>   		tst_brk(TCONF, "Aborting due to unsuitable kernel config, see above!");
>   
> diff --git a/testcases/lib/.gitignore b/testcases/lib/.gitignore
> index e8afd06f3..d0dacf62a 100644
> --- a/testcases/lib/.gitignore
> +++ b/testcases/lib/.gitignore
> @@ -23,3 +23,4 @@
>   /tst_sleep
>   /tst_supported_fs
>   /tst_timeout_kill
> +/tst_res_
> diff --git a/testcases/lib/Makefile b/testcases/lib/Makefile
> index 990b46089..928d76d62 100644
> --- a/testcases/lib/Makefile
> +++ b/testcases/lib/Makefile
> @@ -13,6 +13,6 @@ MAKE_TARGETS		:= tst_sleep tst_random tst_checkpoint tst_rod tst_kvcmp\
>   			   tst_getconf tst_supported_fs tst_check_drivers tst_get_unused_port\
>   			   tst_get_median tst_hexdump tst_get_free_pids tst_timeout_kill\
>   			   tst_check_kconfigs tst_cgctl tst_fsfreeze tst_ns_create tst_ns_exec\
> -			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled
> +			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_
>   
> -include $(top_srcdir)/include/mk/generic_leaf_target.mk
> +include $(top_srcdir)/include/mk/generic_trunk_target.mk
> diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh
> new file mode 100755
> index 000000000..60e7d1bcf
> --- /dev/null
> +++ b/testcases/lib/run_tests.sh
> @@ -0,0 +1,11 @@
> +#!/bin/sh
> +
> +testdir=$(realpath $(dirname $0))
> +export PATH=$PATH:$testdir:$testdir/tests/
> +
> +for i in `seq -w 01 06`; do
> +	echo
> +	echo "*** Running shell_test$i ***"
> +	echo
> +	./tests/shell_test$i
> +done
> diff --git a/testcases/lib/tests/.gitignore b/testcases/lib/tests/.gitignore
> new file mode 100644
> index 000000000..da967c4d6
> --- /dev/null
> +++ b/testcases/lib/tests/.gitignore
> @@ -0,0 +1,6 @@
> +shell_test01
> +shell_test02
> +shell_test03
> +shell_test04
> +shell_test05
> +shell_test06
> diff --git a/testcases/lib/tests/Makefile b/testcases/lib/tests/Makefile
> new file mode 100644
> index 000000000..5a5cf5310
> --- /dev/null
> +++ b/testcases/lib/tests/Makefile
> @@ -0,0 +1,11 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (C) 2009, Cisco Systems Inc.
> +# Ngie Cooper, August 2009
> +
> +top_srcdir		?= ../../..
> +
> +include $(top_srcdir)/include/mk/testcases.mk
> +
> +INSTALL_TARGETS=
> +
> +include $(top_srcdir)/include/mk/generic_leaf_target.mk
> diff --git a/testcases/lib/tests/shell_test01.c b/testcases/lib/tests/shell_test01.c
> new file mode 100644
> index 000000000..b9f07308e
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test01.c
> @@ -0,0 +1,17 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Shell test example.
> + */
> +
> +#include "tst_test.h"
> +
> +static void run_test(void)
> +{
> +	tst_run_script("shell_test_pass.sh", NULL);
> +	tst_res(TINFO, "C test exits now");
> +}
> +
> +static struct tst_test test = {
> +	.runs_script = 1,
> +	.test_all = run_test,
> +};
> diff --git a/testcases/lib/tests/shell_test02.c b/testcases/lib/tests/shell_test02.c
> new file mode 100644
> index 000000000..087055794
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test02.c
> @@ -0,0 +1,18 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Shell test example.
> + */
> +
> +#include "tst_test.h"
> +
> +static void run_test(void)
> +{
> +	tst_run_script("shell_test_pass.sh", NULL);
> +	tst_reap_children();
> +	tst_res(TINFO, "Shell test has finished at this point!");
> +}
> +
> +static struct tst_test test = {
> +	.runs_script = 1,
> +	.test_all = run_test,
> +};
> diff --git a/testcases/lib/tests/shell_test03.c b/testcases/lib/tests/shell_test03.c
> new file mode 100644
> index 000000000..61436891e
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test03.c
> @@ -0,0 +1,25 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Shell test example.
> + */
> +
> +#include <sys/wait.h>
> +#include "tst_test.h"
> +
> +static void run_test(void)
> +{
> +	int pid, status;
> +
> +	pid = tst_run_script("shell_test_pass.sh", NULL);
> +
> +	tst_res(TINFO, "Waiting for the pid %i", pid);
> +
> +	waitpid(pid, &status, 0);
> +
> +	tst_res(TINFO, "Shell test has %s", tst_strstatus(status));
> +}
> +
> +static struct tst_test test = {
> +	.runs_script = 1,
> +	.test_all = run_test,
> +};
> diff --git a/testcases/lib/tests/shell_test04.c b/testcases/lib/tests/shell_test04.c
> new file mode 100644
> index 000000000..a32dd1e9f
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test04.c
> @@ -0,0 +1,18 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Shell test example.
> + */
> +
> +#include "tst_test.h"
> +
> +static void run_test(void)
> +{
> +	char *const params[] = {"param1", "param2", NULL};
> +
> +	tst_run_script("shell_test_check_argv.sh", params);
> +}
> +
> +static struct tst_test test = {
> +	.runs_script = 1,
> +	.test_all = run_test,
> +};
> diff --git a/testcases/lib/tests/shell_test05.c b/testcases/lib/tests/shell_test05.c
> new file mode 100644
> index 000000000..771af8fc3
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test05.c
> @@ -0,0 +1,27 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Shell test example.
> + */
> +
> +#include "tst_test.h"
> +
> +static void run_test(void)
> +{
> +	int pid;
> +
> +	pid = tst_run_script("shell_test_checkpoint.sh", NULL);
> +
> +	tst_res(TINFO, "Waiting for shell to sleep on checkpoint!");
> +
> +	TST_PROCESS_STATE_WAIT(pid, 'S', 10000);
> +
> +	tst_res(TINFO, "Waking shell child!");
> +
> +	TST_CHECKPOINT_WAKE(0);
> +}
> +
> +static struct tst_test test = {
> +	.runs_script = 1,
> +	.needs_checkpoints = 1,
> +	.test_all = run_test,
> +};
> diff --git a/testcases/lib/tests/shell_test06.c b/testcases/lib/tests/shell_test06.c
> new file mode 100644
> index 000000000..89d66bab0
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test06.c
> @@ -0,0 +1,16 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Shell test example.
> + */
> +
> +#include "tst_test.h"
> +
> +static void run_test(void)
> +{
> +	tst_run_script("shell_test_brk.sh", NULL);
> +}
> +
> +static struct tst_test test = {
> +	.runs_script = 1,
> +	.test_all = run_test,
> +};
> diff --git a/testcases/lib/tests/shell_test_brk.sh b/testcases/lib/tests/shell_test_brk.sh
> new file mode 100755
> index 000000000..f266dc3fe
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test_brk.sh
> @@ -0,0 +1,6 @@
> +#!/bin/sh
> +
> +. tst_env.sh
> +
> +tst_brk TCONF "This exits test and the next message should not be reached"
> +tst_res TFAIL "If you see this the test failed"
> diff --git a/testcases/lib/tests/shell_test_check_argv.sh b/testcases/lib/tests/shell_test_check_argv.sh
> new file mode 100755
> index 000000000..ce357027d
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test_check_argv.sh
> @@ -0,0 +1,23 @@
> +#!/bin/sh
> +
> +. tst_env.sh
> +
> +tst_res TINFO "argv = $@"
> +
> +if [ $# -ne 2 ]; then
> +	tst_res TFAIL "Wrong number of parameters got $# expected 2"
> +else
> +	tst_res TPASS "Got 2 parameters"
> +fi
> +
> +if [ "$1" != "param1" ]; then
> +	tst_res TFAIL "First parameter is $1 expected param1"
> +else
> +	tst_res TPASS "First parameter is $1"
> +fi
> +
> +if [ "$2" != "param2" ]; then
> +	tst_res TFAIL "Second parameter is $2 expected param2"
> +else
> +	tst_res TPASS "Second parameter is $2"
> +fi
> diff --git a/testcases/lib/tests/shell_test_checkpoint.sh b/testcases/lib/tests/shell_test_checkpoint.sh
> new file mode 100755
> index 000000000..0ceb7cf66
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test_checkpoint.sh
> @@ -0,0 +1,7 @@
> +#!/bin/sh
> +
> +. tst_env.sh
> +
> +tst_res TINFO "Waiting for a checkpoint 0"
> +tst_checkpoint wait 10000 0
> +tst_res TPASS "Continuing after checkpoint"
> diff --git a/testcases/lib/tests/shell_test_pass.sh b/testcases/lib/tests/shell_test_pass.sh
> new file mode 100755
> index 000000000..fd0684eb2
> --- /dev/null
> +++ b/testcases/lib/tests/shell_test_pass.sh
> @@ -0,0 +1,6 @@
> +#!/bin/sh
> +
> +. tst_env.sh
> +
> +tst_res TPASS "This is called from the shell script!"
> +tst_sleep 100ms
> diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
> new file mode 100644
> index 000000000..948bc5024
> --- /dev/null
> +++ b/testcases/lib/tst_env.sh
> @@ -0,0 +1,21 @@
> +#!/bin/sh
> +
> +tst_script_name=$(basename $0)
> +
> +if [ -z "$LTP_IPC_PATH" ]; then
> +	echo "This script has to be executed from a LTP loader!"
> +	exit 1
> +fi
> +
> +tst_brk_()
> +{
> +	tst_res_ "$@"
> +
> +	case "$3" in
> +		"TBROK") exit 2;;
> +		*) exit 0;;
> +	esac
> +}
> +
> +alias tst_res="tst_res_ $tst_script_name \$LINENO"
> +alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
> diff --git a/testcases/lib/tst_res_.c b/testcases/lib/tst_res_.c
> new file mode 100644
> index 000000000..a43920f36
> --- /dev/null
> +++ b/testcases/lib/tst_res_.c
> @@ -0,0 +1,58 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2024 Cyril Hrubis <chrubis@suse.cz>
> + */
> +
> +#define TST_NO_DEFAULT_MAIN
> +#include "tst_test.h"
> +
> +static void print_help(void)
> +{
> +	printf("Usage: tst_res_ filename lineno [TPASS|TFAIL|TCONF|TINFO|TDEBUG] 'A short description'\n");
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	int type, i;
> +
> +	if (argc < 5)
> +		goto help;
> +
> +	if (!strcmp(argv[3], "TPASS"))
> +		type = TPASS;
> +	else if (!strcmp(argv[3], "TFAIL"))
> +		type = TFAIL;
> +	else if (!strcmp(argv[3], "TCONF"))
> +		type = TCONF;
> +	else if (!strcmp(argv[3], "TINFO"))
> +		type = TINFO;
> +	else if (!strcmp(argv[3], "TDEBUG"))
> +		type = TDEBUG;
> +	else
> +		goto help;
> +
> +	size_t len = 0;
> +
> +	for (i = 4; i < argc; i++)
> +		len += strlen(argv[i]) + 1;
> +
> +	char *msg = SAFE_MALLOC(len);
> +	char *msgp = msg;
> +
> +	for (i = 4; i < argc; i++) {
> +		msgp = strcpy(msgp, argv[i]);
> +		msgp += strlen(argv[i]);
> +		*(msgp++) = ' ';
> +	}
> +
> +	*(msgp - 1) = 0;
> +
> +	tst_reinit();
> +
> +	tst_res_(argv[1], atoi(argv[2]), type, "%s", msg);
> +
> +	return 0;
> +help:
> +	print_help();
> +	return 1;
> +}

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Li Wang]

On Tue, Aug 27, 2024 at 8:04 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
> This is a proof of a concept of a seamless C and shell integration. The
> idea is that with this you can mix shell and C code as much as as you
> wish to get the best of the two worlds.
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> Reviewed-by: Richard Palethorpe <io@richiejp.com>

Reviewed-by: Li Wang <liwang@redhat.com>



--
Regards,
Li Wang


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Petr Vorel]

Hi Cyril,

...
> diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh
I hoped testcases/lib/run_tests.sh could be run in CI (and manually).

We have in the toplevel Makefile other test targets, could we add
testcases/lib/run_tests.sh to test-shell target?

1) Add test target to testcases/lib/tests/Makefile (similar to
metadata/tests/Makefile).

test:
	@echo == Run C - shell integration tests ===
	@./run_tests.sh

2) Add test target to testcases/lib/Makefile which would call test target in the
test/ directory (similar to metadata/Makefile).

test:
	$(MAKE) -C $(abs_srcdir)/tests/ test

3) Add testing to the toplevel test-shell target + build dependency to testcases/lib/.

test-shell: lib-all
ifneq ($(build),$(host))
	$(error running tests on cross-compile build not supported)
endif
	$(call _test,-s)
	$(MAKE) -C $(abs_top_builddir)/testcases/lib
	$(MAKE) -C $(abs_top_srcdir)/testcases/lib test

> --- /dev/null
> +++ b/testcases/lib/run_tests.sh
> @@ -0,0 +1,11 @@
> +#!/bin/sh
> +
> +testdir=$(realpath $(dirname $0))
Not sure if there should be some quotes.

> +export PATH=$PATH:$testdir:$testdir/tests/
Also quotes here.

...
> diff --git a/testcases/lib/tests/Makefile b/testcases/lib/tests/Makefile
> new file mode 100644
> index 000000000..5a5cf5310
> --- /dev/null
> +++ b/testcases/lib/tests/Makefile
> @@ -0,0 +1,11 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (C) 2009, Cisco Systems Inc.
nit: Maybe LTP copyright? (copy paste error)
> +# Ngie Cooper, August 2009
nit: and you, or just replace Ngie?
> +
> +top_srcdir		?= ../../..
> +
> +include $(top_srcdir)/include/mk/testcases.mk
> +
> +INSTALL_TARGETS=
> +
> +include $(top_srcdir)/include/mk/generic_leaf_target.mk

The rest LGTM, but I'll have a deeper look today or tomorrow and try to play
with it.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Cyril Hrubis]

Hi!
> I hoped testcases/lib/run_tests.sh could be run in CI (and manually).
> 
> We have in the toplevel Makefile other test targets, could we add
> testcases/lib/run_tests.sh to test-shell target?
> 
> 1) Add test target to testcases/lib/tests/Makefile (similar to
> metadata/tests/Makefile).
> 
> test:
> 	@echo == Run C - shell integration tests ===
> 	@./run_tests.sh
> 
> 2) Add test target to testcases/lib/Makefile which would call test target in the
> test/ directory (similar to metadata/Makefile).
> 
> test:
> 	$(MAKE) -C $(abs_srcdir)/tests/ test
> 
> 3) Add testing to the toplevel test-shell target + build dependency to testcases/lib/.
> 
> test-shell: lib-all
> ifneq ($(build),$(host))
> 	$(error running tests on cross-compile build not supported)
> endif
> 	$(call _test,-s)
> 	$(MAKE) -C $(abs_top_builddir)/testcases/lib
> 	$(MAKE) -C $(abs_top_srcdir)/testcases/lib test

Quite a few of these testcases fail in order to test failures, we need
to be able to record the test output and compare that with the expected
outcome first, in order to make these tests useful. And I guess that we
need that for the C library tests as well. So I would rather do this
later for both libraries instead.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Li Wang]

Cyril Hrubis <chrubis@suse.cz> wrote:


> diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
> new file mode 100644
> index 000000000..948bc5024
> --- /dev/null
> +++ b/testcases/lib/tst_env.sh
> @@ -0,0 +1,21 @@
> +#!/bin/sh
> +
> +tst_script_name=$(basename $0)
> +
> +if [ -z "$LTP_IPC_PATH" ]; then
> +       echo "This script has to be executed from a LTP loader!"
> +       exit 1
> +fi
> +
> +tst_brk_()
> +{
> +       tst_res_ "$@"
> +
> +       case "$3" in
> +               "TBROK") exit 2;;
> +               *) exit 0;;
> +       esac
> +}
> +
> +alias tst_res="tst_res_ $tst_script_name \$LINENO"
> +alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
>


*** Running shell_test01 ***

tst_test.c:1860: TINFO: LTP version: 20240524
tst_test.c:1864: TINFO: Tested kernel: 6.8.0-1010-raspi #11-Ubuntu SMP
PREEMPT_DYNAMIC Thu Aug  8 23:22:41 UTC 2024 aarch64
tst_test.c:1703: TINFO: Timeout per run is 0h 00m 30s
shell_test01.c:11: TINFO: C test exits now
Usage: tst_res_ filename lineno [TPASS|TFAIL|TCONF|TINFO|TDEBUG] 'A short
description'
tst_test.c:1535: TBROK: Test haven't reported results!

Summary:
passed   0
failed   0
broken   1
skipped  0
warnings 0


Here I got test failures on my RPi4 (bash-5.2.21) that the $LINEON can't
be parsed correctly.

The reason probably is some shells or specific versions might not handle
$LINENO correctly within aliases, especially when the line number needs
to be dynamically determined.

So I suggest using a function instead of the alias.


--- a/testcases/lib/tst_env.sh
+++ b/testcases/lib/tst_env.sh
@@ -21,5 +21,12 @@ tst_brk_()
        esac
 }

-alias tst_res="tst_res_ $tst_script_name \$LINENO"
-alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
+tst_res()
+{
+    tst_res_ "$tst_script_name" "$LINENO" "$@"
+}
+
+tst_brk()
+{
+    tst_brk_ "$tst_script_name" "$LINENO" "$@"
+}


-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Cyril Hrubis]

Hi!
> tst_test.c:1860: TINFO: LTP version: 20240524
> tst_test.c:1864: TINFO: Tested kernel: 6.8.0-1010-raspi #11-Ubuntu SMP
> PREEMPT_DYNAMIC Thu Aug  8 23:22:41 UTC 2024 aarch64
> tst_test.c:1703: TINFO: Timeout per run is 0h 00m 30s
> shell_test01.c:11: TINFO: C test exits now
> Usage: tst_res_ filename lineno [TPASS|TFAIL|TCONF|TINFO|TDEBUG] 'A short
> description'
> tst_test.c:1535: TBROK: Test haven't reported results!
> 
> Summary:
> passed   0
> failed   0
> broken   1
> skipped  0
> warnings 0
> 
> 
> Here I got test failures on my RPi4 (bash-5.2.21) that the $LINEON can't
> be parsed correctly.

Are you sure that this is due to bash? My guess is that on RPi the
default shell is dash because it's debian based.

I will try to reproduce.

> The reason probably is some shells or specific versions might not handle
> $LINENO correctly within aliases, especially when the line number needs
> to be dynamically determined.
> 
> So I suggest using a function instead of the alias.
> 
> 
> --- a/testcases/lib/tst_env.sh
> +++ b/testcases/lib/tst_env.sh
> @@ -21,5 +21,12 @@ tst_brk_()
>         esac
>  }
> 
> -alias tst_res="tst_res_ $tst_script_name \$LINENO"
> -alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
> +tst_res()
> +{
> +    tst_res_ "$tst_script_name" "$LINENO" "$@"
> +}
> +
> +tst_brk()
> +{
> +    tst_brk_ "$tst_script_name" "$LINENO" "$@"
> +}

That actually does not work because unlike the alias the $LINENO is
expanded in the wrong place and the line is incorrect.

The whole reason for this to be alias is that it's expanded on the
correct line in the test source.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Cyril Hrubis]

Hi!
> Are you sure that this is due to bash? My guess is that on RPi the
> default shell is dash because it's debian based.
> 
> I will try to reproduce.

And the whole reason seems to be that dash does not support $LINENO
which is actually required to be there by POSIX.

I quite close to declare dash unsupported because it does not even
support POSIX standard.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Li Wang]

On Fri, Sep 6, 2024 at 5:54 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > tst_test.c:1860: TINFO: LTP version: 20240524
> > tst_test.c:1864: TINFO: Tested kernel: 6.8.0-1010-raspi #11-Ubuntu SMP
> > PREEMPT_DYNAMIC Thu Aug  8 23:22:41 UTC 2024 aarch64
> > tst_test.c:1703: TINFO: Timeout per run is 0h 00m 30s
> > shell_test01.c:11: TINFO: C test exits now
> > Usage: tst_res_ filename lineno [TPASS|TFAIL|TCONF|TINFO|TDEBUG] 'A short
> > description'
> > tst_test.c:1535: TBROK: Test haven't reported results!
> >
> > Summary:
> > passed   0
> > failed   0
> > broken   1
> > skipped  0
> > warnings 0
> >
> >
> > Here I got test failures on my RPi4 (bash-5.2.21) that the $LINEON can't
> > be parsed correctly.
>
> Are you sure that this is due to bash? My guess is that on RPi the
> default shell is dash because it's debian based.
>

Double checked that is indeed bash.

liwang@raspi4:~/ltp/testcases/lib$ echo $0
-bash

liwang@raspi4:~/ltp/testcases/lib$ echo $SHELL
/bin/bash

liwang@raspi4:~/ltp/testcases/lib$ ps -p $$
    PID TTY          TIME CMD
 174997 pts/0    00:00:04 bash

I even manually coded the !#/bin/bash in every test file, the test still
failed.

$ bash run_tests.sh
tst_test.c:1860: TINFO: LTP version: 20240524
tst_test.c:1864: TINFO: Tested kernel: 6.8.0-1010-raspi #11-Ubuntu SMP
PREEMPT_DYNAMIC Thu Aug  8 23:22:41 UTC 2024 aarch64
tst_test.c:1703: TINFO: Timeout per run is 0h 00m 30s
shell_test01.c:11: TINFO: C test exits now
/home/liwang/ltp/testcases/lib/tests//shell_test_pass.sh: line 5: tst_res:
command not found
/bin/bash
tst_test.c:1535: TBROK: Test haven't reported results!

Summary:
passed   0
failed   0
broken   1
skipped  0
warnings 0



>
> I will try to reproduce.
>
> > The reason probably is some shells or specific versions might not handle
> > $LINENO correctly within aliases, especially when the line number needs
> > to be dynamically determined.
> >
> > So I suggest using a function instead of the alias.
> >
> >
> > --- a/testcases/lib/tst_env.sh
> > +++ b/testcases/lib/tst_env.sh
> > @@ -21,5 +21,12 @@ tst_brk_()
> >         esac
> >  }
> >
> > -alias tst_res="tst_res_ $tst_script_name \$LINENO"
> > -alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
> > +tst_res()
> > +{
> > +    tst_res_ "$tst_script_name" "$LINENO" "$@"
> > +}
> > +
> > +tst_brk()
> > +{
> > +    tst_brk_ "$tst_script_name" "$LINENO" "$@"
> > +}
>
> That actually does not work because unlike the alias the $LINENO is
> expanded in the wrong place and the line is incorrect.
>
> The whole reason for this to be alias is that it's expanded on the
> correct line in the test source.
>
> --
> Cyril Hrubis
> chrubis@suse.cz
>
>

-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Cyril Hrubis]

Hi!
> > > Here I got test failures on my RPi4 (bash-5.2.21) that the $LINEON can't
> > > be parsed correctly.
> >
> > Are you sure that this is due to bash? My guess is that on RPi the
> > default shell is dash because it's debian based.
> >
> 
> Double checked that is indeed bash.
> 
> liwang@raspi4:~/ltp/testcases/lib$ echo $0
> -bash
> 
> liwang@raspi4:~/ltp/testcases/lib$ echo $SHELL
> /bin/bash
> 
> liwang@raspi4:~/ltp/testcases/lib$ ps -p $$
>     PID TTY          TIME CMD
>  174997 pts/0    00:00:04 bash
> 
> I even manually coded the !#/bin/bash in every test file, the test still
> failed.

The difference is that /bin/sh links to dash in debian. But as you
pointed out even if all /bin/sh is changed to /bin/bash in the scripts
the tests stil fails.

It looks like on RPi alias with variables does not work at all.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Cyril Hrubis]

Hi!
> The difference is that /bin/sh links to dash in debian. But as you
> pointed out even if all /bin/sh is changed to /bin/bash in the scripts
> the tests stil fails.
> 
> It looks like on RPi alias with variables does not work at all.

So it looks like aliases may not be expanded on non-interactive shells
unless it's enabled explicitely.

This patch uses shopt to enable that and also works around dash that
does not support $LINENO:

diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
index 67ba80744..bb0c586d7 100644
--- a/testcases/lib/tst_env.sh
+++ b/testcases/lib/tst_env.sh
@@ -6,6 +6,16 @@
 
 tst_script_name=$(basename $0)
 
+# bash does not expand aliases in non-iteractive mode, enable it
+if [ -n "$BASH_VERSION" ]; then
+       shopt -s expand_aliases
+fi
+
+# dash does not support line numbers even though this is mandated by POSIX
+if [ -z "$LINENO" ]; then
+       LINENO=-1
+fi
+
 if [ -z "$LTP_IPC_PATH" ]; then
        echo "This script has to be executed from a LTP loader!"
        exit 1


-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Li Wang]

On Fri, Sep 6, 2024 at 9:30 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > The difference is that /bin/sh links to dash in debian. But as you
> > pointed out even if all /bin/sh is changed to /bin/bash in the scripts
> > the tests stil fails.
> >
> > It looks like on RPi alias with variables does not work at all.
>
> So it looks like aliases may not be expanded on non-interactive shells
> unless it's enabled explicitely.
>
> This patch uses shopt to enable that and also works around dash that
> does not support $LINENO:
>
> diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
> index 67ba80744..bb0c586d7 100644
> --- a/testcases/lib/tst_env.sh
> +++ b/testcases/lib/tst_env.sh
> @@ -6,6 +6,16 @@
>
>  tst_script_name=$(basename $0)
>
> +# bash does not expand aliases in non-iteractive mode, enable it
> +if [ -n "$BASH_VERSION" ]; then
> +       shopt -s expand_aliases
> +fi
> +
> +# dash does not support line numbers even though this is mandated by POSIX
> +if [ -z "$LINENO" ]; then
> +       LINENO=-1
> +fi
> +
>  if [ -z "$LTP_IPC_PATH" ]; then
>         echo "This script has to be executed from a LTP loader!"
>         exit 1


LGTM.

This ensures that the script will work in both bash (with alias expansion
enabled)
and dash (with a fallback for LINENO). I confirmed that works correctly in
my RPi4.
Thanks!


-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Li Wang]

On Fri, Sep 6, 2024 at 5:54 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > tst_test.c:1860: TINFO: LTP version: 20240524
> > tst_test.c:1864: TINFO: Tested kernel: 6.8.0-1010-raspi #11-Ubuntu SMP
> > PREEMPT_DYNAMIC Thu Aug  8 23:22:41 UTC 2024 aarch64
> > tst_test.c:1703: TINFO: Timeout per run is 0h 00m 30s
> > shell_test01.c:11: TINFO: C test exits now
> > Usage: tst_res_ filename lineno [TPASS|TFAIL|TCONF|TINFO|TDEBUG] 'A short
> > description'
> > tst_test.c:1535: TBROK: Test haven't reported results!
> >
> > Summary:
> > passed   0
> > failed   0
> > broken   1
> > skipped  0
> > warnings 0
> >
> >
> > Here I got test failures on my RPi4 (bash-5.2.21) that the $LINEON can't
> > be parsed correctly.
>
> Are you sure that this is due to bash? My guess is that on RPi the
> default shell is dash because it's debian based.
>
> I will try to reproduce.
>
> > The reason probably is some shells or specific versions might not handle
> > $LINENO correctly within aliases, especially when the line number needs
> > to be dynamically determined.
> >
> > So I suggest using a function instead of the alias.
> >
> >
> > --- a/testcases/lib/tst_env.sh
> > +++ b/testcases/lib/tst_env.sh
> > @@ -21,5 +21,12 @@ tst_brk_()
> >         esac
> >  }
> >
> > -alias tst_res="tst_res_ $tst_script_name \$LINENO"
> > -alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
> > +tst_res()
> > +{
> > +    tst_res_ "$tst_script_name" "$LINENO" "$@"
> > +}
> > +
> > +tst_brk()
> > +{
> > +    tst_brk_ "$tst_script_name" "$LINENO" "$@"
> > +}
>
> That actually does not work because unlike the alias the $LINENO is
> expanded in the wrong place and the line is incorrect.
>
> The whole reason for this to be alias is that it's expanded on the
> correct line in the test source.
>

Ah, you're right, I wasn't aware of that.

Maybe we can use a trick with eval to delay the expansion
of $LINENO until the function is actually called?

e.g.

tst_res()
{
    eval "tst_res_ \"$tst_script_name\" \$LINENO \"\$@\""
}

tst_brk()
{
    eval "tst_brk_ \"$tst_script_name\" \$LINENO \"\$@\""
}


-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 2/4] Add support for mixing C and shell code

[Li Wang]

On Fri, Sep 6, 2024 at 6:19 PM Li Wang <liwang@redhat.com> wrote:

>
>
> On Fri, Sep 6, 2024 at 5:54 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
>> Hi!
>> > tst_test.c:1860: TINFO: LTP version: 20240524
>> > tst_test.c:1864: TINFO: Tested kernel: 6.8.0-1010-raspi #11-Ubuntu SMP
>> > PREEMPT_DYNAMIC Thu Aug  8 23:22:41 UTC 2024 aarch64
>> > tst_test.c:1703: TINFO: Timeout per run is 0h 00m 30s
>> > shell_test01.c:11: TINFO: C test exits now
>> > Usage: tst_res_ filename lineno [TPASS|TFAIL|TCONF|TINFO|TDEBUG] 'A
>> short
>> > description'
>> > tst_test.c:1535: TBROK: Test haven't reported results!
>> >
>> > Summary:
>> > passed   0
>> > failed   0
>> > broken   1
>> > skipped  0
>> > warnings 0
>> >
>> >
>> > Here I got test failures on my RPi4 (bash-5.2.21) that the $LINEON can't
>> > be parsed correctly.
>>
>> Are you sure that this is due to bash? My guess is that on RPi the
>> default shell is dash because it's debian based.
>>
>> I will try to reproduce.
>>
>> > The reason probably is some shells or specific versions might not handle
>> > $LINENO correctly within aliases, especially when the line number needs
>> > to be dynamically determined.
>> >
>> > So I suggest using a function instead of the alias.
>> >
>> >
>> > --- a/testcases/lib/tst_env.sh
>> > +++ b/testcases/lib/tst_env.sh
>> > @@ -21,5 +21,12 @@ tst_brk_()
>> >         esac
>> >  }
>> >
>> > -alias tst_res="tst_res_ $tst_script_name \$LINENO"
>> > -alias tst_brk="tst_brk_ $tst_script_name \$LINENO"
>> > +tst_res()
>> > +{
>> > +    tst_res_ "$tst_script_name" "$LINENO" "$@"
>> > +}
>> > +
>> > +tst_brk()
>> > +{
>> > +    tst_brk_ "$tst_script_name" "$LINENO" "$@"
>> > +}
>>
>> That actually does not work because unlike the alias the $LINENO is
>> expanded in the wrong place and the line is incorrect.
>>
>> The whole reason for this to be alias is that it's expanded on the
>> correct line in the test source.
>>
>
> Ah, you're right, I wasn't aware of that.
>
> Maybe we can use a trick with eval to delay the expansion
> of $LINENO until the function is actually called?
>

Hmm, seems this is also not working as expected. Please ignore.

$ cat liwang.sh
#!/bin/bash

print_line() {
    eval "echo \"this is line: \$LINENO\""
}

echo "This is line $LINENO"
print_line

liwang@raspi4:~/ltp/testcases/lib$ ./liwang.sh
This is line 7
this is line: 4

>

-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 3/4] libs: Vendor ujson library

[Cyril Hrubis]

See: https://github.com/metan-ucw/ujson

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
Acked-by: Richard Palethorpe <io@richiejp.com>
---
 include/ujson.h           |   13 +
 include/ujson_common.h    |   69 +++
 include/ujson_reader.h    |  543 +++++++++++++++++++
 include/ujson_utf.h       |  168 ++++++
 include/ujson_writer.h    |  224 ++++++++
 libs/ujson/Makefile       |   12 +
 libs/ujson/ujson_common.c |   38 ++
 libs/ujson/ujson_reader.c | 1081 +++++++++++++++++++++++++++++++++++++
 libs/ujson/ujson_utf.c    |  105 ++++
 libs/ujson/ujson_writer.c |  491 +++++++++++++++++
 10 files changed, 2744 insertions(+)
 create mode 100644 include/ujson.h
 create mode 100644 include/ujson_common.h
 create mode 100644 include/ujson_reader.h
 create mode 100644 include/ujson_utf.h
 create mode 100644 include/ujson_writer.h
 create mode 100644 libs/ujson/Makefile
 create mode 100644 libs/ujson/ujson_common.c
 create mode 100644 libs/ujson/ujson_reader.c
 create mode 100644 libs/ujson/ujson_utf.c
 create mode 100644 libs/ujson/ujson_writer.c

diff --git a/include/ujson.h b/include/ujson.h
new file mode 100644
index 000000000..8faeb18f0
--- /dev/null
+++ b/include/ujson.h
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+#ifndef UJSON_H
+#define UJSON_H
+
+#include <ujson_common.h>
+#include <ujson_reader.h>
+#include <ujson_writer.h>
+
+#endif /* UJSON_H */
diff --git a/include/ujson_common.h b/include/ujson_common.h
new file mode 100644
index 000000000..ed31c090d
--- /dev/null
+++ b/include/ujson_common.h
@@ -0,0 +1,69 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+/**
+ * @file ujson_common.h
+ * @brief Common JSON reader/writer definitions.
+ */
+
+#ifndef UJSON_COMMON_H
+#define UJSON_COMMON_H
+
+/** @brief Maximal error message length. */
+#define UJSON_ERR_MAX 128
+/** @brief Maximal id string lenght including terminating null element. */
+#define UJSON_ID_MAX 64
+/** @brief Maximal recursion depth allowed. */
+#define UJSON_RECURSION_MAX 128
+
+#define UJSON_ERR_PRINT ujson_err_handler
+#define UJSON_ERR_PRINT_PRIV stderr
+
+/**
+ * @brief A JSON data type.
+ */
+enum ujson_type {
+	/** @brief No type. Returned when parser finishes. */
+	UJSON_VOID = 0,
+	/** @brief An integer. */
+	UJSON_INT,
+	/** @brief A floating point. */
+	UJSON_FLOAT,
+	/** @brief A boolean. */
+	UJSON_BOOL,
+	/** @brief NULL */
+	UJSON_NULL,
+	/** @brief A string. */
+	UJSON_STR,
+	/** @brief A JSON object. */
+	UJSON_OBJ,
+	/** @brief A JSON array. */
+	UJSON_ARR,
+};
+
+/**
+ * @brief Returns type name.
+ *
+ * @param type A json type.
+ * @return A type name.
+ */
+const char *ujson_type_name(enum ujson_type type);
+
+/**
+ * @brief Default error print handler.
+ *
+ * @param print_priv A json buffer print_priv pointer.
+ * @param line A line of text to be printed.
+ */
+void ujson_err_handler(void *print_priv, const char *line);
+
+typedef struct ujson_reader ujson_reader;
+typedef struct ujson_writer ujson_writer;
+typedef struct ujson_val ujson_val;
+
+/** @brief An array size macro. */
+#define UJSON_ARRAY_SIZE(array) (sizeof(array) / sizeof(*array))
+
+#endif /* UJSON_COMMON_H */
diff --git a/include/ujson_reader.h b/include/ujson_reader.h
new file mode 100644
index 000000000..273fe624a
--- /dev/null
+++ b/include/ujson_reader.h
@@ -0,0 +1,543 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+/**
+ * @file ujson_reader.h
+ * @brief A recursive descend JSON parser.
+ *
+ * All the function that parse JSON return zero on success and non-zero on a
+ * failure. Once an error has happened all subsequent attempts to parse more
+ * return with non-zero exit status immediatelly. This is designed so that we
+ * can parse several values without checking each return value and only check
+ * if error has happened at the end of the sequence.
+ */
+
+#ifndef UJSON_READER_H
+#define UJSON_READER_H
+
+#include <stdio.h>
+#include <ujson_common.h>
+
+/**
+ * @brief An ujson_reader initializer with default values.
+ *
+ * @param buf A pointer to a buffer with JSON data.
+ * @param buf_len A JSON data buffer lenght.
+ * @param rflags enum ujson_reader_flags.
+ *
+ * @return An ujson_reader initialized with default values.
+ */
+#define UJSON_READER_INIT(buf, buf_len, rflags) { \
+	.max_depth = UJSON_RECURSION_MAX, \
+	.err_print = UJSON_ERR_PRINT, \
+	.err_print_priv = UJSON_ERR_PRINT_PRIV, \
+	.json = buf, \
+	.len = buf_len, \
+	.flags = rflags \
+}
+
+/** @brief Reader flags. */
+enum ujson_reader_flags {
+	/** @brief If set warnings are treated as errors. */
+	UJSON_READER_STRICT = 0x01,
+};
+
+/**
+ * @brief A JSON parser internal state.
+ */
+struct ujson_reader {
+	/** Pointer to a null terminated JSON string */
+	const char *json;
+	/** A length of the JSON string */
+	size_t len;
+	/** A current offset into the JSON string */
+	size_t off;
+	/** An offset to the start of the last array or object */
+	size_t sub_off;
+	/** Recursion depth increased when array/object is entered decreased on leave */
+	unsigned int depth;
+	/** Maximal recursion depth */
+	unsigned int max_depth;
+
+	/** Reader flags. */
+	enum ujson_reader_flags flags;
+
+	/** Handler to print errors and warnings */
+	void (*err_print)(void *err_print_priv, const char *line);
+	void *err_print_priv;
+
+	char err[UJSON_ERR_MAX];
+	char buf[];
+};
+
+/**
+ * @brief An ujson_val initializer.
+ *
+ * @param sbuf A pointer to a buffer used for string values.
+ * @param sbuf_size A length of the buffer used for string values.
+ *
+ * @return An ujson_val initialized with default values.
+ */
+#define UJSON_VAL_INIT(sbuf, sbuf_size) { \
+	.buf = sbuf, \
+	.buf_size = sbuf_size, \
+}
+
+/**
+ * @brief A parsed JSON key value pair.
+ */
+struct ujson_val {
+	/**
+	 * @brief A value type
+	 *
+	 * UJSON_VALUE_VOID means that no value was parsed.
+	 */
+	enum ujson_type type;
+
+	/** An user supplied buffer and size to store a string values to. */
+	char *buf;
+	size_t buf_size;
+
+	/**
+	 * @brief An index to attribute list.
+	 *
+	 * This is set by the ujson_obj_first_filter() and
+	 * ujson_obj_next_filter() functions.
+	 */
+	size_t idx;
+
+	/** An union to store the parsed value into. */
+	union {
+		/** @brief A boolean value. */
+		int val_bool;
+		/** @brief An integer value. */
+		long long val_int;
+		/** @brief A string value. */
+		const char *val_str;
+	};
+
+	/**
+	 * @brief A floating point value.
+	 *
+	 * Since integer values are subset of floating point values val_float
+	 * is always set when val_int was set.
+	 */
+	double val_float;
+
+	/** @brief An ID for object values */
+	char id[UJSON_ID_MAX];
+
+	char buf__[];
+};
+
+/**
+ * @brief Allocates a JSON value.
+ *
+ * @param buf_size A maximal buffer size for a string value, pass 0 for default.
+ * @return A newly allocated JSON value.
+ */
+ujson_val *ujson_val_alloc(size_t buf_size);
+
+/**
+ * @brief Frees a JSON value.
+ *
+ * @param self A JSON value previously allocated by ujson_val_alloc().
+ */
+void ujson_val_free(ujson_val *self);
+
+/**
+ * @brief Checks is result has valid type.
+ *
+ * @param res An ujson value.
+ * @return Zero if result is not valid, non-zero otherwise.
+ */
+static inline int ujson_val_valid(struct ujson_val *res)
+{
+	return !!res->type;
+}
+
+/**
+ * @brief Fills the reader error.
+ *
+ * Once buffer error is set all parsing functions return immediatelly with type
+ * set to UJSON_VOID.
+ *
+ * @param self An ujson_reader
+ * @param fmt A printf like format string
+ * @param ... A printf like parameters
+ */
+void ujson_err(ujson_reader *self, const char *fmt, ...)
+               __attribute__((format(printf, 2, 3)));
+
+/**
+ * @brief Prints error stored in the buffer.
+ *
+ * The error takes into consideration the current offset in the buffer and
+ * prints a few preceding lines along with the exact position of the error.
+ *
+ * The error is passed to the err_print() handler.
+ *
+ * @param self A ujson_reader
+ */
+void ujson_err_print(ujson_reader *self);
+
+/**
+ * @brief Prints a warning.
+ *
+ * Uses the print handler in the buffer to print a warning along with a few
+ * lines of context from the JSON at the current position.
+ *
+ * @param self A ujson_reader
+ * @param fmt A printf-like error string.
+ * @param ... A printf-like parameters.
+ */
+void ujson_warn(ujson_reader *self, const char *fmt, ...)
+               __attribute__((format(printf, 2, 3)));
+
+/**
+ * @brief Returns true if error was encountered.
+ *
+ * @param self A ujson_reader
+ * @return True if error was encountered false otherwise.
+ */
+static inline int ujson_reader_err(ujson_reader *self)
+{
+	return !!self->err[0];
+}
+
+/**
+ * @brief Returns the type of next element in buffer.
+ *
+ * @param self An ujson_reader
+ * @return A type of next element in the buffer.
+ */
+enum ujson_type ujson_next_type(ujson_reader *self);
+
+/**
+ * @brief Returns if first element in JSON is object or array.
+ *
+ * @param self A ujson_reader
+ * @return On success returns UJSON_OBJ or UJSON_ARR. On failure UJSON_VOID.
+ */
+enum ujson_type ujson_reader_start(ujson_reader *self);
+
+/**
+ * @brief Starts parsing of a JSON object.
+ *
+ * @param self An ujson_reader
+ * @param res An ujson_val to store the parsed value to.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_obj_first(ujson_reader *self, struct ujson_val *res);
+
+/**
+ * @brief Parses next value from a JSON object.
+ *
+ * If the res->type is UJSON_OBJ or UJSON_ARR it has to be parsed or skipped
+ * before next call to this function.
+ *
+ * @param self An ujson_reader.
+ * @param res A ujson_val to store the parsed value to.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_obj_next(ujson_reader *self, struct ujson_val *res);
+
+/**
+ * @brief A loop over a JSON object.
+ *
+ * @code
+ * UJSON_OBJ_FOREACH(reader, val) {
+ *	printf("Got value id '%s' type '%s'", val->id, ujson_type_name(val->type));
+ *	...
+ * }
+ * @endcode
+ *
+ * @param self An ujson_reader.
+ * @param res An ujson_val to store the next parsed value to.
+ */
+#define UJSON_OBJ_FOREACH(self, res) \
+	for (ujson_obj_first(self, res); ujson_val_valid(res); ujson_obj_next(self, res))
+
+/**
+ * @brief Utility function for log(n) lookup in a sorted array.
+ *
+ * @param list Analphabetically sorted array.
+ * @param list_len Array length.
+ *
+ * @return An array index or (size_t)-1 if key wasn't found.
+ */
+size_t ujson_lookup(const void *arr, size_t memb_size, size_t list_len,
+                    const char *key);
+
+/**
+ * @brief A JSON object attribute description i.e. key and type.
+ */
+typedef struct ujson_obj_attr {
+	/** @brief A JSON object key name. */
+	const char *key;
+	/**
+	 * @brief A JSON object value type.
+	 *
+	 * Note that because integer numbers are subset of floating point
+         * numbers if requested type was UJSON_FLOAT it will match if parsed
+         * type was UJSON_INT and the val_float will be set in addition to
+         * val_int.
+         */
+	enum ujson_type type;
+} ujson_obj_attr;
+
+/** @brief A JSON object description */
+typedef struct ujson_obj {
+	/**
+	 * @brief A list of attributes.
+	 *
+	 * Attributes we are looking for, the parser sets the val->idx for these.
+	 */
+	const ujson_obj_attr *attrs;
+	/** @brief A size of attrs array. */
+	size_t attr_cnt;
+} ujson_obj;
+
+static inline size_t ujson_obj_lookup(const ujson_obj *obj, const char *key)
+{
+	return ujson_lookup(obj->attrs, sizeof(*obj->attrs), obj->attr_cnt, key);
+}
+
+/** @brief An ujson_obj_attr initializer. */
+#define UJSON_OBJ_ATTR(keyv, typev) \
+	{.key = keyv, .type = typev}
+
+/** @brief An ujson_obj_attr intializer with an array index. */
+#define UJSON_OBJ_ATTR_IDX(key_idx, keyv, typev) \
+	[key_idx] = {.key = keyv, .type = typev}
+
+/**
+ * @brief Starts parsing of a JSON object with attribute lists.
+ *
+ * @param self An ujson_reader.
+ * @param res An ujson_val to store the parsed value to.
+ * @param obj An ujson_obj object description.
+ * @param ign A list of keys to ignore.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_obj_first_filter(ujson_reader *self, struct ujson_val *res,
+                           const struct ujson_obj *obj, const struct ujson_obj *ign);
+
+/**
+ * @brief An empty object attribute list.
+ *
+ * To be passed to UJSON_OBJ_FOREACH_FITLER() as ignore list.
+ */
+extern const struct ujson_obj *ujson_empty_obj;
+
+/**
+ * @brief Parses next value from a JSON object with attribute lists.
+ *
+ * If the res->type is UJSON_OBJ or UJSON_ARR it has to be parsed or skipped
+ * before next call to this function.
+ *
+ * @param self An ujson_reader.
+ * @param res An ujson_val to store the parsed value to.
+ * @param obj An ujson_obj object description.
+ * @param ign A list of keys to ignore. If set to NULL all unknown keys are
+ *            ignored, if set to ujson_empty_obj all unknown keys produce warnings.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_obj_next_filter(ujson_reader *self, struct ujson_val *res,
+                          const struct ujson_obj *obj, const struct ujson_obj *ign);
+
+/**
+ * @brief A loop over a JSON object with a pre-defined list of expected attributes.
+ *
+ * @code
+ * static struct ujson_obj_attr attrs[] = {
+ *	UJSON_OBJ_ATTR("bool", UJSON_BOOL),
+ *	UJSON_OBJ_ATTR("number", UJSON_INT),
+ * };
+ *
+ * static struct ujson_obj obj = {
+ *	.attrs = filter_attrs,
+ *	.attr_cnt = UJSON_ARRAY_SIZE(filter_attrs)
+ * };
+ *
+ * UJSON_OBJ_FOREACH_FILTER(reader, val, &obj, NULL) {
+ *	printf("Got value id '%s' type '%s'",
+ *	       attrs[val->idx].id, ujson_type_name(val->type));
+ *	...
+ * }
+ * @endcode
+ *
+ * @param self An ujson_reader.
+ * @param res An ujson_val to store the next parsed value to.
+ * @param obj An ujson_obj with a description of attributes to parse.
+ * @param ign An ujson_obj with a description of attributes to ignore.
+ */
+#define UJSON_OBJ_FOREACH_FILTER(self, res, obj, ign) \
+	for (ujson_obj_first_filter(self, res, obj, ign); \
+	     ujson_val_valid(res); \
+	     ujson_obj_next_filter(self, res, obj, ign))
+
+/**
+ * @brief Skips parsing of a JSON object.
+ *
+ * @param self An ujson_reader.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_obj_skip(ujson_reader *self);
+
+/**
+ * @brief Starts parsing of a JSON array.
+ *
+ * @param self An ujson_reader.
+ * @param res An ujson_val to store the parsed value to.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_arr_first(ujson_reader *self, struct ujson_val *res);
+
+/**
+ * @brief Parses next value from a JSON array.
+ *
+ * If the res->type is UJSON_OBJ or UJSON_ARR it has to be parsed or skipped
+ * before next call to this function.
+ *
+ * @param self An ujson_reader.
+ * @param res An ujson_val to store the parsed value to.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_arr_next(ujson_reader *self, struct ujson_val *res);
+
+/**
+ * @brief A loop over a JSON array.
+ *
+ * @code
+ * UJSON_ARR_FOREACH(reader, val) {
+ *	printf("Got value type '%s'", ujson_type_name(val->type));
+ *	...
+ * }
+ * @endcode
+ *
+ * @param self An ujson_reader.
+ * @param res An ujson_val to store the next parsed value to.
+ */
+#define UJSON_ARR_FOREACH(self, res) \
+	for (ujson_arr_first(self, res); ujson_val_valid(res); ujson_arr_next(self, res))
+
+/**
+ * @brief Skips parsing of a JSON array.
+ *
+ * @param self A ujson_reader.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_arr_skip(ujson_reader *self);
+
+/**
+ * @brief A JSON reader state.
+ */
+typedef struct ujson_reader_state {
+	size_t off;
+	unsigned int depth;
+} ujson_reader_state;
+
+/**
+ * @brief Returns a parser state at the start of current object/array.
+ *
+ * This function could be used for the parser to return to the start of the
+ * currently parsed object or array.
+ *
+ * @param self A ujson_reader
+ * @return A state that points to a start of the last object or array.
+ */
+static inline ujson_reader_state ujson_reader_state_save(ujson_reader *self)
+{
+	struct ujson_reader_state ret = {
+		.off = self->sub_off,
+		.depth = self->depth,
+	};
+
+	return ret;
+}
+
+/**
+ * @brief Returns the parser to a saved state.
+ *
+ * This function could be used for the parser to return to the start of
+ * object or array saved by t the ujson_reader_state_get() function.
+ *
+ * @param self A ujson_reader
+ * @param state An parser state as returned by the ujson_reader_state_get().
+ */
+static inline void ujson_reader_state_load(ujson_reader *self, ujson_reader_state state)
+{
+	if (ujson_reader_err(self))
+		return;
+
+	self->off = state.off;
+	self->sub_off = state.off;
+	self->depth = state.depth;
+}
+
+/**
+ * @brief Resets the parser to a start.
+ *
+ * @param self A ujson_reader
+ */
+static inline void ujson_reader_reset(ujson_reader *self)
+{
+	self->off = 0;
+	self->sub_off = 0;
+	self->depth = 0;
+	self->err[0] = 0;
+}
+
+/**
+ * @brief Loads a file into an ujson_reader buffer.
+ *
+ * The reader has to be later freed by ujson_reader_free().
+ *
+ * @param path A path to a file.
+ * @return A ujson_reader or NULL in a case of a failure.
+ */
+ujson_reader *ujson_reader_load(const char *path);
+
+/**
+ * @brief Frees an ujson_reader buffer.
+ *
+ * @param self A ujson_reader allocated by ujson_reader_load() function.
+ */
+void ujson_reader_free(ujson_reader *self);
+
+/**
+ * @brief Prints errors and warnings at the end of parsing.
+ *
+ * Checks if self->err is set and prints the error with ujson_reader_err()
+ *
+ * Checks if there is any text left after the parser has finished with
+ * ujson_reader_consumed() and prints a warning if there were any non-whitespace
+ * characters left.
+ *
+ * @param self A ujson_reader
+ */
+void ujson_reader_finish(ujson_reader *self);
+
+/**
+ * @brief Returns non-zero if whole buffer has been consumed.
+ *
+ * @param self A ujson_reader.
+ * @return Non-zero if whole buffer was consumed.
+ */
+static inline int ujson_reader_consumed(ujson_reader *self)
+{
+	return self->off >= self->len;
+}
+
+#endif /* UJSON_H */
diff --git a/include/ujson_utf.h b/include/ujson_utf.h
new file mode 100644
index 000000000..f939fbe8c
--- /dev/null
+++ b/include/ujson_utf.h
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2022-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+/**
+ * @file ujson_utf.h
+ * @brief Unicode helper macros and functions.
+ */
+
+#ifndef UJSON_UTF_H
+#define UJSON_UTF_H
+
+#include <stdint.h>
+#include <stddef.h>
+
+/** Returns true if unicode byte is ASCII */
+#define UJSON_UTF8_IS_ASCII(ch) (!((ch) & 0x80))
+/** Returns true if we have first unicode byte of single byte sequence */
+#define UJSON_UTF8_IS_NBYTE(ch) (((ch) & 0xc0) == 0x80)
+/** Returns true if we have first unicode byte of two byte sequence */
+#define UJSON_UTF8_IS_2BYTE(ch) (((ch) & 0xe0) == 0xc0)
+/** Returns true if we have first unicode byte of three byte sequence */
+#define UJSON_UTF8_IS_3BYTE(ch) (((ch) & 0xf0) == 0xe0)
+/** Returns true if we have first unicode byte of four byte sequence */
+#define UJSON_UTF8_IS_4BYTE(ch) (((ch) & 0xf8) == 0xf0)
+
+#define UJSON_UTF8_NBYTE_MASK 0x3f
+
+/**
+ * @brief Parses next unicode character in UTF-8 string.
+ * @param str A pointer to the C string.
+ * @return A unicode character or 0 on error or end of the string.
+ */
+static inline uint32_t ujson_utf8_next(const char **str)
+{
+	uint32_t s0 = *str[0];
+
+	(*str)++;
+
+	if (UJSON_UTF8_IS_ASCII(s0))
+		return s0;
+
+	uint32_t s1 = *str[0];
+
+	if (!UJSON_UTF8_IS_NBYTE(s1))
+		return 0;
+
+	s1 &= UJSON_UTF8_NBYTE_MASK;
+
+	(*str)++;
+
+	if (UJSON_UTF8_IS_2BYTE(s0))
+		return (s0 & 0x1f)<<6 | s1;
+
+	uint32_t s2 = *str[0];
+
+	if (!UJSON_UTF8_IS_NBYTE(s2))
+		return 0;
+
+	s2 &= UJSON_UTF8_NBYTE_MASK;
+
+	(*str)++;
+
+	if (UJSON_UTF8_IS_3BYTE(s0))
+		return (s0 & 0x0f)<<12 | s1<<6 | s2;
+
+	(*str)++;
+
+	uint32_t s3 = *str[0];
+
+	if (!UJSON_UTF8_IS_NBYTE(s2))
+		return 0;
+
+	s3 &= UJSON_UTF8_NBYTE_MASK;
+
+	if (UJSON_UTF8_IS_4BYTE(s0))
+		return (s0 & 0x07)<<18 | s1<<12 | s2<<6 | s3;
+
+	return 0;
+}
+
+/**
+ * @brief Returns number of bytes next character is occupying in an UTF-8 string.
+ *
+ * @param str A pointer to a string.
+ * @param off An offset into the string, must point to a valid multibyte boundary.
+ * @return Number of bytes next character occupies, zero on string end and -1 on failure.
+ */
+int8_t ujson_utf8_next_chsz(const char *str, size_t off);
+
+/**
+ * @brief Returns number of bytes previous character is occupying in an UTF-8 string.
+ *
+ * @param str A pointer to a string.
+ * @param off An offset into the string, must point to a valid multibyte boundary.
+ * @return Number of bytes previous character occupies, and -1 on failure.
+ */
+int8_t ujson_utf8_prev_chsz(const char *str, size_t off);
+
+/**
+ * @brief Returns a number of characters in UTF-8 string.
+ *
+ * Returns number of characters in an UTF-8 string, which may be less or equal
+ * to what strlen() reports.
+ *
+ * @param str An UTF-8 string.
+ * @return Number of characters in the string.
+ */
+size_t ujson_utf8_strlen(const char *str);
+
+/**
+ * @brief Returns a number of bytes needed to store unicode character into UTF-8.
+ *
+ * @param unicode A unicode character.
+ * @return Number of utf8 bytes required to store a unicode character.
+ */
+static inline unsigned int ujson_utf8_bytes(uint32_t unicode)
+{
+	if (unicode < 0x0080)
+		return 1;
+
+	if (unicode < 0x0800)
+		return 2;
+
+	if (unicode < 0x10000)
+		return 3;
+
+	return 4;
+}
+
+/**
+ * @brief Writes an unicode character into a UTF-8 buffer.
+ *
+ * The buffer _must_ be large enough!
+ *
+ * @param unicode A unicode character.
+ * @param buf A byte buffer.
+ * @return A number of bytes written.
+ */
+static inline int ujson_to_utf8(uint32_t unicode, char *buf)
+{
+	if (unicode < 0x0080) {
+		buf[0] = unicode & 0x007f;
+		return 1;
+	}
+
+	if (unicode < 0x0800) {
+		buf[0] = 0xc0 | (0x1f & (unicode>>6));
+		buf[1] = 0x80 | (0x3f & unicode);
+		return 2;
+	}
+
+	if (unicode < 0x10000) {
+		buf[0] = 0xe0 | (0x0f & (unicode>>12));
+		buf[1] = 0x80 | (0x3f & (unicode>>6));
+		buf[2] = 0x80 | (0x3f & unicode);
+		return 3;
+	}
+
+	buf[0] = 0xf0 | (0x07 & (unicode>>18));
+	buf[1] = 0x80 | (0x3f & (unicode>>12));
+	buf[2] = 0x80 | (0x3f & (unicode>>6));
+	buf[3] = 0x80 | (0x3f & unicode);
+	return 4;
+}
+
+#endif /* UJSON_UTF_H */
diff --git a/include/ujson_writer.h b/include/ujson_writer.h
new file mode 100644
index 000000000..dfcc95053
--- /dev/null
+++ b/include/ujson_writer.h
@@ -0,0 +1,224 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+/**
+ * @file ujson_writer.h
+ * @brief A JSON writer.
+ *
+ * All the function that add values return zero on success and non-zero on a
+ * failure. Once an error has happened all subsequent attempts to add more
+ * values return with non-zero exit status immediatelly. This is designed
+ * so that we can add several values without checking each return value
+ * and only check if error has happened at the end of the sequence.
+ *
+ * Failures may occur:
+ * - if we call the functions out of order, e.g. attempt to finish array when
+ *   we are not writing out an array.
+ * - if we run out of recursion stack
+ * - may be propagated from the writer function, e.g. allocation failure, no
+ *   space on disk, etc.
+ */
+
+#ifndef UJSON_WRITER_H
+#define UJSON_WRITER_H
+
+#include <ujson_common.h>
+
+/** @brief A JSON writer */
+struct ujson_writer {
+	unsigned int depth;
+	char depth_type[UJSON_RECURSION_MAX/8];
+	char depth_first[UJSON_RECURSION_MAX/8];
+
+	/** Handler to print errors and warnings */
+	void (*err_print)(void *err_print_priv, const char *line);
+	void *err_print_priv;
+	char err[UJSON_ERR_MAX];
+
+	/** Handler to produce JSON output */
+	int (*out)(struct ujson_writer *self, const char *buf, size_t buf_size);
+	void *out_priv;
+};
+
+/**
+ * @brief An ujson_writer initializer with default values.
+ *
+ * @param vout A pointer to function to write out the data.
+ * @param vout_priv An user pointer passed to the out function.
+ *
+ * @return An ujson_writer initialized with default values.
+ */
+#define UJSON_WRITER_INIT(vout, vout_priv) { \
+	.err_print = UJSON_ERR_PRINT, \
+	.err_print_priv = UJSON_ERR_PRINT_PRIV, \
+	.out = vout, \
+	.out_priv = vout_priv \
+}
+
+/**
+ * @brief Allocates a JSON file writer.
+ *
+ * The call may fail either when file cannot be opened for writing or if
+ * allocation has failed. In all cases errno should be set correctly.
+ *
+ * @param path A path to the file, file is opened for writing and created if it
+ *             does not exist.
+ *
+ * @return A ujson_writer pointer or NULL in a case of failure.
+ */
+ujson_writer *ujson_writer_file_open(const char *path);
+
+/**
+ * @brief Closes and frees a JSON file writer.
+ *
+ * @param self A ujson_writer file writer.
+ *
+ * @return Zero on success, non-zero on a failure and errno is set.
+ */
+int ujson_writer_file_close(ujson_writer *self);
+
+/**
+ * @brief Returns true if writer error happened.
+ *
+ * @param self A JSON writer.
+ *
+ * @return True if error has happened.
+ */
+static inline int ujson_writer_err(ujson_writer *self)
+{
+	return !!self->err[0];
+}
+
+/**
+ * @brief Starts a JSON object.
+ *
+ * For a top level object the id must be NULL, every other object has to have
+ * non-NULL id. The call will also fail if maximal recursion depth
+ * UJSON_RECURSION_MAX has been reached.
+ *
+ * @param self A JSON writer.
+ * @param id An object name.
+ *
+ * @return Zero on a success, non-zero otherwise.
+ */
+int ujson_obj_start(ujson_writer *self, const char *id);
+
+/**
+ * @brief Finishes a JSON object.
+ *
+ * The call will fail if we are currenlty not writing out an object.
+ *
+ * @param self A JSON writer.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_obj_finish(ujson_writer *self);
+
+/**
+ * @brief Starts a JSON array.
+ *
+ * For a top level array the id must be NULL, every other array has to have
+ * non-NULL id. The call will also fail if maximal recursion depth
+ * UJSON_RECURSION_MAX has been reached.
+ *
+ * @param self A JSON writer.
+ * @param id An array name.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_arr_start(ujson_writer *self, const char *id);
+
+/**
+ * @brief Finishes a JSON array.
+ *
+ * The call will fail if we are currenlty not writing out an array.
+ *
+ * @param self A JSON writer.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_arr_finish(ujson_writer *self);
+
+/**
+ * @brief Adds a null value.
+ *
+ * The id must be NULL inside of an array, and must be non-NULL inside of an
+ * object.
+ *
+ * @param self A JSON writer.
+ * @param id A null value name.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_null_add(ujson_writer *self, const char *id);
+
+/**
+ * @brief Adds an integer value.
+ *
+ * The id must be NULL inside of an array, and must be non-NULL inside of an
+ * object.
+ *
+ * @param self A JSON writer.
+ * @param id An integer value name.
+ * @param val An integer value.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_int_add(ujson_writer *self, const char *id, long val);
+
+/**
+ * @brief Adds a bool value.
+ *
+ * The id must be NULL inside of an array, and must be non-NULL inside of an
+ * object.
+ *
+ * @param self A JSON writer.
+ * @param id An boolean value name.
+ * @param val A boolean value.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_bool_add(ujson_writer *self, const char *id, int val);
+
+/**
+ * @brief Adds a float value.
+ *
+ * The id must be NULL inside of an array, and must be non-NULL inside of an
+ * object.
+ *
+ * @param self A JSON writer.
+ * @param id A floating point value name.
+ * @param val A floating point value.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_float_add(ujson_writer *self, const char *id, double val);
+
+/**
+ * @brief Adds a string value.
+ *
+ * The id must be NULL inside of an array, and must be non-NULL inside of an
+ * object.
+ *
+ * @param self A JSON writer.
+ * @param id A string value name.
+ * @param str An UTF8 string value.
+ *
+ * @return Zero on success, non-zero otherwise.
+ */
+int ujson_str_add(ujson_writer *self, const char *id, const char *str);
+
+/**
+ * @brief Finalizes json writer.
+ *
+ * Finalizes the json writer, throws possible errors through the error printing
+ * function.
+ *
+ * @param self A JSON writer.
+ * @return Overall error value.
+ */
+int ujson_writer_finish(ujson_writer *self);
+
+#endif /* UJSON_WRITER_H */
diff --git a/libs/ujson/Makefile b/libs/ujson/Makefile
new file mode 100644
index 000000000..4c8508010
--- /dev/null
+++ b/libs/ujson/Makefile
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright (C) Cyril Hrubis <chrubis@suse.cz>
+
+top_srcdir		?= ../..
+
+include $(top_srcdir)/include/mk/env_pre.mk
+
+INTERNAL_LIB		:= libujson.a
+
+include $(top_srcdir)/include/mk/lib.mk
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
diff --git a/libs/ujson/ujson_common.c b/libs/ujson/ujson_common.c
new file mode 100644
index 000000000..c9cada9a9
--- /dev/null
+++ b/libs/ujson/ujson_common.c
@@ -0,0 +1,38 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+#include <stdio.h>
+#include "ujson_common.h"
+
+void ujson_err_handler(void *err_print_priv, const char *line)
+{
+	fputs(line, err_print_priv);
+	putc('\n', err_print_priv);
+}
+
+const char *ujson_type_name(enum ujson_type type)
+{
+	switch (type) {
+	case UJSON_VOID:
+		return "void";
+	case UJSON_INT:
+		return "integer";
+	case UJSON_FLOAT:
+		return "float";
+	case UJSON_BOOL:
+		return "boolean";
+	case UJSON_NULL:
+		return "null";
+	case UJSON_STR:
+		return "string";
+	case UJSON_OBJ:
+		return "object";
+	case UJSON_ARR:
+		return "array";
+	default:
+		return "invalid";
+	}
+}
+
diff --git a/libs/ujson/ujson_reader.c b/libs/ujson/ujson_reader.c
new file mode 100644
index 000000000..d508f00d3
--- /dev/null
+++ b/libs/ujson/ujson_reader.c
@@ -0,0 +1,1081 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdarg.h>
+#include <stdint.h>
+
+#include "ujson_utf.h"
+#include "ujson_reader.h"
+
+static const struct ujson_obj empty = {};
+const struct ujson_obj *ujson_empty_obj = &empty;
+
+static inline int buf_empty(ujson_reader *buf)
+{
+	return buf->off >= buf->len;
+}
+
+static int eatws(ujson_reader *buf)
+{
+	while (!buf_empty(buf)) {
+		switch (buf->json[buf->off]) {
+		case ' ':
+		case '\t':
+		case '\n':
+		case '\r':
+		break;
+		default:
+			goto ret;
+		}
+
+		buf->off += 1;
+	}
+ret:
+	return buf_empty(buf);
+}
+
+static char getb(ujson_reader *buf)
+{
+	if (buf_empty(buf))
+		return 0;
+
+	return buf->json[buf->off++];
+}
+
+static char peekb_off(ujson_reader *buf, size_t off)
+{
+	if (buf->off + off >= buf->len)
+		return 0;
+
+	return buf->json[buf->off + off];
+}
+
+static char peekb(ujson_reader *buf)
+{
+	if (buf_empty(buf))
+		return 0;
+
+	return buf->json[buf->off];
+}
+
+static int eatb(ujson_reader *buf, char ch)
+{
+	if (peekb(buf) != ch)
+		return 0;
+
+	getb(buf);
+	return 1;
+}
+
+static int eatb2(ujson_reader *buf, char ch1, char ch2)
+{
+	if (peekb(buf) != ch1 && peekb(buf) != ch2)
+		return 0;
+
+	getb(buf);
+	return 1;
+}
+
+static int eatstr(ujson_reader *buf, const char *str)
+{
+	while (*str) {
+		if (!eatb(buf, *str))
+			return 0;
+		str++;
+	}
+
+	return 1;
+}
+
+static int hex2val(unsigned char b)
+{
+	switch (b) {
+	case '0' ... '9':
+		return b - '0';
+	case 'a' ... 'f':
+		return b - 'a' + 10;
+	case 'A' ... 'F':
+		return b - 'A' + 10;
+	default:
+		return -1;
+	}
+}
+
+static int32_t parse_ucode_cp(ujson_reader *buf)
+{
+	int ret = 0, v, i;
+
+	for (i = 0; i < 4; i++) {
+		if ((v = hex2val(getb(buf))) < 0)
+			goto err;
+		ret *= 16;
+		ret += v;
+	}
+
+	return ret;
+err:
+	ujson_err(buf, "Expected four hexadecimal digits");
+	return -1;
+}
+
+static unsigned int parse_ucode_esc(ujson_reader *buf, char *str,
+                                    size_t off, size_t len)
+{
+	int32_t ucode = parse_ucode_cp(buf);
+
+	if (ucode < 0)
+		return 0;
+
+	if (!str)
+		return ucode;
+
+	if (ujson_utf8_bytes(ucode) + 1 >= len - off) {
+		ujson_err(buf, "String buffer too short!");
+		return 0;
+	}
+
+	return ujson_to_utf8(ucode, str+off);
+}
+
+static int copy_str(ujson_reader *buf, char *str, size_t len)
+{
+	size_t pos = 0;
+	int esc = 0;
+	unsigned int l;
+
+	eatb(buf, '"');
+
+	for (;;) {
+		if (buf_empty(buf)) {
+			ujson_err(buf, "Unterminated string");
+			return 1;
+		}
+
+		if (!esc && eatb(buf, '"')) {
+			if (str)
+				str[pos] = 0;
+			return 0;
+		}
+
+		unsigned char b = getb(buf);
+
+		if (b < 0x20) {
+			if (!peekb(buf))
+				ujson_err(buf, "Unterminated string");
+			else
+				ujson_err(buf, "Invalid string character 0x%02x", b);
+			return 1;
+		}
+
+		if (!esc && b == '\\') {
+			esc = 1;
+			continue;
+		}
+
+		if (esc) {
+			switch (b) {
+			case '"':
+			case '\\':
+			case '/':
+			break;
+			case 'b':
+				b = '\b';
+			break;
+			case 'f':
+				b = '\f';
+			break;
+			case 'n':
+				b = '\n';
+			break;
+			case 'r':
+				b = '\r';
+			break;
+			case 't':
+				b = '\t';
+			break;
+			case 'u':
+				if (!(l = parse_ucode_esc(buf, str, pos, len)))
+					return 1;
+				pos += l;
+				b = 0;
+			break;
+			default:
+				ujson_err(buf, "Invalid escape \\%c", b);
+				return 1;
+			}
+			esc = 0;
+		}
+
+		if (str && b) {
+			if (pos + 1 >= len) {
+				ujson_err(buf, "String buffer too short!");
+				return 1;
+			}
+
+			str[pos++] = b;
+		}
+	}
+
+	return 1;
+}
+
+static int copy_id_str(ujson_reader *buf, char *str, size_t len)
+{
+	size_t pos = 0;
+
+	if (eatws(buf))
+		goto err0;
+
+	if (!eatb(buf, '"'))
+		goto err0;
+
+	for (;;) {
+		if (buf_empty(buf)) {
+			ujson_err(buf, "Unterminated ID string");
+			return 1;
+		}
+
+		if (eatb(buf, '"')) {
+			str[pos] = 0;
+			break;
+		}
+
+		if (pos >= len-1) {
+			ujson_err(buf, "ID string too long");
+			return 1;
+		}
+
+		str[pos++] = getb(buf);
+	}
+
+	if (eatws(buf))
+		goto err1;
+
+	if (!eatb(buf, ':'))
+		goto err1;
+
+	return 0;
+err0:
+	ujson_err(buf, "Expected ID string");
+	return 1;
+err1:
+	ujson_err(buf, "Expected ':' after ID string");
+	return 1;
+}
+
+static int is_digit(char b)
+{
+	switch (b) {
+	case '0' ... '9':
+		return 1;
+	default:
+		return 0;
+	}
+}
+
+static int get_int(ujson_reader *buf, struct ujson_val *res)
+{
+	long val = 0;
+	int sign = 1;
+
+	if (eatb(buf, '-')) {
+		sign = -1;
+		if (!is_digit(peekb(buf))) {
+			ujson_err(buf, "Expected digit(s)");
+			return 1;
+		}
+	}
+
+	if (peekb(buf) == '0' && is_digit(peekb_off(buf, 1))) {
+		ujson_err(buf, "Leading zero in number!");
+		return 1;
+	}
+
+	while (is_digit(peekb(buf))) {
+		val *= 10;
+		val += getb(buf) - '0';
+		//TODO: overflow?
+	}
+
+	if (sign < 0)
+		val = -val;
+
+	res->val_int = val;
+	res->val_float = val;
+
+	return 0;
+}
+
+static int eat_digits(ujson_reader *buf)
+{
+	if (!is_digit(peekb(buf))) {
+		ujson_err(buf, "Expected digit(s)");
+		return 1;
+	}
+
+	while (is_digit(peekb(buf)))
+		getb(buf);
+
+	return 0;
+}
+
+static int get_float(ujson_reader *buf, struct ujson_val *res)
+{
+	off_t start = buf->off;
+
+	eatb(buf, '-');
+
+	if (peekb(buf) == '0' && is_digit(peekb_off(buf, 1))) {
+		ujson_err(buf, "Leading zero in float");
+		return 1;
+	}
+
+	if (eat_digits(buf))
+		return 1;
+
+	switch (getb(buf)) {
+	case '.':
+		if (eat_digits(buf))
+			return 1;
+
+		if (!eatb2(buf, 'e', 'E'))
+			break;
+
+		/* fallthrough */
+	case 'e':
+	case 'E':
+		eatb2(buf, '+', '-');
+
+		if (eat_digits(buf))
+			return 1;
+	break;
+	}
+
+	size_t len = buf->off - start;
+	char tmp[len+1];
+
+	memcpy(tmp, buf->json + start, len);
+
+	tmp[len] = 0;
+
+	res->val_float = strtod(tmp, NULL);
+
+	return 0;
+}
+
+static int get_bool(ujson_reader *buf, struct ujson_val *res)
+{
+	switch (peekb(buf)) {
+	case 'f':
+		if (!eatstr(buf, "false")) {
+			ujson_err(buf, "Expected 'false'");
+			return 1;
+		}
+
+		res->val_bool = 0;
+	break;
+	case 't':
+		if (!eatstr(buf, "true")) {
+			ujson_err(buf, "Expected 'true'");
+			return 1;
+		}
+
+		res->val_bool = 1;
+	break;
+	}
+
+	return 0;
+}
+
+static int get_null(ujson_reader *buf)
+{
+	if (!eatstr(buf, "null")) {
+		ujson_err(buf, "Expected 'null'");
+		return 1;
+	}
+
+	return 0;
+}
+
+int ujson_obj_skip(ujson_reader *buf)
+{
+	struct ujson_val res = {};
+
+	UJSON_OBJ_FOREACH(buf, &res) {
+		switch (res.type) {
+		case UJSON_OBJ:
+			if (ujson_obj_skip(buf))
+				return 1;
+		break;
+		case UJSON_ARR:
+			if (ujson_arr_skip(buf))
+				return 1;
+		break;
+		default:
+		break;
+		}
+	}
+
+	return 0;
+}
+
+int ujson_arr_skip(ujson_reader *buf)
+{
+	struct ujson_val res = {};
+
+	UJSON_ARR_FOREACH(buf, &res) {
+		switch (res.type) {
+		case UJSON_OBJ:
+			if (ujson_obj_skip(buf))
+				return 1;
+		break;
+		case UJSON_ARR:
+			if (ujson_arr_skip(buf))
+				return 1;
+		break;
+		default:
+		break;
+		}
+	}
+
+	return 0;
+}
+
+static enum ujson_type next_num_type(ujson_reader *buf)
+{
+	size_t off = 0;
+
+	for (;;) {
+		char b = peekb_off(buf, off++);
+
+		switch (b) {
+		case 0:
+		case ',':
+			return UJSON_INT;
+		case '.':
+		case 'e':
+		case 'E':
+			return UJSON_FLOAT;
+		}
+	}
+
+	return UJSON_VOID;
+}
+
+enum ujson_type ujson_next_type(ujson_reader *buf)
+{
+	if (eatws(buf)) {
+		ujson_err(buf, "Unexpected end");
+		return UJSON_VOID;
+	}
+
+	char b = peekb(buf);
+
+	switch (b) {
+	case '{':
+		return UJSON_OBJ;
+	case '[':
+		return UJSON_ARR;
+	case '"':
+		return UJSON_STR;
+	case '-':
+	case '0' ... '9':
+		return next_num_type(buf);
+	case 'f':
+	case 't':
+		return UJSON_BOOL;
+	break;
+	case 'n':
+		return UJSON_NULL;
+	break;
+	default:
+		ujson_err(buf, "Expected object, array, number or string");
+		return UJSON_VOID;
+	}
+}
+
+enum ujson_type ujson_reader_start(ujson_reader *buf)
+{
+	enum ujson_type type = ujson_next_type(buf);
+
+	switch (type) {
+	case UJSON_ARR:
+	case UJSON_OBJ:
+	case UJSON_VOID:
+	break;
+	default:
+		ujson_err(buf, "JSON can start only with array or object");
+		type = UJSON_VOID;
+	break;
+	}
+
+	return type;
+}
+
+static int get_value(ujson_reader *buf, struct ujson_val *res)
+{
+	int ret = 0;
+
+	res->type = ujson_next_type(buf);
+
+	switch (res->type) {
+	case UJSON_STR:
+		if (copy_str(buf, res->buf, res->buf_size)) {
+			res->type = UJSON_VOID;
+			return 0;
+		}
+		res->val_str = res->buf;
+		return 1;
+	case UJSON_INT:
+		ret = get_int(buf, res);
+	break;
+	case UJSON_FLOAT:
+		ret = get_float(buf, res);
+	break;
+	case UJSON_BOOL:
+		ret = get_bool(buf, res);
+	break;
+	case UJSON_NULL:
+		ret = get_null(buf);
+	break;
+	case UJSON_VOID:
+		return 0;
+	case UJSON_ARR:
+	case UJSON_OBJ:
+		buf->sub_off = buf->off;
+		return 1;
+	}
+
+	if (ret) {
+		res->type = UJSON_VOID;
+		return 0;
+	}
+
+	return 1;
+}
+
+static int pre_next(ujson_reader *buf, struct ujson_val *res)
+{
+	if (!eatb(buf, ',')) {
+		ujson_err(buf, "Expected ','");
+		res->type = UJSON_VOID;
+		return 1;
+	}
+
+	if (eatws(buf)) {
+		ujson_err(buf, "Unexpected end");
+		res->type = UJSON_VOID;
+		return 1;
+	}
+
+	return 0;
+}
+
+static int check_end(ujson_reader *buf, struct ujson_val *res, char b)
+{
+	if (eatws(buf)) {
+		ujson_err(buf, "Unexpected end");
+		return 1;
+	}
+
+	if (eatb(buf, b)) {
+		res->type = UJSON_VOID;
+		eatws(buf);
+		eatb(buf, 0);
+		buf->depth--;
+		return 1;
+	}
+
+	return 0;
+}
+
+/*
+ * This is supposed to return a pointer to a string stored as a first member of
+ * a structure given an array.
+ *
+ * e.g.
+ *
+ *	struct foo {
+ *		const char *key;
+ *		...
+ *	};
+ *
+ *	const struct foo bar[10] = {...};
+ *
+ *      // Returns a pointer to the key string in a second structure in bar[].
+ *	const char *key = list_elem(bar, sizeof(struct foo), 1);
+ */
+static inline const char *list_elem(const void *arr, size_t memb_size, size_t idx)
+{
+	return *(const char**)(arr + idx * memb_size);
+}
+
+size_t ujson_lookup(const void *arr, size_t memb_size, size_t list_len,
+                    const char *key)
+{
+	size_t l = 0;
+	size_t r = list_len-1;
+	size_t mid = -1;
+
+	if (!list_len)
+		return (size_t)-1;
+
+	while (r - l > 1) {
+		mid = (l+r)/2;
+
+		int ret = strcmp(list_elem(arr, memb_size, mid), key);
+		if (!ret)
+			return mid;
+
+		if (ret < 0)
+			l = mid;
+		else
+			r = mid;
+	}
+
+	if (r != mid && !strcmp(list_elem(arr, memb_size, r), key))
+		return r;
+
+	if (l != mid && !strcmp(list_elem(arr, memb_size, l), key))
+		return l;
+
+	return -1;
+}
+
+static int skip_obj_val(ujson_reader *buf)
+{
+	struct ujson_val dummy = {};
+
+	if (!get_value(buf, &dummy))
+		return 0;
+
+	switch (dummy.type) {
+	case UJSON_OBJ:
+		return !ujson_obj_skip(buf);
+	case UJSON_ARR:
+		return !ujson_arr_skip(buf);
+	default:
+		return 1;
+	}
+}
+
+static int obj_next(ujson_reader *buf, struct ujson_val *res)
+{
+	if (copy_id_str(buf, res->id, sizeof(res->id)))
+		return 0;
+
+	return get_value(buf, res);
+}
+
+static int obj_pre_next(ujson_reader *buf, struct ujson_val *res)
+{
+	if (ujson_reader_err(buf))
+		return 1;
+
+	if (check_end(buf, res, '}'))
+		return 1;
+
+	if (pre_next(buf, res))
+		return 1;
+
+	return 0;
+}
+
+static int obj_next_filter(ujson_reader *buf, struct ujson_val *res,
+                           const struct ujson_obj *obj, const struct ujson_obj *ign)
+{
+	const struct ujson_obj_attr *attr;
+
+	for (;;) {
+		if (copy_id_str(buf, res->id, sizeof(res->id)))
+			return 0;
+
+		res->idx = obj ? ujson_obj_lookup(obj, res->id) : (size_t)-1;
+
+		if (res->idx != (size_t)-1) {
+			if (!get_value(buf, res))
+				return 0;
+
+			attr = &obj->attrs[res->idx];
+
+			if (attr->type == UJSON_VOID)
+				return 1;
+
+			if (attr->type == res->type)
+				return 1;
+
+			if (attr->type == UJSON_FLOAT &&
+			    res->type == UJSON_INT)
+				return 1;
+
+			ujson_warn(buf, "Wrong '%s' type expected %s",
+				     attr->key, ujson_type_name(attr->type));
+		} else {
+			if (!skip_obj_val(buf))
+				return 0;
+
+			if (ign && ujson_obj_lookup(ign, res->id) == (size_t)-1)
+				ujson_warn(buf, "Unexpected key '%s'", res->id);
+		}
+
+		if (obj_pre_next(buf, res))
+			return 0;
+	}
+}
+
+static int check_err(ujson_reader *buf, struct ujson_val *res)
+{
+	if (ujson_reader_err(buf)) {
+		res->type = UJSON_VOID;
+		return 1;
+	}
+
+	return 0;
+}
+
+int ujson_obj_next_filter(ujson_reader *buf, struct ujson_val *res,
+                            const struct ujson_obj *obj, const struct ujson_obj *ign)
+{
+	if (check_err(buf, res))
+		return 0;
+
+	if (obj_pre_next(buf, res))
+		return 0;
+
+	return obj_next_filter(buf, res, obj, ign);
+}
+
+int ujson_obj_next(ujson_reader *buf, struct ujson_val *res)
+{
+	if (check_err(buf, res))
+		return 0;
+
+	if (obj_pre_next(buf, res))
+		return 0;
+
+	return obj_next(buf, res);
+}
+
+static int any_first(ujson_reader *buf, char b)
+{
+	if (eatws(buf)) {
+		ujson_err(buf, "Unexpected end");
+		return 1;
+	}
+
+	if (!eatb(buf, b)) {
+		ujson_err(buf, "Expected '%c'", b);
+		return 1;
+	}
+
+	buf->depth++;
+
+	if (buf->depth > buf->max_depth) {
+		ujson_err(buf, "Recursion too deep");
+		return 1;
+	}
+
+	return 0;
+}
+
+int ujson_obj_first_filter(ujson_reader *buf, struct ujson_val *res,
+                             const struct ujson_obj *obj, const struct ujson_obj *ign)
+{
+	if (check_err(buf, res))
+		return 0;
+
+	if (any_first(buf, '{'))
+		return 0;
+
+	if (check_end(buf, res, '}'))
+		return 0;
+
+	return obj_next_filter(buf, res, obj, ign);
+}
+
+int ujson_obj_first(ujson_reader *buf, struct ujson_val *res)
+{
+	if (check_err(buf, res))
+		return 0;
+
+	if (any_first(buf, '{'))
+		return 0;
+
+	if (check_end(buf, res, '}'))
+		return 0;
+
+	return obj_next(buf, res);
+}
+
+static int arr_next(ujson_reader *buf, struct ujson_val *res)
+{
+	return get_value(buf, res);
+}
+
+int ujson_arr_first(ujson_reader *buf, struct ujson_val *res)
+{
+	if (check_err(buf, res))
+		return 0;
+
+	if (any_first(buf, '['))
+		return 0;
+
+	if (check_end(buf, res, ']'))
+		return 0;
+
+	return arr_next(buf, res);
+}
+
+int ujson_arr_next(ujson_reader *buf, struct ujson_val *res)
+{
+	if (check_err(buf, res))
+		return 0;
+
+	if (check_end(buf, res, ']'))
+		return 0;
+
+	if (pre_next(buf, res))
+		return 0;
+
+	return arr_next(buf, res);
+}
+
+static void ujson_err_va(ujson_reader *buf, const char *fmt, va_list va)
+{
+	vsnprintf(buf->err, UJSON_ERR_MAX, fmt, va);
+}
+
+void ujson_err(ujson_reader *buf, const char *fmt, ...)
+{
+	va_list va;
+
+	va_start(va, fmt);
+	ujson_err_va(buf, fmt, va);
+	va_end(va);
+}
+
+static void vprintf_line(ujson_reader *buf, const char *fmt, va_list va)
+{
+	char line[UJSON_ERR_MAX+1];
+
+	vsnprintf(line, sizeof(line), fmt, va);
+
+	line[UJSON_ERR_MAX] = 0;
+
+	buf->err_print(buf->err_print_priv, line);
+}
+
+static void printf_line(ujson_reader *buf, const char *fmt, ...)
+{
+	va_list va;
+
+	va_start(va, fmt);
+	vprintf_line(buf, fmt, va);
+	va_end(va);
+}
+
+static void printf_json_line(ujson_reader *buf, size_t line_nr, const char *buf_pos)
+{
+	char line[UJSON_ERR_MAX+1];
+	size_t plen, i;
+
+	plen = sprintf(line, "%03zu: ", line_nr);
+
+	for (i = 0; i < UJSON_ERR_MAX-plen && buf_pos[i] && buf_pos[i] != '\n'; i++)
+		line[i+plen] = buf_pos[i];
+
+	line[i+plen] = 0;
+
+	buf->err_print(buf->err_print_priv, line);
+}
+
+static void print_arrow(ujson_reader *buf, const char *buf_pos, size_t count)
+{
+	char line[count + 7];
+	size_t i;
+
+	/* The '000: ' prefix */
+	for (i = 0; i <= 5; i++)
+		line[i] = ' ';
+
+	for (i = 0; i < count; i++)
+		line[i+5] = buf_pos[i] == '\t' ? '\t' : ' ';
+
+	line[count+5] = '^';
+	line[count+6] = 0;
+
+	buf->err_print(buf->err_print_priv, line);
+}
+
+#define ERR_LINES 10
+
+#define MIN(A, B) ((A < B) ? (A) : (B))
+
+static void print_snippet(ujson_reader *buf, const char *type)
+{
+	ssize_t i;
+	const char *lines[ERR_LINES] = {};
+	size_t cur_line = 0;
+	size_t cur_off = 0;
+	size_t last_off = buf->off;
+
+	for (;;) {
+		lines[(cur_line++) % ERR_LINES] = buf->json + cur_off;
+
+		while (cur_off < buf->len && buf->json[cur_off] != '\n')
+			cur_off++;
+
+		if (cur_off >= buf->off)
+			break;
+
+		cur_off++;
+		last_off = buf->off - cur_off;
+	}
+
+	printf_line(buf, "%s at line %03zu", type, cur_line);
+	buf->err_print(buf->err_print_priv, "");
+
+	size_t idx = 0;
+
+	for (i = MIN(ERR_LINES, cur_line); i > 0; i--) {
+		idx = (cur_line - i) % ERR_LINES;
+		printf_json_line(buf, cur_line - i + 1, lines[idx]);
+	}
+
+	print_arrow(buf, lines[idx], last_off);
+}
+
+void ujson_err_print(ujson_reader *buf)
+{
+	if (!buf->err_print)
+		return;
+
+	print_snippet(buf, "Parse error");
+	buf->err_print(buf->err_print_priv, buf->err);
+}
+
+void ujson_warn(ujson_reader *buf, const char *fmt, ...)
+{
+	va_list va;
+
+	if (buf->flags & UJSON_READER_STRICT) {
+		va_start(va, fmt);
+		ujson_err_va(buf, fmt, va);
+		va_end(va);
+		return;
+	}
+
+	if (!buf->err_print)
+		return;
+
+	print_snippet(buf, "Warning");
+
+	va_start(va, fmt);
+	vprintf_line(buf, fmt, va);
+	va_end(va);
+}
+
+void ujson_print(void *err_print_priv, const char *line)
+{
+	fputs(line, err_print_priv);
+	putc('\n', err_print_priv);
+}
+
+ujson_reader *ujson_reader_load(const char *path)
+{
+	int fd = open(path, O_RDONLY);
+	ujson_reader *ret;
+	ssize_t res;
+	off_t len, off = 0;
+
+	if (fd < 0)
+		return NULL;
+
+	len = lseek(fd, 0, SEEK_END);
+	if (len == (off_t)-1) {
+		fprintf(stderr, "lseek() failed\n");
+		goto err0;
+	}
+
+	if (lseek(fd, 0, SEEK_SET) == (off_t)-1) {
+		fprintf(stderr, "lseek() failed\n");
+		goto err0;
+	}
+
+	ret = malloc(sizeof(ujson_reader) + len + 1);
+	if (!ret) {
+		fprintf(stderr, "malloc() failed\n");
+		goto err0;
+	}
+
+	memset(ret, 0, sizeof(*ret));
+
+	ret->buf[len] = 0;
+	ret->len = len;
+	ret->max_depth = UJSON_RECURSION_MAX;
+	ret->json = ret->buf;
+	ret->err_print = UJSON_ERR_PRINT;
+	ret->err_print_priv = UJSON_ERR_PRINT_PRIV;
+
+	while (off < len) {
+		res = read(fd, ret->buf + off, len - off);
+		if (res < 0) {
+			fprintf(stderr, "read() failed\n");
+			goto err1;
+		}
+
+		off += res;
+	}
+
+	close(fd);
+
+	return ret;
+err1:
+	free(ret);
+err0:
+	close(fd);
+	return NULL;
+}
+
+void ujson_reader_finish(ujson_reader *self)
+{
+	if (ujson_reader_err(self))
+		ujson_err_print(self);
+	else if (!ujson_reader_consumed(self))
+		ujson_warn(self, "Garbage after JSON string!");
+}
+
+void ujson_reader_free(ujson_reader *buf)
+{
+	free(buf);
+}
+
+ujson_val *ujson_val_alloc(size_t buf_size)
+{
+	buf_size = buf_size == 0 ? 4096 : buf_size;
+	ujson_val *ret;
+
+	ret = malloc(sizeof(ujson_val) + buf_size);
+	if (!ret)
+		return NULL;
+
+	memset(ret, 0, sizeof(ujson_val) + buf_size);
+
+	ret->buf = ret->buf__;
+	ret->buf_size = buf_size;
+
+	return ret;
+}
+
+void ujson_val_free(ujson_val *self)
+{
+	free(self);
+}
diff --git a/libs/ujson/ujson_utf.c b/libs/ujson/ujson_utf.c
new file mode 100644
index 000000000..2c08a39a8
--- /dev/null
+++ b/libs/ujson/ujson_utf.c
@@ -0,0 +1,105 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2022-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+#include <stddef.h>
+#include <ujson_utf.h>
+
+int8_t ujson_utf8_next_chsz(const char *str, size_t off)
+{
+	char ch = str[off];
+	uint8_t len = 0;
+
+	if (!ch)
+		return 0;
+
+	if (UJSON_UTF8_IS_ASCII(ch))
+		return 1;
+
+	if (UJSON_UTF8_IS_2BYTE(ch)) {
+		len = 2;
+		goto ret;
+	}
+
+	if (UJSON_UTF8_IS_3BYTE(ch)) {
+		len = 3;
+		goto ret;
+	}
+
+	if (UJSON_UTF8_IS_4BYTE(ch)) {
+		len = 4;
+		goto ret;
+	}
+
+	return -1;
+ret:
+	if (!UJSON_UTF8_IS_NBYTE(str[off+1]))
+		return -1;
+
+	if (len > 2 && !UJSON_UTF8_IS_NBYTE(str[off+2]))
+		return -1;
+
+	if (len > 3 && !UJSON_UTF8_IS_NBYTE(str[off+3]))
+		return -1;
+
+	return len;
+}
+
+int8_t ujson_utf8_prev_chsz(const char *str, size_t off)
+{
+	char ch;
+
+	if (!off)
+		return 0;
+
+	ch = str[--off];
+
+	if (UJSON_UTF8_IS_ASCII(ch))
+		return 1;
+
+	if (!UJSON_UTF8_IS_NBYTE(ch))
+		return -1;
+
+	if (off < 1)
+		return -1;
+
+	ch = str[--off];
+
+	if (UJSON_UTF8_IS_2BYTE(ch))
+		return 2;
+
+	if (!UJSON_UTF8_IS_NBYTE(ch))
+		return -1;
+
+	if (off < 1)
+		return -1;
+
+	ch = str[--off];
+
+	if (UJSON_UTF8_IS_3BYTE(ch))
+		return 3;
+
+	if (!UJSON_UTF8_IS_NBYTE(ch))
+		return -1;
+
+	if (off < 1)
+		return -1;
+
+	ch = str[--off];
+
+	if (UJSON_UTF8_IS_4BYTE(ch))
+		return 4;
+
+	return -1;
+}
+
+size_t ujson_utf8_strlen(const char *str)
+{
+	size_t cnt = 0;
+
+	while (ujson_utf8_next(&str))
+		cnt++;
+
+	return cnt;
+}
diff --git a/libs/ujson/ujson_writer.c b/libs/ujson/ujson_writer.c
new file mode 100644
index 000000000..6275be1ff
--- /dev/null
+++ b/libs/ujson/ujson_writer.c
@@ -0,0 +1,491 @@
+// SPDX-License-Identifier: LGPL-2.1-or-later
+/*
+ * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
+ */
+
+#include <string.h>
+#include <stdarg.h>
+#include <fcntl.h>
+#include <errno.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+
+#include "ujson_utf.h"
+#include "ujson_writer.h"
+
+static inline int get_depth_bit(ujson_writer *self, char *mask)
+{
+	int depth = self->depth - 1;
+
+	if (depth < 0)
+		return -1;
+
+	return !!(mask[depth/8] & (1<<(depth%8)));
+}
+
+static inline void set_depth_bit(ujson_writer *self, int val)
+{
+	if (val)
+		self->depth_type[self->depth/8] |= (1<<(self->depth%8));
+	else
+		self->depth_type[self->depth/8] &= ~(1<<(self->depth%8));
+
+	self->depth_first[self->depth/8] |= (1<<(self->depth%8));
+
+	self->depth++;
+}
+
+static inline void clear_depth_bit(ujson_writer *self)
+{
+	self->depth--;
+}
+
+static inline int in_arr(ujson_writer *self)
+{
+	return !get_depth_bit(self, self->depth_type);
+}
+
+static inline int in_obj(ujson_writer *self)
+{
+	return get_depth_bit(self, self->depth_type);
+}
+
+static inline void clear_depth_first(ujson_writer *self)
+{
+	int depth = self->depth - 1;
+
+	self->depth_first[depth/8] &= ~(1<<(depth%8));
+}
+
+static inline int is_first(ujson_writer *self)
+{
+	int ret = get_depth_bit(self, self->depth_first);
+
+	if (ret == 1)
+		clear_depth_first(self);
+
+	return ret;
+}
+
+static inline void err(ujson_writer *buf, const char *fmt, ...)
+{
+	va_list va;
+
+	va_start(va, fmt);
+	vsnprintf(buf->err, UJSON_ERR_MAX, fmt, va);
+	va_end(va);
+}
+
+static inline int is_err(ujson_writer *buf)
+{
+	return buf->err[0];
+}
+
+static inline int out(ujson_writer *self, const char *buf, size_t len)
+{
+	return self->out(self, buf, len);
+}
+
+static inline int out_str(ujson_writer *self, const char *str)
+{
+	return out(self, str, strlen(str));
+}
+
+static inline int out_ch(ujson_writer *self, char ch)
+{
+	return out(self, &ch, 1);
+}
+
+#define ESC_FLUSH(esc_char) do {\
+	out(self, val, i); \
+	val += i + 1; \
+	i = 0; \
+	out_str(self, esc_char); \
+} while (0)
+
+static inline int out_esc_str(ujson_writer *self, const char *val)
+{
+	if (out_ch(self, '"'))
+		return 1;
+
+	size_t i = 0;
+	int8_t next_chsz;
+
+	do {
+		next_chsz = ujson_utf8_next_chsz(val, i);
+
+		if (next_chsz == 1) {
+			switch (val[i]) {
+			case '\"':
+				ESC_FLUSH("\\\"");
+			break;
+			case '\\':
+				ESC_FLUSH("\\\\");
+			break;
+			case '/':
+				ESC_FLUSH("\\/");
+			break;
+			case '\b':
+				ESC_FLUSH("\\b");
+			break;
+			case '\f':
+				ESC_FLUSH("\\f");
+			break;
+			case '\n':
+				ESC_FLUSH("\\n");
+			break;
+			case '\r':
+				ESC_FLUSH("\\r");
+			break;
+			case '\t':
+				ESC_FLUSH("\\t");
+			break;
+			default:
+				i += next_chsz;
+			}
+		} else {
+			i += next_chsz;
+		}
+	} while (next_chsz);
+
+	if (i) {
+		if (out(self, val, i))
+			return 1;
+	}
+
+	if (out_ch(self, '"'))
+		return 1;
+
+	return 0;
+}
+
+static int do_padd(ujson_writer *self)
+{
+	unsigned int i;
+
+	for (i = 0; i < self->depth; i++) {
+		if (out_ch(self, ' '))
+			return 1;
+	}
+
+	return 0;
+}
+
+static int newline(ujson_writer *self)
+{
+	if (out_ch(self, '\n'))
+		return 0;
+
+	if (do_padd(self))
+		return 1;
+
+	return 0;
+}
+
+static int add_common(ujson_writer *self, const char *id)
+{
+	if (is_err(self))
+		return 1;
+
+	if (!self->depth) {
+		err(self, "Object/Array has to be started first");
+		return 1;
+	}
+
+	if (in_arr(self)) {
+		if (id) {
+			err(self, "Array entries can't have id");
+			return 1;
+		}
+	} else {
+		if (!id) {
+			err(self, "Object entries must have id");
+			return 1;
+		}
+	}
+
+	if (!is_first(self) && out_ch(self, ','))
+		return 1;
+
+	if (self->depth && newline(self))
+		return 1;
+
+	if (id) {
+		if (out_esc_str(self, id))
+			return 1;
+
+		if (out_str(self, ": "))
+			return 1;
+	}
+
+	return 0;
+}
+
+int ujson_obj_start(ujson_writer *self, const char *id)
+{
+	if (self->depth >= UJSON_RECURSION_MAX)
+		return 1;
+
+	if (!self->depth && id) {
+		err(self, "Top level object cannot have id");
+		return 1;
+	}
+
+	if (self->depth && add_common(self, id))
+		return 1;
+
+	if (out_ch(self, '{'))
+		return 1;
+
+	set_depth_bit(self, 1);
+
+	return 0;
+}
+
+int ujson_obj_finish(ujson_writer *self)
+{
+	if (is_err(self))
+		return 1;
+
+	if (!in_obj(self)) {
+		err(self, "Not in object!");
+		return 1;
+	}
+
+	int first = is_first(self);
+
+	clear_depth_bit(self);
+
+	if (!first)
+		newline(self);
+
+	return out_ch(self, '}');
+}
+
+int ujson_arr_start(ujson_writer *self, const char *id)
+{
+	if (self->depth >= UJSON_RECURSION_MAX) {
+		err(self, "Recursion too deep");
+		return 1;
+	}
+
+	if (!self->depth && id) {
+		err(self, "Top level array cannot have id");
+		return 1;
+	}
+
+	if (self->depth && add_common(self, id))
+		return 1;
+
+	if (out_ch(self, '['))
+		return 1;
+
+	set_depth_bit(self, 0);
+
+	return 0;
+}
+
+int ujson_arr_finish(ujson_writer *self)
+{
+	if (is_err(self))
+		return 1;
+
+	if (!in_arr(self)) {
+		err(self, "Not in array!");
+		return 1;
+	}
+
+	int first = is_first(self);
+
+	clear_depth_bit(self);
+
+	if (!first)
+		newline(self);
+
+	return out_ch(self, ']');
+}
+
+int ujson_null_add(ujson_writer *self, const char *id)
+{
+	if (add_common(self, id))
+		return 1;
+
+	return out_str(self, "null");
+}
+
+int ujson_int_add(ujson_writer *self, const char *id, long val)
+{
+	char buf[64];
+
+	if (add_common(self, id))
+		return 1;
+
+	snprintf(buf, sizeof(buf), "%li", val);
+
+	return out_str(self, buf);
+}
+
+int ujson_bool_add(ujson_writer *self, const char *id, int val)
+{
+	if (add_common(self, id))
+		return 1;
+
+	if (val)
+		return out_str(self, "true");
+	else
+		return out_str(self, "false");
+}
+
+int ujson_str_add(ujson_writer *self, const char *id, const char *val)
+{
+	if (add_common(self, id))
+		return 1;
+
+	if (out_esc_str(self, val))
+		return 1;
+
+	return 0;
+}
+
+int ujson_float_add(ujson_writer *self, const char *id, double val)
+{
+	char buf[64];
+
+	if (add_common(self, id))
+		return 1;
+
+	snprintf(buf, sizeof(buf), "%lg", val);
+
+	return out_str(self, buf);
+}
+
+int ujson_writer_finish(ujson_writer *self)
+{
+	if (is_err(self))
+		goto err;
+
+	if (self->depth) {
+		err(self, "Objects and/or Arrays not finished");
+		goto err;
+	}
+
+	if (newline(self))
+		return 1;
+
+	return 0;
+err:
+	if (self->err_print)
+		self->err_print(self->err_print_priv, self->err);
+
+	return 1;
+}
+
+struct json_writer_file {
+	int fd;
+	size_t buf_used;
+	char buf[1024];
+};
+
+static int out_writer_file_write(ujson_writer *self, int fd, const char *buf, ssize_t buf_len)
+{
+	do {
+		ssize_t ret = write(fd, buf, buf_len);
+		if (ret <= 0) {
+			err(self, "Failed to write to a file");
+			return 1;
+		}
+
+		if (ret > buf_len) {
+			err(self, "Wrote more bytes than requested?!");
+			return 1;
+		}
+
+		buf_len -= ret;
+	} while (buf_len);
+
+	return 0;
+}
+
+static int out_writer_file(ujson_writer *self, const char *buf, size_t buf_len)
+{
+	struct json_writer_file *writer_file = self->out_priv;
+	size_t buf_size = sizeof(writer_file->buf);
+	size_t buf_avail = buf_size - writer_file->buf_used;
+
+	if (buf_len > buf_size/4)
+		return out_writer_file_write(self, writer_file->fd, buf, buf_len);
+
+	if (buf_len >= buf_avail) {
+		if (out_writer_file_write(self, writer_file->fd,
+		                          writer_file->buf, writer_file->buf_used))
+			return 1;
+
+		memcpy(writer_file->buf, buf, buf_len);
+		writer_file->buf_used = buf_len;
+		return 0;
+	}
+
+	memcpy(writer_file->buf + writer_file->buf_used, buf, buf_len);
+	writer_file->buf_used += buf_len;
+
+	return 0;
+}
+
+int ujson_writer_file_close(ujson_writer *self)
+{
+	struct json_writer_file *writer_file = self->out_priv;
+	int saved_errno = 0;
+
+	if (writer_file->buf_used) {
+		if (out_writer_file_write(self, writer_file->fd,
+		                          writer_file->buf, writer_file->buf_used))
+
+			saved_errno = errno;
+	}
+
+	if (close(writer_file->fd)) {
+		if (!saved_errno)
+			saved_errno = errno;
+	}
+
+	free(self);
+
+	if (saved_errno) {
+		errno = saved_errno;
+		return 1;
+	}
+
+	return 0;
+}
+
+ujson_writer *ujson_writer_file_open(const char *path)
+{
+	ujson_writer *ret;
+	struct json_writer_file *writer_file;
+
+	ret = malloc(sizeof(ujson_writer) + sizeof(struct json_writer_file));
+	if (!ret)
+		return NULL;
+
+	writer_file = (void*)ret + sizeof(ujson_writer);
+
+	writer_file->fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0664);
+	if (!writer_file->fd) {
+		free(ret);
+		return NULL;
+	}
+
+	writer_file->buf_used = 0;
+
+	memset(ret, 0, sizeof(*ret));
+
+	ret->err_print = UJSON_ERR_PRINT;
+	ret->err_print_priv = UJSON_ERR_PRINT_PRIV;
+	ret->out = out_writer_file;
+	ret->out_priv = writer_file;
+
+	return ret;
+}
+
+
-- 
2.44.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 3/4] libs: Vendor ujson library

[Andrea Cervesato via ltp]

Hi!

would be nice to use this library also for C tests metadata later on.

Reviewed-by: Andrea Cervesato <andrea.cervesato@suse.com>

On 8/27/24 14:02, Cyril Hrubis wrote:
> See: https://github.com/metan-ucw/ujson
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> Acked-by: Richard Palethorpe <io@richiejp.com>
> ---
>   include/ujson.h           |   13 +
>   include/ujson_common.h    |   69 +++
>   include/ujson_reader.h    |  543 +++++++++++++++++++
>   include/ujson_utf.h       |  168 ++++++
>   include/ujson_writer.h    |  224 ++++++++
>   libs/ujson/Makefile       |   12 +
>   libs/ujson/ujson_common.c |   38 ++
>   libs/ujson/ujson_reader.c | 1081 +++++++++++++++++++++++++++++++++++++
>   libs/ujson/ujson_utf.c    |  105 ++++
>   libs/ujson/ujson_writer.c |  491 +++++++++++++++++
>   10 files changed, 2744 insertions(+)
>   create mode 100644 include/ujson.h
>   create mode 100644 include/ujson_common.h
>   create mode 100644 include/ujson_reader.h
>   create mode 100644 include/ujson_utf.h
>   create mode 100644 include/ujson_writer.h
>   create mode 100644 libs/ujson/Makefile
>   create mode 100644 libs/ujson/ujson_common.c
>   create mode 100644 libs/ujson/ujson_reader.c
>   create mode 100644 libs/ujson/ujson_utf.c
>   create mode 100644 libs/ujson/ujson_writer.c
>
> diff --git a/include/ujson.h b/include/ujson.h
> new file mode 100644
> index 000000000..8faeb18f0
> --- /dev/null
> +++ b/include/ujson.h
> @@ -0,0 +1,13 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +#ifndef UJSON_H
> +#define UJSON_H
> +
> +#include <ujson_common.h>
> +#include <ujson_reader.h>
> +#include <ujson_writer.h>
> +
> +#endif /* UJSON_H */
> diff --git a/include/ujson_common.h b/include/ujson_common.h
> new file mode 100644
> index 000000000..ed31c090d
> --- /dev/null
> +++ b/include/ujson_common.h
> @@ -0,0 +1,69 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +/**
> + * @file ujson_common.h
> + * @brief Common JSON reader/writer definitions.
> + */
> +
> +#ifndef UJSON_COMMON_H
> +#define UJSON_COMMON_H
> +
> +/** @brief Maximal error message length. */
> +#define UJSON_ERR_MAX 128
> +/** @brief Maximal id string lenght including terminating null element. */
> +#define UJSON_ID_MAX 64
> +/** @brief Maximal recursion depth allowed. */
> +#define UJSON_RECURSION_MAX 128
> +
> +#define UJSON_ERR_PRINT ujson_err_handler
> +#define UJSON_ERR_PRINT_PRIV stderr
> +
> +/**
> + * @brief A JSON data type.
> + */
> +enum ujson_type {
> +	/** @brief No type. Returned when parser finishes. */
> +	UJSON_VOID = 0,
> +	/** @brief An integer. */
> +	UJSON_INT,
> +	/** @brief A floating point. */
> +	UJSON_FLOAT,
> +	/** @brief A boolean. */
> +	UJSON_BOOL,
> +	/** @brief NULL */
> +	UJSON_NULL,
> +	/** @brief A string. */
> +	UJSON_STR,
> +	/** @brief A JSON object. */
> +	UJSON_OBJ,
> +	/** @brief A JSON array. */
> +	UJSON_ARR,
> +};
> +
> +/**
> + * @brief Returns type name.
> + *
> + * @param type A json type.
> + * @return A type name.
> + */
> +const char *ujson_type_name(enum ujson_type type);
> +
> +/**
> + * @brief Default error print handler.
> + *
> + * @param print_priv A json buffer print_priv pointer.
> + * @param line A line of text to be printed.
> + */
> +void ujson_err_handler(void *print_priv, const char *line);
> +
> +typedef struct ujson_reader ujson_reader;
> +typedef struct ujson_writer ujson_writer;
> +typedef struct ujson_val ujson_val;
> +
> +/** @brief An array size macro. */
> +#define UJSON_ARRAY_SIZE(array) (sizeof(array) / sizeof(*array))
> +
> +#endif /* UJSON_COMMON_H */
> diff --git a/include/ujson_reader.h b/include/ujson_reader.h
> new file mode 100644
> index 000000000..273fe624a
> --- /dev/null
> +++ b/include/ujson_reader.h
> @@ -0,0 +1,543 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +/**
> + * @file ujson_reader.h
> + * @brief A recursive descend JSON parser.
> + *
> + * All the function that parse JSON return zero on success and non-zero on a
> + * failure. Once an error has happened all subsequent attempts to parse more
> + * return with non-zero exit status immediatelly. This is designed so that we
> + * can parse several values without checking each return value and only check
> + * if error has happened at the end of the sequence.
> + */
> +
> +#ifndef UJSON_READER_H
> +#define UJSON_READER_H
> +
> +#include <stdio.h>
> +#include <ujson_common.h>
> +
> +/**
> + * @brief An ujson_reader initializer with default values.
> + *
> + * @param buf A pointer to a buffer with JSON data.
> + * @param buf_len A JSON data buffer lenght.
> + * @param rflags enum ujson_reader_flags.
> + *
> + * @return An ujson_reader initialized with default values.
> + */
> +#define UJSON_READER_INIT(buf, buf_len, rflags) { \
> +	.max_depth = UJSON_RECURSION_MAX, \
> +	.err_print = UJSON_ERR_PRINT, \
> +	.err_print_priv = UJSON_ERR_PRINT_PRIV, \
> +	.json = buf, \
> +	.len = buf_len, \
> +	.flags = rflags \
> +}
> +
> +/** @brief Reader flags. */
> +enum ujson_reader_flags {
> +	/** @brief If set warnings are treated as errors. */
> +	UJSON_READER_STRICT = 0x01,
> +};
> +
> +/**
> + * @brief A JSON parser internal state.
> + */
> +struct ujson_reader {
> +	/** Pointer to a null terminated JSON string */
> +	const char *json;
> +	/** A length of the JSON string */
> +	size_t len;
> +	/** A current offset into the JSON string */
> +	size_t off;
> +	/** An offset to the start of the last array or object */
> +	size_t sub_off;
> +	/** Recursion depth increased when array/object is entered decreased on leave */
> +	unsigned int depth;
> +	/** Maximal recursion depth */
> +	unsigned int max_depth;
> +
> +	/** Reader flags. */
> +	enum ujson_reader_flags flags;
> +
> +	/** Handler to print errors and warnings */
> +	void (*err_print)(void *err_print_priv, const char *line);
> +	void *err_print_priv;
> +
> +	char err[UJSON_ERR_MAX];
> +	char buf[];
> +};
> +
> +/**
> + * @brief An ujson_val initializer.
> + *
> + * @param sbuf A pointer to a buffer used for string values.
> + * @param sbuf_size A length of the buffer used for string values.
> + *
> + * @return An ujson_val initialized with default values.
> + */
> +#define UJSON_VAL_INIT(sbuf, sbuf_size) { \
> +	.buf = sbuf, \
> +	.buf_size = sbuf_size, \
> +}
> +
> +/**
> + * @brief A parsed JSON key value pair.
> + */
> +struct ujson_val {
> +	/**
> +	 * @brief A value type
> +	 *
> +	 * UJSON_VALUE_VOID means that no value was parsed.
> +	 */
> +	enum ujson_type type;
> +
> +	/** An user supplied buffer and size to store a string values to. */
> +	char *buf;
> +	size_t buf_size;
> +
> +	/**
> +	 * @brief An index to attribute list.
> +	 *
> +	 * This is set by the ujson_obj_first_filter() and
> +	 * ujson_obj_next_filter() functions.
> +	 */
> +	size_t idx;
> +
> +	/** An union to store the parsed value into. */
> +	union {
> +		/** @brief A boolean value. */
> +		int val_bool;
> +		/** @brief An integer value. */
> +		long long val_int;
> +		/** @brief A string value. */
> +		const char *val_str;
> +	};
> +
> +	/**
> +	 * @brief A floating point value.
> +	 *
> +	 * Since integer values are subset of floating point values val_float
> +	 * is always set when val_int was set.
> +	 */
> +	double val_float;
> +
> +	/** @brief An ID for object values */
> +	char id[UJSON_ID_MAX];
> +
> +	char buf__[];
> +};
> +
> +/**
> + * @brief Allocates a JSON value.
> + *
> + * @param buf_size A maximal buffer size for a string value, pass 0 for default.
> + * @return A newly allocated JSON value.
> + */
> +ujson_val *ujson_val_alloc(size_t buf_size);
> +
> +/**
> + * @brief Frees a JSON value.
> + *
> + * @param self A JSON value previously allocated by ujson_val_alloc().
> + */
> +void ujson_val_free(ujson_val *self);
> +
> +/**
> + * @brief Checks is result has valid type.
> + *
> + * @param res An ujson value.
> + * @return Zero if result is not valid, non-zero otherwise.
> + */
> +static inline int ujson_val_valid(struct ujson_val *res)
> +{
> +	return !!res->type;
> +}
> +
> +/**
> + * @brief Fills the reader error.
> + *
> + * Once buffer error is set all parsing functions return immediatelly with type
> + * set to UJSON_VOID.
> + *
> + * @param self An ujson_reader
> + * @param fmt A printf like format string
> + * @param ... A printf like parameters
> + */
> +void ujson_err(ujson_reader *self, const char *fmt, ...)
> +               __attribute__((format(printf, 2, 3)));
> +
> +/**
> + * @brief Prints error stored in the buffer.
> + *
> + * The error takes into consideration the current offset in the buffer and
> + * prints a few preceding lines along with the exact position of the error.
> + *
> + * The error is passed to the err_print() handler.
> + *
> + * @param self A ujson_reader
> + */
> +void ujson_err_print(ujson_reader *self);
> +
> +/**
> + * @brief Prints a warning.
> + *
> + * Uses the print handler in the buffer to print a warning along with a few
> + * lines of context from the JSON at the current position.
> + *
> + * @param self A ujson_reader
> + * @param fmt A printf-like error string.
> + * @param ... A printf-like parameters.
> + */
> +void ujson_warn(ujson_reader *self, const char *fmt, ...)
> +               __attribute__((format(printf, 2, 3)));
> +
> +/**
> + * @brief Returns true if error was encountered.
> + *
> + * @param self A ujson_reader
> + * @return True if error was encountered false otherwise.
> + */
> +static inline int ujson_reader_err(ujson_reader *self)
> +{
> +	return !!self->err[0];
> +}
> +
> +/**
> + * @brief Returns the type of next element in buffer.
> + *
> + * @param self An ujson_reader
> + * @return A type of next element in the buffer.
> + */
> +enum ujson_type ujson_next_type(ujson_reader *self);
> +
> +/**
> + * @brief Returns if first element in JSON is object or array.
> + *
> + * @param self A ujson_reader
> + * @return On success returns UJSON_OBJ or UJSON_ARR. On failure UJSON_VOID.
> + */
> +enum ujson_type ujson_reader_start(ujson_reader *self);
> +
> +/**
> + * @brief Starts parsing of a JSON object.
> + *
> + * @param self An ujson_reader
> + * @param res An ujson_val to store the parsed value to.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_obj_first(ujson_reader *self, struct ujson_val *res);
> +
> +/**
> + * @brief Parses next value from a JSON object.
> + *
> + * If the res->type is UJSON_OBJ or UJSON_ARR it has to be parsed or skipped
> + * before next call to this function.
> + *
> + * @param self An ujson_reader.
> + * @param res A ujson_val to store the parsed value to.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_obj_next(ujson_reader *self, struct ujson_val *res);
> +
> +/**
> + * @brief A loop over a JSON object.
> + *
> + * @code
> + * UJSON_OBJ_FOREACH(reader, val) {
> + *	printf("Got value id '%s' type '%s'", val->id, ujson_type_name(val->type));
> + *	...
> + * }
> + * @endcode
> + *
> + * @param self An ujson_reader.
> + * @param res An ujson_val to store the next parsed value to.
> + */
> +#define UJSON_OBJ_FOREACH(self, res) \
> +	for (ujson_obj_first(self, res); ujson_val_valid(res); ujson_obj_next(self, res))
> +
> +/**
> + * @brief Utility function for log(n) lookup in a sorted array.
> + *
> + * @param list Analphabetically sorted array.
> + * @param list_len Array length.
> + *
> + * @return An array index or (size_t)-1 if key wasn't found.
> + */
> +size_t ujson_lookup(const void *arr, size_t memb_size, size_t list_len,
> +                    const char *key);
> +
> +/**
> + * @brief A JSON object attribute description i.e. key and type.
> + */
> +typedef struct ujson_obj_attr {
> +	/** @brief A JSON object key name. */
> +	const char *key;
> +	/**
> +	 * @brief A JSON object value type.
> +	 *
> +	 * Note that because integer numbers are subset of floating point
> +         * numbers if requested type was UJSON_FLOAT it will match if parsed
> +         * type was UJSON_INT and the val_float will be set in addition to
> +         * val_int.
> +         */
> +	enum ujson_type type;
> +} ujson_obj_attr;
> +
> +/** @brief A JSON object description */
> +typedef struct ujson_obj {
> +	/**
> +	 * @brief A list of attributes.
> +	 *
> +	 * Attributes we are looking for, the parser sets the val->idx for these.
> +	 */
> +	const ujson_obj_attr *attrs;
> +	/** @brief A size of attrs array. */
> +	size_t attr_cnt;
> +} ujson_obj;
> +
> +static inline size_t ujson_obj_lookup(const ujson_obj *obj, const char *key)
> +{
> +	return ujson_lookup(obj->attrs, sizeof(*obj->attrs), obj->attr_cnt, key);
> +}
> +
> +/** @brief An ujson_obj_attr initializer. */
> +#define UJSON_OBJ_ATTR(keyv, typev) \
> +	{.key = keyv, .type = typev}
> +
> +/** @brief An ujson_obj_attr intializer with an array index. */
> +#define UJSON_OBJ_ATTR_IDX(key_idx, keyv, typev) \
> +	[key_idx] = {.key = keyv, .type = typev}
> +
> +/**
> + * @brief Starts parsing of a JSON object with attribute lists.
> + *
> + * @param self An ujson_reader.
> + * @param res An ujson_val to store the parsed value to.
> + * @param obj An ujson_obj object description.
> + * @param ign A list of keys to ignore.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_obj_first_filter(ujson_reader *self, struct ujson_val *res,
> +                           const struct ujson_obj *obj, const struct ujson_obj *ign);
> +
> +/**
> + * @brief An empty object attribute list.
> + *
> + * To be passed to UJSON_OBJ_FOREACH_FITLER() as ignore list.
> + */
> +extern const struct ujson_obj *ujson_empty_obj;
> +
> +/**
> + * @brief Parses next value from a JSON object with attribute lists.
> + *
> + * If the res->type is UJSON_OBJ or UJSON_ARR it has to be parsed or skipped
> + * before next call to this function.
> + *
> + * @param self An ujson_reader.
> + * @param res An ujson_val to store the parsed value to.
> + * @param obj An ujson_obj object description.
> + * @param ign A list of keys to ignore. If set to NULL all unknown keys are
> + *            ignored, if set to ujson_empty_obj all unknown keys produce warnings.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_obj_next_filter(ujson_reader *self, struct ujson_val *res,
> +                          const struct ujson_obj *obj, const struct ujson_obj *ign);
> +
> +/**
> + * @brief A loop over a JSON object with a pre-defined list of expected attributes.
> + *
> + * @code
> + * static struct ujson_obj_attr attrs[] = {
> + *	UJSON_OBJ_ATTR("bool", UJSON_BOOL),
> + *	UJSON_OBJ_ATTR("number", UJSON_INT),
> + * };
> + *
> + * static struct ujson_obj obj = {
> + *	.attrs = filter_attrs,
> + *	.attr_cnt = UJSON_ARRAY_SIZE(filter_attrs)
> + * };
> + *
> + * UJSON_OBJ_FOREACH_FILTER(reader, val, &obj, NULL) {
> + *	printf("Got value id '%s' type '%s'",
> + *	       attrs[val->idx].id, ujson_type_name(val->type));
> + *	...
> + * }
> + * @endcode
> + *
> + * @param self An ujson_reader.
> + * @param res An ujson_val to store the next parsed value to.
> + * @param obj An ujson_obj with a description of attributes to parse.
> + * @param ign An ujson_obj with a description of attributes to ignore.
> + */
> +#define UJSON_OBJ_FOREACH_FILTER(self, res, obj, ign) \
> +	for (ujson_obj_first_filter(self, res, obj, ign); \
> +	     ujson_val_valid(res); \
> +	     ujson_obj_next_filter(self, res, obj, ign))
> +
> +/**
> + * @brief Skips parsing of a JSON object.
> + *
> + * @param self An ujson_reader.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_obj_skip(ujson_reader *self);
> +
> +/**
> + * @brief Starts parsing of a JSON array.
> + *
> + * @param self An ujson_reader.
> + * @param res An ujson_val to store the parsed value to.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_arr_first(ujson_reader *self, struct ujson_val *res);
> +
> +/**
> + * @brief Parses next value from a JSON array.
> + *
> + * If the res->type is UJSON_OBJ or UJSON_ARR it has to be parsed or skipped
> + * before next call to this function.
> + *
> + * @param self An ujson_reader.
> + * @param res An ujson_val to store the parsed value to.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_arr_next(ujson_reader *self, struct ujson_val *res);
> +
> +/**
> + * @brief A loop over a JSON array.
> + *
> + * @code
> + * UJSON_ARR_FOREACH(reader, val) {
> + *	printf("Got value type '%s'", ujson_type_name(val->type));
> + *	...
> + * }
> + * @endcode
> + *
> + * @param self An ujson_reader.
> + * @param res An ujson_val to store the next parsed value to.
> + */
> +#define UJSON_ARR_FOREACH(self, res) \
> +	for (ujson_arr_first(self, res); ujson_val_valid(res); ujson_arr_next(self, res))
> +
> +/**
> + * @brief Skips parsing of a JSON array.
> + *
> + * @param self A ujson_reader.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_arr_skip(ujson_reader *self);
> +
> +/**
> + * @brief A JSON reader state.
> + */
> +typedef struct ujson_reader_state {
> +	size_t off;
> +	unsigned int depth;
> +} ujson_reader_state;
> +
> +/**
> + * @brief Returns a parser state at the start of current object/array.
> + *
> + * This function could be used for the parser to return to the start of the
> + * currently parsed object or array.
> + *
> + * @param self A ujson_reader
> + * @return A state that points to a start of the last object or array.
> + */
> +static inline ujson_reader_state ujson_reader_state_save(ujson_reader *self)
> +{
> +	struct ujson_reader_state ret = {
> +		.off = self->sub_off,
> +		.depth = self->depth,
> +	};
> +
> +	return ret;
> +}
> +
> +/**
> + * @brief Returns the parser to a saved state.
> + *
> + * This function could be used for the parser to return to the start of
> + * object or array saved by t the ujson_reader_state_get() function.
> + *
> + * @param self A ujson_reader
> + * @param state An parser state as returned by the ujson_reader_state_get().
> + */
> +static inline void ujson_reader_state_load(ujson_reader *self, ujson_reader_state state)
> +{
> +	if (ujson_reader_err(self))
> +		return;
> +
> +	self->off = state.off;
> +	self->sub_off = state.off;
> +	self->depth = state.depth;
> +}
> +
> +/**
> + * @brief Resets the parser to a start.
> + *
> + * @param self A ujson_reader
> + */
> +static inline void ujson_reader_reset(ujson_reader *self)
> +{
> +	self->off = 0;
> +	self->sub_off = 0;
> +	self->depth = 0;
> +	self->err[0] = 0;
> +}
> +
> +/**
> + * @brief Loads a file into an ujson_reader buffer.
> + *
> + * The reader has to be later freed by ujson_reader_free().
> + *
> + * @param path A path to a file.
> + * @return A ujson_reader or NULL in a case of a failure.
> + */
> +ujson_reader *ujson_reader_load(const char *path);
> +
> +/**
> + * @brief Frees an ujson_reader buffer.
> + *
> + * @param self A ujson_reader allocated by ujson_reader_load() function.
> + */
> +void ujson_reader_free(ujson_reader *self);
> +
> +/**
> + * @brief Prints errors and warnings at the end of parsing.
> + *
> + * Checks if self->err is set and prints the error with ujson_reader_err()
> + *
> + * Checks if there is any text left after the parser has finished with
> + * ujson_reader_consumed() and prints a warning if there were any non-whitespace
> + * characters left.
> + *
> + * @param self A ujson_reader
> + */
> +void ujson_reader_finish(ujson_reader *self);
> +
> +/**
> + * @brief Returns non-zero if whole buffer has been consumed.
> + *
> + * @param self A ujson_reader.
> + * @return Non-zero if whole buffer was consumed.
> + */
> +static inline int ujson_reader_consumed(ujson_reader *self)
> +{
> +	return self->off >= self->len;
> +}
> +
> +#endif /* UJSON_H */
> diff --git a/include/ujson_utf.h b/include/ujson_utf.h
> new file mode 100644
> index 000000000..f939fbe8c
> --- /dev/null
> +++ b/include/ujson_utf.h
> @@ -0,0 +1,168 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2022-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +/**
> + * @file ujson_utf.h
> + * @brief Unicode helper macros and functions.
> + */
> +
> +#ifndef UJSON_UTF_H
> +#define UJSON_UTF_H
> +
> +#include <stdint.h>
> +#include <stddef.h>
> +
> +/** Returns true if unicode byte is ASCII */
> +#define UJSON_UTF8_IS_ASCII(ch) (!((ch) & 0x80))
> +/** Returns true if we have first unicode byte of single byte sequence */
> +#define UJSON_UTF8_IS_NBYTE(ch) (((ch) & 0xc0) == 0x80)
> +/** Returns true if we have first unicode byte of two byte sequence */
> +#define UJSON_UTF8_IS_2BYTE(ch) (((ch) & 0xe0) == 0xc0)
> +/** Returns true if we have first unicode byte of three byte sequence */
> +#define UJSON_UTF8_IS_3BYTE(ch) (((ch) & 0xf0) == 0xe0)
> +/** Returns true if we have first unicode byte of four byte sequence */
> +#define UJSON_UTF8_IS_4BYTE(ch) (((ch) & 0xf8) == 0xf0)
> +
> +#define UJSON_UTF8_NBYTE_MASK 0x3f
> +
> +/**
> + * @brief Parses next unicode character in UTF-8 string.
> + * @param str A pointer to the C string.
> + * @return A unicode character or 0 on error or end of the string.
> + */
> +static inline uint32_t ujson_utf8_next(const char **str)
> +{
> +	uint32_t s0 = *str[0];
> +
> +	(*str)++;
> +
> +	if (UJSON_UTF8_IS_ASCII(s0))
> +		return s0;
> +
> +	uint32_t s1 = *str[0];
> +
> +	if (!UJSON_UTF8_IS_NBYTE(s1))
> +		return 0;
> +
> +	s1 &= UJSON_UTF8_NBYTE_MASK;
> +
> +	(*str)++;
> +
> +	if (UJSON_UTF8_IS_2BYTE(s0))
> +		return (s0 & 0x1f)<<6 | s1;
> +
> +	uint32_t s2 = *str[0];
> +
> +	if (!UJSON_UTF8_IS_NBYTE(s2))
> +		return 0;
> +
> +	s2 &= UJSON_UTF8_NBYTE_MASK;
> +
> +	(*str)++;
> +
> +	if (UJSON_UTF8_IS_3BYTE(s0))
> +		return (s0 & 0x0f)<<12 | s1<<6 | s2;
> +
> +	(*str)++;
> +
> +	uint32_t s3 = *str[0];
> +
> +	if (!UJSON_UTF8_IS_NBYTE(s2))
> +		return 0;
> +
> +	s3 &= UJSON_UTF8_NBYTE_MASK;
> +
> +	if (UJSON_UTF8_IS_4BYTE(s0))
> +		return (s0 & 0x07)<<18 | s1<<12 | s2<<6 | s3;
> +
> +	return 0;
> +}
> +
> +/**
> + * @brief Returns number of bytes next character is occupying in an UTF-8 string.
> + *
> + * @param str A pointer to a string.
> + * @param off An offset into the string, must point to a valid multibyte boundary.
> + * @return Number of bytes next character occupies, zero on string end and -1 on failure.
> + */
> +int8_t ujson_utf8_next_chsz(const char *str, size_t off);
> +
> +/**
> + * @brief Returns number of bytes previous character is occupying in an UTF-8 string.
> + *
> + * @param str A pointer to a string.
> + * @param off An offset into the string, must point to a valid multibyte boundary.
> + * @return Number of bytes previous character occupies, and -1 on failure.
> + */
> +int8_t ujson_utf8_prev_chsz(const char *str, size_t off);
> +
> +/**
> + * @brief Returns a number of characters in UTF-8 string.
> + *
> + * Returns number of characters in an UTF-8 string, which may be less or equal
> + * to what strlen() reports.
> + *
> + * @param str An UTF-8 string.
> + * @return Number of characters in the string.
> + */
> +size_t ujson_utf8_strlen(const char *str);
> +
> +/**
> + * @brief Returns a number of bytes needed to store unicode character into UTF-8.
> + *
> + * @param unicode A unicode character.
> + * @return Number of utf8 bytes required to store a unicode character.
> + */
> +static inline unsigned int ujson_utf8_bytes(uint32_t unicode)
> +{
> +	if (unicode < 0x0080)
> +		return 1;
> +
> +	if (unicode < 0x0800)
> +		return 2;
> +
> +	if (unicode < 0x10000)
> +		return 3;
> +
> +	return 4;
> +}
> +
> +/**
> + * @brief Writes an unicode character into a UTF-8 buffer.
> + *
> + * The buffer _must_ be large enough!
> + *
> + * @param unicode A unicode character.
> + * @param buf A byte buffer.
> + * @return A number of bytes written.
> + */
> +static inline int ujson_to_utf8(uint32_t unicode, char *buf)
> +{
> +	if (unicode < 0x0080) {
> +		buf[0] = unicode & 0x007f;
> +		return 1;
> +	}
> +
> +	if (unicode < 0x0800) {
> +		buf[0] = 0xc0 | (0x1f & (unicode>>6));
> +		buf[1] = 0x80 | (0x3f & unicode);
> +		return 2;
> +	}
> +
> +	if (unicode < 0x10000) {
> +		buf[0] = 0xe0 | (0x0f & (unicode>>12));
> +		buf[1] = 0x80 | (0x3f & (unicode>>6));
> +		buf[2] = 0x80 | (0x3f & unicode);
> +		return 3;
> +	}
> +
> +	buf[0] = 0xf0 | (0x07 & (unicode>>18));
> +	buf[1] = 0x80 | (0x3f & (unicode>>12));
> +	buf[2] = 0x80 | (0x3f & (unicode>>6));
> +	buf[3] = 0x80 | (0x3f & unicode);
> +	return 4;
> +}
> +
> +#endif /* UJSON_UTF_H */
> diff --git a/include/ujson_writer.h b/include/ujson_writer.h
> new file mode 100644
> index 000000000..dfcc95053
> --- /dev/null
> +++ b/include/ujson_writer.h
> @@ -0,0 +1,224 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +/**
> + * @file ujson_writer.h
> + * @brief A JSON writer.
> + *
> + * All the function that add values return zero on success and non-zero on a
> + * failure. Once an error has happened all subsequent attempts to add more
> + * values return with non-zero exit status immediatelly. This is designed
> + * so that we can add several values without checking each return value
> + * and only check if error has happened at the end of the sequence.
> + *
> + * Failures may occur:
> + * - if we call the functions out of order, e.g. attempt to finish array when
> + *   we are not writing out an array.
> + * - if we run out of recursion stack
> + * - may be propagated from the writer function, e.g. allocation failure, no
> + *   space on disk, etc.
> + */
> +
> +#ifndef UJSON_WRITER_H
> +#define UJSON_WRITER_H
> +
> +#include <ujson_common.h>
> +
> +/** @brief A JSON writer */
> +struct ujson_writer {
> +	unsigned int depth;
> +	char depth_type[UJSON_RECURSION_MAX/8];
> +	char depth_first[UJSON_RECURSION_MAX/8];
> +
> +	/** Handler to print errors and warnings */
> +	void (*err_print)(void *err_print_priv, const char *line);
> +	void *err_print_priv;
> +	char err[UJSON_ERR_MAX];
> +
> +	/** Handler to produce JSON output */
> +	int (*out)(struct ujson_writer *self, const char *buf, size_t buf_size);
> +	void *out_priv;
> +};
> +
> +/**
> + * @brief An ujson_writer initializer with default values.
> + *
> + * @param vout A pointer to function to write out the data.
> + * @param vout_priv An user pointer passed to the out function.
> + *
> + * @return An ujson_writer initialized with default values.
> + */
> +#define UJSON_WRITER_INIT(vout, vout_priv) { \
> +	.err_print = UJSON_ERR_PRINT, \
> +	.err_print_priv = UJSON_ERR_PRINT_PRIV, \
> +	.out = vout, \
> +	.out_priv = vout_priv \
> +}
> +
> +/**
> + * @brief Allocates a JSON file writer.
> + *
> + * The call may fail either when file cannot be opened for writing or if
> + * allocation has failed. In all cases errno should be set correctly.
> + *
> + * @param path A path to the file, file is opened for writing and created if it
> + *             does not exist.
> + *
> + * @return A ujson_writer pointer or NULL in a case of failure.
> + */
> +ujson_writer *ujson_writer_file_open(const char *path);
> +
> +/**
> + * @brief Closes and frees a JSON file writer.
> + *
> + * @param self A ujson_writer file writer.
> + *
> + * @return Zero on success, non-zero on a failure and errno is set.
> + */
> +int ujson_writer_file_close(ujson_writer *self);
> +
> +/**
> + * @brief Returns true if writer error happened.
> + *
> + * @param self A JSON writer.
> + *
> + * @return True if error has happened.
> + */
> +static inline int ujson_writer_err(ujson_writer *self)
> +{
> +	return !!self->err[0];
> +}
> +
> +/**
> + * @brief Starts a JSON object.
> + *
> + * For a top level object the id must be NULL, every other object has to have
> + * non-NULL id. The call will also fail if maximal recursion depth
> + * UJSON_RECURSION_MAX has been reached.
> + *
> + * @param self A JSON writer.
> + * @param id An object name.
> + *
> + * @return Zero on a success, non-zero otherwise.
> + */
> +int ujson_obj_start(ujson_writer *self, const char *id);
> +
> +/**
> + * @brief Finishes a JSON object.
> + *
> + * The call will fail if we are currenlty not writing out an object.
> + *
> + * @param self A JSON writer.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_obj_finish(ujson_writer *self);
> +
> +/**
> + * @brief Starts a JSON array.
> + *
> + * For a top level array the id must be NULL, every other array has to have
> + * non-NULL id. The call will also fail if maximal recursion depth
> + * UJSON_RECURSION_MAX has been reached.
> + *
> + * @param self A JSON writer.
> + * @param id An array name.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_arr_start(ujson_writer *self, const char *id);
> +
> +/**
> + * @brief Finishes a JSON array.
> + *
> + * The call will fail if we are currenlty not writing out an array.
> + *
> + * @param self A JSON writer.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_arr_finish(ujson_writer *self);
> +
> +/**
> + * @brief Adds a null value.
> + *
> + * The id must be NULL inside of an array, and must be non-NULL inside of an
> + * object.
> + *
> + * @param self A JSON writer.
> + * @param id A null value name.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_null_add(ujson_writer *self, const char *id);
> +
> +/**
> + * @brief Adds an integer value.
> + *
> + * The id must be NULL inside of an array, and must be non-NULL inside of an
> + * object.
> + *
> + * @param self A JSON writer.
> + * @param id An integer value name.
> + * @param val An integer value.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_int_add(ujson_writer *self, const char *id, long val);
> +
> +/**
> + * @brief Adds a bool value.
> + *
> + * The id must be NULL inside of an array, and must be non-NULL inside of an
> + * object.
> + *
> + * @param self A JSON writer.
> + * @param id An boolean value name.
> + * @param val A boolean value.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_bool_add(ujson_writer *self, const char *id, int val);
> +
> +/**
> + * @brief Adds a float value.
> + *
> + * The id must be NULL inside of an array, and must be non-NULL inside of an
> + * object.
> + *
> + * @param self A JSON writer.
> + * @param id A floating point value name.
> + * @param val A floating point value.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_float_add(ujson_writer *self, const char *id, double val);
> +
> +/**
> + * @brief Adds a string value.
> + *
> + * The id must be NULL inside of an array, and must be non-NULL inside of an
> + * object.
> + *
> + * @param self A JSON writer.
> + * @param id A string value name.
> + * @param str An UTF8 string value.
> + *
> + * @return Zero on success, non-zero otherwise.
> + */
> +int ujson_str_add(ujson_writer *self, const char *id, const char *str);
> +
> +/**
> + * @brief Finalizes json writer.
> + *
> + * Finalizes the json writer, throws possible errors through the error printing
> + * function.
> + *
> + * @param self A JSON writer.
> + * @return Overall error value.
> + */
> +int ujson_writer_finish(ujson_writer *self);
> +
> +#endif /* UJSON_WRITER_H */
> diff --git a/libs/ujson/Makefile b/libs/ujson/Makefile
> new file mode 100644
> index 000000000..4c8508010
> --- /dev/null
> +++ b/libs/ujson/Makefile
> @@ -0,0 +1,12 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# Copyright (C) Cyril Hrubis <chrubis@suse.cz>
> +
> +top_srcdir		?= ../..
> +
> +include $(top_srcdir)/include/mk/env_pre.mk
> +
> +INTERNAL_LIB		:= libujson.a
> +
> +include $(top_srcdir)/include/mk/lib.mk
> +include $(top_srcdir)/include/mk/generic_leaf_target.mk
> diff --git a/libs/ujson/ujson_common.c b/libs/ujson/ujson_common.c
> new file mode 100644
> index 000000000..c9cada9a9
> --- /dev/null
> +++ b/libs/ujson/ujson_common.c
> @@ -0,0 +1,38 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +#include <stdio.h>
> +#include "ujson_common.h"
> +
> +void ujson_err_handler(void *err_print_priv, const char *line)
> +{
> +	fputs(line, err_print_priv);
> +	putc('\n', err_print_priv);
> +}
> +
> +const char *ujson_type_name(enum ujson_type type)
> +{
> +	switch (type) {
> +	case UJSON_VOID:
> +		return "void";
> +	case UJSON_INT:
> +		return "integer";
> +	case UJSON_FLOAT:
> +		return "float";
> +	case UJSON_BOOL:
> +		return "boolean";
> +	case UJSON_NULL:
> +		return "null";
> +	case UJSON_STR:
> +		return "string";
> +	case UJSON_OBJ:
> +		return "object";
> +	case UJSON_ARR:
> +		return "array";
> +	default:
> +		return "invalid";
> +	}
> +}
> +
> diff --git a/libs/ujson/ujson_reader.c b/libs/ujson/ujson_reader.c
> new file mode 100644
> index 000000000..d508f00d3
> --- /dev/null
> +++ b/libs/ujson/ujson_reader.c
> @@ -0,0 +1,1081 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <stdarg.h>
> +#include <stdint.h>
> +
> +#include "ujson_utf.h"
> +#include "ujson_reader.h"
> +
> +static const struct ujson_obj empty = {};
> +const struct ujson_obj *ujson_empty_obj = &empty;
> +
> +static inline int buf_empty(ujson_reader *buf)
> +{
> +	return buf->off >= buf->len;
> +}
> +
> +static int eatws(ujson_reader *buf)
> +{
> +	while (!buf_empty(buf)) {
> +		switch (buf->json[buf->off]) {
> +		case ' ':
> +		case '\t':
> +		case '\n':
> +		case '\r':
> +		break;
> +		default:
> +			goto ret;
> +		}
> +
> +		buf->off += 1;
> +	}
> +ret:
> +	return buf_empty(buf);
> +}
> +
> +static char getb(ujson_reader *buf)
> +{
> +	if (buf_empty(buf))
> +		return 0;
> +
> +	return buf->json[buf->off++];
> +}
> +
> +static char peekb_off(ujson_reader *buf, size_t off)
> +{
> +	if (buf->off + off >= buf->len)
> +		return 0;
> +
> +	return buf->json[buf->off + off];
> +}
> +
> +static char peekb(ujson_reader *buf)
> +{
> +	if (buf_empty(buf))
> +		return 0;
> +
> +	return buf->json[buf->off];
> +}
> +
> +static int eatb(ujson_reader *buf, char ch)
> +{
> +	if (peekb(buf) != ch)
> +		return 0;
> +
> +	getb(buf);
> +	return 1;
> +}
> +
> +static int eatb2(ujson_reader *buf, char ch1, char ch2)
> +{
> +	if (peekb(buf) != ch1 && peekb(buf) != ch2)
> +		return 0;
> +
> +	getb(buf);
> +	return 1;
> +}
> +
> +static int eatstr(ujson_reader *buf, const char *str)
> +{
> +	while (*str) {
> +		if (!eatb(buf, *str))
> +			return 0;
> +		str++;
> +	}
> +
> +	return 1;
> +}
> +
> +static int hex2val(unsigned char b)
> +{
> +	switch (b) {
> +	case '0' ... '9':
> +		return b - '0';
> +	case 'a' ... 'f':
> +		return b - 'a' + 10;
> +	case 'A' ... 'F':
> +		return b - 'A' + 10;
> +	default:
> +		return -1;
> +	}
> +}
> +
> +static int32_t parse_ucode_cp(ujson_reader *buf)
> +{
> +	int ret = 0, v, i;
> +
> +	for (i = 0; i < 4; i++) {
> +		if ((v = hex2val(getb(buf))) < 0)
> +			goto err;
> +		ret *= 16;
> +		ret += v;
> +	}
> +
> +	return ret;
> +err:
> +	ujson_err(buf, "Expected four hexadecimal digits");
> +	return -1;
> +}
> +
> +static unsigned int parse_ucode_esc(ujson_reader *buf, char *str,
> +                                    size_t off, size_t len)
> +{
> +	int32_t ucode = parse_ucode_cp(buf);
> +
> +	if (ucode < 0)
> +		return 0;
> +
> +	if (!str)
> +		return ucode;
> +
> +	if (ujson_utf8_bytes(ucode) + 1 >= len - off) {
> +		ujson_err(buf, "String buffer too short!");
> +		return 0;
> +	}
> +
> +	return ujson_to_utf8(ucode, str+off);
> +}
> +
> +static int copy_str(ujson_reader *buf, char *str, size_t len)
> +{
> +	size_t pos = 0;
> +	int esc = 0;
> +	unsigned int l;
> +
> +	eatb(buf, '"');
> +
> +	for (;;) {
> +		if (buf_empty(buf)) {
> +			ujson_err(buf, "Unterminated string");
> +			return 1;
> +		}
> +
> +		if (!esc && eatb(buf, '"')) {
> +			if (str)
> +				str[pos] = 0;
> +			return 0;
> +		}
> +
> +		unsigned char b = getb(buf);
> +
> +		if (b < 0x20) {
> +			if (!peekb(buf))
> +				ujson_err(buf, "Unterminated string");
> +			else
> +				ujson_err(buf, "Invalid string character 0x%02x", b);
> +			return 1;
> +		}
> +
> +		if (!esc && b == '\\') {
> +			esc = 1;
> +			continue;
> +		}
> +
> +		if (esc) {
> +			switch (b) {
> +			case '"':
> +			case '\\':
> +			case '/':
> +			break;
> +			case 'b':
> +				b = '\b';
> +			break;
> +			case 'f':
> +				b = '\f';
> +			break;
> +			case 'n':
> +				b = '\n';
> +			break;
> +			case 'r':
> +				b = '\r';
> +			break;
> +			case 't':
> +				b = '\t';
> +			break;
> +			case 'u':
> +				if (!(l = parse_ucode_esc(buf, str, pos, len)))
> +					return 1;
> +				pos += l;
> +				b = 0;
> +			break;
> +			default:
> +				ujson_err(buf, "Invalid escape \\%c", b);
> +				return 1;
> +			}
> +			esc = 0;
> +		}
> +
> +		if (str && b) {
> +			if (pos + 1 >= len) {
> +				ujson_err(buf, "String buffer too short!");
> +				return 1;
> +			}
> +
> +			str[pos++] = b;
> +		}
> +	}
> +
> +	return 1;
> +}
> +
> +static int copy_id_str(ujson_reader *buf, char *str, size_t len)
> +{
> +	size_t pos = 0;
> +
> +	if (eatws(buf))
> +		goto err0;
> +
> +	if (!eatb(buf, '"'))
> +		goto err0;
> +
> +	for (;;) {
> +		if (buf_empty(buf)) {
> +			ujson_err(buf, "Unterminated ID string");
> +			return 1;
> +		}
> +
> +		if (eatb(buf, '"')) {
> +			str[pos] = 0;
> +			break;
> +		}
> +
> +		if (pos >= len-1) {
> +			ujson_err(buf, "ID string too long");
> +			return 1;
> +		}
> +
> +		str[pos++] = getb(buf);
> +	}
> +
> +	if (eatws(buf))
> +		goto err1;
> +
> +	if (!eatb(buf, ':'))
> +		goto err1;
> +
> +	return 0;
> +err0:
> +	ujson_err(buf, "Expected ID string");
> +	return 1;
> +err1:
> +	ujson_err(buf, "Expected ':' after ID string");
> +	return 1;
> +}
> +
> +static int is_digit(char b)
> +{
> +	switch (b) {
> +	case '0' ... '9':
> +		return 1;
> +	default:
> +		return 0;
> +	}
> +}
> +
> +static int get_int(ujson_reader *buf, struct ujson_val *res)
> +{
> +	long val = 0;
> +	int sign = 1;
> +
> +	if (eatb(buf, '-')) {
> +		sign = -1;
> +		if (!is_digit(peekb(buf))) {
> +			ujson_err(buf, "Expected digit(s)");
> +			return 1;
> +		}
> +	}
> +
> +	if (peekb(buf) == '0' && is_digit(peekb_off(buf, 1))) {
> +		ujson_err(buf, "Leading zero in number!");
> +		return 1;
> +	}
> +
> +	while (is_digit(peekb(buf))) {
> +		val *= 10;
> +		val += getb(buf) - '0';
> +		//TODO: overflow?
> +	}
> +
> +	if (sign < 0)
> +		val = -val;
> +
> +	res->val_int = val;
> +	res->val_float = val;
> +
> +	return 0;
> +}
> +
> +static int eat_digits(ujson_reader *buf)
> +{
> +	if (!is_digit(peekb(buf))) {
> +		ujson_err(buf, "Expected digit(s)");
> +		return 1;
> +	}
> +
> +	while (is_digit(peekb(buf)))
> +		getb(buf);
> +
> +	return 0;
> +}
> +
> +static int get_float(ujson_reader *buf, struct ujson_val *res)
> +{
> +	off_t start = buf->off;
> +
> +	eatb(buf, '-');
> +
> +	if (peekb(buf) == '0' && is_digit(peekb_off(buf, 1))) {
> +		ujson_err(buf, "Leading zero in float");
> +		return 1;
> +	}
> +
> +	if (eat_digits(buf))
> +		return 1;
> +
> +	switch (getb(buf)) {
> +	case '.':
> +		if (eat_digits(buf))
> +			return 1;
> +
> +		if (!eatb2(buf, 'e', 'E'))
> +			break;
> +
> +		/* fallthrough */
> +	case 'e':
> +	case 'E':
> +		eatb2(buf, '+', '-');
> +
> +		if (eat_digits(buf))
> +			return 1;
> +	break;
> +	}
> +
> +	size_t len = buf->off - start;
> +	char tmp[len+1];
> +
> +	memcpy(tmp, buf->json + start, len);
> +
> +	tmp[len] = 0;
> +
> +	res->val_float = strtod(tmp, NULL);
> +
> +	return 0;
> +}
> +
> +static int get_bool(ujson_reader *buf, struct ujson_val *res)
> +{
> +	switch (peekb(buf)) {
> +	case 'f':
> +		if (!eatstr(buf, "false")) {
> +			ujson_err(buf, "Expected 'false'");
> +			return 1;
> +		}
> +
> +		res->val_bool = 0;
> +	break;
> +	case 't':
> +		if (!eatstr(buf, "true")) {
> +			ujson_err(buf, "Expected 'true'");
> +			return 1;
> +		}
> +
> +		res->val_bool = 1;
> +	break;
> +	}
> +
> +	return 0;
> +}
> +
> +static int get_null(ujson_reader *buf)
> +{
> +	if (!eatstr(buf, "null")) {
> +		ujson_err(buf, "Expected 'null'");
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +int ujson_obj_skip(ujson_reader *buf)
> +{
> +	struct ujson_val res = {};
> +
> +	UJSON_OBJ_FOREACH(buf, &res) {
> +		switch (res.type) {
> +		case UJSON_OBJ:
> +			if (ujson_obj_skip(buf))
> +				return 1;
> +		break;
> +		case UJSON_ARR:
> +			if (ujson_arr_skip(buf))
> +				return 1;
> +		break;
> +		default:
> +		break;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +int ujson_arr_skip(ujson_reader *buf)
> +{
> +	struct ujson_val res = {};
> +
> +	UJSON_ARR_FOREACH(buf, &res) {
> +		switch (res.type) {
> +		case UJSON_OBJ:
> +			if (ujson_obj_skip(buf))
> +				return 1;
> +		break;
> +		case UJSON_ARR:
> +			if (ujson_arr_skip(buf))
> +				return 1;
> +		break;
> +		default:
> +		break;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static enum ujson_type next_num_type(ujson_reader *buf)
> +{
> +	size_t off = 0;
> +
> +	for (;;) {
> +		char b = peekb_off(buf, off++);
> +
> +		switch (b) {
> +		case 0:
> +		case ',':
> +			return UJSON_INT;
> +		case '.':
> +		case 'e':
> +		case 'E':
> +			return UJSON_FLOAT;
> +		}
> +	}
> +
> +	return UJSON_VOID;
> +}
> +
> +enum ujson_type ujson_next_type(ujson_reader *buf)
> +{
> +	if (eatws(buf)) {
> +		ujson_err(buf, "Unexpected end");
> +		return UJSON_VOID;
> +	}
> +
> +	char b = peekb(buf);
> +
> +	switch (b) {
> +	case '{':
> +		return UJSON_OBJ;
> +	case '[':
> +		return UJSON_ARR;
> +	case '"':
> +		return UJSON_STR;
> +	case '-':
> +	case '0' ... '9':
> +		return next_num_type(buf);
> +	case 'f':
> +	case 't':
> +		return UJSON_BOOL;
> +	break;
> +	case 'n':
> +		return UJSON_NULL;
> +	break;
> +	default:
> +		ujson_err(buf, "Expected object, array, number or string");
> +		return UJSON_VOID;
> +	}
> +}
> +
> +enum ujson_type ujson_reader_start(ujson_reader *buf)
> +{
> +	enum ujson_type type = ujson_next_type(buf);
> +
> +	switch (type) {
> +	case UJSON_ARR:
> +	case UJSON_OBJ:
> +	case UJSON_VOID:
> +	break;
> +	default:
> +		ujson_err(buf, "JSON can start only with array or object");
> +		type = UJSON_VOID;
> +	break;
> +	}
> +
> +	return type;
> +}
> +
> +static int get_value(ujson_reader *buf, struct ujson_val *res)
> +{
> +	int ret = 0;
> +
> +	res->type = ujson_next_type(buf);
> +
> +	switch (res->type) {
> +	case UJSON_STR:
> +		if (copy_str(buf, res->buf, res->buf_size)) {
> +			res->type = UJSON_VOID;
> +			return 0;
> +		}
> +		res->val_str = res->buf;
> +		return 1;
> +	case UJSON_INT:
> +		ret = get_int(buf, res);
> +	break;
> +	case UJSON_FLOAT:
> +		ret = get_float(buf, res);
> +	break;
> +	case UJSON_BOOL:
> +		ret = get_bool(buf, res);
> +	break;
> +	case UJSON_NULL:
> +		ret = get_null(buf);
> +	break;
> +	case UJSON_VOID:
> +		return 0;
> +	case UJSON_ARR:
> +	case UJSON_OBJ:
> +		buf->sub_off = buf->off;
> +		return 1;
> +	}
> +
> +	if (ret) {
> +		res->type = UJSON_VOID;
> +		return 0;
> +	}
> +
> +	return 1;
> +}
> +
> +static int pre_next(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (!eatb(buf, ',')) {
> +		ujson_err(buf, "Expected ','");
> +		res->type = UJSON_VOID;
> +		return 1;
> +	}
> +
> +	if (eatws(buf)) {
> +		ujson_err(buf, "Unexpected end");
> +		res->type = UJSON_VOID;
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +static int check_end(ujson_reader *buf, struct ujson_val *res, char b)
> +{
> +	if (eatws(buf)) {
> +		ujson_err(buf, "Unexpected end");
> +		return 1;
> +	}
> +
> +	if (eatb(buf, b)) {
> +		res->type = UJSON_VOID;
> +		eatws(buf);
> +		eatb(buf, 0);
> +		buf->depth--;
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +/*
> + * This is supposed to return a pointer to a string stored as a first member of
> + * a structure given an array.
> + *
> + * e.g.
> + *
> + *	struct foo {
> + *		const char *key;
> + *		...
> + *	};
> + *
> + *	const struct foo bar[10] = {...};
> + *
> + *      // Returns a pointer to the key string in a second structure in bar[].
> + *	const char *key = list_elem(bar, sizeof(struct foo), 1);
> + */
> +static inline const char *list_elem(const void *arr, size_t memb_size, size_t idx)
> +{
> +	return *(const char**)(arr + idx * memb_size);
> +}
> +
> +size_t ujson_lookup(const void *arr, size_t memb_size, size_t list_len,
> +                    const char *key)
> +{
> +	size_t l = 0;
> +	size_t r = list_len-1;
> +	size_t mid = -1;
> +
> +	if (!list_len)
> +		return (size_t)-1;
> +
> +	while (r - l > 1) {
> +		mid = (l+r)/2;
> +
> +		int ret = strcmp(list_elem(arr, memb_size, mid), key);
> +		if (!ret)
> +			return mid;
> +
> +		if (ret < 0)
> +			l = mid;
> +		else
> +			r = mid;
> +	}
> +
> +	if (r != mid && !strcmp(list_elem(arr, memb_size, r), key))
> +		return r;
> +
> +	if (l != mid && !strcmp(list_elem(arr, memb_size, l), key))
> +		return l;
> +
> +	return -1;
> +}
> +
> +static int skip_obj_val(ujson_reader *buf)
> +{
> +	struct ujson_val dummy = {};
> +
> +	if (!get_value(buf, &dummy))
> +		return 0;
> +
> +	switch (dummy.type) {
> +	case UJSON_OBJ:
> +		return !ujson_obj_skip(buf);
> +	case UJSON_ARR:
> +		return !ujson_arr_skip(buf);
> +	default:
> +		return 1;
> +	}
> +}
> +
> +static int obj_next(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (copy_id_str(buf, res->id, sizeof(res->id)))
> +		return 0;
> +
> +	return get_value(buf, res);
> +}
> +
> +static int obj_pre_next(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (ujson_reader_err(buf))
> +		return 1;
> +
> +	if (check_end(buf, res, '}'))
> +		return 1;
> +
> +	if (pre_next(buf, res))
> +		return 1;
> +
> +	return 0;
> +}
> +
> +static int obj_next_filter(ujson_reader *buf, struct ujson_val *res,
> +                           const struct ujson_obj *obj, const struct ujson_obj *ign)
> +{
> +	const struct ujson_obj_attr *attr;
> +
> +	for (;;) {
> +		if (copy_id_str(buf, res->id, sizeof(res->id)))
> +			return 0;
> +
> +		res->idx = obj ? ujson_obj_lookup(obj, res->id) : (size_t)-1;
> +
> +		if (res->idx != (size_t)-1) {
> +			if (!get_value(buf, res))
> +				return 0;
> +
> +			attr = &obj->attrs[res->idx];
> +
> +			if (attr->type == UJSON_VOID)
> +				return 1;
> +
> +			if (attr->type == res->type)
> +				return 1;
> +
> +			if (attr->type == UJSON_FLOAT &&
> +			    res->type == UJSON_INT)
> +				return 1;
> +
> +			ujson_warn(buf, "Wrong '%s' type expected %s",
> +				     attr->key, ujson_type_name(attr->type));
> +		} else {
> +			if (!skip_obj_val(buf))
> +				return 0;
> +
> +			if (ign && ujson_obj_lookup(ign, res->id) == (size_t)-1)
> +				ujson_warn(buf, "Unexpected key '%s'", res->id);
> +		}
> +
> +		if (obj_pre_next(buf, res))
> +			return 0;
> +	}
> +}
> +
> +static int check_err(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (ujson_reader_err(buf)) {
> +		res->type = UJSON_VOID;
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +int ujson_obj_next_filter(ujson_reader *buf, struct ujson_val *res,
> +                            const struct ujson_obj *obj, const struct ujson_obj *ign)
> +{
> +	if (check_err(buf, res))
> +		return 0;
> +
> +	if (obj_pre_next(buf, res))
> +		return 0;
> +
> +	return obj_next_filter(buf, res, obj, ign);
> +}
> +
> +int ujson_obj_next(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (check_err(buf, res))
> +		return 0;
> +
> +	if (obj_pre_next(buf, res))
> +		return 0;
> +
> +	return obj_next(buf, res);
> +}
> +
> +static int any_first(ujson_reader *buf, char b)
> +{
> +	if (eatws(buf)) {
> +		ujson_err(buf, "Unexpected end");
> +		return 1;
> +	}
> +
> +	if (!eatb(buf, b)) {
> +		ujson_err(buf, "Expected '%c'", b);
> +		return 1;
> +	}
> +
> +	buf->depth++;
> +
> +	if (buf->depth > buf->max_depth) {
> +		ujson_err(buf, "Recursion too deep");
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +int ujson_obj_first_filter(ujson_reader *buf, struct ujson_val *res,
> +                             const struct ujson_obj *obj, const struct ujson_obj *ign)
> +{
> +	if (check_err(buf, res))
> +		return 0;
> +
> +	if (any_first(buf, '{'))
> +		return 0;
> +
> +	if (check_end(buf, res, '}'))
> +		return 0;
> +
> +	return obj_next_filter(buf, res, obj, ign);
> +}
> +
> +int ujson_obj_first(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (check_err(buf, res))
> +		return 0;
> +
> +	if (any_first(buf, '{'))
> +		return 0;
> +
> +	if (check_end(buf, res, '}'))
> +		return 0;
> +
> +	return obj_next(buf, res);
> +}
> +
> +static int arr_next(ujson_reader *buf, struct ujson_val *res)
> +{
> +	return get_value(buf, res);
> +}
> +
> +int ujson_arr_first(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (check_err(buf, res))
> +		return 0;
> +
> +	if (any_first(buf, '['))
> +		return 0;
> +
> +	if (check_end(buf, res, ']'))
> +		return 0;
> +
> +	return arr_next(buf, res);
> +}
> +
> +int ujson_arr_next(ujson_reader *buf, struct ujson_val *res)
> +{
> +	if (check_err(buf, res))
> +		return 0;
> +
> +	if (check_end(buf, res, ']'))
> +		return 0;
> +
> +	if (pre_next(buf, res))
> +		return 0;
> +
> +	return arr_next(buf, res);
> +}
> +
> +static void ujson_err_va(ujson_reader *buf, const char *fmt, va_list va)
> +{
> +	vsnprintf(buf->err, UJSON_ERR_MAX, fmt, va);
> +}
> +
> +void ujson_err(ujson_reader *buf, const char *fmt, ...)
> +{
> +	va_list va;
> +
> +	va_start(va, fmt);
> +	ujson_err_va(buf, fmt, va);
> +	va_end(va);
> +}
> +
> +static void vprintf_line(ujson_reader *buf, const char *fmt, va_list va)
> +{
> +	char line[UJSON_ERR_MAX+1];
> +
> +	vsnprintf(line, sizeof(line), fmt, va);
> +
> +	line[UJSON_ERR_MAX] = 0;
> +
> +	buf->err_print(buf->err_print_priv, line);
> +}
> +
> +static void printf_line(ujson_reader *buf, const char *fmt, ...)
> +{
> +	va_list va;
> +
> +	va_start(va, fmt);
> +	vprintf_line(buf, fmt, va);
> +	va_end(va);
> +}
> +
> +static void printf_json_line(ujson_reader *buf, size_t line_nr, const char *buf_pos)
> +{
> +	char line[UJSON_ERR_MAX+1];
> +	size_t plen, i;
> +
> +	plen = sprintf(line, "%03zu: ", line_nr);
> +
> +	for (i = 0; i < UJSON_ERR_MAX-plen && buf_pos[i] && buf_pos[i] != '\n'; i++)
> +		line[i+plen] = buf_pos[i];
> +
> +	line[i+plen] = 0;
> +
> +	buf->err_print(buf->err_print_priv, line);
> +}
> +
> +static void print_arrow(ujson_reader *buf, const char *buf_pos, size_t count)
> +{
> +	char line[count + 7];
> +	size_t i;
> +
> +	/* The '000: ' prefix */
> +	for (i = 0; i <= 5; i++)
> +		line[i] = ' ';
> +
> +	for (i = 0; i < count; i++)
> +		line[i+5] = buf_pos[i] == '\t' ? '\t' : ' ';
> +
> +	line[count+5] = '^';
> +	line[count+6] = 0;
> +
> +	buf->err_print(buf->err_print_priv, line);
> +}
> +
> +#define ERR_LINES 10
> +
> +#define MIN(A, B) ((A < B) ? (A) : (B))
> +
> +static void print_snippet(ujson_reader *buf, const char *type)
> +{
> +	ssize_t i;
> +	const char *lines[ERR_LINES] = {};
> +	size_t cur_line = 0;
> +	size_t cur_off = 0;
> +	size_t last_off = buf->off;
> +
> +	for (;;) {
> +		lines[(cur_line++) % ERR_LINES] = buf->json + cur_off;
> +
> +		while (cur_off < buf->len && buf->json[cur_off] != '\n')
> +			cur_off++;
> +
> +		if (cur_off >= buf->off)
> +			break;
> +
> +		cur_off++;
> +		last_off = buf->off - cur_off;
> +	}
> +
> +	printf_line(buf, "%s at line %03zu", type, cur_line);
> +	buf->err_print(buf->err_print_priv, "");
> +
> +	size_t idx = 0;
> +
> +	for (i = MIN(ERR_LINES, cur_line); i > 0; i--) {
> +		idx = (cur_line - i) % ERR_LINES;
> +		printf_json_line(buf, cur_line - i + 1, lines[idx]);
> +	}
> +
> +	print_arrow(buf, lines[idx], last_off);
> +}
> +
> +void ujson_err_print(ujson_reader *buf)
> +{
> +	if (!buf->err_print)
> +		return;
> +
> +	print_snippet(buf, "Parse error");
> +	buf->err_print(buf->err_print_priv, buf->err);
> +}
> +
> +void ujson_warn(ujson_reader *buf, const char *fmt, ...)
> +{
> +	va_list va;
> +
> +	if (buf->flags & UJSON_READER_STRICT) {
> +		va_start(va, fmt);
> +		ujson_err_va(buf, fmt, va);
> +		va_end(va);
> +		return;
> +	}
> +
> +	if (!buf->err_print)
> +		return;
> +
> +	print_snippet(buf, "Warning");
> +
> +	va_start(va, fmt);
> +	vprintf_line(buf, fmt, va);
> +	va_end(va);
> +}
> +
> +void ujson_print(void *err_print_priv, const char *line)
> +{
> +	fputs(line, err_print_priv);
> +	putc('\n', err_print_priv);
> +}
> +
> +ujson_reader *ujson_reader_load(const char *path)
> +{
> +	int fd = open(path, O_RDONLY);
> +	ujson_reader *ret;
> +	ssize_t res;
> +	off_t len, off = 0;
> +
> +	if (fd < 0)
> +		return NULL;
> +
> +	len = lseek(fd, 0, SEEK_END);
> +	if (len == (off_t)-1) {
> +		fprintf(stderr, "lseek() failed\n");
> +		goto err0;
> +	}
> +
> +	if (lseek(fd, 0, SEEK_SET) == (off_t)-1) {
> +		fprintf(stderr, "lseek() failed\n");
> +		goto err0;
> +	}
> +
> +	ret = malloc(sizeof(ujson_reader) + len + 1);
> +	if (!ret) {
> +		fprintf(stderr, "malloc() failed\n");
> +		goto err0;
> +	}
> +
> +	memset(ret, 0, sizeof(*ret));
> +
> +	ret->buf[len] = 0;
> +	ret->len = len;
> +	ret->max_depth = UJSON_RECURSION_MAX;
> +	ret->json = ret->buf;
> +	ret->err_print = UJSON_ERR_PRINT;
> +	ret->err_print_priv = UJSON_ERR_PRINT_PRIV;
> +
> +	while (off < len) {
> +		res = read(fd, ret->buf + off, len - off);
> +		if (res < 0) {
> +			fprintf(stderr, "read() failed\n");
> +			goto err1;
> +		}
> +
> +		off += res;
> +	}
> +
> +	close(fd);
> +
> +	return ret;
> +err1:
> +	free(ret);
> +err0:
> +	close(fd);
> +	return NULL;
> +}
> +
> +void ujson_reader_finish(ujson_reader *self)
> +{
> +	if (ujson_reader_err(self))
> +		ujson_err_print(self);
> +	else if (!ujson_reader_consumed(self))
> +		ujson_warn(self, "Garbage after JSON string!");
> +}
> +
> +void ujson_reader_free(ujson_reader *buf)
> +{
> +	free(buf);
> +}
> +
> +ujson_val *ujson_val_alloc(size_t buf_size)
> +{
> +	buf_size = buf_size == 0 ? 4096 : buf_size;
> +	ujson_val *ret;
> +
> +	ret = malloc(sizeof(ujson_val) + buf_size);
> +	if (!ret)
> +		return NULL;
> +
> +	memset(ret, 0, sizeof(ujson_val) + buf_size);
> +
> +	ret->buf = ret->buf__;
> +	ret->buf_size = buf_size;
> +
> +	return ret;
> +}
> +
> +void ujson_val_free(ujson_val *self)
> +{
> +	free(self);
> +}
> diff --git a/libs/ujson/ujson_utf.c b/libs/ujson/ujson_utf.c
> new file mode 100644
> index 000000000..2c08a39a8
> --- /dev/null
> +++ b/libs/ujson/ujson_utf.c
> @@ -0,0 +1,105 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2022-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +#include <stddef.h>
> +#include <ujson_utf.h>
> +
> +int8_t ujson_utf8_next_chsz(const char *str, size_t off)
> +{
> +	char ch = str[off];
> +	uint8_t len = 0;
> +
> +	if (!ch)
> +		return 0;
> +
> +	if (UJSON_UTF8_IS_ASCII(ch))
> +		return 1;
> +
> +	if (UJSON_UTF8_IS_2BYTE(ch)) {
> +		len = 2;
> +		goto ret;
> +	}
> +
> +	if (UJSON_UTF8_IS_3BYTE(ch)) {
> +		len = 3;
> +		goto ret;
> +	}
> +
> +	if (UJSON_UTF8_IS_4BYTE(ch)) {
> +		len = 4;
> +		goto ret;
> +	}
> +
> +	return -1;
> +ret:
> +	if (!UJSON_UTF8_IS_NBYTE(str[off+1]))
> +		return -1;
> +
> +	if (len > 2 && !UJSON_UTF8_IS_NBYTE(str[off+2]))
> +		return -1;
> +
> +	if (len > 3 && !UJSON_UTF8_IS_NBYTE(str[off+3]))
> +		return -1;
> +
> +	return len;
> +}
> +
> +int8_t ujson_utf8_prev_chsz(const char *str, size_t off)
> +{
> +	char ch;
> +
> +	if (!off)
> +		return 0;
> +
> +	ch = str[--off];
> +
> +	if (UJSON_UTF8_IS_ASCII(ch))
> +		return 1;
> +
> +	if (!UJSON_UTF8_IS_NBYTE(ch))
> +		return -1;
> +
> +	if (off < 1)
> +		return -1;
> +
> +	ch = str[--off];
> +
> +	if (UJSON_UTF8_IS_2BYTE(ch))
> +		return 2;
> +
> +	if (!UJSON_UTF8_IS_NBYTE(ch))
> +		return -1;
> +
> +	if (off < 1)
> +		return -1;
> +
> +	ch = str[--off];
> +
> +	if (UJSON_UTF8_IS_3BYTE(ch))
> +		return 3;
> +
> +	if (!UJSON_UTF8_IS_NBYTE(ch))
> +		return -1;
> +
> +	if (off < 1)
> +		return -1;
> +
> +	ch = str[--off];
> +
> +	if (UJSON_UTF8_IS_4BYTE(ch))
> +		return 4;
> +
> +	return -1;
> +}
> +
> +size_t ujson_utf8_strlen(const char *str)
> +{
> +	size_t cnt = 0;
> +
> +	while (ujson_utf8_next(&str))
> +		cnt++;
> +
> +	return cnt;
> +}
> diff --git a/libs/ujson/ujson_writer.c b/libs/ujson/ujson_writer.c
> new file mode 100644
> index 000000000..6275be1ff
> --- /dev/null
> +++ b/libs/ujson/ujson_writer.c
> @@ -0,0 +1,491 @@
> +// SPDX-License-Identifier: LGPL-2.1-or-later
> +/*
> + * Copyright (C) 2021-2024 Cyril Hrubis <metan@ucw.cz>
> + */
> +
> +#include <string.h>
> +#include <stdarg.h>
> +#include <fcntl.h>
> +#include <errno.h>
> +#include <stdio.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +
> +#include "ujson_utf.h"
> +#include "ujson_writer.h"
> +
> +static inline int get_depth_bit(ujson_writer *self, char *mask)
> +{
> +	int depth = self->depth - 1;
> +
> +	if (depth < 0)
> +		return -1;
> +
> +	return !!(mask[depth/8] & (1<<(depth%8)));
> +}
> +
> +static inline void set_depth_bit(ujson_writer *self, int val)
> +{
> +	if (val)
> +		self->depth_type[self->depth/8] |= (1<<(self->depth%8));
> +	else
> +		self->depth_type[self->depth/8] &= ~(1<<(self->depth%8));
> +
> +	self->depth_first[self->depth/8] |= (1<<(self->depth%8));
> +
> +	self->depth++;
> +}
> +
> +static inline void clear_depth_bit(ujson_writer *self)
> +{
> +	self->depth--;
> +}
> +
> +static inline int in_arr(ujson_writer *self)
> +{
> +	return !get_depth_bit(self, self->depth_type);
> +}
> +
> +static inline int in_obj(ujson_writer *self)
> +{
> +	return get_depth_bit(self, self->depth_type);
> +}
> +
> +static inline void clear_depth_first(ujson_writer *self)
> +{
> +	int depth = self->depth - 1;
> +
> +	self->depth_first[depth/8] &= ~(1<<(depth%8));
> +}
> +
> +static inline int is_first(ujson_writer *self)
> +{
> +	int ret = get_depth_bit(self, self->depth_first);
> +
> +	if (ret == 1)
> +		clear_depth_first(self);
> +
> +	return ret;
> +}
> +
> +static inline void err(ujson_writer *buf, const char *fmt, ...)
> +{
> +	va_list va;
> +
> +	va_start(va, fmt);
> +	vsnprintf(buf->err, UJSON_ERR_MAX, fmt, va);
> +	va_end(va);
> +}
> +
> +static inline int is_err(ujson_writer *buf)
> +{
> +	return buf->err[0];
> +}
> +
> +static inline int out(ujson_writer *self, const char *buf, size_t len)
> +{
> +	return self->out(self, buf, len);
> +}
> +
> +static inline int out_str(ujson_writer *self, const char *str)
> +{
> +	return out(self, str, strlen(str));
> +}
> +
> +static inline int out_ch(ujson_writer *self, char ch)
> +{
> +	return out(self, &ch, 1);
> +}
> +
> +#define ESC_FLUSH(esc_char) do {\
> +	out(self, val, i); \
> +	val += i + 1; \
> +	i = 0; \
> +	out_str(self, esc_char); \
> +} while (0)
> +
> +static inline int out_esc_str(ujson_writer *self, const char *val)
> +{
> +	if (out_ch(self, '"'))
> +		return 1;
> +
> +	size_t i = 0;
> +	int8_t next_chsz;
> +
> +	do {
> +		next_chsz = ujson_utf8_next_chsz(val, i);
> +
> +		if (next_chsz == 1) {
> +			switch (val[i]) {
> +			case '\"':
> +				ESC_FLUSH("\\\"");
> +			break;
> +			case '\\':
> +				ESC_FLUSH("\\\\");
> +			break;
> +			case '/':
> +				ESC_FLUSH("\\/");
> +			break;
> +			case '\b':
> +				ESC_FLUSH("\\b");
> +			break;
> +			case '\f':
> +				ESC_FLUSH("\\f");
> +			break;
> +			case '\n':
> +				ESC_FLUSH("\\n");
> +			break;
> +			case '\r':
> +				ESC_FLUSH("\\r");
> +			break;
> +			case '\t':
> +				ESC_FLUSH("\\t");
> +			break;
> +			default:
> +				i += next_chsz;
> +			}
> +		} else {
> +			i += next_chsz;
> +		}
> +	} while (next_chsz);
> +
> +	if (i) {
> +		if (out(self, val, i))
> +			return 1;
> +	}
> +
> +	if (out_ch(self, '"'))
> +		return 1;
> +
> +	return 0;
> +}
> +
> +static int do_padd(ujson_writer *self)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < self->depth; i++) {
> +		if (out_ch(self, ' '))
> +			return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +static int newline(ujson_writer *self)
> +{
> +	if (out_ch(self, '\n'))
> +		return 0;
> +
> +	if (do_padd(self))
> +		return 1;
> +
> +	return 0;
> +}
> +
> +static int add_common(ujson_writer *self, const char *id)
> +{
> +	if (is_err(self))
> +		return 1;
> +
> +	if (!self->depth) {
> +		err(self, "Object/Array has to be started first");
> +		return 1;
> +	}
> +
> +	if (in_arr(self)) {
> +		if (id) {
> +			err(self, "Array entries can't have id");
> +			return 1;
> +		}
> +	} else {
> +		if (!id) {
> +			err(self, "Object entries must have id");
> +			return 1;
> +		}
> +	}
> +
> +	if (!is_first(self) && out_ch(self, ','))
> +		return 1;
> +
> +	if (self->depth && newline(self))
> +		return 1;
> +
> +	if (id) {
> +		if (out_esc_str(self, id))
> +			return 1;
> +
> +		if (out_str(self, ": "))
> +			return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +int ujson_obj_start(ujson_writer *self, const char *id)
> +{
> +	if (self->depth >= UJSON_RECURSION_MAX)
> +		return 1;
> +
> +	if (!self->depth && id) {
> +		err(self, "Top level object cannot have id");
> +		return 1;
> +	}
> +
> +	if (self->depth && add_common(self, id))
> +		return 1;
> +
> +	if (out_ch(self, '{'))
> +		return 1;
> +
> +	set_depth_bit(self, 1);
> +
> +	return 0;
> +}
> +
> +int ujson_obj_finish(ujson_writer *self)
> +{
> +	if (is_err(self))
> +		return 1;
> +
> +	if (!in_obj(self)) {
> +		err(self, "Not in object!");
> +		return 1;
> +	}
> +
> +	int first = is_first(self);
> +
> +	clear_depth_bit(self);
> +
> +	if (!first)
> +		newline(self);
> +
> +	return out_ch(self, '}');
> +}
> +
> +int ujson_arr_start(ujson_writer *self, const char *id)
> +{
> +	if (self->depth >= UJSON_RECURSION_MAX) {
> +		err(self, "Recursion too deep");
> +		return 1;
> +	}
> +
> +	if (!self->depth && id) {
> +		err(self, "Top level array cannot have id");
> +		return 1;
> +	}
> +
> +	if (self->depth && add_common(self, id))
> +		return 1;
> +
> +	if (out_ch(self, '['))
> +		return 1;
> +
> +	set_depth_bit(self, 0);
> +
> +	return 0;
> +}
> +
> +int ujson_arr_finish(ujson_writer *self)
> +{
> +	if (is_err(self))
> +		return 1;
> +
> +	if (!in_arr(self)) {
> +		err(self, "Not in array!");
> +		return 1;
> +	}
> +
> +	int first = is_first(self);
> +
> +	clear_depth_bit(self);
> +
> +	if (!first)
> +		newline(self);
> +
> +	return out_ch(self, ']');
> +}
> +
> +int ujson_null_add(ujson_writer *self, const char *id)
> +{
> +	if (add_common(self, id))
> +		return 1;
> +
> +	return out_str(self, "null");
> +}
> +
> +int ujson_int_add(ujson_writer *self, const char *id, long val)
> +{
> +	char buf[64];
> +
> +	if (add_common(self, id))
> +		return 1;
> +
> +	snprintf(buf, sizeof(buf), "%li", val);
> +
> +	return out_str(self, buf);
> +}
> +
> +int ujson_bool_add(ujson_writer *self, const char *id, int val)
> +{
> +	if (add_common(self, id))
> +		return 1;
> +
> +	if (val)
> +		return out_str(self, "true");
> +	else
> +		return out_str(self, "false");
> +}
> +
> +int ujson_str_add(ujson_writer *self, const char *id, const char *val)
> +{
> +	if (add_common(self, id))
> +		return 1;
> +
> +	if (out_esc_str(self, val))
> +		return 1;
> +
> +	return 0;
> +}
> +
> +int ujson_float_add(ujson_writer *self, const char *id, double val)
> +{
> +	char buf[64];
> +
> +	if (add_common(self, id))
> +		return 1;
> +
> +	snprintf(buf, sizeof(buf), "%lg", val);
> +
> +	return out_str(self, buf);
> +}
> +
> +int ujson_writer_finish(ujson_writer *self)
> +{
> +	if (is_err(self))
> +		goto err;
> +
> +	if (self->depth) {
> +		err(self, "Objects and/or Arrays not finished");
> +		goto err;
> +	}
> +
> +	if (newline(self))
> +		return 1;
> +
> +	return 0;
> +err:
> +	if (self->err_print)
> +		self->err_print(self->err_print_priv, self->err);
> +
> +	return 1;
> +}
> +
> +struct json_writer_file {
> +	int fd;
> +	size_t buf_used;
> +	char buf[1024];
> +};
> +
> +static int out_writer_file_write(ujson_writer *self, int fd, const char *buf, ssize_t buf_len)
> +{
> +	do {
> +		ssize_t ret = write(fd, buf, buf_len);
> +		if (ret <= 0) {
> +			err(self, "Failed to write to a file");
> +			return 1;
> +		}
> +
> +		if (ret > buf_len) {
> +			err(self, "Wrote more bytes than requested?!");
> +			return 1;
> +		}
> +
> +		buf_len -= ret;
> +	} while (buf_len);
> +
> +	return 0;
> +}
> +
> +static int out_writer_file(ujson_writer *self, const char *buf, size_t buf_len)
> +{
> +	struct json_writer_file *writer_file = self->out_priv;
> +	size_t buf_size = sizeof(writer_file->buf);
> +	size_t buf_avail = buf_size - writer_file->buf_used;
> +
> +	if (buf_len > buf_size/4)
> +		return out_writer_file_write(self, writer_file->fd, buf, buf_len);
> +
> +	if (buf_len >= buf_avail) {
> +		if (out_writer_file_write(self, writer_file->fd,
> +		                          writer_file->buf, writer_file->buf_used))
> +			return 1;
> +
> +		memcpy(writer_file->buf, buf, buf_len);
> +		writer_file->buf_used = buf_len;
> +		return 0;
> +	}
> +
> +	memcpy(writer_file->buf + writer_file->buf_used, buf, buf_len);
> +	writer_file->buf_used += buf_len;
> +
> +	return 0;
> +}
> +
> +int ujson_writer_file_close(ujson_writer *self)
> +{
> +	struct json_writer_file *writer_file = self->out_priv;
> +	int saved_errno = 0;
> +
> +	if (writer_file->buf_used) {
> +		if (out_writer_file_write(self, writer_file->fd,
> +		                          writer_file->buf, writer_file->buf_used))
> +
> +			saved_errno = errno;
> +	}
> +
> +	if (close(writer_file->fd)) {
> +		if (!saved_errno)
> +			saved_errno = errno;
> +	}
> +
> +	free(self);
> +
> +	if (saved_errno) {
> +		errno = saved_errno;
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +ujson_writer *ujson_writer_file_open(const char *path)
> +{
> +	ujson_writer *ret;
> +	struct json_writer_file *writer_file;
> +
> +	ret = malloc(sizeof(ujson_writer) + sizeof(struct json_writer_file));
> +	if (!ret)
> +		return NULL;
> +
> +	writer_file = (void*)ret + sizeof(ujson_writer);
> +
> +	writer_file->fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0664);
> +	if (!writer_file->fd) {
> +		free(ret);
> +		return NULL;
> +	}
> +
> +	writer_file->buf_used = 0;
> +
> +	memset(ret, 0, sizeof(*ret));
> +
> +	ret->err_print = UJSON_ERR_PRINT;
> +	ret->err_print_priv = UJSON_ERR_PRINT_PRIV;
> +	ret->out = out_writer_file;
> +	ret->out_priv = writer_file;
> +
> +	return ret;
> +}
> +
> +

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 3/4] libs: Vendor ujson library

[Cyril Hrubis]

Hi!
> would be nice to use this library also for C tests metadata later on.

I had this idea as well, that we could move the metadata into the
comment in C tests as well. I may try to experiment with that later on.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 4/4] testcaes/lib: Add shell loader

[Cyril Hrubis]

This commit implements a shell loader so that we don't have to write a C
loader for each LTP shell test. The idea is simple, the loader parses
the shell test and prepares the tst_test structure accordingly, then
runs the actual shell test.

The format for the metadata in the shell test was choosen to be JSON
because:

- I didn't want to invent an adhoc format and JSON is perfect for
  serializing data structures
- The metadata parser for shell test will be trivial, it will just pick
  the JSON from the comment, no parsing will be required

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
Reviewed-by: Richard Palethorpe <io@richiejp.com>
---
 include/tst_test.h                            |   2 +-
 testcases/lib/.gitignore                      |   1 +
 testcases/lib/Makefile                        |   6 +-
 testcases/lib/run_tests.sh                    |  21 +
 testcases/lib/tests/shell_loader.sh           |  26 +
 .../lib/tests/shell_loader_all_filesystems.sh |  27 +
 .../lib/tests/shell_loader_filesystems.sh     |  33 ++
 .../lib/tests/shell_loader_invalid_block.sh   |  26 +
 .../tests/shell_loader_invalid_metadata.sh    |  15 +
 testcases/lib/tests/shell_loader_kconfigs.sh  |  12 +
 .../lib/tests/shell_loader_no_metadata.sh     |   8 +
 .../lib/tests/shell_loader_supported_archs.sh |  12 +
 testcases/lib/tests/shell_loader_tags.sh      |  15 +
 testcases/lib/tests/shell_loader_tcnt.sh      |  15 +
 .../lib/tests/shell_loader_wrong_metadata.sh  |  15 +
 testcases/lib/tst_env.sh                      |   4 +
 testcases/lib/tst_loader.sh                   |  11 +
 testcases/lib/tst_run_shell.c                 | 491 ++++++++++++++++++
 18 files changed, 738 insertions(+), 2 deletions(-)
 create mode 100755 testcases/lib/tests/shell_loader.sh
 create mode 100755 testcases/lib/tests/shell_loader_all_filesystems.sh
 create mode 100755 testcases/lib/tests/shell_loader_filesystems.sh
 create mode 100755 testcases/lib/tests/shell_loader_invalid_block.sh
 create mode 100755 testcases/lib/tests/shell_loader_invalid_metadata.sh
 create mode 100755 testcases/lib/tests/shell_loader_kconfigs.sh
 create mode 100755 testcases/lib/tests/shell_loader_no_metadata.sh
 create mode 100755 testcases/lib/tests/shell_loader_supported_archs.sh
 create mode 100755 testcases/lib/tests/shell_loader_tags.sh
 create mode 100755 testcases/lib/tests/shell_loader_tcnt.sh
 create mode 100755 testcases/lib/tests/shell_loader_wrong_metadata.sh
 create mode 100644 testcases/lib/tst_loader.sh
 create mode 100644 testcases/lib/tst_run_shell.c

diff --git a/include/tst_test.h b/include/tst_test.h
index 9871676a5..d0fa84a71 100644
--- a/include/tst_test.h
+++ b/include/tst_test.h
@@ -274,7 +274,7 @@ struct tst_fs {
 	const char *const *mkfs_opts;
 	const char *mkfs_size_opt;
 
-	const unsigned int mnt_flags;
+	unsigned int mnt_flags;
 	const void *mnt_data;
 };
 
diff --git a/testcases/lib/.gitignore b/testcases/lib/.gitignore
index d0dacf62a..385f3c3ca 100644
--- a/testcases/lib/.gitignore
+++ b/testcases/lib/.gitignore
@@ -24,3 +24,4 @@
 /tst_supported_fs
 /tst_timeout_kill
 /tst_res_
+/tst_run_shell
diff --git a/testcases/lib/Makefile b/testcases/lib/Makefile
index 928d76d62..b3a9181c1 100644
--- a/testcases/lib/Makefile
+++ b/testcases/lib/Makefile
@@ -4,6 +4,9 @@
 
 top_srcdir		?= ../..
 
+LTPLIBS = ujson
+tst_run_shell: LTPLDLIBS = -lujson
+
 include $(top_srcdir)/include/mk/testcases.mk
 
 INSTALL_TARGETS		:= *.sh
@@ -13,6 +16,7 @@ MAKE_TARGETS		:= tst_sleep tst_random tst_checkpoint tst_rod tst_kvcmp\
 			   tst_getconf tst_supported_fs tst_check_drivers tst_get_unused_port\
 			   tst_get_median tst_hexdump tst_get_free_pids tst_timeout_kill\
 			   tst_check_kconfigs tst_cgctl tst_fsfreeze tst_ns_create tst_ns_exec\
-			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_
+			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_\
+			   tst_run_shell
 
 include $(top_srcdir)/include/mk/generic_trunk_target.mk
diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh
index 60e7d1bcf..e30065f1d 100755
--- a/testcases/lib/run_tests.sh
+++ b/testcases/lib/run_tests.sh
@@ -9,3 +9,24 @@ for i in `seq -w 01 06`; do
 	echo
 	./tests/shell_test$i
 done
+
+for i in shell_loader.sh shell_loader_all_filesystems.sh shell_loader_no_metadata.sh \
+	 shell_loader_wrong_metadata.sh shell_loader_invalid_metadata.sh\
+	 shell_loader_supported_archs.sh shell_loader_filesystems.sh\
+	 shell_loader_tcnt.sh shell_loader_kconfigs.sh shell_loader_tags.sh \
+	 shell_loader_invalid_block.sh; do
+	echo
+	echo "*** Running $i ***"
+	echo
+	$i
+done
+
+echo
+echo "*** Testing LTP test -h option ***"
+echo
+shell_loader.sh -h
+
+echo
+echo "*** Testing LTP test -i option ***"
+echo
+shell_loader.sh -i 2
diff --git a/testcases/lib/tests/shell_loader.sh b/testcases/lib/tests/shell_loader.sh
new file mode 100755
index 000000000..df7f0c0af
--- /dev/null
+++ b/testcases/lib/tests/shell_loader.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+#
+# ---
+# doc
+#
+# [Description]
+#
+# This is a simple shell test loader example.
+# ---
+#
+# ---
+# env
+# {
+#  "needs_tmpdir": true
+# }
+# ---
+
+. tst_loader.sh
+
+tst_res TPASS "Shell loader works fine!"
+case "$PWD" in
+	/tmp/*)
+		tst_res TPASS "We are running in temp directory in $PWD";;
+	*)
+		tst_res TFAIL "We are not running in temp directory but $PWD";;
+esac
diff --git a/testcases/lib/tests/shell_loader_all_filesystems.sh b/testcases/lib/tests/shell_loader_all_filesystems.sh
new file mode 100755
index 000000000..d5943c335
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_all_filesystems.sh
@@ -0,0 +1,27 @@
+#!/bin/sh
+#
+# ---
+# env
+# {
+#  "needs_root": true,
+#  "mount_device": true,
+#  "all_filesystems": true,
+#  "mntpoint": "ltp_mntpoint"
+# }
+# ---
+
+. tst_loader.sh
+
+tst_res TINFO "In shell"
+
+mntpath=$(realpath ltp_mntpoint)
+mounted=$(grep $mntpath /proc/mounts)
+
+if [ -n "$mounted" ]; then
+	device=$(echo $mounted |cut -d' ' -f 1)
+	path=$(echo $mounted |cut -d' ' -f 2)
+
+	tst_res TPASS "$device mounted at $path"
+else
+	tst_res TFAIL "Device not mounted!"
+fi
diff --git a/testcases/lib/tests/shell_loader_filesystems.sh b/testcases/lib/tests/shell_loader_filesystems.sh
new file mode 100755
index 000000000..5d8aa9808
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_filesystems.sh
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# ---
+# env
+# {
+#  "mount_device": true,
+#  "mntpoint": "ltp_mntpoint",
+#  "filesystems": [
+#   {
+#    "type": "btrfs"
+#   },
+#   {
+#    "type": "xfs",
+#    "mkfs_opts": ["-m", "reflink=1"]
+#   }
+#  ]
+# }
+# ---
+
+. tst_loader.sh
+
+tst_res TINFO "In shell"
+
+mntpoint=$(realpath ltp_mntpoint)
+mounted=$(grep $mntpoint /proc/mounts)
+
+if [ -n "$mounted" ]; then
+	fs=$(echo $mounted |cut -d' ' -f 3)
+
+	tst_res TPASS "Mounted device formatted with $fs"
+else
+	tst_res TFAIL "Device not mounted!"
+fi
diff --git a/testcases/lib/tests/shell_loader_invalid_block.sh b/testcases/lib/tests/shell_loader_invalid_block.sh
new file mode 100755
index 000000000..f41de04fd
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_invalid_block.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+#
+# ---
+# doc
+#
+# [Description]
+#
+# This is a simple shell test loader example.
+# ---
+#
+# ---
+# env
+# {
+#  "needs_tmpdir": true
+# }
+# ---
+#
+# ---
+# inv
+#
+# This is an invalid block that breaks the test.
+# ---
+
+. tst_loader.sh
+
+tst_res TPASS "This should pass!"
diff --git a/testcases/lib/tests/shell_loader_invalid_metadata.sh b/testcases/lib/tests/shell_loader_invalid_metadata.sh
new file mode 100755
index 000000000..c10b00f1b
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_invalid_metadata.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+#
+# This test has wrong metadata and should not be run
+#
+# ---
+# env
+# {
+#  {"needs_tmpdir": 42,
+# }
+# ---
+#
+
+. tst_loader.sh
+
+tst_res TFAIL "Shell loader should TBROK the test"
diff --git a/testcases/lib/tests/shell_loader_kconfigs.sh b/testcases/lib/tests/shell_loader_kconfigs.sh
new file mode 100755
index 000000000..7e9a1dce7
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_kconfigs.sh
@@ -0,0 +1,12 @@
+#!/bin/sh
+#
+# ---
+# env
+# {
+#  "needs_kconfigs": ["CONFIG_NUMA=y"]
+# }
+# ---
+
+. tst_loader.sh
+
+tst_res TPASS "Shell loader works fine!"
diff --git a/testcases/lib/tests/shell_loader_no_metadata.sh b/testcases/lib/tests/shell_loader_no_metadata.sh
new file mode 100755
index 000000000..60ba8b889
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_no_metadata.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+#
+# This test has no metadata and should not be executed
+#
+
+. tst_loader.sh
+
+tst_res TFAIL "Shell loader should TBROK the test"
diff --git a/testcases/lib/tests/shell_loader_supported_archs.sh b/testcases/lib/tests/shell_loader_supported_archs.sh
new file mode 100755
index 000000000..45213f840
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_supported_archs.sh
@@ -0,0 +1,12 @@
+#!/bin/sh
+#
+# ---
+# env
+# {
+#  "supported_archs": ["x86", "ppc64", "x86_64"]
+# }
+# ---
+
+. tst_loader.sh
+
+tst_res TPASS "We are running on supported architecture"
diff --git a/testcases/lib/tests/shell_loader_tags.sh b/testcases/lib/tests/shell_loader_tags.sh
new file mode 100755
index 000000000..a6278c37d
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_tags.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+#
+# ---
+# env
+# {
+#  "tags": [
+#   ["linux-git", "832478cd342ab"],
+#   ["CVE", "2099-999"]
+#  ]
+# }
+# ---
+
+. tst_loader.sh
+
+tst_res TFAIL "Fails the test so that tags are shown."
diff --git a/testcases/lib/tests/shell_loader_tcnt.sh b/testcases/lib/tests/shell_loader_tcnt.sh
new file mode 100755
index 000000000..81fc08179
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_tcnt.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+#
+# The script should be executed tcnt times and the iteration number should be in $1
+#
+# ---
+# env
+# {
+#  "tcnt": 2
+# }
+# ---
+#
+
+. tst_loader.sh
+
+tst_res TPASS "Iteration $1"
diff --git a/testcases/lib/tests/shell_loader_wrong_metadata.sh b/testcases/lib/tests/shell_loader_wrong_metadata.sh
new file mode 100755
index 000000000..752e25eea
--- /dev/null
+++ b/testcases/lib/tests/shell_loader_wrong_metadata.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+#
+# This test has wrong metadata and should not be run
+#
+# ---
+# env
+# {
+#  "needs_tmpdir": 42,
+# }
+# ---
+#
+
+. tst_loader.sh
+
+tst_res TFAIL "Shell loader should TBROK the test"
diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
index 948bc5024..67ba80744 100644
--- a/testcases/lib/tst_env.sh
+++ b/testcases/lib/tst_env.sh
@@ -1,4 +1,8 @@
 #!/bin/sh
+#
+# This is a minimal test environment for a shell scripts executed from C by
+# tst_run_shell() function. Shell tests must use the tst_loader.sh instead!
+#
 
 tst_script_name=$(basename $0)
 
diff --git a/testcases/lib/tst_loader.sh b/testcases/lib/tst_loader.sh
new file mode 100644
index 000000000..ed04d0340
--- /dev/null
+++ b/testcases/lib/tst_loader.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+#
+# This is a loader for shell tests that use the C test library.
+#
+
+if [ -z "$LTP_IPC_PATH" ]; then
+	tst_run_shell $(basename "$0") "$@"
+	exit $?
+else
+	. tst_env.sh
+fi
diff --git a/testcases/lib/tst_run_shell.c b/testcases/lib/tst_run_shell.c
new file mode 100644
index 000000000..8ed0f21b6
--- /dev/null
+++ b/testcases/lib/tst_run_shell.c
@@ -0,0 +1,491 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2024 Cyril Hrubis <chrubis@suse.cz>
+ */
+#include <sys/mount.h>
+
+#define TST_NO_DEFAULT_MAIN
+#include "tst_test.h"
+#include "tst_safe_stdio.h"
+#include "ujson.h"
+
+static char *shell_filename;
+
+static void run_shell(void)
+{
+	tst_run_script(shell_filename, NULL);
+}
+
+static void run_shell_tcnt(unsigned int n)
+{
+	char buf[128];
+	char *const params[] = {buf, NULL};
+
+	snprintf(buf, sizeof(buf), "%u", n);
+
+	tst_run_script(shell_filename, params);
+}
+
+struct tst_test test = {
+	.runs_script = 1,
+};
+
+static void print_help(void)
+{
+	printf("Usage: tst_shell_loader ltp_shell_test.sh ...\n");
+}
+
+static char *metadata;
+static size_t metadata_size;
+static size_t metadata_used;
+
+static void metadata_append(const char *line)
+{
+	size_t linelen = strlen(line);
+
+	if (metadata_size - metadata_used < linelen + 1) {
+		metadata_size += 4096;
+		metadata = SAFE_REALLOC(metadata, metadata_size);
+	}
+
+	strcpy(metadata + metadata_used, line);
+	metadata_used += linelen;
+}
+
+enum test_attr_ids {
+	ALL_FILESYSTEMS,
+	DEV_MIN_SIZE,
+	FILESYSTEMS,
+	FORMAT_DEVICE,
+	MIN_CPUS,
+	MIN_MEM_AVAIL,
+	MIN_KVER,
+	MIN_SWAP_AVAIL,
+	MNTPOINT,
+	MOUNT_DEVICE,
+	NEEDS_ABI_BITS,
+	NEEDS_CMDS,
+	NEEDS_DEVFS,
+	NEEDS_DEVICE,
+	NEEDS_DRIVERS,
+	NEEDS_HUGETLBFS,
+	NEEDS_KCONFIGS,
+	NEEDS_ROFS,
+	NEEDS_ROOT,
+	NEEDS_TMPDIR,
+	RESTORE_WALLCLOCK,
+	SKIP_FILESYSTEMS,
+	SKIP_IN_COMPAT,
+	SKIP_IN_LOCKDOWN,
+	SKIP_IN_SECUREBOOT,
+	SUPPORTED_ARCHS,
+	TAGS,
+	TAINT_CHECK,
+	TCNT,
+};
+
+static ujson_obj_attr test_attrs[] = {
+	UJSON_OBJ_ATTR_IDX(ALL_FILESYSTEMS, "all_filesystems", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(DEV_MIN_SIZE, "dev_min_size", UJSON_INT),
+	UJSON_OBJ_ATTR_IDX(FILESYSTEMS, "filesystems", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(FORMAT_DEVICE, "format_device", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(MIN_CPUS, "min_cpus", UJSON_INT),
+	UJSON_OBJ_ATTR_IDX(MIN_MEM_AVAIL, "min_mem_avail", UJSON_INT),
+	UJSON_OBJ_ATTR_IDX(MIN_KVER, "min_kver", UJSON_STR),
+	UJSON_OBJ_ATTR_IDX(MIN_SWAP_AVAIL, "min_swap_avail", UJSON_INT),
+	UJSON_OBJ_ATTR_IDX(MNTPOINT, "mntpoint", UJSON_STR),
+	UJSON_OBJ_ATTR_IDX(MOUNT_DEVICE, "mount_device", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(NEEDS_ABI_BITS, "needs_abi_bits", UJSON_INT),
+	UJSON_OBJ_ATTR_IDX(NEEDS_CMDS, "needs_cmds", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(NEEDS_DEVFS, "needs_devfs", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(NEEDS_DEVICE, "needs_device", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(NEEDS_DRIVERS, "needs_drivers", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(NEEDS_HUGETLBFS, "needs_hugetlbfs", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(NEEDS_KCONFIGS, "needs_kconfigs", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(NEEDS_ROFS, "needs_rofs", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(NEEDS_ROOT, "needs_root", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(NEEDS_TMPDIR, "needs_tmpdir", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(RESTORE_WALLCLOCK, "restore_wallclock", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(SKIP_FILESYSTEMS, "skip_filesystems", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(SKIP_IN_COMPAT, "skip_in_compat", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(SKIP_IN_LOCKDOWN, "skip_in_lockdown", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(SKIP_IN_SECUREBOOT, "skip_in_secureboot", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(SUPPORTED_ARCHS, "supported_archs", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(TAGS, "tags", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(TAINT_CHECK, "taint_check", UJSON_BOOL),
+	UJSON_OBJ_ATTR_IDX(TCNT, "tcnt", UJSON_INT)
+};
+
+static ujson_obj test_obj = {
+	.attrs = test_attrs,
+	.attr_cnt = UJSON_ARRAY_SIZE(test_attrs),
+};
+
+static const char *const *parse_strarr(ujson_reader *reader, ujson_val *val)
+{
+	unsigned int cnt = 0, i = 0;
+	char **ret;
+
+	ujson_reader_state state = ujson_reader_state_save(reader);
+
+	UJSON_ARR_FOREACH(reader, val) {
+		if (val->type != UJSON_STR) {
+			ujson_err(reader, "Expected string!");
+			return NULL;
+		}
+
+		cnt++;
+	}
+
+	ujson_reader_state_load(reader, state);
+
+	ret = SAFE_MALLOC(sizeof(char*) * (cnt + 1));
+
+	UJSON_ARR_FOREACH(reader, val) {
+		ret[i++] = strdup(val->val_str);
+	}
+
+	ret[i] = NULL;
+
+	return (const char *const *)ret;
+}
+
+enum fs_ids {
+	MKFS_OPTS,
+	MKFS_SIZE_OPT,
+	MNT_FLAGS,
+	TYPE,
+};
+
+static ujson_obj_attr fs_attrs[] = {
+	UJSON_OBJ_ATTR_IDX(MKFS_OPTS, "mkfs_opts", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(MKFS_SIZE_OPT, "mkfs_size_opt", UJSON_STR),
+	UJSON_OBJ_ATTR_IDX(MNT_FLAGS, "mnt_flags", UJSON_ARR),
+	UJSON_OBJ_ATTR_IDX(TYPE, "type", UJSON_STR),
+};
+
+static ujson_obj fs_obj = {
+	.attrs = fs_attrs,
+	.attr_cnt = UJSON_ARRAY_SIZE(fs_attrs),
+};
+
+static int parse_mnt_flags(ujson_reader *reader, ujson_val *val)
+{
+	int ret = 0;
+
+	UJSON_ARR_FOREACH(reader, val) {
+		if (val->type != UJSON_STR) {
+			ujson_err(reader, "Expected string!");
+			return ret;
+		}
+
+		if (!strcmp(val->val_str, "RDONLY"))
+			ret |= MS_RDONLY;
+		else if (!strcmp(val->val_str, "NOATIME"))
+			ret |= MS_NOATIME;
+		else if (!strcmp(val->val_str, "NOEXEC"))
+			ret |= MS_NOEXEC;
+		else if (!strcmp(val->val_str, "NOSUID"))
+			ret |= MS_NOSUID;
+		else
+			ujson_err(reader, "Invalid mount flag");
+	}
+
+	return ret;
+}
+
+static struct tst_fs *parse_filesystems(ujson_reader *reader, ujson_val *val)
+{
+	unsigned int i = 0, cnt = 0;
+	struct tst_fs *ret;
+
+	ujson_reader_state state = ujson_reader_state_save(reader);
+
+	UJSON_ARR_FOREACH(reader, val) {
+		if (val->type != UJSON_OBJ) {
+			ujson_err(reader, "Expected object!");
+			return NULL;
+		}
+		ujson_obj_skip(reader);
+		cnt++;
+	}
+
+	ujson_reader_state_load(reader, state);
+
+	ret = SAFE_MALLOC(sizeof(struct tst_fs) * (cnt + 1));
+	memset(ret, 0, sizeof(*ret) * (cnt+1));
+
+	UJSON_ARR_FOREACH(reader, val) {
+		UJSON_OBJ_FOREACH_FILTER(reader, val, &fs_obj, ujson_empty_obj) {
+			switch ((enum fs_ids)val->idx) {
+			case MKFS_OPTS:
+				ret[i].mkfs_opts = parse_strarr(reader, val);
+			break;
+			case MKFS_SIZE_OPT:
+				ret[i].mkfs_size_opt = strdup(val->val_str);
+			break;
+			case MNT_FLAGS:
+				ret[i].mnt_flags = parse_mnt_flags(reader, val);
+			break;
+			case TYPE:
+				ret[i].type = strdup(val->val_str);
+			break;
+			}
+
+		}
+
+		i++;
+	}
+
+	return ret;
+}
+
+static struct tst_tag *parse_tags(ujson_reader *reader, ujson_val *val)
+{
+	unsigned int i = 0, cnt = 0;
+	struct tst_tag *ret;
+
+	ujson_reader_state state = ujson_reader_state_save(reader);
+
+	UJSON_ARR_FOREACH(reader, val) {
+		if (val->type != UJSON_ARR) {
+			ujson_err(reader, "Expected array!");
+			return NULL;
+		}
+		ujson_arr_skip(reader);
+		cnt++;
+	}
+
+	ujson_reader_state_load(reader, state);
+
+	ret = SAFE_MALLOC(sizeof(struct tst_tag) * (cnt + 1));
+	memset(&ret[cnt], 0, sizeof(ret[cnt]));
+
+	UJSON_ARR_FOREACH(reader, val) {
+		char *name = NULL;
+		char *value = NULL;
+
+		UJSON_ARR_FOREACH(reader, val) {
+			if (val->type != UJSON_STR) {
+				ujson_err(reader, "Expected string!");
+				return NULL;
+			}
+
+			if (!name) {
+				name = strdup(val->val_str);
+			} else if (!value) {
+				value = strdup(val->val_str);
+			} else {
+				ujson_err(reader, "Expected only two members!");
+				return NULL;
+			}
+		}
+
+		ret[i].name = name;
+		ret[i].value = value;
+		i++;
+	}
+
+	return ret;
+}
+
+static void parse_metadata(void)
+{
+	ujson_reader reader = UJSON_READER_INIT(metadata, metadata_used, UJSON_READER_STRICT);
+	char str_buf[128];
+	ujson_val val = UJSON_VAL_INIT(str_buf, sizeof(str_buf));
+
+	UJSON_OBJ_FOREACH_FILTER(&reader, &val, &test_obj, ujson_empty_obj) {
+		switch ((enum test_attr_ids)val.idx) {
+		case ALL_FILESYSTEMS:
+			test.all_filesystems = val.val_bool;
+		break;
+		case DEV_MIN_SIZE:
+			if (val.val_int <= 0)
+				ujson_err(&reader, "Device size must be > 0");
+			else
+				test.dev_min_size = val.val_int;
+		break;
+		case FILESYSTEMS:
+			test.filesystems = parse_filesystems(&reader, &val);
+		break;
+		case FORMAT_DEVICE:
+			test.format_device = val.val_bool;
+		break;
+		case MIN_CPUS:
+			if (val.val_int <= 0)
+				ujson_err(&reader, "Minimal number of cpus must be > 0");
+			else
+				test.min_cpus = val.val_int;
+		break;
+		case MIN_MEM_AVAIL:
+			if (val.val_int <= 0)
+				ujson_err(&reader, "Minimal available memory size must be > 0");
+			else
+				test.min_mem_avail = val.val_int;
+		break;
+		case MIN_KVER:
+			test.min_kver = strdup(val.val_str);
+		break;
+		case MIN_SWAP_AVAIL:
+			if (val.val_int <= 0)
+				ujson_err(&reader, "Minimal available swap size must be > 0");
+			else
+				test.min_swap_avail = val.val_int;
+		break;
+		case MNTPOINT:
+			test.mntpoint = strdup(val.val_str);
+		break;
+		case MOUNT_DEVICE:
+			test.mount_device = val.val_bool;
+		break;
+		case NEEDS_ABI_BITS:
+			if (val.val_int == 32 || val.val_int == 64)
+				test.needs_abi_bits = val.val_int;
+			else
+				ujson_err(&reader, "ABI bits must be 32 or 64");
+		break;
+		case NEEDS_CMDS:
+			test.needs_cmds = parse_strarr(&reader, &val);
+		break;
+		case NEEDS_DEVFS:
+			test.needs_devfs = val.val_bool;
+		break;
+		case NEEDS_DEVICE:
+			test.needs_device = val.val_bool;
+		break;
+		case NEEDS_DRIVERS:
+			test.needs_drivers = parse_strarr(&reader, &val);
+		break;
+		case NEEDS_HUGETLBFS:
+			test.needs_hugetlbfs = val.val_bool;
+		break;
+		case NEEDS_KCONFIGS:
+			test.needs_kconfigs = parse_strarr(&reader, &val);
+		break;
+		case NEEDS_ROFS:
+			test.needs_rofs = val.val_bool;
+		break;
+		case NEEDS_ROOT:
+			test.needs_root = val.val_bool;
+		break;
+		case NEEDS_TMPDIR:
+			test.needs_tmpdir = val.val_bool;
+		break;
+		case RESTORE_WALLCLOCK:
+			test.restore_wallclock = val.val_bool;
+		break;
+		case SKIP_FILESYSTEMS:
+			test.skip_filesystems = parse_strarr(&reader, &val);
+		break;
+		case SKIP_IN_COMPAT:
+			test.skip_in_compat = val.val_bool;
+		break;
+		case SKIP_IN_LOCKDOWN:
+			test.skip_in_lockdown = val.val_bool;
+		break;
+		case SKIP_IN_SECUREBOOT:
+			test.skip_in_secureboot = val.val_bool;
+		break;
+		case SUPPORTED_ARCHS:
+			test.supported_archs = parse_strarr(&reader, &val);
+		break;
+		case TAGS:
+			test.tags = parse_tags(&reader, &val);
+		break;
+		case TAINT_CHECK:
+			test.taint_check = val.val_bool;
+		break;
+		case TCNT:
+			if (val.val_int <= 0)
+				ujson_err(&reader, "Number of tests must be > 0");
+			else
+				test.tcnt = val.val_int;
+		break;
+		}
+	}
+
+	ujson_reader_finish(&reader);
+
+	if (ujson_reader_err(&reader))
+		tst_brk(TBROK, "Invalid metadata");
+}
+
+enum parser_state {
+	PAR_NONE,
+	PAR_ESC,
+	PAR_DOC,
+	PAR_ENV,
+};
+
+static void extract_metadata(void)
+{
+	FILE *f;
+	char line[4096];
+	char path[4096];
+	enum parser_state state = PAR_NONE;
+
+	if (tst_get_path(shell_filename, path, sizeof(path)) == -1)
+		tst_brk(TBROK, "Failed to find %s in $PATH", shell_filename);
+
+	f = SAFE_FOPEN(path, "r");
+
+	while (fgets(line, sizeof(line), f)) {
+		switch (state) {
+		case PAR_NONE:
+			if (!strcmp(line, "# ---\n"))
+				state = PAR_ESC;
+		break;
+		case PAR_ESC:
+			if (!strcmp(line, "# env\n"))
+				state = PAR_ENV;
+			else if (!strcmp(line, "# doc\n"))
+				state = PAR_DOC;
+			else
+				tst_brk(TBROK, "Unknown comment block %s", line);
+		break;
+		case PAR_ENV:
+			if (!strcmp(line, "# ---\n"))
+				state = PAR_NONE;
+			else
+				metadata_append(line + 2);
+		break;
+		case PAR_DOC:
+			if (!strcmp(line, "# ---\n"))
+				state = PAR_NONE;
+		break;
+		}
+	}
+
+	fclose(f);
+}
+
+static void prepare_test_struct(void)
+{
+	extract_metadata();
+
+	if (metadata)
+		parse_metadata();
+	else
+		tst_brk(TBROK, "No metadata found!");
+}
+
+int main(int argc, char *argv[])
+{
+	if (argc < 2)
+		goto help;
+
+	shell_filename = argv[1];
+
+	prepare_test_struct();
+
+	if (test.tcnt)
+		test.test = run_shell_tcnt;
+	else
+		test.test_all = run_shell;
+
+	tst_run_tcases(argc - 1, argv + 1, &test);
+help:
+	print_help();
+	return 1;
+}
-- 
2.44.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 4/4] testcaes/lib: Add shell loader

[Andrea Cervesato via ltp]

Hi!

On 8/27/24 14:02, Cyril Hrubis wrote:
> This commit implements a shell loader so that we don't have to write a C
> loader for each LTP shell test. The idea is simple, the loader parses
> the shell test and prepares the tst_test structure accordingly, then
> runs the actual shell test.
>
> The format for the metadata in the shell test was choosen to be JSON
> because:
>
> - I didn't want to invent an adhoc format and JSON is perfect for
>    serializing data structures
> - The metadata parser for shell test will be trivial, it will just pick
>    the JSON from the comment, no parsing will be required
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> Reviewed-by: Richard Palethorpe <io@richiejp.com>
> ---
>   include/tst_test.h                            |   2 +-
>   testcases/lib/.gitignore                      |   1 +
>   testcases/lib/Makefile                        |   6 +-
>   testcases/lib/run_tests.sh                    |  21 +
>   testcases/lib/tests/shell_loader.sh           |  26 +
>   .../lib/tests/shell_loader_all_filesystems.sh |  27 +
>   .../lib/tests/shell_loader_filesystems.sh     |  33 ++
>   .../lib/tests/shell_loader_invalid_block.sh   |  26 +
>   .../tests/shell_loader_invalid_metadata.sh    |  15 +
>   testcases/lib/tests/shell_loader_kconfigs.sh  |  12 +
>   .../lib/tests/shell_loader_no_metadata.sh     |   8 +
>   .../lib/tests/shell_loader_supported_archs.sh |  12 +
>   testcases/lib/tests/shell_loader_tags.sh      |  15 +
>   testcases/lib/tests/shell_loader_tcnt.sh      |  15 +
>   .../lib/tests/shell_loader_wrong_metadata.sh  |  15 +
>   testcases/lib/tst_env.sh                      |   4 +
>   testcases/lib/tst_loader.sh                   |  11 +
>   testcases/lib/tst_run_shell.c                 | 491 ++++++++++++++++++
>   18 files changed, 738 insertions(+), 2 deletions(-)
>   create mode 100755 testcases/lib/tests/shell_loader.sh
>   create mode 100755 testcases/lib/tests/shell_loader_all_filesystems.sh
>   create mode 100755 testcases/lib/tests/shell_loader_filesystems.sh
>   create mode 100755 testcases/lib/tests/shell_loader_invalid_block.sh
>   create mode 100755 testcases/lib/tests/shell_loader_invalid_metadata.sh
>   create mode 100755 testcases/lib/tests/shell_loader_kconfigs.sh
>   create mode 100755 testcases/lib/tests/shell_loader_no_metadata.sh
>   create mode 100755 testcases/lib/tests/shell_loader_supported_archs.sh
>   create mode 100755 testcases/lib/tests/shell_loader_tags.sh
>   create mode 100755 testcases/lib/tests/shell_loader_tcnt.sh
>   create mode 100755 testcases/lib/tests/shell_loader_wrong_metadata.sh
>   create mode 100644 testcases/lib/tst_loader.sh
>   create mode 100644 testcases/lib/tst_run_shell.c
>
> diff --git a/include/tst_test.h b/include/tst_test.h
> index 9871676a5..d0fa84a71 100644
> --- a/include/tst_test.h
> +++ b/include/tst_test.h
> @@ -274,7 +274,7 @@ struct tst_fs {
>   	const char *const *mkfs_opts;
>   	const char *mkfs_size_opt;
>   
> -	const unsigned int mnt_flags;
> +	unsigned int mnt_flags;
>   	const void *mnt_data;
>   };
>   
> diff --git a/testcases/lib/.gitignore b/testcases/lib/.gitignore
> index d0dacf62a..385f3c3ca 100644
> --- a/testcases/lib/.gitignore
> +++ b/testcases/lib/.gitignore
> @@ -24,3 +24,4 @@
>   /tst_supported_fs
>   /tst_timeout_kill
>   /tst_res_
> +/tst_run_shell
> diff --git a/testcases/lib/Makefile b/testcases/lib/Makefile
> index 928d76d62..b3a9181c1 100644
> --- a/testcases/lib/Makefile
> +++ b/testcases/lib/Makefile
> @@ -4,6 +4,9 @@
>   
>   top_srcdir		?= ../..
>   
> +LTPLIBS = ujson
> +tst_run_shell: LTPLDLIBS = -lujson
> +
>   include $(top_srcdir)/include/mk/testcases.mk
>   
>   INSTALL_TARGETS		:= *.sh
> @@ -13,6 +16,7 @@ MAKE_TARGETS		:= tst_sleep tst_random tst_checkpoint tst_rod tst_kvcmp\
>   			   tst_getconf tst_supported_fs tst_check_drivers tst_get_unused_port\
>   			   tst_get_median tst_hexdump tst_get_free_pids tst_timeout_kill\
>   			   tst_check_kconfigs tst_cgctl tst_fsfreeze tst_ns_create tst_ns_exec\
> -			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_
> +			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_\
> +			   tst_run_shell
>   
>   include $(top_srcdir)/include/mk/generic_trunk_target.mk
> diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh
> index 60e7d1bcf..e30065f1d 100755
> --- a/testcases/lib/run_tests.sh
> +++ b/testcases/lib/run_tests.sh
> @@ -9,3 +9,24 @@ for i in `seq -w 01 06`; do
>   	echo
>   	./tests/shell_test$i
>   done
> +
> +for i in shell_loader.sh shell_loader_all_filesystems.sh shell_loader_no_metadata.sh \
> +	 shell_loader_wrong_metadata.sh shell_loader_invalid_metadata.sh\
> +	 shell_loader_supported_archs.sh shell_loader_filesystems.sh\
> +	 shell_loader_tcnt.sh shell_loader_kconfigs.sh shell_loader_tags.sh \
> +	 shell_loader_invalid_block.sh; do
> +	echo
> +	echo "*** Running $i ***"
> +	echo
> +	$i
> +done
> +
> +echo
> +echo "*** Testing LTP test -h option ***"
> +echo
> +shell_loader.sh -h
> +
> +echo
> +echo "*** Testing LTP test -i option ***"
> +echo
> +shell_loader.sh -i 2
> diff --git a/testcases/lib/tests/shell_loader.sh b/testcases/lib/tests/shell_loader.sh
> new file mode 100755
> index 000000000..df7f0c0af
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader.sh
> @@ -0,0 +1,26 @@
> +#!/bin/sh
> +#
> +# ---
> +# doc
> +#
> +# [Description]
> +#
> +# This is a simple shell test loader example.
> +# ---
> +#
> +# ---
> +# env
> +# {
> +#  "needs_tmpdir": true
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "Shell loader works fine!"
> +case "$PWD" in
> +	/tmp/*)
> +		tst_res TPASS "We are running in temp directory in $PWD";;
> +	*)
> +		tst_res TFAIL "We are not running in temp directory but $PWD";;
> +esac
> diff --git a/testcases/lib/tests/shell_loader_all_filesystems.sh b/testcases/lib/tests/shell_loader_all_filesystems.sh
> new file mode 100755
> index 000000000..d5943c335
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_all_filesystems.sh
> @@ -0,0 +1,27 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "needs_root": true,
> +#  "mount_device": true,
> +#  "all_filesystems": true,
> +#  "mntpoint": "ltp_mntpoint"
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TINFO "In shell"
> +
> +mntpath=$(realpath ltp_mntpoint)
> +mounted=$(grep $mntpath /proc/mounts)
> +
> +if [ -n "$mounted" ]; then
> +	device=$(echo $mounted |cut -d' ' -f 1)
> +	path=$(echo $mounted |cut -d' ' -f 2)
> +
> +	tst_res TPASS "$device mounted at $path"
> +else
> +	tst_res TFAIL "Device not mounted!"
> +fi
> diff --git a/testcases/lib/tests/shell_loader_filesystems.sh b/testcases/lib/tests/shell_loader_filesystems.sh
> new file mode 100755
> index 000000000..5d8aa9808
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_filesystems.sh
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "mount_device": true,
> +#  "mntpoint": "ltp_mntpoint",
> +#  "filesystems": [
> +#   {
> +#    "type": "btrfs"
> +#   },
> +#   {
> +#    "type": "xfs",
> +#    "mkfs_opts": ["-m", "reflink=1"]
> +#   }
> +#  ]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TINFO "In shell"
> +
> +mntpoint=$(realpath ltp_mntpoint)
> +mounted=$(grep $mntpoint /proc/mounts)
> +
> +if [ -n "$mounted" ]; then
> +	fs=$(echo $mounted |cut -d' ' -f 3)
> +
> +	tst_res TPASS "Mounted device formatted with $fs"
> +else
> +	tst_res TFAIL "Device not mounted!"
> +fi
> diff --git a/testcases/lib/tests/shell_loader_invalid_block.sh b/testcases/lib/tests/shell_loader_invalid_block.sh
> new file mode 100755
> index 000000000..f41de04fd
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_invalid_block.sh
> @@ -0,0 +1,26 @@
> +#!/bin/sh
> +#
> +# ---
> +# doc
> +#
> +# [Description]
> +#
> +# This is a simple shell test loader example.
> +# ---
> +#
> +# ---
> +# env
> +# {
> +#  "needs_tmpdir": true
> +# }
> +# ---
> +#
> +# ---
> +# inv
> +#
> +# This is an invalid block that breaks the test.
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "This should pass!"
> diff --git a/testcases/lib/tests/shell_loader_invalid_metadata.sh b/testcases/lib/tests/shell_loader_invalid_metadata.sh
> new file mode 100755
> index 000000000..c10b00f1b
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_invalid_metadata.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# This test has wrong metadata and should not be run
> +#
> +# ---
> +# env
> +# {
> +#  {"needs_tmpdir": 42,
> +# }
> +# ---
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Shell loader should TBROK the test"
> diff --git a/testcases/lib/tests/shell_loader_kconfigs.sh b/testcases/lib/tests/shell_loader_kconfigs.sh
> new file mode 100755
> index 000000000..7e9a1dce7
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_kconfigs.sh
> @@ -0,0 +1,12 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "needs_kconfigs": ["CONFIG_NUMA=y"]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "Shell loader works fine!"
> diff --git a/testcases/lib/tests/shell_loader_no_metadata.sh b/testcases/lib/tests/shell_loader_no_metadata.sh
> new file mode 100755
> index 000000000..60ba8b889
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_no_metadata.sh
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +#
> +# This test has no metadata and should not be executed
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Shell loader should TBROK the test"
> diff --git a/testcases/lib/tests/shell_loader_supported_archs.sh b/testcases/lib/tests/shell_loader_supported_archs.sh
> new file mode 100755
> index 000000000..45213f840
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_supported_archs.sh
> @@ -0,0 +1,12 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "supported_archs": ["x86", "ppc64", "x86_64"]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "We are running on supported architecture"
> diff --git a/testcases/lib/tests/shell_loader_tags.sh b/testcases/lib/tests/shell_loader_tags.sh
> new file mode 100755
> index 000000000..a6278c37d
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_tags.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "tags": [
> +#   ["linux-git", "832478cd342ab"],
> +#   ["CVE", "2099-999"]
> +#  ]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Fails the test so that tags are shown."
> diff --git a/testcases/lib/tests/shell_loader_tcnt.sh b/testcases/lib/tests/shell_loader_tcnt.sh
> new file mode 100755
> index 000000000..81fc08179
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_tcnt.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# The script should be executed tcnt times and the iteration number should be in $1
> +#
> +# ---
> +# env
> +# {
> +#  "tcnt": 2
> +# }
> +# ---
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "Iteration $1"
> diff --git a/testcases/lib/tests/shell_loader_wrong_metadata.sh b/testcases/lib/tests/shell_loader_wrong_metadata.sh
> new file mode 100755
> index 000000000..752e25eea
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_wrong_metadata.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# This test has wrong metadata and should not be run
> +#
> +# ---
> +# env
> +# {
> +#  "needs_tmpdir": 42,
> +# }
> +# ---
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Shell loader should TBROK the test"
> diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
> index 948bc5024..67ba80744 100644
> --- a/testcases/lib/tst_env.sh
> +++ b/testcases/lib/tst_env.sh
> @@ -1,4 +1,8 @@
>   #!/bin/sh
> +#
> +# This is a minimal test environment for a shell scripts executed from C by
> +# tst_run_shell() function. Shell tests must use the tst_loader.sh instead!
> +#
>   
>   tst_script_name=$(basename $0)
>   
> diff --git a/testcases/lib/tst_loader.sh b/testcases/lib/tst_loader.sh
> new file mode 100644
> index 000000000..ed04d0340
> --- /dev/null
> +++ b/testcases/lib/tst_loader.sh
> @@ -0,0 +1,11 @@
> +#!/bin/sh
> +#
> +# This is a loader for shell tests that use the C test library.
> +#
> +
> +if [ -z "$LTP_IPC_PATH" ]; then
> +	tst_run_shell $(basename "$0") "$@"
> +	exit $?
> +else
> +	. tst_env.sh
> +fi
> diff --git a/testcases/lib/tst_run_shell.c b/testcases/lib/tst_run_shell.c
> new file mode 100644
> index 000000000..8ed0f21b6
> --- /dev/null
> +++ b/testcases/lib/tst_run_shell.c
> @@ -0,0 +1,491 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2024 Cyril Hrubis <chrubis@suse.cz>
> + */
> +#include <sys/mount.h>
> +
> +#define TST_NO_DEFAULT_MAIN
> +#include "tst_test.h"
> +#include "tst_safe_stdio.h"
> +#include "ujson.h"
> +
> +static char *shell_filename;
> +
> +static void run_shell(void)
> +{
> +	tst_run_script(shell_filename, NULL);
> +}
> +
> +static void run_shell_tcnt(unsigned int n)
> +{
> +	char buf[128];
> +	char *const params[] = {buf, NULL};
> +
> +	snprintf(buf, sizeof(buf), "%u", n);
> +
> +	tst_run_script(shell_filename, params);
> +}
> +
> +struct tst_test test = {
> +	.runs_script = 1,
> +};
> +
> +static void print_help(void)
> +{
> +	printf("Usage: tst_shell_loader ltp_shell_test.sh ...\n");
> +}
> +
> +static char *metadata;
> +static size_t metadata_size;
> +static size_t metadata_used;
> +
> +static void metadata_append(const char *line)
> +{
> +	size_t linelen = strlen(line);
> +
> +	if (metadata_size - metadata_used < linelen + 1) {
> +		metadata_size += 4096;
> +		metadata = SAFE_REALLOC(metadata, metadata_size);
> +	}
> +
> +	strcpy(metadata + metadata_used, line);
> +	metadata_used += linelen;
> +}
> +
> +enum test_attr_ids {
> +	ALL_FILESYSTEMS,
> +	DEV_MIN_SIZE,
> +	FILESYSTEMS,
> +	FORMAT_DEVICE,
> +	MIN_CPUS,
> +	MIN_MEM_AVAIL,
> +	MIN_KVER,
> +	MIN_SWAP_AVAIL,
> +	MNTPOINT,
> +	MOUNT_DEVICE,
> +	NEEDS_ABI_BITS,
> +	NEEDS_CMDS,
> +	NEEDS_DEVFS,
> +	NEEDS_DEVICE,
> +	NEEDS_DRIVERS,
> +	NEEDS_HUGETLBFS,
> +	NEEDS_KCONFIGS,
> +	NEEDS_ROFS,
> +	NEEDS_ROOT,
> +	NEEDS_TMPDIR,
> +	RESTORE_WALLCLOCK,
> +	SKIP_FILESYSTEMS,
> +	SKIP_IN_COMPAT,
> +	SKIP_IN_LOCKDOWN,
> +	SKIP_IN_SECUREBOOT,
> +	SUPPORTED_ARCHS,
> +	TAGS,
> +	TAINT_CHECK,
> +	TCNT,
> +};
> +
> +static ujson_obj_attr test_attrs[] = {
> +	UJSON_OBJ_ATTR_IDX(ALL_FILESYSTEMS, "all_filesystems", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(DEV_MIN_SIZE, "dev_min_size", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(FILESYSTEMS, "filesystems", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(FORMAT_DEVICE, "format_device", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(MIN_CPUS, "min_cpus", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(MIN_MEM_AVAIL, "min_mem_avail", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(MIN_KVER, "min_kver", UJSON_STR),
> +	UJSON_OBJ_ATTR_IDX(MIN_SWAP_AVAIL, "min_swap_avail", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(MNTPOINT, "mntpoint", UJSON_STR),
> +	UJSON_OBJ_ATTR_IDX(MOUNT_DEVICE, "mount_device", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_ABI_BITS, "needs_abi_bits", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_CMDS, "needs_cmds", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_DEVFS, "needs_devfs", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_DEVICE, "needs_device", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_DRIVERS, "needs_drivers", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_HUGETLBFS, "needs_hugetlbfs", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_KCONFIGS, "needs_kconfigs", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_ROFS, "needs_rofs", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_ROOT, "needs_root", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_TMPDIR, "needs_tmpdir", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(RESTORE_WALLCLOCK, "restore_wallclock", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SKIP_FILESYSTEMS, "skip_filesystems", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(SKIP_IN_COMPAT, "skip_in_compat", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SKIP_IN_LOCKDOWN, "skip_in_lockdown", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SKIP_IN_SECUREBOOT, "skip_in_secureboot", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SUPPORTED_ARCHS, "supported_archs", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(TAGS, "tags", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(TAINT_CHECK, "taint_check", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(TCNT, "tcnt", UJSON_INT)
> +};
> +
> +static ujson_obj test_obj = {
> +	.attrs = test_attrs,
> +	.attr_cnt = UJSON_ARRAY_SIZE(test_attrs),
> +};
> +
> +static const char *const *parse_strarr(ujson_reader *reader, ujson_val *val)
> +{
> +	unsigned int cnt = 0, i = 0;
> +	char **ret;
> +
> +	ujson_reader_state state = ujson_reader_state_save(reader);
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_STR) {
> +			ujson_err(reader, "Expected string!");
> +			return NULL;
> +		}
> +
> +		cnt++;
> +	}
> +
> +	ujson_reader_state_load(reader, state);
> +
> +	ret = SAFE_MALLOC(sizeof(char*) * (cnt + 1));
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		ret[i++] = strdup(val->val_str);
> +	}
> +
> +	ret[i] = NULL;
> +
> +	return (const char *const *)ret;
> +}
> +
> +enum fs_ids {
> +	MKFS_OPTS,
> +	MKFS_SIZE_OPT,
> +	MNT_FLAGS,
> +	TYPE,
> +};
> +
> +static ujson_obj_attr fs_attrs[] = {
> +	UJSON_OBJ_ATTR_IDX(MKFS_OPTS, "mkfs_opts", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(MKFS_SIZE_OPT, "mkfs_size_opt", UJSON_STR),
> +	UJSON_OBJ_ATTR_IDX(MNT_FLAGS, "mnt_flags", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(TYPE, "type", UJSON_STR),
> +};
> +
> +static ujson_obj fs_obj = {
> +	.attrs = fs_attrs,
> +	.attr_cnt = UJSON_ARRAY_SIZE(fs_attrs),
> +};
> +
> +static int parse_mnt_flags(ujson_reader *reader, ujson_val *val)
> +{
> +	int ret = 0;
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_STR) {
> +			ujson_err(reader, "Expected string!");
> +			return ret;
> +		}
> +
> +		if (!strcmp(val->val_str, "RDONLY"))
> +			ret |= MS_RDONLY;
> +		else if (!strcmp(val->val_str, "NOATIME"))
> +			ret |= MS_NOATIME;
> +		else if (!strcmp(val->val_str, "NOEXEC"))
> +			ret |= MS_NOEXEC;
> +		else if (!strcmp(val->val_str, "NOSUID"))
> +			ret |= MS_NOSUID;
> +		else
> +			ujson_err(reader, "Invalid mount flag");
> +	}
> +
> +	return ret;
> +}
> +
> +static struct tst_fs *parse_filesystems(ujson_reader *reader, ujson_val *val)
> +{
> +	unsigned int i = 0, cnt = 0;
> +	struct tst_fs *ret;
> +
> +	ujson_reader_state state = ujson_reader_state_save(reader);
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_OBJ) {
> +			ujson_err(reader, "Expected object!");
> +			return NULL;
> +		}
> +		ujson_obj_skip(reader);
> +		cnt++;
> +	}
> +
> +	ujson_reader_state_load(reader, state);
> +
> +	ret = SAFE_MALLOC(sizeof(struct tst_fs) * (cnt + 1));
> +	memset(ret, 0, sizeof(*ret) * (cnt+1));
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		UJSON_OBJ_FOREACH_FILTER(reader, val, &fs_obj, ujson_empty_obj) {
> +			switch ((enum fs_ids)val->idx) {
> +			case MKFS_OPTS:
> +				ret[i].mkfs_opts = parse_strarr(reader, val);
> +			break;
> +			case MKFS_SIZE_OPT:
> +				ret[i].mkfs_size_opt = strdup(val->val_str);
> +			break;
> +			case MNT_FLAGS:
> +				ret[i].mnt_flags = parse_mnt_flags(reader, val);
> +			break;
> +			case TYPE:
> +				ret[i].type = strdup(val->val_str);
> +			break;
> +			}
> +
> +		}
> +
> +		i++;
> +	}
> +
> +	return ret;
> +}
> +
> +static struct tst_tag *parse_tags(ujson_reader *reader, ujson_val *val)
> +{
> +	unsigned int i = 0, cnt = 0;
> +	struct tst_tag *ret;
> +
> +	ujson_reader_state state = ujson_reader_state_save(reader);
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_ARR) {
> +			ujson_err(reader, "Expected array!");
> +			return NULL;
> +		}
> +		ujson_arr_skip(reader);
> +		cnt++;
> +	}
> +
> +	ujson_reader_state_load(reader, state);
> +
> +	ret = SAFE_MALLOC(sizeof(struct tst_tag) * (cnt + 1));
> +	memset(&ret[cnt], 0, sizeof(ret[cnt]));
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		char *name = NULL;
> +		char *value = NULL;
> +
> +		UJSON_ARR_FOREACH(reader, val) {
> +			if (val->type != UJSON_STR) {
> +				ujson_err(reader, "Expected string!");
> +				return NULL;
> +			}
> +
> +			if (!name) {
> +				name = strdup(val->val_str);
> +			} else if (!value) {
> +				value = strdup(val->val_str);
> +			} else {
> +				ujson_err(reader, "Expected only two members!");
> +				return NULL;
> +			}
> +		}
> +
> +		ret[i].name = name;
> +		ret[i].value = value;
> +		i++;
> +	}
> +
> +	return ret;
> +}
> +
> +static void parse_metadata(void)
> +{
> +	ujson_reader reader = UJSON_READER_INIT(metadata, metadata_used, UJSON_READER_STRICT);
> +	char str_buf[128];
> +	ujson_val val = UJSON_VAL_INIT(str_buf, sizeof(str_buf));
> +
> +	UJSON_OBJ_FOREACH_FILTER(&reader, &val, &test_obj, ujson_empty_obj) {
> +		switch ((enum test_attr_ids)val.idx) {
> +		case ALL_FILESYSTEMS:
> +			test.all_filesystems = val.val_bool;
> +		break;
> +		case DEV_MIN_SIZE:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Device size must be > 0");
> +			else
> +				test.dev_min_size = val.val_int;
> +		break;
> +		case FILESYSTEMS:
> +			test.filesystems = parse_filesystems(&reader, &val);
> +		break;
> +		case FORMAT_DEVICE:
> +			test.format_device = val.val_bool;
> +		break;
> +		case MIN_CPUS:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Minimal number of cpus must be > 0");
> +			else
> +				test.min_cpus = val.val_int;
> +		break;
> +		case MIN_MEM_AVAIL:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Minimal available memory size must be > 0");
> +			else
> +				test.min_mem_avail = val.val_int;
> +		break;
> +		case MIN_KVER:
> +			test.min_kver = strdup(val.val_str);
> +		break;
> +		case MIN_SWAP_AVAIL:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Minimal available swap size must be > 0");
> +			else
> +				test.min_swap_avail = val.val_int;
> +		break;
> +		case MNTPOINT:
> +			test.mntpoint = strdup(val.val_str);
> +		break;
> +		case MOUNT_DEVICE:
> +			test.mount_device = val.val_bool;
> +		break;
> +		case NEEDS_ABI_BITS:
> +			if (val.val_int == 32 || val.val_int == 64)
> +				test.needs_abi_bits = val.val_int;
> +			else
> +				ujson_err(&reader, "ABI bits must be 32 or 64");
> +		break;
> +		case NEEDS_CMDS:
> +			test.needs_cmds = parse_strarr(&reader, &val);
> +		break;
> +		case NEEDS_DEVFS:
> +			test.needs_devfs = val.val_bool;
> +		break;
> +		case NEEDS_DEVICE:
> +			test.needs_device = val.val_bool;
> +		break;
> +		case NEEDS_DRIVERS:
> +			test.needs_drivers = parse_strarr(&reader, &val);
> +		break;
> +		case NEEDS_HUGETLBFS:
> +			test.needs_hugetlbfs = val.val_bool;
> +		break;
> +		case NEEDS_KCONFIGS:
> +			test.needs_kconfigs = parse_strarr(&reader, &val);
> +		break;
> +		case NEEDS_ROFS:
> +			test.needs_rofs = val.val_bool;
> +		break;
> +		case NEEDS_ROOT:
> +			test.needs_root = val.val_bool;
> +		break;
> +		case NEEDS_TMPDIR:
> +			test.needs_tmpdir = val.val_bool;
> +		break;
> +		case RESTORE_WALLCLOCK:
> +			test.restore_wallclock = val.val_bool;
> +		break;
> +		case SKIP_FILESYSTEMS:
> +			test.skip_filesystems = parse_strarr(&reader, &val);
> +		break;
> +		case SKIP_IN_COMPAT:
> +			test.skip_in_compat = val.val_bool;
> +		break;
> +		case SKIP_IN_LOCKDOWN:
> +			test.skip_in_lockdown = val.val_bool;
> +		break;
> +		case SKIP_IN_SECUREBOOT:
> +			test.skip_in_secureboot = val.val_bool;
> +		break;
> +		case SUPPORTED_ARCHS:
> +			test.supported_archs = parse_strarr(&reader, &val);
> +		break;
> +		case TAGS:
> +			test.tags = parse_tags(&reader, &val);
> +		break;
> +		case TAINT_CHECK:
> +			test.taint_check = val.val_bool;
> +		break;
> +		case TCNT:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Number of tests must be > 0");
> +			else
> +				test.tcnt = val.val_int;
> +		break;
> +		}
> +	}
> +
> +	ujson_reader_finish(&reader);
> +
> +	if (ujson_reader_err(&reader))
> +		tst_brk(TBROK, "Invalid metadata");
> +}
> +
> +enum parser_state {
> +	PAR_NONE,
> +	PAR_ESC,
> +	PAR_DOC,
> +	PAR_ENV,
> +};
> +
> +static void extract_metadata(void)
> +{
> +	FILE *f;
> +	char line[4096];
> +	char path[4096];
> +	enum parser_state state = PAR_NONE;
> +
> +	if (tst_get_path(shell_filename, path, sizeof(path)) == -1)
> +		tst_brk(TBROK, "Failed to find %s in $PATH", shell_filename);
> +
> +	f = SAFE_FOPEN(path, "r");
> +
> +	while (fgets(line, sizeof(line), f)) {
> +		switch (state) {
> +		case PAR_NONE:
> +			if (!strcmp(line, "# ---\n"))
What if user defines "#---" or "#   ---" ? IMHO it would be better to 
parse it following shell comments standards. In particular, "^#\s+---" 
using a *scan function.
> +				state = PAR_ESC;
> +		break;
> +		case PAR_ESC:
> +			if (!strcmp(line, "# env\n"))
Same apply here and to the others.
> +				state = PAR_ENV;
> +			else if (!strcmp(line, "# doc\n"))
> +				state = PAR_DOC;
> +			else
> +				tst_brk(TBROK, "Unknown comment block %s", line);
> +		break;
> +		case PAR_ENV:
> +			if (!strcmp(line, "# ---\n"))
> +				state = PAR_NONE;
> +			else
> +				metadata_append(line + 2);
> +		break;
> +		case PAR_DOC:
> +			if (!strcmp(line, "# ---\n"))
> +				state = PAR_NONE;
> +		break;
> +		}
> +	}
> +
> +	fclose(f);
> +}
> +
> +static void prepare_test_struct(void)
> +{
> +	extract_metadata();
> +
> +	if (metadata)
> +		parse_metadata();
> +	else
> +		tst_brk(TBROK, "No metadata found!");
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	if (argc < 2)
> +		goto help;
> +
> +	shell_filename = argv[1];
> +
> +	prepare_test_struct();
> +
> +	if (test.tcnt)
> +		test.test = run_shell_tcnt;
> +	else
> +		test.test_all = run_shell;
> +
> +	tst_run_tcases(argc - 1, argv + 1, &test);
> +help:
> +	print_help();
> +	return 1;
> +}
Andrea

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 4/4] testcaes/lib: Add shell loader

[Cyril Hrubis]

Hi!
> > +static void extract_metadata(void)
> > +{
> > +	FILE *f;
> > +	char line[4096];
> > +	char path[4096];
> > +	enum parser_state state = PAR_NONE;
> > +
> > +	if (tst_get_path(shell_filename, path, sizeof(path)) == -1)
> > +		tst_brk(TBROK, "Failed to find %s in $PATH", shell_filename);
> > +
> > +	f = SAFE_FOPEN(path, "r");
> > +
> > +	while (fgets(line, sizeof(line), f)) {
> > +		switch (state) {
> > +		case PAR_NONE:
> > +			if (!strcmp(line, "# ---\n"))
> What if user defines "#---" or "#   ---" ? IMHO it would be better to 
> parse it following shell comments standards. In particular, "^#\s+---" 
> using a *scan function.

If user sets the line any differently the test will end up with pretty
clear error message that metadata are missing, so I do not see this as
an issue.

> > +				state = PAR_ESC;
> > +		break;
> > +		case PAR_ESC:
> > +			if (!strcmp(line, "# env\n"))
> Same apply here and to the others.

Same here as well.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 4/4] testcaes/lib: Add shell loader

[Andrea Cervesato via ltp]

Also is there a reason why we are adding tests to testcases/lib/tests ?

On 8/27/24 14:02, Cyril Hrubis wrote:
> This commit implements a shell loader so that we don't have to write a C
> loader for each LTP shell test. The idea is simple, the loader parses
> the shell test and prepares the tst_test structure accordingly, then
> runs the actual shell test.
>
> The format for the metadata in the shell test was choosen to be JSON
> because:
>
> - I didn't want to invent an adhoc format and JSON is perfect for
>    serializing data structures
> - The metadata parser for shell test will be trivial, it will just pick
>    the JSON from the comment, no parsing will be required
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> Reviewed-by: Richard Palethorpe <io@richiejp.com>
> ---
>   include/tst_test.h                            |   2 +-
>   testcases/lib/.gitignore                      |   1 +
>   testcases/lib/Makefile                        |   6 +-
>   testcases/lib/run_tests.sh                    |  21 +
>   testcases/lib/tests/shell_loader.sh           |  26 +
>   .../lib/tests/shell_loader_all_filesystems.sh |  27 +
>   .../lib/tests/shell_loader_filesystems.sh     |  33 ++
>   .../lib/tests/shell_loader_invalid_block.sh   |  26 +
>   .../tests/shell_loader_invalid_metadata.sh    |  15 +
>   testcases/lib/tests/shell_loader_kconfigs.sh  |  12 +
>   .../lib/tests/shell_loader_no_metadata.sh     |   8 +
>   .../lib/tests/shell_loader_supported_archs.sh |  12 +
>   testcases/lib/tests/shell_loader_tags.sh      |  15 +
>   testcases/lib/tests/shell_loader_tcnt.sh      |  15 +
>   .../lib/tests/shell_loader_wrong_metadata.sh  |  15 +
>   testcases/lib/tst_env.sh                      |   4 +
>   testcases/lib/tst_loader.sh                   |  11 +
>   testcases/lib/tst_run_shell.c                 | 491 ++++++++++++++++++
>   18 files changed, 738 insertions(+), 2 deletions(-)
>   create mode 100755 testcases/lib/tests/shell_loader.sh
>   create mode 100755 testcases/lib/tests/shell_loader_all_filesystems.sh
>   create mode 100755 testcases/lib/tests/shell_loader_filesystems.sh
>   create mode 100755 testcases/lib/tests/shell_loader_invalid_block.sh
>   create mode 100755 testcases/lib/tests/shell_loader_invalid_metadata.sh
>   create mode 100755 testcases/lib/tests/shell_loader_kconfigs.sh
>   create mode 100755 testcases/lib/tests/shell_loader_no_metadata.sh
>   create mode 100755 testcases/lib/tests/shell_loader_supported_archs.sh
>   create mode 100755 testcases/lib/tests/shell_loader_tags.sh
>   create mode 100755 testcases/lib/tests/shell_loader_tcnt.sh
>   create mode 100755 testcases/lib/tests/shell_loader_wrong_metadata.sh
>   create mode 100644 testcases/lib/tst_loader.sh
>   create mode 100644 testcases/lib/tst_run_shell.c
>
> diff --git a/include/tst_test.h b/include/tst_test.h
> index 9871676a5..d0fa84a71 100644
> --- a/include/tst_test.h
> +++ b/include/tst_test.h
> @@ -274,7 +274,7 @@ struct tst_fs {
>   	const char *const *mkfs_opts;
>   	const char *mkfs_size_opt;
>   
> -	const unsigned int mnt_flags;
> +	unsigned int mnt_flags;
>   	const void *mnt_data;
>   };
>   
> diff --git a/testcases/lib/.gitignore b/testcases/lib/.gitignore
> index d0dacf62a..385f3c3ca 100644
> --- a/testcases/lib/.gitignore
> +++ b/testcases/lib/.gitignore
> @@ -24,3 +24,4 @@
>   /tst_supported_fs
>   /tst_timeout_kill
>   /tst_res_
> +/tst_run_shell
> diff --git a/testcases/lib/Makefile b/testcases/lib/Makefile
> index 928d76d62..b3a9181c1 100644
> --- a/testcases/lib/Makefile
> +++ b/testcases/lib/Makefile
> @@ -4,6 +4,9 @@
>   
>   top_srcdir		?= ../..
>   
> +LTPLIBS = ujson
> +tst_run_shell: LTPLDLIBS = -lujson
> +
>   include $(top_srcdir)/include/mk/testcases.mk
>   
>   INSTALL_TARGETS		:= *.sh
> @@ -13,6 +16,7 @@ MAKE_TARGETS		:= tst_sleep tst_random tst_checkpoint tst_rod tst_kvcmp\
>   			   tst_getconf tst_supported_fs tst_check_drivers tst_get_unused_port\
>   			   tst_get_median tst_hexdump tst_get_free_pids tst_timeout_kill\
>   			   tst_check_kconfigs tst_cgctl tst_fsfreeze tst_ns_create tst_ns_exec\
> -			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_
> +			   tst_ns_ifmove tst_lockdown_enabled tst_secureboot_enabled tst_res_\
> +			   tst_run_shell
>   
>   include $(top_srcdir)/include/mk/generic_trunk_target.mk
> diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh
> index 60e7d1bcf..e30065f1d 100755
> --- a/testcases/lib/run_tests.sh
> +++ b/testcases/lib/run_tests.sh
> @@ -9,3 +9,24 @@ for i in `seq -w 01 06`; do
>   	echo
>   	./tests/shell_test$i
>   done
> +
> +for i in shell_loader.sh shell_loader_all_filesystems.sh shell_loader_no_metadata.sh \
> +	 shell_loader_wrong_metadata.sh shell_loader_invalid_metadata.sh\
> +	 shell_loader_supported_archs.sh shell_loader_filesystems.sh\
> +	 shell_loader_tcnt.sh shell_loader_kconfigs.sh shell_loader_tags.sh \
> +	 shell_loader_invalid_block.sh; do
> +	echo
> +	echo "*** Running $i ***"
> +	echo
> +	$i
> +done
> +
> +echo
> +echo "*** Testing LTP test -h option ***"
> +echo
> +shell_loader.sh -h
> +
> +echo
> +echo "*** Testing LTP test -i option ***"
> +echo
> +shell_loader.sh -i 2
> diff --git a/testcases/lib/tests/shell_loader.sh b/testcases/lib/tests/shell_loader.sh
> new file mode 100755
> index 000000000..df7f0c0af
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader.sh
> @@ -0,0 +1,26 @@
> +#!/bin/sh
> +#
> +# ---
> +# doc
> +#
> +# [Description]
> +#
> +# This is a simple shell test loader example.
> +# ---
> +#
> +# ---
> +# env
> +# {
> +#  "needs_tmpdir": true
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "Shell loader works fine!"
> +case "$PWD" in
> +	/tmp/*)
> +		tst_res TPASS "We are running in temp directory in $PWD";;
> +	*)
> +		tst_res TFAIL "We are not running in temp directory but $PWD";;
> +esac
> diff --git a/testcases/lib/tests/shell_loader_all_filesystems.sh b/testcases/lib/tests/shell_loader_all_filesystems.sh
> new file mode 100755
> index 000000000..d5943c335
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_all_filesystems.sh
> @@ -0,0 +1,27 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "needs_root": true,
> +#  "mount_device": true,
> +#  "all_filesystems": true,
> +#  "mntpoint": "ltp_mntpoint"
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TINFO "In shell"
> +
> +mntpath=$(realpath ltp_mntpoint)
> +mounted=$(grep $mntpath /proc/mounts)
> +
> +if [ -n "$mounted" ]; then
> +	device=$(echo $mounted |cut -d' ' -f 1)
> +	path=$(echo $mounted |cut -d' ' -f 2)
> +
> +	tst_res TPASS "$device mounted at $path"
> +else
> +	tst_res TFAIL "Device not mounted!"
> +fi
> diff --git a/testcases/lib/tests/shell_loader_filesystems.sh b/testcases/lib/tests/shell_loader_filesystems.sh
> new file mode 100755
> index 000000000..5d8aa9808
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_filesystems.sh
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "mount_device": true,
> +#  "mntpoint": "ltp_mntpoint",
> +#  "filesystems": [
> +#   {
> +#    "type": "btrfs"
> +#   },
> +#   {
> +#    "type": "xfs",
> +#    "mkfs_opts": ["-m", "reflink=1"]
> +#   }
> +#  ]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TINFO "In shell"
> +
> +mntpoint=$(realpath ltp_mntpoint)
> +mounted=$(grep $mntpoint /proc/mounts)
> +
> +if [ -n "$mounted" ]; then
> +	fs=$(echo $mounted |cut -d' ' -f 3)
> +
> +	tst_res TPASS "Mounted device formatted with $fs"
> +else
> +	tst_res TFAIL "Device not mounted!"
> +fi
> diff --git a/testcases/lib/tests/shell_loader_invalid_block.sh b/testcases/lib/tests/shell_loader_invalid_block.sh
> new file mode 100755
> index 000000000..f41de04fd
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_invalid_block.sh
> @@ -0,0 +1,26 @@
> +#!/bin/sh
> +#
> +# ---
> +# doc
> +#
> +# [Description]
> +#
> +# This is a simple shell test loader example.
> +# ---
> +#
> +# ---
> +# env
> +# {
> +#  "needs_tmpdir": true
> +# }
> +# ---
> +#
> +# ---
> +# inv
> +#
> +# This is an invalid block that breaks the test.
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "This should pass!"
> diff --git a/testcases/lib/tests/shell_loader_invalid_metadata.sh b/testcases/lib/tests/shell_loader_invalid_metadata.sh
> new file mode 100755
> index 000000000..c10b00f1b
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_invalid_metadata.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# This test has wrong metadata and should not be run
> +#
> +# ---
> +# env
> +# {
> +#  {"needs_tmpdir": 42,
> +# }
> +# ---
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Shell loader should TBROK the test"
> diff --git a/testcases/lib/tests/shell_loader_kconfigs.sh b/testcases/lib/tests/shell_loader_kconfigs.sh
> new file mode 100755
> index 000000000..7e9a1dce7
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_kconfigs.sh
> @@ -0,0 +1,12 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "needs_kconfigs": ["CONFIG_NUMA=y"]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "Shell loader works fine!"
> diff --git a/testcases/lib/tests/shell_loader_no_metadata.sh b/testcases/lib/tests/shell_loader_no_metadata.sh
> new file mode 100755
> index 000000000..60ba8b889
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_no_metadata.sh
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +#
> +# This test has no metadata and should not be executed
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Shell loader should TBROK the test"
> diff --git a/testcases/lib/tests/shell_loader_supported_archs.sh b/testcases/lib/tests/shell_loader_supported_archs.sh
> new file mode 100755
> index 000000000..45213f840
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_supported_archs.sh
> @@ -0,0 +1,12 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "supported_archs": ["x86", "ppc64", "x86_64"]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "We are running on supported architecture"
> diff --git a/testcases/lib/tests/shell_loader_tags.sh b/testcases/lib/tests/shell_loader_tags.sh
> new file mode 100755
> index 000000000..a6278c37d
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_tags.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# ---
> +# env
> +# {
> +#  "tags": [
> +#   ["linux-git", "832478cd342ab"],
> +#   ["CVE", "2099-999"]
> +#  ]
> +# }
> +# ---
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Fails the test so that tags are shown."
> diff --git a/testcases/lib/tests/shell_loader_tcnt.sh b/testcases/lib/tests/shell_loader_tcnt.sh
> new file mode 100755
> index 000000000..81fc08179
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_tcnt.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# The script should be executed tcnt times and the iteration number should be in $1
> +#
> +# ---
> +# env
> +# {
> +#  "tcnt": 2
> +# }
> +# ---
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TPASS "Iteration $1"
> diff --git a/testcases/lib/tests/shell_loader_wrong_metadata.sh b/testcases/lib/tests/shell_loader_wrong_metadata.sh
> new file mode 100755
> index 000000000..752e25eea
> --- /dev/null
> +++ b/testcases/lib/tests/shell_loader_wrong_metadata.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +#
> +# This test has wrong metadata and should not be run
> +#
> +# ---
> +# env
> +# {
> +#  "needs_tmpdir": 42,
> +# }
> +# ---
> +#
> +
> +. tst_loader.sh
> +
> +tst_res TFAIL "Shell loader should TBROK the test"
> diff --git a/testcases/lib/tst_env.sh b/testcases/lib/tst_env.sh
> index 948bc5024..67ba80744 100644
> --- a/testcases/lib/tst_env.sh
> +++ b/testcases/lib/tst_env.sh
> @@ -1,4 +1,8 @@
>   #!/bin/sh
> +#
> +# This is a minimal test environment for a shell scripts executed from C by
> +# tst_run_shell() function. Shell tests must use the tst_loader.sh instead!
> +#
>   
>   tst_script_name=$(basename $0)
>   
> diff --git a/testcases/lib/tst_loader.sh b/testcases/lib/tst_loader.sh
> new file mode 100644
> index 000000000..ed04d0340
> --- /dev/null
> +++ b/testcases/lib/tst_loader.sh
> @@ -0,0 +1,11 @@
> +#!/bin/sh
> +#
> +# This is a loader for shell tests that use the C test library.
> +#
> +
> +if [ -z "$LTP_IPC_PATH" ]; then
> +	tst_run_shell $(basename "$0") "$@"
> +	exit $?
> +else
> +	. tst_env.sh
> +fi
> diff --git a/testcases/lib/tst_run_shell.c b/testcases/lib/tst_run_shell.c
> new file mode 100644
> index 000000000..8ed0f21b6
> --- /dev/null
> +++ b/testcases/lib/tst_run_shell.c
> @@ -0,0 +1,491 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2024 Cyril Hrubis <chrubis@suse.cz>
> + */
> +#include <sys/mount.h>
> +
> +#define TST_NO_DEFAULT_MAIN
> +#include "tst_test.h"
> +#include "tst_safe_stdio.h"
> +#include "ujson.h"
> +
> +static char *shell_filename;
> +
> +static void run_shell(void)
> +{
> +	tst_run_script(shell_filename, NULL);
> +}
> +
> +static void run_shell_tcnt(unsigned int n)
> +{
> +	char buf[128];
> +	char *const params[] = {buf, NULL};
> +
> +	snprintf(buf, sizeof(buf), "%u", n);
> +
> +	tst_run_script(shell_filename, params);
> +}
> +
> +struct tst_test test = {
> +	.runs_script = 1,
> +};
> +
> +static void print_help(void)
> +{
> +	printf("Usage: tst_shell_loader ltp_shell_test.sh ...\n");
> +}
> +
> +static char *metadata;
> +static size_t metadata_size;
> +static size_t metadata_used;
> +
> +static void metadata_append(const char *line)
> +{
> +	size_t linelen = strlen(line);
> +
> +	if (metadata_size - metadata_used < linelen + 1) {
> +		metadata_size += 4096;
> +		metadata = SAFE_REALLOC(metadata, metadata_size);
> +	}
> +
> +	strcpy(metadata + metadata_used, line);
> +	metadata_used += linelen;
> +}
> +
> +enum test_attr_ids {
> +	ALL_FILESYSTEMS,
> +	DEV_MIN_SIZE,
> +	FILESYSTEMS,
> +	FORMAT_DEVICE,
> +	MIN_CPUS,
> +	MIN_MEM_AVAIL,
> +	MIN_KVER,
> +	MIN_SWAP_AVAIL,
> +	MNTPOINT,
> +	MOUNT_DEVICE,
> +	NEEDS_ABI_BITS,
> +	NEEDS_CMDS,
> +	NEEDS_DEVFS,
> +	NEEDS_DEVICE,
> +	NEEDS_DRIVERS,
> +	NEEDS_HUGETLBFS,
> +	NEEDS_KCONFIGS,
> +	NEEDS_ROFS,
> +	NEEDS_ROOT,
> +	NEEDS_TMPDIR,
> +	RESTORE_WALLCLOCK,
> +	SKIP_FILESYSTEMS,
> +	SKIP_IN_COMPAT,
> +	SKIP_IN_LOCKDOWN,
> +	SKIP_IN_SECUREBOOT,
> +	SUPPORTED_ARCHS,
> +	TAGS,
> +	TAINT_CHECK,
> +	TCNT,
> +};
> +
> +static ujson_obj_attr test_attrs[] = {
> +	UJSON_OBJ_ATTR_IDX(ALL_FILESYSTEMS, "all_filesystems", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(DEV_MIN_SIZE, "dev_min_size", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(FILESYSTEMS, "filesystems", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(FORMAT_DEVICE, "format_device", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(MIN_CPUS, "min_cpus", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(MIN_MEM_AVAIL, "min_mem_avail", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(MIN_KVER, "min_kver", UJSON_STR),
> +	UJSON_OBJ_ATTR_IDX(MIN_SWAP_AVAIL, "min_swap_avail", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(MNTPOINT, "mntpoint", UJSON_STR),
> +	UJSON_OBJ_ATTR_IDX(MOUNT_DEVICE, "mount_device", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_ABI_BITS, "needs_abi_bits", UJSON_INT),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_CMDS, "needs_cmds", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_DEVFS, "needs_devfs", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_DEVICE, "needs_device", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_DRIVERS, "needs_drivers", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_HUGETLBFS, "needs_hugetlbfs", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_KCONFIGS, "needs_kconfigs", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_ROFS, "needs_rofs", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_ROOT, "needs_root", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(NEEDS_TMPDIR, "needs_tmpdir", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(RESTORE_WALLCLOCK, "restore_wallclock", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SKIP_FILESYSTEMS, "skip_filesystems", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(SKIP_IN_COMPAT, "skip_in_compat", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SKIP_IN_LOCKDOWN, "skip_in_lockdown", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SKIP_IN_SECUREBOOT, "skip_in_secureboot", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(SUPPORTED_ARCHS, "supported_archs", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(TAGS, "tags", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(TAINT_CHECK, "taint_check", UJSON_BOOL),
> +	UJSON_OBJ_ATTR_IDX(TCNT, "tcnt", UJSON_INT)
> +};
> +
> +static ujson_obj test_obj = {
> +	.attrs = test_attrs,
> +	.attr_cnt = UJSON_ARRAY_SIZE(test_attrs),
> +};
> +
> +static const char *const *parse_strarr(ujson_reader *reader, ujson_val *val)
> +{
> +	unsigned int cnt = 0, i = 0;
> +	char **ret;
> +
> +	ujson_reader_state state = ujson_reader_state_save(reader);
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_STR) {
> +			ujson_err(reader, "Expected string!");
> +			return NULL;
> +		}
> +
> +		cnt++;
> +	}
> +
> +	ujson_reader_state_load(reader, state);
> +
> +	ret = SAFE_MALLOC(sizeof(char*) * (cnt + 1));
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		ret[i++] = strdup(val->val_str);
> +	}
> +
> +	ret[i] = NULL;
> +
> +	return (const char *const *)ret;
> +}
> +
> +enum fs_ids {
> +	MKFS_OPTS,
> +	MKFS_SIZE_OPT,
> +	MNT_FLAGS,
> +	TYPE,
> +};
> +
> +static ujson_obj_attr fs_attrs[] = {
> +	UJSON_OBJ_ATTR_IDX(MKFS_OPTS, "mkfs_opts", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(MKFS_SIZE_OPT, "mkfs_size_opt", UJSON_STR),
> +	UJSON_OBJ_ATTR_IDX(MNT_FLAGS, "mnt_flags", UJSON_ARR),
> +	UJSON_OBJ_ATTR_IDX(TYPE, "type", UJSON_STR),
> +};
> +
> +static ujson_obj fs_obj = {
> +	.attrs = fs_attrs,
> +	.attr_cnt = UJSON_ARRAY_SIZE(fs_attrs),
> +};
> +
> +static int parse_mnt_flags(ujson_reader *reader, ujson_val *val)
> +{
> +	int ret = 0;
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_STR) {
> +			ujson_err(reader, "Expected string!");
> +			return ret;
> +		}
> +
> +		if (!strcmp(val->val_str, "RDONLY"))
> +			ret |= MS_RDONLY;
> +		else if (!strcmp(val->val_str, "NOATIME"))
> +			ret |= MS_NOATIME;
> +		else if (!strcmp(val->val_str, "NOEXEC"))
> +			ret |= MS_NOEXEC;
> +		else if (!strcmp(val->val_str, "NOSUID"))
> +			ret |= MS_NOSUID;
> +		else
> +			ujson_err(reader, "Invalid mount flag");
> +	}
> +
> +	return ret;
> +}
> +
> +static struct tst_fs *parse_filesystems(ujson_reader *reader, ujson_val *val)
> +{
> +	unsigned int i = 0, cnt = 0;
> +	struct tst_fs *ret;
> +
> +	ujson_reader_state state = ujson_reader_state_save(reader);
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_OBJ) {
> +			ujson_err(reader, "Expected object!");
> +			return NULL;
> +		}
> +		ujson_obj_skip(reader);
> +		cnt++;
> +	}
> +
> +	ujson_reader_state_load(reader, state);
> +
> +	ret = SAFE_MALLOC(sizeof(struct tst_fs) * (cnt + 1));
> +	memset(ret, 0, sizeof(*ret) * (cnt+1));
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		UJSON_OBJ_FOREACH_FILTER(reader, val, &fs_obj, ujson_empty_obj) {
> +			switch ((enum fs_ids)val->idx) {
> +			case MKFS_OPTS:
> +				ret[i].mkfs_opts = parse_strarr(reader, val);
> +			break;
> +			case MKFS_SIZE_OPT:
> +				ret[i].mkfs_size_opt = strdup(val->val_str);
> +			break;
> +			case MNT_FLAGS:
> +				ret[i].mnt_flags = parse_mnt_flags(reader, val);
> +			break;
> +			case TYPE:
> +				ret[i].type = strdup(val->val_str);
> +			break;
> +			}
> +
> +		}
> +
> +		i++;
> +	}
> +
> +	return ret;
> +}
> +
> +static struct tst_tag *parse_tags(ujson_reader *reader, ujson_val *val)
> +{
> +	unsigned int i = 0, cnt = 0;
> +	struct tst_tag *ret;
> +
> +	ujson_reader_state state = ujson_reader_state_save(reader);
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		if (val->type != UJSON_ARR) {
> +			ujson_err(reader, "Expected array!");
> +			return NULL;
> +		}
> +		ujson_arr_skip(reader);
> +		cnt++;
> +	}
> +
> +	ujson_reader_state_load(reader, state);
> +
> +	ret = SAFE_MALLOC(sizeof(struct tst_tag) * (cnt + 1));
> +	memset(&ret[cnt], 0, sizeof(ret[cnt]));
> +
> +	UJSON_ARR_FOREACH(reader, val) {
> +		char *name = NULL;
> +		char *value = NULL;
> +
> +		UJSON_ARR_FOREACH(reader, val) {
> +			if (val->type != UJSON_STR) {
> +				ujson_err(reader, "Expected string!");
> +				return NULL;
> +			}
> +
> +			if (!name) {
> +				name = strdup(val->val_str);
> +			} else if (!value) {
> +				value = strdup(val->val_str);
> +			} else {
> +				ujson_err(reader, "Expected only two members!");
> +				return NULL;
> +			}
> +		}
> +
> +		ret[i].name = name;
> +		ret[i].value = value;
> +		i++;
> +	}
> +
> +	return ret;
> +}
> +
> +static void parse_metadata(void)
> +{
> +	ujson_reader reader = UJSON_READER_INIT(metadata, metadata_used, UJSON_READER_STRICT);
> +	char str_buf[128];
> +	ujson_val val = UJSON_VAL_INIT(str_buf, sizeof(str_buf));
> +
> +	UJSON_OBJ_FOREACH_FILTER(&reader, &val, &test_obj, ujson_empty_obj) {
> +		switch ((enum test_attr_ids)val.idx) {
> +		case ALL_FILESYSTEMS:
> +			test.all_filesystems = val.val_bool;
> +		break;
> +		case DEV_MIN_SIZE:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Device size must be > 0");
> +			else
> +				test.dev_min_size = val.val_int;
> +		break;
> +		case FILESYSTEMS:
> +			test.filesystems = parse_filesystems(&reader, &val);
> +		break;
> +		case FORMAT_DEVICE:
> +			test.format_device = val.val_bool;
> +		break;
> +		case MIN_CPUS:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Minimal number of cpus must be > 0");
> +			else
> +				test.min_cpus = val.val_int;
> +		break;
> +		case MIN_MEM_AVAIL:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Minimal available memory size must be > 0");
> +			else
> +				test.min_mem_avail = val.val_int;
> +		break;
> +		case MIN_KVER:
> +			test.min_kver = strdup(val.val_str);
> +		break;
> +		case MIN_SWAP_AVAIL:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Minimal available swap size must be > 0");
> +			else
> +				test.min_swap_avail = val.val_int;
> +		break;
> +		case MNTPOINT:
> +			test.mntpoint = strdup(val.val_str);
> +		break;
> +		case MOUNT_DEVICE:
> +			test.mount_device = val.val_bool;
> +		break;
> +		case NEEDS_ABI_BITS:
> +			if (val.val_int == 32 || val.val_int == 64)
> +				test.needs_abi_bits = val.val_int;
> +			else
> +				ujson_err(&reader, "ABI bits must be 32 or 64");
> +		break;
> +		case NEEDS_CMDS:
> +			test.needs_cmds = parse_strarr(&reader, &val);
> +		break;
> +		case NEEDS_DEVFS:
> +			test.needs_devfs = val.val_bool;
> +		break;
> +		case NEEDS_DEVICE:
> +			test.needs_device = val.val_bool;
> +		break;
> +		case NEEDS_DRIVERS:
> +			test.needs_drivers = parse_strarr(&reader, &val);
> +		break;
> +		case NEEDS_HUGETLBFS:
> +			test.needs_hugetlbfs = val.val_bool;
> +		break;
> +		case NEEDS_KCONFIGS:
> +			test.needs_kconfigs = parse_strarr(&reader, &val);
> +		break;
> +		case NEEDS_ROFS:
> +			test.needs_rofs = val.val_bool;
> +		break;
> +		case NEEDS_ROOT:
> +			test.needs_root = val.val_bool;
> +		break;
> +		case NEEDS_TMPDIR:
> +			test.needs_tmpdir = val.val_bool;
> +		break;
> +		case RESTORE_WALLCLOCK:
> +			test.restore_wallclock = val.val_bool;
> +		break;
> +		case SKIP_FILESYSTEMS:
> +			test.skip_filesystems = parse_strarr(&reader, &val);
> +		break;
> +		case SKIP_IN_COMPAT:
> +			test.skip_in_compat = val.val_bool;
> +		break;
> +		case SKIP_IN_LOCKDOWN:
> +			test.skip_in_lockdown = val.val_bool;
> +		break;
> +		case SKIP_IN_SECUREBOOT:
> +			test.skip_in_secureboot = val.val_bool;
> +		break;
> +		case SUPPORTED_ARCHS:
> +			test.supported_archs = parse_strarr(&reader, &val);
> +		break;
> +		case TAGS:
> +			test.tags = parse_tags(&reader, &val);
> +		break;
> +		case TAINT_CHECK:
> +			test.taint_check = val.val_bool;
> +		break;
> +		case TCNT:
> +			if (val.val_int <= 0)
> +				ujson_err(&reader, "Number of tests must be > 0");
> +			else
> +				test.tcnt = val.val_int;
> +		break;
> +		}
> +	}
> +
> +	ujson_reader_finish(&reader);
> +
> +	if (ujson_reader_err(&reader))
> +		tst_brk(TBROK, "Invalid metadata");
> +}
> +
> +enum parser_state {
> +	PAR_NONE,
> +	PAR_ESC,
> +	PAR_DOC,
> +	PAR_ENV,
> +};
> +
> +static void extract_metadata(void)
> +{
> +	FILE *f;
> +	char line[4096];
> +	char path[4096];
> +	enum parser_state state = PAR_NONE;
> +
> +	if (tst_get_path(shell_filename, path, sizeof(path)) == -1)
> +		tst_brk(TBROK, "Failed to find %s in $PATH", shell_filename);
> +
> +	f = SAFE_FOPEN(path, "r");
> +
> +	while (fgets(line, sizeof(line), f)) {
> +		switch (state) {
> +		case PAR_NONE:
> +			if (!strcmp(line, "# ---\n"))
> +				state = PAR_ESC;
> +		break;
> +		case PAR_ESC:
> +			if (!strcmp(line, "# env\n"))
> +				state = PAR_ENV;
> +			else if (!strcmp(line, "# doc\n"))
> +				state = PAR_DOC;
> +			else
> +				tst_brk(TBROK, "Unknown comment block %s", line);
> +		break;
> +		case PAR_ENV:
> +			if (!strcmp(line, "# ---\n"))
> +				state = PAR_NONE;
> +			else
> +				metadata_append(line + 2);
> +		break;
> +		case PAR_DOC:
> +			if (!strcmp(line, "# ---\n"))
> +				state = PAR_NONE;
> +		break;
> +		}
> +	}
> +
> +	fclose(f);
> +}
> +
> +static void prepare_test_struct(void)
> +{
> +	extract_metadata();
> +
> +	if (metadata)
> +		parse_metadata();
> +	else
> +		tst_brk(TBROK, "No metadata found!");
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	if (argc < 2)
> +		goto help;
> +
> +	shell_filename = argv[1];
> +
> +	prepare_test_struct();
> +
> +	if (test.tcnt)
> +		test.test = run_shell_tcnt;
> +	else
> +		test.test_all = run_shell;
> +
> +	tst_run_tcases(argc - 1, argv + 1, &test);
> +help:
> +	print_help();
> +	return 1;
> +}

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 4/4] testcaes/lib: Add shell loader

[Cyril Hrubis]

Hi!
> Also is there a reason why we are adding tests to testcases/lib/tests ?

That is because historically all the libraries for shell tests are in
testcases/lib/. We should probably do a big cleanup and reorganization
of the codebase, but that is something to be done later on.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 4/4] testcaes/lib: Add shell loader

[Li Wang]

On Tue, Aug 27, 2024 at 8:05 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> This commit implements a shell loader so that we don't have to write a C
> loader for each LTP shell test. The idea is simple, the loader parses
> the shell test and prepares the tst_test structure accordingly, then
> runs the actual shell test.
>
> The format for the metadata in the shell test was choosen to be JSON
> because:
>
> - I didn't want to invent an adhoc format and JSON is perfect for
>   serializing data structures
> - The metadata parser for shell test will be trivial, it will just pick
>   the JSON from the comment, no parsing will be required
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> Reviewed-by: Richard Palethorpe <io@richiejp.com>
>

Reviewed-by: Li Wang <liwang@redhat.com>


-- 
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v3 0/4] Shell test library v3

[Cyril Hrubis]

Hi!
As discussed during the LTP montly call I've added all the changes
discussed here to the V3 and pushed.

Thanks a lot for the reviews!

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp