[PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

[PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

[Alexis Lothoré (eBPF Foundation)]

Hello,
this series is part of the larger effort aiming to convert all
standalone tests to the CI runners so that they are properly executed on
patches submission.

Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
and so they do not integrate well in test_progs. This series proposes to
introduce a new runner to support those tests. This new runner is
heavily inspired from test_progs, with slightly less features.

- First commit move the assert macros exposed by test_progs into a
  shared header so that they can be used by the new runner
- Second commit introduces the new runner, as well as a first test
- Third commit adds a second test
- Fourth commit drops the legacy scripts corresponding to those new
  tests

The series only adds the runner, but it is not enough to have it running
in CI. The github actions need to be updated both in [1] and [2], and I
am not sure how it si supposed to be contributed (the part needed in [1]
must likely be integrated in the commit that is systematically brought
on top of the bpf-next_base branch). Still, an example of test_bpftool
run can be found in [3], in which I have forked and updated the needed
Github actions ([4])

[1] https://github.com/kernel-patches/bpf.git
[2] https://github.com/libbpf/ci
[3] https://github.com/kernel-patches/bpf/pull/10711
[4] https://github.com/Tropicao/libbpf-ci

Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
---
Alexis Lothoré (eBPF Foundation) (4):
      bpf/selftests: move assert macros into a dedicated header
      bpf/selftests: introduce bptool test runner and a first test
      selftests/bpf: add bpftool map manipulations tests
      selftests/bpf: remove converted bpftool test scripts

 tools/testing/selftests/bpf/.gitignore             |   1 +
 tools/testing/selftests/bpf/Makefile               |  16 +-
 tools/testing/selftests/bpf/assert_helpers.h       | 231 ++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +
 .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
 .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++
 .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++
 tools/testing/selftests/bpf/test_bpftool.c         | 126 +++++++
 tools/testing/selftests/bpf/test_bpftool.h         |  36 ++
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
 tools/testing/selftests/bpf/test_progs.h           | 226 +-----------
 13 files changed, 1041 insertions(+), 711 deletions(-)
---
base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
change-id: 20251212-bpftool-tests-bb165c4cceb8

Best regards,
-- 
Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>

[PATCH bpf-next 1/4] bpf/selftests: move assert macros into a dedicated header

[Alexis Lothoré (eBPF Foundation)]

The test_progs runner defines a large set of convenient assert macros to
perform all the tests. Writing a new runner involves rewriting some
macros if we want some basic testing features like standardized failure
log.

Export those assert macros from test_progs into a dedicated header so
that we can use those in any test_runner. The sole requirement to be
able to use those macros is to define a test__fail function in the
runner that will be called whenever an assert fails.

Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
---
 tools/testing/selftests/bpf/assert_helpers.h | 231 +++++++++++++++++++++++++++
 tools/testing/selftests/bpf/test_progs.h     | 226 +-------------------------
 2 files changed, 232 insertions(+), 225 deletions(-)

diff --git a/tools/testing/selftests/bpf/assert_helpers.h b/tools/testing/selftests/bpf/assert_helpers.h
new file mode 100644
index 000000000000..93ab5bf39431
--- /dev/null
+++ b/tools/testing/selftests/bpf/assert_helpers.h
@@ -0,0 +1,231 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+#include <stdio.h>
+#include <errno.h>
+#include <stdbool.h>
+
+#define _CHECK(condition, tag, duration, format...) ({			\
+	int __ret = !!(condition);					\
+	int __save_errno = errno;					\
+	if (__ret) {							\
+		test__fail();						\
+		fprintf(stdout, "%s:FAIL:%s ", __func__, tag);		\
+		fprintf(stdout, ##format);				\
+	} else {							\
+		fprintf(stdout, "%s:PASS:%s %d nsec\n",			\
+		       __func__, tag, duration);			\
+	}								\
+	errno = __save_errno;						\
+	__ret;								\
+})
+
+#define CHECK_FAIL(condition) ({					\
+	int __ret = !!(condition);					\
+	int __save_errno = errno;					\
+	if (__ret) {							\
+		test__fail();						\
+		fprintf(stdout, "%s:FAIL:%d\n", __func__, __LINE__);	\
+	}								\
+	errno = __save_errno;						\
+	__ret;								\
+})
+
+#define CHECK(condition, tag, format...) \
+	_CHECK(condition, tag, duration, format)
+#define CHECK_ATTR(condition, tag, format...) \
+	_CHECK(condition, tag, tattr.duration, format)
+
+#define ASSERT_FAIL(fmt, args...) ({					\
+	static int duration;						\
+	CHECK(false, "", fmt"\n", ##args);				\
+	false;								\
+})
+
+#define ASSERT_TRUE(actual, name) ({					\
+	static int duration;						\
+	bool ___ok = (actual);						\
+	CHECK(!___ok, (name), "unexpected %s: got FALSE\n", (name));	\
+	___ok;								\
+})
+
+#define ASSERT_FALSE(actual, name) ({					\
+	static int duration;						\
+	bool ___ok = !(actual);						\
+	CHECK(!___ok, (name), "unexpected %s: got TRUE\n", (name));	\
+	___ok;								\
+})
+
+#define ASSERT_EQ(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act == ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld != expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_NEQ(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act != ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld == expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_LT(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act < ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld >= expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_LE(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act <= ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld > expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_GT(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act > ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld <= expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_GE(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act >= ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld < expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_STREQ(actual, expected, name) ({				\
+	static int duration;						\
+	const char *___act = actual;					\
+	const char *___exp = expected;					\
+	bool ___ok = strcmp(___act, ___exp) == 0;			\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual '%s' != expected '%s'\n",		\
+	      (name), ___act, ___exp);					\
+	___ok;								\
+})
+
+#define ASSERT_STRNEQ(actual, expected, len, name) ({			\
+	static int duration;						\
+	const char *___act = actual;					\
+	const char *___exp = expected;					\
+	int ___len = len;						\
+	bool ___ok = strncmp(___act, ___exp, ___len) == 0;		\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual '%.*s' != expected '%.*s'\n",	\
+	      (name), ___len, ___act, ___len, ___exp);			\
+	___ok;								\
+})
+
+#define ASSERT_HAS_SUBSTR(str, substr, name) ({				\
+	static int duration;						\
+	const char *___str = str;					\
+	const char *___substr = substr;					\
+	bool ___ok = strstr(___str, ___substr) != NULL;			\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: '%s' is not a substring of '%s'\n",	\
+	      (name), ___substr, ___str);				\
+	___ok;								\
+})
+
+#define ASSERT_MEMEQ(actual, expected, len, name) ({			\
+	static int duration;						\
+	const void *__act = actual;					\
+	const void *__exp = expected;					\
+	int __len = len;						\
+	bool ___ok = memcmp(__act, __exp, __len) == 0;			\
+	CHECK(!___ok, (name), "unexpected memory mismatch\n");		\
+	fprintf(stdout, "actual:\n");					\
+	hexdump("\t", __act, __len);					\
+	fprintf(stdout, "expected:\n");					\
+	hexdump("\t", __exp, __len);					\
+	___ok;								\
+})
+
+#define ASSERT_OK(res, name) ({						\
+	static int duration;						\
+	long long ___res = (res);					\
+	bool ___ok = ___res == 0;					\
+	CHECK(!___ok, (name), "unexpected error: %lld (errno %d)\n",	\
+	      ___res, errno);						\
+	___ok;								\
+})
+
+#define ASSERT_ERR(res, name) ({					\
+	static int duration;						\
+	long long ___res = (res);					\
+	bool ___ok = ___res < 0;					\
+	CHECK(!___ok, (name), "unexpected success: %lld\n", ___res);	\
+	___ok;								\
+})
+
+#define ASSERT_NULL(ptr, name) ({					\
+	static int duration;						\
+	const void *___res = (ptr);					\
+	bool ___ok = !___res;						\
+	CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res);	\
+	___ok;								\
+})
+
+#define ASSERT_OK_PTR(ptr, name) ({					\
+	static int duration;						\
+	const void *___res = (ptr);					\
+	int ___err = libbpf_get_error(___res);				\
+	bool ___ok = ___err == 0;					\
+	CHECK(!___ok, (name), "unexpected error: %d\n", ___err);	\
+	___ok;								\
+})
+
+#define ASSERT_ERR_PTR(ptr, name) ({					\
+	static int duration;						\
+	const void *___res = (ptr);					\
+	int ___err = libbpf_get_error(___res);				\
+	bool ___ok = ___err != 0;					\
+	CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res);	\
+	___ok;								\
+})
+
+#define ASSERT_OK_FD(fd, name) ({					\
+	static int duration;						\
+	int ___fd = (fd);						\
+	bool ___ok = ___fd >= 0;					\
+	CHECK(!___ok, (name), "unexpected fd: %d (errno %d)\n",		\
+	      ___fd, errno);						\
+	___ok;								\
+})
+
+#define ASSERT_ERR_FD(fd, name) ({					\
+	static int duration;						\
+	int ___fd = (fd);						\
+	bool ___ok = ___fd < 0;						\
+	CHECK(!___ok, (name), "unexpected fd: %d\n", ___fd);		\
+	___ok;								\
+})
+
diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h
index eebfc18cdcd2..bb876d8f6bcc 100644
--- a/tools/testing/selftests/bpf/test_progs.h
+++ b/tools/testing/selftests/bpf/test_progs.h
@@ -42,6 +42,7 @@ typedef __u16 __sum16;
 #include <bpf/bpf_endian.h>
 #include "trace_helpers.h"
 #include "testing_helpers.h"
+#include "assert_helpers.h"
 
 enum verbosity {
 	VERBOSE_NONE,
@@ -195,231 +196,6 @@ void hexdump(const char *prefix, const void *buf, size_t len);
 		fprintf(stdout, ##format);                                     \
 	})
 
-#define _CHECK(condition, tag, duration, format...) ({			\
-	int __ret = !!(condition);					\
-	int __save_errno = errno;					\
-	if (__ret) {							\
-		test__fail();						\
-		fprintf(stdout, "%s:FAIL:%s ", __func__, tag);		\
-		fprintf(stdout, ##format);				\
-	} else {							\
-		fprintf(stdout, "%s:PASS:%s %d nsec\n",			\
-		       __func__, tag, duration);			\
-	}								\
-	errno = __save_errno;						\
-	__ret;								\
-})
-
-#define CHECK_FAIL(condition) ({					\
-	int __ret = !!(condition);					\
-	int __save_errno = errno;					\
-	if (__ret) {							\
-		test__fail();						\
-		fprintf(stdout, "%s:FAIL:%d\n", __func__, __LINE__);	\
-	}								\
-	errno = __save_errno;						\
-	__ret;								\
-})
-
-#define CHECK(condition, tag, format...) \
-	_CHECK(condition, tag, duration, format)
-#define CHECK_ATTR(condition, tag, format...) \
-	_CHECK(condition, tag, tattr.duration, format)
-
-#define ASSERT_FAIL(fmt, args...) ({					\
-	static int duration = 0;					\
-	CHECK(false, "", fmt"\n", ##args);				\
-	false;								\
-})
-
-#define ASSERT_TRUE(actual, name) ({					\
-	static int duration = 0;					\
-	bool ___ok = (actual);						\
-	CHECK(!___ok, (name), "unexpected %s: got FALSE\n", (name));	\
-	___ok;								\
-})
-
-#define ASSERT_FALSE(actual, name) ({					\
-	static int duration = 0;					\
-	bool ___ok = !(actual);						\
-	CHECK(!___ok, (name), "unexpected %s: got TRUE\n", (name));	\
-	___ok;								\
-})
-
-#define ASSERT_EQ(actual, expected, name) ({				\
-	static int duration = 0;					\
-	typeof(actual) ___act = (actual);				\
-	typeof(expected) ___exp = (expected);				\
-	bool ___ok = ___act == ___exp;					\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual %lld != expected %lld\n",		\
-	      (name), (long long)(___act), (long long)(___exp));	\
-	___ok;								\
-})
-
-#define ASSERT_NEQ(actual, expected, name) ({				\
-	static int duration = 0;					\
-	typeof(actual) ___act = (actual);				\
-	typeof(expected) ___exp = (expected);				\
-	bool ___ok = ___act != ___exp;					\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual %lld == expected %lld\n",		\
-	      (name), (long long)(___act), (long long)(___exp));	\
-	___ok;								\
-})
-
-#define ASSERT_LT(actual, expected, name) ({				\
-	static int duration = 0;					\
-	typeof(actual) ___act = (actual);				\
-	typeof(expected) ___exp = (expected);				\
-	bool ___ok = ___act < ___exp;					\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual %lld >= expected %lld\n",		\
-	      (name), (long long)(___act), (long long)(___exp));	\
-	___ok;								\
-})
-
-#define ASSERT_LE(actual, expected, name) ({				\
-	static int duration = 0;					\
-	typeof(actual) ___act = (actual);				\
-	typeof(expected) ___exp = (expected);				\
-	bool ___ok = ___act <= ___exp;					\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual %lld > expected %lld\n",		\
-	      (name), (long long)(___act), (long long)(___exp));	\
-	___ok;								\
-})
-
-#define ASSERT_GT(actual, expected, name) ({				\
-	static int duration = 0;					\
-	typeof(actual) ___act = (actual);				\
-	typeof(expected) ___exp = (expected);				\
-	bool ___ok = ___act > ___exp;					\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual %lld <= expected %lld\n",		\
-	      (name), (long long)(___act), (long long)(___exp));	\
-	___ok;								\
-})
-
-#define ASSERT_GE(actual, expected, name) ({				\
-	static int duration = 0;					\
-	typeof(actual) ___act = (actual);				\
-	typeof(expected) ___exp = (expected);				\
-	bool ___ok = ___act >= ___exp;					\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual %lld < expected %lld\n",		\
-	      (name), (long long)(___act), (long long)(___exp));	\
-	___ok;								\
-})
-
-#define ASSERT_STREQ(actual, expected, name) ({				\
-	static int duration = 0;					\
-	const char *___act = actual;					\
-	const char *___exp = expected;					\
-	bool ___ok = strcmp(___act, ___exp) == 0;			\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual '%s' != expected '%s'\n",		\
-	      (name), ___act, ___exp);					\
-	___ok;								\
-})
-
-#define ASSERT_STRNEQ(actual, expected, len, name) ({			\
-	static int duration = 0;					\
-	const char *___act = actual;					\
-	const char *___exp = expected;					\
-	int ___len = len;						\
-	bool ___ok = strncmp(___act, ___exp, ___len) == 0;		\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual '%.*s' != expected '%.*s'\n",	\
-	      (name), ___len, ___act, ___len, ___exp);			\
-	___ok;								\
-})
-
-#define ASSERT_HAS_SUBSTR(str, substr, name) ({				\
-	static int duration = 0;					\
-	const char *___str = str;					\
-	const char *___substr = substr;					\
-	bool ___ok = strstr(___str, ___substr) != NULL;			\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: '%s' is not a substring of '%s'\n",	\
-	      (name), ___substr, ___str);				\
-	___ok;								\
-})
-
-#define ASSERT_MEMEQ(actual, expected, len, name) ({			\
-	static int duration = 0;					\
-	const void *__act = actual;					\
-	const void *__exp = expected;					\
-	int __len = len;						\
-	bool ___ok = memcmp(__act, __exp, __len) == 0;			\
-	CHECK(!___ok, (name), "unexpected memory mismatch\n");		\
-	fprintf(stdout, "actual:\n");					\
-	hexdump("\t", __act, __len);					\
-	fprintf(stdout, "expected:\n");					\
-	hexdump("\t", __exp, __len);					\
-	___ok;								\
-})
-
-#define ASSERT_OK(res, name) ({						\
-	static int duration = 0;					\
-	long long ___res = (res);					\
-	bool ___ok = ___res == 0;					\
-	CHECK(!___ok, (name), "unexpected error: %lld (errno %d)\n",	\
-	      ___res, errno);						\
-	___ok;								\
-})
-
-#define ASSERT_ERR(res, name) ({					\
-	static int duration = 0;					\
-	long long ___res = (res);					\
-	bool ___ok = ___res < 0;					\
-	CHECK(!___ok, (name), "unexpected success: %lld\n", ___res);	\
-	___ok;								\
-})
-
-#define ASSERT_NULL(ptr, name) ({					\
-	static int duration = 0;					\
-	const void *___res = (ptr);					\
-	bool ___ok = !___res;						\
-	CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res);	\
-	___ok;								\
-})
-
-#define ASSERT_OK_PTR(ptr, name) ({					\
-	static int duration = 0;					\
-	const void *___res = (ptr);					\
-	int ___err = libbpf_get_error(___res);				\
-	bool ___ok = ___err == 0;					\
-	CHECK(!___ok, (name), "unexpected error: %d\n", ___err);	\
-	___ok;								\
-})
-
-#define ASSERT_ERR_PTR(ptr, name) ({					\
-	static int duration = 0;					\
-	const void *___res = (ptr);					\
-	int ___err = libbpf_get_error(___res);				\
-	bool ___ok = ___err != 0;					\
-	CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res);	\
-	___ok;								\
-})
-
-#define ASSERT_OK_FD(fd, name) ({					\
-	static int duration = 0;					\
-	int ___fd = (fd);						\
-	bool ___ok = ___fd >= 0;					\
-	CHECK(!___ok, (name), "unexpected fd: %d (errno %d)\n",		\
-	      ___fd, errno);						\
-	___ok;								\
-})
-
-#define ASSERT_ERR_FD(fd, name) ({					\
-	static int duration = 0;					\
-	int ___fd = (fd);						\
-	bool ___ok = ___fd < 0;						\
-	CHECK(!___ok, (name), "unexpected fd: %d\n", ___fd);		\
-	___ok;								\
-})
-
 #define SYS(goto_label, fmt, ...)					\
 	({								\
 		char cmd[1024];						\

-- 
2.52.0

Re: [PATCH bpf-next 1/4] bpf/selftests: move assert macros into a dedicated header

[Quentin Monnet]

2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)"
<alexis.lothore@bootlin.com>
> The test_progs runner defines a large set of convenient assert macros to
> perform all the tests. Writing a new runner involves rewriting some
> macros if we want some basic testing features like standardized failure
> log.
> 
> Export those assert macros from test_progs into a dedicated header so
> that we can use those in any test_runner. The sole requirement to be
> able to use those macros is to define a test__fail function in the
> runner that will be called whenever an assert fails.
> 
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>


Acked-by: Quentin Monnet <qmo@kernel.org>

[PATCH bpf-next 2/4] bpf/selftests: introduce bptool test runner and a first test

[Alexis Lothoré (eBPF Foundation)]

The tools/testing/selftests/bpf directory contains multiple scripts
(shell, python, c code, etc) that aim to test some specific features
from bpftool. Those isolated tests are currently not executed by any CI
automation. Create a dedicated runner for any bpftool-related test that
can then be added to the list of executed runners in bpf CI automation.
This new runner (and the corresponding Makefile tooling) is highly
inspired from test_progs, but kept a bit simpler. This version supports
the following features:

- autodetection of bpftool test stored in the in bpftool_tests
  directory
- bpftool binary under test is passed as runner argument
- a few helpers to allow to easily run abpftool commands while possibly
  collecting the output
- usage of assert macros shared with test_progs
- basic sub-tests management
- logs collection, logs being dumped only for failed tests
- exit code reflecting whether all tests have passed or not

As this runner needs at least one test to be implemented to properly
compile, also bring bpftool_metadata, which is the conversion of
test_bpftool_metadata.sh: this test validates that the output of some
basic prog/map listings done with bpftool properly returns the metadata
collected from the .rodata section of eBPF programs.

This new runner gives an output similar to the one generated by
test_progs:

  #2/1	metadata/metadata_unused: OK
  #2/2	metadata/metadata_used: OK
  #2	metadata: OK
  Summary: 1 PASSED, 0 FAILED

Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
---
 tools/testing/selftests/bpf/.gitignore             |   1 +
 tools/testing/selftests/bpf/Makefile               |  14 ++-
 tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +++
 .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
 .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++++++++++++++++
 tools/testing/selftests/bpf/test_bpftool.c         | 126 ++++++++++++++++++++
 tools/testing/selftests/bpf/test_bpftool.h         |  36 ++++++
 8 files changed, 439 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index b8bf51b7a0b0..9498cc11de97 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -2,6 +2,7 @@
 bpftool
 bpf-helpers*
 bpf-syscall*
+test_bpftool
 test_verifier
 test_maps
 test_lru_map
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index fd42b7193d4e..a1fe94efa53c 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -76,7 +76,8 @@ endif
 TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_progs \
 	test_sockmap \
 	test_tcpnotify_user \
-	test_progs-no_alu32
+	test_progs-no_alu32 \
+	test_bpftool
 TEST_INST_SUBDIRS := no_alu32
 
 # Also test bpf-gcc, if present
@@ -791,6 +792,17 @@ TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built)
 TRUNNER_BPF_CFLAGS :=
 $(eval $(call DEFINE_TEST_RUNNER,test_maps))
 
+# Define bpftool test runner.
+TRUNNER_TESTS_DIR := bpftool_tests
+TRUNNER_BPF_PROGS_DIR := progs
+TRUNNER_EXTRA_SOURCES := test_bpftool.c \
+			 bpftool_helpers.c
+TRUNNER_LIB_SOURCES :=
+TRUNNER_EXTRA_FILES :=
+TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
+TRUNNER_BPF_CFLAGS :=
+$(eval $(call DEFINE_TEST_RUNNER,test_bpftool))
+
 # Define test_verifier test runner.
 # It is much simpler than test_maps/test_progs and sufficiently different from
 # them (e.g., test.h is using completely pattern), that it's worth just
diff --git a/tools/testing/selftests/bpf/bpftool_helpers.c b/tools/testing/selftests/bpf/bpftool_helpers.c
new file mode 100644
index 000000000000..ff8084d9a121
--- /dev/null
+++ b/tools/testing/selftests/bpf/bpftool_helpers.c
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include "bpftool_helpers.h"
+#include "test_bpftool.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdbool.h>
+
+#define BPFTOOL_PATH		"./tools/sbin/bpftool"
+#define BPFTOOL_CMD_MAX_LEN	256
+
+static int run_command(char *command, bool get_output, char *output_buf, size_t output_max_len)
+{
+	FILE *f;
+	int ret;
+
+	f = popen(command, "r");
+	if (!f)
+		return 1;
+
+	if (get_output)
+		fread(output_buf, 1, output_max_len, f);
+	ret = pclose(f);
+
+	return ret;
+}
+
+int run_bpftool_command(char *args)
+{
+	char cmd[BPFTOOL_CMD_MAX_LEN];
+	int ret;
+
+	ret = snprintf(cmd, BPFTOOL_CMD_MAX_LEN, "%s %s > /dev/null 2>&1",
+		       env.bpftool_path, args);
+	if (ret !=
+	    strlen(env.bpftool_path) + 1 + strlen(args) + strlen(" > /dev/null 2>&1")) {
+		fprintf(stderr, "Failed to generate bpftool command\n");
+		return 1;
+	}
+
+	return run_command(cmd, false, NULL, 0);
+}
+
+int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len)
+{
+	int ret;
+	char cmd[BPFTOOL_CMD_MAX_LEN];
+
+	ret = snprintf(cmd, BPFTOOL_CMD_MAX_LEN, "%s %s", env.bpftool_path,
+		       args);
+	if (ret != strlen(args) + strlen(env.bpftool_path) + 1) {
+		fprintf(stderr, "Failed to generate bpftool command");
+		return 1;
+	}
+
+	return run_command(cmd, true, output_buf, output_max_len);
+}
+
+void hijack_stdio(void)
+{
+	fflush(stdout);
+	fflush(stderr);
+	if (env.current_subtest) {
+		env.current_test->saved_stdout = stdout;
+		env.current_test->saved_stderr = stderr;
+		stdout = open_memstream(&env.current_subtest->log,
+					&env.current_subtest->log_size);
+
+	} else {
+		env.saved_stdout = stdout;
+		env.saved_stderr = stderr;
+		stdout = open_memstream(&env.current_test->log,
+					&env.current_test->log_size);
+	}
+	stderr = stdout;
+}
+
+void restore_stdio(void)
+{
+	fclose(stdout);
+	if (env.current_subtest) {
+		stdout = env.current_test->saved_stdout;
+		stderr = env.current_test->saved_stderr;
+
+	} else {
+		stdout = env.saved_stdout;
+		stderr = env.saved_stderr;
+	}
+
+}
+
+void test__start_subtest(const char *subtest_name)
+{
+	test__end_subtest();
+	env.current_test->subtests_count++;
+	env.subtest_states = realloc(env.subtest_states,
+				     env.current_test->subtests_count *
+					     sizeof(struct subtest_state));
+	env.current_subtest =
+		&env.subtest_states[env.current_test->subtests_count - 1];
+	memset(env.current_subtest, 0, sizeof(struct subtest_state));
+	env.current_subtest->name = strdup(subtest_name);
+
+	hijack_stdio();
+}
+
+void test__end_subtest(void)
+{
+	if (env.current_subtest) {
+		restore_stdio();
+		env.current_subtest = NULL;
+	}
+}
+
diff --git a/tools/testing/selftests/bpf/bpftool_helpers.h b/tools/testing/selftests/bpf/bpftool_helpers.h
new file mode 100644
index 000000000000..1eacec7936ba
--- /dev/null
+++ b/tools/testing/selftests/bpf/bpftool_helpers.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdbool.h>
+
+#define MAX_BPFTOOL_CMD_LEN	(256)
+
+#ifndef ARRAY_SIZE
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+#endif
+
+int run_bpftool_command(char *args);
+int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len);
+void test__start_subtest(const char *subtests_name);
+void test__end_subtest(void);
+void hijack_stdio(void);
+void restore_stdio(void);
diff --git a/tools/testing/selftests/bpf/bpftool_tests/.gitignore b/tools/testing/selftests/bpf/bpftool_tests/.gitignore
new file mode 100644
index 000000000000..89c4a3d37544
--- /dev/null
+++ b/tools/testing/selftests/bpf/bpftool_tests/.gitignore
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+tests.h
diff --git a/tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c b/tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c
new file mode 100644
index 000000000000..e7146b26f298
--- /dev/null
+++ b/tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c
@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <bpftool_helpers.h>
+#include <test_bpftool.h>
+#include <assert_helpers.h>
+#include <linux/bpf.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+#include <stdbool.h>
+
+#define BPFFS_DIR	"/sys/fs/bpf/test_metadata"
+#define BPFFS_USED	BPFFS_DIR "/used"
+#define BPFFS_UNUSED	BPFFS_DIR "/unused"
+
+#define BPF_FILE_USED	"metadata_used.bpf.o"
+#define BPF_FILE_UNUSED "metadata_unused.bpf.o"
+
+#define MAX_BPFTOOL_OUTPUT_LEN	(100*1000)
+
+#define MAX_TOKENS_TO_CHECK	3
+static char output[MAX_BPFTOOL_OUTPUT_LEN];
+
+struct test_desc {
+	char *name;
+	char *bpf_prog;
+	char *bpffs_path;
+	char *expected_output[MAX_TOKENS_TO_CHECK];
+	char *expected_output_json[MAX_TOKENS_TO_CHECK];
+};
+
+static int setup(struct test_desc *test)
+{
+	return mkdir(BPFFS_DIR, 0700);
+}
+
+static void cleanup(struct test_desc *test)
+{
+	unlink(test->bpffs_path);
+	rmdir(BPFFS_DIR);
+}
+
+static int check_metadata(char *buf, char * const *tokens, int count)
+{
+	int i;
+
+	for (i = 0; i < count && tokens[i]; i++)
+		if (!strstr(buf, tokens[i]))
+			return 1;
+
+	return 0;
+}
+
+static void run_test(struct test_desc *test)
+{
+	int ret;
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+
+	snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog load %s %s",
+			test->bpf_prog, test->bpffs_path);
+	ret = run_bpftool_command(cmd);
+	if (!ASSERT_OK(ret, "load program"))
+		return;
+
+	/* Check output with default format */
+	ret = get_bpftool_command_output("prog show name prog", output,
+			MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info")) {
+		ret = check_metadata(output, test->expected_output,
+				ARRAY_SIZE(test->expected_output));
+		ASSERT_OK(ret, "find metadata");
+	}
+
+	/* Check output with json format */
+	ret = get_bpftool_command_output("prog -j show name prog", output,
+					 MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info in json")) {
+		ret = check_metadata(output, test->expected_output_json,
+				ARRAY_SIZE(test->expected_output_json));
+		ASSERT_OK(ret, "find metadata in json");
+	}
+
+}
+
+struct test_desc tests[] = {
+	{
+		.name = "metadata_unused",
+		.bpf_prog = BPF_FILE_UNUSED,
+		.bpffs_path = BPFFS_UNUSED,
+		.expected_output = {
+			"a = \"foo\"",
+			"b = 1"
+		},
+		.expected_output_json = {
+			"\"metadata\":{\"a\":\"foo\",\"b\":1}"
+		}
+	},
+	{
+		.name = "metadata_used",
+		.bpf_prog = BPF_FILE_USED,
+		.bpffs_path = BPFFS_USED,
+		.expected_output = {
+			"a = \"bar\"",
+			"b = 2"
+		},
+		.expected_output_json = {
+			"\"metadata\":{\"a\":\"bar\",\"b\":2}"
+		}
+	}
+};
+
+static const int tests_count = ARRAY_SIZE(tests);
+
+void test_metadata(void)
+{
+	int i, ret;
+
+	for (i = 0; i < tests_count; i++) {
+		test__start_subtest(tests[i].name);
+		ret = setup(&tests[i]);
+		if (!ASSERT_OK(ret, "setup bpffs pin dir"))
+			continue;
+		run_test(&tests[i]);
+		cleanup(&tests[i]);
+	}
+
+}
+
diff --git a/tools/testing/selftests/bpf/test_bpftool.c b/tools/testing/selftests/bpf/test_bpftool.c
new file mode 100644
index 000000000000..b5fb17d5ea2d
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_bpftool.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <test_bpftool.h>
+#include <bpftool_helpers.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <string.h>
+#include <unistd.h>
+
+struct bpftool_runner_env env = {0};
+
+#define DEFINE_TEST(name) extern void test_##name(void);
+#include <bpftool_tests/tests.h>
+#undef DEFINE_TEST
+
+struct prog_test_def {
+	char *test_name;
+	void (*run_test)(void);
+};
+
+static struct prog_test_def prog_test_defs[] = {
+#define DEFINE_TEST(name) {			\
+	.test_name = #name,			\
+	.run_test = &test_##name,		\
+},
+#include <bpftool_tests/tests.h>
+#undef DEFINE_TEST
+};
+
+
+static const int tests_count = ARRAY_SIZE(prog_test_defs);
+
+/* Needed method for the assert macros exposed by assert_helpers.h */
+void test__fail(void)
+{
+	if (env.current_subtest)
+		env.current_subtest->failed = true;
+	if (!env.current_test->failed)
+		env.failure_cnt++;
+	env.current_test->failed = true;
+}
+
+static void test_setup(struct test_state *test, char *name)
+{
+	env.current_test = test;
+	env.current_test->name = strdup(name);
+}
+
+static void dump_results(struct test_state *test, int test_index)
+{
+	int j;
+
+	if (test->failed)
+		fprintf(stdout, "%s\n", test->log);
+	free(test->log);
+	for (j = 0; j < test->subtests_count; j++) {
+		if (env.subtest_states[j].failed)
+			fprintf(stdout, "%s\n", env.subtest_states[j].log);
+		free(env.subtest_states[j].log);
+		fprintf(stdout, "#%d/%d\t%s/%s: %s\n", test_index+1, j+1,
+				env.current_test->name,
+				env.subtest_states[j].name,
+				env.subtest_states[j].failed ? "KO" : "OK");
+		free(env.subtest_states[j].name);
+	}
+	if (env.current_test->subtests_count) {
+		free(env.subtest_states);
+		env.subtest_states = NULL;
+	}
+	fprintf(stdout, "#%d\t%s: %s\n", test_index + 1, test->name,
+		test->failed ? "KO" : "OK");
+}
+
+static void test_teardown(struct test_state *test, int test_index)
+{
+	dump_results(test, test_index);
+	free(env.current_test->name);
+	env.current_test = NULL;
+}
+
+static int parse_args(int argc, char *argv[])
+{
+	if (argc != 2)
+		return 1;
+	if (access(argv[1], R_OK|X_OK))
+		return 1;
+	env.bpftool_path = argv[1];
+
+	return 0;
+}
+
+static void usage(char *prog)
+{
+	fprintf(stdout, "Usage: %s <bpftool_path>\n", prog);
+	fprintf(stdout, "\t<bpftool_path>: path to the bpftool binary to test\n");
+}
+
+int main(int argc, char *argv[])
+{
+	struct test_state *ctx = NULL;
+	int i;
+
+	if (parse_args(argc, argv)) {
+		fprintf(stderr, "Invalid arguments\n");
+		usage(argv[0]);
+		exit(EXIT_FAILURE);
+	}
+
+	ctx = calloc(tests_count, sizeof(struct test_state));
+	if (!ctx)
+		exit(EXIT_FAILURE);
+
+	for (i = 0; i < tests_count; i++) {
+		test_setup(&ctx[i], prog_test_defs[i].test_name);
+		hijack_stdio();
+		prog_test_defs[i].run_test();
+		test__end_subtest();
+		restore_stdio();
+		test_teardown(&ctx[i], i);
+	}
+
+	fprintf(stdout, "Summary: %d PASSED, %d FAILED\n",
+		tests_count - env.failure_cnt, env.failure_cnt);
+	free(ctx);
+	return env.failure_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
+}
diff --git a/tools/testing/selftests/bpf/test_bpftool.h b/tools/testing/selftests/bpf/test_bpftool.h
new file mode 100644
index 000000000000..a78659eeaf2b
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_bpftool.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+
+#include <stdio.h>
+#include <stdbool.h>
+
+extern struct bpftool_runner_env env;
+
+void test__fail(void);
+
+struct test_state {
+	char *name;
+	char *log;
+	size_t log_size;
+	bool failed;
+	int subtests_count;
+	int subtests_failures;
+	FILE *saved_stdout;
+	FILE *saved_stderr;
+};
+
+struct subtest_state {
+	char *name;
+	char *log;
+	size_t log_size;
+	bool failed;
+};
+struct bpftool_runner_env {
+	char *bpftool_path;
+	int failure_cnt;
+	FILE *saved_stdout;
+	FILE *saved_stderr;
+	struct test_state *current_test;
+	struct subtest_state *current_subtest;
+	struct subtest_state *subtest_states;
+};

-- 
2.52.0

Re: [PATCH bpf-next 2/4] bpf/selftests: introduce bptool test runner and a first test

[Quentin Monnet]

2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)"
<alexis.lothore@bootlin.com>
> The tools/testing/selftests/bpf directory contains multiple scripts
> (shell, python, c code, etc) that aim to test some specific features
> from bpftool. Those isolated tests are currently not executed by any CI
> automation. Create a dedicated runner for any bpftool-related test that
> can then be added to the list of executed runners in bpf CI automation.
> This new runner (and the corresponding Makefile tooling) is highly
> inspired from test_progs, but kept a bit simpler. This version supports
> the following features:
> 
> - autodetection of bpftool test stored in the in bpftool_tests


(Typo: "in the in")


>   directory
> - bpftool binary under test is passed as runner argument
> - a few helpers to allow to easily run abpftool commands while possibly


(Typo: abpftool)


>   collecting the output
> - usage of assert macros shared with test_progs
> - basic sub-tests management
> - logs collection, logs being dumped only for failed tests
> - exit code reflecting whether all tests have passed or not
> 
> As this runner needs at least one test to be implemented to properly
> compile, also bring bpftool_metadata, which is the conversion of
> test_bpftool_metadata.sh: this test validates that the output of some
> basic prog/map listings done with bpftool properly returns the metadata
> collected from the .rodata section of eBPF programs.
> 
> This new runner gives an output similar to the one generated by
> test_progs:
> 
>   #2/1	metadata/metadata_unused: OK
>   #2/2	metadata/metadata_used: OK
>   #2	metadata: OK
>   Summary: 1 PASSED, 0 FAILED
> 
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>


Thanks a lot for this! It looks good, I only have a few minor comments.


> ---
>  tools/testing/selftests/bpf/.gitignore             |   1 +
>  tools/testing/selftests/bpf/Makefile               |  14 ++-
>  tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++++++++++++++
>  tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +++
>  .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
>  .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++++++++++++++++
>  tools/testing/selftests/bpf/test_bpftool.c         | 126 ++++++++++++++++++++
>  tools/testing/selftests/bpf/test_bpftool.h         |  36 ++++++
>  8 files changed, 439 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
> index b8bf51b7a0b0..9498cc11de97 100644
> --- a/tools/testing/selftests/bpf/.gitignore
> +++ b/tools/testing/selftests/bpf/.gitignore
> @@ -2,6 +2,7 @@
>  bpftool
>  bpf-helpers*
>  bpf-syscall*
> +test_bpftool
>  test_verifier
>  test_maps
>  test_lru_map
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index fd42b7193d4e..a1fe94efa53c 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -76,7 +76,8 @@ endif
>  TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_progs \
>  	test_sockmap \
>  	test_tcpnotify_user \
> -	test_progs-no_alu32
> +	test_progs-no_alu32 \
> +	test_bpftool
>  TEST_INST_SUBDIRS := no_alu32
>  
>  # Also test bpf-gcc, if present
> @@ -791,6 +792,17 @@ TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built)
>  TRUNNER_BPF_CFLAGS :=
>  $(eval $(call DEFINE_TEST_RUNNER,test_maps))
>  
> +# Define bpftool test runner.
> +TRUNNER_TESTS_DIR := bpftool_tests
> +TRUNNER_BPF_PROGS_DIR := progs
> +TRUNNER_EXTRA_SOURCES := test_bpftool.c \
> +			 bpftool_helpers.c
> +TRUNNER_LIB_SOURCES :=
> +TRUNNER_EXTRA_FILES :=
> +TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
> +TRUNNER_BPF_CFLAGS :=
> +$(eval $(call DEFINE_TEST_RUNNER,test_bpftool))
> +
>  # Define test_verifier test runner.
>  # It is much simpler than test_maps/test_progs and sufficiently different from
>  # them (e.g., test.h is using completely pattern), that it's worth just
> diff --git a/tools/testing/selftests/bpf/bpftool_helpers.c b/tools/testing/selftests/bpf/bpftool_helpers.c
> new file mode 100644
> index 000000000000..ff8084d9a121
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/bpftool_helpers.c
> @@ -0,0 +1,114 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +#include "bpftool_helpers.h"
> +#include "test_bpftool.h"
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <stdbool.h>
> +
> +#define BPFTOOL_PATH		"./tools/sbin/bpftool"
> +#define BPFTOOL_CMD_MAX_LEN	256
> +
> +static int run_command(char *command, bool get_output, char *output_buf, size_t output_max_len)
> +{
> +	FILE *f;
> +	int ret;
> +
> +	f = popen(command, "r");
> +	if (!f)
> +		return 1;
> +
> +	if (get_output)
> +		fread(output_buf, 1, output_max_len, f);
> +	ret = pclose(f);
> +
> +	return ret;
> +}
> +
> +int run_bpftool_command(char *args)
> +{
> +	char cmd[BPFTOOL_CMD_MAX_LEN];
> +	int ret;
> +
> +	ret = snprintf(cmd, BPFTOOL_CMD_MAX_LEN, "%s %s > /dev/null 2>&1",
> +		       env.bpftool_path, args);
> +	if (ret !=
> +	    strlen(env.bpftool_path) + 1 + strlen(args) + strlen(" > /dev/null 2>&1")) {
> +		fprintf(stderr, "Failed to generate bpftool command\n");
> +		return 1;
> +	}
> +
> +	return run_command(cmd, false, NULL, 0);
> +}
> +
> +int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len)
> +{
> +	int ret;
> +	char cmd[BPFTOOL_CMD_MAX_LEN];
> +
> +	ret = snprintf(cmd, BPFTOOL_CMD_MAX_LEN, "%s %s", env.bpftool_path,
> +		       args);
> +	if (ret != strlen(args) + strlen(env.bpftool_path) + 1) {
> +		fprintf(stderr, "Failed to generate bpftool command");
> +		return 1;
> +	}
> +
> +	return run_command(cmd, true, output_buf, output_max_len);
> +}
> +
> +void hijack_stdio(void)
> +{
> +	fflush(stdout);
> +	fflush(stderr);
> +	if (env.current_subtest) {
> +		env.current_test->saved_stdout = stdout;
> +		env.current_test->saved_stderr = stderr;
> +		stdout = open_memstream(&env.current_subtest->log,
> +					&env.current_subtest->log_size);


Missing check on open_memstream() return value? Same below.


> +


Nit: blank line


> +	} else {
> +		env.saved_stdout = stdout;
> +		env.saved_stderr = stderr;
> +		stdout = open_memstream(&env.current_test->log,
> +					&env.current_test->log_size);
> +	}
> +	stderr = stdout;
> +}
> +
> +void restore_stdio(void)
> +{
> +	fclose(stdout);
> +	if (env.current_subtest) {
> +		stdout = env.current_test->saved_stdout;
> +		stderr = env.current_test->saved_stderr;
> +


Nit: blank line


> +	} else {
> +		stdout = env.saved_stdout;
> +		stderr = env.saved_stderr;
> +	}
> +


Nit: blank line again


> +}
> +
> +void test__start_subtest(const char *subtest_name)
> +{
> +	test__end_subtest();
> +	env.current_test->subtests_count++;
> +	env.subtest_states = realloc(env.subtest_states,
> +				     env.current_test->subtests_count *
> +					     sizeof(struct subtest_state));


Missing check on realloc()'s return value?


> +	env.current_subtest =
> +		&env.subtest_states[env.current_test->subtests_count - 1];
> +	memset(env.current_subtest, 0, sizeof(struct subtest_state));
> +	env.current_subtest->name = strdup(subtest_name);
> +
> +	hijack_stdio();
> +}
> +
> +void test__end_subtest(void)
> +{
> +	if (env.current_subtest) {
> +		restore_stdio();
> +		env.current_subtest = NULL;
> +	}
> +}
> +
> diff --git a/tools/testing/selftests/bpf/bpftool_helpers.h b/tools/testing/selftests/bpf/bpftool_helpers.h
> new file mode 100644
> index 000000000000..1eacec7936ba
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/bpftool_helpers.h
> @@ -0,0 +1,19 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#pragma once
> +
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <stdbool.h>
> +
> +#define MAX_BPFTOOL_CMD_LEN	(256)
> +
> +#ifndef ARRAY_SIZE
> +#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
> +#endif
> +
> +int run_bpftool_command(char *args);
> +int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len);
> +void test__start_subtest(const char *subtests_name);
> +void test__end_subtest(void);
> +void hijack_stdio(void);
> +void restore_stdio(void);
> diff --git a/tools/testing/selftests/bpf/bpftool_tests/.gitignore b/tools/testing/selftests/bpf/bpftool_tests/.gitignore
> new file mode 100644
> index 000000000000..89c4a3d37544
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/bpftool_tests/.gitignore
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +tests.h
> diff --git a/tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c b/tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c
> new file mode 100644
> index 000000000000..e7146b26f298
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c
> @@ -0,0 +1,128 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +#include <bpftool_helpers.h>
> +#include <test_bpftool.h>
> +#include <assert_helpers.h>
> +#include <linux/bpf.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +#include <sys/stat.h>
> +#include <stdbool.h>
> +
> +#define BPFFS_DIR	"/sys/fs/bpf/test_metadata"
> +#define BPFFS_USED	BPFFS_DIR "/used"
> +#define BPFFS_UNUSED	BPFFS_DIR "/unused"
> +
> +#define BPF_FILE_USED	"metadata_used.bpf.o"
> +#define BPF_FILE_UNUSED "metadata_unused.bpf.o"
> +
> +#define MAX_BPFTOOL_OUTPUT_LEN	(100*1000)


Maybe a power of 2 instead, 64k?


> +
> +#define MAX_TOKENS_TO_CHECK	3
> +static char output[MAX_BPFTOOL_OUTPUT_LEN];
> +
> +struct test_desc {
> +	char *name;
> +	char *bpf_prog;
> +	char *bpffs_path;
> +	char *expected_output[MAX_TOKENS_TO_CHECK];
> +	char *expected_output_json[MAX_TOKENS_TO_CHECK];
> +};
> +
> +static int setup(struct test_desc *test)
> +{
> +	return mkdir(BPFFS_DIR, 0700);
> +}
> +
> +static void cleanup(struct test_desc *test)
> +{
> +	unlink(test->bpffs_path);
> +	rmdir(BPFFS_DIR);
> +}
> +
> +static int check_metadata(char *buf, char * const *tokens, int count)
> +{
> +	int i;
> +
> +	for (i = 0; i < count && tokens[i]; i++)
> +		if (!strstr(buf, tokens[i]))
> +			return 1;
> +
> +	return 0;
> +}
> +
> +static void run_test(struct test_desc *test)
> +{
> +	int ret;
> +	char cmd[MAX_BPFTOOL_CMD_LEN];
> +
> +	snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog load %s %s",
> +			test->bpf_prog, test->bpffs_path);
> +	ret = run_bpftool_command(cmd);
> +	if (!ASSERT_OK(ret, "load program"))
> +		return;
> +
> +	/* Check output with default format */
> +	ret = get_bpftool_command_output("prog show name prog", output,


I'd probably use "pinned" + pinned path rather than "name prog",
especially because the name for the program does not otherwise appear in
this file (and same below).


> +			MAX_BPFTOOL_OUTPUT_LEN);
> +	if (ASSERT_OK(ret, "get program info")) {
> +		ret = check_metadata(output, test->expected_output,
> +				ARRAY_SIZE(test->expected_output));
> +		ASSERT_OK(ret, "find metadata");
> +	}
> +
> +	/* Check output with json format */
> +	ret = get_bpftool_command_output("prog -j show name prog", output,
> +					 MAX_BPFTOOL_OUTPUT_LEN);
> +	if (ASSERT_OK(ret, "get program info in json")) {
> +		ret = check_metadata(output, test->expected_output_json,
> +				ARRAY_SIZE(test->expected_output_json));
> +		ASSERT_OK(ret, "find metadata in json");
> +	}
> +


Nit: blank line


> +}
> +
> +struct test_desc tests[] = {
> +	{
> +		.name = "metadata_unused",
> +		.bpf_prog = BPF_FILE_UNUSED,
> +		.bpffs_path = BPFFS_UNUSED,
> +		.expected_output = {
> +			"a = \"foo\"",
> +			"b = 1"
> +		},
> +		.expected_output_json = {
> +			"\"metadata\":{\"a\":\"foo\",\"b\":1}"
> +		}
> +	},
> +	{
> +		.name = "metadata_used",
> +		.bpf_prog = BPF_FILE_USED,
> +		.bpffs_path = BPFFS_USED,
> +		.expected_output = {
> +			"a = \"bar\"",
> +			"b = 2"
> +		},
> +		.expected_output_json = {
> +			"\"metadata\":{\"a\":\"bar\",\"b\":2}"
> +		}
> +	}
> +};
> +
> +static const int tests_count = ARRAY_SIZE(tests);
> +
> +void test_metadata(void)
> +{
> +	int i, ret;
> +
> +	for (i = 0; i < tests_count; i++) {
> +		test__start_subtest(tests[i].name);
> +		ret = setup(&tests[i]);
> +		if (!ASSERT_OK(ret, "setup bpffs pin dir"))
> +			continue;
> +		run_test(&tests[i]);
> +		cleanup(&tests[i]);
> +	}
> +


Nit: blank line


> +}
> +
> diff --git a/tools/testing/selftests/bpf/test_bpftool.c b/tools/testing/selftests/bpf/test_bpftool.c
> new file mode 100644
> index 000000000000..b5fb17d5ea2d
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/test_bpftool.c
> @@ -0,0 +1,126 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +#include <test_bpftool.h>
> +#include <bpftool_helpers.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <stdbool.h>
> +#include <string.h>
> +#include <unistd.h>
> +
> +struct bpftool_runner_env env = {0};
> +
> +#define DEFINE_TEST(name) extern void test_##name(void);
> +#include <bpftool_tests/tests.h>
> +#undef DEFINE_TEST
> +
> +struct prog_test_def {
> +	char *test_name;
> +	void (*run_test)(void);
> +};
> +
> +static struct prog_test_def prog_test_defs[] = {
> +#define DEFINE_TEST(name) {			\
> +	.test_name = #name,			\
> +	.run_test = &test_##name,		\
> +},
> +#include <bpftool_tests/tests.h>
> +#undef DEFINE_TEST
> +};
> +
> +
> +static const int tests_count = ARRAY_SIZE(prog_test_defs);
> +
> +/* Needed method for the assert macros exposed by assert_helpers.h */
> +void test__fail(void)
> +{
> +	if (env.current_subtest)
> +		env.current_subtest->failed = true;
> +	if (!env.current_test->failed)
> +		env.failure_cnt++;
> +	env.current_test->failed = true;
> +}
> +
> +static void test_setup(struct test_state *test, char *name)
> +{
> +	env.current_test = test;
> +	env.current_test->name = strdup(name);
> +}
> +
> +static void dump_results(struct test_state *test, int test_index)


Would you mind renaming this function? It also frees the logs and names
and subtests states. Maybe "dump_results_and_cleanup"? Or move the all
the clean-up to test_teardown(). Even if we loop again over the
subtests, the overhead should be minimal, and I'd find it easier to follow.


> +{
> +	int j;


Let's use "i" rather than "j"


> +
> +	if (test->failed)
> +		fprintf(stdout, "%s\n", test->log);
> +	free(test->log);
> +	for (j = 0; j < test->subtests_count; j++) {
> +		if (env.subtest_states[j].failed)
> +			fprintf(stdout, "%s\n", env.subtest_states[j].log);
> +		free(env.subtest_states[j].log);
> +		fprintf(stdout, "#%d/%d\t%s/%s: %s\n", test_index+1, j+1,
> +				env.current_test->name,
> +				env.subtest_states[j].name,
> +				env.subtest_states[j].failed ? "KO" : "OK");
> +		free(env.subtest_states[j].name);
> +	}
> +	if (env.current_test->subtests_count) {
> +		free(env.subtest_states);
> +		env.subtest_states = NULL;
> +	}
> +	fprintf(stdout, "#%d\t%s: %s\n", test_index + 1, test->name,
> +		test->failed ? "KO" : "OK");
> +}
> +
> +static void test_teardown(struct test_state *test, int test_index)
> +{
> +	dump_results(test, test_index);
> +	free(env.current_test->name);
> +	env.current_test = NULL;
> +}
> +
> +static int parse_args(int argc, char *argv[])
> +{
> +	if (argc != 2)
> +		return 1;
> +	if (access(argv[1], R_OK|X_OK))
> +		return 1;
> +	env.bpftool_path = argv[1];
> +
> +	return 0;
> +}
> +
> +static void usage(char *prog)
> +{
> +	fprintf(stdout, "Usage: %s <bpftool_path>\n", prog);
> +	fprintf(stdout, "\t<bpftool_path>: path to the bpftool binary to test\n");
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	struct test_state *ctx = NULL;
> +	int i;
> +
> +	if (parse_args(argc, argv)) {
> +		fprintf(stderr, "Invalid arguments\n");
> +		usage(argv[0]);
> +		exit(EXIT_FAILURE);
> +	}
> +
> +	ctx = calloc(tests_count, sizeof(struct test_state));
> +	if (!ctx)
> +		exit(EXIT_FAILURE);
> +
> +	for (i = 0; i < tests_count; i++) {
> +		test_setup(&ctx[i], prog_test_defs[i].test_name);
> +		hijack_stdio();


Nit: Move hijack_stdio()/restore_stdio() to test_setup()/test_teardown()?


> +		prog_test_defs[i].run_test();
> +		test__end_subtest();
> +		restore_stdio();
> +		test_teardown(&ctx[i], i);
> +	}
> +
> +	fprintf(stdout, "Summary: %d PASSED, %d FAILED\n",
> +		tests_count - env.failure_cnt, env.failure_cnt);
> +	free(ctx);
> +	return env.failure_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
> +}
> diff --git a/tools/testing/selftests/bpf/test_bpftool.h b/tools/testing/selftests/bpf/test_bpftool.h
> new file mode 100644
> index 000000000000..a78659eeaf2b
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/test_bpftool.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#pragma once
> +
> +#include <stdio.h>
> +#include <stdbool.h>
> +
> +extern struct bpftool_runner_env env;
> +
> +void test__fail(void);
> +
> +struct test_state {
> +	char *name;
> +	char *log;
> +	size_t log_size;
> +	bool failed;
> +	int subtests_count;
> +	int subtests_failures;
> +	FILE *saved_stdout;
> +	FILE *saved_stderr;
> +};
> +
> +struct subtest_state {
> +	char *name;
> +	char *log;
> +	size_t log_size;
> +	bool failed;
> +};
> +struct bpftool_runner_env {
> +	char *bpftool_path;
> +	int failure_cnt;
> +	FILE *saved_stdout;
> +	FILE *saved_stderr;
> +	struct test_state *current_test;
> +	struct subtest_state *current_subtest;
> +	struct subtest_state *subtest_states;
> +};
> 

Re: [PATCH bpf-next 2/4] bpf/selftests: introduce bptool test runner and a first test

[Alexis Lothoré]

Hi Quentin, thanks for the review !

I agree with all your comments. Andrii raised some concerns about
bringing a new runner, I'll wait for this new runner question to be
settled before bringing those changes, as it could make this series
change quite a lot.

Thanks,

Alexis

-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH bpf-next 3/4] selftests/bpf: add bpftool map manipulations tests

[Alexis Lothoré (eBPF Foundation)]

The test_bpftool_map.sh script tests that maps read/write accesses
are being properly allowed/refused by the kernel depending on a specific
fmod_ret program being attached on security_bpf_map function.

Rewrite this test to integrate it in the new test_bpftool runner. The
new test spawns a few subtests:

  #1/1	maps_access/unprotected_unpinned: OK
  #1/2	maps_access/unprotected_pinned: OK
  #1/3	maps_access/protected_unpinned: OK
  #1/4	maps_access/protected_pinned: OK
  #1/5	maps_access/nested_maps: OK
  #1/6	maps_access/btf_list: OK
  #1	maps_access: OK
  #2/1	metadata/metadata_unused: OK
  #2/2	metadata/metadata_used: OK
  #2	metadata: OK
  Summary: 2 PASSED, 0 FAILED

Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
---
 .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++++
 1 file changed, 370 insertions(+)

diff --git a/tools/testing/selftests/bpf/bpftool_tests/bpftool_maps_access.c b/tools/testing/selftests/bpf/bpftool_tests/bpftool_maps_access.c
new file mode 100644
index 000000000000..da4277c79e22
--- /dev/null
+++ b/tools/testing/selftests/bpf/bpftool_tests/bpftool_maps_access.c
@@ -0,0 +1,370 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <bpf/libbpf.h>
+#include <bpftool_helpers.h>
+#include <stdlib.h>
+#include <test_bpftool.h>
+#include <assert_helpers.h>
+#include <linux/bpf.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <stdint.h>
+#include <sys/stat.h>
+#include <stdbool.h>
+#include <bpf/bpf.h>
+#include "security_bpf_map.skel.h"
+
+#define PROTECTED_MAP_NAME	"prot_map"
+#define UNPROTECTED_MAP_NAME	"not_prot_map"
+#define BPF_ITER_FILE		"bpf_iter_map_elem.bpf.o"
+#define BPFFS_PIN_DIR		"/sys/fs/bpf/test_bpftool_map"
+#define INNER_MAP_NAME		"inner_map_tt"
+#define OUTER_MAP_NAME		"outer_map_tt"
+
+#define MAP_NAME_MAX_LEN	64
+#define PATH_MAX_LEN		128
+
+enum map_protection {
+	PROTECTED,
+	UNPROTECTED
+};
+
+struct test_desc {
+	char *name;
+	enum map_protection protection;
+	struct bpf_map *map;
+	char *map_name;
+	bool pinned;
+	char pin_path[PATH_MAX_LEN];
+	bool write_must_fail;
+};
+
+static struct security_bpf_map *general_setup(void)
+{
+	struct security_bpf_map *skel;
+	uint32_t key, value;
+	int ret, i;
+
+	skel = security_bpf_map__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open and load skeleton"))
+		goto end;
+
+	struct bpf_map *maps[] = {skel->maps.prot_map, skel->maps.not_prot_map};
+
+	ret = security_bpf_map__attach(skel);
+	if (!ASSERT_OK(ret, "attach maps security programs"))
+		goto end_destroy;
+
+	for (i = 0; i < sizeof(maps)/sizeof(struct bpf_map *); i++) {
+		for (key = 0; key < 2; key++) {
+			int ret = bpf_map__update_elem(maps[i], &key,
+					sizeof(key), &key, sizeof(key),
+					0);
+			if (!ASSERT_OK(ret, "set initial map value"))
+				goto end_destroy;
+		}
+	}
+
+	key = 0;
+	value = 1;
+	ret = bpf_map__update_elem(skel->maps.prot_status_map, &key,
+			sizeof(key), &value, sizeof(value), 0);
+	if (!ASSERT_OK(ret, "configure map protection"))
+		goto end_destroy;
+
+	if (!ASSERT_OK(mkdir(BPFFS_PIN_DIR, S_IFDIR), "create bpffs pin dir"))
+		goto end_destroy;
+
+	return skel;
+end_destroy:
+	security_bpf_map__destroy(skel);
+end:
+	return NULL;
+}
+
+static void general_cleanup(struct security_bpf_map *skel)
+{
+	rmdir(BPFFS_PIN_DIR);
+	security_bpf_map__destroy(skel);
+}
+
+static void update_test_desc(struct security_bpf_map *skel,
+			      struct test_desc *test)
+{
+	/* Now that the skeleton is loaded, update all missing fields to
+	 * have the subtest properly configured
+	 */
+	if (test->protection == PROTECTED) {
+		test->map = skel->maps.prot_map;
+		test->map_name = PROTECTED_MAP_NAME;
+	} else {
+		test->map = skel->maps.not_prot_map;
+		test->map_name = UNPROTECTED_MAP_NAME;
+	}
+}
+
+static int test_setup(struct security_bpf_map *skel, struct test_desc *desc)
+{
+	int ret;
+
+	update_test_desc(skel, desc);
+
+	if (desc->pinned) {
+		ret = snprintf(desc->pin_path, PATH_MAX_LEN, "%s/%s", BPFFS_PIN_DIR,
+				desc->name);
+		if (!ASSERT_GT(ret, 0, "format pin path"))
+			return 1;
+		ret = bpf_map__pin(desc->map, desc->pin_path);
+		if (!ASSERT_OK(ret, "pin map"))
+			return 1;
+	}
+
+	return 0;
+}
+
+static void test_cleanup(struct test_desc *desc)
+{
+	if (desc->pinned)
+		bpf_map__unpin(desc->map, NULL);
+}
+
+static int lookup_map_value(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "map lookup %s key 0 0 0 0",
+			map_handle);
+	if (!ASSERT_GT(ret, 0, "format map lookup cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int read_map_btf_data(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "btf dump map %s",
+			map_handle);
+	if (!ASSERT_GT(ret, 0, "format map btf dump cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int write_map_value(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN,
+		       "map update %s key 0 0 0 0 value 1 1 1 1", map_handle);
+	if (!ASSERT_GT(ret, 0, "format value write cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int delete_map_value(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN,
+		       "map delete %s key 0 0 0 0", map_handle);
+	if (!ASSERT_GT(ret, 0, "format value deletion cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int iterate_on_map_values(char *map_handle, char *iter_pin_path)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "iter pin %s %s map %s",
+		       BPF_ITER_FILE, iter_pin_path, map_handle);
+	if (!ASSERT_GT(ret, 0, "format iterator cration cmd"))
+		return 1;
+	ret = run_bpftool_command(cmd);
+	if (ret)
+		return ret;
+	ret = snprintf(cmd, MAP_NAME_MAX_LEN, "cat %s", iter_pin_path);
+	if (ret < 0)
+		goto cleanup;
+	ret = system(cmd);
+
+cleanup:
+	unlink(iter_pin_path);
+	return ret;
+}
+
+static int create_inner_map(void)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(
+		cmd, MAX_BPFTOOL_CMD_LEN,
+		"map create %s/%s type array key 4 value 4 entries 4 name %s",
+		BPFFS_PIN_DIR, INNER_MAP_NAME, INNER_MAP_NAME);
+	if (!ASSERT_GT(ret, 0, "format inner map create cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int create_outer_map(void)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(
+		cmd, MAX_BPFTOOL_CMD_LEN,
+		"map create %s/%s type hash_of_maps key 4 value 4 entries 2 name %s inner_map name %s",
+		BPFFS_PIN_DIR, OUTER_MAP_NAME, OUTER_MAP_NAME, INNER_MAP_NAME);
+	if (!ASSERT_GT(ret, 0, "format outer map create cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static void delete_pinned_map(char *map_name)
+{
+	char pin_path[PATH_MAX_LEN];
+	int ret;
+
+	ret = snprintf(pin_path, PATH_MAX_LEN, "%s/%s", BPFFS_PIN_DIR,
+		       map_name);
+	if (ret >= 0)
+		unlink(pin_path);
+}
+
+static int add_outer_map_entry(int key)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(
+		cmd, MAX_BPFTOOL_CMD_LEN,
+		"map update pinned %s/%s key %d 0 0 0 value name %s",
+		BPFFS_PIN_DIR, OUTER_MAP_NAME, key, INNER_MAP_NAME);
+	if (!ASSERT_GT(ret, 0, "format outer map value addition cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static void test_basic_access(struct test_desc *desc)
+{
+	char map_handle[MAP_NAME_MAX_LEN];
+	char iter_pin_path[PATH_MAX_LEN];
+	int ret;
+
+	if (desc->pinned)
+		ret = snprintf(map_handle, MAP_NAME_MAX_LEN, "pinned %s",
+			       desc->pin_path);
+	else
+		ret = snprintf(map_handle, MAP_NAME_MAX_LEN, "name %s",
+			       desc->map_name);
+	if (!ASSERT_GT(ret, 0, "format map handle"))
+		return;
+
+	ret = lookup_map_value(map_handle);
+	ASSERT_OK(ret, "read map value");
+
+	ret = read_map_btf_data(map_handle);
+	ASSERT_OK(ret, "read map btf data");
+
+	ret = write_map_value(map_handle);
+	ASSERT_OK(desc->write_must_fail ? !ret : ret, "write map value");
+
+	ret = delete_map_value(map_handle);
+	ASSERT_OK(desc->write_must_fail ? !ret : ret, "delete map value");
+	/* Restore deleted value */
+	if (!ret)
+		write_map_value(map_handle);
+
+	ret = snprintf(iter_pin_path, PATH_MAX_LEN, "%s/iter", BPFFS_PIN_DIR);
+	if (ASSERT_GT(ret, 0, "format iter pin path")) {
+		ret = iterate_on_map_values(map_handle, iter_pin_path);
+		ASSERT_OK(ret, "iterate on map values");
+	}
+}
+
+static void test_create_nested_maps(void)
+{
+	if (!ASSERT_OK(create_inner_map(), "create inner map"))
+		return;
+	if (!ASSERT_OK(create_outer_map(), "create outer map"))
+		goto end_cleanup_inner;
+	ASSERT_OK(add_outer_map_entry(0), "add a first entry in outer map");
+	ASSERT_OK(add_outer_map_entry(1), "add a second entry in outer map");
+	ASSERT_NEQ(add_outer_map_entry(2), 0, "add a third entry in outer map");
+
+	delete_pinned_map(OUTER_MAP_NAME);
+end_cleanup_inner:
+	delete_pinned_map(INNER_MAP_NAME);
+}
+
+static void test_btf_list(void)
+{
+	ASSERT_OK(run_bpftool_command("btf list"), "list btf data");
+}
+
+static struct test_desc tests[] = {
+	{
+		.name = "unprotected_unpinned",
+		.protection = UNPROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = false,
+		.write_must_fail = false,
+	},
+	{
+		.name = "unprotected_pinned",
+		.protection = UNPROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = true,
+		.write_must_fail = false,
+	},
+	{
+		.name = "protected_unpinned",
+		.protection = PROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = false,
+		.write_must_fail = true,
+	},
+	{
+		.name = "protected_pinned",
+		.protection = PROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = true,
+		.write_must_fail = true,
+	}
+};
+
+static const size_t tests_count = ARRAY_SIZE(tests);
+
+void test_maps_access(void)
+{
+	struct security_bpf_map *skel;
+	struct test_desc *current;
+	int i;
+
+	skel = general_setup();
+	if (!ASSERT_OK_PTR(skel, "prepare programs"))
+		goto cleanup;
+
+	for (i = 0; i < tests_count; i++) {
+		current = &tests[i];
+		test__start_subtest(current->name);
+		if (ASSERT_OK(test_setup(skel, current), "subtest setup"))
+			test_basic_access(current);
+		test_cleanup(current);
+	}
+	test__start_subtest("nested_maps");
+	test_create_nested_maps();
+	test__start_subtest("btf_list");
+	test_btf_list();
+
+cleanup:
+	general_cleanup(skel);
+}
+

-- 
2.52.0

Re: [PATCH bpf-next 3/4] selftests/bpf: add bpftool map manipulations tests

[Quentin Monnet]

2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)"
<alexis.lothore@bootlin.com>
> The test_bpftool_map.sh script tests that maps read/write accesses
> are being properly allowed/refused by the kernel depending on a specific
> fmod_ret program being attached on security_bpf_map function.
> 
> Rewrite this test to integrate it in the new test_bpftool runner. The
> new test spawns a few subtests:
> 
>   #1/1	maps_access/unprotected_unpinned: OK
>   #1/2	maps_access/unprotected_pinned: OK
>   #1/3	maps_access/protected_unpinned: OK
>   #1/4	maps_access/protected_pinned: OK
>   #1/5	maps_access/nested_maps: OK
>   #1/6	maps_access/btf_list: OK
>   #1	maps_access: OK
>   #2/1	metadata/metadata_unused: OK
>   #2/2	metadata/metadata_used: OK
>   #2	metadata: OK
>   Summary: 2 PASSED, 0 FAILED
> 
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>


Acked-by: Quentin Monnet <qmo@kernel.org>

[PATCH bpf-next 4/4] selftests/bpf: remove converted bpftool test scripts

[Alexis Lothoré (eBPF Foundation)]

Now that test_bpftool_map.sh and test_bpftool_metadata.sh are integrated
into test_bpftool runner, remove those.

Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
---
 tools/testing/selftests/bpf/Makefile               |   2 -
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
 3 files changed, 485 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index a1fe94efa53c..daa25d6c8fbb 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -109,8 +109,6 @@ TEST_PROGS := test_kmod.sh \
 	test_xdping.sh \
 	test_bpftool_build.sh \
 	test_bpftool.sh \
-	test_bpftool_map.sh \
-	test_bpftool_metadata.sh \
 	test_doc_build.sh \
 	test_xsk.sh \
 	test_xdp_features.sh
diff --git a/tools/testing/selftests/bpf/test_bpftool_map.sh b/tools/testing/selftests/bpf/test_bpftool_map.sh
deleted file mode 100755
index 515b1df0501e..000000000000
--- a/tools/testing/selftests/bpf/test_bpftool_map.sh
+++ /dev/null
@@ -1,398 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: GPL-2.0
-
-# Kselftest framework requirement - SKIP code is 4.
-ksft_skip=4
-
-TESTNAME="bpftool_map"
-BPF_FILE="security_bpf_map.bpf.o"
-BPF_ITER_FILE="bpf_iter_map_elem.bpf.o"
-PROTECTED_MAP_NAME="prot_map"
-NOT_PROTECTED_MAP_NAME="not_prot_map"
-BPF_FS_TMP_PARENT="/tmp"
-BPF_FS_PARENT=$(awk '$3 == "bpf" {print $2; exit}' /proc/mounts)
-BPF_FS_PARENT=${BPF_FS_PARENT:-$BPF_FS_TMP_PARENT}
-# bpftool will mount bpf file system under BPF_DIR if it is not mounted
-# under BPF_FS_PARENT.
-BPF_DIR="$BPF_FS_PARENT/test_$TESTNAME"
-SCRIPT_DIR=$(dirname $(realpath "$0"))
-BPF_FILE_PATH="$SCRIPT_DIR/$BPF_FILE"
-BPF_ITER_FILE_PATH="$SCRIPT_DIR/$BPF_ITER_FILE"
-BPFTOOL_PATH="bpftool"
-# Assume the script is located under tools/testing/selftests/bpf/
-KDIR_ROOT_DIR=$(realpath "$SCRIPT_DIR"/../../../../)
-
-_cleanup()
-{
-	set +eu
-
-	# If BPF_DIR is a mount point this will not remove the mount point itself.
-	[ -d "$BPF_DIR" ] && rm -rf "$BPF_DIR" 2> /dev/null
-
-	# Unmount if BPF filesystem was temporarily created.
-	if [ "$BPF_FS_PARENT" = "$BPF_FS_TMP_PARENT" ]; then
-		# A loop and recursive unmount are required as bpftool might
-		# create multiple mounts. For example, a bind mount of the directory
-		# to itself. The bind mount is created to change mount propagation
-		# flags on an actual mount point.
-		max_attempts=3
-		attempt=0
-		while mountpoint -q "$BPF_DIR" && [ $attempt -lt $max_attempts ]; do
-			umount -R "$BPF_DIR" 2>/dev/null
-			attempt=$((attempt+1))
-		done
-
-		# The directory still exists. Remove it now.
-		[ -d "$BPF_DIR" ] && rm -rf "$BPF_DIR" 2>/dev/null
-	fi
-}
-
-cleanup_skip()
-{
-	echo "selftests: $TESTNAME [SKIP]"
-	_cleanup
-
-	exit $ksft_skip
-}
-
-cleanup()
-{
-	if [ "$?" = 0 ]; then
-		echo "selftests: $TESTNAME [PASS]"
-	else
-		echo "selftests: $TESTNAME [FAILED]"
-	fi
-	_cleanup
-}
-
-check_root_privileges() {
-	if [ $(id -u) -ne 0 ]; then
-		echo "Need root privileges"
-		exit $ksft_skip
-	fi
-}
-
-# Function to verify bpftool path.
-# Parameters:
-#   $1: bpftool path
-verify_bpftool_path() {
-	local bpftool_path="$1"
-	if ! "$bpftool_path" version > /dev/null 2>&1; then
-		echo "Could not run test without bpftool"
-		exit $ksft_skip
-	fi
-}
-
-# Function to verify BTF support.
-# The test requires BTF support for fmod_ret programs.
-verify_btf_support() {
-	if [ ! -f /sys/kernel/btf/vmlinux ]; then
-		echo "Could not run test without BTF support"
-		exit $ksft_skip
-	fi
-}
-
-# Function to initialize map entries with keys [0..2] and values set to 0.
-# Parameters:
-#  $1: Map name
-#  $2: bpftool path
-initialize_map_entries() {
-	local map_name="$1"
-	local bpftool_path="$2"
-
-	for key in 0 1 2; do
-		"$bpftool_path" map update name "$map_name" key $key 0 0 0 value 0 0 0 $key
-	done
-}
-
-# Test read access to the map.
-# Parameters:
-#   $1: Name command (name/pinned)
-#   $2: Map name
-#   $3: bpftool path
-#   $4: key
-access_for_read() {
-	local name_cmd="$1"
-	local map_name="$2"
-	local bpftool_path="$3"
-	local key="$4"
-
-	# Test read access to the map.
-	if ! "$bpftool_path" map lookup "$name_cmd" "$map_name" key $key 1>/dev/null; then
-		echo " Read access to $key in $map_name failed"
-		exit 1
-	fi
-
-	# Test read access to map's BTF data.
-	if ! "$bpftool_path" btf dump map "$name_cmd" "$map_name" 1>/dev/null; then
-		echo " Read access to $map_name for BTF data failed"
-		exit 1
-	fi
-}
-
-# Test write access to the map.
-# Parameters:
-#   $1: Name command (name/pinned)
-#   $2: Map name
-#   $3: bpftool path
-#   $4: key
-#   $5: Whether write should succeed (true/false)
-access_for_write() {
-	local name_cmd="$1"
-	local map_name="$2"
-	local bpftool_path="$3"
-	local key="$4"
-	local write_should_succeed="$5"
-	local value="1 1 1 1"
-
-	if "$bpftool_path" map update "$name_cmd" "$map_name" key $key value \
-			$value 2>/dev/null; then
-		if [ "$write_should_succeed" = "false" ]; then
-			echo " Write access to $key in $map_name succeeded but should have failed"
-			exit 1
-		fi
-	else
-		if [ "$write_should_succeed" = "true" ]; then
-			echo " Write access to $key in $map_name failed but should have succeeded"
-			exit 1
-		fi
-	fi
-}
-
-# Test entry deletion for the map.
-# Parameters:
-#   $1: Name command (name/pinned)
-#   $2: Map name
-#   $3: bpftool path
-#   $4: key
-#   $5: Whether write should succeed (true/false)
-access_for_deletion() {
-	local name_cmd="$1"
-	local map_name="$2"
-	local bpftool_path="$3"
-	local key="$4"
-	local write_should_succeed="$5"
-	local value="1 1 1 1"
-
-	# Test deletion by key for the map.
-	# Before deleting, check the key exists.
-	if ! "$bpftool_path" map lookup "$name_cmd" "$map_name" key $key 1>/dev/null; then
-		echo " Key $key does not exist in $map_name"
-		exit 1
-	fi
-
-	# Delete by key.
-	if "$bpftool_path" map delete "$name_cmd" "$map_name" key $key 2>/dev/null; then
-		if [ "$write_should_succeed" = "false" ]; then
-			echo " Deletion for $key in $map_name succeeded but should have failed"
-			exit 1
-		fi
-	else
-		if [ "$write_should_succeed" = "true" ]; then
-			echo " Deletion for $key in $map_name failed but should have succeeded"
-			exit 1
-		fi
-	fi
-
-	# After deleting, check the entry existence according to the expected status.
-	if "$bpftool_path" map lookup "$name_cmd" "$map_name" key $key 1>/dev/null; then
-		if [ "$write_should_succeed" = "true" ]; then
-			echo " Key $key for $map_name was not deleted but should have been deleted"
-			exit 1
-		fi
-	else
-		if [ "$write_should_succeed" = "false" ]; then
-			echo "Key $key for $map_name was deleted but should have not been deleted"
-			exit 1
-		fi
-	fi
-
-	# Test creation of map's deleted entry, if deletion was successful.
-	# Otherwise, the entry exists.
-	if "$bpftool_path" map update "$name_cmd" "$map_name" key $key value \
-				$value 2>/dev/null; then
-		if [ "$write_should_succeed" = "false" ]; then
-			echo " Write access to $key in $map_name succeeded after deletion attempt but should have failed"
-			exit 1
-		fi
-	else
-		if [ "$write_should_succeed" = "true" ]; then
-			echo " Write access to $key in $map_name failed after deletion attempt but should have succeeded"
-			exit 1
-		fi
-	fi
-}
-
-# Test map elements iterator.
-# Parameters:
-#   $1: Name command (name/pinned)
-#   $2: Map name
-#   $3: bpftool path
-#   $4: BPF_DIR
-#   $5: bpf iterator object file path
-iterate_map_elem() {
-	local name_cmd="$1"
-	local map_name="$2"
-	local bpftool_path="$3"
-	local bpf_dir="$4"
-	local bpf_file="$5"
-	local pin_path="$bpf_dir/map_iterator"
-
-	"$bpftool_path" iter pin "$bpf_file" "$pin_path" map "$name_cmd" "$map_name"
-	if [ ! -f "$pin_path" ]; then
-		echo " Failed to pin iterator to $pin_path"
-		exit 1
-	fi
-
-	cat "$pin_path" 1>/dev/null
-	rm "$pin_path" 2>/dev/null
-}
-
-# Function to test map access with configurable write expectations
-# Parameters:
-#   $1: Name command (name/pinned)
-#   $2: Map name
-#   $3: bpftool path
-#   $4: key for rw
-#   $5: key to delete
-#   $6: Whether write should succeed (true/false)
-#   $7: BPF_DIR
-#   $8: bpf iterator object file path
-access_map() {
-	local name_cmd="$1"
-	local map_name="$2"
-	local bpftool_path="$3"
-	local key_for_rw="$4"
-	local key_to_del="$5"
-	local write_should_succeed="$6"
-	local bpf_dir="$7"
-	local bpf_iter_file_path="$8"
-
-	access_for_read "$name_cmd" "$map_name" "$bpftool_path" "$key_for_rw"
-	access_for_write "$name_cmd" "$map_name" "$bpftool_path" "$key_for_rw" \
-		"$write_should_succeed"
-	access_for_deletion "$name_cmd" "$map_name" "$bpftool_path" "$key_to_del" \
-		"$write_should_succeed"
-	iterate_map_elem "$name_cmd" "$map_name" "$bpftool_path" "$bpf_dir" \
-		"$bpf_iter_file_path"
-}
-
-# Function to test map access with configurable write expectations
-# Parameters:
-#   $1: Map name
-#   $2: bpftool path
-#   $3: BPF_DIR
-#   $4: Whether write should succeed (true/false)
-#   $5: bpf iterator object file path
-test_map_access() {
-	local map_name="$1"
-	local bpftool_path="$2"
-	local bpf_dir="$3"
-	local pin_path="$bpf_dir/${map_name}_pinned"
-	local write_should_succeed="$4"
-	local bpf_iter_file_path="$5"
-
-	# Test access to the map by name.
-	access_map "name" "$map_name" "$bpftool_path" "0 0 0 0" "1 0 0 0" \
-		"$write_should_succeed" "$bpf_dir" "$bpf_iter_file_path"
-
-	# Pin the map to the BPF filesystem
-	"$bpftool_path" map pin name "$map_name" "$pin_path"
-	if [ ! -e "$pin_path" ]; then
-		echo " Failed to pin $map_name"
-		exit 1
-	fi
-
-	# Test access to the pinned map.
-	access_map "pinned" "$pin_path" "$bpftool_path" "0 0 0 0" "2 0 0 0" \
-		"$write_should_succeed" "$bpf_dir" "$bpf_iter_file_path"
-}
-
-# Function to test map creation and map-of-maps
-# Parameters:
-#   $1: bpftool path
-#   $2: BPF_DIR
-test_map_creation_and_map_of_maps() {
-	local bpftool_path="$1"
-	local bpf_dir="$2"
-	local outer_map_name="outer_map_tt"
-	local inner_map_name="inner_map_tt"
-
-	"$bpftool_path" map create "$bpf_dir/$inner_map_name" type array key 4 \
-		value 4 entries 4 name "$inner_map_name"
-	if [ ! -f "$bpf_dir/$inner_map_name" ]; then
-		echo " Failed to create inner map file at $bpf_dir/$outer_map_name"
-		return 1
-	fi
-
-	"$bpftool_path" map create "$bpf_dir/$outer_map_name" type hash_of_maps \
-		key 4 value 4 entries 2 name "$outer_map_name" inner_map name "$inner_map_name"
-	if [ ! -f "$bpf_dir/$outer_map_name" ]; then
-		echo " Failed to create outer map file at $bpf_dir/$outer_map_name"
-		return 1
-	fi
-
-	# Add entries to the outer map by name and by pinned path.
-	"$bpftool_path" map update pinned "$bpf_dir/$outer_map_name" key 0 0 0 0 \
-		value pinned "$bpf_dir/$inner_map_name"
-	"$bpftool_path" map update name "$outer_map_name" key 1 0 0 0 value \
-		name "$inner_map_name"
-
-	# The outer map should be full by now.
-	# The following map update command is expected to fail.
-	if "$bpftool_path" map update name "$outer_map_name" key 2 0 0 0 value name \
-		"$inner_map_name" 2>/dev/null; then
-		echo " Update for $outer_map_name succeeded but should have failed"
-		exit 1
-	fi
-}
-
-# Function to test map access with the btf list command
-# Parameters:
-#   $1: bpftool path
-test_map_access_with_btf_list() {
-	local bpftool_path="$1"
-
-	# The btf list command iterates over maps for
-	# loaded BPF programs.
-	if ! "$bpftool_path" btf list 1>/dev/null; then
-		echo " Failed to access btf data"
-		exit 1
-	fi
-}
-
-set -eu
-
-trap cleanup_skip EXIT
-
-check_root_privileges
-
-verify_bpftool_path "$BPFTOOL_PATH"
-
-verify_btf_support
-
-trap cleanup EXIT
-
-# Load and attach the BPF programs to control maps access.
-"$BPFTOOL_PATH" prog loadall "$BPF_FILE_PATH" "$BPF_DIR" autoattach
-
-initialize_map_entries "$PROTECTED_MAP_NAME" "$BPFTOOL_PATH"
-initialize_map_entries "$NOT_PROTECTED_MAP_NAME" "$BPFTOOL_PATH"
-
-# Activate the map protection mechanism. Protection status is controlled
-# by a value stored in the prot_status_map at index 0.
-"$BPFTOOL_PATH" map update name prot_status_map key 0 0 0 0 value 1 0 0 0
-
-# Test protected map (write should fail).
-test_map_access "$PROTECTED_MAP_NAME" "$BPFTOOL_PATH" "$BPF_DIR" "false" \
- "$BPF_ITER_FILE_PATH"
-
-# Test not protected map (write should succeed).
-test_map_access "$NOT_PROTECTED_MAP_NAME" "$BPFTOOL_PATH" "$BPF_DIR" "true" \
- "$BPF_ITER_FILE_PATH"
-
-test_map_creation_and_map_of_maps "$BPFTOOL_PATH" "$BPF_DIR"
-
-test_map_access_with_btf_list "$BPFTOOL_PATH"
-
-exit 0
diff --git a/tools/testing/selftests/bpf/test_bpftool_metadata.sh b/tools/testing/selftests/bpf/test_bpftool_metadata.sh
deleted file mode 100755
index b5520692f41b..000000000000
--- a/tools/testing/selftests/bpf/test_bpftool_metadata.sh
+++ /dev/null
@@ -1,85 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: GPL-2.0
-
-# Kselftest framework requirement - SKIP code is 4.
-ksft_skip=4
-
-BPF_FILE_USED="metadata_used.bpf.o"
-BPF_FILE_UNUSED="metadata_unused.bpf.o"
-
-TESTNAME=bpftool_metadata
-BPF_FS=$(awk '$3 == "bpf" {print $2; exit}' /proc/mounts)
-BPF_DIR=$BPF_FS/test_$TESTNAME
-
-_cleanup()
-{
-	set +e
-	rm -rf $BPF_DIR 2> /dev/null
-}
-
-cleanup_skip()
-{
-	echo "selftests: $TESTNAME [SKIP]"
-	_cleanup
-
-	exit $ksft_skip
-}
-
-cleanup()
-{
-	if [ "$?" = 0 ]; then
-		echo "selftests: $TESTNAME [PASS]"
-	else
-		echo "selftests: $TESTNAME [FAILED]"
-	fi
-	_cleanup
-}
-
-if [ $(id -u) -ne 0 ]; then
-	echo "selftests: $TESTNAME [SKIP] Need root privileges"
-	exit $ksft_skip
-fi
-
-if [ -z "$BPF_FS" ]; then
-	echo "selftests: $TESTNAME [SKIP] Could not run test without bpffs mounted"
-	exit $ksft_skip
-fi
-
-if ! bpftool version > /dev/null 2>&1; then
-	echo "selftests: $TESTNAME [SKIP] Could not run test without bpftool"
-	exit $ksft_skip
-fi
-
-set -e
-
-trap cleanup_skip EXIT
-
-mkdir $BPF_DIR
-
-trap cleanup EXIT
-
-bpftool prog load $BPF_FILE_UNUSED $BPF_DIR/unused
-
-METADATA_PLAIN="$(bpftool prog)"
-echo "$METADATA_PLAIN" | grep 'a = "foo"' > /dev/null
-echo "$METADATA_PLAIN" | grep 'b = 1' > /dev/null
-
-bpftool prog --json | grep '"metadata":{"a":"foo","b":1}' > /dev/null
-
-bpftool map | grep 'metadata.rodata' > /dev/null
-
-rm $BPF_DIR/unused
-
-bpftool prog load $BPF_FILE_USED $BPF_DIR/used
-
-METADATA_PLAIN="$(bpftool prog)"
-echo "$METADATA_PLAIN" | grep 'a = "bar"' > /dev/null
-echo "$METADATA_PLAIN" | grep 'b = 2' > /dev/null
-
-bpftool prog --json | grep '"metadata":{"a":"bar","b":2}' > /dev/null
-
-bpftool map | grep 'metadata.rodata' > /dev/null
-
-rm $BPF_DIR/used
-
-exit 0

-- 
2.52.0

Re: [PATCH bpf-next 4/4] selftests/bpf: remove converted bpftool test scripts

[Quentin Monnet]

2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)"
<alexis.lothore@bootlin.com>
> Now that test_bpftool_map.sh and test_bpftool_metadata.sh are integrated
> into test_bpftool runner, remove those.
> 
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>

Acked-by: Quentin Monnet <qmo@kernel.org>

Thanks a lot!

Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

[Andrii Nakryiko]

On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
<alexis.lothore@bootlin.com> wrote:
>
> Hello,
> this series is part of the larger effort aiming to convert all
> standalone tests to the CI runners so that they are properly executed on
> patches submission.
>
> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
> and so they do not integrate well in test_progs. This series proposes to

Can you elaborate why they do not integrate well? In my mind,
test_progs should be the only runner into which we invest effort
(parallel tests, all the different filtering, etc; why would we have
to reimplement subsets of this). The fact that we have test_maps and
test_verifier is historical and if we had enough time we'd merge all
of them into test_progs.

What exactly in test_progs would prevent us from implementing bpftool
test runner?

> introduce a new runner to support those tests. This new runner is
> heavily inspired from test_progs, with slightly less features.
>
> - First commit move the assert macros exposed by test_progs into a
>   shared header so that they can be used by the new runner
> - Second commit introduces the new runner, as well as a first test
> - Third commit adds a second test
> - Fourth commit drops the legacy scripts corresponding to those new
>   tests
>
> The series only adds the runner, but it is not enough to have it running
> in CI. The github actions need to be updated both in [1] and [2], and I
> am not sure how it si supposed to be contributed (the part needed in [1]
> must likely be integrated in the commit that is systematically brought
> on top of the bpf-next_base branch). Still, an example of test_bpftool
> run can be found in [3], in which I have forked and updated the needed
> Github actions ([4])
>
> [1] https://github.com/kernel-patches/bpf.git
> [2] https://github.com/libbpf/ci
> [3] https://github.com/kernel-patches/bpf/pull/10711
> [4] https://github.com/Tropicao/libbpf-ci
>
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
> ---
> Alexis Lothoré (eBPF Foundation) (4):
>       bpf/selftests: move assert macros into a dedicated header
>       bpf/selftests: introduce bptool test runner and a first test
>       selftests/bpf: add bpftool map manipulations tests
>       selftests/bpf: remove converted bpftool test scripts
>
>  tools/testing/selftests/bpf/.gitignore             |   1 +
>  tools/testing/selftests/bpf/Makefile               |  16 +-
>  tools/testing/selftests/bpf/assert_helpers.h       | 231 ++++++++++++
>  tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++
>  tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +
>  .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
>  .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++
>  .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++
>  tools/testing/selftests/bpf/test_bpftool.c         | 126 +++++++
>  tools/testing/selftests/bpf/test_bpftool.h         |  36 ++
>  tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
>  .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
>  tools/testing/selftests/bpf/test_progs.h           | 226 +-----------
>  13 files changed, 1041 insertions(+), 711 deletions(-)
> ---
> base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
> change-id: 20251212-bpftool-tests-bb165c4cceb8
>
> Best regards,
> --
> Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
>

Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

[Alexis Lothoré]

Hi Andrii,

On Thu Jan 15, 2026 at 6:58 PM CET, Andrii Nakryiko wrote:
> On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
> <alexis.lothore@bootlin.com> wrote:
>>
>> Hello,
>> this series is part of the larger effort aiming to convert all
>> standalone tests to the CI runners so that they are properly executed on
>> patches submission.
>>
>> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
>> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
>> and so they do not integrate well in test_progs. This series proposes to
>
> Can you elaborate why they do not integrate well? In my mind,
> test_progs should be the only runner into which we invest effort
> (parallel tests, all the different filtering, etc; why would we have
> to reimplement subsets of this). The fact that we have test_maps and
> test_verifier is historical and if we had enough time we'd merge all
> of them into test_progs.
>
> What exactly in test_progs would prevent us from implementing bpftool
> test runner?

I don't think there is any strong technical blocker preventing from
integrating those tests directly into test_progs. That's rather about
the fact that test_progs tests depends (almost) exclusively on
libbpf/skeletons. Those bpftool tests rather need to directly execute
bpftool and parse its stdout output, so I thought that it made sense to
have a dedicated runner for this. If I'm wrong and so if those tests
should rather be moved in the test_progs runner (eg to avoid duplicating
the runner features), I'm fine with it. Any additional opinion on this
is welcome.

Thanks,

Alexis
-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

[Andrii Nakryiko]

On Thu, Jan 15, 2026 at 11:57 PM Alexis Lothoré
<alexis.lothore@bootlin.com> wrote:
>
> Hi Andrii,
>
> On Thu Jan 15, 2026 at 6:58 PM CET, Andrii Nakryiko wrote:
> > On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
> > <alexis.lothore@bootlin.com> wrote:
> >>
> >> Hello,
> >> this series is part of the larger effort aiming to convert all
> >> standalone tests to the CI runners so that they are properly executed on
> >> patches submission.
> >>
> >> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
> >> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
> >> and so they do not integrate well in test_progs. This series proposes to
> >
> > Can you elaborate why they do not integrate well? In my mind,
> > test_progs should be the only runner into which we invest effort
> > (parallel tests, all the different filtering, etc; why would we have
> > to reimplement subsets of this). The fact that we have test_maps and
> > test_verifier is historical and if we had enough time we'd merge all
> > of them into test_progs.
> >
> > What exactly in test_progs would prevent us from implementing bpftool
> > test runner?
>
> I don't think there is any strong technical blocker preventing from
> integrating those tests directly into test_progs. That's rather about
> the fact that test_progs tests depends (almost) exclusively on
> libbpf/skeletons. Those bpftool tests rather need to directly execute

There are actually plenty of test in test_progs that do networking
setups, calling system() to launch various binaries, etc. So it never
was purely for libbpf/skeletons/whatnot.

So yeah, I think bpftool testing should still be implemented as one
(or many) test_progs tests (and maybe subtests), utilizing
test_progs's generic multi-process testing setup, filtering,
reporting, etc infrastructure. No need to add extra runners.

> bpftool and parse its stdout output, so I thought that it made sense to
> have a dedicated runner for this. If I'm wrong and so if those tests
> should rather be moved in the test_progs runner (eg to avoid duplicating
> the runner features), I'm fine with it. Any additional opinion on this
> is welcome.
>
> Thanks,
>
> Alexis
> --
> Alexis Lothoré, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>