From: Vineet Gupta <vineet.gupta@linux.dev>
To: bpf@vger.kernel.org
Cc: Vineet Gupta <vineet.gupta@linux.dev>
Subject: [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK
Date: Fri, 21 Aug 2026 13:13:15 -0700 [thread overview]
Message-ID: <20260821201316.1059129-3-vineet.gupta@linux.dev> (raw)
In-Reply-To: <20260821201316.1059129-1-vineet.gupta@linux.dev>
Several verifier test files wrap their contents in a feature gate and provide
a one-line placeholder program in the #else arm, so the file still yields
something when the toolchain or JIT cannot support the tested instructions.
However the placeholder is annotated __success, so it reports as a pass,
whereas this is really a SKIP.
| #318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:OK
| #318 verifier_ldsx:OK
This causes skipped coverage to be overlooked: see the next patch where
BPF_GCC builds miss a whole bunch of test due to clang only gate.
Add a __skip("reason") annotation for a compile-time gate, and use it for the
placeholders.
After this change, above tests reported as
| #318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:SKIP
| #318 verifier_ldsx:SKIP
Signed-off-by: Vineet Gupta <vineet.gupta@linux.dev>
---
tools/testing/selftests/bpf/progs/arena_kfunc.c | 1 +
tools/testing/selftests/bpf/progs/bpf_misc.h | 6 ++++++
tools/testing/selftests/bpf/progs/stack_arg_fail.c | 1 +
tools/testing/selftests/bpf/progs/stack_arg_precision.c | 1 +
tools/testing/selftests/bpf/progs/verifier_bswap.c | 1 +
tools/testing/selftests/bpf/progs/verifier_gotol.c | 1 +
tools/testing/selftests/bpf/progs/verifier_ldsx.c | 1 +
.../testing/selftests/bpf/progs/verifier_load_acquire.c | 1 +
tools/testing/selftests/bpf/progs/verifier_movsx.c | 1 +
.../testing/selftests/bpf/progs/verifier_private_stack.c | 1 +
tools/testing/selftests/bpf/progs/verifier_sdiv.c | 1 +
tools/testing/selftests/bpf/progs/verifier_stack_arg.c | 1 +
.../selftests/bpf/progs/verifier_stack_arg_order.c | 1 +
.../testing/selftests/bpf/progs/verifier_store_release.c | 1 +
tools/testing/selftests/bpf/test_loader.c | 9 +++++++++
15 files changed, 28 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/arena_kfunc.c b/tools/testing/selftests/bpf/progs/arena_kfunc.c
index 47bed5c4c488..50609f3b0564 100644
--- a/tools/testing/selftests/bpf/progs/arena_kfunc.c
+++ b/tools/testing/selftests/bpf/progs/arena_kfunc.c
@@ -233,6 +233,7 @@ SEC("syscall")
__arch_x86_64
__arch_arm64
__description("arena_arg_stack: not supported, dummy test")
+__skip("arena_arg_stack: not supported")
__success
int dummy_test(void *ctx)
{
diff --git a/tools/testing/selftests/bpf/progs/bpf_misc.h b/tools/testing/selftests/bpf/progs/bpf_misc.h
index 5eacf1b43252..88b0bfba83bb 100644
--- a/tools/testing/selftests/bpf/progs/bpf_misc.h
+++ b/tools/testing/selftests/bpf/progs/bpf_misc.h
@@ -106,6 +106,11 @@
* __description Text to be used for display and as an additional filter
* alias, while the original program name stays matchable.
*
+ * __skip Report the test as SKIP with the given reason instead of
+ * running it. For placeholder programs that stand in for a
+ * feature the toolchain or JIT cannot provide, so a run does
+ * not report OK for coverage it never executed.
+ *
* __log_level Log level to use for the program, numeric value expected.
*
* __flag Adds one flag use for the program, the following values are valid:
@@ -139,6 +144,7 @@
#define __failure __test_tag("test_expect_failure")
#define __success __test_tag("test_expect_success")
#define __description(desc) __test_tag("test_description=" desc)
+#define __skip(reason) __test_tag("test_skip=" reason)
#define __msg_unpriv(msg) __test_tag("test_expect_msg_unpriv=" msg)
#define __not_msg_unpriv(msg) __test_tag("test_expect_not_msg_unpriv=" msg)
#define __xlated_unpriv(msg) __test_tag("test_expect_xlated_unpriv=" msg)
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_fail.c b/tools/testing/selftests/bpf/progs/stack_arg_fail.c
index 6f6f59306d0e..eed97d582515 100644
--- a/tools/testing/selftests/bpf/progs/stack_arg_fail.c
+++ b/tools/testing/selftests/bpf/progs/stack_arg_fail.c
@@ -103,6 +103,7 @@ __naked void r11_store_zero_off(void)
SEC("tc")
__description("stack_arg_fail: not supported, dummy test")
+__skip("stack_arg_fail: not supported")
__success
int dummy_test(struct __sk_buff *skb)
{
diff --git a/tools/testing/selftests/bpf/progs/stack_arg_precision.c b/tools/testing/selftests/bpf/progs/stack_arg_precision.c
index bee2eeec021d..ce0301a41fa9 100644
--- a/tools/testing/selftests/bpf/progs/stack_arg_precision.c
+++ b/tools/testing/selftests/bpf/progs/stack_arg_precision.c
@@ -124,6 +124,7 @@ __naked void stack_arg_precision_bpf2bpf(void)
SEC("socket")
__description("stack_arg_precision: not supported, dummy test")
+__skip("stack_arg_precision: not supported")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_bswap.c b/tools/testing/selftests/bpf/progs/verifier_bswap.c
index cffaf36192bc..074faef5480e 100644
--- a/tools/testing/selftests/bpf/progs/verifier_bswap.c
+++ b/tools/testing/selftests/bpf/progs/verifier_bswap.c
@@ -117,6 +117,7 @@ l0_%=: \
SEC("socket")
__description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_gotol.c b/tools/testing/selftests/bpf/progs/verifier_gotol.c
index d5d8f24df394..c0fd30a3a4c1 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotol.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotol.c
@@ -52,6 +52,7 @@ __naked void gotol_large_imm(void)
SEC("socket")
__description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_ldsx.c b/tools/testing/selftests/bpf/progs/verifier_ldsx.c
index 41340877dc9d..e58ea69e3854 100644
--- a/tools/testing/selftests/bpf/progs/verifier_ldsx.c
+++ b/tools/testing/selftests/bpf/progs/verifier_ldsx.c
@@ -436,6 +436,7 @@ void kfunc_root(void)
SEC("socket")
__description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_load_acquire.c b/tools/testing/selftests/bpf/progs/verifier_load_acquire.c
index d17026d7480d..bcc34e798c63 100644
--- a/tools/testing/selftests/bpf/progs/verifier_load_acquire.c
+++ b/tools/testing/selftests/bpf/progs/verifier_load_acquire.c
@@ -267,6 +267,7 @@ __naked void load_acquire_with_invalid_reg(void)
SEC("socket")
__description("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support load-acquire, use a dummy test")
+__skip("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support load-acquire")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_movsx.c b/tools/testing/selftests/bpf/progs/verifier_movsx.c
index a4d8814eb5ed..5559c0583dd1 100644
--- a/tools/testing/selftests/bpf/progs/verifier_movsx.c
+++ b/tools/testing/selftests/bpf/progs/verifier_movsx.c
@@ -343,6 +343,7 @@ label_%=: \
SEC("socket")
__description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_private_stack.c b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
index ea0a7e73331d..3d9994a52464 100644
--- a/tools/testing/selftests/bpf/progs/verifier_private_stack.c
+++ b/tools/testing/selftests/bpf/progs/verifier_private_stack.c
@@ -377,6 +377,7 @@ int private_stack_max_depth(void)
SEC("kprobe")
__description("private stack is not supported, use a dummy test")
+__skip("private stack is not supported")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_sdiv.c b/tools/testing/selftests/bpf/progs/verifier_sdiv.c
index 95f3239ce228..515644ace516 100644
--- a/tools/testing/selftests/bpf/progs/verifier_sdiv.c
+++ b/tools/testing/selftests/bpf/progs/verifier_sdiv.c
@@ -1271,6 +1271,7 @@ __naked void smod32_int_min_mod_neg2_imm(void)
SEC("socket")
__description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
index 7e0ce5db28a0..51d22faf4559 100644
--- a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
+++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
@@ -436,6 +436,7 @@ __naked void stack_arg_sequential_calls(void)
SEC("socket")
__description("stack_arg is not supported by compiler or jit, use a dummy test")
+__skip("stack_arg is not supported by compiler or jit")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c
index c9fe4857da3f..8e4325273b3d 100644
--- a/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c
+++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c
@@ -174,6 +174,7 @@ __naked void stack_arg_read_without_write_2(void)
SEC("socket")
__description("stack_arg order is not supported by compiler or jit, use a dummy test")
+__skip("stack_arg order is not supported by compiler or jit")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_store_release.c b/tools/testing/selftests/bpf/progs/verifier_store_release.c
index 72f1eb006074..0abbee5ce109 100644
--- a/tools/testing/selftests/bpf/progs/verifier_store_release.c
+++ b/tools/testing/selftests/bpf/progs/verifier_store_release.c
@@ -290,6 +290,7 @@ __naked void store_release_with_invalid_reg(void)
SEC("socket")
__description("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support store-release, use a dummy test")
+__skip("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support store-release")
__success
int dummy_test(void)
{
diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c
index 07807757b518..794a7dfb0579 100644
--- a/tools/testing/selftests/bpf/test_loader.c
+++ b/tools/testing/selftests/bpf/test_loader.c
@@ -70,6 +70,7 @@ struct test_spec {
int arch_mask;
int load_mask;
int linear_sz;
+ const char *skip_reason;
bool auxiliary;
bool valid;
};
@@ -456,6 +457,8 @@ static int parse_test_spec(struct test_loader *tester,
continue;
if ((val = str_has_pfx(s, "test_description="))) {
description = val;
+ } else if ((val = str_has_pfx(s, "test_skip="))) {
+ spec->skip_reason = val;
} else if (strcmp(s, "test_expect_failure") == 0) {
spec->priv.expect_failure = true;
spec->mode_mask |= PRIV;
@@ -1327,6 +1330,12 @@ void run_subtest(struct test_loader *tester,
if (!test__start_subtest_with_desc(subspec->name, subspec->description))
return;
+ if (spec->skip_reason) {
+ printf("%s:SKIP: %s\n", __func__, spec->skip_reason);
+ test__skip();
+ return;
+ }
+
if ((get_current_arch() & spec->arch_mask) == 0) {
test__skip();
return;
--
2.53.0-Meta
next prev parent reply other threads:[~2026-08-21 20:13 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-21 20:13 [PATCH bpf-next 0/3] selftests/bpf: unblock bpf-gcc test coverage Vineet Gupta
2026-08-21 20:13 ` [PATCH bpf-next 1/3] selftests/bpf: name the remaining placeholder programs dummy_test [NFC] Vineet Gupta
2026-08-21 20:13 ` Vineet Gupta [this message]
2026-08-21 21:00 ` [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK bot+bpf-ci
2026-08-21 23:48 ` Vineet Gupta
2026-08-21 20:13 ` [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc Vineet Gupta
2026-08-21 20:19 ` sashiko-bot
2026-08-21 23:38 ` Vineet Gupta
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260821201316.1059129-3-vineet.gupta@linux.dev \
--to=vineet.gupta@linux.dev \
--cc=bpf@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.