From: "Ricardo B. Marlière" <rbm@suse.com>
To: <bot+bpf-ci@kernel.org>, <rbm@suse.com>, <ast@kernel.org>,
<daniel@iogearbox.net>, <andrii@kernel.org>,
<martin.lau@linux.dev>, <eddyz87@gmail.com>, <memxor@gmail.com>,
<song@kernel.org>, <yonghong.song@linux.dev>, <jolsa@kernel.org>,
<shuah@kernel.org>, <nathan@kernel.org>,
<nick.desaulniers+lkml@gmail.com>, <morbo@google.com>,
<justinstitt@google.com>
Cc: <bpf@vger.kernel.org>, <linux-kselftest@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <llvm@lists.linux.dev>,
<alan.maguire@oracle.com>, <martin.lau@kernel.org>,
<clm@meta.com>, <ihor.solodrai@linux.dev>
Subject: Re: [PATCH bpf-next v10 07/11] selftests/bpf: Skip tests whose objects were not built
Date: Thu, 30 Apr 2026 12:52:28 -0300 [thread overview]
Message-ID: <DI6LZ06Q21O2.38ZFZEES9W13N@suse.com> (raw)
In-Reply-To: <deceeca5bb5e141c8ad07c5ed3754c6b2613fa0b99383614f1cfcc31bd09d39c@mail.kernel.org>
On Thu Apr 30, 2026 at 10:30 AM -03, bot+bpf-ci wrote:
>> diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
>> index cc14b13e23fe..7ba82974ee78 100644
>> --- a/tools/testing/selftests/bpf/test_progs.c
>> +++ b/tools/testing/selftests/bpf/test_progs.c
>
> [ ... ]
>
>> @@ -2046,15 +2069,20 @@ int main(int argc, char **argv)
>> struct prog_test_def *test = &prog_test_defs[i];
>>
>> test->test_num = i + 1;
>> - test->should_run = should_run(&env.test_selector,
>> - test->test_num, test->test_name);
>> + test->selected = should_run(&env.test_selector,
>> + test->test_num, test->test_name);
>> + test->should_run = test->selected;
>>
>> - if ((test->run_test == NULL && test->run_serial_test == NULL) ||
>> - (test->run_test != NULL && test->run_serial_test != NULL)) {
>> + if (test->run_test && test->run_serial_test) {
>> fprintf(stderr, "Test %d:%s must have either test_%s() or serial_test_%sl() defined.\n",
>> test->test_num, test->test_name, test->test_name, test->test_name);
>> exit(EXIT_ERR_SETUP_INFRA);
>> }
>> + if (!test->run_test && !test->run_serial_test) {
>> + test->not_built = true;
>> + test->should_run = false;
>> + continue;
>> + }
>> if (test->should_run)
>> test->should_tmon = should_tmon(&env.tmon_selector, test->test_name);
>> }
>> @@ -2106,9 +2134,18 @@ int main(int argc, char **argv)
>>
>> for (i = 0; i < prog_test_cnt; i++) {
>> struct prog_test_def *test = &prog_test_defs[i];
>> + struct test_state *state = &test_states[i];
>>
>> - if (!test->should_run)
>> + if (!test->should_run) {
>> + if (test->not_built && test->selected &&
>> + !env.get_test_cnt && !env.list_test_names) {
>> + state->tested = true;
>> + state->skip_cnt = 1;
>> + env.not_built_cnt++;
>> + print_test_result(test, state);
>> + }
>> continue;
>> + }
>
> When test->should_run is false (which it is for unbuilt tests due to the
> code above setting test->should_run = false), the continue statement skips
> the env.get_test_cnt and env.list_test_names blocks that follow.
>
> Won't this completely omit unbuilt tests from the output of --list (-l) and
> --count modes? The code only prints unbuilt tests when !env.get_test_cnt &&
> !env.list_test_names is true.
It makes no sense to count and list tests that can't be run.
>
> If CI systems use test_progs -l to discover and distribute tests across
> parallel jobs, they won't be aware of unbuilt tests. Could this undermine
> the stated goal of keeping unbuilt tests visible in CI results?
Only if the CI in question sets BPF_STRICT_BUILD=0, which is not usually
the case and if opted-in then the user will know that some tests might not
appear and be counted.
>
> Reference: sashiko-bot raised this concern in v10 discussion at
> https://lore.kernel.org/bpf/20260430131344.B40DDC2BCB3@smtp.kernel.org/
>
>>
>> if (env.get_test_cnt) {
>> env.succ_cnt++;
>
> [ ... ]
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25167006036
next prev parent reply other threads:[~2026-04-30 15:52 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-30 12:53 [PATCH bpf-next v10 00/11] selftests/bpf: Tolerate partial builds across kernel configs Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 01/11] selftests/bpf: Add BPF_STRICT_BUILD toggle Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 02/11] selftests/bpf: Fix test_kmods KDIR to honor O= and distro kernels Ricardo B. Marlière
2026-04-30 13:30 ` bot+bpf-ci
2026-04-30 14:09 ` Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 03/11] selftests/bpf: Tolerate BPF and skeleton generation failures Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 04/11] selftests/bpf: Avoid rebuilds when running emit_tests Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 05/11] selftests/bpf: Make skeleton headers order-only prerequisites of .test.d Ricardo B. Marlière
2026-04-30 13:30 ` bot+bpf-ci
2026-04-30 15:04 ` Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 06/11] selftests/bpf: Tolerate test file compilation failures Ricardo B. Marlière
2026-04-30 13:30 ` bot+bpf-ci
2026-04-30 12:53 ` [PATCH bpf-next v10 07/11] selftests/bpf: Skip tests whose objects were not built Ricardo B. Marlière
2026-04-30 13:30 ` bot+bpf-ci
2026-04-30 15:52 ` Ricardo B. Marlière [this message]
2026-04-30 12:53 ` [PATCH bpf-next v10 08/11] selftests/bpf: Allow test_progs to link with a partial object set Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 09/11] selftests/bpf: Tolerate benchmark build failures Ricardo B. Marlière
2026-04-30 12:53 ` [PATCH bpf-next v10 10/11] selftests/bpf: Provide weak definitions for cross-test functions Ricardo B. Marlière
2026-04-30 13:30 ` bot+bpf-ci
2026-04-30 12:53 ` [PATCH bpf-next v10 11/11] selftests/bpf: Tolerate missing files during install Ricardo B. Marlière
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DI6LZ06Q21O2.38ZFZEES9W13N@suse.com \
--to=rbm@suse.com \
--cc=alan.maguire@oracle.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bot+bpf-ci@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=clm@meta.com \
--cc=daniel@iogearbox.net \
--cc=eddyz87@gmail.com \
--cc=ihor.solodrai@linux.dev \
--cc=jolsa@kernel.org \
--cc=justinstitt@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=martin.lau@kernel.org \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
--cc=morbo@google.com \
--cc=nathan@kernel.org \
--cc=nick.desaulniers+lkml@gmail.com \
--cc=shuah@kernel.org \
--cc=song@kernel.org \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox