[Question] json parsing and bpf selftests dependencies

[Question] json parsing and bpf selftests dependencies

[Alexis Lothoré]

Hello,
I am pursuing my quest to convert standalone bpf tests (from
tools/testing/selftests/bpf) into the test_progs framework so they can
be executed automatically by the CI tooling.

I would like to continue on the bpftool tests, especially
test_bpftool.py ([1]). This one involves quite a lot of json parsing on
bpftool output (to validate that some entries are present in the
output, depending on the used command), which does not seem to be a use
case currently in bpf selftests. The first tests may be handled by some
manual parsing (eg strstr'ing keys in the output, that's what I've done
for the recently converted bpftool_metadata test, see [2]), but that's a
bit fragile, and there are more complex subtests for which this loosy
strategy will get even more fragile (eg
test_feature_kernel_full_vs_not_full generates json output with two
different commands, and tests the diff).

I then face the need to have some proper json parsing in test_progs. I
kind of understand that there is a will to keep the dependency list
small for test_progs, so I'm asking here what could be the best option
for this:
- is it ok to make test_progs depend on a new json parsing library (eg:
  cJSON) ? and so add the library to CI images ?
- or some implicit dependency on some CLI tooling (eg: jq) ? and so, add
  the cli tool to CI images ?
- test_progs is reusing json_writer.c from bpftool ([2]), should we
  rather write a custom  json_reader.c as well (even if not needed at
  that point by bpftool itself) ?

Any opinion on this ?

Alexis

[1] https://elixir.bootlin.com/linux/v6.19.3/source/tools/testing/selftests/bpf/test_bpftool.py
[2] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c#n49
[3] https://elixir.bootlin.com/linux/v6.19.3/source/tools/bpf/bpftool/json_writer.c

-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [Question] json parsing and bpf selftests dependencies

[Mykyta Yatsenko]

Alexis Lothoré <alexis.lothore@bootlin.com> writes:

> Hello,
> I am pursuing my quest to convert standalone bpf tests (from
> tools/testing/selftests/bpf) into the test_progs framework so they can
> be executed automatically by the CI tooling.
>
> I would like to continue on the bpftool tests, especially
> test_bpftool.py ([1]). This one involves quite a lot of json parsing on
> bpftool output (to validate that some entries are present in the
> output, depending on the used command), which does not seem to be a use
> case currently in bpf selftests. The first tests may be handled by some
> manual parsing (eg strstr'ing keys in the output, that's what I've done
> for the recently converted bpftool_metadata test, see [2]), but that's a
> bit fragile, and there are more complex subtests for which this loosy
> strategy will get even more fragile (eg
> test_feature_kernel_full_vs_not_full generates json output with two
> different commands, and tests the diff).
>
> I then face the need to have some proper json parsing in test_progs. I
> kind of understand that there is a will to keep the dependency list
> small for test_progs, so I'm asking here what could be the best option
> for this:
> - is it ok to make test_progs depend on a new json parsing library (eg:
>   cJSON) ? and so add the library to CI images ?
> - or some implicit dependency on some CLI tooling (eg: jq) ? and so, add
>   the cli tool to CI images ?
> - test_progs is reusing json_writer.c from bpftool ([2]), should we
>   rather write a custom  json_reader.c as well (even if not needed at
>   that point by bpftool itself) ?
>
> Any opinion on this ?
Can we please make that test_progs can be built without json deps and
run tests that do not require them, it's useful because we build tests
in a restricted container image, where adding a new library may be
undesirable (I'm not talking about github CI).
>
> Alexis
>
> [1] https://elixir.bootlin.com/linux/v6.19.3/source/tools/testing/selftests/bpf/test_bpftool.py
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c#n49
> [3] https://elixir.bootlin.com/linux/v6.19.3/source/tools/bpf/bpftool/json_writer.c
>
> -- 
> Alexis Lothoré, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

Re: [Question] json parsing and bpf selftests dependencies

[Alexei Starovoitov]

On Thu, Feb 26, 2026 at 2:36 AM Alexis Lothoré
<alexis.lothore@bootlin.com> wrote:
>
> Hello,
> I am pursuing my quest to convert standalone bpf tests (from
> tools/testing/selftests/bpf) into the test_progs framework so they can
> be executed automatically by the CI tooling.
>
> I would like to continue on the bpftool tests, especially
> test_bpftool.py ([1]).

I think it outlived itself long ago.
Just delete it.
Any new bpftool features can be tested through test_progs.
No need to test json output specifically.
It's diminishing returns.

Re: [Question] json parsing and bpf selftests dependencies

[Alexis Lothoré]

On Thu Feb 26, 2026 at 6:21 PM CET, Alexei Starovoitov wrote:
> On Thu, Feb 26, 2026 at 2:36 AM Alexis Lothoré
> <alexis.lothore@bootlin.com> wrote:
>>
>> Hello,
>> I am pursuing my quest to convert standalone bpf tests (from
>> tools/testing/selftests/bpf) into the test_progs framework so they can
>> be executed automatically by the CI tooling.
>>
>> I would like to continue on the bpftool tests, especially
>> test_bpftool.py ([1]).
>
> I think it outlived itself long ago.
> Just delete it.
> Any new bpftool features can be tested through test_progs.
> No need to test json output specifically.
> It's diminishing returns.

Ok. That's a doubt I had while converting the first round of tests
(bpf_metadata, bpftool_map_access), whether we really want to test exact
bpftool CLI output, or if test_progs->libbpf testing was fine.

I'll remote this test_bpftool.py then, thanks for the feedback.

Alexis

-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com