RE: [PATCH bpf-next 1/2] selftests/bpf: add generic BPF program verification tester

[John Fastabend <john.fastabend@gmail.com>]

Andrii Nakryiko wrote:
> It's become a common pattern to have a collection of small BPF programs
> in one BPF object file, each representing one test case. On user-space
> side of such tests we maintain a table of program names and expected
> failure or success, along with optional expected verifier log message.
> 
> This works, but each set of tests reimplement this mundane code over and
> over again, which is a waste of time for anyone trying to add a new set
> of tests. Furthermore, it's quite error prone as it's way too easy to miss
> some entries in these manually maintained test tables (as evidences by
> dynptr_fail tests, in which ringbuf_release_uninit_dynptr subtest was
> accidentally missed; this is fixed in next patch).
> 
> So this patch implements generic verification_tester, which accepts
> skeleton name and handles the rest of details: opens and loads BPF
> object file, making sure each program is tested in isolation. Optionally
> each test case can specify expected BPF verifier log message. In case of
> failure, tester makes sure to report verifier log, but it also reports
> verifier log in verbose mode unconditionally.
> 
> Now, the interesting deviation from existing custom implementations is
> the use of btf_decl_tag attribute to specify expected-to-fail vs
> expected-to-succeed markers and, optionally, expected log message
> directly next to BPF program source code, eliminating the need to
> manually create and update table of tests.
> 
> We define few macros wrapping btf_decl_tag with a convention that all
> values of btf_decl_tag start with "comment:" prefix, and then utilizing
> a very simple "just_some_text_tag" or "some_key_name=<value>" pattern to
> define things like expected success/failure, expected verifier message,
> extra verifier log level (if necessary). This approach is demonstrated
> by next patch in which two existing sets of failure tests are converted.
> 
> Tester supports both expected-to-fail and expected-to-succeed programs,
> though this patch set didn't convert any existing expected-to-succeed
> programs yet, as existing tests couple BPF program loading with their
> further execution through attach or test_prog_run. One way to allow
> testing scenarios like this would be ability to specify custom callback,
> executed for each successfully loaded BPF program. This is left for
> follow up patches, after some more analysis of existing test cases.
> 
> This verification_tester is, hopefully, a start of a test_verifier
> runner, which allows much better "user experience" of defining low-level
> verification types that can take advantage of all the libbpf-provided
> nicety features on BPF side: global variables, declarative maps, etc.
> All while having a choice of defining it in C or as BPF assembly
> (through __attribute__((naked)) functions and using embedded asm). This
> will be explored in follow up patches as well.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---

LGTM.

Acked-by: John Fastabend <john.fastabend@gmail.com>