Re: [RFC bpf-next 3/5] selftests/bpf: generate boilerplate code for test_loader-based tests

[Eduard Zingerman <eddyz87@gmail.com>]

On Wed, 2023-01-25 at 17:43 -0800, Andrii Nakryiko wrote:
> On Mon, Jan 23, 2023 at 6:52 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > Automatically generate boilerplate code necessary to run tests that
> > use test_loader.c.
> > 
> > Adds a target 'prog_tests/test_loader_auto_wrappers.c' as part of
> > rulesets for 'test_progs' and 'test_progs-no_alu32'. The content of
> > this C file is generated by make and has the following structure:
> > 
> >   #include <test_progs.h>
> > 
> >   #include "some_test_1.skel.h"
> >   #include "some_test_2.skel.h"
> >   ...
> > 
> >   void test_some_test_1(void) { RUN_TESTS(some_test_1); }
> >   void test_some_test_2(void) { RUN_TESTS(some_test_2); }
> >   ...
> > 
> > Here RUN_TESTS is a macro defined in test_progs.h, it expands to a
> > code that uses test_loader.c:test_loader__run_subtests() function to
> > load tests specified by appropriate skel.h.
> > 
> > In order to get the list of tests included in
> > 'test_loader_auto_wrappers.c' the generation script looks for
> > 'progs/*.c' files that contain a special comment:
> > 
> >   /* Use test_loader marker */
> > 
> > Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
> > ---
> 
> It feels like this is a bit of an overkill, tbh. There are
> 
> $ ls verifier/*.c | wc -l
> 94
> 
> files. We can move each migrated set of tests from verifier/xxx.c to
> progs/verifier_xxx.c. And then just have just manually maintained
> prog_tests/verifier.c file where for each converted test we have one
> #include and one void test_some_test_1(void) { RUN_TESTS(some_test_1);
> }.
> 
> It sometimes would useful to add some extra debugging printfs in such
> a file, so having it auto generated would be actually an
> inconvenience. And that on top of further Makefile complication.
> 
> For initial conversion we can auto-generate this file, of course. And
> then for each migrated file adding 2 lines manually doesn't seem like
> a big deal?

Ok, I'll remove the makefile changes.

> 
> 
> 
> >  tools/testing/selftests/bpf/Makefile          | 34 +++++++++++++++++++
> >  .../selftests/bpf/prog_tests/.gitignore       |  1 +
> >  2 files changed, 35 insertions(+)
> > 
> 
> [...]