Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

[Willem de Bruijn <willemdebruijn.kernel@gmail.com>]

Willem de Bruijn wrote:
> Jakub Kicinski wrote:
> > On Fri, 30 Aug 2024 11:20:05 -0400 Willem de Bruijn wrote:
> > > Kselftest install does not preserve directories.
> > > 
> > > So all .pkt files are copied into net/packetdrill root. This is messy.
> > > More fundamentally it breaks the includes in the files (e..g, `source
> > > ../common/defaults.sh`).
> > 
> > Can you show an example of exact commands and what happens?
> 
> Running directly works fine:
> 
>     $ KSELFTEST_PKT_INTERP=packetdrill_ksft.sh
>     $ make -C tools/testing/selftests \
>             TARGETS=net/packetdrill O=/tmp run_tests
> 
>     TAP version 13
>     1..3
>     # timeout set to 45
>     # selftests: net/packetdrill: client.pkt
>     # TAP version 13
>     # 1..2
>     # ok 1 ipv4
>     # ok 2 ipv6
>     # # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
>     ok 1 selftests: net/packetdrill: client.pkt
>     [..etc..]
> 
> Installing does not:
> 
>     $ make -C tools/testing/selftests/ \
>           TARGETS=net/packetdrill \
>           install INSTALL_PATH=$INSTALL_DIR
>     $ cd $INSTALL_DIR
>     $ export KSELFTEST_PKT_INTERP=packetdrill_ksft.sh
>     $ ./run_kselftest.sh -c net/packetdrill
> 
>     TAP version 13
>     1..3
>     # timeout set to 45
>     # selftests: net/packetdrill: client.pkt
>     # TAP version 13
>     # 1..2
>     # sh: line 1: ../common/defaults.sh: No such file or directory
>     # ./client.pkt: error executing init command: non-zero status 127
>     # not ok 1 ipv4
>     # sh: line 1: ../common/defaults.sh: No such file or directory
>     # ./client.pkt: error executing init command: non-zero status 127
>     # not ok 2 ipv6
>     # # Totals: pass:0 fail:2 xfail:0 xpass:0 skip:0 error:0
>     not ok 1 selftests: net/packetdrill: client.pkt # exit=1
> 
> Due to that relative path to defaults.sh inside the scripts.
> 
> It is arguably a bit weird that the relative path of the TEST_PROGS
> differs before and after install.

I guess what's weird is more what I'm trying to do. There has been
no need so far for progs in subdirectories from the TARGET.

One option is to just create a bunch of targets with a wildcard.
Something like below (entirely untested).

TARGETS += net/packetdrill/tcp/$(wildcard *)

> > We have directories in net/lib, and it's a target, and it works, no?
> 
> net/lib is not a TARGET in tools/testing/selftests/Makefile. Its
> Makefile only generates dependencies for other targets: TEST_FILES,
> TEST_GEN_FILES and TEST_INCLUDES.
> 
> This issue with preserving paths until recently also existed for
> helper files (TEST_FILES). TEST_INCLUDES was added expressly to
> preserve those paths (commit 2a0683be5b4c).

Another option is to dive yet deeper into kselftest infra and do the
same for TEST_PROGS, either TEST_PROGS itself or as a new
TEST_PROGS_FULLPATH or so.

It needs a change to INSTALL_RULE (maybe just passing -R to rsync),
to emit_tests to skip basename when writing kselftest-list.txt, and
probably to run_one() to break up the path and chdir. I'll see if I
can make this work without too much churn.