Re: [PATCH net-next v2 2/2] selftests/net: integrate packetdrill with ksft

[Jakub Kicinski <kuba@kernel.org>]

On Fri, 06 Sep 2024 19:28:08 -0400 Willem de Bruijn wrote:
> > > No, we opted for this design exactly to use existing kselftest infra,
> > > rather than reimplementing that in our wrapper, as I did in the RFC.  
> > 
> > OK, I understood from the discussions from the RFC that by using the
> > kselftest infra, the tests would be automatically executed in dedicated
> > netns, and it could also help running tests in parallel. That sounded
> > great to me, but that's not the case by default from what I see.  
> 
> Perhaps that's something to change in the defaults for run_tests.
> 
> Since the infra exist, that is preferable over reimplementing it for
> one particular subset of tests.
> 
> Or if not all kselftests can run in netns (quite likely), this needs
> to be opt-in. Then a variable defined in the Makefile perhaps. To
> tell kselftest to enable the feature for this target.

Indeed, I was thinking along the same lines.

We're closing net-next in a week, it'd be great to have the baseline
ksft interpreter mechanism in place in the next couple of days. 
The exact implementation of packetdrill/ksft_runner.sh can be changed
later as needed, and the current one works fine for NIPA.


Hopefully we can also discuss at LPC/netconf what to do about libraries
(where setup / cleanup code could live). Looking at MPTCP tests - do
they work out of tree? I see mptcp_lib.sh does:

. "$(dirname "${0}")/../lib.sh"
. "$(dirname "${0}")/../net_helper.sh"

but lib/sh and net_helper.sh are not listed in the Makefile. So they
won't get packaged...

We should make sure we support running the tests with make run_tests
and in installed mode. 

If we agree that the current situation with support for library code is
far from ideal, I think we have three(ish) directions to explore:

 1  build netns handling into runner.sh
   + already mostly there
   + simpler tests, no need to worry about netns, it just happens
   - not all tests need netns (HW-adjacent tests especially)
   - netns setup is the main thing we need but not the only thing,
     wait helpers, python code, etc. also need to be handled

 2a improve library bundling at the ksft level
   + we already have a net/lib "meta-target", it kinda works
   + hopefully in a way that lets us Python
   - no idea how

 2b put all the code in kselftest/, like ktap_helpers.sh ?
   + easy to do
   + helps other subsystems
   - could cause git conflicts
   - won't help Python?

 3  give up on target proliferation; on a quick count we have 15 targets
    in ksft for various bits of networking, faaar more than anyone else
   + fewer targets limits the need for libraries, libraries local to
     the target are trivial to handle
   - ksft has no other form of "grouping" tests, if we collapse into 
     a small number of targets it will be hard to run a group of tests