Re: [PATCH net-next 7/7] testing: net-drv: add a driver test for stats reporting

[Petr Machata <petrm@nvidia.com>]

Jakub Kicinski <kuba@kernel.org> writes:

> On Wed, 3 Apr 2024 00:04:14 +0200 Petr Machata wrote:
>> > Yes, I was wondering about that. It must be doable, IIRC 
>> > the multi-threading API "injects" args from a tuple.
>> > I was thinking something along the lines of:
>> >
>> >     with NetDrvEnv(__file__) as cfg:
>> >         ksft_run([check_pause, check_fec, pkt_byte_sum],
>> >                  args=(cfg, ))
>> >
>> > I got lazy, let me take a closer look. Another benefit
>> > will be that once we pass in "env" / cfg - we can "register" 
>> > objects in there for auto-cleanup (in the future, current
>> > tests don't need cleanup)  
>> 
>> Yeah, though some of those should probably just be their own context
>> managers IMHO, not necessarily hooked to cfg. I'm thinking something
>> fairly general, so that the support boilerplate doesn't end up costing
>> an arm and leg:
>> 
>>     with build("ip route add 192.0.2.1/28 nexthop via 192.0.2.17",
>>                "ip route del 192.0.2.1/28"),
>>          build("ip link set dev %s master %s" % (swp1, h1),
>>                "ip link set dev %s nomaster" % swp1):
>>         le_test()
>>
>> Dunno. I guess it makes sense to have some of the common stuff
>> predefined, e.g. "with vrf() as h1". And then the stuff that's typically
>> in lib.sh's setup() and cleanup(), can be losslessly hooked up to cfg.
>
> I was thinking of something along the lines of:
>
> def test_abc(cfg):
>     cfg.build("ip route add 192.0.2.1/28 nexthop via 192.0.2.17",
>               "ip route del 192.0.2.1/28")
>     cfg.build("ip link set dev %s master %s" % (swp1, h1),
>               "ip link set dev %s nomaster" % swp1)
>
> optionally we could then also:
>
>      thing = cfg.build("ip link set dev %s master %s" % (swp1, h1),
>                        "ip link set dev %s nomaster" % swp1)
>
>      # ... some code which may raise ...
>
>      # unlink to do something else with the device
>      del thing
>      # ... more code ... 
>
> cfg may not be best here, could be cleaner to create a "test" object,
> always pass it in as the first param, and destroy it after each test.

I assume above you mean that cfg inherits the thing, but cfg lifetime
currently looks like it spreads across several test cases. ksft_run()
would need to know about it and call something to issue the postponed
cleanups between cases.

Also, it's not clear what "del thing" should do in that context, because
if cfg also keeps a reference, __del__ won't get called. There could be
a direct method, like thing.exit() or whatever, but then you need
bookkeeping so as not to clean up the second time through cfg. It's the
less straightforward way of going about it IMHO.

I know that I must sound like a broken record at this point, but look:

    with build("ip link set dev %s master %s" % (swp1, h1),
               "ip link set dev %s nomaster" % swp1) as thing:
        ... some code which may rise ...
    ... more code, interface detached, `thing' gone ...

It's just as concise, makes it very clear where the device is part of
the bridge and where not anymore, and does away with the intricacies of
lifetime management.

If lifetimes don't nest, I think it's just going to be ugly either way.
But I don't think this comes up often.

I don't really see stuff that you could just throw at cfg to keep track
of, apart from the suite configuration (i.e. topology set up). But I
suppose if it comes up, we can do something like:

    thing = cfg.retain(build(..., ...))

Or maybe have a dedicated retainer object, or whatever, it doesn't
necessarily need to be cfg itself.

>> This is what I ended up gravitating towards after writing a handful of
>> LNST tests anyway. The scoping makes it clear where the object exists,
>> lifetime is taken care of, it's all ponies rainbows basically. At least
>> as long as your object lifetimes can be cleanly nested, which admittedly
>> is not always.
>
> Should be fairly easy to support all cases - "with", "recording on
> cfg/test" and del.  Unfortunately in the two tests I came up with

Yup.

> quickly for this series cleanup is only needed for the env itself.
> It's a bit awkward to add the lifetime helpers without any users.

Yeah. I'm basically delving in this now to kinda try and steer future
expectations.