Re: [PATCH net-next v3 0/7] selftests: net: Switch pmtu.sh to use the internal ovs script.

[Aaron Conole <aconole@redhat.com>]

Jakub Kicinski <kuba@kernel.org> writes:

> On Tue, 25 Jun 2024 13:22:38 -0400 Aaron Conole wrote:
>> Currently, if a user wants to run pmtu.sh and cover all the provided test
>> cases, they need to install the Open vSwitch userspace utilities.  This
>> dependency is difficult for users as well as CI environments, because the
>> userspace build and setup may require lots of support and devel packages
>> to be installed, system setup to be correct, and things like permissions
>> and selinux policies to be properly configured.
>
> Hi Aaron!
>
> I merged this yesterday (with slight alphabetical reshuffling of
> the config options). The pmtu.sh test is solid now, which is great!

:)  Thanks!  That's great to see.

> I also added the OvS tests themselves, and those are not passing, yet:
> https://netdev.bots.linux.dev/contest.html?test=openvswitch-sh
> Could you take a look and LMK if these are likely env issues or
> something bad in the test itself?

I saw that.  I was looking for a place in the nipa repository where I
could submit a small fix, because I noticed in the stdout:

  make -C tools/testing/selftests TARGETS="net/openvswitch"
  TEST_PROGS=openvvswitch.sh TEST_GEN_PROGS="" run_tests
  
and I think the TEST_PROGS=openvvswitch.sh is misspelled (but it seems
to not matter too much for the run_test target).

From what I understand, there are two things causing it to be flaky.
First, the module detection is a bit flaky (and that's why it results is
some 'skip' reports).  Additionally, the connection oriented tests
include negative cases and those hit timeouts.  The default is to
declare failure after 45s.  That can be seen in:

  https://netdev-3.bots.linux.dev/vmksft-net/results/659601/91-openvswitch-sh/stdout
  ...
  # timeout set to 45
  ...
  # TEST: nat_connect_v4                                                [START]
  # Terminated
  # Terminated

This is showing that the timeout is too short.

I have patches ready for these issues, but I didn't know if you would
like me to submit config and settings files to go under net/openvswitch,
or if you would prefer to see the openvswitch.sh script, and
ovs-dpctl.py utilities move out of their net/openvswitch/ directory.  If
the latter, I can submit patches quickly with config and settings (and a
small change to the script itself) that addresses these.  If you'd
prefer the former (moving around the files), I'll need to spend some
additional time modifying pmtu and doing a larger test.  I don't have a
strong opinion on either approach.