Re: [bpf-next PATCH 00/10] bpf: selftests, test_sockmap improvements

[John Fastabend <john.fastabend@gmail.com>]

Jakub Sitnicki wrote:
> On Tue, 05 May 2020 13:49:36 -0700
> John Fastabend <john.fastabend@gmail.com> wrote:
> 
> > Update test_sockmap to add ktls tests and in the process make output
> > easier to understand and reduce overall runtime significantly. Before
> > this series test_sockmap did a poor job of tracking sent bytes causing
> > the recv thread to wait for a timeout even though all expected bytes
> > had been received. Doing this many times causes significant delays.
> > Further, we did many redundant tests because the send/recv test we used
> > was not specific to the parameters we were testing. For example testing
> > a failure case that always fails many times with different send sizes
> > is mostly useless. If the test condition catches 10B in the kernel code
> > testing 100B, 1kB, 4kB, and so on is just noise.
> > 
> > The main motivation for this is to add ktls tests, the last patch. Until
> > now I have been running these locally but we haven't had them checked in
> > to selftests. And finally I'm hoping to get these pushed into the libbpf
> > test infrastructure so we can get more testing. For that to work we need
> > ability to white and blacklist tests based on kernel features so we add
> > that here as well.
> > 
> > The new output looks like this broken into test groups with subtest
> > counters,
> > 
> >  $ time sudo ./test_sockmap
> >  # 1/ 6  sockmap:txmsg test passthrough:OK
> >  # 2/ 6  sockmap:txmsg test redirect:OK
> >  ...
> >  #22/ 1 sockhash:txmsg test push/pop data:OK
> >  Pass: 22 Fail: 0
> > 
> >  real    0m9.790s
> >  user    0m0.093s
> >  sys     0m7.318s
> > 
> > The old output printed individual subtest and was rather noisy
> > 
> >  $ time sudo ./test_sockmap
> >  [TEST 0]: (1, 1, 1, sendmsg, pass,): PASS
> >  ...
> >  [TEST 823]: (16, 1, 100, sendpage, ... ,pop (1599,1609),): PASS
> >  Summary: 824 PASSED 0 FAILED 
> > 
> >  real    0m56.761s
> >  user    0m0.455s
> >  sys     0m31.757s
> > 
> > So we are able to reduce time from ~56s to ~10s. To recover older more
> > verbose output simply run with --verbose option. To whitelist and
> > blacklist tests use the new --whitelist and --blacklist flags added. For
> > example to run cork sockhash tests but only ones that don't have a receive
> > hang (used to test negative cases) we could do,
> > 
> >  $ ./test_sockmap --whitelist="cork" --blacklist="sockmap,hang"
> > 
> > ---
> 
> These are very nice improvements so thanks for putting time into it.
> 
> I run these whenever I touch sockmap, and they do currently take long to
> run, especially on 1 vCPU (which sometimes catches interesting bugs).
> 
> I've also ran before into the CLI quirks you've ironed out, like having
> to pass path to cgroup to get verbose output.
> 
> Feel free to add my:
> 
> Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>

Thanks will do. I'll push a v2 with the other comments addressed after bpf gets
synced into bpf-next.