From: Jakub Sitnicki <jakub@cloudflare.com>
To: John Fastabend <john.fastabend@gmail.com>
Cc: lmb@cloudflare.com, daniel@iogearbox.net, netdev@vger.kernel.org,
bpf@vger.kernel.org, ast@kernel.org
Subject: Re: [bpf-next PATCH 00/10] bpf: selftests, test_sockmap improvements
Date: Thu, 7 May 2020 12:37:07 +0200 [thread overview]
Message-ID: <20200507123707.4b4a0fe1@toad> (raw)
In-Reply-To: <158871160668.7537.2576154513696580062.stgit@john-Precision-5820-Tower>
On Tue, 05 May 2020 13:49:36 -0700
John Fastabend <john.fastabend@gmail.com> wrote:
> Update test_sockmap to add ktls tests and in the process make output
> easier to understand and reduce overall runtime significantly. Before
> this series test_sockmap did a poor job of tracking sent bytes causing
> the recv thread to wait for a timeout even though all expected bytes
> had been received. Doing this many times causes significant delays.
> Further, we did many redundant tests because the send/recv test we used
> was not specific to the parameters we were testing. For example testing
> a failure case that always fails many times with different send sizes
> is mostly useless. If the test condition catches 10B in the kernel code
> testing 100B, 1kB, 4kB, and so on is just noise.
>
> The main motivation for this is to add ktls tests, the last patch. Until
> now I have been running these locally but we haven't had them checked in
> to selftests. And finally I'm hoping to get these pushed into the libbpf
> test infrastructure so we can get more testing. For that to work we need
> ability to white and blacklist tests based on kernel features so we add
> that here as well.
>
> The new output looks like this broken into test groups with subtest
> counters,
>
> $ time sudo ./test_sockmap
> # 1/ 6 sockmap:txmsg test passthrough:OK
> # 2/ 6 sockmap:txmsg test redirect:OK
> ...
> #22/ 1 sockhash:txmsg test push/pop data:OK
> Pass: 22 Fail: 0
>
> real 0m9.790s
> user 0m0.093s
> sys 0m7.318s
>
> The old output printed individual subtest and was rather noisy
>
> $ time sudo ./test_sockmap
> [TEST 0]: (1, 1, 1, sendmsg, pass,): PASS
> ...
> [TEST 823]: (16, 1, 100, sendpage, ... ,pop (1599,1609),): PASS
> Summary: 824 PASSED 0 FAILED
>
> real 0m56.761s
> user 0m0.455s
> sys 0m31.757s
>
> So we are able to reduce time from ~56s to ~10s. To recover older more
> verbose output simply run with --verbose option. To whitelist and
> blacklist tests use the new --whitelist and --blacklist flags added. For
> example to run cork sockhash tests but only ones that don't have a receive
> hang (used to test negative cases) we could do,
>
> $ ./test_sockmap --whitelist="cork" --blacklist="sockmap,hang"
>
> ---
These are very nice improvements so thanks for putting time into it.
I run these whenever I touch sockmap, and they do currently take long to
run, especially on 1 vCPU (which sometimes catches interesting bugs).
I've also ran before into the CLI quirks you've ironed out, like having
to pass path to cgroup to get verbose output.
Feel free to add my:
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
next prev parent reply other threads:[~2020-05-07 10:37 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-05 20:49 [bpf-next PATCH 00/10] bpf: selftests, test_sockmap improvements John Fastabend
2020-05-05 20:49 ` [bpf-next PATCH 01/10] bpf: selftests, move sockmap bpf prog header into progs John Fastabend
2020-05-05 20:50 ` [bpf-next PATCH 02/10] bpf: selftests, remove prints from sockmap tests John Fastabend
2020-05-05 20:50 ` [bpf-next PATCH 03/10] bpf: selftests, sockmap test prog run without setting cgroup John Fastabend
2020-05-07 8:31 ` Jakub Sitnicki
2020-05-07 18:10 ` John Fastabend
2020-05-05 20:50 ` [bpf-next PATCH 04/10] bpf: selftests, print error in test_sockmap error cases John Fastabend
2020-05-05 20:51 ` [bpf-next PATCH 05/10] bpf: selftests, improve test_sockmap total bytes counter John Fastabend
2020-05-07 8:55 ` Jakub Sitnicki
2020-05-05 20:51 ` [bpf-next PATCH 06/10] bpf: selftests, break down test_sockmap into subtests John Fastabend
2020-05-05 20:51 ` [bpf-next PATCH 07/10] bpf: selftests, provide verbose option for selftests execution John Fastabend
2020-05-05 20:52 ` [bpf-next PATCH 08/10] bpf: selftests, add whitelist option to test_sockmap John Fastabend
2020-05-05 20:52 ` [bpf-next PATCH 09/10] bpf: selftests, add blacklist " John Fastabend
2020-05-05 20:52 ` [bpf-next PATCH 10/10] bpf: selftests, add ktls tests " John Fastabend
2020-05-05 21:04 ` [bpf-next PATCH 00/10] bpf: selftests, test_sockmap improvements John Fastabend
2020-05-07 10:37 ` Jakub Sitnicki [this message]
2020-05-07 18:12 ` John Fastabend
2020-05-08 21:34 ` Andrii Nakryiko
2020-05-09 14:55 ` John Fastabend
2020-05-12 3:13 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200507123707.4b4a0fe1@toad \
--to=jakub@cloudflare.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=john.fastabend@gmail.com \
--cc=lmb@cloudflare.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).