[PATCH net-next v4 0/4] selftests: net: add performance metric reporting

[Stanislav Fomichev <sdf.kernel@gmail.com>]

Let Python selftests report numeric measurements, so that CI can track
the performance of drivers over time, and add two performance tests.

ksft_metric() reports a measurement of the current test case, which is
printed right away as JSON:

  # ktap-metric-json: {"goal":"higher","name":"throughput","value":389.127}

The goal tells a performance metric, which is better higher or lower,
from an error metric, which must be zero. Everything else, such as
units, history and regression thresholds, is up to the consumer.

The tests are in a new drivers/net/perf target, apart from the
functional tests: they need real hardware, take a while, and make no
sense on debug kernels. iperf.py measures single-stream TCP throughput
with iperf3, devmem.py the throughput of udmabuf-backed devmem TCP with
kperf [1].

System monitoring stays in NIPA, which runs each case of the perf tests
in a process of its own and attaches CPU and NIC metrics to its results.
The NIPA changes are at [2]; git clone & ./nipa local view starts an
http server on :44109 with two sample runs.

[1] https://github.com/facebookexperimental/kperf
[2] https://github.com/fomichev/nipa/tree/nipa-local-hw-metrics

v4:
- print metrics right away (Jakub)
- mark the exit code of the perf lib/py on import failures as KSFT_SKIP
  (Björn)
- let kperf-server --kill wait for the sessions of the server, drop the
  pgrep loop (Jakub):
  https://github.com/facebookexperimental/kperf/pull/47

v3:
- drop the timestamp of metrics (Jakub)
- drop the note on nested output from the docs (Jakub)
- report plain numbers, drop kind/type (Jakub)
- replace ksft_metric_policy() with a goal per metric: higher, lower
  or zero (Jakub)
- move the tests to drivers/net/perf, with a lib/py of their own (Jakub)
- fix the format of the Makefile lists (Jakub)
- report metrics from the tests rather than the runners (Jakub)
- drop the kperf section of the README (Jakub)
- stop reporting TCP retransmits
- drop the metric labels, report the throughput of the receiver only

v2:
- move system monitoring to NIPA (Jakub)
- move performance tests to drivers/net/hw/performance (Jakub)
- rename metric shape to type and document supported types (Paolo);
  kind= instead of type= (type shadows a builtin)
- remove direction from the metric example (Paolo)
- document policy processing order (Paolo)
- support producer-selected summary operations, including arbitrary pXX
- label throughput metrics in Gbps
- do not mark the iperf benchmark disruptive
- use 2 GiB udmabuf buffers for kperf devmem
- drop the ncdevmem devmem probe
- use a random kperf control port
- report kperf server cleanup failures

Stanislav Fomichev (4):
  selftests: net: py: add metric output
  selftests: drv-net: add an iperf performance test
  selftests: drv-net: add a kperf runner
  selftests: drv-net: measure devmem performance with kperf

 tools/testing/selftests/Makefile              |   2 +-
 .../testing/selftests/drivers/net/README.rst  |  16 ++
 .../selftests/drivers/net/lib/py/__init__.py  |   4 +-
 .../selftests/drivers/net/lib/py/kperf.py     | 157 ++++++++++++++++++
 .../selftests/drivers/net/lib/py/load.py      |  20 ++-
 .../selftests/drivers/net/perf/Makefile       |  12 ++
 .../testing/selftests/drivers/net/perf/config |   4 +
 .../selftests/drivers/net/perf/devmem.py      |  96 +++++++++++
 .../selftests/drivers/net/perf/iperf.py       |  23 +++
 .../drivers/net/perf/lib/py/__init__.py       |  59 +++++++
 .../selftests/drivers/net/perf/settings       |   1 +
 .../testing/selftests/net/lib/py/__init__.py  |   9 +-
 tools/testing/selftests/net/lib/py/ksft.py    |  20 +++
 13 files changed, 414 insertions(+), 9 deletions(-)
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/kperf.py
 create mode 100644 tools/testing/selftests/drivers/net/perf/Makefile
 create mode 100644 tools/testing/selftests/drivers/net/perf/config
 create mode 100755 tools/testing/selftests/drivers/net/perf/devmem.py
 create mode 100755 tools/testing/selftests/drivers/net/perf/iperf.py
 create mode 100644 tools/testing/selftests/drivers/net/perf/lib/py/__init__.py
 create mode 100644 tools/testing/selftests/drivers/net/perf/settings

-- 
2.53.0-Meta