[RFC net-next v2 0/5] selftests: net: add performance metric reporting

[Stanislav Fomichev <sdf.kernel@gmail.com>]

Add timestamped observations and producer-defined metric policy to Python
selftests. Add iperf and kperf performance coverage.

System monitoring can live in either place:

Selftests:
- uses one KTAP format for benchmark and system metrics
- provides system metrics for direct local runs
- lets each test select the exact scope and devices
- adds kernel-side code and maintenance

NIPA:
- keeps generic monitoring out of the kernel tree
- allows collection to evolve without kernel changes
- requires runner-managed test boundaries and local tooling

This version keeps CPU and NIC monitoring in NIPA. NIPA runs each program
in drivers/net/hw/performance one subtest per process, giving the runner
exact monitoring boundaries.

Note that we still keep metric aggregation policy (ksft_metric_policy)
including regression boundaries in the tree. If we were to move it
to NIPA we'd have to have per-metric-name configs or something.

NIPA changes at https://github.com/fomichev/nipa/tree/nipa-local-hw-metrics
have a few examples. git clone & ./nipa local view will start an
http server on :44109 with two sample runs.

v2:
- move system monitoring to NIPA (Jakub)
- move performance tests to drivers/net/hw/performance (Jakub)
- rename metric shape to type and document supported types (Paolo);
  kind= instead of type= (type shadows a builtin)
- remove direction from the metric example (Paolo)
- document policy processing order (Paolo)
- support producer-selected summary operations, including arbitrary pXX
- label throughput metrics in Gbps
- do not mark the iperf benchmark disruptive
- use 2 GiB udmabuf buffers for kperf devmem
- drop the ncdevmem devmem probe
- use a random kperf control port
- report kperf server cleanup failures

Stanislav Fomichev (5):
  selftests: net: py: add timestamped metric output
  selftests: net: py: add metric policy output
  selftests: drv-net: add an iperf performance test
  selftests: drv-net: add a kperf runner
  selftests: drv-net: measure devmem performance with kperf

 tools/testing/selftests/Makefile              |   2 +-
 .../testing/selftests/drivers/net/README.rst  |  57 +++++++
 .../drivers/net/hw/lib/py/__init__.py         |  17 +-
 .../drivers/net/hw/performance/Makefile       |  13 ++
 .../drivers/net/hw/performance/config         |   4 +
 .../drivers/net/hw/performance/devmem.py      |  99 +++++++++++
 .../drivers/net/hw/performance/iperf.py       |  27 +++
 .../drivers/net/hw/performance/settings       |   1 +
 .../selftests/drivers/net/lib/py/__init__.py  |  15 +-
 .../selftests/drivers/net/lib/py/kperf.py     | 161 ++++++++++++++++++
 .../selftests/drivers/net/lib/py/load.py      |  24 ++-
 .../testing/selftests/net/lib/py/__init__.py  |  11 +-
 tools/testing/selftests/net/lib/py/ksft.py    | 159 ++++++++++++++++-
 13 files changed, 571 insertions(+), 19 deletions(-)
 create mode 100644 tools/testing/selftests/drivers/net/hw/performance/Makefile
 create mode 100644 tools/testing/selftests/drivers/net/hw/performance/config
 create mode 100755 tools/testing/selftests/drivers/net/hw/performance/devmem.py
 create mode 100755 tools/testing/selftests/drivers/net/hw/performance/iperf.py
 create mode 100644 tools/testing/selftests/drivers/net/hw/performance/settings
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/kperf.py

-- 
2.53.0-Meta