Re: [PATCH RFC net-next v4 2/2] selftests/dpll: add DPLL system integration selftests

[Jakub Kicinski <kuba@kernel.org>]

On Thu, 23 Nov 2023 05:52:43 -0500 Michal Michalik wrote:
> The tests are written in Python3 (3.7+) and pytest testing framework.
> Framework is basing on the ynl library available in the kernel tree
> at: tools/net/ynl

LGTM!

Somewhat tangential question, a nit, and a comment..
 
> The DPLL system integration tests are meant to be part of selftests, so
> they can be build and run using command:
>   make -C tools/testing/selftests
> 
> Alternatively, they can be run using single command [1]:
>   make kselftest
> 
> If we want to run only DPLL tests, we should set the TARGETS variable:
>   make -C tools/testing/selftests TARGETS=drivers/net/netdevsim/dpll
> 
> They can also be run standalone using starter script:
>   ./run_dpll_tests.sh
> 
> There is a possibliy to set optional PYTEST_PARAMS environment variable
> to set the pytest options, like tests filtering ("-k <filter>") or
> verbose output ("-v").
> 
> [1] https://www.kernel.org/doc/html/v5.0/dev-tools/kselftest.html

nit: s/v5.0/v6.6/ ? Or /v5.0/latest/

Did you try to run it in vmtest or virtme-ng?
https://www.youtube.com/watch?v=NT-325hgXjY
https://lpc.events/event/17/contributions/1506/attachments/1143/2441/virtme-ng.pdf

I'm thinking of using those for continuous testing, curious all 
the Python setup works okay with them.

> +@pytest.fixture(scope="class", params=((0,), (1, 0), (0, 1)))

We have both uses of pytest and unittest in the kernel:

$ git grep --files-with-matches '^import .*unittest'
scripts/rust_is_available_test.py
tools/crypto/ccp/test_dbc.py
tools/perf/pmu-events/metric_test.py
tools/testing/kunit/kunit_tool_test.py
tools/testing/selftests/bpf/test_bpftool.py
tools/testing/selftests/tpm2/tpm2.py
tools/testing/selftests/tpm2/tpm2_tests.py

$ git grep --files-with-matches '^import .*pytest'
scripts/kconfig/tests/conftest.py
tools/testing/selftests/drivers/sdsi/sdsi.sh
tools/testing/selftests/drivers/sdsi/sdsi_test.py
tools/testing/selftests/hid/tests/base.py
tools/testing/selftests/hid/tests/conftest.py
tools/testing/selftests/hid/tests/test_gamepad.py
tools/testing/selftests/hid/tests/test_mouse.py
tools/testing/selftests/hid/tests/test_multitouch.py
tools/testing/selftests/hid/tests/test_sony.py
tools/testing/selftests/hid/tests/test_tablet.py
tools/testing/selftests/hid/tests/test_usb_crash.py
tools/testing/selftests/hid/tests/test_wacom_generic.py

unittest seems a bit more popular but pytest does seem like
a better fit indeed.

Did you see what the sdsi test does? It seems to assume everything 
is installed locally, without the venv. I wonder if that may be simpler
to get going with vmtest?