All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH mptcp-net v3 00/32] selftests: mptcp: skip tests when features are not supported
@ 2023-05-31 14:16 Matthieu Baerts
  2023-05-31 14:16 ` [PATCH mptcp-net v3 01/32] selftests: mptcp: lib: skip if missing symbol Matthieu Baerts
                   ` (32 more replies)
  0 siblings, 33 replies; 35+ messages in thread
From: Matthieu Baerts @ 2023-05-31 14:16 UTC (permalink / raw)
  To: mptcp; +Cc: Paolo Abeni, Matthieu Baerts

Selftests are supposed to run on any kernels, including the old ones not
supporting all MPTCP features.

These different patches allow the different MPTCP selftests to be
executed on older kernels.

I tried to minimise the modifications and do the checks from the bash
script not to have to detect the feature when being used in a .c file,
then pass the info to the bash script, skip the test, clean the
environment, etc. As you can see, even with this minimalistic way, a lot
of modifications have been done to support old kernels.

Note that if we expect to have all features available,
SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var can be set to 1 to mark
the tests as failed instead of skipped and track regressions just in
case one of these checks is wrong. The public CI already set this env
var to 1.

This is a big series impacting new development. Probably best to apply
it "quickly" in our tree and fix issues later to avoid multiple rebases.

Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/368
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
---
Changes in v3:
- Patches 1-7 have been already applied
- check kernel version: new env var that can be set to skip the versions
  check so it works if a new feature is backported to an older kernel.
- get_counter() to get the MIB counters: I can help the stable team with
  the backports that will not be difficult I think but will cause
  conflicts
- check nf/tc issues: looks safer just in case a kconfig is missing or
  an issue with nf/tc but can be removed if you prefer (still I don't
  think we should exit in the middle of a test so I kept it for the
  moment)
- one check per block of tests around the same theme: it would indeed
  avoid re-doing the same check 2 or 3 times but it would break the IDs
  (+maybe less obvious a test might be skipped when reviewing code)
- backports: at the end, we just need all these new checks to be
  backported on the last stable kernel (6.3) because that's what is used
  by the different CIs. It is fine if it is not backported after. (but
  still I can help with the backports, I'm used to :) )
- New patches:
  - selftests: mptcp: join: uniform listener tests
  - Squash to "selftests: mptcp: add MPTCP_FULL_INFO testcase"
- See the changelog on the individual patches
- Link to v2: https://lore.kernel.org/r/20230406-mptcp-issue-368-selftests-old-kernels-v2-0-50313e4f83ab@tessares.net

Changes in v2:
- The first patches are almost the same as in v1, the main modifications
  are in mptcp_lib.sh (and in the commit messages): new helpers but also
  the use of SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var.
- Link to v1: https://lore.kernel.org/r/20230406-mptcp-issue-368-selftests-old-kernels-v1-0-246ac567724d@tessares.net

---
Matthieu Baerts (32):
      selftests: mptcp: lib: skip if missing symbol
      selftests: mptcp: lib: skip if not below kernel version
      selftests: mptcp: connect: skip transp tests if not supported
      selftests: mptcp: connect: skip disconnect tests if not supported
      selftests: mptcp: connect: skip TFO tests if not supported
      selftests: mptcp: diag: skip listen tests if not supported
      selftests: mptcp: diag: skip inuse tests if not supported
      selftests: mptcp: pm nl: remove hardcoded default limits
      selftests: mptcp: pm nl: skip fullmesh flag checks if not supported
      selftests: mptcp: sockopt: relax expected returned size
      selftests: mptcp: sockopt: skip getsockopt checks if not supported
      selftests: mptcp: sockopt: skip TCP_INQ checks if not supported
      selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable
      selftests: mptcp: userspace pm: skip if not supported
      selftests: mptcp: userspace pm: skip PM listener events tests if unavailable
      selftests: mptcp: join: use 'iptables-legacy' if available
      selftests: mptcp: join: helpers to skip tests
      selftests: mptcp: join: skip check if MIB counter not supported
      selftests: mptcp: join: skip test if iptables/tc cmds fail
      selftests: mptcp: join: support local endpoint being tracked or not
      selftests: mptcp: join: skip Fastclose tests if not supported
      selftests: mptcp: join: support RM_ADDR for used endpoints or not
      selftests: mptcp: join: skip implicit tests if not supported
      selftests: mptcp: join: skip backup if set flag on ID not supported
      selftests: mptcp: join: skip fullmesh flag tests if not supported
      selftests: mptcp: join: skip userspace PM tests if not supported
      selftests: mptcp: join: skip fail tests if not supported
      selftests: mptcp: join: skip MPC backups tests if not supported
      selftests: mptcp: join: skip PM listener tests if not supported
      selftests: mptcp: join: uniform listener tests
      selftests: mptcp: join: skip mixed tests if not supported
      Squash to "selftests: mptcp: add MPTCP_FULL_INFO testcase"

 tools/testing/selftests/net/mptcp/config           |   1 +
 tools/testing/selftests/net/mptcp/diag.sh          |  42 +-
 tools/testing/selftests/net/mptcp/mptcp_connect.sh |  20 +
 tools/testing/selftests/net/mptcp/mptcp_join.sh    | 536 +++++++++++++--------
 tools/testing/selftests/net/mptcp/mptcp_lib.sh     |  64 +++
 tools/testing/selftests/net/mptcp/mptcp_sockopt.c  |  26 +-
 tools/testing/selftests/net/mptcp/mptcp_sockopt.sh |  20 +-
 tools/testing/selftests/net/mptcp/pm_netlink.sh    |  27 +-
 tools/testing/selftests/net/mptcp/userspace_pm.sh  |  13 +-
 9 files changed, 506 insertions(+), 243 deletions(-)
---
base-commit: 3d6d7b1262115b5ccc87b47a61f0a2db3d1b2fbf
change-id: 20230406-mptcp-issue-368-selftests-old-kernels-fffd69d67d91

Best regards,
-- 
Matthieu Baerts <matthieu.baerts@tessares.net>


^ permalink raw reply	[flat|nested] 35+ messages in thread

end of thread, other threads:[~2023-06-07  9:47 UTC | newest]

Thread overview: 35+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-05-31 14:16 [PATCH mptcp-net v3 00/32] selftests: mptcp: skip tests when features are not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 01/32] selftests: mptcp: lib: skip if missing symbol Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 02/32] selftests: mptcp: lib: skip if not below kernel version Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 03/32] selftests: mptcp: connect: skip transp tests if not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 04/32] selftests: mptcp: connect: skip disconnect " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 05/32] selftests: mptcp: connect: skip TFO " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 06/32] selftests: mptcp: diag: skip listen " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 07/32] selftests: mptcp: diag: skip inuse " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 08/32] selftests: mptcp: pm nl: remove hardcoded default limits Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 09/32] selftests: mptcp: pm nl: skip fullmesh flag checks if not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 10/32] selftests: mptcp: sockopt: relax expected returned size Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 11/32] selftests: mptcp: sockopt: skip getsockopt checks if not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 12/32] selftests: mptcp: sockopt: skip TCP_INQ " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 13/32] selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 14/32] selftests: mptcp: userspace pm: skip if not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 15/32] selftests: mptcp: userspace pm: skip PM listener events tests if unavailable Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 16/32] selftests: mptcp: join: use 'iptables-legacy' if available Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 17/32] selftests: mptcp: join: helpers to skip tests Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 18/32] selftests: mptcp: join: skip check if MIB counter not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 19/32] selftests: mptcp: join: skip test if iptables/tc cmds fail Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 20/32] selftests: mptcp: join: support local endpoint being tracked or not Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 21/32] selftests: mptcp: join: skip Fastclose tests if not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 22/32] selftests: mptcp: join: support RM_ADDR for used endpoints or not Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 23/32] selftests: mptcp: join: skip implicit tests if not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 24/32] selftests: mptcp: join: skip backup if set flag on ID " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 25/32] selftests: mptcp: join: skip fullmesh flag tests if " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 26/32] selftests: mptcp: join: skip userspace PM " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 27/32] selftests: mptcp: join: skip fail " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 28/32] selftests: mptcp: join: skip MPC backups " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 29/32] selftests: mptcp: join: skip PM listener " Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 30/32] selftests: mptcp: join: uniform listener tests Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 31/32] selftests: mptcp: join: skip mixed tests if not supported Matthieu Baerts
2023-05-31 14:16 ` [PATCH mptcp-net v3 32/32] Squash to "selftests: mptcp: add MPTCP_FULL_INFO testcase" Matthieu Baerts
2023-05-31 15:44   ` Squash to "selftests: mptcp: add MPTCP_FULL_INFO testcase": Tests Results MPTCP CI
2023-06-07  9:47 ` [PATCH mptcp-net v3 00/32] selftests: mptcp: skip tests when features are not supported Matthieu Baerts

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.