Re: [PATCH net-next 00/15] net/mlx5e: PSP cleanups and improvements

[Cosmin Ratiu <cratiu@nvidia.com>]

On Wed, 2026-07-08 at 12:37 +0000, Cosmin Ratiu wrote:
> On Tue, 2026-07-07 at 14:29 -0400, Daniel Zahka wrote:
> > 
> > On 7/7/26 9:08 AM, Tariq Toukan wrote:
> > > Hi,
> > > 
> > > This series by Cosmin refactors mlx5 PSP support in preparation
> > > for
> > > HW-GRO support.
> > > There are almost no functionality changes in all but the last two
> > > patches, which address a long-standing TODO in
> > > mlx5e_psp_set_config().
> > > 
> > > Regards,
> > > Tariq
> > > 
> > > Cosmin Ratiu (15):
> > >    net/mlx5e: psp: Rename the saved psp_dev to 'psd'
> > >    net/mlx5e: psp: Remove PSP steering mutexes
> > >    net/mlx5e: psp: Remove unneeded ref counting for PSP steering
> > >    net/mlx5e: psp: Merge rx_err rule add/delete with ft
> > > create/delete
> > >    net/mlx5e: psp: Use helpers for steering object manipulation
> > >    net/mlx5e: psp: Factor out drop rule creation code
> > >    net/mlx5e: psp: Remove unused PSP syndrome copy action
> > >    net/mlx5e: psp: Rename and consolidate steering functions
> > >    net/mlx5e: psp: Adjust rx_check FT size and use a drop_group
> > >    net/mlx5e: psp: Add an RX steering table
> > >    net/mlx5e: psp: Use a single rx_check table
> > >    net/mlx5e: psp: Flatten steering structures
> > >    net/mlx5e: psp: Make PSP steering config dynamic
> > >    net/mlx5e: Return errors from profile->enable
> > >    net/mlx5e: psp: Report PSP dev registration errors
> > > 
> > >   drivers/net/ethernet/mellanox/mlx5/core/en.h  |    2 +-
> > >   .../net/ethernet/mellanox/mlx5/core/en/fs.h   |    7 +-
> > >   .../mellanox/mlx5/core/en_accel/en_accel.h    |   19 +-
> > >   .../mellanox/mlx5/core/en_accel/psp.c         | 1007 ++++++++--
> > > --
> > > -----
> > >   .../mellanox/mlx5/core/en_accel/psp.h         |   18 +-
> > >   .../mellanox/mlx5/core/en_accel/psp_rxtx.c    |   13 +-
> > >   .../mellanox/mlx5/core/en_accel/psp_rxtx.h    |    3 +-
> > >   .../net/ethernet/mellanox/mlx5/core/en_main.c |   23 +-
> > >   .../net/ethernet/mellanox/mlx5/core/en_rep.c  |    8 +-
> > >   9 files changed, 516 insertions(+), 584 deletions(-)
> > > 
> > > 
> > > base-commit: 31816fc5d9acf8cdf226cdd0dc296e8cf15cc033
> > 
> > Thanks. Excited about the support for mlx5e_psp_set_config(). Jakub
> > and 
> > I had a test case for psp_dev_ops::set_config() that we were
> > waiting
> > to 
> > upstream. I just rebased it onto net-next here: 
> > https://github.com/danieldzahka/linux/commit/b58e9a99573cf6b884e5fe3227c9af7a1f0d80b0
> > 
> > I ran it with the series but am seeing an error trying to catch 
> > undecrypted PSP-UDP packets after disabling all versions with
> > set_config()
> > 
> > TAP version 13
> > 1..30
> > ok 1 psp.data_basic_send.v0_ip4 # SKIP Test requires IPv4
> > connectivity
> > ok 2 psp.data_basic_send.v0_ip6
> > ok 3 psp.data_basic_send.v1_ip4 # SKIP Test requires IPv4
> > connectivity
> > ok 4 psp.data_basic_send.v1_ip6
> > ok 5 psp.data_basic_send.v2_ip4 # SKIP Test requires IPv4
> > connectivity
> > ok 6 psp.data_basic_send.v2_ip6 # SKIP ('PSP version not
> > supported', 
> > 'hdr0-aes-gmac-128')
> > ok 7 psp.data_basic_send.v3_ip4 # SKIP Test requires IPv4
> > connectivity
> > ok 8 psp.data_basic_send.v3_ip6 # SKIP ('PSP version not
> > supported', 
> > 'hdr0-aes-gmac-256')
> > ok 9 psp.data_mss_adjust.ip4 # SKIP Test requires IPv4 connectivity
> > ok 10 psp.data_mss_adjust.ip6
> > ok 11 psp.data_send_off.ip4 # SKIP Test requires IPv4 connectivity
> > # Exception| Traceback (most recent call last):
> > # Exception|   File "/root/ksft-psp-set-config/net/lib/py/ksft.py",
> > line 
> > 420, in ksft_run
> > # Exception|     func(*args)
> > # Exception|   File "/root/./ksft-psp-set-
> > config/drivers/net/psp.py",
> > line 608, in data_send_off
> > # Exception|     udps.recv(8192, socket.MSG_DONTWAIT)
> > # Exception| BlockingIOError: [Errno 11] Resource temporarily
> > unavailable
> > # Exception|
> > not ok 12 psp.data_send_off.ip6
> > ok 13 psp.dev_list_devices
> > ok 14 psp.dev_get_device
> > ok 15 psp.dev_get_device_bad
> > ok 16 psp.dev_rotate
> > ok 17 psp.dev_rotate_spi
> > ok 18 psp.assoc_basic
> > ok 19 psp.assoc_bad_dev
> > ok 20 psp.assoc_sk_only_conn
> > ok 21 psp.assoc_sk_only_mismatch
> > ok 22 psp.assoc_sk_only_mismatch_tx
> > ok 23 psp.assoc_sk_only_unconn
> > ok 24 psp.assoc_version_mismatch
> > ok 25 psp.assoc_twice
> > ok 26 psp.data_send_bad_key
> > ok 27 psp.data_send_disconnect
> > ok 28 psp.data_stale_key
> > ok 29 psp.removal_device_rx # XFAIL Test only works on netdevsim
> > ok 30 psp.removal_device_bi # XFAIL Test only works on netdevsim
> > # Totals: pass:19 fail:1 xfail:2 xpass:0 skip:8 error:0
> > #
> > # Responder logs (0):
> > # STDERR:
> > # #  Set PSP enable on device 1 to 0x3
> > # #  Set PSP enable on device 1 to 0x0
> > 
> > I recall this working on an earlier prototype of this feature for
> > mlx5. 
> > Are the steering rules setup to drop PSP-UDP packets when the 
> > corresponding psp version is disabled?
> > 
> 
> We don't have per-psp version steering rules. If either version is
> requested, steering rules are configured. When all versions are
> disabled, steering rules are removed.
> With no steering rules installed, UDP traffic should not be affected.
> 
> I will take the test and debug what's going on, and get back to you.

So the test relies on TCP retransmissions to catch the encrypted echo
from the responder on the UDP socket. The timeline seems to be:
1. data_send_off disables PSP on its end.
2. data_send_off opens a UDP socket and binds it to port 1000.
3. data_send_off sends "data echo" to psp_responder on the control
connection.
4. psp_responder send "echo" on the now sabotaged PSP connection.
5. psp_responder acks the "echo request" on the control connection.
6. data_send_off receives the ack.
7. data_send_off tries to receive "echo" on the PSP connection but
expect_fail==True so stops after 100 ms.
8. data_send_off reenables PSP on its end.
9. data_send_off waits for "echo" to be received now that connectivity
is back for up to 350 ms.
10. data_send_off asserts that something is in the UDP socket queue.

So UDP packets could be enqueued if PSP packets are received between
steps 4-8.

It seems disabling PSP steering rules isn't as atomic as we thought,
and sometimes the first echo is discarded by steering. The default TCP
retransmission timeout is 200 ms so there are no retransmissions in the
~150-170 ms between steps 4-8.

With a slightly modified test that directly requests data echoes, the
test becomes more reliable. Additionally, you need the UDP_NO_CHECK6_RX
(102) socket options for the ipv6 version, otherwise zero-checksum UDP
packets are discarded by the stack. I vaguely remember doing this
change for this test in Jakub's repo a few years ago.

Anyway, here's the diff that makes both tests reliably pass:

--- a/tools/testing/selftests/drivers/net/psp.py
+++ b/tools/testing/selftests/drivers/net/psp.py
@@ -569,7 +569,8 @@ def _get_psp_ver_ip_variants():
 
 def _get_ip_variants():
     for ipv in ("4", "6"):
-        yield KsftNamedVariant(f"ip{ipv}", ipv)
+        for _ in range(0, 100):
+            yield KsftNamedVariant(f"ip{ipv}", ipv)
 
 
 @ksft_variants(_get_ip_variants())
@@ -604,16 +605,28 @@ def data_send_off(cfg, ipver):
             udps.bind(('0.0.0.0', 1000))
         else:
             udps = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+            # PSP packets have a zero UDP csum, accept them
(UDP_NO_CHECK6_RX)
+            udps.setsockopt(socket.IPPROTO_UDP, 102, 1)
             udps.bind(('::', 1000))
         wait_port_listen(1000, proto="udp")
 
         _req_echo(cfg, s, expect_fail=True)
 
+        echoes = 1
+        for _ in range(10):
+            try:
+                udps.recv(8192, socket.MSG_DONTWAIT | socket.MSG_PEEK)
+                break
+            except BlockingIOError:
+                _send_with_ack(cfg, b'data echo\0')
+                echoes += 1
+                time.sleep(0.05)
+
         cfg.pspnl.dev_set({"id": cfg.psp_dev_id,
                          "psp-versions-ena": info['psp-versions-
ena']})
         info = None
-        # We need some more TCP RTOs so lots of rounds
-        _recv_careful(s, 5, rounds=350)
+        # Drain the socket now that connectivity is back
+        _recv_careful(s, 5 * echoes, rounds=1000)
 
         # Will raise BlockingIOError if there are no packets
         udps.recv(8192, socket.MSG_DONTWAIT)


-------

With this (plus disabling all other test cases), I get:
# ./tools/testing/selftests/drivers/net/psp.py 
TAP version 13
1..200
ok 1 psp.data_send_off.ip4
...
ok 200 psp.data_send_off.ip6
# Totals: pass:200 fail:0 xfail:0 xpass:0 skip:0 error:0

I will continue to dig into what exactly happens in steering when PSP
gets disabled and why a packet is eaten with no traces.

Cosmin.