* [PATCH net v9 0/4] net: hsr: fix super-packet forwarding
@ 2026-11-12 15:33 Xin Xie
2026-11-12 15:33 ` [PATCH net v9 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Xin Xie @ 2026-11-12 15:33 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar
HSR and PRP tag, number and de-duplicate each frame on the wire. GRO
and GSO can hand them one skb holding several frames, which the lower
device then drops or which goes out with a single tag or RCT and
sequence number.
Patch 1 keeps software GRO off on HSR/PRP ports. GRO_HW that the
driver keeps on is left alone.
Patch 2 refuses bridge, bond, team and OVS masters as ports, a macvlan
or MACsec device on top of a bridge, bond, team or HSR/PRP master, and
devices that already have a master or cannot take an RX handler.
This fixes the syzbot recursive seqnr_lock report, as Sebastian
suggested [1].
A port check cannot catch every path back into an HSR master, such
as a tc/BPF redirect or a tunnel; those still nest the lock and can
deadlock on the same device. Fixing them without dropping traffic
needs seqnr_lock released before the port transmit, which can reorder
frames for receivers without aae9d6b616b5 ("hsr: Implement more robust
duplicate discard for HSR"). That is left to net-next, where Hangbin's
patch [2] fits. He dropped it in favor of this series [3]; as this
series no longer narrows seqnr_lock, it can go ahead there.
Patch 3 segments GSO skbs before per-frame processing, outside
seqnr_lock. GSO reaches an HSR/PRP port even with GRO off, typically
from a container attached to a PRP RedBox interlink through veth:
container/netns host PRP RedBox
prp0-peer ---- veth ---- prp0-int (interlink RX)
|
split GSO into frames
|
sequence number + RCT
/ \
LAN A LAN B
Some old Linux PRP senders put one RCT on a whole GSO skb; patch 3
drops such aggregates when it can detect them. With GRO_HW off on
virtio-net, the host may segment them first and the RCT arrives as
payload; upgrade the sender or disable GSO and TSO on its HSR/PRP
master.
Patch 4 adds selftests.
Tested on a lockdep kernel: the new selftests (also with Python 3.9)
and the existing hsr selftests pass, and the syzbot reproducer no
longer reports recursive locking. No new W=1 warnings.
No stable backport is requested. The remaining HSR dev->stats races
are left to a separate series, as Paolo suggested [4].
Based on net 30c71476d842; also applies to net-next. Patch 3
conflicts with Sebastian's pending net-next series [5]; its notes give
the resolution.
Changes since v8:
- Patch 1: the message and comments say which ports the rule covers.
- Patch 2: macvlan and MACsec are refused only on top of a bridge,
bond, team or HSR/PRP master; all ports are checked before any is
changed.
- Patch 3: copy a head shared with a clone before segmenting.
- Patch 4: no fixed sleep, no need for iproute2 to print the HSR
version, new refusal cases.
[1] https://lore.kernel.org/netdev/20261014151838.6RyMIRVM@linutronix.de/
[2] https://lore.kernel.org/netdev/20260805-hsr_deadlock-v1-1-d8a6f06fc5a8@kylinos.cn/
[3] https://lore.kernel.org/netdev/anXnxn-PBv8qH6VB@fedora/
[4] https://lore.kernel.org/netdev/4fc3b9f1-4bef-4b34-ae7a-e89037cce829@redhat.com/
[5] https://lore.kernel.org/netdev/20261104-hsr_ptp-v10-0-2c4a38ad973c@linutronix.de/
v8: https://lore.kernel.org/netdev/20261107220544.20-1-xiexinet@gmail.com/
v7: https://lore.kernel.org/netdev/20261009201324.17-1-xiexinet@gmail.com/
v6: https://lore.kernel.org/netdev/20260809121455.1745-1-xiexinet@gmail.com/
v5: https://lore.kernel.org/netdev/20260807140751.1351-1-xiexinet@gmail.com/
v4: https://lore.kernel.org/netdev/20260803222211.877-1-xiexinet@gmail.com/
v3: https://lore.kernel.org/netdev/20260731090224.18-1-xiexinet@gmail.com/
v2: https://lore.kernel.org/netdev/20260724161253.79-1-xiexinet@gmail.com/
v1: https://lore.kernel.org/netdev/20260722171836.196-1-xiexinet@gmail.com/
Xin Xie (4):
net: hsr: keep GRO disabled on HSR/PRP ports
net: hsr: refuse forwarding devices as HSR/PRP ports
net: hsr: segment GSO before per-frame forwarding
selftests: net: hsr: verify GRO policy and ordered forwarding
.../networking/net_cachelines/net_device.rst | 1 +
include/linux/netdevice.h | 8 +
net/core/dev.c | 15 +
net/hsr/hsr_device.c | 13 +-
net/hsr/hsr_forward.c | 158 ++
net/hsr/hsr_forward.h | 1 +
net/hsr/hsr_slave.c | 145 +-
net/hsr/hsr_slave.h | 2 +
tools/testing/selftests/net/hsr/Makefile | 7 +-
tools/testing/selftests/net/hsr/config | 10 +
.../selftests/net/hsr/hsr_gro_superpacket.py | 1456 +++++++++++++++++
.../selftests/net/hsr/hsr_gro_superpacket.sh | 9 +
.../net/hsr/hsr_ordered_forwarding.sh | 9 +
13 files changed, 1819 insertions(+), 15 deletions(-)
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
create mode 100755 tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
base-commit: 30c71476d842b094651bd7712f6c8ee8716147d0
--
2.53.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH net v9 1/4] net: hsr: keep GRO disabled on HSR/PRP ports
2026-11-12 15:33 [PATCH net v9 0/4] net: hsr: fix super-packet forwarding Xin Xie
@ 2026-11-12 15:33 ` Xin Xie
2026-11-12 15:33 ` [PATCH net v9 2/4] net: hsr: refuse forwarding devices as " Xin Xie
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Xin Xie @ 2026-11-12 15:33 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar, corbet, skhan, rdunlap,
linux-doc
HSR and PRP tag and de-duplicate each wire frame separately. GRO on a
member device merges several frames into one skb that cannot get valid
per-frame metadata, and userspace can turn GRO back on at any time.
Mark direct members (slave A/B, interlink) with a kernel role bit.
While it is set, __netdev_update_features() drops software GRO and
configurable GRO_HW from wanted_features as well as from the computed
features, before ndo_fix_features(). GRO_HW that the driver keeps on
is left alone. The request has to go too: veth adds and deletes its
NAPI instances according to wanted_features, so with GRO still
requested, closing an attached veth spins in napi_disable(), and
detaching a veth that was attached while down and then brought up hits
the BUG_ON() in napi_enable(). As for LRO disabled by an upper device,
GRO is not restored when the port leaves; ethtool can turn it back on.
The rule covers every direct member, PRP ports and ports with HSR
offload included: one simple rule is preferred to per-port exceptions,
at some cost in receive throughput.
The attach fails if software GRO stays active. A failed attach does
not restore the GRO request either. The next patch checks all ports
before any of them is changed, so only a failure at run time, for
example a driver that cannot turn GRO off or the upper device nesting
limit (-EMLINK), leaves a port with its GRO request dropped.
GSO skbs can still arrive, for example from a stacked member whose
lower device runs GRO; a later patch segments them.
Fixes: 5055cccfc2d1 ("net: hsr: Provide RedBox support (HSR-SAN)")
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
.../networking/net_cachelines/net_device.rst | 1 +
include/linux/netdevice.h | 8 +++
net/core/dev.c | 15 ++++++
net/hsr/hsr_slave.c | 54 +++++++++++++++++++
4 files changed, 78 insertions(+)
diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst
index 8eceaa80b686..0d87e4d92a58 100644
--- a/Documentation/networking/net_cachelines/net_device.rst
+++ b/Documentation/networking/net_cachelines/net_device.rst
@@ -169,6 +169,7 @@ unsigned_long:1 see_all_hwtstamp_requests
unsigned_long:1 change_proto_down
unsigned_long:1 netns_immutable
unsigned_long:1 fcoe_mtu
+unsigned_long:1 hsr_port
struct list_head net_notifier_list
struct macsec_ops* macsec_ops
struct udp_tunnel_nic_info* udp_tunnel_nic_info
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 435be7880d7e..0d13c7ecdd1f 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -2110,6 +2110,7 @@ enum netdev_reg_state {
* @change_proto_down: device supports setting carrier via IFLA_PROTO_DOWN
* @netns_immutable: interface can't change network namespaces
* @fcoe_mtu: device supports maximum FCoE MTU, 2158 bytes
+ * @hsr_port: direct HSR/PRP member (slave A/B or interlink)
*
* @net_notifier_list: List of per-net netdev notifier block
* that follow this device when it is moved
@@ -2531,6 +2532,13 @@ struct net_device {
unsigned long change_proto_down:1;
unsigned long netns_immutable:1;
unsigned long fcoe_mtu:1;
+ /* Direct HSR/PRP member: slave A/B or interlink. While set,
+ * software GRO and configurable GRO_HW are dropped from the
+ * request and before ndo_fix_features(); GRO_HW that the driver
+ * keeps on is left alone. Kernel role state only; no user ABI or
+ * offload capability.
+ */
+ unsigned long hsr_port:1;
struct list_head net_notifier_list;
diff --git a/net/core/dev.c b/net/core/dev.c
index df1a840b0334..c8c2de212556 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -11269,6 +11269,21 @@ int __netdev_update_features(struct net_device *dev)
features = netdev_get_wanted_features(dev);
+ /* A direct HSR/PRP port needs per-frame metadata. Drop software
+ * GRO and configurable GRO_HW from the request as well as from the
+ * features passed to ndo_fix_features(), as the core does for
+ * upper-disabled features: drivers such as veth derive state from
+ * wanted_features and need the requested and the active bit to
+ * agree. Fixed-on or driver-required GRO_HW is left to the driver.
+ */
+ if (dev->hsr_port) {
+ netdev_features_t gro = NETIF_F_GRO |
+ (dev->hw_features & NETIF_F_GRO_HW);
+
+ dev->wanted_features &= ~gro;
+ features &= ~gro;
+ }
+
if (dev->netdev_ops->ndo_fix_features)
features = dev->netdev_ops->ndo_fix_features(dev, features);
diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
index a546f70f9cc8..b7765a4bb992 100644
--- a/net/hsr/hsr_slave.c
+++ b/net/hsr/hsr_slave.c
@@ -11,6 +11,7 @@
#include <linux/etherdevice.h>
#include <linux/if_arp.h>
#include <linux/if_vlan.h>
+#include <net/netdev_lock.h>
#include "hsr_main.h"
#include "hsr_device.h"
#include "hsr_forward.h"
@@ -137,6 +138,37 @@ static int hsr_check_dev_ok(struct net_device *dev,
return 0;
}
+/* Member role and feature policy.
+ * Setting the role makes __netdev_update_features() clear GRO
+ * and configurable GRO_HW from the request and before
+ * ndo_fix_features(). A macvlan follows GRO on its lower device,
+ * which can turn it back on later. Only the role and feature
+ * update run under the device ops lock; unlink, promiscuous
+ * updates and the master recompute stay outside.
+ */
+static void hsr_portdev_role_set(struct net_device *dev)
+{
+ netdev_lock_ops(dev);
+ dev->hsr_port = true;
+ netdev_change_features(dev);
+ netdev_unlock_ops(dev);
+}
+
+static void hsr_portdev_role_clear(struct net_device *dev)
+{
+ netdev_lock_ops(dev);
+ dev->hsr_port = false;
+ /* Recompute features without the role. GRO was dropped from
+ * the request while the role was set and is not requested
+ * again; the user can turn it back on with ethtool. A dying
+ * device just drops the role, a netns move stays registered and
+ * is recomputed.
+ */
+ if (dev->reg_state == NETREG_REGISTERED)
+ netdev_change_features(dev);
+ netdev_unlock_ops(dev);
+}
+
/* Setup device to be added to the HSR bridge. */
static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
struct hsr_port *port,
@@ -169,6 +201,22 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
if (res)
goto fail_upper_dev_link;
+ /* Enable the member role and recompute before the RX handler
+ * is published, so that software GRO on this device is off
+ * before frames can arrive. This covers the direct member
+ * only: a stacked member receives from a lower device that
+ * may still run GRO and can deliver merged frames as GSO skbs.
+ * A still-active software GRO rejects the port, a remaining
+ * GRO_HW alone does not.
+ */
+ hsr_portdev_role_set(dev);
+ if (dev->features & NETIF_F_GRO) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "software GRO still on after feature update, cannot join");
+ res = -EBUSY;
+ goto fail_role_policy;
+ }
+
res = netdev_rx_handler_register(dev, hsr_handle_frame, port);
if (res)
goto fail_rx_handler;
@@ -177,7 +225,12 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
return 0;
fail_rx_handler:
+fail_role_policy:
+ /* Rollback order: unlink the upper, clear the role and
+ * recompute, then undo this setup's promiscuous increment.
+ */
netdev_upper_dev_unlink(dev, hsr_dev);
+ hsr_portdev_role_clear(dev);
fail_upper_dev_link:
if (!port->hsr->fwd_offloaded || port->type == HSR_PT_INTERLINK)
dev_set_promiscuity(dev, -1);
@@ -248,6 +301,7 @@ void hsr_del_port(struct hsr_port *port)
if (port->type == HSR_PT_SLAVE_A || port->type == HSR_PT_SLAVE_B)
vlan_vids_del_by_dev(port->dev, master->dev);
netdev_upper_dev_unlink(port->dev, master->dev);
+ hsr_portdev_role_clear(port->dev);
if (hsr->prot_version == PRP_V1 &&
port->type == HSR_PT_SLAVE_B) {
eth_hw_addr_set(port->dev, port->original_macaddress);
--
2.53.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net v9 2/4] net: hsr: refuse forwarding devices as HSR/PRP ports
2026-11-12 15:33 [PATCH net v9 0/4] net: hsr: fix super-packet forwarding Xin Xie
2026-11-12 15:33 ` [PATCH net v9 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
@ 2026-11-12 15:33 ` Xin Xie
2026-11-12 15:33 ` [PATCH net v9 3/4] net: hsr: segment GSO before per-frame forwarding Xin Xie
2026-11-12 15:33 ` [PATCH net v9 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding Xin Xie
3 siblings, 0 replies; 5+ messages in thread
From: Xin Xie @ 2026-11-12 15:33 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar,
syzbot+fbf74291c3b7e753b481, sd
syzbot reported recursive locking with an HSR device whose slave is a
bridge, and that bridge has the master of a second HSR device as a port.
The first device transmits its supervision frames with its seqnr_lock
held; the bridge floods them to the second HSR master, whose
hsr_dev_xmit() takes the seqnr_lock of that device:
WARNING: possible recursive locking detected
...
hsr_dev_xmit+0x18a/0x210 net/hsr/hsr_device.c:234
...
br_dev_xmit+0x1202/0x1b40
...
hsr_forward_skb+0x171c/0x2ac0 net/hsr/hsr_forward.c:728
send_hsr_supervision_frame+0x63b/0xcc0 net/hsr/hsr_device.c:351
hsr_announce+0x1f8/0x3a0 net/hsr/hsr_device.c:408
Interlink RX forwards to the slaves with the same lock held. A chain of
two devices does not deadlock, but such a port makes no sense: an HSR
ring and the two PRP LANs are attached to the node directly.
Refuse bridge and OVS masters and bond and team masters (also without
ports, since ports can be added later) as slave or interlink, and a
macvlan or MACsec device when an HSR/PRP, bridge, bond or team master
is below it; on an ordinary NIC or veth such a device stays valid. This
also refuses one on a DSA user port whose conduit is a bond or team,
since it can gain ports later too. A macvlan takes its GRO setting
from its lower device, so it is accepted only if GRO is off there when
it is attached. The lower devices of other ports are not checked,
because a DSA user port is an upper of its conduit and can offload HSR.
VLAN devices are already refused. The check runs at attach; later
changes below a stacked device are not rechecked. Not covered are a
tunnel whose route leads back through the HSR master, a tc or BPF
redirect to it, other stacked devices, and a software device without
an identifying flag such as batman-adv. Narrowing seqnr_lock, planned
for net-next, addresses that class as a whole.
Also refuse a device whose RX handler is in use or not supported
(IFF_NO_RX_HANDLER: ipvlan, VRF), or that already has a master, such as
a VRF port, with an extack message and the same error codes. All ports
are checked before any of them is changed. Some of these devices, such
as a macvlan lower, ipvlan or VRF, used to fail only in
netdev_rx_handler_register(), after the previous patch had dropped
their GRO request; a refused device now keeps its state, and so do the
ports given with it.
A bridge can no longer be used directly as the interlink; connect it
through a veth pair.
Reported-by: syzbot+fbf74291c3b7e753b481@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=fbf74291c3b7e753b481
Fixes: 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.")
Fixes: 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.")
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
net/hsr/hsr_device.c | 7 ++++
net/hsr/hsr_slave.c | 79 ++++++++++++++++++++++++++++++++++++++++++--
net/hsr/hsr_slave.h | 2 ++
3 files changed, 86 insertions(+), 2 deletions(-)
diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c
index fc512204600c..90ee55eb3c19 100644
--- a/net/hsr/hsr_device.c
+++ b/net/hsr/hsr_device.c
@@ -758,6 +758,13 @@ int hsr_dev_finalize(struct net_device *hsr_dev, struct net_device *slave[2],
INIT_LIST_HEAD(&hsr->proxy_node_db);
spin_lock_init(&hsr->list_lock);
+ /* Refuse a bad port before any port is changed. The same checks
+ * in hsr_add_port() remain as a safety net.
+ */
+ res = hsr_check_ports(slave, interlink, extack);
+ if (res)
+ return res;
+
eth_hw_addr_set(hsr_dev, slave[0]->dev_addr);
/* initialize protocol specific functions */
diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
index b7765a4bb992..666220b24dee 100644
--- a/net/hsr/hsr_slave.c
+++ b/net/hsr/hsr_slave.c
@@ -10,6 +10,7 @@
#include "hsr_slave.h"
#include <linux/etherdevice.h>
#include <linux/if_arp.h>
+#include <linux/if_macvlan.h>
#include <linux/if_vlan.h>
#include <net/netdev_lock.h>
#include "hsr_main.h"
@@ -97,6 +98,16 @@ bool hsr_port_exists(const struct net_device *dev)
return rcu_access_pointer(dev->rx_handler) == hsr_handle_frame;
}
+/* A lower device that forwards into other ports or into a second
+ * HSR/PRP device; called under RTNL by the lower-device walk.
+ */
+static int hsr_lower_forwards(struct net_device *lower,
+ struct netdev_nested_priv *priv)
+{
+ return is_hsr_master(lower) || netif_is_any_bridge_master(lower) ||
+ netif_is_lag_master(lower);
+}
+
static int hsr_check_dev_ok(struct net_device *dev,
struct netlink_ext_ack *extack)
{
@@ -120,6 +131,27 @@ static int hsr_check_dev_ok(struct net_device *dev,
return -EINVAL;
}
+ /* The RX handler is taken by devices such as bridge/bond/team
+ * ports, macvlan and ipvlan lowers and HSR ports, or unsupported
+ * (IFF_NO_RX_HANDLER). Refuse before any state change, so a
+ * failed attach leaves the device as it was.
+ */
+ if (dev->priv_flags & IFF_NO_RX_HANDLER) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "This device does not support an RX handler.");
+ return -EINVAL;
+ }
+ if (netdev_is_rx_handler_busy(dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "RX handler busy: device is already a lower of another device.");
+ return -EBUSY;
+ }
+ /* A master without an RX handler on its ports, such as VRF. */
+ if (netdev_master_upper_dev_get(dev)) {
+ NL_SET_ERR_MSG_MOD(extack, "Device already has a master.");
+ return -EBUSY;
+ }
+
if (is_vlan_dev(dev)) {
NL_SET_ERR_MSG_MOD(extack, "HSR on top of VLAN is not yet supported in this driver.");
return -EINVAL;
@@ -131,13 +163,56 @@ static int hsr_check_dev_ok(struct net_device *dev,
return -EOPNOTSUPP;
}
- /* HSR over bonded devices has not been tested, but I'm not sure it
- * won't work...
+ /* Forwarding devices feed HSR frames into another device's
+ * transmit path and can nest seqnr_lock through a second HSR
+ * device. A bridge, bond, team or OVS master makes no sense as a
+ * ring or LAN port and is refused by type. A macvlan or MACsec
+ * device is refused if such a master or an HSR/PRP master is
+ * below it. Other devices keep their lower devices unchecked:
+ * DSA user ports are uppers of their conduit.
+ */
+ if (netif_is_any_bridge_master(dev) || netif_is_lag_master(dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Cannot use a bridge, bond, team or OVS master as HSR port.");
+ return -EINVAL;
+ }
+ if ((netif_is_macvlan(dev) || netif_is_macsec(dev)) &&
+ netdev_walk_all_lower_dev(dev, hsr_lower_forwards, NULL)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Cannot use a macvlan or MACsec device stacked on an HSR, bridge, bond, team or OVS master as HSR port.");
+ return -EINVAL;
+ }
+ /* A macvlan takes its GRO setting from the lower device, so the
+ * member role cannot turn it off. Refuse at attach, before any
+ * state change, if the lower device runs GRO; it can still turn
+ * GRO on later.
*/
+ if (netif_is_macvlan(dev) &&
+ (macvlan_dev_real_dev(dev)->features & NETIF_F_GRO)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "macvlan inherits GRO from its lower device; turn GRO off on the lower device.");
+ return -EBUSY;
+ }
return 0;
}
+/* Check every port before any of them is changed, so that a refused
+ * port does not leave an earlier one modified.
+ */
+int hsr_check_ports(struct net_device *slave[2], struct net_device *interlink,
+ struct netlink_ext_ack *extack)
+{
+ int res;
+
+ res = hsr_check_dev_ok(slave[0], extack);
+ if (!res)
+ res = hsr_check_dev_ok(slave[1], extack);
+ if (!res && interlink)
+ res = hsr_check_dev_ok(interlink, extack);
+ return res;
+}
+
/* Member role and feature policy.
* Setting the role makes __netdev_update_features() clear GRO
* and configurable GRO_HW from the request and before
diff --git a/net/hsr/hsr_slave.h b/net/hsr/hsr_slave.h
index edc4612bb009..e19c779b500d 100644
--- a/net/hsr/hsr_slave.h
+++ b/net/hsr/hsr_slave.h
@@ -14,6 +14,8 @@
#include <linux/rtnetlink.h>
#include "hsr_main.h"
+int hsr_check_ports(struct net_device *slave[2], struct net_device *interlink,
+ struct netlink_ext_ack *extack);
int hsr_add_port(struct hsr_priv *hsr, struct net_device *dev,
enum hsr_port_type pt, struct netlink_ext_ack *extack);
void hsr_del_port(struct hsr_port *port);
--
2.53.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net v9 3/4] net: hsr: segment GSO before per-frame forwarding
2026-11-12 15:33 [PATCH net v9 0/4] net: hsr: fix super-packet forwarding Xin Xie
2026-11-12 15:33 ` [PATCH net v9 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
2026-11-12 15:33 ` [PATCH net v9 2/4] net: hsr: refuse forwarding devices as " Xin Xie
@ 2026-11-12 15:33 ` Xin Xie
2026-11-12 15:33 ` [PATCH net v9 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding Xin Xie
3 siblings, 0 replies; 5+ messages in thread
From: Xin Xie @ 2026-11-12 15:33 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar, richardcochran
HSR and PRP tag and de-duplicate each wire frame separately. A GSO
super-packet reaching the forwarding path hides several wire frames,
which cannot be tagged or numbered per frame. Such skbs reach a member
even with GRO off, for example from a container or VM attached through
veth, tap or virtio to a software RedBox, or from a member whose GRO_HW
cannot be turned off.
Add one forwarding entry, hsr_forward_input(), for master TX and for
frames received on any port. It segments GSO input outside seqnr_lock;
each segment is then numbered and submitted under the lock, one frame
at a time as before, and follows the normal forwarding and local
delivery paths. Master TX and interlink RX now take the lock there
instead of in their callers.
Classify by the effective inner protocol with the core's bounded VLAN
decoder (802.1Q, 802.1AD, nested or accelerated). Drop aggregates with
inner HSR/PRP, unreadable fragments or undecodable headers, and IPv4
or IPv6 aggregates whose nonzero IP length ends before the skb does,
since the trailing bytes would become segment payload. GSO_PARTIAL and
zero IP lengths are left to the segmenter. Drops are counted once on
the entry device in the core per-CPU drop counters. Segmentation writes
the control block, the headers and the L4 checksum, so a still-shared
input is unshared first, and a head still shared with a clone, for
example from a packet socket, is copied.
Older HSR and PRP senders can pass a whole GSO skb with one tag or one
RCT, for example over tap or virtio. Such aggregates, including locally
destined ones, are now dropped instead of being delivered.
Remove the NETIF_F_GSO_MASK types from the master's hw_features so the
core segments local traffic before ndo_start_xmit(). This also holds
over ports with HSR tag insertion offload, at some cost in performance.
Fixes: f421436a591d ("net/hsr: Add support for the High-availability Seamless Redundancy protocol (HSRv0)")
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
Integration with net-next: Sebastian's pending series "hsr: Allow to
send packets on specific port and pass the timestamp" (v10) changes
hsr_forward_skb() to take the requested TX port and a has_hsr_header
flag, and keeps seqnr_lock in hsr_dev_xmit() and in the interlink path
of hsr_handle_frame(). With both series:
- keep v10's PTP header parsing in hsr_dev_xmit(), then call
hsr_forward_input(skb, master, req_tx_port, has_hsr_header) without
taking seqnr_lock there;
- call hsr_forward_input(skb, port, HSR_PT_NONE, false) from
hsr_handle_frame(), without the interlink lock;
- add both arguments to hsr_forward_input() and hsr_forward_frame()
and pass them to hsr_forward_skb() for the skb and for each segment;
- keep both include sets in hsr_forward.c.
A frame with the PTP header is not GSO and takes the single-frame path
under the lock, as in v10. A GSO input with the header is classified
after the pull: with an HSR header it is dropped like any inner HSR/PRP
aggregate, otherwise each segment keeps req_tx_port.
Checked: on net-next fa5196392691 with v10 applied, this series with
the resolution builds without warnings, and on a lockdep kernel
hsr_ptp.sh, the two tests of patch 4 and the existing hsr selftests
pass.
net/hsr/hsr_device.c | 6 +-
net/hsr/hsr_forward.c | 158 ++++++++++++++++++++++++++++++++++++++++++
net/hsr/hsr_forward.h | 1 +
net/hsr/hsr_slave.c | 12 ++--
4 files changed, 165 insertions(+), 12 deletions(-)
diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c
index 90ee55eb3c19..5db71b49d08e 100644
--- a/net/hsr/hsr_device.c
+++ b/net/hsr/hsr_device.c
@@ -232,9 +232,7 @@ static netdev_tx_t hsr_dev_xmit(struct sk_buff *skb, struct net_device *dev)
skb->dev = master->dev;
skb_reset_mac_header(skb);
skb_reset_mac_len(skb);
- spin_lock_bh(&hsr->seqnr_lock);
- hsr_forward_skb(skb, master);
- spin_unlock_bh(&hsr->seqnr_lock);
+ hsr_forward_input(skb, master);
} else {
dev_core_stats_tx_dropped_inc(dev);
dev_kfree_skb_any(skb);
@@ -686,7 +684,7 @@ void hsr_dev_setup(struct net_device *dev)
dev->needs_free_netdev = true;
dev->hw_features = NETIF_F_SG | NETIF_F_FRAGLIST | NETIF_F_HIGHDMA |
- NETIF_F_GSO_MASK | NETIF_F_HW_CSUM |
+ NETIF_F_HW_CSUM |
NETIF_F_HW_VLAN_CTAG_TX |
NETIF_F_HW_VLAN_CTAG_FILTER;
diff --git a/net/hsr/hsr_forward.c b/net/hsr/hsr_forward.c
index 44f6d9c83180..9aefc5f0249f 100644
--- a/net/hsr/hsr_forward.c
+++ b/net/hsr/hsr_forward.c
@@ -12,6 +12,9 @@
#include <linux/skbuff.h>
#include <linux/etherdevice.h>
#include <linux/if_vlan.h>
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <net/gso.h>
#include "hsr_main.h"
#include "hsr_framereg.h"
@@ -794,3 +797,158 @@ void hsr_forward_skb(struct sk_buff *skb, struct hsr_port *port)
port->dev->stats.tx_dropped++;
kfree_skb(skb);
}
+
+/* Master TX and interlink RX are numbered locally: number and submit
+ * one frame at a time under seqnr_lock, as before. LAN A/B input
+ * keeps its received number and needs no lock.
+ */
+static void hsr_forward_frame(struct sk_buff *skb, struct hsr_port *port)
+{
+ struct hsr_priv *hsr = port->hsr;
+ bool numbered = port->type == HSR_PT_MASTER ||
+ port->type == HSR_PT_INTERLINK;
+
+ if (numbered)
+ spin_lock_bh(&hsr->seqnr_lock);
+ hsr_forward_skb(skb, port);
+ if (numbered)
+ spin_unlock_bh(&hsr->seqnr_lock);
+}
+
+/* One count per rejected GSO input on its entry device. */
+static void hsr_gso_drop_stat(struct hsr_port *port)
+{
+ if (port->type == HSR_PT_MASTER)
+ dev_core_stats_tx_dropped_inc(port->dev);
+ else
+ dev_core_stats_rx_dropped_inc(port->dev);
+}
+
+/* Return the inner EtherType using the core's bounded 802.1Q/802.1AD
+ * decoder. An accelerated outer tag is metadata, not frame bytes.
+ */
+static __be16 hsr_gso_effective_proto(const struct sk_buff *skb, int *l3off)
+{
+ struct ethhdr eh;
+ const struct ethhdr *eth;
+
+ if (skb_vlan_tag_present(skb) &&
+ !eth_type_vlan(skb->vlan_proto))
+ return 0;
+
+ eth = skb_header_pointer(skb, 0, sizeof(eh), &eh);
+ if (!eth)
+ return 0;
+
+ return __vlan_get_protocol(skb, eth->h_proto, l3off);
+}
+
+/* Bytes beyond the IP datagram must not become segment payload. Older
+ * PRP senders can append an RCT to a whole GSO skb. Zero IP lengths and
+ * GSO_PARTIAL headers do not provide the full aggregate length here.
+ */
+static bool hsr_gso_trailing_data(const struct sk_buff *skb, __be16 proto,
+ int l3off)
+{
+ unsigned int l3len;
+
+ if (skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL)
+ return false;
+
+ if (proto == htons(ETH_P_IP)) {
+ struct iphdr buffer;
+ const struct iphdr *iph;
+
+ iph = skb_header_pointer(skb, l3off, sizeof(buffer), &buffer);
+ if (!iph)
+ return true;
+ l3len = ntohs(iph->tot_len);
+ } else if (proto == htons(ETH_P_IPV6)) {
+ struct ipv6hdr buffer;
+ const struct ipv6hdr *ip6h;
+
+ ip6h = skb_header_pointer(skb, l3off, sizeof(buffer), &buffer);
+ if (!ip6h)
+ return true;
+ l3len = ntohs(ip6h->payload_len);
+ if (l3len)
+ l3len += sizeof(*ip6h);
+ } else {
+ return false;
+ }
+
+ return l3len && l3off + l3len < skb->len;
+}
+
+/* Unfold GSO super-packets before per-frame processing so that each
+ * wire frame gets its own HSR tag or PRP RCT and sequence number.
+ * Segmentation runs outside seqnr_lock; every segment is numbered and
+ * submitted under it like any other frame.
+ */
+void hsr_forward_input(struct sk_buff *skb, struct hsr_port *port)
+{
+ struct sk_buff *segs, *next;
+ __be16 proto;
+ int l3off;
+
+ if (likely(!skb_is_gso(skb))) {
+ hsr_forward_frame(skb, port);
+ return;
+ }
+
+ /* The gate is content-based, not port-based. An aggregate
+ * whose effective protocol is ETH_P_HSR or ETH_P_PRP cannot be
+ * segmented safely and is dropped, as is any skb whose headers
+ * cannot be read or that is marked unreadable (net_iov device
+ * memory): software segmentation cannot read its payload, so
+ * each segment would carry uninitialized data. Ordinary VLAN
+ * GSO (802.1Q/802.1AD at any reachable tag depth, accelerated
+ * or in-band) is classified to its inner protocol through the
+ * core VLAN decoder and segmented like plain input. With
+ * NETIF_F_HW_HSR_TAG_RM the lower has already stripped the tag,
+ * so such aggregates arrive plain.
+ */
+ if (!skb_frags_readable(skb))
+ goto drop;
+ proto = hsr_gso_effective_proto(skb, &l3off);
+ if (!proto)
+ goto drop;
+ if (proto == htons(ETH_P_HSR) || proto == htons(ETH_P_PRP))
+ goto drop;
+ if (hsr_gso_trailing_data(skb, proto, l3off))
+ goto drop;
+
+ /* Segmentation writes skb->cb, header metadata and the L4
+ * checksum in the head: work on a private skb if the input is
+ * still shared, and on a private head if a clone (for example
+ * from a packet socket) still shares it.
+ */
+ skb = skb_share_check(skb, GFP_ATOMIC);
+ if (!skb) {
+ hsr_gso_drop_stat(port);
+ return;
+ }
+ if (skb_cow_head(skb, 0))
+ goto drop;
+
+ /* features = 0 requests full software segmentation into single
+ * wire frames; tx_path is true only for locally generated
+ * traffic on the master.
+ */
+ segs = __skb_gso_segment(skb, 0, port->type == HSR_PT_MASTER, 0);
+ if (IS_ERR_OR_NULL(segs))
+ goto drop;
+
+ consume_skb(skb);
+ while (segs) {
+ next = segs->next;
+ segs->next = NULL;
+ hsr_forward_frame(segs, port);
+ segs = next;
+ }
+ return;
+
+drop:
+ hsr_gso_drop_stat(port);
+ kfree_skb(skb);
+}
diff --git a/net/hsr/hsr_forward.h b/net/hsr/hsr_forward.h
index 206636750b30..ec08c4330259 100644
--- a/net/hsr/hsr_forward.h
+++ b/net/hsr/hsr_forward.h
@@ -14,6 +14,7 @@
#include "hsr_main.h"
void hsr_forward_skb(struct sk_buff *skb, struct hsr_port *port);
+void hsr_forward_input(struct sk_buff *skb, struct hsr_port *port);
struct sk_buff *prp_create_tagged_frame(struct hsr_frame_info *frame,
struct hsr_port *port);
struct sk_buff *hsr_create_tagged_frame(struct hsr_frame_info *frame,
diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
index 666220b24dee..7fe7fa68af9e 100644
--- a/net/hsr/hsr_slave.c
+++ b/net/hsr/hsr_slave.c
@@ -76,15 +76,11 @@ static rx_handler_result_t hsr_handle_frame(struct sk_buff **pskb)
skb_reset_mac_len(skb);
/* Only the frames received over the interlink port will assign a
- * sequence number and require synchronisation vs other sender.
+ * sequence number and require synchronisation vs other sender;
+ * hsr_forward_input() takes seqnr_lock for them, one frame at a
+ * time.
*/
- if (port->type == HSR_PT_INTERLINK) {
- spin_lock_bh(&hsr->seqnr_lock);
- hsr_forward_skb(skb, port);
- spin_unlock_bh(&hsr->seqnr_lock);
- } else {
- hsr_forward_skb(skb, port);
- }
+ hsr_forward_input(skb, port);
finish_consume:
return RX_HANDLER_CONSUMED;
--
2.53.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net v9 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding
2026-11-12 15:33 [PATCH net v9 0/4] net: hsr: fix super-packet forwarding Xin Xie
` (2 preceding siblings ...)
2026-11-12 15:33 ` [PATCH net v9 3/4] net: hsr: segment GSO before per-frame forwarding Xin Xie
@ 2026-11-12 15:33 ` Xin Xie
3 siblings, 0 replies; 5+ messages in thread
From: Xin Xie @ 2026-11-12 15:33 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar
Add two tests, run by shell entries that exec one installed Python
helper.
hsr_gro_superpacket.sh checks the member GRO policy: GRO is off,
requested and active, while a veth is a member and after it leaves,
and can then be turned on again; a device refused at attach keeps its
request. It covers close and reopen of an attached veth, a veth
attached while down, the master's GSO type mask, a refused bridge port
that leaves the port given before it unchanged, a busy macvlan lower,
the refusal of bridge, bond and ipvlan devices, of a macvlan on a
bridge or on a bond and of a VRF port without leaving an upper link or
a promiscuity reference on any port, a macvlan whose lower device runs
GRO that is refused as second port with the first port unchanged, and
a macvlan on a veth without GRO that is accepted. Natural TCP from the
interlink is compared byte for byte, retransmissions included, with
both LAN captures. TCP from a SAN on one PRP LAN, in plain and VLAN
forms, is compared the same way with the ingress LAN and the local
delivery, and its reply must appear once on each LAN. Tags/RCT,
sequence numbers and VLAN stacks are checked.
hsr_ordered_forwarding.sh checks local data around interlink data and
UDP GSO aggregates, local and proxy supervision, two concurrent
producers and an observed sequence-number wrap.
Work is bounded to 120 seconds and cleanup to 20; INT/TERM remove the
test's namespaces. Packets are judged only on complete captures without
socket loss or truncation. A missing prerequisite SKIPs the affected
case; query errors and missing packet evidence FAIL, and the cases
after a failure are reported as not ok.
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
tools/testing/selftests/net/hsr/Makefile | 7 +-
tools/testing/selftests/net/hsr/config | 10 +
.../selftests/net/hsr/hsr_gro_superpacket.py | 1456 +++++++++++++++++
.../selftests/net/hsr/hsr_gro_superpacket.sh | 9 +
.../net/hsr/hsr_ordered_forwarding.sh | 9 +
5 files changed, 1490 insertions(+), 1 deletion(-)
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
create mode 100755 tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
diff --git a/tools/testing/selftests/net/hsr/Makefile b/tools/testing/selftests/net/hsr/Makefile
index 2150e487ac7d..70f8a804b44a 100644
--- a/tools/testing/selftests/net/hsr/Makefile
+++ b/tools/testing/selftests/net/hsr/Makefile
@@ -3,6 +3,8 @@
top_srcdir = ../../../../..
TEST_PROGS := \
+ hsr_gro_superpacket.sh \
+ hsr_ordered_forwarding.sh \
hsr_ping.sh \
hsr_prp_redbox.sh \
hsr_redbox.sh \
@@ -10,6 +12,9 @@ TEST_PROGS := \
prp_ping.sh \
# end of TEST_PROGS
-TEST_FILES += hsr_common.sh
+TEST_FILES += \
+ hsr_common.sh \
+ hsr_gro_superpacket.py \
+# end of TEST_FILES
include ../../lib.mk
diff --git a/tools/testing/selftests/net/hsr/config b/tools/testing/selftests/net/hsr/config
index 205cc4d3d64b..d9463ae0025a 100644
--- a/tools/testing/selftests/net/hsr/config
+++ b/tools/testing/selftests/net/hsr/config
@@ -1,6 +1,16 @@
+CONFIG_BONDING=y
CONFIG_BRIDGE=y
CONFIG_HSR=y
+CONFIG_INET=y
+CONFIG_IP_ADVANCED_ROUTER=y
+CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IPV6=y
+CONFIG_IPV6_MULTIPLE_TABLES=y
+CONFIG_IPVLAN=y
+CONFIG_MACVLAN=y
+CONFIG_NET_L3_MASTER_DEV=y
CONFIG_NET_SCH_NETEM=m
+CONFIG_NET_VRF=y
+CONFIG_PACKET=y
CONFIG_VETH=y
CONFIG_VLAN_8021Q=m
diff --git a/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
new file mode 100755
index 000000000000..3548ee0adfae
--- /dev/null
+++ b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
@@ -0,0 +1,1456 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""HSR/PRP GRO policy, segmentation and functional ordering regressions."""
+# pylint: disable=too-many-lines
+
+from array import array
+from collections import Counter
+from contextlib import contextmanager
+import ctypes
+import errno
+import json
+import os
+import re
+import selectors
+import shutil
+import signal
+import socket
+import struct
+import subprocess
+import sys
+import threading
+import time
+import uuid
+
+
+class Skip(Exception):
+ """A missing prerequisite for one case."""
+
+
+# INT/TERM only record the signal; Lab.remaining() raises at the next check.
+SIGNALLED = []
+
+# Errors a case reports as "not ok"; anything else is a bug in this script.
+CASE_ERRORS = (OSError, ValueError, RuntimeError, LookupError,
+ subprocess.SubprocessError, struct.error)
+
+
+def require(condition, message):
+ """Raise ValueError(message) unless condition holds."""
+ if not condition:
+ raise ValueError(message)
+
+
+def features(text):
+ """Parse ethtool -k: name -> (active, fixed, requested)."""
+ result = {}
+ for line in text.splitlines():
+ if ":" not in line or line.startswith("Features for"):
+ continue
+ name, state = line.strip().split(":", 1)
+ match = re.fullmatch(r"\s*(on|off)((?:\s+\[[^\]]*\])*)", state)
+ require(match is not None, "malformed feature line: " + line)
+ # ethtool marks a wanted bit that differs from the active one.
+ requested = re.search(r"\[requested (on|off)\]", match[2])
+ result[name] = (match[1], "[fixed]" in match[2],
+ requested[1] if requested else match[1])
+ require(result, "empty feature listing")
+ return result
+
+
+def decode(raw, ancillary=(), prp=False):
+ """Decode one captured frame; PRP RCTs are recognized only if prp."""
+ require(len(raw) >= 14, "short Ethernet capture")
+ proto = struct.unpack_from("!H", raw, 12)[0]
+ off, stack, aux, checksum_partial = 14, [], (), False
+ for level, kind, data in ancillary:
+ if level == 263 and kind == 8:
+ require(len(data) >= 20, "short PACKET_AUXDATA")
+ status, length, snap, _, _, tci, tpid = struct.unpack("IIIHHHH", data[:20])
+ require(length == snap, "truncated packet auxiliary length")
+ checksum_partial = bool(status & 8)
+ if status & 0x10:
+ aux = ((tpid if status & 0x40 else 0x8100, tci),)
+ while proto in (0x8100, 0x88a8):
+ require(off + 4 <= len(raw), "short VLAN header")
+ tci, inner = struct.unpack_from("!HH", raw, off)
+ stack.append((proto, tci))
+ proto, off = inner, off + 4
+ row = {"raw": raw, "size": len(raw) + 4 * len(aux), "vlans": aux + tuple(stack),
+ "aux": aux, "inband": tuple(stack), "src": raw[6:12], "dst": raw[:6],
+ "proto": proto, "seq": None, "path": None, "payload": b"", "ports": (),
+ "tcpseq": None, "supervision": None, "supseq": None, "tag": False,
+ "encoding": "plain", "checksum_partial": checksum_partial}
+ end = len(raw)
+ if proto == 0x892f:
+ require(off + 6 <= end, "short HSR tag")
+ field, seq, proto = struct.unpack_from("!HHH", raw, off)
+ row.update(seq=seq, path=field >> 12, tag=True, encoding="hsr")
+ expected = len(raw) - 14 - (4 if raw[12:14] == b"\x81\x00" else 0)
+ require(field & 0xfff == expected, "HSR LSDU size differs")
+ off += 6
+ elif prp and end >= off + 6 and raw[-2:] == b"\x88\xfb":
+ seq, field = struct.unpack_from("!HH", raw, end - 6)
+ expected = len(raw) - 14 - (4 if raw[12:14] == b"\x81\x00" else 0)
+ # Like prp_fill_frame_info(): a size mismatch means a plain frame.
+ if field & 0xfff == expected:
+ row.update(seq=seq, path=field >> 12, tag=True, encoding="prp")
+ end -= 6
+ row["proto"] = proto
+ row["network"] = raw[off:end]
+ if raw[:3] == b"\x01\x15\x4e" and proto == 0x88fb:
+ require(off + 12 <= end, "short supervision payload")
+ _, seq, kind, size = struct.unpack_from("!HHBB", raw, off)
+ require(kind in (20, 21, 22, 23) and size in (6, 12), "invalid supervision TLV")
+ row.update(supervision=raw[off + 6:off + 12], supseq=seq)
+ return row
+ if proto != 0x0800:
+ return row
+ require(off + 20 <= end and raw[off] >> 4 == 4, "invalid IPv4 header")
+ ihl, total = (raw[off] & 15) * 4, struct.unpack_from("!H", raw, off + 2)[0]
+ require(ihl >= 20 and total >= ihl and off + total <= end, "invalid IP extent")
+ protocol, l4 = raw[off + 9], off + ihl
+ ipend = off + total
+ row.update(ip_src=raw[off + 12:off + 16], ip_dst=raw[off + 16:off + 20])
+ if protocol == 6:
+ require(l4 + 20 <= ipend, "short TCP header")
+ size = (raw[l4 + 12] >> 4) * 4
+ require(size >= 20 and l4 + size <= ipend, "invalid TCP extent")
+ row.update(ports=struct.unpack_from("!HH", raw, l4),
+ tcpseq=struct.unpack_from("!I", raw, l4 + 4)[0],
+ tcpflags=raw[l4 + 13], payload=raw[l4 + size:ipend])
+ elif protocol == 17:
+ require(l4 + 8 <= ipend, "short UDP header")
+ size = struct.unpack_from("!H", raw, l4 + 4)[0]
+ require(size >= 8 and l4 + size == ipend, "invalid UDP extent")
+ row.update(ports=struct.unpack_from("!HH", raw, l4), payload=raw[l4 + 8:ipend])
+ return row
+
+
+class Capture:
+ """One AF_PACKET receive socket bound to a single device."""
+
+ def __init__(self, lab, ns, dev, pkttype=None, prp=False):
+ self.lab, self.rows, self.seen = lab, [], 0
+ self.ns, self.dev, self.packets = ns, dev, None
+ self.pkttype, self.name, self.prp = pkttype, ns + "/" + dev, prp
+ self.complete, self.drops, self.truncated = False, None, 0
+ self.last_seen = time.monotonic()
+ # Protocol 0 registers no receive hook until bind() names the device.
+ self.socket = lab.sock(ns, socket.AF_PACKET, socket.SOCK_RAW, 0)
+ self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 8 * 1024 * 1024)
+ self.socket.setsockopt(263, 8, 1)
+ self.socket.bind((dev, 3))
+ self.socket.setblocking(False)
+ lab.selector.register(self.socket, selectors.EVENT_READ, self)
+
+ def drain(self):
+ """Read queued frames; return (count, queue empty)."""
+ count = 0
+ while count < 256:
+ self.lab.remaining()
+ try:
+ raw, anc, flags, addr = self.socket.recvmsg(131072, 256)
+ except BlockingIOError:
+ return count, True
+ self.seen += 1
+ self.last_seen = time.monotonic()
+ count += 1
+ if flags & (socket.MSG_TRUNC | socket.MSG_CTRUNC):
+ self.truncated += 1
+ raise ValueError(self.name + ": capture truncated")
+ if addr[0] == self.dev and (self.pkttype is None or addr[2] == self.pkttype):
+ self.rows.append(decode(raw, anc, self.prp))
+ return count, False
+
+ def validate(self):
+ """Fail unless the capture completed without loss."""
+ require(self.complete and self.drops == 0 and self.truncated == 0,
+ self.name + ": incomplete/dropped/truncated capture")
+
+
+@contextmanager
+def signals_blocked():
+ """Hold INT/TERM back while namespace or handler state switches."""
+ previous = signal.pthread_sigmask(signal.SIG_BLOCK, {signal.SIGINT, signal.SIGTERM})
+ try:
+ yield
+ finally:
+ signal.pthread_sigmask(signal.SIG_SETMASK, previous)
+
+
+class Lab:
+ """Owned namespaces, sockets and captures of one test run."""
+
+ def __init__(self, deadline):
+ self.deadline = deadline
+ self.names, self.sockets, self.captures, self.threads = [], [], [], []
+ self.case_start = 0
+ self.selector = selectors.DefaultSelector()
+ self.original = os.open("/proc/self/ns/net", os.O_RDONLY)
+ self.token, self.restore_failed = uuid.uuid4().hex[:10], False
+ self.stop = threading.Event()
+ self.libc = ctypes.CDLL(None, use_errno=True)
+
+ def begin(self):
+ """Start a case: captures left open by an earlier case are closed."""
+ for cap in self.captures[self.case_start:]:
+ if not cap.complete:
+ self.selector.unregister(cap.socket)
+ cap.socket.close()
+ self.case_start = len(self.captures)
+
+ def remaining(self):
+ """Seconds left in the work deadline; raise once it or a signal hit."""
+ if SIGNALLED:
+ raise InterruptedError(f"signal {SIGNALLED[0]}")
+ value = self.deadline - time.monotonic()
+ require(value > 0, "120-second work deadline expired")
+ return value
+
+ def run(self, args, check=True):
+ """Run a command with a bounded timeout."""
+ require(not self.restore_failed, "network namespace restoration failed")
+ result = subprocess.run([str(arg) for arg in args], text=True, capture_output=True,
+ timeout=min(5, self.remaining()), check=False)
+ require(not check or result.returncode == 0,
+ f"{args}: rc{result.returncode} {result.stderr.strip()}")
+ return result
+
+ def ip(self, ns, *args, check=True):
+ """Run ip(8) in a namespace."""
+ return self.run(["ip", "-n", ns, *args], check)
+
+ def cmd(self, ns, *args, check=True):
+ """Run a command in a namespace."""
+ return self.run(["ip", "netns", "exec", ns, *args], check)
+
+ def features(self, ns, dev):
+ """Return the parsed ethtool -k listing of a device."""
+ return features(self.cmd(ns, "ethtool", "-k", dev).stdout)
+
+ def ns(self):
+ """Create an owned namespace with loopback up."""
+ name = f"hsrp4-{self.token}-{len(self.names)}"
+ require(not os.path.lexists("/var/run/netns/" + name), "namespace collision")
+ self.names.append(name)
+ self.run(["ip", "netns", "add", name])
+ self.ip(name, "link", "set", "dev", "lo", "up")
+ return name
+
+ def veth(self, ns, dev, peer_ns, peer):
+ """Create a veth pair across two namespaces."""
+ self.ip(ns, "link", "add", "name", dev, "type", "veth", "peer", "name", peer,
+ "netns", peer_ns)
+
+ def setns(self, descriptor):
+ """Switch this thread's network namespace."""
+ if self.libc.setns(descriptor, 0x40000000):
+ error = ctypes.get_errno()
+ raise OSError(error, os.strerror(error))
+
+ @contextmanager
+ def enter(self, ns):
+ """Run the body in a namespace and always switch back."""
+ descriptor, entered = None, False
+ try:
+ with signals_blocked():
+ descriptor = os.open("/var/run/netns/" + ns, os.O_RDONLY)
+ self.setns(descriptor)
+ entered = True
+ yield
+ finally:
+ with signals_blocked():
+ try:
+ if entered:
+ self.setns(self.original)
+ except BaseException:
+ self.restore_failed = True
+ raise
+ finally:
+ if descriptor is not None:
+ os.close(descriptor)
+
+ def sock(self, ns, family, kind, proto=0):
+ """Open an owned socket bound to a namespace."""
+ with self.enter(ns):
+ result = socket.socket(family, kind, proto)
+ self.sockets.append(result)
+ result.settimeout(min(3, self.remaining()))
+ return result
+
+ def capture(self, ns, dev, pkttype=None, prp=False):
+ """Start a capture owned by the current case."""
+ result = Capture(self, ns, dev, pkttype, prp)
+ self.captures.append(result)
+ return result
+
+ def observe(self, seconds, pump=None):
+ """Drain ready captures for some time, calling pump() each round."""
+ end = time.monotonic() + seconds
+ while True:
+ self.remaining()
+ if pump is not None:
+ pump()
+ for key, _ in self.selector.select(min(0.01, max(0, end - time.monotonic()))):
+ key.data.drain()
+ if time.monotonic() >= end:
+ return
+
+ def finish(self, done=None):
+ """Complete this case's captures.
+
+ done() states when every expected frame has been captured; the
+ deadline bounds the wait. Only without such a condition does a
+ 100 ms quiet period end the wait.
+ """
+ active = [cap for cap in self.captures[self.case_start:] if not cap.complete]
+ end = min(self.deadline, time.monotonic() + 2)
+ quiet = time.monotonic()
+ while time.monotonic() < end:
+ self.observe(0.01)
+ empty = True
+ for cap in active:
+ count, drained = cap.drain()
+ if count or not drained:
+ empty = False
+ quiet = max([quiet] + [cap.last_seen for cap in active])
+ if done is not None:
+ if done():
+ break
+ elif empty and time.monotonic() - quiet >= 0.1:
+ break
+ else:
+ # A missing expected frame is reported by the case's verifier.
+ require(done is not None, "capture failed to drain within fixed deadline")
+ end = max(end, min(self.deadline, time.monotonic() + 1))
+ for ns in sorted({cap.ns for cap in active}):
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require(ns in self.names, "capture namespace is not owned")
+ self.ip(ns, "link", "set", "dev", "lo", "down")
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ state = link_state(self, ns, "lo")
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require("UP" not in state["flags"],
+ "capture cutoff loopback is still up")
+ # Rebinding to a different down device synchronizes the old receive hook.
+ # Its ENETDOWN socket error must be cleared before draining queued frames.
+ for cap in active:
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require(cap.dev != "lo", "capture already uses the cutoff device")
+ cap.socket.bind(("lo", 0))
+ require(cap.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) == errno.ENETDOWN,
+ cap.name + ": capture cutoff did not report ENETDOWN")
+ for cap in active:
+ while True:
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ if cap.drain()[1]:
+ break
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ packets, cap.drops = struct.unpack("II", cap.socket.getsockopt(263, 6, 8))
+ cap.packets = packets
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require(packets == cap.seen + cap.drops,
+ f"{cap.name}: queued capture remains packets={packets} "
+ f"seen={cap.seen} drops={cap.drops}")
+ cap.complete = True
+ cap.validate()
+ self.selector.unregister(cap.socket)
+ cap.socket.close()
+ print(f"# capture {cap.name} packets={cap.seen} drops=0 truncated=0 complete=1",
+ flush=True)
+
+ def close(self):
+ """Stop threads, close sockets and delete owned namespaces."""
+ self.stop.set()
+ end, errors = time.monotonic() + 20, []
+ for sock in self.sockets:
+ sock.close()
+ for thread in self.threads:
+ if thread.ident is not None:
+ thread.join(max(0, end - time.monotonic()))
+ if thread.is_alive():
+ errors.append("sender thread did not stop")
+ self.selector.close()
+ if self.restore_failed:
+ # ip netns del removes the /var/run/netns bind mount, which does
+ # not depend on the network namespace of this thread.
+ errors.append("network namespace restoration failed")
+ for name in reversed(self.names):
+ if time.monotonic() >= end:
+ errors.append("cleanup deadline expired with owned namespaces remaining")
+ break
+ try:
+ result = subprocess.run(["ip", "netns", "del", name],
+ capture_output=True, text=True, check=False,
+ timeout=min(3, end - time.monotonic()))
+ require(result.returncode == 0 and
+ not os.path.lexists("/var/run/netns/" + name),
+ "namespace cleanup failed: " + name + " " + result.stderr)
+ except (OSError, ValueError, subprocess.SubprocessError) as error:
+ errors.append(str(error))
+ os.close(self.original)
+ directory = os.getenv("HSR_CAPTURE_EVIDENCE")
+ if directory:
+ try:
+ for number, cap in enumerate(self.captures):
+ require(time.monotonic() < end, "cleanup/evidence deadline expired")
+ path = os.path.join(directory, f"{self.token}-{number}.json")
+ record = {"name": cap.name, "seen": cap.seen, "packets": cap.packets,
+ "drops": cap.drops, "truncated": cap.truncated,
+ "complete": cap.complete, "pkttype": cap.pkttype,
+ "rows": cap.rows}
+ with open(path, "x", encoding="utf-8") as stream:
+ json.dump(record, stream, default=lambda value: value.hex())
+ except (OSError, ValueError, TypeError) as error:
+ errors.append("capture evidence: " + str(error))
+ require(not errors, "; ".join(errors))
+
+
+def coverage(frames, expected):
+ """Count how often each stream byte occurs in (offset, payload) frames."""
+ counts = array("I", [0]) * len(expected)
+ for offset, payload in frames:
+ require(payload and offset >= 0 and offset + len(payload) <= len(expected),
+ "invalid stream extent")
+ require(payload == expected[offset:offset + len(payload)], "stream payload differs")
+ for position in range(offset, offset + len(payload)):
+ counts[position] += 1
+ return counts
+
+
+def protocol_order(rows, wrap=False, key="seq"):
+ """Require strictly increasing 16-bit protocol sequence numbers."""
+ numbers = [row[key] for row in rows]
+ require(numbers and all(value is not None for value in numbers), "missing protocol sequence")
+ # Successive pairs; itertools.pairwise() needs Python 3.10.
+ steps = [(numbers[index - 1], numbers[index]) for index in range(1, len(numbers))]
+ require(all(0 < ((right - left) & 0xffff) < 0x8000 for left, right in steps),
+ "protocol sequence out of order")
+ if wrap:
+ require(any(left > right for left, right in steps), "no actual sequence wrap")
+
+
+def contains(have, wanted):
+ """Multiset inclusion of Counters (their >= needs Python 3.10)."""
+ return all(have[key] >= count for key, count in wanted.items())
+
+
+def wire_rows(rows, port, stack, tagged=True):
+ """Return and check one lane's frames of a flow."""
+ data = [row for row in rows if row["ports"] and row["ports"][1] == port and row["payload"]]
+ require(data, "no captured flow payload")
+ for row in data:
+ require(row["tag"] == tagged, "flow tag/RCT missing or unexpected")
+ require(row["encoding"] == ("hsr" if tagged else "plain"), "wrong protocol tag kind")
+ require(row["vlans"] == stack, "flow VLAN stack differs")
+ require(row["size"] <= 1520 + 4 * len(stack), "unsegmented output frame")
+ require(len({row["src"] for row in data}) == 1 and
+ len({row["dst"] for row in data}) == 1, "flow MAC addresses differ within lane")
+ if tagged:
+ protocol_order(data)
+ return data
+
+
+def copy_identity(row):
+ """Fields the two LAN copies of a frame share."""
+ # Lane/path and source substitution differ legitimately between the two ports.
+ return (row["seq"], row["vlans"], row["dst"], row["proto"], row["network"])
+
+
+def verify_stream(captures, port, payload, stack=(), local=False, inband=False, ingress=None):
+ """Check every TCP byte occurrence on the sender against the outputs."""
+ for cap in captures:
+ cap.validate()
+ source = [row for row in captures[0].rows if row["ports"] and
+ row["ports"][1] == port and row["payload"]]
+ require(source, "sender TCP capture empty")
+ syn = [row for row in captures[0].rows if row["ports"] and
+ row["ports"][1] == port and row.get("tcpflags", 0) & 2]
+ require(syn, "missing captured TCP initial sequence")
+ isn = (syn[0]["tcpseq"] + 1) & 0xffffffff
+ source_counts = coverage([((row["tcpseq"] - isn) & 0xffffffff, row["payload"])
+ for row in source], payload)
+ require(0 not in source_counts, "sender did not send complete stream")
+ aggregate = [row for row in source if len(row["payload"]) > 1500]
+ if ingress is not None:
+ ingress.validate()
+ inputs = [row for row in ingress.rows if row["ports"] and
+ row["ports"][1] == port and row["payload"]]
+ require(all(row["vlans"] == stack and row["encoding"] == "plain" for row in inputs),
+ "HSR/PRP ingress classification/VLAN differs")
+ require(coverage([((row["tcpseq"] - isn) & 0xffffffff, row["payload"])
+ for row in inputs], payload) == source_counts,
+ "sender/actual ingress stream occurrence differs")
+ require(not aggregate or any(len(row["payload"]) > 1500 for row in inputs),
+ "no actual ingress aggregate")
+ require(all(row["vlans"] == stack for row in source), "sender VLAN stack differs")
+ if stack:
+ require(all(row["aux"] == (() if inband else stack[:1]) and
+ row["inband"] == (stack if inband else stack[1:]) for row in aggregate),
+ "sender aggregate VLAN encoding differs")
+ if local:
+ outputs = [wire_rows(captures[1].rows, port, stack, tagged=False)]
+ require(all(row["ports"][1] == port for row in outputs[0]),
+ "unexpected local flow direction")
+ else:
+ outputs = [wire_rows(cap.rows, port, stack) for cap in captures[1:]]
+ for index, rows in enumerate(outputs):
+ require(all(row["path"] == index for row in rows), "wrong HSR lane/path")
+ require([copy_identity(row) for row in outputs[0]] ==
+ [copy_identity(row) for row in outputs[1]], "LAN copies differ")
+ for rows in outputs:
+ observed = coverage([((row["tcpseq"] - isn) & 0xffffffff, row["payload"])
+ for row in rows], payload)
+ require(observed == source_counts,
+ "lost/duplicate stream occurrence (including retransmission)")
+ # Missing aggregation is a prerequisite gap only after transfer and all other checks.
+ if not aggregate:
+ raise Skip("no sender GSO aggregate observed")
+ print(f"# TCP bytes={len(payload)} sent-occurrences={sum(source_counts)} "
+ f"aggregates={len(aggregate)} outputs={[len(rows) for rows in outputs]}", flush=True)
+
+
+def verify_udp(case, captures, expected, ports, vlans=(), expect_wrap=False,
+ input_order=False, producer_order=False):
+ """Check both lanes against the expected UDP payloads."""
+ lanes = []
+ for index, cap in enumerate(captures):
+ cap.validate()
+ rows = udp_rows(cap, ports)
+ require(Counter(row["payload"] for row in rows) == Counter(expected),
+ case + ": UDP payload/ID set differs")
+ if input_order:
+ require([row["payload"] for row in rows] == expected, case + ": input order differs")
+ if producer_order:
+ for producer in (1, 2):
+ observed = [row["payload"] for row in rows if row["payload"][6] == producer]
+ require(observed == [body for body in expected if body[6] == producer],
+ case + ": producer order differs")
+ for row in rows:
+ require(row["tag"] and row["size"] <= 1520 + 4 * len(vlans),
+ case + ": missing tag or unsegmented frame")
+ require(row["vlans"] == vlans, case + ": VLAN stack differs")
+ require(row["path"] == index, case + ": wrong HSR lane/path")
+ require(row["encoding"] == "hsr", case + ": wrong protocol tag kind")
+ protocol_order(rows, expect_wrap)
+ lanes.append(rows)
+ require([copy_identity(row) for row in lanes[0]] == [copy_identity(row) for row in lanes[1]],
+ case + ": unequal LAN copies")
+ print(f"# {case} expected={len(expected)} lanes={[len(rows) for rows in lanes]}",
+ flush=True)
+ return lanes
+
+
+def local_pair(lab, ns, dev):
+ """Create an up veth pair dev/devp inside one namespace."""
+ lab.veth(ns, dev, ns, dev + "p")
+ for name in (dev, dev + "p"):
+ lab.ip(ns, "link", "set", "dev", name, "up")
+
+
+def link_state(lab, ns, dev):
+ """Return the ip -j -d link description of a device."""
+ result = lab.ip(ns, "-j", "-d", "link", "show", "dev", dev)
+ rows = json.loads(result.stdout)
+ require(len(rows) == 1, f"{dev}: missing link description")
+ state = rows[0]
+ require(isinstance(state.get("ifindex"), int), "missing ifindex")
+ require(isinstance(state.get("promiscuity"), int),
+ f"{dev}: missing promiscuity count")
+ return state
+
+
+def gro_state(lab, ns, dev, expected, label):
+ """Require software GRO to be expected, as requested and as active."""
+ listing = lab.features(ns, dev)
+ require("generic-receive-offload" in listing,
+ f"{dev}: missing software GRO feature")
+ active, _, requested = listing["generic-receive-offload"]
+ require(active == expected and requested == expected,
+ f"{label}: {dev}: GRO expected {expected}, "
+ f"observed active {active} requested {requested}")
+
+
+def set_gro(lab, ns, dev, wanted, active=None):
+ """Request GRO; the read-back, not ethtool's output, is the oracle."""
+ result = lab.cmd(ns, "ethtool", "-K", dev, "gro", wanted, check=False)
+ # ethtool exits 1 when the kernel keeps a feature from changing.
+ require(result.returncode == 0 or (active is not None and result.returncode == 1),
+ f"{dev}: GRO setter rc{result.returncode} {result.stderr.strip()}")
+ gro_state(lab, ns, dev, active if active is not None else wanted,
+ "requested GRO " + wanted)
+
+
+def interlink_unsupported(stderr):
+ """True if iproute2 does not know the interlink option."""
+ error = stderr.lower()
+ return "interlink" in error and "rtnetlink" not in error and any(
+ word in error for word in ("unknown command", "unknown option",
+ "unknown argument", 'what is "interlink"'))
+
+
+def hsr_attach(lab, ns, master, first, second, interlink=None):
+ """Create an HSRv1 device over the given ports and bring it up."""
+ args = ["link", "add", "name", master, "type", "hsr",
+ "slave1", first, "slave2", second, "version", "1"]
+ if interlink is not None:
+ args.extend(("interlink", interlink))
+ result = lab.ip(ns, *args, check=False)
+ if result.returncode and interlink is not None:
+ # Tool-option absence needs a successful ordinary-HSR control.
+ control = lab.ip(ns, "link", "add", "ctl", "type", "hsr",
+ "slave1", first, "slave2", second,
+ "version", "1", check=False)
+ if control.returncode == 0:
+ lab.ip(ns, "link", "del", "ctl")
+ require(control.returncode == 0,
+ "interlink failed and ordinary HSR control failed: "
+ f"{result.stderr.strip()} / {control.stderr.strip()}")
+ if interlink_unsupported(result.stderr):
+ raise Skip("iproute2 has no interlink option")
+ require(result.returncode == 0,
+ "HSR attach failed: " + result.stderr.strip())
+ lab.ip(ns, "link", "set", "dev", master, "up")
+
+
+def policy(lab):
+ """All roles, initial preferences and last requests, twice each."""
+ ns = lab.ns()
+ for dev in ("m", "p", "q"):
+ local_pair(lab, ns, dev)
+ for role in ("A", "B", "interlink"):
+ for initial in ("on", "off"):
+ for last in ("on", "off"):
+ label = f"{role} initial={initial} last={last}"
+ before = link_state(lab, ns, "m")
+ for _ in range(2):
+ set_gro(lab, ns, "m", initial)
+ first = "m" if role == "A" else "p"
+ second = "m" if role == "B" else "q"
+ hsr_attach(lab, ns, "hsr0", first, second,
+ "m" if role == "interlink" else None)
+ gro_state(lab, ns, "m", "off", label + " after attach")
+ for wanted in ("on", "off", last):
+ set_gro(lab, ns, "m", wanted, active="off")
+ lab.ip(ns, "link", "del", "hsr0")
+ after = link_state(lab, ns, "m")
+ require("master" not in after,
+ label + ": upper remains after detach")
+ require(after["promiscuity"] == before["promiscuity"],
+ label + ": promiscuity changed after detach")
+ # The request was dropped while attached and stays off.
+ gro_state(lab, ns, "m", "off", label + " after detach")
+ set_gro(lab, ns, "m", "on")
+ print(f"# policy {label} cycles=2", flush=True)
+
+
+def master_gso(lab):
+ """The HSR master advertises no GSO types."""
+ ns = lab.ns()
+ for dev in ("a", "b"):
+ local_pair(lab, ns, dev)
+ hsr_attach(lab, ns, "hsr0", "a", "b")
+ listing = lab.features(ns, "hsr0")
+ names = [name for name in listing if
+ ("segmentation" in name or name in
+ ("tx-gso-robust", "tx-gso-partial", "tx-gso-list"))
+ and name != "generic-segmentation-offload"]
+ require(names, "master GSO type mask is absent from feature report")
+ enabled = [name for name in names if listing[name][0] != "off"]
+ require(not enabled, "master retains GSO types: " + ", ".join(enabled))
+ print("# master GSO types off: " + ", ".join(sorted(names)), flush=True)
+
+
+def refused_second_port(lab):
+ """A refused second port (a bridge port) leaves the first port as it was."""
+ ns = lab.ns()
+ for dev in ("a", "b", "c"):
+ local_pair(lab, ns, dev)
+ for dev in ("a", "b"):
+ set_gro(lab, ns, dev, "on")
+ lab.ip(ns, "link", "add", "br0", "type", "bridge")
+ lab.ip(ns, "link", "set", "dev", "b", "master", "br0")
+ lab.ip(ns, "link", "set", "dev", "br0", "up")
+ before = {dev: link_state(lab, ns, dev) for dev in ("a", "b")}
+ result = lab.ip(ns, "link", "add", "hsr0", "type", "hsr",
+ "slave1", "a", "slave2", "b", "version", "1",
+ check=False)
+ require(result.returncode != 0, "second-port attachment unexpectedly passed")
+ require(RX_BUSY in result.stderr,
+ "missing busy RX-handler refusal: " + result.stderr.strip())
+ # B is refused before any state change.
+ refused = link_state(lab, ns, "b")
+ require(refused.get("master") == "br0" and
+ refused["promiscuity"] == before["b"]["promiscuity"],
+ "refused second port changed its bridge membership or promiscuity")
+ gro_state(lab, ns, "b", "on", "refused second port")
+ # Every port is checked before any is changed: A keeps its state.
+ after = link_state(lab, ns, "a")
+ require("master" not in after, "first port A gained an upper")
+ require(after["promiscuity"] == before["a"]["promiscuity"],
+ "first port A changed its promiscuity")
+ gro_state(lab, ns, "a", "on", "first port A")
+ # This positive control detects a residual role or RX handler on A.
+ hsr_attach(lab, ns, "hsr1", "a", "c")
+ lab.ip(ns, "link", "del", "hsr1")
+ gro_state(lab, ns, "a", "off", "attach positive control")
+ set_gro(lab, ns, "a", "on")
+ print("# refused second port left A unchanged; attach of A passed", flush=True)
+
+
+def multicast_flow(lab, ns, phase):
+ """Exact multicast receipt on both lower and macvlan, with no workers."""
+ captures = [lab.capture(ns, dev) for dev in ("m", "mv")]
+ sender = lab.sock(ns, socket.AF_PACKET, socket.SOCK_RAW, 0)
+ sender.bind(("mp", 0))
+ sender.setblocking(False)
+ source = bytes.fromhex(link_state(lab, ns, "mp")["address"].replace(":", ""))
+ prefix = f"hsr-macvlan-{phase}:".encode()
+ frames = [bytes.fromhex("01005e000009") + source + b"\x88\xb5" +
+ prefix + struct.pack("!H", index) + b"m" * 96
+ for index in range(50)]
+ for frame in frames:
+ require(sender.send(frame) == len(frame), "short multicast send")
+ lab.observe(0.002)
+ expected = Counter(frames)
+
+ def copies(capture):
+ return Counter(row["raw"] for row in capture.rows
+ if row["raw"][12:14] == b"\x88\xb5" and
+ row["raw"][14:].startswith(prefix))
+
+ # finish() establishes real empty queues and valid loss/truncation stats.
+ lab.finish(lambda: all(contains(copies(capture), expected) for capture in captures))
+ for dev, capture in zip(("m", "mv"), captures):
+ observed = copies(capture)
+ require(observed == expected,
+ f"{phase} {dev} multicast copies differ: "
+ f"missing={sum((expected - observed).values())} "
+ f"excess={sum((observed - expected).values())}")
+ print(f"# macvlan {phase} multicast lower=50/50 upper=50/50", flush=True)
+
+
+def kind_unavailable(stderr):
+ """True if ip link add failed because the device kind is not available."""
+ error = stderr.lower()
+ return "unknown device type" in error or "operation not supported" in error
+
+
+def rollback_macvlan(lab):
+ """A non-master upper occupies the RX handler; delivery survives."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ result = lab.ip(ns, "link", "add", "link", "m", "name", "mv",
+ "type", "macvlan", "mode", "bridge", check=False)
+ if result.returncode:
+ if kind_unavailable(result.stderr):
+ raise Skip("macvlan device kind unavailable: " + result.stderr.strip())
+ require(False, "macvlan creation failed: " + result.stderr.strip())
+ lab.ip(ns, "link", "set", "dev", "mv", "up")
+ lab.ip(ns, "maddress", "add", "01:00:5e:00:00:09", "dev", "mv")
+ set_gro(lab, ns, "m", "on")
+ before = link_state(lab, ns, "m")
+ require("master" not in before, "macvlan lower already has a master")
+
+ def association():
+ index = lab.cmd(ns, "cat", "/sys/class/net/mv/iflink").stdout.strip()
+ require(index == str(link_state(lab, ns, "m")["ifindex"]),
+ "macvlan no longer belongs to the original lower")
+ membership = lab.ip(ns, "maddress", "show", "dev", "mv").stdout.lower()
+ require("01:00:5e:00:00:09" in membership,
+ "macvlan multicast membership missing")
+
+ association()
+ multicast_flow(lab, ns, "before")
+ result = lab.ip(ns, "link", "add", "hsr0", "type", "hsr",
+ "slave1", "m", "slave2", "p", "version", "1",
+ check=False)
+ require(result.returncode != 0 and RX_BUSY in result.stderr,
+ "expected busy RX-handler refusal: " + result.stderr.strip())
+ after = link_state(lab, ns, "m")
+ require("master" not in after, "failed attachment left an HSR upper")
+ require(after["promiscuity"] == before["promiscuity"],
+ "failed attachment changed promiscuity")
+ # The lower is refused before any state change and keeps its request.
+ gro_state(lab, ns, "m", "on", "refused macvlan lower")
+ association()
+ multicast_flow(lab, ns, "after")
+
+
+def member_down_up(lab):
+ """Close and reopen a member attached with GRO requested."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ set_gro(lab, ns, "m", "on")
+ hsr_attach(lab, ns, "hsr0", "m", "p")
+ gro_state(lab, ns, "m", "off", "attached member")
+ # Each command must return within the bound of Lab.run().
+ lab.ip(ns, "link", "set", "dev", "m", "down")
+ lab.ip(ns, "link", "set", "dev", "m", "up")
+ lab.ip(ns, "link", "del", "hsr0")
+ gro_state(lab, ns, "m", "off", "detached member")
+ set_gro(lab, ns, "m", "on")
+ print("# attached member down/up returned; GRO re-enabled after detach", flush=True)
+
+
+def attach_down_up_detach(lab):
+ """Attach a down member with GRO requested, open it, detach while up."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ lab.ip(ns, "link", "set", "dev", "m", "down")
+ set_gro(lab, ns, "m", "on")
+ hsr_attach(lab, ns, "hsr0", "m", "p")
+ gro_state(lab, ns, "m", "off", "attached down member")
+ lab.ip(ns, "link", "set", "dev", "m", "up")
+ lab.ip(ns, "link", "del", "hsr0")
+ gro_state(lab, ns, "m", "off", "member detached while up")
+ lab.ip(ns, "link", "set", "dev", "m", "down")
+ lab.ip(ns, "link", "set", "dev", "m", "up")
+ set_gro(lab, ns, "m", "on")
+ print("# down member attached, opened, detached up; GRO re-enabled", flush=True)
+
+
+def request_while_attached(lab):
+ """A GRO request on an attached member is dropped, also after detach."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ set_gro(lab, ns, "m", "off")
+ hsr_attach(lab, ns, "hsr0", "m", "p")
+ set_gro(lab, ns, "m", "on", active="off")
+ lab.ip(ns, "link", "del", "hsr0")
+ gro_state(lab, ns, "m", "off", "member after detach")
+ print("# GRO request while attached dropped; stays off after detach", flush=True)
+
+
+PORT_ERROR = "Cannot use a bridge, bond, team or OVS master as HSR port"
+STACKED_ERROR = "Cannot use a macvlan or MACsec device stacked on"
+MACVLAN_GRO = "macvlan inherits GRO from its lower device"
+MASTER_ERROR = "Device already has a master"
+RX_BUSY = "RX handler busy"
+NO_RX_HANDLER = "does not support an RX handler"
+
+
+def port_types(lab):
+ """Forwarding masters, macvlans on them, devices with a master and a
+ macvlan whose lower runs GRO are refused before any port changes; a
+ macvlan on a veth without GRO is accepted."""
+ ns = lab.ns()
+ for dev in ("p", "q", "l", "s"):
+ local_pair(lab, ns, dev)
+ # A macvlan takes the RX handler of its lower: it gets its own bridge.
+ for bridge in ("br0", "br1"):
+ lab.ip(ns, "link", "add", bridge, "type", "bridge")
+ candidates, refused, skipped = [("bridge", "br0", PORT_ERROR)], [], []
+ for kind, dev, lower, args, error in (
+ ("bond", "bond0", None, ("bond0", "type", "bond"), PORT_ERROR),
+ ("macvlan on a bridge", "mvb", None,
+ ("link", "br1", "name", "mvb", "type", "macvlan", "mode", "bridge"), STACKED_ERROR),
+ # Its own bond, as br1 for the macvlan on a bridge.
+ ("macvlan on a bond", "mvo", ("bond1", "type", "bond"),
+ ("link", "bond1", "name", "mvo", "type", "macvlan", "mode", "bridge"), STACKED_ERROR),
+ ("ipvlan", "ipv0", None, ("link", "s", "name", "ipv0", "type", "ipvlan",
+ "mode", "l2"), NO_RX_HANDLER)):
+ result = lab.ip(ns, "link", "add", *lower, check=False) if lower else None
+ if result is None or not result.returncode:
+ result = lab.ip(ns, "link", "add", *args, check=False)
+ if not result.returncode:
+ candidates.append((kind, dev, error))
+ elif kind_unavailable(result.stderr):
+ skipped.append(f"{kind} device kind unavailable")
+ else:
+ require(False, f"{kind} creation failed: {result.stderr.strip()}")
+ before = {dev: link_state(lab, ns, dev)
+ for dev in ("p", "q", *(dev for _, dev, _ in candidates))}
+ for kind, dev, error in candidates:
+ for role, ports in (("slave1", ("slave1", dev, "slave2", "p")),
+ ("slave2", ("slave1", "p", "slave2", dev)),
+ ("interlink", ("slave1", "p", "slave2", "q", "interlink", dev))):
+ label = f"{kind} as {role}"
+ result = lab.ip(ns, "link", "add", "name", "hsr0", "type", "hsr", *ports,
+ "version", "1", check=False)
+ require(result.returncode != 0, label + ": accepted")
+ if role == "interlink" and interlink_unsupported(result.stderr):
+ skipped.append(label + ": iproute2 has no interlink option")
+ continue
+ require(error in result.stderr,
+ f"{label}: missing refusal message: {result.stderr.strip()}")
+ require(lab.ip(ns, "link", "show", "dev", "hsr0", check=False).returncode,
+ label + ": HSR device left behind")
+ for name, state in before.items():
+ after = link_state(lab, ns, name)
+ require("master" not in after and
+ after["promiscuity"] == state["promiscuity"],
+ f"{label}: {name} changed by the refused attach")
+ refused.append(label)
+ # A refused second port leaves the first port as it was.
+ for dev in ("w", "g"):
+ local_pair(lab, ns, dev)
+ set_gro(lab, ns, "p", "on")
+ set_gro(lab, ns, "g", "on")
+ first, second = link_state(lab, ns, "p"), []
+ for kind, dev, args, error, setup in (
+ ("VRF port", "w", ("vrf0", "type", "vrf", "table", "10"), MASTER_ERROR,
+ ("link", "set", "dev", "w", "master", "vrf0")),
+ ("macvlan whose lower runs GRO", "mvg",
+ ("link", "g", "name", "mvg", "type", "macvlan", "mode", "bridge"), MACVLAN_GRO,
+ None)):
+ kind_name = "vrf" if dev == "w" else "macvlan"
+ result = lab.ip(ns, "link", "add", *args, check=False)
+ if not result.returncode:
+ if setup:
+ lab.ip(ns, *setup)
+ second.append((kind, dev, error))
+ elif kind_unavailable(result.stderr):
+ skipped.append(f"{kind_name} device kind unavailable")
+ else:
+ require(False, f"{kind_name} creation failed: {result.stderr.strip()}")
+ for kind, dev, error in second:
+ label = f"{kind} as slave2"
+ result = lab.ip(ns, "link", "add", "name", "hsr0", "type", "hsr",
+ "slave1", "p", "slave2", dev, "version", "1", check=False)
+ require(result.returncode != 0 and error in result.stderr,
+ f"{label}: missing refusal message: {result.stderr.strip()}")
+ require(lab.ip(ns, "link", "show", "dev", "hsr0", check=False).returncode,
+ label + ": HSR device left behind")
+ after = link_state(lab, ns, "p")
+ require("master" not in after and after["promiscuity"] == first["promiscuity"],
+ f"{label}: first port p changed by the refused attach")
+ gro_state(lab, ns, "p", "on", label + ": first port p")
+ refused.append(label)
+ print("# refused: " + ", ".join(refused), flush=True)
+ # A macvlan on a veth without GRO is a valid port (positive control).
+ set_gro(lab, ns, "l", "off")
+ result = lab.ip(ns, "link", "add", "link", "l", "name", "mv0", "type", "macvlan",
+ "mode", "bridge", check=False)
+ if not result.returncode:
+ lab.ip(ns, "link", "set", "dev", "mv0", "up")
+ hsr_attach(lab, ns, "hsr0", "mv0", "p")
+ lab.ip(ns, "link", "del", "hsr0")
+ print("# accepted: macvlan on veth as slave1", flush=True)
+ elif kind_unavailable(result.stderr):
+ skipped.append("macvlan device kind unavailable")
+ else:
+ require(False, "macvlan creation failed: " + result.stderr.strip())
+ if skipped:
+ raise Skip("; ".join(skipped) + "; refused: " + ", ".join(refused))
+
+
+GSO_ROWS = (
+ ("plain", (), False, "A"),
+ ("s-tag-inband", ((0x88a8, 100),), True, "A"),
+ ("c-tag-accelerated", ((0x8100, 50),), False, "A"),
+ ("s-tag-accelerated", ((0x88a8, 100),), False, "B"),
+ ("sc-nested-accelerated", ((0x88a8, 200), (0x8100, 30)), False, "B"),
+ ("cc-nested-inband", ((0x8100, 40), (0x8100, 20)), True, "A"),
+)
+
+
+def vlan_chain(lab, ns, parent, stack, prefix):
+ """Stack VLAN devices on parent; return the innermost one."""
+ for depth, (protocol, identifier) in enumerate(stack):
+ dev = f"{prefix}{depth}"
+ lab.ip(ns, "link", "add", "link", parent, "name", dev,
+ "type", "vlan", "id", str(identifier), "protocol",
+ "802.1AD" if protocol == 0x88a8 else "802.1Q")
+ lab.ip(ns, "link", "set", "dev", dev, "up")
+ parent = dev
+ return parent
+
+
+def vlan_tx_mode(lab, ns, lower, stack, inband):
+ """Select accelerated or in-band insertion of the outer VLAN tag."""
+ if not stack:
+ return
+ aliases = (("tx-vlan-stag-hw-insert", "tx-vlan-stag-offload")
+ if stack[0][0] == 0x88a8 else
+ ("tx-vlan-offload", "tx-vlan-hw-insert"))
+ listing = lab.features(ns, lower)
+ name = next((alias for alias in aliases if alias in listing), None)
+ if name is None:
+ raise Skip("no VLAN insertion feature: " + ", ".join(aliases))
+ wanted = "off" if inband else "on"
+ if listing[name][1] and listing[name][0] != wanted:
+ raise Skip("fixed VLAN insertion feature cannot be " + wanted)
+ lab.cmd(ns, "ethtool", "-K", lower, name, wanted)
+ require(lab.features(ns, lower)[name][0] == wanted,
+ "VLAN insertion mode did not match its requested state")
+
+
+def ordered_payload(producer, index, size=64):
+ """Payload naming its producer and index."""
+ prefix = b"HSRORD" + bytes([producer]) + struct.pack("!I", index)
+ return prefix + bytes([producer ^ 0xA5]) * (size - len(prefix))
+
+
+def ordered_socket(lab, ns, address):
+ """UDP multicast sender bound to address."""
+ sock = lab.sock(ns, socket.AF_INET, socket.SOCK_DGRAM)
+ sock.bind((address, 0))
+ sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 1)
+ sock.settimeout(1)
+ return sock
+
+
+def ordered_lanes(lab, topology):
+ """Capture the frames the DUT transmits on its two LAN ports."""
+ return [lab.capture(topology["dut"], lane, pkttype=4)
+ for lane in ("da", "db")]
+
+
+def ordered_send(lab, sock, producer, first, count, port, segment=0):
+ """Send count datagrams, or UDP GSO aggregates of 16; return the payloads."""
+ expected = []
+ if segment:
+ sock.setsockopt(socket.IPPROTO_UDP, 103, segment)
+ blocks = 16 if segment else 1
+ size = segment or 64
+ for number in range(count):
+ payloads = [ordered_payload(producer, first + number * blocks + n,
+ size) for n in range(blocks)]
+ body = b"".join(payloads)
+ require(sock.sendto(body, ("239.0.0.9", port)) == len(body),
+ "short UDP send")
+ expected.extend(payloads)
+ # Keep capture receive queues bounded throughout long wrap warmups.
+ if number % 8 == 0:
+ lab.observe(0)
+ lab.observe(0.1)
+ return expected
+
+
+def udp_rows(cap, ports):
+ """Captured UDP frames addressed to one of ports."""
+ return [row for row in cap.rows if row["ports"] and row["ports"][1] in ports]
+
+
+def lanes_hold(caps, expected, ports):
+ """Completion condition: every lane captured each expected payload."""
+ wanted = Counter(expected)
+ return lambda: all(contains(Counter(row["payload"] for row in udp_rows(cap, ports)), wanted)
+ for cap in caps)
+
+
+def ordered_data(lab, gso=False):
+ """Local data around interlink data or aggregates keeps its order."""
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ local = ordered_socket(lab, topology["dut"], "100.64.3.1")
+ san = ordered_socket(lab, topology["tx"], "100.64.2.1")
+ # Observe interlink aggregates before the two tagged LAN copies expand.
+ sent = lab.capture(topology["tx"], "t0", pkttype=4)
+ ingress = lab.capture(topology["dut"], "dil")
+ expected = ordered_send(lab, local, 1, 0, 32, 5100)
+ expected += ordered_send(lab, san, 2, 0, 16, 5101, segment=512 if gso else 0)
+ expected += ordered_send(lab, local, 1, 32, 32, 5100)
+ lanes_done = lanes_hold(caps, expected, {5100, 5101})
+ lab.finish(lambda: lanes_done() and
+ len(udp_rows(ingress, {5101})) >= len(udp_rows(sent, {5101})) > 0)
+ verify_udp("local and aggregate boundaries", caps, expected,
+ ports={5100, 5101}, input_order=True)
+ inputs = udp_rows(sent, {5101})
+ if gso:
+ expected_inputs = [b"".join(ordered_payload(2, n * 16 + part, 512)
+ for part in range(16)) for n in range(16)]
+ else:
+ expected_inputs = expected[32:48]
+ aggregated = any(len(row["payload"]) > 1500 for row in inputs)
+ if gso and not aggregated:
+ expected_inputs = expected[32:-32]
+ require([row["payload"] for row in inputs] == expected_inputs,
+ "UDP sender input set/order differs")
+ incoming = udp_rows(ingress, {5101})
+ require([row["payload"] for row in incoming] == expected_inputs,
+ "actual UDP ingress input set/order differs")
+ require(all(row["encoding"] == "plain" and not row["vlans"] for row in inputs + incoming),
+ "UDP input classification/VLAN differs")
+ if gso and not aggregated:
+ raise Skip("interlink UDP GSO aggregate not observed")
+
+
+def ordered_supervision(lab):
+ """Local and proxy supervision frames are numbered with the data."""
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ san = ordered_socket(lab, topology["tx"], "100.64.2.1")
+ expected = ordered_send(lab, san, 3, 0, 8, 5110)
+ # Both timers have time for multiple life checks; no fake worker state.
+ lab.observe(10)
+ data_done = lanes_hold(caps, expected, {5110})
+ macs = (topology["mac_master"], topology["mac_tx"])
+ lab.finish(lambda: data_done() and all(
+ sum(row["supervision"] == mac for row in cap.rows) >= 2
+ for cap in caps for mac in macs))
+ verify_udp("proxy-learning inputs", caps, expected, ports={5110},
+ input_order=True)
+ verify_supervision(caps, topology["mac_master"], topology["mac_tx"], {5110})
+
+
+def ordered_concurrent(lab):
+ """Master and interlink producers on two CPUs keep per-producer order."""
+ cpus = sorted(os.sched_getaffinity(0))[:2]
+ if len(cpus) != 2:
+ raise Skip("two functional producers need two allowed CPUs")
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ sockets = [ordered_socket(lab, topology["tx"], "100.64.2.1"),
+ ordered_socket(lab, topology["dut"], "100.64.3.1")]
+ barrier = threading.Barrier(3)
+ stop = threading.Event()
+ errors = []
+ windows = [(), ()]
+ until = min(lab.deadline, time.monotonic() + 15)
+
+ def producer(index):
+ try:
+ # Socket namespace binding was completed before threads existed.
+ os.sched_setaffinity(0, {cpus[index]})
+ barrier.wait(timeout=2)
+ start = time.monotonic()
+ for number in range(128):
+ require(not stop.is_set() and not lab.stop.is_set() and
+ time.monotonic() < until, "producer work deadline")
+ body = ordered_payload(index + 1, number)
+ require(sockets[index].sendto(body, ("239.0.0.9", 5120 + index)) == len(body),
+ "short producer send")
+ time.sleep(0.001)
+ windows[index] = (start, time.monotonic())
+ except (OSError, ValueError, RuntimeError) as error:
+ errors.append(error)
+ stop.set()
+ barrier.abort()
+
+ threads = [threading.Thread(target=producer, args=(index,))
+ for index in range(2)]
+ unfinished = []
+ try:
+ for thread in threads:
+ lab.threads.append(thread)
+ thread.start()
+ try:
+ barrier.wait(timeout=2)
+ except threading.BrokenBarrierError:
+ # A producer that failed broke the barrier: report its error.
+ if errors:
+ raise errors[0] from None
+ raise
+ while any(thread.is_alive() for thread in threads):
+ require(time.monotonic() < until, "producer join deadline")
+ lab.observe(0.01)
+ finally:
+ stop.set()
+ barrier.abort()
+ join_end = min(lab.deadline, time.monotonic() + 2)
+ for thread in threads:
+ if thread.ident is not None:
+ thread.join(max(0, join_end - time.monotonic()))
+ unfinished = [thread for thread in threads if thread.is_alive()]
+ if unfinished:
+ print("# cleanup failure: sender thread remained alive")
+ if errors:
+ raise errors[0]
+ require(not unfinished, "sender thread remained alive")
+ require(all(windows) and windows[0][0] < windows[1][1] and
+ windows[1][0] < windows[0][1], "producer send windows do not overlap")
+ print(f"# functional producers CPUs={cpus} windows={windows}")
+ expected = [ordered_payload(producer, number)
+ for producer in (1, 2) for number in range(128)]
+ lab.finish(lanes_hold(caps, expected, {5120, 5121}))
+ verify_udp("concurrent master/interlink inputs", caps, expected,
+ ports={5120, 5121}, producer_order=True)
+
+
+def ordered_wrap(lab):
+ """The interlink sequence number wraps with both lanes intact."""
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ sender = ordered_socket(lab, topology["tx"], "100.64.2.1")
+ probe = ordered_send(lab, sender, 4, 0, 1, 5130)
+ # Wait until both lanes hold the probe; a missing copy still fails below.
+ end = time.monotonic() + 5
+ while not all(udp_rows(cap, {5130}) for cap in caps) and time.monotonic() < end:
+ lab.observe(0.01)
+ # This only selects the warmup count; final loss-free captures certify it.
+ rows = [udp_rows(cap, {5130}) for cap in caps]
+ require(all(len(lane) == 1 for lane in rows) and
+ rows[0][0]["seq"] == rows[1][0]["seq"], "wrap probe missing or unequal")
+ start = rows[0][0]["seq"]
+ warm_count = (65400 - ((start + 1) & 0xFFFF)) & 0xFFFF
+ warm = ordered_send(lab, sender, 5, 0, warm_count, 5131)
+ # Separate IDs/ports prevent sequence-space reuse from hiding a loss.
+ expected = ordered_send(lab, sender, 6, 0, 256, 5132)
+ lab.finish(lanes_hold(caps, probe + warm + expected, {5130, 5131, 5132}))
+ verify_udp("wrap start", caps, probe, ports={5130}, input_order=True)
+ if warm:
+ verify_udp("wrap warmup", caps, warm, ports={5131}, input_order=True)
+ verify_udp("actual protocol wrap", caps, expected, ports={5132},
+ expect_wrap=True, input_order=True)
+
+
+ORDERED_CASES = (("local-data-order", ordered_data),
+ ("local-data-aggregate-boundaries", lambda lab: ordered_data(lab, True)),
+ ("local-proxy-supervision", ordered_supervision),
+ ("two-functional-producers", ordered_concurrent),
+ ("actual-sequence-wrap", ordered_wrap))
+
+
+def topo(lab):
+ """SAN - interlink - HSR DUT - two LANs - receiver namespaces."""
+ tx, dut, rx = lab.ns(), lab.ns(), lab.ns()
+ for ns, dev, peer_ns, peer in ((tx, "t0", dut, "dil"),
+ (dut, "da", rx, "ra"), (dut, "db", rx, "rb")):
+ lab.veth(ns, dev, peer_ns, peer)
+ for owner, link in ((ns, dev), (peer_ns, peer)):
+ lab.ip(owner, "link", "set", "dev", link, "up")
+ hsr_attach(lab, dut, "hm", "da", "db", "dil")
+ attributes = link_state(lab, dut, "hm")["linkinfo"]["info_data"]
+ # Some iproute2 versions do not print the HSR version; check it if present.
+ require(attributes.get("proto") == 0 and attributes.get("version", 1) == 1 and
+ all(attributes.get(key) == value for key, value in
+ (("slave1", "da"), ("slave2", "db"), ("interlink", "dil"))),
+ "HSR topology has unexpected protocol/version")
+ lab.ip(tx, "addr", "add", "100.64.2.1/24", "dev", "t0")
+ lab.ip(dut, "addr", "add", "100.64.3.1/24", "dev", "hm")
+ for ns, dev in ((tx, "t0"), (dut, "hm")):
+ lab.ip(ns, "route", "add", "239.0.0.0/8", "dev", dev)
+ return {"tx": tx, "dut": dut, "rx": rx, "master": "hm",
+ "mac_master": bytes.fromhex(link_state(lab, dut, "hm")["address"].replace(":", "")),
+ "mac_tx": bytes.fromhex(link_state(lab, tx, "t0")["address"].replace(":", ""))}
+
+
+def drop_capture_head(lanes, what):
+ """Drop up to two leading supervision frames that only one lane captured."""
+ for this, other in ((0, 1), (1, 0)):
+ if lanes[this] and lanes[other]:
+ ids = [copy_identity(row) for row in lanes[this][:3]]
+ first = copy_identity(lanes[other][0])
+ if first in ids:
+ head = ids.index(first)
+ require(all(row["supervision"] is not None for row in lanes[this][:head]),
+ what + " LAN copies differ before the capture start")
+ lanes[this] = lanes[this][head:]
+
+
+def verify_supervision(captures, master, proxy, ports):
+ """Check local and proxy supervision TLVs and their order on both lanes."""
+ tagged, observed = [], []
+ for lane, cap in enumerate(captures):
+ cap.validate()
+ frames = [row for row in cap.rows if row["encoding"] == "hsr"]
+ protocol_order(frames)
+ # Only this case's frames are compared between the lanes; the
+ # namespaces also send IPv6 neighbor discovery and MLD frames.
+ tagged.append([row for row in frames if row["supervision"] is not None or
+ (row["ports"] and row["ports"][1] in ports)])
+ rows = [row for row in cap.rows if row["supervision"] is not None]
+ require(all(row["encoding"] == "hsr" and row["path"] == lane for row in rows),
+ "wrong supervision tag kind/path")
+ require(all(row["supervision"] in (master, proxy) for row in rows),
+ "unexpected supervised MAC")
+ for mac in (master, proxy):
+ group = [row for row in rows if row["supervision"] == mac]
+ require(len(group) >= 2, "missing repeated local/proxy supervision TLV")
+ protocol_order(group, key="supseq")
+ protocol_order(rows)
+ observed.append(rows)
+ # The two lanes are captured and cut off one after the other while
+ # supervision goes on: a lane may start and end in extra supervision;
+ # the rest must match.
+ for lanes, what in ((observed, "supervision"), (tagged, "data/supervision")):
+ drop_capture_head(lanes, what)
+ common = min(len(rows) for rows in lanes)
+ require([copy_identity(row) for row in lanes[0][:common]] ==
+ [copy_identity(row) for row in lanes[1][:common]],
+ what + " LAN order differs")
+ # The lanes are cut off one synchronize_net() apart. In that window
+ # the two supervision sources (the local node and the proxied SAN),
+ # each sending every two seconds, add at most one frame each.
+ tail = [row for rows in lanes for row in rows[common:]]
+ require(len(tail) <= 2 and all(row["supervision"] is not None for row in tail),
+ what + " LAN copies differ beyond the capture cutoff")
+ print(f"# supervision exact local/proxy TLV sets lanes={[len(rows) for rows in observed]}",
+ flush=True)
+
+
+def tcp_transfer(lab, sender_ns, receiver_ns, dev, source, destination, port):
+ """Send a stream and a reply; the caller completes its captures."""
+ payload = bytes(range(256)) * 512
+ reply = b"HSR-TCP-REPLY"
+ listener = lab.sock(receiver_ns, socket.AF_INET, socket.SOCK_STREAM)
+ listener.bind((destination, port))
+ listener.listen(1)
+ client = lab.sock(sender_ns, socket.AF_INET, socket.SOCK_STREAM)
+ client.setsockopt(socket.SOL_SOCKET, socket.SO_BINDTODEVICE, dev.encode() + b"\0")
+ client.bind((source, 0))
+ client.connect((destination, port))
+ server, _ = listener.accept()
+ lab.sockets.append(server)
+ for sock in (client, server):
+ sock.setblocking(False)
+ sent, received, echoed = 0, bytearray(), bytearray()
+ state = {"reply": False, "write_closed": False}
+ end = min(lab.deadline, time.monotonic() + 12)
+
+ def pump():
+ nonlocal sent
+ if sent < len(payload):
+ try:
+ sent += client.send(payload[sent:sent + 65536])
+ except BlockingIOError:
+ pass
+ elif not state["write_closed"]:
+ client.shutdown(socket.SHUT_WR)
+ state["write_closed"] = True
+ try:
+ data = server.recv(65536)
+ received.extend(data)
+ except BlockingIOError:
+ pass
+ if len(received) == len(payload) and not state["reply"]:
+ require(server.send(reply) == len(reply), "short TCP reply")
+ state["reply"] = True
+ try:
+ echoed.extend(client.recv(65536))
+ except BlockingIOError:
+ pass
+
+ while len(echoed) < len(reply) and time.monotonic() < end:
+ lab.observe(0.01, pump)
+ require(bytes(received) == payload and bytes(echoed) == reply,
+ "TCP application did not receive complete expected bytes")
+ for sock in (server, client, listener):
+ sock.close()
+ return payload, reply
+
+
+def fin_seen(caps, port):
+ """Completion condition: each capture saw a FIN of the connection."""
+ return lambda: all(any(port in row["ports"] and row.get("tcpflags", 0) & 1
+ for row in cap.rows) for cap in caps)
+
+
+def tcp_interlink(lab):
+ """A natural TCP stream from the interlink reaches both LANs intact."""
+ topology = topo(lab)
+ tx, dut, rx = (topology[key] for key in ("tx", "dut", "rx"))
+ lab.ip(rx, "link", "add", "hr", "type", "hsr",
+ "slave1", "ra", "slave2", "rb", "version", "1")
+ lab.ip(rx, "link", "set", "dev", "hr", "up")
+ lab.ip(tx, "addr", "add", "10.67.0.2/24", "dev", "t0")
+ lab.ip(rx, "addr", "add", "10.67.0.3/24", "dev", "hr")
+ lab.cmd(tx, "ethtool", "-K", "t0", "tso", "on", "gso", "on")
+ caps = [lab.capture(tx, "t0", pkttype=4)] + ordered_lanes(lab, topology)
+ ingress = lab.capture(dut, "dil")
+ payload, _ = tcp_transfer(lab, tx, rx, "t0", "10.67.0.2", "10.67.0.3", 5200)
+ lab.finish(fin_seen(caps + [ingress], 5200))
+ verify_stream(caps, 5200, payload, ingress=ingress)
+
+
+def tcp_prp(lab, row):
+ """TCP GSO from a SAN on one PRP LAN is delivered locally, segmented."""
+ name, stack, inband, side = row
+ tx, dut = lab.ns(), lab.ns()
+ for lane in ("a", "b"):
+ lab.veth(tx, "t" + lane, dut, "d" + lane)
+ lab.ip(tx, "link", "set", "dev", "t" + lane, "up")
+ lab.ip(dut, "link", "set", "dev", "d" + lane, "up")
+ lab.ip(dut, "link", "add", "pm", "type", "hsr", "slave1", "da", "slave2", "db", "proto", "1")
+ lab.ip(dut, "link", "set", "dev", "pm", "up")
+ attributes = link_state(lab, dut, "pm")["linkinfo"]["info_data"]
+ require(attributes.get("proto") == 1 and
+ attributes.get("slave1") == "da" and attributes.get("slave2") == "db",
+ "PRP topology has unexpected protocol/ports")
+ lower = "t" + side.lower()
+ sender = vlan_chain(lab, tx, lower, stack, "txv")
+ receiver = vlan_chain(lab, dut, "pm", stack, "rxv")
+ lab.ip(tx, "addr", "add", "10.68.0.2/24", "dev", sender)
+ lab.ip(dut, "addr", "add", "10.68.0.1/24", "dev", receiver)
+ vlan_tx_mode(lab, tx, lower, stack, inband)
+ lab.cmd(tx, "ethtool", "-K", lower, "tso", "on", "gso", "on")
+ caps = [lab.capture(tx, lower, pkttype=4), lab.capture(dut, "pm", pkttype=0)]
+ ingress = lab.capture(dut, "d" + side.lower(), prp=True)
+ lanes = [lab.capture(dut, dev, pkttype=4, prp=True) for dev in ("da", "db")]
+ payload, reply = tcp_transfer(lab, tx, dut, sender, "10.68.0.2", "10.68.0.1", 5201)
+ lab.finish(fin_seen(caps + lanes + [ingress], 5201))
+ verify_stream(caps, 5201, payload, stack, local=True, inband=inband, ingress=ingress)
+ copies = []
+ for index, cap in enumerate(lanes):
+ cap.validate()
+ rows = [frame for frame in cap.rows if frame["ports"] and
+ frame["ports"][0] == 5201 and frame["payload"]]
+ require(len(rows) == 1 and rows[0]["payload"] == reply,
+ "PRP reply not exactly once per LAN")
+ require(rows[0]["tag"] and rows[0]["vlans"] == stack and
+ rows[0]["size"] <= 1520 + 4 * len(stack), "invalid PRP reply tag/size/VLAN")
+ require(rows[0]["path"] == 10 + index, "wrong PRP LAN identifier")
+ require(rows[0]["encoding"] == "prp", "wrong PRP reply tag kind")
+ copies.append(copy_identity(rows[0]))
+ require(copies[0] == copies[1], "PRP reply LAN copies differ")
+ print(f"# PRP LAN {name} local data segmented; reply each LAN=1/1", flush=True)
+
+
+def interrupt(signum, _frame):
+ """INT/TERM handler: record the signal for Lab.remaining()."""
+ SIGNALLED.append(signum)
+
+
+def main():
+ """Run the gro or ordered cases as TAP."""
+ mode = sys.argv[1] if len(sys.argv) == 2 else ""
+ require(mode in ("gro", "ordered"), "usage: hsr_gro_superpacket.py gro|ordered")
+ cases = [("GRO policy", policy), ("master GSO mask", master_gso),
+ ("refused second port", refused_second_port),
+ ("busy-handler rollback", rollback_macvlan),
+ ("member down/up while attached", member_down_up),
+ ("attach down, up, detach while up", attach_down_up_detach),
+ ("GRO request while attached", request_while_attached),
+ ("port type rejection", port_types),
+ ("natural TCP interlink", tcp_interlink)]
+ cases += [("PRP LAN " + row[0], lambda lab, row=row: tcp_prp(lab, row)) for row in GSO_ROWS]
+ if mode == "ordered":
+ cases = ORDERED_CASES
+ print(f"TAP version 13\n1..{len(cases)}", flush=True)
+ missing = [tool for tool in ("ip", "ethtool") if shutil.which(tool) is None]
+ if os.geteuid() or missing:
+ for number, (name, _) in enumerate(cases, 1):
+ print(f"ok {number} - {name} # SKIP need root/ip/ethtool {missing}", flush=True)
+ return 4
+ with signals_blocked():
+ for sig in (signal.SIGINT, signal.SIGTERM):
+ signal.signal(sig, interrupt)
+ lab = Lab(time.monotonic() + 120)
+ failure, passed = None, 0
+ try:
+ for number, (name, function) in enumerate(cases, 1):
+ try:
+ lab.begin()
+ function(lab)
+ except Skip as error:
+ print(f"ok {number} - {name} # SKIP {error}", flush=True)
+ continue
+ except CASE_ERRORS as error:
+ failure = error
+ print(f"not ok {number} - {name}: {error}", flush=True)
+ for later, (other, _) in enumerate(cases[number:], number + 1):
+ print(f"not ok {later} - {other}: stopped after prior failure", flush=True)
+ break
+ print(f"ok {number} - {name}", flush=True)
+ passed += 1
+ finally:
+ with signals_blocked():
+ for sig in (signal.SIGINT, signal.SIGTERM):
+ signal.signal(sig, signal.SIG_IGN)
+ try:
+ lab.close()
+ except CASE_ERRORS as error:
+ print("# cleanup failure: " + str(error), flush=True)
+ failure = failure or error
+ return 1 if failure is not None else (0 if passed else 4)
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
new file mode 100755
index 000000000000..9da711ad2ab2
--- /dev/null
+++ b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
@@ -0,0 +1,9 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+if ! command -v python3 > /dev/null; then
+ echo "SKIP: missing python3"
+ exit 4
+fi
+
+exec python3 -B "$(dirname "$0")/hsr_gro_superpacket.py" gro "$@"
diff --git a/tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh b/tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
new file mode 100755
index 000000000000..b43218709455
--- /dev/null
+++ b/tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
@@ -0,0 +1,9 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+if ! command -v python3 > /dev/null; then
+ echo "SKIP: missing python3"
+ exit 4
+fi
+
+exec python3 -B "$(dirname "$0")/hsr_gro_superpacket.py" ordered "$@"
--
2.53.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-12 15:33 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-12 15:33 [PATCH net v9 0/4] net: hsr: fix super-packet forwarding Xin Xie
2026-11-12 15:33 ` [PATCH net v9 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
2026-11-12 15:33 ` [PATCH net v9 2/4] net: hsr: refuse forwarding devices as " Xin Xie
2026-11-12 15:33 ` [PATCH net v9 3/4] net: hsr: segment GSO before per-frame forwarding Xin Xie
2026-11-12 15:33 ` [PATCH net v9 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding Xin Xie
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox