[PATCH v3 nf-next 0/3] flow offload teardown when layer 2 roaming

[PATCH v3 nf-next 0/3] flow offload teardown when layer 2 roaming

[Eric Woudstra]

This patch-set can be reviewed separately from my submissions concerning
the bridge-fastpath.

In case of a bridge in the forward-fastpath or bridge-fastpath the fdb is
used to create the tuple. In case of roaming at layer 2 level, for example
802.11r, the destination device is changed in the fdb. The destination
device of a direct transmitting tuple is no longer valid and traffic is
send to the wrong destination. Also the hardware offloaded fastpath is not
valid anymore.

This flowentry needs to be torn down asap. Also make sure that the flow
entry is not being used, when marked for teardown.

Changes in v3:
- static nf_flow_table_switchdev_nb.

Changes in v2:
- Unchanged, only tags RFC net-next to PATCH nf-next.

Eric Woudstra (3):
  netfilter: flow: Add bridge_vid member
  netfilter: nf_flow_table_core: teardown direct xmit when destination
    changed
  netfilter: nf_flow_table_ip: don't follow fastpath when marked
    teardown

 include/net/netfilter/nf_flow_table.h |  2 +
 net/netfilter/nf_flow_table_core.c    | 66 +++++++++++++++++++++++++++
 net/netfilter/nf_flow_table_ip.c      |  6 +++
 net/netfilter/nft_flow_offload.c      |  3 ++
 4 files changed, 77 insertions(+)

-- 
2.47.1

[PATCH v3 nf-next 1/3] netfilter: flow: Add bridge_vid member

[Eric Woudstra]

Store the vid used on the bridge in the flow_offload_tuple, so it can be
used later to identify fdb entries that relate to the tuple.

The bridge_vid member is added to the structures nft_forward_info,
nf_flow_route and flow_offload_tuple. It can now be passed from
net_device_path->bridge.vlan_id to flow_offload_tuple->out.bridge_vid.

Signed-off-by: Eric Woudstra <ericwouds@gmail.com>
---
 include/net/netfilter/nf_flow_table.h | 2 ++
 net/netfilter/nf_flow_table_core.c    | 1 +
 net/netfilter/nft_flow_offload.c      | 3 +++
 3 files changed, 6 insertions(+)

diff --git a/include/net/netfilter/nf_flow_table.h b/include/net/netfilter/nf_flow_table.h
index d711642e78b5..9d9363e91587 100644
--- a/include/net/netfilter/nf_flow_table.h
+++ b/include/net/netfilter/nf_flow_table.h
@@ -146,6 +146,7 @@ struct flow_offload_tuple {
 		struct {
 			u32		ifidx;
 			u32		hw_ifidx;
+			u16		bridge_vid;
 			u8		h_source[ETH_ALEN];
 			u8		h_dest[ETH_ALEN];
 		} out;
@@ -212,6 +213,7 @@ struct nf_flow_route {
 		struct {
 			u32			ifindex;
 			u32			hw_ifindex;
+			u16			bridge_vid;
 			u8			h_source[ETH_ALEN];
 			u8			h_dest[ETH_ALEN];
 		} out;
diff --git a/net/netfilter/nf_flow_table_core.c b/net/netfilter/nf_flow_table_core.c
index 9441ac3d8c1a..992958db4a19 100644
--- a/net/netfilter/nf_flow_table_core.c
+++ b/net/netfilter/nf_flow_table_core.c
@@ -128,6 +128,7 @@ static int flow_offload_fill_route(struct flow_offload *flow,
 		       ETH_ALEN);
 		flow_tuple->out.ifidx = route->tuple[dir].out.ifindex;
 		flow_tuple->out.hw_ifidx = route->tuple[dir].out.hw_ifindex;
+		flow_tuple->out.bridge_vid = route->tuple[dir].out.bridge_vid;
 		dst_release(dst);
 		break;
 	case FLOW_OFFLOAD_XMIT_XFRM:
diff --git a/net/netfilter/nft_flow_offload.c b/net/netfilter/nft_flow_offload.c
index fdf927a8252d..31372a8ef37e 100644
--- a/net/netfilter/nft_flow_offload.c
+++ b/net/netfilter/nft_flow_offload.c
@@ -85,6 +85,7 @@ struct nft_forward_info {
 		__u16	id;
 		__be16	proto;
 	} encap[NF_FLOW_TABLE_ENCAP_MAX];
+	u16 bridge_vid;
 	u8 num_encaps;
 	u8 ingress_vlans;
 	u8 h_source[ETH_ALEN];
@@ -159,6 +160,7 @@ static void nft_dev_path_info(const struct net_device_path_stack *stack,
 			case DEV_PATH_BR_VLAN_KEEP:
 				break;
 			}
+			info->bridge_vid = path->bridge.vlan_id;
 			info->xmit_type = FLOW_OFFLOAD_XMIT_DIRECT;
 			break;
 		default:
@@ -223,6 +225,7 @@ static void nft_dev_forward_path(struct nf_flow_route *route,
 		memcpy(route->tuple[dir].out.h_dest, info.h_dest, ETH_ALEN);
 		route->tuple[dir].out.ifindex = info.outdev->ifindex;
 		route->tuple[dir].out.hw_ifindex = info.hw_outdev->ifindex;
+		route->tuple[dir].out.bridge_vid = info.bridge_vid;
 		route->tuple[dir].xmit_type = info.xmit_type;
 	}
 }
-- 
2.47.1

[PATCH v3 nf-next 2/3] netfilter: nf_flow_table_core: teardown direct xmit when destination changed

[Eric Woudstra]

In case of a bridge in the forward-fastpath or bridge-fastpath the fdb is
used to create the tuple. In case of roaming at layer 2 level, for example
802.11r, the destination device is changed in the fdb. The destination
device of a direct transmitting tuple is no longer valid and traffic is
send to the wrong destination. Also the hardware offloaded fastpath is not
valid anymore.

In case of roaming, a switchdev notification is send to delete the old fdb
entry. Upon receiving this notification, mark all direct transmitting flows
with the same ifindex, vid and hardware address as the fdb entry to be
teared down. The hardware offloaded fastpath is still in effect, so
minimize the delay of the work queue by setting the delay to zero.

Signed-off-by: Eric Woudstra <ericwouds@gmail.com>
---
 net/netfilter/nf_flow_table_core.c | 65 ++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/net/netfilter/nf_flow_table_core.c b/net/netfilter/nf_flow_table_core.c
index 992958db4a19..61f03907d103 100644
--- a/net/netfilter/nf_flow_table_core.c
+++ b/net/netfilter/nf_flow_table_core.c
@@ -13,6 +13,7 @@
 #include <net/netfilter/nf_conntrack_core.h>
 #include <net/netfilter/nf_conntrack_l4proto.h>
 #include <net/netfilter/nf_conntrack_tuple.h>
+#include <net/switchdev.h>
 
 static DEFINE_MUTEX(flowtable_lock);
 static LIST_HEAD(flowtables);
@@ -745,6 +746,63 @@ void nf_flow_table_cleanup(struct net_device *dev)
 }
 EXPORT_SYMBOL_GPL(nf_flow_table_cleanup);
 
+struct flow_cleanup_data {
+	const unsigned char *addr;
+	int ifindex;
+	u16 vid;
+	bool found;
+};
+
+static void nf_flow_table_do_cleanup_addr(struct nf_flowtable *flow_table,
+					  struct flow_offload *flow, void *data)
+{
+	struct flow_cleanup_data *cud = data;
+
+	if ((flow->tuplehash[0].tuple.xmit_type == FLOW_OFFLOAD_XMIT_DIRECT &&
+	     flow->tuplehash[0].tuple.out.ifidx == cud->ifindex &&
+	     flow->tuplehash[0].tuple.out.bridge_vid == cud->vid &&
+	     ether_addr_equal(flow->tuplehash[0].tuple.out.h_dest, cud->addr)) ||
+	    (flow->tuplehash[1].tuple.xmit_type == FLOW_OFFLOAD_XMIT_DIRECT &&
+	     flow->tuplehash[1].tuple.out.ifidx == cud->ifindex &&
+	     flow->tuplehash[1].tuple.out.bridge_vid == cud->vid &&
+	     ether_addr_equal(flow->tuplehash[1].tuple.out.h_dest, cud->addr))) {
+		flow_offload_teardown(flow);
+		cud->found = true;
+	}
+}
+
+static int nf_flow_table_switchdev_event(struct notifier_block *unused,
+					 unsigned long event, void *ptr)
+{
+	struct switchdev_notifier_fdb_info *fdb_info;
+	struct nf_flowtable *flowtable;
+	struct flow_cleanup_data cud;
+
+	if (event != SWITCHDEV_FDB_DEL_TO_DEVICE)
+		return NOTIFY_DONE;
+
+	fdb_info = ptr;
+	cud.addr = fdb_info->addr;
+	cud.vid = fdb_info->vid;
+	cud.ifindex = fdb_info->info.dev->ifindex;
+
+	mutex_lock(&flowtable_lock);
+	list_for_each_entry(flowtable, &flowtables, list) {
+		cud.found = false;
+		nf_flow_table_iterate(flowtable, nf_flow_table_do_cleanup_addr, &cud);
+		if (cud.found)
+			mod_delayed_work(system_power_efficient_wq,
+					 &flowtable->gc_work, 0);
+	}
+	mutex_unlock(&flowtable_lock);
+
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block nf_flow_table_switchdev_nb __read_mostly = {
+	.notifier_call = nf_flow_table_switchdev_event,
+};
+
 void nf_flow_table_free(struct nf_flowtable *flow_table)
 {
 	mutex_lock(&flowtable_lock);
@@ -818,6 +876,10 @@ static int __init nf_flow_table_module_init(void)
 	if (ret)
 		goto out_offload;
 
+	ret = register_switchdev_notifier(&nf_flow_table_switchdev_nb);
+	if (ret < 0)
+		goto out_sw_noti;
+
 	ret = nf_flow_register_bpf();
 	if (ret)
 		goto out_bpf;
@@ -825,6 +887,8 @@ static int __init nf_flow_table_module_init(void)
 	return 0;
 
 out_bpf:
+	unregister_switchdev_notifier(&nf_flow_table_switchdev_nb);
+out_sw_noti:
 	nf_flow_table_offload_exit();
 out_offload:
 	unregister_pernet_subsys(&nf_flow_table_net_ops);
@@ -833,6 +897,7 @@ static int __init nf_flow_table_module_init(void)
 
 static void __exit nf_flow_table_module_exit(void)
 {
+	unregister_switchdev_notifier(&nf_flow_table_switchdev_nb);
 	nf_flow_table_offload_exit();
 	unregister_pernet_subsys(&nf_flow_table_net_ops);
 }
-- 
2.47.1

Re: [PATCH v3 nf-next 2/3] netfilter: nf_flow_table_core: teardown direct xmit when destination changed

[Florian Westphal]

Eric Woudstra <ericwouds@gmail.com> wrote:
> +static int nf_flow_table_switchdev_event(struct notifier_block *unused,
> +					 unsigned long event, void *ptr)
> +{
> +	struct switchdev_notifier_fdb_info *fdb_info;
> +	struct nf_flowtable *flowtable;
> +	struct flow_cleanup_data cud;
> +
> +	if (event != SWITCHDEV_FDB_DEL_TO_DEVICE)
> +		return NOTIFY_DONE;
> +
> +	fdb_info = ptr;
> +	cud.addr = fdb_info->addr;
> +	cud.vid = fdb_info->vid;
> +	cud.ifindex = fdb_info->info.dev->ifindex;
> +
> +	mutex_lock(&flowtable_lock);

Please always test your patches with lockdep enabled,
this doesn't work.  Switchdev notifiers are atomic.

BUG: sleeping function called from invalid context at kernel/locking/mutex.c:575
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1570, name: kworker/u32:8
preempt_count: 201, expected: 0
RCU nest depth: 1, expected: 0
6 locks held by kworker/u32:8/1570:
 #0: ffff88810032b948 ((wq_completion)netns){+.+.}-{0:0}, at: process_scheduled_works+0x8d7/0x1460
 #1: ffffc900021e7ba0 (net_cleanup_work){+.+.}-{0:0}, at: process_scheduled_works+0x912/0x1460
 #2: ffffffff848d1eb0 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xf8/0x7b0
 #3: ffffffff848de6e8 (rtnl_mutex){+.+.}-{4:4}, at: ops_undo_list+0x270/0x860
 #4: ffff88811f1f8bd8 (&br->hash_lock){+...}-{3:3}, at: br_fdb_delete_by_port+0x3b/0x290
 #5: ffffffff8457a800 (rcu_read_lock){....}-{1:3}, at: atomic_notifier_call_chain+0x27/0x150
Workqueue: netns cleanup_net
Call Trace:
 __mutex_lock+0xf5/0x14f0
  nf_flow_table_switchdev_event+0x104/0x270
 atomic_notifier_call_chain+0xc5/0x150
 br_switchdev_fdb_notify+0x2b2/0x330
...

[PATCH v3 nf-next 3/3] netfilter: nf_flow_table_ip: don't follow fastpath when marked teardown

[Eric Woudstra]

When a flow is marked for teardown, because the destination is not valid
any more, the software fastpath may still be in effect and traffic is
still send to the wrong destination. Change the ip/ipv6 hooks to not use
the software fastpath for a flow that is marked to be teared down and let
the packet continue along the normal path.

Signed-off-by: Eric Woudstra <ericwouds@gmail.com>
---
 net/netfilter/nf_flow_table_ip.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/net/netfilter/nf_flow_table_ip.c b/net/netfilter/nf_flow_table_ip.c
index 64a12b9668e7..f9bf2b466ca8 100644
--- a/net/netfilter/nf_flow_table_ip.c
+++ b/net/netfilter/nf_flow_table_ip.c
@@ -542,6 +542,9 @@ nf_flow_offload_ip_hook(void *priv, struct sk_buff *skb,
 	dir = tuplehash->tuple.dir;
 	flow = container_of(tuplehash, struct flow_offload, tuplehash[dir]);
 
+	if (test_bit(NF_FLOW_TEARDOWN, &flow->flags))
+		return NF_ACCEPT;
+
 	switch (tuplehash->tuple.xmit_type) {
 	case FLOW_OFFLOAD_XMIT_NEIGH:
 		rt = dst_rtable(tuplehash->tuple.dst_cache);
@@ -838,6 +841,9 @@ nf_flow_offload_ipv6_hook(void *priv, struct sk_buff *skb,
 	dir = tuplehash->tuple.dir;
 	flow = container_of(tuplehash, struct flow_offload, tuplehash[dir]);
 
+	if (test_bit(NF_FLOW_TEARDOWN, &flow->flags))
+		return NF_ACCEPT;
+
 	switch (tuplehash->tuple.xmit_type) {
 	case FLOW_OFFLOAD_XMIT_NEIGH:
 		rt = dst_rt6_info(tuplehash->tuple.dst_cache);
-- 
2.47.1

Re: [PATCH v3 nf-next 3/3] netfilter: nf_flow_table_ip: don't follow fastpath when marked teardown

[Pablo Neira Ayuso]

On Tue, Jun 17, 2025 at 09:00:07AM +0200, Eric Woudstra wrote:
> When a flow is marked for teardown, because the destination is not valid
> any more, the software fastpath may still be in effect and traffic is
> still send to the wrong destination. Change the ip/ipv6 hooks to not use
> the software fastpath for a flow that is marked to be teared down and let
> the packet continue along the normal path.
> 
> Signed-off-by: Eric Woudstra <ericwouds@gmail.com>
> ---
>  net/netfilter/nf_flow_table_ip.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/net/netfilter/nf_flow_table_ip.c b/net/netfilter/nf_flow_table_ip.c
> index 64a12b9668e7..f9bf2b466ca8 100644
> --- a/net/netfilter/nf_flow_table_ip.c
> +++ b/net/netfilter/nf_flow_table_ip.c
> @@ -542,6 +542,9 @@ nf_flow_offload_ip_hook(void *priv, struct sk_buff *skb,
>  	dir = tuplehash->tuple.dir;
>  	flow = container_of(tuplehash, struct flow_offload, tuplehash[dir]);
>  
> +	if (test_bit(NF_FLOW_TEARDOWN, &flow->flags))
> +		return NF_ACCEPT;

nf_flow_offload_lookup() already checks for this bit, I don't a
benefit from this re-check.

Re: [PATCH v3 nf-next 0/3] flow offload teardown when layer 2 roaming

[Eric Woudstra]

On 6/17/25 9:00 AM, Eric Woudstra wrote:
> This patch-set can be reviewed separately from my submissions concerning
> the bridge-fastpath.
> 
> In case of a bridge in the forward-fastpath or bridge-fastpath the fdb is
> used to create the tuple. In case of roaming at layer 2 level, for example
> 802.11r, the destination device is changed in the fdb. The destination
> device of a direct transmitting tuple is no longer valid and traffic is
> send to the wrong destination. Also the hardware offloaded fastpath is not
> valid anymore.
> 
> This flowentry needs to be torn down asap. Also make sure that the flow
> entry is not being used, when marked for teardown.
> 
> Changes in v3:
> - static nf_flow_table_switchdev_nb.
> 
> Changes in v2:
> - Unchanged, only tags RFC net-next to PATCH nf-next.
> 
> Eric Woudstra (3):
>   netfilter: flow: Add bridge_vid member
>   netfilter: nf_flow_table_core: teardown direct xmit when destination
>     changed
>   netfilter: nf_flow_table_ip: don't follow fastpath when marked
>     teardown
> 
>  include/net/netfilter/nf_flow_table.h |  2 +
>  net/netfilter/nf_flow_table_core.c    | 66 +++++++++++++++++++++++++++
>  net/netfilter/nf_flow_table_ip.c      |  6 +++
>  net/netfilter/nft_flow_offload.c      |  3 ++
>  4 files changed, 77 insertions(+)
> 

What is the status of this patch-set? Is it still being considered to be
applied? Should I re-submit it? Anything I can do, please let me know.

Re: [PATCH v3 nf-next 0/3] flow offload teardown when layer 2 roaming

[Florian Westphal]

Eric Woudstra <ericwouds@gmail.com> wrote:
> > Changes in v3:
> > - static nf_flow_table_switchdev_nb.
> > 
> > Changes in v2:
> > - Unchanged, only tags RFC net-next to PATCH nf-next.
> > 
> > Eric Woudstra (3):
> >   netfilter: flow: Add bridge_vid member
> >   netfilter: nf_flow_table_core: teardown direct xmit when destination
> >     changed
> >   netfilter: nf_flow_table_ip: don't follow fastpath when marked
> >     teardown
> > 
> >  include/net/netfilter/nf_flow_table.h |  2 +
> >  net/netfilter/nf_flow_table_core.c    | 66 +++++++++++++++++++++++++++
> >  net/netfilter/nf_flow_table_ip.c      |  6 +++
> >  net/netfilter/nft_flow_offload.c      |  3 ++
> >  4 files changed, 77 insertions(+)
> > 
> 
> What is the status of this patch-set? Is it still being considered to be
> applied? Should I re-submit it? Anything I can do, please let me know.

Its marked as 'Changes Requested' in patchwork but I see no feedback
on patchwork or lore archives.

Pablo, do you have any objections to this patchset?

AFAICS this is ok and I'd apply this patchset.