* [PATCH net-next 1/9] net: ethernet: bgmac: validate RX length against the DMA packet capacity
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 2/9] netfilter: flowtable: reject non-host IPv4 packets before lookup Florian Fainelli
` (8 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
BGMAC_RX_ALLOC_SIZE includes skb_shared_info and alignment space beyond
the DMA receive buffer. An invalid completion length can pass that check
and reach skb_put() with more data than the skb tailroom permits.
Limit received frames to BGMAC_RX_MAX_FRAME_SIZE and reject lengths too
short for the FCS subtraction. Keep invalid DMA completions visible
through rate-limited diagnostics and RX error counters.
Fixes: 6a6c708469c9 ("bgmac: add check for oversized packets")
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
drivers/net/ethernet/broadcom/bgmac.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bgmac.c b/drivers/net/ethernet/broadcom/bgmac.c
index 9ef239bc487d..1c502bd64d7d 100644
--- a/drivers/net/ethernet/broadcom/bgmac.c
+++ b/drivers/net/ethernet/broadcom/bgmac.c
@@ -457,10 +457,14 @@ static int bgmac_dma_rx_read(struct bgmac *bgmac, struct bgmac_dma_ring *ring,
break;
}
- if (len > BGMAC_RX_ALLOC_SIZE) {
+ /* The allocation also contains headroom and skb_shared_info;
+ * only the mapped packet area may be supplied by the DMA.
+ */
+ if (unlikely(len > BGMAC_RX_MAX_FRAME_SIZE || len < ETH_FCS_LEN)) {
if (net_ratelimit())
- netdev_err(bgmac->net_dev, "Found oversized packet at slot %d, DMA issue!\n",
- ring->start);
+ netdev_err(bgmac->net_dev,
+ "Invalid RX length %u at slot %d (flags %#x)\n",
+ len, ring->start, flags);
put_page(virt_to_head_page(buf));
bgmac->net_dev->stats.rx_length_errors++;
bgmac->net_dev->stats.rx_errors++;
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 2/9] netfilter: flowtable: reject non-host IPv4 packets before lookup
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 1/9] net: ethernet: bgmac: validate RX length against the DMA packet capacity Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 3/9] netfilter: flowtable: retain offload ownership until deletion completes Florian Fainelli
` (7 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
The IPv4 forwarding path rejects frames whose packet type is not
PACKET_HOST, but an installed flowtable tuple bypasses that check. A
packet addressed to an unrelated, broadcast or multicast destination MAC
can therefore be forwarded if its IPv4 transport tuple matches a flow.
Return to the normal stack before the flow lookup for non-host packets.
This preserves the IPv4 input path's forwarding decision.
Reproduced on BCM53012 using an active UDP NAT tuple: classic forwarding
dropped the unrelated unicast destination MAC while software flowtable
forwarding transmitted it. Boundary tests cover TCP/UDP, both NAT
initiators and unicast, broadcast and multicast MACs.
Implementation, tests and changelog prepared with OpenAI Codex assistance.
Fixes: 97add9f0d66d ("netfilter: flow table support for IPv4")
Signed-off-by: FLorian Fainelli <florian.fainelli@broadcom.com>
---
net/netfilter/nf_flow_table_ip.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/net/netfilter/nf_flow_table_ip.c b/net/netfilter/nf_flow_table_ip.c
index 19cafae748ab..98bade1b8299 100644
--- a/net/netfilter/nf_flow_table_ip.c
+++ b/net/netfilter/nf_flow_table_ip.c
@@ -911,6 +911,10 @@ nf_flow_offload_ip_hook(void *priv, struct sk_buff *skb,
};
int ret;
+ /* The ingress hook runs before IPv4 rejects non-host Ethernet frames. */
+ if (unlikely(skb->pkt_type != PACKET_HOST))
+ return NF_ACCEPT;
+
if (!nf_flow_skb_encap_protocol(&ctx, skb))
return NF_ACCEPT;
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 3/9] netfilter: flowtable: retain offload ownership until deletion completes
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 1/9] net: ethernet: bgmac: validate RX length against the DMA packet capacity Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 2/9] netfilter: flowtable: reject non-host IPv4 packets before lookup Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 4/9] dt-bindings: net: describe the Northstar flow accelerator Florian Fainelli
` (6 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
Teardown clears IPS_OFFLOAD before asynchronous hardware deletion and
hash removal. A new packet can therefore create a replacement flow on
the same conntrack while the old owner still exists. Repeated teardown
can clear the replacement's IPS_OFFLOAD, and the old hardware delete
worker can clear its IPS_HW_OFFLOAD after a new hardware rule was
installed.
The FA route-change reproducer observed four fresh hardware rows but
only one of two connections still carrying HW_OFFLOAD. Two deterministic
KUnit cases also fail before this change: repeated notification releases
a new claim, and hardware retirement admits a new claim before deletion
finishes.
Release IPS_OFFLOAD after both hash entries have been removed. For
hardware flows, GC reaches this point only after hardware deletion is
complete. NF_FLOW_TEARDOWN immediately excludes the old software fast
path. Packets reaching software still use normal conntrack and
forwarding; TCP state and timeout repair run at first teardown. Hardware
deletion may remain pending. Holding the admission bit prevents
overlapping owners.
The delayed release is observable: conntrack dumps may temporarily
report OFFLOAD or HW_OFFLOAD and omit TCP state after a FIN or RST has
returned to the slow path. The status covers ownership pending cleanup
during the GC and asynchronous delete interval, rather than guaranteeing
acceleration. This trades transient reporting for correct ownership
serialization and prevents an old cleanup from clearing its
replacement's status. Document the interval and use packet
captures/counters to identify the packet path.
Include the private lifetime tests and their KUnit configuration here,
independently of the FA series. They cover real hash removal, deferred
hardware retirement, late RCU notification and balanced conntrack
references. The nf_flow_table.config configuration runs these two cases
on its own.
Fixes: e5eaac2beb54 ("netfilter: flowtable: fix TCP flow teardown")
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
Documentation/networking/nf_flowtable.rst | 13 +++
MAINTAINERS | 1 +
net/netfilter/Kconfig | 10 +++
net/netfilter/nf_flow_table_core.c | 9 +-
net/netfilter/nf_flow_table_core_test.c | 86 +++++++++++++++++++
.../kunit/configs/nf_flow_table.config | 9 ++
6 files changed, 127 insertions(+), 1 deletion(-)
create mode 100644 net/netfilter/nf_flow_table_core_test.c
create mode 100644 tools/testing/kunit/configs/nf_flow_table.config
diff --git a/Documentation/networking/nf_flowtable.rst b/Documentation/networking/nf_flowtable.rst
index d757c21c10f2..b9b444435775 100644
--- a/Documentation/networking/nf_flowtable.rst
+++ b/Documentation/networking/nf_flowtable.rst
@@ -205,6 +205,19 @@ refers to the software offload mode, so there is a distinction between [OFFLOAD]
which refers to the software flowtable fastpath and [HW_OFFLOAD] which refers
to the hardware offload datapath being used by the flow.
+During teardown, software fastpath lookups immediately stop matching the
+flow. Packets reaching software follow the classic forwarding path, while
+hardware deletion may still be pending. The OFFLOAD status
+bit remains set until hash removal and asynchronous hardware deletion are
+complete, preventing a new owner from racing the old owner's cleanup. As a
+result, conntrack dumps can temporarily retain [OFFLOAD] or [HW_OFFLOAD]
+and omit the TCP state after a FIN or RST has returned to the slow path.
+TCP state and timeout repair still run at first teardown. The transient
+status describes ownership pending cleanup, rather than guaranteeing that
+packets are still accelerated. It clears when garbage collection finishes
+the deletion; packet captures and forwarding counters identify the actual
+forwarding path during that interval.
+
The flowtable hardware offload infrastructure also supports for the DSA
(Distributed Switch Architecture).
diff --git a/MAINTAINERS b/MAINTAINERS
index c583367c88bc..56212dbc9a0d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -19396,6 +19396,7 @@ F: net/*/netfilter.c
F: net/*/netfilter/
F: net/bridge/br_netfilter*.c
F: net/netfilter/
+F: tools/testing/kunit/configs/nf_flow_table.config
F: tools/testing/selftests/net/netfilter/
NETRONIX EMBEDDED CONTROLLER
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index 09874c26fd13..f565f87b3a6c 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -718,6 +718,16 @@ config NF_FLOW_TABLE
To compile it as a module, choose M here.
+config NF_FLOW_TABLE_KUNIT_TEST
+ bool "KUnit tests for flowtable ownership" if !KUNIT_ALL_TESTS
+ depends on KUNIT=y && NF_FLOW_TABLE
+ default KUNIT_ALL_TESTS
+ help
+ Test conntrack offload ownership during flowtable teardown.
+ Require hash and asynchronous hardware deletion before re-enrolment.
+ Verify that late notifications cannot release a replacement owner.
+ These private tests are embedded in the flowtable core.
+
config NF_FLOW_TABLE_PROCFS
bool "Supply flow table statistics in procfs"
depends on NF_FLOW_TABLE
diff --git a/net/netfilter/nf_flow_table_core.c b/net/netfilter/nf_flow_table_core.c
index 2aa2647ed9df..4c82097cd50b 100644
--- a/net/netfilter/nf_flow_table_core.c
+++ b/net/netfilter/nf_flow_table_core.c
@@ -393,12 +393,15 @@ static void flow_offload_del(struct nf_flowtable *flow_table,
rhashtable_remove_fast(&flow_table->rhashtable,
&flow->tuplehash[FLOW_OFFLOAD_DIR_REPLY].node,
nf_flow_offload_rhash_params);
+ /* Release admission only after both hashes and async hardware state
+ * have gone. An older owner must not clear its replacement's status.
+ */
+ clear_bit_unlock(IPS_OFFLOAD_BIT, &flow->ct->status);
flow_offload_free(flow);
}
void flow_offload_teardown(struct flow_offload *flow)
{
- clear_bit(IPS_OFFLOAD_BIT, &flow->ct->status);
if (!test_and_set_bit(NF_FLOW_TEARDOWN, &flow->flags))
flow_offload_fixup_ct(flow);
}
@@ -876,3 +879,7 @@ module_exit(nf_flow_table_module_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Pablo Neira Ayuso <pablo@netfilter.org>");
MODULE_DESCRIPTION("Netfilter flow table module");
+
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE_KUNIT_TEST)
+#include "nf_flow_table_core_test.c"
+#endif
diff --git a/net/netfilter/nf_flow_table_core_test.c b/net/netfilter/nf_flow_table_core_test.c
new file mode 100644
index 000000000000..378a1c60b50b
--- /dev/null
+++ b/net/netfilter/nf_flow_table_core_test.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <kunit/test.h>
+
+static void nf_flow_test_teardown_repeat(struct kunit *test)
+{
+ struct flow_offload flow = {};
+ struct nf_conn *ct;
+
+ ct = kunit_kzalloc(test, sizeof(*ct), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, ct);
+ ct->tuplehash[0].tuple.dst.protonum = IPPROTO_ICMP;
+ flow.ct = ct;
+ set_bit(IPS_OFFLOAD_BIT, &ct->status);
+ flow_offload_teardown(&flow);
+ KUNIT_EXPECT_TRUE(test, test_bit(IPS_OFFLOAD_BIT, &ct->status));
+ clear_bit(IPS_OFFLOAD_BIT, &ct->status);
+ /* A late notification for this retired flow cannot clear a new claim. */
+ set_bit(IPS_OFFLOAD_BIT, &ct->status);
+ flow_offload_teardown(&flow);
+ KUNIT_EXPECT_TRUE(test, test_bit(IPS_OFFLOAD_BIT, &ct->status));
+}
+
+static void nf_flow_test_hardware_retirement(struct kunit *test)
+{
+ struct nf_flowtable table = {};
+ struct flow_offload *flow;
+ struct nf_conn *ct;
+ unsigned int dir;
+ int err;
+
+ ct = kunit_kzalloc(test, sizeof(*ct), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, ct);
+ err = rhashtable_init(&table.rhashtable, &nf_flow_offload_rhash_params);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ flow = kzalloc_obj(*flow);
+ if (!flow) {
+ rhashtable_destroy(&table.rhashtable);
+ KUNIT_FAIL(test, "flow allocation failed");
+ return;
+ }
+ ct->tuplehash[0].tuple.dst.protonum = IPPROTO_ICMP;
+ refcount_set(&ct->ct_general.use, 2);
+ flow->ct = ct;
+ set_bit(NF_FLOW_HW, &flow->flags);
+ set_bit(IPS_OFFLOAD_BIT, &ct->status);
+ set_bit(IPS_HW_OFFLOAD_BIT, &ct->status);
+ for (dir = 0; dir < FLOW_OFFLOAD_DIR_MAX; dir++) {
+ flow->tuplehash[dir].tuple.src_port = htons(10000 + dir);
+ err = rhashtable_insert_fast(&table.rhashtable,
+ &flow->tuplehash[dir].node,
+ nf_flow_offload_rhash_params);
+ KUNIT_EXPECT_EQ(test, err, 0);
+ }
+ flow_offload_teardown(flow);
+ /* nft_flow_offload must not admit another owner during async DESTROY. */
+ KUNIT_EXPECT_TRUE(test, test_and_set_bit(IPS_OFFLOAD_BIT, &ct->status));
+ flow_offload_teardown(flow);
+ KUNIT_EXPECT_TRUE(test, test_bit(IPS_OFFLOAD_BIT, &ct->status));
+ /* Model completed hardware deletion before the GC removes the hashes. */
+ clear_bit(IPS_HW_OFFLOAD_BIT, &ct->status);
+ set_bit(NF_FLOW_HW_DEAD, &flow->flags);
+ rcu_read_lock();
+ flow_offload_del(&table, flow);
+ KUNIT_EXPECT_FALSE(test, test_bit(IPS_OFFLOAD_BIT, &ct->status));
+ /* Re-enrolment is safe now; an old RCU reader still cannot undo it. */
+ set_bit(IPS_OFFLOAD_BIT, &ct->status);
+ flow_offload_teardown(flow);
+ KUNIT_EXPECT_TRUE(test, test_bit(IPS_OFFLOAD_BIT, &ct->status));
+ rcu_read_unlock();
+ rcu_barrier();
+ KUNIT_EXPECT_EQ(test, refcount_read(&ct->ct_general.use), 1);
+ rhashtable_destroy(&table.rhashtable);
+}
+
+static struct kunit_case nf_flow_lifetime_cases[] = {
+ KUNIT_CASE(nf_flow_test_teardown_repeat),
+ KUNIT_CASE(nf_flow_test_hardware_retirement),
+ {}
+};
+
+static struct kunit_suite nf_flow_lifetime_suite = {
+ .name = "nf-flow-lifetime",
+ .test_cases = nf_flow_lifetime_cases,
+};
+
+kunit_test_suite(nf_flow_lifetime_suite);
diff --git a/tools/testing/kunit/configs/nf_flow_table.config b/tools/testing/kunit/configs/nf_flow_table.config
new file mode 100644
index 000000000000..7f2aa55190c2
--- /dev/null
+++ b/tools/testing/kunit/configs/nf_flow_table.config
@@ -0,0 +1,9 @@
+CONFIG_KUNIT=y
+CONFIG_NET=y
+CONFIG_INET=y
+CONFIG_NETFILTER=y
+CONFIG_NETFILTER_INGRESS=y
+CONFIG_NF_CONNTRACK=y
+CONFIG_NF_TABLES=y
+CONFIG_NF_FLOW_TABLE=y
+CONFIG_NF_FLOW_TABLE_KUNIT_TEST=y
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 4/9] dt-bindings: net: describe the Northstar flow accelerator
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (2 preceding siblings ...)
2026-11-16 22:25 ` [PATCH net-next 3/9] netfilter: flowtable: retain offload ownership until deletion completes Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 5/9] net: dsa: add Broadcom FA metadata and IPv4 exception paths Florian Fainelli
` (5 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
Describe the BCM4708/BCM53011/BCM53012 accelerator register resource and
the GMAC2 consumer phandle. The accelerator is inline on switch CPU port
8; GMAC0 and GMAC1 bypass it. Document the additional port-5/GMAC0 CPU
connection for exceptions that cannot safely traverse the inline lookup.
Keep existing boards unchanged unless their topology explicitly opts in.
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
.../bindings/net/brcm,bcm4708-fa.yaml | 45 +++++++++++++++++++
.../bindings/net/brcm,bcm5301x-gmac.yaml | 27 +++++++++++
MAINTAINERS | 2 +
3 files changed, 74 insertions(+)
create mode 100644 Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml
create mode 100644 Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml
diff --git a/Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml b/Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml
new file mode 100644
index 000000000000..f744b23433c4
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml
@@ -0,0 +1,45 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/brcm,bcm4708-fa.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom Northstar BCM4708 and BCM5301X flow accelerator
+
+maintainers:
+ - Florian Fainelli <f.fainelli@gmail.com>
+
+description: |
+ The flow accelerator is inline between GMAC2 and switch port 8 (IMP0).
+ GMAC0 and GMAC1 bypass it. The GMAC2 BCMA Ethernet node references this
+ register block through a brcm,flow-accelerator phandle. Miss packets carry
+ a prepended accelerator descriptor in place of the embedded switch tag.
+
+ Switch port 5 (IMP5), connected to GMAC0, provides a second CPU path for
+ exceptions that must bypass the accelerator, including TCP control,
+ low-TTL and oversized packets. Describe that CPU port with an ethernet
+ phandle to GMAC0 and an internal, 1 Gbps full-duplex fixed link in the
+ switch's ports node. Port 8 remains the data connection to GMAC2.
+
+properties:
+ compatible:
+ enum:
+ - brcm,bcm4708-fa
+ - brcm,bcm53011-fa
+ - brcm,bcm53012-fa
+
+ reg:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ flow-accelerator@18027c00 {
+ compatible = "brcm,bcm53011-fa";
+ reg = <0x18027c00 0xc0>;
+ };
diff --git a/Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml b/Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml
new file mode 100644
index 000000000000..76fbe2abf515
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/brcm,bcm5301x-gmac.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom Northstar BCMA GMAC flow accelerator connection
+
+maintainers:
+ - Florian Fainelli <f.fainelli@gmail.com>
+
+description: |
+ BCMA enumerates the Northstar GMACs. This binding describes the optional
+ inline flow accelerator connection on GMAC2, connected to switch IMP0.
+
+select:
+ required:
+ - brcm,flow-accelerator
+
+properties:
+ brcm,flow-accelerator:
+ $ref: /schemas/types.yaml#/definitions/phandle
+ description: |
+ Phandle to the BCM4708, BCM53011 or BCM53012 inline flow accelerator.
+ Only GMAC2 (BCMA Ethernet core unit 2) has this connection.
+
+additionalProperties: true
diff --git a/MAINTAINERS b/MAINTAINERS
index 56212dbc9a0d..dce9a88f4aef 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5669,6 +5669,8 @@ R: Broadcom internal kernel review list <bcm-kernel-feedback-list@broadcom.com>
L: netdev@vger.kernel.org
S: Maintained
F: Documentation/devicetree/bindings/net/brcm,amac.yaml
+F: Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml
+F: Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml
F: drivers/net/ethernet/broadcom/bgmac*
F: drivers/net/ethernet/broadcom/unimac.h
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 5/9] net: dsa: add Broadcom FA metadata and IPv4 exception paths
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (3 preceding siblings ...)
2026-11-16 22:25 ` [PATCH net-next 4/9] dt-bindings: net: describe the Northstar flow accelerator Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 6/9] net: flow_offload: allow drivers to validate and invalidate cached paths Florian Fainelli
` (4 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
Add a brcm-fa tagger for the prepended Northstar miss descriptor, with
hash, bucket and parser status fields distinct from ordinary switch
tags. Select it only for an explicitly connected accelerator on
GMAC2/port 8.
Original FA ignores standalone switch exceptions and its key lacks input
port, TTL, packet length and TCP flags. Reserve B53 CFP rules to
redirect SYN/FIN/RST, TTL 0/1 and lengths above 1500 through port
5/GMAC0, outside FA. Per-port router-MAC allow rules and a default IPv4
trap prevent a live tuple from bypassing Linux ingress policy on another
switch port. Nine masked length prefixes cover 1501 through 65535.
Preserve VLAN, EAP and STP filtering, and keep ordinary data flooding
off the auxiliary CPU.
Update MAC guards through a netdevice notifier after DSA registration.
Coordinate guard and conduit readiness with an RCU-protected backend
registry. Disable admission and retire entries before removing any
exception path. Keep incomplete or failed guards disabled across bridge
join and leave, using the same readiness checks as conduit changes.
Decode embedded auxiliary tags before delivering those packets to Linux;
retain strict validation of ordinary Broadcom tags.
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
drivers/net/dsa/b53/Kconfig | 1 +
drivers/net/dsa/b53/b53_common.c | 423 ++++++++++++++++++++++++++++++-
drivers/net/dsa/b53/b53_priv.h | 6 +
drivers/net/dsa/b53/b53_regs.h | 39 +++
include/linux/dsa/brcm.h | 19 ++
include/net/dsa.h | 2 +
net/dsa/Kconfig | 9 +
net/dsa/tag_brcm.c | 116 ++++++++-
8 files changed, 603 insertions(+), 12 deletions(-)
diff --git a/drivers/net/dsa/b53/Kconfig b/drivers/net/dsa/b53/Kconfig
index 915008e8eff5..8e2a8c20bc4a 100644
--- a/drivers/net/dsa/b53/Kconfig
+++ b/drivers/net/dsa/b53/Kconfig
@@ -7,6 +7,7 @@ menuconfig B53
select NET_DSA_TAG_BRCM_LEGACY
select NET_DSA_TAG_BRCM_LEGACY_FCS
select NET_DSA_TAG_BRCM_PREPEND
+ select NET_DSA_TAG_BRCM_FA
help
This driver adds support for Broadcom managed switch chips. It supports
BCM5325E, BCM5365, BCM539x, BCM53115 and BCM53125 as well as BCM63XX
diff --git a/drivers/net/dsa/b53/b53_common.c b/drivers/net/dsa/b53/b53_common.c
index b7aff5e238e8..3e8c9679836d 100644
--- a/drivers/net/dsa/b53/b53_common.c
+++ b/drivers/net/dsa/b53/b53_common.c
@@ -17,19 +17,25 @@
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
+#include <linux/bitfield.h>
#include <linux/delay.h>
+#include <linux/dsa/brcm.h>
#include <linux/export.h>
#include <linux/gpio/consumer.h>
+#include <linux/in.h>
+#include <linux/iopoll.h>
#include <linux/kernel.h>
#include <linux/math.h>
#include <linux/minmax.h>
#include <linux/module.h>
+#include <linux/of.h>
#include <linux/platform_data/b53.h>
#include <linux/phy.h>
#include <linux/phylink.h>
#include <linux/etherdevice.h>
#include <linux/if_bridge.h>
#include <linux/if_vlan.h>
+#include <linux/unaligned.h>
#include <net/dsa.h>
#include "b53_regs.h"
@@ -667,9 +673,10 @@ static void b53_eee_enable_set(struct dsa_switch *ds, int port, bool enable)
int b53_setup_port(struct dsa_switch *ds, int port)
{
struct b53_device *dev = ds->priv;
+ bool auxiliary = dev->fa_cfp_enabled && port == 5;
- b53_port_set_ucast_flood(dev, port, true);
- b53_port_set_mcast_flood(dev, port, true);
+ b53_port_set_ucast_flood(dev, port, !auxiliary);
+ b53_port_set_mcast_flood(dev, port, !auxiliary);
b53_port_set_learning(dev, port, false);
b53_port_set_isolated(dev, port, false);
@@ -678,7 +685,8 @@ int b53_setup_port(struct dsa_switch *ds, int port)
* dropping frames because it isn't allowed to forward there.
*/
if (dsa_is_user_port(ds, port))
- b53_set_eap_mode(dev, port, EAP_MODE_SIMPLIFIED);
+ b53_set_eap_mode(dev, port, dev->fa_cfp_enabled ? EAP_MODE_BASIC :
+ EAP_MODE_SIMPLIFIED);
if (is5325(dev) &&
in_range(port, 1, 4)) {
@@ -728,6 +736,8 @@ int b53_enable_port(struct dsa_switch *ds, int port, struct phy_device *phy)
pvlan &= ~0x1ff;
pvlan |= BIT(port);
pvlan |= dev->ports[port].vlan_ctl_mask;
+ if (dev->fa_cfp_enabled && (dev->fa_cfp_ports & BIT(port)))
+ pvlan |= BIT(5);
b53_write16(dev, B53_PVLAN_PAGE, B53_PVLAN_PORT_MASK(port), pvlan);
b53_imp_vlan_setup(ds, cpu_port);
@@ -1326,6 +1336,315 @@ int b53_setup_devlink_resources(struct dsa_switch *ds)
}
EXPORT_SYMBOL(b53_setup_devlink_resources);
+static int b53_fa_cfp_wait(struct b53_device *dev, u32 mask)
+{
+ u32 val;
+ int ret, err;
+
+ ret = read_poll_timeout(b53_read32, err, err || !(val & mask),
+ 10, 10000, false, dev, B53_CFP_TCAM_PAGE,
+ B53_CFP_ACC, &val);
+ return ret ?: err;
+}
+
+static int b53_fa_cfp_write(struct b53_device *dev, unsigned int index, u32 ram)
+{
+ int err;
+
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_ACC,
+ CFP_ACC_INDEX(index) | ram | CFP_ACC_WRITE | CFP_ACC_START);
+ return err ?: b53_fa_cfp_wait(dev, CFP_ACC_START);
+}
+
+#define B53_FA_ALLOW_BASE 16
+#define B53_FA_DEFAULT_RULE 24
+
+static void b53_fa_udf(u32 data[8], u32 mask[8], unsigned int udf,
+ u16 value, u16 bits)
+{
+ unsigned int bit = CFP_TCAM_UDF_OFFSET + 16 * udf;
+ unsigned int word = bit / 32;
+ unsigned int shift = bit % 32;
+
+ data[word] |= (u32)value << shift;
+ mask[word] |= (u32)bits << shift;
+ if (shift > 16) {
+ data[word + 1] |= value >> (32 - shift);
+ mask[word + 1] |= bits >> (32 - shift);
+ }
+ data[5] |= CFP_TCAM5_UDF_VALID(udf);
+ mask[5] |= CFP_TCAM5_UDF_VALID(udf);
+}
+
+static int b53_fa_rule(struct b53_device *dev, unsigned int index,
+ const u32 data[8], const u32 mask[8], bool trap)
+{
+ unsigned int i;
+ int err;
+
+ for (i = 0; i < 8; i++) {
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_DATA(i), data[i]);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_MASK(i), mask[i]);
+ if (err)
+ return err;
+ }
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_ACTION(0),
+ trap ? CFP_ACT0_FWD_MAP_REPLACE | CFP_ACT0_DST_PORT5 |
+ CFP_ACT0_REASON_TERMINATION : 0);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_ACTION(1),
+ trap ? CFP_ACT1_CLASS_ID(1) | CFP_ACT1_FWD_MAP_REPLACE |
+ CFP_ACT1_DST_PORT5 : 0);
+ if (!err)
+ err = b53_fa_cfp_write(dev, index, CFP_ACC_ACTION);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_METER, 0);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_METER + 4, 0);
+ if (!err)
+ err = b53_fa_cfp_write(dev, index, CFP_ACC_METER);
+ return err ?: b53_fa_cfp_write(dev, index, CFP_ACC_TCAM);
+}
+
+static const u16 b53_fa_lengths[][2] = {
+ { 1501, 0xffff }, { 1502, 0xfffe }, { 1504, 0xffe0 },
+ { 1536, 0xfe00 }, { 2048, 0xf800 }, { 4096, 0xf000 },
+ { 8192, 0xe000 }, { 16384, 0xc000 }, { 32768, 0x8000 },
+};
+
+static int b53_fa_guards(struct b53_device *dev)
+{
+ u32 data[8] = {}, mask[8] = {};
+ unsigned int i;
+ int err;
+
+ /* UDF A1: IPv4 TTL/protocol; A2: IPv4 total length. */
+ err = b53_write8(dev, B53_CFP_PAGE, B53_CFP_UDF1_A0 + 1,
+ CFP_UDF_BASE_END_L2 | CFP_UDF_OFFSET(4));
+ if (!err)
+ err = b53_write8(dev, B53_CFP_PAGE, B53_CFP_UDF1_A0 + 2,
+ CFP_UDF_BASE_END_L2 | CFP_UDF_OFFSET(1));
+ for (i = 0; !err && i < 3; i++)
+ err = b53_write8(dev, B53_CFP_PAGE, B53_CFP_UDF1_A0 + 3 + i, i);
+ if (err)
+ return err;
+ data[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE1;
+ mask[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE_MASK;
+ mask[6] = CFP_TCAM6_L3_FRAMING; /* IPv4, including options and fragments. */
+ b53_fa_udf(data, mask, 1, 0, 0xfe00); /* TTL 0 or 1. */
+ err = b53_fa_rule(dev, 3, data, mask, true);
+ for (i = 0; !err && i < ARRAY_SIZE(b53_fa_lengths); i++) {
+ memset(data, 0, sizeof(data));
+ memset(mask, 0, sizeof(mask));
+ data[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE1;
+ mask[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE_MASK;
+ mask[6] = CFP_TCAM6_L3_FRAMING;
+ b53_fa_udf(data, mask, 2, b53_fa_lengths[i][0], b53_fa_lengths[i][1]);
+ err = b53_fa_rule(dev, 4 + i, data, mask, true);
+ }
+ if (err)
+ return err;
+ /* Only a port's own router MAC may take the inline data path. These
+ * allow rules are populated after DSA has registered its user devices.
+ */
+ memset(data, 0, sizeof(data));
+ memset(mask, 0, sizeof(mask));
+ data[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE1;
+ mask[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE_MASK;
+ mask[6] = CFP_TCAM6_L3_FRAMING;
+ return b53_fa_rule(dev, B53_FA_DEFAULT_RULE, data, mask, true);
+}
+
+/* Original FA ignores switch exceptions on its inline port. Redirect IPv4
+ * exceptions through GMAC0/IMP5, outside FA, while GMAC2/IMP8 carries data.
+ * Reserve classifier rules only when the board describes both CPU paths.
+ */
+static int b53_fa_setup(struct b53_device *dev)
+{
+ struct dsa_port *dp;
+ unsigned int rule, i;
+ u32 data[8], mask[8];
+ int err;
+
+ if (!is5301x(dev) || dev->tag_protocol != DSA_TAG_PROTO_BRCM_FA ||
+ !dsa_is_cpu_port(dev->ds, 5) || !dsa_is_cpu_port(dev->ds, 8))
+ return 0;
+
+ err = b53_write16(dev, B53_CFP_PAGE, B53_CFP_CTRL, 0);
+ if (err)
+ return err;
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_ACC, CFP_ACC_RESET);
+ if (!err)
+ err = b53_fa_cfp_wait(dev, CFP_ACC_RESET);
+ if (err)
+ return err;
+ /* Slice 1 UDF A0 extracts TCP bytes 12..13, relative to the parsed end of L3.
+ * This also works with IP options. Match the individual
+ * FIN/SYN/RST bits, never payload bytes in UDP packets. Slice 0
+ * with a nonzero class ID would request a chained search instead.
+ */
+ err = b53_write8(dev, B53_CFP_PAGE, B53_CFP_UDF1_A0,
+ CFP_UDF_BASE_END_L3 | CFP_UDF_OFFSET(6));
+ if (err)
+ return err;
+ for (rule = 0; rule < 3; rule++) {
+ memset(data, 0, sizeof(data));
+ memset(mask, 0, sizeof(mask));
+ data[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE1 | CFP_TCAM0_UDF0_FLAG(rule);
+ /* Slice 1; FIN, SYN or RST. */
+ mask[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE_MASK | CFP_TCAM0_UDF0_FLAG(rule);
+ data[5] = CFP_TCAM5_UDF_VALID(0); /* UDF A0 valid. */
+ mask[5] = CFP_TCAM5_UDF_VALID(0);
+ data[6] = FIELD_PREP(CFP_TCAM6_IP_PROTOCOL, IPPROTO_TCP);
+ mask[6] = CFP_TCAM6_L3_FRAMING | CFP_TCAM6_IP_PROTOCOL | CFP_TCAM6_IP_FRAGMENT;
+ for (i = 0; i < ARRAY_SIZE(data); i++) {
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_DATA(i), data[i]);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_MASK(i), mask[i]);
+ if (err)
+ return err;
+ }
+ /* Both meter outcomes redirect to IMP5, with class 1 and a
+ * termination reason. Preserve VLAN, EAP and STP filtering.
+ */
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_ACTION(0),
+ CFP_ACT0_FWD_MAP_REPLACE | CFP_ACT0_DST_PORT5 |
+ CFP_ACT0_REASON_TERMINATION);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_ACTION(1),
+ CFP_ACT1_CLASS_ID(1) | CFP_ACT1_FWD_MAP_REPLACE |
+ CFP_ACT1_DST_PORT5);
+ if (!err)
+ err = b53_fa_cfp_write(dev, rule, CFP_ACC_ACTION);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_METER, 0);
+ if (!err)
+ err = b53_write32(dev, B53_CFP_TCAM_PAGE, B53_CFP_METER + 4, 0);
+ if (!err)
+ err = b53_fa_cfp_write(dev, rule, CFP_ACC_METER);
+ if (!err)
+ err = b53_fa_cfp_write(dev, rule, CFP_ACC_TCAM);
+ if (err)
+ return err;
+ }
+ err = b53_fa_guards(dev);
+ if (err)
+ return err;
+ /* Ordinary flooding must reach only the data CPU, not both MACs. */
+ b53_port_set_ucast_flood(dev, 5, false);
+ b53_port_set_mcast_flood(dev, 5, false);
+ err = b53_write8(dev, B53_CTRL_PAGE, B53_PORT_CTRL(5), PORT_CTRL_RX_UCST_EN);
+ if (err)
+ return err;
+ dev->fa_cfp_ports = 0;
+ dsa_switch_for_each_user_port(dp, dev->ds)
+ dev->fa_cfp_ports |= BIT(dp->index);
+ /* The conduit state callback enables these rules after GMAC0 is up. */
+ err = b53_write16(dev, B53_CFP_PAGE, B53_CFP_CTRL, 0);
+ if (!err)
+ dev->fa_cfp_enabled = true;
+ return err;
+}
+
+static bool b53_fa_ready(struct b53_device *dev, bool operational)
+{
+ return operational && dev->fa_guards_ready && !dev->fa_guards_failed;
+}
+
+static int b53_fa_port(struct b53_device *dev, int port, bool standalone)
+{
+ struct dsa_port *control = dsa_to_port(dev->ds, 5);
+ bool ready;
+ u16 ports;
+ int err;
+
+ if (!dev->fa_cfp_enabled)
+ return 0;
+ ports = dev->fa_cfp_ports & ~BIT(port);
+ if (standalone)
+ ports |= BIT(port);
+ ready = b53_fa_ready(dev, dsa_port_conduit_is_operational(control));
+ err = b53_write16(dev, B53_CFP_PAGE, B53_CFP_CTRL,
+ ready ? ports : 0);
+ if (!err)
+ dev->fa_cfp_ports = ports;
+ return err;
+}
+
+static void b53_fa_conduit_state_change(struct dsa_switch *ds,
+ const struct net_device *conduit, bool operational)
+{
+ struct b53_device *dev = ds->priv;
+ struct dsa_port *control, *data;
+ bool ready;
+ int err;
+
+ if (!dev->fa_cfp_enabled)
+ return;
+ control = dsa_to_port(ds, 5);
+ data = dsa_to_port(ds, 8);
+ if (conduit != control->conduit)
+ return;
+ ready = b53_fa_ready(dev, operational);
+ /* Retire offloads before removing any of the physical exception paths. */
+ brcm_fa_control_set_ready(data->conduit, false);
+ err = b53_write16(dev, B53_CFP_PAGE, B53_CFP_CTRL, ready ? dev->fa_cfp_ports : 0);
+ if (err) {
+ dev_err(ds->dev, "failed to update FA exception path: %d\n", err);
+ return;
+ }
+ if (ready)
+ brcm_fa_control_set_ready(data->conduit, true);
+}
+
+static int b53_fa_netdev_event(struct notifier_block *nb, unsigned long event, void *ptr)
+{
+ struct b53_device *dev = container_of(nb, struct b53_device, fa_netdev_nb);
+ struct net_device *netdev = netdev_notifier_info_to_dev(ptr);
+ struct dsa_port *dp, *control, *data_cpu;
+ u32 data[8] = {}, mask[8] = {};
+ unsigned int i;
+ int err;
+
+ if (dev->fa_stopping || !dsa_user_dev_check(netdev) ||
+ (event != NETDEV_REGISTER && event != NETDEV_UP &&
+ event != NETDEV_PRE_CHANGEADDR && event != NETDEV_CHANGEADDR))
+ return NOTIFY_DONE;
+ dp = dsa_port_from_netdev(netdev);
+ if (dp->ds != dev->ds || dp->index >= 8)
+ return NOTIFY_DONE;
+ control = dsa_to_port(dev->ds, 5);
+ data_cpu = dsa_to_port(dev->ds, 8);
+ brcm_fa_control_set_ready(data_cpu->conduit, false);
+ if (event != NETDEV_PRE_CHANGEADDR) {
+ data[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE1;
+ mask[0] = CFP_TCAM0_VALID | CFP_TCAM0_SLICE_MASK;
+ mask[6] = CFP_TCAM6_L3_FRAMING;
+ data[7] = BIT(dp->index);
+ mask[7] = CFP_TCAM7_SOURCE_PORTS;
+ for (i = 0; i < 3; i++)
+ b53_fa_udf(data, mask, 3 + i, get_unaligned_be16(netdev->dev_addr + 2 * i),
+ 0xffff);
+ }
+ err = b53_fa_rule(dev, B53_FA_ALLOW_BASE + dp->index, data, mask, false);
+ if (err) {
+ dev->fa_guards_failed = true;
+ dev_err(dev->dev, "failed to update FA ingress guard: %d\n", err);
+ } else if (event != NETDEV_PRE_CHANGEADDR) {
+ b53_fa_conduit_state_change(dev->ds, control->conduit,
+ dsa_port_conduit_is_operational(control));
+ }
+ return NOTIFY_DONE;
+}
+
+static void b53_fa_unregister_notifier(void *ptr)
+{
+ struct b53_device *dev = ptr;
+
+ unregister_netdevice_notifier(&dev->fa_netdev_nb);
+}
+
static int b53_setup(struct dsa_switch *ds)
{
struct b53_device *dev = ds->priv;
@@ -1391,11 +1710,20 @@ static int b53_setup(struct dsa_switch *ds)
if (ret)
return ret;
+ ret = b53_fa_setup(dev);
+ if (ret)
+ return ret;
+
return b53_setup_devlink_resources(ds);
}
static void b53_teardown(struct dsa_switch *ds)
{
+ struct b53_device *dev = ds->priv;
+
+ dev->fa_stopping = true;
+ if (dev->fa_cfp_enabled)
+ b53_write16(dev, B53_CFP_PAGE, B53_CFP_CTRL, 0);
dsa_devlink_resources_unregister(ds);
}
@@ -2332,6 +2660,7 @@ int b53_br_join(struct dsa_switch *ds, int port, struct dsa_bridge bridge,
s8 cpu_port = dsa_to_port(ds, port)->cpu_dp->index;
u16 pvlan, reg, pvid;
unsigned int i;
+ int err;
/* On 7278, port 7 which connects to the ASP should only receive
* traffic from matching CFP rules.
@@ -2339,6 +2668,10 @@ int b53_br_join(struct dsa_switch *ds, int port, struct dsa_bridge bridge,
if (dev->chip_id == BCM7278_DEVICE_ID && port == 7)
return -EINVAL;
+ err = b53_fa_port(dev, port, false);
+ if (err)
+ return err;
+
pvid = b53_default_pvid(dev);
vl = &dev->vlans[pvid];
@@ -2417,8 +2750,11 @@ void b53_br_leave(struct dsa_switch *ds, int port, struct dsa_bridge bridge)
}
/* Enable redirection of unknown SA to the CPU port */
- b53_set_eap_mode(dev, port, EAP_MODE_SIMPLIFIED);
+ b53_set_eap_mode(dev, port, dev->fa_cfp_enabled ? EAP_MODE_BASIC :
+ EAP_MODE_SIMPLIFIED);
+ if (dev->fa_cfp_enabled)
+ pvlan |= BIT(5);
b53_write16(dev, B53_PVLAN_PAGE, B53_PVLAN_PORT_MASK(port), pvlan);
dev->ports[port].vlan_ctl_mask = pvlan;
@@ -2439,6 +2775,10 @@ void b53_br_leave(struct dsa_switch *ds, int port, struct dsa_bridge bridge)
vl->members |= BIT(port);
b53_set_vlan_entry(dev, pvid, vl);
}
+ if (b53_fa_port(dev, port, true)) {
+ dev_err(ds->dev, "failed to restore FA classification on port %d\n", port);
+ b53_disable_port(ds, port);
+ }
}
EXPORT_SYMBOL(b53_br_leave);
@@ -2552,6 +2892,7 @@ static bool b53_can_enable_brcm_tags(struct dsa_switch *ds, int port,
switch (tag_protocol) {
case DSA_TAG_PROTO_BRCM:
case DSA_TAG_PROTO_BRCM_PREPEND:
+ case DSA_TAG_PROTO_BRCM_FA:
dev_warn(ds->dev,
"Port %d is stacked to Broadcom tag switch\n", port);
ret = false;
@@ -2564,6 +2905,46 @@ static bool b53_can_enable_brcm_tags(struct dsa_switch *ds, int port,
return ret;
}
+static bool b53_uses_fa_tag(struct dsa_switch *ds, int port)
+{
+ struct device_node *ports, *node, *ethernet = NULL, *fa = NULL;
+ struct b53_device *dev = ds->priv;
+ bool enabled = false;
+ u32 index;
+
+ if (port != B53_CPU_PORT)
+ return false;
+ /* CPU5 may be parsed before CPU8 has a dsa_port firmware node. */
+ ports = of_get_child_by_name(ds->dev->of_node, "ports");
+ if (!ports)
+ ports = of_get_child_by_name(ds->dev->of_node, "ethernet-ports");
+ for_each_available_child_of_node(ports, node) {
+ if (of_property_read_u32(node, "reg", &index) || index != B53_CPU_PORT)
+ continue;
+ ethernet = of_parse_phandle(node, "ethernet", 0);
+ fa = of_parse_phandle(ethernet, "brcm,flow-accelerator", 0);
+ enabled = IS_ENABLED(CONFIG_BCM_NS_FA) && of_device_is_available(fa) &&
+ (dev->chip_id == BCM53011_DEVICE_ID ||
+ dev->chip_id == BCM53012_DEVICE_ID ||
+ (is5301x(dev) && of_device_is_compatible(fa, "brcm,bcm4708-fa")));
+ of_node_put(node);
+ break;
+ }
+ of_node_put(ports);
+ of_node_put(fa);
+ of_node_put(ethernet);
+ return enabled;
+}
+
+static struct dsa_port *b53_preferred_cpu_port(struct dsa_switch *ds)
+{
+ struct b53_device *dev = ds->priv;
+
+ if (is5301x(dev) && dsa_is_cpu_port(ds, B53_CPU_PORT) && b53_uses_fa_tag(ds, B53_CPU_PORT))
+ return dsa_to_port(ds, B53_CPU_PORT);
+ return NULL;
+}
+
enum dsa_tag_protocol b53_get_tag_protocol(struct dsa_switch *ds, int port,
enum dsa_tag_protocol mprot)
{
@@ -2583,6 +2964,11 @@ enum dsa_tag_protocol b53_get_tag_protocol(struct dsa_switch *ds, int port,
goto out;
}
+ if (b53_uses_fa_tag(ds, B53_CPU_PORT)) {
+ dev->tag_protocol = DSA_TAG_PROTO_BRCM_FA;
+ goto out;
+ }
+
/* Broadcom BCM58xx chips have a flow accelerator on Port 8
* which requires us to use the prepended Broadcom tag type
*/
@@ -2763,6 +3149,8 @@ static const struct phylink_mac_ops b53_phylink_mac_ops = {
static const struct dsa_switch_ops b53_switch_ops = {
.get_tag_protocol = b53_get_tag_protocol,
+ .preferred_default_local_cpu_port = b53_preferred_cpu_port,
+ .conduit_state_change = b53_fa_conduit_state_change,
.setup = b53_setup,
.teardown = b53_teardown,
.get_strings = b53_get_strings,
@@ -3385,7 +3773,32 @@ int b53_switch_register(struct b53_device *dev)
dev_info(dev->dev, "found switch: %s, rev %i\n",
dev->name, dev->core_rev);
- return dsa_register_switch(dev->ds);
+ ret = dsa_register_switch(dev->ds);
+ if (ret || !dev->fa_cfp_enabled)
+ return ret;
+ /* Registration replays existing user devices, after their MACs and
+ * DSA private data are initialized. Keep lookup gated until it finishes.
+ */
+ dev->fa_netdev_nb.notifier_call = b53_fa_netdev_event;
+ ret = register_netdevice_notifier(&dev->fa_netdev_nb);
+ if (ret)
+ goto unregister_switch;
+ ret = devm_add_action_or_reset(dev->dev, b53_fa_unregister_notifier, dev);
+ if (ret)
+ goto unregister_switch;
+ dev->fa_guards_ready = true;
+ if (!dev->fa_guards_failed) {
+ struct dsa_port *control = dsa_to_port(dev->ds, 5);
+
+ rtnl_lock();
+ b53_fa_conduit_state_change(dev->ds, control->conduit,
+ dsa_port_conduit_is_operational(control));
+ rtnl_unlock();
+ }
+ return 0;
+unregister_switch:
+ dsa_unregister_switch(dev->ds);
+ return ret;
}
EXPORT_SYMBOL(b53_switch_register);
diff --git a/drivers/net/dsa/b53/b53_priv.h b/drivers/net/dsa/b53/b53_priv.h
index c4d2a41c714f..ed663067a865 100644
--- a/drivers/net/dsa/b53/b53_priv.h
+++ b/drivers/net/dsa/b53/b53_priv.h
@@ -153,6 +153,12 @@ struct b53_device {
u8 num_arl_bins;
u16 num_arl_buckets;
enum dsa_tag_protocol tag_protocol;
+ u16 fa_cfp_ports;
+ bool fa_cfp_enabled;
+ bool fa_guards_ready;
+ bool fa_guards_failed;
+ bool fa_stopping;
+ struct notifier_block fa_netdev_nb;
/* used ports mask */
u16 enabled_ports;
diff --git a/drivers/net/dsa/b53/b53_regs.h b/drivers/net/dsa/b53/b53_regs.h
index cfdb5ed38326..4ca7ea64efc6 100644
--- a/drivers/net/dsa/b53/b53_regs.h
+++ b/drivers/net/dsa/b53/b53_regs.h
@@ -614,4 +614,43 @@
/* CFP Control Register with ports map (8 bit) */
#define B53_CFP_CTRL 0x00
+/* Northstar CFP indirect table port. */
+#define B53_CFP_TCAM_PAGE 0xa0
+#define B53_CFP_ACC 0x00
+#define CFP_ACC_START BIT(0)
+#define CFP_ACC_WRITE (2 << 1)
+#define CFP_ACC_TCAM BIT(10)
+#define CFP_ACC_ACTION (2 << 10)
+#define CFP_ACC_METER (4 << 10)
+#define CFP_ACC_RESET BIT(15)
+#define CFP_ACC_INDEX(n) ((n) << 16)
+#define B53_CFP_DATA(n) (0x10 + (n) * 4)
+#define B53_CFP_MASK(n) (0x30 + (n) * 4)
+#define B53_CFP_ACTION(n) (0x50 + (n) * 4)
+#define B53_CFP_METER 0x60
+#define B53_CFP_UDF1_A0 0x20
+#define CFP_UDF_BASE_END_L2 (2 << 5)
+#define CFP_UDF_BASE_END_L3 (3 << 5)
+#define CFP_UDF_OFFSET(n) (n)
+
+/* Slice 1 IPv4 key fields, indexed by DATA/MASK word. */
+#define CFP_TCAM0_VALID GENMASK(1, 0)
+#define CFP_TCAM0_SLICE1 BIT(2)
+#define CFP_TCAM0_SLICE_MASK GENMASK(3, 2)
+#define CFP_TCAM0_UDF0_FLAG(n) BIT(8 + (n))
+#define CFP_TCAM_UDF_OFFSET 8
+#define CFP_TCAM5_UDF_VALID(n) BIT(24 + (n))
+#define CFP_TCAM6_L3_FRAMING GENMASK(25, 24)
+#define CFP_TCAM6_IP_PROTOCOL GENMASK(15, 8)
+#define CFP_TCAM6_IP_FRAGMENT BIT(7)
+#define CFP_TCAM7_SOURCE_PORTS GENMASK(7, 0)
+
+/* Both meter outcomes replace the destination map with CPU port 5. */
+#define CFP_ACT0_FWD_MAP_REPLACE (2 << 24)
+#define CFP_ACT0_DST_PORT5 BIT(19)
+#define CFP_ACT0_REASON_TERMINATION BIT(6)
+#define CFP_ACT1_CLASS_ID(n) ((n) << 20)
+#define CFP_ACT1_FWD_MAP_REPLACE (2 << 11)
+#define CFP_ACT1_DST_PORT5 BIT(6)
+
#endif /* !__B53_REGS_H */
diff --git a/include/linux/dsa/brcm.h b/include/linux/dsa/brcm.h
index 47545a948784..9bfba22b4977 100644
--- a/include/linux/dsa/brcm.h
+++ b/include/linux/dsa/brcm.h
@@ -8,6 +8,25 @@
#ifndef _NET_DSA_BRCM_H
#define _NET_DSA_BRCM_H
+#include <linux/list.h>
+#include <linux/types.h>
+
+struct net_device;
+struct sk_buff;
+
+/* Original FA control packets arrive through the auxiliary CPU port. */
+struct brcm_fa_control {
+ struct list_head list;
+ struct net_device *conduit;
+ void (*receive)(void *priv, struct sk_buff *skb, unsigned int port);
+ void (*state_change)(void *priv, bool ready);
+ void *priv;
+};
+
+void brcm_fa_control_register(struct brcm_fa_control *control);
+void brcm_fa_control_unregister(struct brcm_fa_control *control);
+void brcm_fa_control_set_ready(struct net_device *conduit, bool ready);
+
/* Broadcom tag specific helpers to insert and extract queue/port number */
#define BRCM_TAG_SET_PORT_QUEUE(p, q) ((p) << 8 | q)
#define BRCM_TAG_GET_PORT(v) ((v) >> 8)
diff --git a/include/net/dsa.h b/include/net/dsa.h
index 5d12191b6f6f..5f926b7fde5b 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -62,6 +62,7 @@ struct tc_action;
#define DSA_TAG_PROTO_KSZ8463_VALUE 34
#define DSA_TAG_PROTO_MT7628_VALUE 35
#define DSA_TAG_PROTO_KS8995_VALUE 36
+#define DSA_TAG_PROTO_BRCM_FA_VALUE 37
enum dsa_tag_protocol {
DSA_TAG_PROTO_NONE = DSA_TAG_PROTO_NONE_VALUE,
@@ -69,6 +70,7 @@ enum dsa_tag_protocol {
DSA_TAG_PROTO_BRCM_LEGACY = DSA_TAG_PROTO_BRCM_LEGACY_VALUE,
DSA_TAG_PROTO_BRCM_LEGACY_FCS = DSA_TAG_PROTO_BRCM_LEGACY_FCS_VALUE,
DSA_TAG_PROTO_BRCM_PREPEND = DSA_TAG_PROTO_BRCM_PREPEND_VALUE,
+ DSA_TAG_PROTO_BRCM_FA = DSA_TAG_PROTO_BRCM_FA_VALUE,
DSA_TAG_PROTO_DSA = DSA_TAG_PROTO_DSA_VALUE,
DSA_TAG_PROTO_EDSA = DSA_TAG_PROTO_EDSA_VALUE,
DSA_TAG_PROTO_GSWIP = DSA_TAG_PROTO_GSWIP_VALUE,
diff --git a/net/dsa/Kconfig b/net/dsa/Kconfig
index 4f44bf3ede23..2dffff2f6c34 100644
--- a/net/dsa/Kconfig
+++ b/net/dsa/Kconfig
@@ -69,6 +69,15 @@ config NET_DSA_TAG_BRCM_PREPEND
Broadcom switches which places the tag before the Ethernet header
(prepended).
+config NET_DSA_TAG_BRCM_FA
+ tristate "Tag driver for the Broadcom Northstar flow accelerator"
+ select NET_DSA_TAG_BRCM_COMMON
+ help
+ Support the prepended four-byte FA metadata tag on Northstar GMAC2.
+ The receive format includes a hash, bucket, parser status, and input
+ switch port. Transmit uses the ordinary prepended Broadcom tag.
+ Enable this for a B53 switch explicitly connected to an accelerator.
+
config NET_DSA_TAG_HELLCREEK
tristate "Tag driver for Hirschmann Hellcreek TSN switches"
help
diff --git a/net/dsa/tag_brcm.c b/net/dsa/tag_brcm.c
index b7c49822ca88..8d13a3c7de5e 100644
--- a/net/dsa/tag_brcm.c
+++ b/net/dsa/tag_brcm.c
@@ -9,6 +9,7 @@
#include <linux/etherdevice.h>
#include <linux/if_vlan.h>
#include <linux/list.h>
+#include <linux/mutex.h>
#include <linux/slab.h>
#include "tag.h"
@@ -17,6 +18,8 @@
#define BRCM_LEGACY_NAME "brcm-legacy"
#define BRCM_LEGACY_FCS_NAME "brcm-legacy-fcs"
#define BRCM_PREPEND_NAME "brcm-prepend"
+#define BRCM_FA_NAME "brcm-fa"
+#define BRCM_FA_OPCODE_MISS 2
/* Legacy Broadcom tag (6 bytes) */
#define BRCM_LEG_TAG_LEN 6
@@ -84,7 +87,8 @@
#define BRCM_EG_PID_MASK 0x1f
#if IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM) || \
- IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM_PREPEND)
+ IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM_PREPEND) || \
+ IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM_FA)
static struct sk_buff *brcm_tag_xmit_ll(struct sk_buff *skb,
struct net_device *dev,
@@ -146,9 +150,10 @@ static struct sk_buff *brcm_tag_xmit_ll(struct sk_buff *skb,
*/
static struct sk_buff *brcm_tag_rcv_ll(struct sk_buff *skb,
struct net_device *dev,
- unsigned int offset)
+ unsigned int offset, bool fa)
{
int source_port;
+ u8 opcode;
u8 *brcm_tag;
if (unlikely(!pskb_may_pull(skb, BRCM_TAG_LEN))) {
@@ -158,8 +163,12 @@ static struct sk_buff *brcm_tag_rcv_ll(struct sk_buff *skb,
brcm_tag = skb->data - offset;
- /* The opcode should never be different than 0b000 */
- if (unlikely((brcm_tag[0] >> BRCM_OPCODE_SHIFT) & BRCM_OPCODE_MASK)) {
+ /* Opcode 2 is a four-byte FA miss descriptor. Its middle bytes
+ * contain the flow hash and parser status, not classification/reason.
+ * Timestamp descriptors (opcode 1, eight bytes) are not supported.
+ */
+ opcode = (brcm_tag[0] >> BRCM_OPCODE_SHIFT) & BRCM_OPCODE_MASK;
+ if (unlikely(opcode && !(fa && opcode == BRCM_FA_OPCODE_MISS))) {
kfree_skb(skb);
return NULL;
}
@@ -167,7 +176,7 @@ static struct sk_buff *brcm_tag_rcv_ll(struct sk_buff *skb,
/* We should never see a reserved reason code without knowing how to
* handle it
*/
- if (unlikely(brcm_tag[2] & BRCM_EG_RC_RSVD)) {
+ if (unlikely(!opcode && (brcm_tag[2] & BRCM_EG_RC_RSVD))) {
kfree_skb(skb);
return NULL;
}
@@ -205,7 +214,7 @@ static struct sk_buff *brcm_tag_rcv(struct sk_buff *skb, struct net_device *dev)
struct sk_buff *nskb;
/* skb->data points to the EtherType, the tag is right before it */
- nskb = brcm_tag_rcv_ll(skb, dev, 2);
+ nskb = brcm_tag_rcv_ll(skb, dev, 2, false);
if (!nskb)
return nskb;
@@ -392,7 +401,7 @@ static struct sk_buff *brcm_tag_rcv_prepend(struct sk_buff *skb,
struct net_device *dev)
{
/* tag is prepended to the packet */
- return brcm_tag_rcv_ll(skb, dev, ETH_HLEN);
+ return brcm_tag_rcv_ll(skb, dev, ETH_HLEN, false);
}
static const struct dsa_device_ops brcm_prepend_netdev_ops = {
@@ -407,7 +416,100 @@ DSA_TAG_DRIVER(brcm_prepend_netdev_ops);
MODULE_ALIAS_DSA_TAG_DRIVER(DSA_TAG_PROTO_BRCM_PREPEND, BRCM_PREPEND_NAME);
#endif
+#if IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM_FA)
+static LIST_HEAD(brcm_fa_controls);
+static DEFINE_MUTEX(brcm_fa_control_lock);
+
+void brcm_fa_control_register(struct brcm_fa_control *control)
+{
+ mutex_lock(&brcm_fa_control_lock);
+ list_add_rcu(&control->list, &brcm_fa_controls);
+ mutex_unlock(&brcm_fa_control_lock);
+}
+EXPORT_SYMBOL_GPL(brcm_fa_control_register);
+
+void brcm_fa_control_unregister(struct brcm_fa_control *control)
+{
+ mutex_lock(&brcm_fa_control_lock);
+ list_del_rcu(&control->list);
+ mutex_unlock(&brcm_fa_control_lock);
+ synchronize_net();
+}
+EXPORT_SYMBOL_GPL(brcm_fa_control_unregister);
+
+void brcm_fa_control_set_ready(struct net_device *conduit, bool ready)
+{
+ struct brcm_fa_control *control;
+
+ /* State callbacks can sleep while retiring hardware entries. */
+ mutex_lock(&brcm_fa_control_lock);
+ list_for_each_entry(control, &brcm_fa_controls, list)
+ if (control->conduit == conduit && control->state_change)
+ control->state_change(control->priv, ready);
+ mutex_unlock(&brcm_fa_control_lock);
+}
+EXPORT_SYMBOL_GPL(brcm_fa_control_set_ready);
+
+static struct sk_buff *brcm_fa_tag_xmit(struct sk_buff *skb,
+ struct net_device *dev)
+{
+ struct dsa_port *dp = dsa_user_to_port(dev);
+
+ return brcm_tag_xmit_ll(skb, dev, dp->cpu_dp->index == 5 ? 2 * ETH_ALEN : 0);
+}
+
+static struct sk_buff *brcm_fa_tag_rcv(struct sk_buff *skb, struct net_device *dev)
+{
+ struct dsa_port *cpu = dev->dsa_ptr;
+ struct brcm_fa_control *control;
+ struct dsa_port *data_cpu;
+ unsigned int port;
+ u8 reason;
+
+ if (cpu->index != 5)
+ return brcm_tag_rcv_ll(skb, dev, ETH_HLEN, true);
+ if (!pskb_may_pull(skb, BRCM_TAG_LEN))
+ goto drop;
+ reason = skb->data[0]; /* Embedded tag byte 2 after eth_type_trans(). */
+ port = skb->data[1] & BRCM_EG_PID_MASK;
+ skb = brcm_tag_rcv_ll(skb, dev, 2, false);
+ if (!skb)
+ return NULL;
+ dsa_strip_etype_header(skb, BRCM_TAG_LEN);
+ skb_set_mac_header(skb, -ETH_HLEN);
+ if (!(reason & BRCM_EG_RC_PROT_TERM))
+ return skb;
+ data_cpu = dsa_to_port(cpu->ds, 8);
+ if (!data_cpu || !dsa_port_is_cpu(data_cpu))
+ return skb;
+ rcu_read_lock();
+ list_for_each_entry_rcu(control, &brcm_fa_controls, list)
+ if (control->conduit == data_cpu->conduit)
+ control->receive(control->priv, skb, port);
+ rcu_read_unlock();
+ return skb;
+drop:
+ kfree_skb(skb);
+ return NULL;
+}
+
+static const struct dsa_device_ops brcm_fa_netdev_ops = {
+ .name = BRCM_FA_NAME,
+ .proto = DSA_TAG_PROTO_BRCM_FA,
+ .xmit = brcm_fa_tag_xmit,
+ .rcv = brcm_fa_tag_rcv,
+ .needed_headroom = BRCM_TAG_LEN,
+ .promisc_on_conduit = true,
+};
+
+DSA_TAG_DRIVER(brcm_fa_netdev_ops);
+MODULE_ALIAS_DSA_TAG_DRIVER(DSA_TAG_PROTO_BRCM_FA, BRCM_FA_NAME);
+#endif
+
static struct dsa_tag_driver *dsa_tag_driver_array[] = {
+#if IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM_FA)
+ &DSA_TAG_DRIVER_NAME(brcm_fa_netdev_ops),
+#endif
#if IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM)
&DSA_TAG_DRIVER_NAME(brcm_netdev_ops),
#endif
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 6/9] net: flow_offload: allow drivers to validate and invalidate cached paths
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (4 preceding siblings ...)
2026-11-16 22:25 ` [PATCH net-next 5/9] net: dsa: add Broadcom FA metadata and IPv4 exception paths Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 7/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (3 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
A classifier does not necessarily contain the mark, DSCP and output-
interface inputs used to select its forwarding path. A partial driver
FIB lookup cannot establish that a cached software route is stale.
Provide optional owner validation and invalidation callbacks. Validation
runs under RCU without sleeping and distinguishes a proven stale path
from unavailable context. Retain direct-path dst references for hardware
flowtables and validate the actual cached routes, cookies and direct
Ethernet neighbours in both directions on demand.
Mark references retained only for driver validation. They must not opt
plain direct flows into packet-path or GC route checks after unrelated
IPv4 generation changes. Keep the existing checks for direct tunnel,
neighbour and XFRM routes. The explicit owner callback bypasses that
policy and validates the retained dst itself. Other hardware users and
tables without a validating driver keep their existing automatic-check
behaviour. Software-only direct flows retain their previous reference
policy. Release retained references with the flow.
Netfilter implements invalidation by marking the owning connection for
teardown. Owners keep cookies valid through an RCU grace period after
DESTROY; drivers stop issuing notifications when their rules are
destroyed.
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
include/net/flow_offload.h | 12 ++++++
include/net/netfilter/nf_flow_table.h | 8 ++--
net/netfilter/nf_flow_table_core.c | 12 +++---
net/netfilter/nf_flow_table_offload.c | 58 +++++++++++++++++++++++++++
net/netfilter/nft_flow_offload.c | 6 ++-
5 files changed, 87 insertions(+), 9 deletions(-)
diff --git a/include/net/flow_offload.h b/include/net/flow_offload.h
index 70a02ee14308..09301a924014 100644
--- a/include/net/flow_offload.h
+++ b/include/net/flow_offload.h
@@ -697,6 +697,18 @@ struct flow_cls_offload {
bool use_act_stats;
unsigned long cookie;
struct flow_rule *rule;
+ /* Optional owner notification when a rule no longer represents the path.
+ * Call under RCU; the owner retains the cookie and callback until a
+ * grace period after DESTROY. Drivers must stop new calls at DESTROY.
+ * The callback must not sleep or call back into the offload driver.
+ */
+ void (*invalidate)(unsigned long cookie);
+ /* Optional cached-path validation, called under RCU during REPLACE.
+ * Return 0 for a valid path, -ESTALE for a demonstrably stale path,
+ * or -EOPNOTSUPP when validation is unavailable. Do not sleep or
+ * call back into the driver. The callback does not invalidate its owner.
+ */
+ int (*validate)(unsigned long cookie);
struct flow_stats stats;
u32 classid;
};
diff --git a/include/net/netfilter/nf_flow_table.h b/include/net/netfilter/nf_flow_table.h
index 5b22d31cb140..99cb087ff0b2 100644
--- a/include/net/netfilter/nf_flow_table.h
+++ b/include/net/netfilter/nf_flow_table.h
@@ -154,7 +154,8 @@ struct flow_offload_tuple {
encap_num:2,
needs_gso_segment:1,
tun_num:2,
- in_vlan_ingress:2;
+ in_vlan_ingress:2,
+ dst_validation_only:1;
u16 mtu;
u32 dst_cookie;
struct dst_entry *dst_cache;
@@ -305,7 +306,7 @@ nf_flow_table_offload_del_cb(struct nf_flowtable *flow_table,
}
void flow_offload_route_init(struct flow_offload *flow,
- struct nf_flow_route *route);
+ struct nf_flow_route *route, bool keep_dst);
int flow_offload_add(struct nf_flowtable *flow_table, struct flow_offload *flow);
void flow_offload_refresh(struct nf_flowtable *flow_table,
@@ -313,7 +314,8 @@ void flow_offload_refresh(struct nf_flowtable *flow_table,
static inline bool nf_flow_dst_check(struct flow_offload_tuple *tuple)
{
- if (!tuple->dst_cache)
+ /* A driver-only reference must not change direct-path/GC semantics. */
+ if (!tuple->dst_cache || tuple->dst_validation_only)
return true;
return dst_check(tuple->dst_cache, tuple->dst_cookie);
diff --git a/net/netfilter/nf_flow_table_core.c b/net/netfilter/nf_flow_table_core.c
index 4c82097cd50b..b16a336101af 100644
--- a/net/netfilter/nf_flow_table_core.c
+++ b/net/netfilter/nf_flow_table_core.c
@@ -96,7 +96,7 @@ static struct dst_entry *nft_route_dst_fetch(struct nf_flow_route *route,
static int flow_offload_fill_route(struct flow_offload *flow,
struct nf_flow_route *route,
- enum flow_offload_tuple_dir dir)
+ enum flow_offload_tuple_dir dir, bool keep_dst)
{
struct flow_offload_tuple *flow_tuple = &flow->tuplehash[dir].tuple;
struct dst_entry *dst = nft_route_dst_fetch(route, dir);
@@ -127,7 +127,9 @@ static int flow_offload_fill_route(struct flow_offload *flow,
switch (route->tuple[dir].xmit_type) {
case FLOW_OFFLOAD_XMIT_DIRECT:
- if (route->tuple[!dir].in.num_tuns) {
+ flow_tuple->dst_validation_only = keep_dst &&
+ !route->tuple[!dir].in.num_tuns;
+ if (keep_dst || route->tuple[!dir].in.num_tuns) {
flow_tuple->dst_cache = dst;
flow_tuple->dst_cookie =
flow_offload_dst_cookie(flow_tuple);
@@ -162,10 +164,10 @@ static void nft_flow_dst_release(struct flow_offload *flow,
}
void flow_offload_route_init(struct flow_offload *flow,
- struct nf_flow_route *route)
+ struct nf_flow_route *route, bool keep_dst)
{
- flow_offload_fill_route(flow, route, FLOW_OFFLOAD_DIR_ORIGINAL);
- flow_offload_fill_route(flow, route, FLOW_OFFLOAD_DIR_REPLY);
+ flow_offload_fill_route(flow, route, FLOW_OFFLOAD_DIR_ORIGINAL, keep_dst);
+ flow_offload_fill_route(flow, route, FLOW_OFFLOAD_DIR_REPLY, keep_dst);
flow->type = NF_FLOW_OFFLOAD_ROUTE;
}
EXPORT_SYMBOL_GPL(flow_offload_route_init);
diff --git a/net/netfilter/nf_flow_table_offload.c b/net/netfilter/nf_flow_table_offload.c
index 4365859220e6..1f3adf70f598 100644
--- a/net/netfilter/nf_flow_table_offload.c
+++ b/net/netfilter/nf_flow_table_offload.c
@@ -885,6 +885,62 @@ static int nf_flow_offload_alloc(const struct flow_offload_work *offload,
return 0;
}
+static struct flow_offload *nf_flow_offload_cookie_flow(unsigned long cookie)
+{
+ struct flow_offload_tuple *tuple = (struct flow_offload_tuple *)cookie;
+ struct flow_offload_tuple_rhash *tuplehash;
+ struct flow_offload *flow;
+
+ tuplehash = container_of(tuple, struct flow_offload_tuple_rhash, tuple);
+ flow = container_of(tuplehash, struct flow_offload, tuplehash[tuple->dir]);
+ return flow;
+}
+
+static void nf_flow_offload_invalidate(unsigned long cookie)
+{
+ flow_offload_teardown(nf_flow_offload_cookie_flow(cookie));
+}
+
+static int nf_flow_offload_validate(unsigned long cookie)
+{
+ struct flow_offload *flow = nf_flow_offload_cookie_flow(cookie);
+ unsigned int dir;
+
+ for (dir = 0; dir < FLOW_OFFLOAD_DIR_MAX; dir++) {
+ struct flow_offload_tuple *tuple = &flow->tuplehash[dir].tuple;
+ struct flow_offload_tuple *other = &flow->tuplehash[!dir].tuple;
+ struct dst_entry *dst = tuple->dst_cache;
+ struct neighbour *neigh;
+ u8 mac[ETH_ALEN];
+ bool valid;
+
+ if (!dst)
+ return -EOPNOTSUPP;
+ /* This is the actual route selected with the packet's policy
+ * context, including mark, DSCP and the reverse lookup's oif.
+ */
+ if (!dst_check(dst, tuple->dst_cookie))
+ return -ESTALE;
+ if (tuple->xmit_type != FLOW_OFFLOAD_XMIT_DIRECT)
+ continue;
+ /* A stacked path may have a different final Ethernet neighbour.
+ * Do not mistake that unsupported context for a stale owner.
+ */
+ if (dst->dev->ifindex != tuple->out.ifidx)
+ return -EOPNOTSUPP;
+ neigh = dst_neigh_lookup(dst, &other->src_v4);
+ if (!neigh)
+ return -ESTALE;
+ neigh_ha_snapshot(mac, neigh, dst->dev);
+ valid = (READ_ONCE(neigh->nud_state) & NUD_VALID) &&
+ ether_addr_equal(mac, tuple->out.h_dest);
+ neigh_release(neigh);
+ if (!valid)
+ return -ESTALE;
+ }
+ return 0;
+}
+
static void nf_flow_offload_init(struct flow_cls_offload *cls_flow,
__be16 proto, int priority,
enum flow_cls_command cmd,
@@ -896,6 +952,8 @@ static void nf_flow_offload_init(struct flow_cls_offload *cls_flow,
cls_flow->common.extack = extack;
cls_flow->command = cmd;
cls_flow->cookie = (unsigned long)tuple;
+ cls_flow->invalidate = nf_flow_offload_invalidate;
+ cls_flow->validate = nf_flow_offload_validate;
}
static int nf_flow_offload_tuple(struct nf_flowtable *flowtable,
diff --git a/net/netfilter/nft_flow_offload.c b/net/netfilter/nft_flow_offload.c
index d3c5651dd699..05207ca808ee 100644
--- a/net/netfilter/nft_flow_offload.c
+++ b/net/netfilter/nft_flow_offload.c
@@ -113,7 +113,11 @@ static void nft_flow_offload_eval(const struct nft_expr *expr,
if (!flow)
goto err_flow_alloc;
- flow_offload_route_init(flow, &route);
+ /* Hardware admission may validate the route used to build a direct path.
+ * Keep its reference and cookie instead of reconstructing a lookup with
+ * incomplete policy-routing inputs in the driver.
+ */
+ flow_offload_route_init(flow, &route, nf_flowtable_hw_offload(flowtable));
if (tcph)
flow_offload_ct_tcp(ct);
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 7/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (5 preceding siblings ...)
2026-11-16 22:25 ` [PATCH net-next 6/9] net: flow_offload: allow drivers to validate and invalidate cached paths Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 8/9] net: add KUnit coverage for FA and flowtable path validation Florian Fainelli
` (2 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
Expose the inline BCM4708/BCM5301X accelerator through TC_SETUP_FT on
GMAC2. Learn placement hashes from miss metadata and install exact IPv4
TCP/UDP SNAT or DNAT rules with shared router-MAC and next-hop
resources. Restrict admission to two standalone untagged DSA ports,
distinct router MACs, MTU 1500 and one flowtable in the initial
namespace. Require an operational GMAC0 exception path for both
protocols and the flowtable FIN/RST exclusion mask for TCP. Reject
unsupported or exhausted rules.
Gate lookup synchronously and retire all hardware owners on every IPv4
route, rule or nexthop notification, including unrelated prefixes and
local-table changes. This is deliberately conservative: the driver lacks
enough policy-routing context to prove an event irrelevant. Coalesce
control work, accepting reduced offload residency during unrelated
churn. Track explicit low-MTU prefixes incrementally from notifier
replay and updates; avoid repeated FIB dumps. Bound tracking memory and
latch software fallback if tracking is exhausted.
Use the owner's actual cached-route and neighbour validation as the only
authority for stale-owner teardown. The driver's partial FIB lookup is
an eligibility probe; it lacks mark, DSCP and output-interface context.
A mismatch or unsupported shape declines hardware offload without
tearing down a valid policy-routed software flow. Reject publication
across invalidation generations. Use RCU lookups and tracked netdevice
references. Drive neighbour probes from hardware counter activity,
including the initial pre-wrap jiffies epoch. Retire owners on
neighbour, topology or exception-path changes that withdraw their
hardware paths.
Quiesce lookups using GMAC2 CPU flow control around indirect table
accesses, drain an in-flight frame, check busy and address completion,
and release read mode before resuming traffic. Preserve Broadcom-tag
relocation when disabling lookup. On an uncertain transaction, gate the
parser before releasing pause, retain uncertain resources and reject new
entries until reprobe. Add optional per-device completion fault
injection.
TCP control packets retire both NAT directions before tuple reuse.
Accumulate packet deltas and preserve the final delta for retired
owners; do not fabricate hardware byte counters. Provide debugfs status
and raw table readback. FA2/FA+, PPPoE and board enablement remain
separate work.
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
MAINTAINERS | 1 +
drivers/net/ethernet/broadcom/Kconfig | 24 +
drivers/net/ethernet/broadcom/Makefile | 1 +
drivers/net/ethernet/broadcom/bcm-ns-fa.c | 1897 ++++++++++++++++++++
drivers/net/ethernet/broadcom/bcm-ns-fa.h | 30 +
drivers/net/ethernet/broadcom/bgmac-bcma.c | 5 +
drivers/net/ethernet/broadcom/bgmac.c | 31 +
drivers/net/ethernet/broadcom/bgmac.h | 2 +
8 files changed, 1991 insertions(+)
create mode 100644 drivers/net/ethernet/broadcom/bcm-ns-fa.c
create mode 100644 drivers/net/ethernet/broadcom/bcm-ns-fa.h
diff --git a/MAINTAINERS b/MAINTAINERS
index dce9a88f4aef..80969b91af40 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5671,6 +5671,7 @@ S: Maintained
F: Documentation/devicetree/bindings/net/brcm,amac.yaml
F: Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml
F: Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml
+F: drivers/net/ethernet/broadcom/bcm-ns-fa*
F: drivers/net/ethernet/broadcom/bgmac*
F: drivers/net/ethernet/broadcom/unimac.h
diff --git a/drivers/net/ethernet/broadcom/Kconfig b/drivers/net/ethernet/broadcom/Kconfig
index f0bac0dd1439..865b346a7249 100644
--- a/drivers/net/ethernet/broadcom/Kconfig
+++ b/drivers/net/ethernet/broadcom/Kconfig
@@ -163,13 +163,36 @@ config BNX2X_SRIOV
config BGMAC
tristate
+ depends on BCM_NS_FA || !BCM_NS_FA
help
This enables the integrated ethernet controller support for many
Broadcom (mostly iProc) SoCs. An appropriate bus interface driver
needs to be enabled to select this.
+config BCM_NS_FA
+ tristate "Broadcom Northstar IPv4 TCP/UDP NAT flow accelerator"
+ depends on ARCH_BCM_5301X || COMPILE_TEST
+ depends on OF && NET_DSA && NF_FLOW_TABLE
+ select NET_DSA_TAG_BRCM_FA
+ help
+ Offload IPv4 TCP/UDP NAT flows from nftables flowtables to the inline
+ accelerator on BCM4708, BCM53011 and BCM53012. Requires GMAC2, a B53 switch
+ using brcm-fa tags, a second CPU link through GMAC0 for IPv4 exceptions,
+ and explicit device tree configuration.
+ VLANs, bridges, and double NAT use the software path.
+
+config BCM_NS_FA_FAULT_INJECTION
+ bool "Fault injection for Northstar FA table transactions"
+ depends on BCM_NS_FA && FAULT_INJECTION_DEBUG_FS
+ help
+ Add per-device fail_table controls in debugfs to simulate an uncertain
+ indirect table completion. The failure disables hardware lookup until
+ reprobe and leaves forwarding to software. Intended for driver testing.
+ Injection is disabled by default and can be armed through debugfs.
+
config BGMAC_BCMA
tristate "Broadcom iProc GBit BCMA support"
+ depends on BCM_NS_FA || !BCM_NS_FA
depends on BCMA && BCMA_HOST_SOC
depends on BCM47XX || ARCH_BCM_5301X || COMPILE_TEST
select BGMAC
@@ -183,6 +206,7 @@ config BGMAC_BCMA
config BGMAC_PLATFORM
tristate "Broadcom iProc GBit platform support"
+ depends on BCM_NS_FA || !BCM_NS_FA
depends on ARCH_BCM_IPROC || COMPILE_TEST
select BGMAC
select PHYLIB
diff --git a/drivers/net/ethernet/broadcom/Makefile b/drivers/net/ethernet/broadcom/Makefile
index 10cc1c92ecfc..1c445d9849ef 100644
--- a/drivers/net/ethernet/broadcom/Makefile
+++ b/drivers/net/ethernet/broadcom/Makefile
@@ -19,3 +19,4 @@ obj-$(CONFIG_SYSTEMPORT) += bcmsysport.o
obj-$(CONFIG_BNXT) += bnxt/
obj-$(CONFIG_BCMASP) += asp2/
obj-$(CONFIG_BNGE) += bnge/
+obj-$(CONFIG_BCM_NS_FA) += bcm-ns-fa.o
diff --git a/drivers/net/ethernet/broadcom/bcm-ns-fa.c b/drivers/net/ethernet/broadcom/bcm-ns-fa.c
new file mode 100644
index 000000000000..8665ba94cbab
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bcm-ns-fa.c
@@ -0,0 +1,1897 @@
+// SPDX-License-Identifier: GPL-2.0-only AND ISC
+/* Broadcom Northstar inline flow accelerator, GMAC2 / switch IMP0.
+ *
+ * Table formats and access sequence follow the BCM5301X FA theory of
+ * operations and the ISC-licensed 2013 Broadcom ctf_config.c reference.
+ * Copyright (C) 2013 Broadcom Corporation
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
+ * SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION
+ * OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
+ * CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/dsa/brcm.h>
+#include <linux/debugfs.h>
+#include <linux/delay.h>
+#include <linux/etherdevice.h>
+#include <linux/fault-inject.h>
+#include <linux/hashtable.h>
+#include <linux/if_vlan.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/ip.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/seq_file.h>
+#include <linux/tcp.h>
+#include <linux/udp.h>
+#include <linux/workqueue.h>
+#include <linux/unaligned.h>
+#include <linux/tc_act/tc_csum.h>
+#include <net/dsa.h>
+#include <net/arp.h>
+#include <net/flow_offload.h>
+#include <net/fib_notifier.h>
+#include <net/ip.h>
+#include <net/ip_fib.h>
+#include <net/neighbour.h>
+#include <net/netevent.h>
+#include <net/net_namespace.h>
+
+#include "bcm-ns-fa.h"
+#include "bgmac.h"
+
+#define FA_CTL 0x00
+#define FA_CTL_BYPASS BIT(1)
+#define FA_CTL_CRC_OVERWRITE BIT(3)
+#define FA_CTL_THRESHOLD GENMASK(12, 4)
+#define FA_CTL_INIT GENMASK(17, 13)
+#define FA_CTL_NO_DA_CHECK BIT(18)
+#define FA_MEM_CTL 0x04
+#define FA_MEM_READ BIT(12)
+#define FA_MEM_TABLE GENMASK(11, 10)
+#define FA_MEM_CURRENT_INDEX GENMASK(22, 13)
+#define FA_BRCM_HDR 0x08
+#define FA_BRCM_HW BIT(0)
+#define FA_BRCM_RX BIT(1)
+#define FA_BRCM_TX BIT(2)
+#define FA_BRCM_PARSE_EXCEPTIONS BIT(3)
+#define FA_L2_SKIP 0x0c
+#define FA_L2_TAG 0x10
+#define FA_L2_ETHERTYPE 0x20
+#define FA_IPV4 0x28
+#define FA_IPV4_CSUM BIT(16)
+#define FA_NAPT 0x2c
+#define FA_NAPT_DEC_TTL BIT(21)
+#define FA_STATUS 0x30
+#define FA_INIT_DONE GENMASK(9, 5)
+#define FA_STATUS_MASK 0x34
+#define FA_RX_STATUS 0x38
+#define FA_HITS 0x3c
+#define FA_MISSES 0x40
+#define FA_ERROR 0x64
+#define FA_ERROR_MASK 0x68
+#define FA_DEBUG_STATUS 0x70
+#define FA_MEM_BUSY BIT(0)
+#define FA_DATA(n) (0xa0 + 4 * (n))
+#define FA_FLOW_TABLE 0
+#define FA_POOL_TABLE 1
+#define FA_NH_TABLE 2
+#define FA_SLOTS 1024
+#define FA_WAYS 4
+#define FA_POOLS 4
+#define FA_NEXTHOPS 128
+#define FA_TAG_LEN 4
+#define FA_MISS_OPCODE 2
+#define FA_FLOW_CLOSING 0
+#define FA_MTU_HASH_BITS 6
+#define FA_MTU_ROUTES_MAX 1024
+
+/* IPv4 flow entry, indexed by DATA word; global bit numbers in AN300 table 3. */
+#define FA_FLOW0_HITS_LO GENMASK(31, 3) /* hits[28:0], bits 31:3 */
+#define FA_FLOW1_HITS_HI GENMASK(2, 0) /* hits[31:29], bits 34:32 */
+#define FA_FLOW1_ACTION GENMASK(4, 3)
+#define FA_FLOW1_ROUTER_MAC GENMASK(7, 6)
+#define FA_FLOW1_NEXT_HOP GENMASK(14, 8)
+#define FA_FLOW1_REMAP_IP_LO GENMASK(31, 15) /* remap_ip[16:0], bits 63:47 */
+#define FA_FLOW2_REMAP_IP_HI GENMASK(14, 0) /* remap_ip[31:17], bits 78:64 */
+#define FA_FLOW2_REMAP_PORT GENMASK(30, 15)
+#define FA_FLOW2_DPORT_LO BIT(31) /* dport[0], bit 95 */
+#define FA_FLOW3_DPORT_HI GENMASK(14, 0) /* dport[15:1], bits 110:96 */
+#define FA_FLOW3_SPORT GENMASK(30, 15)
+#define FA_FLOW3_TCP BIT(31)
+#define FA_FLOW6_DIRECTION BIT(0)
+#define FA_FLOW6_DEST_MAP GENMASK(10, 1)
+#define FA_FLOW6_OPCODE GENMASK(19, 17)
+#define FA_FLOW6_OPCODE_INGRESS 1
+#define FA_FLOW6_VALID BIT(20)
+#define FA_FLOW7_IPV4 BIT(31)
+
+/* Router-MAC pool entry: external in bit 0, MAC in bits 48:1. */
+#define FA_POOL0_EXTERNAL BIT(0)
+#define FA_POOL0_MAC_LO GENMASK(31, 1)
+#define FA_POOL1_MAC_HI GENMASK(16, 0)
+
+/* Next-hop entry: frame type in bit 0, tag op in 2:1, DA in 66:19. */
+#define FA_NH0_TAG_OP GENMASK(2, 1)
+#define FA_NH_TAG_NONE 2
+#define FA_NH0_MAC_LO GENMASK(31, 19)
+#define FA_NH2_MAC_HI GENMASK(2, 0)
+
+/* Flowtable rule validation holds RCU, rather than RTNL. */
+static bool fa_has_upper_dev_rcu(struct net_device *dev)
+{
+ struct list_head *iter = &dev->adj_list.upper;
+
+ return netdev_upper_get_next_dev_rcu(dev, &iter);
+}
+
+/* No implicit padding: also used as the key for the RX hash cache. */
+struct fa_tuple {
+ __be32 src, dst;
+ __be16 sport, dport;
+};
+
+#define FA_ACT_REMAP_IP BIT(0)
+#define FA_ACT_REMAP_PORT BIT(1)
+
+enum fa_direction {
+ FA_DIR_UNSET = -1,
+ FA_DIR_SNAT,
+ FA_DIR_DNAT,
+};
+
+struct fa_rule {
+ struct fa_tuple tuple;
+ struct net_device *in, *out;
+ u8 eth[ETH_ALEN * 2];
+ u8 eth_mask[ETH_ALEN * 2];
+ u32 remap_ip;
+ u16 remap_port;
+ u8 action, l4proto;
+ enum fa_direction direction;
+};
+
+static int fa_direction(struct fa_rule *rule, enum fa_direction direction)
+{
+ if (rule->direction != FA_DIR_UNSET && rule->direction != direction)
+ return -EOPNOTSUPP;
+ rule->direction = direction;
+ return 0;
+}
+
+static int fa_mangle(struct fa_rule *rule, const struct flow_action_entry *act)
+{
+ u32 mask = act->mangle.mask, val = act->mangle.val;
+ unsigned int offset = act->mangle.offset;
+ enum fa_direction dir;
+
+ switch (act->mangle.htype) {
+ case FLOW_ACT_MANGLE_HDR_TYPE_ETH: {
+ u8 *data = rule->eth, *known = rule->eth_mask;
+ unsigned int i;
+
+ if (offset > sizeof(rule->eth) - sizeof(val))
+ return -EOPNOTSUPP;
+ for (i = 0; i < sizeof(val); i++) {
+ u8 m = ((u8 *)&mask)[i];
+
+ data[offset + i] = (data[offset + i] & m) |
+ (((u8 *)&val)[i] & ~m);
+ known[offset + i] |= ~m;
+ }
+ return 0;
+ }
+ case FLOW_ACT_MANGLE_HDR_TYPE_IP4:
+ if (mask || (offset != offsetof(struct iphdr, saddr) &&
+ offset != offsetof(struct iphdr, daddr)))
+ return -EOPNOTSUPP;
+ dir = offset == offsetof(struct iphdr, daddr) ? FA_DIR_DNAT : FA_DIR_SNAT;
+ if (fa_direction(rule, dir) || (rule->action & FA_ACT_REMAP_IP))
+ return -EOPNOTSUPP;
+ rule->remap_ip = be32_to_cpu((__force __be32)val);
+ rule->action |= FA_ACT_REMAP_IP;
+ return 0;
+ case FLOW_ACT_MANGLE_HDR_TYPE_TCP:
+ case FLOW_ACT_MANGLE_HDR_TYPE_UDP:
+ if ((act->mangle.htype == FLOW_ACT_MANGLE_HDR_TYPE_TCP &&
+ rule->l4proto != IPPROTO_TCP) ||
+ (act->mangle.htype == FLOW_ACT_MANGLE_HDR_TYPE_UDP &&
+ rule->l4proto != IPPROTO_UDP))
+ return -EOPNOTSUPP;
+ if (offset)
+ return -EOPNOTSUPP;
+ mask = be32_to_cpu((__force __be32)mask);
+ val = be32_to_cpu((__force __be32)val);
+ if (mask != 0xffff && mask != 0xffff0000)
+ return -EOPNOTSUPP;
+ dir = mask == 0xffff0000 ? FA_DIR_DNAT : FA_DIR_SNAT;
+ if (fa_direction(rule, dir) || (rule->action & FA_ACT_REMAP_PORT))
+ return -EOPNOTSUPP;
+ rule->remap_port = dir ? val & 0xffff : val >> 16;
+ rule->action |= FA_ACT_REMAP_PORT;
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+struct fa_learned {
+ struct fa_tuple tuple;
+ u8 mac[ETH_ALEN];
+ unsigned long generation;
+ u8 hash, port, l4proto;
+ bool valid;
+};
+
+struct fa_mac {
+ u8 addr[ETH_ALEN];
+ bool external;
+ unsigned int refs;
+};
+
+struct fa_neighbour {
+ __be32 address;
+ int ifindex;
+ u8 mac[ETH_ALEN];
+ u8 state;
+ bool valid;
+};
+
+struct fa_flow {
+ struct list_head list;
+ struct rcu_head rcu;
+ void (*invalidate)(unsigned long cookie);
+ struct fa_flow *reverse;
+ struct fa_tuple reply;
+ unsigned long flags;
+ unsigned long cookie;
+ struct fa_tuple tuple;
+ struct net_device *in, *out;
+ netdevice_tracker in_tracker, out_tracker;
+ unsigned long lastused;
+ u64 pending_hits;
+ u32 hits;
+ u8 pool_in, pool_out, nh, l4proto, in_port, out_port;
+ bool installed;
+};
+
+struct fa_mtu_route {
+ struct hlist_node node;
+ u32 dst, tb_id;
+ int prefix_len;
+};
+
+struct bcm_ns_fa {
+ struct fault_attr fail_table;
+ bool fail_table_write_only;
+ unsigned long fault_injections;
+ struct brcm_fa_control control;
+ struct net_device *conduit;
+ void __iomem *regs;
+ struct mutex lock; /* Flow blocks, table port, and software ownership. */
+ spinlock_t learn_lock; /* RX NAPI versus flowtable workqueue. */
+ struct fa_learned *learned;
+ unsigned int learn_next;
+ unsigned int nr_flows;
+ struct fa_flow *flows[FA_SLOTS];
+ struct fa_mac pool[FA_POOLS], nh[FA_NEXTHOPS];
+ struct fa_neighbour neighbours[FA_NEXTHOPS];
+ unsigned int neighbour_next;
+ struct list_head block_list;
+ struct list_head entries; /* RCU readers in NAPI, writers under lock. */
+ struct work_struct control_work;
+ unsigned long control_generation;
+ unsigned long invalidate_generation;
+ unsigned long pending_generation;
+ bool control_pending;
+ bool tcp_ready;
+ bool stopping;
+ bool route_mtu_ok;
+ bool mtu_tracking_failed;
+ DECLARE_HASHTABLE(mtu_routes, FA_MTU_HASH_BITS);
+ unsigned int nr_mtu_routes;
+ unsigned long invalidations;
+ unsigned long neighbour_check;
+ struct notifier_block netdev_nb;
+ struct notifier_block fib_nb;
+ struct notifier_block netevent_nb;
+ struct dentry *debugfs;
+ bool failed;
+ bool enabled;
+};
+
+/* Serialize control changes with the NAPI stop request. */
+static void fa_control_update(struct bcm_ns_fa *fa, u32 set, u32 clear)
+{
+ u32 val;
+
+ spin_lock_bh(&fa->learn_lock);
+ val = readl(fa->regs + FA_CTL);
+ writel((val | set) & ~clear, fa->regs + FA_CTL);
+ readl(fa->regs + FA_CTL);
+ spin_unlock_bh(&fa->learn_lock);
+}
+
+/* Caller holds learn_lock. Stop lookups without bypassing tag relocation:
+ * global bypass wires the switch directly to DMA and breaks brcm-fa frames.
+ */
+static void fa_lookup_update(struct bcm_ns_fa *fa)
+{
+ bool enabled = fa->enabled && fa->tcp_ready && fa->route_mtu_ok && !fa->failed &&
+ !fa->control_pending && !fa->stopping;
+
+ writel(enabled ? ETH_P_IP | ETH_P_IPV6 << 16 : 0, fa->regs + FA_L2_ETHERTYPE);
+ readl(fa->regs + FA_L2_ETHERTYPE);
+}
+
+static void fa_reply_tuple(const struct fa_rule *rule, struct fa_tuple *reply)
+{
+ *reply = (struct fa_tuple){ rule->tuple.dst, rule->tuple.src,
+ rule->tuple.dport, rule->tuple.sport };
+ if (rule->direction == FA_DIR_DNAT) {
+ if (rule->action & FA_ACT_REMAP_IP)
+ reply->src = htonl(rule->remap_ip);
+ if (rule->action & FA_ACT_REMAP_PORT)
+ reply->sport = htons(rule->remap_port);
+ } else {
+ if (rule->action & FA_ACT_REMAP_IP)
+ reply->dst = htonl(rule->remap_ip);
+ if (rule->action & FA_ACT_REMAP_PORT)
+ reply->dport = htons(rule->remap_port);
+ }
+}
+
+static bool fa_control_match(const struct fa_flow *entry,
+ const struct fa_tuple *tuple, unsigned int port,
+ bool ports_valid)
+{
+ size_t len = ports_valid ? sizeof(*tuple) : offsetof(struct fa_tuple, sport);
+
+ return entry->l4proto == IPPROTO_TCP && entry->in_port == port &&
+ !memcmp(&entry->tuple, tuple, len);
+}
+
+/* Switch traps bypass FA lookup. Stop subsequent data lookups before Linux
+ * can reuse their NAT tuple; retire both hardware directions in process context.
+ * The publication lock also closes the race with an in-flight insertion.
+ */
+static void fa_tcp_control(struct bcm_ns_fa *fa, const struct fa_tuple *tuple,
+ unsigned int port, bool ports_valid)
+{
+ struct fa_flow *entry;
+ bool found = false;
+
+ rcu_read_lock();
+ spin_lock_bh(&fa->learn_lock);
+ if (fa->stopping)
+ goto out;
+ fa->control_generation++;
+ list_for_each_entry_rcu(entry, &fa->entries, list) {
+ if (!fa_control_match(entry, tuple, port, ports_valid) ||
+ test_bit(FA_FLOW_CLOSING, &entry->flags))
+ continue;
+ set_bit(FA_FLOW_CLOSING, &entry->flags);
+ found = true;
+ }
+ if (found) {
+ fa->pending_generation++;
+ fa->control_pending = true;
+ fa_lookup_update(fa);
+ queue_work(system_dfl_wq, &fa->control_work);
+ }
+out:
+ spin_unlock_bh(&fa->learn_lock);
+ rcu_read_unlock();
+}
+
+static int fa_wait(struct bcm_ns_fa *fa)
+{
+ u32 val;
+
+ return readl_poll_timeout(fa->regs + FA_DEBUG_STATUS, val,
+ !(val & FA_MEM_BUSY), 1, 1000);
+}
+
+static int fa_wait_index(struct bcm_ns_fa *fa, unsigned int next)
+{
+ u32 val;
+ int err;
+
+ err = readl_poll_timeout(fa->regs + FA_MEM_CTL, val,
+ FIELD_GET(FA_MEM_CURRENT_INDEX, val) == (next & 0x3ff) &&
+ !(readl(fa->regs + FA_DEBUG_STATUS) & FA_MEM_BUSY),
+ 1, 1000);
+ if (err)
+ netdev_err(fa->conduit, "FA index wait: expected %u control %#x busy %#x\n",
+ next, val, readl(fa->regs + FA_DEBUG_STATUS));
+ return err;
+}
+
+static DECLARE_FAULT_ATTR(fa_fail_table_default);
+
+static int fa_table_complete(struct bcm_ns_fa *fa, unsigned int next, bool write)
+{
+ int err = fa_wait_index(fa, next);
+
+ if (!err && IS_ENABLED(CONFIG_BCM_NS_FA_FAULT_INJECTION) &&
+ (!READ_ONCE(fa->fail_table_write_only) || write) &&
+ should_fail(&fa->fail_table, 1)) {
+ fa->fault_injections++;
+ return -ETIMEDOUT;
+ }
+ return err;
+}
+
+static void fa_fail(struct bcm_ns_fa *fa);
+
+struct fa_table_access {
+ u32 mac_control;
+ bool paused;
+};
+
+static int fa_table_begin(struct bcm_ns_fa *fa, struct fa_table_access *access)
+{
+ struct bgmac *bgmac = netdev_priv(fa->conduit);
+ u32 val;
+ int err;
+
+ access->paused = false;
+ if (!netif_running(fa->conduit))
+ return 0;
+ if (readl(fa->regs + FA_CTL) & FA_CTL_BYPASS) {
+ /* Initialization and final release use global bypass. */
+ fsleep(20);
+ return 0;
+ }
+ access->mac_control = bgmac_read(bgmac, BGMAC_DEV_CTL);
+ access->paused = true;
+ /* Keep automatic flow control and request a pause from the CPU too. */
+ val = (access->mac_control & ~BGMAC_DC_FCM_MASK) |
+ FIELD_PREP(BGMAC_DC_FCM_MASK, 2) | BGMAC_DC_CFCO;
+ bgmac_write(bgmac, BGMAC_DEV_CTL, val);
+ err = read_poll_timeout(bgmac_read, val, val & BGMAC_DS_PO, 1, 1000,
+ false, bgmac, BGMAC_DEV_STATUS);
+ if (err)
+ return err;
+ /* Drain a frame already on the switch link before taking the shared
+ * table port. Do not change the FA packet path in the middle of a frame.
+ */
+ fsleep(20);
+ return 0;
+}
+
+static void fa_table_end(struct bcm_ns_fa *fa, struct fa_table_access *access)
+{
+ struct bgmac *bgmac = netdev_priv(fa->conduit);
+
+ if (access->paused) {
+ bgmac_write(bgmac, BGMAC_DEV_CTL, access->mac_control);
+ bgmac_read(bgmac, BGMAC_DEV_CTL);
+ }
+}
+
+/* Reading DATA0 triggers a transaction, including the final read below. */
+static int fa_read(struct bcm_ns_fa *fa, unsigned int table, unsigned int index,
+ u32 data[8])
+{
+ struct fa_table_access access;
+ int i, err;
+
+ err = fa_table_begin(fa, &access);
+ if (err)
+ goto restore;
+ err = fa_wait(fa);
+ if (err)
+ goto restore;
+ writel(FA_MEM_READ | FIELD_PREP(FA_MEM_TABLE, table) | index,
+ fa->regs + FA_MEM_CTL);
+ err = fa_wait_index(fa, index);
+ if (err)
+ goto out;
+ readl(fa->regs + FA_DATA(0));
+ err = fa_table_complete(fa, index + 1, false);
+ if (err)
+ goto out;
+ for (i = 7; i >= 0; i--)
+ data[i] = readl(fa->regs + FA_DATA(i));
+ err = fa_table_complete(fa, index + 2, false);
+out:
+ /* Release the software read port so packet hit counters can update. */
+ writel(0, fa->regs + FA_MEM_CTL);
+ readl(fa->regs + FA_MEM_CTL);
+restore:
+ /* Gate lookups while the switch is still paused on an uncertain access. */
+ if (err)
+ fa_fail(fa);
+ fa_table_end(fa, &access);
+ return err;
+}
+
+static int fa_write(struct bcm_ns_fa *fa, unsigned int table, unsigned int index,
+ const u32 data[8])
+{
+ struct fa_table_access access;
+ int i, err;
+
+ err = fa_table_begin(fa, &access);
+ if (err)
+ goto out;
+ err = fa_wait(fa);
+ if (err)
+ goto out;
+ writel(FIELD_PREP(FA_MEM_TABLE, table) | index, fa->regs + FA_MEM_CTL);
+ err = fa_wait_index(fa, index);
+ if (err)
+ goto out;
+ /* DATA0 commits the complete entry. Never write the valid word last. */
+ for (i = 7; i >= 0; i--)
+ writel(data[i], fa->regs + FA_DATA(i));
+ err = fa_table_complete(fa, index + 1, true);
+out:
+ if (err)
+ fa_fail(fa);
+ fa_table_end(fa, &access);
+ return err;
+}
+
+/* On an uncertain table transaction stop all acceleration. Do not recycle
+ * any table resources until reprobe has reinitialized the hardware.
+ */
+static void fa_fail(struct bcm_ns_fa *fa)
+{
+ if (!fa->failed)
+ netdev_err(fa->conduit, "FA table timeout; offload disabled until reprobe\n");
+ spin_lock_bh(&fa->learn_lock);
+ fa->failed = true;
+ fa_lookup_update(fa);
+ spin_unlock_bh(&fa->learn_lock);
+}
+
+static void fa_mac_encode(const u8 *addr, bool external, bool next_hop, u32 data[8])
+{
+ u64 mac = ether_addr_to_u64(addr);
+
+ if (next_hop) {
+ /* Ethernet II with no VLAN tag; DA spans DATA0 through DATA2. */
+ data[0] = FIELD_PREP(FA_NH0_TAG_OP, FA_NH_TAG_NONE) |
+ FIELD_PREP(FA_NH0_MAC_LO, mac & FIELD_MAX(FA_NH0_MAC_LO));
+ data[1] = mac >> 13;
+ data[2] = FIELD_PREP(FA_NH2_MAC_HI, mac >> 45);
+ } else {
+ data[0] = FIELD_PREP(FA_POOL0_MAC_LO, mac & FIELD_MAX(FA_POOL0_MAC_LO)) |
+ FIELD_PREP(FA_POOL0_EXTERNAL, external);
+ data[1] = FIELD_PREP(FA_POOL1_MAC_HI, mac >> 31);
+ }
+}
+
+static int fa_mac_get(struct bcm_ns_fa *fa, const u8 *addr, bool external,
+ bool next_hop)
+{
+ struct fa_mac *macs = next_hop ? fa->nh : fa->pool;
+ unsigned int count = next_hop ? FA_NEXTHOPS : FA_POOLS;
+ u32 data[8] = {};
+ int i, free = -1, err;
+
+ for (i = 0; i < count; i++) {
+ if (!macs[i].refs) {
+ if (free < 0)
+ free = i;
+ continue;
+ }
+ if (!ether_addr_equal(addr, macs[i].addr))
+ continue;
+ if (!next_hop && external != macs[i].external)
+ return -EOPNOTSUPP;
+ macs[i].refs++;
+ return i;
+ }
+ if (free < 0)
+ return -ENOSPC;
+
+ fa_mac_encode(addr, external, next_hop, data);
+ err = fa_write(fa, next_hop ? FA_NH_TABLE : FA_POOL_TABLE, free, data);
+ if (err) {
+ fa_fail(fa);
+ return err;
+ }
+ ether_addr_copy(macs[free].addr, addr);
+ macs[free].external = external;
+ macs[free].refs = 1;
+ return free;
+}
+
+static int fa_mac_put(struct bcm_ns_fa *fa, unsigned int index, bool next_hop)
+{
+ struct fa_mac *mac = next_hop ? &fa->nh[index] : &fa->pool[index];
+ u32 data[8] = {};
+ int err;
+
+ if (--mac->refs)
+ return 0;
+ err = fa_write(fa, next_hop ? FA_NH_TABLE : FA_POOL_TABLE, index, data);
+ if (err)
+ fa_fail(fa);
+ return err;
+}
+
+static bool fa_receive_tcp_control(struct bcm_ns_fa *fa, const struct iphdr *ip,
+ unsigned int total, unsigned int linear,
+ unsigned int port)
+{
+ struct fa_tuple tuple;
+ const struct tcphdr *tcp;
+
+ tuple.src = get_unaligned(&ip->saddr);
+ tuple.dst = get_unaligned(&ip->daddr);
+ if (ip->ihl != 5 || (ip->frag_off & htons(IP_MF | IP_OFFSET))) {
+ fa_tcp_control(fa, &tuple, port, false);
+ return true;
+ }
+ tcp = (void *)(ip + 1);
+ if (total < sizeof(*ip) + sizeof(*tcp) || linear < sizeof(*ip) + sizeof(*tcp) ||
+ tcp->doff < 5 || total < sizeof(*ip) + tcp->doff * 4)
+ return true;
+ if (!tcp->syn && !tcp->fin && !tcp->rst)
+ return false;
+ tuple.sport = get_unaligned(&tcp->source);
+ tuple.dport = get_unaligned(&tcp->dest);
+ fa_tcp_control(fa, &tuple, port, true);
+ return true;
+}
+
+static void fa_control_receive(void *priv, struct sk_buff *skb, unsigned int port)
+{
+ struct bcm_ns_fa *fa = priv;
+ const struct iphdr *ip = (void *)skb->data;
+ unsigned int ihl, total;
+
+ if (READ_ONCE(fa->stopping) || eth_hdr(skb)->h_proto != htons(ETH_P_IP) ||
+ skb_headlen(skb) < sizeof(*ip))
+ return;
+ ihl = ip->ihl * 4;
+ total = ntohs(ip->tot_len);
+ if (ip->version != 4 || ip->protocol != IPPROTO_TCP || ihl < sizeof(*ip) ||
+ total < ihl || total > skb->len || skb_headlen(skb) < ihl ||
+ ip_compute_csum(ip, ihl))
+ return;
+ fa_receive_tcp_control(fa, ip, total, skb_headlen(skb), port);
+}
+
+static struct net_device *fa_control_conduit(struct bcm_ns_fa *fa)
+{
+ struct dsa_port *data_cpu = fa->conduit->dsa_ptr, *control_cpu;
+
+ if (!data_cpu)
+ return NULL;
+ control_cpu = dsa_to_port(data_cpu->ds, 5);
+ return control_cpu && dsa_port_is_cpu(control_cpu) ? control_cpu->conduit : NULL;
+}
+
+/* The hardware supplies the exact 97-bit tuple CRC in its miss tag. Learning
+ * it avoids assumptions about serial CRC bit order and seed endianness.
+ * Cache eviction only prevents new offloads; installed entries own their slots.
+ */
+void bcm_ns_fa_rx(struct bcm_ns_fa *fa, struct sk_buff *skb)
+{
+ struct fa_learned *learn;
+ const struct ethhdr *eth;
+ const struct iphdr *ip;
+ const __be16 *ports;
+ struct fa_tuple tuple;
+ unsigned int offset = FA_TAG_LEN + ETH_HLEN, ihl, total;
+ u8 *tag;
+
+ if (!fa || READ_ONCE(fa->stopping) || skb_headlen(skb) < offset + sizeof(*ip))
+ return;
+ tag = skb->data;
+ if (tag[0] >> 5 != FA_MISS_OPCODE && tag[0] >> 5 != 0)
+ return;
+ eth = (void *)(tag + FA_TAG_LEN);
+ ip = (void *)(tag + offset);
+ ihl = ip->ihl * 4;
+ total = ntohs(ip->tot_len);
+ if (eth->h_proto != htons(ETH_P_IP) || ip->version != 4 || ihl < sizeof(*ip) ||
+ (ip->protocol != IPPROTO_TCP && ip->protocol != IPPROTO_UDP) ||
+ total < ihl || total > skb->len - offset || skb_headlen(skb) < offset + ihl ||
+ ip_compute_csum(ip, ihl))
+ return;
+ if (ip->protocol == IPPROTO_TCP &&
+ fa_receive_tcp_control(fa, ip, total, skb_headlen(skb) - offset, tag[3] & 0x1f))
+ return;
+ if (ihl != sizeof(*ip) || (ip->frag_off & htons(IP_MF | IP_OFFSET)) ||
+ total < ihl + sizeof(struct udphdr) ||
+ skb_headlen(skb) < offset + ihl + sizeof(struct udphdr))
+ return;
+ tuple.src = get_unaligned(&ip->saddr);
+ tuple.dst = get_unaligned(&ip->daddr);
+ ports = (void *)(ip + 1);
+ tuple.sport = get_unaligned(ports);
+ tuple.dport = get_unaligned(ports + 1);
+ /* Exception tags are delivered unchanged and contain no usable hash. */
+ if (tag[0] >> 5 != FA_MISS_OPCODE || tag[2])
+ return;
+ spin_lock_bh(&fa->learn_lock);
+ if (fa->control_pending) {
+ spin_unlock_bh(&fa->learn_lock);
+ return;
+ }
+ learn = &fa->learned[fa->learn_next++ % FA_SLOTS];
+ learn->tuple = tuple;
+ learn->l4proto = ip->protocol;
+ learn->generation = ip->protocol == IPPROTO_TCP ? fa->control_generation :
+ fa->invalidate_generation;
+ ether_addr_copy(learn->mac, eth->h_dest);
+ learn->hash = tag[1];
+ learn->port = tag[3] & 0x1f;
+ learn->valid = true;
+ spin_unlock_bh(&fa->learn_lock);
+}
+EXPORT_SYMBOL_GPL(bcm_ns_fa_rx);
+
+static bool fa_learn_matches(const struct fa_learned *learn,
+ const struct fa_rule *rule, unsigned long generation)
+{
+ return learn->valid && learn->l4proto == rule->l4proto &&
+ learn->generation == generation &&
+ !memcmp(&learn->tuple, &rule->tuple, sizeof(learn->tuple));
+}
+
+static int fa_learn_lookup(struct bcm_ns_fa *fa, const struct fa_rule *rule,
+ struct fa_learned *result)
+{
+ int i, err = -EAGAIN;
+
+ spin_lock_bh(&fa->learn_lock);
+ for (i = 0; i < FA_SLOTS; i++) {
+ const struct fa_learned *l = &fa->learned[i];
+
+ if (!fa_learn_matches(l, rule, rule->l4proto == IPPROTO_TCP ?
+ fa->control_generation : fa->invalidate_generation))
+ continue;
+ if (l->port != dsa_port_from_netdev(rule->in)->index ||
+ !ether_addr_equal(l->mac, rule->in->dev_addr))
+ continue;
+ *result = *l;
+ err = 0;
+ }
+ spin_unlock_bh(&fa->learn_lock);
+ return err;
+}
+
+static bool fa_port(struct bcm_ns_fa *fa, struct net_device *dev)
+{
+ struct dsa_port *dp;
+
+ if (!dev || !dsa_user_dev_check(dev) || dev_net(dev) != &init_net ||
+ !netif_running(dev) || !netif_carrier_ok(dev) || fa_has_upper_dev_rcu(dev))
+ return false;
+ dp = dsa_port_from_netdev(dev);
+ return dp->ds->index == 0 && dp->index < 8 &&
+ dsa_port_to_conduit(dp) == fa->conduit &&
+ dp->cpu_dp->index == 8 &&
+ dp->ds->dst->tag_ops->proto == DSA_TAG_PROTO_BRCM_FA;
+}
+
+static int fa_parse(struct bcm_ns_fa *fa, struct flow_cls_offload *cls,
+ struct fa_rule *rule)
+{
+ struct flow_rule *flow = flow_cls_offload_flow_rule(cls);
+ struct flow_match_ipv4_addrs ipv4;
+ struct flow_match_control control;
+ struct flow_match_basic basic;
+ struct flow_match_ports ports;
+ struct flow_match_meta meta;
+ const struct flow_action_entry *act;
+ struct dsa_port *dp;
+ struct net_device *control_dev;
+ u64 allowed;
+ bool csum = false, is_tcp;
+ u32 csum_flags;
+ int i, err;
+
+ allowed = BIT_ULL(FLOW_DISSECTOR_KEY_META) | BIT_ULL(FLOW_DISSECTOR_KEY_CONTROL) |
+ BIT_ULL(FLOW_DISSECTOR_KEY_BASIC) | BIT_ULL(FLOW_DISSECTOR_KEY_IPV4_ADDRS) |
+ BIT_ULL(FLOW_DISSECTOR_KEY_PORTS);
+ if (flow_rule_match_key(flow, FLOW_DISSECTOR_KEY_TCP)) {
+ struct flow_match_tcp tcp;
+
+ allowed |= BIT_ULL(FLOW_DISSECTOR_KEY_TCP);
+ flow_rule_match_tcp(flow, &tcp);
+ if (tcp.key->flags || tcp.mask->flags !=
+ cpu_to_be16(be32_to_cpu(TCP_FLAG_FIN | TCP_FLAG_RST) >> 16))
+ return -EOPNOTSUPP;
+ }
+ if (cls->common.chain_index || flow->match.dissector->used_keys != allowed)
+ return -EOPNOTSUPP;
+ flow_rule_match_control(flow, &control);
+ flow_rule_match_basic(flow, &basic);
+ flow_rule_match_ipv4_addrs(flow, &ipv4);
+ flow_rule_match_ports(flow, &ports);
+ flow_rule_match_meta(flow, &meta);
+ if (control.key->addr_type != FLOW_DISSECTOR_KEY_IPV4_ADDRS ||
+ control.mask->addr_type != 0xffff || control.mask->flags ||
+ basic.key->n_proto != htons(ETH_P_IP) ||
+ basic.mask->n_proto != htons(0xffff) ||
+ (basic.key->ip_proto != IPPROTO_UDP && basic.key->ip_proto != IPPROTO_TCP) ||
+ basic.mask->ip_proto != 0xff ||
+ ipv4.mask->src != htonl(~0U) || ipv4.mask->dst != htonl(~0U) ||
+ ports.mask->src != htons(0xffff) || ports.mask->dst != htons(0xffff) ||
+ meta.mask->ingress_ifindex != ~0U || meta.mask->ingress_iftype)
+ return -EOPNOTSUPP;
+
+ is_tcp = basic.key->ip_proto == IPPROTO_TCP;
+ if (flow_rule_match_key(flow, FLOW_DISSECTOR_KEY_TCP) != is_tcp)
+ return -EOPNOTSUPP;
+ control_dev = fa_control_conduit(fa);
+ if (!fa->tcp_ready || !control_dev || !netif_running(control_dev) ||
+ !netif_carrier_ok(control_dev) || control_dev->mtu < ETH_DATA_LEN)
+ return -EOPNOTSUPP;
+ csum_flags = TCA_CSUM_UPDATE_FLAG_IPV4HDR |
+ (is_tcp ? TCA_CSUM_UPDATE_FLAG_TCP : TCA_CSUM_UPDATE_FLAG_UDP);
+ rule->l4proto = basic.key->ip_proto;
+ rule->in = dev_get_by_index_rcu(&init_net, meta.key->ingress_ifindex);
+ if (!fa_port(fa, rule->in))
+ return -EOPNOTSUPP;
+ rule->tuple.src = ipv4.key->src;
+ rule->tuple.dst = ipv4.key->dst;
+ rule->tuple.sport = ports.key->src;
+ rule->tuple.dport = ports.key->dst;
+ rule->direction = FA_DIR_UNSET;
+
+ flow_action_for_each(i, act, &flow->action) {
+ switch (act->id) {
+ case FLOW_ACTION_MANGLE:
+ err = fa_mangle(rule, act);
+ if (err)
+ return err;
+ break;
+ case FLOW_ACTION_CSUM:
+ if (csum || act->csum_flags != csum_flags)
+ return -EOPNOTSUPP;
+ csum = true;
+ break;
+ case FLOW_ACTION_REDIRECT:
+ if (rule->out || i != flow->action.num_entries - 1)
+ return -EOPNOTSUPP;
+ rule->out = act->dev;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ }
+ if (!rule->action || !csum || !fa_port(fa, rule->out) ||
+ rule->in == rule->out || rule->in->mtu != rule->out->mtu ||
+ rule->in->mtu != ETH_DATA_LEN ||
+ ether_addr_equal(rule->in->dev_addr, rule->out->dev_addr) ||
+ !ether_addr_equal(rule->out->dev_addr, rule->eth + ETH_ALEN) ||
+ !is_valid_ether_addr(rule->eth) ||
+ memchr_inv(rule->eth_mask, 0xff, sizeof(rule->eth_mask)))
+ return -EOPNOTSUPP;
+ if (dsa_port_from_netdev(rule->in)->ds != dsa_port_from_netdev(rule->out)->ds)
+ return -EOPNOTSUPP;
+
+ /* The lookup key has no input port or VLAN. Limit the initial driver
+ * to two standalone router ports with distinct router MAC addresses.
+ */
+ dsa_switch_for_each_user_port(dp, dsa_port_from_netdev(rule->in)->ds)
+ if (dsa_port_to_conduit(dp) == fa->conduit &&
+ netif_running(dp->user) && dp->user != rule->in && dp->user != rule->out)
+ return -EOPNOTSUPP;
+ return 0;
+}
+
+static int fa_find(struct bcm_ns_fa *fa, unsigned long cookie)
+{
+ int i;
+
+ for (i = 0; i < FA_SLOTS; i++)
+ if (fa->flows[i] && fa->flows[i]->cookie == cookie)
+ return i;
+ return -ENOENT;
+}
+
+/* Hardware traffic must still drive Linux neighbour reachability probes. */
+static void fa_neighbour_activity(struct bcm_ns_fa *fa)
+{
+ unsigned int i, j;
+
+ if (fa->failed || fa->stopping || time_before(jiffies, fa->neighbour_check + HZ))
+ return;
+ fa->neighbour_check = jiffies;
+ for (i = 0; i < ARRAY_SIZE(fa->neighbours); i++) {
+ struct fa_neighbour cached;
+ struct net_device *dev;
+ struct neighbour *neigh;
+ netdevice_tracker tracker;
+ bool owned = false;
+
+ spin_lock_bh(&fa->learn_lock);
+ cached = fa->neighbours[i];
+ spin_unlock_bh(&fa->learn_lock);
+ if (!cached.valid)
+ continue;
+ for (j = 0; j < FA_NEXTHOPS; j++)
+ if (fa->nh[j].refs && ether_addr_equal(fa->nh[j].addr, cached.mac)) {
+ owned = true;
+ break;
+ }
+ if (!owned)
+ continue;
+ dev = netdev_get_by_index(&init_net, cached.ifindex, &tracker, GFP_KERNEL);
+ if (!dev)
+ continue;
+ rcu_read_lock();
+ owned = fa_port(fa, dev);
+ rcu_read_unlock();
+ if (owned) {
+ neigh = ipv4_neigh_lookup(dev, &cached.address);
+ if (neigh) {
+ neigh_event_send(neigh, NULL);
+ neigh_release(neigh);
+ }
+ }
+ netdev_put(dev, &tracker);
+ }
+}
+
+static u32 fa_hits(const u32 data[8])
+{
+ return FIELD_GET(FA_FLOW0_HITS_LO, data[0]) |
+ FIELD_GET(FA_FLOW1_HITS_HI, data[1]) << 29;
+}
+
+static int fa_counter_read(struct bcm_ns_fa *fa, unsigned int index)
+{
+ struct fa_flow *entry = fa->flows[index];
+ u32 data[8], hits, delta;
+ int err;
+
+ err = fa_read(fa, FA_FLOW_TABLE, index, data);
+ if (err) {
+ fa_fail(fa);
+ return err;
+ }
+ hits = fa_hits(data);
+ delta = hits - entry->hits;
+ entry->hits = hits;
+ entry->pending_hits += delta;
+ if (delta) {
+ entry->lastused = jiffies;
+ fa_neighbour_activity(fa);
+ }
+ return 0;
+}
+
+static int fa_uninstall(struct bcm_ns_fa *fa, unsigned int index)
+{
+ struct fa_flow *entry = fa->flows[index];
+ u32 data[8] = {};
+ int err;
+
+ if (!entry->installed)
+ return 0;
+ if (!fa->failed) {
+ /* Preserve the final packet delta for the next flowtable poll. */
+ fa_counter_read(fa, index);
+ if (!fa->failed) {
+ err = fa_write(fa, FA_FLOW_TABLE, index, data);
+ if (err)
+ fa_fail(fa);
+ }
+ if (!fa->failed)
+ fa_mac_put(fa, entry->pool_in, false);
+ if (!fa->failed)
+ fa_mac_put(fa, entry->pool_out, false);
+ if (!fa->failed)
+ fa_mac_put(fa, entry->nh, true);
+ }
+ entry->installed = false;
+ if (!--fa->nr_flows && !fa->failed)
+ fa_control_update(fa, FA_CTL_NO_DA_CHECK, 0);
+ return fa->failed ? -EIO : 0;
+}
+
+static void fa_control_state_change(void *priv, bool ready)
+{
+ struct bcm_ns_fa *fa = priv;
+ unsigned int i;
+
+ mutex_lock(&fa->lock);
+ spin_lock_bh(&fa->learn_lock);
+ fa->tcp_ready = ready && !READ_ONCE(fa->stopping);
+ if (!fa->tcp_ready) {
+ fa->control_generation++;
+ fa->invalidate_generation++;
+ memset(fa->learned, 0, sizeof(*fa->learned) * FA_SLOTS);
+ }
+ fa_lookup_update(fa);
+ spin_unlock_bh(&fa->learn_lock);
+ if (!fa->tcp_ready)
+ for (i = 0; i < FA_SLOTS; i++) {
+ struct fa_flow *entry = fa->flows[i];
+
+ if (!entry)
+ continue;
+ set_bit(FA_FLOW_CLOSING, &entry->flags);
+ rcu_read_lock();
+ entry->invalidate(entry->cookie);
+ rcu_read_unlock();
+ fa_uninstall(fa, i);
+ }
+ mutex_unlock(&fa->lock);
+}
+
+static int fa_destroy(struct bcm_ns_fa *fa, unsigned int index)
+{
+ struct fa_flow *entry = fa->flows[index];
+ int err = fa_uninstall(fa, index);
+
+ if (entry->reverse)
+ entry->reverse->reverse = NULL;
+ spin_lock_bh(&fa->learn_lock);
+ list_del_rcu(&entry->list);
+ spin_unlock_bh(&fa->learn_lock);
+ netdev_put(entry->in, &entry->in_tracker);
+ netdev_put(entry->out, &entry->out_tracker);
+ fa->flows[index] = NULL;
+ kfree_rcu(entry, rcu);
+ return err;
+}
+
+/* FIB notifications identify the preferred alias for each table/prefix.
+ * Retain only low-MTU prefixes, rather than dumping the FIB after changes.
+ * All accesses, including the registration retry reset, hold learn_lock.
+ */
+static void fa_mtu_route_update(struct bcm_ns_fa *fa, unsigned long event,
+ const struct fib_entry_notifier_info *info)
+{
+ struct fa_mtu_route *route;
+ u32 key = info->dst ^ info->tb_id ^ info->dst_len;
+ bool low = event != FIB_EVENT_ENTRY_DEL && info->fi &&
+ info->fi->fib_mtu && info->fi->fib_mtu < ETH_DATA_LEN;
+
+ hash_for_each_possible(fa->mtu_routes, route, node, key) {
+ if (route->dst != info->dst || route->tb_id != info->tb_id ||
+ route->prefix_len != info->dst_len)
+ continue;
+ if (!low) {
+ hash_del(&route->node);
+ kfree(route);
+ fa->nr_mtu_routes--;
+ }
+ goto out;
+ }
+ if (!low)
+ goto out;
+ if (fa->nr_mtu_routes == FA_MTU_ROUTES_MAX) {
+ fa->mtu_tracking_failed = true;
+ goto out;
+ }
+ route = kzalloc_obj(*route, GFP_ATOMIC);
+ if (!route) {
+ /* Never reject a route because an optional offload ran out of memory. */
+ fa->mtu_tracking_failed = true;
+ goto out;
+ }
+ route->dst = info->dst;
+ route->tb_id = info->tb_id;
+ route->prefix_len = info->dst_len;
+ hash_add(fa->mtu_routes, &route->node, key);
+ fa->nr_mtu_routes++;
+out:
+ fa->route_mtu_ok = !fa->nr_mtu_routes && !fa->mtu_tracking_failed;
+}
+
+static void fa_fib_reset(struct notifier_block *nb)
+{
+ struct bcm_ns_fa *fa = container_of(nb, struct bcm_ns_fa, fib_nb);
+ struct fa_mtu_route *route;
+ struct hlist_node *tmp;
+ unsigned int bucket;
+
+ spin_lock_bh(&fa->learn_lock);
+ hash_for_each_safe(fa->mtu_routes, bucket, tmp, route, node) {
+ hash_del(&route->node);
+ kfree(route);
+ }
+ fa->nr_mtu_routes = 0;
+ fa->mtu_tracking_failed = false;
+ fa->route_mtu_ok = true;
+ spin_unlock_bh(&fa->learn_lock);
+}
+
+/* Caller holds RCU and learn_lock. Stop hardware before notifying owners. */
+static void fa_invalidate_locked(struct bcm_ns_fa *fa)
+{
+ struct fa_flow *entry;
+
+ fa->control_generation++;
+ fa->pending_generation++;
+ fa->control_pending = true;
+ fa->invalidations++;
+ fa->invalidate_generation++;
+ fa_lookup_update(fa);
+ list_for_each_entry_rcu(entry, &fa->entries, list) {
+ set_bit(FA_FLOW_CLOSING, &entry->flags);
+ entry->invalidate(entry->cookie);
+ }
+ queue_work(system_dfl_wq, &fa->control_work);
+}
+
+static void fa_invalidate(struct bcm_ns_fa *fa)
+{
+ rcu_read_lock();
+ spin_lock_bh(&fa->learn_lock);
+ if (!fa->stopping)
+ fa_invalidate_locked(fa);
+ spin_unlock_bh(&fa->learn_lock);
+ rcu_read_unlock();
+}
+
+static int fa_fib_event(struct notifier_block *nb, unsigned long event, void *ptr)
+{
+ struct bcm_ns_fa *fa = container_of(nb, struct bcm_ns_fa, fib_nb);
+ struct fib_notifier_info *info = ptr;
+ bool route = false;
+
+ if (info->family != AF_INET)
+ return NOTIFY_DONE;
+ /* Deliberately invalidate all FA-owned flows for every IPv4 route,
+ * rule or nexthop change, including unrelated and local-table routes.
+ * The offload rule lacks the complete policy context needed to prove
+ * that an event cannot affect an installed path.
+ */
+ switch (event) {
+ case FIB_EVENT_ENTRY_REPLACE:
+ case FIB_EVENT_ENTRY_ADD:
+ case FIB_EVENT_ENTRY_APPEND:
+ case FIB_EVENT_ENTRY_DEL:
+ route = true;
+ break;
+ case FIB_EVENT_RULE_ADD:
+ case FIB_EVENT_RULE_DEL:
+ case FIB_EVENT_NH_ADD:
+ case FIB_EVENT_NH_DEL:
+ break;
+ default:
+ return NOTIFY_DONE;
+ }
+ rcu_read_lock();
+ spin_lock_bh(&fa->learn_lock);
+ if (!fa->stopping) {
+ if (route)
+ fa_mtu_route_update(fa, event, ptr);
+ fa_invalidate_locked(fa);
+ }
+ spin_unlock_bh(&fa->learn_lock);
+ rcu_read_unlock();
+ return NOTIFY_DONE;
+}
+
+static int fa_netevent(struct notifier_block *nb, unsigned long event, void *ptr)
+{
+ struct bcm_ns_fa *fa = container_of(nb, struct bcm_ns_fa, netevent_nb);
+ struct fa_neighbour *cached = NULL;
+ struct neighbour *neighbour = ptr;
+ bool invalidate = false;
+ unsigned int i;
+ char mac[ETH_ALEN];
+ __be32 address;
+ u8 state;
+
+ if (event == NETEVENT_IPV4_MPATH_HASH_UPDATE ||
+ event == NETEVENT_IPV4_FWD_UPDATE_PRIORITY_UPDATE) {
+ if (ptr == &init_net)
+ fa_invalidate(fa);
+ return NOTIFY_DONE;
+ }
+ if (event == NETEVENT_REDIRECT) {
+ struct netevent_redirect *redirect = ptr;
+
+ if (net_eq(dev_net(redirect->new->dev), &init_net))
+ fa_invalidate(fa);
+ return NOTIFY_DONE;
+ }
+ if (event != NETEVENT_NEIGH_UPDATE || neighbour->tbl->family != AF_INET ||
+ !fa_port(fa, neighbour->dev))
+ return NOTIFY_DONE;
+ state = READ_ONCE(neighbour->nud_state);
+ address = get_unaligned((__be32 *)neighbour->primary_key);
+ neigh_ha_snapshot(mac, neighbour, neighbour->dev);
+ spin_lock_bh(&fa->learn_lock);
+ for (i = 0; i < ARRAY_SIZE(fa->neighbours); i++) {
+ struct fa_neighbour *candidate = &fa->neighbours[i];
+
+ if (candidate->valid && candidate->ifindex == neighbour->dev->ifindex &&
+ candidate->address == address) {
+ cached = candidate;
+ invalidate = !(state & NUD_VALID) || memcmp(candidate->mac, mac, ETH_ALEN);
+ break;
+ }
+ }
+ if (!cached) {
+ cached = &fa->neighbours[fa->neighbour_next++ % FA_NEXTHOPS];
+ invalidate = cached->valid || !(state & NUD_VALID) || !list_empty(&fa->entries);
+ }
+ *cached = (struct fa_neighbour) {
+ .address = address, .ifindex = neighbour->dev->ifindex,
+ .state = state, .valid = true,
+ };
+ memcpy(cached->mac, mac, ETH_ALEN);
+ spin_unlock_bh(&fa->learn_lock);
+ if (invalidate)
+ fa_invalidate(fa);
+ return NOTIFY_DONE;
+}
+
+static void fa_control_work(struct work_struct *work)
+{
+ struct bcm_ns_fa *fa = container_of(work, struct bcm_ns_fa, control_work);
+ unsigned long generation;
+ unsigned int i;
+
+ mutex_lock(&fa->lock);
+ for (;;) {
+ spin_lock_bh(&fa->learn_lock);
+ generation = fa->pending_generation;
+ spin_unlock_bh(&fa->learn_lock);
+ for (i = 0; i < FA_SLOTS; i++) {
+ struct fa_flow *entry = fa->flows[i];
+
+ if (entry && test_bit(FA_FLOW_CLOSING, &entry->flags) && entry->reverse)
+ set_bit(FA_FLOW_CLOSING, &entry->reverse->flags);
+ }
+ for (i = 0; i < FA_SLOTS; i++)
+ if (fa->flows[i] && test_bit(FA_FLOW_CLOSING, &fa->flows[i]->flags))
+ fa_uninstall(fa, i);
+ spin_lock_bh(&fa->learn_lock);
+ if (generation == fa->pending_generation) {
+ memset(fa->learned, 0, sizeof(*fa->learned) * FA_SLOTS);
+ fa->control_pending = false;
+ fa_lookup_update(fa);
+ spin_unlock_bh(&fa->learn_lock);
+ break;
+ }
+ spin_unlock_bh(&fa->learn_lock);
+ }
+ mutex_unlock(&fa->lock);
+}
+
+static void fa_encode(const struct fa_rule *rule, unsigned int pool,
+ unsigned int nh, unsigned int port, u32 data[8])
+{
+ data[1] = FIELD_PREP(FA_FLOW1_ACTION, rule->action) |
+ FIELD_PREP(FA_FLOW1_ROUTER_MAC, pool) |
+ FIELD_PREP(FA_FLOW1_NEXT_HOP, nh) |
+ FIELD_PREP(FA_FLOW1_REMAP_IP_LO,
+ rule->remap_ip & FIELD_MAX(FA_FLOW1_REMAP_IP_LO));
+ data[2] = FIELD_PREP(FA_FLOW2_REMAP_IP_HI, rule->remap_ip >> 17) |
+ FIELD_PREP(FA_FLOW2_REMAP_PORT, rule->remap_port) |
+ FIELD_PREP(FA_FLOW2_DPORT_LO,
+ be16_to_cpu(rule->tuple.dport) & FIELD_MAX(FA_FLOW2_DPORT_LO));
+ data[3] = FIELD_PREP(FA_FLOW3_DPORT_HI, be16_to_cpu(rule->tuple.dport) >> 1) |
+ FIELD_PREP(FA_FLOW3_SPORT, be16_to_cpu(rule->tuple.sport)) |
+ (rule->l4proto == IPPROTO_TCP ? FA_FLOW3_TCP : 0);
+ data[4] = be32_to_cpu(rule->tuple.dst);
+ data[5] = be32_to_cpu(rule->tuple.src);
+ /* Direct to UniMAC2; ingress opcode 1, one destination switch port. */
+ data[6] = FA_FLOW6_VALID | FIELD_PREP(FA_FLOW6_DIRECTION, rule->direction) |
+ FIELD_PREP(FA_FLOW6_OPCODE, FA_FLOW6_OPCODE_INGRESS) |
+ FIELD_PREP(FA_FLOW6_DEST_MAP, BIT(port));
+ data[7] = FA_FLOW7_IPV4;
+}
+
+static int fa_insert(struct bcm_ns_fa *fa, struct flow_cls_offload *cls,
+ const struct fa_rule *parsed)
+{
+ struct fa_rule rule = *parsed;
+ struct fa_learned learned;
+ struct fa_flow *entry;
+ u32 data[8] = {};
+ int i, slot = -1, pi, po, nh, err;
+
+ if (fa->failed)
+ return -EIO;
+ /* A shared block may deliver the same cookie through both ports. */
+ i = fa_find(fa, cls->cookie);
+ if (i >= 0)
+ return test_bit(FA_FLOW_CLOSING, &fa->flows[i]->flags) ? -EOPNOTSUPP : 0;
+ err = fa_learn_lookup(fa, &rule, &learned);
+ if (err)
+ return err;
+ for (i = learned.hash * FA_WAYS; i < (learned.hash + 1) * FA_WAYS; i++) {
+ if (fa->flows[i]) {
+ if (fa->flows[i]->installed && fa->flows[i]->l4proto == rule.l4proto &&
+ !memcmp(&fa->flows[i]->tuple, &rule.tuple, sizeof(rule.tuple)))
+ return -EEXIST;
+ } else if (slot < 0) {
+ slot = i;
+ }
+ }
+ if (slot < 0)
+ return -ENOSPC;
+ entry = kzalloc_obj(*entry);
+ if (!entry)
+ return -ENOMEM;
+ pi = fa_mac_get(fa, learned.mac, rule.direction, false);
+ if (pi < 0) {
+ err = pi;
+ goto free;
+ }
+ po = fa_mac_get(fa, rule.eth + ETH_ALEN, !rule.direction, false);
+ if (po < 0) {
+ err = po;
+ goto put_in;
+ }
+ nh = fa_mac_get(fa, rule.eth, false, true);
+ if (nh < 0) {
+ err = nh;
+ goto put_out;
+ }
+
+ entry->cookie = cls->cookie;
+ entry->invalidate = cls->invalidate;
+ entry->tuple = rule.tuple;
+ entry->l4proto = rule.l4proto;
+ entry->pool_in = pi;
+ entry->pool_out = po;
+ entry->nh = nh;
+ entry->in = rule.in;
+ entry->out = rule.out;
+ entry->in_port = dsa_port_from_netdev(rule.in)->index;
+ entry->out_port = dsa_port_from_netdev(rule.out)->index;
+ fa_reply_tuple(&rule, &entry->reply);
+ netdev_hold(entry->in, &entry->in_tracker, GFP_KERNEL);
+ netdev_hold(entry->out, &entry->out_tracker, GFP_KERNEL);
+ spin_lock_bh(&fa->learn_lock);
+ if (learned.generation != (rule.l4proto == IPPROTO_TCP ? fa->control_generation :
+ fa->invalidate_generation) || fa->control_pending) {
+ spin_unlock_bh(&fa->learn_lock);
+ netdev_put(entry->in, &entry->in_tracker);
+ netdev_put(entry->out, &entry->out_tracker);
+ err = -EAGAIN;
+ goto put_nh;
+ }
+ list_add_rcu(&entry->list, &fa->entries);
+ spin_unlock_bh(&fa->learn_lock);
+ fa_encode(&rule, po, nh, entry->out_port, data);
+ /* Publish to NAPI before committing: a concurrent control trap must
+ * stop lookups even if it arrives while this entry is being written.
+ */
+ fa_control_update(fa, 0, FA_CTL_NO_DA_CHECK);
+ err = fa_write(fa, FA_FLOW_TABLE, slot, data);
+ if (err) {
+ fa_fail(fa);
+ list_del_rcu(&entry->list);
+ netdev_put(entry->in, &entry->in_tracker);
+ netdev_put(entry->out, &entry->out_tracker);
+ kfree_rcu(entry, rcu);
+ return err;
+ }
+ entry->installed = true;
+ if (rule.l4proto == IPPROTO_TCP) {
+ struct fa_flow *other;
+
+ list_for_each_entry(other, &fa->entries, list) {
+ if (other == entry || other->l4proto != IPPROTO_TCP || other->reverse ||
+ test_bit(FA_FLOW_CLOSING, &other->flags) ||
+ other->in_port != entry->out_port ||
+ other->out_port != entry->in_port ||
+ memcmp(&other->tuple, &entry->reply, sizeof(other->tuple)) ||
+ memcmp(&other->reply, &entry->tuple, sizeof(other->reply)))
+ continue;
+ entry->reverse = other;
+ other->reverse = entry;
+ break;
+ }
+ }
+ entry->lastused = jiffies;
+ fa->flows[slot] = entry;
+ fa->nr_flows++;
+ return 0;
+put_nh:
+ if (!fa->failed)
+ fa_mac_put(fa, nh, true);
+put_out:
+ if (!fa->failed)
+ fa_mac_put(fa, po, false);
+put_in:
+ if (!fa->failed)
+ fa_mac_put(fa, pi, false);
+free:
+ kfree(entry);
+ return err;
+}
+
+static int fa_route_supported(const struct fa_rule *rule, const struct fib_result *res)
+{
+ const struct fib_nh_common *nhc;
+
+ if (res->type != RTN_UNICAST || !res->fi)
+ return -EOPNOTSUPP;
+ /* These paths may be valid software flowtable routes. Reject hardware
+ * admission without tearing down their software owner on every retry.
+ */
+ if (res->fi->nh || res->fi->fib_nhs != 1 ||
+ (res->fi->fib_mtu && res->fi->fib_mtu < ETH_DATA_LEN))
+ return -EOPNOTSUPP;
+ nhc = FIB_RES_NHC(*res);
+ if (nhc->nhc_lwtstate || (nhc->nhc_gw_family && nhc->nhc_gw_family != AF_INET))
+ return -EOPNOTSUPP;
+ return FIB_RES_DEV(*res) == rule->out ? 0 : -EOPNOTSUPP;
+}
+
+/* This lookup is only a conservative hardware eligibility probe. The
+ * classifier lacks the owner's full mark/DSCP/oif context, so a mismatch
+ * cannot prove that a cached software path is stale. The owner validates
+ * its actual cached route separately. Caller holds RCU.
+ */
+static int fa_route_check(const struct fa_rule *rule)
+{
+ struct flowi4 fl4 = {
+ .flowi4_iif = rule->in->ifindex,
+ .flowi4_proto = rule->l4proto,
+ .saddr = rule->tuple.src,
+ .daddr = rule->tuple.dst,
+ .fl4_sport = rule->tuple.sport,
+ .fl4_dport = rule->tuple.dport,
+ };
+ struct fib_result res = {};
+ struct fib_nh_common *nhc;
+ struct neighbour *neigh;
+ u8 mac[ETH_ALEN];
+ bool valid;
+ __be32 gw;
+ int err;
+
+ if (rule->direction == FA_DIR_DNAT) {
+ if (rule->action & FA_ACT_REMAP_IP)
+ fl4.daddr = htonl(rule->remap_ip);
+ if (rule->action & FA_ACT_REMAP_PORT)
+ fl4.fl4_dport = htons(rule->remap_port);
+ }
+ if (fib_lookup(&init_net, &fl4, &res, 0))
+ return -EOPNOTSUPP;
+ err = fa_route_supported(rule, &res);
+ if (err)
+ return err;
+ nhc = FIB_RES_NHC(res);
+ gw = nhc->nhc_gw_family == AF_INET ? nhc->nhc_gw.ipv4 : fl4.daddr;
+ neigh = ipv4_neigh_lookup(rule->out, &gw);
+ if (!neigh)
+ return -EOPNOTSUPP;
+ neigh_ha_snapshot(mac, neigh, rule->out);
+ valid = (READ_ONCE(neigh->nud_state) & NUD_VALID) && ether_addr_equal(mac, rule->eth);
+ neigh_release(neigh);
+ return valid ? 0 : -EOPNOTSUPP;
+}
+
+static int fa_replace(struct bcm_ns_fa *fa, struct flow_cls_offload *cls)
+{
+ struct fa_rule rule = {};
+ netdevice_tracker in_tracker, out_tracker;
+ int err;
+
+ if (!cls->invalidate || !cls->validate || !fa->enabled ||
+ !READ_ONCE(fa->route_mtu_ok) ||
+ READ_ONCE(fa->control_pending))
+ return -EOPNOTSUPP;
+ rcu_read_lock();
+ err = fa_parse(fa, cls, &rule);
+ if (!err)
+ err = cls->validate(cls->cookie);
+ if (err == -ESTALE) {
+ cls->invalidate(cls->cookie);
+ err = -EOPNOTSUPP;
+ }
+ if (!err)
+ err = fa_route_check(&rule);
+ if (!err) {
+ netdev_hold(rule.in, &in_tracker, GFP_ATOMIC);
+ netdev_hold(rule.out, &out_tracker, GFP_ATOMIC);
+ }
+ rcu_read_unlock();
+ if (err) {
+ NL_SET_ERR_MSG_MOD(cls->common.extack,
+ "FA requires IPv4 TCP/UDP NAT between two standalone ports");
+ return err;
+ }
+ err = fa_insert(fa, cls, &rule);
+ netdev_put(rule.in, &in_tracker);
+ netdev_put(rule.out, &out_tracker);
+ return err;
+}
+
+static int fa_stats(struct bcm_ns_fa *fa, struct flow_cls_offload *cls, int slot)
+{
+ struct fa_flow *entry = fa->flows[slot];
+ int err;
+
+ if (fa->failed)
+ return -EIO;
+ if (entry->installed) {
+ err = fa_counter_read(fa, slot);
+ if (err)
+ return err;
+ }
+ /* Hardware clear-on-read gives unreliable hit counts. Read cumulative
+ * packet counters and compute wrapping deltas in fa_counter_read().
+ * Retired TCP rows retain their final delta until it is reported here.
+ */
+ flow_stats_update(&cls->stats, 0, entry->pending_hits, 0, entry->lastused,
+ FLOW_ACTION_HW_STATS_DELAYED);
+ entry->pending_hits = 0;
+ return 0;
+}
+
+static int fa_block_cb(enum tc_setup_type type, void *type_data, void *cb_priv)
+{
+ struct bcm_ns_fa *fa = cb_priv;
+ struct flow_cls_offload *cls = type_data;
+ int slot, err;
+
+ if (type != TC_SETUP_CLSFLOWER)
+ return -EOPNOTSUPP;
+ mutex_lock(&fa->lock);
+ slot = fa_find(fa, cls->cookie);
+ switch (cls->command) {
+ case FLOW_CLS_REPLACE:
+ err = fa_replace(fa, cls);
+ break;
+ case FLOW_CLS_DESTROY:
+ err = slot < 0 ? slot : fa_destroy(fa, slot);
+ break;
+ case FLOW_CLS_STATS:
+ err = slot < 0 ? slot : fa_stats(fa, cls, slot);
+ break;
+ default:
+ err = -EOPNOTSUPP;
+ }
+ mutex_unlock(&fa->lock);
+ return err;
+}
+
+int bcm_ns_fa_setup_tc(struct bcm_ns_fa *fa, enum tc_setup_type type, void *data)
+{
+ struct flow_block_offload *block = data;
+ struct flow_block_cb *cb;
+
+ if (!fa || type != TC_SETUP_FT ||
+ block->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS)
+ return -EOPNOTSUPP;
+ block->driver_block_list = &fa->block_list;
+ /* DSA forwards each user-port bind to the same conduit. Keep one
+ * callback per flowtable, with a reference for each bound user port.
+ */
+ switch (block->command) {
+ case FLOW_BLOCK_BIND:
+ cb = flow_block_cb_lookup(block->block, fa_block_cb, fa);
+ if (cb) {
+ flow_block_cb_incref(cb);
+ return 0;
+ }
+ /* Different flowtables have independent cookies and policy. The
+ * hardware cannot partition tables by ingress port or netns.
+ */
+ if (!list_empty(&fa->block_list))
+ return -EBUSY;
+ cb = flow_block_cb_alloc(fa_block_cb, fa, fa, NULL);
+ if (IS_ERR(cb))
+ return PTR_ERR(cb);
+ flow_block_cb_incref(cb);
+ flow_block_cb_add(cb, block);
+ list_add_tail(&cb->driver_list, &fa->block_list);
+ return 0;
+ case FLOW_BLOCK_UNBIND:
+ cb = flow_block_cb_lookup(block->block, fa_block_cb, fa);
+ if (!cb)
+ return -ENOENT;
+ bcm_ns_fa_flush(fa);
+ if (!flow_block_cb_decref(cb)) {
+ flow_block_cb_remove(cb, block);
+ list_del(&cb->driver_list);
+ }
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+EXPORT_SYMBOL_GPL(bcm_ns_fa_setup_tc);
+
+void bcm_ns_fa_flush(struct bcm_ns_fa *fa)
+{
+ int i;
+
+ if (!fa)
+ return;
+ mutex_lock(&fa->lock);
+ for (i = 0; i < FA_SLOTS; i++) {
+ if (fa->flows[i]) {
+ rcu_read_lock();
+ fa->flows[i]->invalidate(fa->flows[i]->cookie);
+ rcu_read_unlock();
+ fa_destroy(fa, i);
+ }
+ }
+ spin_lock_bh(&fa->learn_lock);
+ memset(fa->learned, 0, sizeof(*fa->learned) * FA_SLOTS);
+ spin_unlock_bh(&fa->learn_lock);
+ mutex_unlock(&fa->lock);
+}
+EXPORT_SYMBOL_GPL(bcm_ns_fa_flush);
+
+void bcm_ns_fa_set_offload(struct bcm_ns_fa *fa, bool enable)
+{
+ if (!fa)
+ return;
+ mutex_lock(&fa->lock);
+ fa->enabled = enable;
+ spin_lock_bh(&fa->learn_lock);
+ fa_lookup_update(fa);
+ spin_unlock_bh(&fa->learn_lock);
+ mutex_unlock(&fa->lock);
+ if (!enable)
+ bcm_ns_fa_flush(fa);
+}
+EXPORT_SYMBOL_GPL(bcm_ns_fa_set_offload);
+
+static int fa_netdev_event(struct notifier_block *nb, unsigned long event, void *ptr)
+{
+ struct bcm_ns_fa *fa = container_of(nb, struct bcm_ns_fa, netdev_nb);
+ struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+
+ /* New active ports or upper devices change the lookup domain. Flush
+ * synchronously before the new topology becomes usable.
+ */
+ if (event != NETDEV_GOING_DOWN && event != NETDEV_DOWN &&
+ event != NETDEV_PRE_CHANGEADDR &&
+ event != NETDEV_CHANGEADDR &&
+ event != NETDEV_CHANGE &&
+ event != NETDEV_CHANGEMTU &&
+ event != NETDEV_PRECHANGEUPPER && event != NETDEV_CHANGEUPPER &&
+ event != NETDEV_UP && event != NETDEV_UNREGISTER)
+ return NOTIFY_DONE;
+ if (dev == fa->conduit || dev == fa_control_conduit(fa) ||
+ (dsa_user_dev_check(dev) &&
+ dsa_port_to_conduit(dsa_port_from_netdev(dev)) == fa->conduit))
+ bcm_ns_fa_flush(fa);
+ return NOTIFY_DONE;
+}
+
+static int fa_debug_show(struct seq_file *s, void *unused)
+{
+ struct bcm_ns_fa *fa = s->private;
+ unsigned int count = 0;
+ int i;
+
+ mutex_lock(&fa->lock);
+ seq_printf(s, "failed %u\nhits %u\nmisses %u\nerrors %#x\n",
+ fa->failed, readl(fa->regs + FA_HITS),
+ readl(fa->regs + FA_MISSES), readl(fa->regs + FA_ERROR));
+ seq_printf(s, "tcp_ready %u\n", fa->tcp_ready);
+ seq_printf(s, "route_mtu_ok %u\ninvalidations %lu\n", fa->route_mtu_ok,
+ READ_ONCE(fa->invalidations));
+ seq_printf(s, "mtu_routes %u\nmtu_tracking_failed %u\n",
+ READ_ONCE(fa->nr_mtu_routes), READ_ONCE(fa->mtu_tracking_failed));
+ seq_printf(s, "fault_injections %lu\n", fa->fault_injections);
+ seq_puts(s, "slot cookie ingress egress packets\n");
+ for (i = 0; i < FA_SLOTS; i++) {
+ struct fa_flow *entry = fa->flows[i];
+ u32 data[8], hits = 0;
+
+ if (!entry || !entry->installed)
+ continue;
+ count++;
+ if (!fa->failed) {
+ if (fa_read(fa, FA_FLOW_TABLE, i, data))
+ fa_fail(fa);
+ else
+ hits = fa_hits(data);
+ }
+ seq_printf(s, "%u %#lx %s %s %u\n", i, entry->cookie,
+ entry->in->name, entry->out->name, hits);
+ }
+ seq_printf(s, "entries %u\n", count);
+ mutex_unlock(&fa->lock);
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(fa_debug);
+
+static int fa_table_show(struct seq_file *s, void *unused)
+{
+ struct bcm_ns_fa *fa = s->private;
+ unsigned int i;
+
+ mutex_lock(&fa->lock);
+ seq_printf(s, "control %#x napt %#x mem_control %#x\n",
+ readl(fa->regs + FA_CTL), readl(fa->regs + FA_NAPT),
+ readl(fa->regs + FA_MEM_CTL));
+ for (i = 0; i < FA_SLOTS; i++) {
+ struct fa_flow *entry = fa->flows[i];
+ u32 data[8] = {};
+
+ if (!entry || !entry->installed)
+ continue;
+ seq_printf(s, "%u cookie %#lx tuple %pI4:%u %pI4:%u proto %s\n", i,
+ entry->cookie, &entry->tuple.src, ntohs(entry->tuple.sport),
+ &entry->tuple.dst, ntohs(entry->tuple.dport),
+ entry->l4proto == IPPROTO_TCP ? "tcp" : "udp");
+ if (fa->failed)
+ continue;
+ if (fa_read(fa, FA_FLOW_TABLE, i, data)) {
+ fa_fail(fa);
+ continue;
+ }
+ seq_printf(s, "raw %u %08x %08x %08x %08x %08x %08x %08x %08x\n",
+ i, data[7], data[6], data[5], data[4], data[3], data[2],
+ data[1], data[0]);
+ }
+ mutex_unlock(&fa->lock);
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(fa_table);
+
+static void fa_release(void *ptr)
+{
+ struct bcm_ns_fa *fa = ptr;
+
+ spin_lock_bh(&fa->learn_lock);
+ fa->stopping = true;
+ writel(readl(fa->regs + FA_CTL) | FA_CTL_BYPASS, fa->regs + FA_CTL);
+ readl(fa->regs + FA_CTL);
+ spin_unlock_bh(&fa->learn_lock);
+ brcm_fa_control_unregister(&fa->control);
+ unregister_fib_notifier(&init_net, &fa->fib_nb);
+ fa_fib_reset(&fa->fib_nb);
+ unregister_netevent_notifier(&fa->netevent_nb);
+ cancel_work_sync(&fa->control_work);
+ debugfs_remove_recursive(fa->debugfs);
+ unregister_netdevice_notifier(&fa->netdev_nb);
+ bcm_ns_fa_flush(fa);
+}
+
+struct bcm_ns_fa *bcm_ns_fa_init(struct net_device *dev)
+{
+ struct device *parent = dev->dev.parent;
+ struct device_node *np;
+ struct resource res;
+ struct bcm_ns_fa *fa;
+ u32 val;
+ int err;
+
+ np = of_parse_phandle(parent->of_node, "brcm,flow-accelerator", 0);
+ if (!np)
+ return NULL;
+ if (!of_device_is_available(np)) {
+ of_node_put(np);
+ return NULL;
+ }
+ if (!of_device_is_compatible(np, "brcm,bcm4708-fa") &&
+ !of_device_is_compatible(np, "brcm,bcm53011-fa") &&
+ !of_device_is_compatible(np, "brcm,bcm53012-fa")) {
+ of_node_put(np);
+ return ERR_PTR(-ENODEV);
+ }
+ err = of_address_to_resource(np, 0, &res);
+ of_node_put(np);
+ if (err)
+ return ERR_PTR(err);
+ if (resource_size(&res) < FA_DATA(7) + sizeof(u32))
+ return ERR_PTR(-EINVAL);
+ fa = devm_kzalloc(parent, sizeof(*fa), GFP_KERNEL);
+ if (!fa)
+ return ERR_PTR(-ENOMEM);
+ fa->learned = devm_kcalloc(parent, FA_SLOTS, sizeof(*fa->learned), GFP_KERNEL);
+ if (!fa->learned)
+ return ERR_PTR(-ENOMEM);
+ fa->regs = devm_ioremap_resource(parent, &res);
+ if (IS_ERR(fa->regs))
+ return ERR_CAST(fa->regs);
+ fa->conduit = dev;
+ fa->fail_table = fa_fail_table_default;
+#ifdef CONFIG_FAULT_INJECTION
+ fa->fail_table.verbose = 0;
+#endif
+ fa->enabled = true;
+ fa->neighbour_check = jiffies - HZ;
+ mutex_init(&fa->lock);
+ spin_lock_init(&fa->learn_lock);
+ INIT_LIST_HEAD(&fa->block_list);
+ INIT_LIST_HEAD(&fa->entries);
+ INIT_WORK(&fa->control_work, fa_control_work);
+ hash_init(fa->mtu_routes);
+ fa->route_mtu_ok = true;
+
+ /* DA validation is enabled before publishing the first flow.
+ * Mask interrupts: initialization and indirect accesses use bounded polls.
+ */
+ writel(0, fa->regs + FA_STATUS_MASK);
+ writel(0, fa->regs + FA_ERROR_MASK);
+ writel(FA_INIT_DONE, fa->regs + FA_STATUS);
+ val = FA_CTL_BYPASS | FA_CTL_CRC_OVERWRITE |
+ FIELD_PREP(FA_CTL_THRESHOLD, 0x140);
+ writel(val | FA_CTL_INIT, fa->regs + FA_CTL);
+ err = readl_poll_timeout(fa->regs + FA_STATUS, val,
+ (val & FA_INIT_DONE) == FA_INIT_DONE, 100, 50000);
+ if (err)
+ return ERR_PTR(err);
+ /* The pool has no init command of its own. Clear stale firmware state. */
+ for (val = 0; val < FA_POOLS; val++) {
+ u32 zero[8] = {};
+
+ err = fa_write(fa, FA_POOL_TABLE, val, zero);
+ if (err)
+ return ERR_PTR(err);
+ }
+ /* Standalone traffic carries flooding reasons. TCP control packets
+ * use the auxiliary CPU port and physically bypass this inline path.
+ */
+ writel(FA_BRCM_HW | FA_BRCM_RX | FA_BRCM_TX | FA_BRCM_PARSE_EXCEPTIONS,
+ fa->regs + FA_BRCM_HDR);
+ writel(0, fa->regs + FA_L2_SKIP);
+ /* Do not skip VLAN tags: tagged traffic must miss even if its inner
+ * tuple equals an installed untagged flow.
+ */
+ writel(0, fa->regs + FA_L2_TAG);
+ writel(ETH_P_IP | ETH_P_IPV6 << 16, fa->regs + FA_L2_ETHERTYPE);
+ writel(IPPROTO_TCP | IPPROTO_UDP << 8 | FA_IPV4_CSUM, fa->regs + FA_IPV4);
+ /* Zero seed, CCITT CRC; learned hashes are used for table placement. */
+ writel(FA_NAPT_DEC_TTL, fa->regs + FA_NAPT);
+ writel(0xff, fa->regs + FA_RX_STATUS);
+ fa->netdev_nb.notifier_call = fa_netdev_event;
+ err = register_netdevice_notifier(&fa->netdev_nb);
+ if (err)
+ return ERR_PTR(err);
+ fa->netevent_nb.notifier_call = fa_netevent;
+ err = register_netevent_notifier(&fa->netevent_nb);
+ if (err)
+ goto unregister_netdev;
+ fa->fib_nb.notifier_call = fa_fib_event;
+ err = register_fib_notifier(&init_net, &fa->fib_nb, fa_fib_reset, NULL);
+ if (err)
+ goto unregister_netevent;
+ /* Finish initial route replay before allowing hardware admission. */
+ fa_invalidate(fa);
+ fa->debugfs = debugfs_create_dir(dev_name(parent), NULL);
+ if (IS_ENABLED(CONFIG_BCM_NS_FA_FAULT_INJECTION)) {
+ fault_create_debugfs_attr("fail_table", fa->debugfs, &fa->fail_table);
+ debugfs_create_bool("fail_table_write_only", 0600, fa->debugfs,
+ &fa->fail_table_write_only);
+ }
+ debugfs_create_file("fa", 0400, fa->debugfs, fa, &fa_debug_fops);
+ debugfs_create_file("fa_table", 0400, fa->debugfs, fa, &fa_table_fops);
+ fa->control.conduit = dev;
+ fa->control.priv = fa;
+ fa->control.receive = fa_control_receive;
+ fa->control.state_change = fa_control_state_change;
+ brcm_fa_control_register(&fa->control);
+ err = devm_add_action_or_reset(parent, fa_release, fa);
+ if (err)
+ return ERR_PTR(err);
+ writel(FA_CTL_NO_DA_CHECK | FA_CTL_CRC_OVERWRITE |
+ FIELD_PREP(FA_CTL_THRESHOLD, 0x140),
+ fa->regs + FA_CTL);
+ return fa;
+unregister_netevent:
+ fa_fib_reset(&fa->fib_nb);
+ WRITE_ONCE(fa->stopping, true);
+ unregister_netevent_notifier(&fa->netevent_nb);
+ cancel_work_sync(&fa->control_work);
+unregister_netdev:
+ unregister_netdevice_notifier(&fa->netdev_nb);
+ return ERR_PTR(err);
+}
+EXPORT_SYMBOL_GPL(bcm_ns_fa_init);
+
+MODULE_DESCRIPTION("Broadcom Northstar IPv4 TCP/UDP NAT flow accelerator");
+MODULE_LICENSE("GPL");
diff --git a/drivers/net/ethernet/broadcom/bcm-ns-fa.h b/drivers/net/ethernet/broadcom/bcm-ns-fa.h
new file mode 100644
index 000000000000..825b019fb89f
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bcm-ns-fa.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _BCM_NS_FA_H
+#define _BCM_NS_FA_H
+
+#include <linux/netdevice.h>
+
+struct bcm_ns_fa;
+
+#if IS_REACHABLE(CONFIG_BCM_NS_FA)
+struct bcm_ns_fa *bcm_ns_fa_init(struct net_device *dev);
+void bcm_ns_fa_rx(struct bcm_ns_fa *fa, struct sk_buff *skb);
+int bcm_ns_fa_setup_tc(struct bcm_ns_fa *fa, enum tc_setup_type type, void *data);
+void bcm_ns_fa_flush(struct bcm_ns_fa *fa);
+void bcm_ns_fa_set_offload(struct bcm_ns_fa *fa, bool enable);
+#else
+static inline struct bcm_ns_fa *bcm_ns_fa_init(struct net_device *dev)
+{
+ return NULL;
+}
+
+static inline void bcm_ns_fa_rx(struct bcm_ns_fa *fa, struct sk_buff *skb) {}
+static inline void bcm_ns_fa_flush(struct bcm_ns_fa *fa) {}
+static inline void bcm_ns_fa_set_offload(struct bcm_ns_fa *fa, bool enable) {}
+static inline int bcm_ns_fa_setup_tc(struct bcm_ns_fa *fa,
+ enum tc_setup_type type, void *data)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+#endif
diff --git a/drivers/net/ethernet/broadcom/bgmac-bcma.c b/drivers/net/ethernet/broadcom/bgmac-bcma.c
index 36f9bad28e6a..87338e01a825 100644
--- a/drivers/net/ethernet/broadcom/bgmac-bcma.c
+++ b/drivers/net/ethernet/broadcom/bgmac-bcma.c
@@ -11,6 +11,7 @@
#include <linux/bcma/bcma.h>
#include <linux/brcmphy.h>
#include <linux/etherdevice.h>
+#include <linux/of.h>
#include <linux/of_mdio.h>
#include <linux/of_net.h>
#include "bgmac.h"
@@ -130,6 +131,10 @@ static int bgmac_probe(struct bcma_device *core)
const u8 *mac;
int err;
+ if (of_property_present(core->dev.of_node, "brcm,flow-accelerator") &&
+ core->core_unit != 2)
+ return -EINVAL;
+
bgmac = bgmac_alloc(&core->dev);
if (!bgmac)
return -ENOMEM;
diff --git a/drivers/net/ethernet/broadcom/bgmac.c b/drivers/net/ethernet/broadcom/bgmac.c
index 1c502bd64d7d..527f9842eb83 100644
--- a/drivers/net/ethernet/broadcom/bgmac.c
+++ b/drivers/net/ethernet/broadcom/bgmac.c
@@ -486,6 +486,7 @@ static int bgmac_dma_rx_read(struct bgmac *bgmac, struct bgmac_dma_ring *ring,
skb_pull(skb, BGMAC_RX_FRAME_OFFSET +
BGMAC_RX_BUF_OFFSET);
+ bcm_ns_fa_rx(bgmac->fa, skb);
skb_checksum_none_assert(skb);
skb->protocol = eth_type_trans(skb, bgmac->net_dev);
bgmac->net_dev->stats.rx_bytes += len;
@@ -1272,7 +1273,25 @@ static int bgmac_change_mtu(struct net_device *net_dev, int mtu)
return 0;
}
+static int bgmac_set_features(struct net_device *dev, netdev_features_t features)
+{
+ struct bgmac *bgmac = netdev_priv(dev);
+
+ bcm_ns_fa_set_offload(bgmac->fa, features & NETIF_F_HW_TC);
+ return 0;
+}
+
+static int bgmac_setup_tc(struct net_device *dev, enum tc_setup_type type,
+ void *data)
+{
+ struct bgmac *bgmac = netdev_priv(dev);
+
+ return bcm_ns_fa_setup_tc(bgmac->fa, type, data);
+}
+
static const struct net_device_ops bgmac_netdev_ops = {
+ .ndo_set_features = bgmac_set_features,
+ .ndo_setup_tc = bgmac_setup_tc,
.ndo_open = bgmac_open,
.ndo_stop = bgmac_stop,
.ndo_start_xmit = bgmac_start_xmit,
@@ -1560,6 +1579,17 @@ int bgmac_enet_probe(struct bgmac *bgmac)
net_dev->hw_features = net_dev->features;
net_dev->vlan_features = net_dev->features;
+ bgmac->fa = bcm_ns_fa_init(net_dev);
+ if (IS_ERR(bgmac->fa)) {
+ err = PTR_ERR(bgmac->fa);
+ bgmac->fa = NULL;
+ goto err_phy_disconnect;
+ }
+ if (bgmac->fa) {
+ net_dev->features |= NETIF_F_HW_TC;
+ net_dev->hw_features |= NETIF_F_HW_TC;
+ }
+
/* Omit FCS from max MTU size */
net_dev->max_mtu = BGMAC_RX_MAX_FRAME_SIZE - ETH_FCS_LEN;
@@ -1610,6 +1640,7 @@ int bgmac_enet_suspend(struct bgmac *bgmac)
netif_tx_unlock(bgmac->net_dev);
bgmac_chip_intrs_off(bgmac);
+ bcm_ns_fa_flush(bgmac->fa);
bgmac_chip_reset(bgmac);
bgmac_dma_cleanup(bgmac);
diff --git a/drivers/net/ethernet/broadcom/bgmac.h b/drivers/net/ethernet/broadcom/bgmac.h
index 6fee9a41839c..788d5ba9c9b5 100644
--- a/drivers/net/ethernet/broadcom/bgmac.h
+++ b/drivers/net/ethernet/broadcom/bgmac.h
@@ -5,6 +5,7 @@
#include <linux/netdevice.h>
#include "unimac.h"
+#include "bcm-ns-fa.h"
#define BGMAC_DEV_CTL 0x000
#define BGMAC_DC_TSM 0x00000002
@@ -437,6 +438,7 @@ struct bgmac_rx_header {
};
struct bgmac {
+ struct bcm_ns_fa *fa;
union {
struct {
void __iomem *base;
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 8/9] net: add KUnit coverage for FA and flowtable path validation
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (6 preceding siblings ...)
2026-11-16 22:25 ` [PATCH net-next 7/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:25 ` [PATCH net-next 9/9] docs: net: document Northstar FA setup and NAT qualification Florian Fainelli
2026-11-16 22:38 ` [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload netdev-bot+sinfo
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
Exercise Northstar NAT fields, byte order, protocol-aware hash learning,
TCP control identity, lookup gating, MTU tracking and owner
invalidation. Check B53 UDF layout and every IPv4 packet length, failed-
guard readiness, and FA/ordinary switch tags.
Test cached-route owner validation with route-cookie expiry, changed or
invalid neighbours, absent route context and unsupported stacked paths.
Require explicit owner validation to detect an expired validation-only
dst even though the generic packet/GC helper ignores that reference.
Add seven scenarios using real route initialization and the private GC
step: ordinary direct software forwarding, validation-only references,
direct tunnels with and without hardware requested, neighbour routes
with and without hardware requested, and XFRM. Check legacy validation
policy, GC teardown and balanced dst references. The validation-only
scenario fails both packet and GC expectations before the policy fix.
The independent netfilter prerequisite already carries its two lifetime
tests and standalone configuration. Extend that test configuration's
help for these path cases, embed private tests in their parent units,
and keep the combined FA configuration beside the backend.
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
MAINTAINERS | 3 +
drivers/net/dsa/b53/Kconfig | 11 +
drivers/net/dsa/b53/b53_common.c | 4 +
drivers/net/dsa/b53/b53_fa_test.c | 102 +++
drivers/net/ethernet/broadcom/.kunitconfig | 21 +
drivers/net/ethernet/broadcom/Kconfig | 11 +
.../net/ethernet/broadcom/bcm-ns-fa-test.c | 661 ++++++++++++++++++
drivers/net/ethernet/broadcom/bcm-ns-fa.c | 4 +
net/dsa/Kconfig | 11 +
net/dsa/tag_brcm.c | 4 +
net/dsa/tag_brcm_test.c | 199 ++++++
net/netfilter/Kconfig | 10 +-
net/netfilter/nf_flow_table_core_test.c | 85 +++
net/netfilter/nf_flow_table_offload.c | 4 +
net/netfilter/nf_flow_table_offload_test.c | 129 ++++
15 files changed, 1254 insertions(+), 5 deletions(-)
create mode 100644 drivers/net/dsa/b53/b53_fa_test.c
create mode 100644 drivers/net/ethernet/broadcom/.kunitconfig
create mode 100644 drivers/net/ethernet/broadcom/bcm-ns-fa-test.c
create mode 100644 net/dsa/tag_brcm_test.c
create mode 100644 net/netfilter/nf_flow_table_offload_test.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 80969b91af40..c05ac2258889 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5305,6 +5305,7 @@ F: drivers/net/dsa/bcm_sf2*
F: include/linux/dsa/brcm.h
F: include/linux/platform_data/b53.h
F: net/dsa/tag_brcm.c
+F: net/dsa/tag_brcm_test.c
BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE
M: Florian Fainelli <florian.fainelli@broadcom.com>
@@ -5671,6 +5672,8 @@ S: Maintained
F: Documentation/devicetree/bindings/net/brcm,amac.yaml
F: Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml
F: Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml
+F: drivers/net/dsa/b53/b53_fa_test.c
+F: drivers/net/ethernet/broadcom/.kunitconfig
F: drivers/net/ethernet/broadcom/bcm-ns-fa*
F: drivers/net/ethernet/broadcom/bgmac*
F: drivers/net/ethernet/broadcom/unimac.h
diff --git a/drivers/net/dsa/b53/Kconfig b/drivers/net/dsa/b53/Kconfig
index 8e2a8c20bc4a..5a03bacef2e5 100644
--- a/drivers/net/dsa/b53/Kconfig
+++ b/drivers/net/dsa/b53/Kconfig
@@ -13,6 +13,17 @@ menuconfig B53
BCM5325E, BCM5365, BCM539x, BCM53115 and BCM53125 as well as BCM63XX
integrated switches.
+config B53_FA_KUNIT_TEST
+ bool "KUnit tests for B53 FA exception guards" if !KUNIT_ALL_TESTS
+ depends on KUNIT=y && B53
+ default KUNIT_ALL_TESTS
+ help
+ Tests are compiled into the parent driver and require built-in KUnit.
+ Verify the original FA classifier field layout and complete packet
+ length range used to divert unsafe packets around the accelerator.
+ Check cross-word destination MAC encoding and every 16-bit IPv4
+ packet length against the standard MTU without accessing hardware.
+
config B53_SPI_DRIVER
tristate "B53 SPI connected switch driver"
depends on B53 && SPI
diff --git a/drivers/net/dsa/b53/b53_common.c b/drivers/net/dsa/b53/b53_common.c
index 3e8c9679836d..25bb23a51952 100644
--- a/drivers/net/dsa/b53/b53_common.c
+++ b/drivers/net/dsa/b53/b53_common.c
@@ -3802,6 +3802,10 @@ int b53_switch_register(struct b53_device *dev)
}
EXPORT_SYMBOL(b53_switch_register);
+#if IS_ENABLED(CONFIG_B53_FA_KUNIT_TEST)
+#include "b53_fa_test.c"
+#endif
+
MODULE_AUTHOR("Jonas Gorski <jogo@openwrt.org>");
MODULE_DESCRIPTION("B53 switch library");
MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/dsa/b53/b53_fa_test.c b/drivers/net/dsa/b53/b53_fa_test.c
new file mode 100644
index 000000000000..1806c87a54fb
--- /dev/null
+++ b/drivers/net/dsa/b53/b53_fa_test.c
@@ -0,0 +1,102 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <kunit/test.h>
+
+static int b53_fa_test_write16(struct b53_device *dev, u8 page, u8 reg, u16 value)
+{
+ u16 *control = dev->priv;
+
+ if (page != B53_CFP_PAGE || reg != B53_CFP_CTRL)
+ return -EINVAL;
+ *control = value;
+ return 0;
+}
+
+static void b53_fa_test_port_guards(struct kunit *test)
+{
+ static const struct b53_io_ops ops = { .write16 = b53_fa_test_write16 };
+ struct b53_device *dev = kunit_kzalloc(test, sizeof(*dev), GFP_KERNEL);
+ struct dsa_switch_tree tree = {};
+ struct dsa_switch ds = { .dst = &tree };
+ struct dsa_port control = {
+ .ds = &ds, .index = 5, .type = DSA_PORT_TYPE_CPU,
+ .conduit_admin_up = true, .conduit_oper_up = true,
+ };
+ u16 map = 0;
+
+ KUNIT_ASSERT_NOT_NULL(test, dev);
+ INIT_LIST_HEAD(&tree.ports);
+ list_add(&control.list, &tree.ports);
+ dev->ds = &ds;
+ dev->ops = &ops;
+ dev->priv = ↦
+ dev->fa_cfp_enabled = true;
+ dev->fa_cfp_ports = BIT(0) | BIT(1);
+ mutex_init(&dev->reg_mutex);
+
+ /* Bridge changes must not resurrect incomplete or failed guards. */
+ KUNIT_ASSERT_EQ(test, b53_fa_port(dev, 0, false), 0);
+ KUNIT_EXPECT_EQ(test, map, (u16)0);
+ dev->fa_guards_ready = true;
+ KUNIT_ASSERT_EQ(test, b53_fa_port(dev, 0, true), 0);
+ KUNIT_EXPECT_EQ(test, map, (u16)(BIT(0) | BIT(1)));
+ dev->fa_guards_failed = true;
+ KUNIT_ASSERT_EQ(test, b53_fa_port(dev, 0, false), 0);
+ KUNIT_EXPECT_EQ(test, map, (u16)0);
+ KUNIT_ASSERT_EQ(test, b53_fa_port(dev, 0, true), 0);
+ KUNIT_EXPECT_EQ(test, map, (u16)0);
+ KUNIT_EXPECT_EQ(test, dev->fa_cfp_ports, (u16)(BIT(0) | BIT(1)));
+ dev->fa_guards_failed = false;
+ control.conduit_oper_up = false;
+ KUNIT_ASSERT_EQ(test, b53_fa_port(dev, 0, false), 0);
+ KUNIT_EXPECT_EQ(test, map, (u16)0);
+}
+
+static void b53_fa_test_length_range(struct kunit *test)
+{
+ unsigned int length, i;
+
+ for (length = 0; length <= U16_MAX; length++) {
+ bool trap = false;
+
+ for (i = 0; i < ARRAY_SIZE(b53_fa_lengths); i++)
+ if ((length & b53_fa_lengths[i][1]) == b53_fa_lengths[i][0])
+ trap = true;
+ KUNIT_ASSERT_EQ_MSG(test, trap, length > ETH_DATA_LEN, "length %u", length);
+ }
+}
+
+static void b53_fa_test_udf_layout(struct kunit *test)
+{
+ u32 data[8] = {}, mask[8] = {};
+
+ b53_fa_udf(data, mask, 2, 1501, 0xffff);
+ KUNIT_EXPECT_EQ(test, data[1], 0x0005dd00U);
+ KUNIT_EXPECT_EQ(test, mask[1], 0x00ffff00U);
+ KUNIT_EXPECT_EQ(test, mask[5], (u32)BIT(26));
+ memset(data, 0, sizeof(data));
+ memset(mask, 0, sizeof(mask));
+ b53_fa_udf(data, mask, 3, 0x0200, 0xffff);
+ b53_fa_udf(data, mask, 4, 0x0000, 0xffff);
+ b53_fa_udf(data, mask, 5, 0x0101, 0xffff);
+ KUNIT_EXPECT_EQ(test, data[1], 0U);
+ KUNIT_EXPECT_EQ(test, data[2], 0x01000002U);
+ KUNIT_EXPECT_EQ(test, data[3], 1U);
+ KUNIT_EXPECT_EQ(test, mask[1], 0xff000000U);
+ KUNIT_EXPECT_EQ(test, mask[2], 0xffffffffU);
+ KUNIT_EXPECT_EQ(test, mask[3], 0x000000ffU);
+ KUNIT_EXPECT_EQ(test, mask[5], (u32)(BIT(27) | BIT(28) | BIT(29)));
+}
+
+static struct kunit_case b53_fa_cases[] = {
+ KUNIT_CASE(b53_fa_test_port_guards),
+ KUNIT_CASE(b53_fa_test_length_range),
+ KUNIT_CASE(b53_fa_test_udf_layout),
+ {}
+};
+
+static struct kunit_suite b53_fa_suite = {
+ .name = "b53-fa-guards",
+ .test_cases = b53_fa_cases,
+};
+
+kunit_test_suite(b53_fa_suite);
diff --git a/drivers/net/ethernet/broadcom/.kunitconfig b/drivers/net/ethernet/broadcom/.kunitconfig
new file mode 100644
index 000000000000..9772432ffc32
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/.kunitconfig
@@ -0,0 +1,21 @@
+CONFIG_KUNIT=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_NET_DEV_REFCNT_TRACKER=y
+CONFIG_COMPILE_TEST=y
+CONFIG_NET=y
+CONFIG_INET=y
+CONFIG_NETDEVICES=y
+CONFIG_NET_VENDOR_BROADCOM=y
+CONFIG_NETFILTER=y
+CONFIG_NF_CONNTRACK=y
+CONFIG_NF_TABLES=y
+CONFIG_NF_FLOW_TABLE=y
+CONFIG_NF_FLOW_TABLE_KUNIT_TEST=y
+CONFIG_NET_DSA=y
+CONFIG_OF=y
+CONFIG_BCM_NS_FA=y
+CONFIG_BCM_NS_FA_KUNIT_TEST=y
+CONFIG_NET_DSA_TAG_BRCM_FA=y
+CONFIG_NET_DSA_TAG_BRCM_FA_KUNIT_TEST=y
+CONFIG_B53=y
+CONFIG_B53_FA_KUNIT_TEST=y
diff --git a/drivers/net/ethernet/broadcom/Kconfig b/drivers/net/ethernet/broadcom/Kconfig
index 865b346a7249..f6cbe63b576b 100644
--- a/drivers/net/ethernet/broadcom/Kconfig
+++ b/drivers/net/ethernet/broadcom/Kconfig
@@ -190,6 +190,17 @@ config BCM_NS_FA_FAULT_INJECTION
reprobe and leaves forwarding to software. Intended for driver testing.
Injection is disabled by default and can be armed through debugfs.
+config BCM_NS_FA_KUNIT_TEST
+ bool "KUnit tests for the Northstar flow accelerator" if !KUNIT_ALL_TESTS
+ depends on KUNIT=y && BCM_NS_FA
+ default KUNIT_ALL_TESTS
+ help
+ Test NAT action validation, table field encoding, and RX hash learning.
+ Exercise upper-device inspection under RCU with real netdevices.
+ Tests are compiled into the parent driver and require built-in KUnit.
+ These tests run without an accelerator and do not simulate table I/O.
+ Use the hardware validation plan to check actual forwarding.
+
config BGMAC_BCMA
tristate "Broadcom iProc GBit BCMA support"
depends on BCM_NS_FA || !BCM_NS_FA
diff --git a/drivers/net/ethernet/broadcom/bcm-ns-fa-test.c b/drivers/net/ethernet/broadcom/bcm-ns-fa-test.c
new file mode 100644
index 000000000000..59406f84809e
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bcm-ns-fa-test.c
@@ -0,0 +1,661 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Included in bcm-ns-fa.c to exercise private table formats and validation. */
+#include <kunit/test.h>
+#include <net/lwtunnel.h>
+#include <net/nexthop.h>
+#include <linux/rtnetlink.h>
+
+static const struct net_device_ops fa_test_netdev_ops;
+
+static void fa_test_free_netdev(void *data)
+{
+ struct net_device *dev = data;
+
+ if (dev->reg_state == NETREG_REGISTERED)
+ unregister_netdev(dev);
+ free_netdev(dev);
+}
+
+static void fa_test_upper_device_rcu(struct kunit *test)
+{
+ struct net_device *lower, *upper;
+ bool has_upper;
+ int err;
+
+ lower = alloc_etherdev(0);
+ KUNIT_ASSERT_NOT_NULL(test, lower);
+ err = kunit_add_action_or_reset(test, fa_test_free_netdev, lower);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ upper = alloc_etherdev(0);
+ KUNIT_ASSERT_NOT_NULL(test, upper);
+ err = kunit_add_action_or_reset(test, fa_test_free_netdev, upper);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ lower->netdev_ops = &fa_test_netdev_ops;
+ upper->netdev_ops = &fa_test_netdev_ops;
+ eth_hw_addr_random(lower);
+ eth_hw_addr_random(upper);
+ KUNIT_ASSERT_EQ(test, register_netdev(lower), 0);
+ KUNIT_ASSERT_EQ(test, register_netdev(upper), 0);
+
+ /* Rule workers inspect topology under RCU without taking RTNL. */
+ rcu_read_lock();
+ has_upper = fa_has_upper_dev_rcu(lower);
+ rcu_read_unlock();
+ KUNIT_EXPECT_FALSE(test, has_upper);
+
+ rtnl_lock();
+ err = netdev_upper_dev_link(lower, upper, NULL);
+ rtnl_unlock();
+ KUNIT_ASSERT_EQ(test, err, 0);
+ rcu_read_lock();
+ has_upper = fa_has_upper_dev_rcu(lower);
+ rcu_read_unlock();
+ KUNIT_EXPECT_TRUE(test, has_upper);
+
+ rtnl_lock();
+ netdev_upper_dev_unlink(lower, upper);
+ rtnl_unlock();
+ rcu_read_lock();
+ has_upper = fa_has_upper_dev_rcu(lower);
+ rcu_read_unlock();
+ KUNIT_EXPECT_FALSE(test, has_upper);
+}
+
+static u64 fa_test_bits(const u32 *data, unsigned int low, unsigned int width)
+{
+ u64 val = 0;
+ unsigned int i;
+
+ for (i = 0; i < width; i++)
+ if (data[(low + i) / 32] & BIT((low + i) % 32))
+ val |= BIT_ULL(i);
+ return val;
+}
+
+static void fa_test_encode(struct kunit *test)
+{
+ struct fa_rule rule = {
+ .tuple = { cpu_to_be32(0xc0000202), cpu_to_be32(0xc6336402),
+ cpu_to_be16(0x1234), cpu_to_be16(0xabcd) },
+ .remap_ip = 0xcb007109,
+ .remap_port = 0x5678,
+ .action = FA_ACT_REMAP_IP | FA_ACT_REMAP_PORT,
+ .direction = FA_DIR_DNAT,
+ };
+ u32 data[8] = {};
+
+ fa_encode(&rule, 3, 127, 4, data);
+ /* Compare against the bit ranges in the hardware table definition,
+ * including fields that straddle words and host/network byte order.
+ */
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 0, 35), 0ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 35, 2), 3ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 37, 1), 0ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 38, 2), 3ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 40, 7), 127ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 47, 32), 0xcb007109ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 79, 16), 0x5678ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 95, 16), 0xabcdULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 111, 16), 0x1234ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 127, 1), 0ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 128, 32), 0xc6336402ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 160, 32), 0xc0000202ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 192, 1), 1ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 193, 10), 16ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 209, 3), 1ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 212, 1), 1ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 255, 1), 1ULL);
+}
+
+static void fa_test_mac_encode(struct kunit *test)
+{
+ const u8 addr[ETH_ALEN] = { 0xfe, 0xdc, 0xba, 0x98, 0x76, 0x54 };
+ u32 data[8] = {};
+
+ fa_mac_encode(addr, true, false, data);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 0, 1), 1ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 1, 48), 0xfedcba987654ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 49, 15), 0ULL);
+ memset(data, 0, sizeof(data));
+ fa_mac_encode(addr, false, false, data);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 0, 1), 0ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 1, 48), 0xfedcba987654ULL);
+ memset(data, 0, sizeof(data));
+ fa_mac_encode(addr, false, true, data);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 0, 1), 0ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 1, 2), 2ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 3, 16), 0ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 19, 48), 0xfedcba987654ULL);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 67, 29), 0ULL);
+ /* Hit-count bits must exclude the adjacent timestamp and action bits. */
+ data[0] = U32_MAX;
+ data[1] = U32_MAX;
+ KUNIT_EXPECT_EQ(test, fa_hits(data), U32_MAX);
+ data[0] = 7;
+ data[1] = ~7U;
+ KUNIT_EXPECT_EQ(test, fa_hits(data), 0U);
+}
+
+static void fa_test_nat_actions(struct kunit *test)
+{
+ struct fa_rule rule = { .direction = FA_DIR_UNSET, .l4proto = IPPROTO_UDP };
+ struct flow_action_entry act = {
+ .mangle = {
+ .htype = FLOW_ACT_MANGLE_HDR_TYPE_IP4,
+ .offset = offsetof(struct iphdr, saddr),
+ .val = (__force u32)cpu_to_be32(0xc0000201),
+ },
+ };
+
+ KUNIT_ASSERT_EQ(test, fa_mangle(&rule, &act), 0);
+ KUNIT_EXPECT_EQ(test, rule.remap_ip, 0xc0000201U);
+ KUNIT_EXPECT_EQ(test, rule.direction, 0);
+ act.mangle.htype = FLOW_ACT_MANGLE_HDR_TYPE_UDP;
+ act.mangle.offset = 0;
+ act.mangle.mask = (__force u32)cpu_to_be32(0x0000ffff);
+ act.mangle.val = (__force u32)cpu_to_be32(0xabcd0000);
+ KUNIT_ASSERT_EQ(test, fa_mangle(&rule, &act), 0);
+ KUNIT_EXPECT_EQ(test, rule.remap_port, (u16)0xabcd);
+ KUNIT_EXPECT_EQ(test, rule.action, (u8)3);
+ /* Opposite-side rewrites cannot share a hardware entry. */
+ act.mangle.mask = (__force u32)cpu_to_be32(0xffff0000);
+ KUNIT_EXPECT_EQ(test, fa_mangle(&rule, &act), -EOPNOTSUPP);
+ /* Partial IPv4 masks and arbitrary offsets must not be accepted. */
+ act.mangle.htype = FLOW_ACT_MANGLE_HDR_TYPE_IP4;
+ act.mangle.offset = offsetof(struct iphdr, daddr);
+ act.mangle.mask = 1;
+ KUNIT_EXPECT_EQ(test, fa_mangle(&rule, &act), -EOPNOTSUPP);
+ act.mangle.mask = 0;
+ act.mangle.offset = offsetof(struct iphdr, ttl);
+ KUNIT_EXPECT_EQ(test, fa_mangle(&rule, &act), -EOPNOTSUPP);
+}
+
+static void fa_test_ethernet_actions(struct kunit *test)
+{
+ const u8 expected[12] = { 0x02, 1, 2, 3, 4, 5, 0x02, 6, 7, 8, 9, 10 };
+ struct fa_rule rule = {};
+ struct flow_action_entry act = {
+ .mangle.htype = FLOW_ACT_MANGLE_HDR_TYPE_ETH,
+ };
+ int i;
+
+ for (i = 0; i < sizeof(expected); i += 4) {
+ act.mangle.offset = i;
+ memcpy(&act.mangle.val, expected + i, 4);
+ KUNIT_ASSERT_EQ(test, fa_mangle(&rule, &act), 0);
+ }
+ KUNIT_EXPECT_MEMEQ(test, rule.eth, expected, sizeof(expected));
+ KUNIT_EXPECT_PTR_EQ(test, memchr_inv(rule.eth_mask, 0xff, 12), NULL);
+ act.mangle.offset = 9;
+ KUNIT_EXPECT_EQ(test, fa_mangle(&rule, &act), -EOPNOTSUPP);
+}
+
+static void fa_test_rx_learning(struct kunit *test)
+{
+ struct bcm_ns_fa *fa = kunit_kzalloc(test, sizeof(*fa), GFP_KERNEL);
+ struct sk_buff *skb;
+ struct ethhdr *eth;
+ struct iphdr *ip;
+ struct udphdr *udp;
+ u8 *tag;
+
+ KUNIT_ASSERT_NOT_NULL(test, fa);
+ fa->learned = kunit_kcalloc(test, FA_SLOTS, sizeof(*fa->learned), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, fa->learned);
+ spin_lock_init(&fa->learn_lock);
+ INIT_LIST_HEAD(&fa->entries);
+ skb = alloc_skb(64, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ tag = skb_put_zero(skb, 4 + ETH_HLEN + sizeof(*ip) + sizeof(*udp));
+ tag[0] = FA_MISS_OPCODE << 5;
+ tag[1] = 0xfe;
+ tag[3] = 3;
+ eth = (void *)(tag + 4);
+ eth->h_proto = htons(ETH_P_IP);
+ eth->h_dest[0] = 2;
+ ip = (void *)(eth + 1);
+ ip->version = 4;
+ ip->ihl = 5;
+ ip->protocol = IPPROTO_UDP;
+ ip->tot_len = htons(sizeof(*ip) + sizeof(*udp));
+ ip->saddr = htonl(0xc0000201);
+ ip->daddr = htonl(0xc6336402);
+ udp = (void *)(ip + 1);
+ udp->source = htons(1234);
+ udp->dest = htons(5678);
+ ip->check = ip_fast_csum(ip, ip->ihl);
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ KUNIT_EXPECT_TRUE(test, fa->learned[0].valid);
+ KUNIT_EXPECT_EQ(test, fa->learned[0].hash, (u8)0xfe);
+ KUNIT_EXPECT_EQ(test, fa->learned[0].port, (u8)3);
+ KUNIT_EXPECT_EQ(test, fa->learned[0].tuple.src, htonl(0xc0000201));
+ KUNIT_EXPECT_EQ(test, fa->learned[0].tuple.dport, htons(5678));
+ /* Preserve metadata for the brcm-fa DSA tagger. */
+ KUNIT_EXPECT_EQ(test, tag[0], (u8)(FA_MISS_OPCODE << 5));
+ ip->frag_off = htons(IP_MF);
+ ip->check = 0;
+ ip->check = ip_fast_csum(ip, ip->ihl);
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ ip->frag_off = 0;
+ ip->check = 0;
+ ip->check = ip_fast_csum(ip, ip->ihl);
+ tag[2] = 1;
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ tag[2] = 0;
+ ip->protocol = IPPROTO_TCP;
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ ip->protocol = IPPROTO_UDP;
+ eth->h_proto = htons(ETH_P_8021Q);
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ skb_trim(skb, 3);
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ kfree_skb(skb);
+}
+
+static void fa_test_tcp_encode_and_mangle(struct kunit *test)
+{
+ struct fa_rule rule = { .l4proto = IPPROTO_TCP, .direction = FA_DIR_UNSET };
+ struct flow_action_entry act = {
+ .mangle = {
+ .htype = FLOW_ACT_MANGLE_HDR_TYPE_TCP,
+ .mask = (__force u32)cpu_to_be32(0xffff0000),
+ .val = (__force u32)cpu_to_be32(0x00001234),
+ },
+ };
+ u32 data[8] = {};
+
+ KUNIT_ASSERT_EQ(test, fa_mangle(&rule, &act), 0);
+ KUNIT_EXPECT_EQ(test, rule.remap_port, (u16)0x1234);
+ KUNIT_EXPECT_EQ(test, rule.direction, 1);
+ act.mangle.htype = FLOW_ACT_MANGLE_HDR_TYPE_UDP;
+ KUNIT_EXPECT_EQ(test, fa_mangle(&rule, &act), -EOPNOTSUPP);
+ fa_encode(&rule, 1, 2, 3, data);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 127, 1), 1ULL);
+ rule.l4proto = IPPROTO_UDP;
+ fa_encode(&rule, 1, 2, 3, data);
+ KUNIT_EXPECT_EQ(test, fa_test_bits(data, 127, 1), 0ULL);
+}
+
+static void fa_test_tcp_identity(struct kunit *test)
+{
+ struct fa_rule rule = {
+ .tuple = { cpu_to_be32(0xc0000202), cpu_to_be32(0xc6336402),
+ cpu_to_be16(40000), cpu_to_be16(5001) },
+ .l4proto = IPPROTO_TCP,
+ .remap_ip = 0xcb007101,
+ .remap_port = 45000,
+ .action = FA_ACT_REMAP_IP | FA_ACT_REMAP_PORT,
+ };
+ struct fa_flow entry = { .l4proto = IPPROTO_TCP, .in_port = 3 };
+ struct fa_learned learn = { .valid = true, .l4proto = IPPROTO_UDP };
+ struct fa_tuple reply, control;
+
+ fa_reply_tuple(&rule, &reply);
+ KUNIT_EXPECT_EQ(test, reply.src, rule.tuple.dst);
+ KUNIT_EXPECT_EQ(test, reply.dst, htonl(rule.remap_ip));
+ KUNIT_EXPECT_EQ(test, reply.sport, rule.tuple.dport);
+ KUNIT_EXPECT_EQ(test, reply.dport, htons(rule.remap_port));
+ rule.direction = FA_DIR_DNAT;
+ fa_reply_tuple(&rule, &reply);
+ KUNIT_EXPECT_EQ(test, reply.src, htonl(rule.remap_ip));
+ KUNIT_EXPECT_EQ(test, reply.dst, rule.tuple.src);
+ KUNIT_EXPECT_EQ(test, reply.sport, htons(rule.remap_port));
+ KUNIT_EXPECT_EQ(test, reply.dport, rule.tuple.sport);
+
+ entry.tuple = rule.tuple;
+ control = rule.tuple;
+ KUNIT_EXPECT_TRUE(test, fa_control_match(&entry, &control, 3, true));
+ KUNIT_EXPECT_FALSE(test, fa_control_match(&entry, &control, 2, true));
+ control.sport++;
+ KUNIT_EXPECT_FALSE(test, fa_control_match(&entry, &control, 3, true));
+ KUNIT_EXPECT_TRUE(test, fa_control_match(&entry, &control, 3, false));
+ entry.l4proto = IPPROTO_UDP;
+ KUNIT_EXPECT_FALSE(test, fa_control_match(&entry, &control, 3, false));
+
+ /* Equal TCP/UDP four-tuples must never share a learned hash. A TCP
+ * control packet also invalidates observations from the old lifetime.
+ */
+ learn.tuple = rule.tuple;
+ KUNIT_EXPECT_FALSE(test, fa_learn_matches(&learn, &rule, 0));
+ learn.l4proto = IPPROTO_TCP;
+ KUNIT_EXPECT_TRUE(test, fa_learn_matches(&learn, &rule, 0));
+ KUNIT_EXPECT_FALSE(test, fa_learn_matches(&learn, &rule, 1));
+ rule.l4proto = IPPROTO_UDP;
+ learn.l4proto = IPPROTO_UDP;
+ /* UDP uses the route/neighbour epoch, independent of TCP controls. */
+ KUNIT_EXPECT_TRUE(test, fa_learn_matches(&learn, &rule, 0));
+ KUNIT_EXPECT_FALSE(test, fa_learn_matches(&learn, &rule, 1));
+}
+
+static void fa_test_tcp_rx_control(struct kunit *test)
+{
+ struct bcm_ns_fa *fa = kunit_kzalloc(test, sizeof(*fa), GFP_KERNEL);
+ struct sk_buff *skb;
+ struct ethhdr *eth;
+ struct iphdr *ip;
+ struct tcphdr *tcp;
+ u8 *tag;
+
+ KUNIT_ASSERT_NOT_NULL(test, fa);
+ fa->learned = kunit_kcalloc(test, FA_SLOTS, sizeof(*fa->learned), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, fa->learned);
+ INIT_LIST_HEAD(&fa->entries);
+ spin_lock_init(&fa->learn_lock);
+ skb = alloc_skb(128, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ tag = skb_put_zero(skb, FA_TAG_LEN + ETH_HLEN + sizeof(*ip) + sizeof(*tcp));
+ tag[0] = FA_MISS_OPCODE << 5;
+ tag[1] = 0xab;
+ tag[3] = 2;
+ eth = (void *)(tag + FA_TAG_LEN);
+ eth->h_proto = htons(ETH_P_IP);
+ ip = (void *)(eth + 1);
+ ip->version = 4;
+ ip->ihl = 5;
+ ip->protocol = IPPROTO_TCP;
+ ip->tot_len = htons(sizeof(*ip) + sizeof(*tcp));
+ ip->saddr = htonl(0xc0000202);
+ ip->daddr = htonl(0xc6336402);
+ ip->check = ip_fast_csum(ip, ip->ihl);
+ tcp = (void *)(ip + 1);
+ tcp->doff = 5;
+ tcp->source = htons(40000);
+ tcp->dest = htons(5001);
+ tcp->syn = 1;
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 0U);
+ KUNIT_EXPECT_EQ(test, fa->control_generation, 1UL);
+ tcp->syn = 0;
+ tcp->ack = 1;
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ KUNIT_EXPECT_EQ(test, fa->learned[0].l4proto, (u8)IPPROTO_TCP);
+ KUNIT_EXPECT_EQ(test, fa->learned[0].hash, (u8)0xab);
+ KUNIT_EXPECT_EQ(test, fa->learned[0].generation, 1UL);
+ /* Opcode-zero switch traps contain no hash, but still retire TCP. */
+ tag[0] = 0;
+ tag[1] = 1;
+ tag[2] = BIT(3);
+ tcp->fin = 1;
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->learn_next, 1U);
+ KUNIT_EXPECT_EQ(test, fa->control_generation, 2UL);
+ tcp->fin = 0;
+ tcp->rst = 1;
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->control_generation, 3UL);
+ tcp->doff = 4;
+ bcm_ns_fa_rx(fa, skb);
+ KUNIT_EXPECT_EQ(test, fa->control_generation, 3UL);
+ kfree_skb(skb);
+}
+
+static void fa_test_tcp_control_readiness(struct kunit *test)
+{
+ struct bcm_ns_fa *fa = kunit_kzalloc(test, sizeof(*fa), GFP_KERNEL);
+
+ KUNIT_ASSERT_NOT_NULL(test, fa);
+ fa->regs = (void __iomem *)kunit_kzalloc(test, 0xc0, GFP_KERNEL);
+ fa->learned = kunit_kcalloc(test, FA_SLOTS, sizeof(*fa->learned), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, fa->regs);
+ KUNIT_ASSERT_NOT_NULL(test, fa->learned);
+ mutex_init(&fa->lock);
+ spin_lock_init(&fa->learn_lock);
+ fa_control_state_change(fa, true);
+ KUNIT_EXPECT_TRUE(test, fa->tcp_ready);
+ fa_control_state_change(fa, false);
+ KUNIT_EXPECT_FALSE(test, fa->tcp_ready);
+ fa->stopping = true;
+ fa_control_state_change(fa, true);
+ KUNIT_EXPECT_FALSE(test, fa->tcp_ready);
+}
+
+static void fa_test_lookup_gate_preserves_tags(struct kunit *test)
+{
+ struct bcm_ns_fa *fa = kunit_kzalloc(test, sizeof(*fa), GFP_KERNEL);
+ u32 *regs = kunit_kzalloc(test, 0xc0, GFP_KERNEL);
+ u32 control = FA_CTL_CRC_OVERWRITE | FA_CTL_NO_DA_CHECK;
+ u32 header = FA_BRCM_HW | FA_BRCM_RX | FA_BRCM_TX | FA_BRCM_PARSE_EXCEPTIONS;
+
+ KUNIT_ASSERT_NOT_NULL(test, fa);
+ KUNIT_ASSERT_NOT_NULL(test, regs);
+ fa->regs = (void __iomem *)regs;
+ writel(control, fa->regs + FA_CTL);
+ writel(header, fa->regs + FA_BRCM_HDR);
+ fa->enabled = true;
+ fa->tcp_ready = true;
+ fa->route_mtu_ok = true;
+ fa_lookup_update(fa);
+ KUNIT_EXPECT_EQ(test, readl(fa->regs + FA_L2_ETHERTYPE),
+ (u32)(ETH_P_IP | ETH_P_IPV6 << 16));
+ fa->control_pending = true;
+ fa_lookup_update(fa);
+ KUNIT_EXPECT_EQ(test, readl(fa->regs + FA_L2_ETHERTYPE), 0U);
+ fa->control_pending = false;
+ fa_lookup_update(fa);
+ KUNIT_EXPECT_NE(test, readl(fa->regs + FA_L2_ETHERTYPE), 0U);
+ fa->route_mtu_ok = false;
+ fa_lookup_update(fa);
+ KUNIT_EXPECT_EQ(test, readl(fa->regs + FA_L2_ETHERTYPE), 0U);
+ fa->route_mtu_ok = true;
+ fa->tcp_ready = false;
+ fa_lookup_update(fa);
+ KUNIT_EXPECT_EQ(test, readl(fa->regs + FA_L2_ETHERTYPE), 0U);
+ fa->tcp_ready = true;
+ fa->enabled = false;
+ fa_lookup_update(fa);
+ KUNIT_EXPECT_EQ(test, readl(fa->regs + FA_L2_ETHERTYPE), 0U);
+ KUNIT_EXPECT_EQ(test, readl(fa->regs + FA_CTL), control);
+ KUNIT_EXPECT_EQ(test, readl(fa->regs + FA_BRCM_HDR), header);
+}
+
+static void fa_test_mtu_cleanup(void *ptr)
+{
+ struct bcm_ns_fa *fa = ptr;
+
+ fa_fib_reset(&fa->fib_nb);
+}
+
+static void fa_test_route_mtu(struct kunit *test)
+{
+ struct bcm_ns_fa *fa = kunit_kzalloc(test, sizeof(*fa), GFP_KERNEL);
+ struct dst_metrics metrics = {};
+ struct fib_info fi = { .fib_metrics = &metrics };
+ struct fib_entry_notifier_info entry = {
+ .info.family = AF_INET, .fi = &fi, .tb_id = RT_TABLE_MAIN,
+ .dst = 0xc0000200, .dst_len = 24,
+ };
+
+ KUNIT_ASSERT_NOT_NULL(test, fa);
+ spin_lock_init(&fa->learn_lock);
+ hash_init(fa->mtu_routes);
+ KUNIT_ASSERT_EQ(test, kunit_add_action_or_reset(test, fa_test_mtu_cleanup, fa), 0);
+
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ KUNIT_EXPECT_TRUE(test, fa->route_mtu_ok);
+ metrics.metrics[RTAX_MTU - 1] = 1500;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ KUNIT_EXPECT_TRUE(test, fa->route_mtu_ok);
+ metrics.metrics[RTAX_MTU - 1] = 1200;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ KUNIT_EXPECT_FALSE(test, fa->route_mtu_ok);
+ KUNIT_EXPECT_EQ(test, fa->nr_mtu_routes, 1U);
+ /* Replays/replacements are idempotent; tables and prefixes are distinct. */
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ KUNIT_EXPECT_EQ(test, fa->nr_mtu_routes, 1U);
+ entry.tb_id++;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ entry.dst_len++;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ KUNIT_EXPECT_EQ(test, fa->nr_mtu_routes, 3U);
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_DEL, &entry);
+ entry.dst_len--;
+ metrics.metrics[RTAX_MTU - 1] = 1500;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ KUNIT_EXPECT_FALSE(test, fa->route_mtu_ok);
+ entry.tb_id--;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ KUNIT_EXPECT_EQ(test, fa->nr_mtu_routes, 0U);
+ KUNIT_EXPECT_TRUE(test, fa->route_mtu_ok);
+ /* A registration retry must discard a partial replay before starting over. */
+ metrics.metrics[RTAX_MTU - 1] = 1000;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ fa_fib_reset(&fa->fib_nb);
+ KUNIT_EXPECT_TRUE(test, hash_empty(fa->mtu_routes));
+ KUNIT_EXPECT_TRUE(test, fa->route_mtu_ok);
+}
+
+static void fa_test_route_mtu_capacity(struct kunit *test)
+{
+ struct bcm_ns_fa *fa = kunit_kzalloc(test, sizeof(*fa), GFP_KERNEL);
+ struct dst_metrics metrics = {};
+ struct fib_info fi = { .fib_metrics = &metrics };
+ struct fib_entry_notifier_info entry = {
+ .info.family = AF_INET, .fi = &fi, .tb_id = RT_TABLE_MAIN, .dst_len = 32,
+ };
+ unsigned int i;
+
+ KUNIT_ASSERT_NOT_NULL(test, fa);
+ spin_lock_init(&fa->learn_lock);
+ hash_init(fa->mtu_routes);
+ KUNIT_ASSERT_EQ(test, kunit_add_action_or_reset(test, fa_test_mtu_cleanup, fa), 0);
+ /* Ordinary routes consume no tracking memory, regardless of FIB size. */
+ for (i = 0; i < 10000; i++) {
+ entry.dst = i;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ }
+ KUNIT_EXPECT_EQ(test, fa->nr_mtu_routes, 0U);
+ KUNIT_EXPECT_TRUE(test, fa->route_mtu_ok);
+ metrics.metrics[RTAX_MTU - 1] = 1200;
+ for (i = 0; i <= FA_MTU_ROUTES_MAX; i++) {
+ entry.dst = i;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_REPLACE, &entry);
+ }
+ KUNIT_EXPECT_EQ(test, fa->nr_mtu_routes, (unsigned int)FA_MTU_ROUTES_MAX);
+ KUNIT_EXPECT_TRUE(test, fa->mtu_tracking_failed);
+ for (i = 0; i < FA_MTU_ROUTES_MAX; i++) {
+ entry.dst = i;
+ fa_mtu_route_update(fa, FIB_EVENT_ENTRY_DEL, &entry);
+ }
+ KUNIT_EXPECT_EQ(test, fa->nr_mtu_routes, 0U);
+ KUNIT_EXPECT_FALSE(test, fa->route_mtu_ok);
+ fa_fib_reset(&fa->fib_nb);
+ KUNIT_EXPECT_TRUE(test, fa->route_mtu_ok);
+}
+
+static void fa_test_route_suitability(struct kunit *test)
+{
+ struct dst_metrics metrics = {};
+ struct fib_info fi = { .fib_metrics = &metrics, .fib_nhs = 1 };
+ struct fib_nh_common nhc = {};
+ struct fib_result res = { .fi = &fi, .nhc = &nhc, .type = RTN_UNICAST };
+ struct fa_rule rule = {};
+
+ rule.out = kunit_kzalloc(test, sizeof(*rule.out), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, rule.out);
+ nhc.nhc_dev = rule.out;
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), 0);
+ /* A partial driver-side lookup cannot establish a stale owner. */
+ fi.fib_nhs = 2;
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), -EOPNOTSUPP);
+ fi.fib_nhs = 1;
+ fi.nh = kunit_kzalloc(test, sizeof(*fi.nh), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, fi.nh);
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), -EOPNOTSUPP);
+ fi.nh = NULL;
+ nhc.nhc_lwtstate = kunit_kzalloc(test, sizeof(*nhc.nhc_lwtstate), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, nhc.nhc_lwtstate);
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), -EOPNOTSUPP);
+ nhc.nhc_lwtstate = NULL;
+ metrics.metrics[RTAX_MTU - 1] = 1200;
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), -EOPNOTSUPP);
+ metrics.metrics[RTAX_MTU - 1] = 0;
+ nhc.nhc_gw_family = AF_INET6;
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), -EOPNOTSUPP);
+ nhc.nhc_gw_family = 0;
+ nhc.nhc_dev = NULL;
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), -EOPNOTSUPP);
+ nhc.nhc_dev = rule.out;
+ res.type = RTN_BLACKHOLE;
+ KUNIT_EXPECT_EQ(test, fa_route_supported(&rule, &res), -EOPNOTSUPP);
+}
+
+struct fa_test_owner {
+ struct bcm_ns_fa *fa;
+ unsigned int calls;
+ bool lookup_stopped;
+};
+
+static void fa_test_owner_invalidate(unsigned long cookie)
+{
+ struct fa_test_owner *owner = (struct fa_test_owner *)cookie;
+
+ owner->calls++;
+ owner->lookup_stopped = !readl(owner->fa->regs + FA_L2_ETHERTYPE);
+}
+
+static void fa_test_empty_work(struct work_struct *work)
+{
+}
+
+static void fa_test_invalidation(struct kunit *test)
+{
+ struct bcm_ns_fa *fa = kunit_kzalloc(test, sizeof(*fa), GFP_KERNEL);
+ struct fa_test_owner owner = { .fa = fa };
+ struct fa_flow entry = {
+ .cookie = (unsigned long)&owner,
+ .invalidate = fa_test_owner_invalidate,
+ };
+
+ KUNIT_ASSERT_NOT_NULL(test, fa);
+ fa->regs = (void __iomem *)kunit_kzalloc(test, 0xc0, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, fa->regs);
+ spin_lock_init(&fa->learn_lock);
+ INIT_WORK(&fa->control_work, fa_test_empty_work);
+ INIT_LIST_HEAD(&fa->entries);
+ list_add(&entry.list, &fa->entries);
+ fa->enabled = true;
+ fa->tcp_ready = true;
+ fa->route_mtu_ok = true;
+ fa_lookup_update(fa);
+ fa_invalidate(fa);
+ flush_work(&fa->control_work);
+ KUNIT_EXPECT_EQ(test, owner.calls, 1U);
+ KUNIT_EXPECT_TRUE(test, owner.lookup_stopped);
+ KUNIT_EXPECT_TRUE(test, test_bit(FA_FLOW_CLOSING, &entry.flags));
+ KUNIT_EXPECT_TRUE(test, fa->control_pending);
+ KUNIT_EXPECT_TRUE(test, fa->route_mtu_ok);
+ KUNIT_EXPECT_EQ(test, fa->invalidate_generation, 1UL);
+}
+
+static struct kunit_case fa_test_cases[] = {
+ KUNIT_CASE(fa_test_upper_device_rcu),
+ KUNIT_CASE(fa_test_encode),
+ KUNIT_CASE(fa_test_mac_encode),
+ KUNIT_CASE(fa_test_nat_actions),
+ KUNIT_CASE(fa_test_ethernet_actions),
+ KUNIT_CASE(fa_test_rx_learning),
+ KUNIT_CASE(fa_test_tcp_encode_and_mangle),
+ KUNIT_CASE(fa_test_tcp_identity),
+ KUNIT_CASE(fa_test_tcp_rx_control),
+ KUNIT_CASE(fa_test_tcp_control_readiness),
+ KUNIT_CASE(fa_test_lookup_gate_preserves_tags),
+ KUNIT_CASE(fa_test_route_mtu),
+ KUNIT_CASE(fa_test_route_mtu_capacity),
+ KUNIT_CASE(fa_test_route_suitability),
+ KUNIT_CASE(fa_test_invalidation),
+ {}
+};
+
+static struct kunit_suite fa_test_suite = {
+ .name = "bcm-ns-fa",
+ .test_cases = fa_test_cases,
+};
+
+kunit_test_suite(fa_test_suite);
diff --git a/drivers/net/ethernet/broadcom/bcm-ns-fa.c b/drivers/net/ethernet/broadcom/bcm-ns-fa.c
index 8665ba94cbab..4ee8e5c18bb8 100644
--- a/drivers/net/ethernet/broadcom/bcm-ns-fa.c
+++ b/drivers/net/ethernet/broadcom/bcm-ns-fa.c
@@ -1893,5 +1893,9 @@ struct bcm_ns_fa *bcm_ns_fa_init(struct net_device *dev)
}
EXPORT_SYMBOL_GPL(bcm_ns_fa_init);
+#if IS_ENABLED(CONFIG_BCM_NS_FA_KUNIT_TEST)
+#include "bcm-ns-fa-test.c"
+#endif
+
MODULE_DESCRIPTION("Broadcom Northstar IPv4 TCP/UDP NAT flow accelerator");
MODULE_LICENSE("GPL");
diff --git a/net/dsa/Kconfig b/net/dsa/Kconfig
index 2dffff2f6c34..33fa69d3f216 100644
--- a/net/dsa/Kconfig
+++ b/net/dsa/Kconfig
@@ -78,6 +78,17 @@ config NET_DSA_TAG_BRCM_FA
switch port. Transmit uses the ordinary prepended Broadcom tag.
Enable this for a B53 switch explicitly connected to an accelerator.
+config NET_DSA_TAG_BRCM_FA_KUNIT_TEST
+ bool "KUnit tests for Broadcom FA tags" if !KUNIT_ALL_TESTS
+ depends on KUNIT=y && NET_DSA_TAG_BRCM_FA
+ default KUNIT_ALL_TESTS
+ help
+ Tests are compiled into the parent driver and require built-in KUnit.
+ Verify FA miss demultiplexing, parser-status handling, and rejection
+ of unsupported opcodes without changing ordinary Broadcom tagging.
+ Exercise source-port demultiplexing and consumption of the metadata
+ tag through the receive path, without requiring a physical switch.
+
config NET_DSA_TAG_HELLCREEK
tristate "Tag driver for Hirschmann Hellcreek TSN switches"
help
diff --git a/net/dsa/tag_brcm.c b/net/dsa/tag_brcm.c
index 8d13a3c7de5e..4a2c0a51db97 100644
--- a/net/dsa/tag_brcm.c
+++ b/net/dsa/tag_brcm.c
@@ -526,5 +526,9 @@ static struct dsa_tag_driver *dsa_tag_driver_array[] = {
module_dsa_tag_drivers(dsa_tag_driver_array);
+#if IS_ENABLED(CONFIG_NET_DSA_TAG_BRCM_FA_KUNIT_TEST)
+#include "tag_brcm_test.c"
+#endif
+
MODULE_DESCRIPTION("DSA tag driver for Broadcom switches using in-frame headers");
MODULE_LICENSE("GPL");
diff --git a/net/dsa/tag_brcm_test.c b/net/dsa/tag_brcm_test.c
new file mode 100644
index 000000000000..85499461e86b
--- /dev/null
+++ b/net/dsa/tag_brcm_test.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Included in tag_brcm.c: test the real receive path with a minimal DSA tree. */
+#include <kunit/test.h>
+
+struct brcm_fa_test_ctx {
+ struct dsa_switch_tree tree;
+ struct dsa_switch ds;
+ struct dsa_port cpu, user;
+ struct net_device *conduit, *port;
+};
+
+static void brcm_fa_test_free_netdev(void *ptr)
+{
+ free_netdev(ptr);
+}
+
+static int brcm_fa_test_init(struct kunit *test)
+{
+ struct brcm_fa_test_ctx *ctx;
+ struct dsa_user_priv *priv;
+ int err;
+
+ ctx = kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, ctx);
+ ctx->conduit = alloc_etherdev(0);
+ KUNIT_ASSERT_NOT_NULL(test, ctx->conduit);
+ err = kunit_add_action_or_reset(test, brcm_fa_test_free_netdev, ctx->conduit);
+ if (err)
+ return err;
+ ctx->port = alloc_etherdev(sizeof(*priv));
+ KUNIT_ASSERT_NOT_NULL(test, ctx->port);
+ err = kunit_add_action_or_reset(test, brcm_fa_test_free_netdev, ctx->port);
+ if (err)
+ return err;
+ INIT_LIST_HEAD(&ctx->tree.ports);
+ ctx->ds.dst = &ctx->tree;
+ ctx->cpu.index = 8;
+ ctx->cpu.type = DSA_PORT_TYPE_CPU;
+ ctx->cpu.ds = &ctx->ds;
+ ctx->cpu.dst = &ctx->tree;
+ ctx->user.ds = &ctx->ds;
+ ctx->user.dst = &ctx->tree;
+ ctx->user.index = 3;
+ ctx->user.type = DSA_PORT_TYPE_USER;
+ ctx->user.user = ctx->port;
+ list_add(&ctx->user.list, &ctx->tree.ports);
+ ctx->conduit->dsa_ptr = &ctx->cpu;
+ priv = netdev_priv(ctx->port);
+ priv->dp = &ctx->user;
+ test->priv = ctx;
+ return 0;
+}
+
+static struct sk_buff *brcm_fa_test_skb(u8 opcode, u8 status, u8 port)
+{
+ struct sk_buff *skb = alloc_skb(64, GFP_KERNEL);
+ u8 *data;
+
+ if (!skb)
+ return NULL;
+ data = skb_put_zero(skb, 64);
+ data[0] = opcode << BRCM_OPCODE_SHIFT;
+ data[1] = 0xff; /* Hash: not a classification identifier. */
+ data[2] = status;
+ data[3] = port;
+ /* Reproduce eth_type_trans(): the tagger gets data after ETH_HLEN. */
+ skb_reset_mac_header(skb);
+ skb_pull(skb, ETH_HLEN);
+ return skb;
+}
+
+static void brcm_fa_test_miss(struct kunit *test)
+{
+ struct brcm_fa_test_ctx *ctx = test->priv;
+ struct sk_buff *skb, *result;
+ u8 *start;
+
+ /* A parser failure is for the software IP stack to handle; it must
+ * not be confused with the reserved bits in a normal switch reason.
+ */
+ skb = brcm_fa_test_skb(BRCM_FA_OPCODE_MISS, 0xff, 3);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ start = skb->data;
+ result = brcm_fa_tag_rcv(skb, ctx->conduit);
+ KUNIT_ASSERT_NOT_NULL(test, result);
+ KUNIT_EXPECT_PTR_EQ(test, result->dev, ctx->port);
+ KUNIT_EXPECT_PTR_EQ(test, result->data, start + BRCM_TAG_LEN);
+ KUNIT_EXPECT_EQ(test, result->len, 64U - ETH_HLEN - BRCM_TAG_LEN);
+ kfree_skb(result);
+}
+
+static void brcm_fa_test_opcodes(struct kunit *test)
+{
+ struct brcm_fa_test_ctx *ctx = test->priv;
+ struct sk_buff *skb, *result;
+ int opcode;
+
+ for (opcode = 0; opcode < 8; opcode++) {
+ skb = brcm_fa_test_skb(opcode, 0, 3);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ result = brcm_fa_tag_rcv(skb, ctx->conduit);
+ if (!opcode || opcode == BRCM_FA_OPCODE_MISS) {
+ KUNIT_ASSERT_NOT_NULL(test, result);
+ kfree_skb(result);
+ } else {
+ KUNIT_EXPECT_PTR_EQ(test, result, NULL);
+ }
+ }
+}
+
+static void brcm_fa_test_strict_normal_tag(struct kunit *test)
+{
+ struct brcm_fa_test_ctx *ctx = test->priv;
+ struct sk_buff *skb;
+
+ skb = brcm_fa_test_skb(BRCM_FA_OPCODE_MISS, 0, 3);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ KUNIT_EXPECT_PTR_EQ(test, brcm_tag_rcv_ll(skb, ctx->conduit, ETH_HLEN, false), NULL);
+ skb = brcm_fa_test_skb(0, BRCM_EG_RC_RSVD, 3);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ KUNIT_EXPECT_PTR_EQ(test, brcm_fa_tag_rcv(skb, ctx->conduit), NULL);
+ skb = brcm_fa_test_skb(BRCM_FA_OPCODE_MISS, 0, 7);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ KUNIT_EXPECT_PTR_EQ(test, brcm_fa_tag_rcv(skb, ctx->conduit), NULL);
+}
+
+struct brcm_fa_control_test {
+ unsigned int calls, port;
+ __be16 protocol;
+};
+
+static void brcm_fa_test_receive(void *priv, struct sk_buff *skb, unsigned int port)
+{
+ struct brcm_fa_control_test *state = priv;
+
+ state->calls++;
+ state->port = port;
+ state->protocol = eth_hdr(skb)->h_proto;
+}
+
+static void brcm_fa_test_auxiliary_control(struct kunit *test)
+{
+ struct brcm_fa_test_ctx *ctx = test->priv;
+ struct brcm_fa_control_test state = {};
+ struct brcm_fa_control control = {
+ .conduit = ctx->port,
+ .receive = brcm_fa_test_receive,
+ .priv = &state,
+ };
+ struct dsa_port *data_cpu;
+ struct sk_buff *skb, *result;
+ u8 *data;
+
+ data_cpu = kunit_kzalloc(test, sizeof(*data_cpu), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, data_cpu);
+ data_cpu->ds = &ctx->ds;
+ data_cpu->type = DSA_PORT_TYPE_CPU;
+ data_cpu->index = 8;
+ data_cpu->conduit = ctx->port;
+ list_add(&data_cpu->list, &ctx->tree.ports);
+ ctx->cpu.index = 5;
+ skb = alloc_skb(64, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, skb);
+ data = skb_put_zero(skb, 64);
+ data[0] = 2;
+ data[6] = 2;
+ data[13] = 1; /* Classification ID. */
+ data[14] = BRCM_EG_RC_PROT_TERM;
+ data[15] = 3;
+ put_unaligned_be16(ETH_P_IP, data + 16);
+ skb_reset_mac_header(skb);
+ skb_pull(skb, ETH_HLEN);
+ brcm_fa_control_register(&control);
+ result = brcm_fa_tag_rcv(skb, ctx->conduit);
+ brcm_fa_control_unregister(&control);
+ KUNIT_ASSERT_NOT_NULL(test, result);
+ KUNIT_EXPECT_PTR_EQ(test, result->dev, ctx->port);
+ KUNIT_EXPECT_EQ(test, result->len, 64U - ETH_HLEN - BRCM_TAG_LEN);
+ KUNIT_EXPECT_EQ(test, state.calls, 1U);
+ KUNIT_EXPECT_EQ(test, state.port, 3U);
+ KUNIT_EXPECT_EQ(test, state.protocol, htons(ETH_P_IP));
+ kfree_skb(result);
+}
+
+static struct kunit_case brcm_fa_test_cases[] = {
+ KUNIT_CASE(brcm_fa_test_miss),
+ KUNIT_CASE(brcm_fa_test_auxiliary_control),
+ KUNIT_CASE(brcm_fa_test_opcodes),
+ KUNIT_CASE(brcm_fa_test_strict_normal_tag),
+ {}
+};
+
+static struct kunit_suite brcm_fa_test_suite = {
+ .name = "brcm-fa-tag",
+ .init = brcm_fa_test_init,
+ .test_cases = brcm_fa_test_cases,
+};
+
+kunit_test_suite(brcm_fa_test_suite);
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index f565f87b3a6c..311a1910f6be 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -719,14 +719,14 @@ config NF_FLOW_TABLE
To compile it as a module, choose M here.
config NF_FLOW_TABLE_KUNIT_TEST
- bool "KUnit tests for flowtable ownership" if !KUNIT_ALL_TESTS
+ bool "KUnit tests for flowtable ownership and cached paths" if !KUNIT_ALL_TESTS
depends on KUNIT=y && NF_FLOW_TABLE
default KUNIT_ALL_TESTS
help
- Test conntrack offload ownership during flowtable teardown.
- Require hash and asynchronous hardware deletion before re-enrolment.
- Verify that late notifications cannot release a replacement owner.
- These private tests are embedded in the flowtable core.
+ Test flowtable ownership during asynchronous hardware deletion and
+ repeated teardown notifications, and validate cached forwarding paths.
+ Tests cover route expiry, neighbour changes and unsupported contexts.
+ The tests are embedded in their parent translation units.
config NF_FLOW_TABLE_PROCFS
bool "Supply flow table statistics in procfs"
diff --git a/net/netfilter/nf_flow_table_core_test.c b/net/netfilter/nf_flow_table_core_test.c
index 378a1c60b50b..5c7645239d2e 100644
--- a/net/netfilter/nf_flow_table_core_test.c
+++ b/net/netfilter/nf_flow_table_core_test.c
@@ -72,9 +72,94 @@ static void nf_flow_test_hardware_retirement(struct kunit *test)
rhashtable_destroy(&table.rhashtable);
}
+static struct dst_entry *nf_flow_test_gc_dst_check(struct dst_entry *dst, u32 cookie)
+{
+ return dst->error ? NULL : dst;
+}
+
+static void nf_flow_test_gc_routes(struct kunit *test)
+{
+ static const struct {
+ const char *name;
+ enum flow_offload_xmit_type type;
+ bool keep_dst;
+ bool tunnel;
+ bool check;
+ } cases[] = {
+ { "software direct", FLOW_OFFLOAD_XMIT_DIRECT, false, false, false },
+ { "validation only", FLOW_OFFLOAD_XMIT_DIRECT, true, false, false },
+ { "direct tunnel", FLOW_OFFLOAD_XMIT_DIRECT, false, true, true },
+ { "HW direct tunnel", FLOW_OFFLOAD_XMIT_DIRECT, true, true, true },
+ { "neighbour", FLOW_OFFLOAD_XMIT_NEIGH, false, false, true },
+ { "HW neighbour", FLOW_OFFLOAD_XMIT_NEIGH, true, false, true },
+ { "XFRM", FLOW_OFFLOAD_XMIT_XFRM, true, false, true },
+ };
+ static struct dst_ops ops = {
+ .family = AF_INET,
+ .check = nf_flow_test_gc_dst_check,
+ };
+ static struct nf_flowtable_type type;
+ unsigned int i, dir;
+
+ for (i = 0; i < ARRAY_SIZE(cases); i++) {
+ struct nf_flowtable table = { .type = &type };
+ struct nf_flow_route route = {};
+ struct flow_offload *flow;
+ struct net_device *dev;
+ struct rtable *rt;
+ struct nf_conn *ct;
+
+ flow = kunit_kzalloc(test, sizeof(*flow), GFP_KERNEL);
+ dev = kunit_kzalloc(test, sizeof(*dev), GFP_KERNEL);
+ rt = kunit_kcalloc(test, FLOW_OFFLOAD_DIR_MAX, sizeof(*rt), GFP_KERNEL);
+ ct = kunit_kzalloc(test, sizeof(*ct), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, flow);
+ KUNIT_ASSERT_NOT_NULL(test, dev);
+ KUNIT_ASSERT_NOT_NULL(test, rt);
+ KUNIT_ASSERT_NOT_NULL(test, ct);
+ dev_net_set(dev, &init_net);
+ dev->mtu = 1500;
+ ct->tuplehash[0].tuple.dst.protonum = IPPROTO_ICMP;
+ refcount_set(&ct->ct_general.use, 2);
+ flow->ct = ct;
+ flow->timeout = nf_flowtable_time_stamp + NF_FLOW_TIMEOUT;
+ /* Avoid queuing real DELETE/STATS work from this private GC test. */
+ set_bit(NF_FLOW_HW, &flow->flags);
+ set_bit(NF_FLOW_HW_DYING, &flow->flags);
+ set_bit(IPS_OFFLOAD_BIT, &ct->status);
+ for (dir = 0; dir < FLOW_OFFLOAD_DIR_MAX; dir++) {
+ struct dst_entry *dst = &rt[dir].dst;
+
+ dst->dev = dev;
+ dst->ops = &ops;
+ dst->obsolete = DST_OBSOLETE_FORCE_CHK;
+ dst_init_metrics(dst, dst_default_metrics.metrics, true);
+ /* One fixture reference and one transferred route reference. */
+ rcuref_init(&dst->__rcuref, 2);
+ flow->tuplehash[dir].tuple.l3proto = NFPROTO_IPV4;
+ route.tuple[dir].dst = dst;
+ route.tuple[dir].xmit_type = cases[i].type;
+ route.tuple[dir].in.num_tuns = cases[i].tunnel;
+ }
+ flow_offload_route_init(flow, &route, cases[i].keep_dst);
+ /* Simulate the expired dst returned after an IPv4 generation change. */
+ rt[0].dst.error = ENETUNREACH;
+ KUNIT_EXPECT_EQ_MSG(test,
+ !nf_flow_dst_check(&flow->tuplehash[0].tuple), cases[i].check,
+ "packet path: %s", cases[i].name);
+ nf_flow_offload_gc_step(&table, flow, NULL);
+ KUNIT_EXPECT_EQ_MSG(test, test_bit(NF_FLOW_TEARDOWN, &flow->flags),
+ cases[i].check, "GC path: %s", cases[i].name);
+ flow_offload_route_release(flow);
+ for (dir = 0; dir < FLOW_OFFLOAD_DIR_MAX; dir++)
+ KUNIT_EXPECT_EQ(test, rcuref_read(&rt[dir].dst.__rcuref), 1);
+ }
+}
+
static struct kunit_case nf_flow_lifetime_cases[] = {
KUNIT_CASE(nf_flow_test_teardown_repeat),
KUNIT_CASE(nf_flow_test_hardware_retirement),
+ KUNIT_CASE(nf_flow_test_gc_routes),
{}
};
diff --git a/net/netfilter/nf_flow_table_offload.c b/net/netfilter/nf_flow_table_offload.c
index 1f3adf70f598..272e96fa466b 100644
--- a/net/netfilter/nf_flow_table_offload.c
+++ b/net/netfilter/nf_flow_table_offload.c
@@ -1382,3 +1382,7 @@ void nf_flow_table_offload_exit(void)
destroy_workqueue(nf_flow_offload_del_wq);
destroy_workqueue(nf_flow_offload_stats_wq);
}
+
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE_KUNIT_TEST)
+#include "nf_flow_table_offload_test.c"
+#endif
diff --git a/net/netfilter/nf_flow_table_offload_test.c b/net/netfilter/nf_flow_table_offload_test.c
new file mode 100644
index 000000000000..7e1764a8298a
--- /dev/null
+++ b/net/netfilter/nf_flow_table_offload_test.c
@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <kunit/test.h>
+
+struct nf_flow_test_path {
+ struct dst_entry dst;
+ struct neighbour neigh;
+};
+
+struct nf_flow_test_ctx {
+ struct flow_offload flow;
+ struct nf_flow_test_path paths[FLOW_OFFLOAD_DIR_MAX];
+ struct net_device *dev;
+};
+
+static struct dst_entry *nf_flow_test_dst_check(struct dst_entry *dst, u32 cookie)
+{
+ return !dst->error && cookie == 42 ? dst : NULL;
+}
+
+static struct neighbour *nf_flow_test_neigh(const struct dst_entry *dst,
+ struct sk_buff *skb, const void *address)
+{
+ struct nf_flow_test_path *path = container_of(dst, struct nf_flow_test_path, dst);
+
+ return neigh_clone(&path->neigh);
+}
+
+static int nf_flow_test_init(struct kunit *test)
+{
+ static struct dst_ops ops = {
+ .family = AF_INET,
+ .check = nf_flow_test_dst_check,
+ .neigh_lookup = nf_flow_test_neigh,
+ };
+ const u8 mac[ETH_ALEN] = { 2, 0, 0, 0, 0, 1 };
+ struct nf_flow_test_ctx *ctx;
+ unsigned int dir;
+
+ ctx = kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->dev = kunit_kzalloc(test, sizeof(*ctx->dev), GFP_KERNEL);
+ if (!ctx->dev)
+ return -ENOMEM;
+ ctx->dev->ifindex = 10;
+ ctx->dev->addr_len = ETH_ALEN;
+ for (dir = 0; dir < FLOW_OFFLOAD_DIR_MAX; dir++) {
+ struct flow_offload_tuple *tuple = &ctx->flow.tuplehash[dir].tuple;
+ struct nf_flow_test_path *path = &ctx->paths[dir];
+
+ path->dst.dev = ctx->dev;
+ path->dst.ops = &ops;
+ path->dst.obsolete = DST_OBSOLETE_FORCE_CHK;
+ path->neigh.dev = ctx->dev;
+ path->neigh.nud_state = NUD_REACHABLE;
+ refcount_set(&path->neigh.refcnt, 1);
+ seqlock_init(&path->neigh.ha_lock);
+ ether_addr_copy(path->neigh.ha, mac);
+ tuple->dir = dir;
+ tuple->dst_cache = &path->dst;
+ tuple->dst_cookie = 42;
+ tuple->dst_validation_only = true;
+ tuple->xmit_type = FLOW_OFFLOAD_XMIT_DIRECT;
+ tuple->out.ifidx = ctx->dev->ifindex;
+ ether_addr_copy(tuple->out.h_dest, mac);
+ }
+ test->priv = ctx;
+ return 0;
+}
+
+static void nf_flow_test_cached_path(struct kunit *test)
+{
+ struct nf_flow_test_ctx *ctx = test->priv;
+ unsigned long cookie = (unsigned long)&ctx->flow.tuplehash[0].tuple;
+
+ /* Validation uses the selected dst and its cookie, without a new FIB
+ * lookup that could omit mark/DSCP/oif from the owner's routing context.
+ */
+ KUNIT_EXPECT_EQ(test, nf_flow_offload_validate(cookie), 0);
+ ctx->paths[1].dst.error = ENETUNREACH;
+ /* On-demand validation still checks a driver-only reference. */
+ KUNIT_EXPECT_TRUE(test, nf_flow_dst_check(&ctx->flow.tuplehash[1].tuple));
+ KUNIT_EXPECT_EQ(test, nf_flow_offload_validate(cookie), -ESTALE);
+ ctx->paths[1].dst.error = 0;
+ ctx->flow.tuplehash[1].tuple.dst_cookie = 41;
+ KUNIT_EXPECT_EQ(test, nf_flow_offload_validate(cookie), -ESTALE);
+ KUNIT_EXPECT_EQ(test, refcount_read(&ctx->paths[0].neigh.refcnt), 1);
+}
+
+static void nf_flow_test_neighbour_change(struct kunit *test)
+{
+ struct nf_flow_test_ctx *ctx = test->priv;
+ unsigned long cookie = (unsigned long)&ctx->flow.tuplehash[1].tuple;
+
+ ctx->paths[0].neigh.ha[5]++;
+ KUNIT_EXPECT_EQ(test, nf_flow_offload_validate(cookie), -ESTALE);
+ ctx->paths[0].neigh.ha[5]--;
+ ctx->paths[0].neigh.nud_state = NUD_FAILED;
+ KUNIT_EXPECT_EQ(test, nf_flow_offload_validate(cookie), -ESTALE);
+ KUNIT_EXPECT_EQ(test, refcount_read(&ctx->paths[0].neigh.refcnt), 1);
+}
+
+static void nf_flow_test_unsupported_context(struct kunit *test)
+{
+ struct nf_flow_test_ctx *ctx = test->priv;
+ unsigned long cookie = (unsigned long)&ctx->flow.tuplehash[0].tuple;
+ struct dst_entry *dst = ctx->flow.tuplehash[0].tuple.dst_cache;
+
+ ctx->flow.tuplehash[0].tuple.dst_cache = NULL;
+ KUNIT_EXPECT_EQ(test, nf_flow_offload_validate(cookie), -EOPNOTSUPP);
+ ctx->flow.tuplehash[0].tuple.dst_cache = dst;
+ ctx->flow.tuplehash[0].tuple.out.ifidx++;
+ KUNIT_EXPECT_EQ(test, nf_flow_offload_validate(cookie), -EOPNOTSUPP);
+}
+
+static struct kunit_case nf_flow_path_cases[] = {
+ KUNIT_CASE(nf_flow_test_cached_path),
+ KUNIT_CASE(nf_flow_test_neighbour_change),
+ KUNIT_CASE(nf_flow_test_unsupported_context),
+ {}
+};
+
+static struct kunit_suite nf_flow_path_suite = {
+ .name = "nf-flow-path",
+ .init = nf_flow_test_init,
+ .test_cases = nf_flow_path_cases,
+};
+
+kunit_test_suite(nf_flow_path_suite);
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH net-next 9/9] docs: net: document Northstar FA setup and NAT qualification
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (7 preceding siblings ...)
2026-11-16 22:25 ` [PATCH net-next 8/9] net: add KUnit coverage for FA and flowtable path validation Florian Fainelli
@ 2026-11-16 22:25 ` Florian Fainelli
2026-11-16 22:38 ` [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload netdev-bot+sinfo
9 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:25 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
open list, open list:DOCUMENTATION, open list:NETFILTER,
open list:NETFILTER
Describe the supported topology, GMAC2 data and GMAC0 exception wiring,
configuration, counters and software fallback. Provide classic/software/
hardware NAT controls and same-live-tuple TTL, MTU and ingress-policy
checks. Cover live route/neighbour changes, reachability probes and
read/insertion/deletion completion injection with reprobe recovery.
Explain that every IPv4 route, rule or nexthop event retires all
hardware owners, including unrelated and local-table updates. Document
the resulting offload-residency cost and distinguish actual cached-path
validation from the driver's incomplete FIB eligibility probe. Keep
campaign history and review status in the accompanying package.
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
---
.../ethernet/broadcom/bcm-ns-fa.rst | 507 ++++++++++++++++++
.../device_drivers/ethernet/index.rst | 1 +
MAINTAINERS | 1 +
3 files changed, 509 insertions(+)
create mode 100644 Documentation/networking/device_drivers/ethernet/broadcom/bcm-ns-fa.rst
diff --git a/Documentation/networking/device_drivers/ethernet/broadcom/bcm-ns-fa.rst b/Documentation/networking/device_drivers/ethernet/broadcom/bcm-ns-fa.rst
new file mode 100644
index 000000000000..2bc6cc5d94ce
--- /dev/null
+++ b/Documentation/networking/device_drivers/ethernet/broadcom/bcm-ns-fa.rst
@@ -0,0 +1,507 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+Broadcom Northstar flow accelerator
+===================================
+
+Status and scope
+----------------
+
+The driver supports BCM4708, BCM53011 and BCM53012. IPv4 TCP and UDP
+SNAT/DNAT have been validated on BCM953012ER (BCM53012), including normal
+and software-flowtable controls, external wire checks, hardware table
+pressure, software fallback, natural aging, teardown and recovery.
+
+BCM4708 and BCM53011 remain unvalidated. BCM53012 validation covers the
+limited topology below; maximum throughput has not been established.
+
+The driver accepts nftables flowtable offload through ``TC_SETUP_FT`` on
+bgmac. DSA forwards flowtable binds from its user ports to the GMAC2 conduit.
+The FA is inline on the UniMAC2 / switch IMP0 (port 8) path. GMAC0 and GMAC1
+are pass-through paths and cannot accelerate these flows.
+
+The initial supported configuration is deliberately limited:
+
+* IPv4 TCP/UDP with source NAT or destination NAT, including port translation.
+* Two standalone, untagged DSA router ports on the same switch, both using
+ GMAC2 as their conduit, in the initial network namespace.
+* Distinct unicast MAC addresses on the LAN and WAN router interfaces.
+* MTU 1500 on both router interfaces, one hardware flowtable, no upper devices.
+
+IPv6, VLAN devices, PPPoE, tunnels, bridges, bonds, hairpin forwarding,
+plain routing, multipath/nexthop-object routes, and simultaneous source and
+destination NAT in one direction
+are rejected for hardware offload. They can still use software forwarding.
+Hardware admission requires a second, operational CPU connection from switch
+port 5 to GMAC0. B53 CFP redirects IPv4 exceptions through this connection,
+physically bypassing FA. This includes TCP SYN/FIN/RST, TTL 0/1, IP lengths
+above 1500, and packets that do not match the input port's router MAC. TCP
+also requires the flowtable FIN/RST exclusion mask. Boards without the
+auxiliary connection retain software forwarding for both protocols.
+
+The hardware has 1024 directional IPv4 entries, arranged in 256 four-way
+buckets, four router MAC entries, and 128 next-hop entries. Capacity exhaustion
+returns an error to the flowtable infrastructure, leaving traffic in software.
+One connection normally needs two directional entries. Next-hop entries are
+shared by destination MAC; router MACs are shared with an internal/external
+side attribute. The driver learns the tuple hash from opcode-2 miss tags,
+so it does not implement an assumed software equivalent of the hardware CRC.
+A finite RX hash cache can also cause an offload attempt to fall back to
+software under miss floods.
+
+The new ``brcm-fa`` DSA tag protocol carries the prepended FA miss descriptor.
+Its hash, bucket, and parser status fields differ from the ordinary switch
+tag. The bgmac receive hook observes the metadata for hash learning; the DSA
+tagger consumes it and selects the input port. Timestamp descriptors are not
+supported. The ordinary ``brcm`` and ``brcm-prepend`` taggers retain their
+opcode validation.
+
+Implementation notes
+--------------------
+
+Table operations use the documented indirect access sequence. DATA0 triggers
+the transfer; reads discard the initial DATA0 value and wait for the fetch
+before consuming DATA7 through DATA0. Every wait is bounded. The driver keeps
+clear-on-read disabled because clearing on a read gives unreliable hit counts.
+Instead it reads cumulative packet counts and computes wrapping deltas in
+software.
+
+The indirect read port is released after each access, including an access
+that times out. Leaving read mode armed prevents per-entry hit counters from
+updating and breaks the flowtable's activity accounting.
+
+Live packet lookups must not overlap software table transactions. On the
+tested BCM53012, that overlap can advance the indirect address more than
+once and persistently copy one flow's key into another slot. Before each
+transaction, the driver requests GMAC2 CPU flow control, waits for pause to
+assert and allows an in-flight frame to drain. It preserves automatic flow
+control and restores the previous MAC control on every exit path. The switch
+CPU port must honor pause, with matching internal link speeds at both ends.
+The FA stays in normal mode so packet and Broadcom-header processing retain
+their usual paths. Completion checks cover both the busy bit and the
+auto-incremented table address. Frequent statistics and debugfs reads also
+consume time on the internal link; include that work in performance tests.
+
+Router destination MAC validation is enabled before publishing a forwarding
+entry. With no installed entries, it is disabled to obtain hashes for the
+first unknown tuples. VLAN TPIDs are disabled in the FA parser so tagged
+packets cannot match an installed untagged tuple merely by sharing its inner
+five-tuple, including the TCP/UDP protocol bit. When both CPU connections
+are explicitly described, B53 CFP reserves rules for TCP SYN/FIN/RST, TTL
+0/1 and IPv4 total lengths above 1500. These redirect to port 5 with a
+classification and termination reason. Nine masked prefixes cover every
+length from 1501 through 65535. Per-port allow rules require the router's
+destination MAC on that physical input port; a final IPv4 rule redirects
+everything else to port 5. MAC changes disable admission while updating
+these rules. Both
+meter outcomes use the same action. Ordinary data uses port 8 only; flooding
+to the auxiliary CPU is disabled. Standalone ports use basic EAP mode, with
+private VLAN membership restricting forwarding to the CPU connections.
+VLAN, EAP and STP filtering remain enabled. CFP classification is disabled
+on bridge members and CPU ports.
+
+FA must ignore switch exception metadata to accelerate standalone data,
+which normally carries the switch flooding reason. Consequently, an inline
+SYN/FIN/RST trap through port 8 is insufficient: FA would accelerate those
+packets as well. The auxiliary GMAC0 path avoids that lookup entirely. It
+uses embedded Broadcom tags; the port 8 data path uses prepended FA metadata.
+The tagger dispatches control packets to FA before delivering them to Linux.
+The auxiliary conduit uses promiscuous mode to receive the router port MACs.
+Before disabling its CFP redirects, B53 disables hardware lookup, retires
+all entries and blocks admission. Re-enabling the path permits fresh
+connections to offload.
+
+On a valid TCP control packet, NAPI marks the matching direction as closing
+and immediately disables IPv4 lookup in the parser. Broadcom-tag relocation
+remains active so the control packet can traverse the software forwarding
+path. A worker removes both NAT directions before resuming lookups. Global
+bypass is unsuitable here because it also disables tag relocation. IP options
+and fragments retire pairs by their L3 identity before Linux can process a
+reused tuple. RCU protects entry
+identity, and a generation check prevents an insertion learned before the
+control packet from racing past it. Closing rows retain their final packet
+delta for software accounting until the flowtable deletes their ownership.
+
+Hardware packet counters refresh flowtable last-used time and packet
+accounting. The hardware has no per-entry byte counter; reported byte totals
+therefore do not include accelerated traffic. Three-bit timestamps are not
+used for aging. Debugfs reads do not consume the driver's accounting deltas.
+
+Netdevice shutdown, address/MTU changes, new active switch ports, and upper
+interface changes flush entries. Every IPv4 FIB route, rule and nexthop change
+flushes all FA-owned flows, including unrelated prefixes and local-table
+entries. This is intentional: the rule does not carry enough routing-policy
+context to prove that an event cannot affect an installed path. Work is
+coalesced, but even unrelated routing churn can reduce offload residency.
+Redirects and next-hop MAC or validity changes also disable lookup and schedule
+entry retirement. The driver also notifies the flowtable owner to retire
+its cached software path; merely removing hardware rows would leave stale
+direct-transmit routes and destination MACs in use. Generation checks reject observations and publications
+that raced with invalidation. Before admitting a rule, the flowtable owner
+validates its actual cached routes and direct-path neighbours. Hardware
+flowtables retain validation-only route references, so this check preserves the mark,
+DSCP and output-interface context used when those paths were selected.
+These references do not enable automatic route checks for plain direct
+flows in other drivers or hardware-enabled flowtables. Existing direct
+tunnel, neighbour and XFRM route checks keep their previous behaviour.
+A proven stale cached route or neighbour retires the software owner too.
+The driver's separate FIB lookup is only a hardware eligibility probe:
+it lacks the complete policy context, so any mismatch declines hardware
+offload without tearing down a valid software owner. Unsupported route types,
+such as multipath, nexthop objects and lightweight tunnels, decline hardware
+admission without tearing down valid software flowtable forwarding.
+Valid neighbour aging transitions do not unnecessarily flush entries.
+Hardware counter activity drives Linux
+neighbour reachability probes, so an accelerated connection cannot leave a
+stale neighbour untested indefinitely.
+
+The flowtable classifier provides no per-route MTU. The driver therefore
+keeps all flows in software while any IPv4 route in the initial network
+namespace has an explicit MTU below 1500. The initial FIB notifier replay
+seeds a hash of low-MTU table/prefix pairs; route replacements and deletions
+update it incrementally without further FIB dumps. Ordinary routes use no
+tracking memory. If allocation fails or more than 1024 low-MTU prefixes
+need tracking, hardware admission remains disabled until reprobe; route
+updates and software forwarding continue. This conservative restriction also
+applies to unrelated routes. The CFP length trap handles packets exceeding
+the admitted interface MTU, including packets of an already installed tuple.
+
+A table timeout disables parser lookup
+and prevents further hardware insertion until reprobe. Tag relocation remains
+active for software fallback. Reconfiguration can
+leave an existing conntrack entry temporarily labelled HW_OFFLOAD after its
+hardware entries have been flushed. Use fresh connections after topology or
+ruleset changes; the label alone is never proof of forwarding by hardware.
+
+Device tree and kernel configuration
+------------------------------------
+
+Enable at least::
+
+ CONFIG_BGMAC_BCMA=y
+ CONFIG_BCM_NS_FA=y
+ CONFIG_NET_DSA=y
+ CONFIG_B53=y
+ CONFIG_B53_SRAB_DRIVER=y
+ CONFIG_NET_DSA_TAG_BRCM_FA=y
+ CONFIG_NF_CONNTRACK=y
+ CONFIG_NF_TABLES=y
+ CONFIG_NF_FLOW_TABLE=y
+ CONFIG_NF_FLOW_TABLE_INET=y
+ CONFIG_NFT_FLOW_OFFLOAD=y
+ CONFIG_NFT_NAT=y
+ CONFIG_NF_NAT=y
+ CONFIG_DEBUG_FS=y
+
+Modules are supported too. Use the board's existing ARM configuration and
+retain its PHY, switch, filesystem, and boot-device options.
+
+For a BCM53012 board, add the following node under the root address space
+(adjust the compatible to ``brcm,bcm53011-fa`` for BCM53011)::
+
+ / {
+ fa: flow-accelerator@18027c00 {
+ compatible = "brcm,bcm53012-fa";
+ reg = <0x18027c00 0xc0>;
+ };
+ };
+
+ &gmac2 {
+ brcm,flow-accelerator = <&fa>;
+ status = "okay";
+ };
+
+Both tested user ports must use switch port 8 and GMAC2. In the existing B53
+switch node, port 8 must have ``ethernet = <&gmac2>``. Also enable
+switch port 5 with ``ethernet = <&gmac0>``, ``phy-mode = "internal"`` and a
+1 Gbps full-duplex fixed link. Enable GMAC0 and bring its network interface
+up before testing either protocol; retain port 8 as the preferred conduit for
+user ports.
+Remove any other conflicting CPU assignments for this test. B53 selects ``brcm-fa``
+when it finds the enabled accelerator through this phandle. Do not set
+``dsa-tag-protocol``: B53 does not implement the runtime protocol-change
+operation. No existing board is enabled automatically by this series.
+
+The FA resource is outside the GMAC3 DMA register window. It is managed by
+the GMAC2 consumer; it does not need a separate platform driver or character
+device. Leave the accelerator disabled or omit the GMAC2 phandle to retain
+the existing board configuration.
+
+Three-host hardware test
+------------------------
+
+Use a dedicated test router with serial access and two external Linux hosts.
+Traffic generated on the router itself does not exercise the inline receive
+path. Keep all other switch user ports administratively down. Replace the
+interface names below with those on the board::
+
+ LAN host BCM5301X router WAN host
+ 192.0.2.2/24 -- lan1 192.0.2.1 198.51.100.1 wan -- 198.51.100.2/24
+ \ /
+ GMAC2 / FA
+
+Assign distinct router MACs before bringing up the interfaces::
+
+ ip link set lan1 down
+ ip link set wan down
+ ip link set lan1 address 02:00:00:00:01:01
+ ip link set wan address 02:00:00:00:02:01
+ ip addr add 192.0.2.1/24 dev lan1
+ ip addr add 198.51.100.1/24 dev wan
+ ip link set lan1 up
+ ip link set wan up
+ sysctl -w net.ipv4.ip_forward=1
+
+On the LAN host, assign 192.0.2.2/24 and a default route via 192.0.2.1.
+On the WAN host, assign 198.51.100.2/24 and a default route via 198.51.100.1.
+Do not put the router ports in a bridge or create VLAN uppers. Ensure that
+other firewall rules on this lab router permit the test packets.
+
+Mount debugfs if necessary and locate the FA status file::
+
+ mount -t debugfs none /sys/kernel/debug
+ find /sys/kernel/debug -name fa -type f
+ cat /sys/kernel/debug/<GMAC2-device-name>/fa
+
+The file contains ``failed``, global ``hits`` and ``misses``, error bits,
+and a row for every programmed directional entry. Each row identifies its
+slot, cookie, input/output interfaces, and raw hardware packet count. Save
+snapshots with timestamps before, during, and after every test.
+
+The adjacent ``fa_table`` file shows each software-owned tuple and the eight
+hardware words in DATA7 through DATA0 order. It uses the same lock and
+bounded indirect accesses as statistics collection. Save it when an entry
+stops counting to compare the hardware key against software ownership.
+
+Install a uniquely named test table; do not flush unrelated tables::
+
+ table inet fa_test {
+ flowtable ft {
+ hook ingress priority 0;
+ devices = { lan1, wan };
+ flags offload;
+ counter;
+ }
+ chain forward {
+ type filter hook forward priority 0; policy drop;
+ meta l4proto udp ct state established flow add @ft
+ ct state established,related counter accept
+ iifname "lan1" oifname "wan" udp dport 5001 counter accept
+ iifname "wan" oifname "lan1" udp dport 5001 ct status dnat counter accept
+ }
+ chain prerouting {
+ type nat hook prerouting priority dstnat; policy accept;
+ iifname "wan" udp dport 15001 dnat ip to 192.0.2.2:5001
+ }
+ chain postrouting {
+ type nat hook postrouting priority srcnat; policy accept;
+ iifname "lan1" oifname "wan" ip saddr 192.0.2.2 snat ip to 198.51.100.1:45000
+ }
+ }
+
+Save this as a ruleset and load it with ``nft -f <file>``. For the two control
+runs described below, edit only this test table. Delete it between phases
+with ``nft delete table inet fa_test``. Delete matching conntrack entries
+between phases, for example::
+
+ conntrack -D -p udp --orig-src 192.0.2.2 --orig-dst 198.51.100.2 --dport 5001
+
+For SNAT, run a UDP or TCP echo server on the WAN host at
+198.51.100.2:5001. Run a client on the LAN host, bind its source to
+192.0.2.2:40000, and connect to that server. Verify the server observes
+the translated peer 198.51.100.1:45000. Include a sequence number in each
+payload and check every returned payload for loss or corruption.
+
+Run three phases with a fresh connection each time:
+
+1. **Normal forwarding:** omit the ``flow add @ft`` rule. Forward-chain
+ counters should grow throughout the test. FA entry counts and hit counts
+ should not grow for this traffic.
+2. **Software flowtable:** restore ``flow add @ft`` and omit ``flags offload``.
+ Expect OFFLOAD and no hardware entries/hit growth for this flow. Reduced
+ forward-chain counters in this phase demonstrate why those counters alone
+ cannot prove hardware offload.
+3. **Hardware flowtable:** restore ``flags offload``. Wait a few seconds for
+ asynchronous rule installation, then compare multiple debugfs snapshots.
+
+The hardware phase passes only when all of the following hold:
+
+* The WAN server sees the peer as 198.51.100.1:45000, proving address and
+ source-port translation. The client receives the original payloads.
+* ``conntrack -L -p udp -o extended`` identifies the expected original and
+ reply tuples and reports HW_OFFLOAD. OFFLOAD alone is insufficient.
+* There are two hardware entry rows for the connection and **both** packet
+ counters advance while the echo stream runs. The netfilter core can mark
+ HW_OFFLOAD when only one direction is accepted, so check each direction.
+* The global FA hit count advances with the transfer, ``failed`` remains 0,
+ and hardware error bits do not increase.
+* Captures on the external hosts show the expected rewritten IPv4/TCP or
+ IPv4/UDP tuples, valid checksums, unchanged payloads, and exactly one TTL
+ decrement.
+* Router CPU load and GMAC DMA packet growth are substantially below the
+ software phase at the same offered packet rate. A router-side capture
+ should see initial misses but few steady-state packets from this stream.
+ Compare external captures; packet-capture placement and GRO can affect
+ counts on the router.
+
+Use ``tcpdump -ni <host-interface> -vv -w <capture.pcap> udp`` on both external
+hosts. If host checksum offload obscures validation, temporarily disable the
+relevant offloads on the test hosts and restore them afterward. Capture from
+the receiving host or a wire tap when checking checksums.
+
+For DNAT, stop the WAN server and run the echo server on the LAN host at
+192.0.2.2:5001. Run the client on the WAN host, bind it to 198.51.100.2:40001,
+and connect it to 198.51.100.1:15001. Verify destination IP/port translation
+on the LAN capture and the source IP/port restoration on the WAN reply.
+Repeat all three phases and require hit growth in both directions.
+
+Stress, teardown, and negative tests
+------------------------------------
+
+* Change SNAT to a port range for parallel sessions. Exercise multiple
+ clients and next hops, hot buckets, more than 128 next-hop MACs, and over
+ 512 bidirectional sessions. Failures to allocate must fall back to software
+ without modifying a different connection's entry. Four-way collisions can
+ exhaust a bucket before the whole table is full.
+* Keep a densely occupied table active while repeatedly reading statistics.
+ Compare the software tuples with the raw entries in ``fa_table`` before
+ and after the test. Every valid hardware key must match its owning slot,
+ cookies must stay stable and every active direction's counter must grow.
+ Intact echoes and HW_OFFLOAD labels alone can conceal damaged entries
+ whose traffic has fallen back to software.
+* Stop the generator and wait longer than the configured UDP conntrack
+ timeout. Hardware rows must disappear. Run sustained traffic longer than
+ that timeout and verify that counter-based last-used refresh keeps the
+ connection alive. Check packet counter wrap handling during a long soak.
+* Delete the test flowtable during traffic. Entries must disappear and
+ forwarding must continue through the remaining firewall/NAT path. Delete
+ only the flow-add rule first if retaining the table is useful for this test.
+* Disable ``hw-tc-offload`` on the conduit with ethtool. Entries must be
+ flushed and new hardware insertions rejected. Re-enable it and use fresh
+ connections.
+* Test port down/up, a third port up, router MAC changes, MTU changes, a bridge
+ or VLAN upper being added, conduit down/up, module unload/reload, and
+ suspend/resume if supported. Require removal of stale entries and clean
+ resource reuse. Start fresh connections after each topology change.
+* Try IPv6, ICMP, GRE, PPPoE, VLAN and QinQ devices, double NAT, and
+ hairpin NAT. They must not create FA entries. Exercise tagged packets whose
+ inner tuple equals a live untagged offload; they must not hit that entry.
+* Send IPv4 fragments, IP options, bad IP checksums, runt/truncated headers,
+ and switch exception traffic. Require software handling or the normal
+ stack/switch drop behavior, never a stale or unrelated hardware hit.
+* Verify UDP zero-checksum packets, nonzero checksums, packets whose updated
+ checksum folds to zero, minimum-size frames, and maximum MTU packets.
+* Repeat SNAT, DNAT, table pressure, software fallback, aging and recovery
+ with TCP. Require the TCP bit in every owned raw table key and advancing
+ counters in both directions. Compare classic, software and hardware modes.
+* While hardware data counters advance, capture SYN/FIN/RST on the GMAC0
+ auxiliary conduit. Require embedded Broadcom tags and delivery to Linux.
+ Close with FIN and RST, reuse the same tuple, and change its NAT
+ mapping; require immediate removal of the old hardware directions, correct
+ new peers and intact framed payloads. Repeat with options and fragmented
+ control packets. An HW_OFFLOAD label alone does not qualify this test.
+ Bring GMAC0 down while TCP data flows. Require removal of its hardware
+ rows, intact software forwarding, and rejection of new TCP hardware
+ entries. Bring GMAC0 up and require fresh connections to offload again.
+* Run TCP and UDP with identical IP addresses and transport ports. Require
+ independent hashes, hardware protocol bits and correct payload delivery;
+ closing TCP must leave the UDP directions installed and advancing.
+* With a development fault-injection mechanism, force an indirect-port busy
+ timeout. Require bounded completion, ``failed 1``, no further insertions,
+ no reuse of uncertain table resources, and working software forwarding.
+ Reprobe before repeating. The driver provides no writable debugfs register
+ interface; do not poke live registers from userspace.
+
+Boundary and invalidation qualification
+---------------------------------------
+
+Run these tests with UDP and TCP, SNAT and DNAT, and classic, software
+flowtable and hardware forwarding. Install both hardware directions and
+verify their raw keys before injecting packets of that same live tuple:
+
+* TTL 0 and 1 must produce the software baseline's ICMP time-exceeded
+ response without forwarding a wrapped TTL. Ordinary traffic must decrement
+ TTL exactly once.
+* An IP length of 1500 must pass. At 1501, DF must produce ICMP fragmentation
+ needed with MTU 1500; without DF, Linux must fragment into packets no
+ larger than 1500. Check transport checksums after reassembly.
+* Inject the tuple on the opposite physical port, addressed both to the
+ original input MAC and the receiving port's MAC. Require the same ingress
+ firewall policy as the software baseline. Also test unrelated unicast,
+ broadcast and multicast destination MACs. These must not enter IPv4
+ forwarding merely because a five-tuple matches.
+
+While a hardware connection is live, replace its neighbour MAC and require
+all subsequently forwarded frames to use the new MAC. Set the neighbour to
+``NUD_FAILED`` while preventing ARP replies and require forwarding to stop.
+Restore reachability and require recovery. Force a valid neighbour to STALE
+while hardware counters advance and require a Linux ARP reachability probe.
+
+Install a blackhole host route and then a source-policy blackhole rule;
+require forwarding to stop after each netlink acknowledgment. Remove each
+change and check payload delivery and fresh hardware admission. Add an MTU
+1200 host route both during an existing connection and before a fresh one.
+Require zero hardware entries, intact small software-forwarded packets and
+ICMP fragmentation-needed with MTU 1200 for a 1280-byte DF probe. Remove
+that route and verify recovery. Keep the flowtable installed throughout;
+recreating it would conceal missing invalidation.
+
+Fault injection qualification
+-----------------------------
+
+A development kernel can enable ``CONFIG_FAULT_INJECTION``,
+``CONFIG_FAULT_INJECTION_DEBUG_FS`` and
+``CONFIG_BCM_NS_FA_FAULT_INJECTION``. The per-device ``fail_table`` directory
+uses the standard fault-injection attributes. The default probability is
+zero. Injection returns a timeout after an indirect transfer has completed,
+which deliberately makes completion uncertain to its caller. This tests
+software handling of uncertain reads and writes, including a write that may
+already have published an entry; it does not emulate a physical bus hang.
+The ordinary hardware busy/address waits remain bounded independently.
+
+For one injected failure, with the device name adjusted as needed::
+
+ D=/sys/kernel/debug/bcma0:5
+ echo 1 > $D/fail_table/times
+ echo 1 > $D/fail_table/interval
+ echo 0 > $D/fail_table/space
+ echo 100 > $D/fail_table/probability
+
+The next table access fails. To restrict injection to a table write, set
+``fail_table_write_only`` to ``Y`` before arming it, then insert a fresh
+connection or delete an installed one. Run read, insertion and deletion
+cases on separate clean probes. Verify ``failed 1`` and exactly one
+``fault_injections`` increment. Lookup must be disabled before the paused
+switch traffic resumes. Broadcom-tag relocation must remain active, and
+uncertain entries and MAC resources must not be reused until reprobe.
+
+Keep UDP and TCP SNAT/DNAT traffic active across each failure. Validate all
+eight wire directions for correct tuples, payloads, checksums and exactly
+one TTL decrement. Require a fresh connection to work in software and an
+ethtool feature toggle to leave the failed state latched. Stop the test,
+reprobe and require clean hardware admission again. Do not poke live
+registers from userspace; no writable register interface is provided.
+
+Automated validation
+--------------------
+
+Run the supplied KUnit configuration from a clean source tree::
+
+ ./tools/testing/kunit/kunit.py run --arch=x86_64 \
+ --kunitconfig=drivers/net/ethernet/broadcom/.kunitconfig \
+ --build_dir=/tmp/bcm-ns-fa-kunit '*fa*'
+
+The backend, switch and tagger provide KUnit suites named ``bcm-ns-fa``,
+``b53-fa-guards`` and ``brcm-fa-tag``. Enable ``CONFIG_BCM_NS_FA_KUNIT_TEST`` and
+``CONFIG_NET_DSA_TAG_BRCM_FA_KUNIT_TEST`` and ``CONFIG_B53_FA_KUNIT_TEST``
+with KUnit and their parent drivers.
+They cover cross-word field layout and endianness, NAT action rejection,
+miss-hash learning exclusions, opcode validation, source-port demultiplexing,
+consumption of the four-byte metadata tag, route-MTU admission and CFP UDF
+layout. The length-prefix test checks all 65536 possible IPv4 lengths.
+These tests do not simulate the FA.
+
+Build the bgmac, B53, and DSA objects with ``W=1`` for ARM, plus an x86
+COMPILE_TEST configuration; also check the FA-disabled configuration and
+built-in/module combinations. Run ``dt_binding_check`` for both new schemas,
+``checkpatch.pl``, and ``git diff --check``. The hardware matrix above remains
+mandatory even when all automated checks pass.
diff --git a/Documentation/networking/device_drivers/ethernet/index.rst b/Documentation/networking/device_drivers/ethernet/index.rst
index 403ae9c2b06a..007e5c5c892a 100644
--- a/Documentation/networking/device_drivers/ethernet/index.rst
+++ b/Documentation/networking/device_drivers/ethernet/index.rst
@@ -18,6 +18,7 @@ Contents:
amd/pds_vdpa
amd/pds_vfio_pci
aquantia/atlantic
+ broadcom/bcm-ns-fa
chelsio/cxgb
dlink/dl2k
davicom/dm9000
diff --git a/MAINTAINERS b/MAINTAINERS
index c05ac2258889..7fbc78f9dd41 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5672,6 +5672,7 @@ S: Maintained
F: Documentation/devicetree/bindings/net/brcm,amac.yaml
F: Documentation/devicetree/bindings/net/brcm,bcm4708-fa.yaml
F: Documentation/devicetree/bindings/net/brcm,bcm5301x-gmac.yaml
+F: Documentation/networking/device_drivers/ethernet/broadcom/bcm-ns-fa.rst
F: drivers/net/dsa/b53/b53_fa_test.c
F: drivers/net/ethernet/broadcom/.kunitconfig
F: drivers/net/ethernet/broadcom/bcm-ns-fa*
--
2.53.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload
2026-11-16 22:25 [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload Florian Fainelli
` (8 preceding siblings ...)
2026-11-16 22:25 ` [PATCH net-next 9/9] docs: net: document Northstar FA setup and NAT qualification Florian Fainelli
@ 2026-11-16 22:38 ` netdev-bot+sinfo
2026-11-16 22:45 ` Florian Fainelli
9 siblings, 1 reply; 12+ messages in thread
From: netdev-bot+sinfo @ 2026-11-16 22:38 UTC (permalink / raw)
To: Florian Fainelli
Cc: netdev, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter, devicetree, linux-kernel,
linux-doc, netfilter-devel, coreteam
Hi!
This is an automated message. This series looks like a fix, but its
commit messages seem to be missing some information:
- How the issue was discovered, e.g. hit in production, hit during
development, syzbot report, manual code inspection, LLM or static
analysis tool scan.
- Whether the issue was actually triggered, or is only theoretical
(e.g. found by LLM and not proven at runtime). If it was triggered
please include the symptoms, like the stack trace or kernel messages.
Please do not repost the series just to address the above. Instead,
reply to this email with the missing information, so that reviewers
can take it into account.
*IMPORTANT*: If you do repost (for other reasons) - please include
the relevant info in the cover letter or commit messages.
--
The evaluation is done by an LLM so it may be wrong, if you think
that is the case please reply and explain.
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload
2026-11-16 22:38 ` [PATCH net-next 0/9] net: bgmac: add Northstar IPv4 TCP/UDP NAT flowtable offload netdev-bot+sinfo
@ 2026-11-16 22:45 ` Florian Fainelli
0 siblings, 0 replies; 12+ messages in thread
From: Florian Fainelli @ 2026-11-16 22:45 UTC (permalink / raw)
To: netdev-bot+sinfo
Cc: netdev, Rafał Miłecki,
Broadcom internal kernel review list, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Simon Horman,
Jonathan Corbet, Jonas Gorski, Vladimir Oltean, Pablo Neira Ayuso,
Florian Westphal, Phil Sutter, devicetree, linux-kernel,
linux-doc, netfilter-devel, coreteam
On 11/16/26 14:38, netdev-bot+sinfo@kernel.org wrote:
> Hi!
>
> This is an automated message. This series looks like a fix, but its
> commit messages seem to be missing some information:
>
> - How the issue was discovered, e.g. hit in production, hit during
> development, syzbot report, manual code inspection, LLM or static
> analysis tool scan.
The commits with a Fixes: tag were developed in response to a test
failure developing the flow accelerator hardware on BCM5301X SoCs.
--
Florian
^ permalink raw reply [flat|nested] 12+ messages in thread