* [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations
@ 2026-11-13 20:41 Alexander Duyck
2026-11-13 20:41 ` [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle Alexander Duyck
` (5 more replies)
0 siblings, 6 replies; 13+ messages in thread
From: Alexander Duyck @ 2026-11-13 20:41 UTC (permalink / raw)
To: netdev; +Cc: andrew+netdev, davem, edumazet, horms, kernel-team, kuba, pabeni
Follow-up fixes for fbnic on top of the series that was just applied
to net, plus two fixes for FW mailbox allocations under memory pressure.
Patch 1 makes the Tx idle check wait for in-flight PTP timestamps. The
QM idle registers do not cover the TMI completion path, so they can
report idle while a timestamp completion is still outstanding.
Patches 2-4 fix the HW stats: patch 2 stops a failed register read, as
every read is during DPC recovery, from being folded into the totals;
patch 3 puts all of the stats under the stats lock, as the MAC stats
getters no longer run under RTNL; patch 4 captures the baselines before
the stats can be read, at probe and on resume, where they are now
rebased as MMIO access returns.
Patches 5 and 6 address heartbeat failures seen on production hosts
during host memory stalls: the heartbeat and ownership requests are
stamped after their message is allocated, and FW completions, which
carry the whole coredump, are allocated with kvzalloc and the dump size
is bounded.
Patches 1, 3 and 4 were found by code review, the PTP completion
behaviour confirmed against the RTL; patches 2 and 3 were confirmed by
hardware testing with forced AER/DPC recovery, and patches 5 and 6
address failures seen on production hosts. Tested on QEMU with the fbnic
device model, and on a Yosemite V4 host with forced AER/DPC recovery
while reading the stats.
---
Alexander Duyck (6):
fbnic: Wait for in-flight PTP timestamps before treating Tx as idle
fbnic: Do not accumulate HW stats from failed register reads
fbnic: Cover all of the HW stats with the stats lock
fbnic: Initialize and reset the HW stats before they can be read
fbnic: Stamp heartbeat and ownership requests after allocating them
fbnic: Allocate FW completions with kvzalloc
drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 3 +-
.../net/ethernet/meta/fbnic/fbnic_debugfs.c | 3 -
.../net/ethernet/meta/fbnic/fbnic_devlink.c | 5 ++
.../net/ethernet/meta/fbnic/fbnic_ethtool.c | 8 ++
drivers/net/ethernet/meta/fbnic/fbnic_fw.c | 19 +++--
drivers/net/ethernet/meta/fbnic/fbnic_fw.h | 5 ++
.../net/ethernet/meta/fbnic/fbnic_hw_stats.c | 32 ++++----
.../net/ethernet/meta/fbnic/fbnic_hw_stats.h | 1 +
drivers/net/ethernet/meta/fbnic/fbnic_mac.c | 8 ++
drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 16 ++--
drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 73 +++++++++++++++----
11 files changed, 131 insertions(+), 42 deletions(-)
--
^ permalink raw reply [flat|nested] 13+ messages in thread
* [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle
2026-11-13 20:41 [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations Alexander Duyck
@ 2026-11-13 20:41 ` Alexander Duyck
2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-13 20:41 ` [net PATCH 2/6] fbnic: Do not accumulate HW stats from failed register reads Alexander Duyck
` (4 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Alexander Duyck @ 2026-11-13 20:41 UTC (permalink / raw)
To: netdev; +Cc: andrew+netdev, davem, edumazet, horms, kernel-team, kuba, pabeni
From: Alexander Duyck <alexanderduyck@fb.com>
The fbnic_wait_all_queues_idle function treats the Tx path as idle once
the QM idle registers report every queue idle. Those registers do not
cover the TMI completion path, so they can report idle while a PTP
timestamp request is still outstanding in the MAC or queued in the TMI
completion FIFO.
The check can then pass without a flush, and the completion arrives
after the rings have been flushed, where the fbnic_clean_tsq function
either reports "Tx timestamp without matching packet" or, if the
interface has come back up and a new timestamped frame is already
waiting, matches the stale completion to that frame and hands an
SO_TIMESTAMPING user the wrong Tx timestamp. After a forced flush it can
also turn FBNIC_TMI_DROP_CTRL_CMPL_DIS back off before the queued
completions have drained, as the completion FIFO is only drained while
that bit is set.
Split the Tx wait out into the fbnic_wait_tx_idle function and have its
idle check require the TCE PTP in-flight count to read zero in addition
to the QM idle registers. The count tracks each timestamp request from
when its frame enters the TCE until its completion leaves the TMI
completion FIFO, so the Tx path is only treated as idle once none are
outstanding, and a forced flush keeps the completion disable set until
they have drained. The Rx wait still checks only the QM idle registers.
The count is printed with the other Tx idle registers on failure.
Waiting on the in-flight count means the Tx idle check can now keep
reading the full set of QM idle registers for the whole timeout, and the
poll timeout only counts the delay between polls, not the time spent in
the register reads. Raise the delay between polls of the Tx and Rx waits
from 2us to 20us so the PCIe read latency stays a small part of each
interval and the 500ms timeout stays close to wall time.
Define the register inside the TCE CSR section and move
FBNIC_CSR_END_TCE to the last register of the TCE block (0x040B4,
TCE_SPARE2 in the CSR map), so that the ethtool register dump covers it.
The section previously ended at 0x04050, short of the TXB, PTP and TCAM
status registers. Per the CSR map, every register in the added range is
read-only or read/write, so reading them for the dump has no side
effects.
A device that has dropped off the bus, or whose CSR access the driver
has disabled after a failed read, reads all ones. The QM idle bits treat
that as idle, and so does the in-flight check: the count is bounded by
the timestamp requests the Tx path can hold and can never reach all
ones, so teardown of a lost device, or during DPC recovery, does not
wait out the idle timeouts.
The issue was found by code review and confirmed against the RTL.
Fixes: ad3d9f8bc66c ("eth: fbnic: add TX packets timestamping support")
Fixes: 7d4568edce74 ("fbnic: Disable PTP completions during the Tx force-flush")
Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 3 +
drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 73 +++++++++++++++++++++-----
2 files changed, 62 insertions(+), 14 deletions(-)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
index c371ac9f78c1..dc234f4d24f6 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
@@ -422,7 +422,8 @@ enum {
#define FBNIC_TCE_TXB_NTWRK_DWRR_CTRL_EXT \
0x0404E /* 0x10138 */
#define FBNIC_TCE_TXB_BMC_DWRR_CTRL_EXT 0x0404F /* 0x1013c */
-#define FBNIC_CSR_END_TCE 0x04050 /* CSR section delimiter */
+#define FBNIC_TCE_PTP_INFLIGHT 0x040A7 /* 0x1029c */
+#define FBNIC_CSR_END_TCE 0x040B4 /* CSR section delimiter */
/* TCE RAM registers */
#define FBNIC_CSR_START_TCE_RAM 0x04200 /* CSR section delimiter */
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
index 07d4b2e92362..c75c5683b724 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
@@ -2398,38 +2398,85 @@ static void fbnic_idle_dump(struct fbnic_dev *fbd,
fbnic_rd32(fbd, regs[i].reg_base + j));
}
-int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail)
+/* The poll timeout only accounts for the delay between polls, not for
+ * the time spent in the poll itself, and each poll is a series of
+ * non-posted PCIe register reads that take on the order of a
+ * microsecond each. Space the polls out so that the reads stay a small
+ * part of each interval and the timeout stays close to wall time.
+ */
+#define FBNIC_IDLE_POLL_US 20
+#define FBNIC_IDLE_TIMEOUT_US 500000
+
+static bool fbnic_tx_idle(struct fbnic_dev *fbd,
+ const struct fbnic_idle_regs *regs,
+ unsigned int nregs)
+{
+ u32 inflight;
+
+ if (!fbnic_all_idle(fbd, regs, nregs))
+ return false;
+
+ /* The QM idle bits do not cover the TMI completion path, so a
+ * timestamp request can still be waiting on its completion. The
+ * count is bounded by the timestamp requests the Tx path can hold,
+ * so all ones is a failed read, as for the idle bits, not a count.
+ */
+ inflight = fbnic_rd32(fbd, FBNIC_TCE_PTP_INFLIGHT);
+
+ return !inflight || !~inflight;
+}
+
+static int fbnic_wait_tx_idle(struct fbnic_dev *fbd)
{
static const struct fbnic_idle_regs tx[] = {
{ FBNIC_QM_TWQ_IDLE(0), FBNIC_QM_TWQ_IDLE_CNT, },
{ FBNIC_QM_TQS_IDLE(0), FBNIC_QM_TQS_IDLE_CNT, },
{ FBNIC_QM_TDE_IDLE(0), FBNIC_QM_TDE_IDLE_CNT, },
{ FBNIC_QM_TCQ_IDLE(0), FBNIC_QM_TCQ_IDLE_CNT, },
- }, rx[] = {
- { FBNIC_QM_HPQ_IDLE(0), FBNIC_QM_HPQ_IDLE_CNT, },
- { FBNIC_QM_PPQ_IDLE(0), FBNIC_QM_PPQ_IDLE_CNT, },
- { FBNIC_QM_RCQ_IDLE(0), FBNIC_QM_RCQ_IDLE_CNT, },
};
bool idle;
int err;
- err = read_poll_timeout_atomic(fbnic_all_idle, idle, idle, 2, 500000,
- false, fbd, tx, ARRAY_SIZE(tx));
+ err = read_poll_timeout_atomic(fbnic_tx_idle, idle, idle,
+ FBNIC_IDLE_POLL_US,
+ FBNIC_IDLE_TIMEOUT_US, false,
+ fbd, tx, ARRAY_SIZE(tx));
if (err == -ETIMEDOUT) {
fbnic_tx_flush(fbd);
- err = read_poll_timeout_atomic(fbnic_all_idle, idle, idle,
- 2, 500000, false,
+ err = read_poll_timeout_atomic(fbnic_tx_idle, idle, idle,
+ FBNIC_IDLE_POLL_US,
+ FBNIC_IDLE_TIMEOUT_US, false,
fbd, tx, ARRAY_SIZE(tx));
fbnic_tx_flush_off(fbd);
}
if (err) {
fbnic_idle_dump(fbd, tx, ARRAY_SIZE(tx), "Tx", err);
- if (may_fail)
- return err;
+ netdev_err(fbd->netdev, "0x%04x: %08x\n",
+ FBNIC_TCE_PTP_INFLIGHT,
+ fbnic_rd32(fbd, FBNIC_TCE_PTP_INFLIGHT));
}
- err = read_poll_timeout_atomic(fbnic_all_idle, idle, idle, 2, 500000,
- false, fbd, rx, ARRAY_SIZE(rx));
+ return err;
+}
+
+int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail)
+{
+ static const struct fbnic_idle_regs rx[] = {
+ { FBNIC_QM_HPQ_IDLE(0), FBNIC_QM_HPQ_IDLE_CNT, },
+ { FBNIC_QM_PPQ_IDLE(0), FBNIC_QM_PPQ_IDLE_CNT, },
+ { FBNIC_QM_RCQ_IDLE(0), FBNIC_QM_RCQ_IDLE_CNT, },
+ };
+ bool idle;
+ int err;
+
+ err = fbnic_wait_tx_idle(fbd);
+ if (err && may_fail)
+ return err;
+
+ err = read_poll_timeout_atomic(fbnic_all_idle, idle, idle,
+ FBNIC_IDLE_POLL_US,
+ FBNIC_IDLE_TIMEOUT_US, false,
+ fbd, rx, ARRAY_SIZE(rx));
if (err)
fbnic_idle_dump(fbd, rx, ARRAY_SIZE(rx), "Rx", err);
return err;
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [net PATCH 2/6] fbnic: Do not accumulate HW stats from failed register reads
2026-11-13 20:41 [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations Alexander Duyck
2026-11-13 20:41 ` [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle Alexander Duyck
@ 2026-11-13 20:41 ` Alexander Duyck
2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-13 20:41 ` [net PATCH 3/6] fbnic: Cover all of the HW stats with the stats lock Alexander Duyck
` (3 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Alexander Duyck @ 2026-11-13 20:41 UTC (permalink / raw)
To: netdev; +Cc: andrew+netdev, davem, edumazet, horms, kernel-team, kuba, pabeni
From: Alexander Duyck <alexanderduyck@fb.com>
The HW stats are kept as a running total that the driver advances by the
difference between the current counter value and the value it last read,
and the fbnic_reset_hw_stats function only rebases that last value after
a reset so the totals carry over.
When a register read fails, as it does for every read once a DPC
containment event takes the link down, the fbnic_rd32 function returns
all ones and disables CSR access, so every later read also returns all
ones until the device is recovered. A stats read in that window adds the
difference between all ones and the last value to the total, which
subtracts the counter's whole value since the last rebase. Hardware
testing with forced AER recovery while reading the stats showed the
64-bit PCIe counters wrapping below zero on the first recovery and
dropping again on every one after it.
Skip the update when CSR access has been disabled by the time the read
returns, which covers the read that failed as well as any after it. The
last value is left alone, so the next good read after recovery, once the
fbnic_reset_hw_stats function has rebased it, picks up where the total
left off.
Fixes: 25ba596d137d ("eth: fbnic: add PCIe hardware statistics")
Fixes: 4eb7f20bcf06 ("eth: fbnic: Add support to fetch group stats")
Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c | 8 ++++++++
drivers/net/ethernet/meta/fbnic/fbnic_mac.c | 8 ++++++++
2 files changed, 16 insertions(+)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
index 8b9b2076beec..8084a63dd807 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
@@ -22,6 +22,10 @@ static void fbnic_hw_stat_rd32(struct fbnic_dev *fbd, u32 reg,
u32 new_reg_value;
new_reg_value = rd32(fbd, reg);
+ /* A failed read disables CSR access and returns all ones */
+ if (!fbnic_present(fbd))
+ return;
+
stat->value += new_reg_value - stat->u.old_reg_value_32;
stat->u.old_reg_value_32 = new_reg_value;
}
@@ -68,6 +72,10 @@ static void fbnic_hw_stat_rd64(struct fbnic_dev *fbd, u32 reg, s32 offset,
u64 new_reg_value;
new_reg_value = fbnic_stat_rd64(fbd, reg, offset);
+ /* A failed read disables CSR access and returns all ones */
+ if (!fbnic_present(fbd))
+ return;
+
stat->value += new_reg_value - stat->u.old_reg_value_64;
stat->u.old_reg_value_64 = new_reg_value;
}
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
index 16271c717a5a..2aacbf0a6af4 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
@@ -405,6 +405,10 @@ static void __fbnic_mac_stat_rd64(struct fbnic_dev *fbd, bool reset, u32 reg,
u64 new_reg_value;
new_reg_value = fbnic_stat_rd64(fbd, reg, 1);
+ /* A failed read disables CSR access and returns all ones */
+ if (!fbnic_present(fbd))
+ return;
+
if (!reset)
stat->value += new_reg_value - stat->u.old_reg_value_64;
stat->u.old_reg_value_64 = new_reg_value;
@@ -417,6 +421,10 @@ static void fbnic_mac_stat_rd32(struct fbnic_dev *fbd, bool reset, u32 reg,
u32 new_reg_value;
new_reg_value = rd32(fbd, reg);
+ /* A failed read disables CSR access and returns all ones */
+ if (!fbnic_present(fbd))
+ return;
+
if (!reset)
stat->value += new_reg_value - stat->u.old_reg_value_32;
stat->u.old_reg_value_32 = new_reg_value;
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [net PATCH 3/6] fbnic: Cover all of the HW stats with the stats lock
2026-11-13 20:41 [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations Alexander Duyck
2026-11-13 20:41 ` [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle Alexander Duyck
2026-11-13 20:41 ` [net PATCH 2/6] fbnic: Do not accumulate HW stats from failed register reads Alexander Duyck
@ 2026-11-13 20:41 ` Alexander Duyck
2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-13 20:41 ` [net PATCH 4/6] fbnic: Initialize and reset the HW stats before they can be read Alexander Duyck
` (2 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Alexander Duyck @ 2026-11-13 20:41 UTC (permalink / raw)
To: netdev; +Cc: andrew+netdev, davem, edumazet, horms, kernel-team, kuba, pabeni
From: Alexander Duyck <alexanderduyck@fb.com>
The hw_stats lock covers the PHY, TMI, TTI, RPC, RXB, queue and PCIe
stats, but the MAC stats are left to RTNL: the fbnic_reset_hw_stats
function resets them outside the lock and asserts RTNL, on the grounds
that the ethtool API is their only other user and holds RTNL. That is no
longer the case. ethtool only takes RTNL on the Netlink path for the
operations a driver lists in op_needs_rtnl, and fbnic lists only the
pause getter among the stats ones, so the eth-mac, eth-ctrl and rmon
getters update the MAC stats with only the netdev instance lock held,
racing each other and the reset.
Take the stats lock around the MAC stats updates in those getters and in
the reset, and drop the RTNL assertion. Take it in the pause getter as
well: it still runs under RTNL, but that no longer serializes it against
the reset once the reset does not rely on RTNL. The PCIe stats debugfs
read never needed RTNL, as the fbnic_get_hw_stats function updates those
stats under the stats lock, so drop it there too. The values are printed
from their 64-bit fields without the lock, as the counters keep changing
during the read anyway.
The issue was found by code review.
Fixes: 45079e00133e ("net: ethtool: optionally skip rtnl_lock on Netlink path for GET ops")
Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c | 3 ---
drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c | 8 ++++++++
drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c | 13 +------------
3 files changed, 9 insertions(+), 15 deletions(-)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c b/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
index f9e879be254d..65bea571c0cc 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
@@ -3,7 +3,6 @@
#include <linux/debugfs.h>
#include <linux/pci.h>
-#include <linux/rtnetlink.h>
#include <linux/seq_file.h>
#include "fbnic.h"
@@ -607,7 +606,6 @@ static int fbnic_dbg_pcie_stats_show(struct seq_file *s, void *v)
{
struct fbnic_dev *fbd = s->private;
- rtnl_lock();
fbnic_get_hw_stats(fbd);
seq_printf(s, "ob_rd_tlp: %llu\n", fbd->hw_stats.pcie.ob_rd_tlp.value);
@@ -626,7 +624,6 @@ static int fbnic_dbg_pcie_stats_show(struct seq_file *s, void *v)
fbd->hw_stats.pcie.ob_rd_no_cpl_cred.value);
seq_printf(s, "ob_rd_no_np_cred: %llu\n",
fbd->hw_stats.pcie.ob_rd_no_np_cred.value);
- rtnl_unlock();
return 0;
}
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
index e774740fcb5d..b300017b53d4 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
@@ -1867,12 +1867,14 @@ fbnic_get_pause_stats(struct net_device *netdev,
mac_stats = &fbd->hw_stats.mac;
+ spin_lock(&fbd->hw_stats.lock);
fbd->mac->get_pause_stats(fbd, false, &mac_stats->pause);
pause_stats->tx_pause_frames = mac_stats->pause.tx_pause_frames.value;
pause_stats->rx_pause_frames = mac_stats->pause.rx_pause_frames.value;
tx_ps_events = mac_stats->pause.tx_pause_storm_events.value;
pause_stats->tx_pause_storm_events = tx_ps_events;
+ spin_unlock(&fbd->hw_stats.lock);
}
static void
@@ -1928,6 +1930,7 @@ fbnic_get_eth_mac_stats(struct net_device *netdev,
mac_stats = &fbd->hw_stats.mac;
mac = fbd->mac;
+ spin_lock(&fbd->hw_stats.lock);
mac->get_eth_mac_stats(fbd, false, &mac_stats->eth_mac);
fbnic_set_counter(ð_mac_stats->FramesTransmittedOK,
@@ -1956,6 +1959,7 @@ fbnic_get_eth_mac_stats(struct net_device *netdev,
&mac_stats->eth_mac.BroadcastFramesReceivedOK);
fbnic_set_counter(ð_mac_stats->FrameTooLongErrors,
&mac_stats->eth_mac.FrameTooLongErrors);
+ spin_unlock(&fbd->hw_stats.lock);
}
static void
@@ -1968,12 +1972,14 @@ fbnic_get_eth_ctrl_stats(struct net_device *netdev,
mac_stats = &fbd->hw_stats.mac;
+ spin_lock(&fbd->hw_stats.lock);
fbd->mac->get_eth_ctrl_stats(fbd, false, &mac_stats->eth_ctrl);
eth_ctrl_stats->MACControlFramesReceived =
mac_stats->eth_ctrl.MACControlFramesReceived.value;
eth_ctrl_stats->MACControlFramesTransmitted =
mac_stats->eth_ctrl.MACControlFramesTransmitted.value;
+ spin_unlock(&fbd->hw_stats.lock);
}
static const struct ethtool_rmon_hist_range fbnic_rmon_ranges[] = {
@@ -2003,6 +2009,7 @@ fbnic_get_rmon_stats(struct net_device *netdev,
mac_stats = &fbd->hw_stats.mac;
+ spin_lock(&fbd->hw_stats.lock);
fbd->mac->get_rmon_stats(fbd, false, &mac_stats->rmon);
rmon_stats->undersize_pkts =
@@ -2018,6 +2025,7 @@ fbnic_get_rmon_stats(struct net_device *netdev,
rmon_stats->hist[i] = mac_stats->rmon.hist[i].value;
rmon_stats->hist_tx[i] = mac_stats->rmon.hist_tx[i].value;
}
+ spin_unlock(&fbd->hw_stats.lock);
*ranges = fbnic_rmon_ranges;
}
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
index 8084a63dd807..307d5e737848 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
@@ -1,8 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) Meta Platforms, Inc. and affiliates. */
-#include <linux/rtnetlink.h>
-
#include "fbnic.h"
static void fbnic_hw_stat_rst32(struct fbnic_dev *fbd, u32 reg,
@@ -559,17 +557,8 @@ void fbnic_reset_hw_stats(struct fbnic_dev *fbd)
fbnic_reset_rxb_stats(fbd, &fbd->hw_stats.rxb);
fbnic_reset_hw_rxq_stats(fbd, fbd->hw_stats.hw_q);
fbnic_reset_pcie_stats_asic(fbd, &fbd->hw_stats.pcie);
- spin_unlock(&fbd->hw_stats.lock);
-
- /* Once registered, the only other access to MAC stats is via the
- * ethtool API which is protected by the rtnl_lock. The call to
- * fbnic_reset_hw_stats() during PCI recovery is also protected
- * by the rtnl_lock hence, we don't need the spinlock to access
- * the MAC stats.
- */
- if (fbd->netdev)
- ASSERT_RTNL();
fbnic_reset_hw_mac_stats(fbd, &fbd->hw_stats.mac);
+ spin_unlock(&fbd->hw_stats.lock);
}
void fbnic_init_hw_stats(struct fbnic_dev *fbd)
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [net PATCH 4/6] fbnic: Initialize and reset the HW stats before they can be read
2026-11-13 20:41 [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations Alexander Duyck
` (2 preceding siblings ...)
2026-11-13 20:41 ` [net PATCH 3/6] fbnic: Cover all of the HW stats with the stats lock Alexander Duyck
@ 2026-11-13 20:41 ` Alexander Duyck
2026-11-13 20:42 ` [net PATCH 5/6] fbnic: Stamp heartbeat and ownership requests after allocating them Alexander Duyck
2026-11-13 20:42 ` [net PATCH 6/6] fbnic: Allocate FW completions with kvzalloc Alexander Duyck
5 siblings, 0 replies; 13+ messages in thread
From: Alexander Duyck @ 2026-11-13 20:41 UTC (permalink / raw)
To: netdev; +Cc: andrew+netdev, davem, edumazet, horms, kernel-team, kuba, pabeni
From: Alexander Duyck <alexanderduyck@fb.com>
The HW stats are kept as running totals that advance by the difference
between each counter and the value last read, so the last values have
to be captured before the first read, and rebased whenever the hardware
counters are reset. Three paths get the ordering wrong.
The fbnic_probe function creates the debugfs directory, including the
pcie_stats file, before it calls the fbnic_init_hw_stats function. A
read in that window takes the stats lock before it has been initialized,
and accumulates deltas against baselines that have not been captured.
On resume, the __fbnic_pm_attach function rebases the stats only after
it has returned early for a device in init failure mode, though such a
device still exposes the PCIe stats through debugfs. And during PCI
error recovery it runs from the resume callback, after the
__fbnic_pm_resume function has already restored MMIO access from the
slot_reset callback, so a debugfs read in between takes its delta
against the values from before the reset and the totals jump by close
to the full width of each counter.
Initialize the stats before the debugfs directory and the devlink
instance are registered. Rebase them in the __fbnic_pm_resume function,
in the same stats lock section that restores MMIO access, so no read can
see the reset counters first, ahead of its init failure check.
The issue was found by code review, and the recovery case confirmed by
hardware testing with forced AER/DPC recovery.
Fixes: 25ba596d137d ("eth: fbnic: add PCIe hardware statistics")
Fixes: 9f61eb2d185b ("eth: fbnic: add locking support for hw stats")
Fixes: 05ec5504ff69 ("fbnic: Fix service task races with ndo_stop and PM suspend")
Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c | 11 +++++++++--
drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.h | 1 +
drivers/net/ethernet/meta/fbnic/fbnic_pci.c | 16 ++++++++++------
3 files changed, 20 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
index 307d5e737848..966c2d0fb27a 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.c
@@ -547,9 +547,10 @@ static void fbnic_reset_hw_mac_stats(struct fbnic_dev *fbd,
mac->get_rmon_stats(fbd, true, &mac_stats->rmon);
}
-void fbnic_reset_hw_stats(struct fbnic_dev *fbd)
+void __fbnic_reset_hw_stats(struct fbnic_dev *fbd)
{
- spin_lock(&fbd->hw_stats.lock);
+ lockdep_assert_held(&fbd->hw_stats.lock);
+
fbnic_reset_phy_stats(fbd, &fbd->hw_stats.phy);
fbnic_reset_tmi_stats(fbd, &fbd->hw_stats.tmi);
fbnic_reset_tti_stats(fbd, &fbd->hw_stats.tti);
@@ -558,6 +559,12 @@ void fbnic_reset_hw_stats(struct fbnic_dev *fbd)
fbnic_reset_hw_rxq_stats(fbd, fbd->hw_stats.hw_q);
fbnic_reset_pcie_stats_asic(fbd, &fbd->hw_stats.pcie);
fbnic_reset_hw_mac_stats(fbd, &fbd->hw_stats.mac);
+}
+
+void fbnic_reset_hw_stats(struct fbnic_dev *fbd)
+{
+ spin_lock(&fbd->hw_stats.lock);
+ __fbnic_reset_hw_stats(fbd);
spin_unlock(&fbd->hw_stats.lock);
}
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.h b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.h
index caea4be46762..5eb1e8b29ef9 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_hw_stats.h
@@ -154,6 +154,7 @@ struct fbnic_hw_stats {
u64 fbnic_stat_rd64(struct fbnic_dev *fbd, u32 reg, u32 offset);
+void __fbnic_reset_hw_stats(struct fbnic_dev *fbd);
void fbnic_reset_hw_stats(struct fbnic_dev *fbd);
void fbnic_init_hw_stats(struct fbnic_dev *fbd);
void fbnic_get_hw_q_stats(struct fbnic_dev *fbd,
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
index 09ae797b1f0d..cb0191b65204 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
@@ -350,13 +350,13 @@ static int fbnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
*/
fbnic_fw_log_enable(fbd, true);
+ /* Capture snapshot of hardware stats so netdev can calculate delta */
+ fbnic_init_hw_stats(fbd);
+
fbnic_devlink_register(fbd);
fbnic_devlink_otp_check(fbd, "error detected during probe");
fbnic_dbg_fbd_init(fbd);
- /* Capture snapshot of hardware stats so netdev can calculate delta */
- fbnic_init_hw_stats(fbd);
-
fbnic_hwmon_register(fbd);
if (!fbd->dsn) {
@@ -498,10 +498,16 @@ static int __fbnic_pm_resume(struct device *dev)
struct fbnic_net *fbn;
int err;
- /* Restore MMIO access */
+ /* Restore MMIO access and rebase the HW stats in one step, so that
+ * no stats read can take its delta from counters the reset cleared
+ * against the values read before it.
+ */
iomap_table = pcim_iomap_table(to_pci_dev(dev));
+ spin_lock(&fbd->hw_stats.lock);
fbd->uc_addr0 = iomap_table[0];
fbd->uc_addr4 = iomap_table[4];
+ __fbnic_reset_hw_stats(fbd);
+ spin_unlock(&fbd->hw_stats.lock);
/* Rerequest the IRQs */
err = fbnic_alloc_irqs(fbd);
@@ -582,8 +588,6 @@ static void __fbnic_pm_attach(struct device *dev)
rtnl_lock();
netdev_lock(netdev);
- fbnic_reset_hw_stats(fbd);
-
if (netif_running(netdev))
fbnic_up(fbn);
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [net PATCH 5/6] fbnic: Stamp heartbeat and ownership requests after allocating them
2026-11-13 20:41 [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations Alexander Duyck
` (3 preceding siblings ...)
2026-11-13 20:41 ` [net PATCH 4/6] fbnic: Initialize and reset the HW stats before they can be read Alexander Duyck
@ 2026-11-13 20:42 ` Alexander Duyck
2026-11-13 20:42 ` [net PATCH 6/6] fbnic: Allocate FW completions with kvzalloc Alexander Duyck
5 siblings, 0 replies; 13+ messages in thread
From: Alexander Duyck @ 2026-11-13 20:42 UTC (permalink / raw)
To: netdev; +Cc: andrew+netdev, davem, edumazet, horms, kernel-team, kuba, pabeni
From: Alexander Duyck <alexanderduyck@fb.com>
The fbnic_fw_xmit_heartbeat_message function records the request time
on entry, before it allocates the message. The service task sends the
heartbeat with RTNL held, and under memory pressure that GFP_KERNEL
page allocation can sit in direct reclaim for minutes. The request then
goes out stamped at the start of the stall, so the FW_HEARTBEAT_PERIOD
gate in the fbnic_fw_check_heartbeat function has already expired on
the next service task pass. That pass expects the response to have
been processed already, and if the mailbox interrupt thread has not got
to it yet on a host that is still struggling, the heartbeat is reported
as missed and handled as a firmware crash although the firmware
answered.
Take the request time after the allocation, just before the message is
posted. A response still cannot be recorded as older than its request,
and the firmware gets the full period to answer however long the
allocation took.
The fbnic_fw_xmit_ownership_msg function restarts the heartbeat the
same way, stamping the request and the last response before it
allocates the message. It runs from the service task with RTNL held
when recovering from a firmware crash, so move its timestamp after the
allocation as well.
This was seen on production hosts during host memory stalls, where a
page allocation stall of 464 seconds in this allocation was followed by
a "Firmware did not respond to heartbeat message" warning while the
firmware kept running normally.
Fixes: 20d2e88cc746 ("eth: fbnic: Add initial messaging to notify FW of our presence")
Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
drivers/net/ethernet/meta/fbnic/fbnic_fw.c | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
index e00d626f6438..3465f0139883 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
@@ -538,8 +538,8 @@ static bool fbnic_mbx_event(struct fbnic_dev *fbd)
*/
int fbnic_fw_xmit_ownership_msg(struct fbnic_dev *fbd, bool take_ownership)
{
- unsigned long req_time = jiffies;
struct fbnic_tlv_msg *msg;
+ unsigned long req_time;
int err = 0;
if (!fbnic_fw_present(fbd))
@@ -556,8 +556,10 @@ int fbnic_fw_xmit_ownership_msg(struct fbnic_dev *fbd, bool take_ownership)
}
/* Initialize heartbeat, set last response to 1 second in the past
- * so that we will trigger a timeout if the firmware doesn't respond
+ * so that we will trigger a timeout if the firmware doesn't respond.
+ * Take the time after the allocation, as for the heartbeat request.
*/
+ req_time = jiffies;
fbd->last_heartbeat_response = req_time - HZ;
fbd->last_heartbeat_request = req_time;
@@ -793,8 +795,8 @@ static int fbnic_fw_parse_heartbeat_resp(void *opaque,
static int fbnic_fw_xmit_heartbeat_message(struct fbnic_dev *fbd)
{
- unsigned long req_time = jiffies;
struct fbnic_tlv_msg *msg;
+ unsigned long req_time;
int err = 0;
if (!fbnic_fw_present(fbd))
@@ -804,6 +806,12 @@ static int fbnic_fw_xmit_heartbeat_message(struct fbnic_dev *fbd)
if (!msg)
return -ENOMEM;
+ /* Stamp the request after the allocation, which can stall in
+ * reclaim, but before posting it, so the response can never be
+ * recorded as older than the request.
+ */
+ req_time = jiffies;
+
err = fbnic_mbx_map_tlv_msg(fbd, msg);
if (err)
goto free_message;
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [net PATCH 6/6] fbnic: Allocate FW completions with kvzalloc
2026-11-13 20:41 [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations Alexander Duyck
` (4 preceding siblings ...)
2026-11-13 20:42 ` [net PATCH 5/6] fbnic: Stamp heartbeat and ownership requests after allocating them Alexander Duyck
@ 2026-11-13 20:42 ` Alexander Duyck
5 siblings, 0 replies; 13+ messages in thread
From: Alexander Duyck @ 2026-11-13 20:42 UTC (permalink / raw)
To: netdev; +Cc: andrew+netdev, davem, edumazet, horms, kernel-team, kuba, pabeni
From: Alexander Duyck <alexanderduyck@fb.com>
The fbnic_fw_reporter_dump function reads the firmware coredump into a
completion that carries the whole dump, allocated by the
__fbnic_fw_alloc_cmpl function with kzalloc. A dump of a few hundred
kilobytes makes that a high-order physically contiguous allocation.
The dump runs from the devlink health auto-dump when the FW reporter
fires, which happens on the service task with RTNL held, and a reporter
firing on a missed heartbeat is most likely when the host is already
short of memory. That is where such an allocation is least likely to
succeed and most likely to drive reclaim and compaction hard.
Allocate completions with kvzalloc and free them with kvfree. Small
completions are still served by the slab allocator, and the coredump
falls back to vmalloc instead of failing or stalling. Nothing hands the
buffer to the device: the mailbox Rx path copies each chunk in with
memcpy.
The dump size comes from the firmware, and until now kmalloc refusing
anything too large was the only bound on it. The firmware keeps its
core dump, header included, in a 256K region of its flash, so reject a
larger size as bogus instead of allocating and reading it with RTNL
held.
This was seen on production hosts during host memory stalls, where the
auto-dump that followed a missed heartbeat logged an order-6 page
allocation failure in this allocation.
Fixes: 005a54722e9d ("eth: fbnic: add FW health reporter")
Signed-off-by: Alexander Duyck <alexanderduyck@fb.com>
---
drivers/net/ethernet/meta/fbnic/fbnic_devlink.c | 5 +++++
drivers/net/ethernet/meta/fbnic/fbnic_fw.c | 5 +++--
drivers/net/ethernet/meta/fbnic/fbnic_fw.h | 5 +++++
3 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c b/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c
index 0b653c4c20a7..f63cf3224510 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_devlink.c
@@ -414,6 +414,11 @@ static int fbnic_fw_reporter_dump(struct devlink_health_reporter *reporter,
"Firmware core dump returned size 0");
return -EIO;
}
+ if (size > FBNIC_FW_COREDUMP_MAX_SIZE) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Firmware core dump size is too large");
+ return -EIO;
+ }
/* Read the dump, we can only transfer TLV_MAX_DATA at a time */
index_count = DIV_ROUND_UP(size, TLV_MAX_DATA);
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
index 3465f0139883..96cfe0b4cd8d 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
@@ -382,7 +382,7 @@ static void fbnic_fw_release_cmpl_data(struct kref *kref)
cmpl_data = container_of(kref, struct fbnic_fw_completion,
ref_count);
- kfree(cmpl_data);
+ kvfree(cmpl_data);
}
static struct fbnic_fw_completion *
@@ -2108,7 +2108,8 @@ struct fbnic_fw_completion *__fbnic_fw_alloc_cmpl(u32 msg_type,
{
struct fbnic_fw_completion *cmpl;
- cmpl = kzalloc(sizeof(*cmpl) + priv_size, GFP_KERNEL);
+ /* Can be large, and the device never accesses it, so allow vmalloc */
+ cmpl = kvzalloc(sizeof(*cmpl) + priv_size, GFP_KERNEL);
if (!cmpl)
return NULL;
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
index f47669129c9d..52f69274889e 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
@@ -40,6 +40,11 @@ struct fbnic_fw_mbx {
#define FBNIC_FW_MAX_LOG_HISTORY 14
#define FBNIC_MBX_RX_TO_SEC 10
+/* The FW keeps its core dump, header included, in a 256K region of its
+ * QSPI flash, so no valid dump is larger than that.
+ */
+#define FBNIC_FW_COREDUMP_MAX_SIZE SZ_256K
+
struct fbnic_fw_ver {
u32 version;
char commit[FBNIC_FW_CAP_RESP_COMMIT_MAX_SIZE];
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle
2026-11-13 20:41 ` [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle Alexander Duyck
@ 2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-17 21:31 ` Alexander Duyck
0 siblings, 1 reply; 13+ messages in thread
From: netdev-bot+sashiko @ 2026-11-17 21:14 UTC (permalink / raw)
To: alexander.duyck
Cc: netdev, andrew+netdev, davem, edumazet, horms, kernel-team, kuba,
pabeni
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The new comment above FBNIC_IDLE_POLL_US, and the commit
message, say a 20us delay keeps the PCIe reads 'a small part of each…
--
fbnic: Wait for in-flight PTP timestamps before treating Tx as idle
This patch moves the Tx idle wait into its own function,
fbnic_wait_tx_idle(). The Tx path now counts as idle only when the
QM idle registers report idle and FBNIC_TCE_PTP_INFLIGHT reads zero.
The patch also raises the poll delay from 2us to 20us and extends
FBNIC_CSR_END_TCE so the ethtool register dump covers the new register.
> diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> index 07d4b2e923627..c75c5683b724e 100644
> --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> @@ -2398,38 +2398,85 @@ static void fbnic_idle_dump(struct fbnic_dev *fbd,
> fbnic_rd32(fbd, regs[i].reg_base + j));
> }
>
> -int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail)
> +/* The poll timeout only accounts for the delay between polls, not for
> + * the time spent in the poll itself, and each poll is a series of
> + * non-posted PCIe register reads that take on the order of a
> + * microsecond each. Space the polls out so that the reads stay a small
> + * part of each interval and the timeout stays close to wall time.
> + */
> +#define FBNIC_IDLE_POLL_US 20
> +#define FBNIC_IDLE_TIMEOUT_US 500000
> +
> +static bool fbnic_tx_idle(struct fbnic_dev *fbd,
> + const struct fbnic_idle_regs *regs,
> + unsigned int nregs)
> +{
> + u32 inflight;
> +
> + if (!fbnic_all_idle(fbd, regs, nregs))
> + return false;
> +
> + /* The QM idle bits do not cover the TMI completion path, so a
> + * timestamp request can still be waiting on its completion. The
> + * count is bounded by the timestamp requests the Tx path can hold,
> + * so all ones is a failed read, as for the idle bits, not a count.
> + */
> + inflight = fbnic_rd32(fbd, FBNIC_TCE_PTP_INFLIGHT);
[Severity: Medium]
Does a 20us delay really keep the reads "a small part of each interval"
in the case this patch adds, where the QM registers are idle but
FBNIC_TCE_PTP_INFLIGHT is still non-zero?
In that state, every poll in fbnic_tx_idle() reads all 28 Tx QM idle
registers (8 TWQ + 8 TQS + 8 TDE + 4 TCQ) as all ones before it reads
FBNIC_TCE_PTP_INFLIGHT.
For each all-ones read, fbnic_rd32() does a second non-posted read:
drivers/net/ethernet/meta/fbnic/fbnic_pci.c:fbnic_rd32() {
...
if (reg != FBNIC_MASTER_SPARE_0 && ~readl(csr + FBNIC_MASTER_SPARE_0))
return value;
...
}
That makes about 57 reads per poll. At the comment's own figure of
about 1us per read, that is roughly 57us of reads against the 20us
delay, so the reads are about three quarters of each interval.
poll_timeout_us_atomic() only charges the delay against the timeout:
if (__delay_us) {
udelay(__delay_us);
if (__timeout_us)
__left_ns -= __delay_ns;
}
So the 500000us timeout allows about 25000 polls of about 77us each.
That is roughly 1.9s per Tx wait rather than 500ms.
fbnic_wait_tx_idle() can run two of these waits, one before and one
after fbnic_tx_flush(), so about 3.9s. The Rx wait reads 12 registers,
24 reads per poll, so it adds about another 1.1s.
All of this is udelay() busy-waiting while the RTNL/netdev lock is held
in fbnic_down(), fbnic_set_ringparam() and fbnic_set_channels().
Could the delay be sized for the real per-poll read count? Another
option is to bound the wait by elapsed time, for example by reading
FBNIC_TCE_PTP_INFLIGHT first or using a ktime-based deadline. If the
code stays as it is, could the comment and the part of the commit
message saying "the 500ms timeout stays close to wall time" be updated?
> +
> + return !inflight || !~inflight;
> +}
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/179460241885.8819.7179935130629202331.stgit%40ahduyck-xeon-server.home.arpa
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [net PATCH 2/6] fbnic: Do not accumulate HW stats from failed register reads
2026-11-13 20:41 ` [net PATCH 2/6] fbnic: Do not accumulate HW stats from failed register reads Alexander Duyck
@ 2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-17 22:11 ` Alexander Duyck
0 siblings, 1 reply; 13+ messages in thread
From: netdev-bot+sashiko @ 2026-11-17 21:14 UTC (permalink / raw)
To: alexander.duyck
Cc: netdev, andrew+netdev, davem, edumazet, horms, kernel-team, kuba,
pabeni
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.
Critical: 0 · High: 1 · Medium: 0 · Low: 1
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [High] The fix is incomplete: `fbnic_pcs_rsfec_stat_rd32()` in
drivers/net/ethernet/meta/fbnic/fbnic_mac.c still adds failed reads…
- [Low] The commit message describes the failed-read symptom accurately
only for the 64-bit counters.
--
fbnic: Do not accumulate HW stats from failed register reads
This commit changes the fbnic HW stats helpers so they skip updating the
running totals once a failed register read has disabled CSR access. The
check is added to fbnic_hw_stat_rd32(), fbnic_hw_stat_rd64(),
__fbnic_mac_stat_rd64() and fbnic_mac_stat_rd32().
> When a register read fails, as it does for every read once a DPC
> containment event takes the link down, the fbnic_rd32 function returns
> all ones and disables CSR access, so every later read also returns all
> ones until the device is recovered. A stats read in that window adds the
> difference between all ones and the last value to the total, which
> subtracts the counter's whole value since the last rebase.
[Severity: Low]
Does this description hold for the 32-bit counters?
In fbnic_hw_stat_rd32() and fbnic_mac_stat_rd32() the delta is computed
in u32:
stat->value += new_reg_value - stat->u.old_reg_value_32;
When new_reg_value == 0xffffffff, this gives 0xffffffff - old. That value
is zero-extended and added to the u64 stat->value. So the 32-bit counters
(TMI/TTI/RPC/RXB drops, per-queue HW drops, 32-bit MAC stats) would jump
forward by nearly 2^32 instead of dropping.
Only the 64-bit helpers wrap the total downward, because fbnic_stat_rd64()
returns ~0ULL there. The code change looks correct for both widths. Could
the commit message also mention the forward jump in the 32-bit counters?
> diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
> index 16271c717a5aa..2aacbf0a6af40 100644
> --- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
> +++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
[ ... ]
> @@ -417,6 +421,10 @@ static void fbnic_mac_stat_rd32(struct fbnic_dev *fbd, bool reset, u32 reg,
> u32 new_reg_value;
>
> new_reg_value = rd32(fbd, reg);
> + /* A failed read disables CSR access and returns all ones */
> + if (!fbnic_present(fbd))
> + return;
> +
> if (!reset)
> stat->value += new_reg_value - stat->u.old_reg_value_32;
> stat->u.old_reg_value_32 = new_reg_value;
[Severity: High]
Should fbnic_pcs_rsfec_stat_rd32() in this same file get the same check?
It reads the clear-on-read FEC/PCS counters and never calls
fbnic_present():
drivers/net/ethernet/meta/fbnic/fbnic_mac.c:fbnic_pcs_rsfec_stat_rd32() {
...
pcs_rsfec_stat = rd32(fbd, reg) & 0xffff;
pcs_rsfec_stat |= rd32(fbd, reg + 1) << 16;
/* RFSEC registers clear themselves upon being read so there is no
* need to store the old_reg_value.
*/
if (!reset)
stat->value += pcs_rsfec_stat;
}
After fbnic_rd32() clears uc_addr0, both reads return ~0U, so
pcs_rsfec_stat ends up as 0xffffffff. These counters have no baseline to
subtract. As a result, corrected_blocks, uncorrectable_blocks and every
SymbolErrorDuringCarrier lane would grow by about 4G on each stats poll
while the device is unreachable.
One path that reaches it:
fbnic_get_stats64()
fbnic_get_hw_stats()
__fbnic_get_hw_stats32()
fbnic_get_phy_stats32() /* reset == false */
fbnic_mac_get_fec_stats() / fbnic_mac_get_pcs_stats()
fbnic_pcs_rsfec_stat_rd32()
The service task reaches it the same way through fbnic_get_hw_stats32(),
and so does ethtool through fbnic_get_fec_stats().
dev_get_stats() does not check netif_device_present(). A read of
/proc/net/dev during a DPC/AER outage, or after fbnic_pm_suspend(), would
therefore hit this path.
fbnic_reset_hw_stats() calls these readers only with reset == true. That
skips the add but does not rewind stat->value, so the inflated totals
would remain after recovery.
Would adding the same "if (!fbnic_present(fbd)) return;" after the two
reads in fbnic_pcs_rsfec_stat_rd32() close this gap?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/179460241885.8819.7179935130629202331.stgit%40ahduyck-xeon-server.home.arpa
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [net PATCH 3/6] fbnic: Cover all of the HW stats with the stats lock
2026-11-13 20:41 ` [net PATCH 3/6] fbnic: Cover all of the HW stats with the stats lock Alexander Duyck
@ 2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-17 22:01 ` Alexander Duyck
0 siblings, 1 reply; 13+ messages in thread
From: netdev-bot+sashiko @ 2026-11-17 21:14 UTC (permalink / raw)
To: alexander.duyck
Cc: netdev, andrew+netdev, davem, edumazet, horms, kernel-team, kuba,
pabeni
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The commit message and Fixes: tag describe a race that cannot
happen at this commit.
--
fbnic: Cover all of the HW stats with the stats lock
This patch puts the fbnic MAC stats under hw_stats.lock. The lock is now
taken in the ethtool pause, eth-mac, eth-ctrl and rmon getters and in
fbnic_reset_hw_stats(), and the ASSERT_RTNL() is gone. It also drops
rtnl_lock() from the PCIe stats debugfs read.
> That is no
> longer the case. ethtool only takes RTNL on the Netlink path for the
> operations a driver lists in op_needs_rtnl, and fbnic lists only the
> pause getter among the stats ones, so the eth-mac, eth-ctrl and rmon
> getters update the MAC stats with only the netdev instance lock held,
> racing each other and the reset.
[ ... ]
> Fixes: 45079e00133e ("net: ethtool: optionally skip rtnl_lock on Netlink path for GET ops")
[Severity: Medium]
Can the race described here actually happen at this commit?
fbnic is ops-locked. fbnic_netdev_alloc() sets queue_mgmt_ops
unconditionally:
netdev->queue_mgmt_ops = &fbnic_queue_mgmt_ops;
That makes netdev_need_ops_lock() true, so netdev_lock_ops() really takes
netdev->lock for this device.
ethnl_default_doit() takes that lock around prepare_data() whether or not
RTNL is needed:
if (need_rtnl)
rtnl_lock();
netdev_lock_ops(req_info->dev);
}
ret = ops->prepare_data(req_info, reply_data, info);
The dump path does the same, and the notify path asserts the same lock.
The only core callers of the fbnic get_eth_mac_stats, get_eth_ctrl_stats,
get_rmon_stats and get_pause_stats callbacks are stats_prepare_data() and
pause_prepare_data(). So the getters are already serialized against each
other by the instance lock.
fbnic_reset_hw_stats() has two callers. fbnic_init_hw_stats() runs from
fbnic_probe() before the netdev is allocated or registered, so no ethtool
request can reach the device yet. __fbnic_pm_attach() holds the same
instance lock around the reset:
rtnl_lock();
netdev_lock(netdev);
fbnic_reset_hw_stats(fbd);
Nothing else touches fbd->hw_stats.mac or calls the fbd->mac->get_*_stats
callbacks.
After 45079e00133e, the old comment and ASSERT_RTNL() in
fbnic_reset_hw_stats() were out of date. But netdev->lock still seems to
serialize every path that touches the MAC stats.
The locking change itself looks fine as a consistency cleanup. Later in
this series the reset moves into __fbnic_pm_resume() and runs under
hw_stats.lock only, without netdev_lock. That later change is what makes
the lock added here necessary.
Could the commit message say that the instance lock already serializes
these paths, and that this patch makes hw_stats.lock the single
protection to prepare for the later reset move? Could the Fixes: tag
also be dropped, or could the message name a path that actually reaches
the race? As written, the tag may lead to this being backported to
stable as a fix for a race that does not exist there.
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/179460241885.8819.7179935130629202331.stgit%40ahduyck-xeon-server.home.arpa
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle
2026-11-17 21:14 ` netdev-bot+sashiko
@ 2026-11-17 21:31 ` Alexander Duyck
0 siblings, 0 replies; 13+ messages in thread
From: Alexander Duyck @ 2026-11-17 21:31 UTC (permalink / raw)
To: netdev-bot+sashiko
Cc: netdev, andrew+netdev, davem, edumazet, horms, kernel-team, kuba,
pabeni
On Tue, Nov 17, 2026 at 1:14 PM <netdev-bot+sashiko@kernel.org> wrote:
>
> Thank you for your contribution! Sashiko AI review found 1 potential
> issue(s) to consider.
>
> Critical: 0 · High: 0 · Medium: 1 · Low: 0
>
> If you believe none of the issues are legitimate, please reply to each
> issue of Medium severity or higher and explain why the AI is wrong.
> If any of them are legitimate and you plan to respin, please let patchwork
> know by including "pw-bot: cr" as a separate line at the end of your reply
> (one such reply per series is enough).
>
> - [Medium] The new comment above FBNIC_IDLE_POLL_US, and the commit
> message, say a 20us delay keeps the PCIe reads 'a small part of each…
>
> --
>
> fbnic: Wait for in-flight PTP timestamps before treating Tx as idle
>
> This patch moves the Tx idle wait into its own function,
> fbnic_wait_tx_idle(). The Tx path now counts as idle only when the
> QM idle registers report idle and FBNIC_TCE_PTP_INFLIGHT reads zero.
> The patch also raises the poll delay from 2us to 20us and extends
> FBNIC_CSR_END_TCE so the ethtool register dump covers the new register.
>
> > diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> > index 07d4b2e923627..c75c5683b724e 100644
> > --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> > +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> > @@ -2398,38 +2398,85 @@ static void fbnic_idle_dump(struct fbnic_dev *fbd,
> > fbnic_rd32(fbd, regs[i].reg_base + j));
> > }
> >
> > -int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail)
> > +/* The poll timeout only accounts for the delay between polls, not for
> > + * the time spent in the poll itself, and each poll is a series of
> > + * non-posted PCIe register reads that take on the order of a
> > + * microsecond each. Space the polls out so that the reads stay a small
> > + * part of each interval and the timeout stays close to wall time.
> > + */
> > +#define FBNIC_IDLE_POLL_US 20
> > +#define FBNIC_IDLE_TIMEOUT_US 500000
> > +
> > +static bool fbnic_tx_idle(struct fbnic_dev *fbd,
> > + const struct fbnic_idle_regs *regs,
> > + unsigned int nregs)
> > +{
> > + u32 inflight;
> > +
> > + if (!fbnic_all_idle(fbd, regs, nregs))
> > + return false;
> > +
> > + /* The QM idle bits do not cover the TMI completion path, so a
> > + * timestamp request can still be waiting on its completion. The
> > + * count is bounded by the timestamp requests the Tx path can hold,
> > + * so all ones is a failed read, as for the idle bits, not a count.
> > + */
> > + inflight = fbnic_rd32(fbd, FBNIC_TCE_PTP_INFLIGHT);
>
> [Severity: Medium]
> Does a 20us delay really keep the reads "a small part of each interval"
> in the case this patch adds, where the QM registers are idle but
> FBNIC_TCE_PTP_INFLIGHT is still non-zero?
>
> In that state, every poll in fbnic_tx_idle() reads all 28 Tx QM idle
> registers (8 TWQ + 8 TQS + 8 TDE + 4 TCQ) as all ones before it reads
> FBNIC_TCE_PTP_INFLIGHT.
>
> For each all-ones read, fbnic_rd32() does a second non-posted read:
>
> drivers/net/ethernet/meta/fbnic/fbnic_pci.c:fbnic_rd32() {
> ...
> if (reg != FBNIC_MASTER_SPARE_0 && ~readl(csr + FBNIC_MASTER_SPARE_0))
> return value;
> ...
> }
>
> That makes about 57 reads per poll. At the comment's own figure of
> about 1us per read, that is roughly 57us of reads against the 20us
> delay, so the reads are about three quarters of each interval.
>
> poll_timeout_us_atomic() only charges the delay against the timeout:
>
> if (__delay_us) {
> udelay(__delay_us);
> if (__timeout_us)
> __left_ns -= __delay_ns;
> }
>
> So the 500000us timeout allows about 25000 polls of about 77us each.
> That is roughly 1.9s per Tx wait rather than 500ms.
>
> fbnic_wait_tx_idle() can run two of these waits, one before and one
> after fbnic_tx_flush(), so about 3.9s. The Rx wait reads 12 registers,
> 24 reads per poll, so it adds about another 1.1s.
>
> All of this is udelay() busy-waiting while the RTNL/netdev lock is held
> in fbnic_down(), fbnic_set_ringparam() and fbnic_set_channels().
>
> Could the delay be sized for the real per-poll read count? Another
> option is to bound the wait by elapsed time, for example by reading
> FBNIC_TCE_PTP_INFLIGHT first or using a ktime-based deadline. If the
> code stays as it is, could the comment and the part of the commit
> message saying "the 500ms timeout stays close to wall time" be updated?
>
> > +
> > + return !inflight || !~inflight;
> > +}
>
> [ ... ]
This is assuming the worst case, but in the worst case we are talking
about the Terminus Tx or Rx path essentially being locked up to
trigger us being stuck in the wait that long.
The likelihood of this is incredibly low, and the change that was
already made reduced the worst case to under 10 seconds which I feel
is reasonable in the case of shutting down an interface that is
potentially in a stuck setup and requiring additional time to flush.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [net PATCH 3/6] fbnic: Cover all of the HW stats with the stats lock
2026-11-17 21:14 ` netdev-bot+sashiko
@ 2026-11-17 22:01 ` Alexander Duyck
0 siblings, 0 replies; 13+ messages in thread
From: Alexander Duyck @ 2026-11-17 22:01 UTC (permalink / raw)
To: netdev-bot+sashiko
Cc: netdev, andrew+netdev, davem, edumazet, horms, kernel-team, kuba,
pabeni
On Tue, Nov 17, 2026 at 1:14 PM <netdev-bot+sashiko@kernel.org> wrote:
>
> Thank you for your contribution! Sashiko AI review found 1 potential
> issue(s) to consider.
>
> Critical: 0 · High: 0 · Medium: 1 · Low: 0
>
> If you believe none of the issues are legitimate, please reply to each
> issue of Medium severity or higher and explain why the AI is wrong.
> If any of them are legitimate and you plan to respin, please let patchwork
> know by including "pw-bot: cr" as a separate line at the end of your reply
> (one such reply per series is enough).
>
> - [Medium] The commit message and Fixes: tag describe a race that cannot
> happen at this commit.
>
> --
>
> fbnic: Cover all of the HW stats with the stats lock
>
> This patch puts the fbnic MAC stats under hw_stats.lock. The lock is now
> taken in the ethtool pause, eth-mac, eth-ctrl and rmon getters and in
> fbnic_reset_hw_stats(), and the ASSERT_RTNL() is gone. It also drops
> rtnl_lock() from the PCIe stats debugfs read.
>
> > That is no
> > longer the case. ethtool only takes RTNL on the Netlink path for the
> > operations a driver lists in op_needs_rtnl, and fbnic lists only the
> > pause getter among the stats ones, so the eth-mac, eth-ctrl and rmon
> > getters update the MAC stats with only the netdev instance lock held,
> > racing each other and the reset.
>
> [ ... ]
>
> > Fixes: 45079e00133e ("net: ethtool: optionally skip rtnl_lock on Netlink path for GET ops")
>
> [Severity: Medium]
> Can the race described here actually happen at this commit?
>
> fbnic is ops-locked. fbnic_netdev_alloc() sets queue_mgmt_ops
> unconditionally:
>
> netdev->queue_mgmt_ops = &fbnic_queue_mgmt_ops;
>
> That makes netdev_need_ops_lock() true, so netdev_lock_ops() really takes
> netdev->lock for this device.
>
> ethnl_default_doit() takes that lock around prepare_data() whether or not
> RTNL is needed:
>
> if (need_rtnl)
> rtnl_lock();
> netdev_lock_ops(req_info->dev);
> }
> ret = ops->prepare_data(req_info, reply_data, info);
>
> The dump path does the same, and the notify path asserts the same lock.
> The only core callers of the fbnic get_eth_mac_stats, get_eth_ctrl_stats,
> get_rmon_stats and get_pause_stats callbacks are stats_prepare_data() and
> pause_prepare_data(). So the getters are already serialized against each
> other by the instance lock.
>
> fbnic_reset_hw_stats() has two callers. fbnic_init_hw_stats() runs from
> fbnic_probe() before the netdev is allocated or registered, so no ethtool
> request can reach the device yet. __fbnic_pm_attach() holds the same
> instance lock around the reset:
>
> rtnl_lock();
> netdev_lock(netdev);
>
> fbnic_reset_hw_stats(fbd);
>
> Nothing else touches fbd->hw_stats.mac or calls the fbd->mac->get_*_stats
> callbacks.
>
> After 45079e00133e, the old comment and ASSERT_RTNL() in
> fbnic_reset_hw_stats() were out of date. But netdev->lock still seems to
> serialize every path that touches the MAC stats.
>
> The locking change itself looks fine as a consistency cleanup. Later in
> this series the reset moves into __fbnic_pm_resume() and runs under
> hw_stats.lock only, without netdev_lock. That later change is what makes
> the lock added here necessary.
This is the key piece here. The locking change is fine, however it
seems to be hung up on the commit message below.
> Could the commit message say that the instance lock already serializes
> these paths, and that this patch makes hw_stats.lock the single
> protection to prepare for the later reset move? Could the Fixes: tag
> also be dropped, or could the message name a path that actually reaches
> the race? As written, the tag may lead to this being backported to
> stable as a fix for a race that does not exist there.
Admittedly the MAC stats were not really the focus of the change. It
was more about the PCIe stats as we were stuck under the RTNL with it
and it didn't make sense to lock PCIe stats under a networking lock.
The whole point of the patch set was two fold. First was to
consolidate everything under one lock as we were dealing with RTNL,
netdev instance lock, and HW stats lock. Rather than have 3 locks we
could place it all under 1 and that is a significant cleanup and
enables the initialization/reset fixes as the AI agent correctly
pointed out.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [net PATCH 2/6] fbnic: Do not accumulate HW stats from failed register reads
2026-11-17 21:14 ` netdev-bot+sashiko
@ 2026-11-17 22:11 ` Alexander Duyck
0 siblings, 0 replies; 13+ messages in thread
From: Alexander Duyck @ 2026-11-17 22:11 UTC (permalink / raw)
To: netdev-bot+sashiko
Cc: netdev, andrew+netdev, davem, edumazet, horms, kernel-team, kuba,
pabeni
On Tue, Nov 17, 2026 at 1:14 PM <netdev-bot+sashiko@kernel.org> wrote:
>
> Thank you for your contribution! Sashiko AI review found 2 potential
> issue(s) to consider.
>
> Critical: 0 · High: 1 · Medium: 0 · Low: 1
>
> If you believe none of the issues are legitimate, please reply to each
> issue of Medium severity or higher and explain why the AI is wrong.
> If any of them are legitimate and you plan to respin, please let patchwork
> know by including "pw-bot: cr" as a separate line at the end of your reply
> (one such reply per series is enough).
>
> - [High] The fix is incomplete: `fbnic_pcs_rsfec_stat_rd32()` in
> drivers/net/ethernet/meta/fbnic/fbnic_mac.c still adds failed reads…
> - [Low] The commit message describes the failed-read symptom accurately
> only for the 64-bit counters.
>
> --
>
> fbnic: Do not accumulate HW stats from failed register reads
>
> This commit changes the fbnic HW stats helpers so they skip updating the
> running totals once a failed register read has disabled CSR access. The
> check is added to fbnic_hw_stat_rd32(), fbnic_hw_stat_rd64(),
> __fbnic_mac_stat_rd64() and fbnic_mac_stat_rd32().
>
> > When a register read fails, as it does for every read once a DPC
> > containment event takes the link down, the fbnic_rd32 function returns
> > all ones and disables CSR access, so every later read also returns all
> > ones until the device is recovered. A stats read in that window adds the
> > difference between all ones and the last value to the total, which
> > subtracts the counter's whole value since the last rebase.
>
> [Severity: Low]
> Does this description hold for the 32-bit counters?
>
> In fbnic_hw_stat_rd32() and fbnic_mac_stat_rd32() the delta is computed
> in u32:
>
> stat->value += new_reg_value - stat->u.old_reg_value_32;
>
> When new_reg_value == 0xffffffff, this gives 0xffffffff - old. That value
> is zero-extended and added to the u64 stat->value. So the 32-bit counters
> (TMI/TTI/RPC/RXB drops, per-queue HW drops, 32-bit MAC stats) would jump
> forward by nearly 2^32 instead of dropping.
>
> Only the 64-bit helpers wrap the total downward, because fbnic_stat_rd64()
> returns ~0ULL there. The code change looks correct for both widths. Could
> the commit message also mention the forward jump in the 32-bit counters?
If a follow-up is requested we can look at updating the commit
message. The thing to keep in mind is that there aren't that many 32b
registers for the interface, so the much more common visibility to the
issue is the issues seen with the 64b counters.
> > diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
> > index 16271c717a5aa..2aacbf0a6af40 100644
> > --- a/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
> > +++ b/drivers/net/ethernet/meta/fbnic/fbnic_mac.c
>
> [ ... ]
>
> > @@ -417,6 +421,10 @@ static void fbnic_mac_stat_rd32(struct fbnic_dev *fbd, bool reset, u32 reg,
> > u32 new_reg_value;
> >
> > new_reg_value = rd32(fbd, reg);
> > + /* A failed read disables CSR access and returns all ones */
> > + if (!fbnic_present(fbd))
> > + return;
> > +
> > if (!reset)
> > stat->value += new_reg_value - stat->u.old_reg_value_32;
> > stat->u.old_reg_value_32 = new_reg_value;
>
> [Severity: High]
> Should fbnic_pcs_rsfec_stat_rd32() in this same file get the same check?
>
> It reads the clear-on-read FEC/PCS counters and never calls
> fbnic_present():
>
> drivers/net/ethernet/meta/fbnic/fbnic_mac.c:fbnic_pcs_rsfec_stat_rd32() {
> ...
> pcs_rsfec_stat = rd32(fbd, reg) & 0xffff;
> pcs_rsfec_stat |= rd32(fbd, reg + 1) << 16;
>
> /* RFSEC registers clear themselves upon being read so there is no
> * need to store the old_reg_value.
> */
> if (!reset)
> stat->value += pcs_rsfec_stat;
> }
>
> After fbnic_rd32() clears uc_addr0, both reads return ~0U, so
> pcs_rsfec_stat ends up as 0xffffffff. These counters have no baseline to
> subtract. As a result, corrected_blocks, uncorrectable_blocks and every
> SymbolErrorDuringCarrier lane would grow by about 4G on each stats poll
> while the device is unreachable.
>
> One path that reaches it:
>
> fbnic_get_stats64()
> fbnic_get_hw_stats()
> __fbnic_get_hw_stats32()
> fbnic_get_phy_stats32() /* reset == false */
> fbnic_mac_get_fec_stats() / fbnic_mac_get_pcs_stats()
> fbnic_pcs_rsfec_stat_rd32()
>
> The service task reaches it the same way through fbnic_get_hw_stats32(),
> and so does ethtool through fbnic_get_fec_stats().
>
> dev_get_stats() does not check netif_device_present(). A read of
> /proc/net/dev during a DPC/AER outage, or after fbnic_pm_suspend(), would
> therefore hit this path.
>
> fbnic_reset_hw_stats() calls these readers only with reset == true. That
> skips the add but does not rewind stat->value, so the inflated totals
> would remain after recovery.
>
> Would adding the same "if (!fbnic_present(fbd)) return;" after the two
> reads in fbnic_pcs_rsfec_stat_rd32() close this gap?
Yeah, this probably does need the same fix. If possible I would prefer
to do this as a follow-on patch.
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2026-11-17 22:12 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-13 20:41 [net PATCH 0/6] fbnic: Follow-up fixes for the Tx idle check, HW stats and FW mailbox allocations Alexander Duyck
2026-11-13 20:41 ` [net PATCH 1/6] fbnic: Wait for in-flight PTP timestamps before treating Tx as idle Alexander Duyck
2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-17 21:31 ` Alexander Duyck
2026-11-13 20:41 ` [net PATCH 2/6] fbnic: Do not accumulate HW stats from failed register reads Alexander Duyck
2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-17 22:11 ` Alexander Duyck
2026-11-13 20:41 ` [net PATCH 3/6] fbnic: Cover all of the HW stats with the stats lock Alexander Duyck
2026-11-17 21:14 ` netdev-bot+sashiko
2026-11-17 22:01 ` Alexander Duyck
2026-11-13 20:41 ` [net PATCH 4/6] fbnic: Initialize and reset the HW stats before they can be read Alexander Duyck
2026-11-13 20:42 ` [net PATCH 5/6] fbnic: Stamp heartbeat and ownership requests after allocating them Alexander Duyck
2026-11-13 20:42 ` [net PATCH 6/6] fbnic: Allocate FW completions with kvzalloc Alexander Duyck
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox