From: Joe Damato <joe@dama.to>
To: netdev@vger.kernel.org
Cc: andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com,
kuba@kernel.org, pabeni@redhat.com, horms@kernel.org,
michael.chan@broadcom.com, pavan.chebbi@broadcom.com,
linux-kernel@vger.kernel.org, Joe Damato <joe@dama.to>
Subject: [RFC net v2 0/3] bnxt_en: Make RING FREE more robust
Date: Tue, 22 Sep 2026 11:24:00 -0700 [thread overview]
Message-ID: <20260922182405.1290749-1-joe@dama.to> (raw)
Greetings:
This is a follow up to the previous RFC (linked below), which takes a slightly
different approach, is now targeted at net instead of net-next, and has a new
subject line :)
On two production systems, I saw the following dmesg pattern:
NETDEV WATCHDOG: transmit queue 0 timed out 6073 ms
Resp cmpl intr err msg: 0x51 x20
hwrm_ring_free type 1 failed x12
hwrm_ring_free type 2 failed x8
AMD-Vi: IO_PAGE_FAULT x3
This suggests that, for some currently unknown reason, TX completions stall
and the netdev watchdog fires. The driver asks FW to free the rings, this
times out, but the driver ignores the possible failure and frees ring memory.
Since the FW didn't respond to the ring free command, it is possible that the
FW is still DMAing to the memory which was freed.
This series tries to prevent this by:
- Returning and checking ring free command return values
- Examining the FW response if the ring free command times out. It is
possible that, for some reason, the FW did complete the ring free but was
unable to respond with an IRQ. This seems unlikely given what appears to be
a use after free in dmesg, but worth logging just in case.
- Lastly, disable the device to stop DMA before the driver frees ring memory,
which should prevent any possible use after free.
Sending this as an RFC so that the Broadcom folks have some time to take a
look and test as needed.
Thanks,
Joe
v2:
- No changes to patch 1
- Patch 2 from v1 dropped
- Patch 2 in the v2 now checks the response and logs state before giving up
- Patch 3 in the v2 disables the device to stop DMA before freeing ring
memory
RFCv1: https://lore.kernel.org/netdev/20260917233218.1160001-1-joe@dama.to/
Joe Damato (3):
bnxt_en: return the RING_FREE status to callers
bnxt_en: check HWRM response if completion never arrives
bnxt_en: stop DMA before releasing rings the firmware did not free
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 69 +++++++++++------
.../net/ethernet/broadcom/bnxt/bnxt_hwrm.c | 75 ++++++++++++++-----
2 files changed, 102 insertions(+), 42 deletions(-)
base-commit: 17741334d00bf5ebd37f8c1c36bc9c146a351deb
--
2.53.0-Meta
next reply other threads:[~2026-09-22 18:24 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-22 18:24 Joe Damato [this message]
2026-09-22 18:24 ` [RFC net v2 1/3] bnxt_en: return the RING_FREE status to callers Joe Damato
2026-09-22 18:24 ` [RFC net v2 2/3] bnxt_en: check HWRM response if completion never arrives Joe Damato
2026-09-23 4:14 ` Michael Chan
2026-09-22 18:24 ` [RFC net v2 3/3] bnxt_en: stop DMA before releasing rings the firmware did not free Joe Damato
2026-09-23 4:43 ` Michael Chan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260922182405.1290749-1-joe@dama.to \
--to=joe@dama.to \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=horms@kernel.org \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=michael.chan@broadcom.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=pavan.chebbi@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox