From: Long Li <longli@microsoft.com>
To: "K . Y . Srinivasan" <kys@microsoft.com>,
Haiyang Zhang <haiyangz@microsoft.com>,
Wei Liu <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
"David S . Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Shradha Gupta <shradhagupta@linux.microsoft.com>,
Simon Horman <horms@kernel.org>,
Konstantin Taranov <kotaranov@microsoft.com>,
Souradeep Chakrabarti <schakrabarti@linux.microsoft.com>,
Erick Archer <erick.archer@outlook.com>,
linux-hyperv@vger.kernel.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org
Cc: Long Li <longli@microsoft.com>
Subject: [PATCH 0/8] RDMA/mana_ib: Handle service reset for RDMA resources
Date: Fri, 6 Mar 2026 17:44:04 -0800 [thread overview]
Message-ID: <20260307014414.556256-1-longli@microsoft.com> (raw)
When the MANA hardware undergoes a service reset, the ETH auxiliary device
(mana.eth) used by DPDK persists across the reset cycle — it is not removed
and re-added like RC/UD/GSI QPs. This means userspace RDMA consumers such
as DPDK have no way of knowing that firmware handles for their PD, CQ, WQ,
QP and MR resources have become stale.
This series adds per-ucontext resource tracking and a reset notification
mechanism so that:
1. The RDMA driver is informed of service reset events via direct callbacks
from the ETH driver (reset_notify / resume_notify).
2. On reset, all tracked firmware handles are invalidated (set to
INVALID_MANA_HANDLE), user doorbell mappings are revoked via
rdma_user_mmap_disassociate(), and IB_EVENT_PORT_ERR is dispatched to
each affected ucontext so userspace can detect the reset.
3. Destroy callbacks check for INVALID_MANA_HANDLE and skip firmware
commands for resources already invalidated by the reset path,
preventing stale handles from being sent to firmware.
4. A reset_rwsem serializes handle invalidation against resource creation
to avoid races between the reset path and new resource allocation.
Patches 1-6 introduce per-ucontext tracking lists for each resource type.
Patch 7 implements the reset/resume notification mechanism with rwsem
serialization, mmap revocation, and IB event dispatch.
Patch 8 adds INVALID_MANA_HANDLE checks in destroy callbacks.
Tested with DPDK testpmd on Azure VM (linux-next-20260306) — confirmed
IB_EVENT_PORT_ERR (type=10) and IB_EVENT_PORT_ACTIVE (type=9) are delivered
to userspace during service reset, and testpmd tears down cleanly afterwards.
Long Li (8):
RDMA/mana_ib: Track ucontext per device
RDMA/mana_ib: Track PD per ucontext
RDMA/mana_ib: Track CQ per ucontext
RDMA/mana_ib: Track WQ per ucontext
RDMA/mana_ib: Track QP per ucontext
RDMA/mana_ib: Track MR per ucontext
RDMA/mana_ib: Notify service reset events to RDMA devices
RDMA/mana_ib: Skip firmware commands for invalidated handles
drivers/infiniband/hw/mana/cq.c | 44 +++++--
drivers/infiniband/hw/mana/device.c | 105 ++++++++++++++++++
drivers/infiniband/hw/mana/main.c | 56 +++++++++-
drivers/infiniband/hw/mana/mana_ib.h | 19 ++++
drivers/infiniband/hw/mana/mr.c | 33 +++++-
drivers/infiniband/hw/mana/qp.c | 61 +++++++---
drivers/infiniband/hw/mana/wq.c | 24 ++++
drivers/net/ethernet/microsoft/mana/mana_en.c | 14 ++-
include/net/mana/gdma.h | 6 +
9 files changed, 331 insertions(+), 31 deletions(-)
--
2.43.0
reply other threads:[~2026-03-07 1:44 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260307014414.556256-1-longli@microsoft.com \
--to=longli@microsoft.com \
--cc=davem@davemloft.net \
--cc=decui@microsoft.com \
--cc=edumazet@google.com \
--cc=erick.archer@outlook.com \
--cc=haiyangz@microsoft.com \
--cc=horms@kernel.org \
--cc=kotaranov@microsoft.com \
--cc=kuba@kernel.org \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=schakrabarti@linux.microsoft.com \
--cc=shradhagupta@linux.microsoft.com \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox