* Re: (subset) [PATCH rdma-next 0/2] RDMA: Small batch of cleanups
From: Leon Romanovsky @ 2026-07-14 6:53 UTC (permalink / raw)
To: Selvin Xavier, Kalesh AP, Jason Gunthorpe, Michael Margolin,
Gal Pressman, Yossi Leybovich, Cheng Xu, Kai Shen,
Chengchang Tang, Junxian Huang, Tatyana Nikolova, Long Li,
Konstantin Taranov, Yishai Hadas, Michal Kalderon, Nelson Escobar,
Satish Kharat, Bernard Metzler, Leon Romanovsky
Cc: linux-rdma, linux-kernel, linux-hyperv
In-Reply-To: <20260713-fix-destroy-no-udata-v1-0-fcca2e34fd57@nvidia.com>
On Mon, 13 Jul 2026 11:10:33 +0300, Leon Romanovsky wrote:
> This series contains two independent cleanups. One fixes the problematic placemen
> of a newly introduced in-kernel API, which should be called at the beginning of
> destroy functions and not at the end. The other removes a redundant memset().
>
> Thanks.
>
>
> [...]
Applied, thanks!
[2/2] RDMA: Remove redundant memset() from query_device callbacks
https://git.kernel.org/rdma/rdma/c/eeb9697db6c16d
Best regards,
--
Leon Romanovsky <leon@kernel.org>
^ permalink raw reply
* Re: [PATCH rdma-next 0/2] RDMA: Small batch of cleanups
From: Leon Romanovsky @ 2026-07-14 6:54 UTC (permalink / raw)
To: Selvin Xavier, Kalesh AP, Jason Gunthorpe, Michael Margolin,
Gal Pressman, Yossi Leybovich, Cheng Xu, Kai Shen,
Chengchang Tang, Junxian Huang, Tatyana Nikolova, Long Li,
Konstantin Taranov, Yishai Hadas, Michal Kalderon, Nelson Escobar,
Satish Kharat, Bernard Metzler
Cc: linux-rdma, linux-kernel, linux-hyperv
In-Reply-To: <20260713-fix-destroy-no-udata-v1-0-fcca2e34fd57@nvidia.com>
On Mon, Jul 13, 2026 at 11:10:33AM +0300, Leon Romanovsky wrote:
> This series contains two independent cleanups. One fixes the problematic placemen
> of a newly introduced in-kernel API, which should be called at the beginning of
> destroy functions and not at the end. The other removes a redundant memset().
>
> Thanks.
>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
> Leon Romanovsky (2):
> RDMA/bnxt_re: Validate udata before destroying resources
I'll resend this one.
> RDMA: Remove redundant memset() from query_device callbacks
And applied this.
^ permalink raw reply
* [PATCH rdma-next] RDMA: Use ib_no_udata_io() in query_device callbacks
From: Leon Romanovsky @ 2026-07-14 11:46 UTC (permalink / raw)
To: Potnuri Bharat Teja, Jason Gunthorpe, Cheng Xu, Kai Shen,
Chengchang Tang, Junxian Huang, Abhijit Gangurde, Allen Hubbe,
Tatyana Nikolova, Long Li, Konstantin Taranov, Selvin Xavier,
Michal Kalderon, Nelson Escobar, Satish Kharat, Bryan Tan,
Vishnu Dasa, Broadcom internal kernel review list,
Dennis Dalessandro, Zhu Yanjun, Bernard Metzler
Cc: linux-rdma, linux-kernel, linux-hyperv
From: Leon Romanovsky <leonro@nvidia.com>
The query_device callbacks that neither accept driver-specific input nor
return a driver-specific response open-code the empty udata handling as
ib_is_udata_in_empty() on entry and ib_respond_empty_udata() on exit.
ib_no_udata_io() already combines both steps, so replace the entry check
with it and simply return 0 on success.
Unlike the create and destroy flows, query_device owns no uobject or HW
resource - the extended path fills a stack ib_device_attr that the core
discards on error - so clearing the empty response buffer on entry rather
than on exit is a mechanical change with no functional difference.
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
drivers/infiniband/hw/cxgb4/provider.c | 4 ++--
drivers/infiniband/hw/erdma/erdma_verbs.c | 4 ++--
drivers/infiniband/hw/hns/hns_roce_main.c | 4 ++--
drivers/infiniband/hw/ionic/ionic_ibdev.c | 4 ++--
drivers/infiniband/hw/irdma/verbs.c | 4 ++--
drivers/infiniband/hw/mana/main.c | 4 ++--
drivers/infiniband/hw/mthca/mthca_provider.c | 3 +--
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 4 ++--
drivers/infiniband/hw/qedr/verbs.c | 4 ++--
drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 4 ++--
drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c | 4 ++--
drivers/infiniband/sw/rdmavt/vt.c | 4 ++--
drivers/infiniband/sw/rxe/rxe_verbs.c | 4 ++--
drivers/infiniband/sw/siw/siw_verbs.c | 4 ++--
14 files changed, 27 insertions(+), 28 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index e1eec37ee822..ebe3170a641c 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -263,7 +263,7 @@ static int c4iw_query_device(struct ib_device *ibdev, struct ib_device_attr *pro
pr_debug("ibdev %p\n", ibdev);
- err = ib_is_udata_in_empty(uhw);
+ err = ib_no_udata_io(uhw);
if (err)
return err;
@@ -300,7 +300,7 @@ static int c4iw_query_device(struct ib_device *ibdev, struct ib_device_attr *pro
props->max_fast_reg_page_list_len =
t4_max_fr_depth(dev->rdev.lldi.ulptx_memwrite_dsgl && use_dsgl);
- return ib_respond_empty_udata(uhw);
+ return 0;
}
static int c4iw_query_port(struct ib_device *ibdev, u32 port,
diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.c b/drivers/infiniband/hw/erdma/erdma_verbs.c
index 9491cbab69b3..71e3e8618a61 100644
--- a/drivers/infiniband/hw/erdma/erdma_verbs.c
+++ b/drivers/infiniband/hw/erdma/erdma_verbs.c
@@ -320,7 +320,7 @@ int erdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr,
struct erdma_dev *dev = to_edev(ibdev);
int err;
- err = ib_is_udata_in_empty(udata);
+ err = ib_no_udata_io(udata);
if (err)
return err;
@@ -361,7 +361,7 @@ int erdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr,
addrconf_addr_eui48((u8 *)&attr->sys_image_guid,
dev->netdev->dev_addr);
- return ib_respond_empty_udata(udata);
+ return 0;
}
int erdma_query_gid(struct ib_device *ibdev, u32 port, int idx,
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index 09c07de5f022..662959efcf31 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -223,7 +223,7 @@ static int hns_roce_query_device(struct ib_device *ib_dev,
struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
int ret;
- ret = ib_is_udata_in_empty(uhw);
+ ret = ib_no_udata_io(uhw);
if (ret)
return ret;
@@ -277,7 +277,7 @@ static int hns_roce_query_device(struct ib_device *ib_dev,
if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_XRC)
props->device_cap_flags |= IB_DEVICE_XRC;
- return ib_respond_empty_udata(uhw);
+ return 0;
}
static int hns_roce_query_port(struct ib_device *ib_dev, u32 port_num,
diff --git a/drivers/infiniband/hw/ionic/ionic_ibdev.c b/drivers/infiniband/hw/ionic/ionic_ibdev.c
index b0449c75f893..2b91bd03c73e 100644
--- a/drivers/infiniband/hw/ionic/ionic_ibdev.c
+++ b/drivers/infiniband/hw/ionic/ionic_ibdev.c
@@ -27,7 +27,7 @@ static int ionic_query_device(struct ib_device *ibdev,
struct net_device *ndev;
int err;
- err = ib_is_udata_in_empty(udata);
+ err = ib_no_udata_io(udata);
if (err)
return err;
@@ -74,7 +74,7 @@ static int ionic_query_device(struct ib_device *ibdev,
attr->max_fast_reg_page_list_len = dev->lif_cfg.npts_per_lif / 2;
attr->max_pkeys = IONIC_PKEY_TBL_LEN;
- return ib_respond_empty_udata(udata);
+ return 0;
}
static int ionic_query_port(struct ib_device *ibdev, u32 port,
diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
index c9b606cc67d4..f48cc8b5299b 100644
--- a/drivers/infiniband/hw/irdma/verbs.c
+++ b/drivers/infiniband/hw/irdma/verbs.c
@@ -18,7 +18,7 @@ static int irdma_query_device(struct ib_device *ibdev,
struct irdma_hw_attrs *hw_attrs = &rf->sc_dev.hw_attrs;
int err;
- err = ib_is_udata_in_empty(udata);
+ err = ib_no_udata_io(udata);
if (err)
return err;
@@ -75,7 +75,7 @@ static int irdma_query_device(struct ib_device *ibdev,
if (hw_attrs->uk_attrs.hw_rev >= IRDMA_GEN_3)
props->device_cap_flags |= IB_DEVICE_MEM_WINDOW_TYPE_2B;
- return ib_respond_empty_udata(udata);
+ return 0;
}
/**
diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index a5b3606a1dd5..73f4bfb22b5e 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -575,7 +575,7 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
struct pci_dev *pdev = to_pci_dev(mdev_to_gc(dev)->dev);
int err;
- err = ib_is_udata_in_empty(uhw);
+ err = ib_no_udata_io(uhw);
if (err)
return err;
@@ -604,7 +604,7 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
if (!mana_ib_is_rnic(dev))
props->raw_packet_caps = IB_RAW_PACKET_CAP_IP_CSUM;
- return ib_respond_empty_udata(uhw);
+ return 0;
}
int mana_ib_query_port(struct ib_device *ibdev, u32 port,
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
index e933a53779a4..6575493f4eab 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.c
+++ b/drivers/infiniband/hw/mthca/mthca_provider.c
@@ -58,7 +58,7 @@ static int mthca_query_device(struct ib_device *ibdev, struct ib_device_attr *pr
int err;
struct mthca_dev *mdev = to_mdev(ibdev);
- err = ib_is_udata_in_empty(uhw);
+ err = ib_no_udata_io(uhw);
if (err)
return err;
@@ -112,7 +112,6 @@ static int mthca_query_device(struct ib_device *ibdev, struct ib_device_attr *pr
props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
props->max_mcast_grp;
- err = ib_respond_empty_udata(uhw);
out:
kfree(in_mad);
kfree(out_mad);
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index de83dc0ea79c..cfe3d19b73b3 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -70,7 +70,7 @@ int ocrdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr,
struct ocrdma_dev *dev = get_ocrdma_dev(ibdev);
int err;
- err = ib_is_udata_in_empty(uhw);
+ err = ib_no_udata_io(uhw);
if (err)
return err;
@@ -111,7 +111,7 @@ int ocrdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr,
attr->local_ca_ack_delay = dev->attr.local_ca_ack_delay;
attr->max_fast_reg_page_list_len = dev->attr.max_pages_per_frmr;
attr->max_pkeys = 1;
- return ib_respond_empty_udata(uhw);
+ return 0;
}
static inline void get_link_speed_and_width(struct ocrdma_dev *dev,
diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index 1cf502579ad1..012a0ab98d6b 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -114,7 +114,7 @@ int qedr_query_device(struct ib_device *ibdev,
return -EINVAL;
}
- rc = ib_is_udata_in_empty(udata);
+ rc = ib_no_udata_io(udata);
if (rc)
return rc;
@@ -158,7 +158,7 @@ int qedr_query_device(struct ib_device *ibdev,
attr->max_pkeys = qattr->max_pkey;
attr->max_ah = qattr->max_ah;
- return ib_respond_empty_udata(udata);
+ return 0;
}
static inline void get_link_speed_and_width(int speed, u16 *ib_speed,
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
index 3402c84eb904..1a1647d0e345 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
@@ -278,7 +278,7 @@ int usnic_ib_query_device(struct ib_device *ibdev,
int err;
usnic_dbg("\n");
- err = ib_is_udata_in_empty(uhw);
+ err = ib_no_udata_io(uhw);
if (err)
return err;
@@ -323,7 +323,7 @@ int usnic_ib_query_device(struct ib_device *ibdev,
* max_qp_wr, max_sge, max_sge_rd, max_cqe */
mutex_unlock(&us_ibdev->usdev_lock);
- return ib_respond_empty_udata(uhw);
+ return 0;
}
int usnic_ib_query_port(struct ib_device *ibdev, u32 port,
diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c
index 1d29a535f76a..59c2a88c158a 100644
--- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c
+++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c
@@ -69,7 +69,7 @@ int pvrdma_query_device(struct ib_device *ibdev,
struct pvrdma_dev *dev = to_vdev(ibdev);
int err;
- err = ib_is_udata_in_empty(uhw);
+ err = ib_no_udata_io(uhw);
if (err)
return err;
@@ -116,7 +116,7 @@ int pvrdma_query_device(struct ib_device *ibdev,
props->device_cap_flags |= IB_DEVICE_PORT_ACTIVE_EVENT |
IB_DEVICE_RC_RNR_NAK_GEN;
- return ib_respond_empty_udata(uhw);
+ return 0;
}
/**
diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c
index f37d6d64adb9..1c112f4dc994 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -84,14 +84,14 @@ static int rvt_query_device(struct ib_device *ibdev,
struct rvt_dev_info *rdi = ib_to_rvt(ibdev);
int err;
- err = ib_is_udata_in_empty(uhw);
+ err = ib_no_udata_io(uhw);
if (err)
return err;
/*
* Return rvt_dev_info.dparms.props contents
*/
*props = rdi->dparms.props;
- return ib_respond_empty_udata(uhw);
+ return 0;
}
static int rvt_get_numa_node(struct ib_device *ibdev)
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 1ec130fee8ea..c8562866e21f 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -22,13 +22,13 @@ static int rxe_query_device(struct ib_device *ibdev,
struct rxe_dev *rxe = to_rdev(ibdev);
int err;
- err = ib_is_udata_in_empty(udata);
+ err = ib_no_udata_io(udata);
if (err)
return err;
memcpy(attr, &rxe->attr, sizeof(*attr));
- return ib_respond_empty_udata(udata);
+ return 0;
}
static int rxe_query_port(struct ib_device *ibdev,
diff --git a/drivers/infiniband/sw/siw/siw_verbs.c b/drivers/infiniband/sw/siw/siw_verbs.c
index e281517fc6f9..4ba6d6d5afe9 100644
--- a/drivers/infiniband/sw/siw/siw_verbs.c
+++ b/drivers/infiniband/sw/siw/siw_verbs.c
@@ -132,7 +132,7 @@ int siw_query_device(struct ib_device *base_dev, struct ib_device_attr *attr,
struct siw_device *sdev = to_siw_dev(base_dev);
int rv;
- rv = ib_is_udata_in_empty(udata);
+ rv = ib_no_udata_io(udata);
if (rv)
return rv;
@@ -165,7 +165,7 @@ int siw_query_device(struct ib_device *base_dev, struct ib_device_attr *attr,
addrconf_addr_eui48((u8 *)&attr->sys_image_guid,
sdev->raw_gid);
- return ib_respond_empty_udata(udata);
+ return 0;
}
int siw_query_port(struct ib_device *base_dev, u32 port,
---
base-commit: eeb9697db6c16d9bb2ce7b7ddf95aa20305aa9f2
change-id: 20260714-convert-to-noio-udata-28749ec5a250
Best regards,
--
Leon Romanovsky <leonro@nvidia.com>
^ permalink raw reply related
* RE: [PATCH v2 3/4] iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest
From: Michael Kelley @ 2026-07-14 15:29 UTC (permalink / raw)
To: Yu Zhang, Michael Kelley
Cc: linux-kernel@vger.kernel.org, linux-hyperv@vger.kernel.org,
iommu@lists.linux.dev, linux-pci@vger.kernel.org,
linux-arch@vger.kernel.org, wei.liu@kernel.org, kys@microsoft.com,
haiyangz@microsoft.com, decui@microsoft.com, longli@microsoft.com,
joro@8bytes.org, will@kernel.org, robin.murphy@arm.com,
bhelgaas@google.com, kwilczynski@kernel.org,
lpieralisi@kernel.org, mani@kernel.org, robh@kernel.org,
arnd@arndb.de, jgg@ziepe.ca, jacob.pan@linux.microsoft.com,
tgopinath@linux.microsoft.com,
easwar.hariharan@linux.microsoft.com, mrathor@linux.microsoft.com
In-Reply-To: <r35cz2l3yz2wyi5d663mxwqoiz2rrl6kzpiw6rsruh7ofk6vu6@temnhcfsp6fa>
From: Yu Zhang <zhangyu1@linux.microsoft.com> Sent: Monday, July 13, 2026 8:34 PM
>
> On Mon, Jul 13, 2026 at 05:37:51PM +0000, Michael Kelley wrote:
> > From: Yu Zhang <zhangyu1@linux.microsoft.com> Sent: Monday, July 13, 2026 9:46 AM
> > >
> > > On Sat, Jul 11, 2026 at 06:31:15PM +0000, Michael Kelley wrote:
> > > > From: Yu Zhang <zhangyu1@linux.microsoft.com> Sent: Friday, July 10, 2026 12:34 AM
> >
> > [snip]
> >
> > > >
> > > > One new thought: Have you considered the hibernate/resume
> > > > cycle? Does anything need to be done with the pvIOMMU to
> > > > make it functional again after resume? I see that the Intel and
> > > > AMD IOMMU drivers have suspend and resume functions. I
> > > > don't know enough about the Hyper-V pvIOMMU to know if it
> > > > might also need suspend and resume functions.
> > > >
> > >
> > > Thanks for raising this, Michael. We have not considered such support.
> > >
> > > My understanding is that the Intel and AMD drivers only disable the
> > > IOMMU translation, flush the IOTLB during the suspend and re-enable/
> > > reload the preserved root tables and other HW state during in the
> > > resueme.
> > >
> > > But for pvIOMMU, I guess such job shall be done by the hypervisor?
> > > For a device resumed on the same VM, its logical device ID should
> > > also remain unchanged? And the corresponding Hyper-V domain objects,
> > > configuration, and device attachments shall be preserved and restored
> > > by hypervisor? I don't think the current Hyper-V ABI explicitly defines
> > > this. But maybe if we want such feature, it could be done by the
> > > hypervisor transparently?
> > >
> >
> > I agree with your and Jacob's comments that the guest doesn't have
> > any responsibility for saving/restoring IOMMU hardware state, as the
> > Intel and AMD IOMMU drivers do.
> >
> > But yes, I'm wondering about the Hyper-V domain objects and device
> > attachments. I doubt Hyper-V can do anything to save and restore
> > them. Hibernation is a Linux concept that the Hyper-V host doesn't
> > know anything about.
> >
> > Hibernation is already complicated, and in a VM it is even worse. :-(
> > As a start, see Documentation/virt/hyperv/hibernation.rst, which I
> > wrote about 18 months ago. It provides some basics as well as outlines
> > the additional complexity in a Hyper-V guest VM. I'll also try to spend
> > some time thinking through the implications for a pvIOMMU, and let
> > you know if I have any more thoughts.
> >
>
> Thank you, Michael, and thanks for pointing us to the documentation. I
> need some time to better understand the Linux guest hibernation and resume
> flow and its implications for pvIOMMU.
>
> Meanwhile, do you think this limitation should be documented in the
> commit message or the cover letter?
>
Yes, I'd say that the lack of hibernation/resume support should be noted in
a commit message, probably with Patch 3 of the series.
Michael
^ permalink raw reply
* Re: [PATCH v8 0/8] mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings
From: Stanislav Kinsburskii @ 2026-07-14 16:09 UTC (permalink / raw)
To: Andrew Morton
Cc: airlied, akhilesh, corbet, dakr, david, decui, haiyangz, jgg,
kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, surenb, tzimmermann, vbabka,
wei.liu, dri-devel, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <20260713154535.7656b3a630e2f6f076b4e76e@linux-foundation.org>
On Mon, Jul 13, 2026 at 03:45:35PM -0700, Andrew Morton wrote:
> On Mon, 13 Jul 2026 13:57:55 -0700 Stanislav Kinsburskii <skinsburskii@gmail.com> wrote:
>
> > > > I rebased this series on top of mm-new right before sending it out.
> > > > Should I have used a different branch?
> > >
> > > mm-new is good - Sashiko attempts that. But it's changing rapidly at
> > > this point in the development cycle.
> > >
> >
> > I’d like to send another revision addressing a few comments and also
> > replace the `max/max_t` check with something simpler.
> >
> > Which branch should I base it on so that Sashiko can apply it
> > successfully?
>
> mainline Linus would be safest.
>
Looks like linux-next/master has been updated with the v8 of the series.
I have v9 with a few small fixes, but it is too late to send it out already?
If it's not, then what should I base it on?
Thanks,
Stanislav
> > Or would it be better to send fixups against `mm-new`?
>
> That should work as well, but it doesn't give us a Sashiko scan of the
> whole patchset.
^ permalink raw reply
* Re: [PATCH v8 0/8] mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings
From: Andrew Morton @ 2026-07-14 17:57 UTC (permalink / raw)
To: Stanislav Kinsburskii
Cc: airlied, akhilesh, corbet, dakr, david, decui, haiyangz, jgg,
kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, surenb, tzimmermann, vbabka,
wei.liu, dri-devel, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <alZfTxfypj39OrCE@skinsburskii>
On Tue, 14 Jul 2026 09:09:51 -0700 Stanislav Kinsburskii <skinsburskii@gmail.com> wrote:
> On Mon, Jul 13, 2026 at 03:45:35PM -0700, Andrew Morton wrote:
> > On Mon, 13 Jul 2026 13:57:55 -0700 Stanislav Kinsburskii <skinsburskii@gmail.com> wrote:
> >
> > > > > I rebased this series on top of mm-new right before sending it out.
> > > > > Should I have used a different branch?
> > > >
> > > > mm-new is good - Sashiko attempts that. But it's changing rapidly at
> > > > this point in the development cycle.
> > > >
> > >
> > > I’d like to send another revision addressing a few comments and also
> > > replace the `max/max_t` check with something simpler.
> > >
> > > Which branch should I base it on so that Sashiko can apply it
> > > successfully?
> >
> > mainline Linus would be safest.
> >
>
> Looks like linux-next/master has been updated with the v8 of the series.
That's because v8 is in mm.git's mm-unstable branch.
> I have v9 with a few small fixes, but it is too late to send it out already?
It's called "unstable" for a reason! Material in mm-unstable is still
under review, test and the latest stages of development. Getting
things finalized for movement into the non-rebasing mm-stable branch,
then into mainline.
So altering or replacing patchsets while they're in mm-unstable is
perfectly OK and expected.
> If it's not, then what should I base it on?
Well it's a bit tricky to replace a series when it's in mm-unstable.
One can do a git-checkout of the commit which precedes the v8 series.
Or base on current Linus mainline, which usually works out.
Sending little fixup patches against what's presently in mm-unstable
also works. I'll queue each one immediately behind the patch which it
alters then squash them into their parent patch before moving the series
into mm-stable.
A third alternative is for me to drop v8 from mm-unstable, then you
wait until that has propagated onto the servers or into linux-next,
then base on that. This approach is OK but I kinda unprefer it because
there's a bit of latency and it makes it harder for me to prepare my
"here's how v9 altered mm.git" summaries.
Which would you prefer?
^ permalink raw reply
* Re: [PATCH v8 0/8] mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings
From: Stanislav Kinsburskii @ 2026-07-14 20:05 UTC (permalink / raw)
To: Andrew Morton
Cc: airlied, akhilesh, corbet, dakr, david, decui, haiyangz, jgg,
kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, surenb, tzimmermann, vbabka,
wei.liu, dri-devel, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <20260714105751.f45ec4c70702c222febdbf07@linux-foundation.org>
On Tue, Jul 14, 2026 at 10:57:51AM -0700, Andrew Morton wrote:
> On Tue, 14 Jul 2026 09:09:51 -0700 Stanislav Kinsburskii <skinsburskii@gmail.com> wrote:
>
> > On Mon, Jul 13, 2026 at 03:45:35PM -0700, Andrew Morton wrote:
> > > On Mon, 13 Jul 2026 13:57:55 -0700 Stanislav Kinsburskii <skinsburskii@gmail.com> wrote:
> > >
> > > > > > I rebased this series on top of mm-new right before sending it out.
> > > > > > Should I have used a different branch?
> > > > >
> > > > > mm-new is good - Sashiko attempts that. But it's changing rapidly at
> > > > > this point in the development cycle.
> > > > >
> > > >
> > > > I’d like to send another revision addressing a few comments and also
> > > > replace the `max/max_t` check with something simpler.
> > > >
> > > > Which branch should I base it on so that Sashiko can apply it
> > > > successfully?
> > >
> > > mainline Linus would be safest.
> > >
> >
> > Looks like linux-next/master has been updated with the v8 of the series.
>
> That's because v8 is in mm.git's mm-unstable branch.
>
> > I have v9 with a few small fixes, but it is too late to send it out already?
>
> It's called "unstable" for a reason! Material in mm-unstable is still
> under review, test and the latest stages of development. Getting
> things finalized for movement into the non-rebasing mm-stable branch,
> then into mainline.
>
> So altering or replacing patchsets while they're in mm-unstable is
> perfectly OK and expected.
>
> > If it's not, then what should I base it on?
>
> Well it's a bit tricky to replace a series when it's in mm-unstable.
> One can do a git-checkout of the commit which precedes the v8 series.
> Or base on current Linus mainline, which usually works out.
>
> Sending little fixup patches against what's presently in mm-unstable
> also works. I'll queue each one immediately behind the patch which it
> alters then squash them into their parent patch before moving the series
> into mm-stable.
>
> A third alternative is for me to drop v8 from mm-unstable, then you
> wait until that has propagated onto the servers or into linux-next,
> then base on that. This approach is OK but I kinda unprefer it because
> there's a bit of latency and it makes it harder for me to prepare my
> "here's how v9 altered mm.git" summaries.
>
>
> Which would you prefer?
Well, given that Sashiko didn’t pick up my series based on mm-new, I’d
prefer to send a few follow-up patches in the hope that they will be
reviewed in the context of the original series.
Thanks,
Stanislav
^ permalink raw reply
* [PATCH] Drivers: hv: vmbus: add VTL2 redirect connection ID
From: Hardik Garg @ 2026-07-14 21:38 UTC (permalink / raw)
To: K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
linux-hyperv, linux-kernel
VMBus sends CHANNELMSG_INITIATE_CONTACT through a Hyper-V message
connection ID. Older protocol versions use VMBUS_MESSAGE_CONNECTION_ID,
while protocol version 5.0 and newer normally use
VMBUS_MESSAGE_CONNECTION_ID_4.
When the Linux VMBus driver runs at VTL2, the VMBus control plane may
be reached through a VMBus relay instead of the standard host endpoint.
In that setup the relay listens on the redirect message connection ID,
and an Initiate Contact message sent to the standard ID is not delivered
to the control plane.
The connection ID selects the Hyper-V message port used to reach the
VMBus control plane. If Linux uses the wrong port, the host does not
receive the Initiate Contact message.
For Linux running at VTL2 with VMBus protocol 5.0 or newer, try the
redirect connection ID first. In VTL2, the redirect connection ID is the
VMBus relay endpoint. If the relay is present, it accepts Initiate
Contact and completes the normal version-response handshake. Systems
without the relay reject the redirect ID synchronously with
HV_STATUS_INVALID_CONNECTION_ID, allowing fallback to
VMBUS_MESSAGE_CONNECTION_ID_4.
Return a distinct error for an invalid Initiate Contact connection ID so
this fallback does not mask other post-message failures or
protocol-version rejections. For older protocol versions, and for Linux
below VTL2, keep the existing connection ID selection.
Signed-off-by: Hardik Garg <hargar@linux.microsoft.com>
---
drivers/hv/connection.c | 76 ++++++++++++++++++++++++++++++++++++++++-------
drivers/hv/hyperv_vmbus.h | 2 ++
2 files changed, 67 insertions(+), 11 deletions(-)
diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 1fe3573ae52a4..eb871f87a819d 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -71,7 +71,19 @@ module_param(max_version, uint, S_IRUGO);
MODULE_PARM_DESC(max_version,
"Maximal VMBus protocol version which can be negotiated");
-int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
+/* Connection IDs to try for VTL2 VMBus protocol 5.0 and newer. */
+static const u32 connection_ids[] = {
+ VMBUS_MESSAGE_CONNECTION_ID_REDIRECT,
+ VMBUS_MESSAGE_CONNECTION_ID_4,
+};
+
+/*
+ * Send one CHANNELMSG_INITIATE_CONTACT attempt.
+ * The caller supplies the message connection ID and owns retry/fallback
+ * policy.
+ */
+static int vmbus_try_connection_id(struct vmbus_channel_msginfo *msginfo,
+ u32 version, u32 connection_id)
{
int ret = 0;
struct vmbus_channel_initiate_contact *msg;
@@ -87,7 +99,7 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
/*
* VMBus protocol 5.0 (VERSION_WIN10_V5) and higher require that we must
- * use VMBUS_MESSAGE_CONNECTION_ID_4 for the Initiate Contact Message,
+ * use connection_id for the Initiate Contact Message,
* and for subsequent messages, we must use the Message Connection ID
* field in the host-returned Version Response Message. And, with
* VERSION_WIN10_V5 and higher, we don't use msg->interrupt_page, but we
@@ -99,7 +111,7 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
if (version >= VERSION_WIN10_V5) {
msg->msg_sint = VMBUS_MESSAGE_SINT;
msg->msg_vtl = ms_hyperv.vtl;
- vmbus_connection.msg_conn_id = VMBUS_MESSAGE_CONNECTION_ID_4;
+ vmbus_connection.msg_conn_id = connection_id;
} else {
msg->interrupt_page = virt_to_phys(vmbus_connection.int_page);
vmbus_connection.msg_conn_id = VMBUS_MESSAGE_CONNECTION_ID;
@@ -161,6 +173,51 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
return ret;
}
+/*
+ * Negotiate the given VMBus protocol version with the host.
+ * Protocol-specific connection ID policy is handled here so the single-try
+ * helper stays simple.
+ */
+int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
+{
+ int ret;
+ size_t j;
+
+ /*
+ * The redirect ID is not a speculative endpoint. If the VMBus relay is
+ * present, it accepts INITIATE_CONTACT and completes the normal version
+ * response. Systems without the relay reject the ID synchronously, so
+ * negotiation falls back to VMBUS_MESSAGE_CONNECTION_ID_4.
+ */
+ if (version >= VERSION_WIN10_V5 && ms_hyperv.vtl >= 2) {
+ for (j = 0; j < ARRAY_SIZE(connection_ids); j++) {
+ ret = vmbus_try_connection_id(msginfo, version,
+ connection_ids[j]);
+ if (vmbus_connection.conn_state == CONNECTED)
+ return 0;
+
+ if (ret == -ETIMEDOUT)
+ return ret;
+
+ if (connection_ids[j] ==
+ VMBUS_MESSAGE_CONNECTION_ID_REDIRECT &&
+ ret == -ENXIO)
+ continue;
+
+ return ret;
+ }
+ return ret;
+ }
+
+ /*
+ * Non-redirect path. Protocol 5.0+ below VTL2 uses the standard
+ * v5 connection ID; pre-v5 ignores the supplied ID and uses the
+ * legacy connection ID.
+ */
+ return vmbus_try_connection_id(msginfo, version,
+ VMBUS_MESSAGE_CONNECTION_ID_4);
+}
+
/*
* vmbus_connect - Sends a connect request on the partition service connection
*/
@@ -455,17 +507,14 @@ int vmbus_post_msg(void *buffer, size_t buflen, bool can_sleep)
switch (ret) {
case HV_STATUS_INVALID_CONNECTION_ID:
/*
- * See vmbus_negotiate_version(): VMBus protocol 5.0
- * and higher require that we must use
- * VMBUS_MESSAGE_CONNECTION_ID_4 for the Initiate
- * Contact message, but on old hosts that only
- * support VMBus protocol 4.0 or lower, here we get
- * HV_STATUS_INVALID_CONNECTION_ID and we should
- * return an error immediately without retrying.
+ * Let negotiation distinguish an unusable
+ * endpoint from other failures.
+ *
+ * Other messages keep retry behavior.
*/
hdr = buffer;
if (hdr->msgtype == CHANNELMSG_INITIATE_CONTACT)
- return -EINVAL;
+ return -ENXIO;
/*
* We could get this if we send messages too
* frequently.
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 49a72a4f3f6a7..951bdebdc0526 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -109,6 +109,8 @@ struct hv_input_post_message {
enum {
VMBUS_MESSAGE_CONNECTION_ID = 1,
VMBUS_MESSAGE_CONNECTION_ID_4 = 4,
+ /* VMBus relay port used for INITIATE_CONTACT probing. */
+ VMBUS_MESSAGE_CONNECTION_ID_REDIRECT = 0x800074,
VMBUS_MESSAGE_PORT_ID = 1,
VMBUS_EVENT_CONNECTION_ID = 2,
VMBUS_EVENT_PORT_ID = 2,
--
2.34.1
^ permalink raw reply related
* [PATCH net-next 0/7] net: mana: harden the HWC and add dynamic queue depth
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
This series hardens the MANA Hardware Channel (HWC) control-plane path
and then builds on that to support a dynamic HWC queue depth.
The HWC is the command channel the driver uses to talk to the device.
Today it is created at a fixed depth of one outstanding request, and
several of its lookup and teardown paths predate the RCU and DMA-lifetime
rules they now need to follow. Raising the queue depth and allowing
concurrent commands makes those latent races reachable, so the fixes come
first and the feature builds on them.
Patches 1-5 are fixes for pre-existing HWC bugs, each with a Fixes: tag:
1: cq_table was a plain pointer array freed with no grace period while
the EQ interrupt handler dereferenced it; put it under RCU.
2: the HWC RQ and SQ were sized with each other's message size, so a
response could overflow the RQ buffer and the RX slot stride was
computed with the wrong size.
3: comp_buf was freed before the EQ was destroyed, so a late completion
handler could touch freed memory.
4: the RX path consumed device-supplied lengths and indices without
validation; validate them before use (this matters for confidential
VMs, where the DMA buffer is shared with the host).
5: a failed mana_hwc_establish_channel() could leave live MST entries
while the driver freed the queue buffers, and destroy_channel() freed
the TXQ/RXQ before the EQ was quiesced; add a setup_active teardown
gate and destroy the CQ first.
Patches 6-7 add the feature:
6: replace the depth-1 semaphore with a slot bitmap and per-slot
synchronization so several management commands can be in flight,
with teardown that drains in-flight senders before freeing the HWC.
7: bootstrap the HWC at depth 1, query the device maximum and, if it is
larger, tear down and rebuild the queues at that depth. The reported
dimensions are validated before they size DMA allocations, and the
capability is advertised so firmware enables it only when the driver
supports it.
The fixes are grouped ahead of the feature they enable rather than sent
separately to net, since the HWC runs at depth 1 today and the races are
reached only once the later patches raise the depth.
Long Li (7):
net: mana: RCU-protect gc->cq_table lookups against concurrent CQ
destroy
net: mana: fix HWC RQ/SQ buffer size swap
net: mana: free HWC comp_buf after destroying the EQ
net: mana: validate hardware-supplied values in the HWC RX path
net: mana: fix HWC teardown safety with setup_active flag and destroy
ordering
net: mana: support concurrent HWC requests with proper synchronization
net: mana: add dynamic HWC queue depth with reinit path
drivers/infiniband/hw/mana/cq.c | 46 +-
.../net/ethernet/microsoft/mana/gdma_main.c | 78 +-
.../net/ethernet/microsoft/mana/hw_channel.c | 707 ++++++++++++++++--
drivers/net/ethernet/microsoft/mana/mana_en.c | 22 +-
include/net/mana/gdma.h | 48 +-
include/net/mana/hw_channel.h | 40 +-
6 files changed, 849 insertions(+), 92 deletions(-)
base-commit: f6f3b36c15ed44de1fbb44e645e4fae8c4a4453e
--
2.43.0
^ permalink raw reply
* [PATCH net-next 1/7] net: mana: RCU-protect gc->cq_table lookups against concurrent CQ destroy
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260715032942.3945317-1-longli@microsoft.com>
The EQ interrupt handler (mana_gd_process_eqe) looks up the completing CQ
in gc->cq_table[cq_id] and runs its callback, concurrently with CQ
teardown on another CPU that clears the slot and frees the CQ. cq_table
was a plain pointer array freed with no grace period, so the two race
into a use-after-free:
CPU A (mana_gd_intr, hard IRQ) CPU B (CQ destroy)
---------------------------------- ------------------------------
cq = gc->cq_table[cq_id]; // valid
gc->cq_table[id] = NULL;
kfree(cq); // freed
cq->cq.callback(ctx, cq); // use-after-free
The handler's existing rcu_read_lock() only guards the per-IRQ EQ list
traversal; cq_table was never under any RCU contract, and a read-side
lock is inert unless the freer also defers the free past a grace period.
Put cq_table under RCU: annotate the base pointer and entries __rcu, read
with rcu_dereference() in the handler, publish with rcu_assign_pointer(),
and on teardown clear the slot then synchronize_rcu() before freeing the
CQ. The grace period blocks until every in-flight handler has dropped
the old pointer, so the kfree() can no longer race the callback.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Long Li <longli@microsoft.com>
---
drivers/infiniband/hw/mana/cq.c | 46 ++++++++++++++++---
.../net/ethernet/microsoft/mana/gdma_main.c | 25 ++++++++--
.../net/ethernet/microsoft/mana/hw_channel.c | 29 ++++++++----
drivers/net/ethernet/microsoft/mana/mana_en.c | 22 +++++++--
include/net/mana/gdma.h | 21 ++++++++-
5 files changed, 119 insertions(+), 24 deletions(-)
diff --git a/drivers/infiniband/hw/mana/cq.c b/drivers/infiniband/hw/mana/cq.c
index f2547989f422..2bf4be21cede 100644
--- a/drivers/infiniband/hw/mana/cq.c
+++ b/drivers/infiniband/hw/mana/cq.c
@@ -131,12 +131,20 @@ static void mana_ib_cq_handler(void *ctx, struct gdma_queue *gdma_cq)
int mana_ib_install_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq)
{
struct gdma_context *gc = mdev_to_gc(mdev);
+ struct gdma_queue __rcu **cq_table;
struct gdma_queue *gdma_cq;
- if (cq->queue.id >= gc->max_num_cqs)
+ /* No rcu_read_lock(): install/remove run within the IB device
+ * lifetime, which mana_rdma_remove() (ib_unregister_device) drains
+ * before the base cq_table can be freed. See gdma_context::cq_table
+ * in gdma.h for why "true" is sound.
+ */
+ cq_table = rcu_dereference_protected(gc->cq_table, true);
+ if (!cq_table || cq->queue.id >= gc->max_num_cqs)
return -EINVAL;
+
/* Create CQ table entry, sharing a CQ between WQs is not supported */
- if (gc->cq_table[cq->queue.id])
+ if (rcu_access_pointer(cq_table[cq->queue.id]))
return -EINVAL;
if (cq->queue.kmem)
gdma_cq = cq->queue.kmem;
@@ -149,23 +157,49 @@ int mana_ib_install_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq)
gdma_cq->type = GDMA_CQ;
gdma_cq->cq.callback = mana_ib_cq_handler;
gdma_cq->id = cq->queue.id;
- gc->cq_table[cq->queue.id] = gdma_cq;
+ rcu_assign_pointer(cq_table[cq->queue.id], gdma_cq);
return 0;
}
void mana_ib_remove_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq)
{
struct gdma_context *gc = mdev_to_gc(mdev);
+ struct gdma_queue __rcu **cq_table;
+ struct gdma_queue *gdma_cq;
- if (cq->queue.id >= gc->max_num_cqs || cq->queue.id == INVALID_QUEUE_ID)
+ if (cq->queue.id == INVALID_QUEUE_ID)
return;
if (cq->queue.kmem)
/* Then it will be cleaned and removed by the mana */
return;
- kfree(gc->cq_table[cq->queue.id]);
- gc->cq_table[cq->queue.id] = NULL;
+ /* No rcu_read_lock(): like mana_ib_install_cq_cb(), this runs within
+ * the IB device lifetime that mana_rdma_remove() drains before the
+ * base cq_table can be freed. See gdma_context::cq_table in gdma.h.
+ */
+ cq_table = rcu_dereference_protected(gc->cq_table, true);
+ if (!cq_table || cq->queue.id >= gc->max_num_cqs)
+ return;
+ /* Removers for a given CQ are serialized by the IB core, so the slot
+ * is read and cleared without rcu_read_lock() or atomicity: a CQ is
+ * never torn down while a live QP references it (cq->usecnt), nor
+ * while the QP-create that installed the entry is still running (that
+ * create holds a reference on the CQ uobject across its error path,
+ * before usecnt is taken). Any double-remove is therefore sequential
+ * -- the later caller sees the NULL stored below and returns.
+ */
+ gdma_cq = rcu_dereference_protected(cq_table[cq->queue.id], true);
+ if (!gdma_cq)
+ return; /* already removed by a prior teardown path */
+
+ rcu_assign_pointer(cq_table[cq->queue.id], NULL);
+
+ /* Wait for in-flight EQ handlers that may have loaded the old
+ * pointer via rcu_dereference() to finish before freeing.
+ */
+ synchronize_rcu();
+ kfree(gdma_cq);
}
int mana_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index aef3b77229c1..c52ef566dc0c 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -761,6 +761,7 @@ static void mana_gd_process_eqe(struct gdma_queue *eq)
union gdma_eqe_info eqe_info;
enum gdma_eqe_type type;
struct gdma_event event;
+ struct gdma_queue __rcu **cq_table;
struct gdma_queue *cq;
struct gdma_eqe *eqe;
u32 cq_id;
@@ -772,10 +773,11 @@ static void mana_gd_process_eqe(struct gdma_queue *eq)
switch (type) {
case GDMA_EQE_COMPLETION:
cq_id = eqe->details[0] & 0xFFFFFF;
- if (WARN_ON_ONCE(cq_id >= gc->max_num_cqs))
+ cq_table = rcu_dereference(gc->cq_table);
+ if (WARN_ON_ONCE(cq_id >= gc->max_num_cqs || !cq_table))
break;
- cq = gc->cq_table[cq_id];
+ cq = rcu_dereference(cq_table[cq_id]);
if (WARN_ON_ONCE(!cq || cq->type != GDMA_CQ || cq->id != cq_id))
break;
@@ -1082,15 +1084,28 @@ static void mana_gd_create_cq(const struct gdma_queue_spec *spec,
static void mana_gd_destroy_cq(struct gdma_context *gc,
struct gdma_queue *queue)
{
+ struct gdma_queue __rcu **cq_table;
u32 id = queue->id;
- if (id >= gc->max_num_cqs)
+ /* No rcu_read_lock() here: mana_gd_destroy_cq() runs only on the
+ * CQ-destroy/teardown path, where the base cq_table is stable. See
+ * the lifecycle note on gdma_context::cq_table in gdma.h for why the
+ * "true" predicate is sound.
+ */
+ cq_table = rcu_dereference_protected(gc->cq_table, true);
+ if (!cq_table || id >= gc->max_num_cqs)
return;
- if (!gc->cq_table[id])
+ if (!rcu_access_pointer(cq_table[id]))
return;
- gc->cq_table[id] = NULL;
+ rcu_assign_pointer(cq_table[id], NULL);
+
+ /* Wait for in-flight EQ handlers that may have loaded the old
+ * pointer via rcu_dereference() to finish before the caller
+ * frees the CQ memory.
+ */
+ synchronize_rcu();
}
int mana_gd_create_hwc_queue(struct gdma_dev *gd,
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index e3c24d50dad0..409e20caeccd 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -674,6 +674,7 @@ static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_depth,
struct gdma_queue *sq = hwc->txq->gdma_wq;
struct gdma_queue *eq = hwc->cq->gdma_eq;
struct gdma_queue *cq = hwc->cq->gdma_cq;
+ struct gdma_queue __rcu **cq_table;
int err;
init_completion(&hwc->hwc_init_eqe_comp);
@@ -698,11 +699,15 @@ static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_depth,
if (WARN_ON(cq->id >= gc->max_num_cqs))
return -EPROTO;
- gc->cq_table = vcalloc(gc->max_num_cqs, sizeof(struct gdma_queue *));
- if (!gc->cq_table)
+ cq_table = vcalloc(gc->max_num_cqs, sizeof(*cq_table));
+ if (!cq_table)
return -ENOMEM;
- gc->cq_table[cq->id] = cq;
+ rcu_assign_pointer(cq_table[cq->id], cq);
+ /* Publish the fully-initialised table last; pairs with the
+ * rcu_dereference(gc->cq_table) in mana_gd_process_eqe().
+ */
+ rcu_assign_pointer(gc->cq_table, cq_table);
return 0;
}
@@ -811,6 +816,7 @@ int mana_hwc_create_channel(struct gdma_context *gc)
void mana_hwc_destroy_channel(struct gdma_context *gc)
{
struct hw_channel_context *hwc = gc->hwc.driver_data;
+ struct gdma_queue __rcu **old_cq_table;
if (!hwc)
return;
@@ -818,10 +824,8 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
/* gc->max_num_cqs is set in mana_hwc_init_event_handler(). If it's
* non-zero, the HWC worked and we should tear down the HWC here.
*/
- if (gc->max_num_cqs > 0) {
+ if (gc->max_num_cqs > 0)
mana_smc_teardown_hwc(&gc->shm_channel, false);
- gc->max_num_cqs = 0;
- }
if (hwc->txq)
mana_hwc_destroy_wq(hwc, hwc->txq);
@@ -832,6 +836,14 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
if (hwc->cq)
mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq);
+ /* Reset only after mana_hwc_destroy_cq() above has run with a valid
+ * max_num_cqs so mana_gd_destroy_cq() clears the CQ table slot and
+ * waits out in-flight EQ handlers (synchronize_rcu) before the CQ is
+ * freed. Clearing it earlier would make that path early-return and
+ * skip the slot clear, leaving a dangling cq_table entry.
+ */
+ gc->max_num_cqs = 0;
+
kfree(hwc->caller_ctx);
hwc->caller_ctx = NULL;
@@ -848,8 +860,9 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
gc->hwc.driver_data = NULL;
gc->hwc.gdma_context = NULL;
- vfree(gc->cq_table);
- gc->cq_table = NULL;
+ old_cq_table = rcu_replace_pointer(gc->cq_table, NULL, true);
+ synchronize_rcu();
+ vfree(old_cq_table);
}
int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len,
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 89e7f59f635d..05b33a1a374d 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -2637,6 +2637,7 @@ static int mana_create_txq(struct mana_port_context *apc,
struct mana_obj_spec cq_spec;
struct gdma_queue_spec spec;
struct gdma_context *gc;
+ struct gdma_queue __rcu **cq_table;
struct mana_txq *txq;
struct mana_cq *cq;
u32 txq_size;
@@ -2742,12 +2743,18 @@ static int mana_create_txq(struct mana_port_context *apc,
cq->gdma_id = cq->gdma_cq->id;
- if (WARN_ON(cq->gdma_id >= gc->max_num_cqs)) {
+ /* No rcu_read_lock(): mana_create_txq runs under RTNL during
+ * netdev bring-up, inside the netdev lifetime that
+ * mana_remove() drains before the base cq_table can be freed.
+ * See gdma_context::cq_table in gdma.h for why "true" is sound.
+ */
+ cq_table = rcu_dereference_protected(gc->cq_table, true);
+ if (WARN_ON(!cq_table || cq->gdma_id >= gc->max_num_cqs)) {
err = -EINVAL;
goto out;
}
- gc->cq_table[cq->gdma_id] = cq->gdma_cq;
+ rcu_assign_pointer(cq_table[cq->gdma_id], cq->gdma_cq);
mana_create_txq_debugfs(apc, i);
@@ -2975,6 +2982,7 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
struct gdma_queue_spec spec;
struct mana_cq *cq = NULL;
struct gdma_context *gc;
+ struct gdma_queue __rcu **cq_table;
u32 cq_size, rq_size;
struct mana_rxq *rxq;
int err;
@@ -3064,12 +3072,18 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
if (err)
goto out;
- if (WARN_ON(cq->gdma_id >= gc->max_num_cqs)) {
+ /* No rcu_read_lock(): mana_create_rxq runs under RTNL during netdev
+ * bring-up, inside the netdev lifetime that mana_remove() drains
+ * before the base cq_table can be freed. See gdma_context::cq_table
+ * in gdma.h for why "true" is sound.
+ */
+ cq_table = rcu_dereference_protected(gc->cq_table, true);
+ if (WARN_ON(!cq_table || cq->gdma_id >= gc->max_num_cqs)) {
err = -EINVAL;
goto out;
}
- gc->cq_table[cq->gdma_id] = cq->gdma_cq;
+ rcu_assign_pointer(cq_table[cq->gdma_id], cq->gdma_cq);
netif_napi_add_weight_locked(ndev, &cq->napi, mana_poll, 1);
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 8529cef0d7c4..da52701e7816 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -430,7 +430,26 @@ struct gdma_context {
/* This maps a CQ index to the queue structure. */
unsigned int max_num_cqs;
- struct gdma_queue **cq_table;
+ /* Both the base pointer and each entry are RCU-managed. The fast
+ * path (mana_gd_process_eqe) reads the base via rcu_dereference()
+ * under rcu_read_lock(), so the table is freed with
+ * rcu_assign_pointer(NULL) + synchronize_rcu() and an in-flight
+ * reader can never observe freed memory.
+ *
+ * The slow paths -- mana_gd_destroy_cq() and the CQ install/remove
+ * callers (mana_create_txq/_rxq, mana_ib_install/remove_cq_cb) --
+ * instead read the base with rcu_dereference_protected(cq_table,
+ * true). The bare "true" is justified by teardown ordering, not by
+ * a lock: the base table is replaced+freed only by
+ * mana_hwc_destroy_channel() (and the create-time reinit), and every
+ * teardown path first runs mana_remove() + mana_rdma_remove(), which
+ * synchronously drain the netdev and the IB device
+ * (unregister_netdevice / ib_unregister_device) that bound all
+ * install/remove callers; the reinit case runs before either
+ * consumer is probed. So no slow-path caller can run while the base
+ * table is being freed.
+ */
+ struct gdma_queue __rcu * __rcu *cq_table;
/* Protect eq_test_event and test_event_eq_id */
struct mutex eq_test_event_mutex;
--
2.43.0
^ permalink raw reply related
* [PATCH net-next 2/7] net: mana: fix HWC RQ/SQ buffer size swap
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260715032942.3945317-1-longli@microsoft.com>
The HWC RQ receives responses and the SQ sends requests, but
mana_hwc_init_queues() sized the RQ with max_req_msg_size and the SQ with
max_resp_msg_size -- backwards. A response larger than the undersized RQ
buffer could overflow it, and mana_hwc_rx_event_handler() recovered the
RX slot index by dividing by the wrong size (max_req_msg_size).
Size the RQ by max_resp_msg_size and the SQ by max_req_msg_size, store
max_resp_msg_size in hw_channel_context, and use it as the RX slot
stride.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Long Li <longli@microsoft.com>
---
drivers/net/ethernet/microsoft/mana/hw_channel.c | 7 ++++---
include/net/mana/hw_channel.h | 1 +
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index 409e20caeccd..3f011ebbe7b3 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -263,7 +263,7 @@ static void mana_hwc_rx_event_handler(void *ctx, u32 gdma_rxq_id,
/* Select the RX work request for virtual address and for reposting. */
rq_base_addr = hwc_rxq->msg_buf->mem_info.dma_handle;
- rx_req_idx = (sge->address - rq_base_addr) / hwc->max_req_msg_size;
+ rx_req_idx = (sge->address - rq_base_addr) / hwc->max_resp_msg_size;
if (rx_req_idx >= hwc_rxq->msg_buf->num_reqs) {
dev_err(hwc->dev, "HWC RX: wrong rx_req_idx=%llu, num_reqs=%u\n",
@@ -733,14 +733,14 @@ static int mana_hwc_init_queues(struct hw_channel_context *hwc, u16 q_depth,
goto out;
}
- err = mana_hwc_create_wq(hwc, GDMA_RQ, q_depth, max_req_msg_size,
+ err = mana_hwc_create_wq(hwc, GDMA_RQ, q_depth, max_resp_msg_size,
hwc->cq, &hwc->rxq);
if (err) {
dev_err(hwc->dev, "Failed to create HWC RQ: %d\n", err);
goto out;
}
- err = mana_hwc_create_wq(hwc, GDMA_SQ, q_depth, max_resp_msg_size,
+ err = mana_hwc_create_wq(hwc, GDMA_SQ, q_depth, max_req_msg_size,
hwc->cq, &hwc->txq);
if (err) {
dev_err(hwc->dev, "Failed to create HWC SQ: %d\n", err);
@@ -749,6 +749,7 @@ static int mana_hwc_init_queues(struct hw_channel_context *hwc, u16 q_depth,
hwc->num_inflight_msg = q_depth;
hwc->max_req_msg_size = max_req_msg_size;
+ hwc->max_resp_msg_size = max_resp_msg_size;
return 0;
out:
diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h
index 16feb39616c1..73671f479399 100644
--- a/include/net/mana/hw_channel.h
+++ b/include/net/mana/hw_channel.h
@@ -181,6 +181,7 @@ struct hw_channel_context {
u16 num_inflight_msg;
u32 max_req_msg_size;
+ u32 max_resp_msg_size;
u16 hwc_init_q_depth_max;
u32 hwc_init_max_req_msg_size;
--
2.43.0
^ permalink raw reply related
* [PATCH net-next 3/7] net: mana: free HWC comp_buf after destroying the EQ
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260715032942.3945317-1-longli@microsoft.com>
mana_hwc_destroy_cq() freed hwc_cq->comp_buf before destroying the CQ and
EQ. comp_buf is dereferenced by mana_hwc_comp_event(), which the EQ
interrupt handler invokes; freeing it while the EQ was still registered
let a late handler touch freed memory.
Destroy the CQ and EQ first -- the EQ teardown deregisters the IRQ and
fences in-flight handlers -- then free comp_buf and hwc_cq.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Long Li <longli@microsoft.com>
---
drivers/net/ethernet/microsoft/mana/hw_channel.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index 3f011ebbe7b3..2239fdeda57c 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -384,14 +384,20 @@ static void mana_hwc_comp_event(void *ctx, struct gdma_queue *q_self)
static void mana_hwc_destroy_cq(struct gdma_context *gc, struct hwc_cq *hwc_cq)
{
- kfree(hwc_cq->comp_buf);
-
if (hwc_cq->gdma_cq)
mana_gd_destroy_queue(gc, hwc_cq->gdma_cq);
+ /* comp_buf is reached only by mana_hwc_comp_event(), which the
+ * EQ handler invokes via cq_table[id]. The CQ destroy above
+ * already cleared that slot and ran synchronize_rcu(), so no
+ * handler can reach comp_buf once it returns. Destroying the EQ
+ * here additionally tears down the IRQ (defense in depth) before
+ * comp_buf and hwc_cq are freed below.
+ */
if (hwc_cq->gdma_eq)
mana_gd_destroy_queue(gc, hwc_cq->gdma_eq);
+ kfree(hwc_cq->comp_buf);
kfree(hwc_cq);
}
--
2.43.0
^ permalink raw reply related
* [PATCH net-next 4/7] net: mana: validate hardware-supplied values in the HWC RX path
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260715032942.3945317-1-longli@microsoft.com>
mana_hwc_rx_event_handler() and mana_hwc_handle_resp() consumed lengths
and indices taken straight from device DMA without validation. A buggy
firmware or a malicious host (in a confidential VM, where the DMA buffer
is shared) could drive a wrong or reused in-flight request to completion
or index out of bounds. Validate before use:
- match the SGE address against the address the driver posted for that
slot, not just an in-range index -- an in-range but wrong SGE would
otherwise truncate onto a neighbouring slot and read a stale response;
- require the response to cover a full gdma_resp_hdr before reading
hwc_msg_id, so a short response cannot complete a slot with stale
bytes left by the buffer's previous occupant;
- bounds-check hwc_msg_id in mana_hwc_handle_resp() before indexing the
inflight bitmap and caller_ctx;
- reject a resp_len larger than the RX buffer.
Repost the RX WQE on every validation early-return so a rejected response
does not permanently shrink the posted RQ depth. The one path that
cannot identify the slot (SGE mismatch) intentionally leaks a single WQE
rather than risk reposting the wrong one.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/hw_channel.c | 58 +++++++++++++++++--
1 file changed, 54 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index 2239fdeda57c..68236727aee8 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -83,6 +83,17 @@ static void mana_hwc_handle_resp(struct hw_channel_context *hwc, u32 resp_len,
struct hwc_caller_ctx *ctx;
int err;
+ /* Validate msg_id is in range before using it to index bitmap
+ * and caller_ctx array. Malicious firmware could send
+ * out-of-range msg_id causing out-of-bounds access.
+ */
+ if (msg_id >= hwc->num_inflight_msg) {
+ dev_err(hwc->dev, "hwc_rx: msg_id %u >= max %u\n",
+ msg_id, hwc->num_inflight_msg);
+ mana_hwc_post_rx_wqe(hwc->rxq, rx_req);
+ return;
+ }
+
if (!test_bit(msg_id, hwc->inflight_msg_res.map)) {
dev_err(hwc->dev, "hwc_rx: invalid msg_id = %u\n", msg_id);
mana_hwc_post_rx_wqe(hwc->rxq, rx_req);
@@ -90,6 +101,18 @@ static void mana_hwc_handle_resp(struct hw_channel_context *hwc, u32 resp_len,
}
ctx = hwc->caller_ctx + msg_id;
+
+ /* Reject responses larger than the RX DMA buffer — the SGE
+ * limits what hardware can DMA, so an oversized resp_len
+ * indicates a firmware bug. Fail rather than silently
+ * truncating.
+ */
+ if (resp_len > rx_req->buf_len) {
+ dev_err(hwc->dev, "HWC RX: resp_len %u > buf_len %u\n",
+ resp_len, rx_req->buf_len);
+ resp_len = 0;
+ }
+
err = mana_hwc_verify_resp_msg(ctx, resp_msg, resp_len);
if (err)
goto out;
@@ -261,19 +284,45 @@ static void mana_hwc_rx_event_handler(void *ctx, u32 gdma_rxq_id,
sge = (struct gdma_sge *)(wqe + 8 + dma_oob->inline_oob_size_div4 * 4);
- /* Select the RX work request for virtual address and for reposting. */
+ /* Recover the originating RX slot from the SGE address. Of the three
+ * terms here only sge->address lives in device-accessible RQ memory;
+ * rq_base_addr and max_resp_msg_size are driver-private constants. An
+ * in-range but wrong/unaligned SGE (corrupted WQE, or a malicious host
+ * in a CVM) would otherwise truncate onto a neighbouring slot, letting
+ * us read a stale response that could complete the wrong, reused
+ * in-flight request. Require the index to be in range AND the address
+ * to exactly match the value the driver posted for that slot.
+ */
rq_base_addr = hwc_rxq->msg_buf->mem_info.dma_handle;
rx_req_idx = (sge->address - rq_base_addr) / hwc->max_resp_msg_size;
- if (rx_req_idx >= hwc_rxq->msg_buf->num_reqs) {
- dev_err(hwc->dev, "HWC RX: wrong rx_req_idx=%llu, num_reqs=%u\n",
- rx_req_idx, hwc_rxq->msg_buf->num_reqs);
+ if (rx_req_idx >= hwc_rxq->queue_depth ||
+ sge->address != (u64)hwc_rxq->msg_buf->reqs[rx_req_idx].buf_sge_addr) {
+ /* Cannot trust which WQE this is, so we cannot safely repost
+ * it; leak one RX WQE and bail. This permanently leaks one
+ * RX WQE but indicates a corrupted SGE from hardware (or host
+ * tampering), which is an unrecoverable device error.
+ */
+ dev_err(hwc->dev, "HWC RX: invalid SGE address %llx (idx=%llu)\n",
+ sge->address, rx_req_idx);
return;
}
rx_req = &hwc_rxq->msg_buf->reqs[rx_req_idx];
resp = (struct gdma_resp_hdr *)rx_req->buf_va;
+ /* Validate resp_len covers the response header before reading
+ * hwc_msg_id. A short response leaves stale data from the
+ * previous buffer occupant, which could match a live slot and
+ * complete the wrong request.
+ */
+ if (rx_oob->tx_oob_data_size < sizeof(*resp)) {
+ dev_err(hwc->dev, "HWC RX: short resp_len=%u\n",
+ rx_oob->tx_oob_data_size);
+ mana_hwc_post_rx_wqe(hwc_rxq, rx_req);
+ return;
+ }
+
/* Read msg_id once from DMA buffer to prevent TOCTOU:
* DMA memory is shared/unencrypted in CVMs - host can
* modify it between reads.
@@ -281,6 +330,7 @@ static void mana_hwc_rx_event_handler(void *ctx, u32 gdma_rxq_id,
msg_id = READ_ONCE(resp->response.hwc_msg_id);
if (msg_id >= hwc->num_inflight_msg) {
dev_err(hwc->dev, "HWC RX: wrong msg_id=%u\n", msg_id);
+ mana_hwc_post_rx_wqe(hwc_rxq, rx_req);
return;
}
--
2.43.0
^ permalink raw reply related
* [PATCH net-next 5/7] net: mana: fix HWC teardown safety with setup_active flag and destroy ordering
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260715032942.3945317-1-longli@microsoft.com>
Two teardown hazards let the hardware touch memory the driver freed.
First, once mana_smc_setup_hwc() succeeds the device has active MST
entries and can DMA into the HWC queue buffers. If a later step in
mana_hwc_establish_channel() fails, the caller had no reliable way to
know teardown was required and could free those buffers while the
mappings were still live -- a DMA-after-free. max_num_cqs was used as a
"HWC is up" proxy, but it is only set when the init EQE arrives.
Add a setup_active flag, set the moment setup_hwc activates MST entries.
establish_channel() now tears down on any later failure and clears
setup_active once teardown succeeds; destroy_channel() gates teardown on
setup_active. max_num_cqs is no longer reset: it is an immutable bound
(see gdma.h) and cq_table == NULL is the sole teardown signal.
Second, destroy_channel() freed the TXQ/RXQ buffers while the HWC EQ was
still on the interrupt dispatch list, so an in-flight interrupt could run
the handler against freed buffers:
CPU A (mana_gd_intr, hard IRQ) CPU B (destroy_channel)
---------------------------------- ------------------------------
free TXQ/RXQ DMA buffers
handler accesses RQ/TXQ buffers (EQ still registered)
Destroy the CQ first: mana_hwc_destroy_cq() -> mana_gd_deregister_irq()
removes the EQ via list_del_rcu() + synchronize_rcu(), after which no
handler can reach the queues; only then free the TXQ and RXQ.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/hw_channel.c | 106 ++++++++++++++----
include/net/mana/gdma.h | 8 +-
include/net/mana/hw_channel.h | 8 ++
3 files changed, 100 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index 68236727aee8..b26c2122ebf5 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -4,6 +4,7 @@
#include <net/mana/gdma.h>
#include <net/mana/mana.h>
#include <net/mana/hw_channel.h>
+#include <linux/pci.h>
#include <linux/vmalloc.h>
static int mana_hwc_get_msg_index(struct hw_channel_context *hwc, u16 *msg_id)
@@ -744,20 +745,33 @@ static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_depth,
if (err)
return err;
- if (!wait_for_completion_timeout(&hwc->hwc_init_eqe_comp, 60 * HZ))
- return -ETIMEDOUT;
+ /* setup_hwc activated MST entries — hardware can now DMA into
+ * our queue buffers. If anything below fails, we must tear
+ * down before returning so the caller doesn't need to track
+ * whether setup_hwc succeeded.
+ */
+ hwc->setup_active = true;
+
+ if (!wait_for_completion_timeout(&hwc->hwc_init_eqe_comp, 60 * HZ)) {
+ err = -ETIMEDOUT;
+ goto teardown;
+ }
*q_depth = hwc->hwc_init_q_depth_max;
*max_req_msg_size = hwc->hwc_init_max_req_msg_size;
*max_resp_msg_size = hwc->hwc_init_max_resp_msg_size;
/* Both were set in mana_hwc_init_event_handler(). */
- if (WARN_ON(cq->id >= gc->max_num_cqs))
- return -EPROTO;
+ if (WARN_ON(cq->id >= gc->max_num_cqs)) {
+ err = -EPROTO;
+ goto teardown;
+ }
cq_table = vcalloc(gc->max_num_cqs, sizeof(*cq_table));
- if (!cq_table)
- return -ENOMEM;
+ if (!cq_table) {
+ err = -ENOMEM;
+ goto teardown;
+ }
rcu_assign_pointer(cq_table[cq->id], cq);
/* Publish the fully-initialised table last; pairs with the
@@ -766,6 +780,16 @@ static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_depth,
rcu_assign_pointer(gc->cq_table, cq_table);
return 0;
+
+teardown:
+ {
+ int td_err = mana_smc_teardown_hwc(&gc->shm_channel, false);
+
+ if (!td_err)
+ hwc->setup_active = false;
+
+ return td_err ? td_err : err;
+ }
}
static int mana_hwc_init_queues(struct hw_channel_context *hwc, u16 q_depth,
@@ -878,11 +902,62 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
if (!hwc)
return;
- /* gc->max_num_cqs is set in mana_hwc_init_event_handler(). If it's
- * non-zero, the HWC worked and we should tear down the HWC here.
+ /* Tear down the HWC if setup_hwc previously activated MST entries.
+ * This is the definitive flag — unlike max_num_cqs which depends
+ * on the init EQE arriving.
*/
- if (gc->max_num_cqs > 0)
- mana_smc_teardown_hwc(&gc->shm_channel, false);
+ if (hwc->setup_active) {
+ int td_err = mana_smc_teardown_hwc(&gc->shm_channel, false);
+
+ if (td_err) {
+ dev_err(gc->dev, "HWC teardown failed: %d, issuing FLR\n",
+ td_err);
+
+ /* On systems without IOMMU, freeing DMA memory with
+ * active hardware MST mappings risks memory corruption.
+ * Issue FLR to force-reset the device and invalidate
+ * all hardware state including MST entries.
+ */
+ td_err = pcie_flr(to_pci_dev(gc->dev));
+ if (td_err) {
+ /* Device is wedged: teardown and FLR both failed.
+ * Hardware may still have active MST entries that
+ * allow DMA into our queue buffers.
+ *
+ * On IOMMU systems: dma_free_coherent() would unmap
+ * the IOVA, causing hardware DMA to fault at the
+ * IOMMU (safe). But on non-IOMMU systems, freeing
+ * the physical pages allows them to be reused for
+ * other purposes while hardware can still DMA to
+ * them (unsafe).
+ *
+ * but leak all DMA buffers to prevent corruption.
+ */
+
+ dev_warn(gc->dev,
+ "Leaked HWC DMA buffers (CQ/RQ/TXQ) to prevent memory corruption. Device is no longer usable.\n");
+
+ /* Do NOT proceed to mana_hwc_destroy_cq/wq — they
+ * would call dma_free_coherent(). Leave hwc, cq,
+ * rxq, txq allocated forever.
+ */
+ return;
+ }
+
+ dev_info(gc->dev, "FLR succeeded, hardware state cleared\n");
+ }
+
+ hwc->setup_active = false;
+ }
+
+ /* Tear down the HWC CQ object first — mana_hwc_destroy_cq()
+ * both unpublishes the CQ from cq_table (+synchronize_rcu) and
+ * deregisters the HWC EQ from the interrupt handler list (via
+ * mana_gd_deregister_irq + synchronize_rcu), guaranteeing no
+ * interrupt handler can access RQ/TXQ buffers after this point.
+ */
+ if (hwc->cq)
+ mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq);
if (hwc->txq)
mana_hwc_destroy_wq(hwc, hwc->txq);
@@ -890,17 +965,6 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
if (hwc->rxq)
mana_hwc_destroy_wq(hwc, hwc->rxq);
- if (hwc->cq)
- mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq);
-
- /* Reset only after mana_hwc_destroy_cq() above has run with a valid
- * max_num_cqs so mana_gd_destroy_cq() clears the CQ table slot and
- * waits out in-flight EQ handlers (synchronize_rcu) before the CQ is
- * freed. Clearing it earlier would make that path early-return and
- * skip the slot clear, leaving a dangling cq_table entry.
- */
- gc->max_num_cqs = 0;
-
kfree(hwc->caller_ctx);
hwc->caller_ctx = NULL;
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index da52701e7816..9ca7cf523366 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -428,7 +428,13 @@ struct gdma_context {
/* L2 MTU */
u16 adapter_mtu;
- /* This maps a CQ index to the queue structure. */
+ /* Size of cq_table, i.e. the largest valid CQ index + 1. Set once
+ * when cq_table is allocated and treated as immutable for the
+ * table's lifetime (a bound only) -- it is never reset on teardown.
+ * cq_table == NULL is the sole "table torn down" signal, so every
+ * cq_table[id] access must guard with both !cq_table (gone) and
+ * id >= max_num_cqs (out of bounds).
+ */
unsigned int max_num_cqs;
/* Both the base pointer and each entry are RCU-managed. The fast
* path (mana_gd_process_eqe) reads the base via rcu_dereference()
diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h
index 73671f479399..684dcec8e612 100644
--- a/include/net/mana/hw_channel.h
+++ b/include/net/mana/hw_channel.h
@@ -200,6 +200,14 @@ struct hw_channel_context {
u32 pf_dest_vrcq_id;
u32 hwc_timeout;
+ /* Set after mana_smc_setup_hwc() succeeds (hardware has active
+ * MST entries). On recoverable paths (establish_channel)
+ * cleared only after successful teardown so a retry remains
+ * possible. On the terminal destroy_channel path, cleared
+ * unconditionally since hwc is about to be freed.
+ */
+ bool setup_active;
+
struct hwc_caller_ctx *caller_ctx;
};
--
2.43.0
^ permalink raw reply related
* [PATCH net-next 6/7] net: mana: support concurrent HWC requests with proper synchronization
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260715032942.3945317-1-longli@microsoft.com>
The HWC serialized all management commands behind a depth-1 semaphore, a
bottleneck when many commands must be issued concurrently. Allowing
multiple in-flight requests exposes several shared-state races that this
patch addresses together.
- Replace the semaphore with a waitqueue + bitmap scheme so senders
acquire message slots in parallel and sleep only when all slots are
busy.
- A response is delivered by handle_resp() in CQ interrupt context
while the issuing sender may concurrently time out and return; both
touch the same caller_ctx slot (output_buf, output_buflen, error):
CPU A (sender timeout) CPU B (handle_resp, IRQ)
------------------------------ ------------------------------
read/clear output_buf write response into output_buf,
complete the slot
Add a per-slot spinlock to make these mutually exclusive, a per-slot
refcount so the last of {sender, handle_resp} releases the bitmap
slot (no double-release, no stuck slot), and an embedded completion
the sender waits on. The sender always NULLs output_buf after
waking, so a late handle_resp() skips the copy.
- Add a per-queue lock to hwc_wq; mana_gd_post_and_ring() is not safe
to call concurrently on the same queue.
- Add channel_up (cleared under the bitmap lock in destroy_channel) and
hwc_timed_out flags so new slot acquisitions are rejected during
teardown and after a timeout.
- destroy_channel() must not free the HWC while senders are still in
flight. Track senders with an atomic refcount; destroy_channel()
force-completes the in-flight slots (-ENODEV) and then waits for the
count to reach zero before freeing. The drain waitqueue lives on
gdma_context, not hwc, so the final wake_up() after the last
atomic_dec does not dereference freed hwc memory.
- The sender looks up the channel via gc->hwc.driver_data, which
destroy_channel() clears and then frees. Without serialization the
lookup and the reference can straddle the free:
CPU A (mana_gd_send_request) CPU B (destroy_channel)
------------------------------ ------------------------------
hwc = gc->hwc.driver_data; // ok
driver_data = NULL;
wait active_senders == 0; // 0!
kfree(hwc);
atomic_inc(&hwc->active_senders); // use-after-free
Guard driver_data with a new gc->hwc_lock spinlock, taken by the
readers (mana_gd_send_request, mana_need_log, mana_serv_reset) and by
the publish/clear, so "load the pointer + take a sender reference" is
atomic against the clear. After the clear a sender either already
holds a reference (and is waited for) or observes NULL and returns
-ENODEV. These are all control-plane paths (HWC commands sleep,
reset runs on a workqueue), so a plain spinlock -- not RCU -- is
sufficient.
This adds the concurrency infrastructure at the bootstrap queue depth of
1; the next patch raises it to the device-reported maximum.
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/gdma_main.c | 53 ++-
.../net/ethernet/microsoft/mana/hw_channel.c | 314 +++++++++++++++---
include/net/mana/gdma.h | 15 +
include/net/mana/hw_channel.h | 29 +-
4 files changed, 365 insertions(+), 46 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index c52ef566dc0c..d44a3e2c4add 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -161,6 +161,8 @@ static int mana_gd_init_registers(struct pci_dev *pdev)
bool mana_need_log(struct gdma_context *gc, int err)
{
struct hw_channel_context *hwc;
+ bool need_log = true;
+ unsigned long flags;
if (err != -ETIMEDOUT)
return true;
@@ -168,11 +170,13 @@ bool mana_need_log(struct gdma_context *gc, int err)
if (!gc)
return true;
+ spin_lock_irqsave(&gc->hwc_lock, flags);
hwc = gc->hwc.driver_data;
if (hwc && hwc->hwc_timeout == 0)
- return false;
+ need_log = false;
+ spin_unlock_irqrestore(&gc->hwc_lock, flags);
- return true;
+ return need_log;
}
static int mana_gd_query_max_resources(struct pci_dev *pdev)
@@ -367,9 +371,25 @@ static int mana_gd_detect_devices(struct pci_dev *pdev)
int mana_gd_send_request(struct gdma_context *gc, u32 req_len, const void *req,
u32 resp_len, void *resp)
{
- struct hw_channel_context *hwc = gc->hwc.driver_data;
+ struct hw_channel_context *hwc;
+ unsigned long flags;
+ int err;
+
+ spin_lock_irqsave(&gc->hwc_lock, flags);
+ hwc = gc->hwc.driver_data;
+ if (!hwc) {
+ spin_unlock_irqrestore(&gc->hwc_lock, flags);
+ return -ENODEV;
+ }
+ atomic_inc(&hwc->active_senders);
+ spin_unlock_irqrestore(&gc->hwc_lock, flags);
+
+ err = mana_hwc_send_request(hwc, req_len, req, resp_len, resp);
- return mana_hwc_send_request(hwc, req_len, req, resp_len, resp);
+ if (atomic_dec_and_test(&hwc->active_senders))
+ wake_up(&gc->hwc_drain_waitq);
+
+ return err;
}
EXPORT_SYMBOL_NS(mana_gd_send_request, "NET_MANA");
@@ -622,6 +642,7 @@ static void mana_serv_reset(struct pci_dev *pdev)
{
struct gdma_context *gc = pci_get_drvdata(pdev);
struct hw_channel_context *hwc;
+ unsigned long flags;
int ret;
if (!gc) {
@@ -631,14 +652,17 @@ static void mana_serv_reset(struct pci_dev *pdev)
return;
}
+ spin_lock_irqsave(&gc->hwc_lock, flags);
hwc = gc->hwc.driver_data;
if (!hwc) {
+ spin_unlock_irqrestore(&gc->hwc_lock, flags);
dev_err(&pdev->dev, "MANA service: no HWC\n");
goto out;
}
/* HWC is not responding in this case, so don't wait */
hwc->hwc_timeout = 0;
+ spin_unlock_irqrestore(&gc->hwc_lock, flags);
dev_info(&pdev->dev, "MANA reset cycle start\n");
@@ -1200,6 +1224,16 @@ static int mana_gd_create_dma_region(struct gdma_dev *gd,
if (!MANA_PAGE_ALIGNED(gmi->virt_addr))
return -EINVAL;
+ /* No RCU needed: this runs only on the data-path queue-creation
+ * path (mana_gd_create_mana_eq/mana_gd_create_mana_wq_cq, called
+ * by mana_en under RTNL and by mana_ib RDMA verbs, or during
+ * init). Every teardown path — mana_gd_remove, mana_gd_suspend,
+ * and the HWC reset/service path (which goes through
+ * mana_gd_suspend) — drains those consumers via mana_rdma_remove()
+ * + mana_remove() before mana_hwc_destroy_channel() clears
+ * gc->hwc.driver_data, so no concurrent destroy can race with
+ * this dereference.
+ */
hwc = gc->hwc.driver_data;
req_msg_size = struct_size(req, page_addr_list, num_page);
if (req_msg_size > hwc->max_req_msg_size)
@@ -1389,7 +1423,17 @@ int mana_gd_verify_vf_version(struct pci_dev *pdev)
struct hw_channel_context *hwc;
int err;
+ /* No RCU needed: this runs only inside mana_gd_setup, on the
+ * probe and resume paths. The PCI/PM core holds device_lock
+ * across .probe/.resume and .remove/.suspend, so setup cannot
+ * overlap teardown of the same device. The HWC reset/service
+ * path is additionally serialized by GC_IN_SERVICE and runs
+ * suspend (destroy) then resume (this) sequentially in one work
+ * item. driver_data was just set by mana_hwc_create_channel
+ * earlier in this same setup call, so it is live here.
+ */
hwc = gc->hwc.driver_data;
+
mana_gd_init_req_hdr(&req.hdr, GDMA_VERIFY_VF_DRIVER_VERSION,
sizeof(req), sizeof(resp));
@@ -2379,6 +2423,7 @@ static int mana_gd_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
mutex_init(&gc->eq_test_event_mutex);
mutex_init(&gc->gic_mutex);
+ spin_lock_init(&gc->hwc_lock);
pci_set_drvdata(pdev, gc);
gc->bar0_pa = pci_resource_start(pdev, 0);
gc->bar0_size = pci_resource_len(pdev, 0);
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index b26c2122ebf5..9ba4e75a4dd3 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -7,25 +7,52 @@
#include <linux/pci.h>
#include <linux/vmalloc.h>
+/* Acquire a free message slot from the inflight bitmap. Returns
+ * -ENODEV if the channel is torn down, or -ETIMEDOUT if a prior HWC
+ * command has timed out (preserving the error code callers expect).
+ */
static int mana_hwc_get_msg_index(struct hw_channel_context *hwc, u16 *msg_id)
{
struct gdma_resource *r = &hwc->inflight_msg_res;
unsigned long flags;
u32 index;
- down(&hwc->sema);
+ for (;;) {
+ spin_lock_irqsave(&r->lock, flags);
- spin_lock_irqsave(&r->lock, flags);
+ if (!hwc->channel_up || hwc->hwc_timed_out) {
+ spin_unlock_irqrestore(&r->lock, flags);
+ return hwc->channel_up ? -ETIMEDOUT : -ENODEV;
+ }
- index = find_first_zero_bit(hwc->inflight_msg_res.map,
- hwc->inflight_msg_res.size);
+ index = find_first_zero_bit(r->map, r->size);
+ if (index < r->size) {
+ struct hwc_caller_ctx *ctx;
+
+ bitmap_set(r->map, index, 1);
+ ctx = &hwc->caller_ctx[index];
+ reinit_completion(&ctx->comp_event);
+ refcount_set(&ctx->refcnt, 1);
+ ctx->msg_id = index;
+ ctx->error = -EINPROGRESS;
+ spin_unlock_irqrestore(&r->lock, flags);
+ break;
+ }
+ spin_unlock_irqrestore(&r->lock, flags);
- bitmap_set(hwc->inflight_msg_res.map, index, 1);
+ wait_event(hwc->msg_waitq,
+ !hwc->channel_up ||
+ hwc->hwc_timed_out ||
+ !bitmap_full(r->map, r->size));
- spin_unlock_irqrestore(&r->lock, flags);
+ if (!hwc->channel_up)
+ return -ENODEV;
- *msg_id = index;
+ if (hwc->hwc_timed_out)
+ return -ETIMEDOUT;
+ }
+ *msg_id = index;
return 0;
}
@@ -35,10 +62,17 @@ static void mana_hwc_put_msg_index(struct hw_channel_context *hwc, u16 msg_id)
unsigned long flags;
spin_lock_irqsave(&r->lock, flags);
- bitmap_clear(hwc->inflight_msg_res.map, msg_id, 1);
+ bitmap_clear(r->map, msg_id, 1);
spin_unlock_irqrestore(&r->lock, flags);
- up(&hwc->sema);
+ wake_up(&hwc->msg_waitq);
+}
+
+static void hwc_ctx_put(struct hw_channel_context *hwc,
+ struct hwc_caller_ctx *ctx)
+{
+ if (refcount_dec_and_test(&ctx->refcnt))
+ mana_hwc_put_msg_index(hwc, ctx->msg_id);
}
static int mana_hwc_verify_resp_msg(const struct hwc_caller_ctx *caller_ctx,
@@ -114,22 +148,32 @@ static void mana_hwc_handle_resp(struct hw_channel_context *hwc, u32 resp_len,
resp_len = 0;
}
- err = mana_hwc_verify_resp_msg(ctx, resp_msg, resp_len);
- if (err)
- goto out;
+ spin_lock(&ctx->lock);
- ctx->status_code = resp_msg->status;
+ err = mana_hwc_verify_resp_msg(ctx, resp_msg, resp_len);
- memcpy(ctx->output_buf, resp_msg, resp_len);
-out:
- ctx->error = err;
+ if (!err && ctx->output_buf) {
+ ctx->status_code = resp_msg->status;
+ memcpy(ctx->output_buf, resp_msg, resp_len);
+ ctx->error = 0;
+ } else if (ctx->output_buf) {
+ /* Only overwrite error if the sender hasn't timed out
+ * or been force-completed by destroy. When output_buf
+ * is NULL, a terminal error (-ENODEV or timeout) has
+ * already been set — preserve it so the sender doesn't
+ * see a spurious success.
+ */
+ ctx->error = err;
+ }
- /* Must post rx wqe before complete(), otherwise the next rx may
- * hit no_wqe error.
+ /* Post RX WQE before completing — the next response may arrive
+ * immediately and needs a posted buffer.
*/
mana_hwc_post_rx_wqe(hwc->rxq, rx_req);
-
complete(&ctx->comp_event);
+ spin_unlock(&ctx->lock);
+
+ hwc_ctx_put(hwc, ctx);
}
static void mana_hwc_init_event_handler(void *ctx, struct gdma_queue *q_self,
@@ -617,6 +661,7 @@ static int mana_hwc_create_wq(struct hw_channel_context *hwc,
hwc_wq->gdma_wq = queue;
hwc_wq->queue_depth = q_depth;
hwc_wq->hwc_cq = hwc_cq;
+ spin_lock_init(&hwc_wq->lock);
err = mana_hwc_alloc_dma_buf(hwc, q_depth, max_msg_size,
&hwc_wq->msg_buf);
@@ -634,7 +679,7 @@ static int mana_hwc_create_wq(struct hw_channel_context *hwc,
return err;
}
-static int mana_hwc_post_tx_wqe(const struct hwc_wq *hwc_txq,
+static int mana_hwc_post_tx_wqe(struct hwc_wq *hwc_txq,
struct hwc_work_request *req,
u32 dest_virt_rq_id, u32 dest_virt_rcq_id,
bool dest_pf)
@@ -673,7 +718,11 @@ static int mana_hwc_post_tx_wqe(const struct hwc_wq *hwc_txq,
req->wqe_req.inline_oob_data = tx_oob;
req->wqe_req.client_data_unit = 0;
+ /* Serialize WQE posting — multiple senders may call concurrently. */
+ spin_lock(&hwc_txq->lock);
err = mana_gd_post_and_ring(hwc_txq->gdma_wq, &req->wqe_req, NULL);
+ spin_unlock(&hwc_txq->lock);
+
if (err)
dev_err(dev, "Failed to post WQE on HWC SQ: %d\n", err);
return err;
@@ -684,7 +733,7 @@ static int mana_hwc_init_inflight_msg(struct hw_channel_context *hwc,
{
int err;
- sema_init(&hwc->sema, num_msg);
+ init_waitqueue_head(&hwc->msg_waitq);
err = mana_gd_alloc_res_map(num_msg, &hwc->inflight_msg_res);
if (err)
@@ -714,18 +763,34 @@ static int mana_hwc_test_channel(struct hw_channel_context *hwc, u16 q_depth,
if (!ctx)
return -ENOMEM;
- for (i = 0; i < q_depth; ++i)
+ for (i = 0; i < q_depth; ++i) {
+ spin_lock_init(&ctx[i].lock);
init_completion(&ctx[i].comp_event);
+ }
hwc->caller_ctx = ctx;
- return mana_gd_test_eq(gc, hwc->cq->gdma_eq);
+ /* channel_up must be set before the test EQ request, because
+ * the request goes through mana_hwc_get_msg_index() which
+ * checks channel_up. caller_ctx is allocated above, so
+ * concurrent access to a NULL caller_ctx is not possible.
+ */
+ hwc->channel_up = true;
+
+ err = mana_gd_test_eq(gc, hwc->cq->gdma_eq);
+ if (err)
+ hwc->channel_up = false;
+
+ return err;
}
static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_depth,
u32 *max_req_msg_size,
u32 *max_resp_msg_size)
{
+ /* No RCU needed: called only from mana_hwc_create_channel
+ * during init, before the channel is published to senders.
+ */
struct hw_channel_context *hwc = gc->hwc.driver_data;
struct gdma_queue *rq = hwc->rxq->gdma_wq;
struct gdma_queue *sq = hwc->txq->gdma_wq;
@@ -842,6 +907,7 @@ int mana_hwc_create_channel(struct gdma_context *gc)
u32 max_req_msg_size, max_resp_msg_size;
struct gdma_dev *gd = &gc->hwc;
struct hw_channel_context *hwc;
+ unsigned long flags;
u16 q_depth_max;
int err;
@@ -850,10 +916,11 @@ int mana_hwc_create_channel(struct gdma_context *gc)
return -ENOMEM;
gd->gdma_context = gc;
- gd->driver_data = hwc;
hwc->gdma_dev = gd;
hwc->dev = gc->dev;
hwc->hwc_timeout = HW_CHANNEL_WAIT_RESOURCE_TIMEOUT_MS;
+ atomic_set(&hwc->active_senders, 0);
+ init_waitqueue_head(&gc->hwc_drain_waitq);
/* HWC's instance number is always 0. */
gd->dev_id.as_uint32 = 0;
@@ -862,6 +929,15 @@ int mana_hwc_create_channel(struct gdma_context *gc)
gd->pdid = INVALID_PDID;
gd->doorbell = INVALID_DOORBELL;
+ /* Publish driver_data last, under hwc_lock: the lock orders the hwc
+ * initialisation above before the pointer becomes visible and
+ * serialises the publish against the control-plane readers in
+ * mana_gd_send_request(), mana_need_log() and mana_serv_reset().
+ */
+ spin_lock_irqsave(&gc->hwc_lock, flags);
+ gc->hwc.driver_data = hwc;
+ spin_unlock_irqrestore(&gc->hwc_lock, flags);
+
/* mana_hwc_init_queues() only creates the required data structures,
* and doesn't touch the HWC device.
*/
@@ -898,13 +974,47 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
{
struct hw_channel_context *hwc = gc->hwc.driver_data;
struct gdma_queue __rcu **old_cq_table;
+ unsigned long flags;
if (!hwc)
return;
+ /* Prevent new requests from starting and wake any
+ * threads waiting for a free msg slot. Set channel_up under
+ * the bitmap lock so get_msg_index() cannot acquire a slot
+ * and increment active_senders after this point.
+ *
+ * If channel_up is already false (e.g. init failed before
+ * the channel was established), skip the lock — it may not
+ * have been initialized yet, and no senders can be active.
+ */
+ if (hwc->channel_up) {
+ spin_lock_irqsave(&hwc->inflight_msg_res.lock, flags);
+ hwc->channel_up = false;
+ spin_unlock_irqrestore(&hwc->inflight_msg_res.lock, flags);
+ wake_up_all(&hwc->msg_waitq);
+ }
+
+ /* Clear the pointer under hwc_lock so new callers in
+ * mana_gd_send_request() see NULL and return -ENODEV. The lock
+ * makes the readers' "load driver_data + atomic_inc(active_senders)"
+ * atomic against this store, so once it returns no new sender can
+ * take a reference; the active_senders drain below waits out those
+ * that already did, before their hwc is freed.
+ */
+ spin_lock_irqsave(&gc->hwc_lock, flags);
+ gc->hwc.driver_data = NULL;
+ spin_unlock_irqrestore(&gc->hwc_lock, flags);
+
/* Tear down the HWC if setup_hwc previously activated MST entries.
* This is the definitive flag — unlike max_num_cqs which depends
* on the init EQE arriving.
+ *
+ * The return value is intentionally not checked. This is the
+ * terminal cleanup path — resources must be freed regardless.
+ * If teardown fails, hardware may still have active MST entries,
+ * but the EQ deregistration and IOMMU unmapping below prevent
+ * stale hardware accesses from reaching kernel memory.
*/
if (hwc->setup_active) {
int td_err = mana_smc_teardown_hwc(&gc->shm_channel, false);
@@ -950,11 +1060,49 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
hwc->setup_active = false;
}
+ /* After SMC teardown, no more hardware events should arrive.
+ * Force-complete any remaining in-flight senders so they can
+ * exit and drop their refs.
+ */
+ if (hwc->caller_ctx) {
+ struct hwc_caller_ctx *ctx;
+ int i;
+
+ for (i = 0; i < hwc->num_inflight_msg; i++) {
+ if (!test_bit(i, hwc->inflight_msg_res.map))
+ continue;
+
+ ctx = &hwc->caller_ctx[i];
+
+ /* Wake senders blocked on wait_for_completion.
+ * Set error under lock to avoid racing with
+ * handle_resp() which writes error under the
+ * same lock. The sender NULLs output_buf
+ * after waking — doing it here would race
+ * with a sender that hasn't set output_buf yet.
+ */
+ spin_lock_irqsave(&ctx->lock, flags);
+ ctx->error = -ENODEV;
+ complete(&ctx->comp_event);
+ spin_unlock_irqrestore(&ctx->lock, flags);
+ }
+ }
+
+ /* Wait for all sender threads to finish and drop their refs.
+ * After this, only slots held by timed-out senders whose
+ * handle_resp() never ran remain in the bitmap.
+ */
+ wait_event(gc->hwc_drain_waitq,
+ atomic_read(&hwc->active_senders) == 0);
+
/* Tear down the HWC CQ object first — mana_hwc_destroy_cq()
* both unpublishes the CQ from cq_table (+synchronize_rcu) and
- * deregisters the HWC EQ from the interrupt handler list (via
- * mana_gd_deregister_irq + synchronize_rcu), guaranteeing no
- * interrupt handler can access RQ/TXQ buffers after this point.
+ * deregisters the HWC EQ from the interrupt handler's RCU list
+ * (via mana_gd_deregister_irq + synchronize_rcu), guaranteeing
+ * no interrupt handler can access RQ/TXQ buffers after this
+ * point. The active_senders drain above ensures no sender is
+ * accessing the CQ via txq->hwc_cq when it is destroyed. Then
+ * destroy TXQ and RQ safely.
*/
if (hwc->cq)
mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq);
@@ -965,6 +1113,23 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
if (hwc->rxq)
mana_hwc_destroy_wq(hwc, hwc->rxq);
+ /* Release any slots still held — these belong to timed-out
+ * senders where handle_resp() never ran (refcount = 1 with
+ * handle_resp's ref still outstanding).
+ */
+ if (hwc->caller_ctx) {
+ struct hwc_caller_ctx *ctx;
+ int i;
+
+ for (i = 0; i < hwc->num_inflight_msg; i++) {
+ if (!test_bit(i, hwc->inflight_msg_res.map))
+ continue;
+
+ ctx = &hwc->caller_ctx[i];
+ hwc_ctx_put(hwc, ctx);
+ }
+ }
+
kfree(hwc->caller_ctx);
hwc->caller_ctx = NULL;
@@ -978,7 +1143,6 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
hwc->hwc_timeout = 0;
kfree(hwc);
- gc->hwc.driver_data = NULL;
gc->hwc.gdma_context = NULL;
old_cq_table = rcu_replace_pointer(gc->cq_table, NULL, true);
@@ -994,13 +1158,17 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len,
struct hwc_wq *txq = hwc->txq;
struct gdma_req_hdr *req_msg;
struct hwc_caller_ctx *ctx;
+ unsigned long flags;
u32 dest_vrcq = 0;
u32 dest_vrq = 0;
u32 command;
+ u32 status;
u16 msg_id;
int err;
- mana_hwc_get_msg_index(hwc, &msg_id);
+ err = mana_hwc_get_msg_index(hwc, &msg_id);
+ if (err)
+ return err;
tx_wr = &txq->msg_buf->reqs[msg_id];
@@ -1012,8 +1180,11 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len,
}
ctx = hwc->caller_ctx + msg_id;
+
+ spin_lock_irqsave(&ctx->lock, flags);
ctx->output_buf = resp;
ctx->output_buflen = resp_len;
+ spin_unlock_irqrestore(&ctx->lock, flags);
req_msg = (struct gdma_req_hdr *)tx_wr->buf_va;
if (req)
@@ -1029,8 +1200,14 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len,
dest_vrcq = hwc->pf_dest_vrcq_id;
}
+ /* Take handle_resp's ref before posting — hardware can respond
+ * immediately after the doorbell ring.
+ */
+ refcount_inc(&ctx->refcnt);
+
err = mana_hwc_post_tx_wqe(txq, tx_wr, dest_vrq, dest_vrcq, false);
if (err) {
+ refcount_dec(&ctx->refcnt);
dev_err(hwc->dev, "HWC: Failed to post send WQE: %d\n", err);
goto out;
}
@@ -1041,31 +1218,86 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len,
dev_err(hwc->dev, "Command 0x%x timed out: %u ms\n",
command, hwc->hwc_timeout);
- /* Reduce further waiting if HWC no response */
+ /* NULL out output_buf so a late handle_resp() won't write
+ * into the caller's buffer after the sender returns, then
+ * check whether handle_resp() already delivered a valid
+ * response between the timeout firing and this lock
+ * acquisition — ctx->error != -EINPROGRESS means it ran.
+ */
+ spin_lock_irqsave(&ctx->lock, flags);
+ ctx->output_buf = NULL;
+ err = ctx->error;
+ status = ctx->status_code;
+ spin_unlock_irqrestore(&ctx->lock, flags);
+
+ if (err != -EINPROGRESS) {
+ /* handle_resp() delivered a valid response just after
+ * the timeout fired. The hardware is alive, so use the
+ * response and leave the channel usable — do not latch
+ * hwc_timed_out or degrade hwc_timeout for what turned
+ * out to be a transient race.
+ */
+ hwc_ctx_put(hwc, ctx);
+ goto check_status;
+ }
+
+ /* Genuine timeout: no response arrived. Reduce further
+ * waiting, and mark the channel timed out under the bitmap
+ * lock so get_msg_index() cannot acquire new slots after this.
+ */
if (hwc->hwc_timeout > 1)
hwc->hwc_timeout = 1;
+ spin_lock_irqsave(&hwc->inflight_msg_res.lock, flags);
+ hwc->hwc_timed_out = true;
+ spin_unlock_irqrestore(&hwc->inflight_msg_res.lock, flags);
+ wake_up_all(&hwc->msg_waitq);
+
err = -ETIMEDOUT;
- goto out;
+ hwc_ctx_put(hwc, ctx);
+ goto done;
}
- if (ctx->error) {
- err = ctx->error;
- goto out;
- }
+ /* NULL output_buf so a late handle_resp() won't memcpy into
+ * the caller's buffer after the sender exits. Read error and
+ * status_code under the same lock — after hwc_ctx_put the slot
+ * may be reused and these fields overwritten.
+ */
+ spin_lock_irqsave(&ctx->lock, flags);
+ ctx->output_buf = NULL;
+ err = ctx->error;
+ status = ctx->status_code;
+ spin_unlock_irqrestore(&ctx->lock, flags);
+ hwc_ctx_put(hwc, ctx);
+
+check_status:
+ if (err)
+ goto done;
- if (ctx->status_code && ctx->status_code != GDMA_STATUS_MORE_ENTRIES) {
- if (ctx->status_code == GDMA_STATUS_CMD_UNSUPPORTED) {
+ if (status && status != GDMA_STATUS_MORE_ENTRIES) {
+ if (status == GDMA_STATUS_CMD_UNSUPPORTED) {
err = -EOPNOTSUPP;
- goto out;
+ goto done;
}
+
if (command != MANA_QUERY_PHY_STAT)
dev_err(hwc->dev, "Command 0x%x failed with status: 0x%x\n",
- command, ctx->status_code);
+ command, status);
err = -EPROTO;
- goto out;
+ goto done;
}
+
+ err = 0;
+ goto done;
out:
- mana_hwc_put_msg_index(hwc, msg_id);
+ /* Pre-post error paths: no WQE was submitted so handle_resp()
+ * cannot race here. refcount is 1 (no second ref taken).
+ */
+ ctx = hwc->caller_ctx + msg_id;
+ spin_lock_irqsave(&ctx->lock, flags);
+ ctx->output_buf = NULL;
+ spin_unlock_irqrestore(&ctx->lock, flags);
+ hwc_ctx_put(hwc, ctx);
+done:
return err;
}
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 9ca7cf523366..01e845237b6a 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -481,6 +481,21 @@ struct gdma_context {
/* Hardware communication channel (HWC) */
struct gdma_dev hwc;
+ /* destroy_channel() waits here for all HWC senders to exit.
+ * Lives on gc (not hwc) so wake_up() after the last sender's
+ * atomic_dec doesn't dereference freed hwc memory.
+ */
+ wait_queue_head_t hwc_drain_waitq;
+
+ /* Serializes hwc.driver_data (the hw_channel_context pointer)
+ * between the control-plane readers in mana_gd_send_request(),
+ * mana_need_log() and mana_serv_reset() and the publish/clear in
+ * mana_hwc_create_channel()/mana_hwc_destroy_channel(). All users
+ * are control-plane (HWC commands sleep; reset runs on a workqueue),
+ * so a plain spinlock -- not RCU -- is sufficient.
+ */
+ spinlock_t hwc_lock;
+
/* Azure network adapter */
struct gdma_dev mana;
diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h
index 684dcec8e612..bc62d1ce7bd4 100644
--- a/include/net/mana/hw_channel.h
+++ b/include/net/mana/hw_channel.h
@@ -164,6 +164,9 @@ struct hwc_wq {
u16 queue_depth;
struct hwc_cq *hwc_cq;
+
+ /* Serializes concurrent mana_gd_post_and_ring() calls. */
+ spinlock_t lock;
};
struct hwc_caller_ctx {
@@ -173,6 +176,17 @@ struct hwc_caller_ctx {
u32 error; /* Linux error code */
u32 status_code;
+
+ /* Protects output_buf against concurrent access from
+ * handle_resp() (CQ interrupt) and the sender timeout path.
+ */
+ spinlock_t lock;
+
+ /* Tracks sender + handle_resp ownership. The last put
+ * (refcount reaches 0) releases the bitmap slot.
+ */
+ refcount_t refcnt;
+ u16 msg_id;
};
struct hw_channel_context {
@@ -193,13 +207,24 @@ struct hw_channel_context {
struct hwc_wq *txq;
struct hwc_cq *cq;
- struct semaphore sema;
struct gdma_resource inflight_msg_res;
+ /* Waitqueue for senders blocked on a full inflight bitmap. */
+ wait_queue_head_t msg_waitq;
u32 pf_dest_vrq_id;
u32 pf_dest_vrcq_id;
u32 hwc_timeout;
+ /* Set after channel is fully established; cleared on teardown to
+ * abort waiters in mana_hwc_get_msg_index() and reject new sends.
+ */
+ bool channel_up;
+
+ /* Set on first HWC timeout. Causes get_msg_index() to return
+ * -ETIMEDOUT instead of waiting, draining all queued senders.
+ */
+ bool hwc_timed_out;
+
/* Set after mana_smc_setup_hwc() succeeds (hardware has active
* MST entries). On recoverable paths (establish_channel)
* cleared only after successful teardown so a retry remains
@@ -208,6 +233,8 @@ struct hw_channel_context {
*/
bool setup_active;
+ atomic_t active_senders;
+
struct hwc_caller_ctx *caller_ctx;
};
--
2.43.0
^ permalink raw reply related
* [PATCH net-next 7/7] net: mana: add dynamic HWC queue depth with reinit path
From: Long Li @ 2026-07-15 3:29 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta, Simon Horman
Cc: netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260715032942.3945317-1-longli@microsoft.com>
The HWC is first established at a bootstrap queue depth of 1. Query the
device's maximum supported depth and, if larger, tear down and rebuild
the HWC queues at that depth before re-establishing the channel, so more
management commands can be in flight. Advertise
GDMA_DRV_CAP_FLAG_1_DYN_HWC_QUEUE_DEPTH so the firmware enables this only
when the driver supports it.
mana_hwc_destroy_queues() tears down the CQ first, which deregisters the
EQ IRQ (mana_gd_deregister_irq() + synchronize_rcu()) so no interrupt
handler can touch the queues, then the TXQ, RXQ and inflight resources.
Validate the device-reported dimensions before they size DMA
allocations: enforce the request/response header minimums, ensure
q_depth * max_msg_size plus alignment fits in u32, and cap CQ depth to
U16_MAX/2. Carry the depth as u32 -- the device field is 24-bit, so
truncating to u16 on receipt could wrap a large value to a small depth
and silently pass these checks.
If reinit fails, fall back to the bootstrap-depth channel when its
teardown succeeded; if teardown also failed, hardware mappings may still
be active, so abort channel creation instead.
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/hw_channel.c | 223 +++++++++++++++++-
include/net/mana/gdma.h | 4 +
include/net/mana/hw_channel.h | 2 +-
3 files changed, 218 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index 9ba4e75a4dd3..3d0f17de3442 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -214,7 +214,12 @@ static void mana_hwc_init_event_handler(void *ctx, struct gdma_queue *q_self,
break;
case HWC_INIT_DATA_QUEUE_DEPTH:
- hwc->hwc_init_q_depth_max = (u16)val;
+ /* HWC_INIT_DATA_QUEUE_DEPTH is a 24-bit field. Keep
+ * the full device-reported value here; it is clamped
+ * and validated in mana_hwc_create_channel() rather
+ * than silently truncated to u16.
+ */
+ hwc->hwc_init_q_depth_max = val;
break;
case HWC_INIT_DATA_MAX_REQUEST:
@@ -784,7 +789,7 @@ static int mana_hwc_test_channel(struct hw_channel_context *hwc, u16 q_depth,
return err;
}
-static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_depth,
+static int mana_hwc_establish_channel(struct gdma_context *gc, u32 *q_depth,
u32 *max_req_msg_size,
u32 *max_resp_msg_size)
{
@@ -862,6 +867,12 @@ static int mana_hwc_init_queues(struct hw_channel_context *hwc, u16 q_depth,
{
int err;
+ /* CQ depth is q_depth * 2 (SQ + RQ) passed as u16 to create_cq.
+ * Cap to prevent u16 truncation.
+ */
+ if (q_depth > U16_MAX / 2)
+ q_depth = U16_MAX / 2;
+
err = mana_hwc_init_inflight_msg(hwc, q_depth);
if (err)
return err;
@@ -902,13 +913,62 @@ static int mana_hwc_init_queues(struct hw_channel_context *hwc, u16 q_depth,
return err;
}
+/* Tear down all HWC queues and free associated resources. Used on
+ * the reinit-with-higher-queue-depth path and reinit fallback.
+ *
+ * PRECONDITION: must be called only during channel bring-up in
+ * mana_hwc_create_channel(), before the channel is published to
+ * senders. There the setup thread is effectively single-threaded
+ * (serialized against teardown by the PCI/PM device_lock or, on the
+ * service path, GC_IN_SERVICE, with the data path not yet probed),
+ * channel_up is still false, caller_ctx is not yet allocated, and
+ * active_senders is 0 — so no concurrent request/response user can
+ * touch these queues. That is why this skips the hwc_lock-protected
+ * driver_data clear + active_senders drain that
+ * mana_hwc_destroy_channel() needs for the runtime teardown race;
+ * only the CQ-first ordering below (to fence off a pending interrupt)
+ * is required. Calling this on a live, published channel would be a
+ * use-after-free.
+ */
+static void mana_hwc_destroy_queues(struct hw_channel_context *hwc)
+{
+ struct gdma_context *gc = hwc->gdma_dev->gdma_context;
+
+ /* Destroy CQ first to deregister the EQ from the interrupt
+ * handler list before freeing caller_ctx, TXQ, or RXQ memory.
+ * A pending interrupt handler could still reach handle_resp()
+ * which dereferences caller_ctx.
+ */
+ if (hwc->cq) {
+ mana_hwc_destroy_cq(gc, hwc->cq);
+ hwc->cq = NULL;
+ }
+
+ kfree(hwc->caller_ctx);
+ hwc->caller_ctx = NULL;
+
+ if (hwc->txq) {
+ mana_hwc_destroy_wq(hwc, hwc->txq);
+ hwc->txq = NULL;
+ }
+
+ if (hwc->rxq) {
+ mana_hwc_destroy_wq(hwc, hwc->rxq);
+ hwc->rxq = NULL;
+ }
+
+ mana_gd_free_res_map(&hwc->inflight_msg_res);
+ hwc->num_inflight_msg = 0;
+}
+
int mana_hwc_create_channel(struct gdma_context *gc)
{
u32 max_req_msg_size, max_resp_msg_size;
struct gdma_dev *gd = &gc->hwc;
struct hw_channel_context *hwc;
+ struct gdma_queue __rcu **old_cq_table;
unsigned long flags;
- u16 q_depth_max;
+ u32 q_depth_max;
int err;
hwc = kzalloc_obj(*hwc);
@@ -956,8 +1016,130 @@ int mana_hwc_create_channel(struct gdma_context *gc)
goto out;
}
+ /* The channel was bootstrapped at a minimal queue depth. If the
+ * device reports a higher maximum, tear down and rebuild with
+ * the larger depth so more HWC commands can be in flight.
+ */
+ if (q_depth_max > HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH) {
+ /* q_depth_max now carries the full device-reported value
+ * (HWC_INIT_DATA_QUEUE_DEPTH is 24-bit). Clamp it to the
+ * depth the rest of the driver supports — create_cq() takes
+ * q_depth * 2 as a u16 — before the overflow check below, so
+ * an over-large but otherwise-valid depth is reduced to the
+ * maximum instead of wrapping or being rejected.
+ */
+ if (q_depth_max > U16_MAX / 2)
+ q_depth_max = U16_MAX / 2;
+
+ /* Sanity-check device-reported values before using them
+ * to size DMA allocations. Enforce protocol minimums
+ * for message sizes and check that q_depth * max_msg_size
+ * plus alignment headroom fits in u32 (for
+ * mana_hwc_alloc_dma_buf's MANA_PAGE_ALIGN).
+ */
+ if (!max_req_msg_size || !max_resp_msg_size ||
+ max_req_msg_size < sizeof(struct gdma_req_hdr) ||
+ max_resp_msg_size < sizeof(struct gdma_resp_hdr) ||
+ (u64)q_depth_max * max_req_msg_size >
+ U32_MAX - MANA_PAGE_SIZE ||
+ (u64)q_depth_max * max_resp_msg_size >
+ U32_MAX - MANA_PAGE_SIZE) {
+ dev_err(hwc->dev,
+ "HWC: invalid dims q=%u req=%u resp=%u\n",
+ q_depth_max, max_req_msg_size,
+ max_resp_msg_size);
+ q_depth_max = HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH;
+ goto skip_reinit;
+ }
+
+ err = mana_smc_teardown_hwc(&gc->shm_channel, false);
+ if (err) {
+ /* Keep using the bootstrap-depth channel. */
+ dev_err(hwc->dev,
+ "Failed to teardown HWC for reinit: %d\n",
+ err);
+ q_depth_max = HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH;
+ goto skip_reinit;
+ }
+
+ hwc->setup_active = false;
+
+ /* Destroy queues first — mana_gd_destroy_cq inside
+ * unpublishes the CQ from cq_table via
+ * rcu_assign_pointer(NULL) + synchronize_rcu.
+ * Must happen while cq_table is still valid.
+ */
+ mana_hwc_destroy_queues(hwc);
+
+ old_cq_table = rcu_replace_pointer(gc->cq_table, NULL, true);
+ synchronize_rcu();
+ vfree(old_cq_table);
+
+ err = mana_hwc_init_queues(hwc, q_depth_max,
+ max_req_msg_size,
+ max_resp_msg_size);
+ if (err) {
+ dev_err(hwc->dev, "Failed to reinit HWC: %d\n", err);
+ goto reinit_fallback;
+ }
+
+ err = mana_hwc_establish_channel(gc, &q_depth_max,
+ &max_req_msg_size,
+ &max_resp_msg_size);
+ if (err) {
+ dev_err(hwc->dev, "Failed to re-establish HWC: %d\n",
+ err);
+ /* establish_channel does internal teardown on
+ * failure. If teardown succeeded (setup_active
+ * cleared), MST entries are invalidated and we
+ * can try the bootstrap fallback. If teardown
+ * also failed (setup_active still set), hardware
+ * mappings may still be active — skip fallback.
+ */
+ if (hwc->setup_active)
+ goto out;
+ goto reinit_fallback;
+ }
+ }
+
+ goto skip_reinit;
+
+reinit_fallback:
+ /* Restore bootstrap-depth channel so the device remains functional.
+ * Free cq_table if it was allocated by a partially successful
+ * establish attempt.
+ */
+ dev_warn(hwc->dev, "HWC reinit failed, falling back to bootstrap depth\n");
+
+ mana_hwc_destroy_queues(hwc);
+
+ old_cq_table = rcu_replace_pointer(gc->cq_table, NULL, true);
+ synchronize_rcu();
+ vfree(old_cq_table);
+
+ err = mana_hwc_init_queues(hwc, HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH,
+ HW_CHANNEL_MAX_REQUEST_SIZE,
+ HW_CHANNEL_MAX_RESPONSE_SIZE);
+ if (err) {
+ dev_err(hwc->dev, "Failed to restore bootstrap HWC: %d\n", err);
+ goto out;
+ }
+
+ err = mana_hwc_establish_channel(gc, &q_depth_max, &max_req_msg_size,
+ &max_resp_msg_size);
+ if (err) {
+ dev_err(hwc->dev, "Failed to re-establish bootstrap HWC: %d\n",
+ err);
+ goto out;
+ }
+
+skip_reinit:
+
+ /* No RCU needed: still in mana_hwc_create_channel, the
+ * pointer has not been published to concurrent senders yet.
+ */
err = mana_hwc_test_channel(gc->hwc.driver_data,
- HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH,
+ hwc->num_inflight_msg,
max_req_msg_size, max_resp_msg_size);
if (err) {
dev_err(hwc->dev, "Failed to test HWC: %d\n", err);
@@ -972,6 +1154,10 @@ int mana_hwc_create_channel(struct gdma_context *gc)
void mana_hwc_destroy_channel(struct gdma_context *gc)
{
+ /* This is the only destroy entry point. driver_data is read
+ * plainly here (teardown is serialised against other teardown);
+ * it is cleared under hwc_lock below before hwc is freed.
+ */
struct hw_channel_context *hwc = gc->hwc.driver_data;
struct gdma_queue __rcu **old_cq_table;
unsigned long flags;
@@ -1011,10 +1197,21 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
* on the init EQE arriving.
*
* The return value is intentionally not checked. This is the
- * terminal cleanup path — resources must be freed regardless.
- * If teardown fails, hardware may still have active MST entries,
- * but the EQ deregistration and IOMMU unmapping below prevent
- * stale hardware accesses from reaching kernel memory.
+ * terminal cleanup path (device removal, suspend, or init
+ * failure) — resources must be freed regardless. If teardown
+ * fails, hardware may still have active MST entries, but:
+ *
+ * - Interrupts: mana_hwc_destroy_cq() below calls
+ * mana_gd_deregister_irq() which removes the HWC EQ from
+ * the interrupt dispatch list via list_del_rcu() +
+ * synchronize_rcu(). After that, no interrupt handler can
+ * invoke handle_resp() or access CQ/RQ buffers — even if
+ * the IRQ is shared with data path queues.
+ *
+ * - DMA: mana_hwc_destroy_wq() frees DMA buffers via
+ * dma_free_coherent() which unmaps the IOVA from the
+ * IOMMU. Any stale hardware DMA to the old address
+ * faults at the IOMMU, not in kernel memory.
*/
if (hwc->setup_active) {
int td_err = mana_smc_teardown_hwc(&gc->shm_channel, false);
@@ -1042,6 +1239,9 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
* them (unsafe).
*
* but leak all DMA buffers to prevent corruption.
+ * Also leak the EQ IRQ registration since freeing
+ * it safely requires accessing queue structures we're
+ * leaving allocated.
*/
dev_warn(gc->dev,
@@ -1061,8 +1261,11 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
}
/* After SMC teardown, no more hardware events should arrive.
- * Force-complete any remaining in-flight senders so they can
- * exit and drop their refs.
+ * If teardown failed, handle_resp() may still race with this
+ * loop via a late interrupt — this is safe because the per-slot
+ * refcount model tolerates a concurrent complete() and both
+ * paths (handle_resp and this loop) will correctly drop their
+ * respective refs without double-releasing the slot.
*/
if (hwc->caller_ctx) {
struct hwc_caller_ctx *ctx;
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 01e845237b6a..a4eac6f7c366 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -708,6 +708,9 @@ enum {
/* Driver supports dynamic interrupt moderation - DIM */
#define GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION BIT(28)
+/* Driver supports dynamic queue depth for HWC */
+#define GDMA_DRV_CAP_FLAG_1_DYN_HWC_QUEUE_DEPTH BIT(29)
+
#define GDMA_DRV_CAP_FLAGS1 \
(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \
GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX | \
@@ -723,6 +726,7 @@ enum {
GDMA_DRV_CAP_FLAG_1_PROBE_RECOVERY | \
GDMA_DRV_CAP_FLAG_1_HANDLE_STALL_SQ_RECOVERY | \
GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY | \
+ GDMA_DRV_CAP_FLAG_1_DYN_HWC_QUEUE_DEPTH | \
GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT | \
GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION)
diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h
index bc62d1ce7bd4..fb9725e30c8c 100644
--- a/include/net/mana/hw_channel.h
+++ b/include/net/mana/hw_channel.h
@@ -197,7 +197,7 @@ struct hw_channel_context {
u32 max_req_msg_size;
u32 max_resp_msg_size;
- u16 hwc_init_q_depth_max;
+ u32 hwc_init_q_depth_max;
u32 hwc_init_max_req_msg_size;
u32 hwc_init_max_resp_msg_size;
--
2.43.0
^ permalink raw reply related
* [bug report] net: mana: Add support for auxiliary device
From: Dan Carpenter @ 2026-07-15 5:52 UTC (permalink / raw)
To: Long Li; +Cc: linux-hyperv
Hello Long Li,
Commit a69839d4327d ("net: mana: Add support for auxiliary device")
from Nov 3, 2022 (linux-next), leads to the following Smatch static
checker warning:
drivers/net/ethernet/microsoft/mana/mana_en.c:3893 add_adev()
warn: 'madev' was already freed. (line 3868)
drivers/net/ethernet/microsoft/mana/mana_en.c
3848 static int add_adev(struct gdma_dev *gd, const char *name)
3849 {
3850 struct auxiliary_device *adev;
3851 struct mana_adev *madev;
3852 int ret;
3853 int id;
3854
3855 madev = kzalloc_obj(*madev);
3856 if (!madev)
3857 return -ENOMEM;
3858
3859 adev = &madev->adev;
3860 ret = mana_adev_idx_alloc();
3861 if (ret < 0)
3862 goto idx_fail;
3863 id = ret;
3864 adev->id = id;
3865
3866 adev->name = name;
3867 adev->dev.parent = gd->gdma_context->dev;
3868 adev->dev.release = adev_release;
The adev_release() is assigned here.
3869 madev->mdev = gd;
3870
3871 ret = auxiliary_device_init(adev);
3872 if (ret)
3873 goto init_fail;
3874
3875 /* madev is owned by the auxiliary device */
3876 madev = NULL;
3877 ret = auxiliary_device_add(adev);
3878 if (ret)
3879 goto add_fail;
This goto...
3880
3881 gd->adev = adev;
3882 dev_dbg(gd->gdma_context->dev,
3883 "Auxiliary device added successfully\n");
3884 return 0;
3885
3886 add_fail:
3887 auxiliary_device_uninit(adev);
Free adev.
3888
3889 init_fail:
3890 mana_adev_idx_free(id);
3891
3892 idx_fail:
--> 3893 kfree(madev);
Double free.
3894
3895 return ret;
3896 }
This email is a free service from the Smatch-CI project [smatch.sf.net].
regards,
dan carpenter
^ permalink raw reply
* Re: RE: [PATCH 1/2] hv: vmbus: Replace lockdep_hardirq_threaded() with lockdep annotation
From: Sebastian Andrzej Siewior @ 2026-07-15 14:32 UTC (permalink / raw)
To: Michael Kelley
Cc: linux-hyperv@vger.kernel.org, linux-rt-devel@lists.linux.dev,
K. Y. Srinivasan, Dexuan Cui, Haiyang Zhang, Jan Kiszka, Long Li,
Wei Liu
In-Reply-To: <SN6PR02MB4157524E39FC4ABD058A4A52D45FA@SN6PR02MB4157.namprd02.prod.outlook.com>
On 2026-04-04 01:22:08 [+0000], Michael Kelley wrote:
> Nit: For historical consistency, use "Drivers: hv: vmbus:" as the prefix for the
> patch "Subject:" line. Same in Patch 2 of this series.
okay.
> Also, any reason not to have copied linux-kernel@vger.kernel.org? I know this
> is pretty much just a Hyper-V thing, but I would have liked to see what the
> Sashiko AI did with these two patches. :-)
Can do.
> > lockdep_hardirq_threaded() is supposed to be used within IRQ core code
> > and not within drivers. It is not obvious from within the driver, that
> > this is the only interrupt service routing and that it is not shared
> > handler.
>
> I presume you meant "routine". And what do you mean by "the only interrupt
> service routine"? And why is the lack of obviousness relevant here? I don't have
> deep expertise in lockdep, but evidently there's some conclusion to reach and it
> would have helped me to have it spelled out.
You use lockdep_hardirq_threaded() which marks the hardirq as a thread
for lockdep purposes. The reason is that the IRQ core will force-thread
the handler and the whole routine will be threaded. It does invoke the
function and the very begin and nothing else will be done.
vmbus_isr() is using it before __vmbus_isr() is invoked and there can be
other functions/ ISRs that are invoked afterwards which would be wrongly
recognized.
To make it more concrete: sysvec_hyperv_callback() invokes mshv_handler
and vmbus_handler. Would vmbus_handler() be invoked first then it will
"spill" this lockdep attribute into mshv_handler(). Nested locks in
mshv_handler() would not be visible then.
This override limits it only to __vmbus_isr() while the RT case above is
using vmbus_irqd_wake().
> >
> > Replace lockdep_hardirq_threaded() with a lockdep annotation limiting
> > threaded context on PREEMPT_RT to __vmbus_isr().
>
> Again, I'm not clear what "limiting threaded context" means. But see my
> additional comment further down.
Does the explanation above make it clear?
> >
> > Fixes: f8e6343b7a89c ("Drivers: hv: vmbus: Use kthread for vmbus interrupts on PREEMPT_RT")
> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> > ---
> > drivers/hv/vmbus_drv.c | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> > index bc4fc1951ae1c..e44275370ac2a 100644
> > --- a/drivers/hv/vmbus_drv.c
> > +++ b/drivers/hv/vmbus_drv.c
> > @@ -1407,8 +1407,11 @@ void vmbus_isr(void)
> > if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
> > vmbus_irqd_wake();
> > } else {
> > - lockdep_hardirq_threaded();
>
> I see two similar occurrences of LD_WAIT_CONFIG in the kernel:
> __kfree_rcu_sheaf() and adjacent to printk_legacy_allow_spinlock_enter().
> Both occurrences have a multi-line comment explaining the "why". I'd like
> to see a similar comment here so that drive-by readers of the code have
> some idea of what's going on. My suggestion is something like this:
>
> vmbus_isr() always runs at hard IRQ level -- the interrupt is not threaded. It
> calls __vmbus_isr() here, which may obtain the spinlock_t sched_lock for
> a VMBus channel in vmbus_chan_sched(). If CONFIG_PROVE_LOCKING=y,
> lockdep complains because obtaining spinlock_t's is not permitted at hard
> IRQ level in PREEMPT_RT configurations. However, the PREEMPT_RT path
> is handled separately above, so there's actually not a problem. Tell
> lockdep that acquiring the spinlock_t is valid by temporarily raising
> the wait-type to LD_WAIT_CONFIG using the "fake" lock vmbus_map.
> If lockdep is not enabled, the acquire & release of the fake lock are no-ops,
> so performance is not impacted.
>
> Please review my suggested text and revise as appropriate, as I'm far
> from an expert on any of this. The above is based on what I've learned
> just now from a bit of research.
What about?
vmbus_isr is never force-threaded and always invoked at hard IRQ level.
__vmbus_isr() below can acquire a spinlock_t which becomes a sleeping
lock and must not be acquired in this context. Therefore on PREEMPT_RT
this will be threaded via vmbus_irqd_wake(). On non-PREEMPT the
annotation lets lockdep know that acquiring a spinlock_t is not an
issue.
> And thanks for jumping in and making all this better ....
so I've been looking at __vmbus_isr() and vmbus_chan_sched() seems fine.
vmbus_message_sched() on the handler invokes hv_stimer0_isr() which
should not be done in the thread but in the hardirq. Is this the only
timer in the system or some legacy thingy that is no longer used?
And there is a lot of tasklet involved which mandates a
local_bh_disable()/enable() in run_vmbus_irqd() around the function or
everything gets pushed into ksoftirqd.
> Michael
Sebastian
^ permalink raw reply
* [PATCH v9 0/8] mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings
From: Stanislav Kinsburskii @ 2026-07-15 18:15 UTC (permalink / raw)
To: airlied, akhilesh, akpm, corbet, dakr, david, decui, haiyangz,
jgg, kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, skinsburskii, surenb,
tzimmermann, vbabka, wei.liu, skinsburskii
Cc: dri-devel, intel-xe, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
This series extends the HMM framework to support userfaultfd-backed memory
by allowing the mmap read lock to be dropped during hmm_range_fault().
Some page fault handlers — most notably userfaultfd — require the mmap lock
to be released so that userspace can resolve the fault. The current HMM
interface never sets FAULT_FLAG_ALLOW_RETRY, making it impossible to fault
in pages from userfaultfd-registered regions.
This series follows the established int *locked pattern from
get_user_pages_remote() in mm/gup.c. A new helper function,
hmm_range_fault_locked(), accepts an int *locked parameter. When the
mmap lock is dropped during fault resolution (VM_FAULT_RETRY or
VM_FAULT_COMPLETED), the function returns 0 with *locked = 0, signalling
the caller to restart its walk. The existing hmm_range_fault() is
refactored into a thin wrapper that passes NULL, preserving current
behavior for all existing callers.
Possible approaches to lift this limitation are documented in
Documentation/mm/hmm.rst.
Changes in v9:
- Folded the fixups into the full 8-patch series instead of sending a
separate fixup series.
- Clarified that the HMM timeout bounds repeated HMM/mmu-notifier retry
attempts, with the helper refreshing range->notifier_seq internally.
- Kept nouveau’s explicit outer deadline because its retry loop runs in a GPU
fault worker and cannot rely on fatal signals from the faulting process.
- Converted amdxdna, and GPU SVM callers to pass the shole timeout budget to
hmm_range_fault_unlocked_timeout().
Changes in v8:
- Make hmm_range_fault_unlocked_timeout() the primary documented HMM
range-fault API, and move hmm_range_fault() into the “use only if the
caller really must hold mmap_lock” category.
- Clarify that the timeout is a retry budget for repeated mmu-notifier
invalidation retries. HMM does not interrupt an in-progress page fault
when the timeout expires.
- Restart the retry timeout only when handle_mm_fault() dropped
mmap_lock, because that indicates a lock-dropping fault handler such as
userfaultfd made progress. Ordinary -EBUSY retries keep the existing
deadline.
- Remove the attempted timeout selftest. The remaining selftest covers the
intended userfaultfd path by resolving missing-page faults through
HMM_DMIRROR_READ_UNLOCKED and hmm_range_fault_unlocked_timeout(..., 0).
Changes in v7:
- Replaced the unlocked HMM API with
hmm_range_fault_unlocked_timeout(). The helper now takes a timeout in
jiffies, with 0 meaning retry indefinitely.
- Moved -EBUSY retry handling into the HMM helper for the unlocked path.
The helper refreshes range->notifier_seq internally before each retry.
- Switched the unlocked path to mmap_read_lock_killable() and return
-EINTR if mmap lock acquisition is interrupted or a fatal signal is
pending during retry handling.
- Removed the redundant non-timeout hmm_range_fault_unlocked() interface.
- Updated Documentation/mm/hmm.rst and kernel-doc to describe the timeout API
and the intended caller pattern.
- Updated the HMM selftests to use hmm_range_fault_unlocked_timeout()
only, including coverage for the finite-timeout path.
- Added in-tree users of the new helper:
- mshv
- nouveau
- RDMA/umem
- amdxdna
- drm/gpusvm
- Preserved each converted driver’s existing timeout convention:
- unbounded retry where the old code retried indefinitely,
- HMM_RANGE_DEFAULT_TIMEOUT where the old code used that budget,
- existing driver-specific timeout return values such as -ETIME.
- Used max_t(long, timeout - jiffies, 1) when passing remaining time from
absolute jiffies deadlines to avoid unsigned underflow while keeping a
minimum one-jiffy retry window.
- Left callers on hmm_range_fault() when they already need to hold
mmap_lock across surrounding work, such as drm_gpusvm_check_pages().
Changes in v6:
- Reworked the new API from the external int *locked pattern to
hmm_range_fault_unlocked(), which owns mmap_read_lock() internally.
- Changed the dropped-lock contract: hmm_range_fault_unlocked() now returns
-EBUSY when the mmap lock is dropped, and callers restart with a fresh
mmu_interval_read_begin() sequence.
- Kept hmm_range_fault() as the locked variant for existing users, preserving
its caller-held mmap lock contract.
- Added an in-tree user by converting the MSHV region fault path to
hmm_range_fault_unlocked().
- Updated Documentation/mm/hmm.rst and kernel-doc to describe the unlocked
helper and retry pattern.
- Updated commit messages to match the new API and return semantics.
- Kept the userfaultfd HMM selftest using the test_hmm unlocked read ioctl
path.
Changes in v5:
- Rework hmm_range_fault_unlockable() retry handling to retry
VM_FAULT_RETRY internally with FAULT_FLAG_TRIED set, matching the
fixup_user_fault() pattern and avoiding repeated first-retry lock drops.
- Distinguish VM_FAULT_RETRY from VM_FAULT_COMPLETED: retry faults now
reacquire the mmap lock internally, while completed faults return to the
caller with *locked = 0 so the caller can restart with a fresh notifier
sequence.
- Document the two *locked return states, including the -EINTR case when a
fatal signal is pending after the mmap lock has already been dropped.
- Update comments around HMM_FAULT_UNLOCKED and the HMM fault loop to match
the current hmm_range_fault_unlockable() implementation.
Changes in v4:
- Rebased on 7.2-rc1
Changes in v3:
- Return -EFAULT from dmirror_fault_unlockable() when the mirrored mm can
no longer be pinned.
- Add an eventfd stop signal for the userfaultfd handler thread to avoid
waiting for the poll timeout on successful test completion.
Changes in v2:
- Split into a preparatory refactor (new patch 1) that moves
handle_mm_fault() out of the walk callbacks, plus a smaller feature
patch on top. Suggested by David Hildenbrand.
- Hugetlb regions are now supported on the unlockable path; the v1
-EFAULT short-circuit and the hugetlb_vma_lock_read drop/retake
dance are gone.
- Distinct internal sentinels for "needs fault" (HMM_FAULT_PENDING)
and "lock dropped" (HMM_FAULT_UNLOCKED).
- Outer loop now re-walks after a successful internal fault so the
faulted pfns end up in range->hmm_pfns.
- Kernel-doc on hmm_range_fault_unlockable() and the
Documentation/mm/hmm.rst example match the implementation.
- Dropped the mshv driver conversion (v1 patch 2); will post
separately.
- Selftest converted to drive the path through test_hmm with a
userfaultfd handler (new HMM_DMIRROR_READ_UNLOCKABLE ioctl).
---
Stanislav Kinsburskii (8):
mm/hmm: move page fault handling out of walk callbacks
mm/hmm: add hmm_range_fault_unlocked_timeout() for mmap lock-drop support
selftests/mm: add HMM test for mmap lock-dropping faults
mshv: Use hmm_range_fault_unlocked_timeout() for region faults
drm/nouveau: Use hmm_range_fault_unlocked_timeout() for SVM faults
RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults
accel/amdxdna: Use hmm_range_fault_unlocked_timeout() for range population
drm/gpusvm: Use hmm_range_fault_unlocked_timeout() for range faults
Documentation/mm/hmm.rst | 79 +++++++---
drivers/accel/amdxdna/aie2_ctx.c | 23 ---
drivers/gpu/drm/drm_gpusvm.c | 61 +-------
drivers/gpu/drm/nouveau/nouveau_svm.c | 20 +-
drivers/hv/mshv_regions.c | 54 +------
drivers/infiniband/core/umem_odp.c | 18 +-
include/linux/hmm.h | 2
lib/test_hmm.c | 107 +++++++++++++
lib/test_hmm_uapi.h | 1
mm/hmm.c | 259 +++++++++++++++++++++++++-------
tools/testing/selftests/mm/hmm-tests.c | 150 +++++++++++++++++++
11 files changed, 551 insertions(+), 223 deletions(-)
^ permalink raw reply
* [PATCH v9 1/8] mm/hmm: move page fault handling out of walk callbacks
From: Stanislav Kinsburskii @ 2026-07-15 18:15 UTC (permalink / raw)
To: airlied, akhilesh, akpm, corbet, dakr, david, decui, haiyangz,
jgg, kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, skinsburskii, surenb,
tzimmermann, vbabka, wei.liu, skinsburskii
Cc: dri-devel, intel-xe, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <178413903133.1155966.3904063656020521607.stgit@skinsburskii>
hmm_range_fault() currently triggers page faults from inside the page-table
walk callbacks: hmm_vma_walk_pmd(), hmm_vma_walk_pud(),
hmm_vma_walk_hugetlb_entry() and the pte-level helper all call
hmm_vma_fault(), which in turn calls handle_mm_fault() while the walker
still holds nested locks. The pte spinlock is dropped explicitly by each
caller, and the hugetlb path manually drops and retakes
hugetlb_vma_lock_read around the fault to dodge a deadlock against the walk
framework's unconditional unlock.
This layering does not extend cleanly to fault handlers that may release
mmap_lock (VM_FAULT_RETRY, VM_FAULT_COMPLETED). If the lock is dropped
while walk_page_range() is mid-traversal, the VMA can be freed before the
walk framework's matching hugetlb_vma_unlock_read(), turning that unlock
into a use-after-free.
Split the responsibilities the way get_user_pages() does. Walk callbacks
become inspect-only: when they detect a range that needs to be faulted in,
they record it in struct hmm_vma_walk and return a private sentinel
(HMM_FAULT_PENDING). The outer loop in hmm_range_fault() then drops out of
walk_page_range(), invokes a new helper hmm_do_fault() that calls
handle_mm_fault() with only mmap_lock held, and restarts the walk so the
now-present entries are collected into hmm_pfns.
No functional change for existing callers. As a side effect the hugetlb
callback no longer needs the hugetlb_vma_{un}lock_read dance, and every
fault-path exit from the callbacks now releases the pte spinlock on a
single, common path. This refactor is also a precursor for adding an
unlockable variant of hmm_range_fault() in a follow-up patch.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
mm/hmm.c | 118 +++++++++++++++++++++++++++++++++++++++-----------------------
1 file changed, 75 insertions(+), 43 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index e5c1f4deed24..bc9361a715fa 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -33,8 +33,17 @@
struct hmm_vma_walk {
struct hmm_range *range;
unsigned long last;
+ unsigned long end;
+ unsigned int required_fault;
};
+/*
+ * Internal sentinel returned by walk callbacks when they need a page fault.
+ * The callback stores end/required_fault in hmm_vma_walk; the outer loop
+ * consumes the sentinel and never propagates it to the caller.
+ */
+#define HMM_FAULT_PENDING -EAGAIN
+
enum {
HMM_NEED_FAULT = 1 << 0,
HMM_NEED_WRITE_FAULT = 1 << 1,
@@ -60,37 +69,25 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end,
}
/*
- * hmm_vma_fault() - fault in a range lacking valid pmd or pte(s)
- * @addr: range virtual start address (inclusive)
- * @end: range virtual end address (exclusive)
- * @required_fault: HMM_NEED_* flags
- * @walk: mm_walk structure
- * Return: -EBUSY after page fault, or page fault error
+ * hmm_record_fault() - record a range that needs to be faulted in
*
- * This function will be called whenever pmd_none() or pte_none() returns true,
- * or whenever there is no page directory covering the virtual address range.
+ * Called by the walk callbacks when they discover that part of the range
+ * needs a page fault. The callback records what to fault and returns
+ * HMM_FAULT_PENDING; the outer loop in hmm_range_fault() drops back out of
+ * walk_page_range() and invokes handle_mm_fault() from a context where no
+ * page-table or hugetlb_vma_lock is held.
*/
-static int hmm_vma_fault(unsigned long addr, unsigned long end,
- unsigned int required_fault, struct mm_walk *walk)
+static int hmm_record_fault(unsigned long addr, unsigned long end,
+ unsigned int required_fault,
+ struct mm_walk *walk)
{
struct hmm_vma_walk *hmm_vma_walk = walk->private;
- struct vm_area_struct *vma = walk->vma;
- unsigned int fault_flags = FAULT_FLAG_REMOTE;
WARN_ON_ONCE(!required_fault);
hmm_vma_walk->last = addr;
-
- if (required_fault & HMM_NEED_WRITE_FAULT) {
- if (!(vma->vm_flags & VM_WRITE))
- return -EPERM;
- fault_flags |= FAULT_FLAG_WRITE;
- }
-
- for (; addr < end; addr += PAGE_SIZE)
- if (handle_mm_fault(vma, addr, fault_flags, NULL) &
- VM_FAULT_ERROR)
- return -EFAULT;
- return -EBUSY;
+ hmm_vma_walk->end = end;
+ hmm_vma_walk->required_fault = required_fault;
+ return HMM_FAULT_PENDING;
}
static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk,
@@ -174,7 +171,7 @@ static int hmm_vma_walk_hole(unsigned long addr, unsigned long end,
return hmm_pfns_fill(addr, end, range, HMM_PFN_ERROR);
}
if (required_fault)
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
return hmm_pfns_fill(addr, end, range, 0);
}
@@ -209,7 +206,7 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr,
required_fault =
hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, cpu_flags);
if (required_fault)
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
pfn = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT);
for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) {
@@ -328,7 +325,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr,
fault:
pte_unmap(ptep);
/* Fault any virtual address we were asked to fault */
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
}
#ifdef CONFIG_ARCH_HAS_PMD_SOFTLEAVES
@@ -371,7 +368,7 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *walk, unsigned long start,
npages, 0);
if (required_fault) {
if (softleaf_is_device_private(entry))
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
else
return -EFAULT;
}
@@ -517,7 +514,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end,
npages, cpu_flags);
if (required_fault) {
spin_unlock(ptl);
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
}
pfn = pud_pfn(pud) + ((addr & ~PUD_MASK) >> PAGE_SHIFT);
@@ -564,21 +561,8 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask,
required_fault =
hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, cpu_flags);
if (required_fault) {
- int ret;
-
spin_unlock(ptl);
- hugetlb_vma_unlock_read(vma);
- /*
- * Avoid deadlock: drop the vma lock before calling
- * hmm_vma_fault(), which will itself potentially take and
- * drop the vma lock. This is also correct from a
- * protection point of view, because there is no further
- * use here of either pte or ptl after dropping the vma
- * lock.
- */
- ret = hmm_vma_fault(addr, end, required_fault, walk);
- hugetlb_vma_lock_read(vma);
- return ret;
+ return hmm_record_fault(addr, end, required_fault, walk);
}
pfn = pte_pfn(entry) + ((start & ~hmask) >> PAGE_SHIFT);
@@ -637,6 +621,44 @@ static const struct mm_walk_ops hmm_walk_ops = {
.walk_lock = PGWALK_RDLOCK,
};
+/*
+ * hmm_do_fault - fault in a range recorded by a walk callback
+ *
+ * Called from the outer loop in hmm_range_fault() after a callback
+ * returned HMM_FAULT_PENDING. At this point we hold only mmap_lock;
+ * the page-table spinlock and any hugetlb_vma_lock acquired by the walk
+ * framework have already been released by the unwind.
+ *
+ * Returns -EBUSY on success (all pages faulted, caller should re-walk).
+ * Returns a negative errno on failure.
+ */
+static int hmm_do_fault(struct mm_struct *mm,
+ struct hmm_vma_walk *hmm_vma_walk)
+{
+ unsigned long addr = hmm_vma_walk->last;
+ unsigned long end = hmm_vma_walk->end;
+ unsigned int required_fault = hmm_vma_walk->required_fault;
+ unsigned int fault_flags = FAULT_FLAG_REMOTE;
+ struct vm_area_struct *vma;
+
+ vma = vma_lookup(mm, addr);
+ if (!vma)
+ return -EFAULT;
+
+ if (required_fault & HMM_NEED_WRITE_FAULT) {
+ if (!(vma->vm_flags & VM_WRITE))
+ return -EPERM;
+ fault_flags |= FAULT_FLAG_WRITE;
+ }
+
+ for (; addr < end; addr += PAGE_SIZE)
+ if (handle_mm_fault(vma, addr, fault_flags, NULL) &
+ VM_FAULT_ERROR)
+ return -EFAULT;
+
+ return -EBUSY;
+}
+
/**
* hmm_range_fault - try to fault some address in a virtual address range
* @range: argument structure
@@ -674,6 +696,16 @@ int hmm_range_fault(struct hmm_range *range)
return -EBUSY;
ret = walk_page_range(mm, hmm_vma_walk.last, range->end,
&hmm_walk_ops, &hmm_vma_walk);
+ /*
+ * When HMM_FAULT_PENDING is returned a walk callback
+ * recorded a range that needs handle_mm_fault();
+ * hmm_do_fault() runs the fault outside walk_page_range()
+ * (so no page-table or hugetlb_vma_lock is held) and
+ * returns -EBUSY so the loop re-walks and picks up the
+ * now-present entries.
+ */
+ if (ret == HMM_FAULT_PENDING)
+ ret = hmm_do_fault(mm, &hmm_vma_walk);
/*
* When -EBUSY is returned the loop restarts with
* hmm_vma_walk.last set to an address that has not been stored
^ permalink raw reply related
* [PATCH v9 2/8] mm/hmm: add hmm_range_fault_unlocked_timeout() for mmap lock-drop support
From: Stanislav Kinsburskii @ 2026-07-15 18:16 UTC (permalink / raw)
To: airlied, akhilesh, akpm, corbet, dakr, david, decui, haiyangz,
jgg, kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, skinsburskii, surenb,
tzimmermann, vbabka, wei.liu, skinsburskii
Cc: dri-devel, intel-xe, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <178413903133.1155966.3904063656020521607.stgit@skinsburskii>
hmm_range_fault() requires the caller to hold the mmap read lock for the
duration of the call. This is incompatible with mappings whose fault
handler may release the mmap lock, notably userfaultfd-managed regions,
where handle_mm_fault() can return VM_FAULT_RETRY or VM_FAULT_COMPLETED
after dropping the lock. Drivers that need to populate device page tables
for such mappings have no way to do so today.
Add hmm_range_fault_unlocked_timeout() for callers that do not need to hold
mmap_lock across any work outside the HMM fault itself. The helper takes
mmap_read_lock_killable() internally, calls the common HMM fault
implementation, and releases the lock before returning if it is still held.
The timeout is specified in jiffies; passing 0 retries indefinitely, while
a non-zero timeout makes the helper return -EBUSY when the retry budget
expires.
When handle_mm_fault() drops mmap_lock, or when the range is invalidated,
hmm_range_fault_unlocked_timeout() refreshes range->notifier_seq and
retries the walk internally. If the lock was dropped, the retry deadline is
also restarted because a lock-dropping fault handler made progress.
Ordinary -EBUSY retries keep the existing deadline, preserving the caller's
timeout policy for repeated mmu-notifier invalidations.
The caller only needs to perform the usual post-success
mmu_interval_read_retry() check while holding its update lock before
consuming the pfns. If mmap_lock acquisition is interrupted or a fatal
signal is pending during retry handling, -EINTR is returned instead.
The common implementation conditionally sets FAULT_FLAG_ALLOW_RETRY and
FAULT_FLAG_KILLABLE only for hmm_range_fault_unlocked_timeout(). The
existing hmm_range_fault() path still passes no locked state, does not
allow handle_mm_fault() to drop mmap_lock, and remains a thin wrapper
preserving the existing API contract for current callers.
The previous refactor that moved page fault handling out of the page-table
walk callbacks is what makes this change small. Faults now run after
walk_page_range() has unwound, with only mmap_lock held, so dropping it
does not interact with the walker's pte spinlock or hugetlb_vma_lock.
Hugetlb regions therefore participate in the unlocked path uniformly with
PTE- and PMD-level mappings; no special case is required.
Documentation/mm/hmm.rst is updated with a description of the new API and
the recommended caller pattern.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
Documentation/mm/hmm.rst | 79 ++++++++++++++++------
include/linux/hmm.h | 2 +
mm/hmm.c | 165 ++++++++++++++++++++++++++++++++++++++--------
3 files changed, 194 insertions(+), 52 deletions(-)
diff --git a/Documentation/mm/hmm.rst b/Documentation/mm/hmm.rst
index 7d61b7a8b65b..e021218ada58 100644
--- a/Documentation/mm/hmm.rst
+++ b/Documentation/mm/hmm.rst
@@ -156,42 +156,57 @@ During the ops->invalidate() callback the device driver must perform the
update action to the range (mark range read only, or fully unmap, etc.). The
device must complete the update before the driver callback returns.
-When the device driver wants to populate a range of virtual addresses, it can
-use::
+When the device driver wants to populate a range of virtual addresses, the
+normal interface is::
- int hmm_range_fault(struct hmm_range *range);
+ int hmm_range_fault_unlocked_timeout(struct hmm_range *range,
+ unsigned long timeout);
It will trigger a page fault on missing or read-only entries if write access is
requested (see below). Page faults use the generic mm page fault code path just
-like a CPU page fault. The usage pattern is::
+like a CPU page fault.
+
+The caller must not hold ``mmap_read_lock`` before the call.
+``hmm_range_fault_unlocked_timeout()`` takes the mmap read lock internally and
+allows ``handle_mm_fault()`` to drop it during fault handling. This is required
+for VMAs whose fault handlers may release the mmap lock, for example regions
+managed by ``userfaultfd``.
+
+If the mmap lock is dropped or the range is invalidated, the function refreshes
+``range->notifier_seq`` and restarts the walk internally. ``-EINTR`` is returned
+if mmap lock acquisition is interrupted or a fatal signal is pending during
+retry handling.
+
+The timeout is specified in jiffies; passing ``0`` means retry indefinitely. The
+timeout exists to preserve caller policy for repeated mmu-notifier invalidation
+and is checked between retry attempts. HMM does not interrupt page fault
+handling when the timeout expires, but returns ``-EBUSY`` if the retry budget is
+exhausted before a stable range is obtained.
+
+The usage pattern is::
int driver_populate_range(...)
{
struct hmm_range range;
+ unsigned long timeout;
...
+ timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
range.notifier = &interval_sub;
range.start = ...;
range.end = ...;
range.hmm_pfns = ...;
- if (!mmget_not_zero(interval_sub->notifier.mm))
+ if (!mmget_not_zero(interval_sub.mm))
return -EFAULT;
again:
- range.notifier_seq = mmu_interval_read_begin(&interval_sub);
- mmap_read_lock(mm);
- ret = hmm_range_fault(&range);
- if (ret) {
- mmap_read_unlock(mm);
- if (ret == -EBUSY)
- goto again;
- return ret;
- }
- mmap_read_unlock(mm);
+ ret = hmm_range_fault_unlocked_timeout(&range, timeout);
+ if (ret)
+ goto out_put;
take_lock(driver->update);
- if (mmu_interval_read_retry(&ni, range.notifier_seq) {
+ if (mmu_interval_read_retry(range.notifier, range.notifier_seq)) {
release_lock(driver->update);
goto again;
}
@@ -200,13 +215,31 @@ like a CPU page fault. The usage pattern is::
* under the update lock */
release_lock(driver->update);
- return 0;
+ ret = 0;
+
+ out_put:
+ mmput(interval_sub.mm);
+ return ret;
}
The driver->update lock is the same lock that the driver takes inside its
invalidate() callback. That lock must be held before calling
mmu_interval_read_retry() to avoid any race with a concurrent CPU page table
-update.
+update. The retry check must use the same notifier and sequence number stored
+in ``range`` by ``hmm_range_fault_unlocked_timeout()``.
+
+Holding the mmap lock across HMM faults
+=======================================
+
+Most callers should use ``hmm_range_fault_unlocked_timeout()``. If a driver
+really needs to hold the mmap lock across work outside HMM, it can use::
+
+ int hmm_range_fault(struct hmm_range *range);
+
+The mmap lock must be held by the caller and will remain held on return. This
+interface cannot support VMAs whose fault handlers need to drop the mmap lock.
+New callers should prefer ``hmm_range_fault_unlocked_timeout()`` unless they
+have a specific requirement to keep the mmap lock held across the call.
Leverage default_flags and pfn_flags_mask
=========================================
@@ -221,8 +254,8 @@ permission, it sets::
range->default_flags = HMM_PFN_REQ_FAULT;
range->pfn_flags_mask = 0;
-and calls hmm_range_fault() as described above. This will fill fault all pages
-in the range with at least read permission.
+and calls the HMM range fault helper as described above. This will fault
+all pages in the range with at least read permission.
Now let's say the driver wants to do the same except for one page in the range for
which it wants to have write permission. Now driver set::
@@ -236,9 +269,9 @@ address == range->start + (index_of_write << PAGE_SHIFT) it will fault with
write permission i.e., if the CPU pte does not have write permission set then HMM
will call handle_mm_fault().
-After hmm_range_fault completes the flag bits are set to the current state of
-the page tables, ie HMM_PFN_VALID | HMM_PFN_WRITE will be set if the page is
-writable.
+After the HMM range fault helper completes the flag bits are set to the
+current state of the page tables, ie HMM_PFN_VALID | HMM_PFN_WRITE will be
+set if the page is writable.
Represent and manage device memory from core kernel point of view
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index db75ffc949a7..6f04e3932f5b 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -123,6 +123,8 @@ struct hmm_range {
* Please see Documentation/mm/hmm.rst for how to use the range API.
*/
int hmm_range_fault(struct hmm_range *range);
+int hmm_range_fault_unlocked_timeout(struct hmm_range *range,
+ unsigned long timeout);
/*
* HMM_RANGE_DEFAULT_TIMEOUT - default timeout (ms) when waiting for a range
diff --git a/mm/hmm.c b/mm/hmm.c
index bc9361a715fa..fc2e1cd0cb22 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -32,6 +32,7 @@
struct hmm_vma_walk {
struct hmm_range *range;
+ bool *locked;
unsigned long last;
unsigned long end;
unsigned int required_fault;
@@ -44,6 +45,14 @@ struct hmm_vma_walk {
*/
#define HMM_FAULT_PENDING -EAGAIN
+/*
+ * Internal sentinel returned by hmm_do_fault() when handle_mm_fault()
+ * completes a page fault with the mmap lock dropped. hmm_do_fault() sets
+ * *locked = false; the outer loop consumes the sentinel and never propagates
+ * it to the caller.
+ */
+#define HMM_FAULT_UNLOCKED -ENOLCK
+
enum {
HMM_NEED_FAULT = 1 << 0,
HMM_NEED_WRITE_FAULT = 1 << 1,
@@ -73,9 +82,9 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end,
*
* Called by the walk callbacks when they discover that part of the range
* needs a page fault. The callback records what to fault and returns
- * HMM_FAULT_PENDING; the outer loop in hmm_range_fault() drops back out of
- * walk_page_range() and invokes handle_mm_fault() from a context where no
- * page-table or hugetlb_vma_lock is held.
+ * HMM_FAULT_PENDING; the outer loop in hmm_range_fault_locked() drops
+ * back out of walk_page_range() and invokes handle_mm_fault() from a context
+ * where no page-table or hugetlb_vma_lock is held.
*/
static int hmm_record_fault(unsigned long addr, unsigned long end,
unsigned int required_fault,
@@ -624,7 +633,7 @@ static const struct mm_walk_ops hmm_walk_ops = {
/*
* hmm_do_fault - fault in a range recorded by a walk callback
*
- * Called from the outer loop in hmm_range_fault() after a callback
+ * Called from the outer loop in hmm_range_fault_locked() after a callback
* returned HMM_FAULT_PENDING. At this point we hold only mmap_lock;
* the page-table spinlock and any hugetlb_vma_lock acquired by the walk
* framework have already been released by the unwind.
@@ -641,6 +650,9 @@ static int hmm_do_fault(struct mm_struct *mm,
unsigned int fault_flags = FAULT_FLAG_REMOTE;
struct vm_area_struct *vma;
+ if (hmm_vma_walk->locked)
+ fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+
vma = vma_lookup(mm, addr);
if (!vma)
return -EFAULT;
@@ -651,37 +663,34 @@ static int hmm_do_fault(struct mm_struct *mm,
fault_flags |= FAULT_FLAG_WRITE;
}
- for (; addr < end; addr += PAGE_SIZE)
- if (handle_mm_fault(vma, addr, fault_flags, NULL) &
- VM_FAULT_ERROR)
- return -EFAULT;
+ for (; addr < end; addr += PAGE_SIZE) {
+ vm_fault_t ret;
+
+ ret = handle_mm_fault(vma, addr, fault_flags, NULL);
+
+ if (ret & (VM_FAULT_COMPLETED | VM_FAULT_RETRY)) {
+ *hmm_vma_walk->locked = false;
+ return HMM_FAULT_UNLOCKED;
+ }
+
+ if (ret & VM_FAULT_ERROR) {
+ int err = vm_fault_to_errno(ret, 0);
+
+ if (WARN_ON(!err))
+ err = -EINVAL;
+
+ return err;
+ }
+ }
return -EBUSY;
}
-/**
- * hmm_range_fault - try to fault some address in a virtual address range
- * @range: argument structure
- *
- * Returns 0 on success or one of the following error codes:
- *
- * -EINVAL: Invalid arguments or mm or virtual address is in an invalid vma
- * (e.g., device file vma).
- * -ENOMEM: Out of memory.
- * -EPERM: Invalid permission (e.g., asking for write and range is read
- * only).
- * -EBUSY: The range has been invalidated and the caller needs to wait for
- * the invalidation to finish.
- * -EFAULT: A page was requested to be valid and could not be made valid
- * ie it has no backing VMA or it is illegal to access
- *
- * This is similar to get_user_pages(), except that it can read the page tables
- * without mutating them (ie causing faults).
- */
-int hmm_range_fault(struct hmm_range *range)
+static int hmm_range_fault_locked(struct hmm_range *range, bool *locked)
{
struct hmm_vma_walk hmm_vma_walk = {
.range = range,
+ .locked = locked,
.last = range->start,
};
struct mm_struct *mm = range->notifier->mm;
@@ -704,8 +713,14 @@ int hmm_range_fault(struct hmm_range *range)
* returns -EBUSY so the loop re-walks and picks up the
* now-present entries.
*/
- if (ret == HMM_FAULT_PENDING)
+ if (ret == HMM_FAULT_PENDING) {
ret = hmm_do_fault(mm, &hmm_vma_walk);
+ if (ret == HMM_FAULT_UNLOCKED) {
+ if (fatal_signal_pending(current))
+ return -EINTR;
+ return -EBUSY;
+ }
+ }
/*
* When -EBUSY is returned the loop restarts with
* hmm_vma_walk.last set to an address that has not been stored
@@ -715,8 +730,100 @@ int hmm_range_fault(struct hmm_range *range)
} while (ret == -EBUSY);
return ret;
}
+
+/**
+ * hmm_range_fault - try to fault some address in a virtual address range
+ * @range: argument structure
+ *
+ * Returns 0 on success or one of the following error codes:
+ *
+ * -EINVAL: Invalid arguments or mm or virtual address is in an invalid vma
+ * (e.g., device file vma).
+ * -ENOMEM: Out of memory.
+ * -EPERM: Invalid permission (e.g., asking for write and range is read
+ * only).
+ * -EBUSY: The range has been invalidated and the caller needs to wait for
+ * the invalidation to finish.
+ * -EFAULT: A page was requested to be valid and could not be made valid
+ * ie it has no backing VMA or it is illegal to access
+ *
+ * This is similar to get_user_pages(), except that it can read the page tables
+ * without mutating them (ie causing faults).
+ *
+ * The mmap lock must be held by the caller and will remain held on return.
+ * New users should prefer hmm_range_fault_unlocked_timeout() unless they
+ * specifically need to keep the mmap lock held across the call. This helper
+ * cannot support VMAs whose fault handlers need to drop the mmap lock.
+ */
+int hmm_range_fault(struct hmm_range *range)
+{
+ return hmm_range_fault_locked(range, NULL);
+}
EXPORT_SYMBOL(hmm_range_fault);
+/**
+ * hmm_range_fault_unlocked_timeout - fault in a range with a retry timeout
+ * @range: argument structure
+ * @timeout: timeout in jiffies for internal -EBUSY retries, or 0 to retry
+ * indefinitely
+ *
+ * The caller must not hold the mmap lock. The function takes the mmap read
+ * lock internally and allows handle_mm_fault() to drop it during faults. If
+ * the mmap lock is dropped or the range is invalidated, the function refreshes
+ * range->notifier_seq and restarts the walk internally.
+ *
+ * Passing 0 for @timeout retries indefinitely. A non-zero @timeout is a caller
+ * policy limit for repeated mmu-notifier invalidation retries. HMM does not
+ * interrupt page fault handling when the timeout expires, but returns -EBUSY
+ * if the retry budget is exhausted before a stable range is obtained.
+ *
+ * Returns 0 on success or one of the error codes documented for
+ * hmm_range_fault(). -EINTR is returned if mmap_lock acquisition is
+ * interrupted or a fatal signal is pending during retry handling.
+ */
+int hmm_range_fault_unlocked_timeout(struct hmm_range *range,
+ unsigned long timeout)
+{
+ struct mm_struct *mm = range->notifier->mm;
+ unsigned long deadline = 0;
+ bool locked = false;
+ int ret;
+
+ do {
+ if (fatal_signal_pending(current))
+ return -EINTR;
+
+ if (timeout) {
+ /*
+ * If the previous fault dropped mmap_lock, then the fault
+ * handler made progress. Restart the retry timeout in that
+ * case, but keep the existing deadline for ordinary -EBUSY
+ * retries.
+ */
+ if (!locked)
+ deadline = jiffies + timeout;
+
+ if (time_after(jiffies, deadline))
+ return -EBUSY;
+ }
+
+ range->notifier_seq =
+ mmu_interval_read_begin(range->notifier);
+
+ ret = mmap_read_lock_killable(mm);
+ if (ret)
+ return ret;
+
+ locked = true;
+ ret = hmm_range_fault_locked(range, &locked);
+ if (locked)
+ mmap_read_unlock(mm);
+ } while (ret == -EBUSY);
+
+ return ret;
+}
+EXPORT_SYMBOL(hmm_range_fault_unlocked_timeout);
+
/**
* hmm_dma_map_alloc - Allocate HMM map structure
* @dev: device to allocate structure for
^ permalink raw reply related
* [PATCH v9 3/8] selftests/mm: add HMM test for mmap lock-dropping faults
From: Stanislav Kinsburskii @ 2026-07-15 18:16 UTC (permalink / raw)
To: airlied, akhilesh, akpm, corbet, dakr, david, decui, haiyangz,
jgg, kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, skinsburskii, surenb,
tzimmermann, vbabka, wei.liu, skinsburskii
Cc: dri-devel, intel-xe, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <178413903133.1155966.3904063656020521607.stgit@skinsburskii>
Add test_hmm coverage for the HMM lock-dropping fault path. The test module
gets a new HMM_DMIRROR_READ_UNLOCKED ioctl that calls
hmm_range_fault_unlocked_timeout() with a timeout of 0, exercising the
unbounded retry mode while allowing the mmap lock to be dropped during
fault handling.
Add a userfaultfd_read selftest that registers an anonymous mapping with
UFFDIO_REGISTER_MODE_MISSING, services the faults from a handler thread
with UFFDIO_COPY, and verifies that HMM can read back the data supplied by
the handler. This exercises the path where handle_mm_fault() drops
mmap_lock and hmm_range_fault_unlocked_timeout() restarts the walk
internally.
Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
lib/test_hmm.c | 107 +++++++++++++++++++++++
lib/test_hmm_uapi.h | 1
tools/testing/selftests/mm/hmm-tests.c | 150 ++++++++++++++++++++++++++++++++
3 files changed, 257 insertions(+), 1 deletion(-)
diff --git a/lib/test_hmm.c b/lib/test_hmm.c
index 45c0cb992218..6205fb313bd0 100644
--- a/lib/test_hmm.c
+++ b/lib/test_hmm.c
@@ -389,6 +389,67 @@ static int dmirror_range_fault(struct dmirror *dmirror,
return ret;
}
+static int dmirror_range_fault_unlocked(struct dmirror *dmirror,
+ struct hmm_range *range,
+ unsigned long timeout)
+{
+ int ret;
+
+ while (true) {
+ ret = hmm_range_fault_unlocked_timeout(range, timeout);
+ if (ret)
+ goto out;
+
+ mutex_lock(&dmirror->mutex);
+ if (mmu_interval_read_retry(range->notifier,
+ range->notifier_seq)) {
+ mutex_unlock(&dmirror->mutex);
+ continue;
+ }
+ break;
+ }
+
+ ret = dmirror_do_fault(dmirror, range);
+
+ mutex_unlock(&dmirror->mutex);
+out:
+ return ret;
+}
+
+static int dmirror_fault_unlocked(struct dmirror *dmirror,
+ unsigned long start,
+ unsigned long end, bool write,
+ unsigned long timeout)
+{
+ struct mm_struct *mm = dmirror->notifier.mm;
+ unsigned long addr;
+ unsigned long pfns[32];
+ struct hmm_range range = {
+ .notifier = &dmirror->notifier,
+ .hmm_pfns = pfns,
+ .pfn_flags_mask = 0,
+ .default_flags =
+ HMM_PFN_REQ_FAULT | (write ? HMM_PFN_REQ_WRITE : 0),
+ .dev_private_owner = dmirror->mdevice,
+ };
+ int ret = 0;
+
+ if (!mmget_not_zero(mm))
+ return -EFAULT;
+
+ for (addr = start; addr < end; addr = range.end) {
+ range.start = addr;
+ range.end = min(addr + (ARRAY_SIZE(pfns) << PAGE_SHIFT), end);
+
+ ret = dmirror_range_fault_unlocked(dmirror, &range, timeout);
+ if (ret)
+ break;
+ }
+
+ mmput(mm);
+ return ret;
+}
+
static int dmirror_fault(struct dmirror *dmirror, unsigned long start,
unsigned long end, bool write)
{
@@ -488,6 +549,48 @@ static int dmirror_read(struct dmirror *dmirror, struct hmm_dmirror_cmd *cmd)
return ret;
}
+static int dmirror_read_unlocked(struct dmirror *dmirror,
+ struct hmm_dmirror_cmd *cmd,
+ unsigned long timeout)
+{
+ struct dmirror_bounce bounce;
+ unsigned long start, end;
+ unsigned long size = cmd->npages << PAGE_SHIFT;
+ int ret;
+
+ start = cmd->addr;
+ end = start + size;
+ if (end < start)
+ return -EINVAL;
+
+ ret = dmirror_bounce_init(&bounce, start, size);
+ if (ret)
+ return ret;
+
+ while (1) {
+ mutex_lock(&dmirror->mutex);
+ ret = dmirror_do_read(dmirror, start, end, &bounce);
+ mutex_unlock(&dmirror->mutex);
+ if (ret != -ENOENT)
+ break;
+
+ start = cmd->addr + (bounce.cpages << PAGE_SHIFT);
+ ret = dmirror_fault_unlocked(dmirror, start, end, false, timeout);
+ if (ret)
+ break;
+ cmd->faults++;
+ }
+
+ if (ret == 0) {
+ if (copy_to_user(u64_to_user_ptr(cmd->ptr), bounce.ptr,
+ bounce.size))
+ ret = -EFAULT;
+ }
+ cmd->cpages = bounce.cpages;
+ dmirror_bounce_fini(&bounce);
+ return ret;
+}
+
static int dmirror_do_write(struct dmirror *dmirror, unsigned long start,
unsigned long end, struct dmirror_bounce *bounce)
{
@@ -1572,7 +1675,9 @@ static long dmirror_fops_unlocked_ioctl(struct file *filp,
dmirror->flags = cmd.npages;
ret = 0;
break;
-
+ case HMM_DMIRROR_READ_UNLOCKED:
+ ret = dmirror_read_unlocked(dmirror, &cmd, 0);
+ break;
default:
return -EINVAL;
}
diff --git a/lib/test_hmm_uapi.h b/lib/test_hmm_uapi.h
index f94c6d457338..ea9b0ec404fb 100644
--- a/lib/test_hmm_uapi.h
+++ b/lib/test_hmm_uapi.h
@@ -38,6 +38,7 @@ struct hmm_dmirror_cmd {
#define HMM_DMIRROR_CHECK_EXCLUSIVE _IOWR('H', 0x06, struct hmm_dmirror_cmd)
#define HMM_DMIRROR_RELEASE _IOWR('H', 0x07, struct hmm_dmirror_cmd)
#define HMM_DMIRROR_FLAGS _IOWR('H', 0x08, struct hmm_dmirror_cmd)
+#define HMM_DMIRROR_READ_UNLOCKED _IOWR('H', 0x09, struct hmm_dmirror_cmd)
#define HMM_DMIRROR_FLAG_FAIL_ALLOC (1ULL << 0)
diff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftests/mm/hmm-tests.c
index 6fccbdab02ee..5acb728666f8 100644
--- a/tools/testing/selftests/mm/hmm-tests.c
+++ b/tools/testing/selftests/mm/hmm-tests.c
@@ -29,6 +29,10 @@
#include <sys/mman.h>
#include <sys/ioctl.h>
#include <sys/time.h>
+#include <sys/syscall.h>
+#include <sys/eventfd.h>
+#include <linux/userfaultfd.h>
+#include <poll.h>
/*
* This is a private UAPI to the kernel test module so it isn't exported
@@ -2952,4 +2956,150 @@ TEST_F_TIMEOUT(hmm, benchmark_thp_migration, 120)
&thp_results, ®ular_results);
}
}
+/*
+ * Test that HMM can fault in pages backed by userfaultfd using the
+ * hmm_range_fault_unlocked_timeout() path with no timeout. This exercises
+ * the lock-drop retry logic in the HMM framework.
+ */
+struct uffd_thread_args {
+ int uffd;
+ int stop_fd;
+ void *page_buffer;
+ unsigned long page_size;
+};
+
+static void *uffd_handler_thread(void *arg)
+{
+ struct uffd_thread_args *args = arg;
+ struct uffd_msg msg;
+ struct uffdio_copy copy;
+ struct pollfd pollfd[2];
+ int ret;
+
+ pollfd[0].fd = args->uffd;
+ pollfd[0].events = POLLIN;
+ pollfd[1].fd = args->stop_fd;
+ pollfd[1].events = POLLIN;
+
+ while (1) {
+ ret = poll(pollfd, 2, -1);
+ if (ret <= 0)
+ break;
+ if (pollfd[1].revents)
+ break;
+ if (!(pollfd[0].revents & POLLIN))
+ break;
+
+ ret = read(args->uffd, &msg, sizeof(msg));
+ if (ret != sizeof(msg))
+ break;
+
+ if (msg.event != UFFD_EVENT_PAGEFAULT)
+ break;
+
+ /* Fill the page with a known pattern */
+ memset(args->page_buffer, 0xAB, args->page_size);
+
+ copy.dst = msg.arg.pagefault.address & ~(args->page_size - 1);
+ copy.src = (unsigned long)args->page_buffer;
+ copy.len = args->page_size;
+ copy.mode = 0;
+ copy.copy = 0;
+
+ ret = ioctl(args->uffd, UFFDIO_COPY, ©);
+ if (ret < 0)
+ break;
+ }
+
+ return NULL;
+}
+
+TEST_F(hmm, userfaultfd_read)
+{
+ struct hmm_buffer *buffer;
+ struct uffd_thread_args uffd_args;
+ unsigned long npages;
+ unsigned long size;
+ unsigned long i;
+ unsigned char *ptr;
+ pthread_t thread;
+ int uffd;
+ int stop_fd;
+ int ret;
+ struct uffdio_api api;
+ struct uffdio_register reg;
+ uint64_t stop = 1;
+ ssize_t nwrite;
+
+ npages = 4;
+ size = npages << self->page_shift;
+
+ /* Create userfaultfd */
+ uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK);
+ if (uffd < 0)
+ SKIP(return, "userfaultfd not available");
+
+ api.api = UFFD_API;
+ api.features = 0;
+ ret = ioctl(uffd, UFFDIO_API, &api);
+ ASSERT_EQ(ret, 0);
+
+ buffer = malloc(sizeof(*buffer));
+ ASSERT_NE(buffer, NULL);
+
+ buffer->fd = -1;
+ buffer->size = size;
+ buffer->mirror = malloc(size);
+ ASSERT_NE(buffer->mirror, NULL);
+
+ /* Create anonymous mapping */
+ buffer->ptr = mmap(NULL, size,
+ PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS,
+ -1, 0);
+ ASSERT_NE(buffer->ptr, MAP_FAILED);
+
+ /* Register the region with userfaultfd */
+ reg.range.start = (unsigned long)buffer->ptr;
+ reg.range.len = size;
+ reg.mode = UFFDIO_REGISTER_MODE_MISSING;
+ ret = ioctl(uffd, UFFDIO_REGISTER, ®);
+ ASSERT_EQ(ret, 0);
+
+ /* Set up the handler thread */
+ uffd_args.uffd = uffd;
+ stop_fd = eventfd(0, EFD_CLOEXEC);
+ ASSERT_GE(stop_fd, 0);
+ uffd_args.stop_fd = stop_fd;
+ uffd_args.page_buffer = malloc(self->page_size);
+ ASSERT_NE(uffd_args.page_buffer, NULL);
+ uffd_args.page_size = self->page_size;
+
+ ret = pthread_create(&thread, NULL, uffd_handler_thread, &uffd_args);
+ ASSERT_EQ(ret, 0);
+
+ /*
+ * Use the unlocked read path which allows the mmap lock to be
+ * dropped during the fault, enabling userfaultfd resolution.
+ */
+ ret = hmm_dmirror_cmd(self->fd, HMM_DMIRROR_READ_UNLOCKED,
+ buffer, npages);
+ ASSERT_EQ(ret, 0);
+ ASSERT_EQ(buffer->cpages, npages);
+
+ /* Verify the device read the data filled by the uffd handler */
+ ptr = buffer->mirror;
+ for (i = 0; i < size; ++i)
+ ASSERT_EQ(ptr[i], (unsigned char)0xAB);
+
+ nwrite = write(stop_fd, &stop, sizeof(stop));
+ ASSERT_EQ(nwrite, sizeof(stop));
+ pthread_join(thread, NULL);
+ close(stop_fd);
+ free(uffd_args.page_buffer);
+ close(uffd);
+ hmm_buffer_free(buffer);
+}
+
+
TEST_HARNESS_MAIN
^ permalink raw reply related
* [PATCH v9 4/8] mshv: Use hmm_range_fault_unlocked_timeout() for region faults
From: Stanislav Kinsburskii @ 2026-07-15 18:16 UTC (permalink / raw)
To: airlied, akhilesh, akpm, corbet, dakr, david, decui, haiyangz,
jgg, kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, skinsburskii, surenb,
tzimmermann, vbabka, wei.liu, skinsburskii
Cc: dri-devel, intel-xe, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <178413903133.1155966.3904063656020521607.stgit@skinsburskii>
MSHV currently faults movable memory regions by taking mmap_read_lock()
around hmm_range_fault(). That prevents the fault path from handling VMAs
whose fault handlers need to drop mmap_lock, such as userfaultfd-backed
mappings.
Use hmm_range_fault_unlocked_timeout() instead. Passing a timeout of 0
preserves MSHV's existing unbounded retry behavior while letting the HMM
helper own mmap_lock acquisition and refresh range->notifier_seq internally
before walking the range. After the fault succeeds, MSHV still takes
mreg_mutex and checks mmu_interval_read_retry() before installing the pages
into the region, so the existing invalidation synchronization is preserved.
Fold the small fault-and-lock helper into mshv_region_range_fault(), since
the remaining retry path is just the standard "fault, take the driver lock,
check the interval notifier sequence" pattern.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/hv/mshv_regions.c | 54 ++++++++-------------------------------------
1 file changed, 10 insertions(+), 44 deletions(-)
diff --git a/drivers/hv/mshv_regions.c b/drivers/hv/mshv_regions.c
index 6d65e5b42152..dddaade31b5d 100644
--- a/drivers/hv/mshv_regions.c
+++ b/drivers/hv/mshv_regions.c
@@ -381,46 +381,6 @@ int mshv_region_get(struct mshv_mem_region *region)
return kref_get_unless_zero(®ion->mreg_refcount);
}
-/**
- * mshv_region_hmm_fault_and_lock - Handle HMM faults and lock the memory region
- * @region: Pointer to the memory region structure
- * @range: Pointer to the HMM range structure
- *
- * This function performs the following steps:
- * 1. Reads the notifier sequence for the HMM range.
- * 2. Acquires a read lock on the memory map.
- * 3. Handles HMM faults for the specified range.
- * 4. Releases the read lock on the memory map.
- * 5. If successful, locks the memory region mutex.
- * 6. Verifies if the notifier sequence has changed during the operation.
- * If it has, releases the mutex and returns -EBUSY to match with
- * hmm_range_fault() return code for repeating.
- *
- * Return: 0 on success, a negative error code otherwise.
- */
-static int mshv_region_hmm_fault_and_lock(struct mshv_mem_region *region,
- struct hmm_range *range)
-{
- int ret;
-
- range->notifier_seq = mmu_interval_read_begin(range->notifier);
- mmap_read_lock(region->mreg_mni.mm);
- ret = hmm_range_fault(range);
- mmap_read_unlock(region->mreg_mni.mm);
- if (ret)
- return ret;
-
- mutex_lock(®ion->mreg_mutex);
-
- if (mmu_interval_read_retry(range->notifier, range->notifier_seq)) {
- mutex_unlock(®ion->mreg_mutex);
- cond_resched();
- return -EBUSY;
- }
-
- return 0;
-}
-
/**
* mshv_region_range_fault - Handle memory range faults for a given region.
* @region: Pointer to the memory region structure.
@@ -452,13 +412,19 @@ static int mshv_region_range_fault(struct mshv_mem_region *region,
range.start = region->start_uaddr + page_offset * HV_HYP_PAGE_SIZE;
range.end = range.start + page_count * HV_HYP_PAGE_SIZE;
- do {
- ret = mshv_region_hmm_fault_and_lock(region, &range);
- } while (ret == -EBUSY);
-
+again:
+ ret = hmm_range_fault_unlocked_timeout(&range, 0);
if (ret)
goto out;
+ mutex_lock(®ion->mreg_mutex);
+
+ if (mmu_interval_read_retry(range.notifier, range.notifier_seq)) {
+ mutex_unlock(®ion->mreg_mutex);
+ cond_resched();
+ goto again;
+ }
+
for (i = 0; i < page_count; i++)
region->mreg_pages[page_offset + i] = hmm_pfn_to_page(pfns[i]);
^ permalink raw reply related
* [PATCH v9 5/8] drm/nouveau: Use hmm_range_fault_unlocked_timeout() for SVM faults
From: Stanislav Kinsburskii @ 2026-07-15 18:16 UTC (permalink / raw)
To: airlied, akhilesh, akpm, corbet, dakr, david, decui, haiyangz,
jgg, kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, skinsburskii, surenb,
tzimmermann, vbabka, wei.liu, skinsburskii
Cc: dri-devel, intel-xe, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <178413903133.1155966.3904063656020521607.stgit@skinsburskii>
nouveau_range_fault() takes mmap_read_lock() only to call
hmm_range_fault(). It also keeps a single HMM_RANGE_DEFAULT_TIMEOUT
deadline across both HMM -EBUSY retries and post-fault
mmu_interval_read_retry() retries.
Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns
the mmap lock and refreshes range->notifier_seq for its internal retries.
Nouveau keeps its existing absolute deadline in the outer loop and passes
the remaining jiffies to the helper for each fault attempt, so retries
caused by mmu_interval_read_retry() do not reset the overall retry budget.
Nouveau still validates the interval notifier sequence while holding
svmm->mutex before programming the GPU mapping.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/gpu/drm/nouveau/nouveau_svm.c | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c
index dcc92131488e..58735446d783 100644
--- a/drivers/gpu/drm/nouveau/nouveau_svm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_svm.c
@@ -678,20 +678,22 @@ static int nouveau_range_fault(struct nouveau_svmm *svmm,
range.end = notifier->notifier.interval_tree.last + 1;
while (true) {
- if (time_after(jiffies, timeout)) {
+ long remaining = timeout - jiffies;
+
+ /*
+ * The HMM timeout only bounds retries while HMM is walking and
+ * faulting the range. This fault is handled by a kernel worker,
+ * so fatal signals from the faulting process cannot stop an
+ * endless stream of invalidations here.
+ */
+ if (time_after_eq(jiffies, timeout)) {
ret = -EBUSY;
goto out;
}
- range.notifier_seq = mmu_interval_read_begin(range.notifier);
- mmap_read_lock(mm);
- ret = hmm_range_fault(&range);
- mmap_read_unlock(mm);
- if (ret) {
- if (ret == -EBUSY)
- continue;
+ ret = hmm_range_fault_unlocked_timeout(&range, remaining);
+ if (ret)
goto out;
- }
mutex_lock(&svmm->mutex);
if (mmu_interval_read_retry(range.notifier,
^ permalink raw reply related
* [PATCH v9 6/8] RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults
From: Stanislav Kinsburskii @ 2026-07-15 18:16 UTC (permalink / raw)
To: airlied, akhilesh, akpm, corbet, dakr, david, decui, haiyangz,
jgg, kees, kys, leon, liam, lizhi.hou, ljs, longli, lyude,
maarten.lankhorst, mamin506, mhocko, mripard, nouveau, ogabbay,
oleg, rppt, shuah, simona, skhan, skinsburskii, surenb,
tzimmermann, vbabka, wei.liu, skinsburskii
Cc: dri-devel, intel-xe, linux-mm, linux-doc, linux-hyperv,
linux-kernel, linux-kselftest, linux-rdma
In-Reply-To: <178413903133.1155966.3904063656020521607.stgit@skinsburskii>
ib_umem_odp_map_dma_and_lock() takes mmap_read_lock() only around
hmm_range_fault(), then retries -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT
expires.
Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns
the mmap lock and refreshes range->notifier_seq for its internal retries.
ODP keeps using HMM_RANGE_DEFAULT_TIMEOUT for each HMM fault attempt,
while interval invalidation retries continue to be handled by the existing
outer loop.
ODP still validates the interval notifier sequence while holding umem_mutex
before DMA mapping pages.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/infiniband/core/umem_odp.c | 18 +++++-------------
1 file changed, 5 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c
index 404fa1cc3254..9cc21cd762d9 100644
--- a/drivers/infiniband/core/umem_odp.c
+++ b/drivers/infiniband/core/umem_odp.c
@@ -329,7 +329,7 @@ int ib_umem_odp_map_dma_and_lock(struct ib_umem_odp *umem_odp, u64 user_virt,
struct mm_struct *owning_mm = umem_odp->umem.owning_mm;
int pfn_index, dma_index, ret = 0, start_idx;
unsigned int page_shift, hmm_order, pfn_start_idx;
- unsigned long num_pfns, current_seq;
+ unsigned long num_pfns;
struct hmm_range range = {};
unsigned long timeout;
@@ -363,26 +363,18 @@ int ib_umem_odp_map_dma_and_lock(struct ib_umem_odp *umem_odp, u64 user_virt,
}
range.hmm_pfns = &(umem_odp->map.pfn_list[pfn_start_idx]);
- timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
retry:
- current_seq = range.notifier_seq =
- mmu_interval_read_begin(&umem_odp->notifier);
-
- mmap_read_lock(owning_mm);
- ret = hmm_range_fault(&range);
- mmap_read_unlock(owning_mm);
- if (unlikely(ret)) {
- if (ret == -EBUSY && !time_after(jiffies, timeout))
- goto retry;
+ ret = hmm_range_fault_unlocked_timeout(&range, timeout);
+ if (unlikely(ret))
goto out_put_mm;
- }
start_idx = (range.start - ib_umem_start(umem_odp)) >> page_shift;
dma_index = start_idx;
mutex_lock(&umem_odp->umem_mutex);
- if (mmu_interval_read_retry(&umem_odp->notifier, current_seq)) {
+ if (mmu_interval_read_retry(&umem_odp->notifier, range.notifier_seq)) {
mutex_unlock(&umem_odp->umem_mutex);
goto retry;
}
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox