* Re: [PATCH v2] scsi: storvsc: Replace symbolic permissions with octal
From: Martin K. Petersen @ 2026-05-15 1:45 UTC (permalink / raw)
To: Md Shofiqul Islam
Cc: linux-scsi, linux-hyperv, linux-kernel, longli, kys, haiyangz,
wei.liu, decui, mhklinux
In-Reply-To: <20260506004948.2172-1-shofiqtest@gmail.com>
> Symbolic permissions like S_IRUGO and S_IWUSR are not preferred by
> checkpatch. Replace with their octal equivalents:
Applied to 7.2/scsi-staging, thanks!
--
Martin K. Petersen
^ permalink raw reply
* Re: [PATCH net-next v9 2/6] net: mana: Query device capabilities and configure MSI-X sharing for EQs
From: sashiko-bot @ 2026-05-14 22:10 UTC (permalink / raw)
To: Long Li; +Cc: linux-hyperv
In-Reply-To: <20260513220956.402058-3-longli@microsoft.com>
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The explicit `msi_sharing = true` requirement for platforms without dynamic MSI-X support is unconditionally overwritten, leading to incorrect MSI-X configuration.
--
commit 113e4b9419d10774e0a86653bf123b1add7cbc2e
Author: Long Li <longli@microsoft.com>
net: mana: Query device capabilities and configure MSI-X sharing for EQs
This commit adjusts the max number of queues to allow dedicated MSI-X vectors
for each vPort and enables MSI-X sharing among vPorts when there are not
enough vectors or the platform does not support dynamic MSI-X allocation.
> diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> index 3bc3fff559991..bbd055849e363 100644
> --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
> +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> @@ -179,8 +179,18 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev)
> struct gdma_context *gc = pci_get_drvdata(pdev);
> struct gdma_query_max_resources_resp resp = {};
> struct gdma_general_req req = {};
> + unsigned int max_num_queues;
> + u8 bm_hostmode;
> + u16 num_ports;
> int err;
>
> + /* Reset msi_sharing so it is recomputed from current hardware
> + * state. On resume, num_online_cpus() or num_msix_usable may
> + * have changed, making dedicated MSI-X feasible where it was
> + * not before.
> + */
> + gc->msi_sharing = false;
> +
Does this unconditional reset unintentionally overwrite the platform constraint
for systems without dynamic MSI-X support?
[ ... ]
> @@ -1889,6 +1936,7 @@ static int mana_gd_setup_hwc_irqs(struct pci_dev *pdev)
> /* Need 1 interrupt for HWC */
> max_irqs = min(num_online_cpus(), MANA_MAX_NUM_QUEUES) + 1;
> min_irqs = 2;
> + gc->msi_sharing = true;
> }
>
> nvec = pci_alloc_irq_vectors(pdev, min_irqs, max_irqs, PCI_IRQ_MSIX);
The commit message states that sharing is always used when the platform does
not support dynamic MSI-X allocation. This is successfully set here in
mana_gd_setup_hwc_irqs().
However, looking at mana_gd_setup(), mana_gd_query_max_resources() is called
after mana_gd_setup_hwc_irqs().
Will this execution order cause the driver to ignore the limitation and
attempt to use dedicated MSI-X vectors if there happen to be enough
pre-allocated vectors to satisfy the per-vPort math later in the function?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260513220956.402058-1-longli@microsoft.com?part=2
^ permalink raw reply
* Re: [PATCH net-next v9 1/6] net: mana: Create separate EQs for each vPort
From: sashiko-bot @ 2026-05-14 22:10 UTC (permalink / raw)
To: Long Li; +Cc: linux-hyperv
In-Reply-To: <20260513220956.402058-2-longli@microsoft.com>
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] Race condition in `mana_set_channels()` allows an RDMA application to steal vport ownership during an Ethernet reconfiguration, leading to out-of-bounds memory accesses and interface breakage.
- [Critical] Per-PD reference counting for per-port EQs causes a scope mismatch, leading to NULL pointer dereferences and resource leaks when a Protection Domain is shared across multiple ports.
--
commit 31a11c151b6c97fb0cd32cd8787a2fa1cd92856b
Author: Long Li <longli@microsoft.com>
Subject: net: mana: Create separate EQs for each vPort
To prepare for assigning vPorts to dedicated MSI-X vectors, remove EQ
sharing among the vPorts and create dedicated EQs for each vPort.
> diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
> index ac5e75dd3494d..6159bd03a0217 100644
> --- a/drivers/infiniband/hw/mana/main.c
> +++ b/drivers/infiniband/hw/mana/main.c
[ ... ]
> @@ -55,15 +57,23 @@ int mana_ib_cfg_vport(struct mana_ib_dev *dev, u32 port, struct mana_ib_pd *pd,
> return err;
> }
>
> - mutex_unlock(&pd->vport_mutex);
>
> - pd->tx_shortform_allowed = mpc->tx_shortform_allowed;
> - pd->tx_vp_offset = mpc->tx_vp_offset;
> + err = mana_create_eq(mpc);
> + if (err) {
> + mana_uncfg_vport(mpc);
> + pd->vport_use_count--;
> + } else {
> + pd->tx_shortform_allowed = mpc->tx_shortform_allowed;
> + pd->tx_vp_offset = mpc->tx_vp_offset;
> + }
>
> - ibdev_dbg(&dev->ib_dev, "vport handle %llx pdid %x doorbell_id %x\n",
> - mpc->port_handle, pd->pdn, doorbell_id);
> + mutex_unlock(&pd->vport_mutex);
Could this lead to a NULL pointer dereference or resource leak if a Protection
Domain is shared across multiple ports?
The vport_use_count is tracked per-PD (pd->vport_use_count), but Event Queues
are now managed per-port (mpc->eqs).
If a user creates a QP on Port 1, pd->vport_use_count becomes 1 and mpc1->eqs
are allocated. If the user then creates a QP on Port 2 using the same PD, the
code in mana_ib_cfg_vport() will see pd->vport_use_count > 1 and bypass
configuration for Port 2.
This leaves mpc2->eqs as NULL if the Ethernet interface is down, leading to a
NULL dereference when EQs are accessed. When QPs are later destroyed, it
will also destroy the context of whichever port the last QP belonged to,
permanently leaking the hardware configuration and EQs of the port that was
actually initialized.
[ ... ]
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> index 04350973e19e3..e121834d17f3a 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> @@ -454,18 +454,40 @@ static int mana_set_coalesce(struct net_device *ndev,
> return err;
> }
>
> +/* mana_set_channels - change the number of queues on a port
> + *
> + * Returns -EBUSY if the port is down and RDMA holds the vport with
> + * EQs sized to the current num_queues.
> + */
> static int mana_set_channels(struct net_device *ndev,
> struct ethtool_channels *channels)
> {
> struct mana_port_context *apc = netdev_priv(ndev);
> unsigned int new_count = channels->combined_count;
> unsigned int old_count = apc->num_queues;
> + bool locked = false;
> int err;
>
> + /* When the port is down, hold vport_mutex for the entire
> + * operation to serialize against RDMA's mana_cfg_vport().
> + * This is safe because mana_detach()/mana_attach() skip
> + * vport teardown/setup when port_st_save is false.
> + * When the port is up, Ethernet owns the vport exclusively
> + * so no locking against RDMA is needed.
> + */
> + if (!apc->port_is_up) {
> + mutex_lock(&apc->vport_mutex);
> + if (apc->vport_use_count) {
> + mutex_unlock(&apc->vport_mutex);
> + return -EBUSY;
> + }
> + locked = true;
> + }
>
> err = mana_pre_alloc_rxbufs(apc, ndev->mtu, new_count);
> if (err) {
> netdev_err(ndev, "Insufficient memory for new allocations");
> - return err;
> + goto unlock;
> }
>
> err = mana_detach(ndev, false);
Can this lead to a race condition with RDMA operations when the port is up?
If apc->port_is_up is true, the function does not acquire apc->vport_mutex.
It proceeds to call mana_detach(), which calls mana_uncfg_vport() and lowers
apc->vport_use_count to 0.
Because no lock is held across the detach/attach window, a concurrent RDMA
operation like mana_ib_create_qp_raw() could grab apc->vport_mutex, observe
a use count of 0, and steal ownership of the vport.
If the RDMA thread allocates apc->eqs based on old_count, and then
mana_set_channels() updates apc->num_queues = new_count, subsequent RDMA
queue indexing operations may read out of bounds. The Ethernet interface will
also fail to reattach because RDMA now owns the vport, leaving the interface
broken.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260513220956.402058-1-longli@microsoft.com?part=1
^ permalink raw reply
* RE: [PATCH v1 4/4] iommu/hyperv: Add page-selective IOTLB flush support
From: Michael Kelley @ 2026-05-14 21:16 UTC (permalink / raw)
To: Michael Kelley, Yu Zhang, linux-kernel@vger.kernel.org,
linux-hyperv@vger.kernel.org, iommu@lists.linux.dev,
linux-pci@vger.kernel.org, linux-arch@vger.kernel.org
Cc: wei.liu@kernel.org, kys@microsoft.com, haiyangz@microsoft.com,
decui@microsoft.com, longli@microsoft.com, joro@8bytes.org,
will@kernel.org, robin.murphy@arm.com, bhelgaas@google.com,
kwilczynski@kernel.org, lpieralisi@kernel.org, mani@kernel.org,
robh@kernel.org, arnd@arndb.de, jgg@ziepe.ca,
jacob.pan@linux.microsoft.com, tgopinath@linux.microsoft.com,
easwar.hariharan@linux.microsoft.com
In-Reply-To: <SN6PR02MB41577D5EEC884EAE8AF5E14ED4072@SN6PR02MB4157.namprd02.prod.outlook.com>
From: Michael Kelley <mhklinux@outlook.com> Sent: Thursday, May 14, 2026 11:14 AM
>
> From: Yu Zhang <zhangyu1@linux.microsoft.com> Sent: Monday, May 11, 2026 9:24 AM
> >
> > Add page-selective IOTLB flush using HVCALL_FLUSH_DEVICE_DOMAIN_LIST.
> > This hypercall accepts a list of (page_number, page_mask_shift) entries,
> > enabling finer-grained IOTLB invalidation compared to the domain-wide
> > HVCALL_FLUSH_DEVICE_DOMAIN used by hv_iommu_flush_iotlb_all().
> >
> > hv_iommu_fill_iova_list() decomposes a contiguous IOVA range into a
> > minimal set of aligned power-of-two regions that fit in a single
> > hypercall input page. When the range exceeds the page capacity, the
> > code falls back to a full domain flush automatically.
> >
> > Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
> > Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
> > ---
> > drivers/iommu/hyperv/iommu.c | 91 +++++++++++++++++++++++++++++++++++-
> > include/hyperv/hvgdk_mini.h | 1 +
> > include/hyperv/hvhdk_mini.h | 17 +++++++
> > 3 files changed, 108 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/iommu/hyperv/iommu.c b/drivers/iommu/hyperv/iommu.c
> > index e5fc625314b5..3bca362b7815 100644
> > --- a/drivers/iommu/hyperv/iommu.c
> > +++ b/drivers/iommu/hyperv/iommu.c
> > @@ -486,10 +486,98 @@ static void hv_iommu_flush_iotlb_all(struct iommu_domain *domain)
> > hv_flush_device_domain(to_hv_iommu_domain(domain));
> > }
> >
> > +/* Max number of iova_list entries in a single hypercall input page. */
> > +#define HV_IOMMU_MAX_FLUSH_VA_COUNT \
> > + ((HV_HYP_PAGE_SIZE - sizeof(struct hv_input_flush_device_domain_list)) / \
> > + sizeof(union hv_iommu_flush_va))
> > +
> > +/* Returned by hv_iommu_fill_iova_list() when the range exceeds the capacity */
> > +#define HV_IOMMU_FLUSH_VA_OVERFLOW U16_MAX
> > +
> > +static inline u16 hv_iommu_fill_iova_list(union hv_iommu_flush_va *iova_list,
> > + unsigned long start,
> > + unsigned long end)
> > +{
> > + unsigned long start_pfn = start >> PAGE_SHIFT;
> > + unsigned long end_pfn = PAGE_ALIGN(end) >> PAGE_SHIFT;
>
> "end" is an inclusive end address per comment in struct iommu_iotlb_gather.
> So a page aligned value would typically have 0xFFF as the low order 12 bits,
> and PAGE_ALIGN() will do the right thing. But I don't think the value is
> *required* to be page aligned. If the value of "end" had 0x000 as the
> low order 12 bits, the above calculation would fail to include the page
> that has the address ending in 0x000. I think it needs to be
> PAGE_ALIGN(end + 1) in order to work correctly for this corner case.
>
One follow-on comment: the macros HVPFN_UP() and HVPFN_DOWN()
would likely be useful in setting start_pfn and end_pfn.
Michael
^ permalink raw reply
* [PATCH net] net: mana: Fix TOCTOU double-fetch of hwc_msg_id from DMA buffer
From: Erni Sri Satya Vennela @ 2026-05-14 19:41 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, longli, andrew+netdev, davem,
edumazet, kuba, pabeni, dipayanroy, horms, ernis, kees, shacharr,
stephen, linux-hyperv, netdev, linux-kernel
In mana_hwc_rx_event_handler(), resp->response.hwc_msg_id is read from
DMA-coherent memory and bounds-checked, then mana_hwc_handle_resp()
re-reads the same field from the same DMA buffer for test_bit() and
pointer arithmetic.
DMA-coherent memory is mapped uncacheable on x86 and is shared,
unencrypted, in Confidential VMs (SEV-SNP/TDX), so each load goes
directly to host-visible memory. A H/W can modify the value
between the check and the use, bypassing the bounds validation.
Fix this by reading hwc_msg_id exactly once using READ_ONCE() into a
stack-local variable in mana_hwc_rx_event_handler(), and passing the
validated value as a parameter to mana_hwc_handle_resp().
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
---
.../net/ethernet/microsoft/mana/hw_channel.c | 23 +++++++++++--------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index dbbde0fa57e7..fd8b324d7fb6 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -77,21 +77,19 @@ static int mana_hwc_post_rx_wqe(const struct hwc_wq *hwc_rxq,
}
static void mana_hwc_handle_resp(struct hw_channel_context *hwc, u32 resp_len,
- struct hwc_work_request *rx_req)
+ struct hwc_work_request *rx_req, u16 msg_id)
{
const struct gdma_resp_hdr *resp_msg = rx_req->buf_va;
struct hwc_caller_ctx *ctx;
int err;
- if (!test_bit(resp_msg->response.hwc_msg_id,
- hwc->inflight_msg_res.map)) {
- dev_err(hwc->dev, "hwc_rx: invalid msg_id = %u\n",
- resp_msg->response.hwc_msg_id);
+ if (!test_bit(msg_id, hwc->inflight_msg_res.map)) {
+ dev_err(hwc->dev, "hwc_rx: invalid msg_id = %u\n", msg_id);
mana_hwc_post_rx_wqe(hwc->rxq, rx_req);
return;
}
- ctx = hwc->caller_ctx + resp_msg->response.hwc_msg_id;
+ ctx = hwc->caller_ctx + msg_id;
err = mana_hwc_verify_resp_msg(ctx, resp_msg, resp_len);
if (err)
goto out;
@@ -251,6 +249,7 @@ static void mana_hwc_rx_event_handler(void *ctx, u32 gdma_rxq_id,
struct gdma_sge *sge;
u64 rq_base_addr;
u64 rx_req_idx;
+ u16 msg_id;
u8 *wqe;
if (WARN_ON_ONCE(hwc_rxq->gdma_wq->id != gdma_rxq_id))
@@ -269,13 +268,17 @@ static void mana_hwc_rx_event_handler(void *ctx, u32 gdma_rxq_id,
rx_req = &hwc_rxq->msg_buf->reqs[rx_req_idx];
resp = (struct gdma_resp_hdr *)rx_req->buf_va;
- if (resp->response.hwc_msg_id >= hwc->num_inflight_msg) {
- dev_err(hwc->dev, "HWC RX: wrong msg_id=%u\n",
- resp->response.hwc_msg_id);
+ /* Read msg_id once from DMA buffer to prevent TOCTOU:
+ * DMA memory is shared/unencrypted in CVMs - host can
+ * modify it between reads.
+ */
+ msg_id = READ_ONCE(resp->response.hwc_msg_id);
+ if (msg_id >= hwc->num_inflight_msg) {
+ dev_err(hwc->dev, "HWC RX: wrong msg_id=%u\n", msg_id);
return;
}
- mana_hwc_handle_resp(hwc, rx_oob->tx_oob_data_size, rx_req);
+ mana_hwc_handle_resp(hwc, rx_oob->tx_oob_data_size, rx_req, msg_id);
/* Can no longer use 'resp', because the buffer is posted to the HW
* in mana_hwc_handle_resp() above.
--
2.34.1
^ permalink raw reply related
* RE: [PATCH v1 4/4] iommu/hyperv: Add page-selective IOTLB flush support
From: Michael Kelley @ 2026-05-14 18:14 UTC (permalink / raw)
To: Yu Zhang, linux-kernel@vger.kernel.org,
linux-hyperv@vger.kernel.org, iommu@lists.linux.dev,
linux-pci@vger.kernel.org, linux-arch@vger.kernel.org
Cc: wei.liu@kernel.org, kys@microsoft.com, haiyangz@microsoft.com,
decui@microsoft.com, longli@microsoft.com, joro@8bytes.org,
will@kernel.org, robin.murphy@arm.com, bhelgaas@google.com,
kwilczynski@kernel.org, lpieralisi@kernel.org, mani@kernel.org,
robh@kernel.org, arnd@arndb.de, jgg@ziepe.ca, Michael Kelley,
jacob.pan@linux.microsoft.com, tgopinath@linux.microsoft.com,
easwar.hariharan@linux.microsoft.com
In-Reply-To: <20260511162408.1180069-5-zhangyu1@linux.microsoft.com>
From: Yu Zhang <zhangyu1@linux.microsoft.com> Sent: Monday, May 11, 2026 9:24 AM
>
> Add page-selective IOTLB flush using HVCALL_FLUSH_DEVICE_DOMAIN_LIST.
> This hypercall accepts a list of (page_number, page_mask_shift) entries,
> enabling finer-grained IOTLB invalidation compared to the domain-wide
> HVCALL_FLUSH_DEVICE_DOMAIN used by hv_iommu_flush_iotlb_all().
>
> hv_iommu_fill_iova_list() decomposes a contiguous IOVA range into a
> minimal set of aligned power-of-two regions that fit in a single
> hypercall input page. When the range exceeds the page capacity, the
> code falls back to a full domain flush automatically.
>
> Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
> Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
> ---
> drivers/iommu/hyperv/iommu.c | 91 +++++++++++++++++++++++++++++++++++-
> include/hyperv/hvgdk_mini.h | 1 +
> include/hyperv/hvhdk_mini.h | 17 +++++++
> 3 files changed, 108 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/hyperv/iommu.c b/drivers/iommu/hyperv/iommu.c
> index e5fc625314b5..3bca362b7815 100644
> --- a/drivers/iommu/hyperv/iommu.c
> +++ b/drivers/iommu/hyperv/iommu.c
> @@ -486,10 +486,98 @@ static void hv_iommu_flush_iotlb_all(struct iommu_domain *domain)
> hv_flush_device_domain(to_hv_iommu_domain(domain));
> }
>
> +/* Max number of iova_list entries in a single hypercall input page. */
> +#define HV_IOMMU_MAX_FLUSH_VA_COUNT \
> + ((HV_HYP_PAGE_SIZE - sizeof(struct hv_input_flush_device_domain_list)) / \
> + sizeof(union hv_iommu_flush_va))
> +
> +/* Returned by hv_iommu_fill_iova_list() when the range exceeds the capacity */
> +#define HV_IOMMU_FLUSH_VA_OVERFLOW U16_MAX
> +
> +static inline u16 hv_iommu_fill_iova_list(union hv_iommu_flush_va *iova_list,
> + unsigned long start,
> + unsigned long end)
> +{
> + unsigned long start_pfn = start >> PAGE_SHIFT;
> + unsigned long end_pfn = PAGE_ALIGN(end) >> PAGE_SHIFT;
"end" is an inclusive end address per comment in struct iommu_iotlb_gather.
So a page aligned value would typically have 0xFFF as the low order 12 bits,
and PAGE_ALIGN() will do the right thing. But I don't think the value is
*required* to be page aligned. If the value of "end" had 0x000 as the
low order 12 bits, the above calculation would fail to include the page
that has the address ending in 0x000. I think it needs to be
PAGE_ALIGN(end + 1) in order to work correctly for this corner case.
> + unsigned long nr_pages = end_pfn - start_pfn;
> + u16 count = 0;
> +
> + while (nr_pages > 0) {
> + unsigned long flush_pages;
> + int order;
> + unsigned long pfn_align;
> + unsigned long size_align;
> +
> + if (count >= HV_IOMMU_MAX_FLUSH_VA_COUNT) {
> + count = HV_IOMMU_FLUSH_VA_OVERFLOW;
> + break;
> + }
> +
> + if (start_pfn)
> + pfn_align = __ffs(start_pfn);
I don't understand why __ffs() is correct here. I would expect
__fls() so it is consistent with the calculation of size_align. But I
can only surmise how the hypercall works since there's no
documentation, so maybe my understanding of the hypercall is
wrong. If __ffs really is correct, a comment explaining why
would help. :-)
> + else
> + pfn_align = BITS_PER_LONG - 1;
> +
> + size_align = __fls(nr_pages);
> + order = min(pfn_align, size_align);
> + iova_list[count].page_mask_shift = order;
> + iova_list[count].page_number = start_pfn;
> +
> + flush_pages = 1UL << order;
> + start_pfn += flush_pages;
> + nr_pages -= flush_pages;
> + count++;
> + }
> +
> + return count;
> +}
> +
> +static void hv_flush_device_domain_list(struct hv_iommu_domain *hv_domain,
> + struct iommu_iotlb_gather *iotlb_gather)
> +{
> + u64 status;
> + u16 count;
> + unsigned long flags;
> + struct hv_input_flush_device_domain_list *input;
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(input, 0, sizeof(*input));
> +
> + input->device_domain = hv_domain->device_domain;
> + input->flags |= HV_FLUSH_DEVICE_DOMAIN_LIST_IOMMU_FORMAT;
I would suggest moving the memset() and setting the input fields down
under the "else" below so that they are parallel with the flush all case.
> + count = hv_iommu_fill_iova_list(input->iova_list,
> + iotlb_gather->start,
> + iotlb_gather->end);
> + if (count == HV_IOMMU_FLUSH_VA_OVERFLOW) {
> + /*
> + * Range exceeds hypercall page capacity. Fall back to a full
> + * domain flush.
> + */
> + struct hv_input_flush_device_domain *flush_all = (void *)input;
> +
> + memset(flush_all, 0, sizeof(*flush_all));
> + flush_all->device_domain = hv_domain->device_domain;
> + status = hv_do_hypercall(HVCALL_FLUSH_DEVICE_DOMAIN,
> + flush_all, NULL);
> + } else {
> + status = hv_do_rep_hypercall(
> + HVCALL_FLUSH_DEVICE_DOMAIN_LIST,
> + count, 0, input, NULL);
> + }
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_FLUSH_DEVICE_DOMAIN_LIST failed, status %lld\n", status);
As Sashiko pointed out, a failure here can lead to all kinds of trouble because
of leaving unflushed entries. Maybe a WARN() is more appropriate? Also, maybe
a failure in the list flush should try a flush all as a fallback, with the WARN()
only if the flush all fails.
> +}
> +
> static void hv_iommu_iotlb_sync(struct iommu_domain *domain,
> struct iommu_iotlb_gather *iotlb_gather)
> {
> - hv_flush_device_domain(to_hv_iommu_domain(domain));
> + hv_flush_device_domain_list(to_hv_iommu_domain(domain), iotlb_gather);
>
> iommu_put_pages_list(&iotlb_gather->freelist);
> }
> @@ -543,6 +631,7 @@ static struct iommu_domain *hv_iommu_domain_alloc_paging(struct device *dev)
>
> cfg.common.hw_max_vasz_lg2 = hv_iommu_device->max_iova_width;
> cfg.common.hw_max_oasz_lg2 = 52;
> + cfg.common.features |= BIT(PT_FEAT_FLUSH_RANGE);
> cfg.top_level = (hv_iommu_device->max_iova_width > 48) ? 4 : 3;
>
> ret = pt_iommu_x86_64_init(&hv_domain->pt_iommu_x86_64, &cfg,
> GFP_KERNEL);
> diff --git a/include/hyperv/hvgdk_mini.h b/include/hyperv/hvgdk_mini.h
> index 5bdbb44da112..eaaf87171478 100644
> --- a/include/hyperv/hvgdk_mini.h
> +++ b/include/hyperv/hvgdk_mini.h
> @@ -496,6 +496,7 @@ union hv_vp_assist_msr_contents { /*
> HV_REGISTER_VP_ASSIST_PAGE */
> #define HVCALL_GET_GPA_PAGES_ACCESS_STATES 0x00c9
> #define HVCALL_CONFIGURE_DEVICE_DOMAIN 0x00ce
> #define HVCALL_FLUSH_DEVICE_DOMAIN 0x00d0
> +#define HVCALL_FLUSH_DEVICE_DOMAIN_LIST 0x00d1
> #define HVCALL_ACQUIRE_SPARSE_SPA_PAGE_HOST_ACCESS 0x00d7
> #define HVCALL_RELEASE_SPARSE_SPA_PAGE_HOST_ACCESS 0x00d8
> #define HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY 0x00db
> diff --git a/include/hyperv/hvhdk_mini.h b/include/hyperv/hvhdk_mini.h
> index 493608e791b4..f51d5d9467f1 100644
> --- a/include/hyperv/hvhdk_mini.h
> +++ b/include/hyperv/hvhdk_mini.h
> @@ -671,4 +671,21 @@ struct hv_input_flush_device_domain {
> u32 reserved;
> } __packed;
>
> +union hv_iommu_flush_va {
> + u64 iova;
> + struct {
> + u64 page_mask_shift : 12;
> + u64 page_number : 52;
> + };
> +} __packed;
> +
> +
> +struct hv_input_flush_device_domain_list {
> + struct hv_input_device_domain device_domain;
> +#define HV_FLUSH_DEVICE_DOMAIN_LIST_IOMMU_FORMAT (1 << 0)
> + u32 flags;
> + u32 reserved;
> + union hv_iommu_flush_va iova_list[];
> +} __packed;
> +
> #endif /* _HV_HVHDK_MINI_H */
> --
> 2.52.0
>
^ permalink raw reply
* RE: [PATCH v1 3/4] iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest
From: Michael Kelley @ 2026-05-14 18:13 UTC (permalink / raw)
To: Yu Zhang, linux-kernel@vger.kernel.org,
linux-hyperv@vger.kernel.org, iommu@lists.linux.dev,
linux-pci@vger.kernel.org, linux-arch@vger.kernel.org
Cc: wei.liu@kernel.org, kys@microsoft.com, haiyangz@microsoft.com,
decui@microsoft.com, longli@microsoft.com, joro@8bytes.org,
will@kernel.org, robin.murphy@arm.com, bhelgaas@google.com,
kwilczynski@kernel.org, lpieralisi@kernel.org, mani@kernel.org,
robh@kernel.org, arnd@arndb.de, jgg@ziepe.ca, Michael Kelley,
jacob.pan@linux.microsoft.com, tgopinath@linux.microsoft.com,
easwar.hariharan@linux.microsoft.com
In-Reply-To: <20260511162408.1180069-4-zhangyu1@linux.microsoft.com>
From: Yu Zhang <zhangyu1@linux.microsoft.com> Sent: Monday, May 11, 2026 9:24 AM
>
> Add a para-virtualized IOMMU driver for Linux guests running on Hyper-V.
> This driver implements stage-1 IO translation within the guest OS.
> It integrates with the Linux IOMMU core, utilizing Hyper-V hypercalls
> for:
> - Capability discovery
> - Domain allocation, configuration, and deallocation
> - Device attachment and detachment
> - IOTLB invalidation
>
> The driver constructs x86-compatible stage-1 IO page tables in the
> guest memory using consolidated IO page table helpers. This allows
> the guest to manage stage-1 translations independently of vendor-
> specific drivers (like Intel VT-d or AMD IOMMU).
>
> Hyper-V consumes this stage-1 IO page table when a device domain is
> created and configured, and nests it with the host's stage-2 IO page
> tables, therefore eliminating the VM exits for guest IOMMU mapping
> operations. For unmapping operations, VM exits to perform the IOTLB
> flush are still unavoidable.
>
> Hyper-V identifies each PCI pass-thru device by a logical device ID
> in its hypercall interface. The vPCI driver (pci-hyperv) registers the
> per-bus portion of this ID with the pvIOMMU driver during bus probe.
> The pvIOMMU driver stores this mapping and combines it with the function
> number of the endpoint PCI device to form the complete ID for hypercalls.
As you are probably aware, Mukesh's patch series to support PCI
pass-thru devices also needs to get the logical device ID. Maybe the
registration mechanism needs to move somewhere that can be shared
with his code.
>
> Co-developed-by: Wei Liu <wei.liu@kernel.org>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> Co-developed-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
> Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
> Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
> ---
> arch/x86/hyperv/hv_init.c | 4 +
> arch/x86/include/asm/mshyperv.h | 4 +
> drivers/iommu/hyperv/Kconfig | 17 +
> drivers/iommu/hyperv/Makefile | 1 +
> drivers/iommu/hyperv/iommu.c | 705 ++++++++++++++++++++++++++++
> drivers/iommu/hyperv/iommu.h | 54 +++
> drivers/pci/controller/pci-hyperv.c | 19 +-
> include/asm-generic/mshyperv.h | 12 +
> 8 files changed, 815 insertions(+), 1 deletion(-)
> create mode 100644 drivers/iommu/hyperv/iommu.c
> create mode 100644 drivers/iommu/hyperv/iommu.h
>
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index 323adc93f2dc..2c8ff8e06249 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -578,6 +578,10 @@ void __init hyperv_init(void)
> old_setup_percpu_clockev = x86_init.timers.setup_percpu_clockev;
> x86_init.timers.setup_percpu_clockev = hv_stimer_setup_percpu_clockev;
>
> +#ifdef CONFIG_HYPERV_PVIOMMU
> + x86_init.iommu.iommu_init = hv_iommu_init;
> +#endif
> +
> hv_apic_init();
>
> x86_init.pci.arch_init = hv_pci_init;
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
> index f64393e853ee..20d947c2c758 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -313,6 +313,10 @@ static inline void mshv_vtl_return_hypercall(void) {}
> static inline void __mshv_vtl_return_call(struct mshv_vtl_cpu_context *vtl0) {}
> #endif
>
> +#ifdef CONFIG_HYPERV_PVIOMMU
> +int __init hv_iommu_init(void);
> +#endif
> +
> #include <asm-generic/mshyperv.h>
>
> #endif
> diff --git a/drivers/iommu/hyperv/Kconfig b/drivers/iommu/hyperv/Kconfig
> index 30f40d867036..9e658d5c9a77 100644
> --- a/drivers/iommu/hyperv/Kconfig
> +++ b/drivers/iommu/hyperv/Kconfig
> @@ -8,3 +8,20 @@ config HYPERV_IOMMU
> help
> Stub IOMMU driver to handle IRQs to support Hyper-V Linux
> guest and root partitions.
> +
> +if HYPERV_IOMMU
> +config HYPERV_PVIOMMU
> + bool "Microsoft Hypervisor para-virtualized IOMMU support"
> + depends on X86 && HYPERV
What is the intent w.r.t. 32-bit builds? Using X86 instead of X86_64
allows it. I did a 32-bit build and didn't get any build failures, which is
good. But I can't run it to see if the pvIOMMU actually works in a
32-bit build. I don't know how building X86_64 generic PT entries
would fare.
> + select IOMMU_API
> + select GENERIC_PT
> + select IOMMU_PT
> + select IOMMU_PT_X86_64
> + select IOMMU_IOVA
> + default HYPERV
> + help
> + Para-virtualized IOMMU driver for Linux guests running on
> + Microsoft Hyper-V. Provides DMA remapping and IOTLB
> + flush support to enable DMA isolation for devices
> + assigned to the guest.
> +endif
> diff --git a/drivers/iommu/hyperv/Makefile b/drivers/iommu/hyperv/Makefile
> index 9f557bad94ff..8669741c0a51 100644
> --- a/drivers/iommu/hyperv/Makefile
> +++ b/drivers/iommu/hyperv/Makefile
> @@ -1,2 +1,3 @@
> # SPDX-License-Identifier: GPL-2.0
> obj-$(CONFIG_HYPERV_IOMMU) += irq_remapping.o
> +obj-$(CONFIG_HYPERV_PVIOMMU) += iommu.o
> diff --git a/drivers/iommu/hyperv/iommu.c b/drivers/iommu/hyperv/iommu.c
> new file mode 100644
> index 000000000000..e5fc625314b5
> --- /dev/null
> +++ b/drivers/iommu/hyperv/iommu.c
> @@ -0,0 +1,705 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +/*
> + * Hyper-V IOMMU driver.
> + *
> + * Copyright (C) 2019, 2024-2026 Microsoft, Inc.
> + */
> +
> +#define pr_fmt(fmt) "Hyper-V pvIOMMU: " fmt
> +#define dev_fmt(fmt) pr_fmt(fmt)
> +
> +#include <linux/iommu.h>
> +#include <linux/pci.h>
> +#include <linux/dma-map-ops.h>
> +#include <linux/generic_pt/iommu.h>
> +#include <linux/pci-ats.h>
> +
> +#include <asm/iommu.h>
> +#include <asm/hypervisor.h>
> +#include <asm/mshyperv.h>
> +
> +#include "iommu.h"
> +#include "../iommu-pages.h"
> +
> +struct hv_iommu_dev *hv_iommu_device;
> +
> +/*
> + * Identity and blocking domains are static singletons: identity is a 1:1
> + * passthrough with no page table, blocking rejects all DMA. Neither holds
> + * per-IOMMU state, so one instance suffices even with multiple vIOMMUs.
> + */
> +static struct hv_iommu_domain hv_identity_domain;
> +static struct hv_iommu_domain hv_blocking_domain;
> +static const struct iommu_domain_ops hv_iommu_identity_domain_ops;
> +static const struct iommu_domain_ops hv_iommu_blocking_domain_ops;
> +static struct iommu_ops hv_iommu_ops;
> +static LIST_HEAD(hv_iommu_pci_bus_list);
> +static DEFINE_SPINLOCK(hv_iommu_pci_bus_lock);
> +
> +#define hv_iommu_present(iommu_cap) (iommu_cap & HV_IOMMU_CAP_PRESENT)
> +#define hv_iommu_s1_domain_supported(iommu_cap) (iommu_cap & HV_IOMMU_CAP_S1)
> +#define hv_iommu_5lvl_supported(iommu_cap) (iommu_cap & HV_IOMMU_CAP_S1_5LVL)
> +#define hv_iommu_ats_supported(iommu_cap) (iommu_cap & HV_IOMMU_CAP_ATS)
> +
> +int hv_iommu_register_pci_bus(int pci_domain_nr, u32 logical_dev_id_prefix)
> +{
> + struct hv_pci_busdata *bus, *new;
> + int ret = 0;
> +
> + if (no_iommu || !iommu_detected)
> + return 0;
> +
> + new = kzalloc_obj(*new, GFP_KERNEL);
> + if (!new)
> + return -ENOMEM;
> +
> + spin_lock(&hv_iommu_pci_bus_lock);
> + list_for_each_entry(bus, &hv_iommu_pci_bus_list, list) {
> + if (bus->pci_domain_nr != pci_domain_nr)
> + continue;
> +
> + if (bus->logical_dev_id_prefix != logical_dev_id_prefix) {
> + pr_err("stale registration for PCI domain %d (old prefix 0x%08x, new 0x%08x)\n",
> + pci_domain_nr, bus->logical_dev_id_prefix,
> + logical_dev_id_prefix);
> + ret = -EEXIST;
> + }
> +
> + goto out_free;
> + }
> +
> + new->pci_domain_nr = pci_domain_nr;
> + new->logical_dev_id_prefix = logical_dev_id_prefix;
> + list_add(&new->list, &hv_iommu_pci_bus_list);
> + spin_unlock(&hv_iommu_pci_bus_lock);
> + return 0;
> +
> +out_free:
> + spin_unlock(&hv_iommu_pci_bus_lock);
> + kfree(new);
> + return ret;
> +}
> +EXPORT_SYMBOL_FOR_MODULES(hv_iommu_register_pci_bus, "pci-hyperv");
> +
> +void hv_iommu_unregister_pci_bus(int pci_domain_nr)
> +{
> + struct hv_pci_busdata *bus, *tmp;
> +
> + spin_lock(&hv_iommu_pci_bus_lock);
> + list_for_each_entry_safe(bus, tmp, &hv_iommu_pci_bus_list, list) {
> + if (bus->pci_domain_nr == pci_domain_nr) {
> + list_del(&bus->list);
> + kfree(bus);
> + break;
> + }
> + }
> + spin_unlock(&hv_iommu_pci_bus_lock);
> +}
> +EXPORT_SYMBOL_FOR_MODULES(hv_iommu_unregister_pci_bus, "pci-hyperv");
> +
> +/*
> + * Look up the logical device ID for a vPCI device. Returns 0 on success
> + * with *logical_id filled in; -ENODEV if no entry registered for this
> + * device's vPCI bus.
> + */
> +static int hv_iommu_lookup_logical_dev_id(struct pci_dev *pdev, u64 *logical_id)
> +{
> + struct hv_pci_busdata *bus;
> + int domain = pci_domain_nr(pdev->bus);
> + int ret = -ENODEV;
> +
> + spin_lock(&hv_iommu_pci_bus_lock);
> + list_for_each_entry(bus, &hv_iommu_pci_bus_list, list) {
> + if (bus->pci_domain_nr == domain) {
> + *logical_id = (u64)bus->logical_dev_id_prefix |
> + PCI_FUNC(pdev->devfn);
> + ret = 0;
> + break;
> + }
> + }
> + spin_unlock(&hv_iommu_pci_bus_lock);
> + return ret;
> +}
> +
> +static int hv_create_device_domain(struct hv_iommu_domain *hv_domain, u32 domain_stage)
> +{
> + int ret;
> + u64 status;
> + unsigned long flags;
> + struct hv_input_create_device_domain *input;
> +
> + ret = ida_alloc_range(&hv_iommu_device->domain_ids,
> + hv_iommu_device->first_domain, hv_iommu_device->last_domain,
> + GFP_KERNEL);
> + if (ret < 0)
> + return ret;
> +
> + hv_domain->device_domain.partition_id = HV_PARTITION_ID_SELF;
> + hv_domain->device_domain.domain_id.type = domain_stage;
> + hv_domain->device_domain.domain_id.id = ret;
> + hv_domain->hv_iommu = hv_iommu_device;
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(input, 0, sizeof(*input));
> + input->device_domain = hv_domain->device_domain;
> + input->create_device_domain_flags.forward_progress_required = 1;
> + input->create_device_domain_flags.inherit_owning_vtl = 0;
> + status = hv_do_hypercall(HVCALL_CREATE_DEVICE_DOMAIN, input, NULL);
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status)) {
> + pr_err("HVCALL_CREATE_DEVICE_DOMAIN failed, status %lld\n", status);
> + ida_free(&hv_iommu_device->domain_ids, hv_domain->device_domain.domain_id.id);
> + }
> +
> + return hv_result_to_errno(status);
> +}
> +
> +static void hv_delete_device_domain(struct hv_iommu_domain *hv_domain)
> +{
> + u64 status;
> + unsigned long flags;
> + struct hv_input_delete_device_domain *input;
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(input, 0, sizeof(*input));
> + input->device_domain = hv_domain->device_domain;
> + status = hv_do_hypercall(HVCALL_DELETE_DEVICE_DOMAIN, input, NULL);
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_DELETE_DEVICE_DOMAIN failed, status %lld\n", status);
> +
> + ida_free(&hv_domain->hv_iommu->domain_ids, hv_domain->device_domain.domain_id.id);
> +}
> +
> +static bool hv_iommu_capable(struct device *dev, enum iommu_cap cap)
> +{
> + switch (cap) {
> + case IOMMU_CAP_CACHE_COHERENCY:
> + return true;
> + case IOMMU_CAP_DEFERRED_FLUSH:
> + return true;
> + default:
> + return false;
> + }
> +}
> +
> +static void hv_flush_device_domain(struct hv_iommu_domain *hv_domain)
> +{
> + u64 status;
> + unsigned long flags;
> + struct hv_input_flush_device_domain *input;
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(input, 0, sizeof(*input));
> + input->device_domain = hv_domain->device_domain;
The previous version of this patch had code to set several other fields in
the input. I wanted to confirm that not setting them in this version is
intentional. Were they not needed?
> + status = hv_do_hypercall(HVCALL_FLUSH_DEVICE_DOMAIN, input, NULL);
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_FLUSH_DEVICE_DOMAIN failed, status %lld\n", status);
> +}
> +
> +static void hv_iommu_detach_dev(struct iommu_domain *domain, struct device *dev)
> +{
> + u64 status;
> + unsigned long flags;
> + struct hv_input_detach_device_domain *input;
> + struct pci_dev *pdev;
> + struct hv_iommu_domain *hv_domain = to_hv_iommu_domain(domain);
> + struct hv_iommu_endpoint *vdev = dev_iommu_priv_get(dev);
> +
> + /* See the attach function, only PCI devices for now */
> + if (!dev_is_pci(dev) || vdev->hv_domain != hv_domain)
> + return;
Are these sanity checks necessary? The only caller is hv_iommu_attach_dev()
and it has already done the checks.
> +
> + pdev = to_pci_dev(dev);
> +
> + dev_dbg(dev, "detaching from domain %d\n", hv_domain->device_domain.domain_id.id);
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(input, 0, sizeof(*input));
> + input->partition_id = HV_PARTITION_ID_SELF;
> + if (hv_iommu_lookup_logical_dev_id(pdev, &input->device_id.as_uint64)) {
As Sashiko and Jacob Pan pointed out, doing the lookup while interrupts are disabled
is problematic. My suggestion would be to just do the lookup into a local variable
before disabling interrupts (rather than using a raw spin lock as Jacob suggested).
Same situation occurs in hv_iommu_attach_dev() and
hv_iommu_get_logical_device_property().
> + local_irq_restore(flags);
> + dev_warn(&pdev->dev, "no IOMMU registration for vPCI bus on detach\n");
> + return;
> + }
> + status = hv_do_hypercall(HVCALL_DETACH_DEVICE_DOMAIN, input, NULL);
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_DETACH_DEVICE_DOMAIN failed, status %lld\n", status);
> +
> + hv_flush_device_domain(hv_domain);
> +
> + vdev->hv_domain = NULL;
> +}
> +
> +static int hv_iommu_attach_dev(struct iommu_domain *domain, struct device *dev,
> + struct iommu_domain *old)
> +{
> + u64 status;
> + unsigned long flags;
> + struct pci_dev *pdev;
> + struct hv_input_attach_device_domain *input;
> + struct hv_iommu_endpoint *vdev = dev_iommu_priv_get(dev);
> + struct hv_iommu_domain *hv_domain = to_hv_iommu_domain(domain);
> + int ret;
> +
> + /* Only allow PCI devices for now */
> + if (!dev_is_pci(dev))
> + return -EINVAL;
> +
> + if (vdev->hv_domain == hv_domain)
> + return 0;
> +
> + if (vdev->hv_domain)
> + hv_iommu_detach_dev(&vdev->hv_domain->domain, dev);
> +
> + pdev = to_pci_dev(dev);
> + dev_dbg(dev, "attaching to domain %d\n",
> + hv_domain->device_domain.domain_id.id);
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(input, 0, sizeof(*input));
> + input->device_domain = hv_domain->device_domain;
> + ret = hv_iommu_lookup_logical_dev_id(pdev, &input->device_id.as_uint64);
> + if (ret) {
> + local_irq_restore(flags);
> + dev_err(&pdev->dev, "no IOMMU registration for vPCI bus\n");
> + return ret;
> + }
> + status = hv_do_hypercall(HVCALL_ATTACH_DEVICE_DOMAIN, input, NULL);
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_ATTACH_DEVICE_DOMAIN failed, status %lld\n", status);
> + else
> + vdev->hv_domain = hv_domain;
> +
> + return hv_result_to_errno(status);
> +}
> +
> +static int hv_iommu_get_logical_device_property(struct device *dev,
> + u32 code,
> + struct hv_output_get_logical_device_property *property)
> +{
> + u64 status, lid;
> + unsigned long flags;
> + int ret;
> + struct hv_input_get_logical_device_property *input;
> + struct hv_output_get_logical_device_property *output;
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + output = *this_cpu_ptr(hyperv_pcpu_input_arg) + sizeof(*input);
Nit: The other way to set output is:
output = input + 1;
I think this produces slightly better code because of not needing to
reference the per-cpu variable hyperv_pcpu_input_arg a 2nd time.
> + memset(input, 0, sizeof(*input));
> + input->partition_id = HV_PARTITION_ID_SELF;
> + ret = hv_iommu_lookup_logical_dev_id(to_pci_dev(dev), &lid);
> + if (ret) {
> + local_irq_restore(flags);
> + return ret;
> + }
> + input->logical_device_id = lid;
> + input->code = code;
> + status = hv_do_hypercall(HVCALL_GET_LOGICAL_DEVICE_PROPERTY, input, output);
> + *property = *output;
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_GET_LOGICAL_DEVICE_PROPERTY failed, status %lld\n", status);
> +
> + return hv_result_to_errno(status);
> +}
> +
> +static struct iommu_device *hv_iommu_probe_device(struct device *dev)
> +{
> + struct pci_dev *pdev;
> + struct hv_iommu_endpoint *vdev;
> + struct hv_output_get_logical_device_property device_iommu_property = {0};
> +
> + if (!dev_is_pci(dev))
> + return ERR_PTR(-ENODEV);
> +
> + pdev = to_pci_dev(dev);
> +
> + if (hv_iommu_get_logical_device_property(dev,
> + HV_LOGICAL_DEVICE_PROPERTY_PVIOMMU,
> + &device_iommu_property) ||
> + !(device_iommu_property.device_iommu & HV_DEVICE_IOMMU_ENABLED))
> + return ERR_PTR(-ENODEV);
> +
> + vdev = kzalloc_obj(*vdev, GFP_KERNEL);
> + if (!vdev)
> + return ERR_PTR(-ENOMEM);
> +
> + vdev->dev = dev;
> + vdev->hv_iommu = hv_iommu_device;
> + dev_iommu_priv_set(dev, vdev);
> +
> + if (hv_iommu_ats_supported(hv_iommu_device->cap) &&
> + pci_ats_supported(pdev))
> + pci_enable_ats(pdev, __ffs(hv_iommu_device->pgsize_bitmap));
> +
> + return &vdev->hv_iommu->iommu;
> +}
> +
> +static void hv_iommu_release_device(struct device *dev)
> +{
> + struct hv_iommu_endpoint *vdev = dev_iommu_priv_get(dev);
> + struct pci_dev *pdev = to_pci_dev(dev);
> +
> + if (pdev->ats_enabled)
> + pci_disable_ats(pdev);
> +
> + dev_iommu_priv_set(dev, NULL);
> + set_dma_ops(dev, NULL);
Previous versions of this function did hv_iommu_detach_dev(). With that call
removed from here, hv_iommu_detach_dev() is only called when attaching a
domain to a device that already has a domain attached. Is it the case that
Hyper-V doesn't require the detach as a cleanup step?
> +
> + kfree(vdev);
> +}
> +
> +static struct iommu_group *hv_iommu_device_group(struct device *dev)
> +{
> + if (dev_is_pci(dev))
> + return pci_device_group(dev);
> + else
> + return generic_device_group(dev);
> +}
> +
> +static int hv_configure_device_domain(struct hv_iommu_domain *hv_domain, u32 domain_type)
> +{
> + u64 status;
> + unsigned long flags;
> + struct pt_iommu_x86_64_hw_info pt_info;
> + struct hv_input_configure_device_domain *input;
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(input, 0, sizeof(*input));
> + input->device_domain = hv_domain->device_domain;
> + input->settings.flags.blocked = (domain_type == IOMMU_DOMAIN_BLOCKED);
> + input->settings.flags.translation_enabled = (domain_type != IOMMU_DOMAIN_IDENTITY);
> +
> + if (domain_type & __IOMMU_DOMAIN_PAGING) {
> + pt_iommu_x86_64_hw_info(&hv_domain->pt_iommu_x86_64, &pt_info);
> + input->settings.page_table_root = pt_info.gcr3_pt;
> + input->settings.flags.first_stage_paging_mode =
> + pt_info.levels == 5;
> + }
> + status = hv_do_hypercall(HVCALL_CONFIGURE_DEVICE_DOMAIN, input, NULL);
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_CONFIGURE_DEVICE_DOMAIN failed, status %lld\n", status);
> +
> + return hv_result_to_errno(status);
> +}
> +
> +static int __init hv_initialize_static_domains(void)
> +{
> + int ret;
> + struct hv_iommu_domain *hv_domain;
> +
> + /* Default stage-1 identity domain */
> + hv_domain = &hv_identity_domain;
> +
> + ret = hv_create_device_domain(hv_domain, HV_DEVICE_DOMAIN_TYPE_S1);
> + if (ret)
> + return ret;
> +
> + ret = hv_configure_device_domain(hv_domain, IOMMU_DOMAIN_IDENTITY);
> + if (ret)
> + goto delete_identity_domain;
> +
> + hv_domain->domain.type = IOMMU_DOMAIN_IDENTITY;
> + hv_domain->domain.ops = &hv_iommu_identity_domain_ops;
> + hv_domain->domain.owner = &hv_iommu_ops;
> + hv_domain->domain.geometry = hv_iommu_device->geometry;
> + hv_domain->domain.pgsize_bitmap = hv_iommu_device->pgsize_bitmap;
> +
> + /* Default stage-1 blocked domain */
> + hv_domain = &hv_blocking_domain;
> +
> + ret = hv_create_device_domain(hv_domain, HV_DEVICE_DOMAIN_TYPE_S1);
> + if (ret)
> + goto delete_identity_domain;
> +
> + ret = hv_configure_device_domain(hv_domain, IOMMU_DOMAIN_BLOCKED);
> + if (ret)
> + goto delete_blocked_domain;
> +
> + hv_domain->domain.type = IOMMU_DOMAIN_BLOCKED;
> + hv_domain->domain.ops = &hv_iommu_blocking_domain_ops;
> + hv_domain->domain.owner = &hv_iommu_ops;
> + hv_domain->domain.geometry = hv_iommu_device->geometry;
> + hv_domain->domain.pgsize_bitmap = hv_iommu_device->pgsize_bitmap;
> +
> + return 0;
> +
> +delete_blocked_domain:
> + hv_delete_device_domain(&hv_blocking_domain);
> +delete_identity_domain:
> + hv_delete_device_domain(&hv_identity_domain);
> + return ret;
> +}
> +
> +#define INTERRUPT_RANGE_START (0xfee00000)
> +#define INTERRUPT_RANGE_END (0xfeefffff)
> +static void hv_iommu_get_resv_regions(struct device *dev,
> + struct list_head *head)
> +{
> + struct iommu_resv_region *region;
> +
> + region = iommu_alloc_resv_region(INTERRUPT_RANGE_START,
> + INTERRUPT_RANGE_END - INTERRUPT_RANGE_START + 1,
> + 0, IOMMU_RESV_MSI, GFP_KERNEL);
> + if (!region)
> + return;
> +
> + list_add_tail(®ion->list, head);
> +}
> +
> +static void hv_iommu_flush_iotlb_all(struct iommu_domain *domain)
> +{
> + hv_flush_device_domain(to_hv_iommu_domain(domain));
> +}
> +
> +static void hv_iommu_iotlb_sync(struct iommu_domain *domain,
> + struct iommu_iotlb_gather *iotlb_gather)
> +{
> + hv_flush_device_domain(to_hv_iommu_domain(domain));
> +
> + iommu_put_pages_list(&iotlb_gather->freelist);
> +}
> +
> +static void hv_iommu_paging_domain_free(struct iommu_domain *domain)
> +{
> + struct hv_iommu_domain *hv_domain = to_hv_iommu_domain(domain);
> +
> + /* Free all remaining mappings */
> + pt_iommu_deinit(&hv_domain->pt_iommu);
> +
> + hv_delete_device_domain(hv_domain);
> +
> + kfree(hv_domain);
> +}
> +
> +static const struct iommu_domain_ops hv_iommu_identity_domain_ops = {
> + .attach_dev = hv_iommu_attach_dev,
> +};
> +
> +static const struct iommu_domain_ops hv_iommu_blocking_domain_ops = {
> + .attach_dev = hv_iommu_attach_dev,
> +};
> +
> +static const struct iommu_domain_ops hv_iommu_paging_domain_ops = {
> + .attach_dev = hv_iommu_attach_dev,
> + IOMMU_PT_DOMAIN_OPS(x86_64),
> + .flush_iotlb_all = hv_iommu_flush_iotlb_all,
> + .iotlb_sync = hv_iommu_iotlb_sync,
> + .free = hv_iommu_paging_domain_free,
> +};
> +
> +static struct iommu_domain *hv_iommu_domain_alloc_paging(struct device *dev)
> +{
> + int ret;
> + struct hv_iommu_domain *hv_domain;
> + struct pt_iommu_x86_64_cfg cfg = {};
> +
> + hv_domain = kzalloc_obj(*hv_domain, GFP_KERNEL);
> + if (!hv_domain)
> + return ERR_PTR(-ENOMEM);
> +
> + ret = hv_create_device_domain(hv_domain, HV_DEVICE_DOMAIN_TYPE_S1);
> + if (ret) {
> + kfree(hv_domain);
> + return ERR_PTR(ret);
> + }
> +
> + hv_domain->domain.geometry = hv_iommu_device->geometry;
> + hv_domain->pt_iommu.nid = dev_to_node(dev);
> +
> + cfg.common.hw_max_vasz_lg2 = hv_iommu_device->max_iova_width;
> + cfg.common.hw_max_oasz_lg2 = 52;
> + cfg.top_level = (hv_iommu_device->max_iova_width > 48) ? 4 : 3;
> +
> + ret = pt_iommu_x86_64_init(&hv_domain->pt_iommu_x86_64, &cfg, GFP_KERNEL);
> + if (ret) {
> + hv_delete_device_domain(hv_domain);
> + kfree(hv_domain);
> + return ERR_PTR(ret);
> + }
> +
> + /* Constrain to page sizes the hypervisor supports */
> + hv_domain->domain.pgsize_bitmap &= hv_iommu_device->pgsize_bitmap;
> +
> + hv_domain->domain.ops = &hv_iommu_paging_domain_ops;
> +
> + ret = hv_configure_device_domain(hv_domain, __IOMMU_DOMAIN_PAGING);
> + if (ret) {
> + pt_iommu_deinit(&hv_domain->pt_iommu);
> + hv_delete_device_domain(hv_domain);
> + kfree(hv_domain);
> + return ERR_PTR(ret);
> + }
> +
> + return &hv_domain->domain;
I think this function would be better if the error paths did "goto"
a cascading set of error labels. That's the typical pattern, and it's what you
use in hv_iommu_init(), for example.
> +}
> +
> +static struct iommu_ops hv_iommu_ops = {
> + .capable = hv_iommu_capable,
> + .domain_alloc_paging = hv_iommu_domain_alloc_paging,
> + .probe_device = hv_iommu_probe_device,
> + .release_device = hv_iommu_release_device,
> + .device_group = hv_iommu_device_group,
> + .get_resv_regions = hv_iommu_get_resv_regions,
> + .owner = THIS_MODULE,
> + .identity_domain = &hv_identity_domain.domain,
> + .blocked_domain = &hv_blocking_domain.domain,
> + .release_domain = &hv_blocking_domain.domain,
> +};
> +
> +static int hv_iommu_detect(struct hv_output_get_iommu_capabilities *hv_iommu_cap)
> +{
> + u64 status;
> + unsigned long flags;
> + struct hv_input_get_iommu_capabilities *input;
> + struct hv_output_get_iommu_capabilities *output;
> +
> + local_irq_save(flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + output = *this_cpu_ptr(hyperv_pcpu_input_arg) + sizeof(*input);
Potentially use "output = input + 1" here as well.
> + memset(input, 0, sizeof(*input));
> + input->partition_id = HV_PARTITION_ID_SELF;
> + status = hv_do_hypercall(HVCALL_GET_IOMMU_CAPABILITIES, input, output);
> + *hv_iommu_cap = *output;
> +
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status))
> + pr_err("HVCALL_GET_IOMMU_CAPABILITIES failed, status %lld\n", status);
> +
> + return hv_result_to_errno(status);
> +}
> +
> +static void __init hv_init_iommu_device(struct hv_iommu_dev *hv_iommu,
> + struct hv_output_get_iommu_capabilities *hv_iommu_cap)
> +{
> + ida_init(&hv_iommu->domain_ids);
> +
> + hv_iommu->cap = hv_iommu_cap->iommu_cap;
> + hv_iommu->max_iova_width = hv_iommu_cap->max_iova_width;
> + if (!hv_iommu_5lvl_supported(hv_iommu->cap) &&
> + hv_iommu->max_iova_width > 48) {
> + pr_info("5-level paging not supported, limiting iova width to 48.\n");
> + hv_iommu->max_iova_width = 48;
> + }
> +
> + hv_iommu->geometry = (struct iommu_domain_geometry) {
> + .aperture_start = 0,
> + .aperture_end = (((u64)1) << hv_iommu->max_iova_width) - 1,
> + .force_aperture = true,
> + };
> +
> + hv_iommu->first_domain = HV_DEVICE_DOMAIN_ID_DEFAULT + 1;
> + hv_iommu->last_domain = HV_DEVICE_DOMAIN_ID_NULL - 1;
> + /* Only x86 page sizes (4K/2M/1G) are supported */
> + hv_iommu->pgsize_bitmap = hv_iommu_cap->pgsize_bitmap &
> + (SZ_4K | SZ_2M | SZ_1G);
> + if (hv_iommu->pgsize_bitmap != hv_iommu_cap->pgsize_bitmap)
> + pr_warn("unsupported page sizes masked: 0x%llx -> 0x%llx\n",
> + hv_iommu_cap->pgsize_bitmap, hv_iommu->pgsize_bitmap);
> + if (!hv_iommu->pgsize_bitmap) {
> + pr_warn("no supported page sizes, defaulting to 4K\n");
> + hv_iommu->pgsize_bitmap = SZ_4K;
> + }
> + hv_iommu_device = hv_iommu;
> +}
> +
> +int __init hv_iommu_init(void)
> +{
> + int ret = 0;
> + struct hv_iommu_dev *hv_iommu = NULL;
> + struct hv_output_get_iommu_capabilities hv_iommu_cap = {0};
> +
> + if (no_iommu || iommu_detected)
> + return -ENODEV;
> +
> + if (!hv_is_hyperv_initialized())
> + return -ENODEV;
> +
> + ret = hv_iommu_detect(&hv_iommu_cap);
> + if (ret) {
> + pr_err("HVCALL_GET_IOMMU_CAPABILITIES failed: %d\n", ret);
> + return -ENODEV;
> + }
> +
> + if (!hv_iommu_present(hv_iommu_cap.iommu_cap) ||
> + !hv_iommu_s1_domain_supported(hv_iommu_cap.iommu_cap)) {
> + pr_err("IOMMU capabilities not sufficient: cap=0x%llx\n",
> + hv_iommu_cap.iommu_cap);
> + return -ENODEV;
> + }
> +
> + iommu_detected = 1;
> + pci_request_acs();
> +
> + hv_iommu = kzalloc_obj(*hv_iommu, GFP_KERNEL);
> + if (!hv_iommu)
> + return -ENOMEM;
> +
> + hv_init_iommu_device(hv_iommu, &hv_iommu_cap);
> +
> + ret = hv_initialize_static_domains();
> + if (ret) {
> + pr_err("static domains init failed: %d\n", ret);
> + goto err_free;
> + }
> +
> + ret = iommu_device_sysfs_add(&hv_iommu->iommu, NULL, NULL, "%s", "hv-iommu");
> + if (ret) {
> + pr_err("iommu_device_sysfs_add failed: %d\n", ret);
> + goto err_delete_static_domains;
> + }
> +
> + ret = iommu_device_register(&hv_iommu->iommu, &hv_iommu_ops, NULL);
> + if (ret) {
> + pr_err("iommu_device_register failed: %d\n", ret);
> + goto err_sysfs_remove;
> + }
> +
> + pr_info("successfully initialized\n");
> + return 0;
> +
> +err_sysfs_remove:
> + iommu_device_sysfs_remove(&hv_iommu->iommu);
> +err_delete_static_domains:
> + hv_delete_device_domain(&hv_blocking_domain);
> + hv_delete_device_domain(&hv_identity_domain);
> +err_free:
> + kfree(hv_iommu);
> + return ret;
> +}
> diff --git a/drivers/iommu/hyperv/iommu.h b/drivers/iommu/hyperv/iommu.h
> new file mode 100644
> index 000000000000..43f20d371245
> --- /dev/null
> +++ b/drivers/iommu/hyperv/iommu.h
> @@ -0,0 +1,54 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * Hyper-V IOMMU driver.
> + *
> + * Copyright (C) 2024-2025, Microsoft, Inc.
> + *
> + */
> +
> +#ifndef _HYPERV_IOMMU_H
> +#define _HYPERV_IOMMU_H
> +
> +struct hv_iommu_dev {
> + struct iommu_device iommu;
> + struct ida domain_ids;
> +
> + /* Device configuration */
> + u8 max_iova_width;
> + u8 max_pasid_width;
> + u64 cap;
> + u64 pgsize_bitmap;
> +
> + struct iommu_domain_geometry geometry;
> + u64 first_domain;
> + u64 last_domain;
> +};
> +
> +struct hv_iommu_domain {
> + union {
> + struct iommu_domain domain;
> + struct pt_iommu pt_iommu;
> + struct pt_iommu_x86_64 pt_iommu_x86_64;
> + };
> + struct hv_iommu_dev *hv_iommu;
> + struct hv_input_device_domain device_domain;
> + u64 pgsize_bitmap;
> +};
> +
> +struct hv_pci_busdata {
> + int pci_domain_nr;
> + u32 logical_dev_id_prefix;
> + struct list_head list;
> +};
> +
> +struct hv_iommu_endpoint {
> + struct device *dev;
> + struct hv_iommu_dev *hv_iommu;
> + struct hv_iommu_domain *hv_domain;
> +};
> +
> +#define to_hv_iommu_domain(d) \
> + container_of(d, struct hv_iommu_domain, domain)
> +
> +#endif /* _HYPERV_IOMMU_H */
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index cfc8fa403dad..a4af9c8c2220 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -3715,6 +3715,7 @@ static int hv_pci_probe(struct hv_device *hdev,
> struct hv_pcibus_device *hbus;
> int ret, dom;
> u16 dom_req;
> + u32 prefix;
> char *name;
>
> bridge = devm_pci_alloc_host_bridge(&hdev->device, 0);
> @@ -3857,13 +3858,25 @@ static int hv_pci_probe(struct hv_device *hdev,
>
> hbus->state = hv_pcibus_probed;
>
> - ret = create_root_hv_pci_bus(hbus);
> + /* Notify pvIOMMU before any device on the bus is scanned. */
> + prefix = (hdev->dev_instance.b[5] << 24) |
> + (hdev->dev_instance.b[4] << 16) |
> + (hdev->dev_instance.b[7] << 8) |
> + (hdev->dev_instance.b[6] & 0xf8);
This assembling of the logical device id prefix duplicates the
code in hv_irq_retarget_interrupt(). Could this code save the
prefix in struct hv_pcibus_device, and then have
hv_irq_retarget_interrupt() use it? Then it would be clear
that HVCALL_RETARGET_INTERRUPT is using exactly the same
logical device id as the IOMMU hypercalls.
> +
> + ret = hv_iommu_register_pci_bus(dom, prefix);
> if (ret)
> goto free_windows;
>
> + ret = create_root_hv_pci_bus(hbus);
> + if (ret)
> + goto unregister_pviommu;
> +
> mutex_unlock(&hbus->state_lock);
> return 0;
>
> +unregister_pviommu:
> + hv_iommu_unregister_pci_bus(dom);
> free_windows:
> hv_pci_free_bridge_windows(hbus);
> exit_d0:
> @@ -3974,8 +3987,10 @@ static int hv_pci_bus_exit(struct hv_device *hdev, bool
> keep_devs)
> static void hv_pci_remove(struct hv_device *hdev)
> {
> struct hv_pcibus_device *hbus;
> + int dom;
>
> hbus = hv_get_drvdata(hdev);
> + dom = hbus->bridge->domain_nr;
Nit: Setting "dom" here feels a little weird because the value is only needed
under the "if" statement. The value must be read before the root bus is
removed, but even so moving it under the "if" statement would make more
sense to me.
> if (hbus->state == hv_pcibus_installed) {
> tasklet_disable(&hdev->channel->callback_event);
> hbus->state = hv_pcibus_removing;
> @@ -3994,6 +4009,8 @@ static void hv_pci_remove(struct hv_device *hdev)
> hv_pci_remove_slots(hbus);
> pci_remove_root_bus(hbus->bridge->bus);
> pci_unlock_rescan_remove();
> +
> + hv_iommu_unregister_pci_bus(dom);
> }
>
> hv_pci_bus_exit(hdev, false);
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
> index bf601d67cecb..b71345c74568 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -73,6 +73,18 @@ extern enum hv_partition_type hv_curr_partition_type;
> extern void * __percpu *hyperv_pcpu_input_arg;
> extern void * __percpu *hyperv_pcpu_output_arg;
>
> +#ifdef CONFIG_HYPERV_PVIOMMU
> +int hv_iommu_register_pci_bus(int pci_domain_nr, u32 logical_dev_id_prefix);
> +void hv_iommu_unregister_pci_bus(int pci_domain_nr);
> +#else
> +static inline int hv_iommu_register_pci_bus(int pci_domain_nr,
> + u32 logical_dev_id_prefix)
> +{
> + return 0;
> +}
> +static inline void hv_iommu_unregister_pci_bus(int pci_domain_nr) { }
> +#endif
> +
> u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputaddr);
> u64 hv_do_fast_hypercall8(u16 control, u64 input8);
> u64 hv_do_fast_hypercall16(u16 control, u64 input1, u64 input2);
> --
> 2.52.0
>
^ permalink raw reply
* Re: [PATCH v4 16/18] mshv: Validate scheduler message bounds from hypervisor
From: Stanislav Kinsburskii @ 2026-05-14 15:17 UTC (permalink / raw)
To: Anirudh Rayabharam
Cc: kys, haiyangz, wei.liu, decui, longli, linux-hyperv, linux-kernel
In-Reply-To: <20260514-efficient-frisky-mastiff-ccdaf7@anirudhrb>
On Thu, May 14, 2026 at 05:49:01AM +0000, Anirudh Rayabharam wrote:
> On Wed, May 13, 2026 at 10:39:31AM -0700, Stanislav Kinsburskii wrote:
> > On Wed, May 13, 2026 at 11:12:05AM +0000, Anirudh Rayabharam wrote:
> > > On Thu, May 07, 2026 at 03:44:26PM +0000, Stanislav Kinsburskii wrote:
> > > > handle_pair_message() iterates up to msg->vp_count without verifying it
> > > > against HV_MESSAGE_MAX_PARTITION_VP_PAIR_COUNT. Since vp_count is read
> > > > from untrusted hypervisor data, a malformed message with a large value
> > > > would cause out-of-bounds reads from the partition_ids and vp_indexes
> > > > arrays.
> > > >
> > > > handle_bitset_message() iterates over set bits in valid_bank_mask (up to
> > > > 64) and advances bank_contents for each one. However, the payload buffer
> > > > only has space for 16 bank entries. A valid_bank_mask with more than 16
> > > > bits set causes bank_contents to read beyond the message buffer.
> > > >
> > > > Fix both by adding bounds validation:
> > > > - Clamp vp_count to HV_MESSAGE_MAX_PARTITION_VP_PAIR_COUNT
> > > > - Track banks consumed and stop before exceeding buffer capacity
> > > >
> > > > Fixes: 621191d709b1 ("Drivers: hv: Introduce mshv_root module to expose /dev/mshv to VMMs")
> > > > Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
> > > > ---
> > > > drivers/hv/mshv_synic.c | 20 ++++++++++++++++++--
> > > > 1 file changed, 18 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/drivers/hv/mshv_synic.c b/drivers/hv/mshv_synic.c
> > > > index 89207aad7cf1f..5d509299f14d7 100644
> > > > --- a/drivers/hv/mshv_synic.c
> > > > +++ b/drivers/hv/mshv_synic.c
> > > > @@ -190,7 +190,9 @@ static void kick_vp(struct mshv_vp *vp)
> > > > static void
> > > > handle_bitset_message(const struct hv_vp_signal_bitset_scheduler_message *msg)
> > > > {
> > > > - int bank_idx, vps_signaled = 0, bank_mask_size;
> > > > + int bank_idx, vps_signaled = 0, bank_mask_size, banks_used = 0;
> > > > + const int max_banks = sizeof(msg->vp_bitset.bitset_buffer) /
> > > > + sizeof(u64) - 2; /* subtract format + mask */
> > >
> > > Could this be a constant in the header?
> > >
> >
> > Yes, it could. But it the only place it's used and it's pretty
> > self-explanatory, so I don't think it needs to be.
>
> The "subtract format+mask" part is a bit concerning. We might forget to update
> this code if the struct layout ever changes. Whereas if the constant is
> right next to the definition in the header, it is unlikely to be missed.
>
Fair enough. But I'd suggest sending this a a follow up for the series.
What do you think?
Thanks,
Stanislav
> Thanks,
> Anirudh.
>
^ permalink raw reply
* Re: [PATCH v3 00/10] Convert all drivers to the new udata response flow
From: Leon Romanovsky @ 2026-05-14 8:22 UTC (permalink / raw)
To: Abhijit Gangurde, Allen Hubbe,
Broadcom internal kernel review list, Bernard Metzler,
Potnuri Bharat Teja, Bryan Tan, Cheng Xu, Dennis Dalessandro,
Gal Pressman, Junxian Huang, Kai Shen, Kalesh AP,
Konstantin Taranov, Krzysztof Czurylo, linux-hyperv, linux-rdma,
Long Li, Michal Kalderon, Michael Margolin, Nelson Escobar,
Satish Kharat, Selvin Xavier, Yossi Leybovich, Chengchang Tang,
Tatyana Nikolova, Vishnu Dasa, Yishai Hadas, Jason Gunthorpe
Cc: patches
In-Reply-To: <0-v3-4effdebad75a+e1-rdma_udata_rep_jgg@nvidia.com>
On Mon, 11 May 2026 21:09:29 -0300, Jason Gunthorpe wrote:
> Go through the drivers and migrate them to use ib_respond_udata(). Remove
> debugging prints on failure paths. Ensure the error propagates from
> ib_respond_udata(). Use the = {} pattern to initialize the uresp.
>
> There are a couple of oddball cases which are fixed up in their own
> commits, but otherwise this is fairly straightforward.
>
> [...]
Applied, thanks!
[01/10] RDMA: Use ib_is_udata_in_empty() for places calling ib_is_udata_cleared()
https://git.kernel.org/rdma/rdma/c/41480529abf89b
[02/10] IB/rdmavt: Don't abuse udata and ib_respond_udata()
https://git.kernel.org/rdma/rdma/c/34705a1ae3e700
[03/10] RDMA: Convert drivers using min to ib_respond_udata()
https://git.kernel.org/rdma/rdma/c/0d1e825104e8bd
[04/10] RDMA: Convert drivers using sizeof() to ib_respond_udata()
https://git.kernel.org/rdma/rdma/c/051ac78d04654f
[05/10] RDMA/cxgb4: Convert to ib_respond_udata()
https://git.kernel.org/rdma/rdma/c/07a642d0284c48
[06/10] RDMA/qedr: Replace qedr_ib_copy_to_udata() with ib_respond_udata()
https://git.kernel.org/rdma/rdma/c/17bf245156ac71
[07/10] RDMA/mlx: Replace response_len with ib_respond_udata()
https://git.kernel.org/rdma/rdma/c/f2d022ef99ec62
[08/10] RDMA: Use proper driver data response structs instead of open coding
https://git.kernel.org/rdma/rdma/c/d06310eda43097
[09/10] RDMA: Add missed = {} initialization to uresp structs
https://git.kernel.org/rdma/rdma/c/b0e60caf6c9d0c
[10/10] RDMA: Replace memset with = {} pattern for ib_respond_udata()
https://git.kernel.org/rdma/rdma/c/be4bca92cb86a6
Best regards,
--
Leon Romanovsky <leon@kernel.org>
^ permalink raw reply
* Re: [PATCH v4 08/18] mshv: Fix level-triggered check on uninitialized data
From: Anirudh Rayabharam @ 2026-05-14 5:49 UTC (permalink / raw)
To: Stanislav Kinsburskii
Cc: kys, haiyangz, wei.liu, decui, longli, linux-hyperv, linux-kernel
In-Reply-To: <agS3GOct6mPN6sN9@skinsburskii.localdomain>
On Wed, May 13, 2026 at 10:38:32AM -0700, Stanislav Kinsburskii wrote:
> On Wed, May 13, 2026 at 12:14:49PM +0000, Anirudh Rayabharam wrote:
> > On Thu, May 07, 2026 at 03:43:43PM +0000, Stanislav Kinsburskii wrote:
> > > In mshv_irqfd_assign(), the level-triggered validation for resample
> > > irqfds checks irqfd_lapic_irq.lapic_control.level_triggered before
> > > mshv_irqfd_update() has populated the field. Since the irqfd struct is
> > > zero-allocated, level_triggered is always 0 at that point, causing the
> > > check to always reject resample irqfds with -EINVAL. This makes
> > > level-triggered interrupt resampling — used to avoid interrupt storms
> > > with assigned devices — completely non-functional.
> >
> > What bugs would this manifest as? Why haven't we seen any such bugs so
> > far?
> >
>
> This patch fixes a logical error.
> Whtout the change this hunk always fails:
>
> if (args->flags & BIT(MSHV_IRQFD_BIT_RESAMPLE) &&
> !irqfd->irqfd_lapic_irq.lapic_control.level_triggered) {
>
> and the reason we never seen it as that we never used
> register_irqfd_with_resample() function of the mshv crate.
I see.
Reviewed-by: Anirudh Rayabharam (Microsoft) <anirudh@anirudhrb.com>
^ permalink raw reply
* Re: [PATCH v4 16/18] mshv: Validate scheduler message bounds from hypervisor
From: Anirudh Rayabharam @ 2026-05-14 5:49 UTC (permalink / raw)
To: Stanislav Kinsburskii
Cc: kys, haiyangz, wei.liu, decui, longli, linux-hyperv, linux-kernel
In-Reply-To: <agS3U8CRnqfYaDuI@skinsburskii.localdomain>
On Wed, May 13, 2026 at 10:39:31AM -0700, Stanislav Kinsburskii wrote:
> On Wed, May 13, 2026 at 11:12:05AM +0000, Anirudh Rayabharam wrote:
> > On Thu, May 07, 2026 at 03:44:26PM +0000, Stanislav Kinsburskii wrote:
> > > handle_pair_message() iterates up to msg->vp_count without verifying it
> > > against HV_MESSAGE_MAX_PARTITION_VP_PAIR_COUNT. Since vp_count is read
> > > from untrusted hypervisor data, a malformed message with a large value
> > > would cause out-of-bounds reads from the partition_ids and vp_indexes
> > > arrays.
> > >
> > > handle_bitset_message() iterates over set bits in valid_bank_mask (up to
> > > 64) and advances bank_contents for each one. However, the payload buffer
> > > only has space for 16 bank entries. A valid_bank_mask with more than 16
> > > bits set causes bank_contents to read beyond the message buffer.
> > >
> > > Fix both by adding bounds validation:
> > > - Clamp vp_count to HV_MESSAGE_MAX_PARTITION_VP_PAIR_COUNT
> > > - Track banks consumed and stop before exceeding buffer capacity
> > >
> > > Fixes: 621191d709b1 ("Drivers: hv: Introduce mshv_root module to expose /dev/mshv to VMMs")
> > > Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
> > > ---
> > > drivers/hv/mshv_synic.c | 20 ++++++++++++++++++--
> > > 1 file changed, 18 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/drivers/hv/mshv_synic.c b/drivers/hv/mshv_synic.c
> > > index 89207aad7cf1f..5d509299f14d7 100644
> > > --- a/drivers/hv/mshv_synic.c
> > > +++ b/drivers/hv/mshv_synic.c
> > > @@ -190,7 +190,9 @@ static void kick_vp(struct mshv_vp *vp)
> > > static void
> > > handle_bitset_message(const struct hv_vp_signal_bitset_scheduler_message *msg)
> > > {
> > > - int bank_idx, vps_signaled = 0, bank_mask_size;
> > > + int bank_idx, vps_signaled = 0, bank_mask_size, banks_used = 0;
> > > + const int max_banks = sizeof(msg->vp_bitset.bitset_buffer) /
> > > + sizeof(u64) - 2; /* subtract format + mask */
> >
> > Could this be a constant in the header?
> >
>
> Yes, it could. But it the only place it's used and it's pretty
> self-explanatory, so I don't think it needs to be.
The "subtract format+mask" part is a bit concerning. We might forget to update
this code if the struct layout ever changes. Whereas if the constant is
right next to the definition in the header, it is unlikely to be missed.
Thanks,
Anirudh.
^ permalink raw reply
* Re: [PATCH v4] mshv: support 1G hugepages by passing them as 2M-aligned chunks
From: sashiko-bot @ 2026-05-14 5:47 UTC (permalink / raw)
Cc: linux-hyperv
In-Reply-To: <20260513-huge_1g-v4-1-33cda59e4a70@anirudhrb.com>
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Missing physical contiguity validation in large folio chunking leads to guest memory corruption and host-guest VMA inconsistencies.
--
commit 8234ab081cfc742915853b13b8eaeeb12adcc021
Author: Anirudh Rayabharam (Microsoft) <anirudh@anirudhrb.com>
mshv: support 1G hugepages by passing them as 2M-aligned chunks
This commit modifies the mshv driver to support 1G hugepages by treating
them as a sequence of 2M chunks, since the hypervisor map GPA hypercall
coalesces contiguous 2M-aligned chunks. It uses a 2M stride for folios
of at least PMD_ORDER.
> diff --git a/drivers/hv/mshv_regions.c b/drivers/hv/mshv_regions.c
> index fdffd4f002f6f..6d65e5b421529 100644
> --- a/drivers/hv/mshv_regions.c
> +++ b/drivers/hv/mshv_regions.c
[ ... ]
> @@ -86,15 +84,14 @@ static long mshv_region_process_chunk(struct mshv_mem_region *region,
> u64 gfn = region->start_gfn + page_offset;
> u64 count;
> struct page *page;
> - int stride, ret;
> + unsigned int stride;
> + int ret;
>
> page = region->mreg_pages[page_offset];
> if (!page)
> return -EINVAL;
>
> stride = mshv_chunk_stride(page, gfn, page_count);
> - if (stride < 0)
> - return stride;
>
> /* Start at stride since the first stride is validated */
> for (count = stride; count < page_count; count += stride) {
Does this code assume that all intermediate pages within the stride are
physically contiguous simply because the first page belongs to a large folio?
If a VMM constructs a fragmented VMA, for example by using MADV_DONTNEED
on a shmem folio for guest memory ballooning, or MAP_FIXED to remap
specific pages, get_user_pages() could populate mreg_pages with a mix of
different pages.
Since the loop advances by the stride without inspecting the intermediate
pages between the stride boundaries, could this instruct the hypervisor to
map the contiguous physical block of the large folio into the guest,
ignoring the actual VMA mappings?
Would this cause the guest to read and write the host large folio's tail
pages instead of the VMM's intended pages?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260513-huge_1g-v4-1-33cda59e4a70@anirudhrb.com?part=1
^ permalink raw reply
* Re: [EXTERNAL] [PATCH] x86/VMBus: Confidential VMBus for dynamic DMA transfers
From: Tianyu Lan @ 2026-05-14 2:26 UTC (permalink / raw)
To: Long Li
Cc: KY Srinivasan, Haiyang Zhang, wei.liu@kernel.org, Dexuan Cui,
James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com,
Allen Pais, Tianyu Lan, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org,
vdso@hexbites.dev, mhklinux@outlook.com
In-Reply-To: <SA1PR21MB6683C18151A933242F826BCDCE062@SA1PR21MB6683.namprd21.prod.outlook.com>
On Thu, May 14, 2026 at 2:30 AM Long Li <longli@microsoft.com> wrote:
>
> > Hyper-V provides Confidential VMBus to communicate between device model
> > and device guest driver via encrypted/private memory in Confidential VM. The
> > device model is in OpenHCL
> > (https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopenvm
> > m.dev%2Fguide%2Fuser_guide%2Fopenhcl.html&data=05%7C02%7Clongli%40mi
> > crosoft.com%7C0ccfea7cda8e4500ae9808de9540d01e%7C72f988bf86f141af91a
> > b2d7cd011db47%7C1%7C0%7C639112302777934798%7CUnknown%7CTWFpbG
> > Zsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIk
> > FOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=5Uc%2FM4ZVgJT1
> > NAq08cIlNtfF5oW4n%2FTj%2Bqg3YqBUeZg%3D&reserved=0) that plays the
> > paravisor role.
> >
> > For a VMBus device, there are two communication methods to talk with
> > Host/Hypervisor. 1) VMBUS Ring buffer 2) Dynamic DMA transfer.
> >
> > The Confidential VMBus Ring buffer has been upstreamed by Roman Kisel(commit
> > 6802d8af47d1).
> >
> > The dynamic DMA transition of VMBus device normally goes through DMA core
> > and it uses SWIOTLB as bounce buffer in a CoCo VM.
> >
> > The Confidential VMBus device can do DMA directly to private/encrypted
> > memory. Because the swiotlb is decrypted memory, the DMA transfer must not
> > be bounced through the swiotlb, so as to preserve confidentiality. This is different
> > from the default for Linux CoCo VMs, so not use DMA(SWIOTLB) API in VMBus
> > driver when confidential dynamic DMA transfers capability is present.
> >
> > Signed-off-by: Tianyu Lan <tiala@microsoft.com>
> > ---
> > drivers/scsi/storvsc_drv.c | 28 +++++++++++++++++++++-------
> > include/linux/hyperv.h | 1 +
> > 2 files changed, 22 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c index
> > ae1abab97835..79b7611518b7 100644
> > --- a/drivers/scsi/storvsc_drv.c
> > +++ b/drivers/scsi/storvsc_drv.c
> > @@ -1316,7 +1316,8 @@ static void storvsc_on_channel_callback(void *context)
> > continue;
> > }
> > request = (struct storvsc_cmd_request
> > *)scsi_cmd_priv(scmnd);
> > - scsi_dma_unmap(scmnd);
> > + if (!device->co_external_memory)
> > + scsi_dma_unmap(scmnd);
> > }
> >
> > storvsc_on_receive(stor_device, packet, request); @@ -
> > 1339,6 +1340,8 @@ static int storvsc_connect_to_vsp(struct hv_device *device,
> > u32 ring_size,
> >
> > device->channel->max_pkt_size = STORVSC_MAX_PKT_SIZE;
> > device->channel->next_request_id_callback = storvsc_next_request_id;
> > + if (device->channel->co_external_memory)
> > + device->co_external_memory = true;
> >
> > ret = vmbus_open(device->channel,
> > ring_size,
> > @@ -1805,7 +1808,7 @@ static enum scsi_qc_status
> > storvsc_queuecommand(struct Scsi_Host *host,
> > unsigned long offset_in_hvpg = offset_in_hvpage(sgl->offset);
> > unsigned int hvpg_count = HVPFN_UP(offset_in_hvpg + length);
> > struct scatterlist *sg;
> > - unsigned long hvpfn, hvpfns_to_add;
> > + unsigned long hvpfn, hvpfns_to_add, hvpgoff;
> > int j, i = 0, sg_count;
> >
> > payload_sz = (hvpg_count * sizeof(u64) + @@ -1821,7 +1824,11
> > @@ static enum scsi_qc_status storvsc_queuecommand(struct Scsi_Host *host,
> > payload->range.len = length;
> > payload->range.offset = offset_in_hvpg;
> >
> > - sg_count = scsi_dma_map(scmnd);
> > + if (dev->co_external_memory)
> > + sg_count = scsi_sg_count(scmnd);
>
> scsi_sg_count() returns unsigned int, sg_count can't be negative. The check for sg_count < 0 below becomes dead code. Add a comment to say this is expected behavior.
>
Hi Long:
Thanks for your review. Nice catch and will update.
> > + else
> > + sg_count = scsi_dma_map(scmnd);
> > +
> > if (sg_count < 0) {
> > ret = SCSI_MLQUEUE_DEVICE_BUSY;
> > goto err_free_payload;
> > @@ -1836,9 +1843,16 @@ static enum scsi_qc_status
> > storvsc_queuecommand(struct Scsi_Host *host,
> > * Such offsets are handled even on other than the first
> > * sgl entry, provided they are a multiple of PAGE_SIZE.
> > */
> > - hvpfn = HVPFN_DOWN(sg_dma_address(sg));
> > - hvpfns_to_add = HVPFN_UP(sg_dma_address(sg) +
> > - sg_dma_len(sg)) - hvpfn;
> > + if (dev->co_external_memory) {
> > + hvpgoff = HVPFN_DOWN(sg->offset);
> > + hvpfn = page_to_hvpfn(sg_page(sg)) + hvpgoff;
> > + hvpfns_to_add = HVPFN_UP(sg->offset
> > + sg->length) -
> > + hvpgoff;
> > + } else {
> > + hvpfn = HVPFN_DOWN(sg_dma_address(sg));
> > + hvpfns_to_add =
> > HVPFN_UP(sg_dma_address(sg) +
> > + sg_dma_len(sg)) -
> > hvpfn;
> > + }
> >
> > /*
> > * Fill the next portion of the PFN array with @@ -1860,7
> > +1874,7 @@ static enum scsi_qc_status storvsc_queuecommand(struct
> > Scsi_Host *host,
> > ret = storvsc_do_io(dev, cmd_request, smp_processor_id());
> > migrate_enable();
> >
> > - if (ret)
> > + if (ret && (!dev->co_external_memory))
> > scsi_dma_unmap(scmnd);
> >
> > if (ret == -EAGAIN) {
> > diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h index
> > dfc516c1c719..bcb143766d6e 100644
> > --- a/include/linux/hyperv.h
> > +++ b/include/linux/hyperv.h
> > @@ -1285,6 +1285,7 @@ struct hv_device {
> >
> > /* place holder to keep track of the dir for hv device in debugfs */
> > struct dentry *debug_dir;
> > + bool co_external_memory;
>
> You don't need to introduce co_external_memory in hv_device, vmbus_channel already has co_external_memory. Is it possible that you can check the vmbus_channel->co_external_memory directly? If you can remove this, you can reword this patch to " scsi: storvsc: Confidential VMBus for dynamic DMA transfers".
>
Good idea. Will update in the next version.
--
Thanks
Tianyu Lan
^ permalink raw reply
* Re: [PATCH v3 10/10] RDMA: Replace memset with = {} pattern for ib_respond_udata()
From: Jason Gunthorpe @ 2026-05-13 23:23 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-hyperv
In-Reply-To: <20260513205916.926F0C19425@smtp.kernel.org>
On Wed, May 13, 2026 at 08:59:15PM +0000, sashiko-bot@kernel.org wrote:
> The commit message notes that qedr_copy_qp_uresp() is already called with
> zeroed memory. If the caller is using aggregate initialization (= {}) on
> the stack, this may not completely zero the memory.
This isn't true, c23 standardized ={} as "empty initializtion"
clearing padding and the kernel has long relied on this behavior.
Jason
^ permalink raw reply
* RE: [EXTERNAL] Re: [PATCH net-next v7 0/6] net: mana: Per-vPort EQ and MSI-X interrupt management
From: Long Li @ 2026-05-13 22:43 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Konstantin Taranov, David S . Miller, Paolo Abeni, Eric Dumazet,
Andrew Lunn, Jason Gunthorpe, Leon Romanovsky, Haiyang Zhang,
KY Srinivasan, Wei Liu, Dexuan Cui,
shradhagupta@linux.microsoft.com, Simon Horman,
netdev@vger.kernel.org, linux-rdma@vger.kernel.org,
linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <20260511191540.630e09b3@kernel.org>
> On Thu, 7 May 2026 12:12:31 -0700 Long Li wrote:
> > This series adds per-vPort Event Queue (EQ) allocation and MSI-X
> > interrupt management for the MANA driver. Previously, all vPorts
> > shared a single set of EQs. This change enables dedicated EQs per
> > vPort with support for both dedicated and shared MSI-X vector allocation
> modes.
>
> Once all the AI review comments are address / only false positives remain - could
> you pop these patches on a branch and add PR info to the cover letter so that
> both RDMA and netdev can pull this?
I have sent v9 with PR info in the cover letter.
Thanks,
Long
^ permalink raw reply
* Re: [PATCH v2 1/2] Drivers: hv: vmbus: Provide option to skip VMBus unload on panic
From: Wei Liu @ 2026-05-13 22:32 UTC (permalink / raw)
To: Michael Kelley
Cc: wei.liu@kernel.org, tzimmermann@suse.de, longli@microsoft.com,
jfalempe@redhat.com, drawat.floss@gmail.com,
maarten.lankhorst@linux.intel.com, mripard@kernel.org,
airlied@gmail.com, simona@ffwll.ch, kys@microsoft.com,
haiyangz@microsoft.com, decui@microsoft.com, ryasuoka@redhat.com,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-hyperv@vger.kernel.org, stable@vger.kernel.org
In-Reply-To: <SN6PR02MB415786A3C7D10C22FAB12E3ED4312@SN6PR02MB4157.namprd02.prod.outlook.com>
On Mon, May 04, 2026 at 08:47:50PM +0000, Michael Kelley wrote:
> From: Michael Kelley <mhklkml@zohomail.com> Sent: Tuesday, February 17, 2026 10:24 AM
> >
>
> Wei and Thomas --
>
> This small patch series has been neglected. Patch 2 of the series is here [1].
>
> Long Li < longli@microsoft.com> has given a Reviewed-by on this patch,
> and Jocelyn Falempe <jfalempe@redhat.com> has given a Reviewed-by
> on Patch 2 of the series, modulo a comment which I have incorporated.
> See [2]. But I neglected to add her R-b when I spun v2 of the series.
>
> Any reason this can't be picked up as a bug fix for 7.1? I just checked,
> and it applies cleanly to a recent linux-next (20260423). I'd suggest
> going through the hyperv tree, as these two patches should be kept
> together in sequence.
>
I thought they were not reviewed yet. I have now applied them to hyperv-fixes.
Thanks for the reminder.
Wei
^ permalink raw reply
* [PATCH net-next v9 6/6] RDMA/mana_ib: Allocate interrupt contexts on EQs
From: Long Li @ 2026-05-13 22:09 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta
Cc: Simon Horman, netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260513220956.402058-1-longli@microsoft.com>
Use the GIC functions to allocate interrupt contexts for RDMA EQs. These
interrupt contexts may be shared with Ethernet EQs when MSI-X vectors
are limited.
The driver now supports allocating dedicated MSI-X for each EQ. Indicate
this capability through driver capability bits. The RDMA EQs pass
use_msi_bitmap=false to share MSI-X vectors with Ethernet, while the
capability flag advertises that the driver supports per-vPort EQ
separation when hardware has sufficient vectors.
Populate eq.irq on all RDMA EQs for consistency with the Ethernet path.
Also relocate the GDMA_DRV_CAP_FLAG_1_HW_VPORT_LINK_AWARE define to its
numeric BIT(6) position among the other capability flags.
Signed-off-by: Long Li <longli@microsoft.com>
---
drivers/infiniband/hw/mana/main.c | 43 +++++++++++++++++++++++++------
include/net/mana/gdma.h | 7 +++--
2 files changed, 40 insertions(+), 10 deletions(-)
diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index 6159bd03a021..47e5322bebca 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -750,7 +750,8 @@ int mana_ib_create_eqs(struct mana_ib_dev *mdev)
{
struct gdma_context *gc = mdev_to_gc(mdev);
struct gdma_queue_spec spec = {};
- int err, i;
+ struct gdma_irq_context *gic;
+ int err, i, msi;
spec.type = GDMA_EQ;
spec.monitor_avl_buf = false;
@@ -758,11 +759,19 @@ int mana_ib_create_eqs(struct mana_ib_dev *mdev)
spec.eq.callback = mana_ib_event_handler;
spec.eq.context = mdev;
spec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE;
- spec.eq.msix_index = 0;
+
+ msi = 0;
+ gic = mana_gd_get_gic(gc, false, &msi);
+ if (!gic)
+ return -ENOMEM;
+ spec.eq.msix_index = msi;
err = mana_gd_create_mana_eq(mdev->gdma_dev, &spec, &mdev->fatal_err_eq);
- if (err)
+ if (err) {
+ mana_gd_put_gic(gc, false, 0);
return err;
+ }
+ mdev->fatal_err_eq->eq.irq = gic->irq;
mdev->eqs = kzalloc_objs(struct gdma_queue *,
mdev->ib_dev.num_comp_vectors);
@@ -772,32 +781,50 @@ int mana_ib_create_eqs(struct mana_ib_dev *mdev)
}
spec.eq.callback = NULL;
for (i = 0; i < mdev->ib_dev.num_comp_vectors; i++) {
- spec.eq.msix_index = (i + 1) % gc->num_msix_usable;
+ msi = (i + 1) % gc->num_msix_usable;
+
+ gic = mana_gd_get_gic(gc, false, &msi);
+ if (!gic) {
+ err = -ENOMEM;
+ goto destroy_eqs;
+ }
+ spec.eq.msix_index = msi;
+
err = mana_gd_create_mana_eq(mdev->gdma_dev, &spec, &mdev->eqs[i]);
- if (err)
+ if (err) {
+ mana_gd_put_gic(gc, false, msi);
goto destroy_eqs;
+ }
+ mdev->eqs[i]->eq.irq = gic->irq;
}
return 0;
destroy_eqs:
- while (i-- > 0)
+ while (i-- > 0) {
mana_gd_destroy_queue(gc, mdev->eqs[i]);
+ mana_gd_put_gic(gc, false, (i + 1) % gc->num_msix_usable);
+ }
kfree(mdev->eqs);
destroy_fatal_eq:
mana_gd_destroy_queue(gc, mdev->fatal_err_eq);
+ mana_gd_put_gic(gc, false, 0);
return err;
}
void mana_ib_destroy_eqs(struct mana_ib_dev *mdev)
{
struct gdma_context *gc = mdev_to_gc(mdev);
- int i;
+ int i, msi;
mana_gd_destroy_queue(gc, mdev->fatal_err_eq);
+ mana_gd_put_gic(gc, false, 0);
- for (i = 0; i < mdev->ib_dev.num_comp_vectors; i++)
+ for (i = 0; i < mdev->ib_dev.num_comp_vectors; i++) {
mana_gd_destroy_queue(gc, mdev->eqs[i]);
+ msi = (i + 1) % gc->num_msix_usable;
+ mana_gd_put_gic(gc, false, msi);
+ }
kfree(mdev->eqs);
}
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 6c138cc77407..d84e474309a3 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -615,6 +615,7 @@ enum {
#define GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG BIT(3)
#define GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB BIT(4)
#define GDMA_DRV_CAP_FLAG_1_VARIABLE_INDIRECTION_TABLE_SUPPORT BIT(5)
+#define GDMA_DRV_CAP_FLAG_1_HW_VPORT_LINK_AWARE BIT(6)
/* Driver can handle holes (zeros) in the device list */
#define GDMA_DRV_CAP_FLAG_1_DEV_LIST_HOLES_SUP BIT(11)
@@ -631,7 +632,8 @@ enum {
/* Driver detects stalled send queues and recovers them */
#define GDMA_DRV_CAP_FLAG_1_HANDLE_STALL_SQ_RECOVERY BIT(18)
-#define GDMA_DRV_CAP_FLAG_1_HW_VPORT_LINK_AWARE BIT(6)
+/* Driver supports separate EQ/MSIs for each vPort */
+#define GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT BIT(19)
/* Driver supports linearizing the skb when num_sge exceeds hardware limit */
#define GDMA_DRV_CAP_FLAG_1_SKB_LINEARIZE BIT(20)
@@ -659,7 +661,8 @@ enum {
GDMA_DRV_CAP_FLAG_1_SKB_LINEARIZE | \
GDMA_DRV_CAP_FLAG_1_PROBE_RECOVERY | \
GDMA_DRV_CAP_FLAG_1_HANDLE_STALL_SQ_RECOVERY | \
- GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY)
+ GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY | \
+ GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT)
#define GDMA_DRV_CAP_FLAGS2 0
--
2.43.0
^ permalink raw reply related
* [PATCH net-next v9 5/6] net: mana: Allocate interrupt context for each EQ when creating vPort
From: Long Li @ 2026-05-13 22:09 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta
Cc: Simon Horman, netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260513220956.402058-1-longli@microsoft.com>
Use GIC functions to create a dedicated interrupt context or acquire a
shared interrupt context for each EQ when setting up a vPort.
The caller now owns the GIC reference across the EQ create/destroy
lifecycle: mana_create_eq() calls mana_gd_get_gic() before creating
each EQ and mana_destroy_eq() calls mana_gd_put_gic() after destroying
it. The msix_index invalidation is moved from mana_gd_deregister_irq()
to the mana_gd_create_eq() error path so that mana_destroy_eq() can
read the index before teardown.
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/gdma_main.c | 2 +-
drivers/net/ethernet/microsoft/mana/mana_en.c | 18 +++++++++++++++++-
include/net/mana/gdma.h | 1 +
3 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index e1a0e897b1b9..53281fef2ccd 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -894,7 +894,6 @@ static void mana_gd_deregister_irq(struct gdma_queue *queue)
}
spin_unlock_irqrestore(&gic->lock, flags);
- queue->eq.msix_index = INVALID_PCI_MSIX_INDEX;
synchronize_rcu();
}
@@ -1009,6 +1008,7 @@ static int mana_gd_create_eq(struct gdma_dev *gd,
out:
dev_err(dev, "Failed to create EQ: %d\n", err);
mana_gd_destroy_eq(gc, false, queue);
+ queue->eq.msix_index = INVALID_PCI_MSIX_INDEX;
return err;
}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 3ee74e7e300c..433ec88d0d69 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1625,6 +1625,7 @@ void mana_destroy_eq(struct mana_port_context *apc)
struct mana_context *ac = apc->ac;
struct gdma_context *gc = ac->gdma_dev->gdma_context;
struct gdma_queue *eq;
+ unsigned int msi;
int i;
if (!apc->eqs)
@@ -1638,7 +1639,9 @@ void mana_destroy_eq(struct mana_port_context *apc)
if (!eq)
continue;
+ msi = eq->eq.msix_index;
mana_gd_destroy_queue(gc, eq);
+ mana_gd_put_gic(gc, !gc->msi_sharing, msi);
}
kfree(apc->eqs);
@@ -1655,6 +1658,7 @@ static void mana_create_eq_debugfs(struct mana_port_context *apc, int i)
eq.mana_eq_debugfs = debugfs_create_dir(eqnum, apc->mana_eqs_debugfs);
debugfs_create_u32("head", 0400, eq.mana_eq_debugfs, &eq.eq->head);
debugfs_create_u32("tail", 0400, eq.mana_eq_debugfs, &eq.eq->tail);
+ debugfs_create_u32("irq", 0400, eq.mana_eq_debugfs, &eq.eq->eq.irq);
debugfs_create_file("eq_dump", 0400, eq.mana_eq_debugfs, eq.eq, &mana_dbg_q_fops);
}
@@ -1663,7 +1667,9 @@ int mana_create_eq(struct mana_port_context *apc)
struct gdma_dev *gd = apc->ac->gdma_dev;
struct gdma_context *gc = gd->gdma_context;
struct gdma_queue_spec spec = {};
+ struct gdma_irq_context *gic;
int err;
+ int msi;
int i;
if (WARN_ON(apc->eqs))
@@ -1683,12 +1689,22 @@ int mana_create_eq(struct mana_port_context *apc)
debugfs_create_dir("EQs", apc->mana_port_debugfs);
for (i = 0; i < apc->num_queues; i++) {
- spec.eq.msix_index = (i + 1) % gc->num_msix_usable;
+ msi = (i + 1) % gc->num_msix_usable;
+
+ gic = mana_gd_get_gic(gc, !gc->msi_sharing, &msi);
+ if (!gic) {
+ err = -ENOMEM;
+ goto out;
+ }
+ spec.eq.msix_index = msi;
+
err = mana_gd_create_mana_eq(gd, &spec, &apc->eqs[i].eq);
if (err) {
dev_err(gc->dev, "Failed to create EQ %d : %d\n", i, err);
+ mana_gd_put_gic(gc, !gc->msi_sharing, msi);
goto out;
}
+ apc->eqs[i].eq->eq.irq = gic->irq;
mana_create_eq_debugfs(apc, i);
}
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index fbe3c1427b45..6c138cc77407 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -342,6 +342,7 @@ struct gdma_queue {
void *context;
unsigned int msix_index;
+ unsigned int irq;
u32 log2_throttle_limit;
} eq;
--
2.43.0
^ permalink raw reply related
* [PATCH net-next v9 4/6] net: mana: Use GIC functions to allocate global EQs
From: Long Li @ 2026-05-13 22:09 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta
Cc: Simon Horman, netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260513220956.402058-1-longli@microsoft.com>
Replace the GDMA global interrupt setup code with the new GIC allocation
and release functions for managing interrupt contexts.
This changes the per-queue interrupt names in /proc/interrupts from
mana_q0, mana_q1, ... to mana_msi1, mana_msi2, ... to reflect the
MSI-X index rather than a zero-based queue number. The HWC interrupt
name (mana_hwc) is unchanged.
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/gdma_main.c | 96 +++----------------
1 file changed, 13 insertions(+), 83 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index fdd2ef24414b..e1a0e897b1b9 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -1915,7 +1915,7 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)
struct gdma_context *gc = pci_get_drvdata(pdev);
struct gdma_irq_context *gic;
bool skip_first_cpu = false;
- int *irqs, irq, err, i;
+ int *irqs, err, i;
irqs = kmalloc_objs(int, nvec);
if (!irqs)
@@ -1928,30 +1928,13 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)
* further used in irq_setup()
*/
for (i = 1; i <= nvec; i++) {
- gic = kzalloc_obj(*gic);
+ gic = mana_gd_get_gic(gc, false, &i);
if (!gic) {
err = -ENOMEM;
goto free_irq;
}
- gic->handler = mana_gd_process_eq_events;
- INIT_LIST_HEAD(&gic->eq_list);
- spin_lock_init(&gic->lock);
-
- snprintf(gic->name, MANA_IRQ_NAME_SZ, "mana_q%d@pci:%s",
- i - 1, pci_name(pdev));
-
- /* one pci vector is already allocated for HWC */
- irqs[i - 1] = pci_irq_vector(pdev, i);
- if (irqs[i - 1] < 0) {
- err = irqs[i - 1];
- goto free_current_gic;
- }
-
- err = request_irq(irqs[i - 1], mana_gd_intr, 0, gic->name, gic);
- if (err)
- goto free_current_gic;
- xa_store(&gc->irq_contexts, i, gic, GFP_KERNEL);
+ irqs[i - 1] = gic->irq;
}
/*
@@ -1973,20 +1956,9 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec)
kfree(irqs);
return 0;
-free_current_gic:
- kfree(gic);
free_irq:
- for (i -= 1; i > 0; i--) {
- irq = pci_irq_vector(pdev, i);
- gic = xa_load(&gc->irq_contexts, i);
- if (WARN_ON(!gic))
- continue;
-
- irq_update_affinity_hint(irq, NULL);
- free_irq(irq, gic);
- xa_erase(&gc->irq_contexts, i);
- kfree(gic);
- }
+ for (i -= 1; i > 0; i--)
+ mana_gd_put_gic(gc, false, i);
kfree(irqs);
return err;
}
@@ -1995,7 +1967,7 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev, int nvec)
{
struct gdma_context *gc = pci_get_drvdata(pdev);
struct gdma_irq_context *gic;
- int *irqs, *start_irqs, irq;
+ int *irqs, *start_irqs;
unsigned int cpu;
int err, i;
@@ -2006,34 +1978,13 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev, int nvec)
start_irqs = irqs;
for (i = 0; i < nvec; i++) {
- gic = kzalloc_obj(*gic);
+ gic = mana_gd_get_gic(gc, false, &i);
if (!gic) {
err = -ENOMEM;
goto free_irq;
}
- gic->handler = mana_gd_process_eq_events;
- INIT_LIST_HEAD(&gic->eq_list);
- spin_lock_init(&gic->lock);
-
- if (!i)
- snprintf(gic->name, MANA_IRQ_NAME_SZ, "mana_hwc@pci:%s",
- pci_name(pdev));
- else
- snprintf(gic->name, MANA_IRQ_NAME_SZ, "mana_q%d@pci:%s",
- i - 1, pci_name(pdev));
-
- irqs[i] = pci_irq_vector(pdev, i);
- if (irqs[i] < 0) {
- err = irqs[i];
- goto free_current_gic;
- }
-
- err = request_irq(irqs[i], mana_gd_intr, 0, gic->name, gic);
- if (err)
- goto free_current_gic;
-
- xa_store(&gc->irq_contexts, i, gic, GFP_KERNEL);
+ irqs[i] = gic->irq;
}
/* If number of IRQ is one extra than number of online CPUs,
@@ -2062,20 +2013,9 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev, int nvec)
kfree(start_irqs);
return 0;
-free_current_gic:
- kfree(gic);
free_irq:
- for (i -= 1; i >= 0; i--) {
- irq = pci_irq_vector(pdev, i);
- gic = xa_load(&gc->irq_contexts, i);
- if (WARN_ON(!gic))
- continue;
-
- irq_update_affinity_hint(irq, NULL);
- free_irq(irq, gic);
- xa_erase(&gc->irq_contexts, i);
- kfree(gic);
- }
+ for (i -= 1; i >= 0; i--)
+ mana_gd_put_gic(gc, false, i);
kfree(start_irqs);
return err;
@@ -2149,26 +2089,16 @@ static int mana_gd_setup_remaining_irqs(struct pci_dev *pdev)
static void mana_gd_remove_irqs(struct pci_dev *pdev)
{
struct gdma_context *gc = pci_get_drvdata(pdev);
- struct gdma_irq_context *gic;
- int irq, i;
+ int i;
if (gc->max_num_msix < 1)
return;
for (i = 0; i < gc->max_num_msix; i++) {
- irq = pci_irq_vector(pdev, i);
- if (irq < 0)
- continue;
-
- gic = xa_load(&gc->irq_contexts, i);
- if (WARN_ON(!gic))
+ if (!xa_load(&gc->irq_contexts, i))
continue;
- /* Need to clear the hint before free_irq */
- irq_update_affinity_hint(irq, NULL);
- free_irq(irq, gic);
- xa_erase(&gc->irq_contexts, i);
- kfree(gic);
+ mana_gd_put_gic(gc, false, i);
}
pci_free_irq_vectors(pdev);
--
2.43.0
^ permalink raw reply related
* [PATCH net-next v9 3/6] net: mana: Introduce GIC context with refcounting for interrupt management
From: Long Li @ 2026-05-13 22:09 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta
Cc: Simon Horman, netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260513220956.402058-1-longli@microsoft.com>
To allow Ethernet EQs to use dedicated or shared MSI-X vectors and RDMA
EQs to share the same MSI-X, introduce a GIC (GDMA IRQ Context) with
reference counting. This allows the driver to create an interrupt context
on an assigned or unassigned MSI-X vector and share it across multiple
EQ consumers.
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/gdma_main.c | 159 ++++++++++++++++++
include/net/mana/gdma.h | 12 ++
2 files changed, 171 insertions(+)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index bbd055849e36..fdd2ef24414b 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -1655,6 +1655,164 @@ static irqreturn_t mana_gd_intr(int irq, void *arg)
return IRQ_HANDLED;
}
+void mana_gd_put_gic(struct gdma_context *gc, bool use_msi_bitmap, int msi)
+{
+ struct pci_dev *dev = to_pci_dev(gc->dev);
+ struct gdma_irq_context *gic;
+ struct msi_map irq_map;
+ int irq;
+
+ mutex_lock(&gc->gic_mutex);
+
+ gic = xa_load(&gc->irq_contexts, msi);
+ if (WARN_ON(!gic)) {
+ mutex_unlock(&gc->gic_mutex);
+ return;
+ }
+
+ if (use_msi_bitmap)
+ gic->bitmap_refs--;
+
+ if (use_msi_bitmap && gic->bitmap_refs == 0)
+ clear_bit(msi, gc->msi_bitmap);
+
+ if (!refcount_dec_and_test(&gic->refcount))
+ goto out;
+
+ irq = gic->irq;
+
+ irq_update_affinity_hint(irq, NULL);
+ free_irq(irq, gic);
+
+ if (gic->dyn_msix) {
+ irq_map.virq = irq;
+ irq_map.index = msi;
+ pci_msix_free_irq(dev, irq_map);
+ }
+
+ xa_erase(&gc->irq_contexts, msi);
+ kfree(gic);
+
+out:
+ mutex_unlock(&gc->gic_mutex);
+}
+EXPORT_SYMBOL_NS(mana_gd_put_gic, "NET_MANA");
+
+/*
+ * Get a GIC (GDMA IRQ Context) on a MSI vector
+ * a MSI can be shared between different EQs, this function supports setting
+ * up separate MSIs using a bitmap, or directly using the MSI index
+ *
+ * @use_msi_bitmap:
+ * True if MSI is assigned by this function on available slots from bitmap.
+ * False if MSI is passed from *msi_requested
+ */
+struct gdma_irq_context *mana_gd_get_gic(struct gdma_context *gc,
+ bool use_msi_bitmap,
+ int *msi_requested)
+{
+ struct pci_dev *dev = to_pci_dev(gc->dev);
+ struct gdma_irq_context *gic;
+ struct msi_map irq_map = { };
+ int irq;
+ int msi;
+ int err;
+
+ mutex_lock(&gc->gic_mutex);
+
+ if (use_msi_bitmap) {
+ msi = find_first_zero_bit(gc->msi_bitmap, gc->num_msix_usable);
+ if (msi >= gc->num_msix_usable) {
+ dev_err(gc->dev, "No free MSI vectors available\n");
+ gic = NULL;
+ goto out;
+ }
+ *msi_requested = msi;
+ } else {
+ msi = *msi_requested;
+ }
+
+ gic = xa_load(&gc->irq_contexts, msi);
+ if (gic) {
+ refcount_inc(&gic->refcount);
+ if (use_msi_bitmap) {
+ gic->bitmap_refs++;
+ set_bit(msi, gc->msi_bitmap);
+ }
+ goto out;
+ }
+
+ irq = pci_irq_vector(dev, msi);
+ if (irq == -EINVAL) {
+ irq_map = pci_msix_alloc_irq_at(dev, msi, NULL);
+ if (!irq_map.virq) {
+ err = irq_map.index;
+ dev_err(gc->dev,
+ "Failed to alloc irq_map msi %d err %d\n",
+ msi, err);
+ gic = NULL;
+ goto out;
+ }
+ irq = irq_map.virq;
+ msi = irq_map.index;
+ }
+
+ gic = kzalloc(sizeof(*gic), GFP_KERNEL);
+ if (!gic) {
+ if (irq_map.virq)
+ pci_msix_free_irq(dev, irq_map);
+ goto out;
+ }
+
+ gic->handler = mana_gd_process_eq_events;
+ gic->msi = msi;
+ gic->irq = irq;
+ INIT_LIST_HEAD(&gic->eq_list);
+ spin_lock_init(&gic->lock);
+
+ if (!gic->msi)
+ snprintf(gic->name, MANA_IRQ_NAME_SZ, "mana_hwc@pci:%s",
+ pci_name(dev));
+ else
+ snprintf(gic->name, MANA_IRQ_NAME_SZ, "mana_msi%d@pci:%s",
+ gic->msi, pci_name(dev));
+
+ err = request_irq(irq, mana_gd_intr, 0, gic->name, gic);
+ if (err) {
+ dev_err(gc->dev, "Failed to request irq %d %s\n",
+ irq, gic->name);
+ kfree(gic);
+ gic = NULL;
+ if (irq_map.virq)
+ pci_msix_free_irq(dev, irq_map);
+ goto out;
+ }
+
+ gic->dyn_msix = !!irq_map.virq;
+ refcount_set(&gic->refcount, 1);
+ gic->bitmap_refs = use_msi_bitmap ? 1 : 0;
+
+ err = xa_err(xa_store(&gc->irq_contexts, msi, gic, GFP_KERNEL));
+ if (err) {
+ dev_err(gc->dev, "Failed to store irq context for msi %d: %d\n",
+ msi, err);
+ free_irq(irq, gic);
+ kfree(gic);
+ gic = NULL;
+ if (irq_map.virq)
+ pci_msix_free_irq(dev, irq_map);
+ goto out;
+ }
+
+ if (use_msi_bitmap)
+ set_bit(msi, gc->msi_bitmap);
+
+out:
+ mutex_unlock(&gc->gic_mutex);
+ return gic;
+}
+EXPORT_SYMBOL_NS(mana_gd_get_gic, "NET_MANA");
+
int mana_gd_alloc_res_map(u32 res_avail, struct gdma_resource *r)
{
r->map = bitmap_zalloc(res_avail, GFP_KERNEL);
@@ -2144,6 +2302,7 @@ static int mana_gd_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
goto release_region;
mutex_init(&gc->eq_test_event_mutex);
+ mutex_init(&gc->gic_mutex);
pci_set_drvdata(pdev, gc);
gc->bar0_pa = pci_resource_start(pdev, 0);
gc->bar0_size = pci_resource_len(pdev, 0);
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 9c05b1e15c3e..fbe3c1427b45 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -388,6 +388,11 @@ struct gdma_irq_context {
spinlock_t lock;
struct list_head eq_list;
char name[MANA_IRQ_NAME_SZ];
+ unsigned int msi;
+ unsigned int irq;
+ refcount_t refcount;
+ unsigned int bitmap_refs;
+ bool dyn_msix;
};
enum gdma_context_flags {
@@ -449,6 +454,9 @@ struct gdma_context {
unsigned long flags;
+ /* Protect access to GIC context */
+ struct mutex gic_mutex;
+
/* Indicate if this device is sharing MSI for EQs on MANA */
bool msi_sharing;
@@ -1026,6 +1034,10 @@ int mana_gd_resume(struct pci_dev *pdev);
bool mana_need_log(struct gdma_context *gc, int err);
+struct gdma_irq_context *mana_gd_get_gic(struct gdma_context *gc,
+ bool use_msi_bitmap,
+ int *msi_requested);
+void mana_gd_put_gic(struct gdma_context *gc, bool use_msi_bitmap, int msi);
int mana_gd_query_device_cfg(struct gdma_context *gc, u32 proto_major_ver,
u32 proto_minor_ver, u32 proto_micro_ver,
u16 *max_num_vports, u8 *bm_hostmode);
--
2.43.0
^ permalink raw reply related
* [PATCH net-next v9 2/6] net: mana: Query device capabilities and configure MSI-X sharing for EQs
From: Long Li @ 2026-05-13 22:09 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta
Cc: Simon Horman, netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260513220956.402058-1-longli@microsoft.com>
When querying the device, adjust the max number of queues to allow
dedicated MSI-X vectors for each vPort. The per-vPort queue count is
clamped towards MANA_DEF_NUM_QUEUES but will not exceed the hardware
maximum reported by the device.
MSI-X sharing among vPorts is enabled when there are not enough MSI-X
vectors for dedicated allocation, or when the platform does not support
dynamic MSI-X allocation (in which case all vectors are pre-allocated
at probe time and sharing is always used). The msi_sharing flag is
reset at the top of mana_gd_query_max_resources() so it is recomputed
from current hardware state on each probe or resume cycle.
A device reporting zero ports now results in a fatal probe error since
the per-vPort MSI-X math requires at least one port.
Rename mana_query_device_cfg() to mana_gd_query_device_cfg() as it is
used at GDMA device probe time for querying device capabilities.
Signed-off-by: Long Li <longli@microsoft.com>
---
.../net/ethernet/microsoft/mana/gdma_main.c | 66 ++++++++++++++++++-
drivers/net/ethernet/microsoft/mana/mana_en.c | 40 ++++++-----
include/net/mana/gdma.h | 13 +++-
3 files changed, 100 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index 3bc3fff55999..bbd055849e36 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -179,8 +179,18 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev)
struct gdma_context *gc = pci_get_drvdata(pdev);
struct gdma_query_max_resources_resp resp = {};
struct gdma_general_req req = {};
+ unsigned int max_num_queues;
+ u8 bm_hostmode;
+ u16 num_ports;
int err;
+ /* Reset msi_sharing so it is recomputed from current hardware
+ * state. On resume, num_online_cpus() or num_msix_usable may
+ * have changed, making dedicated MSI-X feasible where it was
+ * not before.
+ */
+ gc->msi_sharing = false;
+
mana_gd_init_req_hdr(&req.hdr, GDMA_QUERY_MAX_RESOURCES,
sizeof(req), sizeof(resp));
@@ -227,6 +237,43 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev)
if (gc->max_num_queues == 0)
return -ENOSPC;
+ err = mana_gd_query_device_cfg(gc, MANA_MAJOR_VERSION,
+ MANA_MINOR_VERSION,
+ MANA_MICRO_VERSION,
+ &num_ports, &bm_hostmode);
+ if (err)
+ return err;
+
+ if (!num_ports)
+ return -EINVAL;
+
+ /*
+ * Adjust the per-vPort max queue count to allow dedicated
+ * MSIx for each vPort. Clamp to no less than MANA_DEF_NUM_QUEUES.
+ */
+ max_num_queues = (gc->num_msix_usable - 1) / num_ports;
+ max_num_queues = rounddown_pow_of_two(max(max_num_queues, 1U));
+ if (max_num_queues < MANA_DEF_NUM_QUEUES)
+ max_num_queues = MANA_DEF_NUM_QUEUES;
+
+ /*
+ * Use dedicated MSIx for EQs whenever possible, use MSIx sharing for
+ * Ethernet EQs when (max_num_queues * num_ports > num_msix_usable - 1).
+ */
+ max_num_queues = min(gc->max_num_queues, max_num_queues);
+ if (max_num_queues * num_ports > gc->num_msix_usable - 1)
+ gc->msi_sharing = true;
+
+ /* If MSI is shared, use max allowed value */
+ if (gc->msi_sharing)
+ gc->max_num_queues_vport = min(gc->num_msix_usable - 1,
+ gc->max_num_queues);
+ else
+ gc->max_num_queues_vport = max_num_queues;
+
+ dev_info(gc->dev, "MSI sharing mode %d max queues %d\n",
+ gc->msi_sharing, gc->max_num_queues);
+
return 0;
}
@@ -1889,6 +1936,7 @@ static int mana_gd_setup_hwc_irqs(struct pci_dev *pdev)
/* Need 1 interrupt for HWC */
max_irqs = min(num_online_cpus(), MANA_MAX_NUM_QUEUES) + 1;
min_irqs = 2;
+ gc->msi_sharing = true;
}
nvec = pci_alloc_irq_vectors(pdev, min_irqs, max_irqs, PCI_IRQ_MSIX);
@@ -1967,6 +2015,8 @@ static void mana_gd_remove_irqs(struct pci_dev *pdev)
pci_free_irq_vectors(pdev);
+ bitmap_free(gc->msi_bitmap);
+ gc->msi_bitmap = NULL;
gc->max_num_msix = 0;
gc->num_msix_usable = 0;
}
@@ -2001,6 +2051,10 @@ static int mana_gd_setup(struct pci_dev *pdev)
if (err)
goto destroy_hwc;
+ err = mana_gd_detect_devices(pdev);
+ if (err)
+ goto destroy_hwc;
+
err = mana_gd_query_max_resources(pdev);
if (err)
goto destroy_hwc;
@@ -2011,9 +2065,15 @@ static int mana_gd_setup(struct pci_dev *pdev)
goto destroy_hwc;
}
- err = mana_gd_detect_devices(pdev);
- if (err)
- goto destroy_hwc;
+ if (!gc->msi_sharing) {
+ gc->msi_bitmap = bitmap_zalloc(gc->num_msix_usable, GFP_KERNEL);
+ if (!gc->msi_bitmap) {
+ err = -ENOMEM;
+ goto destroy_hwc;
+ }
+ /* Set bit for HWC */
+ set_bit(0, gc->msi_bitmap);
+ }
dev_dbg(&pdev->dev, "mana gdma setup successful\n");
return 0;
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index f1f6f7940b61..3ee74e7e300c 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1007,10 +1007,9 @@ static int mana_init_port_context(struct mana_port_context *apc)
return !apc->rxqs ? -ENOMEM : 0;
}
-static int mana_send_request(struct mana_context *ac, void *in_buf,
- u32 in_len, void *out_buf, u32 out_len)
+static int gdma_mana_send_request(struct gdma_context *gc, void *in_buf,
+ u32 in_len, void *out_buf, u32 out_len)
{
- struct gdma_context *gc = ac->gdma_dev->gdma_context;
struct gdma_resp_hdr *resp = out_buf;
struct gdma_req_hdr *req = in_buf;
struct device *dev = gc->dev;
@@ -1044,6 +1043,14 @@ static int mana_send_request(struct mana_context *ac, void *in_buf,
return 0;
}
+static int mana_send_request(struct mana_context *ac, void *in_buf,
+ u32 in_len, void *out_buf, u32 out_len)
+{
+ struct gdma_context *gc = ac->gdma_dev->gdma_context;
+
+ return gdma_mana_send_request(gc, in_buf, in_len, out_buf, out_len);
+}
+
static int mana_verify_resp_hdr(const struct gdma_resp_hdr *resp_hdr,
const enum mana_command_code expected_code,
const u32 min_size)
@@ -1177,11 +1184,10 @@ static void mana_pf_deregister_filter(struct mana_port_context *apc)
err, resp.hdr.status);
}
-static int mana_query_device_cfg(struct mana_context *ac, u32 proto_major_ver,
- u32 proto_minor_ver, u32 proto_micro_ver,
- u16 *max_num_vports, u8 *bm_hostmode)
+int mana_gd_query_device_cfg(struct gdma_context *gc, u32 proto_major_ver,
+ u32 proto_minor_ver, u32 proto_micro_ver,
+ u16 *max_num_vports, u8 *bm_hostmode)
{
- struct gdma_context *gc = ac->gdma_dev->gdma_context;
struct mana_query_device_cfg_resp resp = {};
struct mana_query_device_cfg_req req = {};
struct device *dev = gc->dev;
@@ -1196,7 +1202,8 @@ static int mana_query_device_cfg(struct mana_context *ac, u32 proto_major_ver,
req.proto_minor_ver = proto_minor_ver;
req.proto_micro_ver = proto_micro_ver;
- err = mana_send_request(ac, &req, sizeof(req), &resp, sizeof(resp));
+ err = gdma_mana_send_request(gc, &req, sizeof(req),
+ &resp, sizeof(resp));
if (err) {
dev_err(dev, "Failed to query config: %d", err);
return err;
@@ -1230,8 +1237,6 @@ static int mana_query_device_cfg(struct mana_context *ac, u32 proto_major_ver,
else
*bm_hostmode = 0;
- debugfs_create_u16("adapter-MTU", 0400, gc->mana_pci_debugfs, &gc->adapter_mtu);
-
return 0;
}
@@ -3416,7 +3421,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
int err;
ndev = alloc_etherdev_mq(sizeof(struct mana_port_context),
- gc->max_num_queues);
+ gc->max_num_queues_vport);
if (!ndev)
return -ENOMEM;
@@ -3425,9 +3430,9 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
apc = netdev_priv(ndev);
apc->ac = ac;
apc->ndev = ndev;
- apc->max_queues = gc->max_num_queues;
+ apc->max_queues = gc->max_num_queues_vport;
/* Use MANA_DEF_NUM_QUEUES as default, still honoring the HW limit */
- apc->num_queues = min(gc->max_num_queues, MANA_DEF_NUM_QUEUES);
+ apc->num_queues = min(gc->max_num_queues_vport, MANA_DEF_NUM_QUEUES);
apc->tx_queue_size = DEF_TX_BUFFERS_PER_QUEUE;
apc->rx_queue_size = DEF_RX_BUFFERS_PER_QUEUE;
apc->port_handle = INVALID_MANA_HANDLE;
@@ -3691,13 +3696,18 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
INIT_DELAYED_WORK(&ac->gf_stats_work, mana_gf_stats_work_handler);
- err = mana_query_device_cfg(ac, MANA_MAJOR_VERSION, MANA_MINOR_VERSION,
- MANA_MICRO_VERSION, &num_ports, &bm_hostmode);
+ err = mana_gd_query_device_cfg(gc, MANA_MAJOR_VERSION,
+ MANA_MINOR_VERSION,
+ MANA_MICRO_VERSION,
+ &num_ports, &bm_hostmode);
if (err)
goto out;
ac->bm_hostmode = bm_hostmode;
+ debugfs_create_u16("adapter-MTU", 0400,
+ gc->mana_pci_debugfs, &gc->adapter_mtu);
+
if (!resuming) {
ac->num_ports = num_ports;
} else {
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 6d836060976a..9c05b1e15c3e 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -399,8 +399,10 @@ struct gdma_context {
struct device *dev;
struct dentry *mana_pci_debugfs;
- /* Per-vPort max number of queues */
+ /* Hardware max number of queues */
unsigned int max_num_queues;
+ /* Per-vPort max number of queues */
+ unsigned int max_num_queues_vport;
unsigned int max_num_msix;
unsigned int num_msix_usable;
struct xarray irq_contexts;
@@ -446,6 +448,12 @@ struct gdma_context {
struct workqueue_struct *service_wq;
unsigned long flags;
+
+ /* Indicate if this device is sharing MSI for EQs on MANA */
+ bool msi_sharing;
+
+ /* Bitmap tracks where MSI is allocated when it is not shared for EQs */
+ unsigned long *msi_bitmap;
};
static inline bool mana_gd_is_mana(struct gdma_dev *gd)
@@ -1018,4 +1026,7 @@ int mana_gd_resume(struct pci_dev *pdev);
bool mana_need_log(struct gdma_context *gc, int err);
+int mana_gd_query_device_cfg(struct gdma_context *gc, u32 proto_major_ver,
+ u32 proto_minor_ver, u32 proto_micro_ver,
+ u16 *max_num_vports, u8 *bm_hostmode);
#endif /* _GDMA_H */
--
2.43.0
^ permalink raw reply related
* [PATCH net-next v9 1/6] net: mana: Create separate EQs for each vPort
From: Long Li @ 2026-05-13 22:09 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta
Cc: Simon Horman, netdev, linux-rdma, linux-hyperv, linux-kernel
In-Reply-To: <20260513220956.402058-1-longli@microsoft.com>
To prepare for assigning vPorts to dedicated MSI-X vectors, remove EQ
sharing among the vPorts and create dedicated EQs for each vPort.
Move the EQ definition from struct mana_context to struct mana_port_context
and update related support functions. Export mana_create_eq() and
mana_destroy_eq() for use by the MANA RDMA driver.
RSS QPs now take a vport reference via pd->vport_use_count to ensure
EQs outlive all QP consumers. The vport must already be configured by
a raw QP before an RSS QP can be created. EQs are only destroyed when
the last QP (raw or RSS) on the PD releases its reference.
Serialize mana_set_channels() against RDMA vport configuration to
prevent num_queues from changing while RDMA holds EQs sized to the
current value. When the port is down, apc->vport_mutex is held for
the entire operation since mana_detach()/mana_attach() do not take
vport_mutex in that case. When the port is up, Ethernet owns the
vport exclusively so no additional locking is needed.
Signed-off-by: Long Li <longli@microsoft.com>
---
drivers/infiniband/hw/mana/main.c | 24 ++--
drivers/infiniband/hw/mana/qp.c | 37 +++++-
drivers/net/ethernet/microsoft/mana/mana_en.c | 112 +++++++++++-------
.../ethernet/microsoft/mana/mana_ethtool.c | 27 ++++-
include/net/mana/mana.h | 7 +-
5 files changed, 145 insertions(+), 62 deletions(-)
diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index ac5e75dd3494..6159bd03a021 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -20,8 +20,10 @@ void mana_ib_uncfg_vport(struct mana_ib_dev *dev, struct mana_ib_pd *pd,
pd->vport_use_count--;
WARN_ON(pd->vport_use_count < 0);
- if (!pd->vport_use_count)
+ if (!pd->vport_use_count) {
+ mana_destroy_eq(mpc);
mana_uncfg_vport(mpc);
+ }
mutex_unlock(&pd->vport_mutex);
}
@@ -55,15 +57,23 @@ int mana_ib_cfg_vport(struct mana_ib_dev *dev, u32 port, struct mana_ib_pd *pd,
return err;
}
- mutex_unlock(&pd->vport_mutex);
- pd->tx_shortform_allowed = mpc->tx_shortform_allowed;
- pd->tx_vp_offset = mpc->tx_vp_offset;
+ err = mana_create_eq(mpc);
+ if (err) {
+ mana_uncfg_vport(mpc);
+ pd->vport_use_count--;
+ } else {
+ pd->tx_shortform_allowed = mpc->tx_shortform_allowed;
+ pd->tx_vp_offset = mpc->tx_vp_offset;
+ }
- ibdev_dbg(&dev->ib_dev, "vport handle %llx pdid %x doorbell_id %x\n",
- mpc->port_handle, pd->pdn, doorbell_id);
+ mutex_unlock(&pd->vport_mutex);
- return 0;
+ if (!err)
+ ibdev_dbg(&dev->ib_dev, "vport handle %llx pdid %x doorbell_id %x\n",
+ mpc->port_handle, pd->pdn, doorbell_id);
+
+ return err;
}
int mana_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)
diff --git a/drivers/infiniband/hw/mana/qp.c b/drivers/infiniband/hw/mana/qp.c
index 0fbcf449c134..108ec4c5ce51 100644
--- a/drivers/infiniband/hw/mana/qp.c
+++ b/drivers/infiniband/hw/mana/qp.c
@@ -79,6 +79,7 @@ static int mana_ib_create_qp_rss(struct ib_qp *ibqp, struct ib_pd *pd,
struct ib_qp_init_attr *attr,
struct ib_udata *udata)
{
+ struct mana_ib_pd *mana_pd = container_of(pd, struct mana_ib_pd, ibpd);
struct mana_ib_qp *qp = container_of(ibqp, struct mana_ib_qp, ibqp);
struct mana_ib_dev *mdev =
container_of(pd->device, struct mana_ib_dev, ib_dev);
@@ -155,6 +156,18 @@ static int mana_ib_create_qp_rss(struct ib_qp *ibqp, struct ib_pd *pd,
qp->port = port;
+ /* Take a reference on the vport to ensure EQs outlive this QP.
+ * The vport must already be configured by a raw QP.
+ */
+ mutex_lock(&mana_pd->vport_mutex);
+ if (!mana_pd->vport_use_count) {
+ mutex_unlock(&mana_pd->vport_mutex);
+ ret = -EINVAL;
+ goto fail;
+ }
+ mana_pd->vport_use_count++;
+ mutex_unlock(&mana_pd->vport_mutex);
+
for (i = 0; i < ind_tbl_size; i++) {
struct mana_obj_spec wq_spec = {};
struct mana_obj_spec cq_spec = {};
@@ -171,13 +184,13 @@ static int mana_ib_create_qp_rss(struct ib_qp *ibqp, struct ib_pd *pd,
cq_spec.gdma_region = cq->queue.gdma_region;
cq_spec.queue_size = cq->cqe * COMP_ENTRY_SIZE;
cq_spec.modr_ctx_id = 0;
- eq = &mpc->ac->eqs[cq->comp_vector];
+ eq = &mpc->eqs[cq->comp_vector % mpc->num_queues];
cq_spec.attached_eq = eq->eq->id;
ret = mana_create_wq_obj(mpc, mpc->port_handle, GDMA_RQ,
&wq_spec, &cq_spec, &wq->rx_object);
if (ret)
- goto fail;
+ goto free_vport;
/* The GDMA regions are now owned by the WQ object */
wq->queue.gdma_region = GDMA_INVALID_DMA_REGION;
@@ -199,7 +212,7 @@ static int mana_ib_create_qp_rss(struct ib_qp *ibqp, struct ib_pd *pd,
ret = mana_ib_install_cq_cb(mdev, cq);
if (ret) {
mana_destroy_wq_obj(mpc, GDMA_RQ, wq->rx_object);
- goto fail;
+ goto free_vport;
}
}
resp.num_entries = i;
@@ -210,7 +223,7 @@ static int mana_ib_create_qp_rss(struct ib_qp *ibqp, struct ib_pd *pd,
ucmd.rx_hash_key_len,
ucmd.rx_hash_key);
if (ret)
- goto fail;
+ goto free_vport;
ret = ib_copy_to_udata(udata, &resp, sizeof(resp));
if (ret) {
@@ -226,7 +239,7 @@ static int mana_ib_create_qp_rss(struct ib_qp *ibqp, struct ib_pd *pd,
err_disable_vport_rx:
mana_disable_vport_rx(mpc);
-fail:
+free_vport:
while (i-- > 0) {
ibwq = ind_tbl->ind_tbl[i];
ibcq = ibwq->cq;
@@ -237,6 +250,9 @@ static int mana_ib_create_qp_rss(struct ib_qp *ibqp, struct ib_pd *pd,
mana_destroy_wq_obj(mpc, GDMA_RQ, wq->rx_object);
}
+ mana_ib_uncfg_vport(mdev, mana_pd, port);
+
+fail:
kfree(mana_ind_table);
return ret;
@@ -321,7 +337,11 @@ static int mana_ib_create_qp_raw(struct ib_qp *ibqp, struct ib_pd *ibpd,
cq_spec.queue_size = send_cq->cqe * COMP_ENTRY_SIZE;
cq_spec.modr_ctx_id = 0;
eq_vec = send_cq->comp_vector;
- eq = &mpc->ac->eqs[eq_vec];
+ if (!mpc->eqs) {
+ err = -EINVAL;
+ goto err_destroy_queue;
+ }
+ eq = &mpc->eqs[eq_vec % mpc->num_queues];
cq_spec.attached_eq = eq->eq->id;
err = mana_create_wq_obj(mpc, mpc->port_handle, GDMA_SQ, &wq_spec,
@@ -785,14 +805,17 @@ static int mana_ib_destroy_qp_rss(struct mana_ib_qp *qp,
{
struct mana_ib_dev *mdev =
container_of(qp->ibqp.device, struct mana_ib_dev, ib_dev);
+ struct ib_pd *ibpd = qp->ibqp.pd;
struct mana_port_context *mpc;
struct net_device *ndev;
+ struct mana_ib_pd *pd;
struct mana_ib_wq *wq;
struct ib_wq *ibwq;
int i;
ndev = mana_ib_get_netdev(qp->ibqp.device, qp->port);
mpc = netdev_priv(ndev);
+ pd = container_of(ibpd, struct mana_ib_pd, ibpd);
/* Disable vPort RX steering before destroying RX WQ objects.
* Otherwise firmware still routes traffic to the destroyed queues,
@@ -817,6 +840,8 @@ static int mana_ib_destroy_qp_rss(struct mana_ib_qp *qp,
mana_destroy_wq_obj(mpc, GDMA_RQ, wq->rx_object);
}
+ mana_ib_uncfg_vport(mdev, pd, qp->port);
+
return 0;
}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index b2faa7cf398f..f1f6f7940b61 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1615,78 +1615,84 @@ void mana_destroy_wq_obj(struct mana_port_context *apc, u32 wq_type,
}
EXPORT_SYMBOL_NS(mana_destroy_wq_obj, "NET_MANA");
-static void mana_destroy_eq(struct mana_context *ac)
+void mana_destroy_eq(struct mana_port_context *apc)
{
+ struct mana_context *ac = apc->ac;
struct gdma_context *gc = ac->gdma_dev->gdma_context;
struct gdma_queue *eq;
int i;
- if (!ac->eqs)
+ if (!apc->eqs)
return;
- debugfs_remove_recursive(ac->mana_eqs_debugfs);
- ac->mana_eqs_debugfs = NULL;
+ debugfs_remove_recursive(apc->mana_eqs_debugfs);
+ apc->mana_eqs_debugfs = NULL;
- for (i = 0; i < gc->max_num_queues; i++) {
- eq = ac->eqs[i].eq;
+ for (i = 0; i < apc->num_queues; i++) {
+ eq = apc->eqs[i].eq;
if (!eq)
continue;
mana_gd_destroy_queue(gc, eq);
}
- kfree(ac->eqs);
- ac->eqs = NULL;
+ kfree(apc->eqs);
+ apc->eqs = NULL;
}
+EXPORT_SYMBOL_NS(mana_destroy_eq, "NET_MANA");
-static void mana_create_eq_debugfs(struct mana_context *ac, int i)
+static void mana_create_eq_debugfs(struct mana_port_context *apc, int i)
{
- struct mana_eq eq = ac->eqs[i];
+ struct mana_eq eq = apc->eqs[i];
char eqnum[32];
sprintf(eqnum, "eq%d", i);
- eq.mana_eq_debugfs = debugfs_create_dir(eqnum, ac->mana_eqs_debugfs);
+ eq.mana_eq_debugfs = debugfs_create_dir(eqnum, apc->mana_eqs_debugfs);
debugfs_create_u32("head", 0400, eq.mana_eq_debugfs, &eq.eq->head);
debugfs_create_u32("tail", 0400, eq.mana_eq_debugfs, &eq.eq->tail);
debugfs_create_file("eq_dump", 0400, eq.mana_eq_debugfs, eq.eq, &mana_dbg_q_fops);
}
-static int mana_create_eq(struct mana_context *ac)
+int mana_create_eq(struct mana_port_context *apc)
{
- struct gdma_dev *gd = ac->gdma_dev;
+ struct gdma_dev *gd = apc->ac->gdma_dev;
struct gdma_context *gc = gd->gdma_context;
struct gdma_queue_spec spec = {};
int err;
int i;
- ac->eqs = kzalloc_objs(struct mana_eq, gc->max_num_queues);
- if (!ac->eqs)
+ if (WARN_ON(apc->eqs))
+ return -EEXIST;
+ apc->eqs = kzalloc_objs(struct mana_eq, apc->num_queues);
+ if (!apc->eqs)
return -ENOMEM;
spec.type = GDMA_EQ;
spec.monitor_avl_buf = false;
spec.queue_size = EQ_SIZE;
spec.eq.callback = NULL;
- spec.eq.context = ac->eqs;
+ spec.eq.context = apc->eqs;
spec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE;
- ac->mana_eqs_debugfs = debugfs_create_dir("EQs", gc->mana_pci_debugfs);
+ apc->mana_eqs_debugfs =
+ debugfs_create_dir("EQs", apc->mana_port_debugfs);
- for (i = 0; i < gc->max_num_queues; i++) {
+ for (i = 0; i < apc->num_queues; i++) {
spec.eq.msix_index = (i + 1) % gc->num_msix_usable;
- err = mana_gd_create_mana_eq(gd, &spec, &ac->eqs[i].eq);
+ err = mana_gd_create_mana_eq(gd, &spec, &apc->eqs[i].eq);
if (err) {
dev_err(gc->dev, "Failed to create EQ %d : %d\n", i, err);
goto out;
}
- mana_create_eq_debugfs(ac, i);
+ mana_create_eq_debugfs(apc, i);
}
return 0;
out:
- mana_destroy_eq(ac);
+ mana_destroy_eq(apc);
return err;
}
+EXPORT_SYMBOL_NS(mana_create_eq, "NET_MANA");
static int mana_fence_rq(struct mana_port_context *apc, struct mana_rxq *rxq)
{
@@ -2451,7 +2457,7 @@ static int mana_create_txq(struct mana_port_context *apc,
spec.monitor_avl_buf = false;
spec.queue_size = cq_size;
spec.cq.callback = mana_schedule_napi;
- spec.cq.parent_eq = ac->eqs[i].eq;
+ spec.cq.parent_eq = apc->eqs[i].eq;
spec.cq.context = cq;
err = mana_gd_create_mana_wq_cq(gd, &spec, &cq->gdma_cq);
if (err)
@@ -2844,13 +2850,12 @@ static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx)
static int mana_add_rx_queues(struct mana_port_context *apc,
struct net_device *ndev)
{
- struct mana_context *ac = apc->ac;
struct mana_rxq *rxq;
int err = 0;
int i;
for (i = 0; i < apc->num_queues; i++) {
- rxq = mana_create_rxq(apc, i, &ac->eqs[i], ndev);
+ rxq = mana_create_rxq(apc, i, &apc->eqs[i], ndev);
if (!rxq) {
err = -ENOMEM;
netdev_err(ndev, "Failed to create rxq %d : %d\n", i, err);
@@ -2869,9 +2874,8 @@ static int mana_add_rx_queues(struct mana_port_context *apc,
return err;
}
-static void mana_destroy_vport(struct mana_port_context *apc)
+static void mana_destroy_rxqs(struct mana_port_context *apc)
{
- struct gdma_dev *gd = apc->ac->gdma_dev;
struct mana_rxq *rxq;
u32 rxq_idx;
@@ -2883,8 +2887,12 @@ static void mana_destroy_vport(struct mana_port_context *apc)
mana_destroy_rxq(apc, rxq, true);
apc->rxqs[rxq_idx] = NULL;
}
+}
+
+static void mana_destroy_vport(struct mana_port_context *apc)
+{
+ struct gdma_dev *gd = apc->ac->gdma_dev;
- mana_destroy_txq(apc);
mana_uncfg_vport(apc);
if (gd->gdma_context->is_pf && !apc->ac->bm_hostmode)
@@ -2905,11 +2913,7 @@ static int mana_create_vport(struct mana_port_context *apc,
return err;
}
- err = mana_cfg_vport(apc, gd->pdid, gd->doorbell);
- if (err)
- return err;
-
- return mana_create_txq(apc, net);
+ return mana_cfg_vport(apc, gd->pdid, gd->doorbell);
}
static int mana_rss_table_alloc(struct mana_port_context *apc)
@@ -3195,21 +3199,36 @@ int mana_alloc_queues(struct net_device *ndev)
err = mana_create_vport(apc, ndev);
if (err) {
- netdev_err(ndev, "Failed to create vPort %u : %d\n", apc->port_idx, err);
+ netdev_err(ndev, "Failed to create vPort %u : %d\n",
+ apc->port_idx, err);
return err;
}
+ err = mana_create_eq(apc);
+ if (err) {
+ netdev_err(ndev, "Failed to create EQ on vPort %u: %d\n",
+ apc->port_idx, err);
+ goto destroy_vport;
+ }
+
+ err = mana_create_txq(apc, ndev);
+ if (err) {
+ netdev_err(ndev, "Failed to create TXQ on vPort %u: %d\n",
+ apc->port_idx, err);
+ goto destroy_eq;
+ }
+
err = netif_set_real_num_tx_queues(ndev, apc->num_queues);
if (err) {
netdev_err(ndev,
"netif_set_real_num_tx_queues () failed for ndev with num_queues %u : %d\n",
apc->num_queues, err);
- goto destroy_vport;
+ goto destroy_txq;
}
err = mana_add_rx_queues(apc, ndev);
if (err)
- goto destroy_vport;
+ goto destroy_rxq;
apc->rss_state = apc->num_queues > 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;
@@ -3218,7 +3237,7 @@ int mana_alloc_queues(struct net_device *ndev)
netdev_err(ndev,
"netif_set_real_num_rx_queues () failed for ndev with num_queues %u : %d\n",
apc->num_queues, err);
- goto destroy_vport;
+ goto destroy_rxq;
}
mana_rss_table_init(apc);
@@ -3226,19 +3245,25 @@ int mana_alloc_queues(struct net_device *ndev)
err = mana_config_rss(apc, TRI_STATE_TRUE, true, true);
if (err) {
netdev_err(ndev, "Failed to configure RSS table: %d\n", err);
- goto destroy_vport;
+ goto destroy_rxq;
}
if (gd->gdma_context->is_pf && !apc->ac->bm_hostmode) {
err = mana_pf_register_filter(apc);
if (err)
- goto destroy_vport;
+ goto destroy_rxq;
}
mana_chn_setxdp(apc, mana_xdp_get(apc));
return 0;
+destroy_rxq:
+ mana_destroy_rxqs(apc);
+destroy_txq:
+ mana_destroy_txq(apc);
+destroy_eq:
+ mana_destroy_eq(apc);
destroy_vport:
mana_destroy_vport(apc);
return err;
@@ -3343,6 +3368,9 @@ static int mana_dealloc_queues(struct net_device *ndev)
mana_fence_rqs(apc);
/* Even in err case, still need to cleanup the vPort */
+ mana_destroy_rxqs(apc);
+ mana_destroy_txq(apc);
+ mana_destroy_eq(apc);
mana_destroy_vport(apc);
return 0;
@@ -3663,12 +3691,6 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
INIT_DELAYED_WORK(&ac->gf_stats_work, mana_gf_stats_work_handler);
- err = mana_create_eq(ac);
- if (err) {
- dev_err(dev, "Failed to create EQs: %d\n", err);
- goto out;
- }
-
err = mana_query_device_cfg(ac, MANA_MAJOR_VERSION, MANA_MINOR_VERSION,
MANA_MICRO_VERSION, &num_ports, &bm_hostmode);
if (err)
@@ -3808,8 +3830,6 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
free_netdev(ndev);
}
- mana_destroy_eq(ac);
-
if (ac->per_port_queue_reset_wq) {
destroy_workqueue(ac->per_port_queue_reset_wq);
ac->per_port_queue_reset_wq = NULL;
diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
index 04350973e19e..e121834d17f3 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
@@ -454,18 +454,40 @@ static int mana_set_coalesce(struct net_device *ndev,
return err;
}
+/* mana_set_channels - change the number of queues on a port
+ *
+ * Returns -EBUSY if the port is down and RDMA holds the vport with
+ * EQs sized to the current num_queues.
+ */
static int mana_set_channels(struct net_device *ndev,
struct ethtool_channels *channels)
{
struct mana_port_context *apc = netdev_priv(ndev);
unsigned int new_count = channels->combined_count;
unsigned int old_count = apc->num_queues;
+ bool locked = false;
int err;
+ /* When the port is down, hold vport_mutex for the entire
+ * operation to serialize against RDMA's mana_cfg_vport().
+ * This is safe because mana_detach()/mana_attach() skip
+ * vport teardown/setup when port_st_save is false.
+ * When the port is up, Ethernet owns the vport exclusively
+ * so no locking against RDMA is needed.
+ */
+ if (!apc->port_is_up) {
+ mutex_lock(&apc->vport_mutex);
+ if (apc->vport_use_count) {
+ mutex_unlock(&apc->vport_mutex);
+ return -EBUSY;
+ }
+ locked = true;
+ }
+
err = mana_pre_alloc_rxbufs(apc, ndev->mtu, new_count);
if (err) {
netdev_err(ndev, "Insufficient memory for new allocations");
- return err;
+ goto unlock;
}
err = mana_detach(ndev, false);
@@ -483,6 +505,9 @@ static int mana_set_channels(struct net_device *ndev,
out:
mana_pre_dealloc_rxbufs(apc);
+unlock:
+ if (locked)
+ mutex_unlock(&apc->vport_mutex);
return err;
}
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index aa90a858c8e3..c8e7d16f6685 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -480,8 +480,6 @@ struct mana_context {
u8 bm_hostmode;
struct mana_ethtool_hc_stats hc_stats;
- struct mana_eq *eqs;
- struct dentry *mana_eqs_debugfs;
struct workqueue_struct *per_port_queue_reset_wq;
/* Workqueue for querying hardware stats */
struct delayed_work gf_stats_work;
@@ -501,6 +499,9 @@ struct mana_port_context {
u8 mac_addr[ETH_ALEN];
+ struct mana_eq *eqs;
+ struct dentry *mana_eqs_debugfs;
+
enum TRI_STATE rss_state;
mana_handle_t default_rxobj;
@@ -1034,6 +1035,8 @@ void mana_destroy_wq_obj(struct mana_port_context *apc, u32 wq_type,
int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id,
u32 doorbell_pg_id);
void mana_uncfg_vport(struct mana_port_context *apc);
+int mana_create_eq(struct mana_port_context *apc);
+void mana_destroy_eq(struct mana_port_context *apc);
struct net_device *mana_get_primary_netdev(struct mana_context *ac,
u32 port_index,
--
2.43.0
^ permalink raw reply related
* [PATCH net-next v9 0/6] net: mana: Per-vPort EQ and MSI-X interrupt management
From: Long Li @ 2026-05-13 22:09 UTC (permalink / raw)
To: Long Li, Konstantin Taranov, Jakub Kicinski, David S . Miller,
Paolo Abeni, Eric Dumazet, Andrew Lunn, Jason Gunthorpe,
Leon Romanovsky, Haiyang Zhang, K . Y . Srinivasan, Wei Liu,
Dexuan Cui, shradhagupta
Cc: Simon Horman, netdev, linux-rdma, linux-hyperv, linux-kernel
This series moves EQ ownership from the shared mana_context to per-vPort
mana_port_context, enabling each vPort to have dedicated MSI-X vectors
when the hardware provides enough vectors. When vectors are limited, the
driver falls back to sharing MSI-X among vPorts.
The series introduces a GDMA IRQ Context (GIC) abstraction with reference
counting to manage interrupt context lifecycle. This allows both Ethernet
and RDMA EQs to dynamically acquire dedicated or shared MSI-X vectors at
vPort creation time rather than pre-allocating all vectors at probe time.
This series touches both the net and RDMA MANA drivers and is intended
to go through the net-next tree. The patches are available on a shared
branch for both netdev and RDMA maintainers to review.
The following changes since commit 73d587ae684d176fac9db94173f77d78a794ea4f:
net: ethtool: fix missing closing paren in rings_reply_size() (2026-05-11 18:42:25 -0700)
are available in the Git repository at:
https://github.com/longlimsft/linux.git tags/mana-eq-msi-v9
for you to fetch changes up to 8249f52c3a065d92d24f27ab12c0b4d197ba14c4:
RDMA/mana_ib: Allocate interrupt contexts on EQs
Changes in v9:
- RSS QPs now take a vport reference via pd->vport_use_count to ensure
EQs outlive all QP consumers. EQs are only destroyed when the last
QP (raw or RSS) on the PD releases its reference (patch 1)
- Serialize mana_set_channels() against RDMA vport configuration via
apc->vport_mutex when the port is down. When the port is up, Ethernet
owns the vport exclusively so no locking is needed (patch 1)
- Change WARN_ON(apc->eqs) to bail out with -EEXIST to prevent
leaking prior EQ array if invariant is violated (patch 1)
- Only commit pd->tx_shortform_allowed and pd->tx_vp_offset after
mana_create_eq() succeeds (patch 1)
- Reset gc->msi_sharing at the top of mana_gd_query_max_resources()
so it is recomputed from current hardware state on resume (patch 2)
- Fix reverse Christmas tree variable declaration ordering (patches
1, 3, 5)
Changes in v8:
- Fix comment to reference per-vPort queue count instead of
gc->max_num_queues (patch 2)
- Remove duplicate irq_update_affinity_hint() calls from error paths
and mana_gd_remove_irqs(); the clearing is now centralized in
mana_gd_put_gic() (patch 4)
- Note the IRQ name change (mana_q -> mana_msi) in the commit
message (patch 4)
- Remove dead conditional write to spec.eq.msix_index (patch 5)
- Document GIC ownership contract and msix_index invariant change
in commit message (patch 5)
- Populate eq.irq on RDMA EQs for consistency with the Ethernet
path (patch 6)
- Document BIT(6) relocation and capability flag semantics in
commit message (patch 6)
- Fix checkpatch --strict alignment and line length warnings
Changes in v7:
- Use rounddown_pow_of_two() instead of roundup_pow_of_two() when
computing per-vPort queue count to avoid unnecessarily forcing shared
MSI-X mode (patch 2)
- Call mana_gd_setup_remaining_irqs() unconditionally to ensure
irq_contexts are populated in both dedicated and shared MSI-X modes,
fixing bisectability between patches 2 and 5 (patch 2)
- Guard ibdev_dbg() in mana_ib_cfg_vport() with error check so the
vport handle is not logged on the failure path (patch 1)
- Use cached gic->irq instead of pci_irq_vector() lookup in
mana_gd_put_gic() for consistency with the allocation path (patch 3)
- Fix unsigned int* to int* pointer type mismatch when calling
mana_gd_get_gic() by using a local int variable for the MSI index
(patches 5, 6)
Changes in v6:
- Rebased on net-next/main (v7.1-rc1)
Changes in v5:
- Rebased on net-next/main
Changes in v4:
- Rebased on net-next/main 7.0-rc4
- Patch 2: Use MANA_DEF_NUM_QUEUES instead of hardcoded 16 for
max_num_queues clamping
- Patch 3: Track dyn_msix in GIC context instead of re-checking
pci_msix_can_alloc_dyn() on each call; improved remove_irqs iteration
to skip unallocated entries
Changes in v3:
- Rebased on net-next/main
- Patch 1: Added NULL check for mpc->eqs in mana_ib_create_qp_rss() to
prevent NULL pointer dereference when RSS QP is created before a raw QP
has configured the vport and allocated EQs
Changes in v2:
- Rebased on net-next/main (adapted to kzalloc_objs/kzalloc_obj macros,
new GDMA_DRV_CAP_FLAG definitions)
- Patch 2: Fixed misleading comment for max_num_queues vs
max_num_queues_vport in gdma.h
- Patch 3: Fixed spelling typo in gdma_main.c ("difference" -> "different")
Long Li (6):
net: mana: Create separate EQs for each vPort
net: mana: Query device capabilities and configure MSI-X sharing for
EQs
net: mana: Introduce GIC context with refcounting for interrupt
management
net: mana: Use GIC functions to allocate global EQs
net: mana: Allocate interrupt context for each EQ when creating vPort
RDMA/mana_ib: Allocate interrupt contexts on EQs
drivers/infiniband/hw/mana/main.c | 67 +++-
drivers/infiniband/hw/mana/qp.c | 37 +-
.../net/ethernet/microsoft/mana/gdma_main.c | 323 +++++++++++++-----
drivers/net/ethernet/microsoft/mana/mana_en.c | 170 +++++----
.../ethernet/microsoft/mana/mana_ethtool.c | 27 +-
include/net/mana/gdma.h | 33 +-
include/net/mana/mana.h | 7 +-
7 files changed, 488 insertions(+), 176 deletions(-)
--
2.43.0
^ permalink raw reply
* Re: [PATCH v3 10/10] RDMA: Replace memset with = {} pattern for ib_respond_udata()
From: sashiko-bot @ 2026-05-13 20:59 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: linux-hyperv
In-Reply-To: <10-v3-4effdebad75a+e1-rdma_udata_rep_jgg@nvidia.com>
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Removal of `memset` in `qedr_copy_qp_uresp` introduces a kernel stack information leak to user-space due to uninitialized struct padding bytes.
--
commit 5d1e218c098790b95c443563dc2cc0974927c81a
Author: Jason Gunthorpe <jgg@ziepe.ca>
RDMA: Replace memset with = {} pattern for ib_respond_udata()
This patch replaces multiple instances of memset() used for zeroing
response structures with the aggregate initialization pattern = {}. It
also removes a redundant memset() in qedr_copy_qp_uresp().
> diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
> index 79190c5b8b50b..1af908275ca72 100644
> --- a/drivers/infiniband/hw/qedr/verbs.c
> +++ b/drivers/infiniband/hw/qedr/verbs.c
[ ... ]
> @@ -1283,8 +1281,6 @@ static int qedr_copy_qp_uresp(struct qedr_dev *dev,
> struct qedr_qp *qp, struct ib_udata *udata,
> struct qedr_create_qp_uresp *uresp)
> {
> - memset(uresp, 0, sizeof(*uresp));
> -
> if (qedr_qp_has_sq(qp))
> qedr_copy_sq_uresp(dev, uresp, qp);
Does this change expose kernel stack memory to userspace?
The commit message notes that qedr_copy_qp_uresp() is already called with
zeroed memory. If the caller is using aggregate initialization (= {}) on
the stack, this may not completely zero the memory.
Unlike memset(), the C standard does not guarantee that = {} will zero out
implicit padding bytes. The compiler might optimize the initialization by
only zeroing the named fields.
Looking at struct qedr_create_qp_uresp, there appear to be 4 bytes of
implicit padding due to field alignment:
include/uapi/rdma/qedr-abi.h:struct qedr_create_qp_uresp {
__u32 qp_id;
__u32 atomic_supported;
/* SQ */
__u32 sq_db_offset;
__u16 sq_icid;
<--- 2 bytes of padding here to align rq_db_offset
/* RQ */
__u32 rq_db_offset;
__u16 rq_icid;
<--- 2 bytes of padding here to align rq_db2_offset
__u32 rq_db2_offset;
__u32 reserved;
...
Since ib_respond_udata() copies sizeof(*uresp) to userspace, could the
removal of memset() cause residual stack data in these padding holes to be
leaked?
--
Sashiko AI review · https://sashiko.dev/#/patchset/0-v3-4effdebad75a+e1-rdma_udata_rep_jgg@nvidia.com?part=10
^ permalink raw reply
* Re: [PATCH V3 11/11] mshv: Mark mem regions as non-movable upfront if device passthru
From: sashiko-bot @ 2026-05-13 20:00 UTC (permalink / raw)
To: Mukesh R; +Cc: linux-pci, linux-hyperv
In-Reply-To: <20260512020259.1678627-12-mrathor@linux.microsoft.com>
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] Failing `copy_to_user` during device creation bypasses the `pt_regions_pinned` enforcement while leaving the device fully accessible.
- [High] Device creation does not validate or pin pre-existing memory regions, allowing an unsafe mix of movable regions and device passthrough.
- [High] Missing synchronization between device creation and region creation allows concurrent bypass of the pinning constraint.
--
commit add8cd40c15400cf2b19c8993ce14d4b4a7adef6
Author: Mukesh R <mrathor@linux.microsoft.com>
mshv: Mark mem regions as non-movable upfront if device passthru
This commit ensures that if a VM is started with a device attached, its
memory regions are marked as non-movable upfront to allow the use of SLAT
for IOMMU. It also adds a boot parameter to disable movable pages globally.
> diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
> index a7864463961bc..ac71534733bd8 100644
> --- a/drivers/hv/mshv_root_main.c
> +++ b/drivers/hv/mshv_root_main.c
[ ... ]
> @@ -1303,6 +1307,12 @@ static void mshv_async_hvcall_handler(void *data, u64 *status)
> *status = partition->async_hypercall_status;
> }
>
> +static bool mshv_do_pt_regions_pinned(struct mshv_partition *pt)
> +{
> + return pt->pt_regions_pinned || mshv_partition_encrypted(pt) ||
> + hv_no_movbl_pgs;
> +}
Is there a potential race condition here between device creation and region
creation?
pt->pt_regions_pinned is written in mshv_partition_ioctl_create_device()
without holding any locks. Concurrently, mshv_partition_create_region()
reads this flag via mshv_do_pt_regions_pinned() without synchronization.
If a region is created while a concurrent device creation thread is about to
set pt_regions_pinned to true, could the region incorrectly be instantiated
as movable while a passthrough device is active?
[ ... ]
> @@ -1808,6 +1818,9 @@ static long mshv_partition_ioctl_create_device(struct mshv_partition *partition,
> if (copy_to_user(uarg, &devargk, sizeof(devargk)))
> return -EFAULT; /* cleanup in mshv_device_fop_release() */
>
> + /* For now, all regions must be pinned if there is device passthru. */
> + partition->pt_regions_pinned = true;
If the copy_to_user() fails, the function returns early and skips setting
pt_regions_pinned to true. However, anon_inode_getfd() has already installed
the file descriptor in the process's file descriptor table, meaning the
device remains fully accessible. Can this allow userspace to bypass the
pt_regions_pinned enforcement?
Additionally, when a device is attached, this sets pt_regions_pinned to true
for future memory region creations. Does this handle the case where a
partition already has existing memory regions?
If userspace creates movable regions first and then attaches a device, is
there a risk that page migration could cause the device to DMA into incorrect
physical memory?
> return 0;
>
> undo_out:
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260512020259.1678627-1-mrathor@linux.microsoft.com?part=11
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox