* RE: [PATCH hyperv-next v3 00/15] Confidential VMBus
From: Michael Kelley @ 2025-06-18 16:13 UTC (permalink / raw)
To: Roman Kisel, alok.a.tiwari@oracle.com, arnd@arndb.de,
bp@alien8.de, corbet@lwn.net, dave.hansen@linux.intel.com,
decui@microsoft.com, haiyangz@microsoft.com, hpa@zytor.com,
kys@microsoft.com, mingo@redhat.com, tglx@linutronix.de,
wei.liu@kernel.org, linux-arch@vger.kernel.org,
linux-doc@vger.kernel.org, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, x86@kernel.org
Cc: apais@microsoft.com, benhill@microsoft.com,
bperkins@microsoft.com, sunilmut@microsoft.com
In-Reply-To: <20250604004341.7194-1-romank@linux.microsoft.com>
From: Roman Kisel <romank@linux.microsoft.com> Sent: Tuesday, June 3, 2025 5:43 PM
>
> The guests running on Hyper-V can be confidential where the memory and the
> register content are encrypted, provided that the hardware supports that
> (currently support AMD SEV-SNP and Intel TDX is implemented) and the guest
> is capable of using these features. The confidential guests cannot be
> introspected by the host nor the hypervisor without the guest sharing the
> memory contents upon doing which the memory is decrypted.
>
> In the confidential guests, neither the host nor the hypervisor need to be
> trusted, and the guests processing sensitive data can take advantage of that.
>
> Not trusting the host and the hypervisor (removing them from the Trusted
> Computing Base aka TCB) ncessitates that the method of communication
> between the host and the guest be changed. Below there is the breakdown of
> the options used in the both cases (in the diagrams below the server is
> marked as S, the client is marked as C):
>
> 1. Without the paravisoor the devices are connected to the host, and the
> host provides the device emulation or translation to the guest:
>
> +---- GUEST ----+ +----- DEVICE ----+ +----- HOST -----+
> | | | | | |
> | | | | | |
> | | | ========== |
> | | | | | |
> | | | | | |
> | | | | | |
> +----- C -------+ +-----------------+ +------- S ------+
> || ||
> || ||
> +------||------------------ VMBus --------------------------||------+
> | Interrupts, MMIO |
> +-------------------------------------------------------------------+
>
> 2. With the paravisor, the devices are connected to the paravisor, and
> the paravisor provides the device emulation or translation to the guest.
> The guest doesn't communicate with the host directly, and the guest
> communicates with the paravisor via the VMBus. The host is not trusted
> in this model, and the paravisor is trusted:
>
> +---- GUEST --------------- VTL0 ------+ +-- DEVICE --+
> | | | |
> | +- PARAVISOR --------- VTL2 -----+ | | |
> | | +-- VMBus Relay ------+ ====+================ |
> | | | Interrupts, MMIO | | | | |
> | | +-------- S ----------+ | | +------------+
> | | || | |
> | +---------+ || | |
> | | Linux | || OpenHCL | |
> | | kernel | || | |
> | +---- C --+-----||---------------+ |
> | || || |
> +-------++------- C -------------------+ +------------+
> || | HOST |
> || +---- S -----+
> +-------||----------------- VMBus ---------------------------||-----+
> | Interrupts, MMIO |
> +-------------------------------------------------------------------+
>
> Note that in the second case the guest doesn't need to share the memory
> with the host as it communicates only with the paravisor within their
> partition boundary. That is precisely the raison d'etre and the value
> proposition of this patch series: equip the confidential guest to use
> private (encrypted) memory and rely on the paravisor when this is
> available to be more secure.
I still have a fairly fundamental question about the value proposition.
The paravisor runs as part of the VM context and so is outside the host.
It should not trust the host any more than the guest does. For operations
that require the host, such as doing disk I/O, does the paravisor provide
any security benefit over the guest just talking directly to the host? Relying
on the paravisor is great, but doesn't that just push the problem from the
guest to the paravisor, which is no better equipped to solve it than the guest?
That's what I'm trying to get clear on.
>
> An implementation of the VMBus relay that offers the Confidential VMBus channels
> is available in the OpenVMM project as a part of the OpenHCL paravisor. Please
> refer to https://openvmm.dev/guide/ for more
> information about the OpenHCL paravisor.
>
> I'd like to thank the following people for their help with this
> patch series:
>
> * Dexuan for help with validation and the fruitful discussions,
> * Easwar for reviewing the refactoring of the page allocating and
> freeing in `hv.c`,
> * John and Sven for the design,
> * Mike for helping to avoid pitfalls when dealing with the GFP flags,
> * Sven for blazing the trail and implementing the design in few
> codebases.
>
> I made sure to validate the patch series on
>
> {TrustedLaunch(x86_64), OpenHCL} x
> {SNP(x86_64), TDX(x86_64), No hardware isolation, No paravisor} x
> {VMBus 5.0, VMBus 6.0} x
> {arm64, x86_64}.
>
> [V3]
> - The patch series is rebased on top of the latest hyperv-next branch.
> - Reworked the "wiring" diagram in the cover letter, added links to the
> OpenVMM project and the OpenHCL paravisor.
>
> - More precise wording in the comments and clearer code.
> **Thank you, Alok!**
>
> - Reworked the documentation patch.
> - Split the patchset into much more granular patches.
> - Various fixes and improvements throughout the patch series.
> **Thank you, Michael!**
I have fully reviewed v3 and am sending comments on most of the
individual patches. The split into more granular patches works really
well and was a big help in doing the review. I did not have comments
on a couple of the patches, but I have not given my Reviewed-by yet,
pending resolution of the other issues I've pointed out.
Michael
>
> [V2] https://lore.kernel.org/linux-hyperv/20250511230758.160674-1-romank@linux.microsoft.com/
> - The patch series is rebased on top of the latest hyperv-next branch.
>
> - Better wording in the commit messages and the Documentation.
> **Thank you, Alok and Wei!**
>
> - Removed the patches 5 and 6 concerning turning bounce buffering off from
> the previous version of the patch series as they were found to be
> architecturally unsound. The value proposition of the patch series is not
> diminished by this removal: these patches were an optimization and only for
> the storage (for the simplicity sake) but not for the network. These changes
> might be proposed in the future again after revolving the issues.
> ** Thanks you, Christoph, Dexuan, Dan, Michael, James, Robin! **
>
> [V1] https://lore.kernel.org/linux-hyperv/20250409000835.285105-1-romank@linux.microsoft.com/
>
> Roman Kisel (15):
> Documentation: hyperv: Confidential VMBus
> drivers: hv: VMBus protocol version 6.0
> arch: hyperv: Get/set SynIC synth.registers via paravisor
> arch/x86: mshyperv: Trap on access for some synthetic MSRs
> Drivers: hv: Rename fields for SynIC message and event pages
> Drivers: hv: Allocate the paravisor SynIC pages when required
> Drivers: hv: Post messages via the confidential VMBus if available
> Drivers: hv: remove stale comment
> Drivers: hv: Use memunmap() to check if the address is in IO map
> Drivers: hv: Rename the SynIC enable and disable routines
> Drivers: hv: Functions for setting up and tearing down the paravisor
> SynIC
> Drivers: hv: Allocate encrypted buffers when requested
> Drivers: hv: Support confidential VMBus channels
> Drivers: hv: Support establishing the confidential VMBus connection
> Drivers: hv: Set the default VMBus version to 6.0
>
> Documentation/virt/hyperv/coco.rst | 125 ++++++++-
> arch/x86/kernel/cpu/mshyperv.c | 67 ++++-
> drivers/hv/channel.c | 43 ++--
> drivers/hv/channel_mgmt.c | 27 +-
> drivers/hv/connection.c | 6 +-
> drivers/hv/hv.c | 399 ++++++++++++++++++++---------
> drivers/hv/hv_common.c | 13 +
> drivers/hv/hyperv_vmbus.h | 28 +-
> drivers/hv/mshv_root.h | 2 +-
> drivers/hv/mshv_synic.c | 6 +-
> drivers/hv/ring_buffer.c | 5 +-
> drivers/hv/vmbus_drv.c | 187 +++++++++-----
> include/asm-generic/mshyperv.h | 3 +
> include/linux/hyperv.h | 69 +++--
> 14 files changed, 740 insertions(+), 240 deletions(-)
>
>
> base-commit: 96959283a58d91ae20d025546f00e16f0a555208
> --
> 2.43.0
^ permalink raw reply
* Re: [PATCH] firmware: smccc: support both conduits for getting hyp UUID
From: Anirudh Rayabharam @ 2025-06-18 12:20 UTC (permalink / raw)
To: Sudeep Holla
Cc: Roman Kisel, linux-arm-kernel, linux-hyperv, linux-kernel,
lpieralisi, mark.rutland
In-Reply-To: <20250611-wandering-juicy-magpie-ed7f46@sudeepholla>
On Wed, Jun 11, 2025 at 01:52:32PM +0100, Sudeep Holla wrote:
> On Tue, Jun 10, 2025 at 09:06:48AM -0700, Roman Kisel wrote:
> > > (sorry for the delay, found the patch in the spam 🙁)
> >
> > "b4" shows the the mail server used for the patch submission
> > doesn't pass the DKIM check, so finding the patch in the spam seems
> > expected :) Thanks for your help!
> >
>
> Thought so looking at the header but I just have very basic knowledge
> there, so couldn't comment.
>
> > >
> > > On Wed, May 21, 2025 at 09:40:48AM +0000, Anirudh Rayabharam wrote:
> > >> From: Anirudh Rayabharam (Microsoft) <anirudh@anirudhrb.com>
> > >>
> > >> When Linux is running as the root partition under Microsoft Hypervisor
> > >> (MSHV) a.k.a Hyper-V, smc is used as the conduit for smc calls.
> > >>
> > >> Extend arm_smccc_hypervisor_has_uuid() to support this usecase. Use
> > >> arm_smccc_1_1_invoke to retrieve and use the appropriate conduit instead
> > >> of supporting only hvc.
> > >>
> > >> Boot tested on MSHV guest, MSHV root & KVM guest.
> > >>
> > >
> > > Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
> > >
> > > Are they any dependent patches or series using this ? Do you plan to
> > > route it via KVM tree if there are any dependency. Or else I can push
> > > it through (arm-)soc tree. Let me know.
> >
> > Anirudh had been OOF for some time and would be for another
> > week iiuc so I thought I'd reply.
> >
> > The patch this depends on is 13423063c7cb
> > ("arm64: kvm, smccc: Introduce and use API for getting hypervisor UUID"),
> > and this patch has already been pulled into the Linus'es tree.
> >
>
> Had a quick look at the commit to refresh my memory and as you mentioned
> it is new feature. I was checking to see if this is a fix.
>
> > As for routing, (arm-)soc should be good it appears as the change
> > is contained within the firmware drivers path. Although I'd trust more to your,
> > Arnd's or Wei's opinion than mine!
> >
>
> I will queue this once I start collecting patches for v6.17 in 1/2 weeks'
> time, so expect silence until then 😄.
Sounds good, thanks!
Regards,
Anirudh.
>
> --
> Regards,
> Sudeep
^ permalink raw reply
* Re: [PATCH] firmware: smccc: support both conduits for getting hyp UUID
From: Anirudh Rayabharam @ 2025-06-18 12:20 UTC (permalink / raw)
To: Roman Kisel
Cc: sudeep.holla, linux-arm-kernel, linux-hyperv, linux-kernel,
lpieralisi, mark.rutland
In-Reply-To: <20250610160656.11984-1-romank@linux.microsoft.com>
On Tue, Jun 10, 2025 at 09:06:48AM -0700, Roman Kisel wrote:
> > (sorry for the delay, found the patch in the spam 🙁)
>
> "b4" shows the the mail server used for the patch submission
> doesn't pass the DKIM check, so finding the patch in the spam seems
How do you check this? I mean, what b4 command do you run?
I think it should be fix now. Let's see...
> expected :) Thanks for your help!
>
> >
> > On Wed, May 21, 2025 at 09:40:48AM +0000, Anirudh Rayabharam wrote:
> >> From: Anirudh Rayabharam (Microsoft) <anirudh@anirudhrb.com>
> >>
> >> When Linux is running as the root partition under Microsoft Hypervisor
> >> (MSHV) a.k.a Hyper-V, smc is used as the conduit for smc calls.
> >>
> >> Extend arm_smccc_hypervisor_has_uuid() to support this usecase. Use
> >> arm_smccc_1_1_invoke to retrieve and use the appropriate conduit instead
> >> of supporting only hvc.
> >>
> >> Boot tested on MSHV guest, MSHV root & KVM guest.
> >>
> >
> > Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
> >
> > Are they any dependent patches or series using this ? Do you plan to
> > route it via KVM tree if there are any dependency. Or else I can push
> > it through (arm-)soc tree. Let me know.
>
> Anirudh had been OOF for some time and would be for another
> week iiuc so I thought I'd reply.
Thanks Roman!
Regards,
Anirudh
>
> The patch this depends on is 13423063c7cb
> ("arm64: kvm, smccc: Introduce and use API for getting hypervisor UUID"),
> and this patch has already been pulled into the Linus'es tree.
>
> As for routing, (arm-)soc should be good it appears as the change
> is contained within the firmware drivers path. Although I'd trust more to your,
> Arnd's or Wei's opinion than mine!
>
> >
> > --
> > Regards,
> > Sudeep
^ permalink raw reply
* [Patch net-next v3] net: mana: Record doorbell physical address in PF mode
From: longli @ 2025-06-18 1:36 UTC (permalink / raw)
To: K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Shradha Gupta, Simon Horman, Konstantin Taranov,
Souradeep Chakrabarti, Erick Archer, linux-hyperv, netdev,
linux-kernel, linux-rdma
Cc: Long Li
From: Long Li <longli@microsoft.com>
MANA supports RDMA in PF mode. The driver should record the doorbell
physical address when in PF mode.
The doorbell physical address is used by the RDMA driver to map
doorbell pages of the device to user-mode applications through RDMA
verbs interface. In the past, they have been mapped to user-mode while
the device is in VF mode. With the support for PF mode implemented,
also expose those pages in PF mode.
Support for PF mode is implemented in
290e5d3c49f6 ("net: mana: Add support for Multi Vports on Bare metal")
Signed-off-by: Long Li <longli@microsoft.com>
---
Changes
v2: add more details in commit message on how the doorbell physical address is used
v3: add the early commit detail where the support for PF mode is implemented
drivers/net/ethernet/microsoft/mana/gdma_main.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index 3504507477c6..52cf7112762c 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -31,6 +31,9 @@ static void mana_gd_init_pf_regs(struct pci_dev *pdev)
gc->db_page_base = gc->bar0_va +
mana_gd_r64(gc, GDMA_PF_REG_DB_PAGE_OFF);
+ gc->phys_db_page_base = gc->bar0_pa +
+ mana_gd_r64(gc, GDMA_PF_REG_DB_PAGE_OFF);
+
sriov_base_off = mana_gd_r64(gc, GDMA_SRIOV_REG_CFG_BASE_OFF);
sriov_base_va = gc->bar0_va + sriov_base_off;
--
2.25.1
^ permalink raw reply related
* RE: [EXTERNAL] Re: [Patch net-next v2] net: mana: Record doorbell physical address in PF mode
From: Long Li @ 2025-06-18 1:36 UTC (permalink / raw)
To: Jakub Kicinski, longli@linuxonhyperv.com
Cc: KY Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
David S. Miller, Eric Dumazet, Paolo Abeni, Shradha Gupta,
Simon Horman, Konstantin Taranov, Souradeep Chakrabarti,
Erick Archer, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-rdma@vger.kernel.org
In-Reply-To: <20250616152318.26e1ea3e@kernel.org>
> Subject: Re: [Patch net-next v2] net: mana: Record doorbell physical
> address in PF mode
>
> On Fri, 13 Jun 2025 10:46:05 -0700 longli@linuxonhyperv.com wrote:
> > MANA supports RDMA in PF mode. The driver should record the doorbell
> > physical address when in PF mode.
> >
> > The doorbell physical address is used by the RDMA driver to map
> > doorbell pages of the device to user-mode applications through RDMA
> > verbs interface. In the past, they have been mapped to user-mode while
> > the device is in VF mode. With the support for PF mode implemented,
> > also expose those pages in PF mode.
>
> It'd be good to indicate if the PF mode support is implemented as in ready to be
> posted to the list, or implemented as in already in the tree? If it's in tree then
> please quote the commit. If upcoming please rephrase.
>
> This commit msg is much better, thank you, but to a person handling backports it
> will still not be immediately clear whether this is prep work for a new code path,
> or a bug fix.
> --
> pw-bot: cr
Thank you. I'm sending v3.
Long
^ permalink raw reply
* Re: [patch V4 11/14] PCI/MSI: Provide a sane mechanism for TPH
From: Bjorn Helgaas @ 2025-06-17 23:25 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Marc Zyngier, Peter Zijlstra, Bjorn Helgaas, Wei Huang,
linux-pci, Jonathan Cameron, Nishanth Menon, Dhruva Gole,
Tero Kristo, Santosh Shilimkar, Logan Gunthorpe, Dave Jiang,
Jon Mason, Allen Hubbe, ntb, Michael Kelley, Wei Liu,
Haiyang Zhang, linux-hyperv, Manivannan Sadhasivam,
James E.J. Bottomley, Martin K. Petersen, linux-scsi,
Jonathan Cameron
In-Reply-To: <20250617232232.GA1175597@bhelgaas>
On Tue, Jun 17, 2025 at 06:22:34PM -0500, Bjorn Helgaas wrote:
> On Wed, Mar 19, 2025 at 11:56:57AM +0100, Thomas Gleixner wrote:
> > The PCI/TPH driver fiddles with the MSI-X control word of an active
> > interrupt completely unserialized against concurrent operations issued
> > from the interrupt core. It also brings the PCI/MSI-X internal cached
> > control word out of sync.
> >
> > Provide a function, which has the required serialization and keeps the
> > control word cache in sync.
> >
> > Unfortunately this requires to look up and lock the interrupt descriptor,
> > which should be only done in the interrupt core code. But confining this
> > particular oddity in the PCI/MSI core is the lesser of all evil. A
> > interrupt core implementation would require a larger pile of infrastructure
> > and indirections for dubious value.
> >
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Acked-by: Bjorn Helgaas <bhelgaas@google.com>
> > Cc: Wei Huang <wei.huang2@amd.com>
> > Cc: linux-pci@vger.kernel.org
> >
> >
> >
> > ---
> > drivers/pci/msi/msi.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
> > drivers/pci/pci.h | 9 +++++++++
> > 2 files changed, 56 insertions(+)
> >
> > --- a/drivers/pci/msi/msi.c
> > +++ b/drivers/pci/msi/msi.c
> > @@ -910,6 +910,53 @@ void pci_free_msi_irqs(struct pci_dev *d
> > }
> > }
> >
> > +#ifdef CONFIG_PCIE_TPH
> > +/**
> > + * pci_msix_write_tph_tag - Update the TPH tag for a given MSI-X vector
> > + * @pdev: The PCIe device to update
> > + * @index: The MSI-X index to update
> > + * @tag: The tag to write
> > + *
> > + * Returns: 0 on success, error code on failure
> > + */
> > +int pci_msix_write_tph_tag(struct pci_dev *pdev, unsigned int index, u16 tag)
> > +{
> > + struct msi_desc *msi_desc;
> > + struct irq_desc *irq_desc;
> > + unsigned int virq;
> > +
> > + if (!pdev->msix_enabled)
> > + return -ENXIO;
> > +
> > + guard(msi_descs_lock)(&pdev->dev);
> > + virq = msi_get_virq(&pdev->dev, index);
> > + if (!virq)
> > + return -ENXIO;
> > + /*
> > + * This is a horrible hack, but short of implementing a PCI
> > + * specific interrupt chip callback and a huge pile of
> > + * infrastructure, this is the minor nuissance. It provides the
> > + * protection against concurrent operations on this entry and keeps
> > + * the control word cache in sync.
> > + */
> > + irq_desc = irq_to_desc(virq);
> > + if (!irq_desc)
> > + return -ENXIO;
> > +
> > + guard(raw_spinlock_irq)(&irq_desc->lock);
> > + msi_desc = irq_data_get_msi_desc(&irq_desc->irq_data);
> > + if (!msi_desc || msi_desc->pci.msi_attrib.is_virtual)
> > + return -ENXIO;
> > +
> > + msi_desc->pci.msix_ctrl &= ~PCI_MSIX_ENTRY_CTRL_ST;
> > + msi_desc->pci.msix_ctrl |= FIELD_PREP(PCI_MSIX_ENTRY_CTRL_ST, tag);
> > + pci_msix_write_vector_ctrl(msi_desc, msi_desc->pci.msix_ctrl);
> > + /* Flush the write */
> > + readl(pci_msix_desc_addr(msi_desc));
> > + return 0;
> > +}
>
> Looks like this change might add this warning, which I don't claim to
> understand:
>
> $ make C=2 drivers/pci/msi/msi.o
> drivers/pci/msi/msi.c:928:5: warning: context imbalance in 'pci_msix_write_tph_tag' - wrong count at exit
>
> This appeared in v6.16-rc1 as d5124a9957b2 ("PCI/MSI: Provide a sane
> mechanism for TPH")
Sorry, I guess I just rediscovered this sparse deficiency:
https://lore.kernel.org/r/20250422174156.GA344533@bhelgaas
Never mind, I'll just try to remember to ignore these warnings.
^ permalink raw reply
* Re: [patch V4 11/14] PCI/MSI: Provide a sane mechanism for TPH
From: Bjorn Helgaas @ 2025-06-17 23:22 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Marc Zyngier, Peter Zijlstra, Bjorn Helgaas, Wei Huang,
linux-pci, Jonathan Cameron, Nishanth Menon, Dhruva Gole,
Tero Kristo, Santosh Shilimkar, Logan Gunthorpe, Dave Jiang,
Jon Mason, Allen Hubbe, ntb, Michael Kelley, Wei Liu,
Haiyang Zhang, linux-hyperv, Manivannan Sadhasivam,
James E.J. Bottomley, Martin K. Petersen, linux-scsi,
Jonathan Cameron
In-Reply-To: <20250319105506.683663807@linutronix.de>
On Wed, Mar 19, 2025 at 11:56:57AM +0100, Thomas Gleixner wrote:
> The PCI/TPH driver fiddles with the MSI-X control word of an active
> interrupt completely unserialized against concurrent operations issued
> from the interrupt core. It also brings the PCI/MSI-X internal cached
> control word out of sync.
>
> Provide a function, which has the required serialization and keeps the
> control word cache in sync.
>
> Unfortunately this requires to look up and lock the interrupt descriptor,
> which should be only done in the interrupt core code. But confining this
> particular oddity in the PCI/MSI core is the lesser of all evil. A
> interrupt core implementation would require a larger pile of infrastructure
> and indirections for dubious value.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Wei Huang <wei.huang2@amd.com>
> Cc: linux-pci@vger.kernel.org
>
>
>
> ---
> drivers/pci/msi/msi.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
> drivers/pci/pci.h | 9 +++++++++
> 2 files changed, 56 insertions(+)
>
> --- a/drivers/pci/msi/msi.c
> +++ b/drivers/pci/msi/msi.c
> @@ -910,6 +910,53 @@ void pci_free_msi_irqs(struct pci_dev *d
> }
> }
>
> +#ifdef CONFIG_PCIE_TPH
> +/**
> + * pci_msix_write_tph_tag - Update the TPH tag for a given MSI-X vector
> + * @pdev: The PCIe device to update
> + * @index: The MSI-X index to update
> + * @tag: The tag to write
> + *
> + * Returns: 0 on success, error code on failure
> + */
> +int pci_msix_write_tph_tag(struct pci_dev *pdev, unsigned int index, u16 tag)
> +{
> + struct msi_desc *msi_desc;
> + struct irq_desc *irq_desc;
> + unsigned int virq;
> +
> + if (!pdev->msix_enabled)
> + return -ENXIO;
> +
> + guard(msi_descs_lock)(&pdev->dev);
> + virq = msi_get_virq(&pdev->dev, index);
> + if (!virq)
> + return -ENXIO;
> + /*
> + * This is a horrible hack, but short of implementing a PCI
> + * specific interrupt chip callback and a huge pile of
> + * infrastructure, this is the minor nuissance. It provides the
> + * protection against concurrent operations on this entry and keeps
> + * the control word cache in sync.
> + */
> + irq_desc = irq_to_desc(virq);
> + if (!irq_desc)
> + return -ENXIO;
> +
> + guard(raw_spinlock_irq)(&irq_desc->lock);
> + msi_desc = irq_data_get_msi_desc(&irq_desc->irq_data);
> + if (!msi_desc || msi_desc->pci.msi_attrib.is_virtual)
> + return -ENXIO;
> +
> + msi_desc->pci.msix_ctrl &= ~PCI_MSIX_ENTRY_CTRL_ST;
> + msi_desc->pci.msix_ctrl |= FIELD_PREP(PCI_MSIX_ENTRY_CTRL_ST, tag);
> + pci_msix_write_vector_ctrl(msi_desc, msi_desc->pci.msix_ctrl);
> + /* Flush the write */
> + readl(pci_msix_desc_addr(msi_desc));
> + return 0;
> +}
Looks like this change might add this warning, which I don't claim to
understand:
$ make C=2 drivers/pci/msi/msi.o
drivers/pci/msi/msi.c:928:5: warning: context imbalance in 'pci_msix_write_tph_tag' - wrong count at exit
This appeared in v6.16-rc1 as d5124a9957b2 ("PCI/MSI: Provide a sane
mechanism for TPH")
^ permalink raw reply
* Re: [PATCH v6 0/5] Allow dyn MSI-X vector allocation of MANA
From: patchwork-bot+netdevbpf @ 2025-06-17 22:00 UTC (permalink / raw)
To: Shradha Gupta
Cc: linux-hyperv, linux-pci, linux-kernel, nipun.gupta, yury.norov,
jgg, Jonathan.Cameron, anna-maria, kevin.tian, longli, tglx,
bhelgaas, robh, manivannan.sadhasivam, kw, lpieralisi, decui,
wei.liu, haiyangz, kys, andrew+netdev, davem, edumazet, kuba,
pabeni, kotaranov, horms, leon, mlevitsk, ernis, peterz, netdev,
linux-rdma, paulros, shradhagupta
In-Reply-To: <1749650984-9193-1-git-send-email-shradhagupta@linux.microsoft.com>
Hello:
This series was applied to netdev/net-next.git (main)
by Shradha Gupta <shradhagupta@linux.microsoft.com>:
On Wed, 11 Jun 2025 07:09:44 -0700 you wrote:
> In this patchset we want to enable the MANA driver to be able to
> allocate MSI-X vectors in PCI dynamically.
>
> The first patch exports pci_msix_prepare_desc() in PCI to be able to
> correctly prepare descriptors for dynamically added MSI-X vectors.
>
> The second patch adds the support of dynamic vector allocation in
> pci-hyperv PCI controller by enabling the MSI_FLAG_PCI_MSIX_ALLOC_DYN
> flag and using the pci_msix_prepare_desc() exported in first patch.
>
> [...]
Here is the summary with links:
- [v6,1/5] PCI/MSI: Export pci_msix_prepare_desc() for dynamic MSI-X allocations
https://git.kernel.org/netdev/net-next/c/5da8a8b8090b
- [v6,2/5] PCI: hv: Allow dynamic MSI-X vector allocation
https://git.kernel.org/netdev/net-next/c/ad518f2557b9
- [v6,3/5] net: mana: explain irq_setup() algorithm
https://git.kernel.org/netdev/net-next/c/4607617af1b4
- [v6,4/5] net: mana: Allow irq_setup() to skip cpus for affinity
https://git.kernel.org/netdev/net-next/c/845c62c543d6
- [v6,5/5] net: mana: Allocate MSI-X vectors dynamically
https://git.kernel.org/netdev/net-next/c/755391121038
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply
* RE: [EXTERNAL] [PATCH net-next v3 3/4] net: mana: Add speed support in mana_get_link_ksettings
From: Long Li @ 2025-06-17 15:54 UTC (permalink / raw)
To: Erni Sri Satya Vennela, KY Srinivasan, Haiyang Zhang,
wei.liu@kernel.org, Dexuan Cui, andrew+netdev@lunn.ch,
davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
pabeni@redhat.com, Konstantin Taranov, horms@kernel.org,
Shiraz Saleem, leon@kernel.org, shradhagupta@linux.microsoft.com,
schakrabarti@linux.microsoft.com, gerhard@engleder-embedded.com,
rosenp@gmail.com, sdf@fomichev.me, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-rdma@vger.kernel.org
In-Reply-To: <1750144656-2021-4-git-send-email-ernis@linux.microsoft.com>
> Subject: [PATCH net-next v3 3/4] net: mana: Add speed support in
> mana_get_link_ksettings
>
> Allow mana ethtool get_link_ksettings operation to report the maximum speed
> supported by the SKU in mbps.
>
> The driver retrieves this information by issuing a HWC command to the hardware
> via mana_query_link_cfg(), which retrieves the SKU's maximum supported speed.
>
> These APIs when invoked on hardware that are older/do not support these APIs,
> the speed would be reported as UNKNOWN.
>
> Before:
> $ethtool enP30832s1
> > Settings for enP30832s1:
> Supported ports: [ ]
> Supported link modes: Not reported
> Supported pause frame use: No
> Supports auto-negotiation: No
> Supported FEC modes: Not reported
> Advertised link modes: Not reported
> Advertised pause frame use: No
> Advertised auto-negotiation: No
> Advertised FEC modes: Not reported
> Speed: Unknown!
> Duplex: Full
> Auto-negotiation: off
> Port: Other
> PHYAD: 0
> Transceiver: internal
> Link detected: yes
>
> After:
> $ethtool enP30832s1
> > Settings for enP30832s1:
> Supported ports: [ ]
> Supported link modes: Not reported
> Supported pause frame use: No
> Supports auto-negotiation: No
> Supported FEC modes: Not reported
> Advertised link modes: Not reported
> Advertised pause frame use: No
> Advertised auto-negotiation: No
> Advertised FEC modes: Not reported
> Speed: 16000Mb/s
> Duplex: Full
> Auto-negotiation: off
> Port: Other
> PHYAD: 0
> Transceiver: internal
> Link detected: yes
>
> Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
> Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
> Reviewed-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
> Reviewed-by: Saurabh Singh Sengar <ssengar@linux.microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>
> ---
> Changes in v3:
> * Rebase to latest net-next branch.
> Changes in v2:
> * No change.
> ---
> drivers/net/ethernet/microsoft/mana/mana_en.c | 1 +
> drivers/net/ethernet/microsoft/mana/mana_ethtool.c | 6 ++++++
> include/net/mana/mana.h | 2 ++
> 3 files changed, 9 insertions(+)
>
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c
> b/drivers/net/ethernet/microsoft/mana/mana_en.c
> index 547dff450b6d..d7079e05dfb8 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_en.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
> @@ -1272,6 +1272,7 @@ int mana_query_link_cfg(struct mana_port_context
> *apc)
> return err;
> }
> apc->speed = resp.link_speed_mbps;
> + apc->max_speed = resp.qos_speed_mbps;
> return 0;
> }
>
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> index 4fb3a04994a2..a1afa75a9463 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> @@ -495,6 +495,12 @@ static int mana_set_ringparam(struct net_device *ndev,
> static int mana_get_link_ksettings(struct net_device *ndev,
> struct ethtool_link_ksettings *cmd) {
> + struct mana_port_context *apc = netdev_priv(ndev);
> + int err;
> +
> + err = mana_query_link_cfg(apc);
> + cmd->base.speed = (err) ? SPEED_UNKNOWN : apc->max_speed;
> +
> cmd->base.duplex = DUPLEX_FULL;
> cmd->base.port = PORT_OTHER;
>
> diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h index
> 038b18340e51..e1030a7d2daa 100644
> --- a/include/net/mana/mana.h
> +++ b/include/net/mana/mana.h
> @@ -533,6 +533,8 @@ struct mana_port_context {
> u16 port_idx;
> /* Currently configured speed (mbps) */
> u32 speed;
> + /* Maximum speed supported by the SKU (mbps) */
> + u32 max_speed;
>
> bool port_is_up;
> bool port_st_save; /* Saved port state */
> --
> 2.34.1
^ permalink raw reply
* RE: [PATCH net-next] net: mana: Set tx_packets to post gso processing packet count
From: Haiyang Zhang @ 2025-06-17 15:31 UTC (permalink / raw)
To: Shradha Gupta, Dexuan Cui, Wei Liu, KY Srinivasan, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Konstantin Taranov, Souradeep Chakrabarti, Erni Sri Satya Vennela,
Long Li, Dipayaan Roy, Shiraz Saleem
Cc: netdev@vger.kernel.org, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, Paul Rosswurm, Shradha Gupta
In-Reply-To: <1750160021-24589-1-git-send-email-shradhagupta@linux.microsoft.com>
> -----Original Message-----
> From: Shradha Gupta <shradhagupta@linux.microsoft.com>
> Sent: Tuesday, June 17, 2025 7:34 AM
> To: Dexuan Cui <decui@microsoft.com>; Wei Liu <wei.liu@kernel.org>;
> Haiyang Zhang <haiyangz@microsoft.com>; KY Srinivasan <kys@microsoft.com>;
> Andrew Lunn <andrew+netdev@lunn.ch>; David S. Miller
> <davem@davemloft.net>; Eric Dumazet <edumazet@google.com>; Jakub Kicinski
> <kuba@kernel.org>; Paolo Abeni <pabeni@redhat.com>; Konstantin Taranov
> <kotaranov@microsoft.com>; Souradeep Chakrabarti
> <schakrabarti@linux.microsoft.com>; Erni Sri Satya Vennela
> <ernis@linux.microsoft.com>; Long Li <longli@microsoft.com>; Dipayaan Roy
> <dipayanroy@linux.microsoft.com>; Shiraz Saleem
> <shirazsaleem@microsoft.com>
> Cc: Shradha Gupta <shradhagupta@linux.microsoft.com>;
> netdev@vger.kernel.org; linux-hyperv@vger.kernel.org; linux-
> kernel@vger.kernel.org; Paul Rosswurm <paulros@microsoft.com>; Shradha
> Gupta <shradhagupta@microsoft.com>
> Subject: [PATCH net-next] net: mana: Set tx_packets to post gso processing
> packet count
>
> Allow tx_packets and tx_bytes counter in the driver to represent
> the packets transmitted post GSO processing.
>
> Currently they are populated as bigger pre-GSO packets and bytes
>
> Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
^ permalink raw reply
* Re: [PATCH net-next v3 1/3] vsock: Add support for SIOCINQ ioctl
From: Stefano Garzarella @ 2025-06-17 14:39 UTC (permalink / raw)
To: Xuewei Niu, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
linux-hyperv
Cc: mst, pabeni, jasowang, xuanzhuo, davem, netdev, stefanha,
leonardi, virtualization, kvm, linux-kernel, fupan.lfp,
Xuewei Niu
In-Reply-To: <20250617045347.1233128-2-niuxuewei.nxw@antgroup.com>
CCin hyper-v maintainers and list since I have a question about hyperv
transport.
On Tue, Jun 17, 2025 at 12:53:44PM +0800, Xuewei Niu wrote:
>Add support for SIOCINQ ioctl, indicating the length of bytes unread in the
>socket. The value is obtained from `vsock_stream_has_data()`.
>
>Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
>---
> net/vmw_vsock/af_vsock.c | 22 ++++++++++++++++++++++
> 1 file changed, 22 insertions(+)
>
>diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c
>index 2e7a3034e965..bae6b89bb5fb 100644
>--- a/net/vmw_vsock/af_vsock.c
>+++ b/net/vmw_vsock/af_vsock.c
>@@ -1389,6 +1389,28 @@ static int vsock_do_ioctl(struct socket *sock, unsigned int cmd,
> vsk = vsock_sk(sk);
>
> switch (cmd) {
>+ case SIOCINQ: {
>+ ssize_t n_bytes;
>+
>+ if (!vsk->transport) {
>+ ret = -EOPNOTSUPP;
>+ break;
>+ }
>+
>+ if (sock_type_connectible(sk->sk_type) &&
>+ sk->sk_state == TCP_LISTEN) {
>+ ret = -EINVAL;
>+ break;
>+ }
>+
>+ n_bytes = vsock_stream_has_data(vsk);
Now looks better to me, I just checked transports: vmci and virtio/vhost
returns what we want, but for hyperv we have:
static s64 hvs_stream_has_data(struct vsock_sock *vsk)
{
struct hvsock *hvs = vsk->trans;
s64 ret;
if (hvs->recv_data_len > 0)
return 1;
@Hyper-v maintainers: do you know why we don't return `recv_data_len`?
Do you think we can do that to support this new feature?
Thanks,
Stefano
>+ if (n_bytes < 0) {
>+ ret = n_bytes;
>+ break;
>+ }
>+ ret = put_user(n_bytes, arg);
>+ break;
>+ }
> case SIOCOUTQ: {
> ssize_t n_bytes;
>
>--
>2.34.1
>
^ permalink raw reply
* [PATCH net-next] net: mana: Set tx_packets to post gso processing packet count
From: Shradha Gupta @ 2025-06-17 11:33 UTC (permalink / raw)
To: Dexuan Cui, Wei Liu, Haiyang Zhang, K. Y. Srinivasan, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Konstantin Taranov, Souradeep Chakrabarti, Erni Sri Satya Vennela,
Long Li, Dipayaan Roy, Shiraz Saleem
Cc: Shradha Gupta, netdev, linux-hyperv, linux-kernel, Paul Rosswurm,
Shradha Gupta
Allow tx_packets and tx_bytes counter in the driver to represent
the packets transmitted post GSO processing.
Currently they are populated as bigger pre-GSO packets and bytes
Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
---
drivers/net/ethernet/microsoft/mana/mana_en.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index e68b8190bb7a..c3b6478cb1d3 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -251,10 +251,10 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
struct netdev_queue *net_txq;
struct mana_stats_tx *tx_stats;
struct gdma_queue *gdma_sq;
+ int err, len, num_gso_seg;
unsigned int csum_type;
struct mana_txq *txq;
struct mana_cq *cq;
- int err, len;
if (unlikely(!apc->port_is_up))
goto tx_drop;
@@ -407,6 +407,7 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
skb_queue_tail(&txq->pending_skbs, skb);
len = skb->len;
+ num_gso_seg = skb_is_gso(skb) ? skb_shinfo(skb)->gso_segs : 1;
net_txq = netdev_get_tx_queue(ndev, txq_idx);
err = mana_gd_post_work_request(gdma_sq, &pkg.wqe_req,
@@ -431,10 +432,13 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
/* skb may be freed after mana_gd_post_work_request. Do not use it. */
skb = NULL;
+ /* Populated the packet and bytes counters based on post GSO packet
+ * calculations
+ */
tx_stats = &txq->stats;
u64_stats_update_begin(&tx_stats->syncp);
- tx_stats->packets++;
- tx_stats->bytes += len;
+ tx_stats->packets += num_gso_seg;
+ tx_stats->bytes += len + ((num_gso_seg - 1) * gso_hs);
u64_stats_update_end(&tx_stats->syncp);
tx_busy:
base-commit: 8909f5f4ecd551c2299b28e05254b77424c8c7dc
--
2.34.1
^ permalink raw reply related
* Re: [PATCH v6 0/5] Allow dyn MSI-X vector allocation of MANA
From: Shradha Gupta @ 2025-06-17 10:53 UTC (permalink / raw)
To: Jakub Kicinski
Cc: linux-hyperv, linux-pci, linux-kernel, Nipun Gupta, Yury Norov,
Jason Gunthorpe, Jonathan Cameron, Anna-Maria Behnsen, Kevin Tian,
Long Li, Thomas Gleixner, Bjorn Helgaas, Rob Herring,
Manivannan Sadhasivam, Krzysztof Wilczy???~Dski,
Lorenzo Pieralisi, Dexuan Cui, Wei Liu, Haiyang Zhang,
K. Y. Srinivasan, Andrew Lunn, David S. Miller, Eric Dumazet,
Paolo Abeni, Konstantin Taranov, Simon Horman, Leon Romanovsky,
Maxim Levitsky, Erni Sri Satya Vennela, Peter Zijlstra, netdev,
linux-rdma, Paul Rosswurm, Shradha Gupta
In-Reply-To: <20250616181148.2aed5dfe@kernel.org>
On Mon, Jun 16, 2025 at 06:11:48PM -0700, Jakub Kicinski wrote:
> On Wed, 11 Jun 2025 23:10:55 -0700 Shradha Gupta wrote:
> > On Wed, Jun 11, 2025 at 08:54:16AM -0700, Jakub Kicinski wrote:
> > > On Wed, 11 Jun 2025 07:09:44 -0700 Shradha Gupta wrote:
> > > > Changes in v6
> > > > * rebased to linux-next's v6.16-rc1 as per Jakub's suggestion
> > >
> > > I meant a branch, basically apply the patches on the v6.16-rc1 tag
> > > and push it out to GitHub, kernel.org or somewhere else public.
> > > Then we can pull it in and maintain the stable commit IDs.
> > > No need to repost the patches, FWIW, just share the branch here
> > > once you pushed it out..
> >
> > Oh, understood. Thanks for the clarity. Here is a github repo branch
> > with the changes on v6.16-rc1 tag
> > https://github.com/shradhagupta6/linux/tree/shradha_v6.16-rc1
>
> The tag was good, but when I pulled it my check scripts complained:
>
> Commit a19036b86845 ("net: mana: Allocate MSI-X vectors dynamically")
> committer Signed-off-by missing
> author email: shradhagupta@linux.microsoft.com
> committer email: shradhagupta@microsoft.com
> Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
>
> etc. You seem to have committed the patches with a slightly different
> email address. Not a huge deal but better to fix it if we can.
>
> So please base the tag. The code can stay the same just adjust the
> committer or author/signoff email addrs. We can use this as an
> opportunity to add Bjorn's email.
>
> No need to repost the code just ping here once you updated the tag.
Hi Jakub,
I have updated the tag with the corrected author and committer details
and added Bjorn's ack:
https://github.com/shradhagupta6/linux/tree/shradha_v6.16-rc1
By 'please base the tag', did you mean we rebase the changes with rc2?
If so, I have also created a rc2 tag branch, JFYI
https://github.com/shradhagupta6/linux/tree/shradha_v6.16-rc2
Hope this helps.
Thanks,
Shradha.
^ permalink raw reply
* [PATCH net-next v3 4/4] net: mana: Handle unsupported HWC commands
From: Erni Sri Satya Vennela @ 2025-06-17 7:17 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, andrew+netdev, davem, edumazet,
kuba, pabeni, longli, kotaranov, horms, shirazsaleem, leon, ernis,
shradhagupta, schakrabarti, gerhard, rosenp, sdf, linux-hyperv,
netdev, linux-kernel, linux-rdma
In-Reply-To: <1750144656-2021-1-git-send-email-ernis@linux.microsoft.com>
If any of the HWC commands are not recognized by the
underlying hardware, the hardware returns the response
header status of -1. Log the information using
netdev_info_once to avoid multiple error logs in dmesg.
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
Reviewed-by: Saurabh Singh Sengar <ssengar@linux.microsoft.com>
Reviewed-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
---
Changes in v3:
* Rebase to latest net-next branch.
Changes in v2:
* Define GDMA_STATUS_CMD_UNSUPPORTED for unsupported HWC status code
instead of using -1.
---
drivers/net/ethernet/microsoft/mana/hw_channel.c | 4 ++++
drivers/net/ethernet/microsoft/mana/mana_en.c | 11 +++++++++++
include/net/mana/gdma.h | 1 +
3 files changed, 16 insertions(+)
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index 3d3677c0d014..650d22654d49 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -891,6 +891,10 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len,
}
if (ctx->status_code && ctx->status_code != GDMA_STATUS_MORE_ENTRIES) {
+ if (ctx->status_code == GDMA_STATUS_CMD_UNSUPPORTED) {
+ err = -EOPNOTSUPP;
+ goto out;
+ }
if (req_msg->req.msg_type != MANA_QUERY_PHY_STAT)
dev_err(hwc->dev, "HWC: Failed hw_channel req: 0x%x\n",
ctx->status_code);
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index d7079e05dfb8..5aee7bda1504 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -847,6 +847,9 @@ static int mana_send_request(struct mana_context *ac, void *in_buf,
err = mana_gd_send_request(gc, in_len, in_buf, out_len,
out_buf);
if (err || resp->status) {
+ if (err == -EOPNOTSUPP)
+ return err;
+
if (req->req.msg_type != MANA_QUERY_PHY_STAT)
dev_err(dev, "Failed to send mana message: %d, 0x%x\n",
err, resp->status);
@@ -1252,6 +1255,10 @@ int mana_query_link_cfg(struct mana_port_context *apc)
sizeof(resp));
if (err) {
+ if (err == -EOPNOTSUPP) {
+ netdev_info_once(ndev, "MANA_QUERY_LINK_CONFIG not supported\n");
+ return err;
+ }
netdev_err(ndev, "Failed to query link config: %d\n", err);
return err;
}
@@ -1294,6 +1301,10 @@ int mana_set_bw_clamp(struct mana_port_context *apc, u32 speed,
sizeof(resp));
if (err) {
+ if (err == -EOPNOTSUPP) {
+ netdev_info_once(ndev, "MANA_SET_BW_CLAMP not supported\n");
+ return err;
+ }
netdev_err(ndev, "Failed to set bandwidth clamp for speed %u, err = %d",
speed, err);
return err;
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index bfae59202669..fdf144988cec 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -10,6 +10,7 @@
#include "shm_channel.h"
#define GDMA_STATUS_MORE_ENTRIES 0x00000105
+#define GDMA_STATUS_CMD_UNSUPPORTED 0xffffffff
/* Structures labeled with "HW DATA" are exchanged with the hardware. All of
* them are naturally aligned and hence don't need __packed.
--
2.34.1
^ permalink raw reply related
* [PATCH net-next v3 3/4] net: mana: Add speed support in mana_get_link_ksettings
From: Erni Sri Satya Vennela @ 2025-06-17 7:17 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, andrew+netdev, davem, edumazet,
kuba, pabeni, longli, kotaranov, horms, shirazsaleem, leon, ernis,
shradhagupta, schakrabarti, gerhard, rosenp, sdf, linux-hyperv,
netdev, linux-kernel, linux-rdma
In-Reply-To: <1750144656-2021-1-git-send-email-ernis@linux.microsoft.com>
Allow mana ethtool get_link_ksettings operation to report
the maximum speed supported by the SKU in mbps.
The driver retrieves this information by issuing a
HWC command to the hardware via mana_query_link_cfg(),
which retrieves the SKU's maximum supported speed.
These APIs when invoked on hardware that are older/do
not support these APIs, the speed would be reported as UNKNOWN.
Before:
$ethtool enP30832s1
> Settings for enP30832s1:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
Link detected: yes
After:
$ethtool enP30832s1
> Settings for enP30832s1:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 16000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
Link detected: yes
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
Reviewed-by: Saurabh Singh Sengar <ssengar@linux.microsoft.com>
---
Changes in v3:
* Rebase to latest net-next branch.
Changes in v2:
* No change.
---
drivers/net/ethernet/microsoft/mana/mana_en.c | 1 +
drivers/net/ethernet/microsoft/mana/mana_ethtool.c | 6 ++++++
include/net/mana/mana.h | 2 ++
3 files changed, 9 insertions(+)
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 547dff450b6d..d7079e05dfb8 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1272,6 +1272,7 @@ int mana_query_link_cfg(struct mana_port_context *apc)
return err;
}
apc->speed = resp.link_speed_mbps;
+ apc->max_speed = resp.qos_speed_mbps;
return 0;
}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
index 4fb3a04994a2..a1afa75a9463 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
@@ -495,6 +495,12 @@ static int mana_set_ringparam(struct net_device *ndev,
static int mana_get_link_ksettings(struct net_device *ndev,
struct ethtool_link_ksettings *cmd)
{
+ struct mana_port_context *apc = netdev_priv(ndev);
+ int err;
+
+ err = mana_query_link_cfg(apc);
+ cmd->base.speed = (err) ? SPEED_UNKNOWN : apc->max_speed;
+
cmd->base.duplex = DUPLEX_FULL;
cmd->base.port = PORT_OTHER;
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index 038b18340e51..e1030a7d2daa 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -533,6 +533,8 @@ struct mana_port_context {
u16 port_idx;
/* Currently configured speed (mbps) */
u32 speed;
+ /* Maximum speed supported by the SKU (mbps) */
+ u32 max_speed;
bool port_is_up;
bool port_st_save; /* Saved port state */
--
2.34.1
^ permalink raw reply related
* [PATCH net-next v3 2/4] net: mana: Add support for net_shaper_ops
From: Erni Sri Satya Vennela @ 2025-06-17 7:17 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, andrew+netdev, davem, edumazet,
kuba, pabeni, longli, kotaranov, horms, shirazsaleem, leon, ernis,
shradhagupta, schakrabarti, gerhard, rosenp, sdf, linux-hyperv,
netdev, linux-kernel, linux-rdma
In-Reply-To: <1750144656-2021-1-git-send-email-ernis@linux.microsoft.com>
Introduce support for net_shaper_ops in the MANA driver,
enabling configuration of rate limiting on the MANA NIC.
To apply rate limiting, the driver issues a HWC command via
mana_set_bw_clamp() and updates the corresponding shaper object
in the net_shaper cache. If an error occurs during this process,
the driver restores the previous speed by querying the current link
configuration using mana_query_link_cfg().
The minimum supported bandwidth is 100 Mbps, and only values that are
exact multiples of 100 Mbps are allowed. Any other values are rejected.
To remove a shaper, the driver resets the bandwidth to the maximum
supported by the SKU using mana_set_bw_clamp() and clears the
associated cache entry. If an error occurs during this process,
the shaper details are retained.
On the hardware that does not support these APIs, the net-shaper
calls to set speed would fail.
Set the speed:
./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/net_shaper.yaml \
--do set --json '{"ifindex":'$IFINDEX',
"handle":{"scope": "netdev", "id":'$ID' },
"bw-max": 200000000 }'
Get the shaper details:
./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/net_shaper.yaml \
--do get --json '{"ifindex":'$IFINDEX',
"handle":{"scope": "netdev", "id":'$ID' }}'
> {'bw-max': 200000000,
> 'handle': {'scope': 'netdev'},
> 'ifindex': $IFINDEX,
> 'metric': 'bps'}
Delete the shaper object:
./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/net_shaper.yaml \
--do delete --json '{"ifindex":'$IFINDEX',
"handle":{"scope": "netdev","id":'$ID' }}'
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
Reviewed-by: Saurabh Singh Sengar <ssengar@linux.microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>
---
Changes in v3:
* Rebase to latest net-next branch.
Changes in v2:
* No change.
---
drivers/net/ethernet/microsoft/mana/mana_en.c | 155 ++++++++++++++++++
include/net/mana/mana.h | 40 +++++
2 files changed, 195 insertions(+)
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index bcc33ea7aca3..547dff450b6d 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -719,6 +719,78 @@ static int mana_change_mtu(struct net_device *ndev, int new_mtu)
return err;
}
+static int mana_shaper_set(struct net_shaper_binding *binding,
+ const struct net_shaper *shaper,
+ struct netlink_ext_ack *extack)
+{
+ struct mana_port_context *apc = netdev_priv(binding->netdev);
+ u32 old_speed, rate;
+ int err;
+
+ if (shaper->handle.scope != NET_SHAPER_SCOPE_NETDEV) {
+ NL_SET_ERR_MSG_MOD(extack, "net shaper scope should be netdev");
+ return -EINVAL;
+ }
+
+ if (apc->handle.id && shaper->handle.id != apc->handle.id) {
+ NL_SET_ERR_MSG_MOD(extack, "Cannot create multiple shapers");
+ return -EOPNOTSUPP;
+ }
+
+ if (!shaper->bw_max || (shaper->bw_max % 100000000)) {
+ NL_SET_ERR_MSG_MOD(extack, "Please use multiples of 100Mbps for bandwidth");
+ return -EINVAL;
+ }
+
+ rate = div_u64(shaper->bw_max, 1000); /* Convert bps to Kbps */
+ rate = div_u64(rate, 1000); /* Convert Kbps to Mbps */
+
+ /* Get current speed */
+ err = mana_query_link_cfg(apc);
+ old_speed = (err) ? SPEED_UNKNOWN : apc->speed;
+
+ if (!err) {
+ err = mana_set_bw_clamp(apc, rate, TRI_STATE_TRUE);
+ apc->speed = (err) ? old_speed : rate;
+ apc->handle = (err) ? apc->handle : shaper->handle;
+ }
+
+ return err;
+}
+
+static int mana_shaper_del(struct net_shaper_binding *binding,
+ const struct net_shaper_handle *handle,
+ struct netlink_ext_ack *extack)
+{
+ struct mana_port_context *apc = netdev_priv(binding->netdev);
+ int err;
+
+ err = mana_set_bw_clamp(apc, 0, TRI_STATE_FALSE);
+
+ if (!err) {
+ /* Reset mana port context parameters */
+ apc->handle.id = 0;
+ apc->handle.scope = NET_SHAPER_SCOPE_UNSPEC;
+ apc->speed = 0;
+ }
+
+ return err;
+}
+
+static void mana_shaper_cap(struct net_shaper_binding *binding,
+ enum net_shaper_scope scope,
+ unsigned long *flags)
+{
+ *flags = BIT(NET_SHAPER_A_CAPS_SUPPORT_BW_MAX) |
+ BIT(NET_SHAPER_A_CAPS_SUPPORT_METRIC_BPS);
+}
+
+static const struct net_shaper_ops mana_shaper_ops = {
+ .set = mana_shaper_set,
+ .delete = mana_shaper_del,
+ .capabilities = mana_shaper_cap,
+};
+
static const struct net_device_ops mana_devops = {
.ndo_open = mana_open,
.ndo_stop = mana_close,
@@ -729,6 +801,7 @@ static const struct net_device_ops mana_devops = {
.ndo_bpf = mana_bpf,
.ndo_xdp_xmit = mana_xdp_xmit,
.ndo_change_mtu = mana_change_mtu,
+ .net_shaper_ops = &mana_shaper_ops,
};
static void mana_cleanup_port_context(struct mana_port_context *apc)
@@ -1162,6 +1235,86 @@ static int mana_cfg_vport_steering(struct mana_port_context *apc,
return err;
}
+int mana_query_link_cfg(struct mana_port_context *apc)
+{
+ struct net_device *ndev = apc->ndev;
+ struct mana_query_link_config_resp resp = {};
+ struct mana_query_link_config_req req = {};
+ int err;
+
+ mana_gd_init_req_hdr(&req.hdr, MANA_QUERY_LINK_CONFIG,
+ sizeof(req), sizeof(resp));
+
+ req.vport = apc->port_handle;
+ req.hdr.resp.msg_version = GDMA_MESSAGE_V2;
+
+ err = mana_send_request(apc->ac, &req, sizeof(req), &resp,
+ sizeof(resp));
+
+ if (err) {
+ netdev_err(ndev, "Failed to query link config: %d\n", err);
+ return err;
+ }
+
+ err = mana_verify_resp_hdr(&resp.hdr, MANA_QUERY_LINK_CONFIG,
+ sizeof(resp));
+
+ if (err || resp.hdr.status) {
+ netdev_err(ndev, "Failed to query link config: %d, 0x%x\n", err,
+ resp.hdr.status);
+ if (!err)
+ err = -EOPNOTSUPP;
+ return err;
+ }
+
+ if (resp.qos_unconfigured) {
+ err = -EINVAL;
+ return err;
+ }
+ apc->speed = resp.link_speed_mbps;
+ return 0;
+}
+
+int mana_set_bw_clamp(struct mana_port_context *apc, u32 speed,
+ int enable_clamping)
+{
+ struct mana_set_bw_clamp_resp resp = {};
+ struct mana_set_bw_clamp_req req = {};
+ struct net_device *ndev = apc->ndev;
+ int err;
+
+ mana_gd_init_req_hdr(&req.hdr, MANA_SET_BW_CLAMP,
+ sizeof(req), sizeof(resp));
+ req.vport = apc->port_handle;
+ req.link_speed_mbps = speed;
+ req.enable_clamping = enable_clamping;
+
+ err = mana_send_request(apc->ac, &req, sizeof(req), &resp,
+ sizeof(resp));
+
+ if (err) {
+ netdev_err(ndev, "Failed to set bandwidth clamp for speed %u, err = %d",
+ speed, err);
+ return err;
+ }
+
+ err = mana_verify_resp_hdr(&resp.hdr, MANA_SET_BW_CLAMP,
+ sizeof(resp));
+
+ if (err || resp.hdr.status) {
+ netdev_err(ndev, "Failed to set bandwidth clamp: %d, 0x%x\n", err,
+ resp.hdr.status);
+ if (!err)
+ err = -EOPNOTSUPP;
+ return err;
+ }
+
+ if (resp.qos_unconfigured)
+ netdev_info(ndev, "QoS is unconfigured\n");
+
+ return 0;
+}
+
int mana_create_wq_obj(struct mana_port_context *apc,
mana_handle_t vport,
u32 wq_type, struct mana_obj_spec *wq_spec,
@@ -3011,6 +3164,8 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
goto free_indir;
}
+ debugfs_create_u32("current_speed", 0400, apc->mana_port_debugfs, &apc->speed);
+
return 0;
free_indir:
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index 4176edf1be71..038b18340e51 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -5,6 +5,7 @@
#define _MANA_H
#include <net/xdp.h>
+#include <net/net_shaper.h>
#include "gdma.h"
#include "hw_channel.h"
@@ -526,7 +527,12 @@ struct mana_port_context {
struct mutex vport_mutex;
int vport_use_count;
+ /* Net shaper handle*/
+ struct net_shaper_handle handle;
+
u16 port_idx;
+ /* Currently configured speed (mbps) */
+ u32 speed;
bool port_is_up;
bool port_st_save; /* Saved port state */
@@ -562,6 +568,9 @@ struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);
void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog);
int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);
void mana_query_gf_stats(struct mana_port_context *apc);
+int mana_query_link_cfg(struct mana_port_context *apc);
+int mana_set_bw_clamp(struct mana_port_context *apc, u32 speed,
+ int enable_clamping);
void mana_query_phy_stats(struct mana_port_context *apc);
int mana_pre_alloc_rxbufs(struct mana_port_context *apc, int mtu, int num_queues);
void mana_pre_dealloc_rxbufs(struct mana_port_context *apc);
@@ -589,6 +598,8 @@ enum mana_command_code {
MANA_FENCE_RQ = 0x20006,
MANA_CONFIG_VPORT_RX = 0x20007,
MANA_QUERY_VPORT_CONFIG = 0x20008,
+ MANA_QUERY_LINK_CONFIG = 0x2000A,
+ MANA_SET_BW_CLAMP = 0x2000B,
MANA_QUERY_PHY_STAT = 0x2000c,
/* Privileged commands for the PF mode */
@@ -598,6 +609,35 @@ enum mana_command_code {
MANA_DEREGISTER_HW_PORT = 0x28004,
};
+/* Query Link Configuration*/
+struct mana_query_link_config_req {
+ struct gdma_req_hdr hdr;
+ mana_handle_t vport;
+}; /* HW DATA */
+
+struct mana_query_link_config_resp {
+ struct gdma_resp_hdr hdr;
+ u32 qos_speed_mbps;
+ u8 qos_unconfigured;
+ u8 reserved1[3];
+ u32 link_speed_mbps;
+ u8 reserved2[4];
+}; /* HW DATA */
+
+/* Set Bandwidth Clamp*/
+struct mana_set_bw_clamp_req {
+ struct gdma_req_hdr hdr;
+ mana_handle_t vport;
+ enum TRI_STATE enable_clamping;
+ u32 link_speed_mbps;
+}; /* HW DATA */
+
+struct mana_set_bw_clamp_resp {
+ struct gdma_resp_hdr hdr;
+ u8 qos_unconfigured;
+ u8 reserved[7];
+}; /* HW DATA */
+
/* Query Device Configuration */
struct mana_query_device_cfg_req {
struct gdma_req_hdr hdr;
--
2.34.1
^ permalink raw reply related
* [PATCH net-next v3 1/4] net: mana: Fix potential deadlocks in mana napi ops
From: Erni Sri Satya Vennela @ 2025-06-17 7:17 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, andrew+netdev, davem, edumazet,
kuba, pabeni, longli, kotaranov, horms, shirazsaleem, leon, ernis,
shradhagupta, schakrabarti, gerhard, rosenp, sdf, linux-hyperv,
netdev, linux-kernel, linux-rdma
In-Reply-To: <1750144656-2021-1-git-send-email-ernis@linux.microsoft.com>
When net_shaper_ops are enabled for MANA, netdev_ops_lock
becomes active.
MANA VF setup/teardown by netvsc follows this call chain:
netvsc_vf_setup()
dev_change_flags()
...
__dev_open() OR __dev_close()
dev_change_flags() holds the netdev mutex via netdev_lock_ops.
Meanwhile, mana_create_txq() and mana_create_rxq() in mana_open()
path call NAPI APIs (netif_napi_add_tx(), netif_napi_add_weight(),
napi_enable()), which also try to acquire the same lock, risking
deadlock.
Similarly in the teardown path (mana_close()), netif_napi_disable()
and netif_napi_del(), contend for the same lock.
Switch to the _locked variants of these APIs to avoid deadlocks
when the netdev_ops_lock is held.
Fixes: d4c22ec680c8 ("net: hold netdev instance lock during ndo_open/ndo_stop")
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
Reviewed-by: Saurabh Singh Sengar <ssengar@linux.microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>
---
Changes in v3:
* Rebase to latest net-next branch.
Changes in v2:
* Use netdev_lock_ops_to_full() instead of if...else statements for napi
APIs.
* Edit commit message.
---
drivers/net/ethernet/microsoft/mana/mana_en.c | 28 +++++++++++++------
1 file changed, 19 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index e68b8190bb7a..bcc33ea7aca3 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1912,8 +1912,10 @@ static void mana_destroy_txq(struct mana_port_context *apc)
napi = &apc->tx_qp[i].tx_cq.napi;
if (apc->tx_qp[i].txq.napi_initialized) {
napi_synchronize(napi);
- napi_disable(napi);
- netif_napi_del(napi);
+ netdev_lock_ops_to_full(napi->dev);
+ napi_disable_locked(napi);
+ netif_napi_del_locked(napi);
+ netdev_unlock_full_to_ops(napi->dev);
apc->tx_qp[i].txq.napi_initialized = false;
}
mana_destroy_wq_obj(apc, GDMA_SQ, apc->tx_qp[i].tx_object);
@@ -2065,8 +2067,11 @@ static int mana_create_txq(struct mana_port_context *apc,
mana_create_txq_debugfs(apc, i);
- netif_napi_add_tx(net, &cq->napi, mana_poll);
- napi_enable(&cq->napi);
+ set_bit(NAPI_STATE_NO_BUSY_POLL, &cq->napi.state);
+ netdev_lock_ops_to_full(net);
+ netif_napi_add_locked(net, &cq->napi, mana_poll);
+ napi_enable_locked(&cq->napi);
+ netdev_unlock_full_to_ops(net);
txq->napi_initialized = true;
mana_gd_ring_cq(cq->gdma_cq, SET_ARM_BIT);
@@ -2102,9 +2107,10 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
if (napi_initialized) {
napi_synchronize(napi);
- napi_disable(napi);
-
- netif_napi_del(napi);
+ netdev_lock_ops_to_full(napi->dev);
+ napi_disable_locked(napi);
+ netif_napi_del_locked(napi);
+ netdev_unlock_full_to_ops(napi->dev);
}
xdp_rxq_info_unreg(&rxq->xdp_rxq);
@@ -2355,14 +2361,18 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
gc->cq_table[cq->gdma_id] = cq->gdma_cq;
- netif_napi_add_weight(ndev, &cq->napi, mana_poll, 1);
+ netdev_lock_ops_to_full(ndev);
+ netif_napi_add_weight_locked(ndev, &cq->napi, mana_poll, 1);
+ netdev_unlock_full_to_ops(ndev);
WARN_ON(xdp_rxq_info_reg(&rxq->xdp_rxq, ndev, rxq_idx,
cq->napi.napi_id));
WARN_ON(xdp_rxq_info_reg_mem_model(&rxq->xdp_rxq, MEM_TYPE_PAGE_POOL,
rxq->page_pool));
- napi_enable(&cq->napi);
+ netdev_lock_ops_to_full(ndev);
+ napi_enable_locked(&cq->napi);
+ netdev_unlock_full_to_ops(ndev);
mana_gd_ring_cq(cq->gdma_cq, SET_ARM_BIT);
out:
--
2.34.1
^ permalink raw reply related
* [PATCH net-next v3 0/4] Support bandwidth clamping in mana using net shapers
From: Erni Sri Satya Vennela @ 2025-06-17 7:17 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, andrew+netdev, davem, edumazet,
kuba, pabeni, longli, kotaranov, horms, shirazsaleem, leon, ernis,
shradhagupta, schakrabarti, gerhard, rosenp, sdf, linux-hyperv,
netdev, linux-kernel, linux-rdma
This patchset introduces hardware-backed bandwidth rate limiting
for MANA NICs via the net_shaper_ops interface, enabling efficient and
fine-grained traffic shaping directly on the device.
Previously, MANA lacked a mechanism for user-configurable bandwidth
control. With this addition, users can now configure shaping parameters,
allowing better traffic management and performance isolation.
The implementation includes the net_shaper_ops callbacks in the MANA
driver and supports one shaper per vport. Add shaping support via
mana_set_bw_clamp(), allowing the configuration of bandwidth rates
in 100 Mbps increments (minimum 100 Mbps). The driver validates input
and rejects unsupported values. On failure, it restores the previous
configuration which is queried using mana_query_link_cfg() or
retains the current state.
To prevent potential deadlocks introduced by net_shaper_ops, switch to
_locked variants of NAPI APIs when netdevops_lock is held during
VF setup and teardown.
Also, Add support for ethtool get_link_ksettings to report the maximum
link speed supported by the SKU in mbps.
These APIs when invoked on hardware that are older or that do
not support these APIs, the speed would be reported as UNKNOWN and
the net-shaper calls to set speed would fail.
---
Changes in v3:
* Rebase to latest net-next branch.
Changes in v2:
* Rebase to latest net-next branch.
* Edit commit message in
"net: mana: Fix potential deadlocks in mana napi ops".
* Use netdev_lock_ops_to_full() instead of explicit if..else statements
for napi APIs.
* Define GDMA_STATUS_CMD_UNSUPPORTED for unsupported HWC status code.
---
Erni Sri Satya Vennela (4):
net: mana: Fix potential deadlocks in mana napi ops
net: mana: Add support for net_shaper_ops
net: mana: Add speed support in mana_get_link_ksettings
net: mana: Handle unsupported HWC commands
.../net/ethernet/microsoft/mana/hw_channel.c | 4 +
drivers/net/ethernet/microsoft/mana/mana_en.c | 195 +++++++++++++++++-
.../ethernet/microsoft/mana/mana_ethtool.c | 6 +
include/net/mana/gdma.h | 1 +
include/net/mana/mana.h | 42 ++++
5 files changed, 239 insertions(+), 9 deletions(-)
--
2.34.1
^ permalink raw reply
* Re: [PATCH net-next v2 0/4] Support bandwidth clamping in mana using net shapers
From: Erni Sri Satya Vennela @ 2025-06-17 6:16 UTC (permalink / raw)
To: Jakub Kicinski
Cc: kys, haiyangz, wei.liu, decui, andrew+netdev, davem, edumazet,
pabeni, longli, kotaranov, horms, shirazsaleem, leon,
shradhagupta, schakrabarti, gerhard, rosenp, sdf, linux-hyperv,
netdev, linux-kernel, linux-rdma
In-Reply-To: <20250614130354.0a53214e@kernel.org>
On Sat, Jun 14, 2025 at 01:03:54PM -0700, Jakub Kicinski wrote:
> On Fri, 13 Jun 2025 04:20:23 -0700 Erni Sri Satya Vennela wrote:
> > This patchset introduces hardware-backed bandwidth rate limiting
> > for MANA NICs via the net_shaper_ops interface, enabling efficient and
> > fine-grained traffic shaping directly on the device.
> >
> > Previously, MANA lacked a mechanism for user-configurable bandwidth
> > control. With this addition, users can now configure shaping parameters,
> > allowing better traffic management and performance isolation.
> >
> > The implementation includes the net_shaper_ops callbacks in the MANA
> > driver and supports one shaper per vport. Add shaping support via
> > mana_set_bw_clamp(), allowing the configuration of bandwidth rates
> > in 100 Mbps increments (minimum 100 Mbps). The driver validates input
> > and rejects unsupported values. On failure, it restores the previous
> > configuration which is queried using mana_query_link_cfg() or
> > retains the current state.
> >
> > To prevent potential deadlocks introduced by net_shaper_ops, switch to
> > _locked variants of NAPI APIs when netdevops_lock is held during
> > VF setup and teardown.
> >
> > Also, Add support for ethtool get_link_ksettings to report the maximum
> > link speed supported by the SKU in mbps.
> >
> > These APIs when invoked on hardware that are older or that do
> > not support these APIs, the speed would be reported as UNKNOWN and
> > the net-shaper calls to set speed would fail.
>
> Failed to apply patch:
> Applying: net: mana: Fix potential deadlocks in mana napi ops
> error: patch fragment without header at line 23: @@ -2102,9 +2108,11 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
> error: could not build fake ancestor
> hint: Use 'git am --show-current-patch=diff' to see the failed patch
> hint: When you have resolved this problem, run "git am --continue".
> hint: If you prefer to skip this patch, run "git am --skip" instead.
> hint: To restore the original branch and stop patching, run "git am --abort".
> hint: Disable this message with "git config set advice.mergeConflict false"
> Patch failed at 0001 net: mana: Fix potential deadlocks in mana napi ops
>
> please rebase
Hi Jakub,
Thank you for your reply. I will rebase and repost.
- Vennela
> --
> pw-bot: cr
^ permalink raw reply
* Re: [PATCH v6 0/5] Allow dyn MSI-X vector allocation of MANA
From: Jakub Kicinski @ 2025-06-17 1:11 UTC (permalink / raw)
To: Shradha Gupta
Cc: linux-hyperv, linux-pci, linux-kernel, Nipun Gupta, Yury Norov,
Jason Gunthorpe, Jonathan Cameron, Anna-Maria Behnsen, Kevin Tian,
Long Li, Thomas Gleixner, Bjorn Helgaas, Rob Herring,
Manivannan Sadhasivam, Krzysztof Wilczy???~Dski,
Lorenzo Pieralisi, Dexuan Cui, Wei Liu, Haiyang Zhang,
K. Y. Srinivasan, Andrew Lunn, David S. Miller, Eric Dumazet,
Paolo Abeni, Konstantin Taranov, Simon Horman, Leon Romanovsky,
Maxim Levitsky, Erni Sri Satya Vennela, Peter Zijlstra, netdev,
linux-rdma, Paul Rosswurm, Shradha Gupta
In-Reply-To: <20250612061055.GA20126@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net>
On Wed, 11 Jun 2025 23:10:55 -0700 Shradha Gupta wrote:
> On Wed, Jun 11, 2025 at 08:54:16AM -0700, Jakub Kicinski wrote:
> > On Wed, 11 Jun 2025 07:09:44 -0700 Shradha Gupta wrote:
> > > Changes in v6
> > > * rebased to linux-next's v6.16-rc1 as per Jakub's suggestion
> >
> > I meant a branch, basically apply the patches on the v6.16-rc1 tag
> > and push it out to GitHub, kernel.org or somewhere else public.
> > Then we can pull it in and maintain the stable commit IDs.
> > No need to repost the patches, FWIW, just share the branch here
> > once you pushed it out..
>
> Oh, understood. Thanks for the clarity. Here is a github repo branch
> with the changes on v6.16-rc1 tag
> https://github.com/shradhagupta6/linux/tree/shradha_v6.16-rc1
The tag was good, but when I pulled it my check scripts complained:
Commit a19036b86845 ("net: mana: Allocate MSI-X vectors dynamically")
committer Signed-off-by missing
author email: shradhagupta@linux.microsoft.com
committer email: shradhagupta@microsoft.com
Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
etc. You seem to have committed the patches with a slightly different
email address. Not a huge deal but better to fix it if we can.
So please base the tag. The code can stay the same just adjust the
committer or author/signoff email addrs. We can use this as an
opportunity to add Bjorn's email.
No need to repost the code just ping here once you updated the tag.
^ permalink raw reply
* Re: [PATCH net-next,v8] net: mana: Add handler for hardware servicing events
From: patchwork-bot+netdevbpf @ 2025-06-16 22:30 UTC (permalink / raw)
To: Haiyang Zhang
Cc: linux-hyperv, netdev, haiyangz, decui, stephen, kys, paulros,
olaf, vkuznets, davem, wei.liu, edumazet, kuba, pabeni, leon,
longli, ssengar, linux-rdma, daniel, john.fastabend, bpf, ast,
hawk, tglx, shradhagupta, andrew+netdev, kotaranov, horms,
linux-kernel
In-Reply-To: <1749834034-18498-1-git-send-email-haiyangz@linux.microsoft.com>
Hello:
This patch was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:
On Fri, 13 Jun 2025 10:00:34 -0700 you wrote:
> From: Haiyang Zhang <haiyangz@microsoft.com>
>
> To collaborate with hardware servicing events, upon receiving the special
> EQE notification from the HW channel, remove the devices on this bus.
> Then, after a waiting period based on the device specs, rescan the parent
> bus to recover the devices.
>
> [...]
Here is the summary with links:
- [net-next,v8] net: mana: Add handler for hardware servicing events
https://git.kernel.org/netdev/net-next/c/7768c5f41733
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply
* Re: [Patch net-next v2] net: mana: Record doorbell physical address in PF mode
From: Jakub Kicinski @ 2025-06-16 22:23 UTC (permalink / raw)
To: longli
Cc: longli, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
David S. Miller, Eric Dumazet, Paolo Abeni, Shradha Gupta,
Simon Horman, Konstantin Taranov, Souradeep Chakrabarti,
Erick Archer, linux-hyperv, netdev, linux-kernel, linux-rdma
In-Reply-To: <1749836765-28886-1-git-send-email-longli@linuxonhyperv.com>
On Fri, 13 Jun 2025 10:46:05 -0700 longli@linuxonhyperv.com wrote:
> MANA supports RDMA in PF mode. The driver should record the doorbell
> physical address when in PF mode.
>
> The doorbell physical address is used by the RDMA driver to map
> doorbell pages of the device to user-mode applications through RDMA
> verbs interface. In the past, they have been mapped to user-mode while
> the device is in VF mode. With the support for PF mode implemented,
> also expose those pages in PF mode.
It'd be good to indicate if the PF mode support is implemented as in
ready to be posted to the list, or implemented as in already in the
tree? If it's in tree then please quote the commit. If upcoming please
rephrase.
This commit msg is much better, thank you, but to a person handling
backports it will still not be immediately clear whether this is prep
work for a new code path, or a bug fix.
--
pw-bot: cr
^ permalink raw reply
* Re: [PATCH 2/4] Drivers: hv: Use nested hypercall for post message and signal event
From: Nuno Das Neves @ 2025-06-16 20:18 UTC (permalink / raw)
To: Michael Kelley, linux-hyperv@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org
Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
decui@microsoft.com, catalin.marinas@arm.com, will@kernel.org,
tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
dave.hansen@linux.intel.com, hpa@zytor.com, lpieralisi@kernel.org,
kw@linux.com, manivannan.sadhasivam@linaro.org, robh@kernel.org,
bhelgaas@google.com, jinankjain@linux.microsoft.com,
skinsburskii@linux.microsoft.com, mrathor@linux.microsoft.com,
x86@kernel.org
In-Reply-To: <SN6PR02MB4157ECC740A762C9C3B9EA6AD475A@SN6PR02MB4157.namprd02.prod.outlook.com>
On 6/11/2025 4:06 PM, Michael Kelley wrote:
> From: Nuno Das Neves <nunodasneves@linux.microsoft.com> Sent: Tuesday, June 10, 2025 4:52 PM
>>
>> When running nested, these hypercalls must be sent to the L0 hypervisor
>> or vmbus will fail.
>
> s/vmbus/VMBus/
>
Ack
>>
>> Add ARM64 stubs for the nested hypercall helpers to not break
>> compilation (nested is still only supported in x86).
>>
>> Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
>> ---
>> arch/arm64/include/asm/mshyperv.h | 10 ++++++++++
>> drivers/hv/connection.c | 3 +++
>> drivers/hv/hv.c | 3 +++
>> 3 files changed, 16 insertions(+)
>>
>> diff --git a/arch/arm64/include/asm/mshyperv.h
>> b/arch/arm64/include/asm/mshyperv.h
>> index b721d3134ab6..893d6a2e8dab 100644
>> --- a/arch/arm64/include/asm/mshyperv.h
>> +++ b/arch/arm64/include/asm/mshyperv.h
>> @@ -53,6 +53,16 @@ static inline u64 hv_get_non_nested_msr(unsigned int reg)
>> return hv_get_msr(reg);
>> }
>>
>> +static inline u64 hv_do_nested_hypercall(u64 control, void *input, void *output)
>> +{
>> + return U64_MAX;
>> +}
>> +
>> +static inline u64 hv_do_fast_nested_hypercall8(u64 control, u64 input1)
>> +{
>> + return U64_MAX;
>> +}
>
> I think the definitions of hv_do_nested_hypercall() and
> hv_do_fast_nested_hypercall8() are architecture independent. All
> they do is add the HV_HYPERCALL_NESTED flag, which when
> implemented for ARM64, will presumably be the same flag as
> currently defined for x86. As such, couldn't the definitions of
> hv_do_nested_hypercall() and hv_do_fast_nested_hypercall8()
> be moved to asm-generic/mshyperv.h? Then stubs would not
> be needed for ARM64. These two functions would never be
> called on ARM64 because hv_nested is never true on ARM64
> (at least for now), but the code would compile. And if either
> function was erroneously called on ARM64, presumably
> Hyper-V would return an error because HV_HYPERCALL_NESTED
> is set.
>
Good point - letting the hypervisor return the error is fine.
>> +
>> /* SMCCC hypercall parameters */
>> #define HV_SMCCC_FUNC_NUMBER 1
>> #define HV_FUNC_ID ARM_SMCCC_CALL_VAL( \
>> diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
>> index be490c598785..992022bc770c 100644
>> --- a/drivers/hv/connection.c
>> +++ b/drivers/hv/connection.c
>> @@ -518,6 +518,9 @@ void vmbus_set_event(struct vmbus_channel *channel)
>> channel->sig_event, 0);
>> else
>> WARN_ON_ONCE(1);
>> + } else if (hv_nested) {
>> + hv_do_fast_nested_hypercall8(HVCALL_SIGNAL_EVENT,
>> + channel->sig_event);
>> } else {
>> hv_do_fast_hypercall8(HVCALL_SIGNAL_EVENT, channel->sig_event);
>> }
>> diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
>> index 308c8f279df8..99b73e779bf0 100644
>> --- a/drivers/hv/hv.c
>> +++ b/drivers/hv/hv.c
>> @@ -84,6 +84,9 @@ int hv_post_message(union hv_connection_id connection_id,
>> sizeof(*aligned_msg));
>> else
>> status = HV_STATUS_INVALID_PARAMETER;
>> + } else if (hv_nested) {
>> + status = hv_do_nested_hypercall(HVCALL_POST_MESSAGE,
>> + aligned_msg, NULL);
>> } else {
>> status = hv_do_hypercall(HVCALL_POST_MESSAGE,
>> aligned_msg, NULL);
>
> Are HVCALL_SIGNAL_EVENT and HVCALL_POST_MESSAGE the only two
> hypercalls that are ever expected to need a "nested" version? I'm
> wondering if the function hv_do_nested_hypercall() and
> hv_do_fast_nested_hypercall8() could be dropped entirely, and just
> pass the first argument to hv_do_hypercall() or hv_do_fast_hypercall8()
> as <hypercall_name> | HV_HYPERCALL_NESTED. For only two cases, a
> little bit of open coding might be preferable to the overhead of defining
> functions just to wrap the or'ing of HV_HYPERCALL_NESTED.
>
> The code above could then look like:
>
> } else {
> u64 control = HVCALL_POST_MESSAGE;
>
> control |= hv_nested ? HV_HYPERCALL_NESTED : 0;
> status = hv_do_hypercall(control, aligned_msg, NULL);
> }
>
> Again, ARM64 is implicitly handled because hv_nested is never set.
>
> This is just a suggestion. It's motivated by the fact that we already have
> three flavors of hypercall for HVCALL_SIGNAL_EVENT and
> HVCALL_POST_MESSAGE, and I was looking for a way to avoid adding
> a fourth flavor. But it's a marginal win, and if you prefer to keep the
> inline functions, I'm OK with that.
>
I like the suggestion to open-code these cases.
There are several places we need to get/set nested MSRs, but as for hypercalls
it is just these 2, so far. When I consider it from that angle, it feels cleaner
to just open-code them, and remove the existing '_nested' versions of the
hypercall helpers for now.
Thanks
Nuno
> Michael
^ permalink raw reply
* Re: [PATCH 1/4] PCI: hv: Do not do vmbus initialization on baremetal
From: Nuno Das Neves @ 2025-06-16 20:06 UTC (permalink / raw)
To: Michael Kelley, linux-hyperv@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org
Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
decui@microsoft.com, catalin.marinas@arm.com, will@kernel.org,
tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
dave.hansen@linux.intel.com, hpa@zytor.com, lpieralisi@kernel.org,
kw@linux.com, manivannan.sadhasivam@linaro.org, robh@kernel.org,
bhelgaas@google.com, jinankjain@linux.microsoft.com,
skinsburskii@linux.microsoft.com, mrathor@linux.microsoft.com,
x86@kernel.org
In-Reply-To: <SN6PR02MB41574163515870DB8E430C56D475A@SN6PR02MB4157.namprd02.prod.outlook.com>
On 6/11/2025 4:06 PM, Michael Kelley wrote:
> From: Nuno Das Neves <nunodasneves@linux.microsoft.com> Sent: Tuesday, June 10, 2025 4:52 PM
>>
>> From: Mukesh Rathor <mrathor@linux.microsoft.com>
>
> The patch Subject line is confusing to me. Perhaps this better captures
> the intent:
>
> "PCI: hv: Don't load the driver for the root partition on a bare-metal hypervisor"
>
Thanks, that does make more sense.
>>
>> init_hv_pci_drv() is not relevant for root partition on baremetal as there
>> is no vmbus. On nested (with a Windows L1 root), vmbus is present.
>
> This needs more precision. init_hv_pci_drv() isn't what is
> "not relevant". It's the entire driver that doesn't need to be loaded
> because the root partition on a bare-metal hypervisor doesn't use
> VMBus. It's only when the root partition is running on a nested
> hypervisor that it uses VMBus, and hence may need the Hyper-V
> PCI driver to be loaded.
>
I'll update it so it is clearer, thanks.
>>
>> Signed-off-by: Mukesh Rathor <mrathor@linux.microsoft.com>
>> Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
>> ---
>> drivers/pci/controller/pci-hyperv.c | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
>> index b4f29ee75848..4d25754dfe2f 100644
>> --- a/drivers/pci/controller/pci-hyperv.c
>> +++ b/drivers/pci/controller/pci-hyperv.c
>> @@ -4150,6 +4150,9 @@ static int __init init_hv_pci_drv(void)
>> if (!hv_is_hyperv_initialized())
>> return -ENODEV;
>>
>> + if (hv_root_partition() && !hv_nested)
>> + return -ENODEV;
>> +
>> ret = hv_pci_irqchip_init();
>> if (ret)
>> return ret;
>> --
>> 2.34.1
^ permalink raw reply
* Re: [Patch net-next v2] net: mana: Record doorbell physical address in PF mode
From: Shradha Gupta @ 2025-06-16 15:32 UTC (permalink / raw)
To: longli
Cc: K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, Konstantin Taranov, Souradeep Chakrabarti,
Erick Archer, linux-hyperv, netdev, linux-kernel, linux-rdma
In-Reply-To: <1749836765-28886-1-git-send-email-longli@linuxonhyperv.com>
On Fri, Jun 13, 2025 at 10:46:05AM -0700, longli@linuxonhyperv.com wrote:
> From: Long Li <longli@microsoft.com>
>
> MANA supports RDMA in PF mode. The driver should record the doorbell
> physical address when in PF mode.
>
> The doorbell physical address is used by the RDMA driver to map
> doorbell pages of the device to user-mode applications through RDMA
> verbs interface. In the past, they have been mapped to user-mode while
> the device is in VF mode. With the support for PF mode implemented,
> also expose those pages in PF mode.
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
> Changes
> v2: add more details in commit message on how the doorbell physical address is used
>
> drivers/net/ethernet/microsoft/mana/gdma_main.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> index 3504507477c6..52cf7112762c 100644
> --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
> +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> @@ -31,6 +31,9 @@ static void mana_gd_init_pf_regs(struct pci_dev *pdev)
> gc->db_page_base = gc->bar0_va +
> mana_gd_r64(gc, GDMA_PF_REG_DB_PAGE_OFF);
>
> + gc->phys_db_page_base = gc->bar0_pa +
> + mana_gd_r64(gc, GDMA_PF_REG_DB_PAGE_OFF);
> +
> sriov_base_off = mana_gd_r64(gc, GDMA_SRIOV_REG_CFG_BASE_OFF);
>
> sriov_base_va = gc->bar0_va + sriov_base_off;
> --
> 2.25.1
Reviewed-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox