Re: [PATCH v2] PCI: vmd: Clear PCI_INTERRUPT_LINE for VMD sub-devices

[Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>]

On Thu, Sep 12, 2024 at 10:11:25AM -0700, Dan Williams wrote:
> Manivannan Sadhasivam wrote:
> > On Thu, Aug 22, 2024 at 11:30:10AM -0700, Nirmal Patel wrote:
> > > On Thu, 22 Aug 2024 15:18:06 +0530
> > > Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> wrote:
> > > 
> > > > On Tue, Aug 20, 2024 at 03:32:13PM -0700, Nirmal Patel wrote:
> > > > > VMD does not support INTx for devices downstream from a VMD
> > > > > endpoint. So initialize the PCI_INTERRUPT_LINE to 0 for all NVMe
> > > > > devices under VMD to ensure other applications don't try to set up
> > > > > an INTx for them.
> > > > > 
> > > > > Signed-off-by: Nirmal Patel <nirmal.patel@linux.intel.com>  
> > > > 
> > > > I shared a diff to put it in pci_assign_irq() and you said that you
> > > > were going to test it [1]. I don't see a reply to that and now you
> > > > came up with another approach.
> > > > 
> > > > What happened inbetween?
> > > 
> > > Apologies, I did perform the tests and the patch worked fine. However, I
> > > was able to see lot of bridge devices had the register set to 0xFF and I
> > > didn't want to alter them.
> > 
> > You should've either replied to my comment or mentioned it in the changelog.
> > 
> > > Also pci_assign_irg would still set the
> > > interrupt line register to 0 with or without VMD. Since I didn't want to
> > > introduce issues for non-VMD setup, I decide to keep the change limited
> > > only to the VMD.
> > > 
> > 
> > Sorry no. SPDK usecase is not specific to VMD and so is the issue. So this
> > should be fixed in the PCI core as I proposed. What if another bridge also wants
> > to do the same?
> 
> Going to say this rather harshly, but there is no conceivable universe I
> can imagine where the Linux PCI core should be bothered with the
> idiosyncracies of VMD. VMD is not a PCI bridge.
> 

I don't think the issue should be constrained to VMD only. Based on my
conversation with Nirmal [1], I understood that it is SPDK that makes wrong
assumption if the device's PCI_INTERRUPT_LINE is non-zero (and I assumed that
other application could do the same). In that case, how it can be classified
as the "idiosyncracy" of VMD? SPDK is not tied to VMD, isn't it?

- Mani

[1] https://lore.kernel.org/linux-pci/20240730052830.GA3122@thinkpad/

> SPDK is fully capable of doing this fixup itself in the presence of VMD.
> Unless and until this problem is apparent in more than just a kernel
> bypass development kit should the kernel worry about it, and even then
> the fixup must be contained to the VMD driver with all its other
> workarounds that to try to get back to standards compliant PCI bridge
> behavior.

-- 
மணிவண்ணன் சதாசிவம்