Re: [RFC] PCI/MSI: Warning observed for NVMe with ACPI

Thomas Gleixner
2021-12-11T09:50:33Z

On Fri, Dec 10 2021 at 11:39, Marc Zyngier wrote:
> On Fri, 10 Dec 2021 10:48:22 +0000,
> Jon Hunter <jonathanh@nvidia.com> wrote:
>> + select PCI_MSI_ARCH_FALLBACKS if PCI_MSI
>> select POWER_RESET
>> select POWER_SUPPLY
>> select SPARSE_IRQ
>
> +Thomas, as he's neck-deep in the MSI rework.
>
> No, this definitely is the wrong solution.

Correct.

> arm64 doesn't need any arch fallback (I actually went out of my way to
> kill them on this architecture), and requires the individual MSI
> controller drivers to do the right thing by using MSI domains. Adding
> this config option makes the warning disappear, but the core issue is
> that you have a device that doesn't have a MSI domain associated with
> it.
>
> So either your device isn't MSI capable (odd), your host bridge
> doesn't make the link with the MSI controller to advertise the MSI
> domain (this should normally be dealt with via IORT), or there is a
> bug of a similar sort somewhere else.

What's even more odd is:

>> [ 2.725479] WARNING: CPU: 0 PID: 94 at include/linux/msi.h:264 free_msi_irqs+0x84/0x188
>> [ 2.825275] free_msi_irqs+0x84/0x188
>> [ 2.825288] __pci_enable_msix_range+0x380/0x530
>> [ 2.825299] pci_alloc_irq_vectors_affinity+0x158/0x168

From __pci_enable_msix_range() there are two ways to reach free_msi_irqs():

1) pci_alloc_irq_vectors_affinity()
__pci_enable_msix_range()
__pci_enable_msix()
msix_capability_init()
msix_setup_entries()
alloc_msi_entry(() -> allocation fail

2) pci_alloc_irq_vectors_affinity()
__pci_enable_msix_range()
__pci_enable_msix()
msix_capability_init()
pci_msi_setup_msi_irqs(); -> any failure after this succeeded

#1 is unlikely

#2 is odd because if the irqdomain of the device is not hierarchical,
then the same warning should trigger already in
pci_msi_setup_msi_irqs() via arch_setup_msi_irqs().

Strange.

Thanks,

tglx