[RFC] PCI/MSI: Warning observed for NVMe with ACPI

Jon Hunter
2021-12-10T10:48:32Z

Hi all,

Since Linux v5.13, we have noticed that following warning splat when
booting Tegra (ARM64) with ACPI ...

[ 2.725479] WARNING: CPU: 0 PID: 94 at include/linux/msi.h:264 free_msi_irqs+0x84/0x188
[ 2.736137] Modules linked in:
[ 2.736147] CPU: 0 PID: 94 Comm: kworker/u16:1 Tainted: G W 5.12.0-rc2-00008-g658376bd3e5-dirty #36
[ 2.736160] Workqueue: nvme-reset-wq nvme_reset_work
[ 2.746470] pstate: 60400009 (nZCv daif +PAN -UAO -TCO BTYPE=--)
[ 2.757713] pc : free_msi_irqs+0x84/0x188
[ 2.757726] lr : __pci_enable_msix_range+0x380/0x530
[ 2.757735] sp : ffff800012813b00
[ 2.757739] x29: ffff800012813b00
[ 2.768371] x28: 00000000ffffffed
[ 2.768382] x27: 0000000000000001 x26: 0000000000000000
[ 2.768393] x25: ffff0000809362e8 x24: 0000000000000000
[ 2.768407] x23: 000000000000000c x22: ffff000080936000
[ 2.768418] x21: ffff0000809362e8 x20: ffff0000809362e8
[ 2.775320] x19: ffff000080936000
[ 2.785950] x18: ffffffffffffffff
[ 2.785961] x17: 0000000000000007 x16: 0000000000000001
[ 2.785975] x15: ffff800011bf9948
[ 2.793997] x14: ffff8000928137e7
[ 2.794009] x13: ffff8000128137f5 x12: ffff800011c19640
[ 2.794023] x11: fffffffffffe5788 x10: 0000000005f5e0ff
[ 2.794034] x9 : 00000000ffffffd0 x8 : 203a737542204f49
[ 2.803737] x7 : 444d206465786946 x6 : ffff800011ee1fd7
[ 2.803750] x5 : 0000000000000000 x4 : 0000000000000000
[ 2.815286] x3 : 00000000ffffffff x2 : ffff0000809362e8
[ 2.815300] x1 : ffff0000809362e8 x0 : 0000000000000000
[ 2.825270] Call trace:
[ 2.825275] free_msi_irqs+0x84/0x188
[ 2.825288] __pci_enable_msix_range+0x380/0x530
[ 2.825299] pci_alloc_irq_vectors_affinity+0x158/0x168
[ 2.825309] nvme_reset_work+0x214/0x15b8
[ 2.829340] dwc-eth-dwmac NVDA1160:00: SPH feature enabled
[ 2.832986] process_one_work+0x1cc/0x360
[ 2.833002] worker_thread+0x48/0x450
[ 2.833012] kthread+0x120/0x150
[ 2.833020] ret_from_fork+0x10/0x18

Bisecting this I found that started to occur because with Linux v5.13,
CONFIG_PCI_MSI_ARCH_FALLBACKS was no longer enabled by default and only
happened to be enabled because Renesas R-Car was enabling it.

When booting with ACPI, I see that when pci_msi_setup_msi_irqs() is
called, it ends up calling arch_setup_msi_irqs() and if
CONFIG_PCI_MSI_ARCH_FALLBACKS is not enabled, then this will call
WARN_ON_ONCE(1).

So the question is, should this be enabled by default for ARM64? I see
a lot of other architectures enabling this when PCI_MSI is enabled. So
I am wondering if we should be doing something like ...

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1f212b47a48a..4bbd81bab809 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -202,6 +202,7 @@ config ARM64
select PCI_DOMAINS_GENERIC if PCI
select PCI_ECAM if (ACPI && PCI)
select PCI_SYSCALL if PCI
+ select PCI_MSI_ARCH_FALLBACKS if PCI_MSI
select POWER_RESET
select POWER_SUPPLY
select SPARSE_IRQ

Cheers
Jon

--
nvpublic