Re: [PATCH v4 1/4] PCI/MSI: Conservatively generalize no_64bit_msi into msi_addr_mask

Mark Bloch
2026-02-26T18:25:46Z

On 29/01/2026 3:56, Vivian Wang wrote:
> Some PCI devices have PCI_MSI_FLAGS_64BIT in the MSI capability, but
> implement less than 64 address bits. This breaks on platforms where such
> a device is assigned an MSI address higher than what's reachable.
>
> Currently, the no_64bit_msi bit is set for these devices, meaning that
> only 32-bit MSI addresses are allowed for them. However, on some
> platforms the MSI doorbell address is above the 32-bit limit but within
> the addressable range of the device.
>
> As a first step to enabling MSI on those combinations of devices and
> platforms, conservatively generalize the single-bit flag no_64bit_msi
> into msi_addr_mask. (The name msi_addr_mask is chosen to avoid confusion
> with msi_mask.)
>
> The translation is essentially:
>
> - no_64bit_msi = 1 -> msi_addr_mask = DMA_BIT_MASK(32)
> - no_64bit_msi = 0 -> msi_addr_mask = DMA_BIT_MASK(64)
> - if (no_64bit_msi) -> if (msi_addr_mask < DMA_BIT_MASK(64))
>

Hey Vivian,

We are seeing issues while reloading mlx5 on a PPC64 platform.
We see the following messages in dmesg:

mlx5_core 0000:00:08.0: mlx5_load:1266:(pid 1283): Failed to alloc IRQs
mlx5_core 0000:00:08.0: E-Switch: cleanup
mlx5_core 0000:00:08.0: probe_one:1959:(pid 1283): mlx5_init_one failed with error code -19
mlx5_core 0000:00:08.1: lsa_required: 0, lsa_enabled: 0, direct mapping: 0
mlx5_core 0000:00:08.1: lsa_required: 0, lsa_enabled: 0, direct mapping: 0
mlx5_core 0000:00:08.1: firmware version: 16.35.4506
mlx5_core 0000:00:08.1: 0.000 Gb/s available PCIe bandwidth (Unknown x16 link)
mlx5_core 0000:00:08.1: Rate limit: 127 rates are supported, range: 0Mbps to 97656Mbps
mlx5_core 0000:00:08.1: E-Switch: Total vports 2, per vport: max uc(128) max mc(2048)
mlx5_core 0000:00:08.1: Flow counters bulk query buffer size increased, bulk_query_len(8)
mlx5_core 0000:00:08.1: mlx5_load:1266:(pid 1283): Failed to alloc IRQs
mlx5_core 0000:00:08.1: E-Switch: cleanup
mlx5_core 0000:00:08.1: probe_one:1959:(pid 1283): mlx5_init_one failed with error code -19

We've bisectedthe issue to this patch.
We've pointed some AI tool to look at the reproducation script and this patch and it came
up with the following idea:

"
The bug is a missing initialization of msi_addr_mask on PPC64LE, caused by a PPC-specific PCI device creation path.
On most architectures, PCI devices are enumerated via the standard path:
> pci_scan_device() -> pci_alloc_dev() (kzalloc) -> pci_setup_device() -> pci_device_add()
But PPC has its own device-tree-based PCI enumeration in arch/powerpc/kernel/pci_of_scan.c.
The function of_create_pci_dev() does this:

pci_of_scan.cLines 215-235

dev = pci_alloc_dev(bus); // kzalloc -> all fields zeroed, msi_addr_mask = 0
// ... manually sets vendor, device, class, dma_mask, etc.
// ... does NOT call pci_setup_device() !!!
pci_device_add(dev, bus); // calls pci_init_capabilities -> pci_msi_init

Inside pci_msi_init(), the code only sets msi_addr_mask = DMA_BIT_MASK(32) when 64-bit MSI is not supported.
If the device does support 64-bit MSI (like mlx5), it leaves msi_addr_mask untouched -- meaning it stays at 0 from the kzalloc.
"

We tried a patch that moved dev->msi_addr_mask = DMA_BIT_MASK(64) from pci_setup_device()
to pci_alloc_dev() and it solved the issue we were seeing (don't know if this is the proper fix).

Can you please have a look?

Mark

> Since no values other than DMA_BIT_MASK(32) and DMA_BIT_MASK(64) are
> used, no functional change is intended. Future patches that make use of
> intermediate values of msi_addr_mask will follow, allowing devices that
> cannot use full 64-bit addresses for MSI to work on platforms with MSI
> doorbell above the 32-bit limit.
>
> Acked-by: Takashi Iwai <tiwai@suse.de> # sound
> Reviewed-by: Brett Creeley <brett.creeley@amd.com> # ionic
> Reviewed-by: Thomas Gleixner <tglx@kernel.org>
> Signed-off-by: Vivian Wang <wangruikang@iscas.ac.cn>
> ---
> v4: Patch message rewording (Thomas)
>
> checkpatch complains about the comment include/linux/pci.h, which I have
> formatted similarly with other comments in the vicinity.
> ---
> arch/powerpc/platforms/powernv/pci-ioda.c | 2 +-
> arch/powerpc/platforms/pseries/msi.c | 4 ++--
> drivers/gpu/drm/radeon/radeon_irq_kms.c | 2 +-
> drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c | 2 +-
> drivers/pci/msi/msi.c | 2 +-
> drivers/pci/msi/pcidev_msi.c | 2 +-
> drivers/pci/probe.c | 7 +++++++
> include/linux/pci.h | 8 +++++++-
> sound/hda/controllers/intel.c | 2 +-
> 9 files changed, 22 insertions(+), 9 deletions(-)
>
> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
> index b0c1d9d16fb5..1c78fdfb7b03 100644
> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> @@ -1666,7 +1666,7 @@ static int __pnv_pci_ioda_msi_setup(struct pnv_phb *phb, struct pci_dev *dev,
> return -ENXIO;
>
> /* Force 32-bit MSI on some broken devices */
> - if (dev->no_64bit_msi)
> + if (dev->msi_addr_mask < DMA_BIT_MASK(64))
> is_64 = 0;
>
> /* Assign XIVE to PE */
> diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
> index a82aaa786e9e..7473c7ca1db0 100644
> --- a/arch/powerpc/platforms/pseries/msi.c
> +++ b/arch/powerpc/platforms/pseries/msi.c
> @@ -383,7 +383,7 @@ static int rtas_prepare_msi_irqs(struct pci_dev *pdev, int nvec_in, int type,
> */
> again:
> if (type == PCI_CAP_ID_MSI) {
> - if (pdev->no_64bit_msi) {
> + if (pdev->msi_addr_mask < DMA_BIT_MASK(64)) {
> rc = rtas_change_msi(pdn, RTAS_CHANGE_32MSI_FN, nvec);
> if (rc < 0) {
> /*
> @@ -409,7 +409,7 @@ static int rtas_prepare_msi_irqs(struct pci_dev *pdev, int nvec_in, int type,
> if (use_32bit_msi_hack && rc > 0)
> rtas_hack_32bit_msi_gen2(pdev);
> } else {
> - if (pdev->no_64bit_msi)
> + if (pdev->msi_addr_mask < DMA_BIT_MASK(64))
> rc = rtas_change_msi(pdn, RTAS_CHANGE_32MSIX_FN, nvec);
> else
> rc = rtas_change_msi(pdn, RTAS_CHANGE_MSIX_FN, nvec);
> diff --git a/drivers/gpu/drm/radeon/radeon_irq_kms.c b/drivers/gpu/drm/radeon/radeon_irq_kms.c
> index 9961251b44ba..d550554a6f3f 100644
> --- a/drivers/gpu/drm/radeon/radeon_irq_kms.c
> +++ b/drivers/gpu/drm/radeon/radeon_irq_kms.c
> @@ -252,7 +252,7 @@ static bool radeon_msi_ok(struct radeon_device *rdev)
> */
> if (rdev->family < CHIP_BONAIRE) {
> dev_info(rdev->dev, "radeon: MSI limited to 32-bit\n");
> - rdev->pdev->no_64bit_msi = 1;
> + rdev->pdev->msi_addr_mask = DMA_BIT_MASK(32);
> }
>
> /* force MSI on */
> diff --git a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
> index 70d86c5f52fb..0671deae9a28 100644
> --- a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
> +++ b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
> @@ -331,7 +331,7 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
>
> #ifdef CONFIG_PPC64
> /* Ensure MSI/MSI-X interrupts lie within addressable physical memory */
> - pdev->no_64bit_msi = 1;
> + pdev->msi_addr_mask = DMA_BIT_MASK(32);
> #endif
>
> err = ionic_setup_one(ionic);
> diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c
> index 34d664139f48..48f5f03d1479 100644
> --- a/drivers/pci/msi/msi.c
> +++ b/drivers/pci/msi/msi.c
> @@ -322,7 +322,7 @@ static int msi_verify_entries(struct pci_dev *dev)
> {
> struct msi_desc *entry;
>
> - if (!dev->no_64bit_msi)
> + if (dev->msi_addr_mask == DMA_BIT_MASK(64))
> return 0;
>
> msi_for_each_desc(entry, &dev->dev, MSI_DESC_ALL) {
> diff --git a/drivers/pci/msi/pcidev_msi.c b/drivers/pci/msi/pcidev_msi.c
> index 5520aff53b56..0b0346813092 100644
> --- a/drivers/pci/msi/pcidev_msi.c
> +++ b/drivers/pci/msi/pcidev_msi.c
> @@ -24,7 +24,7 @@ void pci_msi_init(struct pci_dev *dev)
> }
>
> if (!(ctrl & PCI_MSI_FLAGS_64BIT))
> - dev->no_64bit_msi = 1;
> + dev->msi_addr_mask = DMA_BIT_MASK(32);
> }
>
> void pci_msix_init(struct pci_dev *dev)
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 41183aed8f5d..a2bff57176a3 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -2047,6 +2047,13 @@ int pci_setup_device(struct pci_dev *dev)
> */
> dev->dma_mask = 0xffffffff;
>
> + /*
> + * Assume 64-bit addresses for MSI initially. Will be changed to 32-bit
> + * if MSI (rather than MSI-X) capability does not have
> + * PCI_MSI_FLAGS_64BIT. Can also be overridden by driver.
> + */
> + dev->msi_addr_mask = DMA_BIT_MASK(64);
> +
> dev_set_name(&dev->dev, "%04x:%02x:%02x.%d", pci_domain_nr(dev->bus),
> dev->bus->number, PCI_SLOT(dev->devfn),
> PCI_FUNC(dev->devfn));
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 864775651c6f..0fe32fef0331 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -377,6 +377,13 @@ struct pci_dev {
> 0xffffffff. You only need to change
> this if your device has broken DMA
> or supports 64-bit transfers. */
> + u64 msi_addr_mask; /* Mask of the bits of bus address for
> + MSI that this device implements.
> + Normally set based on device
> + capabilities. You only need to
> + change this if your device claims
> + to support 64-bit MSI but implements
> + fewer than 64 address bits. */
>
> struct device_dma_parameters dma_parms;
>
> @@ -441,7 +448,6 @@ struct pci_dev {
>
> unsigned int is_busmaster:1; /* Is busmaster */
> unsigned int no_msi:1; /* May not use MSI */
> - unsigned int no_64bit_msi:1; /* May only use 32-bit MSIs */
> unsigned int block_cfg_access:1; /* Config space access blocked */
> unsigned int broken_parity_status:1; /* Generates false positive parity */
> unsigned int irq_reroute_variant:2; /* Needs IRQ rerouting variant */
> diff --git a/sound/hda/controllers/intel.c b/sound/hda/controllers/intel.c
> index 1e8e3d61291a..c9542ebdf7e2 100644
> --- a/sound/hda/controllers/intel.c
> +++ b/sound/hda/controllers/intel.c
> @@ -1905,7 +1905,7 @@ static int azx_first_init(struct azx *chip)
>
> if (chip->msi && chip->driver_caps & AZX_DCAPS_NO_MSI64) {
> dev_dbg(card->dev, "Disabling 64bit MSI\n");
> - pci->no_64bit_msi = true;
> + pci->msi_addr_mask = DMA_BIT_MASK(32);
> }
>
> pci_set_master(pci);
>