Re: [patch 1/2] irqchip/msi-lib: Honor the MSI_FLAG_PCI_MSI_MASK_PARENT flag

[Marc Zyngier <maz@kernel.org>]

On Sat, 20 Dec 2025 19:31:19 +0000,
Luigi Rizzo <lrizzo@google.com> wrote:
> 
> There are platforms (including some ARM SoC) where the MSIx
> writes are a performance killer, because they are exceedingly
> serializing on the PCIe root port.
> 
> These platforms are the key motivation for Global Software
> Interrupt Moderation (GSIM) which relies on actually masking
> device interrupts so the MSIx writes are not generated.
> https://lore.kernel.org/all/20251217112128.1401896-1-lrizzo@google.com/
> 
> Overriding mask/unmask with irq_chip_mask_parent() makes software
> moderation ineffective. GSIM works great on ARM platforms before
> this patch, but becomes ineffective afterwards, e.g. on linux 6.18.

You do realise that "ARM platforms" means nothing at all, right? What
you actually mean is "the ARM machines I have access to exhibit some
platform-specific behaviour that may or may not be a general
behaviour".

Your particular circumstances are not in any way something you can
generalise, unless you demonstrate this is caused by an architectural
requirement rather than an implementation defect.

> The round trip through the PCI endpoint for mask_irq(), caused by the
> readback to make sure the PCI write has been sent, is almost always
> (or really always) unnecessary.  Masking is inherently racy; waiting
> that the PCIe write has arrived at the device won't guarantee that an
> interrupt has arrived in the meantime, so there is really no benefit
> in the readback (which, for instance, can be conditionally removed with
> code like the one below).
> 
> I measured the cost of pci_irq_mask_msix() and it goes from 1000-1500ns
> with the readl(), down to 40-50ns without it.
> 
> Once we remove the costly readback, is there any remaining reason
> to overwrite [un]mask_irq() with irq_chip_[un]mask_parent() ?

So you are effectively not masking at all and just rely on hope
instead. I have the utmost confidence in this sort of stuff. Totally.

What you missing is that hitting the config space is causing pretty
high overhead in KVM guests, where the accesses (write and read to the
MSI masks) are trapped all the way to userspace (and back into VFIO),
while the masking at the ITS level is much cheaper.

Masking at the ITS level (and only there) also means that the VM can
be migrated without having to worry about the PBA in each device,
because the pending state is already part of the VM's memory, nicely
tucked away in the RD tables.

Finally, it aligns PCI devices with non-PCI device behaviour,
something that is highly desirable.

For me, that totally beats your interrupt mitigation thing.

Thanks,

	M.

-- 
Jazz isn't dead. It just smells funny.