Re: [PATCH] PCI: dwc: ep: Flush before unmap in dw_pcie_ep_raise_msix_irq()

[Bjorn Helgaas <helgaas@kernel.org>]

On Wed, Feb 11, 2026 at 06:55:41PM +0100, Niklas Cassel wrote:
> When running e.g. fio with a larger queue depth against nvmet-pci-epf we
> get IOMMU errors on the host, e.g.:
> 
> arm-smmu-v3 fc900000.iommu:      0x0000010000000010
> arm-smmu-v3 fc900000.iommu:      0x0000020000000000
> arm-smmu-v3 fc900000.iommu:      0x000000090000f040
> arm-smmu-v3 fc900000.iommu:      0x0000000000000000
> arm-smmu-v3 fc900000.iommu: event: F_TRANSLATION client: 0000:01:00.0 sid: 0x100 ssid: 0x0 iova: 0x90000f040 ipa: 0x0
> arm-smmu-v3 fc900000.iommu: unpriv data write s1 "Input address caused fault" stag: 0x0
> 
> The reason for this is that the writel() is immediately followed by a call
> to unmap(), which will tear down the outbound address translation.
> 
> PCI writes are posted, i.e. don't wait for a completion. Thus, when the
> writel() returns, might not have completed yet, and could even still be
> buffered in the PCI bridge, at the time unmap() is called.
> 
> Flush the write by performing a read() of the same address, to ensure that
> the write has reached the destination before calling unmap().
> 
> This will add some latency, but that is certainly preferred over corrupting
> the host memory.
> 
> The same problem was solved for dw_pcie_ep_raise_msi_irq(), in commit
> 8719c64e76bf ("PCI: dwc: ep: Cache MSI outbound iATU mapping"), however
> there it was solved by dedicating an outbound iATU only for MSI. For MSI-X,
> we can't do the same, as each vector can have a different msg_addr, and
> because the msg_addr is allowed to be changed while the vector is masked.
> 
> Fixes: beb4641a787d ("PCI: dwc: Add MSI-X callbacks handler")
> Signed-off-by: Niklas Cassel <cassel@kernel.org>

beb4641a787d appeared in v4.19 (2018!) so it doesn't strictly qualify
as a post-merge window fix, but I do understand that it fixes a
problem similar to the 8719c64e76bf bug that we added in v7.0.

I don't think 8719c64e76bf and its fix make it any more likely that
we'll hit *this* problem, but it's certainly a trivial low-risk
change.

I put this on pci/for-linus for v7.0, thanks!

> ---
>  drivers/pci/controller/dwc/pcie-designware-ep.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 5d8024d5e5c6..aef41f0218a3 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -1005,6 +1005,9 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
>  
>  	writel(msg_data, ep->msi_mem + offset);
>  
> +	/* flush posted write before unmap */
> +	readl(ep->msi_mem + offset);
> +
>  	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
>  
>  	return 0;
> -- 
> 2.53.0
>