[PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[Sumit Kumar]

An outbound iATU region matches any local (AXI/application-side)
address that falls within its Base/Limit window and rewrites it to
the PCIe-side address programmed in that region's Target Address
Registers before the transaction goes out on the link. This matching
does not distinguish CPU-initiated transactions from ones issued by
the internal DMA engine (eDMA/HDMA).

Example: outbound region 0 is programmed with Base/Limit covering
local addresses 0x8000_0000-0x8000_0FFF and Target 0x1_0000_0000, for
CPU-initiated accesses. The DMA engine is later given a descriptor
whose destination is the host address 0x8000_0080, which happens to
fall inside region 0's Base/Limit range. The iATU matches the DMA
transaction against region 0 too and rewrites it to 0x1_0000_0080
instead of passing 0x8000_0080 through unchanged. The transaction
then lands on the wrong host address, causing data corruption or an
IOMMU fault when the rewritten address has no valid mapping.

Set the DMA_BYPASS bit (bit 27) in PCIE_ATU_REGION_CTRL2 when
programming outbound iATU regions so only CPU/AXI-initiated
transactions are matched against the region; DMA engine transactions
bypass the translation and go out on the PCIe link with their
original address.

Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
---
pci: designware: Set DMA_BYPASS in outbound IATU to prevent HDMA
interception
---
Changes in v2:
- Rewrote the commit message according to suggestion(Mani).
- Link to v1: https://lore.kernel.org/r/20260826-dma_bypass-v1-1-3a53dc748ce2@oss.qualcomm.com
---
 drivers/pci/controller/dwc/pcie-designware.c | 2 ++
 drivers/pci/controller/dwc/pcie-designware.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 593388f29bdd48381cb04710f445042aaf2be29f..9297d0e607e3a1caaeb585b23742f9a29607b7f2 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -577,6 +577,8 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
 		/* The data-less messages only for now */
 		val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code;
 	}
+	if (dw_pcie_ver_is_ge(pci, 460A))
+		val |= PCIE_ATU_DMA_BYPASS;
 	dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val);
 
 	/*
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 0735ae94092404bc2976af67a58f6842cc5e5c50..668a5ba1c6f59aef1aaf15566b77a649bd282ce0 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -201,6 +201,7 @@
 #define PCIE_ATU_ENABLE			BIT(31)
 #define PCIE_ATU_BAR_MODE_ENABLE	BIT(30)
 #define PCIE_ATU_CFG_SHIFT_MODE_ENABLE	BIT(28)
+#define PCIE_ATU_DMA_BYPASS		BIT(27)
 #define PCIE_ATU_INHIBIT_PAYLOAD	BIT(22)
 #define PCIE_ATU_FUNC_NUM_MATCH_EN      BIT(19)
 #define PCIE_ATU_LOWER_BASE		0x008

---
base-commit: 45c13f3f9e3bb15fd89ff2864c6f627a3b4b4229
change-id: 20260826-dma_bypass-bbbf8a0c9af7

Best regards,
-- 
Sumit Kumar <sumit.kumar@oss.qualcomm.com>

Re: [PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[sashiko-bot]

> An outbound iATU region matches any local (AXI/application-side)
> address that falls within its Base/Limit window and rewrites it to
> the PCIe-side address programmed in that region's Target Address
> Registers before the transaction goes out on the link. This matching
> does not distinguish CPU-initiated transactions from ones issued by
> the internal DMA engine (eDMA/HDMA).
> 
> Example: outbound region 0 is programmed with Base/Limit covering
> local addresses 0x8000_0000-0x8000_0FFF and Target 0x1_0000_0000, for
> CPU-initiated accesses. The DMA engine is later given a descriptor
> whose destination is the host address 0x8000_0080, which happens to
> fall inside region 0's Base/Limit range. The iATU matches the DMA
> transaction against region 0 too and rewrites it to 0x1_0000_0080
> instead of passing 0x8000_0080 through unchanged. The transaction
> then lands on the wrong host address, causing data corruption or an
> IOMMU fault when the rewritten address has no valid mapping.
> 
> Set the DMA_BYPASS bit (bit 27) in PCIE_ATU_REGION_CTRL2 when
> programming outbound iATU regions so only CPU/AXI-initiated
> transactions are matched against the region; DMA engine transactions
> bypass the translation and go out on the PCIe link with their
> original address.
> 
> Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260908-dma_bypass-v2-1-c3c81d408c0c@oss.qualcomm.com?part=1

Re: [PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[Niklas Cassel]

Hello Sumit,

On Tue, Sep 08, 2026 at 11:48:22AM +0530, Sumit Kumar wrote:
> An outbound iATU region matches any local (AXI/application-side)
> address that falls within its Base/Limit window and rewrites it to
> the PCIe-side address programmed in that region's Target Address
> Registers before the transaction goes out on the link. This matching
> does not distinguish CPU-initiated transactions from ones issued by
> the internal DMA engine (eDMA/HDMA).
> 
> Example: outbound region 0 is programmed with Base/Limit covering
> local addresses 0x8000_0000-0x8000_0FFF and Target 0x1_0000_0000, for
> CPU-initiated accesses. The DMA engine is later given a descriptor
> whose destination is the host address 0x8000_0080, which happens to
> fall inside region 0's Base/Limit range. The iATU matches the DMA
> transaction against region 0 too and rewrites it to 0x1_0000_0080
> instead of passing 0x8000_0080 through unchanged. The transaction
> then lands on the wrong host address, causing data corruption or an
> IOMMU fault when the rewritten address has no valid mapping.
> 
> Set the DMA_BYPASS bit (bit 27) in PCIE_ATU_REGION_CTRL2 when
> programming outbound iATU regions so only CPU/AXI-initiated
> transactions are matched against the region; DMA engine transactions
> bypass the translation and go out on the PCIe link with their
> original address.
> 
> Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>

If you look at e.g.
drivers/pci/endpoint/functions/pci-epf-test.c:pci_epf_test_read()

you can see that it does call pci_epc_mem_map(), which will end up
as a call to dw_pcie_prog_outbound_atu() to setup an iATU mapping
on DWC based PCIe controllers.


If you configure DWC based controllers to bypass the iATU, this
call to pci_epc_mem_map() is completely useless.


Perhaps the DWC driver can set a DMA_SLAVE_SKIP_MEM_MAP flag
or similar when registering the eDMA, which pci-epf-test then could check,
and not call pci_epc_mem_map() if DMA_SLAVE_SKIP_MEM_MAP is set.


Because, while I can read in the databook that this is not needed
for DWC based controllers, I have no idea if the pci_epc_mem_map()
is still needed for e.g.:
drivers/pci/controller/cadence/pcie-cadence-ep.c
drivers/pci/controller/pcie-rcar-ep.c
drivers/pci/controller/pcie-rockchip-ep.c


Kind regards,
Niklas

Re: [PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[Manivannan Sadhasivam]

On Tue, Sep 08, 2026 at 05:08:26PM +0200, Niklas Cassel wrote:
> Hello Sumit,
> 
> On Tue, Sep 08, 2026 at 11:48:22AM +0530, Sumit Kumar wrote:
> > An outbound iATU region matches any local (AXI/application-side)
> > address that falls within its Base/Limit window and rewrites it to
> > the PCIe-side address programmed in that region's Target Address
> > Registers before the transaction goes out on the link. This matching
> > does not distinguish CPU-initiated transactions from ones issued by
> > the internal DMA engine (eDMA/HDMA).
> > 
> > Example: outbound region 0 is programmed with Base/Limit covering
> > local addresses 0x8000_0000-0x8000_0FFF and Target 0x1_0000_0000, for
> > CPU-initiated accesses. The DMA engine is later given a descriptor
> > whose destination is the host address 0x8000_0080, which happens to
> > fall inside region 0's Base/Limit range. The iATU matches the DMA
> > transaction against region 0 too and rewrites it to 0x1_0000_0080
> > instead of passing 0x8000_0080 through unchanged. The transaction
> > then lands on the wrong host address, causing data corruption or an
> > IOMMU fault when the rewritten address has no valid mapping.
> > 
> > Set the DMA_BYPASS bit (bit 27) in PCIE_ATU_REGION_CTRL2 when
> > programming outbound iATU regions so only CPU/AXI-initiated
> > transactions are matched against the region; DMA engine transactions
> > bypass the translation and go out on the PCIe link with their
> > original address.
> > 
> > Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
> 
> If you look at e.g.
> drivers/pci/endpoint/functions/pci-epf-test.c:pci_epf_test_read()
> 
> you can see that it does call pci_epc_mem_map(), which will end up
> as a call to dw_pcie_prog_outbound_atu() to setup an iATU mapping
> on DWC based PCIe controllers.
> 

pci_epc_mem_map() is already useless in the eDMA path today. So this patch
doesn't change that.

> 
> If you configure DWC based controllers to bypass the iATU, this
> call to pci_epc_mem_map() is completely useless.
> 
> 
> Perhaps the DWC driver can set a DMA_SLAVE_SKIP_MEM_MAP flag
> or similar when registering the eDMA, which pci-epf-test then could check,
> and not call pci_epc_mem_map() if DMA_SLAVE_SKIP_MEM_MAP is set.
> 
> 
> Because, while I can read in the databook that this is not needed
> for DWC based controllers, I have no idea if the pci_epc_mem_map()
> is still needed for e.g.:
> drivers/pci/controller/cadence/pcie-cadence-ep.c
> drivers/pci/controller/pcie-rcar-ep.c
> drivers/pci/controller/pcie-rockchip-ep.c
> 

Fair point. But this should be done as a separate improvement, not related to
this patch.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[Niklas Cassel]

On Sun, Sep 13, 2026 at 11:35:59AM +0200, Manivannan Sadhasivam wrote:
> 
> pci_epc_mem_map() is already useless in the eDMA path today. So this patch
> doesn't change that.

I'm not sure if useless is the proper word here.
Redundant is perhaps a better word.


Let me explain using pci-epf-test.c:pci_epf_test_read() as an example:

Without the patch in $subject, on a DWC based platform:

pci_epc_mem_map() will setup an iATU entry, and the eDMA transaction will then
be translated using that iATU entry, before being sent out on the PCIe bus.

So the iATU entry is actually being used.


After the patch in $subject, on a DWC based platform:

pci_epc_mem_map() will setup an iATU entry, however, because of the DMA_BYPASS
bit, that iATU entry will not be used, and the transaction will be send out on
the PCIe bus untranslated.


So I would argue that it is only after this patch that pci_epc_mem_map() will
setup an iATU entry that will then never be used.

Yes, I think I understand that you are trying to say that, for DWC based
drivers, we could theoretically skip the pci_epc_mem_map() call, and send the
transaction untranslated.

My point is that it feels silly to set the DMA_BYPASS bit without also making
sure that the pci_epc_mem_map() calls are either removed or (at least for DWC
based platforms) modified to not set up an iATU entry in the DMA path in the
first place.

I am happy that this is getting cleaned up, I just don't like a half cleanup
where we, after this commit, will setup+teardown iATU entries that will then
never be used for the actual transaction.


Kind regards,
Niklas

Re: [PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[Manivannan Sadhasivam]

On 2026-09-13 19:16 +0200, Niklas Cassel wrote:
> On Sun, Sep 13, 2026 at 11:35:59AM +0200, Manivannan Sadhasivam wrote:
> > 
> > pci_epc_mem_map() is already useless in the eDMA path today. So this patch
> > doesn't change that.
> 
> I'm not sure if useless is the proper word here.
> Redundant is perhaps a better word.
> 

No, it is useless indeed. See below.

> 
> Let me explain using pci-epf-test.c:pci_epf_test_read() as an example:
> 
> Without the patch in $subject, on a DWC based platform:
> 
> pci_epc_mem_map() will setup an iATU entry, and the eDMA transaction will then
> be translated using that iATU entry, before being sent out on the PCIe bus.
> 

No, that's not true. While the pci_epc_mem_map() does setup an iATU entry, eDMA
transaction doesn't use the mapped address.

If you look into the eDMA read path:
	pci_epc_mem_map()
		-> Allocates map.phys_addr and sets up an iATU entry
	pci_epf_test_data_transfer()
		-> dma_local points to address allocated by dma_map_single()
		   (dma_dst)
		-> dma_remote points to RC address referenced in src_addr
		-> These two were passed to dmaengine_prep_config_single()
		   ignoring the map.phys_addr passed through dma_src parameter.

For eDMA write:
	pci_epc_mem_map()
		-> Allocates map.phys_addr and sets up an iATU entry
	pci_epf_test_data_transfer()
		-> dma_local points to address allocated by dma_map_single()
		   (dma_src)
		-> dma_remote points to RC address referenced in dst_addr
		-> These two were passed to dmaengine_prep_config_single()
		   ignoring the map.phys_addr passed through dma_dst parameter.

So essentially, the address used by the eDMA is the raw RC PCIe address, not
translated by the iATU. In fact, the Designware Databook also doesn't recommend
setting up the iATU entry for the eDMA path. So the allocated iATU entry is
useless.

But the issue this patch is trying to address is, if the raw PCI address or the
dma_map_single address used by the eDMA happens to be present in one of the
existing iATU mappings, then at that point, the address can get translated
causing wrong address to be sent over the bus.

> So the iATU entry is actually being used.
> 
> 
> After the patch in $subject, on a DWC based platform:
> 
> pci_epc_mem_map() will setup an iATU entry, however, because of the DMA_BYPASS
> bit, that iATU entry will not be used, and the transaction will be send out on
> the PCIe bus untranslated.
> 

This is the current behaviour as well.

> 
> So I would argue that it is only after this patch that pci_epc_mem_map() will
> setup an iATU entry that will then never be used.
> 
> Yes, I think I understand that you are trying to say that, for DWC based
> drivers, we could theoretically skip the pci_epc_mem_map() call, and send the
> transaction untranslated.
> 
> My point is that it feels silly to set the DMA_BYPASS bit without also making
> sure that the pci_epc_mem_map() calls are either removed or (at least for DWC
> based platforms) modified to not set up an iATU entry in the DMA path in the
> first place.
> 

This I agree. But this also needs some careful analysis as other DMA controllers
other than eDMA might need the mapping. For eDMA, it is also not straightforward
as we do not pass any 'I'm using DMA' flag to map_addr() callback.

> I am happy that this is getting cleaned up, I just don't like a half cleanup
> where we, after this commit, will setup+teardown iATU entries that will then
> never be used for the actual transaction.
> 

This should be done as a follow-up, not blocking this patch.

- Mani

Re: [PATCH v2] pci: designware: Set DMA_BYPASS bit in outbound IATU CTRL2 register

[Niklas Cassel]

On Tue, Oct 13, 2026 at 10:48:37PM +0530, Manivannan Sadhasivam wrote:
> On 2026-09-13 19:16 +0200, Niklas Cassel wrote:
> > 
> > Without the patch in $subject, on a DWC based platform:
> > 
> > pci_epc_mem_map() will setup an iATU entry, and the eDMA transaction will then
> > be translated using that iATU entry, before being sent out on the PCIe bus.
> > 
> 
> No, that's not true. While the pci_epc_mem_map() does setup an iATU entry, eDMA
> transaction doesn't use the mapped address.
> 
> If you look into the eDMA read path:
> 	pci_epc_mem_map()
> 		-> Allocates map.phys_addr and sets up an iATU entry
> 	pci_epf_test_data_transfer()
> 		-> dma_local points to address allocated by dma_map_single()
> 		   (dma_dst)
> 		-> dma_remote points to RC address referenced in src_addr
> 		-> These two were passed to dmaengine_prep_config_single()
> 		   ignoring the map.phys_addr passed through dma_src parameter.
> 
> For eDMA write:
> 	pci_epc_mem_map()
> 		-> Allocates map.phys_addr and sets up an iATU entry
> 	pci_epf_test_data_transfer()
> 		-> dma_local points to address allocated by dma_map_single()
> 		   (dma_src)
> 		-> dma_remote points to RC address referenced in dst_addr
> 		-> These two were passed to dmaengine_prep_config_single()
> 		   ignoring the map.phys_addr passed through dma_dst parameter.
> 
> So essentially, the address used by the eDMA is the raw RC PCIe address, not
> translated by the iATU. In fact, the Designware Databook also doesn't recommend
> setting up the iATU entry for the eDMA path. So the allocated iATU entry is
> useless.

I see, you are correct.
The pci-epf-test.c driver is very misleading, because if does:

    ...
    pci_epc_mem_map();

    if (flags & FLAG_USE_DMA) {
        dma_map_single();
        pci_epf_test_data_transfer();
    } else {
        memcpy_fromio();
    }

    pci_epc_mem_unmap();


This suggests to the reader that the mapping is used in both cases...
Why else would the mapping be done unconditionally?

I strongly suggest that the pci_epc_mem_map() and pci_epc_mem_unmap()
calls are moved into the "else" clause then.

Since this patch is already touching behavior related to this, it would be
nice if the patch author could also write a patch that cleans up this mess.


> 
> But the issue this patch is trying to address is, if the raw PCI address or the
> dma_map_single address used by the eDMA happens to be present in one of the
> existing iATU mappings, then at that point, the address can get translated
> causing wrong address to be sent over the bus.

I do understand the problem that this patch is trying to solve.


> > I am happy that this is getting cleaned up, I just don't like a half cleanup
> > where we, after this commit, will setup+teardown iATU entries that will then
> > never be used for the actual transaction.
> > 
> 
> This should be done as a follow-up, not blocking this patch.

Fine with me.


Kind regards,
Niklas