[PATCH v2 0/2] PCI: dwc: Support large BARs on UltraRISC DP1000

[PATCH v2 0/2] PCI: dwc: Support large BARs on UltraRISC DP1000

[Xincheng Zhang]

This series lets DWC host drivers skip generic outbound iATU programming
for host bridge MEM ranges, which is useful for some compute cards' very
large BAR size demand (>100GB).

DP1000 has 16 outbound iATU windows, and each window can map up to
4GB. This limits its allocatable MEM64 BAR size to 4GB*(16-3)=52GB
(-3 for CFG, I/O and MEM32).

But some EPs like compute cards demand large BAR size that exceeds the
mapping limit of the 13 iATUs. DWC IP spec allows RC to skip MEM iATU setup
and simply uses a direct mapping. By introducing a `bypass_ob_mem_iatu`
flag at the RC level, this patch enables large BARs on UR-DP1000
RISC-V CPU.

Tested on UltraRISC DP1000 EVB and Titan board. The system booted
successfully with PCIe links up on all three controllers and NVMe rootfs
mounted. MMIO read/write speed test within the large BAR space goes well.

Changes in v2:
- Validate skipped MEM ranges are identity-mapped before bypassing outbound
  iATU setup.
- Document the DWC PCIe RP Controller Databook v6.00a no-match pass-through
  behavior.
- Clarify that the DP1000 bypass is MEM-only; CFG and I/O iATU setup is
  unchanged.

Link: https://lore.kernel.org/r/20260827055448.1818858-1-zhangxincheng@ultrarisc.com

Xincheng Zhang (2):
  PCI: dwc: Allow drivers to skip MEM outbound iATU setup
  PCI: ultrarisc: Skip MEM outbound iATU setup on DP1000

 .../pci/controller/dwc/pcie-designware-host.c   | 17 +++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.h    |  1 +
 drivers/pci/controller/dwc/pcie-ultrarisc.c     |  6 ++++++
 3 files changed, 22 insertions(+), 2 deletions(-)

-- 
2.34.1

[PATCH v2 1/2] PCI: dwc: Allow drivers to skip MEM outbound iATU setup

[Xincheng Zhang]

Some DWC-based controllers do not require outbound iATU windows for MEM
transactions. For those platforms, programming outbound iATU windows for
host bridge MEM ranges is unnecessary and may constrain the available MEM
aperture.

The DWC PCIe RP Controller Databook v6.00a, section 3.12.2, describes the
default outbound no-match behavior as pass-through. Section 3.12.5.6, "No
Address Match Result", says the address is not translated and programmable
TLP header fields come from the application transmit interface.

This no-match path cannot translate between different parent-bus and PCI
addresses, so require skipped MEM ranges to be identity-mapped.

Add dw_pcie_rp::bypass_ob_mem_iatu so drivers can skip generic MEM iATU
setup while using the common DWC host init path.

Existing drivers keep the current behavior because the flag defaults to
false. The I/O iATU setup does not use the MEM resource iterator, and the
iterator may be uninitialized when MEM setup is skipped. Avoid using it in
the I/O iATU error path.

Signed-off-by: Xincheng Zhang <zhangxincheng@ultrarisc.com>
---
 .../pci/controller/dwc/pcie-designware-host.c   | 17 +++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.h    |  1 +
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index f5a38e6fd8d79..b06797b4e31d2 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -918,6 +918,20 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 		atu.parent_bus_addr = entry->res->start - pci->parent_bus_offset;
 		atu.pci_addr = entry->res->start - entry->offset;
 
+		if (pp->bypass_ob_mem_iatu) {
+			/*
+			 * The no-match pass-through path preserves the correct
+			 * PCI address only for identity-mapped MEM ranges.
+			 */
+			if (atu.parent_bus_addr != atu.pci_addr) {
+				dev_err(pci->dev,
+					"Cannot bypass outbound iATU for non-identity MEM range %pr\n",
+					entry->res);
+				return -EINVAL;
+			}
+			continue;
+		}
+
 		/* Adjust iATU size if MSG TLP region was allocated before */
 		if (pp->msg_res && pp->msg_res->parent == entry->res)
 			res_size = resource_size(entry->res) -
@@ -964,8 +978,7 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 
 			ret = dw_pcie_prog_outbound_atu(pci, &atu);
 			if (ret) {
-				dev_err(pci->dev, "Failed to set IO range %pr\n",
-					entry->res);
+				dev_err(pci->dev, "Failed to set IO range\n");
 				return ret;
 			}
 			ob_iatu_index++;
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 0735ae9409240..d89badcca2689 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -469,6 +469,7 @@ struct dw_pcie_rp {
 	struct pci_config_window *cfg;
 	bool			ecam_enabled;
 	bool			native_ecam;
+	bool			bypass_ob_mem_iatu;
 	bool                    skip_l23_ready;
 	bool			skip_pwrctrl_off;
 };
-- 
2.34.1

[PATCH v2 2/2] PCI: ultrarisc: Skip MEM outbound iATU setup on DP1000

[Xincheng Zhang]

DP1000 has 16 outbound iATU windows and each window can map up to 4GB. One
window is used for CFG accesses, one for I/O space and one for the 32-bit
MEM aperture, leaving only 13 windows for the 64-bit MEM aperture.

This limits the 64-bit MEM aperture that can be described with outbound
iATU windows. DP1000 routes MEM transactions without using outbound iATU
windows, so skip generic DWC MEM outbound iATU setup to allow a larger
64-bit MEM aperture.

DP1000 does not use ECAM for this controller. Config space accesses keep
using the normal DWC config access path and CFG outbound iATU programming.
I/O space is also unchanged and still uses an outbound iATU window. The new
flag only skips generic MEM-window programming.

The DP1000 integration has SoC wrapper address decode for the PCIe MEM
apertures described by the "ranges" property. CPU physical accesses to
those MEM apertures are routed directly as PCIe MEM transactions with the
corresponding PCIe bus address, without consuming outbound iATU windows.

Set dw_pcie_rp::bypass_ob_mem_iatu for this controller.

Signed-off-by: Xincheng Zhang <zhangxincheng@ultrarisc.com>
---
 drivers/pci/controller/dwc/pcie-ultrarisc.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-ultrarisc.c b/drivers/pci/controller/dwc/pcie-ultrarisc.c
index 51e89180490d4..0d4989a0f3afc 100644
--- a/drivers/pci/controller/dwc/pcie-ultrarisc.c
+++ b/drivers/pci/controller/dwc/pcie-ultrarisc.c
@@ -154,6 +154,12 @@ static int ultrarisc_pcie_probe(struct platform_device *pdev)
 
 	pp = &pci->pp;
 
+	/*
+	 * DP1000 routes MEM transactions without outbound iATU windows, which
+	 * avoids the 16-window/4GB-per-window aperture limit.
+	 */
+	pp->bypass_ob_mem_iatu = true;
+
 	platform_set_drvdata(pdev, ultra);
 
 	ret = ultrarisc_pcie_init_clks(ultra);
-- 
2.34.1

Re: [PATCH v2 1/2] PCI: dwc: Allow drivers to skip MEM outbound iATU setup

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Callers of dw_pcie_setup_rc() ignore its return value, missing potential errors.
--

commit b48f3b5259744f25f4f7ea094bd8ed4860783aa6
Author: Xincheng Zhang <zhangxincheng@ultrarisc.com>

PCI: dwc: Allow drivers to skip MEM outbound iATU setup

This commit adds a flag to bypass outbound iATU programming for
identity-mapped MEM ranges on DWC controllers. It introduces a check
that returns -EINVAL if a skipped MEM range is not identity-mapped.

> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -918,6 +918,20 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
>  		atu.parent_bus_addr = entry->res->start - pci->parent_bus_offset;
>  		atu.pci_addr = entry->res->start - entry->offset;
>  
> +		if (pp->bypass_ob_mem_iatu) {
> +			/*
> +			 * The no-match pass-through path preserves the correct
> +			 * PCI address only for identity-mapped MEM ranges.
> +			 */
> +			if (atu.parent_bus_addr != atu.pci_addr) {
> +				dev_err(pci->dev,
> +					"Cannot bypass outbound iATU for non-identity MEM range %pr\n",
> +					entry->res);
> +				return -EINVAL;

[Severity: High]
This isn't a bug introduced by this patch, but adding this new failure path
highlights that callers of dw_pcie_setup_rc() (which calls
dw_pcie_iatu_setup()) ignore its return value.

For example, in drivers/pci/controller/dwc/pcie-designware-host.c during
system resume:

dw_pcie_resume_noirq() {
    ...
    dw_pcie_setup_rc(&pci->pp);
    ...
}

The core suspend/resume framework ignores the setup return value here.
The same pattern exists in driver-specific functions like
exynos_pcie_resume_noirq(), tegra_pcie_dw_resume_noirq(), and
tegra_pcie_dw_start_link().

If dw_pcie_setup_rc() fails because of an invalid ATU configuration
(like this new -EINVAL path) or an iATU enablement timeout, the controller
proceeds with link initialization without proper configuration. Missing
outbound memory windows or broken MSI can result in system crashes or silent
I/O failures when accessing PCI memory.

Could the callers of dw_pcie_setup_rc() be updated to check and handle
these failures appropriately?

> +			}
> +			continue;
> +		}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260905025635.211390-1-zhangxincheng@ultrarisc.com?part=1

Re: [PATCH v2 2/2] PCI: ultrarisc: Skip MEM outbound iATU setup on DP1000

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing `.remove` callback in platform driver causes memory leaks, use-after-free, and unbalanced clocks on module unload.
- [High] Ignored return value of `dw_pcie_setup_rc()` during resume leads to unconditionally accessing uninitialized hardware.
- [High] Missing `dw_pcie_dbi_ro_wr_dis()` on the error path leaves DBI read-only registers writable.
--

commit c554fce09754058c518d7827f376e2e64ea19f5a
Author: Xincheng Zhang <zhangxincheng@ultrarisc.com>

PCI: ultrarisc: Skip MEM outbound iATU setup on DP1000

This patch sets the bypass_ob_mem_iatu flag for the DP1000 controller to
skip generic DWC MEM outbound iATU setup. This avoids the 16-window limit
and allows for a larger 64-bit MEM aperture.

> diff --git a/drivers/pci/controller/dwc/pcie-ultrarisc.c b/drivers/pci/controller/dwc/pcie-ultrarisc.c
> --- a/drivers/pci/controller/dwc/pcie-ultrarisc.c
> +++ b/drivers/pci/controller/dwc/pcie-ultrarisc.c
[ ... ]
>  static struct platform_driver ultrarisc_pcie_driver = {
>  	.driver = {
>  		.name	= "ultrarisc-pcie",
>  		.of_match_table = ultrarisc_pcie_of_match,
>  		.suppress_bind_attrs = true,
>  		.pm = &ultrarisc_pcie_pm_ops,
>  	},
>  	.probe = ultrarisc_pcie_probe,
>  };
>  module_platform_driver(ultrarisc_pcie_driver);

[Severity: High]
This isn't a bug introduced by this patch, but is it safe to leave out
the .remove callback in the platform driver?

Since dw_pcie_host_init() is not devres-managed, unloading the module 
triggers devres release which frees driver memory and disables clocks, but
leaves the PCI host bridge and devices registered. Could this cause a
use-after-free and hardware hang if those devices are accessed later?

> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
[ ... ]
>  	if (pp->bridge->child_ops == &dw_child_pcie_ops || pp->ecam_enabled) {
>  		ret = dw_pcie_iatu_setup(pp);
>  		if (ret)
>  			return ret;
>  	}
>  
>  	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
[ ... ]
>  	dw_pcie_dbi_ro_wr_dis(pci);

[Severity: High]
This is a pre-existing issue, but does this error path bypass restoring the 
read-only write protection? 

The function dw_pcie_setup_rc() disables read-only protection at the 
beginning via dw_pcie_dbi_ro_wr_en(). If dw_pcie_iatu_setup() fails, it 
returns immediately. Does this permanently leave critical PCI hardware 
configuration registers writable?

[ ... ]
>  	if (pci->pp.ops->init) {
>  		ret = pci->pp.ops->init(&pci->pp);
>  		if (ret) {
>  			dev_err(pci->dev, "Host init failed: %d\n", ret);
>  			return ret;
>  		}
>  	}
>  
>  	dw_pcie_setup_rc(&pci->pp);
>  
>  	ret = dw_pcie_start_link(pci);
>  	if (ret)
>  		goto err_deinit;

[Severity: High]
This is a pre-existing issue, but what happens if dw_pcie_setup_rc() fails
during resume?

It appears the return value of dw_pcie_setup_rc() is completely ignored. 
If it encounters an error like an ATU enable timeout returning -EBUSY, the 
driver proceeds to call dw_pcie_start_link() and re-enable endpoints while 
the Root Complex hardware is incorrectly initialized. Could this lead to 
accessing uninitialized hardware and causing a synchronous external abort?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260905025635.211390-1-zhangxincheng@ultrarisc.com?part=2