[PATCH v9 0/4] PCI: dwc: Add ECAM support with iATU configuration

[PATCH v9 0/4] PCI: dwc: Add ECAM support with iATU configuration

[Manivannan Sadhasivam]

The current implementation requires iATU for every configuration
space access which increases latency & cpu utilization.

Designware databook 5.20a, section 3.10.10.3 says about CFG Shift Feature,
which shifts/maps the BDF (bits [31:16] of the third header DWORD, which
would be matched against the Base and Limit addresses) of the incoming
CfgRd0/CfgWr0 down to bits[27:12]of the translated address.

Configuring iATU in config shift mode enables ECAM feature to access the
config space, which avoids iATU configuration for every config access.

Add cfg_shft_mode into struct dw_pcie_ob_atu_cfg to enable config shift mode.

As DBI comes under config space, this avoids remapping of DBI space
separately. Instead, it uses the mapped config space address returned from
ECAM initialization. Change the order of dw_pcie_get_resources() execution
to acheive this.

Enable the ECAM feature if the config space size is equal to size required
to represent number of buses in the bus range property.

ELBI registers are optional registers which are part of dwc. So move
ELBI resource mapping to dwc. Also change the dtbinding and devicetree
to make the elbi registers as optional one. Having ELBI as the required
one is making the ecam feature complicated.

The ELBI registers falls after the DBI space, PARF_SLV_DBI_ELBI register
gives us the offset from which ELBI starts. so use this offset and cfg
win to map these regions instead of doing the ioremap again.

On root bus, we have only the root port. Any access other than that
should not go out of the link and should return all F's. Since the iATU
is configured for the buses which starts after root bus, block the
transactions starting from function 1 of the root bus to the end of
the root bus (i.e from dbi_base + 4kb to dbi_base + 1MB) from going
outside the link through ECAM blocker through PARF registers.

Increase the configuration size to 256MB as required by the ECAM feature
and also move config space, DBI, iATU to upper space and use lower space
entirely for BAR region.

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
Signed-off-by: Manivannan Sadhasivam <mani@kernel.org>
---
Changes in v9:
 - Splitted the ECAM enablement change to avoid bisect hole and excluded pcie-al
   driver from DWC ECAM since it uses its own ECAM mechanism.
 - Squashed the patches 2 and 3 as patch 2 won't work without 3.

For the rest of the history, please refer v8 changeset:
https://lore.kernel.org/linux-pci/20250828-ecam_v4-v8-0-92a30e0fa02d@oss.qualcomm.com

---
Krishna Chaitanya Chundru (4):
      PCI: dwc: Add support for ELBI resource mapping
      PCI: dwc: Prepare the driver for enabling ECAM mechanism using iATU 'CFG Shift Feature'
      PCI: qcom: Prepare for the DWC ECAM enablement
      PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

 drivers/pci/controller/dwc/Kconfig                |   1 +
 drivers/pci/controller/dwc/pci-exynos.c           |  62 ++++-----
 drivers/pci/controller/dwc/pcie-al.c              |   1 +
 drivers/pci/controller/dwc/pcie-designware-host.c | 148 ++++++++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.c      |  10 +-
 drivers/pci/controller/dwc/pcie-designware.h      |   7 +
 drivers/pci/controller/dwc/pcie-qcom-ep.c         |  15 +--
 drivers/pci/controller/dwc/pcie-qcom.c            |  85 +++++++++++--
 8 files changed, 262 insertions(+), 67 deletions(-)
---
base-commit: 8f5ae30d69d7543eee0d70083daf4de8fe15d585
change-id: 20250908-controller-dwc-ecam-4c7c3136de14

Best regards,
-- 
Manivannan Sadhasivam <mani@kernel.org>

[PATCH v9 1/4] PCI: dwc: Add support for ELBI resource mapping

[Manivannan Sadhasivam]

From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>

External Local Bus Interface (ELBI) is an optional register space for all
DWC IPs containing the vendor specific registers. There is no need for the
vendor glue drivers to fetch and map the ELBI region separately.

Hence, optionally fetch and map the resource from DT in the DWC core. This
also warrants dropping the corresponding code from glue drivers. Hence,
drop the ELBI resource fetch and map logic from glue drivers and convert
them to use 'dw_pci::elbi_base'.

Note that the pcie-qcom-ep driver used devm_pci_remap_cfg_resource() to map
the ELBI resource previously. But it was a mistake since
devm_pci_remap_cfg_resource() should only be used for mapping the PCIe
config space region as it maps the region as Non-Posted. As ELBI is used to
hold vendor specific registers, there is no need to map the region as
Non-Posted. With this conversion, the region will get mapped as normal MMIO
memory.

Suggested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
[mani: removed elbi override, converted glue drivers and reworded description]
Signed-off-by: Manivannan Sadhasivam <mani@kernel.org>
---
 drivers/pci/controller/dwc/pci-exynos.c      | 62 ++++++++++++++--------------
 drivers/pci/controller/dwc/pcie-designware.c |  8 ++++
 drivers/pci/controller/dwc/pcie-designware.h |  1 +
 drivers/pci/controller/dwc/pcie-qcom-ep.c    | 15 ++-----
 drivers/pci/controller/dwc/pcie-qcom.c       | 16 ++++---
 5 files changed, 50 insertions(+), 52 deletions(-)

diff --git a/drivers/pci/controller/dwc/pci-exynos.c b/drivers/pci/controller/dwc/pci-exynos.c
index 1f0e98d07109353e7321667e98ead2695151184c..0bb7d4f5d7840eaed75b2ba0884f045e4d1c6d2f 100644
--- a/drivers/pci/controller/dwc/pci-exynos.c
+++ b/drivers/pci/controller/dwc/pci-exynos.c
@@ -53,7 +53,6 @@
 
 struct exynos_pcie {
 	struct dw_pcie			pci;
-	void __iomem			*elbi_base;
 	struct clk_bulk_data		*clks;
 	struct phy			*phy;
 	struct regulator_bulk_data	supplies[2];
@@ -71,73 +70,78 @@ static u32 exynos_pcie_readl(void __iomem *base, u32 reg)
 
 static void exynos_pcie_sideband_dbi_w_mode(struct exynos_pcie *ep, bool on)
 {
+	struct dw_pcie *pci = &ep->pci;
 	u32 val;
 
-	val = exynos_pcie_readl(ep->elbi_base, PCIE_ELBI_SLV_AWMISC);
+	val = exynos_pcie_readl(pci->elbi_base, PCIE_ELBI_SLV_AWMISC);
 	if (on)
 		val |= PCIE_ELBI_SLV_DBI_ENABLE;
 	else
 		val &= ~PCIE_ELBI_SLV_DBI_ENABLE;
-	exynos_pcie_writel(ep->elbi_base, val, PCIE_ELBI_SLV_AWMISC);
+	exynos_pcie_writel(pci->elbi_base, val, PCIE_ELBI_SLV_AWMISC);
 }
 
 static void exynos_pcie_sideband_dbi_r_mode(struct exynos_pcie *ep, bool on)
 {
+	struct dw_pcie *pci = &ep->pci;
 	u32 val;
 
-	val = exynos_pcie_readl(ep->elbi_base, PCIE_ELBI_SLV_ARMISC);
+	val = exynos_pcie_readl(pci->elbi_base, PCIE_ELBI_SLV_ARMISC);
 	if (on)
 		val |= PCIE_ELBI_SLV_DBI_ENABLE;
 	else
 		val &= ~PCIE_ELBI_SLV_DBI_ENABLE;
-	exynos_pcie_writel(ep->elbi_base, val, PCIE_ELBI_SLV_ARMISC);
+	exynos_pcie_writel(pci->elbi_base, val, PCIE_ELBI_SLV_ARMISC);
 }
 
 static void exynos_pcie_assert_core_reset(struct exynos_pcie *ep)
 {
+	struct dw_pcie *pci = &ep->pci;
 	u32 val;
 
-	val = exynos_pcie_readl(ep->elbi_base, PCIE_CORE_RESET);
+	val = exynos_pcie_readl(pci->elbi_base, PCIE_CORE_RESET);
 	val &= ~PCIE_CORE_RESET_ENABLE;
-	exynos_pcie_writel(ep->elbi_base, val, PCIE_CORE_RESET);
-	exynos_pcie_writel(ep->elbi_base, 0, PCIE_STICKY_RESET);
-	exynos_pcie_writel(ep->elbi_base, 0, PCIE_NONSTICKY_RESET);
+	exynos_pcie_writel(pci->elbi_base, val, PCIE_CORE_RESET);
+	exynos_pcie_writel(pci->elbi_base, 0, PCIE_STICKY_RESET);
+	exynos_pcie_writel(pci->elbi_base, 0, PCIE_NONSTICKY_RESET);
 }
 
 static void exynos_pcie_deassert_core_reset(struct exynos_pcie *ep)
 {
+	struct dw_pcie *pci = &ep->pci;
 	u32 val;
 
-	val = exynos_pcie_readl(ep->elbi_base, PCIE_CORE_RESET);
+	val = exynos_pcie_readl(pci->elbi_base, PCIE_CORE_RESET);
 	val |= PCIE_CORE_RESET_ENABLE;
 
-	exynos_pcie_writel(ep->elbi_base, val, PCIE_CORE_RESET);
-	exynos_pcie_writel(ep->elbi_base, 1, PCIE_STICKY_RESET);
-	exynos_pcie_writel(ep->elbi_base, 1, PCIE_NONSTICKY_RESET);
-	exynos_pcie_writel(ep->elbi_base, 1, PCIE_APP_INIT_RESET);
-	exynos_pcie_writel(ep->elbi_base, 0, PCIE_APP_INIT_RESET);
+	exynos_pcie_writel(pci->elbi_base, val, PCIE_CORE_RESET);
+	exynos_pcie_writel(pci->elbi_base, 1, PCIE_STICKY_RESET);
+	exynos_pcie_writel(pci->elbi_base, 1, PCIE_NONSTICKY_RESET);
+	exynos_pcie_writel(pci->elbi_base, 1, PCIE_APP_INIT_RESET);
+	exynos_pcie_writel(pci->elbi_base, 0, PCIE_APP_INIT_RESET);
 }
 
 static int exynos_pcie_start_link(struct dw_pcie *pci)
 {
-	struct exynos_pcie *ep = to_exynos_pcie(pci);
 	u32 val;
 
-	val = exynos_pcie_readl(ep->elbi_base, PCIE_SW_WAKE);
+	val = exynos_pcie_readl(pci->elbi_base, PCIE_SW_WAKE);
 	val &= ~PCIE_BUS_EN;
-	exynos_pcie_writel(ep->elbi_base, val, PCIE_SW_WAKE);
+	exynos_pcie_writel(pci->elbi_base, val, PCIE_SW_WAKE);
 
 	/* assert LTSSM enable */
-	exynos_pcie_writel(ep->elbi_base, PCIE_ELBI_LTSSM_ENABLE,
+	exynos_pcie_writel(pci->elbi_base, PCIE_ELBI_LTSSM_ENABLE,
 			  PCIE_APP_LTSSM_ENABLE);
 	return 0;
 }
 
 static void exynos_pcie_clear_irq_pulse(struct exynos_pcie *ep)
 {
-	u32 val = exynos_pcie_readl(ep->elbi_base, PCIE_IRQ_PULSE);
+	struct dw_pcie *pci = &ep->pci;
 
-	exynos_pcie_writel(ep->elbi_base, val, PCIE_IRQ_PULSE);
+	u32 val = exynos_pcie_readl(pci->elbi_base, PCIE_IRQ_PULSE);
+
+	exynos_pcie_writel(pci->elbi_base, val, PCIE_IRQ_PULSE);
 }
 
 static irqreturn_t exynos_pcie_irq_handler(int irq, void *arg)
@@ -150,12 +154,14 @@ static irqreturn_t exynos_pcie_irq_handler(int irq, void *arg)
 
 static void exynos_pcie_enable_irq_pulse(struct exynos_pcie *ep)
 {
+	struct dw_pcie *pci = &ep->pci;
+
 	u32 val = IRQ_INTA_ASSERT | IRQ_INTB_ASSERT |
 		  IRQ_INTC_ASSERT | IRQ_INTD_ASSERT;
 
-	exynos_pcie_writel(ep->elbi_base, val, PCIE_IRQ_EN_PULSE);
-	exynos_pcie_writel(ep->elbi_base, 0, PCIE_IRQ_EN_LEVEL);
-	exynos_pcie_writel(ep->elbi_base, 0, PCIE_IRQ_EN_SPECIAL);
+	exynos_pcie_writel(pci->elbi_base, val, PCIE_IRQ_EN_PULSE);
+	exynos_pcie_writel(pci->elbi_base, 0, PCIE_IRQ_EN_LEVEL);
+	exynos_pcie_writel(pci->elbi_base, 0, PCIE_IRQ_EN_SPECIAL);
 }
 
 static u32 exynos_pcie_read_dbi(struct dw_pcie *pci, void __iomem *base,
@@ -211,8 +217,7 @@ static struct pci_ops exynos_pci_ops = {
 
 static bool exynos_pcie_link_up(struct dw_pcie *pci)
 {
-	struct exynos_pcie *ep = to_exynos_pcie(pci);
-	u32 val = exynos_pcie_readl(ep->elbi_base, PCIE_ELBI_RDLH_LINKUP);
+	u32 val = exynos_pcie_readl(pci->elbi_base, PCIE_ELBI_RDLH_LINKUP);
 
 	return val & PCIE_ELBI_XMLH_LINKUP;
 }
@@ -295,11 +300,6 @@ static int exynos_pcie_probe(struct platform_device *pdev)
 	if (IS_ERR(ep->phy))
 		return PTR_ERR(ep->phy);
 
-	/* External Local Bus interface (ELBI) registers */
-	ep->elbi_base = devm_platform_ioremap_resource_byname(pdev, "elbi");
-	if (IS_ERR(ep->elbi_base))
-		return PTR_ERR(ep->elbi_base);
-
 	ret = devm_clk_bulk_get_all_enabled(dev, &ep->clks);
 	if (ret < 0)
 		return ret;
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 89aad5a08928cc29870ab258d33bee9ff8f83143..93da8a26c9313dfdd6b269a90bbfb017aab2abe7 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -167,6 +167,14 @@ int dw_pcie_get_resources(struct dw_pcie *pci)
 		}
 	}
 
+	/* ELBI is an optional resource */
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "elbi");
+	if (res) {
+		pci->elbi_base = devm_ioremap_resource(pci->dev, res);
+		if (IS_ERR(pci->elbi_base))
+			return PTR_ERR(pci->elbi_base);
+	}
+
 	/* LLDD is supposed to manually switch the clocks and resets state */
 	if (dw_pcie_cap_is(pci, REQ_RES)) {
 		ret = dw_pcie_get_clocks(pci);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 00f52d472dcdd794013a865ad6c4c7cc251edb48..ceb022506c3191cd8fe580411526e20cc3758fed 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -492,6 +492,7 @@ struct dw_pcie {
 	resource_size_t		dbi_phys_addr;
 	void __iomem		*dbi_base2;
 	void __iomem		*atu_base;
+	void __iomem		*elbi_base;
 	resource_size_t		atu_phys_addr;
 	size_t			atu_size;
 	resource_size_t		parent_bus_offset;
diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c
index bf7c6ac0f3e3962de8346ab6e75dfff1d9642aad..2030f3b0f7f98f30b625740cc05cda694fe64125 100644
--- a/drivers/pci/controller/dwc/pcie-qcom-ep.c
+++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c
@@ -179,7 +179,6 @@ struct qcom_pcie_ep_cfg {
  * struct qcom_pcie_ep - Qualcomm PCIe Endpoint Controller
  * @pci: Designware PCIe controller struct
  * @parf: Qualcomm PCIe specific PARF register base
- * @elbi: Designware PCIe specific ELBI register base
  * @mmio: MMIO register base
  * @perst_map: PERST regmap
  * @mmio_res: MMIO region resource
@@ -202,7 +201,6 @@ struct qcom_pcie_ep {
 	struct dw_pcie pci;
 
 	void __iomem *parf;
-	void __iomem *elbi;
 	void __iomem *mmio;
 	struct regmap *perst_map;
 	struct resource *mmio_res;
@@ -267,10 +265,9 @@ static void qcom_pcie_ep_configure_tcsr(struct qcom_pcie_ep *pcie_ep)
 
 static bool qcom_pcie_dw_link_up(struct dw_pcie *pci)
 {
-	struct qcom_pcie_ep *pcie_ep = to_pcie_ep(pci);
 	u32 reg;
 
-	reg = readl_relaxed(pcie_ep->elbi + ELBI_SYS_STTS);
+	reg = readl_relaxed(pci->elbi_base + ELBI_SYS_STTS);
 
 	return reg & XMLH_LINK_UP;
 }
@@ -294,16 +291,15 @@ static void qcom_pcie_dw_stop_link(struct dw_pcie *pci)
 static void qcom_pcie_dw_write_dbi2(struct dw_pcie *pci, void __iomem *base,
 				    u32 reg, size_t size, u32 val)
 {
-	struct qcom_pcie_ep *pcie_ep = to_pcie_ep(pci);
 	int ret;
 
-	writel(1, pcie_ep->elbi + ELBI_CS2_ENABLE);
+	writel(1, pci->elbi_base + ELBI_CS2_ENABLE);
 
 	ret = dw_pcie_write(pci->dbi_base2 + reg, size, val);
 	if (ret)
 		dev_err(pci->dev, "Failed to write DBI2 register (0x%x): %d\n", reg, ret);
 
-	writel(0, pcie_ep->elbi + ELBI_CS2_ENABLE);
+	writel(0, pci->elbi_base + ELBI_CS2_ENABLE);
 }
 
 static void qcom_pcie_ep_icc_update(struct qcom_pcie_ep *pcie_ep)
@@ -583,11 +579,6 @@ static int qcom_pcie_ep_get_io_resources(struct platform_device *pdev,
 		return PTR_ERR(pci->dbi_base);
 	pci->dbi_base2 = pci->dbi_base;
 
-	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "elbi");
-	pcie_ep->elbi = devm_pci_remap_cfg_resource(dev, res);
-	if (IS_ERR(pcie_ep->elbi))
-		return PTR_ERR(pcie_ep->elbi);
-
 	pcie_ep->mmio_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
 							 "mmio");
 	if (!pcie_ep->mmio_res) {
diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
index 294babe1816e4d0c2b2343fe22d89af72afcd6cd..5092752de23866ef95036bb3f8fae9bb06e8ea1e 100644
--- a/drivers/pci/controller/dwc/pcie-qcom.c
+++ b/drivers/pci/controller/dwc/pcie-qcom.c
@@ -276,7 +276,6 @@ struct qcom_pcie_port {
 struct qcom_pcie {
 	struct dw_pcie *pci;
 	void __iomem *parf;			/* DT parf */
-	void __iomem *elbi;			/* DT elbi */
 	void __iomem *mhi;
 	union qcom_pcie_resources res;
 	struct phy *phy;
@@ -414,12 +413,17 @@ static void qcom_pcie_configure_dbi_atu_base(struct qcom_pcie *pcie)
 
 static void qcom_pcie_2_1_0_ltssm_enable(struct qcom_pcie *pcie)
 {
+	struct dw_pcie *pci = pcie->pci;
 	u32 val;
 
+	if (!pci->elbi_base) {
+		dev_err(pci->dev, "ELBI is not present\n");
+		return;
+	}
 	/* enable link training */
-	val = readl(pcie->elbi + ELBI_SYS_CTRL);
+	val = readl(pci->elbi_base + ELBI_SYS_CTRL);
 	val |= ELBI_SYS_CTRL_LT_ENABLE;
-	writel(val, pcie->elbi + ELBI_SYS_CTRL);
+	writel(val, pci->elbi_base + ELBI_SYS_CTRL);
 }
 
 static int qcom_pcie_get_resources_2_1_0(struct qcom_pcie *pcie)
@@ -1861,12 +1865,6 @@ static int qcom_pcie_probe(struct platform_device *pdev)
 		goto err_pm_runtime_put;
 	}
 
-	pcie->elbi = devm_platform_ioremap_resource_byname(pdev, "elbi");
-	if (IS_ERR(pcie->elbi)) {
-		ret = PTR_ERR(pcie->elbi);
-		goto err_pm_runtime_put;
-	}
-
 	/* MHI region is optional */
 	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "mhi");
 	if (res) {

-- 
2.45.2

[PATCH v9 2/4] PCI: dwc: Prepare the driver for enabling ECAM mechanism using iATU 'CFG Shift Feature'

[Manivannan Sadhasivam]

From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>

In order to enable PCIe ECAM mechanism in DWC driver as per the 'CFG Shift
Feature' documented in Designware databook r5.20a, sec 3.10.10.3, prepare
the driver to handle the one time iATU setup and creating ECAM window.

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
[mani: splitted the preparatory code into a separate commit for bisectability]
Signed-off-by: Manivannan Sadhasivam <mani@kernel.org>
---
 drivers/pci/controller/dwc/Kconfig                |   1 +
 drivers/pci/controller/dwc/pcie-designware-host.c | 116 +++++++++++++++++++---
 drivers/pci/controller/dwc/pcie-designware.c      |   2 +-
 drivers/pci/controller/dwc/pcie-designware.h      |   5 +
 4 files changed, 109 insertions(+), 15 deletions(-)

diff --git a/drivers/pci/controller/dwc/Kconfig b/drivers/pci/controller/dwc/Kconfig
index ff6b6d9e18ecfa44273e87931551f9e63fbe3cba..a0e7ad3fb5afec63b0f919732a50147229623186 100644
--- a/drivers/pci/controller/dwc/Kconfig
+++ b/drivers/pci/controller/dwc/Kconfig
@@ -20,6 +20,7 @@ config PCIE_DW_HOST
 	bool
 	select PCIE_DW
 	select IRQ_MSI_LIB
+	select PCI_HOST_COMMON
 
 config PCIE_DW_EP
 	bool
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 952f8594b501254d2b2de5d5e056e16d2aa8d4b7..94e0fe11a0b062d0f14e09fe586e20bde46a4266 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -413,6 +413,67 @@ static void dw_pcie_host_request_msg_tlp_res(struct dw_pcie_rp *pp)
 	}
 }
 
+static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct dw_pcie_ob_atu_cfg atu = {0};
+	resource_size_t bus_range_max;
+	struct resource_entry *bus;
+	int ret;
+
+	bus = resource_list_first_type(&pp->bridge->windows, IORESOURCE_BUS);
+
+	/*
+	 * Root bus under the host bridge doesn't require any iATU configuration
+	 * as DBI region will be used to access root bus config space.
+	 * Immediate bus under Root Bus, needs type 0 iATU configuration and
+	 * remaining buses need type 1 iATU configuration.
+	 */
+	atu.index = 0;
+	atu.type = PCIE_ATU_TYPE_CFG0;
+	atu.parent_bus_addr = pp->cfg0_base + SZ_1M;
+	/* 1MiB is to cover 1 (bus) * 32 (devices) * 8 (functions) */
+	atu.size = SZ_1M;
+	atu.ctrl2 = PCIE_ATU_CFG_SHIFT_MODE_ENABLE;
+	ret = dw_pcie_prog_outbound_atu(pci, &atu);
+	if (ret)
+		return ret;
+
+	bus_range_max = resource_size(bus->res);
+
+	if (bus_range_max < 2)
+		return 0;
+
+	/* Configure remaining buses in type 1 iATU configuration */
+	atu.index = 1;
+	atu.type = PCIE_ATU_TYPE_CFG1;
+	atu.parent_bus_addr = pp->cfg0_base + SZ_2M;
+	atu.size = (SZ_1M * bus_range_max) - SZ_2M;
+	atu.ctrl2 = PCIE_ATU_CFG_SHIFT_MODE_ENABLE;
+
+	return dw_pcie_prog_outbound_atu(pci, &atu);
+}
+
+static int dw_pcie_create_ecam_window(struct dw_pcie_rp *pp, struct resource *res)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct device *dev = pci->dev;
+	struct resource_entry *bus;
+
+	bus = resource_list_first_type(&pp->bridge->windows, IORESOURCE_BUS);
+	if (!bus)
+		return -ENODEV;
+
+	pp->cfg = pci_ecam_create(dev, res, bus->res, &pci_generic_ecam_ops);
+	if (IS_ERR(pp->cfg))
+		return PTR_ERR(pp->cfg);
+
+	pci->dbi_base = pp->cfg->win;
+	pci->dbi_phys_addr = res->start;
+
+	return 0;
+}
+
 static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -422,10 +483,6 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 	struct resource *res;
 	int ret;
 
-	ret = dw_pcie_get_resources(pci);
-	if (ret)
-		return ret;
-
 	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
 	if (!res) {
 		dev_err(dev, "Missing \"config\" reg space\n");
@@ -435,9 +492,31 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 	pp->cfg0_size = resource_size(res);
 	pp->cfg0_base = res->start;
 
-	pp->va_cfg0_base = devm_pci_remap_cfg_resource(dev, res);
-	if (IS_ERR(pp->va_cfg0_base))
-		return PTR_ERR(pp->va_cfg0_base);
+	if (pp->ecam_enabled) {
+		ret = dw_pcie_create_ecam_window(pp, res);
+		if (ret)
+			return ret;
+
+		pp->bridge->ops = (struct pci_ops *)&pci_generic_ecam_ops.pci_ops;
+		pp->bridge->sysdata = pp->cfg;
+		pp->cfg->priv = pp;
+	} else {
+		pp->va_cfg0_base = devm_pci_remap_cfg_resource(dev, res);
+		if (IS_ERR(pp->va_cfg0_base))
+			return PTR_ERR(pp->va_cfg0_base);
+
+		/* Set default bus ops */
+		pp->bridge->ops = &dw_pcie_ops;
+		pp->bridge->child_ops = &dw_child_pcie_ops;
+		pp->bridge->sysdata = pp;
+	}
+
+	ret = dw_pcie_get_resources(pci);
+	if (ret) {
+		if (pp->cfg)
+			pci_ecam_free(pp->cfg);
+		return ret;
+	}
 
 	/* Get the I/O range from DT */
 	win = resource_list_first_type(&pp->bridge->windows, IORESOURCE_IO);
@@ -476,14 +555,10 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 	if (ret)
 		return ret;
 
-	/* Set default bus ops */
-	bridge->ops = &dw_pcie_ops;
-	bridge->child_ops = &dw_child_pcie_ops;
-
 	if (pp->ops->init) {
 		ret = pp->ops->init(pp);
 		if (ret)
-			return ret;
+			goto err_free_ecam;
 	}
 
 	if (pci_msi_enabled()) {
@@ -525,6 +600,14 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 	if (ret)
 		goto err_free_msi;
 
+	if (pp->ecam_enabled) {
+		ret = dw_pcie_config_ecam_iatu(pp);
+		if (ret) {
+			dev_err(dev, "Failed to configure iATU in ECAM mode\n");
+			goto err_free_msi;
+		}
+	}
+
 	/*
 	 * Allocate the resource for MSG TLP before programming the iATU
 	 * outbound window in dw_pcie_setup_rc(). Since the allocation depends
@@ -560,8 +643,6 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 		/* Ignore errors, the link may come up later */
 		dw_pcie_wait_for_link(pci);
 
-	bridge->sysdata = pp;
-
 	ret = pci_host_probe(bridge);
 	if (ret)
 		goto err_stop_link;
@@ -587,6 +668,10 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 	if (pp->ops->deinit)
 		pp->ops->deinit(pp);
 
+err_free_ecam:
+	if (pp->cfg)
+		pci_ecam_free(pp->cfg);
+
 	return ret;
 }
 EXPORT_SYMBOL_GPL(dw_pcie_host_init);
@@ -609,6 +694,9 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp)
 
 	if (pp->ops->deinit)
 		pp->ops->deinit(pp);
+
+	if (pp->cfg)
+		pci_ecam_free(pp->cfg);
 }
 EXPORT_SYMBOL_GPL(dw_pcie_host_deinit);
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 93da8a26c9313dfdd6b269a90bbfb017aab2abe7..ee8caae1edbd44c10e6986430a6932c10f20a9eb 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -575,7 +575,7 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
 		val = dw_pcie_enable_ecrc(val);
 	dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL1, val);
 
-	val = PCIE_ATU_ENABLE;
+	val = PCIE_ATU_ENABLE | atu->ctrl2;
 	if (atu->type == PCIE_ATU_TYPE_MSG) {
 		/* The data-less messages only for now */
 		val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code;
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index ceb022506c3191cd8fe580411526e20cc3758fed..cfeb99b89c3739b867a54da38e0f94835c2ce3a0 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -20,6 +20,7 @@
 #include <linux/irq.h>
 #include <linux/msi.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/reset.h>
 
 #include <linux/pci-epc.h>
@@ -169,6 +170,7 @@
 #define PCIE_ATU_REGION_CTRL2		0x004
 #define PCIE_ATU_ENABLE			BIT(31)
 #define PCIE_ATU_BAR_MODE_ENABLE	BIT(30)
+#define PCIE_ATU_CFG_SHIFT_MODE_ENABLE	BIT(28)
 #define PCIE_ATU_INHIBIT_PAYLOAD	BIT(22)
 #define PCIE_ATU_FUNC_NUM_MATCH_EN      BIT(19)
 #define PCIE_ATU_LOWER_BASE		0x008
@@ -387,6 +389,7 @@ struct dw_pcie_ob_atu_cfg {
 	u8 func_no;
 	u8 code;
 	u8 routing;
+	u32 ctrl2;
 	u64 parent_bus_addr;
 	u64 pci_addr;
 	u64 size;
@@ -425,6 +428,8 @@ struct dw_pcie_rp {
 	struct resource		*msg_res;
 	bool			use_linkup_irq;
 	struct pci_eq_presets	presets;
+	struct pci_config_window *cfg;
+	bool			ecam_enabled;
 };
 
 struct dw_pcie_ep_ops {

-- 
2.45.2

[PATCH v9 3/4] PCI: qcom: Prepare for the DWC ECAM enablement

[Manivannan Sadhasivam]

From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>

To support the DWC ECAM mechanism, prepare the driver by performing below
configurations:

  1. Since the ELBI region will be covered by the ECAM 'config' space,
     override the 'elbi_base' with the address derived from 'dbi_base' and
     the offset from PARF_SLV_DBI_ELBI register.

  2. Block the transactions from the host bridge to devices other than Root
     Port on the root bus to return all F's. This is required when the 'CFG
     Shift Feature' of iATU is enabled.

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
[mani: code split, reworded subject/description and comments]
Signed-off-by: Manivannan Sadhasivam <mani@kernel.org>
---
 drivers/pci/controller/dwc/pcie-qcom.c | 69 ++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
index 5092752de23866ef95036bb3f8fae9bb06e8ea1e..4324555de72adbdd3dcdf59497f9bd067d9c90ab 100644
--- a/drivers/pci/controller/dwc/pcie-qcom.c
+++ b/drivers/pci/controller/dwc/pcie-qcom.c
@@ -55,6 +55,7 @@
 #define PARF_AXI_MSTR_WR_ADDR_HALT_V2		0x1a8
 #define PARF_Q2A_FLUSH				0x1ac
 #define PARF_LTSSM				0x1b0
+#define PARF_SLV_DBI_ELBI			0x1b4
 #define PARF_INT_ALL_STATUS			0x224
 #define PARF_INT_ALL_CLEAR			0x228
 #define PARF_INT_ALL_MASK			0x22c
@@ -64,6 +65,16 @@
 #define PARF_DBI_BASE_ADDR_V2_HI		0x354
 #define PARF_SLV_ADDR_SPACE_SIZE_V2		0x358
 #define PARF_SLV_ADDR_SPACE_SIZE_V2_HI		0x35c
+#define PARF_BLOCK_SLV_AXI_WR_BASE		0x360
+#define PARF_BLOCK_SLV_AXI_WR_BASE_HI		0x364
+#define PARF_BLOCK_SLV_AXI_WR_LIMIT		0x368
+#define PARF_BLOCK_SLV_AXI_WR_LIMIT_HI		0x36c
+#define PARF_BLOCK_SLV_AXI_RD_BASE		0x370
+#define PARF_BLOCK_SLV_AXI_RD_BASE_HI		0x374
+#define PARF_BLOCK_SLV_AXI_RD_LIMIT		0x378
+#define PARF_BLOCK_SLV_AXI_RD_LIMIT_HI		0x37c
+#define PARF_ECAM_BASE				0x380
+#define PARF_ECAM_BASE_HI			0x384
 #define PARF_NO_SNOOP_OVERRIDE			0x3d4
 #define PARF_ATU_BASE_ADDR			0x634
 #define PARF_ATU_BASE_ADDR_HI			0x638
@@ -87,6 +98,7 @@
 
 /* PARF_SYS_CTRL register fields */
 #define MAC_PHY_POWERDOWN_IN_P2_D_MUX_EN	BIT(29)
+#define PCIE_ECAM_BLOCKER_EN			BIT(26)
 #define MST_WAKEUP_EN				BIT(13)
 #define SLV_WAKEUP_EN				BIT(12)
 #define MSTR_ACLK_CGC_DIS			BIT(10)
@@ -134,6 +146,9 @@
 /* PARF_LTSSM register fields */
 #define LTSSM_EN				BIT(8)
 
+/* PARF_SLV_DBI_ELBI */
+#define SLV_DBI_ELBI_ADDR_BASE			GENMASK(11, 0)
+
 /* PARF_INT_ALL_{STATUS/CLEAR/MASK} register fields */
 #define PARF_INT_ALL_LINK_UP			BIT(13)
 #define PARF_INT_MSI_DEV_0_7			GENMASK(30, 23)
@@ -317,6 +332,47 @@ static void qcom_ep_reset_deassert(struct qcom_pcie *pcie)
 	qcom_perst_assert(pcie, false);
 }
 
+static void qcom_pci_config_ecam(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct qcom_pcie *pcie = to_qcom_pcie(pci);
+	u64 addr, addr_end;
+	u32 val;
+
+	writel_relaxed(lower_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE);
+	writel_relaxed(upper_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE_HI);
+
+	/*
+	 * The only device on root bus is a single Root Port. So if PCI core
+	 * tries to access any devices other than Device/Function (0.0) in Bus
+	 * 0, the TLP will go outside of the controller to the PCI bus. But with
+	 * CFG Shift Feature (ECAM) enabled in iATU, there is no guarantee that
+	 * the response is going to be all F's. Hence, to make sure that the
+	 * requester gets all F's response for accesses other than the Root
+	 * Port, configure iATU to block the transactions starting from function
+	 * 1 of the root bus to the end of the root bus (i.e from dbi_base + 4kb
+	 * to dbi_base + 1MB).
+	 */
+	addr = pci->dbi_phys_addr + SZ_4K;
+	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE);
+	writel_relaxed(upper_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE_HI);
+
+	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE);
+	writel_relaxed(upper_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE_HI);
+
+	addr_end = pci->dbi_phys_addr + SZ_1M - 1;
+
+	writel_relaxed(lower_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT);
+	writel_relaxed(upper_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT_HI);
+
+	writel_relaxed(lower_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT);
+	writel_relaxed(upper_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT_HI);
+
+	val = readl_relaxed(pcie->parf + PARF_SYS_CTRL);
+	val |= PCIE_ECAM_BLOCKER_EN;
+	writel_relaxed(val, pcie->parf + PARF_SYS_CTRL);
+}
+
 static int qcom_pcie_start_link(struct dw_pcie *pci)
 {
 	struct qcom_pcie *pcie = to_qcom_pcie(pci);
@@ -326,6 +382,9 @@ static int qcom_pcie_start_link(struct dw_pcie *pci)
 		qcom_pcie_common_set_16gt_lane_margining(pci);
 	}
 
+	if (pci->pp.ecam_enabled)
+		qcom_pci_config_ecam(&pci->pp);
+
 	/* Enable Link Training state machine */
 	if (pcie->cfg->ops->ltssm_enable)
 		pcie->cfg->ops->ltssm_enable(pcie);
@@ -1314,6 +1373,7 @@ static int qcom_pcie_host_init(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
 	struct qcom_pcie *pcie = to_qcom_pcie(pci);
+	u16 offset;
 	int ret;
 
 	qcom_ep_reset_assert(pcie);
@@ -1322,6 +1382,15 @@ static int qcom_pcie_host_init(struct dw_pcie_rp *pp)
 	if (ret)
 		return ret;
 
+	if (pp->ecam_enabled) {
+		/*
+		 * Override ELBI when ECAM is enabled, as when ECAM is enabled,
+		 * ELBI moves under the 'config' space.
+		 */
+		offset = FIELD_GET(SLV_DBI_ELBI_ADDR_BASE, readl(pcie->parf + PARF_SLV_DBI_ELBI));
+		pci->elbi_base = pci->dbi_base + offset;
+	}
+
 	ret = qcom_pcie_phy_power_on(pcie);
 	if (ret)
 		goto err_deinit;

-- 
2.45.2

[PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Manivannan Sadhasivam]

From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>

Designware databook r5.20a, sec 3.10.10.3 documents the 'CFG Shift Feature'
of the internal Address Translation Unit (iATU). When this feature is
enabled, it shifts/maps the BDF contained in the bits [27:12] of the target
address in MEM TLP to become BDF of the CFG TLP. This essentially
implements the Enhanced Configuration Address Mapping (ECAM) mechanism as
defined in PCIe r6.0, sec 7.2.2.

Currently, the driver is not making use of this CFG shift feature, thereby
creating the iATU outbound map for each config access to the devices,
causing latency and wasting CPU cycles.

So to avoid this, configure the controller to enable CFG shift feature by
enabling the 'CFG Shift' bit of the 'iATU Control 2 Register'.

As a result of enabling CFG shift (ECAM), there is longer a need to map the
DBI register space separately as the DBI region falls under the 'config'
space used for ECAM (as DBI is used to access the Root Port).

For enabling ECAM using CFG shift, the platform has to satisfy following
requirements:

  1. Size of the 'config' memory space to be used as ECAM memory should be
     able to accommodate the number of buses defined in the 'bus-range'
     property of the host bridge DT node.

  2. The 'config' memory space should be 256 MiB aligned. This requirement
     comes from PCIe r6.0, sec 7.2.2, which says the base address of ECAM
     memory should be aligned to a 2^(n+20) byte address boundary. For the
     DWC cores, n is 8, so this results in 2^28 byte alignment requirement.

It should be noted that some DWC vendor glue drivers like pcie-al may use
their own ECAM mechanism. For those controllers, set
'dw_pcie_rp::native_ecam' flag and skip enabling the CFG Shift feature in
the DWC core.

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
[mani: code split, reworded subject/description, comment, native_ecam flag]
Signed-off-by: Manivannan Sadhasivam <mani@kernel.org>
---
 drivers/pci/controller/dwc/pcie-al.c              |  1 +
 drivers/pci/controller/dwc/pcie-designware-host.c | 32 +++++++++++++++++++++++
 drivers/pci/controller/dwc/pcie-designware.h      |  1 +
 3 files changed, 34 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-al.c b/drivers/pci/controller/dwc/pcie-al.c
index 643115f74092d1c9319e9738db6e94b2752d30c4..345c281c74fefd2113233ef5461f96834b3765de 100644
--- a/drivers/pci/controller/dwc/pcie-al.c
+++ b/drivers/pci/controller/dwc/pcie-al.c
@@ -352,6 +352,7 @@ static int al_pcie_probe(struct platform_device *pdev)
 		return -ENOENT;
 	}
 	al_pcie->ecam_size = resource_size(ecam_res);
+	pci->pp.native_ecam = true;
 
 	controller_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
 						      "controller");
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 94e0fe11a0b062d0f14e09fe586e20bde46a4266..20c9333bcb1c4812e2fd96047a49944574df1e6f 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -8,6 +8,7 @@
  * Author: Jingoo Han <jg1.han@samsung.com>
  */
 
+#include <linux/align.h>
 #include <linux/iopoll.h>
 #include <linux/irqchip/chained_irq.h>
 #include <linux/irqchip/irq-msi-lib.h>
@@ -32,6 +33,8 @@ static struct pci_ops dw_child_pcie_ops;
 				     MSI_FLAG_PCI_MSIX			| \
 				     MSI_GENERIC_FLAGS_MASK)
 
+#define IS_256MB_ALIGNED(x) IS_ALIGNED(x, SZ_256M)
+
 static const struct msi_parent_ops dw_pcie_msi_parent_ops = {
 	.required_flags		= DW_PCIE_MSI_FLAGS_REQUIRED,
 	.supported_flags	= DW_PCIE_MSI_FLAGS_SUPPORTED,
@@ -474,6 +477,34 @@ static int dw_pcie_create_ecam_window(struct dw_pcie_rp *pp, struct resource *re
 	return 0;
 }
 
+static bool dw_pcie_ecam_enabled(struct dw_pcie_rp *pp, struct resource *config_res)
+{
+	struct resource *bus_range;
+	u64 nr_buses;
+
+	/* Vendor glue drivers may implement their own ECAM mechanism */
+	if (pp->native_ecam)
+		return false;
+
+	/*
+	 * PCIe spec r6.0, sec 7.2.2 mandates the base address used for ECAM to
+	 * be aligned on a 2^(n+20) byte boundary, where n is the number of bits
+	 * used for representing 'bus' in BDF. Since the DWC cores always use 8
+	 * bits for representing 'bus', the base address has to be aligned to
+	 * 2^28 byte boundary, which is 256 MiB.
+	 */
+	if (!IS_256MB_ALIGNED(config_res->start))
+		return false;
+
+	bus_range = resource_list_first_type(&pp->bridge->windows, IORESOURCE_BUS)->res;
+	if (!bus_range)
+		return false;
+
+	nr_buses = resource_size(config_res) >> PCIE_ECAM_BUS_SHIFT;
+
+	return nr_buses >= resource_size(bus_range);
+}
+
 static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -492,6 +523,7 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 	pp->cfg0_size = resource_size(res);
 	pp->cfg0_base = res->start;
 
+	pp->ecam_enabled = dw_pcie_ecam_enabled(pp, res);
 	if (pp->ecam_enabled) {
 		ret = dw_pcie_create_ecam_window(pp, res);
 		if (ret)
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index cfeb99b89c3739b867a54da38e0f94835c2ce3a0..6ec0a737294b2a69bbcc05bbdbe0a11c5f8310bb 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -430,6 +430,7 @@ struct dw_pcie_rp {
 	struct pci_eq_presets	presets;
 	struct pci_config_window *cfg;
 	bool			ecam_enabled;
+	bool			native_ecam;
 };
 
 struct dw_pcie_ep_ops {

-- 
2.45.2

Re: [PATCH v9 3/4] PCI: qcom: Prepare for the DWC ECAM enablement

[Bjorn Helgaas]

Sorry, I missed your repost of this series, Mani.  I'll reiterate my
questions here.

I also deleted the pci/controller/dwc-ecam branch, where Krishna's v8
series was queued up, to avoid confusion (it looked like that branch
was ready to be included in linux-next, but it's not).

On Tue, Sep 09, 2025 at 12:37:52PM +0530, Manivannan Sadhasivam wrote:
> From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
> 
> To support the DWC ECAM mechanism, prepare the driver by performing below
> configurations:
> 
>   1. Since the ELBI region will be covered by the ECAM 'config' space,
>      override the 'elbi_base' with the address derived from 'dbi_base' and
>      the offset from PARF_SLV_DBI_ELBI register.
> 
>   2. Block the transactions from the host bridge to devices other than Root
>      Port on the root bus to return all F's. This is required when the 'CFG
>      Shift Feature' of iATU is enabled.

> +++ b/drivers/pci/controller/dwc/pcie-qcom.c

> +static void qcom_pci_config_ecam(struct dw_pcie_rp *pp)
> +{
> +	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	struct qcom_pcie *pcie = to_qcom_pcie(pci);
> +	u64 addr, addr_end;
> +	u32 val;
> +
> +	writel_relaxed(lower_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE);
> +	writel_relaxed(upper_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE_HI);
> +
> +	/*
> +	 * The only device on root bus is a single Root Port. So if PCI core
> +	 * tries to access any devices other than Device/Function (0.0) in Bus
> +	 * 0, the TLP will go outside of the controller to the PCI bus. But with
> +	 * CFG Shift Feature (ECAM) enabled in iATU, there is no guarantee that
> +	 * the response is going to be all F's. Hence, to make sure that the
> +	 * requester gets all F's response for accesses other than the Root
> +	 * Port, configure iATU to block the transactions starting from function
> +	 * 1 of the root bus to the end of the root bus (i.e from dbi_base + 4kb
> +	 * to dbi_base + 1MB).
> +	 */
> +	addr = pci->dbi_phys_addr + SZ_4K;
> +	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE);
> +	writel_relaxed(upper_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE_HI);
> +
> +	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE);
> +	writel_relaxed(upper_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE_HI);
> +
> +	addr_end = pci->dbi_phys_addr + SZ_1M - 1;
> +
> +	writel_relaxed(lower_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT);
> +	writel_relaxed(upper_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT_HI);
> +
> +	writel_relaxed(lower_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT);
> +	writel_relaxed(upper_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT_HI);
> +
> +	val = readl_relaxed(pcie->parf + PARF_SYS_CTRL);
> +	val |= PCIE_ECAM_BLOCKER_EN;
> +	writel_relaxed(val, pcie->parf + PARF_SYS_CTRL);

The driver already supported ECAM in the existing "firmware_managed"
path (which looks untouched by this series and doesn't do any of this
iATU configuration).

And IIUC, this series adds support for ECAM whenever the DT 'config'
range is sufficiently aligned.  In this new ECAM support, it looks
like we look for and pay attention to 'bus-range' in this path:

  qcom_pcie_probe
    dw_pcie_host_init
      devm_pci_alloc_host_bridge
        devm_of_pci_bridge_init
          pci_parse_request_of_pci_ranges
            devm_of_pci_get_host_bridge_resources
              of_pci_parse_bus_range
                of_property_read_u32_array(node, "bus-range", ...)
      dw_pcie_host_get_resources
        res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config")
        pp->ecam_enabled = dw_pcie_ecam_enabled(pp, res)

Since qcom_pci_config_ecam() doesn't look at the root bus number at
all, is this also an implicit restriction that the root bus must be
bus 0?  Does qcom support root buses other than 0?

> +}
> +
>  static int qcom_pcie_start_link(struct dw_pcie *pci)
>  {
>  	struct qcom_pcie *pcie = to_qcom_pcie(pci);
> @@ -326,6 +382,9 @@ static int qcom_pcie_start_link(struct dw_pcie *pci)
>  		qcom_pcie_common_set_16gt_lane_margining(pci);
>  	}
>  
> +	if (pci->pp.ecam_enabled)
> +		qcom_pci_config_ecam(&pci->pp);

qcom_pcie_start_link() seems like a strange place to do this
ECAM-related iATU configuration.  ECAM is a function of the host
bridge, not of any particular Root Port or link.

>  	/* Enable Link Training state machine */
>  	if (pcie->cfg->ops->ltssm_enable)
>  		pcie->cfg->ops->ltssm_enable(pcie);

Re: [PATCH v9 3/4] PCI: qcom: Prepare for the DWC ECAM enablement

[Bjorn Helgaas]

On Tue, Sep 09, 2025 at 12:37:52PM +0530, Manivannan Sadhasivam wrote:
> From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
> 
> To support the DWC ECAM mechanism, prepare the driver by performing below
> configurations:
> 
>   1. Since the ELBI region will be covered by the ECAM 'config' space,
>      override the 'elbi_base' with the address derived from 'dbi_base' and
>      the offset from PARF_SLV_DBI_ELBI register.
> 
>   2. Block the transactions from the host bridge to devices other than Root
>      Port on the root bus to return all F's. This is required when the 'CFG
>      Shift Feature' of iATU is enabled.

FWIW, before I noticed your v9, I had updated the comments here to fix
a few inconsistencies.  Here's the diff:

diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
index 7c2b2c8c61c2..962f0311a23a 100644
--- a/drivers/pci/controller/dwc/pcie-qcom.c
+++ b/drivers/pci/controller/dwc/pcie-qcom.c
@@ -343,15 +343,15 @@ static void qcom_pci_config_ecam(struct dw_pcie_rp *pp)
 	writel_relaxed(upper_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE_HI);
 
 	/*
-	 * The only device on root bus is a single Root Port. So if PCI core
-	 * tries to access any devices other than Device/Function (0.0) in Bus
-	 * 0, the TLP will go outside of the controller to the PCI bus. But with
-	 * CFG Shift Feature (ECAM) enabled in iATU, there is no guarantee that
-	 * the response is going to be all F's. Hence, to make sure that the
+	 * The only device on the root bus is a single Root Port. If we try to
+	 * access any devices other than Device/Function 00.0 on Bus 0, the TLP
+	 * will go outside of the controller to the PCI bus. But with CFG Shift
+	 * Feature (ECAM) enabled in iATU, there is no guarantee that the
+	 * response is going to be all F's. Hence, to make sure that the
 	 * requester gets all F's response for accesses other than the Root
-	 * Port, configure iATU to block the transactions starting from function
-	 * 1 of the root bus to the end of the root bus (i.e from dbi_base + 4kb
-	 * to dbi_base + 1MB).
+	 * Port, configure iATU to block the transactions starting from
+	 * function 1 of the root bus to the end of the root bus (i.e., from
+	 * dbi_base + 4KB to dbi_base + 1MB).
 	 */
 	addr = pci->dbi_phys_addr + SZ_4K;
 	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE);
@@ -1385,7 +1385,7 @@ static int qcom_pcie_host_init(struct dw_pcie_rp *pp)
 	if (pp->ecam_enabled) {
 		/*
 		 * Override ELBI when ECAM is enabled, as when ECAM
-		 * is enabled ELBI moves along with the dbi config space.
+		 * is enabled ELBI moves along with the DBI config space.
 		 */
 		offset = FIELD_GET(SLV_DBI_ELBI_ADDR_BASE, readl(pcie->parf + PARF_SLV_DBI_ELBI));
 		pci->elbi_base = pci->dbi_base + offset;

Re: [PATCH v9 3/4] PCI: qcom: Prepare for the DWC ECAM enablement

[Manivannan Sadhasivam]

On Fri, Sep 12, 2025 at 04:44:32PM GMT, Bjorn Helgaas wrote:
> Sorry, I missed your repost of this series, Mani.  I'll reiterate my
> questions here.
> 
> I also deleted the pci/controller/dwc-ecam branch, where Krishna's v8
> series was queued up, to avoid confusion (it looked like that branch
> was ready to be included in linux-next, but it's not).
> 
> On Tue, Sep 09, 2025 at 12:37:52PM +0530, Manivannan Sadhasivam wrote:
> > From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
> > 
> > To support the DWC ECAM mechanism, prepare the driver by performing below
> > configurations:
> > 
> >   1. Since the ELBI region will be covered by the ECAM 'config' space,
> >      override the 'elbi_base' with the address derived from 'dbi_base' and
> >      the offset from PARF_SLV_DBI_ELBI register.
> > 
> >   2. Block the transactions from the host bridge to devices other than Root
> >      Port on the root bus to return all F's. This is required when the 'CFG
> >      Shift Feature' of iATU is enabled.
> 
> > +++ b/drivers/pci/controller/dwc/pcie-qcom.c
> 
> > +static void qcom_pci_config_ecam(struct dw_pcie_rp *pp)
> > +{
> > +	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> > +	struct qcom_pcie *pcie = to_qcom_pcie(pci);
> > +	u64 addr, addr_end;
> > +	u32 val;
> > +
> > +	writel_relaxed(lower_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE);
> > +	writel_relaxed(upper_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE_HI);
> > +
> > +	/*
> > +	 * The only device on root bus is a single Root Port. So if PCI core
> > +	 * tries to access any devices other than Device/Function (0.0) in Bus
> > +	 * 0, the TLP will go outside of the controller to the PCI bus. But with
> > +	 * CFG Shift Feature (ECAM) enabled in iATU, there is no guarantee that
> > +	 * the response is going to be all F's. Hence, to make sure that the
> > +	 * requester gets all F's response for accesses other than the Root
> > +	 * Port, configure iATU to block the transactions starting from function
> > +	 * 1 of the root bus to the end of the root bus (i.e from dbi_base + 4kb
> > +	 * to dbi_base + 1MB).
> > +	 */
> > +	addr = pci->dbi_phys_addr + SZ_4K;
> > +	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE);
> > +	writel_relaxed(upper_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE_HI);
> > +
> > +	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE);
> > +	writel_relaxed(upper_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE_HI);
> > +
> > +	addr_end = pci->dbi_phys_addr + SZ_1M - 1;
> > +
> > +	writel_relaxed(lower_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT);
> > +	writel_relaxed(upper_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT_HI);
> > +
> > +	writel_relaxed(lower_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT);
> > +	writel_relaxed(upper_32_bits(addr_end), pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT_HI);
> > +
> > +	val = readl_relaxed(pcie->parf + PARF_SYS_CTRL);
> > +	val |= PCIE_ECAM_BLOCKER_EN;
> > +	writel_relaxed(val, pcie->parf + PARF_SYS_CTRL);
> 
> The driver already supported ECAM in the existing "firmware_managed"
> path (which looks untouched by this series and doesn't do any of this
> iATU configuration).
> 

'firmware_managed' refers to the Qcom platforms supporting BIOS/bootloader that
has initialized the PCIe controller and configured ECAM mode. The driver doesn't
need to do any resource handling on its own including iATU as everything would
be handled by the BIOS/bootloader. This mode is used by the Qcom Automotive
SoCs.

And This mode is different from the DWC ECAM mode implemented in this series as
this one requires doing all iATU configuration in the driver itself. This mode
will be used by the Mobile/IoT/Compute SoCs.

Qcom SoCs have different requirement for different market segments. This is the
reason why we have many different implementations.

> And IIUC, this series adds support for ECAM whenever the DT 'config'
> range is sufficiently aligned.  In this new ECAM support, it looks
> like we look for and pay attention to 'bus-range' in this path:
> 
>   qcom_pcie_probe
>     dw_pcie_host_init
>       devm_pci_alloc_host_bridge
>         devm_of_pci_bridge_init
>           pci_parse_request_of_pci_ranges
>             devm_of_pci_get_host_bridge_resources
>               of_pci_parse_bus_range
>                 of_property_read_u32_array(node, "bus-range", ...)
>       dw_pcie_host_get_resources
>         res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config")
>         pp->ecam_enabled = dw_pcie_ecam_enabled(pp, res)
> 
> Since qcom_pci_config_ecam() doesn't look at the root bus number at
> all, is this also an implicit restriction that the root bus must be
> bus 0?  Does qcom support root buses other than 0?
> 

No, not at the moment.

> > +}
> > +
> >  static int qcom_pcie_start_link(struct dw_pcie *pci)
> >  {
> >  	struct qcom_pcie *pcie = to_qcom_pcie(pci);
> > @@ -326,6 +382,9 @@ static int qcom_pcie_start_link(struct dw_pcie *pci)
> >  		qcom_pcie_common_set_16gt_lane_margining(pci);
> >  	}
> >  
> > +	if (pci->pp.ecam_enabled)
> > +		qcom_pci_config_ecam(&pci->pp);
> 
> qcom_pcie_start_link() seems like a strange place to do this
> ECAM-related iATU configuration.  ECAM is a function of the host
> bridge, not of any particular Root Port or link.
> 

I thought qcom_pci_config_ecam() needs to be called after
dw_pcie_config_ecam_iatu() and before pci_host_probe() in dw_pcie_host_init().
Since there were no other callbacks in-between other than this start_link()
callback, Krishna has called it from here.

But looks like qcom_pci_config_ecam() could be called before
dw_pcie_config_ecam_iatu() as replied by Krishna [1]. So I will move it to
qcom_pcie_host_init().

- Mani

[1] https://lore.kernel.org/linux-pci/d12e002b-e99e-4963-a732-4873e13c5419@oss.qualcomm.com

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v9 3/4] PCI: qcom: Prepare for the DWC ECAM enablement

[Manivannan Sadhasivam]

On Fri, Sep 12, 2025 at 04:50:53PM GMT, Bjorn Helgaas wrote:
> On Tue, Sep 09, 2025 at 12:37:52PM +0530, Manivannan Sadhasivam wrote:
> > From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
> > 
> > To support the DWC ECAM mechanism, prepare the driver by performing below
> > configurations:
> > 
> >   1. Since the ELBI region will be covered by the ECAM 'config' space,
> >      override the 'elbi_base' with the address derived from 'dbi_base' and
> >      the offset from PARF_SLV_DBI_ELBI register.
> > 
> >   2. Block the transactions from the host bridge to devices other than Root
> >      Port on the root bus to return all F's. This is required when the 'CFG
> >      Shift Feature' of iATU is enabled.
> 
> FWIW, before I noticed your v9, I had updated the comments here to fix
> a few inconsistencies.  Here's the diff:
> 

I will incorporate this in next version.

- Mani

> diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
> index 7c2b2c8c61c2..962f0311a23a 100644
> --- a/drivers/pci/controller/dwc/pcie-qcom.c
> +++ b/drivers/pci/controller/dwc/pcie-qcom.c
> @@ -343,15 +343,15 @@ static void qcom_pci_config_ecam(struct dw_pcie_rp *pp)
>  	writel_relaxed(upper_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE_HI);
>  
>  	/*
> -	 * The only device on root bus is a single Root Port. So if PCI core
> -	 * tries to access any devices other than Device/Function (0.0) in Bus
> -	 * 0, the TLP will go outside of the controller to the PCI bus. But with
> -	 * CFG Shift Feature (ECAM) enabled in iATU, there is no guarantee that
> -	 * the response is going to be all F's. Hence, to make sure that the
> +	 * The only device on the root bus is a single Root Port. If we try to
> +	 * access any devices other than Device/Function 00.0 on Bus 0, the TLP
> +	 * will go outside of the controller to the PCI bus. But with CFG Shift
> +	 * Feature (ECAM) enabled in iATU, there is no guarantee that the
> +	 * response is going to be all F's. Hence, to make sure that the
>  	 * requester gets all F's response for accesses other than the Root
> -	 * Port, configure iATU to block the transactions starting from function
> -	 * 1 of the root bus to the end of the root bus (i.e from dbi_base + 4kb
> -	 * to dbi_base + 1MB).
> +	 * Port, configure iATU to block the transactions starting from
> +	 * function 1 of the root bus to the end of the root bus (i.e., from
> +	 * dbi_base + 4KB to dbi_base + 1MB).
>  	 */
>  	addr = pci->dbi_phys_addr + SZ_4K;
>  	writel_relaxed(lower_32_bits(addr), pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE);
> @@ -1385,7 +1385,7 @@ static int qcom_pcie_host_init(struct dw_pcie_rp *pp)
>  	if (pp->ecam_enabled) {
>  		/*
>  		 * Override ELBI when ECAM is enabled, as when ECAM
> -		 * is enabled ELBI moves along with the dbi config space.
> +		 * is enabled ELBI moves along with the DBI config space.
>  		 */
>  		offset = FIELD_GET(SLV_DBI_ELBI_ADDR_BASE, readl(pcie->parf + PARF_SLV_DBI_ELBI));
>  		pci->elbi_base = pci->dbi_base + offset;

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[ALOK TIWARI]

On 9/9/2025 12:37 PM, Manivannan Sadhasivam wrote:
> Currently, the driver is not making use of this CFG shift feature, thereby
> creating the iATU outbound map for each config access to the devices,
> causing latency and wasting CPU cycles.
> 
> So to avoid this, configure the controller to enable CFG shift feature by
> enabling the 'CFG Shift' bit of the 'iATU Control 2 Register'.
> 
> As a result of enabling CFG shift (ECAM), there is longer a need to map the
> DBI register space separately as the DBI region falls under the 'config'
> space used for ECAM (as DBI is used to access the Root Port).

nit : there is no longer a need

Thanks,
Alok

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Maciej W. Rozycki]

On Tue, 9 Sep 2025, Manivannan Sadhasivam wrote:

> From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
> 
> Designware databook r5.20a, sec 3.10.10.3 documents the 'CFG Shift Feature'
> of the internal Address Translation Unit (iATU). When this feature is
> enabled, it shifts/maps the BDF contained in the bits [27:12] of the target
> address in MEM TLP to become BDF of the CFG TLP. This essentially
> implements the Enhanced Configuration Address Mapping (ECAM) mechanism as
> defined in PCIe r6.0, sec 7.2.2.

 So this broke a parallel port on my HiFive Unmatched machine (a SiFive 
FU740-C000 based system), the driver no longer registers the device, no 
/dev/parport0 anymore.

 I've had to bisect it with commit a1978b692a39 ("PCI: dwc: Use custom 
pci_ops for root bus DBI vs ECAM config access") and commit fc2bc2623e3a 
("Revert "PCI: qcom: Prepare for the DWC ECAM enablement"") applied on top 
and it's affirmative it's this change, i.e. upstream commit 0da48c5b2fa7 
("PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'").

 Here's the relevant part of a diff between bootstrap logs:

--- dmesg-good.log	2025-11-28 03:41:18.943097032 +0100
+++ dmesg-bad.log	2025-11-28 03:47:29.582049781 +0100
@@ -1,5 +1,5 @@
-Booting Linux on hartid 3
-Linux version 6.17.0-rc1-00008-g4660e50cf818-dirty (macro@angie) (riscv64-linux-gnu-gcc (GCC) 13.0.0 20220602 (experimental), GNU ld (GNU Binutils) 2.38.50.20220503) #19 SMP Fri Nov 28 02:37:51 GMT 2025
+Booting Linux on hartid 1
+Linux version 6.17.0-rc1-00009-g0da48c5b2fa7-dirty (macro@angie) (riscv64-linux-gnu-gcc (GCC) 13.0.0 20220602 (experimental), GNU ld (GNU Binutils) 2.38.50.20220503) #20 SMP Fri Nov 28 02:43:00 GMT 2025
 Machine model: SiFive HiFive Unmatched A00
 SBI specification v0.3 detected
 SBI implementation ID=0x1 Version=0x9
@@ -61,7 +61,7 @@
 EFI services will not be available.
 smp: Bringing up secondary CPUs ...
 smp: Brought up 1 node, 4 CPUs
-Memory: 16383064K/16777216K available (10746K kernel code, 2200K rwdata, 4972K rodata, 537K init, 371K bss, 389448K reserved, 0K cma-reserved)
+Memory: 16383064K/16777216K available (10746K kernel code, 2200K rwdata, 4972K rodata, 536K init, 371K bss, 389448K reserved, 0K cma-reserved)
 devtmpfs: initialized
 clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
 posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)
@@ -161,6 +161,7 @@
 fu740-pcie e00000000.pcie:       IO 0x0060080000..0x006008ffff -> 0x0060080000
 fu740-pcie e00000000.pcie:      MEM 0x0060090000..0x007fffffff -> 0x0060090000
 fu740-pcie e00000000.pcie:      MEM 0x2000000000..0x3fffffffff -> 0x2000000000
+fu740-pcie e00000000.pcie: ECAM at [mem 0xdf0000000-0xdffffffff] for [bus 00-ff]
 fu740-pcie e00000000.pcie: Using 256 MSI vectors
 fu740-pcie e00000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 4096G
 fu740-pcie e00000000.pcie: cap_exp at 70
@@ -655,7 +656,7 @@
 usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.17
 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
 usb usb1: Product: xHCI Host Controller
-usb usb1: Manufacturer: Linux 6.17.0-rc1-00008-g4660e50cf818-dirty xhci-hcd
+usb usb1: Manufacturer: Linux 6.17.0-rc1-00009-g0da48c5b2fa7-dirty xhci-hcd
 usb usb1: SerialNumber: 0000:04:00.0
 hub 1-0:1.0: USB hub found
 hub 1-0:1.0: 2 ports detected
@@ -663,7 +664,7 @@
 usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.17
 usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
 usb usb2: Product: xHCI Host Controller
-usb usb2: Manufacturer: Linux 6.17.0-rc1-00008-g4660e50cf818-dirty xhci-hcd
+usb usb2: Manufacturer: Linux 6.17.0-rc1-00009-g0da48c5b2fa7-dirty xhci-hcd
 usb usb2: SerialNumber: 0000:04:00.0
 hub 2-0:1.0: USB hub found
 hub 2-0:1.0: 2 ports detected
@@ -735,8 +736,6 @@
 pcieport 0000:06:01.0: enabling bus mastering
 parport_pc 0000:07:00.0: enabling device (0000 -> 0001)
 PCI parallel port detected: 1415:c118, I/O at 0x1000(0x1008), IRQ 35
-parport0: PC-style at 0x1000 (0x1008), irq 35, using FIFO [PCSPP,TRISTATE,EPP,ECP]
-lp0: using parport0 (interrupt-driven).
 parport_pc 0000:07:00.0: vgaarb: pci_notify
 serial 0000:07:00.3: vgaarb: pci_notify
 serial 0000:07:00.3: assign IRQ: got 40

and then it goes on with insignificant changes only owing to differences 
in the order of messages produced, the kernel version ID or date stamps.  
As you can see the PCIe parallel port device continues being accessible, 
it's only the driver that doesn't pick up the device anymore.

 I'm stumped as to where it might be coming from.  Any ideas?

  Maciej

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Krishna Chaitanya Chundru]

On 11/28/2025 8:47 AM, Maciej W. Rozycki wrote:
> On Tue, 9 Sep 2025, Manivannan Sadhasivam wrote:
>
>> From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
>>
>> Designware databook r5.20a, sec 3.10.10.3 documents the 'CFG Shift Feature'
>> of the internal Address Translation Unit (iATU). When this feature is
>> enabled, it shifts/maps the BDF contained in the bits [27:12] of the target
>> address in MEM TLP to become BDF of the CFG TLP. This essentially
>> implements the Enhanced Configuration Address Mapping (ECAM) mechanism as
>> defined in PCIe r6.0, sec 7.2.2.
>   So this broke a parallel port on my HiFive Unmatched machine (a SiFive
> FU740-C000 based system), the driver no longer registers the device, no
> /dev/parport0 anymore.
Hi Maciej, can you share us lspci -vvv o/p with working & non working 
case and also can you point us parport driver. - Krishna Chaitanya.
>   I've had to bisect it with commit a1978b692a39 ("PCI: dwc: Use custom
> pci_ops for root bus DBI vs ECAM config access") and commit fc2bc2623e3a
> ("Revert "PCI: qcom: Prepare for the DWC ECAM enablement"") applied on top
> and it's affirmative it's this change, i.e. upstream commit 0da48c5b2fa7
> ("PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'").
>
>   Here's the relevant part of a diff between bootstrap logs:
>
> --- dmesg-good.log	2025-11-28 03:41:18.943097032 +0100
> +++ dmesg-bad.log	2025-11-28 03:47:29.582049781 +0100
> @@ -1,5 +1,5 @@
> -Booting Linux on hartid 3
> -Linux version 6.17.0-rc1-00008-g4660e50cf818-dirty (macro@angie) (riscv64-linux-gnu-gcc (GCC) 13.0.0 20220602 (experimental), GNU ld (GNU Binutils) 2.38.50.20220503) #19 SMP Fri Nov 28 02:37:51 GMT 2025
> +Booting Linux on hartid 1
> +Linux version 6.17.0-rc1-00009-g0da48c5b2fa7-dirty (macro@angie) (riscv64-linux-gnu-gcc (GCC) 13.0.0 20220602 (experimental), GNU ld (GNU Binutils) 2.38.50.20220503) #20 SMP Fri Nov 28 02:43:00 GMT 2025
>   Machine model: SiFive HiFive Unmatched A00
>   SBI specification v0.3 detected
>   SBI implementation ID=0x1 Version=0x9
> @@ -61,7 +61,7 @@
>   EFI services will not be available.
>   smp: Bringing up secondary CPUs ...
>   smp: Brought up 1 node, 4 CPUs
> -Memory: 16383064K/16777216K available (10746K kernel code, 2200K rwdata, 4972K rodata, 537K init, 371K bss, 389448K reserved, 0K cma-reserved)
> +Memory: 16383064K/16777216K available (10746K kernel code, 2200K rwdata, 4972K rodata, 536K init, 371K bss, 389448K reserved, 0K cma-reserved)
>   devtmpfs: initialized
>   clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
>   posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)
> @@ -161,6 +161,7 @@
>   fu740-pcie e00000000.pcie:       IO 0x0060080000..0x006008ffff -> 0x0060080000
>   fu740-pcie e00000000.pcie:      MEM 0x0060090000..0x007fffffff -> 0x0060090000
>   fu740-pcie e00000000.pcie:      MEM 0x2000000000..0x3fffffffff -> 0x2000000000
> +fu740-pcie e00000000.pcie: ECAM at [mem 0xdf0000000-0xdffffffff] for [bus 00-ff]
>   fu740-pcie e00000000.pcie: Using 256 MSI vectors
>   fu740-pcie e00000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 4096G
>   fu740-pcie e00000000.pcie: cap_exp at 70
> @@ -655,7 +656,7 @@
>   usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.17
>   usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>   usb usb1: Product: xHCI Host Controller
> -usb usb1: Manufacturer: Linux 6.17.0-rc1-00008-g4660e50cf818-dirty xhci-hcd
> +usb usb1: Manufacturer: Linux 6.17.0-rc1-00009-g0da48c5b2fa7-dirty xhci-hcd
>   usb usb1: SerialNumber: 0000:04:00.0
>   hub 1-0:1.0: USB hub found
>   hub 1-0:1.0: 2 ports detected
> @@ -663,7 +664,7 @@
>   usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.17
>   usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>   usb usb2: Product: xHCI Host Controller
> -usb usb2: Manufacturer: Linux 6.17.0-rc1-00008-g4660e50cf818-dirty xhci-hcd
> +usb usb2: Manufacturer: Linux 6.17.0-rc1-00009-g0da48c5b2fa7-dirty xhci-hcd
>   usb usb2: SerialNumber: 0000:04:00.0
>   hub 2-0:1.0: USB hub found
>   hub 2-0:1.0: 2 ports detected
> @@ -735,8 +736,6 @@
>   pcieport 0000:06:01.0: enabling bus mastering
>   parport_pc 0000:07:00.0: enabling device (0000 -> 0001)
>   PCI parallel port detected: 1415:c118, I/O at 0x1000(0x1008), IRQ 35
> -parport0: PC-style at 0x1000 (0x1008), irq 35, using FIFO [PCSPP,TRISTATE,EPP,ECP]
> -lp0: using parport0 (interrupt-driven).
>   parport_pc 0000:07:00.0: vgaarb: pci_notify
>   serial 0000:07:00.3: vgaarb: pci_notify
>   serial 0000:07:00.3: assign IRQ: got 40
>
> and then it goes on with insignificant changes only owing to differences
> in the order of messages produced, the kernel version ID or date stamps.
> As you can see the PCIe parallel port device continues being accessible,
> it's only the driver that doesn't pick up the device anymore.
>
>   I'm stumped as to where it might be coming from.  Any ideas?
>
>    Maciej

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Maciej W. Rozycki]

[-- Attachment #1: Type: text/plain, Size: 2567 bytes --]

On Fri, 28 Nov 2025, Krishna Chaitanya Chundru wrote:

> > > Designware databook r5.20a, sec 3.10.10.3 documents the 'CFG Shift
> > > Feature'
> > > of the internal Address Translation Unit (iATU). When this feature is
> > > enabled, it shifts/maps the BDF contained in the bits [27:12] of the
> > > target
> > > address in MEM TLP to become BDF of the CFG TLP. This essentially
> > > implements the Enhanced Configuration Address Mapping (ECAM) mechanism as
> > > defined in PCIe r6.0, sec 7.2.2.
> >   So this broke a parallel port on my HiFive Unmatched machine (a SiFive
> > FU740-C000 based system), the driver no longer registers the device, no
> > /dev/parport0 anymore.
> Hi Maciej, can you share us lspci -vvv o/p with working & non working case and
> also can you point us parport driver. - Krishna Chaitanya.

 I'm not sure what you mean as to the parport driver; it's standard stuff:

$ zgrep PARPORT /proc/config.gz
CONFIG_PARPORT=y
CONFIG_PARPORT_PC=y
# CONFIG_PARPORT_SERIAL is not set
CONFIG_PARPORT_PC_FIFO=y
CONFIG_PARPORT_1284=y
# CONFIG_PATA_PARPORT is not set
# CONFIG_I2C_PARPORT is not set
# CONFIG_USB_SERIAL_MOS7715_PARPORT is not set
$ 

I've attached output from `lspci -xxxx' so that you can decode it yourself 
however you need, though I fail to see anything standing out there.

 If you can't figure out what's going on here, then I'll try to poke at 
the driver to see what exactly it is that causes it to fail there, but I'm 
a little constrained on the resources and completely unfamiliar with the 
ECAM feature (and the lack of documentation for the DW IP does not help).

 I have no slightest idea why it should cause a regression such as this, 
it seems totally unrelated.  Yet it's 100% reproducible.  Could this be 
because it's the only device in the system that actually uses PCI/e port 
I/O?

# cat /proc/ioports
00000000-0000ffff : pcie@e00000000
  00001000-00002fff : PCI Bus 0000:01
    00001000-00002fff : PCI Bus 0000:02
      00001000-00002fff : PCI Bus 0000:05
        00001000-00002fff : PCI Bus 0000:06
          00001000-00001fff : PCI Bus 0000:07
          00001000-00001007 : 0000:07:00.0
          00001000-00001002 : parport0
          00001003-00001007 : parport0
          00001008-0000100b : 0000:07:00.0
          00001008-0000100a : parport0
          00002000-00002fff : PCI Bus 0000:08
          00002000-00002fff : PCI Bus 0000:09
          00002000-000020ff : 0000:09:01.0
          00002100-0000217f : 0000:09:02.0
# 

(Hmm, indentation does not appear correct to me for buses below 0000:07.)

  Maciej

[-- Attachment #2: Type: application/x-xz, Size: 4144 bytes --]

[-- Attachment #3: Type: application/x-xz, Size: 4208 bytes --]

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Krishna Chaitanya Chundru]

[-- Attachment #1: Type: text/plain, Size: 2816 bytes --]



On 11/28/2025 2:07 PM, Maciej W. Rozycki wrote:
> On Fri, 28 Nov 2025, Krishna Chaitanya Chundru wrote:
>
>>>> Designware databook r5.20a, sec 3.10.10.3 documents the 'CFG Shift
>>>> Feature'
>>>> of the internal Address Translation Unit (iATU). When this feature is
>>>> enabled, it shifts/maps the BDF contained in the bits [27:12] of the
>>>> target
>>>> address in MEM TLP to become BDF of the CFG TLP. This essentially
>>>> implements the Enhanced Configuration Address Mapping (ECAM) mechanism as
>>>> defined in PCIe r6.0, sec 7.2.2.
>>>    So this broke a parallel port on my HiFive Unmatched machine (a SiFive
>>> FU740-C000 based system), the driver no longer registers the device, no
>>> /dev/parport0 anymore.
>> Hi Maciej, can you share us lspci -vvv o/p with working & non working case and
>> also can you point us parport driver. - Krishna Chaitanya.
>   I'm not sure what you mean as to the parport driver; it's standard stuff:
>
> $ zgrep PARPORT /proc/config.gz
> CONFIG_PARPORT=y
> CONFIG_PARPORT_PC=y
> # CONFIG_PARPORT_SERIAL is not set
> CONFIG_PARPORT_PC_FIFO=y
> CONFIG_PARPORT_1284=y
> # CONFIG_PATA_PARPORT is not set
> # CONFIG_I2C_PARPORT is not set
> # CONFIG_USB_SERIAL_MOS7715_PARPORT is not set
> $
>
> I've attached output from `lspci -xxxx' so that you can decode it yourself
> however you need, though I fail to see anything standing out there.
>
>   If you can't figure out what's going on here, then I'll try to poke at
> the driver to see what exactly it is that causes it to fail there, but I'm
> a little constrained on the resources and completely unfamiliar with the
> ECAM feature (and the lack of documentation for the DW IP does not help).
>
>   I have no slightest idea why it should cause a regression such as this,
> it seems totally unrelated.  Yet it's 100% reproducible.  Could this be
> because it's the only device in the system that actually uses PCI/e port
> I/O?
Hi Maciej, Can you try attached patch and let me know if that is helping 
you or not. - Krishna Chaitanya.
> # cat /proc/ioports
> 00000000-0000ffff : pcie@e00000000
>    00001000-00002fff : PCI Bus 0000:01
>      00001000-00002fff : PCI Bus 0000:02
>        00001000-00002fff : PCI Bus 0000:05
>          00001000-00002fff : PCI Bus 0000:06
>            00001000-00001fff : PCI Bus 0000:07
>            00001000-00001007 : 0000:07:00.0
>            00001000-00001002 : parport0
>            00001003-00001007 : parport0
>            00001008-0000100b : 0000:07:00.0
>            00001008-0000100a : parport0
>            00002000-00002fff : PCI Bus 0000:08
>            00002000-00002fff : PCI Bus 0000:09
>            00002000-000020ff : 0000:09:01.0
>            00002100-0000217f : 0000:09:02.0
> #
>
> (Hmm, indentation does not appear correct to me for buses below 0000:07.)
>
>    Maciej

[-- Attachment #2: 0001-PCI-qcom-Enable-iATU-mapping-for-memory-IO-regions.patch --]
[-- Type: text/plain, Size: 4676 bytes --]

From ad002661c2e559ee3ec523b00e23948407968cd7 Mon Sep 17 00:00:00 2001
From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
Date: Fri, 28 Nov 2025 16:44:17 +0530
Subject: [PATCH] PCI: qcom: Enable iATU mapping for memory & IO regions

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
---
 .../pci/controller/dwc/pcie-designware-host.c | 24 ++++++++++++++-----
 drivers/pci/controller/dwc/pcie-designware.c  |  3 +++
 drivers/pci/controller/dwc/pcie-designware.h  |  2 +-
 3 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index e92513c5bda5..a60f1539fadc 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -36,6 +36,7 @@ static struct pci_ops dw_child_pcie_ops;
 
 #define IS_256MB_ALIGNED(x) IS_ALIGNED(x, SZ_256M)
 
+static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp);
 static const struct msi_parent_ops dw_pcie_msi_parent_ops = {
 	.required_flags		= DW_PCIE_MSI_FLAGS_REQUIRED,
 	.supported_flags	= DW_PCIE_MSI_FLAGS_SUPPORTED,
@@ -427,13 +428,17 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp)
 
 	bus = resource_list_first_type(&pp->bridge->windows, IORESOURCE_BUS);
 
+	ret = dw_pcie_iatu_setup(pp);
+	if (ret)
+		return ret;
+
 	/*
 	 * Root bus under the host bridge doesn't require any iATU configuration
 	 * as DBI region will be used to access root bus config space.
 	 * Immediate bus under Root Bus, needs type 0 iATU configuration and
 	 * remaining buses need type 1 iATU configuration.
 	 */
-	atu.index = 0;
+	atu.index = pp->ob_atu_index;
 	atu.type = PCIE_ATU_TYPE_CFG0;
 	atu.parent_bus_addr = pp->cfg0_base + SZ_1M;
 	/* 1MiB is to cover 1 (bus) * 32 (devices) * 8 (functions) */
@@ -443,19 +448,26 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp)
 	if (ret)
 		return ret;
 
+
 	bus_range_max = resource_size(bus->res);
 
 	if (bus_range_max < 2)
 		return 0;
 
+	pp->ob_atu_index++;
+
 	/* Configure remaining buses in type 1 iATU configuration */
-	atu.index = 1;
+	atu.index = pp->ob_atu_index;
 	atu.type = PCIE_ATU_TYPE_CFG1;
 	atu.parent_bus_addr = pp->cfg0_base + SZ_2M;
 	atu.size = (SZ_1M * bus_range_max) - SZ_2M;
 	atu.ctrl2 = PCIE_ATU_CFG_SHIFT_MODE_ENABLE;
 
-	return dw_pcie_prog_outbound_atu(pci, &atu);
+	ret = dw_pcie_prog_outbound_atu(pci, &atu);
+	if (!ret)
+		pp->ob_atu_index++;
+
+	return ret;
 }
 
 static int dw_pcie_create_ecam_window(struct dw_pcie_rp *pp, struct resource *res)
@@ -942,7 +954,7 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 		dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
 			 pci->num_ob_windows);
 
-	pp->msg_atu_index = i;
+	pp->ob_atu_index = i;
 
 	i = 0;
 	resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
@@ -1113,7 +1125,7 @@ static int dw_pcie_pme_turn_off(struct dw_pcie *pci)
 	void __iomem *mem;
 	int ret;
 
-	if (pci->num_ob_windows <= pci->pp.msg_atu_index)
+	if (pci->num_ob_windows <= pci->pp.ob_atu_index)
 		return -ENOSPC;
 
 	if (!pci->pp.msg_res)
@@ -1123,7 +1135,7 @@ static int dw_pcie_pme_turn_off(struct dw_pcie *pci)
 	atu.routing = PCIE_MSG_TYPE_R_BC;
 	atu.type = PCIE_ATU_TYPE_MSG;
 	atu.size = resource_size(pci->pp.msg_res);
-	atu.index = pci->pp.msg_atu_index;
+	atu.index = pci->pp.ob_atu_index;
 
 	atu.parent_bus_addr = pci->pp.msg_res->start - pci->parent_bus_offset;
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index c644216995f6..d27b469b417b 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -478,6 +478,9 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
 
 	limit_addr = parent_bus_addr + atu->size - 1;
 
+	if (atu->index > pci->num_ob_windows)
+		return -ENOSPC;
+
 	if ((limit_addr & ~pci->region_limit) != (parent_bus_addr & ~pci->region_limit) ||
 	    !IS_ALIGNED(parent_bus_addr, pci->region_align) ||
 	    !IS_ALIGNED(atu->pci_addr, pci->region_align) || !atu->size) {
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index e995f692a1ec..69d0bd8b3c57 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -423,8 +423,8 @@ struct dw_pcie_rp {
 	struct pci_host_bridge  *bridge;
 	raw_spinlock_t		lock;
 	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
+	int			ob_atu_index;
 	bool			use_atu_msg;
-	int			msg_atu_index;
 	struct resource		*msg_res;
 	bool			use_linkup_irq;
 	struct pci_eq_presets	presets;
-- 
2.34.1

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Maciej W. Rozycki]

On Fri, 28 Nov 2025, Krishna Chaitanya Chundru wrote:

> >   I have no slightest idea why it should cause a regression such as this,
> > it seems totally unrelated.  Yet it's 100% reproducible.  Could this be
> > because it's the only device in the system that actually uses PCI/e port
> > I/O?
> Hi Maciej, Can you try attached patch and let me know if that is helping you
> or not. - Krishna Chaitanya.

 No change, it's still broken, sorry.

  Maciej

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Krishna Chaitanya Chundru]

On 11/28/2025 10:46 PM, Maciej W. Rozycki wrote:
> On Fri, 28 Nov 2025, Krishna Chaitanya Chundru wrote:
>
>>>    I have no slightest idea why it should cause a regression such as this,
>>> it seems totally unrelated.  Yet it's 100% reproducible.  Could this be
>>> because it's the only device in the system that actually uses PCI/e port
>>> I/O?
>> Hi Maciej, Can you try attached patch and let me know if that is helping you
>> or not. - Krishna Chaitanya.
>   No change, it's still broken, sorry.
HI Maciej,
For the previous patch can you apply this diff and share me dmesg o/p
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -448,7 +448,6 @@ static int dw_pcie_config_ecam_iatu(struct 
dw_pcie_rp *pp)
         if (ret)
                 return ret;

-
         bus_range_max = resource_size(bus->res);

         if (bus_range_max < 2)
@@ -456,6 +455,8 @@ static int dw_pcie_config_ecam_iatu(struct 
dw_pcie_rp *pp)

         pp->ob_atu_index++;

+       dev_err(pci->dev, "Current iATU OB index %d\n", pp->ob_atu_index);
+
         /* Configure remaining buses in type 1 iATU configuration */
         atu.index = pp->ob_atu_index;
         atu.type = PCIE_ATU_TYPE_CFG1;
@@ -931,6 +932,7 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
                 }
         }

+       dev_err(pci->dev, "Current iATU OB index %d\n", i);
         if (pp->io_size) {
                 if (pci->num_ob_windows > ++i) {
                         atu.index = i;
@@ -946,6 +948,7 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
                                 return ret;
                         }
                 } else {
+                       dev_err(pci->dev, "Using shared io index %d\n", i);
                         pp->cfg0_io_shared = true;
                 }
         }

- Krishna Chaitanya.
>    Maciej

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Maciej W. Rozycki]

[-- Attachment #1: Type: text/plain, Size: 3464 bytes --]

On Sat, 29 Nov 2025, Krishna Chaitanya Chundru wrote:

> > > Hi Maciej, Can you try attached patch and let me know if that is helping
> > > you
> > > or not. - Krishna Chaitanya.
> >   No change, it's still broken, sorry.
> HI Maciej,
> For the previous patch can you apply this diff and share me dmesg o/p

 Your patch came though garbled, likely due to:

Content-Type: text/plain; charset=UTF-8; format=flowed

Please refer Documentation/process/email-clients.rst and reconfigure your 
e-mail client if possible.

 Regardless, I've fixed it up by hand and the only difference in the log, 
except for usual noise which I removed, is this:

--- dmesg-bad.log	2025-11-28 03:47:29.582049781 +0100
+++ dmesg-debug.log	2025-11-29 05:41:23.384645926 +0100
@@ -164,6 +164,8 @@
 fu740-pcie e00000000.pcie: ECAM at [mem 0xdf0000000-0xdffffffff] for [bus 00-ff]
 fu740-pcie e00000000.pcie: Using 256 MSI vectors
 fu740-pcie e00000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 4096G
+fu740-pcie e00000000.pcie: Current iATU OB index 2
+fu740-pcie e00000000.pcie: Current iATU OB index 4
 fu740-pcie e00000000.pcie: cap_exp at 70
 fu740-pcie e00000000.pcie: PCIe Gen.1 x8 link up
 fu740-pcie e00000000.pcie: changing speed back to original

I've attached a full copy of the debug log too.  I hope this helps you 
narrow the issue down or otherwise let me know what to try next.

 NB I note that code you've been poking at only refers resources of the 
IORESOURCE_MEM type.  What about IORESOURCE_IO, which seems more relevant 
here?

 Also as a quick check I've now reconfigured the defxx driver for PCI port 
I/O (which is a one-liner; the mapping used to be selectable by hand, but 
distributions got it wrong for systems w/o PCI port I/O, so I switched the 
driver to an automatic choice a few years ago, but the logic remains):

# cat /proc/ioports
00000000-0000ffff : pcie@e00000000
  00001000-00002fff : PCI Bus 0000:01
    00001000-00002fff : PCI Bus 0000:02
      00001000-00002fff : PCI Bus 0000:05
        00001000-00002fff : PCI Bus 0000:06
          00001000-00001fff : PCI Bus 0000:07
          00001000-00001007 : 0000:07:00.0
          00001000-00001002 : parport0
          00001003-00001007 : parport0
          00001008-0000100b : 0000:07:00.0
          00001008-0000100a : parport0
          00002000-00002fff : PCI Bus 0000:08
          00002000-00002fff : PCI Bus 0000:09
          00002000-000020ff : 0000:09:01.0
          00002100-0000217f : 0000:09:02.0
          00002100-0000217f : defxx
# 

and:

defxx 0000:09:02.0: assign IRQ: got 40
defxx: v1.12 2021/03/10  Lawrence V. Stefani and others
defxx 0000:09:02.0: enabling device (0000 -> 0003)
defxx 0000:09:02.0: enabling bus mastering
0000:09:02.0: DEFPA at I/O addr = 0x2100, IRQ = 40, Hardware addr = 00-60-6d-xx-xx-xx
0000:09:02.0: registered as fddi0

(as at commit 4660e50cf818) and likewise it has stopped working here from 
commit 0da48c5b2fa7 onwards:

defxx 0000:09:02.0: assign IRQ: got 40
defxx: v1.12 2021/03/10  Lawrence V. Stefani and others
defxx 0000:09:02.0: enabling device (0000 -> 0003)
defxx 0000:09:02.0: enabling bus mastering
0000:09:02.0: Could not read adapter factory MAC address!

So it's definitely nothing specific to the parport driver, but rather a 
general issue with PCI/e port I/O not working anymore.  I do hope these 
observations will let you address the issue now.  You might be able to 
reproduce it with hardware you have available even.

  Maciej

[-- Attachment #2: Type: application/x-xz, Size: 8572 bytes --]

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Manivannan Sadhasivam]

On Sat, Nov 29, 2025 at 06:04:24AM +0000, Maciej W. Rozycki wrote:
> On Sat, 29 Nov 2025, Krishna Chaitanya Chundru wrote:
> 
> > > > Hi Maciej, Can you try attached patch and let me know if that is helping
> > > > you
> > > > or not. - Krishna Chaitanya.
> > >   No change, it's still broken, sorry.
> > HI Maciej,
> > For the previous patch can you apply this diff and share me dmesg o/p
> 
>  Your patch came though garbled, likely due to:
> 
> Content-Type: text/plain; charset=UTF-8; format=flowed
> 
> Please refer Documentation/process/email-clients.rst and reconfigure your 
> e-mail client if possible.
> 
>  Regardless, I've fixed it up by hand and the only difference in the log, 
> except for usual noise which I removed, is this:
> 
> --- dmesg-bad.log	2025-11-28 03:47:29.582049781 +0100
> +++ dmesg-debug.log	2025-11-29 05:41:23.384645926 +0100
> @@ -164,6 +164,8 @@
>  fu740-pcie e00000000.pcie: ECAM at [mem 0xdf0000000-0xdffffffff] for [bus 00-ff]
>  fu740-pcie e00000000.pcie: Using 256 MSI vectors
>  fu740-pcie e00000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 4096G
> +fu740-pcie e00000000.pcie: Current iATU OB index 2
> +fu740-pcie e00000000.pcie: Current iATU OB index 4
>  fu740-pcie e00000000.pcie: cap_exp at 70
>  fu740-pcie e00000000.pcie: PCIe Gen.1 x8 link up
>  fu740-pcie e00000000.pcie: changing speed back to original
> 
> I've attached a full copy of the debug log too.  I hope this helps you 
> narrow the issue down or otherwise let me know what to try next.
> 
>  NB I note that code you've been poking at only refers resources of the 
> IORESOURCE_MEM type.  What about IORESOURCE_IO, which seems more relevant 
> here?
> 
>  Also as a quick check I've now reconfigured the defxx driver for PCI port 
> I/O (which is a one-liner; the mapping used to be selectable by hand, but 
> distributions got it wrong for systems w/o PCI port I/O, so I switched the 
> driver to an automatic choice a few years ago, but the logic remains):
> 
> # cat /proc/ioports
> 00000000-0000ffff : pcie@e00000000
>   00001000-00002fff : PCI Bus 0000:01
>     00001000-00002fff : PCI Bus 0000:02
>       00001000-00002fff : PCI Bus 0000:05
>         00001000-00002fff : PCI Bus 0000:06
>           00001000-00001fff : PCI Bus 0000:07
>           00001000-00001007 : 0000:07:00.0
>           00001000-00001002 : parport0
>           00001003-00001007 : parport0
>           00001008-0000100b : 0000:07:00.0
>           00001008-0000100a : parport0
>           00002000-00002fff : PCI Bus 0000:08
>           00002000-00002fff : PCI Bus 0000:09
>           00002000-000020ff : 0000:09:01.0
>           00002100-0000217f : 0000:09:02.0
>           00002100-0000217f : defxx
> # 
> 
> and:
> 
> defxx 0000:09:02.0: assign IRQ: got 40
> defxx: v1.12 2021/03/10  Lawrence V. Stefani and others
> defxx 0000:09:02.0: enabling device (0000 -> 0003)
> defxx 0000:09:02.0: enabling bus mastering
> 0000:09:02.0: DEFPA at I/O addr = 0x2100, IRQ = 40, Hardware addr = 00-60-6d-xx-xx-xx
> 0000:09:02.0: registered as fddi0
> 
> (as at commit 4660e50cf818) and likewise it has stopped working here from 
> commit 0da48c5b2fa7 onwards:
> 
> defxx 0000:09:02.0: assign IRQ: got 40
> defxx: v1.12 2021/03/10  Lawrence V. Stefani and others
> defxx 0000:09:02.0: enabling device (0000 -> 0003)
> defxx 0000:09:02.0: enabling bus mastering
> 0000:09:02.0: Could not read adapter factory MAC address!
> 
> So it's definitely nothing specific to the parport driver, but rather a 
> general issue with PCI/e port I/O not working anymore.  I do hope these 
> observations will let you address the issue now.  You might be able to 
> reproduce it with hardware you have available even.
> 

Yes, looks like the I/O port access is not working with the CFG Shift feature.
The spec says that both I/O and MEM TLPs should be handled by this feature, so
we are currently unsure why MEM works, but not I/O.

The issue you reported with parport_pc driver is that the driver gets probed,
but it fails to detect the parallel ports on the device. More precisely, it
fails due to the parport_SPP_supported() check in drivers/parport/parport_pc.c.
This function performs some read/write checks to make sure that the port exists,
but most likely the read value doesn't match the written one. And since there is
no log printed in this function, it just failed silently.

We will check why I/O access fails with ECAM mode and revert back asap. Since
the merge window is now open, it becomes difficult to revert the CFG shift
feature cleanly. The timing of the report also made it difficult to fix the
issue in v6.18. Hopefully, we can backport the fix once we identify the culprit.

Sorry for the inconvenience!

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Manivannan Sadhasivam]

[-- Attachment #1: Type: text/plain, Size: 5154 bytes --]

On Mon, Dec 01, 2025 at 05:21:50PM +0530, Manivannan Sadhasivam wrote:
> On Sat, Nov 29, 2025 at 06:04:24AM +0000, Maciej W. Rozycki wrote:
> > On Sat, 29 Nov 2025, Krishna Chaitanya Chundru wrote:
> > 
> > > > > Hi Maciej, Can you try attached patch and let me know if that is helping
> > > > > you
> > > > > or not. - Krishna Chaitanya.
> > > >   No change, it's still broken, sorry.
> > > HI Maciej,
> > > For the previous patch can you apply this diff and share me dmesg o/p
> > 
> >  Your patch came though garbled, likely due to:
> > 
> > Content-Type: text/plain; charset=UTF-8; format=flowed
> > 
> > Please refer Documentation/process/email-clients.rst and reconfigure your 
> > e-mail client if possible.
> > 
> >  Regardless, I've fixed it up by hand and the only difference in the log, 
> > except for usual noise which I removed, is this:
> > 
> > --- dmesg-bad.log	2025-11-28 03:47:29.582049781 +0100
> > +++ dmesg-debug.log	2025-11-29 05:41:23.384645926 +0100
> > @@ -164,6 +164,8 @@
> >  fu740-pcie e00000000.pcie: ECAM at [mem 0xdf0000000-0xdffffffff] for [bus 00-ff]
> >  fu740-pcie e00000000.pcie: Using 256 MSI vectors
> >  fu740-pcie e00000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 4096G
> > +fu740-pcie e00000000.pcie: Current iATU OB index 2
> > +fu740-pcie e00000000.pcie: Current iATU OB index 4
> >  fu740-pcie e00000000.pcie: cap_exp at 70
> >  fu740-pcie e00000000.pcie: PCIe Gen.1 x8 link up
> >  fu740-pcie e00000000.pcie: changing speed back to original
> > 
> > I've attached a full copy of the debug log too.  I hope this helps you 
> > narrow the issue down or otherwise let me know what to try next.
> > 
> >  NB I note that code you've been poking at only refers resources of the 
> > IORESOURCE_MEM type.  What about IORESOURCE_IO, which seems more relevant 
> > here?
> > 
> >  Also as a quick check I've now reconfigured the defxx driver for PCI port 
> > I/O (which is a one-liner; the mapping used to be selectable by hand, but 
> > distributions got it wrong for systems w/o PCI port I/O, so I switched the 
> > driver to an automatic choice a few years ago, but the logic remains):
> > 
> > # cat /proc/ioports
> > 00000000-0000ffff : pcie@e00000000
> >   00001000-00002fff : PCI Bus 0000:01
> >     00001000-00002fff : PCI Bus 0000:02
> >       00001000-00002fff : PCI Bus 0000:05
> >         00001000-00002fff : PCI Bus 0000:06
> >           00001000-00001fff : PCI Bus 0000:07
> >           00001000-00001007 : 0000:07:00.0
> >           00001000-00001002 : parport0
> >           00001003-00001007 : parport0
> >           00001008-0000100b : 0000:07:00.0
> >           00001008-0000100a : parport0
> >           00002000-00002fff : PCI Bus 0000:08
> >           00002000-00002fff : PCI Bus 0000:09
> >           00002000-000020ff : 0000:09:01.0
> >           00002100-0000217f : 0000:09:02.0
> >           00002100-0000217f : defxx
> > # 
> > 
> > and:
> > 
> > defxx 0000:09:02.0: assign IRQ: got 40
> > defxx: v1.12 2021/03/10  Lawrence V. Stefani and others
> > defxx 0000:09:02.0: enabling device (0000 -> 0003)
> > defxx 0000:09:02.0: enabling bus mastering
> > 0000:09:02.0: DEFPA at I/O addr = 0x2100, IRQ = 40, Hardware addr = 00-60-6d-xx-xx-xx
> > 0000:09:02.0: registered as fddi0
> > 
> > (as at commit 4660e50cf818) and likewise it has stopped working here from 
> > commit 0da48c5b2fa7 onwards:
> > 
> > defxx 0000:09:02.0: assign IRQ: got 40
> > defxx: v1.12 2021/03/10  Lawrence V. Stefani and others
> > defxx 0000:09:02.0: enabling device (0000 -> 0003)
> > defxx 0000:09:02.0: enabling bus mastering
> > 0000:09:02.0: Could not read adapter factory MAC address!
> > 
> > So it's definitely nothing specific to the parport driver, but rather a 
> > general issue with PCI/e port I/O not working anymore.  I do hope these 
> > observations will let you address the issue now.  You might be able to 
> > reproduce it with hardware you have available even.
> > 
> 
> Yes, looks like the I/O port access is not working with the CFG Shift feature.
> The spec says that both I/O and MEM TLPs should be handled by this feature, so
> we are currently unsure why MEM works, but not I/O.
> 
> The issue you reported with parport_pc driver is that the driver gets probed,
> but it fails to detect the parallel ports on the device. More precisely, it
> fails due to the parport_SPP_supported() check in drivers/parport/parport_pc.c.
> This function performs some read/write checks to make sure that the port exists,
> but most likely the read value doesn't match the written one. And since there is
> no log printed in this function, it just failed silently.
> 
> We will check why I/O access fails with ECAM mode and revert back asap. Since
> the merge window is now open, it becomes difficult to revert the CFG shift
> feature cleanly. The timing of the report also made it difficult to fix the
> issue in v6.18. Hopefully, we can backport the fix once we identify the culprit.
> 

Can you try the attached patch? It is a reworked version of Krishna's patch. I
just moved things around to check potential override issue.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

[-- Attachment #2: 0001-PCI-qcom-Enable-iATU-mapping-for-memory-IO-regions.patch --]
[-- Type: text/x-diff, Size: 5466 bytes --]

From b35d92c71d40d3f6471900372f73e07079f8ee34 Mon Sep 17 00:00:00 2001
From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
Date: Fri, 28 Nov 2025 16:44:17 +0530
Subject: [PATCH] PCI: qcom: Enable iATU mapping for memory & IO regions

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@oss.qualcomm.com>
---
 .../pci/controller/dwc/pcie-designware-host.c | 37 +++++++++++--------
 drivers/pci/controller/dwc/pcie-designware.c  |  3 ++
 drivers/pci/controller/dwc/pcie-designware.h  |  2 +-
 3 files changed, 26 insertions(+), 16 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index e92513c5bda5..cffd66d51d02 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -36,6 +36,7 @@ static struct pci_ops dw_child_pcie_ops;
 
 #define IS_256MB_ALIGNED(x) IS_ALIGNED(x, SZ_256M)
 
+static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp);
 static const struct msi_parent_ops dw_pcie_msi_parent_ops = {
 	.required_flags		= DW_PCIE_MSI_FLAGS_REQUIRED,
 	.supported_flags	= DW_PCIE_MSI_FLAGS_SUPPORTED,
@@ -433,7 +434,7 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp)
 	 * Immediate bus under Root Bus, needs type 0 iATU configuration and
 	 * remaining buses need type 1 iATU configuration.
 	 */
-	atu.index = 0;
+	atu.index = pp->ob_atu_index;
 	atu.type = PCIE_ATU_TYPE_CFG0;
 	atu.parent_bus_addr = pp->cfg0_base + SZ_1M;
 	/* 1MiB is to cover 1 (bus) * 32 (devices) * 8 (functions) */
@@ -448,14 +449,20 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp)
 	if (bus_range_max < 2)
 		return 0;
 
+	pp->ob_atu_index++;
+
 	/* Configure remaining buses in type 1 iATU configuration */
-	atu.index = 1;
+	atu.index = pp->ob_atu_index;
 	atu.type = PCIE_ATU_TYPE_CFG1;
 	atu.parent_bus_addr = pp->cfg0_base + SZ_2M;
 	atu.size = (SZ_1M * bus_range_max) - SZ_2M;
 	atu.ctrl2 = PCIE_ATU_CFG_SHIFT_MODE_ENABLE;
 
-	return dw_pcie_prog_outbound_atu(pci, &atu);
+	ret = dw_pcie_prog_outbound_atu(pci, &atu);
+	if (!ret)
+		pp->ob_atu_index++;
+
+	return ret;
 }
 
 static int dw_pcie_create_ecam_window(struct dw_pcie_rp *pp, struct resource *res)
@@ -630,14 +637,6 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 	if (ret)
 		goto err_free_msi;
 
-	if (pp->ecam_enabled) {
-		ret = dw_pcie_config_ecam_iatu(pp);
-		if (ret) {
-			dev_err(dev, "Failed to configure iATU in ECAM mode\n");
-			goto err_free_msi;
-		}
-	}
-
 	/*
 	 * Allocate the resource for MSG TLP before programming the iATU
 	 * outbound window in dw_pcie_setup_rc(). Since the allocation depends
@@ -942,7 +941,7 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 		dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
 			 pci->num_ob_windows);
 
-	pp->msg_atu_index = i;
+	pp->ob_atu_index = i;
 
 	i = 0;
 	resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
@@ -1086,12 +1085,20 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 	 * the platform uses its own address translation component rather than
 	 * ATU, so we should not program the ATU here.
 	 */
-	if (pp->bridge->child_ops == &dw_child_pcie_ops) {
+	if (pp->bridge->child_ops == &dw_child_pcie_ops || pp->ecam_enabled) {
 		ret = dw_pcie_iatu_setup(pp);
 		if (ret)
 			return ret;
 	}
 
+	if (pp->ecam_enabled) {
+		ret = dw_pcie_config_ecam_iatu(pp);
+		if (ret) {
+			dev_err(pci->dev, "Failed to configure iATU in ECAM mode\n");
+			return ret;
+		}
+	}
+
 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
 
 	/* Program correct class for RC */
@@ -1113,7 +1120,7 @@ static int dw_pcie_pme_turn_off(struct dw_pcie *pci)
 	void __iomem *mem;
 	int ret;
 
-	if (pci->num_ob_windows <= pci->pp.msg_atu_index)
+	if (pci->num_ob_windows <= pci->pp.ob_atu_index)
 		return -ENOSPC;
 
 	if (!pci->pp.msg_res)
@@ -1123,7 +1130,7 @@ static int dw_pcie_pme_turn_off(struct dw_pcie *pci)
 	atu.routing = PCIE_MSG_TYPE_R_BC;
 	atu.type = PCIE_ATU_TYPE_MSG;
 	atu.size = resource_size(pci->pp.msg_res);
-	atu.index = pci->pp.msg_atu_index;
+	atu.index = pci->pp.ob_atu_index;
 
 	atu.parent_bus_addr = pci->pp.msg_res->start - pci->parent_bus_offset;
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index c644216995f6..d27b469b417b 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -478,6 +478,9 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
 
 	limit_addr = parent_bus_addr + atu->size - 1;
 
+	if (atu->index > pci->num_ob_windows)
+		return -ENOSPC;
+
 	if ((limit_addr & ~pci->region_limit) != (parent_bus_addr & ~pci->region_limit) ||
 	    !IS_ALIGNED(parent_bus_addr, pci->region_align) ||
 	    !IS_ALIGNED(atu->pci_addr, pci->region_align) || !atu->size) {
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index e995f692a1ec..69d0bd8b3c57 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -423,8 +423,8 @@ struct dw_pcie_rp {
 	struct pci_host_bridge  *bridge;
 	raw_spinlock_t		lock;
 	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
+	int			ob_atu_index;
 	bool			use_atu_msg;
-	int			msg_atu_index;
 	struct resource		*msg_res;
 	bool			use_linkup_irq;
 	struct pci_eq_presets	presets;
-- 
2.48.1

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Maciej W. Rozycki]

On Mon, 1 Dec 2025, Manivannan Sadhasivam wrote:

> > > So it's definitely nothing specific to the parport driver, but rather a 
> > > general issue with PCI/e port I/O not working anymore.  I do hope these 
> > > observations will let you address the issue now.  You might be able to 
> > > reproduce it with hardware you have available even.
> > > 
> > 
> > Yes, looks like the I/O port access is not working with the CFG Shift feature.
> > The spec says that both I/O and MEM TLPs should be handled by this feature, so
> > we are currently unsure why MEM works, but not I/O.

 As I say, last time I checked (for another reason) documentation was not 
available to the general public, so I can't help with that.

> > The issue you reported with parport_pc driver is that the driver gets probed,
> > but it fails to detect the parallel ports on the device. More precisely, it
> > fails due to the parport_SPP_supported() check in drivers/parport/parport_pc.c.
> > This function performs some read/write checks to make sure that the port exists,
> > but most likely the read value doesn't match the written one. And since there is
> > no log printed in this function, it just failed silently.

 Whatever the exact transaction conditions are port I/O TLPs seem not to 
make it through to the requested target device anymore.

 FWIW the defxx driver issues a command to the device's command register 
and wants to see a successful completion status in the status register 
before retrieving the MAC address via the data register.  So it's not a 
simple case of poking at a register and reading it back, but the end 
result is the same: the device cannot be talked to.

> > We will check why I/O access fails with ECAM mode and revert back asap. Since
> > the merge window is now open, it becomes difficult to revert the CFG shift
> > feature cleanly. The timing of the report also made it difficult to fix the
> > issue in v6.18. Hopefully, we can backport the fix once we identify the culprit.

 No worries, I've been around for long enough (short of 30 years) to know 
the process.

 FWIW the original change would've best been reverted for 6.18 as a fatal 
regression, however port I/O is uncommon enough nowadays we can defer any 
final decision to 6.19 I suppose.  I'm glad I've tripped over this in the 
first place as I'm not eager to upgrade all my lab devices all the time, 
and it was owing to another issue only that I chose this moment to move 
forward, not so long after the original commit.

> Can you try the attached patch? It is a reworked version of Krishna's patch. I
> just moved things around to check potential override issue.

 No change in behaviour, sorry.  I suppose it's this range of host address 
decoding:

fu740-pcie e00000000.pcie:       IO 0x0060080000..0x006008ffff -> 0x0060080000

aka:

pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x60080000-0x6008ffff])

that you're after.  Are you sure your code discovers it correctly?  As I 
say I can only see IORESOURCE_MEM references and no IORESOURCE_IO ones as 
would be appropriate for the root bus resource quoted.

  Maciej

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Manivannan Sadhasivam]

[-- Attachment #1: Type: text/plain, Size: 4119 bytes --]

On Mon, Dec 01, 2025 at 04:42:42PM +0000, Maciej W. Rozycki wrote:
> On Mon, 1 Dec 2025, Manivannan Sadhasivam wrote:
> 
> > > > So it's definitely nothing specific to the parport driver, but rather a 
> > > > general issue with PCI/e port I/O not working anymore.  I do hope these 
> > > > observations will let you address the issue now.  You might be able to 
> > > > reproduce it with hardware you have available even.
> > > > 
> > > 
> > > Yes, looks like the I/O port access is not working with the CFG Shift feature.
> > > The spec says that both I/O and MEM TLPs should be handled by this feature, so
> > > we are currently unsure why MEM works, but not I/O.
> 
>  As I say, last time I checked (for another reason) documentation was not 
> available to the general public, so I can't help with that.
> 

Sure. I know that the DWC documentation is well secured behind firewalls. So not
asking for help here.

> > > The issue you reported with parport_pc driver is that the driver gets probed,
> > > but it fails to detect the parallel ports on the device. More precisely, it
> > > fails due to the parport_SPP_supported() check in drivers/parport/parport_pc.c.
> > > This function performs some read/write checks to make sure that the port exists,
> > > but most likely the read value doesn't match the written one. And since there is
> > > no log printed in this function, it just failed silently.
> 
>  Whatever the exact transaction conditions are port I/O TLPs seem not to 
> make it through to the requested target device anymore.
> 
>  FWIW the defxx driver issues a command to the device's command register 
> and wants to see a successful completion status in the status register 
> before retrieving the MAC address via the data register.  So it's not a 
> simple case of poking at a register and reading it back, but the end 
> result is the same: the device cannot be talked to.
> 
> > > We will check why I/O access fails with ECAM mode and revert back asap. Since
> > > the merge window is now open, it becomes difficult to revert the CFG shift
> > > feature cleanly. The timing of the report also made it difficult to fix the
> > > issue in v6.18. Hopefully, we can backport the fix once we identify the culprit.
> 
>  No worries, I've been around for long enough (short of 30 years) to know 
> the process.
> 
>  FWIW the original change would've best been reverted for 6.18 as a fatal 
> regression, however port I/O is uncommon enough nowadays we can defer any 
> final decision to 6.19 I suppose.  I'm glad I've tripped over this in the 
> first place as I'm not eager to upgrade all my lab devices all the time, 
> and it was owing to another issue only that I chose this moment to move 
> forward, not so long after the original commit.
> 
> > Can you try the attached patch? It is a reworked version of Krishna's patch. I
> > just moved things around to check potential override issue.
> 
>  No change in behaviour, sorry.  I suppose it's this range of host address 
> decoding:
> 
> fu740-pcie e00000000.pcie:       IO 0x0060080000..0x006008ffff -> 0x0060080000
> 
> aka:
> 
> pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x60080000-0x6008ffff])
> 
> that you're after.  Are you sure your code discovers it correctly?  As I 
> say I can only see IORESOURCE_MEM references and no IORESOURCE_IO ones as 
> would be appropriate for the root bus resource quoted.
> 

The I/O resource is discovered by the driver correctly as seen from the logs:

pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x60080000-0x6008ffff])
pci_bus 0000:00: root bus resource [mem 0x60090000-0x7fffffff]
pci_bus 0000:00: root bus resource [mem 0x2000000000-0x3fffffffff pref]

But we believe that the iATU is not programmed for the I/O port, resulting in
the I/O access not going out to the device.

Krishna found an issue in the previous patch that got shared. So I've attached a
new one. Could you please try and let us know? If it didn't help, please share
the dmesg log that will have some more info.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

[-- Attachment #2: 0001-PCI-qcom-Enable-iATU-mapping-for-memory-IO-regions.patch --]
[-- Type: text/x-diff, Size: 5930 bytes --]

From 5acab1289f8fd20ecead7c517e7e282584946ff0 Mon Sep 17 00:00:00 2001
From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
Date: Fri, 28 Nov 2025 16:44:17 +0530
Subject: [PATCH] PCI: qcom: Enable iATU mapping for memory & IO regions

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@oss.qualcomm.com>
---
 .../pci/controller/dwc/pcie-designware-host.c | 42 ++++++++++++-------
 drivers/pci/controller/dwc/pcie-designware.c  |  3 ++
 drivers/pci/controller/dwc/pcie-designware.h  |  2 +-
 3 files changed, 31 insertions(+), 16 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index e92513c5bda5..d977bb92cf3b 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -36,6 +36,7 @@ static struct pci_ops dw_child_pcie_ops;
 
 #define IS_256MB_ALIGNED(x) IS_ALIGNED(x, SZ_256M)
 
+static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp);
 static const struct msi_parent_ops dw_pcie_msi_parent_ops = {
 	.required_flags		= DW_PCIE_MSI_FLAGS_REQUIRED,
 	.supported_flags	= DW_PCIE_MSI_FLAGS_SUPPORTED,
@@ -433,7 +434,7 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp)
 	 * Immediate bus under Root Bus, needs type 0 iATU configuration and
 	 * remaining buses need type 1 iATU configuration.
 	 */
-	atu.index = 0;
+	atu.index = pp->ob_atu_index;
 	atu.type = PCIE_ATU_TYPE_CFG0;
 	atu.parent_bus_addr = pp->cfg0_base + SZ_1M;
 	/* 1MiB is to cover 1 (bus) * 32 (devices) * 8 (functions) */
@@ -448,14 +449,20 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp)
 	if (bus_range_max < 2)
 		return 0;
 
+	pp->ob_atu_index++;
+
 	/* Configure remaining buses in type 1 iATU configuration */
-	atu.index = 1;
+	atu.index = pp->ob_atu_index;
 	atu.type = PCIE_ATU_TYPE_CFG1;
 	atu.parent_bus_addr = pp->cfg0_base + SZ_2M;
 	atu.size = (SZ_1M * bus_range_max) - SZ_2M;
 	atu.ctrl2 = PCIE_ATU_CFG_SHIFT_MODE_ENABLE;
 
-	return dw_pcie_prog_outbound_atu(pci, &atu);
+	ret = dw_pcie_prog_outbound_atu(pci, &atu);
+	if (!ret)
+		pp->ob_atu_index++;
+
+	return ret;
 }
 
 static int dw_pcie_create_ecam_window(struct dw_pcie_rp *pp, struct resource *res)
@@ -630,14 +637,6 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 	if (ret)
 		goto err_free_msi;
 
-	if (pp->ecam_enabled) {
-		ret = dw_pcie_config_ecam_iatu(pp);
-		if (ret) {
-			dev_err(dev, "Failed to configure iATU in ECAM mode\n");
-			goto err_free_msi;
-		}
-	}
-
 	/*
 	 * Allocate the resource for MSG TLP before programming the iATU
 	 * outbound window in dw_pcie_setup_rc(). Since the allocation depends
@@ -919,6 +918,8 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 		}
 	}
 
+	dev_info(pci->dev, "%s: %d MEM index: %d", __func__, __LINE__, i);
+
 	if (pp->io_size) {
 		if (pci->num_ob_windows > ++i) {
 			atu.index = i;
@@ -936,13 +937,16 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
 		} else {
 			pp->cfg0_io_shared = true;
 		}
+
+		dev_info(pci->dev, "%s: %d I/O index: %d", __func__, __LINE__, i);
 	}
 
 	if (pci->num_ob_windows <= i)
 		dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
 			 pci->num_ob_windows);
 
-	pp->msg_atu_index = i;
+	pp->ob_atu_index = ++i;
+	dev_info(pci->dev, "%s: %d Final index: %d", __func__, __LINE__, i);
 
 	i = 0;
 	resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
@@ -1086,12 +1090,20 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 	 * the platform uses its own address translation component rather than
 	 * ATU, so we should not program the ATU here.
 	 */
-	if (pp->bridge->child_ops == &dw_child_pcie_ops) {
+	if (pp->bridge->child_ops == &dw_child_pcie_ops || pp->ecam_enabled) {
 		ret = dw_pcie_iatu_setup(pp);
 		if (ret)
 			return ret;
 	}
 
+	if (pp->ecam_enabled) {
+		ret = dw_pcie_config_ecam_iatu(pp);
+		if (ret) {
+			dev_err(pci->dev, "Failed to configure iATU in ECAM mode\n");
+			return ret;
+		}
+	}
+
 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
 
 	/* Program correct class for RC */
@@ -1113,7 +1125,7 @@ static int dw_pcie_pme_turn_off(struct dw_pcie *pci)
 	void __iomem *mem;
 	int ret;
 
-	if (pci->num_ob_windows <= pci->pp.msg_atu_index)
+	if (pci->num_ob_windows <= pci->pp.ob_atu_index)
 		return -ENOSPC;
 
 	if (!pci->pp.msg_res)
@@ -1123,7 +1135,7 @@ static int dw_pcie_pme_turn_off(struct dw_pcie *pci)
 	atu.routing = PCIE_MSG_TYPE_R_BC;
 	atu.type = PCIE_ATU_TYPE_MSG;
 	atu.size = resource_size(pci->pp.msg_res);
-	atu.index = pci->pp.msg_atu_index;
+	atu.index = pci->pp.ob_atu_index;
 
 	atu.parent_bus_addr = pci->pp.msg_res->start - pci->parent_bus_offset;
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index c644216995f6..d27b469b417b 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -478,6 +478,9 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
 
 	limit_addr = parent_bus_addr + atu->size - 1;
 
+	if (atu->index > pci->num_ob_windows)
+		return -ENOSPC;
+
 	if ((limit_addr & ~pci->region_limit) != (parent_bus_addr & ~pci->region_limit) ||
 	    !IS_ALIGNED(parent_bus_addr, pci->region_align) ||
 	    !IS_ALIGNED(atu->pci_addr, pci->region_align) || !atu->size) {
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index e995f692a1ec..69d0bd8b3c57 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -423,8 +423,8 @@ struct dw_pcie_rp {
 	struct pci_host_bridge  *bridge;
 	raw_spinlock_t		lock;
 	DECLARE_BITMAP(msi_irq_in_use, MAX_MSI_IRQS);
+	int			ob_atu_index;
 	bool			use_atu_msg;
-	int			msg_atu_index;
 	struct resource		*msg_res;
 	bool			use_linkup_irq;
 	struct pci_eq_presets	presets;
-- 
2.48.1

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Maciej W. Rozycki]

On Tue, 2 Dec 2025, Manivannan Sadhasivam wrote:

> >  No change in behaviour, sorry.  I suppose it's this range of host address 
> > decoding:
> > 
> > fu740-pcie e00000000.pcie:       IO 0x0060080000..0x006008ffff -> 0x0060080000
> > 
> > aka:
> > 
> > pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x60080000-0x6008ffff])
> > 
> > that you're after.  Are you sure your code discovers it correctly?  As I 
> > say I can only see IORESOURCE_MEM references and no IORESOURCE_IO ones as 
> > would be appropriate for the root bus resource quoted.
> 
> The I/O resource is discovered by the driver correctly as seen from the logs:
> 
> pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x60080000-0x6008ffff])
> pci_bus 0000:00: root bus resource [mem 0x60090000-0x7fffffff]
> pci_bus 0000:00: root bus resource [mem 0x2000000000-0x3fffffffff pref]
> 
> But we believe that the iATU is not programmed for the I/O port, resulting in
> the I/O access not going out to the device.
> 
> Krishna found an issue in the previous patch that got shared. So I've attached a
> new one. Could you please try and let us know? If it didn't help, please share
> the dmesg log that will have some more info.

 This does work correctly, thank you; see the log diff below (I checked 
the defxx driver separately too).  Please make a proper submission and 
I'll give it a Tested-by: tag after final verification against 6.18.

--- dmesg-bad.log	2025-11-28 03:47:29.582049781 +0100
+++ dmesg-fixed.log	2025-12-02 13:58:56.627947450 +0100
@@ -1,5 +1,5 @@
-Booting Linux on hartid 1
-Linux version 6.17.0-rc1-00009-g0da48c5b2fa7-dirty (macro@angie) (riscv64-linux-gnu-gcc (GCC) 13.0.0 20220602 (experimental), GNU ld (GNU Binutils) 2.38.50.20220503) #20 SMP Fri Nov 28 02:43:00 GMT 2025
+Booting Linux on hartid 4
+Linux version 6.17.0-rc1-00009-g0da48c5b2fa7-dirty (macro@angie) (riscv64-linux-gnu-gcc (GCC) 13.0.0 20220602 (experimental), GNU ld (GNU Binutils) 2.38.50.20220503) #32 SMP Tue Dec  2 12:46:23 GMT 2025
 Machine model: SiFive HiFive Unmatched A00
 SBI specification v0.3 detected
 SBI implementation ID=0x1 Version=0x9
@@ -164,6 +164,9 @@
 fu740-pcie e00000000.pcie: ECAM at [mem 0xdf0000000-0xdffffffff] for [bus 00-ff]
 fu740-pcie e00000000.pcie: Using 256 MSI vectors
 fu740-pcie e00000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 4096G
+fu740-pcie e00000000.pcie: dw_pcie_iatu_setup: 921 MEM index: 2
+fu740-pcie e00000000.pcie: dw_pcie_iatu_setup: 941 I/O index: 3
+fu740-pcie e00000000.pcie: dw_pcie_iatu_setup: 949 Final index: 4
 fu740-pcie e00000000.pcie: cap_exp at 70
 fu740-pcie e00000000.pcie: PCIe Gen.1 x8 link up
 fu740-pcie e00000000.pcie: changing speed back to original
@@ -736,6 +739,8 @@
 pcieport 0000:06:01.0: enabling bus mastering
 parport_pc 0000:07:00.0: enabling device (0000 -> 0001)
 PCI parallel port detected: 1415:c118, I/O at 0x1000(0x1008), IRQ 35
+parport0: PC-style at 0x1000 (0x1008), irq 35, using FIFO [PCSPP,TRISTATE,EPP,ECP]
+lp0: using parport0 (interrupt-driven).
 parport_pc 0000:07:00.0: vgaarb: pci_notify
 serial 0000:07:00.3: vgaarb: pci_notify
 serial 0000:07:00.3: assign IRQ: got 40

  Maciej

Re: [PATCH v9 4/4] PCI: dwc: Support ECAM mechanism by enabling iATU 'CFG Shift Feature'

[Manivannan Sadhasivam]

On Tue, Dec 02, 2025 at 01:39:14PM +0000, Maciej W. Rozycki wrote:
> On Tue, 2 Dec 2025, Manivannan Sadhasivam wrote:
> 
> > >  No change in behaviour, sorry.  I suppose it's this range of host address 
> > > decoding:
> > > 
> > > fu740-pcie e00000000.pcie:       IO 0x0060080000..0x006008ffff -> 0x0060080000
> > > 
> > > aka:
> > > 
> > > pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x60080000-0x6008ffff])
> > > 
> > > that you're after.  Are you sure your code discovers it correctly?  As I 
> > > say I can only see IORESOURCE_MEM references and no IORESOURCE_IO ones as 
> > > would be appropriate for the root bus resource quoted.
> > 
> > The I/O resource is discovered by the driver correctly as seen from the logs:
> > 
> > pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0x60080000-0x6008ffff])
> > pci_bus 0000:00: root bus resource [mem 0x60090000-0x7fffffff]
> > pci_bus 0000:00: root bus resource [mem 0x2000000000-0x3fffffffff pref]
> > 
> > But we believe that the iATU is not programmed for the I/O port, resulting in
> > the I/O access not going out to the device.
> > 
> > Krishna found an issue in the previous patch that got shared. So I've attached a
> > new one. Could you please try and let us know? If it didn't help, please share
> > the dmesg log that will have some more info.
> 
>  This does work correctly, thank you; see the log diff below (I checked 
> the defxx driver separately too).

Cool! Thanks a lot for testing and sorry for the breakage once again. Btw, the
issue that was fixed in the last diff has been present for some time. It was
fortunate that you didn't hit that before.

>  Please make a proper submission and 
> I'll give it a Tested-by: tag after final verification against 6.18.
> 

Sure thing. Krishna will post the fix(es) as there are two separate issues and
will CC you.

- Mani

-- 
மணிவண்ணன் சதாசிவம்