[PATCH v24 00/16] PCI: dwc: rcar-gen4: Add R-Car Gen4 PCIe support

[PATCH v24 00/16] PCI: dwc: rcar-gen4: Add R-Car Gen4 PCIe support

[Yoshihiro Shimoda]

Add R-Car S4-8 (R-Car Gen4) PCIe controller for both host and endpoint modes.
To support them, modify PCIe DesignWare common codes.

Changes from v23:
https://lore.kernel.org/linux-pci/20230926122431.3974714-1-yoshihiro.shimoda.uh@renesas.com/
 - Based on the latest pci.git / next branch.
 - Remove Link tag and Krzysztof's s-o-b tags from all patches.
 - Drop "PCI: dwc: Expose dw_pcie_write_dbi2() to module".
 - Add "PCI: dwc: Disable two BARs to avoid unnecessary memory assignment".
 - Add Reviewed-by in the patch 0[137]/16 and the patch 1[24]/16.
 - Fix typo in the patch 14/16.
 - Fix error messages and commit descriptions in the patch 1[234]/16.
 - Change the subjects in the patch 1[34]/16.
 - Use aplhabetical order on Kconfig in the patch 1[34]/16.
 - Drop DWC spec code in the patch 13/16. 

Changes from v22:
https://lore.kernel.org/linux-pci/20230925072130.3901087-1-yoshihiro.shimoda.uh@renesas.com/
 - Based on the latest pci.git / next branch.
 - Add Reviewed-by in the patch 12/16.
 - Move a few endpoint related code from the patch 13/16 to 14/16.
 - Change subjects in the patch 1[34]/16.
 - Modify the code for readability in the patch 14/16.

Changes from v21:
https://lore.kernel.org/linux-pci/20230922065331.3806925-1-yoshihiro.shimoda.uh@renesas.com/
 - Based on the latest pci.git / next branch.
 - Fix -Wmissing-prototypes warnings that was reported by kernel test robot:
   https://lore.kernel.org/linux-pci/202309222125.KiN4nFhD-lkp@intel.com/

Yoshihiro Shimoda (16):
  PCI: dwc: endpoint: Add multiple PFs support for dbi2
  PCI: dwc: Add dw_pcie_link_set_max_link_width()
  PCI: dwc: Add missing PCI_EXP_LNKCAP_MLW handling
  PCI: tegra194: Drop PCI_EXP_LNKSTA_NLW setting
  PCI: dwc: Add EDMA_UNROLL capability flag
  PCI: dwc: Expose dw_pcie_ep_exit() to module
  PCI: dwc: endpoint: Introduce .pre_init() and .deinit()
  PCI: dwc: Disable two BARs to avoid unnecessary memory assignment
  dt-bindings: PCI: dwc: Update maxItems of reg and reg-names
  dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Host
  dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Endpoint
  PCI: Add T_PVPERL macro
  PCI: rcar-gen4: Add R-Car Gen4 PCIe controller support for host mode
  PCI: rcar-gen4: Add endpoint mode support
  MAINTAINERS: Update PCI DRIVER FOR RENESAS R-CAR for R-Car Gen4
  misc: pci_endpoint_test: Add Device ID for R-Car S4-8 PCIe controller

 .../bindings/pci/rcar-gen4-pci-ep.yaml        | 115 ++++
 .../bindings/pci/rcar-gen4-pci-host.yaml      | 127 +++++
 .../bindings/pci/snps,dw-pcie-common.yaml     |   4 +-
 .../bindings/pci/snps,dw-pcie-ep.yaml         |   4 +-
 .../devicetree/bindings/pci/snps,dw-pcie.yaml |   4 +-
 MAINTAINERS                                   |   1 +
 drivers/misc/pci_endpoint_test.c              |   4 +
 drivers/pci/controller/dwc/Kconfig            |  25 +
 drivers/pci/controller/dwc/Makefile           |   1 +
 .../pci/controller/dwc/pcie-designware-ep.c   |  45 +-
 .../pci/controller/dwc/pcie-designware-host.c |   8 +
 drivers/pci/controller/dwc/pcie-designware.c  | 101 ++--
 drivers/pci/controller/dwc/pcie-designware.h  |   8 +-
 drivers/pci/controller/dwc/pcie-rcar-gen4.c   | 517 ++++++++++++++++++
 drivers/pci/controller/dwc/pcie-tegra194.c    |   6 -
 drivers/pci/pci.h                             |   3 +
 16 files changed, 904 insertions(+), 69 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/pci/rcar-gen4-pci-ep.yaml
 create mode 100644 Documentation/devicetree/bindings/pci/rcar-gen4-pci-host.yaml
 create mode 100644 drivers/pci/controller/dwc/pcie-rcar-gen4.c

-- 
2.25.1

[PATCH v24 01/16] PCI: dwc: endpoint: Add multiple PFs support for dbi2

[Yoshihiro Shimoda]

The commit 24ede430fa49 ("PCI: designware-ep: Add multiple PFs support
for DWC") added .func_conf_select() to get the configuration space of
different PFs and assumed that the offsets between dbi and dbi2 would
be the same.

However, Renesas R-Car Gen4 PCIe controllers have different offsets of
function 1: dbi (+0x1000) and dbi2 (+0x800). To get the offset for dbi2,
add .get_dbi2_offset() and dw_pcie_ep_get_dbi2_offset().

Note:
 - .func_conf_select() should be renamed later.
 - dw_pcie_ep_get_dbi2_offset() will call .func_conf_select()
   if .get_dbi2_offset() doesn't exist for backward compatibility.
 - dw_pcie_writeX_{dbi/dbi2} APIs accepted the func_no argument,
   so that these offset calculations are contained in the API
   definitions itself as it should.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 .../pci/controller/dwc/pcie-designware-ep.c   | 32 ++++++++++++++-----
 drivers/pci/controller/dwc/pcie-designware.h  |  1 +
 2 files changed, 25 insertions(+), 8 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index f9182f8d552f..851538ddec0a 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -52,21 +52,35 @@ static unsigned int dw_pcie_ep_func_select(struct dw_pcie_ep *ep, u8 func_no)
 	return func_offset;
 }
 
+static unsigned int dw_pcie_ep_get_dbi2_offset(struct dw_pcie_ep *ep, u8 func_no)
+{
+	unsigned int dbi2_offset = 0;
+
+	if (ep->ops->get_dbi2_offset)
+		dbi2_offset = ep->ops->get_dbi2_offset(ep, func_no);
+	else if (ep->ops->func_conf_select)     /* for backward compatibility */
+		dbi2_offset = ep->ops->func_conf_select(ep, func_no);
+
+	return dbi2_offset;
+}
+
 static void __dw_pcie_ep_reset_bar(struct dw_pcie *pci, u8 func_no,
 				   enum pci_barno bar, int flags)
 {
-	u32 reg;
-	unsigned int func_offset = 0;
+	unsigned int func_offset, dbi2_offset;
 	struct dw_pcie_ep *ep = &pci->ep;
+	u32 reg, reg_dbi2;
 
 	func_offset = dw_pcie_ep_func_select(ep, func_no);
+	dbi2_offset = dw_pcie_ep_get_dbi2_offset(ep, func_no);
 
 	reg = func_offset + PCI_BASE_ADDRESS_0 + (4 * bar);
+	reg_dbi2 = dbi2_offset + PCI_BASE_ADDRESS_0 + (4 * bar);
 	dw_pcie_dbi_ro_wr_en(pci);
-	dw_pcie_writel_dbi2(pci, reg, 0x0);
+	dw_pcie_writel_dbi2(pci, reg_dbi2, 0x0);
 	dw_pcie_writel_dbi(pci, reg, 0x0);
 	if (flags & PCI_BASE_ADDRESS_MEM_TYPE_64) {
-		dw_pcie_writel_dbi2(pci, reg + 4, 0x0);
+		dw_pcie_writel_dbi2(pci, reg_dbi2 + 4, 0x0);
 		dw_pcie_writel_dbi(pci, reg + 4, 0x0);
 	}
 	dw_pcie_dbi_ro_wr_dis(pci);
@@ -228,16 +242,18 @@ static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
 {
 	struct dw_pcie_ep *ep = epc_get_drvdata(epc);
 	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+	unsigned int func_offset, dbi2_offset;
 	enum pci_barno bar = epf_bar->barno;
 	size_t size = epf_bar->size;
 	int flags = epf_bar->flags;
-	unsigned int func_offset = 0;
+	u32 reg, reg_dbi2;
 	int ret, type;
-	u32 reg;
 
 	func_offset = dw_pcie_ep_func_select(ep, func_no);
+	dbi2_offset = dw_pcie_ep_get_dbi2_offset(ep, func_no);
 
 	reg = PCI_BASE_ADDRESS_0 + (4 * bar) + func_offset;
+	reg_dbi2 = PCI_BASE_ADDRESS_0 + (4 * bar) + dbi2_offset;
 
 	if (!(flags & PCI_BASE_ADDRESS_SPACE))
 		type = PCIE_ATU_TYPE_MEM;
@@ -253,11 +269,11 @@ static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
 
 	dw_pcie_dbi_ro_wr_en(pci);
 
-	dw_pcie_writel_dbi2(pci, reg, lower_32_bits(size - 1));
+	dw_pcie_writel_dbi2(pci, reg_dbi2, lower_32_bits(size - 1));
 	dw_pcie_writel_dbi(pci, reg, flags);
 
 	if (flags & PCI_BASE_ADDRESS_MEM_TYPE_64) {
-		dw_pcie_writel_dbi2(pci, reg + 4, upper_32_bits(size - 1));
+		dw_pcie_writel_dbi2(pci, reg_dbi2 + 4, upper_32_bits(size - 1));
 		dw_pcie_writel_dbi(pci, reg + 4, 0);
 	}
 
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index ef0b2efa9f93..6189884b4efa 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -341,6 +341,7 @@ struct dw_pcie_ep_ops {
 	 * driver.
 	 */
 	unsigned int (*func_conf_select)(struct dw_pcie_ep *ep, u8 func_no);
+	unsigned int (*get_dbi2_offset)(struct dw_pcie_ep *ep, u8 func_no);
 };
 
 struct dw_pcie_ep_func {
-- 
2.25.1

[PATCH v24 02/16] PCI: dwc: Add dw_pcie_link_set_max_link_width()

[Yoshihiro Shimoda]

This is a preparation before adding the Max-Link-width capability
setup which would in its turn complete the max-link-width setup
procedure defined by Synopsys in the HW-manual.

Seeing there is a max-link-speed setup method defined in the DW PCIe
core driver it would be good to have a similar function for the link
width setup.

That's why we need to define a dedicated function first from already
implemented but incomplete link-width setting up code.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
---
 drivers/pci/controller/dwc/pcie-designware.c | 86 ++++++++++----------
 1 file changed, 41 insertions(+), 45 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 1c1c7348972b..da4aba4aee62 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -732,6 +732,46 @@ static void dw_pcie_link_set_max_speed(struct dw_pcie *pci, u32 link_gen)
 
 }
 
+static void dw_pcie_link_set_max_link_width(struct dw_pcie *pci, u32 num_lanes)
+{
+	u32 lwsc, plc;
+
+	if (!num_lanes)
+		return;
+
+	/* Set the number of lanes */
+	plc = dw_pcie_readl_dbi(pci, PCIE_PORT_LINK_CONTROL);
+	plc &= ~PORT_LINK_FAST_LINK_MODE;
+	plc &= ~PORT_LINK_MODE_MASK;
+
+	/* Set link width speed control register */
+	lwsc = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	lwsc &= ~PORT_LOGIC_LINK_WIDTH_MASK;
+	switch (num_lanes) {
+	case 1:
+		plc |= PORT_LINK_MODE_1_LANES;
+		lwsc |= PORT_LOGIC_LINK_WIDTH_1_LANES;
+		break;
+	case 2:
+		plc |= PORT_LINK_MODE_2_LANES;
+		lwsc |= PORT_LOGIC_LINK_WIDTH_2_LANES;
+		break;
+	case 4:
+		plc |= PORT_LINK_MODE_4_LANES;
+		lwsc |= PORT_LOGIC_LINK_WIDTH_4_LANES;
+		break;
+	case 8:
+		plc |= PORT_LINK_MODE_8_LANES;
+		lwsc |= PORT_LOGIC_LINK_WIDTH_8_LANES;
+		break;
+	default:
+		dev_err(pci->dev, "num-lanes %u: invalid value\n", num_lanes);
+		return;
+	}
+	dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, plc);
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, lwsc);
+}
+
 void dw_pcie_iatu_detect(struct dw_pcie *pci)
 {
 	int max_region, ob, ib;
@@ -1013,49 +1053,5 @@ void dw_pcie_setup(struct dw_pcie *pci)
 	val |= PORT_LINK_DLL_LINK_EN;
 	dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, val);
 
-	if (!pci->num_lanes) {
-		dev_dbg(pci->dev, "Using h/w default number of lanes\n");
-		return;
-	}
-
-	/* Set the number of lanes */
-	val &= ~PORT_LINK_FAST_LINK_MODE;
-	val &= ~PORT_LINK_MODE_MASK;
-	switch (pci->num_lanes) {
-	case 1:
-		val |= PORT_LINK_MODE_1_LANES;
-		break;
-	case 2:
-		val |= PORT_LINK_MODE_2_LANES;
-		break;
-	case 4:
-		val |= PORT_LINK_MODE_4_LANES;
-		break;
-	case 8:
-		val |= PORT_LINK_MODE_8_LANES;
-		break;
-	default:
-		dev_err(pci->dev, "num-lanes %u: invalid value\n", pci->num_lanes);
-		return;
-	}
-	dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, val);
-
-	/* Set link width speed control register */
-	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
-	val &= ~PORT_LOGIC_LINK_WIDTH_MASK;
-	switch (pci->num_lanes) {
-	case 1:
-		val |= PORT_LOGIC_LINK_WIDTH_1_LANES;
-		break;
-	case 2:
-		val |= PORT_LOGIC_LINK_WIDTH_2_LANES;
-		break;
-	case 4:
-		val |= PORT_LOGIC_LINK_WIDTH_4_LANES;
-		break;
-	case 8:
-		val |= PORT_LOGIC_LINK_WIDTH_8_LANES;
-		break;
-	}
-	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+	dw_pcie_link_set_max_link_width(pci, pci->num_lanes);
 }
-- 
2.25.1

[PATCH v24 03/16] PCI: dwc: Add missing PCI_EXP_LNKCAP_MLW handling

[Yoshihiro Shimoda]

Update dw_pcie_link_set_max_link_width() to set PCI_EXP_LNKCAP_MLW.

In accordance with the DW PCIe RC/EP HW manuals [1,2,3,...] aside with
the PORT_LINK_CTRL_OFF.LINK_CAPABLE and GEN2_CTRL_OFF.NUM_OF_LANES[8:0]
field there is another one which needs to be updated.

It's LINK_CAPABILITIES_REG.PCIE_CAP_MAX_LINK_WIDTH. If it isn't done at
the very least the maximum link-width capability CSR won't expose the
actual maximum capability.

[1] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
    Version 4.60a, March 2015, p.1032
[2] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
    Version 4.70a, March 2016, p.1065
[3] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
    Version 4.90a, March 2016, p.1057
...
[X] DesignWare Cores PCI Express Controller Databook - DWC PCIe Endpoint,
      Version 5.40a, March 2019, p.1396
[X+1] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
      Version 5.40a, March 2019, p.1266

Suggested-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pcie-designware.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index da4aba4aee62..2b60d20dfdf5 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -734,7 +734,8 @@ static void dw_pcie_link_set_max_speed(struct dw_pcie *pci, u32 link_gen)
 
 static void dw_pcie_link_set_max_link_width(struct dw_pcie *pci, u32 num_lanes)
 {
-	u32 lwsc, plc;
+	u32 lnkcap, lwsc, plc;
+	u8 cap;
 
 	if (!num_lanes)
 		return;
@@ -770,6 +771,12 @@ static void dw_pcie_link_set_max_link_width(struct dw_pcie *pci, u32 num_lanes)
 	}
 	dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, plc);
 	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, lwsc);
+
+	cap = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
+	lnkcap = dw_pcie_readl_dbi(pci, cap + PCI_EXP_LNKCAP);
+	lnkcap &= ~PCI_EXP_LNKCAP_MLW;
+	lnkcap |= FIELD_PREP(PCI_EXP_LNKCAP_MLW, num_lanes);
+	dw_pcie_writel_dbi(pci, cap + PCI_EXP_LNKCAP, lnkcap);
 }
 
 void dw_pcie_iatu_detect(struct dw_pcie *pci)
-- 
2.25.1

[PATCH v24 04/16] PCI: tegra194: Drop PCI_EXP_LNKSTA_NLW setting

[Yoshihiro Shimoda]

dw_pcie_setup() is already setting PCI_EXP_LNKCAP_MLW to pcie->num_lanes
in the PCI_EXP_LNKCAP register for programming maximum link width.

Hence, remove the redundant setting here.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 4bba31502ce1..2debdf1a0b52 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -917,12 +917,6 @@ static int tegra_pcie_dw_host_init(struct dw_pcie_rp *pp)
 		AMBA_ERROR_RESPONSE_CRS_SHIFT);
 	dw_pcie_writel_dbi(pci, PORT_LOGIC_AMBA_ERROR_RESPONSE_DEFAULT, val);
 
-	/* Configure Max lane width from DT */
-	val = dw_pcie_readl_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKCAP);
-	val &= ~PCI_EXP_LNKCAP_MLW;
-	val |= (pcie->num_lanes << PCI_EXP_LNKSTA_NLW_SHIFT);
-	dw_pcie_writel_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKCAP, val);
-
 	/* Clear Slot Clock Configuration bit if SRNS configuration */
 	if (pcie->enable_srns) {
 		val_16 = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base +
-- 
2.25.1

[PATCH v24 05/16] PCI: dwc: Add EDMA_UNROLL capability flag

[Yoshihiro Shimoda]

Renesas R-Car Gen4 PCIe controllers have an unexpected register value in
the eDMA CTRL register.

So, add a new capability flag "EDMA_UNROLL" which would force the unrolled
eDMA mapping for the problematic device.

Suggested-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pcie-designware.c | 8 +++++++-
 drivers/pci/controller/dwc/pcie-designware.h | 5 +++--
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 2b60d20dfdf5..1f900be94556 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -887,8 +887,14 @@ static int dw_pcie_edma_find_chip(struct dw_pcie *pci)
 	 * Indirect eDMA CSRs access has been completely removed since v5.40a
 	 * thus no space is now reserved for the eDMA channels viewport and
 	 * former DMA CTRL register is no longer fixed to FFs.
+	 *
+	 * Note that Renesas R-Car S4-8's PCIe controllers for unknown reason
+	 * have zeros in the eDMA CTRL register even though the HW-manual
+	 * explicitly states there must FFs if the unrolled mapping is enabled.
+	 * For such cases the low-level drivers are supposed to manually
+	 * activate the unrolled mapping to bypass the auto-detection procedure.
 	 */
-	if (dw_pcie_ver_is_ge(pci, 540A))
+	if (dw_pcie_ver_is_ge(pci, 540A) || dw_pcie_cap_is(pci, EDMA_UNROLL))
 		val = 0xFFFFFFFF;
 	else
 		val = dw_pcie_readl_dbi(pci, PCIE_DMA_VIEWPORT_BASE + PCIE_DMA_CTRL);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 6189884b4efa..e10f7e18b13a 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -51,8 +51,9 @@
 
 /* DWC PCIe controller capabilities */
 #define DW_PCIE_CAP_REQ_RES		0
-#define DW_PCIE_CAP_IATU_UNROLL		1
-#define DW_PCIE_CAP_CDM_CHECK		2
+#define DW_PCIE_CAP_EDMA_UNROLL		1
+#define DW_PCIE_CAP_IATU_UNROLL		2
+#define DW_PCIE_CAP_CDM_CHECK		3
 
 #define dw_pcie_cap_is(_pci, _cap) \
 	test_bit(DW_PCIE_CAP_ ## _cap, &(_pci)->caps)
-- 
2.25.1

[PATCH v24 06/16] PCI: dwc: Expose dw_pcie_ep_exit() to module

[Yoshihiro Shimoda]

Since no PCIe controller drivers call this, this change is not required
for now. But, Renesas R-Car Gen4 PCIe controller driver will call this
and if the controller driver is built as a kernel module, the following
build error happens:

  ERROR: modpost: "dw_pcie_ep_exit" [drivers/pci/controller/dwc/pcie-rcar-gen4-ep-drv.ko] undefined!

So, expose dw_pcie_ep_exit() for it.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pcie-designware-ep.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 851538ddec0a..a8bcbc57ef86 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -638,6 +638,7 @@ void dw_pcie_ep_exit(struct dw_pcie_ep *ep)
 
 	pci_epc_mem_exit(epc);
 }
+EXPORT_SYMBOL_GPL(dw_pcie_ep_exit);
 
 static unsigned int dw_pcie_ep_find_ext_capability(struct dw_pcie *pci, int cap)
 {
-- 
2.25.1

[PATCH v24 07/16] PCI: dwc: endpoint: Introduce .pre_init() and .deinit()

[Yoshihiro Shimoda]

Renesas R-Car Gen4 PCIe controllers require vendor-specific
initialization before .init().

To use dw->dbi and dw->num-lanes in the initialization code,
introduce .pre_init() into struct dw_pcie_ep_ops. While at it,
also introduce .deinit() to disable the controller by using
vendor-specific de-initialization.

Note that the ep_init in the struct dw_pcie_ep_ops should be
renamed to init later.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pcie-designware-ep.c | 12 +++++++++++-
 drivers/pci/controller/dwc/pcie-designware.h    |  2 ++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index a8bcbc57ef86..d34a5e87ad18 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -637,6 +637,9 @@ void dw_pcie_ep_exit(struct dw_pcie_ep *ep)
 			      epc->mem->window.page_size);
 
 	pci_epc_mem_exit(epc);
+
+	if (ep->ops->deinit)
+		ep->ops->deinit(ep);
 }
 EXPORT_SYMBOL_GPL(dw_pcie_ep_exit);
 
@@ -740,6 +743,9 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 	ep->phys_base = res->start;
 	ep->addr_size = resource_size(res);
 
+	if (ep->ops->pre_init)
+		ep->ops->pre_init(ep);
+
 	dw_pcie_version_detect(pci);
 
 	dw_pcie_iatu_detect(pci);
@@ -794,7 +800,7 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 			       ep->page_size);
 	if (ret < 0) {
 		dev_err(dev, "Failed to initialize address space\n");
-		return ret;
+		goto err_ep_deinit;
 	}
 
 	ep->msi_mem = pci_epc_mem_alloc_addr(epc, &ep->msi_mem_phys,
@@ -831,6 +837,10 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep)
 err_exit_epc_mem:
 	pci_epc_mem_exit(epc);
 
+err_ep_deinit:
+	if (ep->ops->deinit)
+		ep->ops->deinit(ep);
+
 	return ret;
 }
 EXPORT_SYMBOL_GPL(dw_pcie_ep_init);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index e10f7e18b13a..8c637392ab08 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -330,7 +330,9 @@ struct dw_pcie_rp {
 };
 
 struct dw_pcie_ep_ops {
+	void	(*pre_init)(struct dw_pcie_ep *ep);
 	void	(*ep_init)(struct dw_pcie_ep *ep);
+	void	(*deinit)(struct dw_pcie_ep *ep);
 	int	(*raise_irq)(struct dw_pcie_ep *ep, u8 func_no,
 			     enum pci_epc_irq_type type, u16 interrupt_num);
 	const struct pci_epc_features* (*get_features)(struct dw_pcie_ep *ep);
-- 
2.25.1

[PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Yoshihiro Shimoda]

According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
Rev.5.20a, we should disable two BARs to avoid unnecessary memory
assignment during device enumeration. Otherwise, Renesas R-Car Gen4
PCIe controllers cannot work correctly in host mode.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
---
 drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index a7170fd0e847..56cc7ff6d508 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 	u32 val, ctrl, num_ctrls;
 	int ret;
 
+	/*
+	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
+	 * Rev.5.20a, we should disable two BARs to avoid unnecessary memory
+	 * assignment during device enumeration.
+	 */
+	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
+	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
+
 	/*
 	 * Enable DBI read-only registers for writing/updating configuration.
 	 * Write permission gets disabled towards the end of this function.
-- 
2.25.1

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Serge Semin]

On Wed, Oct 11, 2023 at 04:14:15PM +0900, Yoshihiro Shimoda wrote:
> According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> assignment during device enumeration. Otherwise, Renesas R-Car Gen4
> PCIe controllers cannot work correctly in host mode.
> 
> Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
> ---
>  drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index a7170fd0e847..56cc7ff6d508 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>  	u32 val, ctrl, num_ctrls;
>  	int ret;
>  
> +	/*
> +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> +	 * Rev.5.20a,

and 3.5.6.1 "RC mode" in DWC PCIe RC databook v5.20a.

> +      ... we should disable two BARs to avoid unnecessary memory
> +	 * assignment during device enumeration.
> +	 */
> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> +

What's the point in doing this
	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004);
	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_1, 0x00000000);
        ...
        dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
afterward?

I guess if the BARs are disabled there is no need in having them
touched. Am I wrong?

BTW I failed to understand why the BARs inits was originally needed:
first merging the BAR0 and BAR1 into a single 64-bit BAR, then
switching it back to two 32-bit BARs. Moreover here is what prior DW
PCIe RC v5.x databooks say about the BARs:

"3.5.6 BAR Details
Base Address Registers (Offset: 0x10-x14)
The Synopsys core does not implement the optional BARs for the RC
product. This is based on the assumption that the RC host probably has
registers on some other internal bus and has knowledge and setup
access to these registers already."

I am not sure I fully understand what it means, but it seems as DW
PCIe cores didn't have anything behind the RC BARs even back then. So
it seems to me that the BARs manipulation was the Exinos PCIe host
specific, from which driver they are originating - commit 340cba6092c2
("pci: Add PCIe driver for Samsung Exynos").

* BTW Yoshihiro, I am sorry to see your patchset is still under review...(

-Serge(y)

>  	/*
>  	 * Enable DBI read-only registers for writing/updating configuration.
>  	 * Write permission gets disabled towards the end of this function.
> -- 
> 2.25.1
> 

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Krzysztof Wilczyński]

Hello,

[...]
> > +	/*
> > +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > +	 * Rev.5.20a,
> 
> and 3.5.6.1 "RC mode" in DWC PCIe RC databook v5.20a.

OK.  I can fix this citation later.

> > +      ... we should disable two BARs to avoid unnecessary memory
> > +	 * assignment during device enumeration.
> > +	 */
> > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> > +
> 
> What's the point in doing this
> 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004);
> 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_1, 0x00000000);
>         ...
>         dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
> afterward?
> 
> I guess if the BARs are disabled there is no need in having them
> touched. Am I wrong?
> 
> BTW I failed to understand why the BARs inits was originally needed:
> first merging the BAR0 and BAR1 into a single 64-bit BAR, then
> switching it back to two 32-bit BARs. Moreover here is what prior DW
> PCIe RC v5.x databooks say about the BARs:
> 
> "3.5.6 BAR Details
> Base Address Registers (Offset: 0x10-x14)
> The Synopsys core does not implement the optional BARs for the RC
> product. This is based on the assumption that the RC host probably has
> registers on some other internal bus and has knowledge and setup
> access to these registers already."
> 
> I am not sure I fully understand what it means, but it seems as DW
> PCIe cores didn't have anything behind the RC BARs even back then. So
> it seems to me that the BARs manipulation was the Exinos PCIe host
> specific, from which driver they are originating - commit 340cba6092c2
> ("pci: Add PCIe driver for Samsung Exynos").

Would any of the above be something we need to address before this series
can be successfully merged?  I am asking if this is a show stopper,
something we can fix later, or even something I could address once I take
this series again.

Thoughts?

> * BTW Yoshihiro, I am sorry to see your patchset is still under review...(

Yes, we need to draw a line somewhere. :)  I am happy to take this series
so we don't miss another merge window.  We can always fix other bits and
pieces later and iron out any kinks that might have fallen through the
cracks, so to speak.

	Krzysztof

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Manivannan Sadhasivam]

On Wed, Oct 11, 2023 at 10:07:27PM +0900, Krzysztof Wilczyński wrote:
> Hello,
> 
> [...]
> > > +	/*
> > > +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > > +	 * Rev.5.20a,
> > 
> > and 3.5.6.1 "RC mode" in DWC PCIe RC databook v5.20a.
> 
> OK.  I can fix this citation later.
> 
> > > +      ... we should disable two BARs to avoid unnecessary memory
> > > +	 * assignment during device enumeration.
> > > +	 */
> > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> > > +
> > 
> > What's the point in doing this
> > 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004);
> > 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_1, 0x00000000);
> >         ...
> >         dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
> > afterward?
> > 
> > I guess if the BARs are disabled there is no need in having them
> > touched. Am I wrong?
> > 
> > BTW I failed to understand why the BARs inits was originally needed:
> > first merging the BAR0 and BAR1 into a single 64-bit BAR, then
> > switching it back to two 32-bit BARs. Moreover here is what prior DW
> > PCIe RC v5.x databooks say about the BARs:
> > 
> > "3.5.6 BAR Details
> > Base Address Registers (Offset: 0x10-x14)
> > The Synopsys core does not implement the optional BARs for the RC
> > product. This is based on the assumption that the RC host probably has
> > registers on some other internal bus and has knowledge and setup
> > access to these registers already."
> > 
> > I am not sure I fully understand what it means, but it seems as DW
> > PCIe cores didn't have anything behind the RC BARs even back then. So
> > it seems to me that the BARs manipulation was the Exinos PCIe host
> > specific, from which driver they are originating - commit 340cba6092c2
> > ("pci: Add PCIe driver for Samsung Exynos").
> 
> Would any of the above be something we need to address before this series
> can be successfully merged?  I am asking if this is a show stopper,
> something we can fix later, or even something I could address once I take
> this series again.
> 
> Thoughts?
> 

If Yoshihiro can confirm that his controller can work without this patch, then
I'd vote for dropping this patch and applying the rest.

This can be submitted later if required.

- Mani

> > * BTW Yoshihiro, I am sorry to see your patchset is still under review...(
> 
> Yes, we need to draw a line somewhere. :)  I am happy to take this series
> so we don't miss another merge window.  We can always fix other bits and
> pieces later and iron out any kinks that might have fallen through the
> cracks, so to speak.
> 
> 	Krzysztof

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Serge Semin]

Hello Krzysztof, Mani

On Wed, Oct 11, 2023 at 06:48:40PM +0530, Manivannan Sadhasivam wrote:
> On Wed, Oct 11, 2023 at 10:07:27PM +0900, Krzysztof Wilczyński wrote:
> > Hello,
> > 
> > [...]
> > > > +	/*
> > > > +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > > > +	 * Rev.5.20a,
> > > 
> > > and 3.5.6.1 "RC mode" in DWC PCIe RC databook v5.20a.
> > 
> > OK.  I can fix this citation later.
> > 
> > > > +      ... we should disable two BARs to avoid unnecessary memory
> > > > +	 * assignment during device enumeration.
> > > > +	 */
> > > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> > > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> > > > +
> > > 
> > > What's the point in doing this
> > > 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004);
> > > 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_1, 0x00000000);
> > >         ...
> > >         dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
> > > afterward?
> > > 
> > > I guess if the BARs are disabled there is no need in having them
> > > touched. Am I wrong?
> > > 
> > > BTW I failed to understand why the BARs inits was originally needed:
> > > first merging the BAR0 and BAR1 into a single 64-bit BAR, then
> > > switching it back to two 32-bit BARs. Moreover here is what prior DW
> > > PCIe RC v5.x databooks say about the BARs:
> > > 
> > > "3.5.6 BAR Details
> > > Base Address Registers (Offset: 0x10-x14)
> > > The Synopsys core does not implement the optional BARs for the RC
> > > product. This is based on the assumption that the RC host probably has
> > > registers on some other internal bus and has knowledge and setup
> > > access to these registers already."
> > > 
> > > I am not sure I fully understand what it means, but it seems as DW
> > > PCIe cores didn't have anything behind the RC BARs even back then. So
> > > it seems to me that the BARs manipulation was the Exinos PCIe host
> > > specific, from which driver they are originating - commit 340cba6092c2
> > > ("pci: Add PCIe driver for Samsung Exynos").
> > 

> > Would any of the above be something we need to address before this series
> > can be successfully merged?  I am asking if this is a show stopper,
> > something we can fix later, or even something I could address once I take
> > this series again.
> > 
> > Thoughts?
> >

I can't confirm for sure that the BARs manipulations in this patch
will work on the older IP-cores (prior 5.10a) or will be required for
all new controllers (5.10a and newer). Based on the BARs description
posted in the IP-core HW manuals, the CSRs semantic has changed
between the major releases. Old DW PCIe RC IP-core HW-manuals
explicitly state that the BARs are unavailable:

"The Synopsys core does not implement the optional BARs for the RC
product"

New DW PCIe RC IP-cores manual say that the BARs exist, but are
normally unused:

"Two BARs are present but are not expected to be used. You should
disable them to avoid unnecessary memory assignment during device
enumeration. If you do use a BAR, then you should program it to
capture TLPs that are targeted to your local non-application memory
space.... The BAR range must be outside of the three Base/Limit
regions..."

So in theory it's possible to have platforms with the BARs somehow
utilized even in the Root Ports. Though currently AFAICS we don't
have such devices supported in kernel.

> 
> If Yoshihiro can confirm that his controller can work without this patch, then
> I'd vote for dropping this patch and applying the rest.

AFAIR Yoshihiro insisted to have the BARs reset because without
it something didn't work, so he added some comment to justify it:
https://lore.kernel.org/linux-pci/TYBPR01MB534104389952D87385E8745ED8879@TYBPR01MB5341.jpnprd01.prod.outlook.com/
Though based on the comment the BARs reset still seems optional.

One more low-level driver which already does what is implemented in
this patch is the Keystone PCI host-controller driver (see,
pci-keystone.c also activates dbi_cs2 and zeros out the
PCI_TYPE0_BAR0_ENABLED flag). Moreover something similar is done in
the generic DW PCIe EP driver in the framework of the
__dw_pcie_ep_reset_bar() method including the direct BARs zeroing out
(which I questioned in my initial message in this thread). So seeing
this patch would re-do what is already done for the Keystone device
and would add a partly duplicated code it would be reasonable to drop
the patch for now and get the BARs reset back to the Rcar host
low-level driver as it was in v23. We can get back to the topic
afterward and see whether the BARs reset could be done generically for
the RPs. If we figure out that it's required at least for the new
controllers then we'll be able to implement a generic RP/EP BARs reset
method, have it utilized in both DW PCIe core drivers and drop the
respective code from both Rcar and Keystone LLDDs.

-Serge(y)

> 
> This can be submitted later if required.
> 
> - Mani
> 
> > > * BTW Yoshihiro, I am sorry to see your patchset is still under review...(
> > 
> > Yes, we need to draw a line somewhere. :)  I am happy to take this series
> > so we don't miss another merge window.  We can always fix other bits and
> > pieces later and iron out any kinks that might have fallen through the
> > cracks, so to speak.
> > 
> > 	Krzysztof
> 
> -- 
> மணிவண்ணன் சதாசிவம்

RE: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Yoshihiro Shimoda]

Hello Serge, Krzysztof, Mani,

> From: Serge Semin, Sent: Wednesday, October 11, 2023 11:50 PM
> 
> Hello Krzysztof, Mani
> 
> On Wed, Oct 11, 2023 at 06:48:40PM +0530, Manivannan Sadhasivam wrote:
> > On Wed, Oct 11, 2023 at 10:07:27PM +0900, Krzysztof Wilczyński wrote:
> > > Hello,
> > >
> > > [...]
> > > > > +	/*
> > > > > +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > > > > +	 * Rev.5.20a,
> > > >
> > > > and 3.5.6.1 "RC mode" in DWC PCIe RC databook v5.20a.
> > >
> > > OK.  I can fix this citation later.
> > >
> > > > > +      ... we should disable two BARs to avoid unnecessary memory
> > > > > +	 * assignment during device enumeration.
> > > > > +	 */
> > > > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> > > > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> > > > > +
> > > >
> > > > What's the point in doing this
> > > > 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004);
> > > > 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_1, 0x00000000);
> > > >         ...
> > > >         dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
> > > > afterward?
> > > >
> > > > I guess if the BARs are disabled there is no need in having them
> > > > touched. Am I wrong?

I tried to remove these setting on my environment, and then it also worked.
So, as you mentioned, there is no need, I think.

> > > > BTW I failed to understand why the BARs inits was originally needed:
> > > > first merging the BAR0 and BAR1 into a single 64-bit BAR, then
> > > > switching it back to two 32-bit BARs. Moreover here is what prior DW
> > > > PCIe RC v5.x databooks say about the BARs:
> > > >
> > > > "3.5.6 BAR Details
> > > > Base Address Registers (Offset: 0x10-x14)
> > > > The Synopsys core does not implement the optional BARs for the RC
> > > > product. This is based on the assumption that the RC host probably has
> > > > registers on some other internal bus and has knowledge and setup
> > > > access to these registers already."
> > > >
> > > > I am not sure I fully understand what it means, but it seems as DW
> > > > PCIe cores didn't have anything behind the RC BARs even back then. So
> > > > it seems to me that the BARs manipulation was the Exinos PCIe host
> > > > specific, from which driver they are originating - commit 340cba6092c2
> > > > ("pci: Add PCIe driver for Samsung Exynos").
> > >
> 
> > > Would any of the above be something we need to address before this series
> > > can be successfully merged?  I am asking if this is a show stopper,
> > > something we can fix later, or even something I could address once I take
> > > this series again.
> > >
> > > Thoughts?
> > >
> 
> I can't confirm for sure that the BARs manipulations in this patch
> will work on the older IP-cores (prior 5.10a) or will be required for
> all new controllers (5.10a and newer). Based on the BARs description
> posted in the IP-core HW manuals, the CSRs semantic has changed
> between the major releases. Old DW PCIe RC IP-core HW-manuals
> explicitly state that the BARs are unavailable:
> 
> "The Synopsys core does not implement the optional BARs for the RC
> product"
> 
> New DW PCIe RC IP-cores manual say that the BARs exist, but are
> normally unused:
> 
> "Two BARs are present but are not expected to be used. You should
> disable them to avoid unnecessary memory assignment during device
> enumeration. If you do use a BAR, then you should program it to
> capture TLPs that are targeted to your local non-application memory
> space.... The BAR range must be outside of the three Base/Limit
> regions..."
> 
> So in theory it's possible to have platforms with the BARs somehow
> utilized even in the Root Ports. Though currently AFAICS we don't
> have such devices supported in kernel.
> 
> >
> > If Yoshihiro can confirm that his controller can work without this patch, then
> > I'd vote for dropping this patch and applying the rest.
> 
> AFAIR Yoshihiro insisted to have the BARs reset because without
> it something didn't work, so he added some comment to justify it:
> https://lore.kernel.org/linux-pci/TYBPR01MB534104389952D87385E8745ED8879@TYBPR01MB5341.jpnprd01.prod.outlook.com/

Yes, the settings are really needed on my environment.
# I checked this again today.

> Though based on the comment the BARs reset still seems optional.
> 
> One more low-level driver which already does what is implemented in
> this patch is the Keystone PCI host-controller driver (see,
> pci-keystone.c also activates dbi_cs2 and zeros out the
> PCI_TYPE0_BAR0_ENABLED flag).

Thank you for the information. I could not find the settings...
-----
static void ks_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
{
...
        /* Disable BARs for inbound access */
        ks_pcie_set_dbi_mode(ks_pcie);
        dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0);
        dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_1, 0);
        ks_pcie_clear_dbi_mode(ks_pcie);
-----

> Moreover something similar is done in
> the generic DW PCIe EP driver in the framework of the
> __dw_pcie_ep_reset_bar() method including the direct BARs zeroing out
> (which I questioned in my initial message in this thread). So seeing
> this patch would re-do what is already done for the Keystone device
> and would add a partly duplicated code it would be reasonable to drop
> the patch for now and get the BARs reset back to the Rcar host
> low-level driver as it was in v23. We can get back to the topic
> afterward and see whether the BARs reset could be done generically for
> the RPs. If we figure out that it's required at least for the new
> controllers then we'll be able to implement a generic RP/EP BARs reset
> method, have it utilized in both DW PCIe core drivers and drop the
> respective code from both Rcar and Keystone LLDDs.

If possible, I would like to have the setting on pcie-rcar-gen4.c only,
for avoiding any trouble, especially on other DWC drivers.

> -Serge(y)
> 
> >
> > This can be submitted later if required.
> >
> > - Mani
> >
> > > > * BTW Yoshihiro, I am sorry to see your patchset is still under review...(

Serge, no worries! Thank you very much for your support again and again!

> > > Yes, we need to draw a line somewhere. :)  I am happy to take this series
> > > so we don't miss another merge window.  We can always fix other bits and
> > > pieces later and iron out any kinks that might have fallen through the
> > > cracks, so to speak.

Krzysztof, thank you very much for your support! I hope that this patchset can be
merged into v6.7-rc1...

Best regards,
Yoshihiro Shimoda

> > > 	Krzysztof
> >
> > --
> > மணிவண்ணன் சதாசிவம்

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Bjorn Helgaas]

[+cc Siddharth, Ravi, Sriramakrishnan]

On Wed, Oct 11, 2023 at 04:14:15PM +0900, Yoshihiro Shimoda wrote:
> According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> assignment during device enumeration. Otherwise, Renesas R-Car Gen4
> PCIe controllers cannot work correctly in host mode.
> 
> Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
> ---
>  drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index a7170fd0e847..56cc7ff6d508 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>  	u32 val, ctrl, num_ctrls;
>  	int ret;
>  
> +	/*
> +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> +	 * Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> +	 * assignment during device enumeration.
> +	 */
> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);

I cc'd Siddharth and others because they are working on a Keystone
issue with MSI-X that requires BAR0; see
https://lore.kernel.org/r/20231011123451.34827-1-s-vadapalli@ti.com 

I assume any DWC controller that uses MSI-X would require BAR0 or BAR1
for the MSI-X Table.

I don't have any of the DWC specs and don't know whether any
controllers use MSI-X, so just heads up in case they do.  This patch
was recently merged and will appear in v6.7.

Bjorn

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Siddharth Vadapalli]

Hello Bjorn,

On 17/10/23 03:18, Bjorn Helgaas wrote:
> [+cc Siddharth, Ravi, Sriramakrishnan]
> 
> On Wed, Oct 11, 2023 at 04:14:15PM +0900, Yoshihiro Shimoda wrote:
>> According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
>> Rev.5.20a, we should disable two BARs to avoid unnecessary memory
>> assignment during device enumeration. Otherwise, Renesas R-Car Gen4
>> PCIe controllers cannot work correctly in host mode.
>>
>> Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
>> ---
>>  drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
>> index a7170fd0e847..56cc7ff6d508 100644
>> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
>> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
>> @@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>>  	u32 val, ctrl, num_ctrls;
>>  	int ret;
>>  
>> +	/*
>> +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
>> +	 * Rev.5.20a, we should disable two BARs to avoid unnecessary memory
>> +	 * assignment during device enumeration.
>> +	 */
>> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
>> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> 
> I cc'd Siddharth and others because they are working on a Keystone
> issue with MSI-X that requires BAR0; see
> https://lore.kernel.org/r/20231011123451.34827-1-s-vadapalli@ti.com 
> 
> I assume any DWC controller that uses MSI-X would require BAR0 or BAR1
> for the MSI-X Table.

Disabling BAR0 in this section will not affect the pci-keystone.c driver. The
MSI-X setup in the pci-keystone.c driver is done via the ks_pcie_v3_65_add_bus()
function which is invoked in the pci_host_probe(bridge) function call within
dw_pcie_host_init(). Since pci_host_probe(bridge) is invoked *after*
dw_pcie_setup_rc(), the MSI-X setup which involves enabling BAR0 will be
performed after BAR0 is disabled in this patch, due to which BAR0 will remain
enabled as far as MSI-X setup is concerned.

Also, the DW PCIe IP version corresponding to the Keystone PCIe host controller
is 4.90a. Even in the 4.90a Databook, in section 3.5.7.2 RC Mode, it is
suggested that the BARs should be disabled.

> 
> I don't have any of the DWC specs and don't know whether any
> controllers use MSI-X, so just heads up in case they do.  This patch
> was recently merged and will appear in v6.7.
> 
> Bjorn

-- 
Regards,
Siddharth.

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Marek Szyprowski]

Dear All,

On 11.10.2023 09:14, Yoshihiro Shimoda wrote:
> According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> assignment during device enumeration. Otherwise, Renesas R-Car Gen4
> PCIe controllers cannot work correctly in host mode.
>
> Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>

This patch landed in today's linux-next 20231017 as commit e308528cac3e 
("PCI: dwc: Disable two BARs to avoid unnecessary memory assignment"). 
Unfortunately it causes the following kernel panic on Samsung 
Exynos5433-based TM2e board:

exynos-pcie 15700000.pcie: host bridge /soc@0/pcie@15700000 ranges:
exynos-pcie 15700000.pcie:       IO 0x000c001000..0x000c010fff -> 
0x0000000000
exynos-pcie 15700000.pcie:      MEM 0x000c011000..0x000ffffffe -> 
0x000c011000
exynos-pcie 15700000.pcie: iATU: unroll F, 3 ob, 5 ib, align 4K, limit 4G
Unable to handle kernel paging request at virtual address ffff800084196010
Mem abort info:
...
Data abort info:
...
swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000022047000
[ffff800084196010] pgd=10000000df6ff003, p4d=10000000df6ff003, 
pud=10000000df6fe003, pmd=1000000024ad9003, pte=0000000000000000
Internal error: Oops: 0000000096000047 [#1] PREEMPT SMP
Modules linked in:
CPU: 4 PID: 55 Comm: kworker/u18:0 Not tainted 6.6.0-rc1+ #14129
Hardware name: Samsung TM2E board (DT)
Workqueue: events_unbound deferred_probe_work_func
pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : dw_pcie_write_dbi2+0xb8/0xc8
lr : dw_pcie_setup_rc+0x30/0x4e4
...
Call trace:
  dw_pcie_write_dbi2+0xb8/0xc8
  dw_pcie_setup_rc+0x30/0x4e4
  dw_pcie_host_init+0x238/0x608
  exynos_pcie_probe+0x23c/0x340
  platform_probe+0x68/0xd8
  really_probe+0x148/0x2b4
  __driver_probe_device+0x78/0x12c
  driver_probe_device+0xd8/0x160
  __device_attach_driver+0xb8/0x138
  bus_for_each_drv+0x84/0xe0
  __device_attach+0xa8/0x1b0
  device_initial_probe+0x14/0x20
  bus_probe_device+0xb0/0xb4
  deferred_probe_work_func+0x8c/0xc8
  process_one_work+0x1ec/0x53c
  worker_thread+0x298/0x408
  kthread+0x124/0x128
  ret_from_fork+0x10/0x20
Code: d50332bf 79000023 17ffffe2 d50332bf (b9000023)
---[ end trace 0000000000000000 ]---
Kernel panic - not syncing: Oops: Fatal exception
SMP: stopping secondary CPUs
Kernel Offset: disabled
CPU features: 0x8c00020e,3c020000,0000421b
Memory Limit: none
---[ end Kernel panic - not syncing: Oops: Fatal exception ]---

I've observed similar issue on Qualcomm's RB5 platform with some 
additional not-yet merged patches enabling PCIe support. Reverting 
$subject on top of linux-next fixes this issue.

Let me know if you need more information.

> ---
>   drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
>   1 file changed, 8 insertions(+)
>
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index a7170fd0e847..56cc7ff6d508 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
>   	u32 val, ctrl, num_ctrls;
>   	int ret;
>   
> +	/*
> +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> +	 * Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> +	 * assignment during device enumeration.
> +	 */
> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> +
>   	/*
>   	 * Enable DBI read-only registers for writing/updating configuration.
>   	 * Write permission gets disabled towards the end of this function.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

RE: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Yoshihiro Shimoda]

Dear Marek,

> From: Marek Szyprowski, Sent: Tuesday, October 17, 2023 6:19 PM
> 
> Dear All,
> 
> On 11.10.2023 09:14, Yoshihiro Shimoda wrote:
> > According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> > assignment during device enumeration. Otherwise, Renesas R-Car Gen4
> > PCIe controllers cannot work correctly in host mode.
> >
> > Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
> 
> This patch landed in today's linux-next 20231017 as commit e308528cac3e
> ("PCI: dwc: Disable two BARs to avoid unnecessary memory assignment").
> Unfortunately it causes the following kernel panic on Samsung
> Exynos5433-based TM2e board:
> 
> exynos-pcie 15700000.pcie: host bridge /soc@0/pcie@15700000 ranges:
> exynos-pcie 15700000.pcie:       IO 0x000c001000..0x000c010fff ->
> 0x0000000000
> exynos-pcie 15700000.pcie:      MEM 0x000c011000..0x000ffffffe ->
> 0x000c011000
> exynos-pcie 15700000.pcie: iATU: unroll F, 3 ob, 5 ib, align 4K, limit 4G
> Unable to handle kernel paging request at virtual address ffff800084196010
> Mem abort info:
> ...
> Data abort info:
> ...
> swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000022047000
> [ffff800084196010] pgd=10000000df6ff003, p4d=10000000df6ff003,
> pud=10000000df6fe003, pmd=1000000024ad9003, pte=0000000000000000
> Internal error: Oops: 0000000096000047 [#1] PREEMPT SMP
> Modules linked in:
> CPU: 4 PID: 55 Comm: kworker/u18:0 Not tainted 6.6.0-rc1+ #14129
> Hardware name: Samsung TM2E board (DT)
> Workqueue: events_unbound deferred_probe_work_func
> pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : dw_pcie_write_dbi2+0xb8/0xc8
> lr : dw_pcie_setup_rc+0x30/0x4e4
> ...
> Call trace:
>   dw_pcie_write_dbi2+0xb8/0xc8
>   dw_pcie_setup_rc+0x30/0x4e4
>   dw_pcie_host_init+0x238/0x608
>   exynos_pcie_probe+0x23c/0x340
>   platform_probe+0x68/0xd8
>   really_probe+0x148/0x2b4
>   __driver_probe_device+0x78/0x12c
>   driver_probe_device+0xd8/0x160
>   __device_attach_driver+0xb8/0x138
>   bus_for_each_drv+0x84/0xe0
>   __device_attach+0xa8/0x1b0
>   device_initial_probe+0x14/0x20
>   bus_probe_device+0xb0/0xb4
>   deferred_probe_work_func+0x8c/0xc8
>   process_one_work+0x1ec/0x53c
>   worker_thread+0x298/0x408
>   kthread+0x124/0x128
>   ret_from_fork+0x10/0x20
> Code: d50332bf 79000023 17ffffe2 d50332bf (b9000023)
> ---[ end trace 0000000000000000 ]---
> Kernel panic - not syncing: Oops: Fatal exception
> SMP: stopping secondary CPUs
> Kernel Offset: disabled
> CPU features: 0x8c00020e,3c020000,0000421b
> Memory Limit: none
> ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---
> 
> I've observed similar issue on Qualcomm's RB5 platform with some
> additional not-yet merged patches enabling PCIe support. Reverting
> $subject on top of linux-next fixes this issue.
> 
> Let me know if you need more information.

Thank you for the report. I guess that the issue is related to
out-of-range access of dbi2:
- In arch/arm64/boot/dts/exynos/exynos5433.dtsi, the dbi reg size is 0x1000
  like below:
-----
                pcie: pcie@15700000 {
                        compatible = "samsung,exynos5433-pcie";
                        reg = <0x15700000 0x1000>, <0x156b0000 0x1000>,
                              <0x0c000000 0x1000>;
                        reg-names = "dbi", "elbi", "config";
...
-----

- In drivers/pci/controller/dwc/pcie-designware.c, "dbi2" area is calculated
  by the following if reg-names "dbi2" didn't exist:
-----
                        pci->dbi_base2 = pci->dbi_base + SZ_4K;
-----

- However, this is out-of-memory on exynos5433.dtsi because the "dbi" size is
  0x1000 only.
- And then, this patch always writes PCI_BASE_ADDRESS_[01] to dbi2 area.
  So, since this is out-of-range, the kernel panic happens.

Perhaps, we should revert this patch at first. And, add the settings into
my environment (pcie-rcar-gen4.c) only. I also have alternative solution which
modifies the "dbi2" area calculation and avoid out-of-range access somehow.
But, it may complicate source code...

Best regards,
Yoshihiro Shimoda

> > ---
> >   drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
> >   1 file changed, 8 insertions(+)
> >
> > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> > index a7170fd0e847..56cc7ff6d508 100644
> > --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> > @@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
> >   	u32 val, ctrl, num_ctrls;
> >   	int ret;
> >
> > +	/*
> > +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > +	 * Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> > +	 * assignment during device enumeration.
> > +	 */
> > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> > +
> >   	/*
> >   	 * Enable DBI read-only registers for writing/updating configuration.
> >   	 * Write permission gets disabled towards the end of this function.
> 
> Best regards
> --
> Marek Szyprowski, PhD
> Samsung R&D Institute Poland

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[mani]

On Tue, Oct 17, 2023 at 12:05:12PM +0000, Yoshihiro Shimoda wrote:
> Dear Marek,
> 
> > From: Marek Szyprowski, Sent: Tuesday, October 17, 2023 6:19 PM
> > 
> > Dear All,
> > 
> > On 11.10.2023 09:14, Yoshihiro Shimoda wrote:
> > > According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > > Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> > > assignment during device enumeration. Otherwise, Renesas R-Car Gen4
> > > PCIe controllers cannot work correctly in host mode.
> > >
> > > Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
> > 
> > This patch landed in today's linux-next 20231017 as commit e308528cac3e
> > ("PCI: dwc: Disable two BARs to avoid unnecessary memory assignment").
> > Unfortunately it causes the following kernel panic on Samsung
> > Exynos5433-based TM2e board:
> > 
> > exynos-pcie 15700000.pcie: host bridge /soc@0/pcie@15700000 ranges:
> > exynos-pcie 15700000.pcie:       IO 0x000c001000..0x000c010fff ->
> > 0x0000000000
> > exynos-pcie 15700000.pcie:      MEM 0x000c011000..0x000ffffffe ->
> > 0x000c011000
> > exynos-pcie 15700000.pcie: iATU: unroll F, 3 ob, 5 ib, align 4K, limit 4G
> > Unable to handle kernel paging request at virtual address ffff800084196010
> > Mem abort info:
> > ...
> > Data abort info:
> > ...
> > swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000022047000
> > [ffff800084196010] pgd=10000000df6ff003, p4d=10000000df6ff003,
> > pud=10000000df6fe003, pmd=1000000024ad9003, pte=0000000000000000
> > Internal error: Oops: 0000000096000047 [#1] PREEMPT SMP
> > Modules linked in:
> > CPU: 4 PID: 55 Comm: kworker/u18:0 Not tainted 6.6.0-rc1+ #14129
> > Hardware name: Samsung TM2E board (DT)
> > Workqueue: events_unbound deferred_probe_work_func
> > pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > pc : dw_pcie_write_dbi2+0xb8/0xc8
> > lr : dw_pcie_setup_rc+0x30/0x4e4
> > ...
> > Call trace:
> >   dw_pcie_write_dbi2+0xb8/0xc8
> >   dw_pcie_setup_rc+0x30/0x4e4
> >   dw_pcie_host_init+0x238/0x608
> >   exynos_pcie_probe+0x23c/0x340
> >   platform_probe+0x68/0xd8
> >   really_probe+0x148/0x2b4
> >   __driver_probe_device+0x78/0x12c
> >   driver_probe_device+0xd8/0x160
> >   __device_attach_driver+0xb8/0x138
> >   bus_for_each_drv+0x84/0xe0
> >   __device_attach+0xa8/0x1b0
> >   device_initial_probe+0x14/0x20
> >   bus_probe_device+0xb0/0xb4
> >   deferred_probe_work_func+0x8c/0xc8
> >   process_one_work+0x1ec/0x53c
> >   worker_thread+0x298/0x408
> >   kthread+0x124/0x128
> >   ret_from_fork+0x10/0x20
> > Code: d50332bf 79000023 17ffffe2 d50332bf (b9000023)
> > ---[ end trace 0000000000000000 ]---
> > Kernel panic - not syncing: Oops: Fatal exception
> > SMP: stopping secondary CPUs
> > Kernel Offset: disabled
> > CPU features: 0x8c00020e,3c020000,0000421b
> > Memory Limit: none
> > ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---
> > 
> > I've observed similar issue on Qualcomm's RB5 platform with some
> > additional not-yet merged patches enabling PCIe support. Reverting
> > $subject on top of linux-next fixes this issue.
> > 
> > Let me know if you need more information.
> 
> Thank you for the report. I guess that the issue is related to
> out-of-range access of dbi2:
> - In arch/arm64/boot/dts/exynos/exynos5433.dtsi, the dbi reg size is 0x1000
>   like below:
> -----
>                 pcie: pcie@15700000 {
>                         compatible = "samsung,exynos5433-pcie";
>                         reg = <0x15700000 0x1000>, <0x156b0000 0x1000>,
>                               <0x0c000000 0x1000>;
>                         reg-names = "dbi", "elbi", "config";
> ...
> -----
> 
> - In drivers/pci/controller/dwc/pcie-designware.c, "dbi2" area is calculated
>   by the following if reg-names "dbi2" didn't exist:
> -----
>                         pci->dbi_base2 = pci->dbi_base + SZ_4K;
> -----
> 
> - However, this is out-of-memory on exynos5433.dtsi because the "dbi" size is
>   0x1000 only.
> - And then, this patch always writes PCI_BASE_ADDRESS_[01] to dbi2 area.
>   So, since this is out-of-range, the kernel panic happens.
> 

I could reproduce the issue Marek reported on RB5. As you pointed out, it is due
to not mapping dbi2 explicitly. But we were not using DBI2 on the host earlier
and it looks to me that DBI2 may not be implemented on Qcom host platforms.

Atleast on EP, I confirmed with Qcom that DBI=DBI2 as represented in the driver,
but I couldn't confirm if it is same for host as well.

> Perhaps, we should revert this patch at first. And, add the settings into
> my environment (pcie-rcar-gen4.c) only. I also have alternative solution which
> modifies the "dbi2" area calculation and avoid out-of-range access somehow.
> But, it may complicate source code...
> 

Yes, let's revert this patch for now and move it to rcar driver.

- Mani

> Best regards,
> Yoshihiro Shimoda
> 
> > > ---
> > >   drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
> > >   1 file changed, 8 insertions(+)
> > >
> > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> > > index a7170fd0e847..56cc7ff6d508 100644
> > > --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> > > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> > > @@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
> > >   	u32 val, ctrl, num_ctrls;
> > >   	int ret;
> > >
> > > +	/*
> > > +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > > +	 * Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> > > +	 * assignment during device enumeration.
> > > +	 */
> > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> > > +
> > >   	/*
> > >   	 * Enable DBI read-only registers for writing/updating configuration.
> > >   	 * Write permission gets disabled towards the end of this function.
> > 
> > Best regards
> > --
> > Marek Szyprowski, PhD
> > Samsung R&D Institute Poland
> 

-- 
மணிவண்ணன் சதாசிவம்

RE: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Yoshihiro Shimoda]

> From: mani@kernel.org, Sent: Wednesday, October 18, 2023 12:16 AM
> 
> On Tue, Oct 17, 2023 at 12:05:12PM +0000, Yoshihiro Shimoda wrote:
> > Dear Marek,
> >
> > > From: Marek Szyprowski, Sent: Tuesday, October 17, 2023 6:19 PM
> > >
> > > Dear All,
> > >
> > > On 11.10.2023 09:14, Yoshihiro Shimoda wrote:
> > > > According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > > > Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> > > > assignment during device enumeration. Otherwise, Renesas R-Car Gen4
> > > > PCIe controllers cannot work correctly in host mode.
> > > >
> > > > Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
> > >
> > > This patch landed in today's linux-next 20231017 as commit e308528cac3e
> > > ("PCI: dwc: Disable two BARs to avoid unnecessary memory assignment").
> > > Unfortunately it causes the following kernel panic on Samsung
> > > Exynos5433-based TM2e board:
> > >
> > > exynos-pcie 15700000.pcie: host bridge /soc@0/pcie@15700000 ranges:
> > > exynos-pcie 15700000.pcie:       IO 0x000c001000..0x000c010fff ->
> > > 0x0000000000
> > > exynos-pcie 15700000.pcie:      MEM 0x000c011000..0x000ffffffe ->
> > > 0x000c011000
> > > exynos-pcie 15700000.pcie: iATU: unroll F, 3 ob, 5 ib, align 4K, limit 4G
> > > Unable to handle kernel paging request at virtual address ffff800084196010
> > > Mem abort info:
> > > ...
> > > Data abort info:
> > > ...
> > > swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000022047000
> > > [ffff800084196010] pgd=10000000df6ff003, p4d=10000000df6ff003,
> > > pud=10000000df6fe003, pmd=1000000024ad9003, pte=0000000000000000
> > > Internal error: Oops: 0000000096000047 [#1] PREEMPT SMP
> > > Modules linked in:
> > > CPU: 4 PID: 55 Comm: kworker/u18:0 Not tainted 6.6.0-rc1+ #14129
> > > Hardware name: Samsung TM2E board (DT)
> > > Workqueue: events_unbound deferred_probe_work_func
> > > pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > > pc : dw_pcie_write_dbi2+0xb8/0xc8
> > > lr : dw_pcie_setup_rc+0x30/0x4e4
> > > ...
> > > Call trace:
> > >   dw_pcie_write_dbi2+0xb8/0xc8
> > >   dw_pcie_setup_rc+0x30/0x4e4
> > >   dw_pcie_host_init+0x238/0x608
> > >   exynos_pcie_probe+0x23c/0x340
> > >   platform_probe+0x68/0xd8
> > >   really_probe+0x148/0x2b4
> > >   __driver_probe_device+0x78/0x12c
> > >   driver_probe_device+0xd8/0x160
> > >   __device_attach_driver+0xb8/0x138
> > >   bus_for_each_drv+0x84/0xe0
> > >   __device_attach+0xa8/0x1b0
> > >   device_initial_probe+0x14/0x20
> > >   bus_probe_device+0xb0/0xb4
> > >   deferred_probe_work_func+0x8c/0xc8
> > >   process_one_work+0x1ec/0x53c
> > >   worker_thread+0x298/0x408
> > >   kthread+0x124/0x128
> > >   ret_from_fork+0x10/0x20
> > > Code: d50332bf 79000023 17ffffe2 d50332bf (b9000023)
> > > ---[ end trace 0000000000000000 ]---
> > > Kernel panic - not syncing: Oops: Fatal exception
> > > SMP: stopping secondary CPUs
> > > Kernel Offset: disabled
> > > CPU features: 0x8c00020e,3c020000,0000421b
> > > Memory Limit: none
> > > ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---
> > >
> > > I've observed similar issue on Qualcomm's RB5 platform with some
> > > additional not-yet merged patches enabling PCIe support. Reverting
> > > $subject on top of linux-next fixes this issue.
> > >
> > > Let me know if you need more information.
> >
> > Thank you for the report. I guess that the issue is related to
> > out-of-range access of dbi2:
> > - In arch/arm64/boot/dts/exynos/exynos5433.dtsi, the dbi reg size is 0x1000
> >   like below:
> > -----
> >                 pcie: pcie@15700000 {
> >                         compatible = "samsung,exynos5433-pcie";
> >                         reg = <0x15700000 0x1000>, <0x156b0000 0x1000>,
> >                               <0x0c000000 0x1000>;
> >                         reg-names = "dbi", "elbi", "config";
> > ...
> > -----
> >
> > - In drivers/pci/controller/dwc/pcie-designware.c, "dbi2" area is calculated
> >   by the following if reg-names "dbi2" didn't exist:
> > -----
> >                         pci->dbi_base2 = pci->dbi_base + SZ_4K;
> > -----
> >
> > - However, this is out-of-memory on exynos5433.dtsi because the "dbi" size is
> >   0x1000 only.
> > - And then, this patch always writes PCI_BASE_ADDRESS_[01] to dbi2 area.
> >   So, since this is out-of-range, the kernel panic happens.
> >
> 
> I could reproduce the issue Marek reported on RB5. As you pointed out, it is due
> to not mapping dbi2 explicitly. But we were not using DBI2 on the host earlier
> and it looks to me that DBI2 may not be implemented on Qcom host platforms.
> 
> Atleast on EP, I confirmed with Qcom that DBI=DBI2 as represented in the driver,
> but I couldn't confirm if it is same for host as well.

Thank you for the confirmation. I understood it.

> > Perhaps, we should revert this patch at first. And, add the settings into
> > my environment (pcie-rcar-gen4.c) only. I also have alternative solution which
> > modifies the "dbi2" area calculation and avoid out-of-range access somehow.
> > But, it may complicate source code...
> >
> 
> Yes, let's revert this patch for now and move it to rcar driver.

I got it.

Best regards,
Yoshihiro Shimoda

> - Mani
> 
> > Best regards,
> > Yoshihiro Shimoda
> >
> > > > ---
> > > >   drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
> > > >   1 file changed, 8 insertions(+)
> > > >
> > > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> > > > index a7170fd0e847..56cc7ff6d508 100644
> > > > --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> > > > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> > > > @@ -737,6 +737,14 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
> > > >   	u32 val, ctrl, num_ctrls;
> > > >   	int ret;
> > > >
> > > > +	/*
> > > > +	 * According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > > > +	 * Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> > > > +	 * assignment during device enumeration.
> > > > +	 */
> > > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_0, 0x0);
> > > > +	dw_pcie_writel_dbi2(pci, PCI_BASE_ADDRESS_1, 0x0);
> > > > +
> > > >   	/*
> > > >   	 * Enable DBI read-only registers for writing/updating configuration.
> > > >   	 * Write permission gets disabled towards the end of this function.
> > >
> > > Best regards
> > > --
> > > Marek Szyprowski, PhD
> > > Samsung R&D Institute Poland
> >
> 
> --
> மணிவண்ணன் சதாசிவம்

Re: [PATCH v24 08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment

[Bjorn Helgaas]

On Tue, Oct 17, 2023 at 11:19:21AM +0200, Marek Szyprowski wrote:
> Dear All,
> 
> On 11.10.2023 09:14, Yoshihiro Shimoda wrote:
> > According to the section 3.5.7.2 "RC Mode" in DWC PCIe Dual Mode
> > Rev.5.20a, we should disable two BARs to avoid unnecessary memory
> > assignment during device enumeration. Otherwise, Renesas R-Car Gen4
> > PCIe controllers cannot work correctly in host mode.
> >
> > Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
> 
> This patch landed in today's linux-next 20231017 as commit e308528cac3e 
> ("PCI: dwc: Disable two BARs to avoid unnecessary memory assignment"). 
> Unfortunately it causes the following kernel panic on Samsung 
> Exynos5433-based TM2e board:

I dropped the pci/controller/rcar branch for today until we resolve
this.

If you want to reorder the series like this:

  PCI: Add T_PVPERL macro
  PCI: dwc: Add dw_pcie_link_set_max_link_width()
  PCI: dwc: Add missing PCI_EXP_LNKCAP_MLW handling
  PCI: tegra194: Drop PCI_EXP_LNKSTA_NLW setting

  PCI: dwc: endpoint: Add multiple PFs support for dbi2
  PCI: dwc: Add EDMA_UNROLL capability flag
  PCI: dwc: Expose dw_pcie_ep_exit() to module
  PCI: dwc: endpoint: Introduce .pre_init() and .deinit()
  PCI: dwc: Disable two BARs to avoid unnecessary memory assignment
  dt-bindings: PCI: dwc: Update maxItems of reg and reg-names
  dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Host
  dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Endpoint
  PCI: rcar-gen4: Add R-Car Gen4 PCIe controller support for host mode
  PCI: rcar-gen4: Add endpoint mode support
  MAINTAINERS: Update PCI DRIVER FOR RENESAS R-CAR for R-Car Gen4
  misc: pci_endpoint_test: Add Device ID for R-Car S4-8 PCIe controller

I think the first four patches could be applied because they're useful
regardless of R-Car support.

The rest are only needed for R-Car, and it seems like we should apply
them all together as soon as this regression is solved.

Bjorn

[PATCH v24 09/16] dt-bindings: PCI: dwc: Update maxItems of reg and reg-names

[Yoshihiro Shimoda]

Update maxItems of reg and reg-names on both host and endpoint
for supporting Renesas R-Car Gen4 PCIe controllers later.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Acked-by: Manivannan Sadhasivam <mani@kernel.org>
Acked-by: Rob Herring <robh@kernel.org>
---
 .../devicetree/bindings/pci/snps,dw-pcie-common.yaml          | 4 ++--
 Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml    | 4 ++--
 Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml       | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/Documentation/devicetree/bindings/pci/snps,dw-pcie-common.yaml b/Documentation/devicetree/bindings/pci/snps,dw-pcie-common.yaml
index d87e13496834..dc05761c5cf9 100644
--- a/Documentation/devicetree/bindings/pci/snps,dw-pcie-common.yaml
+++ b/Documentation/devicetree/bindings/pci/snps,dw-pcie-common.yaml
@@ -33,11 +33,11 @@ properties:
       specific for each activated function, while the rest of the sub-spaces
       are common for all of them (if there are more than one).
     minItems: 2
-    maxItems: 6
+    maxItems: 7
 
   reg-names:
     minItems: 2
-    maxItems: 6
+    maxItems: 7
 
   interrupts:
     description:
diff --git a/Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml b/Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml
index 8fc2151691a4..bbdb01d22848 100644
--- a/Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml
+++ b/Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml
@@ -33,11 +33,11 @@ properties:
       normal controller functioning. iATU memory IO region is also required
       if the space is unrolled (IP-core version >= 4.80a).
     minItems: 2
-    maxItems: 5
+    maxItems: 7
 
   reg-names:
     minItems: 2
-    maxItems: 5
+    maxItems: 7
     items:
       oneOf:
         - description:
diff --git a/Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml b/Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml
index 8bbdeb8821f8..022055edbf9e 100644
--- a/Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml
@@ -42,11 +42,11 @@ properties:
       are required for the normal controller work. iATU memory IO region is
       also required if the space is unrolled (IP-core version >= 4.80a).
     minItems: 2
-    maxItems: 5
+    maxItems: 7
 
   reg-names:
     minItems: 2
-    maxItems: 5
+    maxItems: 7
     items:
       oneOf:
         - description:
-- 
2.25.1

[PATCH v24 10/16] dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Host

[Yoshihiro Shimoda]

Document bindings for Renesas R-Car Gen4 and R-Car S4-8 (R8A779F0)
PCIe host module.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
---
 .../bindings/pci/rcar-gen4-pci-host.yaml      | 127 ++++++++++++++++++
 1 file changed, 127 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/rcar-gen4-pci-host.yaml

diff --git a/Documentation/devicetree/bindings/pci/rcar-gen4-pci-host.yaml b/Documentation/devicetree/bindings/pci/rcar-gen4-pci-host.yaml
new file mode 100644
index 000000000000..ffb34339b637
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/rcar-gen4-pci-host.yaml
@@ -0,0 +1,127 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (C) 2022-2023 Renesas Electronics Corp.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pci/rcar-gen4-pci-host.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Renesas R-Car Gen4 PCIe Host
+
+maintainers:
+  - Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
+
+allOf:
+  - $ref: snps,dw-pcie.yaml#
+
+properties:
+  compatible:
+    items:
+      - const: renesas,r8a779f0-pcie   # R-Car S4-8
+      - const: renesas,rcar-gen4-pcie  # R-Car Gen4
+
+  reg:
+    maxItems: 7
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: dbi2
+      - const: atu
+      - const: dma
+      - const: app
+      - const: phy
+      - const: config
+
+  interrupts:
+    maxItems: 4
+
+  interrupt-names:
+    items:
+      - const: msi
+      - const: dma
+      - const: sft_ce
+      - const: app
+
+  clocks:
+    maxItems: 2
+
+  clock-names:
+    items:
+      - const: core
+      - const: ref
+
+  power-domains:
+    maxItems: 1
+
+  resets:
+    maxItems: 1
+
+  reset-names:
+    items:
+      - const: pwr
+
+  max-link-speed:
+    maximum: 4
+
+  num-lanes:
+    maximum: 4
+
+required:
+  - compatible
+  - reg
+  - reg-names
+  - interrupts
+  - interrupt-names
+  - clocks
+  - clock-names
+  - power-domains
+  - resets
+  - reset-names
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/clock/r8a779f0-cpg-mssr.h>
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/power/r8a779f0-sysc.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        pcie: pcie@e65d0000 {
+            compatible = "renesas,r8a779f0-pcie", "renesas,rcar-gen4-pcie";
+            reg = <0 0xe65d0000 0 0x1000>, <0 0xe65d2000 0 0x0800>,
+                  <0 0xe65d3000 0 0x2000>, <0 0xe65d5000 0 0x1200>,
+                  <0 0xe65d6200 0 0x0e00>, <0 0xe65d7000 0 0x0400>,
+                  <0 0xfe000000 0 0x400000>;
+            reg-names = "dbi", "dbi2", "atu", "dma", "app", "phy", "config";
+            interrupts = <GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>,
+                         <GIC_SPI 417 IRQ_TYPE_LEVEL_HIGH>,
+                         <GIC_SPI 418 IRQ_TYPE_LEVEL_HIGH>,
+                         <GIC_SPI 422 IRQ_TYPE_LEVEL_HIGH>;
+            interrupt-names = "msi", "dma", "sft_ce", "app";
+            clocks = <&cpg CPG_MOD 624>, <&pcie0_clkref>;
+            clock-names = "core", "ref";
+            power-domains = <&sysc R8A779F0_PD_ALWAYS_ON>;
+            resets = <&cpg 624>;
+            reset-names = "pwr";
+            max-link-speed = <4>;
+            num-lanes = <2>;
+            #address-cells = <3>;
+            #size-cells = <2>;
+            bus-range = <0x00 0xff>;
+            device_type = "pci";
+            ranges = <0x01000000 0 0x00000000 0 0xfe000000 0 0x00400000>,
+                     <0x02000000 0 0x30000000 0 0x30000000 0 0x10000000>;
+            dma-ranges = <0x42000000 0 0x00000000 0 0x00000000 1 0x00000000>;
+            #interrupt-cells = <1>;
+            interrupt-map-mask = <0 0 0 7>;
+            interrupt-map = <0 0 0 1 &gic GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>,
+                            <0 0 0 2 &gic GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>,
+                            <0 0 0 3 &gic GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>,
+                            <0 0 0 4 &gic GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>;
+            snps,enable-cdm-check;
+        };
+    };
-- 
2.25.1

[PATCH v24 11/16] dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Endpoint

[Yoshihiro Shimoda]

Document bindings for Renesas R-Car Gen4 and R-Car S4-8 (R8A779F0)
PCIe endpoint module.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Acked-by: Manivannan Sadhasivam <mani@kernel.org>
---
 .../bindings/pci/rcar-gen4-pci-ep.yaml        | 115 ++++++++++++++++++
 1 file changed, 115 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/rcar-gen4-pci-ep.yaml

diff --git a/Documentation/devicetree/bindings/pci/rcar-gen4-pci-ep.yaml b/Documentation/devicetree/bindings/pci/rcar-gen4-pci-ep.yaml
new file mode 100644
index 000000000000..fe38f62da066
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/rcar-gen4-pci-ep.yaml
@@ -0,0 +1,115 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (C) 2022-2023 Renesas Electronics Corp.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pci/rcar-gen4-pci-ep.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Renesas R-Car Gen4 PCIe Endpoint
+
+maintainers:
+  - Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
+
+allOf:
+  - $ref: snps,dw-pcie-ep.yaml#
+
+properties:
+  compatible:
+    items:
+      - const: renesas,r8a779f0-pcie-ep   # R-Car S4-8
+      - const: renesas,rcar-gen4-pcie-ep  # R-Car Gen4
+
+  reg:
+    maxItems: 7
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: dbi2
+      - const: atu
+      - const: dma
+      - const: app
+      - const: phy
+      - const: addr_space
+
+  interrupts:
+    maxItems: 3
+
+  interrupt-names:
+    items:
+      - const: dma
+      - const: sft_ce
+      - const: app
+
+  clocks:
+    maxItems: 2
+
+  clock-names:
+    items:
+      - const: core
+      - const: ref
+
+  power-domains:
+    maxItems: 1
+
+  resets:
+    maxItems: 1
+
+  reset-names:
+    items:
+      - const: pwr
+
+  max-link-speed:
+    maximum: 4
+
+  num-lanes:
+    maximum: 4
+
+  max-functions:
+    maximum: 2
+
+required:
+  - compatible
+  - reg
+  - reg-names
+  - interrupts
+  - interrupt-names
+  - clocks
+  - clock-names
+  - power-domains
+  - resets
+  - reset-names
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/clock/r8a779f0-cpg-mssr.h>
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/power/r8a779f0-sysc.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        pcie0_ep: pcie-ep@e65d0000 {
+            compatible = "renesas,r8a779f0-pcie-ep", "renesas,rcar-gen4-pcie-ep";
+            reg = <0 0xe65d0000 0 0x2000>, <0 0xe65d2000 0 0x1000>,
+                  <0 0xe65d3000 0 0x2000>, <0 0xe65d5000 0 0x1200>,
+                  <0 0xe65d6200 0 0x0e00>, <0 0xe65d7000 0 0x0400>,
+                  <0 0xfe000000 0 0x400000>;
+            reg-names = "dbi", "dbi2", "atu", "dma", "app", "phy", "addr_space";
+            interrupts = <GIC_SPI 417 IRQ_TYPE_LEVEL_HIGH>,
+                         <GIC_SPI 418 IRQ_TYPE_LEVEL_HIGH>,
+                         <GIC_SPI 422 IRQ_TYPE_LEVEL_HIGH>;
+            interrupt-names = "dma", "sft_ce", "app";
+            clocks = <&cpg CPG_MOD 624>, <&pcie0_clkref>;
+            clock-names = "core", "ref";
+            power-domains = <&sysc R8A779F0_PD_ALWAYS_ON>;
+            resets = <&cpg 624>;
+            reset-names = "pwr";
+            max-link-speed = <4>;
+            num-lanes = <2>;
+            max-functions = /bits/ 8 <2>;
+        };
+    };
-- 
2.25.1

[PATCH v24 12/16] PCI: Add T_PVPERL macro

[Yoshihiro Shimoda]

According to the PCIe CEM r5.0, sec 2.9.2, Power stable to PERST#
inactive interval is 100 ms as minimum. Add a macro so that the PCIe
controller drivers can make use of it.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/pci.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 39a8932dc340..5ecbcf041179 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -13,6 +13,9 @@
 
 #define PCIE_LINK_RETRAIN_TIMEOUT_MS	1000
 
+/* Power stable to PERST# inactive from PCIe card Electromechanical Spec */
+#define PCIE_T_PVPERL_MS		100
+
 /*
  * PCIe r6.0, sec 5.3.3.2.1 <PME Synchronization>
  * Recommends 1ms to 10ms timeout to check L2 ready.
-- 
2.25.1

[PATCH v24 13/16] PCI: rcar-gen4: Add R-Car Gen4 PCIe controller support for host mode

[Yoshihiro Shimoda]

Add R-Car Gen4 PCIe controller support for host mode.

This controller is based on Synopsys DesignWare PCIe. However, this
particular controller has a number of vendor-specific registers, and as
such, requires initialization code like mode setting and retraining and
so on.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
---
 drivers/pci/controller/dwc/Kconfig          |  14 +
 drivers/pci/controller/dwc/Makefile         |   1 +
 drivers/pci/controller/dwc/pcie-rcar-gen4.c | 369 ++++++++++++++++++++
 3 files changed, 384 insertions(+)
 create mode 100644 drivers/pci/controller/dwc/pcie-rcar-gen4.c

diff --git a/drivers/pci/controller/dwc/Kconfig b/drivers/pci/controller/dwc/Kconfig
index ab96da43e0c2..306fc71493fc 100644
--- a/drivers/pci/controller/dwc/Kconfig
+++ b/drivers/pci/controller/dwc/Kconfig
@@ -286,6 +286,20 @@ config PCIE_QCOM_EP
 	  to work in endpoint mode. The PCIe controller uses the DesignWare core
 	  plus Qualcomm-specific hardware wrappers.
 
+config PCIE_RCAR_GEN4
+	tristate
+
+config PCIE_RCAR_GEN4_HOST
+	tristate "Renesas R-Car Gen4 PCIe controller (host mode)"
+	depends on ARCH_RENESAS || COMPILE_TEST
+	depends on PCI_MSI
+	select PCIE_DW_HOST
+	select PCIE_RCAR_GEN4
+	help
+	  Say Y here if you want PCIe controller (host mode) on R-Car Gen4 SoCs.
+	  To compile this driver as a module, choose M here: the module will be
+	  called pcie-rcar-gen4.ko. This uses the DesignWare core.
+
 config PCIE_ROCKCHIP_DW_HOST
 	bool "Rockchip DesignWare PCIe controller"
 	select PCIE_DW
diff --git a/drivers/pci/controller/dwc/Makefile b/drivers/pci/controller/dwc/Makefile
index bf5c311875a1..bac103faa523 100644
--- a/drivers/pci/controller/dwc/Makefile
+++ b/drivers/pci/controller/dwc/Makefile
@@ -26,6 +26,7 @@ obj-$(CONFIG_PCIE_TEGRA194) += pcie-tegra194.o
 obj-$(CONFIG_PCIE_UNIPHIER) += pcie-uniphier.o
 obj-$(CONFIG_PCIE_UNIPHIER_EP) += pcie-uniphier-ep.o
 obj-$(CONFIG_PCIE_VISCONTI_HOST) += pcie-visconti.o
+obj-$(CONFIG_PCIE_RCAR_GEN4) += pcie-rcar-gen4.o
 
 # The following drivers are for devices that use the generic ACPI
 # pci_root.c driver but don't support standard ECAM config access.
diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
new file mode 100644
index 000000000000..098b840bcc31
--- /dev/null
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -0,0 +1,369 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * PCIe controller driver for Renesas R-Car Gen4 Series SoCs
+ * Copyright (C) 2022-2023 Renesas Electronics Corporation
+ */
+
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/reset.h>
+
+#include "../../pci.h"
+#include "pcie-designware.h"
+
+/* Renesas-specific */
+/* PCIe Mode Setting Register 0 */
+#define PCIEMSR0		0x0000
+#define BIFUR_MOD_SET_ON	BIT(0)
+#define DEVICE_TYPE_RC		BIT(4)
+
+/* PCIe Interrupt Status 0 */
+#define PCIEINTSTS0		0x0084
+
+/* PCIe Interrupt Status 0 Enable */
+#define PCIEINTSTS0EN		0x0310
+#define MSI_CTRL_INT		BIT(26)
+#define SMLH_LINK_UP		BIT(7)
+#define RDLH_LINK_UP		BIT(6)
+
+/* PCIe DMA Interrupt Status Enable */
+#define PCIEDMAINTSTSEN		0x0314
+#define PCIEDMAINTSTSEN_INIT	GENMASK(15, 0)
+
+/* PCIe Reset Control Register 1 */
+#define PCIERSTCTRL1		0x0014
+#define APP_HOLD_PHY_RST	BIT(16)
+#define APP_LTSSM_ENABLE	BIT(0)
+
+#define RCAR_NUM_SPEED_CHANGE_RETRIES	10
+#define RCAR_MAX_LINK_SPEED		4
+
+struct rcar_gen4_pcie {
+	struct dw_pcie dw;
+	void __iomem *base;
+	struct platform_device *pdev;
+	enum dw_pcie_device_mode mode;
+};
+#define to_rcar_gen4_pcie(_dw)	container_of(_dw, struct rcar_gen4_pcie, dw)
+
+static void rcar_gen4_pcie_ltssm_enable(struct rcar_gen4_pcie *rcar,
+					bool enable)
+{
+	u32 val;
+
+	val = readl(rcar->base + PCIERSTCTRL1);
+	if (enable) {
+		val |= APP_LTSSM_ENABLE;
+		val &= ~APP_HOLD_PHY_RST;
+	} else {
+		/*
+		 * Since the datasheet of R-Car doesn't mention how to assert
+		 * the APP_HOLD_PHY_RST, don't assert it again. Otherwise,
+		 * hang-up issue happened in the dw_edma_core_off() when
+		 * the controller didn't detect a PCI device.
+		 */
+		val &= ~APP_LTSSM_ENABLE;
+	}
+	writel(val, rcar->base + PCIERSTCTRL1);
+}
+
+static int rcar_gen4_pcie_link_up(struct dw_pcie *dw)
+{
+	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
+	u32 val, mask;
+
+	val = readl(rcar->base + PCIEINTSTS0);
+	mask = RDLH_LINK_UP | SMLH_LINK_UP;
+
+	return (val & mask) == mask;
+}
+
+/*
+ * Manually initiate the speed change. Return 0 if change succeeded; otherwise
+ * -ETIMEDOUT.
+ */
+static int rcar_gen4_pcie_speed_change(struct dw_pcie *dw)
+{
+	u32 val;
+	int i;
+
+	val = dw_pcie_readl_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_SPEED_CHANGE;
+	dw_pcie_writel_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+
+	val = dw_pcie_readl_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val |= PORT_LOGIC_SPEED_CHANGE;
+	dw_pcie_writel_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+
+	for (i = 0; i < RCAR_NUM_SPEED_CHANGE_RETRIES; i++) {
+		val = dw_pcie_readl_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL);
+		if (!(val & PORT_LOGIC_SPEED_CHANGE))
+			return 0;
+		usleep_range(10000, 11000);
+	}
+
+	return -ETIMEDOUT;
+}
+
+/*
+ * Enable LTSSM of this controller and manually initiate the speed change.
+ * Always return 0.
+ */
+static int rcar_gen4_pcie_start_link(struct dw_pcie *dw)
+{
+	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
+	int i, changes;
+
+	rcar_gen4_pcie_ltssm_enable(rcar, true);
+
+	/*
+	 * Require direct speed change with retrying here if the link_gen is
+	 * PCIe Gen2 or higher.
+	 */
+	changes = min_not_zero(dw->link_gen, RCAR_MAX_LINK_SPEED) - 1;
+
+	/*
+	 * Since dw_pcie_setup_rc() sets it once, PCIe Gen2 will be trained.
+	 * So, this needs remaining times for up to PCIe Gen4 if RC mode.
+	 */
+	if (changes && rcar->mode == DW_PCIE_RC_TYPE)
+		changes--;
+
+	for (i = 0; i < changes; i++) {
+		/* It may not be connected in EP mode yet. So, break the loop */
+		if (rcar_gen4_pcie_speed_change(dw))
+			break;
+	}
+
+	return 0;
+}
+
+static void rcar_gen4_pcie_stop_link(struct dw_pcie *dw)
+{
+	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
+
+	rcar_gen4_pcie_ltssm_enable(rcar, false);
+}
+
+static int rcar_gen4_pcie_common_init(struct rcar_gen4_pcie *rcar)
+{
+	struct dw_pcie *dw = &rcar->dw;
+	u32 val;
+	int ret;
+
+	ret = clk_bulk_prepare_enable(DW_PCIE_NUM_CORE_CLKS, dw->core_clks);
+	if (ret) {
+		dev_err(dw->dev, "Enabling core clocks failed\n");
+		return ret;
+	}
+
+	if (!reset_control_status(dw->core_rsts[DW_PCIE_PWR_RST].rstc))
+		reset_control_assert(dw->core_rsts[DW_PCIE_PWR_RST].rstc);
+
+	val = readl(rcar->base + PCIEMSR0);
+	if (rcar->mode == DW_PCIE_RC_TYPE) {
+		val |= DEVICE_TYPE_RC;
+	} else {
+		ret = -EINVAL;
+		goto err_unprepare;
+	}
+
+	if (dw->num_lanes < 4)
+		val |= BIFUR_MOD_SET_ON;
+
+	writel(val, rcar->base + PCIEMSR0);
+
+	ret = reset_control_deassert(dw->core_rsts[DW_PCIE_PWR_RST].rstc);
+	if (ret)
+		goto err_unprepare;
+
+	return 0;
+
+err_unprepare:
+	clk_bulk_disable_unprepare(DW_PCIE_NUM_CORE_CLKS, dw->core_clks);
+
+	return ret;
+}
+
+static void rcar_gen4_pcie_common_deinit(struct rcar_gen4_pcie *rcar)
+{
+	struct dw_pcie *dw = &rcar->dw;
+
+	reset_control_assert(dw->core_rsts[DW_PCIE_PWR_RST].rstc);
+	clk_bulk_disable_unprepare(DW_PCIE_NUM_CORE_CLKS, dw->core_clks);
+}
+
+static int rcar_gen4_pcie_prepare(struct rcar_gen4_pcie *rcar)
+{
+	struct device *dev = rcar->dw.dev;
+	int err;
+
+	pm_runtime_enable(dev);
+	err = pm_runtime_resume_and_get(dev);
+	if (err < 0) {
+		dev_err(dev, "Runtime resume failed\n");
+		pm_runtime_disable(dev);
+	}
+
+	return err;
+}
+
+static void rcar_gen4_pcie_unprepare(struct rcar_gen4_pcie *rcar)
+{
+	struct device *dev = rcar->dw.dev;
+
+	pm_runtime_put(dev);
+	pm_runtime_disable(dev);
+}
+
+static int rcar_gen4_pcie_get_resources(struct rcar_gen4_pcie *rcar)
+{
+	/* Renesas-specific registers */
+	rcar->base = devm_platform_ioremap_resource_byname(rcar->pdev, "app");
+
+	return PTR_ERR_OR_ZERO(rcar->base);
+}
+
+static const struct dw_pcie_ops dw_pcie_ops = {
+	.start_link = rcar_gen4_pcie_start_link,
+	.stop_link = rcar_gen4_pcie_stop_link,
+	.link_up = rcar_gen4_pcie_link_up,
+};
+
+static struct rcar_gen4_pcie *rcar_gen4_pcie_alloc(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct rcar_gen4_pcie *rcar;
+
+	rcar = devm_kzalloc(dev, sizeof(*rcar), GFP_KERNEL);
+	if (!rcar)
+		return ERR_PTR(-ENOMEM);
+
+	rcar->dw.ops = &dw_pcie_ops;
+	rcar->dw.dev = dev;
+	rcar->pdev = pdev;
+	dw_pcie_cap_set(&rcar->dw, EDMA_UNROLL);
+	dw_pcie_cap_set(&rcar->dw, REQ_RES);
+	platform_set_drvdata(pdev, rcar);
+
+	return rcar;
+}
+
+/* Host mode */
+static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *dw = to_dw_pcie_from_pp(pp);
+	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
+	int ret;
+	u32 val;
+
+	gpiod_set_value_cansleep(dw->pe_rst, 1);
+
+	ret = rcar_gen4_pcie_common_init(rcar);
+	if (ret)
+		return ret;
+
+	/* Enable MSI interrupt signal */
+	val = readl(rcar->base + PCIEINTSTS0EN);
+	val |= MSI_CTRL_INT;
+	writel(val, rcar->base + PCIEINTSTS0EN);
+
+	msleep(PCIE_T_PVPERL_MS);	/* pe_rst requires 100msec delay */
+
+	gpiod_set_value_cansleep(dw->pe_rst, 0);
+
+	return 0;
+}
+
+static void rcar_gen4_pcie_host_deinit(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *dw = to_dw_pcie_from_pp(pp);
+	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
+
+	gpiod_set_value_cansleep(dw->pe_rst, 1);
+	rcar_gen4_pcie_common_deinit(rcar);
+}
+
+static const struct dw_pcie_host_ops rcar_gen4_pcie_host_ops = {
+	.host_init = rcar_gen4_pcie_host_init,
+	.host_deinit = rcar_gen4_pcie_host_deinit,
+};
+
+static int rcar_gen4_add_dw_pcie_rp(struct rcar_gen4_pcie *rcar)
+{
+	struct dw_pcie_rp *pp = &rcar->dw.pp;
+
+	pp->num_vectors = MAX_MSI_IRQS;
+	pp->ops = &rcar_gen4_pcie_host_ops;
+	rcar->mode = DW_PCIE_RC_TYPE;
+
+	return dw_pcie_host_init(pp);
+}
+
+static void rcar_gen4_remove_dw_pcie_rp(struct rcar_gen4_pcie *rcar)
+{
+	dw_pcie_host_deinit(&rcar->dw.pp);
+}
+
+static int rcar_gen4_pcie_probe(struct platform_device *pdev)
+{
+	struct rcar_gen4_pcie *rcar;
+	int err;
+
+	rcar = rcar_gen4_pcie_alloc(pdev);
+	if (IS_ERR(rcar))
+		return PTR_ERR(rcar);
+
+	err = rcar_gen4_pcie_get_resources(rcar);
+	if (err)
+		return err;
+
+	err = rcar_gen4_pcie_prepare(rcar);
+	if (err)
+		return err;
+
+	err = rcar_gen4_add_dw_pcie_rp(rcar);
+	if (err)
+		goto err_unprepare;
+
+	return 0;
+
+err_unprepare:
+	rcar_gen4_pcie_unprepare(rcar);
+
+	return err;
+}
+
+static void rcar_gen4_pcie_remove(struct platform_device *pdev)
+{
+	struct rcar_gen4_pcie *rcar = platform_get_drvdata(pdev);
+
+	rcar_gen4_remove_dw_pcie_rp(rcar);
+	rcar_gen4_pcie_unprepare(rcar);
+}
+
+static const struct of_device_id rcar_gen4_pcie_of_match[] = {
+	{ .compatible = "renesas,rcar-gen4-pcie", },
+	{},
+};
+MODULE_DEVICE_TABLE(of, rcar_gen4_pcie_of_match);
+
+static struct platform_driver rcar_gen4_pcie_driver = {
+	.driver = {
+		.name = "pcie-rcar-gen4",
+		.of_match_table = rcar_gen4_pcie_of_match,
+		.probe_type = PROBE_PREFER_ASYNCHRONOUS,
+	},
+	.probe = rcar_gen4_pcie_probe,
+	.remove_new = rcar_gen4_pcie_remove,
+};
+module_platform_driver(rcar_gen4_pcie_driver);
+
+MODULE_DESCRIPTION("Renesas R-Car Gen4 PCIe controller driver");
+MODULE_LICENSE("GPL");
-- 
2.25.1

[PATCH v24 14/16] PCI: rcar-gen4: Add endpoint mode support

[Yoshihiro Shimoda]

Add R-Car Gen4 PCIe controller for endpoint mode. This controller is based
on Synopsys DesignWare PCIe.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/Kconfig          |  11 ++
 drivers/pci/controller/dwc/pcie-rcar-gen4.c | 156 +++++++++++++++++++-
 2 files changed, 163 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/dwc/Kconfig b/drivers/pci/controller/dwc/Kconfig
index 306fc71493fc..5ac021dbd46a 100644
--- a/drivers/pci/controller/dwc/Kconfig
+++ b/drivers/pci/controller/dwc/Kconfig
@@ -300,6 +300,17 @@ config PCIE_RCAR_GEN4_HOST
 	  To compile this driver as a module, choose M here: the module will be
 	  called pcie-rcar-gen4.ko. This uses the DesignWare core.
 
+config PCIE_RCAR_GEN4_EP
+	tristate "Renesas R-Car Gen4 PCIe controller (endpoint mode)"
+	depends on ARCH_RENESAS || COMPILE_TEST
+	depends on PCI_ENDPOINT
+	select PCIE_DW_EP
+	select PCIE_RCAR_GEN4
+	help
+	  Say Y here if you want PCIe controller (endpoint mode) on R-Car Gen4
+	  SoCs. To compile this driver as a module, choose M here: the module
+	  will be called pcie-rcar-gen4.ko. This uses the DesignWare core.
+
 config PCIE_ROCKCHIP_DW_HOST
 	bool "Rockchip DesignWare PCIe controller"
 	select PCIE_DW
diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
index 098b840bcc31..619262d32f4e 100644
--- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -21,6 +21,7 @@
 /* PCIe Mode Setting Register 0 */
 #define PCIEMSR0		0x0000
 #define BIFUR_MOD_SET_ON	BIT(0)
+#define DEVICE_TYPE_EP		0
 #define DEVICE_TYPE_RC		BIT(4)
 
 /* PCIe Interrupt Status 0 */
@@ -44,6 +45,9 @@
 #define RCAR_NUM_SPEED_CHANGE_RETRIES	10
 #define RCAR_MAX_LINK_SPEED		4
 
+#define RCAR_GEN4_PCIE_EP_FUNC_DBI_OFFSET	0x1000
+#define RCAR_GEN4_PCIE_EP_FUNC_DBI2_OFFSET	0x800
+
 struct rcar_gen4_pcie {
 	struct dw_pcie dw;
 	void __iomem *base;
@@ -52,6 +56,7 @@ struct rcar_gen4_pcie {
 };
 #define to_rcar_gen4_pcie(_dw)	container_of(_dw, struct rcar_gen4_pcie, dw)
 
+/* Common */
 static void rcar_gen4_pcie_ltssm_enable(struct rcar_gen4_pcie *rcar,
 					bool enable)
 {
@@ -169,6 +174,8 @@ static int rcar_gen4_pcie_common_init(struct rcar_gen4_pcie *rcar)
 	val = readl(rcar->base + PCIEMSR0);
 	if (rcar->mode == DW_PCIE_RC_TYPE) {
 		val |= DEVICE_TYPE_RC;
+	} else if (rcar->mode == DW_PCIE_EP_TYPE) {
+		val |= DEVICE_TYPE_EP;
 	} else {
 		ret = -EINVAL;
 		goto err_unprepare;
@@ -299,9 +306,11 @@ static int rcar_gen4_add_dw_pcie_rp(struct rcar_gen4_pcie *rcar)
 {
 	struct dw_pcie_rp *pp = &rcar->dw.pp;
 
+	if (!IS_ENABLED(CONFIG_PCIE_RCAR_GEN4_HOST))
+		return -ENODEV;
+
 	pp->num_vectors = MAX_MSI_IRQS;
 	pp->ops = &rcar_gen4_pcie_host_ops;
-	rcar->mode = DW_PCIE_RC_TYPE;
 
 	return dw_pcie_host_init(pp);
 }
@@ -311,6 +320,124 @@ static void rcar_gen4_remove_dw_pcie_rp(struct rcar_gen4_pcie *rcar)
 	dw_pcie_host_deinit(&rcar->dw.pp);
 }
 
+/* Endpoint mode */
+static void rcar_gen4_pcie_ep_pre_init(struct dw_pcie_ep *ep)
+{
+	struct dw_pcie *dw = to_dw_pcie_from_ep(ep);
+	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
+	int ret;
+
+	ret = rcar_gen4_pcie_common_init(rcar);
+	if (ret)
+		return;
+
+	writel(PCIEDMAINTSTSEN_INIT, rcar->base + PCIEDMAINTSTSEN);
+}
+
+static void rcar_gen4_pcie_ep_init(struct dw_pcie_ep *ep)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+	enum pci_barno bar;
+
+	for (bar = 0; bar < PCI_STD_NUM_BARS; bar++)
+		dw_pcie_ep_reset_bar(pci, bar);
+}
+
+static void rcar_gen4_pcie_ep_deinit(struct dw_pcie_ep *ep)
+{
+	struct dw_pcie *dw = to_dw_pcie_from_ep(ep);
+	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
+
+	writel(0, rcar->base + PCIEDMAINTSTSEN);
+	rcar_gen4_pcie_common_deinit(rcar);
+}
+
+static int rcar_gen4_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
+				       unsigned int type, u16 interrupt_num)
+{
+	struct dw_pcie *dw = to_dw_pcie_from_ep(ep);
+
+	switch (type) {
+	case PCI_IRQ_LEGACY:
+		return dw_pcie_ep_raise_legacy_irq(ep, func_no);
+	case PCI_IRQ_MSI:
+		return dw_pcie_ep_raise_msi_irq(ep, func_no, interrupt_num);
+	default:
+		dev_err(dw->dev, "Unknown IRQ type\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static const struct pci_epc_features rcar_gen4_pcie_epc_features = {
+	.linkup_notifier = false,
+	.msi_capable = true,
+	.msix_capable = false,
+	.reserved_bar = 1 << BAR_1 | 1 << BAR_3 | 1 << BAR_5,
+	.align = SZ_1M,
+};
+
+static const struct pci_epc_features*
+rcar_gen4_pcie_ep_get_features(struct dw_pcie_ep *ep)
+{
+	return &rcar_gen4_pcie_epc_features;
+}
+
+static unsigned int rcar_gen4_pcie_ep_func_conf_select(struct dw_pcie_ep *ep,
+						       u8 func_no)
+{
+	return func_no * RCAR_GEN4_PCIE_EP_FUNC_DBI_OFFSET;
+}
+
+static unsigned int rcar_gen4_pcie_ep_get_dbi2_offset(struct dw_pcie_ep *ep,
+						      u8 func_no)
+{
+	return func_no * RCAR_GEN4_PCIE_EP_FUNC_DBI2_OFFSET;
+}
+
+static const struct dw_pcie_ep_ops pcie_ep_ops = {
+	.pre_init = rcar_gen4_pcie_ep_pre_init,
+	.ep_init = rcar_gen4_pcie_ep_init,
+	.deinit = rcar_gen4_pcie_ep_deinit,
+	.raise_irq = rcar_gen4_pcie_ep_raise_irq,
+	.get_features = rcar_gen4_pcie_ep_get_features,
+	.func_conf_select = rcar_gen4_pcie_ep_func_conf_select,
+	.get_dbi2_offset = rcar_gen4_pcie_ep_get_dbi2_offset,
+};
+
+static int rcar_gen4_add_dw_pcie_ep(struct rcar_gen4_pcie *rcar)
+{
+	struct dw_pcie_ep *ep = &rcar->dw.ep;
+
+	if (!IS_ENABLED(CONFIG_PCIE_RCAR_GEN4_EP))
+		return -ENODEV;
+
+	ep->ops = &pcie_ep_ops;
+
+	return dw_pcie_ep_init(ep);
+}
+
+static void rcar_gen4_remove_dw_pcie_ep(struct rcar_gen4_pcie *rcar)
+{
+	dw_pcie_ep_exit(&rcar->dw.ep);
+}
+
+/* Common */
+static int rcar_gen4_add_dw_pcie(struct rcar_gen4_pcie *rcar)
+{
+	rcar->mode = (enum dw_pcie_device_mode)of_device_get_match_data(&rcar->pdev->dev);
+
+	switch (rcar->mode) {
+	case DW_PCIE_RC_TYPE:
+		return rcar_gen4_add_dw_pcie_rp(rcar);
+	case DW_PCIE_EP_TYPE:
+		return rcar_gen4_add_dw_pcie_ep(rcar);
+	default:
+		return -EINVAL;
+	}
+}
+
 static int rcar_gen4_pcie_probe(struct platform_device *pdev)
 {
 	struct rcar_gen4_pcie *rcar;
@@ -328,7 +455,7 @@ static int rcar_gen4_pcie_probe(struct platform_device *pdev)
 	if (err)
 		return err;
 
-	err = rcar_gen4_add_dw_pcie_rp(rcar);
+	err = rcar_gen4_add_dw_pcie(rcar);
 	if (err)
 		goto err_unprepare;
 
@@ -340,16 +467,37 @@ static int rcar_gen4_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static void rcar_gen4_remove_dw_pcie(struct rcar_gen4_pcie *rcar)
+{
+	switch (rcar->mode) {
+	case DW_PCIE_RC_TYPE:
+		rcar_gen4_remove_dw_pcie_rp(rcar);
+		break;
+	case DW_PCIE_EP_TYPE:
+		rcar_gen4_remove_dw_pcie_ep(rcar);
+		break;
+	default:
+		break;
+	}
+}
+
 static void rcar_gen4_pcie_remove(struct platform_device *pdev)
 {
 	struct rcar_gen4_pcie *rcar = platform_get_drvdata(pdev);
 
-	rcar_gen4_remove_dw_pcie_rp(rcar);
+	rcar_gen4_remove_dw_pcie(rcar);
 	rcar_gen4_pcie_unprepare(rcar);
 }
 
 static const struct of_device_id rcar_gen4_pcie_of_match[] = {
-	{ .compatible = "renesas,rcar-gen4-pcie", },
+	{
+		.compatible = "renesas,rcar-gen4-pcie",
+		.data = (void *)DW_PCIE_RC_TYPE,
+	},
+	{
+		.compatible = "renesas,rcar-gen4-pcie-ep",
+		.data = (void *)DW_PCIE_EP_TYPE,
+	},
 	{},
 };
 MODULE_DEVICE_TABLE(of, rcar_gen4_pcie_of_match);
-- 
2.25.1

[PATCH v24 15/16] MAINTAINERS: Update PCI DRIVER FOR RENESAS R-CAR for R-Car Gen4

[Yoshihiro Shimoda]

Update this entry for R-Car Gen4's source code.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Manivannan Sadhasivam <mani@kernel.org>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 90f13281d297..4115e648b4af 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -16424,6 +16424,7 @@ L:	linux-renesas-soc@vger.kernel.org
 S:	Maintained
 F:	Documentation/devicetree/bindings/pci/*rcar*
 F:	drivers/pci/controller/*rcar*
+F:	drivers/pci/controller/dwc/*rcar*
 
 PCI DRIVER FOR SAMSUNG EXYNOS
 M:	Jingoo Han <jingoohan1@gmail.com>
-- 
2.25.1

[PATCH v24 16/16] misc: pci_endpoint_test: Add Device ID for R-Car S4-8 PCIe controller

[Yoshihiro Shimoda]

Add Renesas R8A779F0 in pci_device_id table so that pci-epf-test
can be used for testing PCIe EP on R-Car S4-8.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: Manivannan Sadhasivam <mani@kernel.org>
---
 drivers/misc/pci_endpoint_test.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/misc/pci_endpoint_test.c b/drivers/misc/pci_endpoint_test.c
index ed4d0ef5e5c3..150083dab71a 100644
--- a/drivers/misc/pci_endpoint_test.c
+++ b/drivers/misc/pci_endpoint_test.c
@@ -81,6 +81,7 @@
 #define PCI_DEVICE_ID_RENESAS_R8A774B1		0x002b
 #define PCI_DEVICE_ID_RENESAS_R8A774C0		0x002d
 #define PCI_DEVICE_ID_RENESAS_R8A774E1		0x0025
+#define PCI_DEVICE_ID_RENESAS_R8A779F0		0x0031
 
 static DEFINE_IDA(pci_endpoint_test_ida);
 
@@ -990,6 +991,9 @@ static const struct pci_device_id pci_endpoint_test_tbl[] = {
 	{ PCI_DEVICE(PCI_VENDOR_ID_RENESAS, PCI_DEVICE_ID_RENESAS_R8A774B1),},
 	{ PCI_DEVICE(PCI_VENDOR_ID_RENESAS, PCI_DEVICE_ID_RENESAS_R8A774C0),},
 	{ PCI_DEVICE(PCI_VENDOR_ID_RENESAS, PCI_DEVICE_ID_RENESAS_R8A774E1),},
+	{ PCI_DEVICE(PCI_VENDOR_ID_RENESAS, PCI_DEVICE_ID_RENESAS_R8A779F0),
+	  .driver_data = (kernel_ulong_t)&default_data,
+	},
 	{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_J721E),
 	  .driver_data = (kernel_ulong_t)&j721e_data,
 	},
-- 
2.25.1

Re: [PATCH v24 00/16] PCI: dwc: rcar-gen4: Add R-Car Gen4 PCIe support

[Krzysztof Wilczyński]

Hello,

> Add R-Car S4-8 (R-Car Gen4) PCIe controller for both host and endpoint modes.
> To support them, modify PCIe DesignWare common codes.

Applied to controller/rcar, thank you!

[01/16] PCI: dwc: endpoint: Add multiple PFs support for dbi2
        https://git.kernel.org/pci/pci/c/cca7bd87028c
[02/16] PCI: dwc: Add dw_pcie_link_set_max_link_width()
        https://git.kernel.org/pci/pci/c/4f3d04e53b9d
[03/16] PCI: dwc: Add missing PCI_EXP_LNKCAP_MLW handling
        https://git.kernel.org/pci/pci/c/0d8ef6f3065e
[04/16] PCI: tegra194: Drop PCI_EXP_LNKSTA_NLW setting
        https://git.kernel.org/pci/pci/c/42a0aaac1df2
[05/16] PCI: dwc: Add EDMA_UNROLL capability flag
        https://git.kernel.org/pci/pci/c/9ece6ac45d1b
[06/16] PCI: dwc: Expose dw_pcie_ep_exit() to module
        https://git.kernel.org/pci/pci/c/3f643b8dd982
[07/16] PCI: dwc: endpoint: Introduce .pre_init() and .deinit()
        https://git.kernel.org/pci/pci/c/b4e028978e1e
[08/16] PCI: dwc: Disable two BARs to avoid unnecessary memory assignment
        https://git.kernel.org/pci/pci/c/e308528cac3e
[09/16] dt-bindings: PCI: dwc: Update maxItems of reg and reg-names
        https://git.kernel.org/pci/pci/c/e7bb27bdb333
[10/16] dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Host
        https://git.kernel.org/pci/pci/c/888d9525eaee
[11/16] dt-bindings: PCI: renesas: Add R-Car Gen4 PCIe Endpoint
        https://git.kernel.org/pci/pci/c/323d6545bc64
[12/16] PCI: Add T_PVPERL macro
        https://git.kernel.org/pci/pci/c/bfd7a42cd940
[13/16] PCI: rcar-gen4: Add R-Car Gen4 PCIe controller support for host mode
        https://git.kernel.org/pci/pci/c/8386896d2eb6
[14/16] PCI: rcar-gen4: Add endpoint mode support
        https://git.kernel.org/pci/pci/c/32b83c68d634
[15/16] MAINTAINERS: Update PCI DRIVER FOR RENESAS R-CAR for R-Car Gen4
        https://git.kernel.org/pci/pci/c/714e7cd07645
[16/16] misc: pci_endpoint_test: Add Device ID for R-Car S4-8 PCIe controller
        https://git.kernel.org/pci/pci/c/de45624e69e1

	Krzysztof