[PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Qu Wenruo]

This series is based on the existing upstream work from Sebastian
Reichel:
https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588

And I'm a completely newbie to arm64 world, thus if there is something
wrong, feel free to point out and I'm pretty happy to learn from the
failure.

[BACKGROUND]
RK3588S and RK3588 have PCIE supports, it's done by the following 3
controllers:

- PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
- PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
  Thes two are all connected to a naneng combo phy each, normally shared
  with SATA or USB.

- PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
  This one is also connected to a naneng combo phy, normally shared
  with SATA or USB.

- PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)

And unlike other boards, ROCK5B is utilizing PCIE extensively, its
network controller (RTL8125 2.5Gbps Ethernet) is connected to the
PCIE2.0 lane at fe190000, and an M.2 slot is attached to the PCIE3.0x4
lanes.

[WORKING]
Currently the series is able to bring up the PCIE3.0x4 lanes and
properly boot from an NVME at that M.2 slot of Rock5B boards.

[NOT WORKING]
All PCIE2.0 lanes connected to naneng combo phy are not working.
I tried forward porting the extra handling from downstream, but it only
results hanging at probing (causing RCU stall).

[EXTRA WANRING]
- PCI MSI initialization warning
  WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 pci_msi_setup_msi_irqs+0x38/0x4c

  This seems to be caused by the fact that we are still using legcacy
  msi irqs?

  I checked up the gic and its dts, can not figure out why (all pretty
  the same just like rk3399 and rk3568).
  Any help would be appreciated.

- pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
  The vendoer kernel also has this problem, but my RK3399 board with
  upstream kernel didn't trigger this at all, but something else like:

    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring

  Then:

    pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01

  Not sure if it's something missing or can be just ignored.

[PCI DMESG]
With this patchset, the PCI initialization and nvme would look like this:

[    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
[    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
[    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
[    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
[    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
[    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
[    0.362236] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
[    0.363113] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
[    0.363744] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
[    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
[    0.365005] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
[    0.365583] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
[    0.366159] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[    0.366801] pci 0000:00:00.0: supports D1 D2
[    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
[    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
[    0.370277] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
[    0.370975] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
[    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
[    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
[    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
[    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
[    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 0xf0200000-0xf02fffff]
[    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref]
[    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 0xf0200000-0xf021ffff pref]
[    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0220000-0xf022ffff 64bit]
[    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    0.377762] pci 0000:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
[    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
[    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
[    0.625353] nvme nvme0: pci function 0000:01:00.0
[    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
[    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
[    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
[    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
[    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
[    0.820678]  nvme0n1: p1 p2

Qu Wenruo (5):
  drivers: phy: rockhip: remove 24M and 25M clock handling for naneng
    combphy
  dt-bindings: pci: controller: add pcie controller binding for RK3588
  drivers: pci: controller: add PCIE controller driver for RK3588
  arm64: dts: rockchip: add PCIE3 controller and phy for RK3588
  arm64: dts: rockchip: enable PCIE3 controller and phy for Rock5B
    boards

 .../bindings/pci/rockchip-dw-pcie.yaml        |   5 +-
 .../boot/dts/rockchip/rk3588-rock-5b.dts      |  22 +++
 arch/arm64/boot/dts/rockchip/rk3588.dtsi      | 128 ++++++++++++++++++
 arch/arm64/boot/dts/rockchip/rk3588s.dtsi     |  16 +++
 drivers/pci/controller/dwc/pcie-dw-rockchip.c |   1 +
 .../rockchip/phy-rockchip-naneng-combphy.c    |  17 ---
 6 files changed, 170 insertions(+), 19 deletions(-)

-- 
2.39.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 1/5] drivers: phy: rockhip: remove 24M and 25M clock handling for naneng combphy

[Qu Wenruo]

Although the combphy supports 24M and 25M clocks, they are not utilized
at any upstream dts (RK3568) nor downstream vendor kernel (RK3568,
RK3588S, RK3588).

Another thing is, with those two clocks removed, it's easier to port the
rk3588 combphy, as 3588 combphy needs to write into cfg->pipe_clk_24m
for 24M clock case.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../phy/rockchip/phy-rockchip-naneng-combphy.c  | 17 -----------------
 1 file changed, 17 deletions(-)

diff --git a/drivers/phy/rockchip/phy-rockchip-naneng-combphy.c b/drivers/phy/rockchip/phy-rockchip-naneng-combphy.c
index 7b213825fb5d..ae7083ae17a2 100644
--- a/drivers/phy/rockchip/phy-rockchip-naneng-combphy.c
+++ b/drivers/phy/rockchip/phy-rockchip-naneng-combphy.c
@@ -94,7 +94,6 @@ struct rockchip_combphy_grfcfg {
 	struct combphy_reg pipe_rxterm_set;
 	struct combphy_reg pipe_txelec_set;
 	struct combphy_reg pipe_txcomp_set;
-	struct combphy_reg pipe_clk_25m;
 	struct combphy_reg pipe_clk_100m;
 	struct combphy_reg pipe_phymode_sel;
 	struct combphy_reg pipe_rate_sel;
@@ -454,21 +453,6 @@ static int rk3568_combphy_cfg(struct rockchip_combphy_priv *priv)
 	rate = clk_get_rate(priv->refclk);
 
 	switch (rate) {
-	case REF_CLOCK_24MHz:
-		if (priv->type == PHY_TYPE_USB3 || priv->type == PHY_TYPE_SATA) {
-			/* Set ssc_cnt[9:0]=0101111101 & 31.5KHz. */
-			val = PHYREG15_SSC_CNT_VALUE << PHYREG15_SSC_CNT_SHIFT;
-			rockchip_combphy_updatel(priv, PHYREG15_SSC_CNT_MASK,
-						 val, PHYREG15);
-
-			writel(PHYREG16_SSC_CNT_VALUE, priv->mmio + PHYREG16);
-		}
-		break;
-
-	case REF_CLOCK_25MHz:
-		rockchip_combphy_param_write(priv->phy_grf, &cfg->pipe_clk_25m, true);
-		break;
-
 	case REF_CLOCK_100MHz:
 		rockchip_combphy_param_write(priv->phy_grf, &cfg->pipe_clk_100m, true);
 		if (priv->type == PHY_TYPE_PCIE) {
@@ -530,7 +514,6 @@ static const struct rockchip_combphy_grfcfg rk3568_combphy_grfcfgs = {
 	.pipe_rxterm_set	= { 0x0000, 12, 12, 0x00, 0x01 },
 	.pipe_txelec_set	= { 0x0004, 1, 1, 0x00, 0x01 },
 	.pipe_txcomp_set	= { 0x0004, 4, 4, 0x00, 0x01 },
-	.pipe_clk_25m		= { 0x0004, 14, 13, 0x00, 0x01 },
 	.pipe_clk_100m		= { 0x0004, 14, 13, 0x00, 0x02 },
 	.pipe_phymode_sel	= { 0x0008, 1, 1, 0x00, 0x01 },
 	.pipe_rate_sel		= { 0x0008, 2, 2, 0x00, 0x01 },
-- 
2.39.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 2/5] dt-bindings: pci: controller: add pcie controller binding for RK3588

[Qu Wenruo]

All the properties are the same with RK3568 PCIE controllers, and in
fact the driver can also be reused between RK3568 and RK3588
PCIE controllers.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 Documentation/devicetree/bindings/pci/rockchip-dw-pcie.yaml | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/pci/rockchip-dw-pcie.yaml b/Documentation/devicetree/bindings/pci/rockchip-dw-pcie.yaml
index 2be72ae1169f..473c90a163b1 100644
--- a/Documentation/devicetree/bindings/pci/rockchip-dw-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/rockchip-dw-pcie.yaml
@@ -21,8 +21,9 @@ allOf:
 
 properties:
   compatible:
-    items:
-      - const: rockchip,rk3568-pcie
+    enum:
+      - rockchip,rk3568-pcie
+      - rockchip,rk3588-pcie
 
   reg:
     items:
-- 
2.39.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 2/5] dt-bindings: pci: controller: add pcie controller binding for RK3588

[Krzysztof Kozlowski]

On 04/02/2023 09:47, Qu Wenruo wrote:
> All the properties are the same with RK3568 PCIE controllers, and in

Subject: drop second/last, redundant "binding". The "dt-bindings" prefix
is already stating that these are bindings.


> fact the driver can also be reused between RK3568 and RK3588
> PCIE controllers.

Please use scripts/get_maintainers.pl to get a list of necessary people
and lists to CC.  It might happen, that command when run on an older
kernel, gives you outdated entries.  Therefore please be sure you base
your patches on recent Linux kernel.

> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---

Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

Best regards,
Krzysztof


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 3/5] drivers: pci: controller: add PCIE controller driver for RK3588

[Qu Wenruo]

According to the downstream code, the RK3568 and RK3588 share the same
driver.

Downstream code only has extra handling for the following cases:

- RK1808 driver
- Bifurcation handling
- Endpoint handling

All of the above features are not available upstream anyway.

Thus here we only need to add a new compatible string for the existing
driver.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 drivers/pci/controller/dwc/pcie-dw-rockchip.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/pci/controller/dwc/pcie-dw-rockchip.c b/drivers/pci/controller/dwc/pcie-dw-rockchip.c
index c1e7653e508e..435b717e5bc6 100644
--- a/drivers/pci/controller/dwc/pcie-dw-rockchip.c
+++ b/drivers/pci/controller/dwc/pcie-dw-rockchip.c
@@ -354,6 +354,7 @@ static int rockchip_pcie_probe(struct platform_device *pdev)
 
 static const struct of_device_id rockchip_pcie_of_match[] = {
 	{ .compatible = "rockchip,rk3568-pcie", },
+	{ .compatible = "rockchip,rk3588-pcie", },
 	{},
 };
 
-- 
2.39.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 4/5] arm64: dts: rockchip: add PCIE3 controller and phy for RK3588

[Qu Wenruo]

RK3588 has one PCIE3 x4 lanes, with one dedicated PCIE phy.

This introduces the following needed nodes:

- its0 and its1 sub-nodes of gic

- pcie30_phy_grf node
  With the existing rk3588-pcie3-phy-grf compatible string.

- pcie3x4 and pcie3x2 nodes
  The main nodes for the pcie controller.
  They share the same phy, thus only one of them can be enabled.

- pcie30phy node
  It's the dedicated PCIE phy.

The differences between upstream and downstream dts are:

- The interrupt cells
  Upstream has two ppi partitions, thus requires #interrupt-cells = <4>
  to specify which partition is preferred.

- The configuration space
  The downstream goes with an invalid range window, and modifies
  DesignWare driver to use that invalid range as configure space.

  Change the cursed method to use the proper "config" reg.

- Reg names
  The downstream always has unnecessary "pcie-" prefix, while upstream
  DesignWare driver never checks such prefix.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 arch/arm64/boot/dts/rockchip/rk3588.dtsi  | 128 ++++++++++++++++++++++
 arch/arm64/boot/dts/rockchip/rk3588s.dtsi |  16 +++
 2 files changed, 144 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk3588.dtsi b/arch/arm64/boot/dts/rockchip/rk3588.dtsi
index d085e57fbc4c..371d7330a498 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588.dtsi
@@ -7,6 +7,121 @@
 #include "rk3588-pinctrl.dtsi"
 
 / {
+	pcie30_phy_grf: syscon@fd5b8000 {
+		compatible = "rockchip,rk3588-pcie3-phy-grf", "syscon";
+		reg = <0x0 0xfd5b8000 0x0 0x10000>;
+	};
+
+	pcie3x4: pcie@fe150000 {
+		compatible = "rockchip,rk3588-pcie";
+		#address-cells = <3>;
+		#size-cells = <2>;
+		bus-range = <0x00 0x0f>;
+		clocks = <&cru ACLK_PCIE_4L_MSTR>, <&cru ACLK_PCIE_4L_SLV>,
+			 <&cru ACLK_PCIE_4L_DBI>, <&cru PCLK_PCIE_4L>,
+			 <&cru CLK_PCIE_AUX0>, <&cru CLK_PCIE4L_PIPE>;
+		clock-names = "aclk_mst", "aclk_slv",
+			      "aclk_dbi", "pclk",
+			      "aux", "pipe";
+		device_type = "pci";
+		interrupts = <GIC_SPI 263 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 262 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 261 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 260 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 259 IRQ_TYPE_LEVEL_HIGH 0>;
+		interrupt-names = "sys", "pmc", "msg", "legacy", "err";
+		#interrupt-cells = <1>;
+		interrupt-map-mask = <0 0 0 7>;
+		interrupt-map = <0 0 0 1 &pcie3x4_intc 0>,
+				<0 0 0 2 &pcie3x4_intc 1>,
+				<0 0 0 3 &pcie3x4_intc 2>,
+				<0 0 0 4 &pcie3x4_intc 3>;
+		linux,pci-domain = <0>;
+		num-ib-windows = <16>;
+		num-ob-windows = <16>;
+		num-viewport = <8>;
+		max-link-speed = <3>;
+		msi-map = <0x0000 &its1 0x0000 0x1000>;
+		num-lanes = <4>;
+		phys = <&pcie30phy>;
+		phy-names = "pcie-phy";
+		power-domains = <&power RK3588_PD_PCIE>;
+		ranges = <0x81000000 0x0 0xf0100000 0x0 0xf0100000 0x0 0x100000
+			  0x82000000 0x0 0xf0200000 0x0 0xf0200000 0x0 0xe00000
+			  0xc3000000 0x9 0x00000000 0x9 0x00000000 0x0 0x40000000>;
+		reg = <0x0 0xfe150000 0x0 0x10000>,
+		      <0xa 0x40000000 0x0 0x400000>,
+		      <0x0 0xf0000000 0x0 0x100000>;
+		reg-names = "apb", "dbi", "config";
+		resets = <&cru SRST_PCIE0_POWER_UP>, <&cru SRST_P_PCIE0>;
+		reset-names = "pcie", "periph";
+		rockchip,pipe-grf = <&php_grf>;
+		status = "disabled";
+
+		pcie3x4_intc: legacy-interrupt-controller {
+			interrupt-controller;
+			#address-cells = <0>;
+			#interrupt-cells = <1>;
+			interrupt-parent = <&gic>;
+			interrupts = <GIC_SPI 260 IRQ_TYPE_EDGE_RISING 0>;
+		};
+	};
+
+	pcie3x2: pcie@fe160000 {
+		compatible = "rockchip,rk3588-pcie", "snps,dw-pcie";
+		#address-cells = <3>;
+		#size-cells = <2>;
+		bus-range = <0x10 0x1f>;
+		clocks = <&cru ACLK_PCIE_2L_MSTR>, <&cru ACLK_PCIE_2L_SLV>,
+			 <&cru ACLK_PCIE_2L_DBI>, <&cru PCLK_PCIE_2L>,
+			 <&cru CLK_PCIE_AUX1>, <&cru CLK_PCIE2L_PIPE>;
+		clock-names = "aclk_mst", "aclk_slv",
+			      "aclk_dbi", "pclk",
+			      "aux", "pipe";
+		device_type = "pci";
+		interrupts = <GIC_SPI 258 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 257 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 256 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 255 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 254 IRQ_TYPE_LEVEL_HIGH 0>;
+		interrupt-names = "sys", "pmc", "msg", "legacy", "err";
+		#interrupt-cells = <1>;
+		interrupt-map-mask = <0 0 0 7>;
+		interrupt-map = <0 0 0 1 &pcie3x2_intc 0>,
+				<0 0 0 2 &pcie3x2_intc 1>,
+				<0 0 0 3 &pcie3x2_intc 2>,
+				<0 0 0 4 &pcie3x2_intc 3>;
+		linux,pci-domain = <1>;
+		num-ib-windows = <16>;
+		num-ob-windows = <16>;
+		num-viewport = <8>;
+		max-link-speed = <3>;
+		msi-map = <0x1000 &its1 0x1000 0x1000>;
+		num-lanes = <2>;
+		phys = <&pcie30phy>;
+		phy-names = "pcie-phy";
+		power-domains = <&power RK3588_PD_PCIE>;
+		ranges = <0x81000000 0x0 0xf1100000 0x0 0xf1100000 0x0 0x100000
+			  0x82000000 0x0 0xf1200000 0x0 0xf1200000 0x0 0xe00000
+			  0xc3000000 0x9 0x40000000 0x9 0x40000000 0x0 0x40000000>;
+		reg = <0x0 0xfe160000 0x0 0x10000>,
+		      <0xa 0x40400000 0x0 0x400000>,
+		      <0x0 0xf1000000 0x0 0x100000>;
+		reg-names = "apb", "dbi", "config";
+		resets = <&cru SRST_PCIE1_POWER_UP>, <&cru SRST_P_PCIE1>;
+		reset-names = "pcie", "periph";
+		rockchip,pipe-grf = <&php_grf>;
+		status = "disabled";
+
+		pcie3x2_intc: legacy-interrupt-controller {
+			interrupt-controller;
+			#address-cells = <0>;
+			#interrupt-cells = <1>;
+			interrupt-parent = <&gic>;
+			interrupts = <GIC_SPI 255 IRQ_TYPE_EDGE_RISING 0>;
+		};
+	};
+
 	gmac0: ethernet@fe1b0000 {
 		compatible = "rockchip,rk3588-gmac", "snps,dwmac-4.20a";
 		reg = <0x0 0xfe1b0000 0x0 0x10000>;
@@ -55,4 +170,17 @@ gmac0_mtl_tx_setup: tx-queues-config {
 			queue1 {};
 		};
 	};
+
+	pcie30phy: phy@fee80000 {
+		compatible = "rockchip,rk3588-pcie3-phy";
+		reg = <0x0 0xfee80000 0x0 0x20000>;
+		#phy-cells = <0>;
+		clocks = <&cru PCLK_PCIE_COMBO_PIPE_PHY>;
+		clock-names = "pclk";
+		resets = <&cru SRST_PCIE30_PHY>;
+		reset-names = "phy";
+		rockchip,pipe-grf = <&php_grf>;
+		rockchip,phy-grf = <&pcie30_phy_grf>;
+		status = "disabled";
+	};
 };
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index 24919cb5c153..3af31e12d04c 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -1594,6 +1594,22 @@ gic: interrupt-controller@fe600000 {
 		mbi-ranges = <424 56>;
 		msi-controller;
 		#interrupt-cells = <4>;
+		#address-cells = <2>;
+		#size-cells = <2>;
+
+		its0: interrupt-controller@fe640000 {
+			compatible = "arm,gic-v3-its";
+			msi-controller;
+			#msi-cells = <1>;
+			reg = <0x0 0xfe640000 0x0 0x20000>;
+		};
+
+		its1: interrupt-controller@fe660000 {
+			compatible = "arm,gic-v3-its";
+			msi-controller;
+			#msi-cells = <1>;
+			reg = <0x0 0xfe660000 0x0 0x20000>;
+		};
 
 		ppi-partitions {
 			ppi_partition0: interrupt-partition-0 {
-- 
2.39.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 5/5] arm64: dts: rockchip: enable PCIE3 controller and phy for Rock5B boards

[Qu Wenruo]

We only needs to define the needed 3v3 voltage for the PCIE controller,
and enable both the controller and phy.

Since I only own Rock5B boards for RK3588, I can not test for sure for
other boards, thus this patch only enables the PCIE controller/phy for
Rock5B.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../boot/dts/rockchip/rk3588-rock-5b.dts      | 22 +++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts b/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts
index d2f1e963ce06..75015f065ceb 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts
@@ -2,6 +2,7 @@
 
 /dts-v1/;
 
+#include <dt-bindings/gpio/gpio.h>
 #include "rk3588.dtsi"
 
 / {
@@ -25,6 +26,17 @@ vcc5v0_sys: vcc5v0-sys-regulator {
 		regulator-min-microvolt = <5000000>;
 		regulator-max-microvolt = <5000000>;
 	};
+
+	vcc3v3_pcie30: vcc3v3-pcie30 {
+		compatible = "regulator-fixed";
+		regulator-name = "vcc3v3_pcie30";
+		regulator-min-microvolt = <3300000>;
+		regulator-max-microvolt = <3300000>;
+		enable-active-high;
+		gpios = <&gpio1 RK_PA4 GPIO_ACTIVE_HIGH>;
+		startup-delay-us = <5000>;
+		vin-supply = <&vcc5v0_sys>;
+	};
 };
 
 &sdhci {
@@ -38,6 +50,16 @@ &sdhci {
 	status = "okay";
 };
 
+&pcie30phy {
+	status = "okay";
+};
+
+&pcie3x4 {
+	reset-gpios = <&gpio4 RK_PB6 GPIO_ACTIVE_HIGH>;
+	vpcie3v3-supply = <&vcc3v3_pcie30>;
+	status = "okay";
+};
+
 &uart2 {
 	pinctrl-0 = <&uart2m0_xfer>;
 	status = "okay";
-- 
2.39.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Piotr Oniszczuk]

> Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 04.02.2023, o godz. 09:47:
> 
> This series is based on the existing upstream work from Sebastian
> Reichel:
> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588
> 
> And I'm a completely newbie to arm64 world, thus if there is something
> wrong, feel free to point out and I'm pretty happy to learn from the
> failure.
> 
> [BACKGROUND]
> RK3588S and RK3588 have PCIE supports, it's done by the following 3
> controllers:
> 
> - PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
> - PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
>  Thes two are all connected to a naneng combo phy each, normally shared
>  with SATA or USB.
> 
> - PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
>  This one is also connected to a naneng combo phy, normally shared
>  with SATA or USB.
> 
> - PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)
> 
> And unlike other boards, ROCK5B is utilizing PCIE extensively, its
> network controller (RTL8125 2.5Gbps Ethernet) is connected to the
> PCIE2.0 lane at fe190000, and an M.2 slot is attached to the PCIE3.0x4
> lanes.
> 
> [WORKING]
> Currently the series is able to bring up the PCIE3.0x4 lanes and
> properly boot from an NVME at that M.2 slot of Rock5B boards.
> 
> [NOT WORKING]
> All PCIE2.0 lanes connected to naneng combo phy are not working.
> I tried forward porting the extra handling from downstream, but it only
> results hanging at probing (causing RCU stall).
> 
> [EXTRA WANRING]
> - PCI MSI initialization warning
>  WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 pci_msi_setup_msi_irqs+0x38/0x4c
> 
>  This seems to be caused by the fact that we are still using legcacy
>  msi irqs?
> 
>  I checked up the gic and its dts, can not figure out why (all pretty
>  the same just like rk3399 and rk3568).
>  Any help would be appreciated.
> 
> - pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
>  The vendoer kernel also has this problem, but my RK3399 board with
>  upstream kernel didn't trigger this at all, but something else like:
> 
>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> 
>  Then:
> 
>    pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01
> 
>  Not sure if it's something missing or can be just ignored.
> 
> [PCI DMESG]
> With this patchset, the PCI initialization and nvme would look like this:
> 
> [    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
> [    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
> [    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> [    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
> [    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
> [    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
> [    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
> [    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
> [    0.362236] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
> [    0.363113] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
> [    0.363744] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
> [    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> [    0.365005] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> [    0.365583] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> [    0.366159] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> [    0.366801] pci 0000:00:00.0: supports D1 D2
> [    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> [    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> [    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
> [    0.370277] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
> [    0.370975] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> [    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> [    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
> [    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> [    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
> [    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 0xf0200000-0xf02fffff]
> [    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref]
> [    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 0xf0200000-0xf021ffff pref]
> [    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0220000-0xf022ffff 64bit]
> [    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> [    0.377762] pci 0000:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
> [    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
> [    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
> [    0.625353] nvme nvme0: pci function 0000:01:00.0
> [    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
> [    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
> [    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
> [    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
> [    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
> [    0.820678]  nvme0n1: p1 p2
> 
> 

(resend as plain TXT. Sorry for previous RTF!)

Qu, all

I’m playing with your work on my rock5b as I want to have working Eth on rock5b.

My code is from https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 + your’s PCIE3 patches. 

SBC boots from sd card, I see PCIE related logs in dmesg. but no rtl8125 is detected.
PCIE logs are like this:

    8.207810] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
[    8.208501] rockchip-dw-pcie fe150000.pcie: Parsing ranges property...
[    8.209089] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[    8.209944] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
[    8.210740] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
[    8.218918] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    9.324473] rockchip-dw-pcie fe150000.pcie: Phy link never came up
[    9.325186] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
[    9.325786] pci_bus 0000:00: root bus resource [bus 00-0f]
[    9.326266] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
[    9.327097] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
[    9.327713] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
[    9.328364] pci_bus 0000:00: scanning bus
[    9.328729] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
[    9.329258] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
[    9.329807] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
[    9.330354] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[    9.330984] pci 0000:00:00.0: supports D1 D2
[    9.331356] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    9.331858] pci 0000:00:00.0: PME# disabled
[    9.332309] pci 0000:00:00.0: vgaarb: pci_notify
[    9.333735] pci_bus 0000:00: fixups for bus
[    9.334106] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
[    9.334731] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
[    9.335668] pci_bus 0000:01: scanning bus
[    9.336052] pci_bus 0000:01: fixups for bus
[    9.336416] pci_bus 0000:01: bus scan returning with max=01
[    9.336903] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
[    9.337503] pci_bus 0000:00: bus scan returning with max=ff
[    9.337994] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
[    9.338570] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
[    9.339175] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
[    9.339749] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
[    9.340356] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0200000-0xf020ffff pref]
[    9.340991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]

and nothing more :-(

Are you progressing maybe with pcie on rock5b?


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Qu Wenruo]

On 2023/2/21 02:25, Piotr Oniszczuk wrote:
> 
> 
>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com 
>> <mailto:wqu@suse.com>> w dniu 04.02.2023, o godz. 09:47:
>>
>> This series is based on the existing upstream work from Sebastian
>> Reichel:
>> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588>
>>
>> And I'm a completely newbie to arm64 world, thus if there is something
>> wrong, feel free to point out and I'm pretty happy to learn from the
>> failure.
>>
>> [BACKGROUND]
>> RK3588S and RK3588 have PCIE supports, it's done by the following 3
>> controllers:
>>
>> - PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
>> - PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
>>  Thes two are all connected to a naneng combo phy each, normally shared
>>  with SATA or USB.
>>
>> - PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
>>  This one is also connected to a naneng combo phy, normally shared
>>  with SATA or USB.
>>
>> - PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)
>>
>> And unlike other boards, ROCK5B is utilizing PCIE extensively, its
>> network controller (RTL8125 2.5Gbps Ethernet) is connected to the
>> PCIE2.0 lane at fe190000, and an M.2 slot is attached to the PCIE3.0x4
>> lanes.
>>
>> [WORKING]
>> Currently the series is able to bring up the PCIE3.0x4 lanes and
>> properly boot from an NVME at that M.2 slot of Rock5B boards.
>>
>> [NOT WORKING]
>> All PCIE2.0 lanes connected to naneng combo phy are not working.
>> I tried forward porting the extra handling from downstream, but it only
>> results hanging at probing (causing RCU stall).
>>
>> [EXTRA WANRING]
>> - PCI MSI initialization warning
>>  WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 
>> pci_msi_setup_msi_irqs+0x38/0x4c
>>
>>  This seems to be caused by the fact that we are still using legcacy
>>  msi irqs?
>>
>>  I checked up the gic and its dts, can not figure out why (all pretty
>>  the same just like rk3399 and rk3568).
>>  Any help would be appreciated.
>>
>> - pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 
>> 00-0f] (conflicts with (null) [bus 00-0f])
>>  The vendoer kernel also has this problem, but my RK3399 board with
>>  upstream kernel didn't trigger this at all, but something else like:
>>
>>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), 
>> reconfiguring
>>
>>  Then:
>>
>>    pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01
>>
>>  Not sure if it's something missing or can be just ignored.
>>
>> [PCI DMESG]
>> With this patchset, the PCI initialization and nvme would look like this:
>>
>> [    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge 
>> /pcie@fe150000 ranges:
>> [    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 
>> 0x00f0100000..0x00f01fffff -> 0x00f0100000
>> [    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 
>> 0x00f0200000..0x00f0ffffff -> 0x00f0200000
>> [    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 
>> 0x0900000000..0x093fffffff -> 0x0900000000
>> [    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 
>> ib, align 64K, limit 8G
>> [    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
>> [    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 
>> 0000:00
>> [    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
>> [    0.362236] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] 
>> (bus address [0xf0100000-0xf01fffff])
>> [    0.363113] pci_bus 0000:00: root bus resource [mem 
>> 0xf0200000-0xf0ffffff]
>> [    0.363744] pci_bus 0000:00: root bus resource [mem 
>> 0x900000000-0x93fffffff pref]
>> [    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
>> [    0.365005] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
>> [    0.365583] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
>> [    0.366159] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff 
>> pref]
>> [    0.366801] pci 0000:00:00.0: supports D1 D2
>> [    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
>> [    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] 
>> under [bus 00-0f] (conflicts with (null) [bus 00-0f])
>> [    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
>> [    0.370277] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 
>> 64bit]
>> [    0.370975] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff 
>> pref]
>> [    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
>> [    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem size 
>> 0x40000000]
>> [    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
>> [    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem size 
>> 0x40000000]
>> [    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 
>> 0xf0200000-0xf02fffff]
>> [    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 
>> 0xf0300000-0xf030ffff pref]
>> [    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 
>> 0xf0200000-0xf021ffff pref]
>> [    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 
>> 0xf0220000-0xf022ffff 64bit]
>> [    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>> [    0.377762] pci 0000:00:00.0:   bridge window [mem 
>> 0xf0200000-0xf02fffff]
>> [    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
>> [    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
>> [    0.625353] nvme nvme0: pci function 0000:01:00.0
>> [    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
>> [    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
>> [    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
>> [    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
>> [    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
>> [    0.820678]  nvme0n1: p1 p2
>>
>>
> 
> Qu, all
> 
> I’m playing with your work on my rock5b as I want to have working Eth on 
> rock5b.
> 
> My code is from 
> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588> + your’s PCIE3 patches.
> 
> SBC boots from sd card, I see PCIE related logs in dmesg. but no rtl8125 
> is detected.
> PCIE logs are like this:
> 
>      8.207810] rockchip-dw-pcie fe150000.pcie: host bridge 
> /pcie@fe150000 ranges:
> [    8.208501] rockchip-dw-pcie fe150000.pcie: Parsing ranges property...
> [    8.209089] rockchip-dw-pcie fe150000.pcie:       IO 
> 0x00f0100000..0x00f01fffff -> 0x00f0100000
> [    8.209944] rockchip-dw-pcie fe150000.pcie:      MEM 
> 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> [    8.210740] rockchip-dw-pcie fe150000.pcie:      MEM 
> 0x0900000000..0x093fffffff -> 0x0900000000
> [    8.218918] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 
> ib, align 64K, limit 8G
> [    9.324473] rockchip-dw-pcie fe150000.pcie: Phy link never came up
> [    9.325186] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 
> 0000:00
> [    9.325786] pci_bus 0000:00: root bus resource [bus 00-0f]
> [    9.326266] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] 
> (bus address [0xf0100000-0xf01fffff])
> [    9.327097] pci_bus 0000:00: root bus resource [mem 
> 0xf0200000-0xf0ffffff]
> [    9.327713] pci_bus 0000:00: root bus resource [mem 
> 0x900000000-0x93fffffff pref]
> [    9.328364] pci_bus 0000:00: scanning bus
> [    9.328729] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> [    9.329258] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> [    9.329807] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> [    9.330354] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> [    9.330984] pci 0000:00:00.0: supports D1 D2
> [    9.331356] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> [    9.331858] pci 0000:00:00.0: PME# disabled
> [    9.332309] pci 0000:00:00.0: vgaarb: pci_notify
> [    9.333735] pci_bus 0000:00: fixups for bus
> [    9.334106] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
> [    9.334731] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] 
> under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> [    9.335668] pci_bus 0000:01: scanning bus
> [    9.336052] pci_bus 0000:01: fixups for bus
> [    9.336416] pci_bus 0000:01: bus scan returning with max=01
> [    9.336903] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
> [    9.337503] pci_bus 0000:00: bus scan returning with max=ff
> [    9.337994] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> [    9.338570] pci 0000:00:00.0: BAR 0: failed to assign [mem size 
> 0x40000000]
> [    9.339175] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> [    9.339749] pci 0000:00:00.0: BAR 1: failed to assign [mem size 
> 0x40000000]
> [    9.340356] pci 0000:00:00.0: BAR 6: assigned [mem 
> 0xf0200000-0xf020ffff pref]
> [    9.340991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> 
> and nothing more :-(
> 
> Are you progressing maybe with pcie on rock5b?
> 
> 
fe150000 is the PCIE3.0 controller, on Rock5B, that's the M.2 slot.

But for R8125, it's on a PCIE2.0 controller, which is using naneng combo 
phy.

I'm not able to bring the PCIE2.0 part up yet, it always hangs at 
PCIE2.0 initialization, thus only the PCIE3.0 part is submitted to the list.

Thanks,
Qu

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Piotr Oniszczuk]

> Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 21.02.2023, o godz. 01:14:
> 
> 
> 
> On 2023/2/21 02:25, Piotr Oniszczuk wrote:
>>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com <mailto:wqu@suse.com>> w dniu 04.02.2023, o godz. 09:47:
>>> 
>>> This series is based on the existing upstream work from Sebastian
>>> Reichel:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588>
>>> 
>>> And I'm a completely newbie to arm64 world, thus if there is something
>>> wrong, feel free to point out and I'm pretty happy to learn from the
>>> failure.
>>> 
>>> [BACKGROUND]
>>> RK3588S and RK3588 have PCIE supports, it's done by the following 3
>>> controllers:
>>> 
>>> - PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
>>> - PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
>>>  Thes two are all connected to a naneng combo phy each, normally shared
>>>  with SATA or USB.
>>> 
>>> - PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
>>>  This one is also connected to a naneng combo phy, normally shared
>>>  with SATA or USB.
>>> 
>>> - PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)
>>> 
>>> And unlike other boards, ROCK5B is utilizing PCIE extensively, its
>>> network controller (RTL8125 2.5Gbps Ethernet) is connected to the
>>> PCIE2.0 lane at fe190000, and an M.2 slot is attached to the PCIE3.0x4
>>> lanes.
>>> 
>>> [WORKING]
>>> Currently the series is able to bring up the PCIE3.0x4 lanes and
>>> properly boot from an NVME at that M.2 slot of Rock5B boards.
>>> 
>>> [NOT WORKING]
>>> All PCIE2.0 lanes connected to naneng combo phy are not working.
>>> I tried forward porting the extra handling from downstream, but it only
>>> results hanging at probing (causing RCU stall).
>>> 
>>> [EXTRA WANRING]
>>> - PCI MSI initialization warning
>>>  WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 pci_msi_setup_msi_irqs+0x38/0x4c
>>> 
>>>  This seems to be caused by the fact that we are still using legcacy
>>>  msi irqs?
>>> 
>>>  I checked up the gic and its dts, can not figure out why (all pretty
>>>  the same just like rk3399 and rk3568).
>>>  Any help would be appreciated.
>>> 
>>> - pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
>>>  The vendoer kernel also has this problem, but my RK3399 board with
>>>  upstream kernel didn't trigger this at all, but something else like:
>>> 
>>>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>> 
>>>  Then:
>>> 
>>>    pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01
>>> 
>>>  Not sure if it's something missing or can be just ignored.
>>> 
>>> [PCI DMESG]
>>> With this patchset, the PCI initialization and nvme would look like this:
>>> 
>>> [    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
>>> [    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
>>> [    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
>>> [    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
>>> [    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
>>> [    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
>>> [    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
>>> [    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
>>> [    0.362236] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
>>> [    0.363113] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
>>> [    0.363744] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
>>> [    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
>>> [    0.365005] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
>>> [    0.365583] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
>>> [    0.366159] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
>>> [    0.366801] pci 0000:00:00.0: supports D1 D2
>>> [    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
>>> [    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
>>> [    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
>>> [    0.370277] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
>>> [    0.370975] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
>>> [    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
>>> [    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
>>> [    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
>>> [    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
>>> [    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 0xf0200000-0xf02fffff]
>>> [    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref]
>>> [    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 0xf0200000-0xf021ffff pref]
>>> [    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0220000-0xf022ffff 64bit]
>>> [    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>>> [    0.377762] pci 0000:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
>>> [    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
>>> [    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
>>> [    0.625353] nvme nvme0: pci function 0000:01:00.0
>>> [    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
>>> [    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
>>> [    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
>>> [    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
>>> [    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
>>> [    0.820678]  nvme0n1: p1 p2
>>> 
>>> 
>> Qu, all
>> I’m playing with your work on my rock5b as I want to have working Eth on rock5b.
>> My code is from https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588> + your’s PCIE3 patches.
>> SBC boots from sd card, I see PCIE related logs in dmesg. but no rtl8125 is detected.
>> PCIE logs are like this:
>>     8.207810] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
>> [    8.208501] rockchip-dw-pcie fe150000.pcie: Parsing ranges property...
>> [    8.209089] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
>> [    8.209944] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
>> [    8.210740] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
>> [    8.218918] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
>> [    9.324473] rockchip-dw-pcie fe150000.pcie: Phy link never came up
>> [    9.325186] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
>> [    9.325786] pci_bus 0000:00: root bus resource [bus 00-0f]
>> [    9.326266] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
>> [    9.327097] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
>> [    9.327713] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
>> [    9.328364] pci_bus 0000:00: scanning bus
>> [    9.328729] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
>> [    9.329258] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
>> [    9.329807] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
>> [    9.330354] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
>> [    9.330984] pci 0000:00:00.0: supports D1 D2
>> [    9.331356] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
>> [    9.331858] pci 0000:00:00.0: PME# disabled
>> [    9.332309] pci 0000:00:00.0: vgaarb: pci_notify
>> [    9.333735] pci_bus 0000:00: fixups for bus
>> [    9.334106] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
>> [    9.334731] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
>> [    9.335668] pci_bus 0000:01: scanning bus
>> [    9.336052] pci_bus 0000:01: fixups for bus
>> [    9.336416] pci_bus 0000:01: bus scan returning with max=01
>> [    9.336903] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
>> [    9.337503] pci_bus 0000:00: bus scan returning with max=ff
>> [    9.337994] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
>> [    9.338570] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
>> [    9.339175] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
>> [    9.339749] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
>> [    9.340356] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0200000-0xf020ffff pref]
>> [    9.340991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>> and nothing more :-(
>> Are you progressing maybe with pcie on rock5b?
> fe150000 is the PCIE3.0 controller, on Rock5B, that's the M.2 slot.
> 
> But for R8125, it's on a PCIE2.0 controller, which is using naneng combo phy.
> 
> I'm not able to bring the PCIE2.0 part up yet, it always hangs at PCIE2.0 initialization, thus only the PCIE3.0 part is submitted to the list.
> 
> Thanks,
> Qu

Yes. Indeed.
I'm trying to add pci2.0 and it looks i meet the same problem probably.

I backported (from well working neggles quartz64pro repo I assume):
https://github.com/neggles/linux-quartz64/commit/2ad84e89fc75b82c783345b72c97f5d7e3d45723
https://github.com/neggles/linux-quartz64/commit/4ac44c2b53758eca671d695d19b456d1849d7e14
https://github.com/neggles/linux-quartz64/commit/714c5e01d8f0f73b3a49cbdee29e3ffe0f3353dd
https://github.com/neggles/linux-quartz64/commit/64971f9c85f27e29c44b31a0c326ea4bb8ec3c56

then i added rock5b pcie dt enablements (basing on radxa BSP):
https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.2/files/1058-arm64-dts-rockchip-enable-pcie-rock5b.patch


this gives me quite clean 6.2 mainline boot log with hang at pcie2 init (pls see below)

I assume https://github.com/neggles/linux-quartz64/commits/linux-quartz64 repo works well on Quartsz64 board - so I indirectly assume above backports should give us working pice2.0.

It fails on rock5b - so I conclude: issue/error is on my side and is related to my rock5b specifics.
Unfortunately I don’t owe Quartz64 board so can't verify correctness of my backports by testing on Quartz64pro.

I’m curious about opinion of smarter than me guys…
  



my kernel log:

Starting kernel ...

[    0.000000] efi: UEFI not found.
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000200000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000001ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000200000-0x00000000efffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x00000001ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000200000-0x00000001ffffffff]
[    0.000000] On node 0, zone DMA: 512 pages in unavailable ranges
[    0.000000] cma: Reserved 64 MiB at 0x00000000ec000000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 29 pages/cpu s80872 r8192 d29720 u118784
[    0.000000] pcpu-alloc: s80872 r8192 d29720 u118784 alloc=29*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7 
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: detected: Qualcomm erratum 1009, or ARM erratum 1286807, 2441009
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] alternatives: applying boot alternatives
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1999368
[    0.000000] Kernel command line: root=/dev/mmcblk1p2 rw rootwait earlycon=uart8250,mmio32,0xfeb50000 console=ttyS2,1500000n8 debug MM_DEBUG="yes"
[    0.000000] Unknown kernel command line parameters "MM_DEBUG=yes", will be passed to user space.
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: area num 8.
[    0.000000] software IO TLB: mapped [mem 0x00000000e8000000-0x00000000ec000000] (64MB)
[    0.000000] Memory: 7803944K/8124416K available (14592K kernel code, 3524K rwdata, 6420K rodata, 6720K init, 576K bss, 254936K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-1, MinObjects=0, CPUs=8, Nodes=1
[    0.000000] ftrace: allocating 51240 entries in 201 pages
[    0.000000] ftrace: allocated 201 pages with 4 groups
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] .Trampoline variant of Tasks RCU enabled.
[    0.000000] .Rude variant of Tasks RCU enabled.
[    0.000000] .Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 30 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 480 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: GICv3 features: 16 PPIs
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fe680000
[    0.000000] ITS [mem 0xfe640000-0xfe65ffff]
[    0.000000] GIC: enabling workaround for ITS: Rockchip RK3568 force no_local_cache
[    0.000000] ITS@0x00000000fe640000: allocated 8192 Devices @210000 (indirect, esz 8, psz 64K, shr 0)
[    0.000000] ITS@0x00000000fe640000: allocated 32768 Interrupt Collections @220000 (flat, esz 2, psz 64K, shr 0)
[    0.000000] ITS: using cache flushing for cmd queue
[    0.000000] ITS [mem 0xfe660000-0xfe67ffff]
[    0.000000] GIC: enabling workaround for ITS: Rockchip RK3568 force no_local_cache
[    0.000000] ITS@0x00000000fe660000: allocated 8192 Devices @240000 (indirect, esz 8, psz 64K, shr 0)
[    0.000000] ITS@0x00000000fe660000: allocated 32768 Interrupt Collections @250000 (flat, esz 2, psz 64K, shr 0)
[    0.000000] ITS: using cache flushing for cmd queue
[    0.000000] ITS ALLOCATE PROP WORKAROUND
[    0.000000] GICv3: using LPI property table @0x0000000000260000
[    0.000000] GIC: using cache flushing for LPI property table
[    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000000270000
[    0.000000] GICv3: GIC: PPI partition interrupt-partition-0[0] { /cpus/cpu@0[0] /cpus/cpu@100[1] /cpus/cpu@200[2] /cpus/cpu@300[3] }
[    0.000000] GICv3: GIC: PPI partition interrupt-partition-1[1] { /cpus/cpu@400[4] /cpus/cpu@500[5] /cpus/cpu@600[6] /cpus/cpu@700[7] }
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns
[    0.000000] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns
[    0.001642] Console: colour dummy device 80x25
[    0.002112] Calibrating delay loop (skipped), value calculated using timer frequency.. 48.00 BogoMIPS (lpj=80000)
[    0.003044] pid_max: default: 32768 minimum: 301
[    0.003584] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.004286] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.006812] cblist_init_generic: Setting adjustable number of callback queues.
[    0.007472] cblist_init_generic: Setting shift to 3 and lim to 1.
[    0.008124] cblist_init_generic: Setting shift to 3 and lim to 1.
[    0.008787] cblist_init_generic: Setting shift to 3 and lim to 1.
[    0.009551] rcu: Hierarchical SRCU implementation.
[    0.009987] rcu: .Max phase no-delay instances is 1000.
[    0.011795] Platform MSI: msi-controller@fe640000 domain created
[    0.012382] Platform MSI: msi-controller@fe660000 domain created
[    0.013066] PCI/MSI: /interrupt-controller@fe600000/msi-controller@fe640000 domain created
[    0.013841] PCI/MSI: /interrupt-controller@fe600000/msi-controller@fe660000 domain created
[    0.015804] EFI services will not be available.
[    0.016811] smp: Bringing up secondary CPUs ...
[    0.017822] Detected VIPT I-cache on CPU1
[    0.017887] GICv3: CPU1: found redistributor 100 region 0:0x00000000fe6a0000
[    0.017899] GICv3: CPU1: using allocated LPI pending table @0x0000000000280000
[    0.017937] CPU1: Booted secondary processor 0x0000000100 [0x412fd050]
[    0.018627] Detected VIPT I-cache on CPU2
[    0.018687] GICv3: CPU2: found redistributor 200 region 0:0x00000000fe6c0000
[    0.018698] GICv3: CPU2: using allocated LPI pending table @0x0000000000290000
[    0.018733] CPU2: Booted secondary processor 0x0000000200 [0x412fd050]
[    0.019427] Detected VIPT I-cache on CPU3
[    0.019486] GICv3: CPU3: found redistributor 300 region 0:0x00000000fe6e0000
[    0.019497] GICv3: CPU3: using allocated LPI pending table @0x00000000002a0000
[    0.019529] CPU3: Booted secondary processor 0x0000000300 [0x412fd050]
[    0.020182] CPU features: detected: Spectre-v4
[    0.020186] CPU features: detected: Spectre-BHB
[    0.020189] Detected PIPT I-cache on CPU4
[    0.020223] GICv3: CPU4: found redistributor 400 region 0:0x00000000fe700000
[    0.020231] GICv3: CPU4: using allocated LPI pending table @0x00000000002b0000
[    0.020251] CPU4: Booted secondary processor 0x0000000400 [0x414fd0b0]
[    0.020868] Detected PIPT I-cache on CPU5
[    0.020907] GICv3: CPU5: found redistributor 500 region 0:0x00000000fe720000
[    0.020914] GICv3: CPU5: using allocated LPI pending table @0x00000000002c0000
[    0.020935] CPU5: Booted secondary processor 0x0000000500 [0x414fd0b0]
[    0.021554] Detected PIPT I-cache on CPU6
[    0.021592] GICv3: CPU6: found redistributor 600 region 0:0x00000000fe740000
[    0.021599] GICv3: CPU6: using allocated LPI pending table @0x00000000002d0000
[    0.021619] CPU6: Booted secondary processor 0x0000000600 [0x414fd0b0]
[    0.022222] Detected PIPT I-cache on CPU7
[    0.022262] GICv3: CPU7: found redistributor 700 region 0:0x00000000fe760000
[    0.022269] GICv3: CPU7: using allocated LPI pending table @0x00000000002e0000
[    0.022290] CPU7: Booted secondary processor 0x0000000700 [0x414fd0b0]
[    0.022353] smp: Brought up 1 node, 8 CPUs
[    0.039159] SMP: Total of 8 processors activated.
[    0.039588] CPU features: detected: 32-bit EL0 Support
[    0.040053] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.040827] CPU features: detected: Common not Private translations
[    0.041404] CPU features: detected: CRC32 instructions
[    0.041869] CPU features: detected: RCpc load-acquire (LDAPR)
[    0.042386] CPU features: detected: LSE atomic instructions
[    0.042886] CPU features: detected: Privileged Access Never
[    0.043389] CPU features: detected: RAS Extension Support
[    0.043880] CPU features: detected: Speculative Store Bypassing Safe (SSBS)
[    0.044543] spectre-bhb mitigation disabled by compile time option
[    0.045156] CPU: All CPU(s) started at EL2
[    0.045530] alternatives: applying system-wide alternatives
[    0.049962] devtmpfs: initialized
[    0.056059] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 6370867519511994 ns
[    0.056955] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.058021] pinctrl core: initialized pinctrl subsystem
[    0.058735] DMI not present or invalid.
[    0.059290] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.060548] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[    0.061386] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.062270] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.063020] audit: initializing netlink subsys (disabled)
[    0.063616] audit: type=2000 audit(0.059:1): state=initialized audit_enabled=0 res=1
[    0.064188] thermal_sys: Registered thermal governor 'fair_share'
[    0.064321] thermal_sys: Registered thermal governor 'bang_bang'
[    0.064879] thermal_sys: Registered thermal governor 'step_wise'
[    0.065429] thermal_sys: Registered thermal governor 'user_space'
[    0.066001] cpuidle: using governor ladder
[    0.066937] cpuidle: using governor menu
[    0.067335] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.068059] ASID allocator initialised with 65536 entries
[    0.068921] Serial: AMBA PL011 UART driver
[    0.078586] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    0.079547] rockchip-gpio fd8a0000.gpio: probed /pinctrl/gpio@fd8a0000
[    0.080262] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    0.081177] rockchip-gpio fec20000.gpio: probed /pinctrl/gpio@fec20000
[    0.081877] gpio gpiochip2: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    0.082788] rockchip-gpio fec30000.gpio: probed /pinctrl/gpio@fec30000
[    0.083509] gpio gpiochip3: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    0.084419] rockchip-gpio fec40000.gpio: probed /pinctrl/gpio@fec40000
[    0.085117] gpio gpiochip4: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    0.086028] rockchip-gpio fec50000.gpio: probed /pinctrl/gpio@fec50000
[    0.089315] cryptd: max_cpu_qlen set to 1000
[    0.090138] ACPI: Interpreter disabled.
[    0.147600] iommu: Default domain type: Translated 
[    0.148049] iommu: DMA domain TLB invalidation policy: strict mode 
[    0.148902] SCSI subsystem initialized
[    0.149323] usbcore: registered new interface driver usbfs
[    0.149836] usbcore: registered new interface driver hub
[    0.150335] usbcore: registered new device driver usb
[    0.150996] pps_core: LinuxPPS API ver. 1 registered
[    0.151450] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.152298] PTP clock support registered
[    0.152822] arm-scmi firmware:scmi: Enabled polling mode TX channel - prot_id:16
[    0.153552] arm-scmi firmware:scmi: SCMI Notifications - Core Enabled.
[    0.154173] arm-scmi firmware:scmi: SCMI Protocol v2.0 'rockchip:' Firmware version 0x0
[    0.155148] Advanced Linux Sound Architecture Driver Initialized.
[    0.156011] vgaarb: loaded
[    0.156430] clocksource: Switched to clocksource arch_sys_counter
[    0.157182] pnp: PnP ACPI: disabled
[    0.162252] NET: Registered PF_INET protocol family
[    0.162807] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.166343] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)
[    0.167185] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.167898] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.168950] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)
[    0.170880] TCP: Hash tables configured (established 65536 bind 65536)
[    0.171520] UDP hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.172259] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.173088] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.173861] RPC: Registered named UNIX socket transport module.
[    0.174405] RPC: Registered udp transport module.
[    0.174834] RPC: Registered tcp transport module.
[    0.175265] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.175856] NET: Registered PF_XDP protocol family
[    0.176298] PCI: CLS 0 bytes, default 64
[    0.177335] hw perfevents: enabled with armv8_cortex_a55 PMU driver, 7 counters available
[    0.178272] hw perfevents: enabled with armv8_cortex_a76 PMU driver, 7 counters available
[    0.179256] kvm [1]: IPA Size Limit: 40 bits
[    0.179657] kvm [1]: GICv3: no GICV resource entry
[    0.180102] kvm [1]: disabling GICv2 emulation
[    0.180518] kvm [1]: GIC system register CPU interface enabled
[    0.181159] kvm [1]: vgic interrupt IRQ18
[    0.181647] kvm [1]: VHE mode initialized successfully
[    0.537847] Initialise system trusted keyrings
[    0.538394] workingset: timestamp_bits=46 max_order=21 bucket_order=0
[    0.539013] zbud: loaded
[    0.539434] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.540171] NFS: Registering the id_resolver key type
[    0.540652] Key type id_resolver registered
[    0.541034] Key type id_legacy registered
[    0.541409] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.542023] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    0.563252] NET: Registered PF_ALG protocol family
[    0.563695] Key type asymmetric registered
[    0.564071] Asymmetric key parser 'x509' registered
[    0.564518] Asymmetric key parser 'pkcs8' registered
[    0.564973] Key type pkcs7_test registered
[    0.565368] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    0.566148] io scheduler mq-deadline registered
[    0.566569] io scheduler kyber registered
[    0.566944] io scheduler bfq registered
[    0.571636] dw-pcie a41000000.pcie: host bridge /pcie@fe190000 ranges:
[    0.572241] dw-pcie a41000000.pcie: Parsing ranges property...
[    0.572780] dw-pcie a41000000.pcie:       IO 0x00f4100000..0x00f41fffff -> 0x00f4100000
[    0.573524] dw-pcie a41000000.pcie:      MEM 0x00f4200000..0x00f4ffffff -> 0x00f4200000
[    0.574259] dw-pcie a41000000.pcie:      MEM 0x0a00000000..0x0a3fffffff -> 0x0a00000000
[   65.195794] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Peter Geis]

On Tue, Feb 21, 2023 at 1:04 PM Piotr Oniszczuk
<piotr.oniszczuk@gmail.com> wrote:
>
>
>
> > Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 21.02.2023, o godz. 01:14:
> >
> >
> >
> > On 2023/2/21 02:25, Piotr Oniszczuk wrote:
> >>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com <mailto:wqu@suse.com>> w dniu 04.02.2023, o godz. 09:47:
> >>>
> >>> This series is based on the existing upstream work from Sebastian
> >>> Reichel:
> >>> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588>
> >>>
> >>> And I'm a completely newbie to arm64 world, thus if there is something
> >>> wrong, feel free to point out and I'm pretty happy to learn from the
> >>> failure.
> >>>
> >>> [BACKGROUND]
> >>> RK3588S and RK3588 have PCIE supports, it's done by the following 3
> >>> controllers:
> >>>
> >>> - PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
> >>> - PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
> >>>  Thes two are all connected to a naneng combo phy each, normally shared
> >>>  with SATA or USB.
> >>>
> >>> - PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
> >>>  This one is also connected to a naneng combo phy, normally shared
> >>>  with SATA or USB.
> >>>
> >>> - PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)
> >>>
> >>> And unlike other boards, ROCK5B is utilizing PCIE extensively, its
> >>> network controller (RTL8125 2.5Gbps Ethernet) is connected to the
> >>> PCIE2.0 lane at fe190000, and an M.2 slot is attached to the PCIE3.0x4
> >>> lanes.
> >>>
> >>> [WORKING]
> >>> Currently the series is able to bring up the PCIE3.0x4 lanes and
> >>> properly boot from an NVME at that M.2 slot of Rock5B boards.
> >>>
> >>> [NOT WORKING]
> >>> All PCIE2.0 lanes connected to naneng combo phy are not working.
> >>> I tried forward porting the extra handling from downstream, but it only
> >>> results hanging at probing (causing RCU stall).
> >>>
> >>> [EXTRA WANRING]
> >>> - PCI MSI initialization warning
> >>>  WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 pci_msi_setup_msi_irqs+0x38/0x4c
> >>>
> >>>  This seems to be caused by the fact that we are still using legcacy
> >>>  msi irqs?
> >>>
> >>>  I checked up the gic and its dts, can not figure out why (all pretty
> >>>  the same just like rk3399 and rk3568).
> >>>  Any help would be appreciated.
> >>>
> >>> - pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> >>>  The vendoer kernel also has this problem, but my RK3399 board with
> >>>  upstream kernel didn't trigger this at all, but something else like:
> >>>
> >>>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >>>
> >>>  Then:
> >>>
> >>>    pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01
> >>>
> >>>  Not sure if it's something missing or can be just ignored.
> >>>
> >>> [PCI DMESG]
> >>> With this patchset, the PCI initialization and nvme would look like this:
> >>>
> >>> [    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
> >>> [    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
> >>> [    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> >>> [    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
> >>> [    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
> >>> [    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
> >>> [    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
> >>> [    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
> >>> [    0.362236] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
> >>> [    0.363113] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
> >>> [    0.363744] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
> >>> [    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> >>> [    0.365005] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> >>> [    0.365583] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> >>> [    0.366159] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> >>> [    0.366801] pci 0000:00:00.0: supports D1 D2
> >>> [    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> >>> [    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> >>> [    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
> >>> [    0.370277] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
> >>> [    0.370975] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> >>> [    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> >>> [    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
> >>> [    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> >>> [    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
> >>> [    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 0xf0200000-0xf02fffff]
> >>> [    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref]
> >>> [    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 0xf0200000-0xf021ffff pref]
> >>> [    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0220000-0xf022ffff 64bit]
> >>> [    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> >>> [    0.377762] pci 0000:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
> >>> [    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
> >>> [    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
> >>> [    0.625353] nvme nvme0: pci function 0000:01:00.0
> >>> [    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
> >>> [    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
> >>> [    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
> >>> [    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
> >>> [    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
> >>> [    0.820678]  nvme0n1: p1 p2
> >>>
> >>>
> >> Qu, all
> >> I’m playing with your work on my rock5b as I want to have working Eth on rock5b.
> >> My code is from https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588> + your’s PCIE3 patches.
> >> SBC boots from sd card, I see PCIE related logs in dmesg. but no rtl8125 is detected.
> >> PCIE logs are like this:
> >>     8.207810] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
> >> [    8.208501] rockchip-dw-pcie fe150000.pcie: Parsing ranges property...
> >> [    8.209089] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
> >> [    8.209944] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> >> [    8.210740] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
> >> [    8.218918] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
> >> [    9.324473] rockchip-dw-pcie fe150000.pcie: Phy link never came up
> >> [    9.325186] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
> >> [    9.325786] pci_bus 0000:00: root bus resource [bus 00-0f]
> >> [    9.326266] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
> >> [    9.327097] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
> >> [    9.327713] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
> >> [    9.328364] pci_bus 0000:00: scanning bus
> >> [    9.328729] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> >> [    9.329258] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> >> [    9.329807] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> >> [    9.330354] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> >> [    9.330984] pci 0000:00:00.0: supports D1 D2
> >> [    9.331356] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> >> [    9.331858] pci 0000:00:00.0: PME# disabled
> >> [    9.332309] pci 0000:00:00.0: vgaarb: pci_notify
> >> [    9.333735] pci_bus 0000:00: fixups for bus
> >> [    9.334106] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
> >> [    9.334731] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> >> [    9.335668] pci_bus 0000:01: scanning bus
> >> [    9.336052] pci_bus 0000:01: fixups for bus
> >> [    9.336416] pci_bus 0000:01: bus scan returning with max=01
> >> [    9.336903] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
> >> [    9.337503] pci_bus 0000:00: bus scan returning with max=ff
> >> [    9.337994] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> >> [    9.338570] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
> >> [    9.339175] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> >> [    9.339749] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
> >> [    9.340356] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0200000-0xf020ffff pref]
> >> [    9.340991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> >> and nothing more :-(
> >> Are you progressing maybe with pcie on rock5b?
> > fe150000 is the PCIE3.0 controller, on Rock5B, that's the M.2 slot.
> >
> > But for R8125, it's on a PCIE2.0 controller, which is using naneng combo phy.
> >
> > I'm not able to bring the PCIE2.0 part up yet, it always hangs at PCIE2.0 initialization, thus only the PCIE3.0 part is submitted to the list.
> >
> > Thanks,
> > Qu
>
> Yes. Indeed.
> I'm trying to add pci2.0 and it looks i meet the same problem probably.
>
> I backported (from well working neggles quartz64pro repo I assume):
> https://github.com/neggles/linux-quartz64/commit/2ad84e89fc75b82c783345b72c97f5d7e3d45723
> https://github.com/neggles/linux-quartz64/commit/4ac44c2b53758eca671d695d19b456d1849d7e14
> https://github.com/neggles/linux-quartz64/commit/714c5e01d8f0f73b3a49cbdee29e3ffe0f3353dd
> https://github.com/neggles/linux-quartz64/commit/64971f9c85f27e29c44b31a0c326ea4bb8ec3c56
>
> then i added rock5b pcie dt enablements (basing on radxa BSP):
> https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.2/files/1058-arm64-dts-rockchip-enable-pcie-rock5b.patch
>
>
> this gives me quite clean 6.2 mainline boot log with hang at pcie2 init (pls see below)
>
> I assume https://github.com/neggles/linux-quartz64/commits/linux-quartz64 repo works well on Quartsz64 board - so I indirectly assume above backports should give us working pice2.0.
>
> It fails on rock5b - so I conclude: issue/error is on my side and is related to my rock5b specifics.
> Unfortunately I don’t owe Quartz64 board so can't verify correctness of my backports by testing on Quartz64pro.
>
> I’m curious about opinion of smarter than me guys…

If the phy is misconfigured, not powered, or the clocks aren't going
active, you'll hang when the controller tries to touch it. Unless
someone has completed the combophy rk3588 bits the driver is not
functional yet for rk3588.

Looking at the current 6.2 release, I only see the rk3568 compatible,
so you'll need to add support for rk3588 before it will work.

Very Respectfully,
Peter Geis

>
>
>
>
> my kernel log:
>
> Starting kernel ...
>
> [    0.000000] efi: UEFI not found.
> [    0.000000] Zone ranges:
> [    0.000000]   DMA      [mem 0x0000000000200000-0x00000000ffffffff]
> [    0.000000]   DMA32    empty
> [    0.000000]   Normal   [mem 0x0000000100000000-0x00000001ffffffff]
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000000200000-0x00000000efffffff]
> [    0.000000]   node   0: [mem 0x0000000100000000-0x00000001ffffffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000000200000-0x00000001ffffffff]
> [    0.000000] On node 0, zone DMA: 512 pages in unavailable ranges
> [    0.000000] cma: Reserved 64 MiB at 0x00000000ec000000
> [    0.000000] psci: probing for conduit method from DT.
> [    0.000000] psci: PSCIv1.1 detected in firmware.
> [    0.000000] psci: Using standard PSCI v0.2 function IDs
> [    0.000000] psci: MIGRATE_INFO_TYPE not supported.
> [    0.000000] psci: SMC Calling Convention v1.2
> [    0.000000] percpu: Embedded 29 pages/cpu s80872 r8192 d29720 u118784
> [    0.000000] pcpu-alloc: s80872 r8192 d29720 u118784 alloc=29*4096
> [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7
> [    0.000000] Detected VIPT I-cache on CPU0
> [    0.000000] CPU features: detected: GIC system register CPU interface
> [    0.000000] CPU features: detected: Virtualization Host Extensions
> [    0.000000] CPU features: detected: Qualcomm erratum 1009, or ARM erratum 1286807, 2441009
> [    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
> [    0.000000] alternatives: applying boot alternatives
> [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1999368
> [    0.000000] Kernel command line: root=/dev/mmcblk1p2 rw rootwait earlycon=uart8250,mmio32,0xfeb50000 console=ttyS2,1500000n8 debug MM_DEBUG="yes"
> [    0.000000] Unknown kernel command line parameters "MM_DEBUG=yes", will be passed to user space.
> [    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
> [    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
> [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
> [    0.000000] software IO TLB: area num 8.
> [    0.000000] software IO TLB: mapped [mem 0x00000000e8000000-0x00000000ec000000] (64MB)
> [    0.000000] Memory: 7803944K/8124416K available (14592K kernel code, 3524K rwdata, 6420K rodata, 6720K init, 576K bss, 254936K reserved, 65536K cma-reserved)
> [    0.000000] SLUB: HWalign=64, Order=0-1, MinObjects=0, CPUs=8, Nodes=1
> [    0.000000] ftrace: allocating 51240 entries in 201 pages
> [    0.000000] ftrace: allocated 201 pages with 4 groups
> [    0.000000] trace event string verifier disabled
> [    0.000000] rcu: Preemptible hierarchical RCU implementation.
> [    0.000000] .Trampoline variant of Tasks RCU enabled.
> [    0.000000] .Rude variant of Tasks RCU enabled.
> [    0.000000] .Tracing variant of Tasks RCU enabled.
> [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 30 jiffies.
> [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
> [    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
> [    0.000000] GICv3: 480 SPIs implemented
> [    0.000000] GICv3: 0 Extended SPIs implemented
> [    0.000000] Root IRQ handler: gic_handle_irq
> [    0.000000] GICv3: GICv3 features: 16 PPIs
> [    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fe680000
> [    0.000000] ITS [mem 0xfe640000-0xfe65ffff]
> [    0.000000] GIC: enabling workaround for ITS: Rockchip RK3568 force no_local_cache
> [    0.000000] ITS@0x00000000fe640000: allocated 8192 Devices @210000 (indirect, esz 8, psz 64K, shr 0)
> [    0.000000] ITS@0x00000000fe640000: allocated 32768 Interrupt Collections @220000 (flat, esz 2, psz 64K, shr 0)
> [    0.000000] ITS: using cache flushing for cmd queue
> [    0.000000] ITS [mem 0xfe660000-0xfe67ffff]
> [    0.000000] GIC: enabling workaround for ITS: Rockchip RK3568 force no_local_cache
> [    0.000000] ITS@0x00000000fe660000: allocated 8192 Devices @240000 (indirect, esz 8, psz 64K, shr 0)
> [    0.000000] ITS@0x00000000fe660000: allocated 32768 Interrupt Collections @250000 (flat, esz 2, psz 64K, shr 0)
> [    0.000000] ITS: using cache flushing for cmd queue
> [    0.000000] ITS ALLOCATE PROP WORKAROUND
> [    0.000000] GICv3: using LPI property table @0x0000000000260000
> [    0.000000] GIC: using cache flushing for LPI property table
> [    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000000270000
> [    0.000000] GICv3: GIC: PPI partition interrupt-partition-0[0] { /cpus/cpu@0[0] /cpus/cpu@100[1] /cpus/cpu@200[2] /cpus/cpu@300[3] }
> [    0.000000] GICv3: GIC: PPI partition interrupt-partition-1[1] { /cpus/cpu@400[4] /cpus/cpu@500[5] /cpus/cpu@600[6] /cpus/cpu@700[7] }
> [    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
> [    0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (phys).
> [    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns
> [    0.000000] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns
> [    0.001642] Console: colour dummy device 80x25
> [    0.002112] Calibrating delay loop (skipped), value calculated using timer frequency.. 48.00 BogoMIPS (lpj=80000)
> [    0.003044] pid_max: default: 32768 minimum: 301
> [    0.003584] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
> [    0.004286] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
> [    0.006812] cblist_init_generic: Setting adjustable number of callback queues.
> [    0.007472] cblist_init_generic: Setting shift to 3 and lim to 1.
> [    0.008124] cblist_init_generic: Setting shift to 3 and lim to 1.
> [    0.008787] cblist_init_generic: Setting shift to 3 and lim to 1.
> [    0.009551] rcu: Hierarchical SRCU implementation.
> [    0.009987] rcu: .Max phase no-delay instances is 1000.
> [    0.011795] Platform MSI: msi-controller@fe640000 domain created
> [    0.012382] Platform MSI: msi-controller@fe660000 domain created
> [    0.013066] PCI/MSI: /interrupt-controller@fe600000/msi-controller@fe640000 domain created
> [    0.013841] PCI/MSI: /interrupt-controller@fe600000/msi-controller@fe660000 domain created
> [    0.015804] EFI services will not be available.
> [    0.016811] smp: Bringing up secondary CPUs ...
> [    0.017822] Detected VIPT I-cache on CPU1
> [    0.017887] GICv3: CPU1: found redistributor 100 region 0:0x00000000fe6a0000
> [    0.017899] GICv3: CPU1: using allocated LPI pending table @0x0000000000280000
> [    0.017937] CPU1: Booted secondary processor 0x0000000100 [0x412fd050]
> [    0.018627] Detected VIPT I-cache on CPU2
> [    0.018687] GICv3: CPU2: found redistributor 200 region 0:0x00000000fe6c0000
> [    0.018698] GICv3: CPU2: using allocated LPI pending table @0x0000000000290000
> [    0.018733] CPU2: Booted secondary processor 0x0000000200 [0x412fd050]
> [    0.019427] Detected VIPT I-cache on CPU3
> [    0.019486] GICv3: CPU3: found redistributor 300 region 0:0x00000000fe6e0000
> [    0.019497] GICv3: CPU3: using allocated LPI pending table @0x00000000002a0000
> [    0.019529] CPU3: Booted secondary processor 0x0000000300 [0x412fd050]
> [    0.020182] CPU features: detected: Spectre-v4
> [    0.020186] CPU features: detected: Spectre-BHB
> [    0.020189] Detected PIPT I-cache on CPU4
> [    0.020223] GICv3: CPU4: found redistributor 400 region 0:0x00000000fe700000
> [    0.020231] GICv3: CPU4: using allocated LPI pending table @0x00000000002b0000
> [    0.020251] CPU4: Booted secondary processor 0x0000000400 [0x414fd0b0]
> [    0.020868] Detected PIPT I-cache on CPU5
> [    0.020907] GICv3: CPU5: found redistributor 500 region 0:0x00000000fe720000
> [    0.020914] GICv3: CPU5: using allocated LPI pending table @0x00000000002c0000
> [    0.020935] CPU5: Booted secondary processor 0x0000000500 [0x414fd0b0]
> [    0.021554] Detected PIPT I-cache on CPU6
> [    0.021592] GICv3: CPU6: found redistributor 600 region 0:0x00000000fe740000
> [    0.021599] GICv3: CPU6: using allocated LPI pending table @0x00000000002d0000
> [    0.021619] CPU6: Booted secondary processor 0x0000000600 [0x414fd0b0]
> [    0.022222] Detected PIPT I-cache on CPU7
> [    0.022262] GICv3: CPU7: found redistributor 700 region 0:0x00000000fe760000
> [    0.022269] GICv3: CPU7: using allocated LPI pending table @0x00000000002e0000
> [    0.022290] CPU7: Booted secondary processor 0x0000000700 [0x414fd0b0]
> [    0.022353] smp: Brought up 1 node, 8 CPUs
> [    0.039159] SMP: Total of 8 processors activated.
> [    0.039588] CPU features: detected: 32-bit EL0 Support
> [    0.040053] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
> [    0.040827] CPU features: detected: Common not Private translations
> [    0.041404] CPU features: detected: CRC32 instructions
> [    0.041869] CPU features: detected: RCpc load-acquire (LDAPR)
> [    0.042386] CPU features: detected: LSE atomic instructions
> [    0.042886] CPU features: detected: Privileged Access Never
> [    0.043389] CPU features: detected: RAS Extension Support
> [    0.043880] CPU features: detected: Speculative Store Bypassing Safe (SSBS)
> [    0.044543] spectre-bhb mitigation disabled by compile time option
> [    0.045156] CPU: All CPU(s) started at EL2
> [    0.045530] alternatives: applying system-wide alternatives
> [    0.049962] devtmpfs: initialized
> [    0.056059] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 6370867519511994 ns
> [    0.056955] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
> [    0.058021] pinctrl core: initialized pinctrl subsystem
> [    0.058735] DMI not present or invalid.
> [    0.059290] NET: Registered PF_NETLINK/PF_ROUTE protocol family
> [    0.060548] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
> [    0.061386] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
> [    0.062270] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
> [    0.063020] audit: initializing netlink subsys (disabled)
> [    0.063616] audit: type=2000 audit(0.059:1): state=initialized audit_enabled=0 res=1
> [    0.064188] thermal_sys: Registered thermal governor 'fair_share'
> [    0.064321] thermal_sys: Registered thermal governor 'bang_bang'
> [    0.064879] thermal_sys: Registered thermal governor 'step_wise'
> [    0.065429] thermal_sys: Registered thermal governor 'user_space'
> [    0.066001] cpuidle: using governor ladder
> [    0.066937] cpuidle: using governor menu
> [    0.067335] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
> [    0.068059] ASID allocator initialised with 65536 entries
> [    0.068921] Serial: AMBA PL011 UART driver
> [    0.078586] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation.
> [    0.079547] rockchip-gpio fd8a0000.gpio: probed /pinctrl/gpio@fd8a0000
> [    0.080262] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation.
> [    0.081177] rockchip-gpio fec20000.gpio: probed /pinctrl/gpio@fec20000
> [    0.081877] gpio gpiochip2: Static allocation of GPIO base is deprecated, use dynamic allocation.
> [    0.082788] rockchip-gpio fec30000.gpio: probed /pinctrl/gpio@fec30000
> [    0.083509] gpio gpiochip3: Static allocation of GPIO base is deprecated, use dynamic allocation.
> [    0.084419] rockchip-gpio fec40000.gpio: probed /pinctrl/gpio@fec40000
> [    0.085117] gpio gpiochip4: Static allocation of GPIO base is deprecated, use dynamic allocation.
> [    0.086028] rockchip-gpio fec50000.gpio: probed /pinctrl/gpio@fec50000
> [    0.089315] cryptd: max_cpu_qlen set to 1000
> [    0.090138] ACPI: Interpreter disabled.
> [    0.147600] iommu: Default domain type: Translated
> [    0.148049] iommu: DMA domain TLB invalidation policy: strict mode
> [    0.148902] SCSI subsystem initialized
> [    0.149323] usbcore: registered new interface driver usbfs
> [    0.149836] usbcore: registered new interface driver hub
> [    0.150335] usbcore: registered new device driver usb
> [    0.150996] pps_core: LinuxPPS API ver. 1 registered
> [    0.151450] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
> [    0.152298] PTP clock support registered
> [    0.152822] arm-scmi firmware:scmi: Enabled polling mode TX channel - prot_id:16
> [    0.153552] arm-scmi firmware:scmi: SCMI Notifications - Core Enabled.
> [    0.154173] arm-scmi firmware:scmi: SCMI Protocol v2.0 'rockchip:' Firmware version 0x0
> [    0.155148] Advanced Linux Sound Architecture Driver Initialized.
> [    0.156011] vgaarb: loaded
> [    0.156430] clocksource: Switched to clocksource arch_sys_counter
> [    0.157182] pnp: PnP ACPI: disabled
> [    0.162252] NET: Registered PF_INET protocol family
> [    0.162807] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)
> [    0.166343] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)
> [    0.167185] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
> [    0.167898] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)
> [    0.168950] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)
> [    0.170880] TCP: Hash tables configured (established 65536 bind 65536)
> [    0.171520] UDP hash table entries: 4096 (order: 5, 131072 bytes, linear)
> [    0.172259] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes, linear)
> [    0.173088] NET: Registered PF_UNIX/PF_LOCAL protocol family
> [    0.173861] RPC: Registered named UNIX socket transport module.
> [    0.174405] RPC: Registered udp transport module.
> [    0.174834] RPC: Registered tcp transport module.
> [    0.175265] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [    0.175856] NET: Registered PF_XDP protocol family
> [    0.176298] PCI: CLS 0 bytes, default 64
> [    0.177335] hw perfevents: enabled with armv8_cortex_a55 PMU driver, 7 counters available
> [    0.178272] hw perfevents: enabled with armv8_cortex_a76 PMU driver, 7 counters available
> [    0.179256] kvm [1]: IPA Size Limit: 40 bits
> [    0.179657] kvm [1]: GICv3: no GICV resource entry
> [    0.180102] kvm [1]: disabling GICv2 emulation
> [    0.180518] kvm [1]: GIC system register CPU interface enabled
> [    0.181159] kvm [1]: vgic interrupt IRQ18
> [    0.181647] kvm [1]: VHE mode initialized successfully
> [    0.537847] Initialise system trusted keyrings
> [    0.538394] workingset: timestamp_bits=46 max_order=21 bucket_order=0
> [    0.539013] zbud: loaded
> [    0.539434] squashfs: version 4.0 (2009/01/31) Phillip Lougher
> [    0.540171] NFS: Registering the id_resolver key type
> [    0.540652] Key type id_resolver registered
> [    0.541034] Key type id_legacy registered
> [    0.541409] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
> [    0.542023] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
> [    0.563252] NET: Registered PF_ALG protocol family
> [    0.563695] Key type asymmetric registered
> [    0.564071] Asymmetric key parser 'x509' registered
> [    0.564518] Asymmetric key parser 'pkcs8' registered
> [    0.564973] Key type pkcs7_test registered
> [    0.565368] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
> [    0.566148] io scheduler mq-deadline registered
> [    0.566569] io scheduler kyber registered
> [    0.566944] io scheduler bfq registered
> [    0.571636] dw-pcie a41000000.pcie: host bridge /pcie@fe190000 ranges:
> [    0.572241] dw-pcie a41000000.pcie: Parsing ranges property...
> [    0.572780] dw-pcie a41000000.pcie:       IO 0x00f4100000..0x00f41fffff -> 0x00f4100000
> [    0.573524] dw-pcie a41000000.pcie:      MEM 0x00f4200000..0x00f4ffffff -> 0x00f4200000
> [    0.574259] dw-pcie a41000000.pcie:      MEM 0x0a00000000..0x0a3fffffff -> 0x0a00000000
> [   65.195794] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
>
>
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Sebastian Reichel]

[-- Attachment #1.1: Type: text/plain, Size: 13992 bytes --]

Hi everyone,

On Tue, Feb 21, 2023 at 01:55:48PM -0500, Peter Geis wrote:
> On Tue, Feb 21, 2023 at 1:04 PM Piotr Oniszczuk
> <piotr.oniszczuk@gmail.com> wrote:
> > > Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 21.02.2023, o godz. 01:14:
> > > On 2023/2/21 02:25, Piotr Oniszczuk wrote:
> > >>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com <mailto:wqu@suse.com>> w dniu 04.02.2023, o godz. 09:47:
> > >>>
> > >>> This series is based on the existing upstream work from Sebastian
> > >>> Reichel:
> > >>> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588>
> > >>>
> > >>> And I'm a completely newbie to arm64 world, thus if there is something
> > >>> wrong, feel free to point out and I'm pretty happy to learn from the
> > >>> failure.
> > >>>
> > >>> [BACKGROUND]
> > >>> RK3588S and RK3588 have PCIE supports, it's done by the following 3
> > >>> controllers:
> > >>>
> > >>> - PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
> > >>> - PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
> > >>>  Thes two are all connected to a naneng combo phy each, normally shared
> > >>>  with SATA or USB.
> > >>>
> > >>> - PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
> > >>>  This one is also connected to a naneng combo phy, normally shared
> > >>>  with SATA or USB.
> > >>>
> > >>> - PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)
> > >>>
> > >>> And unlike other boards, ROCK5B is utilizing PCIE extensively, its
> > >>> network controller (RTL8125 2.5Gbps Ethernet) is connected to the
> > >>> PCIE2.0 lane at fe190000, and an M.2 slot is attached to the PCIE3.0x4
> > >>> lanes.
> > >>>
> > >>> [WORKING]
> > >>> Currently the series is able to bring up the PCIE3.0x4 lanes and
> > >>> properly boot from an NVME at that M.2 slot of Rock5B boards.
> > >>>
> > >>> [NOT WORKING]
> > >>> All PCIE2.0 lanes connected to naneng combo phy are not working.
> > >>> I tried forward porting the extra handling from downstream, but it only
> > >>> results hanging at probing (causing RCU stall).
> > >>>
> > >>> [EXTRA WANRING]
> > >>> - PCI MSI initialization warning
> > >>>  WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 pci_msi_setup_msi_irqs+0x38/0x4c
> > >>>
> > >>>  This seems to be caused by the fact that we are still using legcacy
> > >>>  msi irqs?
> > >>>
> > >>>  I checked up the gic and its dts, can not figure out why (all pretty
> > >>>  the same just like rk3399 and rk3568).
> > >>>  Any help would be appreciated.
> > >>>
> > >>> - pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> > >>>  The vendoer kernel also has this problem, but my RK3399 board with
> > >>>  upstream kernel didn't trigger this at all, but something else like:
> > >>>
> > >>>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > >>>
> > >>>  Then:
> > >>>
> > >>>    pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01
> > >>>
> > >>>  Not sure if it's something missing or can be just ignored.
> > >>>
> > >>> [PCI DMESG]
> > >>> With this patchset, the PCI initialization and nvme would look like this:
> > >>>
> > >>> [    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
> > >>> [    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
> > >>> [    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> > >>> [    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
> > >>> [    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
> > >>> [    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
> > >>> [    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
> > >>> [    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
> > >>> [    0.362236] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
> > >>> [    0.363113] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
> > >>> [    0.363744] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
> > >>> [    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> > >>> [    0.365005] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> > >>> [    0.365583] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> > >>> [    0.366159] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> > >>> [    0.366801] pci 0000:00:00.0: supports D1 D2
> > >>> [    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> > >>> [    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> > >>> [    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
> > >>> [    0.370277] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
> > >>> [    0.370975] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> > >>> [    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> > >>> [    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
> > >>> [    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> > >>> [    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
> > >>> [    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 0xf0200000-0xf02fffff]
> > >>> [    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref]
> > >>> [    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 0xf0200000-0xf021ffff pref]
> > >>> [    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0220000-0xf022ffff 64bit]
> > >>> [    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> > >>> [    0.377762] pci 0000:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
> > >>> [    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
> > >>> [    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
> > >>> [    0.625353] nvme nvme0: pci function 0000:01:00.0
> > >>> [    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
> > >>> [    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
> > >>> [    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
> > >>> [    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
> > >>> [    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
> > >>> [    0.820678]  nvme0n1: p1 p2
> > >>>
> > >>>
> > >> Qu, all
> > >> I’m playing with your work on my rock5b as I want to have working Eth on rock5b.
> > >> My code is from https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588> + your’s PCIE3 patches.
> > >> SBC boots from sd card, I see PCIE related logs in dmesg. but no rtl8125 is detected.
> > >> PCIE logs are like this:
> > >>     8.207810] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
> > >> [    8.208501] rockchip-dw-pcie fe150000.pcie: Parsing ranges property...
> > >> [    8.209089] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
> > >> [    8.209944] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> > >> [    8.210740] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
> > >> [    8.218918] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
> > >> [    9.324473] rockchip-dw-pcie fe150000.pcie: Phy link never came up
> > >> [    9.325186] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
> > >> [    9.325786] pci_bus 0000:00: root bus resource [bus 00-0f]
> > >> [    9.326266] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
> > >> [    9.327097] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
> > >> [    9.327713] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
> > >> [    9.328364] pci_bus 0000:00: scanning bus
> > >> [    9.328729] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> > >> [    9.329258] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> > >> [    9.329807] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> > >> [    9.330354] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> > >> [    9.330984] pci 0000:00:00.0: supports D1 D2
> > >> [    9.331356] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> > >> [    9.331858] pci 0000:00:00.0: PME# disabled
> > >> [    9.332309] pci 0000:00:00.0: vgaarb: pci_notify
> > >> [    9.333735] pci_bus 0000:00: fixups for bus
> > >> [    9.334106] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
> > >> [    9.334731] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> > >> [    9.335668] pci_bus 0000:01: scanning bus
> > >> [    9.336052] pci_bus 0000:01: fixups for bus
> > >> [    9.336416] pci_bus 0000:01: bus scan returning with max=01
> > >> [    9.336903] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
> > >> [    9.337503] pci_bus 0000:00: bus scan returning with max=ff
> > >> [    9.337994] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> > >> [    9.338570] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
> > >> [    9.339175] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> > >> [    9.339749] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
> > >> [    9.340356] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0200000-0xf020ffff pref]
> > >> [    9.340991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> > >> and nothing more :-(
> > >> Are you progressing maybe with pcie on rock5b?
> > > fe150000 is the PCIE3.0 controller, on Rock5B, that's the M.2 slot.
> > >
> > > But for R8125, it's on a PCIE2.0 controller, which is using naneng combo phy.
> > >
> > > I'm not able to bring the PCIE2.0 part up yet, it always hangs at PCIE2.0 initialization, thus only the PCIE3.0 part is submitted to the list.
> > >
> > > Thanks,
> > > Qu
> >
> > Yes. Indeed.
> > I'm trying to add pci2.0 and it looks i meet the same problem probably.
> >
> > I backported (from well working neggles quartz64pro repo I assume):
> > https://github.com/neggles/linux-quartz64/commit/2ad84e89fc75b82c783345b72c97f5d7e3d45723
> > https://github.com/neggles/linux-quartz64/commit/4ac44c2b53758eca671d695d19b456d1849d7e14
> > https://github.com/neggles/linux-quartz64/commit/714c5e01d8f0f73b3a49cbdee29e3ffe0f3353dd
> > https://github.com/neggles/linux-quartz64/commit/64971f9c85f27e29c44b31a0c326ea4bb8ec3c56
> >
> > then i added rock5b pcie dt enablements (basing on radxa BSP):
> > https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.2/files/1058-arm64-dts-rockchip-enable-pcie-rock5b.patch
> >
> >
> > this gives me quite clean 6.2 mainline boot log with hang at pcie2 init (pls see below)
> >
> > I assume https://github.com/neggles/linux-quartz64/commits/linux-quartz64 repo works well on Quartsz64 board - so I indirectly assume above backports should give us working pice2.0.
> >
> > It fails on rock5b - so I conclude: issue/error is on my side and is related to my rock5b specifics.
> > Unfortunately I don’t owe Quartz64 board so can't verify correctness of my backports by testing on Quartz64pro.
> >
> > I’m curious about opinion of smarter than me guys…
> 
> If the phy is misconfigured, not powered, or the clocks aren't going
> active, you'll hang when the controller tries to touch it. Unless
> someone has completed the combophy rk3588 bits the driver is not
> functional yet for rk3588.
> 
> Looking at the current 6.2 release, I only see the rk3568 compatible,
> so you'll need to add support for rk3588 before it will work.
> 
> Very Respectfully,
> Peter Geis

Sorry for being late to the game. Life is busy right now :)

I haven't looked into PCIe myself so far, but some of my colleagues
are looking into native network support on Rock 5B (and thus PCIe2
controller).

Apart from the obvious (missing rk3588 support in the combophy driver),
the clocks will need some work. The clock tree implementation I upstreamed
is different from the downstream implementation. Downstream has some
clocks that have two parent clocks using a hacked implementation
that's obviously not upstreamable as is. The upstream implementation
currently only describes the first parent. More details are in the
following comment:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/clk/rockchip/clk-rk3588.c#n16

Back than I wrote that I do not understand the exact need (the TRM
does not describe the clock tree unfortuantely), I found this
explanation:

https://lore.kernel.org/dri-devel/20220309094139198367142@rock-chips.com/

To get the advanced blocks properly running upstream this needs a
solution for this. Note that trying to access registers that are
not clocked properly will result in a hang (as Peter already wrote).

Apart from that the power-domain controller might need some of the
extra bits downstream has.

Last but not least the GIC controller is handled differently in
downstream. For that following the workaround that has been used for
rk356x should also work for rk3588.

TLDR: This is not trivial. It's really unfortunate, that the board
is not just using the native ethernet :(

P.S.: We try to keep a rk3588 / Rock 5 status matrix maintained here:

https://gitlab.collabora.com/hardware-enablement/rockchip-3588/notes-for-rockchip-3588/-/blob/main/mainline-status.md

Greetings,

-- Sebastian

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Qu Wenruo]

On 2023/2/22 05:45, Sebastian Reichel wrote:
> Hi everyone,
> 
[...]
>>
>> If the phy is misconfigured, not powered, or the clocks aren't going
>> active, you'll hang when the controller tries to touch it. Unless
>> someone has completed the combophy rk3588 bits the driver is not
>> functional yet for rk3588.
>>
>> Looking at the current 6.2 release, I only see the rk3568 compatible,
>> so you'll need to add support for rk3588 before it will work.
>>
>> Very Respectfully,
>> Peter Geis
> 
> Sorry for being late to the game. Life is busy right now :)

Welcome back!

> 
> I haven't looked into PCIe myself so far, but some of my colleagues
> are looking into native network support on Rock 5B (and thus PCIe2
> controller).
> 
> Apart from the obvious (missing rk3588 support in the combophy driver),
> the clocks will need some work. The clock tree implementation I upstreamed
> is different from the downstream implementation. Downstream has some
> clocks that have two parent clocks using a hacked implementation
> that's obviously not upstreamable as is. The upstream implementation
> currently only describes the first parent. More details are in the
> following comment:

Thanks for the details!

No wonder why the combo phy doesn't work.

I tried checking the downstream driver, and can only find trivial 
differences between rk3588 and rk3568 naneng combo drivers.

But didn't notice the clock problems.

BTW, is there any docs on all the registers?
For the combo phy drivers, there are some registers only utilized by 
3588 but not 356x, thus docs on this would be very helpful.

Thanks,
Qu

> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/clk/rockchip/clk-rk3588.c#n16
> 
> Back than I wrote that I do not understand the exact need (the TRM
> does not describe the clock tree unfortuantely), I found this
> explanation:
> 
> https://lore.kernel.org/dri-devel/20220309094139198367142@rock-chips.com/
> 
> To get the advanced blocks properly running upstream this needs a
> solution for this. Note that trying to access registers that are
> not clocked properly will result in a hang (as Peter already wrote).
> 
> Apart from that the power-domain controller might need some of the
> extra bits downstream has.
> 
> Last but not least the GIC controller is handled differently in
> downstream. For that following the workaround that has been used for
> rk356x should also work for rk3588.
> 
> TLDR: This is not trivial. It's really unfortunate, that the board
> is not just using the native ethernet :(
> 
> P.S.: We try to keep a rk3588 / Rock 5 status matrix maintained here:
> 
> https://gitlab.collabora.com/hardware-enablement/rockchip-3588/notes-for-rockchip-3588/-/blob/main/mainline-status.md
> 
> Greetings,
> 
> -- Sebastian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Peter Geis]

On Tue, Feb 21, 2023 at 4:45 PM Sebastian Reichel
<sebastian.reichel@collabora.com> wrote:
>
> Hi everyone,
>
> On Tue, Feb 21, 2023 at 01:55:48PM -0500, Peter Geis wrote:
> > On Tue, Feb 21, 2023 at 1:04 PM Piotr Oniszczuk
> > <piotr.oniszczuk@gmail.com> wrote:
> > > > Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 21.02.2023, o godz. 01:14:
> > > > On 2023/2/21 02:25, Piotr Oniszczuk wrote:
> > > >>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com <mailto:wqu@suse.com>> w dniu 04.02.2023, o godz. 09:47:
> > > >>>
> > > >>> This series is based on the existing upstream work from Sebastian
> > > >>> Reichel:
> > > >>> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588>
> > > >>>
> > > >>> And I'm a completely newbie to arm64 world, thus if there is something
> > > >>> wrong, feel free to point out and I'm pretty happy to learn from the
> > > >>> failure.
> > > >>>
> > > >>> [BACKGROUND]
> > > >>> RK3588S and RK3588 have PCIE supports, it's done by the following 3
> > > >>> controllers:
> > > >>>
> > > >>> - PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
> > > >>> - PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
> > > >>>  Thes two are all connected to a naneng combo phy each, normally shared
> > > >>>  with SATA or USB.
> > > >>>
> > > >>> - PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
> > > >>>  This one is also connected to a naneng combo phy, normally shared
> > > >>>  with SATA or USB.
> > > >>>
> > > >>> - PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)
> > > >>>
> > > >>> And unlike other boards, ROCK5B is utilizing PCIE extensively, its
> > > >>> network controller (RTL8125 2.5Gbps Ethernet) is connected to the
> > > >>> PCIE2.0 lane at fe190000, and an M.2 slot is attached to the PCIE3.0x4
> > > >>> lanes.
> > > >>>
> > > >>> [WORKING]
> > > >>> Currently the series is able to bring up the PCIE3.0x4 lanes and
> > > >>> properly boot from an NVME at that M.2 slot of Rock5B boards.
> > > >>>
> > > >>> [NOT WORKING]
> > > >>> All PCIE2.0 lanes connected to naneng combo phy are not working.
> > > >>> I tried forward porting the extra handling from downstream, but it only
> > > >>> results hanging at probing (causing RCU stall).
> > > >>>
> > > >>> [EXTRA WANRING]
> > > >>> - PCI MSI initialization warning
> > > >>>  WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 pci_msi_setup_msi_irqs+0x38/0x4c
> > > >>>
> > > >>>  This seems to be caused by the fact that we are still using legcacy
> > > >>>  msi irqs?
> > > >>>
> > > >>>  I checked up the gic and its dts, can not figure out why (all pretty
> > > >>>  the same just like rk3399 and rk3568).
> > > >>>  Any help would be appreciated.
> > > >>>
> > > >>> - pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> > > >>>  The vendoer kernel also has this problem, but my RK3399 board with
> > > >>>  upstream kernel didn't trigger this at all, but something else like:
> > > >>>
> > > >>>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > > >>>
> > > >>>  Then:
> > > >>>
> > > >>>    pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01
> > > >>>
> > > >>>  Not sure if it's something missing or can be just ignored.
> > > >>>
> > > >>> [PCI DMESG]
> > > >>> With this patchset, the PCI initialization and nvme would look like this:
> > > >>>
> > > >>> [    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
> > > >>> [    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
> > > >>> [    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> > > >>> [    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
> > > >>> [    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
> > > >>> [    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
> > > >>> [    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
> > > >>> [    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
> > > >>> [    0.362236] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
> > > >>> [    0.363113] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
> > > >>> [    0.363744] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
> > > >>> [    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> > > >>> [    0.365005] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> > > >>> [    0.365583] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> > > >>> [    0.366159] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> > > >>> [    0.366801] pci 0000:00:00.0: supports D1 D2
> > > >>> [    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> > > >>> [    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> > > >>> [    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
> > > >>> [    0.370277] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
> > > >>> [    0.370975] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> > > >>> [    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> > > >>> [    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
> > > >>> [    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> > > >>> [    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
> > > >>> [    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 0xf0200000-0xf02fffff]
> > > >>> [    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref]
> > > >>> [    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 0xf0200000-0xf021ffff pref]
> > > >>> [    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0220000-0xf022ffff 64bit]
> > > >>> [    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> > > >>> [    0.377762] pci 0000:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
> > > >>> [    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
> > > >>> [    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
> > > >>> [    0.625353] nvme nvme0: pci function 0000:01:00.0
> > > >>> [    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
> > > >>> [    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
> > > >>> [    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
> > > >>> [    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
> > > >>> [    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
> > > >>> [    0.820678]  nvme0n1: p1 p2
> > > >>>
> > > >>>
> > > >> Qu, all
> > > >> I’m playing with your work on my rock5b as I want to have working Eth on rock5b.
> > > >> My code is from https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588> + your’s PCIE3 patches.
> > > >> SBC boots from sd card, I see PCIE related logs in dmesg. but no rtl8125 is detected.
> > > >> PCIE logs are like this:
> > > >>     8.207810] rockchip-dw-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
> > > >> [    8.208501] rockchip-dw-pcie fe150000.pcie: Parsing ranges property...
> > > >> [    8.209089] rockchip-dw-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
> > > >> [    8.209944] rockchip-dw-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
> > > >> [    8.210740] rockchip-dw-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
> > > >> [    8.218918] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
> > > >> [    9.324473] rockchip-dw-pcie fe150000.pcie: Phy link never came up
> > > >> [    9.325186] rockchip-dw-pcie fe150000.pcie: PCI host bridge to bus 0000:00
> > > >> [    9.325786] pci_bus 0000:00: root bus resource [bus 00-0f]
> > > >> [    9.326266] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
> > > >> [    9.327097] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
> > > >> [    9.327713] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
> > > >> [    9.328364] pci_bus 0000:00: scanning bus
> > > >> [    9.328729] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
> > > >> [    9.329258] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
> > > >> [    9.329807] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
> > > >> [    9.330354] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> > > >> [    9.330984] pci 0000:00:00.0: supports D1 D2
> > > >> [    9.331356] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> > > >> [    9.331858] pci 0000:00:00.0: PME# disabled
> > > >> [    9.332309] pci 0000:00:00.0: vgaarb: pci_notify
> > > >> [    9.333735] pci_bus 0000:00: fixups for bus
> > > >> [    9.334106] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
> > > >> [    9.334731] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
> > > >> [    9.335668] pci_bus 0000:01: scanning bus
> > > >> [    9.336052] pci_bus 0000:01: fixups for bus
> > > >> [    9.336416] pci_bus 0000:01: bus scan returning with max=01
> > > >> [    9.336903] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
> > > >> [    9.337503] pci_bus 0000:00: bus scan returning with max=ff
> > > >> [    9.337994] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
> > > >> [    9.338570] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
> > > >> [    9.339175] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
> > > >> [    9.339749] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
> > > >> [    9.340356] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0200000-0xf020ffff pref]
> > > >> [    9.340991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> > > >> and nothing more :-(
> > > >> Are you progressing maybe with pcie on rock5b?
> > > > fe150000 is the PCIE3.0 controller, on Rock5B, that's the M.2 slot.
> > > >
> > > > But for R8125, it's on a PCIE2.0 controller, which is using naneng combo phy.
> > > >
> > > > I'm not able to bring the PCIE2.0 part up yet, it always hangs at PCIE2.0 initialization, thus only the PCIE3.0 part is submitted to the list.
> > > >
> > > > Thanks,
> > > > Qu
> > >
> > > Yes. Indeed.
> > > I'm trying to add pci2.0 and it looks i meet the same problem probably.
> > >
> > > I backported (from well working neggles quartz64pro repo I assume):
> > > https://github.com/neggles/linux-quartz64/commit/2ad84e89fc75b82c783345b72c97f5d7e3d45723
> > > https://github.com/neggles/linux-quartz64/commit/4ac44c2b53758eca671d695d19b456d1849d7e14
> > > https://github.com/neggles/linux-quartz64/commit/714c5e01d8f0f73b3a49cbdee29e3ffe0f3353dd
> > > https://github.com/neggles/linux-quartz64/commit/64971f9c85f27e29c44b31a0c326ea4bb8ec3c56
> > >
> > > then i added rock5b pcie dt enablements (basing on radxa BSP):
> > > https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.2/files/1058-arm64-dts-rockchip-enable-pcie-rock5b.patch
> > >
> > >
> > > this gives me quite clean 6.2 mainline boot log with hang at pcie2 init (pls see below)
> > >
> > > I assume https://github.com/neggles/linux-quartz64/commits/linux-quartz64 repo works well on Quartsz64 board - so I indirectly assume above backports should give us working pice2.0.
> > >
> > > It fails on rock5b - so I conclude: issue/error is on my side and is related to my rock5b specifics.
> > > Unfortunately I don’t owe Quartz64 board so can't verify correctness of my backports by testing on Quartz64pro.
> > >
> > > I’m curious about opinion of smarter than me guys…
> >
> > If the phy is misconfigured, not powered, or the clocks aren't going
> > active, you'll hang when the controller tries to touch it. Unless
> > someone has completed the combophy rk3588 bits the driver is not
> > functional yet for rk3588.
> >
> > Looking at the current 6.2 release, I only see the rk3568 compatible,
> > so you'll need to add support for rk3588 before it will work.
> >
> > Very Respectfully,
> > Peter Geis
>
> Sorry for being late to the game. Life is busy right now :)
>
> I haven't looked into PCIe myself so far, but some of my colleagues
> are looking into native network support on Rock 5B (and thus PCIe2
> controller).
>
> Apart from the obvious (missing rk3588 support in the combophy driver),
> the clocks will need some work. The clock tree implementation I upstreamed
> is different from the downstream implementation. Downstream has some
> clocks that have two parent clocks using a hacked implementation
> that's obviously not upstreamable as is. The upstream implementation
> currently only describes the first parent. More details are in the
> following comment:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/clk/rockchip/clk-rk3588.c#n16
>
> Back than I wrote that I do not understand the exact need (the TRM
> does not describe the clock tree unfortuantely), I found this
> explanation:
>
> https://lore.kernel.org/dri-devel/20220309094139198367142@rock-chips.com/
>
> To get the advanced blocks properly running upstream this needs a
> solution for this. Note that trying to access registers that are
> not clocked properly will result in a hang (as Peter already wrote).

TLDR: There are now hardware blocks called a "Native Interface Unit"
(NIU) that gate the clocks to devices behind them. Essentially it
makes certain device clocks dependent on multiple parents being active
at the same time, which the current clock structure does not support.
It was decided that until the clock structure is updated to support
this, the NIU gate clocks would be marked critical and left always on.
Better to burn a miniscule more power than risk locking the board up.

Though the issue here is much simpler than that, the controller simply
can't operate without a functional phy attached to it. They are
physically tied together and it's the controller < - > phy link that
doesn't function without the phy being configured. The combophy config
for rk3588 isn't significantly more complicated than rk356x, but one
can't mainline code that can't be tested and rk3588 wasn't available
at that time.

>
> Apart from that the power-domain controller might need some of the
> extra bits downstream has.

If PCIe 3 works, then this likely won't affect PCIe 2, though it
limits our ability to implement runtime power management.

>
> Last but not least the GIC controller is handled differently in
> downstream. For that following the workaround that has been used for
> rk356x should also work for rk3588.

The hack for the GIC controller that I maintain for rk356x is not
upstreamable. For the ITS to be supported in mainline we still require
an official errata from Rockchip documenting the issues here. The
mbi-alias method, while it works for simple configs, isn't scalable.

>
> TLDR: This is not trivial. It's really unfortunate, that the board
> is not just using the native ethernet :(
>
> P.S.: We try to keep a rk3588 / Rock 5 status matrix maintained here:
>
> https://gitlab.collabora.com/hardware-enablement/rockchip-3588/notes-for-rockchip-3588/-/blob/main/mainline-status.md
>
> Greetings,
>
> -- Sebastian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Qu Wenruo]

On 2023/3/9 20:02, Piotr Oniszczuk wrote:
> 
> 
>> Wiadomość napisana przez Peter Geis <pgwipeout@gmail.com 
>> <mailto:pgwipeout@gmail.com>> w dniu 22.02.2023, o godz. 02:25:
>>
>> On Tue, Feb 21, 2023 at 4:45 PM Sebastian Reichel
>> <sebastian.reichel@collabora.com 
>> <mailto:sebastian.reichel@collabora.com>> wrote:
>>>
>>> Hi everyone,
>>>
>>> On Tue, Feb 21, 2023 at 01:55:48PM -0500, Peter Geis wrote:
>>>> On Tue, Feb 21, 2023 at 1:04 PM Piotr Oniszczuk
>>>> <piotr.oniszczuk@gmail.com <mailto:piotr.oniszczuk@gmail.com>> wrote:
>>>>>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com 
>>>>>> <mailto:wqu@suse.com>> w dniu 21.02.2023, o godz. 01:14:
>>>>>> On 2023/2/21 02:25, Piotr Oniszczuk wrote:
>>>>>>>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com 
>>>>>>>> <mailto:wqu@suse.com> <mailto:wqu@suse.com 
>>>>>>>> <mailto:wqu@suse.com>>> w dniu 04.02.2023, o godz. 09:47:
>>>>>>>>
>>>>>>>> This series is based on the existing upstream work from Sebastian
>>>>>>>> Reichel:
>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588> <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588>>
>>>>>>>>
>>>>>>>> And I'm a completely newbie to arm64 world, thus if there is 
>>>>>>>> something
>>>>>>>> wrong, feel free to point out and I'm pretty happy to learn from the
>>>>>>>> failure.
>>>>>>>>
>>>>>>>> [BACKGROUND]
>>>>>>>> RK3588S and RK3588 have PCIE supports, it's done by the following 3
>>>>>>>> controllers:
>>>>>>>>
>>>>>>>> - PCIE2.0x1 lane @fe180000 (both RK3588S and RK3588)
>>>>>>>> - PCIE2.0x1 lane @fe190000 (both RK3588S and RK3588)
>>>>>>>> Thes two are all connected to a naneng combo phy each, normally 
>>>>>>>> shared
>>>>>>>> with SATA or USB.
>>>>>>>>
>>>>>>>> - PCIE2.0x1 lane @fe170000 (RK3588 exlusive)
>>>>>>>> This one is also connected to a naneng combo phy, normally shared
>>>>>>>> with SATA or USB.
>>>>>>>>
>>>>>>>> - PCIE3.0x4 lanes @fe15000 (RK3588 exclusive)
>>>>>>>>
>>>>>>>> And unlike other boards, ROCK5B is utilizing PCIE extensively, its
>>>>>>>> network controller (RTL8125 2.5Gbps Ethernet) is connected to the
>>>>>>>> PCIE2.0 lane at fe190000, and an M.2 slot is attached to the 
>>>>>>>> PCIE3.0x4
>>>>>>>> lanes.
>>>>>>>>
>>>>>>>> [WORKING]
>>>>>>>> Currently the series is able to bring up the PCIE3.0x4 lanes and
>>>>>>>> properly boot from an NVME at that M.2 slot of Rock5B boards.
>>>>>>>>
>>>>>>>> [NOT WORKING]
>>>>>>>> All PCIE2.0 lanes connected to naneng combo phy are not working.
>>>>>>>> I tried forward porting the extra handling from downstream, but 
>>>>>>>> it only
>>>>>>>> results hanging at probing (causing RCU stall).
>>>>>>>>
>>>>>>>> [EXTRA WANRING]
>>>>>>>> - PCI MSI initialization warning
>>>>>>>> WARNING: CPU: 7 PID: 1 at drivers/pci/msi/msi.h:121 
>>>>>>>> pci_msi_setup_msi_irqs+0x38/0x4c
>>>>>>>>
>>>>>>>> This seems to be caused by the fact that we are still using legcacy
>>>>>>>> msi irqs?
>>>>>>>>
>>>>>>>> I checked up the gic and its dts, can not figure out why (all pretty
>>>>>>>> the same just like rk3399 and rk3568).
>>>>>>>> Any help would be appreciated.
>>>>>>>>
>>>>>>>> - pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under 
>>>>>>>> [bus 00-0f] (conflicts with (null) [bus 00-0f])
>>>>>>>> The vendoer kernel also has this problem, but my RK3399 board with
>>>>>>>> upstream kernel didn't trigger this at all, but something else like:
>>>>>>>>
>>>>>>>>   pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), 
>>>>>>>> reconfiguring
>>>>>>>>
>>>>>>>> Then:
>>>>>>>>
>>>>>>>>   pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01
>>>>>>>>
>>>>>>>> Not sure if it's something missing or can be just ignored.
>>>>>>>>
>>>>>>>> [PCI DMESG]
>>>>>>>> With this patchset, the PCI initialization and nvme would look 
>>>>>>>> like this:
>>>>>>>>
>>>>>>>> [    0.142984] rockchip-dw-pcie fe150000.pcie: host bridge 
>>>>>>>> /pcie@fe150000 ranges:
>>>>>>>> [    0.143653] rockchip-dw-pcie fe150000.pcie:       IO 
>>>>>>>> 0x00f0100000..0x00f01fffff -> 0x00f0100000
>>>>>>>> [    0.144463] rockchip-dw-pcie fe150000.pcie:      MEM 
>>>>>>>> 0x00f0200000..0x00f0ffffff -> 0x00f0200000
>>>>>>>> [    0.145261] rockchip-dw-pcie fe150000.pcie:      MEM 
>>>>>>>> 0x0900000000..0x093fffffff -> 0x0900000000
>>>>>>>> [    0.154022] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 
>>>>>>>> ob, 8 ib, align 64K, limit 8G
>>>>>>>> [    0.360415] rockchip-dw-pcie fe150000.pcie: PCIe Gen.3 x4 link up
>>>>>>>> [    0.361099] rockchip-dw-pcie fe150000.pcie: PCI host bridge 
>>>>>>>> to bus 0000:00
>>>>>>>> [    0.361731] pci_bus 0000:00: root bus resource [bus 00-0f]
>>>>>>>> [    0.362236] pci_bus 0000:00: root bus resource [io 
>>>>>>>>  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
>>>>>>>> [    0.363113] pci_bus 0000:00: root bus resource [mem 
>>>>>>>> 0xf0200000-0xf0ffffff]
>>>>>>>> [    0.363744] pci_bus 0000:00: root bus resource [mem 
>>>>>>>> 0x900000000-0x93fffffff pref]
>>>>>>>> [    0.364450] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
>>>>>>>> [    0.365005] pci 0000:00:00.0: reg 0x10: [mem 
>>>>>>>> 0x00000000-0x3fffffff]
>>>>>>>> [    0.365583] pci 0000:00:00.0: reg 0x14: [mem 
>>>>>>>> 0x00000000-0x3fffffff]
>>>>>>>> [    0.366159] pci 0000:00:00.0: reg 0x38: [mem 
>>>>>>>> 0x00000000-0x0000ffff pref]
>>>>>>>> [    0.366801] pci 0000:00:00.0: supports D1 D2
>>>>>>>> [    0.367193] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
>>>>>>>> [    0.368647] pci_bus 0000:01: busn_res: can not insert [bus 
>>>>>>>> 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
>>>>>>>> [    0.369681] pci 0000:01:00.0: [1d97:5216] type 00 class 0x010802
>>>>>>>> [    0.370277] pci 0000:01:00.0: reg 0x10: [mem 
>>>>>>>> 0x00000000-0x0000ffff 64bit]
>>>>>>>> [    0.370975] pci 0000:01:00.0: reg 0x30: [mem 
>>>>>>>> 0x00000000-0x0001ffff pref]
>>>>>>>> [    0.372130] pci 0000:00:00.0: BAR 0: no space for [mem size 
>>>>>>>> 0x40000000]
>>>>>>>> [    0.372742] pci 0000:00:00.0: BAR 0: failed to assign [mem 
>>>>>>>> size 0x40000000]
>>>>>>>> [    0.373381] pci 0000:00:00.0: BAR 1: no space for [mem size 
>>>>>>>> 0x40000000]
>>>>>>>> [    0.373988] pci 0000:00:00.0: BAR 1: failed to assign [mem 
>>>>>>>> size 0x40000000]
>>>>>>>> [    0.374628] pci 0000:00:00.0: BAR 14: assigned [mem 
>>>>>>>> 0xf0200000-0xf02fffff]
>>>>>>>> [    0.375259] pci 0000:00:00.0: BAR 6: assigned [mem 
>>>>>>>> 0xf0300000-0xf030ffff pref]
>>>>>>>> [    0.375923] pci 0000:01:00.0: BAR 6: assigned [mem 
>>>>>>>> 0xf0200000-0xf021ffff pref]
>>>>>>>> [    0.376590] pci 0000:01:00.0: BAR 0: assigned [mem 
>>>>>>>> 0xf0220000-0xf022ffff 64bit]
>>>>>>>> [    0.377281] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>>>>>>>> [    0.377762] pci 0000:00:00.0:   bridge window [mem 
>>>>>>>> 0xf0200000-0xf02fffff]
>>>>>>>> [    0.426841] pcieport 0000:00:00.0: PME: Signaling with IRQ 33
>>>>>>>> [    0.427487] pcieport 0000:00:00.0: AER: enabled with IRQ 33
>>>>>>>> [    0.625353] nvme nvme0: pci function 0000:01:00.0
>>>>>>>> [    0.625774] nvme 0000:01:00.0: enabling device (0000 -> 0002)
>>>>>>>> [    0.717069] nvme nvme0: Shutdown timeout set to 8 seconds
>>>>>>>> [    0.723025] nvme nvme0: allocated 64 MiB host memory buffer.
>>>>>>>> [    0.816820] nvme nvme0: 1/0/0 default/read/poll queues
>>>>>>>> [    0.818079] sdhci-pltfm: SDHCI platform and OF driver helper
>>>>>>>> [    0.820678]  nvme0n1: p1 p2
>>>>>>>>
>>>>>>>>
>>>>>>> Qu, all
>>>>>>> I’m playing with your work on my rock5b as I want to have working 
>>>>>>> Eth on rock5b.
>>>>>>> My code is from 
>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588> <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588 <https://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-misc.git/log/?h=rk3588>> + your’s PCIE3 patches.
>>>>>>> SBC boots from sd card, I see PCIE related logs in dmesg. but no 
>>>>>>> rtl8125 is detected.
>>>>>>> PCIE logs are like this:
>>>>>>>    8.207810] rockchip-dw-pcie fe150000.pcie: host bridge 
>>>>>>> /pcie@fe150000 ranges:
>>>>>>> [    8.208501] rockchip-dw-pcie fe150000.pcie: Parsing ranges 
>>>>>>> property...
>>>>>>> [    8.209089] rockchip-dw-pcie fe150000.pcie:       IO 
>>>>>>> 0x00f0100000..0x00f01fffff -> 0x00f0100000
>>>>>>> [    8.209944] rockchip-dw-pcie fe150000.pcie:      MEM 
>>>>>>> 0x00f0200000..0x00f0ffffff -> 0x00f0200000
>>>>>>> [    8.210740] rockchip-dw-pcie fe150000.pcie:      MEM 
>>>>>>> 0x0900000000..0x093fffffff -> 0x0900000000
>>>>>>> [    8.218918] rockchip-dw-pcie fe150000.pcie: iATU: unroll T, 8 
>>>>>>> ob, 8 ib, align 64K, limit 8G
>>>>>>> [    9.324473] rockchip-dw-pcie fe150000.pcie: Phy link never came up
>>>>>>> [    9.325186] rockchip-dw-pcie fe150000.pcie: PCI host bridge to 
>>>>>>> bus 0000:00
>>>>>>> [    9.325786] pci_bus 0000:00: root bus resource [bus 00-0f]
>>>>>>> [    9.326266] pci_bus 0000:00: root bus resource [io 
>>>>>>>  0x0000-0xfffff] (bus address [0xf0100000-0xf01fffff])
>>>>>>> [    9.327097] pci_bus 0000:00: root bus resource [mem 
>>>>>>> 0xf0200000-0xf0ffffff]
>>>>>>> [    9.327713] pci_bus 0000:00: root bus resource [mem 
>>>>>>> 0x900000000-0x93fffffff pref]
>>>>>>> [    9.328364] pci_bus 0000:00: scanning bus
>>>>>>> [    9.328729] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
>>>>>>> [    9.329258] pci 0000:00:00.0: reg 0x10: [mem 
>>>>>>> 0x00000000-0x3fffffff]
>>>>>>> [    9.329807] pci 0000:00:00.0: reg 0x14: [mem 
>>>>>>> 0x00000000-0x3fffffff]
>>>>>>> [    9.330354] pci 0000:00:00.0: reg 0x38: [mem 
>>>>>>> 0x00000000-0x0000ffff pref]
>>>>>>> [    9.330984] pci 0000:00:00.0: supports D1 D2
>>>>>>> [    9.331356] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
>>>>>>> [    9.331858] pci 0000:00:00.0: PME# disabled
>>>>>>> [    9.332309] pci 0000:00:00.0: vgaarb: pci_notify
>>>>>>> [    9.333735] pci_bus 0000:00: fixups for bus
>>>>>>> [    9.334106] pci 0000:00:00.0: scanning [bus 01-ff] behind 
>>>>>>> bridge, pass 0
>>>>>>> [    9.334731] pci_bus 0000:01: busn_res: can not insert [bus 
>>>>>>> 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
>>>>>>> [    9.335668] pci_bus 0000:01: scanning bus
>>>>>>> [    9.336052] pci_bus 0000:01: fixups for bus
>>>>>>> [    9.336416] pci_bus 0000:01: bus scan returning with max=01
>>>>>>> [    9.336903] pci 0000:00:00.0: scanning [bus 01-ff] behind 
>>>>>>> bridge, pass 1
>>>>>>> [    9.337503] pci_bus 0000:00: bus scan returning with max=ff
>>>>>>> [    9.337994] pci 0000:00:00.0: BAR 0: no space for [mem size 
>>>>>>> 0x40000000]
>>>>>>> [    9.338570] pci 0000:00:00.0: BAR 0: failed to assign [mem 
>>>>>>> size 0x40000000]
>>>>>>> [    9.339175] pci 0000:00:00.0: BAR 1: no space for [mem size 
>>>>>>> 0x40000000]
>>>>>>> [    9.339749] pci 0000:00:00.0: BAR 1: failed to assign [mem 
>>>>>>> size 0x40000000]
>>>>>>> [    9.340356] pci 0000:00:00.0: BAR 6: assigned [mem 
>>>>>>> 0xf0200000-0xf020ffff pref]
>>>>>>> [    9.340991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>>>>>>> and nothing more :-(
>>>>>>> Are you progressing maybe with pcie on rock5b?
>>>>>> fe150000 is the PCIE3.0 controller, on Rock5B, that's the M.2 slot.
>>>>>>
>>>>>> But for R8125, it's on a PCIE2.0 controller, which is using naneng 
>>>>>> combo phy.
>>>>>>
>>>>>> I'm not able to bring the PCIE2.0 part up yet, it always hangs at 
>>>>>> PCIE2.0 initialization, thus only the PCIE3.0 part is submitted to 
>>>>>> the list.
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>
>>>>> Yes. Indeed.
>>>>> I'm trying to add pci2.0 and it looks i meet the same problem probably.
>>>>>
>>>>> I backported (from well working neggles quartz64pro repo I assume):
>>>>> https://github.com/neggles/linux-quartz64/commit/2ad84e89fc75b82c783345b72c97f5d7e3d45723 <https://github.com/neggles/linux-quartz64/commit/2ad84e89fc75b82c783345b72c97f5d7e3d45723>
>>>>> https://github.com/neggles/linux-quartz64/commit/4ac44c2b53758eca671d695d19b456d1849d7e14
>>>>> https://github.com/neggles/linux-quartz64/commit/714c5e01d8f0f73b3a49cbdee29e3ffe0f3353dd
>>>>> https://github.com/neggles/linux-quartz64/commit/64971f9c85f27e29c44b31a0c326ea4bb8ec3c56
>>>>>
>>>>> then i added rock5b pcie dt enablements (basing on radxa BSP):
>>>>> https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.2/files/1058-arm64-dts-rockchip-enable-pcie-rock5b.patch
>>>>>
>>>>>
>>>>> this gives me quite clean 6.2 mainline boot log with hang at pcie2 
>>>>> init (pls see below)
>>>>>
>>>>> I assume 
>>>>> https://github.com/neggles/linux-quartz64/commits/linux-quartz64 
>>>>> repo works well on Quartsz64 board - so I indirectly assume above 
>>>>> backports should give us working pice2.0.
>>>>>
>>>>> It fails on rock5b - so I conclude: issue/error is on my side and 
>>>>> is related to my rock5b specifics.
>>>>> Unfortunately I don’t owe Quartz64 board so can't verify 
>>>>> correctness of my backports by testing on Quartz64pro.
>>>>>
>>>>> I’m curious about opinion of smarter than me guys…
>>>>
>>>> If the phy is misconfigured, not powered, or the clocks aren't going
>>>> active, you'll hang when the controller tries to touch it. Unless
>>>> someone has completed the combophy rk3588 bits the driver is not
>>>> functional yet for rk3588.
>>>>
>>>> Looking at the current 6.2 release, I only see the rk3568 compatible,
>>>> so you'll need to add support for rk3588 before it will work.
>>>>
>>>> Very Respectfully,
>>>> Peter Geis
>>>
>>> Sorry for being late to the game. Life is busy right now :)
>>>
>>> I haven't looked into PCIe myself so far, but some of my colleagues
>>> are looking into native network support on Rock 5B (and thus PCIe2
>>> controller).
>>>
>>> Apart from the obvious (missing rk3588 support in the combophy driver),
>>> the clocks will need some work. The clock tree implementation I 
>>> upstreamed
>>> is different from the downstream implementation. Downstream has some
>>> clocks that have two parent clocks using a hacked implementation
>>> that's obviously not upstreamable as is. The upstream implementation
>>> currently only describes the first parent. More details are in the
>>> following comment:
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/clk/rockchip/clk-rk3588.c#n16 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/clk/rockchip/clk-rk3588.c#n16>
>>>
>>> Back than I wrote that I do not understand the exact need (the TRM
>>> does not describe the clock tree unfortuantely), I found this
>>> explanation:
>>>
>>> https://lore.kernel.org/dri-devel/20220309094139198367142@rock-chips.com/
>>>
>>> To get the advanced blocks properly running upstream this needs a
>>> solution for this. Note that trying to access registers that are
>>> not clocked properly will result in a hang (as Peter already wrote).
>>
>> TLDR: There are now hardware blocks called a "Native Interface Unit"
>> (NIU) that gate the clocks to devices behind them. Essentially it
>> makes certain device clocks dependent on multiple parents being active
>> at the same time, which the current clock structure does not support.
>> It was decided that until the clock structure is updated to support
>> this, the NIU gate clocks would be marked critical and left always on.
>> Better to burn a miniscule more power than risk locking the board up.
>>
>> Though the issue here is much simpler than that, the controller simply
>> can't operate without a functional phy attached to it. They are
>> physically tied together and it's the controller < - > phy link that
>> doesn't function without the phy being configured. The combophy config
>> for rk3588 isn't significantly more complicated than rk356x, but one
>> can't mainline code that can't be tested and rk3588 wasn't available
>> at that time.
>>
>>>
>>> Apart from that the power-domain controller might need some of the
>>> extra bits downstream has.
>>
>> If PCIe 3 works, then this likely won't affect PCIe 2, though it
>> limits our ability to implement runtime power management.
>>
>>>
>>> Last but not least the GIC controller is handled differently in
>>> downstream. For that following the workaround that has been used for
>>> rk356x should also work for rk3588.
>>
>> The hack for the GIC controller that I maintain for rk356x is not
>> upstreamable. For the ITS to be supported in mainline we still require
>> an official errata from Rockchip documenting the issues here. The
>> mbi-alias method, while it works for simple configs, isn't scalable.
>>
>>>
>>> TLDR: This is not trivial. It's really unfortunate, that the board
>>> is not just using the native ethernet :(
>>>
>>> P.S.: We try to keep a rk3588 / Rock 5 status matrix maintained here:
>>>
>>> https://gitlab.collabora.com/hardware-enablement/rockchip-3588/notes-for-rockchip-3588/-/blob/main/mainline-status.md <https://gitlab.collabora.com/hardware-enablement/rockchip-3588/notes-for-rockchip-3588/-/blob/main/mainline-status.md>
>>>
>>> Greetings,
>>>
>>> — Sebastian
> 
> Qu,
> 
> Just fyi,
> 
> I got Eth finally working on rock5b, so - if 6.2 kernel with mmc 
> bootable rock5b with ths, dvfs, mmc, pcie2, pcie3 and usb2 - will be in 
> any way helpful for you: patches over 6.2.2 mainline are here:
> 
> https://github.com/warpme/minimyth2/tree/master/script/kernel/linux-6.2-with-rk3588/files <https://github.com/warpme/minimyth2/tree/master/script/kernel/linux-6.2-with-rk3588/files>
> 
> pls look on patches: 1001-xx onwards + linux-6.2-arm64-armv8.config
> 
> Code quality is far from upstreamable.
> Hope it will be convenient for any devel using rock5a/rock5b

Awesome! I can finally get rid of the stupid out-of-tree r8125 driver.

Would definitely have a good look into the patches and provide some 
feedbacks, just mind to share a git tree for easier reviewing?

Thanks,
Qu

> 
> btw: if anybody is interested - i can provide Archlinux ARM sd card 
> bootstrap image for these boards…
> btw2: this huge list of patches is because i’m building single ArchLinux 
> ARM aarch64 kernel for 13+ SoCs in 17 different boards/tvboxes


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Piotr Oniszczuk]

> Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 09.03.2023, o godz. 13:17:
> 
> 
> 
> Awesome! I can finally get rid of the stupid out-of-tree r8125 driver.
> 
> Would definitely have a good look into the patches and provide some feedbacks, just mind to share a git tree for easier reviewing?
> 
> 

Ah - git tree... my distro builder uses model with: mainline src + patches - so effectively i don’t have git tree for this code…

  
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Qu Wenruo]

On 2023/3/9 21:00, Piotr Oniszczuk wrote:
> 
> 
>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 09.03.2023, o godz. 13:17:
>>
>>
>>
>> Awesome! I can finally get rid of the stupid out-of-tree r8125 driver.
>>
>> Would definitely have a good look into the patches and provide some feedbacks, just mind to share a git tree for easier reviewing?
>>
>>
> 
> Ah - git tree... my distro builder uses model with: mainline src + patches - so effectively i don’t have git tree for this code…
> 

The ABS is fine if the number of patches is small, but when patches go 
larger it can go a little out of control.

Never mind, I can still fetch the patches and find out what's the 
difference for the PCIE2 naneng combo phy.

Just to mention, some kernel AUR also go git tree to simplify the PKGBUILD.

Thanks,
Qu

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/5] arm64: rockchip: enable PCIE3 controller and its phy for Rock5B boards

[Lucas Tanure]

On 10-03-2023 00:16, Qu Wenruo wrote:
> 
> 
> On 2023/3/9 21:00, Piotr Oniszczuk wrote:
>>
>>
>>> Wiadomość napisana przez Qu Wenruo <wqu@suse.com> w dniu 09.03.2023, 
>>> o godz. 13:17:
>>>
>>>
>>>
>>> Awesome! I can finally get rid of the stupid out-of-tree r8125 driver.
>>>
>>> Would definitely have a good look into the patches and provide some 
>>> feedbacks, just mind to share a git tree for easier reviewing?
>>>
>>>
>>
>> Ah - git tree... my distro builder uses model with: mainline src + 
>> patches - so effectively i don’t have git tree for this code…
>>
> 
> The ABS is fine if the number of patches is small, but when patches go 
> larger it can go a little out of control.
> 
> Never mind, I can still fetch the patches and find out what's the 
> difference for the PCIE2 naneng combo phy.
> 
> Just to mention, some kernel AUR also go git tree to simplify the PKGBUILD.
> 
> Thanks,
> Qu
Hi,

I just sent a patch series to support PCIe2 for RK3588s:
https://lore.kernel.org/lkml/20230310080518.78054-1-lucas.tanure@collabora.com/T/#t

The same patches are here:
https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/tree/wip_pcie2_rk3588

Thanks
Lucas

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel