[PATCH v2] riscv: dts: spacemit: k1: give each PCIe port its own DMA window

[PATCH v2] riscv: dts: spacemit: k1: give each PCIe port its own DMA window

[Vlad Banea]

On K1 a PCIe master only sees DDR: bus addresses 0-2 GB map to CPU
0-2 GB and bus 2-16 GB to CPU 4-18 GB. A root port does not forward
upstream an inbound access that falls inside its own memory window, so
dma-ranges must skip each port's own outbound window; the other ports'
windows reach memory.

The shared pcie-bus skips all three windows, bus 0x8000_0000-0xb7ff_ffff,
which leaves CPU addresses 0x1_0000_0000-0x1_37ff_ffff unreachable from
every port. Give each port its own bus that skips only its own window.
On boards with more than 2 GB of RAM, each port can then reach another
512-640 MB without SWIOTLB bouncing.

Signed-off-by: Vlad Banea <montrealtango@gmail.com>
---

Notes:
    Changes in v2:
    - describe the K1 PCIe address map instead of citing the vendor DT (Troy)
    - "more than 2 GB" instead of 4 GB (Troy)
    - drop the RFC tag
    - RFC: https://lore.kernel.org/all/20261112015148.3911178-1-montrealtango@gmail.com/
    
    Tested on an OrangePi R2S (8 GB, RTL8125B on pcie1 and pcie2): SWIOTLB
    use under load went from up to 87 slots to none.

 arch/riscv/boot/dts/spacemit/k1.dtsi | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/boot/dts/spacemit/k1.dtsi b/arch/riscv/boot/dts/spacemit/k1.dtsi
index 3fabfd34114b..949843bb7789 100644
--- a/arch/riscv/boot/dts/spacemit/k1.dtsi
+++ b/arch/riscv/boot/dts/spacemit/k1.dtsi
@@ -1152,13 +1152,14 @@ eth1: ethernet@cac81000 {
 			};
 		};
 
-		pcie-bus {
+		pcie0-bus {
 			compatible = "simple-bus";
 			ranges;
 			#address-cells = <2>;
 			#size-cells = <2>;
+			/* skips pcie0's own window, bus 0x8000_0000-0x8fff_ffff */
 			dma-ranges = <0x0 0x00000000 0x0 0x00000000 0x0 0x80000000>,
-				     <0x0 0xb8000000 0x1 0x38000000 0x3 0x48000000>;
+				     <0x0 0x90000000 0x1 0x10000000 0x3 0x70000000>;
 
 			pcie0: pcie@ca000000 {
 				device_type = "pci";
@@ -1203,6 +1204,17 @@ pcie0_port: pcie@0 {
 					ranges;
 				};
 			};
+		};
+
+		pcie1-bus {
+			compatible = "simple-bus";
+			ranges;
+			#address-cells = <2>;
+			#size-cells = <2>;
+			/* skips pcie1's own window, bus 0x9000_0000-0x9fff_ffff */
+			dma-ranges = <0x0 0x00000000 0x0 0x00000000 0x0 0x80000000>,
+				     <0x0 0x80000000 0x1 0x00000000 0x0 0x10000000>,
+				     <0x0 0xa0000000 0x1 0x20000000 0x3 0x60000000>;
 
 			pcie1: pcie@ca400000 {
 				device_type = "pci";
@@ -1247,6 +1259,17 @@ pcie1_port: pcie@0 {
 					ranges;
 				};
 			};
+		};
+
+		pcie2-bus {
+			compatible = "simple-bus";
+			ranges;
+			#address-cells = <2>;
+			#size-cells = <2>;
+			/* skips pcie2's own window, bus 0xa000_0000-0xb7ff_ffff */
+			dma-ranges = <0x0 0x00000000 0x0 0x00000000 0x0 0x80000000>,
+				     <0x0 0x80000000 0x1 0x00000000 0x0 0x20000000>,
+				     <0x0 0xb8000000 0x1 0x38000000 0x3 0x48000000>;
 
 			pcie2: pcie@ca800000 {
 				device_type = "pci";

base-commit: d05c6236496071916e19012cda7149465f15e66e
-- 
2.53.0

Re: [PATCH v2] riscv: dts: spacemit: k1: give each PCIe port its own DMA window

[Troy Mitchell]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

On Fri Nov 13, 2026 at 2:31 AM GMT, Vlad Banea wrote:
> On K1 a PCIe master only sees DDR: bus addresses 0-2 GB map to CPU
> 0-2 GB and bus 2-16 GB to CPU 4-18 GB. A root port does not forward
> upstream an inbound access that falls inside its own memory window, so
> dma-ranges must skip each port's own outbound window; the other ports'
> windows reach memory.
>
> The shared pcie-bus skips all three windows, bus 0x8000_0000-0xb7ff_ffff,
> which leaves CPU addresses 0x1_0000_0000-0x1_37ff_ffff unreachable from
> every port. Give each port its own bus that skips only its own window.
> On boards with more than 2 GB of RAM, each port can then reach another
> 512-640 MB without SWIOTLB bouncing.
>
> Signed-off-by: Vlad Banea <montrealtango@gmail.com>
Reviewed-by: Troy Mitchell <troy.mitchell@linux.spacemit.com>


-- 
Troy Mitchell


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]