Re: [RFC PATCH] riscv: dts: spacemit: k1: give each PCIe port its own DMA window

["Troy Mitchell" <troy.mitchell@linux.spacemit.com>]

[-- Attachment #1: Type: text/plain, Size: 2456 bytes --]

On Wed, 11 Nov 2026 20:51:47 -0500, Vlad Banea wrote:
> The dma-ranges of the shared pcie-bus skip bus addresses
> 0x8000_0000-0xb7ff_ffff, the union of the three ports' outbound memory
> windows. CPU addresses 0x1_0000_0000-0x1_37ff_ffff are therefore
> unreachable from all three ports.
>
> SpacemiT's vendor device tree gives each port its own map that skips
> only that port's window (dram_range2/6/7). Do the same here. On boards
> with more than 4 GB of RAM, each port can then reach another 512-640 MB
> without SWIOTLB bouncing.
This already applies above 2 GB: RAM past the first 2 GB starts at
0x1_0000_0000, so 4 GB boards hit this hole too. Please say "more
than 2 GB", give the hardware reason below instead of pointing at
the vendor tree, and drop the RFC tag.

> Signed-off-by: Vlad Banea <montrealtango@gmail.com>
> ---
>
> Notes:
>     RFC: the per-port maps come from SpacemiT's Bianbu kernel (linux-6.6,
>     k1-bl-v2.2.y, arch/riscv/boot/dts/spacemit/k1-x.dtsi). They imply that a
>     port's inbound DMA to another port's outbound window reaches memory. I
>     have not verified that on hardware. Yixun, Encrow, can SpacemiT confirm
>     that this is how the K1 routes inbound PCIe traffic?
Yes, it reaches memory. On K1 a PCIe master only sees DDR: bus 0-2G
is CPU 0-2G and bus 2G-16G is CPU 4G-18G. The 2G-4G I/O region with
the outbound windows is not reachable from PCIe.
Only the port's own window has to be skipped: it falls inside the
root port's memory window, so the root port does not forward it
upstream.

[...]
> diff --git a/arch/riscv/boot/dts/spacemit/k1.dtsi b/arch/riscv/boot/dts/spacemit/k1.dtsi
> index 3fabfd34114b..949843bb7789 100644
> --- a/arch/riscv/boot/dts/spacemit/k1.dtsi
> +++ b/arch/riscv/boot/dts/spacemit/k1.dtsi
> @@ -1152,13 +1152,14 @@ eth1: ethernet@cac81000 {
>  			};
>  		};
>
> -		pcie-bus {
> +		pcie0-bus {
>  			compatible = "simple-bus";
>  			ranges;
>  			#address-cells = <2>;
>  			#size-cells = <2>;
> +			/* skips pcie0's own window, bus 0x8000_0000-0x8fff_ffff */
>  			dma-ranges = <0x0 0x00000000 0x0 0x00000000 0x0 0x80000000>,
> -				     <0x0 0xb8000000 0x1 0x38000000 0x3 0x48000000>;
> +				     <0x0 0x90000000 0x1 0x10000000 0x3 0x70000000>;
The maps for all three ports are correct: each one skips only its
own window and maps the other two windows to DDR, which matches the
address map above.

-- 
Troy Mitchell

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]