[PATCH v2 1/3] arm64: dts: microchip: sparx5: do not use PSCI on reference boards

[PATCH v2 1/3] arm64: dts: microchip: sparx5: do not use PSCI on reference boards

[Robert Marko]

PSCI is not implemented on SparX-5 at all, there is no ATF and U-boot that
is shipped does not implement it as well.

I have tried flashing the latest BSP 2022.12 U-boot which did not work.
After contacting Microchip, they confirmed that there is no ATF for the
SoC nor PSCI implementation which is unfortunate in 2023.

So, disable PSCI as otherwise kernel crashes as soon as it tries probing
PSCI with, and the crash is only visible if earlycon is used.

Since PSCI is not implemented, switch core bringup to use spin-tables
which are implemented in the vendor U-boot and actually work.

Tested on PCB134 with eMMC (VSC5640EV).

Fixes: 6694aee00a4b ("arm64: dts: sparx5: Add basic cpu support")
Signed-off-by: Robert Marko <robert.marko@sartura.hr>
---
Changes in v2:
* As suggested by Arnd, disable PSCI only on reference boards
---
 arch/arm64/boot/dts/microchip/sparx5.dtsi            |  2 +-
 arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi | 12 ++++++++++++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/boot/dts/microchip/sparx5.dtsi b/arch/arm64/boot/dts/microchip/sparx5.dtsi
index 0367a00a269b3..5eae6e7fd248e 100644
--- a/arch/arm64/boot/dts/microchip/sparx5.dtsi
+++ b/arch/arm64/boot/dts/microchip/sparx5.dtsi
@@ -61,7 +61,7 @@ arm-pmu {
 		interrupt-affinity = <&cpu0>, <&cpu1>;
 	};
 
-	psci {
+	psci: psci {
 		compatible = "arm,psci-0.2";
 		method = "smc";
 	};
diff --git a/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi b/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi
index 9d1a082de3e29..32bb76b3202a0 100644
--- a/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi
+++ b/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi
@@ -6,6 +6,18 @@
 /dts-v1/;
 #include "sparx5.dtsi"
 
+&psci {
+	status = "disabled";
+};
+
+&cpu0 {
+	enable-method = "spin-table";
+};
+
+&cpu1 {
+	enable-method = "spin-table";
+};
+
 &uart0 {
 	status = "okay";
 };
-- 
2.39.2

[PATCH v2 2/3] arm64: dts: microchip: sparx5: correct CPU address-cells

[Robert Marko]

There is no reason for CPU node #address-cells to be set at 2, so lets
change them to 1 and update the reg property accordingly.

Signed-off-by: Robert Marko <robert.marko@sartura.hr>
---
 arch/arm64/boot/dts/microchip/sparx5.dtsi | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/boot/dts/microchip/sparx5.dtsi b/arch/arm64/boot/dts/microchip/sparx5.dtsi
index 5eae6e7fd248e..a4fabacf5c2f7 100644
--- a/arch/arm64/boot/dts/microchip/sparx5.dtsi
+++ b/arch/arm64/boot/dts/microchip/sparx5.dtsi
@@ -24,7 +24,7 @@ chosen {
 	};
 
 	cpus {
-		#address-cells = <2>;
+		#address-cells = <1>;
 		#size-cells = <0>;
 		cpu-map {
 			cluster0 {
@@ -39,14 +39,14 @@ core1 {
 		cpu0: cpu@0 {
 			compatible = "arm,cortex-a53";
 			device_type = "cpu";
-			reg = <0x0 0x0>;
+			reg = <0x0>;
 			enable-method = "psci";
 			next-level-cache = <&L2_0>;
 		};
 		cpu1: cpu@1 {
 			compatible = "arm,cortex-a53";
 			device_type = "cpu";
-			reg = <0x0 0x1>;
+			reg = <0x1>;
 			enable-method = "psci";
 			next-level-cache = <&L2_0>;
 		};
-- 
2.39.2

[PATCH v2 3/3] arm64: dts: microchip: sparx5: add missing L1/L2 cache information

[Robert Marko]

Currently, when booting on SparX-5 you will get the following error:
[    0.050132] Early cacheinfo failed, ret = -22

This is due to L2 cache node missing cache-level property to indicate its
level, so populate it to let the kernel know its L2 cache.

However, that alone is enough to get rid of the error, but then the
following warnings appear:
[    0.050162] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    0.093256] cacheinfo: Unable to detect cache hierarchy for CPU 1

So, lets completely populate both the L1 and L2 cache info based off the
SoC datasheet[1] and ARM A53 technical reference manual[2].

Now "lscpu -C" provides:
NAME ONE-SIZE ALL-SIZE WAYS TYPE        LEVEL SETS PHY-LINE COHERENCY-SIZE
L1d       32K      64K    4 Data            1  128                      64
L1i       32K      64K    2 Instruction     1  256                      64
L2       256K     256K   16 Unified         2  256                      64

Tested on PCB134 (eMMC).

[1] https://ww1.microchip.com/downloads/aemDocuments/documents/UNG/ProductDocuments/DataSheets/SparX-5_Family_L2L3_Enterprise_25G_Ethernet_Switches_Datasheet_00003823D.pdf
[2] https://developer.arm.com/documentation/ddi0500/j/Level-1-Memory-System/About-the-L1-memory-system?lang=en

Fixes: 6694aee00a4b ("arm64: dts: sparx5: Add basic cpu support")
Signed-off-by: Robert Marko <robert.marko@sartura.hr>
---
 arch/arm64/boot/dts/microchip/sparx5.dtsi | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/arch/arm64/boot/dts/microchip/sparx5.dtsi b/arch/arm64/boot/dts/microchip/sparx5.dtsi
index a4fabacf5c2f7..950ba78bf73f3 100644
--- a/arch/arm64/boot/dts/microchip/sparx5.dtsi
+++ b/arch/arm64/boot/dts/microchip/sparx5.dtsi
@@ -41,6 +41,12 @@ cpu0: cpu@0 {
 			device_type = "cpu";
 			reg = <0x0>;
 			enable-method = "psci";
+			d-cache-size = <0x8000>;
+			d-cache-line-size = <64>;
+			d-cache-sets = <128>; // 32KiB(size)/64(line-size)=512ways/4-way set
+			i-cache-size = <0x8000>;
+			i-cache-line-size = <64>;
+			i-cache-sets = <256>; // 32KiB(size)/64(line-size)=512ways/2-way set
 			next-level-cache = <&L2_0>;
 		};
 		cpu1: cpu@1 {
@@ -48,10 +54,21 @@ cpu1: cpu@1 {
 			device_type = "cpu";
 			reg = <0x1>;
 			enable-method = "psci";
+			d-cache-size = <0x8000>;
+			d-cache-line-size = <64>;
+			d-cache-sets = <128>; // 32KiB(size)/64(line-size)=512ways/4-way set
+			i-cache-size = <0x8000>;
+			i-cache-line-size = <64>;
+			i-cache-sets = <256>; // 32KiB(size)/64(line-size)=512ways/2-way set
 			next-level-cache = <&L2_0>;
 		};
 		L2_0: l2-cache0 {
 			compatible = "cache";
+			cache-unified;
+			cache-size = <0x40000>;
+			cache-line-size = <64>;
+			cache-sets = <256>; // 256KB(size)/64(line-size)=4096ways/16-way set
+			cache-level = <2>;
 		};
 	};
 
-- 
2.39.2

Re: [PATCH v2 1/3] arm64: dts: microchip: sparx5: do not use PSCI on reference boards

[Steen Hegelund]

Hi Robert,

I find this very useful since the Sparx5 SoC does not support TFA and therefore
cannot provide PSCI on its own, as you also state below.

Acked-by: Steen Hegelund <Steen.Hegelund@microchip.com>

On Tue, 2023-02-21 at 11:50 +0100, Robert Marko wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the
> content is safe
> 
> PSCI is not implemented on SparX-5 at all, there is no ATF and U-boot that
> is shipped does not implement it as well.
> 
> I have tried flashing the latest BSP 2022.12 U-boot which did not work.
> After contacting Microchip, they confirmed that there is no ATF for the
> SoC nor PSCI implementation which is unfortunate in 2023.
> 
> So, disable PSCI as otherwise kernel crashes as soon as it tries probing
> PSCI with, and the crash is only visible if earlycon is used.
> 
> Since PSCI is not implemented, switch core bringup to use spin-tables
> which are implemented in the vendor U-boot and actually work.
> 
> Tested on PCB134 with eMMC (VSC5640EV).
> 
> Fixes: 6694aee00a4b ("arm64: dts: sparx5: Add basic cpu support")
> Signed-off-by: Robert Marko <robert.marko@sartura.hr>
> ---
> Changes in v2:
> * As suggested by Arnd, disable PSCI only on reference boards
> ---
>  arch/arm64/boot/dts/microchip/sparx5.dtsi            |  2 +-
>  arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi | 12 ++++++++++++
>  2 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/boot/dts/microchip/sparx5.dtsi
> b/arch/arm64/boot/dts/microchip/sparx5.dtsi
> index 0367a00a269b3..5eae6e7fd248e 100644
> --- a/arch/arm64/boot/dts/microchip/sparx5.dtsi
> +++ b/arch/arm64/boot/dts/microchip/sparx5.dtsi
> @@ -61,7 +61,7 @@ arm-pmu {
>                 interrupt-affinity = <&cpu0>, <&cpu1>;
>         };
> 
> -       psci {
> +       psci: psci {
>                 compatible = "arm,psci-0.2";
>                 method = "smc";
>         };
> diff --git a/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi
> b/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi
> index 9d1a082de3e29..32bb76b3202a0 100644
> --- a/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi
> +++ b/arch/arm64/boot/dts/microchip/sparx5_pcb_common.dtsi
> @@ -6,6 +6,18 @@
>  /dts-v1/;
>  #include "sparx5.dtsi"
> 
> +&psci {
> +       status = "disabled";
> +};
> +
> +&cpu0 {
> +       enable-method = "spin-table";
> +};
> +
> +&cpu1 {
> +       enable-method = "spin-table";
> +};
> +
>  &uart0 {
>         status = "okay";
>  };
> --
> 2.39.2
> 


BR
Steen

Re: [PATCH v2 1/3] arm64: dts: microchip: sparx5: do not use PSCI on reference boards

[Krzysztof Kozlowski]

On 21/02/2023 11:50, Robert Marko wrote:
> PSCI is not implemented on SparX-5 at all, there is no ATF and U-boot that
> is shipped does not implement it as well.
> 
> I have tried flashing the latest BSP 2022.12 U-boot which did not work.
> After contacting Microchip, they confirmed that there is no ATF for the
> SoC nor PSCI implementation which is unfortunate in 2023.
> 
> So, disable PSCI as otherwise kernel crashes as soon as it tries probing
> PSCI with, and the crash is only visible if earlycon is used.
> 
> Since PSCI is not implemented, switch core bringup to use spin-tables
> which are implemented in the vendor U-boot and actually work.
> 
> Tested on PCB134 with eMMC (VSC5640EV).
> 
> Fixes: 6694aee00a4b ("arm64: dts: sparx5: Add basic cpu support")
> Signed-off-by: Robert Marko <robert.marko@sartura.hr>
> ---
> Changes in v2:
> * As suggested by Arnd, disable PSCI only on reference boards

Patch seemed forgotten, so I applied it.
https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux-dt.git/log/?h=next/dt64


Best regards,
Krzysztof

Re: [PATCH v2 2/3] arm64: dts: microchip: sparx5: correct CPU address-cells

[Krzysztof Kozlowski]

On 21/02/2023 11:50, Robert Marko wrote:
> There is no reason for CPU node #address-cells to be set at 2, so lets
> change them to 1 and update the reg property accordingly.
> 
> Signed-off-by: Robert Marko <robert.marko@sartura.hr>
> ---
>  arch/arm64/boot/dts/microchip/sparx5.dtsi | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 

Patch seemed forgotten, so I applied it.
https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux-dt.git/log/?h=next/dt64

Best regards,
Krzysztof

Re: [PATCH v2 3/3] arm64: dts: microchip: sparx5: add missing L1/L2 cache information

[Krzysztof Kozlowski]

On 21/02/2023 11:50, Robert Marko wrote:
> Currently, when booting on SparX-5 you will get the following error:
> [    0.050132] Early cacheinfo failed, ret = -22
> 
> This is due to L2 cache node missing cache-level property to indicate its
> level, so populate it to let the kernel know its L2 cache.
> 
> However, that alone is enough to get rid of the error, but then the
> following warnings appear:
> [    0.050162] cacheinfo: Unable to detect cache hierarchy for CPU 0
> [    0.093256] cacheinfo: Unable to detect cache hierarchy for CPU 1
> 

This did not apply, skipped.

Best regards,
Krzysztof