[PATCH 0/5] MIPS: Allow using multi-cluster with a broken HCI.

[PATCH 0/5] MIPS: Allow using multi-cluster with a broken HCI.

[Gregory CLEMENT]

Hello,

Some CM3.5 reports indicate that Hardware Cache Initialization is
complete, but in reality it's not the case. They also incorrectly show
that Hardware Cache Initialization is supported. Unfortunately, it is
not possible to detect this issue at runtime and the information has
to be passed by the device tree.

I initially proposed to add this information as a CPU property, but as
rightfully pointed out by Jiaxun, it should be more of a Coherence
Manager property. A few months ago, Jiaxun proposed a series to pass
the address of the CM through the device tree when it was not possible
to get it at runtime. This series introduced the binding for the CM
that I reused. However, there were some parts of this series that were
commented on and needed to be addressed, and I don't have the hardware
for it. Therefore, I've kept only the binding and relaxed the need for
the reg property.

My initial proposal was integrated into the series set by Aleksandar
here [1]. And the series adding the CM binding was here: [2]. The
patches 1,2,3, and 5 have no dependencies while patch 4 should depend
on this series [1]. Actually, those five patches should replace
patches 10, 11, and 12.

Gregory

[1]: https://lore.kernel.org/all/20241028175935.51250-1-arikalo@gmail.com/
[2]: https://lore.kernel.org/all/20240612-cm_probe-v2-5-a5b55440563c@flygoat.com/

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
---
Gregory CLEMENT (4):
      dt-bindings: mips: mips-cm: Add property for broken HCI information
      MIPS: cm: Detect CM quirks from device tree
      MIPS: CPS: Support broken HCI for multicluster
      MIPS: mobileye: dts: eyeq6h: Enable cluster support

Jiaxun Yang (1):
      dt-bindings: mips: Document mti,mips-cm

 .../devicetree/bindings/mips/mti,mips-cm.yaml      | 43 ++++++++++++++++++++++
 arch/mips/boot/dts/mobileye/eyeq6h.dtsi            |  5 +++
 arch/mips/include/asm/mips-cm.h                    | 22 +++++++++++
 arch/mips/kernel/mips-cm.c                         | 16 ++++++++
 arch/mips/kernel/smp-cps.c                         |  5 ++-
 5 files changed, 90 insertions(+), 1 deletion(-)
---
base-commit: e87100a0b90df5f239e6b290f7db67e16bcda85f
change-id: 20241115-cluster-hci-broken-840a78f72aae

Best regards,
-- 
Grégory CLEMENT, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH 1/5] dt-bindings: mips: Document mti,mips-cm

[Gregory CLEMENT]

From: Jiaxun Yang <jiaxun.yang@flygoat.com>

Add devicetree binding documentation for MIPS Coherence Manager.

gc: reg is no more mandatory
Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
---
 .../devicetree/bindings/mips/mti,mips-cm.yaml      | 37 ++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..03a5ba5624a429c428ee2afca73b3e29127e02f9
--- /dev/null
+++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/mips/mti,mips-cm.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: MIPS Coherence Manager
+
+description: |
+  Defines a location of the MIPS Coherence Manager registers.
+
+maintainers:
+  - Jiaxun Yang <jiaxun.yang@flygoat.com>
+
+properties:
+  compatible:
+    const: mti,mips-cm
+
+  reg:
+    description:
+      Base address and size of an unoccupied region in system's MMIO address
+      space, which will be used to map the MIPS CM global control registers
+      block. It is conventionally decided by the system integrator.
+    maxItems: 1
+
+required:
+  - compatible
+
+additionalProperties: false
+
+examples:
+  - |
+    coherency-manager@1fbf8000 {
+      compatible = "mti,mips-cm";
+      reg = <0x1bde8000 0x8000>;
+    };
+...

-- 
2.45.2

Re: [PATCH 1/5] dt-bindings: mips: Document mti,mips-cm

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]

On Fri, Nov 15, 2024 at 04:29:54PM +0100, Gregory CLEMENT wrote:
> From: Jiaxun Yang <jiaxun.yang@flygoat.com>
> 
> Add devicetree binding documentation for MIPS Coherence Manager.
> 
> gc: reg is no more mandatory

That's not enough, you need to explain somewhere why it's not required
anymore. Without a reg property, what does this even convey that cannot
be derived from a compatible etc?

> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
> ---
>  .../devicetree/bindings/mips/mti,mips-cm.yaml      | 37 ++++++++++++++++++++++
>  1 file changed, 37 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
> new file mode 100644
> index 0000000000000000000000000000000000000000..03a5ba5624a429c428ee2afca73b3e29127e02f9
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
> @@ -0,0 +1,37 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/mips/mti,mips-cm.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: MIPS Coherence Manager
> +
> +description: |
> +  Defines a location of the MIPS Coherence Manager registers.
> +
> +maintainers:
> +  - Jiaxun Yang <jiaxun.yang@flygoat.com>
> +
> +properties:
> +  compatible:
> +    const: mti,mips-cm
> +
> +  reg:
> +    description:
> +      Base address and size of an unoccupied region in system's MMIO address
> +      space, which will be used to map the MIPS CM global control registers
> +      block. It is conventionally decided by the system integrator.
> +    maxItems: 1
> +
> +required:
> +  - compatible
> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    coherency-manager@1fbf8000 {
> +      compatible = "mti,mips-cm";
> +      reg = <0x1bde8000 0x8000>;
> +    };
> +...
> 
> -- 
> 2.45.2
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH 1/5] dt-bindings: mips: Document mti,mips-cm

[Rob Herring]

On Fri, Nov 15, 2024 at 04:29:54PM +0100, Gregory CLEMENT wrote:
> From: Jiaxun Yang <jiaxun.yang@flygoat.com>
> 
> Add devicetree binding documentation for MIPS Coherence Manager.
> 
> gc: reg is no more mandatory

The h/w either has registers or it doesn't. Can't be both ways.

> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
> ---
>  .../devicetree/bindings/mips/mti,mips-cm.yaml      | 37 ++++++++++++++++++++++
>  1 file changed, 37 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
> new file mode 100644
> index 0000000000000000000000000000000000000000..03a5ba5624a429c428ee2afca73b3e29127e02f9
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
> @@ -0,0 +1,37 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/mips/mti,mips-cm.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: MIPS Coherence Manager
> +
> +description: |

Don't need '|' if no formatting to preserve.

> +  Defines a location of the MIPS Coherence Manager registers.
> +
> +maintainers:
> +  - Jiaxun Yang <jiaxun.yang@flygoat.com>
> +
> +properties:
> +  compatible:
> +    const: mti,mips-cm
> +
> +  reg:
> +    description:
> +      Base address and size of an unoccupied region in system's MMIO address
> +      space, which will be used to map the MIPS CM global control registers
> +      block. It is conventionally decided by the system integrator.
> +    maxItems: 1
> +
> +required:
> +  - compatible
> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    coherency-manager@1fbf8000 {
> +      compatible = "mti,mips-cm";
> +      reg = <0x1bde8000 0x8000>;
> +    };
> +...
> 
> -- 
> 2.45.2
> 

Re: [PATCH 1/5] dt-bindings: mips: Document mti,mips-cm

[Gregory CLEMENT]

Rob Herring <robh@kernel.org> writes:

> On Fri, Nov 15, 2024 at 04:29:54PM +0100, Gregory CLEMENT wrote:
>> From: Jiaxun Yang <jiaxun.yang@flygoat.com>
>> 
>> Add devicetree binding documentation for MIPS Coherence Manager.
>> 
>> gc: reg is no more mandatory
>
> The h/w either has registers or it doesn't. Can't be both ways.

The register addresses are retrieved dynamically on most
hardware. However, for some of them, retrieval was broken, requiring it
to pass through the device tree. This was the initial purpose behind
introducing this binding. In the eyequ6 case, we have a hardware issue
in a different way; the address registers are correct, but the Hardware
Cache Instruction is broken. As such, we do not want to provide a
register address, instead, notifying about this issue through a property
or compatible string.

That's the reason why I want to make the register optional; however, if
I use a dedicated compatible string, then I can just remove the 'reg'
property for this binding.

Gregory

>
>> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
>> Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
>> ---
>>  .../devicetree/bindings/mips/mti,mips-cm.yaml      | 37 ++++++++++++++++++++++
>>  1 file changed, 37 insertions(+)
>> 
>> diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..03a5ba5624a429c428ee2afca73b3e29127e02f9
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
>> @@ -0,0 +1,37 @@
>> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/mips/mti,mips-cm.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: MIPS Coherence Manager
>> +
>> +description: |
>
> Don't need '|' if no formatting to preserve.
>
>> +  Defines a location of the MIPS Coherence Manager registers.
>> +
>> +maintainers:
>> +  - Jiaxun Yang <jiaxun.yang@flygoat.com>
>> +
>> +properties:
>> +  compatible:
>> +    const: mti,mips-cm
>> +
>> +  reg:
>> +    description:
>> +      Base address and size of an unoccupied region in system's MMIO address
>> +      space, which will be used to map the MIPS CM global control registers
>> +      block. It is conventionally decided by the system integrator.
>> +    maxItems: 1
>> +
>> +required:
>> +  - compatible
>> +
>> +additionalProperties: false
>> +
>> +examples:
>> +  - |
>> +    coherency-manager@1fbf8000 {
>> +      compatible = "mti,mips-cm";
>> +      reg = <0x1bde8000 0x8000>;
>> +    };
>> +...
>> 
>> -- 
>> 2.45.2
>> 

-- 
Grégory CLEMENT, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH 2/5] dt-bindings: mips: mips-cm: Add property for broken HCI information

[Gregory CLEMENT]

Some CM3.5 reports show that Hardware Cache Initialization is
complete, but in reality it's not the case. They also incorrectly
indicate that Hardware Cache Initialization is supported. This
optional property allows warning about this broken feature that cannot
be detected at runtime.

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
---
 Documentation/devicetree/bindings/mips/mti,mips-cm.yaml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
index 03a5ba5624a429c428ee2afca73b3e29127e02f9..eab31a1022c8cbdee86081110516579ebe99e640 100644
--- a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
+++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
@@ -23,6 +23,12 @@ properties:
       block. It is conventionally decided by the system integrator.
     maxItems: 1
 
+  cm3-l2-config-hci-broken:
+    type: boolean
+    description:
+      If present, indicates that the HCI (Hardware Cache Initialization)
+      information for the L2 cache in multi-cluster configuration is broken.
+
 required:
   - compatible
 

-- 
2.45.2

Re: [PATCH 2/5] dt-bindings: mips: mips-cm: Add property for broken HCI information

[Rob Herring]

On Fri, Nov 15, 2024 at 04:29:55PM +0100, Gregory CLEMENT wrote:
> Some CM3.5 reports show that Hardware Cache Initialization is
> complete, but in reality it's not the case. They also incorrectly
> indicate that Hardware Cache Initialization is supported. This
> optional property allows warning about this broken feature that cannot
> be detected at runtime.
> 
> Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
> ---
>  Documentation/devicetree/bindings/mips/mti,mips-cm.yaml | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
> index 03a5ba5624a429c428ee2afca73b3e29127e02f9..eab31a1022c8cbdee86081110516579ebe99e640 100644
> --- a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
> +++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
> @@ -23,6 +23,12 @@ properties:
>        block. It is conventionally decided by the system integrator.
>      maxItems: 1
>  
> +  cm3-l2-config-hci-broken:
> +    type: boolean
> +    description:
> +      If present, indicates that the HCI (Hardware Cache Initialization)
> +      information for the L2 cache in multi-cluster configuration is broken.
> +

This should be implied by a specific compatible.

>  required:
>    - compatible
>  
> 
> -- 
> 2.45.2
> 

[PATCH 3/5] MIPS: cm: Detect CM quirks from device tree

[Gregory CLEMENT]

Some information that should be retrieved at runtime for the Coherence
Manager can be either absent or wrong. This patch allows checking if
some of this information is available from the device tree and updates
the internal variable accordingly.

For now, only the HCI broken-related property is being retrieved.

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
---
 arch/mips/include/asm/mips-cm.h | 22 ++++++++++++++++++++++
 arch/mips/kernel/mips-cm.c      | 16 ++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h
index 1afa85db1fb37d1017fbe7d6b7a2b7d2470e8257..3bfe0633b57639bfb05b7692e4bb83ba7c0b2523 100644
--- a/arch/mips/include/asm/mips-cm.h
+++ b/arch/mips/include/asm/mips-cm.h
@@ -59,6 +59,16 @@ extern phys_addr_t mips_cm_l2sync_phys_base(void);
  */
 extern int mips_cm_is64;
 
+/*
+ * mips_cm_is_l2_hci_broken  - determine if HCI is broken
+ *
+ * Some CM reports show that Hardware Cache Initialization is
+ * complete, but in reality it's not the case. They also incorrectly
+ * indicate that Hardware Cache Initialization is supported. This
+ * flags allows warning about this broken feature.
+ */
+extern bool mips_cm_is_l2_hci_broken;
+
 /**
  * mips_cm_error_report - Report CM cache errors
  */
@@ -97,6 +107,18 @@ static inline bool mips_cm_present(void)
 #endif
 }
 
+/**
+ * mips_cm_update_property - update property from the device tree
+ *
+ * Retrieve the properties from the device tree if a CM node exist and
+ * update the internal variable based on this.
+ */
+#ifdef CONFIG_MIPS_CM
+extern void mips_cm_update_property(void);
+#else
+static void mips_cm_update_property(void) {}
+#endif
+
 /**
  * mips_cm_has_l2sync - determine whether an L2-only sync region is present
  *
diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c
index 9854bc2b6895d4db67d216586f65e4810661d29b..a2010b4d54c93175b63763bd5639c12e4583f58f 100644
--- a/arch/mips/kernel/mips-cm.c
+++ b/arch/mips/kernel/mips-cm.c
@@ -5,6 +5,7 @@
  */
 
 #include <linux/errno.h>
+#include <linux/of.h>
 #include <linux/percpu.h>
 #include <linux/spinlock.h>
 
@@ -14,6 +15,7 @@
 void __iomem *mips_gcr_base;
 void __iomem *mips_cm_l2sync_base;
 int mips_cm_is64;
+bool mips_cm_is_l2_hci_broken;
 
 static char *cm2_tr[8] = {
 	"mem",	"gcr",	"gic",	"mmio",
@@ -237,6 +239,20 @@ static void mips_cm_probe_l2sync(void)
 	mips_cm_l2sync_base = ioremap(addr, MIPS_CM_L2SYNC_SIZE);
 }
 
+void mips_cm_update_property(void)
+{
+	struct device_node *cm_node;
+
+	cm_node = of_find_compatible_node(of_root, NULL, "mti,mips-cm");
+	if (!cm_node)
+		return;
+	if (of_property_read_bool(cm_node, "cm3-l2-config-hci-broken")) {
+		pr_info("HCI (Hardware Cache Init for the L2 cache) in GCR_L2_RAM_CONFIG from the CM3 is broken");
+		mips_cm_is_l2_hci_broken = true;
+	}
+	of_node_put(cm_node);
+}
+
 int mips_cm_probe(void)
 {
 	phys_addr_t addr;

-- 
2.45.2

[PATCH 4/5] MIPS: CPS: Support broken HCI for multicluster

[Gregory CLEMENT]

Some CM3.5 devices incorrectly report that hardware cache
initialization has completed, and also claim to support hardware cache
initialization when they don't actually do so. This commit fixes this
issue by retrieving the correct information from the device tree and
allowing the system to bypass the hardware cache initialization
step. Instead, it relies on manual operation. As a result, multi-user
support is now possible for these CPUs.

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
---
 arch/mips/kernel/smp-cps.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c
index b20ea4048429e1aab2bffbada793ee594bee1e05..e85bd087467e8caf0640ad247ee5f8eb65107591 100644
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -333,6 +333,9 @@ static void __init cps_prepare_cpus(unsigned int max_cpus)
 					   sizeof(*mips_cps_cluster_bootcfg),
 					   GFP_KERNEL);
 
+	if (nclusters > 1)
+		mips_cm_update_property();
+
 	for (cl = 0; cl < nclusters; cl++) {
 		/* Allocate core boot configuration structs */
 		ncores = mips_cps_numcores(cl);
@@ -394,7 +397,7 @@ static void init_cluster_l2(void)
 {
 	u32 l2_cfg, l2sm_cop, result;
 
-	while (1) {
+	while (!mips_cm_is_l2_hci_broken) {
 		l2_cfg = read_gcr_redir_l2_ram_config();
 
 		/* If HCI is not supported, use the state machine below */

-- 
2.45.2

[PATCH 5/5] MIPS: mobileye: dts: eyeq6h: Enable cluster support

[Gregory CLEMENT]

The CM3.5 device used in EyeQ6H SoCs incorrectly reports the status
for Hardware Cache Initialization (HCI). This commit adds a property
to acknowledge this issue, which enables the use of the second CPU
cluster.

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
---
 arch/mips/boot/dts/mobileye/eyeq6h.dtsi | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/mips/boot/dts/mobileye/eyeq6h.dtsi b/arch/mips/boot/dts/mobileye/eyeq6h.dtsi
index 1db3c3cda2e395025075387bcb66ea0737fd37f6..0195b5e5227c60031607a1707f152910ab610d4c 100644
--- a/arch/mips/boot/dts/mobileye/eyeq6h.dtsi
+++ b/arch/mips/boot/dts/mobileye/eyeq6h.dtsi
@@ -32,6 +32,11 @@ cpu_intc: interrupt-controller {
 		#interrupt-cells = <1>;
 	};
 
+	coherency-manager {
+		compatible = "mti,mips-cm";
+		cm3-l2-config-hci-broken;
+	};
+
 	soc: soc {
 		compatible = "simple-bus";
 		#address-cells = <2>;

-- 
2.45.2