[PATCH 0/4] mailbox: qcom-apcs-ipc: Avoid circular dependency with clock controller

[PATCH 0/4] mailbox: qcom-apcs-ipc: Avoid circular dependency with clock controller

[Stephan Gerhold]

The APCS "global" node in the device tree currently combines two distinct
use cases in a single device tree node: a mailbox to communicate with other
remoteprocs in the system, and a clock for controlling the CPU frequency.

These two use cases have unavoidable circular dependencies: the mailbox is
needed as early as possible during boot to start controlling shared
resources like clocks and power domains, while the clock controller needs
one of these shared clocks as its parent. Currently, there is no way to
distinguish these two use cases for generic mechanisms like fw_devlink.

Break up the circular dependency chain in the device tree by separating the
clock controller into a separate child node.

The patches in this series should be merged together in one tree to avoid
potential bisect problems. Given the majority of the changes is in the
mailbox subsystem and the QC clock drivers only have trivial 1-line
changes, I propose merging all of these through the mailbox subsystem.

@Bjorn: If this sounds good to you, could you provide an Acked-by for the
two "clk: qcom:" patches?

Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
---
Stephan Gerhold (4):
      dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller
      mailbox: qcom-apcs-ipc: Assign OF node to clock controller child device
      clk: qcom: apcs-msm8916: Obtain clock from own OF node
      clk: qcom: apcs-sdx55: Obtain clock from own OF node

 .../bindings/mailbox/qcom,apcs-kpss-global.yaml    | 169 ++++++++++++++-------
 drivers/clk/qcom/apcs-msm8916.c                    |   2 +-
 drivers/clk/qcom/apcs-sdx55.c                      |   2 +-
 drivers/mailbox/qcom-apcs-ipc-mailbox.c            |  16 +-
 4 files changed, 132 insertions(+), 57 deletions(-)
---
base-commit: 0af2f6be1b4281385b618cb86ad946eded089ac8
change-id: 20250505-qcom-apcs-mailbox-cc-6e292bb6d40e

Best regards,
-- 
Stephan Gerhold <stephan.gerhold@linaro.org>

[PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephan Gerhold]

APCS "global" is sort of a "miscellaneous" hardware block that combines
multiple registers inside the application processor subsystem. Two distinct
use cases are currently stuffed together in a single device tree node:

 - Mailbox: to communicate with other remoteprocs in the system.
 - Clock: for controlling the CPU frequency.

These two use cases have unavoidable circular dependencies: the mailbox is
needed as early as possible during boot to start controlling shared
resources like clocks and power domains, while the clock controller needs
one of these shared clocks as its parent. Currently, there is no way to
distinguish these two use cases for generic mechanisms like fw_devlink.

This is currently blocking conversion of the deprecated custom "qcom,ipc"
properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9
("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"):
  1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
  2. The clock controller inside &apcs1_mbox needs
     clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
  3. &rpmcc is a child of remoteproc &rpm

The mailbox itself does not need any clocks and should probe early to
unblock the rest of the boot process. The "clocks" are only needed for the
separate clock controller. In Linux, these are already two separate drivers
that can probe independently.

Break up the circular dependency chain in the device tree by separating the
clock controller into a separate child node. Deprecate the old approach of
specifying the clock properties as part of the root node, but keep them for
backwards compatibility.

Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
---
 .../bindings/mailbox/qcom,apcs-kpss-global.yaml    | 169 ++++++++++++++-------
 1 file changed, 118 insertions(+), 51 deletions(-)

diff --git a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml
index a58a018f3f7b9f8edd70d7c1bd137844ff2549df..3a0a304bb65a68b2d4a1df79b3243ddac6bf88b2 100644
--- a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml
+++ b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml
@@ -72,6 +72,7 @@ properties:
     description: phandles to the parent clocks of the clock driver
     minItems: 2
     maxItems: 3
+    deprecated: true
 
   '#mbox-cells':
     const: 1
@@ -82,6 +83,23 @@ properties:
   clock-names:
     minItems: 2
     maxItems: 3
+    deprecated: true
+
+  clock-controller:
+    type: object
+    additionalProperties: false
+    properties:
+      clocks:
+        description: phandles to the parent clocks of the clock driver
+        minItems: 2
+        maxItems: 3
+
+      '#clock-cells':
+        enum: [0, 1]
+
+      clock-names:
+        minItems: 2
+        maxItems: 3
 
 required:
   - compatible
@@ -90,6 +108,76 @@ required:
 
 additionalProperties: false
 
+# Clocks should be specified either on the parent node or on the child node
+oneOf:
+  - required:
+      - clock-controller
+    properties:
+      clocks: false
+      clock-names: false
+      '#clock-cells': false
+  - properties:
+      clock-controller: false
+
+$defs:
+  msm8916-apcs-clock-controller:
+    properties:
+      clocks:
+        items:
+          - description: primary pll parent of the clock driver
+          - description: auxiliary parent
+      clock-names:
+        items:
+          - const: pll
+          - const: aux
+      '#clock-cells':
+        const: 0
+
+  msm8939-apcs-clock-controller:
+    properties:
+      clocks:
+        items:
+          - description: primary pll parent of the clock driver
+          - description: auxiliary parent
+          - description: reference clock
+      clock-names:
+        items:
+          - const: pll
+          - const: aux
+          - const: ref
+      '#clock-cells':
+        const: 0
+
+  sdx55-apcs-clock-controller:
+    properties:
+      clocks:
+        items:
+          - description: reference clock
+          - description: primary pll parent of the clock driver
+          - description: auxiliary parent
+      clock-names:
+        items:
+          - const: ref
+          - const: pll
+          - const: aux
+      '#clock-cells':
+        const: 0
+
+  ipq6018-apcs-clock-controller:
+    properties:
+      clocks:
+        items:
+          - description: primary pll parent of the clock driver
+          - description: XO clock
+          - description: GCC GPLL0 clock source
+      clock-names:
+        items:
+          - const: pll
+          - const: xo
+          - const: gpll0
+      '#clock-cells':
+        const: 1
+
 allOf:
   - if:
       properties:
@@ -98,15 +186,10 @@ allOf:
             enum:
               - qcom,msm8916-apcs-kpss-global
     then:
+      $ref: "#/$defs/msm8916-apcs-clock-controller"
       properties:
-        clocks:
-          items:
-            - description: primary pll parent of the clock driver
-            - description: auxiliary parent
-        clock-names:
-          items:
-            - const: pll
-            - const: aux
+        clock-controller:
+          $ref: "#/$defs/msm8916-apcs-clock-controller"
 
   - if:
       properties:
@@ -115,17 +198,10 @@ allOf:
             enum:
               - qcom,msm8939-apcs-kpss-global
     then:
+      $ref: "#/$defs/msm8939-apcs-clock-controller"
       properties:
-        clocks:
-          items:
-            - description: primary pll parent of the clock driver
-            - description: auxiliary parent
-            - description: reference clock
-        clock-names:
-          items:
-            - const: pll
-            - const: aux
-            - const: ref
+        clock-controller:
+          $ref: "#/$defs/msm8939-apcs-clock-controller"
 
   - if:
       properties:
@@ -134,17 +210,10 @@ allOf:
             enum:
               - qcom,sdx55-apcs-gcc
     then:
+      $ref: "#/$defs/sdx55-apcs-clock-controller"
       properties:
-        clocks:
-          items:
-            - description: reference clock
-            - description: primary pll parent of the clock driver
-            - description: auxiliary parent
-        clock-names:
-          items:
-            - const: ref
-            - const: pll
-            - const: aux
+        clock-controller:
+          $ref: "#/$defs/sdx55-apcs-clock-controller"
 
   - if:
       properties:
@@ -153,17 +222,10 @@ allOf:
             enum:
               - qcom,ipq6018-apcs-apps-global
     then:
+      $ref: "#/$defs/ipq6018-apcs-clock-controller"
       properties:
-        clocks:
-          items:
-            - description: primary pll parent of the clock driver
-            - description: XO clock
-            - description: GCC GPLL0 clock source
-        clock-names:
-          items:
-            - const: pll
-            - const: xo
-            - const: gpll0
+        clock-controller:
+          $ref: "#/$defs/ipq6018-apcs-clock-controller"
 
   - if:
       properties:
@@ -179,19 +241,7 @@ allOf:
       properties:
         clocks: false
         clock-names: false
-
-  - if:
-      properties:
-        compatible:
-          contains:
-            enum:
-              - qcom,ipq6018-apcs-apps-global
-    then:
-      properties:
-        '#clock-cells':
-          const: 1
-    else:
-      properties:
+        clock-controller: false
         '#clock-cells':
           const: 0
 
@@ -216,6 +266,23 @@ examples:
     };
 
   # Example apcs with qcs404
+  - |
+    #define GCC_APSS_AHB_CLK_SRC  1
+    #define GCC_GPLL0_AO_OUT_MAIN 123
+    mailbox@b011000 {
+        compatible = "qcom,qcs404-apcs-apps-global",
+                     "qcom,msm8916-apcs-kpss-global", "syscon";
+        reg = <0x0b011000 0x1000>;
+        #mbox-cells = <1>;
+
+        apcs_clk: clock-controller {
+          clocks = <&apcs_hfpll>, <&gcc GCC_GPLL0_AO_OUT_MAIN>;
+          clock-names = "pll", "aux";
+          #clock-cells = <0>;
+        };
+    };
+
+  # Example apcs with qcs404 (deprecated: use clock-controller subnode)
   - |
     #define GCC_APSS_AHB_CLK_SRC  1
     #define GCC_GPLL0_AO_OUT_MAIN 123

-- 
2.47.2

[PATCH 2/4] mailbox: qcom-apcs-ipc: Assign OF node to clock controller child device

[Stephan Gerhold]

Currently, the child device for the clock controller inside the APCS block
is created without any OF node assigned, so the drivers need to rely on the
parent device for obtaining any resources.

Add support for defining the clock controller inside a "clock-controller"
subnode to break up circular dependencies between the mailbox and required
parent clocks of the clock controller. For backwards compatibility, if the
subnode is not defined, reuse the OF node from the parent device.

Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
---
 drivers/mailbox/qcom-apcs-ipc-mailbox.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/mailbox/qcom-apcs-ipc-mailbox.c b/drivers/mailbox/qcom-apcs-ipc-mailbox.c
index 11c41e935a3619b74ad0f5e2d82699ca8aa05722..8b24ec0fa191efc975625d9b9270140ad1fe7b9b 100644
--- a/drivers/mailbox/qcom-apcs-ipc-mailbox.c
+++ b/drivers/mailbox/qcom-apcs-ipc-mailbox.c
@@ -116,10 +116,18 @@ static int qcom_apcs_ipc_probe(struct platform_device *pdev)
 	}
 
 	if (apcs_data->clk_name) {
-		apcs->clk = platform_device_register_data(&pdev->dev,
-							  apcs_data->clk_name,
-							  PLATFORM_DEVID_AUTO,
-							  NULL, 0);
+		struct device_node *np = of_get_child_by_name(pdev->dev.of_node,
+							      "clock-controller");
+		struct platform_device_info pdevinfo = {
+			.parent = &pdev->dev,
+			.name = apcs_data->clk_name,
+			.id = PLATFORM_DEVID_AUTO,
+			.fwnode = of_fwnode_handle(np) ?: pdev->dev.fwnode,
+			.of_node_reused = !np,
+		};
+
+		apcs->clk = platform_device_register_full(&pdevinfo);
+		of_node_put(np);
 		if (IS_ERR(apcs->clk))
 			dev_err(&pdev->dev, "failed to register APCS clk\n");
 	}

-- 
2.47.2

[PATCH 3/4] clk: qcom: apcs-msm8916: Obtain clock from own OF node

[Stephan Gerhold]

So far we had to obtain all resources like clocks from the parent device,
because the qcom-apcs-msm8916-clk platform device did not have an own OF
node assigned. Now that the parent mailbox driver assigns this, obtain the
resources directly from the assigned OF node to add support for describing
the clock controller in a separate child node. This allows breaking up
circular dependencies between the mailbox and the clock controller.

Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
---
@Bjorn: If this looks good to you and you are fine with merging this
through the mailbox subsystem, could you provide an Acked-by here?
---
 drivers/clk/qcom/apcs-msm8916.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/clk/qcom/apcs-msm8916.c b/drivers/clk/qcom/apcs-msm8916.c
index ef31386831ebd2404b99edaeff4c95a31eb68477..af60f3ab1f82068e5ab046ee4a2231c7fb85ff41 100644
--- a/drivers/clk/qcom/apcs-msm8916.c
+++ b/drivers/clk/qcom/apcs-msm8916.c
@@ -82,7 +82,7 @@ static int qcom_apcs_msm8916_clk_probe(struct platform_device *pdev)
 	a53cc->src_shift = 8;
 	a53cc->parent_map = gpll0_a53cc_map;
 
-	a53cc->pclk = devm_clk_get(parent, NULL);
+	a53cc->pclk = devm_clk_get(dev, NULL);
 	if (IS_ERR(a53cc->pclk)) {
 		ret = PTR_ERR(a53cc->pclk);
 		if (ret != -EPROBE_DEFER)

-- 
2.47.2

[PATCH 4/4] clk: qcom: apcs-sdx55: Obtain clock from own OF node

[Stephan Gerhold]

So far we had to obtain all resources like clocks from the parent device,
because the qcom-sdx55-acps-clk platform device did not have an own OF node
assigned. Now that the parent mailbox driver assigns this, obtain the
resources directly from the assigned OF node to add support for describing
the clock controller in a separate child node. This allows breaking up
circular dependencies between the mailbox and the clock controller.

Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
---
@Bjorn: If this looks good to you and you are fine with merging this
through the mailbox subsystem, could you provide an Acked-by here?
---
 drivers/clk/qcom/apcs-sdx55.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/clk/qcom/apcs-sdx55.c b/drivers/clk/qcom/apcs-sdx55.c
index 76ece6c4a969266aecb32806269c791712f98804..e0e779a81f2c6b8b9cd34d839eee9aaf9fa33f03 100644
--- a/drivers/clk/qcom/apcs-sdx55.c
+++ b/drivers/clk/qcom/apcs-sdx55.c
@@ -79,7 +79,7 @@ static int qcom_apcs_sdx55_clk_probe(struct platform_device *pdev)
 	a7cc->src_shift = 8;
 	a7cc->parent_map = apcs_mux_clk_parent_map;
 
-	a7cc->pclk = devm_clk_get(parent, "pll");
+	a7cc->pclk = devm_clk_get(dev, "pll");
 	if (IS_ERR(a7cc->pclk))
 		return dev_err_probe(dev, PTR_ERR(a7cc->pclk),
 				     "Failed to get PLL clk\n");

-- 
2.47.2

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Bjorn Andersson]

On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote:
> APCS "global" is sort of a "miscellaneous" hardware block that combines
> multiple registers inside the application processor subsystem. Two distinct
> use cases are currently stuffed together in a single device tree node:
> 
>  - Mailbox: to communicate with other remoteprocs in the system.
>  - Clock: for controlling the CPU frequency.
> 
> These two use cases have unavoidable circular dependencies: the mailbox is
> needed as early as possible during boot to start controlling shared
> resources like clocks and power domains, while the clock controller needs
> one of these shared clocks as its parent. Currently, there is no way to
> distinguish these two use cases for generic mechanisms like fw_devlink.
> 
> This is currently blocking conversion of the deprecated custom "qcom,ipc"
> properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9
> ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"):
>   1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
>   2. The clock controller inside &apcs1_mbox needs
>      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
>   3. &rpmcc is a child of remoteproc &rpm
> 
> The mailbox itself does not need any clocks and should probe early to
> unblock the rest of the boot process. The "clocks" are only needed for the
> separate clock controller. In Linux, these are already two separate drivers
> that can probe independently.
> 

Why does this circular dependency need to be broken in the DeviceTree
representation?

As you describe, the mailbox probes and register the mailbox controller
and it registers the clock controller. The mailbox device isn't affected
by the clock controller failing to find rpmcc...

Regards,
Bjorn

> Break up the circular dependency chain in the device tree by separating the
> clock controller into a separate child node. Deprecate the old approach of
> specifying the clock properties as part of the root node, but keep them for
> backwards compatibility.
> 
> Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
> ---
>  .../bindings/mailbox/qcom,apcs-kpss-global.yaml    | 169 ++++++++++++++-------
>  1 file changed, 118 insertions(+), 51 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml
> index a58a018f3f7b9f8edd70d7c1bd137844ff2549df..3a0a304bb65a68b2d4a1df79b3243ddac6bf88b2 100644
> --- a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml
> +++ b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml
> @@ -72,6 +72,7 @@ properties:
>      description: phandles to the parent clocks of the clock driver
>      minItems: 2
>      maxItems: 3
> +    deprecated: true
>  
>    '#mbox-cells':
>      const: 1
> @@ -82,6 +83,23 @@ properties:
>    clock-names:
>      minItems: 2
>      maxItems: 3
> +    deprecated: true
> +
> +  clock-controller:
> +    type: object
> +    additionalProperties: false
> +    properties:
> +      clocks:
> +        description: phandles to the parent clocks of the clock driver
> +        minItems: 2
> +        maxItems: 3
> +
> +      '#clock-cells':
> +        enum: [0, 1]
> +
> +      clock-names:
> +        minItems: 2
> +        maxItems: 3
>  
>  required:
>    - compatible
> @@ -90,6 +108,76 @@ required:
>  
>  additionalProperties: false
>  
> +# Clocks should be specified either on the parent node or on the child node
> +oneOf:
> +  - required:
> +      - clock-controller
> +    properties:
> +      clocks: false
> +      clock-names: false
> +      '#clock-cells': false
> +  - properties:
> +      clock-controller: false
> +
> +$defs:
> +  msm8916-apcs-clock-controller:
> +    properties:
> +      clocks:
> +        items:
> +          - description: primary pll parent of the clock driver
> +          - description: auxiliary parent
> +      clock-names:
> +        items:
> +          - const: pll
> +          - const: aux
> +      '#clock-cells':
> +        const: 0
> +
> +  msm8939-apcs-clock-controller:
> +    properties:
> +      clocks:
> +        items:
> +          - description: primary pll parent of the clock driver
> +          - description: auxiliary parent
> +          - description: reference clock
> +      clock-names:
> +        items:
> +          - const: pll
> +          - const: aux
> +          - const: ref
> +      '#clock-cells':
> +        const: 0
> +
> +  sdx55-apcs-clock-controller:
> +    properties:
> +      clocks:
> +        items:
> +          - description: reference clock
> +          - description: primary pll parent of the clock driver
> +          - description: auxiliary parent
> +      clock-names:
> +        items:
> +          - const: ref
> +          - const: pll
> +          - const: aux
> +      '#clock-cells':
> +        const: 0
> +
> +  ipq6018-apcs-clock-controller:
> +    properties:
> +      clocks:
> +        items:
> +          - description: primary pll parent of the clock driver
> +          - description: XO clock
> +          - description: GCC GPLL0 clock source
> +      clock-names:
> +        items:
> +          - const: pll
> +          - const: xo
> +          - const: gpll0
> +      '#clock-cells':
> +        const: 1
> +
>  allOf:
>    - if:
>        properties:
> @@ -98,15 +186,10 @@ allOf:
>              enum:
>                - qcom,msm8916-apcs-kpss-global
>      then:
> +      $ref: "#/$defs/msm8916-apcs-clock-controller"
>        properties:
> -        clocks:
> -          items:
> -            - description: primary pll parent of the clock driver
> -            - description: auxiliary parent
> -        clock-names:
> -          items:
> -            - const: pll
> -            - const: aux
> +        clock-controller:
> +          $ref: "#/$defs/msm8916-apcs-clock-controller"
>  
>    - if:
>        properties:
> @@ -115,17 +198,10 @@ allOf:
>              enum:
>                - qcom,msm8939-apcs-kpss-global
>      then:
> +      $ref: "#/$defs/msm8939-apcs-clock-controller"
>        properties:
> -        clocks:
> -          items:
> -            - description: primary pll parent of the clock driver
> -            - description: auxiliary parent
> -            - description: reference clock
> -        clock-names:
> -          items:
> -            - const: pll
> -            - const: aux
> -            - const: ref
> +        clock-controller:
> +          $ref: "#/$defs/msm8939-apcs-clock-controller"
>  
>    - if:
>        properties:
> @@ -134,17 +210,10 @@ allOf:
>              enum:
>                - qcom,sdx55-apcs-gcc
>      then:
> +      $ref: "#/$defs/sdx55-apcs-clock-controller"
>        properties:
> -        clocks:
> -          items:
> -            - description: reference clock
> -            - description: primary pll parent of the clock driver
> -            - description: auxiliary parent
> -        clock-names:
> -          items:
> -            - const: ref
> -            - const: pll
> -            - const: aux
> +        clock-controller:
> +          $ref: "#/$defs/sdx55-apcs-clock-controller"
>  
>    - if:
>        properties:
> @@ -153,17 +222,10 @@ allOf:
>              enum:
>                - qcom,ipq6018-apcs-apps-global
>      then:
> +      $ref: "#/$defs/ipq6018-apcs-clock-controller"
>        properties:
> -        clocks:
> -          items:
> -            - description: primary pll parent of the clock driver
> -            - description: XO clock
> -            - description: GCC GPLL0 clock source
> -        clock-names:
> -          items:
> -            - const: pll
> -            - const: xo
> -            - const: gpll0
> +        clock-controller:
> +          $ref: "#/$defs/ipq6018-apcs-clock-controller"
>  
>    - if:
>        properties:
> @@ -179,19 +241,7 @@ allOf:
>        properties:
>          clocks: false
>          clock-names: false
> -
> -  - if:
> -      properties:
> -        compatible:
> -          contains:
> -            enum:
> -              - qcom,ipq6018-apcs-apps-global
> -    then:
> -      properties:
> -        '#clock-cells':
> -          const: 1
> -    else:
> -      properties:
> +        clock-controller: false
>          '#clock-cells':
>            const: 0
>  
> @@ -216,6 +266,23 @@ examples:
>      };
>  
>    # Example apcs with qcs404
> +  - |
> +    #define GCC_APSS_AHB_CLK_SRC  1
> +    #define GCC_GPLL0_AO_OUT_MAIN 123
> +    mailbox@b011000 {
> +        compatible = "qcom,qcs404-apcs-apps-global",
> +                     "qcom,msm8916-apcs-kpss-global", "syscon";
> +        reg = <0x0b011000 0x1000>;
> +        #mbox-cells = <1>;
> +
> +        apcs_clk: clock-controller {
> +          clocks = <&apcs_hfpll>, <&gcc GCC_GPLL0_AO_OUT_MAIN>;
> +          clock-names = "pll", "aux";
> +          #clock-cells = <0>;
> +        };
> +    };
> +
> +  # Example apcs with qcs404 (deprecated: use clock-controller subnode)
>    - |
>      #define GCC_APSS_AHB_CLK_SRC  1
>      #define GCC_GPLL0_AO_OUT_MAIN 123
> 
> -- 
> 2.47.2
> 

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephan Gerhold]

On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote:
> On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote:
> > APCS "global" is sort of a "miscellaneous" hardware block that combines
> > multiple registers inside the application processor subsystem. Two distinct
> > use cases are currently stuffed together in a single device tree node:
> > 
> >  - Mailbox: to communicate with other remoteprocs in the system.
> >  - Clock: for controlling the CPU frequency.
> > 
> > These two use cases have unavoidable circular dependencies: the mailbox is
> > needed as early as possible during boot to start controlling shared
> > resources like clocks and power domains, while the clock controller needs
> > one of these shared clocks as its parent. Currently, there is no way to
> > distinguish these two use cases for generic mechanisms like fw_devlink.
> > 
> > This is currently blocking conversion of the deprecated custom "qcom,ipc"
> > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9
> > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"):
> >   1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
> >   2. The clock controller inside &apcs1_mbox needs
> >      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
> >   3. &rpmcc is a child of remoteproc &rpm
> > 
> > The mailbox itself does not need any clocks and should probe early to
> > unblock the rest of the boot process. The "clocks" are only needed for the
> > separate clock controller. In Linux, these are already two separate drivers
> > that can probe independently.
> > 
> 
> Why does this circular dependency need to be broken in the DeviceTree
> representation?
> 
> As you describe, the mailbox probes and register the mailbox controller
> and it registers the clock controller. The mailbox device isn't affected
> by the clock controller failing to find rpmcc...
> 

That's right, but the problem is that the probe() function of the
mailbox driver won't be called at all. The device tree *looks* like the
mailbox depends on the clock, so fw_devlink tries to defer probing until
the clock is probed (which won't ever happen, because the mailbox is
needed to make the clock available).

I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
them anyway, but fact is that we need to split this up in order to avoid
warnings and have the supplies/consumers set up properly. Those device
links are created based on the device tree and not the drivers.

Thanks,
Stephan

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Rob Herring]

On Tue, May 13, 2025 at 02:16:59PM +0100, Stephan Gerhold wrote:
> On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote:
> > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote:
> > > APCS "global" is sort of a "miscellaneous" hardware block that combines
> > > multiple registers inside the application processor subsystem. Two distinct
> > > use cases are currently stuffed together in a single device tree node:
> > > 
> > >  - Mailbox: to communicate with other remoteprocs in the system.
> > >  - Clock: for controlling the CPU frequency.
> > > 
> > > These two use cases have unavoidable circular dependencies: the mailbox is
> > > needed as early as possible during boot to start controlling shared
> > > resources like clocks and power domains, while the clock controller needs
> > > one of these shared clocks as its parent. Currently, there is no way to
> > > distinguish these two use cases for generic mechanisms like fw_devlink.
> > > 
> > > This is currently blocking conversion of the deprecated custom "qcom,ipc"
> > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9
> > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"):
> > >   1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
> > >   2. The clock controller inside &apcs1_mbox needs
> > >      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
> > >   3. &rpmcc is a child of remoteproc &rpm
> > > 
> > > The mailbox itself does not need any clocks and should probe early to
> > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > separate clock controller. In Linux, these are already two separate drivers
> > > that can probe independently.
> > > 
> > 
> > Why does this circular dependency need to be broken in the DeviceTree
> > representation?
> > 
> > As you describe, the mailbox probes and register the mailbox controller
> > and it registers the clock controller. The mailbox device isn't affected
> > by the clock controller failing to find rpmcc...
> > 
> 
> That's right, but the problem is that the probe() function of the
> mailbox driver won't be called at all. The device tree *looks* like the
> mailbox depends on the clock, so fw_devlink tries to defer probing until
> the clock is probed (which won't ever happen, because the mailbox is
> needed to make the clock available).
> 
> I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> them anyway, but fact is that we need to split this up in order to avoid
> warnings and have the supplies/consumers set up properly. Those device
> links are created based on the device tree and not the drivers.

Does "post-init-providers" providers solve your problem?

Rob

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephan Gerhold]

On Wed, May 14, 2025 at 11:08:41AM -0500, Rob Herring wrote:
> On Tue, May 13, 2025 at 02:16:59PM +0100, Stephan Gerhold wrote:
> > On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote:
> > > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote:
> > > > APCS "global" is sort of a "miscellaneous" hardware block that combines
> > > > multiple registers inside the application processor subsystem. Two distinct
> > > > use cases are currently stuffed together in a single device tree node:
> > > > 
> > > >  - Mailbox: to communicate with other remoteprocs in the system.
> > > >  - Clock: for controlling the CPU frequency.
> > > > 
> > > > These two use cases have unavoidable circular dependencies: the mailbox is
> > > > needed as early as possible during boot to start controlling shared
> > > > resources like clocks and power domains, while the clock controller needs
> > > > one of these shared clocks as its parent. Currently, there is no way to
> > > > distinguish these two use cases for generic mechanisms like fw_devlink.
> > > > 
> > > > This is currently blocking conversion of the deprecated custom "qcom,ipc"
> > > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9
> > > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"):
> > > >   1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
> > > >   2. The clock controller inside &apcs1_mbox needs
> > > >      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
> > > >   3. &rpmcc is a child of remoteproc &rpm
> > > > 
> > > > The mailbox itself does not need any clocks and should probe early to
> > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > separate clock controller. In Linux, these are already two separate drivers
> > > > that can probe independently.
> > > > 
> > > 
> > > Why does this circular dependency need to be broken in the DeviceTree
> > > representation?
> > > 
> > > As you describe, the mailbox probes and register the mailbox controller
> > > and it registers the clock controller. The mailbox device isn't affected
> > > by the clock controller failing to find rpmcc...
> > > 
> > 
> > That's right, but the problem is that the probe() function of the
> > mailbox driver won't be called at all. The device tree *looks* like the
> > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > the clock is probed (which won't ever happen, because the mailbox is
> > needed to make the clock available).
> > 
> > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > them anyway, but fact is that we need to split this up in order to avoid
> > warnings and have the supplies/consumers set up properly. Those device
> > links are created based on the device tree and not the drivers.
> 
> Does "post-init-providers" providers solve your problem?
> 

I would expect that it does, but it feels like the wrong solution to the
problem to me. The clock is not really a post-init provider: It's not
consumed at all by the mailbox and needed immediately to initialize the
clock controller. The real problem in my opinion is that we're
describing two essentially distinct devices/drivers in a single device
node, and there is no way to distinguish that.

By splitting up the two distinct components into separate device tree
nodes, the relation between the providers/consumers is clearly
described.

Thanks,
Stephan

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Krzysztof Kozlowski]

On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
> > > > > The mailbox itself does not need any clocks and should probe early to

... so probe it early.

> > > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > > separate clock controller. In Linux, these are already two separate drivers
> > > > > that can probe independently.

They can probe later, no problem and DT does not stop that. Linux, not
DT, controls the ways of probing of devices and their children.

> > > > > 
> > > > 
> > > > Why does this circular dependency need to be broken in the DeviceTree
> > > > representation?
> > > > 
> > > > As you describe, the mailbox probes and register the mailbox controller
> > > > and it registers the clock controller. The mailbox device isn't affected
> > > > by the clock controller failing to find rpmcc...
> > > > 
> > > 
> > > That's right, but the problem is that the probe() function of the
> > > mailbox driver won't be called at all. The device tree *looks* like the
> > > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > > the clock is probed (which won't ever happen, because the mailbox is
> > > needed to make the clock available).
> > > 
> > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > > them anyway, but fact is that we need to split this up in order to avoid
> > > warnings and have the supplies/consumers set up properly. Those device
> > > links are created based on the device tree and not the drivers.
> > 
> > Does "post-init-providers" providers solve your problem?
> > 
> 
> I would expect that it does, but it feels like the wrong solution to the
> problem to me. The clock is not really a post-init provider: It's not
> consumed at all by the mailbox and needed immediately to initialize the
> clock controller. The real problem in my opinion is that we're
> describing two essentially distinct devices/drivers in a single device
> node, and there is no way to distinguish that.
> 
> By splitting up the two distinct components into separate device tree
> nodes, the relation between the providers/consumers is clearly
> described.

You can split devices without splitting the nodes. I do not see reason
why the DT is the problem here.

Best regards,
Krzysztof

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephan Gerhold]

+Saravana

On Wed, May 21, 2025 at 11:20:40AM +0200, Krzysztof Kozlowski wrote:
> On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
> > > > > > The mailbox itself does not need any clocks and should probe early to
> 
> ... so probe it early.
> 
> > > > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > > > separate clock controller. In Linux, these are already two separate drivers
> > > > > > that can probe independently.
> 
> They can probe later, no problem and DT does not stop that. Linux, not
> DT, controls the ways of probing of devices and their children.
> 
> > > > > > 
> > > > > 
> > > > > Why does this circular dependency need to be broken in the DeviceTree
> > > > > representation?
> > > > > 
> > > > > As you describe, the mailbox probes and register the mailbox controller
> > > > > and it registers the clock controller. The mailbox device isn't affected
> > > > > by the clock controller failing to find rpmcc...
> > > > > 
> > > > 
> > > > That's right, but the problem is that the probe() function of the
> > > > mailbox driver won't be called at all. The device tree *looks* like the
> > > > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > > > the clock is probed (which won't ever happen, because the mailbox is
> > > > needed to make the clock available).
> > > > 
> > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > > > them anyway, but fact is that we need to split this up in order to avoid
> > > > warnings and have the supplies/consumers set up properly. Those device
> > > > links are created based on the device tree and not the drivers.
> > > 
> > > Does "post-init-providers" providers solve your problem?
> > > 
> > 
> > I would expect that it does, but it feels like the wrong solution to the
> > problem to me. The clock is not really a post-init provider: It's not
> > consumed at all by the mailbox and needed immediately to initialize the
> > clock controller. The real problem in my opinion is that we're
> > describing two essentially distinct devices/drivers in a single device
> > node, and there is no way to distinguish that.
> > 
> > By splitting up the two distinct components into separate device tree
> > nodes, the relation between the providers/consumers is clearly
> > described.
> 
> You can split devices without splitting the nodes. I do not see reason
> why the DT is the problem here.
> 

The Linux drivers for this particular mailbox/clock controller already
work exactly the way you propose. They are split into two devices that
can probe independently.

The problem is outside of the drivers, because fw_devlink in Linux
blocks probing until all resources specified in the device tree nodes
become available. fw_devlink has no knowledge that the mailbox described
by this peculiar device tree node does not actually need the clocks:

	apcs1_mbox: mailbox@b011000 {
		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
		reg = <0x0b011000 0x1000>;
		#mbox-cells = <1>;
		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
		clock-names = "pll", "aux", "ref";
		#clock-cells = <0>;
	};

Without device-specific quirks in fw_devlink, the fact that these clocks
are only used by an unrelated clock controller only becomes clear if we
split the device tree node like I propose in this series:

	apcs1_mbox: mailbox@b011000 {
		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
		reg = <0x0b011000 0x1000>;
		#mbox-cells = <1>;

		apcs1_clk: clock-controller {
			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
			clock-names = "pll", "aux", "ref";
			#clock-cells = <0>;
		};
	};

It is easy to say that the problem is in Linux (and not the DT), but
unless you are suggesting to remove fw_devlink from Linux, or to add
more device-specific quirks to the generic fw_devlink code, I'm only
aware of the following two options to make this work (both already
discussed in this email thread):

 1. post-init-providers (as suggested by Rob):

		post-init-providers = <&a53pll_c1>, <&gcc>, <&rpmcc>;

    To repeat my previous email: IMHO this is a crude workaround for
    this situation. The clock is not really a post-init provider: It's
    not consumed at all by the mailbox and needed immediately to
    initialize the clock controller.

    With this approach, there are no device links created for the
    clocks, so we don't get the proper probe/suspend ordering that
    fw_devlink normally provides.

 2. Split up device tree node (this patch series): With this approach,
    the mailbox can probe early and the clock controller child device
    gets the expected consumer/supplier device links to the clocks. IMHO
    this is the cleanest solution to go for.

@Saravana: Is there any other option that I missed? Or perhaps you have
any other suggestions how we should handle this?

To summarize the series and previous emails, the dependency cycle that
was in msm8939.dtsi before commit d92e9ea2f0f9 ("arm64: dts: qcom:
msm8939: revert use of APCS mbox for RPM") is:

  1. The clock controller inside &apcs1_mbox needs
     clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
  2. &rpmcc is a child of remoteproc &rpm
  3. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;

This is not a real dependency cycle, the clocks in the mailbox@ node are
not needed for the mailbox. They are only used and needed for the clock
controller child device that makes use of the same device tree node.

At runtime this cycle currently results in none of the devices probing:

[   13.281637] platform remoteproc: deferred probe pending: qcom-rpm-proc: Failed to register smd-edge
[   13.296257] platform b011000.mailbox: deferred probe pending: platform: supplier b016000.clock not ready
[   13.308397] platform b016000.clock: deferred probe pending: platform: wait for supplier /remoteproc/smd-edge/rpm-requests/clock-controller

Thanks,
Stephan

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Krzysztof Kozlowski]

On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
> On Wed, May 14, 2025 at 11:08:41AM -0500, Rob Herring wrote:
> > On Tue, May 13, 2025 at 02:16:59PM +0100, Stephan Gerhold wrote:
> > > On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote:
> > > > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote:
> > > > > APCS "global" is sort of a "miscellaneous" hardware block that combines
> > > > > multiple registers inside the application processor subsystem. Two distinct
> > > > > use cases are currently stuffed together in a single device tree node:
> > > > > 
> > > > >  - Mailbox: to communicate with other remoteprocs in the system.
> > > > >  - Clock: for controlling the CPU frequency.
> > > > > 
> > > > > These two use cases have unavoidable circular dependencies: the mailbox is
> > > > > needed as early as possible during boot to start controlling shared
> > > > > resources like clocks and power domains, while the clock controller needs
> > > > > one of these shared clocks as its parent. Currently, there is no way to
> > > > > distinguish these two use cases for generic mechanisms like fw_devlink.
> > > > > 
> > > > > This is currently blocking conversion of the deprecated custom "qcom,ipc"
> > > > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9
> > > > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"):
> > > > >   1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
> > > > >   2. The clock controller inside &apcs1_mbox needs
> > > > >      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
> > > > >   3. &rpmcc is a child of remoteproc &rpm
> > > > > 
> > > > > The mailbox itself does not need any clocks and should probe early to
> > > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > > separate clock controller. In Linux, these are already two separate drivers
> > > > > that can probe independently.
> > > > > 
> > > > 
> > > > Why does this circular dependency need to be broken in the DeviceTree
> > > > representation?
> > > > 
> > > > As you describe, the mailbox probes and register the mailbox controller
> > > > and it registers the clock controller. The mailbox device isn't affected
> > > > by the clock controller failing to find rpmcc...
> > > > 
> > > 
> > > That's right, but the problem is that the probe() function of the
> > > mailbox driver won't be called at all. The device tree *looks* like the
> > > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > > the clock is probed (which won't ever happen, because the mailbox is
> > > needed to make the clock available).
> > > 
> > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > > them anyway, but fact is that we need to split this up in order to avoid
> > > warnings and have the supplies/consumers set up properly. Those device
> > > links are created based on the device tree and not the drivers.
> > 
> > Does "post-init-providers" providers solve your problem?
> > 
> 
> I would expect that it does, but it feels like the wrong solution to the
> problem to me. The clock is not really a post-init provider: It's not

But the entire node (so the mbox containing clocks) is a provider. This
looks exactly like the case for post-init or do I miss here something.
First, I do not see circular dependencies in the DT.

Best regards,
Krzysztof

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephan Gerhold]

On Fri, May 23, 2025 at 10:42:01AM +0200, Krzysztof Kozlowski wrote:
> On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
> > On Wed, May 14, 2025 at 11:08:41AM -0500, Rob Herring wrote:
> > > On Tue, May 13, 2025 at 02:16:59PM +0100, Stephan Gerhold wrote:
> > > > On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote:
> > > > > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote:
> > > > > > APCS "global" is sort of a "miscellaneous" hardware block that combines
> > > > > > multiple registers inside the application processor subsystem. Two distinct
> > > > > > use cases are currently stuffed together in a single device tree node:
> > > > > > 
> > > > > >  - Mailbox: to communicate with other remoteprocs in the system.
> > > > > >  - Clock: for controlling the CPU frequency.
> > > > > > 
> > > > > > These two use cases have unavoidable circular dependencies: the mailbox is
> > > > > > needed as early as possible during boot to start controlling shared
> > > > > > resources like clocks and power domains, while the clock controller needs
> > > > > > one of these shared clocks as its parent. Currently, there is no way to
> > > > > > distinguish these two use cases for generic mechanisms like fw_devlink.
> > > > > > 
> > > > > > This is currently blocking conversion of the deprecated custom "qcom,ipc"
> > > > > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9
> > > > > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"):
> > > > > >   1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
> > > > > >   2. The clock controller inside &apcs1_mbox needs
> > > > > >      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
> > > > > >   3. &rpmcc is a child of remoteproc &rpm
> > > > > > 
> > > > > > The mailbox itself does not need any clocks and should probe early to
> > > > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > > > separate clock controller. In Linux, these are already two separate drivers
> > > > > > that can probe independently.
> > > > > > 
> > > > > 
> > > > > Why does this circular dependency need to be broken in the DeviceTree
> > > > > representation?
> > > > > 
> > > > > As you describe, the mailbox probes and register the mailbox controller
> > > > > and it registers the clock controller. The mailbox device isn't affected
> > > > > by the clock controller failing to find rpmcc...
> > > > > 
> > > > 
> > > > That's right, but the problem is that the probe() function of the
> > > > mailbox driver won't be called at all. The device tree *looks* like the
> > > > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > > > the clock is probed (which won't ever happen, because the mailbox is
> > > > needed to make the clock available).
> > > > 
> > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > > > them anyway, but fact is that we need to split this up in order to avoid
> > > > warnings and have the supplies/consumers set up properly. Those device
> > > > links are created based on the device tree and not the drivers.
> > > 
> > > Does "post-init-providers" providers solve your problem?
> > > 
> > 
> > I would expect that it does, but it feels like the wrong solution to the
> > problem to me. The clock is not really a post-init provider: It's not
> 
> But the entire node (so the mbox containing clocks) is a provider. This
> looks exactly like the case for post-init or do I miss here something.
> First, I do not see circular dependencies in the DT.
> 

Please see my reply from yesterday, I have explained the disadvantages
of using post-init-provider there and also showed the problematic
circular dependency again [1].

Thanks,
Stephan

[1]: https://lore.kernel.org/linux-arm-msm/aC-AqDa8cjq2AYeM@linaro.org/

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Krzysztof Kozlowski]

On Thu, May 22, 2025 at 09:53:12PM GMT, Stephan Gerhold wrote:
> +Saravana
> 
> On Wed, May 21, 2025 at 11:20:40AM +0200, Krzysztof Kozlowski wrote:
> > On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
> > > > > > > The mailbox itself does not need any clocks and should probe early to
> > 
> > ... so probe it early.
> > 
> > > > > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > > > > separate clock controller. In Linux, these are already two separate drivers
> > > > > > > that can probe independently.
> > 
> > They can probe later, no problem and DT does not stop that. Linux, not
> > DT, controls the ways of probing of devices and their children.
> > 
> > > > > > > 
> > > > > > 
> > > > > > Why does this circular dependency need to be broken in the DeviceTree
> > > > > > representation?
> > > > > > 
> > > > > > As you describe, the mailbox probes and register the mailbox controller
> > > > > > and it registers the clock controller. The mailbox device isn't affected
> > > > > > by the clock controller failing to find rpmcc...
> > > > > > 
> > > > > 
> > > > > That's right, but the problem is that the probe() function of the
> > > > > mailbox driver won't be called at all. The device tree *looks* like the
> > > > > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > > > > the clock is probed (which won't ever happen, because the mailbox is
> > > > > needed to make the clock available).
> > > > > 
> > > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > > > > them anyway, but fact is that we need to split this up in order to avoid
> > > > > warnings and have the supplies/consumers set up properly. Those device
> > > > > links are created based on the device tree and not the drivers.
> > > > 
> > > > Does "post-init-providers" providers solve your problem?
> > > > 
> > > 
> > > I would expect that it does, but it feels like the wrong solution to the
> > > problem to me. The clock is not really a post-init provider: It's not
> > > consumed at all by the mailbox and needed immediately to initialize the
> > > clock controller. The real problem in my opinion is that we're
> > > describing two essentially distinct devices/drivers in a single device
> > > node, and there is no way to distinguish that.
> > > 
> > > By splitting up the two distinct components into separate device tree
> > > nodes, the relation between the providers/consumers is clearly
> > > described.
> > 
> > You can split devices without splitting the nodes. I do not see reason
> > why the DT is the problem here.
> > 
> 
> The Linux drivers for this particular mailbox/clock controller already
> work exactly the way you propose. They are split into two devices that
> can probe independently.
> 
> The problem is outside of the drivers, because fw_devlink in Linux
> blocks probing until all resources specified in the device tree nodes
> become available. fw_devlink has no knowledge that the mailbox described
> by this peculiar device tree node does not actually need the clocks:
> 
> 	apcs1_mbox: mailbox@b011000 {
> 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> 		reg = <0x0b011000 0x1000>;
> 		#mbox-cells = <1>;
> 		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> 		clock-names = "pll", "aux", "ref";
> 		#clock-cells = <0>;
> 	};
> 
> Without device-specific quirks in fw_devlink, the fact that these clocks
> are only used by an unrelated clock controller only becomes clear if we
> split the device tree node like I propose in this series:
> 
> 	apcs1_mbox: mailbox@b011000 {
> 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> 		reg = <0x0b011000 0x1000>;
> 		#mbox-cells = <1>;
> 
> 		apcs1_clk: clock-controller {
> 			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> 			clock-names = "pll", "aux", "ref";
> 			#clock-cells = <0>;
> 		};
> 	};

Above code suggests that clocks are not needed for the mailbox at all.
You need to be really sure of that. If that's the case, then this
description looks like correct hardware description, more detailed then
the first case, though.

> 
> It is easy to say that the problem is in Linux (and not the DT), but
> unless you are suggesting to remove fw_devlink from Linux, or to add
> more device-specific quirks to the generic fw_devlink code, I'm only
> aware of the following two options to make this work (both already
> discussed in this email thread):
> 
>  1. post-init-providers (as suggested by Rob):
> 
> 		post-init-providers = <&a53pll_c1>, <&gcc>, <&rpmcc>;
> 
>     To repeat my previous email: IMHO this is a crude workaround for
>     this situation. The clock is not really a post-init provider: It's
>     not consumed at all by the mailbox and needed immediately to
>     initialize the clock controller.

Clock is a provider - it has clock-cells - and it is post-init, because
mailbox (parent) does not need it to initialize itself. Only part of its
functionality - clocks - need it.

> 
>     With this approach, there are no device links created for the
>     clocks, so we don't get the proper probe/suspend ordering that
>     fw_devlink normally provides.

This probably could be fixed in the Linux?

> 
>  2. Split up device tree node (this patch series): With this approach,
>     the mailbox can probe early and the clock controller child device
>     gets the expected consumer/supplier device links to the clocks. IMHO
>     this is the cleanest solution to go for.
> 
> @Saravana: Is there any other option that I missed? Or perhaps you have
> any other suggestions how we should handle this?
> 
> To summarize the series and previous emails, the dependency cycle that
> was in msm8939.dtsi before commit d92e9ea2f0f9 ("arm64: dts: qcom:
> msm8939: revert use of APCS mbox for RPM") is:
> 
>   1. The clock controller inside &apcs1_mbox needs
>      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
>   2. &rpmcc is a child of remoteproc &rpm
>   3. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
> 
> This is not a real dependency cycle, the clocks in the mailbox@ node are
> not needed for the mailbox. They are only used and needed for the clock
> controller child device that makes use of the same device tree node.
> 
> At runtime this cycle currently results in none of the devices probing:
> 
> [   13.281637] platform remoteproc: deferred probe pending: qcom-rpm-proc: Failed to register smd-edge
> [   13.296257] platform b011000.mailbox: deferred probe pending: platform: supplier b016000.clock not ready
> [   13.308397] platform b016000.clock: deferred probe pending: platform: wait for supplier /remoteproc/smd-edge/rpm-requests/clock-controller
> 

Except missing dev_links you mentioned, this feels for me like job for
post-init-providers (option 1), but I am also fine with option 2,
because it fees like more appropriate hardware description.

Best regards,
Krzysztof

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Krzysztof Kozlowski]

On Tue, May 06, 2025 at 03:10:08PM GMT, Stephan Gerhold wrote:
> APCS "global" is sort of a "miscellaneous" hardware block that combines
> multiple registers inside the application processor subsystem. Two distinct
> use cases are currently stuffed together in a single device tree node:
> 
>  - Mailbox: to communicate with other remoteprocs in the system.
>  - Clock: for controlling the CPU frequency.

Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

Best regards,
Krzysztof

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephan Gerhold]

On Fri, May 23, 2025 at 11:06:04AM +0200, Krzysztof Kozlowski wrote:
> On Thu, May 22, 2025 at 09:53:12PM GMT, Stephan Gerhold wrote:
> > +Saravana
> > 
> > On Wed, May 21, 2025 at 11:20:40AM +0200, Krzysztof Kozlowski wrote:
> > > On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
> > > > > > > > The mailbox itself does not need any clocks and should probe early to
> > > 
> > > ... so probe it early.
> > > 
> > > > > > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > > > > > separate clock controller. In Linux, these are already two separate drivers
> > > > > > > > that can probe independently.
> > > 
> > > They can probe later, no problem and DT does not stop that. Linux, not
> > > DT, controls the ways of probing of devices and their children.
> > > 
> > > > > > > > 
> > > > > > > 
> > > > > > > Why does this circular dependency need to be broken in the DeviceTree
> > > > > > > representation?
> > > > > > > 
> > > > > > > As you describe, the mailbox probes and register the mailbox controller
> > > > > > > and it registers the clock controller. The mailbox device isn't affected
> > > > > > > by the clock controller failing to find rpmcc...
> > > > > > > 
> > > > > > 
> > > > > > That's right, but the problem is that the probe() function of the
> > > > > > mailbox driver won't be called at all. The device tree *looks* like the
> > > > > > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > > > > > the clock is probed (which won't ever happen, because the mailbox is
> > > > > > needed to make the clock available).
> > > > > > 
> > > > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > > > > > them anyway, but fact is that we need to split this up in order to avoid
> > > > > > warnings and have the supplies/consumers set up properly. Those device
> > > > > > links are created based on the device tree and not the drivers.
> > > > > 
> > > > > Does "post-init-providers" providers solve your problem?
> > > > > 
> > > > 
> > > > I would expect that it does, but it feels like the wrong solution to the
> > > > problem to me. The clock is not really a post-init provider: It's not
> > > > consumed at all by the mailbox and needed immediately to initialize the
> > > > clock controller. The real problem in my opinion is that we're
> > > > describing two essentially distinct devices/drivers in a single device
> > > > node, and there is no way to distinguish that.
> > > > 
> > > > By splitting up the two distinct components into separate device tree
> > > > nodes, the relation between the providers/consumers is clearly
> > > > described.
> > > 
> > > You can split devices without splitting the nodes. I do not see reason
> > > why the DT is the problem here.
> > > 
> > 
> > The Linux drivers for this particular mailbox/clock controller already
> > work exactly the way you propose. They are split into two devices that
> > can probe independently.
> > 
> > The problem is outside of the drivers, because fw_devlink in Linux
> > blocks probing until all resources specified in the device tree nodes
> > become available. fw_devlink has no knowledge that the mailbox described
> > by this peculiar device tree node does not actually need the clocks:
> > 
> > 	apcs1_mbox: mailbox@b011000 {
> > 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> > 		reg = <0x0b011000 0x1000>;
> > 		#mbox-cells = <1>;
> > 		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> > 		clock-names = "pll", "aux", "ref";
> > 		#clock-cells = <0>;
> > 	};
> > 
> > Without device-specific quirks in fw_devlink, the fact that these clocks
> > are only used by an unrelated clock controller only becomes clear if we
> > split the device tree node like I propose in this series:
> > 
> > 	apcs1_mbox: mailbox@b011000 {
> > 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> > 		reg = <0x0b011000 0x1000>;
> > 		#mbox-cells = <1>;
> > 
> > 		apcs1_clk: clock-controller {
> > 			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> > 			clock-names = "pll", "aux", "ref";
> > 			#clock-cells = <0>;
> > 		};
> > 	};
> 
> Above code suggests that clocks are not needed for the mailbox at all.
> You need to be really sure of that. If that's the case, then this
> description looks like correct hardware description, more detailed then
> the first case, though.
> 

Yes, these clocks are not needed for the mailbox.

> > 
> > It is easy to say that the problem is in Linux (and not the DT), but
> > unless you are suggesting to remove fw_devlink from Linux, or to add
> > more device-specific quirks to the generic fw_devlink code, I'm only
> > aware of the following two options to make this work (both already
> > discussed in this email thread):
> > 
> >  1. post-init-providers (as suggested by Rob):
> > 
> > 		post-init-providers = <&a53pll_c1>, <&gcc>, <&rpmcc>;
> > 
> >     To repeat my previous email: IMHO this is a crude workaround for
> >     this situation. The clock is not really a post-init provider: It's
> >     not consumed at all by the mailbox and needed immediately to
> >     initialize the clock controller.
> 
> Clock is a provider - it has clock-cells - and it is post-init, because
> mailbox (parent) does not need it to initialize itself. Only part of its
> functionality - clocks - need it.
> 

Right.

> > 
> >     With this approach, there are no device links created for the
> >     clocks, so we don't get the proper probe/suspend ordering that
> >     fw_devlink normally provides.
> 
> This probably could be fixed in the Linux?
> 

This is probably something for Saravana to answer, but I think
post-init-providers is more intended to break up actual circular
dependencies, where you want to omit the device links for one of the
sides. For this specific case this is not necessary, because we can
avoid creating the cycle by describing the hardware more accurately.

> > 
> >  2. Split up device tree node (this patch series): With this approach,
> >     the mailbox can probe early and the clock controller child device
> >     gets the expected consumer/supplier device links to the clocks. IMHO
> >     this is the cleanest solution to go for.
> > 
> > @Saravana: Is there any other option that I missed? Or perhaps you have
> > any other suggestions how we should handle this?
> > 
> > To summarize the series and previous emails, the dependency cycle that
> > was in msm8939.dtsi before commit d92e9ea2f0f9 ("arm64: dts: qcom:
> > msm8939: revert use of APCS mbox for RPM") is:
> > 
> >   1. The clock controller inside &apcs1_mbox needs
> >      clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>.
> >   2. &rpmcc is a child of remoteproc &rpm
> >   3. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>;
> > 
> > This is not a real dependency cycle, the clocks in the mailbox@ node are
> > not needed for the mailbox. They are only used and needed for the clock
> > controller child device that makes use of the same device tree node.
> > 
> > At runtime this cycle currently results in none of the devices probing:
> > 
> > [   13.281637] platform remoteproc: deferred probe pending: qcom-rpm-proc: Failed to register smd-edge
> > [   13.296257] platform b011000.mailbox: deferred probe pending: platform: supplier b016000.clock not ready
> > [   13.308397] platform b016000.clock: deferred probe pending: platform: wait for supplier /remoteproc/smd-edge/rpm-requests/clock-controller
> > 
> 
> Except missing dev_links you mentioned, this feels for me like job for
> post-init-providers (option 1), but I am also fine with option 2,
> because it fees like more appropriate hardware description.
> 

Great, thanks for the review!

Stephan

Re: [PATCH 2/4] mailbox: qcom-apcs-ipc: Assign OF node to clock controller child device

[Jassi Brar]

On Tue, May 6, 2025 at 8:10 AM Stephan Gerhold
<stephan.gerhold@linaro.org> wrote:
>
> Currently, the child device for the clock controller inside the APCS block
> is created without any OF node assigned, so the drivers need to rely on the
> parent device for obtaining any resources.
>
> Add support for defining the clock controller inside a "clock-controller"
> subnode to break up circular dependencies between the mailbox and required
> parent clocks of the clock controller. For backwards compatibility, if the
> subnode is not defined, reuse the OF node from the parent device.
>
> Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
> ---
>  drivers/mailbox/qcom-apcs-ipc-mailbox.c | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/mailbox/qcom-apcs-ipc-mailbox.c b/drivers/mailbox/qcom-apcs-ipc-mailbox.c
> index 11c41e935a3619b74ad0f5e2d82699ca8aa05722..8b24ec0fa191efc975625d9b9270140ad1fe7b9b 100644
> --- a/drivers/mailbox/qcom-apcs-ipc-mailbox.c
> +++ b/drivers/mailbox/qcom-apcs-ipc-mailbox.c
> @@ -116,10 +116,18 @@ static int qcom_apcs_ipc_probe(struct platform_device *pdev)
>         }
>
>         if (apcs_data->clk_name) {
> -               apcs->clk = platform_device_register_data(&pdev->dev,
> -                                                         apcs_data->clk_name,
> -                                                         PLATFORM_DEVID_AUTO,
> -                                                         NULL, 0);
> +               struct device_node *np = of_get_child_by_name(pdev->dev.of_node,
> +                                                             "clock-controller");
> +               struct platform_device_info pdevinfo = {
> +                       .parent = &pdev->dev,
> +                       .name = apcs_data->clk_name,
> +                       .id = PLATFORM_DEVID_AUTO,
> +                       .fwnode = of_fwnode_handle(np) ?: pdev->dev.fwnode,
> +                       .of_node_reused = !np,
> +               };
> +
> +               apcs->clk = platform_device_register_full(&pdevinfo);
> +               of_node_put(np);
>                 if (IS_ERR(apcs->clk))
>                         dev_err(&pdev->dev, "failed to register APCS clk\n");
>         }
>

I see the dt change is acked by the DT maintainer. I have no problem
with this patch and can merge, but do you want to wait for ack from
some Qcom dev?

thanks

Re: [PATCH 2/4] mailbox: qcom-apcs-ipc: Assign OF node to clock controller child device

[Dmitry Baryshkov]

On Tue, May 06, 2025 at 03:10:09PM +0200, Stephan Gerhold wrote:
> Currently, the child device for the clock controller inside the APCS block
> is created without any OF node assigned, so the drivers need to rely on the
> parent device for obtaining any resources.
> 
> Add support for defining the clock controller inside a "clock-controller"
> subnode to break up circular dependencies between the mailbox and required
> parent clocks of the clock controller. For backwards compatibility, if the
> subnode is not defined, reuse the OF node from the parent device.
> 
> Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org>
> ---
>  drivers/mailbox/qcom-apcs-ipc-mailbox.c | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
> 

Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>

-- 
With best wishes
Dmitry

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Bjorn Andersson]

On Fri, May 23, 2025 at 11:06:04AM +0200, Krzysztof Kozlowski wrote:
> On Thu, May 22, 2025 at 09:53:12PM GMT, Stephan Gerhold wrote:
> > +Saravana
> > 
> > On Wed, May 21, 2025 at 11:20:40AM +0200, Krzysztof Kozlowski wrote:
> > > On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
> > > > > > > > The mailbox itself does not need any clocks and should probe early to
> > > 
> > > ... so probe it early.
> > > 
> > > > > > > > unblock the rest of the boot process. The "clocks" are only needed for the
> > > > > > > > separate clock controller. In Linux, these are already two separate drivers
> > > > > > > > that can probe independently.
> > > 
> > > They can probe later, no problem and DT does not stop that. Linux, not
> > > DT, controls the ways of probing of devices and their children.
> > > 
> > > > > > > > 
> > > > > > > 
> > > > > > > Why does this circular dependency need to be broken in the DeviceTree
> > > > > > > representation?
> > > > > > > 
> > > > > > > As you describe, the mailbox probes and register the mailbox controller
> > > > > > > and it registers the clock controller. The mailbox device isn't affected
> > > > > > > by the clock controller failing to find rpmcc...
> > > > > > > 
> > > > > > 
> > > > > > That's right, but the problem is that the probe() function of the
> > > > > > mailbox driver won't be called at all. The device tree *looks* like the
> > > > > > mailbox depends on the clock, so fw_devlink tries to defer probing until
> > > > > > the clock is probed (which won't ever happen, because the mailbox is
> > > > > > needed to make the clock available).
> > > > > > 
> > > > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
> > > > > > them anyway, but fact is that we need to split this up in order to avoid
> > > > > > warnings and have the supplies/consumers set up properly. Those device
> > > > > > links are created based on the device tree and not the drivers.
> > > > > 
> > > > > Does "post-init-providers" providers solve your problem?
> > > > > 
> > > > 
> > > > I would expect that it does, but it feels like the wrong solution to the
> > > > problem to me. The clock is not really a post-init provider: It's not
> > > > consumed at all by the mailbox and needed immediately to initialize the
> > > > clock controller. The real problem in my opinion is that we're
> > > > describing two essentially distinct devices/drivers in a single device
> > > > node, and there is no way to distinguish that.
> > > > 
> > > > By splitting up the two distinct components into separate device tree
> > > > nodes, the relation between the providers/consumers is clearly
> > > > described.
> > > 
> > > You can split devices without splitting the nodes. I do not see reason
> > > why the DT is the problem here.
> > > 
> > 
> > The Linux drivers for this particular mailbox/clock controller already
> > work exactly the way you propose. They are split into two devices that
> > can probe independently.
> > 
> > The problem is outside of the drivers, because fw_devlink in Linux
> > blocks probing until all resources specified in the device tree nodes
> > become available. fw_devlink has no knowledge that the mailbox described
> > by this peculiar device tree node does not actually need the clocks:
> > 
> > 	apcs1_mbox: mailbox@b011000 {
> > 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> > 		reg = <0x0b011000 0x1000>;
> > 		#mbox-cells = <1>;
> > 		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> > 		clock-names = "pll", "aux", "ref";
> > 		#clock-cells = <0>;
> > 	};
> > 
> > Without device-specific quirks in fw_devlink, the fact that these clocks
> > are only used by an unrelated clock controller only becomes clear if we
> > split the device tree node like I propose in this series:
> > 
> > 	apcs1_mbox: mailbox@b011000 {
> > 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> > 		reg = <0x0b011000 0x1000>;
> > 		#mbox-cells = <1>;
> > 
> > 		apcs1_clk: clock-controller {
> > 			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> > 			clock-names = "pll", "aux", "ref";
> > 			#clock-cells = <0>;
> > 		};
> > 	};
> 
> Above code suggests that clocks are not needed for the mailbox at all.
> You need to be really sure of that. If that's the case, then this
> description looks like correct hardware description, more detailed then
> the first case, though.
> 

I'm still sceptical here.

In the first snippet above, we describe a single IP block which provides
mailboxes and clocks.

In the second snippet we're saying that the IP block is a mailbox, and
then it somehow have a subcomponent which is a clock provider.

It seems to me that we're choosing the second option because it better
fits the Linux implementation, rather than that it would be a better
representation of the hardware. To the point that we can't even describe
the register range of the subcomponent...


Can you confirm that this is the path we want to go here?

Regards,
Bjorn

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Krzysztof Kozlowski]

On 11/06/2025 05:31, Bjorn Andersson wrote:
> On Fri, May 23, 2025 at 11:06:04AM +0200, Krzysztof Kozlowski wrote:
>> On Thu, May 22, 2025 at 09:53:12PM GMT, Stephan Gerhold wrote:
>>> +Saravana
>>>
>>> On Wed, May 21, 2025 at 11:20:40AM +0200, Krzysztof Kozlowski wrote:
>>>> On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote:
>>>>>>>>> The mailbox itself does not need any clocks and should probe early to
>>>>
>>>> ... so probe it early.
>>>>
>>>>>>>>> unblock the rest of the boot process. The "clocks" are only needed for the
>>>>>>>>> separate clock controller. In Linux, these are already two separate drivers
>>>>>>>>> that can probe independently.
>>>>
>>>> They can probe later, no problem and DT does not stop that. Linux, not
>>>> DT, controls the ways of probing of devices and their children.
>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> Why does this circular dependency need to be broken in the DeviceTree
>>>>>>>> representation?
>>>>>>>>
>>>>>>>> As you describe, the mailbox probes and register the mailbox controller
>>>>>>>> and it registers the clock controller. The mailbox device isn't affected
>>>>>>>> by the clock controller failing to find rpmcc...
>>>>>>>>
>>>>>>>
>>>>>>> That's right, but the problem is that the probe() function of the
>>>>>>> mailbox driver won't be called at all. The device tree *looks* like the
>>>>>>> mailbox depends on the clock, so fw_devlink tries to defer probing until
>>>>>>> the clock is probed (which won't ever happen, because the mailbox is
>>>>>>> needed to make the clock available).
>>>>>>>
>>>>>>> I'm not sure why fw_devlink doesn't detect this cycle and tries to probe
>>>>>>> them anyway, but fact is that we need to split this up in order to avoid
>>>>>>> warnings and have the supplies/consumers set up properly. Those device
>>>>>>> links are created based on the device tree and not the drivers.
>>>>>>
>>>>>> Does "post-init-providers" providers solve your problem?
>>>>>>
>>>>>
>>>>> I would expect that it does, but it feels like the wrong solution to the
>>>>> problem to me. The clock is not really a post-init provider: It's not
>>>>> consumed at all by the mailbox and needed immediately to initialize the
>>>>> clock controller. The real problem in my opinion is that we're
>>>>> describing two essentially distinct devices/drivers in a single device
>>>>> node, and there is no way to distinguish that.
>>>>>
>>>>> By splitting up the two distinct components into separate device tree
>>>>> nodes, the relation between the providers/consumers is clearly
>>>>> described.
>>>>
>>>> You can split devices without splitting the nodes. I do not see reason
>>>> why the DT is the problem here.
>>>>
>>>
>>> The Linux drivers for this particular mailbox/clock controller already
>>> work exactly the way you propose. They are split into two devices that
>>> can probe independently.
>>>
>>> The problem is outside of the drivers, because fw_devlink in Linux
>>> blocks probing until all resources specified in the device tree nodes
>>> become available. fw_devlink has no knowledge that the mailbox described
>>> by this peculiar device tree node does not actually need the clocks:
>>>
>>> 	apcs1_mbox: mailbox@b011000 {
>>> 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
>>> 		reg = <0x0b011000 0x1000>;
>>> 		#mbox-cells = <1>;
>>> 		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
>>> 		clock-names = "pll", "aux", "ref";
>>> 		#clock-cells = <0>;
>>> 	};
>>>
>>> Without device-specific quirks in fw_devlink, the fact that these clocks
>>> are only used by an unrelated clock controller only becomes clear if we
>>> split the device tree node like I propose in this series:
>>>
>>> 	apcs1_mbox: mailbox@b011000 {
>>> 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
>>> 		reg = <0x0b011000 0x1000>;
>>> 		#mbox-cells = <1>;
>>>
>>> 		apcs1_clk: clock-controller {
>>> 			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
>>> 			clock-names = "pll", "aux", "ref";
>>> 			#clock-cells = <0>;
>>> 		};
>>> 	};
>>
>> Above code suggests that clocks are not needed for the mailbox at all.
>> You need to be really sure of that. If that's the case, then this
>> description looks like correct hardware description, more detailed then
>> the first case, though.
>>
> 
> I'm still sceptical here.
> 
> In the first snippet above, we describe a single IP block which provides
> mailboxes and clocks.
> 
> In the second snippet we're saying that the IP block is a mailbox, and
> then it somehow have a subcomponent which is a clock provider.
> 
> It seems to me that we're choosing the second option because it better
> fits the Linux implementation, rather than that it would be a better

I initially commented in similar way, however some more explanations
were provided.

> representation of the hardware. To the point that we can't even describe
> the register range of the subcomponent...

I did not check in any manual, so all my comments here are based on
above explanations and DTS.

Subnodes are allowed if they come with their own resources. You are
right there is no separate address space, so that's argument against
subnode. But there is separate clock, not needed for the parent (!!!),
which is an argument in favor.

> 
> 
> Can you confirm that this is the path we want to go here?

It is an acceptable solution to me, but I am not saying that every
device should be converted that way.

Best regards,
Krzysztof

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephen Boyd]

Quoting Bjorn Andersson (2025-06-10 20:31:57)
> 
> I'm still sceptical here.
> 
> In the first snippet above, we describe a single IP block which provides
> mailboxes and clocks.
> 
> In the second snippet we're saying that the IP block is a mailbox, and
> then it somehow have a subcomponent which is a clock provider.
> 
> It seems to me that we're choosing the second option because it better
> fits the Linux implementation, rather than that it would be a better
> representation of the hardware. To the point that we can't even describe
> the register range of the subcomponent...
> 

Agreed. Don't workaround problems in the kernel by changing the binding
to have sub-nodes.

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Stephan Gerhold]

wOn Sat, Jun 21, 2025 at 01:51:16PM -0700, Stephen Boyd wrote:
> Quoting Bjorn Andersson (2025-06-10 20:31:57)
> > I'm still sceptical here.
> > 
> > In the first snippet above, we describe a single IP block which provides
> > mailboxes and clocks.
> > 
> > In the second snippet we're saying that the IP block is a mailbox, and
> > then it somehow have a subcomponent which is a clock provider.
> > 
> > It seems to me that we're choosing the second option because it better
> > fits the Linux implementation, rather than that it would be a better
> > representation of the hardware. To the point that we can't even describe
> > the register range of the subcomponent...
> > 
> 
> Agreed. Don't workaround problems in the kernel by changing the binding
> to have sub-nodes.

I can describe the register range for the subcomponent if you prefer
(it's reg = <0x50 0xc>; within the parent component). That would be easy
to add.

Your more fundamental concern (working around problems in the kernel by
changing the binding) is a more tricky and subtle one. I had exactly the
same thought when I started making this patch series. However, if you
start looking more closely you will see that this is much easier said
than done. I tried to explain the problem already a few times (in the
cover letter, the commit messages and responses to this series), but let
me try again. Perhaps in different words it will become more
understandable.

Just for clarity, let's take the current device tree description again:

	apcs1_mbox: mailbox@b011000 {
		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
		reg = <0x0b011000 0x1000>;
		#mbox-cells = <1>;
		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
		clock-names = "pll", "aux", "ref";
		#clock-cells = <0>;
	};

Clearly this is a mailbox (#mbox-cells) and a clock controller
(#clock-cells). In the hardware these are stuffed into one register
region, but they don't have anything to do with each other. In
particular, the specified clocks are only used by the clock controller.
They are not used or related in any way to the mailbox component.

We need to have the mailbox available early to proceed with booting. The
clock controller can probe anytime later. The &rpmcc clock required by
the clock controller depends on having the mailbox available.

In Linux, I cannot get the mailbox driver to probe as long as the &rpmcc
clock is specified inside this device tree node (or by using
post-init-providers, but see [1]). This is not something I can fix in
the driver. The "problem in the kernel" you are referring to is
essentially "fw_devlink". Independent of the device-specific bindings we
define, it is built with the assumption that resources specified in a
device tree node are required to get a device functioning.

We usually want this behavior, but it doesn't work in this case. I argue
this is because we describe *two* devices as part of a *single* device
tree node. By splitting the *two* devices into *two* device tree nodes,
it is clear which resources belong to which device, and fw_devlink can
function correctly.

You argue this is a problem to be solved in the kernel. In practice,
this would mean one of the following:

 - Remove fw_devlink from Linux.
 - Start adding device-specific quirks into the generic fw_devlink code.
   Hardcode device links that cannot be deferred from the device tree
   because our hardware description is too broad.

Both of these are not really desirable, right?

I don't think there is a good way around making the hardware description
more precise by giving the two devices separate device tree nodes. There
are many different options for modelling these, and I would be fine with
all of them if you think one of them fits better:

Top-level siblings:

	apcs1_mbox: mailbox@b011008 {
		compatible = "qcom,msm8939-apcs-mbox";
		reg = <0x0b011008 0x4>;
		#mbox-cells = <1>;
	};

	apcs1_clk: clock-controller@b011050 {
		compatible = "qcom,msm8939-apcs-clk";
		reg = <0x0b011050 0xc>;
		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
		clock-names = "pll", "aux", "ref";
		#clock-cells = <0>;		
	};

Top-level syscon wrapper with two children:

	syscon@b011000 {
		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
		reg = <0x0b011000 0x1000>;
		#adress-cells = <1>;
		#size-cells = <1>;
		ranges = <0 0x0b011000 0x1000>;

		apcs1_mbox: mailbox@8 {
			compatible = "qcom,msm8939-apcs-mbox";
			reg = <0x8 0x4>;
			#mbox-cells = <1>;
		};

		apcs1_clk: clock-controller@50 {
			compatible = "qcom,msm8939-apcs-clk";
			reg = <0x0b011050 0xc>;
			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
			clock-names = "pll", "aux", "ref";
			#clock-cells = <0>;
		};
	};

Mailbox as parent (what I did in this series):

	apcs1_mbox: mailbox@b011000 {
		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
		reg = <0x0b011000 0x1000>;
		#mbox-cells = <1>;

		apcs1_clk: clock-controller {
			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
			clock-names = "pll", "aux", "ref";
			#clock-cells = <0>;
		};
	};

Maybe it makes more sense with this explanation and the other options.
Let me know what you think!

Thanks,
Stephan

[1]: https://lore.kernel.org/linux-arm-msm/aC-AqDa8cjq2AYeM@linaro.org/

Re: [PATCH 1/4] dt-bindings: mailbox: qcom,apcs: Add separate node for clock-controller

[Bjorn Andersson]

On Mon, Jun 23, 2025 at 03:17:33PM +0200, Stephan Gerhold wrote:
> wOn Sat, Jun 21, 2025 at 01:51:16PM -0700, Stephen Boyd wrote:
> > Quoting Bjorn Andersson (2025-06-10 20:31:57)
> > > I'm still sceptical here.
> > > 
> > > In the first snippet above, we describe a single IP block which provides
> > > mailboxes and clocks.
> > > 
> > > In the second snippet we're saying that the IP block is a mailbox, and
> > > then it somehow have a subcomponent which is a clock provider.
> > > 
> > > It seems to me that we're choosing the second option because it better
> > > fits the Linux implementation, rather than that it would be a better
> > > representation of the hardware. To the point that we can't even describe
> > > the register range of the subcomponent...
> > > 
> > 
> > Agreed. Don't workaround problems in the kernel by changing the binding
> > to have sub-nodes.
> 
> I can describe the register range for the subcomponent if you prefer
> (it's reg = <0x50 0xc>; within the parent component). That would be easy
> to add.
> 
> Your more fundamental concern (working around problems in the kernel by
> changing the binding) is a more tricky and subtle one. I had exactly the
> same thought when I started making this patch series. However, if you
> start looking more closely you will see that this is much easier said
> than done. I tried to explain the problem already a few times (in the
> cover letter, the commit messages and responses to this series), but let
> me try again. Perhaps in different words it will become more
> understandable.
> 
> Just for clarity, let's take the current device tree description again:
> 
> 	apcs1_mbox: mailbox@b011000 {
> 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> 		reg = <0x0b011000 0x1000>;
> 		#mbox-cells = <1>;
> 		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> 		clock-names = "pll", "aux", "ref";
> 		#clock-cells = <0>;
> 	};
> 
> Clearly this is a mailbox (#mbox-cells) and a clock controller
> (#clock-cells). In the hardware these are stuffed into one register
> region, but they don't have anything to do with each other. In
> particular, the specified clocks are only used by the clock controller.
> They are not used or related in any way to the mailbox component.
> 
> We need to have the mailbox available early to proceed with booting. The
> clock controller can probe anytime later. The &rpmcc clock required by
> the clock controller depends on having the mailbox available.
> 
> In Linux, I cannot get the mailbox driver to probe as long as the &rpmcc
> clock is specified inside this device tree node (or by using
> post-init-providers, but see [1]). This is not something I can fix in
> the driver. The "problem in the kernel" you are referring to is
> essentially "fw_devlink". Independent of the device-specific bindings we
> define, it is built with the assumption that resources specified in a
> device tree node are required to get a device functioning.
> 
> We usually want this behavior, but it doesn't work in this case. I argue
> this is because we describe *two* devices as part of a *single* device
> tree node. By splitting the *two* devices into *two* device tree nodes,
> it is clear which resources belong to which device, and fw_devlink can
> function correctly.
> 

We have many cases where there are cyclic links in the DeviceTree
representation - clock controllers depending on clock providers that
depend on the same clock controller, regulators being supplied by
regulators of the same PMIC etc.

From a DeviceTree point of view this looks quite similar, but from an
implementation perspective this is simpler than those examples - we
don't even need to do async resolution per resource here.

> You argue this is a problem to be solved in the kernel. In practice,
> this would mean one of the following:
> 
>  - Remove fw_devlink from Linux.
>  - Start adding device-specific quirks into the generic fw_devlink code.
>    Hardcode device links that cannot be deferred from the device tree
>    because our hardware description is too broad.
> 
> Both of these are not really desirable, right?
> 

fw_devlink is supposed to optimize the probe order, it must not prevent
the system from booting just because there is cyclic dependencies
between IP-blocks.

If fw_devlink is authoritative in determining which order things must
happen, then it needs to deal with these things.

> I don't think there is a good way around making the hardware description
> more precise by giving the two devices separate device tree nodes. There
> are many different options for modelling these, and I would be fine with
> all of them if you think one of them fits better:
> 
> Top-level siblings:
> 
> 	apcs1_mbox: mailbox@b011008 {
> 		compatible = "qcom,msm8939-apcs-mbox";
> 		reg = <0x0b011008 0x4>;
> 		#mbox-cells = <1>;
> 	};
> 
> 	apcs1_clk: clock-controller@b011050 {
> 		compatible = "qcom,msm8939-apcs-clk";
> 		reg = <0x0b011050 0xc>;
> 		clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> 		clock-names = "pll", "aux", "ref";
> 		#clock-cells = <0>;		
> 	};

This doesn't scale to any example where you have more than single
resources laid out in a convenient order.

> 
> Top-level syscon wrapper with two children:
> 
> 	syscon@b011000 {
> 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> 		reg = <0x0b011000 0x1000>;
> 		#adress-cells = <1>;
> 		#size-cells = <1>;
> 		ranges = <0 0x0b011000 0x1000>;
> 
> 		apcs1_mbox: mailbox@8 {
> 			compatible = "qcom,msm8939-apcs-mbox";
> 			reg = <0x8 0x4>;
> 			#mbox-cells = <1>;
> 		};
> 
> 		apcs1_clk: clock-controller@50 {
> 			compatible = "qcom,msm8939-apcs-clk";
> 			reg = <0x0b011050 0xc>;
> 			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> 			clock-names = "pll", "aux", "ref";
> 			#clock-cells = <0>;
> 		};
> 	};

This is clearly implementation-driven. Until fw_devlink "forced" you
into this model we considered APCS KPSS global to be one entity.

> 
> Mailbox as parent (what I did in this series):
> 
> 	apcs1_mbox: mailbox@b011000 {
> 		compatible = "qcom,msm8939-apcs-kpss-global", "syscon";
> 		reg = <0x0b011000 0x1000>;
> 		#mbox-cells = <1>;
> 
> 		apcs1_clk: clock-controller {
> 			clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
> 			clock-names = "pll", "aux", "ref";
> 			#clock-cells = <0>;
> 		};
> 	};

This is just a pragmatic variant of above.

> 
> Maybe it makes more sense with this explanation and the other options.
> Let me know what you think!
> 

Regards,
Bjorn

> Thanks,
> Stephan
> 
> [1]: https://lore.kernel.org/linux-arm-msm/aC-AqDa8cjq2AYeM@linaro.org/