* [PATCH v5 1/4] dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings
2023-09-28 15:58 [PATCH v5 0/4] add zynqmp TCM bindings Tanmay Shah
@ 2023-09-28 15:58 ` Tanmay Shah
2023-09-28 15:58 ` [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc Tanmay Shah
` (2 subsequent siblings)
3 siblings, 0 replies; 15+ messages in thread
From: Tanmay Shah @ 2023-09-28 15:58 UTC (permalink / raw)
To: andersson, mathieu.poirier, robh+dt, krzysztof.kozlowski+dt,
michal.simek
Cc: radhey.shyam.pandey, tanmay.shah, ben.levinsky, linux-remoteproc,
devicetree, linux-arm-kernel, Rob Herring
From: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Introduce bindings for TCM memory address space on AMD-xilinx Zynq
UltraScale+ platform. It will help in defining TCM in device-tree
and make it's access platform agnostic and data-driven.
Tightly-coupled memories(TCMs) are low-latency memory that provides
predictable instruction execution and predictable data load/store
timing. Each Cortex-R5F processor contains two 64-bit wide 64 KB memory
banks on the ATCM and BTCM ports, for a total of 128 KB of memory.
The TCM resources(reg, reg-names and power-domain) are documented for
each TCM in the R5 node. The reg and reg-names are made as required
properties as we don't want to hardcode TCM addresses for future
platforms and for zu+ legacy implementation will ensure that the
old dts w/o reg/reg-names works and stable ABI is maintained.
It also extends the examples for TCM split and lockstep modes.
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
Acked-by: Rob Herring <robh@kernel.org>
---
.../remoteproc/xlnx,zynqmp-r5fss.yaml | 131 +++++++++++++++---
1 file changed, 113 insertions(+), 18 deletions(-)
diff --git a/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml b/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml
index 78aac69f1060..9ecd63ea1b38 100644
--- a/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml
@@ -20,6 +20,17 @@ properties:
compatible:
const: xlnx,zynqmp-r5fss
+ "#address-cells":
+ const: 2
+
+ "#size-cells":
+ const: 2
+
+ ranges:
+ description: |
+ Standard ranges definition providing address translations for
+ local R5F TCM address spaces to bus addresses.
+
xlnx,cluster-mode:
$ref: /schemas/types.yaml#/definitions/uint32
enum: [0, 1, 2]
@@ -37,7 +48,7 @@ properties:
2: single cpu mode
patternProperties:
- "^r5f-[a-f0-9]+$":
+ "^r5f@[0-9a-f]+$":
type: object
description: |
The RPU is located in the Low Power Domain of the Processor Subsystem.
@@ -54,8 +65,19 @@ patternProperties:
compatible:
const: xlnx,zynqmp-r5f
+ reg:
+ items:
+ - description: ATCM internal memory region
+ - description: BTCM internal memory region
+
+ reg-names:
+ items:
+ - const: atcm
+ - const: btcm
+
power-domains:
- maxItems: 1
+ minItems: 1
+ maxItems: 3
mboxes:
minItems: 1
@@ -102,34 +124,107 @@ patternProperties:
required:
- compatible
- power-domains
+ - reg
+ - reg-names
unevaluatedProperties: false
required:
- compatible
+ - "#address-cells"
+ - "#size-cells"
+ - ranges
additionalProperties: false
examples:
- |
- remoteproc {
- compatible = "xlnx,zynqmp-r5fss";
- xlnx,cluster-mode = <1>;
-
- r5f-0 {
- compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware 0x7>;
- memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>, <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
- mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
- mbox-names = "tx", "rx";
+ #include <dt-bindings/power/xlnx-zynqmp-power.h>
+
+ //Split mode configuration
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ remoteproc@ffe00000 {
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <0>;
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
+ <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
+
+ r5f@0 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
+ reg-names = "atcm", "btcm";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>;
+ memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>,
+ <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
+ mbox-names = "tx", "rx";
+ };
+
+ r5f@1 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm", "btcm";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>,
+ <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
+ mbox-names = "tx", "rx";
+ };
};
+ };
- r5f-1 {
- compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware 0x8>;
- memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>, <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
- mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
- mbox-names = "tx", "rx";
+ - |
+ //Lockstep configuration
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ remoteproc@ffe00000 {
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <1>;
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x20000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x20000>;
+
+ r5f@0 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x0 0x0 0x0 0x20000>, <0x0 0x20000 0x0 0x20000>;
+ reg-names = "atcm", "btcm";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>;
+ memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>,
+ <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
+ mbox-names = "tx", "rx";
+ };
+
+ r5f@1 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm", "btcm";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>,
+ <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
+ mbox-names = "tx", "rx";
+ };
};
};
...
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-09-28 15:58 [PATCH v5 0/4] add zynqmp TCM bindings Tanmay Shah
2023-09-28 15:58 ` [PATCH v5 1/4] dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings Tanmay Shah
@ 2023-09-28 15:58 ` Tanmay Shah
2023-10-02 15:55 ` Mathieu Poirier
2023-09-28 15:58 ` [PATCH v5 3/4] remoteproc: zynqmp: add pm domains support Tanmay Shah
2023-09-28 15:59 ` [PATCH v5 4/4] remoteproc: zynqmp: parse TCM from device tree Tanmay Shah
3 siblings, 1 reply; 15+ messages in thread
From: Tanmay Shah @ 2023-09-28 15:58 UTC (permalink / raw)
To: andersson, mathieu.poirier, robh+dt, krzysztof.kozlowski+dt,
michal.simek
Cc: radhey.shyam.pandey, tanmay.shah, ben.levinsky, linux-remoteproc,
devicetree, linux-arm-kernel
Add properties as per new bindings in zynqmp remoteproc node
to represent TCM address and size. This patch configures
RPU in split mode and adds TCM information accordingly.
Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
---
arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
1 file changed, 22 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
index b61fc99cd911..01e12894c88e 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
+++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
@@ -247,19 +247,35 @@ fpga_full: fpga-full {
ranges;
};
- remoteproc {
+ remoteproc@ffe00000 {
compatible = "xlnx,zynqmp-r5fss";
- xlnx,cluster-mode = <1>;
+ xlnx,cluster-mode = <0>;
- r5f-0 {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
+ <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
+
+ r5f@0 {
compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware PD_RPU_0>;
+ reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
+ reg-names = "atcm", "btcm";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>;
memory-region = <&rproc_0_fw_image>;
};
- r5f-1 {
+ r5f@1 {
compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware PD_RPU_1>;
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm", "btcm";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
memory-region = <&rproc_1_fw_image>;
};
};
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-09-28 15:58 ` [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc Tanmay Shah
@ 2023-10-02 15:55 ` Mathieu Poirier
2023-10-02 16:25 ` Tanmay Shah
0 siblings, 1 reply; 15+ messages in thread
From: Mathieu Poirier @ 2023-10-02 15:55 UTC (permalink / raw)
To: Tanmay Shah
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On Thu, Sep 28, 2023 at 08:58:58AM -0700, Tanmay Shah wrote:
> Add properties as per new bindings in zynqmp remoteproc node
> to represent TCM address and size. This patch configures
> RPU in split mode and adds TCM information accordingly.
>
Why is this changed from lockstep to split mode? What about all the people out
there that are expecting a lockstep mode?
> Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> ---
> arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
> 1 file changed, 22 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> index b61fc99cd911..01e12894c88e 100644
> --- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> +++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> @@ -247,19 +247,35 @@ fpga_full: fpga-full {
> ranges;
> };
>
> - remoteproc {
> + remoteproc@ffe00000 {
> compatible = "xlnx,zynqmp-r5fss";
> - xlnx,cluster-mode = <1>;
> + xlnx,cluster-mode = <0>;
>
> - r5f-0 {
> + #address-cells = <2>;
> + #size-cells = <2>;
> +
> + ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
> + <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
> + <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
> + <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
> +
> + r5f@0 {
> compatible = "xlnx,zynqmp-r5f";
> - power-domains = <&zynqmp_firmware PD_RPU_0>;
> + reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
> + reg-names = "atcm", "btcm";
> + power-domains = <&zynqmp_firmware PD_RPU_0>,
> + <&zynqmp_firmware PD_R5_0_ATCM>,
> + <&zynqmp_firmware PD_R5_0_BTCM>;
> memory-region = <&rproc_0_fw_image>;
> };
>
> - r5f-1 {
> + r5f@1 {
> compatible = "xlnx,zynqmp-r5f";
> - power-domains = <&zynqmp_firmware PD_RPU_1>;
> + reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
> + reg-names = "atcm", "btcm";
> + power-domains = <&zynqmp_firmware PD_RPU_1>,
> + <&zynqmp_firmware PD_R5_1_ATCM>,
> + <&zynqmp_firmware PD_R5_1_BTCM>;
> memory-region = <&rproc_1_fw_image>;
> };
> };
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-10-02 15:55 ` Mathieu Poirier
@ 2023-10-02 16:25 ` Tanmay Shah
2023-10-02 17:12 ` Tanmay Shah
0 siblings, 1 reply; 15+ messages in thread
From: Tanmay Shah @ 2023-10-02 16:25 UTC (permalink / raw)
To: Mathieu Poirier
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
Hi Mathieu,
Thanks for the reviews. Please find my comments below.
On 10/2/23 10:55 AM, Mathieu Poirier wrote:
> On Thu, Sep 28, 2023 at 08:58:58AM -0700, Tanmay Shah wrote:
> > Add properties as per new bindings in zynqmp remoteproc node
> > to represent TCM address and size. This patch configures
> > RPU in split mode and adds TCM information accordingly.
> >
>
> Why is this changed from lockstep to split mode? What about all the people out
> there that are expecting a lockstep mode?
I agree, this should have been in split mode in the first place as we would like to demonstrate use of both
RPUs with two separate demo firmwares which is the best use of the
hardware and the most preferred use of zynqmp platform by people. That motivates to change
this to split mode.
Now changing this may not be problem for lot of people with following reasons.
The firmwares that are only using first 64KB of TCM memory, they can easily run in split mode as well.
Also rpmsg vring information isn't available in device-tree yet, so I am hoping that firmware that
are using upstream device-tree are not that big yet.
If we change this to split mode before introducing rpmsg related nodes, I bet it will affect very less number of people.
For lockstep mode the example is available in dt-bindings document.
So, if people need lockstep mode for any reason, all they need to change is xlnx,cluster-mode property from 0 to 1 and TCM nodes
from bindings document.
If you think it's crucial to mention all above, I can send new patch with all above info in commit message.
>
> > Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> > ---
> > arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
> > 1 file changed, 22 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > index b61fc99cd911..01e12894c88e 100644
> > --- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > +++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > @@ -247,19 +247,35 @@ fpga_full: fpga-full {
> > ranges;
> > };
> >
> > - remoteproc {
> > + remoteproc@ffe00000 {
> > compatible = "xlnx,zynqmp-r5fss";
> > - xlnx,cluster-mode = <1>;
> > + xlnx,cluster-mode = <0>;
> >
> > - r5f-0 {
> > + #address-cells = <2>;
> > + #size-cells = <2>;
> > +
> > + ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
> > + <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
> > + <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
> > + <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
> > +
> > + r5f@0 {
> > compatible = "xlnx,zynqmp-r5f";
> > - power-domains = <&zynqmp_firmware PD_RPU_0>;
> > + reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
> > + reg-names = "atcm", "btcm";
> > + power-domains = <&zynqmp_firmware PD_RPU_0>,
> > + <&zynqmp_firmware PD_R5_0_ATCM>,
> > + <&zynqmp_firmware PD_R5_0_BTCM>;
> > memory-region = <&rproc_0_fw_image>;
> > };
> >
> > - r5f-1 {
> > + r5f@1 {
> > compatible = "xlnx,zynqmp-r5f";
> > - power-domains = <&zynqmp_firmware PD_RPU_1>;
> > + reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
> > + reg-names = "atcm", "btcm";
> > + power-domains = <&zynqmp_firmware PD_RPU_1>,
> > + <&zynqmp_firmware PD_R5_1_ATCM>,
> > + <&zynqmp_firmware PD_R5_1_BTCM>;
> > memory-region = <&rproc_1_fw_image>;
> > };
> > };
> > --
> > 2.25.1
> >
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-10-02 16:25 ` Tanmay Shah
@ 2023-10-02 17:12 ` Tanmay Shah
2023-10-02 20:17 ` Mathieu Poirier
0 siblings, 1 reply; 15+ messages in thread
From: Tanmay Shah @ 2023-10-02 17:12 UTC (permalink / raw)
To: Mathieu Poirier
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On 10/2/23 11:25 AM, Tanmay Shah wrote:
> Hi Mathieu,
>
> Thanks for the reviews. Please find my comments below.
>
> On 10/2/23 10:55 AM, Mathieu Poirier wrote:
> > On Thu, Sep 28, 2023 at 08:58:58AM -0700, Tanmay Shah wrote:
> > > Add properties as per new bindings in zynqmp remoteproc node
> > > to represent TCM address and size. This patch configures
> > > RPU in split mode and adds TCM information accordingly.
> > >
> >
> > Why is this changed from lockstep to split mode? What about all the people out
> > there that are expecting a lockstep mode?
>
> I agree, this should have been in split mode in the first place as we would like to demonstrate use of both
>
> RPUs with two separate demo firmwares which is the best use of the
>
> hardware and the most preferred use of zynqmp platform by people. That motivates to change
>
> this to split mode.
>
>
> Now changing this may not be problem for lot of people with following reasons.
>
> The firmwares that are only using first 64KB of TCM memory, they can easily run in split mode as well.
>
> Also rpmsg vring information isn't available in device-tree yet, so I am hoping that firmware that
>
> are using upstream device-tree are not that big yet.
>
> If we change this to split mode before introducing rpmsg related nodes, I bet it will affect very less number of people.
>
>
> For lockstep mode the example is available in dt-bindings document.
>
> So, if people need lockstep mode for any reason, all they need to change is xlnx,cluster-mode property from 0 to 1 and TCM nodes
>
> from bindings document.
>
>
> If you think it's crucial to mention all above, I can send new patch with all above info in commit message.
Something to add to this. So, let's say if we don't change it now, what would be good time to change it?
As I am hopping to use RPU1 as well with upstream device-tree. Please let me know some suggestion to work this.
Thanks and again as always appreciate complete reviews,
Tanmay
>
>
> >
> > > Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> > > ---
> > > arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
> > > 1 file changed, 22 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > index b61fc99cd911..01e12894c88e 100644
> > > --- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > +++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > @@ -247,19 +247,35 @@ fpga_full: fpga-full {
> > > ranges;
> > > };
> > >
> > > - remoteproc {
> > > + remoteproc@ffe00000 {
> > > compatible = "xlnx,zynqmp-r5fss";
> > > - xlnx,cluster-mode = <1>;
> > > + xlnx,cluster-mode = <0>;
> > >
> > > - r5f-0 {
> > > + #address-cells = <2>;
> > > + #size-cells = <2>;
> > > +
> > > + ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
> > > + <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
> > > + <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
> > > + <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
> > > +
> > > + r5f@0 {
> > > compatible = "xlnx,zynqmp-r5f";
> > > - power-domains = <&zynqmp_firmware PD_RPU_0>;
> > > + reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
> > > + reg-names = "atcm", "btcm";
> > > + power-domains = <&zynqmp_firmware PD_RPU_0>,
> > > + <&zynqmp_firmware PD_R5_0_ATCM>,
> > > + <&zynqmp_firmware PD_R5_0_BTCM>;
> > > memory-region = <&rproc_0_fw_image>;
> > > };
> > >
> > > - r5f-1 {
> > > + r5f@1 {
> > > compatible = "xlnx,zynqmp-r5f";
> > > - power-domains = <&zynqmp_firmware PD_RPU_1>;
> > > + reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
> > > + reg-names = "atcm", "btcm";
> > > + power-domains = <&zynqmp_firmware PD_RPU_1>,
> > > + <&zynqmp_firmware PD_R5_1_ATCM>,
> > > + <&zynqmp_firmware PD_R5_1_BTCM>;
> > > memory-region = <&rproc_1_fw_image>;
> > > };
> > > };
> > > --
> > > 2.25.1
> > >
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-10-02 17:12 ` Tanmay Shah
@ 2023-10-02 20:17 ` Mathieu Poirier
2023-10-02 20:54 ` Tanmay Shah
0 siblings, 1 reply; 15+ messages in thread
From: Mathieu Poirier @ 2023-10-02 20:17 UTC (permalink / raw)
To: Tanmay Shah
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On Mon, 2 Oct 2023 at 11:12, Tanmay Shah <tanmay.shah@amd.com> wrote:
>
>
> On 10/2/23 11:25 AM, Tanmay Shah wrote:
> > Hi Mathieu,
> >
> > Thanks for the reviews. Please find my comments below.
> >
> > On 10/2/23 10:55 AM, Mathieu Poirier wrote:
> > > On Thu, Sep 28, 2023 at 08:58:58AM -0700, Tanmay Shah wrote:
> > > > Add properties as per new bindings in zynqmp remoteproc node
> > > > to represent TCM address and size. This patch configures
> > > > RPU in split mode and adds TCM information accordingly.
> > > >
> > >
> > > Why is this changed from lockstep to split mode? What about all the people out
> > > there that are expecting a lockstep mode?
> >
> > I agree, this should have been in split mode in the first place as we would like to demonstrate use of both
> >
> > RPUs with two separate demo firmwares which is the best use of the
> >
> > hardware and the most preferred use of zynqmp platform by people. That motivates to change
> >
> > this to split mode.
> >
> >
> > Now changing this may not be problem for lot of people with following reasons.
> >
> > The firmwares that are only using first 64KB of TCM memory, they can easily run in split mode as well.
> >
> > Also rpmsg vring information isn't available in device-tree yet, so I am hoping that firmware that
> >
> > are using upstream device-tree are not that big yet.
> >
> > If we change this to split mode before introducing rpmsg related nodes, I bet it will affect very less number of people.
> >
> >
> > For lockstep mode the example is available in dt-bindings document.
> >
I could use the same argument for the split mode, i.e default is
lockstep and there is an example in the dt-bindings document for split
mode.
> > So, if people need lockstep mode for any reason, all they need to change is xlnx,cluster-mode property from 0 to 1 and TCM nodes
> >
> > from bindings document.
> >
> >
> > If you think it's crucial to mention all above, I can send new patch with all above info in commit message.
>
> Something to add to this. So, let's say if we don't change it now, what would be good time to change it?
>
The best way to go about this is to introduce another DT that is
tailored for split mode. That way people can choose to boot their
device in a specific mode using the DT. If you decide to go this way,
look at how ST has split their DT for different boards - search for
"m4_rproc" under " arch/arm/boot/dts/st".
> As I am hopping to use RPU1 as well with upstream device-tree. Please let me know some suggestion to work this.
>
> Thanks and again as always appreciate complete reviews,
>
> Tanmay
>
>
> >
> >
> > >
> > > > Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> > > > ---
> > > > arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
> > > > 1 file changed, 22 insertions(+), 6 deletions(-)
> > > >
> > > > diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > index b61fc99cd911..01e12894c88e 100644
> > > > --- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > +++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > @@ -247,19 +247,35 @@ fpga_full: fpga-full {
> > > > ranges;
> > > > };
> > > >
> > > > - remoteproc {
> > > > + remoteproc@ffe00000 {
> > > > compatible = "xlnx,zynqmp-r5fss";
> > > > - xlnx,cluster-mode = <1>;
> > > > + xlnx,cluster-mode = <0>;
> > > >
> > > > - r5f-0 {
> > > > + #address-cells = <2>;
> > > > + #size-cells = <2>;
> > > > +
> > > > + ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
> > > > + <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
> > > > + <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
> > > > + <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
> > > > +
> > > > + r5f@0 {
> > > > compatible = "xlnx,zynqmp-r5f";
> > > > - power-domains = <&zynqmp_firmware PD_RPU_0>;
> > > > + reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
> > > > + reg-names = "atcm", "btcm";
> > > > + power-domains = <&zynqmp_firmware PD_RPU_0>,
> > > > + <&zynqmp_firmware PD_R5_0_ATCM>,
> > > > + <&zynqmp_firmware PD_R5_0_BTCM>;
> > > > memory-region = <&rproc_0_fw_image>;
> > > > };
> > > >
> > > > - r5f-1 {
> > > > + r5f@1 {
> > > > compatible = "xlnx,zynqmp-r5f";
> > > > - power-domains = <&zynqmp_firmware PD_RPU_1>;
> > > > + reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
> > > > + reg-names = "atcm", "btcm";
> > > > + power-domains = <&zynqmp_firmware PD_RPU_1>,
> > > > + <&zynqmp_firmware PD_R5_1_ATCM>,
> > > > + <&zynqmp_firmware PD_R5_1_BTCM>;
> > > > memory-region = <&rproc_1_fw_image>;
> > > > };
> > > > };
> > > > --
> > > > 2.25.1
> > > >
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-10-02 20:17 ` Mathieu Poirier
@ 2023-10-02 20:54 ` Tanmay Shah
2023-10-03 15:31 ` Mathieu Poirier
0 siblings, 1 reply; 15+ messages in thread
From: Tanmay Shah @ 2023-10-02 20:54 UTC (permalink / raw)
To: Mathieu Poirier
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On 10/2/23 3:17 PM, Mathieu Poirier wrote:
> On Mon, 2 Oct 2023 at 11:12, Tanmay Shah <tanmay.shah@amd.com> wrote:
> >
> >
> > On 10/2/23 11:25 AM, Tanmay Shah wrote:
> > > Hi Mathieu,
> > >
> > > Thanks for the reviews. Please find my comments below.
> > >
> > > On 10/2/23 10:55 AM, Mathieu Poirier wrote:
> > > > On Thu, Sep 28, 2023 at 08:58:58AM -0700, Tanmay Shah wrote:
> > > > > Add properties as per new bindings in zynqmp remoteproc node
> > > > > to represent TCM address and size. This patch configures
> > > > > RPU in split mode and adds TCM information accordingly.
> > > > >
> > > >
> > > > Why is this changed from lockstep to split mode? What about all the people out
> > > > there that are expecting a lockstep mode?
> > >
> > > I agree, this should have been in split mode in the first place as we would like to demonstrate use of both
> > >
> > > RPUs with two separate demo firmwares which is the best use of the
> > >
> > > hardware and the most preferred use of zynqmp platform by people. That motivates to change
> > >
> > > this to split mode.
> > >
> > >
> > > Now changing this may not be problem for lot of people with following reasons.
> > >
> > > The firmwares that are only using first 64KB of TCM memory, they can easily run in split mode as well.
> > >
> > > Also rpmsg vring information isn't available in device-tree yet, so I am hoping that firmware that
> > >
> > > are using upstream device-tree are not that big yet.
> > >
> > > If we change this to split mode before introducing rpmsg related nodes, I bet it will affect very less number of people.
> > >
> > >
> > > For lockstep mode the example is available in dt-bindings document.
> > >
>
> I could use the same argument for the split mode, i.e default is
> lockstep and there is an example in the dt-bindings document for split
> mode.
>
> > > So, if people need lockstep mode for any reason, all they need to change is xlnx,cluster-mode property from 0 to 1 and TCM nodes
> > >
> > > from bindings document.
> > >
> > >
> > > If you think it's crucial to mention all above, I can send new patch with all above info in commit message.
> >
> > Something to add to this. So, let's say if we don't change it now, what would be good time to change it?
> >
>
> The best way to go about this is to introduce another DT that is
> tailored for split mode. That way people can choose to boot their
> device in a specific mode using the DT. If you decide to go this way,
> look at how ST has split their DT for different boards - search for
> "m4_rproc" under " arch/arm/boot/dts/st".
Thanks for the suggestion. I looked at the example and I think it will work.
I have following idea.
So, if I understand this correctly, we introduce two separate nodes in device-tree.
SOC dtsi file: zynqmp.dtsi
remoteproc_lockstep: remoteproc@... {
. . .
status = "disabled";
};
remoteproc_split: remoteproc@... {
. . .
status = "disabled";
};
And then in board dts enable whatever mode is needed for that board:
*zcu102*.dts
&remoteproc_split {
status = "okay";
};
This sounds like good idea, I hope this is what you mean.
Please let me know if I am missing something.
>
> > As I am hopping to use RPU1 as well with upstream device-tree. Please let me know some suggestion to work this.
> >
> > Thanks and again as always appreciate complete reviews,
> >
> > Tanmay
> >
> >
> > >
> > >
> > > >
> > > > > Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> > > > > ---
> > > > > arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
> > > > > 1 file changed, 22 insertions(+), 6 deletions(-)
> > > > >
> > > > > diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > index b61fc99cd911..01e12894c88e 100644
> > > > > --- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > +++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > @@ -247,19 +247,35 @@ fpga_full: fpga-full {
> > > > > ranges;
> > > > > };
> > > > >
> > > > > - remoteproc {
> > > > > + remoteproc@ffe00000 {
> > > > > compatible = "xlnx,zynqmp-r5fss";
> > > > > - xlnx,cluster-mode = <1>;
> > > > > + xlnx,cluster-mode = <0>;
> > > > >
> > > > > - r5f-0 {
> > > > > + #address-cells = <2>;
> > > > > + #size-cells = <2>;
> > > > > +
> > > > > + ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
> > > > > + <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
> > > > > + <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
> > > > > + <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
> > > > > +
> > > > > + r5f@0 {
> > > > > compatible = "xlnx,zynqmp-r5f";
> > > > > - power-domains = <&zynqmp_firmware PD_RPU_0>;
> > > > > + reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
> > > > > + reg-names = "atcm", "btcm";
> > > > > + power-domains = <&zynqmp_firmware PD_RPU_0>,
> > > > > + <&zynqmp_firmware PD_R5_0_ATCM>,
> > > > > + <&zynqmp_firmware PD_R5_0_BTCM>;
> > > > > memory-region = <&rproc_0_fw_image>;
> > > > > };
> > > > >
> > > > > - r5f-1 {
> > > > > + r5f@1 {
> > > > > compatible = "xlnx,zynqmp-r5f";
> > > > > - power-domains = <&zynqmp_firmware PD_RPU_1>;
> > > > > + reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
> > > > > + reg-names = "atcm", "btcm";
> > > > > + power-domains = <&zynqmp_firmware PD_RPU_1>,
> > > > > + <&zynqmp_firmware PD_R5_1_ATCM>,
> > > > > + <&zynqmp_firmware PD_R5_1_BTCM>;
> > > > > memory-region = <&rproc_1_fw_image>;
> > > > > };
> > > > > };
> > > > > --
> > > > > 2.25.1
> > > > >
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-10-02 20:54 ` Tanmay Shah
@ 2023-10-03 15:31 ` Mathieu Poirier
2023-10-03 16:32 ` Tanmay Shah
0 siblings, 1 reply; 15+ messages in thread
From: Mathieu Poirier @ 2023-10-03 15:31 UTC (permalink / raw)
To: Tanmay Shah
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On Mon, Oct 02, 2023 at 03:54:30PM -0500, Tanmay Shah wrote:
>
> On 10/2/23 3:17 PM, Mathieu Poirier wrote:
> > On Mon, 2 Oct 2023 at 11:12, Tanmay Shah <tanmay.shah@amd.com> wrote:
> > >
> > >
> > > On 10/2/23 11:25 AM, Tanmay Shah wrote:
> > > > Hi Mathieu,
> > > >
> > > > Thanks for the reviews. Please find my comments below.
> > > >
> > > > On 10/2/23 10:55 AM, Mathieu Poirier wrote:
> > > > > On Thu, Sep 28, 2023 at 08:58:58AM -0700, Tanmay Shah wrote:
> > > > > > Add properties as per new bindings in zynqmp remoteproc node
> > > > > > to represent TCM address and size. This patch configures
> > > > > > RPU in split mode and adds TCM information accordingly.
> > > > > >
> > > > >
> > > > > Why is this changed from lockstep to split mode? What about all the people out
> > > > > there that are expecting a lockstep mode?
> > > >
> > > > I agree, this should have been in split mode in the first place as we would like to demonstrate use of both
> > > >
> > > > RPUs with two separate demo firmwares which is the best use of the
> > > >
> > > > hardware and the most preferred use of zynqmp platform by people. That motivates to change
> > > >
> > > > this to split mode.
> > > >
> > > >
> > > > Now changing this may not be problem for lot of people with following reasons.
> > > >
> > > > The firmwares that are only using first 64KB of TCM memory, they can easily run in split mode as well.
> > > >
> > > > Also rpmsg vring information isn't available in device-tree yet, so I am hoping that firmware that
> > > >
> > > > are using upstream device-tree are not that big yet.
> > > >
> > > > If we change this to split mode before introducing rpmsg related nodes, I bet it will affect very less number of people.
> > > >
> > > >
> > > > For lockstep mode the example is available in dt-bindings document.
> > > >
> >
> > I could use the same argument for the split mode, i.e default is
> > lockstep and there is an example in the dt-bindings document for split
> > mode.
> >
> > > > So, if people need lockstep mode for any reason, all they need to change is xlnx,cluster-mode property from 0 to 1 and TCM nodes
> > > >
> > > > from bindings document.
> > > >
> > > >
> > > > If you think it's crucial to mention all above, I can send new patch with all above info in commit message.
> > >
> > > Something to add to this. So, let's say if we don't change it now, what would be good time to change it?
> > >
> >
> > The best way to go about this is to introduce another DT that is
> > tailored for split mode. That way people can choose to boot their
> > device in a specific mode using the DT. If you decide to go this way,
> > look at how ST has split their DT for different boards - search for
> > "m4_rproc" under " arch/arm/boot/dts/st".
>
> Thanks for the suggestion. I looked at the example and I think it will work.
>
> I have following idea.
>
> So, if I understand this correctly, we introduce two separate nodes in device-tree.
>
> SOC dtsi file: zynqmp.dtsi
>
> remoteproc_lockstep: remoteproc@... {
>
> . . .
>
> status = "disabled";
I don't think you need the "status"
>
> };
>
>
> remoteproc_split: remoteproc@... {
>
> . . .
>
> status = "disabled";
>
> };
>
>
> And then in board dts enable whatever mode is needed for that board:
>
> *zcu102*.dts
>
> &remoteproc_split {
>
> status = "okay";
>
> };
Exactly. Again, I don't think the "status" is needed.
>
> This sounds like good idea, I hope this is what you mean.
>
> Please let me know if I am missing something.
>
>
> >
> > > As I am hopping to use RPU1 as well with upstream device-tree. Please let me know some suggestion to work this.
> > >
> > > Thanks and again as always appreciate complete reviews,
> > >
> > > Tanmay
> > >
> > >
> > > >
> > > >
> > > > >
> > > > > > Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> > > > > > ---
> > > > > > arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
> > > > > > 1 file changed, 22 insertions(+), 6 deletions(-)
> > > > > >
> > > > > > diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > > index b61fc99cd911..01e12894c88e 100644
> > > > > > --- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > > +++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > > @@ -247,19 +247,35 @@ fpga_full: fpga-full {
> > > > > > ranges;
> > > > > > };
> > > > > >
> > > > > > - remoteproc {
> > > > > > + remoteproc@ffe00000 {
> > > > > > compatible = "xlnx,zynqmp-r5fss";
> > > > > > - xlnx,cluster-mode = <1>;
> > > > > > + xlnx,cluster-mode = <0>;
> > > > > >
> > > > > > - r5f-0 {
> > > > > > + #address-cells = <2>;
> > > > > > + #size-cells = <2>;
> > > > > > +
> > > > > > + ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
> > > > > > + <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
> > > > > > + <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
> > > > > > + <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
> > > > > > +
> > > > > > + r5f@0 {
> > > > > > compatible = "xlnx,zynqmp-r5f";
> > > > > > - power-domains = <&zynqmp_firmware PD_RPU_0>;
> > > > > > + reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
> > > > > > + reg-names = "atcm", "btcm";
> > > > > > + power-domains = <&zynqmp_firmware PD_RPU_0>,
> > > > > > + <&zynqmp_firmware PD_R5_0_ATCM>,
> > > > > > + <&zynqmp_firmware PD_R5_0_BTCM>;
> > > > > > memory-region = <&rproc_0_fw_image>;
> > > > > > };
> > > > > >
> > > > > > - r5f-1 {
> > > > > > + r5f@1 {
> > > > > > compatible = "xlnx,zynqmp-r5f";
> > > > > > - power-domains = <&zynqmp_firmware PD_RPU_1>;
> > > > > > + reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
> > > > > > + reg-names = "atcm", "btcm";
> > > > > > + power-domains = <&zynqmp_firmware PD_RPU_1>,
> > > > > > + <&zynqmp_firmware PD_R5_1_ATCM>,
> > > > > > + <&zynqmp_firmware PD_R5_1_BTCM>;
> > > > > > memory-region = <&rproc_1_fw_image>;
> > > > > > };
> > > > > > };
> > > > > > --
> > > > > > 2.25.1
> > > > > >
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc
2023-10-03 15:31 ` Mathieu Poirier
@ 2023-10-03 16:32 ` Tanmay Shah
0 siblings, 0 replies; 15+ messages in thread
From: Tanmay Shah @ 2023-10-03 16:32 UTC (permalink / raw)
To: Mathieu Poirier
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On 10/3/23 10:31 AM, Mathieu Poirier wrote:
> On Mon, Oct 02, 2023 at 03:54:30PM -0500, Tanmay Shah wrote:
> >
> > On 10/2/23 3:17 PM, Mathieu Poirier wrote:
> > > On Mon, 2 Oct 2023 at 11:12, Tanmay Shah <tanmay.shah@amd.com> wrote:
> > > >
> > > >
> > > > On 10/2/23 11:25 AM, Tanmay Shah wrote:
> > > > > Hi Mathieu,
> > > > >
> > > > > Thanks for the reviews. Please find my comments below.
> > > > >
> > > > > On 10/2/23 10:55 AM, Mathieu Poirier wrote:
> > > > > > On Thu, Sep 28, 2023 at 08:58:58AM -0700, Tanmay Shah wrote:
> > > > > > > Add properties as per new bindings in zynqmp remoteproc node
> > > > > > > to represent TCM address and size. This patch configures
> > > > > > > RPU in split mode and adds TCM information accordingly.
> > > > > > >
> > > > > >
> > > > > > Why is this changed from lockstep to split mode? What about all the people out
> > > > > > there that are expecting a lockstep mode?
> > > > >
> > > > > I agree, this should have been in split mode in the first place as we would like to demonstrate use of both
> > > > >
> > > > > RPUs with two separate demo firmwares which is the best use of the
> > > > >
> > > > > hardware and the most preferred use of zynqmp platform by people. That motivates to change
> > > > >
> > > > > this to split mode.
> > > > >
> > > > >
> > > > > Now changing this may not be problem for lot of people with following reasons.
> > > > >
> > > > > The firmwares that are only using first 64KB of TCM memory, they can easily run in split mode as well.
> > > > >
> > > > > Also rpmsg vring information isn't available in device-tree yet, so I am hoping that firmware that
> > > > >
> > > > > are using upstream device-tree are not that big yet.
> > > > >
> > > > > If we change this to split mode before introducing rpmsg related nodes, I bet it will affect very less number of people.
> > > > >
> > > > >
> > > > > For lockstep mode the example is available in dt-bindings document.
> > > > >
> > >
> > > I could use the same argument for the split mode, i.e default is
> > > lockstep and there is an example in the dt-bindings document for split
> > > mode.
> > >
> > > > > So, if people need lockstep mode for any reason, all they need to change is xlnx,cluster-mode property from 0 to 1 and TCM nodes
> > > > >
> > > > > from bindings document.
> > > > >
> > > > >
> > > > > If you think it's crucial to mention all above, I can send new patch with all above info in commit message.
> > > >
> > > > Something to add to this. So, let's say if we don't change it now, what would be good time to change it?
> > > >
> > >
> > > The best way to go about this is to introduce another DT that is
> > > tailored for split mode. That way people can choose to boot their
> > > device in a specific mode using the DT. If you decide to go this way,
> > > look at how ST has split their DT for different boards - search for
> > > "m4_rproc" under " arch/arm/boot/dts/st".
> >
> > Thanks for the suggestion. I looked at the example and I think it will work.
> >
> > I have following idea.
> >
> > So, if I understand this correctly, we introduce two separate nodes in device-tree.
> >
> > SOC dtsi file: zynqmp.dtsi
> >
> > remoteproc_lockstep: remoteproc@... {
> >
> > . . .
> >
> > status = "disabled";
>
> I don't think you need the "status"
>
> >
> > };
> >
> >
> > remoteproc_split: remoteproc@... {
> >
> > . . .
> >
> > status = "disabled";
> >
> > };
> >
> >
> > And then in board dts enable whatever mode is needed for that board:
> >
> > *zcu102*.dts
> >
> > &remoteproc_split {
> >
> > status = "okay";
> >
> > };
>
> Exactly. Again, I don't think the "status" is needed.
Thanks for the suggestion. Yes I think if we don't include status then both nodes are enabled by default, and people can
disable in board dts what they don't want.
Instead I keep status disabled, for one of the node, (for split probably as lockstep is already enabled) and people
can choose what they wnt.
Thanks,
Tanmay
>
> >
> > This sounds like good idea, I hope this is what you mean.
> >
> > Please let me know if I am missing something.
> >
> >
> > >
> > > > As I am hopping to use RPU1 as well with upstream device-tree. Please let me know some suggestion to work this.
> > > >
> > > > Thanks and again as always appreciate complete reviews,
> > > >
> > > > Tanmay
> > > >
> > > >
> > > > >
> > > > >
> > > > > >
> > > > > > > Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> > > > > > > ---
> > > > > > > arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 28 ++++++++++++++++++++------
> > > > > > > 1 file changed, 22 insertions(+), 6 deletions(-)
> > > > > > >
> > > > > > > diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > > > index b61fc99cd911..01e12894c88e 100644
> > > > > > > --- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > > > +++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
> > > > > > > @@ -247,19 +247,35 @@ fpga_full: fpga-full {
> > > > > > > ranges;
> > > > > > > };
> > > > > > >
> > > > > > > - remoteproc {
> > > > > > > + remoteproc@ffe00000 {
> > > > > > > compatible = "xlnx,zynqmp-r5fss";
> > > > > > > - xlnx,cluster-mode = <1>;
> > > > > > > + xlnx,cluster-mode = <0>;
> > > > > > >
> > > > > > > - r5f-0 {
> > > > > > > + #address-cells = <2>;
> > > > > > > + #size-cells = <2>;
> > > > > > > +
> > > > > > > + ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
> > > > > > > + <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
> > > > > > > + <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
> > > > > > > + <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
> > > > > > > +
> > > > > > > + r5f@0 {
> > > > > > > compatible = "xlnx,zynqmp-r5f";
> > > > > > > - power-domains = <&zynqmp_firmware PD_RPU_0>;
> > > > > > > + reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
> > > > > > > + reg-names = "atcm", "btcm";
> > > > > > > + power-domains = <&zynqmp_firmware PD_RPU_0>,
> > > > > > > + <&zynqmp_firmware PD_R5_0_ATCM>,
> > > > > > > + <&zynqmp_firmware PD_R5_0_BTCM>;
> > > > > > > memory-region = <&rproc_0_fw_image>;
> > > > > > > };
> > > > > > >
> > > > > > > - r5f-1 {
> > > > > > > + r5f@1 {
> > > > > > > compatible = "xlnx,zynqmp-r5f";
> > > > > > > - power-domains = <&zynqmp_firmware PD_RPU_1>;
> > > > > > > + reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
> > > > > > > + reg-names = "atcm", "btcm";
> > > > > > > + power-domains = <&zynqmp_firmware PD_RPU_1>,
> > > > > > > + <&zynqmp_firmware PD_R5_1_ATCM>,
> > > > > > > + <&zynqmp_firmware PD_R5_1_BTCM>;
> > > > > > > memory-region = <&rproc_1_fw_image>;
> > > > > > > };
> > > > > > > };
> > > > > > > --
> > > > > > > 2.25.1
> > > > > > >
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v5 3/4] remoteproc: zynqmp: add pm domains support
2023-09-28 15:58 [PATCH v5 0/4] add zynqmp TCM bindings Tanmay Shah
2023-09-28 15:58 ` [PATCH v5 1/4] dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings Tanmay Shah
2023-09-28 15:58 ` [PATCH v5 2/4] dts: zynqmp: add properties for TCM in remoteproc Tanmay Shah
@ 2023-09-28 15:58 ` Tanmay Shah
2023-10-02 17:11 ` Mathieu Poirier
2023-09-28 15:59 ` [PATCH v5 4/4] remoteproc: zynqmp: parse TCM from device tree Tanmay Shah
3 siblings, 1 reply; 15+ messages in thread
From: Tanmay Shah @ 2023-09-28 15:58 UTC (permalink / raw)
To: andersson, mathieu.poirier, robh+dt, krzysztof.kozlowski+dt,
michal.simek
Cc: radhey.shyam.pandey, tanmay.shah, ben.levinsky, linux-remoteproc,
devicetree, linux-arm-kernel
Use TCM pm domains extracted from device-tree
to power on/off TCM using general pm domain framework
Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
---
drivers/remoteproc/xlnx_r5_remoteproc.c | 224 ++++++++++++++++++++++--
1 file changed, 205 insertions(+), 19 deletions(-)
diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
index 4395edea9a64..27ed2c070ebb 100644
--- a/drivers/remoteproc/xlnx_r5_remoteproc.c
+++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
@@ -16,6 +16,7 @@
#include <linux/of_reserved_mem.h>
#include <linux/platform_device.h>
#include <linux/remoteproc.h>
+#include <linux/pm_domain.h>
#include "remoteproc_internal.h"
@@ -102,6 +103,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
* @rproc: rproc handle
* @pm_domain_id: RPU CPU power domain id
* @ipi: pointer to mailbox information
+ * @num_pm_dev: number of tcm pm domain devices for this core
+ * @pm_dev1: pm domain virtual devices for power domain framework
+ * @pm_dev_link1: pm domain device links after registration
+ * @pm_dev2: used only in lockstep mode. second core's pm domain virtual devices
+ * @pm_dev_link2: used only in lockstep mode. second core's pm device links after
+ * registration
*/
struct zynqmp_r5_core {
struct device *dev;
@@ -111,6 +118,11 @@ struct zynqmp_r5_core {
struct rproc *rproc;
u32 pm_domain_id;
struct mbox_info *ipi;
+ int num_pm_dev;
+ struct device **pm_dev1;
+ struct device_link **pm_dev_link1;
+ struct device **pm_dev2;
+ struct device_link **pm_dev_link2;
};
/**
@@ -575,12 +587,21 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
bank_size = r5_core->tcm_banks[i]->size;
pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
- ret = zynqmp_pm_request_node(pm_domain_id,
- ZYNQMP_PM_CAPABILITY_ACCESS, 0,
- ZYNQMP_PM_REQUEST_ACK_BLOCKING);
- if (ret < 0) {
- dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
- goto release_tcm_split;
+ /*
+ * If TCM information is available in device-tree then
+ * in that case, pm domain framework will power on/off TCM.
+ * In that case pm_domain_id is set to 0. If hardcode
+ * bindings from driver is used, then only this driver will
+ * use pm_domain_id.
+ */
+ if (pm_domain_id) {
+ ret = zynqmp_pm_request_node(pm_domain_id,
+ ZYNQMP_PM_CAPABILITY_ACCESS, 0,
+ ZYNQMP_PM_REQUEST_ACK_BLOCKING);
+ if (ret < 0) {
+ dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
+ goto release_tcm_split;
+ }
}
dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
@@ -646,13 +667,16 @@ static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
for (i = 0; i < num_banks; i++) {
pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
- /* Turn on each TCM bank individually */
- ret = zynqmp_pm_request_node(pm_domain_id,
- ZYNQMP_PM_CAPABILITY_ACCESS, 0,
- ZYNQMP_PM_REQUEST_ACK_BLOCKING);
- if (ret < 0) {
- dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
- goto release_tcm_lockstep;
+ if (pm_domain_id) {
+ /* Turn on each TCM bank individually */
+ ret = zynqmp_pm_request_node(pm_domain_id,
+ ZYNQMP_PM_CAPABILITY_ACCESS, 0,
+ ZYNQMP_PM_REQUEST_ACK_BLOCKING);
+ if (ret < 0) {
+ dev_err(dev, "failed to turn on TCM 0x%x",
+ pm_domain_id);
+ goto release_tcm_lockstep;
+ }
}
bank_size = r5_core->tcm_banks[i]->size;
@@ -687,8 +711,10 @@ static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
/* If failed, Turn off all TCM banks turned on before */
for (i--; i >= 0; i--) {
pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
- zynqmp_pm_release_node(pm_domain_id);
+ if (pm_domain_id)
+ zynqmp_pm_release_node(pm_domain_id);
}
+
return ret;
}
@@ -758,6 +784,153 @@ static int zynqmp_r5_parse_fw(struct rproc *rproc, const struct firmware *fw)
return ret;
}
+static void zynqmp_r5_remove_pm_domains(struct rproc *rproc)
+{
+ struct zynqmp_r5_core *r5_core = rproc->priv;
+ struct device *dev = r5_core->dev;
+ struct zynqmp_r5_cluster *cluster;
+ int i;
+
+ cluster = platform_get_drvdata(to_platform_device(dev->parent));
+
+ for (i = 0; i < r5_core->num_pm_dev; i++) {
+ if (r5_core->pm_dev_link1 && r5_core->pm_dev_link1[i])
+ device_link_del(r5_core->pm_dev_link1[i]);
+ if (r5_core->pm_dev1 && !IS_ERR_OR_NULL(r5_core->pm_dev1[i]))
+ dev_pm_domain_detach(r5_core->pm_dev1[i], false);
+ }
+
+ kfree(r5_core->pm_dev1);
+ r5_core->pm_dev1 = NULL;
+ kfree(r5_core->pm_dev_link1);
+ r5_core->pm_dev_link1 = NULL;
+
+ if (cluster->mode == SPLIT_MODE) {
+ r5_core->num_pm_dev = 0;
+ return;
+ }
+
+ for (i = 0; i < r5_core->num_pm_dev; i++) {
+ if (r5_core->pm_dev_link2 && r5_core->pm_dev_link2[i])
+ device_link_del(r5_core->pm_dev_link2[i]);
+ if (r5_core->pm_dev2 && !IS_ERR_OR_NULL(r5_core->pm_dev2[i]))
+ dev_pm_domain_detach(r5_core->pm_dev2[i], false);
+ }
+
+ kfree(r5_core->pm_dev2);
+ r5_core->pm_dev2 = NULL;
+ kfree(r5_core->pm_dev_link2);
+ r5_core->pm_dev_link2 = NULL;
+ r5_core->num_pm_dev = 0;
+}
+
+static int zynqmp_r5_add_pm_domains(struct rproc *rproc)
+{
+ struct zynqmp_r5_core *r5_core = rproc->priv;
+ struct device *dev = r5_core->dev, *dev2;
+ struct zynqmp_r5_cluster *cluster;
+ struct platform_device *pdev;
+ struct device_node *np;
+ int i, num_pm_dev, ret;
+
+ cluster = platform_get_drvdata(to_platform_device(dev->parent));
+
+ /* get number of power-domains */
+ num_pm_dev = of_count_phandle_with_args(r5_core->np, "power-domains",
+ "#power-domain-cells");
+
+ if (num_pm_dev <= 0)
+ return -EINVAL;
+
+ r5_core->pm_dev1 = kcalloc(num_pm_dev,
+ sizeof(struct device *),
+ GFP_KERNEL);
+ if (!r5_core->pm_dev1)
+ ret = -ENOMEM;
+
+ r5_core->pm_dev_link1 = kcalloc(num_pm_dev,
+ sizeof(struct device_link *),
+ GFP_KERNEL);
+ if (!r5_core->pm_dev_link1)
+ return -ENOMEM;
+
+ r5_core->num_pm_dev = num_pm_dev;
+
+ /* for zynqmp we only add TCM power domains and not core's power domain */
+ for (i = 1; i < r5_core->num_pm_dev; i++) {
+ r5_core->pm_dev1[i] = dev_pm_domain_attach_by_id(dev, i);
+ if (IS_ERR(r5_core->pm_dev1[i])) {
+ dev_dbg(dev, "failed to attach pm domain %d\n", i);
+ return PTR_ERR(r5_core->pm_dev1[i]);
+ }
+ if (!r5_core->pm_dev1[i]) {
+ dev_dbg(dev, "can't attach to pm domain %d\n", i);
+ return -EINVAL;
+ }
+
+ r5_core->pm_dev_link1[i] = device_link_add(dev, r5_core->pm_dev1[i],
+ DL_FLAG_STATELESS |
+ DL_FLAG_RPM_ACTIVE |
+ DL_FLAG_PM_RUNTIME);
+ if (!r5_core->pm_dev_link1[i]) {
+ dev_pm_domain_detach(r5_core->pm_dev1[i], true);
+ r5_core->pm_dev1[i] = NULL;
+ return -EINVAL;
+ }
+ }
+
+ if (cluster->mode == SPLIT_MODE)
+ return 0;
+
+ r5_core->pm_dev2 = kcalloc(num_pm_dev,
+ sizeof(struct device *),
+ GFP_KERNEL);
+ if (!r5_core->pm_dev2)
+ return -ENOMEM;
+
+ r5_core->pm_dev_link2 = kcalloc(num_pm_dev,
+ sizeof(struct device_link *),
+ GFP_KERNEL);
+ if (!r5_core->pm_dev_link2)
+ return -ENOMEM;
+
+ /* get second core's device to detach its power-domains */
+ np = of_get_next_child(cluster->dev->of_node, of_node_get(dev->of_node));
+
+ pdev = of_find_device_by_node(np);
+ if (!pdev) {
+ dev_err(cluster->dev, "core1 platform device not available\n");
+ return -EINVAL;
+ }
+
+ dev2 = &pdev->dev;
+
+ /* for zynqmp we only add TCM power domains and not core's power domain */
+ for (i = 1; i < r5_core->num_pm_dev; i++) {
+ r5_core->pm_dev2[i] = dev_pm_domain_attach_by_id(dev2, i);
+ if (IS_ERR(r5_core->pm_dev2[i])) {
+ dev_dbg(dev, "can't attach to pm domain %d\n", i);
+ return PTR_ERR(r5_core->pm_dev2[i]);
+ }
+ if (!r5_core->pm_dev2[i]) {
+ dev_dbg(dev, "can't attach to pm domain %d\n", i);
+ return -EINVAL;
+ }
+
+ r5_core->pm_dev_link2[i] = device_link_add(dev, r5_core->pm_dev2[i],
+ DL_FLAG_STATELESS |
+ DL_FLAG_RPM_ACTIVE |
+ DL_FLAG_PM_RUNTIME);
+ if (!r5_core->pm_dev_link2[i]) {
+ dev_pm_domain_detach(r5_core->pm_dev2[i], true);
+ r5_core->pm_dev2[i] = NULL;
+ return -ENODEV;
+ }
+ }
+
+ return 0;
+}
+
/**
* zynqmp_r5_rproc_prepare()
* adds carveouts for TCM bank and reserved memory regions
@@ -770,19 +943,30 @@ static int zynqmp_r5_rproc_prepare(struct rproc *rproc)
{
int ret;
+ ret = zynqmp_r5_add_pm_domains(rproc);
+ if (ret) {
+ dev_err(&rproc->dev, "failed to add pm domains\n");
+ goto fail_prepare;
+ }
+
ret = add_tcm_banks(rproc);
if (ret) {
dev_err(&rproc->dev, "failed to get TCM banks, err %d\n", ret);
- return ret;
+ goto fail_prepare;
}
ret = add_mem_regions_carveout(rproc);
if (ret) {
dev_err(&rproc->dev, "failed to get reserve mem regions %d\n", ret);
- return ret;
+ goto fail_prepare;
}
return 0;
+
+fail_prepare:
+ zynqmp_r5_remove_pm_domains(rproc);
+
+ return ret;
}
/**
@@ -801,11 +985,13 @@ static int zynqmp_r5_rproc_unprepare(struct rproc *rproc)
r5_core = rproc->priv;
+ zynqmp_r5_remove_pm_domains(rproc);
+
for (i = 0; i < r5_core->tcm_bank_count; i++) {
pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
- if (zynqmp_pm_release_node(pm_domain_id))
- dev_warn(r5_core->dev,
- "can't turn off TCM bank 0x%x", pm_domain_id);
+ if (pm_domain_id && zynqmp_pm_release_node(pm_domain_id))
+ dev_dbg(r5_core->dev,
+ "can't turn off TCM bank 0x%x", pm_domain_id);
}
return 0;
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v5 3/4] remoteproc: zynqmp: add pm domains support
2023-09-28 15:58 ` [PATCH v5 3/4] remoteproc: zynqmp: add pm domains support Tanmay Shah
@ 2023-10-02 17:11 ` Mathieu Poirier
2023-10-02 20:39 ` Tanmay Shah
0 siblings, 1 reply; 15+ messages in thread
From: Mathieu Poirier @ 2023-10-02 17:11 UTC (permalink / raw)
To: Tanmay Shah
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On Thu, Sep 28, 2023 at 08:58:59AM -0700, Tanmay Shah wrote:
> Use TCM pm domains extracted from device-tree
> to power on/off TCM using general pm domain framework
>
> Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> ---
> drivers/remoteproc/xlnx_r5_remoteproc.c | 224 ++++++++++++++++++++++--
> 1 file changed, 205 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> index 4395edea9a64..27ed2c070ebb 100644
> --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> @@ -16,6 +16,7 @@
> #include <linux/of_reserved_mem.h>
> #include <linux/platform_device.h>
> #include <linux/remoteproc.h>
> +#include <linux/pm_domain.h>
>
> #include "remoteproc_internal.h"
>
> @@ -102,6 +103,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
> * @rproc: rproc handle
> * @pm_domain_id: RPU CPU power domain id
> * @ipi: pointer to mailbox information
> + * @num_pm_dev: number of tcm pm domain devices for this core
> + * @pm_dev1: pm domain virtual devices for power domain framework
> + * @pm_dev_link1: pm domain device links after registration
> + * @pm_dev2: used only in lockstep mode. second core's pm domain virtual devices
> + * @pm_dev_link2: used only in lockstep mode. second core's pm device links after
> + * registration
> */
> struct zynqmp_r5_core {
> struct device *dev;
> @@ -111,6 +118,11 @@ struct zynqmp_r5_core {
> struct rproc *rproc;
> u32 pm_domain_id;
> struct mbox_info *ipi;
> + int num_pm_dev;
> + struct device **pm_dev1;
> + struct device_link **pm_dev_link1;
> + struct device **pm_dev2;
> + struct device_link **pm_dev_link2;
> };
>
> /**
> @@ -575,12 +587,21 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> bank_size = r5_core->tcm_banks[i]->size;
> pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
>
> - ret = zynqmp_pm_request_node(pm_domain_id,
> - ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> - ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> - if (ret < 0) {
> - dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> - goto release_tcm_split;
> + /*
> + * If TCM information is available in device-tree then
> + * in that case, pm domain framework will power on/off TCM.
> + * In that case pm_domain_id is set to 0. If hardcode
> + * bindings from driver is used, then only this driver will
> + * use pm_domain_id.
> + */
> + if (pm_domain_id) {
> + ret = zynqmp_pm_request_node(pm_domain_id,
> + ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> + ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> + if (ret < 0) {
> + dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> + goto release_tcm_split;
> + }
> }
>
> dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
> @@ -646,13 +667,16 @@ static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
> for (i = 0; i < num_banks; i++) {
> pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
>
> - /* Turn on each TCM bank individually */
> - ret = zynqmp_pm_request_node(pm_domain_id,
> - ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> - ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> - if (ret < 0) {
> - dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> - goto release_tcm_lockstep;
> + if (pm_domain_id) {
> + /* Turn on each TCM bank individually */
> + ret = zynqmp_pm_request_node(pm_domain_id,
> + ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> + ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> + if (ret < 0) {
> + dev_err(dev, "failed to turn on TCM 0x%x",
> + pm_domain_id);
> + goto release_tcm_lockstep;
> + }
> }
>
> bank_size = r5_core->tcm_banks[i]->size;
> @@ -687,8 +711,10 @@ static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
> /* If failed, Turn off all TCM banks turned on before */
> for (i--; i >= 0; i--) {
> pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> - zynqmp_pm_release_node(pm_domain_id);
> + if (pm_domain_id)
> + zynqmp_pm_release_node(pm_domain_id);
> }
> +
Spurious change
> return ret;
> }
>
> @@ -758,6 +784,153 @@ static int zynqmp_r5_parse_fw(struct rproc *rproc, const struct firmware *fw)
> return ret;
> }
>
> +static void zynqmp_r5_remove_pm_domains(struct rproc *rproc)
> +{
> + struct zynqmp_r5_core *r5_core = rproc->priv;
> + struct device *dev = r5_core->dev;
> + struct zynqmp_r5_cluster *cluster;
> + int i;
> +
> + cluster = platform_get_drvdata(to_platform_device(dev->parent));
> +
> + for (i = 0; i < r5_core->num_pm_dev; i++) {
> + if (r5_core->pm_dev_link1 && r5_core->pm_dev_link1[i])
> + device_link_del(r5_core->pm_dev_link1[i]);
> + if (r5_core->pm_dev1 && !IS_ERR_OR_NULL(r5_core->pm_dev1[i]))
> + dev_pm_domain_detach(r5_core->pm_dev1[i], false);
> + }
> +
A global function such as this one should not have to deal with error
conditions. Those should be dealt with in the allocation function where cleanup
is done on error conditions. See my comment below in
zynqmp_r5_add_pm_domains().
> + kfree(r5_core->pm_dev1);
> + r5_core->pm_dev1 = NULL;
> + kfree(r5_core->pm_dev_link1);
> + r5_core->pm_dev_link1 = NULL;
> +
> + if (cluster->mode == SPLIT_MODE) {
> + r5_core->num_pm_dev = 0;
> + return;
> + }
> +
> + for (i = 0; i < r5_core->num_pm_dev; i++) {
> + if (r5_core->pm_dev_link2 && r5_core->pm_dev_link2[i])
> + device_link_del(r5_core->pm_dev_link2[i]);
> + if (r5_core->pm_dev2 && !IS_ERR_OR_NULL(r5_core->pm_dev2[i]))
> + dev_pm_domain_detach(r5_core->pm_dev2[i], false);
> + }
> +
> + kfree(r5_core->pm_dev2);
> + r5_core->pm_dev2 = NULL;
> + kfree(r5_core->pm_dev_link2);
> + r5_core->pm_dev_link2 = NULL;
> + r5_core->num_pm_dev = 0;
> +}
> +
> +static int zynqmp_r5_add_pm_domains(struct rproc *rproc)
> +{
> + struct zynqmp_r5_core *r5_core = rproc->priv;
> + struct device *dev = r5_core->dev, *dev2;
> + struct zynqmp_r5_cluster *cluster;
> + struct platform_device *pdev;
> + struct device_node *np;
> + int i, num_pm_dev, ret;
I'm not sure 'ret' is needed - see below.
> +
> + cluster = platform_get_drvdata(to_platform_device(dev->parent));
Why not use dev_get_drvdata() as it is done elsewhere in this driver?
> +
> + /* get number of power-domains */
> + num_pm_dev = of_count_phandle_with_args(r5_core->np, "power-domains",
> + "#power-domain-cells");
> +
> + if (num_pm_dev <= 0)
> + return -EINVAL;
> +
> + r5_core->pm_dev1 = kcalloc(num_pm_dev,
> + sizeof(struct device *),
> + GFP_KERNEL);
> + if (!r5_core->pm_dev1)
> + ret = -ENOMEM;
What's the goal of the assignment? Did you mean to return an error instead?
> +
> + r5_core->pm_dev_link1 = kcalloc(num_pm_dev,
> + sizeof(struct device_link *),
> + GFP_KERNEL);
> + if (!r5_core->pm_dev_link1)
> + return -ENOMEM;
In case of error, always cleanup the work done in the current function. That
way cleanup functions such as zynqmp_r5_remove_pm_domains() are simple and easy
to read.
> +
> + r5_core->num_pm_dev = num_pm_dev;
> +
> + /* for zynqmp we only add TCM power domains and not core's power domain */
> + for (i = 1; i < r5_core->num_pm_dev; i++) {
> + r5_core->pm_dev1[i] = dev_pm_domain_attach_by_id(dev, i);
> + if (IS_ERR(r5_core->pm_dev1[i])) {
> + dev_dbg(dev, "failed to attach pm domain %d\n", i);
> + return PTR_ERR(r5_core->pm_dev1[i]);
> + }
> + if (!r5_core->pm_dev1[i]) {
> + dev_dbg(dev, "can't attach to pm domain %d\n", i);
> + return -EINVAL;
> + }
> +
> + r5_core->pm_dev_link1[i] = device_link_add(dev, r5_core->pm_dev1[i],
> + DL_FLAG_STATELESS |
> + DL_FLAG_RPM_ACTIVE |
> + DL_FLAG_PM_RUNTIME);
> + if (!r5_core->pm_dev_link1[i]) {
> + dev_pm_domain_detach(r5_core->pm_dev1[i], true);
> + r5_core->pm_dev1[i] = NULL;
> + return -EINVAL;
> + }
> + }
> +
> + if (cluster->mode == SPLIT_MODE)
> + return 0;
> +
> + r5_core->pm_dev2 = kcalloc(num_pm_dev,
> + sizeof(struct device *),
> + GFP_KERNEL);
> + if (!r5_core->pm_dev2)
> + return -ENOMEM;
> +
> + r5_core->pm_dev_link2 = kcalloc(num_pm_dev,
> + sizeof(struct device_link *),
> + GFP_KERNEL);
> + if (!r5_core->pm_dev_link2)
> + return -ENOMEM;
> +
> + /* get second core's device to detach its power-domains */
> + np = of_get_next_child(cluster->dev->of_node, of_node_get(dev->of_node));
> +
> + pdev = of_find_device_by_node(np);
> + if (!pdev) {
> + dev_err(cluster->dev, "core1 platform device not available\n");
> + return -EINVAL;
> + }
> +
> + dev2 = &pdev->dev;
> +
> + /* for zynqmp we only add TCM power domains and not core's power domain */
> + for (i = 1; i < r5_core->num_pm_dev; i++) {
> + r5_core->pm_dev2[i] = dev_pm_domain_attach_by_id(dev2, i);
> + if (IS_ERR(r5_core->pm_dev2[i])) {
> + dev_dbg(dev, "can't attach to pm domain %d\n", i);
> + return PTR_ERR(r5_core->pm_dev2[i]);
> + }
> + if (!r5_core->pm_dev2[i]) {
> + dev_dbg(dev, "can't attach to pm domain %d\n", i);
> + return -EINVAL;
> + }
> +
> + r5_core->pm_dev_link2[i] = device_link_add(dev, r5_core->pm_dev2[i],
> + DL_FLAG_STATELESS |
> + DL_FLAG_RPM_ACTIVE |
> + DL_FLAG_PM_RUNTIME);
> + if (!r5_core->pm_dev_link2[i]) {
> + dev_pm_domain_detach(r5_core->pm_dev2[i], true);
> + r5_core->pm_dev2[i] = NULL;
> + return -ENODEV;
> + }
> + }
> +
> + return 0;
> +}
> +
> /**
> * zynqmp_r5_rproc_prepare()
> * adds carveouts for TCM bank and reserved memory regions
> @@ -770,19 +943,30 @@ static int zynqmp_r5_rproc_prepare(struct rproc *rproc)
> {
> int ret;
>
> + ret = zynqmp_r5_add_pm_domains(rproc);
> + if (ret) {
> + dev_err(&rproc->dev, "failed to add pm domains\n");
> + goto fail_prepare;
> + }
> +
> ret = add_tcm_banks(rproc);
> if (ret) {
> dev_err(&rproc->dev, "failed to get TCM banks, err %d\n", ret);
> - return ret;
> + goto fail_prepare;
> }
>
> ret = add_mem_regions_carveout(rproc);
> if (ret) {
> dev_err(&rproc->dev, "failed to get reserve mem regions %d\n", ret);
> - return ret;
> + goto fail_prepare;
> }
>
> return 0;
> +
> +fail_prepare:
> + zynqmp_r5_remove_pm_domains(rproc);
> +
> + return ret;
> }
>
> /**
> @@ -801,11 +985,13 @@ static int zynqmp_r5_rproc_unprepare(struct rproc *rproc)
>
> r5_core = rproc->priv;
>
> + zynqmp_r5_remove_pm_domains(rproc);
> +
> for (i = 0; i < r5_core->tcm_bank_count; i++) {
> pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> - if (zynqmp_pm_release_node(pm_domain_id))
> - dev_warn(r5_core->dev,
> - "can't turn off TCM bank 0x%x", pm_domain_id);
> + if (pm_domain_id && zynqmp_pm_release_node(pm_domain_id))
> + dev_dbg(r5_core->dev,
> + "can't turn off TCM bank 0x%x", pm_domain_id);
> }
>
> return 0;
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v5 3/4] remoteproc: zynqmp: add pm domains support
2023-10-02 17:11 ` Mathieu Poirier
@ 2023-10-02 20:39 ` Tanmay Shah
0 siblings, 0 replies; 15+ messages in thread
From: Tanmay Shah @ 2023-10-02 20:39 UTC (permalink / raw)
To: Mathieu Poirier
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
Hello Mathiew,
Thanks for reviews.
On 10/2/23 12:11 PM, Mathieu Poirier wrote:
> On Thu, Sep 28, 2023 at 08:58:59AM -0700, Tanmay Shah wrote:
> > Use TCM pm domains extracted from device-tree
> > to power on/off TCM using general pm domain framework
> >
> > Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> > ---
> > drivers/remoteproc/xlnx_r5_remoteproc.c | 224 ++++++++++++++++++++++--
> > 1 file changed, 205 insertions(+), 19 deletions(-)
> >
> > diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > index 4395edea9a64..27ed2c070ebb 100644
> > --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> > +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > @@ -16,6 +16,7 @@
> > #include <linux/of_reserved_mem.h>
> > #include <linux/platform_device.h>
> > #include <linux/remoteproc.h>
> > +#include <linux/pm_domain.h>
> >
> > #include "remoteproc_internal.h"
> >
> > @@ -102,6 +103,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
> > * @rproc: rproc handle
> > * @pm_domain_id: RPU CPU power domain id
> > * @ipi: pointer to mailbox information
> > + * @num_pm_dev: number of tcm pm domain devices for this core
> > + * @pm_dev1: pm domain virtual devices for power domain framework
> > + * @pm_dev_link1: pm domain device links after registration
> > + * @pm_dev2: used only in lockstep mode. second core's pm domain virtual devices
> > + * @pm_dev_link2: used only in lockstep mode. second core's pm device links after
> > + * registration
> > */
> > struct zynqmp_r5_core {
> > struct device *dev;
> > @@ -111,6 +118,11 @@ struct zynqmp_r5_core {
> > struct rproc *rproc;
> > u32 pm_domain_id;
> > struct mbox_info *ipi;
> > + int num_pm_dev;
> > + struct device **pm_dev1;
> > + struct device_link **pm_dev_link1;
> > + struct device **pm_dev2;
> > + struct device_link **pm_dev_link2;
> > };
> >
> > /**
> > @@ -575,12 +587,21 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > bank_size = r5_core->tcm_banks[i]->size;
> > pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> >
> > - ret = zynqmp_pm_request_node(pm_domain_id,
> > - ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> > - ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > - if (ret < 0) {
> > - dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> > - goto release_tcm_split;
> > + /*
> > + * If TCM information is available in device-tree then
> > + * in that case, pm domain framework will power on/off TCM.
> > + * In that case pm_domain_id is set to 0. If hardcode
> > + * bindings from driver is used, then only this driver will
> > + * use pm_domain_id.
> > + */
> > + if (pm_domain_id) {
> > + ret = zynqmp_pm_request_node(pm_domain_id,
> > + ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> > + ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > + if (ret < 0) {
> > + dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> > + goto release_tcm_split;
> > + }
> > }
> >
> > dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
> > @@ -646,13 +667,16 @@ static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
> > for (i = 0; i < num_banks; i++) {
> > pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> >
> > - /* Turn on each TCM bank individually */
> > - ret = zynqmp_pm_request_node(pm_domain_id,
> > - ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> > - ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > - if (ret < 0) {
> > - dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> > - goto release_tcm_lockstep;
> > + if (pm_domain_id) {
> > + /* Turn on each TCM bank individually */
> > + ret = zynqmp_pm_request_node(pm_domain_id,
> > + ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> > + ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > + if (ret < 0) {
> > + dev_err(dev, "failed to turn on TCM 0x%x",
> > + pm_domain_id);
> > + goto release_tcm_lockstep;
> > + }
> > }
> >
> > bank_size = r5_core->tcm_banks[i]->size;
> > @@ -687,8 +711,10 @@ static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
> > /* If failed, Turn off all TCM banks turned on before */
> > for (i--; i >= 0; i--) {
> > pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > - zynqmp_pm_release_node(pm_domain_id);
> > + if (pm_domain_id)
> > + zynqmp_pm_release_node(pm_domain_id);
> > }
> > +
>
> Spurious change
Ack, will remove it.
>
> > return ret;
> > }
> >
> > @@ -758,6 +784,153 @@ static int zynqmp_r5_parse_fw(struct rproc *rproc, const struct firmware *fw)
> > return ret;
> > }
> >
> > +static void zynqmp_r5_remove_pm_domains(struct rproc *rproc)
> > +{
> > + struct zynqmp_r5_core *r5_core = rproc->priv;
> > + struct device *dev = r5_core->dev;
> > + struct zynqmp_r5_cluster *cluster;
> > + int i;
> > +
> > + cluster = platform_get_drvdata(to_platform_device(dev->parent));
> > +
> > + for (i = 0; i < r5_core->num_pm_dev; i++) {
> > + if (r5_core->pm_dev_link1 && r5_core->pm_dev_link1[i])
> > + device_link_del(r5_core->pm_dev_link1[i]);
> > + if (r5_core->pm_dev1 && !IS_ERR_OR_NULL(r5_core->pm_dev1[i]))
> > + dev_pm_domain_detach(r5_core->pm_dev1[i], false);
> > + }
> > +
>
> A global function such as this one should not have to deal with error
> conditions. Those should be dealt with in the allocation function where cleanup
> is done on error conditions. See my comment below in
> zynqmp_r5_add_pm_domains().
Ack, I won't use IS_ERR_OR_NULL here. Instead will make sure that if this function is called,
all the pm domains are acquired successfully.
> > + kfree(r5_core->pm_dev1);
> > + r5_core->pm_dev1 = NULL;
> > + kfree(r5_core->pm_dev_link1);
> > + r5_core->pm_dev_link1 = NULL;
> > +
> > + if (cluster->mode == SPLIT_MODE) {
> > + r5_core->num_pm_dev = 0;
> > + return;
> > + }
> > +
> > + for (i = 0; i < r5_core->num_pm_dev; i++) {
> > + if (r5_core->pm_dev_link2 && r5_core->pm_dev_link2[i])
> > + device_link_del(r5_core->pm_dev_link2[i]);
> > + if (r5_core->pm_dev2 && !IS_ERR_OR_NULL(r5_core->pm_dev2[i]))
> > + dev_pm_domain_detach(r5_core->pm_dev2[i], false);
> > + }
> > +
> > + kfree(r5_core->pm_dev2);
> > + r5_core->pm_dev2 = NULL;
> > + kfree(r5_core->pm_dev_link2);
> > + r5_core->pm_dev_link2 = NULL;
> > + r5_core->num_pm_dev = 0;
> > +}
> > +
> > +static int zynqmp_r5_add_pm_domains(struct rproc *rproc)
> > +{
> > + struct zynqmp_r5_core *r5_core = rproc->priv;
> > + struct device *dev = r5_core->dev, *dev2;
> > + struct zynqmp_r5_cluster *cluster;
> > + struct platform_device *pdev;
> > + struct device_node *np;
> > + int i, num_pm_dev, ret;
>
> I'm not sure 'ret' is needed - see below.
You are right. It's not needed. will fix it.
>
> > +
> > + cluster = platform_get_drvdata(to_platform_device(dev->parent));
>
> Why not use dev_get_drvdata() as it is done elsewhere in this driver?
>
> > +
> > + /* get number of power-domains */
> > + num_pm_dev = of_count_phandle_with_args(r5_core->np, "power-domains",
> > + "#power-domain-cells");
> > +
> > + if (num_pm_dev <= 0)
> > + return -EINVAL;
> > +
> > + r5_core->pm_dev1 = kcalloc(num_pm_dev,
> > + sizeof(struct device *),
> > + GFP_KERNEL);
> > + if (!r5_core->pm_dev1)
> > + ret = -ENOMEM;
>
> What's the goal of the assignment? Did you mean to return an error instead?
Ack, it should have been return. I implemented multiple ways to return errors, and missed to fix this before sending the patch.
>
> > +
> > + r5_core->pm_dev_link1 = kcalloc(num_pm_dev,
> > + sizeof(struct device_link *),
> > + GFP_KERNEL);
> > + if (!r5_core->pm_dev_link1)
> > + return -ENOMEM;
>
> In case of error, always cleanup the work done in the current function. That
> way cleanup functions such as zynqmp_r5_remove_pm_domains() are simple and easy
> to read.
Ack. I will make sure that if we fail here for any reason, all the memory allocated is freed
here. Also any acquired devices and pm domains should be detached and released in this function
only upon failure in between.
Thanks,
Tanmay
>
> > +
> > + r5_core->num_pm_dev = num_pm_dev;
> > +
> > + /* for zynqmp we only add TCM power domains and not core's power domain */
> > + for (i = 1; i < r5_core->num_pm_dev; i++) {
> > + r5_core->pm_dev1[i] = dev_pm_domain_attach_by_id(dev, i);
> > + if (IS_ERR(r5_core->pm_dev1[i])) {
> > + dev_dbg(dev, "failed to attach pm domain %d\n", i);
> > + return PTR_ERR(r5_core->pm_dev1[i]);
> > + }
> > + if (!r5_core->pm_dev1[i]) {
> > + dev_dbg(dev, "can't attach to pm domain %d\n", i);
> > + return -EINVAL;
> > + }
> > +
> > + r5_core->pm_dev_link1[i] = device_link_add(dev, r5_core->pm_dev1[i],
> > + DL_FLAG_STATELESS |
> > + DL_FLAG_RPM_ACTIVE |
> > + DL_FLAG_PM_RUNTIME);
> > + if (!r5_core->pm_dev_link1[i]) {
> > + dev_pm_domain_detach(r5_core->pm_dev1[i], true);
> > + r5_core->pm_dev1[i] = NULL;
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (cluster->mode == SPLIT_MODE)
> > + return 0;
> > +
> > + r5_core->pm_dev2 = kcalloc(num_pm_dev,
> > + sizeof(struct device *),
> > + GFP_KERNEL);
> > + if (!r5_core->pm_dev2)
> > + return -ENOMEM;
> > +
> > + r5_core->pm_dev_link2 = kcalloc(num_pm_dev,
> > + sizeof(struct device_link *),
> > + GFP_KERNEL);
> > + if (!r5_core->pm_dev_link2)
> > + return -ENOMEM;
> > +
> > + /* get second core's device to detach its power-domains */
> > + np = of_get_next_child(cluster->dev->of_node, of_node_get(dev->of_node));
> > +
> > + pdev = of_find_device_by_node(np);
> > + if (!pdev) {
> > + dev_err(cluster->dev, "core1 platform device not available\n");
> > + return -EINVAL;
> > + }
> > +
> > + dev2 = &pdev->dev;
> > +
> > + /* for zynqmp we only add TCM power domains and not core's power domain */
> > + for (i = 1; i < r5_core->num_pm_dev; i++) {
> > + r5_core->pm_dev2[i] = dev_pm_domain_attach_by_id(dev2, i);
> > + if (IS_ERR(r5_core->pm_dev2[i])) {
> > + dev_dbg(dev, "can't attach to pm domain %d\n", i);
> > + return PTR_ERR(r5_core->pm_dev2[i]);
> > + }
> > + if (!r5_core->pm_dev2[i]) {
> > + dev_dbg(dev, "can't attach to pm domain %d\n", i);
> > + return -EINVAL;
> > + }
> > +
> > + r5_core->pm_dev_link2[i] = device_link_add(dev, r5_core->pm_dev2[i],
> > + DL_FLAG_STATELESS |
> > + DL_FLAG_RPM_ACTIVE |
> > + DL_FLAG_PM_RUNTIME);
> > + if (!r5_core->pm_dev_link2[i]) {
> > + dev_pm_domain_detach(r5_core->pm_dev2[i], true);
> > + r5_core->pm_dev2[i] = NULL;
> > + return -ENODEV;
> > + }
> > + }
> > +
> > + return 0;
> > +}
> > +
> > /**
> > * zynqmp_r5_rproc_prepare()
> > * adds carveouts for TCM bank and reserved memory regions
> > @@ -770,19 +943,30 @@ static int zynqmp_r5_rproc_prepare(struct rproc *rproc)
> > {
> > int ret;
> >
> > + ret = zynqmp_r5_add_pm_domains(rproc);
> > + if (ret) {
> > + dev_err(&rproc->dev, "failed to add pm domains\n");
> > + goto fail_prepare;
> > + }
> > +
> > ret = add_tcm_banks(rproc);
> > if (ret) {
> > dev_err(&rproc->dev, "failed to get TCM banks, err %d\n", ret);
> > - return ret;
> > + goto fail_prepare;
> > }
> >
> > ret = add_mem_regions_carveout(rproc);
> > if (ret) {
> > dev_err(&rproc->dev, "failed to get reserve mem regions %d\n", ret);
> > - return ret;
> > + goto fail_prepare;
> > }
> >
> > return 0;
> > +
> > +fail_prepare:
> > + zynqmp_r5_remove_pm_domains(rproc);
> > +
> > + return ret;
> > }
> >
> > /**
> > @@ -801,11 +985,13 @@ static int zynqmp_r5_rproc_unprepare(struct rproc *rproc)
> >
> > r5_core = rproc->priv;
> >
> > + zynqmp_r5_remove_pm_domains(rproc);
> > +
> > for (i = 0; i < r5_core->tcm_bank_count; i++) {
> > pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > - if (zynqmp_pm_release_node(pm_domain_id))
> > - dev_warn(r5_core->dev,
> > - "can't turn off TCM bank 0x%x", pm_domain_id);
> > + if (pm_domain_id && zynqmp_pm_release_node(pm_domain_id))
> > + dev_dbg(r5_core->dev,
> > + "can't turn off TCM bank 0x%x", pm_domain_id);
> > }
> >
> > return 0;
> > --
> > 2.25.1
> >
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v5 4/4] remoteproc: zynqmp: parse TCM from device tree
2023-09-28 15:58 [PATCH v5 0/4] add zynqmp TCM bindings Tanmay Shah
` (2 preceding siblings ...)
2023-09-28 15:58 ` [PATCH v5 3/4] remoteproc: zynqmp: add pm domains support Tanmay Shah
@ 2023-09-28 15:59 ` Tanmay Shah
2023-10-02 17:39 ` Mathieu Poirier
3 siblings, 1 reply; 15+ messages in thread
From: Tanmay Shah @ 2023-09-28 15:59 UTC (permalink / raw)
To: andersson, mathieu.poirier, robh+dt, krzysztof.kozlowski+dt,
michal.simek
Cc: radhey.shyam.pandey, tanmay.shah, ben.levinsky, linux-remoteproc,
devicetree, linux-arm-kernel
ZynqMP TCM information is fixed in driver. Now ZynqMP TCM information
is available in device-tree. Parse TCM information in driver
as per new bindings
Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
---
drivers/remoteproc/xlnx_r5_remoteproc.c | 122 ++++++++++++++++++++++--
1 file changed, 114 insertions(+), 8 deletions(-)
diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
index 27ed2c070ebb..749e9da68906 100644
--- a/drivers/remoteproc/xlnx_r5_remoteproc.c
+++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
@@ -75,8 +75,8 @@ struct mbox_info {
};
/*
- * Hardcoded TCM bank values. This will be removed once TCM bindings are
- * accepted for system-dt specifications and upstreamed in linux kernel
+ * Hardcoded TCM bank values. This will stay in driver to maintain backward
+ * compatibility with device-tree that does not have TCM information.
*/
static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
{0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
@@ -613,7 +613,8 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
bank_name);
if (!rproc_mem) {
ret = -ENOMEM;
- zynqmp_pm_release_node(pm_domain_id);
+ if (pm_domain_id)
+ zynqmp_pm_release_node(pm_domain_id);
goto release_tcm_split;
}
@@ -626,7 +627,8 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
/* If failed, Turn off all TCM banks turned on before */
for (i--; i >= 0; i--) {
pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
- zynqmp_pm_release_node(pm_domain_id);
+ if (pm_domain_id)
+ zynqmp_pm_release_node(pm_domain_id);
}
return ret;
}
@@ -1064,6 +1066,107 @@ static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev)
return ERR_PTR(ret);
}
+static int zynqmp_r5_get_tcm_node_from_dt(struct zynqmp_r5_cluster *cluster)
+{
+ int i, j, tcm_bank_count, ret = -EINVAL;
+ struct zynqmp_r5_core *r5_core;
+ struct platform_device *cpdev;
+ struct resource *res = NULL;
+ u64 abs_addr = 0, size = 0;
+ struct mem_bank_data *tcm;
+ struct device_node *np;
+ struct device *dev;
+
+ for (i = 0; i < cluster->core_count; i++) {
+ r5_core = cluster->r5_cores[i];
+ dev = r5_core->dev;
+ np = dev_of_node(dev);
+
+ /* we have address cell 2 and size cell as 2 */
+ ret = of_property_count_elems_of_size(np, "reg",
+ 4 * sizeof(u32));
+ if (ret == 0) {
+ ret = -EINVAL;
+ goto fail_tcm;
+ }
+ if (ret < 0)
+ goto fail_tcm;
+
+ tcm_bank_count = ret;
+
+ r5_core->tcm_banks = devm_kcalloc(dev, tcm_bank_count,
+ sizeof(struct mem_bank_data *),
+ GFP_KERNEL);
+ if (!r5_core->tcm_banks) {
+ ret = -ENOMEM;
+ goto fail_tcm;
+ }
+
+ r5_core->tcm_bank_count = tcm_bank_count;
+ for (j = 0; j < tcm_bank_count; j++) {
+ tcm = devm_kzalloc(dev, sizeof(struct mem_bank_data *),
+ GFP_KERNEL);
+ if (!tcm) {
+ ret = -ENOMEM;
+ goto fail_tcm;
+ }
+
+ r5_core->tcm_banks[j] = tcm;
+
+ /* get tcm address without translation */
+ ret = of_property_read_reg(np, j, &abs_addr, &size);
+ if (ret) {
+ dev_err(dev, "failed to get reg property\n");
+ goto fail_tcm;
+ }
+
+ /*
+ * remote processor can address only 32 bits
+ * so convert 64-bits into 32-bits. This will discard
+ * any unwanted upper 32-bits.
+ */
+ tcm->da = (u32)abs_addr;
+ tcm->size = (u32)size;
+
+ cpdev = to_platform_device(dev);
+ res = platform_get_resource(cpdev, IORESOURCE_MEM, j);
+ if (!res) {
+ dev_err(dev, "failed to get tcm resource\n");
+ ret = -EINVAL;
+ goto fail_tcm;
+ }
+
+ tcm->addr = (u32)res->start;
+ tcm->bank_name = (char *)res->name;
+ res = devm_request_mem_region(dev, tcm->addr, tcm->size,
+ tcm->bank_name);
+ if (!res) {
+ dev_err(dev, "failed to request tcm resource\n");
+ ret = -EINVAL;
+ goto fail_tcm;
+ }
+ }
+ }
+
+ return 0;
+
+fail_tcm:
+ while (i >= 0) {
+ r5_core = cluster->r5_cores[i];
+ for (j = 0; j < r5_core->tcm_bank_count; j++) {
+ if (!r5_core->tcm_banks || !r5_core->tcm_banks[j])
+ continue;
+ tcm = r5_core->tcm_banks[j];
+ devm_kfree(r5_core->dev, tcm);
+ }
+ devm_kfree(r5_core->dev, r5_core->tcm_banks);
+ r5_core->tcm_banks = NULL;
+ i--;
+ }
+
+ return ret;
+}
+
/**
* zynqmp_r5_get_tcm_node()
* Ideally this function should parse tcm node and store information
@@ -1142,10 +1245,13 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster,
struct zynqmp_r5_core *r5_core;
int ret, i;
- ret = zynqmp_r5_get_tcm_node(cluster);
- if (ret < 0) {
- dev_err(dev, "can't get tcm node, err %d\n", ret);
- return ret;
+ ret = zynqmp_r5_get_tcm_node_from_dt(cluster);
+ if (ret) {
+ ret = zynqmp_r5_get_tcm_node(cluster);
+ if (ret < 0) {
+ dev_err(dev, "can't get tcm node, err %d\n", ret);
+ return ret;
+ }
}
for (i = 0; i < cluster->core_count; i++) {
--
2.25.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v5 4/4] remoteproc: zynqmp: parse TCM from device tree
2023-09-28 15:59 ` [PATCH v5 4/4] remoteproc: zynqmp: parse TCM from device tree Tanmay Shah
@ 2023-10-02 17:39 ` Mathieu Poirier
0 siblings, 0 replies; 15+ messages in thread
From: Mathieu Poirier @ 2023-10-02 17:39 UTC (permalink / raw)
To: Tanmay Shah
Cc: andersson, robh+dt, krzysztof.kozlowski+dt, michal.simek,
radhey.shyam.pandey, ben.levinsky, linux-remoteproc, devicetree,
linux-arm-kernel
On Thu, Sep 28, 2023 at 08:59:00AM -0700, Tanmay Shah wrote:
> ZynqMP TCM information is fixed in driver. Now ZynqMP TCM information
> is available in device-tree. Parse TCM information in driver
> as per new bindings
Missing '.' at the end of the sentence.
>
> Signed-off-by: Tanmay Shah <tanmay.shah@amd.com>
> ---
> drivers/remoteproc/xlnx_r5_remoteproc.c | 122 ++++++++++++++++++++++--
> 1 file changed, 114 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> index 27ed2c070ebb..749e9da68906 100644
> --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> @@ -75,8 +75,8 @@ struct mbox_info {
> };
>
> /*
> - * Hardcoded TCM bank values. This will be removed once TCM bindings are
> - * accepted for system-dt specifications and upstreamed in linux kernel
> + * Hardcoded TCM bank values. This will stay in driver to maintain backward
> + * compatibility with device-tree that does not have TCM information.
> */
> static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
> {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
> @@ -613,7 +613,8 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> bank_name);
> if (!rproc_mem) {
> ret = -ENOMEM;
> - zynqmp_pm_release_node(pm_domain_id);
> + if (pm_domain_id)
> + zynqmp_pm_release_node(pm_domain_id);
> goto release_tcm_split;
> }
>
> @@ -626,7 +627,8 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> /* If failed, Turn off all TCM banks turned on before */
> for (i--; i >= 0; i--) {
> pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> - zynqmp_pm_release_node(pm_domain_id);
> + if (pm_domain_id)
> + zynqmp_pm_release_node(pm_domain_id);
> }
> return ret;
> }
> @@ -1064,6 +1066,107 @@ static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev)
> return ERR_PTR(ret);
> }
>
> +static int zynqmp_r5_get_tcm_node_from_dt(struct zynqmp_r5_cluster *cluster)
> +{
> + int i, j, tcm_bank_count, ret = -EINVAL;
> + struct platform_device *cpdev;
> + struct resource *res = NULL;
> + u64 abs_addr = 0, size = 0;
> + struct mem_bank_data *tcm;
> + struct device_node *np;
> + struct device *dev;
As far as I can tell, none of the above initialization is needed. I have
commented on that before.
> +
> + for (i = 0; i < cluster->core_count; i++) {
> + r5_core = cluster->r5_cores[i];
> + dev = r5_core->dev;
> + np = dev_of_node(dev);
> +
> + /* we have address cell 2 and size cell as 2 */
> + ret = of_property_count_elems_of_size(np, "reg",
> + 4 * sizeof(u32));
> + if (ret == 0) {
> + ret = -EINVAL;
> + goto fail_tcm;
> + }
> + if (ret < 0)
> + goto fail_tcm;
if (ret <= 0) {
ret = -EINVAL;
goto fail_tcm;
}
> +
> + tcm_bank_count = ret;
> +
> + r5_core->tcm_banks = devm_kcalloc(dev, tcm_bank_count,
> + sizeof(struct mem_bank_data *),
> + GFP_KERNEL);
> + if (!r5_core->tcm_banks) {
> + ret = -ENOMEM;
> + goto fail_tcm;
> + }
> +
> + r5_core->tcm_bank_count = tcm_bank_count;
> + for (j = 0; j < tcm_bank_count; j++) {
> + tcm = devm_kzalloc(dev, sizeof(struct mem_bank_data *),
> + GFP_KERNEL);
> + if (!tcm) {
> + ret = -ENOMEM;
> + goto fail_tcm;
> + }
> +
> + r5_core->tcm_banks[j] = tcm;
> +
> + /* get tcm address without translation */
> + ret = of_property_read_reg(np, j, &abs_addr, &size);
> + if (ret) {
> + dev_err(dev, "failed to get reg property\n");
> + goto fail_tcm;
> + }
> +
> + /*
> + * remote processor can address only 32 bits
> + * so convert 64-bits into 32-bits. This will discard
> + * any unwanted upper 32-bits.
> + */
> + tcm->da = (u32)abs_addr;
> + tcm->size = (u32)size;
> +
> + cpdev = to_platform_device(dev);
> + res = platform_get_resource(cpdev, IORESOURCE_MEM, j);
> + if (!res) {
> + dev_err(dev, "failed to get tcm resource\n");
> + ret = -EINVAL;
> + goto fail_tcm;
> + }
> +
> + tcm->addr = (u32)res->start;
> + tcm->bank_name = (char *)res->name;
> + res = devm_request_mem_region(dev, tcm->addr, tcm->size,
> + tcm->bank_name);
> + if (!res) {
> + dev_err(dev, "failed to request tcm resource\n");
> + ret = -EINVAL;
> + goto fail_tcm;
> + }
> + }
> + }
> +
> + return 0;
> +
> +fail_tcm:
> + while (i >= 0) {
> + r5_core = cluster->r5_cores[i];
> + for (j = 0; j < r5_core->tcm_bank_count; j++) {
> + if (!r5_core->tcm_banks || !r5_core->tcm_banks[j])
> + continue;
> + tcm = r5_core->tcm_banks[j];
> + devm_kfree(r5_core->dev, tcm);
> + }
> + devm_kfree(r5_core->dev, r5_core->tcm_banks);
> + r5_core->tcm_banks = NULL;
> + i--;
> + }
Given the devm_xyz() API used in this function, is the above needed?
Thanks,
Mathieu
> +
> + return ret;
> +}
> +
> /**
> * zynqmp_r5_get_tcm_node()
> * Ideally this function should parse tcm node and store information
> @@ -1142,10 +1245,13 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster,
> struct zynqmp_r5_core *r5_core;
> int ret, i;
>
> - ret = zynqmp_r5_get_tcm_node(cluster);
> - if (ret < 0) {
> - dev_err(dev, "can't get tcm node, err %d\n", ret);
> - return ret;
> + ret = zynqmp_r5_get_tcm_node_from_dt(cluster);
> + if (ret) {
> + ret = zynqmp_r5_get_tcm_node(cluster);
> + if (ret < 0) {
> + dev_err(dev, "can't get tcm node, err %d\n", ret);
> + return ret;
> + }
> }
>
> for (i = 0; i < cluster->core_count; i++) {
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 15+ messages in thread