mixing SCMI and PSCI power domain hierarchy

mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman]

Hello SCMI folks,

I'm trying to figure out how to model a power-domain hierarchy when
there is a mixture of SCMI and PSCI domains.

Let's say I have a top-level PD, managed by PSCI, but inside it it has
both a CPU cluster (with cluster & CPU PDs controlled by PSCI) as well
as some other sub-domains for leaf devices that are controlled by SCMI.
A simplified version looks something like this:

SOC
 |
 |- TOP1_PD (PSCI)
     |
     |-- LEAF1_PD (SCMI)
     |-- LEAF2_PD (SCMI)
     |
     \-- CLUSTER_PD (PSCI)
         |
         |-- CPU1_PD (PSCI)
         \-- CPU2_PD (PSCI)

             
So the main question is: how do I describe the SCMI part of this today
in DT?  Currently, I have something like this for the SCMI-controlled
PDs:

	scmi_pds: protocol@11 {
		reg = <0x11>;
		#power-domain-cells = <1>;
		bootph-all;
	};

and the leaf devices under LEAF1_PD and LEAF2_PD would have
`power-domains = <&scmi_pds N>` properties, indicating their PD is
managed by SCMI.  That part works fine.  And I'm also able to model the
CPU PDs and CLUSTER_PD just fine, including using
domain-idle-states... (Hi Ulf ;)

But... how do I describe the relationship of this hierarchy?  In
particular, when the SCMI-controlled PDs are actually subdomains of a
top-level, non-SCMI PD.  I tried adding `power-domains = <&TOP1_PD>`
inside the scmi_pds node, but that property seems to be ignored.  So it
seems that currently it has not been considered to have SCMI PDs as
sub-domains of other PDs.  Is that correct?  Is this something anyone
else has considered adding?

The hierarchy I describe above is not made up, it's for the TI AM62L
SoC, which is in the process of being upstreamed[1], and we're trying to
figure out the proper way to describe the power domain hierarchy of this
SoC in a way that can support the multiple low-power idle states that
the SoC family plans to support.

All suggestions, redirections, corrections welcome!

Thanks,

Kevin

[1]
https://lore.kernel.org/r/20250109-am62lx-v3-0-ef171e789527@ti.com

Re: mixing SCMI and PSCI power domain hierarchy

[Ulf Hansson]

On Tue, 8 Apr 2025 at 01:31, Kevin Hilman <khilman@baylibre.com> wrote:
>
> Hello SCMI folks,
>
> I'm trying to figure out how to model a power-domain hierarchy when
> there is a mixture of SCMI and PSCI domains.
>
> Let's say I have a top-level PD, managed by PSCI, but inside it it has
> both a CPU cluster (with cluster & CPU PDs controlled by PSCI) as well
> as some other sub-domains for leaf devices that are controlled by SCMI.
> A simplified version looks something like this:
>
> SOC
>  |
>  |- TOP1_PD (PSCI)
>      |
>      |-- LEAF1_PD (SCMI)
>      |-- LEAF2_PD (SCMI)
>      |
>      \-- CLUSTER_PD (PSCI)
>          |
>          |-- CPU1_PD (PSCI)
>          \-- CPU2_PD (PSCI)
>
>
> So the main question is: how do I describe the SCMI part of this today
> in DT?  Currently, I have something like this for the SCMI-controlled
> PDs:
>
>         scmi_pds: protocol@11 {
>                 reg = <0x11>;
>                 #power-domain-cells = <1>;
>                 bootph-all;
>         };
>
> and the leaf devices under LEAF1_PD and LEAF2_PD would have
> `power-domains = <&scmi_pds N>` properties, indicating their PD is
> managed by SCMI.  That part works fine.  And I'm also able to model the
> CPU PDs and CLUSTER_PD just fine, including using
> domain-idle-states... (Hi Ulf ;)
>
> But... how do I describe the relationship of this hierarchy?  In
> particular, when the SCMI-controlled PDs are actually subdomains of a
> top-level, non-SCMI PD.  I tried adding `power-domains = <&TOP1_PD>`
> inside the scmi_pds node, but that property seems to be ignored.  So it
> seems that currently it has not been considered to have SCMI PDs as
> sub-domains of other PDs.  Is that correct?  Is this something anyone
> else has considered adding?

I haven't heard of this topology before, but I believe I can
understand the need for it. Especially when using the PSCI OSI mode.

In principle, if any of those LEAF1_PD/LEAF2_PD are being in use we
must not allow the PSCI state to be selected that corresponds to the
TOP1_PD. Right?

I can think of two ways forward, there may be others:
1) Put the leaf devices themselves in TOP1_PD *and* LEAF1_PD/LEAF2_PD.
Thus these devices would have two power-domains. Whether this actually
works for your case, I am not sure of, but I think it could be an
option.

2) Enable parents to be described in the "scmi_pds" node. To do that,
we need to extend the DT bindings a bit, as we need the SCMI index
that should correspond to the SCMI-child-power-domain. In other words
"power-domains = <&TOP1_PD>" would not be sufficient.

>
> The hierarchy I describe above is not made up, it's for the TI AM62L
> SoC, which is in the process of being upstreamed[1], and we're trying to
> figure out the proper way to describe the power domain hierarchy of this
> SoC in a way that can support the multiple low-power idle states that
> the SoC family plans to support.
>
> All suggestions, redirections, corrections welcome!
>
> Thanks,
>
> Kevin
>
> [1]
> https://lore.kernel.org/r/20250109-am62lx-v3-0-ef171e789527@ti.com

Kind regards
Uffe

Re: mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman]

Ulf Hansson <ulf.hansson@linaro.org> writes:

> On Tue, 8 Apr 2025 at 01:31, Kevin Hilman <khilman@baylibre.com> wrote:
>>
>> Hello SCMI folks,
>>
>> I'm trying to figure out how to model a power-domain hierarchy when
>> there is a mixture of SCMI and PSCI domains.
>>
>> Let's say I have a top-level PD, managed by PSCI, but inside it it has
>> both a CPU cluster (with cluster & CPU PDs controlled by PSCI) as well
>> as some other sub-domains for leaf devices that are controlled by SCMI.
>> A simplified version looks something like this:
>>
>> SOC
>>  |
>>  |- TOP1_PD (PSCI)
>>      |
>>      |-- LEAF1_PD (SCMI)
>>      |-- LEAF2_PD (SCMI)
>>      |
>>      \-- CLUSTER_PD (PSCI)
>>          |
>>          |-- CPU1_PD (PSCI)
>>          \-- CPU2_PD (PSCI)
>>
>>
>> So the main question is: how do I describe the SCMI part of this today
>> in DT?  Currently, I have something like this for the SCMI-controlled
>> PDs:
>>
>>         scmi_pds: protocol@11 {
>>                 reg = <0x11>;
>>                 #power-domain-cells = <1>;
>>                 bootph-all;
>>         };
>>
>> and the leaf devices under LEAF1_PD and LEAF2_PD would have
>> `power-domains = <&scmi_pds N>` properties, indicating their PD is
>> managed by SCMI.  That part works fine.  And I'm also able to model the
>> CPU PDs and CLUSTER_PD just fine, including using
>> domain-idle-states... (Hi Ulf ;)
>>
>> But... how do I describe the relationship of this hierarchy?  In
>> particular, when the SCMI-controlled PDs are actually subdomains of a
>> top-level, non-SCMI PD.  I tried adding `power-domains = <&TOP1_PD>`
>> inside the scmi_pds node, but that property seems to be ignored.  So it
>> seems that currently it has not been considered to have SCMI PDs as
>> sub-domains of other PDs.  Is that correct?  Is this something anyone
>> else has considered adding?
>
> I haven't heard of this topology before, but I believe I can
> understand the need for it. Especially when using the PSCI OSI mode.
>
> In principle, if any of those LEAF1_PD/LEAF2_PD are being in use we
> must not allow the PSCI state to be selected that corresponds to the
> TOP1_PD. Right?

Correct (mostly.)  TOP_PD may potentially have multiple domain-idle
states, and if any LEAFx_PD are active, it could just mean that only the
retention state(s) of TOP_PD should be selected, not ones that involve
context loss.

> I can think of two ways forward, there may be others:
> 1) Put the leaf devices themselves in TOP1_PD *and* LEAF1_PD/LEAF2_PD.
> Thus these devices would have two power-domains. Whether this actually
> works for your case, I am not sure of, but I think it could be an
> option.

Hmm, I don't follow.  How/why would this help?

And more importantly, does the current PM domain code support devices
that have multiple PM domains?  I see there are examples in the binding
docs, but I have not seen that in practice anywhere yet.  I did a quick
test by adding

   power-domains = <&scmi_pds N>, <&TOP_PD>;

in one of the DT nodes, and at least according to
<debugfs>/pm_genpd_summary, not only does it disappear from the SCMI PD,
it also doesn't show up in TOP_PD.  It seems it can be in one or the
other, but not both.

> 2) Enable parents to be described in the "scmi_pds" node. To do that,
> we need to extend the DT bindings a bit, as we need the SCMI index
> that should correspond to the SCMI-child-power-domain. In other words
> "power-domains = <&TOP1_PD>" would not be sufficient.

This is the direction that I was thinking makes the most sense, and
actually reflects real hardware.

If you have any thoughts about what that binding change should look
like, maybe I can have a look at implementing it.

Kevin

>> The hierarchy I describe above is not made up, it's for the TI AM62L
>> SoC, which is in the process of being upstreamed[1], and we're trying to
>> figure out the proper way to describe the power domain hierarchy of this
>> SoC in a way that can support the multiple low-power idle states that
>> the SoC family plans to support.
>>
>> All suggestions, redirections, corrections welcome!
>>
>> Thanks,
>>
>> Kevin
>>
>> [1]
>> https://lore.kernel.org/r/20250109-am62lx-v3-0-ef171e789527@ti.com
>
> Kind regards
> Uffe

Re: mixing SCMI and PSCI power domain hierarchy

[Ulf Hansson]

On Wed, 16 Apr 2025 at 02:22, Kevin Hilman <khilman@baylibre.com> wrote:
>
> Ulf Hansson <ulf.hansson@linaro.org> writes:
>
> > On Tue, 8 Apr 2025 at 01:31, Kevin Hilman <khilman@baylibre.com> wrote:
> >>
> >> Hello SCMI folks,
> >>
> >> I'm trying to figure out how to model a power-domain hierarchy when
> >> there is a mixture of SCMI and PSCI domains.
> >>
> >> Let's say I have a top-level PD, managed by PSCI, but inside it it has
> >> both a CPU cluster (with cluster & CPU PDs controlled by PSCI) as well
> >> as some other sub-domains for leaf devices that are controlled by SCMI.
> >> A simplified version looks something like this:
> >>
> >> SOC
> >>  |
> >>  |- TOP1_PD (PSCI)
> >>      |
> >>      |-- LEAF1_PD (SCMI)
> >>      |-- LEAF2_PD (SCMI)
> >>      |
> >>      \-- CLUSTER_PD (PSCI)
> >>          |
> >>          |-- CPU1_PD (PSCI)
> >>          \-- CPU2_PD (PSCI)
> >>
> >>
> >> So the main question is: how do I describe the SCMI part of this today
> >> in DT?  Currently, I have something like this for the SCMI-controlled
> >> PDs:
> >>
> >>         scmi_pds: protocol@11 {
> >>                 reg = <0x11>;
> >>                 #power-domain-cells = <1>;
> >>                 bootph-all;
> >>         };
> >>
> >> and the leaf devices under LEAF1_PD and LEAF2_PD would have
> >> `power-domains = <&scmi_pds N>` properties, indicating their PD is
> >> managed by SCMI.  That part works fine.  And I'm also able to model the
> >> CPU PDs and CLUSTER_PD just fine, including using
> >> domain-idle-states... (Hi Ulf ;)
> >>
> >> But... how do I describe the relationship of this hierarchy?  In
> >> particular, when the SCMI-controlled PDs are actually subdomains of a
> >> top-level, non-SCMI PD.  I tried adding `power-domains = <&TOP1_PD>`
> >> inside the scmi_pds node, but that property seems to be ignored.  So it
> >> seems that currently it has not been considered to have SCMI PDs as
> >> sub-domains of other PDs.  Is that correct?  Is this something anyone
> >> else has considered adding?
> >
> > I haven't heard of this topology before, but I believe I can
> > understand the need for it. Especially when using the PSCI OSI mode.
> >
> > In principle, if any of those LEAF1_PD/LEAF2_PD are being in use we
> > must not allow the PSCI state to be selected that corresponds to the
> > TOP1_PD. Right?
>
> Correct (mostly.)  TOP_PD may potentially have multiple domain-idle
> states, and if any LEAFx_PD are active, it could just mean that only the
> retention state(s) of TOP_PD should be selected, not ones that involve
> context loss.

Okay, I get it.

Do note that, the genpd and its governor doesn't support this today.
Only when the child domain enters the deepest state, the parent domain
can be powered-off (one of its states can be selected). Of course, I
think it should be possible to extend genpd to support this.

>
> > I can think of two ways forward, there may be others:
> > 1) Put the leaf devices themselves in TOP1_PD *and* LEAF1_PD/LEAF2_PD.
> > Thus these devices would have two power-domains. Whether this actually
> > works for your case, I am not sure of, but I think it could be an
> > option.
>
> Hmm, I don't follow.  How/why would this help?
>
> And more importantly, does the current PM domain code support devices
> that have multiple PM domains?  I see there are examples in the binding
> docs, but I have not seen that in practice anywhere yet.  I did a quick
> test by adding
>
>    power-domains = <&scmi_pds N>, <&TOP_PD>;
>
> in one of the DT nodes, and at least according to
> <debugfs>/pm_genpd_summary, not only does it disappear from the SCMI PD,
> it also doesn't show up in TOP_PD.  It seems it can be in one or the
> other, but not both.

Attaching a single PM domain is managed by bus-level code via
dev_pm_domain_attach().

If a device has multiple PM domains, the driver/bus for it needs to
manage that specifically, typically by using
dev_pm_domain_attach_list(). The idea in 1) was based upon that we
could manage the two PM domains via a device_link that we would create
when calling dev_pm_domain_attach_list(). Not sure if it would really
that well, but possibly.

>
> > 2) Enable parents to be described in the "scmi_pds" node. To do that,
> > we need to extend the DT bindings a bit, as we need the SCMI index
> > that should correspond to the SCMI-child-power-domain. In other words
> > "power-domains = <&TOP1_PD>" would not be sufficient.
>
> This is the direction that I was thinking makes the most sense, and
> actually reflects real hardware.
>
> If you have any thoughts about what that binding change should look
> like, maybe I can have a look at implementing it.

Usually we describe parent domains by using the original power-domain
DT binding, but this is limited to work for when the child-domain node
has #power-domain-cells = <0>.

We need to investigate if there are any other DT bindings we can get
some influence from, perhaps clocks have something similar?

Anyway, without further thinking my idea would be to add a new
property ("power-domain-child-ids") where we can put a list of indexes
for the child-domain that corresponds to each of the parents that we
can list in "power-domains".

[...]

Kind regards
Uffe

Re: mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman]

Ulf Hansson <ulf.hansson@linaro.org> writes:

> On Wed, 16 Apr 2025 at 02:22, Kevin Hilman <khilman@baylibre.com> wrote:
>>
>> Ulf Hansson <ulf.hansson@linaro.org> writes:
>>
>> > On Tue, 8 Apr 2025 at 01:31, Kevin Hilman <khilman@baylibre.com> wrote:
>> >>
>> >> Hello SCMI folks,
>> >>
>> >> I'm trying to figure out how to model a power-domain hierarchy when
>> >> there is a mixture of SCMI and PSCI domains.
>> >>
>> >> Let's say I have a top-level PD, managed by PSCI, but inside it it has
>> >> both a CPU cluster (with cluster & CPU PDs controlled by PSCI) as well
>> >> as some other sub-domains for leaf devices that are controlled by SCMI.
>> >> A simplified version looks something like this:
>> >>
>> >> SOC
>> >>  |
>> >>  |- TOP1_PD (PSCI)
>> >>      |
>> >>      |-- LEAF1_PD (SCMI)
>> >>      |-- LEAF2_PD (SCMI)
>> >>      |
>> >>      \-- CLUSTER_PD (PSCI)
>> >>          |
>> >>          |-- CPU1_PD (PSCI)
>> >>          \-- CPU2_PD (PSCI)
>> >>
>> >>
>> >> So the main question is: how do I describe the SCMI part of this today
>> >> in DT?  Currently, I have something like this for the SCMI-controlled
>> >> PDs:
>> >>
>> >>         scmi_pds: protocol@11 {
>> >>                 reg = <0x11>;
>> >>                 #power-domain-cells = <1>;
>> >>                 bootph-all;
>> >>         };
>> >>
>> >> and the leaf devices under LEAF1_PD and LEAF2_PD would have
>> >> `power-domains = <&scmi_pds N>` properties, indicating their PD is
>> >> managed by SCMI.  That part works fine.  And I'm also able to model the
>> >> CPU PDs and CLUSTER_PD just fine, including using
>> >> domain-idle-states... (Hi Ulf ;)
>> >>
>> >> But... how do I describe the relationship of this hierarchy?  In
>> >> particular, when the SCMI-controlled PDs are actually subdomains of a
>> >> top-level, non-SCMI PD.  I tried adding `power-domains = <&TOP1_PD>`
>> >> inside the scmi_pds node, but that property seems to be ignored.  So it
>> >> seems that currently it has not been considered to have SCMI PDs as
>> >> sub-domains of other PDs.  Is that correct?  Is this something anyone
>> >> else has considered adding?
>> >
>> > I haven't heard of this topology before, but I believe I can
>> > understand the need for it. Especially when using the PSCI OSI mode.
>> >
>> > In principle, if any of those LEAF1_PD/LEAF2_PD are being in use we
>> > must not allow the PSCI state to be selected that corresponds to the
>> > TOP1_PD. Right?
>>
>> Correct (mostly.)  TOP_PD may potentially have multiple domain-idle
>> states, and if any LEAFx_PD are active, it could just mean that only the
>> retention state(s) of TOP_PD should be selected, not ones that involve
>> context loss.
>
> Okay, I get it.
>
> Do note that, the genpd and its governor doesn't support this today.
> Only when the child domain enters the deepest state, the parent domain
> can be powered-off (one of its states can be selected). Of course, I
> think it should be possible to extend genpd to support this.

Yes, I'm aware of the current limitations here.  I hope to contribute
some support for this also.

>> > I can think of two ways forward, there may be others:
>> > 1) Put the leaf devices themselves in TOP1_PD *and* LEAF1_PD/LEAF2_PD.
>> > Thus these devices would have two power-domains. Whether this actually
>> > works for your case, I am not sure of, but I think it could be an
>> > option.
>>
>> Hmm, I don't follow.  How/why would this help?
>>
>> And more importantly, does the current PM domain code support devices
>> that have multiple PM domains?  I see there are examples in the binding
>> docs, but I have not seen that in practice anywhere yet.  I did a quick
>> test by adding
>>
>>    power-domains = <&scmi_pds N>, <&TOP_PD>;
>>
>> in one of the DT nodes, and at least according to
>> <debugfs>/pm_genpd_summary, not only does it disappear from the SCMI PD,
>> it also doesn't show up in TOP_PD.  It seems it can be in one or the
>> other, but not both.
>
> Attaching a single PM domain is managed by bus-level code via
> dev_pm_domain_attach().
>
> If a device has multiple PM domains, the driver/bus for it needs to
> manage that specifically, typically by using
> dev_pm_domain_attach_list().

OK, I see.  But in my quick experiment with this, I found that adding a
2nd domain to a devices power-domain property, it disappeared from the
first domain also.  Any idea what is happening there?

> The idea in 1) was based upon that we
> could manage the two PM domains via a device_link that we would create
> when calling dev_pm_domain_attach_list(). Not sure if it would really
> that well, but possibly.

OK, I can have a look at this.  But describing things in DT this way
( a leaf device listing 2 PDs ) doesn't seem like a correct description
of the hardware, and more of a workaround of Linux limitations. 

>> > 2) Enable parents to be described in the "scmi_pds" node. To do that,
>> > we need to extend the DT bindings a bit, as we need the SCMI index
>> > that should correspond to the SCMI-child-power-domain. In other words
>> > "power-domains = <&TOP1_PD>" would not be sufficient.
>>
>> This is the direction that I was thinking makes the most sense, and
>> actually reflects real hardware.
>>
>> If you have any thoughts about what that binding change should look
>> like, maybe I can have a look at implementing it.
>
> Usually we describe parent domains by using the original power-domain
> DT binding, but this is limited to work for when the child-domain node
> has #power-domain-cells = <0>.
>
> We need to investigate if there are any other DT bindings we can get
> some influence from, perhaps clocks have something similar?
>
> Anyway, without further thinking my idea would be to add a new
> property ("power-domain-child-ids") where we can put a list of indexes
> for the child-domain that corresponds to each of the parents that we
> can list in "power-domains".

OK,  I'll start researching some options here.

Thanks for the guidance,

Kevin

Re: mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman]

Kevin Hilman <khilman@baylibre.com> writes:

> Ulf Hansson <ulf.hansson@linaro.org> writes:

[...]

>>
>> Attaching a single PM domain is managed by bus-level code via
>> dev_pm_domain_attach().
>>
>> If a device has multiple PM domains, the driver/bus for it needs to
>> manage that specifically, typically by using
>> dev_pm_domain_attach_list().
>
> OK, I see.  But in my quick experiment with this, I found that adding a
> 2nd domain to a devices power-domain property, it disappeared from the
> first domain also.  Any idea what is happening there?

Just to answer my own question, I found that genpd_dev_pm_attach() has
an explicit check for the number of PM domains a device has[1].  If it
is not 1, the device is not attached to any domain.

I had assumed that maybe the first domain would get attached and the
others are ignored, but in fact all the domains are ignored.

Kevin

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pmdomain/core.c?h=v6.14#n3035

Re: mixing SCMI and PSCI power domain hierarchy

[Ulf Hansson]

On Wed, 16 Apr 2025 at 21:28, Kevin Hilman <khilman@baylibre.com> wrote:
>
> Kevin Hilman <khilman@baylibre.com> writes:
>
> > Ulf Hansson <ulf.hansson@linaro.org> writes:
>
> [...]
>
> >>
> >> Attaching a single PM domain is managed by bus-level code via
> >> dev_pm_domain_attach().
> >>
> >> If a device has multiple PM domains, the driver/bus for it needs to
> >> manage that specifically, typically by using
> >> dev_pm_domain_attach_list().
> >
> > OK, I see.  But in my quick experiment with this, I found that adding a
> > 2nd domain to a devices power-domain property, it disappeared from the
> > first domain also.  Any idea what is happening there?
>
> Just to answer my own question, I found that genpd_dev_pm_attach() has
> an explicit check for the number of PM domains a device has[1].  If it
> is not 1, the device is not attached to any domain.
>
> I had assumed that maybe the first domain would get attached and the
> others are ignored, but in fact all the domains are ignored.

Correct. It's because we need a different way to manage attaching of
multiple PM domains for a device.

[...]

Br
Uffe

Re: mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman]

Ulf Hansson <ulf.hansson@linaro.org> writes:

> On Wed, 16 Apr 2025 at 02:22, Kevin Hilman <khilman@baylibre.com> wrote:
>>
>> Ulf Hansson <ulf.hansson@linaro.org> writes:

[...]

>> > 2) Enable parents to be described in the "scmi_pds" node. To do that,
>> > we need to extend the DT bindings a bit, as we need the SCMI index
>> > that should correspond to the SCMI-child-power-domain. In other words
>> > "power-domains = <&TOP1_PD>" would not be sufficient.
>>
>> This is the direction that I was thinking makes the most sense, and
>> actually reflects real hardware.
>>
>> If you have any thoughts about what that binding change should look
>> like, maybe I can have a look at implementing it.
>
> Usually we describe parent domains by using the original power-domain
> DT binding, but this is limited to work for when the child-domain node
> has #power-domain-cells = <0>.
>
> We need to investigate if there are any other DT bindings we can get
> some influence from, perhaps clocks have something similar?
>
> Anyway, without further thinking my idea would be to add a new
> property ("power-domain-child-ids") where we can put a list of indexes
> for the child-domain that corresponds to each of the parents that we
> can list in "power-domains".

OK, I have an idea.

First, the normal stuff.  We have the SCMI PDs:

&scmi {
	scmi_pds: protocol@11 {
		reg = <0x11>;
		#power-domain-cells = <1>;
	};
};

Then we have an example leaf node which uses that:

	main_timer0: timer@2400000 {
                [...]
		clocks = <&scmi_clk 47>;
		clock-names = "fck";
		power-domains = <&scmi_pds 15>;
	};

And then, we have the top-level PD controlled by PSCI:

&psci {
	compatible = "arm,psci-1.0";
	method = "smc";

	MAIN_PD: power-controller-main {
		#power-domain-cells = <0>;
	};
};

Now the new idea: The hierarchy I want to represent is that main_timer0
is in SCMI PD 15, which is in the top-level domain MAIN_PD.

So, what about simply extending the SCMI binding where we can
(optionally) define a parent domain for each SCMI ID?  So the SCMI node
would then look like this:

	scmi_pds: protocol@11 {
		reg = <0x11>;
		#power-domain-cells = <1>;

		/* main_timer0: SCMI ID: 15 */
		pd15: domain@15 {
		    reg = <15>;
		    parent-domain = <&MAIN_PD>;
	        };
	};

This requires adding a bit of code to the SCMI PM domain driver to check
for these subnodes with optional parent-domain properties, and I already
have a PoC[1] which works on v6.12.

One other reason I like this solution is that it actually kills two
birds with one stone.  The first one is describing the hieracrhy above,
but the second one is that it solves another limitation of the SCMI PM
domain implementation.

Currently you can only have a single group of SCMI PDs.  More
specifically, you can only have a single "protocol@11" or "reg = <0x11>"
SCMI node.  Which means, you cannot have more than one group of SCMI
PDs.  So if you have (like I do) groups of SCMI-controlled PDs that area
themselves grouped into multiple top-level PDs, there is no way to
describe that today since you can only have a single SCMI PD node.

To get more specific, on this TI SoC I'm working on now, there's more
than one top-level PD controlled by PSCI.  The top-level (PSCI) domains
look like this:

&psci {
	compatible = "arm,psci-1.0";
	method = "smc";

	MAIN_PD: power-controller-main {
		#power-domain-cells = <0>;
	};

	WKUP_PD: power-controller-wkup {
		#power-domain-cells = <0>;
	};
};

And in addition to the main_timer0 leaf device described above, I have
another timer that is also SCMI controlled like this:

	wkup_timer0: timer@2b100000 {
                [...]
		clocks = <&scmi_clk 85>;
		clock-names = "fck";
		power-domains = <&scmi_pds 19>;
	};

but..., this one is not in MAIN_PD, it is in WKUP_PD.  Which means I
need some way to declare that SCMI PDs that are part of the one (and
only) "protocol@11" node could have different parent domains.  With the
new idea, we just do this: (note that the subnodes have different
parent-domain properties):

	scmi_pds: protocol@11 {
		reg = <0x11>;
		#power-domain-cells = <1>;

		/* main_timer0: SCMI ID: 15 */
		pd15: domain@15 {
		    reg = <15>;
		    parent-domain = <&MAIN_PD>;
	        };

		/* wkup_timer0: SCMI ID: 19 */
		pd19: domain@19 {
		    reg = <19>;
		    parent-domain = <&WKUP_PD>;
	        };
	};

So.... what you think about this?  I guess's the "parent-domain" idea is
the really new concept here, but I guess it's not that much different
than a clock divider that points to its parent clock.

Anyways, curious what you think.  I quite like this solution because it
solves two fairly big limitations in the SCMI PM domain implementaion
that exist today with one relatively simple solution.

Kevin

P.S. maybe the most interesting thing about this solution is that it was
     co-developed with AI.  I've been experimenting with Anthropic's
     Claude Code[2] for both brainstorming and coding.  After coming up
     with some other ideas I rejected, Claude came up with this one, and
     when I realized it solved both my problems in one go, I was
     obviously drawn to it, and figured it's worth proposing for
     discussion.  I wrote this email, and had to manually write my own
     DT to test out this solution, but the patch below was entirely
     written by Claude.

[1]
diff --git a/drivers/pmdomain/arm/scmi_pm_domain.c b/drivers/pmdomain/arm/scmi_pm_domain.c
index a7784a8bb5db..1e4aa17ca1d2 100644
--- a/drivers/pmdomain/arm/scmi_pm_domain.c
+++ b/drivers/pmdomain/arm/scmi_pm_domain.c
@@ -54,7 +54,7 @@ static int scmi_pd_power_off(struct generic_pm_domain *domain)
 
 static int scmi_pm_domain_probe(struct scmi_device *sdev)
 {
-	int num_domains, i;
+	int num_domains, i, ret;
 	struct device *dev = &sdev->dev;
 	struct device_node *np = dev->of_node;
 	struct scmi_pm_domain *scmi_pd;
@@ -115,7 +115,53 @@ static int scmi_pm_domain_probe(struct scmi_device *sdev)
 
 	dev_set_drvdata(dev, scmi_pd_data);
 
-	return of_genpd_add_provider_onecell(np, scmi_pd_data);
+	/* First register the SCMI domains with the power domain framework */
+	ret = of_genpd_add_provider_onecell(np, scmi_pd_data);
+	if (ret)
+		return ret;
+
+	/* AFTER registering domains, now try to establish subdomain relationships */
+	for (i = 0; i < num_domains; i++) {
+		struct of_phandle_args parent_pd_args;
+		struct device_node *child_np;
+
+		/* Skip if this domain is not defined */
+		if (!domains[i])
+			continue;
+
+		/* Look for child nodes with parent-domain property */
+		for_each_child_of_node(np, child_np) {
+			u32 domain_id;
+
+			/* Check if this child node refers to the current SCMI domain */
+			if (of_property_read_u32(child_np, "reg", &domain_id) || domain_id != i)
+				continue;
+
+			/* Check if it has a parent-domain property */
+			if (of_parse_phandle_with_args(child_np, "parent-domain", "#power-domain-cells",
+						      0, &parent_pd_args) < 0)
+				continue;
+
+			/* Create a phandle args structure for the SCMI domain */
+			struct of_phandle_args subdomain_args = {
+				.np = np,
+				.args_count = 1,
+				.args = {i}
+			};
+
+			/* Use of_genpd_add_subdomain to directly establish the relationship */
+			ret = of_genpd_add_subdomain(&parent_pd_args, &subdomain_args);
+			if (ret) {
+				dev_warn(dev, "Failed to add SCMI domain %d as subdomain: %d\n",
+					 i, ret);
+			} else {
+				dev_info(dev, "Added SCMI domain %s as subdomain of %s\n",
+					 domains[i]->name, parent_pd_args.np->name);
+			}
+		}
+	}
+
+	return 0;
 }
 
 static void scmi_pm_domain_remove(struct scmi_device *sdev)

[2] https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview

Re: mixing SCMI and PSCI power domain hierarchy

[Ulf Hansson]

On Thu, 17 Apr 2025 at 01:57, Kevin Hilman <khilman@baylibre.com> wrote:
>
> Ulf Hansson <ulf.hansson@linaro.org> writes:
>
> > On Wed, 16 Apr 2025 at 02:22, Kevin Hilman <khilman@baylibre.com> wrote:
> >>
> >> Ulf Hansson <ulf.hansson@linaro.org> writes:
>
> [...]
>
> >> > 2) Enable parents to be described in the "scmi_pds" node. To do that,
> >> > we need to extend the DT bindings a bit, as we need the SCMI index
> >> > that should correspond to the SCMI-child-power-domain. In other words
> >> > "power-domains = <&TOP1_PD>" would not be sufficient.
> >>
> >> This is the direction that I was thinking makes the most sense, and
> >> actually reflects real hardware.
> >>
> >> If you have any thoughts about what that binding change should look
> >> like, maybe I can have a look at implementing it.
> >
> > Usually we describe parent domains by using the original power-domain
> > DT binding, but this is limited to work for when the child-domain node
> > has #power-domain-cells = <0>.
> >
> > We need to investigate if there are any other DT bindings we can get
> > some influence from, perhaps clocks have something similar?
> >
> > Anyway, without further thinking my idea would be to add a new
> > property ("power-domain-child-ids") where we can put a list of indexes
> > for the child-domain that corresponds to each of the parents that we
> > can list in "power-domains".
>
> OK, I have an idea.
>
> First, the normal stuff.  We have the SCMI PDs:
>
> &scmi {
>         scmi_pds: protocol@11 {
>                 reg = <0x11>;
>                 #power-domain-cells = <1>;
>         };
> };
>
> Then we have an example leaf node which uses that:
>
>         main_timer0: timer@2400000 {
>                 [...]
>                 clocks = <&scmi_clk 47>;
>                 clock-names = "fck";
>                 power-domains = <&scmi_pds 15>;
>         };
>
> And then, we have the top-level PD controlled by PSCI:
>
> &psci {
>         compatible = "arm,psci-1.0";
>         method = "smc";
>
>         MAIN_PD: power-controller-main {
>                 #power-domain-cells = <0>;
>         };
> };
>
> Now the new idea: The hierarchy I want to represent is that main_timer0
> is in SCMI PD 15, which is in the top-level domain MAIN_PD.
>
> So, what about simply extending the SCMI binding where we can
> (optionally) define a parent domain for each SCMI ID?  So the SCMI node
> would then look like this:
>
>         scmi_pds: protocol@11 {
>                 reg = <0x11>;
>                 #power-domain-cells = <1>;
>
>                 /* main_timer0: SCMI ID: 15 */
>                 pd15: domain@15 {
>                     reg = <15>;
>                     parent-domain = <&MAIN_PD>;
>                 };
>         };
>
> This requires adding a bit of code to the SCMI PM domain driver to check
> for these subnodes with optional parent-domain properties, and I already
> have a PoC[1] which works on v6.12.
>
> One other reason I like this solution is that it actually kills two
> birds with one stone.  The first one is describing the hieracrhy above,
> but the second one is that it solves another limitation of the SCMI PM
> domain implementation.
>
> Currently you can only have a single group of SCMI PDs.  More
> specifically, you can only have a single "protocol@11" or "reg = <0x11>"
> SCMI node.  Which means, you cannot have more than one group of SCMI
> PDs.  So if you have (like I do) groups of SCMI-controlled PDs that area
> themselves grouped into multiple top-level PDs, there is no way to
> describe that today since you can only have a single SCMI PD node.
>
> To get more specific, on this TI SoC I'm working on now, there's more
> than one top-level PD controlled by PSCI.  The top-level (PSCI) domains
> look like this:
>
> &psci {
>         compatible = "arm,psci-1.0";
>         method = "smc";
>
>         MAIN_PD: power-controller-main {
>                 #power-domain-cells = <0>;
>         };
>
>         WKUP_PD: power-controller-wkup {
>                 #power-domain-cells = <0>;
>         };
> };
>
> And in addition to the main_timer0 leaf device described above, I have
> another timer that is also SCMI controlled like this:
>
>         wkup_timer0: timer@2b100000 {
>                 [...]
>                 clocks = <&scmi_clk 85>;
>                 clock-names = "fck";
>                 power-domains = <&scmi_pds 19>;
>         };
>
> but..., this one is not in MAIN_PD, it is in WKUP_PD.  Which means I
> need some way to declare that SCMI PDs that are part of the one (and
> only) "protocol@11" node could have different parent domains.  With the
> new idea, we just do this: (note that the subnodes have different
> parent-domain properties):
>
>         scmi_pds: protocol@11 {
>                 reg = <0x11>;
>                 #power-domain-cells = <1>;
>
>                 /* main_timer0: SCMI ID: 15 */
>                 pd15: domain@15 {
>                     reg = <15>;
>                     parent-domain = <&MAIN_PD>;
>                 };
>
>                 /* wkup_timer0: SCMI ID: 19 */
>                 pd19: domain@19 {
>                     reg = <19>;
>                     parent-domain = <&WKUP_PD>;
>                 };
>         };
>
> So.... what you think about this?  I guess's the "parent-domain" idea is
> the really new concept here, but I guess it's not that much different
> than a clock divider that points to its parent clock.
>
> Anyways, curious what you think.  I quite like this solution because it
> solves two fairly big limitations in the SCMI PM domain implementaion
> that exist today with one relatively simple solution.

Well, I am not the one to convince in regards to DT bindings. Putting
the DT bindings aside, I think your extensions for the SCMI Power
Domain to support parent-domains makes perfect sense to me!

In regards to the DT bindings (again, I am not the expert here), but
maybe we could try to extend the current way we describe parent
domains instead of inventing something entirely new. Could the below
work?

scmi_pds: protocol@11 {
        reg = <0x11>;
         #power-domain-cells = <1>;
         power-domains = <&MAIN_PD>, <&WKUP_PD>; // a list of phandles
         power-domains-child-ids = <15>, <19> ; // a list of indexes
corresponding to the list of phandles above

[...]

The above said, I have also discussed this with Vincent and we thought
it was a good idea to bring up the topic at the next monthly public
SCMI call.

It would be great if you could join, but let's see if we can find a
reasonable slot as the current is in the middle of the night for you.
Vincent will reach out to you as he is the one hosting the call.

Br
Uffe

Re: mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman]

Ulf Hansson <ulf.hansson@linaro.org> writes:

[...]

>> Anyways, curious what you think.  I quite like this solution because it
>> solves two fairly big limitations in the SCMI PM domain implementaion
>> that exist today with one relatively simple solution.
>
> Well, I am not the one to convince in regards to DT bindings. Putting
> the DT bindings aside, I think your extensions for the SCMI Power
> Domain to support parent-domains makes perfect sense to me!
>
> In regards to the DT bindings (again, I am not the expert here), but
> maybe we could try to extend the current way we describe parent
> domains instead of inventing something entirely new. Could the below
> work?
>
> scmi_pds: protocol@11 {
>         reg = <0x11>;
>          #power-domain-cells = <1>;
>          power-domains = <&MAIN_PD>, <&WKUP_PD>; // a list of phandles
>          power-domains-child-ids = <15>, <19> ; // a list of indexes
> corresponding to the list of phandles above

Yeah, this is probably better.  I'm not crazy about introducing novel
DT bindings either.  I will make a PoC of this approach.

The catch with either approach is that it will be a long list.  e.g. on
the TI AM62L, there are 50+ SCMI IDs.

> The above said, I have also discussed this with Vincent and we thought
> it was a good idea to bring up the topic at the next monthly public
> SCMI call.
>
> It would be great if you could join, but let's see if we can find a
> reasonable slot as the current is in the middle of the night for you.
> Vincent will reach out to you as he is the one hosting the call.

Sounds great, thanks!

Kevin

Re: mixing SCMI and PSCI power domain hierarchy

[Ulf Hansson]

On Fri, 25 Apr 2025 at 16:39, Kevin Hilman <khilman@baylibre.com> wrote:
>
> Ulf Hansson <ulf.hansson@linaro.org> writes:
>
> [...]
>
> >> Anyways, curious what you think.  I quite like this solution because it
> >> solves two fairly big limitations in the SCMI PM domain implementaion
> >> that exist today with one relatively simple solution.
> >
> > Well, I am not the one to convince in regards to DT bindings. Putting
> > the DT bindings aside, I think your extensions for the SCMI Power
> > Domain to support parent-domains makes perfect sense to me!
> >
> > In regards to the DT bindings (again, I am not the expert here), but
> > maybe we could try to extend the current way we describe parent
> > domains instead of inventing something entirely new. Could the below
> > work?
> >
> > scmi_pds: protocol@11 {
> >         reg = <0x11>;
> >          #power-domain-cells = <1>;
> >          power-domains = <&MAIN_PD>, <&WKUP_PD>; // a list of phandles
> >          power-domains-child-ids = <15>, <19> ; // a list of indexes
> > corresponding to the list of phandles above
>
> Yeah, this is probably better.  I'm not crazy about introducing novel
> DT bindings either.  I will make a PoC of this approach.
>
> The catch with either approach is that it will be a long list.  e.g. on
> the TI AM62L, there are 50+ SCMI IDs.

Just wanted to share a brief update. I got the opportunity to chat
with Krzysztof and Abel at Linaro Connect this week, about extending
the power-domain DT bindings, in the way I proposed above. We agreed
that it seems like a reasonable approach.

[...]

Kind regards
Uffe

Re: mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman]

Ulf Hansson <ulf.hansson@linaro.org> writes:

> On Fri, 25 Apr 2025 at 16:39, Kevin Hilman <khilman@baylibre.com> wrote:
>>
>> Ulf Hansson <ulf.hansson@linaro.org> writes:
>>
>> [...]
>>
>> >> Anyways, curious what you think.  I quite like this solution because it
>> >> solves two fairly big limitations in the SCMI PM domain implementaion
>> >> that exist today with one relatively simple solution.
>> >
>> > Well, I am not the one to convince in regards to DT bindings. Putting
>> > the DT bindings aside, I think your extensions for the SCMI Power
>> > Domain to support parent-domains makes perfect sense to me!
>> >
>> > In regards to the DT bindings (again, I am not the expert here), but
>> > maybe we could try to extend the current way we describe parent
>> > domains instead of inventing something entirely new. Could the below
>> > work?
>> >
>> > scmi_pds: protocol@11 {
>> >         reg = <0x11>;
>> >          #power-domain-cells = <1>;
>> >          power-domains = <&MAIN_PD>, <&WKUP_PD>; // a list of phandles
>> >          power-domains-child-ids = <15>, <19> ; // a list of indexes
>> > corresponding to the list of phandles above
>>
>> Yeah, this is probably better.  I'm not crazy about introducing novel
>> DT bindings either.  I will make a PoC of this approach.
>>
>> The catch with either approach is that it will be a long list.  e.g. on
>> the TI AM62L, there are 50+ SCMI IDs.
>
> Just wanted to share a brief update. I got the opportunity to chat
> with Krzysztof and Abel at Linaro Connect this week, about extending
> the power-domain DT bindings, in the way I proposed above. We agreed
> that it seems like a reasonable approach.

OK, great.  Thanks!

I just posted an RFC with an initial implementation:
https://lore.kernel.org/linux-pm/20250528-pmdomain-hierarchy-onecell-v1-1-851780700c68@baylibre.com/T/#u

Kevin