Re: mixing SCMI and PSCI power domain hierarchy

[Kevin Hilman <khilman@baylibre.com>]

Ulf Hansson <ulf.hansson@linaro.org> writes:

> On Wed, 16 Apr 2025 at 02:22, Kevin Hilman <khilman@baylibre.com> wrote:
>>
>> Ulf Hansson <ulf.hansson@linaro.org> writes:
>>
>> > On Tue, 8 Apr 2025 at 01:31, Kevin Hilman <khilman@baylibre.com> wrote:
>> >>
>> >> Hello SCMI folks,
>> >>
>> >> I'm trying to figure out how to model a power-domain hierarchy when
>> >> there is a mixture of SCMI and PSCI domains.
>> >>
>> >> Let's say I have a top-level PD, managed by PSCI, but inside it it has
>> >> both a CPU cluster (with cluster & CPU PDs controlled by PSCI) as well
>> >> as some other sub-domains for leaf devices that are controlled by SCMI.
>> >> A simplified version looks something like this:
>> >>
>> >> SOC
>> >>  |
>> >>  |- TOP1_PD (PSCI)
>> >>      |
>> >>      |-- LEAF1_PD (SCMI)
>> >>      |-- LEAF2_PD (SCMI)
>> >>      |
>> >>      \-- CLUSTER_PD (PSCI)
>> >>          |
>> >>          |-- CPU1_PD (PSCI)
>> >>          \-- CPU2_PD (PSCI)
>> >>
>> >>
>> >> So the main question is: how do I describe the SCMI part of this today
>> >> in DT?  Currently, I have something like this for the SCMI-controlled
>> >> PDs:
>> >>
>> >>         scmi_pds: protocol@11 {
>> >>                 reg = <0x11>;
>> >>                 #power-domain-cells = <1>;
>> >>                 bootph-all;
>> >>         };
>> >>
>> >> and the leaf devices under LEAF1_PD and LEAF2_PD would have
>> >> `power-domains = <&scmi_pds N>` properties, indicating their PD is
>> >> managed by SCMI.  That part works fine.  And I'm also able to model the
>> >> CPU PDs and CLUSTER_PD just fine, including using
>> >> domain-idle-states... (Hi Ulf ;)
>> >>
>> >> But... how do I describe the relationship of this hierarchy?  In
>> >> particular, when the SCMI-controlled PDs are actually subdomains of a
>> >> top-level, non-SCMI PD.  I tried adding `power-domains = <&TOP1_PD>`
>> >> inside the scmi_pds node, but that property seems to be ignored.  So it
>> >> seems that currently it has not been considered to have SCMI PDs as
>> >> sub-domains of other PDs.  Is that correct?  Is this something anyone
>> >> else has considered adding?
>> >
>> > I haven't heard of this topology before, but I believe I can
>> > understand the need for it. Especially when using the PSCI OSI mode.
>> >
>> > In principle, if any of those LEAF1_PD/LEAF2_PD are being in use we
>> > must not allow the PSCI state to be selected that corresponds to the
>> > TOP1_PD. Right?
>>
>> Correct (mostly.)  TOP_PD may potentially have multiple domain-idle
>> states, and if any LEAFx_PD are active, it could just mean that only the
>> retention state(s) of TOP_PD should be selected, not ones that involve
>> context loss.
>
> Okay, I get it.
>
> Do note that, the genpd and its governor doesn't support this today.
> Only when the child domain enters the deepest state, the parent domain
> can be powered-off (one of its states can be selected). Of course, I
> think it should be possible to extend genpd to support this.

Yes, I'm aware of the current limitations here.  I hope to contribute
some support for this also.

>> > I can think of two ways forward, there may be others:
>> > 1) Put the leaf devices themselves in TOP1_PD *and* LEAF1_PD/LEAF2_PD.
>> > Thus these devices would have two power-domains. Whether this actually
>> > works for your case, I am not sure of, but I think it could be an
>> > option.
>>
>> Hmm, I don't follow.  How/why would this help?
>>
>> And more importantly, does the current PM domain code support devices
>> that have multiple PM domains?  I see there are examples in the binding
>> docs, but I have not seen that in practice anywhere yet.  I did a quick
>> test by adding
>>
>>    power-domains = <&scmi_pds N>, <&TOP_PD>;
>>
>> in one of the DT nodes, and at least according to
>> <debugfs>/pm_genpd_summary, not only does it disappear from the SCMI PD,
>> it also doesn't show up in TOP_PD.  It seems it can be in one or the
>> other, but not both.
>
> Attaching a single PM domain is managed by bus-level code via
> dev_pm_domain_attach().
>
> If a device has multiple PM domains, the driver/bus for it needs to
> manage that specifically, typically by using
> dev_pm_domain_attach_list().

OK, I see.  But in my quick experiment with this, I found that adding a
2nd domain to a devices power-domain property, it disappeared from the
first domain also.  Any idea what is happening there?

> The idea in 1) was based upon that we
> could manage the two PM domains via a device_link that we would create
> when calling dev_pm_domain_attach_list(). Not sure if it would really
> that well, but possibly.

OK, I can have a look at this.  But describing things in DT this way
( a leaf device listing 2 PDs ) doesn't seem like a correct description
of the hardware, and more of a workaround of Linux limitations. 

>> > 2) Enable parents to be described in the "scmi_pds" node. To do that,
>> > we need to extend the DT bindings a bit, as we need the SCMI index
>> > that should correspond to the SCMI-child-power-domain. In other words
>> > "power-domains = <&TOP1_PD>" would not be sufficient.
>>
>> This is the direction that I was thinking makes the most sense, and
>> actually reflects real hardware.
>>
>> If you have any thoughts about what that binding change should look
>> like, maybe I can have a look at implementing it.
>
> Usually we describe parent domains by using the original power-domain
> DT binding, but this is limited to work for when the child-domain node
> has #power-domain-cells = <0>.
>
> We need to investigate if there are any other DT bindings we can get
> some influence from, perhaps clocks have something similar?
>
> Anyway, without further thinking my idea would be to add a new
> property ("power-domain-child-ids") where we can put a list of indexes
> for the child-domain that corresponds to each of the parents that we
> can list in "power-domains".

OK,  I'll start researching some options here.

Thanks for the guidance,

Kevin