[PATCH RFC 0/2] pmdomain: ti-sci: Handling DSS boot splash

[PATCH RFC 0/2] pmdomain: ti-sci: Handling DSS boot splash

[Tomi Valkeinen]

The first patch here is a fix for ti_sci_pm_domains to handle the case
where the dts has two nodes which refer to the same PD.

The second is the interesting one, and very much an RFC.

 Tomi

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Tomi Valkeinen (2):
      pmdomain: ti-sci: Fix duplicate PD referrals
      pmdomain: ti-sci: Support retaining PD boot time state

 drivers/pmdomain/ti/ti_sci_pm_domains.c    | 47 ++++++++++++++++++++++++++++--
 include/dt-bindings/soc/ti,sci_pm_domain.h |  1 +
 2 files changed, 45 insertions(+), 3 deletions(-)
---
base-commit: 0bbac3facb5d6cc0171c45c9873a2dc96bea9680
change-id: 20240415-ti-sci-pd-33b39f6b0586

Best regards,
-- 
Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

[PATCH RFC 1/2] pmdomain: ti-sci: Fix duplicate PD referrals

[Tomi Valkeinen]

When the dts file has multiple referrers to a single PD (e.g.
simple-framebuffer and dss nodes both point to the DSS power-domain) the
ti-sci driver will create two power domains, both with the same ID, and
that will cause problems as one of the power domains will hide the other
one.

Fix this checking if a PD with the ID has already been created, and only
create a PD for new IDs.

Fixes: efa5c01cd7ee ("soc: ti: ti_sci_pm_domains: switch to use multiple genpds instead of one")
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/pmdomain/ti/ti_sci_pm_domains.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c
index 9dddf227a3a6..1510d5ddae3d 100644
--- a/drivers/pmdomain/ti/ti_sci_pm_domains.c
+++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c
@@ -114,6 +114,18 @@ static const struct of_device_id ti_sci_pm_domain_matches[] = {
 };
 MODULE_DEVICE_TABLE(of, ti_sci_pm_domain_matches);
 
+static bool ti_sci_pm_idx_exists(struct ti_sci_genpd_provider *pd_provider, u32 idx)
+{
+	struct ti_sci_pm_domain *pd;
+
+	list_for_each_entry(pd, &pd_provider->pd_list, node) {
+		if (pd->idx == idx)
+			return true;
+	}
+
+	return false;
+}
+
 static int ti_sci_pm_domain_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -149,8 +161,14 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
 				break;
 
 			if (args.args_count >= 1 && args.np == dev->of_node) {
-				if (args.args[0] > max_id)
+				if (args.args[0] > max_id) {
 					max_id = args.args[0];
+				} else {
+					if (ti_sci_pm_idx_exists(pd_provider, args.args[0])) {
+						index++;
+						continue;
+					}
+				}
 
 				pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL);
 				if (!pd) {

-- 
2.34.1

Re: [PATCH RFC 1/2] pmdomain: ti-sci: Fix duplicate PD referrals

[Ulf Hansson]

On Mon, 15 Apr 2024 at 18:00, Tomi Valkeinen
<tomi.valkeinen@ideasonboard.com> wrote:
>
> When the dts file has multiple referrers to a single PD (e.g.
> simple-framebuffer and dss nodes both point to the DSS power-domain) the
> ti-sci driver will create two power domains, both with the same ID, and
> that will cause problems as one of the power domains will hide the other
> one.
>
> Fix this checking if a PD with the ID has already been created, and only
> create a PD for new IDs.
>
> Fixes: efa5c01cd7ee ("soc: ti: ti_sci_pm_domains: switch to use multiple genpds instead of one")
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Applied for fixes and by adding a stable tag, thanks!

Kind regards
Uffe


> ---
>  drivers/pmdomain/ti/ti_sci_pm_domains.c | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c
> index 9dddf227a3a6..1510d5ddae3d 100644
> --- a/drivers/pmdomain/ti/ti_sci_pm_domains.c
> +++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c
> @@ -114,6 +114,18 @@ static const struct of_device_id ti_sci_pm_domain_matches[] = {
>  };
>  MODULE_DEVICE_TABLE(of, ti_sci_pm_domain_matches);
>
> +static bool ti_sci_pm_idx_exists(struct ti_sci_genpd_provider *pd_provider, u32 idx)
> +{
> +       struct ti_sci_pm_domain *pd;
> +
> +       list_for_each_entry(pd, &pd_provider->pd_list, node) {
> +               if (pd->idx == idx)
> +                       return true;
> +       }
> +
> +       return false;
> +}
> +
>  static int ti_sci_pm_domain_probe(struct platform_device *pdev)
>  {
>         struct device *dev = &pdev->dev;
> @@ -149,8 +161,14 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
>                                 break;
>
>                         if (args.args_count >= 1 && args.np == dev->of_node) {
> -                               if (args.args[0] > max_id)
> +                               if (args.args[0] > max_id) {
>                                         max_id = args.args[0];
> +                               } else {
> +                                       if (ti_sci_pm_idx_exists(pd_provider, args.args[0])) {
> +                                               index++;
> +                                               continue;
> +                                       }
> +                               }
>
>                                 pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL);
>                                 if (!pd) {
>
> --
> 2.34.1
>

[PATCH RFC 2/2] pmdomain: ti-sci: Support retaining PD boot time state

[Tomi Valkeinen]

Add a new flag, TI_SCI_PD_KEEP_BOOT_STATE, which can be set in the dts
when referring to power domains. When this flag is set, the ti-sci
driver will check if the PD is currently enabled in the HW, and if so,
set the GENPD_FLAG_ALWAYS_ON flag so that the PD will stay enabled.

The main issue I'm trying to solve here is this:

If the Display Subsystem (DSS) has been enabled by the bootloader, the
related PD has also been enabled in the HW. When the tidss driver
probes, the driver framework will automatically enable the PD. While
executing the probe function it is very common for the probe to return
EPROBE_DEFER, and, in rarer cases, an actual error. When this happens
(probe() returns an error), the driver framework will automatically
disable the related PD.

Powering off the PD while the DSS is enabled and displaying a picture
will cause the DSS HW to enter a bad state, from which (afaik) it can't
be woken up except with full power-cycle. Trying to access the DSS in
this state (e.g. when retrying the probe) will usually cause the board
to hang sooner or later.

Even if we wouldn't have this board-hangs issue, it's nice to be able to
keep the DSS PD enabled: we want to keep the DSS enabled when the
bootloader has enabled the screen. If, instead, we disable the PD at the
first EPROBE_DEFER, the screen will (probably) go black.

Another option here would perhaps be to change the driver framework
(drivers/base/platform.c) which attaches and detaches the PD, and make
it somehow optional, allowing the driver the manage the PD. That option
has two downsides: 1) the driver _has_ to manage the PD, which would
rule out the use of simplefb and simpledrm, and 2) it would leave the PD
in off state from Linux's perspective until a driver enables the PD, and
that might mean that the PD gets actually disabled as part of normal
system wide power management (disabling unused resources).

Yet another option would be to do this outside the ti_sci_pm_domains
driver: a piece of code that would somehow be ran after the
ti_sci_pm_domains driver has probed (so that we have the PDs), but
before tidss/simplefb/simpledrm probes. The problem here is the
"somehow" part. Also, this would partly have the same issue 2) as
mentioned above.

TODO: If this approach is ok, sci-pm-domain.yaml needs to be extended.
Also, it sounds a bit like the cell value is not a bit-mask, so maybe
adding TI_SCI_PD_KEEP_BOOT_STATE flag this way is not fine.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/pmdomain/ti/ti_sci_pm_domains.c    | 27 +++++++++++++++++++++++++--
 include/dt-bindings/soc/ti,sci_pm_domain.h |  1 +
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c
index 1510d5ddae3d..b71b390aaa39 100644
--- a/drivers/pmdomain/ti/ti_sci_pm_domains.c
+++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c
@@ -103,7 +103,7 @@ static struct generic_pm_domain *ti_sci_pd_xlate(
 		return ERR_PTR(-ENOENT);
 
 	genpd_to_ti_sci_pd(genpd_data->domains[idx])->exclusive =
-		genpdspec->args[1];
+		genpdspec->args[1] & TI_SCI_PD_EXCLUSIVE;
 
 	return genpd_data->domains[idx];
 }
@@ -161,6 +161,8 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
 				break;
 
 			if (args.args_count >= 1 && args.np == dev->of_node) {
+				bool is_on = false;
+
 				if (args.args[0] > max_id) {
 					max_id = args.args[0];
 				} else {
@@ -189,7 +191,28 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
 				pd->idx = args.args[0];
 				pd->parent = pd_provider;
 
-				pm_genpd_init(&pd->pd, NULL, true);
+				/*
+				 * If TI_SCI_PD_KEEP_BOOT_STATE is set and the
+				 * PD has been enabled by the bootloader, set
+				 * the PD to GENPD_FLAG_ALWAYS_ON. This will
+				 * make sure the PD stays enabled until a driver
+				 * takes over and clears the GENPD_FLAG_ALWAYS_ON
+				 * flag.
+				 */
+				if (args.args_count > 1 &&
+				    args.args[1] & TI_SCI_PD_KEEP_BOOT_STATE) {
+					/*
+					 * We ignore any error here, and in case
+					 * of error just assume the PD is off.
+					 */
+					pd_provider->ti_sci->ops.dev_ops.is_on(pd_provider->ti_sci,
+						pd->idx, NULL, &is_on);
+
+					if (is_on)
+						pd->pd.flags |= GENPD_FLAG_ALWAYS_ON;
+				}
+
+				pm_genpd_init(&pd->pd, NULL, !is_on);
 
 				list_add(&pd->node, &pd_provider->pd_list);
 			}
diff --git a/include/dt-bindings/soc/ti,sci_pm_domain.h b/include/dt-bindings/soc/ti,sci_pm_domain.h
index 8f2a7360b65e..af610208e3a3 100644
--- a/include/dt-bindings/soc/ti,sci_pm_domain.h
+++ b/include/dt-bindings/soc/ti,sci_pm_domain.h
@@ -3,6 +3,7 @@
 #ifndef __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
 #define __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
 
+#define TI_SCI_PD_KEEP_BOOT_STATE 2
 #define TI_SCI_PD_EXCLUSIVE	1
 #define TI_SCI_PD_SHARED	0
 

-- 
2.34.1

Re: [PATCH RFC 2/2] pmdomain: ti-sci: Support retaining PD boot time state

[Tomi Valkeinen]

On 15/04/2024 19:00, Tomi Valkeinen wrote:
> Add a new flag, TI_SCI_PD_KEEP_BOOT_STATE, which can be set in the dts
> when referring to power domains. When this flag is set, the ti-sci
> driver will check if the PD is currently enabled in the HW, and if so,
> set the GENPD_FLAG_ALWAYS_ON flag so that the PD will stay enabled.
> 
> The main issue I'm trying to solve here is this:
> 
> If the Display Subsystem (DSS) has been enabled by the bootloader, the
> related PD has also been enabled in the HW. When the tidss driver
> probes, the driver framework will automatically enable the PD. While
> executing the probe function it is very common for the probe to return
> EPROBE_DEFER, and, in rarer cases, an actual error. When this happens
> (probe() returns an error), the driver framework will automatically
> disable the related PD.
> 
> Powering off the PD while the DSS is enabled and displaying a picture
> will cause the DSS HW to enter a bad state, from which (afaik) it can't
> be woken up except with full power-cycle. Trying to access the DSS in
> this state (e.g. when retrying the probe) will usually cause the board
> to hang sooner or later.
> 
> Even if we wouldn't have this board-hangs issue, it's nice to be able to
> keep the DSS PD enabled: we want to keep the DSS enabled when the
> bootloader has enabled the screen. If, instead, we disable the PD at the
> first EPROBE_DEFER, the screen will (probably) go black.

A few things occurred to me. The driver is supposed to clear the 
GENPD_FLAG_ALWAYS_ON when the driver has probed successfully. There are 
two possible issues with that:

- Afaics, there's no API to do that, and currently I just clear the bit 
in genpd->flags. There's a clear race there, so some locking would be 
required.

- This uses the GENPD_FLAG_ALWAYS_ON flag to say "PD is always on, until 
the driver has started". If the PD would have GENPD_FLAG_ALWAYS_ON set 
for other reasons, the driver would still go and clear the flag, which 
might break things.

Also, unrelated to the above and not a problem in practice at the very 
moment, but I think clocks should also be dealt with somehow. Something, 
at early-ish boot stage, should mark the relevant clocks as in use, so 
that there's no chance they would be turned off when the main kernel has 
started (the main display driver is often a module).

It would be nice to deal with all the above in a single place. I wonder 
if the tidss driver itself could somehow be split into two parts, an 
early part that would probe with minimal dependencies, mainly to reserve 
the core resources without doing any kind of DRM init. And a main part 
which would (somehow) finish the initialization at a later point, when 
we have the filesystem (for firmware) and the other bridge/panel drivers 
have probed.

That can be somewhat achieved with simplefb or simpledrm, though, but we 
can't do any TI DSS specific things there, and it also creates a 
requirement to have either of those drivers built-in, and the related DT 
nodes to be added.

  Tomi

> Another option here would perhaps be to change the driver framework
> (drivers/base/platform.c) which attaches and detaches the PD, and make
> it somehow optional, allowing the driver the manage the PD. That option
> has two downsides: 1) the driver _has_ to manage the PD, which would
> rule out the use of simplefb and simpledrm, and 2) it would leave the PD
> in off state from Linux's perspective until a driver enables the PD, and
> that might mean that the PD gets actually disabled as part of normal
> system wide power management (disabling unused resources).
> 
> Yet another option would be to do this outside the ti_sci_pm_domains
> driver: a piece of code that would somehow be ran after the
> ti_sci_pm_domains driver has probed (so that we have the PDs), but
> before tidss/simplefb/simpledrm probes. The problem here is the
> "somehow" part. Also, this would partly have the same issue 2) as
> mentioned above.
> 
> TODO: If this approach is ok, sci-pm-domain.yaml needs to be extended.
> Also, it sounds a bit like the cell value is not a bit-mask, so maybe
> adding TI_SCI_PD_KEEP_BOOT_STATE flag this way is not fine.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>   drivers/pmdomain/ti/ti_sci_pm_domains.c    | 27 +++++++++++++++++++++++++--
>   include/dt-bindings/soc/ti,sci_pm_domain.h |  1 +
>   2 files changed, 26 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c
> index 1510d5ddae3d..b71b390aaa39 100644
> --- a/drivers/pmdomain/ti/ti_sci_pm_domains.c
> +++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c
> @@ -103,7 +103,7 @@ static struct generic_pm_domain *ti_sci_pd_xlate(
>   		return ERR_PTR(-ENOENT);
>   
>   	genpd_to_ti_sci_pd(genpd_data->domains[idx])->exclusive =
> -		genpdspec->args[1];
> +		genpdspec->args[1] & TI_SCI_PD_EXCLUSIVE;
>   
>   	return genpd_data->domains[idx];
>   }
> @@ -161,6 +161,8 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
>   				break;
>   
>   			if (args.args_count >= 1 && args.np == dev->of_node) {
> +				bool is_on = false;
> +
>   				if (args.args[0] > max_id) {
>   					max_id = args.args[0];
>   				} else {
> @@ -189,7 +191,28 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
>   				pd->idx = args.args[0];
>   				pd->parent = pd_provider;
>   
> -				pm_genpd_init(&pd->pd, NULL, true);
> +				/*
> +				 * If TI_SCI_PD_KEEP_BOOT_STATE is set and the
> +				 * PD has been enabled by the bootloader, set
> +				 * the PD to GENPD_FLAG_ALWAYS_ON. This will
> +				 * make sure the PD stays enabled until a driver
> +				 * takes over and clears the GENPD_FLAG_ALWAYS_ON
> +				 * flag.
> +				 */
> +				if (args.args_count > 1 &&
> +				    args.args[1] & TI_SCI_PD_KEEP_BOOT_STATE) {
> +					/*
> +					 * We ignore any error here, and in case
> +					 * of error just assume the PD is off.
> +					 */
> +					pd_provider->ti_sci->ops.dev_ops.is_on(pd_provider->ti_sci,
> +						pd->idx, NULL, &is_on);
> +
> +					if (is_on)
> +						pd->pd.flags |= GENPD_FLAG_ALWAYS_ON;
> +				}
> +
> +				pm_genpd_init(&pd->pd, NULL, !is_on);
>   
>   				list_add(&pd->node, &pd_provider->pd_list);
>   			}
> diff --git a/include/dt-bindings/soc/ti,sci_pm_domain.h b/include/dt-bindings/soc/ti,sci_pm_domain.h
> index 8f2a7360b65e..af610208e3a3 100644
> --- a/include/dt-bindings/soc/ti,sci_pm_domain.h
> +++ b/include/dt-bindings/soc/ti,sci_pm_domain.h
> @@ -3,6 +3,7 @@
>   #ifndef __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
>   #define __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
>   
> +#define TI_SCI_PD_KEEP_BOOT_STATE 2
>   #define TI_SCI_PD_EXCLUSIVE	1
>   #define TI_SCI_PD_SHARED	0
>   
> 

Re: [PATCH RFC 2/2] pmdomain: ti-sci: Support retaining PD boot time state

[Ulf Hansson]

+ Abel, Saravanna, Stephen

On Mon, 15 Apr 2024 at 19:17, Tomi Valkeinen
<tomi.valkeinen@ideasonboard.com> wrote:
>
> On 15/04/2024 19:00, Tomi Valkeinen wrote:
> > Add a new flag, TI_SCI_PD_KEEP_BOOT_STATE, which can be set in the dts
> > when referring to power domains. When this flag is set, the ti-sci
> > driver will check if the PD is currently enabled in the HW, and if so,
> > set the GENPD_FLAG_ALWAYS_ON flag so that the PD will stay enabled.
> >
> > The main issue I'm trying to solve here is this:
> >
> > If the Display Subsystem (DSS) has been enabled by the bootloader, the
> > related PD has also been enabled in the HW. When the tidss driver
> > probes, the driver framework will automatically enable the PD. While
> > executing the probe function it is very common for the probe to return
> > EPROBE_DEFER, and, in rarer cases, an actual error. When this happens
> > (probe() returns an error), the driver framework will automatically
> > disable the related PD.
> >
> > Powering off the PD while the DSS is enabled and displaying a picture
> > will cause the DSS HW to enter a bad state, from which (afaik) it can't
> > be woken up except with full power-cycle. Trying to access the DSS in
> > this state (e.g. when retrying the probe) will usually cause the board
> > to hang sooner or later.
> >
> > Even if we wouldn't have this board-hangs issue, it's nice to be able to
> > keep the DSS PD enabled: we want to keep the DSS enabled when the
> > bootloader has enabled the screen. If, instead, we disable the PD at the
> > first EPROBE_DEFER, the screen will (probably) go black.
>
> A few things occurred to me. The driver is supposed to clear the
> GENPD_FLAG_ALWAYS_ON when the driver has probed successfully. There are
> two possible issues with that:
>
> - Afaics, there's no API to do that, and currently I just clear the bit
> in genpd->flags. There's a clear race there, so some locking would be
> required.
>
> - This uses the GENPD_FLAG_ALWAYS_ON flag to say "PD is always on, until
> the driver has started". If the PD would have GENPD_FLAG_ALWAYS_ON set
> for other reasons, the driver would still go and clear the flag, which
> might break things.
>
> Also, unrelated to the above and not a problem in practice at the very
> moment, but I think clocks should also be dealt with somehow. Something,
> at early-ish boot stage, should mark the relevant clocks as in use, so
> that there's no chance they would be turned off when the main kernel has
> started (the main display driver is often a module).
>
> It would be nice to deal with all the above in a single place. I wonder
> if the tidss driver itself could somehow be split into two parts, an
> early part that would probe with minimal dependencies, mainly to reserve
> the core resources without doing any kind of DRM init. And a main part
> which would (somehow) finish the initialization at a later point, when
> we have the filesystem (for firmware) and the other bridge/panel drivers
> have probed.
>
> That can be somewhat achieved with simplefb or simpledrm, though, but we
> can't do any TI DSS specific things there, and it also creates a
> requirement to have either of those drivers built-in, and the related DT
> nodes to be added.

Without going into too much detail, this and similar problems have
been discussed in the past. With the fw_devlink and the ->sync_state()
callback we are getting closer to a solution, but for genpd a solution
is still pending.

If you want to read up on earlier discussions and join us moving
forward, that would be great. The last attempt for genpd to move this
forward was posted by Abel Vesa:
https://lore.kernel.org/linux-pm/20230621144019.3219858-1-abel.vesa@linaro.org/

Beyond that, we have also discussed various solutions at the last LPC
in Richmond. I think the consensus at that point was that Saravana
targeted to post something for clocks - and when that was done, we
should do the similar thing for genpd. Anyway, I have looped them into
this thread, so they can share any updates on their side of the
matter.

>
>   Tomi

Kind regards
Uffe

>
> > Another option here would perhaps be to change the driver framework
> > (drivers/base/platform.c) which attaches and detaches the PD, and make
> > it somehow optional, allowing the driver the manage the PD. That option
> > has two downsides: 1) the driver _has_ to manage the PD, which would
> > rule out the use of simplefb and simpledrm, and 2) it would leave the PD
> > in off state from Linux's perspective until a driver enables the PD, and
> > that might mean that the PD gets actually disabled as part of normal
> > system wide power management (disabling unused resources).
> >
> > Yet another option would be to do this outside the ti_sci_pm_domains
> > driver: a piece of code that would somehow be ran after the
> > ti_sci_pm_domains driver has probed (so that we have the PDs), but
> > before tidss/simplefb/simpledrm probes. The problem here is the
> > "somehow" part. Also, this would partly have the same issue 2) as
> > mentioned above.
> >
> > TODO: If this approach is ok, sci-pm-domain.yaml needs to be extended.
> > Also, it sounds a bit like the cell value is not a bit-mask, so maybe
> > adding TI_SCI_PD_KEEP_BOOT_STATE flag this way is not fine.
> >
> > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > ---
> >   drivers/pmdomain/ti/ti_sci_pm_domains.c    | 27 +++++++++++++++++++++++++--
> >   include/dt-bindings/soc/ti,sci_pm_domain.h |  1 +
> >   2 files changed, 26 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c
> > index 1510d5ddae3d..b71b390aaa39 100644
> > --- a/drivers/pmdomain/ti/ti_sci_pm_domains.c
> > +++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c
> > @@ -103,7 +103,7 @@ static struct generic_pm_domain *ti_sci_pd_xlate(
> >               return ERR_PTR(-ENOENT);
> >
> >       genpd_to_ti_sci_pd(genpd_data->domains[idx])->exclusive =
> > -             genpdspec->args[1];
> > +             genpdspec->args[1] & TI_SCI_PD_EXCLUSIVE;
> >
> >       return genpd_data->domains[idx];
> >   }
> > @@ -161,6 +161,8 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
> >                               break;
> >
> >                       if (args.args_count >= 1 && args.np == dev->of_node) {
> > +                             bool is_on = false;
> > +
> >                               if (args.args[0] > max_id) {
> >                                       max_id = args.args[0];
> >                               } else {
> > @@ -189,7 +191,28 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
> >                               pd->idx = args.args[0];
> >                               pd->parent = pd_provider;
> >
> > -                             pm_genpd_init(&pd->pd, NULL, true);
> > +                             /*
> > +                              * If TI_SCI_PD_KEEP_BOOT_STATE is set and the
> > +                              * PD has been enabled by the bootloader, set
> > +                              * the PD to GENPD_FLAG_ALWAYS_ON. This will
> > +                              * make sure the PD stays enabled until a driver
> > +                              * takes over and clears the GENPD_FLAG_ALWAYS_ON
> > +                              * flag.
> > +                              */
> > +                             if (args.args_count > 1 &&
> > +                                 args.args[1] & TI_SCI_PD_KEEP_BOOT_STATE) {
> > +                                     /*
> > +                                      * We ignore any error here, and in case
> > +                                      * of error just assume the PD is off.
> > +                                      */
> > +                                     pd_provider->ti_sci->ops.dev_ops.is_on(pd_provider->ti_sci,
> > +                                             pd->idx, NULL, &is_on);
> > +
> > +                                     if (is_on)
> > +                                             pd->pd.flags |= GENPD_FLAG_ALWAYS_ON;
> > +                             }
> > +
> > +                             pm_genpd_init(&pd->pd, NULL, !is_on);
> >
> >                               list_add(&pd->node, &pd_provider->pd_list);
> >                       }
> > diff --git a/include/dt-bindings/soc/ti,sci_pm_domain.h b/include/dt-bindings/soc/ti,sci_pm_domain.h
> > index 8f2a7360b65e..af610208e3a3 100644
> > --- a/include/dt-bindings/soc/ti,sci_pm_domain.h
> > +++ b/include/dt-bindings/soc/ti,sci_pm_domain.h
> > @@ -3,6 +3,7 @@
> >   #ifndef __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
> >   #define __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
> >
> > +#define TI_SCI_PD_KEEP_BOOT_STATE 2
> >   #define TI_SCI_PD_EXCLUSIVE 1
> >   #define TI_SCI_PD_SHARED    0
> >
> >
>

Re: [PATCH RFC 2/2] pmdomain: ti-sci: Support retaining PD boot time state

[Tomi Valkeinen]

Hi,

On 03/05/2024 16:45, Ulf Hansson wrote:
> + Abel, Saravanna, Stephen
> 
> On Mon, 15 Apr 2024 at 19:17, Tomi Valkeinen
> <tomi.valkeinen@ideasonboard.com> wrote:
>>
>> On 15/04/2024 19:00, Tomi Valkeinen wrote:
>>> Add a new flag, TI_SCI_PD_KEEP_BOOT_STATE, which can be set in the dts
>>> when referring to power domains. When this flag is set, the ti-sci
>>> driver will check if the PD is currently enabled in the HW, and if so,
>>> set the GENPD_FLAG_ALWAYS_ON flag so that the PD will stay enabled.
>>>
>>> The main issue I'm trying to solve here is this:
>>>
>>> If the Display Subsystem (DSS) has been enabled by the bootloader, the
>>> related PD has also been enabled in the HW. When the tidss driver
>>> probes, the driver framework will automatically enable the PD. While
>>> executing the probe function it is very common for the probe to return
>>> EPROBE_DEFER, and, in rarer cases, an actual error. When this happens
>>> (probe() returns an error), the driver framework will automatically
>>> disable the related PD.
>>>
>>> Powering off the PD while the DSS is enabled and displaying a picture
>>> will cause the DSS HW to enter a bad state, from which (afaik) it can't
>>> be woken up except with full power-cycle. Trying to access the DSS in
>>> this state (e.g. when retrying the probe) will usually cause the board
>>> to hang sooner or later.
>>>
>>> Even if we wouldn't have this board-hangs issue, it's nice to be able to
>>> keep the DSS PD enabled: we want to keep the DSS enabled when the
>>> bootloader has enabled the screen. If, instead, we disable the PD at the
>>> first EPROBE_DEFER, the screen will (probably) go black.
>>
>> A few things occurred to me. The driver is supposed to clear the
>> GENPD_FLAG_ALWAYS_ON when the driver has probed successfully. There are
>> two possible issues with that:
>>
>> - Afaics, there's no API to do that, and currently I just clear the bit
>> in genpd->flags. There's a clear race there, so some locking would be
>> required.
>>
>> - This uses the GENPD_FLAG_ALWAYS_ON flag to say "PD is always on, until
>> the driver has started". If the PD would have GENPD_FLAG_ALWAYS_ON set
>> for other reasons, the driver would still go and clear the flag, which
>> might break things.
>>
>> Also, unrelated to the above and not a problem in practice at the very
>> moment, but I think clocks should also be dealt with somehow. Something,
>> at early-ish boot stage, should mark the relevant clocks as in use, so
>> that there's no chance they would be turned off when the main kernel has
>> started (the main display driver is often a module).
>>
>> It would be nice to deal with all the above in a single place. I wonder
>> if the tidss driver itself could somehow be split into two parts, an
>> early part that would probe with minimal dependencies, mainly to reserve
>> the core resources without doing any kind of DRM init. And a main part
>> which would (somehow) finish the initialization at a later point, when
>> we have the filesystem (for firmware) and the other bridge/panel drivers
>> have probed.
>>
>> That can be somewhat achieved with simplefb or simpledrm, though, but we
>> can't do any TI DSS specific things there, and it also creates a
>> requirement to have either of those drivers built-in, and the related DT
>> nodes to be added.
> 
> Without going into too much detail, this and similar problems have
> been discussed in the past. With the fw_devlink and the ->sync_state()
> callback we are getting closer to a solution, but for genpd a solution
> is still pending.
> 
> If you want to read up on earlier discussions and join us moving
> forward, that would be great. The last attempt for genpd to move this
> forward was posted by Abel Vesa:
> https://lore.kernel.org/linux-pm/20230621144019.3219858-1-abel.vesa@linaro.org/
> 
> Beyond that, we have also discussed various solutions at the last LPC
> in Richmond. I think the consensus at that point was that Saravana
> targeted to post something for clocks - and when that was done, we
> should do the similar thing for genpd. Anyway, I have looped them into
> this thread, so they can share any updates on their side of the
> matter.

If I understand the series correctly, it has an issue at least for this 
case/platform.

The devlinks are between the consumer devices and the PD provider 
device. TI SCI PD provider has quite a lot of PDs, and all the consumers 
would have to be probed before any of the PDs could be disabled. So, to 
get the display PD disabled, I would have to load, e.g., the GPU driver 
(which I don't even have).

I believe this is the case for the clocks also.

Perhaps that can be considered a feature, but I fear that in practice it 
would mean that most of the time for most users all the boot-time 
enabled powerdomains would be always on.

Nevertheless, I believe the series would fix the issue mentioned in this 
patch, so I'll see if I can get the series working on the TI platform to 
get a bit more experience on this whole issue.

  Tomi

> 
>>
>>    Tomi
> 
> Kind regards
> Uffe
> 
>>
>>> Another option here would perhaps be to change the driver framework
>>> (drivers/base/platform.c) which attaches and detaches the PD, and make
>>> it somehow optional, allowing the driver the manage the PD. That option
>>> has two downsides: 1) the driver _has_ to manage the PD, which would
>>> rule out the use of simplefb and simpledrm, and 2) it would leave the PD
>>> in off state from Linux's perspective until a driver enables the PD, and
>>> that might mean that the PD gets actually disabled as part of normal
>>> system wide power management (disabling unused resources).
>>>
>>> Yet another option would be to do this outside the ti_sci_pm_domains
>>> driver: a piece of code that would somehow be ran after the
>>> ti_sci_pm_domains driver has probed (so that we have the PDs), but
>>> before tidss/simplefb/simpledrm probes. The problem here is the
>>> "somehow" part. Also, this would partly have the same issue 2) as
>>> mentioned above.
>>>
>>> TODO: If this approach is ok, sci-pm-domain.yaml needs to be extended.
>>> Also, it sounds a bit like the cell value is not a bit-mask, so maybe
>>> adding TI_SCI_PD_KEEP_BOOT_STATE flag this way is not fine.
>>>
>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>> ---
>>>    drivers/pmdomain/ti/ti_sci_pm_domains.c    | 27 +++++++++++++++++++++++++--
>>>    include/dt-bindings/soc/ti,sci_pm_domain.h |  1 +
>>>    2 files changed, 26 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c
>>> index 1510d5ddae3d..b71b390aaa39 100644
>>> --- a/drivers/pmdomain/ti/ti_sci_pm_domains.c
>>> +++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c
>>> @@ -103,7 +103,7 @@ static struct generic_pm_domain *ti_sci_pd_xlate(
>>>                return ERR_PTR(-ENOENT);
>>>
>>>        genpd_to_ti_sci_pd(genpd_data->domains[idx])->exclusive =
>>> -             genpdspec->args[1];
>>> +             genpdspec->args[1] & TI_SCI_PD_EXCLUSIVE;
>>>
>>>        return genpd_data->domains[idx];
>>>    }
>>> @@ -161,6 +161,8 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
>>>                                break;
>>>
>>>                        if (args.args_count >= 1 && args.np == dev->of_node) {
>>> +                             bool is_on = false;
>>> +
>>>                                if (args.args[0] > max_id) {
>>>                                        max_id = args.args[0];
>>>                                } else {
>>> @@ -189,7 +191,28 @@ static int ti_sci_pm_domain_probe(struct platform_device *pdev)
>>>                                pd->idx = args.args[0];
>>>                                pd->parent = pd_provider;
>>>
>>> -                             pm_genpd_init(&pd->pd, NULL, true);
>>> +                             /*
>>> +                              * If TI_SCI_PD_KEEP_BOOT_STATE is set and the
>>> +                              * PD has been enabled by the bootloader, set
>>> +                              * the PD to GENPD_FLAG_ALWAYS_ON. This will
>>> +                              * make sure the PD stays enabled until a driver
>>> +                              * takes over and clears the GENPD_FLAG_ALWAYS_ON
>>> +                              * flag.
>>> +                              */
>>> +                             if (args.args_count > 1 &&
>>> +                                 args.args[1] & TI_SCI_PD_KEEP_BOOT_STATE) {
>>> +                                     /*
>>> +                                      * We ignore any error here, and in case
>>> +                                      * of error just assume the PD is off.
>>> +                                      */
>>> +                                     pd_provider->ti_sci->ops.dev_ops.is_on(pd_provider->ti_sci,
>>> +                                             pd->idx, NULL, &is_on);
>>> +
>>> +                                     if (is_on)
>>> +                                             pd->pd.flags |= GENPD_FLAG_ALWAYS_ON;
>>> +                             }
>>> +
>>> +                             pm_genpd_init(&pd->pd, NULL, !is_on);
>>>
>>>                                list_add(&pd->node, &pd_provider->pd_list);
>>>                        }
>>> diff --git a/include/dt-bindings/soc/ti,sci_pm_domain.h b/include/dt-bindings/soc/ti,sci_pm_domain.h
>>> index 8f2a7360b65e..af610208e3a3 100644
>>> --- a/include/dt-bindings/soc/ti,sci_pm_domain.h
>>> +++ b/include/dt-bindings/soc/ti,sci_pm_domain.h
>>> @@ -3,6 +3,7 @@
>>>    #ifndef __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
>>>    #define __DT_BINDINGS_TI_SCI_PM_DOMAIN_H
>>>
>>> +#define TI_SCI_PD_KEEP_BOOT_STATE 2
>>>    #define TI_SCI_PD_EXCLUSIVE 1
>>>    #define TI_SCI_PD_SHARED    0
>>>
>>>
>>

Re: [PATCH RFC 2/2] pmdomain: ti-sci: Support retaining PD boot time state

[Ulf Hansson]

On Fri, 10 May 2024 at 12:19, Tomi Valkeinen
<tomi.valkeinen@ideasonboard.com> wrote:
>
> Hi,
>
> On 03/05/2024 16:45, Ulf Hansson wrote:
> > + Abel, Saravanna, Stephen
> >
> > On Mon, 15 Apr 2024 at 19:17, Tomi Valkeinen
> > <tomi.valkeinen@ideasonboard.com> wrote:
> >>
> >> On 15/04/2024 19:00, Tomi Valkeinen wrote:
> >>> Add a new flag, TI_SCI_PD_KEEP_BOOT_STATE, which can be set in the dts
> >>> when referring to power domains. When this flag is set, the ti-sci
> >>> driver will check if the PD is currently enabled in the HW, and if so,
> >>> set the GENPD_FLAG_ALWAYS_ON flag so that the PD will stay enabled.
> >>>
> >>> The main issue I'm trying to solve here is this:
> >>>
> >>> If the Display Subsystem (DSS) has been enabled by the bootloader, the
> >>> related PD has also been enabled in the HW. When the tidss driver
> >>> probes, the driver framework will automatically enable the PD. While
> >>> executing the probe function it is very common for the probe to return
> >>> EPROBE_DEFER, and, in rarer cases, an actual error. When this happens
> >>> (probe() returns an error), the driver framework will automatically
> >>> disable the related PD.
> >>>
> >>> Powering off the PD while the DSS is enabled and displaying a picture
> >>> will cause the DSS HW to enter a bad state, from which (afaik) it can't
> >>> be woken up except with full power-cycle. Trying to access the DSS in
> >>> this state (e.g. when retrying the probe) will usually cause the board
> >>> to hang sooner or later.
> >>>
> >>> Even if we wouldn't have this board-hangs issue, it's nice to be able to
> >>> keep the DSS PD enabled: we want to keep the DSS enabled when the
> >>> bootloader has enabled the screen. If, instead, we disable the PD at the
> >>> first EPROBE_DEFER, the screen will (probably) go black.
> >>
> >> A few things occurred to me. The driver is supposed to clear the
> >> GENPD_FLAG_ALWAYS_ON when the driver has probed successfully. There are
> >> two possible issues with that:
> >>
> >> - Afaics, there's no API to do that, and currently I just clear the bit
> >> in genpd->flags. There's a clear race there, so some locking would be
> >> required.
> >>
> >> - This uses the GENPD_FLAG_ALWAYS_ON flag to say "PD is always on, until
> >> the driver has started". If the PD would have GENPD_FLAG_ALWAYS_ON set
> >> for other reasons, the driver would still go and clear the flag, which
> >> might break things.
> >>
> >> Also, unrelated to the above and not a problem in practice at the very
> >> moment, but I think clocks should also be dealt with somehow. Something,
> >> at early-ish boot stage, should mark the relevant clocks as in use, so
> >> that there's no chance they would be turned off when the main kernel has
> >> started (the main display driver is often a module).
> >>
> >> It would be nice to deal with all the above in a single place. I wonder
> >> if the tidss driver itself could somehow be split into two parts, an
> >> early part that would probe with minimal dependencies, mainly to reserve
> >> the core resources without doing any kind of DRM init. And a main part
> >> which would (somehow) finish the initialization at a later point, when
> >> we have the filesystem (for firmware) and the other bridge/panel drivers
> >> have probed.
> >>
> >> That can be somewhat achieved with simplefb or simpledrm, though, but we
> >> can't do any TI DSS specific things there, and it also creates a
> >> requirement to have either of those drivers built-in, and the related DT
> >> nodes to be added.
> >
> > Without going into too much detail, this and similar problems have
> > been discussed in the past. With the fw_devlink and the ->sync_state()
> > callback we are getting closer to a solution, but for genpd a solution
> > is still pending.
> >
> > If you want to read up on earlier discussions and join us moving
> > forward, that would be great. The last attempt for genpd to move this
> > forward was posted by Abel Vesa:
> > https://lore.kernel.org/linux-pm/20230621144019.3219858-1-abel.vesa@linaro.org/
> >
> > Beyond that, we have also discussed various solutions at the last LPC
> > in Richmond. I think the consensus at that point was that Saravana
> > targeted to post something for clocks - and when that was done, we
> > should do the similar thing for genpd. Anyway, I have looped them into
> > this thread, so they can share any updates on their side of the
> > matter.
>
> If I understand the series correctly, it has an issue at least for this
> case/platform.
>
> The devlinks are between the consumer devices and the PD provider
> device. TI SCI PD provider has quite a lot of PDs, and all the consumers
> would have to be probed before any of the PDs could be disabled. So, to
> get the display PD disabled, I would have to load, e.g., the GPU driver
> (which I don't even have).
>
> I believe this is the case for the clocks also.
>
> Perhaps that can be considered a feature, but I fear that in practice it
> would mean that most of the time for most users all the boot-time
> enabled powerdomains would be always on.
>
> Nevertheless, I believe the series would fix the issue mentioned in this
> patch, so I'll see if I can get the series working on the TI platform to
> get a bit more experience on this whole issue.

[...]

Just to share a brief update on this matter. I have had an offlist
chat with Saravana and some Qcom guys about this topic. In particular
we looked closer at some implementation details.

Allow me a few weeks, then I will post a series for genpd to implement
the above. I will keep you posted and would appreciate reviews and
tests, of course.

Kind regards
Uffe

Re: [PATCH RFC 2/2] pmdomain: ti-sci: Support retaining PD boot time state

[Tomi Valkeinen]

Hi Ulf,

On 03/05/2024 16:45, Ulf Hansson wrote:
> + Abel, Saravanna, Stephen
> 
> On Mon, 15 Apr 2024 at 19:17, Tomi Valkeinen
> <tomi.valkeinen@ideasonboard.com> wrote:
>>
>> On 15/04/2024 19:00, Tomi Valkeinen wrote:
>>> Add a new flag, TI_SCI_PD_KEEP_BOOT_STATE, which can be set in the dts
>>> when referring to power domains. When this flag is set, the ti-sci
>>> driver will check if the PD is currently enabled in the HW, and if so,
>>> set the GENPD_FLAG_ALWAYS_ON flag so that the PD will stay enabled.
>>>
>>> The main issue I'm trying to solve here is this:
>>>
>>> If the Display Subsystem (DSS) has been enabled by the bootloader, the
>>> related PD has also been enabled in the HW. When the tidss driver
>>> probes, the driver framework will automatically enable the PD. While
>>> executing the probe function it is very common for the probe to return
>>> EPROBE_DEFER, and, in rarer cases, an actual error. When this happens
>>> (probe() returns an error), the driver framework will automatically
>>> disable the related PD.
>>>
>>> Powering off the PD while the DSS is enabled and displaying a picture
>>> will cause the DSS HW to enter a bad state, from which (afaik) it can't
>>> be woken up except with full power-cycle. Trying to access the DSS in
>>> this state (e.g. when retrying the probe) will usually cause the board
>>> to hang sooner or later.
>>>
>>> Even if we wouldn't have this board-hangs issue, it's nice to be able to
>>> keep the DSS PD enabled: we want to keep the DSS enabled when the
>>> bootloader has enabled the screen. If, instead, we disable the PD at the
>>> first EPROBE_DEFER, the screen will (probably) go black.
>>
>> A few things occurred to me. The driver is supposed to clear the
>> GENPD_FLAG_ALWAYS_ON when the driver has probed successfully. There are
>> two possible issues with that:
>>
>> - Afaics, there's no API to do that, and currently I just clear the bit
>> in genpd->flags. There's a clear race there, so some locking would be
>> required.
>>
>> - This uses the GENPD_FLAG_ALWAYS_ON flag to say "PD is always on, until
>> the driver has started". If the PD would have GENPD_FLAG_ALWAYS_ON set
>> for other reasons, the driver would still go and clear the flag, which
>> might break things.
>>
>> Also, unrelated to the above and not a problem in practice at the very
>> moment, but I think clocks should also be dealt with somehow. Something,
>> at early-ish boot stage, should mark the relevant clocks as in use, so
>> that there's no chance they would be turned off when the main kernel has
>> started (the main display driver is often a module).
>>
>> It would be nice to deal with all the above in a single place. I wonder
>> if the tidss driver itself could somehow be split into two parts, an
>> early part that would probe with minimal dependencies, mainly to reserve
>> the core resources without doing any kind of DRM init. And a main part
>> which would (somehow) finish the initialization at a later point, when
>> we have the filesystem (for firmware) and the other bridge/panel drivers
>> have probed.
>>
>> That can be somewhat achieved with simplefb or simpledrm, though, but we
>> can't do any TI DSS specific things there, and it also creates a
>> requirement to have either of those drivers built-in, and the related DT
>> nodes to be added.
> 
> Without going into too much detail, this and similar problems have
> been discussed in the past. With the fw_devlink and the ->sync_state()
> callback we are getting closer to a solution, but for genpd a solution
> is still pending.
> 
> If you want to read up on earlier discussions and join us moving
> forward, that would be great. The last attempt for genpd to move this
> forward was posted by Abel Vesa:
> https://lore.kernel.org/linux-pm/20230621144019.3219858-1-abel.vesa@linaro.org/
> 
> Beyond that, we have also discussed various solutions at the last LPC
> in Richmond. I think the consensus at that point was that Saravana
> targeted to post something for clocks - and when that was done, we
> should do the similar thing for genpd. Anyway, I have looped them into
> this thread, so they can share any updates on their side of the
> matter.

Do you know if there's been any recent work related to this? I tried to 
look around on the lists, but nothing caught my eye.

  Tomi