[RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Rafael J. Wysocki]

Hi Everyone,

These two patches complement the recently made PM core changes related to
the async suspend and resume of devices.  They should apply on top of
6.16-rc3.

They were sent along with the other changes mentioned above:

https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/

(and this is v4 because they have been rebased in the meantime), but they don't
make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
that they are actually needed on ARM (or another architecture using DT).

Thanks!

[RFT][PATCH v4 1/2] PM: sleep: Make async resume handle consumers like children

[Rafael J. Wysocki]

From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Avoid starting "async" resume processing upfront for devices that have
suppliers and start "async" resume processing for a device's consumers
right after resuming the device itself.

Suggested-by: Saravana Kannan <saravanak@google.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---

v3 -> v4: Rebase and update the changelog.

---
 drivers/base/power/main.c |   36 ++++++++++++++++++++++++++++--------
 1 file changed, 28 insertions(+), 8 deletions(-)

--- a/drivers/base/power/main.c
+++ b/drivers/base/power/main.c
@@ -647,14 +647,27 @@
 	/*
 	 * Start processing "async" children of the device unless it's been
 	 * started already for them.
-	 *
-	 * This could have been done for the device's "async" consumers too, but
-	 * they either need to wait for their parents or the processing has
-	 * already started for them after their parents were processed.
 	 */
 	device_for_each_child(dev, func, dpm_async_with_cleanup);
 }
 
+static void dpm_async_resume_subordinate(struct device *dev, async_func_t func)
+{
+	struct device_link *link;
+	int idx;
+
+	dpm_async_resume_children(dev, func);
+
+	idx = device_links_read_lock();
+
+	/* Start processing the device's "async" consumers. */
+	list_for_each_entry_rcu(link, &dev->links.consumers, s_node)
+		if (READ_ONCE(link->status) != DL_STATE_DORMANT)
+			dpm_async_with_cleanup(link->consumer, func);
+
+	device_links_read_unlock(idx);
+}
+
 static void dpm_clear_async_state(struct device *dev)
 {
 	reinit_completion(&dev->power.completion);
@@ -663,7 +676,14 @@
 
 static bool dpm_root_device(struct device *dev)
 {
-	return !dev->parent;
+	lockdep_assert_held(&dpm_list_mtx);
+
+	/*
+	 * Since this function is required to run under dpm_list_mtx, the
+	 * list_empty() below will only return true if the device's list of
+	 * consumers is actually empty before calling it.
+	 */
+	return !dev->parent && list_empty(&dev->links.suppliers);
 }
 
 static void async_resume_noirq(void *data, async_cookie_t cookie);
@@ -752,7 +772,7 @@
 		pm_dev_err(dev, state, async ? " async noirq" : " noirq", error);
 	}
 
-	dpm_async_resume_children(dev, async_resume_noirq);
+	dpm_async_resume_subordinate(dev, async_resume_noirq);
 }
 
 static void async_resume_noirq(void *data, async_cookie_t cookie)
@@ -895,7 +915,7 @@
 		pm_dev_err(dev, state, async ? " async early" : " early", error);
 	}
 
-	dpm_async_resume_children(dev, async_resume_early);
+	dpm_async_resume_subordinate(dev, async_resume_early);
 }
 
 static void async_resume_early(void *data, async_cookie_t cookie)
@@ -1071,7 +1091,7 @@
 		pm_dev_err(dev, state, async ? " async" : "", error);
 	}
 
-	dpm_async_resume_children(dev, async_resume);
+	dpm_async_resume_subordinate(dev, async_resume);
 }
 
 static void async_resume(void *data, async_cookie_t cookie)

[RFT][PATCH v4 2/2] PM: sleep: Make async suspend handle suppliers like parents

[Rafael J. Wysocki]

From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Avoid starting "async" suspend processing upfront for devices that have
consumers and start "async" suspend processing for a device's suppliers
right after suspending the device itself.

Suggested-by: Saravana Kannan <saravanak@google.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---

v3 -> v4: Rebase and update the changelog.

---
 drivers/base/power/main.c |   37 +++++++++++++++++++++++++++++++------
 1 file changed, 31 insertions(+), 6 deletions(-)

--- a/drivers/base/power/main.c
+++ b/drivers/base/power/main.c
@@ -1257,10 +1257,15 @@
 		return false;
 	}
 
-	return true;
+	/*
+	 * Since this function is required to run under dpm_list_mtx, the
+	 * list_empty() below will only return true if the device's list of
+	 * consumers is actually empty before calling it.
+	 */
+	return list_empty(&dev->links.consumers);
 }
 
-static void dpm_async_suspend_parent(struct device *dev, async_func_t func)
+static bool dpm_async_suspend_parent(struct device *dev, async_func_t func)
 {
 	guard(mutex)(&dpm_list_mtx);
 
@@ -1272,11 +1277,31 @@
 	 * deleted before it.
 	 */
 	if (!device_pm_initialized(dev))
-		return;
+		return false;
 
 	/* Start processing the device's parent if it is "async". */
 	if (dev->parent)
 		dpm_async_with_cleanup(dev->parent, func);
+
+	return true;
+}
+
+static void dpm_async_suspend_superior(struct device *dev, async_func_t func)
+{
+	struct device_link *link;
+	int idx;
+
+	if (!dpm_async_suspend_parent(dev, func))
+		return;
+
+	idx = device_links_read_lock();
+
+	/* Start processing the device's "async" suppliers. */
+	list_for_each_entry_rcu(link, &dev->links.suppliers, c_node)
+		if (READ_ONCE(link->status) != DL_STATE_DORMANT)
+			dpm_async_with_cleanup(link->supplier, func);
+
+	device_links_read_unlock(idx);
 }
 
 /**
@@ -1400,7 +1425,7 @@
 	if (error || async_error)
 		return error;
 
-	dpm_async_suspend_parent(dev, async_suspend_noirq);
+	dpm_async_suspend_superior(dev, async_suspend_noirq);
 
 	return 0;
 }
@@ -1596,7 +1621,7 @@
 	if (error || async_error)
 		return error;
 
-	dpm_async_suspend_parent(dev, async_suspend_late);
+	dpm_async_suspend_superior(dev, async_suspend_late);
 
 	return 0;
 }
@@ -1887,7 +1912,7 @@
 	if (error || async_error)
 		return error;
 
-	dpm_async_suspend_parent(dev, async_suspend);
+	dpm_async_suspend_superior(dev, async_suspend);
 
 	return 0;
 }

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Ulf Hansson]

On Mon, 23 Jun 2025 at 14:55, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>
> Hi Everyone,
>
> These two patches complement the recently made PM core changes related to
> the async suspend and resume of devices.  They should apply on top of
> 6.16-rc3.
>
> They were sent along with the other changes mentioned above:
>
> https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
> https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
>
> (and this is v4 because they have been rebased in the meantime), but they don't
> make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
> that they are actually needed on ARM (or another architecture using DT).
>
> Thanks!

Hi Rafael,

I haven't yet got the time to test these, but the code looks good to
me, so feel free to add for the series:

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

Kind regards
Uffe

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Mario Limonciello]

On 6/26/2025 4:46 AM, Ulf Hansson wrote:
> On Mon, 23 Jun 2025 at 14:55, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>>
>> Hi Everyone,
>>
>> These two patches complement the recently made PM core changes related to
>> the async suspend and resume of devices.  They should apply on top of
>> 6.16-rc3.
>>
>> They were sent along with the other changes mentioned above:
>>
>> https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
>> https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
>>
>> (and this is v4 because they have been rebased in the meantime), but they don't
>> make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
>> that they are actually needed on ARM (or another architecture using DT).
>>
>> Thanks!
> 
> Hi Rafael,
> 
> I haven't yet got the time to test these, but the code looks good to
> me, so feel free to add for the series:
> 
> Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
> 
> Kind regards
> Uffe

I passed this series to some internal guys to test on a wide variety of 
AMD x86 hardware.  The initial testing looks good.
Will keep you apprised if anything pops up.

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Rafael J. Wysocki]

On Fri, Jun 27, 2025 at 12:28 AM Mario Limonciello
<mario.limonciello@amd.com> wrote:
>
> On 6/26/2025 4:46 AM, Ulf Hansson wrote:
> > On Mon, 23 Jun 2025 at 14:55, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> >>
> >> Hi Everyone,
> >>
> >> These two patches complement the recently made PM core changes related to
> >> the async suspend and resume of devices.  They should apply on top of
> >> 6.16-rc3.
> >>
> >> They were sent along with the other changes mentioned above:
> >>
> >> https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
> >> https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
> >>
> >> (and this is v4 because they have been rebased in the meantime), but they don't
> >> make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
> >> that they are actually needed on ARM (or another architecture using DT).
> >>
> >> Thanks!
> >
> > Hi Rafael,
> >
> > I haven't yet got the time to test these, but the code looks good to
> > me, so feel free to add for the series:
> >
> > Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
> >
> > Kind regards
> > Uffe
>
> I passed this series to some internal guys to test on a wide variety of
> AMD x86 hardware.  The initial testing looks good.
> Will keep you apprised if anything pops up.

Thanks!

It would also help if you could check whether or not there is any
measurable performance (that is, system suspend and resume time)
difference between "before" and "after".

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Mario Limonciello]

On 6/27/2025 5:40 AM, Rafael J. Wysocki wrote:
> On Fri, Jun 27, 2025 at 12:28 AM Mario Limonciello
> <mario.limonciello@amd.com> wrote:
>>
>> On 6/26/2025 4:46 AM, Ulf Hansson wrote:
>>> On Mon, 23 Jun 2025 at 14:55, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>>>>
>>>> Hi Everyone,
>>>>
>>>> These two patches complement the recently made PM core changes related to
>>>> the async suspend and resume of devices.  They should apply on top of
>>>> 6.16-rc3.
>>>>
>>>> They were sent along with the other changes mentioned above:
>>>>
>>>> https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
>>>> https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
>>>>
>>>> (and this is v4 because they have been rebased in the meantime), but they don't
>>>> make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
>>>> that they are actually needed on ARM (or another architecture using DT).
>>>>
>>>> Thanks!
>>>
>>> Hi Rafael,
>>>
>>> I haven't yet got the time to test these, but the code looks good to
>>> me, so feel free to add for the series:
>>>
>>> Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
>>>
>>> Kind regards
>>> Uffe
>>
>> I passed this series to some internal guys to test on a wide variety of
>> AMD x86 hardware.  The initial testing looks good.
>> Will keep you apprised if anything pops up.
> 
> Thanks!
> 
> It would also help if you could check whether or not there is any
> measurable performance (that is, system suspend and resume time)
> difference between "before" and "after".

Sure thing.

Just to make sure we have an aligned measurement methodology:

I asked them to do this both with and without the patches.

* set /sys/power/pm_debug_messages before running and then capture all 
the timing prints.
* add up all suspend events and get a total
* add up all resume events and get a total
* repeat 5 times
* calculate averages for the 5 runs

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Rafael J. Wysocki]

On Fri, Jun 27, 2025 at 4:01 PM Mario Limonciello
<mario.limonciello@amd.com> wrote:
>
> On 6/27/2025 5:40 AM, Rafael J. Wysocki wrote:
> > On Fri, Jun 27, 2025 at 12:28 AM Mario Limonciello
> > <mario.limonciello@amd.com> wrote:
> >>
> >> On 6/26/2025 4:46 AM, Ulf Hansson wrote:
> >>> On Mon, 23 Jun 2025 at 14:55, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> >>>>
> >>>> Hi Everyone,
> >>>>
> >>>> These two patches complement the recently made PM core changes related to
> >>>> the async suspend and resume of devices.  They should apply on top of
> >>>> 6.16-rc3.
> >>>>
> >>>> They were sent along with the other changes mentioned above:
> >>>>
> >>>> https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
> >>>> https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
> >>>>
> >>>> (and this is v4 because they have been rebased in the meantime), but they don't
> >>>> make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
> >>>> that they are actually needed on ARM (or another architecture using DT).
> >>>>
> >>>> Thanks!
> >>>
> >>> Hi Rafael,
> >>>
> >>> I haven't yet got the time to test these, but the code looks good to
> >>> me, so feel free to add for the series:
> >>>
> >>> Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
> >>>
> >>> Kind regards
> >>> Uffe
> >>
> >> I passed this series to some internal guys to test on a wide variety of
> >> AMD x86 hardware.  The initial testing looks good.
> >> Will keep you apprised if anything pops up.
> >
> > Thanks!
> >
> > It would also help if you could check whether or not there is any
> > measurable performance (that is, system suspend and resume time)
> > difference between "before" and "after".
>
> Sure thing.
>
> Just to make sure we have an aligned measurement methodology:
>
> I asked them to do this both with and without the patches.
>
> * set /sys/power/pm_debug_messages before running and then capture all
> the timing prints.
> * add up all suspend events and get a total
> * add up all resume events and get a total
> * repeat 5 times
> * calculate averages for the 5 runs

Sounds good!

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Mario Limonciello]

On 6/27/2025 12:31 PM, Rafael J. Wysocki wrote:
> On Fri, Jun 27, 2025 at 4:01 PM Mario Limonciello
> <mario.limonciello@amd.com> wrote:
>>
>> On 6/27/2025 5:40 AM, Rafael J. Wysocki wrote:
>>> On Fri, Jun 27, 2025 at 12:28 AM Mario Limonciello
>>> <mario.limonciello@amd.com> wrote:
>>>>
>>>> On 6/26/2025 4:46 AM, Ulf Hansson wrote:
>>>>> On Mon, 23 Jun 2025 at 14:55, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> These two patches complement the recently made PM core changes related to
>>>>>> the async suspend and resume of devices.  They should apply on top of
>>>>>> 6.16-rc3.
>>>>>>
>>>>>> They were sent along with the other changes mentioned above:
>>>>>>
>>>>>> https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
>>>>>> https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
>>>>>>
>>>>>> (and this is v4 because they have been rebased in the meantime), but they don't
>>>>>> make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
>>>>>> that they are actually needed on ARM (or another architecture using DT).
>>>>>>
>>>>>> Thanks!
>>>>>
>>>>> Hi Rafael,
>>>>>
>>>>> I haven't yet got the time to test these, but the code looks good to
>>>>> me, so feel free to add for the series:
>>>>>
>>>>> Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
>>>>>
>>>>> Kind regards
>>>>> Uffe
>>>>
>>>> I passed this series to some internal guys to test on a wide variety of
>>>> AMD x86 hardware.  The initial testing looks good.
>>>> Will keep you apprised if anything pops up.
>>>
>>> Thanks!
>>>
>>> It would also help if you could check whether or not there is any
>>> measurable performance (that is, system suspend and resume time)
>>> difference between "before" and "after".
>>
>> Sure thing.
>>
>> Just to make sure we have an aligned measurement methodology:
>>
>> I asked them to do this both with and without the patches.
>>
>> * set /sys/power/pm_debug_messages before running and then capture all
>> the timing prints.
>> * add up all suspend events and get a total
>> * add up all resume events and get a total
>> * repeat 5 times
>> * calculate averages for the 5 runs
> 
> Sounds good!

This is across two different systems.

The first one didn't have a very large difference in average (20ms)

KRK No patch
Suspend 235.6862
Resume 2220.3976

KRK patch
Suspend 233.3544
Resume 2202.199

The second one had about a 15% drop in average suspend time; but I think 
I suspect this isn't a big enough data sample.  I say that because both 
sides had one cycle take longer than the rest on avearge.

STX nopatch
Suspend 774.39638
Resume 1893.5252

STX patch
Suspend 651.9756
Resume 1895.725

If I exclude that long cycle on both (so average of 4) the drop is 10%

STX No patch
Suspend 319.353725
Resume 2256.0025

STX patch
Suspend	292.482
Resume 2257.27


I'm personally thinking 5 cycles isn't enough for showing "real" gains 
are there.
Probably need a much larger sample size to get statistically relevant 
numbers.

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Sudeep Holla]

On Mon, Jun 23, 2025 at 02:44:09PM +0200, Rafael J. Wysocki wrote:
> Hi Everyone,
> 
> These two patches complement the recently made PM core changes related to
> the async suspend and resume of devices.  They should apply on top of
> 6.16-rc3.
> 
> They were sent along with the other changes mentioned above:
> 
> https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
> https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
> 
> (and this is v4 because they have been rebased in the meantime), but they don't
> make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
> that they are actually needed on ARM (or another architecture using DT).
> 

All the changes LGTM.

Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>

I don't have platform to exercise this patch much ATM, so sorry can't do
much testing though.

-- 
Regards,
Sudeep

Re: [RFT][PATCH v4 0/2] PM: sleep: Handle async suppliers like parents and async consumers like children

[Rafael J. Wysocki]

On Mon, Jun 30, 2025 at 3:13 PM Sudeep Holla <sudeep.holla@arm.com> wrote:
>
> On Mon, Jun 23, 2025 at 02:44:09PM +0200, Rafael J. Wysocki wrote:
> > Hi Everyone,
> >
> > These two patches complement the recently made PM core changes related to
> > the async suspend and resume of devices.  They should apply on top of
> > 6.16-rc3.
> >
> > They were sent along with the other changes mentioned above:
> >
> > https://lore.kernel.org/linux-pm/2229735.Mh6RI2rZIc@rjwysocki.net/
> > https://lore.kernel.org/linux-pm/2651185.Lt9SDvczpP@rjwysocki.net/
> >
> > (and this is v4 because they have been rebased in the meantime), but they don't
> > make any difference on my test-bed x86 systems, so I'd appreciate a confirmation
> > that they are actually needed on ARM (or another architecture using DT).
> >
>
> All the changes LGTM.
>
> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>

Thank you!

> I don't have platform to exercise this patch much ATM, so sorry can't do
> much testing though.

No worries.

Everyone seems to be liking this series, so I'm going to queue it up
for 6.17 and we'll see how it goes.

Thanks!