[PATCH v2 0/3] Fixes for hybrid sleep

[PATCH v2 0/3] Fixes for hybrid sleep

[Mario Limonciello (AMD)]

Ionut Nechita reported recently a hibernate failure, but in debugging
the issue it's actually not a hibernate failure; but a hybrid sleep
failure.

Multiple changes related to the change of when swap is disabled in
the suspend sequence contribute to the failure.  See the individual
patches for details.

Link: https://gitlab.freedesktop.org/drm/amd/-/issues/4573

NOTE: I realize this is super late in the cycle, so sorry about that,
but I debugged it as fast as I could as soon as I heard about it.
If it needs to push out to the next cycle it is what it is.

As it touches two subsystems it either needs to go through linux-pm
or drm.  Patch 3 has an Ack from Alex, this should merge through
linux-pm.

---
v2:
 * Fix LKP robot errors without CONFIG_SUSPEND
 * Add tags


Mario Limonciello (AMD) (3):
  PM: hibernate: Fix hybrid-sleep
  PM: hibernate: Add pm_hibernation_mode_is_suspend()
  drm/amd: Fix hybrid sleep

 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c |  2 +-
 include/linux/suspend.h                 |  2 ++
 kernel/power/hibernate.c                | 22 +++++++++++++++++++++-
 3 files changed, 24 insertions(+), 2 deletions(-)

-- 
2.43.0

[PATCH v2 1/3] PM: hibernate: Fix hybrid-sleep

[Mario Limonciello (AMD)]

Hybrid sleep will hibernate the system followed by running through
the suspend routine.  Since both the hibernate and the suspend routine
will call pm_restrict_gfp_mask(), pm_restore_gfp_mask() must be called
before starting the suspend sequence.

Add an explicit call to pm_restore_gfp_mask() to power_down() before
the suspend sequence starts. Don't call pm_restore_gfp_mask() when
exiting suspend sequence it is already called:

```
power_down()
->suspend_devices_and_enter()
-->dpm_resume_end()
```

Reported-by: Ionut Nechita <ionut_n2001@yahoo.com>
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/4573
Tested-by: Ionut Nechita <ionut_n2001@yahoo.com>
Fixes: 12ffc3b1513eb ("PM: Restrict swap use to later in the suspend sequence")
Tested-by: Kenneth Crudup <kenny@panix.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
---
v2:
 * Move under CONFIG_SUSPEND scope (LKP robot)
 * Add tags
---
 kernel/power/hibernate.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
index 2f66ab4538231..52c1818749724 100644
--- a/kernel/power/hibernate.c
+++ b/kernel/power/hibernate.c
@@ -695,6 +695,7 @@ static void power_down(void)
 
 #ifdef CONFIG_SUSPEND
 	if (hibernation_mode == HIBERNATION_SUSPEND) {
+		pm_restore_gfp_mask();
 		error = suspend_devices_and_enter(mem_sleep_current);
 		if (error) {
 			hibernation_mode = hibernation_ops ?
@@ -862,7 +863,15 @@ int hibernate(void)
 				power_down();
 		}
 		in_suspend = 0;
-		pm_restore_gfp_mask();
+		switch (hibernation_mode) {
+#ifdef CONFIG_SUSPEND
+		case HIBERNATION_SUSPEND:
+			break;
+#endif
+		default:
+			pm_restore_gfp_mask();
+			break;
+		}
 	} else {
 		pm_pr_dbg("Hibernation image restored successfully.\n");
 	}
-- 
2.43.0

[PATCH v2 2/3] PM: hibernate: Add pm_hibernation_mode_is_suspend()

[Mario Limonciello (AMD)]

Some drivers have different flows for hibernation and suspend. If
the driver opportunistically will skip thaw() then it needs a hint
to know what is happening after the hibernate.

Introduce a new symbol pm_hibernation_mode_is_suspend() that drivers
can call to determine if suspending the system for this purpose.

Tested-by: Ionut Nechita <ionut_n2001@yahoo.com>
Tested-by: Kenneth Crudup <kenny@panix.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
---
v2:
 * Move under CONFIG_SUSPEND scope (LKP robot)
 * Add tags
---
 include/linux/suspend.h  |  2 ++
 kernel/power/hibernate.c | 11 +++++++++++
 2 files changed, 13 insertions(+)

diff --git a/include/linux/suspend.h b/include/linux/suspend.h
index 317ae31e89b37..0664c685f0b24 100644
--- a/include/linux/suspend.h
+++ b/include/linux/suspend.h
@@ -276,6 +276,7 @@ extern void arch_suspend_enable_irqs(void);
 
 extern int pm_suspend(suspend_state_t state);
 extern bool sync_on_suspend_enabled;
+bool pm_hibernation_mode_is_suspend(void);
 #else /* !CONFIG_SUSPEND */
 #define suspend_valid_only_mem	NULL
 
@@ -288,6 +289,7 @@ static inline bool pm_suspend_via_firmware(void) { return false; }
 static inline bool pm_resume_via_firmware(void) { return false; }
 static inline bool pm_suspend_no_platform(void) { return false; }
 static inline bool pm_suspend_default_s2idle(void) { return false; }
+static inline bool pm_hibernation_mode_is_suspend(void) { return false; }
 
 static inline void suspend_set_ops(const struct platform_suspend_ops *ops) {}
 static inline int pm_suspend(suspend_state_t state) { return -ENOSYS; }
diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
index 52c1818749724..09f5271c98f35 100644
--- a/kernel/power/hibernate.c
+++ b/kernel/power/hibernate.c
@@ -80,6 +80,17 @@ static const struct platform_hibernation_ops *hibernation_ops;
 
 static atomic_t hibernate_atomic = ATOMIC_INIT(1);
 
+#ifdef CONFIG_SUSPEND
+/**
+ * pm_hibernation_mode_is_suspend - Check if hibernation has been set to suspend
+ */
+bool pm_hibernation_mode_is_suspend(void)
+{
+	return hibernation_mode == HIBERNATION_SUSPEND;
+}
+EXPORT_SYMBOL_GPL(pm_hibernation_mode_is_suspend);
+#endif
+
 bool hibernate_acquire(void)
 {
 	return atomic_add_unless(&hibernate_atomic, -1, 0);
-- 
2.43.0

[PATCH v2 3/3] drm/amd: Fix hybrid sleep

[Mario Limonciello (AMD)]

[Why]
commit 530694f54dd5e ("drm/amdgpu: do not resume device in thaw for
normal hibernation") optimized the flow for systems that are going
into S4 where the power would be turned off.  Basically the thaw()
callback wouldn't resume the device if the hibernation image was
successfully created since the system would be powered off.

This however isn't the correct flow for a system entering into
s0i3 after the hibernation image is created.  Some of the amdgpu
callbacks have different behavior depending upon the intended
state of the suspend.

[How]
Use pm_hibernation_mode_is_suspend() as an input to decide whether
to run resume during thaw() callback.

Reported-by: Ionut Nechita <ionut_n2001@yahoo.com>
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/4573
Tested-by: Ionut Nechita <ionut_n2001@yahoo.com>
Fixes: 530694f54dd5e ("drm/amdgpu: do not resume device in thaw for normal hibernation")
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Kenneth Crudup <kenny@panix.com>
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
---
v2:
 * Add tags
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 395c6be901ce7..dcea66aadfa33 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -2665,7 +2665,7 @@ static int amdgpu_pmops_thaw(struct device *dev)
 	struct drm_device *drm_dev = dev_get_drvdata(dev);
 
 	/* do not resume device if it's normal hibernation */
-	if (!pm_hibernate_is_recovering())
+	if (!pm_hibernate_is_recovering() && !pm_hibernation_mode_is_suspend())
 		return 0;
 
 	return amdgpu_device_resume(drm_dev, true);
-- 
2.43.0

Re: [PATCH v2 0/3] Fixes for hybrid sleep

[Rafael J. Wysocki]

On Thu, Sep 25, 2025 at 5:59 PM Mario Limonciello (AMD)
<superm1@kernel.org> wrote:
>
> Ionut Nechita reported recently a hibernate failure, but in debugging
> the issue it's actually not a hibernate failure; but a hybrid sleep
> failure.
>
> Multiple changes related to the change of when swap is disabled in
> the suspend sequence contribute to the failure.  See the individual
> patches for details.
>
> Link: https://gitlab.freedesktop.org/drm/amd/-/issues/4573
>
> NOTE: I realize this is super late in the cycle, so sorry about that,

No worries.

> but I debugged it as fast as I could as soon as I heard about it.
> If it needs to push out to the next cycle it is what it is.

Well, it can go into 6.18.

Fortunately, hybrid sleep isn't too popular with Linux users.

> As it touches two subsystems it either needs to go through linux-pm
> or drm.  Patch 3 has an Ack from Alex, this should merge through
> linux-pm.

Sure.

Thanks!

Re: [PATCH v2 1/3] PM: hibernate: Fix hybrid-sleep

[Rafael J. Wysocki]

On Thu, Sep 25, 2025 at 5:59 PM Mario Limonciello (AMD)
<superm1@kernel.org> wrote:
>
> Hybrid sleep will hibernate the system followed by running through
> the suspend routine.  Since both the hibernate and the suspend routine
> will call pm_restrict_gfp_mask(), pm_restore_gfp_mask() must be called
> before starting the suspend sequence.
>
> Add an explicit call to pm_restore_gfp_mask() to power_down() before
> the suspend sequence starts. Don't call pm_restore_gfp_mask() when
> exiting suspend sequence it is already called:
>
> ```
> power_down()
> ->suspend_devices_and_enter()
> -->dpm_resume_end()
> ```
>
> Reported-by: Ionut Nechita <ionut_n2001@yahoo.com>
> Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/4573
> Tested-by: Ionut Nechita <ionut_n2001@yahoo.com>
> Fixes: 12ffc3b1513eb ("PM: Restrict swap use to later in the suspend sequence")
> Tested-by: Kenneth Crudup <kenny@panix.com>
> Acked-by: Alex Deucher <alexander.deucher@amd.com>
> Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
> ---
> v2:
>  * Move under CONFIG_SUSPEND scope (LKP robot)
>  * Add tags
> ---
>  kernel/power/hibernate.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
> index 2f66ab4538231..52c1818749724 100644
> --- a/kernel/power/hibernate.c
> +++ b/kernel/power/hibernate.c
> @@ -695,6 +695,7 @@ static void power_down(void)
>
>  #ifdef CONFIG_SUSPEND
>         if (hibernation_mode == HIBERNATION_SUSPEND) {
> +               pm_restore_gfp_mask();
>                 error = suspend_devices_and_enter(mem_sleep_current);
>                 if (error) {
>                         hibernation_mode = hibernation_ops ?
> @@ -862,7 +863,15 @@ int hibernate(void)
>                                 power_down();
>                 }
>                 in_suspend = 0;
> -               pm_restore_gfp_mask();
> +               switch (hibernation_mode) {
> +#ifdef CONFIG_SUSPEND
> +               case HIBERNATION_SUSPEND:
> +                       break;
> +#endif
> +               default:
> +                       pm_restore_gfp_mask();
> +                       break;
> +               }

You're breaking HIBERNATION_TEST_RESUME here AFAICS and power_down()
doesn't return.

>         } else {
>                 pm_pr_dbg("Hibernation image restored successfully.\n");
>         }
> --

Re: [PATCH v2 1/3] PM: hibernate: Fix hybrid-sleep

[Rafael J. Wysocki]

On Thu, Sep 25, 2025 at 7:47 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
>
> On Thu, Sep 25, 2025 at 5:59 PM Mario Limonciello (AMD)
> <superm1@kernel.org> wrote:
> >
> > Hybrid sleep will hibernate the system followed by running through
> > the suspend routine.  Since both the hibernate and the suspend routine
> > will call pm_restrict_gfp_mask(), pm_restore_gfp_mask() must be called
> > before starting the suspend sequence.
> >
> > Add an explicit call to pm_restore_gfp_mask() to power_down() before
> > the suspend sequence starts. Don't call pm_restore_gfp_mask() when
> > exiting suspend sequence it is already called:
> >
> > ```
> > power_down()
> > ->suspend_devices_and_enter()
> > -->dpm_resume_end()
> > ```
> >
> > Reported-by: Ionut Nechita <ionut_n2001@yahoo.com>
> > Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/4573
> > Tested-by: Ionut Nechita <ionut_n2001@yahoo.com>
> > Fixes: 12ffc3b1513eb ("PM: Restrict swap use to later in the suspend sequence")
> > Tested-by: Kenneth Crudup <kenny@panix.com>
> > Acked-by: Alex Deucher <alexander.deucher@amd.com>
> > Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
> > ---
> > v2:
> >  * Move under CONFIG_SUSPEND scope (LKP robot)
> >  * Add tags
> > ---
> >  kernel/power/hibernate.c | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
> > index 2f66ab4538231..52c1818749724 100644
> > --- a/kernel/power/hibernate.c
> > +++ b/kernel/power/hibernate.c
> > @@ -695,6 +695,7 @@ static void power_down(void)
> >
> >  #ifdef CONFIG_SUSPEND
> >         if (hibernation_mode == HIBERNATION_SUSPEND) {
> > +               pm_restore_gfp_mask();
> >                 error = suspend_devices_and_enter(mem_sleep_current);
> >                 if (error) {
> >                         hibernation_mode = hibernation_ops ?
> > @@ -862,7 +863,15 @@ int hibernate(void)
> >                                 power_down();
> >                 }
> >                 in_suspend = 0;
> > -               pm_restore_gfp_mask();
> > +               switch (hibernation_mode) {
> > +#ifdef CONFIG_SUSPEND
> > +               case HIBERNATION_SUSPEND:
> > +                       break;
> > +#endif
> > +               default:
> > +                       pm_restore_gfp_mask();
> > +                       break;
> > +               }
>
> You're breaking HIBERNATION_TEST_RESUME here AFAICS

Well, not really because of the hibernation_mode check.

> and power_down() doesn't return.

But this still is true.

>
> >         } else {
> >                 pm_pr_dbg("Hibernation image restored successfully.\n");
> >         }
> > --

Re: [PATCH v2 1/3] PM: hibernate: Fix hybrid-sleep

[Rafael J. Wysocki]

On Thu, Sep 25, 2025 at 7:51 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
>
> On Thu, Sep 25, 2025 at 7:47 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
> >
> > On Thu, Sep 25, 2025 at 5:59 PM Mario Limonciello (AMD)
> > <superm1@kernel.org> wrote:
> > >
> > > Hybrid sleep will hibernate the system followed by running through
> > > the suspend routine.  Since both the hibernate and the suspend routine
> > > will call pm_restrict_gfp_mask(), pm_restore_gfp_mask() must be called
> > > before starting the suspend sequence.
> > >
> > > Add an explicit call to pm_restore_gfp_mask() to power_down() before
> > > the suspend sequence starts. Don't call pm_restore_gfp_mask() when
> > > exiting suspend sequence it is already called:
> > >
> > > ```
> > > power_down()
> > > ->suspend_devices_and_enter()
> > > -->dpm_resume_end()
> > > ```
> > >
> > > Reported-by: Ionut Nechita <ionut_n2001@yahoo.com>
> > > Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/4573
> > > Tested-by: Ionut Nechita <ionut_n2001@yahoo.com>
> > > Fixes: 12ffc3b1513eb ("PM: Restrict swap use to later in the suspend sequence")
> > > Tested-by: Kenneth Crudup <kenny@panix.com>
> > > Acked-by: Alex Deucher <alexander.deucher@amd.com>
> > > Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
> > > ---
> > > v2:
> > >  * Move under CONFIG_SUSPEND scope (LKP robot)
> > >  * Add tags
> > > ---
> > >  kernel/power/hibernate.c | 11 ++++++++++-
> > >  1 file changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
> > > index 2f66ab4538231..52c1818749724 100644
> > > --- a/kernel/power/hibernate.c
> > > +++ b/kernel/power/hibernate.c
> > > @@ -695,6 +695,7 @@ static void power_down(void)
> > >
> > >  #ifdef CONFIG_SUSPEND
> > >         if (hibernation_mode == HIBERNATION_SUSPEND) {
> > > +               pm_restore_gfp_mask();
> > >                 error = suspend_devices_and_enter(mem_sleep_current);
> > >                 if (error) {
> > >                         hibernation_mode = hibernation_ops ?
> > > @@ -862,7 +863,15 @@ int hibernate(void)
> > >                                 power_down();
> > >                 }
> > >                 in_suspend = 0;
> > > -               pm_restore_gfp_mask();
> > > +               switch (hibernation_mode) {
> > > +#ifdef CONFIG_SUSPEND
> > > +               case HIBERNATION_SUSPEND:
> > > +                       break;
> > > +#endif
> > > +               default:
> > > +                       pm_restore_gfp_mask();
> > > +                       break;
> > > +               }
> >
> > You're breaking HIBERNATION_TEST_RESUME here AFAICS
>
> Well, not really because of the hibernation_mode check.
>
> > and power_down() doesn't return.
>
> But this still is true.

Well, except when it does HIBERNATION_SUSPEND.

But can you just make power_down() call pm_restrict_gfp_mask() before
returning and leave the code in hibernate() as is?

Re: [PATCH v2 1/3] PM: hibernate: Fix hybrid-sleep

[Mario Limonciello (AMD) (kernel.org)]

On 9/25/2025 12:55 PM, Rafael J. Wysocki wrote:
> On Thu, Sep 25, 2025 at 7:51 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
>>
>> On Thu, Sep 25, 2025 at 7:47 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
>>>
>>> On Thu, Sep 25, 2025 at 5:59 PM Mario Limonciello (AMD)
>>> <superm1@kernel.org> wrote:
>>>>
>>>> Hybrid sleep will hibernate the system followed by running through
>>>> the suspend routine.  Since both the hibernate and the suspend routine
>>>> will call pm_restrict_gfp_mask(), pm_restore_gfp_mask() must be called
>>>> before starting the suspend sequence.
>>>>
>>>> Add an explicit call to pm_restore_gfp_mask() to power_down() before
>>>> the suspend sequence starts. Don't call pm_restore_gfp_mask() when
>>>> exiting suspend sequence it is already called:
>>>>
>>>> ```
>>>> power_down()
>>>> ->suspend_devices_and_enter()
>>>> -->dpm_resume_end()
>>>> ```
>>>>
>>>> Reported-by: Ionut Nechita <ionut_n2001@yahoo.com>
>>>> Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/4573
>>>> Tested-by: Ionut Nechita <ionut_n2001@yahoo.com>
>>>> Fixes: 12ffc3b1513eb ("PM: Restrict swap use to later in the suspend sequence")
>>>> Tested-by: Kenneth Crudup <kenny@panix.com>
>>>> Acked-by: Alex Deucher <alexander.deucher@amd.com>
>>>> Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
>>>> ---
>>>> v2:
>>>>   * Move under CONFIG_SUSPEND scope (LKP robot)
>>>>   * Add tags
>>>> ---
>>>>   kernel/power/hibernate.c | 11 ++++++++++-
>>>>   1 file changed, 10 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
>>>> index 2f66ab4538231..52c1818749724 100644
>>>> --- a/kernel/power/hibernate.c
>>>> +++ b/kernel/power/hibernate.c
>>>> @@ -695,6 +695,7 @@ static void power_down(void)
>>>>
>>>>   #ifdef CONFIG_SUSPEND
>>>>          if (hibernation_mode == HIBERNATION_SUSPEND) {
>>>> +               pm_restore_gfp_mask();
>>>>                  error = suspend_devices_and_enter(mem_sleep_current);
>>>>                  if (error) {
>>>>                          hibernation_mode = hibernation_ops ?
>>>> @@ -862,7 +863,15 @@ int hibernate(void)
>>>>                                  power_down();
>>>>                  }
>>>>                  in_suspend = 0;
>>>> -               pm_restore_gfp_mask();
>>>> +               switch (hibernation_mode) {
>>>> +#ifdef CONFIG_SUSPEND
>>>> +               case HIBERNATION_SUSPEND:
>>>> +                       break;
>>>> +#endif
>>>> +               default:
>>>> +                       pm_restore_gfp_mask();
>>>> +                       break;
>>>> +               }
>>>
>>> You're breaking HIBERNATION_TEST_RESUME here AFAICS
>>
>> Well, not really because of the hibernation_mode check.
>>
>>> and power_down() doesn't return.
>>
>> But this still is true.
> 
> Well, except when it does HIBERNATION_SUSPEND.
> 
> But can you just make power_down() call pm_restrict_gfp_mask() before
> returning and leave the code in hibernate() as is?

Ah good suggestion.  I believe so.  I'll test it and get out updated 
patches if that works.