* [PATCH] PM: Restrict swap use to later in the suspend sequence
@ 2025-06-13 21:43 Mario Limonciello
2025-06-23 1:29 ` Mario Limonciello
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Mario Limonciello @ 2025-06-13 21:43 UTC (permalink / raw)
To: mario.limonciello, rafael, len.brown, pavel, gregkh, dakr,
Andrew Morton, bhe
Cc: christian.koenig, linux-pm, kexec
From: Mario Limonciello <mario.limonciello@amd.com>
Currently swap is restricted before drivers have had a chance to do their
prepare() PM callbacks. Restricting swap this early means that if a driver
needs to evict some content from memory into sawp in it's prepare callback
it won't be able to.
On AMD dGPUs this can lead to failed suspends under memory pressure
situations as all VRAM must be evicted to system memory or swap.
Move the swap restriction to right after all devices have had a chance to
do the prepare() callback. If there is any problem with the sequence,
restore swap in the appropriate dpm resume callbacks or error handling
paths.
Closes: https://github.com/ROCm/ROCK-Kernel-Driver/issues/174
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/2362
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
drivers/base/power/main.c | 5 ++++-
include/linux/suspend.h | 5 +++++
kernel/kexec_core.c | 1 +
kernel/power/hibernate.c | 3 ---
kernel/power/power.h | 5 -----
kernel/power/suspend.c | 3 +--
6 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
index 0215b20c5e2c8..4970a804afb6d 100644
--- a/drivers/base/power/main.c
+++ b/drivers/base/power/main.c
@@ -1241,6 +1241,7 @@ void dpm_complete(pm_message_t state)
*/
void dpm_resume_end(pm_message_t state)
{
+ pm_restore_gfp_mask();
dpm_resume(state);
dpm_complete(state);
}
@@ -2183,8 +2184,10 @@ int dpm_suspend_start(pm_message_t state)
error = dpm_prepare(state);
if (error)
dpm_save_failed_step(SUSPEND_PREPARE);
- else
+ else {
+ pm_restrict_gfp_mask();
error = dpm_suspend(state);
+ }
dpm_show_time(starttime, state, error, "start");
return error;
diff --git a/include/linux/suspend.h b/include/linux/suspend.h
index b1c76c8f2c822..6a3f920988720 100644
--- a/include/linux/suspend.h
+++ b/include/linux/suspend.h
@@ -446,6 +446,8 @@ extern int unregister_pm_notifier(struct notifier_block *nb);
extern void ksys_sync_helper(void);
extern void pm_report_hw_sleep_time(u64 t);
extern void pm_report_max_hw_sleep(u64 t);
+void pm_restrict_gfp_mask(void);
+void pm_restore_gfp_mask(void);
#define pm_notifier(fn, pri) { \
static struct notifier_block fn##_nb = \
@@ -492,6 +494,9 @@ static inline int unregister_pm_notifier(struct notifier_block *nb)
static inline void pm_report_hw_sleep_time(u64 t) {};
static inline void pm_report_max_hw_sleep(u64 t) {};
+static inline void pm_restrict_gfp_mask(void) {}
+static inline void pm_restore_gfp_mask(void) {}
+
static inline void ksys_sync_helper(void) {}
#define pm_notifier(fn, pri) do { (void)(fn); } while (0)
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index 9c59fa480b0b6..3a9a9f240dbc9 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -1136,6 +1136,7 @@ int kernel_kexec(void)
Resume_devices:
dpm_resume_end(PMSG_RESTORE);
Resume_console:
+ pm_restore_gfp_mask();
console_resume_all();
thaw_processes();
Restore_console:
diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
index 519fb09de5e0c..9216e3b91d3b3 100644
--- a/kernel/power/hibernate.c
+++ b/kernel/power/hibernate.c
@@ -423,7 +423,6 @@ int hibernation_snapshot(int platform_mode)
}
console_suspend_all();
- pm_restrict_gfp_mask();
error = dpm_suspend(PMSG_FREEZE);
@@ -559,7 +558,6 @@ int hibernation_restore(int platform_mode)
pm_prepare_console();
console_suspend_all();
- pm_restrict_gfp_mask();
error = dpm_suspend_start(PMSG_QUIESCE);
if (!error) {
error = resume_target_kernel(platform_mode);
@@ -571,7 +569,6 @@ int hibernation_restore(int platform_mode)
BUG_ON(!error);
}
dpm_resume_end(PMSG_RECOVER);
- pm_restore_gfp_mask();
console_resume_all();
pm_restore_console();
return error;
diff --git a/kernel/power/power.h b/kernel/power/power.h
index cb1d715620020..7ccd709af93f5 100644
--- a/kernel/power/power.h
+++ b/kernel/power/power.h
@@ -239,11 +239,6 @@ static inline void suspend_test_finish(const char *label) {}
/* kernel/power/main.c */
extern int pm_notifier_call_chain_robust(unsigned long val_up, unsigned long val_down);
extern int pm_notifier_call_chain(unsigned long val);
-void pm_restrict_gfp_mask(void);
-void pm_restore_gfp_mask(void);
-#else
-static inline void pm_restrict_gfp_mask(void) {}
-static inline void pm_restore_gfp_mask(void) {}
#endif
#ifdef CONFIG_HIGHMEM
diff --git a/kernel/power/suspend.c b/kernel/power/suspend.c
index 76b141b9aac01..bb608b68fb301 100644
--- a/kernel/power/suspend.c
+++ b/kernel/power/suspend.c
@@ -540,6 +540,7 @@ int suspend_devices_and_enter(suspend_state_t state)
return error;
Recover_platform:
+ pm_restore_gfp_mask();
platform_recover(state);
goto Resume_devices;
}
@@ -606,9 +607,7 @@ static int enter_state(suspend_state_t state)
trace_suspend_resume(TPS("suspend_enter"), state, false);
pm_pr_dbg("Suspending system (%s)\n", mem_sleep_labels[state]);
- pm_restrict_gfp_mask();
error = suspend_devices_and_enter(state);
- pm_restore_gfp_mask();
Finish:
events_check_enabled = false;
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] PM: Restrict swap use to later in the suspend sequence
2025-06-13 21:43 [PATCH] PM: Restrict swap use to later in the suspend sequence Mario Limonciello
@ 2025-06-23 1:29 ` Mario Limonciello
2025-06-23 3:23 ` Baoquan He
2025-06-23 6:25 ` Lucian Langa
2 siblings, 0 replies; 6+ messages in thread
From: Mario Limonciello @ 2025-06-23 1:29 UTC (permalink / raw)
To: mario.limonciello, rafael, len.brown, pavel, gregkh, dakr,
Andrew Morton, bhe
Cc: christian.koenig, linux-pm, kexec, Nat Wittstock
On 6/13/25 4:43 PM, Mario Limonciello wrote:
> From: Mario Limonciello <mario.limonciello@amd.com>
>
> Currently swap is restricted before drivers have had a chance to do their
> prepare() PM callbacks. Restricting swap this early means that if a driver
> needs to evict some content from memory into sawp in it's prepare callback
> it won't be able to.
>
> On AMD dGPUs this can lead to failed suspends under memory pressure
> situations as all VRAM must be evicted to system memory or swap.
>
> Move the swap restriction to right after all devices have had a chance to
> do the prepare() callback. If there is any problem with the sequence,
> restore swap in the appropriate dpm resume callbacks or error handling
> paths.
>
> Closes: https://github.com/ROCm/ROCK-Kernel-Driver/issues/174
> Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/2362
> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Besides my testing for this, on the bugs two people have tested it helps
this issue. One of them had a hard time replying to the lore post, so
I'll proxy the tag for them.
Tested-by: Nat Wittstock <nat@fardog.io>
> ---
> drivers/base/power/main.c | 5 ++++-
> include/linux/suspend.h | 5 +++++
> kernel/kexec_core.c | 1 +
> kernel/power/hibernate.c | 3 ---
> kernel/power/power.h | 5 -----
> kernel/power/suspend.c | 3 +--
> 6 files changed, 11 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
> index 0215b20c5e2c8..4970a804afb6d 100644
> --- a/drivers/base/power/main.c
> +++ b/drivers/base/power/main.c
> @@ -1241,6 +1241,7 @@ void dpm_complete(pm_message_t state)
> */
> void dpm_resume_end(pm_message_t state)
> {
> + pm_restore_gfp_mask();
> dpm_resume(state);
> dpm_complete(state);
> }
> @@ -2183,8 +2184,10 @@ int dpm_suspend_start(pm_message_t state)
> error = dpm_prepare(state);
> if (error)
> dpm_save_failed_step(SUSPEND_PREPARE);
> - else
> + else {
> + pm_restrict_gfp_mask();
> error = dpm_suspend(state);
> + }
>
> dpm_show_time(starttime, state, error, "start");
> return error;
> diff --git a/include/linux/suspend.h b/include/linux/suspend.h
> index b1c76c8f2c822..6a3f920988720 100644
> --- a/include/linux/suspend.h
> +++ b/include/linux/suspend.h
> @@ -446,6 +446,8 @@ extern int unregister_pm_notifier(struct notifier_block *nb);
> extern void ksys_sync_helper(void);
> extern void pm_report_hw_sleep_time(u64 t);
> extern void pm_report_max_hw_sleep(u64 t);
> +void pm_restrict_gfp_mask(void);
> +void pm_restore_gfp_mask(void);
>
> #define pm_notifier(fn, pri) { \
> static struct notifier_block fn##_nb = \
> @@ -492,6 +494,9 @@ static inline int unregister_pm_notifier(struct notifier_block *nb)
> static inline void pm_report_hw_sleep_time(u64 t) {};
> static inline void pm_report_max_hw_sleep(u64 t) {};
>
> +static inline void pm_restrict_gfp_mask(void) {}
> +static inline void pm_restore_gfp_mask(void) {}
> +
> static inline void ksys_sync_helper(void) {}
>
> #define pm_notifier(fn, pri) do { (void)(fn); } while (0)
> diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
> index 9c59fa480b0b6..3a9a9f240dbc9 100644
> --- a/kernel/kexec_core.c
> +++ b/kernel/kexec_core.c
> @@ -1136,6 +1136,7 @@ int kernel_kexec(void)
> Resume_devices:
> dpm_resume_end(PMSG_RESTORE);
> Resume_console:
> + pm_restore_gfp_mask();
> console_resume_all();
> thaw_processes();
> Restore_console:
> diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
> index 519fb09de5e0c..9216e3b91d3b3 100644
> --- a/kernel/power/hibernate.c
> +++ b/kernel/power/hibernate.c
> @@ -423,7 +423,6 @@ int hibernation_snapshot(int platform_mode)
> }
>
> console_suspend_all();
> - pm_restrict_gfp_mask();
>
> error = dpm_suspend(PMSG_FREEZE);
>
> @@ -559,7 +558,6 @@ int hibernation_restore(int platform_mode)
>
> pm_prepare_console();
> console_suspend_all();
> - pm_restrict_gfp_mask();
> error = dpm_suspend_start(PMSG_QUIESCE);
> if (!error) {
> error = resume_target_kernel(platform_mode);
> @@ -571,7 +569,6 @@ int hibernation_restore(int platform_mode)
> BUG_ON(!error);
> }
> dpm_resume_end(PMSG_RECOVER);
> - pm_restore_gfp_mask();
> console_resume_all();
> pm_restore_console();
> return error;
> diff --git a/kernel/power/power.h b/kernel/power/power.h
> index cb1d715620020..7ccd709af93f5 100644
> --- a/kernel/power/power.h
> +++ b/kernel/power/power.h
> @@ -239,11 +239,6 @@ static inline void suspend_test_finish(const char *label) {}
> /* kernel/power/main.c */
> extern int pm_notifier_call_chain_robust(unsigned long val_up, unsigned long val_down);
> extern int pm_notifier_call_chain(unsigned long val);
> -void pm_restrict_gfp_mask(void);
> -void pm_restore_gfp_mask(void);
> -#else
> -static inline void pm_restrict_gfp_mask(void) {}
> -static inline void pm_restore_gfp_mask(void) {}
> #endif
>
> #ifdef CONFIG_HIGHMEM
> diff --git a/kernel/power/suspend.c b/kernel/power/suspend.c
> index 76b141b9aac01..bb608b68fb301 100644
> --- a/kernel/power/suspend.c
> +++ b/kernel/power/suspend.c
> @@ -540,6 +540,7 @@ int suspend_devices_and_enter(suspend_state_t state)
> return error;
>
> Recover_platform:
> + pm_restore_gfp_mask();
> platform_recover(state);
> goto Resume_devices;
> }
> @@ -606,9 +607,7 @@ static int enter_state(suspend_state_t state)
>
> trace_suspend_resume(TPS("suspend_enter"), state, false);
> pm_pr_dbg("Suspending system (%s)\n", mem_sleep_labels[state]);
> - pm_restrict_gfp_mask();
> error = suspend_devices_and_enter(state);
> - pm_restore_gfp_mask();
>
> Finish:
> events_check_enabled = false;
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] PM: Restrict swap use to later in the suspend sequence
2025-06-13 21:43 [PATCH] PM: Restrict swap use to later in the suspend sequence Mario Limonciello
2025-06-23 1:29 ` Mario Limonciello
@ 2025-06-23 3:23 ` Baoquan He
2025-06-23 4:58 ` David Woodhouse
2025-06-23 6:25 ` Lucian Langa
2 siblings, 1 reply; 6+ messages in thread
From: Baoquan He @ 2025-06-23 3:23 UTC (permalink / raw)
To: Mario Limonciello, dwmw2
Cc: mario.limonciello, rafael, len.brown, pavel, gregkh, dakr,
Andrew Morton, christian.koenig, linux-pm, kexec
On 06/13/25 at 04:43pm, Mario Limonciello wrote:
> From: Mario Limonciello <mario.limonciello@amd.com>
>
> Currently swap is restricted before drivers have had a chance to do their
> prepare() PM callbacks. Restricting swap this early means that if a driver
> needs to evict some content from memory into sawp in it's prepare callback
> it won't be able to.
>
> On AMD dGPUs this can lead to failed suspends under memory pressure
> situations as all VRAM must be evicted to system memory or swap.
>
> Move the swap restriction to right after all devices have had a chance to
> do the prepare() callback. If there is any problem with the sequence,
> restore swap in the appropriate dpm resume callbacks or error handling
> paths.
>
> Closes: https://github.com/ROCm/ROCK-Kernel-Driver/issues/174
> Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/2362
> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
> ---
> drivers/base/power/main.c | 5 ++++-
> include/linux/suspend.h | 5 +++++
> kernel/kexec_core.c | 1 +
> kernel/power/hibernate.c | 3 ---
> kernel/power/power.h | 5 -----
> kernel/power/suspend.c | 3 +--
> 6 files changed, 11 insertions(+), 11 deletions(-)
>
......snip...
> diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
> index 9c59fa480b0b6..3a9a9f240dbc9 100644
> --- a/kernel/kexec_core.c
> +++ b/kernel/kexec_core.c
> @@ -1136,6 +1136,7 @@ int kernel_kexec(void)
> Resume_devices:
> dpm_resume_end(PMSG_RESTORE);
> Resume_console:
> + pm_restore_gfp_mask();
> console_resume_all();
> thaw_processes();
> Restore_console:
This change is inside KEXEC_JUMP scope, let David know this because he
ever tested and made change for KEXEC_JUMP.
......
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] PM: Restrict swap use to later in the suspend sequence
2025-06-23 3:23 ` Baoquan He
@ 2025-06-23 4:58 ` David Woodhouse
0 siblings, 0 replies; 6+ messages in thread
From: David Woodhouse @ 2025-06-23 4:58 UTC (permalink / raw)
To: Baoquan He, Mario Limonciello
Cc: mario.limonciello, rafael, len.brown, pavel, gregkh, dakr,
Andrew Morton, christian.koenig, linux-pm, kexec
On 23 June 2025 04:23:43 BST, Baoquan He <bhe@redhat.com> wrote:
>This change is inside KEXEC_JUMP scope, let David know this because he
>ever tested and made change for KEXEC_JUMP.
>......
Thanks. I added a selftest in https://lore.kernel.org/all/20250326142404.256980-5-dwmw2@infradead.org
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] PM: Restrict swap use to later in the suspend sequence
2025-06-13 21:43 [PATCH] PM: Restrict swap use to later in the suspend sequence Mario Limonciello
2025-06-23 1:29 ` Mario Limonciello
2025-06-23 3:23 ` Baoquan He
@ 2025-06-23 6:25 ` Lucian Langa
2025-06-26 18:41 ` Rafael J. Wysocki
2 siblings, 1 reply; 6+ messages in thread
From: Lucian Langa @ 2025-06-23 6:25 UTC (permalink / raw)
To: Mario Limonciello, mario.limonciello, rafael, len.brown, pavel,
gregkh, dakr, Andrew Morton, bhe
Cc: christian.koenig, linux-pm, kexec
On 13.06.2025 16:43, Mario Limonciello wrote:
> From: Mario Limonciello <mario.limonciello@amd.com>
>
> Currently swap is restricted before drivers have had a chance to do their
> prepare() PM callbacks. Restricting swap this early means that if a driver
> needs to evict some content from memory into sawp in it's prepare callback
> it won't be able to.
>
> On AMD dGPUs this can lead to failed suspends under memory pressure
> situations as all VRAM must be evicted to system memory or swap.
>
> Move the swap restriction to right after all devices have had a chance to
> do the prepare() callback. If there is any problem with the sequence,
> restore swap in the appropriate dpm resume callbacks or error handling
> paths.
>
> Closes: https://github.com/ROCm/ROCK-Kernel-Driver/issues/174
> Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/2362
> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Tested this against 6.15.3 and 6.16-rc3.
Adding my tag.
Tested-by: Lucian Langa <lucilanga@7pot.org>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] PM: Restrict swap use to later in the suspend sequence
2025-06-23 6:25 ` Lucian Langa
@ 2025-06-26 18:41 ` Rafael J. Wysocki
0 siblings, 0 replies; 6+ messages in thread
From: Rafael J. Wysocki @ 2025-06-26 18:41 UTC (permalink / raw)
To: Lucian Langa
Cc: Mario Limonciello, mario.limonciello, rafael, len.brown, pavel,
gregkh, dakr, Andrew Morton, bhe, christian.koenig, linux-pm,
kexec
On Mon, Jun 23, 2025 at 8:25 AM Lucian Langa <lucilanga@7pot.org> wrote:
>
> On 13.06.2025 16:43, Mario Limonciello wrote:
> > From: Mario Limonciello <mario.limonciello@amd.com>
> >
> > Currently swap is restricted before drivers have had a chance to do their
> > prepare() PM callbacks. Restricting swap this early means that if a driver
> > needs to evict some content from memory into sawp in it's prepare callback
> > it won't be able to.
> >
> > On AMD dGPUs this can lead to failed suspends under memory pressure
> > situations as all VRAM must be evicted to system memory or swap.
> >
> > Move the swap restriction to right after all devices have had a chance to
> > do the prepare() callback. If there is any problem with the sequence,
> > restore swap in the appropriate dpm resume callbacks or error handling
> > paths.
> >
> > Closes: https://github.com/ROCm/ROCK-Kernel-Driver/issues/174
> > Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/2362
> > Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
>
> Tested this against 6.15.3 and 6.16-rc3.
> Adding my tag.
>
> Tested-by: Lucian Langa <lucilanga@7pot.org>
Applied as 6.16-rc material, thanks!
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-06-26 18:42 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-13 21:43 [PATCH] PM: Restrict swap use to later in the suspend sequence Mario Limonciello
2025-06-23 1:29 ` Mario Limonciello
2025-06-23 3:23 ` Baoquan He
2025-06-23 4:58 ` David Woodhouse
2025-06-23 6:25 ` Lucian Langa
2025-06-26 18:41 ` Rafael J. Wysocki
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox