[PATCH] PM: Use WQ_MEM_RECLAIM flag for PM workqueue

[PATCH] PM: Use WQ_MEM_RECLAIM flag for PM workqueue

[ravisankarreddylingareddy]

Deadlock is seen under below high memory usage scenario.

kswapd waiting to reclaim memory is blocked on i/o kworker and pm_wq
context which needs to wakeup i/o kworker is waiting for kthread
creation which is again blocked in memory reclaim path.

Use WQ_MEM_RECLAIM flag for PM workqueue to avoid this deadlock.

Signed-off-by: Ravisankar Reddy (Sony) <ravisankarreddylingareddy@gmail.com>
---
 kernel/power/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/power/main.c b/kernel/power/main.c
index 40f86ec4ab30..ff5f3349e76f 100644
--- a/kernel/power/main.c
+++ b/kernel/power/main.c
@@ -933,7 +933,7 @@ EXPORT_SYMBOL_GPL(pm_wq);
 
 static int __init pm_start_workqueue(void)
 {
-	pm_wq = alloc_workqueue("pm", WQ_FREEZABLE, 0);
+	pm_wq = alloc_workqueue("pm", WQ_FREEZABLE | WQ_MEM_RECLAIM, 0);
 
 	return pm_wq ? 0 : -ENOMEM;
 }
-- 
2.17.1

Re: [PATCH] PM: Use WQ_MEM_RECLAIM flag for PM workqueue

[Rafael J. Wysocki]

On Tue, Oct 27, 2020 at 11:15 AM <ravisankarreddylingareddy@gmail.com> wrote:
>
> Deadlock is seen under below high memory usage scenario.

I'm not sure what the "under below high" means.

> kswapd waiting to reclaim memory is blocked on i/o kworker and pm_wq
> context which needs to wakeup i/o kworker is waiting for kthread
> creation which is again blocked in memory reclaim path.

Can you explain in more detail how exactly you end up in that situation?

> Use WQ_MEM_RECLAIM flag for PM workqueue to avoid this deadlock.

This works, but this flag doesn't seem to be appropriate here, because
pm_wq doesn't actually participate in memory reclaim.

> Signed-off-by: Ravisankar Reddy (Sony) <ravisankarreddylingareddy@gmail.com>
> ---
>  kernel/power/main.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/power/main.c b/kernel/power/main.c
> index 40f86ec4ab30..ff5f3349e76f 100644
> --- a/kernel/power/main.c
> +++ b/kernel/power/main.c
> @@ -933,7 +933,7 @@ EXPORT_SYMBOL_GPL(pm_wq);
>
>  static int __init pm_start_workqueue(void)
>  {
> -       pm_wq = alloc_workqueue("pm", WQ_FREEZABLE, 0);
> +       pm_wq = alloc_workqueue("pm", WQ_FREEZABLE | WQ_MEM_RECLAIM, 0);
>
>         return pm_wq ? 0 : -ENOMEM;
>  }
> --
> 2.17.1
>

Re: [PATCH] PM: Use WQ_MEM_RECLAIM flag for PM workqueue

[Rafael J. Wysocki]

On 10/24/2020 8:49 AM, Ravisankarreddy lingareddy wrote:
> Deadlock is seen under below high memory usage scenario.
>
> kswapd waiting to reclaim memory is blocked on i/o kworker and pm_wq
> context which needs to wakeup i/o kworker is waiting for kthread
> creation which is again blocked in memory reclaim path.
>
> Use WQ_MEM_RECLAIM flag for PM workqueue to avoid this deadlock.
>
> Signed-off-by: Ravisankarreddy lingareddy <ravisankarreddylingareddy@gmail.com>
> ---
>   kernel/power/main.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/power/main.c b/kernel/power/main.c
> index 40f86ec4ab30..ff5f3349e76f 100644
> --- a/kernel/power/main.c
> +++ b/kernel/power/main.c
> @@ -933,7 +933,7 @@ EXPORT_SYMBOL_GPL(pm_wq);
>   
>   static int __init pm_start_workqueue(void)
>   {
> -	pm_wq = alloc_workqueue("pm", WQ_FREEZABLE, 0);
> +	pm_wq = alloc_workqueue("pm", WQ_FREEZABLE | WQ_MEM_RECLAIM, 0);
>   
>   	return pm_wq ? 0 : -ENOMEM;
>   }

Please resend this with a CC to linux-pm@vger.kernel.org.

Also it would be good to know how to reproduce this issue.

Re: [PATCH] PM: Use WQ_MEM_RECLAIM flag for PM workqueue

[ravisankarreddy lingareddy]

> Please resend this with a CC to linux-pm@vger.kernel.org.

Thank you, I resend with a CC to linux-pm@vger.kernel.org.

> Also it would be good to know how to reproduce this issue.

Commit message says under high memory scenarios, try a memory pressure
test and do repeatedly  until the battery is under 20%.

> @Pavel,

> Does this need some kind of auditing to make sure we don't allocate memory from the code running in the workqueue, and thus further deadlocks?

I used WQ_MEM_RECLAIM in pm_wq workqueue to fix bug and no further
deadlocks are seen and tested on 4.9 and 4.19 kernel.

Best Regards,
Ravi


On Mon, Oct 26, 2020 at 4:23 AM Rafael J. Wysocki
<rafael.j.wysocki@intel.com> wrote:
>
> On 10/24/2020 8:49 AM, Ravisankarreddy lingareddy wrote:
> > Deadlock is seen under below high memory usage scenario.
> >
> > kswapd waiting to reclaim memory is blocked on i/o kworker and pm_wq
> > context which needs to wakeup i/o kworker is waiting for kthread
> > creation which is again blocked in memory reclaim path.
> >
> > Use WQ_MEM_RECLAIM flag for PM workqueue to avoid this deadlock.
> >
> > Signed-off-by: Ravisankarreddy lingareddy <ravisankarreddylingareddy@gmail.com>
> > ---
> >   kernel/power/main.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kernel/power/main.c b/kernel/power/main.c
> > index 40f86ec4ab30..ff5f3349e76f 100644
> > --- a/kernel/power/main.c
> > +++ b/kernel/power/main.c
> > @@ -933,7 +933,7 @@ EXPORT_SYMBOL_GPL(pm_wq);
> >
> >   static int __init pm_start_workqueue(void)
> >   {
> > -     pm_wq = alloc_workqueue("pm", WQ_FREEZABLE, 0);
> > +     pm_wq = alloc_workqueue("pm", WQ_FREEZABLE | WQ_MEM_RECLAIM, 0);
> >
> >       return pm_wq ? 0 : -ENOMEM;
> >   }
>
> Please resend this with a CC to linux-pm@vger.kernel.org.
>
> Also it would be good to know how to reproduce this issue.
>
>