[PATCH] kernel/sysrq: reset watchdog on all CPUs while processing sysrq-w

[PATCH] kernel/sysrq: reset watchdog on all CPUs while processing sysrq-w

[Andrey Ryabinin]

Lengthy output of sysrq-w may take a lot of time on slow serial console.
Currently we reset NMI-watchdog on the current CPU to avoid softlockup.
Sometimes this doesn't work since watchdog might trigger on the other
CPU which is waiting for an IPI to proceed.

Reset watchdog on all CPUs to prevent spurious softlockup messages.

Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7f2cae4..c135eee 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5134,7 +5134,7 @@ void show_state_filter(unsigned long state_filter)
 		 * reset the NMI-timeout, listing all files on a slow
 		 * console might take a lot of time:
 		 */
-		touch_nmi_watchdog();
+		touch_all_softlockup_watchdogs();
 		if (!state_filter || (p->state & state_filter))
 			sched_show_task(p);
 	}
-- 
2.7.3

Re: [PATCH] kernel/sysrq: reset watchdog on all CPUs while processing sysrq-w

[Ingo Molnar]

* Andrey Ryabinin <aryabinin@virtuozzo.com> wrote:

> Lengthy output of sysrq-w may take a lot of time on slow serial console.
> Currently we reset NMI-watchdog on the current CPU to avoid softlockup.
> Sometimes this doesn't work since watchdog might trigger on the other
> CPU which is waiting for an IPI to proceed.
> 
> Reset watchdog on all CPUs to prevent spurious softlockup messages.
> 
> Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
> ---
>  kernel/sched/core.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 7f2cae4..c135eee 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -5134,7 +5134,7 @@ void show_state_filter(unsigned long state_filter)
>  		 * reset the NMI-timeout, listing all files on a slow
>  		 * console might take a lot of time:
>  		 */
> -		touch_nmi_watchdog();
> +		touch_all_softlockup_watchdogs();

So don't we want both?

Thanks,

	Ingo

Re: [PATCH] kernel/sysrq: reset watchdog on all CPUs while processing sysrq-w

[Andrey Ryabinin]

On 06/08/2016 04:13 PM, Ingo Molnar wrote:
> 
> * Andrey Ryabinin <aryabinin@virtuozzo.com> wrote:
> 
>> Lengthy output of sysrq-w may take a lot of time on slow serial console.
>> Currently we reset NMI-watchdog on the current CPU to avoid softlockup.
>> Sometimes this doesn't work since watchdog might trigger on the other
>> CPU which is waiting for an IPI to proceed.
>>
>> Reset watchdog on all CPUs to prevent spurious softlockup messages.
>>
>> Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
>> ---
>>  kernel/sched/core.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index 7f2cae4..c135eee 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -5134,7 +5134,7 @@ void show_state_filter(unsigned long state_filter)
>>  		 * reset the NMI-timeout, listing all files on a slow
>>  		 * console might take a lot of time:
>>  		 */
>> -		touch_nmi_watchdog();
>> +		touch_all_softlockup_watchdogs();
> 
> So don't we want both?
> 

Indeed, we need both.
I was under the false impression that touch_all_softlockup_watchdogs() will reset NMI timeout too

> Thanks,
> 
> 	Ingo
> 

[PATCH v2] kernel/sysrq: reset watchdog on all CPUs while processing sysrq-w

[Andrey Ryabinin]

Lengthy output of sysrq-w may take a lot of time on slow serial console.
Currently we reset NMI-watchdog on the current CPU to avoid spurious
lockup messages. Sometimes this doesn't work since softlockup watchdog
might trigger on another CPU which is waiting for an IPI to proceed.
We reset softlockup watchdogs on all CPUs, but we do this only after
listing all tasks, and this may be too late on a busy system.

So, reset watchdogs CPUs earlier, in for_each_process_thread() loop.

Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
---

Changes since v1:
 - keep touch_nmi_watchdog()
 - touch_all_softlockup_watchdogs() after the loop becomes redundant, so remove it.
 - added comment.

 kernel/sched/core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7f2cae4..32fe0a7 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5133,14 +5133,16 @@ void show_state_filter(unsigned long state_filter)
 		/*
 		 * reset the NMI-timeout, listing all files on a slow
 		 * console might take a lot of time:
+		 * Also, reset softlockup watchdogs on all CPUs, because
+		 * another CPU might be blocked waiting for us to process
+		 * an IPI.
 		 */
 		touch_nmi_watchdog();
+		touch_all_softlockup_watchdogs();
 		if (!state_filter || (p->state & state_filter))
 			sched_show_task(p);
 	}
 
-	touch_all_softlockup_watchdogs();
-
 #ifdef CONFIG_SCHED_DEBUG
 	if (!state_filter)
 		sysrq_sched_debug_show();
-- 
2.7.3

[tip:sched/urgent] kernel/sysrq, watchdog, sched/core: Reset watchdog on all CPUs while processing sysrq-w

[tip-bot for Andrey Ryabinin]

Commit-ID:  57675cb976eff977aefb428e68e4e0236d48a9ff
Gitweb:     http://git.kernel.org/tip/57675cb976eff977aefb428e68e4e0236d48a9ff
Author:     Andrey Ryabinin <aryabinin@virtuozzo.com>
AuthorDate: Thu, 9 Jun 2016 15:20:05 +0300
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 14 Jun 2016 12:48:38 +0200

kernel/sysrq, watchdog, sched/core: Reset watchdog on all CPUs while processing sysrq-w

Lengthy output of sysrq-w may take a lot of time on slow serial console.

Currently we reset NMI-watchdog on the current CPU to avoid spurious
lockup messages. Sometimes this doesn't work since softlockup watchdog
might trigger on another CPU which is waiting for an IPI to proceed.
We reset softlockup watchdogs on all CPUs, but we do this only after
listing all tasks, and this may be too late on a busy system.

So, reset watchdogs CPUs earlier, in for_each_process_thread() loop.

Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: <stable@vger.kernel.org>
Link: http://lkml.kernel.org/r/1465474805-14641-1-git-send-email-aryabinin@virtuozzo.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/sched/core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 13d0896..4135ac8 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5147,14 +5147,16 @@ void show_state_filter(unsigned long state_filter)
 		/*
 		 * reset the NMI-timeout, listing all files on a slow
 		 * console might take a lot of time:
+		 * Also, reset softlockup watchdogs on all CPUs, because
+		 * another CPU might be blocked waiting for us to process
+		 * an IPI.
 		 */
 		touch_nmi_watchdog();
+		touch_all_softlockup_watchdogs();
 		if (!state_filter || (p->state & state_filter))
 			sched_show_task(p);
 	}
 
-	touch_all_softlockup_watchdogs();
-
 #ifdef CONFIG_SCHED_DEBUG
 	if (!state_filter)
 		sysrq_sched_debug_show();