[PATCH] disable softlockup detection at boottime

[PATCH] disable softlockup detection at boottime

[Dimitri Sivanich]

This patch allows softlockup detection to be disabled by specifying
a boottime parameter.

Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>

Index: linux/kernel/softlockup.c
===================================================================
--- linux.orig/kernel/softlockup.c	2008-04-23 13:50:06.000000000 -0500
+++ linux/kernel/softlockup.c	2008-04-23 14:39:36.267398531 -0500
@@ -24,6 +24,7 @@ static DEFINE_PER_CPU(unsigned long, tou
 static DEFINE_PER_CPU(unsigned long, print_timestamp);
 static DEFINE_PER_CPU(struct task_struct *, watchdog_task);
 
+static int __read_mostly softlockup_off;
 static int __read_mostly did_panic;
 unsigned long __read_mostly softlockup_thresh = 60;
 
@@ -53,7 +54,8 @@ void touch_softlockup_watchdog(void)
 {
 	int this_cpu = raw_smp_processor_id();
 
-	__raw_get_cpu_var(touch_timestamp) = get_timestamp(this_cpu);
+	if (!softlockup_off)
+		__raw_get_cpu_var(touch_timestamp) = get_timestamp(this_cpu);
 }
 EXPORT_SYMBOL(touch_softlockup_watchdog);
 
@@ -61,6 +63,9 @@ void touch_all_softlockup_watchdogs(void
 {
 	int cpu;
 
+	if (softlockup_off)
+		return;
+
 	/* Cause each CPU to re-update its timestamp rather than complain */
 	for_each_online_cpu(cpu)
 		per_cpu(touch_timestamp, cpu) = 0;
@@ -79,6 +84,9 @@ void softlockup_tick(void)
 	struct pt_regs *regs = get_irq_regs();
 	unsigned long now;
 
+	if (softlockup_off)
+		return;
+
 	if (touch_timestamp == 0) {
 		touch_softlockup_watchdog();
 		return;
@@ -306,9 +314,19 @@ __init void spawn_softlockup_task(void)
 	void *cpu = (void *)(long)smp_processor_id();
 	int err = cpu_callback(&cpu_nfb, CPU_UP_PREPARE, cpu);
 
+	if (softlockup_off)
+		return;
+
 	BUG_ON(err == NOTIFY_BAD);
 	cpu_callback(&cpu_nfb, CPU_ONLINE, cpu);
 	register_cpu_notifier(&cpu_nfb);
 
 	atomic_notifier_chain_register(&panic_notifier_list, &panic_block);
 }
+
+static int __init disable_softlockup_detection(char *str)
+{
+	softlockup_off = 1;
+	return 1;
+}
+__setup("nosoftlockup", disable_softlockup_detection);

Re: [PATCH] disable softlockup detection at boottime

[Randy Dunlap]

On Wed, 23 Apr 2008 15:16:13 -0500 Dimitri Sivanich wrote:

> This patch allows softlockup detection to be disabled by specifying
> a boottime parameter.

Seems to be missing (a) justification/why and
(b) entry in Documentation/kernel-parameters.txt


> Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>
> 
> Index: linux/kernel/softlockup.c
> ===================================================================
> --- linux.orig/kernel/softlockup.c	2008-04-23 13:50:06.000000000 -0500
> +++ linux/kernel/softlockup.c	2008-04-23 14:39:36.267398531 -0500
> @@ -24,6 +24,7 @@ static DEFINE_PER_CPU(unsigned long, tou
>  static DEFINE_PER_CPU(unsigned long, print_timestamp);
>  static DEFINE_PER_CPU(struct task_struct *, watchdog_task);
>  
> +static int __read_mostly softlockup_off;
>  static int __read_mostly did_panic;
>  unsigned long __read_mostly softlockup_thresh = 60;
>  
> @@ -53,7 +54,8 @@ void touch_softlockup_watchdog(void)
>  {
>  	int this_cpu = raw_smp_processor_id();
>  
> -	__raw_get_cpu_var(touch_timestamp) = get_timestamp(this_cpu);
> +	if (!softlockup_off)
> +		__raw_get_cpu_var(touch_timestamp) = get_timestamp(this_cpu);
>  }
>  EXPORT_SYMBOL(touch_softlockup_watchdog);
>  
> @@ -61,6 +63,9 @@ void touch_all_softlockup_watchdogs(void
>  {
>  	int cpu;
>  
> +	if (softlockup_off)
> +		return;
> +
>  	/* Cause each CPU to re-update its timestamp rather than complain */
>  	for_each_online_cpu(cpu)
>  		per_cpu(touch_timestamp, cpu) = 0;
> @@ -79,6 +84,9 @@ void softlockup_tick(void)
>  	struct pt_regs *regs = get_irq_regs();
>  	unsigned long now;
>  
> +	if (softlockup_off)
> +		return;
> +
>  	if (touch_timestamp == 0) {
>  		touch_softlockup_watchdog();
>  		return;
> @@ -306,9 +314,19 @@ __init void spawn_softlockup_task(void)
>  	void *cpu = (void *)(long)smp_processor_id();
>  	int err = cpu_callback(&cpu_nfb, CPU_UP_PREPARE, cpu);
>  
> +	if (softlockup_off)
> +		return;
> +
>  	BUG_ON(err == NOTIFY_BAD);
>  	cpu_callback(&cpu_nfb, CPU_ONLINE, cpu);
>  	register_cpu_notifier(&cpu_nfb);
>  
>  	atomic_notifier_chain_register(&panic_notifier_list, &panic_block);
>  }
> +
> +static int __init disable_softlockup_detection(char *str)
> +{
> +	softlockup_off = 1;
> +	return 1;
> +}
> +__setup("nosoftlockup", disable_softlockup_detection);
> --

---
~Randy

Re: [PATCH] disable softlockup detection at boottime

[Dimitri Sivanich]

On Wed, Apr 23, 2008 at 02:55:17PM -0700, Randy Dunlap wrote:
> (b) entry in Documentation/kernel-parameters.txt

Oops, my bad, I missed the other 'nosoftlockup'.

Re: [PATCH] disable softlockup detection at boottime

[Ingo Molnar]

* Dimitri Sivanich <sivanich@sgi.com> wrote:

> This patch allows softlockup detection to be disabled by specifying a 
> boottime parameter.

good idea - but why dont you set softlockup_thresh to 0, which is the 
"off" switch already? (and that way it can be turned back on later as 
well, by the sysadmin.)

	Ingo

Re: [PATCH] disable softlockup detection at boottime

[Dimitri Sivanich]

On Mon, Apr 28, 2008 at 06:55:44PM +0200, Ingo Molnar wrote:
> 
> good idea - but why dont you set softlockup_thresh to 0, which is the 
> "off" switch already? (and that way it can be turned back on later as 
> well, by the sysadmin.)
> 

I'm getting unaligned access errors trying to set it to anything, so it's not working for me currently (2.6.25):

It's tripping up on the address of 'one', which is an int that is not properly aligned for the unsigned long comparison in proc_doulongvec_minmax on my 64 bit machine.  Also, the value '0' is invalid for softlockup_thresh, correct?

I temporarily got around these issues with the following hack.

Index: linux/kernel/sysctl.c
===================================================================
--- linux.orig/kernel/sysctl.c  2008-04-16 21:49:44.000000000 -0500
+++ linux/kernel/sysctl.c       2008-04-28 13:37:43.000561710 -0500
@@ -748,9 +748,9 @@ static struct ctl_table kern_table[] = {
                .data           = &softlockup_thresh,
                .maxlen         = sizeof(unsigned long),
                .mode           = 0644,
-               .proc_handler   = &proc_doulongvec_minmax,
+               .proc_handler   = &proc_dointvec_minmax,
                .strategy       = &sysctl_intvec,
-               .extra1         = &one,
+               .extra1         = &zero,
                .extra2         = &sixty,



Also, I'm not convinced that changing this to 0 does indeed switch off softlockup detection (but I could be missing something):

void softlockup_tick(void)
{
..
..
        /* Warn about unreasonable delays: */
        if (now <= (touch_timestamp + softlockup_thresh))
                return;
        
        per_cpu(print_timestamp, this_cpu) = touch_timestamp;
   
        spin_lock(&print_lock);
        printk(KERN_ERR "BUG: soft lockup - CPU#%d stuck for %lus! [%s:%d]\n",
                        this_cpu, now - touch_timestamp,
                        current->comm, task_pid_nr(current));


Dimitri

Re: [PATCH] disable softlockup detection at boottime

[Ingo Molnar]

* Dimitri Sivanich <sivanich@sgi.com> wrote:

> On Mon, Apr 28, 2008 at 06:55:44PM +0200, Ingo Molnar wrote:
> > 
> > good idea - but why dont you set softlockup_thresh to 0, which is the 
> > "off" switch already? (and that way it can be turned back on later as 
> > well, by the sysadmin.)
> > 
> 
> I'm getting unaligned access errors trying to set it to anything, so 
> it's not working for me currently (2.6.25):
> 
> It's tripping up on the address of 'one', which is an int that is not 
> properly aligned for the unsigned long comparison in 
> proc_doulongvec_minmax on my 64 bit machine.  Also, the value '0' is 
> invalid for softlockup_thresh, correct?
> 
> I temporarily got around these issues with the following hack.

ah, sorry. But ... perhaps using threshold -1 would be the most 
intuitive setting? (for 'infinite timeout' ==> softlockup detector 
turned off) That way it all becomes configurable as part of the 
threshold? No strong opinion though.

	Ingo

Re: [PATCH] disable softlockup detection at boottime

[Dimitri Sivanich]

On Tue, Apr 29, 2008 at 02:35:22PM +0200, Ingo Molnar wrote:
> * Dimitri Sivanich <sivanich@sgi.com> wrote:
> 
> > I'm getting unaligned access errors trying to set it to anything, so 
> > it's not working for me currently (2.6.25):
> > 
> > It's tripping up on the address of 'one', which is an int that is not 
> > properly aligned for the unsigned long comparison in 
> > proc_doulongvec_minmax on my 64 bit machine.  Also, the value '0' is 
> > invalid for softlockup_thresh, correct?
> > 
> > I temporarily got around these issues with the following hack.
> 
> ah, sorry. But ... perhaps using threshold -1 would be the most 
> intuitive setting? (for 'infinite timeout' ==> softlockup detector 
> turned off) That way it all becomes configurable as part of the 
> threshold? No strong opinion though.
>

Having the ability to switch it off dynamically would be nice to have, but
will need a little work yet.  Any solution should also minimize the amount
of time spent in softlockup_tick.

Either '0' or '-1' seems OK to me, but if '-1' turns it off, do we allow
a value of '0'?.

Re: [PATCH] disable softlockup detection at boottime

[Ingo Molnar]

* Dimitri Sivanich <sivanich@sgi.com> wrote:

> > ah, sorry. But ... perhaps using threshold -1 would be the most 
> > intuitive setting? (for 'infinite timeout' ==> softlockup detector 
> > turned off) That way it all becomes configurable as part of the 
> > threshold? No strong opinion though.
> 
> Having the ability to switch it off dynamically would be nice to have, 
> but will need a little work yet.  Any solution should also minimize 
> the amount of time spent in softlockup_tick.
> 
> Either '0' or '-1' seems OK to me, but if '-1' turns it off, do we 
> allow a value of '0'?.

i think <= 0 should be the condition for "off". That means both -1 and 0 
would be OK.

	Ingo