Re: broken suspend (sched related) [Was: 2.6.24-rc4-mm1]

[Gautham R Shenoy <ego@in.ibm.com>]

On Mon, Dec 10, 2007 at 11:21:57AM +0100, Ingo Molnar wrote:
> 
> * Gautham R Shenoy <ego@in.ibm.com> wrote:
> 
> > > i'm wondering, what's the proper CPU-hotplug safe sequence here 
> > > then? I'm picking a CPU number from cpu_online_map, and that CPU 
> > > could go away while i'm still using it, right? What's saving us 
> > > here?
> > 
> > In this particular case, we are trying to see if any task on a 
> > particular cpu has not been scheduled for a really long time. If we do 
> > this check on a cpu which has gone offline, then a) If the tasks have 
> > not been migrated on to another cpu yet, we will still perform that 
> > check and yell if something has been holding any task for a 
> > sufficiently long time. b) If the tasks have been migrated off, then 
> > we have nothing to check.
> 
> say we've got 100 CPUs, so we've got 100 watchdog tasks running - one 
> for each CPU. Checking for hung tasks is a global operation not a 
> per-CPU operation (we iterate over the global tasklist), hence only one 
> CPU should really be calling this function. That online-cpus logic 
> achieves this by picking a single CPU. Perhaps it would be better to 
> keep a hung_task_checker_cpu variable that is driven from a 
> CPU-hotplug-down notifier? That way if a CPU is brought down we can 
> update hung_task_checker_cpu to another, still-online CPU. (this would 
> also be faster, because event-driven)

Do you mean something like this?

From: Gautham R Shenoy <ego@in.ibm.com>

softlockup: update check_cpu during cpu-hotplug

Update the check_cpu value during a cpu-hotplug operation
so that we don't check for hung tasks on a cpu which is about 
to go offline.

Signed-off-by: Gautham R Shenoy <ego@in.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linuxtronix.de>
Cc: Jiri Slaby <jirislaby@gmail.com>
---

 kernel/softlockup.c |   14 ++++++++++++--
 1 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/kernel/softlockup.c b/kernel/softlockup.c
index 576eb9c..b1a8c7c 100644
--- a/kernel/softlockup.c
+++ b/kernel/softlockup.c
@@ -194,6 +194,9 @@ static void check_hung_uninterruptible_tasks(int this_cpu)
 	read_unlock(&tasklist_lock);
 }
 
+
+static int check_cpu = -1;
+
 /*
  * The watchdog thread - runs every second and touches the timestamp.
  */
@@ -219,8 +222,6 @@ static int watchdog(void *__bind_cpu)
 		/*
 		 * Only do the hung-tasks check on one CPU:
 		 */
-		check_cpu = any_online_cpu(cpu_online_map);
-
 		if (this_cpu != check_cpu)
 			continue;
 
@@ -255,6 +256,7 @@ cpu_callback(struct notifier_block *nfb, unsigned long action, void *hcpu)
 		break;
 	case CPU_ONLINE:
 	case CPU_ONLINE_FROZEN:
+		check_cpu = any_online_cpu(cpu_online_map);
 		wake_up_process(per_cpu(watchdog_task, hotcpu));
 		break;
 #ifdef CONFIG_HOTPLUG_CPU
@@ -265,6 +267,14 @@ cpu_callback(struct notifier_block *nfb, unsigned long action, void *hcpu)
 		/* Unbind so it can run.  Fall thru. */
 		kthread_bind(per_cpu(watchdog_task, hotcpu),
 			     any_online_cpu(cpu_online_map));
+	case CPU_DOWN_PREPARE:
+	case CPU_DOWN_PREPARE_FROZEN:
+		if (hotcpu == check_cpu) {
+			cpumask_t temp_cpu_online_map = cpu_online_map; 
+			cpu_clear(hotcpu, temp_cpu_online_map);
+			check_cpu = any_online_cpu(temp_cpu_online_map);
+		}
+		break;
 	case CPU_DEAD:
 	case CPU_DEAD_FROZEN:
 		p = per_cpu(watchdog_task, hotcpu);



> 
> 	Ingo

Thanks and Regards
gautham.
-- 
Gautham R Shenoy
Linux Technology Center
IBM India.
"Freedom comes with a price tag of responsibility, which is still a bargain,
because Freedom is priceless!"