[PATCH] XPC heartbeat timer function must run on CPU 0

[PATCH] XPC heartbeat timer function must run on CPU 0

[Dean Nelson]

Currently, XPC's heartbeat timer function runs on whatever CPU modprobe/insmod
ran on when XPC was started. To avoid the heartbeat from being delayed for
long periods the timer function must run on CPU 0.

Signed-off-by: Dean Nelson <dcn@sgi.com>


Index: linux-2.6/arch/ia64/sn/kernel/xpc_main.c
=================================--- linux-2.6.orig/arch/ia64/sn/kernel/xpc_main.c	2007-11-05 13:39:09.024152387 -0600
+++ linux-2.6/arch/ia64/sn/kernel/xpc_main.c	2007-11-05 13:44:28.815893952 -0600
@@ -3,7 +3,7 @@
  * License.  See the file "COPYING" in the main directory of this archive
  * for more details.
  *
- * Copyright (c) 2004-2006 Silicon Graphics, Inc.  All Rights Reserved.
+ * Copyright (c) 2004-2007 Silicon Graphics, Inc.  All Rights Reserved.
  */
 
 
@@ -257,7 +257,9 @@
 
 	set_cpus_allowed(current, cpumask_of_cpu(XPC_HB_CHECK_CPU));
 
+	/* set our heartbeating to other partitions into motion */
 	xpc_hb_check_timeout = jiffies + (xpc_hb_check_interval * HZ);
+	xpc_hb_beater(0);
 
 	while (!(volatile int) xpc_exiting) {
 
@@ -1338,16 +1340,8 @@
 		dev_warn(xpc_part, "can't register die notifier\n");
 	}
 
-
-	/*
-	 * Set the beating to other partitions into motion.  This is
-	 * the last requirement for other partitions' discovery to
-	 * initiate communications with us.
-	 */
 	init_timer(&xpc_hb_timer);
 	xpc_hb_timer.function = xpc_hb_beater;
-	xpc_hb_beater(0);
-
 
 	/*
 	 * The real work-horse behind xpc.  This processes incoming

RE: [PATCH] XPC heartbeat timer function must run on CPU 0

[Luck, Tony]

> Currently, XPC's heartbeat timer function runs on whatever CPU modprobe/insmod
> ran on when XPC was started. To avoid the heartbeat from being delayed for
> long periods the timer function must run on CPU 0.

Does Altix allow hot remove of cpu0?  If so, then you have a problem.

-Tony

Re: [PATCH] XPC heartbeat timer function must run on CPU 0

[Dean Nelson]

On Wed, Nov 07, 2007 at 08:26:59AM -0800, Luck, Tony wrote:
> > Currently, XPC's heartbeat timer function runs on whatever CPU modprobe/insmod
> > ran on when XPC was started. To avoid the heartbeat from being delayed for
> > long periods the timer function must run on CPU 0.
> 
> Does Altix allow hot remove of cpu0?  If so, then you have a problem.

At this time Altix does not support hot remove of cpu0, but it is something
currently being worked on. At the time that feature is ready then this
code would indeed need to be modified. In the meantime, we have an existing
customer in need of a fix.

Thanks,
Dean

Re: [PATCH] XPC heartbeat timer function must run on CPU 0

[Simon Horman]

On Thu, Nov 08, 2007 at 06:11:19AM -0600, Dean Nelson wrote:
> On Wed, Nov 07, 2007 at 08:26:59AM -0800, Luck, Tony wrote:
> > > Currently, XPC's heartbeat timer function runs on whatever CPU modprobe/insmod
> > > ran on when XPC was started. To avoid the heartbeat from being delayed for
> > > long periods the timer function must run on CPU 0.
> > 
> > Does Altix allow hot remove of cpu0?  If so, then you have a problem.
> 
> At this time Altix does not support hot remove of cpu0, but it is something
> currently being worked on. At the time that feature is ready then this
> code would indeed need to be modified. In the meantime, we have an existing
> customer in need of a fix.

Kdump will run the second (crash) kernel on whatever CPU the crash
occurs on, so its quite possible for cpu 0 not to exist in the
second kernel. Would this be a problem for this code?

-- 
Horms, California Edition

Re: [PATCH] XPC heartbeat timer function must run on CPU 0

[Robin Holt]

On Wed, Nov 14, 2007 at 12:49:14PM -0800, Simon Horman wrote:
> On Thu, Nov 08, 2007 at 06:11:19AM -0600, Dean Nelson wrote:
> > On Wed, Nov 07, 2007 at 08:26:59AM -0800, Luck, Tony wrote:
> > > > Currently, XPC's heartbeat timer function runs on whatever CPU modprobe/insmod
> > > > ran on when XPC was started. To avoid the heartbeat from being delayed for
> > > > long periods the timer function must run on CPU 0.
> > > 
> > > Does Altix allow hot remove of cpu0?  If so, then you have a problem.
> > 
> > At this time Altix does not support hot remove of cpu0, but it is something
> > currently being worked on. At the time that feature is ready then this
> > code would indeed need to be modified. In the meantime, we have an existing
> > customer in need of a fix.
> 
> Kdump will run the second (crash) kernel on whatever CPU the crash
> occurs on, so its quite possible for cpu 0 not to exist in the
> second kernel. Would this be a problem for this code?

Wouldn't that cpu appear to the new OS instance as cpu0?  If so, then
it should not be an issue.

Thanks,
Robin

Re: [PATCH] XPC heartbeat timer function must run on CPU 0

[Simon Horman]

On Wed, Nov 14, 2007 at 03:12:38PM -0600, Robin Holt wrote:
> On Wed, Nov 14, 2007 at 12:49:14PM -0800, Simon Horman wrote:
> > On Thu, Nov 08, 2007 at 06:11:19AM -0600, Dean Nelson wrote:
> > > On Wed, Nov 07, 2007 at 08:26:59AM -0800, Luck, Tony wrote:
> > > > > Currently, XPC's heartbeat timer function runs on whatever CPU modprobe/insmod
> > > > > ran on when XPC was started. To avoid the heartbeat from being delayed for
> > > > > long periods the timer function must run on CPU 0.
> > > > 
> > > > Does Altix allow hot remove of cpu0?  If so, then you have a problem.
> > > 
> > > At this time Altix does not support hot remove of cpu0, but it is something
> > > currently being worked on. At the time that feature is ready then this
> > > code would indeed need to be modified. In the meantime, we have an existing
> > > customer in need of a fix.
> > 
> > Kdump will run the second (crash) kernel on whatever CPU the crash
> > occurs on, so its quite possible for cpu 0 not to exist in the
> > second kernel. Would this be a problem for this code?
> 
> Wouldn't that cpu appear to the new OS instance as cpu0?  If so, then
> it should not be an issue.

No, I don't believe that it does.
I can try and test this theory if it helps.
Perhaps my mind is corrupted by x86 thoughts.

-- 
Horms, California Edition

RE: [PATCH] XPC heartbeat timer function must run on CPU 0

[Luck, Tony]

>> Wouldn't that cpu appear to the new OS instance as cpu0?  If so, then
>> it should not be an issue.
>
> No, I don't believe that it does.
> I can try and test this theory if it helps.
> Perhaps my mind is corrupted by x86 thoughts.

We are talking about logical cpu0 here.  Surely the new instance
must call whatever cpu it is running on first "0".  E.g. I could
have used kexec to switch to a UP kernel that only supports cpu0.

-Tony

Re: [PATCH] XPC heartbeat timer function must run on CPU 0

[Simon Horman]

On Wed, Nov 14, 2007 at 02:19:15PM -0800, Luck, Tony wrote:
> >> Wouldn't that cpu appear to the new OS instance as cpu0?  If so, then
> >> it should not be an issue.
> >
> > No, I don't believe that it does.
> > I can try and test this theory if it helps.
> > Perhaps my mind is corrupted by x86 thoughts.
> 
> We are talking about logical cpu0 here.  Surely the new instance
> must call whatever cpu it is running on first "0".  E.g. I could
> have used kexec to switch to a UP kernel that only supports cpu0.

I took a look into this, and yes you are right.
Regardless of which physical CPU is used to boot on,
it is always logical cpu 0. So the problem that I
was thinking about doesn't exist. Silly me.

I also noticed that even if it was posisble for logical cpu 0 to
not be present, the patch will still work as the hearbeat
code would just end up being run on the prevailing logical cpu.

So no problem x2 :-)

-- 
Horms, California Edition