[PATCH] xen: ensure timer tick is resumed even on CPU driving the resume

[PATCH] xen: ensure timer tick is resumed even on CPU driving the resume

[Ian Campbell]

The core suspend/resume code is run from stop_machine on CPU0 but
parts of the suspend/resume machinery (including xen_arch_resume) are
run on whichever CPU happened to schedule the xenwatch kernel thread.

As part of the non-core resume code xen_arch_resume is called in order
to restart the timer tick on non-boot processors. The boot processor
itself is taken care of by core timekeeping code.

xen_arch_resume uses smp_call_function which does not call the given
function on the current processor. This means that we can end up with
one CPU not receiving timer ticks if the xenwatch thread happened to
be scheduled on CPU > 0.

Use on_each_cpu instead of smp_call_function to ensure the timer tick
is resumed everywhere.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Stable Kernel <stable@kernel.org>
---
 arch/x86/xen/suspend.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c
index 987267f..a9c6611 100644
--- a/arch/x86/xen/suspend.c
+++ b/arch/x86/xen/suspend.c
@@ -60,6 +60,6 @@ static void xen_vcpu_notify_restore(void *data)
 
 void xen_arch_resume(void)
 {
-	smp_call_function(xen_vcpu_notify_restore,
-			       (void *)CLOCK_EVT_NOTIFY_RESUME, 1);
+	on_each_cpu(xen_vcpu_notify_restore,
+		    (void *)CLOCK_EVT_NOTIFY_RESUME, 1);
 }
-- 
1.5.6.5

Re: [Xen-devel] [PATCH] xen: ensure timer tick is resumed even on CPU driving the resume

[Jeremy Fitzhardinge]

On 05/19/2010 08:36 AM, Ian Campbell wrote:
> The core suspend/resume code is run from stop_machine on CPU0 but
> parts of the suspend/resume machinery (including xen_arch_resume) are
> run on whichever CPU happened to schedule the xenwatch kernel thread.
>
> As part of the non-core resume code xen_arch_resume is called in order
> to restart the timer tick on non-boot processors. The boot processor
> itself is taken care of by core timekeeping code.
>
> xen_arch_resume uses smp_call_function which does not call the given
> function on the current processor. This means that we can end up with
> one CPU not receiving timer ticks if the xenwatch thread happened to
> be scheduled on CPU > 0.
>
> Use on_each_cpu instead of smp_call_function to ensure the timer tick
> is resumed everywhere.
>   

Argh, that seems to be a pretty common trap to fall into.  Looks OK (but
unfortunately doesn't fix my other problem).

Acked-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

    J

> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Jeremy Fitzhardinge <jeremy@goop.org>
> Cc: Stable Kernel <stable@kernel.org>
> ---
>  arch/x86/xen/suspend.c |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c
> index 987267f..a9c6611 100644
> --- a/arch/x86/xen/suspend.c
> +++ b/arch/x86/xen/suspend.c
> @@ -60,6 +60,6 @@ static void xen_vcpu_notify_restore(void *data)
>  
>  void xen_arch_resume(void)
>  {
> -	smp_call_function(xen_vcpu_notify_restore,
> -			       (void *)CLOCK_EVT_NOTIFY_RESUME, 1);
> +	on_each_cpu(xen_vcpu_notify_restore,
> +		    (void *)CLOCK_EVT_NOTIFY_RESUME, 1);
>  }
>   

Re: [Xen-devel] [PATCH] xen: ensure timer tick is resumed even on CPU driving the resume

[Jeremy Fitzhardinge]

On 05/20/2010 10:42 AM, Shriram Rajagopalan wrote:
> I know this is a long shot but is there an equivalent piece of code in
> the 2.6.18.8-xen kernel where this patch could be applied ? Assuming
> that the 2.6.18 also has a similar bug.

Very unlikely.  Time is completely different in 2.6.18-xen.

    J

[GIT PULL] xen suspend resume fix.

[Ian Campbell]

The following changes since commit aff67593b7fb2efa1ba17eb07ded96519b6b7cc1:
  Ian Campbell (1):
        xenbus: do not hold transaction_mutex when returning to userspace

are available in the git repository at:

  git://xenbits.xensource.com/people/ianc/linux-2.6.git for-jeremy/saverestore

Ian Campbell (1):
      xen: ensure timer tick is resumed even on CPU driving the resume

 arch/x86/xen/suspend.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

(this is based on xen.git#bugfix branch. I didn't base on 2.6.32 since
the prerequisite patches were only in 2.6.33 and 2.6.32.2. this is not
an area with a great deal of churn so it should merge around ok)

[PATCH] xen: ensure timer tick is resumed even on CPU driving the resume

[Ian Campbell]

The core suspend/resume code is run from stop_machine on CPU0 but
parts of the suspend/resume machinery (including xen_arch_resume) are
run on whichever CPU happened to schedule the xenwatch kernel thread.

As part of the non-core resume code xen_arch_resume is called in order
to restart the timer tick on non-boot processors. The boot processor
itself is taken care of by core timekeeping code.

xen_arch_resume uses smp_call_function which does not call the given
function on the current processor. This means that we can end up with
one CPU not receiving timer ticks if the xenwatch thread happened to
be scheduled on CPU > 0.

Use on_each_cpu instead of smp_call_function to ensure the timer tick
is resumed everywhere.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Stable Kernel <stable@kernel.org>
---

Resend with correct in-reply-to.

 arch/x86/xen/suspend.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c
index 987267f..a9c6611 100644
--- a/arch/x86/xen/suspend.c
+++ b/arch/x86/xen/suspend.c
@@ -60,6 +60,6 @@ static void xen_vcpu_notify_restore(void *data)
 
 void xen_arch_resume(void)
 {
-	smp_call_function(xen_vcpu_notify_restore,
-			       (void *)CLOCK_EVT_NOTIFY_RESUME, 1);
+	on_each_cpu(xen_vcpu_notify_restore,
+		    (void *)CLOCK_EVT_NOTIFY_RESUME, 1);
 }
-- 
1.5.6.5