Re: oprofile + hibernation = badness

Robert Richter
2008-09-01T16:35:49Z

Ingo,

this fix (commit id 1404e403) is not yet upstream for v2.6.27. Could
this be added to the next pull request?

Thanks,

-Robert

On 19.08.08 03:13:38, Andi Kleen wrote:
>
>> These keep on coming all through the suspend/shutdown sequence, also
>> intermixing with other messages. I'm guessing oprofile is trying to
>> NMI CPUs that have been brought down?
>
> That should actually work in theory. Linux CPU offlining puts CPUs in a
> state where
> they can still process NMIs. Hmm actually there was a change recently to
> free
> their exception stacks. Maybe it's broken now.
>
> Also oprofile has a suspend method that disables NMIs.
>
>> Now I get some ACPI Exceptions, but I think that these are unrelated
>> to starting oprofile, because I have seen them on regular shutdowns as
>> well:
>> PM: Syncing filesystems ... done.
>> Freezing user space processes ... (elapsed 0.00 seconds) done.
>> Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
>> PM: Shrinking memory... done (0 pages freed)
>> PM: Freed 0 kbytes in 0.44 seconds (0.00 MB/s)
>> Suspending console(s) (use no_console_suspend to debug)
>> ACPI Exception (exoparg2-0444): AE_AML_PACKAGE_LIMIT, Index
>> (000000007) is beyond end of object [20080609]
>> ACPI Error (psparse-0530): Method parse/execution failed
>> [\_SB_.PCI0.IDE0.GTM_] (Node f783bfc0), AE_AML_PACKAGE_LIMIT
>> ACPI Error (psparse-0530): Method parse/execution failed
>> [\_SB_.PCI0.IDE0.CHN0._GTM] (Node f783bb40), AE_AML_PACKAGE_LIMIT
>> ACPI handle has no context!
>> serial 00:0d: disabled
>> serial 00:06: disabled
>> ehci_hcd 0000:00:1d.7: PCI INT A disabled
>> uhci_hcd 0000:00:1d.3: PCI INT D disabled
>> uhci_hcd 0000:00:1d.2: PCI INT C disabled
>> uhci_hcd 0000:00:1d.1: PCI INT B disabled
>> uhci_hcd 0000:00:1d.0: PCI INT A disabled
>> ACPI: Preparing to enter system sleep state S4
>> After that, I see the message "WQ on CPU0, prefer CPU1" a few times.
>> This must also be bad.
>> Now some warnings:
>> ------------[ cut here ]------------
>> WARNING: at
>> /uio/arkimedes/s29/vegardno/git-working/linux-2.6/kernel/smp.c:328 s
>> mp_call_function_mask+0x194/0x1a0()
>
> The usual problem: the suspend function when interrupts are
> already disabled calls smp_call_function which is not allowed with
> interrupt off. But at this point all the other CPUs should be already
> down anyways, so it should be enough to just drop that.
>
> This patch should fix that problem at least by fixing cpu hotplug&
> suspend support. Untested.
>
> -Andi
>

> From 3967a8b03a30f26a6a5f41b4cc6a6772b5d0ce4f Mon Sep 17 00:00:00 2001
> From: Andi Kleen <ak@linux.intel.com>
> Date: Tue, 19 Aug 2008 03:10:03 +0200
> Subject: [PATCH] oprofile: fix cpu hotcplug
>
> Signed-off-by: Andi Kleen <ak@linux.intel.com>
> ---
> arch/x86/oprofile/nmi_int.c | 40 +++++++++++++++++++++++++++++++++++++---
> 1 files changed, 37 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/oprofile/nmi_int.c b/arch/x86/oprofile/nmi_int.c
> index 529d625..6438c32 100644
> --- a/arch/x86/oprofile/nmi_int.c
> +++ b/arch/x86/oprofile/nmi_int.c
> @@ -15,6 +15,7 @@
> #include <linux/slab.h>
> #include <linux/moduleparam.h>
> #include <linux/kdebug.h>
> +#include <linux/cpu.h>
> #include <asm/nmi.h>
> #include <asm/msr.h>
> #include <asm/apic.h>
> @@ -28,23 +29,48 @@ static DEFINE_PER_CPU(unsigned long, saved_lvtpc);
>
> static int nmi_start(void);
> static void nmi_stop(void);
> +static void nmi_cpu_start(void *dummy);
> +static void nmi_cpu_stop(void *dummy);
>
> /* 0 == registered but off, 1 == registered and on */
> static int nmi_enabled = 0;
>
> +#ifdef CONFIG_SMP
> +static int oprofile_cpu_notifier(struct notifier_block *b, unsigned long action,
> + void *data)
> +{
> + int cpu = (unsigned long)data;
> + switch (action) {
> + case CPU_DOWN_FAILED:
> + case CPU_ONLINE:
> + smp_call_function_single(cpu, nmi_cpu_start, NULL, 0);
> + break;
> + case CPU_DOWN_PREPARE:
> + smp_call_function_single(cpu, nmi_cpu_stop, NULL, 1);
> + break;
> + }
> + return NOTIFY_DONE;
> +}
> +
> +static struct notifier_block oprofile_cpu_nb = {
> + .notifier_call = oprofile_cpu_notifier
> +};
> +#endif
> +
> #ifdef CONFIG_PM
>
> static int nmi_suspend(struct sys_device *dev, pm_message_t state)
> {
> + /* Only one CPU left, just stop that one */
> if (nmi_enabled == 1)
> - nmi_stop();
> + nmi_cpu_stop(NULL);
> return 0;
> }
>
> static int nmi_resume(struct sys_device *dev)
> {
> if (nmi_enabled == 1)
> - nmi_start();
> + nmi_cpu_start(NULL);
> return 0;
> }
>
> @@ -66,6 +92,7 @@ static int __init init_sysfs(void)
> error = sysdev_class_register(&oprofile_sysclass);
> if (!error)
> error = sysdev_register(&device_oprofile);
> +
> return error;
> }
>
> @@ -483,6 +510,9 @@ int __init op_nmi_init(struct oprofile_operations *ops)
> }
>
> init_sysfs();
> +#ifdef CONFIG_SMP
> + register_cpu_notifier(&oprofile_cpu_nb);
> +#endif
> using_nmi = 1;
> ops->create_files = nmi_create_files;
> ops->setup = nmi_setup;
> @@ -496,6 +526,10 @@ int __init op_nmi_init(struct oprofile_operations *ops)
>
> void op_nmi_exit(void)
> {
> - if (using_nmi)
> + if (using_nmi) {
> exit_sysfs();
> +#ifdef CONFIG_SMP
> + unregister_cpu_notifier(&oprofile_cpu_nb);
> +#endif
> + }
> }
> --
> 1.5.6
>

--
Advanced Micro Devices, Inc.
Operating System Research Center
email: robert.richter@amd.com