From: Amit Shah <amit.shah@redhat.com>
To: Marcelo Tosatti <mtosatti@redhat.com>
Cc: kvm@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com,
hpa@zytor.com, x86@kernel.org, johnstul@us.ibm.com,
riel@redhat.com, avi@redhat.com, imammedo@redhat.com
Subject: Re: x86: kvmclock: abstract save/restore sched_clock_state
Date: Thu, 9 Feb 2012 17:57:42 +0530 [thread overview]
Message-ID: <20120209122742.GA25505@amit.redhat.com> (raw)
In-Reply-To: <20120207210542.GC20618@amt.cnet>
On (Tue) 07 Feb 2012 [19:05:42], Marcelo Tosatti wrote:
>
> Upon resume from hibernation, CPU 0's hvclock area contains the old
> values for system_time and tsc_timestamp. It is necessary for the
> hypervisor to update these values with uptodate ones before the CPU uses
> them.
>
> Abstract TSC's save/restore sched_clock_state functions and use
> restore_state to write to KVM_SYSTEM_TIME MSR, forcing an update.
>
> Fixes suspend-to-disk with kvmclock.
There are stalls after resume, see trace below.
./x86_64-softmmu/qemu-system-x86_64 -kernel ~/src/linux/arch/x86/boot/bzImage -append 'root=/dev/vda1 console=tty0 console=ttyS0 no_console_suspend' -drive file=/guests/f14-suspend.qcow2,if=none,id=dr0 -device virtio-blk-pci,drive=dr0 -net nic,model=virtio -net user -serial stdio -enable-kvm -m 512 -cpu host -smp 4
<The following is on resume:>
Disabling non-boot CPUs ...
CPU 1 is now offline
CPU 2 is now offline
CPU 3 is now offline
lockdep: fixing up alternatives.
kvm-clock: cpu 0, msr 0:1f1d19c1, primary cpu clock, resume
PM: Restoring platform NVS memory
Enabling non-boot CPUs ...
lockdep: fixing up alternatives.
Booting Node 0 Processor 1 APIC 0x1
Calibrating delay loop (skipped) already calibrated this CPU
INFO: rcu_sched detected stalls on CPUs/tasks: { 1} (detected by 0, t=314192 jiffies)
Pid: 662, comm: bash Not tainted 3.3.0-rc2+ #293
Call Trace:
<IRQ> [<ffffffff8108b6c9>] __rcu_pending+0x268/0x3c5
[<ffffffff8108bef5>] rcu_check_callbacks+0xab/0x108
[<ffffffff81031208>] update_process_times+0x3f/0x75
[<ffffffff810649b5>] tick_sched_timer+0x6d/0x8c
[<ffffffff81042e68>] __run_hrtimer+0xc9/0x15c
[<ffffffff81064948>] ? tick_nohz_handler+0xd5/0xd5
[<ffffffff8104359b>] hrtimer_interrupt+0xea/0x1b4
[<ffffffff81316192>] smp_apic_timer_interrupt+0x76/0x89
[<ffffffff81315833>] apic_timer_interrupt+0x73/0x80
<EOI> [<ffffffff8130b9ec>] ? arch_local_irq_restore+0x6/0xd
[<ffffffff8130840c>] native_cpu_up+0x108/0x127
[<ffffffff81308f2b>] _cpu_up+0x92/0xfc
[<ffffffff812facf7>] enable_nonboot_cpus+0x4d/0xb2
[<ffffffff81059836>] hibernation_snapshot+0x1f3/0x2a0
[<ffffffff81044406>] ? cleanup_srcu_struct+0x52/0x58
[<ffffffff81059e6d>] hibernate+0x97/0x196
[<ffffffff81057eb0>] state_store+0x5c/0x106
[<ffffffff811af49b>] kobj_attr_store+0x17/0x19
[<ffffffff811368fa>] sysfs_write_file+0x10e/0x14a
[<ffffffff810eb5f6>] vfs_write+0xab/0xd2
[<ffffffff810ec955>] ? fget_light+0x3a/0xa1
[<ffffffff810eb7ad>] sys_write+0x4d/0x74
[<ffffffff81314ce9>] system_call_fastpath+0x16/0x1b
INFO: rcu_preempt detected stalls on CPUs/tasks: { 1} (detected by 0, t=314193 jiffies)
Pid: 662, comm: bash Not tainted 3.3.0-rc2+ #293
Call Trace:
<IRQ> [<ffffffff8108b6c9>] __rcu_pending+0x268/0x3c5
[<ffffffff8108bf33>] rcu_check_callbacks+0xe9/0x108
[<ffffffff81031208>] update_process_times+0x3f/0x75
[<ffffffff810649b5>] tick_sched_timer+0x6d/0x8c
[<ffffffff81042e68>] __run_hrtimer+0xc9/0x15c
[<ffffffff81064948>] ? tick_nohz_handler+0xd5/0xd5
[<ffffffff8104359b>] hrtimer_interrupt+0xea/0x1b4
[<ffffffff81316192>] smp_apic_timer_interrupt+0x76/0x89
[<ffffffff81315833>] apic_timer_interrupt+0x73/0x80
<EOI> [<ffffffff8130b9ec>] ? arch_local_irq_restore+0x6/0xd
[<ffffffff8130840c>] native_cpu_up+0x108/0x127
[<ffffffff81308f2b>] _cpu_up+0x92/0xfc
[<ffffffff812facf7>] enable_nonboot_cpus+0x4d/0xb2
[<ffffffff81059836>] hibernation_snapshot+0x1f3/0x2a0
[<ffffffff81044406>] ? cleanup_srcu_struct+0x52/0x58
[<ffffffff81059e6d>] hibernate+0x97/0x196
[<ffffffff81057eb0>] state_store+0x5c/0x106
[<ffffffff811af49b>] kobj_attr_store+0x17/0x19
[<ffffffff811368fa>] sysfs_write_file+0x10e/0x14a
[<ffffffff810eb5f6>] vfs_write+0xab/0xd2
[<ffffffff810ec955>] ? fget_light+0x3a/0xa1
[<ffffffff810eb7ad>] sys_write+0x4d/0x74
[<ffffffff81314ce9>] system_call_fastpath+0x16/0x1b
INFO: rcu_sched detected stalls on CPUs/tasks: { 1} (detected by 0, t=1713579840232 jiffies)
Pid: 662, comm: bash Not tainted 3.3.0-rc2+ #293
Call Trace:
<IRQ> [<ffffffff8108b6c9>] __rcu_pending+0x268/0x3c5
[<ffffffff8108bef5>] rcu_check_callbacks+0xab/0x108
[<ffffffff81031208>] update_process_times+0x3f/0x75
[<ffffffff810649b5>] tick_sched_timer+0x6d/0x8c
[<ffffffff81042e68>] __run_hrtimer+0xc9/0x15c
[<ffffffff81064948>] ? tick_nohz_handler+0xd5/0xd5
[<ffffffff8104359b>] hrtimer_interrupt+0xea/0x1b4
[<ffffffff81316192>] smp_apic_timer_interrupt+0x76/0x89
[<ffffffff81315833>] apic_timer_interrupt+0x73/0x80
<EOI> [<ffffffff8130b9ec>] ? arch_local_irq_restore+0x6/0xd
[<ffffffff8130840c>] native_cpu_up+0x108/0x127
[<ffffffff81308f2b>] _cpu_up+0x92/0xfc
[<ffffffff812facf7>] enable_nonboot_cpus+0x4d/0xb2
[<ffffffff81059836>] hibernation_snapshot+0x1f3/0x2a0
[<ffffffff81044406>] ? cleanup_srcu_struct+0x52/0x58
[<ffffffff81059e6d>] hibernate+0x97/0x196
[<ffffffff81057eb0>] state_store+0x5c/0x106
[<ffffffff811af49b>] kobj_attr_store+0x17/0x19
[<ffffffff811368fa>] sysfs_write_file+0x10e/0x14a
[<ffffffff810eb5f6>] vfs_write+0xab/0xd2
[<ffffffff810ec955>] ? fget_light+0x3a/0xa1
[<ffffffff810eb7ad>] sys_write+0x4d/0x74
[<ffffffff81314ce9>] system_call_fastpath+0x16/0x1b
INFO: rcu_preempt detected stalls on CPUs/tasks: { 1} (detected by 0, t=1713579840233 jiffies)
Pid: 662, comm: bash Not tainted 3.3.0-rc2+ #293
Call Trace:
<IRQ> [<ffffffff8108b6c9>] __rcu_pending+0x268/0x3c5
[<ffffffff8108bf33>] rcu_check_callbacks+0xe9/0x108
[<ffffffff81031208>] update_process_times+0x3f/0x75
[<ffffffff810649b5>] tick_sched_timer+0x6d/0x8c
[<ffffffff81042e68>] __run_hrtimer+0xc9/0x15c
[<ffffffff81064948>] ? tick_nohz_handler+0xd5/0xd5
[<ffffffff8104359b>] hrtimer_interrupt+0xea/0x1b4
[<ffffffff81316192>] smp_apic_timer_interrupt+0x76/0x89
[<ffffffff81315833>] apic_timer_interrupt+0x73/0x80
<EOI> [<ffffffff8130b9ec>] ? arch_local_irq_restore+0x6/0xd
[<ffffffff8130840c>] native_cpu_up+0x108/0x127
[<ffffffff81308f2b>] _cpu_up+0x92/0xfc
[<ffffffff812facf7>] enable_nonboot_cpus+0x4d/0xb2
[<ffffffff81059836>] hibernation_snapshot+0x1f3/0x2a0
[<ffffffff81044406>] ? cleanup_srcu_struct+0x52/0x58
[<ffffffff81059e6d>] hibernate+0x97/0x196
[<ffffffff81057eb0>] state_store+0x5c/0x106
[<ffffffff811af49b>] kobj_attr_store+0x17/0x19
[<ffffffff811368fa>] sysfs_write_file+0x10e/0x14a
[<ffffffff810eb5f6>] vfs_write+0xab/0xd2
[<ffffffff810ec955>] ? fget_light+0x3a/0xa1
[<ffffffff810eb7ad>] sys_write+0x4d/0x74
[<ffffffff81314ce9>] system_call_fastpath+0x16/0x1b
Amit
next prev parent reply other threads:[~2012-02-09 12:28 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-02-07 21:05 x86: kvmclock: abstract save/restore sched_clock_state Marcelo Tosatti
2012-02-08 10:53 ` Igor Mammedov
2012-02-09 12:27 ` Amit Shah [this message]
2012-02-09 15:13 ` Igor Mammedov
2012-02-10 10:02 ` Amit Shah
2012-02-10 10:11 ` Igor Mammedov
2012-02-10 10:23 ` Amit Shah
2012-02-10 12:32 ` Marcelo Tosatti
2012-02-10 12:33 ` Marcelo Tosatti
2012-02-10 12:43 ` Igor Mammedov
2012-02-13 12:46 ` Amit Shah
2012-02-13 12:56 ` Amit Shah
2012-02-10 13:18 ` Amit Shah
2012-02-10 20:58 ` Igor Mammedov
2012-02-13 12:39 ` Amit Shah
2012-02-13 13:07 ` x86: kvmclock: abstract save/restore sched_clock_state (v2) Marcelo Tosatti
2012-02-13 15:20 ` Igor Mammedov
2012-02-13 15:52 ` Marcelo Tosatti
2012-02-15 10:28 ` Avi Kivity
2012-02-13 15:45 ` [PATCH RFC] pvclock: Make pv_clock more robust and fixup it if overflow happens Igor Mammedov
2012-02-13 17:48 ` Marcelo Tosatti
2012-02-13 18:15 ` Igor Mammedov
2012-02-13 16:26 ` x86: kvmclock: abstract save/restore sched_clock_state (v2) Amit Shah
2012-03-01 9:58 ` x86: kvmclock: abstract save/restore sched_clock_state Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120209122742.GA25505@amit.redhat.com \
--to=amit.shah@redhat.com \
--cc=avi@redhat.com \
--cc=hpa@zytor.com \
--cc=imammedo@redhat.com \
--cc=johnstul@us.ibm.com \
--cc=kvm@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=mtosatti@redhat.com \
--cc=riel@redhat.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).