* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ [not found] ` <20151210162733.GA6587@pd.tnic> @ 2015-12-10 16:44 ` Borislav Petkov 2015-12-10 16:49 ` Paolo Bonzini 0 siblings, 1 reply; 4+ messages in thread From: Borislav Petkov @ 2015-12-10 16:44 UTC (permalink / raw) To: kvm; +Cc: Paolo Bonzini, lkml, qemu-devel Yap, this is clearly a qemu/kvm issue. Lemme remove ext4 folks from CC. So here's what happens: I boot a kvm guest, connect to its monitor (qemu is started with "-monitor pty") and on the monitor I issue a couple of times the "nmi" command. It doesn't explode immediately but it happens pretty often and when it happens, the *host*(!) freezes with some nasty corruption, see below. Thoughts, suggestions, ideas? ... [ 13.564192] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 111.524485] kvm: zapping shadow pages for mmio generation wraparound [ 166.414836] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null) [ 172.408126] kvm: zapping shadow pages for mmio generation wraparound [ 200.912827] BUG: unable to handle kernel NULL pointer dereference at (null) [ 200.920716] IP: [< (null)>] (null) [ 200.925796] PGD 0 [ 200.927833] Oops: 0010 [#1] PREEMPT SMP [ 200.931258] Modules linked in: binfmt_misc ipv6 vfat fat fuse dm_crypt dm_mod kvm_amd kvm irqq [ 200.931274] CPU: 0 PID: 3936 Comm: qemu-system-x86 Not tainted 4.4.0-rc4+ #1 [ 200.931275] Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 EVO R2.0, BIOS3 [ 200.931276] task: ffff880037ba0000 ti: ffff8804227d8000 task.ti: ffff8804227d8000 [ 200.931277] RIP: 0010:[<0000000000000000>] [< (null)>] (null) [ 200.931278] RSP: 0018:ffff8804227dbcb0 EFLAGS: 00010292 [ 200.931278] RAX: ffff880037ba0000 RBX: 0000000000000292 RCX: 0000000000000003 [ 200.931279] RDX: ffff880037ba0750 RSI: 0000000000000000 RDI: ffff880037ba0001 [ 200.931280] RBP: ffff8804227dbd08 R08: 0000000000000001 R09: 00000011f89b6000 [ 200.931280] R10: ffffffff825c4fd0 R11: 0000000000000000 R12: 0000000000000000 [ 200.931281] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000 [ 200.931282] FS: 00007fec0faba700(0000) GS:ffff88042c600000(0000) knlGS:0000000000000000 [ 200.931283] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 200.931283] CR2: 0000000000000000 CR3: 00000000b6566000 CR4: 00000000000406f0 [ 200.931283] Stack: [ 200.931285] ffffffff8121004e ffff880400000000 ffffffff00000000 0000000000000292 [ 200.931287] 00000000a0305d3c ffff880414612280 ffff880414612210 ffff8804227dbe90 [ 200.931288] ffff880414612130 ffff880037ba0000 ffffffff8121004e ffff8804227dbd90 [ 200.931288] Call Trace: [ 200.931293] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470 [ 200.931295] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470 [ 200.931298] [<ffffffff816de106>] mutex_lock_nested+0x66/0x3d0 [ 200.931299] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470 [ 200.931301] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470 [ 200.931303] [<ffffffff810a916a>] ? __lock_acquire+0x55a/0x1f80 [ 200.931305] [<ffffffff8121004e>] ext4_file_write_iter+0x8e/0x470 [ 200.931307] [<ffffffff811ac78e>] ? __fget+0x11e/0x210 [ 200.931309] [<ffffffff810a2cb4>] ? percpu_down_read+0x44/0x90 [ 200.931311] [<ffffffff8118d47d>] __vfs_write+0xad/0xe0 [ 200.931313] [<ffffffff8118dcd5>] vfs_write+0xb5/0x1a0 [ 200.931315] [<ffffffff8118e912>] SyS_write+0x52/0xc0 [ 200.931317] [<ffffffff816e2d5b>] entry_SYSCALL_64_fastpath+0x16/0x6f [ 200.931319] Code: Bad RIP value. [ 200.931320] RIP [< (null)>] (null) [ 200.931321] RSP <ffff8804227dbcb0> [ 200.931321] CR2: 0000000000000000 [ 200.939353] ---[ end trace 70675f54590d7755 ]--- [ 200.939370] note: qemu-system-x86[3936] exited with preempt_count 1 -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. -- ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ 2015-12-10 16:44 ` [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ Borislav Petkov @ 2015-12-10 16:49 ` Paolo Bonzini 2015-12-10 16:53 ` Borislav Petkov 0 siblings, 1 reply; 4+ messages in thread From: Paolo Bonzini @ 2015-12-10 16:49 UTC (permalink / raw) To: Borislav Petkov, kvm; +Cc: lkml, qemu-devel On 10/12/2015 17:44, Borislav Petkov wrote: > Yap, > > this is clearly a qemu/kvm issue. Lemme remove ext4 folks from CC. So > here's what happens: > > I boot a kvm guest, connect to its monitor (qemu is started with > "-monitor pty") and on the monitor I issue a couple of times the "nmi" > command. It doesn't explode immediately but it happens pretty often and > when it happens, the *host*(!) freezes with some nasty corruption, see > below. > > Thoughts, suggestions, ideas? Can you try it on Intel? Paolo ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ 2015-12-10 16:49 ` Paolo Bonzini @ 2015-12-10 16:53 ` Borislav Petkov 2015-12-10 17:34 ` Paolo Bonzini 0 siblings, 1 reply; 4+ messages in thread From: Borislav Petkov @ 2015-12-10 16:53 UTC (permalink / raw) To: Paolo Bonzini; +Cc: lkml, kvm, qemu-devel On Thu, Dec 10, 2015 at 05:49:10PM +0100, Paolo Bonzini wrote: > Can you try it on Intel? Just did, there it splats even when booting the guest, without even injecting NMIs: [ 113.233992] =============================== [ 113.238192] [ INFO: suspicious RCU usage. ] [ 113.242393] 4.4.0-rc4+ #1 Not tainted [ 113.246056] ------------------------------- [ 113.250257] arch/x86/kvm/trace.h:29 suspicious rcu_dereference_check() usage! [ 113.257400] other info that might help us debug this: [ 113.265432] RCU used illegally from idle CPU! rcu_scheduler_active = 1, debug_locks = 0 [ 113.276321] RCU used illegally from extended quiescent state! [ 113.282083] 1 lock held by qemu-system-x86/2275: [ 113.286711] #0: (&vcpu->mutex){+.+.+.}, at: [<ffffffffa01d472c>] vcpu_load+0x1c/0x80 [kvm] [ 113.295270] stack backtrace: [ 113.299644] CPU: 4 PID: 2275 Comm: qemu-system-x86 Not tainted 4.4.0-rc4+ #1 [ 113.306706] Hardware name: Dell Inc. Precision T3600/0PTTT9, BIOS A13 05/11/2014 [ 113.314122] 0000000000000001 ffff88042f263d28 ffffffff813c2cfc ffff880432d53000 [ 113.321575] ffff88042f263d58 ffffffff810c5157 ffff88042f268000 0000000000000000 [ 113.329027] 0000000000000000 0000000000000001 ffff88042f263e10 ffffffffa01ee7c8 [ 113.336483] Call Trace: [ 113.338939] [<ffffffff813c2cfc>] dump_stack+0x4e/0x82 [ 113.344098] [<ffffffff810c5157>] lockdep_rcu_suspicious+0xe7/0x120 [ 113.350385] [<ffffffffa01ee7c8>] kvm_arch_vcpu_ioctl_run+0x16f8/0x19c0 [kvm] [ 113.357544] [<ffffffffa01ed1a0>] ? kvm_arch_vcpu_ioctl_run+0xd0/0x19c0 [kvm] [ 113.364695] [<ffffffffa01d4b32>] kvm_vcpu_ioctl+0x342/0x700 [kvm] [ 113.370896] [<ffffffff810c4a7d>] ? __lock_is_held+0x4d/0x70 [ 113.376583] [<ffffffff812351ae>] ? __fget+0xfe/0x200 [ 113.381662] [<ffffffff812291f1>] do_vfs_ioctl+0x301/0x550 [ 113.387156] [<ffffffff8123531a>] ? __fget_light+0x2a/0x90 [ 113.392654] [<ffffffff81229481>] SyS_ioctl+0x41/0x70 [ 113.397739] [<ffffffff8185cb36>] entry_SYSCALL_64_fastpath+0x16/0x7a [ 113.755008] kvm: zapping shadow pages for mmio generation wraparound -- Regards/Gruss, Boris. ECO tip #101: Trim your mails when you reply. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ 2015-12-10 16:53 ` Borislav Petkov @ 2015-12-10 17:34 ` Paolo Bonzini 0 siblings, 0 replies; 4+ messages in thread From: Paolo Bonzini @ 2015-12-10 17:34 UTC (permalink / raw) To: Borislav Petkov; +Cc: lkml, kvm, qemu-devel On 10/12/2015 17:53, Borislav Petkov wrote: > Just did, there it splats even when booting the guest, without even > injecting NMIs: > > [ 113.233992] =============================== > [ 113.238192] [ INFO: suspicious RCU usage. ] > [ 113.242393] 4.4.0-rc4+ #1 Not tainted > [ 113.246056] ------------------------------- > [ 113.250257] arch/x86/kvm/trace.h:29 suspicious rcu_dereference_check() usage! > [ 113.257400] > other info that might help us debug this: > > [ 113.265432] > RCU used illegally from idle CPU! > rcu_scheduler_active = 1, debug_locks = 0 > [ 113.276321] RCU used illegally from extended quiescent state! > [ 113.282083] 1 lock held by qemu-system-x86/2275: > [ 113.286711] #0: (&vcpu->mutex){+.+.+.}, at: [<ffffffffa01d472c>] vcpu_load+0x1c/0x80 [kvm] > [ 113.295270] > stack backtrace: > [ 113.299644] CPU: 4 PID: 2275 Comm: qemu-system-x86 Not tainted 4.4.0-rc4+ #1 > [ 113.306706] Hardware name: Dell Inc. Precision T3600/0PTTT9, BIOS A13 05/11/2014 > [ 113.314122] 0000000000000001 ffff88042f263d28 ffffffff813c2cfc ffff880432d53000 > [ 113.321575] ffff88042f263d58 ffffffff810c5157 ffff88042f268000 0000000000000000 > [ 113.329027] 0000000000000000 0000000000000001 ffff88042f263e10 ffffffffa01ee7c8 > [ 113.336483] Call Trace: > [ 113.338939] [<ffffffff813c2cfc>] dump_stack+0x4e/0x82 > [ 113.344098] [<ffffffff810c5157>] lockdep_rcu_suspicious+0xe7/0x120 > [ 113.350385] [<ffffffffa01ee7c8>] kvm_arch_vcpu_ioctl_run+0x16f8/0x19c0 [kvm] > [ 113.357544] [<ffffffffa01ed1a0>] ? kvm_arch_vcpu_ioctl_run+0xd0/0x19c0 [kvm] > [ 113.364695] [<ffffffffa01d4b32>] kvm_vcpu_ioctl+0x342/0x700 [kvm] > [ 113.370896] [<ffffffff810c4a7d>] ? __lock_is_held+0x4d/0x70 > [ 113.376583] [<ffffffff812351ae>] ? __fget+0xfe/0x200 > [ 113.381662] [<ffffffff812291f1>] do_vfs_ioctl+0x301/0x550 > [ 113.387156] [<ffffffff8123531a>] ? __fget_light+0x2a/0x90 > [ 113.392654] [<ffffffff81229481>] SyS_ioctl+0x41/0x70 > [ 113.397739] [<ffffffff8185cb36>] entry_SYSCALL_64_fastpath+0x16/0x7a > [ 113.755008] kvm: zapping shadow pages for mmio generation wraparound Wow, this must have been there forever... Paolo ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-12-10 17:34 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20151210140404.GA3049@nazgul.tnic>
[not found] ` <20151210162733.GA6587@pd.tnic>
2015-12-10 16:44 ` [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ Borislav Petkov
2015-12-10 16:49 ` Paolo Bonzini
2015-12-10 16:53 ` Borislav Petkov
2015-12-10 17:34 ` Paolo Bonzini
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).