* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+
[not found] ` <20151210162733.GA6587@pd.tnic>
@ 2015-12-10 16:44 ` Borislav Petkov
2015-12-10 16:49 ` Paolo Bonzini
0 siblings, 1 reply; 4+ messages in thread
From: Borislav Petkov @ 2015-12-10 16:44 UTC (permalink / raw)
To: kvm; +Cc: Paolo Bonzini, lkml, qemu-devel
Yap,
this is clearly a qemu/kvm issue. Lemme remove ext4 folks from CC. So
here's what happens:
I boot a kvm guest, connect to its monitor (qemu is started with
"-monitor pty") and on the monitor I issue a couple of times the "nmi"
command. It doesn't explode immediately but it happens pretty often and
when it happens, the *host*(!) freezes with some nasty corruption, see
below.
Thoughts, suggestions, ideas?
...
[ 13.564192] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 111.524485] kvm: zapping shadow pages for mmio generation wraparound
[ 166.414836] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
[ 172.408126] kvm: zapping shadow pages for mmio generation wraparound
[ 200.912827] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 200.920716] IP: [< (null)>] (null)
[ 200.925796] PGD 0
[ 200.927833] Oops: 0010 [#1] PREEMPT SMP
[ 200.931258] Modules linked in: binfmt_misc ipv6 vfat fat fuse dm_crypt dm_mod kvm_amd kvm irqq
[ 200.931274] CPU: 0 PID: 3936 Comm: qemu-system-x86 Not tainted 4.4.0-rc4+ #1
[ 200.931275] Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 EVO R2.0, BIOS3
[ 200.931276] task: ffff880037ba0000 ti: ffff8804227d8000 task.ti: ffff8804227d8000
[ 200.931277] RIP: 0010:[<0000000000000000>] [< (null)>] (null)
[ 200.931278] RSP: 0018:ffff8804227dbcb0 EFLAGS: 00010292
[ 200.931278] RAX: ffff880037ba0000 RBX: 0000000000000292 RCX: 0000000000000003
[ 200.931279] RDX: ffff880037ba0750 RSI: 0000000000000000 RDI: ffff880037ba0001
[ 200.931280] RBP: ffff8804227dbd08 R08: 0000000000000001 R09: 00000011f89b6000
[ 200.931280] R10: ffffffff825c4fd0 R11: 0000000000000000 R12: 0000000000000000
[ 200.931281] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000
[ 200.931282] FS: 00007fec0faba700(0000) GS:ffff88042c600000(0000) knlGS:0000000000000000
[ 200.931283] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 200.931283] CR2: 0000000000000000 CR3: 00000000b6566000 CR4: 00000000000406f0
[ 200.931283] Stack:
[ 200.931285] ffffffff8121004e ffff880400000000 ffffffff00000000 0000000000000292
[ 200.931287] 00000000a0305d3c ffff880414612280 ffff880414612210 ffff8804227dbe90
[ 200.931288] ffff880414612130 ffff880037ba0000 ffffffff8121004e ffff8804227dbd90
[ 200.931288] Call Trace:
[ 200.931293] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470
[ 200.931295] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470
[ 200.931298] [<ffffffff816de106>] mutex_lock_nested+0x66/0x3d0
[ 200.931299] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470
[ 200.931301] [<ffffffff8121004e>] ? ext4_file_write_iter+0x8e/0x470
[ 200.931303] [<ffffffff810a916a>] ? __lock_acquire+0x55a/0x1f80
[ 200.931305] [<ffffffff8121004e>] ext4_file_write_iter+0x8e/0x470
[ 200.931307] [<ffffffff811ac78e>] ? __fget+0x11e/0x210
[ 200.931309] [<ffffffff810a2cb4>] ? percpu_down_read+0x44/0x90
[ 200.931311] [<ffffffff8118d47d>] __vfs_write+0xad/0xe0
[ 200.931313] [<ffffffff8118dcd5>] vfs_write+0xb5/0x1a0
[ 200.931315] [<ffffffff8118e912>] SyS_write+0x52/0xc0
[ 200.931317] [<ffffffff816e2d5b>] entry_SYSCALL_64_fastpath+0x16/0x6f
[ 200.931319] Code: Bad RIP value.
[ 200.931320] RIP [< (null)>] (null)
[ 200.931321] RSP <ffff8804227dbcb0>
[ 200.931321] CR2: 0000000000000000
[ 200.939353] ---[ end trace 70675f54590d7755 ]---
[ 200.939370] note: qemu-system-x86[3936] exited with preempt_count 1
--
Regards/Gruss,
Boris.
ECO tip #101: Trim your mails when you reply.
--
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+
2015-12-10 16:44 ` [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ Borislav Petkov
@ 2015-12-10 16:49 ` Paolo Bonzini
2015-12-10 16:53 ` Borislav Petkov
0 siblings, 1 reply; 4+ messages in thread
From: Paolo Bonzini @ 2015-12-10 16:49 UTC (permalink / raw)
To: Borislav Petkov, kvm; +Cc: lkml, qemu-devel
On 10/12/2015 17:44, Borislav Petkov wrote:
> Yap,
>
> this is clearly a qemu/kvm issue. Lemme remove ext4 folks from CC. So
> here's what happens:
>
> I boot a kvm guest, connect to its monitor (qemu is started with
> "-monitor pty") and on the monitor I issue a couple of times the "nmi"
> command. It doesn't explode immediately but it happens pretty often and
> when it happens, the *host*(!) freezes with some nasty corruption, see
> below.
>
> Thoughts, suggestions, ideas?
Can you try it on Intel?
Paolo
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+
2015-12-10 16:49 ` Paolo Bonzini
@ 2015-12-10 16:53 ` Borislav Petkov
2015-12-10 17:34 ` Paolo Bonzini
0 siblings, 1 reply; 4+ messages in thread
From: Borislav Petkov @ 2015-12-10 16:53 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: lkml, kvm, qemu-devel
On Thu, Dec 10, 2015 at 05:49:10PM +0100, Paolo Bonzini wrote:
> Can you try it on Intel?
Just did, there it splats even when booting the guest, without even
injecting NMIs:
[ 113.233992] ===============================
[ 113.238192] [ INFO: suspicious RCU usage. ]
[ 113.242393] 4.4.0-rc4+ #1 Not tainted
[ 113.246056] -------------------------------
[ 113.250257] arch/x86/kvm/trace.h:29 suspicious rcu_dereference_check() usage!
[ 113.257400]
other info that might help us debug this:
[ 113.265432]
RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
[ 113.276321] RCU used illegally from extended quiescent state!
[ 113.282083] 1 lock held by qemu-system-x86/2275:
[ 113.286711] #0: (&vcpu->mutex){+.+.+.}, at: [<ffffffffa01d472c>] vcpu_load+0x1c/0x80 [kvm]
[ 113.295270]
stack backtrace:
[ 113.299644] CPU: 4 PID: 2275 Comm: qemu-system-x86 Not tainted 4.4.0-rc4+ #1
[ 113.306706] Hardware name: Dell Inc. Precision T3600/0PTTT9, BIOS A13 05/11/2014
[ 113.314122] 0000000000000001 ffff88042f263d28 ffffffff813c2cfc ffff880432d53000
[ 113.321575] ffff88042f263d58 ffffffff810c5157 ffff88042f268000 0000000000000000
[ 113.329027] 0000000000000000 0000000000000001 ffff88042f263e10 ffffffffa01ee7c8
[ 113.336483] Call Trace:
[ 113.338939] [<ffffffff813c2cfc>] dump_stack+0x4e/0x82
[ 113.344098] [<ffffffff810c5157>] lockdep_rcu_suspicious+0xe7/0x120
[ 113.350385] [<ffffffffa01ee7c8>] kvm_arch_vcpu_ioctl_run+0x16f8/0x19c0 [kvm]
[ 113.357544] [<ffffffffa01ed1a0>] ? kvm_arch_vcpu_ioctl_run+0xd0/0x19c0 [kvm]
[ 113.364695] [<ffffffffa01d4b32>] kvm_vcpu_ioctl+0x342/0x700 [kvm]
[ 113.370896] [<ffffffff810c4a7d>] ? __lock_is_held+0x4d/0x70
[ 113.376583] [<ffffffff812351ae>] ? __fget+0xfe/0x200
[ 113.381662] [<ffffffff812291f1>] do_vfs_ioctl+0x301/0x550
[ 113.387156] [<ffffffff8123531a>] ? __fget_light+0x2a/0x90
[ 113.392654] [<ffffffff81229481>] SyS_ioctl+0x41/0x70
[ 113.397739] [<ffffffff8185cb36>] entry_SYSCALL_64_fastpath+0x16/0x7a
[ 113.755008] kvm: zapping shadow pages for mmio generation wraparound
--
Regards/Gruss,
Boris.
ECO tip #101: Trim your mails when you reply.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+
2015-12-10 16:53 ` Borislav Petkov
@ 2015-12-10 17:34 ` Paolo Bonzini
0 siblings, 0 replies; 4+ messages in thread
From: Paolo Bonzini @ 2015-12-10 17:34 UTC (permalink / raw)
To: Borislav Petkov; +Cc: lkml, kvm, qemu-devel
On 10/12/2015 17:53, Borislav Petkov wrote:
> Just did, there it splats even when booting the guest, without even
> injecting NMIs:
>
> [ 113.233992] ===============================
> [ 113.238192] [ INFO: suspicious RCU usage. ]
> [ 113.242393] 4.4.0-rc4+ #1 Not tainted
> [ 113.246056] -------------------------------
> [ 113.250257] arch/x86/kvm/trace.h:29 suspicious rcu_dereference_check() usage!
> [ 113.257400]
> other info that might help us debug this:
>
> [ 113.265432]
> RCU used illegally from idle CPU!
> rcu_scheduler_active = 1, debug_locks = 0
> [ 113.276321] RCU used illegally from extended quiescent state!
> [ 113.282083] 1 lock held by qemu-system-x86/2275:
> [ 113.286711] #0: (&vcpu->mutex){+.+.+.}, at: [<ffffffffa01d472c>] vcpu_load+0x1c/0x80 [kvm]
> [ 113.295270]
> stack backtrace:
> [ 113.299644] CPU: 4 PID: 2275 Comm: qemu-system-x86 Not tainted 4.4.0-rc4+ #1
> [ 113.306706] Hardware name: Dell Inc. Precision T3600/0PTTT9, BIOS A13 05/11/2014
> [ 113.314122] 0000000000000001 ffff88042f263d28 ffffffff813c2cfc ffff880432d53000
> [ 113.321575] ffff88042f263d58 ffffffff810c5157 ffff88042f268000 0000000000000000
> [ 113.329027] 0000000000000000 0000000000000001 ffff88042f263e10 ffffffffa01ee7c8
> [ 113.336483] Call Trace:
> [ 113.338939] [<ffffffff813c2cfc>] dump_stack+0x4e/0x82
> [ 113.344098] [<ffffffff810c5157>] lockdep_rcu_suspicious+0xe7/0x120
> [ 113.350385] [<ffffffffa01ee7c8>] kvm_arch_vcpu_ioctl_run+0x16f8/0x19c0 [kvm]
> [ 113.357544] [<ffffffffa01ed1a0>] ? kvm_arch_vcpu_ioctl_run+0xd0/0x19c0 [kvm]
> [ 113.364695] [<ffffffffa01d4b32>] kvm_vcpu_ioctl+0x342/0x700 [kvm]
> [ 113.370896] [<ffffffff810c4a7d>] ? __lock_is_held+0x4d/0x70
> [ 113.376583] [<ffffffff812351ae>] ? __fget+0xfe/0x200
> [ 113.381662] [<ffffffff812291f1>] do_vfs_ioctl+0x301/0x550
> [ 113.387156] [<ffffffff8123531a>] ? __fget_light+0x2a/0x90
> [ 113.392654] [<ffffffff81229481>] SyS_ioctl+0x41/0x70
> [ 113.397739] [<ffffffff8185cb36>] entry_SYSCALL_64_fastpath+0x16/0x7a
> [ 113.755008] kvm: zapping shadow pages for mmio generation wraparound
Wow, this must have been there forever...
Paolo
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-12-10 17:34 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20151210140404.GA3049@nazgul.tnic>
[not found] ` <20151210162733.GA6587@pd.tnic>
2015-12-10 16:44 ` [Qemu-devel] freeze host when injecting NMIs in the guest, at least in 4.4-rc4+ Borislav Petkov
2015-12-10 16:49 ` Paolo Bonzini
2015-12-10 16:53 ` Borislav Petkov
2015-12-10 17:34 ` Paolo Bonzini
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).