Re: kernel BUG at arch/x86/kvm/mmu.c:655!

[Marcelo Tosatti <mtosatti@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3234 bytes --]

On Fri, Jan 07, 2011 at 09:43:57PM +0100, Tomasz Chmielewski wrote:
> The following happened when I tried to reboot a virtual machine (host running qemu 0.13.0, kernel 2.6.36.2).
> After a while, the server hanged and was no longer reachable.
> 
> kvm: 3927: cpu0 unhandled wrmsr: 0x198 data 0
> kvm: 3927: cpu1 unhandled wrmsr: 0x198 data 0
> rmap_remove:  ffff88060e9437f8 0 1->BUG
> ------------[ cut here ]------------
> kernel BUG at arch/x86/kvm/mmu.c:655!
> invalid opcode: 0000 [#1] SMP 
> last sysfs file: /sys/kernel/uevent_seqnum
> CPU 0 
> Modules linked in: ipt_MASQUERADE vhost_net kvm_intel kvm iptable_filter xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables bridge stp coretemp f71882fg snd_pcm snd_timer tpm_tis snd tpm soundcore tpm_bios snd_page_alloc i2c_i801 i7core_edac pcspkr edac_core shpchp r8169 mii raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx raid1 raid0 ahci libahci sata_nv sata_sil sata_via 3w_9xxx 3w_xxxx [last unloaded: scsi_wait_scan]
> 
> Pid: 3927, comm: kvm Not tainted 2.6.36.2 #1 MSI X58 Pro-E (MS-7522)/MS-7522
> RIP: 0010:[<ffffffffa024261c>]  [<ffffffffa024261c>] drop_spte+0x1ec/0x1f0 [kvm]
> RSP: 0018:ffff88061414bb58  EFLAGS: 00010292
> RAX: 000000000000002e RBX: ffff88060e9437f8 RCX: 00000000ffffffff
> RDX: 0000000000000000 RSI: 0000000000000082 RDI: 0000000000000246
> RBP: ffff88061414bb68 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000001 R11: 0000000000000003 R12: ffff880520e60000
> R13: ffff8804f4514820 R14: ffff880520e60000 R15: 0000000000000000
> FS:  00007f049b859730(0000) GS:ffff880001e00000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007f7f1dc2b000 CR3: 00000004f4464000 CR4: 00000000000026e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process kvm (pid: 3927, threadinfo ffff88061414a000, task ffff8806107596d0)
> Stack:
>  ffff88060e9437f8 00000000000000ff ffff88061414bba8 ffffffffa02427ba
> <0> ffff88061414bbb8 ffff8806140fb140 ffff880520e60000 ffff880520e62320
> <0> ffff88061414bbb8 0000000000000001 ffff88061414bbe8 ffffffffa024334e
> Call Trace:
>  [<ffffffffa02427ba>] kvm_mmu_prepare_zap_page+0x8a/0x2f0 [kvm]
>  [<ffffffffa024334e>] kvm_mmu_zap_all+0x4e/0x90 [kvm]
>  [<ffffffffa0236026>] kvm_arch_flush_shadow+0x16/0x30 [kvm]
>  [<ffffffffa022af20>] __kvm_set_memory_region+0x2e0/0x7e0 [kvm]
>  [<ffffffffa0239dbd>] ? kvm_set_msr_common+0x4fd/0xd40 [kvm]
>  [<ffffffffa024fdc7>] ? pic_unlock+0x27/0xc0 [kvm]
>  [<ffffffffa022b461>] kvm_set_memory_region+0x41/0x70 [kvm]
>  [<ffffffffa022b4ad>] kvm_vm_ioctl_set_memory_region+0x1d/0x30 [kvm]
>  [<ffffffffa022ba7d>] kvm_vm_ioctl+0x3fd/0x440 [kvm]
>  [<ffffffff8106c1b0>] ? do_send_sig_info+0x70/0x90
>  [<ffffffff8124dd36>] ? security_task_kill+0x16/0x20
>  [<ffffffff8114e60f>] do_vfs_ioctl+0x9f/0x550
>  [<ffffffff8108db49>] ? sys_futex+0x89/0x160
>  [<ffffffff8114eb61>] sys_ioctl+0xa1/0xb0
>  [<ffffffff8113e788>] ? sys_read+0x88/0x90
>  [<ffffffff8100a332>] system_call_fastpath+0x16/0x1b

Tomasz,

This should be fixed by commit eb45fda45f915c7, attached.


[-- Attachment #2: kvm-fix-drop-spte.patch --]
[-- Type: text/plain, Size: 1554 bytes --]

commit eb45fda45f915c7ca3e81e005e853cb770da2642
Author: Marcelo Tosatti <mtosatti@redhat.com>
Date:   Mon Oct 25 11:58:22 2010 -0200

    KVM: MMU: fix rmap_remove on non present sptes
    
    drop_spte should not attempt to rmap_remove a non present shadow pte.
    
    This fixes a BUG_ON seen on kvm-autotest.
    
    Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
    Reported-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
    Signed-off-by: Avi Kivity <avi@redhat.com>

diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 908ea54..fb8b376 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -720,7 +720,7 @@ static void rmap_remove(struct kvm *kvm, u64 *spte)
 	}
 }
 
-static void set_spte_track_bits(u64 *sptep, u64 new_spte)
+static int set_spte_track_bits(u64 *sptep, u64 new_spte)
 {
 	pfn_t pfn;
 	u64 old_spte = *sptep;
@@ -731,19 +731,20 @@ static void set_spte_track_bits(u64 *sptep, u64 new_spte)
 		old_spte = __xchg_spte(sptep, new_spte);
 
 	if (!is_rmap_spte(old_spte))
-		return;
+		return 0;
 
 	pfn = spte_to_pfn(old_spte);
 	if (!shadow_accessed_mask || old_spte & shadow_accessed_mask)
 		kvm_set_pfn_accessed(pfn);
 	if (!shadow_dirty_mask || (old_spte & shadow_dirty_mask))
 		kvm_set_pfn_dirty(pfn);
+	return 1;
 }
 
 static void drop_spte(struct kvm *kvm, u64 *sptep, u64 new_spte)
 {
-	set_spte_track_bits(sptep, new_spte);
-	rmap_remove(kvm, sptep);
+	if (set_spte_track_bits(sptep, new_spte))
+		rmap_remove(kvm, sptep);
 }
 
 static u64 *rmap_next(struct kvm *kvm, unsigned long *rmapp, u64 *spte)