Re: KVM guest crashes

[Alexander Graf <agraf@suse.de>]

Alexander Graf wrote:
> Alexander Graf wrote:
>   
>> Alexander Graf wrote:
>>
>> [...]
>>   
>>     
>>> Also after two days of permanent stress testing I also got the Intel
>>> machine w/ current git down:
>>>
>>> + sudo -u contain1 env -i /usr/local/bin/qemu-system-x86_64 -localtime
>>> -kernel virtio-kernel -initrd virtio-initrd -nographic -append 'quiet
>>> clocksource=acpi_pm cifsuser=contain1 cifspass=contain1
>>> root=cifs://contain1:contain1@172.16.1.1/contain1
>>> realroot=//172.16.1.1/users/contain1
>>> ip=172.16.1.2:172.16.1.1::255.255.255.0::eth0:none console=ttyS0
>>> dhcp=off builder=1' -net nic,model=virtio,macaddr=52:54:00:12:34:1 -net
>>> tap,ifname=tap1,script=/bin/true -m 2000 -nographic -smp 8 /dev/null
>>> qemu: loading initrd (0x1daf359 bytes) at 0x000000007b240000
>>> Stuck ??
>>>
>>> No backtrace here though. That's all I got from the serial console.
>>>   
>>>     
>>>       
>> + sudo -u contain1 env -i /usr/local/bin/qemu-system-x86_64 -localtime
>> -kernel virtio-kernel -initrd virtio-initrd -nographic -append 'quiet
>> clocksource=acpi_pm cifsuser=contain1 cifspass=contain1
>> root=cifs://contain1:contain1@172.16.1.1/contain1
>> realroot=//172.16.1.1/users/contain1
>> ip=172.16.1.2:172.16.1.1::255.255.255.0::eth0:none console=ttyS0
>> dhcp=off builder=1' -net nic,model=virtio,macaddr=52:54:00:12:34:1 -net
>> tap,ifname=tap1,script=/bin/true -m 2000 -nographic -smp 8 /dev/null
>> qemu: loading initrd (0x1daf359 bytes) at 0x000000007b240000
>> Stuck ??
>>     
[...]

In order to provide you with more dumps that might point to some
direction (I'm still lost on figuring where to look), here's another AMD
NPT guest crash with current git. It somehow looks as if the guest
pagetable is corrupted.

+ sudo -u contain3 env -i /usr/local/bin/qemu-system-x86_64 -localtime
-kernel virtio-kernel -initrd virtio-initrd -nographic -append 'quiet
clocksource=acpi_pm cifsuser=con
tain3 cifspass=contain3
root=cifs://contain3:contain3@172.16.3.1/contain3
realroot=//172.16.3.1/users/contain3
ip=172.16.3.2:172.16.3.1::255.255.255.0::eth0:none console=tty
S0 dhcp=off builder=1' -net nic,model=virtio,macaddr=52:54:00:12:34:3
-net tap,ifname=tap3,script=/bin/true -m 2000 -nographic -smp 8
-no-kvm-irqchip /dev/null
qemu: loading initrd (0x1daf359 bytes) at 0x000000007b240000
pci 0000:00:01.0: PIIX3: Enabling Passive Release
IP-Config: Device `eth0' not found.
doing fast boot
Creating device nodes with udev
^MBoot logging started on /dev/ttyS0(/dev/console) at Thu Jan 22
23:05:55 2009^M
[NETWORK] using static config based on
ip=172.16.3.2:172.16.3.1::255.255.255.0::eth0:none^M
Trying manual resume from /dev/disk/by-id/ata-ST380815AS_5RW3M74V-part1^M
resume device /dev/disk/by-id/ata-ST380815AS_5RW3M74V-part1 not found
(ignoring)^M
Trying manual resume from /dev/disk/by-id/ata-ST380815AS_5RW3M74V-part1^M
resume device /dev/disk/by-id/ata-ST380815AS_5RW3M74V-part1 not found
(ignoring)^M
node name not found^M
Mounting root //172.16.3.1/contain3^M
RTNETLINK answers: File exists^M
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN ^M
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00^M
    inet 127.0.0.1/8 scope host lo^M
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UNKNOWN qlen 1000^M
    link/ether 52:54:00:12:34:03 brd ff:ff:ff:ff:ff:ff^M
    inet 172.16.3.2 peer 172.16.3.1/24 scope global eth0^M
BUG: unable to handle kernel paging request at 0000000000100100
IP: [<ffffffff8036a603>] strnlen+0x10/0x19
PGD 7c596067 PUD 7c9ed067 PMD 0
Oops: 0000 [1] SMP
last sysfs file: /sys/kernel/uevent_seqnum
CPU 7
Modules linked in: nls_utf8 cifs(X) af_packet virtio_net virtio_pci
virtio_ring virtio edd ext3 mbcache jbd fan ide_pci_generic ide_core
ata_generic sata_nv libata scsi_mod
dock thermal processor thermal_sys hwmon
Supported: Yes, External
Pid: 782, comm: halt Tainted: G S        2.6.27.7-9-default #1
RIP: 0010:[<ffffffff8036a603>]  [<ffffffff8036a603>] strnlen+0x10/0x19
RSP: 0018:ffff88007c46da70  EFLAGS: 00010082
RAX: 0000000000100100 RBX: 0000000000000000 RCX: 00000000ffffffff
RDX: 0000000000100100 RSI: fffffffffffffffe RDI: 0000000000100100
RBP: ffffffff80ae0fad R08: 00000000ffffffff R09: 0000000000000000
R10: 000000000000000a R11: 0000000000000000 R12: 0000000000100100
R13: 00000000ffffffff R14: ffffffff80ae13a0 R15: 00000000ffffffff
FS:  00007f0b2aee06f0(0000) GS:ffff88007a57bf40(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000100100 CR3: 000000007c4e5000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process halt (pid: 782, threadinfo ffff88007c46c000, task ffff88007c17e0c0)
Stack:  ffffffff8036b39d ffff88007c46ddb8 ffffffff80ae0fad ffffffff805d7e29
 0000000000000000 00000000ffffffff ffffffff8036b6f6 00007f0b2ace27e0
 ffff88007c595ab0 ffff88007c0624a8 0000000000000400 ffffffff80ae0fa0
Call Trace:
 [<ffffffff8036b39d>] string+0x34/0x91
 [<ffffffff8036b6f6>] vsnprintf+0x2fc/0x574
 [<ffffffff8036ba56>] vscnprintf+0x9/0x17
 [<ffffffff80241a12>] vprintk+0x12b/0x2df
 [<ffffffff80240e2f>] warn_slowpath+0x9f/0xd1
 [<ffffffff80366da2>] kobject_put+0x2f/0x42
 [<ffffffff8024fe90>] kernel_power_off+0xe/0x3b
 [<ffffffff80250108>] sys_reboot+0xf8/0x179
 [<ffffffff8020c37a>] system_call_fastpath+0x16/0x1b
 [<00007f0b2aa3aa26>] 0x7f0b2aa3aa26


Code: d5 70 80 20 75 eb 48 89 f8 c3 48 89 f8 eb 03 48 ff c0 80 38 00 75
f8 48 29 f8 c3 48 89 f8 eb 03 48 ff c0 48 85 f6 74 08 48 ff ce <80> 38
00 75 f0 48 29 f8 c3 31 c0 eb
12 41 38 c8 74 0a 48 ff c2
RIP  [<ffffffff8036a603>] strnlen+0x10/0x19
 RSP <ffff88007c46da70>
CR2: 0000000000100100
---[ end trace 1c45144e9c9b5946 ]---
boot/84-builder.sh: line 30:   782 Killed                  halt -fp^M


Alex