Re: [Qemu-devel] E5-2620v2 - emulation stop error

Andrey Korolyov
2015-03-25T22:31:33Z

[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]

On Wed, Mar 25, 2015 at 11:54 PM, Kevin O'Connor <kevin@koconnor.net> wrote:
> On Wed, Mar 25, 2015 at 11:43:31PM +0300, Andrey Korolyov wrote:
>> On Mon, Mar 16, 2015 at 10:17 PM, Andrey Korolyov <andrey@xdel.ru> wrote:
>> > For now, it looks like bug have a mixed Murphy-Heisenberg nature, as
>> > it appearance is very rare (compared to the number of actual launches)
>> > and most probably bounded to the physical characteristics of my
>> > production nodes. As soon as I reach any reproducible path for a
>> > regular workstation environment, I`ll let everyone know. Also I am
>> > starting to think that issue can belong to the particular motherboard
>> > firmware revision, despite fact that the CPU microcode is the same
>> > everywhere.
>>
>>
>> Hello everyone, I`ve managed to reproduce this issue
>> *deterministically* with latest seabios with smp fix and 3.18.3. The
>> error occuring just *once* per vm until hypervisor reboots, at least
>> in my setup, this is definitely crazy...
>>
>> - launch two VMs (Centos 7 in my case),
>> - wait a little while they are booting,
>> - attach serial console (I am using virsh list for this exact purpose),
>> - issue acpi reboot or reset, does not matter,
>> - VM always hangs at boot, most times with sgabios initialization
>> string printed out [1], but sometimes it hangs a bit later [2],
>> - no matter how many times I try to relaunch the QEMU afterwards, the
>> issue does not appear on VM which experienced problem once;
>> - trace and sample args can be seen in [3] and [4] respectively.
>
> Can you add something like:
>
> -chardev file,path=seabioslog.`date +%s`,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios
>
> to the qemu command line and forward the resulting log from both a
> succesful boot and a failed one?
>
> -Kevin

Of course, logs are attached.

[-- Attachment #2: reboot.failed --]
[-- Type: application/octet-stream, Size: 6053 bytes --]

SeaBIOS (version 1.8.1-20150325_230423-testnode)
Running on QEMU (i440fx)
Running on KVM
RamSize: 0x40000000 [cmos]
Relocating init from 0x000e11b0 to 0x3ffafd10 (size 66096)
Found QEMU fw_cfg
RamBlock: addr 0x0000000000000000 len 0x0000000040000000 [e820]
Moving pm_base to 0x600
boot order:
1: /pci@i0cf8/scsi@5/disk@0,0
2: HALT
CPU Mhz=2103
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 7 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c06f
PCI: 32: 0000000080000000 - 00000000fec00000
PCI: map device bdf=00:05.0 bar 0, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:04.0 bar 0, addr 0000c040, size 00000020 [io]
PCI: map device bdf=00:01.1 bar 4, addr 0000c060, size 00000010 [io]
PCI: map device bdf=00:03.0 bar 0, addr febf8000, size 00004000 [mem]
PCI: map device bdf=00:04.0 bar 1, addr febfc000, size 00001000 [mem]
PCI: map device bdf=00:05.0 bar 1, addr febfd000, size 00001000 [mem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0x608
PCI: init bdf=00:03.0 id=1033:0194
PCI: init bdf=00:04.0 id=1af4:1003
PCI: init bdf=00:05.0 id=1af4:1001
PCI: No VGA devices found
handle_smp: apic_id=4
handle_smp: apic_id=11
handle_smp: apic_id=3
handle_smp: apic_id=6
handle_smp: apic_id=2
handle_smp: apic_id=8
handle_smp: apic_id=5
handle_smp: apic_id=9
handle_smp: apic_id=1
handle_smp: apic_id=7
handle_smp: apic_id=10
Found 12 cpu(s) max supported 12 cpu(s)
Copying PIR from 0x3ffbfd08 to 0x000f1360
Copying MPTABLE from 0x00006d49/3ffa6c20 to 0x000f1270
Copying SMBIOS entry point from 0x00006d59 to 0x000f10d0
Scan for VGA option rom
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version 1.8.1-20150325_230423-testnode)
Machine UUID 3c78721f-7317-4f85-bcbe-f5ad46d293a1
All threads complete.
Found 0 lpt ports
Found 1 serial ports
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
found virtio-blk at 0:5
Searching bootorder for: /pci@i0cf8/*@5
PS2 keyboard initialized
All threads complete.
Scan for option roms
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
drive 0x000f1050: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=16777216
Running option rom at c100:0003
Space available for UMB: c3800-ed800, f0000-f1050
Returned 131072 bytes of ZoneHigh
e820 map has 7 items:
0: 0000000000000000 - 000000000009f800 = 1 RAM
1: 000000000009f800 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000003ffe0000 = 1 RAM
4: 000000003ffe0000 - 0000000040000000 = 2 RESERVED
5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
enter handle_19:
NULL
Booting from Hard Disk...
Booting from 0000:7c00
In resume (status=0)
In 32bit resume
Attempting a hard reboot
SeaBIOS (version 1.8.1-20150325_230423-testnode)
Running on QEMU (i440fx)
Running on KVM
RamSize: 0x40000000 [cmos]
Relocating init from 0x000e11b0 to 0x3ffafd10 (size 66096)
Found QEMU fw_cfg
RamBlock: addr 0x0000000000000000 len 0x0000000040000000 [e820]
Moving pm_base to 0x600
boot order:
1: /pci@i0cf8/scsi@5/disk@0,0
2: HALT
CPU Mhz=2102
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 7 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c06f
PCI: 32: 0000000080000000 - 00000000fec00000
PCI: map device bdf=00:05.0 bar 0, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:04.0 bar 0, addr 0000c040, size 00000020 [io]
PCI: map device bdf=00:01.1 bar 4, addr 0000c060, size 00000010 [io]
PCI: map device bdf=00:03.0 bar 0, addr febf8000, size 00004000 [mem]
PCI: map device bdf=00:04.0 bar 1, addr febfc000, size 00001000 [mem]
PCI: map device bdf=00:05.0 bar 1, addr febfd000, size 00001000 [mem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0x608
PCI: init bdf=00:03.0 id=1033:0194
PCI: init bdf=00:04.0 id=1af4:1003
PCI: init bdf=00:05.0 id=1af4:1001
PCI: No VGA devices found
handle_smp: apic_id=2
handle_smp: apic_id=5
handle_smp: apic_id=1
handle_smp: apic_id=4
handle_smp: apic_id=9
handle_smp: apic_id=8
handle_smp: apic_id=11
handle_smp: apic_id=6
handle_smp: apic_id=7
handle_smp: apic_id=10
handle_smp: apic_id=3
Found 12 cpu(s) max supported 12 cpu(s)
Copying PIR from 0x3ffbfd08 to 0x000f1360
Copying MPTABLE from 0x00006d49/3ffa6c20 to 0x000f1270
Copying SMBIOS entry point from 0x00006d59 to 0x000f10d0
Scan for VGA option rom
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version 1.8.1-20150325_230423-testnode)
Machine UUID 3c78721f-7317-4f85-bcbe-f5ad46d293a1
All threads complete.
Found 0 lpt ports
Found 1 serial ports
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
found virtio-blk at 0:5
Searching bootorder for: /pci@i0cf8/*@5
PS2 keyboard initialized
All threads complete.
Scan for option roms
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
drive 0x000f1050: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=16777216
Running option rom at c100:0003
Space available for UMB: c3800-ed800, f0000-f1050
Returned 131072 bytes of ZoneHigh
e820 map has 7 items:
0: 0000000000000000 - 000000000009f800 = 1 RAM
1: 000000000009f800 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000003ffe0000 = 1 RAM
4: 000000003ffe0000 - 0000000040000000 = 2 RESERVED
5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED

[-- Attachment #3: reboot.succeeded --]
[-- Type: application/octet-stream, Size: 6126 bytes --]

SeaBIOS (version 1.8.1-20150325_230423-testnode)
Running on QEMU (i440fx)
Running on KVM
RamSize: 0x40000000 [cmos]
Relocating init from 0x000e11b0 to 0x3ffafd10 (size 66096)
Found QEMU fw_cfg
RamBlock: addr 0x0000000000000000 len 0x0000000040000000 [e820]
Moving pm_base to 0x600
boot order:
1: /pci@i0cf8/scsi@5/disk@0,0
2: HALT
CPU Mhz=2102
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 7 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c06f
PCI: 32: 0000000080000000 - 00000000fec00000
PCI: map device bdf=00:05.0 bar 0, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:04.0 bar 0, addr 0000c040, size 00000020 [io]
PCI: map device bdf=00:01.1 bar 4, addr 0000c060, size 00000010 [io]
PCI: map device bdf=00:03.0 bar 0, addr febf8000, size 00004000 [mem]
PCI: map device bdf=00:04.0 bar 1, addr febfc000, size 00001000 [mem]
PCI: map device bdf=00:05.0 bar 1, addr febfd000, size 00001000 [mem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0x608
PCI: init bdf=00:03.0 id=1033:0194
PCI: init bdf=00:04.0 id=1af4:1003
PCI: init bdf=00:05.0 id=1af4:1001
PCI: No VGA devices found
handle_smp: apic_id=4
handle_smp: apic_id=2
handle_smp: apic_id=1
handle_smp: apic_id=9
handle_smp: apic_id=6
handle_smp: apic_id=11
handle_smp: apic_id=7
handle_smp: apic_id=5
handle_smp: apic_id=3
handle_smp: apic_id=8
handle_smp: apic_id=10
Found 12 cpu(s) max supported 12 cpu(s)
Copying PIR from 0x3ffbfd08 to 0x000f1360
Copying MPTABLE from 0x00006d49/3ffa6c20 to 0x000f1270
Copying SMBIOS entry point from 0x00006d59 to 0x000f10d0
Scan for VGA option rom
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version 1.8.1-20150325_230423-testnode)
Machine UUID 3c78721f-7317-4f85-bcbe-f5ad46d293a1
All threads complete.
Found 0 lpt ports
Found 1 serial ports
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
found virtio-blk at 0:5
Searching bootorder for: /pci@i0cf8/*@5
PS2 keyboard initialized
All threads complete.
Scan for option roms
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
drive 0x000f1050: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=16777216
Running option rom at c100:0003
Space available for UMB: c3800-ed800, f0000-f1050
Returned 131072 bytes of ZoneHigh
e820 map has 7 items:
0: 0000000000000000 - 000000000009f800 = 1 RAM
1: 000000000009f800 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000003ffe0000 = 1 RAM
4: 000000003ffe0000 - 0000000040000000 = 2 RESERVED
5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
enter handle_19:
NULL
Booting from Hard Disk...
Booting from 0000:7c00
In resume (status=0)
In 32bit resume
Attempting a hard reboot
SeaBIOS (version 1.8.1-20150325_230423-testnode)
Running on QEMU (i440fx)
Running on KVM
RamSize: 0x40000000 [cmos]
Relocating init from 0x000e11b0 to 0x3ffafd10 (size 66096)
Found QEMU fw_cfg
RamBlock: addr 0x0000000000000000 len 0x0000000040000000 [e820]
Moving pm_base to 0x600
boot order:
1: /pci@i0cf8/scsi@5/disk@0,0
2: HALT
CPU Mhz=2105
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 7 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c06f
PCI: 32: 0000000080000000 - 00000000fec00000
PCI: map device bdf=00:05.0 bar 0, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:04.0 bar 0, addr 0000c040, size 00000020 [io]
PCI: map device bdf=00:01.1 bar 4, addr 0000c060, size 00000010 [io]
PCI: map device bdf=00:03.0 bar 0, addr febf8000, size 00004000 [mem]
PCI: map device bdf=00:04.0 bar 1, addr febfc000, size 00001000 [mem]
PCI: map device bdf=00:05.0 bar 1, addr febfd000, size 00001000 [mem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0x608
PCI: init bdf=00:03.0 id=1033:0194
PCI: init bdf=00:04.0 id=1af4:1003
PCI: init bdf=00:05.0 id=1af4:1001
PCI: No VGA devices found
handle_smp: apic_id=9
handle_smp: apic_id=7
handle_smp: apic_id=1
handle_smp: apic_id=6
handle_smp: apic_id=5
handle_smp: apic_id=10
handle_smp: apic_id=4
handle_smp: apic_id=8
handle_smp: apic_id=11
handle_smp: apic_id=2
handle_smp: apic_id=3
Found 12 cpu(s) max supported 12 cpu(s)
Copying PIR from 0x3ffbfd08 to 0x000f1360
Copying MPTABLE from 0x00006d49/3ffa6c20 to 0x000f1270
Copying SMBIOS entry point from 0x00006d59 to 0x000f10d0
Scan for VGA option rom
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version 1.8.1-20150325_230423-testnode)
Machine UUID 3c78721f-7317-4f85-bcbe-f5ad46d293a1
All threads complete.
Found 0 lpt ports
Found 1 serial ports
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
found virtio-blk at 0:5
Searching bootorder for: /pci@i0cf8/*@5
PS2 keyboard initialized
All threads complete.
Scan for option roms
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
drive 0x000f1050: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=16777216
Running option rom at c100:0003
Space available for UMB: c3800-ed800, f0000-f1050
Returned 131072 bytes of ZoneHigh
e820 map has 7 items:
0: 0000000000000000 - 000000000009f800 = 1 RAM
1: 000000000009f800 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000003ffe0000 = 1 RAM
4: 000000003ffe0000 - 0000000040000000 = 2 RESERVED
5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
enter handle_19:
NULL
Booting from Hard Disk...
Booting from 0000:7c00