Re: Freezing Windows 2008 x64bit guest

Harri Olin
2010-07-21T06:25:16Z

Gleb Natapov kirjoitti:
> On Mon, Jul 19, 2010 at 10:17:02AM +0300, Harri Olin wrote:
>> Gleb Natapov kirjoitti:
>>> On Thu, Jul 15, 2010 at 03:19:44PM +0200, Christoph Adomeit wrote:
>>>> But one Windows 2008 64 Bit Server Standard is freezing regularly.
>>>> This happens sometimes 3 times a day, sometimes it takes 2 days
>>>> until freeze. The Windows Machine is a clean fresh install.
>> I think I have seen same problem occur on my Windows 2008 SBS SP2
>> 64bit system, but a bit less often, only like once a week.
>> Now I haven't seen crashes but only freezes with qemu on 100% and
>> virtual system unresponsive.
>>
>> qemu command line:
>> /usr/local/qemu-kvm-0.11.1/bin/qemu-system-x86_64 -drive
>> file=/dev/rigelvg/w2k8system,cache=none,boot=on -drive
>> file=/dev/rigelvg/w2k8data,cache=none -m 6144 -vnc :1 -net
>> nic,macaddr=C0:FF:12:FB:AA:01,model=e1000 -net tap -smp 4 -localtime
>>
>>
> Try with different model then e1000 please. Default driver that comes
> with Windows known to have problems.

Didn't help, changed to default realtek emulation, but system freezed
again. Command line:
/usr/local/qemu-kvm-0.11.1/bin/qemu-system-x86_64 -drive
file=/dev/rigelvg/w2k8system,cache=none,boot=on -drive
file=/dev/rigelvg/w2k8data,cache=none -m 6144 -vnc :1 -net
nic,macaddr=C0:FF:12:FB:AA:01 -net tap -smp 4 -localtime

This time the virtual system was not totally unresponsive somehow,
system pinged just fine and I think DNS server worked too. However on
console mouse moved but didn't react to clicking, ctrl-alt-del, etc.

>
>>> When hang occurs ensure that problematic vm is the only on on the server and run
>>> kvm_stat. Send output here. Also do the following:
>>>
>>> # mount -t debugfs debugfs /sys/kernel/debug
>>> # echo kvm > /sys/kernel/debug/tracing/set_event
>>> # sleep 1
>>> # cat /sys/kernel/debug/tracing/trace > /tmp/trace
>>>
>>> Send /tmp/trace here too, but it may be huge, so send only last 1000
>>> lines.

Stats from this hang:
log style http://mizar.remote.agasha.com/k/kvm/kvm_stat_2_log.txt

kvm statistics

efer_reload 0 0
exits 5368720296 21258
fpu_reload 2826750721 2216
halt_exits 695286653 0
halt_wakeup 685187661 0
host_state_reload 3010399640 2216
hypercalls 0 0
insn_emulation 2765352032 6542
insn_emulation_fail 477 0
invlpg 42417110 204
io_exits 818764026 122
irq_exits 109659572 9870
irq_injections 788992032 4643
irq_window 22170277 1718
largepages 0 0
mmio_exits 1490912756 69
mmu_cache_miss 2471963 0
mmu_flooded 905690 0
mmu_pde_zapped 1821325 0
mmu_pte_updated 3256583 0
mmu_pte_write 2896199 0
mmu_recycled 259845 0

full trace: http://mizar.remote.agasha.com/k/kvm/kvm_trace_2_full.txt

qemu-system-x86-5155 [002] 1248198.487147: kvm_entry: vcpu 3
qemu-system-x86-5152 [000] 1248198.487147: kvm_inj_virq: irq 209
qemu-system-x86-5152 [000] 1248198.487148: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.487148: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5155 [002] 1248198.487148: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5152 [000] 1248198.487150: kvm_exit: reason
apic_access rip 0xfffff80001c9d5c4
qemu-system-x86-5154 [001] 1248198.487150: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5154 [001] 1248198.487150: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5155 [002] 1248198.487150: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5155 [002] 1248198.487151: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5154 [001] 1248198.487151: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5155 [002] 1248198.487151: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5152 [000] 1248198.487151: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5152 [000] 1248198.487152: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5154 [001] 1248198.487152: kvm_entry: vcpu 2
qemu-system-x86-5155 [002] 1248198.487152: kvm_entry: vcpu 3
qemu-system-x86-5152 [000] 1248198.487152: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5153 [003] 1248198.487153: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5152 [000] 1248198.487153: kvm_entry: vcpu 0
qemu-system-x86-5153 [003] 1248198.487154: kvm_inj_virq: irq 209
qemu-system-x86-5153 [003] 1248198.487155: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.487157: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5153 [003] 1248198.487159: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5153 [003] 1248198.487159: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5153 [003] 1248198.487160: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5153 [003] 1248198.487161: kvm_entry: vcpu 1
qemu-system-x86-5155 [002] 1248198.487198: kvm_exit: reason ext_irq
rip 0xfffff80001ca42c3
qemu-system-x86-5155 [002] 1248198.487199: kvm_entry: vcpu 3
qemu-system-x86-5153 [003] 1248198.487324: kvm_exit: reason ext_irq
rip 0xfffff80001c9e859
qemu-system-x86-5153 [003] 1248198.487324: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.487407: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5153 [003] 1248198.487408: kvm_entry: vcpu 1
qemu-system-x86-5152 [000] 1248198.487947: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5152 [000] 1248198.487948: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.488072: kvm_exit: reason ext_irq
rip 0xfffff80001c9e859
qemu-system-x86-5154 [001] 1248198.488072: kvm_entry: vcpu 2
qemu-system-x86-5146 [003] 1248198.488141: kvm_set_irq: gsi 0 level 1
source 0
qemu-system-x86-5146 [003] 1248198.488141: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5146 [003] 1248198.488142: kvm_apic_accept_irq:
apicid 0 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.488142: kvm_apic_accept_irq:
apicid 1 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.488143: kvm_apic_accept_irq:
apicid 2 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.488143: kvm_apic_accept_irq:
apicid 3 vec 209 (Fixed|edge)
qemu-system-x86-5152 [000] 1248198.488144: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5146 [003] 1248198.488144: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5146 [003] 1248198.488144: kvm_set_irq: gsi 0 level 0
source 0
qemu-system-x86-5152 [000] 1248198.488144: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.488145: kvm_exit: reason ext_irq
rip 0xfffff80001d049f1
qemu-system-x86-5146 [003] 1248198.488145: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5146 [003] 1248198.488145: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5155 [002] 1248198.488145: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5154 [001] 1248198.488145: kvm_inj_virq: irq 209
qemu-system-x86-5155 [002] 1248198.488145: kvm_inj_virq: irq 209
qemu-system-x86-5154 [001] 1248198.488146: kvm_entry: vcpu 2
qemu-system-x86-5152 [000] 1248198.488146: kvm_exit: reason
interrupt_window rip 0xfffff80001ca3002
qemu-system-x86-5155 [002] 1248198.488146: kvm_entry: vcpu 3
qemu-system-x86-5152 [000] 1248198.488146: kvm_inj_virq: irq 209
qemu-system-x86-5152 [000] 1248198.488147: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.488147: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5155 [002] 1248198.488147: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5152 [000] 1248198.488148: kvm_exit: reason
apic_access rip 0xfffff80001c9d5c4
qemu-system-x86-5154 [001] 1248198.488149: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5154 [001] 1248198.488149: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5155 [002] 1248198.488149: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5155 [002] 1248198.488149: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5154 [001] 1248198.488149: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5155 [002] 1248198.488150: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5152 [000] 1248198.488150: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5152 [000] 1248198.488150: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5152 [000] 1248198.488151: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5154 [001] 1248198.488151: kvm_entry: vcpu 2
qemu-system-x86-5155 [002] 1248198.488152: kvm_entry: vcpu 3
qemu-system-x86-5152 [000] 1248198.488152: kvm_entry: vcpu 0
qemu-system-x86-5153 [003] 1248198.488153: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5153 [003] 1248198.488154: kvm_inj_virq: irq 209
qemu-system-x86-5153 [003] 1248198.488154: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.488156: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5153 [003] 1248198.488158: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5153 [003] 1248198.488158: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5153 [003] 1248198.488159: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5153 [003] 1248198.488160: kvm_entry: vcpu 1
qemu-system-x86-5155 [002] 1248198.488197: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5155 [002] 1248198.488197: kvm_entry: vcpu 3
qemu-system-x86-5153 [003] 1248198.488322: kvm_exit: reason ext_irq
rip 0xfffff80001cba795
qemu-system-x86-5153 [003] 1248198.488322: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.488406: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5153 [003] 1248198.488407: kvm_entry: vcpu 1
qemu-system-x86-5152 [000] 1248198.488945: kvm_exit: reason ext_irq
rip 0xfffff80001cdc193
qemu-system-x86-5152 [000] 1248198.488946: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.489070: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5154 [001] 1248198.489071: kvm_entry: vcpu 2
qemu-system-x86-5146 [003] 1248198.489139: kvm_set_irq: gsi 0 level 1
source 0
qemu-system-x86-5146 [003] 1248198.489139: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5146 [003] 1248198.489140: kvm_apic_accept_irq:
apicid 0 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.489140: kvm_apic_accept_irq:
apicid 1 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.489141: kvm_apic_accept_irq:
apicid 2 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.489141: kvm_apic_accept_irq:
apicid 3 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.489142: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5152 [000] 1248198.489142: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5146 [003] 1248198.489142: kvm_set_irq: gsi 0 level 0
source 0
qemu-system-x86-5146 [003] 1248198.489142: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5152 [000] 1248198.489142: kvm_inj_virq: irq 209
qemu-system-x86-5146 [003] 1248198.489143: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5154 [001] 1248198.489143: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5155 [002] 1248198.489143: kvm_exit: reason ext_irq
rip 0xfffff80001d049e4
qemu-system-x86-5152 [000] 1248198.489143: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.489143: kvm_inj_virq: irq 209
qemu-system-x86-5155 [002] 1248198.489143: kvm_inj_virq: irq 209
qemu-system-x86-5154 [001] 1248198.489144: kvm_entry: vcpu 2
qemu-system-x86-5155 [002] 1248198.489144: kvm_entry: vcpu 3
qemu-system-x86-5152 [000] 1248198.489145: kvm_exit: reason
apic_access rip 0xfffff80001c9d5c4
qemu-system-x86-5154 [001] 1248198.489145: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5155 [002] 1248198.489145: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5152 [000] 1248198.489146: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5152 [000] 1248198.489147: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5154 [001] 1248198.489147: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5154 [001] 1248198.489147: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5155 [002] 1248198.489147: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5152 [000] 1248198.489147: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5155 [002] 1248198.489147: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5154 [001] 1248198.489147: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5155 [002] 1248198.489149: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5152 [000] 1248198.489149: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.489150: kvm_entry: vcpu 2
qemu-system-x86-5153 [003] 1248198.489150: kvm_exit: reason ext_irq
rip 0xfffff80001ca468f
qemu-system-x86-5155 [002] 1248198.489151: kvm_entry: vcpu 3
qemu-system-x86-5153 [003] 1248198.489151: kvm_inj_virq: irq 209
qemu-system-x86-5153 [003] 1248198.489152: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.489153: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5153 [003] 1248198.489156: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5153 [003] 1248198.489156: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5153 [003] 1248198.489156: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5153 [003] 1248198.489157: kvm_entry: vcpu 1
qemu-system-x86-5155 [002] 1248198.489195: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5155 [002] 1248198.489195: kvm_entry: vcpu 3
qemu-system-x86-5153 [003] 1248198.489321: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5154 [001] 1248198.489322: kvm_exit: reason ext_irq
rip 0xfffff80001c9e859
qemu-system-x86-5153 [003] 1248198.489322: kvm_entry: vcpu 1
qemu-system-x86-5154 [001] 1248198.489323: kvm_entry: vcpu 2
qemu-system-x86-5153 [003] 1248198.489404: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5153 [003] 1248198.489406: kvm_entry: vcpu 1
qemu-system-x86-5152 [000] 1248198.489944: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5152 [000] 1248198.489945: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.490069: kvm_exit: reason ext_irq
rip 0xfffff80001cdd795
qemu-system-x86-5154 [001] 1248198.490069: kvm_entry: vcpu 2
qemu-system-x86-5146 [003] 1248198.490137: kvm_set_irq: gsi 0 level 1
source 0
qemu-system-x86-5146 [003] 1248198.490137: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5146 [003] 1248198.490138: kvm_apic_accept_irq:
apicid 0 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.490139: kvm_apic_accept_irq:
apicid 1 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.490139: kvm_apic_accept_irq:
apicid 2 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.490140: kvm_apic_accept_irq:
apicid 3 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.490140: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5152 [000] 1248198.490140: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5146 [003] 1248198.490140: kvm_set_irq: gsi 0 level 0
source 0
qemu-system-x86-5152 [000] 1248198.490141: kvm_inj_virq: irq 209
qemu-system-x86-5146 [003] 1248198.490141: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5146 [003] 1248198.490141: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5154 [001] 1248198.490141: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5155 [002] 1248198.490141: kvm_exit: reason ext_irq
rip 0xfffff80001ca2fbf
qemu-system-x86-5152 [000] 1248198.490141: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.490142: kvm_inj_virq: irq 209
qemu-system-x86-5155 [002] 1248198.490142: kvm_entry: vcpu 3
qemu-system-x86-5154 [001] 1248198.490142: kvm_entry: vcpu 2
qemu-system-x86-5152 [000] 1248198.490143: kvm_exit: reason
apic_access rip 0xfffff80001c9d5c4
qemu-system-x86-5155 [002] 1248198.490143: kvm_exit: reason
interrupt_window rip 0xfffff80001ca3002
qemu-system-x86-5154 [001] 1248198.490143: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5155 [002] 1248198.490144: kvm_inj_virq: irq 209
qemu-system-x86-5155 [002] 1248198.490144: kvm_entry: vcpu 3
qemu-system-x86-5152 [000] 1248198.490144: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5152 [000] 1248198.490145: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5152 [000] 1248198.490145: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5154 [001] 1248198.490145: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5154 [001] 1248198.490146: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5155 [002] 1248198.490146: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5154 [001] 1248198.490146: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5152 [000] 1248198.490146: kvm_entry: vcpu 0
qemu-system-x86-5154 [001] 1248198.490147: kvm_entry: vcpu 2
qemu-system-x86-5155 [002] 1248198.490147: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5155 [002] 1248198.490148: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5155 [002] 1248198.490148: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5153 [003] 1248198.490149: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5155 [002] 1248198.490149: kvm_entry: vcpu 3
qemu-system-x86-5153 [003] 1248198.490150: kvm_inj_virq: irq 209
qemu-system-x86-5153 [003] 1248198.490150: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.490152: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5153 [003] 1248198.490154: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5153 [003] 1248198.490154: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5153 [003] 1248198.490154: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5153 [003] 1248198.490155: kvm_entry: vcpu 1
qemu-system-x86-5155 [002] 1248198.490193: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5155 [002] 1248198.490194: kvm_entry: vcpu 3
qemu-system-x86-5153 [003] 1248198.490319: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5153 [003] 1248198.490319: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.490402: kvm_exit: reason ext_irq
rip 0xfffff80001c9e859
qemu-system-x86-5153 [003] 1248198.490404: kvm_entry: vcpu 1
qemu-system-x86-5154 [001] 1248198.491067: kvm_exit: reason ext_irq
rip 0xfffff80001d04a7c
qemu-system-x86-5154 [001] 1248198.491068: kvm_entry: vcpu 2
qemu-system-x86-5146 [003] 1248198.491136: kvm_set_irq: gsi 0 level 1
source 0
qemu-system-x86-5146 [003] 1248198.491137: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5146 [003] 1248198.491138: kvm_apic_accept_irq:
apicid 0 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.491138: kvm_apic_accept_irq:
apicid 1 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.491139: kvm_apic_accept_irq:
apicid 2 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.491139: kvm_apic_accept_irq:
apicid 3 vec 209 (Fixed|edge)
qemu-system-x86-5146 [003] 1248198.491140: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5146 [003] 1248198.491140: kvm_set_irq: gsi 0 level 0
source 0
qemu-system-x86-5146 [003] 1248198.491140: kvm_pic_set_irq: chip 0
pin 0 (edge|masked)
qemu-system-x86-5154 [001] 1248198.491141: kvm_exit: reason ext_irq
rip 0xfffff80001d049f1
qemu-system-x86-5146 [003] 1248198.491141: kvm_ioapic_set_irq: pin 2
dst ff vec=209 (Fixed|physical|edge)
qemu-system-x86-5155 [002] 1248198.491141: kvm_exit: reason ext_irq
rip 0xfffff80001d04a00
qemu-system-x86-5154 [001] 1248198.491141: kvm_inj_virq: irq 209
qemu-system-x86-5155 [002] 1248198.491141: kvm_inj_virq: irq 209
qemu-system-x86-5154 [001] 1248198.491142: kvm_entry: vcpu 2
qemu-system-x86-5155 [002] 1248198.491142: kvm_entry: vcpu 3
qemu-system-x86-5154 [001] 1248198.491143: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5155 [002] 1248198.491143: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5154 [001] 1248198.491145: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5155 [002] 1248198.491145: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5154 [001] 1248198.491145: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5155 [002] 1248198.491145: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5154 [001] 1248198.491146: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5155 [002] 1248198.491146: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5154 [001] 1248198.491147: kvm_entry: vcpu 2
qemu-system-x86-5155 [002] 1248198.491147: kvm_entry: vcpu 3
qemu-system-x86-5153 [003] 1248198.491148: kvm_exit: reason ext_irq
rip 0xfffff80001ca2fbf
qemu-system-x86-5153 [003] 1248198.491149: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491151: kvm_exit: reason
interrupt_window rip 0xfffff80001ca3002
qemu-system-x86-5153 [003] 1248198.491151: kvm_inj_virq: irq 209
qemu-system-x86-5153 [003] 1248198.491152: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491153: kvm_exit: reason
apic_access rip 0xfffff80001c9df13
qemu-system-x86-5153 [003] 1248198.491155: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5153 [003] 1248198.491155: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5153 [003] 1248198.491156: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5153 [003] 1248198.491157: kvm_entry: vcpu 1
qemu-system-x86-5152 [000] 1248198.491187: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5152 [000] 1248198.491189: kvm_entry: vcpu 0
qemu-system-x86-5152 [000] 1248198.491191: kvm_exit: reason
interrupt_window rip 0xfffff80001ca3002
qemu-system-x86-5152 [000] 1248198.491192: kvm_inj_virq: irq 209
qemu-system-x86-5152 [000] 1248198.491192: kvm_entry: vcpu 0
qemu-system-x86-5152 [000] 1248198.491194: kvm_exit: reason
apic_access rip 0xfffff80001c9d5c4
qemu-system-x86-5152 [000] 1248198.491196: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5152 [000] 1248198.491196: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5152 [000] 1248198.491197: kvm_ack_irq: irqchip
IOAPIC pin 2
qemu-system-x86-5152 [000] 1248198.491198: kvm_entry: vcpu 0
qemu-system-x86-5152 [000] 1248198.491203: kvm_exit: reason
apic_access rip 0xfffff80001c14092
qemu-system-x86-5152 [000] 1248198.491205: kvm_apic: apic_read
APIC_ICR = 0x82f
qemu-system-x86-5152 [000] 1248198.491205: kvm_mmio: mmio read len 4
gpa 0xfee00300 val 0x82f
qemu-system-x86-5152 [000] 1248198.491206: kvm_entry: vcpu 0
qemu-system-x86-5152 [000] 1248198.491207: kvm_exit: reason
apic_access rip 0xfffff80001c140a9
qemu-system-x86-5152 [000] 1248198.491208: kvm_mmio: mmio write len 4
gpa 0xfee00310 val 0x2000000
qemu-system-x86-5152 [000] 1248198.491209: kvm_apic: apic_write
APIC_ICR2 = 0x2000000
qemu-system-x86-5152 [000] 1248198.491209: kvm_entry: vcpu 0
qemu-system-x86-5152 [000] 1248198.491210: kvm_exit: reason
apic_access rip 0xfffff80001c140b0
qemu-system-x86-5152 [000] 1248198.491212: kvm_mmio: mmio write len 4
gpa 0xfee00300 val 0x82f
qemu-system-x86-5152 [000] 1248198.491212: kvm_apic: apic_write
APIC_ICR = 0x82f
qemu-system-x86-5152 [000] 1248198.491212: kvm_apic_ipi: dst 2 vec 47
(Fixed|logical|de-assert|edge|dst)
qemu-system-x86-5152 [000] 1248198.491213: kvm_apic_accept_irq:
apicid 1 vec 47 (Fixed|edge)
qemu-system-x86-5152 [000] 1248198.491214: kvm_entry: vcpu 0
qemu-system-x86-5153 [003] 1248198.491215: kvm_exit: reason ext_irq
rip 0xfffff80001ca2f40
qemu-system-x86-5153 [003] 1248198.491215: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491216: kvm_exit: reason
interrupt_window rip 0xfffff80001ca3002
qemu-system-x86-5153 [003] 1248198.491217: kvm_inj_virq: irq 47
qemu-system-x86-5153 [003] 1248198.491218: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491219: kvm_exit: reason
apic_access rip 0xfffff80001cedc67
qemu-system-x86-5153 [003] 1248198.491220: kvm_mmio: mmio write len 4
gpa 0xfee000b0 val 0x0
qemu-system-x86-5153 [003] 1248198.491220: kvm_apic: apic_write
APIC_EOI = 0x0
qemu-system-x86-5153 [003] 1248198.491222: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491224: kvm_exit: reason cr_access
rip 0xfffff80001ca6ff6
qemu-system-x86-5153 [003] 1248198.491224: kvm_cr: cr_write 3 =
0x15f46d000
qemu-system-x86-5153 [003] 1248198.491228: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491229: kvm_exit: reason exception
rip 0xfffff80001ca702c
qemu-system-x86-5153 [003] 1248198.491230: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491238: kvm_exit: reason exception
rip 0xfffff80001c0f11e
qemu-system-x86-5153 [003] 1248198.491239: kvm_page_fault: address
ffffffffffd070f0 error_code 9
qemu-system-x86-5153 [003] 1248198.491243: kvm_mmio: mmio
unsatisfied-read len 4 gpa 0xfed000f0 val 0x0
qemu-system-x86-5153 [003] 1248198.491252: kvm_mmio: mmio read len 4
gpa 0xfed000f0 val 0x993c675d
qemu-system-x86-5153 [003] 1248198.491253: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491259: kvm_exit: reason cr_access
rip 0xfffff80001ca6ff6
qemu-system-x86-5153 [003] 1248198.491260: kvm_cr: cr_write 3 =
0x164599000
qemu-system-x86-5153 [003] 1248198.491262: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491263: kvm_exit: reason exception
rip 0xfffff80001ca702c
qemu-system-x86-5153 [003] 1248198.491263: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491318: kvm_exit: reason ext_irq
rip 0xfffff80001cbad28
qemu-system-x86-5153 [003] 1248198.491319: kvm_entry: vcpu 1
qemu-system-x86-5153 [003] 1248198.491404: kvm_exit: reason ext_irq
rip 0xfffff80001cdc130
qemu-system-x86-5153 [003] 1248198.491405: kvm_entry: vcpu 1