Re: nested KVM slower than QEMU with gnumach guest kernel

[Samuel Thibault <samuel.thibault@ens-lyon.org>]

[-- Attachment #1: Type: text/plain, Size: 1866 bytes --]

Hello,

Jan Kiszka, le Wed 12 Nov 2014 00:42:52 +0100, a écrit :
> On 2014-11-11 19:55, Samuel Thibault wrote:
> > jenkins.debian.net is running inside a KVM VM, and it runs nested
> > KVM guests for its installation attempts.  This goes fine with Linux
> > kernels, but it is extremely slow with gnumach kernels.

> You can try to catch a trace (ftrace) on the physical host.
> 
> I suspect the setup forces a lot of instruction emulation, either on L0
> or L1. And that is slower than QEMU is KVM does not optimize like QEMU does.

Here is a sample of trace-cmd output dump: the same kind of pattern
repeats over and over, with EXTERNAL_INTERRUPT happening mostly
every other microsecond:

 qemu-system-x86-9752  [003]  4106.187755: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
 qemu-system-x86-9752  [003]  4106.187756: kvm_entry:            vcpu 0
 qemu-system-x86-9752  [003]  4106.187757: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
 qemu-system-x86-9752  [003]  4106.187758: kvm_entry:            vcpu 0
 qemu-system-x86-9752  [003]  4106.187759: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
 qemu-system-x86-9752  [003]  4106.187760: kvm_entry:            vcpu 0

The various functions being interrupted are vmx_vcpu_run
(0xffffffffa02848b1 and 0xffffffffa0284972), handle_io
(0xffffffffa027ee62), vmx_get_cpl (0xffffffffa027a7de),
load_vmc12_host_state (0xffffffffa027ea31), native_read_tscp
(0xffffffff81050a84), native_write_msr_safe (0xffffffff81050aa6),
vmx_decache_cr0_guest_bits (0xffffffffa027a384),
vmx_handle_external_intr (0xffffffffa027a54d).

AIUI, the external interrupt is 0xf6, i.e. Linux' IRQ_WORK_VECTOR.  I
however don't see any of them, neither in L0's /proc/interrupts, nor in
L1's /proc/interrupts...

Samuel

[-- Attachment #2: trace.bz2 --]
[-- Type: application/octet-stream, Size: 8303 bytes --]