Re: nested KVM slower than QEMU with gnumach guest kernel

[Gleb Natapov <gleb@kernel.org>]

On Sun, Nov 16, 2014 at 11:18:28PM +0100, Samuel Thibault wrote:
> Hello,
> 
> Jan Kiszka, le Wed 12 Nov 2014 00:42:52 +0100, a écrit :
> > On 2014-11-11 19:55, Samuel Thibault wrote:
> > > jenkins.debian.net is running inside a KVM VM, and it runs nested
> > > KVM guests for its installation attempts.  This goes fine with Linux
> > > kernels, but it is extremely slow with gnumach kernels.
> 
> > You can try to catch a trace (ftrace) on the physical host.
> > 
> > I suspect the setup forces a lot of instruction emulation, either on L0
> > or L1. And that is slower than QEMU is KVM does not optimize like QEMU does.
> 
> Here is a sample of trace-cmd output dump: the same kind of pattern
> repeats over and over, with EXTERNAL_INTERRUPT happening mostly
> every other microsecond:
> 
>  qemu-system-x86-9752  [003]  4106.187755: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187756: kvm_entry:            vcpu 0
>  qemu-system-x86-9752  [003]  4106.187757: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187758: kvm_entry:            vcpu 0
>  qemu-system-x86-9752  [003]  4106.187759: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187760: kvm_entry:            vcpu 0
> 
> The various functions being interrupted are vmx_vcpu_run
> (0xffffffffa02848b1 and 0xffffffffa0284972), handle_io
> (0xffffffffa027ee62), vmx_get_cpl (0xffffffffa027a7de),
> load_vmc12_host_state (0xffffffffa027ea31), native_read_tscp
> (0xffffffff81050a84), native_write_msr_safe (0xffffffff81050aa6),
> vmx_decache_cr0_guest_bits (0xffffffffa027a384),
> vmx_handle_external_intr (0xffffffffa027a54d).
> 
> AIUI, the external interrupt is 0xf6, i.e. Linux' IRQ_WORK_VECTOR.  I
> however don't see any of them, neither in L0's /proc/interrupts, nor in
> L1's /proc/interrupts...
> 
Do you know how gnumach timekeeping works? Does it have a timer that fires each 1ms?
Which clock device is it using?

--
			Gleb.