Re: nested KVM slower than QEMU with gnumach guest kernel

[Jan Kiszka <jan.kiszka@web.de>]

[-- Attachment #1: Type: text/plain, Size: 2365 bytes --]

On 2014-11-16 23:18, Samuel Thibault wrote:
> Hello,
> 
> Jan Kiszka, le Wed 12 Nov 2014 00:42:52 +0100, a écrit :
>> On 2014-11-11 19:55, Samuel Thibault wrote:
>>> jenkins.debian.net is running inside a KVM VM, and it runs nested
>>> KVM guests for its installation attempts.  This goes fine with Linux
>>> kernels, but it is extremely slow with gnumach kernels.
> 
>> You can try to catch a trace (ftrace) on the physical host.
>>
>> I suspect the setup forces a lot of instruction emulation, either on L0
>> or L1. And that is slower than QEMU is KVM does not optimize like QEMU does.
> 
> Here is a sample of trace-cmd output dump: the same kind of pattern
> repeats over and over, with EXTERNAL_INTERRUPT happening mostly
> every other microsecond:
> 
>  qemu-system-x86-9752  [003]  4106.187755: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187756: kvm_entry:            vcpu 0
>  qemu-system-x86-9752  [003]  4106.187757: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187758: kvm_entry:            vcpu 0
>  qemu-system-x86-9752  [003]  4106.187759: kvm_exit:             reason EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187760: kvm_entry:            vcpu 0

You may want to turn on more trace events, if not all, to possibly see
what Linux does then. The next level after that is function tracing (may
require a kernel rebuild or a tracing kernel of the distro).

> 
> The various functions being interrupted are vmx_vcpu_run
> (0xffffffffa02848b1 and 0xffffffffa0284972), handle_io
> (0xffffffffa027ee62), vmx_get_cpl (0xffffffffa027a7de),
> load_vmc12_host_state (0xffffffffa027ea31), native_read_tscp
> (0xffffffff81050a84), native_write_msr_safe (0xffffffff81050aa6),
> vmx_decache_cr0_guest_bits (0xffffffffa027a384),
> vmx_handle_external_intr (0xffffffffa027a54d).
> 
> AIUI, the external interrupt is 0xf6, i.e. Linux' IRQ_WORK_VECTOR.  I
> however don't see any of them, neither in L0's /proc/interrupts, nor in
> L1's /proc/interrupts...

I suppose this is a SMP host and guest? Does reducing CPUs to 1 change
to picture? If not, it may help to understand cause and effect easier.

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]