From mboxrd@z Thu Jan  1 00:00:00 1970
From: Gleb Natapov <gleb@kernel.org>
Subject: Re: nested KVM slower than QEMU with gnumach guest kernel
Date: Mon, 17 Nov 2014 10:58:45 +0200
Message-ID: <20141117085845.GE26187@minantech.com>
References: <20141111185515.GA16376@type.youpi.perso.aquilenet.fr>
 <54629EFC.1050307@web.de>
 <20141116221828.GA13123@type>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: QUOTED-PRINTABLE
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Jan Kiszka <jan.kiszka@web.de>, kvm@vger.kernel.org,
	pbonzini@redhat.com
Return-path: <kvm-owner@vger.kernel.org>
Received: from mail-wi0-f177.google.com ([209.85.212.177]:45941 "EHLO
	mail-wi0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751283AbaKQI6t convert rfc822-to-8bit (ORCPT
	<rfc822;kvm@vger.kernel.org>); Mon, 17 Nov 2014 03:58:49 -0500
Received: by mail-wi0-f177.google.com with SMTP id l15so8290272wiw.4
        for <kvm@vger.kernel.org>; Mon, 17 Nov 2014 00:58:48 -0800 (PST)
Content-Disposition: inline
In-Reply-To: <20141116221828.GA13123@type>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Sun, Nov 16, 2014 at 11:18:28PM +0100, Samuel Thibault wrote:
> Hello,
>=20
> Jan Kiszka, le Wed 12 Nov 2014 00:42:52 +0100, a =C3=A9crit :
> > On 2014-11-11 19:55, Samuel Thibault wrote:
> > > jenkins.debian.net is running inside a KVM VM, and it runs nested
> > > KVM guests for its installation attempts.  This goes fine with Li=
nux
> > > kernels, but it is extremely slow with gnumach kernels.
>=20
> > You can try to catch a trace (ftrace) on the physical host.
> >=20
> > I suspect the setup forces a lot of instruction emulation, either o=
n L0
> > or L1. And that is slower than QEMU is KVM does not optimize like Q=
EMU does.
>=20
> Here is a sample of trace-cmd output dump: the same kind of pattern
> repeats over and over, with EXTERNAL_INTERRUPT happening mostly
> every other microsecond:
>=20
>  qemu-system-x86-9752  [003]  4106.187755: kvm_exit:             reas=
on EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187756: kvm_entry:            vcpu=
 0
>  qemu-system-x86-9752  [003]  4106.187757: kvm_exit:             reas=
on EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187758: kvm_entry:            vcpu=
 0
>  qemu-system-x86-9752  [003]  4106.187759: kvm_exit:             reas=
on EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6
>  qemu-system-x86-9752  [003]  4106.187760: kvm_entry:            vcpu=
 0
>=20
> The various functions being interrupted are vmx_vcpu_run
> (0xffffffffa02848b1 and 0xffffffffa0284972), handle_io
> (0xffffffffa027ee62), vmx_get_cpl (0xffffffffa027a7de),
> load_vmc12_host_state (0xffffffffa027ea31), native_read_tscp
> (0xffffffff81050a84), native_write_msr_safe (0xffffffff81050aa6),
> vmx_decache_cr0_guest_bits (0xffffffffa027a384),
> vmx_handle_external_intr (0xffffffffa027a54d).
>=20
> AIUI, the external interrupt is 0xf6, i.e. Linux' IRQ_WORK_VECTOR.  I
> however don't see any of them, neither in L0's /proc/interrupts, nor =
in
> L1's /proc/interrupts...
>=20
Do you know how gnumach timekeeping works? Does it have a timer that fi=
res each 1ms?
Which clock device is it using?

--
			Gleb.