From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kiszka Subject: Re: nested KVM slower than QEMU with gnumach guest kernel Date: Mon, 17 Nov 2014 07:28:23 +0100 Message-ID: <54699587.9040109@web.de> References: <20141111185515.GA16376@type.youpi.perso.aquilenet.fr> <54629EFC.1050307@web.de> <20141116221828.GA13123@type> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="89dwiopKa3WkRlBC9TKEgkfmVWp8DQG39" To: Samuel Thibault , kvm@vger.kernel.org, gleb@kernel.org, pbonzini@redhat.com Return-path: Received: from mout.web.de ([212.227.15.3]:63031 "EHLO mout.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750920AbaKQG2m (ORCPT ); Mon, 17 Nov 2014 01:28:42 -0500 In-Reply-To: <20141116221828.GA13123@type> Sender: kvm-owner@vger.kernel.org List-ID: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --89dwiopKa3WkRlBC9TKEgkfmVWp8DQG39 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2014-11-16 23:18, Samuel Thibault wrote: > Hello, >=20 > Jan Kiszka, le Wed 12 Nov 2014 00:42:52 +0100, a =E9crit : >> On 2014-11-11 19:55, Samuel Thibault wrote: >>> jenkins.debian.net is running inside a KVM VM, and it runs nested >>> KVM guests for its installation attempts. This goes fine with Linux >>> kernels, but it is extremely slow with gnumach kernels. >=20 >> You can try to catch a trace (ftrace) on the physical host. >> >> I suspect the setup forces a lot of instruction emulation, either on L= 0 >> or L1. And that is slower than QEMU is KVM does not optimize like QEMU= does. >=20 > Here is a sample of trace-cmd output dump: the same kind of pattern > repeats over and over, with EXTERNAL_INTERRUPT happening mostly > every other microsecond: >=20 > qemu-system-x86-9752 [003] 4106.187755: kvm_exit: reason= EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6 > qemu-system-x86-9752 [003] 4106.187756: kvm_entry: vcpu 0= > qemu-system-x86-9752 [003] 4106.187757: kvm_exit: reason= EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6 > qemu-system-x86-9752 [003] 4106.187758: kvm_entry: vcpu 0= > qemu-system-x86-9752 [003] 4106.187759: kvm_exit: reason= EXTERNAL_INTERRUPT rip 0xffffffffa02848b1 info 0 800000f6 > qemu-system-x86-9752 [003] 4106.187760: kvm_entry: vcpu 0= You may want to turn on more trace events, if not all, to possibly see what Linux does then. The next level after that is function tracing (may require a kernel rebuild or a tracing kernel of the distro). >=20 > The various functions being interrupted are vmx_vcpu_run > (0xffffffffa02848b1 and 0xffffffffa0284972), handle_io > (0xffffffffa027ee62), vmx_get_cpl (0xffffffffa027a7de), > load_vmc12_host_state (0xffffffffa027ea31), native_read_tscp > (0xffffffff81050a84), native_write_msr_safe (0xffffffff81050aa6), > vmx_decache_cr0_guest_bits (0xffffffffa027a384), > vmx_handle_external_intr (0xffffffffa027a54d). >=20 > AIUI, the external interrupt is 0xf6, i.e. Linux' IRQ_WORK_VECTOR. I > however don't see any of them, neither in L0's /proc/interrupts, nor in= > L1's /proc/interrupts... I suppose this is a SMP host and guest? Does reducing CPUs to 1 change to picture? If not, it may help to understand cause and effect easier. Jan --89dwiopKa3WkRlBC9TKEgkfmVWp8DQG39 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlRplYcACgkQitSsb3rl5xTjYgCg3Rc0m0lYBjjpjMielmwvYqk9 w4MAn1gAs27cxBVdPiJ8N8GtAOCcJXIa =cxEZ -----END PGP SIGNATURE----- --89dwiopKa3WkRlBC9TKEgkfmVWp8DQG39--