From mboxrd@z Thu Jan  1 00:00:00 1970
From: Jan Kiszka <jan.kiszka@web.de>
Subject: Re: nested KVM slower than QEMU with gnumach guest kernel
Date: Wed, 12 Nov 2014 00:42:52 +0100
Message-ID: <54629EFC.1050307@web.de>
References: <20141111185515.GA16376@type.youpi.perso.aquilenet.fr>
Mime-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
 protocol="application/pgp-signature";
 boundary="XKhuu78lC2VB2RBvlJT3XOL8sbLvWLemJ"
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	kvm@vger.kernel.org, gleb@kernel.org, pbonzini@redhat.com,
	abelg@il.ibm.com, nyh@il.ibm.com, oritw@il.ibm.com,
	benami@il.ibm.com, muli@il.ibm.com
Return-path: <kvm-owner@vger.kernel.org>
Received: from mout.web.de ([212.227.15.3]:60586 "EHLO mout.web.de"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1752320AbaKKXnF (ORCPT <rfc822;kvm@vger.kernel.org>);
	Tue, 11 Nov 2014 18:43:05 -0500
In-Reply-To: <20141111185515.GA16376@type.youpi.perso.aquilenet.fr>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--XKhuu78lC2VB2RBvlJT3XOL8sbLvWLemJ
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On 2014-11-11 19:55, Samuel Thibault wrote:
> Hello,
>=20
> jenkins.debian.net is running inside a KVM VM, and it runs nested
> KVM guests for its installation attempts.  This goes fine with Linux
> kernels, but it is extremely slow with gnumach kernels.  I have
> reproduced the issue with my laptop with a linux 3.17 host kernel, a
> 3.16 L1-guest kernel, and an i7-2720QM CPU, with similar results; it's
> actually even slower than letting qemu emulate the CPU... For these
> tests I'm using the following image:
>=20
> http://people.debian.org/~sthibault/tmp/netinst.iso
>=20
> The reference test here boils down to running qemu -cdrom netinst.iso -=
m
> 512, choosing the "Automated install" choice, and waiting for "Loading
> additional components" step to complete. (yes, the boot menu gets
> mangled ATM, there's apparently currently a bug between qemu and grub)
>=20
> My host is A, my level1-KVM-guest is B.
>=20
> KVM:
> A$ qemu -enable-kvm -cdrom netinst.iso -m 512M
> takes ~1 minute.
>=20
> QEMU:
> A$ qemu             -cdrom netinst.iso -m 512M
> takes ~7 minutes.
>=20
> KVM-in-KVM:
> B$ qemu -enable-kvm -cdrom netinst.iso -m 512M
> takes ~10 minutes, when it doesn't gets completely stuck, which is quit=
e
> often, actually...
>=20
> QEMU-in-KVM:
> B$ qemu             -cdrom netinst.iso -m 512M
> takes ~7 minutes.
>=20
> I don't see such horrible slowdown with a linux image.  Is there
> something particular that could explain such a difference?  What tools
> or counters could I use to investigate which area of KVM is getting
> slow?

You can try to catch a trace (ftrace) on the physical host.

I suspect the setup forces a lot of instruction emulation, either on L0
or L1. And that is slower than QEMU is KVM does not optimize like QEMU do=
es.

Jan


--XKhuu78lC2VB2RBvlJT3XOL8sbLvWLemJ
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iEYEARECAAYFAlRinvwACgkQitSsb3rl5xQzQgCfWKyDeviiXPHtdZ1wCXcUItzf
YjMAoMr00NEAYgKjwUHPq210mZnuJlEd
=n8ID
-----END PGP SIGNATURE-----

--XKhuu78lC2VB2RBvlJT3XOL8sbLvWLemJ--