From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: [Qemu-devel] Anyone seeing huge slowdown launching qemu with Linux 2.6.35? Date: Thu, 05 Aug 2010 05:38:23 +0300 Message-ID: <4C5A241F.4080108@redhat.com> References: <20100803190525.GB16570@redhat.com> <4C586AB9.5040302@codemonkey.ws> <4C586CF9.7030206@redhat.com> <4C588804.5060803@redhat.com> <4C590046.2020705@redhat.com> <4C591D48.9080301@redhat.com> <4C592218.3000901@redhat.com> <4C596549.1070109@codemonkey.ws> <20100804130709.GL10499@redhat.com> <4C5967D8.7080707@codemonkey.ws> <20100804133401.GP10499@redhat.com> <4C5970AC.6060105@codemonkey.ws> <4C5995B4.90505@redhat.com> <4C5996F4.6010205@redhat.com> <721C3046-D6A7-47B2-A18C-4E59F2B68797@suse.de> <4C599B45.6030808@redhat.com> <966B1FAB-7E03-4EA6-AC74-767FC400A9CC@suse.de> <4C599FF3.5070208@redhat.com> <3A326B27-9204-4733-A9B6-25EC44291499@suse.de> <4C59A491.7020904@redhat.com> <4C59C862.60608@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Graf , Anthony Liguori , Gleb Natapov , Gerd Hoffmann , "Richard W.M. Jones" , qemu-devel@nongnu.org, kvm@vger.kernel.org To: "David S. Ahern" Return-path: Received: from mx1.redhat.com ([209.132.183.28]:64746 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758690Ab0HECi6 (ORCPT ); Wed, 4 Aug 2010 22:38:58 -0400 In-Reply-To: <4C59C862.60608@cisco.com> Sender: kvm-owner@vger.kernel.org List-ID: On 08/04/2010 11:06 PM, David S. Ahern wrote: > > On 08/04/10 11:34, Avi Kivity wrote: > >>> And it's awesome for fast prototyping. Of course, once that fast >>> becomes dog slow, it's not useful anymore. >> For the Nth time, it's only slow with 100MB initrds. > 100MB is really not that large for an initrd. > > Consider the deployment of stateless nodes - something that > virtualization allows the rapid deployment of. 1 kernel, 1 initrd with > the various binaries to be run. Create nodes as needed by launching a > shell command - be it for more capacity, isolation, etc. Why require an > iso or disk wrapper for a binary blob that is all to be run out of > memory? It's inefficient. First qemu reads the initrd and stores it in memory (where it is kept while the guest runs in case you migrate or reboot). Then the guest copies it into temporary storage (where we currently have the slowdown). Then the guest decompresses and extracts it to tmpfs (initramfs model). Finally the guest runs init out of initrd, typically using just a part of the 100MB+. Whereas with a disk image, individual pages are copied to the guest on demand without taking space in qemu. With cache=none, they don't even affect host pagecache. > The -append argument allows boot parameters to be specified at > launch. That is a very powerful and simple design option. Good point. You still have it with a small initrd that bootstraps a larger image. Note -append probably works even without -kernel, it's just that the guest isn't tooled to look at it. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=45590 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OgqMD-0006wg-P2 for qemu-devel@nongnu.org; Wed, 04 Aug 2010 22:38:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OgqMC-0002ru-C9 for qemu-devel@nongnu.org; Wed, 04 Aug 2010 22:38:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:10330) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OgqMC-0002rl-0m for qemu-devel@nongnu.org; Wed, 04 Aug 2010 22:38:56 -0400 Message-ID: <4C5A241F.4080108@redhat.com> Date: Thu, 05 Aug 2010 05:38:23 +0300 From: Avi Kivity MIME-Version: 1.0 Subject: Re: [Qemu-devel] Anyone seeing huge slowdown launching qemu with Linux 2.6.35? References: <20100803190525.GB16570@redhat.com> <4C586AB9.5040302@codemonkey.ws> <4C586CF9.7030206@redhat.com> <4C588804.5060803@redhat.com> <4C590046.2020705@redhat.com> <4C591D48.9080301@redhat.com> <4C592218.3000901@redhat.com> <4C596549.1070109@codemonkey.ws> <20100804130709.GL10499@redhat.com> <4C5967D8.7080707@codemonkey.ws> <20100804133401.GP10499@redhat.com> <4C5970AC.6060105@codemonkey.ws> <4C5995B4.90505@redhat.com> <4C5996F4.6010205@redhat.com> <721C3046-D6A7-47B2-A18C-4E59F2B68797@suse.de> <4C599B45.6030808@redhat.com> <966B1FAB-7E03-4EA6-AC74-767FC400A9CC@suse.de> <4C599FF3.5070208@redhat.com> <3A326B27-9204-4733-A9B6-25EC44291499@suse.de> <4C59A491.7020904@redhat.com> <4C59C862.60608@cisco.com> In-Reply-To: <4C59C862.60608@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "David S. Ahern" Cc: Alexander Graf , kvm@vger.kernel.org, Gleb Natapov , qemu-devel@nongnu.org, "Richard W.M. Jones" , Gerd Hoffmann On 08/04/2010 11:06 PM, David S. Ahern wrote: > > On 08/04/10 11:34, Avi Kivity wrote: > >>> And it's awesome for fast prototyping. Of course, once that fast >>> becomes dog slow, it's not useful anymore. >> For the Nth time, it's only slow with 100MB initrds. > 100MB is really not that large for an initrd. > > Consider the deployment of stateless nodes - something that > virtualization allows the rapid deployment of. 1 kernel, 1 initrd with > the various binaries to be run. Create nodes as needed by launching a > shell command - be it for more capacity, isolation, etc. Why require an > iso or disk wrapper for a binary blob that is all to be run out of > memory? It's inefficient. First qemu reads the initrd and stores it in memory (where it is kept while the guest runs in case you migrate or reboot). Then the guest copies it into temporary storage (where we currently have the slowdown). Then the guest decompresses and extracts it to tmpfs (initramfs model). Finally the guest runs init out of initrd, typically using just a part of the 100MB+. Whereas with a disk image, individual pages are copied to the guest on demand without taking space in qemu. With cache=none, they don't even affect host pagecache. > The -append argument allows boot parameters to be specified at > launch. That is a very powerful and simple design option. Good point. You still have it with a small initrd that bootstraps a larger image. Note -append probably works even without -kernel, it's just that the guest isn't tooled to look at it. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.