From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:34034) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RDYQa-0000X5-Qt for qemu-devel@nongnu.org; Tue, 11 Oct 2011 05:15:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RDYQZ-0005PR-DD for qemu-devel@nongnu.org; Tue, 11 Oct 2011 05:15:12 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49291) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RDYQZ-0005PA-5i for qemu-devel@nongnu.org; Tue, 11 Oct 2011 05:15:11 -0400 Message-ID: <4E940919.7010901@redhat.com> Date: Tue, 11 Oct 2011 11:15:05 +0200 From: Avi Kivity MIME-Version: 1.0 References: <20111010170803.GV9408@redhat.com> <4E933F2D.7090703@codemonkey.ws> <20111011082315.GI14627@redhat.com> In-Reply-To: <20111011082315.GI14627@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Daniel P. Berrange" Cc: Gleb Natapov , "Richard W.M. Jones" , Alexander Graf , qemu-devel On 10/11/2011 10:23 AM, Daniel P. Berrange wrote: > - Application sandbox, directly boots the regular host's kernel and > a custom initrd image. The initrd does not contain any files except > for the 9p kernel modules and a custom init binary, which mounts > the guest root FS from a 9p filesystem export. > > The kernel is< 5 MB, while the initrd is approx 700 KB compressed, > or 1.4 MB compressed. Performance for the sandbox is even more > critical than for libguestfs. Even 10's of milliseconds make a > difference here. The commands being run in the sandbox can be > very short lived processes, executed reasonably frequently. The > goal is to have end-to-end runtime overhead of< 2 seconds. This > includes libvirt guest startup, qemu startup/shutdown, bios time, > option ROM time, kernel boot& shutdown time. > > The reason for using a kerenl/initrd instead of a bootable ISO, > is that building an ISO requires time itself, and we need to be > able to easily pass kernel boot arguments via -append. > > > I'm focusing on the last use case, and if the phase of the moon > is correct, I can currently executed a sandbox command with a total > overhead of 3.5 seconds (if using a compressed initrd) of which > the QEMU execution time is 2.5 seconds. > > Of this, 1.4 seconds is the time required by LinuxBoot to copy the > kernel+initrd. If I used an uncompressed initrd, which I really want > to, to avoid decompression overhead, this increases to ~1.7 seconds. > So the LinuxBoot ROM is ~60% of total QEMU execution time, or 40% > of total sandbox execution overhead. One thing we can do is boot a guest and immediately snapshot it, before it runs any application specific code. Subsequent invocations will MAP_PRIVATE the memory image and COW their way. This avoids the kernel initialization time as well. > > For comparison I also did a test building a bootable ISO using ISOLinux. > This required 700 ms for the boot time, which is appoximately 1/2 the > time reqiured for direct kernel/initrd boot. But you have to then add > on time required to build the ISO on every boot, to add custom kernel > command line args. So while ISO is faster than LinuxBoot currently > there is still non-negligable overhead here that I want to avoid. You can accept parameters from virtio-serial or some other channel. Is there any reason you need them specifically as *kernel* command line parameters? > For further comparison I tested with Rich Jones' patches which add a > DMA-like inteface to fw_cfg. With this the time spent in the LinuxBoot > option ROM was as close to zero as matters. > > So obviously, my preference is for -kernel/-initrd to be made very fast > using the DMA-like patches, or any other patches which could achieve > similarly high performance for -kernel/-initd. -- error compiling committee.c: too many arguments to function