From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: Windows slow boot: contractor wanted Date: Wed, 22 Aug 2012 12:08:27 +0300 Message-ID: <5034A18B.5040408@redhat.com> References: <20120816104727.GA17166@alpha.arachsys.com> <502CDC0D.9080004@redhat.com> <20120817123642.GA16736@alpha.arachsys.com> <5030F273.5080706@redhat.com> <20120820135656.GA16676@alpha.arachsys.com> <50334E13.8020100@redhat.com> <20120821152107.GA16363@alpha.arachsys.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, Rik van Riel To: Richard Davies Return-path: Received: from mx1.redhat.com ([209.132.183.28]:57796 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752514Ab2HVJIh (ORCPT ); Wed, 22 Aug 2012 05:08:37 -0400 In-Reply-To: <20120821152107.GA16363@alpha.arachsys.com> Sender: kvm-owner@vger.kernel.org List-ID: On 08/21/2012 06:21 PM, Richard Davies wrote: > Avi Kivity wrote: >> Richard Davies wrote: >> > We're running host kernel 3.5.1 and qemu-kvm 1.1.1. >> > >> > I hadn't though about it, but I agree this is related to cpu overcommit. The >> > slow boots are intermittent (and infrequent) with cpu overcommit whereas I >> > don't think it occurs without cpu overcommit. >> > >> > In addition, if there is a slow boot ongoing, and you kill some other VMs to >> > reduce cpu overcommit then this will sometimes speed it up. >> > >> > I guess the question is why even with overcommit most boots are fine, but >> > some small fraction then go slow? >> >> Could be a bug. The scheduler and the spin-loop handling code fight >> each other instead of working well. >> >> Please provide snapshots of 'perf top' while a slow boot is in progress. > > Below are two 'perf top' snapshots during a slow boot, which appear to me to > support your idea of a spin-lock problem. > > There are a lot more "unprocessable samples recorded" messages at the end of > each snapshot which I haven't included. I think these may be from the guest > OS - the kernel is listed, and qemu-kvm itself is listed on some other > traces which I did, although not these. > > Richard. > > > > PerfTop: 62249 irqs/sec kernel:96.9% exact: 0.0% [4000Hz cycles], (all, 16 CPUs) > -------------------------------------------------------------------------------------------------------------------------------- > > 35.80% [kernel] [k] _raw_spin_lock_irqsave > 21.64% [kernel] [k] isolate_freepages_block Please disable ksm, and if this function persists in the profile, reduce some memory from the guests. > 5.91% [kernel] [k] yield_to > 4.95% [kernel] [k] _raw_spin_lock > 3.37% [kernel] [k] kvm_vcpu_on_spin Except for isolate_freepages_block, all functions up to here have to do with dealing with cpu overcommit. But let's deal with them after we see a profile with isolate_freepages_block removed. -- error compiling committee.c: too many arguments to function