From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: Workload spikes on KVM host when doing IO on a guest... Date: Sun, 20 May 2012 20:49:23 +0300 Message-ID: <4FB92EA3.7070706@redhat.com> References: <4FB84114.1090600@brakkee.org> <4FB8F7A8.5050504@redhat.com> <4FB923BB.9070306@brakkee.org> <4FB924D0.1020704@redhat.com> <4FB929DE.8000500@brakkee.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: KVM list To: Erik Brakkee Return-path: Received: from mx1.redhat.com ([209.132.183.28]:27320 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753765Ab2ETRt1 (ORCPT ); Sun, 20 May 2012 13:49:27 -0400 In-Reply-To: <4FB929DE.8000500@brakkee.org> Sender: kvm-owner@vger.kernel.org List-ID: On 05/20/2012 08:29 PM, Erik Brakkee wrote: > Avi Kivity wrote: >> On 05/20/2012 08:02 PM, Erik Brakkee wrote: >>> [...] >>> Thanks for this information. Unfortunately, io="native" in domain.xml >>> is not supported by opensuse 11.3. It is supported in 12.1 so it >>> appears that the version of KVM I have on the server is too old. I >>> tried it on a system running the newer version and indeed, as you say >>> the load disappears completely when using io="native". >>> >>> I am going to update the host now (probably to centos 6.2) to get rid >>> of this problem. >> To be clear: it's not a problem. It's completely normal, and doesn't >> affect anything. > The only problem with it is that it leads to high workload spikes, > which is normally a reason to have a good look at what is going on. In > this case, the newer version of KVM should help eliminate these > spikes, so that the next time I see a spike in the workload I know > that I have to look into something. Problem is, it doesn't mean anything important. It's the count of running threads plus the count of threads uninterruptibly waiting on a mutex. It's absolutely meaningless. > I noticed the issue after I started monitoring the server and all VMs > using zabbix (www.zabbix.com) and made a graph showing the workload of > the hosts and that of all guests. See below. Falcon is the host and > sparrow is a continuous integration server which is creating an > updated RPM repository and writing a lot of files. > > > > Still the whole area of workload is a bit confusing to me. Is the > effect of native IO simply that some of the IO work is not being > counted anymore as part of the workload because the work is no longer > done in user space? No, it no longer holds a mutex. Yet it does exactly the same thing. That's an indication that the counter is meaningless. (If the counter doesn't drop on an idle machine, that usually indicates trouble; but that's not the case) -- error compiling committee.c: too many arguments to function