From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marcelo Tosatti Subject: Re: does anyone run guests for more than 5 minutes? (virtio-net perf anomaly) Date: Fri, 6 Mar 2009 14:17:40 -0300 Message-ID: <20090306171740.GA14077@amt.cnet> References: <1236111221.13193.223.camel@bling> <20090306000445.GD8820@amt.cnet> <1236312791.6002.4.camel@bling> <1236353193.7276.204.camel@lappy> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm-devel To: Alex Williamson Return-path: Received: from mx2.redhat.com ([66.187.237.31]:45774 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753766AbZCFRSq (ORCPT ); Fri, 6 Mar 2009 12:18:46 -0500 Content-Disposition: inline In-Reply-To: <1236353193.7276.204.camel@lappy> Sender: kvm-owner@vger.kernel.org List-ID: On Fri, Mar 06, 2009 at 08:26:33AM -0700, Alex Williamson wrote: > On Thu, 2009-03-05 at 21:25 -0700, Alex Williamson wrote: > > On Thu, 2009-03-05 at 21:04 -0300, Marcelo Tosatti wrote: > > > On Tue, Mar 03, 2009 at 01:13:41PM -0700, Alex Williamson wrote: > > > > Any guesses as to what might be going on? Can anyone reproduce? I'm > > > > hoping that I'm doing something dumb, but can't figure out what it is. > > > > The system is running v2.6.29-rc6-121-g64e7130 in the guest, > > > > v2.6.29-rc6-123-gbd7b3b4 on the host, kvm module kvm-84-620-g5bffffc and > > > > userspace kvm-84-95-gea1b668. Thanks, > > > > > > Nope. Collect kvm_stat -l before/after the slowdown? > > > > Attached. This shows about 1 minute of data before the slowdown and a > > dramatic change starting around the 80th row. Thanks, > > For a bit easier consumption, here's a google spreadsheet and chart of > what appear to be the interesting columns: > > http://spreadsheets.google.com/pub?key=pdwpc4VwMbjWxyQfvw8AEYg > > Alex irq_injection goes up significantly. Is it virtio_net generating more interrupts? Check the rate of irq's generated by hw/virtio-net.c and compare that with rate seen in /proc/interrupts inside the guest?