From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: [PATCH v4] kvm: Use a bitmap for tracking used GSIs Date: Wed, 13 May 2009 06:55:43 -0600 Message-ID: <1242219343.4786.66.camel@2710p.home> References: <20090512220142.5663.72948.stgit@dl380g6-3.ned.telco.ned.telco> <20090513043835.6696.27384.stgit@dl380g6-3.ned.telco.ned.telco> <4A0A973A.9020502@redhat.com> <1242217702.4786.59.camel@2710p.home> <4A0ABEA8.6030103@redhat.com> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org, sheng.yang@intel.com, mst@redhat.com To: Avi Kivity Return-path: Received: from g1t0028.austin.hp.com ([15.216.28.35]:45790 "EHLO g1t0028.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757166AbZEMMzs (ORCPT ); Wed, 13 May 2009 08:55:48 -0400 In-Reply-To: <4A0ABEA8.6030103@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On Wed, 2009-05-13 at 15:35 +0300, Avi Kivity wrote: > Alex Williamson wrote: > > On Wed, 2009-05-13 at 12:47 +0300, Avi Kivity wrote: > > > >> Alex Williamson wrote: > >> > >>> We're currently using a counter to track the most recent GSI we've > >>> handed out. This quickly hits KVM_MAX_IRQ_ROUTES when using device > >>> assignment with a driver that regularly toggles the MSI enable bit. > >>> This can mean only a few minutes of usable run time. Instead, track > >>> used GSIs in a bitmap. > >>> > >>> Signed-off-by: Alex Williamson > >>> --- > >>> > >>> v2: Added mutex to protect gsi bitmap > >>> > >>> > >> Why is the mutex needed? We already have mutex protection in qemu. > >> > > > > If it's unneeded, I'll happily remove it. I was assuming in a guest > > with multiple devices these could come in parallel, but maybe the guest > > is already serialized for config space accesses via cfc/cf8. > > > > The guest may or may not be serialized; we can't rely on that. But qemu > is, and we can. Ok, I'll drop the mutex here. > >> How often does the driver enable/disable the MSI (and, do you now why)? > >> If it's often enough it may justify kernel support. (We'll need this > >> patch in any case for kernels without this new support). > >> > > > > Seems like multiple times per second. I don't know why. Now I'm > > starting to get curious why nobody else seems to be hitting this. I'm > > seeing it on an e1000e NIC and Qlogic fibre channel. Is everyone else > > using MSI-X or regular interrupts vs MSI? > > > > When you say "multiple times", it is several, or a lot more? > > Maybe it is NAPI? The system would run out of the ~1000 available GSIs in a minute or two with just an e1000e available to the guest. So that's something on the order of 10/s. This also causes a printk in the host ever time the interrupt in enabled, which can't help performance and gets pretty annoying for syslog. I was guessing some kind of interrupt mitigation, such as NAPI, but a qlogic FC card seems to do it too (seemingly at a slower rate). Alex