From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:54185) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TLDr9-0005zg-M7 for qemu-devel@nongnu.org; Mon, 08 Oct 2012 09:58:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TLDr7-00079J-V4 for qemu-devel@nongnu.org; Mon, 08 Oct 2012 09:58:51 -0400 Received: from e23smtp07.au.ibm.com ([202.81.31.140]:36298) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TLDr7-000793-DI for qemu-devel@nongnu.org; Mon, 08 Oct 2012 09:58:49 -0400 Received: from /spool/local by e23smtp07.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 8 Oct 2012 23:55:57 +1000 Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q98Dn3qH33489086 for ; Tue, 9 Oct 2012 00:49:03 +1100 Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q98Dwf4J026474 for ; Tue, 9 Oct 2012 00:58:41 +1100 From: Anthony Liguori In-Reply-To: <87y5jhpuu2.fsf@rustcorp.com.au> References: <87zk4c2tqq.fsf@rustcorp.com.au> <874nmajcmj.fsf@codemonkey.ws> <87y5jhpuu2.fsf@rustcorp.com.au> Date: Mon, 08 Oct 2012 08:58:34 -0500 Message-ID: <87bogddq0l.fsf@codemonkey.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: [Qemu-devel] Using PCI config space to indicate config location List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Rusty Russell Cc: virtualization@lists.linux-foundation.org, qemu-devel , kvm@vger.kernel.org, "Michael S. Tsirkin" Rusty Russell writes: > (Topic updated, cc's trimmed). > > Anthony Liguori writes: >> Rusty Russell writes: >>> 4) The only significant change to the spec is that we use PCI >>> capabilities, so we can have infinite feature bits. >>> (see http://lists.linuxfoundation.org/pipermail/virtualization/2011-December/019198.html) >> >> We discussed this on IRC last night. I don't think PCI capabilites are >> a good mechanism to use... >> >> PCI capabilities are there to organize how the PCI config space is >> allocated to allow vendor extensions to co-exist with future PCI >> extensions. >> >> But we've never used the PCI config space within virtio-pci. We do >> everything in BAR0. I don't think there's any real advantage of using >> the config space vs. a BAR for virtio-pci. > > Note before anyone gets confused; we were talking about using the PCI > config space to indicate what BAR(s) the virtio stuff is in. An > alternative would be to simply specify a new layout format in BAR1. > > The arguments for a more flexible format that I know of: > > 1) virtio-pci has already extended the pci-specific part of the > configuration once (for MSI-X), so I don't want to assume it won't > happen again. "configuration" is the wrong word here. The virtio-pci BAR0 layout is: 0..19 virtio-pci registers 20+ virtio configuration space MSI-X needed to add additional virtio-pci registers, so now we have: 0..19 virtio-pci registers if MSI-X: 20..23 virtio-pci MSI-X registers 24+ virtio configuration space else: 20+ virtio configuration space I agree, this stinks. But I think we could solve this in a different way. I think we could just move the virtio configuration space to BAR1 by using a transport feature bit. That then frees up the entire BAR0 for use as virtio-pci registers. We can then always include the virtio-pci MSI-X register space and introduce all new virtio-pci registers as simply being appended. This new feature bit then becomes essentially a virtio configuration latch. When unacked, virtio configuration hides new registers, when acked, those new registers are exposed. Another option is to simply put new registers after the virtio configuration blob. > 2) ISTR an argument about mapping the ISR register separately, for > performance, but I can't find a reference to it. I think the rationale is that ISR really needs to be PIO but everything else doesn't. PIO is much faster on x86 because it doesn't require walking page tables or instruction emulation to handle the exit. The argument to move the remaining registers to MMIO is to allow 64-bit accesses to registers which isn't possible with PIO. >> This maps really nicely to non-PCI transports too. > > This isn't right. Noone else can use the PCI layout. While parts are > common, other parts are pci-specific (MSI-X and ISR for example), and > yet other parts are specified by PCI elsewhere (eg interrupt numbers). > >> But extending the >> PCI config space (especially dealing with capability allocation) is >> pretty gnarly and there isn't an obvious equivalent outside of PCI. > > That's OK, because general changes should be done with feature bits, and > the others all have an infinite number. Being the first, virtio-pci has > some unique limitations we'd like to fix. > >> There are very devices that we emulate today that make use of extended >> PCI device registers outside the platform devices (that have no BARs). > > This sentence confused me? There is a missing "few". "There are very few devices..." Extending the PCI configuration space is unusual for PCI devices. That was the point. Regards, Anthony Liguori > > Thanks, > Rusty.