From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:37101) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TbV5o-0004x5-JY for qemu-devel@nongnu.org; Thu, 22 Nov 2012 06:37:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TbV5n-0005uZ-2k for qemu-devel@nongnu.org; Thu, 22 Nov 2012 06:37:16 -0500 Date: Thu, 22 Nov 2012 13:39:50 +0200 From: "Michael S. Tsirkin" Message-ID: <20121122113950.GA26018@redhat.com> References: <20121121005705.GM18362@truffula.fritz.box> <20121121113448.GA31911@redhat.com> <20121121123600.GV18362@truffula.fritz.box> <20121121131339.GA5730@redhat.com> <20121121132157.GW18362@truffula.fritz.box> <50ACD6AC.6030001@suse.de> <20121121152737.GC5730@redhat.com> <20121122022718.GY18362@truffula.fritz.box> <20121122072303.GA23260@redhat.com> <3190D38B-7409-455C-89C7-CD1F141EF19A@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3190D38B-7409-455C-89C7-CD1F141EF19A@suse.de> Subject: Re: [Qemu-devel] [Qemu-ppc] [PATCH 12/12] pseries: Generate unique LIOBNs for PCI host bridges List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexander Graf Cc: "qemu-ppc@nongnu.org List" , qemu-devel qemu-devel , David Gibson On Thu, Nov 22, 2012 at 12:27:49PM +0100, Alexander Graf wrote: > > > On 22.11.2012, at 08:23, "Michael S. Tsirkin" wrote: > > > On Thu, Nov 22, 2012 at 01:27:18PM +1100, David Gibson wrote: > >> On Wed, Nov 21, 2012 at 05:27:37PM +0200, Michael S. Tsirkin wrote: > >>> On Wed, Nov 21, 2012 at 02:27:08PM +0100, Alexander Graf wrote: > >>>> On 11/21/2012 02:21 PM, David Gibson wrote: > >>>>> On Wed, Nov 21, 2012 at 03:13:39PM +0200, Michael S. Tsirkin wrote: > >>>>>> On Wed, Nov 21, 2012 at 11:36:00PM +1100, David Gibson wrote: > >>>>>>> On Wed, Nov 21, 2012 at 01:34:48PM +0200, Michael S. Tsirkin wrote: > >>>>>>>> On Wed, Nov 21, 2012 at 11:57:05AM +1100, David Gibson wrote: > >>>>>>>>> On Tue, Nov 20, 2012 at 02:26:09PM +0200, Michael S. Tsirkin wrote: > >>>>>>>>>> On Tue, Nov 20, 2012 at 10:27:11AM +0100, Alexander Graf wrote: > >>>>>>>>>>> On 19.11.2012, at 23:51, David Gibson wrote: > >>>>>>>>>>> > >>>>>>>>>>>> On Mon, Nov 19, 2012 at 05:34:12PM +0100, Alexander Graf wrote: > >>>>>>>>>>>>> On 13.11.2012, at 03:47, David Gibson wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> From: Alexey Kardashevskiy > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> In future (with VFIO) we will have multiple PCI host bridges on > >>>>>>>>>>>>>> pseries. Each one needs a unique LIOBN (IOMMU id). At the moment we > >>>>>>>>>>>>>> derive these from the pci domain number, but the whole notion of > >>>>>>>>>>>>>> domain numbers on the qemu side is bogus and in any case they're not > >>>>>>>>>>>>>> actually uniquely allocated at this point. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> This patch, therefore uses a simple sequence counter to generate > >>>>>>>>>>>>>> unique LIOBNs for PCI host bridges. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Signed-off-by: Alexey Kardashevskiy > >>>>>>>>>>>>>> Signed-off-by: David Gibson > >>>>>>>>>>>>> I don't really like the idea of having a global variable just > >>>>>>>>>>>>> because our domain ID generation seems to not work as > >>>>>>>>>>>>> expected. Michael, any comments here? > >>>>>>>>>>>> Well, the patch I sent which changed domain id generation was > >>>>>>>>>>>> ignored. In any case, as I said, the whole concept of domain numbers > >>>>>>>>>>> Michael? > >>>>>>>>>> This is user visible, right? > >>>>>>>>>> So IMHO we should have the user specify LIOBN through a property, > >>>>>>>>>> rather than assign what's essentially a random value. > >>>>>>>>> Well, I can implement an override through a property, which could be > >>>>>>>>> useful in some circumstances. But we still need to have qemu generate > >>>>>>>>> unique defaults, rather than forcing it to be specified in every case. > >>>>>>>> I don't see why. > >>>>>>>> And if you want automatic defaults then they need to be generated in a > >>>>>>>> way that does not depend on implementation detail such as order of > >>>>>>>> device initialization. > >>>>>>> Because requiring explicit unique liobns to be supplied whenever there > >>>>>>> is more than one PHB is horrible for usability. > >>>>>> We should make simple things simple and complex things possible. > >>>>>> More than one PHB seems like an advanced feature > >>>>> Not for pseries. On real hardware of this type, dozens of PHBs is > >>>>> routine. Plus, vfio passthrough is coming, we need at minimum one PHB > >>>>> for emulated devices and one for passthrough devices. > >>>> > >>>> Yeah, I second Davids opinion here. We need to make this easy for users. > >>> > >>> I think users don't normally create PHBs. They request a disk, a network > >>> device, a pass-through device ... In this case if this requires > >>> more PHBs create them internally but I imagine this doesn't > >>> require any allocation scheme - simply set some address > >>> for virtual devices, some other one for assigned devices ... no? > >> > >> No. One PHB for passthrough and one for emulated is the minimum. > >> Since we don't emulated p2p bridges, > > > > Actually qemu does emulate p2p bridges. > > > >> each PHB can only support a small > >> number of PCI devices, so if enough PCI devices are requested, we will > >> still need to create - and assign numbers to - additional PHBs. > > > > Each PHB can support up to 32 slots right? This seems ample for > > a typical use. If you want many tens of devices you need to > > supply addresses manually, this looks reasonable to me. > > > > Allocating PHBs on the fly seems unencessarily tricky. > > IIRC it's required for fault isolation. > > Alex IIUC separate physical PHBs might help fault isolation of assigned devices. OTOH I don't see why we need to emulate PHBs for this. > > > >>> If user wants to play with low level detail such as PHBs I don't see why > >>> it's not reasonable to require full specification. > >>> > >>>> How do we assign PCI slot IDs today when all you do is a -device? > >>>> This should probably follow the same scheme. > >>>> > >>>> > >>>> Alex > >>> > >>> What we do is find a first free slow. But it's exactly why I worry: > >>> changing pci addresses between qemu releases has been a source of pain > >>> and compatibility hassles in the past. > >>> The problem would be more manageable if you simply reserve some fixed > >>> addresses for internal use, like we reserve slots for VGA and IDE, > >>> though even that becomes a problem as we switch to q35. > >>> > >> > >> -- > >> David Gibson | I'll have my music baroque, and my code > >> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ > >> | _way_ _around_! > >> http://www.ozlabs.org/~dgibson