From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54584) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YW4wx-0001d8-HW for qemu-devel@nongnu.org; Thu, 12 Mar 2015 11:23:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YW4ws-0007uA-CO for qemu-devel@nongnu.org; Thu, 12 Mar 2015 11:23:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49720) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YW4ws-0007tq-4f for qemu-devel@nongnu.org; Thu, 12 Mar 2015 11:22:58 -0400 Date: Thu, 12 Mar 2015 16:22:46 +0100 From: "Michael S. Tsirkin" Message-ID: <20150312161907-mutt-send-email-mst@redhat.com> References: <1425992614-8938-1-git-send-email-blaschka@linux.vnet.ibm.com> <1425992614-8938-3-git-send-email-blaschka@linux.vnet.ibm.com> <20150310152144-mutt-send-email-mst@redhat.com> <20150311143844.GA9483@tuxmaker.boeblingen.de.ibm.com> <20150311183548-mutt-send-email-mst@redhat.com> <20150312095424.GB29840@tuxmaker.boeblingen.de.ibm.com> <20150312105648-mutt-send-email-mst@redhat.com> <20150312105002.GA4870@tuxmaker.boeblingen.de.ibm.com> <20150312131607.GA54047@tuxmaker.boeblingen.de.ibm.com> <5501A9EF.2010601@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5501A9EF.2010601@suse.de> Subject: Re: [Qemu-devel] [PATCH 2/2 RFC] s390x/pci: rework pci infrastructure modeling List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexander Graf Cc: MIHAJLOV@de.ibm.com, Frank Blaschka , qemu-devel@nongnu.org, borntraeger@de.ibm.com, fiuczy@linux.vnet.ibm.com, cornelia.huck@de.ibm.com On Thu, Mar 12, 2015 at 09:59:59AM -0500, Alexander Graf wrote: > > > On 12.03.15 08:16, Frank Blaschka wrote: > > On Thu, Mar 12, 2015 at 11:50:02AM +0100, Frank Blaschka wrote: > >> On Thu, Mar 12, 2015 at 11:03:50AM +0100, Michael S. Tsirkin wrote: > >>> On Thu, Mar 12, 2015 at 10:54:24AM +0100, Frank Blaschka wrote: > >>>> On Wed, Mar 11, 2015 at 06:42:34PM +0100, Michael S. Tsirkin wrote: > >>>>> On Wed, Mar 11, 2015 at 03:38:44PM +0100, Frank Blaschka wrote: > >>>>>> On Tue, Mar 10, 2015 at 03:26:23PM +0100, Michael S. Tsirkin wrote: > >>>>>>> On Tue, Mar 10, 2015 at 02:03:34PM +0100, Frank Blaschka wrote: > >>>>>>>> This patch changes the modeling of the s390 qemu pci infrastructure to > >>>>>>>> better match the actual pci architecture defined by the real hardware. > >>>>>>>> > >>>>>>>> A pci host bridge like device (s390-pcihost) models the abstract view > >>>>>>>> of the bare pci function. It provides s390 specific configuration > >>>>>>>> attributes (fid and uid) for the attached pci device. The host bridge > >>>>>>>> restrict the pci bus to just hold one single pci device. Also we have > >>>>>>>> to make the s390 pci host bridge hot plugable. > >>>>>>> > >>>>>>> This requirement is really because of the 1 device per bus > >>>>>>> limitation, isn't it? > >>>>>>> If you supported many devices per bus, you could use > >>>>>>> hotplug there and there won't be need to support hotplug > >>>>>>> of the host bridge. > >>>>>>> > >>>>>> Absolutely yes. Have you seen my first proposal? > >>>>>> It basically exploits the normal pci bridge/bus/slot mechanism but need > >>>>>> a place to store s390 specific configuration attributes. > >>>>>> > >>>>>> The idea of a host bridge having this attributes and limit the bus > >>>>>> to one slot was an alternate design approach suggested by Alex. > >>>>>> > >>>>>> I like Alex's idea because: > >>>>>> 1) It reflects pretty well the actual nature of the pci system in real s390 hw > >>>>>> 2) It does not create an somehow "artifical" pci topology > >>>>>> > >>>>> > >>>>> I'll have to re-read but here's a thought: use your patch but > >>>>> remove host bridge hotplug support code. > >>>>> Stick a standard bridge with shpc support in the single slot > >>>>> behind your host bridge (existing pci-bridge-dev should do the trick, > >>>>> though not many people use it, so you might > >>>>> run into bugs, but fixing them is a good idea anyway). > >>>>> You can instanciate it automatically like Marcel's patches do > >>>>> for PXB. > >>>> Still don't undertsand so I try to summarize in my words please corrent me > >>>> if I got something wrong > >>>> > >>>> - create a standard host bridge > >>>> - change the s390-pcihost to be a pci 2 pci bridge > >>> > >>> Actually I suggested simply adding a pci 2 pci bridge behind > >>> s390-pcihost. > >>> > >>>> - now we can hotplug the s390-pcihost + hotplug a pci device to this > >>>> s390-pcihost using standard pci hotplug mechanism > >>> > >>> My idea was to just hotplug a pci device behind the standard pci 2 pci > >>> bridge. don't support hotplugging bridge itself or s390-pcihost itself. > >>> > >>>> - we keep the 1 slot limit on the s390-pcihost. We need a place to > >>>> store fid and uid information (see mail thread from my 1 proposal) > >>> > >>> Yes. > >>> > >>>> - If we need more than 32 pci functions we have to extend the primary pci bus > >>>> via standart pci 2 pci bridges or add another standart host bridge > >>>> > >>>> Is this your suggestion? > >>> > >>> Almost, clarifications above. > >>> > >> OK, got your idea. Have to think about it and may do some prototyping. THX! > >> > > > > hm, after thinking more about this I realized this is not working for us. > > Remember we need a place to store the fid and uid attributes. This place > > must be: > > 1) uid/fid per pci device > > 2) uid/fid in a hotplugable device > > > > I have the feeling we are at the beginning again. Although I liked Alex's > > idea (host bridge containing uid/fid and having only 1 slot on the bus), it > > looks like we end up at my first proposal. This does not require any > > modification in base pci/bus code. > > > > Thx to all of you for the discussion and suggestions. > > I disagree with the assessment. The reason mst was opposed to do the > one-phb-per-device implementation (which is the closest we can get to > model things like real hardware FWIW) was that hotplug would work on the > s390 level rather than pci. I don't see how your first proposal fixes that. > > Also Michael, PCI on s390 is very very special. Yes, I'm trying to wrap my head around it all. And is there hotplug support there on real hardware? > You can't plug in > anything that does not come from IBM. There are no PCIe connectors - > instead you have IBM proprietary slots that only work with IBM approved > devices. So things like "we can plug in a PCI bridge" simply don't work > as well in that world. > > > Alex But interestingly, the usage example that Frank gave actually shows e1000 and other non-IBM cards apparently working? This kind of confuses me. -- MST