From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41469) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTBAh-0001cf-NF for qemu-devel@nongnu.org; Wed, 04 Mar 2015 10:25:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YTBAc-0002iX-9g for qemu-devel@nongnu.org; Wed, 04 Mar 2015 10:25:15 -0500 Received: from cantor2.suse.de ([195.135.220.15]:54431 helo=mx2.suse.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTBAc-0002gv-16 for qemu-devel@nongnu.org; Wed, 04 Mar 2015 10:25:10 -0500 Message-ID: <54F723D3.6050105@suse.de> Date: Wed, 04 Mar 2015 16:25:07 +0100 From: Alexander Graf MIME-Version: 1.0 References: <1424951988-40477-2-git-send-email-blaschka@linux.vnet.ibm.com> <54EF3013.8020608@suse.de> <20150226152701.GA31166@tuxmaker.boeblingen.de.ibm.com> <54EF3CEE.5020008@suse.de> <20150303080631.GA20285@tuxmaker.boeblingen.de.ibm.com> <20150303132539.GA20144@tuxmaker.boeblingen.de.ibm.com> <54F61BCD.9060409@suse.de> <20150304134421.GA54616@tuxmaker.boeblingen.de.ibm.com> <54F71B6B.6070909@suse.de> <20150304150746.GA63486@tuxmaker.boeblingen.de.ibm.com> In-Reply-To: <20150304150746.GA63486@tuxmaker.boeblingen.de.ibm.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH RFC 1/1] s390x/pci: Extend pci representation by new zpci device List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Frank Blaschka Cc: "cornelia.huck@de.ibm.com" , "borntraeger@de.ibm.com" , "qemu-devel@nongnu.org" , "Michael S. Tsirkin" On 04.03.15 16:07, Frank Blaschka wrote: > On Wed, Mar 04, 2015 at 03:49:15PM +0100, Alexander Graf wrote: >> >> >> On 04.03.15 14:44, Frank Blaschka wrote: >>> On Tue, Mar 03, 2015 at 09:38:37PM +0100, Alexander Graf wrote: >>>> >>>> >>>> On 03.03.15 14:25, Frank Blaschka wrote: >>>>> On Tue, Mar 03, 2015 at 10:33:05AM +0100, Alexander Graf wrote: >>>>>> >>>>>> >>>>>> >>>>>>> Am 03.03.2015 um 09:06 schrieb Frank Blaschka : >>>>>>> >>>>>>>> On Thu, Feb 26, 2015 at 04:34:06PM +0100, Alexander Graf wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On 26.02.15 16:27, Frank Blaschka wrote: >>>>>>>>>> On Thu, Feb 26, 2015 at 03:39:15PM +0100, Alexander Graf wrote= : >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 26.02.15 12:59, Frank Blaschka wrote: >>>>>>>>>>> This patch extends the current s390 pci implementation to >>>>>>>>>>> provide more flexibility in configuration of s390 specific >>>>>>>>>>> device handling. For this we had to introduce a new facility >>>>>>>>>>> (and bus) to hold devices representing information actually >>>>>>>>>>> provided by s390 firmware and I/O configuration. >>>>>>>>>>> >>>>>>>>>>> On s390 the physical structure of the pci system (bridge, bus= , slot) >>>>>>>>>>> in not shown to the OS. For this the pci bridge and bus creat= ed >>>>>>>>>>> in qemu can also not be shown to the guest. The new zpci devi= ce class >>>>>>>>>>> represents this abstract view on the bare pci function and al= lows to >>>>>>>>>>> provide s390 specific configuration attributes for it. >>>>>>>>>>> >>>>>>>>>>> Sample qemu configuration: >>>>>>>>>>> -device e1000,id=3Dzpci1 >>>>>>>>>>> -device ne2k_pci,id=3Dzpci2 >>>>>>>>>>> -device zpci,fid=3D2,uid=3D1248,pci_id=3Dzpci1 >>>>>>>>>>> -device zpci,fid=3D17,uid=3D2244,pci_id=3Dzpci2 >>>>>>>>>>> >>>>>>>>>>> A zpci device references the corresponding PCI device via dev= ice id. >>>>>>>>>>> The new design allows to define multiple host bridges and sup= port more >>>>>>>>>>> pci devices. >>>>>>>>>> >>>>>>>>>> Isn't this reverse? Shouldn't it rather be >>>>>>>>>> >>>>>>>>>> -device zpci,...,id=3Dzpci1 >>>>>>>>>> -device e1000,bus=3Dzpci1.0 >>>>>>>>>> >>>>>>>>>> with a limit on each virtual zpci bus to only support one devi= ce? >>>>>>>>> >>>>>>>>> Do you mean something like having multiple host bridges (provid= ing a pci bus >>>>>>>>> each) and limit the bus to just one device? >>>>>>>>> >>>>>>>>> -device s390-pcihost,fid=3D16,uid=3D1234 >>>>>>>>> -device s390-pcihost,fid=3D17,uid=3D5678 >>>>>>>>> -device e1000,bus=3Dpci.0 >>>>>>>>> -device ne2k_pci,bus=3Dpci.1 >>>>>>>>> >>>>>>>>> We also discussed this option but we don't like the idea to put= attributes >>>>>>>>> belong to the pci device to the host bridge. >>>>>>>> >>>>>>>> I guess I'm not grasping something obvious here :). What exactly= are the >>>>>>>> attributes again? >>>>>>> Sorry for the late response, I was on vacation the last couple da= ys. >>>>>>> >>>>>>> The fid and uid values are provided by microcode/io layer on the = real hardware. >>>>>> >>>>>> So they are arbitrary numbers? What uniqueness constraints do we h= ave on them? >>>>> fid and uid must be unique within the same qemu. At a first look th= e numbers are >>>>> arbitrary but our configuration folks want explicitly define a part= icular fid and uid >>>>> to better support migration and pass-through scenarios. >>>> >>>> Well, at the end of the day you want to make sure they're identical = on >>>> both sides, yes. >>>> >>>>>> IIUC you can only have a single pcie device behind a virtual "bus"= anyway, so what if we just calculate uid and fid from the bus id? >>>>> I think this similar to the current implementation. We use the slot= (idea for the future was >>>>> bus + slot) to generate uid and fid. But this is not flexible enoug= h. As I said, our >>>>> configuration folks want to be able to specify fid and uid for the = device. >>>> >>>> I don't see how this is different from what PPC does with its LIOBN >>>> which is a property of the PHB. >>>> >>>> >>>> Alex >>>> >>> >>> I played arround with the idea of having multiple host bridges and th= is worked well >>> at least for static (non hotplug) configuration. In case I want to ho= tplug a host >>> bridge I got following error: >>> >>> (qemu) device_add s390-pcihost,fid=3D8,uid=3D9 >>> Bus 'main-system-bus' does not support hotplugging >>> >>> Is there anything I have to enable to support this? >>> >>> I have: has_dynamic_sysbus =3D 1 and cannot_instantiate_with_device_a= dd_yet =3D false >>> but this seems not to help for the hotplug case. >> >> Having s390 devices reside on sysbus is probably a bad idea. Instead, >> they should be on an s390 specific bus which then can implement hotplu= g >> easily. >> >> >> Alex >> >=20 > Hm now I get lost ... >=20 > Do you suggest we should implement a s390 specific device (which is not= derived from > TYPE_PCI_HOST_BRIDGE) but implements a pci bus so we can attach a pci d= evice to this > device? =20 Ugh, PCI_HOST_BRIDGE is a sysbus device. Awesome. Conceptually your PCI bridge is not a sysbus device, since it doesn't live on a flat MMIO + legacy IRQ routing bus. Instead, it lives on its own thing that handles MMIO and IRQs via special backdoor interfaces. How much of the PCI_HOST_BRIDGE device are you actually using? Would it be a lot of effort to have another s390 specific device that exposes a PCIBus, but is not of type PCI_HOST_BRIDGE (and thus sysbus)? Alex