Re: [Qemu-devel] [PATCH 2/2 RFC] s390x/pci: rework pci infrastructure modeling

[Alexander Graf <agraf@suse.de>]

On 12.03.15 08:16, Frank Blaschka wrote:
> On Thu, Mar 12, 2015 at 11:50:02AM +0100, Frank Blaschka wrote:
>> On Thu, Mar 12, 2015 at 11:03:50AM +0100, Michael S. Tsirkin wrote:
>>> On Thu, Mar 12, 2015 at 10:54:24AM +0100, Frank Blaschka wrote:
>>>> On Wed, Mar 11, 2015 at 06:42:34PM +0100, Michael S. Tsirkin wrote:
>>>>> On Wed, Mar 11, 2015 at 03:38:44PM +0100, Frank Blaschka wrote:
>>>>>> On Tue, Mar 10, 2015 at 03:26:23PM +0100, Michael S. Tsirkin wrote:
>>>>>>> On Tue, Mar 10, 2015 at 02:03:34PM +0100, Frank Blaschka wrote:
>>>>>>>> This patch changes the modeling of the s390 qemu pci infrastructure to
>>>>>>>> better match the actual pci architecture defined by the real hardware.
>>>>>>>>
>>>>>>>> A pci host bridge like device (s390-pcihost) models the abstract view
>>>>>>>> of the bare pci function. It provides s390 specific configuration
>>>>>>>> attributes (fid and uid) for the attached pci device. The host bridge
>>>>>>>> restrict the pci bus to just hold one single pci device. Also we have
>>>>>>>> to make the s390 pci host bridge hot plugable.
>>>>>>>
>>>>>>> This requirement is really because of the 1 device per bus
>>>>>>> limitation, isn't it?
>>>>>>> If you supported many devices per bus, you could use
>>>>>>> hotplug there and there won't be need to support hotplug
>>>>>>> of the host bridge.
>>>>>>>
>>>>>> Absolutely yes. Have you seen my first proposal?
>>>>>> It basically exploits the normal pci bridge/bus/slot mechanism but need
>>>>>> a place to store s390 specific configuration attributes.
>>>>>>
>>>>>> The idea of a host bridge having this attributes and limit the bus
>>>>>> to one slot was an alternate design approach suggested by Alex.
>>>>>>
>>>>>> I like Alex's idea because:
>>>>>> 1) It reflects pretty well the actual nature of the pci system in real s390 hw
>>>>>> 2) It does not create an somehow "artifical" pci topology
>>>>>>
>>>>>
>>>>> I'll have to re-read but here's a thought: use your patch but
>>>>> remove host bridge hotplug support code.
>>>>> Stick a standard bridge with shpc support in the single slot
>>>>> behind your host bridge (existing pci-bridge-dev should do the trick,
>>>>> though not many people use it, so you might
>>>>> run into bugs, but fixing them is a good idea anyway).
>>>>> You can instanciate it automatically like Marcel's patches do
>>>>> for PXB.
>>>> Still don't undertsand so I try to summarize in my words please corrent me
>>>> if I got something wrong
>>>>
>>>> - create a standard host bridge
>>>> - change the s390-pcihost to be a pci 2 pci bridge
>>>
>>> Actually I suggested simply adding a pci 2 pci bridge behind
>>> s390-pcihost.
>>>
>>>> - now we can hotplug the s390-pcihost + hotplug a pci device to this
>>>>   s390-pcihost using standard pci hotplug mechanism
>>>
>>> My idea was to just hotplug a pci device behind the standard pci 2 pci
>>> bridge. don't support hotplugging bridge itself or s390-pcihost itself.
>>>
>>>> - we keep the 1 slot limit on the s390-pcihost. We need a place to
>>>>   store fid and uid information (see mail thread from my 1 proposal)
>>>
>>> Yes.
>>>
>>>> - If we need more than 32 pci functions we have to extend the primary pci bus
>>>>   via standart pci 2 pci bridges or add another standart host bridge
>>>>
>>>> Is this your suggestion?
>>>
>>> Almost, clarifications above.
>>>
>> OK, got your idea. Have to think about it and may do some prototyping. THX!
>>
> 
> hm, after thinking more about this I realized this is not working for us.
> Remember we need a place to store the fid and uid attributes. This place
> must be:
> 1) uid/fid per pci device
> 2) uid/fid in a hotplugable device
> 
> I have the feeling we are at the beginning again. Although I liked Alex's
> idea (host bridge containing uid/fid and having only 1 slot on the bus), it
> looks like we end up at my first proposal. This does not require any
> modification in base pci/bus code.
> 
> Thx to all of you for the discussion and suggestions.

I disagree with the assessment. The reason mst was opposed to do the
one-phb-per-device implementation (which is the closest we can get to
model things like real hardware FWIW) was that hotplug would work on the
s390 level rather than pci. I don't see how your first proposal fixes that.

Also Michael, PCI on s390 is very very special. You can't plug in
anything that does not come from IBM. There are no PCIe connectors -
instead you have IBM proprietary slots that only work with IBM approved
devices. So things like "we can plug in a PCI bridge" simply don't work
as well in that world.


Alex