Re: [Qemu-devel] [PATCH] hw/pci: do not update the PCI mappings while Decode (I/O or memory) bit is not set in the Command register

[Marcel Apfelbaum <marcel@redhat.com>]

On 01/14/2016 07:28 PM, Michael S. Tsirkin wrote:
> On Thu, Jan 14, 2016 at 07:20:32PM +0200, Marcel Apfelbaum wrote:
>> On 01/14/2016 05:37 PM, Michael S. Tsirkin wrote:
>>> On Thu, Jan 14, 2016 at 05:23:27PM +0200, Marcel Apfelbaum wrote:
>>>> On 01/14/2016 04:49 PM, Michael S. Tsirkin wrote:
>>>>> On Thu, Jan 14, 2016 at 03:30:41PM +0100, Laszlo Ersek wrote:
>>>>>>> 2. The same as with pxb, disable Integrated End points for pxb-pcie.
>>>>>>
>>>>>> My vote, without a doubt.
>>>>>
>>>>> Yea, me too.
>>>>>
>>>>>
>>>>> On a related note: I wonder whether enough resources will be allocated
>>>>> to the bridge to actually make it possible to add devices by hotplug
>>>>> later.
>>>>>
>>>>
>>>> It works the same as with PXB, but now instead of having one internal PCI-bridge,
>>>> we will have several switches/root ports. Each of them will get the minimum MEM required by
>>>> PCI bridges,
>>>
>>> what does this mean? What if you add a bunch of devices
>>> with large memory BARs? They won't fit will they?
>>>
>>
>> Indeed, devices with over 1 MB (I think) BARs can't be hot-plugged.
>> This is a known design limitation. We can think of a way to handle this,
>> but the real reason we have multiple root bridges is to be able to
>> correlate an assigned device with a NUMA node. In this case the device
>> will be added more likely at boot time.
>
> Ugh. That's pretty nasty, esp considering live
> migration pretty much requires hotplug ATM.
>
>>
>> I think the first step is to have *some* hot-plug support for pxb/pxb-pcie
>> with the current constraints, once it works we can think
>> of a way to make it work for devices with large BARs.
>>
>> Thanks,
>> Marcel
>
> Well OK but I suspect changes will require host/guest interface changes.
> Time enough before 2.6 but I would hate to release 2.6 with this
> limitation in place.

Understood, the amount of work depends on the design:
1. How much memory/IO should we put aside for each root bridge?
  -  we can let the default as is today, and add optional parameters to pxb devices.
2. Pass this to guest firmware?
  - Better not. We let the firmware to config the resources as today, and
    when we build the ACPI tables we just "crop" some extra ranges for
    each pxb based on user input.


Does it sound acceptable?

Thanks,
Marcel




>
> And I'd like to mention a real pci express host won't
> have this issue I think as it is normally allocated
> a range of memory at boot time.
>
>
>>>> however the IO will be allocated only if at least one legacy device
>>>> will be present at boot time. (this is at least what SeaBIOS does, I am going to check OVMF actions)
>>>>
>>>> Also related, checking that PCIe native hotplug works for devices behind
>>>> pxb-pcie bridges is my next step after I fix the current issue.
>>>>
>>>> Thanks,
>>>> Marcel
>>>>
>>>>>
>>>>>>>
>>>>>>> I am going to look at 1., maybe I is doable in a clean way.
>>>>>>
>>>>>> My vote: don't. :)
>>>>>>
>>>>>> Thanks
>>>>>> Laszlo
>>>>>>
>>>>>>> Thanks,
>>>>>>> Marcel
>>>>>>>
>>>>>>>
>>>>>>> [...]