Re: [Qemu-devel] [PATCH] spapr/pci: populate PCI DT in reverse order

[Alexey Kardashevskiy <aik@ozlabs.ru>]

[-- Attachment #1: Type: text/plain, Size: 3800 bytes --]

On 25/02/17 21:40, Greg Kurz wrote:
> On Sat, 25 Feb 2017 20:39:18 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 22/02/17 21:56, Greg Kurz wrote:
>>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>>
>>> Since commit 1d2d974244c6 "spapr_pci: enumerate and add PCI device tree", QEMU
>>> populates the PCI device tree in the opposite order compared to SLOF.
>>>
>>> Before 1d2d974244c6:
>>>
>>> Populating /pci@800000020000000
>>>                      00 0000 (D) : 1af4 1000    virtio [ net ]
>>>                      00 0800 (D) : 1af4 1001    virtio [ block ]
>>>                      00 1000 (D) : 1af4 1009    virtio [ network ]
>>> Populating /pci@800000020000000/unknown-legacy-device@2
>>>
>>> 7e5294b8 :  /pci@800000020000000
>>> 7e52b998 :  |-- ethernet@0
>>> 7e52c0c8 :  |-- scsi@1
>>> 7e52c7e8 :  +-- unknown-legacy-device@2 ok
>>>
>>> Since 1d2d974244c6:
>>>
>>> Populating /pci@800000020000000
>>>                      00 1000 (D) : 1af4 1009    virtio [ network ]
>>> Populating /pci@800000020000000/unknown-legacy-device@2
>>>                      00 0800 (D) : 1af4 1001    virtio [ block ]
>>>                      00 0000 (D) : 1af4 1000    virtio [ net ]
>>>
>>> 7e5e8118 :  /pci@800000020000000
>>> 7e5ea6a0 :  |-- unknown-legacy-device@2
>>> 7e5eadb8 :  |-- scsi@1
>>> 7e5eb4d8 :  +-- ethernet@0 ok
>>>
>>> This behaviour change is not actually a bug since no assumptions should be
>>> made on DT ordering. But it has no real justification either, other than
>>> being the consequence of the way fdt_add_subnode() inserts new elements
>>> to the front of the FDT rather than adding them to the tail.
>>>
>>> This patch reverts to the historical SLOF ordering by walking PCI devices
>>> in reverse order. This reconciles pseries with x86 machine types behavior.
>>> It is expected to make things easier when porting existing applications to
>>> power.
>>>
>>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>> Tested-by: Thomas Huth <thuth@redhat.com>
>>> Reviewed-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
>>> (slight update to the changelog)
>>> Signed-off-by: Greg Kurz <groug@kaod.org>
>>> ---
>>>  hw/pci/pci.c         |   28 ++++++++++++++++++++++++++++
>>>  hw/ppc/spapr_pci.c   |   12 ++++++------
>>>  include/hw/pci/pci.h |    4 ++++
>>>  3 files changed, 38 insertions(+), 6 deletions(-)
>>>
>>> David,
>>>
>>> This patch was posted and already discussed during 2.5 development:
>>>
>>> http://patchwork.ozlabs.org/patch/549925/
>>>
>>> The "consensus" at the time was that guests should not rely on device
>>> ordering (i.e. use persistent naming instead).
>>>
>>> I got recently contacted by OpenStack people who had several complaints
>>> about the reverse ordering of PCI devices in pseries: different behavior
>>> between ppc64 and x86, lots of time spent in debugging when porting
>>> applications from x86 to ppc64 before realizing that it is caused by the
>>> reverse ordering, necessity to carry hacky workarounds...  
>>
>>
>> x86 does not have a device tree, and PCI id (bus:slot:fn) is the same
>> regardless the scanning order, i.e. "lspci" will show the same picture with
>> either order.
>>
>> How could OpenStack tell the difference and require workaround for what
>> precisely?
>>
>> I am definitely missing the point here...
>>
> 
> NICs get probed in reverse order and are assigned different names compared
> to the same setup on x86 (i.e. eth0 becomes eth1). They end up using wrong
> network settings.


The answer I was looking for is that the guest probes devices in the order
from the device tree rather than doing PCI scan itself and this is how the
order in the device tree matters :)

+1 for the change.


-- 
Alexey


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 839 bytes --]