Re: [Qemu-devel] [PATCH 00/17] s390x: the big pci update

[Marcel Apfelbaum <marcel@redhat.com>]

On 06/28/2016 06:02 PM, Cornelia Huck wrote:
> On Tue, 28 Jun 2016 17:33:58 +0300
> Marcel Apfelbaum <marcel@redhat.com> wrote:
>
>> On 06/24/2016 04:28 PM, Cornelia Huck wrote:
>>> We had been looking at remodelling the pci representation for s390x
>>> to handle our slightly odd architecture correctly some time ago
>>> already, but now we have a patchset that we're happy with.
>>>
>>> There's a bunch of bugfixes, cleanups and architecture conformance
>>> changes in there, but the most interesting part is the modelling
>>> (which, in some respects, takes a cue from what sPAPR does).
>>>
>>> We introduce a 'zpci' device to hold s390x-specific properties like
>>> the uid and fid. This 'zpci' device is connected to a run-of-the-mill
>>> pci device via the 'target' property.
>>>
>>> Example command line portion:
>>>
>>> -device zpci,uid=1,fid=1,target=vpci0 \
>>> -device vfio-pci,host=0000:00:00.0,id=vpci0 \
>>> -device zpci,target=vpci1 \
>>> -device vfio-pci,host=0001:00:00.0,id=vpci1 \
>>> -device vfio-pci,host=0002:00:00.0,id=vpci2
>>>
>>> For device vpci0, uid and fid are specified in the associated zpci
>>> device.
>>> For device vpci1, uid and fid are automatically generated.
>>> For device vpci2, first an associated zpci device is generated and
>>> then autogenerated values for uid and fid are placed in it.
>>>
>>> This should accomodate both specifying our special parameters and
>>> re-using standard statements for pci devices.
>>
>> [...]
>>
>>    You can still specify
>>> bus/slot/function for the pci device, but it will not be propagated
>>> into the guest (as the s390 pci architecture does not allow for it;
>>> the guest only sees uid/fid).
>>>
>>
>> Hi,
>>
>> Please excuse my lack of s390 knowledge, but I find hard to understand
>> how it works. If using bus/slot/function is not an option,
>> how can one access the configuration space? This configuration is not PCI compliant?
>
> Yes, pci on s390 is... odd. Basically, we have some special
> instructions for it.
>
> For device discovery, the guest OS uses an instruction CLP (with a
> subfunction) to list the pci functions available: see list_pci() in
> hw/s390x/s390-pci-inst.c. This list contains the function handle (fh)
> and the fid for each function.
>
> For nearly all subsequent operations that target a pci function (like
> config space accesses), the guest OS will use other instructions that
> take the fh as an argument (see also hw/s390x/s390-pci-inst.c, for
> example pci{lg,stg}_service_call() for reading/writing). The fid is
> used for enabling/disabling via yet another instruction.
>
> As an example of how this is used (and how one may construct a pci
> "topology" from this), see arch/s390/pci/ in the Linux kernel. That's
> why we have bus/slot/function in the host, even though they are
> entirely synthetic.
>

Thank you for the detailed explanations.
I had a quick look at the kernel bits and it makes more sense now.
The upper layers call the usual PCI API and the s390 pci code "translates"
it to s390 architecture, so the kernel can work as "usual".

Thanks,
Marcel