Re: PCI Pass-through in Xen ARM - Draft 2.

[Manish Jaggi <mjaggi@caviumnetworks.com>]

On Thursday 09 July 2015 01:38 PM, Julien Grall wrote:
> Hi Manish,
>
> On 09/07/2015 08:13, Manish Jaggi wrote:
>>>
>>> If this was a domctl there might be scope for accepting an
>>> implementation which made assumptions such as sbdf == deviceid. However
>>> I'd still like to see this topic given proper treatment in the design
>>> and not just glossed over with "this is how ThunderX does things".
>> I got your point.
>>> Or maybe the solution is simple and we should just do it now -- i.e. 
>>> can
>>> we add a new field to the PHYSDEVOP_pci_host_bridge_add argument struct
>>> which contains the base deviceid for that bridge
>> deviceId would be same as sbdf. As we dont have a way to translate sbdf
>> to deviceID.
>
> I think we have to be clear in this design document about the 
> different meaning.
>
> When the Device Tree is used, it's assumed that the deviceID will be 
> equal to the requester ID and not the sbdf.
Does SMMU v2 has a concept of requesterID.
I see requesterID term in SMMUv3
>
> Linux provides a function (pci_for_each_dma_alias) which will return a 
> requester ID for a given PCI device. It appears that the BDF (the 's' 
> of sBDF is only internal to Linux and not part of the hardware) is 
> equal to the requester ID on your platform but we can't assume it for 
> anyone else.
so you mean requesterID = pci_for_each_dma_alias(sbdf)
>
> When we have a PCI in hand, we have to find the requester ID for this 
> device.
That is the question. How to map requesterID to sbdf
> On 
Once ?
> we have it we can deduce the streamID and the deviceID. The way to do 
> it will depend on whether we use device tree or ACPI:
>     - For device tree, the streamID, and deviceID will be equal to the 
> requester ID
what do you think should be streamID when a device is PCI EP and is 
enumerated. Also per ARM SMMU 2.0 spec  StreamID is implementation specific.
As per SMMUv3 specs
"For PCI, it is intended that StreamID is generated from the PCI 
RequesterID. The generation function may be 1:1
where one Root Complex is hosted by one SMMU"

>     - For ACPI, we would have to look up in the ACPI IORT.
>
> For the latter, I think they are static tables and therefore can be 
> parse in Xen. So we wouldn't need to PHYSDEVOP_pci_host_bridge_add to 
> pass an offset. This will also avoid any assumption that deviceID for 
> a given root complex are always contiguous and make extendable for any 
> new hardware require a different *ID.
>
> So what we really care is the requester ID. Although, I'm not sure if 
> you can find it in Xen. If not, we may need to customize (i.e adding a 
> new PHYSDEVOP) PCI add device to take a requesterID in parameter.
>
> Now, in the case of the guest, as we are only supporting device tree, 
> we could make the assumption that requesterID == deviceID as long as 
> this is exposed in a DOMCTL to allow us flexibility.
>
> It would make sense to extend DOMCTL_assign_device to take the vBDF 
> (or requesterID?) in parameter.
>
> Regards,
>