Re: Support SVM without PASID

[valmiki <valmikibow@gmail.com>]

On 8/2/2017 12:10 AM, Jean-Philippe Brucker wrote:
> On 01/08/17 18:38, valmiki wrote:
> [...]
>>>> So i digged through your patches and i understood that using BIND ioctls
>>>> satge-1 translations are setup in SMMU for an application.
>>>> If we use VFIO_IOMMU_MAP/UNMAP_DMA ioctls they are setting up stage-2
>>>> translations in SMMU.
>>>> So without PASID support we can only work with stage-2 translations ?
>>>
>>> It depends what type you use when registering the IOMMU with
>>> VFIO_SET_IOMMU:
>>>
>>> * If the type is VFIO_TYPE1v2_IOMMU, then VFIO_IOMMU_MAP/UNMAP_DMA
>>>   affects the stage-1 non-PASID context (already the case in mainline).
>>>   In addition, with my patch the BIND ioctl will affect stage-1 PASID
>>>   contexts, and bind process page directories to the device (host SVM).
>>>
>>> * If the type is VFIO_TYPE1_NESTING_IOMMU, then VFIO_IOMMU_MAP/UNMAP_DMA
>>>   will affect stage-2 mappings (already in mainline).
>>>   With my SMMU patch series, the BIND ioctl is not supported in this mode.
>>>   But in the future, BIND would allow to manage stage-1 as well:
>>>   - bind a process page directory (host SVM with added stage-2), or
>>>   - bind a guest page directory (viommu), or
>>>   - bind the full stage-1 context table (viommu).
>>
>> Hi Jean, Thanks a lot for this information.
>> I tried to understand how stage-1 translations are setup if we use
>> VFIO_TYPE1v2_IOMMU & VFIO_IOMMU_MAP/UNMAP_DMA flow, but I'm not
>> successful, I couldn't find when context descriptor details are updated
>> with stage-1 translation table details in this flow.
>> I found that in BIND flow a new context descriptor being created and
>> assigned with required translation tables.
>>
>> Can you please point to piece of code/patch series where and how context
>> descriptor is updated for struct vfio_iommu_type1_dma_map/UNMAP_DMA flow with
>> VFIO_TYPE1v2_IOMMU.
>
> What happens for SMMU is during initialization, the
> VFIO_GROUP_SET_CONTAINER ioctl calls vfio_iommu_type1_attach_group.
> This function creates a new iommu_domain, and then calls
> iommu_attach_group(domain, group), which calls arm_smmu_attach_dev(domain,
> dev) for all devices in the group.
>
> If necessary we detach the existing domain, by clearing the STE (and all
> contexts). Then, since the new domain is only partially initialized, we
> call arm_smmu_domain_finalise, which allocates the page tables, and ends
> up calling arm_smmu_domain_finalise_s1. That function prepares the
> configuration of context descriptor 0, which is used for non-PASID
> mappings. Afterwards, back in arm_smmu_attach_dev, context descriptor 0 is
> written, along with the STE, and then the domain is live.
>
> Then when userspace does VFIO_IOMMU_MAP/UNMAP_DMA, iommu_map/unmap are
> called on that domain. arm_smmu_map/unmap update the page tables attached
> to context descriptor 0 (but they don't modify the CD in any way).
>
Hi Jean, Thanks a lot, now i understood the flow. From vfio kernel 
documentation we fill vaddr and iova in struct vfio_iommu_type1_dma_map 
and pass them to VFIO. But if we use dynamic allocation in application 
(say malloc), do we need to use dma API to get iova and then call 
VFIO_IOMMU_MAP ioctl ?
If application needs multiple such dynamic allocations, then it need to 
allocate large chunk and program it via VFIO_IOMMU_MAP ioctl and then 
manage rest allocations requirements from this buffer ?

Regards,
valmiki