Re: [RFC] iommu_ops->domain_alloc_paging() enhancement to support AMD IOMMU driver

[Vasant Hegde <vasant.hegde@amd.com>]

Hi,


On 7/11/2024 7:26 PM, Yi Liu wrote:
> On 2024/7/11 18:15, Vasant Hegde wrote:
>> Hi Yi,
>>
>>
>> On 7/3/2024 12:27 PM, Yi Liu wrote:
>>> On 2024/7/3 13:42, Vasant Hegde wrote:
>>>> Jason,
>>>>
>>>>
>>>> On 7/1/2024 10:56 PM, Jason Gunthorpe wrote:
>>>>> On Mon, Jul 01, 2024 at 04:18:01PM +0530, Vasant Hegde wrote:


.../...

>>>
>>> https://lore.kernel.org/kvm/20240412082121.33382-1-yi.l.liu@intel.com/
>>>
>>> BTW. For AMD, can a single device be attached to multiple V1 page table?
>>> IIRC. looks like AMD has only one V1 page table pointer per BDF. But there
>>> is a PASID table which has multiple cr3 page table pointers.
>>
>> Jason covered this part in his response. Basically AMD has two page table :
>>    V1 which doesn't support PASID.
>>    V2 which support PASID
>>
>> By default when host is booted it will be in V1 page table. What we are planning
>> is we will implement domain_alloc_paging() interface in AMD driver. As part of
>> that:
>>    - PASID capable device will be placed in V2 page table, rest will be in V1
>> page table
> 
> looks like the format of V1 and V2 page tables are different. Can the V2
> page table be attached to a device (RID) and used to do address translation
> for the DMAs w/o PASID? I have an impression that a RID can only use V1
> page table. I may be wrong here. :)

We can attach V2 page table to a device. DMAs without PASID will use GCR3[0]
(PASID zero).

V1 is AMD IOMMU specific, V2 is compatible with CPU page table hence it can
support PASID.



> 
>> For UNMANAGED domain (like vfio-pci), we would need a indication from core layer
>> saying whether we want to allocate PASID capable domain or not. So that
>> internally we can allocate appropriate page table type.
>>
>> That means from VFIO layer we need to pass this detail to IOMMU layer.
>> For this part, I will start watching your patch. May be we can start discussion
>> in that thread itself.
>>
> 
> yeah. Looks like you need to pass a hint to iommu driver for domain
> allocation. For the usage vfio+iommufd, it is fine as the domain_alloc_user
> accepts a flag. For legacy vfio (using vfio iommu type1) usage, it's a bit
> tough as the domain allocation is within kernel, userspace does not have
> any input on it so far. And vfio type1 does not expose the domain concept
> to userspace, it only exposes container which is kind of an address space
> object. This means no good way for userspace to control the domain
> allocation.
> 
> If it's only a matter when PASID is enabled, perhaps we can skip vfio iommu
> type1 as it is not supposed to support PASID.

Right. My understanding is with vfio IOMMU we will not support PASID. SO it
should be fine.


-Vasant