Re: [PATCH v3 01/18] iommu: Add device dma ownership set/release interfaces

[Lu Baolu <baolu.lu@linux.intel.com>]

On 12/6/21 11:01 PM, Jason Gunthorpe wrote:
> On Mon, Dec 06, 2021 at 02:35:55PM +0100, Joerg Roedel wrote:
>> On Mon, Dec 06, 2021 at 09:58:46AM +0800, Lu Baolu wrote:
>>> >From the perspective of who is initiating the device to do DMA, device
>>> DMA could be divided into the following types:
>>>
>>>          DMA_OWNER_DMA_API: Device DMAs are initiated by a kernel driver
>>> 			through the kernel DMA API.
>>>          DMA_OWNER_PRIVATE_DOMAIN: Device DMAs are initiated by a kernel
>>> 			driver with its own PRIVATE domain.
>>> 	DMA_OWNER_PRIVATE_DOMAIN_USER: Device DMAs are initiated by
>>> 			userspace.
>>
>> I have looked at the other iommu patches in this series, but I still
>> don't quite get what the difference in the code flow is between
>> DMA_OWNER_PRIVATE_DOMAIN and DMA_OWNER_PRIVATE_DOMAIN_USER. What are the
>> differences in the iommu core behavior based on this setting?
> 
> USER causes the IOMMU code to spend extra work to never assign the
> default domain. Lu, it would be good to update the comment with this
> detail
> 
> Once in USER mode the domain is always a /dev/null domain or a domain
> controlled by userspace. Never a domain pointing at kernel memory.

Yes. The __iommu_detach_group() re-attaches the default domain
automatically. This is not allowed once in USER mode.

I will update the comments whit this detail.

> 
>>>   struct group_device {
>>> @@ -621,6 +624,7 @@ struct iommu_group *iommu_group_alloc(void)
>>>   	INIT_LIST_HEAD(&group->devices);
>>>   	INIT_LIST_HEAD(&group->entry);
>>>   	BLOCKING_INIT_NOTIFIER_HEAD(&group->notifier);
>>> +	group->dma_owner = DMA_OWNER_NONE;
>>
>>
>> DMA_OWNER_NONE is also questionable. All devices are always in one
>> domain, and the default domain is always the one used for DMA-API, so
>> why isn't the initial value DMA_OWNER_DMA_API?
> 
> 'NONE' means the group is in the default domain but no driver is bound
> and thus DMA isn't being used. Seeing NONE is the only condition when
> it is OK to change the domain.
> 
> This could be reworked to instead rely on the refcount == 0 as the
> signal to know it is OK to change the domain and then we never have
> NONE at all. Lu?

NONE is just a parking state. It's okay to rely on the "refcount == 0"
for state transition as far as I see. I will work towards this.

Best regards,
baolu