Re: [PATCH 0/7] Enable shared device assignment

[David Hildenbrand <david@redhat.com>]

On 10.01.25 14:20, Jason Gunthorpe wrote:

Thanks for your reply, I knew CCing you would be very helpful :)

> On Fri, Jan 10, 2025 at 09:26:02AM +0100, David Hildenbrand wrote:
>>>>>>>>>>> One limitation (also discussed in the guest_memfd
>>>>>>>>>>> meeting) is that VFIO expects the DMA mapping for
>>>>>>>>>>> a specific IOVA to be mapped and unmapped with the
>>>>>>>>>>> same granularity.
> 
> Not just same granularity, whatever you map you have to unmap in
> whole. map/unmap must be perfectly paired by userspace.

Right, that's what virtio-mem ends up doing by mapping each memory block 
(e.g., 2 MiB) separately that could be unmapped separately.

It adds "overhead", but at least you don't run into "no, you cannot 
split this region because you would be out of memory/slots" or in the 
past issues with concurrent ongoing DMA.

> 
>>>>>>>>>>> such as converting a small region within a larger
>>>>>>>>>>> region. To prevent such invalid cases, all
>>>>>>>>>>> operations are performed with 4K granularity. The
>>>>>>>>>>> possible solutions we can think of are either to
>>>>>>>>>>> enable VFIO to support partial unmap
> 
> Yes, you can do that, but it is aweful for performance everywhere

Absolutely.


In your commit I read:

"Implement the cut operation to be hitless, changes to the page table
during cutting must cause zero disruption to any ongoing DMA. This is 
the expectation of the VFIO type 1 uAPI. Hitless requires HW support, it 
is incompatible with HW requiring break-before-make."

So I guess that would mean that, depending on HW support, one could 
avoid disabling large pages to still allow for atomic cuts / partial 
unmaps that don't affect concurrent DMA.


What would be your suggestion here to avoid the "map each 4k page 
individually so we can unmap it individually" ? I didn't completely 
grasp that, sorry.

 From "IIRC you can only trigger split using the VFIO type 1 legacy API. 
We would need to formalize split as an IOMMUFD native ioctl.
Nobody should use this stuf through the legacy type 1 API!!!!"

I assume you mean that we can only avoid the 4k map/unmap if we add 
proper support to IOMMUFD native ioctl, and not try making it fly 
somehow with the legacy type 1 API?

-- 
Cheers,

David / dhildenb