Re: [RFC] Experimental DMA-BUF Device Heaps

[James Jones <jajones@nvidia.com>]

On 8/17/20 8:18 AM, Brian Starkey wrote:
> Hi Ezequiel,
> 
> On Sun, Aug 16, 2020 at 02:22:46PM -0300, Ezequiel Garcia wrote:
>> This heap is basically a wrapper around DMA-API dma_alloc_attrs,
>> which will allocate memory suitable for the given device.
>>
>> The implementation is mostly a port of the Contiguous Videobuf2
>> memory allocator (see videobuf2/videobuf2-dma-contig.c)
>> over to the DMA-BUF Heap interface.
>>
>> The intention of this allocator is to provide applications
>> with a more system-agnostic API: the only thing the application
>> needs to know is which device to get the buffer for.
>>
>> Whether the buffer is backed by CMA, IOMMU or a DMA Pool
>> is unknown to the application.
>>
>> I'm not really expecting this patch to be correct or even
>> a good idea, but just submitting it to start a discussion on DMA-BUF
>> heap discovery and negotiation.
>>
> 
> My initial reaction is that I thought dmabuf heaps are meant for use
> to allocate buffers for sharing across devices, which doesn't fit very
> well with having per-device heaps.
> 
> For single-device allocations, would using the buffer allocation
> functionality of that device's native API be better in most
> cases? (Some other possibly relevant discussion at [1])
> 
> I can see that this can save some boilerplate for devices that want
> to expose private chunks of memory, but might it also lead to 100
> aliases for the system's generic coherent memory pool?
> 
> I wonder if a set of helpers to allow devices to expose whatever they
> want with minimal effort would be better.

I'm rather interested on where this goes, as I was toying with using 
some sort of heap ID as a basis for a "device-local" constraint in the 
memory constraints proposals Simon and I will be discussing at XDC this 
year.  It would be rather elegant if there was one type of heap ID used 
universally throughout the kernel that could provide a unique handle for 
the shared system memory heap(s), as well as accelerator-local heaps on 
fancy NICs, GPUs, NN accelerators, capture devices, etc. so apps could 
negotiate a location among themselves.  This patch seems to be a step 
towards that in a way, but I agree it would be counterproductive if a 
bunch of devices that were using the same underlying system memory ended 
up each getting their own heap ID just because they used some SW 
framework that worked that way.

Would appreciate it if you could send along a pointer to your BoF if it 
happens!

Thanks,
-James

> Cheers,
> -Brian
> 
> 1. https://lore.kernel.org/dri-devel/57062477-30e7-a3de-6723-a50d03a402c4@kapsi.fi/
> 
>> Given Plumbers is just a couple weeks from now, I've submitted
>> a BoF proposal to discuss this, as perhaps it would make
>> sense to discuss this live?
>>
>> Not-signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>