From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36D26C433DF for ; Sun, 23 Aug 2020 20:05:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 01F2F206B5 for ; Sun, 23 Aug 2020 20:05:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="sBsMEyZR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726737AbgHWUFm (ORCPT ); Sun, 23 Aug 2020 16:05:42 -0400 Received: from hqnvemgate25.nvidia.com ([216.228.121.64]:6646 "EHLO hqnvemgate25.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725887AbgHWUFm (ORCPT ); Sun, 23 Aug 2020 16:05:42 -0400 Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Sun, 23 Aug 2020 13:04:40 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Sun, 23 Aug 2020 13:05:41 -0700 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Sun, 23 Aug 2020 13:05:41 -0700 Received: from [10.2.52.145] (10.124.1.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 23 Aug 2020 20:05:34 +0000 Subject: Re: [RFC] Experimental DMA-BUF Device Heaps To: Ezequiel Garcia , Brian Starkey CC: , , "Sumit Semwal" , "Andrew F . Davis" , "Benjamin Gaignard" , Liam Mark , "Laura Abbott" , John Stultz , "Laurent Pinchart" , Daniel Stone , Nicolas Dufresne , Robert Beckett , Tomasz Figa , , References: <20200816172246.69146-1-ezequiel@collabora.com> <20200817151813.wet5faqg4fzlfbsh@DESKTOP-E1NTVVP.localdomain> <746a0bb75bd8388a30b53a5ddc56fb24aea308a8.camel@collabora.com> From: James Jones X-Nvconfidentiality: public Message-ID: <4f987e06-7af1-671b-836a-10d3e9623547@nvidia.com> Date: Sun, 23 Aug 2020 13:04:43 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <746a0bb75bd8388a30b53a5ddc56fb24aea308a8.camel@collabora.com> X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1598213081; bh=5pEzn13QiMIX8L39EgxTkAwhyB4PlbqROzdyj4lEnDg=; h=X-PGP-Universal:Subject:To:CC:References:From:X-Nvconfidentiality: Message-ID:Date:User-Agent:MIME-Version:In-Reply-To: X-Originating-IP:X-ClientProxiedBy:Content-Type:Content-Language: Content-Transfer-Encoding; b=sBsMEyZRa9yhgcTl58ozTZDo1KR/wFiD5SuigXnn1Xiyecdv4Yu2J/3KFeZTe/hYi dTNN9fwbaHEDK7ezDbZ0FxDaxg7NDCYZc8Uv9qpvYj6FQLgXlMIzLjHE40U2EjRJCB 0h4aVSorzDt4+ag2FiPLxc+7vLuZVwyYQ07wzP3756uRQZonEl4DQOIVGESLgeI4qC tU1oDuiWM4cel+IU9oZOzHGgr7UGnjH6qoGWTfp2H/7aFbZC8TPXUvv1mBl1kZtv8I J5G0r81LrKBaJzhl0fMCPOPrRL8il8MOeoQ681KmBmyturTwH1PNecxsOe6umz1wF+ mfwVO2Yh/pc0Q== Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org On 8/20/20 1:15 AM, Ezequiel Garcia wrote: > On Mon, 2020-08-17 at 20:49 -0700, James Jones wrote: >> On 8/17/20 8:18 AM, Brian Starkey wrote: >>> Hi Ezequiel, >>> >>> On Sun, Aug 16, 2020 at 02:22:46PM -0300, Ezequiel Garcia wrote: >>>> This heap is basically a wrapper around DMA-API dma_alloc_attrs, >>>> which will allocate memory suitable for the given device. >>>> >>>> The implementation is mostly a port of the Contiguous Videobuf2 >>>> memory allocator (see videobuf2/videobuf2-dma-contig.c) >>>> over to the DMA-BUF Heap interface. >>>> >>>> The intention of this allocator is to provide applications >>>> with a more system-agnostic API: the only thing the application >>>> needs to know is which device to get the buffer for. >>>> >>>> Whether the buffer is backed by CMA, IOMMU or a DMA Pool >>>> is unknown to the application. >>>> >>>> I'm not really expecting this patch to be correct or even >>>> a good idea, but just submitting it to start a discussion on DMA-BUF >>>> heap discovery and negotiation. >>>> >>> >>> My initial reaction is that I thought dmabuf heaps are meant for use >>> to allocate buffers for sharing across devices, which doesn't fit very >>> well with having per-device heaps. >>> >>> For single-device allocations, would using the buffer allocation >>> functionality of that device's native API be better in most >>> cases? (Some other possibly relevant discussion at [1]) >>> >>> I can see that this can save some boilerplate for devices that want >>> to expose private chunks of memory, but might it also lead to 100 >>> aliases for the system's generic coherent memory pool? >>> >>> I wonder if a set of helpers to allow devices to expose whatever they >>> want with minimal effort would be better. >> >> I'm rather interested on where this goes, as I was toying with using >> some sort of heap ID as a basis for a "device-local" constraint in the >> memory constraints proposals Simon and I will be discussing at XDC this >> year. It would be rather elegant if there was one type of heap ID used >> universally throughout the kernel that could provide a unique handle for >> the shared system memory heap(s), as well as accelerator-local heaps on >> fancy NICs, GPUs, NN accelerators, capture devices, etc. so apps could >> negotiate a location among themselves. This patch seems to be a step >> towards that in a way, but I agree it would be counterproductive if a >> bunch of devices that were using the same underlying system memory ended >> up each getting their own heap ID just because they used some SW >> framework that worked that way. >> >> Would appreciate it if you could send along a pointer to your BoF if it >> happens! >> > > Here is it: > > https://linuxplumbersconf.org/event/7/contributions/818/ > > It would be great to see you there and discuss this, > given I was hoping we could talk about how to meet a > userspace allocator library expectations as well. Thanks! I hadn't registered for LPC and it looks like it's sold out, but I'll try to watch the live stream. This is very interesting, in that it looks like we're both trying to solve roughly the same set of problems but approaching it from different angles. From what I gather, your approach is that a "heap" encompasses all the allocation constraints a device may have. The approach Simon Ser and I are tossing around so far is somewhat different, but may potentially leverage dma-buf heaps a bit as well. Our approach looks more like what I described at XDC a few years ago, where memory constraints for a given device's usage of an image are exposed up to applications, which can then somehow perform boolean intersection/union operations on them to arrive at a common set of constraints that describe something compatible with all the devices & usages desired (or fail to do so, and fall back to copying things around presumably). I believe this is more flexible than your initial proposal in that devices often support multiple usages (E.g., different formats, different proprietary layouts represented by format modifiers, etc.), and it avoids adding a combinatorial number of heaps to manage that. In my view, heaps are more like blobs of memory that can be allocated from in various different ways to satisfy constraints. I realize heaps mean something specific in the dma-buf heap design (specifically, something closer to an association between an "allocation mechanism" and "physical memory"), but I hope we don't have massive heap/allocator mechanism proliferation due to constraints alone. Perhaps some constraints, such as contiguous memory or device-local memory, are properly expressed as a specific heap, but consider the proliferation implied by even that simple pair of examples: How do you express contiguous device-local memory? Do you need to spawn two heaps on the underlying device-local memory, one for contiguous allocations and one for non-contiguous allocations? Seems excessive. Of course, our approach also has downsides and is still being worked on. For example, it works best in an ideal world where all the allocators available understand all the constraints that exist. Dealing with a reality where there are probably a handful of allocators, another handful of userspace libraries and APIs, and still more applications trying to make use of all this is one of the larger remaining challenges of the design. We'll present our work at XDC 2020. Hope you can check that out as well! Thanks, -James > Thanks, > Ezequiel > >> Thanks, >> -James >> >>> Cheers, >>> -Brian >>> >>> 1. https://lore.kernel.org/dri-devel/57062477-30e7-a3de-6723-a50d03a402c4@kapsi.fi/ >>> >>>> Given Plumbers is just a couple weeks from now, I've submitted >>>> a BoF proposal to discuss this, as perhaps it would make >>>> sense to discuss this live? >>>> >>>> Not-signed-off-by: Ezequiel Garcia > >