From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Stultz Subject: Discussion starters for ION session at Linux Plumbers Android+Graphics microconf Date: Thu, 5 Sep 2013 17:49:38 -0700 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0829059438==" Return-path: Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by gabe.freedesktop.org (Postfix) with ESMTP id 86996E5CC7 for ; Thu, 5 Sep 2013 17:49:39 -0700 (PDT) Received: by mail-vc0-f182.google.com with SMTP id hf12so1716371vcb.13 for ; Thu, 05 Sep 2013 17:49:38 -0700 (PDT) List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org Errors-To: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org To: dri-devel@lists.freedesktop.org Cc: Android Kernel Team , Laurent Pinchart List-Id: dri-devel@lists.freedesktop.org --===============0829059438== Content-Type: multipart/alternative; boundary=001a11c2c674c7900d04e5ac69fc --001a11c2c674c7900d04e5ac69fc Content-Type: text/plain; charset=ISO-8859-1 Hey everyone, In preparation for the Plumbers Android+Graphics microconf, I wanted to send out some background documentation to try to get all the context we can out there prior to the discussion, as time will be limited and it would be best to spend it discussing solutions rather then re-hashing problems and requirements. I'm sure many folks on this list could probably do a better job summarizing the issues, but I wanted to get this out there to try to enumerate the problems and the different perspectives on the issues that I'm aware of. The document is on LWN here: http://lwn.net/SubscriberLink/565469/9d88daa2282ef6c2/ But I wanted to start a discussion thread here, since the LWN comment threads (while *much* better then most comment sections) aren't really the right place for this sort of thing. So please feel free to reply to correct any inaccuracies in my summary, provide your thoughts on the various proposed solutions, or suggest new solutions that we should also discuss at the micro-conference! thanks -john --001a11c2c674c7900d04e5ac69fc Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hey everyone,
=A0 =A0In preparation for the Plumbe= rs Android+Graphics microconf, I wanted to send out some background documen= tation to try to get all the context we can out there prior to the discussi= on, as time will be limited and it would be best to spend it discussing sol= utions rather then re-hashing problems and requirements.

I'm sure many folks on this list could probably do a better job sum= marizing the issues, but I wanted to get this out there to try to enumerate= the problems and the different perspectives on the issues that I'm awa= re of.

The document is on LWN here:
http://lwn.net/SubscriberLink/565469/9d88daa228= 2ef6c2/

But I wanted to start a discussion thread here, since the LWN comment t= hreads (while *much* better then most comment sections) aren't really t= he right place for this sort of thing.

So please feel free to reply = to correct any inaccuracies in my summary, provide your thoughts on the var= ious proposed solutions, or suggest new solutions that we should also discu= ss at the micro-conference!

thanks
-john
--001a11c2c674c7900d04e5ac69fc-- --===============0829059438== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel --===============0829059438==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rob Clark Subject: Re: Discussion starters for ION session at Linux Plumbers Android+Graphics microconf Date: Thu, 5 Sep 2013 23:26:34 -0400 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail-ie0-f175.google.com (mail-ie0-f175.google.com [209.85.223.175]) by gabe.freedesktop.org (Postfix) with ESMTP id ADD28E5EA5 for ; Thu, 5 Sep 2013 20:26:34 -0700 (PDT) Received: by mail-ie0-f175.google.com with SMTP id u16so5786200iet.34 for ; Thu, 05 Sep 2013 20:26:34 -0700 (PDT) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org Errors-To: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org To: John Stultz Cc: Android Kernel Team , Laurent Pinchart , "dri-devel@lists.freedesktop.org" List-Id: dri-devel@lists.freedesktop.org On Thu, Sep 5, 2013 at 8:49 PM, John Stultz wrote: > Hey everyone, > In preparation for the Plumbers Android+Graphics microconf, I wanted to > send out some background documentation to try to get all the context we can > out there prior to the discussion, as time will be limited and it would be > best to spend it discussing solutions rather then re-hashing problems and > requirements. > > I'm sure many folks on this list could probably do a better job summarizing > the issues, but I wanted to get this out there to try to enumerate the > problems and the different perspectives on the issues that I'm aware of. > > The document is on LWN here: > http://lwn.net/SubscriberLink/565469/9d88daa2282ef6c2/ oh, I had missed that article.. fwiw "Another possible solution is to allow dma-buf exporters to not allocate the backing buffers immediately. This would allow multiple drivers to attach to a dma-buf before the allocation occurs. Then, when the buffer is first used, the allocation is done; at that time, the allocator could scan the list of attached drivers and be able to determine the constraints of the attached devices and allocate memory accordingly. This would allow user space to not have to deal with any constraint solving. " That is actually how dma-buf works today. And at least with GEM buffers exported as dma-buf's, the allocation is deferred. It does require attaching the buffers in all the devices that will be sharing the buffer up front (but I suppose you need to know the involved devices one way or another with any solution, so this approach seems as good as any). We *do* still need to spiff up dev->dma_parms a bit more, and there might be some room for some helpers to figure out the union of all attached devices constraints, and allocate suitable backing pages... so perhaps this is one thing we should be talking about. At any rate, it might not matter if ION cannot import dma-buf's (as long as every other device importing does not have to differentiate between importing dma-buf's that are also ION buffers vs dma-buf's allocated in some other way). But to be useful upstream, we'd have to ensure that existing drm drivers can somehow plug-in their existing allocation mechanisms in to ION. BR, -R > But I wanted to start a discussion thread here, since the LWN comment > threads (while *much* better then most comment sections) aren't really the > right place for this sort of thing. > > So please feel free to reply to correct any inaccuracies in my summary, > provide your thoughts on the various proposed solutions, or suggest new > solutions that we should also discuss at the micro-conference! > > thanks > -john > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/dri-devel > From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Stultz Subject: Re: Discussion starters for ION session at Linux Plumbers Android+Graphics microconf Date: Thu, 05 Sep 2013 22:06:52 -0700 Message-ID: <522962EC.9030506@linaro.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail-pd0-f178.google.com (mail-pd0-f178.google.com [209.85.192.178]) by gabe.freedesktop.org (Postfix) with ESMTP id C459EE5DCB for ; Thu, 5 Sep 2013 22:06:55 -0700 (PDT) Received: by mail-pd0-f178.google.com with SMTP id w10so2740087pde.23 for ; Thu, 05 Sep 2013 22:06:55 -0700 (PDT) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org Errors-To: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org To: Rob Clark Cc: Android Kernel Team , Laurent Pinchart , "dri-devel@lists.freedesktop.org" List-Id: dri-devel@lists.freedesktop.org On 09/05/2013 08:26 PM, Rob Clark wrote: > On Thu, Sep 5, 2013 at 8:49 PM, John Stultz wrote: >> Hey everyone, >> In preparation for the Plumbers Android+Graphics microconf, I wanted to >> send out some background documentation to try to get all the context we can >> out there prior to the discussion, as time will be limited and it would be >> best to spend it discussing solutions rather then re-hashing problems and >> requirements. >> >> I'm sure many folks on this list could probably do a better job summarizing >> the issues, but I wanted to get this out there to try to enumerate the >> problems and the different perspectives on the issues that I'm aware of. >> >> The document is on LWN here: >> http://lwn.net/SubscriberLink/565469/9d88daa2282ef6c2/ > oh, I had missed that article.. fwiw It was published just moments before I sent out this thread, so I wouldn't have expected anyone to have seen it yet. :) > "Another possible solution is to allow dma-buf exporters to not > allocate the backing buffers immediately. This would allow multiple > drivers to attach to a dma-buf before the allocation occurs. Then, > when the buffer is first used, the allocation is done; at that time, > the allocator could scan the list of attached drivers and be able to > determine the constraints of the attached devices and allocate memory > accordingly. This would allow user space to not have to deal with any > constraint solving. " > > That is actually how dma-buf works today. And at least with GEM > buffers exported as dma-buf's, the allocation is deferred. It does > require attaching the buffers in all the devices that will be sharing > the buffer up front (but I suppose you need to know the involved > devices one way or another with any solution, so this approach seems > as good as any). We *do* still need to spiff up dev->dma_parms a bit > more, and there might be some room for some helpers to figure out the > union of all attached devices constraints, and allocate suitable > backing pages... so perhaps this is one thing we should be talking > about. Ok. I had gone looking for an example of the deferred allocation, but didn't find it. I'll go look again, but if you have a pointer, that could be useful. So yea, I do think this is the most promising approach, but sorting out the next steps for doing a proof of concept is one thing I'd like to discuss (as mentioned in the article, via a ion-like generic allocator, or trying to wire in the constraint solving to some limited set of drivers via generic helper functions). As well as getting a better understanding the Android developers concern about any non-deterministic cost of allocating at mmap time. Thanks for the feedback and thoughts! I'm hopeful some approach to resolving the various issues can be found, but I suspect it will have a few different parts. thanks -john From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Vetter Subject: Re: Discussion starters for ION session at Linux Plumbers Android+Graphics microconf Date: Fri, 6 Sep 2013 11:16:57 +0200 Message-ID: <20130906091657.GR27291@phenom.ffwll.local> References: <522962EC.9030506@linaro.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail-ea0-f173.google.com (mail-ea0-f173.google.com [209.85.215.173]) by gabe.freedesktop.org (Postfix) with ESMTP id 142C2E5C87 for ; Fri, 6 Sep 2013 02:16:44 -0700 (PDT) Received: by mail-ea0-f173.google.com with SMTP id g10so1456526eak.18 for ; Fri, 06 Sep 2013 02:16:44 -0700 (PDT) Content-Disposition: inline In-Reply-To: <522962EC.9030506@linaro.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org Errors-To: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org To: John Stultz Cc: Android Kernel Team , Laurent Pinchart , "dri-devel@lists.freedesktop.org" List-Id: dri-devel@lists.freedesktop.org On Thu, Sep 05, 2013 at 10:06:52PM -0700, John Stultz wrote: > On 09/05/2013 08:26 PM, Rob Clark wrote: > > On Thu, Sep 5, 2013 at 8:49 PM, John Stultz wrote: > >> Hey everyone, > >> In preparation for the Plumbers Android+Graphics microconf, I wanted to > >> send out some background documentation to try to get all the context we can > >> out there prior to the discussion, as time will be limited and it would be > >> best to spend it discussing solutions rather then re-hashing problems and > >> requirements. > >> > >> I'm sure many folks on this list could probably do a better job summarizing > >> the issues, but I wanted to get this out there to try to enumerate the > >> problems and the different perspectives on the issues that I'm aware of. > >> > >> The document is on LWN here: > >> http://lwn.net/SubscriberLink/565469/9d88daa2282ef6c2/ > > oh, I had missed that article.. fwiw > > It was published just moments before I sent out this thread, so I > wouldn't have expected anyone to have seen it yet. :) > > > > "Another possible solution is to allow dma-buf exporters to not > > allocate the backing buffers immediately. This would allow multiple > > drivers to attach to a dma-buf before the allocation occurs. Then, > > when the buffer is first used, the allocation is done; at that time, > > the allocator could scan the list of attached drivers and be able to > > determine the constraints of the attached devices and allocate memory > > accordingly. This would allow user space to not have to deal with any > > constraint solving. " > > > > That is actually how dma-buf works today. And at least with GEM > > buffers exported as dma-buf's, the allocation is deferred. It does > > require attaching the buffers in all the devices that will be sharing > > the buffer up front (but I suppose you need to know the involved > > devices one way or another with any solution, so this approach seems > > as good as any). We *do* still need to spiff up dev->dma_parms a bit > > more, and there might be some room for some helpers to figure out the > > union of all attached devices constraints, and allocate suitable > > backing pages... so perhaps this is one thing we should be talking > > about. > > Ok. I had gone looking for an example of the deferred allocation, but > didn't find it. I'll go look again, but if you have a pointer, that > could be useful. > > So yea, I do think this is the most promising approach, but sorting out > the next steps for doing a proof of concept is one thing I'd like to > discuss (as mentioned in the article, via a ion-like generic allocator, > or trying to wire in the constraint solving to some limited set of > drivers via generic helper functions). As well as getting a better > understanding the Android developers concern about any non-deterministic > cost of allocating at mmap time. > > > Thanks for the feedback and thoughts! I'm hopeful some approach to > resolving the various issues can be found, but I suspect it will have a > few different parts. My main gripe with ION is that it creates a parallel infrastructure for figuring out allocation constraints of devices. Upstream already has all the knowledge (or at least most of it) for cache flushing, mapping into iommus and allocating from special pools stored in association with the device structure. So imo an upstream ION thing should reuse the information each device and its driver already has available. Now I also see that a central allocator has upsides since reinventing this wheel for every device driver is not a great idea. One idea to get there and keep the benefits of ION with up-front allocations would be. 1) Allcoate the dma-buf handle at the central allocator. No backing storage gets allocated. 2) Import that dma-buf everywhere you want it to be used. That way userspace doesn't need to deal with whatever hw madness is actually used to implement the drm/v4l/whatever devices nodes internally. 3) Ask the central allocator to finalize the buffer allocation placement and grab backing storage. If any further attaching happens that doesn't work out it would simply fail, and userspace gets to keep the pieces. Which is already the case in today's upstream when userspace is unlucky and doesn't pick the most constrained device. This only tackles the "make memory allocation predictable" issue ION solves, which leaves the optimized cache flushing. We can add caches for pre-flushed objects for that (not rocket science, most of the drm drivers have that wheel reinvented, too). That leaves us with optimizing cache flushes (i.e. leaving them out when switching between devices without cpu accesss in-between). The current linux dma api doesn't really support this, so we need to add a bit of interfaces there to be able to do device-to-device cache flushing (which safe for maybe iommu flushes I expect to be noops). And the central allocator obviously needs to keep track of where the current cache domain is. Aside: Intel Atom SoCs have the same cache flushing challenges since all the gfx blocks (gpu, display, camera, ...) prefer direct main memory access that bypasses gpu caches. Big core stuff is obviously different and fully coherent. So we need a solution for this, too, but unfortunately the camera driver guys haven't yet managed to up stream their driver so not possible for us to demonstrate anything on upstream :( Same story as everywhere else in SoC-land I guess ... Now one thing I've missed from your article on the GEM vs. ION topic is that gem allows buffers to be swapped out. That works by allocating shmemfs nodes, but that doesn't really work together nicely with the current linux dma apis. Which means that drivers have a bunch of hacks to work around this (and ttm has an entire page cache as a 2nd allocation step to get at the right dma api allocated pages). There's been the occasional talk about a gemfs to rectify these allocation issues. If we'd merge this with the central allocator and optionally allow it to swap out/move backing storage pages (and also back them with a fs node ofc) then we could rip out a bit code from drm drivers. I also think that this way would be the only approach to actually make PRIME work together with IOMMUs. There's some really old patches from Chris Wilson to teach i915-gem to directly manage the backing storage swapping, so patching this into the central allocator shouldn't be too nefarious. So that's my rough sketch of the brave new world I have in mind. Please poke holes ;-) Cheers, Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rob Clark Subject: Re: Discussion starters for ION session at Linux Plumbers Android+Graphics microconf Date: Fri, 6 Sep 2013 07:50:10 -0400 Message-ID: References: <522962EC.9030506@linaro.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail-ie0-f175.google.com (mail-ie0-f175.google.com [209.85.223.175]) by gabe.freedesktop.org (Postfix) with ESMTP id 97527435B6 for ; Fri, 6 Sep 2013 04:50:11 -0700 (PDT) Received: by mail-ie0-f175.google.com with SMTP id u16so6594249iet.34 for ; Fri, 06 Sep 2013 04:50:11 -0700 (PDT) In-Reply-To: <522962EC.9030506@linaro.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org Errors-To: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org To: John Stultz Cc: Android Kernel Team , Laurent Pinchart , "dri-devel@lists.freedesktop.org" List-Id: dri-devel@lists.freedesktop.org On Fri, Sep 6, 2013 at 1:06 AM, John Stultz wrote: > On 09/05/2013 08:26 PM, Rob Clark wrote: >> On Thu, Sep 5, 2013 at 8:49 PM, John Stultz wrote: >>> Hey everyone, >>> In preparation for the Plumbers Android+Graphics microconf, I wanted to >>> send out some background documentation to try to get all the context we can >>> out there prior to the discussion, as time will be limited and it would be >>> best to spend it discussing solutions rather then re-hashing problems and >>> requirements. >>> >>> I'm sure many folks on this list could probably do a better job summarizing >>> the issues, but I wanted to get this out there to try to enumerate the >>> problems and the different perspectives on the issues that I'm aware of. >>> >>> The document is on LWN here: >>> http://lwn.net/SubscriberLink/565469/9d88daa2282ef6c2/ >> oh, I had missed that article.. fwiw > > It was published just moments before I sent out this thread, so I > wouldn't have expected anyone to have seen it yet. :) > ahh, ok, that would explain it ;-) > >> "Another possible solution is to allow dma-buf exporters to not >> allocate the backing buffers immediately. This would allow multiple >> drivers to attach to a dma-buf before the allocation occurs. Then, >> when the buffer is first used, the allocation is done; at that time, >> the allocator could scan the list of attached drivers and be able to >> determine the constraints of the attached devices and allocate memory >> accordingly. This would allow user space to not have to deal with any >> constraint solving. " >> >> That is actually how dma-buf works today. And at least with GEM >> buffers exported as dma-buf's, the allocation is deferred. It does >> require attaching the buffers in all the devices that will be sharing >> the buffer up front (but I suppose you need to know the involved >> devices one way or another with any solution, so this approach seems >> as good as any). We *do* still need to spiff up dev->dma_parms a bit >> more, and there might be some room for some helpers to figure out the >> union of all attached devices constraints, and allocate suitable >> backing pages... so perhaps this is one thing we should be talking >> about. > > Ok. I had gone looking for an example of the deferred allocation, but > didn't find it. I'll go look again, but if you have a pointer, that > could be useful. for a "pure GEM" (ie. not TTM) driver, drm_gem_get_pages() is where actual pages get allocated. This is triggered by various things (faulting in page for userspace access, dmabuf map_attachment, etc) BR, -R > So yea, I do think this is the most promising approach, but sorting out > the next steps for doing a proof of concept is one thing I'd like to > discuss (as mentioned in the article, via a ion-like generic allocator, > or trying to wire in the constraint solving to some limited set of > drivers via generic helper functions). As well as getting a better > understanding the Android developers concern about any non-deterministic > cost of allocating at mmap time. > > > Thanks for the feedback and thoughts! I'm hopeful some approach to > resolving the various issues can be found, but I suspect it will have a > few different parts. > > thanks > -john > > > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rob Clark Subject: Re: Discussion starters for ION session at Linux Plumbers Android+Graphics microconf Date: Fri, 6 Sep 2013 08:13:15 -0400 Message-ID: References: <522962EC.9030506@linaro.org> <20130906091657.GR27291@phenom.ffwll.local> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail-ie0-f173.google.com (mail-ie0-f173.google.com [209.85.223.173]) by gabe.freedesktop.org (Postfix) with ESMTP id 6BBDBE5EBE for ; Fri, 6 Sep 2013 05:13:16 -0700 (PDT) Received: by mail-ie0-f173.google.com with SMTP id qa5so5524518ieb.32 for ; Fri, 06 Sep 2013 05:13:16 -0700 (PDT) In-Reply-To: <20130906091657.GR27291@phenom.ffwll.local> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org Errors-To: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org To: Daniel Vetter Cc: Android Kernel Team , Laurent Pinchart , "dri-devel@lists.freedesktop.org" List-Id: dri-devel@lists.freedesktop.org On Fri, Sep 6, 2013 at 5:16 AM, Daniel Vetter wrote: > On Thu, Sep 05, 2013 at 10:06:52PM -0700, John Stultz wrote: >> On 09/05/2013 08:26 PM, Rob Clark wrote: >> > On Thu, Sep 5, 2013 at 8:49 PM, John Stultz wrote: >> >> Hey everyone, >> >> In preparation for the Plumbers Android+Graphics microconf, I wanted to >> >> send out some background documentation to try to get all the context we can >> >> out there prior to the discussion, as time will be limited and it would be >> >> best to spend it discussing solutions rather then re-hashing problems and >> >> requirements. >> >> >> >> I'm sure many folks on this list could probably do a better job summarizing >> >> the issues, but I wanted to get this out there to try to enumerate the >> >> problems and the different perspectives on the issues that I'm aware of. >> >> >> >> The document is on LWN here: >> >> http://lwn.net/SubscriberLink/565469/9d88daa2282ef6c2/ >> > oh, I had missed that article.. fwiw >> >> It was published just moments before I sent out this thread, so I >> wouldn't have expected anyone to have seen it yet. :) >> >> >> > "Another possible solution is to allow dma-buf exporters to not >> > allocate the backing buffers immediately. This would allow multiple >> > drivers to attach to a dma-buf before the allocation occurs. Then, >> > when the buffer is first used, the allocation is done; at that time, >> > the allocator could scan the list of attached drivers and be able to >> > determine the constraints of the attached devices and allocate memory >> > accordingly. This would allow user space to not have to deal with any >> > constraint solving. " >> > >> > That is actually how dma-buf works today. And at least with GEM >> > buffers exported as dma-buf's, the allocation is deferred. It does >> > require attaching the buffers in all the devices that will be sharing >> > the buffer up front (but I suppose you need to know the involved >> > devices one way or another with any solution, so this approach seems >> > as good as any). We *do* still need to spiff up dev->dma_parms a bit >> > more, and there might be some room for some helpers to figure out the >> > union of all attached devices constraints, and allocate suitable >> > backing pages... so perhaps this is one thing we should be talking >> > about. >> >> Ok. I had gone looking for an example of the deferred allocation, but >> didn't find it. I'll go look again, but if you have a pointer, that >> could be useful. >> >> So yea, I do think this is the most promising approach, but sorting out >> the next steps for doing a proof of concept is one thing I'd like to >> discuss (as mentioned in the article, via a ion-like generic allocator, >> or trying to wire in the constraint solving to some limited set of >> drivers via generic helper functions). As well as getting a better >> understanding the Android developers concern about any non-deterministic >> cost of allocating at mmap time. >> >> >> Thanks for the feedback and thoughts! I'm hopeful some approach to >> resolving the various issues can be found, but I suspect it will have a >> few different parts. > > My main gripe with ION is that it creates a parallel infrastructure for > figuring out allocation constraints of devices. Upstream already has all > the knowledge (or at least most of it) for cache flushing, mapping into > iommus and allocating from special pools stored in association with the > device structure. So imo an upstream ION thing should reuse the > information each device and its driver already has available. yeah, we want to make sure that dma-mapping is up to snuff for handling allocations of backing pages meeting the constraints of a set of devices (spiffing up dma_parms, etc, as I mentioned in my first reply). I see a potential upstream ION as just be a sort of convenience wrapper for android userspace rather than an actual allocator of backing pages, etc. Well, maybe some of this is easier to do in userspace/gralloc, but for example to ease "jank" fears, it could pre-attach to all the involved devices for the use-case, and then do a dummy map_attachment to the ION device to force backing page allocation. BR, -R > Now I also see that a central allocator has upsides since reinventing this > wheel for every device driver is not a great idea. One idea to get there > and keep the benefits of ION with up-front allocations would be. > 1) Allcoate the dma-buf handle at the central allocator. No backing > storage gets allocated. > 2) Import that dma-buf everywhere you want it to be used. That way > userspace doesn't need to deal with whatever hw madness is actually used > to implement the drm/v4l/whatever devices nodes internally. > 3) Ask the central allocator to finalize the buffer allocation placement > and grab backing storage. > > If any further attaching happens that doesn't work out it would simply > fail, and userspace gets to keep the pieces. Which is already the case in > today's upstream when userspace is unlucky and doesn't pick the most > constrained device. > > This only tackles the "make memory allocation predictable" issue ION > solves, which leaves the optimized cache flushing. We can add caches for > pre-flushed objects for that (not rocket science, most of the drm drivers > have that wheel reinvented, too). That leaves us with optimizing cache > flushes (i.e. leaving them out when switching between devices without cpu > accesss in-between). The current linux dma api doesn't really support > this, so we need to add a bit of interfaces there to be able to do > device-to-device cache flushing (which safe for maybe iommu flushes I > expect to be noops). And the central allocator obviously needs to keep > track of where the current cache domain is. > > Aside: Intel Atom SoCs have the same cache flushing challenges since all > the gfx blocks (gpu, display, camera, ...) prefer direct main memory > access that bypasses gpu caches. Big core stuff is obviously different and > fully coherent. So we need a solution for this, too, but unfortunately the > camera driver guys haven't yet managed to up stream their driver so not > possible for us to demonstrate anything on upstream :( Same story as > everywhere else in SoC-land I guess ... > > Now one thing I've missed from your article on the GEM vs. ION topic is > that gem allows buffers to be swapped out. That works by allocating > shmemfs nodes, but that doesn't really work together nicely with the > current linux dma apis. Which means that drivers have a bunch of hacks to > work around this (and ttm has an entire page cache as a 2nd allocation > step to get at the right dma api allocated pages). > > There's been the occasional talk about a gemfs to rectify these allocation > issues. If we'd merge this with the central allocator and optionally allow > it to swap out/move backing storage pages (and also back them with a fs > node ofc) then we could rip out a bit code from drm drivers. I also think > that this way would be the only approach to actually make PRIME work > together with IOMMUs. There's some really old patches from Chris Wilson to > teach i915-gem to directly manage the backing storage swapping, so > patching this into the central allocator shouldn't be too nefarious. > > So that's my rough sketch of the brave new world I have in mind. Please > poke holes ;-) > > Cheers, Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch