From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christoph Hellwig Subject: Re: [Linaro-mm-sig] [PATCH 4/8] dma-buf: add peer2peer flag Date: Tue, 24 Apr 2018 11:48:47 -0700 Message-ID: <20180424184847.GA3247@infradead.org> References: <20180403180832.GZ3881@phenom.ffwll.local> <20180416123937.GA9073@infradead.org> <20180419081657.GA16735@infradead.org> <20180420071312.GF31310@phenom.ffwll.local> <3e17afc5-7d6c-5795-07bd-f23e34cf8d4b@gmail.com> <20180420101755.GA11400@infradead.org> <20180420124625.GA31078@infradead.org> <20180420152111.GR31310@phenom.ffwll.local> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20180420152111.GR31310@phenom.ffwll.local> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Christoph Hellwig , Christian =?iso-8859-1?Q?K=F6nig?= , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Linux Kernel Mailing List , amd-gfx list , Jerome Glisse , dri-devel , Dan Williams , Logan Gunthorpe , "open list:DMA BUFFER SHARING FRAMEWORK" List-Id: amd-gfx.lists.freedesktop.org T24gRnJpLCBBcHIgMjAsIDIwMTggYXQgMDU6MjE6MTFQTSArMDIwMCwgRGFuaWVsIFZldHRlciB3 cm90ZToKPiA+IEF0IHRoZSB2ZXJ5IGxvd2VzdCBsZXZlbCB0aGV5IHdpbGwgbmVlZCB0byBiZSBo YW5kbGVkIGRpZmZlcmVudGx5IGZvcgo+ID4gbWFueSBhcmNoaXRlY3R1cmVzLCB0aGUgcXVlc3Rp b25zIGlzIGF0IHdoYXQgcG9pbnQgd2UnbGwgZG8gdGhlCj4gPiBicmFuY2hpbmcgb3V0Lgo+IAo+ IEhhdmluZyBhdCBsZWFzdCBzdHJ1Y3QgcGFnZSBhbHNvIGluIHRoYXQgbGlzdCB3aXRoIChkbWFf YWRkcl90LCBsZW5naHQpCj4gcGFpcnMgaGFzIGEgYnVuY2ggb2YgYmVuZWZpdHMgZm9yIGRyaXZl cnMgaW4gdW5pZnlpbmcgYnVmZmVyIGhhbmRsaW5nCj4gY29kZS4gWW91IGp1c3QgcGFzcyB0aGF0 IG9uZSBzaW5nbGUgbGlzdCBhcm91bmQsIHVzZSB0aGUgZG1hX2FkZHJfdCBzaWRlCj4gZm9yIGdw dSBhY2Nlc3MgKGdlbmVyYWxseSBiYXNoaW5nIGl0IGludG8gZ3B1IHB0ZXMpLiBBbmQgdGhlIHN0 cnVjdCBwYWdlCj4gKGlmIHByZXNlbnQpIGZvciBjcHUgYWNjZXNzLCB1c2luZyBrbWFwIG9yIHZt X2luc2VydF8qLiBXZSBnZW5lcmFsbHkKPiBpZ25vcmUgdmlydCwgaWYgd2UgZG8gbmVlZCBhIGZ1 bGwgbWFwcGluZyB0aGVuIHdlIGNvbnN0cnVjdCBhIHZtYXAgZm9yCj4gdGhhdCBidWZmZXIgb2Yg b3VyIG93bi4KCldlbGwsIGZvciBtYXBwaW5nIGEgcmVzb3VyY2UgKHdoaWNoIGdldHMgYmFjayB0 byB0aGUgc3RhcnQgb2YgdGhlCmRpc2N1c3Npb24pIHlvdSB3aWxsIG5lZWQgYW4gZXhwbGljaXQg dmlydCBwb2ludGVyLiAgWW91IGFsc28gbmVlZAphbiBleHBsaWNpdCB2aXJ0IHBvaW50ZXIgYW5k IG5vdCBqdXN0IHBhZ2VfYWRkcmVzcy9rbWFwIGZvciB1c2VycyBvZgpkbWFfZ2V0X3NndGFibGUs IGJlY2F1c2UgZm9yIG1hbnkgYXJjaGl0ZWN0dXJlcyB5b3Ugd2lsbCBuZWVkIHRvIGZsdXNoCnRo ZSB2aXJ0dWFsIGFkZHJlc3MgdXNlZCB0byBhY2Nlc3MgdGhlIGRhdGEsIHdoaWNoIG1pZ2h0IGJl IGEKdm1hcC9pb3JlbWFwIHN0eWxlIG1hcHBpbmcgcmV0b3VybmVkIGZyb20gZG1hX2FsbG9jX2Fk ZHJlc3MsIGFuZCBub3QKdGhlIGRpcmVjdGx5IG1hcHBlZCBrZXJuZWwgYWRkcmVzcy4KCkhlcmUg aXMgYW5vdGhlciBpZGVhIGF0IHRoZSBsb3ctbGV2ZWwgZG1hIEFQSSBsZXZlbDoKCiAtIGRtYV9n ZXRfc2d0YWJsZSBnb2VzIGF3YXkuICBUaGUgcmVwbGFjZW1lbnQgaXMgYSBuZXcKICAgZG1hX2Fs bG9jX3JlbWFwIGhlbHBlciB0aGF0IHRha2VzIHRoZSB2aXJ0dWFsIGFkZHJlc3MgcmV0dXJuZWQK ICAgZnJvbSBkbWFfYWxsb2NfYXR0cnMvY29oZXJlbnQgYW5kIGNyZWF0ZXMgYSBkbWFfYWRkcl90 IGZvciB0aGUKICAgZ2l2ZW4gbmV3IGRldmljZS4gIElmIHRoZSBvcmlnaW5hbCBhbGxvY2F0aW9u IHdhcyBhIGNvaGVyZW50CiAgIG9uZSBubyBjYWNoZSBmbHVzaGluZyBpcyByZXF1aXJlZCBlaXRo ZXIgKGJlY2F1c2UgdGhlIGFyY2gKICAgbWFkZSBzdXJlIGl0IGlzIGNvaGVyZW50KSwgaWYgdGhl IG9yaWdpbmFsIGFsbG9jYXRpb24gdXNlZAogICBETUFfQVRUUl9OT05fQ09OU0lTVEVOVCB0aGUg bmV3IGFsbG9jYXRpb24gd2lsbCBuZWVkCiAgIGRtYV9jYWNoZV9zeW5jIGNhbGxzIGFzIHdlbGwu CiAtIHlvdSBuZXZlciBldmVuIHRyeSB0byBzaGFyZSBhIG1hcHBpbmcgcmV0b3VybmVkIGZyb20K ICAgZG1hX21hcF9yZXNvdXJjZSAtIGluc3RlYWQgZWFjaCBkZXZpY2UgdXNpbmcgaXQgY3JlYXRl cyBhIG5ldwogICBtYXBwaW5nLCB3aGljaCBtYWtlcyBzZW5zZSBhcyBubyB2aXJ0dWFsIGFkZHJl c3NlcyBhcmUgaW52b2x2ZWQKICAgYXQgYWxsLgoKPiBTbyBtYXliZSBhIGxpc3Qgb2YgKHN0cnVj dCBwYWdlICosIGRtYV9hZGRyX3QsIG51bV9wYWdlcykgd291bGQgc3VpdCBiZXN0LAo+IHdpdGgg c3RydWN0IHBhZ2UgKiBiZWluZyBvcHRpb25hbCAoaWYgaXQncyBhIHJlc291cmNlLCBvciBzb21l dGhpbmcgZWxzZQo+IHRoYXQgdGhlIGtlcm5lbCBjb3JlIG1tIGlzbid0IGF3YXJlIG9mKS4gQnV0 IHRoYXQgb25seSBoYXMgYmVuZWZpdHMgaWYgd2UKPiByZWFsbHkgcm9sbCBpdCBvdXQgZXZlcnl3 aGVyZSwgaW4gYWxsIHRoZSBzdWJzeXN0ZW1zIGFuZCBkcml2ZXJzLCBzaW5jZSBpZgo+IHdlIGRv bid0IHdlJ3ZlIG1hZGUgdGhlIHN0cnVjdCBwYWdlcyAqKiA8LT4gc2d0IGNvbnZlcnNpb24gZnVu IG9ubHkgd29yc2UKPiBieSBhZGRpbmcgYSAzIHJlcHJlc2VudGF0aW9uIG9mIGdwdSBidWZmZXIg b2JqZWN0IGJhY2tpbmcgc3RvcmFnZS4KCkkgdGhpbmsgdGhlIG1vc3QgaW1wb3J0YW50IHRoaW5n IGFib3V0IHN1Y2ggYSBidWZmZXIgb2JqZWN0IGlzIHRoYXQKaXQgY2FuIGRpc3Rpbmd1aXNoIHRo ZSB1bmRlcmx5aW5nIG1hcHBpbmcgdHlwZXMuICBXaGlsZQpkbWFfYWxsb2NfY29oZXJlbnQsIGRt YV9hbGxvY19hdHRycyB3aXRoIERNQV9BVFRSX05PTl9DT05TSVNURU5ULApkbWFfbWFwX3BhZ2Uv ZG1hX21hcF9zaW5nbGUvZG1hX21hcF9zZyBhbmQgZG1hX21hcF9yZXNvdXJjZSBhbGwgZ2l2ZQpi YWNrIGEgZG1hX2FkZHJfdCB0aGV5IGFyZSBpbiBub3cgd2F5IGludGVyY2hhbmdhYmxlLiAgQW5k IHRyeWluZyB0bwpzdHVmZiB0aGVtIGFsbCBpbnRvIGEgc3RydWN0dXJlIGxpa2Ugc3RydWN0IHNj YXR0ZXJsaXN0IHRoYXQgaGFzCm5vIGluZGljYXRpb24gd2hhdCBraW5kIG9mIG1hcHBpbmcgeW91 IGFyZSBkZWFsaW5nIHdpdGggaXMganVzdAphc2tpbmcgZm9yIHRyb3VibGUuCl9fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxp c3QKZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNr dG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from bombadil.infradead.org ([198.137.202.133]:50130 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751980AbeDXSsu (ORCPT ); Tue, 24 Apr 2018 14:48:50 -0400 Date: Tue, 24 Apr 2018 11:48:47 -0700 From: Christoph Hellwig To: Christoph Hellwig , Christian =?iso-8859-1?Q?K=F6nig?= , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Linux Kernel Mailing List , amd-gfx list , Jerome Glisse , dri-devel , Dan Williams , Logan Gunthorpe , "open list:DMA BUFFER SHARING FRAMEWORK" Subject: Re: [Linaro-mm-sig] [PATCH 4/8] dma-buf: add peer2peer flag Message-ID: <20180424184847.GA3247@infradead.org> References: <20180403180832.GZ3881@phenom.ffwll.local> <20180416123937.GA9073@infradead.org> <20180419081657.GA16735@infradead.org> <20180420071312.GF31310@phenom.ffwll.local> <3e17afc5-7d6c-5795-07bd-f23e34cf8d4b@gmail.com> <20180420101755.GA11400@infradead.org> <20180420124625.GA31078@infradead.org> <20180420152111.GR31310@phenom.ffwll.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180420152111.GR31310@phenom.ffwll.local> Sender: linux-media-owner@vger.kernel.org List-ID: On Fri, Apr 20, 2018 at 05:21:11PM +0200, Daniel Vetter wrote: > > At the very lowest level they will need to be handled differently for > > many architectures, the questions is at what point we'll do the > > branching out. > > Having at least struct page also in that list with (dma_addr_t, lenght) > pairs has a bunch of benefits for drivers in unifying buffer handling > code. You just pass that one single list around, use the dma_addr_t side > for gpu access (generally bashing it into gpu ptes). And the struct page > (if present) for cpu access, using kmap or vm_insert_*. We generally > ignore virt, if we do need a full mapping then we construct a vmap for > that buffer of our own. Well, for mapping a resource (which gets back to the start of the discussion) you will need an explicit virt pointer. You also need an explicit virt pointer and not just page_address/kmap for users of dma_get_sgtable, because for many architectures you will need to flush the virtual address used to access the data, which might be a vmap/ioremap style mapping retourned from dma_alloc_address, and not the directly mapped kernel address. Here is another idea at the low-level dma API level: - dma_get_sgtable goes away. The replacement is a new dma_alloc_remap helper that takes the virtual address returned from dma_alloc_attrs/coherent and creates a dma_addr_t for the given new device. If the original allocation was a coherent one no cache flushing is required either (because the arch made sure it is coherent), if the original allocation used DMA_ATTR_NON_CONSISTENT the new allocation will need dma_cache_sync calls as well. - you never even try to share a mapping retourned from dma_map_resource - instead each device using it creates a new mapping, which makes sense as no virtual addresses are involved at all. > So maybe a list of (struct page *, dma_addr_t, num_pages) would suit best, > with struct page * being optional (if it's a resource, or something else > that the kernel core mm isn't aware of). But that only has benefits if we > really roll it out everywhere, in all the subsystems and drivers, since if > we don't we've made the struct pages ** <-> sgt conversion fun only worse > by adding a 3 representation of gpu buffer object backing storage. I think the most important thing about such a buffer object is that it can distinguish the underlying mapping types. While dma_alloc_coherent, dma_alloc_attrs with DMA_ATTR_NON_CONSISTENT, dma_map_page/dma_map_single/dma_map_sg and dma_map_resource all give back a dma_addr_t they are in now way interchangable. And trying to stuff them all into a structure like struct scatterlist that has no indication what kind of mapping you are dealing with is just asking for trouble.