From mboxrd@z Thu Jan 1 00:00:00 1970 From: jilaiw@codeaurora.org Subject: Re: [PATCH 2/3] drm:msm: Initial Add Writeback Support Date: Tue, 7 Apr 2015 15:55:45 -0000 Message-ID: <93d3802305fa348738dae7f458c3fb56.squirrel@www.codeaurora.org> References: <1427922770-13721-1-git-send-email-jilaiw@codeaurora.org> <20150407055949.GI6354@phenom.ffwll.local> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20150407055949.GI6354@phenom.ffwll.local> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Rob Clark , Jilai Wang , linux-arm-msm , Linux Kernel Mailing List , "dri-devel@lists.freedesktop.org" List-Id: linux-arm-msm@vger.kernel.org PiBPbiBUaHUsIEFwciAwMiwgMjAxNSBhdCAxMDoyOTo1MkFNIC0wNDAwLCBSb2IgQ2xhcmsgd3Jv dGU6Cj4+IFNvLCBmcm9tIGEgcXVpY2sgbG9vaywgaXQgc2VlbXMgbGlrZSB0aGVyZSBpcyBhIGxv dCBvZiBwb3RlbnRpYWwgdG8KPj4gc3BsaXQgdGhlIHY0bCBwYXJ0IG91dCBpbnRvIHNvbWUgZHJt IGhlbHBlcnMuLiBpdCBsb29rcyBwcmV0dHkKPj4gZ2VuZXJpYyhpc2gpLCBvciBhdCBsZWFzdCBp dCBjb3VsZCBiZSB3aXRoIHNvbWUgc3RyYXRlZ2ljYWxseSBwbGFjZWQKPj4gdmZ1bmNzIGluIGRy bV92NGwyX2hlbHBlcl9mdW5jcy4KPj4KPj4gSSBkbyB0aGluayB3ZSBuZWVkIHRvIGZpZ3VyZSBv dXQgdGhlIGF1dGgvc2VjdXJpdHkgc2l0dWF0aW9uLiAgV2UKPj4gcHJvYmFibHkgZG9uJ3Qgd2Fu dCB0byBsZXQgYXJiaXRyYXJ5IHByb2Nlc3NlcyBvcGVuIGEgdjRsIGRldmljZSBhbmQKPj4gc25v b3Agb24gdGhlIHNjcmVlbiBjb250ZW50cy4gIFdlIHBlcmhhcHMgY291bGQgcmUtdXNlIHRoZSBk cmkyIGRybQo+PiBhdXRoIHN0dWZmICh2NGwyX2RybV9nZXRfbWFnaWMgaW9jdGw/KS4gIE9yLCB3 ZWxsLCBpdCB3b3VsZCBiZSBuaWNlIGlmCj4+IHRoZSB3YiBkZXZpY2UgY291bGQgYmUgbWFkZSB0 byBub3QgZXhpc3QgaW4gL2RldiBhdCBhbGwsIGFuZAo+PiBwcmUtb3BlbidkIGZkIHJldHVybmVk IGZyb20gYW4gaW9jdGwgb24gdGhlIGRybSBkZXZpY2UsIGJ1dCBub3QgcmVhbGx5Cj4+IHN1cmUg aWYgdGhhdCBpcyBwb3NzaWJsZSAob3IgdG9vIHdlaXJkKS4gIE9uY2UgdGhlIGNvbXBvc2l0b3Ig cHJvY2Vzcwo+PiBoYXMgdGhlIHY0bCBkZXZpY2Ugb3BlbiBhbmQgYXV0aGVudGljYXRlZCBzb21l aG93LCBJIGV4cGVjdCBpdCB3b3VsZAo+PiB1c2UgZmQgcGFzc2luZyB0byBwYXNzIHRoZSBmZCBv ZmYgdG8gYSB0cnVzdGVkIGhlbHBlciBwcm9jZXNzLgo+Cj4gUGxlYXNlIGRvbid0IHJlc3VycmVj dCB0aGUgbWFnaWMgc3R1ZmYgOy0pCj4KPiBBbnl3YXkgSSBkaXNjdXNzZWQgdGhpcyBhIGJpdCB3 aXRoIExhdXJlbnQgYW5kIHdlIGZpZ3VyZWQgdGhlIGJlc3Qgd2F5IHRvCj4gd2lyZSB1cCB3cml0 ZWJhY2sgc3VwcG9ydCBpcyBieSB1c2luZyBkcm0gZnJhbWVidWZmZXJzLiBUaGVuIHlvdSBjYW4g dXNlCj4gYXRvbWljIGZsaXBzIHRvIGNyZWF0ZSBhIG5ldyBzbmFwc2hvdC4gT2YgY291cnNlIHRo YXQgd29uJ3Qgd29yayB3aXRoIGh3Cj4gd2hlcmUgd3JpdGViYWNrIGlzIGNvbnRpbnVvdXMsIHRo ZXJlIHY0bCBpcyBhIG11Y2ggYmV0dGVyIGZpdC4gQW5kIHdlIGFsc28KPiBoYXZlIGhhcmR3YXJl IHdoZXJlIHNvbWUgdjRsIHBpcGVsaW5lIGNvdWxkIGRpcmVjdGx5IGZlZWQgaW50byBhIGRybQo+ IG91dHB1dCBwaXBlbGluZSwgc28gd2UgbmVlZCBhIGdlbmVyaWMgd2F5IHRvIGNvbm5lY3QgdjRs IGFuZCBkcm0gYW55d2F5Lgo+IEZvciB0aGF0IEkgdGhpbmsgd2Ugc2hvdWxkIGFkZCBhIG5ldyBm bGFnIHRvIGFkZGZiMiAob3IgYSBuZXcgYWRkZmJ2NGwpCj4gd2hpY2ggY3JlYXRlcyBhIG1hZ2lj IGZyYW1lYnVmZmVyIGZyb20gYSB2NGwgaW5wdXQvb3V0cHV0LiBTb21lIHZhbHVlcwo+IGxpa2Ug c3RyaWRlIGRvbid0IG1ha2Ugc2Vuc2UgaW4gc3VjaCBhIHZpcnR1YWwgZnJhbWVidWZmZXIsIGJ1 dCBwaXhlbAo+IGZvcm1hdCBhbmQgc2l6ZSBhcmUgYWxsIG5lZWRlZC4KPgo+IFRoaXMgd2F5IHdl IGRvbid0IG5lZWQgcGFyYWxsZWwgYWJpcyBmb3Igc2luZ2xlLXNob3Qgd3JpdGViYWNrIGRpcmVj dGx5Cj4gaW50byBmcmFtZWJ1ZmZlcnMgYW5kIGZvciBjb250aW51b3VzIHdyaXRlYmFjayB0aHJv dWdoIHY0bCwgd2UgY2FuIHJldXNlCj4gdGhlIHNhbWUgZHJtIGZyYW1lYnVmZmVyIG9uZXMuIEFu ZCB0aGlzIGFsc28gc29sdmVzIHRoZSBzZWN1cml0eSBpc3N1ZXMKPiBzaW5jZSBubyBvbmUgY2Fu IHN0YXJ0IHdyaXRlYmFjayB3aXRob3V0IHRoZSBkcm0gZGV2aWNlIG93bmVyJ3MgY29uc2VudCwK PiBzbyBubyBuZWVkIHRvIHJlaW52ZW50IGFueXRoaW5nIHRoZXJlLiBBbmQgd2l0aCBhdG9taWMg d2UgYWxyZWFkeSBoYXZlCj4gYWxtb3N0IGV2ZXJ5dGhpbmcgdGhlcmU6IEZvciB0aGUgd3JpdGVi YWNrIGZyYW1lYnVmZmVyIHdlIG9ubHkgbmVlZCBhIG5ldwo+ICJXUklURUJBQ0siIHByb3BlcnR5 ICh3aGljaCB0YWtlcyBhbiBmYiBpZCkgYW5kIHRoZSBzbWFsbCBleHRlbnNpb24gdG8KPiBjcmVh dGUgdjRsLWJhY2tlZCBmcmFtZWJ1ZmZlcnMuCj4KPiBDaGVlcnMsIERhbmllbAo+IC0tCj4gRGFu aWVsIFZldHRlcgo+IFNvZnR3YXJlIEVuZ2luZWVyLCBJbnRlbCBDb3Jwb3JhdGlvbgo+IGh0dHA6 Ly9ibG9nLmZmd2xsLmNoCj4KSGkgRGFuaWVsLAoKMS4gVGhpcyBjaGFuZ2UgaXMgdG8gaW1wbGVt ZW50IGEgY29udGludW91cyB3cml0ZWJhY2suCjIuIEFzIHlvdSBzYWlkLCB3ZSBuZWVkICJhIGdl bmVyaWMgd2F5IHRvIGNvbm5lY3QgdjRsIGFuZCBkcm0iLgpFc3BlY2lhbGx5IGhvdyB0byBzaGFy ZSB0aGUgYnVmZmVyIGluZm9ybWF0aW9uIGJldHdlZW4gdjRsIGFuZCBkcm0gZm9yCndyaXRlYmFj ayBvdXRwdXQuCgpCZWxvdyBhcmUganVzdCBzb21lIGRldGFpbHMgb2YgdGhpcyBjaGFuZ2U6CgpJ biBjdXJyZW50IGltcGxlbWVudGF0aW9uLCBJIGV4cGVjdCB0aGUgb3V0cHV0IGJ1ZmZlciBpcyBk bWEgYnVmZmVyCndoaWNoIGNvdWxkIGJlIGZyb20gR0VNIG9iamVjdCAoZHJtKSBvciBmcm9tIHZp ZGVvIGVuY29kZXIgKFY0bCkuIE9uY2UKdGhlIGJ1ZmZlciBpcyBxdWV1ZWQgaW50byBWNGwgZHJp dmVyLCBpdCB3aWxsIGJlIGNvbnZlcnRlZCBpbnRvIGEgR0VNCm9iamVjdCBhbmQgdGhlbiBwYXNz IGl0IHRvIGRybSBhcyB3cml0ZWJhY2sgb3V0cHV0IGJ1ZmZlci4gT25jZSB0aGUKYnVmZmVyIGlz IGNhcHR1cmVkLCBpdCB3aWxsIG5vdGlmeSBWNGwgZHJpdmVyIHRvIGxldCB1c2VyIGRlcXVldWUK YnVmZmVyLgoKRHJtIHdpbGwgbm90aWNlIHRoZXJlIGlzIGEgVmlydHVhbCBDb25uZWN0b3IgKG1h eWJlIGEgbmV3IHR5cGUgV1JJVEVCQUNLCmNhbiBiZSBhZGRlZCksIGJ1dCBpdCB3aWxsIG9ubHkg YmUgImNvbm5lY3RlZCIgdW50aWwgVjRsCnN0YXJ0cyBzdHJlYW1pbmcuCgpEdXJpbmcgc3RyZWFt aW5nLCBkcm0gYXBwbGljYXRpb24ganVzdCBkb2VzIG5vcm1hbCBwYWdlX2ZsaXAgd2hpbGUgdjRs CmFwcGxpY2F0aW9uIHF1ZXVlcy9kZXF1ZXVlcyB2NGwgYnVmZmVycy4KClRoYW5rcywKSmlsYWkK CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpkcmktZGV2 ZWwgbWFpbGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cDovL2xp c3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756264AbbDGPzx (ORCPT ); Tue, 7 Apr 2015 11:55:53 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:53703 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754656AbbDGPzq (ORCPT ); Tue, 7 Apr 2015 11:55:46 -0400 Message-ID: <93d3802305fa348738dae7f458c3fb56.squirrel@www.codeaurora.org> In-Reply-To: <20150407055949.GI6354@phenom.ffwll.local> References: <1427922770-13721-1-git-send-email-jilaiw@codeaurora.org> <20150407055949.GI6354@phenom.ffwll.local> Date: Tue, 7 Apr 2015 15:55:45 -0000 Subject: Re: [PATCH 2/3] drm:msm: Initial Add Writeback Support From: jilaiw@codeaurora.org To: "Rob Clark" , "Jilai Wang" , "linux-arm-msm" , "Linux Kernel Mailing List" , "dri-devel@lists.freedesktop.org" User-Agent: SquirrelMail/1.4.22-4.el6 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT X-Priority: 3 (Normal) Importance: Normal Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > On Thu, Apr 02, 2015 at 10:29:52AM -0400, Rob Clark wrote: >> So, from a quick look, it seems like there is a lot of potential to >> split the v4l part out into some drm helpers.. it looks pretty >> generic(ish), or at least it could be with some strategically placed >> vfuncs in drm_v4l2_helper_funcs. >> >> I do think we need to figure out the auth/security situation. We >> probably don't want to let arbitrary processes open a v4l device and >> snoop on the screen contents. We perhaps could re-use the dri2 drm >> auth stuff (v4l2_drm_get_magic ioctl?). Or, well, it would be nice if >> the wb device could be made to not exist in /dev at all, and >> pre-open'd fd returned from an ioctl on the drm device, but not really >> sure if that is possible (or too weird). Once the compositor process >> has the v4l device open and authenticated somehow, I expect it would >> use fd passing to pass the fd off to a trusted helper process. > > Please don't resurrect the magic stuff ;-) > > Anyway I discussed this a bit with Laurent and we figured the best way to > wire up writeback support is by using drm framebuffers. Then you can use > atomic flips to create a new snapshot. Of course that won't work with hw > where writeback is continuous, there v4l is a much better fit. And we also > have hardware where some v4l pipeline could directly feed into a drm > output pipeline, so we need a generic way to connect v4l and drm anyway. > For that I think we should add a new flag to addfb2 (or a new addfbv4l) > which creates a magic framebuffer from a v4l input/output. Some values > like stride don't make sense in such a virtual framebuffer, but pixel > format and size are all needed. > > This way we don't need parallel abis for single-shot writeback directly > into framebuffers and for continuous writeback through v4l, we can reuse > the same drm framebuffer ones. And this also solves the security issues > since no one can start writeback without the drm device owner's consent, > so no need to reinvent anything there. And with atomic we already have > almost everything there: For the writeback framebuffer we only need a new > "WRITEBACK" property (which takes an fb id) and the small extension to > create v4l-backed framebuffers. > > Cheers, Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch > Hi Daniel, 1. This change is to implement a continuous writeback. 2. As you said, we need "a generic way to connect v4l and drm". Especially how to share the buffer information between v4l and drm for writeback output. Below are just some details of this change: In current implementation, I expect the output buffer is dma buffer which could be from GEM object (drm) or from video encoder (V4l). Once the buffer is queued into V4l driver, it will be converted into a GEM object and then pass it to drm as writeback output buffer. Once the buffer is captured, it will notify V4l driver to let user dequeue buffer. Drm will notice there is a Virtual Connector (maybe a new type WRITEBACK can be added), but it will only be "connected" until V4l starts streaming. During streaming, drm application just does normal page_flip while v4l application queues/dequeues v4l buffers. Thanks, Jilai