From mboxrd@z Thu Jan 1 00:00:00 1970 From: jilaiw@codeaurora.org Subject: Re: [PATCH 2/3] drm:msm: Initial Add Writeback Support Date: Wed, 8 Apr 2015 14:40:01 -0000 Message-ID: <6f5bcdfa44daaf8fc78ea3510963094a.squirrel@www.codeaurora.org> References: <1427922770-13721-1-git-send-email-jilaiw@codeaurora.org> <20150407055949.GI6354@phenom.ffwll.local> <93d3802305fa348738dae7f458c3fb56.squirrel@www.codeaurora.org> <20150408080707.GN6354@phenom.ffwll.local> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20150408080707.GN6354@phenom.ffwll.local> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: jilaiw@codeaurora.org, Rob Clark , linux-arm-msm , Linux Kernel Mailing List , "dri-devel@lists.freedesktop.org" List-Id: linux-arm-msm@vger.kernel.org PiBPbiBUdWUsIEFwciAwNywgMjAxNSBhdCAwMzo1NTo0NVBNIC0wMDAwLCBqaWxhaXdAY29kZWF1 cm9yYS5vcmcgd3JvdGU6Cj4+ID4gT24gVGh1LCBBcHIgMDIsIDIwMTUgYXQgMTA6Mjk6NTJBTSAt MDQwMCwgUm9iIENsYXJrIHdyb3RlOgo+PiA+PiBTbywgZnJvbSBhIHF1aWNrIGxvb2ssIGl0IHNl ZW1zIGxpa2UgdGhlcmUgaXMgYSBsb3Qgb2YgcG90ZW50aWFsIHRvCj4+ID4+IHNwbGl0IHRoZSB2 NGwgcGFydCBvdXQgaW50byBzb21lIGRybSBoZWxwZXJzLi4gaXQgbG9va3MgcHJldHR5Cj4+ID4+ IGdlbmVyaWMoaXNoKSwgb3IgYXQgbGVhc3QgaXQgY291bGQgYmUgd2l0aCBzb21lIHN0cmF0ZWdp Y2FsbHkgcGxhY2VkCj4+ID4+IHZmdW5jcyBpbiBkcm1fdjRsMl9oZWxwZXJfZnVuY3MuCj4+ID4+ Cj4+ID4+IEkgZG8gdGhpbmsgd2UgbmVlZCB0byBmaWd1cmUgb3V0IHRoZSBhdXRoL3NlY3VyaXR5 IHNpdHVhdGlvbi4gIFdlCj4+ID4+IHByb2JhYmx5IGRvbid0IHdhbnQgdG8gbGV0IGFyYml0cmFy eSBwcm9jZXNzZXMgb3BlbiBhIHY0bCBkZXZpY2UgYW5kCj4+ID4+IHNub29wIG9uIHRoZSBzY3Jl ZW4gY29udGVudHMuICBXZSBwZXJoYXBzIGNvdWxkIHJlLXVzZSB0aGUgZHJpMiBkcm0KPj4gPj4g YXV0aCBzdHVmZiAodjRsMl9kcm1fZ2V0X21hZ2ljIGlvY3RsPykuICBPciwgd2VsbCwgaXQgd291 bGQgYmUgbmljZQo+PiBpZgo+PiA+PiB0aGUgd2IgZGV2aWNlIGNvdWxkIGJlIG1hZGUgdG8gbm90 IGV4aXN0IGluIC9kZXYgYXQgYWxsLCBhbmQKPj4gPj4gcHJlLW9wZW4nZCBmZCByZXR1cm5lZCBm cm9tIGFuIGlvY3RsIG9uIHRoZSBkcm0gZGV2aWNlLCBidXQgbm90Cj4+IHJlYWxseQo+PiA+PiBz dXJlIGlmIHRoYXQgaXMgcG9zc2libGUgKG9yIHRvbyB3ZWlyZCkuICBPbmNlIHRoZSBjb21wb3Np dG9yIHByb2Nlc3MKPj4gPj4gaGFzIHRoZSB2NGwgZGV2aWNlIG9wZW4gYW5kIGF1dGhlbnRpY2F0 ZWQgc29tZWhvdywgSSBleHBlY3QgaXQgd291bGQKPj4gPj4gdXNlIGZkIHBhc3NpbmcgdG8gcGFz cyB0aGUgZmQgb2ZmIHRvIGEgdHJ1c3RlZCBoZWxwZXIgcHJvY2Vzcy4KPj4gPgo+PiA+IFBsZWFz ZSBkb24ndCByZXN1cnJlY3QgdGhlIG1hZ2ljIHN0dWZmIDstKQo+PiA+Cj4+ID4gQW55d2F5IEkg ZGlzY3Vzc2VkIHRoaXMgYSBiaXQgd2l0aCBMYXVyZW50IGFuZCB3ZSBmaWd1cmVkIHRoZSBiZXN0 IHdheQo+PiB0bwo+PiA+IHdpcmUgdXAgd3JpdGViYWNrIHN1cHBvcnQgaXMgYnkgdXNpbmcgZHJt IGZyYW1lYnVmZmVycy4gVGhlbiB5b3UgY2FuCj4+IHVzZQo+PiA+IGF0b21pYyBmbGlwcyB0byBj cmVhdGUgYSBuZXcgc25hcHNob3QuIE9mIGNvdXJzZSB0aGF0IHdvbid0IHdvcmsgd2l0aAo+PiBo dwo+PiA+IHdoZXJlIHdyaXRlYmFjayBpcyBjb250aW51b3VzLCB0aGVyZSB2NGwgaXMgYSBtdWNo IGJldHRlciBmaXQuIEFuZCB3ZQo+PiBhbHNvCj4+ID4gaGF2ZSBoYXJkd2FyZSB3aGVyZSBzb21l IHY0bCBwaXBlbGluZSBjb3VsZCBkaXJlY3RseSBmZWVkIGludG8gYSBkcm0KPj4gPiBvdXRwdXQg cGlwZWxpbmUsIHNvIHdlIG5lZWQgYSBnZW5lcmljIHdheSB0byBjb25uZWN0IHY0bCBhbmQgZHJt Cj4+IGFueXdheS4KPj4gPiBGb3IgdGhhdCBJIHRoaW5rIHdlIHNob3VsZCBhZGQgYSBuZXcgZmxh ZyB0byBhZGRmYjIgKG9yIGEgbmV3Cj4+IGFkZGZidjRsKQo+PiA+IHdoaWNoIGNyZWF0ZXMgYSBt YWdpYyBmcmFtZWJ1ZmZlciBmcm9tIGEgdjRsIGlucHV0L291dHB1dC4gU29tZSB2YWx1ZXMKPj4g PiBsaWtlIHN0cmlkZSBkb24ndCBtYWtlIHNlbnNlIGluIHN1Y2ggYSB2aXJ0dWFsIGZyYW1lYnVm ZmVyLCBidXQgcGl4ZWwKPj4gPiBmb3JtYXQgYW5kIHNpemUgYXJlIGFsbCBuZWVkZWQuCj4+ID4K Pj4gPiBUaGlzIHdheSB3ZSBkb24ndCBuZWVkIHBhcmFsbGVsIGFiaXMgZm9yIHNpbmdsZS1zaG90 IHdyaXRlYmFjawo+PiBkaXJlY3RseQo+PiA+IGludG8gZnJhbWVidWZmZXJzIGFuZCBmb3IgY29u dGludW91cyB3cml0ZWJhY2sgdGhyb3VnaCB2NGwsIHdlIGNhbgo+PiByZXVzZQo+PiA+IHRoZSBz YW1lIGRybSBmcmFtZWJ1ZmZlciBvbmVzLiBBbmQgdGhpcyBhbHNvIHNvbHZlcyB0aGUgc2VjdXJp dHkKPj4gaXNzdWVzCj4+ID4gc2luY2Ugbm8gb25lIGNhbiBzdGFydCB3cml0ZWJhY2sgd2l0aG91 dCB0aGUgZHJtIGRldmljZSBvd25lcidzCj4+IGNvbnNlbnQsCj4+ID4gc28gbm8gbmVlZCB0byBy ZWludmVudCBhbnl0aGluZyB0aGVyZS4gQW5kIHdpdGggYXRvbWljIHdlIGFscmVhZHkgaGF2ZQo+ PiA+IGFsbW9zdCBldmVyeXRoaW5nIHRoZXJlOiBGb3IgdGhlIHdyaXRlYmFjayBmcmFtZWJ1ZmZl ciB3ZSBvbmx5IG5lZWQgYQo+PiBuZXcKPj4gPiAiV1JJVEVCQUNLIiBwcm9wZXJ0eSAod2hpY2gg dGFrZXMgYW4gZmIgaWQpIGFuZCB0aGUgc21hbGwgZXh0ZW5zaW9uIHRvCj4+ID4gY3JlYXRlIHY0 bC1iYWNrZWQgZnJhbWVidWZmZXJzLgo+PiA+Cj4+ID4gQ2hlZXJzLCBEYW5pZWwKPj4gPiAtLQo+ PiA+IERhbmllbCBWZXR0ZXIKPj4gPiBTb2Z0d2FyZSBFbmdpbmVlciwgSW50ZWwgQ29ycG9yYXRp b24KPj4gPiBodHRwOi8vYmxvZy5mZndsbC5jaAo+PiA+Cj4+IEhpIERhbmllbCwKPj4KPj4gMS4g VGhpcyBjaGFuZ2UgaXMgdG8gaW1wbGVtZW50IGEgY29udGludW91cyB3cml0ZWJhY2suCj4+IDIu IEFzIHlvdSBzYWlkLCB3ZSBuZWVkICJhIGdlbmVyaWMgd2F5IHRvIGNvbm5lY3QgdjRsIGFuZCBk cm0iLgo+PiBFc3BlY2lhbGx5IGhvdyB0byBzaGFyZSB0aGUgYnVmZmVyIGluZm9ybWF0aW9uIGJl dHdlZW4gdjRsIGFuZCBkcm0gZm9yCj4+IHdyaXRlYmFjayBvdXRwdXQuCj4+Cj4+IEJlbG93IGFy ZSBqdXN0IHNvbWUgZGV0YWlscyBvZiB0aGlzIGNoYW5nZToKPj4KPj4gSW4gY3VycmVudCBpbXBs ZW1lbnRhdGlvbiwgSSBleHBlY3QgdGhlIG91dHB1dCBidWZmZXIgaXMgZG1hIGJ1ZmZlcgo+PiB3 aGljaCBjb3VsZCBiZSBmcm9tIEdFTSBvYmplY3QgKGRybSkgb3IgZnJvbSB2aWRlbyBlbmNvZGVy IChWNGwpLiBPbmNlCj4+IHRoZSBidWZmZXIgaXMgcXVldWVkIGludG8gVjRsIGRyaXZlciwgaXQg d2lsbCBiZSBjb252ZXJ0ZWQgaW50byBhIEdFTQo+PiBvYmplY3QgYW5kIHRoZW4gcGFzcyBpdCB0 byBkcm0gYXMgd3JpdGViYWNrIG91dHB1dCBidWZmZXIuIE9uY2UgdGhlCj4+IGJ1ZmZlciBpcyBj YXB0dXJlZCwgaXQgd2lsbCBub3RpZnkgVjRsIGRyaXZlciB0byBsZXQgdXNlciBkZXF1ZXVlCj4+ IGJ1ZmZlci4KPj4KPj4gRHJtIHdpbGwgbm90aWNlIHRoZXJlIGlzIGEgVmlydHVhbCBDb25uZWN0 b3IgKG1heWJlIGEgbmV3IHR5cGUgV1JJVEVCQUNLCj4+IGNhbiBiZSBhZGRlZCksIGJ1dCBpdCB3 aWxsIG9ubHkgYmUgImNvbm5lY3RlZCIgdW50aWwgVjRsCj4+IHN0YXJ0cyBzdHJlYW1pbmcuCj4K PiBZZXMgd2UgZGVmaW5pdGVseSBzaG91bGQgYWRkIGEgbmV3IGNvbm5lY3RvciB0eXBlIFdSSVRF QkFDSy4gQW5kIGp1c3QgdGhlCj4gY29ubmVjdG9yIGtpbmRhIHdvcmtzIGZvciB5b3VyIGh3IGRl c2lnbiB3aGVyZSB3cml0ZWJhY2sgd29ya3MgbGlrZSBhCj4gc2VwYXJhdGUgZW5jb2Rlci4gQnV0 IHRoZXJlJ3MgYWxzbyBodyBvdXQgdGhlcmUgd2hlcmUgYW55IGNydGMgY2FuIGJlCj4gd3JpdHRl biBiYWNrLCBhbmQgZm9yIHRob3NlIGNhc2VzIHdlIG5lZWQgZXhwbGljaXQgcHJvcGVydGllcy4g VGhlbgo+IHRoZXJlJ3MgYWxzbyB0aGUgb25lLXNob3QgdnMuIGNvbnRpbnVvdXMgaXNzdWVzLgp5 ZXMsIHRoaXMgY2hhbmdlIGlzIHNwZWNpZmljIGZvciBtc20gY2hpcC4gSW4gb3JkZXIgdG8gY292 ZXIgdGhlIG90aGVyCndyaXRlYmFjayB1c2UgY2FzZXMsIG5ldyBwcm9wZXJ0aWVzIGFuZCBmcmFt ZXdvcmsgY2hhbmdlcyBhcmUgcmVxdWlyZWQuCgo+Cj4gR2l2ZW4gYWxsIHRoYXQgSSBzdGlsbCB0 aGluayB5b3Ugd2FudCBhbiBleHBsaWNpdCBkcm0gZnJhbWVidWZmZXIgdG8KPiBjb25uZWN0IHRo ZSBrbXMgYW5kIHRoZSB2NGwgc2lkZSBvZiB0aGluZ3MuIFRoYXQgd291bGQgYWxzbyBoZWxwIGEg Yml0Cj4gd2l0aCBtYWtpbmcgaXQgY2xlYXIgd2hpY2ggdjRsIGNvbm5lY3RzIHRvIHdoaWNoIGRy bSBkZXZpY2UuCnllcywgaXQgd2lsbCBoZWxwLgoKPiAtRGFuaWVsCj4gLS0KPiBEYW5pZWwgVmV0 dGVyCj4gU29mdHdhcmUgRW5naW5lZXIsIEludGVsIENvcnBvcmF0aW9uCj4gaHR0cDovL2Jsb2cu ZmZ3bGwuY2gKPgoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9y ZwpodHRwOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932340AbbDHOkZ (ORCPT ); Wed, 8 Apr 2015 10:40:25 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:38022 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932313AbbDHOkD (ORCPT ); Wed, 8 Apr 2015 10:40:03 -0400 Message-ID: <6f5bcdfa44daaf8fc78ea3510963094a.squirrel@www.codeaurora.org> In-Reply-To: <20150408080707.GN6354@phenom.ffwll.local> References: <1427922770-13721-1-git-send-email-jilaiw@codeaurora.org> <20150407055949.GI6354@phenom.ffwll.local> <93d3802305fa348738dae7f458c3fb56.squirrel@www.codeaurora.org> <20150408080707.GN6354@phenom.ffwll.local> Date: Wed, 8 Apr 2015 14:40:01 -0000 Subject: Re: [PATCH 2/3] drm:msm: Initial Add Writeback Support From: jilaiw@codeaurora.org To: jilaiw@codeaurora.org, "Rob Clark" , "linux-arm-msm" , "Linux Kernel Mailing List" , "dri-devel@lists.freedesktop.org" User-Agent: SquirrelMail/1.4.22-4.el6 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT X-Priority: 3 (Normal) Importance: Normal Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > On Tue, Apr 07, 2015 at 03:55:45PM -0000, jilaiw@codeaurora.org wrote: >> > On Thu, Apr 02, 2015 at 10:29:52AM -0400, Rob Clark wrote: >> >> So, from a quick look, it seems like there is a lot of potential to >> >> split the v4l part out into some drm helpers.. it looks pretty >> >> generic(ish), or at least it could be with some strategically placed >> >> vfuncs in drm_v4l2_helper_funcs. >> >> >> >> I do think we need to figure out the auth/security situation. We >> >> probably don't want to let arbitrary processes open a v4l device and >> >> snoop on the screen contents. We perhaps could re-use the dri2 drm >> >> auth stuff (v4l2_drm_get_magic ioctl?). Or, well, it would be nice >> if >> >> the wb device could be made to not exist in /dev at all, and >> >> pre-open'd fd returned from an ioctl on the drm device, but not >> really >> >> sure if that is possible (or too weird). Once the compositor process >> >> has the v4l device open and authenticated somehow, I expect it would >> >> use fd passing to pass the fd off to a trusted helper process. >> > >> > Please don't resurrect the magic stuff ;-) >> > >> > Anyway I discussed this a bit with Laurent and we figured the best way >> to >> > wire up writeback support is by using drm framebuffers. Then you can >> use >> > atomic flips to create a new snapshot. Of course that won't work with >> hw >> > where writeback is continuous, there v4l is a much better fit. And we >> also >> > have hardware where some v4l pipeline could directly feed into a drm >> > output pipeline, so we need a generic way to connect v4l and drm >> anyway. >> > For that I think we should add a new flag to addfb2 (or a new >> addfbv4l) >> > which creates a magic framebuffer from a v4l input/output. Some values >> > like stride don't make sense in such a virtual framebuffer, but pixel >> > format and size are all needed. >> > >> > This way we don't need parallel abis for single-shot writeback >> directly >> > into framebuffers and for continuous writeback through v4l, we can >> reuse >> > the same drm framebuffer ones. And this also solves the security >> issues >> > since no one can start writeback without the drm device owner's >> consent, >> > so no need to reinvent anything there. And with atomic we already have >> > almost everything there: For the writeback framebuffer we only need a >> new >> > "WRITEBACK" property (which takes an fb id) and the small extension to >> > create v4l-backed framebuffers. >> > >> > Cheers, Daniel >> > -- >> > Daniel Vetter >> > Software Engineer, Intel Corporation >> > http://blog.ffwll.ch >> > >> Hi Daniel, >> >> 1. This change is to implement a continuous writeback. >> 2. As you said, we need "a generic way to connect v4l and drm". >> Especially how to share the buffer information between v4l and drm for >> writeback output. >> >> Below are just some details of this change: >> >> In current implementation, I expect the output buffer is dma buffer >> which could be from GEM object (drm) or from video encoder (V4l). Once >> the buffer is queued into V4l driver, it will be converted into a GEM >> object and then pass it to drm as writeback output buffer. Once the >> buffer is captured, it will notify V4l driver to let user dequeue >> buffer. >> >> Drm will notice there is a Virtual Connector (maybe a new type WRITEBACK >> can be added), but it will only be "connected" until V4l >> starts streaming. > > Yes we definitely should add a new connector type WRITEBACK. And just the > connector kinda works for your hw design where writeback works like a > separate encoder. But there's also hw out there where any crtc can be > written back, and for those cases we need explicit properties. Then > there's also the one-shot vs. continuous issues. yes, this change is specific for msm chip. In order to cover the other writeback use cases, new properties and framework changes are required. > > Given all that I still think you want an explicit drm framebuffer to > connect the kms and the v4l side of things. That would also help a bit > with making it clear which v4l connects to which drm device. yes, it will help. > -Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch >