From mboxrd@z Thu Jan 1 00:00:00 1970 From: Laurent Pinchart Subject: Re: DRM Format Modifiers in v4l2 Date: Thu, 31 Aug 2017 17:51:33 +0300 Message-ID: <2089286.L0REA3WtaS@avalon> References: <20170824111430.GB25711@e107564-lin.cambridge.arm.com> <20170830103040.GA19103@e107564-lin.cambridge.arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from galahad.ideasonboard.com (galahad.ideasonboard.com [IPv6:2001:4b98:dc2:45:216:3eff:febb:480d]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2DBB36E730 for ; Thu, 31 Aug 2017 14:51:10 +0000 (UTC) In-Reply-To: <20170830103040.GA19103@e107564-lin.cambridge.arm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Brian Starkey Cc: jonathan.chai@arm.com, dri-devel , Nicolas Dufresne , Hans Verkuil , "linux-media@vger.kernel.org" List-Id: dri-devel@lists.freedesktop.org SGkgQnJpYW4sCgpPbiBXZWRuZXNkYXksIDMwIEF1Z3VzdCAyMDE3IDEzOjMyOjAxIEVFU1QgQnJp YW4gU3RhcmtleSB3cm90ZToKPiBPbiBXZWQsIEF1ZyAzMCwgMjAxNyBhdCAxMTo1Mzo1OEFNICsw MjAwLCBIYW5zIFZlcmt1aWwgd3JvdGU6Cj4gPiBPbiAzMC8wOC8xNyAxMTozNiwgQnJpYW4gU3Rh cmtleSB3cm90ZToKPiA+PiBPbiBXZWQsIEF1ZyAzMCwgMjAxNyBhdCAxMDoxMDowMUFNICswMjAw LCBIYW5zIFZlcmt1aWwgd3JvdGU6Cj4gPj4+IE9uIDMwLzA4LzE3IDA5OjUwLCBEYW5pZWwgVmV0 dGVyIHdyb3RlOgo+ID4+Pj4gT24gVHVlLCBBdWcgMjksIDIwMTcgYXQgMTA6NDc6MDFBTSArMDEw MCwgQnJpYW4gU3RhcmtleSB3cm90ZToKPiA+Pj4+PiBUaGUgZmFjdCBpcywgYWRkaW5nIHNwZWNp YWwgZm9ybWF0cyBmb3IgZWFjaCBjb21iaW5hdGlvbiBpcwo+ID4+Pj4+IHVubWFuYWdlYWJsZSAt IHdlJ3JlIHRhbGtpbmcgZG96ZW5zIGluIHRoZSBjYXNlIG9mIG91ciBoYXJkd2FyZS4KPiA+Pj4+ IAo+ID4+Pj4gSG0gcmlnaHQsIHdlIGNhbiBqdXN0IHJlbWFwIHRoZSBzcGVjaWFsIGNvbWJvcyB0 byB0aGUgZHJtLWZvdXJjYyArCj4gPj4+PiBtb2RpZmllciBzdHlsZS4gQm9udXMgcG9pbnQgaWYg djRsIGRvZXMgdGhhdCBpbiB0aGUgY29yZSBzbyBub3QKPiA+Pj4+IGV2ZXJ5b25lIGhhcyB0byBy ZWludmVudCB0aGF0IHdoZWVsIDotKQo+ID4+PiAKPiA+Pj4gUHJvYmFibHkgbm90IHNvbWV0aGlu ZyB3ZSdsbCBkbzogdGhlcmUgYXJlIEkgYmVsaWV2ZSBvbmx5IHR3byBkcml2ZXJzCj4gPj4+IHRo YXQgYXJlIGFmZmVjdGVkIChleHlub3MgJiBtZWRpYXRlayksIHNvIHRoZXkgY2FuIGRvIHRoYXQg aW4gdGhlaXIKPiA+Pj4gZHJpdmVyLgo+ID4+PiAKPiA+Pj4gUXVlc3Rpb246IGhvdyBtYW55IG1v ZGlmaWVycyB3aWxsIHR5cGljYWxseSBhcHBseSB0byBhIGZvcm1hdD8gSSBhc2sKPiA+Pj4gYmVj YXVzZSBJIHJlYWxpemVkIHRoYXQgVjRMMiBjb3VsZCB1c2UgVklESU9DX0VOVU1GTVQgdG8gbWFr ZSB0aGUgbGluawo+ID4+PiBiZXR3ZWVuIGEgZm91cmNjIGFuZCBtb2RpZmllcnM6Cj4gPj4+IAo+ ID4+PiBodHRwczovL2h2ZXJrdWlsLmhvbWUueHM0YWxsLm5sL3NwZWMvdWFwaS92NGwvdmlkaW9j LWVudW0tZm10Lmh0bWwKPiA+Pj4gCj4gPj4+IFRoZSBfX3UzMiByZXNlcnZlZFs0XSBhcnJheSBj YW4gYmUgdXNlZCB0byBwcm92aWRlIGEgYml0bWFzayB0byBtb2RpZmllcgo+ID4+PiBpbmRpY2Vz IChmb3IgdGhlIGludGVnZXIgbWVudSBjb250cm9sKS4gSXQncyBzaW1pbGFyIHRvIHdoYXQgZHJt IGRvZXMsCj4gPj4+IGV4Y2VwdCBpbnN0ZWFkIG9mIG1vZGlmaWVycyBtYXBwaW5nIHRvIGZvdXJj Y3MgaXQgaXMgdGhlIG90aGVyIHdheQo+ID4+PiBhcm91bmQuCj4gPj4+IAo+ID4+PiBUaGlzIHdv dWxkIGF2b2lkIGhhdmluZyB0byBjaGFuZ2UgdGhlIG1vZGlmaWVycyBjb250cm9sIHdoZW5ldmVy IGEgbmV3Cj4gPj4+IGZvcm1hdCBpcyBzZXQgYW5kIGl0IG1ha2VzIGl0IGVhc3kgdG8gZW51bWVy YXRlIGFsbCBjb21iaW5hdGlvbnMuCj4gPj4+IAo+ID4+PiBCdXQgdGhpcyBvbmx5IHdvcmtzIGlm IHRoZSB0b3RhbCBudW1iZXIgb2YgbW9kaWZpZXJzIHVzZWQgYnkgYSBzaW5nbGUKPiA+Pj4gZHJp dmVyIGlzIGV4cGVjdGVkIHRvIHJlbWFpbiBzbWFsbCAobGV0J3Mgc2F5IG5vIG1vcmUgdGhhbiA2 NCkuCj4gPj4gCj4gPj4gSW4gb3VyIGN1cnJlbnQgKHlldCB0byBiZSBzdWJtaXR0ZWQpIGRlc2Ny aXB0aW9uLCB3ZSd2ZSBnb3QgYXJvdW5kIGEKPiA+PiBkb3plbiBtb2RpZmllcnMgZm9yIGFueSBv bmUgZm9ybWF0IHRvIGRlc2NyaWJlIG91ciBjb21wcmVzc2lvbgo+ID4+IHZhcmlhbnRzLiBXZSBo YXZlIGEgbG90IG9mIG9uL29mZiB0b2dnbGVzIHdoaWNoIGxlYWRzIHRvIGNvbWJpbmF0b3JpYWwK PiA+PiBleHBhbnNpb24sIHNvIGl0IGNhbiBncm93IHByZXR0eSBxdWlja2x5ICh0aG91Z2ggSSBh bSB0cnlpbmcgdG8gbGltaXQKPiA+PiB0aGUgdmFsaWQgY29tYmluYXRpb25zIGFzIG11Y2ggYXMg cG9zc2libGUpLgo+ID4+IAo+ID4+IEhvdyBhYm91dCBpZiB0aGUgbWFzayBmaWxscyB1cCB0aGVu IFZJRElPQ19FTlVNX0ZNVCBjYW4gcmV0dXJuIGFub3RoZXIKPiA+PiBmbXRkc2Mgd2l0aCB0aGUg c2FtZSBGb3VyQ0MgYW5kIGRpZmZlcmVudCBtb2RpZmllciBiaXRtYXNrLCB3aGVyZSB0aGUKPiA+ PiBzZWNvbmQgb25lJ3MgbW9kaWZpZXIgYml0bWFzayBpcyBmb3IgdGhlIG5leHQgIk4iIG1vZGlm aWVycz8KPiA+Cj4gPiBJIHdhcyB0aGlua2luZyBhbG9uZyBzaW1pbGFyIGxpbmVzLCBidXQgaXQg Y291bGQgY2F1c2Ugc29tZSBwcm9ibGVtcyB3aXRoCj4gPiB0aGUgQUJJIHNpbmNlIGFwcGxpY2F0 aW9ucyBjdXJyZW50bHkgYXNzdW1lIHRoYXQgbm8gZm91cmNjIHdpbGwgYXBwZWFyCj4gPiB0d2lj ZSB3aGVuIGVudW1lcmF0aW5nIGZvcm1hdHMuIEFkbWl0dGVkbHksIHdlIG5ldmVyIGV4cGxpY2l0 bHkgc2FpZCBpbgo+ID4gdGhlIHNwZWMgdGhhdCB0aGF0IGNhbid0IGhhcHBlbiwgYnV0IGl0IGlz IGtpbmQgb2YgZXhwZWN0ZWQuCj4gPgo+ID4gVGhlcmUgYXJlIHdheXMgYXJvdW5kIHRoYXQsIGJ1 dCBpZiBwb3NzaWJsZSBJJ2QgbGlrZSB0byBhdm9pZCB0aGF0Lgo+ID4KPiA+IEluIHRoZW9yeSB0 aGVyZSBhcmUgdXAgdG8gMTI4IGJpdHMgYXZhaWxhYmxlIGJ1dCBJIGNhbid0IGhlbHAgdGhpbmtp bmcKPiA+IHRoYXQgaWYgeW91IGNyZWF0ZSBtb3JlIHRoYW4sIHNheSwgNjQgbW9kaWZpZXJzIGZv ciBhIEhXIHBsYXRmb3JtIHlvdQo+ID4gaGF2ZSBhIGJpZyBtZXNzIGFueXdheS4KPiA+Cj4gPiBJ ZiBJIGFtIHdyb25nLCB0aGVuIEkgbmVlZCB0byBrbm93IGJlY2F1c2UgdGhlbiBJIGNhbiBwcmVw YXJlIGZvciBpdAo+ID4gKG9yIHdob2V2ZXIgaXMgZ29pbmcgdG8gYWN0dWFsbHkgaW1wbGVtZW50 IHRoaXMuLi4pCj4gCj4gWW91J3JlIHByb2JhYmx5IHJpZ2h0LCBidXQgSSBjYW4ndCBzcGVhayBm b3IgZXZlcnlvbmUuIEZyb20gdGhlCj4gY3VycmVudCBzdGF0ZSBvZiBkcm1fZm91cmNjLmggaXQg bG9va3MgbGlrZSA2NCB3b3VsZCBiZSBwbGVudHkgKHRoZXJlCj4gYXJlbid0IGFueXdoZXJlIG5l YXIgNjQgbW9kaWZpZXJzIGV2ZW4gZGVmaW5lZCByaWdodCBub3cpLiBBZGRpbmcgaW4KPiB0aGUg QXJtIGNvbXByZXNzaW9uIGZvcm1hdHMgd2lsbCBleHBhbmQgaXQgYSBsb3QsIGJ1dCBzdGlsbCBu b3QgdG8gNjQKPiAoeWV0KS4KCkRvIGFsbCB0aG9zZSBtb2RpZmllcnMgbWFrZSBzZW5zZSBvbiB0 aGUgVjRMMiBzaWRlID8gSSBleHBlY3QgdGhhdCBzb21lIAptb2RpZmllcnMgd2lsbCBtb3N0bHkg YmUgdXNlZCBmb3IgYnVmZmVycyBzaGFyZWQgYmV0d2VlbiB0aGUgR1BVIGFuZCB0aGUgCmRpc3Bs YXkgZW5naW5lLCB3aGlsZSBvdGhlcnMgd2lsbCBiZSB1c2VkIGJ5IGNvZGVjcy4gVGhlIHNldHMg d2lsbCBsaWtlbHkgCm92ZXJsYXAsIGJ1dCBtaWdodCBub3QgYmUgaWRlbnRpY2FsLgoKPiA+IElm IHRoZSBudW1iZXIgb2YgbW9kaWZpZXJzIGlzIGV4cGVjdGVkIHRvIGJlIGxpbWl0ZWQgdGhlbiBt YWtpbmcgNjQgYml0cwo+ID4gYXZhaWxhYmxlIHdvdWxkIGJlIGdvb2QgZW5vdWdoLCBhdCBsZWFz dCBmb3Igbm93Lgo+ID4KPiA+IEJUVywgaXMgYSBtb2RpZmllciBhbHdheXMgb3B0aW9uYWw/IEku ZS4gZm9yIGFsbCBmb3VyY2NzLCBpcyB0aGUKPiA+IHVubW9kaWZpZWQgZm9ybWF0IGFsd2F5cyBh dmFpbGFibGU/IE9yIGFyZSB0aGVyZSBmb3VyY2NzIHRoYXQgcmVxdWlyZSB0aGUKPiA+IHVzZSBv ZiBhIG1vZGlmaWVyPwo+IAo+IFdlIGRvIGFjdHVhbGx5IGhhdmUgb25lIG9yIHR3byBmb3JtYXRz IHdoaWNoIGFyZSBvbmx5IHN1cHBvcnRlZCB3aXRoIGEKPiBtb2RpZmllciAob24gb3VyIEhXKS4K Ci0tIApSZWdhcmRzLAoKTGF1cmVudCBQaW5jaGFydAoKX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxA bGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxt YW4vbGlzdGluZm8vZHJpLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from galahad.ideasonboard.com ([185.26.127.97]:38549 "EHLO galahad.ideasonboard.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751011AbdHaOvK (ORCPT ); Thu, 31 Aug 2017 10:51:10 -0400 From: Laurent Pinchart To: Brian Starkey Cc: Hans Verkuil , Daniel Vetter , Nicolas Dufresne , "linux-media@vger.kernel.org" , jonathan.chai@arm.com, dri-devel Subject: Re: DRM Format Modifiers in v4l2 Date: Thu, 31 Aug 2017 17:51:33 +0300 Message-ID: <2089286.L0REA3WtaS@avalon> In-Reply-To: <20170830103040.GA19103@e107564-lin.cambridge.arm.com> References: <20170824111430.GB25711@e107564-lin.cambridge.arm.com> <20170830103040.GA19103@e107564-lin.cambridge.arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Sender: linux-media-owner@vger.kernel.org List-ID: Hi Brian, On Wednesday, 30 August 2017 13:32:01 EEST Brian Starkey wrote: > On Wed, Aug 30, 2017 at 11:53:58AM +0200, Hans Verkuil wrote: > > On 30/08/17 11:36, Brian Starkey wrote: > >> On Wed, Aug 30, 2017 at 10:10:01AM +0200, Hans Verkuil wrote: > >>> On 30/08/17 09:50, Daniel Vetter wrote: > >>>> On Tue, Aug 29, 2017 at 10:47:01AM +0100, Brian Starkey wrote: > >>>>> The fact is, adding special formats for each combination is > >>>>> unmanageable - we're talking dozens in the case of our hardware. > >>>> > >>>> Hm right, we can just remap the special combos to the drm-fourcc + > >>>> modifier style. Bonus point if v4l does that in the core so not > >>>> everyone has to reinvent that wheel :-) > >>> > >>> Probably not something we'll do: there are I believe only two drivers > >>> that are affected (exynos & mediatek), so they can do that in their > >>> driver. > >>> > >>> Question: how many modifiers will typically apply to a format? I ask > >>> because I realized that V4L2 could use VIDIOC_ENUMFMT to make the link > >>> between a fourcc and modifiers: > >>> > >>> https://hverkuil.home.xs4all.nl/spec/uapi/v4l/vidioc-enum-fmt.html > >>> > >>> The __u32 reserved[4] array can be used to provide a bitmask to modifier > >>> indices (for the integer menu control). It's similar to what drm does, > >>> except instead of modifiers mapping to fourccs it is the other way > >>> around. > >>> > >>> This would avoid having to change the modifiers control whenever a new > >>> format is set and it makes it easy to enumerate all combinations. > >>> > >>> But this only works if the total number of modifiers used by a single > >>> driver is expected to remain small (let's say no more than 64). > >> > >> In our current (yet to be submitted) description, we've got around a > >> dozen modifiers for any one format to describe our compression > >> variants. We have a lot of on/off toggles which leads to combinatorial > >> expansion, so it can grow pretty quickly (though I am trying to limit > >> the valid combinations as much as possible). > >> > >> How about if the mask fills up then VIDIOC_ENUM_FMT can return another > >> fmtdsc with the same FourCC and different modifier bitmask, where the > >> second one's modifier bitmask is for the next "N" modifiers? > > > > I was thinking along similar lines, but it could cause some problems with > > the ABI since applications currently assume that no fourcc will appear > > twice when enumerating formats. Admittedly, we never explicitly said in > > the spec that that can't happen, but it is kind of expected. > > > > There are ways around that, but if possible I'd like to avoid that. > > > > In theory there are up to 128 bits available but I can't help thinking > > that if you create more than, say, 64 modifiers for a HW platform you > > have a big mess anyway. > > > > If I am wrong, then I need to know because then I can prepare for it > > (or whoever is going to actually implement this...) > > You're probably right, but I can't speak for everyone. From the > current state of drm_fourcc.h it looks like 64 would be plenty (there > aren't anywhere near 64 modifiers even defined right now). Adding in > the Arm compression formats will expand it a lot, but still not to 64 > (yet). Do all those modifiers make sense on the V4L2 side ? I expect that some modifiers will mostly be used for buffers shared between the GPU and the display engine, while others will be used by codecs. The sets will likely overlap, but might not be identical. > > If the number of modifiers is expected to be limited then making 64 bits > > available would be good enough, at least for now. > > > > BTW, is a modifier always optional? I.e. for all fourccs, is the > > unmodified format always available? Or are there fourccs that require the > > use of a modifier? > > We do actually have one or two formats which are only supported with a > modifier (on our HW). -- Regards, Laurent Pinchart