From mboxrd@z Thu Jan 1 00:00:00 1970 From: Laurent Pinchart Subject: Re: [PATCH v4 02/14] Documentation: media: description of DMABUF importing in V4L2 Date: Sat, 21 Apr 2012 19:10:27 +0200 Message-ID: <12963949.ALUT5xjWQF@avalon> References: <1334332076-28489-1-git-send-email-t.stanislaws@samsung.com> <4F91559D.6060900@samsung.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: In-Reply-To: Sender: linux-media-owner@vger.kernel.org To: =?ISO-8859-1?Q?R=E9mi?= Denis-Courmont Cc: Tomasz Stanislawski , Mauro Carvalho Chehab , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, airlied@redhat.com, m.szyprowski@samsung.com, kyungmin.park@samsung.com, sumit.semwal@ti.com, daeinki@gmail.com, daniel.vetter@ffwll.ch, robdclark@gmail.com, pawel@osciak.com, linaro-mm-sig@lists.linaro.org, hverkuil@xs4all.nl, subashrp@gmail.com List-Id: dri-devel@lists.freedesktop.org Hi R=E9mi, On Friday 20 April 2012 15:03:17 R=E9mi Denis-Courmont wrote: > On Fri, 20 Apr 2012 14:25:01 +0200, Tomasz Stanislawski wrote: > >>> The USERPTR simplifies userspace code but introduce > >>> a lot of complexity problems for the kernel drivers > >>> and frameworks. > >>=20 > >> It is not only a simplification. In some cases, USERPTR is the onl= y I/O > >> method that supports zero copy in pretty much any circumstance. > >=20 > > Only for devices that have its own IOMMU that can access system mem= ory. >=20 > Newer versions of the UVC driver have USERTPR, and simingly gspca see= ms > too. That is practically all USB capture devices... That might be > irrelevant for a smartphone manufacturer. That is very relevant for d= esktop > applications. >=20 > > Moreover the userptr must come from malloc or be a mmaped file. > > The other case are drivers that touch memory using CPU in the kerne= l > > space like VIVI or USB drivers. >=20 > I'd argue that USB is the most common case of V4L2 on the desktop... >=20 > >> When the user cannot reliably predict the maximum number of requir= ed > >> buffers, predicts a value larger than the device will negotiate, o= r > >> needs buffers to outlive STREAMOFF (?), MMAP requires memory copyi= ng. > >> USERPTR does not. > >=20 > > What does outlive STREAMOFF means in this context? >=20 > Depending how your multimedia pipeline is built, it is plausible that= the > V4L2 source is shutdown (STREAMOFF then close()) before buffers comin= g from > it are released/destroyed downstream. I might be wrong, but I would e= xpect > that V4L2 MMAP buffers become invalid after STREAMOFF+close()? If the buffer is mmap()ed to userspace, it will not be freed before bei= ng=20 munmap()ed. > > Anyway, IMO allocation of the buffers at VIDIOC_REQBUFS was not the= best > > idea because it introduces an allocation overhead for negotiations = of > > the number of the buffers. An allocation at mmap was to late. There= is a > > need for some intermediate state between REQBUFS and mmap. The ioct= l > > BUF_PREPARE may help here. > >=20 > > Can you give me an example of a sane application is forced to negot= iate > > a larger number of buffers than it is actually going to use? >=20 > Outside the embedded world, the application typically does not know w= hat > the latency of the multimedia pipeline is. If the latency is not know= n, the > number of buffers needed for zero copy cannot be precomputed for REQB= UFS, > say: >=20 > count =3D 1 + latency / frame interval. >=20 > Even for a trivial analog TV viewer application, lip synchronization > requires picture frames to be bufferred to be long enough to account = for > the latency of the audio input, dejitter, filtering and audio output.= Those > values are usually not well determined at the time of requesting buff= ers > from the video capture device. Also the application may want to play = nice > with PulseAudio. Then it will get very long audio buffers with very f= ew > audio periods... more latency. >=20 > It gets harder or outright impossible for frameworks dealing with > complicated or arbitrary pipelines such as LibVLC or gstreamer. There= is > far too much unpredictability and variability downstream of the V4L2 = source > to estimate latency, and infer the number of buffers needed. If I'm not mistaken VIDIOC_CREATEBUF allows you to create additional bu= ffers=20 at runtime. You can thus cope with a latency increase (provided that th= e=20 allocation overhead isn't prohibitive, in which case you're stuck whate= ver=20 method you select). Deleting buffers at runtime is currently not possib= le=20 though. > >> Now, I do realize that some devices cannot support USERPTR efficie= ntly, > >> then they should not support USERPTR. > >=20 > > The problem is not there is *NO* device that can handle USERPTR rel= iably. > > The can handle USERPTR generated by malloc or page cache (not sure)= =2E > > Memory mmaped from other devices, frameworks etc may or may not wor= k. > > Even if the device has its IOMMU the DMA layer provides no generic = way to > > transform from one device to the mapping in some other device. >=20 > I'm not saying that USERPTR should replace DMABUF. I'm saying USERPTR= has > advantages over MMAP that DMABUF does not seem to cover as yet (if on= ly > libv4l2 would not inhibit USERPTR...). >=20 > I'm definitely not saying that applications should rely on USERPTR be= ing > supported. We agree that not all devices can support USERPTR. >=20 > > The userptr has its niches were it works pretty well like Web cams = or > > VIVI. > > > > I am saying that if ever DMABUF becomes a good alternative for USER= PTR > > than maybe we should consider encouraging dropping USERPTR in the n= ew > > drivers as 'obsolete' feature and providing some emulation layer in > > libv4l2 for legacy applications. >=20 > Sure. --=20 Regards, Laurent Pinchart