From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ptmx.org (ptmx.org [178.63.28.110]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id 197C0E00A87 for ; Wed, 5 Mar 2014 10:06:22 -0800 (PST) Received: from [10.1.14.248] (unknown [91.114.0.140]) by ptmx.org (Postfix) with ESMTPSA id 8AD4C28A90 for ; Wed, 5 Mar 2014 19:06:21 +0100 (CET) Message-ID: <5317679F.4030109@pseudoterminal.org> Date: Wed, 05 Mar 2014 19:06:23 +0100 From: Carlos Rafael Giani User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: meta-freescale@yoctoproject.org References: <5315EB25.9020903@mlbassoc.com> <53176264.8030108@boundarydevices.com> <53176464.8030109@mlbassoc.com> <5317661E.5000905@boundarydevices.com> In-Reply-To: <5317661E.5000905@boundarydevices.com> Subject: Re: [meta-fsl-arm] Using gstreamer on Nitrogen6x/SabreLite X-BeenThere: meta-freescale@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Usage and development list for the meta-fsl-* layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 18:06:23 -0000 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 2014-03-05 18:59, Eric Nelson wrote: > Hi Gary, > > On 03/05/2014 10:52 AM, Gary Thomas wrote: >> On 2014-03-05 10:44, Eric Nelson wrote: >>> Hi Gary, >>> >>> On 03/04/2014 08:03 AM, Gary Thomas wrote: >>>> I have a SabreLite with the OV5642 camera. I'd like to capture >>>> some video and display it on the screen. Here's my gstreamer >>>> pipeline: >>>> >>>> gst-launch -e -vvv mfw_v4lsrc device=/dev/video0 num-buffers=100 >>>> typefind=true \ >>>> ! "video/x-raw-yuv, format=(fourcc)I420, width=640, >>>> height=480, framerate=(fraction)30/1" \ >>>> ! ffmpegcolorspace \ >>>> ! ximagesink >>>> >>>> What I don't understand is why the format from mfw_v4lsrc has to >>>> be I420 when the OV5642 [kernel] driver seems to only support YUYV >>>> (yuyv422)? >>>> >>>> To further confuse, I can grab a frame like this: >>>> yavta -fYUYV -s640x480 -F -c1 /dev/video0 >>>> and the V4l2 subsystem tells me this sensor is YUYV: >>>> root@nitrogen6x:~# v4l2-ctl -d /dev/video0 --list-formats >>>> ioctl: VIDIOC_ENUM_FMT >>>> Index : 0 >>>> Type : Video Capture >>>> Pixel Format: 'YUYV' >>>> Name : >>> >>> This appears to be a bug in the video driver (mxc_v4l2_capture) with >>> respect to enumeration (not a strong point for the drivers). >>> >>> The driver (mxc_v4l2_capture) appears to support a wide variety of >>> pixel formats through the magic of the IPU. >>> >>> The mfw_v4lsrc plugin appears to be hard-coded to support only >>> UYVY and I420 on i.MX6 though (and NV12 on i.MX51). >>> >>> Note that your pipeline is very expensive, and would benefit from >>> using a sink that can support YUV natively (mfw_isink, mfw_v4lsink) >>> or that can do the conversion in hardware (glimagesink). >> >> Thanks. I only used ximagesink as an example (one that also >> works on the desktop) for others to see. In the end, the video >> will probably be packaged into some container format (.mp4) and/or >> streamed, so those other methods don't help much. >> >> Is it possible to use the v4l2src gstreamer element with this video? >> I haven't been able to make that work at all... >> > Not without some hacking of the kernel driver(s), and after that, > you'd need to figure out how to handle allocation of DMA'able > buffers, and so on. > > If you were to undertake that, I'd recommend taking a hard look > at Carlos's GStreamer-1.0 in the process, since I understand that > the buffer-chaining process is different in 1.0 (and cleaner in 1.0). Philip Craig kindly submitted patches to the gstreamer-imx project to add a v4l2src element that uses i.MX6 specifics. And yes, the allocation scheme is much cleaner in GStreamer 1.0. In 0.10, it was only possible through hacks. I implemented an imxeglvivsink in gstreamer-imx, which is a video sink that outputs YUV frames located in DMA buffers with OpenGL ES and the Vivante GPU drivers' direct texture extension. This extension allows for using DMA buffers as pixel source for OpenGL ES textures. Net result is that the frames are shown directly without involving a CPU copy. I usually measure ~7% CPU usage for typical 1080p videos , more if some nontrivial audio encoding has been used (like AC3). ximagesink should never be used for video (unless it is the only sink that is available). When possible, either use a platform specific sink, like the mfw ones for 0.10 of imxeglvivsink in my 1.0 plugins, or at least xvimagesink. (But I realize that XVideo is not exposed in the X11 drivers..)