From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f49.google.com (mail-la0-f49.google.com [209.85.215.49]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id A9C9CE00343 for ; Fri, 5 Jul 2013 16:04:24 -0700 (PDT) Received: by mail-la0-f49.google.com with SMTP id ea20so2396707lab.22 for ; Fri, 05 Jul 2013 16:04:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=R+OZommp/gEvm8m/VCfx2s6Zaw0jbKeEXr/DLPpiglY=; b=khA2djUWc8+dDODhvu01S/Fx05BgQOxr5+CLwgyQF3PyAr8/JVcOqCGtUUCdm1QkL0 /qpuq0TLk6HAiFMVj5zGxYsOh8EiVpaVijnRGYxTjR7U9GiVI0Aq6PmRMDLtiEDDqU6m KbTEZp7a/G9UMK17Xht/GqsgMnmtSZhEOba0WkPyjQG6+PuGepb/3hWJFgreJI10kna8 md0rrJq6rkxz3U1cDiBUt82pbvpNQZm9mUzy+viPlBFFBfCL9t5zLWCkUfjr09Ypx98+ uHF6f9L7yElJTAmOCYgFWvOWkZLlM3ISan/euvGn9F9YlQHIhsWImqWqe91tEKDBmrXH C3VQ== X-Received: by 10.112.137.225 with SMTP id ql1mr6584237lbb.26.1373065462991; Fri, 05 Jul 2013 16:04:22 -0700 (PDT) Received: from [192.168.0.55] ([70.96.116.236]) by mx.google.com with ESMTPSA id b8sm3590663lbr.12.2013.07.05.16.04.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 05 Jul 2013 16:04:22 -0700 (PDT) Message-ID: <51D750F2.3020209@boundarydevices.com> Date: Fri, 05 Jul 2013 16:04:18 -0700 From: Eric Nelson User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: Otavio Salvador References: <1797012.Po2ZnTUHJW@rudolf> <1891888.TfnZ3vpNCR@rudolf> In-Reply-To: X-Gm-Message-State: ALoCoQmq6Dfy3Ei4jHfHvXtiT3E6kFUmsGzKZ2heD/5LT5FVJsGyW5XlAOwKSw2oSQfYoSON7hzC Cc: "meta-freescale@yoctoproject.org" , Rogerio Nunes Subject: Re: QtMultimedia on i.MX6 X-BeenThere: meta-freescale@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Usage and development list for the meta-fsl-* layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jul 2013 23:04:28 -0000 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 07/04/2013 02:42 PM, Otavio Salvador wrote: > On Thu, Jul 4, 2013 at 12:14 PM, Thomas Senyk > wrote: >> On Thursday, 04 July, 2013 17:03:03 Thomas Senyk wrote: >>> Hi, >>> >>> I've started to investigate to see the QtMultimedia-gstreamer integration on >>> the i.MX6 could be optimized. >>> >>> I'm writing this mail to sync with others who might already done some work >>> or investigations on this matter, to get hints where to start and what's >>> worth to investigate. >>> >>> State right now: >>> >>> The HW-decoders are used (according to gstreamer log). >>> That's a very good start/out of the box experience! Good job..freescale I >>> guess? >>> >>> There is still a lot of cpu load: >>> - 720p: smooth playback but~18% complete cpu load => ~70% of one core >>> - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one >>> core + some additional cpu load >>> >>> The obvious reason for this cpu load is some sort of memcopy and/or >>> conversion. >>> ... gstreamer doesn't render into gpu-memory directly. >>> ... or (if you want it the other way around) the gstreamer (/cpu) memory >>> isn't mapped into gpu-memory (which is possibly on the imx6) >>> >>> Started to do some digging: >>> >>> One of the first things I tried was to compile gst-plugin-gl (which is part >>> of the meta-fsl-arm layer via a .bbappend). >>> The current recipe is driver-version-1.1.0 based and is not building out of >>> the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz >>> (based on 4.0.0) building.** >>> >>> So, after I got it deployed I hoped for magic to happen and everything is >>> perfect now ... no big surprise: it's not :) nothing changed, possibly the >>> gst-pipline build by qtmultimedia is the ame. >>> >>> The reason for this mail / some questions: >>> >>> So I though, I write this mail before I waste a lot of time. >>> What does people think is the best way forward? >>> Try to convince QtMultimedia to use glsink? >>> Try to use a fakesink/memory-based-sink and try to map the cpu memory into >>> gpu memory via vivante APIs? >>> >>> Does anyone have c/c++ code I could try? >>> Ideally for direct framebuffer/EGL use (I'm not using X11) >>> I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git >>> .. but it's crashing rather early and I'm not sure the if the time is well >>> spend fix this code? Anyone used it before? >>> Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups? >>> >>> >>> Greet >>> Thomas >> >> Ok ... what is it with me and sending mails 1minute to early all the time ... >> After I pressed "send", I had a very simple and obvious idea .. why not just >> simply try glimagesink instead of mfw_v4lsink: >> >> gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux ! >> vpudec ! glimagesink >> >> ... long shot, but: >> It's actually working! >> It just opens /dev/fb0 and renders the video into it. >> >> 1080p with 0%(!!) cpu load! >> Nice! I stopped short of testing this out because I thought that X was required. A quick browse of the sources indicates that there's some really good stuff inside, including what appears to be the ability to upload planar YUV into a texture: http://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/src/629e1efaf87f379a36456f4e42cdec0ca2ed7e2a/gst-libs/gst/gl/gstglvivante.c?at=fsl/0.10.3#cl-103 >> I think that pretty much answers all questions I had ;) >> The way forward: patch QtMultimedia to use glimagesink >> ... I hope it's going to be trivial to just give him a active FBO or texture. >> >> >> Sorry to everybody who read my way to long mail ;) > > It was very good read. +1 > By the way, Rogerio (add in Cc) started to try > to get the gst-plugin-gl code more manageble in bitbucket > (https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/all) > but it was not finished. Maybe you could help on this and we could try > to merge the Vivante backend upstream? This would allow us to drop the > bbappend completely. > +1 >>> ** some more info in gst-plugin-gl: >>> It's 'detected' on the board: >>> root@nitrogen-1:~# gst-inspect |grep gl >>> opengl: glupload: OpenGL upload >>> opengl: gldownload: OpenGL video maker >>> opengl: glimagesink: OpenGL video sink >>> opengl: glfiltercube: OpenGL cube filter >>> opengl: gleffects: Gstreamer OpenGL Effects >>> >>> ... if someone is interested on how I got it building, let me know. If I got >>> something useful out of it (read: "it's tested") I'll start a discussion on >>> how to upstream it properly into meta-fsl-arm (by the way: I find the >>> current setup of how to bbappend and it's patch looks like very weird) I'm interested in the details of how you added Qt5 to your build. Qt4 seems to be directly accessible in Yocto, but not Qt5. Regards, Eric