From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-we0-f173.google.com (mail-we0-f173.google.com [74.125.82.173]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id 1C80EE01482 for ; Thu, 4 Jul 2013 08:15:05 -0700 (PDT) Received: by mail-we0-f173.google.com with SMTP id x54so1222078wes.18 for ; Thu, 04 Jul 2013 08:15:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:subject:date:message-id:organization:user-agent:in-reply-to :references:mime-version:content-transfer-encoding:content-type :x-gm-message-state; bh=hspeFdwMKa51QY+rK6VnP7tpkc7iZ7k8Qt0c0qaPv/o=; b=dS96P3Exy2Zfu/ojOlKGbYLXzXmXMYdpQpwsAAlyTWip3mpeV9MgAv2AfjnILEXrX/ 2A2gccWEYrLDRYBcqmJIf0IuBAOmM0VuBscgop9VVIJijxxqd2+yua7F0cSltkASdv/P L3qMkqPBdm7N6YSt+3IEKkIb7Aa0LimNDdMlj+aQUGv6a191Gvpk1+IOWi0dbHjOBDxW B6s3mdZwLwzpcpUChDsGFac1GfCqhNyGB5y2RnWfL/zKLvK4yFQxTO9gUfsb++LCoQ6J O7freGJjbldCdWjpCPodqIN+9btKngugWDRxBFXLjafssi/CHGs4CEsh6USB4mbhNwCT J5rg== X-Received: by 10.194.176.41 with SMTP id cf9mr3673531wjc.66.1372950904873; Thu, 04 Jul 2013 08:15:04 -0700 (PDT) Received: from rudolf.localnet (ppp-82-135-84-132.dynamic.mnet-online.de. [82.135.84.132]) by mx.google.com with ESMTPSA id m6sm1791149wiy.8.2013.07.04.08.15.03 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 04 Jul 2013 08:15:04 -0700 (PDT) From: Thomas Senyk To: meta-freescale@yoctoproject.org Date: Thu, 04 Jul 2013 17:14:46 +0200 Message-ID: <1891888.TfnZ3vpNCR@rudolf> Organization: Pelagicore AG User-Agent: KMail/4.10.4 (Linux/3.9.8-1-ARCH; KDE/4.10.4; x86_64; ; ) In-Reply-To: <1797012.Po2ZnTUHJW@rudolf> References: <1797012.Po2ZnTUHJW@rudolf> MIME-Version: 1.0 X-Gm-Message-State: ALoCoQluaVUPGWt8HJ44Z/fQljKmE7KSjDmEAMxi4E7W8CJ235jHD+fV2gQTUYiUUMY/4N9JQ5jk Subject: Re: QtMultimedia on i.MX6 X-BeenThere: meta-freescale@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Usage and development list for the meta-fsl-* layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jul 2013 15:15:08 -0000 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" On Thursday, 04 July, 2013 17:03:03 Thomas Senyk wrote: > Hi, > > I've started to investigate to see the QtMultimedia-gstreamer integration on > the i.MX6 could be optimized. > > I'm writing this mail to sync with others who might already done some work > or investigations on this matter, to get hints where to start and what's > worth to investigate. > > > > State right now: > > The HW-decoders are used (according to gstreamer log). > That's a very good start/out of the box experience! Good job..freescale I > guess? > > There is still a lot of cpu load: > - 720p: smooth playback but~18% complete cpu load => ~70% of one core > - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one > core + some additional cpu load > > The obvious reason for this cpu load is some sort of memcopy and/or > conversion. > ... gstreamer doesn't render into gpu-memory directly. > ... or (if you want it the other way around) the gstreamer (/cpu) memory > isn't mapped into gpu-memory (which is possibly on the imx6) > > > > > Started to do some digging: > > One of the first things I tried was to compile gst-plugin-gl (which is part > of the meta-fsl-arm layer via a .bbappend). > The current recipe is driver-version-1.1.0 based and is not building out of > the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz > (based on 4.0.0) building.** > > So, after I got it deployed I hoped for magic to happen and everything is > perfect now ... no big surprise: it's not :) nothing changed, possibly the > gst-pipline build by qtmultimedia is the ame. > > > > The reason for this mail / some questions: > > So I though, I write this mail before I waste a lot of time. > What does people think is the best way forward? > Try to convince QtMultimedia to use glsink? > Try to use a fakesink/memory-based-sink and try to map the cpu memory into > gpu memory via vivante APIs? > > Does anyone have c/c++ code I could try? > Ideally for direct framebuffer/EGL use (I'm not using X11) > I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git > .. but it's crashing rather early and I'm not sure the if the time is well > spend fix this code? Anyone used it before? > Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups? > > > Greet > Thomas Ok ... what is it with me and sending mails 1minute to early all the time ... After I pressed "send", I had a very simple and obvious idea .. why not just simply try glimagesink instead of mfw_v4lsink: gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux ! vpudec ! glimagesink ... long shot, but: It's actually working! It just opens /dev/fb0 and renders the video into it. 1080p with 0%(!!) cpu load! I think that pretty much answers all questions I had ;) The way forward: patch QtMultimedia to use glimagesink ... I hope it's going to be trivial to just give him a active FBO or texture. Sorry to everybody who read my way to long mail ;) > > ** some more info in gst-plugin-gl: > It's 'detected' on the board: > root@nitrogen-1:~# gst-inspect |grep gl > opengl: glupload: OpenGL upload > opengl: gldownload: OpenGL video maker > opengl: glimagesink: OpenGL video sink > opengl: glfiltercube: OpenGL cube filter > opengl: gleffects: Gstreamer OpenGL Effects > > ... if someone is interested on how I got it building, let me know. If I got > something useful out of it (read: "it's tested") I'll start a discussion on > how to upstream it properly into meta-fsl-arm (by the way: I find the > current setup of how to bbappend and it's patch looks like very weird)