From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <ed.sutter@alcatel-lucent.com>
Received: by yocto-www.yoctoproject.org (Postfix, from userid 118)
	id EDFA6E00874; Wed,  7 Jan 2015 05:45:40 -0800 (PST)
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	yocto-www.yoctoproject.org
X-Spam-Level: 
X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI
	autolearn=ham version=3.3.1
X-Spam-HAM-Report: * -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at
	http://www.dnswl.org/, high *      trust
	*      [135.245.210.20 listed in list.dnswl.org]
	* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
	*      [score: 0.0000]
Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-01.alcatel-lucent.com
	[135.245.210.20])
	by yocto-www.yoctoproject.org (Postfix) with ESMTP id 96F38E00823
	for <meta-freescale@yoctoproject.org>;
	Wed,  7 Jan 2015 05:45:34 -0800 (PST)
Received: from us70tusmtp2.zam.alcatel-lucent.com (unknown [135.5.2.64])
	by Websense Email Security Gateway with ESMTPS id 9B30A8A8F21B6;
	Wed,  7 Jan 2015 13:45:30 +0000 (GMT)
Received: from usnavsmail2.ndc.alcatel-lucent.com
	(usnavsmail2.ndc.alcatel-lucent.com [135.3.39.10])
	by us70tusmtp2.zam.alcatel-lucent.com (GMO) with ESMTP id
	t07DjU1R014930
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL);
	Wed, 7 Jan 2015 08:45:31 -0500
Received: from umail.lucent.com (umail.ndc.lucent.com [135.3.40.61])
	by usnavsmail2.ndc.alcatel-lucent.com (8.14.3/8.14.3/GMO) with ESMTP id
	t07DjTLw003613
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT);
	Wed, 7 Jan 2015 07:45:30 -0600
Received: from [135.222.138.133] (Ed-PC.mh.lucent.com [135.222.138.133])
	by umail.lucent.com (8.13.8/TPES) with ESMTP id t07DjS3R026981;
	Wed, 7 Jan 2015 07:45:29 -0600 (CST)
Message-ID: <54AD376A.7040606@alcatel-lucent.com>
Date: Wed, 07 Jan 2015 08:40:58 -0500
From: Ed Sutter <ed.sutter@alcatel-lucent.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
	rv:31.0) Gecko/20100101 Thunderbird/31.3.0
MIME-Version: 1.0
To: "Dmitriy B." <rzk333@gmail.com>
References: <54AC1D12.5040603@alcatel-lucent.com>
	<CAP9ODKqH_zE3a9_OgLUiO1ieULOKCLiUFHhYRAGwDzEhZOORSA@mail.gmail.com>
	<54AC324F.1010301@alcatel-lucent.com>
	<CAPhrY5NsnprKLenv6Ax_Cf9=RDPjRB74-Ra1y-Q+i3H-tj0wqg@mail.gmail.com>
	<54AC5D4A.2080501@alcatel-lucent.com>
	<CAPhrY5N+CNq7JK2+CkXHk8eamkdcJK8=py02y5Ru9y8d8FFVcQ@mail.gmail.com>
In-Reply-To: <CAPhrY5N+CNq7JK2+CkXHk8eamkdcJK8=py02y5Ru9y8d8FFVcQ@mail.gmail.com>
X-Scanned-By: MIMEDefang 2.64 on 135.3.39.10
Cc: "meta-freescale@yoctoproject.org" <meta-freescale@yoctoproject.org>
Subject: Re: webrtc question
X-BeenThere: meta-freescale@yoctoproject.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: Usage and development list for the meta-fsl-* layers
	<meta-freescale.yoctoproject.org>
List-Unsubscribe: <https://lists.yoctoproject.org/options/meta-freescale>,
	<mailto:meta-freescale-request@yoctoproject.org?subject=unsubscribe>
List-Archive: <http://lists.yoctoproject.org/pipermail/meta-freescale>
List-Post: <mailto:meta-freescale@yoctoproject.org>
List-Help: <mailto:meta-freescale-request@yoctoproject.org?subject=help>
List-Subscribe: <https://lists.yoctoproject.org/listinfo/meta-freescale>,
	<mailto:meta-freescale-request@yoctoproject.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Jan 2015 13:45:41 -0000
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit


>                From your description it seems that the webRTC will be the
>                 'communication channel' and the processing of the packages
>                 will be a
>                 second step (audio / video / whatever). For VP8 it
>         does work,
>                 as it is
>                 supported in Chromium.
>
>
>         Unfortunately, it is not that simple. webrtc & libjingle is a
>         long going project from Google that is done as a big C++
>         framework that has everything you need: encoding, decoding
>         libs, communication libs and debug stuff. Chrome is just
>         hooked up to use that framework when webrtc actions appear in
>         users javascript. webrtc does not depend on HTML5 video
>         playback for example, where we have with patches imx-vpu in
>         action. At least that is how it did work around a year ago.
>
>     Similarly, HTML5 doesn't use VP8 unless webrtc is involved right?
>     I'm hoping I don't need to get near any HTML at all, and can use
>     libjingle along with the audio and video engines to do this.
>
>
> IIRC, no, <video> tag is handled by Chrome, but when you start "webrtc 
> context", Chrome just proxies everything to webrtc library and waits 
> for answer from it. It is due to webrtc nature of being a separate 
> library ready for integration to any application. This is how Google 
> wants it.
>
> What do you mean by not getting near HTML? If you just want to use 
> webrtc in your application it is one situation, when you want a 
> "webrtc context" inside Chrome being accelerated - that is another 
> situation. Sadly, both of these situations will need you to dig into 
> webrtc sources and add imx-vpu support. There is no "easy way" here.
My use of this will (hopefully) not require a browser; hence, no HTML.  
I just want to use webrtc libraries in my application.
I've dug into this a few years ago (pre-webrtc-availability), using VP8 
alone in a different project.  Wasn't too messy but that
was prior to the point when all this "stuff" was integrated into 
webrtc.   Also, then I was working on a PC (no acceleration needed).
The difference here is that iMX6-quad != Intel-I7-quad; so I assume I'll 
need as much acceleration as I can get.
Really not sure what burden this will put on the iMX6Q, so that is yet 
to be determined.
>
>
>
>             Yea, the first step is to make my iMX6 based device be
>         compatible
>             with the
>             communication channels (and data formats, i.e. VP8/opus/etc..)
>             used/supported
>             by webrtc.  The difference will be that (in certain modes)
>         there
>             is no user on
>             the device; it will be headless and autonomous, so I need
>         to be
>             able to feed
>             media to and extract media from the underlying webrtc base
>             platform on the iMX6.
>             Does Chromium provide any non-browser type of hooks into
>         webrtc
>             down below?
>
>
>         No, chromium links with webrtc which links with libvpx for
>         encoding/decoding. Libvpx then decides what to use.
>
>         I investigated this some time ago while working on a similar
>         device (webrtc app linked with Qt5 front end on ARM
>         linux-native device communicating other similar devices, it
>         was a robotics project). You need to integrate imx vpu
>         encoding/decoding support to webrtc/libvpx internals or use
>         Android where it should be passed through from standard video
>         acceleration frameworks.
>
>     So does that imply that there is no hardware (imx-vpu)
>     acceleration support for VP8?
>     If this is true, is there any data on how the impacts CPU usage
>     when VP8 is used?
>
>
> Chrome HTML5 VP8 video playback acceleration != WebRTC VP8 decoding 
> acceleration! (see explanation above about webrtc being separate 
> library with own encoders/decoders).
> HTML5 VP8 playback works, webrtc video decoding does not.
Hmmm...(get ready for a naive comment)...
I would have thought once you get into VP8 then regardless of who calls 
it, the acceleration would be the same.
I realize it can be tuned and run in different modes; however I would 
have thought that
the bowels of the algorithm that acceleration would be applied to would 
be below this.
>
>     BTW... did you ever complete that project?  If not, was it due to
>     iMX6/webrtc issues?
>
>
> This was a long going project on Samsung Exynos SoCs and we ended up 
> just running it there unaccelerated, just bare webrtc+libvpx, since 
> 480p was enough for robotics. We were trying to move to i.MX, but at 
> that point only LTIB existed and it was soft-float (which was a 
> showstopper).
>
> But now, you have much more chances to get webrtc running great on 
> i.MX when HTML5 acceleration works in Chrome. Just adapt that code to 
> webrtc's internals and link your Chrome with it.
>
> I remember asking Carlos (author of imx patches for Chrome) about this 
> on IRC, but, unfortunately, I don't remember his answers. Hope he will 
> explain better than me :)
>
> Other than here, you might want to ask at webrtc google group 
> (https://groups.google.com/forum/#!forum/discuss-webrtc 
> <https://groups.google.com/forum/#%21forum/discuss-webrtc>) about how 
> to integrate video acceleration to the library.
>
Thanks very much for your comments.
Regards,
Ed