From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ptmx.org (ptmx.org [178.63.28.110]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id 9C5A9E00975 for ; Thu, 20 Mar 2014 01:30:29 -0700 (PDT) Received: from [192.168.178.101] (chello080108088242.14.11.vie.surfer.at [80.108.88.242]) by ptmx.org (Postfix) with ESMTPSA id 91FDC204AD; Thu, 20 Mar 2014 09:30:27 +0100 (CET) Message-ID: <532AA723.1010901@pseudoterminal.org> Date: Thu, 20 Mar 2014 09:30:27 +0100 From: Carlos Rafael Giani User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Eric Nelson , Christian Betz References: <532A02E5.7060607@boundarydevices.com> <532A057E.2010805@pseudoterminal.org> <532A1CC6.1020609@boundarydevices.com> <532A5782.4050809@boundarydevices.com> In-Reply-To: <532A5782.4050809@boundarydevices.com> Cc: "meta-freescale@yoctoproject.org" Subject: Re: Chromium acceleration X-BeenThere: meta-freescale@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Usage and development list for the meta-fsl-* layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 08:30:33 -0000 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi Eric and Christian, I wanted to avoid _any_ copy involving the CPU, including any texture upload. The Vivante GLES direct texture extension can color-convert, yes, but it can also read pixels directly from a DMA buffer, without any copy. This is what I use for the GLES-based sink in my GStreamer plugins, and allows me to play 1080p videos with less than 6% CPU usage. v4lsink would be another idea, but I am a bit skeptical, because it wouldn't play well with any X11 or Wayland compositor, since it would essentially act like an overlay. This is even worse with the isink (if the regular, non-overlay layer is used), because then, the sink simply overwrites pixels drawn by X11. What the isink does is to make sure the pixels are blit in the area of its output window, so it does not overwrite anything outside. But again, with desktop compositing this would break down. (Also, the isink is quite slow.) I would consider this one texture upload only if absolutely necessary. My idea was to actually produce the texture in the decoder myself, and just send the texture GL name around to the renderer. This would require that everything is drawn with GLES of course - what would a software renderer do with a GL name? As for the decoder itself: I implemented it in the Chromium media framework, in media/ . I simply took the vpx decoder code, copied it, and modified it to use the VPU. I had VP8, MPEG2, MPEG4, and h264 decoding working. It wasnt much code, but unfortunately, the interfaces tend to change between versions, so the code would have to be updated for the newest Chromium version. It is not much code, I will try to clean it up a bit and post it. In addition, I will ask the colleagues for the other patches for accelerating the 2D and WebGL rendering. Carlos On 2014-03-20 03:50, Eric Nelson wrote: > Thanks Christian, > > On 03/19/2014 06:29 PM, Christian Betz wrote: >> The Vivante libraries can map the physical buffers produced by >> the VPU directly, so they could do format conversion on their >> way to the graphics stack if the Chromium bindings have access >> to that (the "single copy" I referred to above). >> >> >> in my experience "zero-copy" really means zero memcpy() (or similar >> system call, or really anything that involves the cpu to move around >> image data. /metadata /such as buffer pointers is generally allowed). in >> other words having the GPU do format conversion isn't considered a copy. >> >> here's a great article on this general technique implemented in >> webkit/gtk+: >> http://blogs.igalia.com/vjaquez/2013/07/26/composited-video-support-in-webkitgtk/ >> >> > > That's a nice article, and if Webkit (Blink) retains the same mapping, > video buffers from the VPU would end up hitting a RenderObject and > the only "copy" would be in rendering that to the Framebuffer. > > Since we're on the fat end of the pipe here, that copy can be expensive > and might limit things to 1x1080P decode, but I'd be happy with that. > > In the mfw_v4lsink case at least, this can be bypassed by having > the V4L2 buffer filled by the VPU and also displayed by the IPU > (such that copy), but aiming there would keep the video out of the > rest of the RenderLayer tree (I think). > > This is what I was so awkwardly trying to suggest. > >> moving beyond semantics... let me tell you what i know about this >> particular issue: >> >> there is source code in chromium to accomplish a similar goal. >> specifically there is a "generic" android driver, which works out of the >> box with i.mx6 because freescale implements the standard android video >> decode interface that allows VPU -> GPU texture uploading (don't quote >> me on this, i'm just reading the source code: >> https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/android_video_decode_accelerator.cc) >> >> >> sadly there is no such standard interface on GNU/Linux. outside of >> android (i.e. chromeos), there are drivers for particular chipsets, like >> samsung exynos. here is the source for that: >> https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/exynos_video_decode_accelerator.cc >> >> >> the group should also be aware of the current situation regarding these >> modules /only/ working when certain flags to enable "chromeos" are used >> (explained here: >> https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode). in other >> words: this stuff doesn't work on your x86 with the chrome you download >> from google. >> > > Again, thanks for the info. > > Regards, > > > Eric