From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id 2372BE00B01 for ; Tue, 1 Apr 2014 12:22:20 -0700 (PDT) Received: by mail-pd0-f177.google.com with SMTP id y10so9958021pdj.22 for ; Tue, 01 Apr 2014 12:22:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type; bh=RVze1opjsc7/rq3zcdw+GKYA0aOsIbo9bNJgSMD4Knw=; b=T17bsCXDZANJ2lPQ8xYVFex3ypzGZ2NJk5helxUZqnMNkXWmMqx1UGAr2ooXtkYuJT xYBoIM8NoVF41uforwqKJnHGQOqTmA7yvgmKW0ZXy5CRiETdmQeJoS2pfIbx6DWVCVz0 R10Vkt/qtBqjrBwOVqWml3FEdqzUUYT5CxBsB0T7Nd+Yei+tgI/bNAR2/bdoJJKqC18n +HxNQkh7iyyvdL5Cf5NtL1wXgR7GPgNRk1RCfJ+GIdLj6IVEAmdL9JjU/FzrMwOdPIYN l7+fZS//7XpE8mqTBsDJ6/WmDw0pgjN5YKvEPoZale63QUFehyYQoLhRCUwUplnpZLaM OfQg== X-Gm-Message-State: ALoCoQmPmD63uUP06oaRVXnkLedU110ydQaP17IuEhXl7jImu3vwV08TTTIakqQf+FQAcFmBL1Yf X-Received: by 10.66.5.135 with SMTP id s7mr3763829pas.154.1396380139819; Tue, 01 Apr 2014 12:22:19 -0700 (PDT) Received: from [192.168.0.53] ([63.226.49.26]) by mx.google.com with ESMTPSA id yi3sm51637125pbb.51.2014.04.01.12.22.16 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 12:22:18 -0700 (PDT) Message-ID: <533B11E6.5040103@boundarydevices.com> Date: Tue, 01 Apr 2014 12:22:14 -0700 From: Eric Nelson User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Carlos Rafael Giani , Christian Betz , "meta-freescale@yoctoproject.org" References: <532A02E5.7060607@boundarydevices.com> <532A057E.2010805@pseudoterminal.org> <532A1CC6.1020609@boundarydevices.com> <532A5782.4050809@boundarydevices.com> <532AA723.1010901@pseudoterminal.org> <532B7776.5090700@pseudoterminal.org> In-Reply-To: <532B7776.5090700@pseudoterminal.org> Subject: Re: Chromium acceleration X-BeenThere: meta-freescale@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Usage and development list for the meta-fsl-* layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 19:22:22 -0000 X-Groupsio-MsgNum: 7968 Content-Type: multipart/mixed; boundary="------------090608050908060005050204" --------------090608050908060005050204 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Thanks again Carlos, On 03/20/2014 04:19 PM, Carlos Rafael Giani wrote: > > > > Back then we needed some hw decoding quickly, so we did not look further > into this, since we had spent a lot of time getting the 2D acceleration > stable already. I could only briefly look at the exynos accelerator > code, but it does look like the right direction, although the VPU uses > physical addresses directly instead of dmabuf FDs. I am currently > writing a frontend for the libfslvpuwrap, which takes care of certain > complexities (like being able to pass userdata pointers to frames, to > make it possible to associate input and output frames, which is > essential for correct timestamping, especially when things like h264 > frame reordering are active). This frontend will hide these things in a > future gstreamer-imx release to make the decoder code clearer and easier > to maintain, and is intended to be reusable, for example for Chromium > integration, or XBMC. Beyond that, my patches are probably not that > helpful anymore, since they were based on the vpx decoder way of > decoding, and I agree that the exynos code is a better starting point. > Nevertheless, I attached the patch. Keep in mind that it is very basic, > old, and wasn't further developed on because it was "good enough" for > the project. > > But here are some additional notes from our efforts to accelerate > Chromium on imx6 (keep in mind these apply to version 32.0.1664.3 ) : > > 1. We started the google-chrome binary with these switches: > --ignore-gpu-blacklist --enable-gpu --use-gl=egl > --enable-accelerated-2d-canvas > > 2. To make building Chromium easier, we switched to component build (by > default, it is all linked into one enormous binary). I attached a patch > that was necessary to fix some minor issues. Perhaps it is not necessary > anymore. > > 3. Chromium tried to load libEGL with dlopen() , which caused problems, > because with the Vivante libraries, libEGL also needs libGAL. Usually, > build scripts just add -lEGL -lGAL to the linker line. With dlopen() > this wasn't possible of course. We patched the gyp scripts to link > against this libraries during building instead. I attached this patch. > It is really a hack, because it circumvents the sandboxing process (this > is why Chromium loads EGL and GLESv2 with dlopen() ). > Mahyar updated these patches to apply against the chromium-35.0.1883.0 build currently in meta-browser. Additional notes to follow, but this appears to achieve HTML5 video against Webm/Ogg videos when chromium is started with these command line arguments: --ignore-gpu-blacklist --enable-gpu --usegl-egl Regards, Eric --------------090608050908060005050204 Content-Type: text/x-diff; name="0001-Modified-gl.gyp-to-link-libEGL-and-libGAL-at-build-t.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-Modified-gl.gyp-to-link-libEGL-and-libGAL-at-build-t.pa"; filename*1="tch" >From 03924737a9378db934da9355bb38c916b5e3fa45 Mon Sep 17 00:00:00 2001 From: Eric Nelson Date: Sun, 23 Mar 2014 07:10:36 -0700 Subject: [PATCH 1/3] Modified gl.gyp to link libEGL and libGAL at build time Signed-off-by: Eric Nelson --- ui/gl/gl.gyp | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/ui/gl/gl.gyp b/ui/gl/gl.gyp index 2ec6d76..f7ee1a4 100644 --- a/ui/gl/gl.gyp +++ b/ui/gl/gl.gyp @@ -123,6 +123,12 @@ '<(gl_binding_output_dir)/gl_bindings_autogen_osmesa.cc', '<(gl_binding_output_dir)/gl_bindings_autogen_osmesa.h', ], + 'link_settings': { + 'libraries': [ + '-lEGL', + '-lGAL', + ], + }, # hard_dependency is necessary for this target because it has actions # that generate header files included by dependent targets. The header # files must be generated before the dependents are compiled. The usual -- 1.8.3.2 --------------090608050908060005050204 Content-Type: text/x-diff; name="0002-Added-hardware-accelerated-decoding-with-the-i.MX-VP.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0002-Added-hardware-accelerated-decoding-with-the-i.MX-VP.pa"; filename*1="tch" >From 44ccb7f8a920389897e60e30e3f0ee3319f7bf7d Mon Sep 17 00:00:00 2001 From: Eric Nelson Date: Sun, 23 Mar 2014 07:17:08 -0700 Subject: [PATCH 2/3] Added hardware accelerated decoding with the i.MX VPU --- content/renderer/media/webmediaplayer_impl.cc | 5 + media/filters/vpu_framebuffers.cc | 106 +++++ media/filters/vpu_framebuffers.h | 36 ++ media/filters/vpu_video_decoder.cc | 614 ++++++++++++++++++++++++++ media/filters/vpu_video_decoder.h | 74 ++++ media/media.gyp | 28 ++ 6 files changed, 863 insertions(+) create mode 100644 media/filters/vpu_framebuffers.cc create mode 100644 media/filters/vpu_framebuffers.h create mode 100644 media/filters/vpu_video_decoder.cc create mode 100644 media/filters/vpu_video_decoder.h diff --git a/content/renderer/media/webmediaplayer_impl.cc b/content/renderer/media/webmediaplayer_impl.cc index b7094f6..7ac7318 100644 --- a/content/renderer/media/webmediaplayer_impl.cc +++ b/content/renderer/media/webmediaplayer_impl.cc @@ -57,6 +57,7 @@ #include "media/filters/opus_audio_decoder.h" #include "media/filters/video_renderer_impl.h" #include "media/filters/vpx_video_decoder.h" +#include "media/filters/vpu_video_decoder.h" #include "third_party/WebKit/public/platform/WebContentDecryptionModule.h" #include "third_party/WebKit/public/platform/WebMediaSource.h" #include "third_party/WebKit/public/platform/WebRect.h" @@ -1187,6 +1188,10 @@ void WebMediaPlayerImpl::StartPipeline() { new media::GpuVideoDecoder(gpu_factories_, media_log_)); } +#if !defined(MEDIA_DISABLE_IMXVPU) + video_decoders.push_back(new media::VpuVideoDecoder(media_loop_)); +#endif // !defined(MEDIA_DISABLE_IMXVPU) + #if !defined(MEDIA_DISABLE_LIBVPX) video_decoders.push_back(new media::VpxVideoDecoder(media_loop_)); #endif // !defined(MEDIA_DISABLE_LIBVPX) diff --git a/media/filters/vpu_framebuffers.cc b/media/filters/vpu_framebuffers.cc new file mode 100644 index 0000000..fce43fc --- /dev/null +++ b/media/filters/vpu_framebuffers.cc @@ -0,0 +1,106 @@ +#include +#include +#include "vpu_framebuffers.h" +#include "base/logging.h" + + +#define ALIGN_VAL_TO(LENGTH, ALIGN_SIZE) ( ((uintptr_t)((LENGTH) + (ALIGN_SIZE) - 1) / (ALIGN_SIZE)) * (ALIGN_SIZE) ) +#define FRAME_ALIGN 16 + + +namespace media { + + +VPUFramebuffers::VPUFramebuffers() { +} + + +void VPUFramebuffers::init(VpuDecHandle handle, VpuDecInitInfo const &init_info) { + framebuffers.resize(init_info.nMinFrameBufferCount); + + y_stride = ALIGN_VAL_TO(init_info.nPicWidth, FRAME_ALIGN); + if (init_info.nInterlace) + y_size = y_stride * ALIGN_VAL_TO(init_info.nPicHeight, (2 * FRAME_ALIGN)); + else + y_size = y_stride * ALIGN_VAL_TO(init_info.nPicHeight, FRAME_ALIGN); + + uv_stride = y_stride / 2; + u_size = v_size = mv_size = y_size / 4; + + int alignment = init_info.nAddressAlignment; + if (alignment > 1) + { + y_size = ALIGN_VAL_TO(y_size, alignment); + u_size = ALIGN_VAL_TO(u_size, alignment); + v_size = ALIGN_VAL_TO(v_size, alignment); + mv_size = ALIGN_VAL_TO(mv_size, alignment); + } + + pic_width = init_info.nPicWidth; + pic_height = init_info.nPicHeight; + + total_size = y_size + u_size + v_size + mv_size + alignment; + + LOG(INFO) << "Num VPU framebuffers: " << framebuffers.size(); + LOG(INFO) << "VPU framebuffer memory block size:" + << " total: " << total_size + << " Y " << y_size + << " U: " << u_size + << " V: " << v_size + << " Mv: " << mv_size + << " alignment: " << alignment + ; + + for (Framebuffers::iterator iter = framebuffers.begin(); iter != framebuffers.end(); ++iter) { + unsigned char *virt_ptr, *phys_ptr; + + VpuFrameBuffer &framebuffer = *iter; + + VpuMemDesc mem_desc; + memset(&mem_desc, 0, sizeof(VpuMemDesc)); + mem_desc.nSize = total_size; + VPU_DecGetMem(&mem_desc); + phys_mem_blocks.push_back(mem_desc); + + virt_ptr = (unsigned char*)(mem_desc.nVirtAddr); + phys_ptr = (unsigned char*)(mem_desc.nPhyAddr); + + if (alignment > 1) { + phys_ptr = (unsigned char*)ALIGN_VAL_TO(phys_ptr, alignment); + virt_ptr = (unsigned char*)ALIGN_VAL_TO(virt_ptr, alignment); + } + + framebuffer.nStrideY = y_stride; + framebuffer.nStrideC = uv_stride; + + /* fill phy addr*/ + framebuffer.pbufY = phys_ptr; + framebuffer.pbufCb = phys_ptr + y_size; + framebuffer.pbufCr = phys_ptr + y_size + u_size; + framebuffer.pbufMvCol = phys_ptr + y_size + u_size + v_size; + + /* fill virt addr */ + framebuffer.pbufVirtY = virt_ptr; + framebuffer.pbufVirtCb = virt_ptr + y_size; + framebuffer.pbufVirtCr = virt_ptr + y_size + u_size; + framebuffer.pbufVirtMvCol = virt_ptr + y_size + u_size + v_size; + + framebuffer.pbufY_tilebot = 0; + framebuffer.pbufCb_tilebot = 0; + framebuffer.pbufVirtY_tilebot = 0; + framebuffer.pbufVirtCb_tilebot = 0; + } + + VPU_DecRegisterFrameBuffer(handle, &(framebuffers[0]), framebuffers.size()); +} + + +VPUFramebuffers::~VPUFramebuffers() { + LOG(INFO) << "Freeing VPU framebuffer memory"; + for (PhysMemBlocks::iterator iter = phys_mem_blocks.begin(); iter != phys_mem_blocks.end(); ++iter) + VPU_DecFreeMem(&(*iter)); +} + + +} + diff --git a/media/filters/vpu_framebuffers.h b/media/filters/vpu_framebuffers.h new file mode 100644 index 0000000..bb9b021 --- /dev/null +++ b/media/filters/vpu_framebuffers.h @@ -0,0 +1,36 @@ +#ifndef MEDIA_FILTERS_VPU_FRAMEBUFFERS_H_ +#define MEDIA_FILTERS_VPU_FRAMEBUFFERS_H_ + +#include +#include +#include + + +namespace media { + + +class VPUFramebuffers +{ +public: + explicit VPUFramebuffers(); + ~VPUFramebuffers(); + + void init(VpuDecHandle handle, VpuDecInitInfo const &init_info); + +private: + typedef std::vector < VpuFrameBuffer > Framebuffers; + typedef std::list < VpuMemDesc > PhysMemBlocks; + + PhysMemBlocks phys_mem_blocks; + Framebuffers framebuffers; + int y_stride, uv_stride; + int y_size, u_size, v_size, mv_size; + int total_size; + unsigned int pic_width, pic_height; +}; + + +} + + +#endif diff --git a/media/filters/vpu_video_decoder.cc b/media/filters/vpu_video_decoder.cc new file mode 100644 index 0000000..396d4f1 --- /dev/null +++ b/media/filters/vpu_video_decoder.cc @@ -0,0 +1,614 @@ +#include +#include +#include +#include +#include +#include + +#include + +#include "media/filters/vpu_video_decoder.h" + +#include "base/bind.h" +#include "base/callback_helpers.h" +#include "base/command_line.h" +#include "base/location.h" +#include "base/logging.h" +#include "base/synchronization/lock.h" +#include "base/message_loop/message_loop_proxy.h" +#include "base/strings/string_number_conversions.h" +#include "base/sys_byteorder.h" +#include "media/base/bind_to_loop.h" +#include "media/base/decoder_buffer.h" +#include "media/base/demuxer_stream.h" +#include "media/base/media_switches.h" +#include "media/base/pipeline.h" +#include "media/base/video_decoder_config.h" +#include "media/base/video_frame.h" +#include "media/base/video_util.h" + +#include "media/filters/vpu_framebuffers.h" + +#include + + +#define ALIGN_VAL_TO(LENGTH, ALIGN_SIZE) ( ((uintptr_t)((LENGTH) + (ALIGN_SIZE) - 1) / (ALIGN_SIZE)) * (ALIGN_SIZE) ) + + + +namespace media { + + +namespace { + + +char const *VpuStrerror(VpuDecRetCode const code) { + switch (code) { + case VPU_DEC_RET_SUCCESS: return "success"; + case VPU_DEC_RET_FAILURE: return "failure"; + case VPU_DEC_RET_INVALID_PARAM: return "invalid param"; + case VPU_DEC_RET_INVALID_HANDLE: return "invalid handle"; + case VPU_DEC_RET_INVALID_FRAME_BUFFER: return "invalid frame buffer"; + case VPU_DEC_RET_INSUFFICIENT_FRAME_BUFFERS: return "insufficient frame buffers"; + case VPU_DEC_RET_INVALID_STRIDE: return "invalid stride"; + case VPU_DEC_RET_WRONG_CALL_SEQUENCE: return "wrong call sequence"; + case VPU_DEC_RET_FAILURE_TIMEOUT: return "failure timeout"; + default: + return NULL; + } +} + + +long inst_count = 0; +base::Lock vpu_load_lock; + + +void LoadDecoder() { + VpuDecRetCode ret; + + base::AutoLock alock(vpu_load_lock); + + if (inst_count == 0) { + ret = VPU_DecLoad(); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "VPU loading failed: " << VpuStrerror(ret); + return; + } + + } + + ++inst_count; + + if (inst_count == 1) { + VpuVersionInfo version; + VpuWrapperVersionInfo wrapper_version; + + ret = VPU_DecGetVersionInfo(&version); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Getting version info failed: " << VpuStrerror(ret); + return; + } + + ret = VPU_DecGetWrapperVersionInfo(&wrapper_version); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Getting wrapper version info failed: " << VpuStrerror(ret); + return; + } + + LOG(INFO) << "VPU Loaded"; + LOG(INFO) << "VPU firmware version " << version.nFwMajor << "." << version.nFwMinor << "." << version.nFwRelease << "_r" << version.nFwCode; + LOG(INFO) << "VPU library version " << version.nLibMajor << "." << version.nLibMinor << "." << version.nLibRelease; + LOG(INFO) << "VPU wrapper version " << wrapper_version.nMajor << "." << wrapper_version.nMinor << "." << wrapper_version.nRelease << " " << wrapper_version.pBinary; + } +} + + +void UnloadDecoder() { + base::AutoLock alock(vpu_load_lock); + + if (inst_count == 0) + return; + + --inst_count; + + if (inst_count == 0) + VPU_DecUnLoad(); +} + + +} + + +struct VpuVideoDecoder::Internal +{ + typedef std::vector < uint8_t > VirtMemBlock; + typedef std::list < VirtMemBlock > VirtMemBlocks; + + typedef std::list < VpuMemDesc > PhysMemBlocks; + + VpuDecHandle handle; + + VpuDecInitInfo init_info; + VpuMemInfo mem_info; + + bool vpu_opened; + + VirtMemBlocks virt_mem_blocks; + PhysMemBlocks phys_mem_blocks; + + VPUFramebuffers framebuffers; + + Internal() + : vpu_opened(false) + { + } + + ~Internal() + { + FreeMemBlocks(); + } + + bool AllocMemBlocks() + { + VpuDecRetCode ret; + + memset(&mem_info, 0, sizeof(VpuMemInfo)); + ret = VPU_DecQueryMem(&mem_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get VPU memory information: " << VpuStrerror(ret); + return false; + } + + for (int i = 0; i < mem_info.nSubBlockNum; ++i) + { + int size = mem_info.MemSubBlock[i].nAlignment + mem_info.MemSubBlock[i].nSize; + + LOG(INFO) << "Sub block " << i << " type: " << ((mem_info.MemSubBlock[i].MemType == VPU_MEM_VIRT) ? "virtual" : "phys") << " size: " << size; + + if (mem_info.MemSubBlock[i].MemType == VPU_MEM_VIRT) + { + unsigned char *virt_ptr; + virt_mem_blocks.push_back(VirtMemBlock()); + VirtMemBlock &blk = virt_mem_blocks.back(); + blk.resize(size); + virt_ptr = reinterpret_cast < unsigned char* > (&blk[0]); + + mem_info.MemSubBlock[i].pVirtAddr = (unsigned char *)ALIGN_VAL_TO(virt_ptr, mem_info.MemSubBlock[i].nAlignment); + } + else if (mem_info.MemSubBlock[i].MemType == VPU_MEM_PHY) + { + VpuMemDesc mem_desc; + memset(&mem_desc, 0, sizeof(VpuMemDesc)); + mem_desc.nSize = size; + if (VPU_DecGetMem(&mem_desc) != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Unable to allocate " << size << " bytes of physical memory"; + return false; + } + + phys_mem_blocks.push_back(mem_desc); + + mem_info.MemSubBlock[i].pVirtAddr = (unsigned char *)ALIGN_VAL_TO((unsigned char*)(mem_desc.nVirtAddr), mem_info.MemSubBlock[i].nAlignment); + mem_info.MemSubBlock[i].pPhyAddr = (unsigned char *)ALIGN_VAL_TO((unsigned char*)(mem_desc.nPhyAddr), mem_info.MemSubBlock[i].nAlignment); + } + else + LOG(WARNING) << "Sub block " << i << " type is unknown - skipping"; + } + + LOG(INFO) << "VPU memory blocks allocated"; + + return true; + } + + void FreeMemBlocks() + { + LOG(INFO) << "Freeing VPU memory blocks"; + + for (PhysMemBlocks::iterator iter = phys_mem_blocks.begin(); iter != phys_mem_blocks.end(); ++iter) + VPU_DecFreeMem(&(*iter)); + } +}; + + + +void CopyVpuFramebufferTo(VpuFrameBuffer *framebuffer, scoped_refptr* video_frame, VpuDecInitInfo &init_info, const VideoDecoderConfig& config); + + + + +VpuVideoDecoder::VpuVideoDecoder( + const scoped_refptr& message_loop) + : message_loop_(message_loop), + weak_factory_(this), + state_(kUninitialized) { + LoadDecoder(); + internal = new Internal; +} + +VpuVideoDecoder::~VpuVideoDecoder() { + DCHECK_EQ(kUninitialized, state_); + CloseDecoder(); + delete internal; + UnloadDecoder(); +} + +void VpuVideoDecoder::Initialize(const VideoDecoderConfig& config, + const PipelineStatusCB& status_cb) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(config.IsValidConfig()); + DCHECK(!config.is_encrypted()); + DCHECK(decode_cb_.is_null()); + DCHECK(reset_cb_.is_null()); + + weak_this_ = weak_factory_.GetWeakPtr(); + + if (!internal->AllocMemBlocks()) { + status_cb.Run(PIPELINE_ERROR_INITIALIZATION_FAILED); + return; + } + + if (!ConfigureDecoder(config)) { + status_cb.Run(DECODER_ERROR_NOT_SUPPORTED); + return; + } + + // Success! + LOG(INFO) << "VPU decoder initialization succeeded"; + config_ = config; + state_ = kNormal; + status_cb.Run(PIPELINE_OK); +} + +bool VpuVideoDecoder::ConfigureDecoder(const VideoDecoderConfig& config) { + bool can_handle = false; + + if ((config.format() == VideoFrame::YV12) || (config.format() == VideoFrame::YV12A) || (config.format() == VideoFrame::I420)) { + switch (config.codec()) { + case kCodecH264: + case kCodecVC1: + case kCodecMPEG2: + case kCodecMPEG4: + case kCodecVP8: + can_handle = true; + default: + break; + } + } + + if (!can_handle) { + LOG(INFO) << "Cannot handle video config: " << config.AsHumanReadableString(); + return false; + } + + CloseDecoder(); + return OpenDecoder(config); +} + +bool VpuVideoDecoder::OpenDecoder(const VideoDecoderConfig& config) { + if (internal->vpu_opened) + return true; + + VpuDecOpenParam open_param; + VpuDecRetCode ret; + + memset(&open_param, 0, sizeof(open_param)); + + switch (config.codec()) + { + case kCodecH264: + open_param.CodecFormat = VPU_V_AVC; + open_param.nReorderEnable = 1; + LOG(INFO) << "Setting h.264 as stream format"; + break; + case kCodecVC1: + open_param.CodecFormat = VPU_V_VC1_AP; + LOG(INFO) << "Setting VC1 as stream format"; + break; + case kCodecMPEG2: + open_param.CodecFormat = VPU_V_MPEG2; + LOG(INFO) << "Setting MPEG-2 as stream format"; + break; + case kCodecMPEG4: + open_param.CodecFormat = VPU_V_MPEG4; + LOG(INFO) << "Setting MPEG-4 as stream format"; + break; + case kCodecVP8: + open_param.CodecFormat = VPU_V_VP8; + LOG(INFO) << "Setting VP8 as stream format"; + break; + default: + LOG(ERROR) << "Could not set unknown stream format"; + break; + } + + open_param.nChromaInterleave = 0; + open_param.nMapType = 0; + open_param.nTiled2LinearEnable = 0; + open_param.nEnableFileMode = 0; + open_param.nPicWidth = config.coded_size().width(); + open_param.nPicHeight = config.coded_size().height(); + + ret = VPU_DecOpen(&(internal->handle), &open_param, &(internal->mem_info)); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Opening new VPU handle failed: " << VpuStrerror(ret); + return false; + } + + int config_param; + + config_param = VPU_DEC_SKIPNONE; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_SKIPMODE, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure skip mode: " << VpuStrerror(ret); + return false; + } + + config_param = 0; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_BUFDELAY, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure buffer delay: " << VpuStrerror(ret); + return false; + } + + config_param = VPU_DEC_IN_NORMAL; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_INPUTTYPE, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure input type: " << VpuStrerror(ret); + return false; + } + + internal->vpu_opened = true; + + return true; +} + +void VpuVideoDecoder::CloseDecoder() { + VpuDecRetCode ret; + + if (!(internal->vpu_opened)) + return; + + ret = VPU_DecFlushAll(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Flushing decoder failed: " << VpuStrerror(ret); + VPU_DecClose(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Closing decoder failed: " << VpuStrerror(ret); + + internal->vpu_opened = false; +} + +void VpuVideoDecoder::Decode(const scoped_refptr& buffer, + const DecodeCB& decode_cb) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(!decode_cb.is_null()); + CHECK_NE(state_, kUninitialized); + CHECK(decode_cb_.is_null()) << "Overlapping decodes are not supported."; + + decode_cb_ = BindToCurrentLoop(decode_cb); + + if (state_ == kError) { + base::ResetAndReturn(&decode_cb_).Run(kDecodeError, NULL); + return; + } + + // Return empty frames if decoding has finished. + if (state_ == kDecodeFinished) { + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + return; + } + + DecodeBuffer(buffer); +} + +void VpuVideoDecoder::Reset(const base::Closure& closure) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(reset_cb_.is_null()); + reset_cb_ = BindToCurrentLoop(closure); + + // Defer the reset if a decode is pending. + if (!decode_cb_.is_null()) + return; + + DoReset(); +} + +void VpuVideoDecoder::Stop(const base::Closure& closure) { + DCHECK(message_loop_->BelongsToCurrentThread()); + base::ScopedClosureRunner runner(BindToCurrentLoop(closure)); + + if (state_ == kUninitialized) + return; + + if (!decode_cb_.is_null()) { + base::ResetAndReturn(&decode_cb_).Run(kOk, NULL); + // Reset is pending only when decode is pending. + if (!reset_cb_.is_null()) + base::ResetAndReturn(&reset_cb_).Run(); + } + + state_ = kUninitialized; +} + +bool VpuVideoDecoder::HasAlpha() const { + return false; +} + +void VpuVideoDecoder::DecodeBuffer(const scoped_refptr& buffer) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK_NE(state_, kUninitialized); + DCHECK_NE(state_, kDecodeFinished); + DCHECK_NE(state_, kError); + DCHECK(reset_cb_.is_null()); + DCHECK(!decode_cb_.is_null()); + DCHECK(buffer); + + // Transition to kDecodeFinished on the first end of stream buffer. + if (state_ == kNormal && buffer->end_of_stream()) { + state_ = kDecodeFinished; + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + return; + } + + scoped_refptr video_frame; + if (!VpuDecode(buffer, &video_frame)) { + state_ = kError; + base::ResetAndReturn(&decode_cb_).Run(kDecodeError, NULL); + return; + } + + // If we didn't get a frame we need more data. + if (!video_frame.get()) { + base::ResetAndReturn(&decode_cb_).Run(kNotEnoughData, NULL); + return; + } + + base::ResetAndReturn(&decode_cb_).Run(kOk, video_frame); +} + +bool VpuVideoDecoder::VpuDecode(const scoped_refptr& buffer, + scoped_refptr* video_frame) { + DCHECK(video_frame); + DCHECK(!buffer->end_of_stream()); + + int buffer_ret_code; + VpuBufferNode in_data; + VpuDecRetCode ret; + + memset(&in_data, 0, sizeof(in_data)); + in_data.pVirAddr = (unsigned char *)(buffer->data()); + in_data.nSize = buffer->data_size(); + + ret = VPU_DecDecodeBuf(internal->handle, &in_data, &buffer_ret_code); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Failed to decode frame: " << VpuStrerror(ret); + return false; + } + + VLOG(1) << "VPU_DecDecodeBuf buffer ret code is 0x" << std::hex << buffer_ret_code << std::dec; + + if (buffer_ret_code & VPU_DEC_INIT_OK) { + ret = VPU_DecGetInitialInfo(internal->handle, &(internal->init_info)); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get init info: " << VpuStrerror(ret); + return false; + } + internal->framebuffers.init(internal->handle, internal->init_info); + } + + if (buffer_ret_code & VPU_DEC_FLUSH) { + ret = VPU_DecFlushAll(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Flushing VPU failed: " << VpuStrerror(ret); + return false; + } + + *video_frame = NULL; + + return true; + } + + if (buffer_ret_code & VPU_DEC_NO_ENOUGH_INBUF) { + LOG(INFO) << "Need more input"; + return true; + } + + if (buffer_ret_code & VPU_DEC_ONE_FRM_CONSUMED) { + VpuDecFrameLengthInfo dec_framelen_info; + + ret = VPU_DecGetConsumedFrameInfo(internal->handle, &dec_framelen_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get information about consumed frame: " << VpuStrerror(ret); + } else { + LOG(INFO) << "One frame got consumed:" + << " framebuffer addr: 0x" << std::hex << uintptr_t(dec_framelen_info.pFrame) << std::dec + << " stuff length: " << dec_framelen_info.nStuffLength + << " frame length: " << dec_framelen_info.nFrameLength + ; + } + } + + if (buffer_ret_code & VPU_DEC_OUTPUT_DIS) { + VpuDecOutFrameInfo out_frame_info; + int64 timestamp = buffer->timestamp().InMicroseconds(); + + ret = VPU_DecGetOutputFrame(internal->handle, &out_frame_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get decoded output frame: " << VpuStrerror(ret); + return false; + } + + LOG(INFO) << "Output frame:" + << " pic width: " << internal->init_info.nPicWidth + << " pic height: " << internal->init_info.nPicHeight + << " pic type: " << out_frame_info.ePicType + << " Y stride: " << out_frame_info.pDisplayFrameBuf->nStrideY + << " CbCr stride: " << out_frame_info.pDisplayFrameBuf->nStrideC + ; + + CopyVpuFramebufferTo(out_frame_info.pDisplayFrameBuf, video_frame, internal->init_info, config_); + (*video_frame)->SetTimestamp(base::TimeDelta::FromMicroseconds(timestamp)); + + ret = VPU_DecOutFrameDisplayed(internal->handle, out_frame_info.pDisplayFrameBuf); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Clearing display framebuffer failed: " << VpuStrerror(ret); + } + else + *video_frame = NULL; + + return true; +} + +void VpuVideoDecoder::DoReset() { + DCHECK(decode_cb_.is_null()); + + state_ = kNormal; + reset_cb_.Run(); + reset_cb_.Reset(); +} + +void CopyVpuFramebufferTo(VpuFrameBuffer *framebuffer, scoped_refptr* video_frame, VpuDecInitInfo &init_info, const VideoDecoderConfig& config) { + CHECK(framebuffer); + + gfx::Size size(init_info.nPicWidth, init_info.nPicHeight); + + *video_frame = VideoFrame::CreateFrame( + config.format(), + size, + gfx::Rect(size), + config.natural_size(), + kNoTimestamp()); + + CopyYPlane(framebuffer->pbufVirtY, + framebuffer->nStrideY, + init_info.nPicHeight, + video_frame->get()); + if (config.format() == VideoFrame::I420) { + CopyUPlane(framebuffer->pbufVirtCb, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + CopyVPlane(framebuffer->pbufVirtCr, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + } else { /* YV12 and YV12A */ + /* U and V swapped */ + CopyUPlane(framebuffer->pbufVirtCb, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + CopyVPlane(framebuffer->pbufVirtCr, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + + if (config.format() == VideoFrame::YV12A) { + MakeOpaqueAPlane(framebuffer->nStrideY, + init_info.nPicHeight, + video_frame->get()); + } + } +} + + +} // namespace media diff --git a/media/filters/vpu_video_decoder.h b/media/filters/vpu_video_decoder.h new file mode 100644 index 0000000..d7016aa --- /dev/null +++ b/media/filters/vpu_video_decoder.h @@ -0,0 +1,74 @@ +#ifndef MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ +#define MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ + +#include "base/callback.h" +#include "base/memory/weak_ptr.h" +#include "media/base/demuxer_stream.h" +#include "media/base/video_decoder.h" +#include "media/base/video_decoder_config.h" +#include "media/base/video_frame.h" + +namespace base { +class MessageLoopProxy; +} + +namespace media { + +class MEDIA_EXPORT VpuVideoDecoder : public VideoDecoder { + public: + explicit VpuVideoDecoder( + const scoped_refptr& message_loop); + virtual ~VpuVideoDecoder(); + + // VideoDecoder implementation. + virtual void Initialize(const VideoDecoderConfig& config, + const PipelineStatusCB& status_cb) OVERRIDE; + virtual void Decode(const scoped_refptr& buffer, + const DecodeCB& decode_cb) OVERRIDE; + virtual void Reset(const base::Closure& closure) OVERRIDE; + virtual void Stop(const base::Closure& closure) OVERRIDE; + virtual bool HasAlpha() const OVERRIDE; + + private: + enum DecoderState { + kUninitialized, + kNormal, + kFlushCodec, + kDecodeFinished, + kError + }; + + // Handles (re-)initializing the decoder with a (new) config. + // Returns true when initialization was successful. + bool ConfigureDecoder(const VideoDecoderConfig& config); + + bool OpenDecoder(const VideoDecoderConfig& config); + void CloseDecoder(); + + void DecodeBuffer(const scoped_refptr& buffer); + bool VpuDecode(const scoped_refptr& buffer, + scoped_refptr* video_frame); + + // Reset decoder and call |reset_cb_|. + void DoReset(); + + scoped_refptr message_loop_; + base::WeakPtrFactory weak_factory_; + base::WeakPtr weak_this_; + + DecoderState state_; + + DecodeCB decode_cb_; + base::Closure reset_cb_; + + VideoDecoderConfig config_; + + struct Internal; + Internal *internal; + + DISALLOW_COPY_AND_ASSIGN(VpuVideoDecoder); +}; + +} // namespace media + +#endif // MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ diff --git a/media/media.gyp b/media/media.gyp index 4782741..ec53626 100644 --- a/media/media.gyp +++ b/media/media.gyp @@ -11,6 +11,7 @@ # (DT_NEEDED) instead of using dlopen. This helps with automated # detection of ABI mismatches and prevents silent errors. 'linux_link_pulseaudio%': 0, + 'media_use_imxvpu%': 1, 'conditions': [ ['OS=="android"', { # Android doesn't use ffmpeg. @@ -404,6 +405,10 @@ 'filters/video_frame_painter.h', 'filters/video_renderer_impl.cc', 'filters/video_renderer_impl.h', + 'filters/vpu_framebuffers.cc', + 'filters/vpu_framebuffers.h', + 'filters/vpu_video_decoder.cc', + 'filters/vpu_video_decoder.h', 'filters/vpx_video_decoder.cc', 'filters/vpx_video_decoder.h', 'filters/webvtt_util.h', @@ -658,6 +663,29 @@ 'DISABLE_USER_INPUT_MONITOR', ], }], + ['media_use_imxvpu==1', { + 'cflags': [ + 'From f059d6501b17ab8aed618f53f7e4255cf8d8326b Mon Sep 17 00:00:00 2001 From: Mahyar Yaghmaee Date: Fri, 28 Mar 2014 14:06:40 -0700 Subject: [PATCH 3/3] Update vpu_video_decoder.cc to match chromium-35.0.1883 Signed-off-by: Mahyar Yaghmaee --- media/filters/vpu_video_decoder.cc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/media/filters/vpu_video_decoder.cc b/media/filters/vpu_video_decoder.cc index 396d4f1..9024484 100644 --- a/media/filters/vpu_video_decoder.cc +++ b/media/filters/vpu_video_decoder.cc @@ -18,7 +18,7 @@ #include "base/message_loop/message_loop_proxy.h" #include "base/strings/string_number_conversions.h" #include "base/sys_byteorder.h" -#include "media/base/bind_to_loop.h" +#include "media/base/bind_to_current_loop.h" #include "media/base/decoder_buffer.h" #include "media/base/demuxer_stream.h" #include "media/base/media_switches.h" @@ -393,7 +393,7 @@ void VpuVideoDecoder::Decode(const scoped_refptr& buffer, // Return empty frames if decoding has finished. if (state_ == kDecodeFinished) { - base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEOSFrame()); return; } @@ -445,7 +445,7 @@ void VpuVideoDecoder::DecodeBuffer(const scoped_refptr& buffer) { // Transition to kDecodeFinished on the first end of stream buffer. if (state_ == kNormal && buffer->end_of_stream()) { state_ = kDecodeFinished; - base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEOSFrame()); return; } -- 1.8.3.2 --------------090608050908060005050204--