[PATCH 0/8] Qualcomm video decoder/encoder driver

[PATCH 0/8] Qualcomm video decoder/encoder driver

[Stanimir Varbanov]

This patchset introduces a basic support for Qualcomm video
acceleration hardware used for video stream decoding/encoding.
The video IP can found on various qcom SoCs like apq8084, msm8916
and msm8996, hence it is widly distributed but the driver is 
missing in the mainline.

The v4l2 driver is something like a wrapper over Host Firmware
Interface. The HFI itself is a set of command and message packets
send/received through shared memory, and its purpose is to
comunicate with the firmware which is run on remote processor.
The Venus is the name of the video hardware IP that doing the
video acceleration.

>From the software point of view the HFI interface is implemented
in the files with prefix hfi_xxx. It acts as a translation layer
between HFI and v4l2 layer. There is one special file in the 
driver called hfi_venus which doing most of the driver
orchestration work. Something more it setups Venus core, run it
and handle commands and messages from low-level point of view with
the help of provided functions by HFI interface.

I think that the driver is in good shape for mainline kernel, and
I hope the review comments will help to improve it, so please
do review and make comments.

The driver depends on:
 - venus remoteproc driver posted at [1].
 - out-of-tree qcom IOMMU driver and IOMMU probe deferral support
 at [2].

The driver has been tested on db410c (with apq8016 SoC) with simple 
v4l2 test applications and with gstreamer v4l2 videodec plugin,
and v4l2 h264 out-of-tree gstreamer videoenc plugin.

The output of v4l2-compliance test looks like:

root@dragonboard-410c:/home/linaro# ./v4l2-compliance -d /dev/video0 
v4l2-compliance SHA   : ee1ab491019f80052834d14c76bdd1c1b46f2158

Driver Info:
        Driver name   : vidc
        Card type     : video decoder
        Bus info      : platform:vidc
        Driver version: 4.8.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video0 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
        test VIDIOC_QUERYCTRL: OK
        test VIDIOC_G/S_CTRL: OK
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 7 Private Controls: 0

Format ioctls:
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
        test VIDIOC_G/S_PARM: OK
        test VIDIOC_G_FBUF: OK (Not Supported)
        test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
        test Cropping: OK (Not Supported)
        test Composing: OK (Not Supported)
        test Scaling: OK

Codec ioctls:
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test VIDIOC_EXPBUF: OK

Test input 0:

Total: 43, Succeeded: 43, Failed: 0, Warnings: 0

root@dragonboard-410c:/home/linaro# ./v4l2-compliance -d /dev/video1
v4l2-compliance SHA   : ee1ab491019f80052834d14c76bdd1c1b46f2158

Driver Info:
        Driver name   : vidc
        Card type     : video encoder
        Bus info      : platform:vidc
        Driver version: 4.8.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second video open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
        test VIDIOC_QUERYCTRL: OK
        test VIDIOC_G/S_CTRL: OK
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 32 Private Controls: 0

Format ioctls:
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
        test VIDIOC_G/S_PARM: OK
        test VIDIOC_G_FBUF: OK (Not Supported)
        test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
        test Cropping: OK
        test Composing: OK (Not Supported)
        test Scaling: OK

Codec ioctls:
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test VIDIOC_EXPBUF: OK

Test input 0:

Total: 43, Succeeded: 43, Failed: 0, Warnings: 0

regards,
Stan

[1] https://lkml.org/lkml/2016/8/19/570
[2] https://www.spinics.net/lists/arm-kernel/msg522505.html

Stanimir Varbanov (8):
  doc: DT: vidc: binding document for Qualcomm video driver
  media: vidc: adding core part and helper functions
  media: vidc: decoder: add video decoder files
  media: vidc: encoder: add video encoder files
  media: vidc: add Host Firmware Interface (HFI)
  media: vidc: add Venus HFI files
  media: vidc: add Makefiles and Kconfig files
  media: vidc: enable building of the video codec driver

 .../devicetree/bindings/media/qcom,vidc.txt        |   61 +
 drivers/media/platform/Kconfig                     |    1 +
 drivers/media/platform/Makefile                    |    1 +
 drivers/media/platform/qcom/Kconfig                |    8 +
 drivers/media/platform/qcom/Makefile               |    6 +
 drivers/media/platform/qcom/vidc/Makefile          |   19 +
 drivers/media/platform/qcom/vidc/core.c            |  548 +++++++
 drivers/media/platform/qcom/vidc/core.h            |  196 +++
 drivers/media/platform/qcom/vidc/helpers.c         |  394 +++++
 drivers/media/platform/qcom/vidc/helpers.h         |   43 +
 drivers/media/platform/qcom/vidc/hfi.c             |  622 ++++++++
 drivers/media/platform/qcom/vidc/hfi.h             |  272 ++++
 drivers/media/platform/qcom/vidc/hfi_cmds.c        | 1261 ++++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_cmds.h        |  338 +++++
 drivers/media/platform/qcom/vidc/hfi_helper.h      | 1143 +++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_msgs.c        | 1072 ++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_msgs.h        |  298 ++++
 drivers/media/platform/qcom/vidc/hfi_venus.c       | 1539 ++++++++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_venus.h       |   25 +
 drivers/media/platform/qcom/vidc/hfi_venus_io.h    |   98 ++
 drivers/media/platform/qcom/vidc/int_bufs.c        |  325 +++++
 drivers/media/platform/qcom/vidc/int_bufs.h        |   23 +
 drivers/media/platform/qcom/vidc/load.c            |  104 ++
 drivers/media/platform/qcom/vidc/load.h            |   22 +
 drivers/media/platform/qcom/vidc/mem.c             |   64 +
 drivers/media/platform/qcom/vidc/mem.h             |   32 +
 drivers/media/platform/qcom/vidc/resources.c       |   46 +
 drivers/media/platform/qcom/vidc/resources.h       |   46 +
 drivers/media/platform/qcom/vidc/vdec.c            | 1100 ++++++++++++++
 drivers/media/platform/qcom/vidc/vdec.h            |   27 +
 drivers/media/platform/qcom/vidc/vdec_ctrls.c      |  200 +++
 drivers/media/platform/qcom/vidc/vdec_ctrls.h      |   21 +
 drivers/media/platform/qcom/vidc/venc.c            | 1261 ++++++++++++++++
 drivers/media/platform/qcom/vidc/venc.h            |   27 +
 drivers/media/platform/qcom/vidc/venc_ctrls.c      |  396 +++++
 drivers/media/platform/qcom/vidc/venc_ctrls.h      |   23 +
 36 files changed, 11662 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/qcom,vidc.txt
 create mode 100644 drivers/media/platform/qcom/Kconfig
 create mode 100644 drivers/media/platform/qcom/Makefile
 create mode 100644 drivers/media/platform/qcom/vidc/Makefile
 create mode 100644 drivers/media/platform/qcom/vidc/core.c
 create mode 100644 drivers/media/platform/qcom/vidc/core.h
 create mode 100644 drivers/media/platform/qcom/vidc/helpers.c
 create mode 100644 drivers/media/platform/qcom/vidc/helpers.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_cmds.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_cmds.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_helper.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_msgs.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_msgs.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus_io.h
 create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.c
 create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.h
 create mode 100644 drivers/media/platform/qcom/vidc/load.c
 create mode 100644 drivers/media/platform/qcom/vidc/load.h
 create mode 100644 drivers/media/platform/qcom/vidc/mem.c
 create mode 100644 drivers/media/platform/qcom/vidc/mem.h
 create mode 100644 drivers/media/platform/qcom/vidc/resources.c
 create mode 100644 drivers/media/platform/qcom/vidc/resources.h
 create mode 100644 drivers/media/platform/qcom/vidc/vdec.c
 create mode 100644 drivers/media/platform/qcom/vidc/vdec.h
 create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.c
 create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.h
 create mode 100644 drivers/media/platform/qcom/vidc/venc.c
 create mode 100644 drivers/media/platform/qcom/vidc/venc.h
 create mode 100644 drivers/media/platform/qcom/vidc/venc_ctrls.c
 create mode 100644 drivers/media/platform/qcom/vidc/venc_ctrls.h

-- 
2.7.4

[PATCH 1/8] doc: DT: vidc: binding document for Qualcomm video driver

[Stanimir Varbanov]

Adds binding document for vidc video encoder/decoder driver

Cc: Rob Herring <robh+dt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>
Cc: devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Signed-off-by: Stanimir Varbanov <stanimir.varbanov-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
---
 .../devicetree/bindings/media/qcom,vidc.txt        | 61 ++++++++++++++++++++++
 1 file changed, 61 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/qcom,vidc.txt

diff --git a/Documentation/devicetree/bindings/media/qcom,vidc.txt b/Documentation/devicetree/bindings/media/qcom,vidc.txt
new file mode 100644
index 000000000000..0d50a7b2e3ed
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/qcom,vidc.txt
@@ -0,0 +1,61 @@
+* Qualcomm video encoder/decoder accelerator
+
+- compatible:
+	Usage: required
+	Value type: <stringlist>
+	Definition: Value should contain
+			- "qcom,vidc-msm8916"
+			- "qcom,vidc-msm8996"
+- reg:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: Register ranges as listed in the reg-names property
+
+- interrupts:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition:
+
+- power-domains:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: A phandle and power domain specifier pairs to the
+		    power domain which is responsible for collapsing
+		    and restoring power to the peripheral
+
+- clocks:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: List of phandle and clock specifier pairs as listed
+		    in clock-names property
+- clock-names:
+	Usage: required
+	Value type: <stringlist>
+	Definition: Should contain the following entries
+			- "core"  Core video accelerator clock
+			- "iface" Video accelerator AHB clock
+			- "bus"	  Video accelerator AXI clock
+- rproc:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: A phandle to remote processor responsible for
+		    firmware loading
+
+- iommus:
+	Usage: required
+	Value type: <prop-encoded-array>
+	Definition: A list of phandle and IOMMU specifier pairs
+
+* An Example
+	qcom,vidc@1d00000 {
+		compatible = "qcom,vidc-msm8916";
+		reg = <0x01d00000 0xff000>;
+		clocks = <&gcc GCC_VENUS0_VCODEC0_CLK>,
+			 <&gcc GCC_VENUS0_AHB_CLK>,
+			 <&gcc GCC_VENUS0_AXI_CLK>;
+		clock-names = "core", "iface", "bus";
+		interrupts = <GIC_SPI 44 0>;
+		power-domains = <&gcc VENUS_GDSC>;
+		rproc = <&vidc_rproc>;
+		iommus = <&apps_iommu 5>;
+	};
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/8] media: vidc: adding core part and helper functions

[Stanimir Varbanov]

This adds core part of the vidc driver common helper functions
used by encoder and decoder specific files.

 - core.c has implemented the platform dirver methods, file
operations and v4l2 registration.

 - helpers.c has implemented common helper functions for
buffer management, vb2_ops and functions for format propagation.

 - int_bufs.c implements functions for allocating and freeing
buffers for internal usage. The buffer parameters describing
internal buffers depends on current format, resolution and
codec.

 - load.c consists functions for calculation of current load
of the hardware. Depending on the count of instances and
resolutions it selects the best clock rate for the video
core.

 - mem.c has two functions for memory allocation, currently
those functions are used for internal buffers and to allocate
the shared memory for communication with firmware via HFI
(Host Firmware Interface) interface commands.

 - resources.c exports a structure describing the details
specific to platform and SoC.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/vidc/core.c      | 548 +++++++++++++++++++++++++++
 drivers/media/platform/qcom/vidc/core.h      | 196 ++++++++++
 drivers/media/platform/qcom/vidc/helpers.c   | 394 +++++++++++++++++++
 drivers/media/platform/qcom/vidc/helpers.h   |  43 +++
 drivers/media/platform/qcom/vidc/int_bufs.c  | 325 ++++++++++++++++
 drivers/media/platform/qcom/vidc/int_bufs.h  |  23 ++
 drivers/media/platform/qcom/vidc/load.c      | 104 +++++
 drivers/media/platform/qcom/vidc/load.h      |  22 ++
 drivers/media/platform/qcom/vidc/mem.c       |  64 ++++
 drivers/media/platform/qcom/vidc/mem.h       |  32 ++
 drivers/media/platform/qcom/vidc/resources.c |  46 +++
 drivers/media/platform/qcom/vidc/resources.h |  46 +++
 12 files changed, 1843 insertions(+)
 create mode 100644 drivers/media/platform/qcom/vidc/core.c
 create mode 100644 drivers/media/platform/qcom/vidc/core.h
 create mode 100644 drivers/media/platform/qcom/vidc/helpers.c
 create mode 100644 drivers/media/platform/qcom/vidc/helpers.h
 create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.c
 create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.h
 create mode 100644 drivers/media/platform/qcom/vidc/load.c
 create mode 100644 drivers/media/platform/qcom/vidc/load.h
 create mode 100644 drivers/media/platform/qcom/vidc/mem.c
 create mode 100644 drivers/media/platform/qcom/vidc/mem.h
 create mode 100644 drivers/media/platform/qcom/vidc/resources.c
 create mode 100644 drivers/media/platform/qcom/vidc/resources.h

diff --git a/drivers/media/platform/qcom/vidc/core.c b/drivers/media/platform/qcom/vidc/core.c
new file mode 100644
index 000000000000..e005be178fc0
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/core.c
@@ -0,0 +1,548 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/clk.h>
+#include <linux/init.h>
+#include <linux/ioctl.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/remoteproc.h>
+#include <linux/pm_runtime.h>
+#include <media/videobuf2-v4l2.h>
+#include <media/v4l2-ioctl.h>
+
+#include "core.h"
+#include "resources.h"
+#include "vdec.h"
+#include "venc.h"
+
+static void vidc_add_inst(struct vidc_core *core, struct vidc_inst *inst)
+{
+	mutex_lock(&core->lock);
+	list_add_tail(&inst->list, &core->instances);
+	mutex_unlock(&core->lock);
+}
+
+static void vidc_del_inst(struct vidc_core *core, struct vidc_inst *inst)
+{
+	struct vidc_inst *pos, *n;
+
+	mutex_lock(&core->lock);
+	list_for_each_entry_safe(pos, n, &core->instances, list) {
+		if (pos == inst)
+			list_del(&inst->list);
+	}
+	mutex_unlock(&core->lock);
+}
+
+static int vidc_rproc_boot(struct vidc_core *core)
+{
+	int ret;
+
+	if (core->rproc_booted)
+		return 0;
+
+	ret = rproc_boot(core->rproc);
+	if (ret)
+		return ret;
+
+	core->rproc_booted = true;
+
+	return 0;
+}
+
+static void vidc_rproc_shutdown(struct vidc_core *core)
+{
+	if (!core->rproc_booted)
+		return;
+
+	rproc_shutdown(core->rproc);
+	core->rproc_booted = false;
+}
+
+struct vidc_sys_error {
+	struct vidc_core *core;
+	struct delayed_work work;
+};
+
+static void vidc_sys_error_handler(struct work_struct *work)
+{
+	struct vidc_sys_error *handler =
+		container_of(work, struct vidc_sys_error, work.work);
+	struct vidc_core *core = handler->core;
+	struct hfi_core *hfi = &core->hfi;
+	struct device *dev = core->dev;
+	int ret;
+
+	mutex_lock(&hfi->lock);
+	if (hfi->state != CORE_INVALID)
+		goto exit;
+
+	mutex_unlock(&hfi->lock);
+
+	ret = vidc_hfi_core_deinit(hfi);
+	if (ret)
+		dev_err(dev, "core: deinit failed (%d)\n", ret);
+
+	mutex_lock(&hfi->lock);
+
+	rproc_report_crash(core->rproc, RPROC_FATAL_ERROR);
+
+	vidc_rproc_shutdown(core);
+
+	ret = vidc_rproc_boot(core);
+	if (ret)
+		goto exit;
+
+	hfi->state = CORE_INIT;
+
+exit:
+	mutex_unlock(&hfi->lock);
+	kfree(handler);
+}
+
+static int vidc_event_notify(struct hfi_core *hfi, u32 event)
+{
+	struct vidc_sys_error *handler;
+	struct hfi_inst *inst;
+
+	switch (event) {
+	case EVT_SYS_WATCHDOG_TIMEOUT:
+	case EVT_SYS_ERROR:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	mutex_lock(&hfi->lock);
+
+	hfi->state = CORE_INVALID;
+
+	list_for_each_entry(inst, &hfi->instances, list) {
+		mutex_lock(&inst->lock);
+		inst->state = INST_INVALID;
+		mutex_unlock(&inst->lock);
+	}
+
+	mutex_unlock(&hfi->lock);
+
+	handler = kzalloc(sizeof(*handler), GFP_KERNEL);
+	if (!handler)
+		return -ENOMEM;
+
+	handler->core = container_of(hfi, struct vidc_core, hfi);
+	INIT_DELAYED_WORK(&handler->work, vidc_sys_error_handler);
+
+	/*
+	 * Sleep for 5 sec to ensure venus has completed any
+	 * pending cache operations. Without this sleep, we see
+	 * device reset when firmware is unloaded after a sys
+	 * error.
+	 */
+	schedule_delayed_work(&handler->work, msecs_to_jiffies(5000));
+
+	return 0;
+}
+
+static const struct hfi_core_ops vidc_core_ops = {
+	.event_notify = vidc_event_notify,
+};
+
+static int vidc_open(struct file *file)
+{
+	struct video_device *vdev = video_devdata(file);
+	struct vidc_core *core = video_drvdata(file);
+	struct vidc_inst *inst;
+	int ret = 0;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	mutex_init(&inst->lock);
+
+	INIT_VIDC_LIST(&inst->scratchbufs);
+	INIT_VIDC_LIST(&inst->persistbufs);
+	INIT_VIDC_LIST(&inst->registeredbufs);
+
+	INIT_LIST_HEAD(&inst->bufqueue);
+	mutex_init(&inst->bufqueue_lock);
+
+	if (vdev == &core->vdev_dec)
+		inst->session_type = VIDC_SESSION_TYPE_DEC;
+	else
+		inst->session_type = VIDC_SESSION_TYPE_ENC;
+
+	inst->core = core;
+
+	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
+		ret = vdec_open(inst);
+	else
+		ret = venc_open(inst);
+
+	if (ret)
+		goto err_free_inst;
+
+	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
+		v4l2_fh_init(&inst->fh, &core->vdev_dec);
+	else
+		v4l2_fh_init(&inst->fh, &core->vdev_enc);
+
+	inst->fh.ctrl_handler = &inst->ctrl_handler;
+
+	v4l2_fh_add(&inst->fh);
+
+	file->private_data = &inst->fh;
+
+	vidc_add_inst(core, inst);
+
+	return 0;
+
+err_free_inst:
+	kfree(inst);
+	return ret;
+}
+
+static int vidc_close(struct file *file)
+{
+	struct vidc_inst *inst = to_inst(file);
+	struct vidc_core *core = inst->core;
+
+	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
+		vdec_close(inst);
+	else
+		venc_close(inst);
+
+	vidc_del_inst(core, inst);
+
+	mutex_destroy(&inst->bufqueue_lock);
+	mutex_destroy(&inst->scratchbufs.lock);
+	mutex_destroy(&inst->persistbufs.lock);
+	mutex_destroy(&inst->registeredbufs.lock);
+
+	v4l2_fh_del(&inst->fh);
+	v4l2_fh_exit(&inst->fh);
+
+	kfree(inst);
+	return 0;
+}
+
+static unsigned int vidc_poll(struct file *file, struct poll_table_struct *pt)
+{
+	struct vidc_inst *inst = to_inst(file);
+	struct vb2_queue *outq = &inst->bufq_out;
+	struct vb2_queue *capq = &inst->bufq_cap;
+	unsigned int ret;
+
+	ret = vb2_poll(outq, file, pt);
+	ret |= vb2_poll(capq, file, pt);
+
+	return ret;
+}
+
+static int vidc_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct vidc_inst *inst = to_inst(file);
+	unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
+	int ret;
+
+	if (offset < DST_QUEUE_OFF_BASE) {
+		ret = vb2_mmap(&inst->bufq_out, vma);
+	} else {
+		vma->vm_pgoff -= DST_QUEUE_OFF_BASE >> PAGE_SHIFT;
+		ret = vb2_mmap(&inst->bufq_cap, vma);
+	}
+
+	return ret;
+}
+
+const struct v4l2_file_operations vidc_fops = {
+	.owner = THIS_MODULE,
+	.open = vidc_open,
+	.release = vidc_close,
+	.unlocked_ioctl = video_ioctl2,
+	.poll = vidc_poll,
+	.mmap = vidc_mmap,
+#ifdef CONFIG_COMPAT
+	.compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static irqreturn_t vidc_isr_thread(int irq, void *dev_id)
+{
+	return vidc_hfi_isr_thread(irq, dev_id);
+}
+
+static irqreturn_t vidc_isr(int irq, void *dev)
+{
+	return vidc_hfi_isr(irq, dev);
+}
+
+static int vidc_clks_get(struct vidc_core *core, unsigned int clks_num,
+			 const char * const *clks_id)
+{
+	struct device *dev = core->dev;
+	unsigned int i;
+
+	for (i = 0; i < clks_num; i++) {
+		core->clks[i] = devm_clk_get(dev, clks_id[i]);
+		if (IS_ERR(core->clks[i]))
+			return PTR_ERR(core->clks[i]);
+	}
+
+	return 0;
+}
+
+static int
+vidc_clks_enable(struct vidc_core *core, const struct vidc_resources *res)
+{
+	unsigned int i;
+	int ret;
+
+	for (i = 0; i < res->clks_num; i++) {
+		ret = clk_prepare_enable(core->clks[i]);
+		if (ret)
+			goto err;
+	}
+
+	return 0;
+err:
+	while (--i)
+		clk_disable_unprepare(core->clks[i]);
+
+	return ret;
+}
+
+static void
+vidc_clks_disable(struct vidc_core *core, const struct vidc_resources *res)
+{
+	unsigned int i;
+
+	for (i = 0; i < res->clks_num; i++)
+		clk_disable_unprepare(core->clks[i]);
+}
+
+static const struct of_device_id vidc_dt_match[] = {
+	{ .compatible = "qcom,vidc-msm8916", .data = &msm8916_res, },
+	{ }
+};
+
+MODULE_DEVICE_TABLE(of, vidc_dt_match);
+
+static int vidc_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct vidc_core *core;
+	struct device_node *rproc;
+	struct resource *r;
+	int ret;
+
+	core = devm_kzalloc(dev, sizeof(*core), GFP_KERNEL);
+	if (!core)
+		return -ENOMEM;
+
+	core->dev = dev;
+	platform_set_drvdata(pdev, core);
+
+	rproc = of_parse_phandle(dev->of_node, "rproc", 0);
+	if (IS_ERR(rproc))
+		return PTR_ERR(rproc);
+
+	core->rproc = rproc_get_by_phandle(rproc->phandle);
+	if (IS_ERR(core->rproc))
+		return PTR_ERR(core->rproc);
+	else if (!core->rproc)
+		return -EPROBE_DEFER;
+
+	r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	core->base = devm_ioremap_resource(dev, r);
+	if (IS_ERR(core->base))
+		return PTR_ERR(core->base);
+
+	core->irq = platform_get_irq(pdev, 0);
+	if (core->irq < 0)
+		return core->irq;
+
+	core->res = of_device_get_match_data(dev);
+	if (!core->res)
+		return -ENODEV;
+
+	ret = vidc_clks_get(core, core->res->clks_num, core->res->clks);
+	if (ret)
+		return ret;
+
+	ret = dma_set_mask_and_coherent(dev, core->res->dma_mask);
+	if (ret)
+		return ret;
+
+	INIT_LIST_HEAD(&core->instances);
+	mutex_init(&core->lock);
+
+	ret = devm_request_threaded_irq(dev, core->irq, vidc_isr,
+					vidc_isr_thread,
+					IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
+					"vidc", &core->hfi);
+	if (ret)
+		return ret;
+
+	core->hfi.core_ops = &vidc_core_ops;
+	core->hfi.dev = dev;
+
+	ret = vidc_hfi_create(&core->hfi, core->res, core->base);
+	if (ret)
+		return ret;
+
+	ret = vidc_clks_enable(core, core->res);
+	if (ret)
+		goto err_hfi_destroy;
+
+	ret = vidc_rproc_boot(core);
+	if (ret) {
+		vidc_clks_disable(core, core->res);
+		goto err_hfi_destroy;
+	}
+
+	pm_runtime_enable(dev);
+
+	ret = pm_runtime_get_sync(dev);
+	if (ret < 0)
+		goto err_runtime_disable;
+
+	ret = vidc_hfi_core_init(&core->hfi);
+	if (ret)
+		goto err_rproc_shutdown;
+
+	ret = pm_runtime_put_sync(dev);
+	if (ret)
+		goto err_core_deinit;
+
+	vidc_clks_disable(core, core->res);
+
+	ret = v4l2_device_register(dev, &core->v4l2_dev);
+	if (ret)
+		goto err_core_deinit;
+
+	ret = vdec_init(core, &core->vdev_dec);
+	if (ret)
+		goto err_dev_unregister;
+
+	ret = venc_init(core, &core->vdev_enc);
+	if (ret)
+		goto err_vdec_deinit;
+
+	return 0;
+
+err_vdec_deinit:
+	vdec_deinit(core, &core->vdev_dec);
+err_dev_unregister:
+	v4l2_device_unregister(&core->v4l2_dev);
+err_core_deinit:
+	vidc_hfi_core_deinit(&core->hfi);
+err_rproc_shutdown:
+	vidc_rproc_shutdown(core);
+err_runtime_disable:
+	pm_runtime_set_suspended(dev);
+	pm_runtime_disable(dev);
+err_hfi_destroy:
+	vidc_hfi_destroy(&core->hfi);
+	return ret;
+}
+
+static int vidc_remove(struct platform_device *pdev)
+{
+	struct vidc_core *core = platform_get_drvdata(pdev);
+	int ret;
+
+	ret = pm_runtime_get_sync(&pdev->dev);
+	if (ret < 0)
+		return ret;
+
+	ret = vidc_hfi_core_deinit(&core->hfi);
+	if (ret) {
+		pm_runtime_put_sync(&pdev->dev);
+		return ret;
+	}
+
+	vidc_rproc_shutdown(core);
+
+	ret = pm_runtime_put_sync(&pdev->dev);
+
+	vidc_hfi_destroy(&core->hfi);
+	vdec_deinit(core, &core->vdev_dec);
+	venc_deinit(core, &core->vdev_enc);
+	v4l2_device_unregister(&core->v4l2_dev);
+
+	pm_runtime_disable(core->dev);
+
+	return ret < 0 ? ret : 0;
+}
+
+static int vidc_runtime_suspend(struct device *dev)
+{
+	struct vidc_core *core = dev_get_drvdata(dev);
+	int ret;
+
+	ret = vidc_hfi_core_suspend(&core->hfi);
+
+	vidc_clks_disable(core, core->res);
+
+	return ret;
+}
+
+static int vidc_runtime_resume(struct device *dev)
+{
+	struct vidc_core *core = dev_get_drvdata(dev);
+	int ret;
+
+	ret = vidc_clks_enable(core, core->res);
+	if (ret)
+		return ret;
+
+	return vidc_hfi_core_resume(&core->hfi);
+}
+
+static int vidc_pm_suspend(struct device *dev)
+{
+	return vidc_runtime_suspend(dev);
+}
+
+static int vidc_pm_resume(struct device *dev)
+{
+	return vidc_runtime_resume(dev);
+}
+
+static const struct dev_pm_ops vidc_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(vidc_pm_suspend, vidc_pm_resume)
+	SET_RUNTIME_PM_OPS(vidc_runtime_suspend, vidc_runtime_resume, NULL)
+};
+
+static struct platform_driver qcom_vidc_driver = {
+	.probe = vidc_probe,
+	.remove = vidc_remove,
+	.driver = {
+		.name = "qcom-vidc",
+		.of_match_table = vidc_dt_match,
+		.pm = &vidc_pm_ops,
+	},
+};
+
+module_platform_driver(qcom_vidc_driver);
+
+MODULE_ALIAS("platform:qcom-vidc");
+MODULE_DESCRIPTION("Qualcomm video encoder and decoder driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/qcom/vidc/core.h b/drivers/media/platform/qcom/vidc/core.h
new file mode 100644
index 000000000000..5dc8e05f8c36
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/core.h
@@ -0,0 +1,196 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef __VIDC_CORE_H_
+#define __VIDC_CORE_H_
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/videobuf2-core.h>
+
+#include "resources.h"
+#include "hfi.h"
+
+#define VIDC_DRV_NAME		"vidc"
+
+struct vidc_list {
+	struct list_head list;
+	struct mutex lock;
+};
+
+struct vidc_format {
+	u32 pixfmt;
+	int num_planes;
+	u32 type;
+};
+
+struct vidc_core {
+	struct list_head list;
+	void __iomem *base;
+	int irq;
+	struct clk *clks[VIDC_CLKS_NUM_MAX];
+	struct mutex lock;
+	struct hfi_core hfi;
+	struct video_device vdev_dec;
+	struct video_device vdev_enc;
+	struct v4l2_device v4l2_dev;
+	struct list_head instances;
+	const struct vidc_resources *res;
+	struct rproc *rproc;
+	bool rproc_booted;
+	struct device *dev;
+};
+
+struct vdec_controls {
+	u32 post_loop_deb_mode;
+	u32 profile;
+	u32 level;
+};
+
+struct venc_controls {
+	u16 gop_size;
+	u32 idr_period;
+	u32 num_p_frames;
+	u32 num_b_frames;
+	u32 bitrate_mode;
+	u32 bitrate;
+	u32 bitrate_peak;
+
+	u32 h264_i_period;
+	u32 h264_entropy_mode;
+	u32 h264_i_qp;
+	u32 h264_p_qp;
+	u32 h264_b_qp;
+	u32 h264_min_qp;
+	u32 h264_max_qp;
+	u32 h264_loop_filter_mode;
+	u32 h264_loop_filter_alpha;
+	u32 h264_loop_filter_beta;
+
+	u32 vp8_min_qp;
+	u32 vp8_max_qp;
+
+	u32 multi_slice_mode;
+	u32 multi_slice_max_bytes;
+	u32 multi_slice_max_mb;
+
+	u32 header_mode;
+
+	u32 profile;
+	u32 level;
+};
+
+struct vidc_inst {
+	struct list_head list;
+	struct mutex lock;
+	struct vidc_core *core;
+
+	struct vidc_list scratchbufs;
+	struct vidc_list persistbufs;
+	struct vidc_list registeredbufs;
+
+	struct list_head bufqueue;
+	struct mutex bufqueue_lock;
+
+	int streamoff;
+	int streamon;
+	struct vb2_queue bufq_out;
+	struct vb2_queue bufq_cap;
+
+	struct v4l2_ctrl_handler ctrl_handler;
+	union {
+		struct vdec_controls dec;
+		struct venc_controls enc;
+	} controls;
+	struct v4l2_fh fh;
+
+	struct hfi_inst *hfi_inst;
+
+	/* session fields */
+	u32 session_type;
+	u32 width;
+	u32 height;
+	u32 out_width;
+	u32 out_height;
+	u32 colorspace;
+	u8 ycbcr_enc;
+	u8 quantization;
+	u8 xfer_func;
+	u64 fps;
+	struct v4l2_fract timeperframe;
+	const struct vidc_format *fmt_out;
+	const struct vidc_format *fmt_cap;
+	unsigned int num_input_bufs;
+	unsigned int num_output_bufs;
+	bool in_reconfig;
+	u32 reconfig_width;
+	u32 reconfig_height;
+	u64 sequence;
+};
+
+#define ctrl_to_inst(ctrl)	\
+	container_of(ctrl->handler, struct vidc_inst, ctrl_handler)
+
+struct vidc_ctrl {
+	u32 id;
+	enum v4l2_ctrl_type type;
+	s32 min;
+	s32 max;
+	s32 def;
+	u32 step;
+	u64 menu_skip_mask;
+	u32 flags;
+	const char * const *qmenu;
+};
+
+/*
+ * Offset base for buffers on the destination queue - used to distinguish
+ * between source and destination buffers when mmapping - they receive the same
+ * offsets but for different queues
+ */
+#define DST_QUEUE_OFF_BASE	(1 << 30)
+
+extern const struct v4l2_file_operations vidc_fops;
+
+static inline void INIT_VIDC_LIST(struct vidc_list *mlist)
+{
+	mutex_init(&mlist->lock);
+	INIT_LIST_HEAD(&mlist->list);
+}
+
+static inline struct vidc_inst *to_inst(struct file *filp)
+{
+	return container_of(filp->private_data, struct vidc_inst, fh);
+}
+
+static inline struct hfi_inst *to_hfi_inst(struct file *filp)
+{
+	return to_inst(filp)->hfi_inst;
+}
+
+static inline struct vb2_queue *
+vidc_to_vb2q(struct file *file, enum v4l2_buf_type type)
+{
+	struct vidc_inst *inst = to_inst(file);
+
+	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		return &inst->bufq_cap;
+	else if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		return &inst->bufq_out;
+
+	return NULL;
+}
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/helpers.c b/drivers/media/platform/qcom/vidc/helpers.c
new file mode 100644
index 000000000000..81079f2b5ed1
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/helpers.c
@@ -0,0 +1,394 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/pm_runtime.h>
+#include <media/videobuf2-dma-sg.h>
+
+#include "helpers.h"
+#include "int_bufs.h"
+#include "load.h"
+#include "hfi_helper.h"
+
+static int session_set_buf(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct vb2_queue *q = vb->vb2_queue;
+	struct vidc_inst *inst = vb2_get_drv_priv(q);
+	struct vidc_core *core = inst->core;
+	struct device *dev = core->dev;
+	struct hfi_core *hfi = &core->hfi;
+	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
+	struct hfi_frame_data fdata;
+	int ret;
+
+	memset(&fdata, 0, sizeof(fdata));
+
+	fdata.alloc_len = vb2_plane_size(vb, 0);
+	fdata.device_addr = buf->dma_addr;
+	fdata.timestamp = vb->timestamp;
+	fdata.flags = 0;
+	fdata.clnt_data = buf->dma_addr;
+
+	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		fdata.buffer_type = HFI_BUFFER_INPUT;
+		fdata.filled_len = vb2_get_plane_payload(vb, 0);
+		fdata.offset = vb->planes[0].data_offset;
+
+		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
+			fdata.flags |= HFI_BUFFERFLAG_EOS;
+
+		ret = vidc_hfi_session_etb(hfi, inst->hfi_inst, &fdata);
+	} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		fdata.buffer_type = HFI_BUFFER_OUTPUT;
+		fdata.filled_len = 0;
+		fdata.offset = 0;
+
+		ret = vidc_hfi_session_ftb(hfi, inst->hfi_inst, &fdata);
+	} else {
+		ret = -EINVAL;
+	}
+
+	if (ret) {
+		dev_err(dev, "failed to set session buffer (%d)\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int session_unregister_bufs(struct vidc_inst *inst)
+{
+	struct device *dev = inst->core->dev;
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct hfi_buffer_desc *bd;
+	struct vidc_buffer *buf, *tmp;
+	int ret = 0;
+
+	mutex_lock(&inst->registeredbufs.lock);
+	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
+				 hfi_list) {
+		list_del(&buf->hfi_list);
+		bd = &buf->bd;
+		bd->response_required = 1;
+		ret = vidc_hfi_session_unset_buffers(hfi, inst->hfi_inst, bd);
+		if (ret) {
+			dev_err(dev, "%s: session release buffers failed\n",
+				__func__);
+			break;
+		}
+	}
+	mutex_unlock(&inst->registeredbufs.lock);
+
+	return ret;
+}
+
+static int session_register_bufs(struct vidc_inst *inst)
+{
+	struct device *dev = inst->core->dev;
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct hfi_buffer_desc *bd;
+	struct vidc_buffer *buf, *tmp;
+	int ret = 0;
+
+	mutex_lock(&inst->registeredbufs.lock);
+	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
+				 hfi_list) {
+		bd = &buf->bd;
+		ret = vidc_hfi_session_set_buffers(hfi, inst->hfi_inst, bd);
+		if (ret) {
+			dev_err(dev, "%s: session: set buffer failed\n",
+				__func__);
+			break;
+		}
+	}
+	mutex_unlock(&inst->registeredbufs.lock);
+
+	return ret;
+}
+
+int vidc_buf_descs(struct vidc_inst *inst, u32 type,
+		   struct hfi_buffer_requirements *out)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	u32 ptype = HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS;
+	union hfi_get_property hprop;
+	int ret, i;
+
+	if (out)
+		memset(out, 0, sizeof(*out));
+
+	ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype, &hprop);
+	if (ret)
+		return ret;
+
+	ret = -EINVAL;
+
+	for (i = 0; i < HFI_BUFFER_TYPE_MAX; i++) {
+		if (hprop.bufreq[i].type != type)
+			continue;
+
+		if (out)
+			memcpy(out, &hprop.bufreq[i], sizeof(*out));
+		ret = 0;
+		break;
+	}
+
+	return ret;
+}
+
+int vidc_set_color_format(struct vidc_inst *inst, u32 type, u32 pixfmt)
+{
+	struct hfi_uncompressed_format_select fmt;
+	struct hfi_core *hfi = &inst->core->hfi;
+	u32 ptype = HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT;
+	int ret;
+
+	fmt.buffer_type = type;
+
+	switch (pixfmt) {
+	case V4L2_PIX_FMT_NV12:
+		fmt.format = HFI_COLOR_FORMAT_NV12;
+		break;
+	case V4L2_PIX_FMT_NV21:
+		fmt.format = HFI_COLOR_FORMAT_NV21;
+		break;
+	default:
+		return -ENOTSUPP;
+	}
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fmt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+struct vb2_v4l2_buffer *
+vidc_vb2_find_buf(struct vidc_inst *inst, dma_addr_t addr)
+{
+	struct vidc_buffer *buf;
+	struct vb2_v4l2_buffer *vb = NULL;
+
+	mutex_lock(&inst->bufqueue_lock);
+
+	list_for_each_entry(buf, &inst->bufqueue, list) {
+		if (buf->dma_addr == addr) {
+			vb = &buf->vb;
+			break;
+		}
+	}
+
+	if (vb)
+		list_del(&buf->list);
+
+	mutex_unlock(&inst->bufqueue_lock);
+
+	return vb;
+}
+
+int vidc_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct vb2_queue *q = vb->vb2_queue;
+	struct vidc_inst *inst = vb2_get_drv_priv(q);
+	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
+	struct hfi_buffer_desc *bd = &buf->bd;
+	struct sg_table *sgt;
+
+	memset(bd, 0, sizeof(*bd));
+
+	if (q->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		return 0;
+
+	sgt = vb2_dma_sg_plane_desc(vb, 0);
+	if (!sgt)
+		return -EINVAL;
+
+	bd->buffer_size = vb2_plane_size(vb, 0);
+	bd->buffer_type = HFI_BUFFER_OUTPUT;
+	bd->num_buffers = 1;
+	bd->device_addr = sg_dma_address(sgt->sgl);
+
+	mutex_lock(&inst->registeredbufs.lock);
+	list_add_tail(&buf->hfi_list, &inst->registeredbufs.list);
+	mutex_unlock(&inst->registeredbufs.lock);
+
+	return 0;
+}
+
+int vidc_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
+	struct sg_table *sgt;
+
+	sgt = vb2_dma_sg_plane_desc(vb, 0);
+	if (!sgt)
+		return -EINVAL;
+
+	buf->dma_addr = sg_dma_address(sgt->sgl);
+
+	return 0;
+}
+
+void vidc_vb2_buf_queue(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct vidc_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+	struct vidc_core *core = inst->core;
+	struct device *dev = core->dev;
+	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
+	unsigned int state;
+	int ret;
+
+	mutex_lock(&inst->hfi_inst->lock);
+	state = inst->hfi_inst->state;
+	mutex_unlock(&inst->hfi_inst->lock);
+
+	if (state == INST_INVALID || state >= INST_STOP) {
+		vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
+		dev_dbg(dev, "%s: type:%d, invalid instance state\n", __func__,
+			vb->type);
+		return;
+	}
+
+	mutex_lock(&inst->bufqueue_lock);
+	list_add_tail(&buf->list, &inst->bufqueue);
+	mutex_unlock(&inst->bufqueue_lock);
+
+	if (!vb2_is_streaming(&inst->bufq_cap) ||
+	    !vb2_is_streaming(&inst->bufq_out))
+		return;
+
+	ret = session_set_buf(vb);
+	if (ret)
+		vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
+}
+
+void vidc_vb2_stop_streaming(struct vb2_queue *q)
+{
+	struct vidc_inst *inst = vb2_get_drv_priv(q);
+	struct hfi_inst *hfi_inst = inst->hfi_inst;
+	struct vidc_core *core = inst->core;
+	struct device *dev = core->dev;
+	struct hfi_core *hfi = &core->hfi;
+	int ret, streamoff;
+
+	mutex_lock(&inst->lock);
+	streamoff = inst->streamoff;
+	mutex_unlock(&inst->lock);
+
+	if (streamoff)
+		return;
+
+	mutex_lock(&inst->lock);
+	if (inst->streamon == 0) {
+		mutex_unlock(&inst->lock);
+		return;
+	}
+	mutex_unlock(&inst->lock);
+
+	ret = vidc_hfi_session_stop(hfi, inst->hfi_inst);
+	if (ret) {
+		dev_err(dev, "session: stop failed (%d)\n", ret);
+		goto abort;
+	}
+
+	ret = vidc_hfi_session_unload_res(hfi, inst->hfi_inst);
+	if (ret) {
+		dev_err(dev, "session: release resources failed (%d)\n", ret);
+		goto abort;
+	}
+
+	ret = session_unregister_bufs(inst);
+	if (ret) {
+		dev_err(dev, "failed to release capture buffers: %d\n", ret);
+		goto abort;
+	}
+
+	ret = internal_bufs_free(inst);
+
+	if (hfi_inst->state == INST_INVALID || hfi->state == CORE_INVALID) {
+		ret = -EINVAL;
+		goto abort;
+	}
+
+abort:
+	if (ret)
+		vidc_hfi_session_abort(hfi, inst->hfi_inst);
+
+	vidc_scale_clocks(inst->core);
+
+	ret = vidc_hfi_session_deinit(hfi, inst->hfi_inst);
+
+	mutex_lock(&inst->lock);
+	inst->streamoff = 1;
+	mutex_unlock(&inst->lock);
+
+	if (ret)
+		dev_err(dev, "stop streaming failed type: %d, ret: %d\n",
+			q->type, ret);
+
+	ret = pm_runtime_put_sync(dev);
+	if (ret < 0)
+		dev_err(dev, "%s: pm_runtime_put_sync (%d)\n", __func__, ret);
+}
+
+int vidc_vb2_start_streaming(struct vidc_inst *inst)
+{
+	struct device *dev = inst->core->dev;
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct vidc_buffer *buf, *n;
+	int ret;
+
+	ret = session_register_bufs(inst);
+	if (ret)
+		return ret;
+
+	ret = internal_bufs_alloc(inst);
+	if (ret)
+		return ret;
+
+	vidc_scale_clocks(inst->core);
+
+	ret = vidc_hfi_session_load_res(hfi, inst->hfi_inst);
+	if (ret) {
+		dev_err(dev, "session: load resources (%d)\n", ret);
+		return ret;
+	}
+
+	ret = vidc_hfi_session_start(hfi, inst->hfi_inst);
+	if (ret) {
+		dev_err(dev, "session: start failed (%d)\n", ret);
+		return ret;
+	}
+
+	mutex_lock(&inst->bufqueue_lock);
+	list_for_each_entry_safe(buf, n, &inst->bufqueue, list) {
+		ret = session_set_buf(&buf->vb.vb2_buf);
+		if (ret)
+			break;
+	}
+	mutex_unlock(&inst->bufqueue_lock);
+
+	if (!ret) {
+		mutex_lock(&inst->lock);
+		inst->streamon = 1;
+		mutex_unlock(&inst->lock);
+	}
+
+	return ret;
+}
diff --git a/drivers/media/platform/qcom/vidc/helpers.h b/drivers/media/platform/qcom/vidc/helpers.h
new file mode 100644
index 000000000000..a151c96bf939
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/helpers.h
@@ -0,0 +1,43 @@
+/*
+ * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_COMMON_H__
+#define __VIDC_COMMON_H__
+
+#include <linux/list.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "core.h"
+
+struct vidc_buffer {
+	struct vb2_v4l2_buffer vb;
+	struct list_head list;
+	dma_addr_t dma_addr;
+	struct list_head hfi_list;
+	struct hfi_buffer_desc bd;
+};
+
+#define to_vidc_buffer(buf)	container_of(buf, struct vidc_buffer, vb)
+
+struct vb2_v4l2_buffer *
+vidc_vb2_find_buf(struct vidc_inst *inst, dma_addr_t addr);
+int vidc_vb2_buf_init(struct vb2_buffer *vb);
+int vidc_vb2_buf_prepare(struct vb2_buffer *vb);
+void vidc_vb2_buf_queue(struct vb2_buffer *vb);
+void vidc_vb2_stop_streaming(struct vb2_queue *q);
+int vidc_vb2_start_streaming(struct vidc_inst *inst);
+int vidc_buf_descs(struct vidc_inst *inst, u32 type,
+		   struct hfi_buffer_requirements *out);
+int vidc_set_color_format(struct vidc_inst *inst, u32 type, u32 fmt);
+#endif
diff --git a/drivers/media/platform/qcom/vidc/int_bufs.c b/drivers/media/platform/qcom/vidc/int_bufs.c
new file mode 100644
index 000000000000..393d75785d7a
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/int_bufs.c
@@ -0,0 +1,325 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "helpers.h"
+#include "int_bufs.h"
+#include "mem.h"
+
+struct vidc_internal_buf {
+	struct list_head list;
+	u32 type;
+	struct vidc_mem *mem;
+};
+
+static u32 scratch_buf_sufficient(struct vidc_inst *inst, u32 buffer_type)
+{
+	struct hfi_buffer_requirements bufreq;
+	struct vidc_internal_buf *buf;
+	unsigned int count = 0;
+	int ret;
+
+	ret = vidc_buf_descs(inst, buffer_type, &bufreq);
+	if (ret)
+		return 0;
+
+	/* Check if current scratch buffers are sufficient */
+	mutex_lock(&inst->scratchbufs.lock);
+	list_for_each_entry(buf, &inst->scratchbufs.list, list) {
+		if (buf->type == buffer_type &&
+		    buf->mem->size >= bufreq.size)
+			count++;
+	}
+	mutex_unlock(&inst->scratchbufs.lock);
+
+	if (count != bufreq.count_actual)
+		return 0;
+
+	return buffer_type;
+}
+
+static int internal_set_buf_on_fw(struct vidc_inst *inst, u32 buffer_type,
+				  struct vidc_mem *mem, bool reuse)
+{
+	struct device *dev = inst->core->dev;
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct hfi_buffer_desc bd = {0};
+	int ret;
+
+	bd.buffer_size = mem->size;
+	bd.buffer_type = buffer_type;
+	bd.num_buffers = 1;
+	bd.device_addr = mem->da;
+
+	ret = vidc_hfi_session_set_buffers(hfi, inst->hfi_inst, &bd);
+	if (ret) {
+		dev_err(dev, "set session buffers failed\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static int internal_alloc_and_set(struct vidc_inst *inst,
+				  struct hfi_buffer_requirements *bufreq,
+				  struct vidc_list *buf_list)
+{
+	struct vidc_internal_buf *buf;
+	struct vidc_mem *mem;
+	unsigned int i;
+	int ret = 0;
+
+	if (!bufreq->size)
+		return 0;
+
+	for (i = 0; i < bufreq->count_actual; i++) {
+		mem = mem_alloc(inst->core->dev, bufreq->size, 0);
+		if (IS_ERR(mem)) {
+			ret = PTR_ERR(mem);
+			goto err_no_mem;
+		}
+
+		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
+		if (!buf) {
+			ret = -ENOMEM;
+			goto fail_kzalloc;
+		}
+
+		buf->mem = mem;
+		buf->type = bufreq->type;
+
+		ret = internal_set_buf_on_fw(inst, bufreq->type, mem, false);
+		if (ret)
+			goto fail_set_buffers;
+
+		mutex_lock(&buf_list->lock);
+		list_add_tail(&buf->list, &buf_list->list);
+		mutex_unlock(&buf_list->lock);
+	}
+
+	return ret;
+
+fail_set_buffers:
+	kfree(buf);
+fail_kzalloc:
+	mem_free(mem);
+err_no_mem:
+	return ret;
+}
+
+static bool scratch_reuse_buffer(struct vidc_inst *inst, u32 buffer_type)
+{
+	struct device *dev = inst->core->dev;
+	struct vidc_internal_buf *buf;
+	bool reused = false;
+	int ret = 0;
+
+	mutex_lock(&inst->scratchbufs.lock);
+	list_for_each_entry(buf, &inst->scratchbufs.list, list) {
+		if (buf->type != buffer_type)
+			continue;
+
+		ret = internal_set_buf_on_fw(inst, buffer_type, buf->mem, true);
+		if (ret) {
+			dev_err(dev, "set internal buffers failed\n");
+			reused = false;
+			break;
+		}
+
+		reused = true;
+	}
+	mutex_unlock(&inst->scratchbufs.lock);
+
+	return reused;
+}
+
+static int scratch_set_buffer(struct vidc_inst *inst, u32 type)
+{
+	struct hfi_buffer_requirements bufreq;
+	int ret;
+
+	ret = vidc_buf_descs(inst, type, &bufreq);
+	if (ret)
+		return 0;
+
+	if (scratch_reuse_buffer(inst, type))
+		return 0;
+
+	return internal_alloc_and_set(inst, &bufreq, &inst->scratchbufs);
+}
+
+static int persist_set_buffer(struct vidc_inst *inst, u32 type)
+{
+	struct hfi_buffer_requirements bufreq;
+	int ret;
+
+	ret = vidc_buf_descs(inst, type, &bufreq);
+	if (ret)
+		return 0;
+
+	mutex_lock(&inst->persistbufs.lock);
+	if (!list_empty(&inst->persistbufs.list)) {
+		mutex_unlock(&inst->persistbufs.lock);
+		return 0;
+	}
+	mutex_unlock(&inst->persistbufs.lock);
+
+	return internal_alloc_and_set(inst, &bufreq, &inst->persistbufs);
+}
+
+static int scratch_unset_buffers(struct vidc_inst *inst, bool reuse)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct vidc_internal_buf *buf, *n;
+	struct hfi_buffer_desc bd = {0};
+	u32 sufficient = 0;
+	int ret = 0;
+
+	mutex_lock(&inst->scratchbufs.lock);
+	list_for_each_entry_safe(buf, n, &inst->scratchbufs.list, list) {
+		bd.buffer_size = buf->mem->size;
+		bd.buffer_type = buf->type;
+		bd.num_buffers = 1;
+		bd.device_addr = buf->mem->da;
+		bd.response_required = true;
+
+		ret = vidc_hfi_session_unset_buffers(hfi, inst->hfi_inst, &bd);
+
+		/* If scratch buffers can be reused, do not free the buffers */
+		if (reuse) {
+			sufficient = scratch_buf_sufficient(inst, buf->type);
+			if (sufficient == buf->type)
+				continue;
+		}
+
+		list_del(&buf->list);
+		mem_free(buf->mem);
+		kfree(buf);
+	}
+	mutex_unlock(&inst->scratchbufs.lock);
+
+	return ret;
+}
+
+static int persist_unset_buffers(struct vidc_inst *inst)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct vidc_internal_buf *buf, *n;
+	struct hfi_buffer_desc bd = {0};
+	int ret = 0;
+
+	mutex_lock(&inst->persistbufs.lock);
+	list_for_each_entry_safe(buf, n, &inst->persistbufs.list, list) {
+		bd.buffer_size = buf->mem->size;
+		bd.buffer_type = buf->type;
+		bd.num_buffers = 1;
+		bd.device_addr = buf->mem->da;
+		bd.response_required = true;
+
+		ret = vidc_hfi_session_unset_buffers(hfi, inst->hfi_inst, &bd);
+
+		list_del(&buf->list);
+		mem_free(buf->mem);
+		kfree(buf);
+	}
+	mutex_unlock(&inst->persistbufs.lock);
+
+	return ret;
+}
+
+static int scratch_set_buffers(struct vidc_inst *inst)
+{
+	struct device *dev = inst->core->dev;
+	int ret;
+
+	ret = scratch_unset_buffers(inst, true);
+	if (ret)
+		dev_warn(dev, "Failed to release scratch buffers\n");
+
+	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH);
+	if (ret)
+		goto error;
+
+	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_1);
+	if (ret)
+		goto error;
+
+	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_2);
+	if (ret)
+		goto error;
+
+	return 0;
+error:
+	scratch_unset_buffers(inst, false);
+	return ret;
+}
+
+static int persist_set_buffers(struct vidc_inst *inst)
+{
+	int ret;
+
+	ret = persist_set_buffer(inst, HFI_BUFFER_INTERNAL_PERSIST);
+	if (ret)
+		goto error;
+
+	ret = persist_set_buffer(inst, HFI_BUFFER_INTERNAL_PERSIST_1);
+	if (ret)
+		goto error;
+
+	return 0;
+
+error:
+	persist_unset_buffers(inst);
+	return ret;
+}
+
+int internal_bufs_alloc(struct vidc_inst *inst)
+{
+	struct device *dev = inst->core->dev;
+	int ret;
+
+	ret = scratch_set_buffers(inst);
+	if (ret) {
+		dev_err(dev, "set scratch buffers (%d)\n", ret);
+		return ret;
+	}
+
+	ret = persist_set_buffers(inst);
+	if (ret) {
+		dev_err(dev, "set persist buffers (%d)\n", ret);
+		goto error;
+	}
+
+	return 0;
+
+error:
+	scratch_unset_buffers(inst, false);
+	return ret;
+}
+
+int internal_bufs_free(struct vidc_inst *inst)
+{
+	struct device *dev = inst->core->dev;
+	int ret;
+
+	ret = scratch_unset_buffers(inst, false);
+	if (ret)
+		dev_err(dev, "failed to release scratch buffers: %d\n", ret);
+
+	ret = persist_unset_buffers(inst);
+	if (ret)
+		dev_err(dev, "failed to release persist buffers: %d\n", ret);
+
+	return ret;
+}
diff --git a/drivers/media/platform/qcom/vidc/int_bufs.h b/drivers/media/platform/qcom/vidc/int_bufs.h
new file mode 100644
index 000000000000..5f8b2b85839f
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/int_bufs.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_INTERNAL_BUFFERS_H__
+#define __VIDC_INTERNAL_BUFFERS_H__
+
+struct vidc_inst;
+
+int internal_bufs_alloc(struct vidc_inst *inst);
+int internal_bufs_free(struct vidc_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/load.c b/drivers/media/platform/qcom/vidc/load.c
new file mode 100644
index 000000000000..8ae25fc0e8a5
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/load.c
@@ -0,0 +1,104 @@
+/*
+ * Copyright (c) 2013-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/clk.h>
+
+#include "core.h"
+#include "load.h"
+
+static u32 get_inst_load(struct vidc_inst *inst)
+{
+	int mbs;
+	u32 w = inst->width;
+	u32 h = inst->height;
+
+	if (!inst->hfi_inst || !(inst->hfi_inst->state >= INST_INIT &&
+				 inst->hfi_inst->state < INST_STOP))
+		return 0;
+
+	mbs = (ALIGN(w, 16) / 16) * (ALIGN(h, 16) / 16);
+
+	return mbs * inst->fps;
+}
+
+static u32 get_load(struct vidc_core *core, u32 session_type)
+{
+	struct vidc_inst *inst = NULL;
+	u32 mbs_per_sec = 0;
+
+	mutex_lock(&core->lock);
+	list_for_each_entry(inst, &core->instances, list) {
+		if (inst->session_type != session_type)
+			continue;
+
+		mbs_per_sec += get_inst_load(inst);
+	}
+	mutex_unlock(&core->lock);
+
+	return mbs_per_sec;
+}
+
+static int scale_clocks_load(struct vidc_core *core, u32 mbs_per_sec)
+{
+	const struct freq_tbl *table = core->res->freq_tbl;
+	int num_rows = core->res->freq_tbl_size;
+	struct clk *clk = core->clks[0];
+	struct device *dev = core->dev;
+	unsigned long freq = table[0].freq;
+	int ret, i;
+
+	if (!mbs_per_sec && num_rows > 1) {
+		freq = table[num_rows - 1].freq;
+		goto set_freq;
+	}
+
+	for (i = 0; i < num_rows; i++) {
+		if (mbs_per_sec > table[i].load)
+			break;
+		freq = table[i].freq;
+	}
+
+set_freq:
+
+	ret = clk_set_rate(clk, freq);
+	if (ret) {
+		dev_err(dev, "failed to set clock rate %lu (%d)\n", freq, ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+int vidc_scale_clocks(struct vidc_core *core)
+{
+	struct device *dev = core->dev;
+	u32 mbs_per_sec;
+	int ret;
+
+	mbs_per_sec = get_load(core, VIDC_SESSION_TYPE_ENC) +
+		      get_load(core, VIDC_SESSION_TYPE_DEC);
+
+	if (mbs_per_sec > core->res->max_load) {
+		dev_warn(dev, "HW is overloaded, needed: %d max: %d\n",
+			 mbs_per_sec, core->res->max_load);
+		return -EBUSY;
+	}
+
+	ret = scale_clocks_load(core, mbs_per_sec);
+	if (ret)
+		dev_warn(dev, "failed to scale clocks, performance might be impacted\n");
+
+	return 0;
+}
diff --git a/drivers/media/platform/qcom/vidc/load.h b/drivers/media/platform/qcom/vidc/load.h
new file mode 100644
index 000000000000..b8f8dc57e18f
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/load.h
@@ -0,0 +1,22 @@
+/*
+ * Copyright (c) 2013-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_LOAD_H__
+#define __VIDC_LOAD_H__
+
+struct vidc_core;
+
+int vidc_scale_clocks(struct vidc_core *core);
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/mem.c b/drivers/media/platform/qcom/vidc/mem.c
new file mode 100644
index 000000000000..6a83b5784410
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/mem.c
@@ -0,0 +1,64 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/dma-direction.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/slab.h>
+
+#include "mem.h"
+
+struct vidc_mem *mem_alloc(struct device *dev, size_t size, int map_kernel)
+{
+	struct vidc_mem *mem;
+
+	if (!size)
+		return ERR_PTR(-EINVAL);
+
+	if (IS_ERR(dev))
+		return ERR_CAST(dev);
+
+	mem = kzalloc(sizeof(*mem), GFP_KERNEL);
+	if (!mem)
+		return ERR_PTR(-ENOMEM);
+
+	mem->size = ALIGN(size, SZ_4K);
+	mem->iommu_dev = dev;
+
+	mem->attrs = DMA_ATTR_WRITE_COMBINE;
+
+	if (!map_kernel)
+		mem->attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+
+	mem->kvaddr = dma_alloc_attrs(mem->iommu_dev, mem->size, &mem->da,
+				      GFP_KERNEL, mem->attrs);
+	if (!mem->kvaddr) {
+		kfree(mem);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	return mem;
+}
+
+void mem_free(struct vidc_mem *mem)
+{
+	if (!mem)
+		return;
+
+	dma_free_attrs(mem->iommu_dev, mem->size, mem->kvaddr,
+	       mem->da, mem->attrs);
+	kfree(mem);
+};
diff --git a/drivers/media/platform/qcom/vidc/mem.h b/drivers/media/platform/qcom/vidc/mem.h
new file mode 100644
index 000000000000..cab81aa2f550
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/mem.h
@@ -0,0 +1,32 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef __VIDC_MEM_H__
+#define __VIDC_MEM_H__
+
+struct device;
+
+struct vidc_mem {
+	size_t size;
+	void *kvaddr;
+	dma_addr_t da;
+	unsigned long attrs;
+	struct device *iommu_dev;
+};
+
+struct vidc_mem *mem_alloc(struct device *dev, size_t size, int map_kernel);
+void mem_free(struct vidc_mem *mem);
+
+#endif /* __VIDC_MEM_H__ */
diff --git a/drivers/media/platform/qcom/vidc/resources.c b/drivers/media/platform/qcom/vidc/resources.c
new file mode 100644
index 000000000000..e00ed1caa824
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/resources.c
@@ -0,0 +1,46 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/bug.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+
+#include "hfi.h"
+
+static const struct freq_tbl msm8916_freq_table[] = {
+	{ 352800, 228570000 },	/* 1920x1088 @ 30 + 1280x720 @ 30 */
+	{ 244800, 160000000 },	/* 1920x1088 @ 30 */
+	{ 108000, 100000000 },	/* 1280x720 @ 30 */
+};
+
+static const struct reg_val msm8916_reg_preset[] = {
+	{ 0xe0020, 0x05555556 },
+	{ 0xe0024, 0x05555556 },
+	{ 0x80124, 0x00000003 },
+};
+
+const struct vidc_resources msm8916_res = {
+	.freq_tbl = msm8916_freq_table,
+	.freq_tbl_size = ARRAY_SIZE(msm8916_freq_table),
+	.reg_tbl = msm8916_reg_preset,
+	.reg_tbl_size = ARRAY_SIZE(msm8916_reg_preset),
+	.clks = {"core", "iface", "bus", },
+	.clks_num = 3,
+	.max_load = 352800, /* 720p@30 + 1080p@30 */
+	.hfi_version = 0,
+	.vmem_id = VIDC_RESOURCE_NONE,
+	.vmem_size = 0,
+	.vmem_addr = 0,
+	.dma_mask = 0xddc00000 - 1,
+};
diff --git a/drivers/media/platform/qcom/vidc/resources.h b/drivers/media/platform/qcom/vidc/resources.h
new file mode 100644
index 000000000000..9f2afad78a9f
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/resources.h
@@ -0,0 +1,46 @@
+/*
+ * Copyright (c) 2013-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_RESOURCES_H__
+#define __VIDC_RESOURCES_H__
+
+#define VIDC_CLKS_NUM_MAX	7
+
+struct freq_tbl {
+	unsigned int load;
+	unsigned long freq;
+};
+
+struct reg_val {
+	u32 reg;
+	u32 value;
+};
+
+struct vidc_resources {
+	u64 dma_mask;
+	const struct freq_tbl *freq_tbl;
+	unsigned int freq_tbl_size;
+	const struct reg_val *reg_tbl;
+	unsigned int reg_tbl_size;
+	const char *clks[VIDC_CLKS_NUM_MAX];
+	unsigned int clks_num;
+	unsigned int hfi_version;
+	u32 max_load;
+	unsigned int vmem_id;
+	u32 vmem_size;
+	u32 vmem_addr;
+};
+
+extern const struct vidc_resources msm8916_res;
+#endif
-- 
2.7.4

[PATCH 3/8] media: vidc: decoder: add video decoder files

[Stanimir Varbanov]

This consists of video decoder implementation plus decoder
controls.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/vidc/vdec.c       | 1100 +++++++++++++++++++++++++
 drivers/media/platform/qcom/vidc/vdec.h       |   27 +
 drivers/media/platform/qcom/vidc/vdec_ctrls.c |  200 +++++
 drivers/media/platform/qcom/vidc/vdec_ctrls.h |   21 +
 4 files changed, 1348 insertions(+)
 create mode 100644 drivers/media/platform/qcom/vidc/vdec.c
 create mode 100644 drivers/media/platform/qcom/vidc/vdec.h
 create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.c
 create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.h

diff --git a/drivers/media/platform/qcom/vidc/vdec.c b/drivers/media/platform/qcom/vidc/vdec.c
new file mode 100644
index 000000000000..a631a354742f
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/vdec.c
@@ -0,0 +1,1100 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/slab.h>
+#include <linux/pm_runtime.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ctrls.h>
+#include <media/videobuf2-dma-sg.h>
+
+#include "core.h"
+#include "helpers.h"
+#include "load.h"
+#include "vdec.h"
+#include "vdec_ctrls.h"
+
+#define MACROBLOCKS_PER_PIXEL	(16 * 16)
+
+static u32 get_framesize_nv12(int plane, u32 height, u32 width)
+{
+	u32 y_stride, uv_stride, y_plane;
+	u32 y_sclines, uv_sclines, uv_plane;
+	u32 size;
+
+	y_stride = ALIGN(width, 128);
+	uv_stride = ALIGN(width, 128);
+	y_sclines = ALIGN(height, 32);
+	uv_sclines = ALIGN(((height + 1) >> 1), 16);
+
+	y_plane = y_stride * y_sclines;
+	uv_plane = uv_stride * uv_sclines + SZ_4K;
+	size = y_plane + uv_plane + SZ_8K;
+
+	return ALIGN(size, SZ_4K);
+}
+
+static u32 get_framesize_compressed(u32 mbs_per_frame)
+{
+	return ((mbs_per_frame * MACROBLOCKS_PER_PIXEL * 3 / 2) / 2) + 128;
+}
+
+static const struct vidc_format vdec_formats[] = {
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_MPEG4,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_MPEG2,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_H263,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_VP8,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_XVID,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+};
+
+static const struct vidc_format *find_format(u32 pixfmt, u32 type)
+{
+	const struct vidc_format *fmt = vdec_formats;
+	unsigned int size = ARRAY_SIZE(vdec_formats);
+	unsigned int i;
+
+	for (i = 0; i < size; i++) {
+		if (fmt[i].pixfmt == pixfmt)
+			break;
+	}
+
+	if (i == size || fmt[i].type != type)
+		return NULL;
+
+	return &fmt[i];
+}
+
+static const struct vidc_format *find_format_by_index(int index, u32 type)
+{
+	const struct vidc_format *fmt = vdec_formats;
+	unsigned int size = ARRAY_SIZE(vdec_formats);
+	int i, k = 0;
+
+	if (index < 0 || index > size)
+		return NULL;
+
+	for (i = 0; i < size; i++) {
+		if (fmt[i].type != type)
+			continue;
+		if (k == index)
+			break;
+		k++;
+	}
+
+	if (i == size)
+		return NULL;
+
+	return &fmt[i];
+}
+
+static int vdec_set_properties(struct vidc_inst *inst)
+{
+	struct vdec_controls *ctr = &inst->controls.dec;
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct hfi_enable en = { .enable = 1 };
+	struct hfi_framerate frate;
+	u32 ptype;
+	int ret;
+
+	ptype = HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER;
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &en);
+	if (ret)
+		return ret;
+
+	ptype = HFI_PROPERTY_CONFIG_FRAME_RATE;
+	frate.buffer_type = HFI_BUFFER_INPUT;
+	frate.framerate = inst->fps * (1 << 16);
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &frate);
+	if (ret)
+		return ret;
+
+	if (ctr->post_loop_deb_mode) {
+		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
+		en.enable = 1;
+		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype,
+						    &en);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static const struct vidc_format *
+vdec_try_fmt_common(struct vidc_inst *inst, struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
+	struct hfi_inst *hfi_inst = inst->hfi_inst;
+	const struct vidc_format *fmt;
+	unsigned int p;
+
+	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
+	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+
+	fmt = find_format(pixmp->pixelformat, f->type);
+	if (!fmt) {
+		if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+			pixmp->pixelformat = V4L2_PIX_FMT_NV12;
+		else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+			pixmp->pixelformat = V4L2_PIX_FMT_H264;
+		else
+			return NULL;
+		fmt = find_format(pixmp->pixelformat, f->type);
+		pixmp->width = 1280;
+		pixmp->height = 720;
+	}
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		pixmp->height = ALIGN(pixmp->height, 32);
+
+	pixmp->width = clamp(pixmp->width, hfi_inst->width.min,
+			     hfi_inst->width.max);
+	pixmp->height = clamp(pixmp->height, hfi_inst->height.min,
+			      hfi_inst->height.max);
+	if (pixmp->field == V4L2_FIELD_ANY)
+		pixmp->field = V4L2_FIELD_NONE;
+	pixmp->num_planes = fmt->num_planes;
+	pixmp->flags = 0;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		for (p = 0; p < pixmp->num_planes; p++) {
+			pfmt[p].sizeimage = get_framesize_nv12(p, pixmp->height,
+							       pixmp->width);
+
+			pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
+		}
+	} else {
+		u32 mbs = pixmp->width * pixmp->height / MACROBLOCKS_PER_PIXEL;
+
+		pfmt[0].sizeimage = get_framesize_compressed(mbs);
+		pfmt[0].bytesperline = 0;
+	}
+
+	return fmt;
+}
+
+static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct vidc_inst *inst = to_inst(file);
+	const struct vidc_format *fmt;
+
+	fmt = vdec_try_fmt_common(inst, f);
+	if (!fmt)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct vidc_inst *inst = to_inst(file);
+	const struct vidc_format *fmt = NULL;
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		fmt = inst->fmt_cap;
+	else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		fmt = inst->fmt_out;
+
+	if (inst->in_reconfig) {
+		inst->height = inst->reconfig_height;
+		inst->width = inst->reconfig_width;
+		inst->in_reconfig = false;
+	}
+
+	pixmp->pixelformat = fmt->pixfmt;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		pixmp->width = inst->width;
+		pixmp->height = inst->height;
+		pixmp->colorspace = inst->colorspace;
+		pixmp->ycbcr_enc = inst->ycbcr_enc;
+		pixmp->quantization = inst->quantization;
+		pixmp->xfer_func = inst->xfer_func;
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		pixmp->width = inst->out_width;
+		pixmp->height = inst->out_height;
+	}
+
+	vdec_try_fmt_common(inst, f);
+
+	return 0;
+}
+
+static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct vidc_inst *inst = to_inst(file);
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+	struct v4l2_pix_format_mplane orig_pixmp;
+	const struct vidc_format *fmt;
+	struct v4l2_format format;
+	u32 pixfmt_out = 0, pixfmt_cap = 0;
+
+	orig_pixmp = *pixmp;
+
+	fmt = vdec_try_fmt_common(inst, f);
+	if (!fmt)
+		return -EINVAL;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		pixfmt_out = pixmp->pixelformat;
+		pixfmt_cap = inst->fmt_cap->pixfmt;
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		pixfmt_cap = pixmp->pixelformat;
+		pixfmt_out = inst->fmt_out->pixfmt;
+	}
+
+	memset(&format, 0, sizeof(format));
+
+	format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	format.fmt.pix_mp.pixelformat = pixfmt_out;
+	format.fmt.pix_mp.width = orig_pixmp.width;
+	format.fmt.pix_mp.height = orig_pixmp.height;
+	vdec_try_fmt_common(inst, &format);
+	inst->out_width = format.fmt.pix_mp.width;
+	inst->out_height = format.fmt.pix_mp.height;
+	inst->colorspace = pixmp->colorspace;
+	inst->ycbcr_enc = pixmp->ycbcr_enc;
+	inst->quantization = pixmp->quantization;
+	inst->xfer_func = pixmp->xfer_func;
+
+	memset(&format, 0, sizeof(format));
+
+	format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	format.fmt.pix_mp.pixelformat = pixfmt_cap;
+	format.fmt.pix_mp.width = orig_pixmp.width;
+	format.fmt.pix_mp.height = orig_pixmp.height;
+	vdec_try_fmt_common(inst, &format);
+	inst->width = format.fmt.pix_mp.width;
+	inst->height = format.fmt.pix_mp.height;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		inst->fmt_out = fmt;
+	else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		inst->fmt_cap = fmt;
+
+	return 0;
+}
+
+static int
+vdec_g_selection(struct file *file, void *priv, struct v4l2_selection *s)
+{
+	struct vidc_inst *inst = to_inst(file);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+	case V4L2_SEL_TGT_COMPOSE:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	s->r.top = 0;
+	s->r.left = 0;
+	s->r.width = inst->out_width;
+	s->r.height = inst->out_height;
+
+	return 0;
+}
+
+static int
+vdec_reqbufs(struct file *file, void *fh, struct v4l2_requestbuffers *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	if (!b->count)
+		vb2_core_queue_release(queue);
+
+	return vb2_reqbufs(queue, b);
+}
+
+static int
+vdec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
+{
+	strlcpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
+	strlcpy(cap->card, "video decoder", sizeof(cap->card));
+	strlcpy(cap->bus_info, "platform:vidc", sizeof(cap->bus_info));
+
+	cap->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+
+	return 0;
+}
+
+static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+	const struct vidc_format *fmt;
+
+	memset(f->reserved, 0, sizeof(f->reserved));
+
+	fmt = find_format_by_index(f->index, f->type);
+	if (!fmt)
+		return -EINVAL;
+
+	f->pixelformat = fmt->pixfmt;
+
+	return 0;
+}
+
+static int vdec_querybuf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+	unsigned int p;
+	int ret;
+
+	if (!queue)
+		return -EINVAL;
+
+	ret = vb2_querybuf(queue, b);
+	if (ret)
+		return ret;
+
+	if (b->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    b->memory == V4L2_MEMORY_MMAP) {
+		for (p = 0; p < b->length; p++)
+			b->m.planes[p].m.mem_offset += DST_QUEUE_OFF_BASE;
+	}
+
+	return 0;
+}
+
+static int
+vdec_create_bufs(struct file *file, void *fh, struct v4l2_create_buffers *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->format.type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_create_bufs(queue, b);
+}
+
+static int vdec_prepare_buf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_prepare_buf(queue, b);
+}
+
+static int vdec_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_qbuf(queue, b);
+}
+
+static int
+vdec_exportbuf(struct file *file, void *fh, struct v4l2_exportbuffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_expbuf(queue, b);
+}
+
+static int vdec_dqbuf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_dqbuf(queue, b, file->f_flags & O_NONBLOCK);
+}
+
+static int vdec_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_streamon(queue, type);
+}
+
+static int vdec_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_streamoff(queue, type);
+}
+
+static int vdec_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+	struct vidc_inst *inst = to_inst(file);
+	struct v4l2_captureparm *cap = &a->parm.capture;
+	struct v4l2_fract *timeperframe = &cap->timeperframe;
+	u64 us_per_frame, fps;
+
+	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		return -EINVAL;
+
+	memset(cap->reserved, 0, sizeof(cap->reserved));
+	if (!timeperframe->denominator)
+		timeperframe->denominator = inst->timeperframe.denominator;
+	if (!timeperframe->numerator)
+		timeperframe->numerator = inst->timeperframe.numerator;
+	cap->readbuffers = 0;
+	cap->extendedmode = 0;
+	cap->capability = V4L2_CAP_TIMEPERFRAME;
+	us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
+	do_div(us_per_frame, timeperframe->denominator);
+
+	if (!us_per_frame)
+		return -EINVAL;
+
+	fps = (u64)USEC_PER_SEC;
+	do_div(fps, us_per_frame);
+
+	inst->fps = fps;
+	inst->timeperframe = *timeperframe;
+
+	return 0;
+}
+
+static int vdec_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+	struct vidc_inst *inst = to_inst(file);
+
+	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		return -EINVAL;
+
+	a->parm.capture.capability |= V4L2_CAP_TIMEPERFRAME;
+	a->parm.capture.timeperframe = inst->timeperframe;
+
+	return 0;
+}
+
+static int vdec_enum_framesizes(struct file *file, void *fh,
+				struct v4l2_frmsizeenum *fsize)
+{
+	struct hfi_inst *hfi_inst = to_hfi_inst(file);
+	const struct vidc_format *fmt;
+
+	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+
+	fmt = find_format(fsize->pixel_format,
+			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (!fmt) {
+		fmt = find_format(fsize->pixel_format,
+				  V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+		if (!fmt)
+			return -EINVAL;
+	}
+
+	if (fsize->index)
+		return -EINVAL;
+
+	fsize->stepwise.min_width = hfi_inst->width.min;
+	fsize->stepwise.max_width = hfi_inst->width.max;
+	fsize->stepwise.step_width = hfi_inst->width.step_size;
+	fsize->stepwise.min_height = hfi_inst->height.min;
+	fsize->stepwise.max_height = hfi_inst->height.max;
+	fsize->stepwise.step_height = hfi_inst->height.step_size;
+
+	return 0;
+}
+
+static int vdec_enum_frameintervals(struct file *file, void *fh,
+				    struct v4l2_frmivalenum *fival)
+{
+	struct hfi_inst *hfi_inst = to_hfi_inst(file);
+	const struct vidc_format *fmt;
+
+	fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
+
+	fmt = find_format(fival->pixel_format,
+			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (!fmt)
+		return -EINVAL;
+
+	if (fival->index)
+		return -EINVAL;
+
+	if (!fival->width || !fival->height)
+		return -EINVAL;
+
+	if (fival->width > hfi_inst->width.max ||
+	    fival->width < hfi_inst->width.min ||
+	    fival->height > hfi_inst->height.max ||
+	    fival->height < hfi_inst->height.min)
+		return -EINVAL;
+
+	fival->stepwise.min.numerator = hfi_inst->framerate.min;
+	fival->stepwise.min.denominator = 1;
+	fival->stepwise.max.numerator = hfi_inst->framerate.max;
+	fival->stepwise.max.denominator = 1;
+	fival->stepwise.step.numerator = hfi_inst->framerate.step_size;
+	fival->stepwise.step.denominator = 1;
+
+	return 0;
+}
+
+static int vdec_subscribe_event(struct v4l2_fh *fh,
+				const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_EOS:
+		return v4l2_event_subscribe(fh, sub, 2, NULL);
+	case V4L2_EVENT_SOURCE_CHANGE:
+		return v4l2_src_change_event_subscribe(fh, sub);
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
+	.vidioc_querycap = vdec_querycap,
+	.vidioc_enum_fmt_vid_cap_mplane = vdec_enum_fmt,
+	.vidioc_enum_fmt_vid_out_mplane = vdec_enum_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = vdec_s_fmt,
+	.vidioc_s_fmt_vid_out_mplane = vdec_s_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = vdec_g_fmt,
+	.vidioc_g_fmt_vid_out_mplane = vdec_g_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = vdec_try_fmt,
+	.vidioc_try_fmt_vid_out_mplane = vdec_try_fmt,
+	.vidioc_g_selection = vdec_g_selection,
+	.vidioc_reqbufs = vdec_reqbufs,
+	.vidioc_querybuf = vdec_querybuf,
+	.vidioc_create_bufs = vdec_create_bufs,
+	.vidioc_prepare_buf = vdec_prepare_buf,
+	.vidioc_qbuf = vdec_qbuf,
+	.vidioc_expbuf = vdec_exportbuf,
+	.vidioc_dqbuf = vdec_dqbuf,
+	.vidioc_streamon = vdec_streamon,
+	.vidioc_streamoff = vdec_streamoff,
+	.vidioc_s_parm = vdec_s_parm,
+	.vidioc_g_parm = vdec_g_parm,
+	.vidioc_enum_framesizes = vdec_enum_framesizes,
+	.vidioc_enum_frameintervals = vdec_enum_frameintervals,
+	.vidioc_subscribe_event = vdec_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int vdec_init_session(struct vidc_inst *inst)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	u32 pixfmt = inst->fmt_out->pixfmt;
+	struct hfi_framesize fs;
+	u32 ptype;
+	int ret;
+
+	ret = vidc_hfi_session_init(hfi, inst->hfi_inst, pixfmt,
+				    VIDC_SESSION_TYPE_DEC);
+	if (ret)
+		return ret;
+
+	ptype = HFI_PROPERTY_PARAM_FRAME_SIZE;
+	fs.buffer_type = HFI_BUFFER_INPUT;
+	fs.width = inst->out_width;
+	fs.height = inst->out_height;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
+	if (ret)
+		goto err;
+
+	fs.buffer_type = HFI_BUFFER_OUTPUT;
+	fs.width = inst->width;
+	fs.height = inst->height;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
+	if (ret)
+		goto err;
+
+	pixfmt = inst->fmt_cap->pixfmt;
+
+	ret = vidc_set_color_format(inst, HFI_BUFFER_OUTPUT, pixfmt);
+	if (ret)
+		goto err;
+
+	return 0;
+err:
+	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
+	return ret;
+}
+
+static int vdec_cap_num_buffers(struct vidc_inst *inst,
+				struct hfi_buffer_requirements *bufreq)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct device *dev = inst->core->dev;
+	int ret, ret2;
+
+	ret = pm_runtime_get_sync(dev);
+	if (ret < 0)
+		return ret;
+
+	ret = vdec_init_session(inst);
+	if (ret)
+		goto put_sync;
+
+	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, bufreq);
+
+	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
+
+put_sync:
+	ret2 = pm_runtime_put_sync(dev);
+
+	return ret ? ret : ret2;
+}
+
+static int vdec_queue_setup(struct vb2_queue *q,
+			    unsigned int *num_buffers, unsigned int *num_planes,
+			    unsigned int sizes[], struct device *alloc_devs[])
+{
+	struct vidc_inst *inst = vb2_get_drv_priv(q);
+	struct hfi_buffer_requirements bufreq;
+	unsigned int p;
+	int ret = 0;
+	u32 mbs;
+
+	switch (q->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		*num_planes = inst->fmt_out->num_planes;
+
+		*num_buffers = clamp_val(*num_buffers, 4, VIDEO_MAX_FRAME);
+
+		mbs = inst->out_width * inst->out_height /
+				MACROBLOCKS_PER_PIXEL;
+		for (p = 0; p < *num_planes; p++) {
+			sizes[p] = get_framesize_compressed(mbs);
+			alloc_devs[p] = inst->core->dev;
+		}
+
+		inst->num_input_bufs = *num_buffers;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		*num_planes = inst->fmt_cap->num_planes;
+
+		ret = vdec_cap_num_buffers(inst, &bufreq);
+		if (ret)
+			break;
+
+		*num_buffers = max(*num_buffers, bufreq.count_actual);
+
+		for (p = 0; p < *num_planes; p++) {
+			sizes[p] = get_framesize_nv12(p, inst->height,
+						      inst->width);
+			alloc_devs[p] = inst->core->dev;
+		}
+
+		inst->num_output_bufs = *num_buffers;
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static int vdec_check_configuration(struct vidc_inst *inst)
+{
+	struct hfi_buffer_requirements bufreq;
+	int ret;
+
+	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
+	if (ret)
+		return ret;
+
+	if (inst->num_output_bufs < bufreq.count_actual ||
+	    inst->num_output_bufs < bufreq.count_min)
+		return -EINVAL;
+
+	ret = vidc_buf_descs(inst, HFI_BUFFER_INPUT, &bufreq);
+	if (ret)
+		return ret;
+
+	if (inst->num_input_bufs < bufreq.count_min)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+	struct vidc_inst *inst = vb2_get_drv_priv(q);
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct device *dev = inst->core->dev;
+	struct hfi_buffer_requirements bufreq;
+	struct hfi_buffer_count_actual buf_count;
+	struct vb2_queue *queue;
+	u32 ptype;
+	int ret;
+
+	switch (q->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		queue = &inst->bufq_cap;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		queue = &inst->bufq_out;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	if (!vb2_is_streaming(queue))
+		return 0;
+
+	inst->in_reconfig = false;
+	inst->sequence = 0;
+
+	ret = pm_runtime_get_sync(dev);
+	if (ret < 0)
+		return ret;
+
+	ret = vdec_init_session(inst);
+	if (ret)
+		goto put_sync;
+
+	ret = vdec_set_properties(inst);
+	if (ret)
+		goto deinit_sess;
+
+	ret = vdec_check_configuration(inst);
+	if (ret)
+		goto deinit_sess;
+
+	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
+	buf_count.type = HFI_BUFFER_INPUT;
+	buf_count.count_actual = inst->num_input_bufs;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
+					    ptype, &buf_count);
+	if (ret)
+		goto deinit_sess;
+
+	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
+	if (ret)
+		goto deinit_sess;
+
+	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
+	buf_count.type = HFI_BUFFER_OUTPUT;
+	buf_count.count_actual = inst->num_output_bufs;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
+					    ptype, &buf_count);
+	if (ret)
+		goto deinit_sess;
+
+	if (inst->num_output_bufs != bufreq.count_actual) {
+		struct hfi_buffer_display_hold_count_actual display;
+
+		ptype = HFI_PROPERTY_PARAM_BUFFER_DISPLAY_HOLD_COUNT_ACTUAL;
+		display.type = HFI_BUFFER_OUTPUT;
+		display.hold_count = inst->num_output_bufs -
+				     bufreq.count_actual;
+
+		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
+						    ptype, &display);
+		if (ret)
+			goto deinit_sess;
+	}
+
+	ret = vidc_vb2_start_streaming(inst);
+	if (ret)
+		goto deinit_sess;
+
+	return 0;
+
+deinit_sess:
+	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
+put_sync:
+	pm_runtime_put_sync(dev);
+	return ret;
+}
+
+static const struct vb2_ops vdec_vb2_ops = {
+	.queue_setup = vdec_queue_setup,
+	.buf_init = vidc_vb2_buf_init,
+	.buf_prepare = vidc_vb2_buf_prepare,
+	.start_streaming = vdec_start_streaming,
+	.stop_streaming = vidc_vb2_stop_streaming,
+	.buf_queue = vidc_vb2_buf_queue,
+};
+
+static int vdec_empty_buf_done(struct hfi_inst *hfi_inst, u32 addr,
+			       u32 bytesused, u32 data_offset, u32 flags)
+{
+	struct vidc_inst *inst = hfi_inst->ops_priv;
+	struct vb2_v4l2_buffer *vbuf;
+	struct vb2_buffer *vb;
+
+	vbuf = vidc_vb2_find_buf(inst, addr);
+	if (!vbuf)
+		return -EINVAL;
+
+	vb = &vbuf->vb2_buf;
+	vbuf->flags = flags;
+
+	vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+
+	return 0;
+}
+
+static int vdec_fill_buf_done(struct hfi_inst *hfi_inst, u32 addr,
+			      u32 bytesused, u32 data_offset, u32 flags,
+			      struct timeval *timestamp)
+{
+	struct vidc_inst *inst = hfi_inst->ops_priv;
+	struct vb2_v4l2_buffer *vbuf;
+	struct vb2_buffer *vb;
+
+	vbuf = vidc_vb2_find_buf(inst, addr);
+	if (!vbuf)
+		return -EINVAL;
+
+	vb = &vbuf->vb2_buf;
+	vb->planes[0].bytesused = bytesused;
+	vb->planes[0].data_offset = data_offset;
+	vb->timestamp = timeval_to_ns(timestamp);
+	vbuf->flags = flags;
+	vbuf->sequence = inst->sequence++;
+
+	vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+
+	if (vbuf->flags & V4L2_BUF_FLAG_LAST) {
+		const struct v4l2_event ev = {
+			.type = V4L2_EVENT_EOS
+		};
+
+		v4l2_event_queue_fh(&inst->fh, &ev);
+	}
+
+	return 0;
+}
+
+static int vdec_event_notify(struct hfi_inst *hfi_inst, u32 event,
+			     struct hfi_event_data *data)
+{
+	struct vidc_inst *inst = hfi_inst->ops_priv;
+	struct device *dev = inst->core->dev;
+	const struct v4l2_event ev = { .type = V4L2_EVENT_SOURCE_CHANGE };
+
+	switch (event) {
+	case EVT_SESSION_ERROR:
+		if (hfi_inst) {
+			mutex_lock(&hfi_inst->lock);
+			inst->hfi_inst->state = INST_INVALID;
+			mutex_unlock(&hfi_inst->lock);
+		}
+		dev_err(dev, "dec: event session error (inst:%p)\n", hfi_inst);
+		break;
+	case EVT_SYS_EVENT_CHANGE:
+		switch (data->event_type) {
+		case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES:
+			dev_dbg(dev, "event sufficient resources\n");
+			break;
+		case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES:
+			inst->reconfig_height = data->height;
+			inst->reconfig_width = data->width;
+			inst->in_reconfig = true;
+
+			v4l2_event_queue_fh(&inst->fh, &ev);
+
+			dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
+				data->width, data->height);
+			break;
+		default:
+			break;
+		}
+		break;
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+static const struct hfi_inst_ops vdec_hfi_ops = {
+	.empty_buf_done = vdec_empty_buf_done,
+	.fill_buf_done = vdec_fill_buf_done,
+	.event_notify = vdec_event_notify,
+};
+
+static void vdec_inst_init(struct vidc_inst *inst)
+{
+	struct hfi_inst *hfi_inst = inst->hfi_inst;
+
+	inst->fmt_out = &vdec_formats[6];
+	inst->fmt_cap = &vdec_formats[0];
+	inst->width = 1280;
+	inst->height = ALIGN(720, 32);
+	inst->out_width = 1280;
+	inst->out_height = 720;
+	inst->fps = 30;
+	inst->timeperframe.numerator = 1;
+	inst->timeperframe.denominator = 30;
+
+	hfi_inst->width.min = 64;
+	hfi_inst->width.max = 1920;
+	hfi_inst->width.step_size = 1;
+	hfi_inst->height.min = 64;
+	hfi_inst->height.max = ALIGN(1080, 32);
+	hfi_inst->height.step_size = 1;
+	hfi_inst->framerate.min = 1;
+	hfi_inst->framerate.max = 30;
+	hfi_inst->framerate.step_size = 1;
+	hfi_inst->mbs_per_frame.min = 16;
+	hfi_inst->mbs_per_frame.max = 8160;
+}
+
+int vdec_init(struct vidc_core *core, struct video_device *dec)
+{
+	int ret;
+
+	dec->release = video_device_release_empty;
+	dec->fops = &vidc_fops;
+	dec->ioctl_ops = &vdec_ioctl_ops;
+	dec->vfl_dir = VFL_DIR_M2M;
+	dec->v4l2_dev = &core->v4l2_dev;
+
+	ret = video_register_device(dec, VFL_TYPE_GRABBER, -1);
+	if (ret)
+		return ret;
+
+	video_set_drvdata(dec, core);
+
+	return 0;
+}
+
+void vdec_deinit(struct vidc_core *core, struct video_device *dec)
+{
+	video_unregister_device(dec);
+}
+
+int vdec_open(struct vidc_inst *inst)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct vb2_queue *q;
+	int ret;
+
+	ret = vdec_ctrl_init(inst);
+	if (ret)
+		return ret;
+
+	inst->hfi_inst = vidc_hfi_session_create(hfi, &vdec_hfi_ops, inst);
+	if (IS_ERR(inst->hfi_inst)) {
+		ret = PTR_ERR(inst->hfi_inst);
+		goto err_ctrl_deinit;
+	}
+
+	vdec_inst_init(inst);
+
+	q = &inst->bufq_cap;
+	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	q->io_modes = VB2_MMAP | VB2_DMABUF;
+	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	q->ops = &vdec_vb2_ops;
+	q->mem_ops = &vb2_dma_sg_memops;
+	q->drv_priv = inst;
+	q->buf_struct_size = sizeof(struct vidc_buffer);
+	q->allow_zero_bytesused = 1;
+	ret = vb2_queue_init(q);
+	if (ret)
+		goto err_session_destroy;
+
+	q = &inst->bufq_out;
+	q->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	q->io_modes = VB2_MMAP | VB2_DMABUF;
+	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	q->ops = &vdec_vb2_ops;
+	q->mem_ops = &vb2_dma_sg_memops;
+	q->drv_priv = inst;
+	q->buf_struct_size = sizeof(struct vidc_buffer);
+	q->allow_zero_bytesused = 1;
+	ret = vb2_queue_init(q);
+	if (ret)
+		goto err_cap_queue_release;
+
+	return 0;
+
+err_cap_queue_release:
+	vb2_queue_release(&inst->bufq_cap);
+err_session_destroy:
+	vidc_hfi_session_destroy(hfi, inst->hfi_inst);
+	inst->hfi_inst = NULL;
+err_ctrl_deinit:
+	vdec_ctrl_deinit(inst);
+	return ret;
+}
+
+void vdec_close(struct vidc_inst *inst)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+
+	vb2_queue_release(&inst->bufq_out);
+	vb2_queue_release(&inst->bufq_cap);
+	vdec_ctrl_deinit(inst);
+	vidc_hfi_session_destroy(hfi, inst->hfi_inst);
+}
diff --git a/drivers/media/platform/qcom/vidc/vdec.h b/drivers/media/platform/qcom/vidc/vdec.h
new file mode 100644
index 000000000000..f67de6d73f85
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/vdec.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_VDEC_H__
+#define __VIDC_VDEC_H__
+
+struct vidc_core;
+struct video_device;
+struct vidc_inst;
+
+int vdec_init(struct vidc_core *core, struct video_device *dec);
+void vdec_deinit(struct vidc_core *core, struct video_device *dec);
+int vdec_open(struct vidc_inst *inst);
+void vdec_close(struct vidc_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/vdec_ctrls.c b/drivers/media/platform/qcom/vidc/vdec_ctrls.c
new file mode 100644
index 000000000000..59225d8f1fd9
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/vdec_ctrls.c
@@ -0,0 +1,200 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/types.h>
+#include <media/v4l2-ctrls.h>
+
+#include "core.h"
+
+static struct vidc_ctrl vdec_ctrls[] = {
+	{
+		.id = V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
+		.max = V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_CODING_EFFICIENCY,
+		.def = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
+		.flags = V4L2_CTRL_FLAG_VOLATILE,
+		.menu_skip_mask = ~(
+			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE) |
+			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE)
+		),
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
+		.max = V4L2_MPEG_VIDEO_MPEG4_LEVEL_5,
+		.def = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
+		.flags = V4L2_CTRL_FLAG_VOLATILE,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+		.max = V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH,
+		.def = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+		.flags = V4L2_CTRL_FLAG_VOLATILE,
+		.menu_skip_mask = ~(
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH)
+		),
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+		.max = V4L2_MPEG_VIDEO_H264_LEVEL_5_1,
+		.def = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+		.flags = V4L2_CTRL_FLAG_VOLATILE,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_VPX_PROFILE,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 0,
+		.max = 3,
+		.step = 1,
+		.def = 0,
+		.flags = V4L2_CTRL_FLAG_VOLATILE,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER,
+		.type = V4L2_CTRL_TYPE_BOOLEAN,
+		.def = 0,
+	},
+};
+
+#define NUM_CTRLS	ARRAY_SIZE(vdec_ctrls)
+
+static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct vidc_inst *inst = ctrl_to_inst(ctrl);
+	struct vdec_controls *ctr = &inst->controls.dec;
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
+		ctr->post_loop_deb_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+		ctr->profile = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+		ctr->level = ctrl->val;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct vidc_inst *inst = ctrl_to_inst(ctrl);
+	struct vdec_controls *ctr = &inst->controls.dec;
+	struct hfi_core *hfi = &inst->core->hfi;
+	union hfi_get_property hprop;
+	u32 ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
+	int ret;
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
+						    &hprop);
+		if (!ret)
+			ctr->profile = hprop.profile_level.profile;
+		ctrl->val = ctr->profile;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
+						    &hprop);
+		if (!ret)
+			ctr->level = hprop.profile_level.level;
+		ctrl->val = ctr->level;
+		break;
+	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
+		ctrl->val = ctr->post_loop_deb_mode;
+		break;
+	default:
+		return -EINVAL;
+	};
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops vdec_ctrl_ops = {
+	.s_ctrl = vdec_op_s_ctrl,
+	.g_volatile_ctrl = vdec_op_g_volatile_ctrl,
+};
+
+int vdec_ctrl_init(struct vidc_inst *inst)
+{
+	unsigned int i;
+	int ret;
+
+	ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, NUM_CTRLS);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < NUM_CTRLS; i++) {
+		struct v4l2_ctrl *ctrl;
+
+		if (vdec_ctrls[i].type == V4L2_CTRL_TYPE_MENU) {
+			ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
+					&vdec_ctrl_ops,
+					vdec_ctrls[i].id,
+					vdec_ctrls[i].max,
+					vdec_ctrls[i].menu_skip_mask,
+					vdec_ctrls[i].def);
+		} else {
+			ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler,
+					&vdec_ctrl_ops,
+					vdec_ctrls[i].id,
+					vdec_ctrls[i].min,
+					vdec_ctrls[i].max,
+					vdec_ctrls[i].step,
+					vdec_ctrls[i].def);
+		}
+
+		if (!ctrl)
+			return -EINVAL;
+
+		switch (vdec_ctrls[i].id) {
+		case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+		case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+		case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+		case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+		case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+			ctrl->flags |= vdec_ctrls[i].flags;
+			break;
+		}
+
+		ret = inst->ctrl_handler.error;
+		if (ret) {
+			v4l2_ctrl_handler_free(&inst->ctrl_handler);
+			return ret;
+		}
+	}
+
+	return ret;
+}
+
+void vdec_ctrl_deinit(struct vidc_inst *inst)
+{
+	v4l2_ctrl_handler_free(&inst->ctrl_handler);
+}
diff --git a/drivers/media/platform/qcom/vidc/vdec_ctrls.h b/drivers/media/platform/qcom/vidc/vdec_ctrls.h
new file mode 100644
index 000000000000..18f14d06deed
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/vdec_ctrls.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (c) 2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_VDEC_CTRLS_H__
+#define __VIDC_VDEC_CTRLS_H__
+
+int vdec_ctrl_init(struct vidc_inst *inst);
+void vdec_ctrl_deinit(struct vidc_inst *inst);
+
+#endif
-- 
2.7.4

[PATCH 4/8] media: vidc: encoder: add video encoder files

[Stanimir Varbanov]

This adds encoder part of the driver plus encoder controls.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/vidc/venc.c       | 1261 +++++++++++++++++++++++++
 drivers/media/platform/qcom/vidc/venc.h       |   27 +
 drivers/media/platform/qcom/vidc/venc_ctrls.c |  396 ++++++++
 drivers/media/platform/qcom/vidc/venc_ctrls.h |   23 +
 4 files changed, 1707 insertions(+)
 create mode 100644 drivers/media/platform/qcom/vidc/venc.c
 create mode 100644 drivers/media/platform/qcom/vidc/venc.h
 create mode 100644 drivers/media/platform/qcom/vidc/venc_ctrls.c
 create mode 100644 drivers/media/platform/qcom/vidc/venc_ctrls.h

diff --git a/drivers/media/platform/qcom/vidc/venc.c b/drivers/media/platform/qcom/vidc/venc.c
new file mode 100644
index 000000000000..bc44f419089e
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/venc.c
@@ -0,0 +1,1261 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/slab.h>
+#include <linux/pm_runtime.h>
+#include <media/videobuf2-dma-sg.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ctrls.h>
+
+#include "core.h"
+#include "helpers.h"
+#include "load.h"
+#include "venc_ctrls.h"
+
+#define NUM_B_FRAMES_MAX	4
+
+static u32 get_framesize_nv12(int plane, u32 height, u32 width)
+{
+	u32 y_stride, uv_stride, y_plane;
+	u32 y_sclines, uv_sclines, uv_plane;
+	u32 size;
+
+	y_stride = ALIGN(width, 128);
+	uv_stride = ALIGN(width, 128);
+	y_sclines = ALIGN(height, 32);
+	uv_sclines = ALIGN(((height + 1) >> 1), 16);
+
+	y_plane = y_stride * y_sclines;
+	uv_plane = uv_stride * uv_sclines + SZ_4K;
+	size = y_plane + uv_plane + SZ_8K;
+	size = ALIGN(size, SZ_4K);
+
+	return size;
+}
+
+static u32 get_framesize_compressed(u32 height, u32 width)
+{
+	u32 sz = ALIGN(height, 32) * ALIGN(width, 32) * 3 / 2;
+
+	return ALIGN(sz, SZ_4K);
+}
+
+static const struct vidc_format venc_formats[] = {
+	{
+		.pixfmt = V4L2_PIX_FMT_NV12,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_MPEG4,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_H263,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_VP8,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+	},
+};
+
+static const struct vidc_format *find_format(u32 pixfmt, int type)
+{
+	const struct vidc_format *fmt = venc_formats;
+	unsigned int size = ARRAY_SIZE(venc_formats);
+	unsigned int i;
+
+	for (i = 0; i < size; i++) {
+		if (fmt[i].pixfmt == pixfmt)
+			break;
+	}
+
+	if (i == size || fmt[i].type != type)
+		return NULL;
+
+	return &fmt[i];
+}
+
+static const struct vidc_format *find_format_by_index(int index, int type)
+{
+	const struct vidc_format *fmt = venc_formats;
+	unsigned int size = ARRAY_SIZE(venc_formats);
+	int i, k = 0;
+
+	if (index < 0 || index > size)
+		return NULL;
+
+	for (i = 0; i < size; i++) {
+		if (fmt[i].type != type)
+			continue;
+		if (k == index)
+			break;
+		k++;
+	}
+
+	if (i == size)
+		return NULL;
+
+	return &fmt[i];
+}
+
+static int venc_v4l2_to_hfi(int id, int value)
+{
+	switch (id) {
+	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+		switch (value) {
+		case V4L2_MPEG_VIDEO_MPEG4_LEVEL_0:
+		default:
+			return HFI_MPEG4_LEVEL_0;
+		case V4L2_MPEG_VIDEO_MPEG4_LEVEL_0B:
+			return HFI_MPEG4_LEVEL_0b;
+		case V4L2_MPEG_VIDEO_MPEG4_LEVEL_1:
+			return HFI_MPEG4_LEVEL_1;
+		case V4L2_MPEG_VIDEO_MPEG4_LEVEL_2:
+			return HFI_MPEG4_LEVEL_2;
+		case V4L2_MPEG_VIDEO_MPEG4_LEVEL_3:
+			return HFI_MPEG4_LEVEL_3;
+		case V4L2_MPEG_VIDEO_MPEG4_LEVEL_4:
+			return HFI_MPEG4_LEVEL_4;
+		case V4L2_MPEG_VIDEO_MPEG4_LEVEL_5:
+			return HFI_MPEG4_LEVEL_5;
+		}
+	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+		switch (value) {
+		case V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE:
+		default:
+			return HFI_MPEG4_PROFILE_SIMPLE;
+		case V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE:
+			return HFI_MPEG4_PROFILE_ADVANCEDSIMPLE;
+		}
+	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+		switch (value) {
+		case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+			return HFI_H264_PROFILE_BASELINE;
+		case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE:
+			return HFI_H264_PROFILE_CONSTRAINED_BASE;
+		case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+			return HFI_H264_PROFILE_MAIN;
+		case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+		default:
+			return HFI_H264_PROFILE_HIGH;
+		}
+	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+		switch (value) {
+		case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+			return HFI_H264_LEVEL_1;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_1B:
+			return HFI_H264_LEVEL_1b;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+			return HFI_H264_LEVEL_11;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+			return HFI_H264_LEVEL_12;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+			return HFI_H264_LEVEL_13;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+			return HFI_H264_LEVEL_2;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+			return HFI_H264_LEVEL_21;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+			return HFI_H264_LEVEL_22;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+			return HFI_H264_LEVEL_3;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+			return HFI_H264_LEVEL_31;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+			return HFI_H264_LEVEL_32;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+			return HFI_H264_LEVEL_4;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+			return HFI_H264_LEVEL_41;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_4_2:
+			return HFI_H264_LEVEL_42;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_5_0:
+		default:
+			return HFI_H264_LEVEL_5;
+		case V4L2_MPEG_VIDEO_H264_LEVEL_5_1:
+			return HFI_H264_LEVEL_51;
+		}
+	case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
+		switch (value) {
+		case V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC:
+		default:
+			return HFI_H264_ENTROPY_CAVLC;
+		case V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC:
+			return HFI_H264_ENTROPY_CABAC;
+		}
+	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+		switch (value) {
+		case 0:
+		default:
+			return HFI_VPX_PROFILE_VERSION_0;
+		case 1:
+			return HFI_VPX_PROFILE_VERSION_1;
+		case 2:
+			return HFI_VPX_PROFILE_VERSION_2;
+		case 3:
+			return HFI_VPX_PROFILE_VERSION_3;
+		}
+	}
+
+	return 0;
+}
+
+static int venc_set_properties(struct vidc_inst *inst)
+{
+	struct venc_controls *ctr = &inst->controls.enc;
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct hfi_intra_period intra_period;
+	struct hfi_profile_level pl;
+	struct hfi_framerate frate;
+	struct hfi_bitrate brate;
+	struct hfi_idr_period idrp;
+	u32 ptype, rate_control, bitrate, profile = 0, level = 0;
+	int ret;
+
+	ptype = HFI_PROPERTY_CONFIG_FRAME_RATE;
+	frate.buffer_type = HFI_BUFFER_OUTPUT;
+	frate.framerate = inst->fps * (1 << 16);
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &frate);
+	if (ret)
+		return ret;
+
+	if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H264) {
+		struct hfi_h264_vui_timing_info info;
+
+		ptype = HFI_PROPERTY_PARAM_VENC_H264_VUI_TIMING_INFO;
+		info.enable = 1;
+		info.fixed_framerate = 1;
+		info.time_scale = NSEC_PER_SEC;
+
+		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype,
+						    &info);
+		if (ret)
+			return ret;
+	}
+
+	ptype = HFI_PROPERTY_CONFIG_VENC_IDR_PERIOD;
+	idrp.idr_period = ctr->gop_size;
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &idrp);
+	if (ret)
+		return ret;
+
+	if (ctr->num_b_frames) {
+		u32 max_num_b_frames = NUM_B_FRAMES_MAX;
+
+		ptype = HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES;
+		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
+						    ptype, &max_num_b_frames);
+		if (ret)
+			return ret;
+	}
+
+	/* intra_period = pframes + bframes + 1 */
+	if (!ctr->num_p_frames)
+		ctr->num_p_frames = 2 * 15 - 1,
+
+	ptype = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD;
+	intra_period.pframes = ctr->num_p_frames;
+	intra_period.bframes = ctr->num_b_frames;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype,
+					    &intra_period);
+	if (ret)
+		return ret;
+
+	if (ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR)
+		rate_control = HFI_RATE_CONTROL_VBR_CFR;
+	else
+		rate_control = HFI_RATE_CONTROL_CBR_CFR;
+
+	ptype = HFI_PROPERTY_PARAM_VENC_RATE_CONTROL;
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype,
+					    &rate_control);
+	if (ret)
+		return ret;
+
+	if (!ctr->bitrate)
+		bitrate = 64000;
+	else
+		bitrate = ctr->bitrate;
+
+	ptype = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE;
+	brate.bitrate = bitrate;
+	brate.layer_id = 0;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &brate);
+	if (ret)
+		return ret;
+
+	if (!ctr->bitrate_peak)
+		bitrate *= 2;
+	else
+		bitrate = ctr->bitrate_peak;
+
+	ptype = HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE;
+	brate.bitrate = bitrate;
+	brate.layer_id = 0;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &brate);
+	if (ret)
+		return ret;
+
+	if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H264) {
+		profile = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+					   ctr->profile);
+		level = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+					 ctr->level);
+	} else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_VP8) {
+		profile = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_VPX_PROFILE,
+					   ctr->profile);
+		level = 0;
+	} else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_MPEG4) {
+		profile = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
+					   ctr->profile);
+		level = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
+					 ctr->level);
+	} else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H263) {
+		profile = 0;
+		level = 0;
+	}
+
+	ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
+	pl.profile = profile;
+	pl.level = level;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &pl);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int
+venc_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
+{
+	strlcpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
+	strlcpy(cap->card, "video encoder", sizeof(cap->card));
+	strlcpy(cap->bus_info, "platform:vidc", sizeof(cap->bus_info));
+
+	cap->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+
+	return 0;
+}
+
+static int venc_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+	const struct vidc_format *fmt;
+
+	fmt = find_format_by_index(f->index, f->type);
+
+	memset(f->reserved, 0, sizeof(f->reserved));
+
+	if (!fmt)
+		return -EINVAL;
+
+	f->pixelformat = fmt->pixfmt;
+
+	return 0;
+}
+
+static const struct vidc_format *
+venc_try_fmt_common(struct vidc_inst *inst, struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
+	struct hfi_inst *hfi_inst = inst->hfi_inst;
+	const struct vidc_format *fmt;
+	unsigned int p;
+
+	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
+	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+
+	fmt = find_format(pixmp->pixelformat, f->type);
+	if (!fmt) {
+		if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+			pixmp->pixelformat = V4L2_PIX_FMT_H264;
+		else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+			pixmp->pixelformat = V4L2_PIX_FMT_NV12;
+		else
+			return NULL;
+		fmt = find_format(pixmp->pixelformat, f->type);
+		pixmp->width = 1280;
+		pixmp->height = 720;
+	}
+
+	pixmp->height = ALIGN(pixmp->height, 32);
+
+	pixmp->width = clamp(pixmp->width, hfi_inst->width.min,
+			     hfi_inst->width.max);
+	pixmp->height = clamp(pixmp->height, hfi_inst->height.min,
+			      hfi_inst->height.max);
+	if (pixmp->field == V4L2_FIELD_ANY)
+		pixmp->field = V4L2_FIELD_NONE;
+	pixmp->num_planes = fmt->num_planes;
+	pixmp->flags = 0;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		for (p = 0; p < pixmp->num_planes; p++) {
+			pfmt[p].sizeimage = get_framesize_nv12(p, pixmp->height,
+							       pixmp->width);
+
+			pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
+		}
+	} else {
+		pfmt[0].sizeimage = get_framesize_compressed(pixmp->height,
+							     pixmp->width);
+		pfmt[0].bytesperline = 0;
+	}
+
+	return fmt;
+}
+
+static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct vidc_inst *inst = to_inst(file);
+	const struct vidc_format *fmt;
+
+	fmt = venc_try_fmt_common(inst, f);
+	if (!fmt)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct vidc_inst *inst = to_inst(file);
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+	struct v4l2_pix_format_mplane orig_pixmp;
+	const struct vidc_format *fmt;
+	struct v4l2_format format;
+	u32 pixfmt_out = 0, pixfmt_cap = 0;
+
+	orig_pixmp = *pixmp;
+
+	fmt = venc_try_fmt_common(inst, f);
+	if (!fmt)
+		return -EINVAL;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		pixfmt_out = pixmp->pixelformat;
+		pixfmt_cap = inst->fmt_cap->pixfmt;
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		pixfmt_cap = pixmp->pixelformat;
+		pixfmt_out = inst->fmt_out->pixfmt;
+	}
+
+	memset(&format, 0, sizeof(format));
+
+	format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	format.fmt.pix_mp.pixelformat = pixfmt_out;
+	format.fmt.pix_mp.width = orig_pixmp.width;
+	format.fmt.pix_mp.height = orig_pixmp.height;
+	venc_try_fmt_common(inst, &format);
+	inst->out_width = format.fmt.pix_mp.width;
+	inst->out_height = format.fmt.pix_mp.height;
+	inst->colorspace = pixmp->colorspace;
+	inst->ycbcr_enc = pixmp->ycbcr_enc;
+	inst->quantization = pixmp->quantization;
+	inst->xfer_func = pixmp->xfer_func;
+
+	memset(&format, 0, sizeof(format));
+
+	format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	format.fmt.pix_mp.pixelformat = pixfmt_cap;
+	format.fmt.pix_mp.width = orig_pixmp.width;
+	format.fmt.pix_mp.height = orig_pixmp.height;
+	venc_try_fmt_common(inst, &format);
+	inst->width = format.fmt.pix_mp.width;
+	inst->height = format.fmt.pix_mp.height;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		inst->fmt_out = fmt;
+	else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		inst->fmt_cap = fmt;
+
+	return 0;
+}
+
+static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+	struct vidc_inst *inst = to_inst(file);
+	const struct vidc_format *fmt;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		fmt = inst->fmt_cap;
+	else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		fmt = inst->fmt_out;
+	else
+		return -EINVAL;
+
+	pixmp->pixelformat = fmt->pixfmt;
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		pixmp->width = inst->width;
+		pixmp->height = inst->height;
+		pixmp->colorspace = inst->colorspace;
+		pixmp->ycbcr_enc = inst->ycbcr_enc;
+		pixmp->quantization = inst->quantization;
+		pixmp->xfer_func = inst->xfer_func;
+	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		pixmp->width = inst->out_width;
+		pixmp->height = inst->out_height;
+	}
+
+	venc_try_fmt_common(inst, f);
+
+	return 0;
+}
+
+static int venc_g_selection(struct file *file, void *fh,
+			    struct v4l2_selection *s)
+{
+	struct vidc_inst *inst = to_inst(file);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+		s->r.top = 0;
+		s->r.left = 0;
+		s->r.width = inst->width;
+		s->r.height = inst->height;
+		break;
+	case V4L2_SEL_TGT_CROP:
+		s->r.top = 0;
+		s->r.left = 0;
+		s->r.width = inst->out_width;
+		s->r.height = inst->out_height;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int venc_s_selection(struct file *file, void *fh,
+			    struct v4l2_selection *s)
+{
+	return -EINVAL;
+}
+
+static int
+venc_reqbufs(struct file *file, void *fh, struct v4l2_requestbuffers *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	if (!b->count)
+		vb2_core_queue_release(queue);
+
+	return vb2_reqbufs(queue, b);
+}
+
+static int venc_querybuf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+	unsigned int p;
+	int ret;
+
+	ret = vb2_querybuf(queue, b);
+	if (ret)
+		return ret;
+
+	if (b->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    b->memory == V4L2_MEMORY_MMAP) {
+		for (p = 0; p < b->length; p++)
+			b->m.planes[p].m.mem_offset += DST_QUEUE_OFF_BASE;
+	}
+
+	return 0;
+}
+
+static int venc_create_bufs(struct file *file, void *fh,
+			    struct v4l2_create_buffers *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->format.type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_create_bufs(queue, b);
+}
+
+static int venc_prepare_buf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_prepare_buf(queue, b);
+}
+
+static int venc_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_qbuf(queue, b);
+}
+
+static int
+venc_exportbuf(struct file *file, void *fh, struct v4l2_exportbuffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_expbuf(queue, b);
+}
+
+static int venc_dqbuf(struct file *file, void *fh, struct v4l2_buffer *b)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_dqbuf(queue, b, file->f_flags & O_NONBLOCK);
+}
+
+static int venc_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_streamon(queue, type);
+}
+
+static int venc_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
+{
+	struct vb2_queue *queue = vidc_to_vb2q(file, type);
+
+	if (!queue)
+		return -EINVAL;
+
+	return vb2_streamoff(queue, type);
+}
+
+static int venc_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+	struct vidc_inst *inst = to_inst(file);
+	struct v4l2_outputparm *out = &a->parm.output;
+	struct v4l2_fract *timeperframe = &out->timeperframe;
+	u64 us_per_frame, fps;
+
+	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		return -EINVAL;
+
+	memset(out->reserved, 0, sizeof(out->reserved));
+
+	if (!timeperframe->denominator)
+		timeperframe->denominator = inst->timeperframe.denominator;
+	if (!timeperframe->numerator)
+		timeperframe->numerator = inst->timeperframe.numerator;
+
+	out->capability = V4L2_CAP_TIMEPERFRAME;
+
+	us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
+	do_div(us_per_frame, timeperframe->denominator);
+
+	if (!us_per_frame)
+		return -EINVAL;
+
+	fps = (u64)USEC_PER_SEC;
+	do_div(fps, us_per_frame);
+
+	inst->timeperframe = *timeperframe;
+	inst->fps = fps;
+
+	return 0;
+}
+
+static int venc_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+	struct vidc_inst *inst = to_inst(file);
+
+	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		return -EINVAL;
+
+	a->parm.output.capability |= V4L2_CAP_TIMEPERFRAME;
+	a->parm.output.timeperframe = inst->timeperframe;
+
+	return 0;
+}
+
+static int venc_enum_framesizes(struct file *file, void *fh,
+				struct v4l2_frmsizeenum *fsize)
+{
+	struct hfi_inst *inst = to_hfi_inst(file);
+	const struct vidc_format *fmt;
+
+	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+
+	fmt = find_format(fsize->pixel_format,
+			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (!fmt) {
+		fmt = find_format(fsize->pixel_format,
+				  V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+		if (!fmt)
+			return -EINVAL;
+	}
+
+	if (fsize->index)
+		return -EINVAL;
+
+	fsize->stepwise.min_width = inst->width.min;
+	fsize->stepwise.max_width = inst->width.max;
+	fsize->stepwise.step_width = inst->width.step_size;
+	fsize->stepwise.min_height = inst->height.min;
+	fsize->stepwise.max_height = inst->height.max;
+	fsize->stepwise.step_height = inst->height.step_size;
+
+	return 0;
+}
+
+static int venc_enum_frameintervals(struct file *file, void *fh,
+				    struct v4l2_frmivalenum *fival)
+{
+	struct hfi_inst *inst = to_hfi_inst(file);
+	const struct vidc_format *fmt;
+
+	fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
+
+	fmt = find_format(fival->pixel_format,
+			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (!fmt) {
+		fmt = find_format(fival->pixel_format,
+				  V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+		if (!fmt)
+			return -EINVAL;
+	}
+
+	if (fival->index)
+		return -EINVAL;
+
+	if (!fival->width || !fival->height)
+		return -EINVAL;
+
+	if (fival->width > inst->width.max ||
+	    fival->width < inst->width.min ||
+	    fival->height > inst->height.max ||
+	    fival->height < inst->height.min)
+		return -EINVAL;
+
+	fival->stepwise.min.numerator = 1;
+	fival->stepwise.min.denominator = inst->framerate.max;
+	fival->stepwise.max.numerator = 1;
+	fival->stepwise.max.denominator = inst->framerate.min;
+	fival->stepwise.step.numerator = 1;
+	fival->stepwise.step.denominator = inst->framerate.max;
+
+	return 0;
+}
+
+static int venc_subscribe_event(struct v4l2_fh *fh,
+				const struct  v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_EOS:
+		return v4l2_event_subscribe(fh, sub, 2, NULL);
+	case V4L2_EVENT_SOURCE_CHANGE:
+		return v4l2_src_change_event_subscribe(fh, sub);
+	case V4L2_EVENT_CTRL:
+		return v4l2_ctrl_subscribe_event(fh, sub);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ioctl_ops venc_ioctl_ops = {
+	.vidioc_querycap = venc_querycap,
+	.vidioc_enum_fmt_vid_cap_mplane = venc_enum_fmt,
+	.vidioc_enum_fmt_vid_out_mplane = venc_enum_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = venc_s_fmt,
+	.vidioc_s_fmt_vid_out_mplane = venc_s_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = venc_g_fmt,
+	.vidioc_g_fmt_vid_out_mplane = venc_g_fmt,
+	.vidioc_try_fmt_vid_cap_mplane = venc_try_fmt,
+	.vidioc_try_fmt_vid_out_mplane = venc_try_fmt,
+	.vidioc_g_selection = venc_g_selection,
+	.vidioc_s_selection = venc_s_selection,
+	.vidioc_reqbufs = venc_reqbufs,
+	.vidioc_querybuf = venc_querybuf,
+	.vidioc_create_bufs = venc_create_bufs,
+	.vidioc_prepare_buf = venc_prepare_buf,
+	.vidioc_qbuf = venc_qbuf,
+	.vidioc_expbuf = venc_exportbuf,
+	.vidioc_dqbuf = venc_dqbuf,
+	.vidioc_streamon = venc_streamon,
+	.vidioc_streamoff = venc_streamoff,
+	.vidioc_s_parm = venc_s_parm,
+	.vidioc_g_parm = venc_g_parm,
+	.vidioc_enum_framesizes = venc_enum_framesizes,
+	.vidioc_enum_frameintervals = venc_enum_frameintervals,
+	.vidioc_subscribe_event = venc_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int venc_init_session(struct vidc_inst *inst)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	u32 pixfmt = inst->fmt_cap->pixfmt;
+	struct hfi_framesize fs;
+	u32 ptype;
+	int ret;
+
+	ret = vidc_hfi_session_init(hfi, inst->hfi_inst, pixfmt,
+				    VIDC_SESSION_TYPE_ENC);
+	if (ret)
+		return ret;
+
+	ptype = HFI_PROPERTY_PARAM_FRAME_SIZE;
+	fs.buffer_type = HFI_BUFFER_INPUT;
+	fs.width = inst->out_width;
+	fs.height = inst->out_height;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
+	if (ret)
+		goto err;
+
+	fs.buffer_type = HFI_BUFFER_OUTPUT;
+	fs.width = inst->width;
+	fs.height = inst->height;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
+	if (ret)
+		goto err;
+
+	pixfmt = inst->fmt_out->pixfmt;
+
+	ret = vidc_set_color_format(inst, HFI_BUFFER_INPUT, pixfmt);
+	if (ret)
+		goto err;
+
+	return 0;
+err:
+	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
+	return ret;
+}
+
+static int venc_cap_num_buffers(struct vidc_inst *inst,
+				struct hfi_buffer_requirements *bufreq)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct device *dev = inst->core->dev;
+	int ret, ret2;
+
+	ret = pm_runtime_get_sync(dev);
+	if (ret < 0)
+		return ret;
+
+	ret = venc_init_session(inst);
+	if (ret)
+		goto put_sync;
+
+	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, bufreq);
+
+	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
+
+put_sync:
+	ret2 = pm_runtime_put_sync(dev);
+
+	return ret ? ret : ret2;
+}
+
+static int venc_queue_setup(struct vb2_queue *q,
+			    unsigned int *num_buffers, unsigned int *num_planes,
+			    unsigned int sizes[], struct device *alloc_devs[])
+{
+	struct vidc_inst *inst = vb2_get_drv_priv(q);
+	struct hfi_buffer_requirements bufreq;
+	unsigned int p;
+	int ret = 0;
+
+	switch (q->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		*num_planes = inst->fmt_out->num_planes;
+		*num_buffers = clamp_val(*num_buffers, 5, VIDEO_MAX_FRAME);
+		inst->num_input_bufs = *num_buffers;
+
+		for (p = 0; p < *num_planes; ++p) {
+			sizes[p] = get_framesize_nv12(p, inst->height,
+						      inst->width);
+			alloc_devs[p] = inst->core->dev;
+		}
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		*num_planes = inst->fmt_cap->num_planes;
+
+		ret = venc_cap_num_buffers(inst, &bufreq);
+		if (ret)
+			break;
+
+		*num_buffers = max(*num_buffers, bufreq.count_actual);
+		inst->num_output_bufs = *num_buffers;
+
+		sizes[0] = get_framesize_compressed(inst->height, inst->width);
+		alloc_devs[0] = inst->core->dev;
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static int venc_check_configuration(struct vidc_inst *inst)
+{
+	struct hfi_buffer_requirements bufreq;
+	int ret;
+
+	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
+	if (ret)
+		return ret;
+
+	if (inst->num_output_bufs < bufreq.count_actual ||
+	    inst->num_output_bufs < bufreq.count_min)
+		return -EINVAL;
+
+	ret = vidc_buf_descs(inst, HFI_BUFFER_INPUT, &bufreq);
+	if (ret)
+		return ret;
+
+	if (inst->num_input_bufs < bufreq.count_actual ||
+	    inst->num_input_bufs < bufreq.count_min)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int venc_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+	struct vidc_inst *inst = vb2_get_drv_priv(q);
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct device *dev = inst->core->dev;
+	struct hfi_buffer_count_actual buf_count;
+	struct vb2_queue *queue;
+	u32 ptype;
+	int ret;
+
+	switch (q->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		queue = &inst->bufq_cap;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		queue = &inst->bufq_out;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	if (!vb2_is_streaming(queue))
+		return 0;
+
+	inst->sequence = 0;
+
+	ret = pm_runtime_get_sync(dev);
+	if (ret < 0)
+		return ret;
+
+	ret = venc_init_session(inst);
+	if (ret)
+		goto put_sync;
+
+	ret = venc_set_properties(inst);
+	if (ret)
+		goto deinit_sess;
+
+	ret = venc_check_configuration(inst);
+	if (ret)
+		goto deinit_sess;
+
+	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
+	buf_count.type = HFI_BUFFER_OUTPUT;
+	buf_count.count_actual = inst->num_output_bufs;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype,
+					    &buf_count);
+	if (ret)
+		goto deinit_sess;
+
+	buf_count.type = HFI_BUFFER_INPUT;
+	buf_count.count_actual = inst->num_input_bufs;
+
+	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype,
+					    &buf_count);
+	if (ret)
+		goto deinit_sess;
+
+	ret = vidc_vb2_start_streaming(inst);
+	if (ret)
+		goto deinit_sess;
+
+	return 0;
+
+deinit_sess:
+	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
+put_sync:
+	pm_runtime_put_sync(dev);
+	return ret;
+}
+
+static const struct vb2_ops venc_vb2_ops = {
+	.queue_setup = venc_queue_setup,
+	.buf_init = vidc_vb2_buf_init,
+	.buf_prepare = vidc_vb2_buf_prepare,
+	.start_streaming = venc_start_streaming,
+	.stop_streaming = vidc_vb2_stop_streaming,
+	.buf_queue = vidc_vb2_buf_queue,
+};
+
+static int venc_empty_buf_done(struct hfi_inst *hfi_inst, u32 addr,
+			       u32 bytesused, u32 data_offset, u32 flags)
+{
+	struct vidc_inst *inst = hfi_inst->ops_priv;
+	struct vb2_v4l2_buffer *vbuf;
+	enum vb2_buffer_state state;
+	struct vb2_buffer *vb;
+
+	vbuf = vidc_vb2_find_buf(inst, addr);
+	if (!vbuf)
+		return -EINVAL;
+
+	vb = &vbuf->vb2_buf;
+	vb->planes[0].bytesused = bytesused;
+	vb->planes[0].data_offset = data_offset;
+	vbuf->flags = flags;
+	state = VB2_BUF_STATE_DONE;
+
+	if (data_offset > vb->planes[0].length ||
+	    bytesused > vb->planes[0].length)
+		state = VB2_BUF_STATE_ERROR;
+
+	vb2_buffer_done(vb, state);
+
+	return 0;
+}
+
+static int venc_fill_buf_done(struct hfi_inst *hfi_inst, u32 addr,
+			      u32 bytesused, u32 data_offset, u32 flags,
+			      struct timeval *timestamp)
+{
+	struct vidc_inst *inst = hfi_inst->ops_priv;
+	struct vb2_v4l2_buffer *vbuf;
+	enum vb2_buffer_state state;
+	struct vb2_buffer *vb;
+
+	vbuf = vidc_vb2_find_buf(inst, addr);
+	if (!vbuf)
+		return -EINVAL;
+
+	vb = &vbuf->vb2_buf;
+	vb->planes[0].bytesused = bytesused;
+	vb->planes[0].data_offset = data_offset;
+	vb->timestamp = timeval_to_ns(timestamp);
+	vbuf->flags = flags;
+	vbuf->sequence = inst->sequence++;
+	state = VB2_BUF_STATE_DONE;
+
+	if (data_offset > vb->planes[0].length ||
+	    bytesused > vb->planes[0].length)
+		state = VB2_BUF_STATE_ERROR;
+
+	vb2_buffer_done(vb, state);
+
+	return 0;
+}
+
+static int venc_event_notify(struct hfi_inst *hfi_inst, u32 event,
+			     struct hfi_event_data *data)
+{
+	struct vidc_inst *inst = hfi_inst->ops_priv;
+	struct device *dev = inst->core->dev;
+
+	switch (event) {
+	case EVT_SESSION_ERROR:
+		if (hfi_inst) {
+			mutex_lock(&hfi_inst->lock);
+			inst->hfi_inst->state = INST_INVALID;
+			mutex_unlock(&hfi_inst->lock);
+		}
+		dev_err(dev, "enc: event session error (inst:%p)\n", hfi_inst);
+		break;
+	case EVT_SYS_EVENT_CHANGE:
+		switch (data->event_type) {
+		case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES:
+			dev_dbg(dev, "event sufficient resources\n");
+			break;
+		case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES:
+			inst->reconfig_height = data->height;
+			inst->reconfig_width = data->width;
+			inst->in_reconfig = true;
+			dev_dbg(dev, "event not sufficient resources\n");
+			break;
+		default:
+			break;
+		}
+		break;
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+static const struct hfi_inst_ops venc_hfi_ops = {
+	.empty_buf_done = venc_empty_buf_done,
+	.fill_buf_done = venc_fill_buf_done,
+	.event_notify = venc_event_notify,
+};
+
+static void venc_inst_init(struct vidc_inst *inst)
+{
+	struct hfi_inst *hfi_inst = inst->hfi_inst;
+
+	inst->fmt_cap = &venc_formats[2];
+	inst->fmt_out = &venc_formats[0];
+	inst->width = 1280;
+	inst->height = ALIGN(720, 32);
+	inst->fps = 15;
+	inst->timeperframe.numerator = 1;
+	inst->timeperframe.denominator = 15;
+
+	hfi_inst->width.min = 64;
+	hfi_inst->width.max = 1920;
+	hfi_inst->width.step_size = 1;
+	hfi_inst->height.min = 64;
+	hfi_inst->height.max = ALIGN(1080, 32);
+	hfi_inst->height.step_size = 1;
+	hfi_inst->framerate.min = 1;
+	hfi_inst->framerate.max = 30;
+	hfi_inst->framerate.step_size = 1;
+	hfi_inst->mbs_per_frame.min = 16;
+	hfi_inst->mbs_per_frame.max = 8160;
+}
+
+int venc_init(struct vidc_core *core, struct video_device *enc)
+{
+	int ret;
+
+	enc->release = video_device_release_empty;
+	enc->fops = &vidc_fops;
+	enc->ioctl_ops = &venc_ioctl_ops;
+	enc->vfl_dir = VFL_DIR_M2M;
+	enc->v4l2_dev = &core->v4l2_dev;
+
+	ret = video_register_device(enc, VFL_TYPE_GRABBER, -1);
+	if (ret)
+		return ret;
+
+	video_set_drvdata(enc, core);
+
+	return 0;
+}
+
+void venc_deinit(struct vidc_core *core, struct video_device *enc)
+{
+	video_unregister_device(enc);
+}
+
+int venc_open(struct vidc_inst *inst)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+	struct vb2_queue *q;
+	int ret;
+
+	ret = venc_ctrl_init(inst);
+	if (ret)
+		return ret;
+
+	inst->hfi_inst = vidc_hfi_session_create(hfi, &venc_hfi_ops, inst);
+	if (IS_ERR(inst->hfi_inst)) {
+		ret = PTR_ERR(inst->hfi_inst);
+		goto err_ctrl_deinit;
+	}
+
+	venc_inst_init(inst);
+
+	q = &inst->bufq_cap;
+	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	q->io_modes = VB2_MMAP | VB2_DMABUF;
+	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	q->ops = &venc_vb2_ops;
+	q->mem_ops = &vb2_dma_sg_memops;
+	q->drv_priv = inst;
+	q->buf_struct_size = sizeof(struct vidc_buffer);
+	q->allow_zero_bytesused = 1;
+	ret = vb2_queue_init(q);
+	if (ret)
+		goto err_session_destroy;
+
+	q = &inst->bufq_out;
+	q->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	q->io_modes = VB2_MMAP | VB2_DMABUF;
+	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	q->ops = &venc_vb2_ops;
+	q->mem_ops = &vb2_dma_sg_memops;
+	q->drv_priv = inst;
+	q->buf_struct_size = sizeof(struct vidc_buffer);
+	q->allow_zero_bytesused = 1;
+	ret = vb2_queue_init(q);
+	if (ret)
+		goto err_cap_queue_release;
+
+	return 0;
+
+err_cap_queue_release:
+	vb2_queue_release(&inst->bufq_cap);
+err_session_destroy:
+	vidc_hfi_session_destroy(hfi, inst->hfi_inst);
+	inst->hfi_inst = NULL;
+err_ctrl_deinit:
+	venc_ctrl_deinit(inst);
+	return ret;
+}
+
+void venc_close(struct vidc_inst *inst)
+{
+	struct hfi_core *hfi = &inst->core->hfi;
+
+	vb2_queue_release(&inst->bufq_out);
+	vb2_queue_release(&inst->bufq_cap);
+	venc_ctrl_deinit(inst);
+	vidc_hfi_session_destroy(hfi, inst->hfi_inst);
+}
diff --git a/drivers/media/platform/qcom/vidc/venc.h b/drivers/media/platform/qcom/vidc/venc.h
new file mode 100644
index 000000000000..ef839b59def4
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/venc.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_VENC_H__
+#define __VIDC_VENC_H__
+
+struct vidc_core;
+struct video_device;
+struct vidc_inst;
+
+int venc_init(struct vidc_core *core, struct video_device *enc);
+void venc_deinit(struct vidc_core *core, struct video_device *enc);
+int venc_open(struct vidc_inst *inst);
+void venc_close(struct vidc_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/venc_ctrls.c b/drivers/media/platform/qcom/vidc/venc_ctrls.c
new file mode 100644
index 000000000000..61331f95d54a
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/venc_ctrls.c
@@ -0,0 +1,396 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/types.h>
+#include <media/v4l2-ctrls.h>
+
+#include "core.h"
+
+#define BITRATE_MIN		32000
+#define BITRATE_MAX		160000000
+#define BITRATE_DEFAULT		1000000
+#define BITRATE_DEFAULT_PEAK	(BITRATE_DEFAULT * 2)
+#define BITRATE_STEP		100
+#define SLICE_BYTE_SIZE_MAX	1024
+#define SLICE_BYTE_SIZE_MIN	1024
+#define SLICE_MB_SIZE_MAX	300
+#define INTRA_REFRESH_MBS_MAX	300
+#define AT_SLICE_BOUNDARY	\
+	V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY
+static struct vidc_ctrl venc_ctrls[] = {
+	{
+		.id = V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+		.max = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+		.def = V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+		.menu_skip_mask = ~((1 << V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) |
+				    (1 << V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)),
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_BITRATE,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = BITRATE_MIN,
+		.max = BITRATE_MAX,
+		.def = BITRATE_DEFAULT,
+		.step = BITRATE_STEP,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_BITRATE_PEAK,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = BITRATE_MIN,
+		.max = BITRATE_MAX,
+		.def = BITRATE_DEFAULT_PEAK,
+		.step = BITRATE_STEP,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC,
+		.max = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC,
+		.def = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
+		.max = V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_CODING_EFFICIENCY,
+		.def = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
+		.menu_skip_mask = ~(
+			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE) |
+			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE)
+		),
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
+		.max = V4L2_MPEG_VIDEO_MPEG4_LEVEL_5,
+		.def = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+		.max = V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH,
+		.def = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+		.menu_skip_mask = ~(
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH) |
+		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH)
+		),
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+		.max = V4L2_MPEG_VIDEO_H264_LEVEL_5_1,
+		.def = V4L2_MPEG_VIDEO_H264_LEVEL_5_0,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_VPX_PROFILE,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 0,
+		.max = 3,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = 51,
+		.def = 26,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = 51,
+		.def = 28,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = 51,
+		.def = 30,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_MIN_QP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = 51,
+		.def = 1,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = 51,
+		.def = 51,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE,
+		.max = V4L2_MPEG_VIDEO_MULTI_SICE_MODE_MAX_BYTES,
+		.def = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = SLICE_BYTE_SIZE_MIN,
+		.max = SLICE_BYTE_SIZE_MAX,
+		.def = SLICE_BYTE_SIZE_MIN,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = SLICE_MB_SIZE_MAX,
+		.def = 1,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = -6,
+		.max = 6,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = -6,
+		.max = 6,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_ENABLED,
+		.max = AT_SLICE_BOUNDARY,
+		.def = V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+		.max = V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+		.def = V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+		.menu_skip_mask =
+			1 << V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 0,
+		.max = INTRA_REFRESH_MBS_MAX,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE,
+		.type = V4L2_CTRL_TYPE_BOOLEAN,
+		.min = 0,
+		.max = 1,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC,
+		.type = V4L2_CTRL_TYPE_MENU,
+		.min = V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_UNSPECIFIED,
+		.max = V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_EXTENDED,
+		.def = V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_UNSPECIFIED,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 0,
+		.max = (1 << 16) - 1,
+		.def = 12,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 0,
+		.max = (1 << 16) - 1,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM,
+		.type = V4L2_CTRL_TYPE_BOOLEAN,
+		.min = 0,
+		.max = 1,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_VPX_MIN_QP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = 128,
+		.def = 1,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_VPX_MAX_QP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 1,
+		.max = 128,
+		.def = 128,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_B_FRAMES,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 0,
+		.max = INT_MAX,
+		.def = 0,
+		.step = 1,
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.min = 0,
+		.max = (1 << 16) - 1,
+		.step = 1,
+		.def = 0,
+	},
+};
+
+#define NUM_CTRLS	ARRAY_SIZE(venc_ctrls)
+
+static int venc_op_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct vidc_inst *inst = ctrl_to_inst(ctrl);
+	struct venc_controls *ctr = &inst->controls.enc;
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_BITRATE_MODE:
+		ctr->bitrate_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_BITRATE:
+		ctr->bitrate = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_BITRATE_PEAK:
+		ctr->bitrate_peak = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
+		ctr->h264_entropy_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+		ctr->profile = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+		ctr->level = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP:
+		ctr->h264_i_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP:
+		ctr->h264_p_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP:
+		ctr->h264_b_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_MIN_QP:
+		ctr->h264_min_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+		ctr->h264_max_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE:
+		ctr->multi_slice_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES:
+		ctr->multi_slice_max_bytes = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB:
+		ctr->multi_slice_max_mb = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA:
+		ctr->h264_loop_filter_alpha = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA:
+		ctr->h264_loop_filter_beta = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE:
+		ctr->h264_loop_filter_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
+		ctr->header_mode = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB:
+		break;
+	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+		ctr->gop_size = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
+		ctr->h264_i_period = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE:
+	case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC:
+	case V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE:
+	case V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM:
+		break;
+	case V4L2_CID_MPEG_VIDEO_VPX_MIN_QP:
+		ctr->vp8_min_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_VPX_MAX_QP:
+		ctr->vp8_max_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_B_FRAMES:
+		ctr->num_b_frames = ctrl->val;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops venc_ctrl_ops = {
+	.s_ctrl = venc_op_s_ctrl,
+};
+
+int venc_ctrl_init(struct vidc_inst *inst)
+{
+	unsigned int i;
+	int ret;
+
+	ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, NUM_CTRLS);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < NUM_CTRLS; i++) {
+		struct v4l2_ctrl *ctrl;
+
+		if (venc_ctrls[i].type == V4L2_CTRL_TYPE_MENU) {
+			ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
+					&venc_ctrl_ops, venc_ctrls[i].id,
+					venc_ctrls[i].max,
+					venc_ctrls[i].menu_skip_mask,
+					venc_ctrls[i].def);
+		} else {
+			ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler,
+					&venc_ctrl_ops, venc_ctrls[i].id,
+					venc_ctrls[i].min,
+					venc_ctrls[i].max,
+					venc_ctrls[i].step,
+					venc_ctrls[i].def);
+		}
+
+		ret = inst->ctrl_handler.error;
+		if (ret) {
+			v4l2_ctrl_handler_free(&inst->ctrl_handler);
+			return ret;
+		}
+	}
+
+	return 0;
+}
+
+void venc_ctrl_deinit(struct vidc_inst *inst)
+{
+	v4l2_ctrl_handler_free(&inst->ctrl_handler);
+}
diff --git a/drivers/media/platform/qcom/vidc/venc_ctrls.h b/drivers/media/platform/qcom/vidc/venc_ctrls.h
new file mode 100644
index 000000000000..4441f550f57d
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/venc_ctrls.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VIDC_VENC_CTRLS_H__
+#define __VIDC_VENC_CTRLS_H__
+
+struct vidc_inst;
+
+int venc_ctrl_init(struct vidc_inst *inst);
+void venc_ctrl_deinit(struct vidc_inst *inst);
+
+#endif
-- 
2.7.4

[PATCH 5/8] media: vidc: add Host Firmware Interface (HFI)

[Stanimir Varbanov]

This is the implementation of HFI. It is loaded with the
responsibility to comunicate with the firmware through an
interface commands and messages.

 - hfi.c has interface functions used by the core, decoder
and encoder parts to comunicate with the firmware. For example
there are functions for session and core initialisation.

 - hfi_cmds has packetization operations which preparing
packets to be send from host to firmware.

 - hfi_msgs takes care of messages sent from firmware to the
host.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/vidc/hfi.c        |  622 ++++++++++++
 drivers/media/platform/qcom/vidc/hfi.h        |  272 ++++++
 drivers/media/platform/qcom/vidc/hfi_cmds.c   | 1261 +++++++++++++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_cmds.h   |  338 +++++++
 drivers/media/platform/qcom/vidc/hfi_helper.h | 1143 ++++++++++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_msgs.c   | 1072 +++++++++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_msgs.h   |  298 ++++++
 7 files changed, 5006 insertions(+)
 create mode 100644 drivers/media/platform/qcom/vidc/hfi.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_cmds.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_cmds.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_helper.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_msgs.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_msgs.h

diff --git a/drivers/media/platform/qcom/vidc/hfi.c b/drivers/media/platform/qcom/vidc/hfi.c
new file mode 100644
index 000000000000..030b50082ff2
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi.c
@@ -0,0 +1,622 @@
+/*
+ * Copyright (c) 2012-2013, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/slab.h>
+#include <linux/mutex.h>
+#include <linux/list.h>
+#include <linux/completion.h>
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+
+#include "hfi.h"
+#include "hfi_cmds.h"
+#include "hfi_venus.h"
+
+#define TIMEOUT		msecs_to_jiffies(1000)
+
+static u32 to_codec_type(u32 pixfmt)
+{
+	switch (pixfmt) {
+	case V4L2_PIX_FMT_H264:
+	case V4L2_PIX_FMT_H264_NO_SC:
+		return HFI_VIDEO_CODEC_H264;
+	case V4L2_PIX_FMT_H263:
+		return HFI_VIDEO_CODEC_H263;
+	case V4L2_PIX_FMT_MPEG1:
+		return HFI_VIDEO_CODEC_MPEG1;
+	case V4L2_PIX_FMT_MPEG2:
+		return HFI_VIDEO_CODEC_MPEG2;
+	case V4L2_PIX_FMT_MPEG4:
+		return HFI_VIDEO_CODEC_MPEG4;
+	case V4L2_PIX_FMT_VC1_ANNEX_G:
+	case V4L2_PIX_FMT_VC1_ANNEX_L:
+		return HFI_VIDEO_CODEC_VC1;
+	case V4L2_PIX_FMT_VP8:
+		return HFI_VIDEO_CODEC_VP8;
+	case V4L2_PIX_FMT_XVID:
+		return HFI_VIDEO_CODEC_DIVX;
+	default:
+		return 0;
+	}
+}
+
+int vidc_hfi_core_init(struct hfi_core *hfi)
+{
+	int ret = 0;
+
+	mutex_lock(&hfi->lock);
+
+	if (hfi->state >= CORE_INIT)
+		goto unlock;
+
+	init_completion(&hfi->done);
+
+	ret = call_hfi_op(hfi, core_init, hfi);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&hfi->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+
+	if (hfi->error != HFI_ERR_NONE) {
+		ret = -EIO;
+		goto unlock;
+	}
+
+	hfi->state = CORE_INIT;
+unlock:
+	mutex_unlock(&hfi->lock);
+	return ret;
+}
+
+int vidc_hfi_core_deinit(struct hfi_core *hfi)
+{
+	struct device *dev = hfi->dev;
+	int ret = 0;
+
+	mutex_lock(&hfi->lock);
+
+	if (hfi->state == CORE_UNINIT)
+		goto unlock;
+
+	if (!list_empty(&hfi->instances)) {
+		ret = -EBUSY;
+		goto unlock;
+	}
+
+	/*
+	 * Delay unloading of firmware. This is useful
+	 * in avoiding firmware download delays in cases where we
+	 * will have a burst of back to back video playback sessions
+	 * e.g. thumbnail generation.
+	 */
+	ret = call_hfi_op(hfi, core_deinit, hfi);
+	if (ret)
+		dev_err(dev, "core deinit failed: %d\n", ret);
+
+	hfi->state = CORE_UNINIT;
+
+unlock:
+	mutex_unlock(&hfi->lock);
+	return ret;
+}
+
+int vidc_hfi_core_suspend(struct hfi_core *hfi)
+{
+	return call_hfi_op(hfi, suspend, hfi);
+}
+
+int vidc_hfi_core_resume(struct hfi_core *hfi)
+{
+	return call_hfi_op(hfi, resume, hfi);
+}
+
+int vidc_hfi_core_trigger_ssr(struct hfi_core *hfi, u32 type)
+{
+	int ret;
+
+	ret = call_hfi_op(hfi, core_trigger_ssr, hfi, type);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+int vidc_hfi_core_ping(struct hfi_core *hfi)
+{
+	int ret;
+
+	mutex_lock(&hfi->lock);
+
+	ret = call_hfi_op(hfi, core_ping, hfi, 0xbeef);
+	if (ret)
+		return ret;
+
+	ret = wait_for_completion_timeout(&hfi->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+	ret = 0;
+	if (hfi->error != HFI_ERR_NONE)
+		ret = -ENODEV;
+unlock:
+	mutex_unlock(&hfi->lock);
+	return ret;
+}
+
+struct hfi_inst *
+vidc_hfi_session_create(struct hfi_core *hfi, const struct hfi_inst_ops *ops,
+			void *ops_priv)
+{
+	struct hfi_inst *inst;
+
+	if (!ops)
+		return ERR_PTR(-EINVAL);
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return ERR_PTR(-ENOMEM);
+
+	mutex_init(&inst->lock);
+	INIT_LIST_HEAD(&inst->list);
+	inst->state = INST_UNINIT;
+	inst->ops = ops;
+	inst->ops_priv = ops_priv;
+
+	mutex_lock(&hfi->lock);
+	list_add_tail(&inst->list, &hfi->instances);
+	mutex_unlock(&hfi->lock);
+
+	return inst;
+}
+
+int vidc_hfi_session_init(struct hfi_core *hfi, struct hfi_inst *inst,
+			  u32 pixfmt, u32 session_type)
+{
+	u32 codec;
+	int ret;
+
+	if (!hfi || !inst)
+		return -EINVAL;
+
+	codec = to_codec_type(pixfmt);
+	inst->session_type = session_type;
+	init_completion(&inst->done);
+
+	mutex_lock(&inst->lock);
+
+	ret = call_hfi_op(hfi, session_init, hfi, inst, session_type, codec);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	if (inst->error != HFI_ERR_NONE) {
+		dev_err(hfi->dev, "%s: session init failed (%x)\n", __func__,
+			inst->error);
+		ret = -EIO;
+		goto unlock;
+	}
+
+	ret = 0;
+	inst->state = INST_INIT;
+
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+void vidc_hfi_session_destroy(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	mutex_lock(&hfi->lock);
+	list_del(&inst->list);
+	mutex_unlock(&hfi->lock);
+
+	if (mutex_is_locked(&inst->lock))
+		WARN(1, "session destroy");
+
+	mutex_destroy(&inst->lock);
+	kfree(inst);
+}
+
+int vidc_hfi_session_deinit(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	if (inst->state == INST_UNINIT) {
+		ret = 0;
+		goto unlock;
+	}
+
+	if (inst->state < INST_INIT) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_end, inst);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	if (inst->error != HFI_ERR_NONE) {
+		dev_err(hfi->dev, "session deinit error (%x)\n", inst->error);
+		ret = -EIO;
+		goto unlock;
+	}
+
+	ret = 0;
+	inst->state = INST_UNINIT;
+
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_start(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	if (inst->state != INST_LOAD_RESOURCES) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_start, inst);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+
+	inst->state = INST_START;
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_stop(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	if (inst->state != INST_START) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_stop, inst);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+
+	inst->state = INST_STOP;
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_abort(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_abort, inst);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_load_res(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	if (inst->state != INST_INIT) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_load_res, inst);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+	inst->state = INST_LOAD_RESOURCES;
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_unload_res(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	if (inst->state != INST_STOP) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_release_res, inst);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+	inst->state = INST_RELEASE_RESOURCES;
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_flush(struct hfi_core *hfi, struct hfi_inst *inst)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_flush, inst, HFI_FLUSH_ALL);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_set_buffers(struct hfi_core *hfi, struct hfi_inst *inst,
+				 struct hfi_buffer_desc *bd)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+	ret = call_hfi_op(hfi, session_set_buffers, inst, bd);
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_unset_buffers(struct hfi_core *hfi, struct hfi_inst *inst,
+				   struct hfi_buffer_desc *bd)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_release_buffers, inst, bd);
+	if (ret)
+		goto unlock;
+
+	if (!bd->response_required) {
+		ret = 0;
+		goto unlock;
+	}
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	ret = 0;
+
+	if (inst->error != HFI_ERR_NONE) {
+		dev_dbg(hfi->dev, "unset buffers error (%x)\n", inst->error);
+		ret = -EIO;
+	}
+
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_get_property(struct hfi_core *hfi, struct hfi_inst *inst,
+				  u32 ptype, union hfi_get_property *hprop)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	if (inst->state < INST_INIT || inst->state >= INST_STOP) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	init_completion(&inst->done);
+
+	ret = call_hfi_op(hfi, session_get_property, inst, ptype);
+	if (ret)
+		goto unlock;
+
+	ret = wait_for_completion_timeout(&inst->done, TIMEOUT);
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto unlock;
+	}
+
+	if (inst->error != HFI_ERR_NONE) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	ret = 0;
+	*hprop = inst->hprop;
+unlock:
+	mutex_unlock(&inst->lock);
+
+	return ret;
+}
+
+int vidc_hfi_session_set_property(struct hfi_core *hfi, struct hfi_inst *inst,
+				  u32 ptype, void *pdata)
+{
+	int ret;
+
+	mutex_lock(&inst->lock);
+
+	if (inst->state < INST_INIT || inst->state >= INST_STOP) {
+		ret = -EINVAL;
+		goto unlock;
+	}
+
+	ret = call_hfi_op(hfi, session_set_property, inst, ptype, pdata);
+unlock:
+	mutex_unlock(&inst->lock);
+
+	if (ret)
+		dev_err(hfi->dev, "set property %x failed (%d)\n", ptype, ret);
+
+	return ret;
+}
+
+int vidc_hfi_session_etb(struct hfi_core *hfi, struct hfi_inst *inst,
+			 struct hfi_frame_data *fdata)
+{
+	return call_hfi_op(hfi, session_etb, inst, fdata);
+}
+
+int vidc_hfi_session_ftb(struct hfi_core *hfi, struct hfi_inst *inst,
+			 struct hfi_frame_data *fdata)
+{
+	return call_hfi_op(hfi, session_ftb, inst, fdata);
+}
+
+irqreturn_t vidc_hfi_isr_thread(int irq, void *dev_id)
+{
+	struct hfi_core *hfi = dev_id;
+
+	return call_hfi_op(hfi, isr_thread, irq, hfi);
+}
+
+irqreturn_t vidc_hfi_isr(int irq, void *dev)
+{
+	struct hfi_core *hfi = dev;
+
+	return call_hfi_op(hfi, isr, irq, hfi);
+}
+
+int vidc_hfi_create(struct hfi_core *hfi, const struct vidc_resources *res,
+		    void __iomem *base)
+{
+	if (!hfi->core_ops || !hfi->dev)
+		return -EINVAL;
+
+	if (res->hfi_version)
+		hfi->packetization_type = HFI_PACKETIZATION_3XX;
+	else
+		hfi->packetization_type = HFI_PACKETIZATION_LEGACY;
+	mutex_init(&hfi->lock);
+	INIT_LIST_HEAD(&hfi->instances);
+	hfi->state = CORE_UNINIT;
+
+	hfi->pkt_ops = hfi_get_pkt_ops(hfi->packetization_type);
+	if (!hfi->pkt_ops)
+		return -EINVAL;
+
+	return venus_hfi_create(hfi, res, base);
+}
+
+void vidc_hfi_destroy(struct hfi_core *hfi)
+{
+	venus_hfi_destroy(hfi);
+	mutex_destroy(&hfi->lock);
+}
diff --git a/drivers/media/platform/qcom/vidc/hfi.h b/drivers/media/platform/qcom/vidc/hfi.h
new file mode 100644
index 000000000000..5154709f84a2
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi.h
@@ -0,0 +1,272 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __HFI_H__
+#define __HFI_H__
+
+#include <linux/interrupt.h>
+
+#include "resources.h"
+#include "hfi_helper.h"
+
+#define VIDC_SESSION_TYPE_VPE			0
+#define VIDC_SESSION_TYPE_ENC			1
+#define VIDC_SESSION_TYPE_DEC			2
+
+/* core capabilities */
+#define VIDC_ENC_ROTATION_CAPABILITY		0x1
+#define VIDC_ENC_SCALING_CAPABILITY		0x2
+#define VIDC_ENC_DEINTERLACE_CAPABILITY		0x4
+#define VIDC_DEC_MULTI_STREAM_CAPABILITY	0x8
+
+#define VIDC_RESOURCE_NONE			0
+#define VIDC_RESOURCE_OCMEM			1
+#define VIDC_RESOURCE_VMEM			2
+
+struct hfi_buffer_desc {
+	u32 buffer_type;
+	u32 buffer_size;
+	u32 num_buffers;
+	u32 device_addr;
+	u32 extradata_addr;
+	u32 extradata_size;
+	u32 response_required;
+};
+
+struct hfi_frame_data {
+	u32 buffer_type;
+	u32 device_addr;
+	u32 extradata_addr;
+	u64 timestamp;
+	u32 flags;
+	u32 offset;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 mark_target;
+	u32 mark_data;
+	u32 clnt_data;
+	u32 extradata_size;
+};
+
+union hfi_get_property {
+	struct hfi_profile_level profile_level;
+	struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
+};
+
+/* HFI events */
+#define EVT_SYS_EVENT_CHANGE			1
+#define EVT_SYS_WATCHDOG_TIMEOUT		2
+#define EVT_SYS_ERROR				3
+#define EVT_SESSION_ERROR			4
+
+/* HFI event callback structure */
+struct hfi_event_data {
+	u32 error;
+	u32 height;
+	u32 width;
+	u32 event_type;
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 profile;
+	u32 level;
+};
+
+/* define core states */
+#define CORE_UNINIT				0
+#define CORE_INIT				1
+#define CORE_INVALID				2
+
+/* define instance states */
+#define INST_INVALID				1
+#define INST_UNINIT				2
+#define INST_INIT				3
+#define INST_LOAD_RESOURCES			4
+#define INST_START				5
+#define INST_STOP				6
+#define INST_RELEASE_RESOURCES			7
+
+#define call_hfi_op(hfi, op, args...)	\
+	(((hfi) && (hfi)->ops && (hfi)->ops->op) ?	\
+	((hfi)->ops->op(args)) : 0)
+
+struct hfi_core;
+struct hfi_inst;
+
+struct hfi_core_ops {
+	int (*event_notify)(struct hfi_core *hfi, u32 event);
+};
+
+struct hfi_inst_ops {
+	int (*empty_buf_done)(struct hfi_inst *inst, u32 addr, u32 bytesused,
+			      u32 data_offset, u32 flags);
+	int (*fill_buf_done)(struct hfi_inst *inst, u32 addr, u32 bytesused,
+			     u32 data_offset, u32 flags, struct timeval *ts);
+	int (*event_notify)(struct hfi_inst *inst, u32 event,
+			    struct hfi_event_data *data);
+};
+
+struct hfi_inst {
+	struct list_head list;
+	struct mutex lock;
+	unsigned int state;
+	struct completion done;
+	unsigned int error;
+
+	/* instance operations passed by outside world */
+	const struct hfi_inst_ops *ops;
+	void *ops_priv;
+
+	void *priv;
+
+	u32 session_type;
+	union hfi_get_property hprop;
+
+	/* capabilities filled by session_init */
+	struct hfi_capability width;
+	struct hfi_capability height;
+	struct hfi_capability mbs_per_frame;
+	struct hfi_capability mbs_per_sec;
+	struct hfi_capability framerate;
+	struct hfi_capability scale_x;
+	struct hfi_capability scale_y;
+	struct hfi_capability bitrate;
+	struct hfi_capability hier_p;
+	struct hfi_capability ltr_count;
+	struct hfi_capability secure_output2_threshold;
+	bool alloc_mode_static;
+	bool alloc_mode_dynamic;
+
+	/* profile & level pairs supported */
+	unsigned int pl_count;
+	struct hfi_profile_level pl[HFI_MAX_PROFILE_COUNT];
+
+	/* buffer requirements */
+	struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
+};
+
+struct hfi_core {
+	struct device *dev;	/* mostly used for dev_xxx */
+
+	struct mutex lock;
+	unsigned int state;
+	struct completion done;
+	unsigned int error;
+
+	/*
+	 * list of 'struct hfi_inst' instances which belong to
+	 * this hfi core device
+	 */
+	struct list_head instances;
+
+	/* core operations passed by outside world */
+	const struct hfi_core_ops *core_ops;
+
+	/* filled by sys core init */
+	u32 enc_codecs;
+	u32 dec_codecs;
+
+	/* core capabilities */
+	unsigned int core_caps;
+
+	/* internal hfi operations */
+	void *priv;
+	const struct hfi_ops *ops;
+	const struct hfi_packetization_ops *pkt_ops;
+	enum hfi_packetization_type packetization_type;
+};
+
+struct hfi_ops {
+	int (*core_init)(struct hfi_core *hfi);
+	int (*core_deinit)(struct hfi_core *hfi);
+	int (*core_ping)(struct hfi_core *hfi, u32 cookie);
+	int (*core_trigger_ssr)(struct hfi_core *hfi, u32 trigger_type);
+
+	int (*session_init)(struct hfi_core *hfi, struct hfi_inst *inst,
+			    u32 session_type, u32 codec);
+	int (*session_end)(struct hfi_inst *inst);
+	int (*session_abort)(struct hfi_inst *inst);
+	int (*session_flush)(struct hfi_inst *inst, u32 flush_mode);
+	int (*session_start)(struct hfi_inst *inst);
+	int (*session_stop)(struct hfi_inst *inst);
+	int (*session_etb)(struct hfi_inst *inst,
+			   struct hfi_frame_data *input_frame);
+	int (*session_ftb)(struct hfi_inst *inst,
+			   struct hfi_frame_data *output_frame);
+	int (*session_set_buffers)(struct hfi_inst *inst,
+				   struct hfi_buffer_desc *bd);
+	int (*session_release_buffers)(struct hfi_inst *inst,
+				       struct hfi_buffer_desc *bd);
+	int (*session_load_res)(struct hfi_inst *inst);
+	int (*session_release_res)(struct hfi_inst *inst);
+	int (*session_parse_seq_hdr)(struct hfi_inst *inst, u32 seq_hdr,
+				     u32 seq_hdr_len);
+	int (*session_get_seq_hdr)(struct hfi_inst *inst, u32 seq_hdr,
+				   u32 seq_hdr_len);
+	int (*session_set_property)(struct hfi_inst *inst, u32 ptype,
+				    void *pdata);
+	int (*session_get_property)(struct hfi_inst *inst, u32 ptype);
+
+	int (*resume)(struct hfi_core *hfi);
+	int (*suspend)(struct hfi_core *hfi);
+
+	/* interrupt operations */
+	irqreturn_t (*isr)(int irq, struct hfi_core *hfi);
+	irqreturn_t (*isr_thread)(int irq, struct hfi_core *hfi);
+};
+
+static inline void *to_hfi_priv(struct hfi_core *hfi)
+{
+	return hfi->priv;
+}
+
+int vidc_hfi_create(struct hfi_core *hfi, const struct vidc_resources *res,
+		    void __iomem *base);
+void vidc_hfi_destroy(struct hfi_core *hfi);
+
+int vidc_hfi_core_init(struct hfi_core *hfi);
+int vidc_hfi_core_deinit(struct hfi_core *hfi);
+int vidc_hfi_core_suspend(struct hfi_core *hfi);
+int vidc_hfi_core_resume(struct hfi_core *hfi);
+int vidc_hfi_core_trigger_ssr(struct hfi_core *hfi, u32 type);
+int vidc_hfi_core_ping(struct hfi_core *hfi);
+
+struct hfi_inst *vidc_hfi_session_create(struct hfi_core *hfi,
+					 const struct hfi_inst_ops *ops,
+					 void *ops_priv);
+void vidc_hfi_session_destroy(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_init(struct hfi_core *hfi, struct hfi_inst *inst,
+			  u32 pixfmt, u32 session_type);
+int vidc_hfi_session_deinit(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_start(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_stop(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_abort(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_load_res(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_unload_res(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_flush(struct hfi_core *hfi, struct hfi_inst *inst);
+int vidc_hfi_session_set_buffers(struct hfi_core *hfi, struct hfi_inst *inst,
+				 struct hfi_buffer_desc *bd);
+int vidc_hfi_session_unset_buffers(struct hfi_core *hfi, struct hfi_inst *inst,
+				   struct hfi_buffer_desc *bd);
+int vidc_hfi_session_get_property(struct hfi_core *hfi, struct hfi_inst *inst,
+				  u32 ptype, union hfi_get_property *hprop);
+int vidc_hfi_session_set_property(struct hfi_core *hfi, struct hfi_inst *inst,
+				  u32 ptype, void *pdata);
+int vidc_hfi_session_etb(struct hfi_core *hfi, struct hfi_inst *inst,
+			 struct hfi_frame_data *fdata);
+int vidc_hfi_session_ftb(struct hfi_core *hfi, struct hfi_inst *inst,
+			 struct hfi_frame_data *fdata);
+irqreturn_t vidc_hfi_isr_thread(int irq, void *dev_id);
+irqreturn_t vidc_hfi_isr(int irq, void *dev);
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/hfi_cmds.c b/drivers/media/platform/qcom/vidc/hfi_cmds.c
new file mode 100644
index 000000000000..552d8c558a80
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_cmds.c
@@ -0,0 +1,1261 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/errno.h>
+#include <linux/log2.h>
+#include <linux/hash.h>
+
+#include "hfi_cmds.h"
+
+static void pkt_sys_init(struct hfi_sys_init_pkt *pkt, u32 arch_type)
+{
+	pkt->hdr.size = sizeof(*pkt);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_INIT;
+	pkt->arch_type = arch_type;
+}
+
+static void pkt_sys_pc_prep(struct hfi_sys_pc_prep_pkt *pkt)
+{
+	pkt->hdr.size = sizeof(*pkt);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_PC_PREP;
+}
+
+static void pkt_sys_idle_indicator(struct hfi_sys_set_property_pkt *pkt,
+				   u32 enable)
+{
+	struct hfi_enable *hfi = (struct hfi_enable *) &pkt->data[1];
+
+	pkt->hdr.size = sizeof(*pkt) + sizeof(*hfi) + sizeof(u32);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_SET_PROPERTY;
+	pkt->num_properties = 1;
+	pkt->data[0] = HFI_PROPERTY_SYS_IDLE_INDICATOR;
+	hfi->enable = enable;
+}
+
+static void pkt_sys_debug_config(struct hfi_sys_set_property_pkt *pkt, u32 mode,
+				 u32 config)
+{
+	struct hfi_debug_config *hfi;
+
+	pkt->hdr.size = sizeof(*pkt) + sizeof(*hfi) + sizeof(u32);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_SET_PROPERTY;
+	pkt->num_properties = 1;
+	pkt->data[0] = HFI_PROPERTY_SYS_DEBUG_CONFIG;
+	hfi = (struct hfi_debug_config *) &pkt->data[1];
+	hfi->config = config;
+	hfi->mode = mode;
+}
+
+static void pkt_sys_coverage_config(struct hfi_sys_set_property_pkt *pkt,
+				    u32 mode)
+{
+	pkt->hdr.size = sizeof(*pkt) + sizeof(u32);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_SET_PROPERTY;
+	pkt->num_properties = 1;
+	pkt->data[0] = HFI_PROPERTY_SYS_CONFIG_COVERAGE;
+	pkt->data[1] = mode;
+}
+
+static int pkt_sys_set_resource(struct hfi_sys_set_resource_pkt *pkt,
+				u32 id, u32 size, u32 addr, void *cookie)
+{
+	pkt->hdr.size = sizeof(*pkt);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_SET_RESOURCE;
+	pkt->resource_handle = hash32_ptr(cookie);
+
+	switch (id) {
+	case VIDC_RESOURCE_OCMEM:
+	case VIDC_RESOURCE_VMEM: {
+		struct hfi_resource_ocmem *res =
+			(struct hfi_resource_ocmem *) &pkt->resource_data[0];
+
+		res->size = size;
+		res->mem = addr;
+		pkt->resource_type = HFI_RESOURCE_OCMEM;
+		pkt->hdr.size += sizeof(*res) - sizeof(u32);
+		break;
+	}
+	case VIDC_RESOURCE_NONE:
+	default:
+		return -ENOTSUPP;
+	}
+
+	return 0;
+}
+
+static int pkt_sys_unset_resource(struct hfi_sys_release_resource_pkt *pkt,
+				  u32 id, u32 size, void *cookie)
+{
+	pkt->hdr.size = sizeof(*pkt);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_RELEASE_RESOURCE;
+	pkt->resource_handle = hash32_ptr(cookie);
+
+	switch (id) {
+	case VIDC_RESOURCE_OCMEM:
+	case VIDC_RESOURCE_VMEM:
+		pkt->resource_type = HFI_RESOURCE_OCMEM;
+		break;
+	case VIDC_RESOURCE_NONE:
+		break;
+	default:
+		return -ENOTSUPP;
+	}
+
+	return 0;
+}
+
+static void pkt_sys_ping(struct hfi_sys_ping_pkt *pkt, u32 cookie)
+{
+	pkt->hdr.size = sizeof(*pkt);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_PING;
+	pkt->client_data = cookie;
+}
+
+static int pkt_session_init(struct hfi_session_init_pkt *pkt,
+			    struct hfi_inst *inst, u32 session_type, u32 codec)
+{
+	if (!pkt || !inst || !codec)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SYS_SESSION_INIT;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->session_domain = session_type;
+	pkt->session_codec = codec;
+
+	return 0;
+}
+
+static void pkt_session_cmd(struct hfi_session_pkt *pkt, u32 pkt_type,
+			    struct hfi_inst *inst)
+{
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = pkt_type;
+	pkt->shdr.session_id = hash32_ptr(inst);
+}
+
+static void pkt_sys_power_control(struct hfi_sys_set_property_pkt *pkt,
+				  u32 enable)
+{
+	struct hfi_enable *hfi = (struct hfi_enable *) &pkt->data[1];
+
+	pkt->hdr.size = sizeof(*pkt) + sizeof(*hfi) + sizeof(u32);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_SET_PROPERTY;
+	pkt->num_properties = 1;
+	pkt->data[0] = HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL;
+	hfi->enable = enable;
+}
+
+static int pkt_session_set_buffers(struct hfi_session_set_buffers_pkt *pkt,
+				   struct hfi_inst *inst,
+				   struct hfi_buffer_desc *bd)
+{
+	int i;
+
+	if (!inst || !pkt || !bd)
+		return -EINVAL;
+
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_BUFFERS;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->buffer_size = bd->buffer_size;
+	pkt->min_buffer_size = bd->buffer_size;
+	pkt->num_buffers = bd->num_buffers;
+
+	if (bd->buffer_type == HFI_BUFFER_OUTPUT ||
+	    bd->buffer_type == HFI_BUFFER_OUTPUT2) {
+		struct hfi_buffer_info *bi;
+
+		pkt->extradata_size = bd->extradata_size;
+		pkt->shdr.hdr.size = sizeof(*pkt) - sizeof(u32) +
+			(bd->num_buffers * sizeof(*bi));
+		bi = (struct hfi_buffer_info *) pkt->buffer_info;
+		for (i = 0; i < pkt->num_buffers; i++) {
+			bi->buffer_addr = bd->device_addr;
+			bi->extradata_addr = bd->extradata_addr;
+		}
+	} else {
+		pkt->extradata_size = 0;
+		pkt->shdr.hdr.size = sizeof(*pkt) +
+			((bd->num_buffers - 1) * sizeof(u32));
+		for (i = 0; i < pkt->num_buffers; i++)
+			pkt->buffer_info[i] = bd->device_addr;
+	}
+
+	pkt->buffer_type = bd->buffer_type;
+
+	return 0;
+}
+
+static int
+pkt_session_release_buffers(struct hfi_session_release_buffer_pkt *pkt,
+			    struct hfi_inst *inst, struct hfi_buffer_desc *bd)
+{
+	int i;
+
+	if (!inst || !pkt || !bd)
+		return -EINVAL;
+
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_RELEASE_BUFFERS;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->buffer_size = bd->buffer_size;
+	pkt->num_buffers = bd->num_buffers;
+
+	if (bd->buffer_type == HFI_BUFFER_OUTPUT ||
+	    bd->buffer_type == HFI_BUFFER_OUTPUT2) {
+		struct hfi_buffer_info *bi;
+
+		bi = (struct hfi_buffer_info *) pkt->buffer_info;
+		for (i = 0; i < pkt->num_buffers; i++) {
+			bi->buffer_addr = bd->device_addr;
+			bi->extradata_addr = bd->extradata_addr;
+		}
+		pkt->shdr.hdr.size =
+				sizeof(struct hfi_session_set_buffers_pkt) -
+				sizeof(u32) + (bd->num_buffers * sizeof(*bi));
+	} else {
+		for (i = 0; i < pkt->num_buffers; i++)
+			pkt->buffer_info[i] = bd->device_addr;
+
+		pkt->extradata_size = 0;
+		pkt->shdr.hdr.size =
+				sizeof(struct hfi_session_set_buffers_pkt) +
+				((bd->num_buffers - 1) * sizeof(u32));
+	}
+
+	pkt->response_req = bd->response_required;
+	pkt->buffer_type = bd->buffer_type;
+
+	return 0;
+}
+
+static int
+pkt_session_etb_decoder(struct hfi_session_empty_buffer_compressed_pkt *pkt,
+			struct hfi_inst *inst, struct hfi_frame_data *in_frame)
+{
+	if (!inst || !in_frame->device_addr)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_EMPTY_BUFFER;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->time_stamp_hi = upper_32_bits(in_frame->timestamp);
+	pkt->time_stamp_lo = lower_32_bits(in_frame->timestamp);
+	pkt->flags = in_frame->flags;
+	pkt->mark_target = in_frame->mark_target;
+	pkt->mark_data = in_frame->mark_data;
+	pkt->offset = in_frame->offset;
+	pkt->alloc_len = in_frame->alloc_len;
+	pkt->filled_len = in_frame->filled_len;
+	pkt->input_tag = in_frame->clnt_data;
+	pkt->packet_buffer = in_frame->device_addr;
+
+	return 0;
+}
+
+static int pkt_session_etb_encoder(
+		struct hfi_session_empty_buffer_uncompressed_plane0_pkt *pkt,
+		struct hfi_inst *inst, struct hfi_frame_data *in_frame)
+{
+	if (!inst || !in_frame->device_addr)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_EMPTY_BUFFER;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->view_id = 0;
+	pkt->time_stamp_hi = upper_32_bits(in_frame->timestamp);
+	pkt->time_stamp_lo = lower_32_bits(in_frame->timestamp);
+	pkt->flags = in_frame->flags;
+	pkt->mark_target = in_frame->mark_target;
+	pkt->mark_data = in_frame->mark_data;
+	pkt->offset = in_frame->offset;
+	pkt->alloc_len = in_frame->alloc_len;
+	pkt->filled_len = in_frame->filled_len;
+	pkt->input_tag = in_frame->clnt_data;
+	pkt->packet_buffer = in_frame->device_addr;
+	pkt->extradata_buffer = in_frame->extradata_addr;
+
+	return 0;
+}
+
+static int pkt_session_ftb(struct hfi_session_fill_buffer_pkt *pkt,
+			   struct hfi_inst *inst,
+			   struct hfi_frame_data *out_frame)
+{
+	if (!inst || !out_frame || !out_frame->device_addr)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_FILL_BUFFER;
+	pkt->shdr.session_id = hash32_ptr(inst);
+
+	if (out_frame->buffer_type == HFI_BUFFER_OUTPUT)
+		pkt->stream_id = 0;
+	else if (out_frame->buffer_type == HFI_BUFFER_OUTPUT2)
+		pkt->stream_id = 1;
+
+	pkt->packet_buffer = out_frame->device_addr;
+	pkt->extradata_buffer = out_frame->extradata_addr;
+	pkt->alloc_len = out_frame->alloc_len;
+	pkt->filled_len = out_frame->filled_len;
+	pkt->offset = out_frame->offset;
+	pkt->data[0] = out_frame->extradata_size;
+
+	return 0;
+}
+
+static int pkt_session_parse_seq_header(
+		struct hfi_session_parse_sequence_header_pkt *pkt,
+		struct hfi_inst *inst, u32 seq_hdr, u32 seq_hdr_len)
+{
+	if (!inst || !seq_hdr || !seq_hdr_len)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_PARSE_SEQUENCE_HEADER;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->header_len = seq_hdr_len;
+	pkt->packet_buffer = seq_hdr;
+
+	return 0;
+}
+
+static int
+pkt_session_get_seq_hdr(struct hfi_session_get_sequence_header_pkt *pkt,
+			struct hfi_inst *inst, u32 seq_hdr, u32 seq_hdr_len)
+{
+	if (!inst || !seq_hdr || !seq_hdr_len)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_GET_SEQUENCE_HEADER;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->buffer_len = seq_hdr_len;
+	pkt->packet_buffer = seq_hdr;
+
+	return 0;
+}
+
+static int pkt_session_flush(struct hfi_session_flush_pkt *pkt,
+			     struct hfi_inst *inst, u32 type)
+{
+	switch (type) {
+	case HFI_FLUSH_INPUT:
+	case HFI_FLUSH_OUTPUT:
+	case HFI_FLUSH_OUTPUT2:
+	case HFI_FLUSH_ALL:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->flush_type = type;
+
+	return 0;
+}
+
+static int pkt_session_get_property(struct hfi_session_get_property_pkt *pkt,
+				    struct hfi_inst *inst, u32 ptype)
+{
+	switch (ptype) {
+	case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT:
+	case HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_GET_PROPERTY;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->num_properties = 1;
+	pkt->data[0] = ptype;
+
+	return 0;
+}
+
+static int pkt_session_set_property(struct hfi_session_set_property_pkt *pkt,
+				    struct hfi_inst *inst, u32 ptype,
+				    void *pdata)
+{
+	void *prop_data = &pkt->data[1];
+	int ret = 0;
+
+	if (!pkt || !inst || !pdata)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_PROPERTY;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->num_properties = 1;
+
+	switch (ptype) {
+	case HFI_PROPERTY_CONFIG_FRAME_RATE: {
+		struct hfi_framerate *in = pdata, *frate = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_FRAME_RATE;
+		frate->buffer_type = in->buffer_type;
+		frate->framerate = in->framerate;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*frate);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT: {
+		struct hfi_uncompressed_format_select *in = pdata;
+		struct hfi_uncompressed_format_select *hfi = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT;
+		hfi->buffer_type = in->buffer_type;
+		hfi->format = in->format;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*hfi);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_FRAME_SIZE: {
+		struct hfi_framesize *in = pdata, *fsize = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_FRAME_SIZE;
+		fsize->buffer_type = in->buffer_type;
+		fsize->height = in->height;
+		fsize->width = in->width;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*fsize);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_REALTIME: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_REALTIME;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL: {
+		struct hfi_buffer_count_actual *in = pdata, *count = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
+		count->count_actual = in->count_actual;
+		count->type = in->type;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*count);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL: {
+		struct hfi_buffer_size_actual *in = pdata, *sz = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL;
+		sz->size = in->size;
+		sz->type = in->type;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*sz);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_BUFFER_DISPLAY_HOLD_COUNT_ACTUAL: {
+		struct hfi_buffer_display_hold_count_actual *in = pdata;
+		struct hfi_buffer_display_hold_count_actual *count = prop_data;
+
+		pkt->data[0] =
+			HFI_PROPERTY_PARAM_BUFFER_DISPLAY_HOLD_COUNT_ACTUAL;
+		count->hold_count = in->hold_count;
+		count->type = in->type;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*count);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SELECT: {
+		struct hfi_nal_stream_format_select *in = pdata;
+		struct hfi_nal_stream_format_select *fmt = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SELECT;
+		fmt->format = in->format;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*fmt);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_OUTPUT_ORDER: {
+		u32 *in = pdata;
+
+		switch (*in) {
+		case HFI_OUTPUT_ORDER_DECODE:
+		case HFI_OUTPUT_ORDER_DISPLAY:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_OUTPUT_ORDER;
+		pkt->data[1] = *in;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_PICTURE_TYPE_DECODE: {
+		struct hfi_enable_picture *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_PICTURE_TYPE_DECODE;
+		en->picture_type = in->picture_type;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_OUTPUT2_KEEP_ASPECT_RATIO: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] =
+			HFI_PROPERTY_PARAM_VDEC_OUTPUT2_KEEP_ASPECT_RATIO;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER: {
+		struct hfi_enable *in = pdata;
+		struct hfi_enable *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM: {
+		struct hfi_multi_stream *in = pdata, *multi = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
+		multi->buffer_type = in->buffer_type;
+		multi->enable = in->enable;
+		multi->width = in->width;
+		multi->height = in->height;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*multi);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_DISPLAY_PICTURE_BUFFER_COUNT: {
+		struct hfi_display_picture_buffer_count *in = pdata;
+		struct hfi_display_picture_buffer_count *count = prop_data;
+
+		pkt->data[0] =
+			HFI_PROPERTY_PARAM_VDEC_DISPLAY_PICTURE_BUFFER_COUNT;
+		count->count = in->count;
+		count->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*count);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_DIVX_FORMAT: {
+		u32 *in = pdata;
+
+		switch (*in) {
+		case HFI_DIVX_FORMAT_4:
+		case HFI_DIVX_FORMAT_5:
+		case HFI_DIVX_FORMAT_6:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_DIVX_FORMAT;
+		pkt->data[1] = *in;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VDEC_MB_ERROR_MAP_REPORTING: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VDEC_MB_ERROR_MAP_REPORTING;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_THUMBNAIL_MODE: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_THUMBNAIL_MODE;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] =
+			HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME:
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME;
+		pkt->shdr.hdr.size += sizeof(u32);
+		break;
+	case HFI_PROPERTY_PARAM_VENC_MPEG4_SHORT_HEADER:
+		break;
+	case HFI_PROPERTY_PARAM_VENC_MPEG4_AC_PREDICTION:
+		break;
+	case HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE: {
+		struct hfi_bitrate *in = pdata, *brate = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE;
+		brate->bitrate = in->bitrate;
+		brate->layer_id = in->layer_id;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*brate);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE: {
+		struct hfi_bitrate *in = pdata, *hfi = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE;
+		hfi->bitrate = in->bitrate;
+		hfi->layer_id = in->layer_id;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*hfi);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT: {
+		struct hfi_profile_level *in = pdata, *pl = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
+		pl->level = in->level;
+		pl->profile = in->profile;
+		if (pl->profile <= 0)
+			/* Profile not supported, falling back to high */
+			pl->profile = HFI_H264_PROFILE_HIGH;
+
+		if (!pl->level)
+			/* Level not supported, falling back to 1 */
+			pl->level = 1;
+
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*pl);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_H264_ENTROPY_CONTROL: {
+		struct hfi_h264_entropy_control *in = pdata, *hfi = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_H264_ENTROPY_CONTROL;
+		hfi->entropy_mode = in->entropy_mode;
+		if (hfi->entropy_mode == HFI_H264_ENTROPY_CABAC)
+			hfi->cabac_model = in->cabac_model;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*hfi);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_RATE_CONTROL: {
+		u32 *in = pdata;
+
+		switch (*in) {
+		case HFI_RATE_CONTROL_OFF:
+		case HFI_RATE_CONTROL_CBR_CFR:
+		case HFI_RATE_CONTROL_CBR_VFR:
+		case HFI_RATE_CONTROL_VBR_CFR:
+		case HFI_RATE_CONTROL_VBR_VFR:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_RATE_CONTROL;
+		pkt->data[1] = *in;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_MPEG4_TIME_RESOLUTION: {
+		struct hfi_mpeg4_time_resolution *in = pdata, *res = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_MPEG4_TIME_RESOLUTION;
+		res->time_increment_resolution = in->time_increment_resolution;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*res);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_MPEG4_HEADER_EXTENSION: {
+		struct hfi_mpeg4_header_extension *in = pdata, *ext = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_MPEG4_HEADER_EXTENSION;
+		ext->header_extension = in->header_extension;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*ext);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_H264_DEBLOCK_CONTROL: {
+		struct hfi_h264_db_control *in = pdata, *db = prop_data;
+
+		switch (in->mode) {
+		case HFI_H264_DB_MODE_DISABLE:
+		case HFI_H264_DB_MODE_SKIP_SLICE_BOUNDARY:
+		case HFI_H264_DB_MODE_ALL_BOUNDARY:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_H264_DEBLOCK_CONTROL;
+		db->mode = in->mode;
+		db->slice_alpha_offset = in->slice_alpha_offset;
+		db->slice_beta_offset = in->slice_beta_offset;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*db);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_SESSION_QP: {
+		struct hfi_quantization *in = pdata, *quant = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_SESSION_QP;
+		quant->qp_i = in->qp_i;
+		quant->qp_p = in->qp_p;
+		quant->qp_b = in->qp_b;
+		quant->layer_id = in->layer_id;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*quant);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_SESSION_QP_RANGE: {
+		struct hfi_quantization_range *in = pdata, *range = prop_data;
+		u32 min_qp, max_qp;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_SESSION_QP_RANGE;
+		min_qp = in->min_qp;
+		max_qp = in->max_qp;
+
+		/* We'll be packing in the qp, so make sure we
+		 * won't be losing data when masking
+		 */
+		if (min_qp > 0xff || max_qp > 0xff) {
+			ret = -ERANGE;
+			break;
+		}
+
+		/* When creating the packet, pack the qp value as
+		 * 0xiippbb, where ii = qp range for I-frames,
+		 * pp = qp range for P-frames, etc.
+		 */
+		range->min_qp = min_qp | min_qp << 8 | min_qp << 16;
+		range->max_qp = max_qp | max_qp << 8 | max_qp << 16;
+		range->layer_id = in->layer_id;
+
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*range);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_VC1_PERF_CFG: {
+		struct hfi_vc1e_perf_cfg_type *in = pdata, *perf = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_VC1_PERF_CFG;
+
+		memcpy(perf->search_range_x_subsampled,
+		       in->search_range_x_subsampled,
+		       sizeof(perf->search_range_x_subsampled));
+		memcpy(perf->search_range_y_subsampled,
+		       in->search_range_y_subsampled,
+		       sizeof(perf->search_range_y_subsampled));
+
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*perf);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES: {
+		struct hfi_max_num_b_frames *bframes = prop_data;
+		u32 *in = pdata;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES;
+		bframes->max_num_b_frames = *in;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*bframes);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD: {
+		struct hfi_intra_period *in = pdata, *intra = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD;
+		intra->pframes = in->pframes;
+		intra->bframes = in->bframes;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*intra);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_IDR_PERIOD: {
+		struct hfi_idr_period *in = pdata, *idr = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_IDR_PERIOD;
+		idr->idr_period = in->idr_period;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*idr);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_CONCEAL_COLOR: {
+		struct hfi_conceal_color *color = prop_data;
+		u32 *in = pdata;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_CONCEAL_COLOR;
+		color->conceal_color = *in;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*color);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VPE_OPERATIONS: {
+		struct hfi_operations_type *in = pdata, *ops = prop_data;
+
+		switch (in->rotation) {
+		case HFI_ROTATE_NONE:
+		case HFI_ROTATE_90:
+		case HFI_ROTATE_180:
+		case HFI_ROTATE_270:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		switch (in->flip) {
+		case HFI_FLIP_NONE:
+		case HFI_FLIP_HORIZONTAL:
+		case HFI_FLIP_VERTICAL:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VPE_OPERATIONS;
+		ops->rotation = in->rotation;
+		ops->flip = in->flip;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*ops);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH: {
+		struct hfi_intra_refresh *in = pdata, *intra = prop_data;
+
+		switch (in->mode) {
+		case HFI_INTRA_REFRESH_NONE:
+		case HFI_INTRA_REFRESH_ADAPTIVE:
+		case HFI_INTRA_REFRESH_CYCLIC:
+		case HFI_INTRA_REFRESH_CYCLIC_ADAPTIVE:
+		case HFI_INTRA_REFRESH_RANDOM:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH;
+		intra->mode = in->mode;
+		intra->air_mbs = in->air_mbs;
+		intra->air_ref = in->air_ref;
+		intra->cir_mbs = in->cir_mbs;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*intra);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_MULTI_SLICE_CONTROL: {
+		struct hfi_multi_slice_control *in = pdata, *multi = prop_data;
+
+		switch (in->multi_slice) {
+		case HFI_MULTI_SLICE_OFF:
+		case HFI_MULTI_SLICE_GOB:
+		case HFI_MULTI_SLICE_BY_MB_COUNT:
+		case HFI_MULTI_SLICE_BY_BYTE_COUNT:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_MULTI_SLICE_CONTROL;
+		multi->multi_slice = in->multi_slice;
+		multi->slice_size = in->slice_size;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*multi);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_SLICE_DELIVERY_MODE: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_SLICE_DELIVERY_MODE;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_H264_VUI_TIMING_INFO: {
+		struct hfi_h264_vui_timing_info *in = pdata, *vui = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_H264_VUI_TIMING_INFO;
+		vui->enable = in->enable;
+		vui->fixed_framerate = in->fixed_framerate;
+		vui->time_scale = in->time_scale;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*vui);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VPE_DEINTERLACE: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VPE_DEINTERLACE;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_H264_GENERATE_AUDNAL: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_H264_GENERATE_AUDNAL;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE: {
+		struct hfi_buffer_alloc_mode *in = pdata, *mode = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
+		mode->type = in->type;
+		mode->mode = in->mode;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*mode);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_FRAME_ASSEMBLY: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_FRAME_ASSEMBLY;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_H264_VUI_BITSTREAM_RESTRC: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] =
+			HFI_PROPERTY_PARAM_VENC_H264_VUI_BITSTREAM_RESTRC;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_PRESERVE_TEXT_QUALITY: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_PRESERVE_TEXT_QUALITY;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_SCS_THRESHOLD: {
+		struct hfi_scs_threshold *thres = prop_data;
+		u32 *in = pdata;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_SCS_THRESHOLD;
+		thres->threshold_value = *in;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*thres);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_MVC_BUFFER_LAYOUT: {
+		struct hfi_mvc_buffer_layout_descp_type *in = pdata;
+		struct hfi_mvc_buffer_layout_descp_type *mvc = prop_data;
+
+		switch (in->layout_type) {
+		case HFI_MVC_BUFFER_LAYOUT_TOP_BOTTOM:
+		case HFI_MVC_BUFFER_LAYOUT_SEQ:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_MVC_BUFFER_LAYOUT;
+		mvc->layout_type = in->layout_type;
+		mvc->bright_view_first = in->bright_view_first;
+		mvc->ngap = in->ngap;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*mvc);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_LTRMODE: {
+		struct hfi_ltr_mode *in = pdata, *ltr = prop_data;
+
+		switch (in->ltr_mode) {
+		case HFI_LTR_MODE_DISABLE:
+		case HFI_LTR_MODE_MANUAL:
+		case HFI_LTR_MODE_PERIODIC:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_LTRMODE;
+		ltr->ltr_mode = in->ltr_mode;
+		ltr->ltr_count = in->ltr_count;
+		ltr->trust_mode = in->trust_mode;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*ltr);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_USELTRFRAME: {
+		struct hfi_ltr_use *in = pdata, *ltr_use = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_USELTRFRAME;
+		ltr_use->frames = in->frames;
+		ltr_use->ref_ltr = in->ref_ltr;
+		ltr_use->use_constrnt = in->use_constrnt;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*ltr_use);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME: {
+		struct hfi_ltr_mark *in = pdata, *ltr_mark = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME;
+		ltr_mark->mark_frame = in->mark_frame;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*ltr_mark);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER: {
+		u32 *in = pdata;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER;
+		pkt->data[1] = *in;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER: {
+		u32 *in = pdata;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER;
+		pkt->data[1] = *in;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_DISABLE_RC_TIMESTAMP: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_DISABLE_RC_TIMESTAMP;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_INITIAL_QP: {
+		struct hfi_initial_quantization *in = pdata, *quant = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_INITIAL_QP;
+		quant->init_qp_enable = in->init_qp_enable;
+		quant->qp_i = in->qp_i;
+		quant->qp_p = in->qp_p;
+		quant->qp_b = in->qp_b;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*quant);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VPE_COLOR_SPACE_CONVERSION: {
+		struct hfi_vpe_color_space_conversion *in = pdata;
+		struct hfi_vpe_color_space_conversion *csc = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VPE_COLOR_SPACE_CONVERSION;
+		memcpy(csc->csc_matrix, in->csc_matrix,
+			sizeof(csc->csc_matrix));
+		memcpy(csc->csc_bias, in->csc_bias, sizeof(csc->csc_bias));
+		memcpy(csc->csc_limit, in->csc_limit, sizeof(csc->csc_limit));
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*csc);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_VPX_ERROR_RESILIENCE_MODE: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] =
+			HFI_PROPERTY_PARAM_VENC_VPX_ERROR_RESILIENCE_MODE;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_H264_NAL_SVC_EXT: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_H264_NAL_SVC_EXT;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_PERF_MODE: {
+		u32 *in = pdata;
+
+		pkt->data[0] = HFI_PROPERTY_CONFIG_VENC_PERF_MODE;
+		pkt->data[1] = *in;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_HIER_B_MAX_NUM_ENH_LAYER: {
+		u32 *in = pdata;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_HIER_B_MAX_NUM_ENH_LAYER;
+		pkt->data[1] = *in;
+		pkt->shdr.hdr.size += sizeof(u32) * 2;
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VDEC_NONCP_OUTPUT2: {
+		struct hfi_enable *in = pdata, *en = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_NONCP_OUTPUT2;
+		en->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*en);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_HIER_P_HYBRID_MODE: {
+		struct hfi_hybrid_hierp *in = pdata, *hierp = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_HIER_P_HYBRID_MODE;
+		hierp->layers = in->layers;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*hierp);
+		break;
+	}
+
+	/* FOLLOWING PROPERTIES ARE NOT IMPLEMENTED IN CORE YET */
+	case HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS:
+	case HFI_PROPERTY_CONFIG_PRIORITY:
+	case HFI_PROPERTY_CONFIG_BATCH_INFO:
+	case HFI_PROPERTY_SYS_IDLE_INDICATOR:
+	case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED:
+	case HFI_PROPERTY_PARAM_INTERLACE_FORMAT_SUPPORTED:
+	case HFI_PROPERTY_PARAM_CHROMA_SITE:
+	case HFI_PROPERTY_PARAM_PROPERTIES_SUPPORTED:
+	case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED:
+	case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED:
+	case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SUPPORTED:
+	case HFI_PROPERTY_PARAM_MULTI_VIEW_FORMAT:
+	case HFI_PROPERTY_PARAM_MAX_SEQUENCE_HEADER_SIZE:
+	case HFI_PROPERTY_PARAM_CODEC_SUPPORTED:
+	case HFI_PROPERTY_PARAM_VDEC_MULTI_VIEW_SELECT:
+	case HFI_PROPERTY_PARAM_VDEC_MB_QUANTIZATION:
+	case HFI_PROPERTY_PARAM_VDEC_NUM_CONCEALED_MB:
+	case HFI_PROPERTY_PARAM_VDEC_H264_ENTROPY_SWITCHING:
+	case HFI_PROPERTY_PARAM_VENC_MULTI_SLICE_INFO:
+	default:
+		return -ENOTSUPP;
+	}
+
+	return ret;
+}
+
+static int
+pkt_session_set_property_3xx(struct hfi_session_set_property_pkt *pkt,
+			     struct hfi_inst *inst, u32 ptype, void *pdata)
+{
+	void *prop_data = &pkt->data[1];
+	int ret = 0;
+
+	if (!pkt || !inst || !pdata)
+		return -EINVAL;
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_PROPERTY;
+	pkt->shdr.session_id = hash32_ptr(inst);
+	pkt->num_properties = 1;
+
+	/*
+	 * Any session set property which is different in 3XX packetization
+	 * should be added as a new case below. All unchanged session set
+	 * properties will be handled in the default case.
+	 */
+	switch (ptype) {
+	case HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM: {
+		struct hfi_multi_stream *in = pdata;
+		struct hfi_multi_stream_3x *multi = prop_data;
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
+		multi->buffer_type = in->buffer_type;
+		multi->enable = in->enable;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*multi);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH: {
+		struct hfi_intra_refresh *in = pdata;
+		struct hfi_intra_refresh_3x *intra = prop_data;
+
+		switch (in->mode) {
+		case HFI_INTRA_REFRESH_NONE:
+		case HFI_INTRA_REFRESH_ADAPTIVE:
+		case HFI_INTRA_REFRESH_CYCLIC:
+		case HFI_INTRA_REFRESH_CYCLIC_ADAPTIVE:
+		case HFI_INTRA_REFRESH_RANDOM:
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		pkt->data[0] = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH;
+		intra->mode = in->mode;
+		intra->mbs = in->cir_mbs;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*intra);
+		break;
+	}
+	default:
+		ret = pkt_session_set_property(pkt, inst, ptype, pdata);
+		break;
+	}
+
+	return ret;
+}
+
+static int pkt_ssr_cmd(u32 trigger_type, struct hfi_sys_test_ssr_pkt *pkt)
+{
+	switch (trigger_type) {
+	case HFI_TEST_SSR_SW_ERR_FATAL:
+	case HFI_TEST_SSR_SW_DIV_BY_ZERO:
+	case HFI_TEST_SSR_HW_WDOG_IRQ:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	pkt->hdr.size = sizeof(*pkt);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_TEST_SSR;
+	pkt->trigger_type = trigger_type;
+
+	return 0;
+}
+
+static void pkt_sys_image_version(struct hfi_sys_get_property_pkt *pkt)
+{
+	pkt->hdr.size = sizeof(*pkt);
+	pkt->hdr.pkt_type = HFI_CMD_SYS_GET_PROPERTY;
+	pkt->num_properties = 1;
+	pkt->data[0] = HFI_PROPERTY_SYS_IMAGE_VERSION;
+}
+
+static const struct hfi_packetization_ops hfi_default = {
+	.sys_init = pkt_sys_init,
+	.sys_pc_prep = pkt_sys_pc_prep,
+	.sys_idle_indicator = pkt_sys_idle_indicator,
+	.sys_power_control = pkt_sys_power_control,
+	.sys_set_resource = pkt_sys_set_resource,
+	.sys_release_resource = pkt_sys_unset_resource,
+	.sys_debug_config = pkt_sys_debug_config,
+	.sys_coverage_config = pkt_sys_coverage_config,
+	.sys_ping = pkt_sys_ping,
+	.sys_image_version = pkt_sys_image_version,
+	.ssr_cmd = pkt_ssr_cmd,
+	.session_init = pkt_session_init,
+	.session_cmd = pkt_session_cmd,
+	.session_set_buffers = pkt_session_set_buffers,
+	.session_release_buffers = pkt_session_release_buffers,
+	.session_etb_decoder = pkt_session_etb_decoder,
+	.session_etb_encoder = pkt_session_etb_encoder,
+	.session_ftb = pkt_session_ftb,
+	.session_parse_seq_header = pkt_session_parse_seq_header,
+	.session_get_seq_hdr = pkt_session_get_seq_hdr,
+	.session_flush = pkt_session_flush,
+	.session_get_property = pkt_session_get_property,
+	.session_set_property = pkt_session_set_property,
+};
+
+static const struct hfi_packetization_ops *get_3xx_ops(void)
+{
+	static struct hfi_packetization_ops hfi_3xx;
+
+	hfi_3xx = hfi_default;
+	hfi_3xx.session_set_property = pkt_session_set_property_3xx;
+
+	return &hfi_3xx;
+}
+
+const struct hfi_packetization_ops *
+hfi_get_pkt_ops(enum hfi_packetization_type type)
+{
+	switch (type) {
+	case HFI_PACKETIZATION_LEGACY:
+		return &hfi_default;
+	case HFI_PACKETIZATION_3XX:
+		return get_3xx_ops();
+	}
+
+	return NULL;
+}
diff --git a/drivers/media/platform/qcom/vidc/hfi_cmds.h b/drivers/media/platform/qcom/vidc/hfi_cmds.h
new file mode 100644
index 000000000000..e57a0d4d2b9d
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_cmds.h
@@ -0,0 +1,338 @@
+/*
+ * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __HFI_CMDS_H__
+#define __HFI_CMDS_H__
+
+#include "hfi_helper.h"
+#include "hfi.h"
+
+/* commands */
+#define HFI_CMD_SYS_COMMON_START	(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_COMMON_OFFSET +	\
+					 HFI_CMD_START_OFFSET + 0x0000)
+#define HFI_CMD_SYS_INIT		(HFI_CMD_SYS_COMMON_START + 0x1)
+#define HFI_CMD_SYS_PC_PREP		(HFI_CMD_SYS_COMMON_START + 0x2)
+#define HFI_CMD_SYS_SET_RESOURCE	(HFI_CMD_SYS_COMMON_START + 0x3)
+#define HFI_CMD_SYS_RELEASE_RESOURCE	(HFI_CMD_SYS_COMMON_START + 0x4)
+#define HFI_CMD_SYS_SET_PROPERTY	(HFI_CMD_SYS_COMMON_START + 0x5)
+#define HFI_CMD_SYS_GET_PROPERTY	(HFI_CMD_SYS_COMMON_START + 0x6)
+#define HFI_CMD_SYS_SESSION_INIT	(HFI_CMD_SYS_COMMON_START + 0x7)
+#define HFI_CMD_SYS_SESSION_END		(HFI_CMD_SYS_COMMON_START + 0x8)
+#define HFI_CMD_SYS_SET_BUFFERS		(HFI_CMD_SYS_COMMON_START + 0x9)
+#define HFI_CMD_SYS_TEST_START		(HFI_CMD_SYS_COMMON_START + 0x100)
+#define HFI_CMD_SYS_TEST_SSR		(HFI_CMD_SYS_TEST_START + 0x1)
+
+#define HFI_CMD_SESSION_COMMON_START	(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_COMMON_OFFSET +	\
+					 HFI_CMD_START_OFFSET + 0x1000)
+#define HFI_CMD_SESSION_SET_PROPERTY	(HFI_CMD_SESSION_COMMON_START + 0x1)
+#define HFI_CMD_SESSION_SET_BUFFERS	(HFI_CMD_SESSION_COMMON_START + 0x2)
+#define HFI_CMD_SESSION_GET_SEQUENCE_HEADER	\
+					(HFI_CMD_SESSION_COMMON_START + 0x3)
+
+#define HFI_CMD_SYS_OX_START		(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_OX_OFFSET +		\
+					 HFI_CMD_START_OFFSET + 0x0000)
+#define HFI_CMD_SYS_SESSION_ABORT	(HFI_CMD_SYS_OX_START + 0x1)
+#define HFI_CMD_SYS_PING		(HFI_CMD_SYS_OX_START + 0x2)
+
+#define HFI_CMD_SESSION_OX_START	(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_OX_OFFSET +		\
+					 HFI_CMD_START_OFFSET + 0x1000)
+#define HFI_CMD_SESSION_LOAD_RESOURCES	(HFI_CMD_SESSION_OX_START + 0x1)
+#define HFI_CMD_SESSION_START		(HFI_CMD_SESSION_OX_START + 0x2)
+#define HFI_CMD_SESSION_STOP		(HFI_CMD_SESSION_OX_START + 0x3)
+#define HFI_CMD_SESSION_EMPTY_BUFFER	(HFI_CMD_SESSION_OX_START + 0x4)
+#define HFI_CMD_SESSION_FILL_BUFFER	(HFI_CMD_SESSION_OX_START + 0x5)
+#define HFI_CMD_SESSION_SUSPEND		(HFI_CMD_SESSION_OX_START + 0x6)
+#define HFI_CMD_SESSION_RESUME		(HFI_CMD_SESSION_OX_START + 0x7)
+#define HFI_CMD_SESSION_FLUSH		(HFI_CMD_SESSION_OX_START + 0x8)
+#define HFI_CMD_SESSION_GET_PROPERTY	(HFI_CMD_SESSION_OX_START + 0x9)
+#define HFI_CMD_SESSION_PARSE_SEQUENCE_HEADER	\
+					(HFI_CMD_SESSION_OX_START + 0xa)
+#define HFI_CMD_SESSION_RELEASE_BUFFERS		\
+					(HFI_CMD_SESSION_OX_START + 0xb)
+#define HFI_CMD_SESSION_RELEASE_RESOURCES	\
+					(HFI_CMD_SESSION_OX_START + 0xc)
+
+/* command packets */
+struct hfi_sys_init_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 arch_type;
+};
+
+struct hfi_sys_pc_prep_pkt {
+	struct hfi_pkt_hdr hdr;
+};
+
+struct hfi_sys_set_resource_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 resource_handle;
+	u32 resource_type;
+	u32 resource_data[1];
+};
+
+struct hfi_sys_release_resource_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 resource_type;
+	u32 resource_handle;
+};
+
+struct hfi_sys_set_property_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_sys_get_property_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_sys_set_buffers_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 buffer_type;
+	u32 buffer_size;
+	u32 num_buffers;
+	u32 buffer_addr[1];
+};
+
+struct hfi_sys_ping_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 client_data;
+};
+
+struct hfi_session_init_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 session_domain;
+	u32 session_codec;
+};
+
+struct hfi_session_end_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_abort_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_set_property_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 num_properties;
+	u32 data[0];
+};
+
+struct hfi_session_set_buffers_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 buffer_type;
+	u32 buffer_size;
+	u32 extradata_size;
+	u32 min_buffer_size;
+	u32 num_buffers;
+	u32 buffer_info[1];
+};
+
+struct hfi_session_get_sequence_header_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 buffer_len;
+	u32 packet_buffer;
+};
+
+struct hfi_session_load_resources_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_start_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_stop_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_empty_buffer_compressed_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 time_stamp_hi;
+	u32 time_stamp_lo;
+	u32 flags;
+	u32 mark_target;
+	u32 mark_data;
+	u32 offset;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 input_tag;
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 data[1];
+};
+
+struct hfi_session_empty_buffer_uncompressed_plane0_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 view_id;
+	u32 time_stamp_hi;
+	u32 time_stamp_lo;
+	u32 flags;
+	u32 mark_target;
+	u32 mark_data;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 offset;
+	u32 input_tag;
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 data[1];
+};
+
+struct hfi_session_empty_buffer_uncompressed_plane1_pkt {
+	u32 flags;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 offset;
+	u32 packet_buffer2;
+	u32 data[1];
+};
+
+struct hfi_session_empty_buffer_uncompressed_plane2_pkt {
+	u32 flags;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 offset;
+	u32 packet_buffer3;
+	u32 data[1];
+};
+
+struct hfi_session_fill_buffer_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 stream_id;
+	u32 offset;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 output_tag;
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 data[1];
+};
+
+struct hfi_session_flush_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 flush_type;
+};
+
+struct hfi_session_suspend_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_resume_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_get_property_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_session_release_buffer_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 buffer_type;
+	u32 buffer_size;
+	u32 extradata_size;
+	u32 response_req;
+	u32 num_buffers;
+	u32 buffer_info[1];
+};
+
+struct hfi_session_release_resources_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+struct hfi_session_parse_sequence_header_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 header_len;
+	u32 packet_buffer;
+};
+
+struct hfi_sfr {
+	u32 buf_size;
+	u8 data[1];
+};
+
+struct hfi_sys_test_ssr_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 trigger_type;
+};
+
+#define call_hfi_pkt_op(hfi, op, args...)			\
+	(((hfi) && (hfi)->pkt_ops && (hfi)->pkt_ops->op) ?	\
+	((hfi)->pkt_ops->op(args)) : -EINVAL)
+
+struct hfi_packetization_ops {
+	void (*sys_init)(struct hfi_sys_init_pkt *pkt, u32 arch_type);
+	void (*sys_pc_prep)(struct hfi_sys_pc_prep_pkt *pkt);
+	void (*sys_idle_indicator)(struct hfi_sys_set_property_pkt *pkt,
+				   u32 enable);
+	void (*sys_power_control)(struct hfi_sys_set_property_pkt *pkt,
+				  u32 enable);
+	int (*sys_set_resource)(struct hfi_sys_set_resource_pkt *pkt,
+				u32 id, u32 size, u32 addr, void *cookie);
+	int (*sys_release_resource)(struct hfi_sys_release_resource_pkt *pkt,
+				    u32 id, u32 size, void *cookie);
+	void (*sys_debug_config)(struct hfi_sys_set_property_pkt *pkt, u32 mode,
+				 u32 config);
+	void (*sys_coverage_config)(struct hfi_sys_set_property_pkt *pkt,
+				    u32 mode);
+	void (*sys_ping)(struct hfi_sys_ping_pkt *pkt, u32 cookie);
+	void (*sys_image_version)(struct hfi_sys_get_property_pkt *pkt);
+	int (*ssr_cmd)(u32 trigger_type, struct hfi_sys_test_ssr_pkt *pkt);
+	int (*session_init)(struct hfi_session_init_pkt *pkt,
+			    struct hfi_inst *inst, u32 session_type, u32 codec);
+	void (*session_cmd)(struct hfi_session_pkt *pkt, u32 pkt_type,
+			    struct hfi_inst *inst);
+	int (*session_set_buffers)(struct hfi_session_set_buffers_pkt *pkt,
+				   struct hfi_inst *inst,
+				   struct hfi_buffer_desc *bd);
+	int (*session_release_buffers)(
+		struct hfi_session_release_buffer_pkt *pkt,
+		struct hfi_inst *inst, struct hfi_buffer_desc *bd);
+	int (*session_etb_decoder)(
+		struct hfi_session_empty_buffer_compressed_pkt *pkt,
+		struct hfi_inst *inst,
+		struct hfi_frame_data *input_frame);
+	int (*session_etb_encoder)(
+		struct hfi_session_empty_buffer_uncompressed_plane0_pkt *pkt,
+		struct hfi_inst *inst, struct hfi_frame_data *input_frame);
+	int (*session_ftb)(struct hfi_session_fill_buffer_pkt *pkt,
+		struct hfi_inst *inst, struct hfi_frame_data *output_frame);
+	int (*session_parse_seq_header)(
+		struct hfi_session_parse_sequence_header_pkt *pkt,
+		struct hfi_inst *inst, u32 seq_hdr, u32 seq_hdr_len);
+	int (*session_get_seq_hdr)(
+		struct hfi_session_get_sequence_header_pkt *pkt,
+		struct hfi_inst *inst, u32 seq_hdr, u32 seq_hdr_len);
+	int (*session_flush)(struct hfi_session_flush_pkt *pkt,
+		struct hfi_inst *inst, u32 flush_mode);
+	int (*session_get_property)(
+		struct hfi_session_get_property_pkt *pkt,
+		struct hfi_inst *inst, u32 ptype);
+	int (*session_set_property)(struct hfi_session_set_property_pkt *pkt,
+				    struct hfi_inst *inst,
+				    u32 ptype, void *pdata);
+};
+
+const struct hfi_packetization_ops *
+hfi_get_pkt_ops(enum hfi_packetization_type);
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/hfi_helper.h b/drivers/media/platform/qcom/vidc/hfi_helper.h
new file mode 100644
index 000000000000..c4d2c263bf03
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_helper.h
@@ -0,0 +1,1143 @@
+/*
+ * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __HFI_HELPER_H__
+#define __HFI_HELPER_H__
+
+#define HFI_COMMON_BASE				0
+
+#define HFI_DOMAIN_BASE_COMMON			(HFI_COMMON_BASE + 0)
+#define HFI_DOMAIN_BASE_VDEC			(HFI_COMMON_BASE + 0x1000000)
+#define HFI_DOMAIN_BASE_VENC			(HFI_COMMON_BASE + 0x2000000)
+#define HFI_DOMAIN_BASE_VPE			(HFI_COMMON_BASE + 0x3000000)
+
+#define HFI_VIDEO_ARCH_OX			(HFI_COMMON_BASE + 0x1)
+
+#define HFI_ARCH_COMMON_OFFSET			0
+
+#define HFI_CMD_START_OFFSET			0x10000
+#define HFI_MSG_START_OFFSET			0x20000
+
+#define HFI_ERR_NONE				HFI_COMMON_BASE
+#define HFI_ERR_SYS_FATAL			(HFI_COMMON_BASE + 0x1)
+#define HFI_ERR_SYS_INVALID_PARAMETER		(HFI_COMMON_BASE + 0x2)
+#define HFI_ERR_SYS_VERSION_MISMATCH		(HFI_COMMON_BASE + 0x3)
+#define HFI_ERR_SYS_INSUFFICIENT_RESOURCES	(HFI_COMMON_BASE + 0x4)
+#define HFI_ERR_SYS_MAX_SESSIONS_REACHED	(HFI_COMMON_BASE + 0x5)
+#define HFI_ERR_SYS_UNSUPPORTED_CODEC		(HFI_COMMON_BASE + 0x6)
+#define HFI_ERR_SYS_SESSION_IN_USE		(HFI_COMMON_BASE + 0x7)
+#define HFI_ERR_SYS_SESSION_ID_OUT_OF_RANGE	(HFI_COMMON_BASE + 0x8)
+#define HFI_ERR_SYS_UNSUPPORTED_DOMAIN		(HFI_COMMON_BASE + 0x9)
+
+#define HFI_ERR_SESSION_FATAL			(HFI_COMMON_BASE + 0x1001)
+#define HFI_ERR_SESSION_INVALID_PARAMETER	(HFI_COMMON_BASE + 0x1002)
+#define HFI_ERR_SESSION_BAD_POINTER		(HFI_COMMON_BASE + 0x1003)
+#define HFI_ERR_SESSION_INVALID_SESSION_ID	(HFI_COMMON_BASE + 0x1004)
+#define HFI_ERR_SESSION_INVALID_STREAM_ID	(HFI_COMMON_BASE + 0x1005)
+#define HFI_ERR_SESSION_INCORRECT_STATE_OPERATION	\
+						(HFI_COMMON_BASE + 0x1006)
+#define HFI_ERR_SESSION_UNSUPPORTED_PROPERTY	(HFI_COMMON_BASE + 0x1007)
+#define HFI_ERR_SESSION_UNSUPPORTED_SETTING	(HFI_COMMON_BASE + 0x1008)
+#define HFI_ERR_SESSION_INSUFFICIENT_RESOURCES	(HFI_COMMON_BASE + 0x1009)
+#define HFI_ERR_SESSION_STREAM_CORRUPT_OUTPUT_STALLED	\
+						(HFI_COMMON_BASE + 0x100a)
+#define HFI_ERR_SESSION_STREAM_CORRUPT		(HFI_COMMON_BASE + 0x100b)
+#define HFI_ERR_SESSION_ENC_OVERFLOW		(HFI_COMMON_BASE + 0x100c)
+#define HFI_ERR_SESSION_UNSUPPORTED_STREAM	(HFI_COMMON_BASE + 0x100d)
+#define HFI_ERR_SESSION_CMDSIZE			(HFI_COMMON_BASE + 0x100e)
+#define HFI_ERR_SESSION_UNSUPPORT_CMD		(HFI_COMMON_BASE + 0x100f)
+#define HFI_ERR_SESSION_UNSUPPORT_BUFFERTYPE	(HFI_COMMON_BASE + 0x1010)
+#define HFI_ERR_SESSION_BUFFERCOUNT_TOOSMALL	(HFI_COMMON_BASE + 0x1011)
+#define HFI_ERR_SESSION_INVALID_SCALE_FACTOR	(HFI_COMMON_BASE + 0x1012)
+#define HFI_ERR_SESSION_UPSCALE_NOT_SUPPORTED	(HFI_COMMON_BASE + 0x1013)
+
+#define HFI_EVENT_SYS_ERROR			(HFI_COMMON_BASE + 0x1)
+#define HFI_EVENT_SESSION_ERROR			(HFI_COMMON_BASE + 0x2)
+
+#define HFI_OX_BASE				0x01000000
+
+#define HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES	\
+						(HFI_OX_BASE + 0x1)
+#define HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES	\
+						(HFI_OX_BASE + 0x2)
+#define HFI_EVENT_SESSION_SEQUENCE_CHANGED	(HFI_OX_BASE + 0x3)
+#define HFI_EVENT_SESSION_PROPERTY_CHANGED	(HFI_OX_BASE + 0x4)
+#define HFI_EVENT_SESSION_LTRUSE_FAILED		(HFI_OX_BASE + 0x5)
+#define HFI_EVENT_RELEASE_BUFFER_REFERENCE	(HFI_OX_BASE + 0x6)
+
+#define HFI_BUFFERFLAG_EOS				0x00000001
+#define HFI_BUFFERFLAG_STARTTIME			0x00000002
+#define HFI_BUFFERFLAG_DECODEONLY			0x00000004
+#define HFI_BUFFERFLAG_DATACORRUPT			0x00000008
+#define HFI_BUFFERFLAG_ENDOFFRAME			0x00000010
+#define HFI_BUFFERFLAG_SYNCFRAME			0x00000020
+#define HFI_BUFFERFLAG_EXTRADATA			0x00000040
+#define HFI_BUFFERFLAG_CODECCONFIG			0x00000080
+#define HFI_BUFFERFLAG_TIMESTAMPINVALID			0x00000100
+#define HFI_BUFFERFLAG_READONLY				0x00000200
+#define HFI_BUFFERFLAG_ENDOFSUBFRAME			0x00000400
+#define HFI_BUFFERFLAG_EOSEQ				0x00200000
+#define HFI_BUFFERFLAG_MBAFF				0x08000000
+#define HFI_BUFFERFLAG_VPE_YUV_601_709_CSC_CLAMP	0x10000000
+#define HFI_BUFFERFLAG_DROP_FRAME			0x20000000
+#define HFI_BUFFERFLAG_TEI				0x40000000
+#define HFI_BUFFERFLAG_DISCONTINUITY			0x80000000
+
+#define HFI_ERR_SESSION_EMPTY_BUFFER_DONE_OUTPUT_PENDING	\
+							(HFI_OX_BASE + 0x1001)
+#define HFI_ERR_SESSION_SAME_STATE_OPERATION	\
+							(HFI_OX_BASE + 0x1002)
+#define HFI_ERR_SESSION_SYNC_FRAME_NOT_DETECTED	\
+							(HFI_OX_BASE + 0x1003)
+#define HFI_ERR_SESSION_START_CODE_NOT_FOUND	\
+							(HFI_OX_BASE + 0x1004)
+
+#define HFI_FLUSH_INPUT					(HFI_OX_BASE + 0x1)
+#define HFI_FLUSH_OUTPUT				(HFI_OX_BASE + 0x2)
+#define HFI_FLUSH_OUTPUT2				(HFI_OX_BASE + 0x3)
+#define HFI_FLUSH_ALL					(HFI_OX_BASE + 0x4)
+
+#define HFI_EXTRADATA_NONE				0x00000000
+#define HFI_EXTRADATA_MB_QUANTIZATION			0x00000001
+#define HFI_EXTRADATA_INTERLACE_VIDEO			0x00000002
+#define HFI_EXTRADATA_VC1_FRAMEDISP			0x00000003
+#define HFI_EXTRADATA_VC1_SEQDISP			0x00000004
+#define HFI_EXTRADATA_TIMESTAMP				0x00000005
+#define HFI_EXTRADATA_S3D_FRAME_PACKING			0x00000006
+#define HFI_EXTRADATA_FRAME_RATE			0x00000007
+#define HFI_EXTRADATA_PANSCAN_WINDOW			0x00000008
+#define HFI_EXTRADATA_RECOVERY_POINT_SEI		0x00000009
+#define HFI_EXTRADATA_MPEG2_SEQDISP			0x0000000d
+#define HFI_EXTRADATA_STREAM_USERDATA			0x0000000e
+#define HFI_EXTRADATA_FRAME_QP				0x0000000f
+#define HFI_EXTRADATA_FRAME_BITS_INFO			0x00000010
+#define HFI_EXTRADATA_MULTISLICE_INFO			0x7f100000
+#define HFI_EXTRADATA_NUM_CONCEALED_MB			0x7f100001
+#define HFI_EXTRADATA_INDEX				0x7f100002
+#define HFI_EXTRADATA_METADATA_LTR			0x7f100004
+#define HFI_EXTRADATA_METADATA_FILLER			0x7fe00002
+
+#define HFI_INDEX_EXTRADATA_INPUT_CROP			0x0700000e
+#define HFI_INDEX_EXTRADATA_DIGITAL_ZOOM		0x07000010
+#define HFI_INDEX_EXTRADATA_ASPECT_RATIO		0x7f100003
+
+#define HFI_INTERLACE_FRAME_PROGRESSIVE			0x01
+#define HFI_INTERLACE_INTERLEAVE_FRAME_TOPFIELDFIRST	0x02
+#define HFI_INTERLACE_INTERLEAVE_FRAME_BOTTOMFIELDFIRST	0x04
+#define HFI_INTERLACE_FRAME_TOPFIELDFIRST		0x08
+#define HFI_INTERLACE_FRAME_BOTTOMFIELDFIRST		0x10
+
+#define HFI_ARCH_OX_OFFSET				0x200000
+
+#define HFI_PROPERTY_SYS_OX_START	\
+	(HFI_DOMAIN_BASE_COMMON + HFI_ARCH_OX_OFFSET + 0x0000)
+
+#define HFI_PROPERTY_PARAM_OX_START	\
+	(HFI_DOMAIN_BASE_COMMON + HFI_ARCH_OX_OFFSET + 0x1000)
+#define HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x1)
+#define HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x2)
+#define HFI_PROPERTY_PARAM_INTERLACE_FORMAT_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x3)
+#define HFI_PROPERTY_PARAM_CHROMA_SITE	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x4)
+#define HFI_PROPERTY_PARAM_EXTRA_DATA_HEADER_CONFIG	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x5)
+#define HFI_PROPERTY_PARAM_INDEX_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x6)
+#define HFI_PROPERTY_PARAM_DIVX_FORMAT	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x7)
+#define HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x8)
+#define HFI_PROPERTY_PARAM_S3D_FRAME_PACKING_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_OX_START + 0x9)
+#define HFI_PROPERTY_PARAM_ERR_DETECTION_CODE_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_OX_START + 0xa)
+#define HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_OX_START + 0xb)
+#define HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL	\
+	(HFI_PROPERTY_PARAM_OX_START + 0xc)
+#define HFI_PROPERTY_PARAM_BUFFER_DISPLAY_HOLD_COUNT_ACTUAL	\
+	(HFI_PROPERTY_PARAM_OX_START + 0xd)
+
+#define HFI_PROPERTY_CONFIG_OX_START	\
+	(HFI_DOMAIN_BASE_COMMON + HFI_ARCH_OX_OFFSET + 0x2000)
+#define HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS	\
+	(HFI_PROPERTY_CONFIG_OX_START + 0x1)
+#define HFI_PROPERTY_CONFIG_REALTIME	\
+	(HFI_PROPERTY_CONFIG_OX_START + 0x2)
+#define HFI_PROPERTY_CONFIG_PRIORITY	\
+	(HFI_PROPERTY_CONFIG_OX_START + 0x3)
+#define HFI_PROPERTY_CONFIG_BATCH_INFO	\
+	(HFI_PROPERTY_CONFIG_OX_START + 0x4)
+
+#define HFI_PROPERTY_PARAM_VDEC_OX_START	\
+	(HFI_DOMAIN_BASE_VDEC + HFI_ARCH_OX_OFFSET + 0x3000)
+#define HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x1)
+#define HFI_PROPERTY_PARAM_VDEC_DISPLAY_PICTURE_BUFFER_COUNT	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x2)
+#define HFI_PROPERTY_PARAM_VDEC_MULTI_VIEW_SELECT	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x3)
+#define HFI_PROPERTY_PARAM_VDEC_PICTURE_TYPE_DECODE	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x4)
+#define HFI_PROPERTY_PARAM_VDEC_OUTPUT_ORDER	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x5)
+#define HFI_PROPERTY_PARAM_VDEC_MB_QUANTIZATION	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x6)
+#define HFI_PROPERTY_PARAM_VDEC_NUM_CONCEALED_MB	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x7)
+#define HFI_PROPERTY_PARAM_VDEC_H264_ENTROPY_SWITCHING	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x8)
+#define HFI_PROPERTY_PARAM_VDEC_OUTPUT2_KEEP_ASPECT_RATIO	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x9)
+#define HFI_PROPERTY_PARAM_VDEC_FRAME_RATE_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0xa)
+#define HFI_PROPERTY_PARAM_VDEC_PANSCAN_WNDW_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0xb)
+#define HFI_PROPERTY_PARAM_VDEC_RECOVERY_POINT_SEI_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0xc)
+#define HFI_PROPERTY_PARAM_VDEC_THUMBNAIL_MODE	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0xd)
+#define HFI_PROPERTY_PARAM_VDEC_FRAME_ASSEMBLY	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0xe)
+#define HFI_PROPERTY_PARAM_VDEC_VC1_FRAMEDISP_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x11)
+#define HFI_PROPERTY_PARAM_VDEC_VC1_SEQDISP_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x12)
+#define HFI_PROPERTY_PARAM_VDEC_TIMESTAMP_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x13)
+#define HFI_PROPERTY_PARAM_VDEC_INTERLACE_VIDEO_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x14)
+#define HFI_PROPERTY_PARAM_VDEC_AVC_SESSION_SELECT	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x15)
+#define HFI_PROPERTY_PARAM_VDEC_MPEG2_SEQDISP_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x16)
+#define HFI_PROPERTY_PARAM_VDEC_STREAM_USERDATA_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x17)
+#define HFI_PROPERTY_PARAM_VDEC_FRAME_QP_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x18)
+#define HFI_PROPERTY_PARAM_VDEC_FRAME_BITS_INFO_EXTRADATA	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x19)
+#define HFI_PROPERTY_PARAM_VDEC_SCS_THRESHOLD	\
+	(HFI_PROPERTY_PARAM_VDEC_OX_START + 0x1a)
+
+#define HFI_PROPERTY_CONFIG_VDEC_OX_START	\
+	(HFI_DOMAIN_BASE_VDEC + HFI_ARCH_OX_OFFSET + 0x0000)
+#define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER	\
+	(HFI_PROPERTY_CONFIG_VDEC_OX_START + 0x1)
+#define HFI_PROPERTY_CONFIG_VDEC_MB_ERROR_MAP_REPORTING	\
+	(HFI_PROPERTY_CONFIG_VDEC_OX_START + 0x2)
+#define HFI_PROPERTY_CONFIG_VDEC_MB_ERROR_MAP	\
+	(HFI_PROPERTY_CONFIG_VDEC_OX_START + 0x3)
+
+#define HFI_PROPERTY_PARAM_VENC_OX_START	\
+	(HFI_DOMAIN_BASE_VENC + HFI_ARCH_OX_OFFSET + 0x5000)
+#define  HFI_PROPERTY_PARAM_VENC_MULTI_SLICE_INFO	\
+	(HFI_PROPERTY_PARAM_VENC_OX_START + 0x1)
+#define  HFI_PROPERTY_PARAM_VENC_H264_IDR_S3D_FRAME_PACKING_NAL	\
+	(HFI_PROPERTY_PARAM_VENC_OX_START + 0x2)
+#define  HFI_PROPERTY_PARAM_VENC_LTR_INFO	\
+	(HFI_PROPERTY_PARAM_VENC_OX_START + 0x3)
+#define  HFI_PROPERTY_PARAM_VENC_MBI_DUMPING	\
+	(HFI_PROPERTY_PARAM_VENC_OX_START + 0x5)
+
+#define HFI_PROPERTY_CONFIG_VENC_OX_START	\
+	(HFI_DOMAIN_BASE_VENC + HFI_ARCH_OX_OFFSET + 0x6000)
+#define HFI_PROPERTY_CONFIG_VENC_FRAME_QP	\
+	(HFI_PROPERTY_CONFIG_VENC_OX_START + 0x1)
+
+#define HFI_PROPERTY_PARAM_VPE_OX_START	\
+	(HFI_DOMAIN_BASE_VPE + HFI_ARCH_OX_OFFSET + 0x7000)
+#define HFI_PROPERTY_PARAM_VPE_COLOR_SPACE_CONVERSION	\
+	(HFI_PROPERTY_PARAM_VPE_OX_START + 0x1)
+
+#define HFI_PROPERTY_CONFIG_VPE_OX_START	\
+	(HFI_DOMAIN_BASE_VPE + HFI_ARCH_OX_OFFSET + 0x8000)
+
+#define HFI_CHROMA_SITE_0			(HFI_OX_BASE + 0x1)
+#define HFI_CHROMA_SITE_1			(HFI_OX_BASE + 0x2)
+#define HFI_CHROMA_SITE_2			(HFI_OX_BASE + 0x3)
+#define HFI_CHROMA_SITE_3			(HFI_OX_BASE + 0x4)
+#define HFI_CHROMA_SITE_4			(HFI_OX_BASE + 0x5)
+#define HFI_CHROMA_SITE_5			(HFI_OX_BASE + 0x6)
+
+#define HFI_PRIORITY_LOW			10
+#define HFI_PRIOIRTY_MEDIUM			20
+#define HFI_PRIORITY_HIGH			30
+
+#define HFI_OUTPUT_ORDER_DISPLAY		(HFI_OX_BASE + 0x1)
+#define HFI_OUTPUT_ORDER_DECODE			(HFI_OX_BASE + 0x2)
+
+#define HFI_RATE_CONTROL_OFF			(HFI_OX_BASE + 0x1)
+#define HFI_RATE_CONTROL_VBR_VFR		(HFI_OX_BASE + 0x2)
+#define HFI_RATE_CONTROL_VBR_CFR		(HFI_OX_BASE + 0x3)
+#define HFI_RATE_CONTROL_CBR_VFR		(HFI_OX_BASE + 0x4)
+#define HFI_RATE_CONTROL_CBR_CFR		(HFI_OX_BASE + 0x5)
+
+#define HFI_VIDEO_CODEC_H264			0x00000002
+#define HFI_VIDEO_CODEC_H263			0x00000004
+#define HFI_VIDEO_CODEC_MPEG1			0x00000008
+#define HFI_VIDEO_CODEC_MPEG2			0x00000010
+#define HFI_VIDEO_CODEC_MPEG4			0x00000020
+#define HFI_VIDEO_CODEC_DIVX_311		0x00000040
+#define HFI_VIDEO_CODEC_DIVX			0x00000080
+#define HFI_VIDEO_CODEC_VC1			0x00000100
+#define HFI_VIDEO_CODEC_SPARK			0x00000200
+#define HFI_VIDEO_CODEC_VP8			0x00001000
+#define HFI_VIDEO_CODEC_HEVC			0x00002000
+#define HFI_VIDEO_CODEC_HEVC_HYBRID		0x00004000
+
+#define HFI_H264_PROFILE_BASELINE		0x00000001
+#define HFI_H264_PROFILE_MAIN			0x00000002
+#define HFI_H264_PROFILE_HIGH			0x00000004
+#define HFI_H264_PROFILE_STEREO_HIGH		0x00000008
+#define HFI_H264_PROFILE_MULTIVIEW_HIGH		0x00000010
+#define HFI_H264_PROFILE_CONSTRAINED_BASE	0x00000020
+#define HFI_H264_PROFILE_CONSTRAINED_HIGH	0x00000040
+
+#define HFI_H264_LEVEL_1			0x00000001
+#define HFI_H264_LEVEL_1b			0x00000002
+#define HFI_H264_LEVEL_11			0x00000004
+#define HFI_H264_LEVEL_12			0x00000008
+#define HFI_H264_LEVEL_13			0x00000010
+#define HFI_H264_LEVEL_2			0x00000020
+#define HFI_H264_LEVEL_21			0x00000040
+#define HFI_H264_LEVEL_22			0x00000080
+#define HFI_H264_LEVEL_3			0x00000100
+#define HFI_H264_LEVEL_31			0x00000200
+#define HFI_H264_LEVEL_32			0x00000400
+#define HFI_H264_LEVEL_4			0x00000800
+#define HFI_H264_LEVEL_41			0x00001000
+#define HFI_H264_LEVEL_42			0x00002000
+#define HFI_H264_LEVEL_5			0x00004000
+#define HFI_H264_LEVEL_51			0x00008000
+#define HFI_H264_LEVEL_52			0x00010000
+
+#define HFI_H263_PROFILE_BASELINE		0x00000001
+
+#define HFI_H263_LEVEL_10			0x00000001
+#define HFI_H263_LEVEL_20			0x00000002
+#define HFI_H263_LEVEL_30			0x00000004
+#define HFI_H263_LEVEL_40			0x00000008
+#define HFI_H263_LEVEL_45			0x00000010
+#define HFI_H263_LEVEL_50			0x00000020
+#define HFI_H263_LEVEL_60			0x00000040
+#define HFI_H263_LEVEL_70			0x00000080
+
+#define HFI_MPEG2_PROFILE_SIMPLE		0x00000001
+#define HFI_MPEG2_PROFILE_MAIN			0x00000002
+#define HFI_MPEG2_PROFILE_422			0x00000004
+#define HFI_MPEG2_PROFILE_SNR			0x00000008
+#define HFI_MPEG2_PROFILE_SPATIAL		0x00000010
+#define HFI_MPEG2_PROFILE_HIGH			0x00000020
+
+#define HFI_MPEG2_LEVEL_LL			0x00000001
+#define HFI_MPEG2_LEVEL_ML			0x00000002
+#define HFI_MPEG2_LEVEL_H14			0x00000004
+#define HFI_MPEG2_LEVEL_HL			0x00000008
+
+#define HFI_MPEG4_PROFILE_SIMPLE		0x00000001
+#define HFI_MPEG4_PROFILE_ADVANCEDSIMPLE	0x00000002
+
+#define HFI_MPEG4_LEVEL_0			0x00000001
+#define HFI_MPEG4_LEVEL_0b			0x00000002
+#define HFI_MPEG4_LEVEL_1			0x00000004
+#define HFI_MPEG4_LEVEL_2			0x00000008
+#define HFI_MPEG4_LEVEL_3			0x00000010
+#define HFI_MPEG4_LEVEL_4			0x00000020
+#define HFI_MPEG4_LEVEL_4a			0x00000040
+#define HFI_MPEG4_LEVEL_5			0x00000080
+#define HFI_MPEG4_LEVEL_6			0x00000100
+#define HFI_MPEG4_LEVEL_7			0x00000200
+#define HFI_MPEG4_LEVEL_8			0x00000400
+#define HFI_MPEG4_LEVEL_9			0x00000800
+#define HFI_MPEG4_LEVEL_3b			0x00001000
+
+#define HFI_VC1_PROFILE_SIMPLE			0x00000001
+#define HFI_VC1_PROFILE_MAIN			0x00000002
+#define HFI_VC1_PROFILE_ADVANCED		0x00000004
+
+#define HFI_VC1_LEVEL_LOW			0x00000001
+#define HFI_VC1_LEVEL_MEDIUM			0x00000002
+#define HFI_VC1_LEVEL_HIGH			0x00000004
+#define HFI_VC1_LEVEL_0				0x00000008
+#define HFI_VC1_LEVEL_1				0x00000010
+#define HFI_VC1_LEVEL_2				0x00000020
+#define HFI_VC1_LEVEL_3				0x00000040
+#define HFI_VC1_LEVEL_4				0x00000080
+
+#define HFI_VPX_PROFILE_SIMPLE			0x00000001
+#define HFI_VPX_PROFILE_ADVANCED		0x00000002
+#define HFI_VPX_PROFILE_VERSION_0		0x00000004
+#define HFI_VPX_PROFILE_VERSION_1		0x00000008
+#define HFI_VPX_PROFILE_VERSION_2		0x00000010
+#define HFI_VPX_PROFILE_VERSION_3		0x00000020
+
+#define HFI_DIVX_FORMAT_4			(HFI_COMMON_BASE + 0x1)
+#define HFI_DIVX_FORMAT_5			(HFI_COMMON_BASE + 0x2)
+#define HFI_DIVX_FORMAT_6			(HFI_COMMON_BASE + 0x3)
+
+#define HFI_DIVX_PROFILE_QMOBILE		0x00000001
+#define HFI_DIVX_PROFILE_MOBILE			0x00000002
+#define HFI_DIVX_PROFILE_MT			0x00000004
+#define HFI_DIVX_PROFILE_HT			0x00000008
+#define HFI_DIVX_PROFILE_HD			0x00000010
+
+#define HFI_HEVC_PROFILE_MAIN			0x00000001
+#define HFI_HEVC_PROFILE_MAIN10			0x00000002
+#define HFI_HEVC_PROFILE_MAIN_STILL_PIC		0x00000004
+
+#define HFI_HEVC_LEVEL_1			0x00000001
+#define HFI_HEVC_LEVEL_2			0x00000002
+#define HFI_HEVC_LEVEL_21			0x00000004
+#define HFI_HEVC_LEVEL_3			0x00000008
+#define HFI_HEVC_LEVEL_31			0x00000010
+#define HFI_HEVC_LEVEL_4			0x00000020
+#define HFI_HEVC_LEVEL_41			0x00000040
+#define HFI_HEVC_LEVEL_5			0x00000080
+#define HFI_HEVC_LEVEL_51			0x00000100
+#define HFI_HEVC_LEVEL_52			0x00000200
+#define HFI_HEVC_LEVEL_6			0x00000400
+#define HFI_HEVC_LEVEL_61			0x00000800
+#define HFI_HEVC_LEVEL_62			0x00001000
+
+#define HFI_HEVC_TIER_MAIN			0x1
+#define HFI_HEVC_TIER_HIGH0			0x2
+
+#define HFI_BUFFER_INPUT			(HFI_COMMON_BASE + 0x1)
+#define HFI_BUFFER_OUTPUT			(HFI_COMMON_BASE + 0x2)
+#define HFI_BUFFER_OUTPUT2			(HFI_COMMON_BASE + 0x3)
+#define HFI_BUFFER_INTERNAL_PERSIST		(HFI_COMMON_BASE + 0x4)
+#define HFI_BUFFER_INTERNAL_PERSIST_1		(HFI_COMMON_BASE + 0x5)
+#define HFI_BUFFER_INTERNAL_SCRATCH		(HFI_OX_BASE + 0x1)
+#define HFI_BUFFER_EXTRADATA_INPUT		(HFI_OX_BASE + 0x2)
+#define HFI_BUFFER_EXTRADATA_OUTPUT		(HFI_OX_BASE + 0x3)
+#define HFI_BUFFER_EXTRADATA_OUTPUT2		(HFI_OX_BASE + 0x4)
+#define HFI_BUFFER_INTERNAL_SCRATCH_1		(HFI_OX_BASE + 0x5)
+#define HFI_BUFFER_INTERNAL_SCRATCH_2		(HFI_OX_BASE + 0x6)
+
+#define HFI_BUFFER_TYPE_MAX			11
+
+#define HFI_BUFFER_MODE_STATIC			(HFI_OX_BASE + 0x1)
+#define HFI_BUFFER_MODE_RING			(HFI_OX_BASE + 0x2)
+#define HFI_BUFFER_MODE_DYNAMIC			(HFI_OX_BASE + 0x3)
+
+#define HFI_VENC_PERFMODE_MAX_QUALITY		0x1
+#define HFI_VENC_PERFMODE_POWER_SAVE		0x2
+
+#define HFI_PROPERTY_SYS_COMMON_START	\
+	(HFI_DOMAIN_BASE_COMMON + HFI_ARCH_COMMON_OFFSET + 0x0000)
+#define HFI_PROPERTY_SYS_DEBUG_CONFIG	\
+	(HFI_PROPERTY_SYS_COMMON_START + 0x1)
+#define HFI_PROPERTY_SYS_RESOURCE_OCMEM_REQUIREMENT_INFO	\
+	(HFI_PROPERTY_SYS_COMMON_START + 0x2)
+#define HFI_PROPERTY_SYS_CONFIG_VCODEC_CLKFREQ	\
+	(HFI_PROPERTY_SYS_COMMON_START + 0x3)
+#define HFI_PROPERTY_SYS_IDLE_INDICATOR	\
+	(HFI_PROPERTY_SYS_COMMON_START + 0x4)
+#define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL	\
+	(HFI_PROPERTY_SYS_COMMON_START + 0x5)
+#define HFI_PROPERTY_SYS_IMAGE_VERSION	\
+	(HFI_PROPERTY_SYS_COMMON_START + 0x6)
+#define HFI_PROPERTY_SYS_CONFIG_COVERAGE	\
+	(HFI_PROPERTY_SYS_COMMON_START + 0x7)
+
+#define HFI_PROPERTY_PARAM_COMMON_START	\
+	(HFI_DOMAIN_BASE_COMMON + HFI_ARCH_COMMON_OFFSET + 0x1000)
+#define HFI_PROPERTY_PARAM_FRAME_SIZE	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x1)
+#define HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_INFO	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x2)
+#define HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x3)
+#define HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x4)
+#define HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x5)
+#define HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x6)
+#define HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x7)
+#define HFI_PROPERTY_PARAM_PROPERTIES_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x8)
+#define HFI_PROPERTY_PARAM_CODEC_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0x9)
+#define HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0xa)
+#define HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SELECT	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0xb)
+#define HFI_PROPERTY_PARAM_MULTI_VIEW_FORMAT	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0xc)
+#define  HFI_PROPERTY_PARAM_MAX_SEQUENCE_HEADER_SIZE	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0xd)
+#define  HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0xe)
+#define HFI_PROPERTY_PARAM_MVC_BUFFER_LAYOUT	\
+	(HFI_PROPERTY_PARAM_COMMON_START + 0xf)
+
+#define HFI_PROPERTY_CONFIG_COMMON_START	\
+	(HFI_DOMAIN_BASE_COMMON + HFI_ARCH_COMMON_OFFSET + 0x2000)
+#define HFI_PROPERTY_CONFIG_FRAME_RATE	\
+	(HFI_PROPERTY_CONFIG_COMMON_START + 0x1)
+
+#define HFI_PROPERTY_PARAM_VDEC_COMMON_START	\
+	(HFI_DOMAIN_BASE_VDEC + HFI_ARCH_COMMON_OFFSET + 0x3000)
+#define HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM	\
+	(HFI_PROPERTY_PARAM_VDEC_COMMON_START + 0x1)
+#define HFI_PROPERTY_PARAM_VDEC_CONCEAL_COLOR	\
+	(HFI_PROPERTY_PARAM_VDEC_COMMON_START + 0x2)
+#define HFI_PROPERTY_PARAM_VDEC_NONCP_OUTPUT2	\
+	(HFI_PROPERTY_PARAM_VDEC_COMMON_START + 0x3)
+
+#define HFI_PROPERTY_CONFIG_VDEC_COMMON_START	\
+	(HFI_DOMAIN_BASE_VDEC + HFI_ARCH_COMMON_OFFSET + 0x4000)
+
+#define HFI_PROPERTY_PARAM_VENC_COMMON_START	\
+	(HFI_DOMAIN_BASE_VENC + HFI_ARCH_COMMON_OFFSET + 0x5000)
+#define HFI_PROPERTY_PARAM_VENC_SLICE_DELIVERY_MODE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x1)
+#define HFI_PROPERTY_PARAM_VENC_H264_ENTROPY_CONTROL	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x2)
+#define HFI_PROPERTY_PARAM_VENC_H264_DEBLOCK_CONTROL	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x3)
+#define HFI_PROPERTY_PARAM_VENC_RATE_CONTROL	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x4)
+#define HFI_PROPERTY_PARAM_VENC_H264_PICORDER_CNT_TYPE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x5)
+#define HFI_PROPERTY_PARAM_VENC_SESSION_QP	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x6)
+#define HFI_PROPERTY_PARAM_VENC_MPEG4_AC_PREDICTION	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x7)
+#define HFI_PROPERTY_PARAM_VENC_SESSION_QP_RANGE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x8)
+#define HFI_PROPERTY_PARAM_VENC_MPEG4_TIME_RESOLUTION	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x9)
+#define HFI_PROPERTY_PARAM_VENC_MPEG4_SHORT_HEADER	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0xa)
+#define HFI_PROPERTY_PARAM_VENC_MPEG4_HEADER_EXTENSION	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0xb)
+#define HFI_PROPERTY_PARAM_VENC_OPEN_GOP	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0xc)
+#define HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0xd)
+#define HFI_PROPERTY_PARAM_VENC_MULTI_SLICE_CONTROL	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0xe)
+#define HFI_PROPERTY_PARAM_VENC_VBV_HRD_BUF_SIZE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0xf)
+#define HFI_PROPERTY_PARAM_VENC_QUALITY_VS_SPEED	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x10)
+#define HFI_PROPERTY_PARAM_VENC_ADVANCED	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x12)
+#define HFI_PROPERTY_PARAM_VENC_H264_SPS_ID	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x14)
+#define HFI_PROPERTY_PARAM_VENC_H264_PPS_ID	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x15)
+#define HFI_PROPERTY_PARAM_VENC_H264_GENERATE_AUDNAL	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x16)
+#define HFI_PROPERTY_PARAM_VENC_ASPECT_RATIO	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x17)
+#define HFI_PROPERTY_PARAM_VENC_NUMREF	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x18)
+#define HFI_PROPERTY_PARAM_VENC_MULTIREF_P	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x19)
+#define HFI_PROPERTY_PARAM_VENC_H264_NAL_SVC_EXT	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x1b)
+#define HFI_PROPERTY_PARAM_VENC_LTRMODE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x1c)
+#define HFI_PROPERTY_PARAM_VENC_VIDEO_FULL_RANGE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x1d)
+#define HFI_PROPERTY_PARAM_VENC_H264_VUI_TIMING_INFO	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x1e)
+#define HFI_PROPERTY_PARAM_VENC_VC1_PERF_CFG	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x1f)
+#define HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x20)
+#define HFI_PROPERTY_PARAM_VENC_H264_VUI_BITSTREAM_RESTRC	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x21)
+#define HFI_PROPERTY_PARAM_VENC_PRESERVE_TEXT_QUALITY	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x23)
+#define HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x26)
+#define HFI_PROPERTY_PARAM_VENC_DISABLE_RC_TIMESTAMP	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x27)
+#define HFI_PROPERTY_PARAM_VENC_INITIAL_QP	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x28)
+#define HFI_PROPERTY_PARAM_VENC_VPX_ERROR_RESILIENCE_MODE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x29)
+#define HFI_PROPERTY_PARAM_VENC_HIER_B_MAX_NUM_ENH_LAYER	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x2c)
+#define HFI_PROPERTY_PARAM_VENC_HIER_P_HYBRID_MODE	\
+	(HFI_PROPERTY_PARAM_VENC_COMMON_START + 0x2f)
+
+#define HFI_PROPERTY_CONFIG_VENC_COMMON_START	\
+	(HFI_DOMAIN_BASE_VENC + HFI_ARCH_COMMON_OFFSET + 0x6000)
+#define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x1)
+#define HFI_PROPERTY_CONFIG_VENC_IDR_PERIOD	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x2)
+#define HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x3)
+#define HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x4)
+#define HFI_PROPERTY_CONFIG_VENC_SLICE_SIZE	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x5)
+#define HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x7)
+
+#define HFI_PROPERTY_PARAM_VPE_COMMON_START	\
+	(HFI_DOMAIN_BASE_VPE + HFI_ARCH_COMMON_OFFSET + 0x7000)
+#define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x8)
+#define HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0x9)
+#define HFI_PROPERTY_CONFIG_VENC_USELTRFRAME	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0xa)
+#define HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0xb)
+#define HFI_PROPERTY_CONFIG_VENC_LTRPERIOD	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0xc)
+#define HFI_PROPERTY_CONFIG_VENC_PERF_MODE	\
+	(HFI_PROPERTY_CONFIG_VENC_COMMON_START + 0xe)
+
+#define HFI_PROPERTY_CONFIG_VPE_COMMON_START	\
+	(HFI_DOMAIN_BASE_VPE + HFI_ARCH_COMMON_OFFSET + 0x8000)
+#define HFI_PROPERTY_CONFIG_VPE_DEINTERLACE	\
+	(HFI_PROPERTY_CONFIG_VPE_COMMON_START + 0x1)
+#define HFI_PROPERTY_CONFIG_VPE_OPERATIONS	\
+	(HFI_PROPERTY_CONFIG_VPE_COMMON_START + 0x2)
+
+enum hfi_packetization_type {
+	HFI_PACKETIZATION_LEGACY,
+	HFI_PACKETIZATION_3XX,
+};
+
+struct hfi_buffer_info {
+	u32 buffer_addr;
+	u32 extradata_addr;
+};
+
+struct hfi_bitrate {
+	u32 bitrate;
+	u32 layer_id;
+};
+
+#define HFI_CAPABILITY_FRAME_WIDTH			(HFI_COMMON_BASE + 0x1)
+#define HFI_CAPABILITY_FRAME_HEIGHT			(HFI_COMMON_BASE + 0x2)
+#define HFI_CAPABILITY_MBS_PER_FRAME			(HFI_COMMON_BASE + 0x3)
+#define HFI_CAPABILITY_MBS_PER_SECOND			(HFI_COMMON_BASE + 0x4)
+#define HFI_CAPABILITY_FRAMERATE			(HFI_COMMON_BASE + 0x5)
+#define HFI_CAPABILITY_SCALE_X				(HFI_COMMON_BASE + 0x6)
+#define HFI_CAPABILITY_SCALE_Y				(HFI_COMMON_BASE + 0x7)
+#define HFI_CAPABILITY_BITRATE				(HFI_COMMON_BASE + 0x8)
+#define HFI_CAPABILITY_BFRAME				(HFI_COMMON_BASE + 0x9)
+#define HFI_CAPABILITY_HIER_P_NUM_ENH_LAYERS		(HFI_COMMON_BASE + 0x10)
+#define HFI_CAPABILITY_ENC_LTR_COUNT			(HFI_COMMON_BASE + 0x11)
+#define HFI_CAPABILITY_CP_OUTPUT2_THRESH		(HFI_COMMON_BASE + 0x12)
+#define HFI_CAPABILITY_HIER_P_HYBRID_NUM_ENH_LAYERS	(HFI_COMMON_BASE + 0x15)
+
+struct hfi_capability {
+	u32 capability_type;
+	u32 min;
+	u32 max;
+	u32 step_size;
+};
+
+struct hfi_capabilities {
+	u32 num_capabilities;
+	struct hfi_capability data[1];
+};
+
+#define HFI_DEBUG_MSG_LOW	0x00000001
+#define HFI_DEBUG_MSG_MEDIUM	0x00000002
+#define HFI_DEBUG_MSG_HIGH	0x00000004
+#define HFI_DEBUG_MSG_ERROR	0x00000008
+#define HFI_DEBUG_MSG_FATAL	0x00000010
+#define HFI_DEBUG_MSG_PERF	0x00000020
+
+#define HFI_DEBUG_MODE_QUEUE	0x00000001
+#define HFI_DEBUG_MODE_QDSS	0x00000002
+
+struct hfi_debug_config {
+	u32 config;
+	u32 mode;
+};
+
+struct hfi_enable {
+	u32 enable;
+};
+
+#define HFI_H264_DB_MODE_DISABLE		(HFI_COMMON_BASE + 0x1)
+#define HFI_H264_DB_MODE_SKIP_SLICE_BOUNDARY	(HFI_COMMON_BASE + 0x2)
+#define HFI_H264_DB_MODE_ALL_BOUNDARY		(HFI_COMMON_BASE + 0x3)
+
+struct hfi_h264_db_control {
+	u32 mode;
+	u32 slice_alpha_offset;
+	u32 slice_beta_offset;
+};
+
+#define HFI_H264_ENTROPY_CAVLC			(HFI_COMMON_BASE + 0x1)
+#define HFI_H264_ENTROPY_CABAC			(HFI_COMMON_BASE + 0x2)
+
+#define HFI_H264_CABAC_MODEL_0			(HFI_COMMON_BASE + 0x1)
+#define HFI_H264_CABAC_MODEL_1			(HFI_COMMON_BASE + 0x2)
+#define HFI_H264_CABAC_MODEL_2			(HFI_COMMON_BASE + 0x3)
+
+struct hfi_h264_entropy_control {
+	u32 entropy_mode;
+	u32 cabac_model;
+};
+
+struct hfi_framerate {
+	u32 buffer_type;
+	u32 framerate;
+};
+
+#define HFI_INTRA_REFRESH_NONE			(HFI_COMMON_BASE + 0x1)
+#define HFI_INTRA_REFRESH_CYCLIC		(HFI_COMMON_BASE + 0x2)
+#define HFI_INTRA_REFRESH_ADAPTIVE		(HFI_COMMON_BASE + 0x3)
+#define HFI_INTRA_REFRESH_CYCLIC_ADAPTIVE	(HFI_COMMON_BASE + 0x4)
+#define HFI_INTRA_REFRESH_RANDOM		(HFI_COMMON_BASE + 0x5)
+
+struct hfi_intra_refresh {
+	u32 mode;
+	u32 air_mbs;
+	u32 air_ref;
+	u32 cir_mbs;
+};
+
+struct hfi_intra_refresh_3x {
+	u32 mode;
+	u32 mbs;
+};
+
+struct hfi_idr_period {
+	u32 idr_period;
+};
+
+struct hfi_operations_type {
+	u32 rotation;
+	u32 flip;
+};
+
+struct hfi_max_num_b_frames {
+	u32 max_num_b_frames;
+};
+
+struct hfi_vc1e_perf_cfg_type {
+	u32 search_range_x_subsampled[3];
+	u32 search_range_y_subsampled[3];
+};
+
+struct hfi_conceal_color {
+	u32 conceal_color;
+};
+
+struct hfi_intra_period {
+	u32 pframes;
+	u32 bframes;
+};
+
+struct hfi_mpeg4_header_extension {
+	u32 header_extension;
+};
+
+struct hfi_mpeg4_time_resolution {
+	u32 time_increment_resolution;
+};
+
+struct hfi_multi_stream {
+	u32 buffer_type;
+	u32 enable;
+	u32 width;
+	u32 height;
+};
+
+struct hfi_multi_stream_3x {
+	u32 buffer_type;
+	u32 enable;
+};
+
+struct hfi_multi_view_format {
+	u32 views;
+	u32 view_order[1];
+};
+
+#define HFI_MULTI_SLICE_OFF			(HFI_COMMON_BASE + 0x1)
+#define HFI_MULTI_SLICE_BY_MB_COUNT		(HFI_COMMON_BASE + 0x2)
+#define HFI_MULTI_SLICE_BY_BYTE_COUNT		(HFI_COMMON_BASE + 0x3)
+#define HFI_MULTI_SLICE_GOB			(HFI_COMMON_BASE + 0x4)
+
+struct hfi_multi_slice_control {
+	u32 multi_slice;
+	u32 slice_size;
+};
+
+#define HFI_NAL_FORMAT_STARTCODES		0x1
+#define HFI_NAL_FORMAT_ONE_NAL_PER_BUFFER	0x2
+#define HFI_NAL_FORMAT_ONE_BYTE_LENGTH		0x4
+#define HFI_NAL_FORMAT_TWO_BYTE_LENGTH		0x8
+#define HFI_NAL_FORMAT_FOUR_BYTE_LENGTH		0x10
+
+struct hfi_nal_stream_format {
+	u32 format;
+};
+
+struct hfi_nal_stream_format_select {
+	u32 format;
+};
+
+#define HFI_PICTURE_TYPE_I			0x01
+#define HFI_PICTURE_TYPE_P			0x02
+#define HFI_PICTURE_TYPE_B			0x04
+#define HFI_PICTURE_TYPE_IDR			0x08
+
+struct hfi_profile_level {
+	u32 profile;
+	u32 level;
+};
+
+#define HFI_MAX_PROFILE_COUNT			16
+
+struct hfi_profile_level_supported {
+	u32 profile_count;
+	struct hfi_profile_level profile_level[1];
+};
+
+struct hfi_quality_vs_speed {
+	u32 quality_vs_speed;
+};
+
+struct hfi_quantization {
+	u32 qp_i;
+	u32 qp_p;
+	u32 qp_b;
+	u32 layer_id;
+};
+
+struct hfi_initial_quantization {
+	u32 qp_i;
+	u32 qp_p;
+	u32 qp_b;
+	u32 init_qp_enable;
+};
+
+struct hfi_quantization_range {
+	u32 min_qp;
+	u32 max_qp;
+	u32 layer_id;
+};
+
+#define HFI_LTR_MODE_DISABLE	0x0
+#define HFI_LTR_MODE_MANUAL	0x1
+#define HFI_LTR_MODE_PERIODIC	0x2
+
+struct hfi_ltr_mode {
+	u32 ltr_mode;
+	u32 ltr_count;
+	u32 trust_mode;
+};
+
+struct hfi_ltr_use {
+	u32 ref_ltr;
+	u32 use_constrnt;
+	u32 frames;
+};
+
+struct hfi_ltr_mark {
+	u32 mark_frame;
+};
+
+struct hfi_framesize {
+	u32 buffer_type;
+	u32 width;
+	u32 height;
+};
+
+struct hfi_h264_vui_timing_info {
+	u32 enable;
+	u32 fixed_framerate;
+	u32 time_scale;
+};
+
+#define HFI_COLOR_FORMAT_MONOCHROME		(HFI_COMMON_BASE + 0x1)
+#define HFI_COLOR_FORMAT_NV12			(HFI_COMMON_BASE + 0x2)
+#define HFI_COLOR_FORMAT_NV21			(HFI_COMMON_BASE + 0x3)
+#define HFI_COLOR_FORMAT_NV12_4x4TILE		(HFI_COMMON_BASE + 0x4)
+#define HFI_COLOR_FORMAT_NV21_4x4TILE		(HFI_COMMON_BASE + 0x5)
+#define HFI_COLOR_FORMAT_YUYV			(HFI_COMMON_BASE + 0x6)
+#define HFI_COLOR_FORMAT_YVYU			(HFI_COMMON_BASE + 0x7)
+#define HFI_COLOR_FORMAT_UYVY			(HFI_COMMON_BASE + 0x8)
+#define HFI_COLOR_FORMAT_VYUY			(HFI_COMMON_BASE + 0x9)
+#define HFI_COLOR_FORMAT_RGB565			(HFI_COMMON_BASE + 0xa)
+#define HFI_COLOR_FORMAT_BGR565			(HFI_COMMON_BASE + 0xb)
+#define HFI_COLOR_FORMAT_RGB888			(HFI_COMMON_BASE + 0xc)
+#define HFI_COLOR_FORMAT_BGR888			(HFI_COMMON_BASE + 0xd)
+#define HFI_COLOR_FORMAT_YUV444			(HFI_COMMON_BASE + 0xe)
+#define HFI_COLOR_FORMAT_RGBA8888		(HFI_COMMON_BASE + 0x10)
+
+#define HFI_COLOR_FORMAT_UBWC_BASE		0x8000
+#define HFI_COLOR_FORMAT_10_BIT_BASE		0x4000
+
+#define HFI_COLOR_FORMAT_YUV420_TP10	\
+	(HFI_COLOR_FORMAT_10_BIT_BASE + HFI_COLOR_FORMAT_NV12)
+#define HFI_COLOR_FORMAT_NV12_UBWC	\
+	(HFI_COLOR_FORMAT_UBWC_BASE + HFI_COLOR_FORMAT_NV12)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC	\
+	(HFI_COLOR_FORMAT_UBWC_BASE + HFI_COLOR_FORMAT_YUV420_TP10)
+#define HFI_COLOR_FORMAT_RGBA8888_UBWC	\
+	(HFI_COLOR_FORMAT_UBWC_BASE + HFI_COLOR_FORMAT_RGBA8888)
+
+struct hfi_uncompressed_format_select {
+	u32 buffer_type;
+	u32 format;
+};
+
+struct hfi_uncompressed_format_supported {
+	u32 buffer_type;
+	u32 format_entries;
+	u32 format_info[1];
+};
+
+struct hfi_uncompressed_plane_actual {
+	int actual_stride;
+	u32 actual_plane_buffer_height;
+};
+
+struct hfi_uncompressed_plane_actual_info {
+	u32 buffer_type;
+	u32 num_planes;
+	struct hfi_uncompressed_plane_actual plane_format[1];
+};
+
+struct hfi_uncompressed_plane_constraints {
+	u32 stride_multiples;
+	u32 max_stride;
+	u32 min_plane_buffer_height_multiple;
+	u32 buffer_alignment;
+};
+
+struct hfi_uncompressed_plane_info {
+	u32 format;
+	u32 num_planes;
+	struct hfi_uncompressed_plane_constraints plane_format[1];
+};
+
+struct hfi_uncompressed_plane_actual_constraints_info {
+	u32 buffer_type;
+	u32 num_planes;
+	struct hfi_uncompressed_plane_constraints plane_format[1];
+};
+
+struct hfi_codec_supported {
+	u32 dec_codecs;
+	u32 enc_codecs;
+};
+
+struct hfi_properties_supported {
+	u32 num_properties;
+	u32 properties[1];
+};
+
+#define HFI_MAX_MATRIX_COEFFS	9
+#define HFI_MAX_BIAS_COEFFS	3
+#define HFI_MAX_LIMIT_COEFFS	6
+
+struct hfi_vpe_color_space_conversion {
+	u32 csc_matrix[HFI_MAX_MATRIX_COEFFS];
+	u32 csc_bias[HFI_MAX_BIAS_COEFFS];
+	u32 csc_limit[HFI_MAX_LIMIT_COEFFS];
+};
+
+#define HFI_ROTATE_NONE		(HFI_COMMON_BASE + 0x1)
+#define HFI_ROTATE_90		(HFI_COMMON_BASE + 0x2)
+#define HFI_ROTATE_180		(HFI_COMMON_BASE + 0x3)
+#define HFI_ROTATE_270		(HFI_COMMON_BASE + 0x4)
+
+#define HFI_FLIP_NONE		(HFI_COMMON_BASE + 0x1)
+#define HFI_FLIP_HORIZONTAL	(HFI_COMMON_BASE + 0x2)
+#define HFI_FLIP_VERTICAL	(HFI_COMMON_BASE + 0x3)
+
+struct hfi_operations {
+	u32 rotate;
+	u32 flip;
+};
+
+#define HFI_RESOURCE_OCMEM	0x1
+
+struct hfi_resource_ocmem {
+	u32 size;
+	u32 mem;
+};
+
+struct hfi_resource_ocmem_requirement {
+	u32 session_domain;
+	u32 width;
+	u32 height;
+	u32 size;
+};
+
+struct hfi_resource_ocmem_requirement_info {
+	u32 num_entries;
+	struct hfi_resource_ocmem_requirement requirements[1];
+};
+
+struct hfi_property_sys_image_version_info_type {
+	u32 string_size;
+	u8  str_image_version[1];
+};
+
+struct hfi_seq_header_info {
+	u32 max_hader_len;
+};
+struct hfi_aspect_ratio {
+	u32 aspect_width;
+	u32 aspect_height;
+};
+#define HFI_MVC_BUFFER_LAYOUT_TOP_BOTTOM	0
+#define HFI_MVC_BUFFER_LAYOUT_SIDEBYSIDE	1
+#define HFI_MVC_BUFFER_LAYOUT_SEQ		2
+
+struct hfi_mvc_buffer_layout_descp_type {
+	u32 layout_type;
+	u32 bright_view_first;
+	u32 ngap;
+};
+
+struct hfi_scs_threshold {
+	u32 threshold_value;
+};
+
+#define HFI_TEST_SSR_SW_ERR_FATAL	0x1
+#define HFI_TEST_SSR_SW_DIV_BY_ZERO	0x2
+#define HFI_TEST_SSR_HW_WDOG_IRQ	0x3
+
+struct hfi_buffer_alloc_mode {
+	u32 type;
+	u32 mode;
+};
+
+struct hfi_index_extradata_config {
+	u32 enable;
+	u32 index_extra_data_id;
+};
+
+struct hfi_extradata_header {
+	u32 size;
+	u32 version;
+	u32 port_index;
+	u32 type;
+	u32 data_size;
+	u8 data[1];
+};
+
+struct hfi_batch_info {
+	u32 input_batch_count;
+	u32 output_batch_count;
+};
+
+struct hfi_buffer_count_actual {
+	u32 type;
+	u32 count_actual;
+};
+
+struct hfi_buffer_size_actual {
+	u32 type;
+	u32 size;
+};
+
+struct hfi_buffer_display_hold_count_actual {
+	u32 type;
+	u32 hold_count;
+};
+
+struct hfi_buffer_requirements {
+	u32 type;
+	u32 size;
+	u32 region_size;
+	u32 hold_count;
+	u32 count_min;
+	u32 count_actual;
+	u32 contiguous;
+	u32 alignment;
+};
+
+struct hfi_data_payload {
+	u32 size;
+	u8 data[1];
+};
+
+struct hfi_enable_picture {
+	u32 picture_type;
+};
+
+struct hfi_display_picture_buffer_count {
+	int enable;
+	u32 count;
+};
+
+struct hfi_extra_data_header_config {
+	u32 type;
+	u32 buffer_type;
+	u32 version;
+	u32 port_index;
+	u32 client_extra_data_id;
+};
+
+struct hfi_interlace_format_supported {
+	u32 buffer_type;
+	u32 format;
+};
+
+struct hfi_buffer_alloc_mode_supported {
+	u32 buffer_type;
+	u32 num_entries;
+	u32 data[1];
+};
+
+struct hfi_mb_error_map {
+	u32 error_map_size;
+	u8 error_map[1];
+};
+
+struct hfi_metadata_pass_through {
+	int enable;
+	u32 size;
+};
+
+struct hfi_multi_view_select {
+	u32 view_index;
+};
+
+struct hfi_hybrid_hierp {
+	u32 layers;
+};
+
+struct hfi_pkt_hdr {
+	u32 size;
+	u32 pkt_type;
+};
+
+struct hfi_session_hdr_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 session_id;
+};
+
+struct hfi_session_pkt {
+	struct hfi_session_hdr_pkt shdr;
+};
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/hfi_msgs.c b/drivers/media/platform/qcom/vidc/hfi_msgs.c
new file mode 100644
index 000000000000..0f390af30d6d
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_msgs.c
@@ -0,0 +1,1072 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/hash.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "hfi.h"
+#include "hfi_helper.h"
+#include "hfi_msgs.h"
+
+struct hfi_msg_fbd {
+	u32 stream_id;
+	u32 view_id;
+	u32 timestamp_hi;
+	u32 timestamp_lo;
+	u32 flags1;
+	u32 mark_target;
+	u32 mark_data;
+	u32 stats;
+	u32 alloc_len1;
+	u32 filled_len1;
+	u32 offset1;
+	u32 frame_width;
+	u32 frame_height;
+	u32 start_x_coord;
+	u32 start_y_coord;
+	u32 input_tag;
+	u32 input_tag1;
+	u32 picture_type;
+	u32 packet_buffer1;
+	u32 extradata_buffer;
+	u32 flags2;
+	u32 alloc_len2;
+	u32 filled_len2;
+	u32 offset2;
+	u32 packet_buffer2;
+	u32 flags3;
+	u32 alloc_len3;
+	u32 filled_len3;
+	u32 offset3;
+	u32 packet_buffer3;
+	u32 buffer_type;
+};
+
+static struct hfi_inst *to_hfi_instance(struct hfi_core *hfi, u32 session_id)
+{
+	struct hfi_inst *inst;
+
+	mutex_lock(&hfi->lock);
+	list_for_each_entry(inst, &hfi->instances, list)
+		if (hash32_ptr(inst) == session_id) {
+			mutex_unlock(&hfi->lock);
+			return inst;
+		}
+	mutex_unlock(&hfi->lock);
+
+	return NULL;
+}
+
+static void event_seq_changed(struct hfi_core *hfi, struct hfi_inst *inst,
+			      struct hfi_msg_event_notify_pkt *pkt)
+{
+	struct device *dev = hfi->dev;
+	struct hfi_event_data event = {0};
+	int num_properties_changed;
+	struct hfi_framesize *frame_sz;
+	struct hfi_profile_level *profile_level;
+	u8 *data_ptr;
+	u32 ptype;
+
+	inst->error = HFI_ERR_NONE;
+
+	switch (pkt->event_data1) {
+	case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES:
+	case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES:
+		break;
+	default:
+		inst->error = HFI_ERR_SESSION_INVALID_PARAMETER;
+		goto done;
+	}
+
+	event.event_type = pkt->event_data1;
+
+	num_properties_changed = pkt->event_data2;
+	if (!num_properties_changed) {
+		inst->error = HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
+		goto done;
+	}
+
+	data_ptr = (u8 *) &pkt->ext_event_data[0];
+	do {
+		ptype = *((u32 *)data_ptr);
+		switch (ptype) {
+		case HFI_PROPERTY_PARAM_FRAME_SIZE:
+			data_ptr += sizeof(u32);
+			frame_sz = (struct hfi_framesize *) data_ptr;
+			event.width = frame_sz->width;
+			event.height = frame_sz->height;
+			data_ptr += sizeof(frame_sz);
+			dev_dbg(dev, "%s cmd: frame size: %ux%u\n",
+				__func__, event.width, event.height);
+			break;
+		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT:
+			data_ptr += sizeof(u32);
+			profile_level = (struct hfi_profile_level *) data_ptr;
+			event.profile = profile_level->profile;
+			event.level = profile_level->level;
+			data_ptr += sizeof(profile_level);
+			dev_dbg(dev, "%s cmd: profile-level: %u - %u\n",
+				__func__, event.profile, event.level);
+			break;
+		default:
+			dev_dbg(dev, "%s cmd: %#x not supported\n",
+				__func__, ptype);
+			break;
+		}
+		num_properties_changed--;
+	} while (num_properties_changed > 0);
+
+done:
+	inst->ops->event_notify(inst, EVT_SYS_EVENT_CHANGE, &event);
+}
+
+static void event_release_buffer_ref(struct hfi_core *hfi,
+				     struct hfi_inst *inst,
+				     struct hfi_msg_event_notify_pkt *pkt)
+{
+	struct hfi_event_data event = {0};
+	struct hfi_msg_event_release_buffer_ref_pkt *data;
+
+	data = (struct hfi_msg_event_release_buffer_ref_pkt *)
+		pkt->ext_event_data;
+
+	event.event_type = HFI_EVENT_RELEASE_BUFFER_REFERENCE;
+	event.packet_buffer = data->packet_buffer;
+	event.extradata_buffer = data->extradata_buffer;
+
+	inst->error = HFI_ERR_NONE;
+	inst->ops->event_notify(inst, EVT_SYS_EVENT_CHANGE, &event);
+}
+
+static void event_sys_error(struct hfi_core *hfi, u32 event)
+{
+	hfi->core_ops->event_notify(hfi, event);
+}
+
+static void event_session_error(struct hfi_core *hfi, struct hfi_inst *inst,
+				struct hfi_msg_event_notify_pkt *pkt)
+{
+	struct device *dev = hfi->dev;
+
+	dev_dbg(dev, "session error: event id:%x, session id:%x\n",
+		pkt->event_data1, pkt->shdr.session_id);
+
+	if (!inst)
+		return;
+
+	switch (pkt->event_data1) {
+	/* non fatal session errors */
+	case HFI_ERR_SESSION_INVALID_SCALE_FACTOR:
+	case HFI_ERR_SESSION_UNSUPPORT_BUFFERTYPE:
+	case HFI_ERR_SESSION_UNSUPPORTED_SETTING:
+	case HFI_ERR_SESSION_UPSCALE_NOT_SUPPORTED:
+		inst->error = HFI_ERR_NONE;
+		break;
+	default:
+		dev_err(dev, "session error: event id:%x, session id:%x\n",
+			pkt->event_data1, pkt->shdr.session_id);
+
+		inst->error = pkt->event_data1;
+		inst->ops->event_notify(inst, EVT_SESSION_ERROR, NULL);
+		break;
+	}
+}
+
+static void hfi_event_notify(struct hfi_core *hfi, struct hfi_inst *inst,
+			     void *packet)
+{
+	struct hfi_msg_event_notify_pkt *pkt = packet;
+
+	if (!packet) {
+		dev_err(hfi->dev, "invalid packet\n");
+		return;
+	}
+
+	switch (pkt->event_id) {
+	case HFI_EVENT_SYS_ERROR:
+		event_sys_error(hfi, EVT_SYS_ERROR);
+		break;
+	case HFI_EVENT_SESSION_ERROR:
+		event_session_error(hfi, inst, pkt);
+		break;
+	case HFI_EVENT_SESSION_SEQUENCE_CHANGED:
+		event_seq_changed(hfi, inst, pkt);
+		break;
+	case HFI_EVENT_RELEASE_BUFFER_REFERENCE:
+		event_release_buffer_ref(hfi, inst, pkt);
+		break;
+	case HFI_EVENT_SESSION_PROPERTY_CHANGED:
+		break;
+	default:
+		break;
+	}
+}
+
+static void hfi_sys_init_done(struct hfi_core *hfi, struct hfi_inst *inst,
+			      void *packet)
+{
+	struct hfi_msg_sys_init_done_pkt *pkt = packet;
+	u32 enc_codecs = 0, dec_codecs = 0;
+	u32 rem_bytes, read_bytes = 0, num_properties;
+	u8 *data_ptr;
+	u32 error;
+	u32 ptype;
+
+	error = pkt->error_type;
+	if (error != HFI_ERR_NONE)
+		goto err_no_prop;
+
+	num_properties = pkt->num_properties;
+
+	if (!num_properties) {
+		error = HFI_ERR_SYS_INVALID_PARAMETER;
+		goto err_no_prop;
+	}
+
+	rem_bytes = pkt->hdr.size - sizeof(*pkt) + sizeof(u32);
+
+	if (!rem_bytes) {
+		/* missing property data */
+		error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
+		goto err_no_prop;
+	}
+
+	data_ptr = (u8 *)&pkt->data[0];
+
+	while (num_properties && rem_bytes >= sizeof(u32)) {
+		ptype = *((u32 *)data_ptr);
+		data_ptr += sizeof(u32);
+
+		switch (ptype) {
+		case HFI_PROPERTY_PARAM_CODEC_SUPPORTED: {
+			struct hfi_codec_supported *prop;
+
+			prop = (struct hfi_codec_supported *)data_ptr;
+
+			if (rem_bytes < sizeof(*prop)) {
+				error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
+				break;
+			}
+			dec_codecs = prop->dec_codecs;
+			enc_codecs = prop->enc_codecs;
+			break;
+		}
+		default:
+			error = HFI_ERR_SYS_INVALID_PARAMETER;
+			break;
+		}
+
+		if (!error) {
+			rem_bytes -= read_bytes;
+			data_ptr += read_bytes;
+			num_properties--;
+		}
+	}
+
+	hfi->enc_codecs = enc_codecs;
+	hfi->dec_codecs = dec_codecs;
+
+err_no_prop:
+	hfi->error = error;
+	complete(&hfi->done);
+}
+
+static void
+sys_get_prop_image_version(struct device *dev,
+			   struct hfi_msg_sys_property_info_pkt *pkt)
+{
+	int req_bytes;
+
+	req_bytes = pkt->hdr.size - sizeof(*pkt);
+
+	if (req_bytes < 128 || !pkt->data[1] || pkt->num_properties > 1)
+		/* bad packet */
+		return;
+
+	dev_dbg(dev, "F/W version: %s\n", (u8 *)&pkt->data[1]);
+}
+
+static void hfi_sys_property_info(struct hfi_core *hfi, struct hfi_inst *inst,
+				  void *packet)
+{
+	struct hfi_msg_sys_property_info_pkt *pkt = packet;
+	struct device *dev = hfi->dev;
+
+	if (!pkt->num_properties) {
+		dev_dbg(dev, "%s: no properties\n", __func__);
+		return;
+	}
+
+	switch (pkt->data[0]) {
+	case HFI_PROPERTY_SYS_IMAGE_VERSION:
+		sys_get_prop_image_version(dev, pkt);
+		break;
+	default:
+		dev_dbg(dev, "%s: unknown property data\n", __func__);
+		break;
+	}
+}
+
+static void hfi_sys_rel_resource_done(struct hfi_core *hfi,
+				      struct hfi_inst *inst,
+				      void *packet)
+{
+	struct hfi_msg_sys_release_resource_done_pkt *pkt = packet;
+
+	hfi->error = pkt->error_type;
+	complete(&hfi->done);
+}
+
+static void hfi_sys_ping_done(struct hfi_core *hfi, struct hfi_inst *inst,
+			      void *packet)
+{
+	struct hfi_msg_sys_ping_ack_pkt *pkt = packet;
+
+	hfi->error = HFI_ERR_NONE;
+
+	if (pkt->client_data != 0xbeef)
+		hfi->error = HFI_ERR_SYS_FATAL;
+
+	complete(&hfi->done);
+}
+
+static void hfi_sys_idle_done(struct hfi_core *hfi, struct hfi_inst *inst,
+			      void *packet)
+{
+	dev_dbg(hfi->dev, "sys idle\n");
+}
+
+static void hfi_sys_pc_prepare_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				    void *packet)
+{
+	struct hfi_msg_sys_pc_prep_done_pkt *pkt = packet;
+
+	dev_dbg(hfi->dev, "pc prepare done (error %x)\n", pkt->error_type);
+}
+
+static void hfi_copy_cap_prop(struct hfi_capability *in, struct hfi_inst *inst)
+{
+	if (!in || !inst)
+		return;
+
+	switch (in->capability_type) {
+	case HFI_CAPABILITY_FRAME_WIDTH:
+		inst->width = *in;
+		break;
+	case HFI_CAPABILITY_FRAME_HEIGHT:
+		inst->height = *in;
+		break;
+	case HFI_CAPABILITY_MBS_PER_FRAME:
+		inst->mbs_per_frame = *in;
+		break;
+	case HFI_CAPABILITY_MBS_PER_SECOND:
+		inst->mbs_per_sec = *in;
+		break;
+	case HFI_CAPABILITY_FRAMERATE:
+		inst->framerate = *in;
+		break;
+	case HFI_CAPABILITY_SCALE_X:
+		inst->scale_x = *in;
+		break;
+	case HFI_CAPABILITY_SCALE_Y:
+		inst->scale_y = *in;
+		break;
+	case HFI_CAPABILITY_BITRATE:
+		inst->bitrate = *in;
+		break;
+	case HFI_CAPABILITY_HIER_P_NUM_ENH_LAYERS:
+		inst->hier_p = *in;
+		break;
+	case HFI_CAPABILITY_ENC_LTR_COUNT:
+		inst->ltr_count = *in;
+		break;
+	case HFI_CAPABILITY_CP_OUTPUT2_THRESH:
+		inst->secure_output2_threshold = *in;
+		break;
+	default:
+		break;
+	}
+}
+
+static unsigned int
+session_get_prop_profile_level(struct hfi_msg_session_property_info_pkt *pkt,
+			       struct hfi_profile_level *profile_level)
+{
+	struct hfi_profile_level *hfi;
+	u32 req_bytes;
+
+	req_bytes = pkt->shdr.hdr.size - sizeof(*pkt);
+
+	if (!req_bytes || req_bytes % sizeof(struct hfi_profile_level))
+		/* bad packet */
+		return HFI_ERR_SESSION_INVALID_PARAMETER;
+
+	hfi = (struct hfi_profile_level *)&pkt->data[1];
+	profile_level->profile = hfi->profile;
+	profile_level->level = hfi->level;
+
+	return HFI_ERR_NONE;
+}
+
+static unsigned int
+session_get_prop_buf_req(struct hfi_msg_session_property_info_pkt *pkt,
+			 struct hfi_buffer_requirements *bufreq)
+{
+	struct hfi_buffer_requirements *buf_req;
+	u32 req_bytes;
+	unsigned int idx = 0;
+
+	req_bytes = pkt->shdr.hdr.size - sizeof(*pkt);
+
+	if (!req_bytes || req_bytes % sizeof(*buf_req) || !pkt->data[1])
+		/* bad packet */
+		return HFI_ERR_SESSION_INVALID_PARAMETER;
+
+	buf_req = (struct hfi_buffer_requirements *)&pkt->data[1];
+	if (!buf_req)
+		return HFI_ERR_SESSION_INVALID_PARAMETER;
+
+	while (req_bytes) {
+		memcpy(&bufreq[idx], buf_req, sizeof(*bufreq));
+		idx++;
+
+		if (idx > HFI_BUFFER_TYPE_MAX)
+			return HFI_ERR_SESSION_INVALID_PARAMETER;
+
+		req_bytes -= sizeof(struct hfi_buffer_requirements);
+		buf_req++;
+	}
+
+	return HFI_ERR_NONE;
+}
+
+static void hfi_session_prop_info(struct hfi_core *hfi, struct hfi_inst *inst,
+				  void *packet)
+{
+	struct hfi_msg_session_property_info_pkt *pkt = packet;
+	struct device *dev = hfi->dev;
+	union hfi_get_property *hprop = &inst->hprop;
+	unsigned int error = HFI_ERR_NONE;
+
+	if (!pkt->num_properties) {
+		error = HFI_ERR_SESSION_INVALID_PARAMETER;
+		dev_err(dev, "%s: no properties\n", __func__);
+		goto done;
+	}
+
+	switch (pkt->data[0]) {
+	case HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS:
+		memset(hprop->bufreq, 0, sizeof(hprop->bufreq));
+		error = session_get_prop_buf_req(pkt, hprop->bufreq);
+		break;
+	case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT:
+		memset(&hprop->profile_level, 0, sizeof(hprop->profile_level));
+		error = session_get_prop_profile_level(pkt,
+						       &hprop->profile_level);
+		break;
+	default:
+		dev_dbg(dev, "%s: unknown property id:%x\n", __func__,
+			pkt->data[0]);
+		return;
+	}
+
+done:
+	inst->error = error;
+	complete(&inst->done);
+}
+
+static u32
+session_init_done_read_prop(struct hfi_core *hfi, struct hfi_inst *inst,
+			    struct hfi_msg_session_init_done_pkt *pkt)
+{
+	struct device *dev = hfi->dev;
+	u32 rem_bytes, num_props;
+	u32 ptype, next_offset = 0;
+	u32 err;
+	u8 *data;
+
+	rem_bytes = pkt->shdr.hdr.size - sizeof(*pkt) + sizeof(u32);
+	if (!rem_bytes) {
+		dev_err(dev, "%s: missing property info\n", __func__);
+		return HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
+	}
+
+	err = pkt->error_type;
+	if (err)
+		return err;
+
+	data = (u8 *) &pkt->data[0];
+	num_props = pkt->num_properties;
+
+	while (err == HFI_ERR_NONE && num_props && rem_bytes >= sizeof(u32)) {
+		ptype = *((u32 *)data);
+		next_offset = sizeof(u32);
+
+		switch (ptype) {
+		case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED: {
+			struct hfi_capabilities *caps;
+			struct hfi_capability *cap;
+			u32 num_caps;
+
+			if ((rem_bytes - next_offset) < sizeof(*cap)) {
+				err = HFI_ERR_SESSION_INVALID_PARAMETER;
+				break;
+			}
+
+			caps = (struct hfi_capabilities *)(data + next_offset);
+
+			num_caps = caps->num_capabilities;
+			cap = &caps->data[0];
+			next_offset += sizeof(u32);
+
+			while (num_caps &&
+			      (rem_bytes - next_offset) >= sizeof(u32)) {
+				hfi_copy_cap_prop(cap, inst);
+				cap++;
+				next_offset += sizeof(*cap);
+				num_caps--;
+			}
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED: {
+			struct hfi_uncompressed_format_supported *prop =
+				(struct hfi_uncompressed_format_supported *)
+				(data + next_offset);
+			u32 num_fmt_entries;
+			u8 *fmt;
+			struct hfi_uncompressed_plane_info *inf;
+
+			if ((rem_bytes - next_offset) < sizeof(*prop)) {
+				err = HFI_ERR_SESSION_INVALID_PARAMETER;
+				break;
+			}
+
+			num_fmt_entries = prop->format_entries;
+			next_offset = sizeof(*prop) - sizeof(u32);
+			fmt = (u8 *)&prop->format_info[0];
+
+			dev_dbg(dev, "uncomm format support num entries:%u\n",
+				num_fmt_entries);
+
+			while (num_fmt_entries) {
+				struct hfi_uncompressed_plane_constraints *cnts;
+				u32 bytes_to_skip;
+
+				inf = (struct hfi_uncompressed_plane_info *)fmt;
+
+				if ((rem_bytes - next_offset) < sizeof(*inf)) {
+					err = HFI_ERR_SESSION_INVALID_PARAMETER;
+					break;
+				}
+
+				dev_dbg(dev, "plane info: fmt:%x, planes:%x\n",
+					inf->format, inf->num_planes);
+
+				cnts = &inf->plane_format[0];
+				dev_dbg(dev, "%u %u %u %u\n",
+					cnts->stride_multiples,
+					cnts->max_stride,
+					cnts->min_plane_buffer_height_multiple,
+					cnts->buffer_alignment);
+
+				bytes_to_skip = sizeof(*inf) - sizeof(*cnts) +
+						inf->num_planes * sizeof(*cnts);
+
+				fmt += bytes_to_skip;
+				next_offset += bytes_to_skip;
+				num_fmt_entries--;
+			}
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_PROPERTIES_SUPPORTED: {
+			struct hfi_properties_supported *prop =
+				(struct hfi_properties_supported *)
+				(data + next_offset);
+
+			next_offset += sizeof(*prop) - sizeof(u32)
+					+ prop->num_properties * sizeof(u32);
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED: {
+			struct hfi_profile_level_supported *prop =
+				(struct hfi_profile_level_supported *)
+				(data + next_offset);
+			struct hfi_profile_level *pl;
+			unsigned int prop_count = 0;
+			unsigned int count = 0;
+			u8 *ptr;
+
+			ptr = (u8 *)&prop->profile_level[0];
+			prop_count = prop->profile_count;
+
+			if (prop_count > HFI_MAX_PROFILE_COUNT)
+				prop_count = HFI_MAX_PROFILE_COUNT;
+
+			while (prop_count) {
+				ptr++;
+				pl = (struct hfi_profile_level *)ptr;
+
+				inst->pl[count].profile = pl->profile;
+				inst->pl[count].level = pl->level;
+				prop_count--;
+				count++;
+				ptr += sizeof(*pl) / sizeof(u32);
+			}
+
+			inst->pl_count = count;
+			next_offset += sizeof(*prop) - sizeof(*pl) +
+				       prop->profile_count * sizeof(*pl);
+
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SUPPORTED: {
+			struct hfi_nal_stream_format *nal =
+				(struct hfi_nal_stream_format *)
+				(data + next_offset);
+			dev_dbg(dev, "NAL format: %x\n", nal->format);
+			next_offset += sizeof(*nal);
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SELECT: {
+			next_offset += sizeof(u32);
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_MAX_SEQUENCE_HEADER_SIZE: {
+			u32 *max_seq_sz = (u32 *)(data + next_offset);
+
+			dev_dbg(dev, "max seq header sz: %x\n", *max_seq_sz);
+			next_offset += sizeof(u32);
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH: {
+			next_offset += sizeof(struct hfi_intra_refresh);
+			num_props--;
+			break;
+		}
+		case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED: {
+			struct hfi_buffer_alloc_mode_supported *prop =
+				(struct hfi_buffer_alloc_mode_supported *)
+				(data + next_offset);
+			int i;
+
+			if (prop->buffer_type == HFI_BUFFER_OUTPUT ||
+			    prop->buffer_type == HFI_BUFFER_OUTPUT2) {
+				for (i = 0; i < prop->num_entries; i++) {
+					switch (prop->data[i]) {
+					case HFI_BUFFER_MODE_STATIC:
+						inst->alloc_mode_static = true;
+						break;
+					case HFI_BUFFER_MODE_DYNAMIC:
+						inst->alloc_mode_dynamic = true;
+						break;
+					}
+				}
+			}
+			next_offset += sizeof(*prop) -
+				sizeof(u32) + prop->num_entries * sizeof(u32);
+			num_props--;
+			break;
+		}
+		default:
+			dev_dbg(dev, "%s: default case %#x\n", __func__, ptype);
+			break;
+		}
+
+		rem_bytes -= next_offset;
+		data += next_offset;
+	}
+
+	return err;
+}
+
+static void hfi_session_init_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				  void *packet)
+{
+	struct hfi_msg_session_init_done_pkt *pkt = packet;
+	unsigned int error;
+
+	error = pkt->error_type;
+	if (error != HFI_ERR_NONE)
+		goto done;
+
+	error = session_init_done_read_prop(hfi, inst, pkt);
+
+done:
+	inst->error = error;
+	complete(&inst->done);
+}
+
+static void hfi_session_load_res_done(struct hfi_core *hfi,
+				      struct hfi_inst *inst, void *packet)
+{
+	struct hfi_msg_session_load_resources_done_pkt *pkt = packet;
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_flush_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				   void *packet)
+{
+	struct hfi_msg_session_flush_done_pkt *pkt = packet;
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_etb_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				 void *packet)
+{
+	struct hfi_msg_session_empty_buffer_done_pkt *pkt = packet;
+	u32 flags = 0;
+
+	inst->error = pkt->error_type;
+
+	inst->ops->empty_buf_done(inst, pkt->input_tag, pkt->filled_len,
+				  pkt->offset, flags);
+}
+
+static void hfi_session_ftb_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				 void *packet)
+{
+	u32 session_type = inst->session_type;
+	struct hfi_msg_fbd fbd = {0};
+	struct timeval timestamp;
+	int64_t time_usec = 0;
+	unsigned int error;
+	u32 flags = 0;
+
+	if (session_type == VIDC_SESSION_TYPE_ENC) {
+		struct hfi_msg_session_fbd_compressed_pkt *pkt = packet;
+
+		fbd.timestamp_hi = pkt->time_stamp_hi;
+		fbd.timestamp_lo = pkt->time_stamp_lo;
+		fbd.flags1 = pkt->flags;
+		fbd.offset1 = pkt->offset;
+		fbd.alloc_len1 = pkt->alloc_len;
+		fbd.filled_len1 = pkt->filled_len;
+		fbd.picture_type = pkt->picture_type;
+		fbd.packet_buffer1 = pkt->packet_buffer;
+		fbd.extradata_buffer = pkt->extradata_buffer;
+		fbd.buffer_type = HFI_BUFFER_OUTPUT;
+
+		error = pkt->error_type;
+	} else if (session_type == VIDC_SESSION_TYPE_DEC) {
+		struct hfi_msg_session_fbd_uncompressed_plane0_pkt *pkt =
+			packet;
+
+		fbd.timestamp_hi = pkt->time_stamp_hi;
+		fbd.timestamp_lo = pkt->time_stamp_lo;
+		fbd.flags1 = pkt->flags;
+		fbd.offset1 = pkt->offset;
+		fbd.alloc_len1 = pkt->alloc_len;
+		fbd.filled_len1 = pkt->filled_len;
+		fbd.picture_type = pkt->picture_type;
+		fbd.packet_buffer1 = pkt->packet_buffer;
+		fbd.extradata_buffer = pkt->extradata_buffer;
+
+		if (pkt->stream_id == 0)
+			fbd.buffer_type = HFI_BUFFER_OUTPUT;
+		else if (pkt->stream_id == 1)
+			fbd.buffer_type = HFI_BUFFER_OUTPUT2;
+
+		error = pkt->error_type;
+	} else {
+		error = HFI_ERR_SESSION_INVALID_PARAMETER;
+	}
+
+	if (fbd.buffer_type != HFI_BUFFER_OUTPUT)
+		return;
+
+	if (fbd.flags1 & HFI_BUFFERFLAG_EOS)
+		flags |= V4L2_BUF_FLAG_LAST;
+
+	switch (fbd.picture_type) {
+	case HFI_PICTURE_IDR:
+		flags |= V4L2_BUF_FLAG_KEYFRAME;
+		break;
+	case HFI_PICTURE_I:
+		flags |= V4L2_BUF_FLAG_KEYFRAME;
+		break;
+	case HFI_PICTURE_P:
+		flags |= V4L2_BUF_FLAG_PFRAME;
+		break;
+	case HFI_PICTURE_B:
+		flags |= V4L2_BUF_FLAG_BFRAME;
+		break;
+	case HFI_FRAME_NOTCODED:
+	case HFI_UNUSED_PICT:
+	case HFI_FRAME_YUV:
+	default:
+		break;
+	}
+
+	if (!(fbd.flags1 & HFI_BUFFERFLAG_TIMESTAMPINVALID) &&
+	      fbd.filled_len1) {
+		time_usec = fbd.timestamp_hi;
+		time_usec = (time_usec << 32) | fbd.timestamp_lo;
+	}
+
+	timestamp = ns_to_timeval(time_usec * NSEC_PER_USEC);
+
+	inst->error = error;
+	inst->ops->fill_buf_done(inst, fbd.packet_buffer1, fbd.filled_len1,
+				 fbd.offset1, flags, &timestamp);
+}
+
+static void hfi_session_start_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				   void *packet)
+{
+	struct hfi_msg_session_start_done_pkt *pkt = packet;
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_stop_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				  void *packet)
+{
+	struct hfi_msg_session_stop_done_pkt *pkt = packet;
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_rel_res_done(struct hfi_core *hfi,
+				     struct hfi_inst *inst, void *packet)
+{
+	struct hfi_msg_session_release_resources_done_pkt *pkt = packet;
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_rel_buf_done(struct hfi_core *hfi,
+				     struct hfi_inst *inst, void *packet)
+{
+	struct hfi_msg_session_release_buffers_done_pkt *pkt = packet;
+
+	/*
+	 * the address of the released buffer can be extracted:
+	 * if (pkt->buffer_info) {
+	 *	cmd.data = &pkt->buffer_info;
+	 *	cmd.size = sizeof(struct hfi_buffer_info);
+	 * }
+	 */
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_end_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				 void *packet)
+{
+	struct hfi_msg_session_end_done_pkt *pkt = packet;
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_abort_done(struct hfi_core *hfi, struct hfi_inst *inst,
+				   void *packet)
+{
+	struct hfi_msg_sys_session_abort_done_pkt *pkt = packet;
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+static void hfi_session_get_seq_hdr_done(struct hfi_core *hfi,
+					 struct hfi_inst *inst, void *packet)
+{
+	struct hfi_msg_session_get_sequence_hdr_done_pkt *pkt = packet;
+
+	/*
+	 * output_done.packet_buffer1 = pkt->sequence_header;
+	 * output_done.filled_len1 = pkt->header_len;
+	 */
+
+	inst->error = pkt->error_type;
+	complete(&inst->done);
+}
+
+struct hfi_done_handler {
+	u32 pkt;
+	u32 pkt_sz;
+	u32 pkt_sz2;
+	void (*done)(struct hfi_core *, struct hfi_inst *, void *);
+	bool is_sys_pkt;
+};
+
+static const struct hfi_done_handler handlers[] = {
+	{.pkt = HFI_MSG_EVENT_NOTIFY,
+	 .pkt_sz = sizeof(struct hfi_msg_event_notify_pkt),
+	 .done = hfi_event_notify,
+	},
+	{.pkt = HFI_MSG_SYS_INIT,
+	 .pkt_sz = sizeof(struct hfi_msg_sys_init_done_pkt),
+	 .done = hfi_sys_init_done,
+	 .is_sys_pkt = true,
+	},
+	{.pkt = HFI_MSG_SYS_PROPERTY_INFO,
+	 .pkt_sz = sizeof(struct hfi_msg_sys_property_info_pkt),
+	 .done = hfi_sys_property_info,
+	 .is_sys_pkt = true,
+	},
+	{.pkt = HFI_MSG_SYS_RELEASE_RESOURCE,
+	 .pkt_sz = sizeof(struct hfi_msg_sys_release_resource_done_pkt),
+	 .done = hfi_sys_rel_resource_done,
+	 .is_sys_pkt = true,
+	},
+	{.pkt = HFI_MSG_SYS_PING_ACK,
+	 .pkt_sz = sizeof(struct hfi_msg_sys_ping_ack_pkt),
+	 .done = hfi_sys_ping_done,
+	 .is_sys_pkt = true,
+	},
+	{.pkt = HFI_MSG_SYS_IDLE,
+	 .pkt_sz = sizeof(struct hfi_msg_sys_idle_pkt),
+	 .done = hfi_sys_idle_done,
+	 .is_sys_pkt = true,
+	},
+	{.pkt = HFI_MSG_SYS_PC_PREP,
+	 .pkt_sz = sizeof(struct hfi_msg_sys_pc_prep_done_pkt),
+	 .done = hfi_sys_pc_prepare_done,
+	 .is_sys_pkt = true,
+	},
+	{.pkt = HFI_MSG_SYS_SESSION_INIT,
+	 .pkt_sz = sizeof(struct hfi_msg_session_init_done_pkt),
+	 .done = hfi_session_init_done,
+	},
+	{.pkt = HFI_MSG_SYS_SESSION_END,
+	 .pkt_sz = sizeof(struct hfi_msg_session_end_done_pkt),
+	 .done = hfi_session_end_done,
+	},
+	{.pkt = HFI_MSG_SESSION_LOAD_RESOURCES,
+	 .pkt_sz = sizeof(struct hfi_msg_session_load_resources_done_pkt),
+	 .done = hfi_session_load_res_done,
+	},
+	{.pkt = HFI_MSG_SESSION_START,
+	 .pkt_sz = sizeof(struct hfi_msg_session_start_done_pkt),
+	 .done = hfi_session_start_done,
+	},
+	{.pkt = HFI_MSG_SESSION_STOP,
+	 .pkt_sz = sizeof(struct hfi_msg_session_stop_done_pkt),
+	 .done = hfi_session_stop_done,
+	},
+	{.pkt = HFI_MSG_SYS_SESSION_ABORT,
+	 .pkt_sz = sizeof(struct hfi_msg_sys_session_abort_done_pkt),
+	 .done = hfi_session_abort_done,
+	},
+	{.pkt = HFI_MSG_SESSION_EMPTY_BUFFER,
+	 .pkt_sz = sizeof(struct hfi_msg_session_empty_buffer_done_pkt),
+	 .done = hfi_session_etb_done,
+	},
+	{.pkt = HFI_MSG_SESSION_FILL_BUFFER,
+	 .pkt_sz = sizeof(struct hfi_msg_session_fbd_uncompressed_plane0_pkt),
+	 .pkt_sz2 = sizeof(struct hfi_msg_session_fbd_compressed_pkt),
+	 .done = hfi_session_ftb_done,
+	},
+	{.pkt = HFI_MSG_SESSION_FLUSH,
+	 .pkt_sz = sizeof(struct hfi_msg_session_flush_done_pkt),
+	 .done = hfi_session_flush_done,
+	},
+	{.pkt = HFI_MSG_SESSION_PROPERTY_INFO,
+	 .pkt_sz = sizeof(struct hfi_msg_session_property_info_pkt),
+	 .done = hfi_session_prop_info,
+	},
+	{.pkt = HFI_MSG_SESSION_RELEASE_RESOURCES,
+	 .pkt_sz = sizeof(struct hfi_msg_session_release_resources_done_pkt),
+	 .done = hfi_session_rel_res_done,
+	},
+	{.pkt = HFI_MSG_SESSION_GET_SEQUENCE_HEADER,
+	 .pkt_sz = sizeof(struct hfi_msg_session_get_sequence_hdr_done_pkt),
+	 .done = hfi_session_get_seq_hdr_done,
+	},
+	{.pkt = HFI_MSG_SESSION_RELEASE_BUFFERS,
+	 .pkt_sz = sizeof(struct hfi_msg_session_release_buffers_done_pkt),
+	 .done = hfi_session_rel_buf_done,
+	},
+};
+
+void hfi_process_watchdog_timeout(struct hfi_core *hfi)
+{
+	event_sys_error(hfi, EVT_SYS_WATCHDOG_TIMEOUT);
+}
+
+u32 hfi_process_msg_packet(struct hfi_core *hfi, struct hfi_pkt_hdr *hdr)
+{
+	const struct hfi_done_handler *handler;
+	struct device *dev = hfi->dev;
+	struct hfi_inst *inst;
+	bool found = false;
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(handlers); i++) {
+		handler = &handlers[i];
+
+		if (handler->pkt != hdr->pkt_type)
+			continue;
+
+		found = true;
+		break;
+	}
+
+	if (found == false)
+		return hdr->pkt_type;
+
+	if (hdr->size && hdr->size < handler->pkt_sz &&
+	    hdr->size < handler->pkt_sz2) {
+		dev_err(dev, "bad packet size (%d should be %d, pkt type:%x)\n",
+			hdr->size, handler->pkt_sz, hdr->pkt_type);
+
+		return hdr->pkt_type;
+	}
+
+	if (handler->is_sys_pkt) {
+		inst = NULL;
+	} else {
+		struct hfi_session_pkt *pkt;
+
+		pkt = (struct hfi_session_pkt *)hdr;
+		inst = to_hfi_instance(hfi, pkt->shdr.session_id);
+
+		if (!inst)
+			dev_warn(dev, "no valid instance(pkt session_id:%x)\n",
+				 pkt->shdr.session_id);
+
+		/*
+		 * Event of type HFI_EVENT_SYS_ERROR will not have any session
+		 * associated with it
+		 */
+		if (!inst && hdr->pkt_type != HFI_MSG_EVENT_NOTIFY) {
+			dev_err(dev, "got invalid session id:%d\n",
+				pkt->shdr.session_id);
+			goto invalid_session;
+		}
+	}
+
+	handler->done(hfi, inst, hdr);
+
+invalid_session:
+	return hdr->pkt_type;
+}
diff --git a/drivers/media/platform/qcom/vidc/hfi_msgs.h b/drivers/media/platform/qcom/vidc/hfi_msgs.h
new file mode 100644
index 000000000000..34d1e8679709
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_msgs.h
@@ -0,0 +1,298 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __HFI_MSGS_H__
+#define __HFI_MSGS_H__
+
+#include "hfi_helper.h"
+
+/* message calls */
+#define HFI_MSG_SYS_COMMON_START	(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_COMMON_OFFSET +	\
+					 HFI_MSG_START_OFFSET + 0x0000)
+#define HFI_MSG_SYS_INIT		(HFI_MSG_SYS_COMMON_START + 0x1)
+#define HFI_MSG_SYS_PC_PREP		(HFI_MSG_SYS_COMMON_START + 0x2)
+#define HFI_MSG_SYS_RELEASE_RESOURCE	(HFI_MSG_SYS_COMMON_START + 0x3)
+#define HFI_MSG_SYS_DEBUG		(HFI_MSG_SYS_COMMON_START + 0x4)
+#define HFI_MSG_SYS_SESSION_INIT	(HFI_MSG_SYS_COMMON_START + 0x6)
+#define HFI_MSG_SYS_SESSION_END		(HFI_MSG_SYS_COMMON_START + 0x7)
+#define HFI_MSG_SYS_IDLE		(HFI_MSG_SYS_COMMON_START + 0x8)
+#define HFI_MSG_SYS_COV                 (HFI_MSG_SYS_COMMON_START + 0x9)
+#define HFI_MSG_SYS_PROPERTY_INFO	(HFI_MSG_SYS_COMMON_START + 0xa)
+
+#define HFI_MSG_SESSION_COMMON_START	(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_COMMON_OFFSET +	\
+					 HFI_MSG_START_OFFSET + 0x1000)
+#define HFI_MSG_EVENT_NOTIFY		(HFI_MSG_SESSION_COMMON_START + 0x1)
+#define HFI_MSG_SESSION_GET_SEQUENCE_HEADER \
+					(HFI_MSG_SESSION_COMMON_START + 0x2)
+
+#define HFI_MSG_SYS_OX_START		(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_OX_OFFSET +		\
+					 HFI_MSG_START_OFFSET + 0x0000)
+#define HFI_MSG_SYS_PING_ACK		(HFI_MSG_SYS_OX_START + 0x2)
+#define HFI_MSG_SYS_SESSION_ABORT	(HFI_MSG_SYS_OX_START + 0x4)
+
+#define HFI_MSG_SESSION_OX_START	(HFI_DOMAIN_BASE_COMMON +	\
+					 HFI_ARCH_OX_OFFSET +	\
+					 HFI_MSG_START_OFFSET + 0x1000)
+#define HFI_MSG_SESSION_LOAD_RESOURCES	(HFI_MSG_SESSION_OX_START + 0x1)
+#define HFI_MSG_SESSION_START		(HFI_MSG_SESSION_OX_START + 0x2)
+#define HFI_MSG_SESSION_STOP		(HFI_MSG_SESSION_OX_START + 0x3)
+#define HFI_MSG_SESSION_SUSPEND		(HFI_MSG_SESSION_OX_START + 0x4)
+#define HFI_MSG_SESSION_RESUME		(HFI_MSG_SESSION_OX_START + 0x5)
+#define HFI_MSG_SESSION_FLUSH		(HFI_MSG_SESSION_OX_START + 0x6)
+#define HFI_MSG_SESSION_EMPTY_BUFFER	(HFI_MSG_SESSION_OX_START + 0x7)
+#define HFI_MSG_SESSION_FILL_BUFFER	(HFI_MSG_SESSION_OX_START + 0x8)
+#define HFI_MSG_SESSION_PROPERTY_INFO		(HFI_MSG_SESSION_OX_START + 0x9)
+#define HFI_MSG_SESSION_RELEASE_RESOURCES	(HFI_MSG_SESSION_OX_START + 0xa)
+#define HFI_MSG_SESSION_PARSE_SEQUENCE_HEADER	(HFI_MSG_SESSION_OX_START + 0xb)
+#define HFI_MSG_SESSION_RELEASE_BUFFERS		(HFI_MSG_SESSION_OX_START + 0xc)
+
+#define HFI_PICTURE_I			0x01
+#define HFI_PICTURE_P			0x02
+#define HFI_PICTURE_B			0x04
+#define HFI_PICTURE_IDR			0x08
+#define HFI_FRAME_NOTCODED		0x7f002000
+#define HFI_FRAME_YUV			0x7f004000
+#define HFI_UNUSED_PICT			0x10000000
+
+/* message packets */
+struct hfi_msg_event_notify_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 event_id;
+	u32 event_data1;
+	u32 event_data2;
+	u32 ext_event_data[1];
+};
+
+struct hfi_msg_event_release_buffer_ref_pkt {
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 output_tag;
+};
+
+struct hfi_msg_sys_init_done_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 error_type;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_msg_sys_pc_prep_done_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 error_type;
+};
+
+struct hfi_msg_sys_release_resource_done_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 resource_handle;
+	u32 error_type;
+};
+
+struct hfi_msg_session_init_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_msg_session_end_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_session_get_sequence_hdr_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+	u32 header_len;
+	u32 sequence_header;
+};
+
+struct hfi_msg_sys_session_abort_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_sys_idle_pkt {
+	struct hfi_pkt_hdr hdr;
+};
+
+struct hfi_msg_sys_ping_ack_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 client_data;
+};
+
+struct hfi_msg_sys_property_info_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_msg_session_load_resources_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_session_start_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_session_stop_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_session_suspend_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_session_resume_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_session_flush_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+	u32 flush_type;
+};
+
+struct hfi_msg_session_empty_buffer_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+	u32 offset;
+	u32 filled_len;
+	u32 input_tag;
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 data[0];
+};
+
+struct hfi_msg_session_fbd_compressed_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 time_stamp_hi;
+	u32 time_stamp_lo;
+	u32 error_type;
+	u32 flags;
+	u32 mark_target;
+	u32 mark_data;
+	u32 stats;
+	u32 offset;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 input_tag;
+	u32 output_tag;
+	u32 picture_type;
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 data[0];
+};
+
+struct hfi_msg_session_fbd_uncompressed_plane0_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 stream_id;
+	u32 view_id;
+	u32 error_type;
+	u32 time_stamp_hi;
+	u32 time_stamp_lo;
+	u32 flags;
+	u32 mark_target;
+	u32 mark_data;
+	u32 stats;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 offset;
+	u32 frame_width;
+	u32 frame_height;
+	u32 start_x_coord;
+	u32 start_y_coord;
+	u32 input_tag;
+	u32 input_tag2;
+	u32 output_tag;
+	u32 picture_type;
+	u32 packet_buffer;
+	u32 extradata_buffer;
+	u32 data[0];
+};
+
+struct hfi_msg_session_fbd_uncompressed_plane1_pkt {
+	u32 flags;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 offset;
+	u32 packet_buffer2;
+	u32 data[0];
+};
+
+struct hfi_msg_session_fbd_uncompressed_plane2_pkt {
+	u32 flags;
+	u32 alloc_len;
+	u32 filled_len;
+	u32 offset;
+	u32 packet_buffer3;
+	u32 data[0];
+};
+
+struct hfi_msg_session_parse_sequence_header_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_msg_session_property_info_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 num_properties;
+	u32 data[1];
+};
+
+struct hfi_msg_session_release_resources_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+};
+
+struct hfi_msg_session_release_buffers_done_pkt {
+	struct hfi_session_hdr_pkt shdr;
+	u32 error_type;
+	u32 num_buffers;
+	u32 buffer_info[1];
+};
+
+struct hfi_msg_sys_debug_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 msg_type;
+	u32 msg_size;
+	u32 time_stamp_hi;
+	u32 time_stamp_lo;
+	u8 msg_data[1];
+};
+
+struct hfi_msg_sys_coverage_pkt {
+	struct hfi_pkt_hdr hdr;
+	u32 msg_size;
+	u32 time_stamp_hi;
+	u32 time_stamp_lo;
+	u8 msg_data[1];
+};
+
+struct hfi_core;
+struct hfi_pkt_hdr;
+
+void hfi_process_watchdog_timeout(struct hfi_core *hfi);
+u32 hfi_process_msg_packet(struct hfi_core *hfi, struct hfi_pkt_hdr *hdr);
+
+#endif
-- 
2.7.4

[PATCH 6/8] media: vidc: add Venus HFI files

[Stanimir Varbanov]

Here is the implementation of Venus video accelerator low-level
functionality. It contanins code which setup the registers and
startup uthe processor, allocate and manipulates with the shared
memory used for sending commands and receiving messages.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/vidc/hfi_venus.c    | 1539 +++++++++++++++++++++++
 drivers/media/platform/qcom/vidc/hfi_venus.h    |   25 +
 drivers/media/platform/qcom/vidc/hfi_venus_io.h |   98 ++
 3 files changed, 1662 insertions(+)
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.c
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.h
 create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus_io.h

diff --git a/drivers/media/platform/qcom/vidc/hfi_venus.c b/drivers/media/platform/qcom/vidc/hfi_venus.c
new file mode 100644
index 000000000000..1193fa136711
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_venus.c
@@ -0,0 +1,1539 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/qcom_scm.h>
+#include <linux/slab.h>
+
+#include "hfi_cmds.h"
+#include "hfi_msgs.h"
+#include "mem.h"
+#include "hfi_venus.h"
+#include "hfi_venus_io.h"
+
+#define HFI_MASK_QHDR_TX_TYPE		0xff000000
+#define HFI_MASK_QHDR_RX_TYPE		0x00ff0000
+#define HFI_MASK_QHDR_PRI_TYPE		0x0000ff00
+#define HFI_MASK_QHDR_ID_TYPE		0x000000ff
+
+#define HFI_HOST_TO_CTRL_CMD_Q		0
+#define HFI_CTRL_TO_HOST_MSG_Q		1
+#define HFI_CTRL_TO_HOST_DBG_Q		2
+#define HFI_MASK_QHDR_STATUS		0x000000ff
+
+#define IFACEQ_NUM			3
+#define IFACEQ_CMD_IDX			0
+#define IFACEQ_MSG_IDX			1
+#define IFACEQ_DBG_IDX			2
+#define IFACEQ_MAX_BUF_COUNT		50
+#define IFACEQ_MAX_PARALLEL_CLNTS	16
+#define IFACEQ_DFLT_QHDR		0x01010000
+
+#define POLL_INTERVAL_US		50
+
+#define IFACEQ_MAX_PKT_SIZE		1024
+#define IFACEQ_MED_PKT_SIZE		768
+#define IFACEQ_MIN_PKT_SIZE		8
+#define IFACEQ_VAR_SMALL_PKT_SIZE	100
+#define IFACEQ_VAR_LARGE_PKT_SIZE	512
+#define IFACEQ_VAR_HUGE_PKT_SIZE	(1024 * 12)
+
+enum tzbsp_video_state {
+	TZBSP_VIDEO_STATE_SUSPEND = 0,
+	TZBSP_VIDEO_STATE_RESUME
+};
+
+struct hfi_queue_table_header {
+	u32 version;
+	u32 size;
+	u32 qhdr0_offset;
+	u32 qhdr_size;
+	u32 num_q;
+	u32 num_active_q;
+};
+
+struct hfi_queue_header {
+	u32 status;
+	u32 start_addr;
+	u32 type;
+	u32 q_size;
+	u32 pkt_size;
+	u32 pkt_drop_cnt;
+	u32 rx_wm;
+	u32 tx_wm;
+	u32 rx_req;
+	u32 tx_req;
+	u32 rx_irq_status;
+	u32 tx_irq_status;
+	u32 read_idx;
+	u32 write_idx;
+};
+
+#define IFACEQ_TABLE_SIZE	\
+	(sizeof(struct hfi_queue_table_header) +	\
+	 sizeof(struct hfi_queue_header) * IFACEQ_NUM)
+
+#define IFACEQ_QUEUE_SIZE	(IFACEQ_MAX_PKT_SIZE *	\
+	IFACEQ_MAX_BUF_COUNT * IFACEQ_MAX_PARALLEL_CLNTS)
+
+#define IFACEQ_GET_QHDR_START_ADDR(ptr, i)	\
+	(void *)(((ptr) + sizeof(struct hfi_queue_table_header)) +	\
+		((i) * sizeof(struct hfi_queue_header)))
+
+#define QDSS_SIZE		SZ_4K
+#define SFR_SIZE		SZ_4K
+#define QUEUE_SIZE		\
+	(IFACEQ_TABLE_SIZE + (IFACEQ_QUEUE_SIZE * IFACEQ_NUM))
+
+#define ALIGNED_QDSS_SIZE	ALIGN(QDSS_SIZE, SZ_4K)
+#define ALIGNED_SFR_SIZE	ALIGN(SFR_SIZE, SZ_4K)
+#define ALIGNED_QUEUE_SIZE	ALIGN(QUEUE_SIZE, SZ_4K)
+#define SHARED_QSIZE		ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE + \
+				      ALIGNED_QDSS_SIZE, SZ_1M)
+
+struct mem_desc {
+	u32 da;		/* device address */
+	void *kva;	/* kernel virtual address */
+	u32 size;
+	struct vidc_mem *mem;
+};
+
+struct iface_queue {
+	struct hfi_queue_header *qhdr;
+	struct mem_desc qmem;
+};
+
+enum venus_state {
+	VENUS_STATE_DEINIT = 1,
+	VENUS_STATE_INIT,
+};
+
+struct venus_hfi_device {
+	struct device *dev;
+	void __iomem *base;
+	u32 irq_status;
+	u32 last_packet_type;
+	bool power_enabled;
+	bool suspended;
+	struct completion pwr_collapse_prep;
+	struct completion release_resource;
+	struct mutex lock;
+	struct mem_desc ifaceq_table;
+	struct mem_desc sfr;
+	struct iface_queue queues[IFACEQ_NUM];
+	const struct vidc_resources *res;
+	enum venus_state state;
+	const struct hfi_packetization_ops *pkt_ops;
+	enum hfi_packetization_type packetization_type;
+	u8 pkt_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
+	u8 dbg_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
+};
+
+#define VENUS_FW_DEBUG_MSG_LOW			0x1
+#define VENUS_FW_DEBUG_MSG_MEDIUM		0x2
+#define VENUS_FW_DEBUG_MSG_HIGH			0x4
+#define VENUS_FW_DEBUG_MSG_ERROR		0x8
+#define VENUS_FW_DEBUG_MSG_FATAL		0x10
+
+static bool venus_pkt_debug;
+static int venus_fw_debug = VENUS_FW_DEBUG_MSG_ERROR | VENUS_FW_DEBUG_MSG_FATAL;
+static bool venus_sys_idle_indicator;
+static bool venus_fw_low_power_mode = true;
+static int venus_hw_rsp_timeout = 1000;
+static bool venus_fw_coverage;
+
+static void venus_set_state(struct venus_hfi_device *hdev,
+			    enum venus_state state)
+{
+	mutex_lock(&hdev->lock);
+	hdev->state = state;
+	mutex_unlock(&hdev->lock);
+}
+
+static bool venus_is_valid_state(struct venus_hfi_device *hdev)
+{
+	return hdev->state != VENUS_STATE_DEINIT;
+}
+
+static void venus_dump_packet(struct venus_hfi_device *hdev, u8 *packet)
+{
+	struct device *dev = hdev->dev;
+	u32 c = 0, pkt_size = *(u32 *)packet;
+	const int row_size = 32;
+	/*
+	 * row must contain enough for 0xdeadbaad * 8 to be converted into
+	 * "de ad ba ab " * 8 + '\0'
+	 */
+	char row[3 * row_size];
+
+	if (!venus_pkt_debug)
+		return;
+
+	for (c = 0; c * row_size < pkt_size; ++c) {
+		int bytes_to_read = ((c + 1) * row_size > pkt_size) ?
+			pkt_size % row_size : row_size;
+		hex_dump_to_buffer(packet + c * row_size, bytes_to_read,
+				   row_size, 4, row, sizeof(row), false);
+		dev_dbg(dev, "%s\n", row);
+	}
+}
+
+static int venus_write_queue(struct venus_hfi_device *hdev,
+			     struct iface_queue *queue,
+			     void *packet, u32 *rx_req)
+{
+	struct hfi_queue_header *qhdr;
+	u32 dwords, new_wr_idx;
+	u32 empty_space, rd_idx, wr_idx, qsize;
+	u32 *wr_ptr;
+
+	if (!queue->qmem.kva)
+		return -EINVAL;
+
+	qhdr = queue->qhdr;
+	if (!qhdr)
+		return -EINVAL;
+
+	venus_dump_packet(hdev, packet);
+
+	dwords = (*(u32 *)packet) >> 2;
+	if (!dwords)
+		return -EINVAL;
+
+	rd_idx = qhdr->read_idx;
+	wr_idx = qhdr->write_idx;
+	qsize = qhdr->q_size;
+	/* ensure rd/wr indices's are read from memory */
+	rmb();
+
+	if (wr_idx >= rd_idx)
+		empty_space = qsize - (wr_idx - rd_idx);
+	else
+		empty_space = rd_idx - wr_idx;
+
+	if (empty_space <= dwords) {
+		qhdr->tx_req = 1;
+		/* ensure tx_req is updated in memory */
+		wmb();
+		return -ENOSPC;
+	}
+
+	qhdr->tx_req = 0;
+	/* ensure tx_req is updated in memory */
+	wmb();
+
+	new_wr_idx = wr_idx + dwords;
+	wr_ptr = (u32 *)(queue->qmem.kva + (wr_idx << 2));
+	if (new_wr_idx < qsize) {
+		memcpy(wr_ptr, packet, dwords << 2);
+	} else {
+		size_t len;
+
+		new_wr_idx -= qsize;
+		len = (dwords - new_wr_idx) << 2;
+		memcpy(wr_ptr, packet, len);
+		memcpy(queue->qmem.kva, packet + len, new_wr_idx << 2);
+	}
+
+	/* make sure packet is written before updating the write index */
+	wmb();
+
+	qhdr->write_idx = new_wr_idx;
+	*rx_req = qhdr->rx_req ? 1 : 0;
+
+	/* make sure write index is updated before an interupt is raised */
+	mb();
+
+	return 0;
+}
+
+static int venus_read_queue(struct venus_hfi_device *hdev,
+			    struct iface_queue *queue, void *pkt, u32 *tx_req)
+{
+	struct hfi_queue_header *qhdr;
+	u32 dwords, new_rd_idx;
+	u32 rd_idx, wr_idx, type, qsize;
+	u32 *rd_ptr;
+	u32 recv_request = 0;
+	int ret = 0;
+
+	if (!queue->qmem.kva)
+		return -EINVAL;
+
+	qhdr = queue->qhdr;
+	if (!qhdr)
+		return -EINVAL;
+
+	type = qhdr->type;
+	rd_idx = qhdr->read_idx;
+	wr_idx = qhdr->write_idx;
+	qsize = qhdr->q_size;
+
+	/* make sure data is valid before using it */
+	rmb();
+
+	/*
+	 * Do not set receive request for debug queue, if set, Venus generates
+	 * interrupt for debug messages even when there is no response message
+	 * available. In general debug queue will not become full as it is being
+	 * emptied out for every interrupt from Venus. Venus will anyway
+	 * generates interrupt if it is full.
+	 */
+	if (type & HFI_CTRL_TO_HOST_MSG_Q)
+		recv_request = 1;
+
+	if (rd_idx == wr_idx) {
+		qhdr->rx_req = recv_request;
+		*tx_req = 0;
+		/* update rx_req field in memory */
+		wmb();
+		return -ENODATA;
+	}
+
+	rd_ptr = (u32 *)(queue->qmem.kva + (rd_idx << 2));
+	dwords = *rd_ptr >> 2;
+	if (!dwords)
+		return -EINVAL;
+
+	new_rd_idx = rd_idx + dwords;
+	if (((dwords << 2) <= IFACEQ_VAR_HUGE_PKT_SIZE) && rd_idx <= qsize) {
+		if (new_rd_idx < qsize) {
+			memcpy(pkt, rd_ptr, dwords << 2);
+		} else {
+			size_t len;
+
+			new_rd_idx -= qsize;
+			len = (dwords - new_rd_idx) << 2;
+			memcpy(pkt, rd_ptr, len);
+			memcpy(pkt + len, queue->qmem.kva, new_rd_idx << 2);
+		}
+	} else {
+		/* bad packet received, dropping */
+		new_rd_idx = qhdr->write_idx;
+		ret = -EBADMSG;
+	}
+
+	/* ensure the packet is read before updating read index */
+	rmb();
+
+	qhdr->read_idx = new_rd_idx;
+	/* ensure updating read index */
+	wmb();
+
+	rd_idx = qhdr->read_idx;
+	wr_idx = qhdr->write_idx;
+	/* ensure rd/wr indices are read from memory */
+	rmb();
+
+	if (rd_idx != wr_idx)
+		qhdr->rx_req = 0;
+	else
+		qhdr->rx_req = recv_request;
+
+	*tx_req = qhdr->tx_req ? 1 : 0;
+
+	/* ensure rx_req is stored to memory and tx_req is loaded from memory */
+	mb();
+
+	venus_dump_packet(hdev, pkt);
+
+	return ret;
+}
+
+static int venus_alloc(struct venus_hfi_device *hdev, struct mem_desc *desc,
+		       u32 size)
+{
+	struct vidc_mem *mem;
+
+	mem = mem_alloc(hdev->dev, size, 1);
+	if (IS_ERR(mem))
+		return PTR_ERR(mem);
+
+	desc->size = mem->size;
+	desc->mem = mem;
+	desc->kva = mem->kvaddr;
+	desc->da = mem->da;
+
+	return 0;
+}
+
+static void venus_free(struct vidc_mem *mem)
+{
+	mem_free(mem);
+}
+
+static void venus_writel(struct venus_hfi_device *hdev, u32 reg, u32 value)
+{
+	writel(value, hdev->base + reg);
+}
+
+static u32 venus_readl(struct venus_hfi_device *hdev, u32 reg)
+{
+	return readl(hdev->base + reg);
+}
+
+static void venus_set_registers(struct venus_hfi_device *hdev)
+{
+	const struct reg_val *tbl = hdev->res->reg_tbl;
+	unsigned int count = hdev->res->reg_tbl_size;
+	int i;
+
+	for (i = 0; i < count; i++)
+		venus_writel(hdev, tbl[i].reg, tbl[i].value);
+}
+
+static void venus_soft_int(struct venus_hfi_device *hdev)
+{
+	venus_writel(hdev, CPU_IC_SOFTINT, BIT(CPU_IC_SOFTINT_H2A_SHIFT));
+}
+
+static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
+					 void *pkt)
+{
+	struct device *dev = hdev->dev;
+	struct hfi_pkt_hdr *cmd_packet;
+	struct iface_queue *queue;
+	u32 rx_req;
+	int ret;
+
+	WARN(!mutex_is_locked(&hdev->lock),
+	     "cmd queue write lock must be acquired");
+
+	if (!venus_is_valid_state(hdev)) {
+		dev_err(dev, "%s: fw not in init state\n", __func__);
+		return -EINVAL;
+	}
+
+	cmd_packet = (struct hfi_pkt_hdr *)pkt;
+	hdev->last_packet_type = cmd_packet->pkt_type;
+
+	queue = &hdev->queues[IFACEQ_CMD_IDX];
+
+	ret = venus_write_queue(hdev, queue, pkt, &rx_req);
+	if (ret) {
+		dev_err(dev, "write to iface cmd queue failed (%d)\n", ret);
+		return ret;
+	}
+
+	if (rx_req)
+		venus_soft_int(hdev);
+
+	return 0;
+}
+
+static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt)
+{
+	int ret;
+
+	mutex_lock(&hdev->lock);
+	ret = venus_iface_cmdq_write_nolock(hdev, pkt);
+	mutex_unlock(&hdev->lock);
+
+	return ret;
+}
+
+static int venus_hfi_core_set_resource(struct venus_hfi_device *hdev,
+				       u32 id, u32 size, u32 addr, void *cookie)
+{
+	struct hfi_sys_set_resource_pkt *pkt;
+	u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+	int ret;
+
+	if (id == VIDC_RESOURCE_NONE)
+		return 0;
+
+	pkt = (struct hfi_sys_set_resource_pkt *) packet;
+
+	ret = call_hfi_pkt_op(hdev, sys_set_resource, pkt, id, size, addr,
+			      cookie);
+	if (ret)
+		return ret;
+
+	ret = venus_iface_cmdq_write(hdev, pkt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int venus_tzbsp_set_video_state(enum tzbsp_video_state state)
+{
+	return qcom_scm_video_set_state(state, 0);
+}
+
+static int venus_reset_core(struct venus_hfi_device *hdev)
+{
+	struct device *dev = hdev->dev;
+	u32 ctrl_status = 0, count = 0;
+	int max_tries = 100, ret = 0;
+
+	venus_writel(hdev, VIDC_CTRL_INIT, BIT(VIDC_CTRL_INIT_CTRL_SHIFT));
+	venus_writel(hdev, WRAPPER_INTR_MASK, WRAPPER_INTR_MASK_A2HVCODEC_MASK);
+	venus_writel(hdev, CPU_CS_SCIACMDARG3, 1);
+
+	while (!ctrl_status && count < max_tries) {
+		ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+		if ((ctrl_status & 0xfe) == 0x4) {
+			dev_err(dev, "invalid setting for UC_REGION\n");
+			ret = -EINVAL;
+			break;
+		}
+
+		usleep_range(500, 1000);
+		count++;
+	}
+
+	if (count >= max_tries)
+		ret = -ETIMEDOUT;
+
+	return ret;
+}
+
+static u32 venus_hwversion(struct venus_hfi_device *hdev)
+{
+	struct device *dev = hdev->dev;
+	u32 ver = venus_readl(hdev, WRAPPER_HW_VERSION);
+	u32 major, minor, step;
+
+	major = ver & WRAPPER_HW_VERSION_MAJOR_VERSION_MASK;
+	major = major >> WRAPPER_HW_VERSION_MAJOR_VERSION_SHIFT;
+	minor = ver & WRAPPER_HW_VERSION_MINOR_VERSION_MASK;
+	minor = minor >> WRAPPER_HW_VERSION_MINOR_VERSION_SHIFT;
+	step = ver & WRAPPER_HW_VERSION_STEP_VERSION_MASK;
+
+	dev_dbg(dev, "venus hw version %d.%d.%d\n", major, minor, step);
+
+	return major;
+}
+
+static int venus_run(struct venus_hfi_device *hdev)
+{
+	struct device *dev = hdev->dev;
+	int ret;
+
+	/*
+	 * Re-program all of the registers that get reset as a result of
+	 * regulator_disable() and _enable()
+	 */
+	venus_set_registers(hdev);
+
+	venus_writel(hdev, UC_REGION_ADDR, hdev->ifaceq_table.da);
+	venus_writel(hdev, UC_REGION_SIZE, SHARED_QSIZE);
+	venus_writel(hdev, CPU_CS_SCIACMDARG2, hdev->ifaceq_table.da);
+	venus_writel(hdev, CPU_CS_SCIACMDARG1, 0x01);
+	if (hdev->sfr.da)
+		venus_writel(hdev, SFR_ADDR, hdev->sfr.da);
+
+	ret = venus_reset_core(hdev);
+	if (ret) {
+		dev_err(dev, "failed to reset venus core\n");
+		return ret;
+	}
+
+	venus_hwversion(hdev);
+
+	return 0;
+}
+
+static int venus_halt_axi(struct venus_hfi_device *hdev)
+{
+	struct device *dev = hdev->dev;
+	void __iomem *base = hdev->base;
+	u32 val;
+	int ret;
+
+	/* Halt AXI and AXI IMEM VBIF Access */
+	val = venus_readl(hdev, VBIF_AXI_HALT_CTRL0);
+	val |= VBIF_AXI_HALT_CTRL0_HALT_REQ;
+	venus_writel(hdev, VBIF_AXI_HALT_CTRL0, val);
+
+	/* Request for AXI bus port halt */
+	ret = readl_poll_timeout(base + VBIF_AXI_HALT_CTRL1, val,
+				 val & VBIF_AXI_HALT_CTRL1_HALT_ACK,
+				 POLL_INTERVAL_US,
+				 VBIF_AXI_HALT_ACK_TIMEOUT_US);
+	if (ret) {
+		dev_err(dev, "AXI bus port halt timeout\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static int venus_power_off(struct venus_hfi_device *hdev)
+{
+	int ret;
+
+	if (!hdev->power_enabled)
+		return 0;
+
+	ret = venus_halt_axi(hdev);
+	if (ret)
+		return ret;
+
+	ret = venus_tzbsp_set_video_state(TZBSP_VIDEO_STATE_SUSPEND);
+	if (ret)
+		return ret;
+
+	hdev->power_enabled = false;
+
+	return 0;
+}
+
+static int venus_power_on(struct venus_hfi_device *hdev)
+{
+	int ret;
+
+	if (hdev->power_enabled)
+		return 0;
+
+	ret = venus_tzbsp_set_video_state(TZBSP_VIDEO_STATE_RESUME);
+	if (ret)
+		goto err;
+
+	ret = venus_run(hdev);
+	if (ret)
+		goto err_suspend;
+
+	hdev->power_enabled = true;
+
+	return 0;
+
+err_suspend:
+	venus_tzbsp_set_video_state(TZBSP_VIDEO_STATE_SUSPEND);
+err:
+	hdev->power_enabled = false;
+	return ret;
+}
+
+static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev,
+					void *pkt)
+{
+	struct iface_queue *queue;
+	u32 tx_req;
+	int ret;
+
+	if (!venus_is_valid_state(hdev))
+		return -EINVAL;
+
+	queue = &hdev->queues[IFACEQ_MSG_IDX];
+
+	ret = venus_read_queue(hdev, queue, pkt, &tx_req);
+	if (ret)
+		return ret;
+
+	if (tx_req)
+		venus_soft_int(hdev);
+
+	return 0;
+}
+
+static int venus_iface_msgq_read(struct venus_hfi_device *hdev, void *pkt)
+{
+	int ret;
+
+	mutex_lock(&hdev->lock);
+	ret = venus_iface_msgq_read_nolock(hdev, pkt);
+	mutex_unlock(&hdev->lock);
+
+	return ret;
+}
+
+static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev,
+					void *pkt)
+{
+	struct iface_queue *queue;
+	u32 tx_req;
+	int ret;
+
+	ret = venus_is_valid_state(hdev);
+	if (!ret)
+		return -EINVAL;
+
+	queue = &hdev->queues[IFACEQ_DBG_IDX];
+
+	ret = venus_read_queue(hdev, queue, pkt, &tx_req);
+	if (ret)
+		return ret;
+
+	if (tx_req)
+		venus_soft_int(hdev);
+
+	return 0;
+}
+
+static int venus_iface_dbgq_read(struct venus_hfi_device *hdev, void *pkt)
+{
+	int ret;
+
+	if (!pkt)
+		return -EINVAL;
+
+	mutex_lock(&hdev->lock);
+	ret = venus_iface_dbgq_read_nolock(hdev, pkt);
+	mutex_unlock(&hdev->lock);
+
+	return ret;
+}
+
+static void venus_set_qhdr_defaults(struct hfi_queue_header *qhdr)
+{
+	qhdr->status = 1;
+	qhdr->type = IFACEQ_DFLT_QHDR;
+	qhdr->q_size = IFACEQ_QUEUE_SIZE / 4;
+	qhdr->pkt_size = 0;
+	qhdr->rx_wm = 1;
+	qhdr->tx_wm = 1;
+	qhdr->rx_req = 1;
+	qhdr->tx_req = 0;
+	qhdr->rx_irq_status = 0;
+	qhdr->tx_irq_status = 0;
+	qhdr->read_idx = 0;
+	qhdr->write_idx = 0;
+}
+
+static void venus_interface_queues_release(struct venus_hfi_device *hdev)
+{
+	mutex_lock(&hdev->lock);
+
+	venus_free(hdev->ifaceq_table.mem);
+	venus_free(hdev->sfr.mem);
+
+	memset(hdev->queues, 0, sizeof(hdev->queues));
+	memset(&hdev->ifaceq_table, 0, sizeof(hdev->ifaceq_table));
+	memset(&hdev->sfr, 0, sizeof(hdev->sfr));
+
+	mutex_unlock(&hdev->lock);
+}
+
+static int venus_interface_queues_init(struct venus_hfi_device *hdev)
+{
+	struct hfi_queue_table_header *tbl_hdr;
+	struct iface_queue *queue;
+	struct hfi_sfr *sfr;
+	struct mem_desc desc = {0};
+	unsigned int offset;
+	unsigned int i;
+	int ret;
+
+	ret = venus_alloc(hdev, &desc, ALIGNED_QUEUE_SIZE);
+	if (ret)
+		return ret;
+
+	hdev->ifaceq_table.kva = desc.kva;
+	hdev->ifaceq_table.da = desc.da;
+	hdev->ifaceq_table.size = IFACEQ_TABLE_SIZE;
+	hdev->ifaceq_table.mem = desc.mem;
+	offset = hdev->ifaceq_table.size;
+
+	for (i = 0; i < IFACEQ_NUM; i++) {
+		queue = &hdev->queues[i];
+		queue->qmem.da = desc.da + offset;
+		queue->qmem.kva = desc.kva + offset;
+		queue->qmem.size = IFACEQ_QUEUE_SIZE;
+		queue->qmem.mem = NULL;
+		offset += queue->qmem.size;
+		queue->qhdr =
+			IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
+
+		venus_set_qhdr_defaults(queue->qhdr);
+
+		queue->qhdr->start_addr = queue->qmem.da;
+
+		if (i == IFACEQ_CMD_IDX)
+			queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
+		else if (i == IFACEQ_MSG_IDX)
+			queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
+		else if (i == IFACEQ_DBG_IDX)
+			queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
+	}
+
+	tbl_hdr = hdev->ifaceq_table.kva;
+	tbl_hdr->version = 0;
+	tbl_hdr->size = IFACEQ_TABLE_SIZE;
+	tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
+	tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
+	tbl_hdr->num_q = IFACEQ_NUM;
+	tbl_hdr->num_active_q = IFACEQ_NUM;
+
+	/*
+	 * Set receive request to zero on debug queue as there is no
+	 * need of interrupt from video hardware for debug messages
+	 */
+	queue = &hdev->queues[IFACEQ_DBG_IDX];
+	queue->qhdr->rx_req = 0;
+
+	ret = venus_alloc(hdev, &desc, ALIGNED_SFR_SIZE);
+	if (ret) {
+		hdev->sfr.da = 0;
+	} else {
+		hdev->sfr.da = desc.da;
+		hdev->sfr.kva = desc.kva;
+		hdev->sfr.size = ALIGNED_SFR_SIZE;
+		hdev->sfr.mem = desc.mem;
+		sfr = hdev->sfr.kva;
+		sfr->buf_size = ALIGNED_SFR_SIZE;
+	}
+
+	/* ensure table and queue header structs are settled in memory */
+	wmb();
+
+	return 0;
+}
+
+static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug)
+{
+	struct hfi_sys_set_property_pkt *pkt;
+	u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+	int ret;
+
+	pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+	call_hfi_pkt_op(hdev, sys_debug_config, pkt, HFI_DEBUG_MODE_QUEUE,
+			debug);
+
+	ret = venus_iface_cmdq_write(hdev, pkt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode)
+{
+	struct hfi_sys_set_property_pkt *pkt;
+	u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+	int ret;
+
+	pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+	call_hfi_pkt_op(hdev, sys_coverage_config, pkt, mode);
+
+	ret = venus_iface_cmdq_write(hdev, pkt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int venus_sys_set_idle_message(struct venus_hfi_device *hdev,
+				      bool enable)
+{
+	struct hfi_sys_set_property_pkt *pkt;
+	u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+	int ret;
+
+	if (!enable)
+		return 0;
+
+	pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+	call_hfi_pkt_op(hdev, sys_idle_indicator, pkt, enable);
+
+	ret = venus_iface_cmdq_write(hdev, pkt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int venus_sys_set_power_control(struct venus_hfi_device *hdev,
+				       bool enable)
+{
+	struct hfi_sys_set_property_pkt *pkt;
+	u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+	int ret;
+
+	pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+	call_hfi_pkt_op(hdev, sys_power_control, pkt, enable);
+
+	ret = venus_iface_cmdq_write(hdev, pkt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int venus_get_queue_size(struct venus_hfi_device *hdev,
+				unsigned int index)
+{
+	struct hfi_queue_header *qhdr;
+
+	if (index >= IFACEQ_NUM)
+		return -EINVAL;
+
+	qhdr = hdev->queues[index].qhdr;
+	if (!qhdr)
+		return -EINVAL;
+
+	return abs(qhdr->read_idx - qhdr->write_idx);
+}
+
+static int venus_sys_set_default_properties(struct venus_hfi_device *hdev)
+{
+	struct device *dev = hdev->dev;
+	int ret;
+
+	ret = venus_sys_set_debug(hdev, venus_fw_debug);
+	if (ret)
+		dev_warn(dev, "setting fw debug msg ON failed (%d)\n", ret);
+
+	ret = venus_sys_set_idle_message(hdev, venus_sys_idle_indicator);
+	if (ret)
+		dev_warn(dev, "setting idle response ON failed (%d)\n", ret);
+
+	ret = venus_sys_set_power_control(hdev, venus_fw_low_power_mode);
+	if (ret)
+		dev_warn(dev, "setting hw power collapse ON failed (%d)\n",
+			 ret);
+
+	return ret;
+}
+
+static int venus_session_cmd(struct hfi_inst *inst, u32 pkt_type)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_pkt pkt;
+	int ret;
+
+	call_hfi_pkt_op(hdev, session_cmd, &pkt, pkt_type, inst);
+
+	ret = venus_iface_cmdq_write(hdev, &pkt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static void venus_flush_debug_queue(struct venus_hfi_device *hdev)
+{
+	void *packet = hdev->dbg_buf;
+	struct device *dev = hdev->dev;
+
+	while (!venus_iface_dbgq_read(hdev, packet)) {
+		struct hfi_msg_sys_coverage_pkt *pkt = packet;
+
+		if (pkt->hdr.pkt_type != HFI_MSG_SYS_COV) {
+			struct hfi_msg_sys_debug_pkt *pkt = packet;
+
+			dev_dbg(dev, "%s", pkt->msg_data);
+		}
+	}
+}
+
+static int venus_prepare_power_collapse(struct venus_hfi_device *hdev)
+{
+	unsigned long timeout = msecs_to_jiffies(venus_hw_rsp_timeout);
+	struct hfi_sys_pc_prep_pkt pkt;
+	int ret;
+
+	init_completion(&hdev->pwr_collapse_prep);
+
+	call_hfi_pkt_op(hdev, sys_pc_prep, &pkt);
+
+	ret = venus_iface_cmdq_write(hdev, &pkt);
+	if (ret)
+		return ret;
+
+	ret = wait_for_completion_timeout(&hdev->pwr_collapse_prep, timeout);
+	if (!ret) {
+		venus_flush_debug_queue(hdev);
+		return -ETIMEDOUT;
+	}
+
+	return 0;
+}
+
+static int venus_are_queues_empty(struct venus_hfi_device *hdev)
+{
+	int ret1, ret2;
+
+	ret1 = venus_get_queue_size(hdev, IFACEQ_MSG_IDX);
+	if (ret1 < 0)
+		return ret1;
+
+	ret2 = venus_get_queue_size(hdev, IFACEQ_CMD_IDX);
+	if (ret2 < 0)
+		return ret2;
+
+	if (!ret1 && !ret2)
+		return 1;
+
+	return 0;
+}
+
+static void venus_sfr_print(struct venus_hfi_device *hdev)
+{
+	struct device *dev = hdev->dev;
+	struct hfi_sfr *sfr = hdev->sfr.kva;
+	void *p;
+
+	if (!sfr)
+		return;
+
+	p = memchr(sfr->data, '\0', sfr->buf_size);
+	/*
+	 * SFR isn't guaranteed to be NULL terminated since SYS_ERROR indicates
+	 * that Venus is in the process of crashing.
+	 */
+	if (p == NULL)
+		sfr->data[sfr->buf_size - 1] = '\0';
+
+	dev_err(dev, "SFR message from FW: %s\n", sfr->data);
+}
+
+static void venus_process_msg_sys_error(struct venus_hfi_device *hdev,
+					void *packet)
+{
+	struct hfi_msg_event_notify_pkt *event_pkt = packet;
+
+	if (event_pkt->event_id != HFI_EVENT_SYS_ERROR)
+		return;
+
+	venus_set_state(hdev, VENUS_STATE_DEINIT);
+
+	/*
+	 * Once SYS_ERROR received from HW, it is safe to halt the AXI.
+	 * With SYS_ERROR, Venus FW may have crashed and HW might be
+	 * active and causing unnecessary transactions. Hence it is
+	 * safe to stop all AXI transactions from venus subsystem.
+	 */
+	venus_halt_axi(hdev);
+	venus_sfr_print(hdev);
+}
+
+static irqreturn_t venus_isr_thread(int irq, struct hfi_core *hfi)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	void *pkt = hdev->pkt_buf;
+	u32 msg_ret;
+
+	if (!hdev)
+		return IRQ_NONE;
+
+	if (hdev->irq_status & WRAPPER_INTR_STATUS_A2HWD_MASK) {
+		venus_sfr_print(hdev);
+		hfi_process_watchdog_timeout(hfi);
+	}
+
+	while (!venus_iface_msgq_read(hdev, pkt)) {
+		msg_ret = hfi_process_msg_packet(hfi, pkt);
+		switch (msg_ret) {
+		case HFI_MSG_EVENT_NOTIFY:
+			venus_process_msg_sys_error(hdev, pkt);
+			break;
+		case HFI_MSG_SYS_INIT:
+			venus_hfi_core_set_resource(hdev, hdev->res->vmem_id,
+						    hdev->res->vmem_size,
+						    hdev->res->vmem_addr,
+						    hdev);
+			break;
+		case HFI_MSG_SYS_RELEASE_RESOURCE:
+			complete(&hdev->release_resource);
+			break;
+		case HFI_MSG_SYS_PC_PREP:
+			complete(&hdev->pwr_collapse_prep);
+			break;
+		default:
+			break;
+		}
+	}
+
+	venus_flush_debug_queue(hdev);
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t venus_isr(int irq, struct hfi_core *hfi)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	u32 status;
+
+	if (!hdev)
+		return IRQ_NONE;
+
+	status = venus_readl(hdev, WRAPPER_INTR_STATUS);
+
+	if (status & WRAPPER_INTR_STATUS_A2H_MASK ||
+	    status & WRAPPER_INTR_STATUS_A2HWD_MASK ||
+	    status & CPU_CS_SCIACMDARG0_INIT_IDLE_MSG_MASK)
+		hdev->irq_status = status;
+
+	venus_writel(hdev, CPU_CS_A2HSOFTINTCLR, 1);
+	venus_writel(hdev, WRAPPER_INTR_CLEAR, status);
+
+	return IRQ_WAKE_THREAD;
+}
+
+static int venus_hfi_core_init(struct hfi_core *hfi)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	struct device *dev = hfi->dev;
+	struct hfi_sys_get_property_pkt version_pkt;
+	struct hfi_sys_init_pkt pkt;
+	int ret;
+
+	call_hfi_pkt_op(hdev, sys_init, &pkt, HFI_VIDEO_ARCH_OX);
+
+	ret = venus_iface_cmdq_write(hdev, &pkt);
+	if (ret)
+		return ret;
+
+	call_hfi_pkt_op(hdev, sys_image_version, &version_pkt);
+
+	ret = venus_iface_cmdq_write(hdev, &version_pkt);
+	if (ret)
+		dev_warn(dev, "failed to send image version pkt to fw\n");
+
+	venus_set_state(hdev, VENUS_STATE_INIT);
+
+	return 0;
+}
+
+static int venus_hfi_core_deinit(struct hfi_core *hfi)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+
+	venus_set_state(hdev, VENUS_STATE_DEINIT);
+	hdev->suspended = true;
+
+	return 0;
+}
+
+static int venus_hfi_core_ping(struct hfi_core *hfi, u32 cookie)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	struct hfi_sys_ping_pkt pkt;
+
+	call_hfi_pkt_op(hdev, sys_ping, &pkt, cookie);
+
+	return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_core_trigger_ssr(struct hfi_core *hfi,
+				      u32 trigger_type)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	struct hfi_sys_test_ssr_pkt pkt;
+	int ret;
+
+	ret = call_hfi_pkt_op(hdev, ssr_cmd, trigger_type, &pkt);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+int venus_hfi_session_init(struct hfi_core *hfi, struct hfi_inst *inst,
+			   u32 session_type, u32 codec)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	struct hfi_session_init_pkt pkt;
+	int ret;
+
+	inst->priv = hdev;
+
+	ret = venus_sys_set_default_properties(hdev);
+	if (ret)
+		return ret;
+
+	ret = call_hfi_pkt_op(hdev, session_init, &pkt, inst, session_type,
+			      codec);
+	if (ret)
+		goto err;
+
+	ret = venus_iface_cmdq_write(hdev, &pkt);
+	if (ret)
+		goto err;
+
+	return 0;
+
+err:
+	venus_flush_debug_queue(hdev);
+	return ret;
+}
+
+static int venus_hfi_session_end(struct hfi_inst *inst)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct device *dev = hdev->dev;
+
+	if (venus_fw_coverage) {
+		if (venus_sys_set_coverage(hdev, venus_fw_coverage))
+			dev_warn(dev, "fw coverage msg ON failed\n");
+	}
+
+	return venus_session_cmd(inst, HFI_CMD_SYS_SESSION_END);
+}
+
+static int venus_hfi_session_abort(struct hfi_inst *inst)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+
+	venus_flush_debug_queue(hdev);
+
+	return venus_session_cmd(inst, HFI_CMD_SYS_SESSION_ABORT);
+}
+
+static int venus_hfi_session_flush(struct hfi_inst *inst, u32 flush_mode)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_flush_pkt pkt;
+	int ret;
+
+	ret = call_hfi_pkt_op(hdev, session_flush, &pkt, inst, flush_mode);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_session_start(struct hfi_inst *inst)
+{
+	return venus_session_cmd(inst, HFI_CMD_SESSION_START);
+}
+
+static int venus_hfi_session_stop(struct hfi_inst *inst)
+{
+	return venus_session_cmd(inst, HFI_CMD_SESSION_STOP);
+}
+
+static int venus_hfi_session_etb(struct hfi_inst *inst,
+				 struct hfi_frame_data *in_frame)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	u32 session_type = inst->session_type;
+	int ret;
+
+	if (session_type == VIDC_SESSION_TYPE_DEC) {
+		struct hfi_session_empty_buffer_compressed_pkt pkt;
+
+		ret = call_hfi_pkt_op(hdev, session_etb_decoder, &pkt, inst,
+				      in_frame);
+		if (ret)
+			return ret;
+
+		ret = venus_iface_cmdq_write(hdev, &pkt);
+	} else if (session_type == VIDC_SESSION_TYPE_ENC) {
+		struct hfi_session_empty_buffer_uncompressed_plane0_pkt pkt;
+
+		ret = call_hfi_pkt_op(hdev, session_etb_encoder, &pkt, inst,
+				      in_frame);
+		if (ret)
+			return ret;
+
+		ret = venus_iface_cmdq_write(hdev, &pkt);
+	} else {
+		ret = -EINVAL;
+	}
+
+	return ret;
+}
+
+static int venus_hfi_session_ftb(struct hfi_inst *inst,
+				 struct hfi_frame_data *out_frame)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_fill_buffer_pkt pkt;
+	int ret;
+
+	ret = call_hfi_pkt_op(hdev, session_ftb, &pkt, inst, out_frame);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_session_set_buffers(struct hfi_inst *inst,
+					 struct hfi_buffer_desc *bd)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_set_buffers_pkt *pkt;
+	u8 packet[IFACEQ_VAR_LARGE_PKT_SIZE];
+	int ret;
+
+	if (bd->buffer_type == HFI_BUFFER_INPUT)
+		return 0;
+
+	pkt = (struct hfi_session_set_buffers_pkt *)packet;
+
+	ret = call_hfi_pkt_op(hdev, session_set_buffers, pkt, inst, bd);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_release_buffers(struct hfi_inst *inst,
+					     struct hfi_buffer_desc *bd)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_release_buffer_pkt *pkt;
+	u8 packet[IFACEQ_VAR_LARGE_PKT_SIZE];
+	int ret;
+
+	if (bd->buffer_type == HFI_BUFFER_INPUT)
+		return 0;
+
+	pkt = (struct hfi_session_release_buffer_pkt *) packet;
+
+	ret = call_hfi_pkt_op(hdev, session_release_buffers, pkt, inst, bd);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_load_res(struct hfi_inst *inst)
+{
+	return venus_session_cmd(inst, HFI_CMD_SESSION_LOAD_RESOURCES);
+}
+
+static int venus_hfi_session_release_res(struct hfi_inst *inst)
+{
+	return venus_session_cmd(inst, HFI_CMD_SESSION_RELEASE_RESOURCES);
+}
+
+static int venus_hfi_session_parse_seq_hdr(struct hfi_inst *inst,
+					   u32 seq_hdr, u32 seq_hdr_len)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_parse_sequence_header_pkt *pkt;
+	u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+	int ret;
+
+	pkt = (struct hfi_session_parse_sequence_header_pkt *) packet;
+
+	ret = call_hfi_pkt_op(hdev, session_parse_seq_header,
+			      pkt, inst, seq_hdr, seq_hdr_len);
+	if (ret)
+		return ret;
+
+	ret = venus_iface_cmdq_write(hdev, pkt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int venus_hfi_session_get_seq_hdr(struct hfi_inst *inst,
+					 u32 seq_hdr, u32 seq_hdr_len)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_get_sequence_header_pkt *pkt;
+	u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+	int ret;
+
+	pkt = (struct hfi_session_get_sequence_header_pkt *) packet;
+
+	ret = call_hfi_pkt_op(hdev, session_get_seq_hdr, pkt, inst, seq_hdr,
+			      seq_hdr_len);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_set_property(struct hfi_inst *inst, u32 ptype,
+					  void *pdata)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_set_property_pkt *pkt;
+	u8 packet[IFACEQ_VAR_LARGE_PKT_SIZE];
+	int ret;
+
+	pkt = (struct hfi_session_set_property_pkt *) packet;
+
+	ret = call_hfi_pkt_op(hdev, session_set_property, pkt, inst, ptype,
+			      pdata);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_get_property(struct hfi_inst *inst, u32 ptype)
+{
+	struct venus_hfi_device *hdev = inst->priv;
+	struct hfi_session_get_property_pkt pkt;
+	int ret;
+
+	ret = call_hfi_pkt_op(hdev, session_get_property, &pkt, inst, ptype);
+	if (ret)
+		return ret;
+
+	return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_resume(struct hfi_core *hfi)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	int ret = 0;
+
+	mutex_lock(&hdev->lock);
+
+	if (hdev->suspended == false)
+		goto unlock;
+
+	ret = venus_power_on(hdev);
+
+unlock:
+	if (!ret)
+		hdev->suspended = false;
+
+	mutex_unlock(&hdev->lock);
+
+	return ret;
+}
+
+static int venus_hfi_suspend(struct hfi_core *hfi)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+	struct device *dev = hfi->dev;
+	u32 ctrl_status;
+	int ret;
+
+	if (!hdev->power_enabled || hdev->suspended)
+		return 0;
+
+	mutex_lock(&hdev->lock);
+	ret = venus_is_valid_state(hdev);
+	mutex_unlock(&hdev->lock);
+
+	if (!ret) {
+		dev_err(dev, "bad state, cannot suspend\n");
+		return -EINVAL;
+	}
+
+	ret = venus_prepare_power_collapse(hdev);
+	if (ret) {
+		dev_err(dev, "prepare for power collapse fail (%d)\n", ret);
+		return ret;
+	}
+
+	mutex_lock(&hdev->lock);
+
+	if (hdev->last_packet_type != HFI_CMD_SYS_PC_PREP) {
+		mutex_unlock(&hdev->lock);
+		return -EINVAL;
+	}
+
+	ret = venus_are_queues_empty(hdev);
+	if (ret < 0 || !ret) {
+		mutex_unlock(&hdev->lock);
+		return -EINVAL;
+	}
+
+	ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+	if (!(ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)) {
+		mutex_unlock(&hdev->lock);
+		return -EINVAL;
+	}
+
+	ret = venus_power_off(hdev);
+	if (ret) {
+		mutex_unlock(&hdev->lock);
+		return ret;
+	}
+
+	hdev->suspended = true;
+
+	mutex_unlock(&hdev->lock);
+
+	return 0;
+}
+
+static const struct hfi_ops venus_hfi_ops = {
+	.core_init			= venus_hfi_core_init,
+	.core_deinit			= venus_hfi_core_deinit,
+	.core_ping			= venus_hfi_core_ping,
+	.core_trigger_ssr		= venus_hfi_core_trigger_ssr,
+
+	.session_init			= venus_hfi_session_init,
+	.session_end			= venus_hfi_session_end,
+	.session_abort			= venus_hfi_session_abort,
+	.session_flush			= venus_hfi_session_flush,
+	.session_start			= venus_hfi_session_start,
+	.session_stop			= venus_hfi_session_stop,
+	.session_etb			= venus_hfi_session_etb,
+	.session_ftb			= venus_hfi_session_ftb,
+	.session_set_buffers		= venus_hfi_session_set_buffers,
+	.session_release_buffers	= venus_hfi_session_release_buffers,
+	.session_load_res		= venus_hfi_session_load_res,
+	.session_release_res		= venus_hfi_session_release_res,
+	.session_parse_seq_hdr		= venus_hfi_session_parse_seq_hdr,
+	.session_get_seq_hdr		= venus_hfi_session_get_seq_hdr,
+	.session_set_property		= venus_hfi_session_set_property,
+	.session_get_property		= venus_hfi_session_get_property,
+
+	.resume				= venus_hfi_resume,
+	.suspend			= venus_hfi_suspend,
+
+	.isr				= venus_isr,
+	.isr_thread			= venus_isr_thread,
+};
+
+void venus_hfi_destroy(struct hfi_core *hfi)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
+
+	venus_interface_queues_release(hdev);
+	mutex_destroy(&hdev->lock);
+	kfree(hdev);
+}
+
+int venus_hfi_create(struct hfi_core *hfi, const struct vidc_resources *res,
+		     void __iomem *base)
+{
+	struct venus_hfi_device *hdev;
+	int ret;
+
+	hdev = kzalloc(sizeof(*hdev), GFP_KERNEL);
+	if (!hdev)
+		return -ENOMEM;
+
+	mutex_init(&hdev->lock);
+
+	hdev->res = res;
+	hdev->pkt_ops = hfi->pkt_ops;
+	hdev->packetization_type = HFI_PACKETIZATION_LEGACY;
+	hdev->base = base;
+	hdev->dev = hfi->dev;
+	hdev->suspended = true;
+
+	hfi->priv = hdev;
+	hfi->ops = &venus_hfi_ops;
+	hfi->core_caps = VIDC_ENC_ROTATION_CAPABILITY |
+			 VIDC_ENC_SCALING_CAPABILITY |
+			 VIDC_ENC_DEINTERLACE_CAPABILITY |
+			 VIDC_DEC_MULTI_STREAM_CAPABILITY;
+
+	ret = venus_interface_queues_init(hdev);
+	if (ret)
+		goto err_kfree;
+
+	return 0;
+
+err_kfree:
+	kfree(hdev);
+	hfi->priv = NULL;
+	hfi->ops = NULL;
+	return ret;
+}
diff --git a/drivers/media/platform/qcom/vidc/hfi_venus.h b/drivers/media/platform/qcom/vidc/hfi_venus.h
new file mode 100644
index 000000000000..c2e6e4235a0b
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_venus.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VENUS_HFI_H__
+#define __VENUS_HFI_H__
+
+struct hfi_core;
+struct vidc_resources;
+
+void venus_hfi_destroy(struct hfi_core *hfi);
+int venus_hfi_create(struct hfi_core *hfi, const struct vidc_resources *res,
+		     void __iomem *base);
+
+#endif
diff --git a/drivers/media/platform/qcom/vidc/hfi_venus_io.h b/drivers/media/platform/qcom/vidc/hfi_venus_io.h
new file mode 100644
index 000000000000..7df09fb72011
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/hfi_venus_io.h
@@ -0,0 +1,98 @@
+/*
+ * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2016 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VENUS_HFI_IO_H__
+#define __VENUS_HFI_IO_H__
+
+#define VBIF_BASE				0x80000
+
+#define VBIF_AXI_HALT_CTRL0			(VBIF_BASE + 0x208)
+#define VBIF_AXI_HALT_CTRL1			(VBIF_BASE + 0x20c)
+
+#define VBIF_AXI_HALT_CTRL0_HALT_REQ		BIT(0)
+#define VBIF_AXI_HALT_CTRL1_HALT_ACK		BIT(0)
+#define VBIF_AXI_HALT_ACK_TIMEOUT_US		500000
+
+#define CPU_BASE				0xc0000
+#define CPU_CS_BASE				(CPU_BASE + 0x12000)
+#define CPU_IC_BASE				(CPU_BASE + 0x1f000)
+
+#define CPU_CS_A2HSOFTINTCLR			(CPU_CS_BASE + 0x1c)
+
+#define VIDC_CTRL_INIT				(CPU_CS_BASE + 0x48)
+#define VIDC_CTRL_INIT_RESERVED_BITS31_1_MASK	0xfffffffe
+#define VIDC_CTRL_INIT_RESERVED_BITS31_1_SHIFT	1
+#define VIDC_CTRL_INIT_CTRL_MASK		0x1
+#define VIDC_CTRL_INIT_CTRL_SHIFT		0
+
+/* HFI control status */
+#define CPU_CS_SCIACMDARG0			(CPU_CS_BASE + 0x4c)
+#define CPU_CS_SCIACMDARG0_MASK			0xff
+#define CPU_CS_SCIACMDARG0_SHIFT		0x0
+#define CPU_CS_SCIACMDARG0_ERROR_STATUS_MASK	0xfe
+#define CPU_CS_SCIACMDARG0_ERROR_STATUS_SHIFT	0x1
+#define CPU_CS_SCIACMDARG0_INIT_STATUS_MASK	0x1
+#define CPU_CS_SCIACMDARG0_INIT_STATUS_SHIFT	0x0
+#define CPU_CS_SCIACMDARG0_PC_READY		0x100
+#define CPU_CS_SCIACMDARG0_INIT_IDLE_MSG_MASK	0x40000000
+
+/* HFI queue table info */
+#define CPU_CS_SCIACMDARG1			(CPU_CS_BASE + 0x50)
+
+/* HFI queue table address */
+#define CPU_CS_SCIACMDARG2			(CPU_CS_BASE + 0x54)
+
+/* vidc cpu */
+#define CPU_CS_SCIACMDARG3			(CPU_CS_BASE + 0x58)
+
+#define SFR_ADDR				(CPU_CS_BASE + 0x5c)
+#define MMAP_ADDR				(CPU_CS_BASE + 0x60)
+#define UC_REGION_ADDR				(CPU_CS_BASE + 0x64)
+#define UC_REGION_SIZE				(CPU_CS_BASE + 0x68)
+
+#define CPU_IC_SOFTINT				(CPU_IC_BASE + 0x18)
+#define CPU_IC_SOFTINT_H2A_MASK			0x8000
+#define CPU_IC_SOFTINT_H2A_SHIFT		0xf
+
+/* vidc wrapper */
+#define WRAPPER_BASE				0x000e0000
+
+#define WRAPPER_HW_VERSION			(WRAPPER_BASE + 0x00)
+#define WRAPPER_HW_VERSION_MAJOR_VERSION_MASK	0x78000000
+#define WRAPPER_HW_VERSION_MAJOR_VERSION_SHIFT	28
+#define WRAPPER_HW_VERSION_MINOR_VERSION_MASK	0xfff0000
+#define WRAPPER_HW_VERSION_MINOR_VERSION_SHIFT	16
+#define WRAPPER_HW_VERSION_STEP_VERSION_MASK	0xffff
+
+#define WRAPPER_INTR_STATUS			(WRAPPER_BASE + 0x0c)
+#define WRAPPER_INTR_STATUS_A2HWD_MASK		0x10
+#define WRAPPER_INTR_STATUS_A2HWD_SHIFT		0x4
+#define WRAPPER_INTR_STATUS_A2H_MASK		0x4
+#define WRAPPER_INTR_STATUS_A2H_SHIFT		0x2
+
+#define WRAPPER_INTR_MASK			(WRAPPER_BASE + 0x10)
+#define WRAPPER_INTR_MASK_A2HWD_BASK		0x10
+#define WRAPPER_INTR_MASK_A2HWD_SHIFT		0x4
+#define WRAPPER_INTR_MASK_A2HVCODEC_MASK	0x8
+#define WRAPPER_INTR_MASK_A2HVCODEC_SHIFT	0x3
+#define WRAPPER_INTR_MASK_A2HCPU_MASK		0x4
+#define WRAPPER_INTR_MASK_A2HCPU_SHIFT		0x2
+
+#define WRAPPER_INTR_CLEAR			(WRAPPER_BASE + 0x14)
+#define WRAPPER_INTR_CLEAR_A2HWD_MASK		0x10
+#define WRAPPER_INTR_CLEAR_A2HWD_SHIFT		0x4
+#define WRAPPER_INTR_CLEAR_A2H_MASK		0x4
+#define WRAPPER_INTR_CLEAR_A2H_SHIFT		0x2
+
+#endif
-- 
2.7.4

[PATCH 7/8] media: vidc: add Makefiles and Kconfig files

[Stanimir Varbanov]

Makefile and Kconfig files to build the video codec driver.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/Kconfig       |  8 ++++++++
 drivers/media/platform/qcom/Makefile      |  6 ++++++
 drivers/media/platform/qcom/vidc/Makefile | 19 +++++++++++++++++++
 3 files changed, 33 insertions(+)
 create mode 100644 drivers/media/platform/qcom/Kconfig
 create mode 100644 drivers/media/platform/qcom/Makefile
 create mode 100644 drivers/media/platform/qcom/vidc/Makefile

diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
new file mode 100644
index 000000000000..4bad5c0f68e4
--- /dev/null
+++ b/drivers/media/platform/qcom/Kconfig
@@ -0,0 +1,8 @@
+comment "Qualcomm V4L2 drivers"
+
+menuconfig QCOM_VIDC
+        tristate "Qualcomm V4L2 encoder/decoder driver"
+        depends on ARCH_QCOM && VIDEO_V4L2
+        depends on IOMMU_DMA
+        depends on QCOM_VENUS_PIL
+        select VIDEOBUF2_DMA_SG
diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
new file mode 100644
index 000000000000..150892f6533b
--- /dev/null
+++ b/drivers/media/platform/qcom/Makefile
@@ -0,0 +1,6 @@
+#
+# Makefile for the QCOM spcific video device drivers
+# based on V4L2.
+#
+
+obj-$(CONFIG_QCOM_VIDC)     += vidc/
diff --git a/drivers/media/platform/qcom/vidc/Makefile b/drivers/media/platform/qcom/vidc/Makefile
new file mode 100644
index 000000000000..56f5c3924855
--- /dev/null
+++ b/drivers/media/platform/qcom/vidc/Makefile
@@ -0,0 +1,19 @@
+# Makefile for Qualcomm vidc driver
+
+vidc-objs += \
+		core.o \
+		int_bufs.o \
+		helpers.o \
+		vdec.o \
+		vdec_ctrls.o \
+		venc.o \
+		venc_ctrls.o \
+		mem.o \
+		resources.o \
+		load.o \
+		hfi_venus.o \
+		hfi_msgs.o \
+		hfi_cmds.o \
+		hfi.o \
+
+obj-$(CONFIG_QCOM_VIDC) += vidc.o
-- 
2.7.4

[PATCH 8/8] media: vidc: enable building of the video codec driver

[Stanimir Varbanov]

This adds changes in v4l2 platform directory to include the
vidc driver and show it in kernel config.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/Kconfig  | 1 +
 drivers/media/platform/Makefile | 1 +
 2 files changed, 2 insertions(+)

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index f25344bc7912..e52640417b0a 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -111,6 +111,7 @@ source "drivers/media/platform/s5p-tv/Kconfig"
 source "drivers/media/platform/am437x/Kconfig"
 source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
+source "drivers/media/platform/qcom/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 21771c1a13fb..d8fc75ddcc73 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -51,6 +51,7 @@ obj-$(CONFIG_VIDEO_RENESAS_JPU) 	+= rcar_jpu.o
 obj-$(CONFIG_VIDEO_RENESAS_VSP1)	+= vsp1/
 
 obj-y	+= omap/
+obj-y	+= qcom/
 
 obj-$(CONFIG_VIDEO_AM437X_VPFE)		+= am437x/
 
-- 
2.7.4

Re: [PATCH 2/8] media: vidc: adding core part and helper functions

[Hans Verkuil]

Hi Stanimir,

Thanks for this patch series!

I have some review comments:

On 08/22/2016 03:13 PM, Stanimir Varbanov wrote:

<snip>

> diff --git a/drivers/media/platform/qcom/vidc/core.h b/drivers/media/platform/qcom/vidc/core.h
> new file mode 100644
> index 000000000000..5dc8e05f8c36
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/core.h
> @@ -0,0 +1,196 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +
> +#ifndef __VIDC_CORE_H_
> +#define __VIDC_CORE_H_
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/videobuf2-core.h>
> +
> +#include "resources.h"
> +#include "hfi.h"
> +
> +#define VIDC_DRV_NAME		"vidc"
> +
> +struct vidc_list {
> +	struct list_head list;
> +	struct mutex lock;
> +};
> +
> +struct vidc_format {
> +	u32 pixfmt;
> +	int num_planes;
> +	u32 type;
> +};
> +
> +struct vidc_core {
> +	struct list_head list;
> +	void __iomem *base;
> +	int irq;
> +	struct clk *clks[VIDC_CLKS_NUM_MAX];
> +	struct mutex lock;
> +	struct hfi_core hfi;
> +	struct video_device vdev_dec;
> +	struct video_device vdev_enc;

I know that many drivers embed struct video_device, but this can cause subtle
refcounting problems. I recommend changing this to a pointer and using video_device_alloc().

I have plans to reorganize the way video_devices are allocated and registered in
the near future, and you might just as well prepare this driver for that by switching
to a pointer.

> +	struct v4l2_device v4l2_dev;
> +	struct list_head instances;
> +	const struct vidc_resources *res;
> +	struct rproc *rproc;
> +	bool rproc_booted;
> +	struct device *dev;
> +};
> +
> +struct vdec_controls {
> +	u32 post_loop_deb_mode;
> +	u32 profile;
> +	u32 level;
> +};
> +
> +struct venc_controls {
> +	u16 gop_size;
> +	u32 idr_period;
> +	u32 num_p_frames;
> +	u32 num_b_frames;
> +	u32 bitrate_mode;
> +	u32 bitrate;
> +	u32 bitrate_peak;
> +
> +	u32 h264_i_period;
> +	u32 h264_entropy_mode;
> +	u32 h264_i_qp;
> +	u32 h264_p_qp;
> +	u32 h264_b_qp;
> +	u32 h264_min_qp;
> +	u32 h264_max_qp;
> +	u32 h264_loop_filter_mode;
> +	u32 h264_loop_filter_alpha;
> +	u32 h264_loop_filter_beta;
> +
> +	u32 vp8_min_qp;
> +	u32 vp8_max_qp;
> +
> +	u32 multi_slice_mode;
> +	u32 multi_slice_max_bytes;
> +	u32 multi_slice_max_mb;
> +
> +	u32 header_mode;
> +
> +	u32 profile;
> +	u32 level;
> +};
> +
> +struct vidc_inst {
> +	struct list_head list;
> +	struct mutex lock;
> +	struct vidc_core *core;
> +
> +	struct vidc_list scratchbufs;
> +	struct vidc_list persistbufs;
> +	struct vidc_list registeredbufs;
> +
> +	struct list_head bufqueue;
> +	struct mutex bufqueue_lock;
> +
> +	int streamoff;
> +	int streamon;
> +	struct vb2_queue bufq_out;
> +	struct vb2_queue bufq_cap;
> +
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	union {
> +		struct vdec_controls dec;
> +		struct venc_controls enc;
> +	} controls;
> +	struct v4l2_fh fh;
> +
> +	struct hfi_inst *hfi_inst;
> +
> +	/* session fields */
> +	u32 session_type;
> +	u32 width;
> +	u32 height;
> +	u32 out_width;
> +	u32 out_height;
> +	u32 colorspace;
> +	u8 ycbcr_enc;
> +	u8 quantization;
> +	u8 xfer_func;
> +	u64 fps;
> +	struct v4l2_fract timeperframe;
> +	const struct vidc_format *fmt_out;
> +	const struct vidc_format *fmt_cap;
> +	unsigned int num_input_bufs;
> +	unsigned int num_output_bufs;
> +	bool in_reconfig;
> +	u32 reconfig_width;
> +	u32 reconfig_height;
> +	u64 sequence;
> +};
> +
> +#define ctrl_to_inst(ctrl)	\
> +	container_of(ctrl->handler, struct vidc_inst, ctrl_handler)
> +
> +struct vidc_ctrl {
> +	u32 id;
> +	enum v4l2_ctrl_type type;
> +	s32 min;
> +	s32 max;
> +	s32 def;
> +	u32 step;
> +	u64 menu_skip_mask;
> +	u32 flags;
> +	const char * const *qmenu;
> +};
> +
> +/*
> + * Offset base for buffers on the destination queue - used to distinguish
> + * between source and destination buffers when mmapping - they receive the same
> + * offsets but for different queues
> + */
> +#define DST_QUEUE_OFF_BASE	(1 << 30)
> +
> +extern const struct v4l2_file_operations vidc_fops;
> +
> +static inline void INIT_VIDC_LIST(struct vidc_list *mlist)
> +{
> +	mutex_init(&mlist->lock);
> +	INIT_LIST_HEAD(&mlist->list);
> +}
> +
> +static inline struct vidc_inst *to_inst(struct file *filp)
> +{
> +	return container_of(filp->private_data, struct vidc_inst, fh);
> +}
> +
> +static inline struct hfi_inst *to_hfi_inst(struct file *filp)
> +{
> +	return to_inst(filp)->hfi_inst;
> +}
> +
> +static inline struct vb2_queue *
> +vidc_to_vb2q(struct file *file, enum v4l2_buf_type type)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +
> +	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +		return &inst->bufq_cap;
> +	else if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +		return &inst->bufq_out;
> +
> +	return NULL;
> +}
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vidc/helpers.c b/drivers/media/platform/qcom/vidc/helpers.c
> new file mode 100644
> index 000000000000..81079f2b5ed1
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/helpers.c
> @@ -0,0 +1,394 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/pm_runtime.h>
> +#include <media/videobuf2-dma-sg.h>
> +
> +#include "helpers.h"
> +#include "int_bufs.h"
> +#include "load.h"
> +#include "hfi_helper.h"
> +
> +static int session_set_buf(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct vb2_queue *q = vb->vb2_queue;
> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> +	struct vidc_core *core = inst->core;
> +	struct device *dev = core->dev;
> +	struct hfi_core *hfi = &core->hfi;
> +	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
> +	struct hfi_frame_data fdata;
> +	int ret;
> +
> +	memset(&fdata, 0, sizeof(fdata));
> +
> +	fdata.alloc_len = vb2_plane_size(vb, 0);
> +	fdata.device_addr = buf->dma_addr;
> +	fdata.timestamp = vb->timestamp;
> +	fdata.flags = 0;
> +	fdata.clnt_data = buf->dma_addr;
> +
> +	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		fdata.buffer_type = HFI_BUFFER_INPUT;
> +		fdata.filled_len = vb2_get_plane_payload(vb, 0);
> +		fdata.offset = vb->planes[0].data_offset;
> +
> +		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
> +			fdata.flags |= HFI_BUFFERFLAG_EOS;
> +
> +		ret = vidc_hfi_session_etb(hfi, inst->hfi_inst, &fdata);
> +	} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		fdata.buffer_type = HFI_BUFFER_OUTPUT;
> +		fdata.filled_len = 0;
> +		fdata.offset = 0;
> +
> +		ret = vidc_hfi_session_ftb(hfi, inst->hfi_inst, &fdata);
> +	} else {
> +		ret = -EINVAL;
> +	}
> +
> +	if (ret) {
> +		dev_err(dev, "failed to set session buffer (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int session_unregister_bufs(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct hfi_buffer_desc *bd;
> +	struct vidc_buffer *buf, *tmp;
> +	int ret = 0;
> +
> +	mutex_lock(&inst->registeredbufs.lock);
> +	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
> +				 hfi_list) {
> +		list_del(&buf->hfi_list);
> +		bd = &buf->bd;
> +		bd->response_required = 1;
> +		ret = vidc_hfi_session_unset_buffers(hfi, inst->hfi_inst, bd);
> +		if (ret) {
> +			dev_err(dev, "%s: session release buffers failed\n",
> +				__func__);
> +			break;
> +		}
> +	}
> +	mutex_unlock(&inst->registeredbufs.lock);
> +
> +	return ret;
> +}
> +
> +static int session_register_bufs(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct hfi_buffer_desc *bd;
> +	struct vidc_buffer *buf, *tmp;
> +	int ret = 0;
> +
> +	mutex_lock(&inst->registeredbufs.lock);
> +	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
> +				 hfi_list) {
> +		bd = &buf->bd;
> +		ret = vidc_hfi_session_set_buffers(hfi, inst->hfi_inst, bd);
> +		if (ret) {
> +			dev_err(dev, "%s: session: set buffer failed\n",
> +				__func__);
> +			break;
> +		}
> +	}
> +	mutex_unlock(&inst->registeredbufs.lock);
> +
> +	return ret;
> +}
> +
> +int vidc_buf_descs(struct vidc_inst *inst, u32 type,
> +		   struct hfi_buffer_requirements *out)
> +{
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	u32 ptype = HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS;
> +	union hfi_get_property hprop;
> +	int ret, i;
> +
> +	if (out)
> +		memset(out, 0, sizeof(*out));
> +
> +	ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype, &hprop);
> +	if (ret)
> +		return ret;
> +
> +	ret = -EINVAL;
> +
> +	for (i = 0; i < HFI_BUFFER_TYPE_MAX; i++) {
> +		if (hprop.bufreq[i].type != type)
> +			continue;
> +
> +		if (out)
> +			memcpy(out, &hprop.bufreq[i], sizeof(*out));
> +		ret = 0;
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +int vidc_set_color_format(struct vidc_inst *inst, u32 type, u32 pixfmt)
> +{
> +	struct hfi_uncompressed_format_select fmt;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	u32 ptype = HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT;
> +	int ret;
> +
> +	fmt.buffer_type = type;
> +
> +	switch (pixfmt) {
> +	case V4L2_PIX_FMT_NV12:
> +		fmt.format = HFI_COLOR_FORMAT_NV12;
> +		break;
> +	case V4L2_PIX_FMT_NV21:
> +		fmt.format = HFI_COLOR_FORMAT_NV21;
> +		break;
> +	default:
> +		return -ENOTSUPP;

I'm not really sure how this error code is used, but normally -EINVAL is returned
for invalid pixel formats. -ENOTSUPP is not used by V4L2.

> +	}
> +
> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fmt);
> +	if (ret)
> +		return ret;
> +
> +	return 0;
> +}
> +
> +struct vb2_v4l2_buffer *
> +vidc_vb2_find_buf(struct vidc_inst *inst, dma_addr_t addr)
> +{
> +	struct vidc_buffer *buf;
> +	struct vb2_v4l2_buffer *vb = NULL;
> +
> +	mutex_lock(&inst->bufqueue_lock);
> +
> +	list_for_each_entry(buf, &inst->bufqueue, list) {
> +		if (buf->dma_addr == addr) {
> +			vb = &buf->vb;
> +			break;
> +		}
> +	}
> +
> +	if (vb)
> +		list_del(&buf->list);
> +
> +	mutex_unlock(&inst->bufqueue_lock);
> +
> +	return vb;
> +}
> +
> +int vidc_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct vb2_queue *q = vb->vb2_queue;
> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> +	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
> +	struct hfi_buffer_desc *bd = &buf->bd;
> +	struct sg_table *sgt;
> +
> +	memset(bd, 0, sizeof(*bd));
> +
> +	if (q->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +		return 0;
> +
> +	sgt = vb2_dma_sg_plane_desc(vb, 0);
> +	if (!sgt)
> +		return -EINVAL;
> +
> +	bd->buffer_size = vb2_plane_size(vb, 0);
> +	bd->buffer_type = HFI_BUFFER_OUTPUT;
> +	bd->num_buffers = 1;
> +	bd->device_addr = sg_dma_address(sgt->sgl);
> +
> +	mutex_lock(&inst->registeredbufs.lock);
> +	list_add_tail(&buf->hfi_list, &inst->registeredbufs.list);
> +	mutex_unlock(&inst->registeredbufs.lock);
> +
> +	return 0;
> +}
> +
> +int vidc_vb2_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
> +	struct sg_table *sgt;
> +
> +	sgt = vb2_dma_sg_plane_desc(vb, 0);
> +	if (!sgt)
> +		return -EINVAL;
> +
> +	buf->dma_addr = sg_dma_address(sgt->sgl);
> +
> +	return 0;
> +}
> +
> +void vidc_vb2_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct vidc_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> +	struct vidc_core *core = inst->core;
> +	struct device *dev = core->dev;
> +	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
> +	unsigned int state;
> +	int ret;
> +
> +	mutex_lock(&inst->hfi_inst->lock);
> +	state = inst->hfi_inst->state;
> +	mutex_unlock(&inst->hfi_inst->lock);
> +
> +	if (state == INST_INVALID || state >= INST_STOP) {
> +		vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
> +		dev_dbg(dev, "%s: type:%d, invalid instance state\n", __func__,
> +			vb->type);
> +		return;
> +	}
> +
> +	mutex_lock(&inst->bufqueue_lock);
> +	list_add_tail(&buf->list, &inst->bufqueue);
> +	mutex_unlock(&inst->bufqueue_lock);
> +
> +	if (!vb2_is_streaming(&inst->bufq_cap) ||
> +	    !vb2_is_streaming(&inst->bufq_out))
> +		return;
> +
> +	ret = session_set_buf(vb);
> +	if (ret)
> +		vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
> +}
> +
> +void vidc_vb2_stop_streaming(struct vb2_queue *q)
> +{
> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> +	struct hfi_inst *hfi_inst = inst->hfi_inst;
> +	struct vidc_core *core = inst->core;
> +	struct device *dev = core->dev;
> +	struct hfi_core *hfi = &core->hfi;
> +	int ret, streamoff;
> +
> +	mutex_lock(&inst->lock);
> +	streamoff = inst->streamoff;
> +	mutex_unlock(&inst->lock);
> +
> +	if (streamoff)
> +		return;
> +
> +	mutex_lock(&inst->lock);
> +	if (inst->streamon == 0) {
> +		mutex_unlock(&inst->lock);
> +		return;
> +	}
> +	mutex_unlock(&inst->lock);
> +
> +	ret = vidc_hfi_session_stop(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: stop failed (%d)\n", ret);
> +		goto abort;
> +	}
> +
> +	ret = vidc_hfi_session_unload_res(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: release resources failed (%d)\n", ret);
> +		goto abort;
> +	}
> +
> +	ret = session_unregister_bufs(inst);
> +	if (ret) {
> +		dev_err(dev, "failed to release capture buffers: %d\n", ret);
> +		goto abort;
> +	}
> +
> +	ret = internal_bufs_free(inst);
> +
> +	if (hfi_inst->state == INST_INVALID || hfi->state == CORE_INVALID) {
> +		ret = -EINVAL;
> +		goto abort;
> +	}
> +
> +abort:
> +	if (ret)
> +		vidc_hfi_session_abort(hfi, inst->hfi_inst);
> +
> +	vidc_scale_clocks(inst->core);
> +
> +	ret = vidc_hfi_session_deinit(hfi, inst->hfi_inst);
> +
> +	mutex_lock(&inst->lock);
> +	inst->streamoff = 1;
> +	mutex_unlock(&inst->lock);
> +
> +	if (ret)
> +		dev_err(dev, "stop streaming failed type: %d, ret: %d\n",
> +			q->type, ret);
> +
> +	ret = pm_runtime_put_sync(dev);
> +	if (ret < 0)
> +		dev_err(dev, "%s: pm_runtime_put_sync (%d)\n", __func__, ret);
> +}
> +
> +int vidc_vb2_start_streaming(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct vidc_buffer *buf, *n;
> +	int ret;
> +
> +	ret = session_register_bufs(inst);
> +	if (ret)
> +		return ret;
> +
> +	ret = internal_bufs_alloc(inst);
> +	if (ret)
> +		return ret;
> +
> +	vidc_scale_clocks(inst->core);
> +
> +	ret = vidc_hfi_session_load_res(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: load resources (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	ret = vidc_hfi_session_start(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: start failed (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	mutex_lock(&inst->bufqueue_lock);
> +	list_for_each_entry_safe(buf, n, &inst->bufqueue, list) {
> +		ret = session_set_buf(&buf->vb.vb2_buf);
> +		if (ret)
> +			break;
> +	}
> +	mutex_unlock(&inst->bufqueue_lock);
> +
> +	if (!ret) {
> +		mutex_lock(&inst->lock);
> +		inst->streamon = 1;
> +		mutex_unlock(&inst->lock);
> +	}
> +
> +	return ret;
> +}

Regards,

	Hans

Re: [PATCH 3/8] media: vidc: decoder: add video decoder files

[Hans Verkuil]

On 08/22/2016 03:13 PM, Stanimir Varbanov wrote:
> This consists of video decoder implementation plus decoder
> controls.
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/vidc/vdec.c       | 1100 +++++++++++++++++++++++++
>  drivers/media/platform/qcom/vidc/vdec.h       |   27 +
>  drivers/media/platform/qcom/vidc/vdec_ctrls.c |  200 +++++
>  drivers/media/platform/qcom/vidc/vdec_ctrls.h |   21 +
>  4 files changed, 1348 insertions(+)
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec.c
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec.h
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.c
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.h
> 
> diff --git a/drivers/media/platform/qcom/vidc/vdec.c b/drivers/media/platform/qcom/vidc/vdec.c
> new file mode 100644
> index 000000000000..a631a354742f
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/vdec.c
> @@ -0,0 +1,1100 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#include <linux/slab.h>
> +#include <linux/pm_runtime.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/videobuf2-dma-sg.h>
> +
> +#include "core.h"
> +#include "helpers.h"
> +#include "load.h"
> +#include "vdec.h"
> +#include "vdec_ctrls.h"
> +
> +#define MACROBLOCKS_PER_PIXEL	(16 * 16)
> +
> +static u32 get_framesize_nv12(int plane, u32 height, u32 width)
> +{
> +	u32 y_stride, uv_stride, y_plane;
> +	u32 y_sclines, uv_sclines, uv_plane;
> +	u32 size;
> +
> +	y_stride = ALIGN(width, 128);
> +	uv_stride = ALIGN(width, 128);
> +	y_sclines = ALIGN(height, 32);
> +	uv_sclines = ALIGN(((height + 1) >> 1), 16);
> +
> +	y_plane = y_stride * y_sclines;
> +	uv_plane = uv_stride * uv_sclines + SZ_4K;
> +	size = y_plane + uv_plane + SZ_8K;
> +
> +	return ALIGN(size, SZ_4K);
> +}
> +
> +static u32 get_framesize_compressed(u32 mbs_per_frame)
> +{
> +	return ((mbs_per_frame * MACROBLOCKS_PER_PIXEL * 3 / 2) / 2) + 128;
> +}
> +
> +static const struct vidc_format vdec_formats[] = {
> +	{
> +		.pixfmt = V4L2_PIX_FMT_NV12,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_MPEG4,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_MPEG2,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_H263,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_H264,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_VP8,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_XVID,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	},
> +};
> +
> +static const struct vidc_format *find_format(u32 pixfmt, u32 type)
> +{
> +	const struct vidc_format *fmt = vdec_formats;
> +	unsigned int size = ARRAY_SIZE(vdec_formats);
> +	unsigned int i;
> +
> +	for (i = 0; i < size; i++) {
> +		if (fmt[i].pixfmt == pixfmt)
> +			break;
> +	}
> +
> +	if (i == size || fmt[i].type != type)
> +		return NULL;
> +
> +	return &fmt[i];
> +}
> +
> +static const struct vidc_format *find_format_by_index(int index, u32 type)
> +{
> +	const struct vidc_format *fmt = vdec_formats;
> +	unsigned int size = ARRAY_SIZE(vdec_formats);
> +	int i, k = 0;
> +
> +	if (index < 0 || index > size)
> +		return NULL;
> +
> +	for (i = 0; i < size; i++) {
> +		if (fmt[i].type != type)
> +			continue;
> +		if (k == index)
> +			break;
> +		k++;
> +	}
> +
> +	if (i == size)
> +		return NULL;
> +
> +	return &fmt[i];
> +}
> +
> +static int vdec_set_properties(struct vidc_inst *inst)
> +{
> +	struct vdec_controls *ctr = &inst->controls.dec;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct hfi_enable en = { .enable = 1 };
> +	struct hfi_framerate frate;
> +	u32 ptype;
> +	int ret;
> +
> +	ptype = HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER;
> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &en);
> +	if (ret)
> +		return ret;
> +
> +	ptype = HFI_PROPERTY_CONFIG_FRAME_RATE;
> +	frate.buffer_type = HFI_BUFFER_INPUT;
> +	frate.framerate = inst->fps * (1 << 16);
> +
> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &frate);
> +	if (ret)
> +		return ret;
> +
> +	if (ctr->post_loop_deb_mode) {
> +		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
> +		en.enable = 1;
> +		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype,
> +						    &en);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static const struct vidc_format *
> +vdec_try_fmt_common(struct vidc_inst *inst, struct v4l2_format *f)
> +{
> +	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
> +	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
> +	struct hfi_inst *hfi_inst = inst->hfi_inst;
> +	const struct vidc_format *fmt;
> +	unsigned int p;
> +
> +	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
> +	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
> +
> +	fmt = find_format(pixmp->pixelformat, f->type);
> +	if (!fmt) {
> +		if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +			pixmp->pixelformat = V4L2_PIX_FMT_NV12;
> +		else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +			pixmp->pixelformat = V4L2_PIX_FMT_H264;
> +		else
> +			return NULL;
> +		fmt = find_format(pixmp->pixelformat, f->type);
> +		pixmp->width = 1280;
> +		pixmp->height = 720;
> +	}
> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +		pixmp->height = ALIGN(pixmp->height, 32);
> +
> +	pixmp->width = clamp(pixmp->width, hfi_inst->width.min,
> +			     hfi_inst->width.max);
> +	pixmp->height = clamp(pixmp->height, hfi_inst->height.min,
> +			      hfi_inst->height.max);
> +	if (pixmp->field == V4L2_FIELD_ANY)
> +		pixmp->field = V4L2_FIELD_NONE;
> +	pixmp->num_planes = fmt->num_planes;
> +	pixmp->flags = 0;
> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		for (p = 0; p < pixmp->num_planes; p++) {
> +			pfmt[p].sizeimage = get_framesize_nv12(p, pixmp->height,
> +							       pixmp->width);
> +
> +			pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
> +		}
> +	} else {
> +		u32 mbs = pixmp->width * pixmp->height / MACROBLOCKS_PER_PIXEL;
> +
> +		pfmt[0].sizeimage = get_framesize_compressed(mbs);
> +		pfmt[0].bytesperline = 0;
> +	}
> +
> +	return fmt;
> +}
> +
> +static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +	const struct vidc_format *fmt;
> +
> +	fmt = vdec_try_fmt_common(inst, f);
> +	if (!fmt)
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +
> +static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +	const struct vidc_format *fmt = NULL;
> +	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +		fmt = inst->fmt_cap;
> +	else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +		fmt = inst->fmt_out;
> +
> +	if (inst->in_reconfig) {
> +		inst->height = inst->reconfig_height;
> +		inst->width = inst->reconfig_width;
> +		inst->in_reconfig = false;
> +	}
> +
> +	pixmp->pixelformat = fmt->pixfmt;
> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		pixmp->width = inst->width;
> +		pixmp->height = inst->height;
> +		pixmp->colorspace = inst->colorspace;
> +		pixmp->ycbcr_enc = inst->ycbcr_enc;
> +		pixmp->quantization = inst->quantization;
> +		pixmp->xfer_func = inst->xfer_func;
> +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		pixmp->width = inst->out_width;
> +		pixmp->height = inst->out_height;
> +	}
> +
> +	vdec_try_fmt_common(inst, f);

Is this call really necessary? The current format should always be a valid format,
so this is dubious.

> +
> +	return 0;
> +}
> +
> +static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
> +	struct v4l2_pix_format_mplane orig_pixmp;
> +	const struct vidc_format *fmt;
> +	struct v4l2_format format;
> +	u32 pixfmt_out = 0, pixfmt_cap = 0;
> +
> +	orig_pixmp = *pixmp;
> +
> +	fmt = vdec_try_fmt_common(inst, f);
> +	if (!fmt)
> +		return -EINVAL;
> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		pixfmt_out = pixmp->pixelformat;
> +		pixfmt_cap = inst->fmt_cap->pixfmt;
> +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		pixfmt_cap = pixmp->pixelformat;
> +		pixfmt_out = inst->fmt_out->pixfmt;
> +	}
> +
> +	memset(&format, 0, sizeof(format));
> +
> +	format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> +	format.fmt.pix_mp.pixelformat = pixfmt_out;
> +	format.fmt.pix_mp.width = orig_pixmp.width;
> +	format.fmt.pix_mp.height = orig_pixmp.height;
> +	vdec_try_fmt_common(inst, &format);
> +	inst->out_width = format.fmt.pix_mp.width;
> +	inst->out_height = format.fmt.pix_mp.height;
> +	inst->colorspace = pixmp->colorspace;
> +	inst->ycbcr_enc = pixmp->ycbcr_enc;
> +	inst->quantization = pixmp->quantization;
> +	inst->xfer_func = pixmp->xfer_func;

These four fields can only be set if f->type == VIDEO_OUTPUT.

> +
> +	memset(&format, 0, sizeof(format));
> +
> +	format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> +	format.fmt.pix_mp.pixelformat = pixfmt_cap;
> +	format.fmt.pix_mp.width = orig_pixmp.width;
> +	format.fmt.pix_mp.height = orig_pixmp.height;
> +	vdec_try_fmt_common(inst, &format);
> +	inst->width = format.fmt.pix_mp.width;
> +	inst->height = format.fmt.pix_mp.height;

This doesn't look right.

If I understand this code correctly, the capture and output format
depend on one another. Can you explain a bit more what the dependencies
are? I have to revisit this later, once I have a better idea of what's
going on here.

> +
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +		inst->fmt_out = fmt;
> +	else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +		inst->fmt_cap = fmt;
> +
> +	return 0;
> +}
> +
> +static int
> +vdec_g_selection(struct file *file, void *priv, struct v4l2_selection *s)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +
> +	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +		return -EINVAL;
> +
> +	switch (s->target) {
> +	case V4L2_SEL_TGT_CROP_DEFAULT:
> +	case V4L2_SEL_TGT_CROP_BOUNDS:
> +	case V4L2_SEL_TGT_CROP:
> +	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
> +	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
> +	case V4L2_SEL_TGT_COMPOSE:

You can't set both crop and compose with just a single s->r. That makes no sense.

What exactly are you trying to do here?

> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	s->r.top = 0;
> +	s->r.left = 0;
> +	s->r.width = inst->out_width;
> +	s->r.height = inst->out_height;
> +
> +	return 0;
> +}
> +
> +static int
> +vdec_reqbufs(struct file *file, void *fh, struct v4l2_requestbuffers *b)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	if (!b->count)
> +		vb2_core_queue_release(queue);

Don't do this. Let vb2_reqbufs handle this for you.

> +
> +	return vb2_reqbufs(queue, b);
> +}
> +
> +static int
> +vdec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
> +{
> +	strlcpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
> +	strlcpy(cap->card, "video decoder", sizeof(cap->card));
> +	strlcpy(cap->bus_info, "platform:vidc", sizeof(cap->bus_info));
> +
> +	cap->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> +	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;

Drop these two lines. Instead set the device_caps field of struct video_device
to V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING. The v4l2 core will fill
in these two cap fields for you based on the vdev->device_caps field.

> +
> +	return 0;
> +}
> +
> +static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
> +{
> +	const struct vidc_format *fmt;
> +
> +	memset(f->reserved, 0, sizeof(f->reserved));
> +
> +	fmt = find_format_by_index(f->index, f->type);
> +	if (!fmt)
> +		return -EINVAL;
> +
> +	f->pixelformat = fmt->pixfmt;
> +
> +	return 0;
> +}
> +
> +static int vdec_querybuf(struct file *file, void *fh, struct v4l2_buffer *b)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
> +	unsigned int p;
> +	int ret;
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	ret = vb2_querybuf(queue, b);
> +	if (ret)
> +		return ret;
> +
> +	if (b->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	    b->memory == V4L2_MEMORY_MMAP) {
> +		for (p = 0; p < b->length; p++)
> +			b->m.planes[p].m.mem_offset += DST_QUEUE_OFF_BASE;
> +	}
> +
> +	return 0;
> +}
> +
> +static int
> +vdec_create_bufs(struct file *file, void *fh, struct v4l2_create_buffers *b)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->format.type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	return vb2_create_bufs(queue, b);
> +}
> +
> +static int vdec_prepare_buf(struct file *file, void *fh, struct v4l2_buffer *b)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	return vb2_prepare_buf(queue, b);
> +}
> +
> +static int vdec_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	return vb2_qbuf(queue, b);
> +}
> +
> +static int
> +vdec_exportbuf(struct file *file, void *fh, struct v4l2_exportbuffer *b)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	return vb2_expbuf(queue, b);
> +}
> +
> +static int vdec_dqbuf(struct file *file, void *fh, struct v4l2_buffer *b)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	return vb2_dqbuf(queue, b, file->f_flags & O_NONBLOCK);
> +}
> +
> +static int vdec_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	return vb2_streamon(queue, type);
> +}
> +
> +static int vdec_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
> +{
> +	struct vb2_queue *queue = vidc_to_vb2q(file, type);
> +
> +	if (!queue)
> +		return -EINVAL;
> +
> +	return vb2_streamoff(queue, type);
> +}

Is there a reason why the v4l2-mem2mem framework isn't used? It seems to me that
a lot of this code is in there as well.

> +
> +static int vdec_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +	struct v4l2_captureparm *cap = &a->parm.capture;
> +	struct v4l2_fract *timeperframe = &cap->timeperframe;
> +	u64 us_per_frame, fps;
> +
> +	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +		return -EINVAL;
> +
> +	memset(cap->reserved, 0, sizeof(cap->reserved));
> +	if (!timeperframe->denominator)
> +		timeperframe->denominator = inst->timeperframe.denominator;
> +	if (!timeperframe->numerator)
> +		timeperframe->numerator = inst->timeperframe.numerator;
> +	cap->readbuffers = 0;

Just set readbuffers to the minimum number of required buffers. Hmm, shouldn't
v4l2-compliance complain about a 0 value for readbuffers? Odd.

Also, for output use v4l2_outputparm. I know, they are the same, but I don't
really like it when that fact is (ab)used.

> +	cap->extendedmode = 0;
> +	cap->capability = V4L2_CAP_TIMEPERFRAME;
> +	us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
> +	do_div(us_per_frame, timeperframe->denominator);
> +
> +	if (!us_per_frame)
> +		return -EINVAL;
> +
> +	fps = (u64)USEC_PER_SEC;
> +	do_div(fps, us_per_frame);
> +
> +	inst->fps = fps;
> +	inst->timeperframe = *timeperframe;

Can both capture and output set the timeperframe? Is that intended?

> +
> +	return 0;
> +}
> +
> +static int vdec_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +
> +	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +		return -EINVAL;
> +
> +	a->parm.capture.capability |= V4L2_CAP_TIMEPERFRAME;
> +	a->parm.capture.timeperframe = inst->timeperframe;
> +
> +	return 0;
> +}
> +
> +static int vdec_enum_framesizes(struct file *file, void *fh,
> +				struct v4l2_frmsizeenum *fsize)
> +{
> +	struct hfi_inst *hfi_inst = to_hfi_inst(file);
> +	const struct vidc_format *fmt;
> +
> +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +
> +	fmt = find_format(fsize->pixel_format,
> +			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	if (!fmt) {
> +		fmt = find_format(fsize->pixel_format,
> +				  V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> +		if (!fmt)
> +			return -EINVAL;
> +	}
> +
> +	if (fsize->index)
> +		return -EINVAL;
> +
> +	fsize->stepwise.min_width = hfi_inst->width.min;
> +	fsize->stepwise.max_width = hfi_inst->width.max;
> +	fsize->stepwise.step_width = hfi_inst->width.step_size;
> +	fsize->stepwise.min_height = hfi_inst->height.min;
> +	fsize->stepwise.max_height = hfi_inst->height.max;
> +	fsize->stepwise.step_height = hfi_inst->height.step_size;
> +
> +	return 0;
> +}
> +
> +static int vdec_enum_frameintervals(struct file *file, void *fh,
> +				    struct v4l2_frmivalenum *fival)
> +{
> +	struct hfi_inst *hfi_inst = to_hfi_inst(file);
> +	const struct vidc_format *fmt;
> +
> +	fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
> +
> +	fmt = find_format(fival->pixel_format,
> +			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	if (!fmt)
> +		return -EINVAL;
> +
> +	if (fival->index)
> +		return -EINVAL;
> +
> +	if (!fival->width || !fival->height)
> +		return -EINVAL;
> +
> +	if (fival->width > hfi_inst->width.max ||
> +	    fival->width < hfi_inst->width.min ||
> +	    fival->height > hfi_inst->height.max ||
> +	    fival->height < hfi_inst->height.min)
> +		return -EINVAL;
> +
> +	fival->stepwise.min.numerator = hfi_inst->framerate.min;
> +	fival->stepwise.min.denominator = 1;
> +	fival->stepwise.max.numerator = hfi_inst->framerate.max;
> +	fival->stepwise.max.denominator = 1;
> +	fival->stepwise.step.numerator = hfi_inst->framerate.step_size;
> +	fival->stepwise.step.denominator = 1;
> +
> +	return 0;
> +}
> +
> +static int vdec_subscribe_event(struct v4l2_fh *fh,
> +				const struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_EOS:
> +		return v4l2_event_subscribe(fh, sub, 2, NULL);
> +	case V4L2_EVENT_SOURCE_CHANGE:
> +		return v4l2_src_change_event_subscribe(fh, sub);
> +	case V4L2_EVENT_CTRL:
> +		return v4l2_ctrl_subscribe_event(fh, sub);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
> +	.vidioc_querycap = vdec_querycap,
> +	.vidioc_enum_fmt_vid_cap_mplane = vdec_enum_fmt,
> +	.vidioc_enum_fmt_vid_out_mplane = vdec_enum_fmt,
> +	.vidioc_s_fmt_vid_cap_mplane = vdec_s_fmt,
> +	.vidioc_s_fmt_vid_out_mplane = vdec_s_fmt,
> +	.vidioc_g_fmt_vid_cap_mplane = vdec_g_fmt,
> +	.vidioc_g_fmt_vid_out_mplane = vdec_g_fmt,
> +	.vidioc_try_fmt_vid_cap_mplane = vdec_try_fmt,
> +	.vidioc_try_fmt_vid_out_mplane = vdec_try_fmt,
> +	.vidioc_g_selection = vdec_g_selection,
> +	.vidioc_reqbufs = vdec_reqbufs,
> +	.vidioc_querybuf = vdec_querybuf,
> +	.vidioc_create_bufs = vdec_create_bufs,
> +	.vidioc_prepare_buf = vdec_prepare_buf,
> +	.vidioc_qbuf = vdec_qbuf,
> +	.vidioc_expbuf = vdec_exportbuf,
> +	.vidioc_dqbuf = vdec_dqbuf,
> +	.vidioc_streamon = vdec_streamon,
> +	.vidioc_streamoff = vdec_streamoff,
> +	.vidioc_s_parm = vdec_s_parm,
> +	.vidioc_g_parm = vdec_g_parm,
> +	.vidioc_enum_framesizes = vdec_enum_framesizes,
> +	.vidioc_enum_frameintervals = vdec_enum_frameintervals,
> +	.vidioc_subscribe_event = vdec_subscribe_event,
> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static int vdec_init_session(struct vidc_inst *inst)
> +{
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	u32 pixfmt = inst->fmt_out->pixfmt;
> +	struct hfi_framesize fs;
> +	u32 ptype;
> +	int ret;
> +
> +	ret = vidc_hfi_session_init(hfi, inst->hfi_inst, pixfmt,
> +				    VIDC_SESSION_TYPE_DEC);
> +	if (ret)
> +		return ret;
> +
> +	ptype = HFI_PROPERTY_PARAM_FRAME_SIZE;
> +	fs.buffer_type = HFI_BUFFER_INPUT;
> +	fs.width = inst->out_width;
> +	fs.height = inst->out_height;
> +
> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
> +	if (ret)
> +		goto err;
> +
> +	fs.buffer_type = HFI_BUFFER_OUTPUT;
> +	fs.width = inst->width;
> +	fs.height = inst->height;
> +
> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
> +	if (ret)
> +		goto err;
> +
> +	pixfmt = inst->fmt_cap->pixfmt;
> +
> +	ret = vidc_set_color_format(inst, HFI_BUFFER_OUTPUT, pixfmt);
> +	if (ret)
> +		goto err;
> +
> +	return 0;
> +err:
> +	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
> +	return ret;
> +}
> +
> +static int vdec_cap_num_buffers(struct vidc_inst *inst,
> +				struct hfi_buffer_requirements *bufreq)
> +{
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct device *dev = inst->core->dev;
> +	int ret, ret2;
> +
> +	ret = pm_runtime_get_sync(dev);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = vdec_init_session(inst);
> +	if (ret)
> +		goto put_sync;
> +
> +	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, bufreq);
> +
> +	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
> +
> +put_sync:
> +	ret2 = pm_runtime_put_sync(dev);
> +
> +	return ret ? ret : ret2;
> +}
> +
> +static int vdec_queue_setup(struct vb2_queue *q,
> +			    unsigned int *num_buffers, unsigned int *num_planes,
> +			    unsigned int sizes[], struct device *alloc_devs[])
> +{
> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> +	struct hfi_buffer_requirements bufreq;
> +	unsigned int p;
> +	int ret = 0;
> +	u32 mbs;
> +
> +	switch (q->type) {
> +	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> +		*num_planes = inst->fmt_out->num_planes;
> +
> +		*num_buffers = clamp_val(*num_buffers, 4, VIDEO_MAX_FRAME);

No need to check for VIDEO_MAX_FRAME. I assume 4 is the minimum required
number of buffers. Just set min_buffers_needed in struct vb2_queue to 4 instead.

> +
> +		mbs = inst->out_width * inst->out_height /
> +				MACROBLOCKS_PER_PIXEL;
> +		for (p = 0; p < *num_planes; p++) {
> +			sizes[p] = get_framesize_compressed(mbs);
> +			alloc_devs[p] = inst->core->dev;

Don't do this here. Just set the dev field of vb2_queue to inst->core->dev.
alloc_devs[] is prefilled with q->dev by vb2.

> +		}
> +
> +		inst->num_input_bufs = *num_buffers;
> +		break;
> +	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> +		*num_planes = inst->fmt_cap->num_planes;
> +
> +		ret = vdec_cap_num_buffers(inst, &bufreq);
> +		if (ret)
> +			break;
> +
> +		*num_buffers = max(*num_buffers, bufreq.count_actual);

How is bufreq.count_actual derived? I.e. what does it depend on?

> +
> +		for (p = 0; p < *num_planes; p++) {
> +			sizes[p] = get_framesize_nv12(p, inst->height,
> +						      inst->width);
> +			alloc_devs[p] = inst->core->dev;
> +		}
> +
> +		inst->num_output_bufs = *num_buffers;
> +		break;
> +	default:
> +		ret = -EINVAL;
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +static int vdec_check_configuration(struct vidc_inst *inst)
> +{
> +	struct hfi_buffer_requirements bufreq;
> +	int ret;
> +
> +	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
> +	if (ret)
> +		return ret;
> +
> +	if (inst->num_output_bufs < bufreq.count_actual ||
> +	    inst->num_output_bufs < bufreq.count_min)
> +		return -EINVAL;
> +
> +	ret = vidc_buf_descs(inst, HFI_BUFFER_INPUT, &bufreq);
> +	if (ret)
> +		return ret;
> +
> +	if (inst->num_input_bufs < bufreq.count_min)
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +
> +static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct device *dev = inst->core->dev;
> +	struct hfi_buffer_requirements bufreq;
> +	struct hfi_buffer_count_actual buf_count;
> +	struct vb2_queue *queue;
> +	u32 ptype;
> +	int ret;
> +
> +	switch (q->type) {
> +	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> +		queue = &inst->bufq_cap;
> +		break;
> +	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> +		queue = &inst->bufq_out;
> +		break;
> +	default:

If start_streaming fails, then all pending buffers have to be returned by the driver
by calling vb2_buffer_done(VB2_BUF_STATE_QUEUED). This will give ownership back to
userspace.

> +		return -EINVAL;
> +	}
> +
> +	if (!vb2_is_streaming(queue))
> +		return 0;

Can never happen, no need to test for this.

> +
> +	inst->in_reconfig = false;
> +	inst->sequence = 0;
> +
> +	ret = pm_runtime_get_sync(dev);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = vdec_init_session(inst);
> +	if (ret)
> +		goto put_sync;
> +
> +	ret = vdec_set_properties(inst);
> +	if (ret)
> +		goto deinit_sess;
> +
> +	ret = vdec_check_configuration(inst);
> +	if (ret)
> +		goto deinit_sess;
> +
> +	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
> +	buf_count.type = HFI_BUFFER_INPUT;
> +	buf_count.count_actual = inst->num_input_bufs;
> +
> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
> +					    ptype, &buf_count);
> +	if (ret)
> +		goto deinit_sess;
> +
> +	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
> +	if (ret)
> +		goto deinit_sess;
> +
> +	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
> +	buf_count.type = HFI_BUFFER_OUTPUT;
> +	buf_count.count_actual = inst->num_output_bufs;
> +
> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
> +					    ptype, &buf_count);
> +	if (ret)
> +		goto deinit_sess;
> +
> +	if (inst->num_output_bufs != bufreq.count_actual) {
> +		struct hfi_buffer_display_hold_count_actual display;
> +
> +		ptype = HFI_PROPERTY_PARAM_BUFFER_DISPLAY_HOLD_COUNT_ACTUAL;
> +		display.type = HFI_BUFFER_OUTPUT;
> +		display.hold_count = inst->num_output_bufs -
> +				     bufreq.count_actual;
> +
> +		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
> +						    ptype, &display);
> +		if (ret)
> +			goto deinit_sess;
> +	}
> +
> +	ret = vidc_vb2_start_streaming(inst);
> +	if (ret)
> +		goto deinit_sess;
> +
> +	return 0;
> +
> +deinit_sess:
> +	vidc_hfi_session_deinit(hfi, inst->hfi_inst);

Note that vidc_vb2_start_streaming already calls vidc_hfi_session_deinit on error.
No idea if vidc_hfi_session_deinit can handle that, I just thought I'd mention it.

> +put_sync:
> +	pm_runtime_put_sync(dev);
> +	return ret;
> +}
> +
> +static const struct vb2_ops vdec_vb2_ops = {
> +	.queue_setup = vdec_queue_setup,
> +	.buf_init = vidc_vb2_buf_init,
> +	.buf_prepare = vidc_vb2_buf_prepare,
> +	.start_streaming = vdec_start_streaming,
> +	.stop_streaming = vidc_vb2_stop_streaming,


Note that stop_streaming has to return all pending buffers by calling vb2_buffer_done(VB2_BUF_STATE_ERROR).
This will give ownership back to userspace.

And I don't think vidc_vb2_stop_streaming does that right now.

> +	.buf_queue = vidc_vb2_buf_queue,
> +};
> +
> +static int vdec_empty_buf_done(struct hfi_inst *hfi_inst, u32 addr,
> +			       u32 bytesused, u32 data_offset, u32 flags)
> +{
> +	struct vidc_inst *inst = hfi_inst->ops_priv;
> +	struct vb2_v4l2_buffer *vbuf;
> +	struct vb2_buffer *vb;
> +
> +	vbuf = vidc_vb2_find_buf(inst, addr);
> +	if (!vbuf)
> +		return -EINVAL;
> +
> +	vb = &vbuf->vb2_buf;
> +	vbuf->flags = flags;
> +
> +	vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +
> +	return 0;
> +}
> +
> +static int vdec_fill_buf_done(struct hfi_inst *hfi_inst, u32 addr,
> +			      u32 bytesused, u32 data_offset, u32 flags,
> +			      struct timeval *timestamp)
> +{
> +	struct vidc_inst *inst = hfi_inst->ops_priv;
> +	struct vb2_v4l2_buffer *vbuf;
> +	struct vb2_buffer *vb;
> +
> +	vbuf = vidc_vb2_find_buf(inst, addr);
> +	if (!vbuf)
> +		return -EINVAL;
> +
> +	vb = &vbuf->vb2_buf;
> +	vb->planes[0].bytesused = bytesused;
> +	vb->planes[0].data_offset = data_offset;
> +	vb->timestamp = timeval_to_ns(timestamp);
> +	vbuf->flags = flags;
> +	vbuf->sequence = inst->sequence++;
> +
> +	vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> +
> +	if (vbuf->flags & V4L2_BUF_FLAG_LAST) {
> +		const struct v4l2_event ev = {

I think this can be static const.

> +			.type = V4L2_EVENT_EOS
> +		};
> +
> +		v4l2_event_queue_fh(&inst->fh, &ev);
> +	}
> +
> +	return 0;
> +}
> +
> +static int vdec_event_notify(struct hfi_inst *hfi_inst, u32 event,
> +			     struct hfi_event_data *data)
> +{
> +	struct vidc_inst *inst = hfi_inst->ops_priv;
> +	struct device *dev = inst->core->dev;
> +	const struct v4l2_event ev = { .type = V4L2_EVENT_SOURCE_CHANGE };

I think this can be static const.

> +
> +	switch (event) {
> +	case EVT_SESSION_ERROR:
> +		if (hfi_inst) {
> +			mutex_lock(&hfi_inst->lock);
> +			inst->hfi_inst->state = INST_INVALID;
> +			mutex_unlock(&hfi_inst->lock);
> +		}
> +		dev_err(dev, "dec: event session error (inst:%p)\n", hfi_inst);
> +		break;
> +	case EVT_SYS_EVENT_CHANGE:
> +		switch (data->event_type) {
> +		case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES:
> +			dev_dbg(dev, "event sufficient resources\n");
> +			break;
> +		case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES:
> +			inst->reconfig_height = data->height;
> +			inst->reconfig_width = data->width;
> +			inst->in_reconfig = true;
> +
> +			v4l2_event_queue_fh(&inst->fh, &ev);
> +
> +			dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
> +				data->width, data->height);
> +			break;
> +		default:
> +			break;
> +		}
> +		break;
> +	default:
> +		break;
> +	}
> +
> +	return 0;
> +}
> +
> +static const struct hfi_inst_ops vdec_hfi_ops = {
> +	.empty_buf_done = vdec_empty_buf_done,
> +	.fill_buf_done = vdec_fill_buf_done,
> +	.event_notify = vdec_event_notify,
> +};
> +
> +static void vdec_inst_init(struct vidc_inst *inst)
> +{
> +	struct hfi_inst *hfi_inst = inst->hfi_inst;
> +
> +	inst->fmt_out = &vdec_formats[6];
> +	inst->fmt_cap = &vdec_formats[0];
> +	inst->width = 1280;
> +	inst->height = ALIGN(720, 32);
> +	inst->out_width = 1280;
> +	inst->out_height = 720;
> +	inst->fps = 30;
> +	inst->timeperframe.numerator = 1;
> +	inst->timeperframe.denominator = 30;
> +
> +	hfi_inst->width.min = 64;
> +	hfi_inst->width.max = 1920;
> +	hfi_inst->width.step_size = 1;
> +	hfi_inst->height.min = 64;
> +	hfi_inst->height.max = ALIGN(1080, 32);
> +	hfi_inst->height.step_size = 1;
> +	hfi_inst->framerate.min = 1;
> +	hfi_inst->framerate.max = 30;
> +	hfi_inst->framerate.step_size = 1;
> +	hfi_inst->mbs_per_frame.min = 16;
> +	hfi_inst->mbs_per_frame.max = 8160;
> +}
> +
> +int vdec_init(struct vidc_core *core, struct video_device *dec)
> +{
> +	int ret;
> +
> +	dec->release = video_device_release_empty;
> +	dec->fops = &vidc_fops;
> +	dec->ioctl_ops = &vdec_ioctl_ops;
> +	dec->vfl_dir = VFL_DIR_M2M;
> +	dec->v4l2_dev = &core->v4l2_dev;
> +
> +	ret = video_register_device(dec, VFL_TYPE_GRABBER, -1);
> +	if (ret)
> +		return ret;
> +
> +	video_set_drvdata(dec, core);
> +
> +	return 0;
> +}
> +
> +void vdec_deinit(struct vidc_core *core, struct video_device *dec)
> +{
> +	video_unregister_device(dec);
> +}
> +
> +int vdec_open(struct vidc_inst *inst)
> +{
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct vb2_queue *q;
> +	int ret;
> +
> +	ret = vdec_ctrl_init(inst);
> +	if (ret)
> +		return ret;
> +
> +	inst->hfi_inst = vidc_hfi_session_create(hfi, &vdec_hfi_ops, inst);
> +	if (IS_ERR(inst->hfi_inst)) {
> +		ret = PTR_ERR(inst->hfi_inst);
> +		goto err_ctrl_deinit;
> +	}
> +
> +	vdec_inst_init(inst);
> +
> +	q = &inst->bufq_cap;
> +	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> +	q->io_modes = VB2_MMAP | VB2_DMABUF;
> +	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	q->ops = &vdec_vb2_ops;
> +	q->mem_ops = &vb2_dma_sg_memops;
> +	q->drv_priv = inst;
> +	q->buf_struct_size = sizeof(struct vidc_buffer);
> +	q->allow_zero_bytesused = 1;
> +	ret = vb2_queue_init(q);
> +	if (ret)
> +		goto err_session_destroy;
> +
> +	q = &inst->bufq_out;
> +	q->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> +	q->io_modes = VB2_MMAP | VB2_DMABUF;
> +	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> +	q->ops = &vdec_vb2_ops;
> +	q->mem_ops = &vb2_dma_sg_memops;
> +	q->drv_priv = inst;
> +	q->buf_struct_size = sizeof(struct vidc_buffer);
> +	q->allow_zero_bytesused = 1;
> +	ret = vb2_queue_init(q);
> +	if (ret)
> +		goto err_cap_queue_release;
> +
> +	return 0;
> +
> +err_cap_queue_release:
> +	vb2_queue_release(&inst->bufq_cap);
> +err_session_destroy:
> +	vidc_hfi_session_destroy(hfi, inst->hfi_inst);
> +	inst->hfi_inst = NULL;
> +err_ctrl_deinit:
> +	vdec_ctrl_deinit(inst);
> +	return ret;
> +}
> +
> +void vdec_close(struct vidc_inst *inst)
> +{
> +	struct hfi_core *hfi = &inst->core->hfi;
> +
> +	vb2_queue_release(&inst->bufq_out);
> +	vb2_queue_release(&inst->bufq_cap);
> +	vdec_ctrl_deinit(inst);
> +	vidc_hfi_session_destroy(hfi, inst->hfi_inst);
> +}
> diff --git a/drivers/media/platform/qcom/vidc/vdec.h b/drivers/media/platform/qcom/vidc/vdec.h
> new file mode 100644
> index 000000000000..f67de6d73f85
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/vdec.h
> @@ -0,0 +1,27 @@
> +/*
> + * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#ifndef __VIDC_VDEC_H__
> +#define __VIDC_VDEC_H__
> +
> +struct vidc_core;
> +struct video_device;
> +struct vidc_inst;
> +
> +int vdec_init(struct vidc_core *core, struct video_device *dec);
> +void vdec_deinit(struct vidc_core *core, struct video_device *dec);
> +int vdec_open(struct vidc_inst *inst);
> +void vdec_close(struct vidc_inst *inst);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vidc/vdec_ctrls.c b/drivers/media/platform/qcom/vidc/vdec_ctrls.c
> new file mode 100644
> index 000000000000..59225d8f1fd9
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/vdec_ctrls.c
> @@ -0,0 +1,200 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#include <linux/types.h>
> +#include <media/v4l2-ctrls.h>
> +
> +#include "core.h"
> +
> +static struct vidc_ctrl vdec_ctrls[] = {
> +	{
> +		.id = V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
> +		.type = V4L2_CTRL_TYPE_MENU,
> +		.min = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
> +		.max = V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_CODING_EFFICIENCY,
> +		.def = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
> +		.menu_skip_mask = ~(
> +			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE) |
> +			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE)
> +		),
> +	}, {
> +		.id = V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
> +		.type = V4L2_CTRL_TYPE_MENU,
> +		.min = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
> +		.max = V4L2_MPEG_VIDEO_MPEG4_LEVEL_5,
> +		.def = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
> +	}, {
> +		.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> +		.type = V4L2_CTRL_TYPE_MENU,
> +		.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
> +		.max = V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH,
> +		.def = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
> +		.menu_skip_mask = ~(
> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) |
> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH) |
> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH)
> +		),
> +	}, {
> +		.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL,
> +		.type = V4L2_CTRL_TYPE_MENU,
> +		.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
> +		.max = V4L2_MPEG_VIDEO_H264_LEVEL_5_1,
> +		.def = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
> +	}, {
> +		.id = V4L2_CID_MPEG_VIDEO_VPX_PROFILE,
> +		.type = V4L2_CTRL_TYPE_INTEGER,
> +		.min = 0,
> +		.max = 3,
> +		.step = 1,
> +		.def = 0,
> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
> +	}, {
> +		.id = V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER,
> +		.type = V4L2_CTRL_TYPE_BOOLEAN,
> +		.def = 0,
> +	},
> +};
> +
> +#define NUM_CTRLS	ARRAY_SIZE(vdec_ctrls)
> +
> +static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct vidc_inst *inst = ctrl_to_inst(ctrl);
> +	struct vdec_controls *ctr = &inst->controls.dec;
> +
> +	switch (ctrl->id) {
> +	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
> +		ctr->post_loop_deb_mode = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> +	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
> +	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
> +		ctr->profile = ctrl->val;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> +	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
> +		ctr->level = ctrl->val;
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct vidc_inst *inst = ctrl_to_inst(ctrl);
> +	struct vdec_controls *ctr = &inst->controls.dec;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	union hfi_get_property hprop;
> +	u32 ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
> +	int ret;
> +
> +	switch (ctrl->id) {
> +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> +	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
> +	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
> +						    &hprop);
> +		if (!ret)
> +			ctr->profile = hprop.profile_level.profile;
> +		ctrl->val = ctr->profile;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> +	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
> +						    &hprop);
> +		if (!ret)
> +			ctr->level = hprop.profile_level.level;
> +		ctrl->val = ctr->level;
> +		break;
> +	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
> +		ctrl->val = ctr->post_loop_deb_mode;
> +		break;

Why are these volatile?

> +	default:
> +		return -EINVAL;
> +	};
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ctrl_ops vdec_ctrl_ops = {
> +	.s_ctrl = vdec_op_s_ctrl,
> +	.g_volatile_ctrl = vdec_op_g_volatile_ctrl,
> +};
> +
> +int vdec_ctrl_init(struct vidc_inst *inst)
> +{
> +	unsigned int i;
> +	int ret;
> +
> +	ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, NUM_CTRLS);
> +	if (ret)
> +		return ret;
> +
> +	for (i = 0; i < NUM_CTRLS; i++) {
> +		struct v4l2_ctrl *ctrl;
> +
> +		if (vdec_ctrls[i].type == V4L2_CTRL_TYPE_MENU) {
> +			ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
> +					&vdec_ctrl_ops,
> +					vdec_ctrls[i].id,
> +					vdec_ctrls[i].max,
> +					vdec_ctrls[i].menu_skip_mask,
> +					vdec_ctrls[i].def);
> +		} else {
> +			ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler,
> +					&vdec_ctrl_ops,
> +					vdec_ctrls[i].id,
> +					vdec_ctrls[i].min,
> +					vdec_ctrls[i].max,
> +					vdec_ctrls[i].step,
> +					vdec_ctrls[i].def);
> +		}

Why have this vdec_ctrls array at all? Just call v4l2_ctrl_new_std(_menu)
directly for the controls you want to add. Most drivers do that.

> +
> +		if (!ctrl)
> +			return -EINVAL;
> +
> +		switch (vdec_ctrls[i].id) {
> +		case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
> +		case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
> +		case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
> +		case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
> +		case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
> +			ctrl->flags |= vdec_ctrls[i].flags;
> +			break;
> +		}
> +
> +		ret = inst->ctrl_handler.error;
> +		if (ret) {
> +			v4l2_ctrl_handler_free(&inst->ctrl_handler);
> +			return ret;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +void vdec_ctrl_deinit(struct vidc_inst *inst)
> +{
> +	v4l2_ctrl_handler_free(&inst->ctrl_handler);
> +}
> diff --git a/drivers/media/platform/qcom/vidc/vdec_ctrls.h b/drivers/media/platform/qcom/vidc/vdec_ctrls.h
> new file mode 100644
> index 000000000000..18f14d06deed
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/vdec_ctrls.h
> @@ -0,0 +1,21 @@
> +/*
> + * Copyright (c) 2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#ifndef __VIDC_VDEC_CTRLS_H__
> +#define __VIDC_VDEC_CTRLS_H__
> +
> +int vdec_ctrl_init(struct vidc_inst *inst);
> +void vdec_ctrl_deinit(struct vidc_inst *inst);
> +
> +#endif
> 

Regards,

	Hans

Re: [PATCH 2/8] media: vidc: adding core part and helper functions

[Stanimir Varbanov]

Hi Hans,

Thanks for the express comments!

<cut>

>> +
>> +struct vidc_core {
>> +	struct list_head list;
>> +	void __iomem *base;
>> +	int irq;
>> +	struct clk *clks[VIDC_CLKS_NUM_MAX];
>> +	struct mutex lock;
>> +	struct hfi_core hfi;
>> +	struct video_device vdev_dec;
>> +	struct video_device vdev_enc;
> 
> I know that many drivers embed struct video_device, but this can cause subtle
> refcounting problems. I recommend changing this to a pointer and using video_device_alloc().
> 
> I have plans to reorganize the way video_devices are allocated and registered in
> the near future, and you might just as well prepare this driver for that by switching
> to a pointer.

OK, thanks for the info, I will change to pointers.

<cut>

>> +
>> +int vidc_set_color_format(struct vidc_inst *inst, u32 type, u32 pixfmt)
>> +{
>> +	struct hfi_uncompressed_format_select fmt;
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	u32 ptype = HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT;
>> +	int ret;
>> +
>> +	fmt.buffer_type = type;
>> +
>> +	switch (pixfmt) {
>> +	case V4L2_PIX_FMT_NV12:
>> +		fmt.format = HFI_COLOR_FORMAT_NV12;
>> +		break;
>> +	case V4L2_PIX_FMT_NV21:
>> +		fmt.format = HFI_COLOR_FORMAT_NV21;
>> +		break;
>> +	default:
>> +		return -ENOTSUPP;
> 
> I'm not really sure how this error code is used, but normally -EINVAL is returned
> for invalid pixel formats. -ENOTSUPP is not used by V4L2.
> 

you are right, I need to change this to EINVAL.

-- 
regards,
Stan

Re: [PATCH 2/8] media: vidc: adding core part and helper functions

[Bjorn Andersson]

On Mon 22 Aug 06:13 PDT 2016, Stanimir Varbanov wrote:

Hi Stan,

> This adds core part of the vidc driver common helper functions
> used by encoder and decoder specific files.

I believe "vidc" is short for "video core" and this is not the only
"video core" from Qualcomm. This driver is the v4l2 <-> hfi interface and
uses either two ram based fifos _or_ apr tal for communication with the
implementation.

In the case of apr, the other side is not the venus core but rather the
"VIDC" apr service on the Hexagon DSP. In this case the hfi packets are
encapsulated in apr packets. Although this is not used in 8916 it would
be nice to be able to add this later...


But I think we should call this driver "hfi" - or at least venus, as
it's not compatible with e.g the "blackbird" found in 8064, which is
also called "vidc".

> 
>  - core.c has implemented the platform dirver methods, file
> operations and v4l2 registration.
> 
>  - helpers.c has implemented common helper functions for
> buffer management, vb2_ops and functions for format propagation.
> 
>  - int_bufs.c implements functions for allocating and freeing
> buffers for internal usage. The buffer parameters describing
> internal buffers depends on current format, resolution and
> codec.
> 
>  - load.c consists functions for calculation of current load
> of the hardware. Depending on the count of instances and
> resolutions it selects the best clock rate for the video
> core.
> 
>  - mem.c has two functions for memory allocation, currently
> those functions are used for internal buffers and to allocate
> the shared memory for communication with firmware via HFI
> (Host Firmware Interface) interface commands.

Please drop this; see comments on mem_alloc()

> 
>  - resources.c exports a structure describing the details
> specific to platform and SoC.
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---

This doesn't compile, as it depends on later patches. Also there are
plenty of functions that are related to later patches and would with be
better to include there, to keep the size of this patch down.

>  drivers/media/platform/qcom/vidc/core.c      | 548 +++++++++++++++++++++++++++
>  drivers/media/platform/qcom/vidc/core.h      | 196 ++++++++++
>  drivers/media/platform/qcom/vidc/helpers.c   | 394 +++++++++++++++++++
>  drivers/media/platform/qcom/vidc/helpers.h   |  43 +++
>  drivers/media/platform/qcom/vidc/int_bufs.c  | 325 ++++++++++++++++
>  drivers/media/platform/qcom/vidc/int_bufs.h  |  23 ++
>  drivers/media/platform/qcom/vidc/load.c      | 104 +++++
>  drivers/media/platform/qcom/vidc/load.h      |  22 ++
>  drivers/media/platform/qcom/vidc/mem.c       |  64 ++++
>  drivers/media/platform/qcom/vidc/mem.h       |  32 ++
>  drivers/media/platform/qcom/vidc/resources.c |  46 +++
>  drivers/media/platform/qcom/vidc/resources.h |  46 +++
>  12 files changed, 1843 insertions(+)
>  create mode 100644 drivers/media/platform/qcom/vidc/core.c
>  create mode 100644 drivers/media/platform/qcom/vidc/core.h
>  create mode 100644 drivers/media/platform/qcom/vidc/helpers.c
>  create mode 100644 drivers/media/platform/qcom/vidc/helpers.h
>  create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.c
>  create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.h
>  create mode 100644 drivers/media/platform/qcom/vidc/load.c
>  create mode 100644 drivers/media/platform/qcom/vidc/load.h
>  create mode 100644 drivers/media/platform/qcom/vidc/mem.c
>  create mode 100644 drivers/media/platform/qcom/vidc/mem.h
>  create mode 100644 drivers/media/platform/qcom/vidc/resources.c
>  create mode 100644 drivers/media/platform/qcom/vidc/resources.h
> 
> diff --git a/drivers/media/platform/qcom/vidc/core.c b/drivers/media/platform/qcom/vidc/core.c
> new file mode 100644
> index 000000000000..e005be178fc0
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/core.c
> @@ -0,0 +1,548 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#include <linux/clk.h>
> +#include <linux/init.h>
> +#include <linux/ioctl.h>
> +#include <linux/list.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +#include <linux/remoteproc.h>
> +#include <linux/pm_runtime.h>
> +#include <media/videobuf2-v4l2.h>
> +#include <media/v4l2-ioctl.h>
> +
> +#include "core.h"
> +#include "resources.h"
> +#include "vdec.h"
> +#include "venc.h"
> +
> +static void vidc_add_inst(struct vidc_core *core, struct vidc_inst *inst)
> +{
> +	mutex_lock(&core->lock);
> +	list_add_tail(&inst->list, &core->instances);

There are two different "instances" lists in this implementation, one
keeping track of vidc instances and one keeping track of hfi instances,
at the same time the vidc instances has a reference to its associated
hfi instance.

It should be possible to drop one of those lists.

> +	mutex_unlock(&core->lock);
> +}
> +
> +static void vidc_del_inst(struct vidc_core *core, struct vidc_inst *inst)
> +{
> +	struct vidc_inst *pos, *n;
> +
> +	mutex_lock(&core->lock);
> +	list_for_each_entry_safe(pos, n, &core->instances, list) {
> +		if (pos == inst)
> +			list_del(&inst->list);
> +	}
> +	mutex_unlock(&core->lock);
> +}
> +
> +static int vidc_rproc_boot(struct vidc_core *core)
> +{
> +	int ret;
> +
> +	if (core->rproc_booted)
> +		return 0;

rproc_boot()/rproc_shutdown() is reference counted, so there is no
reason (other than this driver being buggy) to keep track of
"rproc_boot". As such, you can drop vidc_rproc_boot() and
vidc_rproc_shutdown() and just call the rproc functions directly.

> +
> +	ret = rproc_boot(core->rproc);
> +	if (ret)
> +		return ret;
> +
> +	core->rproc_booted = true;
> +
> +	return 0;
> +}
> +
> +static void vidc_rproc_shutdown(struct vidc_core *core)
> +{
> +	if (!core->rproc_booted)
> +		return;
> +
> +	rproc_shutdown(core->rproc);
> +	core->rproc_booted = false;
> +}
> +
> +struct vidc_sys_error {
> +	struct vidc_core *core;
> +	struct delayed_work work;
> +};

This is cool, but during the 5 second delay we should be able to call
remove on the driver and this will dereference a freed hfi instance.

Move the worker to hfi_core and you can cancel it on remove.

> +
> +static void vidc_sys_error_handler(struct work_struct *work)
> +{
> +	struct vidc_sys_error *handler =
> +		container_of(work, struct vidc_sys_error, work.work);
> +	struct vidc_core *core = handler->core;
> +	struct hfi_core *hfi = &core->hfi;
> +	struct device *dev = core->dev;
> +	int ret;
> +
> +	mutex_lock(&hfi->lock);
> +	if (hfi->state != CORE_INVALID)
> +		goto exit;
> +
> +	mutex_unlock(&hfi->lock);
> +
> +	ret = vidc_hfi_core_deinit(hfi);
> +	if (ret)
> +		dev_err(dev, "core: deinit failed (%d)\n", ret);
> +
> +	mutex_lock(&hfi->lock);
> +
> +	rproc_report_crash(core->rproc, RPROC_FATAL_ERROR);

This operation is async, as such I believe this to be fragile. To get
the expected result you should be able to simply call
rproc_shutdown()/rproc_boot() to restart the core...

However, if we at any point would like to be able to get memory dumps
from this core (likely a requirement on the Qualcomm side) we need to
call rproc_report_crash() and let it collect the resources and then
power cycle the core.


As the life cycle of the venus driver goes 1:1 with the rproc driver I
think it would be more suitable to make the v4l driver a child of the
rproc driver and have it probe/remove this driver as the rproc comes and
goes. This would allow us to call rproc_report_crash() here, we will be
removed and when the crash is handled (sometime in the future) we will
be probed again.

> +
> +	vidc_rproc_shutdown(core);
> +
> +	ret = vidc_rproc_boot(core);
> +	if (ret)
> +		goto exit;
> +
> +	hfi->state = CORE_INIT;
> +
> +exit:
> +	mutex_unlock(&hfi->lock);
> +	kfree(handler);
> +}
> +
> +static int vidc_event_notify(struct hfi_core *hfi, u32 event)
> +{
> +	struct vidc_sys_error *handler;
> +	struct hfi_inst *inst;
> +
> +	switch (event) {
> +	case EVT_SYS_WATCHDOG_TIMEOUT:
> +	case EVT_SYS_ERROR:
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	mutex_lock(&hfi->lock);
> +
> +	hfi->state = CORE_INVALID;
> +
> +	list_for_each_entry(inst, &hfi->instances, list) {
> +		mutex_lock(&inst->lock);
> +		inst->state = INST_INVALID;
> +		mutex_unlock(&inst->lock);
> +	}
> +
> +	mutex_unlock(&hfi->lock);
> +
> +	handler = kzalloc(sizeof(*handler), GFP_KERNEL);
> +	if (!handler)
> +		return -ENOMEM;
> +
> +	handler->core = container_of(hfi, struct vidc_core, hfi);
> +	INIT_DELAYED_WORK(&handler->work, vidc_sys_error_handler);
> +
> +	/*
> +	 * Sleep for 5 sec to ensure venus has completed any
> +	 * pending cache operations. Without this sleep, we see
> +	 * device reset when firmware is unloaded after a sys
> +	 * error.
> +	 */
> +	schedule_delayed_work(&handler->work, msecs_to_jiffies(5000));
> +
> +	return 0;
> +}
> +
> +static const struct hfi_core_ops vidc_core_ops = {
> +	.event_notify = vidc_event_notify,
> +};

This is an overly generic way of calling vidc_sys_error_handler().
There is no need for having the hfi_core_ops indirections for a single
op that will only exist in 1 and only 1 variant.

Just replace the two affected event_notify() calls with a direct call to
this function (and clean it up a bit).

> +
> +static int vidc_open(struct file *file)
> +{
> +	struct video_device *vdev = video_devdata(file);
> +	struct vidc_core *core = video_drvdata(file);
> +	struct vidc_inst *inst;
> +	int ret = 0;
> +
> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> +	if (!inst)
> +		return -ENOMEM;
> +
> +	mutex_init(&inst->lock);
> +
> +	INIT_VIDC_LIST(&inst->scratchbufs);

Please inline the mutex_init() and INIT_LIST_HEAD() here and drop the
custom INIT_VIDC_LIST() wrapper macro.

> +	INIT_VIDC_LIST(&inst->persistbufs);
> +	INIT_VIDC_LIST(&inst->registeredbufs);
> +
> +	INIT_LIST_HEAD(&inst->bufqueue);
> +	mutex_init(&inst->bufqueue_lock);
> +
> +	if (vdev == &core->vdev_dec)
> +		inst->session_type = VIDC_SESSION_TYPE_DEC;
> +	else
> +		inst->session_type = VIDC_SESSION_TYPE_ENC;
> +
> +	inst->core = core;
> +
> +	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
> +		ret = vdec_open(inst);
> +	else
> +		ret = venc_open(inst);
> +
> +	if (ret)
> +		goto err_free_inst;
> +
> +	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
> +		v4l2_fh_init(&inst->fh, &core->vdev_dec);
> +	else
> +		v4l2_fh_init(&inst->fh, &core->vdev_enc);

Here we have three sequential conditionals testing for the same thing,
please join them into one.

> +
> +	inst->fh.ctrl_handler = &inst->ctrl_handler;
> +
> +	v4l2_fh_add(&inst->fh);
> +
> +	file->private_data = &inst->fh;
> +
> +	vidc_add_inst(core, inst);
> +
> +	return 0;
> +
> +err_free_inst:
> +	kfree(inst);
> +	return ret;
> +}
> +
> +static int vidc_close(struct file *file)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +	struct vidc_core *core = inst->core;
> +
> +	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
> +		vdec_close(inst);
> +	else
> +		venc_close(inst);
> +
> +	vidc_del_inst(core, inst);
> +
> +	mutex_destroy(&inst->bufqueue_lock);
> +	mutex_destroy(&inst->scratchbufs.lock);
> +	mutex_destroy(&inst->persistbufs.lock);
> +	mutex_destroy(&inst->registeredbufs.lock);

Here's a good reason for dropping the INIT_VIDC_LIST() macro

> +
> +	v4l2_fh_del(&inst->fh);
> +	v4l2_fh_exit(&inst->fh);
> +
> +	kfree(inst);
> +	return 0;
> +}
> +
> +static unsigned int vidc_poll(struct file *file, struct poll_table_struct *pt)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +	struct vb2_queue *outq = &inst->bufq_out;
> +	struct vb2_queue *capq = &inst->bufq_cap;
> +	unsigned int ret;
> +
> +	ret = vb2_poll(outq, file, pt);
> +	ret |= vb2_poll(capq, file, pt);
> +
> +	return ret;
> +}
> +
> +static int vidc_mmap(struct file *file, struct vm_area_struct *vma)
> +{
> +	struct vidc_inst *inst = to_inst(file);
> +	unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
> +	int ret;
> +
> +	if (offset < DST_QUEUE_OFF_BASE) {
> +		ret = vb2_mmap(&inst->bufq_out, vma);
> +	} else {
> +		vma->vm_pgoff -= DST_QUEUE_OFF_BASE >> PAGE_SHIFT;
> +		ret = vb2_mmap(&inst->bufq_cap, vma);
> +	}

This feels hackish, is this really the way to do this?

> +
> +	return ret;
> +}
> +
> +const struct v4l2_file_operations vidc_fops = {
> +	.owner = THIS_MODULE,
> +	.open = vidc_open,
> +	.release = vidc_close,
> +	.unlocked_ioctl = video_ioctl2,
> +	.poll = vidc_poll,
> +	.mmap = vidc_mmap,
> +#ifdef CONFIG_COMPAT
> +	.compat_ioctl32 = v4l2_compat_ioctl32,
> +#endif
> +};
> +
> +static irqreturn_t vidc_isr_thread(int irq, void *dev_id)
> +{
> +	return vidc_hfi_isr_thread(irq, dev_id);
> +}
> +
> +static irqreturn_t vidc_isr(int irq, void *dev)
> +{
> +	return vidc_hfi_isr(irq, dev);
> +}

These two functions indicates that we're requesting the irq in the wrong
layer.

Also, these two functions arrives in a later patchset, so I assume this
doesn't compile...

> +
> +static int vidc_clks_get(struct vidc_core *core, unsigned int clks_num,
> +			 const char * const *clks_id)
> +{
> +	struct device *dev = core->dev;
> +	unsigned int i;
> +
> +	for (i = 0; i < clks_num; i++) {
> +		core->clks[i] = devm_clk_get(dev, clks_id[i]);
> +		if (IS_ERR(core->clks[i]))
> +			return PTR_ERR(core->clks[i]);
> +	}
> +
> +	return 0;
> +}
> +
> +static int
> +vidc_clks_enable(struct vidc_core *core, const struct vidc_resources *res)
> +{
> +	unsigned int i;
> +	int ret;
> +
> +	for (i = 0; i < res->clks_num; i++) {
> +		ret = clk_prepare_enable(core->clks[i]);
> +		if (ret)
> +			goto err;
> +	}
> +
> +	return 0;
> +err:
> +	while (--i)
> +		clk_disable_unprepare(core->clks[i]);
> +
> +	return ret;
> +}
> +
> +static void
> +vidc_clks_disable(struct vidc_core *core, const struct vidc_resources *res)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < res->clks_num; i++)
> +		clk_disable_unprepare(core->clks[i]);
> +}
> +
> +static const struct of_device_id vidc_dt_match[] = {
> +	{ .compatible = "qcom,vidc-msm8916", .data = &msm8916_res, },
> +	{ }
> +};
> +
> +MODULE_DEVICE_TABLE(of, vidc_dt_match);

As you're using of_device_get_match_data() you can move this table to
the bottom of the file.

> +
> +static int vidc_probe(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct vidc_core *core;
> +	struct device_node *rproc;
> +	struct resource *r;
> +	int ret;
> +
> +	core = devm_kzalloc(dev, sizeof(*core), GFP_KERNEL);
> +	if (!core)
> +		return -ENOMEM;
> +
> +	core->dev = dev;
> +	platform_set_drvdata(pdev, core);
> +
> +	rproc = of_parse_phandle(dev->of_node, "rproc", 0);
> +	if (IS_ERR(rproc))
> +		return PTR_ERR(rproc);
> +
> +	core->rproc = rproc_get_by_phandle(rproc->phandle);

FYI, We're hoping to land some patches shortly that will replace this
with rproc_get(pdev->dev.of_node), looking up an rproc by the standard
"rprocs" property...

> +	if (IS_ERR(core->rproc))
> +		return PTR_ERR(core->rproc);
> +	else if (!core->rproc)
> +		return -EPROBE_DEFER;

We're cleaning up this in the core as well.

You need to rproc_put() the rproc pointer after this point.


My question still stands though, if this driver should be probed as the
remoteproc is booted (or the apr service appearing). I will continue to
look at that.

> +
> +	r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> +	core->base = devm_ioremap_resource(dev, r);
> +	if (IS_ERR(core->base))
> +		return PTR_ERR(core->base);
> +
> +	core->irq = platform_get_irq(pdev, 0);
> +	if (core->irq < 0)
> +		return core->irq;
> +
> +	core->res = of_device_get_match_data(dev);
> +	if (!core->res)
> +		return -ENODEV;
> +
> +	ret = vidc_clks_get(core, core->res->clks_num, core->res->clks);
> +	if (ret)
> +		return ret;
> +
> +	ret = dma_set_mask_and_coherent(dev, core->res->dma_mask);
> +	if (ret)
> +		return ret;
> +
> +	INIT_LIST_HEAD(&core->instances);
> +	mutex_init(&core->lock);
> +
> +	ret = devm_request_threaded_irq(dev, core->irq, vidc_isr,
> +					vidc_isr_thread,
> +					IRQF_TRIGGER_HIGH | IRQF_ONESHOT,

Drop this IRQF_TRIGGER_HIGH and have this be specified in devicetree.

> +					"vidc", &core->hfi);
> +	if (ret)
> +		return ret;
> +
> +	core->hfi.core_ops = &vidc_core_ops;
> +	core->hfi.dev = dev;
> +
> +	ret = vidc_hfi_create(&core->hfi, core->res, core->base);
> +	if (ret)
> +		return ret;
> +
> +	ret = vidc_clks_enable(core, core->res);
> +	if (ret)
> +		goto err_hfi_destroy;
> +
> +	ret = vidc_rproc_boot(core);
> +	if (ret) {
> +		vidc_clks_disable(core, core->res);
> +		goto err_hfi_destroy;
> +	}
> +
> +	pm_runtime_enable(dev);
> +
> +	ret = pm_runtime_get_sync(dev);
> +	if (ret < 0)
> +		goto err_runtime_disable;
> +
> +	ret = vidc_hfi_core_init(&core->hfi);
> +	if (ret)
> +		goto err_rproc_shutdown;
> +
> +	ret = pm_runtime_put_sync(dev);
> +	if (ret)
> +		goto err_core_deinit;
> +
> +	vidc_clks_disable(core, core->res);

These operations follow the general pattern of booting other Qualcomm
remoteprocs; acquire and enable some resources, boot the core and
disable the resources. Therefor it looks quite likely that these
operations are related to the life cycle of the venus core, rather than
hfi.

> +
> +	ret = v4l2_device_register(dev, &core->v4l2_dev);
> +	if (ret)
> +		goto err_core_deinit;
> +
> +	ret = vdec_init(core, &core->vdev_dec);
> +	if (ret)
> +		goto err_dev_unregister;
> +
> +	ret = venc_init(core, &core->vdev_enc);
> +	if (ret)
> +		goto err_vdec_deinit;
> +
> +	return 0;
> +
> +err_vdec_deinit:
> +	vdec_deinit(core, &core->vdev_dec);
> +err_dev_unregister:
> +	v4l2_device_unregister(&core->v4l2_dev);
> +err_core_deinit:
> +	vidc_hfi_core_deinit(&core->hfi);
> +err_rproc_shutdown:
> +	vidc_rproc_shutdown(core);
> +err_runtime_disable:
> +	pm_runtime_set_suspended(dev);
> +	pm_runtime_disable(dev);
> +err_hfi_destroy:
> +	vidc_hfi_destroy(&core->hfi);
> +	return ret;
> +}
> +
> +static int vidc_remove(struct platform_device *pdev)
> +{
> +	struct vidc_core *core = platform_get_drvdata(pdev);
> +	int ret;
> +
> +	ret = pm_runtime_get_sync(&pdev->dev);
> +	if (ret < 0)
> +		return ret;

No-one cares about you returning an error here, so you better move
forward and release as much of your resources as possible even though
you didn't get your pm.

> +
> +	ret = vidc_hfi_core_deinit(&core->hfi);
> +	if (ret) {
> +		pm_runtime_put_sync(&pdev->dev);
> +		return ret;
> +	}
> +
> +	vidc_rproc_shutdown(core);
> +
> +	ret = pm_runtime_put_sync(&pdev->dev);
> +
> +	vidc_hfi_destroy(&core->hfi);
> +	vdec_deinit(core, &core->vdev_dec);
> +	venc_deinit(core, &core->vdev_enc);
> +	v4l2_device_unregister(&core->v4l2_dev);
> +
> +	pm_runtime_disable(core->dev);
> +
> +	return ret < 0 ? ret : 0;
> +}
> +
> +static int vidc_runtime_suspend(struct device *dev)
> +{
> +	struct vidc_core *core = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = vidc_hfi_core_suspend(&core->hfi);
> +
> +	vidc_clks_disable(core, core->res);
> +
> +	return ret;
> +}
> +
> +static int vidc_runtime_resume(struct device *dev)
> +{
> +	struct vidc_core *core = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = vidc_clks_enable(core, core->res);
> +	if (ret)
> +		return ret;
> +
> +	return vidc_hfi_core_resume(&core->hfi);
> +}
> +
> +static int vidc_pm_suspend(struct device *dev)
> +{
> +	return vidc_runtime_suspend(dev);
> +}
> +
> +static int vidc_pm_resume(struct device *dev)
> +{
> +	return vidc_runtime_resume(dev);
> +}
> +
> +static const struct dev_pm_ops vidc_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(vidc_pm_suspend, vidc_pm_resume)
> +	SET_RUNTIME_PM_OPS(vidc_runtime_suspend, vidc_runtime_resume, NULL)
> +};
> +
> +static struct platform_driver qcom_vidc_driver = {
> +	.probe = vidc_probe,
> +	.remove = vidc_remove,
> +	.driver = {
> +		.name = "qcom-vidc",
> +		.of_match_table = vidc_dt_match,
> +		.pm = &vidc_pm_ops,
> +	},
> +};
> +
> +module_platform_driver(qcom_vidc_driver);
> +
> +MODULE_ALIAS("platform:qcom-vidc");
> +MODULE_DESCRIPTION("Qualcomm video encoder and decoder driver");
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/media/platform/qcom/vidc/core.h b/drivers/media/platform/qcom/vidc/core.h
> new file mode 100644
> index 000000000000..5dc8e05f8c36
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/core.h
> @@ -0,0 +1,196 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +
> +#ifndef __VIDC_CORE_H_
> +#define __VIDC_CORE_H_
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/videobuf2-core.h>
> +
> +#include "resources.h"
> +#include "hfi.h"
> +
> +#define VIDC_DRV_NAME		"vidc"

Unused

> +
> +struct vidc_list {
> +	struct list_head list;
> +	struct mutex lock;
> +};

Can't we get away without passing around lockable lists? Does these
lists have to be locked independently and should we really pass around
their lock with them?

> +
> +struct vidc_format {
> +	u32 pixfmt;
> +	int num_planes;
> +	u32 type;
> +};
> +
> +struct vidc_core {
> +	struct list_head list;

This list_head seems unused.

> +	void __iomem *base;

base is acquired and passed by value to vidc_hfi_create(), so no need to
keep track of it here.

> +	int irq;

This irq belongs to hfi, so it should probably be kept there.

> +	struct clk *clks[VIDC_CLKS_NUM_MAX];
> +	struct mutex lock;

This "lock" seems to be only related the instances list, please name it
more appropriately - and place it next to the instances member.

> +	struct hfi_core hfi;
> +	struct video_device vdev_dec;
> +	struct video_device vdev_enc;
> +	struct v4l2_device v4l2_dev;
> +	struct list_head instances;
> +	const struct vidc_resources *res;
> +	struct rproc *rproc;
> +	bool rproc_booted;
> +	struct device *dev;
> +};
> +
> +struct vdec_controls {
> +	u32 post_loop_deb_mode;
> +	u32 profile;
> +	u32 level;
> +};
> +
> +struct venc_controls {
> +	u16 gop_size;
> +	u32 idr_period;
> +	u32 num_p_frames;
> +	u32 num_b_frames;
> +	u32 bitrate_mode;
> +	u32 bitrate;
> +	u32 bitrate_peak;
> +
> +	u32 h264_i_period;
> +	u32 h264_entropy_mode;
> +	u32 h264_i_qp;
> +	u32 h264_p_qp;
> +	u32 h264_b_qp;
> +	u32 h264_min_qp;
> +	u32 h264_max_qp;
> +	u32 h264_loop_filter_mode;
> +	u32 h264_loop_filter_alpha;
> +	u32 h264_loop_filter_beta;
> +
> +	u32 vp8_min_qp;
> +	u32 vp8_max_qp;
> +
> +	u32 multi_slice_mode;
> +	u32 multi_slice_max_bytes;
> +	u32 multi_slice_max_mb;
> +
> +	u32 header_mode;
> +
> +	u32 profile;
> +	u32 level;
> +};
> +
> +struct vidc_inst {
> +	struct list_head list;
> +	struct mutex lock;
> +	struct vidc_core *core;
> +
> +	struct vidc_list scratchbufs;
> +	struct vidc_list persistbufs;
> +	struct vidc_list registeredbufs;

Just inline the list_head and mutex here, as it's done for bufqueue.
> +
> +	struct list_head bufqueue;
> +	struct mutex bufqueue_lock;
> +
> +	int streamoff;
> +	int streamon;
> +	struct vb2_queue bufq_out;
> +	struct vb2_queue bufq_cap;
> +
> +	struct v4l2_ctrl_handler ctrl_handler;
> +	union {
> +		struct vdec_controls dec;
> +		struct venc_controls enc;
> +	} controls;
> +	struct v4l2_fh fh;
> +
> +	struct hfi_inst *hfi_inst;
> +
> +	/* session fields */
> +	u32 session_type;
> +	u32 width;
> +	u32 height;
> +	u32 out_width;
> +	u32 out_height;
> +	u32 colorspace;
> +	u8 ycbcr_enc;
> +	u8 quantization;
> +	u8 xfer_func;
> +	u64 fps;
> +	struct v4l2_fract timeperframe;
> +	const struct vidc_format *fmt_out;
> +	const struct vidc_format *fmt_cap;
> +	unsigned int num_input_bufs;
> +	unsigned int num_output_bufs;
> +	bool in_reconfig;
> +	u32 reconfig_width;
> +	u32 reconfig_height;
> +	u64 sequence;
> +};
> +
> +#define ctrl_to_inst(ctrl)	\
> +	container_of(ctrl->handler, struct vidc_inst, ctrl_handler)
> +
> +struct vidc_ctrl {
> +	u32 id;
> +	enum v4l2_ctrl_type type;
> +	s32 min;
> +	s32 max;
> +	s32 def;
> +	u32 step;
> +	u64 menu_skip_mask;
> +	u32 flags;
> +	const char * const *qmenu;
> +};
> +
> +/*
> + * Offset base for buffers on the destination queue - used to distinguish
> + * between source and destination buffers when mmapping - they receive the same
> + * offsets but for different queues
> + */
> +#define DST_QUEUE_OFF_BASE	(1 << 30)
> +
> +extern const struct v4l2_file_operations vidc_fops;

Just pass this to v{dec,enc}_init() rather than back-referencing it
through a global variable. But on the other hand this is unused in this
patchset...

> +
> +static inline void INIT_VIDC_LIST(struct vidc_list *mlist)
> +{
> +	mutex_init(&mlist->lock);
> +	INIT_LIST_HEAD(&mlist->list);
> +}
> +
> +static inline struct vidc_inst *to_inst(struct file *filp)
> +{
> +	return container_of(filp->private_data, struct vidc_inst, fh);
> +}
> +
> +static inline struct hfi_inst *to_hfi_inst(struct file *filp)

Unused

> +{
> +	return to_inst(filp)->hfi_inst;
> +}
> +
> +static inline struct vb2_queue *
> +vidc_to_vb2q(struct file *file, enum v4l2_buf_type type)

Unused

> +{
> +	struct vidc_inst *inst = to_inst(file);
> +
> +	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +		return &inst->bufq_cap;
> +	else if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +		return &inst->bufq_out;
> +
> +	return NULL;
> +}
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vidc/helpers.c b/drivers/media/platform/qcom/vidc/helpers.c
> new file mode 100644
> index 000000000000..81079f2b5ed1
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/helpers.c
> @@ -0,0 +1,394 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/pm_runtime.h>
> +#include <media/videobuf2-dma-sg.h>
> +
> +#include "helpers.h"
> +#include "int_bufs.h"
> +#include "load.h"
> +#include "hfi_helper.h"
> +
> +static int session_set_buf(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct vb2_queue *q = vb->vb2_queue;
> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> +	struct vidc_core *core = inst->core;
> +	struct device *dev = core->dev;
> +	struct hfi_core *hfi = &core->hfi;
> +	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
> +	struct hfi_frame_data fdata;
> +	int ret;
> +
> +	memset(&fdata, 0, sizeof(fdata));
> +
> +	fdata.alloc_len = vb2_plane_size(vb, 0);
> +	fdata.device_addr = buf->dma_addr;
> +	fdata.timestamp = vb->timestamp;
> +	fdata.flags = 0;
> +	fdata.clnt_data = buf->dma_addr;
> +
> +	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		fdata.buffer_type = HFI_BUFFER_INPUT;
> +		fdata.filled_len = vb2_get_plane_payload(vb, 0);
> +		fdata.offset = vb->planes[0].data_offset;
> +
> +		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
> +			fdata.flags |= HFI_BUFFERFLAG_EOS;
> +
> +		ret = vidc_hfi_session_etb(hfi, inst->hfi_inst, &fdata);
> +	} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		fdata.buffer_type = HFI_BUFFER_OUTPUT;
> +		fdata.filled_len = 0;
> +		fdata.offset = 0;
> +
> +		ret = vidc_hfi_session_ftb(hfi, inst->hfi_inst, &fdata);
> +	} else {
> +		ret = -EINVAL;
> +	}
> +
> +	if (ret) {
> +		dev_err(dev, "failed to set session buffer (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int session_unregister_bufs(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct hfi_buffer_desc *bd;
> +	struct vidc_buffer *buf, *tmp;
> +	int ret = 0;
> +
> +	mutex_lock(&inst->registeredbufs.lock);
> +	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
> +				 hfi_list) {
> +		list_del(&buf->hfi_list);

So the hfi_list is the list_head for entries in the _vidc_ instance
list?

> +		bd = &buf->bd;
> +		bd->response_required = 1;
> +		ret = vidc_hfi_session_unset_buffers(hfi, inst->hfi_inst, bd);
> +		if (ret) {
> +			dev_err(dev, "%s: session release buffers failed\n",
> +				__func__);
> +			break;
> +		}
> +	}
> +	mutex_unlock(&inst->registeredbufs.lock);
> +
> +	return ret;
> +}
> +
> +static int session_register_bufs(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct hfi_buffer_desc *bd;
> +	struct vidc_buffer *buf, *tmp;
> +	int ret = 0;
> +
> +	mutex_lock(&inst->registeredbufs.lock);
> +	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
> +				 hfi_list) {
> +		bd = &buf->bd;
> +		ret = vidc_hfi_session_set_buffers(hfi, inst->hfi_inst, bd);
> +		if (ret) {
> +			dev_err(dev, "%s: session: set buffer failed\n",
> +				__func__);
> +			break;
> +		}
> +	}
> +	mutex_unlock(&inst->registeredbufs.lock);
> +
> +	return ret;
> +}
> +
> +int vidc_buf_descs(struct vidc_inst *inst, u32 type,
> +		   struct hfi_buffer_requirements *out)

If you call this vidc_get_buf_requirements() it would actually describe
what's going on. But why is this hfi wrapper in the core, rather than
just have the internal buffer manager call it directly.

The call doesn't seem to depend on the parameters or state, can we
cache the result?

> +{
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	u32 ptype = HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS;
> +	union hfi_get_property hprop;
> +	int ret, i;
> +
> +	if (out)
> +		memset(out, 0, sizeof(*out));
> +
> +	ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype, &hprop);
> +	if (ret)
> +		return ret;
> +
> +	ret = -EINVAL;
> +
> +	for (i = 0; i < HFI_BUFFER_TYPE_MAX; i++) {
> +		if (hprop.bufreq[i].type != type)
> +			continue;
> +
> +		if (out)
> +			memcpy(out, &hprop.bufreq[i], sizeof(*out));
> +		ret = 0;
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
[..]
> +
> +void vidc_vb2_stop_streaming(struct vb2_queue *q)
> +{
> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> +	struct hfi_inst *hfi_inst = inst->hfi_inst;
> +	struct vidc_core *core = inst->core;
> +	struct device *dev = core->dev;
> +	struct hfi_core *hfi = &core->hfi;
> +	int ret, streamoff;
> +
> +	mutex_lock(&inst->lock);
> +	streamoff = inst->streamoff;
> +	mutex_unlock(&inst->lock);
> +
> +	if (streamoff)
> +		return;
> +
> +	mutex_lock(&inst->lock);
> +	if (inst->streamon == 0) {
> +		mutex_unlock(&inst->lock);
> +		return;
> +	}
> +	mutex_unlock(&inst->lock);

Why do we keep track of streamon and stream off, why isn't streamoff
ever cleared? Why don't we check both conditions in one critical region?

> +
> +	ret = vidc_hfi_session_stop(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: stop failed (%d)\n", ret);
> +		goto abort;

When are we going to relaim the buffers in these cases?

> +	}
> +
> +	ret = vidc_hfi_session_unload_res(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: release resources failed (%d)\n", ret);
> +		goto abort;
> +	}
> +
> +	ret = session_unregister_bufs(inst);
> +	if (ret) {
> +		dev_err(dev, "failed to release capture buffers: %d\n", ret);
> +		goto abort;
> +	}
> +
> +	ret = internal_bufs_free(inst);
> +
> +	if (hfi_inst->state == INST_INVALID || hfi->state == CORE_INVALID) {
> +		ret = -EINVAL;
> +		goto abort;
> +	}
> +
> +abort:
> +	if (ret)
> +		vidc_hfi_session_abort(hfi, inst->hfi_inst);
> +
> +	vidc_scale_clocks(inst->core);
> +
> +	ret = vidc_hfi_session_deinit(hfi, inst->hfi_inst);
> +
> +	mutex_lock(&inst->lock);
> +	inst->streamoff = 1;
> +	mutex_unlock(&inst->lock);
> +
> +	if (ret)
> +		dev_err(dev, "stop streaming failed type: %d, ret: %d\n",
> +			q->type, ret);
> +
> +	ret = pm_runtime_put_sync(dev);
> +	if (ret < 0)
> +		dev_err(dev, "%s: pm_runtime_put_sync (%d)\n", __func__, ret);
> +}
> +
> +int vidc_vb2_start_streaming(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	struct hfi_core *hfi = &inst->core->hfi;
> +	struct vidc_buffer *buf, *n;
> +	int ret;
> +
> +	ret = session_register_bufs(inst);
> +	if (ret)
> +		return ret;
> +
> +	ret = internal_bufs_alloc(inst);
> +	if (ret)
> +		return ret;
> +
> +	vidc_scale_clocks(inst->core);
> +
> +	ret = vidc_hfi_session_load_res(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: load resources (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	ret = vidc_hfi_session_start(hfi, inst->hfi_inst);
> +	if (ret) {
> +		dev_err(dev, "session: start failed (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	mutex_lock(&inst->bufqueue_lock);
> +	list_for_each_entry_safe(buf, n, &inst->bufqueue, list) {
> +		ret = session_set_buf(&buf->vb.vb2_buf);
> +		if (ret)
> +			break;
> +	}
> +	mutex_unlock(&inst->bufqueue_lock);
> +
> +	if (!ret) {
> +		mutex_lock(&inst->lock);
> +		inst->streamon = 1;
> +		mutex_unlock(&inst->lock);
> +	}
> +
> +	return ret;
> +}
> diff --git a/drivers/media/platform/qcom/vidc/helpers.h b/drivers/media/platform/qcom/vidc/helpers.h
> new file mode 100644
> index 000000000000..a151c96bf939
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/helpers.h
> @@ -0,0 +1,43 @@
> +/*
> + * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#ifndef __VIDC_COMMON_H__
> +#define __VIDC_COMMON_H__

s/COMMON/HELPERS/

> +
> +#include <linux/list.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#include "core.h"
> +
> +struct vidc_buffer {
> +	struct vb2_v4l2_buffer vb;
> +	struct list_head list;
> +	dma_addr_t dma_addr;
> +	struct list_head hfi_list;

This seems to be the list_head used for associating buffers to the
_vidc_ instances.

> +	struct hfi_buffer_desc bd;
> +};
> +
> +#define to_vidc_buffer(buf)	container_of(buf, struct vidc_buffer, vb)
> +
> +struct vb2_v4l2_buffer *
> +vidc_vb2_find_buf(struct vidc_inst *inst, dma_addr_t addr);
> +int vidc_vb2_buf_init(struct vb2_buffer *vb);
> +int vidc_vb2_buf_prepare(struct vb2_buffer *vb);
> +void vidc_vb2_buf_queue(struct vb2_buffer *vb);
> +void vidc_vb2_stop_streaming(struct vb2_queue *q);
> +int vidc_vb2_start_streaming(struct vidc_inst *inst);
> +int vidc_buf_descs(struct vidc_inst *inst, u32 type,
> +		   struct hfi_buffer_requirements *out);
> +int vidc_set_color_format(struct vidc_inst *inst, u32 type, u32 fmt);
> +#endif
> diff --git a/drivers/media/platform/qcom/vidc/int_bufs.c b/drivers/media/platform/qcom/vidc/int_bufs.c
[..]
> +
> +static int internal_alloc_and_set(struct vidc_inst *inst,
> +				  struct hfi_buffer_requirements *bufreq,
> +				  struct vidc_list *buf_list)
> +{
> +	struct vidc_internal_buf *buf;
> +	struct vidc_mem *mem;
> +	unsigned int i;
> +	int ret = 0;
> +
> +	if (!bufreq->size)
> +		return 0;
> +
> +	for (i = 0; i < bufreq->count_actual; i++) {
> +		mem = mem_alloc(inst->core->dev, bufreq->size, 0);

Inline mem_alloc here; might need to make sure bufreq->size is 4K
aligned.

> +		if (IS_ERR(mem)) {
> +			ret = PTR_ERR(mem);
> +			goto err_no_mem;
> +		}
> +
> +		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
> +		if (!buf) {
> +			ret = -ENOMEM;
> +			goto fail_kzalloc;
> +		}
> +
> +		buf->mem = mem;
> +		buf->type = bufreq->type;
> +
> +		ret = internal_set_buf_on_fw(inst, bufreq->type, mem, false);
> +		if (ret)
> +			goto fail_set_buffers;
> +
> +		mutex_lock(&buf_list->lock);
> +		list_add_tail(&buf->list, &buf_list->list);
> +		mutex_unlock(&buf_list->lock);
> +	}
> +
> +	return ret;
> +
> +fail_set_buffers:
> +	kfree(buf);
> +fail_kzalloc:
> +	mem_free(mem);
> +err_no_mem:
> +	return ret;
> +}
> +
[..]
> +
> +static int persist_set_buffer(struct vidc_inst *inst, u32 type)
> +{
> +	struct hfi_buffer_requirements bufreq;
> +	int ret;
> +
> +	ret = vidc_buf_descs(inst, type, &bufreq);
> +	if (ret)
> +		return 0;
> +
> +	mutex_lock(&inst->persistbufs.lock);
> +	if (!list_empty(&inst->persistbufs.list)) {

This function is called twice, with type HFI_BUFFER_INTERNAL_PERSIST and
HFI_BUFFER_INTERNAL_PERSIST_1 respectively. Unless the buffer
requirements are missing for HFI_BUFFER_INTERNAL_PERSIST persistbufs
won't be empty and we will skip the later allocation.

> +		mutex_unlock(&inst->persistbufs.lock);
> +		return 0;
> +	}
> +	mutex_unlock(&inst->persistbufs.lock);
> +
> +	return internal_alloc_and_set(inst, &bufreq, &inst->persistbufs);
> +}
> +
[..]
> +
> +static int scratch_set_buffers(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	int ret;
> +
> +	ret = scratch_unset_buffers(inst, true);
> +	if (ret)
> +		dev_warn(dev, "Failed to release scratch buffers\n");

internal_bufs_free() calls scratch_unset_buffers(reuse=false) so we're
coming here with an empty scratchbufs either way - meaning that this
whole file can be greatly simplified.

So instead of trying to fix that I would suggest that you just let
internal_bufs_alloc() acquire the buffer requirements and call
internal_alloc_and_set() directly, storing the result in a single list.

And then inline a free method in internal_bufs_free() as well as drop
all reuse-stuff and unused/dead code.

That would simplify this file quite a bit and if there actually is a
need for the reusing of buffer that can be added at some later time.

> +
> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH);
> +	if (ret)
> +		goto error;
> +
> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_1);
> +	if (ret)
> +		goto error;
> +
> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_2);
> +	if (ret)
> +		goto error;
> +
> +	return 0;
> +error:
> +	scratch_unset_buffers(inst, false);
> +	return ret;
> +}
> +
> +static int persist_set_buffers(struct vidc_inst *inst)
> +{
> +	int ret;
> +
> +	ret = persist_set_buffer(inst, HFI_BUFFER_INTERNAL_PERSIST);
> +	if (ret)
> +		goto error;
> +
> +	ret = persist_set_buffer(inst, HFI_BUFFER_INTERNAL_PERSIST_1);
> +	if (ret)
> +		goto error;
> +
> +	return 0;
> +
> +error:
> +	persist_unset_buffers(inst);
> +	return ret;
> +}
> +
> +int internal_bufs_alloc(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	int ret;
> +
> +	ret = scratch_set_buffers(inst);
> +	if (ret) {
> +		dev_err(dev, "set scratch buffers (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	ret = persist_set_buffers(inst);
> +	if (ret) {
> +		dev_err(dev, "set persist buffers (%d)\n", ret);
> +		goto error;
> +	}
> +
> +	return 0;
> +
> +error:
> +	scratch_unset_buffers(inst, false);
> +	return ret;
> +}
> +
> +int internal_bufs_free(struct vidc_inst *inst)
> +{
> +	struct device *dev = inst->core->dev;
> +	int ret;
> +
> +	ret = scratch_unset_buffers(inst, false);
> +	if (ret)
> +		dev_err(dev, "failed to release scratch buffers: %d\n", ret);
> +
> +	ret = persist_unset_buffers(inst);
> +	if (ret)
> +		dev_err(dev, "failed to release persist buffers: %d\n", ret);
> +
> +	return ret;
> +}
> diff --git a/drivers/media/platform/qcom/vidc/int_bufs.h b/drivers/media/platform/qcom/vidc/int_bufs.h
> new file mode 100644
> index 000000000000..5f8b2b85839f
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/int_bufs.h
> @@ -0,0 +1,23 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#ifndef __VIDC_INTERNAL_BUFFERS_H__
> +#define __VIDC_INTERNAL_BUFFERS_H__
> +
> +struct vidc_inst;
> +
> +int internal_bufs_alloc(struct vidc_inst *inst);
> +int internal_bufs_free(struct vidc_inst *inst);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vidc/load.c b/drivers/media/platform/qcom/vidc/load.c
> new file mode 100644
> index 000000000000..8ae25fc0e8a5
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/load.c
> @@ -0,0 +1,104 @@
> +/*
> + * Copyright (c) 2013-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +
> +#include <linux/clk.h>
> +
> +#include "core.h"
> +#include "load.h"
> +
> +static u32 get_inst_load(struct vidc_inst *inst)
> +{
> +	int mbs;
> +	u32 w = inst->width;
> +	u32 h = inst->height;
> +
> +	if (!inst->hfi_inst || !(inst->hfi_inst->state >= INST_INIT &&
> +				 inst->hfi_inst->state < INST_STOP))
> +		return 0;
> +
> +	mbs = (ALIGN(w, 16) / 16) * (ALIGN(h, 16) / 16);
> +
> +	return mbs * inst->fps;
> +}
> +
> +static u32 get_load(struct vidc_core *core, u32 session_type)
> +{
> +	struct vidc_inst *inst = NULL;
> +	u32 mbs_per_sec = 0;
> +
> +	mutex_lock(&core->lock);
> +	list_for_each_entry(inst, &core->instances, list) {
> +		if (inst->session_type != session_type)
> +			continue;
> +
> +		mbs_per_sec += get_inst_load(inst);
> +	}
> +	mutex_unlock(&core->lock);
> +
> +	return mbs_per_sec;
> +}
> +
> +static int scale_clocks_load(struct vidc_core *core, u32 mbs_per_sec)
> +{
> +	const struct freq_tbl *table = core->res->freq_tbl;
> +	int num_rows = core->res->freq_tbl_size;
> +	struct clk *clk = core->clks[0];

Using individual clk pointers instead of this array would make this
"core_clk" easier to follow.

> +	struct device *dev = core->dev;
> +	unsigned long freq = table[0].freq;
> +	int ret, i;
> +
> +	if (!mbs_per_sec && num_rows > 1) {
> +		freq = table[num_rows - 1].freq;
> +		goto set_freq;
> +	}

Here we will set freq to the last entry in the freq table, potentially
table[0] if num_rows == 1, so the second part of the conditional doesn't
add any value and you can skip the early initialization above.

And you can put the loop below in an else block instead of using a goto.
> +
> +	for (i = 0; i < num_rows; i++) {
> +		if (mbs_per_sec > table[i].load)
> +			break;
> +		freq = table[i].freq;
> +	}
> +
> +set_freq:
> +
> +	ret = clk_set_rate(clk, freq);
> +	if (ret) {
> +		dev_err(dev, "failed to set clock rate %lu (%d)\n", freq, ret);
> +		return ret;
> +	}
> +
> +	return 0;

ret will be 0 here, so print the error message conditionally and then
just return ret.

> +}
> +
> +int vidc_scale_clocks(struct vidc_core *core)

This is only called from helpers.c, drop this file and move the
implementation there.

> +{
> +	struct device *dev = core->dev;
> +	u32 mbs_per_sec;
> +	int ret;
> +
> +	mbs_per_sec = get_load(core, VIDC_SESSION_TYPE_ENC) +
> +		      get_load(core, VIDC_SESSION_TYPE_DEC);
> +
> +	if (mbs_per_sec > core->res->max_load) {
> +		dev_warn(dev, "HW is overloaded, needed: %d max: %d\n",
> +			 mbs_per_sec, core->res->max_load);
> +		return -EBUSY;
> +	}
> +
> +	ret = scale_clocks_load(core, mbs_per_sec);
> +	if (ret)
> +		dev_warn(dev, "failed to scale clocks, performance might be impacted\n");
> +
> +	return 0;
> +}
[..]
> diff --git a/drivers/media/platform/qcom/vidc/mem.c b/drivers/media/platform/qcom/vidc/mem.c
> new file mode 100644
> index 000000000000..6a83b5784410
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/mem.c
> @@ -0,0 +1,64 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +
> +#include <linux/device.h>
> +#include <linux/dma-direction.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/slab.h>
> +
> +#include "mem.h"
> +
> +struct vidc_mem *mem_alloc(struct device *dev, size_t size, int map_kernel)

This is a terrible name for a global function.

But I think you can favorably inline this into the two callers. They
both have their own tracking objects. So just drop this entire file.

> +{
> +	struct vidc_mem *mem;
> +
> +	if (!size)
> +		return ERR_PTR(-EINVAL);
> +
> +	if (IS_ERR(dev))
> +		return ERR_CAST(dev);
> +
> +	mem = kzalloc(sizeof(*mem), GFP_KERNEL);
> +	if (!mem)
> +		return ERR_PTR(-ENOMEM);
> +
> +	mem->size = ALIGN(size, SZ_4K);
> +	mem->iommu_dev = dev;
> +
> +	mem->attrs = DMA_ATTR_WRITE_COMBINE;
> +
> +	if (!map_kernel)
> +		mem->attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
> +
> +	mem->kvaddr = dma_alloc_attrs(mem->iommu_dev, mem->size, &mem->da,
> +				      GFP_KERNEL, mem->attrs);
> +	if (!mem->kvaddr) {
> +		kfree(mem);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +
> +	return mem;
> +}
> +
> +void mem_free(struct vidc_mem *mem)
> +{
> +	if (!mem)
> +		return;
> +
> +	dma_free_attrs(mem->iommu_dev, mem->size, mem->kvaddr,
> +	       mem->da, mem->attrs);
> +	kfree(mem);
> +};
[..]
> diff --git a/drivers/media/platform/qcom/vidc/resources.c b/drivers/media/platform/qcom/vidc/resources.c
> new file mode 100644
> index 000000000000..e00ed1caa824
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vidc/resources.c
> @@ -0,0 +1,46 @@
> +/*
> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
> + * Copyright (C) 2016 Linaro Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + */
> +#include <linux/bug.h>
> +#include <linux/types.h>
> +#include <linux/kernel.h>
> +
> +#include "hfi.h"
> +
> +static const struct freq_tbl msm8916_freq_table[] = {
> +	{ 352800, 228570000 },	/* 1920x1088 @ 30 + 1280x720 @ 30 */
> +	{ 244800, 160000000 },	/* 1920x1088 @ 30 */
> +	{ 108000, 100000000 },	/* 1280x720 @ 30 */
> +};
> +
> +static const struct reg_val msm8916_reg_preset[] = {
> +	{ 0xe0020, 0x05555556 },
> +	{ 0xe0024, 0x05555556 },
> +	{ 0x80124, 0x00000003 },
> +};
> +
> +const struct vidc_resources msm8916_res = {
> +	.freq_tbl = msm8916_freq_table,
> +	.freq_tbl_size = ARRAY_SIZE(msm8916_freq_table),
> +	.reg_tbl = msm8916_reg_preset,
> +	.reg_tbl_size = ARRAY_SIZE(msm8916_reg_preset),
> +	.clks = {"core", "iface", "bus", },
> +	.clks_num = 3,
> +	.max_load = 352800, /* 720p@30 + 1080p@30 */
> +	.hfi_version = 0,

Unused

> +	.vmem_id = VIDC_RESOURCE_NONE,

Unused

> +	.vmem_size = 0,

Unused

> +	.vmem_addr = 0,

Unused

> +	.dma_mask = 0xddc00000 - 1,
> +};

These tables could with favor be moved next to the of_table in vidc.c

Regards,
Bjorn

Re: [PATCH 5/8] media: vidc: add Host Firmware Interface (HFI)

[Bjorn Andersson]

On Mon 22 Aug 06:13 PDT 2016, Stanimir Varbanov wrote:

> This is the implementation of HFI. It is loaded with the
> responsibility to comunicate with the firmware through an
> interface commands and messages.
> 
>  - hfi.c has interface functions used by the core, decoder
> and encoder parts to comunicate with the firmware. For example
> there are functions for session and core initialisation.
> 

I can't help feeling that the split between core.c and hfi.c is a
remnant of a vidc driver supporting both HFI and pre-HFI with the same
v4l code.

What do you think about merging vidc_core with hfi_core and vidc_inst
with hfi_inst? Both seems to be in a 1:1 relationship.

>  - hfi_cmds has packetization operations which preparing
> packets to be send from host to firmware.
> 
>  - hfi_msgs takes care of messages sent from firmware to the
> host.
> 
[..]
> diff --git a/drivers/media/platform/qcom/vidc/hfi_cmds.c b/drivers/media/platform/qcom/vidc/hfi_cmds.c
[..]
> +
> +static const struct hfi_packetization_ops hfi_default = {
> +	.sys_init = pkt_sys_init,
> +	.sys_pc_prep = pkt_sys_pc_prep,
> +	.sys_idle_indicator = pkt_sys_idle_indicator,
> +	.sys_power_control = pkt_sys_power_control,
> +	.sys_set_resource = pkt_sys_set_resource,
> +	.sys_release_resource = pkt_sys_unset_resource,
> +	.sys_debug_config = pkt_sys_debug_config,
> +	.sys_coverage_config = pkt_sys_coverage_config,
> +	.sys_ping = pkt_sys_ping,
> +	.sys_image_version = pkt_sys_image_version,
> +	.ssr_cmd = pkt_ssr_cmd,
> +	.session_init = pkt_session_init,
> +	.session_cmd = pkt_session_cmd,
> +	.session_set_buffers = pkt_session_set_buffers,
> +	.session_release_buffers = pkt_session_release_buffers,
> +	.session_etb_decoder = pkt_session_etb_decoder,
> +	.session_etb_encoder = pkt_session_etb_encoder,
> +	.session_ftb = pkt_session_ftb,
> +	.session_parse_seq_header = pkt_session_parse_seq_header,
> +	.session_get_seq_hdr = pkt_session_get_seq_hdr,
> +	.session_flush = pkt_session_flush,
> +	.session_get_property = pkt_session_get_property,
> +	.session_set_property = pkt_session_set_property,
> +};
> +
> +static const struct hfi_packetization_ops *get_3xx_ops(void)
> +{
> +	static struct hfi_packetization_ops hfi_3xx;
> +
> +	hfi_3xx = hfi_default;
> +	hfi_3xx.session_set_property = pkt_session_set_property_3xx;
> +
> +	return &hfi_3xx;
> +}
> +
> +const struct hfi_packetization_ops *
> +hfi_get_pkt_ops(enum hfi_packetization_type type)

The only reasonable argument I can come up with for not just exposing
these as global functions would be that there are 23 of them... Can we
skip the jump table?

> +{
> +	switch (type) {
> +	case HFI_PACKETIZATION_LEGACY:
> +		return &hfi_default;
> +	case HFI_PACKETIZATION_3XX:
> +		return get_3xx_ops();
> +	}
> +
> +	return NULL;
> +}

Regards,
Bjorn

Re: [PATCH 6/8] media: vidc: add Venus HFI files

[Bjorn Andersson]

On Mon 22 Aug 06:13 PDT 2016, Stanimir Varbanov wrote:

> Here is the implementation of Venus video accelerator low-level
> functionality. It contanins code which setup the registers and
> startup uthe processor, allocate and manipulates with the shared
> memory used for sending commands and receiving messages.
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/vidc/hfi_venus.c    | 1539 +++++++++++++++++++++++
>  drivers/media/platform/qcom/vidc/hfi_venus.h    |   25 +
>  drivers/media/platform/qcom/vidc/hfi_venus_io.h |   98 ++
>  3 files changed, 1662 insertions(+)
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.c
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.h
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus_io.h
> 
> diff --git a/drivers/media/platform/qcom/vidc/hfi_venus.c b/drivers/media/platform/qcom/vidc/hfi_venus.c
[..]
> +
> +static const struct hfi_ops venus_hfi_ops = {
> +	.core_init			= venus_hfi_core_init,
> +	.core_deinit			= venus_hfi_core_deinit,
> +	.core_ping			= venus_hfi_core_ping,
> +	.core_trigger_ssr		= venus_hfi_core_trigger_ssr,
> +
> +	.session_init			= venus_hfi_session_init,
> +	.session_end			= venus_hfi_session_end,
> +	.session_abort			= venus_hfi_session_abort,
> +	.session_flush			= venus_hfi_session_flush,
> +	.session_start			= venus_hfi_session_start,
> +	.session_stop			= venus_hfi_session_stop,
> +	.session_etb			= venus_hfi_session_etb,
> +	.session_ftb			= venus_hfi_session_ftb,
> +	.session_set_buffers		= venus_hfi_session_set_buffers,
> +	.session_release_buffers	= venus_hfi_session_release_buffers,
> +	.session_load_res		= venus_hfi_session_load_res,
> +	.session_release_res		= venus_hfi_session_release_res,
> +	.session_parse_seq_hdr		= venus_hfi_session_parse_seq_hdr,
> +	.session_get_seq_hdr		= venus_hfi_session_get_seq_hdr,
> +	.session_set_property		= venus_hfi_session_set_property,
> +	.session_get_property		= venus_hfi_session_get_property,
> +
> +	.resume				= venus_hfi_resume,
> +	.suspend			= venus_hfi_suspend,
> +
> +	.isr				= venus_isr,
> +	.isr_thread			= venus_isr_thread,
> +};
> +
> +void venus_hfi_destroy(struct hfi_core *hfi)
> +{
> +	struct venus_hfi_device *hdev = to_hfi_priv(hfi);
> +
> +	venus_interface_queues_release(hdev);
> +	mutex_destroy(&hdev->lock);
> +	kfree(hdev);
> +}
> +
> +int venus_hfi_create(struct hfi_core *hfi, const struct vidc_resources *res,
> +		     void __iomem *base)
> +{

Rather than having the core figure out which *_hfi_create() to call I
think this should be the probe() entry point, calling into the core
registering the venus_hfi_ops - a common-probe() in the hfi/vidc core
could still do most of the heavy lifting.

Probing the driver up from the transport rather than for the highest
logical layer allows us to inject a separate hfi_ops for the apr tal
case.

> +	struct venus_hfi_device *hdev;
> +	int ret;
> +
> +	hdev = kzalloc(sizeof(*hdev), GFP_KERNEL);
> +	if (!hdev)
> +		return -ENOMEM;
> +
> +	mutex_init(&hdev->lock);
> +
> +	hdev->res = res;
> +	hdev->pkt_ops = hfi->pkt_ops;
> +	hdev->packetization_type = HFI_PACKETIZATION_LEGACY;
> +	hdev->base = base;
> +	hdev->dev = hfi->dev;
> +	hdev->suspended = true;
> +
> +	hfi->priv = hdev;
> +	hfi->ops = &venus_hfi_ops;
> +	hfi->core_caps = VIDC_ENC_ROTATION_CAPABILITY |
> +			 VIDC_ENC_SCALING_CAPABILITY |
> +			 VIDC_ENC_DEINTERLACE_CAPABILITY |
> +			 VIDC_DEC_MULTI_STREAM_CAPABILITY;
> +
> +	ret = venus_interface_queues_init(hdev);
> +	if (ret)
> +		goto err_kfree;
> +
> +	return 0;
> +
> +err_kfree:
> +	kfree(hdev);
> +	hfi->priv = NULL;
> +	hfi->ops = NULL;
> +	return ret;
> +}

I'll try to find some time to do a more detailed review of the
implementation.

Regards,
Bjorn

Re: [PATCH 3/8] media: vidc: decoder: add video decoder files

[Stanimir Varbanov]

Hi Hans,

Thanks for the valuable comments!

<cut>

>> +
>> +static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +	const struct vidc_format *fmt = NULL;
>> +	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
>> +
>> +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
>> +		fmt = inst->fmt_cap;
>> +	else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> +		fmt = inst->fmt_out;
>> +
>> +	if (inst->in_reconfig) {
>> +		inst->height = inst->reconfig_height;
>> +		inst->width = inst->reconfig_width;
>> +		inst->in_reconfig = false;
>> +	}
>> +
>> +	pixmp->pixelformat = fmt->pixfmt;
>> +
>> +	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>> +		pixmp->width = inst->width;
>> +		pixmp->height = inst->height;
>> +		pixmp->colorspace = inst->colorspace;
>> +		pixmp->ycbcr_enc = inst->ycbcr_enc;
>> +		pixmp->quantization = inst->quantization;
>> +		pixmp->xfer_func = inst->xfer_func;
>> +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
>> +		pixmp->width = inst->out_width;
>> +		pixmp->height = inst->out_height;
>> +	}
>> +
>> +	vdec_try_fmt_common(inst, f);
> 
> Is this call really necessary? The current format should always be a valid format,
> so this is dubious.

Agreed, this looks over engineered. The only thing which could be
related to this call could be the reconfiguration above. The
reconfiguration is a event from firmware side that the streaming
resolution (based on stream headers) is different from the initial
resolution set it before stream_on. In this case the decoder refuses to
start and wait for reconfiguration. Also get_fmt can be called after the
stream_on so maybe my idea was to return the changed resolution to
userspace so that it can handle the exception properly.

> 
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
>> +	struct v4l2_pix_format_mplane orig_pixmp;
>> +	const struct vidc_format *fmt;
>> +	struct v4l2_format format;
>> +	u32 pixfmt_out = 0, pixfmt_cap = 0;
>> +
>> +	orig_pixmp = *pixmp;
>> +
>> +	fmt = vdec_try_fmt_common(inst, f);
>> +	if (!fmt)
>> +		return -EINVAL;
>> +
>> +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
>> +		pixfmt_out = pixmp->pixelformat;
>> +		pixfmt_cap = inst->fmt_cap->pixfmt;
>> +	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>> +		pixfmt_cap = pixmp->pixelformat;
>> +		pixfmt_out = inst->fmt_out->pixfmt;
>> +	}
>> +
>> +	memset(&format, 0, sizeof(format));
>> +
>> +	format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>> +	format.fmt.pix_mp.pixelformat = pixfmt_out;
>> +	format.fmt.pix_mp.width = orig_pixmp.width;
>> +	format.fmt.pix_mp.height = orig_pixmp.height;
>> +	vdec_try_fmt_common(inst, &format);
>> +	inst->out_width = format.fmt.pix_mp.width;
>> +	inst->out_height = format.fmt.pix_mp.height;
>> +	inst->colorspace = pixmp->colorspace;
>> +	inst->ycbcr_enc = pixmp->ycbcr_enc;
>> +	inst->quantization = pixmp->quantization;
>> +	inst->xfer_func = pixmp->xfer_func;
> 
> These four fields can only be set if f->type == VIDEO_OUTPUT.

OK, corrected.

> 
>> +
>> +	memset(&format, 0, sizeof(format));
>> +
>> +	format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>> +	format.fmt.pix_mp.pixelformat = pixfmt_cap;
>> +	format.fmt.pix_mp.width = orig_pixmp.width;
>> +	format.fmt.pix_mp.height = orig_pixmp.height;
>> +	vdec_try_fmt_common(inst, &format);
>> +	inst->width = format.fmt.pix_mp.width;
>> +	inst->height = format.fmt.pix_mp.height;
> 
> This doesn't look right.
> 
> If I understand this code correctly, the capture and output format
> depend on one another. Can you explain a bit more what the dependencies
> are? I have to revisit this later, once I have a better idea of what's
> going on here.

The thing is that the capture resolution have hardware alignment
restrictions. Here the height must be aligned to 32 lines. So I got
original pixmp struct and call try_fmt_common to align height to 32 and
save the resolution in the instance height variable.

> 
>> +
>> +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> +		inst->fmt_out = fmt;
>> +	else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
>> +		inst->fmt_cap = fmt;
>> +
>> +	return 0;
>> +}
>> +
>> +static int
>> +vdec_g_selection(struct file *file, void *priv, struct v4l2_selection *s)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +
>> +	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
>> +		return -EINVAL;
>> +
>> +	switch (s->target) {
>> +	case V4L2_SEL_TGT_CROP_DEFAULT:
>> +	case V4L2_SEL_TGT_CROP_BOUNDS:
>> +	case V4L2_SEL_TGT_CROP:
>> +	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
>> +	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
>> +	case V4L2_SEL_TGT_COMPOSE:
> 
> You can't set both crop and compose with just a single s->r. That makes no sense.
> 
> What exactly are you trying to do here?

This is related to hardware alignment restriction. I'm expecting
userspace (gstreamer is one of them) would call g_selection (g_crop) to
got the actual resolution without vertical padding.

> 
>> +		break;
>> +	default:
>> +		return -EINVAL;
>> +	}
>> +
>> +	s->r.top = 0;
>> +	s->r.left = 0;
>> +	s->r.width = inst->out_width;
>> +	s->r.height = inst->out_height;
>> +
>> +	return 0;
>> +}
>> +
>> +static int
>> +vdec_reqbufs(struct file *file, void *fh, struct v4l2_requestbuffers *b)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	if (!b->count)
>> +		vb2_core_queue_release(queue);
> 
> Don't do this. Let vb2_reqbufs handle this for you.

Sorry this is leftover during testing with some custom Gstreamer plugin.

> 
>> +
>> +	return vb2_reqbufs(queue, b);
>> +}
>> +
>> +static int
>> +vdec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
>> +{
>> +	strlcpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
>> +	strlcpy(cap->card, "video decoder", sizeof(cap->card));
>> +	strlcpy(cap->bus_info, "platform:vidc", sizeof(cap->bus_info));
>> +
>> +	cap->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
>> +	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
> 
> Drop these two lines. Instead set the device_caps field of struct video_device
> to V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING. The v4l2 core will fill
> in these two cap fields for you based on the vdev->device_caps field.

OK, done.

> 
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
>> +{
>> +	const struct vidc_format *fmt;
>> +
>> +	memset(f->reserved, 0, sizeof(f->reserved));
>> +
>> +	fmt = find_format_by_index(f->index, f->type);
>> +	if (!fmt)
>> +		return -EINVAL;
>> +
>> +	f->pixelformat = fmt->pixfmt;
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_querybuf(struct file *file, void *fh, struct v4l2_buffer *b)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
>> +	unsigned int p;
>> +	int ret;
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	ret = vb2_querybuf(queue, b);
>> +	if (ret)
>> +		return ret;
>> +
>> +	if (b->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
>> +	    b->memory == V4L2_MEMORY_MMAP) {
>> +		for (p = 0; p < b->length; p++)
>> +			b->m.planes[p].m.mem_offset += DST_QUEUE_OFF_BASE;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int
>> +vdec_create_bufs(struct file *file, void *fh, struct v4l2_create_buffers *b)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->format.type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	return vb2_create_bufs(queue, b);
>> +}
>> +
>> +static int vdec_prepare_buf(struct file *file, void *fh, struct v4l2_buffer *b)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	return vb2_prepare_buf(queue, b);
>> +}
>> +
>> +static int vdec_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	return vb2_qbuf(queue, b);
>> +}
>> +
>> +static int
>> +vdec_exportbuf(struct file *file, void *fh, struct v4l2_exportbuffer *b)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	return vb2_expbuf(queue, b);
>> +}
>> +
>> +static int vdec_dqbuf(struct file *file, void *fh, struct v4l2_buffer *b)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, b->type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	return vb2_dqbuf(queue, b, file->f_flags & O_NONBLOCK);
>> +}
>> +
>> +static int vdec_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	return vb2_streamon(queue, type);
>> +}
>> +
>> +static int vdec_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
>> +{
>> +	struct vb2_queue *queue = vidc_to_vb2q(file, type);
>> +
>> +	if (!queue)
>> +		return -EINVAL;
>> +
>> +	return vb2_streamoff(queue, type);
>> +}
> 
> Is there a reason why the v4l2-mem2mem framework isn't used? It seems to me that
> a lot of this code is in there as well.

So in the begging of driver reworking I have the impression that mem2mem
is the right API to use. Then I decided that this will complicate the
driver a bit without any benefit. Of course I could be wrong.

> 
>> +
>> +static int vdec_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +	struct v4l2_captureparm *cap = &a->parm.capture;
>> +	struct v4l2_fract *timeperframe = &cap->timeperframe;
>> +	u64 us_per_frame, fps;
>> +
>> +	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
>> +	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> +		return -EINVAL;
>> +
>> +	memset(cap->reserved, 0, sizeof(cap->reserved));
>> +	if (!timeperframe->denominator)
>> +		timeperframe->denominator = inst->timeperframe.denominator;
>> +	if (!timeperframe->numerator)
>> +		timeperframe->numerator = inst->timeperframe.numerator;
>> +	cap->readbuffers = 0;
> 
> Just set readbuffers to the minimum number of required buffers. Hmm, shouldn't
> v4l2-compliance complain about a 0 value for readbuffers? Odd.

I guess I misunderstood the meaning of those fields and just zeroing
them to make v4l2-compliance happy.

> 
> Also, for output use v4l2_outputparm. I know, they are the same, but I don't
> really like it when that fact is (ab)used.

sure, I will.

> 
>> +	cap->extendedmode = 0;
>> +	cap->capability = V4L2_CAP_TIMEPERFRAME;
>> +	us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
>> +	do_div(us_per_frame, timeperframe->denominator);
>> +
>> +	if (!us_per_frame)
>> +		return -EINVAL;
>> +
>> +	fps = (u64)USEC_PER_SEC;
>> +	do_div(fps, us_per_frame);
>> +
>> +	inst->fps = fps;
>> +	inst->timeperframe = *timeperframe;
> 
> Can both capture and output set the timeperframe? Is that intended?

>From the firmware point of view the timeperframe is expected to be set
on the input buffers.

> 
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +
>> +	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
>> +	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> +		return -EINVAL;
>> +
>> +	a->parm.capture.capability |= V4L2_CAP_TIMEPERFRAME;
>> +	a->parm.capture.timeperframe = inst->timeperframe;
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_enum_framesizes(struct file *file, void *fh,
>> +				struct v4l2_frmsizeenum *fsize)
>> +{
>> +	struct hfi_inst *hfi_inst = to_hfi_inst(file);
>> +	const struct vidc_format *fmt;
>> +
>> +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
>> +
>> +	fmt = find_format(fsize->pixel_format,
>> +			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>> +	if (!fmt) {
>> +		fmt = find_format(fsize->pixel_format,
>> +				  V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> +		if (!fmt)
>> +			return -EINVAL;
>> +	}
>> +
>> +	if (fsize->index)
>> +		return -EINVAL;
>> +
>> +	fsize->stepwise.min_width = hfi_inst->width.min;
>> +	fsize->stepwise.max_width = hfi_inst->width.max;
>> +	fsize->stepwise.step_width = hfi_inst->width.step_size;
>> +	fsize->stepwise.min_height = hfi_inst->height.min;
>> +	fsize->stepwise.max_height = hfi_inst->height.max;
>> +	fsize->stepwise.step_height = hfi_inst->height.step_size;
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_enum_frameintervals(struct file *file, void *fh,
>> +				    struct v4l2_frmivalenum *fival)
>> +{
>> +	struct hfi_inst *hfi_inst = to_hfi_inst(file);
>> +	const struct vidc_format *fmt;
>> +
>> +	fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
>> +
>> +	fmt = find_format(fival->pixel_format,
>> +			  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>> +	if (!fmt)
>> +		return -EINVAL;
>> +
>> +	if (fival->index)
>> +		return -EINVAL;
>> +
>> +	if (!fival->width || !fival->height)
>> +		return -EINVAL;
>> +
>> +	if (fival->width > hfi_inst->width.max ||
>> +	    fival->width < hfi_inst->width.min ||
>> +	    fival->height > hfi_inst->height.max ||
>> +	    fival->height < hfi_inst->height.min)
>> +		return -EINVAL;
>> +
>> +	fival->stepwise.min.numerator = hfi_inst->framerate.min;
>> +	fival->stepwise.min.denominator = 1;
>> +	fival->stepwise.max.numerator = hfi_inst->framerate.max;
>> +	fival->stepwise.max.denominator = 1;
>> +	fival->stepwise.step.numerator = hfi_inst->framerate.step_size;
>> +	fival->stepwise.step.denominator = 1;
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_subscribe_event(struct v4l2_fh *fh,
>> +				const struct v4l2_event_subscription *sub)
>> +{
>> +	switch (sub->type) {
>> +	case V4L2_EVENT_EOS:
>> +		return v4l2_event_subscribe(fh, sub, 2, NULL);
>> +	case V4L2_EVENT_SOURCE_CHANGE:
>> +		return v4l2_src_change_event_subscribe(fh, sub);
>> +	case V4L2_EVENT_CTRL:
>> +		return v4l2_ctrl_subscribe_event(fh, sub);
>> +	default:
>> +		return -EINVAL;
>> +	}
>> +}
>> +
>> +static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
>> +	.vidioc_querycap = vdec_querycap,
>> +	.vidioc_enum_fmt_vid_cap_mplane = vdec_enum_fmt,
>> +	.vidioc_enum_fmt_vid_out_mplane = vdec_enum_fmt,
>> +	.vidioc_s_fmt_vid_cap_mplane = vdec_s_fmt,
>> +	.vidioc_s_fmt_vid_out_mplane = vdec_s_fmt,
>> +	.vidioc_g_fmt_vid_cap_mplane = vdec_g_fmt,
>> +	.vidioc_g_fmt_vid_out_mplane = vdec_g_fmt,
>> +	.vidioc_try_fmt_vid_cap_mplane = vdec_try_fmt,
>> +	.vidioc_try_fmt_vid_out_mplane = vdec_try_fmt,
>> +	.vidioc_g_selection = vdec_g_selection,
>> +	.vidioc_reqbufs = vdec_reqbufs,
>> +	.vidioc_querybuf = vdec_querybuf,
>> +	.vidioc_create_bufs = vdec_create_bufs,
>> +	.vidioc_prepare_buf = vdec_prepare_buf,
>> +	.vidioc_qbuf = vdec_qbuf,
>> +	.vidioc_expbuf = vdec_exportbuf,
>> +	.vidioc_dqbuf = vdec_dqbuf,
>> +	.vidioc_streamon = vdec_streamon,
>> +	.vidioc_streamoff = vdec_streamoff,
>> +	.vidioc_s_parm = vdec_s_parm,
>> +	.vidioc_g_parm = vdec_g_parm,
>> +	.vidioc_enum_framesizes = vdec_enum_framesizes,
>> +	.vidioc_enum_frameintervals = vdec_enum_frameintervals,
>> +	.vidioc_subscribe_event = vdec_subscribe_event,
>> +	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>> +};
>> +
>> +static int vdec_init_session(struct vidc_inst *inst)
>> +{
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	u32 pixfmt = inst->fmt_out->pixfmt;
>> +	struct hfi_framesize fs;
>> +	u32 ptype;
>> +	int ret;
>> +
>> +	ret = vidc_hfi_session_init(hfi, inst->hfi_inst, pixfmt,
>> +				    VIDC_SESSION_TYPE_DEC);
>> +	if (ret)
>> +		return ret;
>> +
>> +	ptype = HFI_PROPERTY_PARAM_FRAME_SIZE;
>> +	fs.buffer_type = HFI_BUFFER_INPUT;
>> +	fs.width = inst->out_width;
>> +	fs.height = inst->out_height;
>> +
>> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
>> +	if (ret)
>> +		goto err;
>> +
>> +	fs.buffer_type = HFI_BUFFER_OUTPUT;
>> +	fs.width = inst->width;
>> +	fs.height = inst->height;
>> +
>> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst, ptype, &fs);
>> +	if (ret)
>> +		goto err;
>> +
>> +	pixfmt = inst->fmt_cap->pixfmt;
>> +
>> +	ret = vidc_set_color_format(inst, HFI_BUFFER_OUTPUT, pixfmt);
>> +	if (ret)
>> +		goto err;
>> +
>> +	return 0;
>> +err:
>> +	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
>> +	return ret;
>> +}
>> +
>> +static int vdec_cap_num_buffers(struct vidc_inst *inst,
>> +				struct hfi_buffer_requirements *bufreq)
>> +{
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	struct device *dev = inst->core->dev;
>> +	int ret, ret2;
>> +
>> +	ret = pm_runtime_get_sync(dev);
>> +	if (ret < 0)
>> +		return ret;
>> +
>> +	ret = vdec_init_session(inst);
>> +	if (ret)
>> +		goto put_sync;
>> +
>> +	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, bufreq);
>> +
>> +	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
>> +
>> +put_sync:
>> +	ret2 = pm_runtime_put_sync(dev);
>> +
>> +	return ret ? ret : ret2;
>> +}
>> +
>> +static int vdec_queue_setup(struct vb2_queue *q,
>> +			    unsigned int *num_buffers, unsigned int *num_planes,
>> +			    unsigned int sizes[], struct device *alloc_devs[])
>> +{
>> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
>> +	struct hfi_buffer_requirements bufreq;
>> +	unsigned int p;
>> +	int ret = 0;
>> +	u32 mbs;
>> +
>> +	switch (q->type) {
>> +	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>> +		*num_planes = inst->fmt_out->num_planes;
>> +
>> +		*num_buffers = clamp_val(*num_buffers, 4, VIDEO_MAX_FRAME);
> 
> No need to check for VIDEO_MAX_FRAME. I assume 4 is the minimum required
> number of buffers. Just set min_buffers_needed in struct vb2_queue to 4 instead.

Sure, I missed the clamp in videobu2-core.

> 
>> +
>> +		mbs = inst->out_width * inst->out_height /
>> +				MACROBLOCKS_PER_PIXEL;
>> +		for (p = 0; p < *num_planes; p++) {
>> +			sizes[p] = get_framesize_compressed(mbs);
>> +			alloc_devs[p] = inst->core->dev;
> 
> Don't do this here. Just set the dev field of vb2_queue to inst->core->dev.
> alloc_devs[] is prefilled with q->dev by vb2.

Good remark, this change has been made latetly so I have some excuse :)

> 
>> +		}
>> +
>> +		inst->num_input_bufs = *num_buffers;
>> +		break;
>> +	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
>> +		*num_planes = inst->fmt_cap->num_planes;
>> +
>> +		ret = vdec_cap_num_buffers(inst, &bufreq);
>> +		if (ret)
>> +			break;
>> +
>> +		*num_buffers = max(*num_buffers, bufreq.count_actual);
> 
> How is bufreq.count_actual derived? I.e. what does it depend on?

Good question, the count_actual is derived from firmware based on input
and output resolutions, framerate and codec i.e. the firmware based on
those parameters decides the number of buffers need for optimal
performance.

> 
>> +
>> +		for (p = 0; p < *num_planes; p++) {
>> +			sizes[p] = get_framesize_nv12(p, inst->height,
>> +						      inst->width);
>> +			alloc_devs[p] = inst->core->dev;
>> +		}
>> +
>> +		inst->num_output_bufs = *num_buffers;
>> +		break;
>> +	default:
>> +		ret = -EINVAL;
>> +		break;
>> +	}
>> +
>> +	return ret;
>> +}
>> +
>> +static int vdec_check_configuration(struct vidc_inst *inst)
>> +{
>> +	struct hfi_buffer_requirements bufreq;
>> +	int ret;
>> +
>> +	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
>> +	if (ret)
>> +		return ret;
>> +
>> +	if (inst->num_output_bufs < bufreq.count_actual ||
>> +	    inst->num_output_bufs < bufreq.count_min)
>> +		return -EINVAL;
>> +
>> +	ret = vidc_buf_descs(inst, HFI_BUFFER_INPUT, &bufreq);
>> +	if (ret)
>> +		return ret;
>> +
>> +	if (inst->num_input_bufs < bufreq.count_min)
>> +		return -EINVAL;
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
>> +{
>> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	struct device *dev = inst->core->dev;
>> +	struct hfi_buffer_requirements bufreq;
>> +	struct hfi_buffer_count_actual buf_count;
>> +	struct vb2_queue *queue;
>> +	u32 ptype;
>> +	int ret;
>> +
>> +	switch (q->type) {
>> +	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>> +		queue = &inst->bufq_cap;
>> +		break;
>> +	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
>> +		queue = &inst->bufq_out;
>> +		break;
>> +	default:
> 
> If start_streaming fails, then all pending buffers have to be returned by the driver
> by calling vb2_buffer_done(VB2_BUF_STATE_QUEUED). This will give ownership back to
> userspace.

Infact this error path shouldn't be possible, because videobu2-core
should check q->type before jumping here?

> 
>> +		return -EINVAL;
>> +	}
>> +
>> +	if (!vb2_is_streaming(queue))
>> +		return 0;
> 
> Can never happen, no need to test for this.

This can happen cause start_streaming is called for OUTPUT and for
CAPTURE queues, see the above switch (q->type). If start_streaming is
called for the CAPTURE thus vb2_is_streaming() checks does
start_streaming is already called for OUTPUT. So this guarantee that the
firmware will be started when the streaming is started for both queues.

> 
>> +
>> +	inst->in_reconfig = false;
>> +	inst->sequence = 0;
>> +
>> +	ret = pm_runtime_get_sync(dev);
>> +	if (ret < 0)
>> +		return ret;
>> +
>> +	ret = vdec_init_session(inst);
>> +	if (ret)
>> +		goto put_sync;
>> +
>> +	ret = vdec_set_properties(inst);
>> +	if (ret)
>> +		goto deinit_sess;
>> +
>> +	ret = vdec_check_configuration(inst);
>> +	if (ret)
>> +		goto deinit_sess;
>> +
>> +	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
>> +	buf_count.type = HFI_BUFFER_INPUT;
>> +	buf_count.count_actual = inst->num_input_bufs;
>> +
>> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
>> +					    ptype, &buf_count);
>> +	if (ret)
>> +		goto deinit_sess;
>> +
>> +	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
>> +	if (ret)
>> +		goto deinit_sess;
>> +
>> +	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
>> +	buf_count.type = HFI_BUFFER_OUTPUT;
>> +	buf_count.count_actual = inst->num_output_bufs;
>> +
>> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
>> +					    ptype, &buf_count);
>> +	if (ret)
>> +		goto deinit_sess;
>> +
>> +	if (inst->num_output_bufs != bufreq.count_actual) {
>> +		struct hfi_buffer_display_hold_count_actual display;
>> +
>> +		ptype = HFI_PROPERTY_PARAM_BUFFER_DISPLAY_HOLD_COUNT_ACTUAL;
>> +		display.type = HFI_BUFFER_OUTPUT;
>> +		display.hold_count = inst->num_output_bufs -
>> +				     bufreq.count_actual;
>> +
>> +		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
>> +						    ptype, &display);
>> +		if (ret)
>> +			goto deinit_sess;
>> +	}
>> +
>> +	ret = vidc_vb2_start_streaming(inst);
>> +	if (ret)
>> +		goto deinit_sess;
>> +
>> +	return 0;
>> +
>> +deinit_sess:
>> +	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
> 
> Note that vidc_vb2_start_streaming already calls vidc_hfi_session_deinit on error.

Probably you wanted to say vdec_init_session has deinit on error path?
If so I cannot see the problem, vidc_hfi_session_deinit is called only once?

> No idea if vidc_hfi_session_deinit can handle that, I just thought I'd mention it.
> 
>> +put_sync:
>> +	pm_runtime_put_sync(dev);
>> +	return ret;
>> +}
>> +
>> +static const struct vb2_ops vdec_vb2_ops = {
>> +	.queue_setup = vdec_queue_setup,
>> +	.buf_init = vidc_vb2_buf_init,
>> +	.buf_prepare = vidc_vb2_buf_prepare,
>> +	.start_streaming = vdec_start_streaming,
>> +	.stop_streaming = vidc_vb2_stop_streaming,
> 
> 
> Note that stop_streaming has to return all pending buffers by calling vb2_buffer_done(VB2_BUF_STATE_ERROR).
> This will give ownership back to userspace.
> 
> And I don't think vidc_vb2_stop_streaming does that right now.

Me too, I will re-consider it.

> 
>> +	.buf_queue = vidc_vb2_buf_queue,
>> +};
>> +
>> +static int vdec_empty_buf_done(struct hfi_inst *hfi_inst, u32 addr,
>> +			       u32 bytesused, u32 data_offset, u32 flags)
>> +{
>> +	struct vidc_inst *inst = hfi_inst->ops_priv;
>> +	struct vb2_v4l2_buffer *vbuf;
>> +	struct vb2_buffer *vb;
>> +
>> +	vbuf = vidc_vb2_find_buf(inst, addr);
>> +	if (!vbuf)
>> +		return -EINVAL;
>> +
>> +	vb = &vbuf->vb2_buf;
>> +	vbuf->flags = flags;
>> +
>> +	vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_fill_buf_done(struct hfi_inst *hfi_inst, u32 addr,
>> +			      u32 bytesused, u32 data_offset, u32 flags,
>> +			      struct timeval *timestamp)
>> +{
>> +	struct vidc_inst *inst = hfi_inst->ops_priv;
>> +	struct vb2_v4l2_buffer *vbuf;
>> +	struct vb2_buffer *vb;
>> +
>> +	vbuf = vidc_vb2_find_buf(inst, addr);
>> +	if (!vbuf)
>> +		return -EINVAL;
>> +
>> +	vb = &vbuf->vb2_buf;
>> +	vb->planes[0].bytesused = bytesused;
>> +	vb->planes[0].data_offset = data_offset;
>> +	vb->timestamp = timeval_to_ns(timestamp);
>> +	vbuf->flags = flags;
>> +	vbuf->sequence = inst->sequence++;
>> +
>> +	vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
>> +
>> +	if (vbuf->flags & V4L2_BUF_FLAG_LAST) {
>> +		const struct v4l2_event ev = {
> 
> I think this can be static const.

Sure, will do it.

> 
>> +			.type = V4L2_EVENT_EOS
>> +		};
>> +
>> +		v4l2_event_queue_fh(&inst->fh, &ev);
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_event_notify(struct hfi_inst *hfi_inst, u32 event,
>> +			     struct hfi_event_data *data)
>> +{
>> +	struct vidc_inst *inst = hfi_inst->ops_priv;
>> +	struct device *dev = inst->core->dev;
>> +	const struct v4l2_event ev = { .type = V4L2_EVENT_SOURCE_CHANGE };
> 
> I think this can be static const.

Sure, will do it.

> 
>> +
>> +	switch (event) {
>> +	case EVT_SESSION_ERROR:
>> +		if (hfi_inst) {
>> +			mutex_lock(&hfi_inst->lock);
>> +			inst->hfi_inst->state = INST_INVALID;
>> +			mutex_unlock(&hfi_inst->lock);
>> +		}
>> +		dev_err(dev, "dec: event session error (inst:%p)\n", hfi_inst);
>> +		break;
>> +	case EVT_SYS_EVENT_CHANGE:
>> +		switch (data->event_type) {
>> +		case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES:
>> +			dev_dbg(dev, "event sufficient resources\n");
>> +			break;
>> +		case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES:
>> +			inst->reconfig_height = data->height;
>> +			inst->reconfig_width = data->width;
>> +			inst->in_reconfig = true;
>> +
>> +			v4l2_event_queue_fh(&inst->fh, &ev);
>> +
>> +			dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
>> +				data->width, data->height);
>> +			break;
>> +		default:
>> +			break;
>> +		}
>> +		break;
>> +	default:
>> +		break;
>> +	}
>> +
>> +	return 0;
>> +}
>> +

<cut>

>> diff --git a/drivers/media/platform/qcom/vidc/vdec_ctrls.c b/drivers/media/platform/qcom/vidc/vdec_ctrls.c
>> new file mode 100644
>> index 000000000000..59225d8f1fd9
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/vdec_ctrls.c
>> @@ -0,0 +1,200 @@
>> +/*
>> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +#include <linux/types.h>
>> +#include <media/v4l2-ctrls.h>
>> +
>> +#include "core.h"
>> +
>> +static struct vidc_ctrl vdec_ctrls[] = {
>> +	{
>> +		.id = V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
>> +		.type = V4L2_CTRL_TYPE_MENU,
>> +		.min = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
>> +		.max = V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_CODING_EFFICIENCY,
>> +		.def = V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE,
>> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
>> +		.menu_skip_mask = ~(
>> +			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE) |
>> +			(1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE)
>> +		),
>> +	}, {
>> +		.id = V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
>> +		.type = V4L2_CTRL_TYPE_MENU,
>> +		.min = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
>> +		.max = V4L2_MPEG_VIDEO_MPEG4_LEVEL_5,
>> +		.def = V4L2_MPEG_VIDEO_MPEG4_LEVEL_0,
>> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
>> +	}, {
>> +		.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
>> +		.type = V4L2_CTRL_TYPE_MENU,
>> +		.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
>> +		.max = V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH,
>> +		.def = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
>> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
>> +		.menu_skip_mask = ~(
>> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
>> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
>> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
>> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) |
>> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH) |
>> +		(1 << V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH)
>> +		),
>> +	}, {
>> +		.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL,
>> +		.type = V4L2_CTRL_TYPE_MENU,
>> +		.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
>> +		.max = V4L2_MPEG_VIDEO_H264_LEVEL_5_1,
>> +		.def = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
>> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
>> +	}, {
>> +		.id = V4L2_CID_MPEG_VIDEO_VPX_PROFILE,
>> +		.type = V4L2_CTRL_TYPE_INTEGER,
>> +		.min = 0,
>> +		.max = 3,
>> +		.step = 1,
>> +		.def = 0,
>> +		.flags = V4L2_CTRL_FLAG_VOLATILE,
>> +	}, {
>> +		.id = V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER,
>> +		.type = V4L2_CTRL_TYPE_BOOLEAN,
>> +		.def = 0,
>> +	},
>> +};
>> +
>> +#define NUM_CTRLS	ARRAY_SIZE(vdec_ctrls)
>> +
>> +static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
>> +{
>> +	struct vidc_inst *inst = ctrl_to_inst(ctrl);
>> +	struct vdec_controls *ctr = &inst->controls.dec;
>> +
>> +	switch (ctrl->id) {
>> +	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
>> +		ctr->post_loop_deb_mode = ctrl->val;
>> +		break;
>> +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
>> +	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
>> +		ctr->profile = ctrl->val;
>> +		break;
>> +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
>> +		ctr->level = ctrl->val;
>> +		break;
>> +	default:
>> +		return -EINVAL;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
>> +{
>> +	struct vidc_inst *inst = ctrl_to_inst(ctrl);
>> +	struct vdec_controls *ctr = &inst->controls.dec;
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	union hfi_get_property hprop;
>> +	u32 ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
>> +	int ret;
>> +
>> +	switch (ctrl->id) {
>> +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
>> +	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
>> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
>> +						    &hprop);
>> +		if (!ret)
>> +			ctr->profile = hprop.profile_level.profile;
>> +		ctrl->val = ctr->profile;
>> +		break;
>> +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
>> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
>> +						    &hprop);
>> +		if (!ret)
>> +			ctr->level = hprop.profile_level.level;
>> +		ctrl->val = ctr->level;
>> +		break;
>> +	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
>> +		ctrl->val = ctr->post_loop_deb_mode;
>> +		break;
> 
> Why are these volatile?

Because the firmware acording to stream headers that profile and levels
are different.

> 
>> +	default:
>> +		return -EINVAL;
>> +	};
>> +
>> +	return 0;
>> +}
>> +
>> +static const struct v4l2_ctrl_ops vdec_ctrl_ops = {
>> +	.s_ctrl = vdec_op_s_ctrl,
>> +	.g_volatile_ctrl = vdec_op_g_volatile_ctrl,
>> +};
>> +
>> +int vdec_ctrl_init(struct vidc_inst *inst)
>> +{
>> +	unsigned int i;
>> +	int ret;
>> +
>> +	ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, NUM_CTRLS);
>> +	if (ret)
>> +		return ret;
>> +
>> +	for (i = 0; i < NUM_CTRLS; i++) {
>> +		struct v4l2_ctrl *ctrl;
>> +
>> +		if (vdec_ctrls[i].type == V4L2_CTRL_TYPE_MENU) {
>> +			ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
>> +					&vdec_ctrl_ops,
>> +					vdec_ctrls[i].id,
>> +					vdec_ctrls[i].max,
>> +					vdec_ctrls[i].menu_skip_mask,
>> +					vdec_ctrls[i].def);
>> +		} else {
>> +			ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler,
>> +					&vdec_ctrl_ops,
>> +					vdec_ctrls[i].id,
>> +					vdec_ctrls[i].min,
>> +					vdec_ctrls[i].max,
>> +					vdec_ctrls[i].step,
>> +					vdec_ctrls[i].def);
>> +		}
> 
> Why have this vdec_ctrls array at all? Just call v4l2_ctrl_new_std(_menu)
> directly for the controls you want to add. Most drivers do that.

Sure I can give it a try, the only thing I am concerned on is the code size.

<cut>

-- 
regards,
Stan

Re: [PATCH 3/8] media: vidc: decoder: add video decoder files

[Hans Verkuil]

On 08/23/16 14:45, Stanimir Varbanov wrote:
> Hi Hans,
> 
> Thanks for the valuable comments!
> 
> <cut>
> 
>>> +static int vdec_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
>>> +{
>>> +	struct vidc_inst *inst = to_inst(file);
>>> +	struct v4l2_captureparm *cap = &a->parm.capture;
>>> +	struct v4l2_fract *timeperframe = &cap->timeperframe;
>>> +	u64 us_per_frame, fps;
>>> +
>>> +	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
>>> +	    a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>>> +		return -EINVAL;
>>> +
>>> +	memset(cap->reserved, 0, sizeof(cap->reserved));
>>> +	if (!timeperframe->denominator)
>>> +		timeperframe->denominator = inst->timeperframe.denominator;
>>> +	if (!timeperframe->numerator)
>>> +		timeperframe->numerator = inst->timeperframe.numerator;
>>> +	cap->readbuffers = 0;
>>
>> Just set readbuffers to the minimum number of required buffers. Hmm, shouldn't
>> v4l2-compliance complain about a 0 value for readbuffers? Odd.
> 
> I guess I misunderstood the meaning of those fields and just zeroing
> them to make v4l2-compliance happy.

These fields are really left-overs from old times. One of these days I'll
probably remove them. I think that there are still one or two old drivers
that actually use these.

>>> +static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
>>> +{
>>> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
>>> +	struct hfi_core *hfi = &inst->core->hfi;
>>> +	struct device *dev = inst->core->dev;
>>> +	struct hfi_buffer_requirements bufreq;
>>> +	struct hfi_buffer_count_actual buf_count;
>>> +	struct vb2_queue *queue;
>>> +	u32 ptype;
>>> +	int ret;
>>> +
>>> +	switch (q->type) {
>>> +	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>>> +		queue = &inst->bufq_cap;
>>> +		break;
>>> +	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
>>> +		queue = &inst->bufq_out;
>>> +		break;
>>> +	default:
>>
>> If start_streaming fails, then all pending buffers have to be returned by the driver
>> by calling vb2_buffer_done(VB2_BUF_STATE_QUEUED). This will give ownership back to
>> userspace.
> 
> Infact this error path shouldn't be possible, because videobu2-core
> should check q->type before jumping here?

Sorry, I wasn't clear. It is not just this place (and you are right, the error
path here isn't possible), but any place where an error is returned in this
function.

Those should be replaced by a goto fail; and in the fail label all pending buffers
have to be returned. Same code as in stop_streaming, but you call vb2_buffer_done
with state QUEUED instead of state ERROR.

> 
>>
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	if (!vb2_is_streaming(queue))
>>> +		return 0;
>>
>> Can never happen, no need to test for this.
> 
> This can happen cause start_streaming is called for OUTPUT and for
> CAPTURE queues, see the above switch (q->type). If start_streaming is
> called for the CAPTURE thus vb2_is_streaming() checks does
> start_streaming is already called for OUTPUT. So this guarantee that the
> firmware will be started when the streaming is started for both queues.

Ah, now I see it. Please rename 'queue' to 'other_queue' or something like
that. I thought it was the queue that start_streaming was called for, but
that is called 'q'. A bit confusing :-)

> 
>>
>>> +
>>> +	inst->in_reconfig = false;
>>> +	inst->sequence = 0;
>>> +
>>> +	ret = pm_runtime_get_sync(dev);
>>> +	if (ret < 0)
>>> +		return ret;
>>> +
>>> +	ret = vdec_init_session(inst);
>>> +	if (ret)
>>> +		goto put_sync;
>>> +
>>> +	ret = vdec_set_properties(inst);
>>> +	if (ret)
>>> +		goto deinit_sess;
>>> +
>>> +	ret = vdec_check_configuration(inst);
>>> +	if (ret)
>>> +		goto deinit_sess;
>>> +
>>> +	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
>>> +	buf_count.type = HFI_BUFFER_INPUT;
>>> +	buf_count.count_actual = inst->num_input_bufs;
>>> +
>>> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
>>> +					    ptype, &buf_count);
>>> +	if (ret)
>>> +		goto deinit_sess;
>>> +
>>> +	ret = vidc_buf_descs(inst, HFI_BUFFER_OUTPUT, &bufreq);
>>> +	if (ret)
>>> +		goto deinit_sess;
>>> +
>>> +	ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
>>> +	buf_count.type = HFI_BUFFER_OUTPUT;
>>> +	buf_count.count_actual = inst->num_output_bufs;
>>> +
>>> +	ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
>>> +					    ptype, &buf_count);
>>> +	if (ret)
>>> +		goto deinit_sess;
>>> +
>>> +	if (inst->num_output_bufs != bufreq.count_actual) {
>>> +		struct hfi_buffer_display_hold_count_actual display;
>>> +
>>> +		ptype = HFI_PROPERTY_PARAM_BUFFER_DISPLAY_HOLD_COUNT_ACTUAL;
>>> +		display.type = HFI_BUFFER_OUTPUT;
>>> +		display.hold_count = inst->num_output_bufs -
>>> +				     bufreq.count_actual;
>>> +
>>> +		ret = vidc_hfi_session_set_property(hfi, inst->hfi_inst,
>>> +						    ptype, &display);
>>> +		if (ret)
>>> +			goto deinit_sess;
>>> +	}
>>> +
>>> +	ret = vidc_vb2_start_streaming(inst);
>>> +	if (ret)
>>> +		goto deinit_sess;
>>> +
>>> +	return 0;
>>> +
>>> +deinit_sess:
>>> +	vidc_hfi_session_deinit(hfi, inst->hfi_inst);
>>
>> Note that vidc_vb2_start_streaming already calls vidc_hfi_session_deinit on error.
> 
> Probably you wanted to say vdec_init_session has deinit on error path?
> If so I cannot see the problem, vidc_hfi_session_deinit is called only once?

Never mind. I thought vidc_hfi_session_deinit() was called from vidc_vb2_start_streaming,
but it is called from vidc_vb2_stop_streaming. My mistake.

>>> +static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
>>> +{
>>> +	struct vidc_inst *inst = ctrl_to_inst(ctrl);
>>> +	struct vdec_controls *ctr = &inst->controls.dec;
>>> +	struct hfi_core *hfi = &inst->core->hfi;
>>> +	union hfi_get_property hprop;
>>> +	u32 ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
>>> +	int ret;
>>> +
>>> +	switch (ctrl->id) {
>>> +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
>>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
>>> +	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
>>> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
>>> +						    &hprop);
>>> +		if (!ret)
>>> +			ctr->profile = hprop.profile_level.profile;
>>> +		ctrl->val = ctr->profile;
>>> +		break;
>>> +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
>>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
>>> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
>>> +						    &hprop);
>>> +		if (!ret)
>>> +			ctr->level = hprop.profile_level.level;
>>> +		ctrl->val = ctr->level;
>>> +		break;
>>> +	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
>>> +		ctrl->val = ctr->post_loop_deb_mode;
>>> +		break;
>>
>> Why are these volatile?
> 
> Because the firmware acording to stream headers that profile and levels
> are different.

But when these change, isn't the driver told about it? And can these
change midstream? I would expect this to be set once when you start
decoding and not change afterwards.

Regards,

	Hans

Re: [PATCH 2/8] media: vidc: adding core part and helper functions

[Stanimir Varbanov]

Hi Bjorn,

Thanks for the review and comments!

On 08/23/2016 05:50 AM, Bjorn Andersson wrote:
> On Mon 22 Aug 06:13 PDT 2016, Stanimir Varbanov wrote:
> 
> Hi Stan,
> 
>> This adds core part of the vidc driver common helper functions
>> used by encoder and decoder specific files.
> 
> I believe "vidc" is short for "video core" and this is not the only
> "video core" from Qualcomm. This driver is the v4l2 <-> hfi interface and

What other "video core"s do you know?

> uses either two ram based fifos _or_ apr tal for communication with the
> implementation.
> 
> In the case of apr, the other side is not the venus core but rather the
> "VIDC" apr service on the Hexagon DSP. In this case the hfi packets are
> encapsulated in apr packets. Although this is not used in 8916 it would
> be nice to be able to add this later...

OK, you are talking about q6_hfi.c which file is found in msm-3.10 and
maybe older kernel versions.

There is a function vidc_hfi_create() which currently creates venus hfi
interface but it aways could be extended to call q6 DSP specific function.

> 
> 
> But I think we should call this driver "hfi" - or at least venus, as
> it's not compatible with e.g the "blackbird" found in 8064, which is
> also called "vidc".

Do you think that vidc driver for 8064 will ever reach the mainline kernel?

I personally don't like hfi nor venus other suggestions? Does "vidcore"
or "vcore" makes sense?

> 
>>
>>  - core.c has implemented the platform dirver methods, file
>> operations and v4l2 registration.
>>
>>  - helpers.c has implemented common helper functions for
>> buffer management, vb2_ops and functions for format propagation.
>>
>>  - int_bufs.c implements functions for allocating and freeing
>> buffers for internal usage. The buffer parameters describing
>> internal buffers depends on current format, resolution and
>> codec.
>>
>>  - load.c consists functions for calculation of current load
>> of the hardware. Depending on the count of instances and
>> resolutions it selects the best clock rate for the video
>> core.
>>
>>  - mem.c has two functions for memory allocation, currently
>> those functions are used for internal buffers and to allocate
>> the shared memory for communication with firmware via HFI
>> (Host Firmware Interface) interface commands.
> 
> Please drop this; see comments on mem_alloc()

OK.

> 
>>
>>  - resources.c exports a structure describing the details
>> specific to platform and SoC.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
> 
> This doesn't compile, as it depends on later patches. Also there are
> plenty of functions that are related to later patches and would with be
> better to include there, to keep the size of this patch down.
> 
>>  drivers/media/platform/qcom/vidc/core.c      | 548 +++++++++++++++++++++++++++
>>  drivers/media/platform/qcom/vidc/core.h      | 196 ++++++++++
>>  drivers/media/platform/qcom/vidc/helpers.c   | 394 +++++++++++++++++++
>>  drivers/media/platform/qcom/vidc/helpers.h   |  43 +++
>>  drivers/media/platform/qcom/vidc/int_bufs.c  | 325 ++++++++++++++++
>>  drivers/media/platform/qcom/vidc/int_bufs.h  |  23 ++
>>  drivers/media/platform/qcom/vidc/load.c      | 104 +++++
>>  drivers/media/platform/qcom/vidc/load.h      |  22 ++
>>  drivers/media/platform/qcom/vidc/mem.c       |  64 ++++
>>  drivers/media/platform/qcom/vidc/mem.h       |  32 ++
>>  drivers/media/platform/qcom/vidc/resources.c |  46 +++
>>  drivers/media/platform/qcom/vidc/resources.h |  46 +++
>>  12 files changed, 1843 insertions(+)
>>  create mode 100644 drivers/media/platform/qcom/vidc/core.c
>>  create mode 100644 drivers/media/platform/qcom/vidc/core.h
>>  create mode 100644 drivers/media/platform/qcom/vidc/helpers.c
>>  create mode 100644 drivers/media/platform/qcom/vidc/helpers.h
>>  create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.c
>>  create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.h
>>  create mode 100644 drivers/media/platform/qcom/vidc/load.c
>>  create mode 100644 drivers/media/platform/qcom/vidc/load.h
>>  create mode 100644 drivers/media/platform/qcom/vidc/mem.c
>>  create mode 100644 drivers/media/platform/qcom/vidc/mem.h
>>  create mode 100644 drivers/media/platform/qcom/vidc/resources.c
>>  create mode 100644 drivers/media/platform/qcom/vidc/resources.h
>>
>> diff --git a/drivers/media/platform/qcom/vidc/core.c b/drivers/media/platform/qcom/vidc/core.c
>> new file mode 100644
>> index 000000000000..e005be178fc0
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/core.c
>> @@ -0,0 +1,548 @@
>> +/*
>> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +#include <linux/clk.h>
>> +#include <linux/init.h>
>> +#include <linux/ioctl.h>
>> +#include <linux/list.h>
>> +#include <linux/module.h>
>> +#include <linux/of_device.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/slab.h>
>> +#include <linux/types.h>
>> +#include <linux/remoteproc.h>
>> +#include <linux/pm_runtime.h>
>> +#include <media/videobuf2-v4l2.h>
>> +#include <media/v4l2-ioctl.h>
>> +
>> +#include "core.h"
>> +#include "resources.h"
>> +#include "vdec.h"
>> +#include "venc.h"
>> +
>> +static void vidc_add_inst(struct vidc_core *core, struct vidc_inst *inst)
>> +{
>> +	mutex_lock(&core->lock);
>> +	list_add_tail(&inst->list, &core->instances);
> 
> There are two different "instances" lists in this implementation, one
> keeping track of vidc instances and one keeping track of hfi instances,
> at the same time the vidc instances has a reference to its associated
> hfi instance.
> 
> It should be possible to drop one of those lists.

I agree with you. I have thought about this many times during driver
development and it should be possible.

> 
>> +	mutex_unlock(&core->lock);
>> +}
>> +
>> +static void vidc_del_inst(struct vidc_core *core, struct vidc_inst *inst)
>> +{
>> +	struct vidc_inst *pos, *n;
>> +
>> +	mutex_lock(&core->lock);
>> +	list_for_each_entry_safe(pos, n, &core->instances, list) {
>> +		if (pos == inst)
>> +			list_del(&inst->list);
>> +	}
>> +	mutex_unlock(&core->lock);
>> +}
>> +
>> +static int vidc_rproc_boot(struct vidc_core *core)
>> +{
>> +	int ret;
>> +
>> +	if (core->rproc_booted)
>> +		return 0;
> 
> rproc_boot()/rproc_shutdown() is reference counted, so there is no
> reason (other than this driver being buggy) to keep track of
> "rproc_boot". As such, you can drop vidc_rproc_boot() and
> vidc_rproc_shutdown() and just call the rproc functions directly.

You are right, this checks are redundant.

> 
>> +
>> +	ret = rproc_boot(core->rproc);
>> +	if (ret)
>> +		return ret;
>> +
>> +	core->rproc_booted = true;
>> +
>> +	return 0;
>> +}
>> +
>> +static void vidc_rproc_shutdown(struct vidc_core *core)
>> +{
>> +	if (!core->rproc_booted)
>> +		return;
>> +
>> +	rproc_shutdown(core->rproc);
>> +	core->rproc_booted = false;
>> +}
>> +
>> +struct vidc_sys_error {
>> +	struct vidc_core *core;
>> +	struct delayed_work work;
>> +};
> 
> This is cool, but during the 5 second delay we should be able to call
> remove on the driver and this will dereference a freed hfi instance.
> 
> Move the worker to hfi_core and you can cancel it on remove.

OK.

> 
>> +
>> +static void vidc_sys_error_handler(struct work_struct *work)
>> +{
>> +	struct vidc_sys_error *handler =
>> +		container_of(work, struct vidc_sys_error, work.work);
>> +	struct vidc_core *core = handler->core;
>> +	struct hfi_core *hfi = &core->hfi;
>> +	struct device *dev = core->dev;
>> +	int ret;
>> +
>> +	mutex_lock(&hfi->lock);
>> +	if (hfi->state != CORE_INVALID)
>> +		goto exit;
>> +
>> +	mutex_unlock(&hfi->lock);
>> +
>> +	ret = vidc_hfi_core_deinit(hfi);
>> +	if (ret)
>> +		dev_err(dev, "core: deinit failed (%d)\n", ret);
>> +
>> +	mutex_lock(&hfi->lock);
>> +
>> +	rproc_report_crash(core->rproc, RPROC_FATAL_ERROR);
> 
> This operation is async, as such I believe this to be fragile. To get
> the expected result you should be able to simply call
> rproc_shutdown()/rproc_boot() to restart the core...

OK will remove crash report for now.

> 
> However, if we at any point would like to be able to get memory dumps
> from this core (likely a requirement on the Qualcomm side) we need to
> call rproc_report_crash() and let it collect the resources and then
> power cycle the core.
> 
> 
> As the life cycle of the venus driver goes 1:1 with the rproc driver I
> think it would be more suitable to make the v4l driver a child of the
> rproc driver and have it probe/remove this driver as the rproc comes and
> goes. This would allow us to call rproc_report_crash() here, we will be
> removed and when the crash is handled (sometime in the future) we will
> be probed again.

What's the problem with Kconfig "depends on QCOM_VENUS_PIL", isn't that
enough?

> 
>> +
>> +	vidc_rproc_shutdown(core);
>> +
>> +	ret = vidc_rproc_boot(core);
>> +	if (ret)
>> +		goto exit;
>> +
>> +	hfi->state = CORE_INIT;
>> +
>> +exit:
>> +	mutex_unlock(&hfi->lock);
>> +	kfree(handler);
>> +}
>> +
>> +static int vidc_event_notify(struct hfi_core *hfi, u32 event)
>> +{
>> +	struct vidc_sys_error *handler;
>> +	struct hfi_inst *inst;
>> +
>> +	switch (event) {
>> +	case EVT_SYS_WATCHDOG_TIMEOUT:
>> +	case EVT_SYS_ERROR:
>> +		break;
>> +	default:
>> +		return -EINVAL;
>> +	}
>> +
>> +	mutex_lock(&hfi->lock);
>> +
>> +	hfi->state = CORE_INVALID;
>> +
>> +	list_for_each_entry(inst, &hfi->instances, list) {
>> +		mutex_lock(&inst->lock);
>> +		inst->state = INST_INVALID;
>> +		mutex_unlock(&inst->lock);
>> +	}
>> +
>> +	mutex_unlock(&hfi->lock);
>> +
>> +	handler = kzalloc(sizeof(*handler), GFP_KERNEL);
>> +	if (!handler)
>> +		return -ENOMEM;
>> +
>> +	handler->core = container_of(hfi, struct vidc_core, hfi);
>> +	INIT_DELAYED_WORK(&handler->work, vidc_sys_error_handler);
>> +
>> +	/*
>> +	 * Sleep for 5 sec to ensure venus has completed any
>> +	 * pending cache operations. Without this sleep, we see
>> +	 * device reset when firmware is unloaded after a sys
>> +	 * error.
>> +	 */
>> +	schedule_delayed_work(&handler->work, msecs_to_jiffies(5000));
>> +
>> +	return 0;
>> +}
>> +
>> +static const struct hfi_core_ops vidc_core_ops = {
>> +	.event_notify = vidc_event_notify,
>> +};
> 
> This is an overly generic way of calling vidc_sys_error_handler().
> There is no need for having the hfi_core_ops indirections for a single
> op that will only exist in 1 and only 1 variant.

The .event_notify operation is called by hfi part (in hfi_msgs.c) of the
driver and I don't want break the interface. My idea was to have HFI
part and v4l2 part, and each of these parts taking care of their
specifics. The interface between HFI <-> v4l2 should be immutable and
shoudn't be changed when every new version of the hardware IP rise up.

> 
> Just replace the two affected event_notify() calls with a direct call to
> this function (and clean it up a bit).
> 
>> +
>> +static int vidc_open(struct file *file)
>> +{
>> +	struct video_device *vdev = video_devdata(file);
>> +	struct vidc_core *core = video_drvdata(file);
>> +	struct vidc_inst *inst;
>> +	int ret = 0;
>> +
>> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
>> +	if (!inst)
>> +		return -ENOMEM;
>> +
>> +	mutex_init(&inst->lock);
>> +
>> +	INIT_VIDC_LIST(&inst->scratchbufs);
> 
> Please inline the mutex_init() and INIT_LIST_HEAD() here and drop the
> custom INIT_VIDC_LIST() wrapper macro.

OK. I thought I made this already, but seems that I forgot it. Also I
think scratchbufs and persistbufs lists can be merged in one common list.

> 
>> +	INIT_VIDC_LIST(&inst->persistbufs);
>> +	INIT_VIDC_LIST(&inst->registeredbufs);
>> +
>> +	INIT_LIST_HEAD(&inst->bufqueue);
>> +	mutex_init(&inst->bufqueue_lock);
>> +
>> +	if (vdev == &core->vdev_dec)
>> +		inst->session_type = VIDC_SESSION_TYPE_DEC;
>> +	else
>> +		inst->session_type = VIDC_SESSION_TYPE_ENC;
>> +
>> +	inst->core = core;
>> +
>> +	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
>> +		ret = vdec_open(inst);
>> +	else
>> +		ret = venc_open(inst);
>> +
>> +	if (ret)
>> +		goto err_free_inst;
>> +
>> +	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
>> +		v4l2_fh_init(&inst->fh, &core->vdev_dec);
>> +	else
>> +		v4l2_fh_init(&inst->fh, &core->vdev_enc);
> 
> Here we have three sequential conditionals testing for the same thing,
> please join them into one.

OK.

> 
>> +
>> +	inst->fh.ctrl_handler = &inst->ctrl_handler;
>> +
>> +	v4l2_fh_add(&inst->fh);
>> +
>> +	file->private_data = &inst->fh;
>> +
>> +	vidc_add_inst(core, inst);
>> +
>> +	return 0;
>> +
>> +err_free_inst:
>> +	kfree(inst);
>> +	return ret;
>> +}
>> +
>> +static int vidc_close(struct file *file)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +	struct vidc_core *core = inst->core;
>> +
>> +	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
>> +		vdec_close(inst);
>> +	else
>> +		venc_close(inst);
>> +
>> +	vidc_del_inst(core, inst);
>> +
>> +	mutex_destroy(&inst->bufqueue_lock);
>> +	mutex_destroy(&inst->scratchbufs.lock);
>> +	mutex_destroy(&inst->persistbufs.lock);
>> +	mutex_destroy(&inst->registeredbufs.lock);
> 
> Here's a good reason for dropping the INIT_VIDC_LIST() macro

yes indeed :)

> 
>> +
>> +	v4l2_fh_del(&inst->fh);
>> +	v4l2_fh_exit(&inst->fh);
>> +
>> +	kfree(inst);
>> +	return 0;
>> +}
>> +
>> +static unsigned int vidc_poll(struct file *file, struct poll_table_struct *pt)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +	struct vb2_queue *outq = &inst->bufq_out;
>> +	struct vb2_queue *capq = &inst->bufq_cap;
>> +	unsigned int ret;
>> +
>> +	ret = vb2_poll(outq, file, pt);
>> +	ret |= vb2_poll(capq, file, pt);
>> +
>> +	return ret;
>> +}
>> +
>> +static int vidc_mmap(struct file *file, struct vm_area_struct *vma)
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +	unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
>> +	int ret;
>> +
>> +	if (offset < DST_QUEUE_OFF_BASE) {
>> +		ret = vb2_mmap(&inst->bufq_out, vma);
>> +	} else {
>> +		vma->vm_pgoff -= DST_QUEUE_OFF_BASE >> PAGE_SHIFT;
>> +		ret = vb2_mmap(&inst->bufq_cap, vma);
>> +	}
> 
> This feels hackish, is this really the way to do this?

Yes it looks like a hack but there is no other way (to my knowledge),
there are plenty of v4l2 drivers doing like this.

> 
>> +
>> +	return ret;
>> +}
>> +
>> +const struct v4l2_file_operations vidc_fops = {
>> +	.owner = THIS_MODULE,
>> +	.open = vidc_open,
>> +	.release = vidc_close,
>> +	.unlocked_ioctl = video_ioctl2,
>> +	.poll = vidc_poll,
>> +	.mmap = vidc_mmap,
>> +#ifdef CONFIG_COMPAT
>> +	.compat_ioctl32 = v4l2_compat_ioctl32,
>> +#endif
>> +};
>> +
>> +static irqreturn_t vidc_isr_thread(int irq, void *dev_id)
>> +{
>> +	return vidc_hfi_isr_thread(irq, dev_id);
>> +}
>> +
>> +static irqreturn_t vidc_isr(int irq, void *dev)
>> +{
>> +	return vidc_hfi_isr(irq, dev);
>> +}
> 
> These two functions indicates that we're requesting the irq in the wrong
> layer.

IMO the proper place is platform driver .probe method.

> 
> Also, these two functions arrives in a later patchset, so I assume this
> doesn't compile...

That's why I'm adding Makefiles later on patchset. On the other hand I
have splitted the driver by files because I think it is easier for
review. But I might be wrong.

> 
>> +
>> +static int vidc_clks_get(struct vidc_core *core, unsigned int clks_num,
>> +			 const char * const *clks_id)
>> +{
>> +	struct device *dev = core->dev;
>> +	unsigned int i;
>> +
>> +	for (i = 0; i < clks_num; i++) {
>> +		core->clks[i] = devm_clk_get(dev, clks_id[i]);
>> +		if (IS_ERR(core->clks[i]))
>> +			return PTR_ERR(core->clks[i]);
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int
>> +vidc_clks_enable(struct vidc_core *core, const struct vidc_resources *res)
>> +{
>> +	unsigned int i;
>> +	int ret;
>> +
>> +	for (i = 0; i < res->clks_num; i++) {
>> +		ret = clk_prepare_enable(core->clks[i]);
>> +		if (ret)
>> +			goto err;
>> +	}
>> +
>> +	return 0;
>> +err:
>> +	while (--i)
>> +		clk_disable_unprepare(core->clks[i]);
>> +
>> +	return ret;
>> +}
>> +
>> +static void
>> +vidc_clks_disable(struct vidc_core *core, const struct vidc_resources *res)
>> +{
>> +	unsigned int i;
>> +
>> +	for (i = 0; i < res->clks_num; i++)
>> +		clk_disable_unprepare(core->clks[i]);
>> +}
>> +
>> +static const struct of_device_id vidc_dt_match[] = {
>> +	{ .compatible = "qcom,vidc-msm8916", .data = &msm8916_res, },
>> +	{ }
>> +};
>> +
>> +MODULE_DEVICE_TABLE(of, vidc_dt_match);
> 
> As you're using of_device_get_match_data() you can move this table to
> the bottom of the file.

OK.

> 
>> +
>> +static int vidc_probe(struct platform_device *pdev)
>> +{
>> +	struct device *dev = &pdev->dev;
>> +	struct vidc_core *core;
>> +	struct device_node *rproc;
>> +	struct resource *r;
>> +	int ret;
>> +
>> +	core = devm_kzalloc(dev, sizeof(*core), GFP_KERNEL);
>> +	if (!core)
>> +		return -ENOMEM;
>> +
>> +	core->dev = dev;
>> +	platform_set_drvdata(pdev, core);
>> +
>> +	rproc = of_parse_phandle(dev->of_node, "rproc", 0);
>> +	if (IS_ERR(rproc))
>> +		return PTR_ERR(rproc);
>> +
>> +	core->rproc = rproc_get_by_phandle(rproc->phandle);
> 
> FYI, We're hoping to land some patches shortly that will replace this
> with rproc_get(pdev->dev.of_node), looking up an rproc by the standard
> "rprocs" property...

OK, that looks good. But shoudn't be rproc_get(pdev->dev)?

> 
>> +	if (IS_ERR(core->rproc))
>> +		return PTR_ERR(core->rproc);
>> +	else if (!core->rproc)
>> +		return -EPROBE_DEFER;
> 
> We're cleaning up this in the core as well.
> 
> You need to rproc_put() the rproc pointer after this point.

OK, good. When those changes landed I will rework this part.

> 
> 
> My question still stands though, if this driver should be probed as the
> remoteproc is booted (or the apr service appearing). I will continue to
> look at that.

I might be misunderstood your point here. Is your concern related to
EPROBE_DEFFER or some sort of ordering issue in rproc?

Currently the vidc depends on QCOM_VENUS_PIL in Kconfig, so modprobe
vidc should modprobe remoteproc driver cause it depends on it.

> 
>> +
>> +	r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>> +	core->base = devm_ioremap_resource(dev, r);
>> +	if (IS_ERR(core->base))
>> +		return PTR_ERR(core->base);
>> +
>> +	core->irq = platform_get_irq(pdev, 0);
>> +	if (core->irq < 0)
>> +		return core->irq;
>> +
>> +	core->res = of_device_get_match_data(dev);
>> +	if (!core->res)
>> +		return -ENODEV;
>> +
>> +	ret = vidc_clks_get(core, core->res->clks_num, core->res->clks);
>> +	if (ret)
>> +		return ret;
>> +
>> +	ret = dma_set_mask_and_coherent(dev, core->res->dma_mask);
>> +	if (ret)
>> +		return ret;
>> +
>> +	INIT_LIST_HEAD(&core->instances);
>> +	mutex_init(&core->lock);
>> +
>> +	ret = devm_request_threaded_irq(dev, core->irq, vidc_isr,
>> +					vidc_isr_thread,
>> +					IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
> 
> Drop this IRQF_TRIGGER_HIGH and have this be specified in devicetree.

OK can do that, but is there a convention about who is populating the
flag and what is the precedence dt against the driver?

> 
>> +					"vidc", &core->hfi);
>> +	if (ret)
>> +		return ret;
>> +
>> +	core->hfi.core_ops = &vidc_core_ops;
>> +	core->hfi.dev = dev;
>> +
>> +	ret = vidc_hfi_create(&core->hfi, core->res, core->base);
>> +	if (ret)
>> +		return ret;
>> +
>> +	ret = vidc_clks_enable(core, core->res);
>> +	if (ret)
>> +		goto err_hfi_destroy;
>> +
>> +	ret = vidc_rproc_boot(core);
>> +	if (ret) {
>> +		vidc_clks_disable(core, core->res);
>> +		goto err_hfi_destroy;
>> +	}
>> +
>> +	pm_runtime_enable(dev);
>> +
>> +	ret = pm_runtime_get_sync(dev);
>> +	if (ret < 0)
>> +		goto err_runtime_disable;
>> +
>> +	ret = vidc_hfi_core_init(&core->hfi);
>> +	if (ret)
>> +		goto err_rproc_shutdown;
>> +
>> +	ret = pm_runtime_put_sync(dev);
>> +	if (ret)
>> +		goto err_core_deinit;
>> +
>> +	vidc_clks_disable(core, core->res);
> 
> These operations follow the general pattern of booting other Qualcomm
> remoteprocs; acquire and enable some resources, boot the core and
> disable the resources. Therefor it looks quite likely that these
> operations are related to the life cycle of the venus core, rather than
> hfi.

So you saying that

> 
>> +
>> +	ret = v4l2_device_register(dev, &core->v4l2_dev);
>> +	if (ret)
>> +		goto err_core_deinit;
>> +
>> +	ret = vdec_init(core, &core->vdev_dec);
>> +	if (ret)
>> +		goto err_dev_unregister;
>> +
>> +	ret = venc_init(core, &core->vdev_enc);
>> +	if (ret)
>> +		goto err_vdec_deinit;
>> +
>> +	return 0;
>> +
>> +err_vdec_deinit:
>> +	vdec_deinit(core, &core->vdev_dec);
>> +err_dev_unregister:
>> +	v4l2_device_unregister(&core->v4l2_dev);
>> +err_core_deinit:
>> +	vidc_hfi_core_deinit(&core->hfi);
>> +err_rproc_shutdown:
>> +	vidc_rproc_shutdown(core);
>> +err_runtime_disable:
>> +	pm_runtime_set_suspended(dev);
>> +	pm_runtime_disable(dev);
>> +err_hfi_destroy:
>> +	vidc_hfi_destroy(&core->hfi);
>> +	return ret;
>> +}
>> +
>> +static int vidc_remove(struct platform_device *pdev)
>> +{
>> +	struct vidc_core *core = platform_get_drvdata(pdev);
>> +	int ret;
>> +
>> +	ret = pm_runtime_get_sync(&pdev->dev);
>> +	if (ret < 0)
>> +		return ret;
> 
> No-one cares about you returning an error here, so you better move
> forward and release as much of your resources as possible even though
> you didn't get your pm.

Hmm, I don't agree here. The runtime_resume will enable clocks (for
example venus iface clk) and if it fails the subsequent call to
vidc_rproc_shutdown can crash badly.

> 
>> +
>> +	ret = vidc_hfi_core_deinit(&core->hfi);
>> +	if (ret) {
>> +		pm_runtime_put_sync(&pdev->dev);
>> +		return ret;
>> +	}
>> +
>> +	vidc_rproc_shutdown(core);
>> +
>> +	ret = pm_runtime_put_sync(&pdev->dev);
>> +
>> +	vidc_hfi_destroy(&core->hfi);
>> +	vdec_deinit(core, &core->vdev_dec);
>> +	venc_deinit(core, &core->vdev_enc);
>> +	v4l2_device_unregister(&core->v4l2_dev);
>> +
>> +	pm_runtime_disable(core->dev);
>> +
>> +	return ret < 0 ? ret : 0;
>> +}
>> +
>> +static int vidc_runtime_suspend(struct device *dev)
>> +{
>> +	struct vidc_core *core = dev_get_drvdata(dev);
>> +	int ret;
>> +
>> +	ret = vidc_hfi_core_suspend(&core->hfi);
>> +
>> +	vidc_clks_disable(core, core->res);
>> +
>> +	return ret;
>> +}
>> +
>> +static int vidc_runtime_resume(struct device *dev)
>> +{
>> +	struct vidc_core *core = dev_get_drvdata(dev);
>> +	int ret;
>> +
>> +	ret = vidc_clks_enable(core, core->res);
>> +	if (ret)
>> +		return ret;
>> +
>> +	return vidc_hfi_core_resume(&core->hfi);
>> +}
>> +
>> +static int vidc_pm_suspend(struct device *dev)
>> +{
>> +	return vidc_runtime_suspend(dev);
>> +}
>> +
>> +static int vidc_pm_resume(struct device *dev)
>> +{
>> +	return vidc_runtime_resume(dev);
>> +}
>> +
>> +static const struct dev_pm_ops vidc_pm_ops = {
>> +	SET_SYSTEM_SLEEP_PM_OPS(vidc_pm_suspend, vidc_pm_resume)
>> +	SET_RUNTIME_PM_OPS(vidc_runtime_suspend, vidc_runtime_resume, NULL)
>> +};
>> +
>> +static struct platform_driver qcom_vidc_driver = {
>> +	.probe = vidc_probe,
>> +	.remove = vidc_remove,
>> +	.driver = {
>> +		.name = "qcom-vidc",
>> +		.of_match_table = vidc_dt_match,
>> +		.pm = &vidc_pm_ops,
>> +	},
>> +};
>> +
>> +module_platform_driver(qcom_vidc_driver);
>> +
>> +MODULE_ALIAS("platform:qcom-vidc");
>> +MODULE_DESCRIPTION("Qualcomm video encoder and decoder driver");
>> +MODULE_LICENSE("GPL v2");
>> diff --git a/drivers/media/platform/qcom/vidc/core.h b/drivers/media/platform/qcom/vidc/core.h
>> new file mode 100644
>> index 000000000000..5dc8e05f8c36
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/core.h
>> @@ -0,0 +1,196 @@
>> +/*
>> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +
>> +#ifndef __VIDC_CORE_H_
>> +#define __VIDC_CORE_H_
>> +
>> +#include <media/v4l2-device.h>
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/videobuf2-core.h>
>> +
>> +#include "resources.h"
>> +#include "hfi.h"
>> +
>> +#define VIDC_DRV_NAME		"vidc"
> 
> Unused

used in vdec.c and enc.c. But I can delete this in next submission.

> 
>> +
>> +struct vidc_list {
>> +	struct list_head list;
>> +	struct mutex lock;
>> +};
> 
> Can't we get away without passing around lockable lists? Does these
> lists have to be locked independently and should we really pass around
> their lock with them?

I guess it is possible but didn't spent to much time on that (I had more
important problems to solve with downstream driver). So the answer is
yes and I have to re-consider it.

> 
>> +
>> +struct vidc_format {
>> +	u32 pixfmt;
>> +	int num_planes;
>> +	u32 type;
>> +};
>> +
>> +struct vidc_core {
>> +	struct list_head list;
> 
> This list_head seems unused.

Yes, it is.

> 
>> +	void __iomem *base;
> 
> base is acquired and passed by value to vidc_hfi_create(), so no need to
> keep track of it here.
> 
>> +	int irq;
> 
> This irq belongs to hfi, so it should probably be kept there.

Sure, will move those two.

> 
>> +	struct clk *clks[VIDC_CLKS_NUM_MAX];
>> +	struct mutex lock;
> 
> This "lock" seems to be only related the instances list, please name it
> more appropriately - and place it next to the instances member.

OK.

> 
>> +	struct hfi_core hfi;
>> +	struct video_device vdev_dec;
>> +	struct video_device vdev_enc;
>> +	struct v4l2_device v4l2_dev;
>> +	struct list_head instances;
>> +	const struct vidc_resources *res;
>> +	struct rproc *rproc;
>> +	bool rproc_booted;
>> +	struct device *dev;
>> +};
>> +
>> +struct vdec_controls {
>> +	u32 post_loop_deb_mode;
>> +	u32 profile;
>> +	u32 level;
>> +};
>> +
>> +struct venc_controls {
>> +	u16 gop_size;
>> +	u32 idr_period;
>> +	u32 num_p_frames;
>> +	u32 num_b_frames;
>> +	u32 bitrate_mode;
>> +	u32 bitrate;
>> +	u32 bitrate_peak;
>> +
>> +	u32 h264_i_period;
>> +	u32 h264_entropy_mode;
>> +	u32 h264_i_qp;
>> +	u32 h264_p_qp;
>> +	u32 h264_b_qp;
>> +	u32 h264_min_qp;
>> +	u32 h264_max_qp;
>> +	u32 h264_loop_filter_mode;
>> +	u32 h264_loop_filter_alpha;
>> +	u32 h264_loop_filter_beta;
>> +
>> +	u32 vp8_min_qp;
>> +	u32 vp8_max_qp;
>> +
>> +	u32 multi_slice_mode;
>> +	u32 multi_slice_max_bytes;
>> +	u32 multi_slice_max_mb;
>> +
>> +	u32 header_mode;
>> +
>> +	u32 profile;
>> +	u32 level;
>> +};
>> +
>> +struct vidc_inst {
>> +	struct list_head list;
>> +	struct mutex lock;
>> +	struct vidc_core *core;
>> +
>> +	struct vidc_list scratchbufs;
>> +	struct vidc_list persistbufs;
>> +	struct vidc_list registeredbufs;
> 
> Just inline the list_head and mutex here, as it's done for bufqueue.

OK.

>> +
>> +	struct list_head bufqueue;
>> +	struct mutex bufqueue_lock;
>> +
>> +	int streamoff;
>> +	int streamon;
>> +	struct vb2_queue bufq_out;
>> +	struct vb2_queue bufq_cap;
>> +
>> +	struct v4l2_ctrl_handler ctrl_handler;
>> +	union {
>> +		struct vdec_controls dec;
>> +		struct venc_controls enc;
>> +	} controls;
>> +	struct v4l2_fh fh;
>> +
>> +	struct hfi_inst *hfi_inst;
>> +
>> +	/* session fields */
>> +	u32 session_type;
>> +	u32 width;
>> +	u32 height;
>> +	u32 out_width;
>> +	u32 out_height;
>> +	u32 colorspace;
>> +	u8 ycbcr_enc;
>> +	u8 quantization;
>> +	u8 xfer_func;
>> +	u64 fps;
>> +	struct v4l2_fract timeperframe;
>> +	const struct vidc_format *fmt_out;
>> +	const struct vidc_format *fmt_cap;
>> +	unsigned int num_input_bufs;
>> +	unsigned int num_output_bufs;
>> +	bool in_reconfig;
>> +	u32 reconfig_width;
>> +	u32 reconfig_height;
>> +	u64 sequence;
>> +};
>> +
>> +#define ctrl_to_inst(ctrl)	\
>> +	container_of(ctrl->handler, struct vidc_inst, ctrl_handler)
>> +
>> +struct vidc_ctrl {
>> +	u32 id;
>> +	enum v4l2_ctrl_type type;
>> +	s32 min;
>> +	s32 max;
>> +	s32 def;
>> +	u32 step;
>> +	u64 menu_skip_mask;
>> +	u32 flags;
>> +	const char * const *qmenu;
>> +};
>> +
>> +/*
>> + * Offset base for buffers on the destination queue - used to distinguish
>> + * between source and destination buffers when mmapping - they receive the same
>> + * offsets but for different queues
>> + */
>> +#define DST_QUEUE_OFF_BASE	(1 << 30)
>> +
>> +extern const struct v4l2_file_operations vidc_fops;
> 
> Just pass this to v{dec,enc}_init() rather than back-referencing it
> through a global variable. But on the other hand this is unused in this
> patchset...

Very nice idea, thanks.

> 
>> +
>> +static inline void INIT_VIDC_LIST(struct vidc_list *mlist)
>> +{
>> +	mutex_init(&mlist->lock);
>> +	INIT_LIST_HEAD(&mlist->list);
>> +}
>> +
>> +static inline struct vidc_inst *to_inst(struct file *filp)
>> +{
>> +	return container_of(filp->private_data, struct vidc_inst, fh);
>> +}
>> +
>> +static inline struct hfi_inst *to_hfi_inst(struct file *filp)
> 
> Unused

Unsed in this patch, but used by vdec.c and enc.c in subsequent patches.

> 
>> +{
>> +	return to_inst(filp)->hfi_inst;
>> +}
>> +
>> +static inline struct vb2_queue *
>> +vidc_to_vb2q(struct file *file, enum v4l2_buf_type type)
> 
> Unused

Same as above comment.

> 
>> +{
>> +	struct vidc_inst *inst = to_inst(file);
>> +
>> +	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
>> +		return &inst->bufq_cap;
>> +	else if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> +		return &inst->bufq_out;
>> +
>> +	return NULL;
>> +}
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/vidc/helpers.c b/drivers/media/platform/qcom/vidc/helpers.c
>> new file mode 100644
>> index 000000000000..81079f2b5ed1
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/helpers.c
>> @@ -0,0 +1,394 @@
>> +/*
>> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +#include <linux/list.h>
>> +#include <linux/mutex.h>
>> +#include <linux/pm_runtime.h>
>> +#include <media/videobuf2-dma-sg.h>
>> +
>> +#include "helpers.h"
>> +#include "int_bufs.h"
>> +#include "load.h"
>> +#include "hfi_helper.h"
>> +
>> +static int session_set_buf(struct vb2_buffer *vb)
>> +{
>> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>> +	struct vb2_queue *q = vb->vb2_queue;
>> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
>> +	struct vidc_core *core = inst->core;
>> +	struct device *dev = core->dev;
>> +	struct hfi_core *hfi = &core->hfi;
>> +	struct vidc_buffer *buf = to_vidc_buffer(vbuf);
>> +	struct hfi_frame_data fdata;
>> +	int ret;
>> +
>> +	memset(&fdata, 0, sizeof(fdata));
>> +
>> +	fdata.alloc_len = vb2_plane_size(vb, 0);
>> +	fdata.device_addr = buf->dma_addr;
>> +	fdata.timestamp = vb->timestamp;
>> +	fdata.flags = 0;
>> +	fdata.clnt_data = buf->dma_addr;
>> +
>> +	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
>> +		fdata.buffer_type = HFI_BUFFER_INPUT;
>> +		fdata.filled_len = vb2_get_plane_payload(vb, 0);
>> +		fdata.offset = vb->planes[0].data_offset;
>> +
>> +		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
>> +			fdata.flags |= HFI_BUFFERFLAG_EOS;
>> +
>> +		ret = vidc_hfi_session_etb(hfi, inst->hfi_inst, &fdata);
>> +	} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>> +		fdata.buffer_type = HFI_BUFFER_OUTPUT;
>> +		fdata.filled_len = 0;
>> +		fdata.offset = 0;
>> +
>> +		ret = vidc_hfi_session_ftb(hfi, inst->hfi_inst, &fdata);
>> +	} else {
>> +		ret = -EINVAL;
>> +	}
>> +
>> +	if (ret) {
>> +		dev_err(dev, "failed to set session buffer (%d)\n", ret);
>> +		return ret;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int session_unregister_bufs(struct vidc_inst *inst)
>> +{
>> +	struct device *dev = inst->core->dev;
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	struct hfi_buffer_desc *bd;
>> +	struct vidc_buffer *buf, *tmp;
>> +	int ret = 0;
>> +
>> +	mutex_lock(&inst->registeredbufs.lock);
>> +	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
>> +				 hfi_list) {
>> +		list_del(&buf->hfi_list);
> 
> So the hfi_list is the list_head for entries in the _vidc_ instance
> list?

yes, registeredbufs.list is used to keep track of the all buffers that
will be used during the life-cycle of the current session, i.e. the
firmware wants to know all buffer addresses before calling
session_start. On the other side bufqueue list is used for v4l2
queue/dequeue side.

> 
>> +		bd = &buf->bd;
>> +		bd->response_required = 1;
>> +		ret = vidc_hfi_session_unset_buffers(hfi, inst->hfi_inst, bd);
>> +		if (ret) {
>> +			dev_err(dev, "%s: session release buffers failed\n",
>> +				__func__);
>> +			break;
>> +		}
>> +	}
>> +	mutex_unlock(&inst->registeredbufs.lock);
>> +
>> +	return ret;
>> +}
>> +
>> +static int session_register_bufs(struct vidc_inst *inst)
>> +{
>> +	struct device *dev = inst->core->dev;
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	struct hfi_buffer_desc *bd;
>> +	struct vidc_buffer *buf, *tmp;
>> +	int ret = 0;
>> +
>> +	mutex_lock(&inst->registeredbufs.lock);
>> +	list_for_each_entry_safe(buf, tmp, &inst->registeredbufs.list,
>> +				 hfi_list) {
>> +		bd = &buf->bd;
>> +		ret = vidc_hfi_session_set_buffers(hfi, inst->hfi_inst, bd);
>> +		if (ret) {
>> +			dev_err(dev, "%s: session: set buffer failed\n",
>> +				__func__);
>> +			break;
>> +		}
>> +	}
>> +	mutex_unlock(&inst->registeredbufs.lock);
>> +
>> +	return ret;
>> +}
>> +
>> +int vidc_buf_descs(struct vidc_inst *inst, u32 type,
>> +		   struct hfi_buffer_requirements *out)
> 
> If you call this vidc_get_buf_requirements() it would actually describe
> what's going on. But why is this hfi wrapper in the core, rather than

the original name of this function was similar to what you suggest but I
decided in last-minute cleanup to shorten its name.

> just have the internal buffer manager call it directly.

It is in the core cause I used it on few places to gather buffer count
needed depending on parameters (resolution, codec). Good example is
vb2_ops::queue_setup where I need to return num_buffers depending on
resolution, codec, bitrate, framerate and so on.

> 
> The call doesn't seem to depend on the parameters or state, can we
> cache the result?

No, we cannot. see above comment. Something more the scratch and
prersist buffer sizes can also be changed by the firmware depending on
above proparties.

> 
>> +{
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	u32 ptype = HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS;
>> +	union hfi_get_property hprop;
>> +	int ret, i;
>> +
>> +	if (out)
>> +		memset(out, 0, sizeof(*out));
>> +
>> +	ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype, &hprop);
>> +	if (ret)
>> +		return ret;
>> +
>> +	ret = -EINVAL;
>> +
>> +	for (i = 0; i < HFI_BUFFER_TYPE_MAX; i++) {
>> +		if (hprop.bufreq[i].type != type)
>> +			continue;
>> +
>> +		if (out)
>> +			memcpy(out, &hprop.bufreq[i], sizeof(*out));
>> +		ret = 0;
>> +		break;
>> +	}
>> +
>> +	return ret;
>> +}
>> +
> [..]
>> +
>> +void vidc_vb2_stop_streaming(struct vb2_queue *q)
>> +{
>> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
>> +	struct hfi_inst *hfi_inst = inst->hfi_inst;
>> +	struct vidc_core *core = inst->core;
>> +	struct device *dev = core->dev;
>> +	struct hfi_core *hfi = &core->hfi;
>> +	int ret, streamoff;
>> +
>> +	mutex_lock(&inst->lock);
>> +	streamoff = inst->streamoff;
>> +	mutex_unlock(&inst->lock);
>> +
>> +	if (streamoff)
>> +		return;
>> +
>> +	mutex_lock(&inst->lock);
>> +	if (inst->streamon == 0) {
>> +		mutex_unlock(&inst->lock);
>> +		return;
>> +	}
>> +	mutex_unlock(&inst->lock);
> 
> Why do we keep track of streamon and stream off, why isn't streamoff
> ever cleared? Why don't we check both conditions in one critical region?

Probably cause its buggy, I will sort it out.

> 
>> +
>> +	ret = vidc_hfi_session_stop(hfi, inst->hfi_inst);
>> +	if (ret) {
>> +		dev_err(dev, "session: stop failed (%d)\n", ret);
>> +		goto abort;
> 
> When are we going to relaim the buffers in these cases?

session_stop will instruct the firmware return buffers to the v4l2
driver through hfi_inst_ops empty_buf_done and fill_buf_done, those
operations will call vb2_buffer_done.

> 
>> +	}
>> +
>> +	ret = vidc_hfi_session_unload_res(hfi, inst->hfi_inst);
>> +	if (ret) {
>> +		dev_err(dev, "session: release resources failed (%d)\n", ret);
>> +		goto abort;
>> +	}
>> +
>> +	ret = session_unregister_bufs(inst);
>> +	if (ret) {
>> +		dev_err(dev, "failed to release capture buffers: %d\n", ret);
>> +		goto abort;
>> +	}
>> +
>> +	ret = internal_bufs_free(inst);
>> +
>> +	if (hfi_inst->state == INST_INVALID || hfi->state == CORE_INVALID) {
>> +		ret = -EINVAL;
>> +		goto abort;
>> +	}
>> +
>> +abort:
>> +	if (ret)
>> +		vidc_hfi_session_abort(hfi, inst->hfi_inst);
>> +
>> +	vidc_scale_clocks(inst->core);
>> +
>> +	ret = vidc_hfi_session_deinit(hfi, inst->hfi_inst);
>> +
>> +	mutex_lock(&inst->lock);
>> +	inst->streamoff = 1;
>> +	mutex_unlock(&inst->lock);
>> +
>> +	if (ret)
>> +		dev_err(dev, "stop streaming failed type: %d, ret: %d\n",
>> +			q->type, ret);
>> +
>> +	ret = pm_runtime_put_sync(dev);
>> +	if (ret < 0)
>> +		dev_err(dev, "%s: pm_runtime_put_sync (%d)\n", __func__, ret);
>> +}
>> +
>> +int vidc_vb2_start_streaming(struct vidc_inst *inst)
>> +{
>> +	struct device *dev = inst->core->dev;
>> +	struct hfi_core *hfi = &inst->core->hfi;
>> +	struct vidc_buffer *buf, *n;
>> +	int ret;
>> +
>> +	ret = session_register_bufs(inst);
>> +	if (ret)
>> +		return ret;
>> +
>> +	ret = internal_bufs_alloc(inst);
>> +	if (ret)
>> +		return ret;
>> +
>> +	vidc_scale_clocks(inst->core);
>> +
>> +	ret = vidc_hfi_session_load_res(hfi, inst->hfi_inst);
>> +	if (ret) {
>> +		dev_err(dev, "session: load resources (%d)\n", ret);
>> +		return ret;
>> +	}
>> +
>> +	ret = vidc_hfi_session_start(hfi, inst->hfi_inst);
>> +	if (ret) {
>> +		dev_err(dev, "session: start failed (%d)\n", ret);
>> +		return ret;
>> +	}
>> +
>> +	mutex_lock(&inst->bufqueue_lock);
>> +	list_for_each_entry_safe(buf, n, &inst->bufqueue, list) {
>> +		ret = session_set_buf(&buf->vb.vb2_buf);
>> +		if (ret)
>> +			break;
>> +	}
>> +	mutex_unlock(&inst->bufqueue_lock);
>> +
>> +	if (!ret) {
>> +		mutex_lock(&inst->lock);
>> +		inst->streamon = 1;
>> +		mutex_unlock(&inst->lock);
>> +	}
>> +
>> +	return ret;
>> +}
>> diff --git a/drivers/media/platform/qcom/vidc/helpers.h b/drivers/media/platform/qcom/vidc/helpers.h
>> new file mode 100644
>> index 000000000000..a151c96bf939
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/helpers.h
>> @@ -0,0 +1,43 @@
>> +/*
>> + * Copyright (c) 2012-2014, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +#ifndef __VIDC_COMMON_H__
>> +#define __VIDC_COMMON_H__
> 
> s/COMMON/HELPERS/
> 
>> +
>> +#include <linux/list.h>
>> +#include <media/videobuf2-v4l2.h>
>> +
>> +#include "core.h"
>> +
>> +struct vidc_buffer {
>> +	struct vb2_v4l2_buffer vb;
>> +	struct list_head list;
>> +	dma_addr_t dma_addr;
>> +	struct list_head hfi_list;
> 
> This seems to be the list_head used for associating buffers to the
> _vidc_ instances.
> 
>> +	struct hfi_buffer_desc bd;
>> +};
>> +
>> +#define to_vidc_buffer(buf)	container_of(buf, struct vidc_buffer, vb)
>> +
>> +struct vb2_v4l2_buffer *
>> +vidc_vb2_find_buf(struct vidc_inst *inst, dma_addr_t addr);
>> +int vidc_vb2_buf_init(struct vb2_buffer *vb);
>> +int vidc_vb2_buf_prepare(struct vb2_buffer *vb);
>> +void vidc_vb2_buf_queue(struct vb2_buffer *vb);
>> +void vidc_vb2_stop_streaming(struct vb2_queue *q);
>> +int vidc_vb2_start_streaming(struct vidc_inst *inst);
>> +int vidc_buf_descs(struct vidc_inst *inst, u32 type,
>> +		   struct hfi_buffer_requirements *out);
>> +int vidc_set_color_format(struct vidc_inst *inst, u32 type, u32 fmt);
>> +#endif
>> diff --git a/drivers/media/platform/qcom/vidc/int_bufs.c b/drivers/media/platform/qcom/vidc/int_bufs.c
> [..]
>> +
>> +static int internal_alloc_and_set(struct vidc_inst *inst,
>> +				  struct hfi_buffer_requirements *bufreq,
>> +				  struct vidc_list *buf_list)
>> +{
>> +	struct vidc_internal_buf *buf;
>> +	struct vidc_mem *mem;
>> +	unsigned int i;
>> +	int ret = 0;
>> +
>> +	if (!bufreq->size)
>> +		return 0;
>> +
>> +	for (i = 0; i < bufreq->count_actual; i++) {
>> +		mem = mem_alloc(inst->core->dev, bufreq->size, 0);
> 
> Inline mem_alloc here; might need to make sure bufreq->size is 4K
> aligned.

OK, I will give it a try.

> 
>> +		if (IS_ERR(mem)) {
>> +			ret = PTR_ERR(mem);
>> +			goto err_no_mem;
>> +		}
>> +
>> +		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
>> +		if (!buf) {
>> +			ret = -ENOMEM;
>> +			goto fail_kzalloc;
>> +		}
>> +
>> +		buf->mem = mem;
>> +		buf->type = bufreq->type;
>> +
>> +		ret = internal_set_buf_on_fw(inst, bufreq->type, mem, false);
>> +		if (ret)
>> +			goto fail_set_buffers;
>> +
>> +		mutex_lock(&buf_list->lock);
>> +		list_add_tail(&buf->list, &buf_list->list);
>> +		mutex_unlock(&buf_list->lock);
>> +	}
>> +
>> +	return ret;
>> +
>> +fail_set_buffers:
>> +	kfree(buf);
>> +fail_kzalloc:
>> +	mem_free(mem);
>> +err_no_mem:
>> +	return ret;
>> +}
>> +
> [..]
>> +
>> +static int persist_set_buffer(struct vidc_inst *inst, u32 type)
>> +{
>> +	struct hfi_buffer_requirements bufreq;
>> +	int ret;
>> +
>> +	ret = vidc_buf_descs(inst, type, &bufreq);
>> +	if (ret)
>> +		return 0;
>> +
>> +	mutex_lock(&inst->persistbufs.lock);
>> +	if (!list_empty(&inst->persistbufs.list)) {
> 
> This function is called twice, with type HFI_BUFFER_INTERNAL_PERSIST and
> HFI_BUFFER_INTERNAL_PERSIST_1 respectively. Unless the buffer
> requirements are missing for HFI_BUFFER_INTERNAL_PERSIST persistbufs
> won't be empty and we will skip the later allocation.
> 
>> +		mutex_unlock(&inst->persistbufs.lock);
>> +		return 0;
>> +	}
>> +	mutex_unlock(&inst->persistbufs.lock);
>> +
>> +	return internal_alloc_and_set(inst, &bufreq, &inst->persistbufs);
>> +}
>> +
> [..]
>> +
>> +static int scratch_set_buffers(struct vidc_inst *inst)
>> +{
>> +	struct device *dev = inst->core->dev;
>> +	int ret;
>> +
>> +	ret = scratch_unset_buffers(inst, true);
>> +	if (ret)
>> +		dev_warn(dev, "Failed to release scratch buffers\n");
> 
> internal_bufs_free() calls scratch_unset_buffers(reuse=false) so we're
> coming here with an empty scratchbufs either way - meaning that this
> whole file can be greatly simplified.
> 
> So instead of trying to fix that I would suggest that you just let
> internal_bufs_alloc() acquire the buffer requirements and call
> internal_alloc_and_set() directly, storing the result in a single list.
> 
> And then inline a free method in internal_bufs_free() as well as drop
> all reuse-stuff and unused/dead code.
> 
> That would simplify this file quite a bit and if there actually is a
> need for the reusing of buffer that can be added at some later time.
> 

Actially I thought about droping the reuse stuff in the past, so I agree
on that cleanup. The thing which worries me is the size of those buffers
(the biggest is 10-15MB) and also the allocation time. Currently those
buffers are allocate on streamon time, but probably the right place is
on request_buf time.

>> +
>> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH);
>> +	if (ret)
>> +		goto error;
>> +
>> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_1);
>> +	if (ret)
>> +		goto error;
>> +
>> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_2);
>> +	if (ret)
>> +		goto error;
>> +
>> +	return 0;
>> +error:
>> +	scratch_unset_buffers(inst, false);
>> +	return ret;
>> +}
>> +
>> +static int persist_set_buffers(struct vidc_inst *inst)
>> +{
>> +	int ret;
>> +
>> +	ret = persist_set_buffer(inst, HFI_BUFFER_INTERNAL_PERSIST);
>> +	if (ret)
>> +		goto error;
>> +
>> +	ret = persist_set_buffer(inst, HFI_BUFFER_INTERNAL_PERSIST_1);
>> +	if (ret)
>> +		goto error;
>> +
>> +	return 0;
>> +
>> +error:
>> +	persist_unset_buffers(inst);
>> +	return ret;
>> +}
>> +
>> +int internal_bufs_alloc(struct vidc_inst *inst)
>> +{
>> +	struct device *dev = inst->core->dev;
>> +	int ret;
>> +
>> +	ret = scratch_set_buffers(inst);
>> +	if (ret) {
>> +		dev_err(dev, "set scratch buffers (%d)\n", ret);
>> +		return ret;
>> +	}
>> +
>> +	ret = persist_set_buffers(inst);
>> +	if (ret) {
>> +		dev_err(dev, "set persist buffers (%d)\n", ret);
>> +		goto error;
>> +	}
>> +
>> +	return 0;
>> +
>> +error:
>> +	scratch_unset_buffers(inst, false);
>> +	return ret;
>> +}
>> +
>> +int internal_bufs_free(struct vidc_inst *inst)
>> +{
>> +	struct device *dev = inst->core->dev;
>> +	int ret;
>> +
>> +	ret = scratch_unset_buffers(inst, false);
>> +	if (ret)
>> +		dev_err(dev, "failed to release scratch buffers: %d\n", ret);
>> +
>> +	ret = persist_unset_buffers(inst);
>> +	if (ret)
>> +		dev_err(dev, "failed to release persist buffers: %d\n", ret);
>> +
>> +	return ret;
>> +}
>> diff --git a/drivers/media/platform/qcom/vidc/int_bufs.h b/drivers/media/platform/qcom/vidc/int_bufs.h
>> new file mode 100644
>> index 000000000000..5f8b2b85839f
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/int_bufs.h
>> @@ -0,0 +1,23 @@
>> +/*
>> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +#ifndef __VIDC_INTERNAL_BUFFERS_H__
>> +#define __VIDC_INTERNAL_BUFFERS_H__
>> +
>> +struct vidc_inst;
>> +
>> +int internal_bufs_alloc(struct vidc_inst *inst);
>> +int internal_bufs_free(struct vidc_inst *inst);
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/vidc/load.c b/drivers/media/platform/qcom/vidc/load.c
>> new file mode 100644
>> index 000000000000..8ae25fc0e8a5
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/load.c
>> @@ -0,0 +1,104 @@
>> +/*
>> + * Copyright (c) 2013-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +
>> +#include <linux/clk.h>
>> +
>> +#include "core.h"
>> +#include "load.h"
>> +
>> +static u32 get_inst_load(struct vidc_inst *inst)
>> +{
>> +	int mbs;
>> +	u32 w = inst->width;
>> +	u32 h = inst->height;
>> +
>> +	if (!inst->hfi_inst || !(inst->hfi_inst->state >= INST_INIT &&
>> +				 inst->hfi_inst->state < INST_STOP))
>> +		return 0;
>> +
>> +	mbs = (ALIGN(w, 16) / 16) * (ALIGN(h, 16) / 16);
>> +
>> +	return mbs * inst->fps;
>> +}
>> +
>> +static u32 get_load(struct vidc_core *core, u32 session_type)
>> +{
>> +	struct vidc_inst *inst = NULL;
>> +	u32 mbs_per_sec = 0;
>> +
>> +	mutex_lock(&core->lock);
>> +	list_for_each_entry(inst, &core->instances, list) {
>> +		if (inst->session_type != session_type)
>> +			continue;
>> +
>> +		mbs_per_sec += get_inst_load(inst);
>> +	}
>> +	mutex_unlock(&core->lock);
>> +
>> +	return mbs_per_sec;
>> +}
>> +
>> +static int scale_clocks_load(struct vidc_core *core, u32 mbs_per_sec)
>> +{
>> +	const struct freq_tbl *table = core->res->freq_tbl;
>> +	int num_rows = core->res->freq_tbl_size;
>> +	struct clk *clk = core->clks[0];
> 
> Using individual clk pointers instead of this array would make this
> "core_clk" easier to follow.

I have decided to use an array of struct clk pointers is that the number
of clocks depends on SoC, for example 8096 have 9 clks and I wanted to
avoid describing each of them as idividual one.

> 
>> +	struct device *dev = core->dev;
>> +	unsigned long freq = table[0].freq;
>> +	int ret, i;
>> +
>> +	if (!mbs_per_sec num_row&& s > 1) {
>> +		freq = table[num_rows - 1].freq;
>> +		goto set_freq;
>> +	}
> 
> Here we will set freq to the last entry in the freq table, potentially
> table[0] if num_rows == 1, so the second part of the conditional doesn't
> add any value and you can skip the early initialization above.

Ok I will reconsider this part.

> 
> And you can put the loop below in an else block instead of using a goto.
>> +
>> +	for (i = 0; i < num_rows; i++) {
>> +		if (mbs_per_sec > table[i].load)
>> +			break;
>> +		freq = table[i].freq;
>> +	}
>> +
>> +set_freq:
>> +
>> +	ret = clk_set_rate(clk, freq);
>> +	if (ret) {
>> +		dev_err(dev, "failed to set clock rate %lu (%d)\n", freq, ret);
>> +		return ret;
>> +	}
>> +
>> +	return 0;
> 
> ret will be 0 here, so print the error message conditionally and then
> just return ret.
> 
>> +}
>> +
>> +int vidc_scale_clocks(struct vidc_core *core)
> 
> This is only called from helpers.c, drop this file and move the
> implementation there.

OK, agreed.

> 
>> +{
>> +	struct device *dev = core->dev;
>> +	u32 mbs_per_sec;
>> +	int ret;
>> +
>> +	mbs_per_sec = get_load(core, VIDC_SESSION_TYPE_ENC) +
>> +		      get_load(core, VIDC_SESSION_TYPE_DEC);
>> +
>> +	if (mbs_per_sec > core->res->max_load) {
>> +		dev_warn(dev, "HW is overloaded, needed: %d max: %d\n",
>> +			 mbs_per_sec, core->res->max_load);
>> +		return -EBUSY;
>> +	}
>> +
>> +	ret = scale_clocks_load(core, mbs_per_sec);
>> +	if (ret)
>> +		dev_warn(dev, "failed to scale clocks, performance might be impacted\n");
>> +
>> +	return 0;
>> +}
> [..]
>> diff --git a/drivers/media/platform/qcom/vidc/mem.c b/drivers/media/platform/qcom/vidc/mem.c
>> new file mode 100644
>> index 000000000000..6a83b5784410
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/mem.c
>> @@ -0,0 +1,64 @@
>> +/*
>> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +
>> +#include <linux/device.h>
>> +#include <linux/dma-direction.h>
>> +#include <linux/dma-mapping.h>
>> +#include <linux/err.h>
>> +#include <linux/slab.h>
>> +
>> +#include "mem.h"
>> +
>> +struct vidc_mem *mem_alloc(struct device *dev, size_t size, int map_kernel)
> 
> This is a terrible name for a global function.
> 
> But I think you can favorably inline this into the two callers. They
> both have their own tracking objects. So just drop this entire file.

OK, I will delete it.

> 
>> +{
>> +	struct vidc_mem *mem;
>> +
>> +	if (!size)
>> +		return ERR_PTR(-EINVAL);
>> +
>> +	if (IS_ERR(dev))
>> +		return ERR_CAST(dev);
>> +
>> +	mem = kzalloc(sizeof(*mem), GFP_KERNEL);
>> +	if (!mem)
>> +		return ERR_PTR(-ENOMEM);
>> +
>> +	mem->size = ALIGN(size, SZ_4K);
>> +	mem->iommu_dev = dev;
>> +
>> +	mem->attrs = DMA_ATTR_WRITE_COMBINE;
>> +
>> +	if (!map_kernel)
>> +		mem->attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
>> +
>> +	mem->kvaddr = dma_alloc_attrs(mem->iommu_dev, mem->size, &mem->da,
>> +				      GFP_KERNEL, mem->attrs);
>> +	if (!mem->kvaddr) {
>> +		kfree(mem);
>> +		return ERR_PTR(-ENOMEM);
>> +	}
>> +
>> +	return mem;
>> +}
>> +
>> +void mem_free(struct vidc_mem *mem)
>> +{
>> +	if (!mem)
>> +		return;
>> +
>> +	dma_free_attrs(mem->iommu_dev, mem->size, mem->kvaddr,
>> +	       mem->da, mem->attrs);
>> +	kfree(mem);
>> +};
> [..]
>> diff --git a/drivers/media/platform/qcom/vidc/resources.c b/drivers/media/platform/qcom/vidc/resources.c
>> new file mode 100644
>> index 000000000000..e00ed1caa824
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vidc/resources.c
>> @@ -0,0 +1,46 @@
>> +/*
>> + * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
>> + * Copyright (C) 2016 Linaro Ltd.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + */
>> +#include <linux/bug.h>
>> +#include <linux/types.h>
>> +#include <linux/kernel.h>
>> +
>> +#include "hfi.h"
>> +
>> +static const struct freq_tbl msm8916_freq_table[] = {
>> +	{ 352800, 228570000 },	/* 1920x1088 @ 30 + 1280x720 @ 30 */
>> +	{ 244800, 160000000 },	/* 1920x1088 @ 30 */
>> +	{ 108000, 100000000 },	/* 1280x720 @ 30 */
>> +};
>> +
>> +static const struct reg_val msm8916_reg_preset[] = {
>> +	{ 0xe0020, 0x05555556 },
>> +	{ 0xe0024, 0x05555556 },
>> +	{ 0x80124, 0x00000003 },
>> +};
>> +
>> +const struct vidc_resources msm8916_res = {
>> +	.freq_tbl = msm8916_freq_table,
>> +	.freq_tbl_size = ARRAY_SIZE(msm8916_freq_table),
>> +	.reg_tbl = msm8916_reg_preset,
>> +	.reg_tbl_size = ARRAY_SIZE(msm8916_reg_preset),
>> +	.clks = {"core", "iface", "bus", },
>> +	.clks_num = 3,
>> +	.max_load = 352800, /* 720p@30 + 1080p@30 */
>> +	.hfi_version = 0,
> 
> Unused

hfi_version is used from vidc_hfi_create() to decide which packetization
type "3xx" or "legacy" to use. Currently msm8916 use "legacy" but I
guess msm8996 will use "3xx"

> 
>> +	.vmem_id = VIDC_RESOURCE_NONE,
> 
> Unused

Some of the planed SoCs to support with this driver has this fast  video
RAM memory, despite that msm8916 has not.

> 
>> +	.vmem_size = 0,
> 
> Unused

this is for next SoCs which we will support.

> 
>> +	.vmem_addr = 0,
> 
> Unused

same comment as above.

> 
>> +	.dma_mask = 0xddc00000 - 1,
>> +};
> 
> These tables could with favor be moved next to the of_table in vidc.c

yes, makes sense.

-- 
regards,
Stan

Re: [PATCH 2/8] media: vidc: adding core part and helper functions

[Bjorn Andersson]

On Thu 25 Aug 05:59 PDT 2016, Stanimir Varbanov wrote:

> Hi Bjorn,
> 
> Thanks for the review and comments!
> 
> On 08/23/2016 05:50 AM, Bjorn Andersson wrote:
> > On Mon 22 Aug 06:13 PDT 2016, Stanimir Varbanov wrote:
> > 
> > Hi Stan,
> > 
> >> This adds core part of the vidc driver common helper functions
> >> used by encoder and decoder specific files.
> > 
> > I believe "vidc" is short for "video core" and this is not the only
> > "video core" from Qualcomm. This driver is the v4l2 <-> hfi interface and
> 
> What other "video core"s do you know?
> 

The blackbird, or "vidc" as we know it in 8064. That one should likely
be named "mfc" though.

> > uses either two ram based fifos _or_ apr tal for communication with the
> > implementation.
> > 
> > In the case of apr, the other side is not the venus core but rather the
> > "VIDC" apr service on the Hexagon DSP. In this case the hfi packets are
> > encapsulated in apr packets. Although this is not used in 8916 it would
> > be nice to be able to add this later...
> 
> OK, you are talking about q6_hfi.c which file is found in msm-3.10 and
> maybe older kernel versions.
> 

That's the one.

> There is a function vidc_hfi_create() which currently creates venus hfi
> interface but it aways could be extended to call q6 DSP specific function.
> 

As the ADSP comes up with a VIDC service (on the applicable platform(s))
we get an APR channel, the concept is similar from then on but rather
than putting the messages directly into the venus hfi tx/rx fifos a
header is prepended to each hfi message and it's passed to the APR.

In the Qualcomm code the hfi ops are picked as vidc_hfi_create() is
called, but I think flipping that upside down to have the venus_hfi or
q6_hfi be the probe point and then calling the common probe part with an
ops struct associated would be more natural.

I do however not see a problem with doing such refactoring in the
future, I just wanted to bring it up.

> > 
> > 
> > But I think we should call this driver "hfi" - or at least venus, as
> > it's not compatible with e.g the "blackbird" found in 8064, which is
> > also called "vidc".
> 
> Do you think that vidc driver for 8064 will ever reach the mainline kernel?
> 

There are strong wishes for it to be supported, so we should take
reasonable measures to make sure its possible.

> I personally don't like hfi nor venus other suggestions? Does "vidcore"
> or "vcore" makes sense?
> 

These names would imply that we intend to have a single driver for all Qualcomm
video encoder/decoders.

As it's really hard to envision the future, or to argue about 8064 vidc,
I would rather see a driver for the hfi based video encoder/decoders.

If we in 8996+1 see a non-hfi chip benefit greatly from reusing
something from this implementation then those better go into the v4l
core or some other common entity.

[..]
> >> diff --git a/drivers/media/platform/qcom/vidc/core.c b/drivers/media/platform/qcom/vidc/core.c
[..]
> >> +static void vidc_sys_error_handler(struct work_struct *work)
> >> +{
> >> +	struct vidc_sys_error *handler =
> >> +		container_of(work, struct vidc_sys_error, work.work);
> >> +	struct vidc_core *core = handler->core;
> >> +	struct hfi_core *hfi = &core->hfi;
> >> +	struct device *dev = core->dev;
> >> +	int ret;
> >> +
> >> +	mutex_lock(&hfi->lock);
> >> +	if (hfi->state != CORE_INVALID)
> >> +		goto exit;
> >> +
> >> +	mutex_unlock(&hfi->lock);
> >> +
> >> +	ret = vidc_hfi_core_deinit(hfi);
> >> +	if (ret)
> >> +		dev_err(dev, "core: deinit failed (%d)\n", ret);
> >> +
> >> +	mutex_lock(&hfi->lock);
> >> +
> >> +	rproc_report_crash(core->rproc, RPROC_FATAL_ERROR);
> > 
> > This operation is async, as such I believe this to be fragile. To get
> > the expected result you should be able to simply call
> > rproc_shutdown()/rproc_boot() to restart the core...
> 
> OK will remove crash report for now.
> 
> > 
> > However, if we at any point would like to be able to get memory dumps
> > from this core (likely a requirement on the Qualcomm side) we need to
> > call rproc_report_crash() and let it collect the resources and then
> > power cycle the core.
> > 
> > 
> > As the life cycle of the venus driver goes 1:1 with the rproc driver I
> > think it would be more suitable to make the v4l driver a child of the
> > rproc driver and have it probe/remove this driver as the rproc comes and
> > goes. This would allow us to call rproc_report_crash() here, we will be
> > removed and when the crash is handled (sometime in the future) we will
> > be probed again.
> 
> What's the problem with Kconfig "depends on QCOM_VENUS_PIL", isn't that
> enough?
> 

I mean in runtime, not compile time.

Devices instantiated from this driver does not serve a purpose without
either the venus rproc running or the adsp running. Further more as we
detect an issue with the remote core these resources (the venus or adsp)
will be stopped and restarted - potentially with a long delay inbetween
for ramdumping.

So either the venus driver must be resilient towards the remote being
gone or we should tie the venus driver to the running state of the
remoteproc driver.

One way to do that would be to add a of_platform_populate() in
venus_start() and a of_platform_depopulate() in venus_stop() and
represent the hfi-venus driver as a child of the remoteproc driver.

> > 
> >> +
> >> +	vidc_rproc_shutdown(core);
> >> +
> >> +	ret = vidc_rproc_boot(core);
> >> +	if (ret)
> >> +		goto exit;
> >> +
> >> +	hfi->state = CORE_INIT;
> >> +
> >> +exit:
> >> +	mutex_unlock(&hfi->lock);
> >> +	kfree(handler);
> >> +}
> >> +
> >> +static int vidc_event_notify(struct hfi_core *hfi, u32 event)
> >> +{
> >> +	struct vidc_sys_error *handler;
> >> +	struct hfi_inst *inst;
> >> +
> >> +	switch (event) {
> >> +	case EVT_SYS_WATCHDOG_TIMEOUT:
> >> +	case EVT_SYS_ERROR:
> >> +		break;
> >> +	default:
> >> +		return -EINVAL;
> >> +	}
> >> +
> >> +	mutex_lock(&hfi->lock);
> >> +
> >> +	hfi->state = CORE_INVALID;
> >> +
> >> +	list_for_each_entry(inst, &hfi->instances, list) {
> >> +		mutex_lock(&inst->lock);
> >> +		inst->state = INST_INVALID;
> >> +		mutex_unlock(&inst->lock);
> >> +	}
> >> +
> >> +	mutex_unlock(&hfi->lock);
> >> +
> >> +	handler = kzalloc(sizeof(*handler), GFP_KERNEL);
> >> +	if (!handler)
> >> +		return -ENOMEM;
> >> +
> >> +	handler->core = container_of(hfi, struct vidc_core, hfi);
> >> +	INIT_DELAYED_WORK(&handler->work, vidc_sys_error_handler);
> >> +
> >> +	/*
> >> +	 * Sleep for 5 sec to ensure venus has completed any
> >> +	 * pending cache operations. Without this sleep, we see
> >> +	 * device reset when firmware is unloaded after a sys
> >> +	 * error.
> >> +	 */
> >> +	schedule_delayed_work(&handler->work, msecs_to_jiffies(5000));
> >> +
> >> +	return 0;
> >> +}
> >> +
> >> +static const struct hfi_core_ops vidc_core_ops = {
> >> +	.event_notify = vidc_event_notify,
> >> +};
> > 
> > This is an overly generic way of calling vidc_sys_error_handler().
> > There is no need for having the hfi_core_ops indirections for a single
> > op that will only exist in 1 and only 1 variant.
> 
> The .event_notify operation is called by hfi part (in hfi_msgs.c) of the
> driver and I don't want break the interface. My idea was to have HFI
> part and v4l2 part, and each of these parts taking care of their
> specifics. The interface between HFI <-> v4l2 should be immutable and
> shoudn't be changed when every new version of the hardware IP rise up.
> 

I'm fine with having the structural split between the two pieces
(although I see it serving little purpose), but I don't think it's
necessary to use a function pointer interface for something that always
only will have a single possible value.

> > 
> > Just replace the two affected event_notify() calls with a direct call to
> > this function (and clean it up a bit).
> > 
> >> +
[..]
> >> +
> >> +static irqreturn_t vidc_isr_thread(int irq, void *dev_id)
> >> +{
> >> +	return vidc_hfi_isr_thread(irq, dev_id);
> >> +}
> >> +
> >> +static irqreturn_t vidc_isr(int irq, void *dev)
> >> +{
> >> +	return vidc_hfi_isr(irq, dev);
> >> +}
> > 
> > These two functions indicates that we're requesting the irq in the wrong
> > layer.
> 
> IMO the proper place is platform driver .probe method.
> 

If we squash the vidc-core and hfi-core into one this oddness should go
away, right?

> > 
> > Also, these two functions arrives in a later patchset, so I assume this
> > doesn't compile...
> 
> That's why I'm adding Makefiles later on patchset. On the other hand I
> have splitted the driver by files because I think it is easier for
> review. But I might be wrong.
> 

Ok, I'm afraid I don't have any good suggestion to counter this. It's
hard to introduce a minimal functional driver for this and then add
features - as any "minimal" version is large.


Sorry for all the "unused" comments for later patches, I didn't double
check those with the end result.

> > 
> >> +
[..]
> >> +
> >> +static int vidc_probe(struct platform_device *pdev)
> >> +{
> >> +	struct device *dev = &pdev->dev;
> >> +	struct vidc_core *core;
> >> +	struct device_node *rproc;
> >> +	struct resource *r;
> >> +	int ret;
> >> +
> >> +	core = devm_kzalloc(dev, sizeof(*core), GFP_KERNEL);
> >> +	if (!core)
> >> +		return -ENOMEM;
> >> +
> >> +	core->dev = dev;
> >> +	platform_set_drvdata(pdev, core);
> >> +
> >> +	rproc = of_parse_phandle(dev->of_node, "rproc", 0);
> >> +	if (IS_ERR(rproc))
> >> +		return PTR_ERR(rproc);
> >> +
> >> +	core->rproc = rproc_get_by_phandle(rproc->phandle);
> > 
> > FYI, We're hoping to land some patches shortly that will replace this
> > with rproc_get(pdev->dev.of_node), looking up an rproc by the standard
> > "rprocs" property...
> 
> OK, that looks good. But shoudn't be rproc_get(pdev->dev)?
> 

Sorry, the patches in flight introduces of_rproc_get() - not
rproc_get(), which takes a device_node. But it's still pending some DT
discussions.

> > 
> >> +	if (IS_ERR(core->rproc))
> >> +		return PTR_ERR(core->rproc);
> >> +	else if (!core->rproc)
> >> +		return -EPROBE_DEFER;
> > 
> > We're cleaning up this in the core as well.
> > 
> > You need to rproc_put() the rproc pointer after this point.
> 
> OK, good. When those changes landed I will rework this part.
> 
> > 
> > 
> > My question still stands though, if this driver should be probed as the
> > remoteproc is booted (or the apr service appearing). I will continue to
> > look at that.
> 
> I might be misunderstood your point here. Is your concern related to
> EPROBE_DEFFER or some sort of ordering issue in rproc?
> 
> Currently the vidc depends on QCOM_VENUS_PIL in Kconfig, so modprobe
> vidc should modprobe remoteproc driver cause it depends on it.
> 

If we probe the two drivers independently, then you're good. I just
don't see how we (sanely) will communicate the information about when
the remote goes down and comes up related to a crash.

> > 
> >> +
> >> +	r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> >> +	core->base = devm_ioremap_resource(dev, r);
> >> +	if (IS_ERR(core->base))
> >> +		return PTR_ERR(core->base);
> >> +
> >> +	core->irq = platform_get_irq(pdev, 0);
> >> +	if (core->irq < 0)
> >> +		return core->irq;
> >> +
> >> +	core->res = of_device_get_match_data(dev);
> >> +	if (!core->res)
> >> +		return -ENODEV;
> >> +
> >> +	ret = vidc_clks_get(core, core->res->clks_num, core->res->clks);
> >> +	if (ret)
> >> +		return ret;
> >> +
> >> +	ret = dma_set_mask_and_coherent(dev, core->res->dma_mask);
> >> +	if (ret)
> >> +		return ret;
> >> +
> >> +	INIT_LIST_HEAD(&core->instances);
> >> +	mutex_init(&core->lock);
> >> +
> >> +	ret = devm_request_threaded_irq(dev, core->irq, vidc_isr,
> >> +					vidc_isr_thread,
> >> +					IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
> > 
> > Drop this IRQF_TRIGGER_HIGH and have this be specified in devicetree.
> 
> OK can do that, but is there a convention about who is populating the
> flag and what is the precedence dt against the driver?
> 

It seems there's a push lately for moving to getting the trigger from
DT. But I haven't not studied the details or reasoning behind it.

> > 
> >> +					"vidc", &core->hfi);
> >> +	if (ret)
> >> +		return ret;
> >> +
> >> +	core->hfi.core_ops = &vidc_core_ops;
> >> +	core->hfi.dev = dev;
> >> +
> >> +	ret = vidc_hfi_create(&core->hfi, core->res, core->base);
> >> +	if (ret)
> >> +		return ret;
> >> +
> >> +	ret = vidc_clks_enable(core, core->res);
> >> +	if (ret)
> >> +		goto err_hfi_destroy;
> >> +
> >> +	ret = vidc_rproc_boot(core);
> >> +	if (ret) {
> >> +		vidc_clks_disable(core, core->res);
> >> +		goto err_hfi_destroy;
> >> +	}
> >> +
> >> +	pm_runtime_enable(dev);
> >> +
> >> +	ret = pm_runtime_get_sync(dev);
> >> +	if (ret < 0)
> >> +		goto err_runtime_disable;
> >> +
> >> +	ret = vidc_hfi_core_init(&core->hfi);
> >> +	if (ret)
> >> +		goto err_rproc_shutdown;
> >> +
> >> +	ret = pm_runtime_put_sync(dev);
> >> +	if (ret)
> >> +		goto err_core_deinit;
> >> +
> >> +	vidc_clks_disable(core, core->res);
> > 
> > These operations follow the general pattern of booting other Qualcomm
> > remoteprocs; acquire and enable some resources, boot the core and
> > disable the resources. Therefor it looks quite likely that these
> > operations are related to the life cycle of the venus core, rather than
> > hfi.
> 
> So you saying that
> 

I'm saying that I would like for the rproc driver to be able to boot,
shutdown and handle a crash (not trigger) without having to depend on
the venus driver setting up a bunch of resources for it.

Either that or the venus rproc driver should not be a standalone rproc
driver - with its own probe function.

> > 
> >> +
> >> +	ret = v4l2_device_register(dev, &core->v4l2_dev);
> >> +	if (ret)
> >> +		goto err_core_deinit;
> >> +
> >> +	ret = vdec_init(core, &core->vdev_dec);
> >> +	if (ret)
> >> +		goto err_dev_unregister;
> >> +
> >> +	ret = venc_init(core, &core->vdev_enc);
> >> +	if (ret)
> >> +		goto err_vdec_deinit;
> >> +
> >> +	return 0;
> >> +
> >> +err_vdec_deinit:
> >> +	vdec_deinit(core, &core->vdev_dec);
> >> +err_dev_unregister:
> >> +	v4l2_device_unregister(&core->v4l2_dev);
> >> +err_core_deinit:
> >> +	vidc_hfi_core_deinit(&core->hfi);
> >> +err_rproc_shutdown:
> >> +	vidc_rproc_shutdown(core);
> >> +err_runtime_disable:
> >> +	pm_runtime_set_suspended(dev);
> >> +	pm_runtime_disable(dev);
> >> +err_hfi_destroy:
> >> +	vidc_hfi_destroy(&core->hfi);
> >> +	return ret;
> >> +}
> >> +
> >> +static int vidc_remove(struct platform_device *pdev)
> >> +{
> >> +	struct vidc_core *core = platform_get_drvdata(pdev);
> >> +	int ret;
> >> +
> >> +	ret = pm_runtime_get_sync(&pdev->dev);
> >> +	if (ret < 0)
> >> +		return ret;
> > 
> > No-one cares about you returning an error here, so you better move
> > forward and release as much of your resources as possible even though
> > you didn't get your pm.
> 
> Hmm, I don't agree here. The runtime_resume will enable clocks (for
> example venus iface clk) and if it fails the subsequent call to
> vidc_rproc_shutdown can crash badly.
> 

Sorry for not being clear, that part we care about. What I meant was
that in the device core will dismantle your driver regardless of you
returning an error here. So you will leak the resources not freed below.

> > 
> >> +
> >> +	ret = vidc_hfi_core_deinit(&core->hfi);
> >> +	if (ret) {
> >> +		pm_runtime_put_sync(&pdev->dev);
> >> +		return ret;
> >> +	}
> >> +
> >> +	vidc_rproc_shutdown(core);
> >> +
> >> +	ret = pm_runtime_put_sync(&pdev->dev);
> >> +
> >> +	vidc_hfi_destroy(&core->hfi);
> >> +	vdec_deinit(core, &core->vdev_dec);
> >> +	venc_deinit(core, &core->vdev_enc);
> >> +	v4l2_device_unregister(&core->v4l2_dev);
> >> +
> >> +	pm_runtime_disable(core->dev);
> >> +
> >> +	return ret < 0 ? ret : 0;
> >> +}
> >> +
[..]
> >> diff --git a/drivers/media/platform/qcom/vidc/core.h b/drivers/media/platform/qcom/vidc/core.h
[..]
> 
> > 
> >> +
> >> +struct vidc_list {
> >> +	struct list_head list;
> >> +	struct mutex lock;
> >> +};
> > 
> > Can't we get away without passing around lockable lists? Does these
> > lists have to be locked independently and should we really pass around
> > their lock with them?
> 
> I guess it is possible but didn't spent to much time on that (I had more
> important problems to solve with downstream driver). So the answer is
> yes and I have to re-consider it.
> 

We with rework of internal buffers this should be less of a problem, you
could with favor grab the one lock, allocate all internal buffers and
then unlock the lock again.

[..]
> >> diff --git a/drivers/media/platform/qcom/vidc/helpers.c b/drivers/media/platform/qcom/vidc/helpers.c
[..]
> >> +int vidc_buf_descs(struct vidc_inst *inst, u32 type,
> >> +		   struct hfi_buffer_requirements *out)
> > 
> > If you call this vidc_get_buf_requirements() it would actually describe
> > what's going on. But why is this hfi wrapper in the core, rather than
> 
> the original name of this function was similar to what you suggest but I
> decided in last-minute cleanup to shorten its name.
> 

vidc_get_bufreq() should be short enough :)

> > just have the internal buffer manager call it directly.
> 
> It is in the core cause I used it on few places to gather buffer count
> needed depending on parameters (resolution, codec). Good example is
> vb2_ops::queue_setup where I need to return num_buffers depending on
> resolution, codec, bitrate, framerate and so on.
> 

Ok, that makes sense.

> > 
> > The call doesn't seem to depend on the parameters or state, can we
> > cache the result?
> 
> No, we cannot. see above comment. Something more the scratch and
> prersist buffer sizes can also be changed by the firmware depending on
> above proparties.
> 

Okay, I figured that might be the case.

> > 
> >> +{
> >> +	struct hfi_core *hfi = &inst->core->hfi;
> >> +	u32 ptype = HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS;
> >> +	union hfi_get_property hprop;
> >> +	int ret, i;
> >> +
> >> +	if (out)
> >> +		memset(out, 0, sizeof(*out));
> >> +
> >> +	ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype, &hprop);
> >> +	if (ret)
> >> +		return ret;
> >> +
> >> +	ret = -EINVAL;
> >> +
> >> +	for (i = 0; i < HFI_BUFFER_TYPE_MAX; i++) {
> >> +		if (hprop.bufreq[i].type != type)
> >> +			continue;
> >> +
> >> +		if (out)
> >> +			memcpy(out, &hprop.bufreq[i], sizeof(*out));
> >> +		ret = 0;
> >> +		break;
> >> +	}
> >> +
> >> +	return ret;
> >> +}
> >> +
> > [..]
> >> +
> >> +void vidc_vb2_stop_streaming(struct vb2_queue *q)
> >> +{
> >> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
> >> +	struct hfi_inst *hfi_inst = inst->hfi_inst;
> >> +	struct vidc_core *core = inst->core;
> >> +	struct device *dev = core->dev;
> >> +	struct hfi_core *hfi = &core->hfi;
> >> +	int ret, streamoff;
> >> +
> >> +	mutex_lock(&inst->lock);
> >> +	streamoff = inst->streamoff;
> >> +	mutex_unlock(&inst->lock);
> >> +
> >> +	if (streamoff)
> >> +		return;
> >> +
> >> +	mutex_lock(&inst->lock);
> >> +	if (inst->streamon == 0) {
> >> +		mutex_unlock(&inst->lock);
> >> +		return;
> >> +	}
> >> +	mutex_unlock(&inst->lock);
> > 
> > Why do we keep track of streamon and stream off, why isn't streamoff
> > ever cleared? Why don't we check both conditions in one critical region?
> 
> Probably cause its buggy, I will sort it out.
> 
> > 
> >> +
> >> +	ret = vidc_hfi_session_stop(hfi, inst->hfi_inst);
> >> +	if (ret) {
> >> +		dev_err(dev, "session: stop failed (%d)\n", ret);
> >> +		goto abort;
> > 
> > When are we going to relaim the buffers in these cases?
> 
> session_stop will instruct the firmware return buffers to the v4l2
> driver through hfi_inst_ops empty_buf_done and fill_buf_done, those
> operations will call vb2_buffer_done.
> 

Okay, but does that include the internal buffers as well?

> > 
> >> +	}
> >> +
> >> +	ret = vidc_hfi_session_unload_res(hfi, inst->hfi_inst);
> >> +	if (ret) {
> >> +		dev_err(dev, "session: release resources failed (%d)\n", ret);
> >> +		goto abort;
> >> +	}
> >> +
> >> +	ret = session_unregister_bufs(inst);
> >> +	if (ret) {
> >> +		dev_err(dev, "failed to release capture buffers: %d\n", ret);
> >> +		goto abort;
> >> +	}
> >> +
> >> +	ret = internal_bufs_free(inst);
> >> +
> >> +	if (hfi_inst->state == INST_INVALID || hfi->state == CORE_INVALID) {
> >> +		ret = -EINVAL;
> >> +		goto abort;
> >> +	}
> >> +
> >> +abort:
> >> +	if (ret)
> >> +		vidc_hfi_session_abort(hfi, inst->hfi_inst);
> >> +
> >> +	vidc_scale_clocks(inst->core);
> >> +
> >> +	ret = vidc_hfi_session_deinit(hfi, inst->hfi_inst);
> >> +
> >> +	mutex_lock(&inst->lock);
> >> +	inst->streamoff = 1;
> >> +	mutex_unlock(&inst->lock);
> >> +
> >> +	if (ret)
> >> +		dev_err(dev, "stop streaming failed type: %d, ret: %d\n",
> >> +			q->type, ret);
> >> +
> >> +	ret = pm_runtime_put_sync(dev);
> >> +	if (ret < 0)
> >> +		dev_err(dev, "%s: pm_runtime_put_sync (%d)\n", __func__, ret);
> >> +}
> >> +
[..]
> >> diff --git a/drivers/media/platform/qcom/vidc/helpers.h b/drivers/media/platform/qcom/vidc/helpers.h
[..]
> >> +static int scratch_set_buffers(struct vidc_inst *inst)
> >> +{
> >> +	struct device *dev = inst->core->dev;
> >> +	int ret;
> >> +
> >> +	ret = scratch_unset_buffers(inst, true);
> >> +	if (ret)
> >> +		dev_warn(dev, "Failed to release scratch buffers\n");
> > 
> > internal_bufs_free() calls scratch_unset_buffers(reuse=false) so we're
> > coming here with an empty scratchbufs either way - meaning that this
> > whole file can be greatly simplified.
> > 
> > So instead of trying to fix that I would suggest that you just let
> > internal_bufs_alloc() acquire the buffer requirements and call
> > internal_alloc_and_set() directly, storing the result in a single list.
> > 
> > And then inline a free method in internal_bufs_free() as well as drop
> > all reuse-stuff and unused/dead code.
> > 
> > That would simplify this file quite a bit and if there actually is a
> > need for the reusing of buffer that can be added at some later time.
> > 
> 
> Actially I thought about droping the reuse stuff in the past, so I agree
> on that cleanup. The thing which worries me is the size of those buffers
> (the biggest is 10-15MB) and also the allocation time. Currently those
> buffers are allocate on streamon time, but probably the right place is
> on request_buf time.
> 

I think this is definitely worth investigating and should be possible to
add incrementally once the driver has landed. As far as I can see the
code doesn't really work as it is now, so rather than fixing that make
it dead-simple and then add to it later.

> >> +
> >> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH);
> >> +	if (ret)
> >> +		goto error;
> >> +
> >> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_1);
> >> +	if (ret)
> >> +		goto error;
> >> +
> >> +	ret = scratch_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_2);
> >> +	if (ret)
> >> +		goto error;
> >> +
> >> +	return 0;
> >> +error:
> >> +	scratch_unset_buffers(inst, false);
> >> +	return ret;
> >> +}
> >> +

Regards,
Bjorn

Re: [PATCH 5/8] media: vidc: add Host Firmware Interface (HFI)

[Stanimir Varbanov]

Hi Bjorn,

Thanks for the comments!

On 08/23/2016 06:25 AM, Bjorn Andersson wrote:
> On Mon 22 Aug 06:13 PDT 2016, Stanimir Varbanov wrote:
> 
>> This is the implementation of HFI. It is loaded with the
>> responsibility to comunicate with the firmware through an
>> interface commands and messages.
>>
>>  - hfi.c has interface functions used by the core, decoder
>> and encoder parts to comunicate with the firmware. For example
>> there are functions for session and core initialisation.
>>
> 
> I can't help feeling that the split between core.c and hfi.c is a
> remnant of a vidc driver supporting both HFI and pre-HFI with the same
> v4l code.
> 
> What do you think about merging vidc_core with hfi_core and vidc_inst
> with hfi_inst? Both seems to be in a 1:1 relationship.

OK, I can give it a try.

> 
>>  - hfi_cmds has packetization operations which preparing
>> packets to be send from host to firmware.
>>
>>  - hfi_msgs takes care of messages sent from firmware to the
>> host.
>>
> [..]
>> diff --git a/drivers/media/platform/qcom/vidc/hfi_cmds.c b/drivers/media/platform/qcom/vidc/hfi_cmds.c
> [..]
>> +
>> +static const struct hfi_packetization_ops hfi_default = {
>> +	.sys_init = pkt_sys_init,
>> +	.sys_pc_prep = pkt_sys_pc_prep,
>> +	.sys_idle_indicator = pkt_sys_idle_indicator,
>> +	.sys_power_control = pkt_sys_power_control,
>> +	.sys_set_resource = pkt_sys_set_resource,
>> +	.sys_release_resource = pkt_sys_unset_resource,
>> +	.sys_debug_config = pkt_sys_debug_config,
>> +	.sys_coverage_config = pkt_sys_coverage_config,
>> +	.sys_ping = pkt_sys_ping,
>> +	.sys_image_version = pkt_sys_image_version,
>> +	.ssr_cmd = pkt_ssr_cmd,
>> +	.session_init = pkt_session_init,
>> +	.session_cmd = pkt_session_cmd,
>> +	.session_set_buffers = pkt_session_set_buffers,
>> +	.session_release_buffers = pkt_session_release_buffers,
>> +	.session_etb_decoder = pkt_session_etb_decoder,
>> +	.session_etb_encoder = pkt_session_etb_encoder,
>> +	.session_ftb = pkt_session_ftb,
>> +	.session_parse_seq_header = pkt_session_parse_seq_header,
>> +	.session_get_seq_hdr = pkt_session_get_seq_hdr,
>> +	.session_flush = pkt_session_flush,
>> +	.session_get_property = pkt_session_get_property,
>> +	.session_set_property = pkt_session_set_property,
>> +};
>> +
>> +static const struct hfi_packetization_ops *get_3xx_ops(void)
>> +{
>> +	static struct hfi_packetization_ops hfi_3xx;
>> +
>> +	hfi_3xx = hfi_default;
>> +	hfi_3xx.session_set_property = pkt_session_set_property_3xx;
>> +
>> +	return &hfi_3xx;
>> +}
>> +
>> +const struct hfi_packetization_ops *
>> +hfi_get_pkt_ops(enum hfi_packetization_type type)
> 
> The only reasonable argument I can come up with for not just exposing
> these as global functions would be that there are 23 of them... Can we
> skip the jump table?

Of course we can, but what will be the benefit, increased readability ?
Let's keep it for now. Once the driver is merged some smarter guy could
came up with better solution ;)

> 
>> +{
>> +	switch (type) {
>> +	case HFI_PACKETIZATION_LEGACY:
>> +		return &hfi_default;
>> +	case HFI_PACKETIZATION_3XX:
>> +		return get_3xx_ops();
>> +	}
>> +
>> +	return NULL;
>> +}
> 
> Regards,
> Bjorn
> 

-- 
regards,
Stan

Re: [PATCH 3/8] media: vidc: decoder: add video decoder files

[Stanimir Varbanov]

Hi Hans,

<cut>

>>>> +static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
>>>> +{
>>>> +	struct vidc_inst *inst = vb2_get_drv_priv(q);
>>>> +	struct hfi_core *hfi = &inst->core->hfi;
>>>> +	struct device *dev = inst->core->dev;
>>>> +	struct hfi_buffer_requirements bufreq;
>>>> +	struct hfi_buffer_count_actual buf_count;
>>>> +	struct vb2_queue *queue;
>>>> +	u32 ptype;
>>>> +	int ret;
>>>> +
>>>> +	switch (q->type) {
>>>> +	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>>>> +		queue = &inst->bufq_cap;
>>>> +		break;
>>>> +	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
>>>> +		queue = &inst->bufq_out;
>>>> +		break;
>>>> +	default:
>>>
>>> If start_streaming fails, then all pending buffers have to be returned by the driver
>>> by calling vb2_buffer_done(VB2_BUF_STATE_QUEUED). This will give ownership back to
>>> userspace.
>>
>> Infact this error path shouldn't be possible, because videobu2-core
>> should check q->type before jumping here?
> 
> Sorry, I wasn't clear. It is not just this place (and you are right, the error
> path here isn't possible), but any place where an error is returned in this
> function.
> 
> Those should be replaced by a goto fail; and in the fail label all pending buffers
> have to be returned. Same code as in stop_streaming, but you call vb2_buffer_done
> with state QUEUED instead of state ERROR.

OK, I need to call vb2_buffer_done(ERROR) for every queued buffer
(before invocation of STREAM_ON) if start_streaming failed.

I think that in present code I have calls to vb2_buffer_done but with
status DONE.

<cut>

>>>> +static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
>>>> +{
>>>> +	struct vidc_inst *inst = ctrl_to_inst(ctrl);
>>>> +	struct vdec_controls *ctr = &inst->controls.dec;
>>>> +	struct hfi_core *hfi = &inst->core->hfi;
>>>> +	union hfi_get_property hprop;
>>>> +	u32 ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
>>>> +	int ret;
>>>> +
>>>> +	switch (ctrl->id) {
>>>> +	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
>>>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
>>>> +	case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
>>>> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
>>>> +						    &hprop);
>>>> +		if (!ret)
>>>> +			ctr->profile = hprop.profile_level.profile;
>>>> +		ctrl->val = ctr->profile;
>>>> +		break;
>>>> +	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
>>>> +	case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
>>>> +		ret = vidc_hfi_session_get_property(hfi, inst->hfi_inst, ptype,
>>>> +						    &hprop);
>>>> +		if (!ret)
>>>> +			ctr->level = hprop.profile_level.level;
>>>> +		ctrl->val = ctr->level;
>>>> +		break;
>>>> +	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
>>>> +		ctrl->val = ctr->post_loop_deb_mode;
>>>> +		break;
>>>
>>> Why are these volatile?
>>
>> Because the firmware acording to stream headers that profile and levels
>> are different.
> 
> But when these change, isn't the driver told about it? And can these
> change midstream? I would expect this to be set once when you start
> decoding and not change afterwards.

Actually the decoder firmware will detect the profile/level pair itself
based on elementary stream headers. So I'd expect that getting those
parameters by session_get_property API will just return what the decoder
thinks about profile/level.

-- 
regards,
Stan

Re: [PATCH 0/8] Qualcomm video decoder/encoder driver

[Hans Verkuil]

On 08/22/2016 03:13 PM, Stanimir Varbanov wrote:
> This patchset introduces a basic support for Qualcomm video
> acceleration hardware used for video stream decoding/encoding.
> The video IP can found on various qcom SoCs like apq8084, msm8916
> and msm8996, hence it is widly distributed but the driver is 
> missing in the mainline.
> 
> The v4l2 driver is something like a wrapper over Host Firmware
> Interface. The HFI itself is a set of command and message packets
> send/received through shared memory, and its purpose is to
> comunicate with the firmware which is run on remote processor.
> The Venus is the name of the video hardware IP that doing the
> video acceleration.
> 
> From the software point of view the HFI interface is implemented
> in the files with prefix hfi_xxx. It acts as a translation layer
> between HFI and v4l2 layer. There is one special file in the 
> driver called hfi_venus which doing most of the driver
> orchestration work. Something more it setups Venus core, run it
> and handle commands and messages from low-level point of view with
> the help of provided functions by HFI interface.
> 
> I think that the driver is in good shape for mainline kernel, and
> I hope the review comments will help to improve it, so please
> do review and make comments.

I was hoping that I could finish reviewing this patch series today,
but that didn't work out.

I have more time next week, but I wonder if it isn't better if you make a
v2 first, taking my comments into account. Then I can review v2 next week.

Also test with the latest v4l2-compliance (i.e. as of today) since I improved
a few tests relating to g/s_selection and g/s_parm.

Regards,

	Hans

> 
> The driver depends on:
>  - venus remoteproc driver posted at [1].
>  - out-of-tree qcom IOMMU driver and IOMMU probe deferral support
>  at [2].
> 
> The driver has been tested on db410c (with apq8016 SoC) with simple 
> v4l2 test applications and with gstreamer v4l2 videodec plugin,
> and v4l2 h264 out-of-tree gstreamer videoenc plugin.
> 
> The output of v4l2-compliance test looks like:
> 
> root@dragonboard-410c:/home/linaro# ./v4l2-compliance -d /dev/video0 
> v4l2-compliance SHA   : ee1ab491019f80052834d14c76bdd1c1b46f2158
> 
> Driver Info:
>         Driver name   : vidc
>         Card type     : video decoder
>         Bus info      : platform:vidc
>         Driver version: 4.8.0
>         Capabilities  : 0x84204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
> 
> Compliance test for device /dev/video0 (not using libv4l2):
> 
> Required ioctls:
>         test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>         test second video open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
>         test for unlimited opens: OK
> 
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
>         test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>         test VIDIOC_QUERYCTRL: OK
>         test VIDIOC_G/S_CTRL: OK
>         test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>         test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>         test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>         Standard Controls: 7 Private Controls: 0
> 
> Format ioctls:
>         test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>         test VIDIOC_G/S_PARM: OK
>         test VIDIOC_G_FBUF: OK (Not Supported)
>         test VIDIOC_G_FMT: OK
>         test VIDIOC_TRY_FMT: OK
>         test VIDIOC_S_FMT: OK
>         test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>         test Cropping: OK (Not Supported)
>         test Composing: OK (Not Supported)
>         test Scaling: OK
> 
> Codec ioctls:
>         test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>         test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>         test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
>         test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>         test VIDIOC_EXPBUF: OK
> 
> Test input 0:
> 
> Total: 43, Succeeded: 43, Failed: 0, Warnings: 0
> 
> root@dragonboard-410c:/home/linaro# ./v4l2-compliance -d /dev/video1
> v4l2-compliance SHA   : ee1ab491019f80052834d14c76bdd1c1b46f2158
> 
> Driver Info:
>         Driver name   : vidc
>         Card type     : video encoder
>         Bus info      : platform:vidc
>         Driver version: 4.8.0
>         Capabilities  : 0x84204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04204000
>                 Video Memory-to-Memory Multiplanar
>                 Streaming
>                 Extended Pix Format
> 
> Compliance test for device /dev/video1 (not using libv4l2):
> 
> Required ioctls:
>         test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>         test second video open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
>         test for unlimited opens: OK
> 
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
>         test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>         test VIDIOC_QUERYCTRL: OK
>         test VIDIOC_G/S_CTRL: OK
>         test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>         test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>         test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>         Standard Controls: 32 Private Controls: 0
> 
> Format ioctls:
>         test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>         test VIDIOC_G/S_PARM: OK
>         test VIDIOC_G_FBUF: OK (Not Supported)
>         test VIDIOC_G_FMT: OK
>         test VIDIOC_TRY_FMT: OK
>         test VIDIOC_S_FMT: OK
>         test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>         test Cropping: OK
>         test Composing: OK (Not Supported)
>         test Scaling: OK
> 
> Codec ioctls:
>         test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>         test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>         test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
>         test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>         test VIDIOC_EXPBUF: OK
> 
> Test input 0:
> 
> Total: 43, Succeeded: 43, Failed: 0, Warnings: 0
> 
> regards,
> Stan
> 
> [1] https://lkml.org/lkml/2016/8/19/570
> [2] https://www.spinics.net/lists/arm-kernel/msg522505.html
> 
> Stanimir Varbanov (8):
>   doc: DT: vidc: binding document for Qualcomm video driver
>   media: vidc: adding core part and helper functions
>   media: vidc: decoder: add video decoder files
>   media: vidc: encoder: add video encoder files
>   media: vidc: add Host Firmware Interface (HFI)
>   media: vidc: add Venus HFI files
>   media: vidc: add Makefiles and Kconfig files
>   media: vidc: enable building of the video codec driver
> 
>  .../devicetree/bindings/media/qcom,vidc.txt        |   61 +
>  drivers/media/platform/Kconfig                     |    1 +
>  drivers/media/platform/Makefile                    |    1 +
>  drivers/media/platform/qcom/Kconfig                |    8 +
>  drivers/media/platform/qcom/Makefile               |    6 +
>  drivers/media/platform/qcom/vidc/Makefile          |   19 +
>  drivers/media/platform/qcom/vidc/core.c            |  548 +++++++
>  drivers/media/platform/qcom/vidc/core.h            |  196 +++
>  drivers/media/platform/qcom/vidc/helpers.c         |  394 +++++
>  drivers/media/platform/qcom/vidc/helpers.h         |   43 +
>  drivers/media/platform/qcom/vidc/hfi.c             |  622 ++++++++
>  drivers/media/platform/qcom/vidc/hfi.h             |  272 ++++
>  drivers/media/platform/qcom/vidc/hfi_cmds.c        | 1261 ++++++++++++++++
>  drivers/media/platform/qcom/vidc/hfi_cmds.h        |  338 +++++
>  drivers/media/platform/qcom/vidc/hfi_helper.h      | 1143 +++++++++++++++
>  drivers/media/platform/qcom/vidc/hfi_msgs.c        | 1072 ++++++++++++++
>  drivers/media/platform/qcom/vidc/hfi_msgs.h        |  298 ++++
>  drivers/media/platform/qcom/vidc/hfi_venus.c       | 1539 ++++++++++++++++++++
>  drivers/media/platform/qcom/vidc/hfi_venus.h       |   25 +
>  drivers/media/platform/qcom/vidc/hfi_venus_io.h    |   98 ++
>  drivers/media/platform/qcom/vidc/int_bufs.c        |  325 +++++
>  drivers/media/platform/qcom/vidc/int_bufs.h        |   23 +
>  drivers/media/platform/qcom/vidc/load.c            |  104 ++
>  drivers/media/platform/qcom/vidc/load.h            |   22 +
>  drivers/media/platform/qcom/vidc/mem.c             |   64 +
>  drivers/media/platform/qcom/vidc/mem.h             |   32 +
>  drivers/media/platform/qcom/vidc/resources.c       |   46 +
>  drivers/media/platform/qcom/vidc/resources.h       |   46 +
>  drivers/media/platform/qcom/vidc/vdec.c            | 1100 ++++++++++++++
>  drivers/media/platform/qcom/vidc/vdec.h            |   27 +
>  drivers/media/platform/qcom/vidc/vdec_ctrls.c      |  200 +++
>  drivers/media/platform/qcom/vidc/vdec_ctrls.h      |   21 +
>  drivers/media/platform/qcom/vidc/venc.c            | 1261 ++++++++++++++++
>  drivers/media/platform/qcom/vidc/venc.h            |   27 +
>  drivers/media/platform/qcom/vidc/venc_ctrls.c      |  396 +++++
>  drivers/media/platform/qcom/vidc/venc_ctrls.h      |   23 +
>  36 files changed, 11662 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/qcom,vidc.txt
>  create mode 100644 drivers/media/platform/qcom/Kconfig
>  create mode 100644 drivers/media/platform/qcom/Makefile
>  create mode 100644 drivers/media/platform/qcom/vidc/Makefile
>  create mode 100644 drivers/media/platform/qcom/vidc/core.c
>  create mode 100644 drivers/media/platform/qcom/vidc/core.h
>  create mode 100644 drivers/media/platform/qcom/vidc/helpers.c
>  create mode 100644 drivers/media/platform/qcom/vidc/helpers.h
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi.c
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi.h
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_cmds.c
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_cmds.h
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_helper.h
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_msgs.c
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_msgs.h
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.c
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus.h
>  create mode 100644 drivers/media/platform/qcom/vidc/hfi_venus_io.h
>  create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.c
>  create mode 100644 drivers/media/platform/qcom/vidc/int_bufs.h
>  create mode 100644 drivers/media/platform/qcom/vidc/load.c
>  create mode 100644 drivers/media/platform/qcom/vidc/load.h
>  create mode 100644 drivers/media/platform/qcom/vidc/mem.c
>  create mode 100644 drivers/media/platform/qcom/vidc/mem.h
>  create mode 100644 drivers/media/platform/qcom/vidc/resources.c
>  create mode 100644 drivers/media/platform/qcom/vidc/resources.h
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec.c
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec.h
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.c
>  create mode 100644 drivers/media/platform/qcom/vidc/vdec_ctrls.h
>  create mode 100644 drivers/media/platform/qcom/vidc/venc.c
>  create mode 100644 drivers/media/platform/qcom/vidc/venc.h
>  create mode 100644 drivers/media/platform/qcom/vidc/venc_ctrls.c
>  create mode 100644 drivers/media/platform/qcom/vidc/venc_ctrls.h
> 

Re: [PATCH 0/8] Qualcomm video decoder/encoder driver

[Stanimir Varbanov]

Hi Hans,

On 09/05/2016 05:47 PM, Hans Verkuil wrote:
> On 08/22/2016 03:13 PM, Stanimir Varbanov wrote:
>> This patchset introduces a basic support for Qualcomm video
>> acceleration hardware used for video stream decoding/encoding.
>> The video IP can found on various qcom SoCs like apq8084, msm8916
>> and msm8996, hence it is widly distributed but the driver is 
>> missing in the mainline.
>>
>> The v4l2 driver is something like a wrapper over Host Firmware
>> Interface. The HFI itself is a set of command and message packets
>> send/received through shared memory, and its purpose is to
>> comunicate with the firmware which is run on remote processor.
>> The Venus is the name of the video hardware IP that doing the
>> video acceleration.
>>
>> From the software point of view the HFI interface is implemented
>> in the files with prefix hfi_xxx. It acts as a translation layer
>> between HFI and v4l2 layer. There is one special file in the 
>> driver called hfi_venus which doing most of the driver
>> orchestration work. Something more it setups Venus core, run it
>> and handle commands and messages from low-level point of view with
>> the help of provided functions by HFI interface.
>>
>> I think that the driver is in good shape for mainline kernel, and
>> I hope the review comments will help to improve it, so please
>> do review and make comments.
> 
> I was hoping that I could finish reviewing this patch series today,
> but that didn't work out.
> 
> I have more time next week, but I wonder if it isn't better if you make a
> v2 first, taking my comments into account. Then I can review v2 next week.

OK, I have more of your comments addressed plus few of Bjorn's too.

> 
> Also test with the latest v4l2-compliance (i.e. as of today) since I improved
> a few tests relating to g/s_selection and g/s_parm.

Sure, I will retest and post the results in cover letter.

-- 
regards,
Stan