* [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
@ 2023-12-18 11:31 ` Dikshita Agarwal
2023-12-18 18:24 ` Dmitry Baryshkov
2023-12-19 11:40 ` Bryan O'Donoghue
2023-12-18 11:31 ` [PATCH v2 02/34] media: introduce common helpers for queues handling Dikshita Agarwal
` (39 subsequent siblings)
40 siblings, 2 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:31 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Re-organize the video driver code by introducing a new folder
'vcodec' and placing 'venus' driver code inside that.
Introduce common helpers for trustzone based firmware
load/unload etc. which are placed in common folder
i.e. 'vcodec'.
Use these helpers in 'venus' driver. These helpers will be
used by 'iris' driver as well which is introduced later
in this patch series.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/Kconfig | 2 +-
drivers/media/platform/qcom/Makefile | 2 +-
drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
.../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
.../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
.../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
.../media/platform/qcom/{ => vcodec}/venus/core.h | 0
.../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
.../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
.../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
.../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
.../platform/qcom/{ => vcodec}/venus/helpers.c | 0
.../platform/qcom/{ => vcodec}/venus/helpers.h | 0
.../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
.../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
.../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
.../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
.../qcom/{ => vcodec}/venus/hfi_platform.c | 0
.../qcom/{ => vcodec}/venus/hfi_platform.h | 0
.../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
.../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
.../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
.../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
.../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
.../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
.../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
.../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
.../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
.../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
.../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
.../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
drivers/media/platform/qcom/venus/firmware.c | 363 ---------------------
drivers/media/platform/qcom/venus/firmware.h | 26 --
42 files changed, 492 insertions(+), 409 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
index cc5799b..e94142f 100644
--- a/drivers/media/platform/qcom/Kconfig
+++ b/drivers/media/platform/qcom/Kconfig
@@ -3,4 +3,4 @@
comment "Qualcomm media platform drivers"
source "drivers/media/platform/qcom/camss/Kconfig"
-source "drivers/media/platform/qcom/venus/Kconfig"
+source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
index 4f055c3..3d2d82b 100644
--- a/drivers/media/platform/qcom/Makefile
+++ b/drivers/media/platform/qcom/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-y += camss/
-obj-y += venus/
+obj-y += vcodec/venus/
diff --git a/drivers/media/platform/qcom/vcodec/firmware.c b/drivers/media/platform/qcom/vcodec/firmware.c
new file mode 100644
index 0000000..dbc220a
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/firmware.c
@@ -0,0 +1,147 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/firmware.h>
+#include <linux/kernel.h>
+#include <linux/iommu.h>
+#include <linux/of_device.h>
+#include <linux/firmware/qcom/qcom_scm.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/soc/qcom/mdt_loader.h>
+
+#include "firmware.h"
+
+bool use_tz(struct device *core_dev)
+{
+ struct device_node *np;
+
+ np = of_get_child_by_name(core_dev->of_node, "video-firmware");
+ if (!np)
+ return true;
+
+ return false;
+}
+
+int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
+ u32 cp_nonpixel_size, u32 pas_id)
+{
+ int ret;
+ /*
+ * Clues for porting using downstream data:
+ * cp_start = 0
+ * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
+ * This works, as the non-secure context bank is placed
+ * contiguously right after the Content Protection region.
+ *
+ * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
+ * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
+ */
+ ret = qcom_scm_mem_protect_video_var(cp_start,
+ cp_size,
+ cp_nonpixel_start,
+ cp_nonpixel_size);
+ if (ret)
+ qcom_scm_pas_shutdown(pas_id);
+
+ return ret;
+}
+
+int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
+ size_t *mem_size, u32 pas_id, bool use_tz)
+{
+ const struct firmware *firmware = NULL;
+ struct reserved_mem *rmem;
+ struct device_node *node;
+ void *mem_virt = NULL;
+ ssize_t fw_size = 0;
+ int ret;
+
+ if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
+ (use_tz && !qcom_scm_is_available()))
+ return -EPROBE_DEFER;
+
+ if (!fw_name || !(*fw_name))
+ return -EINVAL;
+
+ *mem_phys = 0;
+ *mem_size = 0;
+
+ node = of_parse_phandle(dev->of_node, "memory-region", 0);
+ if (!node) {
+ dev_err(dev, "no memory-region specified\n");
+ return -EINVAL;
+ }
+
+ rmem = of_reserved_mem_lookup(node);
+ of_node_put(node);
+ if (!rmem) {
+ dev_err(dev, "failed to lookup reserved memory-region\n");
+ return -EINVAL;
+ }
+
+ ret = request_firmware(&firmware, fw_name, dev);
+ if (ret) {
+ dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
+ __func__, fw_name, ret);
+ return ret;
+ }
+
+ fw_size = qcom_mdt_get_size(firmware);
+ if (fw_size < 0) {
+ ret = fw_size;
+ dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
+ __func__, fw_size);
+ goto err_release_fw;
+ }
+
+ *mem_phys = rmem->base;
+ *mem_size = rmem->size;
+
+ if (*mem_size < fw_size) {
+ ret = -EINVAL;
+ goto err_release_fw;
+ }
+
+ mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
+ if (!mem_virt) {
+ dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
+ mem_phys, *mem_size);
+ goto err_release_fw;
+ }
+
+ if (use_tz)
+ ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
+ *mem_phys, *mem_size, NULL);
+ else
+ ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id, mem_virt,
+ *mem_phys, *mem_size, NULL);
+ if (ret) {
+ dev_err(dev, "%s: error %d loading fw \"%s\"\n",
+ __func__, ret, fw_name);
+ }
+
+ memunmap(mem_virt);
+err_release_fw:
+ release_firmware(firmware);
+ return ret;
+}
+
+int auth_reset_fw(u32 pas_id)
+{
+ return qcom_scm_pas_auth_and_reset(pas_id);
+}
+
+void unload_fw(u32 pas_id)
+{
+ qcom_scm_pas_shutdown(pas_id);
+}
+
+int set_hw_state(bool resume)
+{
+ return qcom_scm_set_remote_state(resume, 0);
+}
diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
new file mode 100644
index 0000000..7d410a8
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/firmware.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _FIRMWARE_H_
+#define _FIRMWARE_H_
+
+#include <linux/device.h>
+#include <linux/types.h>
+
+bool use_tz(struct device *core_dev);
+int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
+ size_t *mem_size, u32 pas_id, bool use_tz);
+int auth_reset_fw(u32 pas_id);
+int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
+ u32 cp_nonpixel_size, u32 pas_id);
+void unload_fw(u32 pas_id);
+int set_hw_state(bool resume);
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/Kconfig b/drivers/media/platform/qcom/vcodec/venus/Kconfig
similarity index 100%
rename from drivers/media/platform/qcom/venus/Kconfig
rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
similarity index 83%
rename from drivers/media/platform/qcom/venus/Makefile
rename to drivers/media/platform/qcom/vcodec/venus/Makefile
index 91ee6be..f6f3a88 100644
--- a/drivers/media/platform/qcom/venus/Makefile
+++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
@@ -1,7 +1,9 @@
# SPDX-License-Identifier: GPL-2.0
# Makefile for Qualcomm Venus driver
-venus-core-objs += core.o helpers.o firmware.o \
+venus-core-objs += ../firmware.o
+
+venus-core-objs += core.o helpers.o firmware_no_tz.o \
hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
hfi_parser.o pm_helpers.o dbgfs.o \
hfi_platform.o hfi_platform_v4.o \
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/vcodec/venus/core.c
similarity index 91%
rename from drivers/media/platform/qcom/venus/core.c
rename to drivers/media/platform/qcom/vcodec/venus/core.c
index 9cffe97..56d9a53 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/vcodec/venus/core.c
@@ -22,7 +22,8 @@
#include <media/v4l2-ioctl.h>
#include "core.h"
-#include "firmware.h"
+#include "../firmware.h"
+#include "firmware_no_tz.h"
#include "pm_helpers.h"
#include "hfi_venus_io.h"
@@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct work_struct *work)
struct venus_core *core =
container_of(work, struct venus_core, work.work);
int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
+ const struct venus_resources *res = core->res;
+ const char *fwpath = NULL;
const char *err_msg = "";
bool failed = false;
@@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct work_struct *work)
mutex_lock(&core->lock);
- venus_shutdown(core);
+ if (core->use_tz)
+ unload_fw(VENUS_PAS_ID);
+ else
+ unload_fw_no_tz(core);
venus_coredump(core);
@@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct work_struct *work)
failed = true;
}
- ret = venus_boot(core);
+ ret = of_property_read_string_index(core->dev->of_node, "firmware-name", 0,
+ &fwpath);
+ if (ret)
+ fwpath = core->res->fwname;
+
+ ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
+ VENUS_PAS_ID, core->use_tz);
if (ret && !failed) {
- err_msg = "boot Venus";
+ err_msg = "load FW";
failed = true;
}
+ if (core->use_tz)
+ ret = auth_reset_fw(VENUS_PAS_ID);
+ else
+ ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
+ if (ret && !failed) {
+ err_msg = "Auth and Reset";
+ failed = true;
+ }
+
+ if (core->use_tz && res->cp_size) {
+ ret = protect_secure_region(res->cp_start,
+ res->cp_size,
+ res->cp_nonpixel_start,
+ res->cp_nonpixel_size,
+ VENUS_PAS_ID);
+ if (ret && !failed) {
+ err_msg = "Protect CP Mem";
+ failed = true;
+ }
+ }
+
ret = hfi_core_resume(core, true);
if (ret && !failed) {
err_msg = "resume HFI";
@@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void *dev_id)
static int venus_probe(struct platform_device *pdev)
{
+ const struct venus_resources *res;
struct device *dev = &pdev->dev;
+ const char *fwpath = NULL;
struct venus_core *core;
int ret;
@@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev)
if (ret)
goto err_runtime_disable;
- ret = venus_firmware_init(core);
+ core->use_tz = use_tz(core->dev);
+
+ if (!core->use_tz) {
+ ret = init_fw_no_tz(core);
+ if (ret)
+ goto err_of_depopulate;
+ }
+
+ ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
+ &fwpath);
if (ret)
- goto err_of_depopulate;
+ fwpath = core->res->fwname;
- ret = venus_boot(core);
+ ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
+ VENUS_PAS_ID, core->use_tz);
if (ret)
goto err_firmware_deinit;
+ if (core->use_tz)
+ ret = auth_reset_fw(VENUS_PAS_ID);
+ else
+ ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
+ if (ret)
+ goto err_firmware_deinit;
+
+ res = core->res;
+ if (core->use_tz && res->cp_size) {
+ ret = protect_secure_region(res->cp_start,
+ res->cp_size,
+ res->cp_nonpixel_start,
+ res->cp_nonpixel_size,
+ VENUS_PAS_ID);
+ if (ret)
+ goto err_firmware_deinit;
+ }
+
ret = hfi_core_resume(core, true);
if (ret)
goto err_venus_shutdown;
@@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev)
err_dev_unregister:
v4l2_device_unregister(&core->v4l2_dev);
err_venus_shutdown:
- venus_shutdown(core);
+ if (core->use_tz)
+ unload_fw(VENUS_PAS_ID);
+ else
+ unload_fw_no_tz(core);
err_firmware_deinit:
- venus_firmware_deinit(core);
+ if (!core->use_tz)
+ deinit_fw_no_tz(core);
err_of_depopulate:
of_platform_depopulate(dev);
err_runtime_disable:
@@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev)
ret = hfi_core_deinit(core, true);
WARN_ON(ret);
- venus_shutdown(core);
+ if (core->use_tz)
+ unload_fw(VENUS_PAS_ID);
+ else
+ unload_fw_no_tz(core);
+
of_platform_depopulate(dev);
- venus_firmware_deinit(core);
+ if (!core->use_tz)
+ deinit_fw_no_tz(core);
pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
@@ -455,8 +527,12 @@ static void venus_core_shutdown(struct platform_device *pdev)
struct venus_core *core = platform_get_drvdata(pdev);
pm_runtime_get_sync(core->dev);
- venus_shutdown(core);
- venus_firmware_deinit(core);
+ if (core->use_tz) {
+ unload_fw(VENUS_PAS_ID);
+ } else {
+ unload_fw_no_tz(core);
+ deinit_fw_no_tz(core);
+ }
pm_runtime_put_sync(core->dev);
}
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/vcodec/venus/core.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/core.h
rename to drivers/media/platform/qcom/vcodec/venus/core.h
diff --git a/drivers/media/platform/qcom/venus/dbgfs.c b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/dbgfs.c
rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c
diff --git a/drivers/media/platform/qcom/venus/dbgfs.h b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/dbgfs.h
rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h
diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
new file mode 100644
index 0000000..9dca6e23
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
@@ -0,0 +1,194 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 Linaro Ltd.
+ */
+
+#include <linux/iommu.h>
+#include <linux/of_device.h>
+#include "core.h"
+#include "firmware_no_tz.h"
+#include "hfi_venus_io.h"
+
+#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
+#define VENUS_FW_START_ADDR 0x0
+
+int init_fw_no_tz(struct venus_core *core)
+{
+ struct platform_device_info info;
+ struct iommu_domain *iommu_dom;
+ struct platform_device *pdev;
+ struct device_node *np;
+ int ret;
+
+ np = of_get_child_by_name(core->dev->of_node, "video-firmware");
+
+ memset(&info, 0, sizeof(info));
+ info.fwnode = &np->fwnode;
+ info.parent = core->dev;
+ info.name = np->name;
+ info.dma_mask = DMA_BIT_MASK(32);
+
+ pdev = platform_device_register_full(&info);
+ if (IS_ERR(pdev)) {
+ of_node_put(np);
+ return PTR_ERR(pdev);
+ }
+
+ pdev->dev.of_node = np;
+
+ ret = of_dma_configure(&pdev->dev, np, true);
+ if (ret) {
+ dev_err(core->dev, "dma configure fail\n");
+ goto err_unregister;
+ }
+
+ core->fw.dev = &pdev->dev;
+
+ iommu_dom = iommu_domain_alloc(&platform_bus_type);
+ if (!iommu_dom) {
+ dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
+ ret = -ENOMEM;
+ goto err_unregister;
+ }
+
+ ret = iommu_attach_device(iommu_dom, core->fw.dev);
+ if (ret) {
+ dev_err(core->fw.dev, "could not attach device\n");
+ goto err_iommu_free;
+ }
+
+ core->fw.iommu_domain = iommu_dom;
+
+ of_node_put(np);
+
+ return 0;
+
+err_iommu_free:
+ iommu_domain_free(iommu_dom);
+err_unregister:
+ platform_device_unregister(pdev);
+ of_node_put(np);
+ return ret;
+}
+
+void deinit_fw_no_tz(struct venus_core *core)
+{
+ struct iommu_domain *iommu;
+
+ if (!core->fw.dev)
+ return;
+
+ iommu = core->fw.iommu_domain;
+
+ iommu_detach_device(iommu, core->fw.dev);
+
+ if (iommu) {
+ iommu_domain_free(iommu);
+ iommu = NULL;
+ }
+
+ platform_device_unregister(to_platform_device(core->fw.dev));
+}
+
+static void reset_cpu_no_tz(struct venus_core *core)
+{
+ u32 fw_size = core->fw.mapped_mem_size;
+ void __iomem *wrapper_base;
+
+ if (IS_IRIS2_1(core))
+ wrapper_base = core->wrapper_tz_base;
+ else
+ wrapper_base = core->wrapper_base;
+
+ writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
+ writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
+
+ if (IS_IRIS2_1(core)) {
+ /* Bring XTSS out of reset */
+ writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
+ } else {
+ writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
+ writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
+
+ /* Bring ARM9 out of reset */
+ writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
+ }
+}
+
+void set_hw_state_no_tz(struct venus_core *core, bool resume)
+{
+ if (resume) {
+ reset_cpu_no_tz(core);
+ } else {
+ if (IS_IRIS2_1(core))
+ writel(WRAPPER_XTSS_SW_RESET_BIT,
+ core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
+ else
+ writel(WRAPPER_A9SS_SW_RESET_BIT,
+ core->wrapper_base + WRAPPER_A9SS_SW_RESET);
+ }
+}
+
+int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
+ size_t mem_size)
+{
+ struct iommu_domain *iommu;
+ struct device *dev;
+ int ret;
+
+ dev = core->fw.dev;
+ if (!dev)
+ return -EPROBE_DEFER;
+
+ iommu = core->fw.iommu_domain;
+ core->fw.mapped_mem_size = mem_size;
+
+ ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
+ IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
+ if (ret) {
+ dev_err(dev, "could not map video firmware region\n");
+ return ret;
+ }
+
+ reset_cpu_no_tz(core);
+
+ return 0;
+}
+
+void unload_fw_no_tz(struct venus_core *core)
+{
+ const size_t mapped = core->fw.mapped_mem_size;
+ struct iommu_domain *iommu;
+ size_t unmapped;
+ u32 reg;
+ struct device *dev = core->fw.dev;
+ void __iomem *wrapper_base = core->wrapper_base;
+ void __iomem *wrapper_tz_base = core->wrapper_tz_base;
+
+ if (IS_IRIS2_1(core)) {
+ /* Assert the reset to XTSS */
+ reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
+ reg |= WRAPPER_XTSS_SW_RESET_BIT;
+ writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
+ } else {
+ /* Assert the reset to ARM9 */
+ reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
+ reg |= WRAPPER_A9SS_SW_RESET_BIT;
+ writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
+ }
+
+ iommu = core->fw.iommu_domain;
+
+ if (core->fw.mapped_mem_size && iommu) {
+ unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
+
+ if (unmapped != mapped)
+ dev_err(dev, "failed to unmap firmware\n");
+ else
+ core->fw.mapped_mem_size = 0;
+ }
+}
diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
new file mode 100644
index 0000000..5f008ef
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017 Linaro Ltd.
+ */
+#ifndef __FIRMWARE_NO_TZ_H__
+#define __FIRMWARE_NO_TZ_H__
+
+struct device;
+
+#define VENUS_PAS_ID 9
+
+int init_fw_no_tz(struct venus_core *core);
+void deinit_fw_no_tz(struct venus_core *core);
+int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
+ size_t mem_size);
+void unload_fw_no_tz(struct venus_core *core);
+void set_hw_state_no_tz(struct venus_core *core, bool resume);
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/helpers.c
rename to drivers/media/platform/qcom/vcodec/venus/helpers.c
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/helpers.h
rename to drivers/media/platform/qcom/vcodec/venus/helpers.h
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/vcodec/venus/hfi.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi.c
diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/vcodec/venus/hfi.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi.h
diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_cmds.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_cmds.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_helper.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_msgs.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_msgs.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_parser.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_parser.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
similarity index 99%
rename from drivers/media/platform/qcom/venus/hfi_venus.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
index f9437b6..5a68db9 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
@@ -13,11 +13,12 @@
#include <linux/slab.h>
#include "core.h"
+#include "../firmware.h"
#include "hfi_cmds.h"
#include "hfi_msgs.h"
#include "hfi_venus.h"
#include "hfi_venus_io.h"
-#include "firmware.h"
+#include "firmware_no_tz.h"
#define HFI_MASK_QHDR_TX_TYPE 0xff000000
#define HFI_MASK_QHDR_RX_TYPE 0x00ff0000
@@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device *hdev)
if (!hdev->power_enabled)
return 0;
- ret = venus_set_hw_state_suspend(hdev->core);
+ if (hdev->core->use_tz)
+ ret = set_hw_state(false);
+ else
+ set_hw_state_no_tz(hdev->core, false);
if (ret)
return ret;
@@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device *hdev)
if (hdev->power_enabled)
return 0;
- ret = venus_set_hw_state_resume(hdev->core);
+ if (hdev->core->use_tz) {
+ ret = set_hw_state(true);
+ if (ret == -EINVAL)
+ ret = 0;
+ } else {
+ set_hw_state_no_tz(hdev->core, true);
+ }
if (ret)
goto err;
@@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device *hdev)
return 0;
err_suspend:
- venus_set_hw_state_suspend(hdev->core);
+ if (hdev->core->use_tz)
+ set_hw_state(false);
+ else
+ set_hw_state_no_tz(hdev->core, false);
err:
hdev->power_enabled = false;
return ret;
diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_venus.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_venus_io.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/pm_helpers.c
rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/pm_helpers.h
rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/vdec.c
rename to drivers/media/platform/qcom/vcodec/venus/vdec.c
diff --git a/drivers/media/platform/qcom/venus/vdec.h b/drivers/media/platform/qcom/vcodec/venus/vdec.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/vdec.h
rename to drivers/media/platform/qcom/vcodec/venus/vdec.h
diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/vdec_ctrls.c
rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/vcodec/venus/venc.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/venc.c
rename to drivers/media/platform/qcom/vcodec/venus/venc.c
diff --git a/drivers/media/platform/qcom/venus/venc.h b/drivers/media/platform/qcom/vcodec/venus/venc.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/venc.h
rename to drivers/media/platform/qcom/vcodec/venus/venc.h
diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/venc_ctrls.c
rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
diff --git a/drivers/media/platform/qcom/venus/firmware.c b/drivers/media/platform/qcom/venus/firmware.c
deleted file mode 100644
index fe7da2b..0000000
--- a/drivers/media/platform/qcom/venus/firmware.c
+++ /dev/null
@@ -1,363 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (C) 2017 Linaro Ltd.
- */
-
-#include <linux/device.h>
-#include <linux/firmware.h>
-#include <linux/kernel.h>
-#include <linux/iommu.h>
-#include <linux/io.h>
-#include <linux/of.h>
-#include <linux/of_address.h>
-#include <linux/of_reserved_mem.h>
-#include <linux/platform_device.h>
-#include <linux/of_device.h>
-#include <linux/firmware/qcom/qcom_scm.h>
-#include <linux/sizes.h>
-#include <linux/soc/qcom/mdt_loader.h>
-
-#include "core.h"
-#include "firmware.h"
-#include "hfi_venus_io.h"
-
-#define VENUS_PAS_ID 9
-#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
-#define VENUS_FW_START_ADDR 0x0
-
-static void venus_reset_cpu(struct venus_core *core)
-{
- u32 fw_size = core->fw.mapped_mem_size;
- void __iomem *wrapper_base;
-
- if (IS_IRIS2_1(core))
- wrapper_base = core->wrapper_tz_base;
- else
- wrapper_base = core->wrapper_base;
-
- writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
- writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
-
- if (IS_IRIS2_1(core)) {
- /* Bring XTSS out of reset */
- writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
- } else {
- writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
- writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
-
- /* Bring ARM9 out of reset */
- writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
- }
-}
-
-int venus_set_hw_state(struct venus_core *core, bool resume)
-{
- int ret;
-
- if (core->use_tz) {
- ret = qcom_scm_set_remote_state(resume, 0);
- if (resume && ret == -EINVAL)
- ret = 0;
- return ret;
- }
-
- if (resume) {
- venus_reset_cpu(core);
- } else {
- if (IS_IRIS2_1(core))
- writel(WRAPPER_XTSS_SW_RESET_BIT,
- core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
- else
- writel(WRAPPER_A9SS_SW_RESET_BIT,
- core->wrapper_base + WRAPPER_A9SS_SW_RESET);
- }
-
- return 0;
-}
-
-static int venus_load_fw(struct venus_core *core, const char *fwname,
- phys_addr_t *mem_phys, size_t *mem_size)
-{
- const struct firmware *mdt;
- struct reserved_mem *rmem;
- struct device_node *node;
- struct device *dev;
- ssize_t fw_size;
- void *mem_va;
- int ret;
-
- *mem_phys = 0;
- *mem_size = 0;
-
- dev = core->dev;
- node = of_parse_phandle(dev->of_node, "memory-region", 0);
- if (!node) {
- dev_err(dev, "no memory-region specified\n");
- return -EINVAL;
- }
-
- rmem = of_reserved_mem_lookup(node);
- of_node_put(node);
- if (!rmem) {
- dev_err(dev, "failed to lookup reserved memory-region\n");
- return -EINVAL;
- }
-
- ret = request_firmware(&mdt, fwname, dev);
- if (ret < 0)
- return ret;
-
- fw_size = qcom_mdt_get_size(mdt);
- if (fw_size < 0) {
- ret = fw_size;
- goto err_release_fw;
- }
-
- *mem_phys = rmem->base;
- *mem_size = rmem->size;
-
- if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) {
- ret = -EINVAL;
- goto err_release_fw;
- }
-
- mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
- if (!mem_va) {
- dev_err(dev, "unable to map memory region %pa size %#zx\n", mem_phys, *mem_size);
- ret = -ENOMEM;
- goto err_release_fw;
- }
-
- if (core->use_tz)
- ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID,
- mem_va, *mem_phys, *mem_size, NULL);
- else
- ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID,
- mem_va, *mem_phys, *mem_size, NULL);
-
- memunmap(mem_va);
-err_release_fw:
- release_firmware(mdt);
- return ret;
-}
-
-static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys,
- size_t mem_size)
-{
- struct iommu_domain *iommu;
- struct device *dev;
- int ret;
-
- dev = core->fw.dev;
- if (!dev)
- return -EPROBE_DEFER;
-
- iommu = core->fw.iommu_domain;
- core->fw.mapped_mem_size = mem_size;
-
- ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
- IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
- if (ret) {
- dev_err(dev, "could not map video firmware region\n");
- return ret;
- }
-
- venus_reset_cpu(core);
-
- return 0;
-}
-
-static int venus_shutdown_no_tz(struct venus_core *core)
-{
- const size_t mapped = core->fw.mapped_mem_size;
- struct iommu_domain *iommu;
- size_t unmapped;
- u32 reg;
- struct device *dev = core->fw.dev;
- void __iomem *wrapper_base = core->wrapper_base;
- void __iomem *wrapper_tz_base = core->wrapper_tz_base;
-
- if (IS_IRIS2_1(core)) {
- /* Assert the reset to XTSS */
- reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
- reg |= WRAPPER_XTSS_SW_RESET_BIT;
- writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
- } else {
- /* Assert the reset to ARM9 */
- reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
- reg |= WRAPPER_A9SS_SW_RESET_BIT;
- writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
- }
-
- iommu = core->fw.iommu_domain;
-
- if (core->fw.mapped_mem_size && iommu) {
- unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
-
- if (unmapped != mapped)
- dev_err(dev, "failed to unmap firmware\n");
- else
- core->fw.mapped_mem_size = 0;
- }
-
- return 0;
-}
-
-int venus_boot(struct venus_core *core)
-{
- struct device *dev = core->dev;
- const struct venus_resources *res = core->res;
- const char *fwpath = NULL;
- phys_addr_t mem_phys;
- size_t mem_size;
- int ret;
-
- if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
- (core->use_tz && !qcom_scm_is_available()))
- return -EPROBE_DEFER;
-
- ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
- &fwpath);
- if (ret)
- fwpath = core->res->fwname;
-
- ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size);
- if (ret) {
- dev_err(dev, "fail to load video firmware\n");
- return -EINVAL;
- }
-
- core->fw.mem_size = mem_size;
- core->fw.mem_phys = mem_phys;
-
- if (core->use_tz)
- ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID);
- else
- ret = venus_boot_no_tz(core, mem_phys, mem_size);
-
- if (ret)
- return ret;
-
- if (core->use_tz && res->cp_size) {
- /*
- * Clues for porting using downstream data:
- * cp_start = 0
- * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
- * This works, as the non-secure context bank is placed
- * contiguously right after the Content Protection region.
- *
- * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
- * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
- */
- ret = qcom_scm_mem_protect_video_var(res->cp_start,
- res->cp_size,
- res->cp_nonpixel_start,
- res->cp_nonpixel_size);
- if (ret) {
- qcom_scm_pas_shutdown(VENUS_PAS_ID);
- dev_err(dev, "set virtual address ranges fail (%d)\n",
- ret);
- return ret;
- }
- }
-
- return 0;
-}
-
-int venus_shutdown(struct venus_core *core)
-{
- int ret;
-
- if (core->use_tz)
- ret = qcom_scm_pas_shutdown(VENUS_PAS_ID);
- else
- ret = venus_shutdown_no_tz(core);
-
- return ret;
-}
-
-int venus_firmware_init(struct venus_core *core)
-{
- struct platform_device_info info;
- struct iommu_domain *iommu_dom;
- struct platform_device *pdev;
- struct device_node *np;
- int ret;
-
- np = of_get_child_by_name(core->dev->of_node, "video-firmware");
- if (!np) {
- core->use_tz = true;
- return 0;
- }
-
- memset(&info, 0, sizeof(info));
- info.fwnode = &np->fwnode;
- info.parent = core->dev;
- info.name = np->name;
- info.dma_mask = DMA_BIT_MASK(32);
-
- pdev = platform_device_register_full(&info);
- if (IS_ERR(pdev)) {
- of_node_put(np);
- return PTR_ERR(pdev);
- }
-
- pdev->dev.of_node = np;
-
- ret = of_dma_configure(&pdev->dev, np, true);
- if (ret) {
- dev_err(core->dev, "dma configure fail\n");
- goto err_unregister;
- }
-
- core->fw.dev = &pdev->dev;
-
- iommu_dom = iommu_domain_alloc(&platform_bus_type);
- if (!iommu_dom) {
- dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
- ret = -ENOMEM;
- goto err_unregister;
- }
-
- ret = iommu_attach_device(iommu_dom, core->fw.dev);
- if (ret) {
- dev_err(core->fw.dev, "could not attach device\n");
- goto err_iommu_free;
- }
-
- core->fw.iommu_domain = iommu_dom;
-
- of_node_put(np);
-
- return 0;
-
-err_iommu_free:
- iommu_domain_free(iommu_dom);
-err_unregister:
- platform_device_unregister(pdev);
- of_node_put(np);
- return ret;
-}
-
-void venus_firmware_deinit(struct venus_core *core)
-{
- struct iommu_domain *iommu;
-
- if (!core->fw.dev)
- return;
-
- iommu = core->fw.iommu_domain;
-
- iommu_detach_device(iommu, core->fw.dev);
-
- if (core->fw.iommu_domain) {
- iommu_domain_free(iommu);
- core->fw.iommu_domain = NULL;
- }
-
- platform_device_unregister(to_platform_device(core->fw.dev));
-}
diff --git a/drivers/media/platform/qcom/venus/firmware.h b/drivers/media/platform/qcom/venus/firmware.h
deleted file mode 100644
index aaccd84..0000000
--- a/drivers/media/platform/qcom/venus/firmware.h
+++ /dev/null
@@ -1,26 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (C) 2017 Linaro Ltd.
- */
-#ifndef __VENUS_FIRMWARE_H__
-#define __VENUS_FIRMWARE_H__
-
-struct device;
-
-int venus_firmware_init(struct venus_core *core);
-void venus_firmware_deinit(struct venus_core *core);
-int venus_boot(struct venus_core *core);
-int venus_shutdown(struct venus_core *core);
-int venus_set_hw_state(struct venus_core *core, bool suspend);
-
-static inline int venus_set_hw_state_suspend(struct venus_core *core)
-{
- return venus_set_hw_state(core, false);
-}
-
-static inline int venus_set_hw_state_resume(struct venus_core *core)
-{
- return venus_set_hw_state(core, true);
-}
-
-#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-18 11:31 ` [PATCH v2 01/34] media: introduce common helpers for video firmware handling Dikshita Agarwal
@ 2023-12-18 18:24 ` Dmitry Baryshkov
2023-12-20 8:01 ` Dikshita Agarwal
2023-12-19 11:40 ` Bryan O'Donoghue
1 sibling, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:24 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:31, Dikshita Agarwal wrote:
> Re-organize the video driver code by introducing a new folder
> 'vcodec' and placing 'venus' driver code inside that.
>
> Introduce common helpers for trustzone based firmware
> load/unload etc. which are placed in common folder
> i.e. 'vcodec'.
> Use these helpers in 'venus' driver. These helpers will be
> used by 'iris' driver as well which is introduced later
> in this patch series.
But why do you need to move the venus driver to subdir?
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> drivers/media/platform/qcom/Kconfig | 2 +-
> drivers/media/platform/qcom/Makefile | 2 +-
> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ---------------------
> drivers/media/platform/qcom/venus/firmware.h | 26 --
> 42 files changed, 492 insertions(+), 409 deletions(-)
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>
> diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
> index cc5799b..e94142f 100644
> --- a/drivers/media/platform/qcom/Kconfig
> +++ b/drivers/media/platform/qcom/Kconfig
> @@ -3,4 +3,4 @@
> comment "Qualcomm media platform drivers"
>
> source "drivers/media/platform/qcom/camss/Kconfig"
> -source "drivers/media/platform/qcom/venus/Kconfig"
> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
> index 4f055c3..3d2d82b 100644
> --- a/drivers/media/platform/qcom/Makefile
> +++ b/drivers/media/platform/qcom/Makefile
> @@ -1,3 +1,3 @@
> # SPDX-License-Identifier: GPL-2.0-only
> obj-y += camss/
> -obj-y += venus/
> +obj-y += vcodec/venus/
> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c b/drivers/media/platform/qcom/vcodec/firmware.c
> new file mode 100644
> index 0000000..dbc220a
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
> @@ -0,0 +1,147 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/firmware.h>
> +#include <linux/kernel.h>
> +#include <linux/iommu.h>
> +#include <linux/of_device.h>
> +#include <linux/firmware/qcom/qcom_scm.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/platform_device.h>
> +#include <linux/soc/qcom/mdt_loader.h>
> +
> +#include "firmware.h"
> +
> +bool use_tz(struct device *core_dev)
All these functions must get some sane prefix. Otherwise a generic
'use_tz' function is too polluting for the global namespace.
> +{
> + struct device_node *np;
> +
> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
> + if (!np)
> + return true;
> +
> + return false;
> +}
> +
> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> + u32 cp_nonpixel_size, u32 pas_id)
> +{
> + int ret;
> + /*
> + * Clues for porting using downstream data:
> + * cp_start = 0
> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> + * This works, as the non-secure context bank is placed
> + * contiguously right after the Content Protection region.
> + *
> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> + */
> + ret = qcom_scm_mem_protect_video_var(cp_start,
> + cp_size,
> + cp_nonpixel_start,
> + cp_nonpixel_size);
> + if (ret)
> + qcom_scm_pas_shutdown(pas_id);
> +
> + return ret;
> +}
> +
> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> + size_t *mem_size, u32 pas_id, bool use_tz)
> +{
> + const struct firmware *firmware = NULL;
> + struct reserved_mem *rmem;
> + struct device_node *node;
> + void *mem_virt = NULL;
> + ssize_t fw_size = 0;
> + int ret;
> +
> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
Why? Can you just depend on it?
> + (use_tz && !qcom_scm_is_available()))
> + return -EPROBE_DEFER;
> +
> + if (!fw_name || !(*fw_name))
> + return -EINVAL;
> +
> + *mem_phys = 0;
> + *mem_size = 0;
> +
> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
> + if (!node) {
> + dev_err(dev, "no memory-region specified\n");
> + return -EINVAL;
> + }
> +
> + rmem = of_reserved_mem_lookup(node);
> + of_node_put(node);
> + if (!rmem) {
> + dev_err(dev, "failed to lookup reserved memory-region\n");
> + return -EINVAL;
> + }
> +
> + ret = request_firmware(&firmware, fw_name, dev);
> + if (ret) {
> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
> + __func__, fw_name, ret);
> + return ret;
> + }
> +
> + fw_size = qcom_mdt_get_size(firmware);
> + if (fw_size < 0) {
> + ret = fw_size;
> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
> + __func__, fw_size);
> + goto err_release_fw;
> + }
> +
> + *mem_phys = rmem->base;
> + *mem_size = rmem->size;
> +
> + if (*mem_size < fw_size) {
> + ret = -EINVAL;
> + goto err_release_fw;
> + }
> +
> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> + if (!mem_virt) {
> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
> + mem_phys, *mem_size);
> + goto err_release_fw;
> + }
> +
> + if (use_tz)
> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
> + *mem_phys, *mem_size, NULL);
> + else
> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id, mem_virt,
> + *mem_phys, *mem_size, NULL);
> + if (ret) {
> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
> + __func__, ret, fw_name);
> + }
> +
> + memunmap(mem_virt);
> +err_release_fw:
> + release_firmware(firmware);
> + return ret;
> +}
> +
> +int auth_reset_fw(u32 pas_id)
> +{
> + return qcom_scm_pas_auth_and_reset(pas_id);
> +}
> +
> +void unload_fw(u32 pas_id)
> +{
> + qcom_scm_pas_shutdown(pas_id);
> +}
> +
> +int set_hw_state(bool resume)
> +{
> + return qcom_scm_set_remote_state(resume, 0);
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
> new file mode 100644
> index 0000000..7d410a8
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _FIRMWARE_H_
> +#define _FIRMWARE_H_
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +
> +bool use_tz(struct device *core_dev);
> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> + size_t *mem_size, u32 pas_id, bool use_tz);
> +int auth_reset_fw(u32 pas_id);
> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> + u32 cp_nonpixel_size, u32 pas_id);
> +void unload_fw(u32 pas_id);
> +int set_hw_state(bool resume);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/venus/Kconfig b/drivers/media/platform/qcom/vcodec/venus/Kconfig
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/Kconfig
> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
> diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
> similarity index 83%
> rename from drivers/media/platform/qcom/venus/Makefile
> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
> index 91ee6be..f6f3a88 100644
> --- a/drivers/media/platform/qcom/venus/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> @@ -1,7 +1,9 @@
> # SPDX-License-Identifier: GPL-2.0
> # Makefile for Qualcomm Venus driver
>
> -venus-core-objs += core.o helpers.o firmware.o \
> +venus-core-objs += ../firmware.o
> +
> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> hfi_parser.o pm_helpers.o dbgfs.o \
> hfi_platform.o hfi_platform_v4.o \
> diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/vcodec/venus/core.c
> similarity index 91%
> rename from drivers/media/platform/qcom/venus/core.c
> rename to drivers/media/platform/qcom/vcodec/venus/core.c
> index 9cffe97..56d9a53 100644
> --- a/drivers/media/platform/qcom/venus/core.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
> @@ -22,7 +22,8 @@
> #include <media/v4l2-ioctl.h>
>
> #include "core.h"
> -#include "firmware.h"
> +#include "../firmware.h"
> +#include "firmware_no_tz.h"
> #include "pm_helpers.h"
> #include "hfi_venus_io.h"
>
> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct work_struct *work)
> struct venus_core *core =
> container_of(work, struct venus_core, work.work);
> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
> + const struct venus_resources *res = core->res;
> + const char *fwpath = NULL;
> const char *err_msg = "";
> bool failed = false;
>
> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct work_struct *work)
>
> mutex_lock(&core->lock);
>
> - venus_shutdown(core);
> + if (core->use_tz)
> + unload_fw(VENUS_PAS_ID);
> + else
> + unload_fw_no_tz(core);
This is more than introducing helpers.
>
> venus_coredump(core);
>
> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct work_struct *work)
> failed = true;
> }
>
> - ret = venus_boot(core);
> + ret = of_property_read_string_index(core->dev->of_node, "firmware-name", 0,
> + &fwpath);
> + if (ret)
> + fwpath = core->res->fwname;
> +
> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
> + VENUS_PAS_ID, core->use_tz);
So, we had a nice local 'venus_boot'. Instead we now have a pile of code
with non-generic prefixes, etc. If you are introducing helpers, please
refrain from inlining of calling functions, etc. Just move the code to
your helpers.
NAK for the rest of the patch.
> if (ret && !failed) {
> - err_msg = "boot Venus";
> + err_msg = "load FW";
> failed = true;
> }
>
> + if (core->use_tz)
> + ret = auth_reset_fw(VENUS_PAS_ID);
> + else
> + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
> + if (ret && !failed) {
> + err_msg = "Auth and Reset";
> + failed = true;
> + }
> +
> + if (core->use_tz && res->cp_size) {
> + ret = protect_secure_region(res->cp_start,
> + res->cp_size,
> + res->cp_nonpixel_start,
> + res->cp_nonpixel_size,
> + VENUS_PAS_ID);
> + if (ret && !failed) {
> + err_msg = "Protect CP Mem";
> + failed = true;
> + }
> + }
> +
> ret = hfi_core_resume(core, true);
> if (ret && !failed) {
> err_msg = "resume HFI";
> @@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void *dev_id)
>
> static int venus_probe(struct platform_device *pdev)
> {
> + const struct venus_resources *res;
> struct device *dev = &pdev->dev;
> + const char *fwpath = NULL;
> struct venus_core *core;
> int ret;
>
> @@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev)
> if (ret)
> goto err_runtime_disable;
>
> - ret = venus_firmware_init(core);
> + core->use_tz = use_tz(core->dev);
> +
> + if (!core->use_tz) {
> + ret = init_fw_no_tz(core);
> + if (ret)
> + goto err_of_depopulate;
> + }
> +
> + ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
> + &fwpath);
> if (ret)
> - goto err_of_depopulate;
> + fwpath = core->res->fwname;
>
> - ret = venus_boot(core);
> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
> + VENUS_PAS_ID, core->use_tz);
> if (ret)
> goto err_firmware_deinit;
>
> + if (core->use_tz)
> + ret = auth_reset_fw(VENUS_PAS_ID);
> + else
> + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
> + if (ret)
> + goto err_firmware_deinit;
> +
> + res = core->res;
> + if (core->use_tz && res->cp_size) {
> + ret = protect_secure_region(res->cp_start,
> + res->cp_size,
> + res->cp_nonpixel_start,
> + res->cp_nonpixel_size,
> + VENUS_PAS_ID);
> + if (ret)
> + goto err_firmware_deinit;
> + }
> +
> ret = hfi_core_resume(core, true);
> if (ret)
> goto err_venus_shutdown;
> @@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev)
> err_dev_unregister:
> v4l2_device_unregister(&core->v4l2_dev);
> err_venus_shutdown:
> - venus_shutdown(core);
> + if (core->use_tz)
> + unload_fw(VENUS_PAS_ID);
> + else
> + unload_fw_no_tz(core);
> err_firmware_deinit:
> - venus_firmware_deinit(core);
> + if (!core->use_tz)
> + deinit_fw_no_tz(core);
> err_of_depopulate:
> of_platform_depopulate(dev);
> err_runtime_disable:
> @@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev)
> ret = hfi_core_deinit(core, true);
> WARN_ON(ret);
>
> - venus_shutdown(core);
> + if (core->use_tz)
> + unload_fw(VENUS_PAS_ID);
> + else
> + unload_fw_no_tz(core);
> +
> of_platform_depopulate(dev);
>
> - venus_firmware_deinit(core);
> + if (!core->use_tz)
> + deinit_fw_no_tz(core);
>
> pm_runtime_put_sync(dev);
> pm_runtime_disable(dev);
> @@ -455,8 +527,12 @@ static void venus_core_shutdown(struct platform_device *pdev)
> struct venus_core *core = platform_get_drvdata(pdev);
>
> pm_runtime_get_sync(core->dev);
> - venus_shutdown(core);
> - venus_firmware_deinit(core);
> + if (core->use_tz) {
> + unload_fw(VENUS_PAS_ID);
> + } else {
> + unload_fw_no_tz(core);
> + deinit_fw_no_tz(core);
> + }
> pm_runtime_put_sync(core->dev);
> }
>
> diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/vcodec/venus/core.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/core.h
> rename to drivers/media/platform/qcom/vcodec/venus/core.h
> diff --git a/drivers/media/platform/qcom/venus/dbgfs.c b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/dbgfs.c
> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c
> diff --git a/drivers/media/platform/qcom/venus/dbgfs.h b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/dbgfs.h
> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h
> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> new file mode 100644
> index 0000000..9dca6e23
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> @@ -0,0 +1,194 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017 Linaro Ltd.
> + */
> +
> +#include <linux/iommu.h>
> +#include <linux/of_device.h>
> +#include "core.h"
> +#include "firmware_no_tz.h"
> +#include "hfi_venus_io.h"
> +
> +#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
> +#define VENUS_FW_START_ADDR 0x0
> +
> +int init_fw_no_tz(struct venus_core *core)
> +{
> + struct platform_device_info info;
> + struct iommu_domain *iommu_dom;
> + struct platform_device *pdev;
> + struct device_node *np;
> + int ret;
> +
> + np = of_get_child_by_name(core->dev->of_node, "video-firmware");
> +
> + memset(&info, 0, sizeof(info));
> + info.fwnode = &np->fwnode;
> + info.parent = core->dev;
> + info.name = np->name;
> + info.dma_mask = DMA_BIT_MASK(32);
> +
> + pdev = platform_device_register_full(&info);
> + if (IS_ERR(pdev)) {
> + of_node_put(np);
> + return PTR_ERR(pdev);
> + }
> +
> + pdev->dev.of_node = np;
> +
> + ret = of_dma_configure(&pdev->dev, np, true);
> + if (ret) {
> + dev_err(core->dev, "dma configure fail\n");
> + goto err_unregister;
> + }
> +
> + core->fw.dev = &pdev->dev;
> +
> + iommu_dom = iommu_domain_alloc(&platform_bus_type);
> + if (!iommu_dom) {
> + dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
> + ret = -ENOMEM;
> + goto err_unregister;
> + }
> +
> + ret = iommu_attach_device(iommu_dom, core->fw.dev);
> + if (ret) {
> + dev_err(core->fw.dev, "could not attach device\n");
> + goto err_iommu_free;
> + }
> +
> + core->fw.iommu_domain = iommu_dom;
> +
> + of_node_put(np);
> +
> + return 0;
> +
> +err_iommu_free:
> + iommu_domain_free(iommu_dom);
> +err_unregister:
> + platform_device_unregister(pdev);
> + of_node_put(np);
> + return ret;
> +}
> +
> +void deinit_fw_no_tz(struct venus_core *core)
> +{
> + struct iommu_domain *iommu;
> +
> + if (!core->fw.dev)
> + return;
> +
> + iommu = core->fw.iommu_domain;
> +
> + iommu_detach_device(iommu, core->fw.dev);
> +
> + if (iommu) {
> + iommu_domain_free(iommu);
> + iommu = NULL;
> + }
> +
> + platform_device_unregister(to_platform_device(core->fw.dev));
> +}
> +
> +static void reset_cpu_no_tz(struct venus_core *core)
> +{
> + u32 fw_size = core->fw.mapped_mem_size;
> + void __iomem *wrapper_base;
> +
> + if (IS_IRIS2_1(core))
> + wrapper_base = core->wrapper_tz_base;
> + else
> + wrapper_base = core->wrapper_base;
> +
> + writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
> + writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
> +
> + if (IS_IRIS2_1(core)) {
> + /* Bring XTSS out of reset */
> + writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
> + } else {
> + writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
> + writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
> +
> + /* Bring ARM9 out of reset */
> + writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
> + }
> +}
> +
> +void set_hw_state_no_tz(struct venus_core *core, bool resume)
> +{
> + if (resume) {
> + reset_cpu_no_tz(core);
> + } else {
> + if (IS_IRIS2_1(core))
> + writel(WRAPPER_XTSS_SW_RESET_BIT,
> + core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> + else
> + writel(WRAPPER_A9SS_SW_RESET_BIT,
> + core->wrapper_base + WRAPPER_A9SS_SW_RESET);
> + }
> +}
> +
> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
> + size_t mem_size)
> +{
> + struct iommu_domain *iommu;
> + struct device *dev;
> + int ret;
> +
> + dev = core->fw.dev;
> + if (!dev)
> + return -EPROBE_DEFER;
> +
> + iommu = core->fw.iommu_domain;
> + core->fw.mapped_mem_size = mem_size;
> +
> + ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
> + IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
> + if (ret) {
> + dev_err(dev, "could not map video firmware region\n");
> + return ret;
> + }
> +
> + reset_cpu_no_tz(core);
> +
> + return 0;
> +}
> +
> +void unload_fw_no_tz(struct venus_core *core)
> +{
> + const size_t mapped = core->fw.mapped_mem_size;
> + struct iommu_domain *iommu;
> + size_t unmapped;
> + u32 reg;
> + struct device *dev = core->fw.dev;
> + void __iomem *wrapper_base = core->wrapper_base;
> + void __iomem *wrapper_tz_base = core->wrapper_tz_base;
> +
> + if (IS_IRIS2_1(core)) {
> + /* Assert the reset to XTSS */
> + reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> + reg |= WRAPPER_XTSS_SW_RESET_BIT;
> + writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> + } else {
> + /* Assert the reset to ARM9 */
> + reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
> + reg |= WRAPPER_A9SS_SW_RESET_BIT;
> + writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
> + }
> +
> + iommu = core->fw.iommu_domain;
> +
> + if (core->fw.mapped_mem_size && iommu) {
> + unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
> +
> + if (unmapped != mapped)
> + dev_err(dev, "failed to unmap firmware\n");
> + else
> + core->fw.mapped_mem_size = 0;
> + }
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> new file mode 100644
> index 0000000..5f008ef
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> @@ -0,0 +1,19 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017 Linaro Ltd.
> + */
> +#ifndef __FIRMWARE_NO_TZ_H__
> +#define __FIRMWARE_NO_TZ_H__
> +
> +struct device;
> +
> +#define VENUS_PAS_ID 9
> +
> +int init_fw_no_tz(struct venus_core *core);
> +void deinit_fw_no_tz(struct venus_core *core);
> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
> + size_t mem_size);
> +void unload_fw_no_tz(struct venus_core *core);
> +void set_hw_state_no_tz(struct venus_core *core, bool resume);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/helpers.c
> rename to drivers/media/platform/qcom/vcodec/venus/helpers.c
> diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/helpers.h
> rename to drivers/media/platform/qcom/vcodec/venus/helpers.h
> diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/vcodec/venus/hfi.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi.c
> diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/vcodec/venus/hfi.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_cmds.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_cmds.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_helper.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_msgs.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_msgs.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_parser.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_parser.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> similarity index 99%
> rename from drivers/media/platform/qcom/venus/hfi_venus.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> index f9437b6..5a68db9 100644
> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> @@ -13,11 +13,12 @@
> #include <linux/slab.h>
>
> #include "core.h"
> +#include "../firmware.h"
> #include "hfi_cmds.h"
> #include "hfi_msgs.h"
> #include "hfi_venus.h"
> #include "hfi_venus_io.h"
> -#include "firmware.h"
> +#include "firmware_no_tz.h"
>
> #define HFI_MASK_QHDR_TX_TYPE 0xff000000
> #define HFI_MASK_QHDR_RX_TYPE 0x00ff0000
> @@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device *hdev)
> if (!hdev->power_enabled)
> return 0;
>
> - ret = venus_set_hw_state_suspend(hdev->core);
> + if (hdev->core->use_tz)
> + ret = set_hw_state(false);
> + else
> + set_hw_state_no_tz(hdev->core, false);
> if (ret)
> return ret;
>
> @@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device *hdev)
> if (hdev->power_enabled)
> return 0;
>
> - ret = venus_set_hw_state_resume(hdev->core);
> + if (hdev->core->use_tz) {
> + ret = set_hw_state(true);
> + if (ret == -EINVAL)
> + ret = 0;
> + } else {
> + set_hw_state_no_tz(hdev->core, true);
> + }
> if (ret)
> goto err;
>
> @@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device *hdev)
> return 0;
>
> err_suspend:
> - venus_set_hw_state_suspend(hdev->core);
> + if (hdev->core->use_tz)
> + set_hw_state(false);
> + else
> + set_hw_state_no_tz(hdev->core, false);
> err:
> hdev->power_enabled = false;
> return ret;
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_venus.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_venus_io.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/pm_helpers.c
> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/pm_helpers.h
> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/vdec.c
> rename to drivers/media/platform/qcom/vcodec/venus/vdec.c
> diff --git a/drivers/media/platform/qcom/venus/vdec.h b/drivers/media/platform/qcom/vcodec/venus/vdec.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/vdec.h
> rename to drivers/media/platform/qcom/vcodec/venus/vdec.h
> diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/vdec_ctrls.c
> rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
> diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/vcodec/venus/venc.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/venc.c
> rename to drivers/media/platform/qcom/vcodec/venus/venc.c
> diff --git a/drivers/media/platform/qcom/venus/venc.h b/drivers/media/platform/qcom/vcodec/venus/venc.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/venc.h
> rename to drivers/media/platform/qcom/vcodec/venus/venc.h
> diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/venc_ctrls.c
> rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
> diff --git a/drivers/media/platform/qcom/venus/firmware.c b/drivers/media/platform/qcom/venus/firmware.c
> deleted file mode 100644
> index fe7da2b..0000000
> --- a/drivers/media/platform/qcom/venus/firmware.c
> +++ /dev/null
> @@ -1,363 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0-only
> -/*
> - * Copyright (C) 2017 Linaro Ltd.
> - */
> -
> -#include <linux/device.h>
> -#include <linux/firmware.h>
> -#include <linux/kernel.h>
> -#include <linux/iommu.h>
> -#include <linux/io.h>
> -#include <linux/of.h>
> -#include <linux/of_address.h>
> -#include <linux/of_reserved_mem.h>
> -#include <linux/platform_device.h>
> -#include <linux/of_device.h>
> -#include <linux/firmware/qcom/qcom_scm.h>
> -#include <linux/sizes.h>
> -#include <linux/soc/qcom/mdt_loader.h>
> -
> -#include "core.h"
> -#include "firmware.h"
> -#include "hfi_venus_io.h"
> -
> -#define VENUS_PAS_ID 9
> -#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
> -#define VENUS_FW_START_ADDR 0x0
> -
> -static void venus_reset_cpu(struct venus_core *core)
> -{
> - u32 fw_size = core->fw.mapped_mem_size;
> - void __iomem *wrapper_base;
> -
> - if (IS_IRIS2_1(core))
> - wrapper_base = core->wrapper_tz_base;
> - else
> - wrapper_base = core->wrapper_base;
> -
> - writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
> - writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
> -
> - if (IS_IRIS2_1(core)) {
> - /* Bring XTSS out of reset */
> - writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
> - } else {
> - writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
> - writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
> -
> - /* Bring ARM9 out of reset */
> - writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
> - }
> -}
> -
> -int venus_set_hw_state(struct venus_core *core, bool resume)
> -{
> - int ret;
> -
> - if (core->use_tz) {
> - ret = qcom_scm_set_remote_state(resume, 0);
> - if (resume && ret == -EINVAL)
> - ret = 0;
> - return ret;
> - }
> -
> - if (resume) {
> - venus_reset_cpu(core);
> - } else {
> - if (IS_IRIS2_1(core))
> - writel(WRAPPER_XTSS_SW_RESET_BIT,
> - core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> - else
> - writel(WRAPPER_A9SS_SW_RESET_BIT,
> - core->wrapper_base + WRAPPER_A9SS_SW_RESET);
> - }
> -
> - return 0;
> -}
> -
> -static int venus_load_fw(struct venus_core *core, const char *fwname,
> - phys_addr_t *mem_phys, size_t *mem_size)
> -{
> - const struct firmware *mdt;
> - struct reserved_mem *rmem;
> - struct device_node *node;
> - struct device *dev;
> - ssize_t fw_size;
> - void *mem_va;
> - int ret;
> -
> - *mem_phys = 0;
> - *mem_size = 0;
> -
> - dev = core->dev;
> - node = of_parse_phandle(dev->of_node, "memory-region", 0);
> - if (!node) {
> - dev_err(dev, "no memory-region specified\n");
> - return -EINVAL;
> - }
> -
> - rmem = of_reserved_mem_lookup(node);
> - of_node_put(node);
> - if (!rmem) {
> - dev_err(dev, "failed to lookup reserved memory-region\n");
> - return -EINVAL;
> - }
> -
> - ret = request_firmware(&mdt, fwname, dev);
> - if (ret < 0)
> - return ret;
> -
> - fw_size = qcom_mdt_get_size(mdt);
> - if (fw_size < 0) {
> - ret = fw_size;
> - goto err_release_fw;
> - }
> -
> - *mem_phys = rmem->base;
> - *mem_size = rmem->size;
> -
> - if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) {
> - ret = -EINVAL;
> - goto err_release_fw;
> - }
> -
> - mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> - if (!mem_va) {
> - dev_err(dev, "unable to map memory region %pa size %#zx\n", mem_phys, *mem_size);
> - ret = -ENOMEM;
> - goto err_release_fw;
> - }
> -
> - if (core->use_tz)
> - ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID,
> - mem_va, *mem_phys, *mem_size, NULL);
> - else
> - ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID,
> - mem_va, *mem_phys, *mem_size, NULL);
> -
> - memunmap(mem_va);
> -err_release_fw:
> - release_firmware(mdt);
> - return ret;
> -}
> -
> -static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys,
> - size_t mem_size)
> -{
> - struct iommu_domain *iommu;
> - struct device *dev;
> - int ret;
> -
> - dev = core->fw.dev;
> - if (!dev)
> - return -EPROBE_DEFER;
> -
> - iommu = core->fw.iommu_domain;
> - core->fw.mapped_mem_size = mem_size;
> -
> - ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
> - IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
> - if (ret) {
> - dev_err(dev, "could not map video firmware region\n");
> - return ret;
> - }
> -
> - venus_reset_cpu(core);
> -
> - return 0;
> -}
> -
> -static int venus_shutdown_no_tz(struct venus_core *core)
> -{
> - const size_t mapped = core->fw.mapped_mem_size;
> - struct iommu_domain *iommu;
> - size_t unmapped;
> - u32 reg;
> - struct device *dev = core->fw.dev;
> - void __iomem *wrapper_base = core->wrapper_base;
> - void __iomem *wrapper_tz_base = core->wrapper_tz_base;
> -
> - if (IS_IRIS2_1(core)) {
> - /* Assert the reset to XTSS */
> - reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> - reg |= WRAPPER_XTSS_SW_RESET_BIT;
> - writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> - } else {
> - /* Assert the reset to ARM9 */
> - reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
> - reg |= WRAPPER_A9SS_SW_RESET_BIT;
> - writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
> - }
> -
> - iommu = core->fw.iommu_domain;
> -
> - if (core->fw.mapped_mem_size && iommu) {
> - unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
> -
> - if (unmapped != mapped)
> - dev_err(dev, "failed to unmap firmware\n");
> - else
> - core->fw.mapped_mem_size = 0;
> - }
> -
> - return 0;
> -}
> -
> -int venus_boot(struct venus_core *core)
> -{
> - struct device *dev = core->dev;
> - const struct venus_resources *res = core->res;
> - const char *fwpath = NULL;
> - phys_addr_t mem_phys;
> - size_t mem_size;
> - int ret;
> -
> - if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> - (core->use_tz && !qcom_scm_is_available()))
> - return -EPROBE_DEFER;
> -
> - ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
> - &fwpath);
> - if (ret)
> - fwpath = core->res->fwname;
> -
> - ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size);
> - if (ret) {
> - dev_err(dev, "fail to load video firmware\n");
> - return -EINVAL;
> - }
> -
> - core->fw.mem_size = mem_size;
> - core->fw.mem_phys = mem_phys;
> -
> - if (core->use_tz)
> - ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID);
> - else
> - ret = venus_boot_no_tz(core, mem_phys, mem_size);
> -
> - if (ret)
> - return ret;
> -
> - if (core->use_tz && res->cp_size) {
> - /*
> - * Clues for porting using downstream data:
> - * cp_start = 0
> - * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> - * This works, as the non-secure context bank is placed
> - * contiguously right after the Content Protection region.
> - *
> - * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> - * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> - */
> - ret = qcom_scm_mem_protect_video_var(res->cp_start,
> - res->cp_size,
> - res->cp_nonpixel_start,
> - res->cp_nonpixel_size);
> - if (ret) {
> - qcom_scm_pas_shutdown(VENUS_PAS_ID);
> - dev_err(dev, "set virtual address ranges fail (%d)\n",
> - ret);
> - return ret;
> - }
> - }
> -
> - return 0;
> -}
> -
> -int venus_shutdown(struct venus_core *core)
> -{
> - int ret;
> -
> - if (core->use_tz)
> - ret = qcom_scm_pas_shutdown(VENUS_PAS_ID);
> - else
> - ret = venus_shutdown_no_tz(core);
> -
> - return ret;
> -}
> -
> -int venus_firmware_init(struct venus_core *core)
> -{
> - struct platform_device_info info;
> - struct iommu_domain *iommu_dom;
> - struct platform_device *pdev;
> - struct device_node *np;
> - int ret;
> -
> - np = of_get_child_by_name(core->dev->of_node, "video-firmware");
> - if (!np) {
> - core->use_tz = true;
> - return 0;
> - }
> -
> - memset(&info, 0, sizeof(info));
> - info.fwnode = &np->fwnode;
> - info.parent = core->dev;
> - info.name = np->name;
> - info.dma_mask = DMA_BIT_MASK(32);
> -
> - pdev = platform_device_register_full(&info);
> - if (IS_ERR(pdev)) {
> - of_node_put(np);
> - return PTR_ERR(pdev);
> - }
> -
> - pdev->dev.of_node = np;
> -
> - ret = of_dma_configure(&pdev->dev, np, true);
> - if (ret) {
> - dev_err(core->dev, "dma configure fail\n");
> - goto err_unregister;
> - }
> -
> - core->fw.dev = &pdev->dev;
> -
> - iommu_dom = iommu_domain_alloc(&platform_bus_type);
> - if (!iommu_dom) {
> - dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
> - ret = -ENOMEM;
> - goto err_unregister;
> - }
> -
> - ret = iommu_attach_device(iommu_dom, core->fw.dev);
> - if (ret) {
> - dev_err(core->fw.dev, "could not attach device\n");
> - goto err_iommu_free;
> - }
> -
> - core->fw.iommu_domain = iommu_dom;
> -
> - of_node_put(np);
> -
> - return 0;
> -
> -err_iommu_free:
> - iommu_domain_free(iommu_dom);
> -err_unregister:
> - platform_device_unregister(pdev);
> - of_node_put(np);
> - return ret;
> -}
> -
> -void venus_firmware_deinit(struct venus_core *core)
> -{
> - struct iommu_domain *iommu;
> -
> - if (!core->fw.dev)
> - return;
> -
> - iommu = core->fw.iommu_domain;
> -
> - iommu_detach_device(iommu, core->fw.dev);
> -
> - if (core->fw.iommu_domain) {
> - iommu_domain_free(iommu);
> - core->fw.iommu_domain = NULL;
> - }
> -
> - platform_device_unregister(to_platform_device(core->fw.dev));
> -}
> diff --git a/drivers/media/platform/qcom/venus/firmware.h b/drivers/media/platform/qcom/venus/firmware.h
> deleted file mode 100644
> index aaccd84..0000000
> --- a/drivers/media/platform/qcom/venus/firmware.h
> +++ /dev/null
> @@ -1,26 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0-only */
> -/*
> - * Copyright (C) 2017 Linaro Ltd.
> - */
> -#ifndef __VENUS_FIRMWARE_H__
> -#define __VENUS_FIRMWARE_H__
> -
> -struct device;
> -
> -int venus_firmware_init(struct venus_core *core);
> -void venus_firmware_deinit(struct venus_core *core);
> -int venus_boot(struct venus_core *core);
> -int venus_shutdown(struct venus_core *core);
> -int venus_set_hw_state(struct venus_core *core, bool suspend);
> -
> -static inline int venus_set_hw_state_suspend(struct venus_core *core)
> -{
> - return venus_set_hw_state(core, false);
> -}
> -
> -static inline int venus_set_hw_state_resume(struct venus_core *core)
> -{
> - return venus_set_hw_state(core, true);
> -}
> -
> -#endif
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-18 18:24 ` Dmitry Baryshkov
@ 2023-12-20 8:01 ` Dikshita Agarwal
2023-12-20 8:12 ` Dmitry Baryshkov
0 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:01 UTC (permalink / raw)
To: Dmitry Baryshkov, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> Re-organize the video driver code by introducing a new folder
>> 'vcodec' and placing 'venus' driver code inside that.
>>
>> Introduce common helpers for trustzone based firmware
>> load/unload etc. which are placed in common folder
>> i.e. 'vcodec'.
>> Use these helpers in 'venus' driver. These helpers will be
>> used by 'iris' driver as well which is introduced later
>> in this patch series.
>
> But why do you need to move the venus driver to subdir?
Currently venus driver is present in drivers/media/platform/qcom folder
which also has camss folder. We introduced vcodec to keep common code and
moved venus inside that, to indicate that the common code is for vcodec
drivers i.e venus and iris. Keeping this in qcom folder would mean, common
code will be used for camss only which is not the case here.
>
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>> drivers/media/platform/qcom/Kconfig | 2 +-
>> drivers/media/platform/qcom/Makefile | 2 +-
>> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
>> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
>> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
>> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
>> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
>> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
>> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
>> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
>> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
>> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
>> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
>> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
>> drivers/media/platform/qcom/venus/firmware.c | 363
>> ---------------------
>> drivers/media/platform/qcom/venus/firmware.h | 26 --
>> 42 files changed, 492 insertions(+), 409 deletions(-)
>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
>> create mode 100644
>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> create mode 100644
>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
>> (100%)
>> rename drivers/media/platform/qcom/{ =>
>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>>
>> diff --git a/drivers/media/platform/qcom/Kconfig
>> b/drivers/media/platform/qcom/Kconfig
>> index cc5799b..e94142f 100644
>> --- a/drivers/media/platform/qcom/Kconfig
>> +++ b/drivers/media/platform/qcom/Kconfig
>> @@ -3,4 +3,4 @@
>> comment "Qualcomm media platform drivers"
>> source "drivers/media/platform/qcom/camss/Kconfig"
>> -source "drivers/media/platform/qcom/venus/Kconfig"
>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
>> diff --git a/drivers/media/platform/qcom/Makefile
>> b/drivers/media/platform/qcom/Makefile
>> index 4f055c3..3d2d82b 100644
>> --- a/drivers/media/platform/qcom/Makefile
>> +++ b/drivers/media/platform/qcom/Makefile
>> @@ -1,3 +1,3 @@
>> # SPDX-License-Identifier: GPL-2.0-only
>> obj-y += camss/
>> -obj-y += venus/
>> +obj-y += vcodec/venus/
>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
>> b/drivers/media/platform/qcom/vcodec/firmware.c
>> new file mode 100644
>> index 0000000..dbc220a
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
>> @@ -0,0 +1,147 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#include <linux/device.h>
>> +#include <linux/dma-mapping.h>
>> +#include <linux/firmware.h>
>> +#include <linux/kernel.h>
>> +#include <linux/iommu.h>
>> +#include <linux/of_device.h>
>> +#include <linux/firmware/qcom/qcom_scm.h>
>> +#include <linux/of_reserved_mem.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/soc/qcom/mdt_loader.h>
>> +
>> +#include "firmware.h"
>> +
>> +bool use_tz(struct device *core_dev)
>
> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
> function is too polluting for the global namespace.
>
I understand, will check and do the needful.
>> +{
>> + struct device_node *np;
>> +
>> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
>> + if (!np)
>> + return true;
>> +
>> + return false;
>> +}
>> +
>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>> + u32 cp_nonpixel_size, u32 pas_id)
>> +{
>> + int ret;
>> + /*
>> + * Clues for porting using downstream data:
>> + * cp_start = 0
>> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
>> + * This works, as the non-secure context bank is placed
>> + * contiguously right after the Content Protection region.
>> + *
>> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>> + */
>> + ret = qcom_scm_mem_protect_video_var(cp_start,
>> + cp_size,
>> + cp_nonpixel_start,
>> + cp_nonpixel_size);
>> + if (ret)
>> + qcom_scm_pas_shutdown(pas_id);
>> +
>> + return ret;
>> +}
>> +
>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>> + size_t *mem_size, u32 pas_id, bool use_tz)
>> +{
>> + const struct firmware *firmware = NULL;
>> + struct reserved_mem *rmem;
>> + struct device_node *node;
>> + void *mem_virt = NULL;
>> + ssize_t fw_size = 0;
>> + int ret;
>> +
>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>
> Why? Can you just depend on it?
>
Sure, Will check this and get back.
>> + (use_tz && !qcom_scm_is_available()))
>> + return -EPROBE_DEFER;
>> +
>> + if (!fw_name || !(*fw_name))
>> + return -EINVAL;
>> +
>> + *mem_phys = 0;
>> + *mem_size = 0;
>> +
>> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
>> + if (!node) {
>> + dev_err(dev, "no memory-region specified\n");
>> + return -EINVAL;
>> + }
>> +
>> + rmem = of_reserved_mem_lookup(node);
>> + of_node_put(node);
>> + if (!rmem) {
>> + dev_err(dev, "failed to lookup reserved memory-region\n");
>> + return -EINVAL;
>> + }
>> +
>> + ret = request_firmware(&firmware, fw_name, dev);
>> + if (ret) {
>> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
>> + __func__, fw_name, ret);
>> + return ret;
>> + }
>> +
>> + fw_size = qcom_mdt_get_size(firmware);
>> + if (fw_size < 0) {
>> + ret = fw_size;
>> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
>> + __func__, fw_size);
>> + goto err_release_fw;
>> + }
>> +
>> + *mem_phys = rmem->base;
>> + *mem_size = rmem->size;
>> +
>> + if (*mem_size < fw_size) {
>> + ret = -EINVAL;
>> + goto err_release_fw;
>> + }
>> +
>> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>> + if (!mem_virt) {
>> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
>> + mem_phys, *mem_size);
>> + goto err_release_fw;
>> + }
>> +
>> + if (use_tz)
>> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
>> + *mem_phys, *mem_size, NULL);
>> + else
>> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
>> mem_virt,
>> + *mem_phys, *mem_size, NULL);
>> + if (ret) {
>> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
>> + __func__, ret, fw_name);
>> + }
>> +
>> + memunmap(mem_virt);
>> +err_release_fw:
>> + release_firmware(firmware);
>> + return ret;
>> +}
>> +
>> +int auth_reset_fw(u32 pas_id)
>> +{
>> + return qcom_scm_pas_auth_and_reset(pas_id);
>> +}
>> +
>> +void unload_fw(u32 pas_id)
>> +{
>> + qcom_scm_pas_shutdown(pas_id);
>> +}
>> +
>> +int set_hw_state(bool resume)
>> +{
>> + return qcom_scm_set_remote_state(resume, 0);
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
>> b/drivers/media/platform/qcom/vcodec/firmware.h
>> new file mode 100644
>> index 0000000..7d410a8
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
>> @@ -0,0 +1,21 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#ifndef _FIRMWARE_H_
>> +#define _FIRMWARE_H_
>> +
>> +#include <linux/device.h>
>> +#include <linux/types.h>
>> +
>> +bool use_tz(struct device *core_dev);
>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>> + size_t *mem_size, u32 pas_id, bool use_tz);
>> +int auth_reset_fw(u32 pas_id);
>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>> + u32 cp_nonpixel_size, u32 pas_id);
>> +void unload_fw(u32 pas_id);
>> +int set_hw_state(bool resume);
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/Kconfig
>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
>> diff --git a/drivers/media/platform/qcom/venus/Makefile
>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> similarity index 83%
>> rename from drivers/media/platform/qcom/venus/Makefile
>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
>> index 91ee6be..f6f3a88 100644
>> --- a/drivers/media/platform/qcom/venus/Makefile
>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> @@ -1,7 +1,9 @@
>> # SPDX-License-Identifier: GPL-2.0
>> # Makefile for Qualcomm Venus driver
>> -venus-core-objs += core.o helpers.o firmware.o \
>> +venus-core-objs += ../firmware.o
>> +
>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>> hfi_parser.o pm_helpers.o dbgfs.o \
>> hfi_platform.o hfi_platform_v4.o \
>> diff --git a/drivers/media/platform/qcom/venus/core.c
>> b/drivers/media/platform/qcom/vcodec/venus/core.c
>> similarity index 91%
>> rename from drivers/media/platform/qcom/venus/core.c
>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
>> index 9cffe97..56d9a53 100644
>> --- a/drivers/media/platform/qcom/venus/core.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
>> @@ -22,7 +22,8 @@
>> #include <media/v4l2-ioctl.h>
>> #include "core.h"
>> -#include "firmware.h"
>> +#include "../firmware.h"
>> +#include "firmware_no_tz.h"
>> #include "pm_helpers.h"
>> #include "hfi_venus_io.h"
>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
>> work_struct *work)
>> struct venus_core *core =
>> container_of(work, struct venus_core, work.work);
>> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
>> + const struct venus_resources *res = core->res;
>> + const char *fwpath = NULL;
>> const char *err_msg = "";
>> bool failed = false;
>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
>> work_struct *work)
>> mutex_lock(&core->lock);
>> - venus_shutdown(core);
>> + if (core->use_tz)
>> + unload_fw(VENUS_PAS_ID);
>> + else
>> + unload_fw_no_tz(core);
>
> This is more than introducing helpers.
>
The new helpers are written to make the code generic for video drivers.
which requires changes in the calling function also.
>> venus_coredump(core);
>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
>> work_struct *work)
>> failed = true;
>> }
>> - ret = venus_boot(core);
>> + ret = of_property_read_string_index(core->dev->of_node,
>> "firmware-name", 0,
>> + &fwpath);
>> + if (ret)
>> + fwpath = core->res->fwname;
>> +
>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>> &core->fw.mem_size,
>> + VENUS_PAS_ID, core->use_tz);
>
> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
> with non-generic prefixes, etc. If you are introducing helpers, please
> refrain from inlining of calling functions, etc. Just move the code to your
> helpers.
>
As mentioned in above comment, the common helpers are written to make the
code generic. I Will try to make it more clear, working on the same.
> NAK for the rest of the patch.
>
>> if (ret && !failed) {
>> - err_msg = "boot Venus";
>> + err_msg = "load FW";
>> failed = true;
>> }
>> + if (core->use_tz)
>> + ret = auth_reset_fw(VENUS_PAS_ID);
>> + else
>> + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys,
>> core->fw.mem_size);
>> + if (ret && !failed) {
>> + err_msg = "Auth and Reset";
>> + failed = true;
>> + }
>> +
>> + if (core->use_tz && res->cp_size) {
>> + ret = protect_secure_region(res->cp_start,
>> + res->cp_size,
>> + res->cp_nonpixel_start,
>> + res->cp_nonpixel_size,
>> + VENUS_PAS_ID);
>> + if (ret && !failed) {
>> + err_msg = "Protect CP Mem";
>> + failed = true;
>> + }
>> + }
>> +
>> ret = hfi_core_resume(core, true);
>> if (ret && !failed) {
>> err_msg = "resume HFI";
>> @@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void
>> *dev_id)
>> static int venus_probe(struct platform_device *pdev)
>> {
>> + const struct venus_resources *res;
>> struct device *dev = &pdev->dev;
>> + const char *fwpath = NULL;
>> struct venus_core *core;
>> int ret;
>> @@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev)
>> if (ret)
>> goto err_runtime_disable;
>> - ret = venus_firmware_init(core);
>> + core->use_tz = use_tz(core->dev);
>> +
>> + if (!core->use_tz) {
>> + ret = init_fw_no_tz(core);
>> + if (ret)
>> + goto err_of_depopulate;
>> + }
>> +
>> + ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
>> + &fwpath);
>> if (ret)
>> - goto err_of_depopulate;
>> + fwpath = core->res->fwname;
>> - ret = venus_boot(core);
>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>> &core->fw.mem_size,
>> + VENUS_PAS_ID, core->use_tz);
>> if (ret)
>> goto err_firmware_deinit;
>> + if (core->use_tz)
>> + ret = auth_reset_fw(VENUS_PAS_ID);
>> + else
>> + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys,
>> core->fw.mem_size);
>> + if (ret)
>> + goto err_firmware_deinit;
>> +
>> + res = core->res;
>> + if (core->use_tz && res->cp_size) {
>> + ret = protect_secure_region(res->cp_start,
>> + res->cp_size,
>> + res->cp_nonpixel_start,
>> + res->cp_nonpixel_size,
>> + VENUS_PAS_ID);
>> + if (ret)
>> + goto err_firmware_deinit;
>> + }
>> +
>> ret = hfi_core_resume(core, true);
>> if (ret)
>> goto err_venus_shutdown;
>> @@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev)
>> err_dev_unregister:
>> v4l2_device_unregister(&core->v4l2_dev);
>> err_venus_shutdown:
>> - venus_shutdown(core);
>> + if (core->use_tz)
>> + unload_fw(VENUS_PAS_ID);
>> + else
>> + unload_fw_no_tz(core);
>> err_firmware_deinit:
>> - venus_firmware_deinit(core);
>> + if (!core->use_tz)
>> + deinit_fw_no_tz(core);
>> err_of_depopulate:
>> of_platform_depopulate(dev);
>> err_runtime_disable:
>> @@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev)
>> ret = hfi_core_deinit(core, true);
>> WARN_ON(ret);
>> - venus_shutdown(core);
>> + if (core->use_tz)
>> + unload_fw(VENUS_PAS_ID);
>> + else
>> + unload_fw_no_tz(core);
>> +
>> of_platform_depopulate(dev);
>> - venus_firmware_deinit(core);
>> + if (!core->use_tz)
>> + deinit_fw_no_tz(core);
>> pm_runtime_put_sync(dev);
>> pm_runtime_disable(dev);
>> @@ -455,8 +527,12 @@ static void venus_core_shutdown(struct
>> platform_device *pdev)
>> struct venus_core *core = platform_get_drvdata(pdev);
>> pm_runtime_get_sync(core->dev);
>> - venus_shutdown(core);
>> - venus_firmware_deinit(core);
>> + if (core->use_tz) {
>> + unload_fw(VENUS_PAS_ID);
>> + } else {
>> + unload_fw_no_tz(core);
>> + deinit_fw_no_tz(core);
>> + }
>> pm_runtime_put_sync(core->dev);
>> }
>> diff --git a/drivers/media/platform/qcom/venus/core.h
>> b/drivers/media/platform/qcom/vcodec/venus/core.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/core.h
>> rename to drivers/media/platform/qcom/vcodec/venus/core.h
>> diff --git a/drivers/media/platform/qcom/venus/dbgfs.c
>> b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/dbgfs.c
>> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c
>> diff --git a/drivers/media/platform/qcom/venus/dbgfs.h
>> b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/dbgfs.h
>> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> new file mode 100644
>> index 0000000..9dca6e23
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> @@ -0,0 +1,194 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (C) 2017 Linaro Ltd.
>> + */
>> +
>> +#include <linux/iommu.h>
>> +#include <linux/of_device.h>
>> +#include "core.h"
>> +#include "firmware_no_tz.h"
>> +#include "hfi_venus_io.h"
>> +
>> +#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
>> +#define VENUS_FW_START_ADDR 0x0
>> +
>> +int init_fw_no_tz(struct venus_core *core)
>> +{
>> + struct platform_device_info info;
>> + struct iommu_domain *iommu_dom;
>> + struct platform_device *pdev;
>> + struct device_node *np;
>> + int ret;
>> +
>> + np = of_get_child_by_name(core->dev->of_node, "video-firmware");
>> +
>> + memset(&info, 0, sizeof(info));
>> + info.fwnode = &np->fwnode;
>> + info.parent = core->dev;
>> + info.name = np->name;
>> + info.dma_mask = DMA_BIT_MASK(32);
>> +
>> + pdev = platform_device_register_full(&info);
>> + if (IS_ERR(pdev)) {
>> + of_node_put(np);
>> + return PTR_ERR(pdev);
>> + }
>> +
>> + pdev->dev.of_node = np;
>> +
>> + ret = of_dma_configure(&pdev->dev, np, true);
>> + if (ret) {
>> + dev_err(core->dev, "dma configure fail\n");
>> + goto err_unregister;
>> + }
>> +
>> + core->fw.dev = &pdev->dev;
>> +
>> + iommu_dom = iommu_domain_alloc(&platform_bus_type);
>> + if (!iommu_dom) {
>> + dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
>> + ret = -ENOMEM;
>> + goto err_unregister;
>> + }
>> +
>> + ret = iommu_attach_device(iommu_dom, core->fw.dev);
>> + if (ret) {
>> + dev_err(core->fw.dev, "could not attach device\n");
>> + goto err_iommu_free;
>> + }
>> +
>> + core->fw.iommu_domain = iommu_dom;
>> +
>> + of_node_put(np);
>> +
>> + return 0;
>> +
>> +err_iommu_free:
>> + iommu_domain_free(iommu_dom);
>> +err_unregister:
>> + platform_device_unregister(pdev);
>> + of_node_put(np);
>> + return ret;
>> +}
>> +
>> +void deinit_fw_no_tz(struct venus_core *core)
>> +{
>> + struct iommu_domain *iommu;
>> +
>> + if (!core->fw.dev)
>> + return;
>> +
>> + iommu = core->fw.iommu_domain;
>> +
>> + iommu_detach_device(iommu, core->fw.dev);
>> +
>> + if (iommu) {
>> + iommu_domain_free(iommu);
>> + iommu = NULL;
>> + }
>> +
>> + platform_device_unregister(to_platform_device(core->fw.dev));
>> +}
>> +
>> +static void reset_cpu_no_tz(struct venus_core *core)
>> +{
>> + u32 fw_size = core->fw.mapped_mem_size;
>> + void __iomem *wrapper_base;
>> +
>> + if (IS_IRIS2_1(core))
>> + wrapper_base = core->wrapper_tz_base;
>> + else
>> + wrapper_base = core->wrapper_base;
>> +
>> + writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
>> + writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
>> + writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
>> + writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
>> + writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
>> + writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
>> +
>> + if (IS_IRIS2_1(core)) {
>> + /* Bring XTSS out of reset */
>> + writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
>> + } else {
>> + writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
>> + writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
>> +
>> + /* Bring ARM9 out of reset */
>> + writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> + }
>> +}
>> +
>> +void set_hw_state_no_tz(struct venus_core *core, bool resume)
>> +{
>> + if (resume) {
>> + reset_cpu_no_tz(core);
>> + } else {
>> + if (IS_IRIS2_1(core))
>> + writel(WRAPPER_XTSS_SW_RESET_BIT,
>> + core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> + else
>> + writel(WRAPPER_A9SS_SW_RESET_BIT,
>> + core->wrapper_base + WRAPPER_A9SS_SW_RESET);
>> + }
>> +}
>> +
>> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
>> + size_t mem_size)
>> +{
>> + struct iommu_domain *iommu;
>> + struct device *dev;
>> + int ret;
>> +
>> + dev = core->fw.dev;
>> + if (!dev)
>> + return -EPROBE_DEFER;
>> +
>> + iommu = core->fw.iommu_domain;
>> + core->fw.mapped_mem_size = mem_size;
>> +
>> + ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
>> + IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
>> + if (ret) {
>> + dev_err(dev, "could not map video firmware region\n");
>> + return ret;
>> + }
>> +
>> + reset_cpu_no_tz(core);
>> +
>> + return 0;
>> +}
>> +
>> +void unload_fw_no_tz(struct venus_core *core)
>> +{
>> + const size_t mapped = core->fw.mapped_mem_size;
>> + struct iommu_domain *iommu;
>> + size_t unmapped;
>> + u32 reg;
>> + struct device *dev = core->fw.dev;
>> + void __iomem *wrapper_base = core->wrapper_base;
>> + void __iomem *wrapper_tz_base = core->wrapper_tz_base;
>> +
>> + if (IS_IRIS2_1(core)) {
>> + /* Assert the reset to XTSS */
>> + reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> + reg |= WRAPPER_XTSS_SW_RESET_BIT;
>> + writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> + } else {
>> + /* Assert the reset to ARM9 */
>> + reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
>> + reg |= WRAPPER_A9SS_SW_RESET_BIT;
>> + writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> + }
>> +
>> + iommu = core->fw.iommu_domain;
>> +
>> + if (core->fw.mapped_mem_size && iommu) {
>> + unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
>> +
>> + if (unmapped != mapped)
>> + dev_err(dev, "failed to unmap firmware\n");
>> + else
>> + core->fw.mapped_mem_size = 0;
>> + }
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> new file mode 100644
>> index 0000000..5f008ef
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> @@ -0,0 +1,19 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (C) 2017 Linaro Ltd.
>> + */
>> +#ifndef __FIRMWARE_NO_TZ_H__
>> +#define __FIRMWARE_NO_TZ_H__
>> +
>> +struct device;
>> +
>> +#define VENUS_PAS_ID 9
>> +
>> +int init_fw_no_tz(struct venus_core *core);
>> +void deinit_fw_no_tz(struct venus_core *core);
>> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
>> + size_t mem_size);
>> +void unload_fw_no_tz(struct venus_core *core);
>> +void set_hw_state_no_tz(struct venus_core *core, bool resume);
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/venus/helpers.c
>> b/drivers/media/platform/qcom/vcodec/venus/helpers.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/helpers.c
>> rename to drivers/media/platform/qcom/vcodec/venus/helpers.c
>> diff --git a/drivers/media/platform/qcom/venus/helpers.h
>> b/drivers/media/platform/qcom/vcodec/venus/helpers.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/helpers.h
>> rename to drivers/media/platform/qcom/vcodec/venus/helpers.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_cmds.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_cmds.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_helper.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_msgs.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_msgs.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_parser.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_parser.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> similarity index 99%
>> rename from drivers/media/platform/qcom/venus/hfi_venus.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> index f9437b6..5a68db9 100644
>> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> @@ -13,11 +13,12 @@
>> #include <linux/slab.h>
>> #include "core.h"
>> +#include "../firmware.h"
>> #include "hfi_cmds.h"
>> #include "hfi_msgs.h"
>> #include "hfi_venus.h"
>> #include "hfi_venus_io.h"
>> -#include "firmware.h"
>> +#include "firmware_no_tz.h"
>> #define HFI_MASK_QHDR_TX_TYPE 0xff000000
>> #define HFI_MASK_QHDR_RX_TYPE 0x00ff0000
>> @@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device
>> *hdev)
>> if (!hdev->power_enabled)
>> return 0;
>> - ret = venus_set_hw_state_suspend(hdev->core);
>> + if (hdev->core->use_tz)
>> + ret = set_hw_state(false);
>> + else
>> + set_hw_state_no_tz(hdev->core, false);
>> if (ret)
>> return ret;
>> @@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device
>> *hdev)
>> if (hdev->power_enabled)
>> return 0;
>> - ret = venus_set_hw_state_resume(hdev->core);
>> + if (hdev->core->use_tz) {
>> + ret = set_hw_state(true);
>> + if (ret == -EINVAL)
>> + ret = 0;
>> + } else {
>> + set_hw_state_no_tz(hdev->core, true);
>> + }
>> if (ret)
>> goto err;
>> @@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device
>> *hdev)
>> return 0;
>> err_suspend:
>> - venus_set_hw_state_suspend(hdev->core);
>> + if (hdev->core->use_tz)
>> + set_hw_state(false);
>> + else
>> + set_hw_state_no_tz(hdev->core, false);
>> err:
>> hdev->power_enabled = false;
>> return ret;
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_venus.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_venus_io.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
>> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c
>> b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/pm_helpers.c
>> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
>> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h
>> b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/pm_helpers.h
>> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
>> diff --git a/drivers/media/platform/qcom/venus/vdec.c
>> b/drivers/media/platform/qcom/vcodec/venus/vdec.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/vdec.c
>> rename to drivers/media/platform/qcom/vcodec/venus/vdec.c
>> diff --git a/drivers/media/platform/qcom/venus/vdec.h
>> b/drivers/media/platform/qcom/vcodec/venus/vdec.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/vdec.h
>> rename to drivers/media/platform/qcom/vcodec/venus/vdec.h
>> diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c
>> b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/vdec_ctrls.c
>> rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
>> diff --git a/drivers/media/platform/qcom/venus/venc.c
>> b/drivers/media/platform/qcom/vcodec/venus/venc.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/venc.c
>> rename to drivers/media/platform/qcom/vcodec/venus/venc.c
>> diff --git a/drivers/media/platform/qcom/venus/venc.h
>> b/drivers/media/platform/qcom/vcodec/venus/venc.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/venc.h
>> rename to drivers/media/platform/qcom/vcodec/venus/venc.h
>> diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c
>> b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/venc_ctrls.c
>> rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
>> diff --git a/drivers/media/platform/qcom/venus/firmware.c
>> b/drivers/media/platform/qcom/venus/firmware.c
>> deleted file mode 100644
>> index fe7da2b..0000000
>> --- a/drivers/media/platform/qcom/venus/firmware.c
>> +++ /dev/null
>> @@ -1,363 +0,0 @@
>> -// SPDX-License-Identifier: GPL-2.0-only
>> -/*
>> - * Copyright (C) 2017 Linaro Ltd.
>> - */
>> -
>> -#include <linux/device.h>
>> -#include <linux/firmware.h>
>> -#include <linux/kernel.h>
>> -#include <linux/iommu.h>
>> -#include <linux/io.h>
>> -#include <linux/of.h>
>> -#include <linux/of_address.h>
>> -#include <linux/of_reserved_mem.h>
>> -#include <linux/platform_device.h>
>> -#include <linux/of_device.h>
>> -#include <linux/firmware/qcom/qcom_scm.h>
>> -#include <linux/sizes.h>
>> -#include <linux/soc/qcom/mdt_loader.h>
>> -
>> -#include "core.h"
>> -#include "firmware.h"
>> -#include "hfi_venus_io.h"
>> -
>> -#define VENUS_PAS_ID 9
>> -#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
>> -#define VENUS_FW_START_ADDR 0x0
>> -
>> -static void venus_reset_cpu(struct venus_core *core)
>> -{
>> - u32 fw_size = core->fw.mapped_mem_size;
>> - void __iomem *wrapper_base;
>> -
>> - if (IS_IRIS2_1(core))
>> - wrapper_base = core->wrapper_tz_base;
>> - else
>> - wrapper_base = core->wrapper_base;
>> -
>> - writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
>> - writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
>> - writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
>> - writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
>> - writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
>> - writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
>> -
>> - if (IS_IRIS2_1(core)) {
>> - /* Bring XTSS out of reset */
>> - writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
>> - } else {
>> - writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
>> - writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
>> -
>> - /* Bring ARM9 out of reset */
>> - writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> - }
>> -}
>> -
>> -int venus_set_hw_state(struct venus_core *core, bool resume)
>> -{
>> - int ret;
>> -
>> - if (core->use_tz) {
>> - ret = qcom_scm_set_remote_state(resume, 0);
>> - if (resume && ret == -EINVAL)
>> - ret = 0;
>> - return ret;
>> - }
>> -
>> - if (resume) {
>> - venus_reset_cpu(core);
>> - } else {
>> - if (IS_IRIS2_1(core))
>> - writel(WRAPPER_XTSS_SW_RESET_BIT,
>> - core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> - else
>> - writel(WRAPPER_A9SS_SW_RESET_BIT,
>> - core->wrapper_base + WRAPPER_A9SS_SW_RESET);
>> - }
>> -
>> - return 0;
>> -}
>> -
>> -static int venus_load_fw(struct venus_core *core, const char *fwname,
>> - phys_addr_t *mem_phys, size_t *mem_size)
>> -{
>> - const struct firmware *mdt;
>> - struct reserved_mem *rmem;
>> - struct device_node *node;
>> - struct device *dev;
>> - ssize_t fw_size;
>> - void *mem_va;
>> - int ret;
>> -
>> - *mem_phys = 0;
>> - *mem_size = 0;
>> -
>> - dev = core->dev;
>> - node = of_parse_phandle(dev->of_node, "memory-region", 0);
>> - if (!node) {
>> - dev_err(dev, "no memory-region specified\n");
>> - return -EINVAL;
>> - }
>> -
>> - rmem = of_reserved_mem_lookup(node);
>> - of_node_put(node);
>> - if (!rmem) {
>> - dev_err(dev, "failed to lookup reserved memory-region\n");
>> - return -EINVAL;
>> - }
>> -
>> - ret = request_firmware(&mdt, fwname, dev);
>> - if (ret < 0)
>> - return ret;
>> -
>> - fw_size = qcom_mdt_get_size(mdt);
>> - if (fw_size < 0) {
>> - ret = fw_size;
>> - goto err_release_fw;
>> - }
>> -
>> - *mem_phys = rmem->base;
>> - *mem_size = rmem->size;
>> -
>> - if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) {
>> - ret = -EINVAL;
>> - goto err_release_fw;
>> - }
>> -
>> - mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>> - if (!mem_va) {
>> - dev_err(dev, "unable to map memory region %pa size %#zx\n",
>> mem_phys, *mem_size);
>> - ret = -ENOMEM;
>> - goto err_release_fw;
>> - }
>> -
>> - if (core->use_tz)
>> - ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID,
>> - mem_va, *mem_phys, *mem_size, NULL);
>> - else
>> - ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID,
>> - mem_va, *mem_phys, *mem_size, NULL);
>> -
>> - memunmap(mem_va);
>> -err_release_fw:
>> - release_firmware(mdt);
>> - return ret;
>> -}
>> -
>> -static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys,
>> - size_t mem_size)
>> -{
>> - struct iommu_domain *iommu;
>> - struct device *dev;
>> - int ret;
>> -
>> - dev = core->fw.dev;
>> - if (!dev)
>> - return -EPROBE_DEFER;
>> -
>> - iommu = core->fw.iommu_domain;
>> - core->fw.mapped_mem_size = mem_size;
>> -
>> - ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
>> - IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
>> - if (ret) {
>> - dev_err(dev, "could not map video firmware region\n");
>> - return ret;
>> - }
>> -
>> - venus_reset_cpu(core);
>> -
>> - return 0;
>> -}
>> -
>> -static int venus_shutdown_no_tz(struct venus_core *core)
>> -{
>> - const size_t mapped = core->fw.mapped_mem_size;
>> - struct iommu_domain *iommu;
>> - size_t unmapped;
>> - u32 reg;
>> - struct device *dev = core->fw.dev;
>> - void __iomem *wrapper_base = core->wrapper_base;
>> - void __iomem *wrapper_tz_base = core->wrapper_tz_base;
>> -
>> - if (IS_IRIS2_1(core)) {
>> - /* Assert the reset to XTSS */
>> - reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> - reg |= WRAPPER_XTSS_SW_RESET_BIT;
>> - writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> - } else {
>> - /* Assert the reset to ARM9 */
>> - reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
>> - reg |= WRAPPER_A9SS_SW_RESET_BIT;
>> - writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> - }
>> -
>> - iommu = core->fw.iommu_domain;
>> -
>> - if (core->fw.mapped_mem_size && iommu) {
>> - unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
>> -
>> - if (unmapped != mapped)
>> - dev_err(dev, "failed to unmap firmware\n");
>> - else
>> - core->fw.mapped_mem_size = 0;
>> - }
>> -
>> - return 0;
>> -}
>> -
>> -int venus_boot(struct venus_core *core)
>> -{
>> - struct device *dev = core->dev;
>> - const struct venus_resources *res = core->res;
>> - const char *fwpath = NULL;
>> - phys_addr_t mem_phys;
>> - size_t mem_size;
>> - int ret;
>> -
>> - if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>> - (core->use_tz && !qcom_scm_is_available()))
>> - return -EPROBE_DEFER;
>> -
>> - ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
>> - &fwpath);
>> - if (ret)
>> - fwpath = core->res->fwname;
>> -
>> - ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size);
>> - if (ret) {
>> - dev_err(dev, "fail to load video firmware\n");
>> - return -EINVAL;
>> - }
>> -
>> - core->fw.mem_size = mem_size;
>> - core->fw.mem_phys = mem_phys;
>> -
>> - if (core->use_tz)
>> - ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID);
>> - else
>> - ret = venus_boot_no_tz(core, mem_phys, mem_size);
>> -
>> - if (ret)
>> - return ret;
>> -
>> - if (core->use_tz && res->cp_size) {
>> - /*
>> - * Clues for porting using downstream data:
>> - * cp_start = 0
>> - * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and
>> not size!
>> - * This works, as the non-secure context bank is placed
>> - * contiguously right after the Content Protection region.
>> - *
>> - * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>> - * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>> - */
>> - ret = qcom_scm_mem_protect_video_var(res->cp_start,
>> - res->cp_size,
>> - res->cp_nonpixel_start,
>> - res->cp_nonpixel_size);
>> - if (ret) {
>> - qcom_scm_pas_shutdown(VENUS_PAS_ID);
>> - dev_err(dev, "set virtual address ranges fail (%d)\n",
>> - ret);
>> - return ret;
>> - }
>> - }
>> -
>> - return 0;
>> -}
>> -
>> -int venus_shutdown(struct venus_core *core)
>> -{
>> - int ret;
>> -
>> - if (core->use_tz)
>> - ret = qcom_scm_pas_shutdown(VENUS_PAS_ID);
>> - else
>> - ret = venus_shutdown_no_tz(core);
>> -
>> - return ret;
>> -}
>> -
>> -int venus_firmware_init(struct venus_core *core)
>> -{
>> - struct platform_device_info info;
>> - struct iommu_domain *iommu_dom;
>> - struct platform_device *pdev;
>> - struct device_node *np;
>> - int ret;
>> -
>> - np = of_get_child_by_name(core->dev->of_node, "video-firmware");
>> - if (!np) {
>> - core->use_tz = true;
>> - return 0;
>> - }
>> -
>> - memset(&info, 0, sizeof(info));
>> - info.fwnode = &np->fwnode;
>> - info.parent = core->dev;
>> - info.name = np->name;
>> - info.dma_mask = DMA_BIT_MASK(32);
>> -
>> - pdev = platform_device_register_full(&info);
>> - if (IS_ERR(pdev)) {
>> - of_node_put(np);
>> - return PTR_ERR(pdev);
>> - }
>> -
>> - pdev->dev.of_node = np;
>> -
>> - ret = of_dma_configure(&pdev->dev, np, true);
>> - if (ret) {
>> - dev_err(core->dev, "dma configure fail\n");
>> - goto err_unregister;
>> - }
>> -
>> - core->fw.dev = &pdev->dev;
>> -
>> - iommu_dom = iommu_domain_alloc(&platform_bus_type);
>> - if (!iommu_dom) {
>> - dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
>> - ret = -ENOMEM;
>> - goto err_unregister;
>> - }
>> -
>> - ret = iommu_attach_device(iommu_dom, core->fw.dev);
>> - if (ret) {
>> - dev_err(core->fw.dev, "could not attach device\n");
>> - goto err_iommu_free;
>> - }
>> -
>> - core->fw.iommu_domain = iommu_dom;
>> -
>> - of_node_put(np);
>> -
>> - return 0;
>> -
>> -err_iommu_free:
>> - iommu_domain_free(iommu_dom);
>> -err_unregister:
>> - platform_device_unregister(pdev);
>> - of_node_put(np);
>> - return ret;
>> -}
>> -
>> -void venus_firmware_deinit(struct venus_core *core)
>> -{
>> - struct iommu_domain *iommu;
>> -
>> - if (!core->fw.dev)
>> - return;
>> -
>> - iommu = core->fw.iommu_domain;
>> -
>> - iommu_detach_device(iommu, core->fw.dev);
>> -
>> - if (core->fw.iommu_domain) {
>> - iommu_domain_free(iommu);
>> - core->fw.iommu_domain = NULL;
>> - }
>> -
>> - platform_device_unregister(to_platform_device(core->fw.dev));
>> -}
>> diff --git a/drivers/media/platform/qcom/venus/firmware.h
>> b/drivers/media/platform/qcom/venus/firmware.h
>> deleted file mode 100644
>> index aaccd84..0000000
>> --- a/drivers/media/platform/qcom/venus/firmware.h
>> +++ /dev/null
>> @@ -1,26 +0,0 @@
>> -/* SPDX-License-Identifier: GPL-2.0-only */
>> -/*
>> - * Copyright (C) 2017 Linaro Ltd.
>> - */
>> -#ifndef __VENUS_FIRMWARE_H__
>> -#define __VENUS_FIRMWARE_H__
>> -
>> -struct device;
>> -
>> -int venus_firmware_init(struct venus_core *core);
>> -void venus_firmware_deinit(struct venus_core *core);
>> -int venus_boot(struct venus_core *core);
>> -int venus_shutdown(struct venus_core *core);
>> -int venus_set_hw_state(struct venus_core *core, bool suspend);
>> -
>> -static inline int venus_set_hw_state_suspend(struct venus_core *core)
>> -{
>> - return venus_set_hw_state(core, false);
>> -}
>> -
>> -static inline int venus_set_hw_state_resume(struct venus_core *core)
>> -{
>> - return venus_set_hw_state(core, true);
>> -}
>> -
>> -#endif
>
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-20 8:01 ` Dikshita Agarwal
@ 2023-12-20 8:12 ` Dmitry Baryshkov
2023-12-20 17:10 ` Abhinav Kumar
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-20 8:12 UTC (permalink / raw)
To: Dikshita Agarwal
Cc: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm, quic_abhinavk
On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
<quic_dikshita@quicinc.com> wrote:
>
>
>
> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
> > On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >> Re-organize the video driver code by introducing a new folder
> >> 'vcodec' and placing 'venus' driver code inside that.
> >>
> >> Introduce common helpers for trustzone based firmware
> >> load/unload etc. which are placed in common folder
> >> i.e. 'vcodec'.
> >> Use these helpers in 'venus' driver. These helpers will be
> >> used by 'iris' driver as well which is introduced later
> >> in this patch series.
> >
> > But why do you need to move the venus driver to subdir?
>
> Currently venus driver is present in drivers/media/platform/qcom folder
> which also has camss folder. We introduced vcodec to keep common code and
> moved venus inside that, to indicate that the common code is for vcodec
> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
> code will be used for camss only which is not the case here.
you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
.../platform/qcom/venus and .../platform/qcom/iris.
If you were to use build helpers in a proper kernel module, this would
be more obvious.
> >
> >>
> >> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> >> ---
> >> drivers/media/platform/qcom/Kconfig | 2 +-
> >> drivers/media/platform/qcom/Makefile | 2 +-
> >> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
> >> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
> >> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> >> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
> >> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
> >> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> >> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
> >> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
> >> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
> >> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> >> drivers/media/platform/qcom/venus/firmware.c | 363
> >> ---------------------
> >> drivers/media/platform/qcom/venus/firmware.h | 26 --
> >> 42 files changed, 492 insertions(+), 409 deletions(-)
> >> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> >> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> >> create mode 100644
> >> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> >> create mode 100644
> >> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
> >> (100%)
> >> rename drivers/media/platform/qcom/{ =>
> >> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> >> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> >> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
> >>
> >> diff --git a/drivers/media/platform/qcom/Kconfig
> >> b/drivers/media/platform/qcom/Kconfig
> >> index cc5799b..e94142f 100644
> >> --- a/drivers/media/platform/qcom/Kconfig
> >> +++ b/drivers/media/platform/qcom/Kconfig
> >> @@ -3,4 +3,4 @@
> >> comment "Qualcomm media platform drivers"
> >> source "drivers/media/platform/qcom/camss/Kconfig"
> >> -source "drivers/media/platform/qcom/venus/Kconfig"
> >> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> >> diff --git a/drivers/media/platform/qcom/Makefile
> >> b/drivers/media/platform/qcom/Makefile
> >> index 4f055c3..3d2d82b 100644
> >> --- a/drivers/media/platform/qcom/Makefile
> >> +++ b/drivers/media/platform/qcom/Makefile
> >> @@ -1,3 +1,3 @@
> >> # SPDX-License-Identifier: GPL-2.0-only
> >> obj-y += camss/
> >> -obj-y += venus/
> >> +obj-y += vcodec/venus/
> >> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
> >> b/drivers/media/platform/qcom/vcodec/firmware.c
> >> new file mode 100644
> >> index 0000000..dbc220a
> >> --- /dev/null
> >> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
> >> @@ -0,0 +1,147 @@
> >> +// SPDX-License-Identifier: GPL-2.0-only
> >> +/*
> >> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >> reserved.
> >> + */
> >> +
> >> +#include <linux/device.h>
> >> +#include <linux/dma-mapping.h>
> >> +#include <linux/firmware.h>
> >> +#include <linux/kernel.h>
> >> +#include <linux/iommu.h>
> >> +#include <linux/of_device.h>
> >> +#include <linux/firmware/qcom/qcom_scm.h>
> >> +#include <linux/of_reserved_mem.h>
> >> +#include <linux/platform_device.h>
> >> +#include <linux/soc/qcom/mdt_loader.h>
> >> +
> >> +#include "firmware.h"
> >> +
> >> +bool use_tz(struct device *core_dev)
> >
> > All these functions must get some sane prefix. Otherwise a generic 'use_tz'
> > function is too polluting for the global namespace.
> >
> I understand, will check and do the needful.
> >> +{
> >> + struct device_node *np;
> >> +
> >> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
> >> + if (!np)
> >> + return true;
> >> +
> >> + return false;
> >> +}
> >> +
> >> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >> + u32 cp_nonpixel_size, u32 pas_id)
> >> +{
> >> + int ret;
> >> + /*
> >> + * Clues for porting using downstream data:
> >> + * cp_start = 0
> >> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> >> + * This works, as the non-secure context bank is placed
> >> + * contiguously right after the Content Protection region.
> >> + *
> >> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> >> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> >> + */
> >> + ret = qcom_scm_mem_protect_video_var(cp_start,
> >> + cp_size,
> >> + cp_nonpixel_start,
> >> + cp_nonpixel_size);
> >> + if (ret)
> >> + qcom_scm_pas_shutdown(pas_id);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >> + size_t *mem_size, u32 pas_id, bool use_tz)
> >> +{
> >> + const struct firmware *firmware = NULL;
> >> + struct reserved_mem *rmem;
> >> + struct device_node *node;
> >> + void *mem_virt = NULL;
> >> + ssize_t fw_size = 0;
> >> + int ret;
> >> +
> >> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> >
> > Why? Can you just depend on it?
> >
> Sure, Will check this and get back.
> >> + (use_tz && !qcom_scm_is_available()))
> >> + return -EPROBE_DEFER;
> >> +
> >> + if (!fw_name || !(*fw_name))
> >> + return -EINVAL;
> >> +
> >> + *mem_phys = 0;
> >> + *mem_size = 0;
> >> +
> >> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
> >> + if (!node) {
> >> + dev_err(dev, "no memory-region specified\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + rmem = of_reserved_mem_lookup(node);
> >> + of_node_put(node);
> >> + if (!rmem) {
> >> + dev_err(dev, "failed to lookup reserved memory-region\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + ret = request_firmware(&firmware, fw_name, dev);
> >> + if (ret) {
> >> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
> >> + __func__, fw_name, ret);
> >> + return ret;
> >> + }
> >> +
> >> + fw_size = qcom_mdt_get_size(firmware);
> >> + if (fw_size < 0) {
> >> + ret = fw_size;
> >> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
> >> + __func__, fw_size);
> >> + goto err_release_fw;
> >> + }
> >> +
> >> + *mem_phys = rmem->base;
> >> + *mem_size = rmem->size;
> >> +
> >> + if (*mem_size < fw_size) {
> >> + ret = -EINVAL;
> >> + goto err_release_fw;
> >> + }
> >> +
> >> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> >> + if (!mem_virt) {
> >> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
> >> + mem_phys, *mem_size);
> >> + goto err_release_fw;
> >> + }
> >> +
> >> + if (use_tz)
> >> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
> >> + *mem_phys, *mem_size, NULL);
> >> + else
> >> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
> >> mem_virt,
> >> + *mem_phys, *mem_size, NULL);
> >> + if (ret) {
> >> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
> >> + __func__, ret, fw_name);
> >> + }
> >> +
> >> + memunmap(mem_virt);
> >> +err_release_fw:
> >> + release_firmware(firmware);
> >> + return ret;
> >> +}
> >> +
> >> +int auth_reset_fw(u32 pas_id)
> >> +{
> >> + return qcom_scm_pas_auth_and_reset(pas_id);
> >> +}
> >> +
> >> +void unload_fw(u32 pas_id)
> >> +{
> >> + qcom_scm_pas_shutdown(pas_id);
> >> +}
> >> +
> >> +int set_hw_state(bool resume)
> >> +{
> >> + return qcom_scm_set_remote_state(resume, 0);
> >> +}
> >> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
> >> b/drivers/media/platform/qcom/vcodec/firmware.h
> >> new file mode 100644
> >> index 0000000..7d410a8
> >> --- /dev/null
> >> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
> >> @@ -0,0 +1,21 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +/*
> >> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >> reserved.
> >> + */
> >> +
> >> +#ifndef _FIRMWARE_H_
> >> +#define _FIRMWARE_H_
> >> +
> >> +#include <linux/device.h>
> >> +#include <linux/types.h>
> >> +
> >> +bool use_tz(struct device *core_dev);
> >> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >> + size_t *mem_size, u32 pas_id, bool use_tz);
> >> +int auth_reset_fw(u32 pas_id);
> >> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >> + u32 cp_nonpixel_size, u32 pas_id);
> >> +void unload_fw(u32 pas_id);
> >> +int set_hw_state(bool resume);
> >> +
> >> +#endif
> >> diff --git a/drivers/media/platform/qcom/venus/Kconfig
> >> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
> >> similarity index 100%
> >> rename from drivers/media/platform/qcom/venus/Kconfig
> >> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
> >> diff --git a/drivers/media/platform/qcom/venus/Makefile
> >> b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> similarity index 83%
> >> rename from drivers/media/platform/qcom/venus/Makefile
> >> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
> >> index 91ee6be..f6f3a88 100644
> >> --- a/drivers/media/platform/qcom/venus/Makefile
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> @@ -1,7 +1,9 @@
> >> # SPDX-License-Identifier: GPL-2.0
> >> # Makefile for Qualcomm Venus driver
> >> -venus-core-objs += core.o helpers.o firmware.o \
> >> +venus-core-objs += ../firmware.o
> >> +
> >> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
> >> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> >> hfi_parser.o pm_helpers.o dbgfs.o \
> >> hfi_platform.o hfi_platform_v4.o \
> >> diff --git a/drivers/media/platform/qcom/venus/core.c
> >> b/drivers/media/platform/qcom/vcodec/venus/core.c
> >> similarity index 91%
> >> rename from drivers/media/platform/qcom/venus/core.c
> >> rename to drivers/media/platform/qcom/vcodec/venus/core.c
> >> index 9cffe97..56d9a53 100644
> >> --- a/drivers/media/platform/qcom/venus/core.c
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
> >> @@ -22,7 +22,8 @@
> >> #include <media/v4l2-ioctl.h>
> >> #include "core.h"
> >> -#include "firmware.h"
> >> +#include "../firmware.h"
> >> +#include "firmware_no_tz.h"
> >> #include "pm_helpers.h"
> >> #include "hfi_venus_io.h"
> >> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
> >> work_struct *work)
> >> struct venus_core *core =
> >> container_of(work, struct venus_core, work.work);
> >> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
> >> + const struct venus_resources *res = core->res;
> >> + const char *fwpath = NULL;
> >> const char *err_msg = "";
> >> bool failed = false;
> >> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
> >> work_struct *work)
> >> mutex_lock(&core->lock);
> >> - venus_shutdown(core);
> >> + if (core->use_tz)
> >> + unload_fw(VENUS_PAS_ID);
> >> + else
> >> + unload_fw_no_tz(core);
> >
> > This is more than introducing helpers.
> >
> The new helpers are written to make the code generic for video drivers.
> which requires changes in the calling function also.
> >> venus_coredump(core);
> >> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
> >> work_struct *work)
> >> failed = true;
> >> }
> >> - ret = venus_boot(core);
> >> + ret = of_property_read_string_index(core->dev->of_node,
> >> "firmware-name", 0,
> >> + &fwpath);
> >> + if (ret)
> >> + fwpath = core->res->fwname;
> >> +
> >> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
> >> &core->fw.mem_size,
> >> + VENUS_PAS_ID, core->use_tz);
> >
> > So, we had a nice local 'venus_boot'. Instead we now have a pile of code
> > with non-generic prefixes, etc. If you are introducing helpers, please
> > refrain from inlining of calling functions, etc. Just move the code to your
> > helpers.
> >
> As mentioned in above comment, the common helpers are written to make the
> code generic. I Will try to make it more clear, working on the same.
First, you move the code, then you make it generic. Or vice versa.
First you split the code, then you move it. Don't do both in the same
patch.
> > NAK for the rest of the patch.
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-20 8:12 ` Dmitry Baryshkov
@ 2023-12-20 17:10 ` Abhinav Kumar
2023-12-20 20:56 ` Dmitry Baryshkov
0 siblings, 1 reply; 102+ messages in thread
From: Abhinav Kumar @ 2023-12-20 17:10 UTC (permalink / raw)
To: Dmitry Baryshkov, Dikshita Agarwal
Cc: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm
Hi Dmitry
On 12/20/2023 12:12 AM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
> <quic_dikshita@quicinc.com> wrote:
>>
>>
>>
>> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>> Re-organize the video driver code by introducing a new folder
>>>> 'vcodec' and placing 'venus' driver code inside that.
>>>>
>>>> Introduce common helpers for trustzone based firmware
>>>> load/unload etc. which are placed in common folder
>>>> i.e. 'vcodec'.
>>>> Use these helpers in 'venus' driver. These helpers will be
>>>> used by 'iris' driver as well which is introduced later
>>>> in this patch series.
>>>
>>> But why do you need to move the venus driver to subdir?
>>
>> Currently venus driver is present in drivers/media/platform/qcom folder
>> which also has camss folder. We introduced vcodec to keep common code and
>> moved venus inside that, to indicate that the common code is for vcodec
>> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
>> code will be used for camss only which is not the case here.
>
> you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
> .../platform/qcom/venus and .../platform/qcom/iris.
>
> If you were to use build helpers in a proper kernel module, this would
> be more obvious.
>
Although your suggestion is good in terms of avoiding moving venus, I
think the location of venus was wrong to begin with. There should have
always been a vcodec (or similar) folder for venus/iris as that will
establish the boundaries of camss and video sub-system in a cleaner way
I like the mediatek separation that way as it makes the boundaries clear:
drivers/media/platform/mediatek$ ls
jpeg Kconfig Makefile mdp mdp3 vcodec vpu
So I think that this re-org of venus into a vcodec had to happen at some
point. Its just that it aligned with iris addition.
>>>
>>>>
>>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>>> ---
>>>> drivers/media/platform/qcom/Kconfig | 2 +-
>>>> drivers/media/platform/qcom/Makefile | 2 +-
>>>> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
>>>> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
>>>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
>>>> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
>>>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
>>>> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
>>>> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
>>>> drivers/media/platform/qcom/venus/firmware.c | 363
>>>> ---------------------
>>>> drivers/media/platform/qcom/venus/firmware.h | 26 --
>>>> 42 files changed, 492 insertions(+), 409 deletions(-)
>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
>>>> create mode 100644
>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>>>> create mode 100644
>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ =>
>>>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>>>>
>>>> diff --git a/drivers/media/platform/qcom/Kconfig
>>>> b/drivers/media/platform/qcom/Kconfig
>>>> index cc5799b..e94142f 100644
>>>> --- a/drivers/media/platform/qcom/Kconfig
>>>> +++ b/drivers/media/platform/qcom/Kconfig
>>>> @@ -3,4 +3,4 @@
>>>> comment "Qualcomm media platform drivers"
>>>> source "drivers/media/platform/qcom/camss/Kconfig"
>>>> -source "drivers/media/platform/qcom/venus/Kconfig"
>>>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
>>>> diff --git a/drivers/media/platform/qcom/Makefile
>>>> b/drivers/media/platform/qcom/Makefile
>>>> index 4f055c3..3d2d82b 100644
>>>> --- a/drivers/media/platform/qcom/Makefile
>>>> +++ b/drivers/media/platform/qcom/Makefile
>>>> @@ -1,3 +1,3 @@
>>>> # SPDX-License-Identifier: GPL-2.0-only
>>>> obj-y += camss/
>>>> -obj-y += venus/
>>>> +obj-y += vcodec/venus/
>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
>>>> b/drivers/media/platform/qcom/vcodec/firmware.c
>>>> new file mode 100644
>>>> index 0000000..dbc220a
>>>> --- /dev/null
>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
>>>> @@ -0,0 +1,147 @@
>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>> +/*
>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>> reserved.
>>>> + */
>>>> +
>>>> +#include <linux/device.h>
>>>> +#include <linux/dma-mapping.h>
>>>> +#include <linux/firmware.h>
>>>> +#include <linux/kernel.h>
>>>> +#include <linux/iommu.h>
>>>> +#include <linux/of_device.h>
>>>> +#include <linux/firmware/qcom/qcom_scm.h>
>>>> +#include <linux/of_reserved_mem.h>
>>>> +#include <linux/platform_device.h>
>>>> +#include <linux/soc/qcom/mdt_loader.h>
>>>> +
>>>> +#include "firmware.h"
>>>> +
>>>> +bool use_tz(struct device *core_dev)
>>>
>>> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
>>> function is too polluting for the global namespace.
>>>
>> I understand, will check and do the needful.
>>>> +{
>>>> + struct device_node *np;
>>>> +
>>>> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
>>>> + if (!np)
>>>> + return true;
>>>> +
>>>> + return false;
>>>> +}
>>>> +
>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>> + u32 cp_nonpixel_size, u32 pas_id)
>>>> +{
>>>> + int ret;
>>>> + /*
>>>> + * Clues for porting using downstream data:
>>>> + * cp_start = 0
>>>> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
>>>> + * This works, as the non-secure context bank is placed
>>>> + * contiguously right after the Content Protection region.
>>>> + *
>>>> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>>>> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>>>> + */
>>>> + ret = qcom_scm_mem_protect_video_var(cp_start,
>>>> + cp_size,
>>>> + cp_nonpixel_start,
>>>> + cp_nonpixel_size);
>>>> + if (ret)
>>>> + qcom_scm_pas_shutdown(pas_id);
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>> + size_t *mem_size, u32 pas_id, bool use_tz)
>>>> +{
>>>> + const struct firmware *firmware = NULL;
>>>> + struct reserved_mem *rmem;
>>>> + struct device_node *node;
>>>> + void *mem_virt = NULL;
>>>> + ssize_t fw_size = 0;
>>>> + int ret;
>>>> +
>>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>>>
>>> Why? Can you just depend on it?
>>>
>> Sure, Will check this and get back.
>>>> + (use_tz && !qcom_scm_is_available()))
>>>> + return -EPROBE_DEFER;
>>>> +
>>>> + if (!fw_name || !(*fw_name))
>>>> + return -EINVAL;
>>>> +
>>>> + *mem_phys = 0;
>>>> + *mem_size = 0;
>>>> +
>>>> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
>>>> + if (!node) {
>>>> + dev_err(dev, "no memory-region specified\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + rmem = of_reserved_mem_lookup(node);
>>>> + of_node_put(node);
>>>> + if (!rmem) {
>>>> + dev_err(dev, "failed to lookup reserved memory-region\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + ret = request_firmware(&firmware, fw_name, dev);
>>>> + if (ret) {
>>>> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
>>>> + __func__, fw_name, ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + fw_size = qcom_mdt_get_size(firmware);
>>>> + if (fw_size < 0) {
>>>> + ret = fw_size;
>>>> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
>>>> + __func__, fw_size);
>>>> + goto err_release_fw;
>>>> + }
>>>> +
>>>> + *mem_phys = rmem->base;
>>>> + *mem_size = rmem->size;
>>>> +
>>>> + if (*mem_size < fw_size) {
>>>> + ret = -EINVAL;
>>>> + goto err_release_fw;
>>>> + }
>>>> +
>>>> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>>>> + if (!mem_virt) {
>>>> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
>>>> + mem_phys, *mem_size);
>>>> + goto err_release_fw;
>>>> + }
>>>> +
>>>> + if (use_tz)
>>>> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
>>>> + *mem_phys, *mem_size, NULL);
>>>> + else
>>>> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
>>>> mem_virt,
>>>> + *mem_phys, *mem_size, NULL);
>>>> + if (ret) {
>>>> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
>>>> + __func__, ret, fw_name);
>>>> + }
>>>> +
>>>> + memunmap(mem_virt);
>>>> +err_release_fw:
>>>> + release_firmware(firmware);
>>>> + return ret;
>>>> +}
>>>> +
>>>> +int auth_reset_fw(u32 pas_id)
>>>> +{
>>>> + return qcom_scm_pas_auth_and_reset(pas_id);
>>>> +}
>>>> +
>>>> +void unload_fw(u32 pas_id)
>>>> +{
>>>> + qcom_scm_pas_shutdown(pas_id);
>>>> +}
>>>> +
>>>> +int set_hw_state(bool resume)
>>>> +{
>>>> + return qcom_scm_set_remote_state(resume, 0);
>>>> +}
>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
>>>> b/drivers/media/platform/qcom/vcodec/firmware.h
>>>> new file mode 100644
>>>> index 0000000..7d410a8
>>>> --- /dev/null
>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
>>>> @@ -0,0 +1,21 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>> reserved.
>>>> + */
>>>> +
>>>> +#ifndef _FIRMWARE_H_
>>>> +#define _FIRMWARE_H_
>>>> +
>>>> +#include <linux/device.h>
>>>> +#include <linux/types.h>
>>>> +
>>>> +bool use_tz(struct device *core_dev);
>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>> + size_t *mem_size, u32 pas_id, bool use_tz);
>>>> +int auth_reset_fw(u32 pas_id);
>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>> + u32 cp_nonpixel_size, u32 pas_id);
>>>> +void unload_fw(u32 pas_id);
>>>> +int set_hw_state(bool resume);
>>>> +
>>>> +#endif
>>>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
>>>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>> similarity index 100%
>>>> rename from drivers/media/platform/qcom/venus/Kconfig
>>>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>> diff --git a/drivers/media/platform/qcom/venus/Makefile
>>>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>> similarity index 83%
>>>> rename from drivers/media/platform/qcom/venus/Makefile
>>>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
>>>> index 91ee6be..f6f3a88 100644
>>>> --- a/drivers/media/platform/qcom/venus/Makefile
>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>> @@ -1,7 +1,9 @@
>>>> # SPDX-License-Identifier: GPL-2.0
>>>> # Makefile for Qualcomm Venus driver
>>>> -venus-core-objs += core.o helpers.o firmware.o \
>>>> +venus-core-objs += ../firmware.o
>>>> +
>>>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
>>>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>>>> hfi_parser.o pm_helpers.o dbgfs.o \
>>>> hfi_platform.o hfi_platform_v4.o \
>>>> diff --git a/drivers/media/platform/qcom/venus/core.c
>>>> b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>> similarity index 91%
>>>> rename from drivers/media/platform/qcom/venus/core.c
>>>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
>>>> index 9cffe97..56d9a53 100644
>>>> --- a/drivers/media/platform/qcom/venus/core.c
>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>> @@ -22,7 +22,8 @@
>>>> #include <media/v4l2-ioctl.h>
>>>> #include "core.h"
>>>> -#include "firmware.h"
>>>> +#include "../firmware.h"
>>>> +#include "firmware_no_tz.h"
>>>> #include "pm_helpers.h"
>>>> #include "hfi_venus_io.h"
>>>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
>>>> work_struct *work)
>>>> struct venus_core *core =
>>>> container_of(work, struct venus_core, work.work);
>>>> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
>>>> + const struct venus_resources *res = core->res;
>>>> + const char *fwpath = NULL;
>>>> const char *err_msg = "";
>>>> bool failed = false;
>>>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
>>>> work_struct *work)
>>>> mutex_lock(&core->lock);
>>>> - venus_shutdown(core);
>>>> + if (core->use_tz)
>>>> + unload_fw(VENUS_PAS_ID);
>>>> + else
>>>> + unload_fw_no_tz(core);
>>>
>>> This is more than introducing helpers.
>>>
>> The new helpers are written to make the code generic for video drivers.
>> which requires changes in the calling function also.
>>>> venus_coredump(core);
>>>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
>>>> work_struct *work)
>>>> failed = true;
>>>> }
>>>> - ret = venus_boot(core);
>>>> + ret = of_property_read_string_index(core->dev->of_node,
>>>> "firmware-name", 0,
>>>> + &fwpath);
>>>> + if (ret)
>>>> + fwpath = core->res->fwname;
>>>> +
>>>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>>>> &core->fw.mem_size,
>>>> + VENUS_PAS_ID, core->use_tz);
>>>
>>> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
>>> with non-generic prefixes, etc. If you are introducing helpers, please
>>> refrain from inlining of calling functions, etc. Just move the code to your
>>> helpers.
>>>
>> As mentioned in above comment, the common helpers are written to make the
>> code generic. I Will try to make it more clear, working on the same.
>
> First, you move the code, then you make it generic. Or vice versa.
> First you split the code, then you move it. Don't do both in the same
> patch.
>
>>> NAK for the rest of the patch.
>
>
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-20 17:10 ` Abhinav Kumar
@ 2023-12-20 20:56 ` Dmitry Baryshkov
2023-12-20 21:03 ` Abhinav Kumar
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-20 20:56 UTC (permalink / raw)
To: Abhinav Kumar
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue, linux-arm-msm
Hi Abhinav,
On Wed, 20 Dec 2023 at 19:10, Abhinav Kumar <quic_abhinavk@quicinc.com> wrote:
>
> Hi Dmitry
>
> On 12/20/2023 12:12 AM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
> > <quic_dikshita@quicinc.com> wrote:
> >>
> >>
> >>
> >> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
> >>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>> Re-organize the video driver code by introducing a new folder
> >>>> 'vcodec' and placing 'venus' driver code inside that.
> >>>>
> >>>> Introduce common helpers for trustzone based firmware
> >>>> load/unload etc. which are placed in common folder
> >>>> i.e. 'vcodec'.
> >>>> Use these helpers in 'venus' driver. These helpers will be
> >>>> used by 'iris' driver as well which is introduced later
> >>>> in this patch series.
> >>>
> >>> But why do you need to move the venus driver to subdir?
> >>
> >> Currently venus driver is present in drivers/media/platform/qcom folder
> >> which also has camss folder. We introduced vcodec to keep common code and
> >> moved venus inside that, to indicate that the common code is for vcodec
> >> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
> >> code will be used for camss only which is not the case here.
> >
> > you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
> > .../platform/qcom/venus and .../platform/qcom/iris.
> >
> > If you were to use build helpers in a proper kernel module, this would
> > be more obvious.
> >
>
> Although your suggestion is good in terms of avoiding moving venus, I
> think the location of venus was wrong to begin with. There should have
> always been a vcodec (or similar) folder for venus/iris as that will
> establish the boundaries of camss and video sub-system in a cleaner way
>
> I like the mediatek separation that way as it makes the boundaries clear:
>
> drivers/media/platform/mediatek$ ls
> jpeg Kconfig Makefile mdp mdp3 vcodec vpu
>
> So I think that this re-org of venus into a vcodec had to happen at some
> point. Its just that it aligned with iris addition.
Then it should be a clean separate step with its own justification.
>
> >>>
> >>>>
> >>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> >>>> ---
> >>>> drivers/media/platform/qcom/Kconfig | 2 +-
> >>>> drivers/media/platform/qcom/Makefile | 2 +-
> >>>> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
> >>>> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
> >>>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
> >>>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
> >>>> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> >>>> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
> >>>> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
> >>>> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> >>>> drivers/media/platform/qcom/venus/firmware.c | 363
> >>>> ---------------------
> >>>> drivers/media/platform/qcom/venus/firmware.h | 26 --
> >>>> 42 files changed, 492 insertions(+), 409 deletions(-)
> >>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> >>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> >>>> create mode 100644
> >>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> >>>> create mode 100644
> >>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ =>
> >>>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> >>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> >>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
> >>>>
> >>>> diff --git a/drivers/media/platform/qcom/Kconfig
> >>>> b/drivers/media/platform/qcom/Kconfig
> >>>> index cc5799b..e94142f 100644
> >>>> --- a/drivers/media/platform/qcom/Kconfig
> >>>> +++ b/drivers/media/platform/qcom/Kconfig
> >>>> @@ -3,4 +3,4 @@
> >>>> comment "Qualcomm media platform drivers"
> >>>> source "drivers/media/platform/qcom/camss/Kconfig"
> >>>> -source "drivers/media/platform/qcom/venus/Kconfig"
> >>>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> >>>> diff --git a/drivers/media/platform/qcom/Makefile
> >>>> b/drivers/media/platform/qcom/Makefile
> >>>> index 4f055c3..3d2d82b 100644
> >>>> --- a/drivers/media/platform/qcom/Makefile
> >>>> +++ b/drivers/media/platform/qcom/Makefile
> >>>> @@ -1,3 +1,3 @@
> >>>> # SPDX-License-Identifier: GPL-2.0-only
> >>>> obj-y += camss/
> >>>> -obj-y += venus/
> >>>> +obj-y += vcodec/venus/
> >>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
> >>>> b/drivers/media/platform/qcom/vcodec/firmware.c
> >>>> new file mode 100644
> >>>> index 0000000..dbc220a
> >>>> --- /dev/null
> >>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
> >>>> @@ -0,0 +1,147 @@
> >>>> +// SPDX-License-Identifier: GPL-2.0-only
> >>>> +/*
> >>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >>>> reserved.
> >>>> + */
> >>>> +
> >>>> +#include <linux/device.h>
> >>>> +#include <linux/dma-mapping.h>
> >>>> +#include <linux/firmware.h>
> >>>> +#include <linux/kernel.h>
> >>>> +#include <linux/iommu.h>
> >>>> +#include <linux/of_device.h>
> >>>> +#include <linux/firmware/qcom/qcom_scm.h>
> >>>> +#include <linux/of_reserved_mem.h>
> >>>> +#include <linux/platform_device.h>
> >>>> +#include <linux/soc/qcom/mdt_loader.h>
> >>>> +
> >>>> +#include "firmware.h"
> >>>> +
> >>>> +bool use_tz(struct device *core_dev)
> >>>
> >>> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
> >>> function is too polluting for the global namespace.
> >>>
> >> I understand, will check and do the needful.
> >>>> +{
> >>>> + struct device_node *np;
> >>>> +
> >>>> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
> >>>> + if (!np)
> >>>> + return true;
> >>>> +
> >>>> + return false;
> >>>> +}
> >>>> +
> >>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >>>> + u32 cp_nonpixel_size, u32 pas_id)
> >>>> +{
> >>>> + int ret;
> >>>> + /*
> >>>> + * Clues for porting using downstream data:
> >>>> + * cp_start = 0
> >>>> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> >>>> + * This works, as the non-secure context bank is placed
> >>>> + * contiguously right after the Content Protection region.
> >>>> + *
> >>>> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> >>>> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> >>>> + */
> >>>> + ret = qcom_scm_mem_protect_video_var(cp_start,
> >>>> + cp_size,
> >>>> + cp_nonpixel_start,
> >>>> + cp_nonpixel_size);
> >>>> + if (ret)
> >>>> + qcom_scm_pas_shutdown(pas_id);
> >>>> +
> >>>> + return ret;
> >>>> +}
> >>>> +
> >>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >>>> + size_t *mem_size, u32 pas_id, bool use_tz)
> >>>> +{
> >>>> + const struct firmware *firmware = NULL;
> >>>> + struct reserved_mem *rmem;
> >>>> + struct device_node *node;
> >>>> + void *mem_virt = NULL;
> >>>> + ssize_t fw_size = 0;
> >>>> + int ret;
> >>>> +
> >>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> >>>
> >>> Why? Can you just depend on it?
> >>>
> >> Sure, Will check this and get back.
> >>>> + (use_tz && !qcom_scm_is_available()))
> >>>> + return -EPROBE_DEFER;
> >>>> +
> >>>> + if (!fw_name || !(*fw_name))
> >>>> + return -EINVAL;
> >>>> +
> >>>> + *mem_phys = 0;
> >>>> + *mem_size = 0;
> >>>> +
> >>>> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
> >>>> + if (!node) {
> >>>> + dev_err(dev, "no memory-region specified\n");
> >>>> + return -EINVAL;
> >>>> + }
> >>>> +
> >>>> + rmem = of_reserved_mem_lookup(node);
> >>>> + of_node_put(node);
> >>>> + if (!rmem) {
> >>>> + dev_err(dev, "failed to lookup reserved memory-region\n");
> >>>> + return -EINVAL;
> >>>> + }
> >>>> +
> >>>> + ret = request_firmware(&firmware, fw_name, dev);
> >>>> + if (ret) {
> >>>> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
> >>>> + __func__, fw_name, ret);
> >>>> + return ret;
> >>>> + }
> >>>> +
> >>>> + fw_size = qcom_mdt_get_size(firmware);
> >>>> + if (fw_size < 0) {
> >>>> + ret = fw_size;
> >>>> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
> >>>> + __func__, fw_size);
> >>>> + goto err_release_fw;
> >>>> + }
> >>>> +
> >>>> + *mem_phys = rmem->base;
> >>>> + *mem_size = rmem->size;
> >>>> +
> >>>> + if (*mem_size < fw_size) {
> >>>> + ret = -EINVAL;
> >>>> + goto err_release_fw;
> >>>> + }
> >>>> +
> >>>> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> >>>> + if (!mem_virt) {
> >>>> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
> >>>> + mem_phys, *mem_size);
> >>>> + goto err_release_fw;
> >>>> + }
> >>>> +
> >>>> + if (use_tz)
> >>>> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
> >>>> + *mem_phys, *mem_size, NULL);
> >>>> + else
> >>>> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
> >>>> mem_virt,
> >>>> + *mem_phys, *mem_size, NULL);
> >>>> + if (ret) {
> >>>> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
> >>>> + __func__, ret, fw_name);
> >>>> + }
> >>>> +
> >>>> + memunmap(mem_virt);
> >>>> +err_release_fw:
> >>>> + release_firmware(firmware);
> >>>> + return ret;
> >>>> +}
> >>>> +
> >>>> +int auth_reset_fw(u32 pas_id)
> >>>> +{
> >>>> + return qcom_scm_pas_auth_and_reset(pas_id);
> >>>> +}
> >>>> +
> >>>> +void unload_fw(u32 pas_id)
> >>>> +{
> >>>> + qcom_scm_pas_shutdown(pas_id);
> >>>> +}
> >>>> +
> >>>> +int set_hw_state(bool resume)
> >>>> +{
> >>>> + return qcom_scm_set_remote_state(resume, 0);
> >>>> +}
> >>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
> >>>> b/drivers/media/platform/qcom/vcodec/firmware.h
> >>>> new file mode 100644
> >>>> index 0000000..7d410a8
> >>>> --- /dev/null
> >>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
> >>>> @@ -0,0 +1,21 @@
> >>>> +/* SPDX-License-Identifier: GPL-2.0-only */
> >>>> +/*
> >>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >>>> reserved.
> >>>> + */
> >>>> +
> >>>> +#ifndef _FIRMWARE_H_
> >>>> +#define _FIRMWARE_H_
> >>>> +
> >>>> +#include <linux/device.h>
> >>>> +#include <linux/types.h>
> >>>> +
> >>>> +bool use_tz(struct device *core_dev);
> >>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >>>> + size_t *mem_size, u32 pas_id, bool use_tz);
> >>>> +int auth_reset_fw(u32 pas_id);
> >>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >>>> + u32 cp_nonpixel_size, u32 pas_id);
> >>>> +void unload_fw(u32 pas_id);
> >>>> +int set_hw_state(bool resume);
> >>>> +
> >>>> +#endif
> >>>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
> >>>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
> >>>> similarity index 100%
> >>>> rename from drivers/media/platform/qcom/venus/Kconfig
> >>>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
> >>>> diff --git a/drivers/media/platform/qcom/venus/Makefile
> >>>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >>>> similarity index 83%
> >>>> rename from drivers/media/platform/qcom/venus/Makefile
> >>>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
> >>>> index 91ee6be..f6f3a88 100644
> >>>> --- a/drivers/media/platform/qcom/venus/Makefile
> >>>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >>>> @@ -1,7 +1,9 @@
> >>>> # SPDX-License-Identifier: GPL-2.0
> >>>> # Makefile for Qualcomm Venus driver
> >>>> -venus-core-objs += core.o helpers.o firmware.o \
> >>>> +venus-core-objs += ../firmware.o
> >>>> +
> >>>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
> >>>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> >>>> hfi_parser.o pm_helpers.o dbgfs.o \
> >>>> hfi_platform.o hfi_platform_v4.o \
> >>>> diff --git a/drivers/media/platform/qcom/venus/core.c
> >>>> b/drivers/media/platform/qcom/vcodec/venus/core.c
> >>>> similarity index 91%
> >>>> rename from drivers/media/platform/qcom/venus/core.c
> >>>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
> >>>> index 9cffe97..56d9a53 100644
> >>>> --- a/drivers/media/platform/qcom/venus/core.c
> >>>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
> >>>> @@ -22,7 +22,8 @@
> >>>> #include <media/v4l2-ioctl.h>
> >>>> #include "core.h"
> >>>> -#include "firmware.h"
> >>>> +#include "../firmware.h"
> >>>> +#include "firmware_no_tz.h"
> >>>> #include "pm_helpers.h"
> >>>> #include "hfi_venus_io.h"
> >>>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
> >>>> work_struct *work)
> >>>> struct venus_core *core =
> >>>> container_of(work, struct venus_core, work.work);
> >>>> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
> >>>> + const struct venus_resources *res = core->res;
> >>>> + const char *fwpath = NULL;
> >>>> const char *err_msg = "";
> >>>> bool failed = false;
> >>>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
> >>>> work_struct *work)
> >>>> mutex_lock(&core->lock);
> >>>> - venus_shutdown(core);
> >>>> + if (core->use_tz)
> >>>> + unload_fw(VENUS_PAS_ID);
> >>>> + else
> >>>> + unload_fw_no_tz(core);
> >>>
> >>> This is more than introducing helpers.
> >>>
> >> The new helpers are written to make the code generic for video drivers.
> >> which requires changes in the calling function also.
> >>>> venus_coredump(core);
> >>>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
> >>>> work_struct *work)
> >>>> failed = true;
> >>>> }
> >>>> - ret = venus_boot(core);
> >>>> + ret = of_property_read_string_index(core->dev->of_node,
> >>>> "firmware-name", 0,
> >>>> + &fwpath);
> >>>> + if (ret)
> >>>> + fwpath = core->res->fwname;
> >>>> +
> >>>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
> >>>> &core->fw.mem_size,
> >>>> + VENUS_PAS_ID, core->use_tz);
> >>>
> >>> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
> >>> with non-generic prefixes, etc. If you are introducing helpers, please
> >>> refrain from inlining of calling functions, etc. Just move the code to your
> >>> helpers.
> >>>
> >> As mentioned in above comment, the common helpers are written to make the
> >> code generic. I Will try to make it more clear, working on the same.
> >
> > First, you move the code, then you make it generic. Or vice versa.
> > First you split the code, then you move it. Don't do both in the same
> > patch.
> >
> >>> NAK for the rest of the patch.
> >
> >
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-20 20:56 ` Dmitry Baryshkov
@ 2023-12-20 21:03 ` Abhinav Kumar
0 siblings, 0 replies; 102+ messages in thread
From: Abhinav Kumar @ 2023-12-20 21:03 UTC (permalink / raw)
To: Dmitry Baryshkov
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue, linux-arm-msm
On 12/20/2023 12:56 PM, Dmitry Baryshkov wrote:
> Hi Abhinav,
>
> On Wed, 20 Dec 2023 at 19:10, Abhinav Kumar <quic_abhinavk@quicinc.com> wrote:
>>
>> Hi Dmitry
>>
>> On 12/20/2023 12:12 AM, Dmitry Baryshkov wrote:
>>> On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
>>> <quic_dikshita@quicinc.com> wrote:
>>>>
>>>>
>>>>
>>>> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>>>> Re-organize the video driver code by introducing a new folder
>>>>>> 'vcodec' and placing 'venus' driver code inside that.
>>>>>>
>>>>>> Introduce common helpers for trustzone based firmware
>>>>>> load/unload etc. which are placed in common folder
>>>>>> i.e. 'vcodec'.
>>>>>> Use these helpers in 'venus' driver. These helpers will be
>>>>>> used by 'iris' driver as well which is introduced later
>>>>>> in this patch series.
>>>>>
>>>>> But why do you need to move the venus driver to subdir?
>>>>
>>>> Currently venus driver is present in drivers/media/platform/qcom folder
>>>> which also has camss folder. We introduced vcodec to keep common code and
>>>> moved venus inside that, to indicate that the common code is for vcodec
>>>> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
>>>> code will be used for camss only which is not the case here.
>>>
>>> you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
>>> .../platform/qcom/venus and .../platform/qcom/iris.
>>>
>>> If you were to use build helpers in a proper kernel module, this would
>>> be more obvious.
>>>
>>
>> Although your suggestion is good in terms of avoiding moving venus, I
>> think the location of venus was wrong to begin with. There should have
>> always been a vcodec (or similar) folder for venus/iris as that will
>> establish the boundaries of camss and video sub-system in a cleaner way
>>
>> I like the mediatek separation that way as it makes the boundaries clear:
>>
>> drivers/media/platform/mediatek$ ls
>> jpeg Kconfig Makefile mdp mdp3 vcodec vpu
>>
>> So I think that this re-org of venus into a vcodec had to happen at some
>> point. Its just that it aligned with iris addition.
>
> Then it should be a clean separate step with its own justification.
>
Yes, I am fine with this first going as a separate series to move venus
into vcodec and anyway there was a separate request to make patches 1-3
into another series from Bryan.
So this can first be posted separately.
>>
>>>>>
>>>>>>
>>>>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>>>>> ---
>>>>>> drivers/media/platform/qcom/Kconfig | 2 +-
>>>>>> drivers/media/platform/qcom/Makefile | 2 +-
>>>>>> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
>>>>>> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
>>>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
>>>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
>>>>>> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
>>>>>> drivers/media/platform/qcom/venus/firmware.c | 363
>>>>>> ---------------------
>>>>>> drivers/media/platform/qcom/venus/firmware.h | 26 --
>>>>>> 42 files changed, 492 insertions(+), 409 deletions(-)
>>>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
>>>>>> create mode 100644
>>>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>>>>>> create mode 100644
>>>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ =>
>>>>>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
>>>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
>>>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>>>>>>
>>>>>> diff --git a/drivers/media/platform/qcom/Kconfig
>>>>>> b/drivers/media/platform/qcom/Kconfig
>>>>>> index cc5799b..e94142f 100644
>>>>>> --- a/drivers/media/platform/qcom/Kconfig
>>>>>> +++ b/drivers/media/platform/qcom/Kconfig
>>>>>> @@ -3,4 +3,4 @@
>>>>>> comment "Qualcomm media platform drivers"
>>>>>> source "drivers/media/platform/qcom/camss/Kconfig"
>>>>>> -source "drivers/media/platform/qcom/venus/Kconfig"
>>>>>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
>>>>>> diff --git a/drivers/media/platform/qcom/Makefile
>>>>>> b/drivers/media/platform/qcom/Makefile
>>>>>> index 4f055c3..3d2d82b 100644
>>>>>> --- a/drivers/media/platform/qcom/Makefile
>>>>>> +++ b/drivers/media/platform/qcom/Makefile
>>>>>> @@ -1,3 +1,3 @@
>>>>>> # SPDX-License-Identifier: GPL-2.0-only
>>>>>> obj-y += camss/
>>>>>> -obj-y += venus/
>>>>>> +obj-y += vcodec/venus/
>>>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> b/drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> new file mode 100644
>>>>>> index 0000000..dbc220a
>>>>>> --- /dev/null
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> @@ -0,0 +1,147 @@
>>>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>>>> +/*
>>>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>>>> reserved.
>>>>>> + */
>>>>>> +
>>>>>> +#include <linux/device.h>
>>>>>> +#include <linux/dma-mapping.h>
>>>>>> +#include <linux/firmware.h>
>>>>>> +#include <linux/kernel.h>
>>>>>> +#include <linux/iommu.h>
>>>>>> +#include <linux/of_device.h>
>>>>>> +#include <linux/firmware/qcom/qcom_scm.h>
>>>>>> +#include <linux/of_reserved_mem.h>
>>>>>> +#include <linux/platform_device.h>
>>>>>> +#include <linux/soc/qcom/mdt_loader.h>
>>>>>> +
>>>>>> +#include "firmware.h"
>>>>>> +
>>>>>> +bool use_tz(struct device *core_dev)
>>>>>
>>>>> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
>>>>> function is too polluting for the global namespace.
>>>>>
>>>> I understand, will check and do the needful.
>>>>>> +{
>>>>>> + struct device_node *np;
>>>>>> +
>>>>>> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
>>>>>> + if (!np)
>>>>>> + return true;
>>>>>> +
>>>>>> + return false;
>>>>>> +}
>>>>>> +
>>>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>>>> + u32 cp_nonpixel_size, u32 pas_id)
>>>>>> +{
>>>>>> + int ret;
>>>>>> + /*
>>>>>> + * Clues for porting using downstream data:
>>>>>> + * cp_start = 0
>>>>>> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
>>>>>> + * This works, as the non-secure context bank is placed
>>>>>> + * contiguously right after the Content Protection region.
>>>>>> + *
>>>>>> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>>>>>> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>>>>>> + */
>>>>>> + ret = qcom_scm_mem_protect_video_var(cp_start,
>>>>>> + cp_size,
>>>>>> + cp_nonpixel_start,
>>>>>> + cp_nonpixel_size);
>>>>>> + if (ret)
>>>>>> + qcom_scm_pas_shutdown(pas_id);
>>>>>> +
>>>>>> + return ret;
>>>>>> +}
>>>>>> +
>>>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>>>> + size_t *mem_size, u32 pas_id, bool use_tz)
>>>>>> +{
>>>>>> + const struct firmware *firmware = NULL;
>>>>>> + struct reserved_mem *rmem;
>>>>>> + struct device_node *node;
>>>>>> + void *mem_virt = NULL;
>>>>>> + ssize_t fw_size = 0;
>>>>>> + int ret;
>>>>>> +
>>>>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>>>>>
>>>>> Why? Can you just depend on it?
>>>>>
>>>> Sure, Will check this and get back.
>>>>>> + (use_tz && !qcom_scm_is_available()))
>>>>>> + return -EPROBE_DEFER;
>>>>>> +
>>>>>> + if (!fw_name || !(*fw_name))
>>>>>> + return -EINVAL;
>>>>>> +
>>>>>> + *mem_phys = 0;
>>>>>> + *mem_size = 0;
>>>>>> +
>>>>>> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
>>>>>> + if (!node) {
>>>>>> + dev_err(dev, "no memory-region specified\n");
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + rmem = of_reserved_mem_lookup(node);
>>>>>> + of_node_put(node);
>>>>>> + if (!rmem) {
>>>>>> + dev_err(dev, "failed to lookup reserved memory-region\n");
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + ret = request_firmware(&firmware, fw_name, dev);
>>>>>> + if (ret) {
>>>>>> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
>>>>>> + __func__, fw_name, ret);
>>>>>> + return ret;
>>>>>> + }
>>>>>> +
>>>>>> + fw_size = qcom_mdt_get_size(firmware);
>>>>>> + if (fw_size < 0) {
>>>>>> + ret = fw_size;
>>>>>> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
>>>>>> + __func__, fw_size);
>>>>>> + goto err_release_fw;
>>>>>> + }
>>>>>> +
>>>>>> + *mem_phys = rmem->base;
>>>>>> + *mem_size = rmem->size;
>>>>>> +
>>>>>> + if (*mem_size < fw_size) {
>>>>>> + ret = -EINVAL;
>>>>>> + goto err_release_fw;
>>>>>> + }
>>>>>> +
>>>>>> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>>>>>> + if (!mem_virt) {
>>>>>> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
>>>>>> + mem_phys, *mem_size);
>>>>>> + goto err_release_fw;
>>>>>> + }
>>>>>> +
>>>>>> + if (use_tz)
>>>>>> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
>>>>>> + *mem_phys, *mem_size, NULL);
>>>>>> + else
>>>>>> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
>>>>>> mem_virt,
>>>>>> + *mem_phys, *mem_size, NULL);
>>>>>> + if (ret) {
>>>>>> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
>>>>>> + __func__, ret, fw_name);
>>>>>> + }
>>>>>> +
>>>>>> + memunmap(mem_virt);
>>>>>> +err_release_fw:
>>>>>> + release_firmware(firmware);
>>>>>> + return ret;
>>>>>> +}
>>>>>> +
>>>>>> +int auth_reset_fw(u32 pas_id)
>>>>>> +{
>>>>>> + return qcom_scm_pas_auth_and_reset(pas_id);
>>>>>> +}
>>>>>> +
>>>>>> +void unload_fw(u32 pas_id)
>>>>>> +{
>>>>>> + qcom_scm_pas_shutdown(pas_id);
>>>>>> +}
>>>>>> +
>>>>>> +int set_hw_state(bool resume)
>>>>>> +{
>>>>>> + return qcom_scm_set_remote_state(resume, 0);
>>>>>> +}
>>>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> b/drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> new file mode 100644
>>>>>> index 0000000..7d410a8
>>>>>> --- /dev/null
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> @@ -0,0 +1,21 @@
>>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>>>> +/*
>>>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>>>> reserved.
>>>>>> + */
>>>>>> +
>>>>>> +#ifndef _FIRMWARE_H_
>>>>>> +#define _FIRMWARE_H_
>>>>>> +
>>>>>> +#include <linux/device.h>
>>>>>> +#include <linux/types.h>
>>>>>> +
>>>>>> +bool use_tz(struct device *core_dev);
>>>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>>>> + size_t *mem_size, u32 pas_id, bool use_tz);
>>>>>> +int auth_reset_fw(u32 pas_id);
>>>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>>>> + u32 cp_nonpixel_size, u32 pas_id);
>>>>>> +void unload_fw(u32 pas_id);
>>>>>> +int set_hw_state(bool resume);
>>>>>> +
>>>>>> +#endif
>>>>>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
>>>>>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>>>> similarity index 100%
>>>>>> rename from drivers/media/platform/qcom/venus/Kconfig
>>>>>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>>>> diff --git a/drivers/media/platform/qcom/venus/Makefile
>>>>>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>>>> similarity index 83%
>>>>>> rename from drivers/media/platform/qcom/venus/Makefile
>>>>>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
>>>>>> index 91ee6be..f6f3a88 100644
>>>>>> --- a/drivers/media/platform/qcom/venus/Makefile
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>>>> @@ -1,7 +1,9 @@
>>>>>> # SPDX-License-Identifier: GPL-2.0
>>>>>> # Makefile for Qualcomm Venus driver
>>>>>> -venus-core-objs += core.o helpers.o firmware.o \
>>>>>> +venus-core-objs += ../firmware.o
>>>>>> +
>>>>>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
>>>>>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>>>>>> hfi_parser.o pm_helpers.o dbgfs.o \
>>>>>> hfi_platform.o hfi_platform_v4.o \
>>>>>> diff --git a/drivers/media/platform/qcom/venus/core.c
>>>>>> b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>>>> similarity index 91%
>>>>>> rename from drivers/media/platform/qcom/venus/core.c
>>>>>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
>>>>>> index 9cffe97..56d9a53 100644
>>>>>> --- a/drivers/media/platform/qcom/venus/core.c
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>>>> @@ -22,7 +22,8 @@
>>>>>> #include <media/v4l2-ioctl.h>
>>>>>> #include "core.h"
>>>>>> -#include "firmware.h"
>>>>>> +#include "../firmware.h"
>>>>>> +#include "firmware_no_tz.h"
>>>>>> #include "pm_helpers.h"
>>>>>> #include "hfi_venus_io.h"
>>>>>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
>>>>>> work_struct *work)
>>>>>> struct venus_core *core =
>>>>>> container_of(work, struct venus_core, work.work);
>>>>>> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
>>>>>> + const struct venus_resources *res = core->res;
>>>>>> + const char *fwpath = NULL;
>>>>>> const char *err_msg = "";
>>>>>> bool failed = false;
>>>>>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
>>>>>> work_struct *work)
>>>>>> mutex_lock(&core->lock);
>>>>>> - venus_shutdown(core);
>>>>>> + if (core->use_tz)
>>>>>> + unload_fw(VENUS_PAS_ID);
>>>>>> + else
>>>>>> + unload_fw_no_tz(core);
>>>>>
>>>>> This is more than introducing helpers.
>>>>>
>>>> The new helpers are written to make the code generic for video drivers.
>>>> which requires changes in the calling function also.
>>>>>> venus_coredump(core);
>>>>>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
>>>>>> work_struct *work)
>>>>>> failed = true;
>>>>>> }
>>>>>> - ret = venus_boot(core);
>>>>>> + ret = of_property_read_string_index(core->dev->of_node,
>>>>>> "firmware-name", 0,
>>>>>> + &fwpath);
>>>>>> + if (ret)
>>>>>> + fwpath = core->res->fwname;
>>>>>> +
>>>>>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>>>>>> &core->fw.mem_size,
>>>>>> + VENUS_PAS_ID, core->use_tz);
>>>>>
>>>>> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
>>>>> with non-generic prefixes, etc. If you are introducing helpers, please
>>>>> refrain from inlining of calling functions, etc. Just move the code to your
>>>>> helpers.
>>>>>
>>>> As mentioned in above comment, the common helpers are written to make the
>>>> code generic. I Will try to make it more clear, working on the same.
>>>
>>> First, you move the code, then you make it generic. Or vice versa.
>>> First you split the code, then you move it. Don't do both in the same
>>> patch.
>>>
>>>>> NAK for the rest of the patch.
>>>
>>>
>
>
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-18 11:31 ` [PATCH v2 01/34] media: introduce common helpers for video firmware handling Dikshita Agarwal
2023-12-18 18:24 ` Dmitry Baryshkov
@ 2023-12-19 11:40 ` Bryan O'Donoghue
2023-12-19 13:26 ` Dmitry Baryshkov
1 sibling, 1 reply; 102+ messages in thread
From: Bryan O'Donoghue @ 2023-12-19 11:40 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 11:31, Dikshita Agarwal wrote:
> Re-organize the video driver code by introducing a new folder
> 'vcodec' and placing 'venus' driver code inside that.
>
> Introduce common helpers for trustzone based firmware
> load/unload etc. which are placed in common folder
> i.e. 'vcodec'.
> Use these helpers in 'venus' driver. These helpers will be
> used by 'iris' driver as well which is introduced later
> in this patch series.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
This is a very large patch, I think it needs to be broken up into
smaller chunks.
#1 Introduce common helper functions
#2 Use common helper functions
Its alot of code to try to eat in the one go.
Could you consider making patches 1-3 a standalone series to reduce the
amount of code to review here ?
* 77e7025529d7c - (HEAD -> linux-stable-master-23-12-18-iris-v2) media:
iris: add power management for encoder (21 hours ago)
* ceb6a6f023fd3 - (tag: v6.7-rc6, linux-stable/master) Linux 6.7-rc6 (2
days ago)
git diff ceb6a6f023fd3 | wc -l
21243
Also I feel it wouild give more time for the changes to "digest" though
upstream users and to find any unintended bugs.
> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> + size_t *mem_size, u32 pas_id, bool use_tz)
> +{
> + const struct firmware *firmware = NULL;
> + struct reserved_mem *rmem;
> + struct device_node *node;
> + void *mem_virt = NULL;
> + ssize_t fw_size = 0;
> + int ret;
> +
> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> + (use_tz && !qcom_scm_is_available()))
> + return -EPROBE_DEFER;
> +
> + if (!fw_name || !(*fw_name))
> + return -EINVAL;
The parameter check should come before the qcom_scm_is_available()
No matter how many times you -EPROBE_DEFER -EINVAL is still -EINVAL.
> +
> + *mem_phys = 0;
> + *mem_size = 0;
I don't think you need this, you don't appear to use these variables
before you assign them below.
> +
> + *mem_phys = rmem->base;
> + *mem_size = rmem->size;
> +
> +int auth_reset_fw(u32 pas_id)
> +{
> + return qcom_scm_pas_auth_and_reset(pas_id);
> +}
> +
> +void unload_fw(u32 pas_id)
> +{
> + qcom_scm_pas_shutdown(pas_id);
> +}
> +
Do these wrapper functions add anything ? Some kind of validity check on
the pas_id otherwise I'm not sure these are justified.
> +int set_hw_state(bool resume)
> +{
> + return qcom_scm_set_remote_state(resume, 0);
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
> new file mode 100644
> index 0000000..7d410a8
> --- /dev/null
---
bod
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-19 11:40 ` Bryan O'Donoghue
@ 2023-12-19 13:26 ` Dmitry Baryshkov
2023-12-20 8:01 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-19 13:26 UTC (permalink / raw)
To: Bryan O'Donoghue
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
linux-arm-msm, quic_abhinavk
On Tue, 19 Dec 2023 at 13:40, Bryan O'Donoghue
<bryan.odonoghue@linaro.org> wrote:
>
> On 18/12/2023 11:31, Dikshita Agarwal wrote:
> > Re-organize the video driver code by introducing a new folder
> > 'vcodec' and placing 'venus' driver code inside that.
> >
> > Introduce common helpers for trustzone based firmware
> > load/unload etc. which are placed in common folder
> > i.e. 'vcodec'.
> > Use these helpers in 'venus' driver. These helpers will be
> > used by 'iris' driver as well which is introduced later
> > in this patch series.
> >
> > Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> > ---
>
> This is a very large patch, I think it needs to be broken up into
> smaller chunks.
>
> #1 Introduce common helper functions
> #2 Use common helper functions
This will make it harder to review. It's usually preferred to have a
single 'move' patch instead of two (add + remove). But I definitely
agree that the size of the patch is big. Somewhat it is related to the
fact that this doesn't only introduce helpers, but also reshuffles the
rest of the code.
>
> Its alot of code to try to eat in the one go.
>
> Could you consider making patches 1-3 a standalone series to reduce the
> amount of code to review here ?
This sounds like a good idea.
>
> * 77e7025529d7c - (HEAD -> linux-stable-master-23-12-18-iris-v2) media:
> iris: add power management for encoder (21 hours ago)
>
> * ceb6a6f023fd3 - (tag: v6.7-rc6, linux-stable/master) Linux 6.7-rc6 (2
> days ago)
>
> git diff ceb6a6f023fd3 | wc -l
>
> 21243
>
> Also I feel it wouild give more time for the changes to "digest" though
> upstream users and to find any unintended bugs.
>
> > +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> > + size_t *mem_size, u32 pas_id, bool use_tz)
> > +{
> > + const struct firmware *firmware = NULL;
> > + struct reserved_mem *rmem;
> > + struct device_node *node;
> > + void *mem_virt = NULL;
> > + ssize_t fw_size = 0;
> > + int ret;
> > +
> > + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> > + (use_tz && !qcom_scm_is_available()))
> > + return -EPROBE_DEFER;
> > +
> > + if (!fw_name || !(*fw_name))
> > + return -EINVAL;
>
> The parameter check should come before the qcom_scm_is_available()
>
> No matter how many times you -EPROBE_DEFER -EINVAL is still -EINVAL.
>
> > +
> > + *mem_phys = 0;
> > + *mem_size = 0;
>
> I don't think you need this, you don't appear to use these variables
> before you assign them below.
>
>
> > +
> > + *mem_phys = rmem->base;
> > + *mem_size = rmem->size;
>
> > +
> > +int auth_reset_fw(u32 pas_id)
> > +{
> > + return qcom_scm_pas_auth_and_reset(pas_id);
> > +}
> > +
> > +void unload_fw(u32 pas_id)
> > +{
> > + qcom_scm_pas_shutdown(pas_id);
> > +}
> > +
>
> Do these wrapper functions add anything ? Some kind of validity check on
> the pas_id otherwise I'm not sure these are justified.
>
> > +int set_hw_state(bool resume)
> > +{
> > + return qcom_scm_set_remote_state(resume, 0);
> > +}
> > diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
> > new file mode 100644
> > index 0000000..7d410a8
> > --- /dev/null
>
> ---
> bod
>
>
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling
2023-12-19 13:26 ` Dmitry Baryshkov
@ 2023-12-20 8:01 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:01 UTC (permalink / raw)
To: Dmitry Baryshkov, Bryan O'Donoghue
Cc: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, linux-arm-msm,
quic_abhinavk
On 12/19/2023 6:56 PM, Dmitry Baryshkov wrote:
> On Tue, 19 Dec 2023 at 13:40, Bryan O'Donoghue
> <bryan.odonoghue@linaro.org> wrote:
>>
>> On 18/12/2023 11:31, Dikshita Agarwal wrote:
>>> Re-organize the video driver code by introducing a new folder
>>> 'vcodec' and placing 'venus' driver code inside that.
>>>
>>> Introduce common helpers for trustzone based firmware
>>> load/unload etc. which are placed in common folder
>>> i.e. 'vcodec'.
>>> Use these helpers in 'venus' driver. These helpers will be
>>> used by 'iris' driver as well which is introduced later
>>> in this patch series.
>>>
>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>> ---
>>
>> This is a very large patch, I think it needs to be broken up into
>> smaller chunks.
>>
>> #1 Introduce common helper functions
>> #2 Use common helper functions
>
> This will make it harder to review. It's usually preferred to have a
> single 'move' patch instead of two (add + remove). But I definitely
> agree that the size of the patch is big. Somewhat it is related to the
> fact that this doesn't only introduce helpers, but also reshuffles the
> rest of the code.
>
Thanking along the same lines as Dmitry's, we wanted to show the usage of
these common helpers in iris driver as well, hence we added these patches
as part of this series.
I can send these patches 1-3 as separate series and mark the dependency of
iris series to that if you would prefer that way.
Also, as mentioned in comments in previous patches, to make the common
helper code generic, moving just the code from venus to vcodec was not
sufficient and more changes were needed in calling functions of venus as well.
>>
>> Its alot of code to try to eat in the one go.
>>
>> Could you consider making patches 1-3 a standalone series to reduce the
>> amount of code to review here ?
>
> This sounds like a good idea.
> >>
>> * 77e7025529d7c - (HEAD -> linux-stable-master-23-12-18-iris-v2) media:
>> iris: add power management for encoder (21 hours ago)
>>
>> * ceb6a6f023fd3 - (tag: v6.7-rc6, linux-stable/master) Linux 6.7-rc6 (2
>> days ago)
>>
>> git diff ceb6a6f023fd3 | wc -l
>>
>> 21243
>>
>> Also I feel it wouild give more time for the changes to "digest" though
>> upstream users and to find any unintended bugs.
>>
>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>> + size_t *mem_size, u32 pas_id, bool use_tz)
>>> +{
>>> + const struct firmware *firmware = NULL;
>>> + struct reserved_mem *rmem;
>>> + struct device_node *node;
>>> + void *mem_virt = NULL;
>>> + ssize_t fw_size = 0;
>>> + int ret;
>>> +
>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>>> + (use_tz && !qcom_scm_is_available()))
>>> + return -EPROBE_DEFER;
>>> +
>>> + if (!fw_name || !(*fw_name))
>>> + return -EINVAL;
>>
>> The parameter check should come before the qcom_scm_is_available()
>>
>> No matter how many times you -EPROBE_DEFER -EINVAL is still -EINVAL.
>>
Sure, will check and do the needful.
>>> +
>>> + *mem_phys = 0;
>>> + *mem_size = 0;
>>
>> I don't think you need this, you don't appear to use these variables
>> before you assign them below.
>>
That's true, will fix.
>>
>>> +
>>> + *mem_phys = rmem->base;
>>> + *mem_size = rmem->size;
>>
>>> +
>>> +int auth_reset_fw(u32 pas_id)
>>> +{
>>> + return qcom_scm_pas_auth_and_reset(pas_id);
>>> +}
>>> +
>>> +void unload_fw(u32 pas_id)
>>> +{
>>> + qcom_scm_pas_shutdown(pas_id);
>>> +}
>>> +
>>
>> Do these wrapper functions add anything ? Some kind of validity check on
>> the pas_id otherwise I'm not sure these are justified.
>>
>>> +int set_hw_state(bool resume)
>>> +{
>>> + return qcom_scm_set_remote_state(resume, 0);
>>> +}
Will check more on this and do required changes.
>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
>>> new file mode 100644
>>> index 0000000..7d410a8
>>> --- /dev/null
>>
>> ---
>> bod
>>
>>
>
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 02/34] media: introduce common helpers for queues handling
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
2023-12-18 11:31 ` [PATCH v2 01/34] media: introduce common helpers for video firmware handling Dikshita Agarwal
@ 2023-12-18 11:31 ` Dikshita Agarwal
2023-12-18 18:29 ` Dmitry Baryshkov
2023-12-18 11:31 ` [PATCH v2 03/34] media: introduce common helpers for buffer size calculation Dikshita Agarwal
` (38 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:31 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Introduce common helpers for shared queues init, deinit
and read/write queue operations and use them in venus driver.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++++++++++
drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++++++
drivers/media/platform/qcom/vcodec/venus/Makefile | 3 +-
.../media/platform/qcom/vcodec/venus/hfi_venus.c | 500 ++++-----------------
4 files changed, 465 insertions(+), 425 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.c b/drivers/media/platform/qcom/vcodec/hfi_queue.c
new file mode 100644
index 0000000..e3e470b
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/hfi_queue.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "hfi_queue.h"
+
+int write_queue(struct iface_q_info *qinfo, void *packet,
+ u32 packet_size, u32 *rx_req)
+{
+ u32 empty_space, read_idx, write_idx, new_write_idx;
+ struct hfi_queue_header *queue;
+ u32 *write_ptr, residue;
+
+ queue = qinfo->qhdr;
+ if (!queue)
+ return -EINVAL;
+
+ if (!packet_size || packet_size > qinfo->q_array.size)
+ return -EINVAL;
+
+ read_idx = queue->qhdr_read_idx * sizeof(u32);
+ write_idx = queue->qhdr_write_idx * sizeof(u32);
+
+ empty_space = (write_idx >= read_idx) ?
+ (qinfo->q_array.size - (write_idx - read_idx)) :
+ (read_idx - write_idx);
+ if (empty_space <= packet_size) {
+ queue->qhdr_tx_req = 1;
+ return -ENOSPC;
+ }
+
+ queue->qhdr_tx_req = 0;
+
+ new_write_idx = write_idx + packet_size;
+ write_ptr = (u32 *)((u8 *)qinfo->q_array.kernel_vaddr + write_idx);
+
+ if (write_ptr < (u32 *)qinfo->q_array.kernel_vaddr ||
+ write_ptr > (u32 *)(qinfo->q_array.kernel_vaddr +
+ qinfo->q_array.size))
+ return -EINVAL;
+
+ if (new_write_idx < qinfo->q_array.size) {
+ memcpy(write_ptr, packet, packet_size);
+ } else {
+ residue = new_write_idx - qinfo->q_array.size;
+ memcpy(write_ptr, packet, (packet_size - residue));
+ memcpy(qinfo->q_array.kernel_vaddr,
+ packet + (packet_size - residue), residue);
+ new_write_idx = residue;
+ }
+
+ /* Make sure packet is written before updating the write index */
+ mb();
+ queue->qhdr_write_idx = new_write_idx / sizeof(u32);
+ *rx_req = queue->qhdr_rx_req ? 1 : 0;
+
+ /* Make sure write index is updated before an interrupt is raised */
+ mb();
+
+ return 0;
+}
+
+int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req)
+{
+ u32 read_idx, write_idx, new_read_idx;
+ struct hfi_queue_header *queue;
+ u32 receive_request = 0;
+ u32 packet_size, residue;
+ u32 *read_ptr;
+ int ret = 0;
+
+ /* Make sure data is valid before reading it */
+ mb();
+ queue = qinfo->qhdr;
+ if (!queue)
+ return -EINVAL;
+
+ if (queue->qhdr_type & IFACEQ_MSGQ_ID)
+ receive_request = 1;
+
+ read_idx = queue->qhdr_read_idx * sizeof(u32);
+ write_idx = queue->qhdr_write_idx * sizeof(u32);
+
+ if (read_idx == write_idx) {
+ queue->qhdr_rx_req = receive_request;
+ /* Ensure qhdr is updated in main memory */
+ mb();
+ return -ENODATA;
+ }
+
+ read_ptr = (u32 *)(qinfo->q_array.kernel_vaddr + read_idx);
+ if (read_ptr < (u32 *)qinfo->q_array.kernel_vaddr ||
+ read_ptr > (u32 *)(qinfo->q_array.kernel_vaddr +
+ qinfo->q_array.size - sizeof(*read_ptr)))
+ return -ENODATA;
+
+ packet_size = *read_ptr;
+ if (!packet_size)
+ return -EINVAL;
+
+ new_read_idx = read_idx + packet_size;
+ if (packet_size <= IFACEQ_CORE_PKT_SIZE &&
+ read_idx <= qinfo->q_array.size) {
+ if (new_read_idx < qinfo->q_array.size) {
+ memcpy(packet, read_ptr, packet_size);
+ } else {
+ residue = new_read_idx - qinfo->q_array.size;
+ memcpy(packet, read_ptr, (packet_size - residue));
+ memcpy((packet + (packet_size - residue)),
+ qinfo->q_array.kernel_vaddr, residue);
+ new_read_idx = residue;
+ }
+ } else {
+ new_read_idx = write_idx;
+ ret = -EBADMSG;
+ }
+
+ queue->qhdr_rx_req = receive_request;
+
+ queue->qhdr_read_idx = new_read_idx / sizeof(u32);
+
+ *tx_req = queue->qhdr_tx_req ? 1 : 0;
+ /* Ensure qhdr is updated in main memory */
+ mb();
+
+ return ret;
+}
+
+static void __set_queue_hdr_defaults(struct hfi_queue_header *q_hdr)
+{
+ q_hdr->qhdr_status = 0x1;
+ q_hdr->qhdr_type = IFACEQ_DFLT_QHDR;
+ q_hdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4;
+ q_hdr->qhdr_pkt_size = 0;
+ q_hdr->qhdr_rx_wm = 0x1;
+ q_hdr->qhdr_tx_wm = 0x1;
+ q_hdr->qhdr_rx_req = 0x1;
+ q_hdr->qhdr_tx_req = 0x0;
+ q_hdr->qhdr_rx_irq_status = 0x0;
+ q_hdr->qhdr_tx_irq_status = 0x0;
+ q_hdr->qhdr_read_idx = 0x0;
+ q_hdr->qhdr_write_idx = 0x0;
+}
+
+static void hfi_set_queue_header(u32 queue_id, struct iface_q_info *iface_q)
+{
+ __set_queue_hdr_defaults(iface_q->qhdr);
+ iface_q->qhdr->qhdr_start_addr = iface_q->q_array.device_addr;
+ iface_q->qhdr->qhdr_type |= queue_id;
+
+ /*
+ * Set receive request to zero on debug queue as there is no
+ * need of interrupt from video hardware for debug messages
+ */
+ if (queue_id == IFACEQ_DBGQ_ID)
+ iface_q->qhdr->qhdr_rx_req = 0;
+}
+
+static void __queue_init(struct mem_desc *q_table, u32 queue_id, struct iface_q_info *iface_q)
+{
+ unsigned int offset = 0;
+
+ offset = q_table->size + (queue_id * IFACEQ_QUEUE_SIZE);
+ iface_q->q_array.device_addr = q_table->device_addr + offset;
+ iface_q->q_array.kernel_vaddr =
+ (void *)((char *)q_table->kernel_vaddr + offset);
+ iface_q->q_array.size = IFACEQ_QUEUE_SIZE;
+ iface_q->qhdr =
+ IFACEQ_GET_QHDR_START_ADDR(q_table->kernel_vaddr, queue_id);
+
+ memset(iface_q->qhdr, 0, sizeof(struct hfi_queue_header));
+
+ hfi_set_queue_header(queue_id, iface_q);
+}
+
+int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
+ struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
+ struct iface_q_info *dbg_q, void *daddr)
+{
+ struct hfi_queue_table_header *q_tbl_hdr;
+
+ if (q_table->kernel_vaddr) {
+ hfi_set_queue_header(IFACEQ_CMDQ_ID, cmd_q);
+ hfi_set_queue_header(IFACEQ_MSGQ_ID, msg_q);
+ hfi_set_queue_header(IFACEQ_DBGQ_ID, dbg_q);
+ return 0;
+ }
+
+ q_table->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_QUEUE_SIZE,
+ &q_table->device_addr,
+ GFP_KERNEL, DMA_ATTR_WRITE_COMBINE);
+ if (!q_table->kernel_vaddr) {
+ dev_err(dev, "%s: queues alloc and map failed\n", __func__);
+ return -ENOMEM;
+ }
+
+ sfr->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_SFR_SIZE,
+ &sfr->device_addr,
+ GFP_KERNEL, DMA_ATTR_WRITE_COMBINE);
+ if (!sfr->kernel_vaddr) {
+ dev_err(dev, "%s: sfr alloc and map failed\n", __func__);
+ return -ENOMEM;
+ }
+
+ q_table->size = IFACEQ_TABLE_SIZE;
+
+ __queue_init(q_table, IFACEQ_CMDQ_ID, cmd_q);
+ __queue_init(q_table, IFACEQ_MSGQ_ID, msg_q);
+ __queue_init(q_table, IFACEQ_DBGQ_ID, dbg_q);
+
+ q_tbl_hdr = (struct hfi_queue_table_header *)q_table->kernel_vaddr;
+ q_tbl_hdr->qtbl_version = 0;
+ q_tbl_hdr->device_addr = (void *)daddr;
+ strscpy(q_tbl_hdr->name, "hfi-queues", sizeof(q_tbl_hdr->name));
+ q_tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE;
+ q_tbl_hdr->qtbl_qhdr0_offset = sizeof(*q_tbl_hdr);
+ q_tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header);
+ q_tbl_hdr->qtbl_num_q = IFACEQ_NUMQ;
+ q_tbl_hdr->qtbl_num_active_q = IFACEQ_NUMQ;
+
+ sfr->size = ALIGNED_SFR_SIZE;
+ /* Write sfr size in first word to be used by firmware */
+ *((u32 *)sfr->kernel_vaddr) = sfr->size;
+
+ return 0;
+}
+
+static void __queue_deinit(struct iface_q_info *iface_q)
+{
+ iface_q->qhdr = NULL;
+ iface_q->q_array.kernel_vaddr = NULL;
+ iface_q->q_array.device_addr = 0;
+}
+
+void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
+ struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
+ struct iface_q_info *dbg_q)
+{
+ if (!q_table->kernel_vaddr)
+ return;
+
+ dma_free_attrs(dev, q_table->size, q_table->kernel_vaddr,
+ q_table->device_addr, q_table->attrs);
+
+ dma_free_attrs(dev, sfr->size, sfr->kernel_vaddr,
+ sfr->device_addr, sfr->attrs);
+
+ __queue_deinit(cmd_q);
+ __queue_deinit(msg_q);
+ __queue_deinit(dbg_q);
+
+ q_table->kernel_vaddr = NULL;
+ q_table->device_addr = 0;
+
+ sfr->kernel_vaddr = NULL;
+ sfr->device_addr = 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.h b/drivers/media/platform/qcom/vcodec/hfi_queue.h
new file mode 100644
index 0000000..8cc9491
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/hfi_queue.h
@@ -0,0 +1,129 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _HFI_QUEUE_H_
+#define _HFI_QUEUE_H_
+
+#include <linux/types.h>
+#include <media/v4l2-device.h>
+
+/*
+ * Shared queues are used for communication between driver and firmware.
+ * There are 3 types of queues:
+ * Command queue - driver to write any command to firmware.
+ * Message queue - firmware to send any response to driver.
+ * Debug queue - firmware to write debug message.
+ */
+
+/* Host-firmware shared queue ids */
+enum iris_iface_queue {
+ IFACEQ_CMDQ_ID = 0,
+ IFACEQ_MSGQ_ID,
+ IFACEQ_DBGQ_ID,
+ IFACEQ_NUMQ, /* not an index */
+};
+
+#define IFACEQ_MAX_BUF_COUNT 50
+/*
+ * Max session supported are 16.
+ * this value is used to calculate the size of
+ * individual shared queue.
+ */
+#define IFACE_MAX_PARALLEL_CLNTS 16
+#define IFACEQ_DFLT_QHDR 0x01010000
+
+#define IFACEQ_MAX_PKT_SIZE 1024
+#define IFACEQ_CORE_PKT_SIZE (1024 * 4)
+
+#define IFACEQ_TABLE_SIZE (sizeof(struct hfi_queue_table_header) + \
+ sizeof(struct hfi_queue_header) * IFACEQ_NUMQ)
+
+#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \
+ IFACEQ_MAX_BUF_COUNT * IFACE_MAX_PARALLEL_CLNTS)
+
+#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \
+ ((void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \
+ ((i) * sizeof(struct hfi_queue_header))))
+
+#define SFR_SIZE SZ_4K
+#define QUEUE_SIZE (IFACEQ_TABLE_SIZE + \
+ (IFACEQ_QUEUE_SIZE * IFACEQ_NUMQ))
+
+#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K)
+#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K)
+#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE, SZ_1M)
+
+struct mem_desc {
+ dma_addr_t device_addr;
+ void *kernel_vaddr;
+ u32 size;
+ unsigned long attrs;
+};
+
+struct hfi_queue_table_header {
+ u32 qtbl_version;
+ u32 qtbl_size;
+ u32 qtbl_qhdr0_offset;
+ u32 qtbl_qhdr_size;
+ u32 qtbl_num_q;
+ u32 qtbl_num_active_q;
+ void *device_addr;
+ char name[256];
+};
+
+/**
+ * struct hfi_queue_header - queue header definition
+ *
+ * @qhdr_status: bits (7:0) 0x1 - active 0x0 - inactive
+ * @qhdr_start_addr: queue start addr.
+ * @qhdr_type: queue type.
+ * @qhdr_q_size: size in number of HFI_QUEUE_PACKET_TYPE packets it can hold.
+ * @qhdr_pkt_size: size of the Queue packet entries, in bytes.
+ * @qhdr_rx_wm: the receiver watermark in number of queue packets.
+ * @qhdr_tx_wm: the transmitter watermark in number of queue packets.
+ * @qhdr_rx_req: set to request an interrupt from transmitter if qhdr_tx_wm is reached.
+ * @qhdr_tx_req: set to request an interrupt from receiver if qhdr_rx_wm is reached.
+ * @qhdr_rx_irq_status: status of receiver irq.
+ * @qhdr_tx_irq_status: status of transmitter irq.
+ * @qhdr_read_idx: read index, which starts with a value of 0 and wraps around after
+ * (QHDR_Q_SIZE - 1).
+ * @qhdr_write_idx: write index, which starts with a value of 0 and wraps around after
+ * (QHDR_Q_SIZE - 1).
+ */
+
+struct hfi_queue_header {
+ u32 qhdr_status;
+ u32 qhdr_start_addr;
+ u32 qhdr_type;
+ u32 qhdr_q_size;
+ u32 qhdr_pkt_size;
+ u32 qhdr_pkt_drop_cnt;
+ u32 qhdr_rx_wm;
+ u32 qhdr_tx_wm;
+ u32 qhdr_rx_req;
+ u32 qhdr_tx_req;
+ u32 qhdr_rx_irq_status;
+ u32 qhdr_tx_irq_status;
+ u32 qhdr_read_idx;
+ u32 qhdr_write_idx;
+};
+
+struct iface_q_info {
+ struct hfi_queue_header *qhdr;
+ struct mem_desc q_array;
+};
+
+int write_queue(struct iface_q_info *qinfo, void *packet,
+ u32 packet_size, u32 *rx_req);
+int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req);
+int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
+ struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
+ struct iface_q_info *dbg_q, void *daddr);
+
+void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
+ struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
+ struct iface_q_info *dbg_q);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
index f6f3a88..1941ef4 100644
--- a/drivers/media/platform/qcom/vcodec/venus/Makefile
+++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
@@ -1,7 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
# Makefile for Qualcomm Venus driver
-venus-core-objs += ../firmware.o
+venus-core-objs += ../firmware.o \
+ ../hfi_queue.o
venus-core-objs += core.o helpers.o firmware_no_tz.o \
hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
index 5a68db9..41e3f27 100644
--- a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
@@ -16,6 +16,7 @@
#include "../firmware.h"
#include "hfi_cmds.h"
#include "hfi_msgs.h"
+#include "../hfi_queue.h"
#include "hfi_venus.h"
#include "hfi_venus_io.h"
#include "firmware_no_tz.h"
@@ -34,79 +35,13 @@
#define IFACEQ_CMD_IDX 0
#define IFACEQ_MSG_IDX 1
#define IFACEQ_DBG_IDX 2
-#define IFACEQ_MAX_BUF_COUNT 50
-#define IFACEQ_MAX_PARALLEL_CLNTS 16
-#define IFACEQ_DFLT_QHDR 0x01010000
#define POLL_INTERVAL_US 50
-#define IFACEQ_MAX_PKT_SIZE 1024
-#define IFACEQ_MED_PKT_SIZE 768
-#define IFACEQ_MIN_PKT_SIZE 8
#define IFACEQ_VAR_SMALL_PKT_SIZE 100
#define IFACEQ_VAR_LARGE_PKT_SIZE 512
#define IFACEQ_VAR_HUGE_PKT_SIZE (1024 * 12)
-struct hfi_queue_table_header {
- u32 version;
- u32 size;
- u32 qhdr0_offset;
- u32 qhdr_size;
- u32 num_q;
- u32 num_active_q;
-};
-
-struct hfi_queue_header {
- u32 status;
- u32 start_addr;
- u32 type;
- u32 q_size;
- u32 pkt_size;
- u32 pkt_drop_cnt;
- u32 rx_wm;
- u32 tx_wm;
- u32 rx_req;
- u32 tx_req;
- u32 rx_irq_status;
- u32 tx_irq_status;
- u32 read_idx;
- u32 write_idx;
-};
-
-#define IFACEQ_TABLE_SIZE \
- (sizeof(struct hfi_queue_table_header) + \
- sizeof(struct hfi_queue_header) * IFACEQ_NUM)
-
-#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \
- IFACEQ_MAX_BUF_COUNT * IFACEQ_MAX_PARALLEL_CLNTS)
-
-#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \
- (void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \
- ((i) * sizeof(struct hfi_queue_header)))
-
-#define QDSS_SIZE SZ_4K
-#define SFR_SIZE SZ_4K
-#define QUEUE_SIZE \
- (IFACEQ_TABLE_SIZE + (IFACEQ_QUEUE_SIZE * IFACEQ_NUM))
-
-#define ALIGNED_QDSS_SIZE ALIGN(QDSS_SIZE, SZ_4K)
-#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K)
-#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K)
-#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE + \
- ALIGNED_QDSS_SIZE, SZ_1M)
-
-struct mem_desc {
- dma_addr_t da; /* device address */
- void *kva; /* kernel virtual address */
- u32 size;
- unsigned long attrs;
-};
-
-struct iface_queue {
- struct hfi_queue_header *qhdr;
- struct mem_desc qmem;
-};
-
enum venus_state {
VENUS_STATE_DEINIT = 1,
VENUS_STATE_INIT,
@@ -125,12 +60,11 @@ struct venus_hfi_device {
struct completion release_resource;
struct mem_desc ifaceq_table;
struct mem_desc sfr;
- struct iface_queue queues[IFACEQ_NUM];
+ struct iface_q_info queues[IFACEQ_NUM];
u8 pkt_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
u8 dbg_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
};
-static bool venus_pkt_debug;
int venus_fw_debug = HFI_DEBUG_MSG_ERROR | HFI_DEBUG_MSG_FATAL;
static bool venus_fw_low_power_mode = true;
static int venus_hw_rsp_timeout = 1000;
@@ -149,212 +83,6 @@ static bool venus_is_valid_state(struct venus_hfi_device *hdev)
return hdev->state != VENUS_STATE_DEINIT;
}
-static void venus_dump_packet(struct venus_hfi_device *hdev, const void *packet)
-{
- size_t pkt_size = *(u32 *)packet;
-
- if (!venus_pkt_debug)
- return;
-
- print_hex_dump(KERN_DEBUG, "", DUMP_PREFIX_OFFSET, 16, 1, packet,
- pkt_size, true);
-}
-
-static int venus_write_queue(struct venus_hfi_device *hdev,
- struct iface_queue *queue,
- void *packet, u32 *rx_req)
-{
- struct hfi_queue_header *qhdr;
- u32 dwords, new_wr_idx;
- u32 empty_space, rd_idx, wr_idx, qsize;
- u32 *wr_ptr;
-
- if (!queue->qmem.kva)
- return -EINVAL;
-
- qhdr = queue->qhdr;
- if (!qhdr)
- return -EINVAL;
-
- venus_dump_packet(hdev, packet);
-
- dwords = (*(u32 *)packet) >> 2;
- if (!dwords)
- return -EINVAL;
-
- rd_idx = qhdr->read_idx;
- wr_idx = qhdr->write_idx;
- qsize = qhdr->q_size;
- /* ensure rd/wr indices's are read from memory */
- rmb();
-
- if (wr_idx >= rd_idx)
- empty_space = qsize - (wr_idx - rd_idx);
- else
- empty_space = rd_idx - wr_idx;
-
- if (empty_space <= dwords) {
- qhdr->tx_req = 1;
- /* ensure tx_req is updated in memory */
- wmb();
- return -ENOSPC;
- }
-
- qhdr->tx_req = 0;
- /* ensure tx_req is updated in memory */
- wmb();
-
- new_wr_idx = wr_idx + dwords;
- wr_ptr = (u32 *)(queue->qmem.kva + (wr_idx << 2));
-
- if (wr_ptr < (u32 *)queue->qmem.kva ||
- wr_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*wr_ptr)))
- return -EINVAL;
-
- if (new_wr_idx < qsize) {
- memcpy(wr_ptr, packet, dwords << 2);
- } else {
- size_t len;
-
- new_wr_idx -= qsize;
- len = (dwords - new_wr_idx) << 2;
- memcpy(wr_ptr, packet, len);
- memcpy(queue->qmem.kva, packet + len, new_wr_idx << 2);
- }
-
- /* make sure packet is written before updating the write index */
- wmb();
-
- qhdr->write_idx = new_wr_idx;
- *rx_req = qhdr->rx_req ? 1 : 0;
-
- /* make sure write index is updated before an interrupt is raised */
- mb();
-
- return 0;
-}
-
-static int venus_read_queue(struct venus_hfi_device *hdev,
- struct iface_queue *queue, void *pkt, u32 *tx_req)
-{
- struct hfi_queue_header *qhdr;
- u32 dwords, new_rd_idx;
- u32 rd_idx, wr_idx, type, qsize;
- u32 *rd_ptr;
- u32 recv_request = 0;
- int ret = 0;
-
- if (!queue->qmem.kva)
- return -EINVAL;
-
- qhdr = queue->qhdr;
- if (!qhdr)
- return -EINVAL;
-
- type = qhdr->type;
- rd_idx = qhdr->read_idx;
- wr_idx = qhdr->write_idx;
- qsize = qhdr->q_size;
-
- /* make sure data is valid before using it */
- rmb();
-
- /*
- * Do not set receive request for debug queue, if set, Venus generates
- * interrupt for debug messages even when there is no response message
- * available. In general debug queue will not become full as it is being
- * emptied out for every interrupt from Venus. Venus will anyway
- * generates interrupt if it is full.
- */
- if (type & HFI_CTRL_TO_HOST_MSG_Q)
- recv_request = 1;
-
- if (rd_idx == wr_idx) {
- qhdr->rx_req = recv_request;
- *tx_req = 0;
- /* update rx_req field in memory */
- wmb();
- return -ENODATA;
- }
-
- rd_ptr = (u32 *)(queue->qmem.kva + (rd_idx << 2));
-
- if (rd_ptr < (u32 *)queue->qmem.kva ||
- rd_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*rd_ptr)))
- return -EINVAL;
-
- dwords = *rd_ptr >> 2;
- if (!dwords)
- return -EINVAL;
-
- new_rd_idx = rd_idx + dwords;
- if (((dwords << 2) <= IFACEQ_VAR_HUGE_PKT_SIZE) && rd_idx <= qsize) {
- if (new_rd_idx < qsize) {
- memcpy(pkt, rd_ptr, dwords << 2);
- } else {
- size_t len;
-
- new_rd_idx -= qsize;
- len = (dwords - new_rd_idx) << 2;
- memcpy(pkt, rd_ptr, len);
- memcpy(pkt + len, queue->qmem.kva, new_rd_idx << 2);
- }
- } else {
- /* bad packet received, dropping */
- new_rd_idx = qhdr->write_idx;
- ret = -EBADMSG;
- }
-
- /* ensure the packet is read before updating read index */
- rmb();
-
- qhdr->read_idx = new_rd_idx;
- /* ensure updating read index */
- wmb();
-
- rd_idx = qhdr->read_idx;
- wr_idx = qhdr->write_idx;
- /* ensure rd/wr indices are read from memory */
- rmb();
-
- if (rd_idx != wr_idx)
- qhdr->rx_req = 0;
- else
- qhdr->rx_req = recv_request;
-
- *tx_req = qhdr->tx_req ? 1 : 0;
-
- /* ensure rx_req is stored to memory and tx_req is loaded from memory */
- mb();
-
- venus_dump_packet(hdev, pkt);
-
- return ret;
-}
-
-static int venus_alloc(struct venus_hfi_device *hdev, struct mem_desc *desc,
- u32 size)
-{
- struct device *dev = hdev->core->dev;
-
- desc->attrs = DMA_ATTR_WRITE_COMBINE;
- desc->size = ALIGN(size, SZ_4K);
-
- desc->kva = dma_alloc_attrs(dev, desc->size, &desc->da, GFP_KERNEL,
- desc->attrs);
- if (!desc->kva)
- return -ENOMEM;
-
- return 0;
-}
-
-static void venus_free(struct venus_hfi_device *hdev, struct mem_desc *mem)
-{
- struct device *dev = hdev->core->dev;
-
- dma_free_attrs(dev, mem->size, mem->kva, mem->da, mem->attrs);
-}
-
static void venus_set_registers(struct venus_hfi_device *hdev)
{
const struct venus_resources *res = hdev->core->res;
@@ -380,11 +108,11 @@ static void venus_soft_int(struct venus_hfi_device *hdev)
}
static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
- void *pkt, bool sync)
+ void *pkt, u32 pkt_size, bool sync)
{
struct device *dev = hdev->core->dev;
struct hfi_pkt_hdr *cmd_packet;
- struct iface_queue *queue;
+ struct iface_q_info *queue;
u32 rx_req;
int ret;
@@ -396,7 +124,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
queue = &hdev->queues[IFACEQ_CMD_IDX];
- ret = venus_write_queue(hdev, queue, pkt, &rx_req);
+ ret = write_queue(queue, pkt, pkt_size, &rx_req);
if (ret) {
dev_err(dev, "write to iface cmd queue failed (%d)\n", ret);
return ret;
@@ -408,7 +136,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
* commands
*/
queue = &hdev->queues[IFACEQ_MSG_IDX];
- queue->qhdr->rx_req = 1;
+ queue->qhdr->qhdr_rx_req = 1;
/* ensure rx_req is updated in memory */
wmb();
}
@@ -419,12 +147,12 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
return 0;
}
-static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, bool sync)
+static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, u32 pkt_size, bool sync)
{
int ret;
mutex_lock(&hdev->lock);
- ret = venus_iface_cmdq_write_nolock(hdev, pkt, sync);
+ ret = venus_iface_cmdq_write_nolock(hdev, pkt, pkt_size, sync);
mutex_unlock(&hdev->lock);
return ret;
@@ -447,7 +175,7 @@ static int venus_hfi_core_set_resource(struct venus_core *core, u32 id,
if (ret)
return ret;
- ret = venus_iface_cmdq_write(hdev, pkt, false);
+ ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
if (ret)
return ret;
@@ -531,12 +259,12 @@ static int venus_run(struct venus_hfi_device *hdev)
*/
venus_set_registers(hdev);
- writel(hdev->ifaceq_table.da, cpu_cs_base + UC_REGION_ADDR);
+ writel(hdev->ifaceq_table.device_addr, cpu_cs_base + UC_REGION_ADDR);
writel(SHARED_QSIZE, cpu_cs_base + UC_REGION_SIZE);
- writel(hdev->ifaceq_table.da, cpu_cs_base + CPU_CS_SCIACMDARG2);
+ writel(hdev->ifaceq_table.device_addr, cpu_cs_base + CPU_CS_SCIACMDARG2);
writel(0x01, cpu_cs_base + CPU_CS_SCIACMDARG1);
- if (hdev->sfr.da)
- writel(hdev->sfr.da, cpu_cs_base + SFR_ADDR);
+ if (hdev->sfr.device_addr)
+ writel(hdev->sfr.device_addr, cpu_cs_base + SFR_ADDR);
ret = venus_boot_core(hdev);
if (ret) {
@@ -690,7 +418,7 @@ static int venus_power_on(struct venus_hfi_device *hdev)
static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev,
void *pkt)
{
- struct iface_queue *queue;
+ struct iface_q_info *queue;
u32 tx_req;
int ret;
@@ -699,7 +427,7 @@ static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev,
queue = &hdev->queues[IFACEQ_MSG_IDX];
- ret = venus_read_queue(hdev, queue, pkt, &tx_req);
+ ret = read_queue(queue, pkt, &tx_req);
if (ret)
return ret;
@@ -723,7 +451,7 @@ static int venus_iface_msgq_read(struct venus_hfi_device *hdev, void *pkt)
static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev,
void *pkt)
{
- struct iface_queue *queue;
+ struct iface_q_info *queue;
u32 tx_req;
int ret;
@@ -733,7 +461,7 @@ static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev,
queue = &hdev->queues[IFACEQ_DBG_IDX];
- ret = venus_read_queue(hdev, queue, pkt, &tx_req);
+ ret = read_queue(queue, pkt, &tx_req);
if (ret)
return ret;
@@ -759,100 +487,18 @@ static int venus_iface_dbgq_read(struct venus_hfi_device *hdev, void *pkt)
static void venus_set_qhdr_defaults(struct hfi_queue_header *qhdr)
{
- qhdr->status = 1;
- qhdr->type = IFACEQ_DFLT_QHDR;
- qhdr->q_size = IFACEQ_QUEUE_SIZE / 4;
- qhdr->pkt_size = 0;
- qhdr->rx_wm = 1;
- qhdr->tx_wm = 1;
- qhdr->rx_req = 1;
- qhdr->tx_req = 0;
- qhdr->rx_irq_status = 0;
- qhdr->tx_irq_status = 0;
- qhdr->read_idx = 0;
- qhdr->write_idx = 0;
-}
-
-static void venus_interface_queues_release(struct venus_hfi_device *hdev)
-{
- mutex_lock(&hdev->lock);
-
- venus_free(hdev, &hdev->ifaceq_table);
- venus_free(hdev, &hdev->sfr);
-
- memset(hdev->queues, 0, sizeof(hdev->queues));
- memset(&hdev->ifaceq_table, 0, sizeof(hdev->ifaceq_table));
- memset(&hdev->sfr, 0, sizeof(hdev->sfr));
-
- mutex_unlock(&hdev->lock);
-}
-
-static int venus_interface_queues_init(struct venus_hfi_device *hdev)
-{
- struct hfi_queue_table_header *tbl_hdr;
- struct iface_queue *queue;
- struct hfi_sfr *sfr;
- struct mem_desc desc = {0};
- unsigned int offset;
- unsigned int i;
- int ret;
-
- ret = venus_alloc(hdev, &desc, ALIGNED_QUEUE_SIZE);
- if (ret)
- return ret;
-
- hdev->ifaceq_table = desc;
- offset = IFACEQ_TABLE_SIZE;
-
- for (i = 0; i < IFACEQ_NUM; i++) {
- queue = &hdev->queues[i];
- queue->qmem.da = desc.da + offset;
- queue->qmem.kva = desc.kva + offset;
- queue->qmem.size = IFACEQ_QUEUE_SIZE;
- offset += queue->qmem.size;
- queue->qhdr =
- IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
-
- venus_set_qhdr_defaults(queue->qhdr);
-
- queue->qhdr->start_addr = queue->qmem.da;
-
- if (i == IFACEQ_CMD_IDX)
- queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
- else if (i == IFACEQ_MSG_IDX)
- queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
- else if (i == IFACEQ_DBG_IDX)
- queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
- }
-
- tbl_hdr = hdev->ifaceq_table.kva;
- tbl_hdr->version = 0;
- tbl_hdr->size = IFACEQ_TABLE_SIZE;
- tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
- tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
- tbl_hdr->num_q = IFACEQ_NUM;
- tbl_hdr->num_active_q = IFACEQ_NUM;
-
- /*
- * Set receive request to zero on debug queue as there is no
- * need of interrupt from video hardware for debug messages
- */
- queue = &hdev->queues[IFACEQ_DBG_IDX];
- queue->qhdr->rx_req = 0;
-
- ret = venus_alloc(hdev, &desc, ALIGNED_SFR_SIZE);
- if (ret) {
- hdev->sfr.da = 0;
- } else {
- hdev->sfr = desc;
- sfr = hdev->sfr.kva;
- sfr->buf_size = ALIGNED_SFR_SIZE;
- }
-
- /* ensure table and queue header structs are settled in memory */
- wmb();
-
- return 0;
+ qhdr->qhdr_status = 1;
+ qhdr->qhdr_type = IFACEQ_DFLT_QHDR;
+ qhdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4;
+ qhdr->qhdr_pkt_size = 0;
+ qhdr->qhdr_rx_wm = 1;
+ qhdr->qhdr_tx_wm = 1;
+ qhdr->qhdr_rx_req = 1;
+ qhdr->qhdr_tx_req = 0;
+ qhdr->qhdr_rx_irq_status = 0;
+ qhdr->qhdr_tx_irq_status = 0;
+ qhdr->qhdr_read_idx = 0;
+ qhdr->qhdr_write_idx = 0;
}
static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug)
@@ -864,7 +510,7 @@ static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug)
pkt_sys_debug_config(pkt, HFI_DEBUG_MODE_QUEUE, debug);
- return venus_iface_cmdq_write(hdev, pkt, false);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
}
static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode)
@@ -876,7 +522,7 @@ static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode)
pkt_sys_coverage_config(pkt, mode);
- return venus_iface_cmdq_write(hdev, pkt, false);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
}
static int venus_sys_set_idle_message(struct venus_hfi_device *hdev,
@@ -892,7 +538,7 @@ static int venus_sys_set_idle_message(struct venus_hfi_device *hdev,
pkt_sys_idle_indicator(pkt, enable);
- return venus_iface_cmdq_write(hdev, pkt, false);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
}
static int venus_sys_set_power_control(struct venus_hfi_device *hdev,
@@ -905,7 +551,7 @@ static int venus_sys_set_power_control(struct venus_hfi_device *hdev,
pkt_sys_power_control(pkt, enable);
- return venus_iface_cmdq_write(hdev, pkt, false);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
}
static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev)
@@ -919,7 +565,7 @@ static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev)
pkt_sys_ubwc_config(pkt, res->ubwc_conf);
- ret = venus_iface_cmdq_write(hdev, pkt, false);
+ ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
if (ret)
return ret;
@@ -938,7 +584,7 @@ static int venus_get_queue_size(struct venus_hfi_device *hdev,
if (!qhdr)
return -EINVAL;
- return abs(qhdr->read_idx - qhdr->write_idx);
+ return abs(qhdr->qhdr_read_idx - qhdr->qhdr_write_idx);
}
static int venus_sys_set_default_properties(struct venus_hfi_device *hdev)
@@ -980,7 +626,7 @@ static int venus_session_cmd(struct venus_inst *inst, u32 pkt_type, bool sync)
pkt_session_cmd(&pkt, pkt_type, inst);
- return venus_iface_cmdq_write(hdev, &pkt, sync);
+ return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, sync);
}
static void venus_flush_debug_queue(struct venus_hfi_device *hdev)
@@ -1010,7 +656,7 @@ static int venus_prepare_power_collapse(struct venus_hfi_device *hdev,
pkt_sys_pc_prep(&pkt);
- ret = venus_iface_cmdq_write(hdev, &pkt, false);
+ ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
if (ret)
return ret;
@@ -1047,7 +693,7 @@ static int venus_are_queues_empty(struct venus_hfi_device *hdev)
static void venus_sfr_print(struct venus_hfi_device *hdev)
{
struct device *dev = hdev->core->dev;
- struct hfi_sfr *sfr = hdev->sfr.kva;
+ struct hfi_sfr *sfr = hdev->sfr.kernel_vaddr;
void *p;
if (!sfr)
@@ -1163,13 +809,13 @@ static int venus_core_init(struct venus_core *core)
venus_set_state(hdev, VENUS_STATE_INIT);
- ret = venus_iface_cmdq_write(hdev, &pkt, false);
+ ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
if (ret)
return ret;
pkt_sys_image_version(&version_pkt);
- ret = venus_iface_cmdq_write(hdev, &version_pkt, false);
+ ret = venus_iface_cmdq_write(hdev, &version_pkt, version_pkt.hdr.size, false);
if (ret)
dev_warn(dev, "failed to send image version pkt to fw\n");
@@ -1198,7 +844,7 @@ static int venus_core_ping(struct venus_core *core, u32 cookie)
pkt_sys_ping(&pkt, cookie);
- return venus_iface_cmdq_write(hdev, &pkt, false);
+ return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
}
static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type)
@@ -1211,7 +857,7 @@ static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type)
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, &pkt, false);
+ return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
}
static int venus_session_init(struct venus_inst *inst, u32 session_type,
@@ -1229,7 +875,7 @@ static int venus_session_init(struct venus_inst *inst, u32 session_type,
if (ret)
goto err;
- ret = venus_iface_cmdq_write(hdev, &pkt, true);
+ ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
if (ret)
goto err;
@@ -1272,7 +918,7 @@ static int venus_session_flush(struct venus_inst *inst, u32 flush_mode)
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, &pkt, true);
+ return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
}
static int venus_session_start(struct venus_inst *inst)
@@ -1304,7 +950,7 @@ static int venus_session_etb(struct venus_inst *inst,
if (ret)
return ret;
- ret = venus_iface_cmdq_write(hdev, &pkt, false);
+ ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
} else if (session_type == VIDC_SESSION_TYPE_ENC) {
struct hfi_session_empty_buffer_uncompressed_plane0_pkt pkt;
@@ -1312,7 +958,7 @@ static int venus_session_etb(struct venus_inst *inst,
if (ret)
return ret;
- ret = venus_iface_cmdq_write(hdev, &pkt, false);
+ ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
} else {
ret = -EINVAL;
}
@@ -1331,7 +977,7 @@ static int venus_session_ftb(struct venus_inst *inst,
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, &pkt, false);
+ return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
}
static int venus_session_set_buffers(struct venus_inst *inst,
@@ -1351,7 +997,7 @@ static int venus_session_set_buffers(struct venus_inst *inst,
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, pkt, false);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
}
static int venus_session_unset_buffers(struct venus_inst *inst,
@@ -1371,7 +1017,7 @@ static int venus_session_unset_buffers(struct venus_inst *inst,
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, pkt, true);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, true);
}
static int venus_session_load_res(struct venus_inst *inst)
@@ -1398,7 +1044,7 @@ static int venus_session_parse_seq_hdr(struct venus_inst *inst, u32 seq_hdr,
if (ret)
return ret;
- ret = venus_iface_cmdq_write(hdev, pkt, false);
+ ret = venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
if (ret)
return ret;
@@ -1419,7 +1065,7 @@ static int venus_session_get_seq_hdr(struct venus_inst *inst, u32 seq_hdr,
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, pkt, false);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
}
static int venus_session_set_property(struct venus_inst *inst, u32 ptype,
@@ -1438,7 +1084,7 @@ static int venus_session_set_property(struct venus_inst *inst, u32 ptype,
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, pkt, false);
+ return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
}
static int venus_session_get_property(struct venus_inst *inst, u32 ptype)
@@ -1451,7 +1097,7 @@ static int venus_session_get_property(struct venus_inst *inst, u32 ptype)
if (ret)
return ret;
- return venus_iface_cmdq_write(hdev, &pkt, true);
+ return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
}
static int venus_resume(struct venus_core *core)
@@ -1685,7 +1331,11 @@ void venus_hfi_destroy(struct venus_core *core)
struct venus_hfi_device *hdev = to_hfi_priv(core);
core->priv = NULL;
- venus_interface_queues_release(hdev);
+ mutex_lock(&hdev->lock);
+ hfi_queue_deinit(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr,
+ &hdev->queues[0], &hdev->queues[1],
+ &hdev->queues[2]);
+ mutex_unlock(&hdev->lock);
mutex_destroy(&hdev->lock);
kfree(hdev);
core->ops = NULL;
@@ -1707,7 +1357,9 @@ int venus_hfi_create(struct venus_core *core)
core->priv = hdev;
core->ops = &venus_hfi_ops;
- ret = venus_interface_queues_init(hdev);
+ ret = hfi_queue_init(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr,
+ &hdev->queues[0], &hdev->queues[1],
+ &hdev->queues[2], &hdev->core);
if (ret)
goto err_kfree;
@@ -1724,7 +1376,7 @@ void venus_hfi_queues_reinit(struct venus_core *core)
{
struct venus_hfi_device *hdev = to_hfi_priv(core);
struct hfi_queue_table_header *tbl_hdr;
- struct iface_queue *queue;
+ struct iface_q_info *queue;
struct hfi_sfr *sfr;
unsigned int i;
@@ -1733,36 +1385,36 @@ void venus_hfi_queues_reinit(struct venus_core *core)
for (i = 0; i < IFACEQ_NUM; i++) {
queue = &hdev->queues[i];
queue->qhdr =
- IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
+ IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kernel_vaddr, i);
venus_set_qhdr_defaults(queue->qhdr);
- queue->qhdr->start_addr = queue->qmem.da;
+ queue->qhdr->qhdr_start_addr = queue->q_array.device_addr;
if (i == IFACEQ_CMD_IDX)
- queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
+ queue->qhdr->qhdr_type |= HFI_HOST_TO_CTRL_CMD_Q;
else if (i == IFACEQ_MSG_IDX)
- queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
+ queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_MSG_Q;
else if (i == IFACEQ_DBG_IDX)
- queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
+ queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_DBG_Q;
}
- tbl_hdr = hdev->ifaceq_table.kva;
- tbl_hdr->version = 0;
- tbl_hdr->size = IFACEQ_TABLE_SIZE;
- tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
- tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
- tbl_hdr->num_q = IFACEQ_NUM;
- tbl_hdr->num_active_q = IFACEQ_NUM;
+ tbl_hdr = hdev->ifaceq_table.kernel_vaddr;
+ tbl_hdr->qtbl_version = 0;
+ tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE;
+ tbl_hdr->qtbl_qhdr0_offset = sizeof(struct hfi_queue_table_header);
+ tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header);
+ tbl_hdr->qtbl_num_q = IFACEQ_NUM;
+ tbl_hdr->qtbl_num_active_q = IFACEQ_NUM;
/*
* Set receive request to zero on debug queue as there is no
* need of interrupt from video hardware for debug messages
*/
queue = &hdev->queues[IFACEQ_DBG_IDX];
- queue->qhdr->rx_req = 0;
+ queue->qhdr->qhdr_rx_req = 0;
- sfr = hdev->sfr.kva;
+ sfr = hdev->sfr.kernel_vaddr;
sfr->buf_size = ALIGNED_SFR_SIZE;
/* ensure table and queue header structs are settled in memory */
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 02/34] media: introduce common helpers for queues handling
2023-12-18 11:31 ` [PATCH v2 02/34] media: introduce common helpers for queues handling Dikshita Agarwal
@ 2023-12-18 18:29 ` Dmitry Baryshkov
2023-12-20 8:03 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:29 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:31, Dikshita Agarwal wrote:
> Introduce common helpers for shared queues init, deinit
> and read/write queue operations and use them in venus driver.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++++++++++
> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++++++
> drivers/media/platform/qcom/vcodec/venus/Makefile | 3 +-
> .../media/platform/qcom/vcodec/venus/hfi_venus.c | 500 ++++-----------------
> 4 files changed, 465 insertions(+), 425 deletions(-)
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
>
> diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.c b/drivers/media/platform/qcom/vcodec/hfi_queue.c
> new file mode 100644
> index 0000000..e3e470b
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.c
> @@ -0,0 +1,258 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include "hfi_queue.h"
> +
> +int write_queue(struct iface_q_info *qinfo, void *packet,
> + u32 packet_size, u32 *rx_req)
Same issue here: no prefixes.
> +{
> + u32 empty_space, read_idx, write_idx, new_write_idx;
> + struct hfi_queue_header *queue;
> + u32 *write_ptr, residue;
> +
> + queue = qinfo->qhdr;
> + if (!queue)
> + return -EINVAL;
> +
> + if (!packet_size || packet_size > qinfo->q_array.size)
> + return -EINVAL;
> +
> + read_idx = queue->qhdr_read_idx * sizeof(u32);
> + write_idx = queue->qhdr_write_idx * sizeof(u32);
> +
> + empty_space = (write_idx >= read_idx) ?
> + (qinfo->q_array.size - (write_idx - read_idx)) :
> + (read_idx - write_idx);
> + if (empty_space <= packet_size) {
> + queue->qhdr_tx_req = 1;
> + return -ENOSPC;
> + }
> +
> + queue->qhdr_tx_req = 0;
> +
> + new_write_idx = write_idx + packet_size;
> + write_ptr = (u32 *)((u8 *)qinfo->q_array.kernel_vaddr + write_idx);
> +
> + if (write_ptr < (u32 *)qinfo->q_array.kernel_vaddr ||
> + write_ptr > (u32 *)(qinfo->q_array.kernel_vaddr +
> + qinfo->q_array.size))
> + return -EINVAL;
> +
> + if (new_write_idx < qinfo->q_array.size) {
> + memcpy(write_ptr, packet, packet_size);
> + } else {
> + residue = new_write_idx - qinfo->q_array.size;
> + memcpy(write_ptr, packet, (packet_size - residue));
> + memcpy(qinfo->q_array.kernel_vaddr,
> + packet + (packet_size - residue), residue);
> + new_write_idx = residue;
> + }
> +
> + /* Make sure packet is written before updating the write index */
> + mb();
> + queue->qhdr_write_idx = new_write_idx / sizeof(u32);
> + *rx_req = queue->qhdr_rx_req ? 1 : 0;
> +
> + /* Make sure write index is updated before an interrupt is raised */
> + mb();
> +
> + return 0;
> +}
> +
> +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req)
> +{
> + u32 read_idx, write_idx, new_read_idx;
> + struct hfi_queue_header *queue;
> + u32 receive_request = 0;
> + u32 packet_size, residue;
> + u32 *read_ptr;
> + int ret = 0;
> +
> + /* Make sure data is valid before reading it */
> + mb();
> + queue = qinfo->qhdr;
> + if (!queue)
> + return -EINVAL;
> +
> + if (queue->qhdr_type & IFACEQ_MSGQ_ID)
> + receive_request = 1;
> +
> + read_idx = queue->qhdr_read_idx * sizeof(u32);
> + write_idx = queue->qhdr_write_idx * sizeof(u32);
> +
> + if (read_idx == write_idx) {
> + queue->qhdr_rx_req = receive_request;
> + /* Ensure qhdr is updated in main memory */
> + mb();
> + return -ENODATA;
> + }
> +
> + read_ptr = (u32 *)(qinfo->q_array.kernel_vaddr + read_idx);
> + if (read_ptr < (u32 *)qinfo->q_array.kernel_vaddr ||
> + read_ptr > (u32 *)(qinfo->q_array.kernel_vaddr +
> + qinfo->q_array.size - sizeof(*read_ptr)))
> + return -ENODATA;
> +
> + packet_size = *read_ptr;
> + if (!packet_size)
> + return -EINVAL;
> +
> + new_read_idx = read_idx + packet_size;
> + if (packet_size <= IFACEQ_CORE_PKT_SIZE &&
> + read_idx <= qinfo->q_array.size) {
> + if (new_read_idx < qinfo->q_array.size) {
> + memcpy(packet, read_ptr, packet_size);
> + } else {
> + residue = new_read_idx - qinfo->q_array.size;
> + memcpy(packet, read_ptr, (packet_size - residue));
> + memcpy((packet + (packet_size - residue)),
> + qinfo->q_array.kernel_vaddr, residue);
> + new_read_idx = residue;
> + }
> + } else {
> + new_read_idx = write_idx;
> + ret = -EBADMSG;
> + }
> +
> + queue->qhdr_rx_req = receive_request;
> +
> + queue->qhdr_read_idx = new_read_idx / sizeof(u32);
> +
> + *tx_req = queue->qhdr_tx_req ? 1 : 0;
> + /* Ensure qhdr is updated in main memory */
> + mb();
> +
> + return ret;
> +}
> +
> +static void __set_queue_hdr_defaults(struct hfi_queue_header *q_hdr)
> +{
> + q_hdr->qhdr_status = 0x1;
> + q_hdr->qhdr_type = IFACEQ_DFLT_QHDR;
> + q_hdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4;
> + q_hdr->qhdr_pkt_size = 0;
> + q_hdr->qhdr_rx_wm = 0x1;
> + q_hdr->qhdr_tx_wm = 0x1;
> + q_hdr->qhdr_rx_req = 0x1;
> + q_hdr->qhdr_tx_req = 0x0;
> + q_hdr->qhdr_rx_irq_status = 0x0;
> + q_hdr->qhdr_tx_irq_status = 0x0;
> + q_hdr->qhdr_read_idx = 0x0;
> + q_hdr->qhdr_write_idx = 0x0;
> +}
> +
> +static void hfi_set_queue_header(u32 queue_id, struct iface_q_info *iface_q)
> +{
> + __set_queue_hdr_defaults(iface_q->qhdr);
> + iface_q->qhdr->qhdr_start_addr = iface_q->q_array.device_addr;
> + iface_q->qhdr->qhdr_type |= queue_id;
> +
> + /*
> + * Set receive request to zero on debug queue as there is no
> + * need of interrupt from video hardware for debug messages
> + */
> + if (queue_id == IFACEQ_DBGQ_ID)
> + iface_q->qhdr->qhdr_rx_req = 0;
> +}
> +
> +static void __queue_init(struct mem_desc *q_table, u32 queue_id, struct iface_q_info *iface_q)
> +{
> + unsigned int offset = 0;
> +
> + offset = q_table->size + (queue_id * IFACEQ_QUEUE_SIZE);
> + iface_q->q_array.device_addr = q_table->device_addr + offset;
> + iface_q->q_array.kernel_vaddr =
> + (void *)((char *)q_table->kernel_vaddr + offset);
> + iface_q->q_array.size = IFACEQ_QUEUE_SIZE;
> + iface_q->qhdr =
> + IFACEQ_GET_QHDR_START_ADDR(q_table->kernel_vaddr, queue_id);
> +
> + memset(iface_q->qhdr, 0, sizeof(struct hfi_queue_header));
> +
> + hfi_set_queue_header(queue_id, iface_q);
> +}
> +
> +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
> + struct iface_q_info *dbg_q, void *daddr)
> +{
> + struct hfi_queue_table_header *q_tbl_hdr;
> +
> + if (q_table->kernel_vaddr) {
> + hfi_set_queue_header(IFACEQ_CMDQ_ID, cmd_q);
> + hfi_set_queue_header(IFACEQ_MSGQ_ID, msg_q);
> + hfi_set_queue_header(IFACEQ_DBGQ_ID, dbg_q);
> + return 0;
> + }
> +
> + q_table->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_QUEUE_SIZE,
> + &q_table->device_addr,
> + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE);
> + if (!q_table->kernel_vaddr) {
> + dev_err(dev, "%s: queues alloc and map failed\n", __func__);
> + return -ENOMEM;
> + }
> +
> + sfr->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_SFR_SIZE,
> + &sfr->device_addr,
> + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE);
> + if (!sfr->kernel_vaddr) {
> + dev_err(dev, "%s: sfr alloc and map failed\n", __func__);
> + return -ENOMEM;
> + }
> +
> + q_table->size = IFACEQ_TABLE_SIZE;
> +
> + __queue_init(q_table, IFACEQ_CMDQ_ID, cmd_q);
> + __queue_init(q_table, IFACEQ_MSGQ_ID, msg_q);
> + __queue_init(q_table, IFACEQ_DBGQ_ID, dbg_q);
> +
> + q_tbl_hdr = (struct hfi_queue_table_header *)q_table->kernel_vaddr;
> + q_tbl_hdr->qtbl_version = 0;
> + q_tbl_hdr->device_addr = (void *)daddr;
> + strscpy(q_tbl_hdr->name, "hfi-queues", sizeof(q_tbl_hdr->name));
> + q_tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE;
> + q_tbl_hdr->qtbl_qhdr0_offset = sizeof(*q_tbl_hdr);
> + q_tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header);
> + q_tbl_hdr->qtbl_num_q = IFACEQ_NUMQ;
> + q_tbl_hdr->qtbl_num_active_q = IFACEQ_NUMQ;
> +
> + sfr->size = ALIGNED_SFR_SIZE;
> + /* Write sfr size in first word to be used by firmware */
> + *((u32 *)sfr->kernel_vaddr) = sfr->size;
> +
> + return 0;
> +}
> +
> +static void __queue_deinit(struct iface_q_info *iface_q)
> +{
> + iface_q->qhdr = NULL;
> + iface_q->q_array.kernel_vaddr = NULL;
> + iface_q->q_array.device_addr = 0;
> +}
> +
> +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
> + struct iface_q_info *dbg_q)
> +{
> + if (!q_table->kernel_vaddr)
> + return;
> +
> + dma_free_attrs(dev, q_table->size, q_table->kernel_vaddr,
> + q_table->device_addr, q_table->attrs);
> +
> + dma_free_attrs(dev, sfr->size, sfr->kernel_vaddr,
> + sfr->device_addr, sfr->attrs);
> +
> + __queue_deinit(cmd_q);
> + __queue_deinit(msg_q);
> + __queue_deinit(dbg_q);
> +
> + q_table->kernel_vaddr = NULL;
> + q_table->device_addr = 0;
> +
> + sfr->kernel_vaddr = NULL;
> + sfr->device_addr = 0;
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.h b/drivers/media/platform/qcom/vcodec/hfi_queue.h
> new file mode 100644
> index 0000000..8cc9491
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.h
> @@ -0,0 +1,129 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _HFI_QUEUE_H_
> +#define _HFI_QUEUE_H_
> +
> +#include <linux/types.h>
> +#include <media/v4l2-device.h>
> +
> +/*
> + * Shared queues are used for communication between driver and firmware.
> + * There are 3 types of queues:
> + * Command queue - driver to write any command to firmware.
> + * Message queue - firmware to send any response to driver.
> + * Debug queue - firmware to write debug message.
> + */
> +
> +/* Host-firmware shared queue ids */
> +enum iris_iface_queue {
> + IFACEQ_CMDQ_ID = 0,
> + IFACEQ_MSGQ_ID,
> + IFACEQ_DBGQ_ID,
> + IFACEQ_NUMQ, /* not an index */
> +};
> +
> +#define IFACEQ_MAX_BUF_COUNT 50
> +/*
> + * Max session supported are 16.
> + * this value is used to calculate the size of
> + * individual shared queue.
> + */
> +#define IFACE_MAX_PARALLEL_CLNTS 16
> +#define IFACEQ_DFLT_QHDR 0x01010000
> +
> +#define IFACEQ_MAX_PKT_SIZE 1024
> +#define IFACEQ_CORE_PKT_SIZE (1024 * 4)
> +
> +#define IFACEQ_TABLE_SIZE (sizeof(struct hfi_queue_table_header) + \
> + sizeof(struct hfi_queue_header) * IFACEQ_NUMQ)
> +
> +#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \
> + IFACEQ_MAX_BUF_COUNT * IFACE_MAX_PARALLEL_CLNTS)
> +
> +#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \
> + ((void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \
> + ((i) * sizeof(struct hfi_queue_header))))
> +
> +#define SFR_SIZE SZ_4K
> +#define QUEUE_SIZE (IFACEQ_TABLE_SIZE + \
> + (IFACEQ_QUEUE_SIZE * IFACEQ_NUMQ))
> +
> +#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K)
> +#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K)
> +#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE, SZ_1M)
> +
> +struct mem_desc {
> + dma_addr_t device_addr;
> + void *kernel_vaddr;
> + u32 size;
> + unsigned long attrs;
> +};
> +
> +struct hfi_queue_table_header {
> + u32 qtbl_version;
> + u32 qtbl_size;
> + u32 qtbl_qhdr0_offset;
> + u32 qtbl_qhdr_size;
> + u32 qtbl_num_q;
> + u32 qtbl_num_active_q;
> + void *device_addr;
> + char name[256];
> +};
> +
> +/**
> + * struct hfi_queue_header - queue header definition
> + *
> + * @qhdr_status: bits (7:0) 0x1 - active 0x0 - inactive
> + * @qhdr_start_addr: queue start addr.
> + * @qhdr_type: queue type.
> + * @qhdr_q_size: size in number of HFI_QUEUE_PACKET_TYPE packets it can hold.
> + * @qhdr_pkt_size: size of the Queue packet entries, in bytes.
> + * @qhdr_rx_wm: the receiver watermark in number of queue packets.
> + * @qhdr_tx_wm: the transmitter watermark in number of queue packets.
> + * @qhdr_rx_req: set to request an interrupt from transmitter if qhdr_tx_wm is reached.
> + * @qhdr_tx_req: set to request an interrupt from receiver if qhdr_rx_wm is reached.
> + * @qhdr_rx_irq_status: status of receiver irq.
> + * @qhdr_tx_irq_status: status of transmitter irq.
> + * @qhdr_read_idx: read index, which starts with a value of 0 and wraps around after
> + * (QHDR_Q_SIZE - 1).
> + * @qhdr_write_idx: write index, which starts with a value of 0 and wraps around after
> + * (QHDR_Q_SIZE - 1).
> + */
> +
> +struct hfi_queue_header {
> + u32 qhdr_status;
> + u32 qhdr_start_addr;
> + u32 qhdr_type;
> + u32 qhdr_q_size;
> + u32 qhdr_pkt_size;
> + u32 qhdr_pkt_drop_cnt;
> + u32 qhdr_rx_wm;
> + u32 qhdr_tx_wm;
> + u32 qhdr_rx_req;
> + u32 qhdr_tx_req;
> + u32 qhdr_rx_irq_status;
> + u32 qhdr_tx_irq_status;
> + u32 qhdr_read_idx;
> + u32 qhdr_write_idx;
> +};
> +
> +struct iface_q_info {
> + struct hfi_queue_header *qhdr;
> + struct mem_desc q_array;
> +};
> +
> +int write_queue(struct iface_q_info *qinfo, void *packet,
> + u32 packet_size, u32 *rx_req);
> +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req);
> +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
> + struct iface_q_info *dbg_q, void *daddr);
> +
> +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr,
> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
> + struct iface_q_info *dbg_q);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
> index f6f3a88..1941ef4 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> @@ -1,7 +1,8 @@
> # SPDX-License-Identifier: GPL-2.0
> # Makefile for Qualcomm Venus driver
>
> -venus-core-objs += ../firmware.o
> +venus-core-objs += ../firmware.o \
> + ../hfi_queue.o
>
> venus-core-objs += core.o helpers.o firmware_no_tz.o \
> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> index 5a68db9..41e3f27 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> @@ -16,6 +16,7 @@
> #include "../firmware.h"
> #include "hfi_cmds.h"
> #include "hfi_msgs.h"
> +#include "../hfi_queue.h"
> #include "hfi_venus.h"
> #include "hfi_venus_io.h"
> #include "firmware_no_tz.h"
> @@ -34,79 +35,13 @@
> #define IFACEQ_CMD_IDX 0
> #define IFACEQ_MSG_IDX 1
> #define IFACEQ_DBG_IDX 2
> -#define IFACEQ_MAX_BUF_COUNT 50
> -#define IFACEQ_MAX_PARALLEL_CLNTS 16
> -#define IFACEQ_DFLT_QHDR 0x01010000
>
> #define POLL_INTERVAL_US 50
>
> -#define IFACEQ_MAX_PKT_SIZE 1024
> -#define IFACEQ_MED_PKT_SIZE 768
> -#define IFACEQ_MIN_PKT_SIZE 8
> #define IFACEQ_VAR_SMALL_PKT_SIZE 100
> #define IFACEQ_VAR_LARGE_PKT_SIZE 512
> #define IFACEQ_VAR_HUGE_PKT_SIZE (1024 * 12)
>
> -struct hfi_queue_table_header {
> - u32 version;
> - u32 size;
> - u32 qhdr0_offset;
> - u32 qhdr_size;
> - u32 num_q;
> - u32 num_active_q;
> -};
> -
> -struct hfi_queue_header {
> - u32 status;
> - u32 start_addr;
> - u32 type;
> - u32 q_size;
> - u32 pkt_size;
> - u32 pkt_drop_cnt;
> - u32 rx_wm;
> - u32 tx_wm;
> - u32 rx_req;
> - u32 tx_req;
> - u32 rx_irq_status;
> - u32 tx_irq_status;
> - u32 read_idx;
> - u32 write_idx;
> -};
> -
> -#define IFACEQ_TABLE_SIZE \
> - (sizeof(struct hfi_queue_table_header) + \
> - sizeof(struct hfi_queue_header) * IFACEQ_NUM)
> -
> -#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \
> - IFACEQ_MAX_BUF_COUNT * IFACEQ_MAX_PARALLEL_CLNTS)
> -
> -#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \
> - (void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \
> - ((i) * sizeof(struct hfi_queue_header)))
> -
> -#define QDSS_SIZE SZ_4K
> -#define SFR_SIZE SZ_4K
> -#define QUEUE_SIZE \
> - (IFACEQ_TABLE_SIZE + (IFACEQ_QUEUE_SIZE * IFACEQ_NUM))
> -
> -#define ALIGNED_QDSS_SIZE ALIGN(QDSS_SIZE, SZ_4K)
> -#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K)
> -#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K)
> -#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE + \
> - ALIGNED_QDSS_SIZE, SZ_1M)
> -
> -struct mem_desc {
> - dma_addr_t da; /* device address */
> - void *kva; /* kernel virtual address */
> - u32 size;
> - unsigned long attrs;
> -};
> -
> -struct iface_queue {
> - struct hfi_queue_header *qhdr;
> - struct mem_desc qmem;
> -};
> -
> enum venus_state {
> VENUS_STATE_DEINIT = 1,
> VENUS_STATE_INIT,
> @@ -125,12 +60,11 @@ struct venus_hfi_device {
> struct completion release_resource;
> struct mem_desc ifaceq_table;
> struct mem_desc sfr;
> - struct iface_queue queues[IFACEQ_NUM];
> + struct iface_q_info queues[IFACEQ_NUM];
> u8 pkt_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
> u8 dbg_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
> };
>
> -static bool venus_pkt_debug;
> int venus_fw_debug = HFI_DEBUG_MSG_ERROR | HFI_DEBUG_MSG_FATAL;
> static bool venus_fw_low_power_mode = true;
> static int venus_hw_rsp_timeout = 1000;
> @@ -149,212 +83,6 @@ static bool venus_is_valid_state(struct venus_hfi_device *hdev)
> return hdev->state != VENUS_STATE_DEINIT;
> }
>
> -static void venus_dump_packet(struct venus_hfi_device *hdev, const void *packet)
> -{
> - size_t pkt_size = *(u32 *)packet;
> -
> - if (!venus_pkt_debug)
> - return;
> -
> - print_hex_dump(KERN_DEBUG, "", DUMP_PREFIX_OFFSET, 16, 1, packet,
> - pkt_size, true);
> -}
> -
> -static int venus_write_queue(struct venus_hfi_device *hdev,
> - struct iface_queue *queue,
> - void *packet, u32 *rx_req)
> -{
> - struct hfi_queue_header *qhdr;
> - u32 dwords, new_wr_idx;
> - u32 empty_space, rd_idx, wr_idx, qsize;
> - u32 *wr_ptr;
> -
> - if (!queue->qmem.kva)
> - return -EINVAL;
> -
> - qhdr = queue->qhdr;
> - if (!qhdr)
> - return -EINVAL;
> -
> - venus_dump_packet(hdev, packet);
> -
> - dwords = (*(u32 *)packet) >> 2;
> - if (!dwords)
> - return -EINVAL;
> -
> - rd_idx = qhdr->read_idx;
> - wr_idx = qhdr->write_idx;
> - qsize = qhdr->q_size;
> - /* ensure rd/wr indices's are read from memory */
> - rmb();
> -
> - if (wr_idx >= rd_idx)
> - empty_space = qsize - (wr_idx - rd_idx);
> - else
> - empty_space = rd_idx - wr_idx;
> -
> - if (empty_space <= dwords) {
> - qhdr->tx_req = 1;
> - /* ensure tx_req is updated in memory */
> - wmb();
> - return -ENOSPC;
> - }
> -
> - qhdr->tx_req = 0;
> - /* ensure tx_req is updated in memory */
> - wmb();
> -
> - new_wr_idx = wr_idx + dwords;
> - wr_ptr = (u32 *)(queue->qmem.kva + (wr_idx << 2));
> -
> - if (wr_ptr < (u32 *)queue->qmem.kva ||
> - wr_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*wr_ptr)))
> - return -EINVAL;
> -
> - if (new_wr_idx < qsize) {
> - memcpy(wr_ptr, packet, dwords << 2);
> - } else {
> - size_t len;
> -
> - new_wr_idx -= qsize;
> - len = (dwords - new_wr_idx) << 2;
> - memcpy(wr_ptr, packet, len);
> - memcpy(queue->qmem.kva, packet + len, new_wr_idx << 2);
> - }
> -
> - /* make sure packet is written before updating the write index */
> - wmb();
> -
> - qhdr->write_idx = new_wr_idx;
> - *rx_req = qhdr->rx_req ? 1 : 0;
> -
> - /* make sure write index is updated before an interrupt is raised */
> - mb();
> -
> - return 0;
> -}
> -
> -static int venus_read_queue(struct venus_hfi_device *hdev,
> - struct iface_queue *queue, void *pkt, u32 *tx_req)
> -{
> - struct hfi_queue_header *qhdr;
> - u32 dwords, new_rd_idx;
> - u32 rd_idx, wr_idx, type, qsize;
> - u32 *rd_ptr;
> - u32 recv_request = 0;
> - int ret = 0;
> -
> - if (!queue->qmem.kva)
> - return -EINVAL;
> -
> - qhdr = queue->qhdr;
> - if (!qhdr)
> - return -EINVAL;
> -
> - type = qhdr->type;
> - rd_idx = qhdr->read_idx;
> - wr_idx = qhdr->write_idx;
> - qsize = qhdr->q_size;
> -
> - /* make sure data is valid before using it */
> - rmb();
> -
> - /*
> - * Do not set receive request for debug queue, if set, Venus generates
> - * interrupt for debug messages even when there is no response message
> - * available. In general debug queue will not become full as it is being
> - * emptied out for every interrupt from Venus. Venus will anyway
> - * generates interrupt if it is full.
> - */
> - if (type & HFI_CTRL_TO_HOST_MSG_Q)
> - recv_request = 1;
> -
> - if (rd_idx == wr_idx) {
> - qhdr->rx_req = recv_request;
> - *tx_req = 0;
> - /* update rx_req field in memory */
> - wmb();
> - return -ENODATA;
> - }
> -
> - rd_ptr = (u32 *)(queue->qmem.kva + (rd_idx << 2));
> -
> - if (rd_ptr < (u32 *)queue->qmem.kva ||
> - rd_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*rd_ptr)))
> - return -EINVAL;
> -
> - dwords = *rd_ptr >> 2;
> - if (!dwords)
> - return -EINVAL;
> -
> - new_rd_idx = rd_idx + dwords;
> - if (((dwords << 2) <= IFACEQ_VAR_HUGE_PKT_SIZE) && rd_idx <= qsize) {
> - if (new_rd_idx < qsize) {
> - memcpy(pkt, rd_ptr, dwords << 2);
> - } else {
> - size_t len;
> -
> - new_rd_idx -= qsize;
> - len = (dwords - new_rd_idx) << 2;
> - memcpy(pkt, rd_ptr, len);
> - memcpy(pkt + len, queue->qmem.kva, new_rd_idx << 2);
> - }
> - } else {
> - /* bad packet received, dropping */
> - new_rd_idx = qhdr->write_idx;
> - ret = -EBADMSG;
> - }
> -
> - /* ensure the packet is read before updating read index */
> - rmb();
> -
> - qhdr->read_idx = new_rd_idx;
> - /* ensure updating read index */
> - wmb();
> -
> - rd_idx = qhdr->read_idx;
> - wr_idx = qhdr->write_idx;
> - /* ensure rd/wr indices are read from memory */
> - rmb();
> -
> - if (rd_idx != wr_idx)
> - qhdr->rx_req = 0;
> - else
> - qhdr->rx_req = recv_request;
> -
> - *tx_req = qhdr->tx_req ? 1 : 0;
> -
> - /* ensure rx_req is stored to memory and tx_req is loaded from memory */
> - mb();
> -
> - venus_dump_packet(hdev, pkt);
> -
> - return ret;
> -}
> -
> -static int venus_alloc(struct venus_hfi_device *hdev, struct mem_desc *desc,
> - u32 size)
> -{
> - struct device *dev = hdev->core->dev;
> -
> - desc->attrs = DMA_ATTR_WRITE_COMBINE;
> - desc->size = ALIGN(size, SZ_4K);
> -
> - desc->kva = dma_alloc_attrs(dev, desc->size, &desc->da, GFP_KERNEL,
> - desc->attrs);
> - if (!desc->kva)
> - return -ENOMEM;
> -
> - return 0;
> -}
> -
> -static void venus_free(struct venus_hfi_device *hdev, struct mem_desc *mem)
> -{
> - struct device *dev = hdev->core->dev;
> -
> - dma_free_attrs(dev, mem->size, mem->kva, mem->da, mem->attrs);
> -}
> -
> static void venus_set_registers(struct venus_hfi_device *hdev)
> {
> const struct venus_resources *res = hdev->core->res;
> @@ -380,11 +108,11 @@ static void venus_soft_int(struct venus_hfi_device *hdev)
> }
>
> static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
> - void *pkt, bool sync)
> + void *pkt, u32 pkt_size, bool sync)
I'm tempted to say that you should first move the code, then add
pkt_size later. BTW, why do we need it at all and why wasn't it present
beforehand?
> {
> struct device *dev = hdev->core->dev;
> struct hfi_pkt_hdr *cmd_packet;
> - struct iface_queue *queue;
> + struct iface_q_info *queue;
iface_queue was good enough, unless you have a reason.
> u32 rx_req;
> int ret;
>
> @@ -396,7 +124,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
>
> queue = &hdev->queues[IFACEQ_CMD_IDX];
>
> - ret = venus_write_queue(hdev, queue, pkt, &rx_req);
> + ret = write_queue(queue, pkt, pkt_size, &rx_req);
You know, you can call venus_write_queue from your Iris driver, the
kernel won't bite you. But the generic write_queue() function will be
bitten and frowned upon.
> if (ret) {
> dev_err(dev, "write to iface cmd queue failed (%d)\n", ret);
> return ret;
> @@ -408,7 +136,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
> * commands
> */
> queue = &hdev->queues[IFACEQ_MSG_IDX];
> - queue->qhdr->rx_req = 1;
> + queue->qhdr->qhdr_rx_req = 1;
> /* ensure rx_req is updated in memory */
> wmb();
> }
> @@ -419,12 +147,12 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
> return 0;
> }
>
> -static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, bool sync)
> +static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, u32 pkt_size, bool sync)
> {
> int ret;
>
> mutex_lock(&hdev->lock);
> - ret = venus_iface_cmdq_write_nolock(hdev, pkt, sync);
> + ret = venus_iface_cmdq_write_nolock(hdev, pkt, pkt_size, sync);
> mutex_unlock(&hdev->lock);
>
> return ret;
> @@ -447,7 +175,7 @@ static int venus_hfi_core_set_resource(struct venus_core *core, u32 id,
> if (ret)
> return ret;
>
> - ret = venus_iface_cmdq_write(hdev, pkt, false);
> + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
> if (ret)
> return ret;
>
> @@ -531,12 +259,12 @@ static int venus_run(struct venus_hfi_device *hdev)
> */
> venus_set_registers(hdev);
>
> - writel(hdev->ifaceq_table.da, cpu_cs_base + UC_REGION_ADDR);
> + writel(hdev->ifaceq_table.device_addr, cpu_cs_base + UC_REGION_ADDR);
> writel(SHARED_QSIZE, cpu_cs_base + UC_REGION_SIZE);
> - writel(hdev->ifaceq_table.da, cpu_cs_base + CPU_CS_SCIACMDARG2);
> + writel(hdev->ifaceq_table.device_addr, cpu_cs_base + CPU_CS_SCIACMDARG2);
This is a separate patch material.
> writel(0x01, cpu_cs_base + CPU_CS_SCIACMDARG1);
> - if (hdev->sfr.da)
> - writel(hdev->sfr.da, cpu_cs_base + SFR_ADDR);
> + if (hdev->sfr.device_addr)
> + writel(hdev->sfr.device_addr, cpu_cs_base + SFR_ADDR);
>
> ret = venus_boot_core(hdev);
> if (ret) {
> @@ -690,7 +418,7 @@ static int venus_power_on(struct venus_hfi_device *hdev)
> static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev,
> void *pkt)
> {
> - struct iface_queue *queue;
> + struct iface_q_info *queue;
> u32 tx_req;
> int ret;
>
> @@ -699,7 +427,7 @@ static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev,
>
> queue = &hdev->queues[IFACEQ_MSG_IDX];
>
> - ret = venus_read_queue(hdev, queue, pkt, &tx_req);
> + ret = read_queue(queue, pkt, &tx_req);
> if (ret)
> return ret;
>
> @@ -723,7 +451,7 @@ static int venus_iface_msgq_read(struct venus_hfi_device *hdev, void *pkt)
> static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev,
> void *pkt)
> {
> - struct iface_queue *queue;
> + struct iface_q_info *queue;
> u32 tx_req;
> int ret;
>
> @@ -733,7 +461,7 @@ static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev,
>
> queue = &hdev->queues[IFACEQ_DBG_IDX];
>
> - ret = venus_read_queue(hdev, queue, pkt, &tx_req);
> + ret = read_queue(queue, pkt, &tx_req);
> if (ret)
> return ret;
>
> @@ -759,100 +487,18 @@ static int venus_iface_dbgq_read(struct venus_hfi_device *hdev, void *pkt)
>
> static void venus_set_qhdr_defaults(struct hfi_queue_header *qhdr)
> {
> - qhdr->status = 1;
> - qhdr->type = IFACEQ_DFLT_QHDR;
> - qhdr->q_size = IFACEQ_QUEUE_SIZE / 4;
> - qhdr->pkt_size = 0;
> - qhdr->rx_wm = 1;
> - qhdr->tx_wm = 1;
> - qhdr->rx_req = 1;
> - qhdr->tx_req = 0;
> - qhdr->rx_irq_status = 0;
> - qhdr->tx_irq_status = 0;
> - qhdr->read_idx = 0;
> - qhdr->write_idx = 0;
> -}
> -
> -static void venus_interface_queues_release(struct venus_hfi_device *hdev)
> -{
> - mutex_lock(&hdev->lock);
> -
> - venus_free(hdev, &hdev->ifaceq_table);
> - venus_free(hdev, &hdev->sfr);
> -
> - memset(hdev->queues, 0, sizeof(hdev->queues));
> - memset(&hdev->ifaceq_table, 0, sizeof(hdev->ifaceq_table));
> - memset(&hdev->sfr, 0, sizeof(hdev->sfr));
> -
> - mutex_unlock(&hdev->lock);
> -}
> -
> -static int venus_interface_queues_init(struct venus_hfi_device *hdev)
> -{
> - struct hfi_queue_table_header *tbl_hdr;
> - struct iface_queue *queue;
> - struct hfi_sfr *sfr;
> - struct mem_desc desc = {0};
> - unsigned int offset;
> - unsigned int i;
> - int ret;
> -
> - ret = venus_alloc(hdev, &desc, ALIGNED_QUEUE_SIZE);
> - if (ret)
> - return ret;
> -
> - hdev->ifaceq_table = desc;
> - offset = IFACEQ_TABLE_SIZE;
> -
> - for (i = 0; i < IFACEQ_NUM; i++) {
> - queue = &hdev->queues[i];
> - queue->qmem.da = desc.da + offset;
> - queue->qmem.kva = desc.kva + offset;
> - queue->qmem.size = IFACEQ_QUEUE_SIZE;
> - offset += queue->qmem.size;
> - queue->qhdr =
> - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
> -
> - venus_set_qhdr_defaults(queue->qhdr);
> -
> - queue->qhdr->start_addr = queue->qmem.da;
> -
> - if (i == IFACEQ_CMD_IDX)
> - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
> - else if (i == IFACEQ_MSG_IDX)
> - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
> - else if (i == IFACEQ_DBG_IDX)
> - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
> - }
> -
> - tbl_hdr = hdev->ifaceq_table.kva;
> - tbl_hdr->version = 0;
> - tbl_hdr->size = IFACEQ_TABLE_SIZE;
> - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
> - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
> - tbl_hdr->num_q = IFACEQ_NUM;
> - tbl_hdr->num_active_q = IFACEQ_NUM;
> -
> - /*
> - * Set receive request to zero on debug queue as there is no
> - * need of interrupt from video hardware for debug messages
> - */
> - queue = &hdev->queues[IFACEQ_DBG_IDX];
> - queue->qhdr->rx_req = 0;
> -
> - ret = venus_alloc(hdev, &desc, ALIGNED_SFR_SIZE);
> - if (ret) {
> - hdev->sfr.da = 0;
> - } else {
> - hdev->sfr = desc;
> - sfr = hdev->sfr.kva;
> - sfr->buf_size = ALIGNED_SFR_SIZE;
> - }
> -
> - /* ensure table and queue header structs are settled in memory */
> - wmb();
> -
> - return 0;
> + qhdr->qhdr_status = 1;
> + qhdr->qhdr_type = IFACEQ_DFLT_QHDR;
> + qhdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4;
> + qhdr->qhdr_pkt_size = 0;
> + qhdr->qhdr_rx_wm = 1;
> + qhdr->qhdr_tx_wm = 1;
> + qhdr->qhdr_rx_req = 1;
> + qhdr->qhdr_tx_req = 0;
> + qhdr->qhdr_rx_irq_status = 0;
> + qhdr->qhdr_tx_irq_status = 0;
> + qhdr->qhdr_read_idx = 0;
> + qhdr->qhdr_write_idx = 0;
> }
>
> static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug)
> @@ -864,7 +510,7 @@ static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug)
>
> pkt_sys_debug_config(pkt, HFI_DEBUG_MODE_QUEUE, debug);
>
> - return venus_iface_cmdq_write(hdev, pkt, false);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
> }
>
> static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode)
> @@ -876,7 +522,7 @@ static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode)
>
> pkt_sys_coverage_config(pkt, mode);
>
> - return venus_iface_cmdq_write(hdev, pkt, false);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
> }
>
> static int venus_sys_set_idle_message(struct venus_hfi_device *hdev,
> @@ -892,7 +538,7 @@ static int venus_sys_set_idle_message(struct venus_hfi_device *hdev,
>
> pkt_sys_idle_indicator(pkt, enable);
>
> - return venus_iface_cmdq_write(hdev, pkt, false);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
> }
>
> static int venus_sys_set_power_control(struct venus_hfi_device *hdev,
> @@ -905,7 +551,7 @@ static int venus_sys_set_power_control(struct venus_hfi_device *hdev,
>
> pkt_sys_power_control(pkt, enable);
>
> - return venus_iface_cmdq_write(hdev, pkt, false);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
> }
>
> static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev)
> @@ -919,7 +565,7 @@ static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev)
>
> pkt_sys_ubwc_config(pkt, res->ubwc_conf);
>
> - ret = venus_iface_cmdq_write(hdev, pkt, false);
> + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
> if (ret)
> return ret;
>
> @@ -938,7 +584,7 @@ static int venus_get_queue_size(struct venus_hfi_device *hdev,
> if (!qhdr)
> return -EINVAL;
>
> - return abs(qhdr->read_idx - qhdr->write_idx);
> + return abs(qhdr->qhdr_read_idx - qhdr->qhdr_write_idx);
> }
>
> static int venus_sys_set_default_properties(struct venus_hfi_device *hdev)
> @@ -980,7 +626,7 @@ static int venus_session_cmd(struct venus_inst *inst, u32 pkt_type, bool sync)
>
> pkt_session_cmd(&pkt, pkt_type, inst);
>
> - return venus_iface_cmdq_write(hdev, &pkt, sync);
> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, sync);
> }
>
> static void venus_flush_debug_queue(struct venus_hfi_device *hdev)
> @@ -1010,7 +656,7 @@ static int venus_prepare_power_collapse(struct venus_hfi_device *hdev,
>
> pkt_sys_pc_prep(&pkt);
>
> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
> if (ret)
> return ret;
>
> @@ -1047,7 +693,7 @@ static int venus_are_queues_empty(struct venus_hfi_device *hdev)
> static void venus_sfr_print(struct venus_hfi_device *hdev)
> {
> struct device *dev = hdev->core->dev;
> - struct hfi_sfr *sfr = hdev->sfr.kva;
> + struct hfi_sfr *sfr = hdev->sfr.kernel_vaddr;
> void *p;
>
> if (!sfr)
> @@ -1163,13 +809,13 @@ static int venus_core_init(struct venus_core *core)
>
> venus_set_state(hdev, VENUS_STATE_INIT);
>
> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
> if (ret)
> return ret;
>
> pkt_sys_image_version(&version_pkt);
>
> - ret = venus_iface_cmdq_write(hdev, &version_pkt, false);
> + ret = venus_iface_cmdq_write(hdev, &version_pkt, version_pkt.hdr.size, false);
> if (ret)
> dev_warn(dev, "failed to send image version pkt to fw\n");
>
> @@ -1198,7 +844,7 @@ static int venus_core_ping(struct venus_core *core, u32 cookie)
>
> pkt_sys_ping(&pkt, cookie);
>
> - return venus_iface_cmdq_write(hdev, &pkt, false);
> + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
> }
>
> static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type)
> @@ -1211,7 +857,7 @@ static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type)
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, &pkt, false);
> + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
> }
>
> static int venus_session_init(struct venus_inst *inst, u32 session_type,
> @@ -1229,7 +875,7 @@ static int venus_session_init(struct venus_inst *inst, u32 session_type,
> if (ret)
> goto err;
>
> - ret = venus_iface_cmdq_write(hdev, &pkt, true);
> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
> if (ret)
> goto err;
>
> @@ -1272,7 +918,7 @@ static int venus_session_flush(struct venus_inst *inst, u32 flush_mode)
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, &pkt, true);
> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
> }
>
> static int venus_session_start(struct venus_inst *inst)
> @@ -1304,7 +950,7 @@ static int venus_session_etb(struct venus_inst *inst,
> if (ret)
> return ret;
>
> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
> } else if (session_type == VIDC_SESSION_TYPE_ENC) {
> struct hfi_session_empty_buffer_uncompressed_plane0_pkt pkt;
>
> @@ -1312,7 +958,7 @@ static int venus_session_etb(struct venus_inst *inst,
> if (ret)
> return ret;
>
> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
> } else {
> ret = -EINVAL;
> }
> @@ -1331,7 +977,7 @@ static int venus_session_ftb(struct venus_inst *inst,
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, &pkt, false);
> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
> }
>
> static int venus_session_set_buffers(struct venus_inst *inst,
> @@ -1351,7 +997,7 @@ static int venus_session_set_buffers(struct venus_inst *inst,
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, pkt, false);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
> }
>
> static int venus_session_unset_buffers(struct venus_inst *inst,
> @@ -1371,7 +1017,7 @@ static int venus_session_unset_buffers(struct venus_inst *inst,
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, pkt, true);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, true);
> }
>
> static int venus_session_load_res(struct venus_inst *inst)
> @@ -1398,7 +1044,7 @@ static int venus_session_parse_seq_hdr(struct venus_inst *inst, u32 seq_hdr,
> if (ret)
> return ret;
>
> - ret = venus_iface_cmdq_write(hdev, pkt, false);
> + ret = venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
> if (ret)
> return ret;
>
> @@ -1419,7 +1065,7 @@ static int venus_session_get_seq_hdr(struct venus_inst *inst, u32 seq_hdr,
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, pkt, false);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
> }
>
> static int venus_session_set_property(struct venus_inst *inst, u32 ptype,
> @@ -1438,7 +1084,7 @@ static int venus_session_set_property(struct venus_inst *inst, u32 ptype,
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, pkt, false);
> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
> }
>
> static int venus_session_get_property(struct venus_inst *inst, u32 ptype)
> @@ -1451,7 +1097,7 @@ static int venus_session_get_property(struct venus_inst *inst, u32 ptype)
> if (ret)
> return ret;
>
> - return venus_iface_cmdq_write(hdev, &pkt, true);
> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
> }
>
> static int venus_resume(struct venus_core *core)
> @@ -1685,7 +1331,11 @@ void venus_hfi_destroy(struct venus_core *core)
> struct venus_hfi_device *hdev = to_hfi_priv(core);
>
> core->priv = NULL;
> - venus_interface_queues_release(hdev);
> + mutex_lock(&hdev->lock);
> + hfi_queue_deinit(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr,
> + &hdev->queues[0], &hdev->queues[1],
> + &hdev->queues[2]);
> + mutex_unlock(&hdev->lock);
> mutex_destroy(&hdev->lock);
> kfree(hdev);
> core->ops = NULL;
> @@ -1707,7 +1357,9 @@ int venus_hfi_create(struct venus_core *core)
> core->priv = hdev;
> core->ops = &venus_hfi_ops;
>
> - ret = venus_interface_queues_init(hdev);
> + ret = hfi_queue_init(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr,
> + &hdev->queues[0], &hdev->queues[1],
> + &hdev->queues[2], &hdev->core);
> if (ret)
> goto err_kfree;
>
> @@ -1724,7 +1376,7 @@ void venus_hfi_queues_reinit(struct venus_core *core)
> {
> struct venus_hfi_device *hdev = to_hfi_priv(core);
> struct hfi_queue_table_header *tbl_hdr;
> - struct iface_queue *queue;
> + struct iface_q_info *queue;
> struct hfi_sfr *sfr;
> unsigned int i;
>
> @@ -1733,36 +1385,36 @@ void venus_hfi_queues_reinit(struct venus_core *core)
> for (i = 0; i < IFACEQ_NUM; i++) {
> queue = &hdev->queues[i];
> queue->qhdr =
> - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
> + IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kernel_vaddr, i);
>
> venus_set_qhdr_defaults(queue->qhdr);
>
> - queue->qhdr->start_addr = queue->qmem.da;
> + queue->qhdr->qhdr_start_addr = queue->q_array.device_addr;
>
> if (i == IFACEQ_CMD_IDX)
> - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
> + queue->qhdr->qhdr_type |= HFI_HOST_TO_CTRL_CMD_Q;
> else if (i == IFACEQ_MSG_IDX)
> - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
> + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_MSG_Q;
> else if (i == IFACEQ_DBG_IDX)
> - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
> + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_DBG_Q;
> }
>
> - tbl_hdr = hdev->ifaceq_table.kva;
> - tbl_hdr->version = 0;
> - tbl_hdr->size = IFACEQ_TABLE_SIZE;
> - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
> - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
> - tbl_hdr->num_q = IFACEQ_NUM;
> - tbl_hdr->num_active_q = IFACEQ_NUM;
> + tbl_hdr = hdev->ifaceq_table.kernel_vaddr;
> + tbl_hdr->qtbl_version = 0;
> + tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE;
> + tbl_hdr->qtbl_qhdr0_offset = sizeof(struct hfi_queue_table_header);
> + tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header);
> + tbl_hdr->qtbl_num_q = IFACEQ_NUM;
> + tbl_hdr->qtbl_num_active_q = IFACEQ_NUM;
NAK. This is not 'an introduction of helpers'.
>
> /*
> * Set receive request to zero on debug queue as there is no
> * need of interrupt from video hardware for debug messages
> */
> queue = &hdev->queues[IFACEQ_DBG_IDX];
> - queue->qhdr->rx_req = 0;
> + queue->qhdr->qhdr_rx_req = 0;
>
> - sfr = hdev->sfr.kva;
> + sfr = hdev->sfr.kernel_vaddr;
> sfr->buf_size = ALIGNED_SFR_SIZE;
>
> /* ensure table and queue header structs are settled in memory */
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 02/34] media: introduce common helpers for queues handling
2023-12-18 18:29 ` Dmitry Baryshkov
@ 2023-12-20 8:03 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:03 UTC (permalink / raw)
To: Dmitry Baryshkov, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/2023 11:59 PM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> Introduce common helpers for shared queues init, deinit
>> and read/write queue operations and use them in venus driver.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++++++++++
>> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++++++
>> drivers/media/platform/qcom/vcodec/venus/Makefile | 3 +-
>> .../media/platform/qcom/vcodec/venus/hfi_venus.c | 500
>> ++++-----------------
>> 4 files changed, 465 insertions(+), 425 deletions(-)
>> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
>> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
>>
>> diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.c
>> b/drivers/media/platform/qcom/vcodec/hfi_queue.c
>> new file mode 100644
>> index 0000000..e3e470b
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.c
>> @@ -0,0 +1,258 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#include "hfi_queue.h"
>> +
>> +int write_queue(struct iface_q_info *qinfo, void *packet,
>> + u32 packet_size, u32 *rx_req)
>
> Same issue here: no prefixes.
>
Sure, will fix the API names with proper prefixes.
>> +{
>> + u32 empty_space, read_idx, write_idx, new_write_idx;
>> + struct hfi_queue_header *queue;
>> + u32 *write_ptr, residue;
>> +
>> + queue = qinfo->qhdr;
>> + if (!queue)
>> + return -EINVAL;
>> +
>> + if (!packet_size || packet_size > qinfo->q_array.size)
>> + return -EINVAL;
>> +
>> + read_idx = queue->qhdr_read_idx * sizeof(u32);
>> + write_idx = queue->qhdr_write_idx * sizeof(u32);
>> +
>> + empty_space = (write_idx >= read_idx) ?
>> + (qinfo->q_array.size - (write_idx - read_idx)) :
>> + (read_idx - write_idx);
>> + if (empty_space <= packet_size) {
>> + queue->qhdr_tx_req = 1;
>> + return -ENOSPC;
>> + }
>> +
>> + queue->qhdr_tx_req = 0;
>> +
>> + new_write_idx = write_idx + packet_size;
>> + write_ptr = (u32 *)((u8 *)qinfo->q_array.kernel_vaddr + write_idx);
>> +
>> + if (write_ptr < (u32 *)qinfo->q_array.kernel_vaddr ||
>> + write_ptr > (u32 *)(qinfo->q_array.kernel_vaddr +
>> + qinfo->q_array.size))
>> + return -EINVAL;
>> +
>> + if (new_write_idx < qinfo->q_array.size) {
>> + memcpy(write_ptr, packet, packet_size);
>> + } else {
>> + residue = new_write_idx - qinfo->q_array.size;
>> + memcpy(write_ptr, packet, (packet_size - residue));
>> + memcpy(qinfo->q_array.kernel_vaddr,
>> + packet + (packet_size - residue), residue);
>> + new_write_idx = residue;
>> + }
>> +
>> + /* Make sure packet is written before updating the write index */
>> + mb();
>> + queue->qhdr_write_idx = new_write_idx / sizeof(u32);
>> + *rx_req = queue->qhdr_rx_req ? 1 : 0;
>> +
>> + /* Make sure write index is updated before an interrupt is raised */
>> + mb();
>> +
>> + return 0;
>> +}
>> +
>> +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req)
>> +{
>> + u32 read_idx, write_idx, new_read_idx;
>> + struct hfi_queue_header *queue;
>> + u32 receive_request = 0;
>> + u32 packet_size, residue;
>> + u32 *read_ptr;
>> + int ret = 0;
>> +
>> + /* Make sure data is valid before reading it */
>> + mb();
>> + queue = qinfo->qhdr;
>> + if (!queue)
>> + return -EINVAL;
>> +
>> + if (queue->qhdr_type & IFACEQ_MSGQ_ID)
>> + receive_request = 1;
>> +
>> + read_idx = queue->qhdr_read_idx * sizeof(u32);
>> + write_idx = queue->qhdr_write_idx * sizeof(u32);
>> +
>> + if (read_idx == write_idx) {
>> + queue->qhdr_rx_req = receive_request;
>> + /* Ensure qhdr is updated in main memory */
>> + mb();
>> + return -ENODATA;
>> + }
>> +
>> + read_ptr = (u32 *)(qinfo->q_array.kernel_vaddr + read_idx);
>> + if (read_ptr < (u32 *)qinfo->q_array.kernel_vaddr ||
>> + read_ptr > (u32 *)(qinfo->q_array.kernel_vaddr +
>> + qinfo->q_array.size - sizeof(*read_ptr)))
>> + return -ENODATA;
>> +
>> + packet_size = *read_ptr;
>> + if (!packet_size)
>> + return -EINVAL;
>> +
>> + new_read_idx = read_idx + packet_size;
>> + if (packet_size <= IFACEQ_CORE_PKT_SIZE &&
>> + read_idx <= qinfo->q_array.size) {
>> + if (new_read_idx < qinfo->q_array.size) {
>> + memcpy(packet, read_ptr, packet_size);
>> + } else {
>> + residue = new_read_idx - qinfo->q_array.size;
>> + memcpy(packet, read_ptr, (packet_size - residue));
>> + memcpy((packet + (packet_size - residue)),
>> + qinfo->q_array.kernel_vaddr, residue);
>> + new_read_idx = residue;
>> + }
>> + } else {
>> + new_read_idx = write_idx;
>> + ret = -EBADMSG;
>> + }
>> +
>> + queue->qhdr_rx_req = receive_request;
>> +
>> + queue->qhdr_read_idx = new_read_idx / sizeof(u32);
>> +
>> + *tx_req = queue->qhdr_tx_req ? 1 : 0;
>> + /* Ensure qhdr is updated in main memory */
>> + mb();
>> +
>> + return ret;
>> +}
>> +
>> +static void __set_queue_hdr_defaults(struct hfi_queue_header *q_hdr)
>> +{
>> + q_hdr->qhdr_status = 0x1;
>> + q_hdr->qhdr_type = IFACEQ_DFLT_QHDR;
>> + q_hdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4;
>> + q_hdr->qhdr_pkt_size = 0;
>> + q_hdr->qhdr_rx_wm = 0x1;
>> + q_hdr->qhdr_tx_wm = 0x1;
>> + q_hdr->qhdr_rx_req = 0x1;
>> + q_hdr->qhdr_tx_req = 0x0;
>> + q_hdr->qhdr_rx_irq_status = 0x0;
>> + q_hdr->qhdr_tx_irq_status = 0x0;
>> + q_hdr->qhdr_read_idx = 0x0;
>> + q_hdr->qhdr_write_idx = 0x0;
>> +}
>> +
>> +static void hfi_set_queue_header(u32 queue_id, struct iface_q_info
>> *iface_q)
>> +{
>> + __set_queue_hdr_defaults(iface_q->qhdr);
>> + iface_q->qhdr->qhdr_start_addr = iface_q->q_array.device_addr;
>> + iface_q->qhdr->qhdr_type |= queue_id;
>> +
>> + /*
>> + * Set receive request to zero on debug queue as there is no
>> + * need of interrupt from video hardware for debug messages
>> + */
>> + if (queue_id == IFACEQ_DBGQ_ID)
>> + iface_q->qhdr->qhdr_rx_req = 0;
>> +}
>> +
>> +static void __queue_init(struct mem_desc *q_table, u32 queue_id, struct
>> iface_q_info *iface_q)
>> +{
>> + unsigned int offset = 0;
>> +
>> + offset = q_table->size + (queue_id * IFACEQ_QUEUE_SIZE);
>> + iface_q->q_array.device_addr = q_table->device_addr + offset;
>> + iface_q->q_array.kernel_vaddr =
>> + (void *)((char *)q_table->kernel_vaddr + offset);
>> + iface_q->q_array.size = IFACEQ_QUEUE_SIZE;
>> + iface_q->qhdr =
>> + IFACEQ_GET_QHDR_START_ADDR(q_table->kernel_vaddr, queue_id);
>> +
>> + memset(iface_q->qhdr, 0, sizeof(struct hfi_queue_header));
>> +
>> + hfi_set_queue_header(queue_id, iface_q);
>> +}
>> +
>> +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct
>> mem_desc *sfr,
>> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
>> + struct iface_q_info *dbg_q, void *daddr)
>> +{
>> + struct hfi_queue_table_header *q_tbl_hdr;
>> +
>> + if (q_table->kernel_vaddr) {
>> + hfi_set_queue_header(IFACEQ_CMDQ_ID, cmd_q);
>> + hfi_set_queue_header(IFACEQ_MSGQ_ID, msg_q);
>> + hfi_set_queue_header(IFACEQ_DBGQ_ID, dbg_q);
>> + return 0;
>> + }
>> +
>> + q_table->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_QUEUE_SIZE,
>> + &q_table->device_addr,
>> + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE);
>> + if (!q_table->kernel_vaddr) {
>> + dev_err(dev, "%s: queues alloc and map failed\n", __func__);
>> + return -ENOMEM;
>> + }
>> +
>> + sfr->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_SFR_SIZE,
>> + &sfr->device_addr,
>> + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE);
>> + if (!sfr->kernel_vaddr) {
>> + dev_err(dev, "%s: sfr alloc and map failed\n", __func__);
>> + return -ENOMEM;
>> + }
>> +
>> + q_table->size = IFACEQ_TABLE_SIZE;
>> +
>> + __queue_init(q_table, IFACEQ_CMDQ_ID, cmd_q);
>> + __queue_init(q_table, IFACEQ_MSGQ_ID, msg_q);
>> + __queue_init(q_table, IFACEQ_DBGQ_ID, dbg_q);
>> +
>> + q_tbl_hdr = (struct hfi_queue_table_header *)q_table->kernel_vaddr;
>> + q_tbl_hdr->qtbl_version = 0;
>> + q_tbl_hdr->device_addr = (void *)daddr;
>> + strscpy(q_tbl_hdr->name, "hfi-queues", sizeof(q_tbl_hdr->name));
>> + q_tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE;
>> + q_tbl_hdr->qtbl_qhdr0_offset = sizeof(*q_tbl_hdr);
>> + q_tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header);
>> + q_tbl_hdr->qtbl_num_q = IFACEQ_NUMQ;
>> + q_tbl_hdr->qtbl_num_active_q = IFACEQ_NUMQ;
>> +
>> + sfr->size = ALIGNED_SFR_SIZE;
>> + /* Write sfr size in first word to be used by firmware */
>> + *((u32 *)sfr->kernel_vaddr) = sfr->size;
>> +
>> + return 0;
>> +}
>> +
>> +static void __queue_deinit(struct iface_q_info *iface_q)
>> +{
>> + iface_q->qhdr = NULL;
>> + iface_q->q_array.kernel_vaddr = NULL;
>> + iface_q->q_array.device_addr = 0;
>> +}
>> +
>> +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table,
>> struct mem_desc *sfr,
>> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
>> + struct iface_q_info *dbg_q)
>> +{
>> + if (!q_table->kernel_vaddr)
>> + return;
>> +
>> + dma_free_attrs(dev, q_table->size, q_table->kernel_vaddr,
>> + q_table->device_addr, q_table->attrs);
>> +
>> + dma_free_attrs(dev, sfr->size, sfr->kernel_vaddr,
>> + sfr->device_addr, sfr->attrs);
>> +
>> + __queue_deinit(cmd_q);
>> + __queue_deinit(msg_q);
>> + __queue_deinit(dbg_q);
>> +
>> + q_table->kernel_vaddr = NULL;
>> + q_table->device_addr = 0;
>> +
>> + sfr->kernel_vaddr = NULL;
>> + sfr->device_addr = 0;
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.h
>> b/drivers/media/platform/qcom/vcodec/hfi_queue.h
>> new file mode 100644
>> index 0000000..8cc9491
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.h
>> @@ -0,0 +1,129 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#ifndef _HFI_QUEUE_H_
>> +#define _HFI_QUEUE_H_
>> +
>> +#include <linux/types.h>
>> +#include <media/v4l2-device.h>
>> +
>> +/*
>> + * Shared queues are used for communication between driver and firmware.
>> + * There are 3 types of queues:
>> + * Command queue - driver to write any command to firmware.
>> + * Message queue - firmware to send any response to driver.
>> + * Debug queue - firmware to write debug message.
>> + */
>> +
>> +/* Host-firmware shared queue ids */
>> +enum iris_iface_queue {
>> + IFACEQ_CMDQ_ID = 0,
>> + IFACEQ_MSGQ_ID,
>> + IFACEQ_DBGQ_ID,
>> + IFACEQ_NUMQ, /* not an index */
>> +};
>> +
>> +#define IFACEQ_MAX_BUF_COUNT 50
>> +/*
>> + * Max session supported are 16.
>> + * this value is used to calculate the size of
>> + * individual shared queue.
>> + */
>> +#define IFACE_MAX_PARALLEL_CLNTS 16
>> +#define IFACEQ_DFLT_QHDR 0x01010000
>> +
>> +#define IFACEQ_MAX_PKT_SIZE 1024
>> +#define IFACEQ_CORE_PKT_SIZE (1024 * 4)
>> +
>> +#define IFACEQ_TABLE_SIZE (sizeof(struct hfi_queue_table_header) + \
>> + sizeof(struct hfi_queue_header) * IFACEQ_NUMQ)
>> +
>> +#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \
>> + IFACEQ_MAX_BUF_COUNT * IFACE_MAX_PARALLEL_CLNTS)
>> +
>> +#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \
>> + ((void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \
>> + ((i) * sizeof(struct hfi_queue_header))))
>> +
>> +#define SFR_SIZE SZ_4K
>> +#define QUEUE_SIZE (IFACEQ_TABLE_SIZE + \
>> + (IFACEQ_QUEUE_SIZE * IFACEQ_NUMQ))
>> +
>> +#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K)
>> +#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K)
>> +#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE,
>> SZ_1M)
>> +
>> +struct mem_desc {
>> + dma_addr_t device_addr;
>> + void *kernel_vaddr;
>> + u32 size;
>> + unsigned long attrs;
>> +};
>> +
>> +struct hfi_queue_table_header {
>> + u32 qtbl_version;
>> + u32 qtbl_size;
>> + u32 qtbl_qhdr0_offset;
>> + u32 qtbl_qhdr_size;
>> + u32 qtbl_num_q;
>> + u32 qtbl_num_active_q;
>> + void *device_addr;
>> + char name[256];
>> +};
>> +
>> +/**
>> + * struct hfi_queue_header - queue header definition
>> + *
>> + * @qhdr_status: bits (7:0) 0x1 - active 0x0 - inactive
>> + * @qhdr_start_addr: queue start addr.
>> + * @qhdr_type: queue type.
>> + * @qhdr_q_size: size in number of HFI_QUEUE_PACKET_TYPE packets it can
>> hold.
>> + * @qhdr_pkt_size: size of the Queue packet entries, in bytes.
>> + * @qhdr_rx_wm: the receiver watermark in number of queue packets.
>> + * @qhdr_tx_wm: the transmitter watermark in number of queue packets.
>> + * @qhdr_rx_req: set to request an interrupt from transmitter if
>> qhdr_tx_wm is reached.
>> + * @qhdr_tx_req: set to request an interrupt from receiver if qhdr_rx_wm
>> is reached.
>> + * @qhdr_rx_irq_status: status of receiver irq.
>> + * @qhdr_tx_irq_status: status of transmitter irq.
>> + * @qhdr_read_idx: read index, which starts with a value of 0 and wraps
>> around after
>> + * (QHDR_Q_SIZE - 1).
>> + * @qhdr_write_idx: write index, which starts with a value of 0 and
>> wraps around after
>> + * (QHDR_Q_SIZE - 1).
>> + */
>> +
>> +struct hfi_queue_header {
>> + u32 qhdr_status;
>> + u32 qhdr_start_addr;
>> + u32 qhdr_type;
>> + u32 qhdr_q_size;
>> + u32 qhdr_pkt_size;
>> + u32 qhdr_pkt_drop_cnt;
>> + u32 qhdr_rx_wm;
>> + u32 qhdr_tx_wm;
>> + u32 qhdr_rx_req;
>> + u32 qhdr_tx_req;
>> + u32 qhdr_rx_irq_status;
>> + u32 qhdr_tx_irq_status;
>> + u32 qhdr_read_idx;
>> + u32 qhdr_write_idx;
>> +};
>> +
>> +struct iface_q_info {
>> + struct hfi_queue_header *qhdr;
>> + struct mem_desc q_array;
>> +};
>> +
>> +int write_queue(struct iface_q_info *qinfo, void *packet,
>> + u32 packet_size, u32 *rx_req);
>> +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req);
>> +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct
>> mem_desc *sfr,
>> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
>> + struct iface_q_info *dbg_q, void *daddr);
>> +
>> +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table,
>> struct mem_desc *sfr,
>> + struct iface_q_info *cmd_q, struct iface_q_info *msg_q,
>> + struct iface_q_info *dbg_q);
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile
>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> index f6f3a88..1941ef4 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/Makefile
>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> @@ -1,7 +1,8 @@
>> # SPDX-License-Identifier: GPL-2.0
>> # Makefile for Qualcomm Venus driver
>> -venus-core-objs += ../firmware.o
>> +venus-core-objs += ../firmware.o \
>> + ../hfi_queue.o
>> venus-core-objs += core.o helpers.o firmware_no_tz.o \
>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> index 5a68db9..41e3f27 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> @@ -16,6 +16,7 @@
>> #include "../firmware.h"
>> #include "hfi_cmds.h"
>> #include "hfi_msgs.h"
>> +#include "../hfi_queue.h"
>> #include "hfi_venus.h"
>> #include "hfi_venus_io.h"
>> #include "firmware_no_tz.h"
>> @@ -34,79 +35,13 @@
>> #define IFACEQ_CMD_IDX 0
>> #define IFACEQ_MSG_IDX 1
>> #define IFACEQ_DBG_IDX 2
>> -#define IFACEQ_MAX_BUF_COUNT 50
>> -#define IFACEQ_MAX_PARALLEL_CLNTS 16
>> -#define IFACEQ_DFLT_QHDR 0x01010000
>> #define POLL_INTERVAL_US 50
>> -#define IFACEQ_MAX_PKT_SIZE 1024
>> -#define IFACEQ_MED_PKT_SIZE 768
>> -#define IFACEQ_MIN_PKT_SIZE 8
>> #define IFACEQ_VAR_SMALL_PKT_SIZE 100
>> #define IFACEQ_VAR_LARGE_PKT_SIZE 512
>> #define IFACEQ_VAR_HUGE_PKT_SIZE (1024 * 12)
>> -struct hfi_queue_table_header {
>> - u32 version;
>> - u32 size;
>> - u32 qhdr0_offset;
>> - u32 qhdr_size;
>> - u32 num_q;
>> - u32 num_active_q;
>> -};
>> -
>> -struct hfi_queue_header {
>> - u32 status;
>> - u32 start_addr;
>> - u32 type;
>> - u32 q_size;
>> - u32 pkt_size;
>> - u32 pkt_drop_cnt;
>> - u32 rx_wm;
>> - u32 tx_wm;
>> - u32 rx_req;
>> - u32 tx_req;
>> - u32 rx_irq_status;
>> - u32 tx_irq_status;
>> - u32 read_idx;
>> - u32 write_idx;
>> -};
>> -
>> -#define IFACEQ_TABLE_SIZE \
>> - (sizeof(struct hfi_queue_table_header) + \
>> - sizeof(struct hfi_queue_header) * IFACEQ_NUM)
>> -
>> -#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \
>> - IFACEQ_MAX_BUF_COUNT * IFACEQ_MAX_PARALLEL_CLNTS)
>> -
>> -#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \
>> - (void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \
>> - ((i) * sizeof(struct hfi_queue_header)))
>> -
>> -#define QDSS_SIZE SZ_4K
>> -#define SFR_SIZE SZ_4K
>> -#define QUEUE_SIZE \
>> - (IFACEQ_TABLE_SIZE + (IFACEQ_QUEUE_SIZE * IFACEQ_NUM))
>> -
>> -#define ALIGNED_QDSS_SIZE ALIGN(QDSS_SIZE, SZ_4K)
>> -#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K)
>> -#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K)
>> -#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE + \
>> - ALIGNED_QDSS_SIZE, SZ_1M)
>> -
>> -struct mem_desc {
>> - dma_addr_t da; /* device address */
>> - void *kva; /* kernel virtual address */
>> - u32 size;
>> - unsigned long attrs;
>> -};
>> -
>> -struct iface_queue {
>> - struct hfi_queue_header *qhdr;
>> - struct mem_desc qmem;
>> -};
>> -
>> enum venus_state {
>> VENUS_STATE_DEINIT = 1,
>> VENUS_STATE_INIT,
>> @@ -125,12 +60,11 @@ struct venus_hfi_device {
>> struct completion release_resource;
>> struct mem_desc ifaceq_table;
>> struct mem_desc sfr;
>> - struct iface_queue queues[IFACEQ_NUM];
>> + struct iface_q_info queues[IFACEQ_NUM];
>> u8 pkt_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
>> u8 dbg_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
>> };
>> -static bool venus_pkt_debug;
>> int venus_fw_debug = HFI_DEBUG_MSG_ERROR | HFI_DEBUG_MSG_FATAL;
>> static bool venus_fw_low_power_mode = true;
>> static int venus_hw_rsp_timeout = 1000;
>> @@ -149,212 +83,6 @@ static bool venus_is_valid_state(struct
>> venus_hfi_device *hdev)
>> return hdev->state != VENUS_STATE_DEINIT;
>> }
>> -static void venus_dump_packet(struct venus_hfi_device *hdev, const
>> void *packet)
>> -{
>> - size_t pkt_size = *(u32 *)packet;
>> -
>> - if (!venus_pkt_debug)
>> - return;
>> -
>> - print_hex_dump(KERN_DEBUG, "", DUMP_PREFIX_OFFSET, 16, 1, packet,
>> - pkt_size, true);
>> -}
>> -
>> -static int venus_write_queue(struct venus_hfi_device *hdev,
>> - struct iface_queue *queue,
>> - void *packet, u32 *rx_req)
>> -{
>> - struct hfi_queue_header *qhdr;
>> - u32 dwords, new_wr_idx;
>> - u32 empty_space, rd_idx, wr_idx, qsize;
>> - u32 *wr_ptr;
>> -
>> - if (!queue->qmem.kva)
>> - return -EINVAL;
>> -
>> - qhdr = queue->qhdr;
>> - if (!qhdr)
>> - return -EINVAL;
>> -
>> - venus_dump_packet(hdev, packet);
>> -
>> - dwords = (*(u32 *)packet) >> 2;
>> - if (!dwords)
>> - return -EINVAL;
>> -
>> - rd_idx = qhdr->read_idx;
>> - wr_idx = qhdr->write_idx;
>> - qsize = qhdr->q_size;
>> - /* ensure rd/wr indices's are read from memory */
>> - rmb();
>> -
>> - if (wr_idx >= rd_idx)
>> - empty_space = qsize - (wr_idx - rd_idx);
>> - else
>> - empty_space = rd_idx - wr_idx;
>> -
>> - if (empty_space <= dwords) {
>> - qhdr->tx_req = 1;
>> - /* ensure tx_req is updated in memory */
>> - wmb();
>> - return -ENOSPC;
>> - }
>> -
>> - qhdr->tx_req = 0;
>> - /* ensure tx_req is updated in memory */
>> - wmb();
>> -
>> - new_wr_idx = wr_idx + dwords;
>> - wr_ptr = (u32 *)(queue->qmem.kva + (wr_idx << 2));
>> -
>> - if (wr_ptr < (u32 *)queue->qmem.kva ||
>> - wr_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size -
>> sizeof(*wr_ptr)))
>> - return -EINVAL;
>> -
>> - if (new_wr_idx < qsize) {
>> - memcpy(wr_ptr, packet, dwords << 2);
>> - } else {
>> - size_t len;
>> -
>> - new_wr_idx -= qsize;
>> - len = (dwords - new_wr_idx) << 2;
>> - memcpy(wr_ptr, packet, len);
>> - memcpy(queue->qmem.kva, packet + len, new_wr_idx << 2);
>> - }
>> -
>> - /* make sure packet is written before updating the write index */
>> - wmb();
>> -
>> - qhdr->write_idx = new_wr_idx;
>> - *rx_req = qhdr->rx_req ? 1 : 0;
>> -
>> - /* make sure write index is updated before an interrupt is raised */
>> - mb();
>> -
>> - return 0;
>> -}
>> -
>> -static int venus_read_queue(struct venus_hfi_device *hdev,
>> - struct iface_queue *queue, void *pkt, u32 *tx_req)
>> -{
>> - struct hfi_queue_header *qhdr;
>> - u32 dwords, new_rd_idx;
>> - u32 rd_idx, wr_idx, type, qsize;
>> - u32 *rd_ptr;
>> - u32 recv_request = 0;
>> - int ret = 0;
>> -
>> - if (!queue->qmem.kva)
>> - return -EINVAL;
>> -
>> - qhdr = queue->qhdr;
>> - if (!qhdr)
>> - return -EINVAL;
>> -
>> - type = qhdr->type;
>> - rd_idx = qhdr->read_idx;
>> - wr_idx = qhdr->write_idx;
>> - qsize = qhdr->q_size;
>> -
>> - /* make sure data is valid before using it */
>> - rmb();
>> -
>> - /*
>> - * Do not set receive request for debug queue, if set, Venus generates
>> - * interrupt for debug messages even when there is no response message
>> - * available. In general debug queue will not become full as it is
>> being
>> - * emptied out for every interrupt from Venus. Venus will anyway
>> - * generates interrupt if it is full.
>> - */
>> - if (type & HFI_CTRL_TO_HOST_MSG_Q)
>> - recv_request = 1;
>> -
>> - if (rd_idx == wr_idx) {
>> - qhdr->rx_req = recv_request;
>> - *tx_req = 0;
>> - /* update rx_req field in memory */
>> - wmb();
>> - return -ENODATA;
>> - }
>> -
>> - rd_ptr = (u32 *)(queue->qmem.kva + (rd_idx << 2));
>> -
>> - if (rd_ptr < (u32 *)queue->qmem.kva ||
>> - rd_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size -
>> sizeof(*rd_ptr)))
>> - return -EINVAL;
>> -
>> - dwords = *rd_ptr >> 2;
>> - if (!dwords)
>> - return -EINVAL;
>> -
>> - new_rd_idx = rd_idx + dwords;
>> - if (((dwords << 2) <= IFACEQ_VAR_HUGE_PKT_SIZE) && rd_idx <= qsize) {
>> - if (new_rd_idx < qsize) {
>> - memcpy(pkt, rd_ptr, dwords << 2);
>> - } else {
>> - size_t len;
>> -
>> - new_rd_idx -= qsize;
>> - len = (dwords - new_rd_idx) << 2;
>> - memcpy(pkt, rd_ptr, len);
>> - memcpy(pkt + len, queue->qmem.kva, new_rd_idx << 2);
>> - }
>> - } else {
>> - /* bad packet received, dropping */
>> - new_rd_idx = qhdr->write_idx;
>> - ret = -EBADMSG;
>> - }
>> -
>> - /* ensure the packet is read before updating read index */
>> - rmb();
>> -
>> - qhdr->read_idx = new_rd_idx;
>> - /* ensure updating read index */
>> - wmb();
>> -
>> - rd_idx = qhdr->read_idx;
>> - wr_idx = qhdr->write_idx;
>> - /* ensure rd/wr indices are read from memory */
>> - rmb();
>> -
>> - if (rd_idx != wr_idx)
>> - qhdr->rx_req = 0;
>> - else
>> - qhdr->rx_req = recv_request;
>> -
>> - *tx_req = qhdr->tx_req ? 1 : 0;
>> -
>> - /* ensure rx_req is stored to memory and tx_req is loaded from
>> memory */
>> - mb();
>> -
>> - venus_dump_packet(hdev, pkt);
>> -
>> - return ret;
>> -}
>> -
>> -static int venus_alloc(struct venus_hfi_device *hdev, struct mem_desc
>> *desc,
>> - u32 size)
>> -{
>> - struct device *dev = hdev->core->dev;
>> -
>> - desc->attrs = DMA_ATTR_WRITE_COMBINE;
>> - desc->size = ALIGN(size, SZ_4K);
>> -
>> - desc->kva = dma_alloc_attrs(dev, desc->size, &desc->da, GFP_KERNEL,
>> - desc->attrs);
>> - if (!desc->kva)
>> - return -ENOMEM;
>> -
>> - return 0;
>> -}
>> -
>> -static void venus_free(struct venus_hfi_device *hdev, struct mem_desc *mem)
>> -{
>> - struct device *dev = hdev->core->dev;
>> -
>> - dma_free_attrs(dev, mem->size, mem->kva, mem->da, mem->attrs);
>> -}
>> -
>> static void venus_set_registers(struct venus_hfi_device *hdev)
>> {
>> const struct venus_resources *res = hdev->core->res;
>> @@ -380,11 +108,11 @@ static void venus_soft_int(struct venus_hfi_device
>> *hdev)
>> }
>> static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
>> - void *pkt, bool sync)
>> + void *pkt, u32 pkt_size, bool sync)
>
> I'm tempted to say that you should first move the code, then add pkt_size
> later. BTW, why do we need it at all and why wasn't it present beforehand?
>
This was added to make the code generic between venus and iris.
But will check if I can move the code first and then add additional
parameters with iris changes.
>> {
>> struct device *dev = hdev->core->dev;
>> struct hfi_pkt_hdr *cmd_packet;
>> - struct iface_queue *queue;
>> + struct iface_q_info *queue;
>
> iface_queue was good enough, unless you have a reason.
>
Sure, will fix.
>> u32 rx_req;
>> int ret;
>> @@ -396,7 +124,7 @@ static int venus_iface_cmdq_write_nolock(struct
>> venus_hfi_device *hdev,
>> queue = &hdev->queues[IFACEQ_CMD_IDX];
>> - ret = venus_write_queue(hdev, queue, pkt, &rx_req);
>> + ret = write_queue(queue, pkt, pkt_size, &rx_req);
>
> You know, you can call venus_write_queue from your Iris driver, the kernel
> won't bite you. But the generic write_queue() function will be bitten and
> frowned upon.
>
We would prefer to keep the common APIs like this in common vcodec folder,
with proper prefix for sure and use that in both venus and iris driver.
>> if (ret) {
>> dev_err(dev, "write to iface cmd queue failed (%d)\n", ret);
>> return ret;
>> @@ -408,7 +136,7 @@ static int venus_iface_cmdq_write_nolock(struct
>> venus_hfi_device *hdev,
>> * commands
>> */
>> queue = &hdev->queues[IFACEQ_MSG_IDX];
>> - queue->qhdr->rx_req = 1;
>> + queue->qhdr->qhdr_rx_req = 1;
>> /* ensure rx_req is updated in memory */
>> wmb();
>> }
>> @@ -419,12 +147,12 @@ static int venus_iface_cmdq_write_nolock(struct
>> venus_hfi_device *hdev,
>> return 0;
>> }
>> -static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void
>> *pkt, bool sync)
>> +static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void
>> *pkt, u32 pkt_size, bool sync)
>> {
>> int ret;
>> mutex_lock(&hdev->lock);
>> - ret = venus_iface_cmdq_write_nolock(hdev, pkt, sync);
>> + ret = venus_iface_cmdq_write_nolock(hdev, pkt, pkt_size, sync);
>> mutex_unlock(&hdev->lock);
>> return ret;
>> @@ -447,7 +175,7 @@ static int venus_hfi_core_set_resource(struct
>> venus_core *core, u32 id,
>> if (ret)
>> return ret;
>> - ret = venus_iface_cmdq_write(hdev, pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
>> if (ret)
>> return ret;
>> @@ -531,12 +259,12 @@ static int venus_run(struct venus_hfi_device *hdev)
>> */
>> venus_set_registers(hdev);
>> - writel(hdev->ifaceq_table.da, cpu_cs_base + UC_REGION_ADDR);
>> + writel(hdev->ifaceq_table.device_addr, cpu_cs_base + UC_REGION_ADDR);
>> writel(SHARED_QSIZE, cpu_cs_base + UC_REGION_SIZE);
>> - writel(hdev->ifaceq_table.da, cpu_cs_base + CPU_CS_SCIACMDARG2);
>> + writel(hdev->ifaceq_table.device_addr, cpu_cs_base +
>> CPU_CS_SCIACMDARG2);
>
> This is a separate patch material.
>
Sure, will try to separate these changes.
>> writel(0x01, cpu_cs_base + CPU_CS_SCIACMDARG1);
>> - if (hdev->sfr.da)
>> - writel(hdev->sfr.da, cpu_cs_base + SFR_ADDR);
>> + if (hdev->sfr.device_addr)
>> + writel(hdev->sfr.device_addr, cpu_cs_base + SFR_ADDR);
>> ret = venus_boot_core(hdev);
>> if (ret) {
>> @@ -690,7 +418,7 @@ static int venus_power_on(struct venus_hfi_device *hdev)
>> static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev,
>> void *pkt)
>> {
>> - struct iface_queue *queue;
>> + struct iface_q_info *queue;
>> u32 tx_req;
>> int ret;
>> @@ -699,7 +427,7 @@ static int venus_iface_msgq_read_nolock(struct
>> venus_hfi_device *hdev,
>> queue = &hdev->queues[IFACEQ_MSG_IDX];
>> - ret = venus_read_queue(hdev, queue, pkt, &tx_req);
>> + ret = read_queue(queue, pkt, &tx_req);
>> if (ret)
>> return ret;
>> @@ -723,7 +451,7 @@ static int venus_iface_msgq_read(struct
>> venus_hfi_device *hdev, void *pkt)
>> static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev,
>> void *pkt)
>> {
>> - struct iface_queue *queue;
>> + struct iface_q_info *queue;
>> u32 tx_req;
>> int ret;
>> @@ -733,7 +461,7 @@ static int venus_iface_dbgq_read_nolock(struct
>> venus_hfi_device *hdev,
>> queue = &hdev->queues[IFACEQ_DBG_IDX];
>> - ret = venus_read_queue(hdev, queue, pkt, &tx_req);
>> + ret = read_queue(queue, pkt, &tx_req);
>> if (ret)
>> return ret;
>> @@ -759,100 +487,18 @@ static int venus_iface_dbgq_read(struct
>> venus_hfi_device *hdev, void *pkt)
>> static void venus_set_qhdr_defaults(struct hfi_queue_header *qhdr)
>> {
>> - qhdr->status = 1;
>> - qhdr->type = IFACEQ_DFLT_QHDR;
>> - qhdr->q_size = IFACEQ_QUEUE_SIZE / 4;
>> - qhdr->pkt_size = 0;
>> - qhdr->rx_wm = 1;
>> - qhdr->tx_wm = 1;
>> - qhdr->rx_req = 1;
>> - qhdr->tx_req = 0;
>> - qhdr->rx_irq_status = 0;
>> - qhdr->tx_irq_status = 0;
>> - qhdr->read_idx = 0;
>> - qhdr->write_idx = 0;
>> -}
>> -
>> -static void venus_interface_queues_release(struct venus_hfi_device *hdev)
>> -{
>> - mutex_lock(&hdev->lock);
>> -
>> - venus_free(hdev, &hdev->ifaceq_table);
>> - venus_free(hdev, &hdev->sfr);
>> -
>> - memset(hdev->queues, 0, sizeof(hdev->queues));
>> - memset(&hdev->ifaceq_table, 0, sizeof(hdev->ifaceq_table));
>> - memset(&hdev->sfr, 0, sizeof(hdev->sfr));
>> -
>> - mutex_unlock(&hdev->lock);
>> -}
>> -
>> -static int venus_interface_queues_init(struct venus_hfi_device *hdev)
>> -{
>> - struct hfi_queue_table_header *tbl_hdr;
>> - struct iface_queue *queue;
>> - struct hfi_sfr *sfr;
>> - struct mem_desc desc = {0};
>> - unsigned int offset;
>> - unsigned int i;
>> - int ret;
>> -
>> - ret = venus_alloc(hdev, &desc, ALIGNED_QUEUE_SIZE);
>> - if (ret)
>> - return ret;
>> -
>> - hdev->ifaceq_table = desc;
>> - offset = IFACEQ_TABLE_SIZE;
>> -
>> - for (i = 0; i < IFACEQ_NUM; i++) {
>> - queue = &hdev->queues[i];
>> - queue->qmem.da = desc.da + offset;
>> - queue->qmem.kva = desc.kva + offset;
>> - queue->qmem.size = IFACEQ_QUEUE_SIZE;
>> - offset += queue->qmem.size;
>> - queue->qhdr =
>> - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
>> -
>> - venus_set_qhdr_defaults(queue->qhdr);
>> -
>> - queue->qhdr->start_addr = queue->qmem.da;
>> -
>> - if (i == IFACEQ_CMD_IDX)
>> - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
>> - else if (i == IFACEQ_MSG_IDX)
>> - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
>> - else if (i == IFACEQ_DBG_IDX)
>> - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
>> - }
>> -
>> - tbl_hdr = hdev->ifaceq_table.kva;
>> - tbl_hdr->version = 0;
>> - tbl_hdr->size = IFACEQ_TABLE_SIZE;
>> - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
>> - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
>> - tbl_hdr->num_q = IFACEQ_NUM;
>> - tbl_hdr->num_active_q = IFACEQ_NUM;
>> -
>> - /*
>> - * Set receive request to zero on debug queue as there is no
>> - * need of interrupt from video hardware for debug messages
>> - */
>> - queue = &hdev->queues[IFACEQ_DBG_IDX];
>> - queue->qhdr->rx_req = 0;
>> -
>> - ret = venus_alloc(hdev, &desc, ALIGNED_SFR_SIZE);
>> - if (ret) {
>> - hdev->sfr.da = 0;
>> - } else {
>> - hdev->sfr = desc;
>> - sfr = hdev->sfr.kva;
>> - sfr->buf_size = ALIGNED_SFR_SIZE;
>> - }
>> -
>> - /* ensure table and queue header structs are settled in memory */
>> - wmb();
>> -
>> - return 0;
>> + qhdr->qhdr_status = 1;
>> + qhdr->qhdr_type = IFACEQ_DFLT_QHDR;
>> + qhdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4;
>> + qhdr->qhdr_pkt_size = 0;
>> + qhdr->qhdr_rx_wm = 1;
>> + qhdr->qhdr_tx_wm = 1;
>> + qhdr->qhdr_rx_req = 1;
>> + qhdr->qhdr_tx_req = 0;
>> + qhdr->qhdr_rx_irq_status = 0;
>> + qhdr->qhdr_tx_irq_status = 0;
>> + qhdr->qhdr_read_idx = 0;
>> + qhdr->qhdr_write_idx = 0;
>> }
>> static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug)
>> @@ -864,7 +510,7 @@ static int venus_sys_set_debug(struct
>> venus_hfi_device *hdev, u32 debug)
>> pkt_sys_debug_config(pkt, HFI_DEBUG_MODE_QUEUE, debug);
>> - return venus_iface_cmdq_write(hdev, pkt, false);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
>> }
>> static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32
>> mode)
>> @@ -876,7 +522,7 @@ static int venus_sys_set_coverage(struct
>> venus_hfi_device *hdev, u32 mode)
>> pkt_sys_coverage_config(pkt, mode);
>> - return venus_iface_cmdq_write(hdev, pkt, false);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
>> }
>> static int venus_sys_set_idle_message(struct venus_hfi_device *hdev,
>> @@ -892,7 +538,7 @@ static int venus_sys_set_idle_message(struct
>> venus_hfi_device *hdev,
>> pkt_sys_idle_indicator(pkt, enable);
>> - return venus_iface_cmdq_write(hdev, pkt, false);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
>> }
>> static int venus_sys_set_power_control(struct venus_hfi_device *hdev,
>> @@ -905,7 +551,7 @@ static int venus_sys_set_power_control(struct
>> venus_hfi_device *hdev,
>> pkt_sys_power_control(pkt, enable);
>> - return venus_iface_cmdq_write(hdev, pkt, false);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
>> }
>> static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev)
>> @@ -919,7 +565,7 @@ static int venus_sys_set_ubwc_config(struct
>> venus_hfi_device *hdev)
>> pkt_sys_ubwc_config(pkt, res->ubwc_conf);
>> - ret = venus_iface_cmdq_write(hdev, pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false);
>> if (ret)
>> return ret;
>> @@ -938,7 +584,7 @@ static int venus_get_queue_size(struct
>> venus_hfi_device *hdev,
>> if (!qhdr)
>> return -EINVAL;
>> - return abs(qhdr->read_idx - qhdr->write_idx);
>> + return abs(qhdr->qhdr_read_idx - qhdr->qhdr_write_idx);
>> }
>> static int venus_sys_set_default_properties(struct venus_hfi_device
>> *hdev)
>> @@ -980,7 +626,7 @@ static int venus_session_cmd(struct venus_inst *inst,
>> u32 pkt_type, bool sync)
>> pkt_session_cmd(&pkt, pkt_type, inst);
>> - return venus_iface_cmdq_write(hdev, &pkt, sync);
>> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, sync);
>> }
>> static void venus_flush_debug_queue(struct venus_hfi_device *hdev)
>> @@ -1010,7 +656,7 @@ static int venus_prepare_power_collapse(struct
>> venus_hfi_device *hdev,
>> pkt_sys_pc_prep(&pkt);
>> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
>> if (ret)
>> return ret;
>> @@ -1047,7 +693,7 @@ static int venus_are_queues_empty(struct
>> venus_hfi_device *hdev)
>> static void venus_sfr_print(struct venus_hfi_device *hdev)
>> {
>> struct device *dev = hdev->core->dev;
>> - struct hfi_sfr *sfr = hdev->sfr.kva;
>> + struct hfi_sfr *sfr = hdev->sfr.kernel_vaddr;
>> void *p;
>> if (!sfr)
>> @@ -1163,13 +809,13 @@ static int venus_core_init(struct venus_core *core)
>> venus_set_state(hdev, VENUS_STATE_INIT);
>> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
>> if (ret)
>> return ret;
>> pkt_sys_image_version(&version_pkt);
>> - ret = venus_iface_cmdq_write(hdev, &version_pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, &version_pkt,
>> version_pkt.hdr.size, false);
>> if (ret)
>> dev_warn(dev, "failed to send image version pkt to fw\n");
>> @@ -1198,7 +844,7 @@ static int venus_core_ping(struct venus_core
>> *core, u32 cookie)
>> pkt_sys_ping(&pkt, cookie);
>> - return venus_iface_cmdq_write(hdev, &pkt, false);
>> + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
>> }
>> static int venus_core_trigger_ssr(struct venus_core *core, u32
>> trigger_type)
>> @@ -1211,7 +857,7 @@ static int venus_core_trigger_ssr(struct venus_core
>> *core, u32 trigger_type)
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, &pkt, false);
>> + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false);
>> }
>> static int venus_session_init(struct venus_inst *inst, u32 session_type,
>> @@ -1229,7 +875,7 @@ static int venus_session_init(struct venus_inst
>> *inst, u32 session_type,
>> if (ret)
>> goto err;
>> - ret = venus_iface_cmdq_write(hdev, &pkt, true);
>> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
>> if (ret)
>> goto err;
>> @@ -1272,7 +918,7 @@ static int venus_session_flush(struct venus_inst
>> *inst, u32 flush_mode)
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, &pkt, true);
>> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
>> }
>> static int venus_session_start(struct venus_inst *inst)
>> @@ -1304,7 +950,7 @@ static int venus_session_etb(struct venus_inst *inst,
>> if (ret)
>> return ret;
>> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
>> } else if (session_type == VIDC_SESSION_TYPE_ENC) {
>> struct hfi_session_empty_buffer_uncompressed_plane0_pkt pkt;
>> @@ -1312,7 +958,7 @@ static int venus_session_etb(struct venus_inst *inst,
>> if (ret)
>> return ret;
>> - ret = venus_iface_cmdq_write(hdev, &pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
>> } else {
>> ret = -EINVAL;
>> }
>> @@ -1331,7 +977,7 @@ static int venus_session_ftb(struct venus_inst *inst,
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, &pkt, false);
>> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false);
>> }
>> static int venus_session_set_buffers(struct venus_inst *inst,
>> @@ -1351,7 +997,7 @@ static int venus_session_set_buffers(struct
>> venus_inst *inst,
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, pkt, false);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
>> }
>> static int venus_session_unset_buffers(struct venus_inst *inst,
>> @@ -1371,7 +1017,7 @@ static int venus_session_unset_buffers(struct
>> venus_inst *inst,
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, pkt, true);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, true);
>> }
>> static int venus_session_load_res(struct venus_inst *inst)
>> @@ -1398,7 +1044,7 @@ static int venus_session_parse_seq_hdr(struct
>> venus_inst *inst, u32 seq_hdr,
>> if (ret)
>> return ret;
>> - ret = venus_iface_cmdq_write(hdev, pkt, false);
>> + ret = venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
>> if (ret)
>> return ret;
>> @@ -1419,7 +1065,7 @@ static int venus_session_get_seq_hdr(struct
>> venus_inst *inst, u32 seq_hdr,
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, pkt, false);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
>> }
>> static int venus_session_set_property(struct venus_inst *inst, u32
>> ptype,
>> @@ -1438,7 +1084,7 @@ static int venus_session_set_property(struct
>> venus_inst *inst, u32 ptype,
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, pkt, false);
>> + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false);
>> }
>> static int venus_session_get_property(struct venus_inst *inst, u32
>> ptype)
>> @@ -1451,7 +1097,7 @@ static int venus_session_get_property(struct
>> venus_inst *inst, u32 ptype)
>> if (ret)
>> return ret;
>> - return venus_iface_cmdq_write(hdev, &pkt, true);
>> + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true);
>> }
>> static int venus_resume(struct venus_core *core)
>> @@ -1685,7 +1331,11 @@ void venus_hfi_destroy(struct venus_core *core)
>> struct venus_hfi_device *hdev = to_hfi_priv(core);
>> core->priv = NULL;
>> - venus_interface_queues_release(hdev);
>> + mutex_lock(&hdev->lock);
>> + hfi_queue_deinit(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr,
>> + &hdev->queues[0], &hdev->queues[1],
>> + &hdev->queues[2]);
>> + mutex_unlock(&hdev->lock);
>> mutex_destroy(&hdev->lock);
>> kfree(hdev);
>> core->ops = NULL;
>> @@ -1707,7 +1357,9 @@ int venus_hfi_create(struct venus_core *core)
>> core->priv = hdev;
>> core->ops = &venus_hfi_ops;
>> - ret = venus_interface_queues_init(hdev);
>> + ret = hfi_queue_init(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr,
>> + &hdev->queues[0], &hdev->queues[1],
>> + &hdev->queues[2], &hdev->core);
>> if (ret)
>> goto err_kfree;
>> @@ -1724,7 +1376,7 @@ void venus_hfi_queues_reinit(struct venus_core
>> *core)
>> {
>> struct venus_hfi_device *hdev = to_hfi_priv(core);
>> struct hfi_queue_table_header *tbl_hdr;
>> - struct iface_queue *queue;
>> + struct iface_q_info *queue;
>> struct hfi_sfr *sfr;
>> unsigned int i;
>> @@ -1733,36 +1385,36 @@ void venus_hfi_queues_reinit(struct venus_core
>> *core)
>> for (i = 0; i < IFACEQ_NUM; i++) {
>> queue = &hdev->queues[i];
>> queue->qhdr =
>> - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
>> + IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kernel_vaddr, i);
>> venus_set_qhdr_defaults(queue->qhdr);
>> - queue->qhdr->start_addr = queue->qmem.da;
>> + queue->qhdr->qhdr_start_addr = queue->q_array.device_addr;
>> if (i == IFACEQ_CMD_IDX)
>> - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
>> + queue->qhdr->qhdr_type |= HFI_HOST_TO_CTRL_CMD_Q;
>> else if (i == IFACEQ_MSG_IDX)
>> - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
>> + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_MSG_Q;
>> else if (i == IFACEQ_DBG_IDX)
>> - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
>> + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_DBG_Q;
>> }
>> - tbl_hdr = hdev->ifaceq_table.kva;
>> - tbl_hdr->version = 0;
>> - tbl_hdr->size = IFACEQ_TABLE_SIZE;
>> - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
>> - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
>> - tbl_hdr->num_q = IFACEQ_NUM;
>> - tbl_hdr->num_active_q = IFACEQ_NUM;
>> + tbl_hdr = hdev->ifaceq_table.kernel_vaddr;
>> + tbl_hdr->qtbl_version = 0;
>> + tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE;
>> + tbl_hdr->qtbl_qhdr0_offset = sizeof(struct hfi_queue_table_header);
>> + tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header);
>> + tbl_hdr->qtbl_num_q = IFACEQ_NUM;
>> + tbl_hdr->qtbl_num_active_q = IFACEQ_NUM;
>
> NAK. This is not 'an introduction of helpers'.
Sure, will fix this.
>
>> /*
>> * Set receive request to zero on debug queue as there is no
>> * need of interrupt from video hardware for debug messages
>> */
>> queue = &hdev->queues[IFACEQ_DBG_IDX];
>> - queue->qhdr->rx_req = 0;
>> + queue->qhdr->qhdr_rx_req = 0;
>> - sfr = hdev->sfr.kva;
>> + sfr = hdev->sfr.kernel_vaddr;
>> sfr->buf_size = ALIGNED_SFR_SIZE;
>> /* ensure table and queue header structs are settled in memory */
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 03/34] media: introduce common helpers for buffer size calculation
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
2023-12-18 11:31 ` [PATCH v2 01/34] media: introduce common helpers for video firmware handling Dikshita Agarwal
2023-12-18 11:31 ` [PATCH v2 02/34] media: introduce common helpers for queues handling Dikshita Agarwal
@ 2023-12-18 11:31 ` Dikshita Agarwal
2023-12-18 18:32 ` Dmitry Baryshkov
2023-12-18 11:31 ` [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema Dikshita Agarwal
` (37 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:31 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Introduce helper to calculate size of pixel buffer and
use in venus driver.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/buffers.c | 103 ++++++++++++
drivers/media/platform/qcom/vcodec/buffers.h | 15 ++
drivers/media/platform/qcom/vcodec/venus/Makefile | 4 +-
drivers/media/platform/qcom/vcodec/venus/helpers.c | 172 ++++-----------------
drivers/media/platform/qcom/vcodec/venus/helpers.h | 2 +-
.../platform/qcom/vcodec/venus/hfi_plat_bufs.h | 4 +-
.../platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c | 10 +-
drivers/media/platform/qcom/vcodec/venus/vdec.c | 5 +-
8 files changed, 157 insertions(+), 158 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
diff --git a/drivers/media/platform/qcom/vcodec/buffers.c b/drivers/media/platform/qcom/vcodec/buffers.c
new file mode 100644
index 0000000..58ab3b0
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/buffers.c
@@ -0,0 +1,103 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#include <linux/videodev2.h>
+
+#include "buffers.h"
+
+u32 video_raw_buffer_size(u32 colorformat,
+ u32 pix_width,
+ u32 pix_height)
+{
+ u32 size = 0;
+ u32 y_plane, uv_plane, y_stride,
+ uv_stride, y_sclines, uv_sclines;
+ u32 y_ubwc_plane = 0, uv_ubwc_plane = 0;
+ u32 y_meta_stride = 0, y_meta_scanlines = 0;
+ u32 uv_meta_stride = 0, uv_meta_scanlines = 0;
+ u32 y_meta_plane = 0, uv_meta_plane = 0;
+
+ if (!pix_width || !pix_height)
+ goto invalid_input;
+
+ switch (colorformat) {
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV21:
+ y_stride = ALIGN(pix_width, 128);
+ uv_stride = ALIGN(pix_width, 128);
+ y_sclines = ALIGN(pix_height, 32);
+ uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
+ y_plane = y_stride * y_sclines;
+ uv_plane = uv_stride * uv_sclines;
+ size = y_plane + uv_plane;
+ break;
+ case V4L2_PIX_FMT_QC08C:
+ y_stride = ALIGN(pix_width, 128);
+ uv_stride = ALIGN(pix_width, 128);
+ y_sclines = ALIGN(pix_height, 32);
+ uv_sclines = ALIGN((pix_height + 1) >> 1, 32);
+ y_meta_stride = ALIGN(DIV_ROUND_UP(pix_width, 32), 64);
+ uv_meta_stride = ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 16), 64);
+ y_ubwc_plane =
+ ALIGN(y_stride * y_sclines, 4096);
+ uv_ubwc_plane =
+ ALIGN(uv_stride * uv_sclines, 4096);
+ y_meta_scanlines =
+ ALIGN(DIV_ROUND_UP(pix_height, 8), 16);
+ y_meta_plane =
+ ALIGN(y_meta_stride * y_meta_scanlines, 4096);
+ uv_meta_scanlines =
+ ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 8), 16);
+ uv_meta_plane =
+ ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
+ size = (y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
+ uv_meta_plane);
+ break;
+ case V4L2_PIX_FMT_QC10C:
+ y_stride =
+ ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
+ uv_stride =
+ ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
+ y_sclines =
+ ALIGN(pix_height, 16);
+ uv_sclines =
+ ALIGN((pix_height + 1) >> 1, 16);
+ y_ubwc_plane =
+ ALIGN(y_stride * y_sclines, 4096);
+ uv_ubwc_plane =
+ ALIGN(uv_stride * uv_sclines, 4096);
+ y_meta_stride =
+ ALIGN(DIV_ROUND_UP(pix_width, 48), 64);
+ y_meta_scanlines =
+ ALIGN(DIV_ROUND_UP(pix_height, 4), 16);
+ y_meta_plane =
+ ALIGN(y_meta_stride * y_meta_scanlines, 4096);
+ uv_meta_stride =
+ ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 24), 64);
+ uv_meta_scanlines =
+ ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 4), 16);
+ uv_meta_plane =
+ ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
+
+ size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
+ uv_meta_plane;
+ break;
+ case V4L2_PIX_FMT_P010:
+ y_stride = ALIGN(pix_width * 2, 128);
+ uv_stride = ALIGN(pix_width * 2, 128);
+ y_sclines = ALIGN(pix_height, 32);
+ uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
+ y_plane = y_stride * y_sclines;
+ uv_plane = uv_stride * uv_sclines;
+
+ size = y_plane + uv_plane;
+ break;
+ default:
+ break;
+ }
+
+invalid_input:
+
+ return ALIGN(size, 4096);
+}
diff --git a/drivers/media/platform/qcom/vcodec/buffers.h b/drivers/media/platform/qcom/vcodec/buffers.h
new file mode 100644
index 0000000..ac1d052
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/buffers.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _BUFFERS_H_
+#define _BUFFERS_H_
+
+#include <linux/types.h>
+
+u32 video_raw_buffer_size(u32 colorformat,
+ u32 pix_width,
+ u32 pix_height);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
index 1941ef4..6abd54a 100644
--- a/drivers/media/platform/qcom/vcodec/venus/Makefile
+++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
@@ -1,8 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
# Makefile for Qualcomm Venus driver
-venus-core-objs += ../firmware.o \
- ../hfi_queue.o
+venus-core-objs += ../firmware.o ../hfi_queue.o ../buffers.o
venus-core-objs += core.o helpers.o firmware_no_tz.o \
hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
@@ -10,6 +9,7 @@ venus-core-objs += core.o helpers.o firmware_no_tz.o \
hfi_platform.o hfi_platform_v4.o \
hfi_platform_v6.o hfi_plat_bufs_v6.o \
+venus-dec-objs += ../buffers.o
venus-dec-objs += vdec.o vdec_ctrls.o
venus-enc-objs += venc.o venc_ctrls.o
diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c
index 8295542..95e4424 100644
--- a/drivers/media/platform/qcom/vcodec/venus/helpers.c
+++ b/drivers/media/platform/qcom/vcodec/venus/helpers.c
@@ -12,6 +12,7 @@
#include <media/v4l2-mem2mem.h>
#include <asm/div64.h>
+#include "../buffers.h"
#include "core.h"
#include "helpers.h"
#include "hfi_helper.h"
@@ -616,6 +617,27 @@ static u32 to_hfi_raw_fmt(u32 v4l2_fmt)
return 0;
}
+u32 to_v4l2_raw_fmt(u32 hfi_color_fmt)
+{
+ switch (hfi_color_fmt) {
+ case HFI_COLOR_FORMAT_NV12:
+ return V4L2_PIX_FMT_NV12;
+ case HFI_COLOR_FORMAT_NV21:
+ return V4L2_PIX_FMT_NV21;
+ case HFI_COLOR_FORMAT_NV12_UBWC:
+ return V4L2_PIX_FMT_QC08C;
+ case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
+ return V4L2_PIX_FMT_QC10C;
+ case HFI_COLOR_FORMAT_P010:
+ return V4L2_PIX_FMT_P010;
+ default:
+ break;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(to_v4l2_raw_fmt);
+
static int platform_get_bufreq(struct venus_inst *inst, u32 buftype,
struct hfi_buffer_requirements *req)
{
@@ -639,20 +661,20 @@ static int platform_get_bufreq(struct venus_inst *inst, u32 buftype,
params.out_width = inst->out_width;
params.out_height = inst->out_height;
params.codec = inst->fmt_out->pixfmt;
- params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_cap->pixfmt);
+ params.color_fmt = inst->fmt_cap->pixfmt;
params.dec.max_mbs_per_frame = mbs_per_frame_max(inst);
params.dec.buffer_size_limit = 0;
params.dec.is_secondary_output =
inst->opb_buftype == HFI_BUFFER_OUTPUT2;
if (params.dec.is_secondary_output)
- params.hfi_dpb_color_fmt = inst->dpb_fmt;
+ params.dpb_color_fmt = to_v4l2_raw_fmt(inst->dpb_fmt);
params.dec.is_interlaced =
inst->pic_struct != HFI_INTERLACE_FRAME_PROGRESSIVE;
} else {
params.width = inst->out_width;
params.height = inst->out_height;
params.codec = inst->fmt_cap->pixfmt;
- params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_out->pixfmt);
+ params.color_fmt = inst->fmt_out->pixfmt;
params.enc.work_mode = VIDC_WORK_MODE_2;
params.enc.rc_type = HFI_RATE_CONTROL_OFF;
if (enc_ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ)
@@ -942,146 +964,10 @@ int venus_helper_set_profile_level(struct venus_inst *inst, u32 profile, u32 lev
}
EXPORT_SYMBOL_GPL(venus_helper_set_profile_level);
-static u32 get_framesize_raw_nv12(u32 width, u32 height)
-{
- u32 y_stride, uv_stride, y_plane;
- u32 y_sclines, uv_sclines, uv_plane;
- u32 size;
-
- y_stride = ALIGN(width, 128);
- uv_stride = ALIGN(width, 128);
- y_sclines = ALIGN(height, 32);
- uv_sclines = ALIGN(((height + 1) >> 1), 16);
-
- y_plane = y_stride * y_sclines;
- uv_plane = uv_stride * uv_sclines + SZ_4K;
- size = y_plane + uv_plane + SZ_8K;
-
- return ALIGN(size, SZ_4K);
-}
-
-static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height)
-{
- u32 y_meta_stride, y_meta_plane;
- u32 y_stride, y_plane;
- u32 uv_meta_stride, uv_meta_plane;
- u32 uv_stride, uv_plane;
- u32 extradata = SZ_16K;
-
- y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
- y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(height, 8), 16);
- y_meta_plane = ALIGN(y_meta_plane, SZ_4K);
-
- y_stride = ALIGN(width, 128);
- y_plane = ALIGN(y_stride * ALIGN(height, 32), SZ_4K);
-
- uv_meta_stride = ALIGN(DIV_ROUND_UP(width / 2, 16), 64);
- uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(height / 2, 8), 16);
- uv_meta_plane = ALIGN(uv_meta_plane, SZ_4K);
-
- uv_stride = ALIGN(width, 128);
- uv_plane = ALIGN(uv_stride * ALIGN(height / 2, 32), SZ_4K);
-
- return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane +
- max(extradata, y_stride * 48), SZ_4K);
-}
-
-static u32 get_framesize_raw_p010(u32 width, u32 height)
-{
- u32 y_plane, uv_plane, y_stride, uv_stride, y_sclines, uv_sclines;
-
- y_stride = ALIGN(width * 2, 128);
- uv_stride = ALIGN(width * 2, 128);
- y_sclines = ALIGN(height, 32);
- uv_sclines = ALIGN((height + 1) >> 1, 16);
- y_plane = y_stride * y_sclines;
- uv_plane = uv_stride * uv_sclines;
-
- return ALIGN((y_plane + uv_plane), SZ_4K);
-}
-
-static u32 get_framesize_raw_p010_ubwc(u32 width, u32 height)
-{
- u32 y_stride, uv_stride, y_sclines, uv_sclines;
- u32 y_ubwc_plane, uv_ubwc_plane;
- u32 y_meta_stride, y_meta_scanlines;
- u32 uv_meta_stride, uv_meta_scanlines;
- u32 y_meta_plane, uv_meta_plane;
- u32 size;
-
- y_stride = ALIGN(width * 2, 256);
- uv_stride = ALIGN(width * 2, 256);
- y_sclines = ALIGN(height, 16);
- uv_sclines = ALIGN((height + 1) >> 1, 16);
-
- y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
- uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
- y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
- y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
- y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
- uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64);
- uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
- uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
-
- size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
-
- return ALIGN(size, SZ_4K);
-}
-
-static u32 get_framesize_raw_yuv420_tp10_ubwc(u32 width, u32 height)
-{
- u32 y_stride, uv_stride, y_sclines, uv_sclines;
- u32 y_ubwc_plane, uv_ubwc_plane;
- u32 y_meta_stride, y_meta_scanlines;
- u32 uv_meta_stride, uv_meta_scanlines;
- u32 y_meta_plane, uv_meta_plane;
- u32 extradata = SZ_16K;
- u32 size;
-
- y_stride = ALIGN(width * 4 / 3, 256);
- uv_stride = ALIGN(width * 4 / 3, 256);
- y_sclines = ALIGN(height, 16);
- uv_sclines = ALIGN((height + 1) >> 1, 16);
-
- y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
- uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
- y_meta_stride = ALIGN(DIV_ROUND_UP(width, 48), 64);
- y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
- y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
- uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64);
- uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
- uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
-
- size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
- size += max(extradata + SZ_8K, y_stride * 48);
-
- return ALIGN(size, SZ_4K);
-}
-
-u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height)
-{
- switch (hfi_fmt) {
- case HFI_COLOR_FORMAT_NV12:
- case HFI_COLOR_FORMAT_NV21:
- return get_framesize_raw_nv12(width, height);
- case HFI_COLOR_FORMAT_NV12_UBWC:
- return get_framesize_raw_nv12_ubwc(width, height);
- case HFI_COLOR_FORMAT_P010:
- return get_framesize_raw_p010(width, height);
- case HFI_COLOR_FORMAT_P010_UBWC:
- return get_framesize_raw_p010_ubwc(width, height);
- case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
- return get_framesize_raw_yuv420_tp10_ubwc(width, height);
- default:
- return 0;
- }
-}
-EXPORT_SYMBOL_GPL(venus_helper_get_framesz_raw);
-
u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
{
- u32 hfi_fmt, sz;
bool compressed;
+ u32 sz;
switch (v4l2_fmt) {
case V4L2_PIX_FMT_MPEG:
@@ -1112,11 +998,7 @@ u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
return ALIGN(sz, SZ_4K);
}
- hfi_fmt = to_hfi_raw_fmt(v4l2_fmt);
- if (!hfi_fmt)
- return 0;
-
- return venus_helper_get_framesz_raw(hfi_fmt, width, height);
+ return video_raw_buffer_size(v4l2_fmt, width, height);
}
EXPORT_SYMBOL_GPL(venus_helper_get_framesz);
diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h
index 358e4f3..9b72d18 100644
--- a/drivers/media/platform/qcom/vcodec/venus/helpers.h
+++ b/drivers/media/platform/qcom/vcodec/venus/helpers.h
@@ -11,6 +11,7 @@
struct venus_inst;
struct venus_core;
+u32 to_v4l2_raw_fmt(u32 hfi_color_fmt);
bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt);
struct vb2_v4l2_buffer *venus_helper_find_buf(struct venus_inst *inst,
unsigned int type, u32 idx);
@@ -29,7 +30,6 @@ void venus_helper_m2m_device_run(void *priv);
void venus_helper_m2m_job_abort(void *priv);
int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
struct hfi_buffer_requirements *req);
-u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height);
u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height);
int venus_helper_set_input_resolution(struct venus_inst *inst,
unsigned int width, unsigned int height);
diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
index 25e6074..20f684e 100644
--- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
+++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
@@ -15,8 +15,8 @@ struct hfi_plat_buffers_params {
u32 out_width;
u32 out_height;
u32 codec;
- u32 hfi_color_fmt;
- u32 hfi_dpb_color_fmt;
+ u32 color_fmt;
+ u32 dpb_color_fmt;
enum hfi_version version;
u32 num_vpp_pipes;
union {
diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
index f5a6559..3e06516 100644
--- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
+++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
@@ -6,6 +6,7 @@
#include <linux/sizes.h>
#include <linux/videodev2.h>
+#include "../buffers.h"
#include "hfi.h"
#include "hfi_plat_bufs.h"
#include "helpers.h"
@@ -1233,13 +1234,11 @@ static int bufreq_dec(struct hfi_plat_buffers_params *params, u32 buftype,
buffer_size_limit);
} else if (buftype == HFI_BUFFER_OUTPUT || buftype == HFI_BUFFER_OUTPUT2) {
hfi_bufreq_set_count_min(bufreq, version, out_min_count);
- bufreq->size =
- venus_helper_get_framesz_raw(params->hfi_color_fmt,
+ bufreq->size = video_raw_buffer_size(params->color_fmt,
out_width, out_height);
if (buftype == HFI_BUFFER_OUTPUT &&
params->dec.is_secondary_output)
- bufreq->size =
- venus_helper_get_framesz_raw(params->hfi_dpb_color_fmt,
+ bufreq->size = video_raw_buffer_size(params->dpb_color_fmt,
out_width, out_height);
} else if (buftype == HFI_BUFFER_INTERNAL_SCRATCH(version)) {
bufreq->size = dec_ops->scratch(width, height, is_interlaced);
@@ -1297,8 +1296,7 @@ static int bufreq_enc(struct hfi_plat_buffers_params *params, u32 buftype,
if (buftype == HFI_BUFFER_INPUT) {
hfi_bufreq_set_count_min(bufreq, version, MIN_INPUT_BUFFERS);
- bufreq->size =
- venus_helper_get_framesz_raw(params->hfi_color_fmt,
+ bufreq->size = video_raw_buffer_size(params->color_fmt,
width, height);
} else if (buftype == HFI_BUFFER_OUTPUT ||
buftype == HFI_BUFFER_OUTPUT2) {
diff --git a/drivers/media/platform/qcom/vcodec/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c
index dbf305c..e6316be 100644
--- a/drivers/media/platform/qcom/vcodec/venus/vdec.c
+++ b/drivers/media/platform/qcom/vcodec/venus/vdec.c
@@ -15,6 +15,7 @@
#include <media/v4l2-mem2mem.h>
#include <media/videobuf2-dma-contig.h>
+#include "../buffers.h"
#include "hfi_venus_io.h"
#include "hfi_parser.h"
#include "core.h"
@@ -777,9 +778,9 @@ static int vdec_output_conf(struct venus_inst *inst)
return ret;
inst->output_buf_size =
- venus_helper_get_framesz_raw(out_fmt, width, height);
+ video_raw_buffer_size(to_v4l2_raw_fmt(out_fmt), width, height);
inst->output2_buf_size =
- venus_helper_get_framesz_raw(out2_fmt, width, height);
+ video_raw_buffer_size(to_v4l2_raw_fmt(out2_fmt), width, height);
if (is_ubwc_fmt(out_fmt)) {
inst->opb_buftype = HFI_BUFFER_OUTPUT2;
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 03/34] media: introduce common helpers for buffer size calculation
2023-12-18 11:31 ` [PATCH v2 03/34] media: introduce common helpers for buffer size calculation Dikshita Agarwal
@ 2023-12-18 18:32 ` Dmitry Baryshkov
2023-12-20 8:04 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:32 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:31, Dikshita Agarwal wrote:
> Introduce helper to calculate size of pixel buffer and
> use in venus driver.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++++++++++++
> drivers/media/platform/qcom/vcodec/buffers.h | 15 ++
> drivers/media/platform/qcom/vcodec/venus/Makefile | 4 +-
> drivers/media/platform/qcom/vcodec/venus/helpers.c | 172 ++++-----------------
> drivers/media/platform/qcom/vcodec/venus/helpers.h | 2 +-
> .../platform/qcom/vcodec/venus/hfi_plat_bufs.h | 4 +-
> .../platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c | 10 +-
> drivers/media/platform/qcom/vcodec/venus/vdec.c | 5 +-
> 8 files changed, 157 insertions(+), 158 deletions(-)
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
>
> diff --git a/drivers/media/platform/qcom/vcodec/buffers.c b/drivers/media/platform/qcom/vcodec/buffers.c
> new file mode 100644
> index 0000000..58ab3b0
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/buffers.c
> @@ -0,0 +1,103 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +#include <linux/videodev2.h>
> +
> +#include "buffers.h"
> +
> +u32 video_raw_buffer_size(u32 colorformat,
Usual drill, non-prefixed name.
> + u32 pix_width,
> + u32 pix_height)
My gut tells me that this is misaligned.
> +{
> + u32 size = 0;
> + u32 y_plane, uv_plane, y_stride,
> + uv_stride, y_sclines, uv_sclines;
> + u32 y_ubwc_plane = 0, uv_ubwc_plane = 0;
> + u32 y_meta_stride = 0, y_meta_scanlines = 0;
> + u32 uv_meta_stride = 0, uv_meta_scanlines = 0;
> + u32 y_meta_plane = 0, uv_meta_plane = 0;
> +
> + if (!pix_width || !pix_height)
> + goto invalid_input;
> +
> + switch (colorformat) {
> + case V4L2_PIX_FMT_NV12:
> + case V4L2_PIX_FMT_NV21:
> + y_stride = ALIGN(pix_width, 128);
> + uv_stride = ALIGN(pix_width, 128);
> + y_sclines = ALIGN(pix_height, 32);
> + uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
> + y_plane = y_stride * y_sclines;
> + uv_plane = uv_stride * uv_sclines;
> + size = y_plane + uv_plane;
> + break;
> + case V4L2_PIX_FMT_QC08C:
> + y_stride = ALIGN(pix_width, 128);
> + uv_stride = ALIGN(pix_width, 128);
> + y_sclines = ALIGN(pix_height, 32);
> + uv_sclines = ALIGN((pix_height + 1) >> 1, 32);
> + y_meta_stride = ALIGN(DIV_ROUND_UP(pix_width, 32), 64);
> + uv_meta_stride = ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 16), 64);
> + y_ubwc_plane =
> + ALIGN(y_stride * y_sclines, 4096);
> + uv_ubwc_plane =
> + ALIGN(uv_stride * uv_sclines, 4096);
> + y_meta_scanlines =
> + ALIGN(DIV_ROUND_UP(pix_height, 8), 16);
> + y_meta_plane =
> + ALIGN(y_meta_stride * y_meta_scanlines, 4096);
> + uv_meta_scanlines =
> + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 8), 16);
> + uv_meta_plane =
> + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
> + size = (y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
> + uv_meta_plane);
> + break;
> + case V4L2_PIX_FMT_QC10C:
> + y_stride =
> + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
> + uv_stride =
> + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
> + y_sclines =
> + ALIGN(pix_height, 16);
> + uv_sclines =
> + ALIGN((pix_height + 1) >> 1, 16);
> + y_ubwc_plane =
> + ALIGN(y_stride * y_sclines, 4096);
> + uv_ubwc_plane =
> + ALIGN(uv_stride * uv_sclines, 4096);
> + y_meta_stride =
> + ALIGN(DIV_ROUND_UP(pix_width, 48), 64);
> + y_meta_scanlines =
> + ALIGN(DIV_ROUND_UP(pix_height, 4), 16);
> + y_meta_plane =
> + ALIGN(y_meta_stride * y_meta_scanlines, 4096);
> + uv_meta_stride =
> + ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 24), 64);
> + uv_meta_scanlines =
> + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 4), 16);
> + uv_meta_plane =
> + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
> +
> + size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
> + uv_meta_plane;
> + break;
> + case V4L2_PIX_FMT_P010:
> + y_stride = ALIGN(pix_width * 2, 128);
> + uv_stride = ALIGN(pix_width * 2, 128);
> + y_sclines = ALIGN(pix_height, 32);
> + uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
> + y_plane = y_stride * y_sclines;
> + uv_plane = uv_stride * uv_sclines;
> +
> + size = y_plane + uv_plane;
> + break;
> + default:
> + break;
> + }
> +
> +invalid_input:
> +
> + return ALIGN(size, 4096);
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/buffers.h b/drivers/media/platform/qcom/vcodec/buffers.h
> new file mode 100644
> index 0000000..ac1d052
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/buffers.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _BUFFERS_H_
> +#define _BUFFERS_H_
> +
> +#include <linux/types.h>
> +
> +u32 video_raw_buffer_size(u32 colorformat,
> + u32 pix_width,
> + u32 pix_height);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
> index 1941ef4..6abd54a 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> @@ -1,8 +1,7 @@
> # SPDX-License-Identifier: GPL-2.0
> # Makefile for Qualcomm Venus driver
>
> -venus-core-objs += ../firmware.o \
> - ../hfi_queue.o
> +venus-core-objs += ../firmware.o ../hfi_queue.o ../buffers.o
Ugh. I missed that in the previous patches. This is not how the helpers
are done. Add normal kernel module instead.
>
> venus-core-objs += core.o helpers.o firmware_no_tz.o \
> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> @@ -10,6 +9,7 @@ venus-core-objs += core.o helpers.o firmware_no_tz.o \
> hfi_platform.o hfi_platform_v4.o \
> hfi_platform_v6.o hfi_plat_bufs_v6.o \
>
> +venus-dec-objs += ../buffers.o
I really wonder, why doesn't this end up with 'symbol defined multiple
times' error.
> venus-dec-objs += vdec.o vdec_ctrls.o
> venus-enc-objs += venc.o venc_ctrls.o
>
> diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c
> index 8295542..95e4424 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/helpers.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.c
> @@ -12,6 +12,7 @@
> #include <media/v4l2-mem2mem.h>
> #include <asm/div64.h>
>
> +#include "../buffers.h"
> #include "core.h"
> #include "helpers.h"
> #include "hfi_helper.h"
> @@ -616,6 +617,27 @@ static u32 to_hfi_raw_fmt(u32 v4l2_fmt)
> return 0;
> }
>
> +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt)
> +{
> + switch (hfi_color_fmt) {
> + case HFI_COLOR_FORMAT_NV12:
> + return V4L2_PIX_FMT_NV12;
> + case HFI_COLOR_FORMAT_NV21:
> + return V4L2_PIX_FMT_NV21;
> + case HFI_COLOR_FORMAT_NV12_UBWC:
> + return V4L2_PIX_FMT_QC08C;
> + case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
> + return V4L2_PIX_FMT_QC10C;
> + case HFI_COLOR_FORMAT_P010:
> + return V4L2_PIX_FMT_P010;
> + default:
> + break;
> + }
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(to_v4l2_raw_fmt);
> +
> static int platform_get_bufreq(struct venus_inst *inst, u32 buftype,
> struct hfi_buffer_requirements *req)
> {
> @@ -639,20 +661,20 @@ static int platform_get_bufreq(struct venus_inst *inst, u32 buftype,
> params.out_width = inst->out_width;
> params.out_height = inst->out_height;
> params.codec = inst->fmt_out->pixfmt;
> - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_cap->pixfmt);
> + params.color_fmt = inst->fmt_cap->pixfmt;
> params.dec.max_mbs_per_frame = mbs_per_frame_max(inst);
> params.dec.buffer_size_limit = 0;
> params.dec.is_secondary_output =
> inst->opb_buftype == HFI_BUFFER_OUTPUT2;
> if (params.dec.is_secondary_output)
> - params.hfi_dpb_color_fmt = inst->dpb_fmt;
> + params.dpb_color_fmt = to_v4l2_raw_fmt(inst->dpb_fmt);
> params.dec.is_interlaced =
> inst->pic_struct != HFI_INTERLACE_FRAME_PROGRESSIVE;
> } else {
> params.width = inst->out_width;
> params.height = inst->out_height;
> params.codec = inst->fmt_cap->pixfmt;
> - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_out->pixfmt);
> + params.color_fmt = inst->fmt_out->pixfmt;
> params.enc.work_mode = VIDC_WORK_MODE_2;
> params.enc.rc_type = HFI_RATE_CONTROL_OFF;
> if (enc_ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ)
> @@ -942,146 +964,10 @@ int venus_helper_set_profile_level(struct venus_inst *inst, u32 profile, u32 lev
> }
> EXPORT_SYMBOL_GPL(venus_helper_set_profile_level);
>
> -static u32 get_framesize_raw_nv12(u32 width, u32 height)
> -{
> - u32 y_stride, uv_stride, y_plane;
> - u32 y_sclines, uv_sclines, uv_plane;
> - u32 size;
> -
> - y_stride = ALIGN(width, 128);
> - uv_stride = ALIGN(width, 128);
> - y_sclines = ALIGN(height, 32);
> - uv_sclines = ALIGN(((height + 1) >> 1), 16);
> -
> - y_plane = y_stride * y_sclines;
> - uv_plane = uv_stride * uv_sclines + SZ_4K;
> - size = y_plane + uv_plane + SZ_8K;
> -
> - return ALIGN(size, SZ_4K);
> -}
> -
> -static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height)
> -{
> - u32 y_meta_stride, y_meta_plane;
> - u32 y_stride, y_plane;
> - u32 uv_meta_stride, uv_meta_plane;
> - u32 uv_stride, uv_plane;
> - u32 extradata = SZ_16K;
> -
> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
> - y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(height, 8), 16);
> - y_meta_plane = ALIGN(y_meta_plane, SZ_4K);
> -
> - y_stride = ALIGN(width, 128);
> - y_plane = ALIGN(y_stride * ALIGN(height, 32), SZ_4K);
> -
> - uv_meta_stride = ALIGN(DIV_ROUND_UP(width / 2, 16), 64);
> - uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(height / 2, 8), 16);
> - uv_meta_plane = ALIGN(uv_meta_plane, SZ_4K);
> -
> - uv_stride = ALIGN(width, 128);
> - uv_plane = ALIGN(uv_stride * ALIGN(height / 2, 32), SZ_4K);
> -
> - return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane +
> - max(extradata, y_stride * 48), SZ_4K);
> -}
> -
> -static u32 get_framesize_raw_p010(u32 width, u32 height)
> -{
> - u32 y_plane, uv_plane, y_stride, uv_stride, y_sclines, uv_sclines;
> -
> - y_stride = ALIGN(width * 2, 128);
> - uv_stride = ALIGN(width * 2, 128);
> - y_sclines = ALIGN(height, 32);
> - uv_sclines = ALIGN((height + 1) >> 1, 16);
> - y_plane = y_stride * y_sclines;
> - uv_plane = uv_stride * uv_sclines;
> -
> - return ALIGN((y_plane + uv_plane), SZ_4K);
> -}
> -
> -static u32 get_framesize_raw_p010_ubwc(u32 width, u32 height)
> -{
> - u32 y_stride, uv_stride, y_sclines, uv_sclines;
> - u32 y_ubwc_plane, uv_ubwc_plane;
> - u32 y_meta_stride, y_meta_scanlines;
> - u32 uv_meta_stride, uv_meta_scanlines;
> - u32 y_meta_plane, uv_meta_plane;
> - u32 size;
> -
> - y_stride = ALIGN(width * 2, 256);
> - uv_stride = ALIGN(width * 2, 256);
> - y_sclines = ALIGN(height, 16);
> - uv_sclines = ALIGN((height + 1) >> 1, 16);
> -
> - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
> - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
> - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
> - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
> - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64);
> - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
> - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
> -
> - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
> -
> - return ALIGN(size, SZ_4K);
> -}
> -
> -static u32 get_framesize_raw_yuv420_tp10_ubwc(u32 width, u32 height)
> -{
> - u32 y_stride, uv_stride, y_sclines, uv_sclines;
> - u32 y_ubwc_plane, uv_ubwc_plane;
> - u32 y_meta_stride, y_meta_scanlines;
> - u32 uv_meta_stride, uv_meta_scanlines;
> - u32 y_meta_plane, uv_meta_plane;
> - u32 extradata = SZ_16K;
> - u32 size;
> -
> - y_stride = ALIGN(width * 4 / 3, 256);
> - uv_stride = ALIGN(width * 4 / 3, 256);
> - y_sclines = ALIGN(height, 16);
> - uv_sclines = ALIGN((height + 1) >> 1, 16);
> -
> - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
> - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 48), 64);
> - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
> - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
> - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64);
> - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
> - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
> -
> - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
> - size += max(extradata + SZ_8K, y_stride * 48);
> -
> - return ALIGN(size, SZ_4K);
> -}
> -
> -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height)
> -{
> - switch (hfi_fmt) {
> - case HFI_COLOR_FORMAT_NV12:
> - case HFI_COLOR_FORMAT_NV21:
> - return get_framesize_raw_nv12(width, height);
> - case HFI_COLOR_FORMAT_NV12_UBWC:
> - return get_framesize_raw_nv12_ubwc(width, height);
> - case HFI_COLOR_FORMAT_P010:
> - return get_framesize_raw_p010(width, height);
> - case HFI_COLOR_FORMAT_P010_UBWC:
> - return get_framesize_raw_p010_ubwc(width, height);
> - case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
> - return get_framesize_raw_yuv420_tp10_ubwc(width, height);
> - default:
> - return 0;
> - }
> -}
> -EXPORT_SYMBOL_GPL(venus_helper_get_framesz_raw);
> -
> u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
> {
> - u32 hfi_fmt, sz;
> bool compressed;
> + u32 sz;
>
> switch (v4l2_fmt) {
> case V4L2_PIX_FMT_MPEG:
> @@ -1112,11 +998,7 @@ u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
> return ALIGN(sz, SZ_4K);
> }
>
> - hfi_fmt = to_hfi_raw_fmt(v4l2_fmt);
> - if (!hfi_fmt)
> - return 0;
> -
> - return venus_helper_get_framesz_raw(hfi_fmt, width, height);
> + return video_raw_buffer_size(v4l2_fmt, width, height);
> }
> EXPORT_SYMBOL_GPL(venus_helper_get_framesz);
>
> diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h
> index 358e4f3..9b72d18 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/helpers.h
> +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.h
> @@ -11,6 +11,7 @@
> struct venus_inst;
> struct venus_core;
>
> +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt);
> bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt);
> struct vb2_v4l2_buffer *venus_helper_find_buf(struct venus_inst *inst,
> unsigned int type, u32 idx);
> @@ -29,7 +30,6 @@ void venus_helper_m2m_device_run(void *priv);
> void venus_helper_m2m_job_abort(void *priv);
> int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
> struct hfi_buffer_requirements *req);
> -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height);
> u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height);
> int venus_helper_set_input_resolution(struct venus_inst *inst,
> unsigned int width, unsigned int height);
> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> index 25e6074..20f684e 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> @@ -15,8 +15,8 @@ struct hfi_plat_buffers_params {
> u32 out_width;
> u32 out_height;
> u32 codec;
> - u32 hfi_color_fmt;
> - u32 hfi_dpb_color_fmt;
> + u32 color_fmt;
> + u32 dpb_color_fmt;
As usual. This is not a helper introduction. This is field rename. Could
you please split that.
> enum hfi_version version;
> u32 num_vpp_pipes;
> union {
> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> index f5a6559..3e06516 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> @@ -6,6 +6,7 @@
> #include <linux/sizes.h>
> #include <linux/videodev2.h>
>
> +#include "../buffers.h"
> #include "hfi.h"
> #include "hfi_plat_bufs.h"
> #include "helpers.h"
> @@ -1233,13 +1234,11 @@ static int bufreq_dec(struct hfi_plat_buffers_params *params, u32 buftype,
> buffer_size_limit);
> } else if (buftype == HFI_BUFFER_OUTPUT || buftype == HFI_BUFFER_OUTPUT2) {
> hfi_bufreq_set_count_min(bufreq, version, out_min_count);
> - bufreq->size =
> - venus_helper_get_framesz_raw(params->hfi_color_fmt,
> + bufreq->size = video_raw_buffer_size(params->color_fmt,
> out_width, out_height);
> if (buftype == HFI_BUFFER_OUTPUT &&
> params->dec.is_secondary_output)
> - bufreq->size =
> - venus_helper_get_framesz_raw(params->hfi_dpb_color_fmt,
> + bufreq->size = video_raw_buffer_size(params->dpb_color_fmt,
> out_width, out_height);
> } else if (buftype == HFI_BUFFER_INTERNAL_SCRATCH(version)) {
> bufreq->size = dec_ops->scratch(width, height, is_interlaced);
> @@ -1297,8 +1296,7 @@ static int bufreq_enc(struct hfi_plat_buffers_params *params, u32 buftype,
>
> if (buftype == HFI_BUFFER_INPUT) {
> hfi_bufreq_set_count_min(bufreq, version, MIN_INPUT_BUFFERS);
> - bufreq->size =
> - venus_helper_get_framesz_raw(params->hfi_color_fmt,
> + bufreq->size = video_raw_buffer_size(params->color_fmt,
> width, height);
> } else if (buftype == HFI_BUFFER_OUTPUT ||
> buftype == HFI_BUFFER_OUTPUT2) {
> diff --git a/drivers/media/platform/qcom/vcodec/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c
> index dbf305c..e6316be 100644
> --- a/drivers/media/platform/qcom/vcodec/venus/vdec.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/vdec.c
> @@ -15,6 +15,7 @@
> #include <media/v4l2-mem2mem.h>
> #include <media/videobuf2-dma-contig.h>
>
> +#include "../buffers.h"
> #include "hfi_venus_io.h"
> #include "hfi_parser.h"
> #include "core.h"
> @@ -777,9 +778,9 @@ static int vdec_output_conf(struct venus_inst *inst)
> return ret;
>
> inst->output_buf_size =
> - venus_helper_get_framesz_raw(out_fmt, width, height);
> + video_raw_buffer_size(to_v4l2_raw_fmt(out_fmt), width, height);
> inst->output2_buf_size =
> - venus_helper_get_framesz_raw(out2_fmt, width, height);
> + video_raw_buffer_size(to_v4l2_raw_fmt(out2_fmt), width, height);
>
> if (is_ubwc_fmt(out_fmt)) {
> inst->opb_buftype = HFI_BUFFER_OUTPUT2;
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 03/34] media: introduce common helpers for buffer size calculation
2023-12-18 18:32 ` Dmitry Baryshkov
@ 2023-12-20 8:04 ` Dikshita Agarwal
2023-12-20 8:15 ` Dmitry Baryshkov
0 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:04 UTC (permalink / raw)
To: Dmitry Baryshkov, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 12:02 AM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> Introduce helper to calculate size of pixel buffer and
>> use in venus driver.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++++++++++++
>> drivers/media/platform/qcom/vcodec/buffers.h | 15 ++
>> drivers/media/platform/qcom/vcodec/venus/Makefile | 4 +-
>> drivers/media/platform/qcom/vcodec/venus/helpers.c | 172
>> ++++-----------------
>> drivers/media/platform/qcom/vcodec/venus/helpers.h | 2 +-
>> .../platform/qcom/vcodec/venus/hfi_plat_bufs.h | 4 +-
>> .../platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c | 10 +-
>> drivers/media/platform/qcom/vcodec/venus/vdec.c | 5 +-
>> 8 files changed, 157 insertions(+), 158 deletions(-)
>> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
>> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
>>
>> diff --git a/drivers/media/platform/qcom/vcodec/buffers.c
>> b/drivers/media/platform/qcom/vcodec/buffers.c
>> new file mode 100644
>> index 0000000..58ab3b0
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/buffers.c
>> @@ -0,0 +1,103 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +#include <linux/videodev2.h>
>> +
>> +#include "buffers.h"
>> +
>> +u32 video_raw_buffer_size(u32 colorformat,
>
> Usual drill, non-prefixed name.
Will fix in next version.
>
>> + u32 pix_width,
>> + u32 pix_height)
>
> My gut tells me that this is misaligned.
>
>> +{
>> + u32 size = 0;
>> + u32 y_plane, uv_plane, y_stride,
>> + uv_stride, y_sclines, uv_sclines;
>> + u32 y_ubwc_plane = 0, uv_ubwc_plane = 0;
>> + u32 y_meta_stride = 0, y_meta_scanlines = 0;
>> + u32 uv_meta_stride = 0, uv_meta_scanlines = 0;
>> + u32 y_meta_plane = 0, uv_meta_plane = 0;
>> +
>> + if (!pix_width || !pix_height)
>> + goto invalid_input;
>> +
>> + switch (colorformat) {
>> + case V4L2_PIX_FMT_NV12:
>> + case V4L2_PIX_FMT_NV21:
>> + y_stride = ALIGN(pix_width, 128);
>> + uv_stride = ALIGN(pix_width, 128);
>> + y_sclines = ALIGN(pix_height, 32);
>> + uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
>> + y_plane = y_stride * y_sclines;
>> + uv_plane = uv_stride * uv_sclines;
>> + size = y_plane + uv_plane;
>> + break;
>> + case V4L2_PIX_FMT_QC08C:
>> + y_stride = ALIGN(pix_width, 128);
>> + uv_stride = ALIGN(pix_width, 128);
>> + y_sclines = ALIGN(pix_height, 32);
>> + uv_sclines = ALIGN((pix_height + 1) >> 1, 32);
>> + y_meta_stride = ALIGN(DIV_ROUND_UP(pix_width, 32), 64);
>> + uv_meta_stride = ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 16), 64);
>> + y_ubwc_plane =
>> + ALIGN(y_stride * y_sclines, 4096);
>> + uv_ubwc_plane =
>> + ALIGN(uv_stride * uv_sclines, 4096);
>> + y_meta_scanlines =
>> + ALIGN(DIV_ROUND_UP(pix_height, 8), 16);
>> + y_meta_plane =
>> + ALIGN(y_meta_stride * y_meta_scanlines, 4096);
>> + uv_meta_scanlines =
>> + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 8), 16);
>> + uv_meta_plane =
>> + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
>> + size = (y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
>> + uv_meta_plane);
>> + break;
>> + case V4L2_PIX_FMT_QC10C:
>> + y_stride =
>> + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
>> + uv_stride =
>> + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
>> + y_sclines =
>> + ALIGN(pix_height, 16);
>> + uv_sclines =
>> + ALIGN((pix_height + 1) >> 1, 16);
>> + y_ubwc_plane =
>> + ALIGN(y_stride * y_sclines, 4096);
>> + uv_ubwc_plane =
>> + ALIGN(uv_stride * uv_sclines, 4096);
>> + y_meta_stride =
>> + ALIGN(DIV_ROUND_UP(pix_width, 48), 64);
>> + y_meta_scanlines =
>> + ALIGN(DIV_ROUND_UP(pix_height, 4), 16);
>> + y_meta_plane =
>> + ALIGN(y_meta_stride * y_meta_scanlines, 4096);
>> + uv_meta_stride =
>> + ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 24), 64);
>> + uv_meta_scanlines =
>> + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 4), 16);
>> + uv_meta_plane =
>> + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
>> +
>> + size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
>> + uv_meta_plane;
>> + break;
>> + case V4L2_PIX_FMT_P010:
>> + y_stride = ALIGN(pix_width * 2, 128);
>> + uv_stride = ALIGN(pix_width * 2, 128);
>> + y_sclines = ALIGN(pix_height, 32);
>> + uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
>> + y_plane = y_stride * y_sclines;
>> + uv_plane = uv_stride * uv_sclines;
>> +
>> + size = y_plane + uv_plane;
>> + break;
>> + default:
>> + break;
>> + }
>> +
>> +invalid_input:
>> +
>> + return ALIGN(size, 4096);
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/buffers.h
>> b/drivers/media/platform/qcom/vcodec/buffers.h
>> new file mode 100644
>> index 0000000..ac1d052
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/buffers.h
>> @@ -0,0 +1,15 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#ifndef _BUFFERS_H_
>> +#define _BUFFERS_H_
>> +
>> +#include <linux/types.h>
>> +
>> +u32 video_raw_buffer_size(u32 colorformat,
>> + u32 pix_width,
>> + u32 pix_height);
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile
>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> index 1941ef4..6abd54a 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/Makefile
>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> @@ -1,8 +1,7 @@
>> # SPDX-License-Identifier: GPL-2.0
>> # Makefile for Qualcomm Venus driver
>> -venus-core-objs += ../firmware.o \
>> - ../hfi_queue.o
>> +venus-core-objs += ../firmware.o ../hfi_queue.o ../buffers.o
>
> Ugh. I missed that in the previous patches. This is not how the helpers are
> done. Add normal kernel module instead.
>
I unnderstand this looks little messy, I can try to move the makefile one
level up (in new vcodec folder), compile the common object by default and
compile venus or iris based on config.
Please let me know what do you think of this apporach.
>> venus-core-objs += core.o helpers.o firmware_no_tz.o \
>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>> @@ -10,6 +9,7 @@ venus-core-objs += core.o helpers.o firmware_no_tz.o \
>> hfi_platform.o hfi_platform_v4.o \
>> hfi_platform_v6.o hfi_plat_bufs_v6.o \
>> +venus-dec-objs += ../buffers.o
>
> I really wonder, why doesn't this end up with 'symbol defined multiple
> times' error.
>
>> venus-dec-objs += vdec.o vdec_ctrls.o
>> venus-enc-objs += venc.o venc_ctrls.o
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.c
>> b/drivers/media/platform/qcom/vcodec/venus/helpers.c
>> index 8295542..95e4424 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/helpers.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.c
>> @@ -12,6 +12,7 @@
>> #include <media/v4l2-mem2mem.h>
>> #include <asm/div64.h>
>> +#include "../buffers.h"
>> #include "core.h"
>> #include "helpers.h"
>> #include "hfi_helper.h"
>> @@ -616,6 +617,27 @@ static u32 to_hfi_raw_fmt(u32 v4l2_fmt)
>> return 0;
>> }
>> +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt)
>> +{
>> + switch (hfi_color_fmt) {
>> + case HFI_COLOR_FORMAT_NV12:
>> + return V4L2_PIX_FMT_NV12;
>> + case HFI_COLOR_FORMAT_NV21:
>> + return V4L2_PIX_FMT_NV21;
>> + case HFI_COLOR_FORMAT_NV12_UBWC:
>> + return V4L2_PIX_FMT_QC08C;
>> + case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
>> + return V4L2_PIX_FMT_QC10C;
>> + case HFI_COLOR_FORMAT_P010:
>> + return V4L2_PIX_FMT_P010;
>> + default:
>> + break;
>> + }
>> +
>> + return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(to_v4l2_raw_fmt);
>> +
>> static int platform_get_bufreq(struct venus_inst *inst, u32 buftype,
>> struct hfi_buffer_requirements *req)
>> {
>> @@ -639,20 +661,20 @@ static int platform_get_bufreq(struct venus_inst
>> *inst, u32 buftype,
>> params.out_width = inst->out_width;
>> params.out_height = inst->out_height;
>> params.codec = inst->fmt_out->pixfmt;
>> - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_cap->pixfmt);
>> + params.color_fmt = inst->fmt_cap->pixfmt;
>> params.dec.max_mbs_per_frame = mbs_per_frame_max(inst);
>> params.dec.buffer_size_limit = 0;
>> params.dec.is_secondary_output =
>> inst->opb_buftype == HFI_BUFFER_OUTPUT2;
>> if (params.dec.is_secondary_output)
>> - params.hfi_dpb_color_fmt = inst->dpb_fmt;
>> + params.dpb_color_fmt = to_v4l2_raw_fmt(inst->dpb_fmt);
>> params.dec.is_interlaced =
>> inst->pic_struct != HFI_INTERLACE_FRAME_PROGRESSIVE;
>> } else {
>> params.width = inst->out_width;
>> params.height = inst->out_height;
>> params.codec = inst->fmt_cap->pixfmt;
>> - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_out->pixfmt);
>> + params.color_fmt = inst->fmt_out->pixfmt;
>> params.enc.work_mode = VIDC_WORK_MODE_2;
>> params.enc.rc_type = HFI_RATE_CONTROL_OFF;
>> if (enc_ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ)
>> @@ -942,146 +964,10 @@ int venus_helper_set_profile_level(struct
>> venus_inst *inst, u32 profile, u32 lev
>> }
>> EXPORT_SYMBOL_GPL(venus_helper_set_profile_level);
>> -static u32 get_framesize_raw_nv12(u32 width, u32 height)
>> -{
>> - u32 y_stride, uv_stride, y_plane;
>> - u32 y_sclines, uv_sclines, uv_plane;
>> - u32 size;
>> -
>> - y_stride = ALIGN(width, 128);
>> - uv_stride = ALIGN(width, 128);
>> - y_sclines = ALIGN(height, 32);
>> - uv_sclines = ALIGN(((height + 1) >> 1), 16);
>> -
>> - y_plane = y_stride * y_sclines;
>> - uv_plane = uv_stride * uv_sclines + SZ_4K;
>> - size = y_plane + uv_plane + SZ_8K;
>> -
>> - return ALIGN(size, SZ_4K);
>> -}
>> -
>> -static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height)
>> -{
>> - u32 y_meta_stride, y_meta_plane;
>> - u32 y_stride, y_plane;
>> - u32 uv_meta_stride, uv_meta_plane;
>> - u32 uv_stride, uv_plane;
>> - u32 extradata = SZ_16K;
>> -
>> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
>> - y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(height, 8), 16);
>> - y_meta_plane = ALIGN(y_meta_plane, SZ_4K);
>> -
>> - y_stride = ALIGN(width, 128);
>> - y_plane = ALIGN(y_stride * ALIGN(height, 32), SZ_4K);
>> -
>> - uv_meta_stride = ALIGN(DIV_ROUND_UP(width / 2, 16), 64);
>> - uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(height / 2, 8),
>> 16);
>> - uv_meta_plane = ALIGN(uv_meta_plane, SZ_4K);
>> -
>> - uv_stride = ALIGN(width, 128);
>> - uv_plane = ALIGN(uv_stride * ALIGN(height / 2, 32), SZ_4K);
>> -
>> - return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane +
>> - max(extradata, y_stride * 48), SZ_4K);
>> -}
>> -
>> -static u32 get_framesize_raw_p010(u32 width, u32 height)
>> -{
>> - u32 y_plane, uv_plane, y_stride, uv_stride, y_sclines, uv_sclines;
>> -
>> - y_stride = ALIGN(width * 2, 128);
>> - uv_stride = ALIGN(width * 2, 128);
>> - y_sclines = ALIGN(height, 32);
>> - uv_sclines = ALIGN((height + 1) >> 1, 16);
>> - y_plane = y_stride * y_sclines;
>> - uv_plane = uv_stride * uv_sclines;
>> -
>> - return ALIGN((y_plane + uv_plane), SZ_4K);
>> -}
>> -
>> -static u32 get_framesize_raw_p010_ubwc(u32 width, u32 height)
>> -{
>> - u32 y_stride, uv_stride, y_sclines, uv_sclines;
>> - u32 y_ubwc_plane, uv_ubwc_plane;
>> - u32 y_meta_stride, y_meta_scanlines;
>> - u32 uv_meta_stride, uv_meta_scanlines;
>> - u32 y_meta_plane, uv_meta_plane;
>> - u32 size;
>> -
>> - y_stride = ALIGN(width * 2, 256);
>> - uv_stride = ALIGN(width * 2, 256);
>> - y_sclines = ALIGN(height, 16);
>> - uv_sclines = ALIGN((height + 1) >> 1, 16);
>> -
>> - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
>> - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
>> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
>> - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
>> - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
>> - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64);
>> - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
>> - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
>> -
>> - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
>> -
>> - return ALIGN(size, SZ_4K);
>> -}
>> -
>> -static u32 get_framesize_raw_yuv420_tp10_ubwc(u32 width, u32 height)
>> -{
>> - u32 y_stride, uv_stride, y_sclines, uv_sclines;
>> - u32 y_ubwc_plane, uv_ubwc_plane;
>> - u32 y_meta_stride, y_meta_scanlines;
>> - u32 uv_meta_stride, uv_meta_scanlines;
>> - u32 y_meta_plane, uv_meta_plane;
>> - u32 extradata = SZ_16K;
>> - u32 size;
>> -
>> - y_stride = ALIGN(width * 4 / 3, 256);
>> - uv_stride = ALIGN(width * 4 / 3, 256);
>> - y_sclines = ALIGN(height, 16);
>> - uv_sclines = ALIGN((height + 1) >> 1, 16);
>> -
>> - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
>> - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
>> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 48), 64);
>> - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
>> - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
>> - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64);
>> - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
>> - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
>> -
>> - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
>> - size += max(extradata + SZ_8K, y_stride * 48);
>> -
>> - return ALIGN(size, SZ_4K);
>> -}
>> -
>> -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height)
>> -{
>> - switch (hfi_fmt) {
>> - case HFI_COLOR_FORMAT_NV12:
>> - case HFI_COLOR_FORMAT_NV21:
>> - return get_framesize_raw_nv12(width, height);
>> - case HFI_COLOR_FORMAT_NV12_UBWC:
>> - return get_framesize_raw_nv12_ubwc(width, height);
>> - case HFI_COLOR_FORMAT_P010:
>> - return get_framesize_raw_p010(width, height);
>> - case HFI_COLOR_FORMAT_P010_UBWC:
>> - return get_framesize_raw_p010_ubwc(width, height);
>> - case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
>> - return get_framesize_raw_yuv420_tp10_ubwc(width, height);
>> - default:
>> - return 0;
>> - }
>> -}
>> -EXPORT_SYMBOL_GPL(venus_helper_get_framesz_raw);
>> -
>> u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
>> {
>> - u32 hfi_fmt, sz;
>> bool compressed;
>> + u32 sz;
>> switch (v4l2_fmt) {
>> case V4L2_PIX_FMT_MPEG:
>> @@ -1112,11 +998,7 @@ u32 venus_helper_get_framesz(u32 v4l2_fmt, u32
>> width, u32 height)
>> return ALIGN(sz, SZ_4K);
>> }
>> - hfi_fmt = to_hfi_raw_fmt(v4l2_fmt);
>> - if (!hfi_fmt)
>> - return 0;
>> -
>> - return venus_helper_get_framesz_raw(hfi_fmt, width, height);
>> + return video_raw_buffer_size(v4l2_fmt, width, height);
>> }
>> EXPORT_SYMBOL_GPL(venus_helper_get_framesz);
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.h
>> b/drivers/media/platform/qcom/vcodec/venus/helpers.h
>> index 358e4f3..9b72d18 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/helpers.h
>> +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.h
>> @@ -11,6 +11,7 @@
>> struct venus_inst;
>> struct venus_core;
>> +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt);
>> bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt);
>> struct vb2_v4l2_buffer *venus_helper_find_buf(struct venus_inst *inst,
>> unsigned int type, u32 idx);
>> @@ -29,7 +30,6 @@ void venus_helper_m2m_device_run(void *priv);
>> void venus_helper_m2m_job_abort(void *priv);
>> int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
>> struct hfi_buffer_requirements *req);
>> -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height);
>> u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height);
>> int venus_helper_set_input_resolution(struct venus_inst *inst,
>> unsigned int width, unsigned int height);
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> index 25e6074..20f684e 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> @@ -15,8 +15,8 @@ struct hfi_plat_buffers_params {
>> u32 out_width;
>> u32 out_height;
>> u32 codec;
>> - u32 hfi_color_fmt;
>> - u32 hfi_dpb_color_fmt;
>> + u32 color_fmt;
>> + u32 dpb_color_fmt;
>
> As usual. This is not a helper introduction. This is field rename. Could
> you please split that.
>
>> enum hfi_version version;
>> u32 num_vpp_pipes;
>> union {
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> index f5a6559..3e06516 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> @@ -6,6 +6,7 @@
>> #include <linux/sizes.h>
>> #include <linux/videodev2.h>
>> +#include "../buffers.h"
>> #include "hfi.h"
>> #include "hfi_plat_bufs.h"
>> #include "helpers.h"
>> @@ -1233,13 +1234,11 @@ static int bufreq_dec(struct
>> hfi_plat_buffers_params *params, u32 buftype,
>> buffer_size_limit);
>> } else if (buftype == HFI_BUFFER_OUTPUT || buftype ==
>> HFI_BUFFER_OUTPUT2) {
>> hfi_bufreq_set_count_min(bufreq, version, out_min_count);
>> - bufreq->size =
>> - venus_helper_get_framesz_raw(params->hfi_color_fmt,
>> + bufreq->size = video_raw_buffer_size(params->color_fmt,
>> out_width, out_height);
>> if (buftype == HFI_BUFFER_OUTPUT &&
>> params->dec.is_secondary_output)
>> - bufreq->size =
>> - venus_helper_get_framesz_raw(params->hfi_dpb_color_fmt,
>> + bufreq->size = video_raw_buffer_size(params->dpb_color_fmt,
>> out_width, out_height);
>> } else if (buftype == HFI_BUFFER_INTERNAL_SCRATCH(version)) {
>> bufreq->size = dec_ops->scratch(width, height, is_interlaced);
>> @@ -1297,8 +1296,7 @@ static int bufreq_enc(struct
>> hfi_plat_buffers_params *params, u32 buftype,
>> if (buftype == HFI_BUFFER_INPUT) {
>> hfi_bufreq_set_count_min(bufreq, version, MIN_INPUT_BUFFERS);
>> - bufreq->size =
>> - venus_helper_get_framesz_raw(params->hfi_color_fmt,
>> + bufreq->size = video_raw_buffer_size(params->color_fmt,
>> width, height);
>> } else if (buftype == HFI_BUFFER_OUTPUT ||
>> buftype == HFI_BUFFER_OUTPUT2) {
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/vdec.c
>> b/drivers/media/platform/qcom/vcodec/venus/vdec.c
>> index dbf305c..e6316be 100644
>> --- a/drivers/media/platform/qcom/vcodec/venus/vdec.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/vdec.c
>> @@ -15,6 +15,7 @@
>> #include <media/v4l2-mem2mem.h>
>> #include <media/videobuf2-dma-contig.h>
>> +#include "../buffers.h"
>> #include "hfi_venus_io.h"
>> #include "hfi_parser.h"
>> #include "core.h"
>> @@ -777,9 +778,9 @@ static int vdec_output_conf(struct venus_inst *inst)
>> return ret;
>> inst->output_buf_size =
>> - venus_helper_get_framesz_raw(out_fmt, width, height);
>> + video_raw_buffer_size(to_v4l2_raw_fmt(out_fmt), width, height);
>> inst->output2_buf_size =
>> - venus_helper_get_framesz_raw(out2_fmt, width, height);
>> + video_raw_buffer_size(to_v4l2_raw_fmt(out2_fmt), width,
>> height);
>> if (is_ubwc_fmt(out_fmt)) {
>> inst->opb_buftype = HFI_BUFFER_OUTPUT2;
>
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 03/34] media: introduce common helpers for buffer size calculation
2023-12-20 8:04 ` Dikshita Agarwal
@ 2023-12-20 8:15 ` Dmitry Baryshkov
0 siblings, 0 replies; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-20 8:15 UTC (permalink / raw)
To: Dikshita Agarwal
Cc: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm, quic_abhinavk
On Wed, 20 Dec 2023 at 10:04, Dikshita Agarwal
<quic_dikshita@quicinc.com> wrote:
>
>
>
> On 12/19/2023 12:02 AM, Dmitry Baryshkov wrote:
> > On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >> Introduce helper to calculate size of pixel buffer and
> >> use in venus driver.
> >>
> >> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> >> ---
> >> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++++++++++++
> >> drivers/media/platform/qcom/vcodec/buffers.h | 15 ++
> >> drivers/media/platform/qcom/vcodec/venus/Makefile | 4 +-
> >> drivers/media/platform/qcom/vcodec/venus/helpers.c | 172
> >> ++++-----------------
> >> drivers/media/platform/qcom/vcodec/venus/helpers.h | 2 +-
> >> .../platform/qcom/vcodec/venus/hfi_plat_bufs.h | 4 +-
> >> .../platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c | 10 +-
> >> drivers/media/platform/qcom/vcodec/venus/vdec.c | 5 +-
> >> 8 files changed, 157 insertions(+), 158 deletions(-)
> >> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> >> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
> >>
> >> diff --git a/drivers/media/platform/qcom/vcodec/buffers.c
> >> b/drivers/media/platform/qcom/vcodec/buffers.c
> >> new file mode 100644
> >> index 0000000..58ab3b0
> >> --- /dev/null
> >> +++ b/drivers/media/platform/qcom/vcodec/buffers.c
> >> @@ -0,0 +1,103 @@
> >> +// SPDX-License-Identifier: GPL-2.0-only
> >> +/*
> >> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >> reserved.
> >> + */
> >> +#include <linux/videodev2.h>
> >> +
> >> +#include "buffers.h"
> >> +
> >> +u32 video_raw_buffer_size(u32 colorformat,
> >
> > Usual drill, non-prefixed name.
> Will fix in next version.
> >
> >> + u32 pix_width,
> >> + u32 pix_height)
> >
> > My gut tells me that this is misaligned.
> >
> >> +{
> >> + u32 size = 0;
> >> + u32 y_plane, uv_plane, y_stride,
> >> + uv_stride, y_sclines, uv_sclines;
> >> + u32 y_ubwc_plane = 0, uv_ubwc_plane = 0;
> >> + u32 y_meta_stride = 0, y_meta_scanlines = 0;
> >> + u32 uv_meta_stride = 0, uv_meta_scanlines = 0;
> >> + u32 y_meta_plane = 0, uv_meta_plane = 0;
> >> +
> >> + if (!pix_width || !pix_height)
> >> + goto invalid_input;
> >> +
> >> + switch (colorformat) {
> >> + case V4L2_PIX_FMT_NV12:
> >> + case V4L2_PIX_FMT_NV21:
> >> + y_stride = ALIGN(pix_width, 128);
> >> + uv_stride = ALIGN(pix_width, 128);
> >> + y_sclines = ALIGN(pix_height, 32);
> >> + uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
> >> + y_plane = y_stride * y_sclines;
> >> + uv_plane = uv_stride * uv_sclines;
> >> + size = y_plane + uv_plane;
> >> + break;
> >> + case V4L2_PIX_FMT_QC08C:
> >> + y_stride = ALIGN(pix_width, 128);
> >> + uv_stride = ALIGN(pix_width, 128);
> >> + y_sclines = ALIGN(pix_height, 32);
> >> + uv_sclines = ALIGN((pix_height + 1) >> 1, 32);
> >> + y_meta_stride = ALIGN(DIV_ROUND_UP(pix_width, 32), 64);
> >> + uv_meta_stride = ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 16), 64);
> >> + y_ubwc_plane =
> >> + ALIGN(y_stride * y_sclines, 4096);
> >> + uv_ubwc_plane =
> >> + ALIGN(uv_stride * uv_sclines, 4096);
> >> + y_meta_scanlines =
> >> + ALIGN(DIV_ROUND_UP(pix_height, 8), 16);
> >> + y_meta_plane =
> >> + ALIGN(y_meta_stride * y_meta_scanlines, 4096);
> >> + uv_meta_scanlines =
> >> + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 8), 16);
> >> + uv_meta_plane =
> >> + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
> >> + size = (y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
> >> + uv_meta_plane);
> >> + break;
> >> + case V4L2_PIX_FMT_QC10C:
> >> + y_stride =
> >> + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
> >> + uv_stride =
> >> + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256);
> >> + y_sclines =
> >> + ALIGN(pix_height, 16);
> >> + uv_sclines =
> >> + ALIGN((pix_height + 1) >> 1, 16);
> >> + y_ubwc_plane =
> >> + ALIGN(y_stride * y_sclines, 4096);
> >> + uv_ubwc_plane =
> >> + ALIGN(uv_stride * uv_sclines, 4096);
> >> + y_meta_stride =
> >> + ALIGN(DIV_ROUND_UP(pix_width, 48), 64);
> >> + y_meta_scanlines =
> >> + ALIGN(DIV_ROUND_UP(pix_height, 4), 16);
> >> + y_meta_plane =
> >> + ALIGN(y_meta_stride * y_meta_scanlines, 4096);
> >> + uv_meta_stride =
> >> + ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 24), 64);
> >> + uv_meta_scanlines =
> >> + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 4), 16);
> >> + uv_meta_plane =
> >> + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096);
> >> +
> >> + size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane +
> >> + uv_meta_plane;
> >> + break;
> >> + case V4L2_PIX_FMT_P010:
> >> + y_stride = ALIGN(pix_width * 2, 128);
> >> + uv_stride = ALIGN(pix_width * 2, 128);
> >> + y_sclines = ALIGN(pix_height, 32);
> >> + uv_sclines = ALIGN((pix_height + 1) >> 1, 16);
> >> + y_plane = y_stride * y_sclines;
> >> + uv_plane = uv_stride * uv_sclines;
> >> +
> >> + size = y_plane + uv_plane;
> >> + break;
> >> + default:
> >> + break;
> >> + }
> >> +
> >> +invalid_input:
> >> +
> >> + return ALIGN(size, 4096);
> >> +}
> >> diff --git a/drivers/media/platform/qcom/vcodec/buffers.h
> >> b/drivers/media/platform/qcom/vcodec/buffers.h
> >> new file mode 100644
> >> index 0000000..ac1d052
> >> --- /dev/null
> >> +++ b/drivers/media/platform/qcom/vcodec/buffers.h
> >> @@ -0,0 +1,15 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +/*
> >> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >> reserved.
> >> + */
> >> +
> >> +#ifndef _BUFFERS_H_
> >> +#define _BUFFERS_H_
> >> +
> >> +#include <linux/types.h>
> >> +
> >> +u32 video_raw_buffer_size(u32 colorformat,
> >> + u32 pix_width,
> >> + u32 pix_height);
> >> +
> >> +#endif
> >> diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> index 1941ef4..6abd54a 100644
> >> --- a/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> @@ -1,8 +1,7 @@
> >> # SPDX-License-Identifier: GPL-2.0
> >> # Makefile for Qualcomm Venus driver
> >> -venus-core-objs += ../firmware.o \
> >> - ../hfi_queue.o
> >> +venus-core-objs += ../firmware.o ../hfi_queue.o ../buffers.o
> >
> > Ugh. I missed that in the previous patches. This is not how the helpers are
> > done. Add normal kernel module instead.
> >
> I unnderstand this looks little messy, I can try to move the makefile one
> level up (in new vcodec folder), compile the common object by default and
> compile venus or iris based on config.
> Please let me know what do you think of this apporach.
I think that you want a venus-common or qcom-vcodec-common kernel
module instead. It will have a clean API and clean interfaces.
But, what we really want is to have iris as a subdir and subdriver inside venus.
>
> >> venus-core-objs += core.o helpers.o firmware_no_tz.o \
> >> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> >> @@ -10,6 +9,7 @@ venus-core-objs += core.o helpers.o firmware_no_tz.o \
> >> hfi_platform.o hfi_platform_v4.o \
> >> hfi_platform_v6.o hfi_plat_bufs_v6.o \
> >> +venus-dec-objs += ../buffers.o
> >
> > I really wonder, why doesn't this end up with 'symbol defined multiple
> > times' error.
> >
> >> venus-dec-objs += vdec.o vdec_ctrls.o
> >> venus-enc-objs += venc.o venc_ctrls.o
> >> diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.c
> >> b/drivers/media/platform/qcom/vcodec/venus/helpers.c
> >> index 8295542..95e4424 100644
> >> --- a/drivers/media/platform/qcom/vcodec/venus/helpers.c
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.c
> >> @@ -12,6 +12,7 @@
> >> #include <media/v4l2-mem2mem.h>
> >> #include <asm/div64.h>
> >> +#include "../buffers.h"
> >> #include "core.h"
> >> #include "helpers.h"
> >> #include "hfi_helper.h"
> >> @@ -616,6 +617,27 @@ static u32 to_hfi_raw_fmt(u32 v4l2_fmt)
> >> return 0;
> >> }
> >> +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt)
> >> +{
> >> + switch (hfi_color_fmt) {
> >> + case HFI_COLOR_FORMAT_NV12:
> >> + return V4L2_PIX_FMT_NV12;
> >> + case HFI_COLOR_FORMAT_NV21:
> >> + return V4L2_PIX_FMT_NV21;
> >> + case HFI_COLOR_FORMAT_NV12_UBWC:
> >> + return V4L2_PIX_FMT_QC08C;
> >> + case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
> >> + return V4L2_PIX_FMT_QC10C;
> >> + case HFI_COLOR_FORMAT_P010:
> >> + return V4L2_PIX_FMT_P010;
> >> + default:
> >> + break;
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +EXPORT_SYMBOL_GPL(to_v4l2_raw_fmt);
> >> +
> >> static int platform_get_bufreq(struct venus_inst *inst, u32 buftype,
> >> struct hfi_buffer_requirements *req)
> >> {
> >> @@ -639,20 +661,20 @@ static int platform_get_bufreq(struct venus_inst
> >> *inst, u32 buftype,
> >> params.out_width = inst->out_width;
> >> params.out_height = inst->out_height;
> >> params.codec = inst->fmt_out->pixfmt;
> >> - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_cap->pixfmt);
> >> + params.color_fmt = inst->fmt_cap->pixfmt;
> >> params.dec.max_mbs_per_frame = mbs_per_frame_max(inst);
> >> params.dec.buffer_size_limit = 0;
> >> params.dec.is_secondary_output =
> >> inst->opb_buftype == HFI_BUFFER_OUTPUT2;
> >> if (params.dec.is_secondary_output)
> >> - params.hfi_dpb_color_fmt = inst->dpb_fmt;
> >> + params.dpb_color_fmt = to_v4l2_raw_fmt(inst->dpb_fmt);
> >> params.dec.is_interlaced =
> >> inst->pic_struct != HFI_INTERLACE_FRAME_PROGRESSIVE;
> >> } else {
> >> params.width = inst->out_width;
> >> params.height = inst->out_height;
> >> params.codec = inst->fmt_cap->pixfmt;
> >> - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_out->pixfmt);
> >> + params.color_fmt = inst->fmt_out->pixfmt;
> >> params.enc.work_mode = VIDC_WORK_MODE_2;
> >> params.enc.rc_type = HFI_RATE_CONTROL_OFF;
> >> if (enc_ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ)
> >> @@ -942,146 +964,10 @@ int venus_helper_set_profile_level(struct
> >> venus_inst *inst, u32 profile, u32 lev
> >> }
> >> EXPORT_SYMBOL_GPL(venus_helper_set_profile_level);
> >> -static u32 get_framesize_raw_nv12(u32 width, u32 height)
> >> -{
> >> - u32 y_stride, uv_stride, y_plane;
> >> - u32 y_sclines, uv_sclines, uv_plane;
> >> - u32 size;
> >> -
> >> - y_stride = ALIGN(width, 128);
> >> - uv_stride = ALIGN(width, 128);
> >> - y_sclines = ALIGN(height, 32);
> >> - uv_sclines = ALIGN(((height + 1) >> 1), 16);
> >> -
> >> - y_plane = y_stride * y_sclines;
> >> - uv_plane = uv_stride * uv_sclines + SZ_4K;
> >> - size = y_plane + uv_plane + SZ_8K;
> >> -
> >> - return ALIGN(size, SZ_4K);
> >> -}
> >> -
> >> -static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height)
> >> -{
> >> - u32 y_meta_stride, y_meta_plane;
> >> - u32 y_stride, y_plane;
> >> - u32 uv_meta_stride, uv_meta_plane;
> >> - u32 uv_stride, uv_plane;
> >> - u32 extradata = SZ_16K;
> >> -
> >> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
> >> - y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(height, 8), 16);
> >> - y_meta_plane = ALIGN(y_meta_plane, SZ_4K);
> >> -
> >> - y_stride = ALIGN(width, 128);
> >> - y_plane = ALIGN(y_stride * ALIGN(height, 32), SZ_4K);
> >> -
> >> - uv_meta_stride = ALIGN(DIV_ROUND_UP(width / 2, 16), 64);
> >> - uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(height / 2, 8),
> >> 16);
> >> - uv_meta_plane = ALIGN(uv_meta_plane, SZ_4K);
> >> -
> >> - uv_stride = ALIGN(width, 128);
> >> - uv_plane = ALIGN(uv_stride * ALIGN(height / 2, 32), SZ_4K);
> >> -
> >> - return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane +
> >> - max(extradata, y_stride * 48), SZ_4K);
> >> -}
> >> -
> >> -static u32 get_framesize_raw_p010(u32 width, u32 height)
> >> -{
> >> - u32 y_plane, uv_plane, y_stride, uv_stride, y_sclines, uv_sclines;
> >> -
> >> - y_stride = ALIGN(width * 2, 128);
> >> - uv_stride = ALIGN(width * 2, 128);
> >> - y_sclines = ALIGN(height, 32);
> >> - uv_sclines = ALIGN((height + 1) >> 1, 16);
> >> - y_plane = y_stride * y_sclines;
> >> - uv_plane = uv_stride * uv_sclines;
> >> -
> >> - return ALIGN((y_plane + uv_plane), SZ_4K);
> >> -}
> >> -
> >> -static u32 get_framesize_raw_p010_ubwc(u32 width, u32 height)
> >> -{
> >> - u32 y_stride, uv_stride, y_sclines, uv_sclines;
> >> - u32 y_ubwc_plane, uv_ubwc_plane;
> >> - u32 y_meta_stride, y_meta_scanlines;
> >> - u32 uv_meta_stride, uv_meta_scanlines;
> >> - u32 y_meta_plane, uv_meta_plane;
> >> - u32 size;
> >> -
> >> - y_stride = ALIGN(width * 2, 256);
> >> - uv_stride = ALIGN(width * 2, 256);
> >> - y_sclines = ALIGN(height, 16);
> >> - uv_sclines = ALIGN((height + 1) >> 1, 16);
> >> -
> >> - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
> >> - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
> >> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
> >> - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
> >> - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
> >> - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64);
> >> - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
> >> - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
> >> -
> >> - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
> >> -
> >> - return ALIGN(size, SZ_4K);
> >> -}
> >> -
> >> -static u32 get_framesize_raw_yuv420_tp10_ubwc(u32 width, u32 height)
> >> -{
> >> - u32 y_stride, uv_stride, y_sclines, uv_sclines;
> >> - u32 y_ubwc_plane, uv_ubwc_plane;
> >> - u32 y_meta_stride, y_meta_scanlines;
> >> - u32 uv_meta_stride, uv_meta_scanlines;
> >> - u32 y_meta_plane, uv_meta_plane;
> >> - u32 extradata = SZ_16K;
> >> - u32 size;
> >> -
> >> - y_stride = ALIGN(width * 4 / 3, 256);
> >> - uv_stride = ALIGN(width * 4 / 3, 256);
> >> - y_sclines = ALIGN(height, 16);
> >> - uv_sclines = ALIGN((height + 1) >> 1, 16);
> >> -
> >> - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K);
> >> - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K);
> >> - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 48), 64);
> >> - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16);
> >> - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K);
> >> - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64);
> >> - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
> >> - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K);
> >> -
> >> - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane;
> >> - size += max(extradata + SZ_8K, y_stride * 48);
> >> -
> >> - return ALIGN(size, SZ_4K);
> >> -}
> >> -
> >> -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height)
> >> -{
> >> - switch (hfi_fmt) {
> >> - case HFI_COLOR_FORMAT_NV12:
> >> - case HFI_COLOR_FORMAT_NV21:
> >> - return get_framesize_raw_nv12(width, height);
> >> - case HFI_COLOR_FORMAT_NV12_UBWC:
> >> - return get_framesize_raw_nv12_ubwc(width, height);
> >> - case HFI_COLOR_FORMAT_P010:
> >> - return get_framesize_raw_p010(width, height);
> >> - case HFI_COLOR_FORMAT_P010_UBWC:
> >> - return get_framesize_raw_p010_ubwc(width, height);
> >> - case HFI_COLOR_FORMAT_YUV420_TP10_UBWC:
> >> - return get_framesize_raw_yuv420_tp10_ubwc(width, height);
> >> - default:
> >> - return 0;
> >> - }
> >> -}
> >> -EXPORT_SYMBOL_GPL(venus_helper_get_framesz_raw);
> >> -
> >> u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
> >> {
> >> - u32 hfi_fmt, sz;
> >> bool compressed;
> >> + u32 sz;
> >> switch (v4l2_fmt) {
> >> case V4L2_PIX_FMT_MPEG:
> >> @@ -1112,11 +998,7 @@ u32 venus_helper_get_framesz(u32 v4l2_fmt, u32
> >> width, u32 height)
> >> return ALIGN(sz, SZ_4K);
> >> }
> >> - hfi_fmt = to_hfi_raw_fmt(v4l2_fmt);
> >> - if (!hfi_fmt)
> >> - return 0;
> >> -
> >> - return venus_helper_get_framesz_raw(hfi_fmt, width, height);
> >> + return video_raw_buffer_size(v4l2_fmt, width, height);
> >> }
> >> EXPORT_SYMBOL_GPL(venus_helper_get_framesz);
> >> diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.h
> >> b/drivers/media/platform/qcom/vcodec/venus/helpers.h
> >> index 358e4f3..9b72d18 100644
> >> --- a/drivers/media/platform/qcom/vcodec/venus/helpers.h
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.h
> >> @@ -11,6 +11,7 @@
> >> struct venus_inst;
> >> struct venus_core;
> >> +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt);
> >> bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt);
> >> struct vb2_v4l2_buffer *venus_helper_find_buf(struct venus_inst *inst,
> >> unsigned int type, u32 idx);
> >> @@ -29,7 +30,6 @@ void venus_helper_m2m_device_run(void *priv);
> >> void venus_helper_m2m_job_abort(void *priv);
> >> int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
> >> struct hfi_buffer_requirements *req);
> >> -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height);
> >> u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height);
> >> int venus_helper_set_input_resolution(struct venus_inst *inst,
> >> unsigned int width, unsigned int height);
> >> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> >> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> >> index 25e6074..20f684e 100644
> >> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> >> @@ -15,8 +15,8 @@ struct hfi_plat_buffers_params {
> >> u32 out_width;
> >> u32 out_height;
> >> u32 codec;
> >> - u32 hfi_color_fmt;
> >> - u32 hfi_dpb_color_fmt;
> >> + u32 color_fmt;
> >> + u32 dpb_color_fmt;
> >
> > As usual. This is not a helper introduction. This is field rename. Could
> > you please split that.
> >
> >> enum hfi_version version;
> >> u32 num_vpp_pipes;
> >> union {
> >> diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> >> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> >> index f5a6559..3e06516 100644
> >> --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> >> @@ -6,6 +6,7 @@
> >> #include <linux/sizes.h>
> >> #include <linux/videodev2.h>
> >> +#include "../buffers.h"
> >> #include "hfi.h"
> >> #include "hfi_plat_bufs.h"
> >> #include "helpers.h"
> >> @@ -1233,13 +1234,11 @@ static int bufreq_dec(struct
> >> hfi_plat_buffers_params *params, u32 buftype,
> >> buffer_size_limit);
> >> } else if (buftype == HFI_BUFFER_OUTPUT || buftype ==
> >> HFI_BUFFER_OUTPUT2) {
> >> hfi_bufreq_set_count_min(bufreq, version, out_min_count);
> >> - bufreq->size =
> >> - venus_helper_get_framesz_raw(params->hfi_color_fmt,
> >> + bufreq->size = video_raw_buffer_size(params->color_fmt,
> >> out_width, out_height);
> >> if (buftype == HFI_BUFFER_OUTPUT &&
> >> params->dec.is_secondary_output)
> >> - bufreq->size =
> >> - venus_helper_get_framesz_raw(params->hfi_dpb_color_fmt,
> >> + bufreq->size = video_raw_buffer_size(params->dpb_color_fmt,
> >> out_width, out_height);
> >> } else if (buftype == HFI_BUFFER_INTERNAL_SCRATCH(version)) {
> >> bufreq->size = dec_ops->scratch(width, height, is_interlaced);
> >> @@ -1297,8 +1296,7 @@ static int bufreq_enc(struct
> >> hfi_plat_buffers_params *params, u32 buftype,
> >> if (buftype == HFI_BUFFER_INPUT) {
> >> hfi_bufreq_set_count_min(bufreq, version, MIN_INPUT_BUFFERS);
> >> - bufreq->size =
> >> - venus_helper_get_framesz_raw(params->hfi_color_fmt,
> >> + bufreq->size = video_raw_buffer_size(params->color_fmt,
> >> width, height);
> >> } else if (buftype == HFI_BUFFER_OUTPUT ||
> >> buftype == HFI_BUFFER_OUTPUT2) {
> >> diff --git a/drivers/media/platform/qcom/vcodec/venus/vdec.c
> >> b/drivers/media/platform/qcom/vcodec/venus/vdec.c
> >> index dbf305c..e6316be 100644
> >> --- a/drivers/media/platform/qcom/vcodec/venus/vdec.c
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/vdec.c
> >> @@ -15,6 +15,7 @@
> >> #include <media/v4l2-mem2mem.h>
> >> #include <media/videobuf2-dma-contig.h>
> >> +#include "../buffers.h"
> >> #include "hfi_venus_io.h"
> >> #include "hfi_parser.h"
> >> #include "core.h"
> >> @@ -777,9 +778,9 @@ static int vdec_output_conf(struct venus_inst *inst)
> >> return ret;
> >> inst->output_buf_size =
> >> - venus_helper_get_framesz_raw(out_fmt, width, height);
> >> + video_raw_buffer_size(to_v4l2_raw_fmt(out_fmt), width, height);
> >> inst->output2_buf_size =
> >> - venus_helper_get_framesz_raw(out2_fmt, width, height);
> >> + video_raw_buffer_size(to_v4l2_raw_fmt(out2_fmt), width,
> >> height);
> >> if (is_ubwc_fmt(out_fmt)) {
> >> inst->opb_buftype = HFI_BUFFER_OUTPUT2;
> >
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (2 preceding siblings ...)
2023-12-18 11:31 ` [PATCH v2 03/34] media: introduce common helpers for buffer size calculation Dikshita Agarwal
@ 2023-12-18 11:31 ` Dikshita Agarwal
2023-12-18 16:10 ` Krzysztof Kozlowski
2023-12-18 18:17 ` Dmitry Baryshkov
2023-12-18 11:32 ` [PATCH v2 05/34] media: MAINTAINERS: Add Qualcomm Iris video accelerator driver Dikshita Agarwal
` (36 subsequent siblings)
40 siblings, 2 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:31 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Add a schema description for the iris video encoder/decoder
on sm8550.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../bindings/media/qcom,sm8550-iris.yaml | 177 +++++++++++++++++++++
1 file changed, 177 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
diff --git a/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
new file mode 100644
index 0000000..a3d9233
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
@@ -0,0 +1,177 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/qcom,sm8550-iris.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm IRIS video encode and decode accelerators
+
+maintainers:
+ - Vikash Garodia <quic_vgarodia@quicinc.com>
+ - Dikshita Agarwal <quic_dikshita@quicinc.com>
+
+description:
+ The Iris video processing unit is a video encode and decode accelerator
+ present on Qualcomm platforms.
+
+properties:
+ compatible:
+ oneOf:
+ - enum:
+ - qcom,sm8550-iris
+
+ reg:
+ maxItems: 2
+
+ interrupts:
+ maxItems: 1
+
+ power-domains:
+ minItems: 2
+ maxItems: 4
+
+ power-domain-names:
+ oneOf:
+ - items:
+ - const: iris-ctl
+ - const: vcodec
+ - const: mxc
+ - const: mmcx
+
+ operating-points-v2: true
+
+ clocks:
+ maxItems: 3
+
+ clock-names:
+ items:
+ - const: gcc_video_axi0
+ - const: core_clk
+ - const: vcodec_core
+
+ interconnects:
+ maxItems: 2
+
+ interconnect-names:
+ items:
+ - const: iris-cnoc
+ - const: iris-ddr
+
+ memory-region:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ reset-names:
+ items:
+ - const: video_axi_reset
+
+ iommus:
+ maxItems: 2
+
+ dma-coherent: true
+
+ opp-table:
+ type: object
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - power-domains
+ - power-domain-names
+ - clocks
+ - clock-names
+ - interconnects
+ - interconnect-names
+ - memory-region
+ - resets
+ - reset-names
+ - iommus
+ - dma-coherent
+ - opp-table
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,rpmh.h>
+ #include <dt-bindings/clock/qcom,sm8550-gcc.h>
+ #include <dt-bindings/clock/qcom,sm8450-videocc.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interconnect/qcom,icc.h>
+ #include <dt-bindings/interconnect/qcom,sm8550-rpmh.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ iris: video-codec@aa00000 {
+ compatible = "qcom,sm8550-iris";
+
+ reg = <0 0x0aa00000 0 0xf0000>;
+ interrupts = <GIC_SPI 174 IRQ_TYPE_LEVEL_HIGH>;
+
+ power-domains = <&videocc VIDEO_CC_MVS0C_GDSC>,
+ <&videocc VIDEO_CC_MVS0_GDSC>,
+ <&rpmhpd SM8550_MXC>,
+ <&rpmhpd SM8550_MMCX>;
+ power-domain-names = "iris-ctl", "vcodec", "mxc", "mmcx";
+ operating-points-v2 = <&iris_opp_table>;
+
+ clocks = <&gcc GCC_VIDEO_AXI0_CLK>,
+ <&videocc VIDEO_CC_MVS0C_CLK>,
+ <&videocc VIDEO_CC_MVS0_CLK>;
+ clock-names = "gcc_video_axi0", "core_clk", "vcodec_core";
+
+ interconnects = <&gem_noc MASTER_APPSS_PROC QCOM_ICC_TAG_ALWAYS
+ &config_noc SLAVE_VENUS_CFG QCOM_ICC_TAG_ALWAYS>,
+ <&mmss_noc MASTER_VIDEO QCOM_ICC_TAG_ALWAYS
+ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>;
+ interconnect-names = "iris-cnoc", "iris-ddr";
+
+ /* FW load region */
+ memory-region = <&video_mem>;
+
+ resets = <&gcc GCC_VIDEO_AXI0_CLK_ARES>;
+ reset-names = "video_axi_reset";
+
+ iommus = <&apps_smmu 0x1940 0x0000>,
+ <&apps_smmu 0x1947 0x0000>;
+ dma-coherent;
+
+ status = "disabled";
+
+ iris_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-240000000 {
+ opp-hz = /bits/ 64 <240000000>;
+ required-opps = <&rpmhpd_opp_svs>,
+ <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-338000000 {
+ opp-hz = /bits/ 64 <338000000>;
+ required-opps = <&rpmhpd_opp_svs>,
+ <&rpmhpd_opp_svs>;
+ };
+
+ opp-366000000 {
+ opp-hz = /bits/ 64 <366000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>,
+ <&rpmhpd_opp_svs_l1>;
+ };
+
+ opp-444000000 {
+ opp-hz = /bits/ 64 <444000000>;
+ required-opps = <&rpmhpd_opp_turbo>,
+ <&rpmhpd_opp_turbo>;
+ };
+
+ opp-533333334 {
+ opp-hz = /bits/ 64 <533333334>;
+ required-opps = <&rpmhpd_opp_turbo_l1>,
+ <&rpmhpd_opp_turbo_l1>;
+ };
+ };
+ };
+...
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema
2023-12-18 11:31 ` [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema Dikshita Agarwal
@ 2023-12-18 16:10 ` Krzysztof Kozlowski
2023-12-18 18:17 ` Dmitry Baryshkov
1 sibling, 0 replies; 102+ messages in thread
From: Krzysztof Kozlowski @ 2023-12-18 16:10 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 12:31, Dikshita Agarwal wrote:
> Add a schema description for the iris video encoder/decoder
> on sm8550.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> .../bindings/media/qcom,sm8550-iris.yaml | 177 +++++++++++++++++++++
> 1 file changed, 177 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
NAK, I don't understand why we need to repeat it some many times.
Please use scripts/get_maintainers.pl to get a list of necessary people
and lists to CC. It might happen, that command when run on an older
kernel, gives you outdated entries. Therefore please be sure you base
your patches on recent Linux kernel.
You missed at least devicetree list (maybe more), so this won't be
tested by automated tooling. Performing review on untested code might be
a waste of time, thus I will skip this patch entirely till you follow
the process allowing the patch to be tested.
Please kindly resend and include all necessary To/Cc entries.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema
2023-12-18 11:31 ` [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema Dikshita Agarwal
2023-12-18 16:10 ` Krzysztof Kozlowski
@ 2023-12-18 18:17 ` Dmitry Baryshkov
1 sibling, 0 replies; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:17 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:31, Dikshita Agarwal wrote:
> Add a schema description for the iris video encoder/decoder
> on sm8550.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> .../bindings/media/qcom,sm8550-iris.yaml | 177 +++++++++++++++++++++
> 1 file changed, 177 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> new file mode 100644
> index 0000000..a3d9233
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> @@ -0,0 +1,177 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/qcom,sm8550-iris.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Qualcomm IRIS video encode and decode accelerators
> +
> +maintainers:
> + - Vikash Garodia <quic_vgarodia@quicinc.com>
> + - Dikshita Agarwal <quic_dikshita@quicinc.com>
> +
> +description:
> + The Iris video processing unit is a video encode and decode accelerator
> + present on Qualcomm platforms.
> +
> +properties:
> + compatible:
> + oneOf:
> + - enum:
> + - qcom,sm8550-iris
> +
> + reg:
> + maxItems: 2
> +
> + interrupts:
> + maxItems: 1
> +
> + power-domains:
> + minItems: 2
> + maxItems: 4
> +
> + power-domain-names:
> + oneOf:
> + - items: > + - const: iris-ctl
"iris" or even "venus"
> + - const: vcodec
"vcodec0"
> + - const: mxc
> + - const: mmcx
> +
> + operating-points-v2: true
> +
> + clocks:
> + maxItems: 3
> +
> + clock-names:
> + items:
> + - const: gcc_video_axi0
"iface"
> + - const: core_clk
Drop the _clk
> + - const: vcodec_core
"vcodec0_core' will be more Venus-compatible
> +
> + interconnects:
> + maxItems: 2
> +
> + interconnect-names:
> + items:
> + - const: iris-cnoc
"cpu-cfg"
> + - const: iris-ddr
"video-mem" to be closer to Venus
> +
> + memory-region:
> + maxItems: 1
> +
> + resets:
> + maxItems: 1
> +
> + reset-names:
> + items:
> + - const: video_axi_reset
Just 'bus'
> +
> + iommus:
> + maxItems: 2
> +
> + dma-coherent: true
> +
> + opp-table:
> + type: object
> +
> +required:
> + - compatible
> + - reg
> + - interrupts
> + - power-domains
> + - power-domain-names
> + - clocks
> + - clock-names
> + - interconnects
> + - interconnect-names
> + - memory-region
> + - resets
> + - reset-names
> + - iommus
> + - dma-coherent
> + - opp-table
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/clock/qcom,rpmh.h>
> + #include <dt-bindings/clock/qcom,sm8550-gcc.h>
> + #include <dt-bindings/clock/qcom,sm8450-videocc.h>
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> + #include <dt-bindings/interconnect/qcom,icc.h>
> + #include <dt-bindings/interconnect/qcom,sm8550-rpmh.h>
> + #include <dt-bindings/power/qcom-rpmpd.h>
> +
> + iris: video-codec@aa00000 {
> + compatible = "qcom,sm8550-iris";
> +
> + reg = <0 0x0aa00000 0 0xf0000>;
> + interrupts = <GIC_SPI 174 IRQ_TYPE_LEVEL_HIGH>;
> +
> + power-domains = <&videocc VIDEO_CC_MVS0C_GDSC>,
> + <&videocc VIDEO_CC_MVS0_GDSC>,
> + <&rpmhpd SM8550_MXC>,
> + <&rpmhpd SM8550_MMCX>;
> + power-domain-names = "iris-ctl", "vcodec", "mxc", "mmcx";
> + operating-points-v2 = <&iris_opp_table>;
> +
> + clocks = <&gcc GCC_VIDEO_AXI0_CLK>,
> + <&videocc VIDEO_CC_MVS0C_CLK>,
> + <&videocc VIDEO_CC_MVS0_CLK>;
> + clock-names = "gcc_video_axi0", "core_clk", "vcodec_core";
> +
> + interconnects = <&gem_noc MASTER_APPSS_PROC QCOM_ICC_TAG_ALWAYS
> + &config_noc SLAVE_VENUS_CFG QCOM_ICC_TAG_ALWAYS>,
> + <&mmss_noc MASTER_VIDEO QCOM_ICC_TAG_ALWAYS
> + &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>;
> + interconnect-names = "iris-cnoc", "iris-ddr";
> +
> + /* FW load region */
> + memory-region = <&video_mem>;
> +
> + resets = <&gcc GCC_VIDEO_AXI0_CLK_ARES>;
> + reset-names = "video_axi_reset";
> +
> + iommus = <&apps_smmu 0x1940 0x0000>,
> + <&apps_smmu 0x1947 0x0000>;
> + dma-coherent;
> +
> + status = "disabled";
> +
> + iris_opp_table: opp-table {
> + compatible = "operating-points-v2";
> +
> + opp-240000000 {
> + opp-hz = /bits/ 64 <240000000>;
> + required-opps = <&rpmhpd_opp_svs>,
> + <&rpmhpd_opp_low_svs>;
> + };
> +
> + opp-338000000 {
> + opp-hz = /bits/ 64 <338000000>;
> + required-opps = <&rpmhpd_opp_svs>,
> + <&rpmhpd_opp_svs>;
> + };
> +
> + opp-366000000 {
> + opp-hz = /bits/ 64 <366000000>;
> + required-opps = <&rpmhpd_opp_svs_l1>,
> + <&rpmhpd_opp_svs_l1>;
> + };
> +
> + opp-444000000 {
> + opp-hz = /bits/ 64 <444000000>;
> + required-opps = <&rpmhpd_opp_turbo>,
> + <&rpmhpd_opp_turbo>;
> + };
> +
> + opp-533333334 {
> + opp-hz = /bits/ 64 <533333334>;
> + required-opps = <&rpmhpd_opp_turbo_l1>,
> + <&rpmhpd_opp_turbo_l1>;
> + };
> + };
> + };
> +...
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 05/34] media: MAINTAINERS: Add Qualcomm Iris video accelerator driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (3 preceding siblings ...)
2023-12-18 11:31 ` [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 06/34] media: iris: register video device to platform driver Dikshita Agarwal
` (35 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Add an entry for Iris video encoder/decoder accelerator driver.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
MAINTAINERS | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 10a981a..a727144 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17980,6 +17980,17 @@ S: Maintained
F: Documentation/devicetree/bindings/regulator/vqmmc-ipq4019-regulator.yaml
F: drivers/regulator/vqmmc-ipq4019-regulator.c
+QUALCOMM IRIS VIDEO ACCELERATOR DRIVER
+M: Vikash Garodia <quic_vgarodia@quicinc.com>
+M: Dikshita Agarwal <quic_dikshita@quicinc.com>
+R: Abhinav Kumar <quic_abhinavk@quicinc.com>
+L: linux-media@vger.kernel.org
+L: linux-arm-msm@vger.kernel.org
+S: Maintained
+T: git git://linuxtv.org/media_tree.git
+F: Documentation/devicetree/bindings/media/qcom,*-iris.yaml
+F: drivers/media/platform/qcom/iris/
+
QUALCOMM NAND CONTROLLER DRIVER
M: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
L: linux-mtd@lists.infradead.org
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 06/34] media: iris: register video device to platform driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (4 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 05/34] media: MAINTAINERS: Add Qualcomm Iris video accelerator driver Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 18:40 ` Dmitry Baryshkov
2023-12-18 11:32 ` [PATCH v2 07/34] media: iris: initialize power resources Dikshita Agarwal
` (34 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Iris is a multi pipe based video acceleration hardware
block that offloads video stream encoding and decoding
from the application processor (AP). It supports H.264
and H.265 encoding and decoding, as well as VP9 decoding.
The AP communicates with hardware through a well defined
protocol which provides fine-grained and asynchronous
control over individual hardware features.
This patch introduces basic probe and remove functions.
It handles setting up a video device as well as registering
it with the V4L2 subsystem.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/Kconfig | 1 +
drivers/media/platform/qcom/Makefile | 1 +
drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +++
drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 34 +++++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 111 +++++++++++++++++++++
6 files changed, 163 insertions(+)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
index e94142f..7c88837 100644
--- a/drivers/media/platform/qcom/Kconfig
+++ b/drivers/media/platform/qcom/Kconfig
@@ -4,3 +4,4 @@ comment "Qualcomm media platform drivers"
source "drivers/media/platform/qcom/camss/Kconfig"
source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
+source "drivers/media/platform/qcom/vcodec/iris/Kconfig"
diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
index 3d2d82b..3c76969 100644
--- a/drivers/media/platform/qcom/Makefile
+++ b/drivers/media/platform/qcom/Makefile
@@ -1,3 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-y += camss/
obj-y += vcodec/venus/
+obj-y += vcodec/iris/
diff --git a/drivers/media/platform/qcom/vcodec/iris/Kconfig b/drivers/media/platform/qcom/vcodec/iris/Kconfig
new file mode 100644
index 0000000..850a5b4
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/Kconfig
@@ -0,0 +1,13 @@
+config VIDEO_QCOM_IRIS
+ tristate "Qualcomm Iris V4L2 encoder/decoder driver"
+ depends on VIDEO_DEV
+ depends on ARCH_QCOM
+ select QCOM_MDT_LOADER if ARCH_QCOM
+ select QCOM_SCM
+ select DMABUF_HEAPS
+ select DMABUF_HEAPS_SYSTEM
+ help
+ This is a V4L2 driver for Qualcomm Iris video accelerator
+ hardware. It accelerates encoding and decoding operations
+ on various Qualcomm SoCs.
+ To compile this driver as a module choose m here.
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
new file mode 100644
index 0000000..5536ae0
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -0,0 +1,3 @@
+iris-objs += iris_probe.o
+
+obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
new file mode 100644
index 0000000..ab7fcee
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_CORE_H_
+#define _IRIS_CORE_H_
+
+#include <linux/types.h>
+#include <media/v4l2-device.h>
+
+/**
+ * struct iris_core - holds core parameters valid for all instances
+ *
+ * @dev: reference to device structure
+ * @reg_base: IO memory base address
+ * @irq: iris irq
+ * @v4l2_dev: a holder for v4l2 device structure
+ * @vdev_dec: iris video device structure for decoder
+ * @v4l2_file_ops: iris v4l2 file ops
+ * @v4l2_ioctl_ops: iris v4l2 ioctl ops
+ */
+
+struct iris_core {
+ struct device *dev;
+ void __iomem *reg_base;
+ int irq;
+ struct v4l2_device v4l2_dev;
+ struct video_device *vdev_dec;
+ const struct v4l2_file_operations *v4l2_file_ops;
+ const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
+};
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
new file mode 100644
index 0000000..2e93118
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+
+#include "iris_core.h"
+
+static int iris_register_video_device(struct iris_core *core)
+{
+ struct video_device *vdev;
+ int ret;
+
+ vdev = video_device_alloc();
+ if (!vdev)
+ return -ENOMEM;
+
+ strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
+ vdev->release = video_device_release;
+ vdev->fops = core->v4l2_file_ops;
+ vdev->ioctl_ops = core->v4l2_ioctl_ops;
+ vdev->vfl_dir = VFL_DIR_M2M;
+ vdev->v4l2_dev = &core->v4l2_dev;
+ vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto err_vdev_release;
+
+ core->vdev_dec = vdev;
+ video_set_drvdata(vdev, core);
+
+ return ret;
+
+err_vdev_release:
+ video_device_release(vdev);
+
+ return ret;
+}
+
+static void iris_remove(struct platform_device *pdev)
+{
+ struct iris_core *core;
+
+ core = platform_get_drvdata(pdev);
+ if (!core)
+ return;
+
+ video_unregister_device(core->vdev_dec);
+
+ v4l2_device_unregister(&core->v4l2_dev);
+}
+
+static int iris_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct iris_core *core;
+ int ret;
+
+ core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
+ if (!core)
+ return -ENOMEM;
+ core->dev = dev;
+
+ core->reg_base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(core->reg_base))
+ return PTR_ERR(core->reg_base);
+
+ core->irq = platform_get_irq(pdev, 0);
+ if (core->irq < 0)
+ return core->irq;
+
+ ret = v4l2_device_register(dev, &core->v4l2_dev);
+ if (ret)
+ return ret;
+
+ ret = iris_register_video_device(core);
+ if (ret)
+ goto err_v4l2_unreg;
+
+ platform_set_drvdata(pdev, core);
+
+ return ret;
+
+err_v4l2_unreg:
+ v4l2_device_unregister(&core->v4l2_dev);
+
+ return ret;
+}
+
+static const struct of_device_id iris_dt_match[] = {
+ { .compatible = "qcom,sm8550-iris", },
+ { },
+};
+MODULE_DEVICE_TABLE(of, iris_dt_match);
+
+static struct platform_driver qcom_iris_driver = {
+ .probe = iris_probe,
+ .remove_new = iris_remove,
+ .driver = {
+ .name = "qcom-iris",
+ .of_match_table = iris_dt_match,
+ },
+};
+
+module_platform_driver(qcom_iris_driver);
+MODULE_DESCRIPTION("Qualcomm Iris video driver");
+MODULE_LICENSE("GPL");
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 06/34] media: iris: register video device to platform driver
2023-12-18 11:32 ` [PATCH v2 06/34] media: iris: register video device to platform driver Dikshita Agarwal
@ 2023-12-18 18:40 ` Dmitry Baryshkov
0 siblings, 0 replies; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:40 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:32, Dikshita Agarwal wrote:
> Iris is a multi pipe based video acceleration hardware
> block that offloads video stream encoding and decoding
> from the application processor (AP). It supports H.264
> and H.265 encoding and decoding, as well as VP9 decoding.
> The AP communicates with hardware through a well defined
> protocol which provides fine-grained and asynchronous
> control over individual hardware features.
>
> This patch introduces basic probe and remove functions.
Documentation/process/submitting-patches.rst
Other than that LGTM
> It handles setting up a video device as well as registering
> it with the V4L2 subsystem.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> drivers/media/platform/qcom/Kconfig | 1 +
> drivers/media/platform/qcom/Makefile | 1 +
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +++
> drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +
> .../media/platform/qcom/vcodec/iris/iris_core.h | 34 +++++++
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 111 +++++++++++++++++++++
> 6 files changed, 163 insertions(+)
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>
> diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
> index e94142f..7c88837 100644
> --- a/drivers/media/platform/qcom/Kconfig
> +++ b/drivers/media/platform/qcom/Kconfig
> @@ -4,3 +4,4 @@ comment "Qualcomm media platform drivers"
>
> source "drivers/media/platform/qcom/camss/Kconfig"
> source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> +source "drivers/media/platform/qcom/vcodec/iris/Kconfig"
> diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
> index 3d2d82b..3c76969 100644
> --- a/drivers/media/platform/qcom/Makefile
> +++ b/drivers/media/platform/qcom/Makefile
> @@ -1,3 +1,4 @@
> # SPDX-License-Identifier: GPL-2.0-only
> obj-y += camss/
> obj-y += vcodec/venus/
> +obj-y += vcodec/iris/
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Kconfig b/drivers/media/platform/qcom/vcodec/iris/Kconfig
> new file mode 100644
> index 0000000..850a5b4
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/Kconfig
> @@ -0,0 +1,13 @@
> +config VIDEO_QCOM_IRIS
> + tristate "Qualcomm Iris V4L2 encoder/decoder driver"
> + depends on VIDEO_DEV
> + depends on ARCH_QCOM
> + select QCOM_MDT_LOADER if ARCH_QCOM
> + select QCOM_SCM
> + select DMABUF_HEAPS
> + select DMABUF_HEAPS_SYSTEM
> + help
> + This is a V4L2 driver for Qualcomm Iris video accelerator
> + hardware. It accelerates encoding and decoding operations
> + on various Qualcomm SoCs.
> + To compile this driver as a module choose m here.
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
> new file mode 100644
> index 0000000..5536ae0
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
> @@ -0,0 +1,3 @@
> +iris-objs += iris_probe.o
> +
> +obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> new file mode 100644
> index 0000000..ab7fcee
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> @@ -0,0 +1,34 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _IRIS_CORE_H_
> +#define _IRIS_CORE_H_
> +
> +#include <linux/types.h>
> +#include <media/v4l2-device.h>
> +
> +/**
> + * struct iris_core - holds core parameters valid for all instances
> + *
> + * @dev: reference to device structure
> + * @reg_base: IO memory base address
> + * @irq: iris irq
> + * @v4l2_dev: a holder for v4l2 device structure
> + * @vdev_dec: iris video device structure for decoder
> + * @v4l2_file_ops: iris v4l2 file ops
> + * @v4l2_ioctl_ops: iris v4l2 ioctl ops
> + */
> +
> +struct iris_core {
> + struct device *dev;
> + void __iomem *reg_base;
> + int irq;
> + struct v4l2_device v4l2_dev;
> + struct video_device *vdev_dec;
> + const struct v4l2_file_operations *v4l2_file_ops;
> + const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
> +};
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> new file mode 100644
> index 0000000..2e93118
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> @@ -0,0 +1,111 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +
> +#include "iris_core.h"
> +
> +static int iris_register_video_device(struct iris_core *core)
> +{
> + struct video_device *vdev;
> + int ret;
> +
> + vdev = video_device_alloc();
> + if (!vdev)
> + return -ENOMEM;
> +
> + strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
> + vdev->release = video_device_release;
> + vdev->fops = core->v4l2_file_ops;
> + vdev->ioctl_ops = core->v4l2_ioctl_ops;
> + vdev->vfl_dir = VFL_DIR_M2M;
> + vdev->v4l2_dev = &core->v4l2_dev;
> + vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret)
> + goto err_vdev_release;
> +
> + core->vdev_dec = vdev;
> + video_set_drvdata(vdev, core);
> +
> + return ret;
> +
> +err_vdev_release:
> + video_device_release(vdev);
> +
> + return ret;
> +}
> +
> +static void iris_remove(struct platform_device *pdev)
> +{
> + struct iris_core *core;
> +
> + core = platform_get_drvdata(pdev);
> + if (!core)
> + return;
> +
> + video_unregister_device(core->vdev_dec);
> +
> + v4l2_device_unregister(&core->v4l2_dev);
> +}
> +
> +static int iris_probe(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct iris_core *core;
> + int ret;
> +
> + core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
> + if (!core)
> + return -ENOMEM;
> + core->dev = dev;
> +
> + core->reg_base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(core->reg_base))
> + return PTR_ERR(core->reg_base);
> +
> + core->irq = platform_get_irq(pdev, 0);
> + if (core->irq < 0)
> + return core->irq;
> +
> + ret = v4l2_device_register(dev, &core->v4l2_dev);
> + if (ret)
> + return ret;
> +
> + ret = iris_register_video_device(core);
> + if (ret)
> + goto err_v4l2_unreg;
> +
> + platform_set_drvdata(pdev, core);
> +
> + return ret;
> +
> +err_v4l2_unreg:
> + v4l2_device_unregister(&core->v4l2_dev);
> +
> + return ret;
> +}
> +
> +static const struct of_device_id iris_dt_match[] = {
> + { .compatible = "qcom,sm8550-iris", },
> + { },
> +};
> +MODULE_DEVICE_TABLE(of, iris_dt_match);
> +
> +static struct platform_driver qcom_iris_driver = {
> + .probe = iris_probe,
> + .remove_new = iris_remove,
> + .driver = {
> + .name = "qcom-iris",
> + .of_match_table = iris_dt_match,
> + },
> +};
> +
> +module_platform_driver(qcom_iris_driver);
> +MODULE_DESCRIPTION("Qualcomm Iris video driver");
> +MODULE_LICENSE("GPL");
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 07/34] media: iris: initialize power resources
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (5 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 06/34] media: iris: register video device to platform driver Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 15:09 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 08/34] media: iris: introduce state machine for iris core Dikshita Agarwal
` (33 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Add support for initializing Iris "resources", which are clocks,
interconnects, power domains, reset clocks, and clock frequencies
used for Iris hardware.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +-
.../media/platform/qcom/vcodec/iris/iris_core.h | 16 ++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 8 +
.../media/platform/qcom/vcodec/iris/resources.c | 232 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/resources.h | 36 ++++
5 files changed, 294 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 5536ae0..0748819 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -1,3 +1,4 @@
-iris-objs += iris_probe.o
+iris-objs += iris_probe.o \
+ resources.o
obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index ab7fcee..c2bc95d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -19,6 +19,14 @@
* @vdev_dec: iris video device structure for decoder
* @v4l2_file_ops: iris v4l2 file ops
* @v4l2_ioctl_ops: iris v4l2 ioctl ops
+ * @bus_tbl: table of iris buses
+ * @bus_count: count of iris buses
+ * @power_domain_tbl: table of iris power domains
+ * @pd_count: count of iris power domains
+ * @clock_tbl: table of iris clocks
+ * @clk_count: count of iris clocks
+ * @reset_tbl: table of iris reset clocks
+ * @reset_count: count of iris reset clocks
*/
struct iris_core {
@@ -29,6 +37,14 @@ struct iris_core {
struct video_device *vdev_dec;
const struct v4l2_file_operations *v4l2_file_ops;
const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
+ struct bus_info *bus_tbl;
+ u32 bus_count;
+ struct power_domain_info *power_domain_tbl;
+ u32 pd_count;
+ struct clock_info *clock_tbl;
+ u32 clk_count;
+ struct reset_info *reset_tbl;
+ u32 reset_count;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 2e93118..7bb9c92 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -8,6 +8,7 @@
#include <linux/platform_device.h>
#include "iris_core.h"
+#include "resources.h"
static int iris_register_video_device(struct iris_core *core)
{
@@ -73,6 +74,13 @@ static int iris_probe(struct platform_device *pdev)
if (core->irq < 0)
return core->irq;
+ ret = init_resources(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init resource failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = v4l2_device_register(dev, &core->v4l2_dev);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c
new file mode 100644
index 0000000..5aebbe4
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/resources.c
@@ -0,0 +1,232 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <dt-bindings/clock/qcom,sm8550-gcc.h>
+#include <dt-bindings/clock/qcom,sm8450-videocc.h>
+
+#include <linux/clk.h>
+#include <linux/interconnect.h>
+#include <linux/pm_domain.h>
+#include <linux/pm_opp.h>
+#include <linux/pm_runtime.h>
+#include <linux/reset.h>
+#include <linux/sort.h>
+
+#include "iris_core.h"
+#include "resources.h"
+
+static const struct bus_info plat_bus_table[] = {
+ { NULL, "iris-cnoc", 1000, 1000 },
+ { NULL, "iris-ddr", 1000, 15000000 },
+};
+
+static const char * const plat_pd_table[] = { "iris-ctl", "vcodec", NULL };
+#define PD_COUNT 2
+
+static const char * const plat_opp_pd_table[] = { "mxc", "mmcx", NULL };
+#define OPP_PD_COUNT 2
+
+static const struct clock_info plat_clk_table[] = {
+ { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0, 0 },
+ { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0, 0 },
+ { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1, 0 },
+};
+
+static const char * const plat_clk_reset_table[] = { "video_axi_reset", NULL };
+#define RESET_COUNT 1
+
+static void iris_pd_release(void *res)
+{
+ struct device *pd = (struct device *)res;
+
+ dev_pm_domain_detach(pd, true);
+}
+
+static int iris_pd_get(struct iris_core *core, struct power_domain_info *pdinfo)
+{
+ int ret;
+
+ pdinfo->genpd_dev = dev_pm_domain_attach_by_name(core->dev, pdinfo->name);
+ if (IS_ERR_OR_NULL(pdinfo->genpd_dev))
+ ret = PTR_ERR(pdinfo->genpd_dev) ? : -ENODATA;
+
+ ret = devm_add_action_or_reset(core->dev, iris_pd_release, (void *)pdinfo->genpd_dev);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+static void iris_opp_dl_release(void *res)
+{
+ struct device_link *link = (struct device_link *)res;
+
+ device_link_del(link);
+}
+
+static int iris_opp_dl_get(struct device *dev, struct device *supplier)
+{
+ u32 flag = DL_FLAG_RPM_ACTIVE | DL_FLAG_PM_RUNTIME | DL_FLAG_STATELESS;
+ struct device_link *link = NULL;
+ int ret;
+
+ link = device_link_add(dev, supplier, flag);
+ if (!link)
+ return -EINVAL;
+
+ ret = devm_add_action_or_reset(dev, iris_opp_dl_release, (void *)link);
+
+ return ret;
+}
+
+static int init_bus(struct iris_core *core)
+{
+ struct bus_info *binfo = NULL;
+ u32 i = 0;
+
+ core->bus_count = ARRAY_SIZE(plat_bus_table);
+ core->bus_tbl = devm_kzalloc(core->dev,
+ sizeof(struct bus_info) * core->bus_count,
+ GFP_KERNEL);
+ if (!core->bus_tbl)
+ return -ENOMEM;
+
+ for (i = 0; i < core->bus_count; i++) {
+ binfo = &core->bus_tbl[i];
+ binfo->name = plat_bus_table[i].name;
+ binfo->bw_min_kbps = plat_bus_table[i].bw_min_kbps;
+ binfo->bw_max_kbps = plat_bus_table[i].bw_max_kbps;
+ binfo->icc = devm_of_icc_get(core->dev, binfo->name);
+ if (IS_ERR(binfo->icc)) {
+ dev_err(core->dev,
+ "%s: failed to get bus: %s\n", __func__, binfo->name);
+ return PTR_ERR(binfo->icc);
+ }
+ }
+
+ return 0;
+}
+
+static int init_power_domains(struct iris_core *core)
+{
+ struct power_domain_info *pdinfo = NULL;
+ struct device **opp_vdevs = NULL;
+ int ret;
+ u32 i;
+
+ core->pd_count = PD_COUNT;
+ core->power_domain_tbl = devm_kzalloc(core->dev,
+ sizeof(struct power_domain_info) * core->pd_count,
+ GFP_KERNEL);
+ if (!core->power_domain_tbl)
+ return -ENOMEM;
+
+ for (i = 0; i < core->pd_count; i++) {
+ pdinfo = &core->power_domain_tbl[i];
+ pdinfo->name = plat_pd_table[i];
+ ret = iris_pd_get(core, pdinfo);
+ if (ret) {
+ dev_err(core->dev,
+ "%s: failed to get pd: %s\n", __func__, pdinfo->name);
+ return ret;
+ }
+ }
+
+ ret = devm_pm_opp_attach_genpd(core->dev, plat_opp_pd_table, &opp_vdevs);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < OPP_PD_COUNT; i++) {
+ ret = iris_opp_dl_get(core->dev, opp_vdevs[i]);
+ if (ret) {
+ dev_err(core->dev, "%s: failed to create dl: %s\n",
+ __func__, dev_name(opp_vdevs[i]));
+ return ret;
+ }
+ }
+
+ ret = devm_pm_opp_of_add_table(core->dev);
+ if (ret) {
+ dev_err(core->dev, "%s: failed to add opp table\n", __func__);
+ return ret;
+ }
+
+ return ret;
+}
+
+static int init_clocks(struct iris_core *core)
+{
+ struct clock_info *cinfo = NULL;
+ u32 i;
+
+ core->clk_count = ARRAY_SIZE(plat_clk_table);
+ core->clock_tbl = devm_kzalloc(core->dev,
+ sizeof(struct clock_info) * core->clk_count,
+ GFP_KERNEL);
+ if (!core->clock_tbl)
+ return -ENOMEM;
+
+ for (i = 0; i < core->clk_count; i++) {
+ cinfo = &core->clock_tbl[i];
+ cinfo->name = plat_clk_table[i].name;
+ cinfo->clk_id = plat_clk_table[i].clk_id;
+ cinfo->has_scaling = plat_clk_table[i].has_scaling;
+ cinfo->clk = devm_clk_get(core->dev, cinfo->name);
+ if (IS_ERR(cinfo->clk)) {
+ dev_err(core->dev,
+ "%s: failed to get clock: %s\n", __func__, cinfo->name);
+ return PTR_ERR(cinfo->clk);
+ }
+ }
+
+ return 0;
+}
+
+static int init_reset_clocks(struct iris_core *core)
+{
+ struct reset_info *rinfo = NULL;
+ u32 i = 0;
+
+ core->reset_count = RESET_COUNT;
+ core->reset_tbl = devm_kzalloc(core->dev,
+ sizeof(struct reset_info) * core->reset_count,
+ GFP_KERNEL);
+ if (!core->reset_tbl)
+ return -ENOMEM;
+
+ for (i = 0; i < core->reset_count; i++) {
+ rinfo = &core->reset_tbl[i];
+ rinfo->name = plat_clk_reset_table[i];
+ rinfo->rst = devm_reset_control_get(core->dev, rinfo->name);
+ if (IS_ERR(rinfo->rst)) {
+ dev_err(core->dev,
+ "%s: failed to get reset clock: %s\n", __func__, rinfo->name);
+ return PTR_ERR(rinfo->rst);
+ }
+ }
+
+ return 0;
+}
+
+int init_resources(struct iris_core *core)
+{
+ int ret;
+
+ ret = init_bus(core);
+ if (ret)
+ return ret;
+
+ ret = init_power_domains(core);
+ if (ret)
+ return ret;
+
+ ret = init_clocks(core);
+ if (ret)
+ return ret;
+
+ ret = init_reset_clocks(core);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h
new file mode 100644
index 0000000..d21bcc7e
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/resources.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _RESOURCES_H_
+#define _RESOURCES_H_
+
+struct bus_info {
+ struct icc_path *icc;
+ const char *name;
+ u32 bw_min_kbps;
+ u32 bw_max_kbps;
+};
+
+struct power_domain_info {
+ struct device *genpd_dev;
+ const char *name;
+};
+
+struct clock_info {
+ struct clk *clk;
+ const char *name;
+ u32 clk_id;
+ bool has_scaling;
+ u64 prev;
+};
+
+struct reset_info {
+ struct reset_control *rst;
+ const char *name;
+};
+
+int init_resources(struct iris_core *core);
+
+#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 07/34] media: iris: initialize power resources
2023-12-18 11:32 ` [PATCH v2 07/34] media: iris: initialize power resources Dikshita Agarwal
@ 2023-12-18 15:09 ` Konrad Dybcio
2023-12-20 8:04 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-18 15:09 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Add support for initializing Iris "resources", which are clocks,
> interconnects, power domains, reset clocks, and clock frequencies
> used for Iris hardware.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> + ret = init_resources(core);
> + if (ret) {
> + dev_err_probe(core->dev, ret,
> + "%s: init resource failed with %d\n", __func__, ret);
> + return ret;
You're supposed to return dev_err_probe, it propagates the errors
this way
Also, I think __func__ is excessive, throughout the code. You can
very quickly grep for the error messages, which are quite unique.
[...]
> +
> +static const struct bus_info plat_bus_table[] = {
> + { NULL, "iris-cnoc", 1000, 1000 },
> + { NULL, "iris-ddr", 1000, 15000000 },
> +};
> +
> +static const char * const plat_pd_table[] = { "iris-ctl", "vcodec", NULL };
> +#define PD_COUNT 2
> +
> +static const char * const plat_opp_pd_table[] = { "mxc", "mmcx", NULL };
> +#define OPP_PD_COUNT 2
> +
> +static const struct clock_info plat_clk_table[] = {
> + { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0, 0 },
> + { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0, 0 },
> + { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1, 0 },
> +};
> +
> +static const char * const plat_clk_reset_table[] = { "video_axi_reset", NULL };
> +#define RESET_COUNT 1
Are you sure this won't change between platforms?
[...]
> +static int init_bus(struct iris_core *core)
> +{
> + struct bus_info *binfo = NULL;
> + u32 i = 0;
no need to initialize
[...]
> +static int init_clocks(struct iris_core *core)
> +{
> + struct clock_info *cinfo = NULL;
> + u32 i;
> +
> + core->clk_count = ARRAY_SIZE(plat_clk_table);
> + core->clock_tbl = devm_kzalloc(core->dev,
> + sizeof(struct clock_info) * core->clk_count,
> + GFP_KERNEL);
> + if (!core->clock_tbl)
> + return -ENOMEM;
> +
> + for (i = 0; i < core->clk_count; i++) {
> + cinfo = &core->clock_tbl[i];
> + cinfo->name = plat_clk_table[i].name;
> + cinfo->clk_id = plat_clk_table[i].clk_id;
> + cinfo->has_scaling = plat_clk_table[i].has_scaling;
> + cinfo->clk = devm_clk_get(core->dev, cinfo->name);
> + if (IS_ERR(cinfo->clk)) {
> + dev_err(core->dev,
> + "%s: failed to get clock: %s\n", __func__, cinfo->name);
> + return PTR_ERR(cinfo->clk);
> + }
> + }
Are you not going to use OPP for scaling the main RPMhPD with the core
clock?
> +
> + return 0;
> +}
> +
> +static int init_reset_clocks(struct iris_core *core)
init_resets
'reset clocks' is an old downstream concept
> +{
> + struct reset_info *rinfo = NULL;
> + u32 i = 0;
unnecessary initializations
[...]
> +
> +int init_resources(struct iris_core *core)
> +{
> + int ret;
> +
> + ret = init_bus(core);
> + if (ret)
> + return ret;
> +
> + ret = init_power_domains(core);
> + if (ret)
> + return ret;
> +
> + ret = init_clocks(core);
> + if (ret)
> + return ret;
> +
> + ret = init_reset_clocks(core);
> +
> + return ret;
return init_reset_clocks(core);
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h
> new file mode 100644
> index 0000000..d21bcc7e
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/resources.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _RESOURCES_H_
> +#define _RESOURCES_H_
> +
> +struct bus_info {
> + struct icc_path *icc;
> + const char *name;
> + u32 bw_min_kbps;
> + u32 bw_max_kbps;
u64?
> +};
> +
> +struct power_domain_info {
> + struct device *genpd_dev;
> + const char *name;
> +};
> +
> +struct clock_info {
> + struct clk *clk;
> + const char *name;
I'm not sure why you need it
> + u32 clk_id;
Or this
> + bool has_scaling;
Or this
you could probably do something like this:
struct big_iris_struct {
[...]
struct clk *core_clk;
struct clk *memory_clk;
struct clk *some_important_scaled_clock;
struct clk_bulk_data less_important_nonscaling_clocks[X]
}
and then make use of all of the great common upstream APIs to manage
them!
> + u64 prev;
> +};
> +
> +struct reset_info {
> + struct reset_control *rst;
> + const char *name;
this seems a bit excessive as well
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 07/34] media: iris: initialize power resources
2023-12-18 15:09 ` Konrad Dybcio
@ 2023-12-20 8:04 ` Dikshita Agarwal
2024-01-03 13:45 ` Konrad Dybcio
0 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:04 UTC (permalink / raw)
To: Konrad Dybcio, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/2023 8:39 PM, Konrad Dybcio wrote:
> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>> Add support for initializing Iris "resources", which are clocks,
>> interconnects, power domains, reset clocks, and clock frequencies
>> used for Iris hardware.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
> [...]
>
>> + ret = init_resources(core);
>> + if (ret) {
>> + dev_err_probe(core->dev, ret,
>> + "%s: init resource failed with %d\n", __func__, ret);
>> + return ret;
> You're supposed to return dev_err_probe, it propagates the errors
> this way
>
Sure, fix this.
> Also, I think __func__ is excessive, throughout the code. You can
> very quickly grep for the error messages, which are quite unique.
>
Ok, will remove __func__
> [...]
>
>> +
>> +static const struct bus_info plat_bus_table[] = {
>> + { NULL, "iris-cnoc", 1000, 1000 },
>> + { NULL, "iris-ddr", 1000, 15000000 },
>> +};
>> +
>> +static const char * const plat_pd_table[] = { "iris-ctl", "vcodec", NULL };
>> +#define PD_COUNT 2
>> +
>> +static const char * const plat_opp_pd_table[] = { "mxc", "mmcx", NULL };
>> +#define OPP_PD_COUNT 2
>> +
>> +static const struct clock_info plat_clk_table[] = {
>> + { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0, 0 },
>> + { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0, 0 },
>> + { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1, 0 },
>> +};
>> +
>> +static const char * const plat_clk_reset_table[] = { "video_axi_reset", NULL };
>> +#define RESET_COUNT 1
> Are you sure this won't change between platforms?
> [...]
>
yes, these will change, but since at this point in the patches, I have not
introduced platform specific file, added these tables here.
I am moving these to platform specific file when I introduce that in
patch-13 [1].
[1]
https://patchwork.kernel.org/project/linux-media/patch/1702899149-21321-14-git-send-email-quic_dikshita@quicinc.com/
>> +static int init_bus(struct iris_core *core)
>> +{
>> + struct bus_info *binfo = NULL;
>> + u32 i = 0;
> no need to initialize
>
Sure, will fix.
> [...]
>
>> +static int init_clocks(struct iris_core *core)
>> +{
>> + struct clock_info *cinfo = NULL;
>> + u32 i;
>> +
>> + core->clk_count = ARRAY_SIZE(plat_clk_table);
>> + core->clock_tbl = devm_kzalloc(core->dev,
>> + sizeof(struct clock_info) * core->clk_count,
>> + GFP_KERNEL);
>> + if (!core->clock_tbl)
>> + return -ENOMEM;
>> +
>> + for (i = 0; i < core->clk_count; i++) {
>> + cinfo = &core->clock_tbl[i];
>> + cinfo->name = plat_clk_table[i].name;
>> + cinfo->clk_id = plat_clk_table[i].clk_id;
>> + cinfo->has_scaling = plat_clk_table[i].has_scaling;
>> + cinfo->clk = devm_clk_get(core->dev, cinfo->name);
>> + if (IS_ERR(cinfo->clk)) {
>> + dev_err(core->dev,
>> + "%s: failed to get clock: %s\n", __func__, cinfo->name);
>> + return PTR_ERR(cinfo->clk);
>> + }
>> + }
> Are you not going to use OPP for scaling the main RPMhPD with the core
> clock?
>
We are using OPP for scaling the vcodec clk.
Could you please elaborate you query here, may be I didn't understand fully.
>> +
>> + return 0;
>> +}
>> +
>> +static int init_reset_clocks(struct iris_core *core)
> init_resets
>
> 'reset clocks' is an old downstream concept
>
Sure, I can rename it.
>> +{
>> + struct reset_info *rinfo = NULL;
>> + u32 i = 0;
> unnecessary initializations
>
Sure, will fix.
> [...]
>
>> +
>> +int init_resources(struct iris_core *core)
>> +{
>> + int ret;
>> +
>> + ret = init_bus(core);
>> + if (ret)
>> + return ret;
>> +
>> + ret = init_power_domains(core);
>> + if (ret)
>> + return ret;
>> +
>> + ret = init_clocks(core);
>> + if (ret)
>> + return ret;
>> +
>> + ret = init_reset_clocks(core);
>> +
>> + return ret;
> return init_reset_clocks(core);
>
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h
>> new file mode 100644
>> index 0000000..d21bcc7e
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/iris/resources.h
>> @@ -0,0 +1,36 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
>> + */
>> +
>> +#ifndef _RESOURCES_H_
>> +#define _RESOURCES_H_
>> +
>> +struct bus_info {
>> + struct icc_path *icc;
>> + const char *name;
>> + u32 bw_min_kbps;
>> + u32 bw_max_kbps;
> u64?
>
Will check and do the needful.
>> +};
>> +
>> +struct power_domain_info {
>> + struct device *genpd_dev;
>> + const char *name;
>> +};
>> +
>> +struct clock_info {
>> + struct clk *clk;
>> + const char *name;
> I'm not sure why you need it
>
>> + u32 clk_id;
> Or this
>
>> + bool has_scaling;
> Or this
>
> you could probably do something like this:
>
> struct big_iris_struct {
> [...]
> struct clk *core_clk;
> struct clk *memory_clk;
> struct clk *some_important_scaled_clock;
> struct clk_bulk_data less_important_nonscaling_clocks[X]
> }
>
> and then make use of all of the great common upstream APIs to manage
> them!
>
Will explore this and get back.
>> + u64 prev;
>> +};
>> +
>> +struct reset_info {
>> + struct reset_control *rst;
>> + const char *name;
> this seems a bit excessive as well
>
> Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 07/34] media: iris: initialize power resources
2023-12-20 8:04 ` Dikshita Agarwal
@ 2024-01-03 13:45 ` Konrad Dybcio
0 siblings, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2024-01-03 13:45 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 20.12.2023 09:04, Dikshita Agarwal wrote:
>
>
> On 12/18/2023 8:39 PM, Konrad Dybcio wrote:
>> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>>> Add support for initializing Iris "resources", which are clocks,
>>> interconnects, power domains, reset clocks, and clock frequencies
>>> used for Iris hardware.
>>>
>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>> ---
[...]
>>> +
>>> + for (i = 0; i < core->clk_count; i++) {
>>> + cinfo = &core->clock_tbl[i];
>>> + cinfo->name = plat_clk_table[i].name;
>>> + cinfo->clk_id = plat_clk_table[i].clk_id;
>>> + cinfo->has_scaling = plat_clk_table[i].has_scaling;
>>> + cinfo->clk = devm_clk_get(core->dev, cinfo->name);
>>> + if (IS_ERR(cinfo->clk)) {
>>> + dev_err(core->dev,
>>> + "%s: failed to get clock: %s\n", __func__, cinfo->name);
>>> + return PTR_ERR(cinfo->clk);
>>> + }
>>> + }
>> Are you not going to use OPP for scaling the main RPMhPD with the core
>> clock?
>>
> We are using OPP for scaling the vcodec clk.
> Could you please elaborate you query here, may be I didn't understand fully.
It's just that this approach of scanning everything we know and
expect about the clock seems a bit unnecessary.. Going with the
approach I suggested below (i.e. separate struct clk for important
ones like core or mem clock) simplify this to the point where you
just set opp_set_rate on the imporant ones and you can throw
clk_bulk_ operations at the ones that simply need to be en/disabled.
Konrad
[...]
>>> +
>>> +struct power_domain_info {
>>> + struct device *genpd_dev;
>>> + const char *name;
>>> +};
>>> +
>>> +struct clock_info {
>>> + struct clk *clk;
>>> + const char *name;
>> I'm not sure why you need it
>>
>>> + u32 clk_id;
>> Or this
>>
>>> + bool has_scaling;
>> Or this
>>
>> you could probably do something like this:
>>
>> struct big_iris_struct {
>> [...]
>> struct clk *core_clk;
>> struct clk *memory_clk;
>> struct clk *some_important_scaled_clock;
>> struct clk_bulk_data less_important_nonscaling_clocks[X]
>> }
>>
>> and then make use of all of the great common upstream APIs to manage
>> them!
>>
> Will explore this and get back.
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 08/34] media: iris: introduce state machine for iris core
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (6 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 07/34] media: iris: initialize power resources Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 18:46 ` Dmitry Baryshkov
2023-12-18 11:32 ` [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication Dikshita Agarwal
` (32 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Introduces different states for core. State transitions
are defined, based on which they would be allowed/rejected/ignored.
IRIS_CORE_DEINIT - default state.
IRIS_CORE_INIT_WAIT - wait state for video core to complete
initialization.
IRIS_CORE_INIT - core state with core initialized. FW loaded and
HW brought out of reset, shared queues established between host
driver and firmware.
IRIS_CORE_ERROR - error state.
|
v
-----------
+--->| DEINIT |<---+
| ----------- |
| | |
| v |
| ----------- |
| |INIT_WAIT| |
| ----------- |
| / \ |
| / \ |
| / \ |
| v v v
----------- ----------.
| INIT |-->| ERROR |
----------- ----------.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +-
.../media/platform/qcom/vcodec/iris/iris_core.h | 4 ++
.../media/platform/qcom/vcodec/iris/iris_state.c | 64 ++++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++++
4 files changed, 92 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 0748819..12a74de 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -1,4 +1,5 @@
iris-objs += iris_probe.o \
- resources.o
+ resources.o \
+ iris_state.o
obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index c2bc95d..56a5b7d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -9,6 +9,8 @@
#include <linux/types.h>
#include <media/v4l2-device.h>
+#include "iris_state.h"
+
/**
* struct iris_core - holds core parameters valid for all instances
*
@@ -27,6 +29,7 @@
* @clk_count: count of iris clocks
* @reset_tbl: table of iris reset clocks
* @reset_count: count of iris reset clocks
+ * @state: current state of core
*/
struct iris_core {
@@ -45,6 +48,7 @@ struct iris_core {
u32 clk_count;
struct reset_info *reset_tbl;
u32 reset_count;
+ enum iris_core_state state;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
new file mode 100644
index 0000000..22557af
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_state.h"
+
+#define IRIS_STATE(name)[IRIS_CORE_##name] = "CORE_"#name
+
+static const char * const core_state_names[] = {
+ IRIS_STATE(DEINIT),
+ IRIS_STATE(INIT_WAIT),
+ IRIS_STATE(INIT),
+ IRIS_STATE(ERROR),
+};
+
+#undef IRIS_STATE
+
+bool core_in_valid_state(struct iris_core *core)
+{
+ return core->state == IRIS_CORE_INIT ||
+ core->state == IRIS_CORE_INIT_WAIT;
+}
+
+static const char *core_state_name(enum iris_core_state state)
+{
+ if ((unsigned int)state < ARRAY_SIZE(core_state_names))
+ return core_state_names[state];
+
+ return "UNKNOWN_STATE";
+}
+
+static bool iris_allow_core_state_change(struct iris_core *core,
+ enum iris_core_state req_state)
+{
+ if (core->state == IRIS_CORE_DEINIT)
+ return req_state == IRIS_CORE_INIT_WAIT || req_state == IRIS_CORE_ERROR;
+ else if (core->state == IRIS_CORE_INIT_WAIT)
+ return req_state == IRIS_CORE_INIT || req_state == IRIS_CORE_ERROR;
+ else if (core->state == IRIS_CORE_INIT)
+ return req_state == IRIS_CORE_DEINIT || req_state == IRIS_CORE_ERROR;
+ else if (core->state == IRIS_CORE_ERROR)
+ return req_state == IRIS_CORE_DEINIT;
+
+ dev_warn(core->dev, "core state change %s -> %s is not allowed\n",
+ core_state_name(core->state), core_state_name(req_state));
+
+ return false;
+}
+
+int iris_change_core_state(struct iris_core *core,
+ enum iris_core_state request_state)
+{
+ if (core->state == request_state)
+ return 0;
+
+ if (!iris_allow_core_state_change(core, request_state))
+ return -EINVAL;
+
+ core->state = request_state;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
new file mode 100644
index 0000000..ee20842
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_STATE_H_
+#define _IRIS_STATE_H_
+
+struct iris_core;
+
+enum iris_core_state {
+ IRIS_CORE_DEINIT,
+ IRIS_CORE_INIT_WAIT,
+ IRIS_CORE_INIT,
+ IRIS_CORE_ERROR,
+};
+
+bool core_in_valid_state(struct iris_core *core);
+int iris_change_core_state(struct iris_core *core,
+ enum iris_core_state request_state);
+
+#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 08/34] media: iris: introduce state machine for iris core
2023-12-18 11:32 ` [PATCH v2 08/34] media: iris: introduce state machine for iris core Dikshita Agarwal
@ 2023-12-18 18:46 ` Dmitry Baryshkov
2023-12-20 8:05 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:46 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:32, Dikshita Agarwal wrote:
> Introduces different states for core. State transitions
> are defined, based on which they would be allowed/rejected/ignored.
>
> IRIS_CORE_DEINIT - default state.
> IRIS_CORE_INIT_WAIT - wait state for video core to complete
> initialization.
> IRIS_CORE_INIT - core state with core initialized. FW loaded and
> HW brought out of reset, shared queues established between host
> driver and firmware.
> IRIS_CORE_ERROR - error state.
>
> |
> v
> -----------
> +--->| DEINIT |<---+
> | ----------- |
> | | |
> | v |
> | ----------- |
> | |INIT_WAIT| |
> | ----------- |
> | / \ |
> | / \ |
> | / \ |
> | v v v
> ----------- ----------.
> | INIT |-->| ERROR |
> ----------- ----------.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
Can we see users for this API please? Otherwise it's just a dead code.
> ---
> drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +-
> .../media/platform/qcom/vcodec/iris/iris_core.h | 4 ++
> .../media/platform/qcom/vcodec/iris/iris_state.c | 64 ++++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++++
> 4 files changed, 92 insertions(+), 1 deletion(-)
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
>
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
> index 0748819..12a74de 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
> @@ -1,4 +1,5 @@
> iris-objs += iris_probe.o \
> - resources.o
> + resources.o \
> + iris_state.o
>
> obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> index c2bc95d..56a5b7d 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> @@ -9,6 +9,8 @@
> #include <linux/types.h>
> #include <media/v4l2-device.h>
>
> +#include "iris_state.h"
> +
> /**
> * struct iris_core - holds core parameters valid for all instances
> *
> @@ -27,6 +29,7 @@
> * @clk_count: count of iris clocks
> * @reset_tbl: table of iris reset clocks
> * @reset_count: count of iris reset clocks
> + * @state: current state of core
> */
>
> struct iris_core {
> @@ -45,6 +48,7 @@ struct iris_core {
> u32 clk_count;
> struct reset_info *reset_tbl;
> u32 reset_count;
> + enum iris_core_state state;
> };
>
> #endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
> new file mode 100644
> index 0000000..22557af
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
> @@ -0,0 +1,64 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include "iris_core.h"
> +#include "iris_state.h"
> +
> +#define IRIS_STATE(name)[IRIS_CORE_##name] = "CORE_"#name
Inline this please.
> +
> +static const char * const core_state_names[] = {
> + IRIS_STATE(DEINIT),
> + IRIS_STATE(INIT_WAIT),
> + IRIS_STATE(INIT),
> + IRIS_STATE(ERROR),
> +};
> +
> +#undef IRIS_STATE
> +
> +bool core_in_valid_state(struct iris_core *core)
So, even in your driver you have global names? That's really ugh. Please
fix them.
> +{
> + return core->state == IRIS_CORE_INIT ||
> + core->state == IRIS_CORE_INIT_WAIT;
> +}
> +
> +static const char *core_state_name(enum iris_core_state state)
> +{
> + if ((unsigned int)state < ARRAY_SIZE(core_state_names))
> + return core_state_names[state];
> +
> + return "UNKNOWN_STATE";
> +}
> +
> +static bool iris_allow_core_state_change(struct iris_core *core,
> + enum iris_core_state req_state)
> +{
> + if (core->state == IRIS_CORE_DEINIT)
> + return req_state == IRIS_CORE_INIT_WAIT || req_state == IRIS_CORE_ERROR;
> + else if (core->state == IRIS_CORE_INIT_WAIT)
> + return req_state == IRIS_CORE_INIT || req_state == IRIS_CORE_ERROR;
> + else if (core->state == IRIS_CORE_INIT)
> + return req_state == IRIS_CORE_DEINIT || req_state == IRIS_CORE_ERROR;
> + else if (core->state == IRIS_CORE_ERROR)
> + return req_state == IRIS_CORE_DEINIT;
> +
> + dev_warn(core->dev, "core state change %s -> %s is not allowed\n",
> + core_state_name(core->state), core_state_name(req_state));
> +
> + return false;
> +}
> +
> +int iris_change_core_state(struct iris_core *core,
> + enum iris_core_state request_state)
> +{
> + if (core->state == request_state)
> + return 0;
> +
> + if (!iris_allow_core_state_change(core, request_state))
> + return -EINVAL;
> +
> + core->state = request_state;
> +
> + return 0;
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
> new file mode 100644
> index 0000000..ee20842
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _IRIS_STATE_H_
> +#define _IRIS_STATE_H_
> +
> +struct iris_core;
> +
> +enum iris_core_state {
> + IRIS_CORE_DEINIT,
> + IRIS_CORE_INIT_WAIT,
> + IRIS_CORE_INIT,
> + IRIS_CORE_ERROR,
> +};
> +
> +bool core_in_valid_state(struct iris_core *core);
> +int iris_change_core_state(struct iris_core *core,
> + enum iris_core_state request_state);
> +
> +#endif
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 08/34] media: iris: introduce state machine for iris core
2023-12-18 18:46 ` Dmitry Baryshkov
@ 2023-12-20 8:05 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:05 UTC (permalink / raw)
To: Dmitry Baryshkov, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 12:16 AM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:32, Dikshita Agarwal wrote:
>> Introduces different states for core. State transitions
>> are defined, based on which they would be allowed/rejected/ignored.
>>
>> IRIS_CORE_DEINIT - default state.
>> IRIS_CORE_INIT_WAIT - wait state for video core to complete
>> initialization.
>> IRIS_CORE_INIT - core state with core initialized. FW loaded and
>> HW brought out of reset, shared queues established between host
>> driver and firmware.
>> IRIS_CORE_ERROR - error state.
>>
>> |
>> v
>> -----------
>> +--->| DEINIT |<---+
>> | ----------- |
>> | | |
>> | v |
>> | ----------- |
>> | |INIT_WAIT| |
>> | ----------- |
>> | / \ |
>> | / \ |
>> | / \ |
>> | v v v
>> ----------- ----------.
>> | INIT |-->| ERROR |
>> ----------- ----------.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>
> Can we see users for this API please? Otherwise it's just a dead code.
>
This is like a preparation patch to add the core state.
These APIs are used in patch-10[1] onward.
[1]:
https://patchwork.kernel.org/project/linux-media/patch/1702899149-21321-11-git-send-email-quic_dikshita@quicinc.com/
>> ---
>> drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +-
>> .../media/platform/qcom/vcodec/iris/iris_core.h | 4 ++
>> .../media/platform/qcom/vcodec/iris/iris_state.c | 64
>> ++++++++++++++++++++++
>> .../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++++
>> 4 files changed, 92 insertions(+), 1 deletion(-)
>> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
>> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
>>
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile
>> b/drivers/media/platform/qcom/vcodec/iris/Makefile
>> index 0748819..12a74de 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
>> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
>> @@ -1,4 +1,5 @@
>> iris-objs += iris_probe.o \
>> - resources.o
>> + resources.o \
>> + iris_state.o
>> obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> index c2bc95d..56a5b7d 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> @@ -9,6 +9,8 @@
>> #include <linux/types.h>
>> #include <media/v4l2-device.h>
>> +#include "iris_state.h"
>> +
>> /**
>> * struct iris_core - holds core parameters valid for all instances
>> *
>> @@ -27,6 +29,7 @@
>> * @clk_count: count of iris clocks
>> * @reset_tbl: table of iris reset clocks
>> * @reset_count: count of iris reset clocks
>> + * @state: current state of core
>> */
>> struct iris_core {
>> @@ -45,6 +48,7 @@ struct iris_core {
>> u32 clk_count;
>> struct reset_info *reset_tbl;
>> u32 reset_count;
>> + enum iris_core_state state;
>> };
>> #endif
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
>> b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
>> new file mode 100644
>> index 0000000..22557af
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
>> @@ -0,0 +1,64 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#include "iris_core.h"
>> +#include "iris_state.h"
>> +
>> +#define IRIS_STATE(name)[IRIS_CORE_##name] = "CORE_"#name
>
> Inline this please.
>
>> +
>> +static const char * const core_state_names[] = {
>> + IRIS_STATE(DEINIT),
>> + IRIS_STATE(INIT_WAIT),
>> + IRIS_STATE(INIT),
>> + IRIS_STATE(ERROR),
>> +};
>> +
>> +#undef IRIS_STATE
>> +
>> +bool core_in_valid_state(struct iris_core *core)
>
> So, even in your driver you have global names? That's really ugh. Please
> fix them.
>
Sure will fix these names.
>> +{
>> + return core->state == IRIS_CORE_INIT ||
>> + core->state == IRIS_CORE_INIT_WAIT;
>> +}
>> +
>> +static const char *core_state_name(enum iris_core_state state)
>> +{
>> + if ((unsigned int)state < ARRAY_SIZE(core_state_names))
>> + return core_state_names[state];
>> +
>> + return "UNKNOWN_STATE";
>> +}
>> +
>> +static bool iris_allow_core_state_change(struct iris_core *core,
>> + enum iris_core_state req_state)
>> +{
>> + if (core->state == IRIS_CORE_DEINIT)
>> + return req_state == IRIS_CORE_INIT_WAIT || req_state ==
>> IRIS_CORE_ERROR;
>> + else if (core->state == IRIS_CORE_INIT_WAIT)
>> + return req_state == IRIS_CORE_INIT || req_state == IRIS_CORE_ERROR;
>> + else if (core->state == IRIS_CORE_INIT)
>> + return req_state == IRIS_CORE_DEINIT || req_state ==
>> IRIS_CORE_ERROR;
>> + else if (core->state == IRIS_CORE_ERROR)
>> + return req_state == IRIS_CORE_DEINIT;
>> +
>> + dev_warn(core->dev, "core state change %s -> %s is not allowed\n",
>> + core_state_name(core->state), core_state_name(req_state));
>> +
>> + return false;
>> +}
>> +
>> +int iris_change_core_state(struct iris_core *core,
>> + enum iris_core_state request_state)
>> +{
>> + if (core->state == request_state)
>> + return 0;
>> +
>> + if (!iris_allow_core_state_change(core, request_state))
>> + return -EINVAL;
>> +
>> + core->state = request_state;
>> +
>> + return 0;
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
>> b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
>> new file mode 100644
>> index 0000000..ee20842
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
>> @@ -0,0 +1,22 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#ifndef _IRIS_STATE_H_
>> +#define _IRIS_STATE_H_
>> +
>> +struct iris_core;
>> +
>> +enum iris_core_state {
>> + IRIS_CORE_DEINIT,
>> + IRIS_CORE_INIT_WAIT,
>> + IRIS_CORE_INIT,
>> + IRIS_CORE_ERROR,
>> +};
>> +
>> +bool core_in_valid_state(struct iris_core *core);
>> +int iris_change_core_state(struct iris_core *core,
>> + enum iris_core_state request_state);
>> +
>> +#endif
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (7 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 08/34] media: iris: introduce state machine for iris core Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 18:46 ` Dmitry Baryshkov
2023-12-18 21:33 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 10/34] media: iris: add PIL functionality for video firmware Dikshita Agarwal
` (31 subsequent siblings)
40 siblings, 2 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Shared queues are used for communication between driver and firmware.
There are 3 types of queues:
Command queue - driver to write any command to firmware.
Message queue - firmware to send any response to driver.
Debug queue - firmware to write debug message.
Above queues are initialized and configured to firmware during probe.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 2 ++
.../media/platform/qcom/vcodec/iris/iris_core.h | 11 ++++++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 31 ++++++++++++++++++++++
3 files changed, 44 insertions(+)
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 12a74de..59798e5d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -1,3 +1,5 @@
+iris-objs += ../hfi_queue.o
+
iris-objs += iris_probe.o \
resources.o \
iris_state.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 56a5b7d..77124f9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -9,6 +9,7 @@
#include <linux/types.h>
#include <media/v4l2-device.h>
+#include "../hfi_queue.h"
#include "iris_state.h"
/**
@@ -30,6 +31,11 @@
* @reset_tbl: table of iris reset clocks
* @reset_count: count of iris reset clocks
* @state: current state of core
+ * @iface_q_table: Interface queue table memory
+ * @command_queue: shared interface queue to send commands to firmware
+ * @message_queue: shared interface queue to receive responses from firmware
+ * @debug_queue: shared interface queue to receive debug info from firmware
+ * @sfr: SFR register memory
*/
struct iris_core {
@@ -49,6 +55,11 @@ struct iris_core {
struct reset_info *reset_tbl;
u32 reset_count;
enum iris_core_state state;
+ struct mem_desc iface_q_table;
+ struct iface_q_info command_queue;
+ struct iface_q_info message_queue;
+ struct iface_q_info debug_queue;
+ struct mem_desc sfr;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 7bb9c92..fd349a3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -7,6 +7,7 @@
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include "../hfi_queue.h"
#include "iris_core.h"
#include "resources.h"
@@ -50,6 +51,10 @@ static void iris_remove(struct platform_device *pdev)
if (!core)
return;
+ hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
+ &core->command_queue, &core->message_queue,
+ &core->debug_queue);
+
video_unregister_device(core->vdev_dec);
v4l2_device_unregister(&core->v4l2_dev);
@@ -59,6 +64,7 @@ static int iris_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct iris_core *core;
+ u64 dma_mask;
int ret;
core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
@@ -91,8 +97,33 @@ static int iris_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, core);
+ /*
+ * Specify the max value of address space, which can be used
+ * for buffer transactions.
+ */
+ dma_mask = DMA_BIT_MASK(32);
+ dma_mask &= ~BIT(29);
+
+ ret = dma_set_mask_and_coherent(dev, dma_mask);
+ if (ret)
+ goto err_vdev_unreg;
+
+ dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
+ dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));
+
+ ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
+ &core->command_queue, &core->message_queue,
+ &core->debug_queue, core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: interface queues init failed\n", __func__);
+ goto err_vdev_unreg;
+ }
+
return ret;
+err_vdev_unreg:
+ iris_unregister_video_device(core);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication
2023-12-18 11:32 ` [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication Dikshita Agarwal
@ 2023-12-18 18:46 ` Dmitry Baryshkov
2023-12-20 8:05 ` Dikshita Agarwal
2023-12-18 21:33 ` Konrad Dybcio
1 sibling, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:46 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:32, Dikshita Agarwal wrote:
> Shared queues are used for communication between driver and firmware.
> There are 3 types of queues:
> Command queue - driver to write any command to firmware.
> Message queue - firmware to send any response to driver.
> Debug queue - firmware to write debug message.
>
> Above queues are initialized and configured to firmware during probe.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> drivers/media/platform/qcom/vcodec/iris/Makefile | 2 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 11 ++++++++
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 31 ++++++++++++++++++++++
> 3 files changed, 44 insertions(+)
>
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
> index 12a74de..59798e5d 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
> @@ -1,3 +1,5 @@
> +iris-objs += ../hfi_queue.o
> +
> iris-objs += iris_probe.o \
> resources.o \
> iris_state.o
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> index 56a5b7d..77124f9 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> @@ -9,6 +9,7 @@
> #include <linux/types.h>
> #include <media/v4l2-device.h>
>
> +#include "../hfi_queue.h"
> #include "iris_state.h"
>
> /**
> @@ -30,6 +31,11 @@
> * @reset_tbl: table of iris reset clocks
> * @reset_count: count of iris reset clocks
> * @state: current state of core
> + * @iface_q_table: Interface queue table memory
> + * @command_queue: shared interface queue to send commands to firmware
> + * @message_queue: shared interface queue to receive responses from firmware
> + * @debug_queue: shared interface queue to receive debug info from firmware
> + * @sfr: SFR register memory
> */
>
> struct iris_core {
> @@ -49,6 +55,11 @@ struct iris_core {
> struct reset_info *reset_tbl;
> u32 reset_count;
> enum iris_core_state state;
> + struct mem_desc iface_q_table;
> + struct iface_q_info command_queue;
> + struct iface_q_info message_queue;
> + struct iface_q_info debug_queue;
> + struct mem_desc sfr;
> };
>
> #endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> index 7bb9c92..fd349a3 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> @@ -7,6 +7,7 @@
> #include <linux/of_device.h>
> #include <linux/platform_device.h>
>
> +#include "../hfi_queue.h"
> #include "iris_core.h"
> #include "resources.h"
>
> @@ -50,6 +51,10 @@ static void iris_remove(struct platform_device *pdev)
> if (!core)
> return;
>
> + hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
> + &core->command_queue, &core->message_queue,
> + &core->debug_queue);
> +
> video_unregister_device(core->vdev_dec);
>
> v4l2_device_unregister(&core->v4l2_dev);
> @@ -59,6 +64,7 @@ static int iris_probe(struct platform_device *pdev)
> {
> struct device *dev = &pdev->dev;
> struct iris_core *core;
> + u64 dma_mask;
> int ret;
>
> core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
> @@ -91,8 +97,33 @@ static int iris_probe(struct platform_device *pdev)
>
> platform_set_drvdata(pdev, core);
>
> + /*
> + * Specify the max value of address space, which can be used
> + * for buffer transactions.
> + */
> + dma_mask = DMA_BIT_MASK(32);
> + dma_mask &= ~BIT(29);
> +
> + ret = dma_set_mask_and_coherent(dev, dma_mask);
> + if (ret)
> + goto err_vdev_unreg;
> +
> + dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
> + dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));
This isn't related to queues.
> +
> + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
> + &core->command_queue, &core->message_queue,
> + &core->debug_queue, core);
> + if (ret) {
> + dev_err_probe(core->dev, ret,
> + "%s: interface queues init failed\n", __func__);
> + goto err_vdev_unreg;
> + }
> +
> return ret;
>
> +err_vdev_unreg:
> + iris_unregister_video_device(core);
> err_v4l2_unreg:
> v4l2_device_unregister(&core->v4l2_dev);
>
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication
2023-12-18 18:46 ` Dmitry Baryshkov
@ 2023-12-20 8:05 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:05 UTC (permalink / raw)
To: Dmitry Baryshkov, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 12:16 AM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:32, Dikshita Agarwal wrote:
>> Shared queues are used for communication between driver and firmware.
>> There are 3 types of queues:
>> Command queue - driver to write any command to firmware.
>> Message queue - firmware to send any response to driver.
>> Debug queue - firmware to write debug message.
>>
>> Above queues are initialized and configured to firmware during probe.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>> drivers/media/platform/qcom/vcodec/iris/Makefile | 2 ++
>> .../media/platform/qcom/vcodec/iris/iris_core.h | 11 ++++++++
>> .../media/platform/qcom/vcodec/iris/iris_probe.c | 31
>> ++++++++++++++++++++++
>> 3 files changed, 44 insertions(+)
>>
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile
>> b/drivers/media/platform/qcom/vcodec/iris/Makefile
>> index 12a74de..59798e5d 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
>> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
>> @@ -1,3 +1,5 @@
>> +iris-objs += ../hfi_queue.o
>> +
>> iris-objs += iris_probe.o \
>> resources.o \
>> iris_state.o
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> index 56a5b7d..77124f9 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> @@ -9,6 +9,7 @@
>> #include <linux/types.h>
>> #include <media/v4l2-device.h>
>> +#include "../hfi_queue.h"
>> #include "iris_state.h"
>> /**
>> @@ -30,6 +31,11 @@
>> * @reset_tbl: table of iris reset clocks
>> * @reset_count: count of iris reset clocks
>> * @state: current state of core
>> + * @iface_q_table: Interface queue table memory
>> + * @command_queue: shared interface queue to send commands to firmware
>> + * @message_queue: shared interface queue to receive responses from
>> firmware
>> + * @debug_queue: shared interface queue to receive debug info from firmware
>> + * @sfr: SFR register memory
>> */
>> struct iris_core {
>> @@ -49,6 +55,11 @@ struct iris_core {
>> struct reset_info *reset_tbl;
>> u32 reset_count;
>> enum iris_core_state state;
>> + struct mem_desc iface_q_table;
>> + struct iface_q_info command_queue;
>> + struct iface_q_info message_queue;
>> + struct iface_q_info debug_queue;
>> + struct mem_desc sfr;
>> };
>> #endif
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> index 7bb9c92..fd349a3 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> @@ -7,6 +7,7 @@
>> #include <linux/of_device.h>
>> #include <linux/platform_device.h>
>> +#include "../hfi_queue.h"
>> #include "iris_core.h"
>> #include "resources.h"
>> @@ -50,6 +51,10 @@ static void iris_remove(struct platform_device *pdev)
>> if (!core)
>> return;
>> + hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
>> + &core->command_queue, &core->message_queue,
>> + &core->debug_queue);
>> +
>> video_unregister_device(core->vdev_dec);
>> v4l2_device_unregister(&core->v4l2_dev);
>> @@ -59,6 +64,7 @@ static int iris_probe(struct platform_device *pdev)
>> {
>> struct device *dev = &pdev->dev;
>> struct iris_core *core;
>> + u64 dma_mask;
>> int ret;
>> core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
>> @@ -91,8 +97,33 @@ static int iris_probe(struct platform_device *pdev)
>> platform_set_drvdata(pdev, core);
>> + /*
>> + * Specify the max value of address space, which can be used
>> + * for buffer transactions.
>> + */
>> + dma_mask = DMA_BIT_MASK(32);
>> + dma_mask &= ~BIT(29);
>> +
>> + ret = dma_set_mask_and_coherent(dev, dma_mask);
>> + if (ret)
>> + goto err_vdev_unreg;
>> +
>> + dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
>> + dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));
>
> This isn't related to queues.
>
Using this DMA mask, we define the upper limit of memory to be used to
allocate the queues, hence this is added with this patch.
>> +
>> + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
>> + &core->command_queue, &core->message_queue,
>> + &core->debug_queue, core);
>> + if (ret) {
>> + dev_err_probe(core->dev, ret,
>> + "%s: interface queues init failed\n", __func__);
>> + goto err_vdev_unreg;
>> + }
>> +
>> return ret;
>> +err_vdev_unreg:
>> + iris_unregister_video_device(core);
>> err_v4l2_unreg:
>> v4l2_device_unregister(&core->v4l2_dev);
>>
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication
2023-12-18 11:32 ` [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication Dikshita Agarwal
2023-12-18 18:46 ` Dmitry Baryshkov
@ 2023-12-18 21:33 ` Konrad Dybcio
2023-12-20 8:05 ` Dikshita Agarwal
1 sibling, 1 reply; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-18 21:33 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/23 12:32, Dikshita Agarwal wrote:
> Shared queues are used for communication between driver and firmware.
> There are 3 types of queues:
> Command queue - driver to write any command to firmware.
> Message queue - firmware to send any response to driver.
> Debug queue - firmware to write debug message.
>
> Above queues are initialized and configured to firmware during probe.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
> + &core->command_queue, &core->message_queue,
> + &core->debug_queue, core);
> + if (ret) {
> + dev_err_probe(core->dev, ret,
> + "%s: interface queues init failed\n", __func__);
> + goto err_vdev_unreg;
> + }
> +
> return ret;
Like before, you're suppose to return dev_err_probe
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication
2023-12-18 21:33 ` Konrad Dybcio
@ 2023-12-20 8:05 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:05 UTC (permalink / raw)
To: Konrad Dybcio, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 3:03 AM, Konrad Dybcio wrote:
>
>
> On 12/18/23 12:32, Dikshita Agarwal wrote:
>> Shared queues are used for communication between driver and firmware.
>> There are 3 types of queues:
>> Command queue - driver to write any command to firmware.
>> Message queue - firmware to send any response to driver.
>> Debug queue - firmware to write debug message.
>>
>> Above queues are initialized and configured to firmware during probe.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
> [...]
>
>> + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
>> + &core->command_queue, &core->message_queue,
>> + &core->debug_queue, core);
>> + if (ret) {
>> + dev_err_probe(core->dev, ret,
>> + "%s: interface queues init failed\n", __func__);
>> + goto err_vdev_unreg;
>> + }
>> +
>> return ret;
> Like before, you're suppose to return dev_err_probe
Sure, will fix all such instances.
>
> Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 10/34] media: iris: add PIL functionality for video firmware
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (8 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 21:40 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets Dikshita Agarwal
` (30 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Load/unload firmware in memory via mdt loader.
Firmware loading is part of core initialization
and unloading is part of core de-initialization.
This also changes the core states accordingly.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 10 ++-
.../media/platform/qcom/vcodec/iris/iris_core.c | 70 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_core.h | 8 +++
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 15 +++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 13 ++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 72 ++++++++++++++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 14 +++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 19 +++++-
.../media/platform/qcom/vcodec/iris/iris_state.c | 9 ++-
9 files changed, 225 insertions(+), 5 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 59798e5d..74bd344 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -1,7 +1,11 @@
-iris-objs += ../hfi_queue.o
+iris-objs += ../hfi_queue.o ../firmware.o
iris-objs += iris_probe.o \
- resources.o \
- iris_state.o
+ iris_state.o \
+ iris_core.o \
+ iris_state.o \
+ iris_helpers.o \
+ iris_hfi.o \
+ resources.o
obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
new file mode 100644
index 0000000..ba8960d
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_state.h"
+
+static int iris_core_deinit_locked(struct iris_core *core)
+{
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (core->state == IRIS_CORE_DEINIT)
+ return 0;
+
+ iris_hfi_core_deinit(core);
+
+ iris_change_core_state(core, IRIS_CORE_DEINIT);
+
+ return ret;
+}
+
+int iris_core_deinit(struct iris_core *core)
+{
+ int ret;
+
+ mutex_lock(&core->lock);
+ ret = iris_core_deinit_locked(core);
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_core_init(struct iris_core *core)
+{
+ int ret = 0;
+
+ mutex_lock(&core->lock);
+ if (core_in_valid_state(core)) {
+ goto unlock;
+ } else if (core->state == IRIS_CORE_ERROR) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (iris_change_core_state(core, IRIS_CORE_INIT_WAIT)) {
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = iris_hfi_core_init(core);
+ if (ret) {
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ dev_err(core->dev, "%s: core init failed\n", __func__);
+ iris_core_deinit_locked(core);
+ goto unlock;
+ }
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 77124f9..2740ff1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -11,6 +11,7 @@
#include "../hfi_queue.h"
#include "iris_state.h"
+#include "resources.h"
/**
* struct iris_core - holds core parameters valid for all instances
@@ -36,6 +37,8 @@
* @message_queue: shared interface queue to receive responses from firmware
* @debug_queue: shared interface queue to receive debug info from firmware
* @sfr: SFR register memory
+ * @lock: a lock for this strucure
+ * @use_tz: a flag that suggests presence of trustzone
*/
struct iris_core {
@@ -60,6 +63,11 @@ struct iris_core {
struct iface_q_info message_queue;
struct iface_q_info debug_queue;
struct mem_desc sfr;
+ struct mutex lock; /* lock for core structure */
+ unsigned int use_tz;
};
+int iris_core_init(struct iris_core *core);
+int iris_core_deinit(struct iris_core *core);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
new file mode 100644
index 0000000..22c706a
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_helpers.h"
+
+int check_core_lock(struct iris_core *core)
+{
+ bool fatal = !mutex_is_locked(&core->lock);
+
+ WARN_ON(fatal);
+
+ return fatal ? -EINVAL : 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
new file mode 100644
index 0000000..314a8d75
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_HELPERS_H_
+#define _IRIS_HELPERS_H_
+
+#include "iris_core.h"
+
+int check_core_lock(struct iris_core *core);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
new file mode 100644
index 0000000..4f51a8c
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -0,0 +1,72 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "../firmware.h"
+#include "iris_helpers.h"
+#include "iris_hfi.h"
+
+#define CP_START 0
+#define CP_SIZE 0x25800000
+#define CP_NONPIXEL_START 0x01000000
+#define CP_NONPIXEL_SIZE 0x24800000
+
+#define FW_NAME "vpu30_4v.mbn"
+#define IRIS_PAS_ID 9
+
+int iris_hfi_core_init(struct iris_core *core)
+{
+ phys_addr_t mem_phys = 0;
+ size_t mem_size = 0;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
+ &core->command_queue, &core->message_queue,
+ &core->debug_queue, core);
+ if (ret)
+ goto error;
+
+ core->use_tz = use_tz(core->dev);
+ if (!core->use_tz)
+ goto error;
+
+ ret = load_fw(core->dev, FW_NAME, &mem_phys, &mem_size,
+ IRIS_PAS_ID, core->use_tz);
+ if (ret)
+ goto error;
+
+ ret = auth_reset_fw(IRIS_PAS_ID);
+ if (ret)
+ goto error;
+
+ ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START,
+ CP_NONPIXEL_SIZE, IRIS_PAS_ID);
+
+ return ret;
+
+error:
+ dev_err(core->dev, "%s(): failed with ret %d\n", __func__, ret);
+
+ return ret;
+}
+
+int iris_hfi_core_deinit(struct iris_core *core)
+{
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (core->state == IRIS_CORE_DEINIT)
+ return 0;
+
+ unload_fw(IRIS_PAS_ID);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
new file mode 100644
index 0000000..fcf9f28
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_HFI_H_
+#define _IRIS_HFI_H_
+
+#include "iris_core.h"
+
+int iris_hfi_core_init(struct iris_core *core);
+int iris_hfi_core_deinit(struct iris_core *core);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index fd349a3..f39b4aa 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -51,6 +51,7 @@ static void iris_remove(struct platform_device *pdev)
if (!core)
return;
+ iris_core_deinit(core);
hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
&core->command_queue, &core->message_queue,
&core->debug_queue);
@@ -58,6 +59,9 @@ static void iris_remove(struct platform_device *pdev)
video_unregister_device(core->vdev_dec);
v4l2_device_unregister(&core->v4l2_dev);
+
+ mutex_destroy(&core->lock);
+ core->state = IRIS_CORE_DEINIT;
}
static int iris_probe(struct platform_device *pdev)
@@ -72,6 +76,9 @@ static int iris_probe(struct platform_device *pdev)
return -ENOMEM;
core->dev = dev;
+ core->state = IRIS_CORE_DEINIT;
+ mutex_init(&core->lock);
+
core->reg_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(core->reg_base))
return PTR_ERR(core->reg_base);
@@ -120,10 +127,20 @@ static int iris_probe(struct platform_device *pdev)
goto err_vdev_unreg;
}
+ ret = iris_core_init(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret, "%s: core init failed\n", __func__);
+ goto err_queue_deinit;
+ }
+
return ret;
+err_queue_deinit:
+ hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
+ &core->command_queue, &core->message_queue,
+ &core->debug_queue);
err_vdev_unreg:
- iris_unregister_video_device(core);
+ video_unregister_device(core->vdev_dec);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 22557af..83bbc6b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -4,6 +4,7 @@
*/
#include "iris_core.h"
+#include "iris_helpers.h"
#include "iris_state.h"
#define IRIS_STATE(name)[IRIS_CORE_##name] = "CORE_"#name
@@ -52,6 +53,12 @@ static bool iris_allow_core_state_change(struct iris_core *core,
int iris_change_core_state(struct iris_core *core,
enum iris_core_state request_state)
{
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
if (core->state == request_state)
return 0;
@@ -60,5 +67,5 @@ int iris_change_core_state(struct iris_core *core,
core->state = request_state;
- return 0;
+ return ret;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 10/34] media: iris: add PIL functionality for video firmware
2023-12-18 11:32 ` [PATCH v2 10/34] media: iris: add PIL functionality for video firmware Dikshita Agarwal
@ 2023-12-18 21:40 ` Konrad Dybcio
2023-12-20 8:15 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-18 21:40 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/23 12:32, Dikshita Agarwal wrote:
> Load/unload firmware in memory via mdt loader.
> Firmware loading is part of core initialization
> and unloading is part of core de-initialization.
> This also changes the core states accordingly.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> +
> +#include "iris_core.h"
> +#include "iris_helpers.h"
> +#include "iris_hfi.h"
> +#include "iris_state.h"
> +
> +static int iris_core_deinit_locked(struct iris_core *core)
I suppose you meant to call this something like _nolock, as
you're calling it with a lock around it
> +{
> + int ret;
> +
> + ret = check_core_lock(core);
> + if (ret)
> + return ret;
> +
> + if (core->state == IRIS_CORE_DEINIT)
> + return 0;
> +
> + iris_hfi_core_deinit(core);
> +
> + iris_change_core_state(core, IRIS_CORE_DEINIT);
You're casually ignoring the return value of the two
above funcs here :/
> +
> + return ret;
> +}
> +
> +int iris_core_deinit(struct iris_core *core)
> +{
> + int ret;
> +
> + mutex_lock(&core->lock);
> + ret = iris_core_deinit_locked(core);
> + mutex_unlock(&core->lock);
> +
> + return ret;
> +}
> +
> +int iris_core_init(struct iris_core *core)
> +{
> + int ret = 0;
> +
> + mutex_lock(&core->lock);
You may be interested in scoped mutexes
> + if (core_in_valid_state(core)) {
> + goto unlock;
> + } else if (core->state == IRIS_CORE_ERROR) {
> + ret = -EINVAL;
> + goto unlock;
> + }
> +
> + if (iris_change_core_state(core, IRIS_CORE_INIT_WAIT)) {
> + iris_change_core_state(core, IRIS_CORE_ERROR);
> + ret = -EINVAL;
> + goto unlock;
> + }
> +
> + ret = iris_hfi_core_init(core);
> + if (ret) {
> + iris_change_core_state(core, IRIS_CORE_ERROR);
> + dev_err(core->dev, "%s: core init failed\n", __func__);
__func__ still seems overly verbose in my eyes
[...]
> +
> +int check_core_lock(struct iris_core *core)
> +{
> + bool fatal = !mutex_is_locked(&core->lock);
> +
> + WARN_ON(fatal);
> +
> + return fatal ? -EINVAL : 0;
You can simply inline this:
if (WARN_ON(!mutex_is_locked(&core->lock))
return -EINVAL;
[...]
> +#define CP_START 0
> +#define CP_SIZE 0x25800000
> +#define CP_NONPIXEL_START 0x01000000
> +#define CP_NONPIXEL_SIZE 0x24800000
> +
> +#define FW_NAME "vpu30_4v.mbn"
This doesn't scale, at all.
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 10/34] media: iris: add PIL functionality for video firmware
2023-12-18 21:40 ` Konrad Dybcio
@ 2023-12-20 8:15 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:15 UTC (permalink / raw)
To: Konrad Dybcio, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 3:10 AM, Konrad Dybcio wrote:
>
>
> On 12/18/23 12:32, Dikshita Agarwal wrote:
>> Load/unload firmware in memory via mdt loader.
>> Firmware loading is part of core initialization
>> and unloading is part of core de-initialization.
>> This also changes the core states accordingly.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
> [...]
>
>> +
>> +#include "iris_core.h"
>> +#include "iris_helpers.h"
>> +#include "iris_hfi.h"
>> +#include "iris_state.h"
>> +
>> +static int iris_core_deinit_locked(struct iris_core *core)
> I suppose you meant to call this something like _nolock, as
> you're calling it with a lock around it
>
We are trying to coney that this particular API should be called with
locked context only.
doesn't _nolock mean other way? please correct if my understanding is wrong.
>> +{
>> + int ret;
>> +
>> + ret = check_core_lock(core);
>> + if (ret)
>> + return ret;
>> +
>> + if (core->state == IRIS_CORE_DEINIT)
>> + return 0;
>> +
>> + iris_hfi_core_deinit(core);
>> +
>> + iris_change_core_state(core, IRIS_CORE_DEINIT);
> You're casually ignoring the return value of the two
> above funcs here :/
>
Right, since this is the tear-down sequence, we don't care of the return
value here.
>> +
>> + return ret;
>> +}
>> +
>> +int iris_core_deinit(struct iris_core *core)
>> +{
>> + int ret;
>> +
>> + mutex_lock(&core->lock);
>> + ret = iris_core_deinit_locked(core);
>> + mutex_unlock(&core->lock);
>> +
>> + return ret;
>> +}
>> +
>> +int iris_core_init(struct iris_core *core)
>> +{
>> + int ret = 0;
>> +
>> + mutex_lock(&core->lock);
> You may be interested in scoped mutexes
>
Will explore this.
>> + if (core_in_valid_state(core)) {
>> + goto unlock;
>> + } else if (core->state == IRIS_CORE_ERROR) {
>> + ret = -EINVAL;
>> + goto unlock;
>> + }
>> +
>> + if (iris_change_core_state(core, IRIS_CORE_INIT_WAIT)) {
>> + iris_change_core_state(core, IRIS_CORE_ERROR);
>> + ret = -EINVAL;
>> + goto unlock;
>> + }
>> +
>> + ret = iris_hfi_core_init(core);
>> + if (ret) {
>> + iris_change_core_state(core, IRIS_CORE_ERROR);
>> + dev_err(core->dev, "%s: core init failed\n", __func__);
> __func__ still seems overly verbose in my eyes
>
Sure, will remove such instances.
> [...]
>
>> +
>> +int check_core_lock(struct iris_core *core)
>> +{
>> + bool fatal = !mutex_is_locked(&core->lock);
>> +
>> + WARN_ON(fatal);
>> +
>> + return fatal ? -EINVAL : 0;
> You can simply inline this:
>
> if (WARN_ON(!mutex_is_locked(&core->lock))
> return -EINVAL;
>
Thanks for the suggestion, will update this.
> [...]
>> +#define CP_START 0
>> +#define CP_SIZE 0x25800000
>> +#define CP_NONPIXEL_START 0x01000000
>> +#define CP_NONPIXEL_SIZE 0x24800000
>> +
>> +#define FW_NAME "vpu30_4v.mbn"
> This doesn't scale, at all.
>
As mentioned in previous patches, we have not introduced platform specific
file yet, when I introduce that in later patch, this will be moved to
platform specific file.
> Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (9 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 10/34] media: iris: add PIL functionality for video firmware Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 21:50 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling Dikshita Agarwal
` (29 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Host firmware interface (HFI) is well defined set of interfaces
for communication between host driver and firmware.
The command and responses are exchanged in form of packets.
One or multiple packets are grouped under packet header.
Each packet has packet type which describes the specific HFI
and payload which holds the corresponding value for that HFI.
Sys_init is the first packets sent to firmware, which initializes
the firmware. Sys_image_version packet is to get the firmware
version string.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 15 +++
.../media/platform/qcom/vcodec/iris/iris_core.h | 10 ++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 69 +++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 129 +++++++++++++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 71 ++++++++++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 5 +
7 files changed, 300 insertions(+)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 74bd344..a2d5d74 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -6,6 +6,7 @@ iris-objs += iris_probe.o \
iris_state.o \
iris_helpers.o \
iris_hfi.o \
+ iris_hfi_packet.o \
resources.o
obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
new file mode 100644
index 0000000..4c48c90
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _HFI_DEFINES_H_
+#define _HFI_DEFINES_H_
+
+#define HFI_VIDEO_ARCH_LX 0x1
+
+#define HFI_CMD_INIT 0x01000001
+
+#define HFI_PROP_IMAGE_VERSION 0x03000001
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 2740ff1..de0cfef 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -39,6 +39,11 @@
* @sfr: SFR register memory
* @lock: a lock for this strucure
* @use_tz: a flag that suggests presence of trustzone
+ * @packet: pointer to packet from driver to fw
+ * @packet_size: size of packet
+ * @sys_init_id: id of sys init packet
+ * @header_id: id of packet header
+ * @packet_id: id of packet
*/
struct iris_core {
@@ -65,6 +70,11 @@ struct iris_core {
struct mem_desc sfr;
struct mutex lock; /* lock for core structure */
unsigned int use_tz;
+ u8 *packet;
+ u32 packet_size;
+ u32 sys_init_id;
+ u32 header_id;
+ u32 packet_id;
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 4f51a8c..fe16448 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -4,8 +4,67 @@
*/
#include "../firmware.h"
+#include "../hfi_queue.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
+
+static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
+{
+ struct iface_q_info *q_info;
+ struct hfi_header *header;
+ u32 packet_size, rx_req;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ q_info = &core->command_queue;
+ if (!q_info || !q_info->q_array.kernel_vaddr || !pkt) {
+ dev_err(core->dev, "cannot write to shared CMD Q's\n");
+ return -ENODATA;
+ }
+
+ header = pkt;
+ packet_size = header->size;
+
+ if (!write_queue(q_info, pkt, packet_size, &rx_req)) {
+ dev_err(core->dev, "queue full\n");
+ return -ENODATA;
+ }
+
+ return ret;
+}
+
+static int sys_init(struct iris_core *core)
+{
+ int ret;
+
+ ret = hfi_packet_sys_init(core, core->packet, core->packet_size);
+ if (ret)
+ return ret;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+
+ return ret;
+}
+
+static int sys_image_version(struct iris_core *core)
+{
+ int ret;
+
+ ret = hfi_packet_image_version(core, core->packet, core->packet_size);
+ if (ret)
+ return ret;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+
+ return ret;
+}
#define CP_START 0
#define CP_SIZE 0x25800000
@@ -46,6 +105,16 @@ int iris_hfi_core_init(struct iris_core *core)
ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START,
CP_NONPIXEL_SIZE, IRIS_PAS_ID);
+ if (ret)
+ goto error;
+
+ ret = sys_init(core);
+ if (ret)
+ goto error;
+
+ ret = sys_image_version(core);
+ if (ret)
+ goto error;
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
new file mode 100644
index 0000000..73bba07
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_hfi_packet.h"
+#include "hfi_defines.h"
+
+static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
+ u32 header_id)
+{
+ struct hfi_header *hdr = (struct hfi_header *)packet;
+
+ if (!packet || packet_size < sizeof(*hdr))
+ return -EINVAL;
+
+ memset(hdr, 0, sizeof(*hdr));
+
+ hdr->size = sizeof(*hdr);
+ hdr->session_id = session_id;
+ hdr->header_id = header_id;
+ hdr->num_packets = 0;
+
+ return 0;
+}
+
+static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
+ u32 pkt_flags, u32 payload_type, u32 port,
+ u32 packet_id, void *payload, u32 payload_size)
+{
+ struct hfi_header *hdr;
+ struct hfi_packet *pkt;
+ u32 pkt_size;
+
+ if (!packet)
+ return -EINVAL;
+
+ hdr = (struct hfi_header *)packet;
+ if (hdr->size < sizeof(*hdr))
+ return -EINVAL;
+
+ pkt = (struct hfi_packet *)(packet + hdr->size);
+ pkt_size = sizeof(*pkt) + payload_size;
+ if (packet_size < hdr->size + pkt_size)
+ return -EINVAL;
+
+ memset(pkt, 0, pkt_size);
+ pkt->size = pkt_size;
+ pkt->type = pkt_type;
+ pkt->flags = pkt_flags;
+ pkt->payload_info = payload_type;
+ pkt->port = port;
+ pkt->packet_id = packet_id;
+ if (payload_size)
+ memcpy((u8 *)pkt + sizeof(*pkt),
+ payload, payload_size);
+
+ hdr->num_packets++;
+ hdr->size += pkt->size;
+
+ return 0;
+}
+
+int hfi_packet_sys_init(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ u32 payload = 0;
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0,
+ core->header_id++);
+ if (ret)
+ goto error;
+
+ payload = HFI_VIDEO_ARCH_LX;
+ core->sys_init_id = core->packet_id++;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_CMD_INIT,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->sys_init_id,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ dev_err(core->dev, "%s: create sys init packet failed\n", __func__);
+
+ return ret;
+}
+
+int hfi_packet_image_version(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0,
+ core->header_id++);
+ if (ret)
+ goto error;
+
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_IMAGE_VERSION,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_GET_PROPERTY),
+ HFI_PAYLOAD_NONE,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ NULL, 0);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ dev_err(core->dev, "%s: create image version packet failed\n", __func__);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
new file mode 100644
index 0000000..e36612c
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_HFI_PACKET_H_
+#define _IRIS_HFI_PACKET_H_
+
+struct hfi_header {
+ u32 size;
+ u32 session_id;
+ u32 header_id;
+ u32 reserved[4];
+ u32 num_packets;
+};
+
+struct hfi_packet {
+ u32 size;
+ u32 type;
+ u32 flags;
+ u32 payload_info;
+ u32 port;
+ u32 packet_id;
+ u32 reserved[2];
+};
+
+enum hfi_packet_host_flags {
+ HFI_HOST_FLAGS_NONE = 0x00000000,
+ HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001,
+ HFI_HOST_FLAGS_RESPONSE_REQUIRED = 0x00000002,
+ HFI_HOST_FLAGS_NON_DISCARDABLE = 0x00000004,
+ HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008,
+};
+
+enum hfi_packet_firmware_flags {
+ HFI_FW_FLAGS_NONE = 0x00000000,
+ HFI_FW_FLAGS_SUCCESS = 0x00000001,
+ HFI_FW_FLAGS_INFORMATION = 0x00000002,
+ HFI_FW_FLAGS_SESSION_ERROR = 0x00000004,
+ HFI_FW_FLAGS_SYSTEM_ERROR = 0x00000008,
+};
+
+enum hfi_packet_payload_info {
+ HFI_PAYLOAD_NONE = 0x00000000,
+ HFI_PAYLOAD_U32 = 0x00000001,
+ HFI_PAYLOAD_S32 = 0x00000002,
+ HFI_PAYLOAD_U64 = 0x00000003,
+ HFI_PAYLOAD_S64 = 0x00000004,
+ HFI_PAYLOAD_STRUCTURE = 0x00000005,
+ HFI_PAYLOAD_BLOB = 0x00000006,
+ HFI_PAYLOAD_STRING = 0x00000007,
+ HFI_PAYLOAD_Q16 = 0x00000008,
+ HFI_PAYLOAD_U32_ENUM = 0x00000009,
+ HFI_PAYLOAD_32_PACKED = 0x0000000a,
+ HFI_PAYLOAD_U32_ARRAY = 0x0000000b,
+ HFI_PAYLOAD_S32_ARRAY = 0x0000000c,
+ HFI_PAYLOAD_64_PACKED = 0x0000000d,
+};
+
+enum hfi_packet_port_type {
+ HFI_PORT_NONE = 0x00000000,
+ HFI_PORT_BITSTREAM = 0x00000001,
+ HFI_PORT_RAW = 0x00000002,
+};
+
+int hfi_packet_sys_init(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);
+int hfi_packet_image_version(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index f39b4aa..570c64e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -79,6 +79,11 @@ static int iris_probe(struct platform_device *pdev)
core->state = IRIS_CORE_DEINIT;
mutex_init(&core->lock);
+ core->packet_size = IFACEQ_CORE_PKT_SIZE;
+ core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL);
+ if (!core->packet)
+ return -ENOMEM;
+
core->reg_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(core->reg_base))
return PTR_ERR(core->reg_base);
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets
2023-12-18 11:32 ` [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets Dikshita Agarwal
@ 2023-12-18 21:50 ` Konrad Dybcio
0 siblings, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-18 21:50 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/23 12:32, Dikshita Agarwal wrote:
> Host firmware interface (HFI) is well defined set of interfaces
> for communication between host driver and firmware.
> The command and responses are exchanged in form of packets.
> One or multiple packets are grouped under packet header.
> Each packet has packet type which describes the specific HFI
> and payload which holds the corresponding value for that HFI.
>
> Sys_init is the first packets sent to firmware, which initializes
> the firmware. Sys_image_version packet is to get the firmware
> version string.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> struct iris_core {
> @@ -65,6 +70,11 @@ struct iris_core {
> struct mem_desc sfr;
> struct mutex lock; /* lock for core structure */
> unsigned int use_tz;
> + u8 *packet;
> + u32 packet_size;
I'm not sure it's necessary to always keep a reference to the last
packet in the core struct, especially since it needs to be allocated
first anyway
> + u32 sys_init_id;
This looks like a hyper-defensive measure against some firmware
overtaking attacks.. Or a way to spot random/unwanted resets of
the firmware core..
Is it actually necessary, or does this just serve as a debug
feature?
> + u32 header_id;
Similar to above..
> + u32 packet_id;
And here.
I performed some quick CTRL-F-agge around the series and this is
never reset.. Can the firmware cope with this? What if I watch a
veeeery long youtube video that ends up creating more than
(1<<32)-1 HFI packets while playing?
> +
> +enum hfi_packet_host_flags {
> + HFI_HOST_FLAGS_NONE = 0x00000000,
> + HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001,
> + HFI_HOST_FLAGS_RESPONSE_REQUIRED = 0x00000002,
> + HFI_HOST_FLAGS_NON_DISCARDABLE = 0x00000004,
> + HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008,
BIT(n)?
> +};
> +
> +enum hfi_packet_firmware_flags {
> + HFI_FW_FLAGS_NONE = 0x00000000,
> + HFI_FW_FLAGS_SUCCESS = 0x00000001,
> + HFI_FW_FLAGS_INFORMATION = 0x00000002,
> + HFI_FW_FLAGS_SESSION_ERROR = 0x00000004,
> + HFI_FW_FLAGS_SYSTEM_ERROR = 0x00000008,
BIT(n)?
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (10 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 16:19 ` Krzysztof Kozlowski
2023-12-18 22:00 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance Dikshita Agarwal
` (28 subsequent siblings)
40 siblings, 2 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Registers are defined differently for different VPUs.
Define ops for VPU specific handling to accommodate
different VPUs. Implement boot sequence of firmware and interrupt
programming.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +-
.../media/platform/qcom/vcodec/iris/iris_core.h | 3 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 7 +
.../media/platform/qcom/vcodec/iris/iris_probe.c | 7 +
.../media/platform/qcom/vcodec/iris/vpu_common.c | 71 +++++++++
.../media/platform/qcom/vcodec/iris/vpu_common.h | 32 ++++
.../media/platform/qcom/vcodec/iris/vpu_iris3.c | 166 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 ++
8 files changed, 302 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index a2d5d74..90241b5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -7,6 +7,8 @@ iris-objs += iris_probe.o \
iris_helpers.o \
iris_hfi.o \
iris_hfi_packet.o \
- resources.o
+ resources.o \
+ vpu_common.o \
+ vpu_iris3.o
obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index de0cfef..64678fd 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -12,6 +12,7 @@
#include "../hfi_queue.h"
#include "iris_state.h"
#include "resources.h"
+#include "vpu_common.h"
/**
* struct iris_core - holds core parameters valid for all instances
@@ -44,6 +45,7 @@
* @sys_init_id: id of sys init packet
* @header_id: id of packet header
* @packet_id: id of packet
+ * @vpu_ops: a pointer to vpu ops
*/
struct iris_core {
@@ -75,6 +77,7 @@ struct iris_core {
u32 sys_init_id;
u32 header_id;
u32 packet_id;
+ const struct vpu_ops *vpu_ops;
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index fe16448..7b3cbbc 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -8,6 +8,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
+#include "vpu_common.h"
static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
{
@@ -33,6 +34,8 @@ static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
packet_size = header->size;
if (!write_queue(q_info, pkt, packet_size, &rx_req)) {
+ call_vpu_op(core, raise_interrupt, core);
+ } else {
dev_err(core->dev, "queue full\n");
return -ENODATA;
}
@@ -108,6 +111,10 @@ int iris_hfi_core_init(struct iris_core *core)
if (ret)
goto error;
+ ret = call_vpu_op(core, boot_firmware, core);
+ if (ret)
+ goto error;
+
ret = sys_init(core);
if (ret)
goto error;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 570c64e..773481f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -92,6 +92,13 @@ static int iris_probe(struct platform_device *pdev)
if (core->irq < 0)
return core->irq;
+ ret = init_vpu(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init vpu failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = init_resources(core);
if (ret) {
dev_err_probe(core->dev, ret,
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
new file mode 100644
index 0000000..3282510
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "vpu_iris3.h"
+#include "iris_core.h"
+#include "iris_helpers.h"
+#include "vpu_common.h"
+
+int write_register(struct iris_core *core, u32 reg, u32 value)
+{
+ void __iomem *base_addr;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ base_addr = core->reg_base;
+ base_addr += reg;
+ writel_relaxed(value, base_addr);
+
+ /* Make sure value is written into the register */
+ wmb();
+
+ return ret;
+}
+
+int read_register(struct iris_core *core, u32 reg, u32 *value)
+{
+ void __iomem *base_addr;
+
+ base_addr = core->reg_base;
+
+ *value = readl_relaxed(base_addr + reg);
+
+ /* Make sure value is read correctly from the register */
+ rmb();
+
+ return 0;
+}
+
+static const struct compat_handle compat_handle[] = {
+ {
+ .compat = "qcom,sm8550-iris",
+ .init = init_iris3,
+ },
+};
+
+int init_vpu(struct iris_core *core)
+{
+ struct device *dev = NULL;
+ int i, ret = 0;
+
+ dev = core->dev;
+
+ for (i = 0; i < ARRAY_SIZE(compat_handle); i++) {
+ if (of_device_is_compatible(dev->of_node, compat_handle[i].compat)) {
+ ret = compat_handle[i].init(core);
+ if (ret)
+ return ret;
+ break;
+ }
+ }
+
+ if (i == ARRAY_SIZE(compat_handle))
+ return -EINVAL;
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
new file mode 100644
index 0000000..790496a
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _VPU_COMMON_H_
+#define _VPU_COMMON_H_
+
+#include <linux/types.h>
+
+struct iris_core;
+
+#define call_vpu_op(d, op, ...) \
+ (((d) && (d)->vpu_ops && (d)->vpu_ops->op) ? \
+ ((d)->vpu_ops->op(__VA_ARGS__)) : 0)
+
+struct compat_handle {
+ const char *compat;
+ int (*init)(struct iris_core *core);
+};
+
+struct vpu_ops {
+ int (*boot_firmware)(struct iris_core *core);
+ int (*raise_interrupt)(struct iris_core *core);
+};
+
+int init_vpu(struct iris_core *core);
+
+int write_register(struct iris_core *core, u32 reg, u32 value);
+int read_register(struct iris_core *core, u32 reg, u32 *value);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
new file mode 100644
index 0000000..95bf223
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
@@ -0,0 +1,166 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/delay.h>
+
+#include "vpu_iris3.h"
+
+#define VIDEO_ARCH_LX 1
+
+#define CPU_BASE_OFFS_IRIS3 0x000A0000
+
+#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
+#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
+
+#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
+#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
+
+/* HFI_CTRL_INIT */
+#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48)
+
+/* HFI_CTRL_STATUS */
+#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C)
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000
+
+#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148)
+
+#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168)
+
+/* UC_REGION_ADDR */
+#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64)
+
+/* UC_REGION_ADDR */
+#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68)
+
+/* HFI_QTBL_INFO */
+#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50)
+
+/* HFI_QTBL_ADDR */
+#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54)
+
+/* SFR_ADDR */
+#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C)
+
+#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3
+#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3
+
+#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3
+#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3
+
+#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3
+
+#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3
+
+#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe
+#define CTRL_ERROR_STATUS__M_IRIS3 \
+ CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3
+
+#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150)
+#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0
+
+static int setup_ucregion_memory_map_iris3(struct iris_core *core)
+{
+ int ret;
+ u32 value;
+
+ value = (u32)core->iface_q_table.device_addr;
+ ret = write_register(core, UC_REGION_ADDR_IRIS3, value);
+ if (ret)
+ return ret;
+
+ value = SHARED_QSIZE;
+ ret = write_register(core, UC_REGION_SIZE_IRIS3, value);
+ if (ret)
+ return ret;
+
+ value = (u32)core->iface_q_table.device_addr;
+ ret = write_register(core, QTBL_ADDR_IRIS3, value);
+ if (ret)
+ return ret;
+
+ ret = write_register(core, QTBL_INFO_IRIS3, 0x01);
+ if (ret)
+ return ret;
+
+ value = (u32)((u64)core->iface_q_table.kernel_vaddr);
+ ret = write_register(core, CPU_CS_VCICMDARG0_IRIS3, value);
+ if (ret)
+ return ret;
+
+ value = (u32)((u64)core->iface_q_table.kernel_vaddr >> 32);
+ ret = write_register(core, CPU_CS_VCICMDARG1_IRIS3, value);
+ if (ret)
+ return ret;
+
+ if (core->sfr.device_addr) {
+ value = (u32)core->sfr.device_addr + VIDEO_ARCH_LX;
+ ret = write_register(core, SFR_ADDR_IRIS3, value);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+static int boot_firmware_iris3(struct iris_core *core)
+{
+ u32 ctrl_init = 0, ctrl_status = 0, count = 0, max_tries = 1000;
+ int ret;
+
+ ret = setup_ucregion_memory_map_iris3(core);
+ if (ret)
+ return ret;
+
+ ctrl_init = BIT(0);
+
+ ret = write_register(core, CTRL_INIT_IRIS3, ctrl_init);
+ if (ret)
+ return ret;
+
+ while (!ctrl_status && count < max_tries) {
+ ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status);
+ if (ret)
+ return ret;
+
+ if ((ctrl_status & CTRL_ERROR_STATUS__M_IRIS3) == 0x4) {
+ dev_err(core->dev, "invalid setting for UC_REGION\n");
+ break;
+ }
+
+ usleep_range(50, 100);
+ count++;
+ }
+
+ if (count >= max_tries) {
+ dev_err(core->dev, "Error booting up vidc firmware\n");
+ return -ETIME;
+ }
+
+ ret = write_register(core, CPU_CS_H2XSOFTINTEN_IRIS3, 0x1);
+ if (ret)
+ return ret;
+
+ ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x0);
+
+ return ret;
+}
+
+static int raise_interrupt_iris3(struct iris_core *core)
+{
+ return write_register(core, CPU_IC_SOFTINT_IRIS3, 1 << CPU_IC_SOFTINT_H2A_SHFT_IRIS3);
+}
+
+static const struct vpu_ops iris3_ops = {
+ .boot_firmware = boot_firmware_iris3,
+ .raise_interrupt = raise_interrupt_iris3,
+};
+
+int init_iris3(struct iris_core *core)
+{
+ core->vpu_ops = &iris3_ops;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
new file mode 100644
index 0000000..1424a5f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _VPU_IRIS3_H_
+#define _VPU_IRIS3_H_
+
+#include "iris_core.h"
+
+int init_iris3(struct iris_core *core);
+
+#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling
2023-12-18 11:32 ` [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling Dikshita Agarwal
@ 2023-12-18 16:19 ` Krzysztof Kozlowski
2023-12-18 22:00 ` Konrad Dybcio
1 sibling, 0 replies; 102+ messages in thread
From: Krzysztof Kozlowski @ 2023-12-18 16:19 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 12:32, Dikshita Agarwal wrote:
> Registers are defined differently for different VPUs.
> Define ops for VPU specific handling to accommodate
> different VPUs. Implement boot sequence of firmware and interrupt
> programming.
>
...
> +
> +int write_register(struct iris_core *core, u32 reg, u32 value)
> +{
> + void __iomem *base_addr;
> + int ret;
> +
> + ret = check_core_lock(core);
> + if (ret)
> + return ret;
> +
> + base_addr = core->reg_base;
> + base_addr += reg;
> + writel_relaxed(value, base_addr);
> +
> + /* Make sure value is written into the register */
> + wmb();
Just don't use relaxed method. The same applies to other places like that.
> +
> + return ret;
> +}
> +
> +int read_register(struct iris_core *core, u32 reg, u32 *value)
> +{
> + void __iomem *base_addr;
> +
> + base_addr = core->reg_base;
> +
> + *value = readl_relaxed(base_addr + reg);
> +
> + /* Make sure value is read correctly from the register */
> + rmb();
> +
> + return 0;
> +}
> +
> +static const struct compat_handle compat_handle[] = {
> + {
> + .compat = "qcom,sm8550-iris",
> + .init = init_iris3,
Uh...
> + },
> +};
> +
> +int init_vpu(struct iris_core *core)
> +{
> + struct device *dev = NULL;
> + int i, ret = 0;
> +
> + dev = core->dev;
> +
> + for (i = 0; i < ARRAY_SIZE(compat_handle); i++) {
> + if (of_device_is_compatible(dev->of_node, compat_handle[i].compat)) {
> + ret = compat_handle[i].init(core);
This does not look good. Use flags, quirks, type, pointer ops in
structures. Just look at existing code in Linux kernel. Do not
reimplement driver match data.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling
2023-12-18 11:32 ` [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling Dikshita Agarwal
2023-12-18 16:19 ` Krzysztof Kozlowski
@ 2023-12-18 22:00 ` Konrad Dybcio
1 sibling, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-18 22:00 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/23 12:32, Dikshita Agarwal wrote:
> Registers are defined differently for different VPUs.
> Define ops for VPU specific handling to accommodate
> different VPUs. Implement boot sequence of firmware and interrupt
> programming.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> +int write_register(struct iris_core *core, u32 reg, u32 value)
> +{
> + void __iomem *base_addr;
> + int ret;
> +
> + ret = check_core_lock(core);
> + if (ret)
> + return ret;
> +
> + base_addr = core->reg_base;
> + base_addr += reg;
> + writel_relaxed(value, base_addr);> +
> + /* Make sure value is written into the register */
IIUC barriers only ensure the prior writes need to be submitted
before the next ones, they don't actually guarantee the value
arrives at the destination. You would probably want to read the
register back here to guarantee that.
> + wmb();
> +
> + return ret;
> +}
> +
> +int read_register(struct iris_core *core, u32 reg, u32 *value)
> +{
> + void __iomem *base_addr;
> +
> + base_addr = core->reg_base;
> +
> + *value = readl_relaxed(base_addr + reg);
> +
> + /* Make sure value is read correctly from the register */
> + rmb();
You can drop _relaxed for that and simply use readl() instead of
this entire wrapper..
> +
> + return 0;
> +}
> +
> +static const struct compat_handle compat_handle[] = {
> + {
> + .compat = "qcom,sm8550-iris",
> + .init = init_iris3,
> + },
> +};
> +
> +int init_vpu(struct iris_core *core)
> +{
> + struct device *dev = NULL;
> + int i, ret = 0;
> +
> + dev = core->dev;
> +
> + for (i = 0; i < ARRAY_SIZE(compat_handle); i++) {
> + if (of_device_is_compatible(dev->of_node, compat_handle[i].compat)) {
> + ret = compat_handle[i].init(core);
> + if (ret)
> + return ret;
> + break;
> + }
> + }
> +
> + if (i == ARRAY_SIZE(compat_handle))
> + return -EINVAL;
> +
> + return ret;
> +}
what's wrong with of_match_data?
[...]
> +
> +#define call_vpu_op(d, op, ...) \
> + (((d) && (d)->vpu_ops && (d)->vpu_ops->op) ? \
> + ((d)->vpu_ops->op(__VA_ARGS__)) : 0)
> +
> +struct compat_handle {
> + const char *compat;
> + int (*init)(struct iris_core *core);
> +};
> +
> +struct vpu_ops {
> + int (*boot_firmware)(struct iris_core *core);
> + int (*raise_interrupt)(struct iris_core *core);
> +};
or you can simply create functions like
boot_firmware(...)
raise_interrupt(...)
that call other functions as needed (unless there's no need if e.g.
the cores are so similar)
and drop this sugar (well, bitter sugar at least to my taste) syntax
[...]
> + int ret;
> + u32 value;
reverse-Christmas-tree, please
(Christmas is in a week, get festive! :D)
> +
> + value = (u32)core->iface_q_table.device_addr;
> + ret = write_register(core, UC_REGION_ADDR_IRIS3, value);
> + if (ret)
> + return ret;
> +
> + value = SHARED_QSIZE;
> + ret = write_register(core, UC_REGION_SIZE_IRIS3, value);
> + if (ret)
> + return ret;
> +
> + value = (u32)core->iface_q_table.device_addr;
> + ret = write_register(core, QTBL_ADDR_IRIS3, value);
> + if (ret)
> + return ret;
> +
> + ret = write_register(core, QTBL_INFO_IRIS3, 0x01);
> + if (ret)
> + return ret;
> +
> + value = (u32)((u64)core->iface_q_table.kernel_vaddr);
lower_32_bits()
> + ret = write_register(core, CPU_CS_VCICMDARG0_IRIS3, value);
> + if (ret)
> + return ret;
> +
> + value = (u32)((u64)core->iface_q_table.kernel_vaddr >> 32);
upper_32_bits()
> + ret = write_register(core, CPU_CS_VCICMDARG1_IRIS3, value);
> + if (ret)
> + return ret;
> +
> + if (core->sfr.device_addr) {
> + value = (u32)core->sfr.device_addr + VIDEO_ARCH_LX;
> + ret = write_register(core, SFR_ADDR_IRIS3, value);
> + if (ret)
> + return ret;
you're returning ret 3 lines below anyway
> + }
> +
> + return ret;
> +}
> +
> +static int boot_firmware_iris3(struct iris_core *core)
> +{
> + u32 ctrl_init = 0, ctrl_status = 0, count = 0, max_tries = 1000;
> + int ret;
> +
> + ret = setup_ucregion_memory_map_iris3(core);
> + if (ret)
> + return ret;
> +
> + ctrl_init = BIT(0);
this should be a named #define used inline
> +
> + ret = write_register(core, CTRL_INIT_IRIS3, ctrl_init);
> + if (ret)
> + return ret;
> +
> + while (!ctrl_status && count < max_tries) {
if you take the previous feedback into account, this can become
readl_poll_timeout()
> + ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status);
> + if (ret)
> + return ret;
> +
> + if ((ctrl_status & CTRL_ERROR_STATUS__M_IRIS3) == 0x4) {
> + dev_err(core->dev, "invalid setting for UC_REGION\n");
> + break;
> + }
> +
> + usleep_range(50, 100);
> + count++;
> + }
> +
> + if (count >= max_tries) {
> + dev_err(core->dev, "Error booting up vidc firmware\n");
> + return -ETIME;
> + }
> +
> + ret = write_register(core, CPU_CS_H2XSOFTINTEN_IRIS3, 0x1);
0x1? BIT(0)? probably a named BIT(0) that deserves its own #define?
> + if (ret)
> + return ret;
> +
> + ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x0);
similarly here
btw you can just return this
> +
> + return ret;
> +}
> +
> +static int raise_interrupt_iris3(struct iris_core *core)
> +{
> + return write_register(core, CPU_IC_SOFTINT_IRIS3, 1 << CPU_IC_SOFTINT_H2A_SHFT_IRIS3);
FIELD_PREP/GET, please
> +}
> +
> +static const struct vpu_ops iris3_ops = {
> + .boot_firmware = boot_firmware_iris3,
> + .raise_interrupt = raise_interrupt_iris3,
> +};
> +
> +int init_iris3(struct iris_core *core)
> +{
> + core->vpu_ops = &iris3_ops;
> +
> + return 0;
> +}
what is this dead function for?
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (11 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 22:08 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 14/34] media: iris: implement iris v4l2 file ops Dikshita Agarwal
` (27 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Capabilities are set of video specifications and features supported
by a specific platform(SOC). Each capability is defined with
min, max, range, default value and corresponding HFI.
Also, platform data defines different resource details for
a specific platform(SOC). This change defines resource tables
for sm8550 platform data and use for initializing these resources.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +-
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 42 ++-
.../media/platform/qcom/vcodec/iris/iris_core.h | 5 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 19 ++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 20 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 84 ++++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 27 +-
.../platform/qcom/vcodec/iris/platform_common.c | 29 ++
.../platform/qcom/vcodec/iris/platform_common.h | 194 +++++++++++++
.../platform/qcom/vcodec/iris/platform_sm8550.c | 316 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/resources.c | 74 +++--
12 files changed, 759 insertions(+), 56 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 90241b5..c50e3241 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -9,6 +9,8 @@ iris-objs += iris_probe.o \
iris_hfi_packet.o \
resources.o \
vpu_common.o \
- vpu_iris3.o
+ vpu_iris3.o \
+ platform_common.o \
+ platform_sm8550.o
obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 4c48c90..fb383b2 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -6,10 +6,46 @@
#ifndef _HFI_DEFINES_H_
#define _HFI_DEFINES_H_
-#define HFI_VIDEO_ARCH_LX 0x1
+#define HFI_VIDEO_ARCH_LX 0x1
-#define HFI_CMD_INIT 0x01000001
+#define HFI_CMD_INIT 0x01000001
-#define HFI_PROP_IMAGE_VERSION 0x03000001
+#define HFI_PROP_IMAGE_VERSION 0x03000001
+
+#define HFI_PROP_UBWC_MAX_CHANNELS 0x03000003
+#define HFI_PROP_UBWC_MAL_LENGTH 0x03000004
+#define HFI_PROP_UBWC_HBB 0x03000005
+#define HFI_PROP_UBWC_BANK_SWZL_LEVEL1 0x03000006
+#define HFI_PROP_UBWC_BANK_SWZL_LEVEL2 0x03000007
+#define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008
+#define HFI_PROP_UBWC_BANK_SPREADING 0x03000009
+
+#define HFI_PROP_PROFILE 0x03000107
+
+#define HFI_PROP_LEVEL 0x03000108
+
+#define HFI_PROP_TIER 0x03000109
+
+#define HFI_PROP_STAGE 0x0300010a
+
+#define HFI_PROP_PIPE 0x0300010b
+
+#define HFI_PROP_FRAME_RATE 0x0300010c
+
+#define HFI_PROP_LUMA_CHROMA_BIT_DEPTH 0x0300010f
+
+#define HFI_PROP_CODED_FRAMES 0x03000120
+
+#define HFI_PROP_CABAC_SESSION 0x03000121
+
+#define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123
+
+#define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128
+
+#define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b
+
+#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
+
+#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 64678fd..3c8497a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -11,6 +11,7 @@
#include "../hfi_queue.h"
#include "iris_state.h"
+#include "platform_common.h"
#include "resources.h"
#include "vpu_common.h"
@@ -46,6 +47,8 @@
* @header_id: id of packet header
* @packet_id: id of packet
* @vpu_ops: a pointer to vpu ops
+ * @platform_data: a structure for platform data
+ * @cap: an array for supported core capabilities
*/
struct iris_core {
@@ -78,6 +81,8 @@ struct iris_core {
u32 header_id;
u32 packet_id;
const struct vpu_ops *vpu_ops;
+ struct platform_data *platform_data;
+ struct plat_core_cap cap[CORE_CAP_MAX + 1];
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 22c706a..c31dfd5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -13,3 +13,22 @@ int check_core_lock(struct iris_core *core)
return fatal ? -EINVAL : 0;
}
+
+int iris_init_core_caps(struct iris_core *core)
+{
+ struct plat_core_cap *core_platform_data;
+ int i, num_core_caps;
+
+ core_platform_data = core->platform_data->core_data;
+ if (!core_platform_data)
+ return -EINVAL;
+
+ num_core_caps = core->platform_data->core_data_size;
+
+ for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) {
+ core->cap[core_platform_data[i].type].type = core_platform_data[i].type;
+ core->cap[core_platform_data[i].type].value = core_platform_data[i].value;
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 314a8d75..4c7ddbf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -9,5 +9,6 @@
#include "iris_core.h"
int check_core_lock(struct iris_core *core);
+int iris_init_core_caps(struct iris_core *core);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 7b3cbbc..b0390b542 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -79,8 +79,12 @@ static int sys_image_version(struct iris_core *core)
int iris_hfi_core_init(struct iris_core *core)
{
+ u32 cp_nonpixel_start, cp_nonpixel_size;
phys_addr_t mem_phys = 0;
+ u32 cp_start, cp_size;
+ const char *fw_name;
size_t mem_size = 0;
+ u32 pas_id;
int ret;
ret = check_core_lock(core);
@@ -97,17 +101,23 @@ int iris_hfi_core_init(struct iris_core *core)
if (!core->use_tz)
goto error;
- ret = load_fw(core->dev, FW_NAME, &mem_phys, &mem_size,
- IRIS_PAS_ID, core->use_tz);
+ pas_id = core->platform_data->pas_id;
+ fw_name = core->platform_data->fwname;
+ ret = load_fw(core->dev, fw_name, &mem_phys, &mem_size,
+ pas_id, core->use_tz);
if (ret)
goto error;
- ret = auth_reset_fw(IRIS_PAS_ID);
+ ret = auth_reset_fw(pas_id);
if (ret)
goto error;
- ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START,
- CP_NONPIXEL_SIZE, IRIS_PAS_ID);
+ cp_start = core->cap[CP_START].value;
+ cp_size = core->cap[CP_SIZE].value;
+ cp_nonpixel_start = core->cap[CP_NONPIXEL_START].value;
+ cp_nonpixel_size = core->cap[CP_NONPIXEL_SIZE].value;
+ ret = protect_secure_region(cp_start, cp_size, cp_nonpixel_start,
+ cp_nonpixel_size, pas_id);
if (ret)
goto error;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index 73bba07..f31dd84 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -89,6 +89,90 @@ int hfi_packet_sys_init(struct iris_core *core,
if (ret)
goto error;
+ payload = core->platform_data->ubwc_config->max_channels;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_UBWC_MAX_CHANNELS,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ payload = core->platform_data->ubwc_config->mal_length;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_UBWC_MAL_LENGTH,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ payload = core->platform_data->ubwc_config->highest_bank_bit;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_UBWC_HBB,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ payload = core->platform_data->ubwc_config->bank_swzl_level;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_UBWC_BANK_SWZL_LEVEL1,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ payload = core->platform_data->ubwc_config->bank_swz2_level;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_UBWC_BANK_SWZL_LEVEL2,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ payload = core->platform_data->ubwc_config->bank_swz3_level;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_UBWC_BANK_SWZL_LEVEL3,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ payload = core->platform_data->ubwc_config->bank_spreading;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_UBWC_BANK_SPREADING,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
return ret;
error:
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 773481f..9d6a6c5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -9,6 +9,7 @@
#include "../hfi_queue.h"
#include "iris_core.h"
+#include "iris_helpers.h"
#include "resources.h"
static int iris_register_video_device(struct iris_core *core)
@@ -92,6 +93,13 @@ static int iris_probe(struct platform_device *pdev)
if (core->irq < 0)
return core->irq;
+ ret = init_platform(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init platform failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = init_vpu(core);
if (ret) {
dev_err_probe(core->dev, ret,
@@ -106,6 +114,13 @@ static int iris_probe(struct platform_device *pdev)
return ret;
}
+ ret = iris_init_core_caps(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init core caps failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = v4l2_device_register(dev, &core->v4l2_dev);
if (ret)
return ret;
@@ -116,12 +131,7 @@ static int iris_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, core);
- /*
- * Specify the max value of address space, which can be used
- * for buffer transactions.
- */
- dma_mask = DMA_BIT_MASK(32);
- dma_mask &= ~BIT(29);
+ dma_mask = core->cap[DMA_MASK].value;
ret = dma_set_mask_and_coherent(dev, dma_mask);
if (ret)
@@ -160,7 +170,10 @@ static int iris_probe(struct platform_device *pdev)
}
static const struct of_device_id iris_dt_match[] = {
- { .compatible = "qcom,sm8550-iris", },
+ {
+ .compatible = "qcom,sm8550-iris",
+ .data = &sm8550_data,
+ },
{ },
};
MODULE_DEVICE_TABLE(of, iris_dt_match);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.c b/drivers/media/platform/qcom/vcodec/iris/platform_common.c
new file mode 100644
index 0000000..043adc8
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.c
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+
+#include "iris_core.h"
+#include "platform_common.h"
+
+int init_platform(struct iris_core *core)
+{
+ struct platform_data *platform = NULL;
+
+ platform = devm_kzalloc(core->dev, sizeof(*platform),
+ GFP_KERNEL);
+ if (!platform)
+ return -ENOMEM;
+
+ core->platform_data = platform;
+
+ core->platform_data = (struct platform_data *)of_device_get_match_data(core->dev);
+ if (!core->platform_data)
+ return -ENODEV;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
new file mode 100644
index 0000000..e478b02
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -0,0 +1,194 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _PLATFORM_COMMON_H_
+#define _PLATFORM_COMMON_H_
+
+#include <linux/bits.h>
+
+struct iris_core;
+
+#define HW_RESPONSE_TIMEOUT_VALUE (1000)
+
+#define BIT_DEPTH_8 (8 << 16 | 8)
+#define BIT_DEPTH_10 (10 << 16 | 10)
+
+#define CODED_FRAMES_PROGRESSIVE 0x0
+#define CODED_FRAMES_INTERLACE 0x1
+
+#define DEFAULT_MAX_HOST_BUF_COUNT 64
+#define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256
+
+#define UBWC_CONFIG(mc, ml, hbb, bs1, bs2, bs3, bsp) \
+{ \
+ .max_channels = mc, \
+ .mal_length = ml, \
+ .highest_bank_bit = hbb, \
+ .bank_swzl_level = bs1, \
+ .bank_swz2_level = bs2, \
+ .bank_swz3_level = bs3, \
+ .bank_spreading = bsp, \
+}
+
+enum codec_type {
+ H264 = BIT(0),
+ HEVC = BIT(1),
+ VP9 = BIT(2),
+};
+
+enum colorformat_type {
+ FMT_NONE = 0,
+ FMT_NV12C = BIT(0),
+ FMT_NV12 = BIT(1),
+ FMT_NV21 = BIT(2),
+ FMT_TP10C = BIT(3),
+};
+
+enum stage_type {
+ STAGE_NONE = 0,
+ STAGE_1 = 1,
+ STAGE_2 = 2,
+};
+
+enum pipe_type {
+ PIPE_NONE = 0,
+ PIPE_1 = 1,
+ PIPE_2 = 2,
+ PIPE_4 = 4,
+};
+
+extern struct platform_data sm8550_data;
+
+struct bw_info {
+ u32 mbs_per_sec;
+ u32 bw_ddr;
+ u32 bw_ddr_10bit;
+};
+
+struct reg_preset_info {
+ u32 reg;
+ u32 value;
+ u32 mask;
+};
+
+struct ubwc_config_data {
+ u32 max_channels;
+ u32 mal_length;
+ u32 highest_bank_bit;
+ u32 bank_swzl_level;
+ u32 bank_swz2_level;
+ u32 bank_swz3_level;
+ u32 bank_spreading;
+};
+
+enum plat_core_cap_type {
+ CORE_CAP_NONE = 0,
+ DEC_CODECS,
+ MAX_SESSION_COUNT,
+ MAX_MBPF,
+ MAX_MBPS,
+ MAX_MBPF_HQ,
+ MAX_MBPS_HQ,
+ MAX_MBPF_B_FRAME,
+ MAX_MBPS_B_FRAME,
+ MAX_ENH_LAYER_COUNT,
+ NUM_VPP_PIPE,
+ FW_UNLOAD,
+ FW_UNLOAD_DELAY,
+ HW_RESPONSE_TIMEOUT,
+ NON_FATAL_FAULTS,
+ DMA_MASK,
+ CP_START,
+ CP_SIZE,
+ CP_NONPIXEL_START,
+ CP_NONPIXEL_SIZE,
+ CORE_CAP_MAX,
+};
+
+struct plat_core_cap {
+ enum plat_core_cap_type type;
+ u32 value;
+};
+
+enum plat_inst_cap_type {
+ INST_CAP_NONE = 0,
+ FRAME_WIDTH,
+ FRAME_HEIGHT,
+ PIX_FMTS,
+ MBPF,
+ QUEUED_RATE,
+ MB_CYCLES_VSP,
+ MB_CYCLES_VPP,
+ MB_CYCLES_LP,
+ MB_CYCLES_FW,
+ MB_CYCLES_FW_VPP,
+ NUM_COMV,
+ PROFILE,
+ LEVEL,
+ HEVC_TIER,
+ DISPLAY_DELAY_ENABLE,
+ DISPLAY_DELAY,
+ OUTPUT_ORDER,
+ INPUT_BUF_HOST_MAX_COUNT,
+ STAGE,
+ PIPE,
+ POC,
+ CODED_FRAMES,
+ BIT_DEPTH,
+ DEFAULT_HEADER,
+ RAP_FRAME,
+ INST_CAP_MAX,
+};
+
+enum plat_inst_cap_flags {
+ CAP_FLAG_NONE = 0,
+ CAP_FLAG_DYNAMIC_ALLOWED = BIT(0),
+ CAP_FLAG_MENU = BIT(1),
+ CAP_FLAG_INPUT_PORT = BIT(2),
+ CAP_FLAG_OUTPUT_PORT = BIT(3),
+ CAP_FLAG_CLIENT_SET = BIT(4),
+ CAP_FLAG_BITMASK = BIT(5),
+ CAP_FLAG_VOLATILE = BIT(6),
+};
+
+struct plat_inst_cap {
+ enum plat_inst_cap_type cap_id;
+ enum codec_type codec;
+ s32 min;
+ s32 max;
+ u32 step_or_mask;
+ s32 value;
+ u32 v4l2_id;
+ u32 hfi_id;
+ enum plat_inst_cap_flags flags;
+};
+
+struct platform_data {
+ const struct bus_info *bus_tbl;
+ unsigned int bus_tbl_size;
+ const struct bw_info *bw_tbl_dec;
+ unsigned int bw_tbl_dec_size;
+ const char * const *pd_tbl;
+ unsigned int pd_tbl_size;
+ const char * const *opp_pd_tbl;
+ unsigned int opp_pd_tbl_size;
+ const struct clock_info *clk_tbl;
+ unsigned int clk_tbl_size;
+ const char * const *clk_rst_tbl;
+ unsigned int clk_rst_tbl_size;
+ const struct reg_preset_info *reg_prst_tbl;
+ unsigned int reg_prst_tbl_size;
+ struct ubwc_config_data *ubwc_config;
+ const char *fwname;
+ u32 pas_id;
+ struct plat_core_cap *core_data;
+ u32 core_data_size;
+ struct plat_inst_cap *inst_cap_data;
+ u32 inst_cap_data_size;
+};
+
+int init_platform(struct iris_core *core);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
new file mode 100644
index 0000000..c75017e
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -0,0 +1,316 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <dt-bindings/clock/qcom,sm8550-gcc.h>
+#include <dt-bindings/clock/qcom,sm8450-videocc.h>
+
+#include <media/v4l2-ctrls.h>
+
+#include "hfi_defines.h"
+#include "platform_common.h"
+#include "resources.h"
+
+#define CODECS_ALL (H264 | HEVC | VP9)
+
+#define DEFAULT_FPS 30
+#define MINIMUM_FPS 1
+#define MAXIMUM_FPS 480
+
+static struct plat_core_cap core_data_sm8550[] = {
+ {DEC_CODECS, H264 | HEVC | VP9},
+ {MAX_SESSION_COUNT, 16},
+ {MAX_MBPF, 278528}, /* ((8192x4352)/256) * 2 */
+ {MAX_MBPS, 7833600}, /* max_load 7680x4320@60fps */
+ {NUM_VPP_PIPE, 4},
+ {HW_RESPONSE_TIMEOUT, HW_RESPONSE_TIMEOUT_VALUE},
+ {DMA_MASK, GENMASK(31, 29) - 1},
+ {CP_START, 0},
+ {CP_SIZE, 0x25800000},
+ {CP_NONPIXEL_START, 0x01000000},
+ {CP_NONPIXEL_SIZE, 0x24800000},
+};
+
+static struct plat_inst_cap instance_cap_data_sm8550[] = {
+ {FRAME_WIDTH, CODECS_ALL, 96, 8192, 1, 1920},
+
+ {FRAME_WIDTH, VP9, 96, 4096, 1, 1920},
+
+ {FRAME_HEIGHT, CODECS_ALL, 96, 8192, 1, 1080},
+
+ {FRAME_HEIGHT, VP9, 96, 4096, 1, 1080},
+
+ {PIX_FMTS, H264,
+ FMT_NV12,
+ FMT_NV12C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C,
+ FMT_NV12C},
+
+ {PIX_FMTS, HEVC | VP9,
+ FMT_NV12,
+ FMT_TP10C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
+ FMT_NV12C},
+
+ {MBPF, CODECS_ALL, 36, 138240, 1, 138240},
+
+ /* (4096 * 2304) / 256 */
+ {MBPF, VP9, 36, 36864, 1, 36864},
+
+ {QUEUED_RATE, CODECS_ALL,
+ (MINIMUM_FPS << 16), INT_MAX,
+ 1, (DEFAULT_FPS << 16)},
+
+ {MB_CYCLES_VSP, CODECS_ALL, 25, 25, 1, 25},
+
+ {MB_CYCLES_VSP, VP9, 60, 60, 1, 60},
+
+ {MB_CYCLES_VPP, CODECS_ALL, 200, 200, 1, 200},
+
+ {MB_CYCLES_LP, CODECS_ALL, 200, 200, 1, 200},
+
+ {MB_CYCLES_FW, CODECS_ALL, 489583, 489583, 1, 489583},
+
+ {MB_CYCLES_FW_VPP, CODECS_ALL, 66234, 66234, 1, 66234},
+
+ {NUM_COMV, CODECS_ALL,
+ 0, INT_MAX, 1, 0},
+
+ {PROFILE, H264,
+ V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH,
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH),
+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ HFI_PROP_PROFILE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {PROFILE, HEVC,
+ V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+ V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
+ BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10),
+ V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+ V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+ HFI_PROP_PROFILE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {PROFILE, VP9,
+ V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+ V4L2_MPEG_VIDEO_VP9_PROFILE_2,
+ BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) |
+ BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_2),
+ V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+ V4L2_CID_MPEG_VIDEO_VP9_PROFILE,
+ HFI_PROP_PROFILE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {LEVEL, H264,
+ V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+ V4L2_MPEG_VIDEO_H264_LEVEL_6_2,
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_2),
+ V4L2_MPEG_VIDEO_H264_LEVEL_6_1,
+ V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ HFI_PROP_LEVEL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {LEVEL, HEVC,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2,
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2),
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1,
+ V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+ HFI_PROP_LEVEL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {LEVEL, VP9,
+ V4L2_MPEG_VIDEO_VP9_LEVEL_1_0,
+ V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_1) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_0) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_0) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_0) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_0) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_6_0),
+ V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
+ V4L2_CID_MPEG_VIDEO_VP9_LEVEL,
+ HFI_PROP_LEVEL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {HEVC_TIER, HEVC,
+ V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
+ V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
+ BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_TIER_HIGH),
+ V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
+ V4L2_CID_MPEG_VIDEO_HEVC_TIER,
+ HFI_PROP_TIER,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {DISPLAY_DELAY_ENABLE, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE,
+ HFI_PROP_DECODE_ORDER_OUTPUT,
+ CAP_FLAG_INPUT_PORT},
+
+ {DISPLAY_DELAY, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY,
+ HFI_PROP_DECODE_ORDER_OUTPUT,
+ CAP_FLAG_INPUT_PORT},
+
+ {OUTPUT_ORDER, CODECS_ALL,
+ 0, 1, 1, 0,
+ 0,
+ HFI_PROP_DECODE_ORDER_OUTPUT,
+ CAP_FLAG_INPUT_PORT},
+
+ {INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL,
+ DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT,
+ 1, DEFAULT_MAX_HOST_BUF_COUNT,
+ 0,
+ HFI_PROP_BUFFER_HOST_MAX_COUNT,
+ CAP_FLAG_INPUT_PORT},
+
+ {STAGE, CODECS_ALL,
+ STAGE_1,
+ STAGE_2, 1,
+ STAGE_2,
+ 0,
+ HFI_PROP_STAGE},
+
+ {PIPE, CODECS_ALL,
+ PIPE_1,
+ PIPE_4, 1,
+ PIPE_4,
+ 0,
+ HFI_PROP_PIPE},
+
+ {POC, H264, 0, 2, 1, 1,
+ 0,
+ HFI_PROP_PIC_ORDER_CNT_TYPE},
+
+ {CODED_FRAMES, H264 | HEVC,
+ CODED_FRAMES_PROGRESSIVE, CODED_FRAMES_PROGRESSIVE,
+ 0, CODED_FRAMES_PROGRESSIVE,
+ 0,
+ HFI_PROP_CODED_FRAMES},
+
+ {BIT_DEPTH, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8,
+ 0,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH},
+
+ {DEFAULT_HEADER, CODECS_ALL,
+ 0, 1, 1, 0,
+ 0,
+ HFI_PROP_DEC_DEFAULT_HEADER},
+
+ {RAP_FRAME, CODECS_ALL,
+ 0, 1, 1, 1,
+ 0,
+ HFI_PROP_DEC_START_FROM_RAP_FRAME,
+ CAP_FLAG_INPUT_PORT},
+};
+
+static const struct bus_info sm8550_bus_table[] = {
+ { NULL, "iris-cnoc", 1000, 1000 },
+ { NULL, "iris-ddr", 1000, 15000000 },
+};
+
+static const struct clock_info sm8550_clk_table[] = {
+ { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0 },
+ { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0 },
+ { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1 },
+};
+
+static const char * const sm8550_clk_reset_table[] = { "video_axi_reset", NULL };
+
+static const char * const sm8550_pd_table[] = { "iris-ctl", "vcodec", NULL };
+
+static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx", NULL };
+
+static const struct bw_info sm8550_bw_table_dec[] = {
+ { 2073600, 1608000, 2742000 }, /* 4096x2160@60 */
+ { 1036800, 826000, 1393000 }, /* 4096x2160@30 */
+ { 489600, 567000, 723000 }, /* 1920x1080@60 */
+ { 244800, 294000, 372000 }, /* 1920x1080@30 */
+};
+
+static const struct reg_preset_info sm8550_reg_preset_table[] = {
+ { 0xB0088, 0x0, 0x11 },
+};
+
+static struct ubwc_config_data ubwc_config_sm8550[] = {
+ UBWC_CONFIG(8, 32, 16, 0, 1, 1, 1),
+};
+
+struct platform_data sm8550_data = {
+ .bus_tbl = sm8550_bus_table,
+ .bus_tbl_size = ARRAY_SIZE(sm8550_bus_table),
+ .clk_tbl = sm8550_clk_table,
+ .clk_tbl_size = ARRAY_SIZE(sm8550_clk_table),
+ .clk_rst_tbl = sm8550_clk_reset_table,
+ .clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table),
+
+ .bw_tbl_dec = sm8550_bw_table_dec,
+ .bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec),
+
+ .pd_tbl = sm8550_pd_table,
+ .pd_tbl_size = ARRAY_SIZE(sm8550_pd_table),
+ .opp_pd_tbl = sm8550_opp_pd_table,
+ .opp_pd_tbl_size = ARRAY_SIZE(sm8550_opp_pd_table),
+
+ .reg_prst_tbl = sm8550_reg_preset_table,
+ .reg_prst_tbl_size = ARRAY_SIZE(sm8550_reg_preset_table),
+ .fwname = "vpu30_4v.mbn",
+ .pas_id = 9,
+
+ .core_data = core_data_sm8550,
+ .core_data_size = ARRAY_SIZE(core_data_sm8550),
+ .inst_cap_data = instance_cap_data_sm8550,
+ .inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550),
+ .ubwc_config = ubwc_config_sm8550,
+};
diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c
index 5aebbe4..8cfdcd6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/resources.c
+++ b/drivers/media/platform/qcom/vcodec/iris/resources.c
@@ -3,9 +3,6 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
-#include <dt-bindings/clock/qcom,sm8550-gcc.h>
-#include <dt-bindings/clock/qcom,sm8450-videocc.h>
-
#include <linux/clk.h>
#include <linux/interconnect.h>
#include <linux/pm_domain.h>
@@ -15,28 +12,9 @@
#include <linux/sort.h>
#include "iris_core.h"
+#include "platform_common.h"
#include "resources.h"
-static const struct bus_info plat_bus_table[] = {
- { NULL, "iris-cnoc", 1000, 1000 },
- { NULL, "iris-ddr", 1000, 15000000 },
-};
-
-static const char * const plat_pd_table[] = { "iris-ctl", "vcodec", NULL };
-#define PD_COUNT 2
-
-static const char * const plat_opp_pd_table[] = { "mxc", "mmcx", NULL };
-#define OPP_PD_COUNT 2
-
-static const struct clock_info plat_clk_table[] = {
- { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0, 0 },
- { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0, 0 },
- { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1, 0 },
-};
-
-static const char * const plat_clk_reset_table[] = { "video_axi_reset", NULL };
-#define RESET_COUNT 1
-
static void iris_pd_release(void *res)
{
struct device *pd = (struct device *)res;
@@ -83,10 +61,13 @@ static int iris_opp_dl_get(struct device *dev, struct device *supplier)
static int init_bus(struct iris_core *core)
{
+ const struct bus_info *bus_tbl;
struct bus_info *binfo = NULL;
u32 i = 0;
- core->bus_count = ARRAY_SIZE(plat_bus_table);
+ bus_tbl = core->platform_data->bus_tbl;
+
+ core->bus_count = core->platform_data->bus_tbl_size;
core->bus_tbl = devm_kzalloc(core->dev,
sizeof(struct bus_info) * core->bus_count,
GFP_KERNEL);
@@ -95,9 +76,9 @@ static int init_bus(struct iris_core *core)
for (i = 0; i < core->bus_count; i++) {
binfo = &core->bus_tbl[i];
- binfo->name = plat_bus_table[i].name;
- binfo->bw_min_kbps = plat_bus_table[i].bw_min_kbps;
- binfo->bw_max_kbps = plat_bus_table[i].bw_max_kbps;
+ binfo->name = bus_tbl[i].name;
+ binfo->bw_min_kbps = bus_tbl[i].bw_min_kbps;
+ binfo->bw_max_kbps = bus_tbl[i].bw_max_kbps;
binfo->icc = devm_of_icc_get(core->dev, binfo->name);
if (IS_ERR(binfo->icc)) {
dev_err(core->dev,
@@ -112,20 +93,24 @@ static int init_bus(struct iris_core *core)
static int init_power_domains(struct iris_core *core)
{
struct power_domain_info *pdinfo = NULL;
+ const char * const *opp_pd_tbl;
+ const char * const *pd_tbl;
struct device **opp_vdevs = NULL;
+ u32 opp_pd_cnt, i;
int ret;
- u32 i;
- core->pd_count = PD_COUNT;
+ pd_tbl = core->platform_data->pd_tbl;
+
+ core->pd_count = core->platform_data->pd_tbl_size;
core->power_domain_tbl = devm_kzalloc(core->dev,
sizeof(struct power_domain_info) * core->pd_count,
GFP_KERNEL);
if (!core->power_domain_tbl)
return -ENOMEM;
- for (i = 0; i < core->pd_count; i++) {
+ for (i = 0; i < (core->pd_count - 1); i++) {
pdinfo = &core->power_domain_tbl[i];
- pdinfo->name = plat_pd_table[i];
+ pdinfo->name = pd_tbl[i];
ret = iris_pd_get(core, pdinfo);
if (ret) {
dev_err(core->dev,
@@ -134,11 +119,14 @@ static int init_power_domains(struct iris_core *core)
}
}
- ret = devm_pm_opp_attach_genpd(core->dev, plat_opp_pd_table, &opp_vdevs);
+ opp_pd_tbl = core->platform_data->opp_pd_tbl;
+ opp_pd_cnt = core->platform_data->opp_pd_tbl_size;
+
+ ret = devm_pm_opp_attach_genpd(core->dev, opp_pd_tbl, &opp_vdevs);
if (ret)
return ret;
- for (i = 0; i < OPP_PD_COUNT; i++) {
+ for (i = 0; i < (opp_pd_cnt - 1) ; i++) {
ret = iris_opp_dl_get(core->dev, opp_vdevs[i]);
if (ret) {
dev_err(core->dev, "%s: failed to create dl: %s\n",
@@ -158,10 +146,13 @@ static int init_power_domains(struct iris_core *core)
static int init_clocks(struct iris_core *core)
{
+ const struct clock_info *clk_tbl;
struct clock_info *cinfo = NULL;
u32 i;
- core->clk_count = ARRAY_SIZE(plat_clk_table);
+ clk_tbl = core->platform_data->clk_tbl;
+
+ core->clk_count = core->platform_data->clk_tbl_size;
core->clock_tbl = devm_kzalloc(core->dev,
sizeof(struct clock_info) * core->clk_count,
GFP_KERNEL);
@@ -170,9 +161,9 @@ static int init_clocks(struct iris_core *core)
for (i = 0; i < core->clk_count; i++) {
cinfo = &core->clock_tbl[i];
- cinfo->name = plat_clk_table[i].name;
- cinfo->clk_id = plat_clk_table[i].clk_id;
- cinfo->has_scaling = plat_clk_table[i].has_scaling;
+ cinfo->name = clk_tbl[i].name;
+ cinfo->clk_id = clk_tbl[i].clk_id;
+ cinfo->has_scaling = clk_tbl[i].has_scaling;
cinfo->clk = devm_clk_get(core->dev, cinfo->name);
if (IS_ERR(cinfo->clk)) {
dev_err(core->dev,
@@ -187,18 +178,21 @@ static int init_clocks(struct iris_core *core)
static int init_reset_clocks(struct iris_core *core)
{
struct reset_info *rinfo = NULL;
+ const char * const *rst_tbl;
u32 i = 0;
- core->reset_count = RESET_COUNT;
+ rst_tbl = core->platform_data->clk_rst_tbl;
+
+ core->reset_count = core->platform_data->clk_rst_tbl_size;
core->reset_tbl = devm_kzalloc(core->dev,
sizeof(struct reset_info) * core->reset_count,
GFP_KERNEL);
if (!core->reset_tbl)
return -ENOMEM;
- for (i = 0; i < core->reset_count; i++) {
+ for (i = 0; i < (core->reset_count - 1); i++) {
rinfo = &core->reset_tbl[i];
- rinfo->name = plat_clk_reset_table[i];
+ rinfo->name = rst_tbl[i];
rinfo->rst = devm_reset_control_get(core->dev, rinfo->name);
if (IS_ERR(rinfo->rst)) {
dev_err(core->dev,
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance
2023-12-18 11:32 ` [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance Dikshita Agarwal
@ 2023-12-18 22:08 ` Konrad Dybcio
0 siblings, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-18 22:08 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/23 12:32, Dikshita Agarwal wrote:
> Capabilities are set of video specifications and features supported
> by a specific platform(SOC). Each capability is defined with
> min, max, range, default value and corresponding HFI.
>
> Also, platform data defines different resource details for
> a specific platform(SOC). This change defines resource tables
> for sm8550 platform data and use for initializing these resources.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
[...]
> - ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START,
> - CP_NONPIXEL_SIZE, IRIS_PAS_ID);
> + cp_start = core->cap[CP_START].value;
> + cp_size = core->cap[CP_SIZE].value;
> + cp_nonpixel_start = core->cap[CP_NONPIXEL_START].value;
> + cp_nonpixel_size = core->cap[CP_NONPIXEL_SIZE].value;
but you just hardcoded these a couple patches ago.. are they
variable after all?
[...]
> + {DEC_CODECS, H264 | HEVC | VP9},
> + {MAX_SESSION_COUNT, 16},
> + {MAX_MBPF, 278528}, /* ((8192x4352)/256) * 2 */
> + {MAX_MBPS, 7833600}, /* max_load 7680x4320@60fps */
> + {NUM_VPP_PIPE, 4},
> + {HW_RESPONSE_TIMEOUT, HW_RESPONSE_TIMEOUT_VALUE},
> + {DMA_MASK, GENMASK(31, 29) - 1},
> + {CP_START, 0},
> + {CP_SIZE, 0x25800000},
> + {CP_NONPIXEL_START, 0x01000000},
> + {CP_NONPIXEL_SIZE, 0x24800000},
Why this and not an array of enum-indexed u32s?
> +};
> +
> +static struct plat_inst_cap instance_cap_data_sm8550[] = {
you know all of the possible members here as well, you can just
create a struct of actual configurations instead of turning them
into generic capabilities that you have to parse either way at
some point
[...]
> +
> +static const struct bus_info sm8550_bus_table[] = {
> + { NULL, "iris-cnoc", 1000, 1000 },
> + { NULL, "iris-ddr", 1000, 15000000 },
This is a copy of the common data from the previous patches that you're
now dropping (why was it there in the first place then?). Is it specific
to this platform, or can it be reused?
> +};
> +
> +static const struct clock_info sm8550_clk_table[] = {
> + { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0 },
> + { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0 },
> + { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1 },
> +};
Are these the input pad names?
> +
> +static const char * const sm8550_clk_reset_table[] = { "video_axi_reset", NULL };
Ditto
> +
> +static const char * const sm8550_pd_table[] = { "iris-ctl", "vcodec", NULL };
Ditto
> +
> +static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx", NULL };
Ditto
> +
> +static const struct bw_info sm8550_bw_table_dec[] = {
> + { 2073600, 1608000, 2742000 }, /* 4096x2160@60 */
> + { 1036800, 826000, 1393000 }, /* 4096x2160@30 */
> + { 489600, 567000, 723000 }, /* 1920x1080@60 */
> + { 244800, 294000, 372000 }, /* 1920x1080@30 */
> +};
> +
> +static const struct reg_preset_info sm8550_reg_preset_table[] = {
> + { 0xB0088, 0x0, 0x11 },
lowercase hex for non-defines, please
> +};
> +
> +static struct ubwc_config_data ubwc_config_sm8550[] = {
> + UBWC_CONFIG(8, 32, 16, 0, 1, 1, 1),
I think it would be far more telling to drop this #define and fill
in the values inline
> +};
> +
> +struct platform_data sm8550_data = {
> + .bus_tbl = sm8550_bus_table,
> + .bus_tbl_size = ARRAY_SIZE(sm8550_bus_table),
> + .clk_tbl = sm8550_clk_table,
> + .clk_tbl_size = ARRAY_SIZE(sm8550_clk_table),
> + .clk_rst_tbl = sm8550_clk_reset_table,
> + .clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table),
> +
> + .bw_tbl_dec = sm8550_bw_table_dec,
> + .bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec),
> +
> + .pd_tbl = sm8550_pd_table,
> + .pd_tbl_size = ARRAY_SIZE(sm8550_pd_table),
> + .opp_pd_tbl = sm8550_opp_pd_table,
> + .opp_pd_tbl_size = ARRAY_SIZE(sm8550_opp_pd_table),
> +
> + .reg_prst_tbl = sm8550_reg_preset_table,
> + .reg_prst_tbl_size = ARRAY_SIZE(sm8550_reg_preset_table),
> + .fwname = "vpu30_4v.mbn",
> + .pas_id = 9,
> +
> + .core_data = core_data_sm8550,
> + .core_data_size = ARRAY_SIZE(core_data_sm8550),
> + .inst_cap_data = instance_cap_data_sm8550,
> + .inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550),
> + .ubwc_config = ubwc_config_sm8550,
Unless any of these are going to be reused, please inline all of the
values here..
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 14/34] media: iris: implement iris v4l2 file ops
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (12 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 11:57 ` Bryan O'Donoghue
2023-12-18 11:32 ` [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware Dikshita Agarwal
` (26 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
From: Vikash Garodia <quic_vgarodia@quicinc.com>
Implements iris v4l2 file ops - open, close and poll.
Open:
Configure the vb2 queue and v4l2 file handler. Initialize
a video instance and add the instance to core instance list.
Open a session in video firmware via HFI_CMD_OPEN.
Close:
De-initialize the instance and remove it from core
instance list. Close the session in firmware via HFI_CMD_CLOSE.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 2 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_common.h | 26 ++
.../media/platform/qcom/vcodec/iris/iris_core.c | 42 ++-
.../media/platform/qcom/vcodec/iris/iris_core.h | 8 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 62 +++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 2 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 115 +++++++-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 7 +
.../platform/qcom/vcodec/iris/iris_instance.h | 48 ++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 10 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 47 ++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 14 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 298 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 ++
17 files changed, 719 insertions(+), 10 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index c50e3241..3c076d0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -3,6 +3,8 @@ iris-objs += ../hfi_queue.o ../firmware.o
iris-objs += iris_probe.o \
iris_state.o \
iris_core.o \
+ iris_vidc.o \
+ iris_vdec.o \
iris_state.o \
iris_helpers.o \
iris_hfi.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index fb383b2..423ba1a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -9,6 +9,8 @@
#define HFI_VIDEO_ARCH_LX 0x1
#define HFI_CMD_INIT 0x01000001
+#define HFI_CMD_OPEN 0x01000003
+#define HFI_CMD_CLOSE 0x01000004
#define HFI_PROP_IMAGE_VERSION 0x03000001
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
new file mode 100644
index 0000000..3e4dd71
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#ifndef _IRIS_COMMON_H_
+#define _IRIS_COMMON_H_
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-v4l2.h>
+
+#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
+#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
+#define DEFAULT_WIDTH 320
+#define DEFAULT_HEIGHT 240
+#define DEFAULT_BUF_SIZE 115200
+
+enum signal_session_response {
+ SIGNAL_CMD_STOP_INPUT = 0,
+ SIGNAL_CMD_STOP_OUTPUT,
+ SIGNAL_CMD_CLOSE,
+ MAX_SIGNAL,
+};
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
index ba8960d..174fad9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
@@ -3,12 +3,14 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include <linux/delay.h>
+
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_state.h"
-static int iris_core_deinit_locked(struct iris_core *core)
+int iris_core_deinit_locked(struct iris_core *core)
{
int ret;
@@ -68,3 +70,41 @@ int iris_core_init(struct iris_core *core)
return ret;
}
+
+int iris_core_init_wait(struct iris_core *core)
+{
+ const int interval = 10;
+ int max_tries, count = 0, ret = 0;
+
+ mutex_lock(&core->lock);
+ if (!core_in_valid_state(core)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (core->state == IRIS_CORE_INIT)
+ goto unlock;
+
+ max_tries = core->cap[HW_RESPONSE_TIMEOUT].value / interval;
+ while (count < max_tries) {
+ if (core->state != IRIS_CORE_INIT_WAIT)
+ break;
+ msleep(interval);
+ count++;
+ }
+
+ if (core->state == IRIS_CORE_INIT) {
+ ret = 0;
+ goto unlock;
+ } else {
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ iris_core_deinit_locked(core);
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 3c8497a..b47520a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -33,6 +33,8 @@
* @clk_count: count of iris clocks
* @reset_tbl: table of iris reset clocks
* @reset_count: count of iris reset clocks
+ * @vb2_ops: iris vb2 ops
+ * @vb2_mem_ops: iris vb2 memory ops
* @state: current state of core
* @iface_q_table: Interface queue table memory
* @command_queue: shared interface queue to send commands to firmware
@@ -49,6 +51,7 @@
* @vpu_ops: a pointer to vpu ops
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
+ * @instances: a list_head of all instances
*/
struct iris_core {
@@ -67,6 +70,8 @@ struct iris_core {
u32 clk_count;
struct reset_info *reset_tbl;
u32 reset_count;
+ const struct vb2_ops *vb2_ops;
+ struct vb2_mem_ops *vb2_mem_ops;
enum iris_core_state state;
struct mem_desc iface_q_table;
struct iface_q_info command_queue;
@@ -83,9 +88,12 @@ struct iris_core {
const struct vpu_ops *vpu_ops;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
+ struct list_head instances;
};
int iris_core_init(struct iris_core *core);
+int iris_core_init_wait(struct iris_core *core);
int iris_core_deinit(struct iris_core *core);
+int iris_core_deinit_locked(struct iris_core *core);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index c31dfd5..872268d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -4,6 +4,8 @@
*/
#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_instance.h"
int check_core_lock(struct iris_core *core)
{
@@ -32,3 +34,63 @@ int iris_init_core_caps(struct iris_core *core)
return 0;
}
+
+static int process_inst_timeout(struct iris_inst *inst)
+{
+ struct iris_inst *instance;
+ struct iris_core *core;
+ bool found = false;
+ int ret = 0;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(instance, &core->instances, list) {
+ if (instance == inst) {
+ found = true;
+ break;
+ }
+ }
+ if (!found) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+
+ iris_core_deinit_locked(core);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int close_session(struct iris_inst *inst)
+{
+ u32 hw_response_timeout_val;
+ bool wait_for_response;
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+ hw_response_timeout_val = core->cap[HW_RESPONSE_TIMEOUT].value;
+ wait_for_response = true;
+ ret = iris_hfi_session_close(inst);
+ if (ret)
+ wait_for_response = false;
+
+ kfree(inst->packet);
+ inst->packet = NULL;
+
+ if (wait_for_response) {
+ ret = wait_for_completion_timeout(&inst->completions[SIGNAL_CMD_CLOSE],
+ msecs_to_jiffies(hw_response_timeout_val));
+ if (!ret) {
+ ret = -ETIMEDOUT;
+ process_inst_timeout(inst);
+ }
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 4c7ddbf..b9a6485 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -7,8 +7,10 @@
#define _IRIS_HELPERS_H_
#include "iris_core.h"
+#include "iris_instance.h"
int check_core_lock(struct iris_core *core);
int iris_init_core_caps(struct iris_core *core);
+int close_session(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index b0390b542..d144ae5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -8,6 +8,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
+#include "hfi_defines.h"
#include "vpu_common.h"
static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
@@ -43,6 +44,27 @@ static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
return ret;
}
+static bool validate_session(struct iris_core *core,
+ struct iris_inst *inst)
+{
+ struct iris_inst *temp;
+ bool valid = false;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return false;
+
+ list_for_each_entry(temp, &core->instances, list) {
+ if (temp == inst) {
+ valid = true;
+ break;
+ }
+ }
+
+ return valid;
+}
+
static int sys_init(struct iris_core *core)
{
int ret;
@@ -69,14 +91,6 @@ static int sys_image_version(struct iris_core *core)
return ret;
}
-#define CP_START 0
-#define CP_SIZE 0x25800000
-#define CP_NONPIXEL_START 0x01000000
-#define CP_NONPIXEL_SIZE 0x24800000
-
-#define FW_NAME "vpu30_4v.mbn"
-#define IRIS_PAS_ID 9
-
int iris_hfi_core_init(struct iris_core *core)
{
u32 cp_nonpixel_start, cp_nonpixel_size;
@@ -152,7 +166,90 @@ int iris_hfi_core_deinit(struct iris_core *core)
if (core->state == IRIS_CORE_DEINIT)
return 0;
- unload_fw(IRIS_PAS_ID);
+ unload_fw(core->platform_data->pas_id);
+
+ return ret;
+}
+
+int iris_hfi_session_open(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int ret;
+
+ inst->packet_size = 4096;
+ inst->packet = kzalloc(inst->packet_size, GFP_KERNEL);
+ if (!inst->packet)
+ return -ENOMEM;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto fail_free_packet;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_OPEN,
+ HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED,
+ HFI_PORT_NONE,
+ 0,
+ HFI_PAYLOAD_U32,
+ &inst->session_id,
+ sizeof(u32));
+ if (ret)
+ goto fail_free_packet;
+
+ ret = iris_hfi_queue_cmd_write(core, inst->packet);
+ if (ret)
+ goto fail_free_packet;
+
+ mutex_unlock(&core->lock);
+
+ return ret;
+
+fail_free_packet:
+ kfree(inst->packet);
+ inst->packet = NULL;
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_session_close(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_CLOSE,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ HFI_PORT_NONE,
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
return ret;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index fcf9f28..8a057cc 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -6,9 +6,12 @@
#ifndef _IRIS_HFI_H_
#define _IRIS_HFI_H_
+#include "iris_instance.h"
#include "iris_core.h"
int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
+int iris_hfi_session_open(struct iris_inst *inst);
+int iris_hfi_session_close(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index f31dd84..1ed572e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -211,3 +211,31 @@ int hfi_packet_image_version(struct iris_core *core,
return ret;
}
+
+int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
+ u32 flags, u32 port, u32 session_id,
+ u32 payload_type, void *payload,
+ u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+
+ ret = hfi_create_header(inst->packet, inst->packet_size,
+ session_id, core->header_id++);
+ if (ret)
+ return ret;
+
+ ret = hfi_create_packet(inst->packet,
+ inst->packet_size,
+ pkt_type,
+ flags,
+ payload_type,
+ port,
+ core->packet_id++,
+ payload,
+ payload_size);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index e36612c..9e476e9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -6,6 +6,9 @@
#ifndef _IRIS_HFI_PACKET_H_
#define _IRIS_HFI_PACKET_H_
+#include "iris_core.h"
+#include "iris_instance.h"
+
struct hfi_header {
u32 size;
u32 session_id;
@@ -67,5 +70,9 @@ int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
u8 *pkt, u32 pkt_size);
+int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
+ u32 flags, u32 port, u32 session_id,
+ u32 payload_type, void *payload,
+ u32 payload_size);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
new file mode 100644
index 0000000..1bbb533
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_INSTANCE_H_
+#define _IRIS_INSTANCE_H_
+
+#include <media/v4l2-ctrls.h>
+
+#include "iris_core.h"
+#include "iris_common.h"
+
+/**
+ * struct iris_inst - holds per video instance parameters
+ *
+ * @list: used for attach an instance to the core
+ * @core: pointer to core structure
+ * @session_id: id of current video session
+ * @vb2q_src: source vb2 queue
+ * @vb2q_dst: destination vb2 queue
+ * @ctx_q_lock: lock to serialize queues related ioctls
+ * @fh: reference of v4l2 file handler
+ * @fmt_src: structure of v4l2_format for source
+ * @fmt_dst: structure of v4l2_format for destination
+ * @ctrl_handler: reference of v4l2 ctrl handler
+ * @packet: HFI packet
+ * @packet_size: HFI packet size
+ * @completions: structure of signal completions
+ */
+
+struct iris_inst {
+ struct list_head list;
+ struct iris_core *core;
+ u32 session_id;
+ struct vb2_queue *vb2q_src;
+ struct vb2_queue *vb2q_dst;
+ struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */
+ struct v4l2_fh fh;
+ struct v4l2_format *fmt_src;
+ struct v4l2_format *fmt_dst;
+ struct v4l2_ctrl_handler ctrl_handler;
+ void *packet;
+ u32 packet_size;
+ struct completion completions[MAX_SIGNAL];
+};
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 9d6a6c5..4f20da8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -11,6 +11,7 @@
#include "iris_core.h"
#include "iris_helpers.h"
#include "resources.h"
+#include "iris_vidc.h"
static int iris_register_video_device(struct iris_core *core)
{
@@ -85,6 +86,8 @@ static int iris_probe(struct platform_device *pdev)
if (!core->packet)
return -ENOMEM;
+ INIT_LIST_HEAD(&core->instances);
+
core->reg_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(core->reg_base))
return PTR_ERR(core->reg_base);
@@ -107,6 +110,13 @@ static int iris_probe(struct platform_device *pdev)
return ret;
}
+ ret = init_ops(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init ops failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = init_resources(core);
if (ret) {
dev_err_probe(core->dev, ret,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
new file mode 100644
index 0000000..984be34
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_common.h"
+#include "iris_vdec.h"
+
+void vdec_inst_init(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+
+ inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL);
+ inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL);
+ inst->vb2q_src = kzalloc(sizeof(*inst->vb2q_src), GFP_KERNEL);
+ inst->vb2q_dst = kzalloc(sizeof(*inst->vb2q_dst), GFP_KERNEL);
+
+ f = inst->fmt_src;
+ f->type = INPUT_MPLANE;
+ f->fmt.pix_mp.width = DEFAULT_WIDTH;
+ f->fmt.pix_mp.height = DEFAULT_HEIGHT;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+
+ f = inst->fmt_dst;
+ f->type = OUTPUT_MPLANE;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_QC08C;
+ f->fmt.pix_mp.width = ALIGN(DEFAULT_WIDTH, 128);
+ f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32);
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128);
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+}
+
+void vdec_inst_deinit(struct iris_inst *inst)
+{
+ kfree(inst->fmt_dst);
+ kfree(inst->fmt_src);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
new file mode 100644
index 0000000..dc8f43f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VDEC_H_
+#define _IRIS_VDEC_H_
+
+#include "iris_instance.h"
+
+void vdec_inst_init(struct iris_inst *inst);
+void vdec_inst_deinit(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
new file mode 100644
index 0000000..d02da8b
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -0,0 +1,298 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_common.h"
+#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_instance.h"
+#include "iris_vdec.h"
+#include "iris_vidc.h"
+
+static int vidc_v4l2_fh_init(struct iris_inst *inst)
+{
+ struct iris_core *core;
+
+ core = inst->core;
+
+ if (inst->fh.vdev)
+ return -EINVAL;
+
+ v4l2_fh_init(&inst->fh, core->vdev_dec);
+ inst->fh.ctrl_handler = &inst->ctrl_handler;
+ v4l2_fh_add(&inst->fh);
+
+ return 0;
+}
+
+static int vidc_v4l2_fh_deinit(struct iris_inst *inst)
+{
+ if (!inst->fh.vdev)
+ return 0;
+
+ v4l2_fh_del(&inst->fh);
+ inst->fh.ctrl_handler = NULL;
+ v4l2_fh_exit(&inst->fh);
+
+ return 0;
+}
+
+static int vb2q_init(struct iris_inst *inst,
+ struct vb2_queue *q, enum v4l2_buf_type type)
+{
+ struct iris_core *core;
+
+ core = inst->core;
+
+ q->lock = &inst->ctx_q_lock;
+ q->type = type;
+ q->io_modes = VB2_MMAP | VB2_DMABUF;
+ q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ q->ops = core->vb2_ops;
+ q->mem_ops = core->vb2_mem_ops;
+ q->drv_priv = inst;
+ q->copy_timestamp = 1;
+ q->min_buffers_needed = 0;
+ return vb2_queue_init(q);
+}
+
+static int vidc_vb2_queue_init(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = vb2q_init(inst, inst->vb2q_src, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = vb2q_init(inst, inst->vb2q_dst, OUTPUT_MPLANE);
+ if (ret)
+ goto fail_vb2q_src_deinit;
+
+ return ret;
+
+fail_vb2q_src_deinit:
+ vb2_queue_release(inst->vb2q_src);
+
+ return ret;
+}
+
+static int vidc_vb2_queue_deinit(struct iris_inst *inst)
+{
+ vb2_queue_release(inst->vb2q_src);
+ kfree(inst->vb2q_src);
+ inst->vb2q_src = NULL;
+
+ vb2_queue_release(inst->vb2q_dst);
+ kfree(inst->vb2q_dst);
+ inst->vb2q_dst = NULL;
+
+ return 0;
+}
+
+static int vidc_add_session(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ struct iris_inst *i;
+ u32 count = 0;
+ int ret = 0;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ if (core->state != IRIS_CORE_INIT) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ list_for_each_entry(i, &core->instances, list)
+ count++;
+
+ if (count < core->cap[MAX_SESSION_COUNT].value)
+ list_add_tail(&inst->list, &core->instances);
+ else
+ ret = -EAGAIN;
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int vidc_remove_session(struct iris_inst *inst)
+{
+ struct iris_inst *i, *temp;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry_safe(i, temp, &core->instances, list) {
+ if (i->session_id == inst->session_id) {
+ list_del_init(&i->list);
+ break;
+ }
+ }
+ mutex_unlock(&core->lock);
+
+ return 0;
+}
+
+static struct iris_inst *get_vidc_inst(struct file *filp, void *fh)
+{
+ if (!filp || !filp->private_data)
+ return NULL;
+
+ return container_of(filp->private_data,
+ struct iris_inst, fh);
+}
+
+int vidc_open(struct file *filp)
+{
+ struct iris_core *core = video_drvdata(filp);
+ struct iris_inst *inst = NULL;
+ int i = 0;
+ int ret;
+
+ ret = iris_core_init(core);
+ if (ret)
+ return ret;
+
+ ret = iris_core_init_wait(core);
+ if (ret)
+ return ret;
+
+ inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ inst->core = core;
+ inst->session_id = hash32_ptr(inst);
+ mutex_init(&inst->ctx_q_lock);
+ for (i = 0; i < MAX_SIGNAL; i++)
+ init_completion(&inst->completions[i]);
+
+ ret = vidc_add_session(inst);
+ if (ret)
+ goto fail_free_inst;
+
+ ret = vidc_v4l2_fh_init(inst);
+ if (ret)
+ goto fail_remove_session;
+
+ vdec_inst_init(inst);
+
+ ret = vidc_vb2_queue_init(inst);
+ if (ret)
+ goto fail_inst_deinit;
+
+ ret = iris_hfi_session_open(inst);
+ if (ret) {
+ dev_err(core->dev, "%s: session open failed\n", __func__);
+ goto fail_core_deinit;
+ }
+ filp->private_data = &inst->fh;
+
+ return 0;
+
+fail_core_deinit:
+ iris_core_deinit(core);
+ vidc_vb2_queue_deinit(inst);
+fail_inst_deinit:
+ vdec_inst_deinit(inst);
+ vidc_v4l2_fh_deinit(inst);
+fail_remove_session:
+ vidc_remove_session(inst);
+fail_free_inst:
+ mutex_destroy(&inst->ctx_q_lock);
+ kfree(inst);
+
+ return ret;
+}
+
+int vidc_close(struct file *filp)
+{
+ struct iris_inst *inst;
+
+ inst = get_vidc_inst(filp, NULL);
+ if (!inst)
+ return -EINVAL;
+
+ vdec_inst_deinit(inst);
+ close_session(inst);
+ vidc_vb2_queue_deinit(inst);
+ vidc_v4l2_fh_deinit(inst);
+ vidc_remove_session(inst);
+ mutex_destroy(&inst->ctx_q_lock);
+ kfree(inst);
+
+ filp->private_data = NULL;
+
+ return 0;
+}
+
+static __poll_t get_poll_flags(struct iris_inst *inst, u32 plane)
+{
+ struct vb2_buffer *vb = NULL;
+ struct vb2_queue *q = NULL;
+ unsigned long flags = 0;
+ __poll_t poll = 0;
+
+ if (plane == INPUT_MPLANE)
+ q = inst->vb2q_src;
+ else if (plane == OUTPUT_MPLANE)
+ q = inst->vb2q_dst;
+
+ if (!q)
+ return EPOLLERR;
+
+ spin_lock_irqsave(&q->done_lock, flags);
+ if (!list_empty(&q->done_list))
+ vb = list_first_entry(&q->done_list, struct vb2_buffer,
+ done_entry);
+ if (vb && (vb->state == VB2_BUF_STATE_DONE ||
+ vb->state == VB2_BUF_STATE_ERROR)) {
+ if (plane == OUTPUT_MPLANE)
+ poll |= EPOLLIN | EPOLLRDNORM;
+ else if (plane == INPUT_MPLANE)
+ poll |= EPOLLOUT | EPOLLWRNORM;
+ }
+ spin_unlock_irqrestore(&q->done_lock, flags);
+
+ return poll;
+}
+
+static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
+{
+ struct iris_inst *inst;
+ __poll_t poll = 0;
+
+ inst = get_vidc_inst(filp, NULL);
+ if (!inst)
+ return EPOLLERR;
+
+ poll_wait(filp, &inst->fh.wait, pt);
+ poll_wait(filp, &inst->vb2q_src->done_wq, pt);
+ poll_wait(filp, &inst->vb2q_dst->done_wq, pt);
+
+ if (v4l2_event_pending(&inst->fh))
+ poll |= EPOLLPRI;
+
+ poll |= get_poll_flags(inst, INPUT_MPLANE);
+ poll |= get_poll_flags(inst, OUTPUT_MPLANE);
+
+ return poll;
+}
+
+static const struct v4l2_file_operations v4l2_file_ops = {
+ .owner = THIS_MODULE,
+ .open = vidc_open,
+ .release = vidc_close,
+ .unlocked_ioctl = video_ioctl2,
+ .poll = vidc_poll,
+};
+
+int init_ops(struct iris_core *core)
+{
+ core->v4l2_file_ops = &v4l2_file_ops;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
new file mode 100644
index 0000000..aebff44
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VIDC_H_
+#define _IRIS_VIDC_H_
+
+#include "iris_core.h"
+
+int init_ops(struct iris_core *core);
+int vidc_open(struct file *filp);
+int vidc_close(struct file *filp);
+
+#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 14/34] media: iris: implement iris v4l2 file ops
2023-12-18 11:32 ` [PATCH v2 14/34] media: iris: implement iris v4l2 file ops Dikshita Agarwal
@ 2023-12-19 11:57 ` Bryan O'Donoghue
0 siblings, 0 replies; 102+ messages in thread
From: Bryan O'Donoghue @ 2023-12-19 11:57 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 11:32, Dikshita Agarwal wrote:
> From: Vikash Garodia <quic_vgarodia@quicinc.com>
>
> Implements iris v4l2 file ops - open, close and poll.
>
> Open:
> Configure the vb2 queue and v4l2 file handler. Initialize
> a video instance and add the instance to core instance list.
> Open a session in video firmware via HFI_CMD_OPEN.
>
> Close:
> De-initialize the instance and remove it from core
> instance list. Close the session in firmware via HFI_CMD_CLOSE.
>
> Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> drivers/media/platform/qcom/vcodec/iris/Makefile | 2 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 26 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 42 ++-
> .../media/platform/qcom/vcodec/iris/iris_core.h | 8 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 62 +++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 2 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 115 +++++++-
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 7 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 48 ++++
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 10 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 47 ++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 14 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 298 +++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 ++
> 17 files changed, 719 insertions(+), 10 deletions(-)
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
>
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
> index c50e3241..3c076d0 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
> @@ -3,6 +3,8 @@ iris-objs += ../hfi_queue.o ../firmware.o
> iris-objs += iris_probe.o \
> iris_state.o \
> iris_core.o \
> + iris_vidc.o \
> + iris_vdec.o \
> iris_state.o \
> iris_helpers.o \
> iris_hfi.o \
> diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> index fb383b2..423ba1a 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> @@ -9,6 +9,8 @@
> #define HFI_VIDEO_ARCH_LX 0x1
>
> #define HFI_CMD_INIT 0x01000001
> +#define HFI_CMD_OPEN 0x01000003
> +#define HFI_CMD_CLOSE 0x01000004
>
> #define HFI_PROP_IMAGE_VERSION 0x03000001
>
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
> new file mode 100644
> index 0000000..3e4dd71
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
> @@ -0,0 +1,26 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +#ifndef _IRIS_COMMON_H_
> +#define _IRIS_COMMON_H_
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
> +#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
> +#define DEFAULT_WIDTH 320
> +#define DEFAULT_HEIGHT 240
> +#define DEFAULT_BUF_SIZE 115200
> +
> +enum signal_session_response {
> + SIGNAL_CMD_STOP_INPUT = 0,
> + SIGNAL_CMD_STOP_OUTPUT,
> + SIGNAL_CMD_CLOSE,
> + MAX_SIGNAL,
> +};
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
> index ba8960d..174fad9 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.c
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
> @@ -3,12 +3,14 @@
> * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> */
>
> +#include <linux/delay.h>
> +
> #include "iris_core.h"
> #include "iris_helpers.h"
> #include "iris_hfi.h"
> #include "iris_state.h"
>
> -static int iris_core_deinit_locked(struct iris_core *core)
> +int iris_core_deinit_locked(struct iris_core *core)
> {
> int ret;
>
> @@ -68,3 +70,41 @@ int iris_core_init(struct iris_core *core)
>
> return ret;
> }
> +
> +int iris_core_init_wait(struct iris_core *core)
> +{
> + const int interval = 10;
> + int max_tries, count = 0, ret = 0;
> +
> + mutex_lock(&core->lock);
> + if (!core_in_valid_state(core)) {
> + ret = -EINVAL;
> + goto unlock;
> + }
> +
> + if (core->state == IRIS_CORE_INIT)
> + goto unlock;
> +
> + max_tries = core->cap[HW_RESPONSE_TIMEOUT].value / interval;
> + while (count < max_tries) {
> + if (core->state != IRIS_CORE_INIT_WAIT)
> + break;
> + msleep(interval);
> + count++;
> + }
Is the intention here really to poll and sleep for 10 milliseconds and
if so why ?
Shouldn't there be some sort of bootup IRQ or event that we can trap ?
That's how it works in the venus driver right ? Just kick a workqueue
when the IRQ fires and have the iris_core_init_wait() do a
wait_for_completion_timeout().
> +int iris_hfi_session_open(struct iris_inst *inst)
> +{
> + struct iris_core *core;
> + int ret;
> +
> + inst->packet_size = 4096;
> + inst->packet = kzalloc(inst->packet_size, GFP_KERNEL);
> + if (!inst->packet)
> + return -ENOMEM;
Who is responsible for cleaning up this allocation ?
> +
> +fail_free_packet:
> + kfree(inst->packet);
freed here.
> +int iris_hfi_session_close(struct iris_inst *inst)
> +{
> }
but not here ?
Still finding this alot of code to go through.
Please try to break this up into smaller series.
---
bod
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (13 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 14/34] media: iris: implement iris v4l2 file ops Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 11:48 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities Dikshita Agarwal
` (25 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Allocate interrupt resources, enable the interrupt line and IRQ handling.
Register the IRQ handler to be called when interrupt occurs and
the function to be called from IRQ handler thread.
The threads invoke the driver's response handler which handles
all different responses from firmware.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 10 ++
.../media/platform/qcom/vcodec/iris/iris_core.h | 4 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 29 +++-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 184 +++++++++++++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 27 +++
.../media/platform/qcom/vcodec/iris/vpu_common.h | 2 +
.../media/platform/qcom/vcodec/iris/vpu_iris3.c | 44 +++++
10 files changed, 323 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 3c076d0..85fdf63 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -8,6 +8,7 @@ iris-objs += iris_probe.o \
iris_state.o \
iris_helpers.o \
iris_hfi.o \
+ iris_hfi_response.o \
iris_hfi_packet.o \
resources.o \
vpu_common.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 423ba1a..9dd5f11 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -6,6 +6,8 @@
#ifndef _HFI_DEFINES_H_
#define _HFI_DEFINES_H_
+#include <linux/types.h>
+
#define HFI_VIDEO_ARCH_LX 0x1
#define HFI_CMD_INIT 0x01000001
@@ -50,4 +52,12 @@
#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
+#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+
+struct hfi_debug_header {
+ u32 size;
+ u32 debug_level;
+ u32 reserved[2];
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index b47520a..c125bce 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -45,6 +45,7 @@
* @use_tz: a flag that suggests presence of trustzone
* @packet: pointer to packet from driver to fw
* @packet_size: size of packet
+ * @response_packet: a pointer to response packet from fw to driver
* @sys_init_id: id of sys init packet
* @header_id: id of packet header
* @packet_id: id of packet
@@ -52,6 +53,7 @@
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
* @instances: a list_head of all instances
+ * @intr_status: interrupt status
*/
struct iris_core {
@@ -82,6 +84,7 @@ struct iris_core {
unsigned int use_tz;
u8 *packet;
u32 packet_size;
+ u8 *response_packet;
u32 sys_init_id;
u32 header_id;
u32 packet_id;
@@ -89,6 +92,7 @@ struct iris_core {
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
struct list_head instances;
+ u32 intr_status;
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index d144ae5..4b8d8c7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -5,10 +5,11 @@
#include "../firmware.h"
#include "../hfi_queue.h"
+#include "hfi_defines.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
-#include "hfi_defines.h"
+#include "iris_hfi_response.h"
#include "vpu_common.h"
static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
@@ -253,3 +254,29 @@ int iris_hfi_session_close(struct iris_inst *inst)
return ret;
}
+
+irqreturn_t iris_hfi_isr(int irq, void *data)
+{
+ disable_irq_nosync(irq);
+
+ return IRQ_WAKE_THREAD;
+}
+
+irqreturn_t iris_hfi_isr_handler(int irq, void *data)
+{
+ struct iris_core *core = data;
+
+ if (!core)
+ return IRQ_NONE;
+
+ mutex_lock(&core->lock);
+ call_vpu_op(core, clear_interrupt, core);
+ mutex_unlock(&core->lock);
+
+ __response_handler(core);
+
+ if (!call_vpu_op(core, watchdog, core, core->intr_status))
+ enable_irq(irq);
+
+ return IRQ_HANDLED;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 8a057cc..8a62986 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -14,4 +14,7 @@ int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
int iris_hfi_session_close(struct iris_inst *inst);
+irqreturn_t iris_hfi_isr(int irq, void *data);
+irqreturn_t iris_hfi_isr_handler(int irq, void *data);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
new file mode 100644
index 0000000..829f3f6
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -0,0 +1,184 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "hfi_defines.h"
+#include "iris_hfi_packet.h"
+#include "iris_hfi_response.h"
+
+static void print_sfr_message(struct iris_core *core)
+{
+ struct sfr_buffer *vsfr = NULL;
+ u32 vsfr_size = 0;
+ void *p = NULL;
+
+ vsfr = (struct sfr_buffer *)core->sfr.kernel_vaddr;
+ if (vsfr) {
+ if (vsfr->bufsize != core->sfr.size)
+ return;
+ vsfr_size = vsfr->bufsize - sizeof(u32);
+ p = memchr(vsfr->rg_data, '\0', vsfr_size);
+ /* SFR isn't guaranteed to be NULL terminated */
+ if (!p)
+ vsfr->rg_data[vsfr_size - 1] = '\0';
+ }
+}
+
+static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt,
+ u32 core_resp_pkt_size)
+{
+ u32 response_pkt_size = 0;
+ u8 *response_limit;
+
+ if (!response_pkt || !core_resp_pkt || !core_resp_pkt_size)
+ return -EINVAL;
+
+ response_limit = core_resp_pkt + core_resp_pkt_size;
+
+ if (response_pkt < core_resp_pkt || response_pkt > response_limit)
+ return -EINVAL;
+
+ response_pkt_size = *(u32 *)response_pkt;
+ if (!response_pkt_size)
+ return -EINVAL;
+
+ if (response_pkt_size < sizeof(struct hfi_packet))
+ return -EINVAL;
+
+ if (response_pkt + response_pkt_size > response_limit)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int validate_hdr_packet(struct iris_core *core,
+ struct hfi_header *hdr)
+{
+ struct hfi_packet *packet;
+ int i, ret = 0;
+ u8 *pkt;
+
+ if (hdr->size < sizeof(*hdr) + sizeof(*packet))
+ return -EINVAL;
+
+ pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+
+ for (i = 0; i < hdr->num_packets; i++) {
+ packet = (struct hfi_packet *)pkt;
+ ret = validate_packet(pkt, core->response_packet, core->packet_size);
+ if (ret)
+ return ret;
+
+ pkt += packet->size;
+ }
+
+ return ret;
+}
+
+static int handle_system_error(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ print_sfr_message(core);
+
+ iris_core_deinit(core);
+
+ return 0;
+}
+
+static int handle_response(struct iris_core *core, void *response)
+{
+ struct hfi_header *hdr;
+ int ret;
+
+ hdr = (struct hfi_header *)response;
+ ret = validate_hdr_packet(core, hdr);
+ if (ret)
+ return handle_system_error(core, NULL);
+
+ return ret;
+}
+
+static int iris_hfi_queue_read(struct iris_core *core, void *pkt,
+ struct iface_q_info *q_info)
+{
+ u32 tx_req;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ if (!q_info || !q_info->q_array.kernel_vaddr || !pkt) {
+ dev_err(core->dev, "cannot read from shared Q's\n");
+ return -ENODATA;
+ }
+
+ if (read_queue(q_info, pkt, &tx_req))
+ return -ENODATA;
+
+ return 0;
+}
+
+int __response_handler(struct iris_core *core)
+{
+ int ret = 0;
+
+ if (call_vpu_op(core, watchdog, core, core->intr_status)) {
+ struct hfi_packet pkt = {.type = HFI_SYS_ERROR_WD_TIMEOUT};
+
+ mutex_lock(&core->lock);
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ dev_err(core->dev, "%s: CPU WD error received\n", __func__);
+ mutex_unlock(&core->lock);
+
+ return handle_system_error(core, &pkt);
+ }
+
+ memset(core->response_packet, 0, core->packet_size);
+ while (!iris_hfi_queue_read(core, core->response_packet, &core->message_queue)) {
+ ret = handle_response(core, core->response_packet);
+ if (ret)
+ continue;
+ if (core->state != IRIS_CORE_INIT)
+ break;
+ memset(core->response_packet, 0, core->packet_size);
+ }
+
+ iris_flush_debug_queue(core, core->response_packet, core->packet_size);
+
+ return ret;
+}
+
+void iris_flush_debug_queue(struct iris_core *core,
+ u8 *packet, u32 packet_size)
+{
+ struct hfi_debug_header *pkt;
+ bool local_packet = false;
+ u8 *log;
+
+ if (!packet || !packet_size) {
+ packet = kzalloc(IFACEQ_CORE_PKT_SIZE, GFP_KERNEL);
+ if (!packet)
+ return;
+
+ packet_size = IFACEQ_CORE_PKT_SIZE;
+
+ local_packet = true;
+ }
+
+ while (!iris_hfi_queue_read(core, packet, &core->debug_queue)) {
+ pkt = (struct hfi_debug_header *)packet;
+
+ if (pkt->size < sizeof(*pkt))
+ continue;
+
+ if (pkt->size >= packet_size)
+ continue;
+
+ packet[pkt->size] = '\0';
+ log = (u8 *)packet + sizeof(*pkt) + 1;
+ dev_dbg(core->dev, "%s", log);
+ }
+
+ if (local_packet)
+ kfree(packet);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
new file mode 100644
index 0000000..c1d2d27
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_HFI_RESPONSE_H_
+#define _IRIS_HFI_RESPONSE_H_
+
+#include "iris_core.h"
+
+struct sfr_buffer {
+ u32 bufsize;
+ u8 rg_data[];
+};
+
+int __response_handler(struct iris_core *core);
+void iris_flush_debug_queue(struct iris_core *core,
+ u8 *packet, u32 packet_size);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 4f20da8..8c591da 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -10,9 +10,25 @@
#include "../hfi_queue.h"
#include "iris_core.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
#include "resources.h"
#include "iris_vidc.h"
+static int init_iris_isr(struct iris_core *core)
+{
+ int ret;
+
+ ret = devm_request_threaded_irq(core->dev, core->irq, iris_hfi_isr,
+ iris_hfi_isr_handler, IRQF_TRIGGER_HIGH, "iris", core);
+ if (ret) {
+ dev_err(core->dev, "%s: Failed to allocate iris IRQ\n", __func__);
+ return ret;
+ }
+ disable_irq_nosync(core->irq);
+
+ return ret;
+}
+
static int iris_register_video_device(struct iris_core *core)
{
struct video_device *vdev;
@@ -86,6 +102,10 @@ static int iris_probe(struct platform_device *pdev)
if (!core->packet)
return -ENOMEM;
+ core->response_packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL);
+ if (!core->response_packet)
+ return -ENOMEM;
+
INIT_LIST_HEAD(&core->instances);
core->reg_base = devm_platform_ioremap_resource(pdev, 0);
@@ -96,6 +116,13 @@ static int iris_probe(struct platform_device *pdev)
if (core->irq < 0)
return core->irq;
+ ret = init_iris_isr(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: Failed to init isr with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = init_platform(core);
if (ret) {
dev_err_probe(core->dev, ret,
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
index 790496a..6512039 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
@@ -22,6 +22,8 @@ struct compat_handle {
struct vpu_ops {
int (*boot_firmware)(struct iris_core *core);
int (*raise_interrupt)(struct iris_core *core);
+ int (*clear_interrupt)(struct iris_core *core);
+ int (*watchdog)(struct iris_core *core, u32 intr_status);
};
int init_vpu(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
index 95bf223..a34d0ed 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
@@ -17,6 +17,8 @@
#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
+#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
+
/* HFI_CTRL_INIT */
#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48)
@@ -57,10 +59,19 @@
#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe
#define CTRL_ERROR_STATUS__M_IRIS3 \
CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3
+#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 \
+ CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3
+
+#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000
+#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C)
+#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4
#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150)
#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0
+#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+
static int setup_ucregion_memory_map_iris3(struct iris_core *core)
{
int ret;
@@ -153,9 +164,42 @@ static int raise_interrupt_iris3(struct iris_core *core)
return write_register(core, CPU_IC_SOFTINT_IRIS3, 1 << CPU_IC_SOFTINT_H2A_SHFT_IRIS3);
}
+static int clear_interrupt_iris3(struct iris_core *core)
+{
+ u32 intr_status = 0, mask = 0;
+ int ret;
+
+ ret = read_register(core, WRAPPER_INTR_STATUS_IRIS3, &intr_status);
+ if (ret)
+ return ret;
+
+ mask = (WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 |
+ WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 |
+ CTRL_INIT_IDLE_MSG_BMSK_IRIS3);
+
+ if (intr_status & mask)
+ core->intr_status |= intr_status;
+
+ ret = write_register(core, CPU_CS_A2HSOFTINTCLR_IRIS3, 1);
+
+ return ret;
+}
+
+static int watchdog_iris3(struct iris_core *core, u32 intr_status)
+{
+ if (intr_status & WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3) {
+ dev_err(core->dev, "%s: received watchdog interrupt\n", __func__);
+ return -ETIME;
+ }
+
+ return 0;
+}
+
static const struct vpu_ops iris3_ops = {
.boot_firmware = boot_firmware_iris3,
.raise_interrupt = raise_interrupt_iris3,
+ .clear_interrupt = clear_interrupt_iris3,
+ .watchdog = watchdog_iris3,
};
int init_iris3(struct iris_core *core)
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware
2023-12-18 11:32 ` [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware Dikshita Agarwal
@ 2023-12-19 11:48 ` Konrad Dybcio
0 siblings, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-19 11:48 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Allocate interrupt resources, enable the interrupt line and IRQ handling.
> Register the IRQ handler to be called when interrupt occurs and
> the function to be called from IRQ handler thread.
> The threads invoke the driver's response handler which handles
> all different responses from firmware.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> +
> +irqreturn_t iris_hfi_isr_handler(int irq, void *data)
> +{
> + struct iris_core *core = data;
> +
> + if (!core)
> + return IRQ_NONE;
Should this even be possible?
> +
> + mutex_lock(&core->lock);
> + call_vpu_op(core, clear_interrupt, core);
> + mutex_unlock(&core->lock);
> +
> + __response_handler(core);
> +
> + if (!call_vpu_op(core, watchdog, core, core->intr_status))
> + enable_irq(irq);
> +
> + return IRQ_HANDLED;
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> index 8a057cc..8a62986 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> @@ -14,4 +14,7 @@ int iris_hfi_core_deinit(struct iris_core *core);
> int iris_hfi_session_open(struct iris_inst *inst);
> int iris_hfi_session_close(struct iris_inst *inst);
>
> +irqreturn_t iris_hfi_isr(int irq, void *data);
> +irqreturn_t iris_hfi_isr_handler(int irq, void *data);
> +
> #endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> new file mode 100644
> index 0000000..829f3f6
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> @@ -0,0 +1,184 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include "hfi_defines.h"
> +#include "iris_hfi_packet.h"
> +#include "iris_hfi_response.h"
> +
> +static void print_sfr_message(struct iris_core *core)
I'm not sure how 'print' relates to what this function does
[...]
> +static int handle_system_error(struct iris_core *core,
> + struct hfi_packet *pkt)
> +{
> + print_sfr_message(core);
> +
> + iris_core_deinit(core);
Either take the return value of /\ into account, or make this function
void.
> +
> + return 0;
> +}
[...]
> +
> +struct sfr_buffer {
> + u32 bufsize;
> + u8 rg_data[];
Looks like you skipped static code checks.. Use __counted_by
[...]
> @@ -17,6 +17,8 @@
> #define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
> #define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
>
> +#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
You're mixing upper and lowercase hex throughout your defines.
[...]
> +static int clear_interrupt_iris3(struct iris_core *core)
> +{
> + u32 intr_status = 0, mask = 0;
> + int ret;
> +
> + ret = read_register(core, WRAPPER_INTR_STATUS_IRIS3, &intr_status);
> + if (ret)
> + return ret;
> +
> + mask = (WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 |
> + WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 |
> + CTRL_INIT_IDLE_MSG_BMSK_IRIS3);
unnecesary parentheses
> +
> + if (intr_status & mask)
> + core->intr_status |= intr_status;
> +
> + ret = write_register(core, CPU_CS_A2HSOFTINTCLR_IRIS3, 1);
> +
> + return ret;
why not return write_register directly?
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (14 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 11:50 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 17/34] media: iris: implement vb2_ops queue setup Dikshita Agarwal
` (24 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Implement iris v4l2_ctrl_ops - s_ctrl and g_volatile_ctrl.
Core capability structure has all the supported capabilities
for all supported codecs. Initializes instance capability
from core capability based on the codec type.
Add following to each capability:
Children: define dependencies for a specific capability.
Adjust: define function to adjust the value of capability
based on client configuration or dependency with
other capability.
Set: define function to set the adjusted value to firmware.
Prepare dependency graph for all inter-dependent capabilities.
This is used to adjust the value of capabilities and set the
same to firmware.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 679 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 30 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 31 +-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 13 +-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 29 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 26 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 3 +
.../platform/qcom/vcodec/iris/iris_instance.h | 9 +
.../media/platform/qcom/vcodec/iris/iris_probe.c | 8 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 16 +
.../platform/qcom/vcodec/iris/platform_common.h | 13 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 84 ++-
15 files changed, 921 insertions(+), 28 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 85fdf63..7fdee5b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -6,6 +6,7 @@ iris-objs += iris_probe.o \
iris_vidc.o \
iris_vdec.o \
iris_state.o \
+ iris_ctrls.o \
iris_helpers.o \
iris_hfi.o \
iris_hfi_response.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index c125bce..30f7ad7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -50,8 +50,10 @@
* @header_id: id of packet header
* @packet_id: id of packet
* @vpu_ops: a pointer to vpu ops
+ * @dec_codecs_count: supported codec count for decoder
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
+ * @inst_caps: a pointer to supported instance capabilities
* @instances: a list_head of all instances
* @intr_status: interrupt status
*/
@@ -89,8 +91,10 @@ struct iris_core {
u32 header_id;
u32 packet_id;
const struct vpu_ops *vpu_ops;
+ u32 dec_codecs_count;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
+ struct plat_inst_caps *inst_caps;
struct list_head instances;
u32 intr_status;
};
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
new file mode 100644
index 0000000..6b7aeaa
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -0,0 +1,679 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#include <linux/vmalloc.h>
+#include <linux/types.h>
+#include <linux/list.h>
+
+#include "platform_common.h"
+#include "iris_helpers.h"
+#include "iris_hfi_packet.h"
+#include "iris_hfi.h"
+#include "iris_ctrls.h"
+
+#define MIN_CAPTURE_BUFFERS 4
+#define MIN_OUTPUT_BUFFERS 4
+
+static inline bool are_all_childrens_visited(struct plat_inst_cap *cap,
+ bool lookup[INST_CAP_MAX])
+{
+ bool found = true;
+ int i;
+
+ for (i = 0; i < MAX_NUM_CHILD; i++) {
+ if (cap->children[i] == INST_CAP_NONE)
+ continue;
+
+ if (!lookup[cap->children[i]]) {
+ found = false;
+ break;
+ }
+ }
+
+ return found;
+}
+
+static bool is_valid_cap_id(enum plat_inst_cap_type cap_id)
+{
+ return cap_id > INST_CAP_NONE && cap_id < INST_CAP_MAX;
+}
+
+static enum plat_inst_cap_type get_cap_id(struct iris_inst *inst, u32 id)
+{
+ enum plat_inst_cap_type iter = INST_CAP_NONE;
+ enum plat_inst_cap_type cap_id = INST_CAP_NONE;
+
+ do {
+ if (inst->cap[iter].v4l2_id == id) {
+ cap_id = inst->cap[iter].cap_id;
+ break;
+ }
+ iter++;
+ } while (iter < INST_CAP_MAX);
+
+ return cap_id;
+}
+
+static int add_node_list(struct list_head *list, enum plat_inst_cap_type cap_id)
+{
+ struct cap_entry *entry = NULL;
+
+ entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&entry->list);
+ entry->cap_id = cap_id;
+ list_add(&entry->list, list);
+
+ return 0;
+}
+
+static int add_children(struct list_head *list,
+ struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ struct plat_inst_cap *cap;
+ int i, ret = 0;
+
+ cap = &inst->cap[cap_id];
+
+ for (i = 0; i < MAX_NUM_CHILD; i++) {
+ if (!cap->children[i])
+ break;
+
+ if (!is_valid_cap_id(cap->children[i]))
+ continue;
+
+ ret = add_node_list(list, cap->children[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+static int adjust_cap(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id,
+ struct v4l2_ctrl *ctrl)
+{
+ struct plat_inst_cap *cap;
+
+ cap = &inst->cap[cap_id];
+ if (!inst->cap[cap_id].cap_id)
+ return 0;
+
+ if (!cap->adjust) {
+ if (ctrl)
+ inst->cap[cap_id].value = ctrl->val;
+ return 0;
+ }
+
+ return cap->adjust(inst, ctrl);
+}
+
+static int adjust_dynamic_property(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id,
+ struct v4l2_ctrl *ctrl,
+ struct list_head *firmware_list,
+ struct list_head *children_list,
+ bool cap_present[INST_CAP_MAX])
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct plat_inst_cap *cap;
+ s32 prev_value;
+ int ret;
+
+ cap = &inst->cap[0];
+
+ if (!(cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED))
+ return -EBUSY;
+
+ prev_value = cap[cap_id].value;
+ ret = adjust_cap(inst, cap_id, ctrl);
+ if (ret)
+ return ret;
+
+ ret = add_node_list(firmware_list, cap_id);
+ if (ret)
+ return ret;
+ cap_present[cap->cap_id] = true;
+
+ if (cap[cap_id].value == prev_value)
+ return 0;
+
+ ret = add_children(children_list, inst, cap_id);
+ if (ret)
+ return ret;
+
+ list_for_each_entry_safe(entry, temp, children_list, list) {
+ if (!cap[entry->cap_id].adjust) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ continue;
+ }
+
+ prev_value = cap[entry->cap_id].value;
+ ret = adjust_cap(inst, entry->cap_id, NULL);
+ if (ret)
+ return ret;
+
+ if (cap[entry->cap_id].value != prev_value) {
+ if (!cap_present[cap->cap_id]) {
+ ret = add_node_list(firmware_list, cap_id);
+ if (ret)
+ return ret;
+ cap_present[cap->cap_id] = true;
+ }
+
+ ret = add_children(children_list, inst, entry->cap_id);
+ if (ret)
+ return ret;
+ }
+
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ if (!list_empty(children_list))
+ return -EINVAL;
+
+ return ret;
+}
+
+static int set_dynamic_property(struct iris_inst *inst,
+ struct list_head *firmware_list)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct plat_inst_cap *cap;
+ int ret = 0;
+
+ list_for_each_entry_safe(entry, temp, firmware_list, list) {
+ cap = &inst->cap[entry->cap_id];
+ if (cap->set) {
+ ret = cap->set(inst, entry->cap_id);
+ if (ret)
+ return -EINVAL;
+ }
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ return ret;
+}
+
+static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+{
+ enum plat_inst_cap_type cap_id;
+ struct iris_inst *inst = NULL;
+ int ret = 0;
+
+ switch (ctrl->id) {
+ case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
+ ctrl->val = MIN_CAPTURE_BUFFERS;
+ break;
+ case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
+ ctrl->val = MIN_OUTPUT_BUFFERS;
+ break;
+ default:
+ inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
+ cap_id = get_cap_id(inst, ctrl->id);
+ if (is_valid_cap_id(cap_id))
+ ctrl->val = inst->cap[cap_id].value;
+ else
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct list_head children_list, firmware_list;
+ enum plat_inst_cap_type cap_id;
+ bool cap_present[INST_CAP_MAX];
+ struct plat_inst_cap *cap;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
+ cap = &inst->cap[0];
+
+ INIT_LIST_HEAD(&firmware_list);
+ INIT_LIST_HEAD(&children_list);
+ memset(&cap_present, 0, sizeof(cap_present));
+
+ cap_id = get_cap_id(inst, ctrl->id);
+ if (!is_valid_cap_id(cap_id))
+ return -EINVAL;
+
+ cap[cap_id].flags |= CAP_FLAG_CLIENT_SET;
+
+ if (!inst->vb2q_src->streaming) {
+ inst->cap[cap_id].value = ctrl->val;
+ } else {
+ ret = adjust_dynamic_property(inst, cap_id, ctrl,
+ &firmware_list, &children_list,
+ cap_present);
+ if (ret)
+ goto free_list;
+
+ ret = set_dynamic_property(inst, &firmware_list);
+ }
+
+free_list:
+ list_for_each_entry_safe(entry, temp, &children_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ list_for_each_entry_safe(entry, temp, &firmware_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops ctrl_ops = {
+ .s_ctrl = vdec_op_s_ctrl,
+ .g_volatile_ctrl = vdec_op_g_volatile_ctrl,
+};
+
+int ctrls_init(struct iris_inst *inst)
+{
+ int num_ctrls = 0, ctrl_idx = 0;
+ struct plat_inst_cap *cap;
+ struct iris_core *core;
+ int idx = 0;
+ int ret = 0;
+
+ core = inst->core;
+ cap = &inst->cap[0];
+
+ for (idx = 0; idx < INST_CAP_MAX; idx++) {
+ if (cap[idx].v4l2_id)
+ num_ctrls++;
+ }
+ if (!num_ctrls)
+ return -EINVAL;
+
+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
+ INST_CAP_MAX * core->dec_codecs_count);
+ if (ret)
+ return ret;
+
+ for (idx = 0; idx < INST_CAP_MAX; idx++) {
+ struct v4l2_ctrl *ctrl;
+
+ if (!cap[idx].v4l2_id)
+ continue;
+
+ if (ctrl_idx >= num_ctrls) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ if (cap[idx].flags & CAP_FLAG_MENU) {
+ ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
+ &ctrl_ops,
+ cap[idx].v4l2_id,
+ cap[idx].max,
+ ~(cap[idx].step_or_mask),
+ cap[idx].value);
+ } else {
+ ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler,
+ &ctrl_ops,
+ cap[idx].v4l2_id,
+ cap[idx].min,
+ cap[idx].max,
+ cap[idx].step_or_mask,
+ cap[idx].value);
+ }
+ if (!ctrl) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ ret = inst->ctrl_handler.error;
+ if (ret)
+ goto error;
+
+ if ((cap[idx].flags & CAP_FLAG_VOLATILE) ||
+ (ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_CAPTURE ||
+ ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_OUTPUT))
+ ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ ctrl->flags |= V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
+ ctrl_idx++;
+ }
+ inst->num_ctrls = num_ctrls;
+
+ return 0;
+error:
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
+
+ return ret;
+}
+
+int iris_init_core_caps(struct iris_core *core)
+{
+ struct plat_core_cap *core_platform_data;
+ int i, num_core_caps;
+
+ core_platform_data = core->platform_data->core_data;
+ if (!core_platform_data)
+ return -EINVAL;
+
+ num_core_caps = core->platform_data->core_data_size;
+
+ for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) {
+ core->cap[core_platform_data[i].type].type = core_platform_data[i].type;
+ core->cap[core_platform_data[i].type].value = core_platform_data[i].value;
+ }
+
+ return 0;
+}
+
+static int update_inst_capability(struct plat_inst_cap *in,
+ struct plat_inst_caps *capability)
+{
+ if (!in || !capability)
+ return -EINVAL;
+
+ if (in->cap_id >= INST_CAP_MAX)
+ return -EINVAL;
+
+ capability->cap[in->cap_id].cap_id = in->cap_id;
+ capability->cap[in->cap_id].min = in->min;
+ capability->cap[in->cap_id].max = in->max;
+ capability->cap[in->cap_id].step_or_mask = in->step_or_mask;
+ capability->cap[in->cap_id].value = in->value;
+ capability->cap[in->cap_id].flags = in->flags;
+ capability->cap[in->cap_id].v4l2_id = in->v4l2_id;
+ capability->cap[in->cap_id].hfi_id = in->hfi_id;
+ memcpy(capability->cap[in->cap_id].children, in->children,
+ sizeof(capability->cap[in->cap_id].children));
+ capability->cap[in->cap_id].adjust = in->adjust;
+ capability->cap[in->cap_id].set = in->set;
+
+ return 0;
+}
+
+int iris_init_instance_caps(struct iris_core *core)
+{
+ struct plat_inst_cap *inst_plat_cap_data = NULL;
+ u8 dec_codecs_count = 0;
+ int num_inst_cap;
+ u32 dec_valid_codecs;
+ int i, j, check_bit = 0;
+ int ret = 0;
+
+ inst_plat_cap_data = core->platform_data->inst_cap_data;
+ if (!inst_plat_cap_data)
+ return -EINVAL;
+
+ dec_valid_codecs = core->cap[DEC_CODECS].value;
+ dec_codecs_count = hweight32(dec_valid_codecs);
+ core->dec_codecs_count = dec_codecs_count;
+
+ core->inst_caps = devm_kzalloc(core->dev,
+ dec_codecs_count * sizeof(struct plat_inst_caps),
+ GFP_KERNEL);
+ if (!core->inst_caps)
+ return -ENOMEM;
+
+ for (i = 0; i < dec_codecs_count; i++) {
+ while (check_bit < (sizeof(dec_valid_codecs) * 8)) {
+ if (dec_valid_codecs & BIT(check_bit)) {
+ core->inst_caps[i].codec = dec_valid_codecs &
+ BIT(check_bit);
+ check_bit++;
+ break;
+ }
+ check_bit++;
+ }
+ }
+
+ num_inst_cap = core->platform_data->inst_cap_data_size;
+
+ for (i = 0; i < num_inst_cap; i++) {
+ for (j = 0; j < dec_codecs_count; j++) {
+ if ((inst_plat_cap_data[i].codec &
+ core->inst_caps[j].codec)) {
+ ret = update_inst_capability(&inst_plat_cap_data[i],
+ &core->inst_caps[j]);
+ if (ret)
+ return ret;
+ }
+ }
+ }
+
+ return ret;
+}
+
+int get_inst_capability(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int i;
+
+ core = inst->core;
+
+ for (i = 0; i < core->dec_codecs_count; i++) {
+ if (core->inst_caps[i].codec == inst->codec) {
+ memcpy(&inst->cap[0], &core->inst_caps[i].cap[0],
+ (INST_CAP_MAX + 1) * sizeof(struct plat_inst_cap));
+ }
+ }
+
+ return 0;
+}
+
+int prepare_dependency_list(struct iris_inst *inst)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct plat_inst_cap *cap, *temp_cap;
+ int caps_to_prepare, pending_list_counter,
+ pending_at_start = 0;
+ struct list_head prepared_list, pending_list;
+ bool is_prepared[INST_CAP_MAX];
+ bool is_pending[INST_CAP_MAX];
+ int i, ret = 0;
+
+ cap = &inst->cap[0];
+
+ if (!list_empty(&inst->caps_list))
+ return 0;
+
+ INIT_LIST_HEAD(&prepared_list);
+ INIT_LIST_HEAD(&pending_list);
+ memset(&is_prepared, 0, sizeof(is_prepared));
+ memset(&is_pending, 0, sizeof(is_pending));
+
+ for (i = 1; i < INST_CAP_MAX; i++) {
+ temp_cap = &cap[i];
+ if (!is_valid_cap_id(temp_cap->cap_id))
+ continue;
+
+ if (!temp_cap->children[0]) {
+ if (!is_prepared[temp_cap->cap_id]) {
+ ret = add_node_list(&prepared_list, temp_cap->cap_id);
+ if (ret)
+ goto free_list;
+ is_prepared[temp_cap->cap_id] = true;
+ }
+ } else {
+ if (!is_pending[temp_cap->cap_id]) {
+ ret = add_node_list(&pending_list, temp_cap->cap_id);
+ if (ret)
+ goto free_list;
+ is_pending[temp_cap->cap_id] = true;
+ }
+ }
+ }
+
+ list_for_each_entry(entry, &pending_list, list)
+ pending_at_start++;
+
+ caps_to_prepare = pending_at_start;
+ pending_list_counter = pending_at_start;
+
+ list_for_each_entry_safe(entry, temp, &pending_list, list) {
+ list_del_init(&entry->list);
+ is_pending[entry->cap_id] = false;
+ pending_list_counter--;
+ temp_cap = &cap[entry->cap_id];
+
+ if (are_all_childrens_visited(temp_cap, is_prepared)) {
+ list_add(&entry->list, &prepared_list);
+ is_prepared[entry->cap_id] = true;
+ caps_to_prepare--;
+ } else {
+ list_add_tail(&entry->list, &pending_list);
+ is_pending[entry->cap_id] = true;
+ }
+
+ if (!pending_list_counter) {
+ if (pending_at_start == caps_to_prepare) {
+ ret = -EINVAL;
+ goto free_list;
+ }
+ pending_at_start = caps_to_prepare;
+ pending_list_counter = caps_to_prepare;
+ }
+ }
+
+ if (!list_empty(&pending_list)) {
+ ret = -EINVAL;
+ goto free_list;
+ }
+
+ list_replace_init(&prepared_list, &inst->caps_list);
+
+free_list:
+ list_for_each_entry_safe(entry, temp, &pending_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+ list_for_each_entry_safe(entry, temp, &prepared_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ return ret;
+}
+
+int set_u32_enum(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_value, sizeof(u32));
+}
+
+int set_u32(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
+int set_stage(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ struct v4l2_format *inp_f;
+ u32 work_mode = STAGE_2;
+ u32 width, height;
+ u32 hfi_id;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ inp_f = inst->fmt_src;
+ height = inp_f->fmt.pix_mp.height;
+ width = inp_f->fmt.pix_mp.width;
+ if (res_is_less_than(width, height, 1280, 720))
+ work_mode = STAGE_1;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &work_mode, sizeof(u32));
+}
+
+int set_pipe(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 work_route, hfi_id;
+
+ work_route = inst->cap[PIPE].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &work_route, sizeof(u32));
+}
+
+int adjust_v4l2_properties(struct iris_inst *inst)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ int ret = 0;
+
+ list_for_each_entry_safe(entry, temp, &inst->caps_list, list) {
+ ret = adjust_cap(inst, entry->cap_id, NULL);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int adjust_output_order(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 display_delay = -1, display_delay_enable = -1;
+ u32 adjusted_value;
+
+ adjusted_value = ctrl ? ctrl->val :
+ inst->cap[OUTPUT_ORDER].value;
+
+ display_delay = inst->cap[DISPLAY_DELAY].value;
+ display_delay_enable = inst->cap[DISPLAY_DELAY_ENABLE].value;
+
+ if (display_delay_enable && !display_delay)
+ adjusted_value = 1;
+
+ inst->cap[OUTPUT_ORDER].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 adjusted_value;
+ s32 pix_fmt = -1;
+
+ adjusted_value = ctrl ? ctrl->val :
+ inst->cap[PROFILE].value;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+
+ if (pix_fmt == FMT_TP10C)
+ adjusted_value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10;
+ else
+ adjusted_value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN;
+
+ inst->cap[PROFILE].value = adjusted_value;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
new file mode 100644
index 0000000..0f67f4f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_CTRLS_H_
+#define _IRIS_CTRLS_H_
+
+#include "iris_instance.h"
+
+struct cap_entry {
+ struct list_head list;
+ enum plat_inst_cap_type cap_id;
+};
+
+int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_pipe(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_u32(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int adjust_output_order(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int vidc_ctrl_handler_deinit(struct iris_inst *inst);
+int prepare_dependency_list(struct iris_inst *inst);
+int iris_init_instance_caps(struct iris_core *core);
+int iris_init_core_caps(struct iris_core *core);
+int get_inst_capability(struct iris_inst *inst);
+int adjust_v4l2_properties(struct iris_inst *inst);
+int ctrls_init(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 872268d..8d8bc3a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -5,6 +5,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
#include "iris_instance.h"
int check_core_lock(struct iris_core *core)
@@ -16,23 +17,29 @@ int check_core_lock(struct iris_core *core)
return fatal ? -EINVAL : 0;
}
-int iris_init_core_caps(struct iris_core *core)
+bool res_is_less_than(u32 width, u32 height,
+ u32 ref_width, u32 ref_height)
{
- struct plat_core_cap *core_platform_data;
- int i, num_core_caps;
+ u32 num_mbs = NUM_MBS_PER_FRAME(height, width);
+ u32 max_side = max(ref_width, ref_height);
- core_platform_data = core->platform_data->core_data;
- if (!core_platform_data)
- return -EINVAL;
+ if (num_mbs < NUM_MBS_PER_FRAME(ref_height, ref_width) &&
+ width < max_side &&
+ height < max_side)
+ return true;
- num_core_caps = core->platform_data->core_data_size;
+ return false;
+}
- for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) {
- core->cap[core_platform_data[i].type].type = core_platform_data[i].type;
- core->cap[core_platform_data[i].type].value = core_platform_data[i].value;
- }
+u32 get_port_info(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT)
+ return HFI_PORT_BITSTREAM;
+ else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT)
+ return HFI_PORT_RAW;
- return 0;
+ return HFI_PORT_NONE;
}
static int process_inst_timeout(struct iris_inst *inst)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index b9a6485..60c79124 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -6,11 +6,22 @@
#ifndef _IRIS_HELPERS_H_
#define _IRIS_HELPERS_H_
+#include <linux/align.h>
+#include <linux/types.h>
+
#include "iris_core.h"
#include "iris_instance.h"
+#include "platform_common.h"
+
+#define NUM_MBS_PER_FRAME(__height, __width) \
+ ((ALIGN(__height, 16) / 16) * (ALIGN(__width, 16) / 16))
int check_core_lock(struct iris_core *core);
-int iris_init_core_caps(struct iris_core *core);
+bool res_is_less_than(u32 width, u32 height,
+ u32 ref_width, u32 ref_height);
+u32 get_port_info(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+
int close_session(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 4b8d8c7..24ddb98 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -10,6 +10,7 @@
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
#include "iris_hfi_response.h"
+#include "iris_helpers.h"
#include "vpu_common.h"
static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
@@ -280,3 +281,31 @@ irqreturn_t iris_hfi_isr_handler(int irq, void *data)
return IRQ_HANDLED;
}
+
+int iris_hfi_set_property(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ ret = hfi_packet_session_property(inst,
+ packet_type,
+ flag,
+ plane,
+ payload_type,
+ payload,
+ payload_size);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 8a62986..bf991bb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -13,6 +13,9 @@ int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
int iris_hfi_session_close(struct iris_inst *inst);
+int iris_hfi_set_property(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size);
irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index 1ed572e..749d978 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -239,3 +239,29 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
return ret;
}
+
+int hfi_packet_session_property(struct iris_inst *inst,
+ u32 pkt_type, u32 flags, u32 port,
+ u32 payload_type, void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+
+ ret = hfi_create_header(inst->packet, inst->packet_size,
+ inst->session_id, core->header_id++);
+ if (ret)
+ return ret;
+
+ ret = hfi_create_packet(inst->packet, inst->packet_size,
+ pkt_type,
+ flags,
+ payload_type,
+ port,
+ core->packet_id++,
+ payload,
+ payload_size);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 9e476e9..bea7ed9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -74,5 +74,8 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
u32 flags, u32 port, u32 session_id,
u32 payload_type, void *payload,
u32 payload_size);
+int hfi_packet_session_property(struct iris_inst *inst,
+ u32 pkt_type, u32 flags, u32 port,
+ u32 payload_type, void *payload, u32 payload_size);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 1bbb533..f6a3066 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -10,6 +10,7 @@
#include "iris_core.h"
#include "iris_common.h"
+#include "platform_common.h"
/**
* struct iris_inst - holds per video instance parameters
@@ -27,6 +28,10 @@
* @packet: HFI packet
* @packet_size: HFI packet size
* @completions: structure of signal completions
+ * @cap: array of supported instance capabilities
+ * @num_ctrls: supported number of controls
+ * @caps_list: list head of capability
+ * @codec: codec type
*/
struct iris_inst {
@@ -43,6 +48,10 @@ struct iris_inst {
void *packet;
u32 packet_size;
struct completion completions[MAX_SIGNAL];
+ struct plat_inst_cap cap[INST_CAP_MAX + 1];
+ u32 num_ctrls;
+ struct list_head caps_list;
+ enum codec_type codec;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 8c591da..50fb93e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -13,6 +13,7 @@
#include "iris_hfi.h"
#include "resources.h"
#include "iris_vidc.h"
+#include "iris_ctrls.h"
static int init_iris_isr(struct iris_core *core)
{
@@ -158,6 +159,13 @@ static int iris_probe(struct platform_device *pdev)
return ret;
}
+ ret = iris_init_instance_caps(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init inst caps failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = v4l2_device_register(dev, &core->v4l2_dev);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index d02da8b..3a26edb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -9,6 +9,7 @@
#include "iris_instance.h"
#include "iris_vdec.h"
#include "iris_vidc.h"
+#include "iris_ctrls.h"
static int vidc_v4l2_fh_init(struct iris_inst *inst)
{
@@ -174,6 +175,10 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_free_inst;
+ INIT_LIST_HEAD(&inst->caps_list);
+ for (i = 0; i < MAX_SIGNAL; i++)
+ init_completion(&inst->completions[i]);
+
ret = vidc_v4l2_fh_init(inst);
if (ret)
goto fail_remove_session;
@@ -184,6 +189,14 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_inst_deinit;
+ ret = get_inst_capability(inst);
+ if (ret)
+ goto fail_queue_deinit;
+
+ ret = ctrls_init(inst);
+ if (ret)
+ goto fail_queue_deinit;
+
ret = iris_hfi_session_open(inst);
if (ret) {
dev_err(core->dev, "%s: session open failed\n", __func__);
@@ -194,7 +207,9 @@ int vidc_open(struct file *filp)
return 0;
fail_core_deinit:
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
iris_core_deinit(core);
+fail_queue_deinit:
vidc_vb2_queue_deinit(inst);
fail_inst_deinit:
vdec_inst_deinit(inst);
@@ -216,6 +231,7 @@ int vidc_close(struct file *filp)
if (!inst)
return -EINVAL;
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
close_session(inst);
vidc_vb2_queue_deinit(inst);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index e478b02..8305c65 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -7,8 +7,10 @@
#define _PLATFORM_COMMON_H_
#include <linux/bits.h>
+#include <media/v4l2-ctrls.h>
struct iris_core;
+struct iris_inst;
#define HW_RESPONSE_TIMEOUT_VALUE (1000)
@@ -17,6 +19,7 @@ struct iris_core;
#define CODED_FRAMES_PROGRESSIVE 0x0
#define CODED_FRAMES_INTERLACE 0x1
+#define MAX_NUM_CHILD 10
#define DEFAULT_MAX_HOST_BUF_COUNT 64
#define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256
@@ -163,6 +166,16 @@ struct plat_inst_cap {
u32 v4l2_id;
u32 hfi_id;
enum plat_inst_cap_flags flags;
+ enum plat_inst_cap_type children[MAX_NUM_CHILD];
+ int (*adjust)(struct iris_inst *inst,
+ struct v4l2_ctrl *ctrl);
+ int (*set)(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+};
+
+struct plat_inst_caps {
+ enum codec_type codec;
+ struct plat_inst_cap cap[INST_CAP_MAX + 1];
};
struct platform_data {
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index c75017e..0759ac5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -9,6 +9,7 @@
#include <media/v4l2-ctrls.h>
#include "hfi_defines.h"
+#include "iris_ctrls.h"
#include "platform_common.h"
#include "resources.h"
@@ -47,7 +48,16 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
FMT_NV12 | FMT_NV21 | FMT_NV12C,
FMT_NV12C},
- {PIX_FMTS, HEVC | VP9,
+ {PIX_FMTS, HEVC,
+ FMT_NV12,
+ FMT_TP10C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
+ FMT_NV12C,
+ 0, 0,
+ CAP_FLAG_NONE,
+ {PROFILE}},
+
+ {PIX_FMTS, VP9,
FMT_NV12,
FMT_TP10C,
FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
@@ -88,7 +98,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
V4L2_CID_MPEG_VIDEO_H264_PROFILE,
HFI_PROP_PROFILE,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},
{PROFILE, HEVC,
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
@@ -99,7 +112,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
HFI_PROP_PROFILE,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ adjust_profile,
+ set_u32_enum},
{PROFILE, VP9,
V4L2_MPEG_VIDEO_VP9_PROFILE_0,
@@ -109,7 +125,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_VP9_PROFILE_0,
V4L2_CID_MPEG_VIDEO_VP9_PROFILE,
HFI_PROP_PROFILE,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},
{LEVEL, H264,
V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
@@ -137,7 +156,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_H264_LEVEL_6_1,
V4L2_CID_MPEG_VIDEO_H264_LEVEL,
HFI_PROP_LEVEL,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},
{LEVEL, HEVC,
V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
@@ -158,7 +180,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1,
V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
HFI_PROP_LEVEL,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},
{LEVEL, VP9,
V4L2_MPEG_VIDEO_VP9_LEVEL_1_0,
@@ -178,7 +203,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
V4L2_CID_MPEG_VIDEO_VP9_LEVEL,
HFI_PROP_LEVEL,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},
{HEVC_TIER, HEVC,
V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
@@ -188,46 +216,69 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
V4L2_CID_MPEG_VIDEO_HEVC_TIER,
HFI_PROP_TIER,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},
{DISPLAY_DELAY_ENABLE, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE,
HFI_PROP_DECODE_ORDER_OUTPUT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {OUTPUT_ORDER},
+ NULL,
+ NULL},
{DISPLAY_DELAY, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY,
HFI_PROP_DECODE_ORDER_OUTPUT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {OUTPUT_ORDER},
+ NULL,
+ NULL},
{OUTPUT_ORDER, CODECS_ALL,
0, 1, 1, 0,
0,
HFI_PROP_DECODE_ORDER_OUTPUT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {0},
+ adjust_output_order,
+ set_u32},
{INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL,
DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT,
1, DEFAULT_MAX_HOST_BUF_COUNT,
0,
HFI_PROP_BUFFER_HOST_MAX_COUNT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {0},
+ NULL,
+ set_u32},
{STAGE, CODECS_ALL,
STAGE_1,
STAGE_2, 1,
STAGE_2,
0,
- HFI_PROP_STAGE},
+ HFI_PROP_STAGE,
+ CAP_FLAG_NONE,
+ {0},
+ NULL,
+ set_stage},
{PIPE, CODECS_ALL,
PIPE_1,
PIPE_4, 1,
PIPE_4,
0,
- HFI_PROP_PIPE},
+ HFI_PROP_PIPE,
+ CAP_FLAG_NONE,
+ {0},
+ NULL,
+ set_pipe},
{POC, H264, 0, 2, 1, 1,
0,
@@ -252,7 +303,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
0, 1, 1, 1,
0,
HFI_PROP_DEC_START_FROM_RAP_FRAME,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {0},
+ NULL,
+ set_u32},
};
static const struct bus_info sm8550_bus_table[] = {
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
2023-12-18 11:32 ` [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities Dikshita Agarwal
@ 2023-12-19 11:50 ` Konrad Dybcio
0 siblings, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-19 11:50 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Implement iris v4l2_ctrl_ops - s_ctrl and g_volatile_ctrl.
>
> Core capability structure has all the supported capabilities
> for all supported codecs. Initializes instance capability
> from core capability based on the codec type.
>
> Add following to each capability:
> Children: define dependencies for a specific capability.
> Adjust: define function to adjust the value of capability
> based on client configuration or dependency with
> other capability.
> Set: define function to set the adjusted value to firmware.
>
> Prepare dependency graph for all inter-dependent capabilities.
> This is used to adjust the value of capabilities and set the
> same to firmware.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> +static inline bool are_all_childrens_visited(struct plat_inst_cap *cap,
> + bool lookup[INST_CAP_MAX])
he's making a list, he's checking it twice..
I still think moving to a predefined config struct instead would
be a good idea
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 17/34] media: iris: implement vb2_ops queue setup
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (15 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 11:56 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops Dikshita Agarwal
` (23 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Implement queue_setup vb2_ops.
Calculate the buffer count and buffer size as par video
hardware requirement and updates to client.
Also, allocate the video driver buffers for output and
capture plane.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +-
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 179 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 49 ++++++
.../media/platform/qcom/vcodec/iris/iris_common.h | 71 +++++++-
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 24 +++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +-
.../platform/qcom/vcodec/iris/iris_instance.h | 6 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 71 ++++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 15 ++
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 5 +-
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 19 ++-
.../platform/qcom/vcodec/iris/platform_common.h | 16 +-
12 files changed, 442 insertions(+), 21 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 7fdee5b..a94e36b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -1,9 +1,10 @@
-iris-objs += ../hfi_queue.o ../firmware.o
+iris-objs += ../hfi_queue.o ../firmware.o ../buffers.o
iris-objs += iris_probe.o \
iris_state.o \
iris_core.o \
iris_vidc.o \
+ iris_vb2.o \
iris_vdec.o \
iris_state.o \
iris_ctrls.o \
@@ -11,6 +12,7 @@ iris-objs += iris_probe.o \
iris_hfi.o \
iris_hfi_response.o \
iris_hfi_packet.o \
+ iris_buffer.o \
resources.o \
vpu_common.o \
vpu_iris3.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
new file mode 100644
index 0000000..b9cffbf
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -0,0 +1,179 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "../buffers.h"
+#include "iris_buffer.h"
+#include "iris_helpers.h"
+#include "iris_instance.h"
+
+static int input_min_count(struct iris_inst *inst)
+{
+ return MIN_BUFFERS;
+}
+
+static int output_min_count(struct iris_inst *inst)
+{
+ int output_min_count;
+
+ switch (inst->codec) {
+ case H264:
+ case HEVC:
+ output_min_count = 4;
+ break;
+ case VP9:
+ output_min_count = 9;
+ break;
+ default:
+ output_min_count = 4;
+ break;
+ }
+
+ return output_min_count;
+}
+
+int iris_get_buf_min_count(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return input_min_count(inst);
+ case BUF_OUTPUT:
+ return output_min_count(inst);
+ default:
+ return 0;
+ }
+}
+
+static u32 input_buffer_size(struct iris_inst *inst)
+{
+ u32 base_res_mbs = NUM_MBS_4k;
+ u32 frame_size, num_mbs;
+ struct v4l2_format *f;
+ u32 div_factor = 1;
+ u32 codec;
+
+ f = inst->fmt_src;
+ codec = f->fmt.pix_mp.pixelformat;
+
+ num_mbs = get_mbpf(inst);
+ if (num_mbs > NUM_MBS_4k) {
+ div_factor = 4;
+ base_res_mbs = inst->cap[MBPF].value;
+ } else {
+ base_res_mbs = NUM_MBS_4k;
+ if (codec == V4L2_PIX_FMT_VP9)
+ div_factor = 1;
+ else
+ div_factor = 2;
+ }
+
+ frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor;
+
+ /* multiply by 10/8 (1.25) to get size for 10 bit case */
+ if (codec == V4L2_PIX_FMT_VP9 || codec == V4L2_PIX_FMT_HEVC)
+ frame_size = frame_size + (frame_size >> 2);
+
+ return ALIGN(frame_size, SZ_4K);
+}
+
+static u32 output_buffer_size(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ u32 size;
+
+ f = inst->fmt_dst;
+
+ size = video_raw_buffer_size(f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
+ f->fmt.pix_mp.height);
+ return size;
+}
+
+int iris_get_buffer_size(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return input_buffer_size(inst);
+ case BUF_OUTPUT:
+ return output_buffer_size(inst);
+ default:
+ return 0;
+ }
+}
+
+struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return &inst->buffers.input;
+ case BUF_OUTPUT:
+ return &inst->buffers.output;
+ case BUF_READ_ONLY:
+ return &inst->buffers.read_only;
+ case BUF_BIN:
+ return &inst->buffers.bin;
+ case BUF_ARP:
+ return &inst->buffers.arp;
+ case BUF_COMV:
+ return &inst->buffers.comv;
+ case BUF_NON_COMV:
+ return &inst->buffers.non_comv;
+ case BUF_LINE:
+ return &inst->buffers.line;
+ case BUF_DPB:
+ return &inst->buffers.dpb;
+ case BUF_PERSIST:
+ return &inst->buffers.persist;
+ case BUF_VPSS:
+ return &inst->buffers.vpss;
+ default:
+ return NULL;
+ }
+}
+
+int iris_allocate_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type,
+ u32 num_buffers)
+{
+ struct iris_buffer *buf = NULL;
+ struct iris_buffers *buffers;
+ int idx = 0;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return -EINVAL;
+
+ for (idx = 0; idx < num_buffers; idx++) {
+ buf = kzalloc(sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ return -EINVAL;
+
+ INIT_LIST_HEAD(&buf->list);
+ list_add_tail(&buf->list, &buffers->list);
+ buf->type = buf_type;
+ buf->index = idx;
+ }
+
+ return 0;
+}
+
+int iris_free_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ list_del_init(&buf->list);
+ kfree(buf);
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
new file mode 100644
index 0000000..1cd76a9
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_BUFFER_H_
+#define _IRIS_BUFFER_H_
+
+#define MIN_BUFFERS 4
+
+#include "iris_common.h"
+
+struct iris_inst;
+
+struct iris_buffers {
+ struct list_head list; // list of "struct iris_buffer"
+ u32 min_count;
+ u32 actual_count;
+ u32 size;
+ bool reuse;
+};
+
+struct iris_buffers_info {
+ struct iris_buffers input;
+ struct iris_buffers output;
+ struct iris_buffers read_only;
+ struct iris_buffers bin;
+ struct iris_buffers arp;
+ struct iris_buffers comv;
+ struct iris_buffers non_comv;
+ struct iris_buffers line;
+ struct iris_buffers dpb;
+ struct iris_buffers persist;
+ struct iris_buffers vpss;
+};
+
+int iris_get_buf_min_count(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+int iris_get_buffer_size(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+int iris_allocate_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type,
+ u32 num_buffers);
+int iris_free_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 3e4dd71..4edadc3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -5,6 +5,7 @@
#ifndef _IRIS_COMMON_H_
#define _IRIS_COMMON_H_
+#include <media/v4l2-dev.h>
#include <media/v4l2-device.h>
#include <media/v4l2-event.h>
#include <media/v4l2-ioctl.h>
@@ -14,7 +15,31 @@
#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
#define DEFAULT_WIDTH 320
#define DEFAULT_HEIGHT 240
-#define DEFAULT_BUF_SIZE 115200
+
+#define MB_IN_PIXEL (16 * 16)
+
+#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
+
+enum codec_type {
+ H264 = BIT(0),
+ HEVC = BIT(1),
+ VP9 = BIT(2),
+};
+
+enum colorformat_type {
+ FMT_NONE = 0,
+ FMT_NV12C = BIT(0),
+ FMT_NV12 = BIT(1),
+ FMT_NV21 = BIT(2),
+ FMT_TP10C = BIT(3),
+};
+
+struct rect_desc {
+ u32 left;
+ u32 top;
+ u32 width;
+ u32 height;
+};
enum signal_session_response {
SIGNAL_CMD_STOP_INPUT = 0,
@@ -23,4 +48,48 @@ enum signal_session_response {
MAX_SIGNAL,
};
+enum iris_buffer_type {
+ BUF_NONE,
+ BUF_INPUT,
+ BUF_OUTPUT,
+ BUF_READ_ONLY,
+ BUF_BIN,
+ BUF_ARP,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+ BUF_DPB,
+ BUF_PERSIST,
+ BUF_VPSS,
+};
+
+enum iris_buffer_attributes {
+ BUF_ATTR_DEFERRED = BIT(0),
+ BUF_ATTR_READ_ONLY = BIT(1),
+ BUF_ATTR_PENDING_RELEASE = BIT(2),
+ BUF_ATTR_QUEUED = BIT(3),
+ BUF_ATTR_DEQUEUED = BIT(4),
+ BUF_ATTR_BUFFER_DONE = BIT(5),
+};
+
+struct iris_buffer {
+ struct list_head list;
+ struct iris_inst *inst;
+ enum iris_buffer_type type;
+ u32 index;
+ int fd;
+ u32 buffer_size;
+ u32 data_offset;
+ u32 data_size;
+ u64 device_addr;
+ void *kvaddr;
+ unsigned long dma_attrs;
+ u32 flags;
+ u64 timestamp;
+ enum iris_buffer_attributes attr;
+ void *dmabuf;
+ struct sg_table *sg_table;
+ struct dma_buf_attachment *attach;
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 8d8bc3a..54a7851 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -42,6 +42,30 @@ u32 get_port_info(struct iris_inst *inst,
return HFI_PORT_NONE;
}
+enum iris_buffer_type v4l2_type_to_driver(u32 type)
+{
+ switch (type) {
+ case INPUT_MPLANE:
+ return BUF_INPUT;
+ case OUTPUT_MPLANE:
+ return BUF_OUTPUT;
+ default:
+ return 0;
+ }
+}
+
+int get_mbpf(struct iris_inst *inst)
+{
+ int height = 0, width = 0;
+ struct v4l2_format *inp_f;
+
+ inp_f = inst->fmt_src;
+ width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
+ height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+
+ return NUM_MBS_PER_FRAME(height, width);
+}
+
static int process_inst_timeout(struct iris_inst *inst)
{
struct iris_inst *instance;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 60c79124..3bae969 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -11,6 +11,7 @@
#include "iris_core.h"
#include "iris_instance.h"
+#include "iris_buffer.h"
#include "platform_common.h"
#define NUM_MBS_PER_FRAME(__height, __width) \
@@ -21,7 +22,8 @@ bool res_is_less_than(u32 width, u32 height,
u32 ref_width, u32 ref_height);
u32 get_port_info(struct iris_inst *inst,
enum plat_inst_cap_type cap_id);
-
+enum iris_buffer_type v4l2_type_to_driver(u32 type);
+int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index f6a3066..a5c6cb48 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -8,6 +8,8 @@
#include <media/v4l2-ctrls.h>
+#include "iris_buffer.h"
+#include "iris_common.h"
#include "iris_core.h"
#include "iris_common.h"
#include "platform_common.h"
@@ -25,6 +27,7 @@
* @fmt_src: structure of v4l2_format for source
* @fmt_dst: structure of v4l2_format for destination
* @ctrl_handler: reference of v4l2 ctrl handler
+ * @crop: structure of crop info
* @packet: HFI packet
* @packet_size: HFI packet size
* @completions: structure of signal completions
@@ -32,6 +35,7 @@
* @num_ctrls: supported number of controls
* @caps_list: list head of capability
* @codec: codec type
+ * @buffers: structure of buffer info
*/
struct iris_inst {
@@ -45,6 +49,7 @@ struct iris_inst {
struct v4l2_format *fmt_src;
struct v4l2_format *fmt_dst;
struct v4l2_ctrl_handler ctrl_handler;
+ struct rect_desc crop;
void *packet;
u32 packet_size;
struct completion completions[MAX_SIGNAL];
@@ -52,6 +57,7 @@ struct iris_inst {
u32 num_ctrls;
struct list_head caps_list;
enum codec_type codec;
+ struct iris_buffers_info buffers;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
new file mode 100644
index 0000000..b040d27
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_buffer.h"
+#include "iris_core.h"
+#include "iris_helpers.h"
+#include "iris_instance.h"
+#include "iris_vb2.h"
+
+int iris_vb2_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers, unsigned int *num_planes,
+ unsigned int sizes[], struct device *alloc_devs[])
+{
+ enum iris_buffer_type buffer_type = 0;
+ struct iris_buffers *buffers;
+ struct iris_inst *inst;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ int ret;
+
+ if (!q || !num_buffers || !num_planes || !sizes)
+ return -EINVAL;
+
+ inst = vb2_get_drv_priv(q);
+ if (!inst || !inst->core)
+ return -EINVAL;
+
+ core = inst->core;
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ f = inst->fmt_src;
+ else
+ f = inst->fmt_dst;
+
+ if (*num_planes) {
+ if (*num_planes != f->fmt.pix_mp.num_planes ||
+ sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage)
+ return -EINVAL;
+ }
+
+ buffer_type = v4l2_type_to_driver(q->type);
+ if (!buffer_type)
+ return -EINVAL;
+
+ ret = iris_free_buffers(inst, buffer_type);
+ if (ret)
+ return ret;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ buffers->min_count = iris_get_buf_min_count(inst, buffer_type);
+ if (*num_buffers < buffers->min_count)
+ *num_buffers = buffers->min_count;
+ buffers->actual_count = *num_buffers;
+ *num_planes = 1;
+
+ buffers->size = iris_get_buffer_size(inst, buffer_type);
+
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = buffers->size;
+ sizes[0] = f->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ ret = iris_allocate_buffers(inst, buffer_type, *num_buffers);
+
+ q->dev = core->dev;
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
new file mode 100644
index 0000000..8a8e1039
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VB2_H_
+#define _IRIS_VB2_H_
+
+#include <media/videobuf2-v4l2.h>
+
+int iris_vb2_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers, unsigned int *num_planes,
+ unsigned int sizes[], struct device *alloc_devs[]);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 984be34..1b957a13 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -3,6 +3,7 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_vdec.h"
@@ -22,7 +23,7 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;
f->fmt.pix_mp.num_planes = 1;
f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
- f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT);
f->fmt.pix_mp.field = V4L2_FIELD_NONE;
f = inst->fmt_dst;
@@ -32,7 +33,7 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32);
f->fmt.pix_mp.num_planes = 1;
f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128);
- f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
f->fmt.pix_mp.field = V4L2_FIELD_NONE;
f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 3a26edb..410de720 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -10,6 +10,7 @@
#include "iris_vdec.h"
#include "iris_vidc.h"
#include "iris_ctrls.h"
+#include "iris_vb2.h"
static int vidc_v4l2_fh_init(struct iris_inst *inst)
{
@@ -168,13 +169,22 @@ int vidc_open(struct file *filp)
inst->core = core;
inst->session_id = hash32_ptr(inst);
mutex_init(&inst->ctx_q_lock);
- for (i = 0; i < MAX_SIGNAL; i++)
- init_completion(&inst->completions[i]);
ret = vidc_add_session(inst);
if (ret)
goto fail_free_inst;
+ INIT_LIST_HEAD(&inst->buffers.input.list);
+ INIT_LIST_HEAD(&inst->buffers.output.list);
+ INIT_LIST_HEAD(&inst->buffers.read_only.list);
+ INIT_LIST_HEAD(&inst->buffers.bin.list);
+ INIT_LIST_HEAD(&inst->buffers.arp.list);
+ INIT_LIST_HEAD(&inst->buffers.comv.list);
+ INIT_LIST_HEAD(&inst->buffers.non_comv.list);
+ INIT_LIST_HEAD(&inst->buffers.line.list);
+ INIT_LIST_HEAD(&inst->buffers.dpb.list);
+ INIT_LIST_HEAD(&inst->buffers.persist.list);
+ INIT_LIST_HEAD(&inst->buffers.vpss.list);
INIT_LIST_HEAD(&inst->caps_list);
for (i = 0; i < MAX_SIGNAL; i++)
init_completion(&inst->completions[i]);
@@ -306,9 +316,14 @@ static const struct v4l2_file_operations v4l2_file_ops = {
.poll = vidc_poll,
};
+static const struct vb2_ops iris_vb2_ops = {
+ .queue_setup = iris_vb2_queue_setup,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
+ core->vb2_ops = &iris_vb2_ops;
return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 8305c65..e242614 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -9,6 +9,8 @@
#include <linux/bits.h>
#include <media/v4l2-ctrls.h>
+#include "iris_common.h"
+
struct iris_core;
struct iris_inst;
@@ -35,20 +37,6 @@ struct iris_inst;
.bank_spreading = bsp, \
}
-enum codec_type {
- H264 = BIT(0),
- HEVC = BIT(1),
- VP9 = BIT(2),
-};
-
-enum colorformat_type {
- FMT_NONE = 0,
- FMT_NV12C = BIT(0),
- FMT_NV12 = BIT(1),
- FMT_NV21 = BIT(2),
- FMT_TP10C = BIT(3),
-};
-
enum stage_type {
STAGE_NONE = 0,
STAGE_1 = 1,
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 17/34] media: iris: implement vb2_ops queue setup
2023-12-18 11:32 ` [PATCH v2 17/34] media: iris: implement vb2_ops queue setup Dikshita Agarwal
@ 2023-12-19 11:56 ` Konrad Dybcio
2023-12-20 8:37 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-19 11:56 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Implement queue_setup vb2_ops.
> Calculate the buffer count and buffer size as par video
> hardware requirement and updates to client.
> Also, allocate the video driver buffers for output and
> capture plane.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> +static int input_min_count(struct iris_inst *inst)
> +{
> + return MIN_BUFFERS;
> +}
Why is this a function?
> +
> +static int output_min_count(struct iris_inst *inst)
> +{
> + int output_min_count;
> +
> + switch (inst->codec) {
> + case H264:
> + case HEVC:
> + output_min_count = 4;
> + break;
> + case VP9:
> + output_min_count = 9;
> + break;
> + default:
> + output_min_count = 4;
> + break;
> + }
switch (inst->codec) {
case VP9:
return 9;
case H264:
case HEVC:
default:
return 4;
}
> +
> + return output_min_count;
> +}
> +
> +int iris_get_buf_min_count(struct iris_inst *inst,
> + enum iris_buffer_type buffer_type)
> +{
> + switch (buffer_type) {
> + case BUF_INPUT:
> + return input_min_count(inst);
> + case BUF_OUTPUT:
> + return output_min_count(inst);
> + default:
> + return 0;
> + }
> +}
> +
> +static u32 input_buffer_size(struct iris_inst *inst)
> +{
> + u32 base_res_mbs = NUM_MBS_4k;
> + u32 frame_size, num_mbs;
> + struct v4l2_format *f;
> + u32 div_factor = 1;
> + u32 codec;
> +
> + f = inst->fmt_src;
> + codec = f->fmt.pix_mp.pixelformat;
> +
> + num_mbs = get_mbpf(inst);
> + if (num_mbs > NUM_MBS_4k) {
> + div_factor = 4;
> + base_res_mbs = inst->cap[MBPF].value;
> + } else {
> + base_res_mbs = NUM_MBS_4k;
> + if (codec == V4L2_PIX_FMT_VP9)
> + div_factor = 1;
> + else
> + div_factor = 2;
> + }
> +
> + frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor;
that's a bit magic..
> +
> + /* multiply by 10/8 (1.25) to get size for 10 bit case */
misaligned
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 17/34] media: iris: implement vb2_ops queue setup
2023-12-19 11:56 ` Konrad Dybcio
@ 2023-12-20 8:37 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:37 UTC (permalink / raw)
To: Konrad Dybcio, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 5:26 PM, Konrad Dybcio wrote:
> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>> Implement queue_setup vb2_ops.
>> Calculate the buffer count and buffer size as par video
>> hardware requirement and updates to client.
>> Also, allocate the video driver buffers for output and
>> capture plane.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
> [...]
>
>> +static int input_min_count(struct iris_inst *inst)
>> +{
>> + return MIN_BUFFERS;
>> +}
> Why is this a function?
>
Some extra checks will be performed to calculate the input buffers for
encoder, hence it was made a function. May be I can directly use the macro
at this point and introduce the function later in the patch with encoder
patches.
>> +
>> +static int output_min_count(struct iris_inst *inst)
>> +{
>> + int output_min_count;
>> +
>> + switch (inst->codec) {
>> + case H264:
>> + case HEVC:
>> + output_min_count = 4;
>> + break;
>> + case VP9:
>> + output_min_count = 9;
>> + break;
>> + default:
>> + output_min_count = 4;
>> + break;
>> + }
>
> switch (inst->codec) {
> case VP9:
> return 9;
> case H264:
> case HEVC:
> default:
> return 4;
> }
>
Right, this is better, will make the change.
>> +
>> + return output_min_count;
>> +}
>> +
>> +int iris_get_buf_min_count(struct iris_inst *inst,
>> + enum iris_buffer_type buffer_type)
>> +{
>> + switch (buffer_type) {
>> + case BUF_INPUT:
>> + return input_min_count(inst);
>> + case BUF_OUTPUT:
>> + return output_min_count(inst);
>> + default:
>> + return 0;
>> + }
>> +}
>> +
>> +static u32 input_buffer_size(struct iris_inst *inst)
>> +{
>> + u32 base_res_mbs = NUM_MBS_4k;
>> + u32 frame_size, num_mbs;
>> + struct v4l2_format *f;
>> + u32 div_factor = 1;
>> + u32 codec;
>> +
>> + f = inst->fmt_src;
>> + codec = f->fmt.pix_mp.pixelformat;
>> +
>> + num_mbs = get_mbpf(inst);
>> + if (num_mbs > NUM_MBS_4k) {
>> + div_factor = 4;
>> + base_res_mbs = inst->cap[MBPF].value;
>> + } else {
>> + base_res_mbs = NUM_MBS_4k;
>> + if (codec == V4L2_PIX_FMT_VP9)
>> + div_factor = 1;
>> + else
>> + div_factor = 2;
>> + }
>> +
>> + frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor;
> that's a bit magic..
>
This is the formula to calculate video frame size, no magic here :)
>> +
>> + /* multiply by 10/8 (1.25) to get size for 10 bit case */
> misaligned
>
Oh! wasn't caught by check-patch.
Thanks for pointing this, Will fix.
>
> Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (16 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 17/34] media: iris: implement vb2_ops queue setup Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 11:58 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 19/34] media: iris: implement HFI to queue and release buffers Dikshita Agarwal
` (22 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
From: Vikash Garodia <quic_vgarodia@quicinc.com>
Implement the iris vb2 mem ops for buffer management for
DMABUF streaming mode. Update video driver buffer
with dma buf information.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/iris_probe.c | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 147 +++++++++++++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 10 ++
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 11 ++
4 files changed, 169 insertions(+)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 50fb93e..bf484a3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -233,5 +233,6 @@ static struct platform_driver qcom_iris_driver = {
};
module_platform_driver(qcom_iris_driver);
+MODULE_IMPORT_NS(DMA_BUF);
MODULE_DESCRIPTION("Qualcomm Iris video driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index b040d27..a57b5fb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -69,3 +69,150 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
return ret;
}
+
+void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
+ struct dma_buf *dbuf, unsigned long size)
+{
+ enum iris_buffer_type buf_type;
+ struct iris_buffers *buffers;
+ struct iris_buffer *iter;
+ struct iris_buffer *buf;
+ struct iris_inst *inst;
+ bool found = false;
+
+ if (!vb || !dev || !dbuf || !vb->vb2_queue)
+ return ERR_PTR(-EINVAL);
+
+ inst = vb->vb2_queue->drv_priv;
+
+ buf_type = v4l2_type_to_driver(vb->type);
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return NULL;
+
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->index == vb->index) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+
+ if (!found)
+ return NULL;
+
+ buf->inst = inst;
+ buf->dmabuf = dbuf;
+
+ buf->attach = dma_buf_attach(dbuf, dev);
+ if (IS_ERR(buf->attach)) {
+ buf->attach = NULL;
+ return NULL;
+ }
+
+ return buf;
+}
+
+int iris_vb2_map_dmabuf(void *buf_priv)
+{
+ struct iris_buffer *buf = buf_priv;
+ struct iris_core *core;
+ struct iris_inst *inst;
+
+ if (!buf || !buf->inst)
+ return -EINVAL;
+
+ inst = buf->inst;
+ core = inst->core;
+
+ if (!buf->attach) {
+ dev_err(core->dev, "trying to map a non attached buffer\n");
+ return -EINVAL;
+ }
+
+ buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL);
+ if (IS_ERR(buf->sg_table))
+ return -EINVAL;
+
+ if (!buf->sg_table->sgl) {
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ buf->sg_table = NULL;
+ return -EINVAL;
+ }
+
+ buf->device_addr = sg_dma_address(buf->sg_table->sgl);
+
+ return 0;
+}
+
+void iris_vb2_unmap_dmabuf(void *buf_priv)
+{
+ struct iris_buffer *buf = buf_priv;
+ struct iris_core *core;
+ struct iris_inst *inst;
+
+ if (!buf || !buf->inst)
+ return;
+
+ inst = buf->inst;
+ core = inst->core;
+
+ if (!buf->attach) {
+ dev_err(core->dev, "trying to unmap a non attached buffer\n");
+ return;
+ }
+
+ if (!buf->sg_table) {
+ dev_err(core->dev, "dmabuf buffer is already unmapped\n");
+ return;
+ }
+
+ if (buf->attach && buf->sg_table) {
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ buf->sg_table = NULL;
+ buf->device_addr = 0x0;
+ }
+}
+
+void iris_vb2_detach_dmabuf(void *buf_priv)
+{
+ struct iris_buffer *buf = buf_priv;
+ struct iris_core *core;
+ struct iris_inst *inst;
+
+ if (!buf || !buf->inst)
+ return;
+
+ inst = buf->inst;
+ core = inst->core;
+
+ if (buf->sg_table) {
+ dev_err(core->dev, "trying to detach an unmapped buffer\n");
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ buf->sg_table = NULL;
+ }
+
+ if (buf->attach && buf->dmabuf) {
+ dma_buf_detach(buf->dmabuf, buf->attach);
+ buf->attach = NULL;
+ }
+
+ buf->dmabuf = NULL;
+ buf->inst = NULL;
+}
+
+void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev,
+ unsigned long size)
+{
+ return (void *)0xdeadbeef;
+}
+
+void iris_vb2_put(void *buf_priv)
+{
+}
+
+int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma)
+{
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
index 8a8e1039..4342034 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -12,4 +12,14 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
unsigned int sizes[], struct device *alloc_devs[]);
+/* vb2_mem_ops */
+void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size);
+void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, struct dma_buf *dbuf,
+ unsigned long size);
+void iris_vb2_put(void *buf_priv);
+int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma);
+void iris_vb2_detach_dmabuf(void *buf_priv);
+int iris_vb2_map_dmabuf(void *buf_priv);
+void iris_vb2_unmap_dmabuf(void *buf_priv);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 410de720..124333a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -320,10 +320,21 @@ static const struct vb2_ops iris_vb2_ops = {
.queue_setup = iris_vb2_queue_setup,
};
+static struct vb2_mem_ops iris_vb2_mem_ops = {
+ .alloc = iris_vb2_alloc,
+ .put = iris_vb2_put,
+ .mmap = iris_vb2_mmap,
+ .attach_dmabuf = iris_vb2_attach_dmabuf,
+ .detach_dmabuf = iris_vb2_detach_dmabuf,
+ .map_dmabuf = iris_vb2_map_dmabuf,
+ .unmap_dmabuf = iris_vb2_unmap_dmabuf,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
core->vb2_ops = &iris_vb2_ops;
+ core->vb2_mem_ops = &iris_vb2_mem_ops;
return 0;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops
2023-12-18 11:32 ` [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops Dikshita Agarwal
@ 2023-12-19 11:58 ` Konrad Dybcio
0 siblings, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-19 11:58 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> From: Vikash Garodia <quic_vgarodia@quicinc.com>
>
> Implement the iris vb2 mem ops for buffer management for
> DMABUF streaming mode. Update video driver buffer
> with dma buf information.
>
> Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> +
> +void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev,
> + unsigned long size)
> +{
> + return (void *)0xdeadbeef;
> +}
> +
> +void iris_vb2_put(void *buf_priv)
> +{
> +}
> +
> +int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma)
> +{
> + return 0;
> +}
Are you sure these are enough for this driver to function?
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 19/34] media: iris: implement HFI to queue and release buffers
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (17 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation Dikshita Agarwal
` (21 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Introduce and implement HFIs to queue and release
all input, output and internal buffers to/from firmware.
HFI_CMD_BUFFER - to submit the buffer to firmware.
HFI_CMD_BUFFER with HFI_BUF_HOST_FLAG_RELEASE - to request
firmware to release the buffer.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 25 +++++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 85 ++++++++++++++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 78 ++++++++++++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 15 ++++
5 files changed, 207 insertions(+)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 9dd5f11..c044f78 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -13,6 +13,7 @@
#define HFI_CMD_INIT 0x01000001
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
+#define HFI_CMD_BUFFER 0x01000009
#define HFI_PROP_IMAGE_VERSION 0x03000001
@@ -60,4 +61,28 @@ struct hfi_debug_header {
u32 reserved[2];
};
+enum hfi_buffer_type {
+ HFI_BUFFER_BITSTREAM = 0x00000001,
+ HFI_BUFFER_RAW = 0x00000002,
+ HFI_BUFFER_METADATA = 0x00000003,
+ HFI_BUFFER_SUBCACHE = 0x00000004,
+ HFI_BUFFER_PARTIAL_DATA = 0x00000005,
+ HFI_BUFFER_DPB = 0x00000006,
+ HFI_BUFFER_BIN = 0x00000007,
+ HFI_BUFFER_LINE = 0x00000008,
+ HFI_BUFFER_ARP = 0x00000009,
+ HFI_BUFFER_COMV = 0x0000000A,
+ HFI_BUFFER_NON_COMV = 0x0000000B,
+ HFI_BUFFER_PERSIST = 0x0000000C,
+ HFI_BUFFER_VPSS = 0x0000000D,
+};
+
+enum hfi_buffer_host_flags {
+ HFI_BUF_HOST_FLAG_NONE = 0x00000000,
+ HFI_BUF_HOST_FLAG_RELEASE = 0x00000001,
+ HFI_BUF_HOST_FLAG_READONLY = 0x00000010,
+ HFI_BUF_HOST_FLAG_CODEC_CONFIG = 0x00000100,
+ HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200,
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 24ddb98..8f1e456 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -309,3 +309,88 @@ int iris_hfi_set_property(struct iris_inst *inst,
return ret;
}
+
+int iris_hfi_queue_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer)
+{
+ struct hfi_buffer hfi_buffer;
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = get_hfi_buffer(buffer, &hfi_buffer);
+ if (ret)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_BUFFER,
+ HFI_HOST_FLAGS_INTR_REQUIRED,
+ get_hfi_port_from_buffer_type(buffer->type),
+ inst->session_id,
+ HFI_PAYLOAD_STRUCTURE,
+ &hfi_buffer,
+ sizeof(hfi_buffer));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_release_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer)
+{
+ struct hfi_buffer hfi_buffer;
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet || !buffer)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = get_hfi_buffer(buffer, &hfi_buffer);
+ if (ret)
+ goto unlock;
+
+ hfi_buffer.flags |= HFI_BUF_HOST_FLAG_RELEASE;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_BUFFER,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port_from_buffer_type(buffer->type),
+ inst->session_id,
+ HFI_PAYLOAD_STRUCTURE,
+ &hfi_buffer,
+ sizeof(hfi_buffer));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index bf991bb..4aefdc4 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -19,5 +19,9 @@ int iris_hfi_set_property(struct iris_inst *inst,
irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
+int iris_hfi_queue_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
+int iris_hfi_release_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index 749d978..a3544d8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -4,9 +4,87 @@
*/
#include "iris_core.h"
+#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "hfi_defines.h"
+u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type)
+{
+ u32 hfi_port = HFI_PORT_NONE;
+
+ switch (buffer_type) {
+ case BUF_INPUT:
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case BUF_OUTPUT:
+ case BUF_DPB:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case BUF_PERSIST:
+ hfi_port = HFI_PORT_NONE;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_port;
+}
+
+static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return HFI_BUFFER_BITSTREAM;
+ case BUF_OUTPUT:
+ return HFI_BUFFER_RAW;
+ case BUF_BIN:
+ return HFI_BUFFER_BIN;
+ case BUF_COMV:
+ return HFI_BUFFER_COMV;
+ case BUF_NON_COMV:
+ return HFI_BUFFER_NON_COMV;
+ case BUF_LINE:
+ return HFI_BUFFER_LINE;
+ case BUF_DPB:
+ return HFI_BUFFER_DPB;
+ case BUF_PERSIST:
+ return HFI_BUFFER_PERSIST;
+ default:
+ return 0;
+ }
+}
+
+int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
+{
+ memset(buf, 0, sizeof(*buf));
+ buf->type = hfi_buf_type_from_driver(buffer->type);
+ buf->index = buffer->index;
+ buf->base_address = buffer->device_addr;
+ buf->addr_offset = 0;
+ buf->buffer_size = buffer->buffer_size;
+ /*
+ * for decoder input buffers, firmware (BSE HW) needs 256 aligned
+ * buffer size otherwise it will truncate or ignore the data after 256
+ * aligned size which may lead to error concealment
+ */
+ if (buffer->type == BUF_INPUT)
+ buf->buffer_size = ALIGN(buffer->buffer_size, 256);
+ buf->data_offset = buffer->data_offset;
+ buf->data_size = buffer->data_size;
+ if (buffer->attr & BUF_ATTR_READ_ONLY)
+ buf->flags |= HFI_BUF_HOST_FLAG_READONLY;
+ if (buffer->attr & BUF_ATTR_PENDING_RELEASE)
+ buf->flags |= HFI_BUF_HOST_FLAG_RELEASE;
+ buf->flags |= HFI_BUF_HOST_FLAGS_CB_NON_SECURE;
+ buf->timestamp = buffer->timestamp;
+
+ return 0;
+}
+
static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
u32 header_id)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index bea7ed9..bf18553 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -27,6 +27,19 @@ struct hfi_packet {
u32 reserved[2];
};
+struct hfi_buffer {
+ u32 type;
+ u32 index;
+ u64 base_address;
+ u32 addr_offset;
+ u32 buffer_size;
+ u32 data_offset;
+ u32 data_size;
+ u64 timestamp;
+ u32 flags;
+ u32 reserved[5];
+};
+
enum hfi_packet_host_flags {
HFI_HOST_FLAGS_NONE = 0x00000000,
HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001,
@@ -66,6 +79,8 @@ enum hfi_packet_port_type {
HFI_PORT_RAW = 0x00000002,
};
+u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
+int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (18 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 19/34] media: iris: implement HFI to queue and release buffers Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 12:06 ` Bryan O'Donoghue
2023-12-18 11:32 ` [PATCH v2 21/34] media: iris: implement internal buffer management Dikshita Agarwal
` (20 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Video driver needs various kind of internal buffers for
frame processing. Internal buffer size calculation depends
on hardware architecture, color format, resolution and codec.
Add APIs to calculate min count and size of different
internal buffers for different codecs for iris3.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++
.../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 +
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
.../platform/qcom/vcodec/iris/iris_instance.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 +
.../media/platform/qcom/vcodec/iris/vpu_common.h | 8 +
.../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 +
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++
12 files changed, 1132 insertions(+)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index a94e36b..7e3d9f1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -16,6 +16,7 @@ iris-objs += iris_probe.o \
resources.o \
vpu_common.o \
vpu_iris3.o \
+ vpu_iris3_buffer.o \
platform_common.o \
platform_sm8550.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index b9cffbf..6d4e722 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -17,6 +17,14 @@ static int output_min_count(struct iris_inst *inst)
{
int output_min_count;
+ /* fw_min_count > 0 indicates reconfig event has already arrived */
+ if (inst->fw_min_count) {
+ if (is_split_mode_enabled(inst) && inst->codec == VP9)
+ return min_t(u32, 4, inst->fw_min_count);
+ else
+ return inst->fw_min_count;
+ }
+
switch (inst->codec) {
case H264:
case HEVC:
@@ -33,6 +41,38 @@ static int output_min_count(struct iris_inst *inst)
return output_min_count;
}
+static u32 internal_buffer_count(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ u32 count = 0;
+
+ if (buffer_type == BUF_BIN || buffer_type == BUF_LINE ||
+ buffer_type == BUF_PERSIST) {
+ count = 1;
+ } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
+ if (inst->codec == H264 || inst->codec == HEVC)
+ count = 1;
+ else
+ count = 0;
+ } else {
+ count = 0;
+ }
+
+ return count;
+}
+
+static int dpb_count(struct iris_inst *inst)
+{
+ int count = 0;
+
+ if (is_split_mode_enabled(inst)) {
+ count = inst->fw_min_count ?
+ inst->fw_min_count : inst->buffers.output.min_count;
+ }
+
+ return count;
+}
+
int iris_get_buf_min_count(struct iris_inst *inst,
enum iris_buffer_type buffer_type)
{
@@ -41,6 +81,14 @@ int iris_get_buf_min_count(struct iris_inst *inst,
return input_min_count(inst);
case BUF_OUTPUT:
return output_min_count(inst);
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ case BUF_PERSIST:
+ return internal_buffer_count(inst, buffer_type);
+ case BUF_DPB:
+ return dpb_count(inst);
default:
return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 4edadc3..a83d1c1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -15,6 +15,7 @@
#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
#define DEFAULT_WIDTH 320
#define DEFAULT_HEIGHT 240
+#define DEFAULT_BSE_VPP_DELAY 2
#define MB_IN_PIXEL (16 * 16)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 30f7ad7..154991c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -50,6 +50,7 @@
* @header_id: id of packet header
* @packet_id: id of packet
* @vpu_ops: a pointer to vpu ops
+ * @session_ops: a pointer to session level ops
* @dec_codecs_count: supported codec count for decoder
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
@@ -91,6 +92,7 @@ struct iris_core {
u32 header_id;
u32 packet_id;
const struct vpu_ops *vpu_ops;
+ const struct vpu_session_ops *session_ops;
u32 dec_codecs_count;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 54a7851..335885f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -66,6 +66,19 @@ int get_mbpf(struct iris_inst *inst)
return NUM_MBS_PER_FRAME(height, width);
}
+bool is_linear_colorformat(u32 colorformat)
+{
+ return colorformat == V4L2_PIX_FMT_NV12 || colorformat == V4L2_PIX_FMT_NV21;
+}
+
+bool is_split_mode_enabled(struct iris_inst *inst)
+{
+ if (is_linear_colorformat(inst->fmt_dst->fmt.pix_mp.pixelformat))
+ return true;
+
+ return false;
+}
+
static int process_inst_timeout(struct iris_inst *inst)
{
struct iris_inst *instance;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 3bae969..9e85510 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -12,6 +12,7 @@
#include "iris_core.h"
#include "iris_instance.h"
#include "iris_buffer.h"
+#include "iris_instance.h"
#include "platform_common.h"
#define NUM_MBS_PER_FRAME(__height, __width) \
@@ -26,4 +27,7 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type);
int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);
+bool is_linear_colorformat(u32 colorformat);
+bool is_split_mode_enabled(struct iris_inst *inst);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index a5c6cb48..5d4c856 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -36,6 +36,7 @@
* @caps_list: list head of capability
* @codec: codec type
* @buffers: structure of buffer info
+ * @fw_min_count: minimnum count of buffers needed by fw
*/
struct iris_inst {
@@ -58,6 +59,7 @@ struct iris_inst {
struct list_head caps_list;
enum codec_type codec;
struct iris_buffers_info buffers;
+ u32 fw_min_count;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 1b957a13..b131a50 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -39,6 +39,7 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->fw_min_count = 0;
}
void vdec_inst_deinit(struct iris_inst *inst)
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
index 6512039..7fba8c2 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
@@ -26,6 +26,14 @@ struct vpu_ops {
int (*watchdog)(struct iris_core *core, u32 intr_status);
};
+#define call_session_op(c, op, ...) \
+ (((c) && (c)->session_ops && (c)->session_ops->op) ? \
+ ((c)->session_ops->op(__VA_ARGS__)) : 0)
+
+struct vpu_session_ops {
+ int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type);
+};
+
int init_vpu(struct iris_core *core);
int write_register(struct iris_core *core, u32 reg, u32 value);
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
index a34d0ed..efea5aa 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
@@ -6,6 +6,7 @@
#include <linux/delay.h>
#include "vpu_iris3.h"
+#include "vpu_iris3_buffer.h"
#define VIDEO_ARCH_LX 1
@@ -202,9 +203,14 @@ static const struct vpu_ops iris3_ops = {
.watchdog = watchdog_iris3,
};
+static const struct vpu_session_ops iris3_session_ops = {
+ .int_buf_size = iris_int_buf_size_iris3,
+};
+
int init_iris3(struct iris_core *core)
{
core->vpu_ops = &iris3_ops;
+ core->session_ops = &iris3_session_ops;
return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
new file mode 100644
index 0000000..44f9342
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
@@ -0,0 +1,201 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_helpers.h"
+#include "iris_instance.h"
+#include "vpu_iris3_buffer.h"
+
+static u32 dec_bin_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_bin_h264d(width, height, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_bin_h265d(width, height, num_vpp_pipes);
+ else if (inst->codec == VP9)
+ size = hfi_buffer_bin_vp9d(width, height,
+ num_vpp_pipes);
+ return size;
+}
+
+static u32 dec_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_comv;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ num_comv = inst->buffers.output.min_count;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_comv_h264d(width, height, num_comv);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_comv_h265d(width, height, num_comv);
+
+ inst->cap[NUM_COMV].value = num_comv;
+
+ return size;
+}
+
+static u32 dec_non_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_non_comv_h264d(width, height, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_non_comv_h265d(width, height, num_vpp_pipes);
+
+ return size;
+}
+
+static u32 dec_line_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, out_min_count, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+ bool is_opb;
+
+ core = inst->core;
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+
+ is_opb = true;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ out_min_count = inst->buffers.output.min_count;
+ if (inst->codec == H264)
+ size = hfi_buffer_line_h264d(width, height, is_opb,
+ num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_line_h265d(width, height, is_opb,
+ num_vpp_pipes);
+ else if (inst->codec == VP9)
+ size = hfi_buffer_line_vp9d(width, height, out_min_count,
+ is_opb, num_vpp_pipes);
+ return size;
+}
+
+static u32 dec_persist_size_iris3(struct iris_inst *inst)
+{
+ u32 size = 0;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_persist_h264d(0);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_persist_h265d(0);
+ else if (inst->codec == VP9)
+ size = hfi_buffer_persist_vp9d();
+
+ return size;
+}
+
+static u32 dec_dpb_size_iris3(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ u32 width, height;
+ u32 color_fmt;
+ u32 size = 0;
+
+ f = inst->fmt_dst;
+ color_fmt = f->fmt.pix_mp.pixelformat;
+ if (!is_linear_colorformat(color_fmt))
+ return size;
+
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (color_fmt == V4L2_PIX_FMT_NV12 ||
+ color_fmt == V4L2_PIX_FMT_QC08C) {
+ size =
+ hfi_nv12_ubwc_il_calc_buf_size_v2(width, height,
+ ALIGN(width, 128),
+ ALIGN(height, 32),
+ ALIGN(width, 128),
+ ALIGN((height + 1) >> 1, 32),
+ ALIGN(DIV_ROUND_UP(width, 32), 64),
+ ALIGN(DIV_ROUND_UP(height, 8), 16),
+ ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64),
+ ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16));
+ } else if (color_fmt == V4L2_PIX_FMT_QC10C) {
+ size =
+ hfi_yuv420_tp10_ubwc_calc_buf_size(ALIGN(ALIGN(width, 192) * 4 / 3, 256),
+ ALIGN(height, 16),
+ ALIGN(ALIGN(width, 192) * 4 / 3, 256),
+ ALIGN((height + 1) >> 1, 16),
+ ALIGN(DIV_ROUND_UP(width, 48), 64),
+ ALIGN(DIV_ROUND_UP(height, 4), 16),
+ ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64),
+ ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16));
+ }
+
+ return size;
+}
+
+struct iris_buf_type_handle {
+ enum iris_buffer_type type;
+ u32 (*handle)(struct iris_inst *inst);
+};
+
+int iris_int_buf_size_iris3(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ const struct iris_buf_type_handle *buf_type_handle_arr = NULL;
+ u32 size = 0, buf_type_handle_size = 0;
+ int i;
+
+ static const struct iris_buf_type_handle dec_internal_buf_type_handle[] = {
+ {BUF_BIN, dec_bin_size_iris3 },
+ {BUF_COMV, dec_comv_size_iris3 },
+ {BUF_NON_COMV, dec_non_comv_size_iris3 },
+ {BUF_LINE, dec_line_size_iris3 },
+ {BUF_PERSIST, dec_persist_size_iris3 },
+ {BUF_DPB, dec_dpb_size_iris3 },
+ };
+
+ buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle);
+ buf_type_handle_arr = dec_internal_buf_type_handle;
+
+ if (!buf_type_handle_arr || !buf_type_handle_size)
+ return size;
+
+ for (i = 0; i < buf_type_handle_size; i++) {
+ if (buf_type_handle_arr[i].type == buffer_type) {
+ size = buf_type_handle_arr[i].handle(inst);
+ break;
+ }
+ }
+
+ return size;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
new file mode 100644
index 0000000..b520c79
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
@@ -0,0 +1,845 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _VPU_IRIS3_BUFFER_H_
+#define _VPU_IRIS3_BUFFER_H_
+
+#include <linux/types.h>
+#include <linux/minmax.h>
+#include <linux/align.h>
+
+#include "iris_instance.h"
+
+#define DMA_ALIGNMENT 256
+
+#define BUFFER_ALIGNMENT_512_BYTES 512
+#define BUFFER_ALIGNMENT_256_BYTES 256
+#define BUFFER_ALIGNMENT_64_BYTES 64
+#define BUFFER_ALIGNMENT_32_BYTES 32
+#define BUFFER_ALIGNMENT_16_BYTES 16
+
+#define HFI_ALIGNMENT_4096 (4096)
+
+#define HFI_COL_FMT_NV12C_Y_TILE_HEIGHT (8)
+#define HFI_COL_FMT_NV12C_Y_TILE_WIDTH (32)
+#define HFI_COL_FMT_NV12C_UV_TILE_HEIGHT (8)
+#define HFI_COL_FMT_NV12C_UV_TILE_WIDTH (16)
+
+#define NUM_HW_PIC_BUF 32
+#define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf))
+
+#define MAX_TILE_COLUMNS 32
+
+#define LCU_MAX_SIZE_PELS 64
+#define LCU_MIN_SIZE_PELS 16
+
+#define HDR10_HIST_EXTRADATA_SIZE (4 * 1024)
+
+#define BIN_BUFFER_THRESHOLD (1280 * 736)
+
+#define VPP_CMD_MAX_SIZE (BIT(20))
+
+#define H264D_MAX_SLICE 1800
+
+#define SIZE_H264D_BUFTAB_T (256)
+#define SIZE_H264D_HW_PIC_T (BIT(11))
+#define SIZE_H264D_BSE_CMD_PER_BUF (32 * 4)
+#define SIZE_H264D_VPP_CMD_PER_BUF (512)
+
+#define NUM_SLIST_BUF_H264 (256 + 32)
+#define SIZE_SLIST_BUF_H264 (512)
+#define H264_DISPLAY_BUF_SIZE (3328)
+#define H264_NUM_FRM_INFO (66)
+
+#define H265_NUM_TILE_COL 32
+#define H265_NUM_TILE_ROW 128
+#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
+#define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32))
+
+#define NUM_SLIST_BUF_H265 (80 + 20)
+#define SIZE_SLIST_BUF_H265 (BIT(10))
+#define H265_DISPLAY_BUF_SIZE (3072)
+#define H265_NUM_FRM_INFO (48)
+
+#define VP9_NUM_FRAME_INFO_BUF 32
+#define VP9_NUM_PROBABILITY_TABLE_BUF (VP9_NUM_FRAME_INFO_BUF + 4)
+#define VP9_PROB_TABLE_SIZE (3840)
+#define VP9_FRAME_INFO_BUF_SIZE (6144)
+
+#define VP9_UDC_HEADER_BUF_SIZE (3 * 128)
+#define MAX_SUPERFRAME_HEADER_LEN (34)
+#define CCE_TILE_OFFSET_SIZE ALIGN(32 * 4 * 4, BUFFER_ALIGNMENT_32_BYTES)
+
+#define SIZE_SEI_USERDATA (4096)
+#define SIZE_DOLBY_RPU_METADATA (41 * 1024)
+
+#define H264_CABAC_HDR_RATIO_HD_TOT 1
+#define H264_CABAC_RES_RATIO_HD_TOT 3
+
+#define H265D_MAX_SLICE 1200
+#define SIZE_H265D_HW_PIC_T SIZE_H264D_HW_PIC_T
+#define H265_CABAC_HDR_RATIO_HD_TOT 2
+#define H265_CABAC_RES_RATIO_HD_TOT 2
+#define SIZE_H265D_VPP_CMD_PER_BUF (256)
+
+#define VPX_DECODER_FRAME_CONCURENCY_LVL (2)
+#define VPX_DECODER_FRAME_BIN_HDR_BUDGET 1
+#define VPX_DECODER_FRAME_BIN_RES_BUDGET 3
+#define VPX_DECODER_FRAME_BIN_DENOMINATOR 2
+
+#define VPX_DECODER_FRAME_BIN_RES_BUDGET_RATIO (3 / 2)
+
+#define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 64
+#define MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE 64
+#define MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE 64
+
+#define MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE (128 / 8)
+#define MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE (128 / 8)
+#define MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE (128 / 8)
+
+#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE (64 * 2 * 3)
+#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640
+
+static inline
+u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 size_yuv, size_bin_hdr, size_bin_res;
+
+ size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ?
+ ((BIN_BUFFER_THRESHOLD * 3) >> 1) :
+ ((frame_width * frame_height * 3) >> 1);
+ size_bin_hdr = size_yuv * H264_CABAC_HDR_RATIO_HD_TOT;
+ size_bin_res = size_yuv * H264_CABAC_RES_RATIO_HD_TOT;
+ size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+ size_bin_res = ALIGN(size_bin_res / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+
+ return size_bin_hdr + size_bin_res;
+}
+
+static inline
+u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 n_aligned_w, n_aligned_h;
+
+ n_aligned_w =
+ ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES);
+ n_aligned_h =
+ ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES);
+
+ return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h,
+ num_vpp_pipes);
+}
+
+static inline
+u32 size_h265d_hw_bin_buffer(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 size_yuv, size_bin_hdr, size_bin_res;
+
+ size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ?
+ ((BIN_BUFFER_THRESHOLD * 3) >> 1) :
+ ((frame_width * frame_height * 3) >> 1);
+ size_bin_hdr = size_yuv * H265_CABAC_HDR_RATIO_HD_TOT;
+ size_bin_res = size_yuv * H265_CABAC_RES_RATIO_HD_TOT;
+ size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, DMA_ALIGNMENT) *
+ num_vpp_pipes;
+ size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, DMA_ALIGNMENT) *
+ num_vpp_pipes;
+
+ return size_bin_hdr + size_bin_res;
+}
+
+static inline
+u32 hfi_buffer_bin_h265d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 n_aligned_w, n_aligned_h;
+
+ n_aligned_w = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES);
+ n_aligned_h = ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES);
+ return size_h265d_hw_bin_buffer(n_aligned_w, n_aligned_h,
+ num_vpp_pipes);
+}
+
+static inline
+u32 hfi_buffer_bin_vp9d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 _size_yuv, _size;
+
+ _size_yuv = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES) *
+ ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES) * 3 / 2;
+ _size = ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) *
+ VPX_DECODER_FRAME_BIN_HDR_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR *
+ VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes),
+ DMA_ALIGNMENT) +
+ ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) *
+ VPX_DECODER_FRAME_BIN_RES_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR *
+ VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes),
+ DMA_ALIGNMENT);
+
+ return _size * num_vpp_pipes;
+}
+
+static inline
+u32 hfi_buffer_comv_h264d(u32 frame_width, u32 frame_height,
+ u32 _comv_bufcount)
+{
+ u32 frame_width_in_mbs = ((frame_width + 15) >> 4);
+ u32 frame_height_in_mbs = ((frame_height + 15) >> 4);
+ u32 col_mv_aligned_width = (frame_width_in_mbs << 7);
+ u32 col_zero_aligned_width = (frame_width_in_mbs << 2);
+ u32 col_zero_size = 0, size_colloc = 0;
+
+ col_mv_aligned_width =
+ ALIGN(col_mv_aligned_width, BUFFER_ALIGNMENT_16_BYTES);
+ col_zero_aligned_width =
+ ALIGN(col_zero_aligned_width, BUFFER_ALIGNMENT_16_BYTES);
+ col_zero_size = col_zero_aligned_width *
+ ((frame_height_in_mbs + 1) >> 1);
+ col_zero_size =
+ ALIGN(col_zero_size, BUFFER_ALIGNMENT_64_BYTES);
+ col_zero_size <<= 1;
+ col_zero_size =
+ ALIGN(col_zero_size, BUFFER_ALIGNMENT_512_BYTES);
+ size_colloc = col_mv_aligned_width *
+ ((frame_height_in_mbs + 1) >> 1);
+ size_colloc =
+ ALIGN(size_colloc, BUFFER_ALIGNMENT_64_BYTES);
+ size_colloc <<= 1;
+ size_colloc =
+ ALIGN(size_colloc, BUFFER_ALIGNMENT_512_BYTES);
+ size_colloc += (col_zero_size + SIZE_H264D_BUFTAB_T * 2);
+
+ return (size_colloc * (_comv_bufcount)) +
+ BUFFER_ALIGNMENT_512_BYTES;
+}
+
+static inline
+u32 hfi_buffer_comv_h265d(u32 frame_width, u32 frame_height,
+ u32 _comv_bufcount)
+{
+ u32 _size;
+
+ _size = ALIGN(((((frame_width + 15) >> 4) *
+ ((frame_height + 15) >> 4)) << 8),
+ BUFFER_ALIGNMENT_512_BYTES);
+ _size *= _comv_bufcount;
+ _size += BUFFER_ALIGNMENT_512_BYTES;
+
+ return _size;
+}
+
+static inline
+u32 size_h264d_bse_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _height = ALIGN(frame_height,
+ BUFFER_ALIGNMENT_32_BYTES);
+ return min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) *
+ SIZE_H264D_BSE_CMD_PER_BUF;
+}
+
+static inline
+u32 size_h264d_vpp_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _size, _height;
+
+ _height = ALIGN(frame_height, BUFFER_ALIGNMENT_32_BYTES);
+ _size = min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) *
+ SIZE_H264D_VPP_CMD_PER_BUF;
+
+ if (_size > VPP_CMD_MAX_SIZE)
+ _size = VPP_CMD_MAX_SIZE;
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 _size_bse, _size_vpp, _size;
+
+ _size_bse = size_h264d_bse_cmd_buf(frame_width, frame_height);
+ _size_vpp = size_h264d_vpp_cmd_buf(frame_width, frame_height);
+ _size = ALIGN(_size_bse, DMA_ALIGNMENT) +
+ ALIGN(_size_vpp, DMA_ALIGNMENT) +
+ ALIGN(SIZE_HW_PIC(SIZE_H264D_HW_PIC_T), DMA_ALIGNMENT);
+
+ return ALIGN(_size, DMA_ALIGNMENT);
+}
+
+static inline
+u32 size_h265d_bse_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _size;
+
+ _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) *
+ NUM_HW_PIC_BUF, DMA_ALIGNMENT);
+ _size = min_t(u32, _size, H265D_MAX_SLICE + 1);
+ _size = 2 * _size * SIZE_H265D_BSE_CMD_PER_BUF;
+
+ return _size;
+}
+
+static inline
+u32 size_h265d_vpp_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _size;
+
+ _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) *
+ NUM_HW_PIC_BUF, DMA_ALIGNMENT);
+ _size = min_t(u32, _size, H265D_MAX_SLICE + 1);
+ _size = ALIGN(_size, 4);
+ _size = 2 * _size * SIZE_H265D_VPP_CMD_PER_BUF;
+ if (_size > VPP_CMD_MAX_SIZE)
+ _size = VPP_CMD_MAX_SIZE;
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_non_comv_h265d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 _size_bse, _size_vpp, _size;
+
+ _size_bse = size_h265d_bse_cmd_buf(frame_width, frame_height);
+ _size_vpp = size_h265d_vpp_cmd_buf(frame_width, frame_height);
+ _size = ALIGN(_size_bse, DMA_ALIGNMENT) +
+ ALIGN(_size_vpp, DMA_ALIGNMENT) +
+ ALIGN(NUM_HW_PIC_BUF * 20 * 22 * 4, DMA_ALIGNMENT) +
+ ALIGN(2 * sizeof(u16) *
+ (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS),
+ DMA_ALIGNMENT) +
+ ALIGN(SIZE_HW_PIC(SIZE_H265D_HW_PIC_T),
+ DMA_ALIGNMENT) +
+ HDR10_HIST_EXTRADATA_SIZE;
+
+ return ALIGN(_size, DMA_ALIGNMENT);
+}
+
+static inline u32 size_h264d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * ALIGN(frame_width, 16) * 3;
+}
+
+static inline u32 size_h264d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return (((frame_width + 15) >> 4) << 7);
+}
+
+static inline u32 size_h264d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+ return ALIGN(frame_height, 16) * 32;
+}
+
+static inline u32 size_h264d_qp(u32 frame_width, u32 frame_height)
+{
+ return ((frame_width + 63) >> 6) * ((frame_height + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_vpss_lb(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+ u32 vpss_4tap_left_buffer_size = 0, vpss_div2_left_buffer_size = 0;
+ u32 vpss_4tap_top_buffer_size = 0, vpss_div2_top_buffer_size = 0;
+ u32 opb_lb_wr_llb_y_buffer_size, opb_lb_wr_llb_uv_buffer_size;
+ u32 opb_wr_top_line_chroma_buffer_size;
+ u32 opb_wr_top_line_luma_buffer_size;
+ u32 macrotiling_size = 32, size;
+
+ opb_wr_top_line_luma_buffer_size =
+ ALIGN(frame_width, macrotiling_size) /
+ macrotiling_size * 256;
+ opb_wr_top_line_luma_buffer_size =
+ ALIGN(opb_wr_top_line_luma_buffer_size, DMA_ALIGNMENT) +
+ (MAX_TILE_COLUMNS - 1) * 256;
+ opb_wr_top_line_luma_buffer_size =
+ max_t(u32, opb_wr_top_line_luma_buffer_size,
+ (32 * ALIGN(frame_height, 8)));
+ opb_wr_top_line_chroma_buffer_size =
+ opb_wr_top_line_luma_buffer_size;
+ opb_lb_wr_llb_uv_buffer_size =
+ ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64,
+ BUFFER_ALIGNMENT_32_BYTES);
+ opb_lb_wr_llb_y_buffer_size =
+ ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64,
+ BUFFER_ALIGNMENT_32_BYTES);
+ size = num_vpp_pipes * 2 *
+ (vpss_4tap_top_buffer_size + vpss_div2_top_buffer_size) +
+ 2 * (vpss_4tap_left_buffer_size + vpss_div2_left_buffer_size) +
+ opb_wr_top_line_luma_buffer_size +
+ opb_wr_top_line_chroma_buffer_size +
+ opb_lb_wr_llb_uv_buffer_size +
+ opb_lb_wr_llb_y_buffer_size;
+
+ return size;
+}
+
+static inline
+u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height,
+ bool is_opb, u32 num_vpp_pipes)
+{
+ u32 vpss_lb_size = 0;
+ u32 _size;
+
+ _size = ALIGN(size_h264d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h264d_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h264d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height),
+ DMA_ALIGNMENT) * 2 +
+ ALIGN(size_h264d_qp(frame_width, frame_height),
+ DMA_ALIGNMENT);
+ _size = ALIGN(_size, DMA_ALIGNMENT);
+ if (is_opb)
+ vpss_lb_size = size_vpss_lb(frame_width, frame_height,
+ num_vpp_pipes);
+
+ _size = ALIGN((_size + vpss_lb_size),
+ DMA_ALIGNMENT);
+
+ return _size;
+}
+
+static inline
+u32 size_h265d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE *
+ (ALIGN(frame_width, 64) + 8) * 2;
+}
+
+static inline
+u32 size_h265d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE *
+ (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (LCU_MAX_SIZE_PELS / 8 * (128 / 8)) * ((frame_width + 15) >> 4);
+}
+
+static inline
+u32 size_h265d_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return max_t(u32, ((frame_height + 16 - 1) / 8) *
+ MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max_t(u32, ((frame_height + 32 - 1) / 8) *
+ MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 64 - 1) / 8) *
+ MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_h265d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE *
+ (ALIGN(frame_width, LCU_MIN_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return ((frame_width + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_h265d_lb_vsp_left(u32 frame_width, u32 frame_height)
+{
+ return ((frame_height + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_h265d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height);
+}
+
+static inline
+u32 size_h265d_qp(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_qp(frame_width, frame_height);
+}
+
+static inline
+u32 hfi_buffer_line_h265d(u32 frame_width, u32 frame_height,
+ bool is_opb, u32 num_vpp_pipes)
+{
+ u32 vpss_lb_size = 0, _size;
+
+ _size = ALIGN(size_h265d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h265d_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h265d_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_vsp_left(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h265d_lb_recon_dma_metadata_wr(frame_width, frame_height),
+ DMA_ALIGNMENT) * 4 +
+ ALIGN(size_h265d_qp(frame_width, frame_height),
+ DMA_ALIGNMENT);
+ if (is_opb)
+ vpss_lb_size = size_vpss_lb(frame_width, frame_height, num_vpp_pipes);
+
+ return ALIGN((_size + vpss_lb_size), DMA_ALIGNMENT);
+}
+
+static inline
+u32 size_vpxd_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return max_t(u32, ((frame_height + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max_t(u32, ((frame_height + 31) >> 5) * MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 63) >> 6) * MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_vpxd_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return ((ALIGN(frame_width, 64) + 8) * 10 * 2);
+}
+
+static inline
+u32 size_vpxd_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return ((frame_width + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE;
+}
+
+static inline
+u32 size_vpxd_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return max_t(u32, ((frame_height + 15) >> 4) * MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max_t(u32, ((frame_height + 31) >> 5) * MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 63) >> 6) * MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_vpxd_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+ return ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64,
+ BUFFER_ALIGNMENT_32_BYTES);
+}
+
+static inline
+u32 size_mp2d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return ((ALIGN(frame_width, 16) + 8) * 10 * 2);
+}
+
+static inline
+u32 size_vp9d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return (ALIGN(ALIGN(frame_width, 8), 64) + 8) * 10 * 2;
+}
+
+static inline
+u32 size_vp9d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return ((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 176);
+}
+
+static inline
+u32 size_vp9d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return (((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 64 * 8) + 256);
+}
+
+static inline u32 hfi_iris3_vp9d_comv_size(void)
+{
+ return (((8192 + 63) >> 6) * ((4320 + 63) >> 6) * 8 * 8 * 2 * 8);
+}
+
+static inline
+u32 size_vp9d_qp(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_qp(frame_width, frame_height);
+}
+
+static inline
+u32 hfi_iris3_vp9d_lb_size(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ return ALIGN(size_vpxd_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_vpxd_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_vp9d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vpxd_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ 2 * ALIGN(size_vpxd_lb_recon_dma_metadata_wr(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vpxd_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vp9d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vp9d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vp9d_qp(frame_width, frame_height),
+ DMA_ALIGNMENT);
+}
+
+static inline
+u32 hfi_buffer_line_vp9d(u32 frame_width, u32 frame_height,
+ u32 _yuv_bufcount_min, bool is_opb,
+ u32 num_vpp_pipes)
+{
+ u32 vpss_lb_size = 0;
+ u32 _lb_size = 0;
+
+ _lb_size = hfi_iris3_vp9d_lb_size(frame_width, frame_height,
+ num_vpp_pipes);
+
+ if (is_opb)
+ vpss_lb_size = size_vpss_lb(frame_width, frame_height,
+ num_vpp_pipes);
+
+ return _lb_size + vpss_lb_size;
+}
+
+static inline u32 hfi_buffer_persist_h264d(u32 rpu_enabled)
+{
+ return ALIGN(SIZE_SLIST_BUF_H264 * NUM_SLIST_BUF_H264 +
+ H264_DISPLAY_BUF_SIZE * H264_NUM_FRM_INFO +
+ NUM_HW_PIC_BUF * SIZE_SEI_USERDATA +
+ (rpu_enabled) * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA,
+ DMA_ALIGNMENT);
+}
+
+static inline
+u32 hfi_buffer_persist_h265d(u32 rpu_enabled)
+{
+ return ALIGN((SIZE_SLIST_BUF_H265 * NUM_SLIST_BUF_H265 +
+ H265_NUM_FRM_INFO * H265_DISPLAY_BUF_SIZE +
+ H265_NUM_TILE * sizeof(u32) +
+ NUM_HW_PIC_BUF * SIZE_SEI_USERDATA +
+ rpu_enabled * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA),
+ DMA_ALIGNMENT);
+}
+
+static inline u32 hfi_buffer_persist_vp9d(void)
+{
+ return ALIGN(VP9_NUM_PROBABILITY_TABLE_BUF * VP9_PROB_TABLE_SIZE,
+ DMA_ALIGNMENT) +
+ ALIGN(hfi_iris3_vp9d_comv_size(), DMA_ALIGNMENT) +
+ ALIGN(MAX_SUPERFRAME_HEADER_LEN, DMA_ALIGNMENT) +
+ ALIGN(VP9_UDC_HEADER_BUF_SIZE, DMA_ALIGNMENT) +
+ ALIGN(VP9_NUM_FRAME_INFO_BUF * CCE_TILE_OFFSET_SIZE,
+ DMA_ALIGNMENT) +
+ ALIGN(VP9_NUM_FRAME_INFO_BUF * VP9_FRAME_INFO_BUF_SIZE,
+ DMA_ALIGNMENT) +
+ HDR10_HIST_EXTRADATA_SIZE;
+}
+
+static inline
+u32 hfi_nv12_ubwc_il_calc_y_buf_size(u32 frame_width, u32 frame_height,
+ u32 stride_multiple,
+ u32 min_buf_height_multiple)
+{
+ u32 stride, buf_height;
+
+ stride = ALIGN(frame_width, stride_multiple);
+ buf_height = ALIGN(frame_height, min_buf_height_multiple);
+
+ return ALIGN(stride * buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_nv12_ubwc_il_calc_uv_buf_size(u32 frame_width, u32 frame_height,
+ u32 stride_multiple,
+ u32 min_buf_height_multiple)
+{
+ u32 uv_stride, uv_buf_height;
+
+ uv_stride = ALIGN(frame_width, stride_multiple);
+ uv_buf_height = ALIGN(((frame_height + 1) >> 1),
+ min_buf_height_multiple);
+
+ return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_ubwc_calc_metadata_plane_stride(u32 frame_width,
+ u32 metadata_stride_multiple,
+ u32 tile_width_in_pels)
+{
+ return ALIGN(((frame_width + (tile_width_in_pels - 1)) / tile_width_in_pels),
+ metadata_stride_multiple);
+}
+
+static inline
+u32 hfi_ubwc_metadata_plane_bufheight(u32 frame_height,
+ u32 metadata_height_multiple,
+ u32 tile_height_in_pels)
+{
+ return ALIGN(((frame_height + (tile_height_in_pels - 1)) / tile_height_in_pels),
+ metadata_height_multiple);
+}
+
+static inline
+u32 hfi_ubwc_uv_metadata_plane_stride(u32 frame_width,
+ u32 metadata_stride_multiple,
+ u32 tile_width_in_pels)
+{
+ return ALIGN(((((frame_width + 1) >> 1) + tile_width_in_pels - 1) /
+ tile_width_in_pels), metadata_stride_multiple);
+}
+
+static inline
+u32 hfi_ubwc_uv_metadata_plane_bufheight(u32 frame_height,
+ u32 metadata_height_multiple,
+ u32 tile_height_in_pels)
+{
+ return ALIGN(((((frame_height + 1) >> 1) + tile_height_in_pels - 1) /
+ tile_height_in_pels), metadata_height_multiple);
+}
+
+static inline
+u32 hfi_ubwc_metadata_plane_buffer_size(u32 _metadata_tride, u32 _metadata_buf_height)
+{
+ return ALIGN(_metadata_tride * _metadata_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_nv12_ubwc_il_calc_buf_size_v2(u32 frame_width,
+ u32 frame_height,
+ u32 y_stride_multiple,
+ u32 y_buffer_height_multiple,
+ u32 uv_stride_multiple,
+ u32 uv_buffer_height_multiple,
+ u32 y_metadata_stride_multiple,
+ u32 y_metadata_buffer_height_multiple,
+ u32 uv_metadata_stride_multiple,
+ u32 uv_metadata_buffer_height_multiple)
+{
+ u32 y_buf_size, uv_buf_size, y_meta_size, uv_meta_size;
+ u32 half_height = (frame_height + 1) >> 1;
+ u32 stride, _height;
+
+ y_buf_size =
+ hfi_nv12_ubwc_il_calc_y_buf_size(frame_width, half_height,
+ y_stride_multiple,
+ y_buffer_height_multiple);
+ uv_buf_size =
+ hfi_nv12_ubwc_il_calc_uv_buf_size(frame_width, half_height,
+ uv_stride_multiple,
+ uv_buffer_height_multiple);
+ stride =
+ hfi_ubwc_calc_metadata_plane_stride(frame_width,
+ y_metadata_stride_multiple,
+ HFI_COL_FMT_NV12C_Y_TILE_WIDTH);
+ _height =
+ hfi_ubwc_metadata_plane_bufheight(half_height,
+ y_metadata_buffer_height_multiple,
+ HFI_COL_FMT_NV12C_Y_TILE_HEIGHT);
+ y_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height);
+ stride =
+ hfi_ubwc_uv_metadata_plane_stride(frame_width,
+ uv_metadata_stride_multiple,
+ HFI_COL_FMT_NV12C_UV_TILE_WIDTH);
+ _height =
+ hfi_ubwc_uv_metadata_plane_bufheight(half_height,
+ uv_metadata_buffer_height_multiple,
+ HFI_COL_FMT_NV12C_UV_TILE_HEIGHT);
+ uv_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height);
+
+ return (y_buf_size + uv_buf_size + y_meta_size + uv_meta_size) << 1;
+}
+
+static inline
+u32 hfi_yuv420_tp10_ubwc_calc_y_buf_size(u32 y_stride,
+ u32 y_buf_height)
+{
+ return ALIGN(y_stride * y_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_yuv420_tp10_ubwc_calc_uv_buf_size(u32 uv_stride,
+ u32 uv_buf_height)
+{
+ return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_yuv420_tp10_ubwc_calc_buf_size(u32 y_stride, u32 y_buf_height,
+ u32 uv_stride, u32 uv_buf_height,
+ u32 y_md_stride, u32 y_md_height,
+ u32 uv_md_stride, u32 uv_md_height)
+{
+ u32 y_data_size, uv_data_size, y_md_size, uv_md_size;
+
+ y_data_size = hfi_yuv420_tp10_ubwc_calc_y_buf_size(y_stride, y_buf_height);
+ uv_data_size = hfi_yuv420_tp10_ubwc_calc_uv_buf_size(uv_stride, uv_buf_height);
+ y_md_size = hfi_ubwc_metadata_plane_buffer_size(y_md_stride, y_md_height);
+ uv_md_size = hfi_ubwc_metadata_plane_buffer_size(uv_md_stride, uv_md_height);
+
+ return y_data_size + uv_data_size + y_md_size + uv_md_size;
+}
+
+int iris_int_buf_size_iris3(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+
+#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation
2023-12-18 11:32 ` [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation Dikshita Agarwal
@ 2023-12-19 12:06 ` Bryan O'Donoghue
2023-12-20 8:29 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Bryan O'Donoghue @ 2023-12-19 12:06 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 11:32, Dikshita Agarwal wrote:
> drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++
> .../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
> .../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 2 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 +
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 8 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++
So, unless the code you are adding effects existing upstream venus, I
think it would be a bit easier to review if you squashed down changes
that pertain to Iris only.
For example this patch seems to relate to Iris only, so why is it a
progressive change within your series.
Similar comment for "add vb2 streaming and buffer ops" and other "add
feature x" patches in this series.
If the change is contained to your own codebase, then progressive
changes are more noise than content.
Please try to squash down changes - to reduce the number of patches and
the total LOC being proposed here.
---
bod
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation
2023-12-19 12:06 ` Bryan O'Donoghue
@ 2023-12-20 8:29 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:29 UTC (permalink / raw)
To: Bryan O'Donoghue, linux-media, linux-kernel,
stanimir.k.varbanov, quic_vgarodia, agross, andersson,
konrad.dybcio, mchehab
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 5:36 PM, Bryan O'Donoghue wrote:
> On 18/12/2023 11:32, Dikshita Agarwal wrote:
>> drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
>> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++
>> .../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
>> .../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
>> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 +
>> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
>> .../platform/qcom/vcodec/iris/iris_instance.h | 2 +
>> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 +
>> .../media/platform/qcom/vcodec/iris/vpu_common.h | 8 +
>> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 +
>> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++
>> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++
>
> So, unless the code you are adding effects existing upstream venus, I think
> it would be a bit easier to review if you squashed down changes that
> pertain to Iris only.
>
> For example this patch seems to relate to Iris only, so why is it a
> progressive change within your series.
>
> Similar comment for "add vb2 streaming and buffer ops" and other "add
> feature x" patches in this series.
>
> If the change is contained to your own codebase, then progressive changes
> are more noise than content.
>
> Please try to squash down changes - to reduce the number of patches and the
> total LOC being proposed here.
>
As requested in V1, we separated out the iris driver code in functional
patches. Also, as commented in 1st patch, we are OK to have the common
helpers as separate series.
Patch 6 onward all the code is contained to iris driver only.
Could you please elaborate, what do you mean by squashing down the changes?
which patches you want us to squash?
> ---
> bod
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 21/34] media: iris: implement internal buffer management
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (19 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 22/34] media: iris: introduce instance states Dikshita Agarwal
` (19 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Implement functions for creating, queueing, releasing
and destroying buffers for internal usage.
During any bitstream parameter change, these internal buffers
needs to be reallocated and requeued. To optimize the same,
driver checks for reusability of any buffer and skip
re-allocation/re-queue.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 312 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 12 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 3 +
5 files changed, 333 insertions(+)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index c044f78..ce4eaff 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -53,6 +53,8 @@
#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
+#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
+
#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
struct hfi_debug_header {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index 6d4e722..2a3989c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -6,8 +6,22 @@
#include "../buffers.h"
#include "iris_buffer.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "hfi_defines.h"
+#include "iris_hfi_packet.h"
#include "iris_instance.h"
+static const u32 dec_ip_int_buf_type[] = {
+ BUF_BIN,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+};
+
+static const u32 dec_op_int_buf_type[] = {
+ BUF_DPB,
+};
+
static int input_min_count(struct iris_inst *inst)
{
return MIN_BUFFERS;
@@ -225,3 +239,301 @@ int iris_free_buffers(struct iris_inst *inst,
return 0;
}
+
+static int iris_get_internal_buf_info(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffers *buffers;
+ struct iris_core *core;
+ u32 buf_count;
+ u32 buf_size;
+
+ core = inst->core;
+
+ buf_size = call_session_op(core, int_buf_size,
+ inst, buffer_type);
+
+ buf_count = iris_get_buf_min_count(inst, buffer_type);
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buf_size && buf_size <= buffers->size &&
+ buf_count && buf_count <= buffers->min_count) {
+ buffers->reuse = true;
+ } else {
+ buffers->reuse = false;
+ buffers->size = buf_size;
+ buffers->min_count = buf_count;
+ }
+
+ return 0;
+}
+
+int iris_get_internal_buffers(struct iris_inst *inst,
+ u32 plane)
+{
+ int ret = 0;
+ u32 i = 0;
+
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else {
+ return iris_get_internal_buf_info(inst, BUF_DPB);
+ }
+
+ return ret;
+}
+
+static int iris_create_internal_buffer(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type, u32 index)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buffer;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (!buffers->size)
+ return 0;
+
+ buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
+ if (!buffer)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&buffer->list);
+ buffer->type = buffer_type;
+ buffer->index = index;
+ buffer->buffer_size = buffers->size;
+ buffer->dma_attrs = DMA_ATTR_WRITE_COMBINE | DMA_ATTR_NO_KERNEL_MAPPING;
+ list_add_tail(&buffer->list, &buffers->list);
+
+ buffer->kvaddr = dma_alloc_attrs(core->dev, buffer->buffer_size,
+ &buffer->device_addr, GFP_KERNEL, buffer->dma_attrs);
+
+ if (!buffer->kvaddr)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int iris_create_internal_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffers *buffers;
+ int ret = 0;
+ int i;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buffers->reuse)
+ return 0;
+
+ for (i = 0; i < buffers->min_count; i++) {
+ ret = iris_create_internal_buffer(inst, buffer_type, i);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_create_input_internal_buffers(struct iris_inst *inst)
+{
+ int ret = 0;
+ u32 i = 0;
+
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_create_output_internal_buffers(struct iris_inst *inst)
+{
+ return iris_create_internal_buffers(inst, BUF_DPB);
+}
+
+static int set_num_comv(struct iris_inst *inst)
+{
+ u32 num_comv;
+
+ num_comv = inst->cap[NUM_COMV].value;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_COMV_BUFFER_COUNT,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_BITSTREAM,
+ HFI_PAYLOAD_U32,
+ &num_comv,
+ sizeof(u32));
+}
+
+static int iris_queue_internal_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffer *buffer, *dummy;
+ struct iris_buffers *buffers;
+ int ret = 0;
+
+ if (buffer_type == BUF_COMV) {
+ ret = set_num_comv(inst);
+ if (ret)
+ return ret;
+ }
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buffer, dummy, &buffers->list, list) {
+ if (buffer->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ if (buffer->attr & BUF_ATTR_QUEUED)
+ continue;
+ ret = iris_hfi_queue_buffer(inst, buffer);
+ if (ret)
+ return ret;
+ buffer->attr |= BUF_ATTR_QUEUED;
+ }
+
+ return ret;
+}
+
+int iris_queue_input_internal_buffers(struct iris_inst *inst)
+{
+ int ret = 0;
+ u32 i;
+
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_queue_output_internal_buffers(struct iris_inst *inst)
+{
+ return iris_queue_internal_buffers(inst, BUF_DPB);
+}
+
+int iris_destroy_internal_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ buffers = iris_get_buffer_list(inst, buffer->type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->device_addr == buffer->device_addr) {
+ list_del(&buf->list);
+ dma_free_attrs(core->dev, buf->buffer_size, buf->kvaddr,
+ buf->device_addr, buf->dma_attrs);
+ buf->kvaddr = NULL;
+ buf->device_addr = 0;
+ kfree(buf);
+ break;
+ }
+ }
+
+ return 0;
+}
+
+int iris_destroy_internal_buffers(struct iris_inst *inst,
+ u32 plane)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+ const u32 *internal_buf_type;
+ int ret = 0;
+ u32 i, len;
+
+ if (plane == INPUT_MPLANE) {
+ internal_buf_type = dec_ip_int_buf_type;
+ len = ARRAY_SIZE(dec_ip_int_buf_type);
+ } else {
+ internal_buf_type = dec_op_int_buf_type;
+ len = ARRAY_SIZE(dec_op_int_buf_type);
+ }
+
+ for (i = 0; i < len; i++) {
+ buffers = iris_get_buffer_list(inst, internal_buf_type[i]);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buffers->reuse)
+ continue;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ ret = iris_destroy_internal_buffer(inst, buf);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
+}
+
+static int iris_release_internal_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffer *buffer, *dummy;
+ struct iris_buffers *buffers;
+ int ret = 0;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buffers->reuse)
+ return 0;
+
+ list_for_each_entry_safe(buffer, dummy, &buffers->list, list) {
+ if (buffer->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ if (!(buffer->attr & BUF_ATTR_QUEUED))
+ continue;
+ ret = iris_hfi_release_buffer(inst, buffer);
+ if (ret)
+ return ret;
+ buffer->attr |= BUF_ATTR_PENDING_RELEASE;
+ }
+
+ return ret;
+}
+
+int iris_release_input_internal_buffers(struct iris_inst *inst)
+{
+ int ret = 0;
+ u32 i = 0;
+
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index 1cd76a9..ece894e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -46,4 +46,16 @@ int iris_allocate_buffers(struct iris_inst *inst,
int iris_free_buffers(struct iris_inst *inst,
enum iris_buffer_type buf_type);
+int iris_get_internal_buffers(struct iris_inst *inst,
+ u32 plane);
+int iris_create_input_internal_buffers(struct iris_inst *inst);
+int iris_create_output_internal_buffers(struct iris_inst *inst);
+int iris_queue_input_internal_buffers(struct iris_inst *inst);
+int iris_queue_output_internal_buffers(struct iris_inst *inst);
+int iris_destroy_internal_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
+int iris_destroy_internal_buffers(struct iris_inst *inst,
+ u32 plane);
+int iris_release_input_internal_buffers(struct iris_inst *inst);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 4aefdc4..ca2339e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -16,6 +16,10 @@ int iris_hfi_session_close(struct iris_inst *inst);
int iris_hfi_set_property(struct iris_inst *inst,
u32 packet_type, u32 flag, u32 plane, u32 payload_type,
void *payload, u32 payload_size);
+int iris_hfi_queue_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
+int iris_hfi_release_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index b131a50..566048e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -39,6 +39,9 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ inst->buffers.output.actual_count = inst->buffers.output.min_count;
+ inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
inst->fw_min_count = 0;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 22/34] media: iris: introduce instance states
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (20 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 21/34] media: iris: implement internal buffer management Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 23/34] media: iris: implement iris v4l2 ioctl ops supported by decoder Dikshita Agarwal
` (18 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Introduce different states for instances. State transitions
are defined, based on which they would be allowed/rejected/ignored.
IRIS_INST_OPEN - video instance is opened.
IRIS_INST_INPUT_STREAMING - stream on is completed on output plane.
IRIS_INST_OUTPUT_STREAMING - stream on is completed on capture
plane.
IRIS_INST_STREAMING - stream on is completed on both output and
capture planes.
IRIS_INST_CLOSE - video instance is closed.
IRIS_INST_ERROR - error state.
|
v
-------------
+---------| OPEN |--------- +
| ------------- |
| ^ ^ |
| / \ |
| / \ |
| v v |
| ----------- ----------- |
| | INPUT OUTPUT | |
|---| STREAMING STREAMING |---|
| ----------- ----------- |
| ^ ^ |
| \ / |
| \ / |
| v v |
| ------------- |
|--------| STREAMING |-----------|
| ------------- |
| | |
| | |
| v |
| ----------- |
+-------->| CLOSE |<----------+
| ----------- |
| | |
| | |
| v |
| ---------- |
+-------->| ERROR |<-----------+
----------
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../platform/qcom/vcodec/iris/iris_instance.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 86 ++++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 3 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 5 ++
5 files changed, 118 insertions(+)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 5d4c856..275efa8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -37,6 +37,7 @@
* @codec: codec type
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
+ * @state: instance state
*/
struct iris_inst {
@@ -60,6 +61,7 @@ struct iris_inst {
enum codec_type codec;
struct iris_buffers_info buffers;
u32 fw_min_count;
+ enum iris_inst_state state;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 83bbc6b..9bf79a0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -69,3 +69,89 @@ int iris_change_core_state(struct iris_core *core,
return ret;
}
+
+struct iris_inst_state_change_allow {
+ enum iris_inst_state from;
+ enum iris_inst_state to;
+ enum state_change allow;
+};
+
+static enum state_change iris_allow_inst_state_change(struct iris_inst *inst,
+ enum iris_inst_state req_state)
+{
+ enum state_change allow = STATE_CHANGE_ALLOW;
+ int cnt;
+ static struct iris_inst_state_change_allow state[] = {
+ /* from, to, allow */
+ {IRIS_INST_OPEN, IRIS_INST_OPEN, STATE_CHANGE_IGNORE },
+ {IRIS_INST_OPEN, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_CLOSE, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE },
+ {IRIS_INST_CLOSE, IRIS_INST_ERROR, STATE_CHANGE_IGNORE },
+
+ {IRIS_INST_ERROR, IRIS_INST_OPEN, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_ERROR, STATE_CHANGE_IGNORE },
+ };
+
+ for (cnt = 0; cnt < ARRAY_SIZE(state); cnt++) {
+ if (state[cnt].from == inst->state && state[cnt].to == req_state) {
+ allow = state[cnt].allow;
+ break;
+ }
+ }
+
+ return allow;
+}
+
+int iris_inst_change_state(struct iris_inst *inst,
+ enum iris_inst_state request_state)
+{
+ enum state_change allow;
+
+ if (IS_SESSION_ERROR(inst))
+ return 0;
+
+ if (inst->state == request_state)
+ return 0;
+
+ allow = iris_allow_inst_state_change(inst, request_state);
+ if (allow != STATE_CHANGE_ALLOW)
+ return (allow == STATE_CHANGE_DISALLOW ? -EINVAL : 0);
+
+ inst->state = request_state;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index ee20842..6db95a1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -7,6 +7,7 @@
#define _IRIS_STATE_H_
struct iris_core;
+struct iris_inst;
enum iris_core_state {
IRIS_CORE_DEINIT,
@@ -15,8 +16,29 @@ enum iris_core_state {
IRIS_CORE_ERROR,
};
+enum iris_inst_state {
+ IRIS_INST_OPEN,
+ IRIS_INST_INPUT_STREAMING,
+ IRIS_INST_OUTPUT_STREAMING,
+ IRIS_INST_STREAMING,
+ IRIS_INST_CLOSE,
+ IRIS_INST_ERROR,
+};
+
+enum state_change {
+ STATE_CHANGE_ALLOW,
+ STATE_CHANGE_DISALLOW,
+ STATE_CHANGE_IGNORE,
+};
+
+#define IS_SESSION_ERROR(inst) \
+((inst)->state == IRIS_INST_ERROR)
+
bool core_in_valid_state(struct iris_core *core);
int iris_change_core_state(struct iris_core *core,
enum iris_core_state request_state);
+int iris_inst_change_state(struct iris_inst *inst,
+ enum iris_inst_state request_state);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index a57b5fb..8f499a9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -34,6 +34,9 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
else
f = inst->fmt_dst;
+ if (inst->state == IRIS_INST_STREAMING)
+ return -EINVAL;
+
if (*num_planes) {
if (*num_planes != f->fmt.pix_mp.num_planes ||
sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 124333a..68ba75f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -168,6 +168,7 @@ int vidc_open(struct file *filp)
inst->core = core;
inst->session_id = hash32_ptr(inst);
+ iris_inst_change_state(inst, IRIS_INST_OPEN);
mutex_init(&inst->ctx_q_lock);
ret = vidc_add_session(inst);
@@ -244,6 +245,7 @@ int vidc_close(struct file *filp)
v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
close_session(inst);
+ iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
vidc_remove_session(inst);
@@ -295,6 +297,9 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
if (!inst)
return EPOLLERR;
+ if (IS_SESSION_ERROR(inst))
+ return EPOLLERR;
+
poll_wait(filp, &inst->fh.wait, pt);
poll_wait(filp, &inst->vb2q_src->done_wq, pt);
poll_wait(filp, &inst->vb2q_dst->done_wq, pt);
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 23/34] media: iris: implement iris v4l2 ioctl ops supported by decoder
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (21 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 22/34] media: iris: introduce instance states Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 24/34] media: iris: subscribe input and output properties to firmware Dikshita Agarwal
` (17 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
From: Vikash Garodia <quic_vgarodia@quicinc.com>
Implement all iris v4l2 ioctls ops supported by decoder.
Add state checks to ensure ioctl are allowed in valid
instance state only. Codec format can be changed by client
during s_fmt. Update the v4l2 control values according
to the updated codec format.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 23 +
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_common.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 48 +-
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 6 +-
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 212 +++++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 8 +
.../platform/qcom/vcodec/iris/iris_instance.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 38 ++
.../media/platform/qcom/vcodec/iris/iris_state.h | 6 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 17 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 281 ++++++++-
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 6 +-
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 632 ++++++++++++++++++++-
.../platform/qcom/vcodec/iris/platform_common.h | 18 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 42 ++
16 files changed, 1323 insertions(+), 21 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index 2a3989c..1ee840e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -55,6 +55,29 @@ static int output_min_count(struct iris_inst *inst)
return output_min_count;
}
+int update_buffer_count(struct iris_inst *inst, u32 plane)
+{
+ switch (plane) {
+ case INPUT_MPLANE:
+ inst->buffers.input.min_count = input_min_count(inst);
+ if (inst->buffers.input.actual_count < inst->buffers.input.min_count)
+ inst->buffers.input.actual_count = inst->buffers.input.min_count;
+
+ break;
+ case OUTPUT_MPLANE:
+ if (!inst->vb2q_src->streaming)
+ inst->buffers.output.min_count = output_min_count(inst);
+ if (inst->buffers.output.actual_count < inst->buffers.output.min_count)
+ inst->buffers.output.actual_count = inst->buffers.output.min_count;
+
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static u32 internal_buffer_count(struct iris_inst *inst,
enum iris_buffer_type buffer_type)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index ece894e..bdef15f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -34,6 +34,7 @@ struct iris_buffers_info {
struct iris_buffers vpss;
};
+int update_buffer_count(struct iris_inst *inst, u32 plane);
int iris_get_buf_min_count(struct iris_inst *inst,
enum iris_buffer_type buffer_type);
int iris_get_buffer_size(struct iris_inst *inst,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index a83d1c1..6b771f8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -17,6 +17,8 @@
#define DEFAULT_HEIGHT 240
#define DEFAULT_BSE_VPP_DELAY 2
+#define MAX_EVENTS 30
+
#define MB_IN_PIXEL (16 * 16)
#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 6b7aeaa..28977e8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -209,15 +209,15 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
struct iris_inst *inst = NULL;
int ret = 0;
+ inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
switch (ctrl->id) {
case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
- ctrl->val = MIN_CAPTURE_BUFFERS;
+ ctrl->val = inst->buffers.output.min_count;
break;
case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
- ctrl->val = MIN_OUTPUT_BUFFERS;
+ ctrl->val = inst->buffers.input.min_count;
break;
default:
- inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
cap_id = get_cap_id(inst, ctrl->id);
if (is_valid_cap_id(cap_id))
ctrl->val = inst->cap[cap_id].value;
@@ -232,12 +232,17 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct cap_entry *entry = NULL, *temp = NULL;
struct list_head children_list, firmware_list;
+ struct ctrl_data *priv_ctrl_data;
enum plat_inst_cap_type cap_id;
bool cap_present[INST_CAP_MAX];
struct plat_inst_cap *cap;
struct iris_inst *inst;
int ret = 0;
+ priv_ctrl_data = ctrl->priv ? ctrl->priv : NULL;
+ if (priv_ctrl_data && priv_ctrl_data->skip_s_ctrl)
+ return 0;
+
inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
cap = &inst->cap[0];
@@ -249,6 +254,9 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
if (!is_valid_cap_id(cap_id))
return -EINVAL;
+ if (!allow_s_ctrl(inst, cap_id))
+ return -EBUSY;
+
cap[cap_id].flags |= CAP_FLAG_CLIENT_SET;
if (!inst->vb2q_src->streaming) {
@@ -282,11 +290,12 @@ static const struct v4l2_ctrl_ops ctrl_ops = {
.g_volatile_ctrl = vdec_op_g_volatile_ctrl,
};
-int ctrls_init(struct iris_inst *inst)
+int ctrls_init(struct iris_inst *inst, bool init)
{
int num_ctrls = 0, ctrl_idx = 0;
struct plat_inst_cap *cap;
struct iris_core *core;
+ u64 step_or_mask;
int idx = 0;
int ret = 0;
@@ -300,10 +309,12 @@ int ctrls_init(struct iris_inst *inst)
if (!num_ctrls)
return -EINVAL;
- ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
- INST_CAP_MAX * core->dec_codecs_count);
- if (ret)
- return ret;
+ if (init) {
+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
+ INST_CAP_MAX * core->dec_codecs_count);
+ if (ret)
+ return ret;
+ }
for (idx = 0; idx < INST_CAP_MAX; idx++) {
struct v4l2_ctrl *ctrl;
@@ -316,6 +327,27 @@ int ctrls_init(struct iris_inst *inst)
goto error;
}
+ if (!init) {
+ struct ctrl_data ctrl_priv_data;
+
+ ctrl = v4l2_ctrl_find(&inst->ctrl_handler, cap[idx].v4l2_id);
+ if (ctrl) {
+ step_or_mask = (cap[idx].flags & CAP_FLAG_MENU) ?
+ ~(cap[idx].step_or_mask) :
+ cap[idx].step_or_mask;
+ memset(&ctrl_priv_data, 0, sizeof(ctrl_priv_data));
+ ctrl_priv_data.skip_s_ctrl = true;
+ ctrl->priv = &ctrl_priv_data;
+ v4l2_ctrl_modify_range(ctrl,
+ cap[idx].min,
+ cap[idx].max,
+ step_or_mask,
+ cap[idx].value);
+ ctrl->priv = NULL;
+ continue;
+ }
+ }
+
if (cap[idx].flags & CAP_FLAG_MENU) {
ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
&ctrl_ops,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 0f67f4f..22ee6c4b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -13,6 +13,10 @@ struct cap_entry {
enum plat_inst_cap_type cap_id;
};
+struct ctrl_data {
+ bool skip_s_ctrl;
+};
+
int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_pipe(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
@@ -25,6 +29,6 @@ int iris_init_instance_caps(struct iris_core *core);
int iris_init_core_caps(struct iris_core *core);
int get_inst_capability(struct iris_inst *inst);
int adjust_v4l2_properties(struct iris_inst *inst);
-int ctrls_init(struct iris_inst *inst);
+int ctrls_init(struct iris_inst *inst, bool init);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 335885f..ff44cda 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -79,6 +79,112 @@ bool is_split_mode_enabled(struct iris_inst *inst)
return false;
}
+inline bool is_10bit_colorformat(enum colorformat_type colorformat)
+{
+ return colorformat == FMT_TP10C;
+}
+
+inline bool is_8bit_colorformat(enum colorformat_type colorformat)
+{
+ return colorformat == FMT_NV12 ||
+ colorformat == FMT_NV12C ||
+ colorformat == FMT_NV21;
+}
+
+u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec)
+{
+ const struct codec_info *codec_info;
+ struct iris_core *core;
+ u32 v4l2_codec = 0;
+ u32 i, size;
+
+ core = inst->core;
+ codec_info = core->platform_data->format_data->codec_info;
+ size = core->platform_data->format_data->codec_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (codec_info[i].codec == codec)
+ return codec_info[i].v4l2_codec;
+ }
+
+ return v4l2_codec;
+}
+
+enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec)
+{
+ const struct codec_info *codec_info;
+ enum codec_type codec = 0;
+ struct iris_core *core;
+ u32 i, size;
+
+ core = inst->core;
+ codec_info = core->platform_data->format_data->codec_info;
+ size = core->platform_data->format_data->codec_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (codec_info[i].v4l2_codec == v4l2_codec)
+ return codec_info[i].codec;
+ }
+
+ return codec;
+}
+
+u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat)
+{
+ const struct color_format_info *color_format_info;
+ u32 v4l2_colorformat = 0;
+ struct iris_core *core;
+ u32 i, size;
+
+ core = inst->core;
+ color_format_info = core->platform_data->format_data->color_format_info;
+ size = core->platform_data->format_data->color_format_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (color_format_info[i].color_format == colorformat)
+ return color_format_info[i].v4l2_color_format;
+ }
+
+ return v4l2_colorformat;
+}
+
+enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat)
+{
+ const struct color_format_info *color_format_info;
+ enum colorformat_type colorformat = 0;
+ struct iris_core *core;
+ u32 i, size;
+
+ core = inst->core;
+ color_format_info = core->platform_data->format_data->color_format_info;
+ size = core->platform_data->format_data->color_format_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (color_format_info[i].v4l2_color_format == v4l2_colorformat)
+ return color_format_info[i].color_format;
+ }
+
+ return colorformat;
+}
+
+struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type)
+{
+ struct vb2_queue *vb2q = NULL;
+
+ switch (type) {
+ case INPUT_MPLANE:
+ vb2q = inst->vb2q_src;
+ break;
+ case OUTPUT_MPLANE:
+ vb2q = inst->vb2q_dst;
+ break;
+ default:
+ return NULL;
+ }
+
+ return vb2q;
+}
+
static int process_inst_timeout(struct iris_inst *inst)
{
struct iris_inst *instance;
@@ -128,13 +234,119 @@ int close_session(struct iris_inst *inst)
inst->packet = NULL;
if (wait_for_response) {
+ mutex_unlock(&inst->lock);
ret = wait_for_completion_timeout(&inst->completions[SIGNAL_CMD_CLOSE],
msecs_to_jiffies(hw_response_timeout_val));
if (!ret) {
ret = -ETIMEDOUT;
process_inst_timeout(inst);
}
+ mutex_lock(&inst->lock);
}
return ret;
}
+
+static int check_core_mbps_mbpf(struct iris_inst *inst)
+{
+ u32 mbpf = 0, mbps = 0, total_mbpf = 0, total_mbps = 0;
+ struct iris_core *core;
+ struct iris_inst *instance;
+ u32 fps;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(instance, &core->instances, list) {
+ fps = inst->cap[QUEUED_RATE].value >> 16;
+ mbpf = get_mbpf(inst);
+ mbps = mbpf * fps;
+ total_mbpf += mbpf;
+ total_mbps += mbps;
+ }
+ mutex_unlock(&core->lock);
+
+ if (total_mbps > core->cap[MAX_MBPS].value ||
+ total_mbpf > core->cap[MAX_MBPF].value)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int check_inst_mbpf(struct iris_inst *inst)
+{
+ u32 mbpf = 0, max_mbpf = 0;
+
+ max_mbpf = inst->cap[MBPF].max;
+ mbpf = get_mbpf(inst);
+ if (mbpf > max_mbpf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int check_resolution_supported(struct iris_inst *inst)
+{
+ u32 width = 0, height = 0, min_width, min_height,
+ max_width, max_height;
+
+ width = inst->fmt_src->fmt.pix_mp.width;
+ height = inst->fmt_src->fmt.pix_mp.height;
+
+ min_width = inst->cap[FRAME_WIDTH].min;
+ max_width = inst->cap[FRAME_WIDTH].max;
+ min_height = inst->cap[FRAME_HEIGHT].min;
+ max_height = inst->cap[FRAME_HEIGHT].max;
+
+ if (!(min_width <= width && width <= max_width) ||
+ !(min_height <= height && height <= max_height))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int check_max_sessions(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ u32 num_sessions = 0;
+ struct iris_inst *i;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+ list_for_each_entry(i, &core->instances, list) {
+ num_sessions++;
+ }
+ mutex_unlock(&core->lock);
+
+ if (num_sessions > core->cap[MAX_SESSION_COUNT].value)
+ return -ENOMEM;
+
+ return 0;
+}
+
+int check_session_supported(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_core_mbps_mbpf(inst);
+ if (ret)
+ goto exit;
+
+ ret = check_inst_mbpf(inst);
+ if (ret)
+ goto exit;
+
+ ret = check_resolution_supported(inst);
+ if (ret)
+ goto exit;
+
+ ret = check_max_sessions(inst);
+ if (ret)
+ goto exit;
+
+ return ret;
+exit:
+ dev_err(inst->core->dev, "current session not supported(%d)\n", ret);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 9e85510..fe85d23 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -28,6 +28,14 @@ int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);
bool is_linear_colorformat(u32 colorformat);
+bool is_10bit_colorformat(enum colorformat_type colorformat);
+bool is_8bit_colorformat(enum colorformat_type colorformat);
bool is_split_mode_enabled(struct iris_inst *inst);
+u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
+enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
+u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat);
+enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat);
+struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type);
+int check_session_supported(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 275efa8..365f844 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -23,6 +23,7 @@
* @vb2q_src: source vb2 queue
* @vb2q_dst: destination vb2 queue
* @ctx_q_lock: lock to serialize queues related ioctls
+ * @lock: lock to seralise forward and reverse threads
* @fh: reference of v4l2 file handler
* @fmt_src: structure of v4l2_format for source
* @fmt_dst: structure of v4l2_format for destination
@@ -38,6 +39,7 @@
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
+ * @ipsc_properties_set: boolean to set ipsc properties to fw
*/
struct iris_inst {
@@ -47,6 +49,7 @@ struct iris_inst {
struct vb2_queue *vb2q_src;
struct vb2_queue *vb2q_dst;
struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */
+ struct mutex lock;
struct v4l2_fh fh;
struct v4l2_format *fmt_src;
struct v4l2_format *fmt_dst;
@@ -62,6 +65,7 @@ struct iris_inst {
struct iris_buffers_info buffers;
u32 fw_min_count;
enum iris_inst_state state;
+ bool ipsc_properties_set;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 9bf79a0..4cf6b69 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -155,3 +155,41 @@ int iris_inst_change_state(struct iris_inst *inst,
return 0;
}
+
+bool allow_s_fmt(struct iris_inst *inst, u32 type)
+{
+ return (inst->state == IRIS_INST_OPEN) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING);
+}
+
+bool allow_reqbufs(struct iris_inst *inst, u32 type)
+{
+ return (inst->state == IRIS_INST_OPEN) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING);
+}
+
+bool allow_streamon(struct iris_inst *inst, u32 type)
+{
+ return (type == INPUT_MPLANE && inst->state == IRIS_INST_OPEN) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OPEN) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING);
+}
+
+bool allow_streamoff(struct iris_inst *inst, u32 type)
+{
+ return (type == INPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_STREAMING);
+}
+
+bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id)
+{
+ return ((inst->state == IRIS_INST_OPEN) ||
+ ((inst->cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED) &&
+ (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_STREAMING)));
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 6db95a1..35263e8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -41,4 +41,10 @@ int iris_change_core_state(struct iris_core *core,
int iris_inst_change_state(struct iris_inst *inst,
enum iris_inst_state request_state);
+bool allow_s_fmt(struct iris_inst *inst, u32 type);
+bool allow_reqbufs(struct iris_inst *inst, u32 type);
+bool allow_streamon(struct iris_inst *inst, u32 type);
+bool allow_streamoff(struct iris_inst *inst, u32 type);
+bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index 8f499a9..66b5295 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -4,6 +4,7 @@
*/
#include "iris_buffer.h"
+#include "iris_ctrls.h"
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_instance.h"
@@ -47,6 +48,22 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
if (!buffer_type)
return -EINVAL;
+ if (list_empty(&inst->caps_list)) {
+ ret = prepare_dependency_list(inst);
+ if (ret)
+ return ret;
+ }
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ret = adjust_v4l2_properties(inst);
+ if (ret)
+ return ret;
+ }
+
+ ret = check_session_supported(inst);
+ if (ret)
+ return ret;
+
ret = iris_free_buffers(inst, buffer_type);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 566048e..64067d5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -3,13 +3,51 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include <media/v4l2-event.h>
+
#include "iris_buffer.h"
#include "iris_common.h"
+#include "iris_ctrls.h"
+#include "iris_helpers.h"
#include "iris_vdec.h"
-void vdec_inst_init(struct iris_inst *inst)
+static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
+{
+ bool session_init = false;
+ int ret;
+
+ if (!inst->codec)
+ session_init = true;
+
+ if (inst->codec && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec)
+ return 0;
+
+ inst->codec = v4l2_codec_to_driver(inst, v4l2_codec);
+ if (!inst->codec)
+ return -EINVAL;
+
+ inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec;
+ ret = get_inst_capability(inst);
+ if (ret)
+ return ret;
+
+ ret = ctrls_init(inst, session_init);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
+
+int vdec_inst_init(struct iris_inst *inst)
{
struct v4l2_format *f;
+ int ret;
inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL);
inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL);
@@ -43,6 +81,10 @@ void vdec_inst_init(struct iris_inst *inst)
inst->buffers.output.actual_count = inst->buffers.output.min_count;
inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
inst->fw_min_count = 0;
+
+ ret = vdec_codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat);
+
+ return ret;
}
void vdec_inst_deinit(struct iris_inst *inst)
@@ -50,3 +92,240 @@ void vdec_inst_deinit(struct iris_inst *inst)
kfree(inst->fmt_dst);
kfree(inst->fmt_src);
}
+
+static int vdec_check_colorformat_supported(struct iris_inst *inst,
+ enum colorformat_type colorformat)
+{
+ bool supported = true;
+
+ if (!inst->vb2q_src->streaming)
+ return true;
+
+ if (inst->cap[BIT_DEPTH].value == BIT_DEPTH_8 &&
+ !is_8bit_colorformat(colorformat))
+ supported = false;
+ if (inst->cap[BIT_DEPTH].value == BIT_DEPTH_10 &&
+ !is_10bit_colorformat(colorformat))
+ supported = false;
+ if (inst->cap[CODED_FRAMES].value == CODED_FRAMES_INTERLACE)
+ supported = false;
+
+ return supported;
+}
+
+int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f)
+{
+ struct iris_core *core;
+ u32 array[32] = {0};
+ u32 i = 0;
+
+ if (f->index >= ARRAY_SIZE(array))
+ return -EINVAL;
+
+ core = inst->core;
+ if (f->type == INPUT_MPLANE) {
+ u32 codecs = core->cap[DEC_CODECS].value;
+ u32 codecs_count = hweight32(codecs);
+ u32 idx = 0;
+
+ for (i = 0; i <= codecs_count; i++) {
+ if (codecs & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ array[idx] = codecs & BIT(i);
+ idx++;
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_codec_from_driver(inst, array[f->index]);
+ f->flags = V4L2_FMT_FLAG_COMPRESSED;
+ strscpy(f->description, "codec", sizeof(f->description));
+ } else if (f->type == OUTPUT_MPLANE) {
+ u32 formats = inst->cap[PIX_FMTS].step_or_mask;
+ u32 idx = 0;
+
+ for (i = 0; i <= 31; i++) {
+ if (formats & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ if (vdec_check_colorformat_supported(inst, formats & BIT(i))) {
+ array[idx] = formats & BIT(i);
+ idx++;
+ }
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_colorformat_from_driver(inst, array[f->index]);
+ strscpy(f->description, "colorformat", sizeof(f->description));
+ }
+
+ if (!f->pixelformat)
+ return -EINVAL;
+
+ memset(f->reserved, 0, sizeof(f->reserved));
+
+ return 0;
+}
+
+int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ struct v4l2_format *f_inst;
+ u32 pix_fmt;
+
+ memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+ if (f->type == INPUT_MPLANE) {
+ pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f_inst = inst->fmt_src;
+ f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height;
+ f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat;
+ pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ }
+ } else if (f->type == OUTPUT_MPLANE) {
+ pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f_inst = inst->fmt_dst;
+ f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat;
+ f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height;
+ }
+ if (inst->vb2q_src->streaming) {
+ f_inst = inst->fmt_src;
+ f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height;
+ f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ if (pixmp->field == V4L2_FIELD_ANY)
+ pixmp->field = V4L2_FIELD_NONE;
+
+ pixmp->num_planes = 1;
+
+ return 0;
+}
+
+int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_format *fmt, *output_fmt;
+ enum colorformat_type colorformat;
+ u32 codec_align, stride = 0;
+ int ret = 0;
+
+ vdec_try_fmt(inst, f);
+
+ if (f->type == INPUT_MPLANE) {
+ if (inst->fmt_src->fmt.pix_mp.pixelformat !=
+ f->fmt.pix_mp.pixelformat) {
+ ret = vdec_codec_change(inst, f->fmt.pix_mp.pixelformat);
+ if (ret)
+ return ret;
+ }
+
+ fmt = inst->fmt_src;
+ fmt->type = INPUT_MPLANE;
+
+ codec_align = inst->fmt_src->fmt.pix_mp.pixelformat ==
+ V4L2_PIX_FMT_HEVC ? 32 : 16;
+ fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align);
+ fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align);
+ fmt->fmt.pix_mp.num_planes = 1;
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT);
+ inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT);
+ if (inst->buffers.input.actual_count <
+ inst->buffers.input.min_count) {
+ inst->buffers.input.actual_count =
+ inst->buffers.input.min_count;
+ }
+ inst->buffers.input.size =
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+
+ output_fmt = inst->fmt_dst;
+ output_fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ output_fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ output_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ output_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+
+ inst->crop.left = 0;
+ inst->crop.top = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+ } else if (f->type == OUTPUT_MPLANE) {
+ fmt = inst->fmt_dst;
+ fmt->type = OUTPUT_MPLANE;
+ if (inst->vb2q_src->streaming) {
+ f->fmt.pix_mp.height = inst->fmt_src->fmt.pix_mp.height;
+ f->fmt.pix_mp.width = inst->fmt_src->fmt.pix_mp.width;
+ }
+ fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat;
+ codec_align = f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C ? 192 : 128;
+ fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align);
+ codec_align = f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C ? 16 : 32;
+ fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align);
+ fmt->fmt.pix_mp.num_planes = 1;
+ if (f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C) {
+ stride = ALIGN(f->fmt.pix_mp.width, 192);
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(stride * 4 / 3, 256);
+ } else {
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(f->fmt.pix_mp.width, 128);
+ }
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
+
+ if (!inst->vb2q_src->streaming)
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ if (inst->buffers.output.actual_count <
+ inst->buffers.output.min_count) {
+ inst->buffers.output.actual_count =
+ inst->buffers.output.min_count;
+ }
+
+ colorformat = v4l2_colorformat_to_driver(inst, fmt->fmt.pix_mp.pixelformat);
+ inst->buffers.output.size =
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+ inst->cap[PIX_FMTS].value = colorformat;
+
+ if (!inst->vb2q_src->streaming) {
+ inst->crop.top = 0;
+ inst->crop.left = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+ }
+ } else {
+ return -EINVAL;
+ }
+ memcpy(f, fmt, sizeof(*fmt));
+
+ return ret;
+}
+
+int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub)
+{
+ int ret = 0;
+
+ switch (sub->type) {
+ case V4L2_EVENT_EOS:
+ ret = v4l2_event_subscribe(&inst->fh, sub, MAX_EVENTS, NULL);
+ break;
+ case V4L2_EVENT_SOURCE_CHANGE:
+ ret = v4l2_src_change_event_subscribe(&inst->fh, sub);
+ break;
+ case V4L2_EVENT_CTRL:
+ ret = v4l2_ctrl_subscribe_event(&inst->fh, sub);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index dc8f43f..a2f159d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -8,7 +8,11 @@
#include "iris_instance.h"
-void vdec_inst_init(struct iris_inst *inst);
+int vdec_inst_init(struct iris_inst *inst);
void vdec_inst_deinit(struct iris_inst *inst);
+int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
+int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 68ba75f..5c76821 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -3,6 +3,11 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include <linux/videodev2.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+
+#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
@@ -12,6 +17,9 @@
#include "iris_ctrls.h"
#include "iris_vb2.h"
+#define VIDC_DRV_NAME "iris_driver"
+#define VIDC_BUS_NAME "platform:iris_bus"
+
static int vidc_v4l2_fh_init(struct iris_inst *inst)
{
struct iris_core *core;
@@ -169,6 +177,7 @@ int vidc_open(struct file *filp)
inst->core = core;
inst->session_id = hash32_ptr(inst);
iris_inst_change_state(inst, IRIS_INST_OPEN);
+ mutex_init(&inst->lock);
mutex_init(&inst->ctx_q_lock);
ret = vidc_add_session(inst);
@@ -200,14 +209,6 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_inst_deinit;
- ret = get_inst_capability(inst);
- if (ret)
- goto fail_queue_deinit;
-
- ret = ctrls_init(inst);
- if (ret)
- goto fail_queue_deinit;
-
ret = iris_hfi_session_open(inst);
if (ret) {
dev_err(core->dev, "%s: session open failed\n", __func__);
@@ -220,7 +221,6 @@ int vidc_open(struct file *filp)
fail_core_deinit:
v4l2_ctrl_handler_free(&inst->ctrl_handler);
iris_core_deinit(core);
-fail_queue_deinit:
vidc_vb2_queue_deinit(inst);
fail_inst_deinit:
vdec_inst_deinit(inst);
@@ -229,6 +229,7 @@ int vidc_open(struct file *filp)
vidc_remove_session(inst);
fail_free_inst:
mutex_destroy(&inst->ctx_q_lock);
+ mutex_destroy(&inst->lock);
kfree(inst);
return ret;
@@ -244,14 +245,16 @@ int vidc_close(struct file *filp)
v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
+ mutex_lock(&inst->lock);
close_session(inst);
iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
vidc_remove_session(inst);
+ mutex_unlock(&inst->lock);
mutex_destroy(&inst->ctx_q_lock);
+ mutex_destroy(&inst->lock);
kfree(inst);
-
filp->private_data = NULL;
return 0;
@@ -313,6 +316,588 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
return poll;
}
+static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_enum_fmt(inst, f);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (!allow_s_fmt(inst, f->type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_try_fmt(inst, f);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (!allow_s_fmt(inst, f->type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_s_fmt(inst, f);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_g_fmt(struct file *filp, void *fh, struct v4l2_format *f)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (f->type == INPUT_MPLANE)
+ memcpy(f, inst->fmt_src, sizeof(*f));
+ else if (f->type == OUTPUT_MPLANE)
+ memcpy(f, inst->fmt_dst, sizeof(*f));
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_enum_framesizes(struct file *filp, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ enum colorformat_type colorfmt;
+ struct iris_inst *inst;
+ enum codec_type codec;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !fsize)
+ return -EINVAL;
+
+ if (fsize->index)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ codec = v4l2_codec_to_driver(inst, fsize->pixel_format);
+ if (!codec) {
+ colorfmt = v4l2_colorformat_to_driver(inst, fsize->pixel_format);
+ if (colorfmt == FMT_NONE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise.min_width = inst->cap[FRAME_WIDTH].min;
+ fsize->stepwise.max_width = inst->cap[FRAME_WIDTH].max;
+ fsize->stepwise.step_width = inst->cap[FRAME_WIDTH].step_or_mask;
+ fsize->stepwise.min_height = inst->cap[FRAME_HEIGHT].min;
+ fsize->stepwise.max_height = inst->cap[FRAME_HEIGHT].max;
+ fsize->stepwise.step_height = inst->cap[FRAME_HEIGHT].step_or_mask;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_reqbufs(struct file *filp, void *fh, struct v4l2_requestbuffers *b)
+{
+ struct vb2_queue *vb2q = NULL;
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (!allow_reqbufs(inst, b->type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ vb2q = get_vb2q(inst, b->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_reqbufs(vb2q, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_querybuf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct vb2_queue *vb2q = NULL;
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ vb2q = get_vb2q(inst, b->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_querybuf(vb2q, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_create_bufs(struct file *filp, void *fh, struct v4l2_create_buffers *b)
+{
+ struct iris_inst *inst;
+ struct vb2_queue *vb2q;
+ struct v4l2_format *f;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ f = &b->format;
+ vb2q = get_vb2q(inst, f->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_create_bufs(vb2q, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_prepare_buf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct video_device *vdev;
+ struct iris_inst *inst;
+ struct vb2_queue *vb2q;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ vdev = video_devdata(filp);
+ if (!inst || !vdev || !vdev->v4l2_dev->mdev)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ vb2q = get_vb2q(inst, b->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_prepare_buf(vb2q, vdev->v4l2_dev->mdev, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_qbuf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct video_device *vdev;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ vdev = video_devdata(filp);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (b->type == INPUT_MPLANE)
+ ret = vb2_qbuf(inst->vb2q_src, vdev->v4l2_dev->mdev, b);
+ else if (b->type == OUTPUT_MPLANE)
+ ret = vb2_qbuf(inst->vb2q_dst, vdev->v4l2_dev->mdev, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_dqbuf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (b->type == INPUT_MPLANE)
+ ret = vb2_dqbuf(inst->vb2q_src, b, true);
+ else if (b->type == OUTPUT_MPLANE)
+ ret = vb2_dqbuf(inst->vb2q_dst, b, true);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_streamon(struct file *filp, void *fh, enum v4l2_buf_type type)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (!allow_streamon(inst, type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (type == INPUT_MPLANE)
+ ret = vb2_streamon(inst->vb2q_src, type);
+ else if (type == OUTPUT_MPLANE)
+ ret = vb2_streamon(inst->vb2q_dst, type);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_streamoff(struct file *filp, void *fh, enum v4l2_buf_type type)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (!allow_streamoff(inst, type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (type == INPUT_MPLANE)
+ ret = vb2_streamoff(inst->vb2q_src, type);
+ else if (type == OUTPUT_MPLANE)
+ ret = vb2_streamoff(inst->vb2q_dst, type);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_querycap(struct file *filp, void *fh, struct v4l2_capability *cap)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ strscpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
+ strscpy(cap->bus_info, VIDC_BUS_NAME, sizeof(cap->bus_info));
+ memset(cap->reserved, 0, sizeof(cap->reserved));
+ strscpy(cap->card, "iris_decoder", sizeof(cap->card));
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_queryctrl(struct file *filp, void *fh, struct v4l2_queryctrl *q_ctrl)
+{
+ struct v4l2_ctrl *ctrl;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !q_ctrl)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ctrl = v4l2_ctrl_find(&inst->ctrl_handler, q_ctrl->id);
+ if (!ctrl) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ q_ctrl->minimum = ctrl->minimum;
+ q_ctrl->maximum = ctrl->maximum;
+ q_ctrl->default_value = ctrl->default_value;
+ q_ctrl->flags = 0;
+ q_ctrl->step = ctrl->step;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qmenu)
+{
+ struct v4l2_ctrl *ctrl;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !qmenu)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ctrl = v4l2_ctrl_find(&inst->ctrl_handler, qmenu->id);
+ if (!ctrl) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (ctrl->type != V4L2_CTRL_TYPE_MENU) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (qmenu->index < ctrl->minimum || qmenu->index > ctrl->maximum) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (ctrl->menu_skip_mask & (1 << qmenu->index)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = container_of(fh, struct iris_inst, fh);
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_subscribe_event(inst, sub);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_unsubscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = container_of(fh, struct iris_inst, fh);
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = v4l2_event_unsubscribe(&inst->fh, sub);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection *s)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !s)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP:
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE:
+ s->r.left = inst->crop.left;
+ s->r.top = inst->crop.top;
+ s->r.width = inst->crop.width;
+ s->r.height = inst->crop.height;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static const struct v4l2_file_operations v4l2_file_ops = {
.owner = THIS_MODULE,
.open = vidc_open,
@@ -335,11 +920,38 @@ static struct vb2_mem_ops iris_vb2_mem_ops = {
.unmap_dmabuf = iris_vb2_unmap_dmabuf,
};
+static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
+ .vidioc_enum_fmt_vid_cap = vidc_enum_fmt,
+ .vidioc_enum_fmt_vid_out = vidc_enum_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt,
+ .vidioc_try_fmt_vid_out_mplane = vidc_try_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = vidc_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = vidc_s_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = vidc_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = vidc_g_fmt,
+ .vidioc_enum_framesizes = vidc_enum_framesizes,
+ .vidioc_reqbufs = vidc_reqbufs,
+ .vidioc_querybuf = vidc_querybuf,
+ .vidioc_create_bufs = vidc_create_bufs,
+ .vidioc_prepare_buf = vidc_prepare_buf,
+ .vidioc_qbuf = vidc_qbuf,
+ .vidioc_dqbuf = vidc_dqbuf,
+ .vidioc_streamon = vidc_streamon,
+ .vidioc_streamoff = vidc_streamoff,
+ .vidioc_querycap = vidc_querycap,
+ .vidioc_queryctrl = vidc_queryctrl,
+ .vidioc_querymenu = vidc_querymenu,
+ .vidioc_subscribe_event = vidc_subscribe_event,
+ .vidioc_unsubscribe_event = vidc_unsubscribe_event,
+ .vidioc_g_selection = vidc_g_selection,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
core->vb2_ops = &iris_vb2_ops;
core->vb2_mem_ops = &iris_vb2_mem_ops;
+ core->v4l2_ioctl_ops = &v4l2_ioctl_ops;
return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index e242614..abd11fa 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -166,6 +166,23 @@ struct plat_inst_caps {
struct plat_inst_cap cap[INST_CAP_MAX + 1];
};
+struct codec_info {
+ u32 v4l2_codec;
+ enum codec_type codec;
+};
+
+struct color_format_info {
+ u32 v4l2_color_format;
+ enum colorformat_type color_format;
+};
+
+struct format_capability {
+ struct codec_info *codec_info;
+ u32 codec_info_size;
+ struct color_format_info *color_format_info;
+ u32 color_format_info_size;
+};
+
struct platform_data {
const struct bus_info *bus_tbl;
unsigned int bus_tbl_size;
@@ -182,6 +199,7 @@ struct platform_data {
const struct reg_preset_info *reg_prst_tbl;
unsigned int reg_prst_tbl_size;
struct ubwc_config_data *ubwc_config;
+ struct format_capability *format_data;
const char *fwname;
u32 pas_id;
struct plat_core_cap *core_data;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 0759ac5..85bc677 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -19,6 +19,40 @@
#define MINIMUM_FPS 1
#define MAXIMUM_FPS 480
+static struct codec_info codec_data_sm8550[] = {
+ {
+ .v4l2_codec = V4L2_PIX_FMT_H264,
+ .codec = H264,
+ },
+ {
+ .v4l2_codec = V4L2_PIX_FMT_HEVC,
+ .codec = HEVC,
+ },
+ {
+ .v4l2_codec = V4L2_PIX_FMT_VP9,
+ .codec = VP9,
+ },
+};
+
+static struct color_format_info color_format_data_sm8550[] = {
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_NV12,
+ .color_format = FMT_NV12,
+ },
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_NV21,
+ .color_format = FMT_NV21,
+ },
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_QC08C,
+ .color_format = FMT_NV12C,
+ },
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_QC10C,
+ .color_format = FMT_TP10C,
+ },
+};
+
static struct plat_core_cap core_data_sm8550[] = {
{DEC_CODECS, H264 | HEVC | VP9},
{MAX_SESSION_COUNT, 16},
@@ -341,6 +375,13 @@ static struct ubwc_config_data ubwc_config_sm8550[] = {
UBWC_CONFIG(8, 32, 16, 0, 1, 1, 1),
};
+static struct format_capability format_data_sm8550 = {
+ .codec_info = codec_data_sm8550,
+ .codec_info_size = ARRAY_SIZE(codec_data_sm8550),
+ .color_format_info = color_format_data_sm8550,
+ .color_format_info_size = ARRAY_SIZE(color_format_data_sm8550),
+};
+
struct platform_data sm8550_data = {
.bus_tbl = sm8550_bus_table,
.bus_tbl_size = ARRAY_SIZE(sm8550_bus_table),
@@ -367,4 +408,5 @@ struct platform_data sm8550_data = {
.inst_cap_data = instance_cap_data_sm8550,
.inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550),
.ubwc_config = ubwc_config_sm8550,
+ .format_data = &format_data_sm8550,
};
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 24/34] media: iris: subscribe input and output properties to firmware
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (22 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 23/34] media: iris: implement iris v4l2 ioctl ops supported by decoder Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 25/34] media: iris: subscribe src change and handle firmware responses Dikshita Agarwal
` (16 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
From: Vikash Garodia <quic_vgarodia@quicinc.com>
Driver can subscribe to different properties for which
it expects a response from firmware. Different SOCs
firmware can support different properties which can be
subscribed by driver.
HFI_CMD_SUBSCRIBE_MODE with HFI_MODE_PROPERTY - to subscribe
any property to firmware.
Also, set below mandatory properties on capture plane:
HFI_PROP_COLOR_FORMAT
HFI_PROP_LINEAR_STRIDE_SCANLINE
HFI_PROP_UBWC_STRIDE_SCANLINE.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 31 +++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 38 +++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 42 ++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 3 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 154 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 2 +
.../platform/qcom/vcodec/iris/platform_common.h | 8 ++
.../platform/qcom/vcodec/iris/platform_sm8550.c | 29 ++++
9 files changed, 310 insertions(+)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index ce4eaff..a6078a5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -25,6 +25,29 @@
#define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008
#define HFI_PROP_UBWC_BANK_SPREADING 0x03000009
+enum hfi_property_mode_type {
+ HFI_MODE_NONE = 0x00000000,
+ HFI_MODE_PORT_SETTINGS_CHANGE = 0x00000001,
+ HFI_MODE_PROPERTY = 0x00000002,
+};
+
+#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B
+
+enum hfi_color_format {
+ HFI_COLOR_FMT_OPAQUE = 0,
+ HFI_COLOR_FMT_NV12 = 1,
+ HFI_COLOR_FMT_NV12_UBWC = 2,
+ HFI_COLOR_FMT_P010 = 3,
+ HFI_COLOR_FMT_TP10_UBWC = 4,
+ HFI_COLOR_FMT_RGBA8888 = 5,
+ HFI_COLOR_FMT_RGBA8888_UBWC = 6,
+ HFI_COLOR_FMT_NV21 = 7,
+};
+
+#define HFI_PROP_COLOR_FORMAT 0x03000101
+
+#define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104
+
#define HFI_PROP_PROFILE 0x03000107
#define HFI_PROP_LEVEL 0x03000108
@@ -49,10 +72,18 @@
#define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b
+#define HFI_PROP_PICTURE_TYPE 0x03000162
+
#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
+#define HFI_PROP_NO_OUTPUT 0x0300016a
+
+#define HFI_PROP_DPB_LIST 0x0300017A
+
+#define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190
+
#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 8f1e456..d15ce5a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -219,6 +219,44 @@ int iris_hfi_session_open(struct iris_inst *inst)
return ret;
}
+int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
+ u32 cmd, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ cmd,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ payload_type,
+ payload,
+ payload_size);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
int iris_hfi_session_close(struct iris_inst *inst)
{
struct iris_core *core;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index ca2339e..d6b3fca 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -13,6 +13,9 @@ int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
int iris_hfi_session_close(struct iris_inst *inst);
+int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
+ u32 cmd, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size);
int iris_hfi_set_property(struct iris_inst *inst,
u32 packet_type, u32 flag, u32 plane, u32 payload_type,
void *payload, u32 payload_size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index a3544d8..d4cdfcf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -34,6 +34,24 @@ u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type)
return hfi_port;
}
+u32 get_hfi_port(u32 plane)
+{
+ u32 hfi_port = HFI_PORT_NONE;
+
+ switch (plane) {
+ case INPUT_MPLANE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case OUTPUT_MPLANE:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_port;
+}
+
static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
{
switch (buffer_type) {
@@ -58,6 +76,30 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}
+u32 get_hfi_colorformat(u32 colorformat)
+{
+ u32 hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC;
+
+ switch (colorformat) {
+ case V4L2_PIX_FMT_NV12:
+ hfi_colorformat = HFI_COLOR_FMT_NV12;
+ break;
+ case V4L2_PIX_FMT_QC08C:
+ hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC;
+ break;
+ case V4L2_PIX_FMT_QC10C:
+ hfi_colorformat = HFI_COLOR_FMT_TP10_UBWC;
+ break;
+ case V4L2_PIX_FMT_NV21:
+ hfi_colorformat = HFI_COLOR_FMT_NV21;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_colorformat;
+}
+
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
{
memset(buf, 0, sizeof(*buf));
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index bf18553..4276d6d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -81,6 +81,9 @@ enum hfi_packet_port_type {
u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
+u32 get_hfi_colorformat(u32 colorformat);
+u32 get_hfi_port(u32 plane);
+
int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 64067d5..7d16c96 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -5,10 +5,13 @@
#include <media/v4l2-event.h>
+#include "hfi_defines.h"
#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_ctrls.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
#include "iris_vdec.h"
static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
@@ -329,3 +332,154 @@ int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscri
return ret;
}
+
+int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
+{
+ const u32 *subcribe_prop = NULL;
+ u32 subscribe_prop_size = 0;
+ struct iris_core *core;
+ u32 payload[32] = {0};
+ u32 i;
+
+ core = inst->core;
+
+ payload[0] = HFI_MODE_PROPERTY;
+
+ if (plane == INPUT_MPLANE) {
+ subscribe_prop_size = core->platform_data->dec_input_prop_size;
+ subcribe_prop = core->platform_data->dec_input_prop;
+ } else if (plane == OUTPUT_MPLANE) {
+ if (inst->codec == H264) {
+ subscribe_prop_size = core->platform_data->dec_output_prop_size_avc;
+ subcribe_prop = core->platform_data->dec_output_prop_avc;
+ } else if (inst->codec == HEVC) {
+ subscribe_prop_size = core->platform_data->dec_output_prop_size_hevc;
+ subcribe_prop = core->platform_data->dec_output_prop_hevc;
+ } else if (inst->codec == VP9) {
+ subscribe_prop_size = core->platform_data->dec_output_prop_size_vp9;
+ subcribe_prop = core->platform_data->dec_output_prop_vp9;
+ } else {
+ return -EINVAL;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ for (i = 0; i < subscribe_prop_size; i++)
+ payload[i + 1] = subcribe_prop[i];
+
+ return iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ plane,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ (subscribe_prop_size + 1) * sizeof(u32));
+}
+
+static int vdec_set_colorformat(struct iris_inst *inst)
+{
+ u32 hfi_colorformat;
+ u32 pixelformat;
+
+ pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ hfi_colorformat = get_hfi_colorformat(pixelformat);
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_COLOR_FORMAT,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &hfi_colorformat,
+ sizeof(u32));
+}
+
+static int vdec_set_linear_stride_scanline(struct iris_inst *inst)
+{
+ u32 stride_y, scanline_y, stride_uv, scanline_uv;
+ u32 pixelformat;
+ u32 payload[2];
+
+ pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat;
+
+ if (!is_linear_colorformat(pixelformat))
+ return 0;
+
+ stride_y = inst->fmt_dst->fmt.pix_mp.width;
+ scanline_y = inst->fmt_dst->fmt.pix_mp.height;
+ stride_uv = stride_y;
+ scanline_uv = scanline_y / 2;
+
+ payload[0] = stride_y << 16 | scanline_y;
+ payload[1] = stride_uv << 16 | scanline_uv;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LINEAR_STRIDE_SCANLINE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ HFI_PAYLOAD_U64,
+ &payload,
+ sizeof(u64));
+}
+
+static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst)
+{
+ u32 meta_stride_y, meta_scanline_y, meta_stride_uv, meta_scanline_uv;
+ u32 stride_y, scanline_y, stride_uv, scanline_uv;
+ u32 pix_fmt, width, height;
+ u32 payload[4];
+
+ pix_fmt = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ width = inst->fmt_dst->fmt.pix_mp.width;
+ height = inst->fmt_dst->fmt.pix_mp.height;
+
+ if (is_linear_colorformat(pix_fmt))
+ return 0;
+
+ if (pix_fmt == V4L2_PIX_FMT_QC08C) {
+ stride_y = ALIGN(width, 128);
+ scanline_y = ALIGN(height, 32);
+ stride_uv = ALIGN(width, 128);
+ scanline_uv = ALIGN((height + 1) >> 1, 32);
+ meta_stride_y = ALIGN(DIV_ROUND_UP(width, 32), 64);
+ meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 8), 16);
+ meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64);
+ meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16);
+ } else {
+ stride_y = ALIGN(ALIGN(width, 192) * 4 / 3, 256);
+ scanline_y = ALIGN(height, 16);
+ stride_uv = ALIGN(ALIGN(width, 192) * 4 / 3, 256);
+ scanline_uv = ALIGN((height + 1) >> 1, 16);
+ meta_stride_y = ALIGN(DIV_ROUND_UP(width, 48), 64);
+ meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 4), 16);
+ meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64);
+ meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
+ }
+
+ payload[0] = stride_y << 16 | scanline_y;
+ payload[1] = stride_uv << 16 | scanline_uv;
+ payload[2] = meta_stride_y << 16 | meta_scanline_y;
+ payload[3] = meta_stride_uv << 16 | meta_scanline_uv;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_UBWC_STRIDE_SCANLINE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ sizeof(u32) * 4);
+}
+
+int vdec_set_output_property(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = vdec_set_colorformat(inst);
+ if (ret)
+ return ret;
+
+ ret = vdec_set_linear_stride_scanline(inst);
+ if (ret)
+ return ret;
+
+ return vdec_set_ubwc_stride_scanline(inst);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index a2f159d..6b0306c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -14,5 +14,7 @@ int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);
+int vdec_subscribe_property(struct iris_inst *inst, u32 plane);
+int vdec_set_output_property(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index abd11fa..fc12bde 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -206,6 +206,14 @@ struct platform_data {
u32 core_data_size;
struct plat_inst_cap *inst_cap_data;
u32 inst_cap_data_size;
+ const u32 *dec_input_prop;
+ unsigned int dec_input_prop_size;
+ const u32 *dec_output_prop_avc;
+ unsigned int dec_output_prop_size_avc;
+ const u32 *dec_output_prop_hevc;
+ unsigned int dec_output_prop_size_hevc;
+ const u32 *dec_output_prop_vp9;
+ unsigned int dec_output_prop_size_vp9;
};
int init_platform(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 85bc677..6a4bfa3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -382,6 +382,26 @@ static struct format_capability format_data_sm8550 = {
.color_format_info_size = ARRAY_SIZE(color_format_data_sm8550),
};
+static const u32 sm8550_vdec_input_properties[] = {
+ HFI_PROP_NO_OUTPUT,
+};
+
+static const u32 sm8550_vdec_output_properties_avc[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_DPB_LIST,
+ HFI_PROP_CABAC_SESSION,
+};
+
+static const u32 sm8550_vdec_output_properties_hevc[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_DPB_LIST,
+};
+
+static const u32 sm8550_vdec_output_properties_vp9[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_DPB_LIST,
+};
+
struct platform_data sm8550_data = {
.bus_tbl = sm8550_bus_table,
.bus_tbl_size = ARRAY_SIZE(sm8550_bus_table),
@@ -409,4 +429,13 @@ struct platform_data sm8550_data = {
.inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550),
.ubwc_config = ubwc_config_sm8550,
.format_data = &format_data_sm8550,
+
+ .dec_input_prop = sm8550_vdec_input_properties,
+ .dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_input_properties),
+ .dec_output_prop_avc = sm8550_vdec_output_properties_avc,
+ .dec_output_prop_size_avc = ARRAY_SIZE(sm8550_vdec_output_properties_avc),
+ .dec_output_prop_hevc = sm8550_vdec_output_properties_hevc,
+ .dec_output_prop_size_hevc = ARRAY_SIZE(sm8550_vdec_output_properties_hevc),
+ .dec_output_prop_vp9 = sm8550_vdec_output_properties_vp9,
+ .dec_output_prop_size_vp9 = ARRAY_SIZE(sm8550_vdec_output_properties_vp9),
};
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 25/34] media: iris: subscribe src change and handle firmware responses
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (23 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 24/34] media: iris: subscribe input and output properties to firmware Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 26/34] media: iris: implement vb2 streaming ops on capture and output planes Dikshita Agarwal
` (15 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
From: Vikash Garodia <quic_vgarodia@quicinc.com>
Driver can subscribe to different bitstream parameters
for which it expects a response from firmware.
These are vital bitstream parameters which requires
reconfiguration of buffers or sending updated info
to client. Incase of any change in these subscribed
parameters, firmware sends a response, which is then
sent to client as source change event.
HFI_CMD_SUBSCRIBE_MODE with HFI_MODE_PORT_SETTINGS_CHANGE
- to subscribe for different bitstream parameters.
Below properties are set to firmware as part of this:
HFI_PROP_BITSTREAM_RESOLUTION
HFI_PROP_CROP_OFFSETS
HFI_PROP_LUMA_CHROMA_BIT_DEPTH
HFI_PROP_CODED_FRAMES
HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT
HFI_PROP_PIC_ORDER_CNT_TYPE
HFI_PROP_SIGNAL_COLOR_INFO
HFI_PROP_PROFILE
HFI_PROP_LEVEL
HFI_PROP_TIER
Add the handling of different responses
from firmware. Below are different types of responses:
System responses:
- System error - response in case of system error occurred.
- System property - response for system level property - sys image version.
- System init - response for video core init completion.
Session responses:
- Session error - response in case of any session level error.
- Session property - response for all session properties.
HFI_PROP_BITSTREAM_RESOLUTION
HFI_PROP_CROP_OFFSETS
HFI_PROP_LUMA_CHROMA_BIT_DEPTH
HFI_PROP_CODED_FRAMES
HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT
HFI_PROP_PIC_ORDER_CNT_TYPE
HFI_PROP_SIGNAL_COLOR_INFO
HFI_PROP_PROFILE
HFI_PROP_LEVEL
HFI_PROP_TIER
- Session command - response for session level commands.
HFI_CMD_OPEN - response for open competition.
HFI_CMD_CLOSE - response for open competition.
HFI_CMD_SETTINGS_CHANGE - response for change in subscribe parameters.
HFI_CMD_SUBSCRIBE_MODE - response for info on subscribed properties.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 114 +++-
.../media/platform/qcom/vcodec/iris/iris_common.h | 25 +-
.../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 29 +-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 195 +++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 11 +-
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 364 ++++++++++++-
.../platform/qcom/vcodec/iris/iris_instance.h | 8 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 580 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 4 +
.../platform/qcom/vcodec/iris/platform_common.h | 6 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 44 ++
13 files changed, 1375 insertions(+), 11 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index a6078a5..0ef6bad 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -10,6 +10,7 @@
#define HFI_VIDEO_ARCH_LX 0x1
+#define HFI_CMD_BEGIN 0x01000000
#define HFI_CMD_INIT 0x01000001
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
@@ -31,7 +32,18 @@ enum hfi_property_mode_type {
HFI_MODE_PROPERTY = 0x00000002,
};
-#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B
+#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B
+#define HFI_CMD_SETTINGS_CHANGE 0x0100000C
+#define HFI_CMD_END 0x01FFFFFF
+
+#define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000
+#define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff
+
+#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001
+
+#define HFI_PROP_BEGIN 0x03000000
+
+#define HFI_PROP_IMAGE_VERSION 0x03000001
enum hfi_color_format {
HFI_COLOR_FMT_OPAQUE = 0,
@@ -46,8 +58,12 @@ enum hfi_color_format {
#define HFI_PROP_COLOR_FORMAT 0x03000101
+#define HFI_PROP_BITSTREAM_RESOLUTION 0x03000103
+
#define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104
+#define HFI_PROP_CROP_OFFSETS 0x03000105
+
#define HFI_PROP_PROFILE 0x03000107
#define HFI_PROP_LEVEL 0x03000108
@@ -68,13 +84,83 @@ enum hfi_color_format {
#define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123
+#define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124
+
#define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128
+#define HFI_PROP_QUALITY_MODE 0x03000148
+
+enum hfi_color_primaries {
+ HFI_PRIMARIES_RESERVED = 0,
+ HFI_PRIMARIES_BT709 = 1,
+ HFI_PRIMARIES_UNSPECIFIED = 2,
+ HFI_PRIMARIES_BT470_SYSTEM_M = 4,
+ HFI_PRIMARIES_BT470_SYSTEM_BG = 5,
+ HFI_PRIMARIES_BT601_525 = 6,
+ HFI_PRIMARIES_SMPTE_ST240M = 7,
+ HFI_PRIMARIES_GENERIC_FILM = 8,
+ HFI_PRIMARIES_BT2020 = 9,
+ HFI_PRIMARIES_SMPTE_ST428_1 = 10,
+ HFI_PRIMARIES_SMPTE_RP431_2 = 11,
+ HFI_PRIMARIES_SMPTE_EG431_1 = 12,
+ HFI_PRIMARIES_SMPTE_EBU_TECH = 22,
+};
+
+enum hfi_transfer_characteristics {
+ HFI_TRANSFER_RESERVED = 0,
+ HFI_TRANSFER_BT709 = 1,
+ HFI_TRANSFER_UNSPECIFIED = 2,
+ HFI_TRANSFER_BT470_SYSTEM_M = 4,
+ HFI_TRANSFER_BT470_SYSTEM_BG = 5,
+ HFI_TRANSFER_BT601_525_OR_625 = 6,
+ HFI_TRANSFER_SMPTE_ST240M = 7,
+ HFI_TRANSFER_LINEAR = 8,
+ HFI_TRANSFER_LOG_100_1 = 9,
+ HFI_TRANSFER_LOG_SQRT = 10,
+ HFI_TRANSFER_XVYCC = 11,
+ HFI_TRANSFER_BT1361_0 = 12,
+ HFI_TRANSFER_SRGB_SYCC = 13,
+ HFI_TRANSFER_BT2020_14 = 14,
+ HFI_TRANSFER_BT2020_15 = 15,
+ HFI_TRANSFER_SMPTE_ST2084_PQ = 16,
+ HFI_TRANSFER_SMPTE_ST428_1 = 17,
+ HFI_TRANSFER_BT2100_2_HLG = 18,
+};
+
+enum hfi_matrix_coefficients {
+ HFI_MATRIX_COEFF_SRGB_SMPTE_ST428_1 = 0,
+ HFI_MATRIX_COEFF_BT709 = 1,
+ HFI_MATRIX_COEFF_UNSPECIFIED = 2,
+ HFI_MATRIX_COEFF_RESERVED = 3,
+ HFI_MATRIX_COEFF_FCC_TITLE_47 = 4,
+ HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625 = 5,
+ HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625 = 6,
+ HFI_MATRIX_COEFF_SMPTE_ST240 = 7,
+ HFI_MATRIX_COEFF_YCGCO = 8,
+ HFI_MATRIX_COEFF_BT2020_NON_CONSTANT = 9,
+ HFI_MATRIX_COEFF_BT2020_CONSTANT = 10,
+ HFI_MATRIX_COEFF_SMPTE_ST2085 = 11,
+ HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_NON_CONSTANT = 12,
+ HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_CONSTANT = 13,
+ HFI_MATRIX_COEFF_BT2100 = 14,
+};
+
+#define HFI_PROP_SIGNAL_COLOR_INFO 0x03000155
+
#define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b
+enum hfi_picture_type {
+ HFI_PICTURE_IDR = 0x00000001,
+ HFI_PICTURE_P = 0x00000002,
+ HFI_PICTURE_B = 0x00000004,
+ HFI_PICTURE_I = 0x00000008,
+ HFI_PICTURE_CRA = 0x00000010,
+ HFI_PICTURE_BLA = 0x00000020,
+};
+
#define HFI_PROP_PICTURE_TYPE 0x03000162
-#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
+#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
@@ -86,7 +172,29 @@ enum hfi_color_format {
#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
-#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+#define HFI_PROP_END 0x03FFFFFF
+
+#define HFI_SESSION_ERROR_BEGIN 0x04000000
+
+#define HFI_ERROR_UNKNOWN_SESSION 0x04000001
+
+#define HFI_ERROR_MAX_SESSIONS 0x04000002
+
+#define HFI_ERROR_FATAL 0x04000003
+
+#define HFI_ERROR_INVALID_STATE 0x04000004
+
+#define HFI_ERROR_INSUFFICIENT_RESOURCES 0x04000005
+
+#define HFI_ERROR_BUFFER_NOT_SET 0x04000006
+
+#define HFI_SESSION_ERROR_END 0x04FFFFFF
+
+#define HFI_SYSTEM_ERROR_BEGIN 0x05000000
+
+#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+
+#define HFI_SYSTEM_ERROR_END 0x05FFFFFF
struct hfi_debug_header {
u32 size;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 6b771f8..0fbd139 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -8,7 +8,6 @@
#include <media/v4l2-dev.h>
#include <media/v4l2-device.h>
#include <media/v4l2-event.h>
-#include <media/v4l2-ioctl.h>
#include <media/videobuf2-v4l2.h>
#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
@@ -16,6 +15,7 @@
#define DEFAULT_WIDTH 320
#define DEFAULT_HEIGHT 240
#define DEFAULT_BSE_VPP_DELAY 2
+#define IRIS_VERSION_LENGTH 128
#define MAX_EVENTS 30
@@ -23,6 +23,9 @@
#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
+#define MAX_DPB_LIST_ARRAY_SIZE (16 * 4)
+#define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4)
+
enum codec_type {
H264 = BIT(0),
HEVC = BIT(1),
@@ -95,4 +98,24 @@ struct iris_buffer {
struct dma_buf_attachment *attach;
};
+struct subscription_params {
+ u32 bitstream_resolution;
+ u32 crop_offsets[2];
+ u32 bit_depth;
+ u32 coded_frames;
+ u32 fw_min_count;
+ u32 pic_order_cnt;
+ u32 color_info;
+ u32 profile;
+ u32 level;
+ u32 tier;
+};
+
+struct iris_hfi_frame_info {
+ u32 picture_type;
+ u32 no_output;
+ u32 data_corrupt;
+ u32 overflow;
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 154991c..6452ea7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -57,6 +57,7 @@
* @inst_caps: a pointer to supported instance capabilities
* @instances: a list_head of all instances
* @intr_status: interrupt status
+ * @fw_version: firmware version
*/
struct iris_core {
@@ -99,6 +100,7 @@ struct iris_core {
struct plat_inst_caps *inst_caps;
struct list_head instances;
u32 intr_status;
+ char fw_version[IRIS_VERSION_LENGTH];
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index ff44cda..4cad673 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -3,6 +3,8 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include "hfi_defines.h"
+#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
@@ -66,7 +68,7 @@ int get_mbpf(struct iris_inst *inst)
return NUM_MBS_PER_FRAME(height, width);
}
-bool is_linear_colorformat(u32 colorformat)
+inline bool is_linear_colorformat(u32 colorformat)
{
return colorformat == V4L2_PIX_FMT_NV12 || colorformat == V4L2_PIX_FMT_NV21;
}
@@ -350,3 +352,28 @@ int check_session_supported(struct iris_inst *inst)
return ret;
}
+
+int signal_session_msg_receipt(struct iris_inst *inst,
+ enum signal_session_response cmd)
+{
+ if (cmd < MAX_SIGNAL)
+ complete(&inst->completions[cmd]);
+
+ return 0;
+}
+
+struct iris_inst *to_instance(struct iris_core *core, u32 session_id)
+{
+ struct iris_inst *inst = NULL;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(inst, &core->instances, list) {
+ if (inst->session_id == session_id) {
+ mutex_unlock(&core->lock);
+ return inst;
+ }
+ }
+ mutex_unlock(&core->lock);
+
+ return NULL;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index fe85d23..cb22adf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -31,6 +31,10 @@ bool is_linear_colorformat(u32 colorformat);
bool is_10bit_colorformat(enum colorformat_type colorformat);
bool is_8bit_colorformat(enum colorformat_type colorformat);
bool is_split_mode_enabled(struct iris_inst *inst);
+int signal_session_msg_receipt(struct iris_inst *inst,
+ enum signal_session_response cmd);
+struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
+
u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index d4cdfcf..dc7157d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -100,6 +100,201 @@ u32 get_hfi_colorformat(u32 colorformat)
return hfi_colorformat;
}
+u32 get_hfi_color_primaries(u32 primaries)
+{
+ u32 hfi_primaries = HFI_PRIMARIES_RESERVED;
+
+ switch (primaries) {
+ case V4L2_COLORSPACE_DEFAULT:
+ hfi_primaries = HFI_PRIMARIES_RESERVED;
+ break;
+ case V4L2_COLORSPACE_REC709:
+ hfi_primaries = HFI_PRIMARIES_BT709;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_M:
+ hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_M;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_BG:
+ hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_BG;
+ break;
+ case V4L2_COLORSPACE_SMPTE170M:
+ hfi_primaries = HFI_PRIMARIES_BT601_525;
+ break;
+ case V4L2_COLORSPACE_SMPTE240M:
+ hfi_primaries = HFI_PRIMARIES_SMPTE_ST240M;
+ break;
+ case V4L2_COLORSPACE_BT2020:
+ hfi_primaries = HFI_PRIMARIES_BT2020;
+ break;
+ case V4L2_COLORSPACE_DCI_P3:
+ hfi_primaries = HFI_PRIMARIES_SMPTE_RP431_2;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_primaries;
+}
+
+u32 get_hfi_transfer_char(u32 characterstics)
+{
+ u32 hfi_characterstics = HFI_TRANSFER_RESERVED;
+
+ switch (characterstics) {
+ case V4L2_XFER_FUNC_DEFAULT:
+ hfi_characterstics = HFI_TRANSFER_RESERVED;
+ break;
+ case V4L2_XFER_FUNC_709:
+ hfi_characterstics = HFI_TRANSFER_BT709;
+ break;
+ case V4L2_XFER_FUNC_SMPTE240M:
+ hfi_characterstics = HFI_TRANSFER_SMPTE_ST240M;
+ break;
+ case V4L2_XFER_FUNC_SRGB:
+ hfi_characterstics = HFI_TRANSFER_SRGB_SYCC;
+ break;
+ case V4L2_XFER_FUNC_SMPTE2084:
+ hfi_characterstics = HFI_TRANSFER_SMPTE_ST2084_PQ;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_characterstics;
+}
+
+u32 get_hfi_matrix_coefficients(u32 coefficients)
+{
+ u32 hfi_coefficients = HFI_MATRIX_COEFF_RESERVED;
+
+ switch (coefficients) {
+ case V4L2_YCBCR_ENC_DEFAULT:
+ hfi_coefficients = HFI_MATRIX_COEFF_RESERVED;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT709;
+ break;
+ case V4L2_YCBCR_ENC_XV709:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT709;
+ break;
+ case V4L2_YCBCR_ENC_XV601:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625;
+ break;
+ case V4L2_YCBCR_ENC_601:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625;
+ break;
+ case V4L2_YCBCR_ENC_SMPTE240M:
+ hfi_coefficients = HFI_MATRIX_COEFF_SMPTE_ST240;
+ break;
+ case V4L2_YCBCR_ENC_BT2020:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT2020_NON_CONSTANT;
+ break;
+ case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT2020_CONSTANT;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_coefficients;
+}
+
+u32 get_v4l2_color_primaries(u32 hfi_primaries)
+{
+ u32 primaries = V4L2_COLORSPACE_DEFAULT;
+
+ switch (hfi_primaries) {
+ case HFI_PRIMARIES_RESERVED:
+ primaries = V4L2_COLORSPACE_DEFAULT;
+ break;
+ case HFI_PRIMARIES_BT709:
+ primaries = V4L2_COLORSPACE_REC709;
+ break;
+ case HFI_PRIMARIES_BT470_SYSTEM_M:
+ primaries = V4L2_COLORSPACE_470_SYSTEM_M;
+ break;
+ case HFI_PRIMARIES_BT470_SYSTEM_BG:
+ primaries = V4L2_COLORSPACE_470_SYSTEM_BG;
+ break;
+ case HFI_PRIMARIES_BT601_525:
+ primaries = V4L2_COLORSPACE_SMPTE170M;
+ break;
+ case HFI_PRIMARIES_SMPTE_ST240M:
+ primaries = V4L2_COLORSPACE_SMPTE240M;
+ break;
+ case HFI_PRIMARIES_BT2020:
+ primaries = V4L2_COLORSPACE_BT2020;
+ break;
+ case V4L2_COLORSPACE_DCI_P3:
+ primaries = HFI_PRIMARIES_SMPTE_RP431_2;
+ break;
+ default:
+ break;
+ }
+
+ return primaries;
+}
+
+u32 get_v4l2_transfer_char(u32 hfi_characterstics)
+{
+ u32 characterstics = V4L2_XFER_FUNC_DEFAULT;
+
+ switch (hfi_characterstics) {
+ case HFI_TRANSFER_RESERVED:
+ characterstics = V4L2_XFER_FUNC_DEFAULT;
+ break;
+ case HFI_TRANSFER_BT709:
+ characterstics = V4L2_XFER_FUNC_709;
+ break;
+ case HFI_TRANSFER_SMPTE_ST240M:
+ characterstics = V4L2_XFER_FUNC_SMPTE240M;
+ break;
+ case HFI_TRANSFER_SRGB_SYCC:
+ characterstics = V4L2_XFER_FUNC_SRGB;
+ break;
+ case HFI_TRANSFER_SMPTE_ST2084_PQ:
+ characterstics = V4L2_XFER_FUNC_SMPTE2084;
+ break;
+ default:
+ break;
+ }
+
+ return characterstics;
+}
+
+u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients)
+{
+ u32 coefficients = V4L2_YCBCR_ENC_DEFAULT;
+
+ switch (hfi_coefficients) {
+ case HFI_MATRIX_COEFF_RESERVED:
+ coefficients = V4L2_YCBCR_ENC_DEFAULT;
+ break;
+ case HFI_MATRIX_COEFF_BT709:
+ coefficients = V4L2_YCBCR_ENC_709;
+ break;
+ case HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625:
+ coefficients = V4L2_YCBCR_ENC_XV601;
+ break;
+ case HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625:
+ coefficients = V4L2_YCBCR_ENC_601;
+ break;
+ case HFI_MATRIX_COEFF_SMPTE_ST240:
+ coefficients = V4L2_YCBCR_ENC_SMPTE240M;
+ break;
+ case HFI_MATRIX_COEFF_BT2020_NON_CONSTANT:
+ coefficients = V4L2_YCBCR_ENC_BT2020;
+ break;
+ case HFI_MATRIX_COEFF_BT2020_CONSTANT:
+ coefficients = V4L2_YCBCR_ENC_BT2020_CONST_LUM;
+ break;
+ default:
+ break;
+ }
+
+ return coefficients;
+}
+
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
{
memset(buf, 0, sizeof(*buf));
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 4276d6d..f813116 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -80,10 +80,17 @@ enum hfi_packet_port_type {
};
u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
-int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
-u32 get_hfi_colorformat(u32 colorformat);
u32 get_hfi_port(u32 plane);
+u32 get_hfi_colorformat(u32 colorformat);
+u32 get_hfi_color_primaries(u32 primaries);
+u32 get_hfi_transfer_char(u32 characterstics);
+u32 get_hfi_matrix_coefficients(u32 coefficients);
+u32 get_v4l2_color_primaries(u32 hfi_primaries);
+u32 get_v4l2_transfer_char(u32 hfi_characterstics);
+u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients);
+int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
+
int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 829f3f6..4ca9314 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -4,8 +4,27 @@
*/
#include "hfi_defines.h"
+#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "iris_hfi_response.h"
+#include "iris_vdec.h"
+
+struct iris_core_hfi_range {
+ u32 begin;
+ u32 end;
+ int (*handle)(struct iris_core *core, struct hfi_packet *pkt);
+};
+
+struct iris_inst_hfi_range {
+ u32 begin;
+ u32 end;
+ int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
+};
+
+struct iris_hfi_packet_handle {
+ enum hfi_buffer_type type;
+ int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
+};
static void print_sfr_message(struct iris_core *core)
{
@@ -25,8 +44,7 @@ static void print_sfr_message(struct iris_core *core)
}
}
-static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt,
- u32 core_resp_pkt_size)
+static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, u32 core_resp_pkt_size)
{
u32 response_pkt_size = 0;
u8 *response_limit;
@@ -52,8 +70,7 @@ static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt,
return 0;
}
-static int validate_hdr_packet(struct iris_core *core,
- struct hfi_header *hdr)
+static int validate_hdr_packet(struct iris_core *core, struct hfi_header *hdr)
{
struct hfi_packet *packet;
int i, ret = 0;
@@ -76,6 +93,45 @@ static int validate_hdr_packet(struct iris_core *core,
return ret;
}
+static int handle_session_error(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ struct iris_core *core;
+ char *error;
+
+ core = inst->core;
+
+ switch (pkt->type) {
+ case HFI_ERROR_MAX_SESSIONS:
+ error = "exceeded max sessions";
+ break;
+ case HFI_ERROR_UNKNOWN_SESSION:
+ error = "unknown session id";
+ break;
+ case HFI_ERROR_INVALID_STATE:
+ error = "invalid operation for current state";
+ break;
+ case HFI_ERROR_INSUFFICIENT_RESOURCES:
+ error = "insufficient resources";
+ break;
+ case HFI_ERROR_BUFFER_NOT_SET:
+ error = "internal buffers not set";
+ break;
+ case HFI_ERROR_FATAL:
+ error = "fatal error";
+ break;
+ default:
+ error = "unknown";
+ break;
+ }
+
+ dev_err(core->dev, "session error received %#x: %s\n",
+ pkt->type, error);
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return 0;
+}
+
static int handle_system_error(struct iris_core *core,
struct hfi_packet *pkt)
{
@@ -86,6 +142,301 @@ static int handle_system_error(struct iris_core *core,
return 0;
}
+static int handle_system_init(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ if (!(pkt->flags & HFI_FW_FLAGS_SUCCESS))
+ return 0;
+
+ mutex_lock(&core->lock);
+ if (pkt->packet_id != core->sys_init_id)
+ goto unlock;
+
+ iris_change_core_state(core, IRIS_CORE_INIT);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return 0;
+}
+
+static int handle_session_close(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ signal_session_msg_receipt(inst, SIGNAL_CMD_CLOSE);
+
+ return 0;
+}
+
+static int handle_src_change(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+
+ if (pkt->port == HFI_PORT_BITSTREAM)
+ ret = vdec_src_change(inst);
+ else if (pkt->port == HFI_PORT_RAW)
+ ret = 0;
+
+ if (ret)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return ret;
+}
+
+static int handle_session_command(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int i, ret = 0;
+ static const struct iris_hfi_packet_handle hfi_pkt_handle[] = {
+ {HFI_CMD_OPEN, NULL },
+ {HFI_CMD_CLOSE, handle_session_close },
+ {HFI_CMD_SETTINGS_CHANGE, handle_src_change },
+ {HFI_CMD_SUBSCRIBE_MODE, NULL },
+ };
+
+ for (i = 0; i < ARRAY_SIZE(hfi_pkt_handle); i++) {
+ if (hfi_pkt_handle[i].type == pkt->type) {
+ if (hfi_pkt_handle[i].handle) {
+ ret = hfi_pkt_handle[i].handle(inst, pkt);
+ if (ret)
+ return ret;
+ }
+ break;
+ }
+ }
+
+ if (i == ARRAY_SIZE(hfi_pkt_handle))
+ return -EINVAL;
+
+ return ret;
+}
+
+static int handle_dpb_list_property(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ u8 *payload_start;
+ u32 payload_size;
+
+ payload_size = pkt->size - sizeof(*pkt);
+ payload_start = (u8 *)((u8 *)pkt + sizeof(*pkt));
+ memset(inst->dpb_list_payload, 0, MAX_DPB_LIST_ARRAY_SIZE);
+
+ if (payload_size > MAX_DPB_LIST_PAYLOAD_SIZE) {
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ return -EINVAL;
+ }
+
+ memcpy(inst->dpb_list_payload, payload_start, payload_size);
+
+ return 0;
+}
+
+static int handle_session_property(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ u32 *payload_ptr = NULL;
+ int ret = 0;
+
+ if (pkt->port != HFI_PORT_BITSTREAM)
+ return 0;
+
+ if (pkt->flags & HFI_FW_FLAGS_INFORMATION)
+ return 0;
+
+ payload_ptr = (u32 *)((u8 *)pkt + sizeof(*pkt));
+ if (!payload_ptr)
+ return -EINVAL;
+
+ switch (pkt->type) {
+ case HFI_PROP_BITSTREAM_RESOLUTION:
+ inst->src_subcr_params.bitstream_resolution = payload_ptr[0];
+ break;
+ case HFI_PROP_CROP_OFFSETS:
+ inst->src_subcr_params.crop_offsets[0] = payload_ptr[0];
+ inst->src_subcr_params.crop_offsets[1] = payload_ptr[1];
+ break;
+ case HFI_PROP_LUMA_CHROMA_BIT_DEPTH:
+ inst->src_subcr_params.bit_depth = payload_ptr[0];
+ break;
+ case HFI_PROP_CODED_FRAMES:
+ inst->src_subcr_params.coded_frames = payload_ptr[0];
+ break;
+ case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT:
+ inst->src_subcr_params.fw_min_count = payload_ptr[0];
+ break;
+ case HFI_PROP_PIC_ORDER_CNT_TYPE:
+ inst->src_subcr_params.pic_order_cnt = payload_ptr[0];
+ break;
+ case HFI_PROP_SIGNAL_COLOR_INFO:
+ inst->src_subcr_params.color_info = payload_ptr[0];
+ break;
+ case HFI_PROP_PROFILE:
+ inst->src_subcr_params.profile = payload_ptr[0];
+ break;
+ case HFI_PROP_LEVEL:
+ inst->src_subcr_params.level = payload_ptr[0];
+ break;
+ case HFI_PROP_TIER:
+ inst->src_subcr_params.tier = payload_ptr[0];
+ break;
+ case HFI_PROP_PICTURE_TYPE:
+ inst->hfi_frame_info.picture_type = payload_ptr[0];
+ break;
+ case HFI_PROP_DPB_LIST:
+ ret = handle_dpb_list_property(inst, pkt);
+ if (ret)
+ break;
+ break;
+ case HFI_PROP_NO_OUTPUT:
+ inst->hfi_frame_info.no_output = 1;
+ break;
+ case HFI_PROP_QUALITY_MODE:
+ case HFI_PROP_STAGE:
+ case HFI_PROP_PIPE:
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+static int handle_image_version_property(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ u8 *str_image_version;
+ u32 req_bytes;
+ u32 i = 0;
+
+ req_bytes = pkt->size - sizeof(*pkt);
+ if (req_bytes < IRIS_VERSION_LENGTH - 1)
+ return -EINVAL;
+
+ str_image_version = (u8 *)pkt + sizeof(*pkt);
+
+ for (i = 0; i < IRIS_VERSION_LENGTH - 1; i++) {
+ if (str_image_version[i] != '\0')
+ core->fw_version[i] = str_image_version[i];
+ else
+ core->fw_version[i] = ' ';
+ }
+ core->fw_version[i] = '\0';
+
+ return 0;
+}
+
+static int handle_system_property(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+
+ switch (pkt->type) {
+ case HFI_PROP_IMAGE_VERSION:
+ ret = handle_image_version_property(core, pkt);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+static int handle_system_response(struct iris_core *core,
+ struct hfi_header *hdr)
+{
+ struct hfi_packet *packet;
+ u8 *pkt, *start_pkt;
+ int ret = 0;
+ int i, j;
+ static const struct iris_core_hfi_range be[] = {
+ {HFI_SYSTEM_ERROR_BEGIN, HFI_SYSTEM_ERROR_END, handle_system_error },
+ {HFI_PROP_BEGIN, HFI_PROP_END, handle_system_property },
+ {HFI_CMD_BEGIN, HFI_CMD_END, handle_system_init },
+ };
+
+ start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+ for (i = 0; i < ARRAY_SIZE(be); i++) {
+ pkt = start_pkt;
+ for (j = 0; j < hdr->num_packets; j++) {
+ packet = (struct hfi_packet *)pkt;
+ if (packet->flags & HFI_FW_FLAGS_SYSTEM_ERROR) {
+ ret = handle_system_error(core, packet);
+ return ret;
+ }
+
+ if (packet->type > be[i].begin && packet->type < be[i].end) {
+ ret = be[i].handle(core, packet);
+ if (ret)
+ return ret;
+
+ if (packet->type > HFI_SYSTEM_ERROR_BEGIN &&
+ packet->type < HFI_SYSTEM_ERROR_END)
+ return 0;
+ }
+ pkt += packet->size;
+ }
+ }
+
+ return ret;
+}
+
+static int handle_session_response(struct iris_core *core,
+ struct hfi_header *hdr)
+{
+ struct hfi_packet *packet;
+ struct iris_inst *inst;
+ u8 *pkt, *start_pkt;
+ int ret = 0;
+ int i, j;
+ static const struct iris_inst_hfi_range be[] = {
+ {HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, handle_session_error },
+ {HFI_PROP_BEGIN, HFI_PROP_END, handle_session_property },
+ {HFI_CMD_BEGIN, HFI_CMD_END, handle_session_command },
+ };
+
+ inst = to_instance(core, hdr->session_id);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info));
+
+ pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+ for (i = 0; i < hdr->num_packets; i++) {
+ packet = (struct hfi_packet *)pkt;
+ if (packet->type == HFI_CMD_SETTINGS_CHANGE) {
+ if (packet->port == HFI_PORT_BITSTREAM) {
+ vdec_init_src_change_param(inst);
+ break;
+ }
+ }
+ pkt += packet->size;
+ }
+
+ start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+ for (i = 0; i < ARRAY_SIZE(be); i++) {
+ pkt = start_pkt;
+ for (j = 0; j < hdr->num_packets; j++) {
+ packet = (struct hfi_packet *)pkt;
+ if (packet->flags & HFI_FW_FLAGS_SESSION_ERROR)
+ handle_session_error(inst, packet);
+
+ if (packet->type > be[i].begin && packet->type < be[i].end) {
+ ret = be[i].handle(inst, packet);
+ if (ret)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ }
+ pkt += packet->size;
+ }
+ }
+
+ memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info));
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static int handle_response(struct iris_core *core, void *response)
{
struct hfi_header *hdr;
@@ -96,6 +447,11 @@ static int handle_response(struct iris_core *core, void *response)
if (ret)
return handle_system_error(core, NULL);
+ if (!hdr->session_id)
+ return handle_system_response(core, hdr);
+ else
+ return handle_session_response(core, hdr);
+
return ret;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 365f844..4f51d68 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -40,6 +40,10 @@
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
* @ipsc_properties_set: boolean to set ipsc properties to fw
+ * @hfi_frame_info: structure of frame info
+ * @src_subcr_params: subscription params to fw on input port
+ * @dst_subcr_params: subscription params to fw on output port
+ * @dpb_list_payload: array of dpb buffers
*/
struct iris_inst {
@@ -66,6 +70,10 @@ struct iris_inst {
u32 fw_min_count;
enum iris_inst_state state;
bool ipsc_properties_set;
+ struct iris_hfi_frame_info hfi_frame_info;
+ struct subscription_params src_subcr_params;
+ struct subscription_params dst_subcr_params;
+ u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE];
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 7d16c96..ac47fc0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -14,6 +14,13 @@
#include "iris_hfi_packet.h"
#include "iris_vdec.h"
+#define UNSPECIFIED_COLOR_FORMAT 5
+
+struct vdec_prop_type_handle {
+ u32 type;
+ int (*handle)(struct iris_inst *inst);
+};
+
static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
{
bool session_init = false;
@@ -376,6 +383,455 @@ int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
(subscribe_prop_size + 1) * sizeof(u32));
}
+static int vdec_set_bitstream_resolution(struct iris_inst *inst)
+{
+ u32 resolution;
+
+ resolution = inst->fmt_src->fmt.pix_mp.width << 16 |
+ inst->fmt_src->fmt.pix_mp.height;
+ inst->src_subcr_params.bitstream_resolution = resolution;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &resolution,
+ sizeof(u32));
+}
+
+static int vdec_set_crop_offsets(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ u32 payload[2] = {0};
+
+ left_offset = inst->crop.left;
+ top_offset = inst->crop.top;
+ right_offset = (inst->fmt_src->fmt.pix_mp.width -
+ inst->crop.width);
+ bottom_offset = (inst->fmt_src->fmt.pix_mp.height -
+ inst->crop.height);
+
+ payload[0] = left_offset << 16 | top_offset;
+ payload[1] = right_offset << 16 | bottom_offset;
+ inst->src_subcr_params.crop_offsets[0] = payload[0];
+ inst->src_subcr_params.crop_offsets[1] = payload[1];
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &payload,
+ sizeof(u64));
+}
+
+static int vdec_set_bit_depth(struct iris_inst *inst)
+{
+ u32 bitdepth = 8 << 16 | 8;
+ u32 pix_fmt;
+
+ pix_fmt = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ if (is_10bit_colorformat(pix_fmt))
+ bitdepth = 10 << 16 | 10;
+
+ inst->src_subcr_params.bit_depth = bitdepth;
+ inst->cap[BIT_DEPTH].value = bitdepth;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &bitdepth,
+ sizeof(u32));
+}
+
+static int vdec_set_coded_frames(struct iris_inst *inst)
+{
+ u32 coded_frames = 0;
+
+ if (inst->cap[CODED_FRAMES].value == CODED_FRAMES_PROGRESSIVE)
+ coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG;
+ inst->src_subcr_params.coded_frames = coded_frames;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CODED_FRAMES,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &coded_frames,
+ sizeof(u32));
+}
+
+static int vdec_set_min_output_count(struct iris_inst *inst)
+{
+ u32 min_output;
+
+ min_output = inst->buffers.output.min_count;
+ inst->src_subcr_params.fw_min_count = min_output;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &min_output,
+ sizeof(u32));
+}
+
+static int vdec_set_picture_order_count(struct iris_inst *inst)
+{
+ u32 poc = 0;
+
+ inst->src_subcr_params.pic_order_cnt = poc;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_PIC_ORDER_CNT_TYPE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &poc,
+ sizeof(u32));
+}
+
+static int vdec_set_colorspace(struct iris_inst *inst)
+{
+ u32 video_signal_type_present_flag = 0, color_info = 0;
+ u32 matrix_coeff = HFI_MATRIX_COEFF_RESERVED;
+ u32 video_format = UNSPECIFIED_COLOR_FORMAT;
+ struct v4l2_pix_format_mplane *pixmp = NULL;
+ u32 full_range = V4L2_QUANTIZATION_DEFAULT;
+ u32 transfer_char = HFI_TRANSFER_RESERVED;
+ u32 colour_description_present_flag = 0;
+ u32 primaries = HFI_PRIMARIES_RESERVED;
+
+ int ret;
+
+ if (inst->codec == VP9)
+ return 0;
+
+ pixmp = &inst->fmt_src->fmt.pix_mp;
+ if (pixmp->colorspace != V4L2_COLORSPACE_DEFAULT ||
+ pixmp->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT ||
+ pixmp->xfer_func != V4L2_XFER_FUNC_DEFAULT) {
+ colour_description_present_flag = 1;
+ video_signal_type_present_flag = 1;
+ primaries = get_hfi_color_primaries(pixmp->colorspace);
+ matrix_coeff = get_hfi_matrix_coefficients(pixmp->ycbcr_enc);
+ transfer_char = get_hfi_transfer_char(pixmp->xfer_func);
+ }
+
+ if (pixmp->quantization != V4L2_QUANTIZATION_DEFAULT) {
+ video_signal_type_present_flag = 1;
+ full_range = pixmp->quantization ==
+ V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0;
+ }
+
+ color_info = (matrix_coeff & 0xFF) |
+ ((transfer_char << 8) & 0xFF00) |
+ ((primaries << 16) & 0xFF0000) |
+ ((colour_description_present_flag << 24) & 0x1000000) |
+ ((full_range << 25) & 0x2000000) |
+ ((video_format << 26) & 0x1C000000) |
+ ((video_signal_type_present_flag << 29) & 0x20000000);
+
+ inst->src_subcr_params.color_info = color_info;
+
+ ret = iris_hfi_set_property(inst,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &color_info,
+ sizeof(u32));
+
+ return ret;
+}
+
+static int vdec_set_profile(struct iris_inst *inst)
+{
+ u32 profile;
+
+ profile = inst->cap[PROFILE].value;
+ inst->src_subcr_params.profile = profile;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_PROFILE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &profile,
+ sizeof(u32));
+}
+
+static int vdec_set_level(struct iris_inst *inst)
+{
+ u32 level;
+
+ level = inst->cap[LEVEL].value;
+ inst->src_subcr_params.level = level;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LEVEL,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &level,
+ sizeof(u32));
+}
+
+static int vdec_set_tier(struct iris_inst *inst)
+{
+ u32 tier;
+
+ tier = inst->cap[HEVC_TIER].value;
+ inst->src_subcr_params.tier = tier;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_TIER,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &tier,
+ sizeof(u32));
+}
+
+int vdec_subscribe_src_change_param(struct iris_inst *inst)
+{
+ const u32 *src_change_param;
+ u32 src_change_param_size;
+ struct iris_core *core;
+ u32 payload[32] = {0};
+ int ret;
+ u32 i, j;
+
+ static const struct vdec_prop_type_handle prop_type_handle_arr[] = {
+ {HFI_PROP_BITSTREAM_RESOLUTION, vdec_set_bitstream_resolution },
+ {HFI_PROP_CROP_OFFSETS, vdec_set_crop_offsets },
+ {HFI_PROP_LUMA_CHROMA_BIT_DEPTH, vdec_set_bit_depth },
+ {HFI_PROP_CODED_FRAMES, vdec_set_coded_frames },
+ {HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, vdec_set_min_output_count },
+ {HFI_PROP_PIC_ORDER_CNT_TYPE, vdec_set_picture_order_count },
+ {HFI_PROP_SIGNAL_COLOR_INFO, vdec_set_colorspace },
+ {HFI_PROP_PROFILE, vdec_set_profile },
+ {HFI_PROP_LEVEL, vdec_set_level },
+ {HFI_PROP_TIER, vdec_set_tier },
+ };
+
+ core = inst->core;
+
+ payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
+ if (inst->codec == H264) {
+ src_change_param_size = core->platform_data->avc_subscribe_param_size;
+ src_change_param = core->platform_data->avc_subscribe_param;
+ } else if (inst->codec == HEVC) {
+ src_change_param_size = core->platform_data->hevc_subscribe_param_size;
+ src_change_param = core->platform_data->hevc_subscribe_param;
+ } else if (inst->codec == VP9) {
+ src_change_param_size = core->platform_data->vp9_subscribe_param_size;
+ src_change_param = core->platform_data->vp9_subscribe_param;
+ } else {
+ src_change_param = NULL;
+ return -EINVAL;
+ }
+
+ if (!src_change_param || !src_change_param_size)
+ return -EINVAL;
+
+ for (i = 0; i < src_change_param_size; i++)
+ payload[i + 1] = src_change_param[i];
+
+ ret = iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ INPUT_MPLANE,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ ((src_change_param_size + 1) * sizeof(u32)));
+ if (ret)
+ return ret;
+
+ for (i = 0; i < src_change_param_size; i++) {
+ for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) {
+ if (prop_type_handle_arr[j].type == src_change_param[i]) {
+ ret = prop_type_handle_arr[j].handle(inst);
+ if (ret)
+ return ret;
+ break;
+ }
+ }
+ }
+
+ return ret;
+}
+
+int vdec_init_src_change_param(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op;
+ u32 primaries, matrix_coeff, transfer_char;
+ struct subscription_params *subsc_params;
+ u32 colour_description_present_flag = 0;
+ u32 video_signal_type_present_flag = 0;
+ u32 full_range = 0, video_format = 0;
+
+ subsc_params = &inst->src_subcr_params;
+ pixmp_ip = &inst->fmt_src->fmt.pix_mp;
+ pixmp_op = &inst->fmt_dst->fmt.pix_mp;
+
+ subsc_params->bitstream_resolution =
+ pixmp_ip->width << 16 | pixmp_ip->height;
+
+ left_offset = inst->crop.left;
+ top_offset = inst->crop.top;
+ right_offset = (pixmp_ip->width - inst->crop.width);
+ bottom_offset = (pixmp_ip->height - inst->crop.height);
+ subsc_params->crop_offsets[0] =
+ left_offset << 16 | top_offset;
+ subsc_params->crop_offsets[1] =
+ right_offset << 16 | bottom_offset;
+
+ subsc_params->fw_min_count = inst->buffers.output.min_count;
+
+ primaries = get_hfi_color_primaries(pixmp_op->colorspace);
+ matrix_coeff = get_hfi_matrix_coefficients(pixmp_op->ycbcr_enc);
+ transfer_char = get_hfi_transfer_char(pixmp_op->xfer_func);
+ full_range = pixmp_op->quantization == V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0;
+ subsc_params->color_info =
+ (matrix_coeff & 0xFF) |
+ ((transfer_char << 8) & 0xFF00) |
+ ((primaries << 16) & 0xFF0000) |
+ ((colour_description_present_flag << 24) & 0x1000000) |
+ ((full_range << 25) & 0x2000000) |
+ ((video_format << 26) & 0x1C000000) |
+ ((video_signal_type_present_flag << 29) & 0x20000000);
+
+ subsc_params->profile = inst->cap[PROFILE].value;
+ subsc_params->level = inst->cap[LEVEL].value;
+ subsc_params->tier = inst->cap[HEVC_TIER].value;
+ subsc_params->pic_order_cnt = inst->cap[POC].value;
+ subsc_params->bit_depth = inst->cap[BIT_DEPTH].value;
+ if (inst->cap[CODED_FRAMES].value ==
+ CODED_FRAMES_PROGRESSIVE)
+ subsc_params->coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG;
+ else
+ subsc_params->coded_frames = 0;
+
+ return 0;
+}
+
+static int vdec_read_input_subcr_params(struct iris_inst *inst)
+{
+ struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op;
+ u32 primaries, matrix_coeff, transfer_char;
+ struct subscription_params subsc_params;
+ u32 colour_description_present_flag = 0;
+ u32 video_signal_type_present_flag = 0;
+ u32 full_range = 0;
+ u32 width, height;
+
+ subsc_params = inst->src_subcr_params;
+ pixmp_ip = &inst->fmt_src->fmt.pix_mp;
+ pixmp_op = &inst->fmt_dst->fmt.pix_mp;
+ width = (subsc_params.bitstream_resolution &
+ HFI_BITMASK_BITSTREAM_WIDTH) >> 16;
+ height = subsc_params.bitstream_resolution &
+ HFI_BITMASK_BITSTREAM_HEIGHT;
+
+ pixmp_ip->width = width;
+ pixmp_ip->height = height;
+
+ pixmp_op->width = pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ?
+ ALIGN(width, 192) : ALIGN(width, 128);
+ pixmp_op->height = pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ?
+ ALIGN(height, 16) : ALIGN(height, 32);
+ pixmp_op->plane_fmt[0].bytesperline =
+ pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ?
+ ALIGN(ALIGN(width, 192) * 4 / 3, 256) :
+ ALIGN(width, 128);
+ pixmp_op->plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
+
+ matrix_coeff = subsc_params.color_info & 0xFF;
+ transfer_char = (subsc_params.color_info & 0xFF00) >> 8;
+ primaries = (subsc_params.color_info & 0xFF0000) >> 16;
+ colour_description_present_flag =
+ (subsc_params.color_info & 0x1000000) >> 24;
+ full_range = (subsc_params.color_info & 0x2000000) >> 25;
+ video_signal_type_present_flag =
+ (subsc_params.color_info & 0x20000000) >> 29;
+
+ pixmp_op->colorspace = V4L2_COLORSPACE_DEFAULT;
+ pixmp_op->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ pixmp_op->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ pixmp_op->quantization = V4L2_QUANTIZATION_DEFAULT;
+
+ if (video_signal_type_present_flag) {
+ pixmp_op->quantization =
+ full_range ?
+ V4L2_QUANTIZATION_FULL_RANGE :
+ V4L2_QUANTIZATION_LIM_RANGE;
+ if (colour_description_present_flag) {
+ pixmp_op->colorspace =
+ get_v4l2_color_primaries(primaries);
+ pixmp_op->xfer_func =
+ get_v4l2_transfer_char(transfer_char);
+ pixmp_op->ycbcr_enc =
+ get_v4l2_matrix_coefficients(matrix_coeff);
+ }
+ }
+
+ pixmp_ip->colorspace = pixmp_op->colorspace;
+ pixmp_ip->xfer_func = pixmp_op->xfer_func;
+ pixmp_ip->ycbcr_enc = pixmp_op->ycbcr_enc;
+ pixmp_ip->quantization = pixmp_op->quantization;
+
+ inst->crop.top = subsc_params.crop_offsets[0] & 0xFFFF;
+ inst->crop.left = (subsc_params.crop_offsets[0] >> 16) & 0xFFFF;
+ inst->crop.height = pixmp_ip->height -
+ (subsc_params.crop_offsets[1] & 0xFFFF) - inst->crop.top;
+ inst->crop.width = pixmp_ip->width -
+ ((subsc_params.crop_offsets[1] >> 16) & 0xFFFF) - inst->crop.left;
+
+ inst->cap[PROFILE].value = subsc_params.profile;
+ inst->cap[LEVEL].value = subsc_params.level;
+ inst->cap[HEVC_TIER].value = subsc_params.tier;
+ inst->cap[POC].value = subsc_params.pic_order_cnt;
+
+ if (subsc_params.bit_depth == BIT_DEPTH_8)
+ inst->cap[BIT_DEPTH].value = BIT_DEPTH_8;
+ else
+ inst->cap[BIT_DEPTH].value = BIT_DEPTH_10;
+
+ if (subsc_params.coded_frames & HFI_BITMASK_FRAME_MBS_ONLY_FLAG)
+ inst->cap[CODED_FRAMES].value = CODED_FRAMES_PROGRESSIVE;
+ else
+ inst->cap[CODED_FRAMES].value = CODED_FRAMES_INTERLACE;
+
+ inst->fw_min_count = subsc_params.fw_min_count;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+
+ return 0;
+}
+
+int vdec_src_change(struct iris_inst *inst)
+{
+ struct v4l2_event event = {0};
+ u32 ret;
+
+ if (!inst->vb2q_src->streaming)
+ return 0;
+
+ ret = vdec_read_input_subcr_params(inst);
+ if (ret)
+ return ret;
+
+ event.type = V4L2_EVENT_SOURCE_CHANGE;
+ event.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION;
+ v4l2_event_queue_fh(&inst->fh, &event);
+
+ return ret;
+}
+
static int vdec_set_colorformat(struct iris_inst *inst)
{
u32 hfi_colorformat;
@@ -483,3 +939,127 @@ int vdec_set_output_property(struct iris_inst *inst)
return vdec_set_ubwc_stride_scanline(inst);
}
+
+int vdec_subscribe_dst_change_param(struct iris_inst *inst)
+{
+ u32 prop_type, payload_size, payload_type;
+ struct subscription_params subsc_params;
+ const u32 *dst_change_param = NULL;
+ u32 dst_change_param_size = 0;
+ struct iris_core *core;
+ u32 payload[32] = {0};
+ int ret;
+ u32 i;
+
+ core = inst->core;
+
+ payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
+ if (inst->codec == H264) {
+ dst_change_param_size = core->platform_data->avc_subscribe_param_size;
+ dst_change_param = core->platform_data->avc_subscribe_param;
+ } else if (inst->codec == HEVC) {
+ dst_change_param_size = core->platform_data->hevc_subscribe_param_size;
+ dst_change_param = core->platform_data->hevc_subscribe_param;
+ } else if (inst->codec == VP9) {
+ dst_change_param_size = core->platform_data->vp9_subscribe_param_size;
+ dst_change_param = core->platform_data->vp9_subscribe_param;
+ } else {
+ dst_change_param = NULL;
+ return -EINVAL;
+ }
+
+ if (!dst_change_param || !dst_change_param_size)
+ return -EINVAL;
+
+ payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
+ for (i = 0; i < dst_change_param_size; i++)
+ payload[i + 1] = dst_change_param[i];
+
+ ret = iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ OUTPUT_MPLANE,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ ((dst_change_param_size + 1) * sizeof(u32)));
+ if (ret)
+ return ret;
+
+ subsc_params = inst->dst_subcr_params;
+ for (i = 0; i < dst_change_param_size; i++) {
+ payload[0] = 0;
+ payload[1] = 0;
+ payload_size = 0;
+ payload_type = 0;
+ prop_type = dst_change_param[i];
+ switch (prop_type) {
+ case HFI_PROP_BITSTREAM_RESOLUTION:
+ payload[0] = subsc_params.bitstream_resolution;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_CROP_OFFSETS:
+ payload[0] = subsc_params.crop_offsets[0];
+ payload[1] = subsc_params.crop_offsets[1];
+ payload_size = sizeof(u64);
+ payload_type = HFI_PAYLOAD_64_PACKED;
+ break;
+ case HFI_PROP_LUMA_CHROMA_BIT_DEPTH:
+ payload[0] = subsc_params.bit_depth;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_CODED_FRAMES:
+ payload[0] = subsc_params.coded_frames;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT:
+ payload[0] = subsc_params.fw_min_count;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_PIC_ORDER_CNT_TYPE:
+ payload[0] = subsc_params.pic_order_cnt;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_SIGNAL_COLOR_INFO:
+ payload[0] = subsc_params.color_info;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_PROFILE:
+ payload[0] = subsc_params.profile;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_LEVEL:
+ payload[0] = subsc_params.level;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_TIER:
+ payload[0] = subsc_params.tier;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ default:
+ prop_type = 0;
+ ret = -EINVAL;
+ break;
+ }
+ if (prop_type) {
+ ret = iris_hfi_set_property(inst,
+ prop_type,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ payload_type,
+ &payload,
+ payload_size);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index 6b0306c..e0db653 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -16,5 +16,9 @@ int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);
int vdec_subscribe_property(struct iris_inst *inst, u32 plane);
int vdec_set_output_property(struct iris_inst *inst);
+int vdec_init_src_change_param(struct iris_inst *inst);
+int vdec_src_change(struct iris_inst *inst);
+int vdec_subscribe_src_change_param(struct iris_inst *inst);
+int vdec_subscribe_dst_change_param(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index fc12bde..22a8f5b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -206,6 +206,12 @@ struct platform_data {
u32 core_data_size;
struct plat_inst_cap *inst_cap_data;
u32 inst_cap_data_size;
+ const u32 *avc_subscribe_param;
+ unsigned int avc_subscribe_param_size;
+ const u32 *hevc_subscribe_param;
+ unsigned int hevc_subscribe_param_size;
+ const u32 *vp9_subscribe_param;
+ unsigned int vp9_subscribe_param_size;
const u32 *dec_input_prop;
unsigned int dec_input_prop_size;
const u32 *dec_output_prop_avc;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 6a4bfa3..7ae9715 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -382,6 +382,38 @@ static struct format_capability format_data_sm8550 = {
.color_format_info_size = ARRAY_SIZE(color_format_data_sm8550),
};
+static const u32 sm8550_vdec_src_change_param_avc[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_CODED_FRAMES,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PIC_ORDER_CNT_TYPE,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+};
+
+static const u32 sm8550_vdec_src_change_param_hevc[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+ HFI_PROP_TIER,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+};
+
+static const u32 sm8550_vdec_src_change_param_vp9[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+};
+
static const u32 sm8550_vdec_input_properties[] = {
HFI_PROP_NO_OUTPUT,
};
@@ -430,6 +462,18 @@ struct platform_data sm8550_data = {
.ubwc_config = ubwc_config_sm8550,
.format_data = &format_data_sm8550,
+ .avc_subscribe_param =
+ sm8550_vdec_src_change_param_avc,
+ .avc_subscribe_param_size =
+ ARRAY_SIZE(sm8550_vdec_src_change_param_avc),
+ .hevc_subscribe_param =
+ sm8550_vdec_src_change_param_hevc,
+ .hevc_subscribe_param_size =
+ ARRAY_SIZE(sm8550_vdec_src_change_param_hevc),
+ .vp9_subscribe_param =
+ sm8550_vdec_src_change_param_vp9,
+ .vp9_subscribe_param_size =
+ ARRAY_SIZE(sm8550_vdec_src_change_param_vp9),
.dec_input_prop = sm8550_vdec_input_properties,
.dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_input_properties),
.dec_output_prop_avc = sm8550_vdec_output_properties_avc,
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 26/34] media: iris: implement vb2 streaming ops on capture and output planes
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (24 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 25/34] media: iris: subscribe src change and handle firmware responses Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response Dikshita Agarwal
` (14 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
During stream on, set some mandatory properties
to firmware to start a session. Subscribe for different
bitstream parameters to get notified for change in any of
the subscribed parameters. Set all v4l2 properties set
by client, to firmware prepared with the dependency graph.
Also, configure the hardware internal buffers required
for frame processing. Send HFI_CMD_START on capture and
output planes to start the processing on respective planes.
The instance state is changed accordingly.
During stream off, send HFI_CMD_STOP to firmware which is
a synchronous command. After the response is received from
firmware, the session is closed on firmware and
instance state is changed accordingly.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 12 ++
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 21 +++
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 3 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 28 ++++
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 51 ++++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 173 +++++++++++++++++++--
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 7 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 14 ++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 2 +-
.../platform/qcom/vcodec/iris/iris_instance.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 38 +++++
.../media/platform/qcom/vcodec/iris/iris_state.h | 3 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 78 ++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 138 +++++++++++++++-
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 6 +-
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 4 +
19 files changed, 566 insertions(+), 20 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 0ef6bad..dfd1a4c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -14,6 +14,8 @@
#define HFI_CMD_INIT 0x01000001
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
+#define HFI_CMD_START 0x01000005
+#define HFI_CMD_STOP 0x01000006
#define HFI_CMD_BUFFER 0x01000009
#define HFI_PROP_IMAGE_VERSION 0x03000001
@@ -45,6 +47,16 @@ enum hfi_property_mode_type {
#define HFI_PROP_IMAGE_VERSION 0x03000001
+enum hfi_codec_type {
+ HFI_CODEC_DECODE_AVC = 1,
+ HFI_CODEC_ENCODE_AVC = 2,
+ HFI_CODEC_DECODE_HEVC = 3,
+ HFI_CODEC_ENCODE_HEVC = 4,
+ HFI_CODEC_DECODE_VP9 = 5,
+};
+
+#define HFI_PROP_CODEC 0x03000100
+
enum hfi_color_format {
HFI_COLOR_FMT_OPAQUE = 0,
HFI_COLOR_FMT_NV12 = 1,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index 1ee840e..ab25026 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -560,3 +560,24 @@ int iris_release_input_internal_buffers(struct iris_inst *inst)
return ret;
}
+
+int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ int ret;
+
+ if (buffer_type != BUF_PERSIST)
+ return -EINVAL;
+
+ ret = iris_get_internal_buf_info(inst, buffer_type);
+ if (ret)
+ return ret;
+
+ ret = iris_create_internal_buffers(inst, buffer_type);
+ if (ret)
+ return ret;
+
+ ret = iris_queue_internal_buffers(inst, buffer_type);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index bdef15f..8769c3d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -59,4 +59,7 @@ int iris_destroy_internal_buffers(struct iris_inst *inst,
u32 plane);
int iris_release_input_internal_buffers(struct iris_inst *inst);
+int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 28977e8..94fff74 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -113,6 +113,20 @@ static int adjust_cap(struct iris_inst *inst,
return cap->adjust(inst, ctrl);
}
+static int set_cap(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ struct plat_inst_cap *cap;
+
+ cap = &inst->cap[cap_id];
+ if (!inst->cap[cap_id].cap_id)
+ return 0;
+
+ if (!cap->set)
+ return 0;
+
+ return cap->set(inst, cap_id);
+}
+
static int adjust_dynamic_property(struct iris_inst *inst,
enum plat_inst_cap_type cap_id,
struct v4l2_ctrl *ctrl,
@@ -657,6 +671,20 @@ int set_pipe(struct iris_inst *inst,
&work_route, sizeof(u32));
}
+int set_v4l2_properties(struct iris_inst *inst)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ int ret = 0;
+
+ list_for_each_entry_safe(entry, temp, &inst->caps_list, list) {
+ ret = set_cap(inst, entry->cap_id);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
int adjust_v4l2_properties(struct iris_inst *inst)
{
struct cap_entry *entry = NULL, *temp = NULL;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 22ee6c4b..28ce767 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -28,6 +28,7 @@ int prepare_dependency_list(struct iris_inst *inst);
int iris_init_instance_caps(struct iris_core *core);
int iris_init_core_caps(struct iris_core *core);
int get_inst_capability(struct iris_inst *inst);
+int set_v4l2_properties(struct iris_inst *inst);
int adjust_v4l2_properties(struct iris_inst *inst);
int ctrls_init(struct iris_inst *inst, bool init);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 4cad673..7868566 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -9,6 +9,7 @@
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
#include "iris_instance.h"
+#include "iris_vidc.h"
int check_core_lock(struct iris_core *core)
{
@@ -377,3 +378,53 @@ struct iris_inst *to_instance(struct iris_core *core, u32 session_id)
return NULL;
}
+
+static int kill_session(struct iris_inst *inst)
+{
+ if (!inst->session_id)
+ return 0;
+
+ close_session(inst);
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return 0;
+}
+
+int session_streamoff(struct iris_inst *inst, u32 plane)
+{
+ enum signal_session_response signal_type;
+ u32 hw_response_timeout_val;
+ struct iris_core *core;
+ int ret;
+
+ ret = iris_hfi_stop(inst, plane);
+ if (ret)
+ goto error;
+
+ core = inst->core;
+ hw_response_timeout_val = core->cap[HW_RESPONSE_TIMEOUT].value;
+ mutex_unlock(&inst->lock);
+ ret = wait_for_completion_timeout(&inst->completions[signal_type],
+ msecs_to_jiffies(hw_response_timeout_val));
+ if (!ret) {
+ ret = -ETIMEDOUT;
+ process_inst_timeout(inst);
+ } else {
+ ret = 0;
+ }
+ mutex_lock(&inst->lock);
+
+ if (ret)
+ goto error;
+
+ ret = iris_inst_state_change_streamoff(inst, plane);
+ if (ret)
+ goto error;
+
+ return 0;
+
+error:
+ kill_session(inst);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index cb22adf..47a017d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -34,6 +34,7 @@ bool is_split_mode_enabled(struct iris_inst *inst);
int signal_session_msg_receipt(struct iris_inst *inst,
enum signal_session_response cmd);
struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
+int session_streamoff(struct iris_inst *inst, u32 plane);
u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index d15ce5a..e0da3be 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -219,9 +219,7 @@ int iris_hfi_session_open(struct iris_inst *inst)
return ret;
}
-int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
- u32 cmd, u32 plane, u32 payload_type,
- void *payload, u32 payload_size)
+int iris_hfi_session_close(struct iris_inst *inst)
{
struct iris_core *core;
int ret;
@@ -238,14 +236,126 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
}
ret = hfi_packet_session_command(inst,
- cmd,
+ HFI_CMD_CLOSE,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ HFI_PORT_NONE,
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_session_set_codec(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int ret;
+ u32 codec;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ codec = get_hfi_codec(inst);
+ ret = hfi_packet_session_property(inst,
+ HFI_PROP_CODEC,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_NONE,
+ HFI_PAYLOAD_U32_ENUM,
+ &codec,
+ sizeof(u32));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_session_set_default_header(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ u32 default_header = false;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ default_header = inst->cap[DEFAULT_HEADER].value;
+ ret = hfi_packet_session_property(inst,
+ HFI_PROP_DEC_DEFAULT_HEADER,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &default_header,
+ sizeof(u32));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_start(struct iris_inst *inst, u32 plane)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_START,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
get_hfi_port(plane),
inst->session_id,
- payload_type,
- payload,
- payload_size);
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
if (ret)
goto unlock;
@@ -257,10 +367,10 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
return ret;
}
-int iris_hfi_session_close(struct iris_inst *inst)
+int iris_hfi_stop(struct iris_inst *inst, u32 plane)
{
struct iris_core *core;
- int ret;
+ int ret = 0;
if (!inst->packet)
return -EINVAL;
@@ -273,12 +383,15 @@ int iris_hfi_session_close(struct iris_inst *inst)
goto unlock;
}
+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
ret = hfi_packet_session_command(inst,
- HFI_CMD_CLOSE,
+ HFI_CMD_STOP,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED |
HFI_HOST_FLAGS_NON_DISCARDABLE),
- HFI_PORT_NONE,
+ get_hfi_port(plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -294,6 +407,44 @@ int iris_hfi_session_close(struct iris_inst *inst)
return ret;
}
+int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
+ u32 cmd, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ cmd,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ payload_type,
+ payload,
+ payload_size);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
irqreturn_t iris_hfi_isr(int irq, void *data)
{
disable_irq_nosync(irq);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index d6b3fca..f054c2d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -19,6 +19,13 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
int iris_hfi_set_property(struct iris_inst *inst,
u32 packet_type, u32 flag, u32 plane, u32 payload_type,
void *payload, u32 payload_size);
+
+int iris_hfi_session_set_codec(struct iris_inst *inst);
+int iris_hfi_session_set_default_header(struct iris_inst *inst);
+
+int iris_hfi_start(struct iris_inst *inst, u32 plane);
+int iris_hfi_stop(struct iris_inst *inst, u32 plane);
+
int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_hfi_release_buffer(struct iris_inst *inst,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index dc7157d..b8785a9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -76,6 +76,20 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}
+u32 get_hfi_codec(struct iris_inst *inst)
+{
+ switch (inst->codec) {
+ case H264:
+ return HFI_CODEC_DECODE_AVC;
+ case HEVC:
+ return HFI_CODEC_DECODE_HEVC;
+ case VP9:
+ return HFI_CODEC_DECODE_VP9;
+ default:
+ return 0;
+ }
+}
+
u32 get_hfi_colorformat(u32 colorformat)
{
u32 hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index f813116..cf0960b6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -81,8 +81,8 @@ enum hfi_packet_port_type {
u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
u32 get_hfi_port(u32 plane);
-
u32 get_hfi_colorformat(u32 colorformat);
+u32 get_hfi_codec(struct iris_inst *inst);
u32 get_hfi_color_primaries(u32 primaries);
u32 get_hfi_transfer_char(u32 characterstics);
u32 get_hfi_matrix_coefficients(u32 coefficients);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 4f51d68..cddf143 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -40,10 +40,12 @@
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
* @ipsc_properties_set: boolean to set ipsc properties to fw
+ * @opsc_properties_set: boolean to set opsc properties to fw
* @hfi_frame_info: structure of frame info
* @src_subcr_params: subscription params to fw on input port
* @dst_subcr_params: subscription params to fw on output port
* @dpb_list_payload: array of dpb buffers
+ * @once_per_session_set: boolean to set once per session property
*/
struct iris_inst {
@@ -70,10 +72,12 @@ struct iris_inst {
u32 fw_min_count;
enum iris_inst_state state;
bool ipsc_properties_set;
+ bool opsc_properties_set;
struct iris_hfi_frame_info hfi_frame_info;
struct subscription_params src_subcr_params;
struct subscription_params dst_subcr_params;
u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE];
+ bool once_per_session_set;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 4cf6b69..6553029 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -193,3 +193,41 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id)
(inst->state == IRIS_INST_INPUT_STREAMING ||
inst->state == IRIS_INST_STREAMING)));
}
+
+int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane)
+{
+ enum iris_inst_state new_state = IRIS_INST_ERROR;
+
+ if (plane == INPUT_MPLANE) {
+ if (inst->state == IRIS_INST_OPEN)
+ new_state = IRIS_INST_INPUT_STREAMING;
+ else if (inst->state == IRIS_INST_OUTPUT_STREAMING)
+ new_state = IRIS_INST_STREAMING;
+ } else if (plane == OUTPUT_MPLANE) {
+ if (inst->state == IRIS_INST_OPEN)
+ new_state = IRIS_INST_OUTPUT_STREAMING;
+ else if (inst->state == IRIS_INST_INPUT_STREAMING)
+ new_state = IRIS_INST_STREAMING;
+ }
+
+ return iris_inst_change_state(inst, new_state);
+}
+
+int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane)
+{
+ enum iris_inst_state new_state = IRIS_INST_ERROR;
+
+ if (plane == INPUT_MPLANE) {
+ if (inst->state == IRIS_INST_INPUT_STREAMING)
+ new_state = IRIS_INST_OPEN;
+ else if (inst->state == IRIS_INST_STREAMING)
+ new_state = IRIS_INST_OUTPUT_STREAMING;
+ } else if (plane == OUTPUT_MPLANE) {
+ if (inst->state == IRIS_INST_OUTPUT_STREAMING)
+ new_state = IRIS_INST_OPEN;
+ else if (inst->state == IRIS_INST_STREAMING)
+ new_state = IRIS_INST_INPUT_STREAMING;
+ }
+
+ return iris_inst_change_state(inst, new_state);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 35263e8..28d5380 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -47,4 +47,7 @@ bool allow_streamon(struct iris_inst *inst, u32 type);
bool allow_streamoff(struct iris_inst *inst, u32 type);
bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);
+int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane);
+int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index 66b5295..d599366 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -6,9 +6,12 @@
#include "iris_buffer.h"
#include "iris_ctrls.h"
#include "iris_core.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
#include "iris_instance.h"
#include "iris_vb2.h"
+#include "iris_vdec.h"
int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
@@ -90,6 +93,81 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
return ret;
}
+int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ if (!q)
+ return -EINVAL;
+
+ inst = vb2_get_drv_priv(q);
+ if (!inst || !inst->core)
+ return -EINVAL;
+
+ if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ if (!inst->once_per_session_set) {
+ inst->once_per_session_set = true;
+ ret = iris_hfi_session_set_codec(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_hfi_session_set_default_header(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
+ if (ret)
+ goto error;
+ }
+
+ if (q->type == INPUT_MPLANE)
+ ret = vdec_streamon_input(inst);
+ else if (q->type == OUTPUT_MPLANE)
+ ret = vdec_streamon_output(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+error:
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return ret;
+}
+
+void iris_vb2_stop_streaming(struct vb2_queue *q)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ if (!q)
+ return;
+
+ inst = vb2_get_drv_priv(q);
+ if (!inst)
+ return;
+
+ if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE)
+ goto error;
+
+ if (q->type == INPUT_MPLANE)
+ ret = session_streamoff(inst, INPUT_MPLANE);
+ else if (q->type == OUTPUT_MPLANE)
+ ret = session_streamoff(inst, OUTPUT_MPLANE);
+
+ if (ret)
+ goto error;
+
+ return;
+
+error:
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+}
+
void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
struct dma_buf *dbuf, unsigned long size)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
index 4342034..fc0e804 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -11,6 +11,8 @@
int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
unsigned int sizes[], struct device *alloc_devs[]);
+int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count);
+void iris_vb2_stop_streaming(struct vb2_queue *q);
/* vb2_mem_ops */
void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index ac47fc0..0d8ca4b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -340,7 +340,7 @@ int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscri
return ret;
}
-int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
+static int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
{
const u32 *subcribe_prop = NULL;
u32 subscribe_prop_size = 0;
@@ -597,7 +597,7 @@ static int vdec_set_tier(struct iris_inst *inst)
sizeof(u32));
}
-int vdec_subscribe_src_change_param(struct iris_inst *inst)
+static int vdec_subscribe_src_change_param(struct iris_inst *inst)
{
const u32 *src_change_param;
u32 src_change_param_size;
@@ -925,7 +925,7 @@ static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst)
sizeof(u32) * 4);
}
-int vdec_set_output_property(struct iris_inst *inst)
+static int vdec_set_output_property(struct iris_inst *inst)
{
int ret;
@@ -940,7 +940,7 @@ int vdec_set_output_property(struct iris_inst *inst)
return vdec_set_ubwc_stride_scanline(inst);
}
-int vdec_subscribe_dst_change_param(struct iris_inst *inst)
+static int vdec_subscribe_dst_change_param(struct iris_inst *inst)
{
u32 prop_type, payload_size, payload_type;
struct subscription_params subsc_params;
@@ -1063,3 +1063,133 @@ int vdec_subscribe_dst_change_param(struct iris_inst *inst)
return ret;
}
+
+static int process_streamon_input(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_start(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+int vdec_streamon_input(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_session_supported(inst);
+ if (ret)
+ return ret;
+
+ ret = set_v4l2_properties(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_get_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_destroy_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_create_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_queue_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ if (!inst->ipsc_properties_set) {
+ ret = vdec_subscribe_src_change_param(inst);
+ if (ret)
+ return ret;
+ inst->ipsc_properties_set = true;
+ }
+
+ ret = vdec_subscribe_property(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = process_streamon_input(inst);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+static int process_streamon_output(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_start(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+int vdec_streamon_output(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_session_supported(inst);
+ if (ret)
+ return ret;
+
+ ret = vdec_set_output_property(inst);
+ if (ret)
+ goto error;
+
+ if (!inst->opsc_properties_set) {
+ memcpy(&inst->dst_subcr_params,
+ &inst->src_subcr_params,
+ sizeof(inst->src_subcr_params));
+ ret = vdec_subscribe_dst_change_param(inst);
+ if (ret)
+ goto error;
+ inst->opsc_properties_set = true;
+ }
+
+ ret = vdec_subscribe_property(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_get_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_destroy_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_create_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = process_streamon_output(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_queue_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ session_streamoff(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index e0db653..0722da1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -14,11 +14,9 @@ int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);
-int vdec_subscribe_property(struct iris_inst *inst, u32 plane);
-int vdec_set_output_property(struct iris_inst *inst);
int vdec_init_src_change_param(struct iris_inst *inst);
int vdec_src_change(struct iris_inst *inst);
-int vdec_subscribe_src_change_param(struct iris_inst *inst);
-int vdec_subscribe_dst_change_param(struct iris_inst *inst);
+int vdec_streamon_input(struct iris_inst *inst);
+int vdec_streamon_output(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 5c76821..14d0077 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -176,6 +176,8 @@ int vidc_open(struct file *filp)
inst->core = core;
inst->session_id = hash32_ptr(inst);
+ inst->ipsc_properties_set = false;
+ inst->opsc_properties_set = false;
iris_inst_change_state(inst, IRIS_INST_OPEN);
mutex_init(&inst->lock);
mutex_init(&inst->ctx_q_lock);
@@ -908,6 +910,8 @@ static const struct v4l2_file_operations v4l2_file_ops = {
static const struct vb2_ops iris_vb2_ops = {
.queue_setup = iris_vb2_queue_setup,
+ .start_streaming = iris_vb2_start_streaming,
+ .stop_streaming = iris_vb2_stop_streaming,
};
static struct vb2_mem_ops iris_vb2_mem_ops = {
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (25 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 26/34] media: iris: implement vb2 streaming ops on capture and output planes Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 12:21 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 28/34] media: iris: add instance sub states and implement DRC and Drain sequence Dikshita Agarwal
` (13 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Implement vb2 ops for buf queue. Below are the different
buffer attributes:
BUF_ATTR_DEFERRED - buffer queued by client but not submitted
to firmware.
BUF_ATTR_READ_ONLY - processed buffer received from firmware
as read only. These buffers are held in firmware as reference
for future frame processing.
BUF_ATTR_PENDING_RELEASE - buffers requested to be released
from firmware.
BUF_ATTR_QUEUED - buffers submitted to firmware.
BUF_ATTR_DEQUEUED - buffers received from firmware.
BUF_ATTR_BUFFER_DONE - buffers sent back to vb2.
Buffers are submitted and received via HFI_CMD_BUFFER.
Firmware associates below flags during buffer response:
HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware.
HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware.
Input buffers dequeued from firmware are sent directly to vb2.
Output buffers if read only, are sent to vb2 and also maintained
in read only list. If the same read only buffer is received form
client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and
submitted to firmware. Once the buffer is received from firmware
as non read only, it is removed from read only list.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 6 +
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 1 -
.../media/platform/qcom/vcodec/iris/iris_common.h | 9 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 369 +++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 11 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 2 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 392 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_state.c | 8 +
.../media/platform/qcom/vcodec/iris/iris_state.h | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 81 +++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 49 +++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 2 +
15 files changed, 959 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index dfd1a4c..104ef9c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -238,4 +238,10 @@ enum hfi_buffer_host_flags {
HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200,
};
+enum hfi_buffer_firmware_flags {
+ HFI_BUF_FW_FLAG_NONE = 0x00000000,
+ HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001,
+ HFI_BUF_FW_FLAG_READONLY = 0x00000010,
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index 8769c3d..59c4a10 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -58,7 +58,6 @@ int iris_destroy_internal_buffer(struct iris_inst *inst,
int iris_destroy_internal_buffers(struct iris_inst *inst,
u32 plane);
int iris_release_input_internal_buffers(struct iris_inst *inst);
-
int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
enum iris_buffer_type buffer_type);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 0fbd139..5fd96d3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -23,6 +23,8 @@
#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
+#define MAX_DPB_COUNT 32
+
#define MAX_DPB_LIST_ARRAY_SIZE (16 * 4)
#define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4)
@@ -69,6 +71,13 @@ enum iris_buffer_type {
BUF_VPSS,
};
+enum iris_buffer_flags {
+ BUF_FLAG_KEYFRAME = 0x00000008,
+ BUF_FLAG_PFRAME = 0x00000010,
+ BUF_FLAG_BFRAME = 0x00000020,
+ BUF_FLAG_ERROR = 0x00000040,
+};
+
enum iris_buffer_attributes {
BUF_ATTR_DEFERRED = BIT(0),
BUF_ATTR_READ_ONLY = BIT(1),
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 7868566..a5b8aef 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -57,6 +57,18 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type)
}
}
+u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return INPUT_MPLANE;
+ case BUF_OUTPUT:
+ return OUTPUT_MPLANE;
+ default:
+ return 0;
+ }
+}
+
int get_mbpf(struct iris_inst *inst)
{
int height = 0, width = 0;
@@ -390,13 +402,359 @@ static int kill_session(struct iris_inst *inst)
return 0;
}
+struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index)
+{
+ struct iris_buffer *iter = NULL;
+ struct iris_buffer *buf = NULL;
+ enum iris_buffer_type buf_type;
+ struct iris_buffers *buffers;
+
+ bool found = false;
+
+ buf_type = v4l2_type_to_driver(plane);
+ if (!buf_type)
+ return NULL;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return NULL;
+
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->index == index) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+
+ if (!found)
+ return NULL;
+
+ return buf;
+}
+
+static void process_requeued_readonly_buffers(struct iris_inst *inst,
+ struct iris_buffer *buf)
+{
+ struct iris_buffer *ro_buf, *dummy;
+
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->device_addr != buf->device_addr)
+ continue;
+ if (ro_buf->attr & BUF_ATTR_READ_ONLY &&
+ !(ro_buf->attr & BUF_ATTR_PENDING_RELEASE)) {
+ buf->attr |= BUF_ATTR_READ_ONLY;
+
+ list_del_init(&ro_buf->list);
+ kfree(ro_buf);
+ break;
+ }
+ }
+}
+
+int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf)
+{
+ int ret;
+
+ if (buf->type == BUF_OUTPUT)
+ process_requeued_readonly_buffers(inst, buf);
+
+ ret = iris_hfi_queue_buffer(inst, buf);
+ if (ret)
+ return ret;
+
+ buf->attr &= ~BUF_ATTR_DEFERRED;
+ buf->attr |= BUF_ATTR_QUEUED;
+
+ return ret;
+}
+
+int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf;
+ int ret = 0;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry(buf, &buffers->list, list) {
+ if (!(buf->attr & BUF_ATTR_DEFERRED))
+ continue;
+ ret = queue_buffer(inst, buf);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_release_nonref_buffers(struct iris_inst *inst)
+{
+ u32 fw_ro_count = 0, nonref_ro_count = 0;
+ struct iris_buffer *ro_buf;
+ bool found = false;
+ int ret = 0;
+ int i = 0;
+
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ if (!(ro_buf->attr & BUF_ATTR_READ_ONLY))
+ continue;
+ if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ fw_ro_count++;
+ }
+
+ if (fw_ro_count <= MAX_DPB_COUNT)
+ return 0;
+
+ /*
+ * Mark the read only buffers present in read_only list as
+ * non-reference if it's not part of dpb_list_payload.
+ * dpb_list_payload details:
+ * payload[0-1] : 64 bits base_address of DPB-1
+ * payload[2] : 32 bits addr_offset of DPB-1
+ * payload[3] : 32 bits data_offset of DPB-1
+ */
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ found = false;
+ if (!(ro_buf->attr & BUF_ATTR_READ_ONLY))
+ continue;
+ if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) {
+ if (ro_buf->device_addr == inst->dpb_list_payload[i] &&
+ ro_buf->data_offset == inst->dpb_list_payload[i + 3]) {
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ nonref_ro_count++;
+ }
+
+ if (nonref_ro_count <= inst->buffers.output.min_count)
+ return 0;
+
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ found = false;
+ if (!(ro_buf->attr & BUF_ATTR_READ_ONLY))
+ continue;
+ if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) {
+ if (ro_buf->device_addr == inst->dpb_list_payload[i] &&
+ ro_buf->data_offset == inst->dpb_list_payload[i + 3]) {
+ found = true;
+ break;
+ }
+ }
+ if (!found) {
+ ro_buf->attr |= BUF_ATTR_PENDING_RELEASE;
+ ret = iris_hfi_release_buffer(inst, ro_buf);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
+}
+
+int iris_vb2_buffer_done(struct iris_inst *inst,
+ struct iris_buffer *buf)
+{
+ struct vb2_v4l2_buffer *vbuf;
+ struct vb2_queue *q = NULL;
+ struct vb2_buffer *iter;
+ struct vb2_buffer *vb2;
+ int type, state;
+ bool found;
+
+ type = v4l2_type_from_driver(buf->type);
+ if (!type)
+ return -EINVAL;
+
+ if (type == INPUT_MPLANE)
+ q = inst->vb2q_src;
+ else if (type == OUTPUT_MPLANE)
+ q = inst->vb2q_dst;
+ if (!q || !q->streaming)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &q->queued_list, queued_entry) {
+ if (iter->state != VB2_BUF_STATE_ACTIVE)
+ continue;
+ if (iter->index == buf->index) {
+ found = true;
+ vb2 = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ if (buf->flags & BUF_FLAG_ERROR)
+ state = VB2_BUF_STATE_ERROR;
+ else
+ state = VB2_BUF_STATE_DONE;
+
+ vbuf = to_vb2_v4l2_buffer(vb2);
+ vbuf->flags = buf->flags;
+ vb2->timestamp = buf->timestamp;
+ vb2->planes[0].bytesused = buf->data_size + vb2->planes[0].data_offset;
+ vb2_buffer_done(vb2, state);
+
+ return 0;
+}
+
+static int iris_flush_deferred_buffers(struct iris_inst *inst,
+ enum iris_buffer_type type)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+
+ buffers = iris_get_buffer_list(inst, type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->attr & BUF_ATTR_DEFERRED) {
+ if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) {
+ buf->attr |= BUF_ATTR_BUFFER_DONE;
+ buf->data_size = 0;
+ iris_vb2_buffer_done(inst, buf);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int iris_flush_read_only_buffers(struct iris_inst *inst,
+ enum iris_buffer_type type)
+{
+ struct iris_buffer *ro_buf, *dummy;
+
+ if (type != BUF_OUTPUT)
+ return 0;
+
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->attr & BUF_ATTR_READ_ONLY)
+ continue;
+ if (ro_buf->attach && ro_buf->sg_table)
+ dma_buf_unmap_attachment(ro_buf->attach, ro_buf->sg_table,
+ DMA_BIDIRECTIONAL);
+ if (ro_buf->attach && ro_buf->dmabuf)
+ dma_buf_detach(ro_buf->dmabuf, ro_buf->attach);
+ ro_buf->attach = NULL;
+ ro_buf->sg_table = NULL;
+ ro_buf->dmabuf = NULL;
+ ro_buf->device_addr = 0x0;
+ list_del_init(&ro_buf->list);
+ kfree(ro_buf);
+ }
+
+ return 0;
+}
+
+void iris_destroy_buffers(struct iris_inst *inst)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+
+ static const enum iris_buffer_type ext_buf_types[] = {
+ BUF_INPUT,
+ BUF_OUTPUT,
+ };
+ static const enum iris_buffer_type internal_buf_types[] = {
+ BUF_BIN,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+ BUF_DPB,
+ BUF_PERSIST,
+ };
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(internal_buf_types); i++) {
+ buffers = iris_get_buffer_list(inst, internal_buf_types[i]);
+ if (!buffers)
+ continue;
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list)
+ iris_destroy_internal_buffer(inst, buf);
+ }
+
+ list_for_each_entry_safe(buf, dummy, &inst->buffers.read_only.list, list) {
+ if (buf->attach && buf->sg_table)
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ if (buf->attach && buf->dmabuf)
+ dma_buf_detach(buf->dmabuf, buf->attach);
+ list_del_init(&buf->list);
+ kfree(buf);
+ }
+
+ for (i = 0; i < ARRAY_SIZE(ext_buf_types); i++) {
+ buffers = iris_get_buffer_list(inst, ext_buf_types[i]);
+ if (!buffers)
+ continue;
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->attach && buf->sg_table)
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table,
+ DMA_BIDIRECTIONAL);
+ if (buf->attach && buf->dmabuf)
+ dma_buf_detach(buf->dmabuf, buf->attach);
+ list_del_init(&buf->list);
+ kfree(buf);
+ }
+ }
+}
+
+static int get_num_queued_buffers(struct iris_inst *inst,
+ enum iris_buffer_type type)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *vbuf;
+ int count = 0;
+
+ if (type == BUF_INPUT)
+ buffers = &inst->buffers.input;
+ else if (type == BUF_OUTPUT)
+ buffers = &inst->buffers.output;
+ else
+ return count;
+
+ list_for_each_entry(vbuf, &buffers->list, list) {
+ if (vbuf->type != type)
+ continue;
+ if (!(vbuf->attr & BUF_ATTR_QUEUED))
+ continue;
+ count++;
+ }
+
+ return count;
+}
+
int session_streamoff(struct iris_inst *inst, u32 plane)
{
enum signal_session_response signal_type;
+ enum iris_buffer_type buffer_type;
u32 hw_response_timeout_val;
struct iris_core *core;
+ int count = 0;
int ret;
+ if (plane == INPUT_MPLANE) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ buffer_type = BUF_INPUT;
+ } else if (plane == OUTPUT_MPLANE) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ buffer_type = BUF_OUTPUT;
+ } else {
+ return -EINVAL;
+ }
+
ret = iris_hfi_stop(inst, plane);
if (ret)
goto error;
@@ -417,14 +775,25 @@ int session_streamoff(struct iris_inst *inst, u32 plane)
if (ret)
goto error;
+ /* no more queued buffers after streamoff */
+ count = get_num_queued_buffers(inst, buffer_type);
+ if (count) {
+ ret = -EINVAL;
+ goto error;
+ }
+
ret = iris_inst_state_change_streamoff(inst, plane);
if (ret)
goto error;
+ iris_flush_deferred_buffers(inst, buffer_type);
+ iris_flush_read_only_buffers(inst, buffer_type);
return 0;
error:
kill_session(inst);
+ iris_flush_deferred_buffers(inst, buffer_type);
+ iris_flush_read_only_buffers(inst, buffer_type);
return ret;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 47a017d..3a9889e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -24,6 +24,7 @@ bool res_is_less_than(u32 width, u32 height,
u32 get_port_info(struct iris_inst *inst,
enum plat_inst_cap_type cap_id);
enum iris_buffer_type v4l2_type_to_driver(u32 type);
+u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type);
int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);
@@ -34,7 +35,6 @@ bool is_split_mode_enabled(struct iris_inst *inst);
int signal_session_msg_receipt(struct iris_inst *inst,
enum signal_session_response cmd);
struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
-int session_streamoff(struct iris_inst *inst, u32 plane);
u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
@@ -43,4 +43,13 @@ enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l
struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type);
int check_session_supported(struct iris_inst *inst);
+struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index);
+int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf);
+int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type);
+int iris_vb2_buffer_done(struct iris_inst *inst,
+ struct iris_buffer *buf);
+int iris_release_nonref_buffers(struct iris_inst *inst);
+void iris_destroy_buffers(struct iris_inst *inst);
+int session_streamoff(struct iris_inst *inst, u32 plane);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index b8785a9..8dafd04 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -76,6 +76,34 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}
+u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type)
+{
+ switch (buf_type) {
+ case HFI_BUFFER_BITSTREAM:
+ return BUF_INPUT;
+ case HFI_BUFFER_RAW:
+ return BUF_OUTPUT;
+ case HFI_BUFFER_BIN:
+ return BUF_BIN;
+ case HFI_BUFFER_ARP:
+ return BUF_ARP;
+ case HFI_BUFFER_COMV:
+ return BUF_COMV;
+ case HFI_BUFFER_NON_COMV:
+ return BUF_NON_COMV;
+ case HFI_BUFFER_LINE:
+ return BUF_LINE;
+ case HFI_BUFFER_DPB:
+ return BUF_DPB;
+ case HFI_BUFFER_PERSIST:
+ return BUF_PERSIST;
+ case HFI_BUFFER_VPSS:
+ return BUF_VPSS;
+ default:
+ return 0;
+ }
+}
+
u32 get_hfi_codec(struct iris_inst *inst)
{
switch (inst->codec) {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index cf0960b6..8999c28 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -6,6 +6,7 @@
#ifndef _IRIS_HFI_PACKET_H_
#define _IRIS_HFI_PACKET_H_
+#include "hfi_defines.h"
#include "iris_core.h"
#include "iris_instance.h"
@@ -89,6 +90,7 @@ u32 get_hfi_matrix_coefficients(u32 coefficients);
u32 get_v4l2_color_primaries(u32 hfi_primaries);
u32 get_v4l2_transfer_char(u32 hfi_characterstics);
u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients);
+u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type);
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
int hfi_packet_sys_init(struct iris_core *core,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 4ca9314..b1236dd 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -4,6 +4,7 @@
*/
#include "hfi_defines.h"
+#include "iris_buffer.h"
#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "iris_hfi_response.h"
@@ -21,6 +22,11 @@ struct iris_inst_hfi_range {
int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
};
+struct iris_hfi_buffer_handle {
+ enum hfi_buffer_type type;
+ int (*handle)(struct iris_inst *inst, struct hfi_buffer *buffer);
+};
+
struct iris_hfi_packet_handle {
enum hfi_buffer_type type;
int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
@@ -44,6 +50,94 @@ static void print_sfr_message(struct iris_core *core)
}
}
+static bool is_valid_hfi_buffer_type(u32 buffer_type)
+{
+ if (buffer_type != HFI_BUFFER_BITSTREAM &&
+ buffer_type != HFI_BUFFER_RAW &&
+ buffer_type != HFI_BUFFER_BIN &&
+ buffer_type != HFI_BUFFER_ARP &&
+ buffer_type != HFI_BUFFER_COMV &&
+ buffer_type != HFI_BUFFER_NON_COMV &&
+ buffer_type != HFI_BUFFER_LINE &&
+ buffer_type != HFI_BUFFER_DPB &&
+ buffer_type != HFI_BUFFER_PERSIST &&
+ buffer_type != HFI_BUFFER_VPSS) {
+ return false;
+ }
+
+ return true;
+}
+
+static bool is_valid_hfi_port(u32 port, u32 buffer_type)
+{
+ if (port == HFI_PORT_NONE &&
+ buffer_type != HFI_BUFFER_PERSIST)
+ return false;
+
+ if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW)
+ return false;
+
+ return true;
+}
+
+static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags)
+{
+ u32 driver_flags = 0;
+
+ if (inst->hfi_frame_info.picture_type & HFI_PICTURE_IDR)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_P)
+ driver_flags |= BUF_FLAG_PFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_B)
+ driver_flags |= BUF_FLAG_BFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_I)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_CRA)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_BLA)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+
+ if (inst->hfi_frame_info.data_corrupt)
+ driver_flags |= BUF_FLAG_ERROR;
+
+ if (inst->hfi_frame_info.overflow)
+ driver_flags |= BUF_FLAG_ERROR;
+
+ return driver_flags;
+}
+
+static bool validate_packet_payload(struct hfi_packet *pkt)
+{
+ u32 payload_size = 0;
+
+ switch (pkt->payload_info) {
+ case HFI_PAYLOAD_U32:
+ case HFI_PAYLOAD_S32:
+ case HFI_PAYLOAD_Q16:
+ case HFI_PAYLOAD_U32_ENUM:
+ case HFI_PAYLOAD_32_PACKED:
+ payload_size = 4;
+ break;
+ case HFI_PAYLOAD_U64:
+ case HFI_PAYLOAD_S64:
+ case HFI_PAYLOAD_64_PACKED:
+ payload_size = 8;
+ break;
+ case HFI_PAYLOAD_STRUCTURE:
+ if (pkt->type == HFI_CMD_BUFFER)
+ payload_size = sizeof(struct hfi_buffer);
+ break;
+ default:
+ payload_size = 0;
+ break;
+ }
+
+ if (pkt->size < sizeof(struct hfi_packet) + payload_size)
+ return false;
+
+ return true;
+}
+
static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, u32 core_resp_pkt_size)
{
u32 response_pkt_size = 0;
@@ -168,6 +262,293 @@ static int handle_session_close(struct iris_inst *inst,
return 0;
}
+static int handle_read_only_buffer(struct iris_inst *inst,
+ struct iris_buffer *buf)
+{
+ struct iris_buffer *ro_buf, *iter;
+ bool found = false;
+
+ list_for_each_entry(iter, &inst->buffers.read_only.list, list) {
+ if (iter->device_addr == buf->device_addr) {
+ found = true;
+ ro_buf = iter;
+ break;
+ }
+ }
+
+ if (!found) {
+ ro_buf = kzalloc(sizeof(*ro_buf), GFP_KERNEL);
+ if (!ro_buf)
+ return -ENOMEM;
+ ro_buf->index = -1;
+ ro_buf->inst = inst;
+ ro_buf->type = buf->type;
+ ro_buf->fd = buf->fd;
+ ro_buf->dmabuf = buf->dmabuf;
+ ro_buf->device_addr = buf->device_addr;
+ ro_buf->data_offset = buf->data_offset;
+ INIT_LIST_HEAD(&ro_buf->list);
+ list_add_tail(&ro_buf->list, &inst->buffers.read_only.list);
+ }
+ ro_buf->attr |= BUF_ATTR_READ_ONLY;
+
+ return 0;
+}
+
+static int handle_non_read_only_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffer *ro_buf;
+
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ if (ro_buf->device_addr == buffer->base_address) {
+ ro_buf->attr &= ~BUF_ATTR_READ_ONLY;
+ break;
+ }
+ }
+
+ return 0;
+}
+
+static int handle_release_output_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffer *buf, *iter;
+ bool found = false;
+
+ list_for_each_entry(iter, &inst->buffers.read_only.list, list) {
+ if (iter->device_addr == buffer->base_address &&
+ iter->attr & BUF_ATTR_PENDING_RELEASE) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ buf->attr &= ~BUF_ATTR_READ_ONLY;
+ buf->attr &= ~BUF_ATTR_PENDING_RELEASE;
+
+ return 0;
+}
+
+static int handle_input_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf, *iter;
+ bool found;
+
+ buffers = iris_get_buffer_list(inst, BUF_INPUT);
+ if (!buffers)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->index == buffer->index) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ if (!(buf->attr & BUF_ATTR_QUEUED))
+ return 0;
+
+ buf->data_size = buffer->data_size;
+ buf->attr &= ~BUF_ATTR_QUEUED;
+ buf->attr |= BUF_ATTR_DEQUEUED;
+
+ buf->flags = get_driver_buffer_flags(inst, buffer->flags);
+
+ return 0;
+}
+
+static int handle_output_buffer(struct iris_inst *inst,
+ struct hfi_buffer *hfi_buffer)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf, *iter;
+ bool found;
+ int ret = 0;
+
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
+ return handle_release_output_buffer(inst, hfi_buffer);
+
+ if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
+ ret = handle_non_read_only_buffer(inst, hfi_buffer);
+
+ buffers = iris_get_buffer_list(inst, BUF_OUTPUT);
+ if (!buffers)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (!(iter->attr & BUF_ATTR_QUEUED))
+ continue;
+
+ found = (iter->index == hfi_buffer->index &&
+ iter->device_addr == hfi_buffer->base_address &&
+ iter->data_offset == hfi_buffer->data_offset);
+
+ if (found) {
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return 0;
+
+ buf->data_offset = hfi_buffer->data_offset;
+ buf->data_size = hfi_buffer->data_size;
+ buf->timestamp = hfi_buffer->timestamp;
+
+ buf->attr &= ~BUF_ATTR_QUEUED;
+ buf->attr |= BUF_ATTR_DEQUEUED;
+
+ if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) {
+ buf->attr |= BUF_ATTR_READ_ONLY;
+ ret = handle_read_only_buffer(inst, buf);
+ } else {
+ buf->attr &= ~BUF_ATTR_READ_ONLY;
+ }
+
+ buf->flags = get_driver_buffer_flags(inst, hfi_buffer->flags);
+
+ return ret;
+}
+
+static int handle_dequeue_buffers(struct iris_inst *inst)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *dummy;
+ struct iris_buffer *buf;
+ int ret = 0;
+ int i;
+ static const enum iris_buffer_type buffer_type[] = {
+ BUF_INPUT,
+ BUF_OUTPUT,
+ };
+
+ for (i = 0; i < ARRAY_SIZE(buffer_type); i++) {
+ buffers = iris_get_buffer_list(inst, buffer_type[i]);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->attr & BUF_ATTR_DEQUEUED) {
+ buf->attr &= ~BUF_ATTR_DEQUEUED;
+ if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) {
+ buf->attr |= BUF_ATTR_BUFFER_DONE;
+ ret = iris_vb2_buffer_done(inst, buf);
+ if (ret)
+ ret = 0;
+ }
+ }
+ }
+ }
+
+ return ret;
+}
+
+static int handle_release_internal_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf, *iter;
+ int ret = 0;
+ bool found;
+
+ buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(buffer->type));
+ if (!buffers)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->device_addr == buffer->base_address) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ buf->attr &= ~BUF_ATTR_QUEUED;
+
+ if (buf->attr & BUF_ATTR_PENDING_RELEASE)
+ ret = iris_destroy_internal_buffer(inst, buf);
+
+ return ret;
+}
+
+static int handle_session_buffer(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ struct hfi_buffer *buffer;
+ u32 hfi_handle_size = 0;
+ int i, ret = 0;
+ const struct iris_hfi_buffer_handle *hfi_handle_arr = NULL;
+ static const struct iris_hfi_buffer_handle input_hfi_handle[] = {
+ {HFI_BUFFER_BITSTREAM, handle_input_buffer },
+ {HFI_BUFFER_BIN, handle_release_internal_buffer },
+ {HFI_BUFFER_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_NON_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_LINE, handle_release_internal_buffer },
+ {HFI_BUFFER_PERSIST, handle_release_internal_buffer },
+ };
+ static const struct iris_hfi_buffer_handle output_hfi_handle[] = {
+ {HFI_BUFFER_RAW, handle_output_buffer },
+ {HFI_BUFFER_DPB, handle_release_internal_buffer },
+ };
+
+ if (pkt->payload_info == HFI_PAYLOAD_NONE)
+ return 0;
+
+ if (!validate_packet_payload(pkt)) {
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ return 0;
+ }
+
+ buffer = (struct hfi_buffer *)((u8 *)pkt + sizeof(*pkt));
+ if (!is_valid_hfi_buffer_type(buffer->type))
+ return 0;
+
+ if (!is_valid_hfi_port(pkt->port, buffer->type))
+ return 0;
+
+ if (pkt->port == HFI_PORT_BITSTREAM) {
+ hfi_handle_size = ARRAY_SIZE(input_hfi_handle);
+ hfi_handle_arr = input_hfi_handle;
+ } else if (pkt->port == HFI_PORT_RAW) {
+ hfi_handle_size = ARRAY_SIZE(output_hfi_handle);
+ hfi_handle_arr = output_hfi_handle;
+ }
+
+ if (!hfi_handle_arr || !hfi_handle_size)
+ return -EINVAL;
+
+ for (i = 0; i < hfi_handle_size; i++) {
+ if (hfi_handle_arr[i].type == buffer->type) {
+ ret = hfi_handle_arr[i].handle(inst, buffer);
+ if (ret)
+ return ret;
+ break;
+ }
+ }
+
+ if (i == hfi_handle_size)
+ return -EINVAL;
+
+ return ret;
+}
+
static int handle_src_change(struct iris_inst *inst,
struct hfi_packet *pkt)
{
@@ -191,6 +572,7 @@ static int handle_session_command(struct iris_inst *inst,
static const struct iris_hfi_packet_handle hfi_pkt_handle[] = {
{HFI_CMD_OPEN, NULL },
{HFI_CMD_CLOSE, handle_session_close },
+ {HFI_CMD_BUFFER, handle_session_buffer },
{HFI_CMD_SETTINGS_CHANGE, handle_src_change },
{HFI_CMD_SUBSCRIBE_MODE, NULL },
};
@@ -386,6 +768,7 @@ static int handle_session_response(struct iris_core *core,
{
struct hfi_packet *packet;
struct iris_inst *inst;
+ bool dequeue = false;
u8 *pkt, *start_pkt;
int ret = 0;
int i, j;
@@ -423,6 +806,7 @@ static int handle_session_response(struct iris_core *core,
handle_session_error(inst, packet);
if (packet->type > be[i].begin && packet->type < be[i].end) {
+ dequeue |= (packet->type == HFI_CMD_BUFFER);
ret = be[i].handle(inst, packet);
if (ret)
iris_inst_change_state(inst, IRIS_INST_ERROR);
@@ -431,7 +815,15 @@ static int handle_session_response(struct iris_core *core,
}
}
+ if (dequeue) {
+ ret = handle_dequeue_buffers(inst);
+ if (ret)
+ goto unlock;
+ }
+
memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info));
+
+unlock:
mutex_unlock(&inst->lock);
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 6553029..04e7dc8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -170,6 +170,14 @@ bool allow_reqbufs(struct iris_inst *inst, u32 type)
(type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING);
}
+bool allow_qbuf(struct iris_inst *inst, u32 type)
+{
+ return (type == INPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_STREAMING);
+}
+
bool allow_streamon(struct iris_inst *inst, u32 type)
{
return (type == INPUT_MPLANE && inst->state == IRIS_INST_OPEN) ||
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 28d5380..47558ed 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -43,6 +43,7 @@ int iris_inst_change_state(struct iris_inst *inst,
bool allow_s_fmt(struct iris_inst *inst, u32 type);
bool allow_reqbufs(struct iris_inst *inst, u32 type);
+bool allow_qbuf(struct iris_inst *inst, u32 type);
bool allow_streamon(struct iris_inst *inst, u32 type);
bool allow_streamoff(struct iris_inst *inst, u32 type);
bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index d599366..f34434fc 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -95,6 +95,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
{
+ enum iris_buffer_type buf_type;
struct iris_inst *inst;
int ret = 0;
@@ -132,6 +133,16 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
if (ret)
goto error;
+ buf_type = v4l2_type_to_driver(q->type);
+ if (!buf_type) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ ret = queue_deferred_buffers(inst, buf_type);
+ if (ret)
+ goto error;
+
return ret;
error:
iris_inst_change_state(inst, IRIS_INST_ERROR);
@@ -168,9 +179,33 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)
iris_inst_change_state(inst, IRIS_INST_ERROR);
}
+void iris_vb2_buf_queue(struct vb2_buffer *vb2)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = vb2_get_drv_priv(vb2->vb2_queue);
+ if (!inst || !inst->core)
+ return;
+
+ if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ ret = vdec_qbuf(inst, vb2);
+
+exit:
+ if (ret) {
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ vb2_buffer_done(vb2, VB2_BUF_STATE_ERROR);
+ }
+}
+
void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
struct dma_buf *dbuf, unsigned long size)
{
+ struct iris_buffer *ro_buf, *dummy;
enum iris_buffer_type buf_type;
struct iris_buffers *buffers;
struct iris_buffer *iter;
@@ -203,6 +238,16 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
buf->inst = inst;
buf->dmabuf = dbuf;
+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ buf->attach = ro_buf->attach;
+ ro_buf->attach = NULL;
+ return buf;
+ }
+ }
+
buf->attach = dma_buf_attach(dbuf, dev);
if (IS_ERR(buf->attach)) {
buf->attach = NULL;
@@ -214,6 +259,7 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
int iris_vb2_map_dmabuf(void *buf_priv)
{
+ struct iris_buffer *ro_buf, *dummy;
struct iris_buffer *buf = buf_priv;
struct iris_core *core;
struct iris_inst *inst;
@@ -229,6 +275,17 @@ int iris_vb2_map_dmabuf(void *buf_priv)
return -EINVAL;
}
+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ buf->sg_table = ro_buf->sg_table;
+ buf->device_addr = ro_buf->device_addr;
+ ro_buf->sg_table = NULL;
+ return 0;
+ }
+ }
+
buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL);
if (IS_ERR(buf->sg_table))
return -EINVAL;
@@ -246,6 +303,7 @@ int iris_vb2_map_dmabuf(void *buf_priv)
void iris_vb2_unmap_dmabuf(void *buf_priv)
{
+ struct iris_buffer *ro_buf, *dummy;
struct iris_buffer *buf = buf_priv;
struct iris_core *core;
struct iris_inst *inst;
@@ -266,6 +324,17 @@ void iris_vb2_unmap_dmabuf(void *buf_priv)
return;
}
+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ ro_buf->sg_table = buf->sg_table;
+ buf->sg_table = NULL;
+ buf->device_addr = 0x0;
+ return;
+ }
+ }
+
if (buf->attach && buf->sg_table) {
dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
buf->sg_table = NULL;
@@ -275,6 +344,7 @@ void iris_vb2_unmap_dmabuf(void *buf_priv)
void iris_vb2_detach_dmabuf(void *buf_priv)
{
+ struct iris_buffer *ro_buf, *dummy;
struct iris_buffer *buf = buf_priv;
struct iris_core *core;
struct iris_inst *inst;
@@ -291,11 +361,22 @@ void iris_vb2_detach_dmabuf(void *buf_priv)
buf->sg_table = NULL;
}
+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ ro_buf->attach = buf->attach;
+ buf->attach = NULL;
+ goto exit;
+ }
+ }
+
if (buf->attach && buf->dmabuf) {
dma_buf_detach(buf->dmabuf, buf->attach);
buf->attach = NULL;
}
+exit:
buf->dmabuf = NULL;
buf->inst = NULL;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
index fc0e804..0757f01 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -13,6 +13,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int sizes[], struct device *alloc_devs[]);
int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count);
void iris_vb2_stop_streaming(struct vb2_queue *q);
+void iris_vb2_buf_queue(struct vb2_buffer *vb2);
/* vb2_mem_ops */
void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 0d8ca4b..7eb74c3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -1193,3 +1193,52 @@ int vdec_streamon_output(struct iris_inst *inst)
return ret;
}
+
+static int vb2_buffer_to_driver(struct vb2_buffer *vb2,
+ struct iris_buffer *buf)
+{
+ struct vb2_v4l2_buffer *vbuf;
+
+ if (!vb2 || !buf)
+ return -EINVAL;
+
+ vbuf = to_vb2_v4l2_buffer(vb2);
+
+ buf->fd = vb2->planes[0].m.fd;
+ buf->data_offset = vb2->planes[0].data_offset;
+ buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset;
+ buf->buffer_size = vb2->planes[0].length;
+ buf->timestamp = vb2->timestamp;
+ buf->flags = vbuf->flags;
+ buf->attr = 0;
+
+ return 0;
+}
+
+int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
+{
+ struct iris_buffer *buf = NULL;
+ int ret = 0;
+
+ buf = get_driver_buf(inst, vb2->type, vb2->index);
+ if (!buf)
+ return -EINVAL;
+
+ ret = vb2_buffer_to_driver(vb2, buf);
+ if (ret)
+ return ret;
+
+ if (!allow_qbuf(inst, vb2->type)) {
+ buf->attr |= BUF_ATTR_DEFERRED;
+ return 0;
+ }
+
+ ret = queue_buffer(inst, buf);
+ if (ret)
+ return ret;
+
+ if (vb2->type == OUTPUT_MPLANE)
+ ret = iris_release_nonref_buffers(inst);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index 0722da1..d666c54 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -18,5 +18,6 @@ int vdec_init_src_change_param(struct iris_inst *inst);
int vdec_src_change(struct iris_inst *inst);
int vdec_streamon_input(struct iris_inst *inst);
int vdec_streamon_output(struct iris_inst *inst);
+int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 14d0077..0165340 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -252,6 +252,7 @@ int vidc_close(struct file *filp)
iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
+ iris_destroy_buffers(inst);
vidc_remove_session(inst);
mutex_unlock(&inst->lock);
mutex_destroy(&inst->ctx_q_lock);
@@ -912,6 +913,7 @@ static const struct vb2_ops iris_vb2_ops = {
.queue_setup = iris_vb2_queue_setup,
.start_streaming = iris_vb2_start_streaming,
.stop_streaming = iris_vb2_stop_streaming,
+ .buf_queue = iris_vb2_buf_queue,
};
static struct vb2_mem_ops iris_vb2_mem_ops = {
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response
2023-12-18 11:32 ` [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response Dikshita Agarwal
@ 2023-12-19 12:21 ` Konrad Dybcio
2023-12-20 8:25 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-19 12:21 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Implement vb2 ops for buf queue. Below are the different
> buffer attributes:
> BUF_ATTR_DEFERRED - buffer queued by client but not submitted
> to firmware.
> BUF_ATTR_READ_ONLY - processed buffer received from firmware
> as read only. These buffers are held in firmware as reference
> for future frame processing.
> BUF_ATTR_PENDING_RELEASE - buffers requested to be released
> from firmware.
> BUF_ATTR_QUEUED - buffers submitted to firmware.
> BUF_ATTR_DEQUEUED - buffers received from firmware.
> BUF_ATTR_BUFFER_DONE - buffers sent back to vb2.
>
> Buffers are submitted and received via HFI_CMD_BUFFER.
> Firmware associates below flags during buffer response:
> HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware.
> HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware.
>
> Input buffers dequeued from firmware are sent directly to vb2.
>
> Output buffers if read only, are sent to vb2 and also maintained
> in read only list. If the same read only buffer is received form
> client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and
> submitted to firmware. Once the buffer is received from firmware
> as non read only, it is removed from read only list.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
>
> +enum iris_buffer_flags {
> + BUF_FLAG_KEYFRAME = 0x00000008,
> + BUF_FLAG_PFRAME = 0x00000010,
> + BUF_FLAG_BFRAME = 0x00000020,
> + BUF_FLAG_ERROR = 0x00000040,
BIT(3), 4, 5, 6?
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response
2023-12-19 12:21 ` Konrad Dybcio
@ 2023-12-20 8:25 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:25 UTC (permalink / raw)
To: Konrad Dybcio, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 5:51 PM, Konrad Dybcio wrote:
> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>> Implement vb2 ops for buf queue. Below are the different
>> buffer attributes:
>> BUF_ATTR_DEFERRED - buffer queued by client but not submitted
>> to firmware.
>> BUF_ATTR_READ_ONLY - processed buffer received from firmware
>> as read only. These buffers are held in firmware as reference
>> for future frame processing.
>> BUF_ATTR_PENDING_RELEASE - buffers requested to be released
>> from firmware.
>> BUF_ATTR_QUEUED - buffers submitted to firmware.
>> BUF_ATTR_DEQUEUED - buffers received from firmware.
>> BUF_ATTR_BUFFER_DONE - buffers sent back to vb2.
>>
>> Buffers are submitted and received via HFI_CMD_BUFFER.
>> Firmware associates below flags during buffer response:
>> HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware.
>> HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware.
>>
>> Input buffers dequeued from firmware are sent directly to vb2.
>>
>> Output buffers if read only, are sent to vb2 and also maintained
>> in read only list. If the same read only buffer is received form
>> client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and
>> submitted to firmware. Once the buffer is received from firmware
>> as non read only, it is removed from read only list.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
> [...]
>
>>
>> +enum iris_buffer_flags {
>> + BUF_FLAG_KEYFRAME = 0x00000008,
>> + BUF_FLAG_PFRAME = 0x00000010,
>> + BUF_FLAG_BFRAME = 0x00000020,
>> + BUF_FLAG_ERROR = 0x00000040,
> BIT(3), 4, 5, 6?
Sure, will update.
>
> Konrad
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 28/34] media: iris: add instance sub states and implement DRC and Drain sequence
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (26 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 29/34] media: iris: implement power management Dikshita Agarwal
` (12 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
From: Vikash Garodia <quic_vgarodia@quicinc.com>
Implement decoder commands for CMD_START and CMD_STOP.
HFI_CMD_DRAIN - set to firmware as part of stop command
to initiate drain.
HFI_CMD_RESUME - set to firmware as part of start command
to resume the buffer processing on specific plane.
Also, introduce different sub states for instances.
Sub state transitions are defined, based on which they would
be allowed/rejected/ignored.
- IRIS_INST_SUB_DRAIN - indicates drain is in progress.
- IRIS_INST_SUB_DRC - indicates source change is received from
firmware and source change event is sent to client.
- IRIS_INST_SUB_DRAIN_LAST - indicates last buffer is received
from firmware as part of drain sequence .
- IRIS_INST_SUB_DRC_LAST - indicates last buffer is received
from firmware as part of source change.
- IRIS_INST_SUB_INPUT_PAUSE - source change is received form
firmware. This indicates that firmware is paused to process
any further input frames.
- IRIS_INST_SUB_OUTPUT_PAUSE - last buffer is received form
firmware as part of drain or drc sequence . This indicates
that firmware is paused to process any further output frames.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 19 ++
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 56 +++++-
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 3 +-
.../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 118 ++++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 +-
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 106 ++++++++++-
.../platform/qcom/vcodec/iris/iris_instance.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 211 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_state.h | 24 +++
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 145 ++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 79 +++++++-
13 files changed, 760 insertions(+), 10 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 104ef9c..bbe8ca6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -16,7 +16,10 @@
#define HFI_CMD_CLOSE 0x01000004
#define HFI_CMD_START 0x01000005
#define HFI_CMD_STOP 0x01000006
+#define HFI_CMD_DRAIN 0x01000007
+#define HFI_CMD_RESUME 0x01000008
#define HFI_CMD_BUFFER 0x01000009
+#define HFI_CMD_PAUSE 0x01000011
#define HFI_PROP_IMAGE_VERSION 0x03000001
@@ -208,6 +211,20 @@ enum hfi_picture_type {
#define HFI_SYSTEM_ERROR_END 0x05FFFFFF
+#define HFI_INFORMATION_BEGIN 0x06000000
+
+#define HFI_INFO_UNSUPPORTED 0x06000001
+
+#define HFI_INFO_DATA_CORRUPT 0x06000002
+
+#define HFI_INFO_BUFFER_OVERFLOW 0x06000004
+
+#define HFI_INFO_HFI_FLAG_DRAIN_LAST 0x06000006
+
+#define HFI_INFO_HFI_FLAG_PSC_LAST 0x06000007
+
+#define HFI_INFORMATION_END 0x06FFFFFF
+
struct hfi_debug_header {
u32 size;
u32 debug_level;
@@ -242,6 +259,8 @@ enum hfi_buffer_firmware_flags {
HFI_BUF_FW_FLAG_NONE = 0x00000000,
HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001,
HFI_BUF_FW_FLAG_READONLY = 0x00000010,
+ HFI_BUF_FW_FLAG_LAST = 0x10000000,
+ HFI_BUF_FW_FLAG_PSC_LAST = 0x20000000,
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index ab25026..f85b52c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -547,7 +547,7 @@ static int iris_release_internal_buffers(struct iris_inst *inst,
return ret;
}
-int iris_release_input_internal_buffers(struct iris_inst *inst)
+static int iris_release_input_internal_buffers(struct iris_inst *inst)
{
int ret = 0;
u32 i = 0;
@@ -581,3 +581,57 @@ int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
return ret;
}
+
+int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_get_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_release_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_create_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_queue_input_internal_buffers(inst);
+
+ return ret;
+}
+
+int iris_alloc_and_queue_additional_dpb_buffers(struct iris_inst *inst)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buffer;
+ int cur_min_count = 0;
+ int ret;
+ int i;
+
+ ret = iris_get_internal_buf_info(inst, BUF_DPB);
+ if (ret)
+ return ret;
+
+ buffers = iris_get_buffer_list(inst, BUF_DPB);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry(buffer, &buffers->list, list)
+ cur_min_count++;
+
+ if (cur_min_count >= buffers->min_count)
+ return 0;
+
+ for (i = cur_min_count; i < buffers->min_count; i++) {
+ ret = iris_create_internal_buffers(inst, BUF_DPB);
+ if (ret)
+ return ret;
+ }
+
+ ret = iris_queue_internal_buffers(inst, BUF_DPB);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index 59c4a10..6ab5f22 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -57,8 +57,9 @@ int iris_destroy_internal_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_destroy_internal_buffers(struct iris_inst *inst,
u32 plane);
-int iris_release_input_internal_buffers(struct iris_inst *inst);
int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
enum iris_buffer_type buffer_type);
+int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst);
+int iris_alloc_and_queue_additional_dpb_buffers(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 5fd96d3..24b322a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -76,6 +76,7 @@ enum iris_buffer_flags {
BUF_FLAG_PFRAME = 0x00000010,
BUF_FLAG_BFRAME = 0x00000020,
BUF_FLAG_ERROR = 0x00000040,
+ BUF_FLAG_LAST = 0x00100000,
};
enum iris_buffer_attributes {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index e0da3be..2b3fa47 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -445,6 +445,124 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
return ret;
}
+int iris_hfi_pause(struct iris_inst *inst, u32 plane)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_PAUSE,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 payload)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_RESUME,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ HFI_PAYLOAD_U32,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_drain(struct iris_inst *inst, u32 plane)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_DRAIN,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ get_hfi_port(plane),
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
irqreturn_t iris_hfi_isr(int irq, void *data)
{
disable_irq_nosync(irq);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index f054c2d..ab479a9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -25,7 +25,9 @@ int iris_hfi_session_set_default_header(struct iris_inst *inst);
int iris_hfi_start(struct iris_inst *inst, u32 plane);
int iris_hfi_stop(struct iris_inst *inst, u32 plane);
-
+int iris_hfi_drain(struct iris_inst *inst, u32 plane);
+int iris_hfi_pause(struct iris_inst *inst, u32 plane);
+int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 cmd);
int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_hfi_release_buffer(struct iris_inst *inst,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index b1236dd..3548f9d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -103,6 +103,10 @@ static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags)
if (inst->hfi_frame_info.overflow)
driver_flags |= BUF_FLAG_ERROR;
+ if (hfi_flags & HFI_BUF_FW_FLAG_LAST ||
+ hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST)
+ driver_flags |= BUF_FLAG_LAST;
+
return driver_flags;
}
@@ -187,6 +191,46 @@ static int validate_hdr_packet(struct iris_core *core, struct hfi_header *hdr)
return ret;
}
+static int handle_session_info(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ struct iris_core *core;
+ int ret = 0;
+ char *info;
+
+ core = inst->core;
+
+ switch (pkt->type) {
+ case HFI_INFO_UNSUPPORTED:
+ info = "unsupported";
+ break;
+ case HFI_INFO_DATA_CORRUPT:
+ info = "data corrupt";
+ inst->hfi_frame_info.data_corrupt = 1;
+ break;
+ case HFI_INFO_BUFFER_OVERFLOW:
+ info = "buffer overflow";
+ inst->hfi_frame_info.overflow = 1;
+ break;
+ case HFI_INFO_HFI_FLAG_DRAIN_LAST:
+ info = "drain last flag";
+ ret = iris_inst_sub_state_change_drain_last(inst);
+ break;
+ case HFI_INFO_HFI_FLAG_PSC_LAST:
+ info = "drc last flag";
+ ret = iris_inst_sub_state_change_drc_last(inst);
+ break;
+ default:
+ info = "unknown";
+ break;
+ }
+
+ dev_dbg(core->dev, "session info received %#x: %s\n",
+ pkt->type, info);
+
+ return ret;
+}
+
static int handle_session_error(struct iris_inst *inst,
struct hfi_packet *pkt)
{
@@ -375,9 +419,21 @@ static int handle_output_buffer(struct iris_inst *inst,
bool found;
int ret = 0;
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) {
+ ret = iris_inst_sub_state_change_drain_last(inst);
+ if (ret)
+ return ret;
+ }
+
if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
return handle_release_output_buffer(inst, hfi_buffer);
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) {
+ ret = iris_inst_sub_state_change_drc_last(inst);
+ if (ret)
+ return ret;
+ }
+
if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
ret = handle_non_read_only_buffer(inst, hfi_buffer);
@@ -488,6 +544,26 @@ static int handle_release_internal_buffer(struct iris_inst *inst,
return ret;
}
+static int handle_session_stop(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+ enum signal_session_response signal_type = -1;
+
+ if (pkt->port == HFI_PORT_RAW) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ }
+
+ if (signal_type != -1)
+ signal_session_msg_receipt(inst, signal_type);
+
+ return ret;
+}
+
static int handle_session_buffer(struct iris_inst *inst,
struct hfi_packet *pkt)
{
@@ -549,18 +625,30 @@ static int handle_session_buffer(struct iris_inst *inst,
return ret;
}
+static int handle_session_drain(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRAIN)
+ ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_INPUT_PAUSE);
+
+ return ret;
+}
+
static int handle_src_change(struct iris_inst *inst,
struct hfi_packet *pkt)
{
- int ret = 0;
+ int ret;
- if (pkt->port == HFI_PORT_BITSTREAM)
- ret = vdec_src_change(inst);
- else if (pkt->port == HFI_PORT_RAW)
- ret = 0;
+ if (pkt->port != HFI_PORT_BITSTREAM)
+ return 0;
+ ret = iris_inst_sub_state_change_drc(inst);
if (ret)
- iris_inst_change_state(inst, IRIS_INST_ERROR);
+ return ret;
+
+ ret = vdec_src_change(inst);
return ret;
}
@@ -572,9 +660,14 @@ static int handle_session_command(struct iris_inst *inst,
static const struct iris_hfi_packet_handle hfi_pkt_handle[] = {
{HFI_CMD_OPEN, NULL },
{HFI_CMD_CLOSE, handle_session_close },
+ {HFI_CMD_START, NULL },
+ {HFI_CMD_STOP, handle_session_stop },
+ {HFI_CMD_DRAIN, handle_session_drain },
{HFI_CMD_BUFFER, handle_session_buffer },
{HFI_CMD_SETTINGS_CHANGE, handle_src_change },
{HFI_CMD_SUBSCRIBE_MODE, NULL },
+ {HFI_CMD_PAUSE, NULL },
+ {HFI_CMD_RESUME, NULL },
};
for (i = 0; i < ARRAY_SIZE(hfi_pkt_handle); i++) {
@@ -774,6 +867,7 @@ static int handle_session_response(struct iris_core *core,
int i, j;
static const struct iris_inst_hfi_range be[] = {
{HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, handle_session_error },
+ {HFI_INFORMATION_BEGIN, HFI_INFORMATION_END, handle_session_info },
{HFI_PROP_BEGIN, HFI_PROP_END, handle_session_property },
{HFI_CMD_BEGIN, HFI_CMD_END, handle_session_command },
};
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index cddf143..50154bf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -39,6 +39,7 @@
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
+ * @sub_state: instance sub state
* @ipsc_properties_set: boolean to set ipsc properties to fw
* @opsc_properties_set: boolean to set opsc properties to fw
* @hfi_frame_info: structure of frame info
@@ -71,6 +72,7 @@ struct iris_inst {
struct iris_buffers_info buffers;
u32 fw_min_count;
enum iris_inst_state state;
+ enum iris_inst_sub_state sub_state;
bool ipsc_properties_set;
bool opsc_properties_set;
struct iris_hfi_frame_info hfi_frame_info;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 04e7dc8..952ba2a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -239,3 +239,214 @@ int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane)
return iris_inst_change_state(inst, new_state);
}
+
+struct iris_inst_sub_state_change_allow {
+ enum iris_inst_state state;
+ enum state_change allow;
+ u32 sub_state_mask;
+};
+
+static int iris_inst_allow_sub_state(struct iris_inst *inst, enum iris_inst_sub_state sub_state)
+{
+ int i;
+ static struct iris_inst_sub_state_change_allow sub_state_allow[] = {
+ {IRIS_INST_OPEN, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_INPUT_PAUSE },
+
+ {IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_INPUT_PAUSE },
+
+ {IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_CLOSE, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_ERROR, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+ };
+
+ if (!sub_state)
+ return 0;
+
+ for (i = 0; i < ARRAY_SIZE(sub_state_allow); i++) {
+ if (sub_state_allow[i].state != inst->state)
+ continue;
+
+ if (sub_state_allow[i].allow == STATE_CHANGE_DISALLOW)
+ if (sub_state_allow[i].sub_state_mask & sub_state) {
+ dev_dbg(inst->core->dev, "state %d with disallowed sub state %x\n",
+ inst->state, sub_state);
+ return -EINVAL;
+ }
+
+ if (sub_state_allow[i].allow == STATE_CHANGE_ALLOW) {
+ if ((sub_state_allow[i].sub_state_mask & sub_state) != sub_state)
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+int iris_inst_change_sub_state(struct iris_inst *inst,
+ enum iris_inst_sub_state clear_sub_state,
+ enum iris_inst_sub_state set_sub_state)
+{
+ enum iris_inst_sub_state prev_sub_state;
+ int ret;
+
+ if (IS_SESSION_ERROR(inst))
+ return 0;
+
+ if (!clear_sub_state && !set_sub_state)
+ return 0;
+
+ if ((clear_sub_state & set_sub_state) ||
+ set_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE ||
+ clear_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE)
+ return -EINVAL;
+
+ prev_sub_state = inst->sub_state;
+
+ ret = iris_inst_allow_sub_state(inst, set_sub_state);
+ if (ret)
+ return ret;
+
+ inst->sub_state |= set_sub_state;
+ inst->sub_state &= ~clear_sub_state;
+
+ if (inst->sub_state != prev_sub_state)
+ dev_dbg(inst->core->dev, "state %d and sub state changed to %x\n",
+ inst->sub_state, inst->sub_state);
+
+ return 0;
+}
+
+int iris_inst_sub_state_change_drc(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = 0;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC)
+ return STATE_CHANGE_DISALLOW;
+
+ if (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_OPEN)
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ else
+ set_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_INPUT_PAUSE;
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+int iris_inst_sub_state_change_drain_last(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRAIN_LAST)
+ return STATE_CHANGE_DISALLOW;
+
+ if (!(inst->sub_state & IRIS_INST_SUB_DRAIN) ||
+ !(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE))
+ return STATE_CHANGE_DISALLOW;
+
+ set_sub_state = IRIS_INST_SUB_DRAIN_LAST | IRIS_INST_SUB_OUTPUT_PAUSE;
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+int iris_inst_sub_state_change_drc_last(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC_LAST)
+ return STATE_CHANGE_DISALLOW;
+
+ if (!(inst->sub_state & IRIS_INST_SUB_DRC) ||
+ !(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE))
+ return STATE_CHANGE_DISALLOW;
+
+ set_sub_state = IRIS_INST_SUB_DRC_LAST | IRIS_INST_SUB_OUTPUT_PAUSE;
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+
+ if (plane == INPUT_MPLANE) {
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ !(inst->sub_state & IRIS_INST_SUB_DRC_LAST))
+ return STATE_CHANGE_DISALLOW;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ !(inst->sub_state & IRIS_INST_SUB_DRAIN_LAST))
+ return STATE_CHANGE_DISALLOW;
+
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ } else {
+ set_sub_state = IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+bool is_drc_pending(struct iris_inst *inst)
+{
+ return inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST;
+}
+
+bool is_drain_pending(struct iris_inst *inst)
+{
+ return inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
+}
+
+bool allow_cmd(struct iris_inst *inst, u32 cmd)
+{
+ if (cmd == V4L2_DEC_CMD_START) {
+ if (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_OUTPUT_STREAMING ||
+ inst->state == IRIS_INST_STREAMING)
+ if (is_drc_pending(inst) || is_drain_pending(inst))
+ return true;
+ } else if (cmd == V4L2_DEC_CMD_STOP) {
+ if (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_STREAMING)
+ if (inst->sub_state != IRIS_INST_SUB_DRAIN)
+ return true;
+ }
+
+ return false;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 47558ed..76db8f5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -25,6 +25,19 @@ enum iris_inst_state {
IRIS_INST_ERROR,
};
+#define IRIS_INST_SUB_NONE 0
+#define IRIS_INST_SUB_STATES 6
+#define IRIS_INST_MAX_SUB_STATE_VALUE ((1 << IRIS_INST_SUB_STATES) - 1)
+
+enum iris_inst_sub_state {
+ IRIS_INST_SUB_DRAIN = BIT(0),
+ IRIS_INST_SUB_DRC = BIT(1),
+ IRIS_INST_SUB_DRAIN_LAST = BIT(2),
+ IRIS_INST_SUB_DRC_LAST = BIT(3),
+ IRIS_INST_SUB_INPUT_PAUSE = BIT(4),
+ IRIS_INST_SUB_OUTPUT_PAUSE = BIT(5),
+};
+
enum state_change {
STATE_CHANGE_ALLOW,
STATE_CHANGE_DISALLOW,
@@ -40,6 +53,9 @@ int iris_change_core_state(struct iris_core *core,
int iris_inst_change_state(struct iris_inst *inst,
enum iris_inst_state request_state);
+int iris_inst_change_sub_state(struct iris_inst *inst,
+ enum iris_inst_sub_state clear_sub_state,
+ enum iris_inst_sub_state set_sub_state);
bool allow_s_fmt(struct iris_inst *inst, u32 type);
bool allow_reqbufs(struct iris_inst *inst, u32 type);
@@ -51,4 +67,12 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);
int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane);
int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane);
+int iris_inst_sub_state_change_drc(struct iris_inst *inst);
+int iris_inst_sub_state_change_drain_last(struct iris_inst *inst);
+int iris_inst_sub_state_change_drc_last(struct iris_inst *inst);
+int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane);
+bool is_drc_pending(struct iris_inst *inst);
+bool is_drain_pending(struct iris_inst *inst);
+bool allow_cmd(struct iris_inst *inst, u32 cmd);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 7eb74c3..7884ba6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -1066,16 +1066,35 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst)
static int process_streamon_input(struct iris_inst *inst)
{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
int ret;
ret = iris_hfi_start(inst, INPUT_MPLANE);
if (ret)
return ret;
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0);
+ if (ret)
+ return ret;
+ }
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC ||
+ inst->sub_state & IRIS_INST_SUB_DRAIN) {
+ if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) {
+ ret = iris_hfi_pause(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
if (ret)
return ret;
+ ret = iris_inst_change_sub_state(inst, 0, set_sub_state);
+
return ret;
}
@@ -1127,16 +1146,51 @@ int vdec_streamon_input(struct iris_inst *inst)
static int process_streamon_output(struct iris_inst *inst)
{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ bool drain_pending = false;
int ret;
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST)
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_alloc_and_queue_input_int_bufs(inst);
+ if (ret)
+ return ret;
+ ret = set_stage(inst, STAGE);
+ if (ret)
+ return ret;
+ ret = set_pipe(inst, PIPE);
+ if (ret)
+ return ret;
+ }
+
+ drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
+
+ if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) {
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
ret = iris_hfi_start(inst, OUTPUT_MPLANE);
if (ret)
return ret;
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE)
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+
ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
if (ret)
return ret;
+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
return ret;
}
@@ -1242,3 +1296,94 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
return ret;
}
+
+static int process_resume(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ int ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ } else if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST;
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ }
+
+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
+ return ret;
+}
+
+int vdec_start_cmd(struct iris_inst *inst)
+{
+ int ret;
+
+ vb2_clear_last_buffer_dequeued(inst->vb2q_dst);
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST &&
+ inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_alloc_and_queue_input_int_bufs(inst);
+ if (ret)
+ return ret;
+
+ ret = set_stage(inst, STAGE);
+ if (ret)
+ return ret;
+
+ ret = set_pipe(inst, PIPE);
+ if (ret)
+ return ret;
+ }
+
+ ret = iris_alloc_and_queue_additional_dpb_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = queue_deferred_buffers(inst, BUF_OUTPUT);
+ if (ret)
+ return ret;
+
+ ret = process_resume(inst);
+
+ return ret;
+}
+
+int vdec_stop_cmd(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_drain(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_DRAIN);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index d666c54..974b95e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -19,5 +19,7 @@ int vdec_src_change(struct iris_inst *inst);
int vdec_streamon_input(struct iris_inst *inst);
int vdec_streamon_output(struct iris_inst *inst);
int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2);
+int vdec_start_cmd(struct iris_inst *inst);
+int vdec_stop_cmd(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 0165340..d37ef04 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -901,7 +901,82 @@ static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection *
return ret;
}
-static const struct v4l2_file_operations v4l2_file_ops = {
+static int vidc_try_dec_cmd(struct file *filp, void *fh,
+ struct v4l2_decoder_cmd *dec)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !dec)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (dec->cmd != V4L2_DEC_CMD_STOP && dec->cmd != V4L2_DEC_CMD_START) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ dec->flags = 0;
+ if (dec->cmd == V4L2_DEC_CMD_STOP) {
+ dec->stop.pts = 0;
+ } else if (dec->cmd == V4L2_DEC_CMD_START) {
+ dec->start.speed = 0;
+ dec->start.format = V4L2_DEC_START_FMT_NONE;
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_dec_cmd(struct file *filp, void *fh,
+ struct v4l2_decoder_cmd *dec)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !dec)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (dec->cmd != V4L2_DEC_CMD_START &&
+ dec->cmd != V4L2_DEC_CMD_STOP) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (inst->state == IRIS_INST_OPEN)
+ goto unlock;
+
+ if (!allow_cmd(inst, dec->cmd)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (dec->cmd == V4L2_DEC_CMD_START)
+ ret = vdec_start_cmd(inst);
+ else if (dec->cmd == V4L2_DEC_CMD_STOP)
+ ret = vdec_stop_cmd(inst);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static struct v4l2_file_operations v4l2_file_ops = {
.owner = THIS_MODULE,
.open = vidc_open,
.release = vidc_close,
@@ -950,6 +1025,8 @@ static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
.vidioc_subscribe_event = vidc_subscribe_event,
.vidioc_unsubscribe_event = vidc_unsubscribe_event,
.vidioc_g_selection = vidc_g_selection,
+ .vidioc_try_decoder_cmd = vidc_try_dec_cmd,
+ .vidioc_decoder_cmd = vidc_dec_cmd,
};
int init_ops(struct iris_core *core)
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 29/34] media: iris: implement power management
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (27 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 28/34] media: iris: add instance sub states and implement DRC and Drain sequence Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 12:24 ` Konrad Dybcio
2023-12-18 11:32 ` [PATCH v2 30/34] media: iris: register video encoder device to platform driver Dikshita Agarwal
` (11 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
From: Vikash Garodia <quic_vgarodia@quicinc.com>
Hardware specific power sequence include programming specific
sequence of registers. Implements this sequence for iris3.
Introduce video device suspend and resume using runtime PM.
Also, implement iris3 specific calculation for clock and
bus bandwidth which depends on hardware configuration,
codec format, resolution and frame rate.
Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 2 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 7 +
.../media/platform/qcom/vcodec/iris/iris_common.h | 7 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 66 +++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 175 +++++++-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 5 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 48 +++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 4 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 2 +
.../platform/qcom/vcodec/iris/iris_instance.h | 10 +
.../media/platform/qcom/vcodec/iris/iris_power.c | 173 ++++++++
.../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
.../media/platform/qcom/vcodec/iris/iris_probe.c | 118 +++++-
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 42 +-
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 7 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 36 +-
.../platform/qcom/vcodec/iris/platform_common.h | 18 +
.../media/platform/qcom/vcodec/iris/resources.c | 280 +++++++++++++
.../media/platform/qcom/vcodec/iris/resources.h | 8 +
.../media/platform/qcom/vcodec/iris/vpu_common.c | 87 ++++
.../media/platform/qcom/vcodec/iris/vpu_common.h | 10 +
.../media/platform/qcom/vcodec/iris/vpu_iris3.c | 454 ++++++++++++++++++---
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 97 +++++
.../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
26 files changed, 1614 insertions(+), 78 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 7e3d9f1..4c8f8f6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -13,10 +13,12 @@ iris-objs += iris_probe.o \
iris_hfi_response.o \
iris_hfi_packet.o \
iris_buffer.o \
+ iris_power.o \
resources.o \
vpu_common.o \
vpu_iris3.o \
vpu_iris3_buffer.o \
+ vpu_iris3_power.o \
platform_common.o \
platform_sm8550.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index bbe8ca6..872674e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -12,6 +12,7 @@
#define HFI_CMD_BEGIN 0x01000000
#define HFI_CMD_INIT 0x01000001
+#define HFI_CMD_POWER_COLLAPSE 0x01000002
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
#define HFI_CMD_START 0x01000005
@@ -46,10 +47,16 @@ enum hfi_property_mode_type {
#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001
+#define HFI_TRUE 0x00000001
+
+#define HFI_FALSE 0x00000000
+
#define HFI_PROP_BEGIN 0x03000000
#define HFI_PROP_IMAGE_VERSION 0x03000001
+#define HFI_PROP_INTRA_FRAME_POWER_COLLAPSE 0x03000002
+
enum hfi_codec_type {
HFI_CODEC_DECODE_AVC = 1,
HFI_CODEC_ENCODE_AVC = 2,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 24b322a..3ab4767 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -28,6 +28,8 @@
#define MAX_DPB_LIST_ARRAY_SIZE (16 * 4)
#define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4)
+#define INPUT_TIMER_LIST_SIZE 30
+
enum codec_type {
H264 = BIT(0),
HEVC = BIT(1),
@@ -128,4 +130,9 @@ struct iris_hfi_frame_info {
u32 overflow;
};
+struct iris_input_timer {
+ struct list_head list;
+ u64 time_us;
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 6452ea7..50553d2 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -101,6 +101,10 @@ struct iris_core {
struct list_head instances;
u32 intr_status;
char fw_version[IRIS_VERSION_LENGTH];
+ struct mutex pm_lock; /* lock for pm operations */
+ u32 skip_pc_count;
+ bool power_enabled;
+ struct iris_core_power power;
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index a5b8aef..395a189 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -3,12 +3,15 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include <linux/pm_runtime.h>
+
#include "hfi_defines.h"
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
#include "iris_instance.h"
+#include "iris_power.h"
#include "iris_vidc.h"
int check_core_lock(struct iris_core *core)
@@ -479,6 +482,8 @@ int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_typ
if (!buffers)
return -EINVAL;
+ iris_scale_power(inst);
+
list_for_each_entry(buf, &buffers->list, list) {
if (!(buf->attr & BUF_ATTR_DEFERRED))
continue;
@@ -775,6 +780,9 @@ int session_streamoff(struct iris_inst *inst, u32 plane)
if (ret)
goto error;
+ if (plane == INPUT_MPLANE)
+ iris_flush_input_timer(inst);
+
/* no more queued buffers after streamoff */
count = get_num_queued_buffers(inst, buffer_type);
if (count) {
@@ -797,3 +805,61 @@ int session_streamoff(struct iris_inst *inst, u32 plane)
return ret;
}
+
+int iris_pm_get(struct iris_core *core)
+{
+ int ret;
+
+ mutex_lock(&core->pm_lock);
+ ret = pm_runtime_resume_and_get(core->dev);
+ mutex_unlock(&core->pm_lock);
+
+ return ret;
+}
+
+int iris_pm_put(struct iris_core *core, bool autosuspend)
+{
+ int ret;
+
+ mutex_lock(&core->pm_lock);
+
+ if (autosuspend)
+ ret = pm_runtime_put_autosuspend(core->dev);
+ else
+ ret = pm_runtime_put_sync(core->dev);
+ if (ret > 0)
+ ret = 0;
+
+ mutex_unlock(&core->pm_lock);
+
+ return ret;
+}
+
+int iris_pm_get_put(struct iris_core *core)
+{
+ int ret = 0;
+
+ mutex_lock(&core->pm_lock);
+
+ if (pm_runtime_suspended(core->dev)) {
+ ret = pm_runtime_resume_and_get(core->dev);
+ if (ret < 0)
+ goto error;
+
+ ret = pm_runtime_put_autosuspend(core->dev);
+ }
+ if (ret > 0)
+ ret = 0;
+
+error:
+ mutex_unlock(&core->pm_lock);
+
+ return ret;
+}
+
+void iris_pm_touch(struct iris_core *core)
+{
+ mutex_lock(&core->pm_lock);
+ pm_runtime_mark_last_busy(core->dev);
+ mutex_unlock(&core->pm_lock);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 3a9889e..c628b2e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -51,5 +51,9 @@ int iris_vb2_buffer_done(struct iris_inst *inst,
int iris_release_nonref_buffers(struct iris_inst *inst);
void iris_destroy_buffers(struct iris_inst *inst);
int session_streamoff(struct iris_inst *inst, u32 plane);
+int iris_pm_get(struct iris_core *core);
+int iris_pm_put(struct iris_core *core, bool autosuspend);
+int iris_pm_get_put(struct iris_core *core);
+void iris_pm_touch(struct iris_core *core);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 2b3fa47..2850fd5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -67,6 +67,41 @@ static bool validate_session(struct iris_core *core,
return valid;
}
+static int iris_power_off(struct iris_core *core)
+{
+ int ret;
+
+ if (!core->power_enabled)
+ return 0;
+
+ ret = call_vpu_op(core, power_off, core);
+ if (ret) {
+ dev_err(core->dev, "Failed to power off, err: %d\n", ret);
+ return ret;
+ }
+ core->power_enabled = false;
+
+ return ret;
+}
+
+static int iris_power_on(struct iris_core *core)
+{
+ int ret;
+
+ if (core->power_enabled)
+ return 0;
+
+ ret = call_vpu_op(core, power_on, core);
+ if (ret) {
+ dev_err(core->dev, "Failed to power on, err: %d\n", ret);
+ return ret;
+ }
+
+ core->power_enabled = true;
+
+ return ret;
+}
+
static int sys_init(struct iris_core *core)
{
int ret;
@@ -113,20 +148,24 @@ int iris_hfi_core_init(struct iris_core *core)
if (ret)
goto error;
+ ret = iris_power_on(core);
+ if (ret)
+ goto error;
+
core->use_tz = use_tz(core->dev);
if (!core->use_tz)
- goto error;
+ goto error_power_off;
pas_id = core->platform_data->pas_id;
fw_name = core->platform_data->fwname;
ret = load_fw(core->dev, fw_name, &mem_phys, &mem_size,
pas_id, core->use_tz);
if (ret)
- goto error;
+ goto error_power_off;
ret = auth_reset_fw(pas_id);
if (ret)
- goto error;
+ goto error_power_off;
cp_start = core->cap[CP_START].value;
cp_size = core->cap[CP_SIZE].value;
@@ -135,22 +174,24 @@ int iris_hfi_core_init(struct iris_core *core)
ret = protect_secure_region(cp_start, cp_size, cp_nonpixel_start,
cp_nonpixel_size, pas_id);
if (ret)
- goto error;
+ goto error_power_off;
ret = call_vpu_op(core, boot_firmware, core);
if (ret)
- goto error;
+ goto error_power_off;
ret = sys_init(core);
if (ret)
- goto error;
+ goto error_power_off;
ret = sys_image_version(core);
if (ret)
- goto error;
+ goto error_power_off;
return ret;
+error_power_off:
+ iris_power_off(core);
error:
dev_err(core->dev, "%s(): failed with ret %d\n", __func__, ret);
@@ -170,6 +211,8 @@ int iris_hfi_core_deinit(struct iris_core *core)
unload_fw(core->platform_data->pas_id);
+ iris_power_off(core);
+
return ret;
}
@@ -573,16 +616,24 @@ irqreturn_t iris_hfi_isr(int irq, void *data)
irqreturn_t iris_hfi_isr_handler(int irq, void *data)
{
struct iris_core *core = data;
+ int ret;
if (!core)
return IRQ_NONE;
+ ret = iris_pm_get(core);
+ if (ret)
+ goto exit;
+
mutex_lock(&core->lock);
call_vpu_op(core, clear_interrupt, core);
mutex_unlock(&core->lock);
__response_handler(core);
+ iris_pm_put(core, true);
+
+exit:
if (!call_vpu_op(core, watchdog, core, core->intr_status))
enable_irq(irq);
@@ -701,3 +752,113 @@ int iris_hfi_release_buffer(struct iris_inst *inst,
return ret;
}
+
+int prepare_pc(struct iris_core *core)
+{
+ int ret;
+
+ ret = hfi_packet_sys_pc_prep(core, core->packet, core->packet_size);
+ if (ret)
+ goto err_pc_prep;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+ if (ret)
+ goto err_pc_prep;
+
+ return ret;
+
+err_pc_prep:
+ dev_err(core->dev, "Failed to prepare iris for power off\n");
+
+ return ret;
+}
+
+int iris_hfi_pm_suspend(struct iris_core *core)
+{
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ if (!core->power_enabled)
+ return 0;
+
+ if (core->skip_pc_count >= MAX_PC_SKIP_COUNT) {
+ dev_err(core->dev, "Failed to PC for %d times\n", core->skip_pc_count);
+ core->skip_pc_count = 0;
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ iris_core_deinit_locked(core);
+ return -EINVAL;
+ }
+
+ iris_flush_debug_queue(core, core->packet, core->packet_size);
+
+ ret = call_vpu_op(core, prepare_pc, core);
+ if (ret) {
+ core->skip_pc_count++;
+ iris_pm_touch(core);
+ return -EAGAIN;
+ }
+
+ ret = set_hw_state(false);
+ if (ret)
+ return ret;
+
+ ret = iris_power_off(core);
+ if (ret)
+ return ret;
+
+ core->skip_pc_count = 0;
+
+ return ret;
+}
+
+int iris_hfi_pm_resume(struct iris_core *core)
+{
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ if (core->power_enabled)
+ return 0;
+
+ ret = iris_power_on(core);
+ if (ret)
+ goto error;
+
+ ret = set_hw_state(true);
+ if (ret)
+ goto err_power_off;
+
+ ret = call_vpu_op(core, boot_firmware, core);
+ if (ret)
+ goto err_suspend_hw;
+
+ ret = hfi_packet_sys_interframe_powercollapse(core, core->packet, core->packet_size);
+ if (ret)
+ goto err_suspend_hw;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+ if (ret)
+ goto err_suspend_hw;
+
+ return ret;
+
+err_suspend_hw:
+ set_hw_state(false);
+err_power_off:
+ iris_power_off(core);
+error:
+ dev_err(core->dev, "Failed to Resume\n");
+
+ return -EBUSY;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index ab479a9..465bfc5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -9,6 +9,8 @@
#include "iris_instance.h"
#include "iris_core.h"
+#define MAX_PC_SKIP_COUNT 10
+
int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
@@ -32,6 +34,9 @@ int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_hfi_release_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
+int iris_hfi_pm_suspend(struct iris_core *core);
+int iris_hfi_pm_resume(struct iris_core *core);
+int prepare_pc(struct iris_core *core);
irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index 8dafd04..fab206cf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -622,3 +622,51 @@ int hfi_packet_session_property(struct iris_inst *inst,
return ret;
}
+
+int hfi_packet_sys_interframe_powercollapse(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ u32 payload = 0;
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0 /*session_id*/,
+ core->header_id++);
+ if (ret)
+ return ret;
+
+ payload = HFI_TRUE;
+
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_INTRA_FRAME_POWER_COLLAPSE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+
+ return ret;
+}
+
+int hfi_packet_sys_pc_prep(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0 /*session_id*/,
+ core->header_id++);
+ if (ret)
+ return ret;
+
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_CMD_POWER_COLLAPSE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_NONE,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ NULL, 0);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 8999c28..849b585 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -104,5 +104,9 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
int hfi_packet_session_property(struct iris_inst *inst,
u32 pkt_type, u32 flags, u32 port,
u32 payload_type, void *payload, u32 payload_size);
+int hfi_packet_sys_interframe_powercollapse(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);
+int hfi_packet_sys_pc_prep(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 3548f9d..a4d5d9c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -928,6 +928,8 @@ static int handle_response(struct iris_core *core, void *response)
struct hfi_header *hdr;
int ret;
+ iris_pm_touch(core);
+
hdr = (struct hfi_header *)response;
ret = validate_hdr_packet(core, hdr);
if (ret)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 50154bf..3c85e78 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -47,6 +47,11 @@
* @dst_subcr_params: subscription params to fw on output port
* @dpb_list_payload: array of dpb buffers
* @once_per_session_set: boolean to set once per session property
+ * @max_rate: max input rate
+ * @max_input_data_size: max size of input data
+ * @power: structure of power info
+ * @bus_data: structure of bus data
+ * @input_timer_list: list head of input timer
*/
struct iris_inst {
@@ -80,6 +85,11 @@ struct iris_inst {
struct subscription_params dst_subcr_params;
u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE];
bool once_per_session_set;
+ u32 max_rate;
+ u32 max_input_data_size;
+ struct iris_inst_power power;
+ struct bus_vote_data bus_data;
+ struct list_head input_timer_list;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.c b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
new file mode 100644
index 0000000..77439e3
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
@@ -0,0 +1,173 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_power.h"
+#include "iris_helpers.h"
+#include "resources.h"
+
+static int iris_set_buses(struct iris_inst *inst)
+{
+ struct iris_inst *instance;
+ struct iris_core *core;
+ u64 total_bw_ddr = 0;
+ int ret;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(instance, &core->instances, list) {
+ if (!instance->max_input_data_size)
+ continue;
+
+ total_bw_ddr += instance->power.bus_bw;
+ }
+
+ ret = vote_buses(core, total_bw_ddr);
+
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int iris_vote_buses(struct iris_inst *inst)
+{
+ struct v4l2_format *out_f, *inp_f;
+ struct bus_vote_data *vote_data;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ vote_data = &inst->bus_data;
+
+ out_f = inst->fmt_dst;
+ inp_f = inst->fmt_src;
+
+ vote_data->width = inp_f->fmt.pix_mp.width;
+ vote_data->height = inp_f->fmt.pix_mp.height;
+ vote_data->fps = inst->max_rate;
+
+ if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) {
+ vote_data->color_formats[0] = V4L2_PIX_FMT_NV12;
+ vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat;
+ } else {
+ vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat;
+ }
+
+ call_session_op(core, calc_bw, inst, vote_data);
+
+ inst->power.bus_bw = vote_data->bus_bw;
+
+ return iris_set_buses(inst);
+}
+
+static int iris_set_clocks(struct iris_inst *inst)
+{
+ struct iris_inst *instance;
+ struct iris_core *core;
+ int ret = 0;
+ u64 freq;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+
+ freq = 0;
+ list_for_each_entry(instance, &core->instances, list) {
+ if (!instance->max_input_data_size)
+ continue;
+
+ freq += instance->power.min_freq;
+ }
+
+ core->power.clk_freq = freq;
+
+ ret = opp_set_rate(core, freq);
+
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int iris_scale_clocks(struct iris_inst *inst)
+{
+ struct iris_buffer *vbuf;
+ struct iris_core *core;
+ u32 data_size = 0;
+
+ core = inst->core;
+
+ list_for_each_entry(vbuf, &inst->buffers.input.list, list)
+ data_size = max(data_size, vbuf->data_size);
+
+ inst->max_input_data_size = data_size;
+
+ inst->max_rate = inst->cap[QUEUED_RATE].value >> 16;
+
+ if (!inst->max_input_data_size)
+ return 0;
+
+ inst->power.min_freq = call_session_op(core, calc_freq, inst,
+ inst->max_input_data_size);
+ iris_set_clocks(inst);
+
+ return 0;
+}
+
+int iris_scale_power(struct iris_inst *inst)
+{
+ iris_scale_clocks(inst);
+ iris_vote_buses(inst);
+
+ return 0;
+}
+
+int iris_update_input_rate(struct iris_inst *inst, u64 time_us)
+{
+ struct iris_input_timer *prev_timer = NULL;
+ struct iris_input_timer *input_timer;
+ u64 input_timer_sum_us = 0;
+ u64 counter = 0;
+
+ input_timer = kzalloc(sizeof(*input_timer), GFP_KERNEL);
+ if (!input_timer)
+ return -ENOMEM;
+
+ input_timer->time_us = time_us;
+ INIT_LIST_HEAD(&input_timer->list);
+ list_add_tail(&input_timer->list, &inst->input_timer_list);
+ list_for_each_entry(input_timer, &inst->input_timer_list, list) {
+ if (prev_timer) {
+ input_timer_sum_us += input_timer->time_us - prev_timer->time_us;
+ counter++;
+ }
+ prev_timer = input_timer;
+ }
+
+ if (input_timer_sum_us && counter >= INPUT_TIMER_LIST_SIZE)
+ inst->cap[QUEUED_RATE].value =
+ (s32)(DIV64_U64_ROUND_CLOSEST(counter * 1000000,
+ input_timer_sum_us) << 16);
+
+ if (counter >= INPUT_TIMER_LIST_SIZE) {
+ input_timer = list_first_entry(&inst->input_timer_list,
+ struct iris_input_timer, list);
+ list_del_init(&input_timer->list);
+ kfree(input_timer);
+ }
+
+ return 0;
+}
+
+int iris_flush_input_timer(struct iris_inst *inst)
+{
+ struct iris_input_timer *input_timer, *dummy_timer;
+
+ list_for_each_entry_safe(input_timer, dummy_timer, &inst->input_timer_list, list) {
+ list_del_init(&input_timer->list);
+ kfree(input_timer);
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.h b/drivers/media/platform/qcom/vcodec/iris/iris_power.h
new file mode 100644
index 0000000..28dbd5f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_POWER_H_
+#define _IRIS_POWER_H_
+
+#include "iris_instance.h"
+
+int iris_scale_power(struct iris_inst *inst);
+int iris_update_input_rate(struct iris_inst *inst, u64 time_us);
+int iris_flush_input_timer(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index bf484a3..2571e27 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -6,14 +6,15 @@
#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
#include "../hfi_queue.h"
#include "iris_core.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
-#include "resources.h"
#include "iris_vidc.h"
-#include "iris_ctrls.h"
+#include "resources.h"
static int init_iris_isr(struct iris_core *core)
{
@@ -70,6 +71,8 @@ static void iris_remove(struct platform_device *pdev)
if (!core)
return;
+ iris_pm_get(core);
+
iris_core_deinit(core);
hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
&core->command_queue, &core->message_queue,
@@ -79,7 +82,10 @@ static void iris_remove(struct platform_device *pdev)
v4l2_device_unregister(&core->v4l2_dev);
+ iris_pm_put(core, false);
+
mutex_destroy(&core->lock);
+ mutex_destroy(&core->pm_lock);
core->state = IRIS_CORE_DEINIT;
}
@@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev)
core->state = IRIS_CORE_DEINIT;
mutex_init(&core->lock);
+ mutex_init(&core->pm_lock);
core->packet_size = IFACEQ_CORE_PKT_SIZE;
core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL);
@@ -117,58 +124,66 @@ static int iris_probe(struct platform_device *pdev)
if (core->irq < 0)
return core->irq;
+ pm_runtime_set_autosuspend_delay(core->dev, AUTOSUSPEND_DELAY_VALUE);
+ pm_runtime_use_autosuspend(core->dev);
+ ret = devm_pm_runtime_enable(core->dev);
+ if (ret) {
+ dev_err(core->dev, "failed to enable runtime pm\n");
+ goto err_runtime_disable;
+ }
+
ret = init_iris_isr(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: Failed to init isr with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}
ret = init_platform(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init platform failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}
ret = init_vpu(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init vpu failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}
ret = init_ops(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init ops failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}
ret = init_resources(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init resource failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}
ret = iris_init_core_caps(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init core caps failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}
ret = iris_init_instance_caps(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init inst caps failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}
ret = v4l2_device_register(dev, &core->v4l2_dev);
if (ret)
- return ret;
+ goto err_runtime_disable;
ret = iris_register_video_device(core);
if (ret)
@@ -194,14 +209,29 @@ static int iris_probe(struct platform_device *pdev)
goto err_vdev_unreg;
}
+ ret = iris_pm_get(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret, "%s: failed to get runtime pm\n", __func__);
+ goto err_queue_deinit;
+ }
+
ret = iris_core_init(core);
if (ret) {
dev_err_probe(core->dev, ret, "%s: core init failed\n", __func__);
goto err_queue_deinit;
}
+ ret = iris_pm_put(core, false);
+ if (ret) {
+ pm_runtime_get_noresume(core->dev);
+ dev_err_probe(core->dev, ret, "%s: failed to put runtime pm\n", __func__);
+ goto err_core_deinit;
+ }
+
return ret;
+err_core_deinit:
+ iris_core_deinit(core);
err_queue_deinit:
hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
&core->command_queue, &core->message_queue,
@@ -210,10 +240,77 @@ static int iris_probe(struct platform_device *pdev)
video_unregister_device(core->vdev_dec);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
+err_runtime_disable:
+ pm_runtime_put_noidle(core->dev);
+ pm_runtime_set_suspended(core->dev);
return ret;
}
+static int iris_pm_suspend(struct device *dev)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!dev || !dev->driver)
+ return 0;
+
+ core = dev_get_drvdata(dev);
+
+ mutex_lock(&core->lock);
+ if (!core_in_valid_state(core)) {
+ ret = 0;
+ goto unlock;
+ }
+
+ if (!core->power_enabled) {
+ ret = 0;
+ goto unlock;
+ }
+
+ ret = iris_hfi_pm_suspend(core);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int iris_pm_resume(struct device *dev)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!dev || !dev->driver)
+ return 0;
+
+ core = dev_get_drvdata(dev);
+
+ mutex_lock(&core->lock);
+ if (!core_in_valid_state(core)) {
+ ret = 0;
+ goto unlock;
+ }
+
+ if (core->power_enabled) {
+ ret = 0;
+ goto unlock;
+ }
+
+ ret = iris_hfi_pm_resume(core);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static const struct dev_pm_ops iris_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(iris_pm_suspend, iris_pm_resume, NULL)
+};
+
static const struct of_device_id iris_dt_match[] = {
{
.compatible = "qcom,sm8550-iris",
@@ -229,6 +326,7 @@ static struct platform_driver qcom_iris_driver = {
.driver = {
.name = "qcom-iris",
.of_match_table = iris_dt_match,
+ .pm = &iris_pm_ops,
},
};
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index f34434fc..c0878f1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -10,6 +10,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_instance.h"
+#include "iris_power.h"
#include "iris_vb2.h"
#include "iris_vdec.h"
@@ -111,39 +112,52 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
goto error;
}
+ ret = iris_pm_get(inst->core);
+ if (ret)
+ goto error;
+
if (!inst->once_per_session_set) {
inst->once_per_session_set = true;
ret = iris_hfi_session_set_codec(inst);
if (ret)
- goto error;
+ goto err_pm_get;
ret = iris_hfi_session_set_default_header(inst);
if (ret)
- goto error;
+ goto err_pm_get;
ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
if (ret)
- goto error;
+ goto err_pm_get;
}
+ iris_scale_power(inst);
+
if (q->type == INPUT_MPLANE)
ret = vdec_streamon_input(inst);
else if (q->type == OUTPUT_MPLANE)
ret = vdec_streamon_output(inst);
if (ret)
- goto error;
+ goto err_pm_get;
buf_type = v4l2_type_to_driver(q->type);
if (!buf_type) {
ret = -EINVAL;
- goto error;
+ goto err_pm_get;
}
ret = queue_deferred_buffers(inst, buf_type);
if (ret)
+ goto err_pm_get;
+
+ ret = iris_pm_put(inst->core, true);
+ if (ret)
goto error;
return ret;
+
+err_pm_get:
+ iris_pm_put(inst->core, false);
error:
iris_inst_change_state(inst, IRIS_INST_ERROR);
@@ -165,6 +179,10 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)
if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE)
goto error;
+ ret = iris_pm_get_put(inst->core);
+ if (ret)
+ goto error;
+
if (q->type == INPUT_MPLANE)
ret = session_streamoff(inst, INPUT_MPLANE);
else if (q->type == OUTPUT_MPLANE)
@@ -181,6 +199,8 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)
void iris_vb2_buf_queue(struct vb2_buffer *vb2)
{
+ u64 ktime_ns = ktime_get_ns();
+ struct iris_core *core;
struct iris_inst *inst;
int ret;
@@ -188,11 +208,23 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2)
if (!inst || !inst->core)
return;
+ core = inst->core;
+
if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) {
ret = -EINVAL;
goto exit;
}
+ if (vb2->type == INPUT_MPLANE) {
+ ret = iris_update_input_rate(inst, div_u64(ktime_ns, 1000));
+ if (ret)
+ goto exit;
+ }
+
+ ret = iris_pm_get_put(core);
+ if (ret)
+ goto exit;
+
ret = vdec_qbuf(inst, vb2);
exit:
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 7884ba6..371615e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -12,6 +12,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
+#include "iris_power.h"
#include "iris_vdec.h"
#define UNSPECIFIED_COLOR_FORMAT 5
@@ -1069,6 +1070,8 @@ static int process_streamon_input(struct iris_inst *inst)
enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
int ret;
+ iris_scale_power(inst);
+
ret = iris_hfi_start(inst, INPUT_MPLANE);
if (ret)
return ret;
@@ -1150,6 +1153,8 @@ static int process_streamon_output(struct iris_inst *inst)
bool drain_pending = false;
int ret;
+ iris_scale_power(inst);
+
if (inst->sub_state & IRIS_INST_SUB_DRC &&
inst->sub_state & IRIS_INST_SUB_DRC_LAST)
clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
@@ -1287,6 +1292,8 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
return 0;
}
+ iris_scale_power(inst);
+
ret = queue_buffer(inst, buf);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index d37ef04..437d6b4 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -9,12 +9,13 @@
#include "iris_buffer.h"
#include "iris_common.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_instance.h"
+#include "iris_power.h"
#include "iris_vdec.h"
#include "iris_vidc.h"
-#include "iris_ctrls.h"
#include "iris_vb2.h"
#define VIDC_DRV_NAME "iris_driver"
@@ -162,17 +163,23 @@ int vidc_open(struct file *filp)
int i = 0;
int ret;
- ret = iris_core_init(core);
+ ret = iris_pm_get(core);
if (ret)
return ret;
+ ret = iris_core_init(core);
+ if (ret)
+ goto fail_pm_put;
+
ret = iris_core_init_wait(core);
if (ret)
- return ret;
+ goto fail_pm_put;
inst = kzalloc(sizeof(*inst), GFP_KERNEL);
- if (!inst)
- return -ENOMEM;
+ if (!inst) {
+ ret = -ENOMEM;
+ goto fail_pm_put;
+ }
inst->core = core;
inst->session_id = hash32_ptr(inst);
@@ -198,6 +205,7 @@ int vidc_open(struct file *filp)
INIT_LIST_HEAD(&inst->buffers.persist.list);
INIT_LIST_HEAD(&inst->buffers.vpss.list);
INIT_LIST_HEAD(&inst->caps_list);
+ INIT_LIST_HEAD(&inst->input_timer_list);
for (i = 0; i < MAX_SIGNAL; i++)
init_completion(&inst->completions[i]);
@@ -211,11 +219,16 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_inst_deinit;
+ iris_scale_power(inst);
+
ret = iris_hfi_session_open(inst);
if (ret) {
dev_err(core->dev, "%s: session open failed\n", __func__);
goto fail_core_deinit;
}
+
+ iris_pm_put(core, true);
+
filp->private_data = &inst->fh;
return 0;
@@ -233,6 +246,8 @@ int vidc_open(struct file *filp)
mutex_destroy(&inst->ctx_q_lock);
mutex_destroy(&inst->lock);
kfree(inst);
+fail_pm_put:
+ iris_pm_put(core, false);
return ret;
}
@@ -240,20 +255,25 @@ int vidc_open(struct file *filp)
int vidc_close(struct file *filp)
{
struct iris_inst *inst;
+ struct iris_core *core;
inst = get_vidc_inst(filp, NULL);
if (!inst)
return -EINVAL;
+ core = inst->core;
+
v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
mutex_lock(&inst->lock);
+ iris_pm_get(core);
close_session(inst);
iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
iris_destroy_buffers(inst);
vidc_remove_session(inst);
+ iris_pm_put(core, false);
mutex_unlock(&inst->lock);
mutex_destroy(&inst->ctx_q_lock);
mutex_destroy(&inst->lock);
@@ -965,11 +985,17 @@ static int vidc_dec_cmd(struct file *filp, void *fh,
goto unlock;
}
+ ret = iris_pm_get(inst->core);
+ if (ret)
+ goto unlock;
+
if (dec->cmd == V4L2_DEC_CMD_START)
ret = vdec_start_cmd(inst);
else if (dec->cmd == V4L2_DEC_CMD_STOP)
ret = vdec_stop_cmd(inst);
+ iris_pm_put(inst->core, true);
+
unlock:
mutex_unlock(&inst->lock);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 22a8f5b..81de610 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -15,6 +15,7 @@ struct iris_core;
struct iris_inst;
#define HW_RESPONSE_TIMEOUT_VALUE (1000)
+#define AUTOSUSPEND_DELAY_VALUE (HW_RESPONSE_TIMEOUT_VALUE + 500)
#define BIT_DEPTH_8 (8 << 16 | 8)
#define BIT_DEPTH_10 (10 << 16 | 10)
@@ -64,6 +65,11 @@ struct reg_preset_info {
u32 mask;
};
+struct iris_core_power {
+ u64 clk_freq;
+ u64 bus_bw;
+};
+
struct ubwc_config_data {
u32 max_channels;
u32 mal_length;
@@ -74,6 +80,18 @@ struct ubwc_config_data {
u32 bank_spreading;
};
+struct bus_vote_data {
+ u32 color_formats[2];
+ int height, width;
+ u32 fps;
+ u64 bus_bw;
+};
+
+struct iris_inst_power {
+ u64 min_freq;
+ u32 bus_bw;
+};
+
enum plat_core_cap_type {
CORE_CAP_NONE = 0,
DEC_CODECS,
diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c
index 8cfdcd6..c9e3754 100644
--- a/drivers/media/platform/qcom/vcodec/iris/resources.c
+++ b/drivers/media/platform/qcom/vcodec/iris/resources.c
@@ -4,6 +4,8 @@
*/
#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
#include <linux/interconnect.h>
#include <linux/pm_domain.h>
#include <linux/pm_opp.h>
@@ -15,6 +17,8 @@
#include "platform_common.h"
#include "resources.h"
+#define BW_THRESHOLD 50000
+
static void iris_pd_release(void *res)
{
struct device *pd = (struct device *)res;
@@ -59,6 +63,34 @@ static int iris_opp_dl_get(struct device *dev, struct device *supplier)
return ret;
}
+int opp_set_rate(struct iris_core *core, u64 freq)
+{
+ unsigned long opp_freq = 0;
+ struct dev_pm_opp *opp;
+ int ret;
+
+ opp_freq = freq;
+
+ opp = dev_pm_opp_find_freq_ceil(core->dev, &opp_freq);
+ if (IS_ERR(opp)) {
+ opp = dev_pm_opp_find_freq_floor(core->dev, &opp_freq);
+ if (IS_ERR(opp)) {
+ dev_err(core->dev,
+ "unable to find freq %lld in opp table\n", freq);
+ return -EINVAL;
+ }
+ }
+ dev_pm_opp_put(opp);
+
+ ret = dev_pm_opp_set_rate(core->dev, opp_freq);
+ if (ret) {
+ dev_err(core->dev, "failed to set rate\n");
+ return ret;
+ }
+
+ return ret;
+}
+
static int init_bus(struct iris_core *core)
{
const struct bus_info *bus_tbl;
@@ -135,6 +167,10 @@ static int init_power_domains(struct iris_core *core)
}
}
+ ret = devm_pm_opp_set_clkname(core->dev, "vcodec_core");
+ if (ret)
+ return ret;
+
ret = devm_pm_opp_of_add_table(core->dev);
if (ret) {
dev_err(core->dev, "%s: failed to add opp table\n", __func__);
@@ -144,6 +180,56 @@ static int init_power_domains(struct iris_core *core)
return ret;
}
+int enable_power_domains(struct iris_core *core, const char *name)
+{
+ struct power_domain_info *pdinfo = NULL;
+ int ret;
+ u32 i;
+
+ ret = opp_set_rate(core, ULONG_MAX);
+ if (ret)
+ return ret;
+
+ core->pd_count = core->platform_data->pd_tbl_size;
+ for (i = 0; i < (core->pd_count - 1); i++) {
+ pdinfo = &core->power_domain_tbl[i];
+ if (strcmp(pdinfo->name, name))
+ continue;
+ ret = pm_runtime_get_sync(pdinfo->genpd_dev);
+ if (ret < 0)
+ return ret;
+ }
+
+ ret = opp_set_rate(core, ULONG_MAX);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+int disable_power_domains(struct iris_core *core, const char *name)
+{
+ struct power_domain_info *pdinfo = NULL;
+ int ret;
+ u32 i;
+
+ ret = opp_set_rate(core, 0);
+ if (ret)
+ return ret;
+
+ core->pd_count = core->platform_data->pd_tbl_size;
+ for (i = 0; i < (core->pd_count - 1); i++) {
+ pdinfo = &core->power_domain_tbl[i];
+ if (strcmp(pdinfo->name, name))
+ continue;
+ ret = pm_runtime_put_sync(pdinfo->genpd_dev);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
static int init_clocks(struct iris_core *core)
{
const struct clock_info *clk_tbl;
@@ -204,6 +290,200 @@ static int init_reset_clocks(struct iris_core *core)
return 0;
}
+int unvote_buses(struct iris_core *core)
+{
+ struct bus_info *bus = NULL;
+ int ret = 0;
+ u32 i;
+
+ core->power.bus_bw = 0;
+ core->bus_count = core->platform_data->bus_tbl_size;
+
+ for (i = 0; i < core->bus_count; i++) {
+ bus = &core->bus_tbl[i];
+ if (!bus->icc)
+ return -EINVAL;
+
+ ret = icc_set_bw(bus->icc, 0, 0);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int vote_buses(struct iris_core *core, unsigned long bus_bw)
+{
+ unsigned long bw_kbps = 0, bw_prev = 0;
+ struct bus_info *bus = NULL;
+ int ret = 0;
+ u32 i;
+
+ core->bus_count = core->platform_data->bus_tbl_size;
+
+ for (i = 0; i < core->bus_count; i++) {
+ bus = &core->bus_tbl[i];
+ if (bus && bus->icc) {
+ if (!strcmp(bus->name, "iris-ddr")) {
+ bw_kbps = bus_bw;
+ bw_prev = core->power.bus_bw;
+ } else {
+ bw_kbps = bus->bw_max_kbps;
+ bw_prev = core->power.bus_bw ?
+ bw_kbps : 0;
+ }
+
+ bw_kbps = clamp_t(typeof(bw_kbps), bw_kbps,
+ bus->bw_min_kbps, bus->bw_max_kbps);
+
+ if (abs(bw_kbps - bw_prev) < BW_THRESHOLD && bw_prev)
+ continue;
+
+ ret = icc_set_bw(bus->icc, bw_kbps, 0);
+ if (ret)
+ return ret;
+
+ if (!strcmp(bus->name, "iris-ddr"))
+ core->power.bus_bw = bw_kbps;
+ }
+ }
+
+ return ret;
+}
+
+static int deassert_reset_control(struct iris_core *core)
+{
+ struct reset_info *rcinfo = NULL;
+ int ret = 0;
+ u32 i;
+
+ core->reset_count = core->platform_data->clk_rst_tbl_size;
+
+ for (i = 0; i < (core->reset_count - 1); i++) {
+ rcinfo = &core->reset_tbl[i];
+ ret = reset_control_deassert(rcinfo->rst);
+ if (ret) {
+ dev_err(core->dev, "deassert reset control failed. ret = %d\n", ret);
+ continue;
+ }
+ }
+
+ return ret;
+}
+
+static int assert_reset_control(struct iris_core *core)
+{
+ struct reset_info *rcinfo = NULL;
+ int ret = 0, cnt = 0;
+ u32 i;
+
+ core->reset_count = core->platform_data->clk_rst_tbl_size;
+
+ for (i = 0; i < (core->reset_count - 1); i++) {
+ rcinfo = &core->reset_tbl[i];
+ if (!rcinfo->rst)
+ return -EINVAL;
+
+ ret = reset_control_assert(rcinfo->rst);
+ if (ret) {
+ dev_err(core->dev, "failed to assert reset control %s, ret = %d\n",
+ rcinfo->name, ret);
+ goto deassert_reset_control;
+ }
+ cnt++;
+
+ usleep_range(1000, 1100);
+ }
+
+ return ret;
+deassert_reset_control:
+ for (i = 0; i < cnt; i++) {
+ rcinfo = &core->reset_tbl[i];
+ reset_control_deassert(rcinfo->rst);
+ }
+
+ return ret;
+}
+
+int reset_ahb2axi_bridge(struct iris_core *core)
+{
+ int ret;
+
+ ret = assert_reset_control(core);
+ if (ret)
+ return ret;
+
+ ret = deassert_reset_control(core);
+
+ return ret;
+}
+
+int disable_unprepare_clock(struct iris_core *core, const char *clk_name)
+{
+ struct clock_info *cl;
+ bool found = false;
+ u32 i;
+
+ core->clk_count = core->platform_data->clk_tbl_size;
+
+ for (i = 0; i < core->clk_count; i++) {
+ cl = &core->clock_tbl[i];
+ if (!cl->clk)
+ return -EINVAL;
+
+ if (strcmp(cl->name, clk_name))
+ continue;
+
+ found = true;
+ clk_disable_unprepare(cl->clk);
+ cl->prev = 0;
+ break;
+ }
+
+ if (!found)
+ return -EINVAL;
+
+ return 0;
+}
+
+int prepare_enable_clock(struct iris_core *core, const char *clk_name)
+{
+ struct clock_info *cl;
+ bool found = false;
+ int ret = 0;
+ u32 i;
+
+ core->clk_count = core->platform_data->clk_tbl_size;
+
+ for (i = 0; i < core->clk_count; i++) {
+ cl = &core->clock_tbl[i];
+ if (!cl->clk)
+ return -EINVAL;
+
+ if (strcmp(cl->name, clk_name))
+ continue;
+
+ found = true;
+
+ ret = clk_prepare_enable(cl->clk);
+ if (ret) {
+ dev_err(core->dev, "failed to enable clock %s\n", cl->name);
+ return ret;
+ }
+
+ if (!__clk_is_enabled(cl->clk)) {
+ clk_disable_unprepare(cl->clk);
+ return -EINVAL;
+ }
+ break;
+ }
+
+ if (!found)
+ return -EINVAL;
+
+ return ret;
+}
+
int init_resources(struct iris_core *core)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h
index d21bcc7e..09857ed 100644
--- a/drivers/media/platform/qcom/vcodec/iris/resources.h
+++ b/drivers/media/platform/qcom/vcodec/iris/resources.h
@@ -31,6 +31,14 @@ struct reset_info {
const char *name;
};
+int enable_power_domains(struct iris_core *core, const char *name);
+int disable_power_domains(struct iris_core *core, const char *name);
+int unvote_buses(struct iris_core *core);
+int vote_buses(struct iris_core *core, unsigned long bus_bw);
+int reset_ahb2axi_bridge(struct iris_core *core);
+int opp_set_rate(struct iris_core *core, u64 freq);
+int disable_unprepare_clock(struct iris_core *core, const char *clk_name);
+int prepare_enable_clock(struct iris_core *core, const char *clk_name);
int init_resources(struct iris_core *core);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
index 3282510..8ef1029 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
@@ -3,6 +3,8 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include <linux/iopoll.h>
+
#include "vpu_iris3.h"
#include "iris_core.h"
#include "iris_helpers.h"
@@ -17,6 +19,9 @@ int write_register(struct iris_core *core, u32 reg, u32 value)
if (ret)
return ret;
+ if (!core->power_enabled)
+ return -EINVAL;
+
base_addr = core->reg_base;
base_addr += reg;
writel_relaxed(value, base_addr);
@@ -27,10 +32,46 @@ int write_register(struct iris_core *core, u32 reg, u32 value)
return ret;
}
+int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask)
+{
+ void __iomem *base_addr;
+ u32 prev_val, new_val;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core->power_enabled)
+ return -EINVAL;
+
+ base_addr = core->reg_base;
+ base_addr += reg;
+
+ prev_val = readl_relaxed(base_addr);
+ /*
+ * Memory barrier to ensure register read is correct
+ */
+ rmb();
+
+ new_val = (prev_val & ~mask) | (value & mask);
+
+ writel_relaxed(new_val, base_addr);
+ /*
+ * Memory barrier to make sure value is written into the register.
+ */
+ wmb();
+
+ return ret;
+}
+
int read_register(struct iris_core *core, u32 reg, u32 *value)
{
void __iomem *base_addr;
+ if (!core->power_enabled)
+ return -EINVAL;
+
base_addr = core->reg_base;
*value = readl_relaxed(base_addr + reg);
@@ -41,6 +82,52 @@ int read_register(struct iris_core *core, u32 reg, u32 *value)
return 0;
}
+int read_register_with_poll_timeout(struct iris_core *core, u32 reg,
+ u32 mask, u32 exp_val, u32 sleep_us,
+ u32 timeout_us)
+{
+ void __iomem *base_addr;
+ u32 val = 0;
+ int ret;
+
+ if (!core->power_enabled)
+ return -EINVAL;
+
+ base_addr = core->reg_base;
+
+ ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val),
+ sleep_us, timeout_us);
+ /*
+ * Memory barrier to make sure value is read correctly from the
+ * register.
+ */
+ rmb();
+
+ return ret;
+}
+
+int set_preset_registers(struct iris_core *core)
+{
+ const struct reg_preset_info *reg_prst;
+ unsigned int prst_count;
+ int cnt, ret = 0;
+
+ reg_prst = core->platform_data->reg_prst_tbl;
+ prst_count = core->platform_data->reg_prst_tbl_size;
+
+ if (!reg_prst || !prst_count)
+ return 0;
+
+ for (cnt = 0; cnt < prst_count; cnt++) {
+ ret = write_register_masked(core, reg_prst[cnt].reg,
+ reg_prst[cnt].value, reg_prst[cnt].mask);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
static const struct compat_handle compat_handle[] = {
{
.compat = "qcom,sm8550-iris",
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
index 7fba8c2..aef5606 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
@@ -24,6 +24,9 @@ struct vpu_ops {
int (*raise_interrupt)(struct iris_core *core);
int (*clear_interrupt)(struct iris_core *core);
int (*watchdog)(struct iris_core *core, u32 intr_status);
+ int (*power_on)(struct iris_core *core);
+ int (*power_off)(struct iris_core *core);
+ int (*prepare_pc)(struct iris_core *core);
};
#define call_session_op(c, op, ...) \
@@ -32,11 +35,18 @@ struct vpu_ops {
struct vpu_session_ops {
int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type);
+ u64 (*calc_freq)(struct iris_inst *inst, u32 data_size);
+ int (*calc_bw)(struct iris_inst *inst, struct bus_vote_data *data);
};
int init_vpu(struct iris_core *core);
int write_register(struct iris_core *core, u32 reg, u32 value);
+int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask);
int read_register(struct iris_core *core, u32 reg, u32 *value);
+int read_register_with_poll_timeout(struct iris_core *core, u32 reg,
+ u32 mask, u32 exp_val, u32 sleep_us,
+ u32 timeout_us);
+int set_preset_registers(struct iris_core *core);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
index efea5aa..85ed38d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
@@ -5,73 +5,105 @@
#include <linux/delay.h>
+#include "iris_hfi.h"
#include "vpu_iris3.h"
#include "vpu_iris3_buffer.h"
+#include "vpu_iris3_power.h"
#define VIDEO_ARCH_LX 1
-#define CPU_BASE_OFFS_IRIS3 0x000A0000
-
-#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
-#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
-
-#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
-#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
-
-#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
+#define VCODEC_BASE_OFFS_IRIS3 0x00000000
+#define AON_MVP_NOC_RESET 0x0001F000
+#define CPU_BASE_OFFS_IRIS3 0x000A0000
+#define AON_BASE_OFFS 0x000E0000
+#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
+#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
+#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
+#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
+#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
/* HFI_CTRL_INIT */
-#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48)
-
+#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48)
/* HFI_CTRL_STATUS */
-#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C)
-#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000
-
-#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148)
-
-#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168)
-
-/* UC_REGION_ADDR */
-#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64)
-
-/* UC_REGION_ADDR */
-#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68)
-
+#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C)
/* HFI_QTBL_INFO */
-#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50)
-
+#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50)
/* HFI_QTBL_ADDR */
-#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54)
-
+#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54)
/* SFR_ADDR */
-#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C)
-
-#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3
-#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3
-
-#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3
-#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3
-
-#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3
+#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C)
+#define CPU_CS_SCIBCMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x60)
+/* UC_REGION_ADDR */
+#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64)
+/* UC_REGION_ADDR */
+#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68)
+#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148)
+#define CPU_CS_AHB_BRIDGE_SYNC_RESET (CPU_CS_BASE_OFFS_IRIS3 + 0x160)
+#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168)
-#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3
+#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150)
+#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0
-#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3
-#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe
-#define CTRL_ERROR_STATUS__M_IRIS3 \
- CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3
-#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 \
- CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3 0x100
+
+#define AON_WRAPPER_MVP_NOC_RESET_REQ (AON_MVP_NOC_RESET + 0x000)
+#define AON_WRAPPER_MVP_NOC_RESET_ACK (AON_MVP_NOC_RESET + 0x004)
+
+#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000
+#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C)
+#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4
+
+#define WRAPPER_INTR_MASK_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x10)
+#define WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 0x8
+#define WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3 0x4
+
+#define WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x54)
+#define WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x58)
+#define WRAPPER_IRIS_CPU_NOC_LPI_CONTROL (WRAPPER_BASE_OFFS_IRIS3 + 0x5C)
+#define WRAPPER_IRIS_CPU_NOC_LPI_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x60)
+#define WRAPPER_CORE_POWER_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x80)
+#define WRAPPER_CORE_CLOCK_CONFIG_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x88)
+
+#define WRAPPER_TZ_BASE_OFFS 0x000C0000
+#define WRAPPER_TZ_CPU_STATUS (WRAPPER_TZ_BASE_OFFS + 0x10)
+#define WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG (WRAPPER_TZ_BASE_OFFS + 0x14)
+#define WRAPPER_TZ_QNS4PDXFIFO_RESET (WRAPPER_TZ_BASE_OFFS + 0x18)
+
+#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3
+#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3
+#define CTRL_ERROR_STATUS__M_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3
+#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3
+#define CTRL_STATUS_PC_READY_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3
+
+#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3
+#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3
+#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3
+#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3
+#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3
+
+#define AON_WRAPPER_MVP_NOC_LPI_CONTROL (AON_BASE_OFFS)
+#define AON_WRAPPER_MVP_NOC_LPI_STATUS (AON_BASE_OFFS + 0x4)
+
+#define VCODEC_SS_IDLE_STATUSN (VCODEC_BASE_OFFS_IRIS3 + 0x70)
+
+static int interrupt_init_iris3(struct iris_core *core)
+{
+ u32 mask_val;
+ int ret;
-#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000
-#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C)
-#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
-#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4
+ ret = read_register(core, WRAPPER_INTR_MASK_IRIS3, &mask_val);
+ if (ret)
+ return ret;
-#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150)
-#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0
+ mask_val &= ~(WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 |
+ WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3);
+ ret = write_register(core, WRAPPER_INTR_MASK_IRIS3, mask_val);
-#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+ return ret;
+}
static int setup_ucregion_memory_map_iris3(struct iris_core *core)
{
@@ -196,15 +228,335 @@ static int watchdog_iris3(struct iris_core *core, u32 intr_status)
return 0;
}
+static bool is_iris3_hw_power_collapsed(struct iris_core *core)
+{
+ u32 value = 0, pwr_status = 0;
+ int ret;
+
+ ret = read_register(core, WRAPPER_CORE_POWER_STATUS, &value);
+ if (ret)
+ return false;
+
+ pwr_status = value & BIT(1);
+
+ return pwr_status ? false : true;
+}
+
+static int power_off_iris3_hardware(struct iris_core *core)
+{
+ u32 value = 0;
+ int ret, i;
+
+ if (is_iris3_hw_power_collapsed(core))
+ goto disable_power;
+
+ dev_err(core->dev, "Video hw is power ON\n");
+
+ ret = read_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, &value);
+ if (ret)
+ goto disable_power;
+
+ if (value) {
+ ret = write_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, 0);
+ if (ret)
+ goto disable_power;
+ }
+
+ for (i = 0; i < core->cap[NUM_VPP_PIPE].value; i++) {
+ ret = read_register_with_poll_timeout(core, VCODEC_SS_IDLE_STATUSN + 4 * i,
+ 0x400000, 0x400000, 2000, 20000);
+ }
+
+ ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x3);
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
+ 0x3, 0x3, 200, 2000);
+ ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x0);
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
+ 0x3, 0x0, 200, 2000);
+
+ ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x3);
+ if (ret)
+ goto disable_power;
+ ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x2);
+ if (ret)
+ goto disable_power;
+ ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x0);
+ if (ret)
+ goto disable_power;
+
+disable_power:
+ ret = disable_power_domains(core, "vcodec");
+ if (ret) {
+ dev_err(core->dev, "disable power domain vcodec failed\n");
+ ret = 0;
+ }
+
+ disable_unprepare_clock(core, "vcodec_core");
+ if (ret) {
+ dev_err(core->dev, "disable unprepare vcodec_core failed\n");
+ ret = 0;
+ }
+
+ return ret;
+}
+
+static int power_off_iris3_controller(struct iris_core *core)
+{
+ int ret;
+
+ ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x3);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register_masked(core, AON_WRAPPER_MVP_NOC_LPI_CONTROL,
+ 0x1, BIT(0));
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_LPI_STATUS,
+ 0x1, 0x1, 200, 2000);
+
+ ret = write_register_masked(core, WRAPPER_IRIS_CPU_NOC_LPI_CONTROL,
+ 0x1, BIT(0));
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, WRAPPER_IRIS_CPU_NOC_LPI_STATUS,
+ 0x1, 0x1, 200, 2000);
+
+ ret = write_register(core, WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3, 0x0);
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3,
+ 0xffffffff, 0x0, 200, 2000);
+
+ ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x3);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x1);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x0);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x0);
+ if (ret)
+ goto disable_power;
+
+disable_power:
+ disable_unprepare_clock(core, "core_clk");
+ if (ret) {
+ dev_err(core->dev, "disable unprepare core_clk failed\n");
+ ret = 0;
+ }
+
+ /* power down process */
+ ret = disable_power_domains(core, "iris-ctl");
+ if (ret) {
+ dev_err(core->dev, "disable power domain iris-ctl failed\n");
+ ret = 0;
+ }
+
+ return ret;
+}
+
+static int power_off_iris3(struct iris_core *core)
+{
+ if (!core->power_enabled)
+ return 0;
+
+ opp_set_rate(core, 0);
+ power_off_iris3_hardware(core);
+ power_off_iris3_controller(core);
+ unvote_buses(core);
+
+ if (!call_vpu_op(core, watchdog, core, core->intr_status))
+ disable_irq_nosync(core->irq);
+
+ core->power_enabled = false;
+
+ return 0;
+}
+
+static int power_on_iris3_controller(struct iris_core *core)
+{
+ int ret;
+
+ ret = enable_power_domains(core, "iris-ctl");
+ if (ret)
+ return ret;
+
+ ret = reset_ahb2axi_bridge(core);
+ if (ret)
+ goto err_disable_power;
+
+ ret = prepare_enable_clock(core, "gcc_video_axi0");
+ if (ret)
+ goto err_disable_power;
+
+ ret = prepare_enable_clock(core, "core_clk");
+ if (ret)
+ goto err_disable_clock;
+
+ return ret;
+
+err_disable_clock:
+ disable_unprepare_clock(core, "gcc_video_axi0");
+err_disable_power:
+ disable_power_domains(core, "iris-ctl");
+
+ return ret;
+}
+
+static int power_on_iris3_hardware(struct iris_core *core)
+{
+ int ret;
+
+ ret = enable_power_domains(core, "vcodec");
+ if (ret)
+ return ret;
+
+ ret = prepare_enable_clock(core, "vcodec_core");
+ if (ret)
+ goto err_disable_power;
+
+ return ret;
+
+err_disable_power:
+ disable_power_domains(core, "vcodec");
+
+ return ret;
+}
+
+static int power_on_iris3(struct iris_core *core)
+{
+ u32 freq = 0;
+ int ret;
+
+ if (core->power_enabled)
+ return 0;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ ret = vote_buses(core, INT_MAX);
+ if (ret)
+ goto err;
+
+ ret = power_on_iris3_controller(core);
+ if (ret)
+ goto err_unvote_bus;
+
+ ret = power_on_iris3_hardware(core);
+ if (ret)
+ goto err_power_off_ctrl;
+
+ core->power_enabled = true;
+
+ freq = core->power.clk_freq ? core->power.clk_freq :
+ (u32)ULONG_MAX;
+
+ opp_set_rate(core, freq);
+
+ set_preset_registers(core);
+
+ interrupt_init_iris3(core);
+ core->intr_status = 0;
+ enable_irq(core->irq);
+
+ return ret;
+
+err_power_off_ctrl:
+ power_off_iris3_controller(core);
+err_unvote_bus:
+ unvote_buses(core);
+err:
+ core->power_enabled = false;
+
+ return ret;
+}
+
+static int prepare_pc_iris3(struct iris_core *core)
+{
+ u32 wfi_status = 0, idle_status = 0, pc_ready = 0;
+ u32 ctrl_status = 0;
+ int ret;
+
+ ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status);
+ if (ret)
+ return ret;
+
+ pc_ready = ctrl_status & CTRL_STATUS_PC_READY_IRIS3;
+ idle_status = ctrl_status & BIT(30);
+
+ if (pc_ready)
+ return 0;
+
+ ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status);
+ if (ret)
+ return ret;
+
+ wfi_status &= BIT(0);
+ if (!wfi_status || !idle_status)
+ goto skip_power_off;
+
+ ret = prepare_pc(core);
+ if (ret)
+ goto skip_power_off;
+
+ ret = read_register_with_poll_timeout(core, CTRL_STATUS_IRIS3,
+ CTRL_STATUS_PC_READY_IRIS3,
+ CTRL_STATUS_PC_READY_IRIS3, 250, 2500);
+ if (ret)
+ goto skip_power_off;
+
+ ret = read_register_with_poll_timeout(core, WRAPPER_TZ_CPU_STATUS,
+ BIT(0), 0x1, 250, 2500);
+ if (ret)
+ goto skip_power_off;
+
+ return ret;
+
+skip_power_off:
+ ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status);
+ if (ret)
+ return ret;
+
+ ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status);
+ if (ret)
+ return ret;
+
+ wfi_status &= BIT(0);
+ dev_err(core->dev, "Skip PC, wfi=%#x, idle=%#x, pcr=%#x, ctrl=%#x)\n",
+ wfi_status, idle_status, pc_ready, ctrl_status);
+
+ return -EAGAIN;
+}
+
static const struct vpu_ops iris3_ops = {
.boot_firmware = boot_firmware_iris3,
.raise_interrupt = raise_interrupt_iris3,
.clear_interrupt = clear_interrupt_iris3,
.watchdog = watchdog_iris3,
+ .power_on = power_on_iris3,
+ .power_off = power_off_iris3,
+ .prepare_pc = prepare_pc_iris3,
};
static const struct vpu_session_ops iris3_session_ops = {
.int_buf_size = iris_int_buf_size_iris3,
+ .calc_freq = iris_calc_freq_iris3,
+ .calc_bw = iris_calc_bw_iris3,
};
int init_iris3(struct iris_core *core)
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
new file mode 100644
index 0000000..5a02f24
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_instance.h"
+#include "iris_helpers.h"
+#include "platform_common.h"
+#include "vpu_iris3_power.h"
+
+u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
+{
+ u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0;
+ u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0;
+ u32 base_cycles = 0, fps, mbpf;
+ u32 height = 0, width = 0;
+ struct v4l2_format *inp_f;
+ u32 mbs_per_second;
+
+ inp_f = inst->fmt_src;
+ width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
+ height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+
+ mbpf = NUM_MBS_PER_FRAME(height, width);
+ fps = inst->max_rate;
+ mbs_per_second = mbpf * fps;
+
+ fw_cycles = fps * inst->cap[MB_CYCLES_FW].value;
+ fw_vpp_cycles = fps * inst->cap[MB_CYCLES_FW_VPP].value;
+
+ vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value /
+ inst->cap[PIPE].value;
+ vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles);
+
+ if (inst->cap[PIPE].value > 1)
+ vpp_cycles += div_u64(vpp_cycles * 59, 1000);
+
+ base_cycles = inst->cap[MB_CYCLES_VSP].value;
+ bitrate = fps * data_size * 8;
+ vsp_cycles = bitrate;
+
+ if (inst->codec == VP9) {
+ vsp_cycles = div_u64(vsp_cycles * 170, 100);
+ } else {
+ base_cycles = 0;
+ vsp_cycles = div_u64(vsp_cycles, 2);
+ }
+ vsp_cycles = div_u64(vsp_cycles * 21, 20);
+
+ if (inst->cap[STAGE].value == STAGE_1)
+ vsp_cycles = vsp_cycles * 3;
+
+ vsp_cycles += mbs_per_second * base_cycles;
+
+ freq = max3(vpp_cycles, vsp_cycles, fw_cycles);
+
+ return freq;
+}
+
+int iris_calc_bw_iris3(struct iris_inst *inst, struct bus_vote_data *data)
+{
+ const struct bw_info *bw_tbl = NULL;
+ unsigned int num_rows = 0;
+ unsigned int i, mbs, mbps;
+ struct iris_core *core;
+
+ if (!data)
+ return 0;
+
+ core = inst->core;
+
+ mbs = (ALIGN(data->height, 16) / 16) * (ALIGN(data->width, 16) / 16);
+ mbps = mbs * data->fps;
+ if (mbps == 0)
+ return 0;
+
+ bw_tbl = core->platform_data->bw_tbl_dec;
+ num_rows = core->platform_data->bw_tbl_dec_size;
+
+ if (!bw_tbl || num_rows == 0)
+ return 0;
+
+ for (i = 0; i < num_rows; i++) {
+ if (i != 0 && mbps > bw_tbl[i].mbs_per_sec)
+ break;
+
+ if (is_10bit_colorformat(data->color_formats[0]))
+ data->bus_bw = bw_tbl[i].bw_ddr_10bit;
+ else
+ data->bus_bw = bw_tbl[i].bw_ddr;
+ }
+
+ dev_info(core->dev, "bus_bw %llu\n", data->bus_bw);
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
new file mode 100644
index 0000000..2a37a9e
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef __H_VPU_IRIS3_POWER_H__
+#define __H_VPU_IRIS3_POWER_H__
+
+u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size);
+int iris_calc_bw_iris3(struct iris_inst *inst,
+ struct bus_vote_data *vote_data);
+
+#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 29/34] media: iris: implement power management
2023-12-18 11:32 ` [PATCH v2 29/34] media: iris: implement power management Dikshita Agarwal
@ 2023-12-19 12:24 ` Konrad Dybcio
2023-12-20 8:23 ` Dikshita Agarwal
0 siblings, 1 reply; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-19 12:24 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> From: Vikash Garodia <quic_vgarodia@quicinc.com>
>
> Hardware specific power sequence include programming specific
> sequence of registers. Implements this sequence for iris3.
>
> Introduce video device suspend and resume using runtime PM.
>
> Also, implement iris3 specific calculation for clock and
> bus bandwidth which depends on hardware configuration,
> codec format, resolution and frame rate.
>
> Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> @@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev)
return dev_err_probe, drop __func__
[...]
> + ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val),
> + sleep_us, timeout_us);
> + /*
> + * Memory barrier to make sure value is read correctly from the
> + * register.
> + */
> + rmb();
just drop _relaxed and return directly
> +
> + return ret;
> +}
[...]
> + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
> + 0x3, 0x3, 200, 2000);
that looks like a lot of bits/bitfields that deserve #defining in this
and some other functions
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 29/34] media: iris: implement power management
2023-12-19 12:24 ` Konrad Dybcio
@ 2023-12-20 8:23 ` Dikshita Agarwal
0 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-20 8:23 UTC (permalink / raw)
To: Konrad Dybcio, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/19/2023 5:54 PM, Konrad Dybcio wrote:
> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>> From: Vikash Garodia <quic_vgarodia@quicinc.com>
>>
>> Hardware specific power sequence include programming specific
>> sequence of registers. Implements this sequence for iris3.
>>
>> Introduce video device suspend and resume using runtime PM.
>>
>> Also, implement iris3 specific calculation for clock and
>> bus bandwidth which depends on hardware configuration,
>> codec format, resolution and frame rate.
>>
>> Signed-off-by: Vikash Garodia <quic_vgarodia@quicinc.com>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
> [...]
>
>> @@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev)
> return dev_err_probe, drop __func__
Sure
>
> [...]
>
>> + ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val),
>> + sleep_us, timeout_us);
>> + /*
>> + * Memory barrier to make sure value is read correctly from the
>> + * register.
>> + */
>> + rmb();
> just drop _relaxed and return directly
>
Will check and do the needful.
>> +
>> + return ret;
>> +}
> [...]
>
>> + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
>> + 0x3, 0x3, 200, 2000);
> that looks like a lot of bits/bitfields that deserve #defining in this
> and some other functions
>
Will check this.
> Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH v2 30/34] media: iris: register video encoder device to platform driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (28 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 29/34] media: iris: implement power management Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 31/34] media: iris: add platform specific instance capabilities for encoder Dikshita Agarwal
` (10 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Register/unregister video Encoder device.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/iris_common.h | 5 +++
.../media/platform/qcom/vcodec/iris/iris_core.h | 2 ++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 42 ++++++++++++++++------
3 files changed, 38 insertions(+), 11 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 3ab4767..b1273d0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -30,6 +30,11 @@
#define INPUT_TIMER_LIST_SIZE 30
+enum domain_type {
+ ENCODER = BIT(0),
+ DECODER = BIT(1),
+};
+
enum codec_type {
H264 = BIT(0),
HEVC = BIT(1),
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 50553d2..c56eb24 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -23,6 +23,7 @@
* @irq: iris irq
* @v4l2_dev: a holder for v4l2 device structure
* @vdev_dec: iris video device structure for decoder
+ * @vdev_enc: iris video device structure for encoder
* @v4l2_file_ops: iris v4l2 file ops
* @v4l2_ioctl_ops: iris v4l2 ioctl ops
* @bus_tbl: table of iris buses
@@ -66,6 +67,7 @@ struct iris_core {
int irq;
struct v4l2_device v4l2_dev;
struct video_device *vdev_dec;
+ struct video_device *vdev_enc;
const struct v4l2_file_operations *v4l2_file_ops;
const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
struct bus_info *bus_tbl;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 2571e27..b487e83 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -31,16 +31,15 @@ static int init_iris_isr(struct iris_core *core)
return ret;
}
-static int iris_register_video_device(struct iris_core *core)
+static int iris_register_video_device(struct iris_core *core, enum domain_type type)
{
struct video_device *vdev;
- int ret;
+ int ret = 0;
vdev = video_device_alloc();
if (!vdev)
return -ENOMEM;
- strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
vdev->release = video_device_release;
vdev->fops = core->v4l2_file_ops;
vdev->ioctl_ops = core->v4l2_ioctl_ops;
@@ -48,11 +47,24 @@ static int iris_register_video_device(struct iris_core *core)
vdev->v4l2_dev = &core->v4l2_dev;
vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
- ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
- if (ret)
- goto err_vdev_release;
+ if (type == DECODER) {
+ strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto err_vdev_release;
+
+ core->vdev_dec = vdev;
+ } else if (type == ENCODER) {
+ strscpy(vdev->name, "qcom-iris-encoder", sizeof(vdev->name));
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto err_vdev_release;
+
+ core->vdev_enc = vdev;
+ }
- core->vdev_dec = vdev;
video_set_drvdata(vdev, core);
return ret;
@@ -80,6 +92,8 @@ static void iris_remove(struct platform_device *pdev)
video_unregister_device(core->vdev_dec);
+ video_unregister_device(core->vdev_enc);
+
v4l2_device_unregister(&core->v4l2_dev);
iris_pm_put(core, false);
@@ -185,17 +199,21 @@ static int iris_probe(struct platform_device *pdev)
if (ret)
goto err_runtime_disable;
- ret = iris_register_video_device(core);
+ ret = iris_register_video_device(core, DECODER);
if (ret)
goto err_v4l2_unreg;
+ ret = iris_register_video_device(core, ENCODER);
+ if (ret)
+ goto err_vdev_unreg_dec;
+
platform_set_drvdata(pdev, core);
dma_mask = core->cap[DMA_MASK].value;
ret = dma_set_mask_and_coherent(dev, dma_mask);
if (ret)
- goto err_vdev_unreg;
+ goto err_vdev_unreg_enc;
dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));
@@ -206,7 +224,7 @@ static int iris_probe(struct platform_device *pdev)
if (ret) {
dev_err_probe(core->dev, ret,
"%s: interface queues init failed\n", __func__);
- goto err_vdev_unreg;
+ goto err_vdev_unreg_enc;
}
ret = iris_pm_get(core);
@@ -236,7 +254,9 @@ static int iris_probe(struct platform_device *pdev)
hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
&core->command_queue, &core->message_queue,
&core->debug_queue);
-err_vdev_unreg:
+err_vdev_unreg_enc:
+ video_unregister_device(core->vdev_enc);
+err_vdev_unreg_dec:
video_unregister_device(core->vdev_dec);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 31/34] media: iris: add platform specific instance capabilities for encoder
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (29 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 30/34] media: iris: register video encoder device to platform driver Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 32/34] media: iris: implement iris v4l2 ioctl ops supported by encoder Dikshita Agarwal
` (9 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Capabilities are set of video specifications and features supported
by a specific platform(SOC). Each capability is defined with
min, max, range, default value and corresponding HFI.
Also, platform data defines different resource details for
a specific platform(SOC). This change defines resource tables
for sm8550 platform data and use for initializing these resources.
Add Children, Set, Adjust functions for to each capability for
Encoder.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 128 ++-
.../media/platform/qcom/vcodec/iris/iris_common.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 981 ++++++++++++++++++++-
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 35 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 16 +-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 53 ++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 7 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 10 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 5 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 6 +
.../platform/qcom/vcodec/iris/iris_instance.h | 6 +
.../platform/qcom/vcodec/iris/platform_common.h | 74 +-
.../platform/qcom/vcodec/iris/platform_sm8550.c | 768 +++++++++++++++-
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 3 +
14 files changed, 2024 insertions(+), 72 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 872674e..bc32c99 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -45,6 +45,8 @@ enum hfi_property_mode_type {
#define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000
#define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff
+#define HFI_LEVEL_NONE 0xFFFFFFFF
+
#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001
#define HFI_TRUE 0x00000001
@@ -68,14 +70,14 @@ enum hfi_codec_type {
#define HFI_PROP_CODEC 0x03000100
enum hfi_color_format {
- HFI_COLOR_FMT_OPAQUE = 0,
- HFI_COLOR_FMT_NV12 = 1,
- HFI_COLOR_FMT_NV12_UBWC = 2,
- HFI_COLOR_FMT_P010 = 3,
- HFI_COLOR_FMT_TP10_UBWC = 4,
- HFI_COLOR_FMT_RGBA8888 = 5,
- HFI_COLOR_FMT_RGBA8888_UBWC = 6,
- HFI_COLOR_FMT_NV21 = 7,
+ HFI_COLOR_FMT_OPAQUE = 0,
+ HFI_COLOR_FMT_NV12 = 1,
+ HFI_COLOR_FMT_NV12_UBWC = 2,
+ HFI_COLOR_FMT_P010 = 3,
+ HFI_COLOR_FMT_TP10_UBWC = 4,
+ HFI_COLOR_FMT_RGBA8888 = 5,
+ HFI_COLOR_FMT_RGBA8888_UBWC = 6,
+ HFI_COLOR_FMT_NV21 = 7,
};
#define HFI_PROP_COLOR_FORMAT 0x03000101
@@ -104,14 +106,108 @@ enum hfi_color_format {
#define HFI_PROP_CABAC_SESSION 0x03000121
+#define HFI_PROP_8X8_TRANSFORM 0x03000122
+
#define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123
#define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124
#define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128
+enum hfi_syncframe_request_mode {
+ HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR = 0x00000001,
+ HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR = 0x00000002,
+};
+
+enum hfi_rate_control {
+ HFI_RC_VBR_CFR = 0x00000000,
+ HFI_RC_CBR_CFR = 0x00000001,
+ HFI_RC_CQ = 0x00000002,
+ HFI_RC_OFF = 0x00000003,
+ HFI_RC_CBR_VFR = 0x00000004,
+ HFI_RC_LOSSLESS = 0x00000005,
+};
+
+#define HFI_PROP_RATE_CONTROL 0x0300012a
+
+#define HFI_PROP_QP_PACKED 0x0300012e
+
+#define HFI_PROP_MIN_QP_PACKED 0x0300012f
+
+#define HFI_PROP_MAX_QP_PACKED 0x03000130
+
+enum hfi_layer_enc_type {
+ HFI_HIER_P_SLIDING_WINDOW = 0x1,
+ HFI_HIER_P_HYBRID_LTR = 0x2,
+ HFI_HIER_B = 0x3,
+};
+
+#define HFI_PROP_IR_RANDOM_PERIOD 0x03000131
+
+#define HFI_PROP_MULTI_SLICE_MB_COUNT 0x03000132
+
+#define HFI_PROP_MULTI_SLICE_BYTES_COUNT 0x03000133
+
+#define HFI_PROP_LTR_COUNT 0x03000134
+
+#define HFI_PROP_LTR_MARK 0x03000135
+
+#define HFI_PROP_LTR_USE 0x03000136
+
+#define HFI_PROP_LAYER_ENCODING_TYPE 0x03000138
+
+#define HFI_PROP_LAYER_COUNT 0x03000139
+
+#define HFI_PROP_TOTAL_BITRATE 0x0300013b
+
+#define HFI_PROP_BITRATE_LAYER1 0x0300013c
+
+#define HFI_PROP_BITRATE_LAYER2 0x0300013d
+
+#define HFI_PROP_BITRATE_LAYER3 0x0300013e
+
+#define HFI_PROP_BITRATE_LAYER4 0x0300013f
+
+#define HFI_PROP_BITRATE_LAYER5 0x03000140
+
+#define HFI_PROP_BITRATE_LAYER6 0x03000141
+
+#define HFI_PROP_BASELAYER_PRIORITYID 0x03000142
+
+#define HFI_PROP_REQUEST_SYNC_FRAME 0x03000145
+
+#define HFI_PROP_MAX_GOP_FRAMES 0x03000146
+
+#define HFI_PROP_MAX_B_FRAMES 0x03000147
+
#define HFI_PROP_QUALITY_MODE 0x03000148
+enum hfi_seq_header_mode {
+ HFI_SEQ_HEADER_SEPERATE_FRAME = 0x00000001,
+ HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME = 0x00000002,
+ HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME = 0x00000004,
+ HFI_SEQ_HEADER_METADATA = 0x00000008,
+};
+
+#define HFI_PROP_SEQ_HEADER_MODE 0x03000149
+
+enum hfi_rotation {
+ HFI_ROTATION_NONE = 0x00000000,
+ HFI_ROTATION_90 = 0x00000001,
+ HFI_ROTATION_180 = 0x00000002,
+ HFI_ROTATION_270 = 0x00000003,
+};
+
+#define HFI_PROP_ROTATION 0x0300014b
+
+enum hfi_flip {
+ HFI_DISABLE_FLIP = 0x00000000,
+ HFI_HORIZONTAL_FLIP = 0x00000001,
+ HFI_VERTICAL_FLIP = 0x00000002,
+};
+
+#define HFI_PROP_FLIP 0x0300014c
+
enum hfi_color_primaries {
HFI_PRIMARIES_RESERVED = 0,
HFI_PRIMARIES_BT709 = 1,
@@ -182,7 +278,7 @@ enum hfi_picture_type {
#define HFI_PROP_PICTURE_TYPE 0x03000162
-#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
+#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
@@ -190,6 +286,12 @@ enum hfi_picture_type {
#define HFI_PROP_DPB_LIST 0x0300017A
+#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
+
+#define HFI_PROP_MAINTAIN_MIN_QUALITY 0x0300017D
+
+#define HFI_PROP_IR_CYCLIC_PERIOD 0x0300017E
+
#define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190
#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
@@ -214,7 +316,7 @@ enum hfi_picture_type {
#define HFI_SYSTEM_ERROR_BEGIN 0x05000000
-#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
#define HFI_SYSTEM_ERROR_END 0x05FFFFFF
@@ -233,9 +335,9 @@ enum hfi_picture_type {
#define HFI_INFORMATION_END 0x06FFFFFF
struct hfi_debug_header {
- u32 size;
- u32 debug_level;
- u32 reserved[2];
+ u32 size;
+ u32 debug_level;
+ u32 reserved[2];
};
enum hfi_buffer_type {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index b1273d0..ca5406a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -30,6 +30,10 @@
#define INPUT_TIMER_LIST_SIZE 30
+#define CABAC_MAX_BITRATE 160000000
+
+#define CAVLC_MAX_BITRATE 220000000
+
enum domain_type {
ENCODER = BIT(0),
DECODER = BIT(1),
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 94fff74..a648cc1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -611,6 +611,45 @@ int prepare_dependency_list(struct iris_inst *inst)
return ret;
}
+static inline bool is_layer_bitrate_set(struct iris_inst *inst)
+{
+ u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR};
+ u32 cap_id = 0, i, enh_layer_count;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ for (i = 0; i <= enh_layer_count; i++) {
+ if (i >= ARRAY_SIZE(layer_br_caps))
+ break;
+
+ cap_id = layer_br_caps[i];
+ if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET))
+ return false;
+ }
+
+ return true;
+}
+
+static inline u32 get_cumulative_bitrate(struct iris_inst *inst)
+{
+ u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR};
+ u32 cumulative_br = 0;
+ s32 enh_layer_count;
+ u32 cap_id = 0;
+ int i;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ for (i = 0; i <= enh_layer_count; i++) {
+ if (i >= ARRAY_SIZE(layer_br_caps))
+ break;
+ cap_id = layer_br_caps[i];
+ cumulative_br += inst->cap[cap_id].value;
+ }
+
+ return cumulative_br;
+}
+
int set_u32_enum(struct iris_inst *inst,
enum plat_inst_cap_type cap_id)
{
@@ -635,6 +674,18 @@ int set_u32(struct iris_inst *inst,
&hfi_value, sizeof(u32));
}
+int set_q16(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_Q16,
+ &hfi_value, sizeof(u32));
+}
+
int set_stage(struct iris_inst *inst,
enum plat_inst_cap_type cap_id)
{
@@ -662,7 +713,7 @@ int set_pipe(struct iris_inst *inst,
{
u32 work_route, hfi_id;
- work_route = inst->cap[PIPE].value;
+ work_route = inst->cap[cap_id].value;
hfi_id = inst->cap[cap_id].hfi_id;
return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
@@ -671,6 +722,419 @@ int set_pipe(struct iris_inst *inst,
&work_route, sizeof(u32));
}
+int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET))
+ hfi_value = HFI_LEVEL_NONE;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_value, sizeof(u32));
+}
+
+int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+ s32 prepend_sps_pps;
+
+ prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ if (prepend_sps_pps)
+ hfi_val = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR;
+ else
+ hfi_val = HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_val, sizeof(u32));
+}
+
+int set_flip(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hflip, vflip, ret = 0;
+
+ u32 hfi_value = HFI_DISABLE_FLIP;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ hflip = inst->cap[HFLIP].value;
+ vflip = inst->cap[VFLIP].value;
+
+ if (hflip)
+ hfi_value |= HFI_HORIZONTAL_FLIP;
+
+ if (vflip)
+ hfi_value |= HFI_VERTICAL_FLIP;
+
+ if (inst->vb2q_dst->streaming) {
+ if (hfi_value != HFI_DISABLE_FLIP) {
+ ret = set_req_sync_frame(inst, REQUEST_I_FRAME);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_value, sizeof(u32));
+}
+
+int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ switch (inst->cap[cap_id].value) {
+ case 0:
+ hfi_val = HFI_ROTATION_NONE;
+ break;
+ case 90:
+ hfi_val = HFI_ROTATION_90;
+ break;
+ case 180:
+ hfi_val = HFI_ROTATION_180;
+ break;
+ case 270:
+ hfi_val = HFI_ROTATION_270;
+ break;
+ default:
+ hfi_val = HFI_ROTATION_NONE;
+ break;
+ }
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 header_mode, hfi_id, hfi_val;
+ s32 prepend_sps_pps;
+
+ prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value;
+ header_mode = inst->cap[cap_id].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ if (prepend_sps_pps)
+ hfi_val = HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME;
+ else if (header_mode == V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME)
+ hfi_val = HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME;
+ else
+ hfi_val = HFI_SEQ_HEADER_SEPERATE_FRAME;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_val, sizeof(u32));
+}
+
+int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value, hfi_id;
+
+ if (inst->vb2q_dst->streaming) {
+ if (inst->hfi_layer_type == HFI_HIER_B)
+ return 0;
+ }
+
+ hfi_value = inst->cap[GOP_SIZE].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
+int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET)
+ goto set_total_bitrate;
+
+ if (inst->vb2q_dst->streaming)
+ return 0;
+
+set_total_bitrate:
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_val = inst->cap[cap_id].value;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = 0;
+ u32 hfi_id;
+
+ if (!inst->vb2q_dst->streaming)
+ return 0;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET)
+ return 0;
+
+ if (!inst->cap[ENH_LAYER_COUNT].max ||
+ !is_layer_bitrate_set(inst))
+ return 0;
+
+ hfi_value = inst->cap[BIT_RATE].value;
+ hfi_id = inst->cap[BIT_RATE].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
+int set_peak_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+ s32 rc_mode;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_val = inst->cap[cap_id].value;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)
+ return 0;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_use_and_mark_ltr(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_val = inst->cap[cap_id].value;
+
+ if (!inst->cap[LTR_COUNT].value ||
+ inst->cap[cap_id].value == INVALID_DEFAULT_MARK_OR_USE_LTR)
+ return 0;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_ir_period(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id = 0, hfi_val;
+
+ hfi_val = inst->cap[cap_id].value;
+
+ if (inst->cap[IR_TYPE].value ==
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) {
+ hfi_id = HFI_PROP_IR_RANDOM_PERIOD;
+ } else if (inst->cap[IR_TYPE].value ==
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC) {
+ hfi_id = HFI_PROP_IR_CYCLIC_PERIOD;
+ }
+
+ return iris_hfi_set_ir_period(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, min_qp_enable = 0;
+ u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0;
+ u32 client_qp_enable = 0, hfi_value = 0;
+ u32 hfi_id;
+
+ if (inst->cap[MIN_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ min_qp_enable = 1;
+
+ if (min_qp_enable ||
+ (inst->cap[I_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET))
+ i_qp_enable = 1;
+ if (min_qp_enable ||
+ (inst->cap[P_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET))
+ p_qp_enable = 1;
+ if (min_qp_enable ||
+ (inst->cap[B_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET))
+ b_qp_enable = 1;
+
+ client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2;
+ if (!client_qp_enable)
+ return 0;
+
+ i_frame_qp = max(inst->cap[I_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value);
+ p_frame_qp = max(inst->cap[P_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value);
+ b_frame_qp = max(inst->cap[B_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value);
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 |
+ client_qp_enable << 24;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_32_PACKED,
+ &hfi_value, sizeof(u32));
+}
+
+int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, max_qp_enable = 0;
+ u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0;
+ u32 client_qp_enable = 0, hfi_value = 0;
+ u32 hfi_id;
+
+ if (inst->cap[MAX_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ max_qp_enable = 1;
+
+ if (max_qp_enable ||
+ (inst->cap[I_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET))
+ i_qp_enable = 1;
+ if (max_qp_enable ||
+ (inst->cap[P_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET))
+ p_qp_enable = 1;
+ if (max_qp_enable ||
+ (inst->cap[B_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET))
+ b_qp_enable = 1;
+
+ client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2;
+ if (!client_qp_enable)
+ return 0;
+
+ i_frame_qp = min(inst->cap[I_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value);
+ p_frame_qp = min(inst->cap[P_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value);
+ b_frame_qp = min(inst->cap[B_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value);
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 |
+ client_qp_enable << 24;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_32_PACKED,
+ &hfi_value, sizeof(u32));
+}
+
+int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0;
+ u32 client_qp_enable = 0, hfi_value = 0;
+ s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0;
+ s32 rc_type = -1;
+ u32 hfi_id;
+
+ rc_type = inst->hfi_rc_type;
+ if (inst->vb2q_dst->streaming) {
+ if (rc_type != HFI_RC_OFF)
+ return 0;
+ }
+
+ if (rc_type == HFI_RC_OFF) {
+ i_qp_enable = 1;
+ p_qp_enable = 1;
+ b_qp_enable = 1;
+ } else {
+ if (inst->cap[I_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ i_qp_enable = 1;
+ if (inst->cap[P_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ p_qp_enable = 1;
+ if (inst->cap[B_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ b_qp_enable = 1;
+ }
+
+ client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2;
+ if (!client_qp_enable)
+ return 0;
+
+ i_frame_qp = inst->cap[I_FRAME_QP].value;
+ p_frame_qp = inst->cap[P_FRAME_QP].value;
+ b_frame_qp = inst->cap[B_FRAME_QP].value;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 |
+ client_qp_enable << 24;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_32_PACKED,
+ &hfi_value, sizeof(u32));
+}
+
+int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_layer_count, hfi_layer_type = 0;
+ int ret, hfi_id;
+
+ if (!inst->vb2q_dst->streaming) {
+ hfi_layer_type = inst->hfi_layer_type;
+ hfi_id = inst->cap[LAYER_TYPE].hfi_id;
+
+ ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, LAYER_TYPE),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_layer_type, sizeof(u32));
+ if (ret)
+ return ret;
+ } else {
+ if (inst->hfi_layer_type == HFI_HIER_B)
+ return 0;
+ }
+
+ hfi_id = inst->cap[ENH_LAYER_COUNT].hfi_id;
+ hfi_layer_count = inst->cap[ENH_LAYER_COUNT].value + 1;
+
+ ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, ENH_LAYER_COUNT),
+ HFI_PAYLOAD_U32,
+ &hfi_layer_count, sizeof(u32));
+
+ return ret;
+}
+
+int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = 0, set_cap_id = 0, hfi_id;
+ s32 slice_mode = -1;
+
+ slice_mode = inst->cap[SLICE_MODE].value;
+
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE)
+ return 0;
+
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) {
+ hfi_value = (inst->codec == HEVC) ?
+ (inst->cap[SLICE_MAX_MB].value + 3) / 4 :
+ inst->cap[SLICE_MAX_MB].value;
+ set_cap_id = SLICE_MAX_MB;
+ } else if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES) {
+ hfi_value = inst->cap[SLICE_MAX_BYTES].value;
+ set_cap_id = SLICE_MAX_BYTES;
+ }
+
+ hfi_id = inst->cap[set_cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, set_cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
int set_v4l2_properties(struct iris_inst *inst)
{
struct cap_entry *entry = NULL, *temp = NULL;
@@ -723,8 +1187,7 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
u32 adjusted_value;
s32 pix_fmt = -1;
- adjusted_value = ctrl ? ctrl->val :
- inst->cap[PROFILE].value;
+ adjusted_value = ctrl ? ctrl->val : inst->cap[PROFILE].value;
pix_fmt = inst->cap[PIX_FMTS].value;
@@ -737,3 +1200,515 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
return 0;
}
+
+int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR};
+ u32 adjusted_value, cumulative_bitrate, cap_id, cap_val, i;
+ s32 layer_count, max_bitrate = 0, entropy_mode;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[BIT_RATE].value;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) {
+ inst->cap[BIT_RATE].value = adjusted_value;
+ return 0;
+ }
+
+ entropy_mode = inst->cap[ENTROPY_MODE].value;
+
+ if (inst->codec == HEVC)
+ max_bitrate = CABAC_MAX_BITRATE;
+
+ if (inst->codec == H264) {
+ if (entropy_mode == V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC)
+ max_bitrate = CABAC_MAX_BITRATE;
+ else
+ max_bitrate = CAVLC_MAX_BITRATE;
+ }
+
+ if (inst->cap[BIT_RATE].value > max_bitrate)
+ inst->cap[BIT_RATE].value = max_bitrate;
+
+ layer_count = inst->cap[ENH_LAYER_COUNT].value;
+ if (!layer_count)
+ return 0;
+
+ if (!is_layer_bitrate_set(inst))
+ return 0;
+
+ cumulative_bitrate = get_cumulative_bitrate(inst);
+
+ if (cumulative_bitrate > max_bitrate) {
+ u32 decrement_in_value = 0;
+ u32 decrement_in_percent = ((cumulative_bitrate - max_bitrate) * 100) /
+ max_bitrate;
+
+ cumulative_bitrate = 0;
+ for (i = 0; i <= layer_count; i++) {
+ if (i >= ARRAY_SIZE(layer_br_caps))
+ break;
+ cap_id = layer_br_caps[i];
+ cap_val = inst->cap[cap_id].value;
+ decrement_in_value = (cap_val * decrement_in_percent) / 100;
+ cumulative_bitrate += (cap_val - decrement_in_value);
+ inst->cap[cap_id].value = cap_val - decrement_in_value;
+ }
+ inst->cap[BIT_RATE].value = cumulative_bitrate;
+ }
+
+ return 0;
+}
+
+int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 old_br = 0, new_br = 0, exceeded_br = 0;
+ u32 client_set_cap_id = INST_CAP_NONE;
+ u32 cumulative_bitrate = 0;
+ s32 max_bitrate;
+
+ if (!ctrl)
+ return 0;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET ||
+ !inst->vb2q_dst->streaming)
+ return 0;
+
+ if (!inst->cap[ENH_LAYER_COUNT].max)
+ return -EINVAL;
+
+ if (!is_layer_bitrate_set(inst))
+ return 0;
+
+ client_set_cap_id = get_cap_id(inst, ctrl->id);
+ if (!is_valid_cap_id(client_set_cap_id))
+ return -EINVAL;
+
+ cumulative_bitrate = get_cumulative_bitrate(inst);
+ max_bitrate = inst->cap[BIT_RATE].max;
+ old_br = inst->cap[client_set_cap_id].value;
+ new_br = ctrl->val;
+
+ if ((cumulative_bitrate - old_br + new_br) > max_bitrate) {
+ exceeded_br = (cumulative_bitrate - old_br + new_br) - max_bitrate;
+ new_br = ctrl->val - exceeded_br;
+ }
+ inst->cap[client_set_cap_id].value = new_br;
+
+ inst->cap[BIT_RATE].value = get_cumulative_bitrate(inst);
+
+ return 0;
+}
+
+int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 adjusted_value;
+ s32 rc_mode, bitrate;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[PEAK_BITRATE].value;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)
+ return 0;
+
+ bitrate = inst->cap[BIT_RATE].value;
+
+ if (inst->cap[PEAK_BITRATE].flags & CAP_FLAG_CLIENT_SET) {
+ if (adjusted_value < bitrate)
+ adjusted_value = bitrate;
+ } else {
+ adjusted_value = inst->cap[BIT_RATE].value;
+ }
+
+ inst->cap[PEAK_BITRATE].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 frame_rc, bitrate_mode, frame_skip;
+
+ bitrate_mode = inst->cap[BITRATE_MODE].value;
+ frame_rc = inst->cap[FRAME_RC_ENABLE].value;
+ frame_skip = inst->cap[FRAME_SKIP_MODE].value;
+
+ if (!frame_rc) {
+ inst->hfi_rc_type = HFI_RC_OFF;
+ return 0;
+ }
+
+ if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) {
+ inst->hfi_rc_type = HFI_RC_VBR_CFR;
+ } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) {
+ if (frame_skip)
+ inst->hfi_rc_type = HFI_RC_CBR_VFR;
+ else
+ inst->hfi_rc_type = HFI_RC_CBR_CFR;
+ } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) {
+ inst->hfi_rc_type = HFI_RC_CQ;
+ }
+
+ return 0;
+}
+
+int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, enh_layer_count;
+ u32 min_gop_size, num_subgops;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[GOP_SIZE].value;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ if (!enh_layer_count)
+ goto exit;
+
+ /*
+ * Layer encoding needs GOP size to be multiple of subgop size
+ * And subgop size is 2 ^ number of enhancement layers.
+ */
+ min_gop_size = 1 << enh_layer_count;
+ num_subgops = (adjusted_value + (min_gop_size >> 1)) / min_gop_size;
+ if (num_subgops)
+ adjusted_value = num_subgops * min_gop_size;
+ else
+ adjusted_value = min_gop_size;
+
+exit:
+ inst->cap[GOP_SIZE].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, enh_layer_count = -1;
+ const u32 max_bframe_size = 7;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[B_FRAME].value;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ if (!enh_layer_count || inst->hfi_layer_type != HFI_HIER_B) {
+ adjusted_value = 0;
+ goto exit;
+ }
+
+ adjusted_value = (1 << enh_layer_count) - 1;
+
+ if (adjusted_value > max_bframe_size)
+ adjusted_value = max_bframe_size;
+
+exit:
+ inst->cap[B_FRAME].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, rc_mode, pix_fmt;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[LTR_COUNT].value;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR ||
+ inst->hfi_rc_type != HFI_RC_OFF)
+ adjusted_value = 0;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+ if (is_10bit_colorformat(pix_fmt))
+ adjusted_value = 0;
+
+ inst->cap[LTR_COUNT].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, ltr_count;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[USE_LTR].value;
+
+ ltr_count = inst->cap[LTR_COUNT].value;
+ if (!ltr_count)
+ return 0;
+
+ /*
+ * USE_LTR is bitmask value, hence should be
+ * > 0 and <= (2 ^ LTR_COUNT) - 1
+ */
+ if (adjusted_value <= 0 ||
+ adjusted_value > (1 << ltr_count) - 1)
+ return 0;
+
+ inst->cap[USE_LTR].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, ltr_count;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[MARK_LTR].value;
+
+ ltr_count = inst->cap[LTR_COUNT].value;
+ if (!ltr_count)
+ return 0;
+
+ if (adjusted_value < 0 || adjusted_value > ltr_count - 1)
+ return 0;
+
+ inst->cap[MARK_LTR].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, pix_fmt, rc_mode;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[IR_PERIOD].value;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+ if (is_10bit_colorformat(pix_fmt))
+ adjusted_value = 0;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)
+ adjusted_value = 0;
+
+ inst->cap[IR_PERIOD].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, pix_fmt, rc_mode, layer_count;
+ u32 width, height, frame_rate;
+ struct v4l2_format *f;
+
+ if (inst->vb2q_dst->streaming)
+ return 0;
+
+ adjusted_value = MAX_SUPPORTED_MIN_QUALITY;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_VBR)
+ adjusted_value = 0;
+
+ layer_count = inst->cap[ENH_LAYER_COUNT].value;
+ if (layer_count && inst->hfi_layer_type != HFI_HIER_B)
+ adjusted_value = 0;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+ if (is_10bit_colorformat(pix_fmt))
+ adjusted_value = 0;
+
+ frame_rate = inst->cap[FRAME_RATE].value >> 16;
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (!res_is_less_than(width, height, 1920, 1080))
+ adjusted_value = 0;
+
+ if (frame_rate > 60)
+ adjusted_value = 0;
+
+ inst->cap[MIN_QUALITY].value = adjusted_value;
+
+ return 0;
+}
+
+static int adjust_static_layer_count_and_type(struct iris_inst *inst, s32 layer_count)
+{
+ bool hb_requested = false;
+ s32 max_enh_count = 0;
+
+ if (!layer_count)
+ goto exit;
+
+ if (inst->hfi_rc_type == HFI_RC_CQ) {
+ layer_count = 0;
+ goto exit;
+ }
+
+ if (inst->codec == H264) {
+ if (!inst->cap[LAYER_ENABLE].value) {
+ layer_count = 0;
+ goto exit;
+ }
+ hb_requested = inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B;
+ } else if (inst->codec == HEVC) {
+ hb_requested = inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B;
+ }
+
+ if (hb_requested && inst->hfi_rc_type != HFI_RC_VBR_CFR) {
+ layer_count = 0;
+ goto exit;
+ }
+
+ inst->hfi_layer_type = hb_requested ? HFI_HIER_B :
+ (inst->codec == H264 && inst->hfi_rc_type == HFI_RC_VBR_CFR) ?
+ HFI_HIER_P_HYBRID_LTR : HFI_HIER_P_SLIDING_WINDOW;
+
+ max_enh_count = inst->hfi_layer_type == HFI_HIER_B ? MAX_ENH_LAYER_HB :
+ inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR ? MAX_AVC_ENH_LAYER_HYBRID_HP :
+ inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW ?
+ (inst->codec == H264 ? MAX_AVC_ENH_LAYER_SLIDING_WINDOW :
+ (inst->codec == HEVC && inst->hfi_rc_type == HFI_RC_VBR_CFR) ?
+ MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW :
+ MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW) :
+ layer_count;
+
+ layer_count = min(layer_count, max_enh_count);
+
+exit:
+ inst->cap[ENH_LAYER_COUNT].value = layer_count;
+ inst->cap[ENH_LAYER_COUNT].max = layer_count;
+
+ return 0;
+}
+
+int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 client_layer_count;
+ int ret = 0;
+
+ client_layer_count = ctrl ? ctrl->val : inst->cap[ENH_LAYER_COUNT].value;
+
+ if (!inst->vb2q_dst->streaming) {
+ ret = adjust_static_layer_count_and_type(inst, client_layer_count);
+ if (ret)
+ return ret;
+ } else {
+ if (inst->hfi_rc_type == HFI_RC_CBR_CFR ||
+ inst->hfi_rc_type == HFI_RC_CBR_VFR)
+ return ret;
+
+ if (inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR ||
+ inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW)
+ inst->cap[ENH_LAYER_COUNT].value =
+ min(client_layer_count, inst->cap[ENH_LAYER_COUNT].max);
+ }
+
+ return ret;
+}
+
+int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value;
+ s32 profile = -1;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[ENTROPY_MODE].value;
+
+ profile = inst->cap[PROFILE].value;
+ if (profile == V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE ||
+ profile == V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE)
+ adjusted_value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC;
+
+ inst->cap[ENTROPY_MODE].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, rc_type = -1, slice_mode, enh_layer_count = 0;
+ u32 slice_val, mbpf = 0, mbps = 0, max_mbpf = 0, max_mbps = 0, bitrate = 0;
+ u32 update_cap, max_avg_slicesize, output_width, output_height;
+ u32 min_width, min_height, max_width, max_height, fps;
+
+ slice_mode = ctrl ? ctrl->val : inst->cap[SLICE_MODE].value;
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE)
+ return 0;
+
+ bitrate = inst->cap[BIT_RATE].value;
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+ if (enh_layer_count && is_layer_bitrate_set(inst))
+ bitrate = get_cumulative_bitrate(inst);
+
+ rc_type = inst->hfi_rc_type;
+ fps = inst->cap[FRAME_RATE].value >> 16;
+ if (fps > MAX_SLICES_FRAME_RATE ||
+ (rc_type != HFI_RC_OFF && rc_type != HFI_RC_CBR_CFR &&
+ rc_type != HFI_RC_CBR_VFR && rc_type != HFI_RC_VBR_CFR)) {
+ adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE;
+ update_cap = SLICE_MODE;
+ goto exit;
+ }
+
+ output_width = inst->fmt_dst->fmt.pix_mp.width;
+ output_height = inst->fmt_dst->fmt.pix_mp.height;
+
+ max_width = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ?
+ MAX_MB_SLICE_WIDTH : MAX_BYTES_SLICE_WIDTH;
+ max_height = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ?
+ MAX_MB_SLICE_HEIGHT : MAX_BYTES_SLICE_HEIGHT;
+ min_width = (inst->codec == HEVC) ?
+ MIN_HEVC_SLICE_WIDTH : MIN_AVC_SLICE_WIDTH;
+ min_height = MIN_SLICE_HEIGHT;
+
+ if (output_width < min_width || output_height < min_height ||
+ output_width > max_width || output_height > max_width) {
+ adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE;
+ update_cap = SLICE_MODE;
+ goto exit;
+ }
+
+ mbpf = NUM_MBS_PER_FRAME(output_height, output_width);
+ mbps = mbpf * fps;
+ max_mbpf = NUM_MBS_PER_FRAME(max_height, max_width);
+ max_mbps = max_mbpf * MAX_SLICES_FRAME_RATE;
+
+ if (mbpf > max_mbpf || mbps > max_mbps) {
+ adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE;
+ update_cap = SLICE_MODE;
+ goto exit;
+ }
+
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) {
+ update_cap = SLICE_MAX_MB;
+ slice_val = inst->cap[SLICE_MAX_MB].value;
+ slice_val = max(slice_val, mbpf / MAX_SLICES_PER_FRAME);
+ } else {
+ slice_val = inst->cap[SLICE_MAX_BYTES].value;
+ update_cap = SLICE_MAX_BYTES;
+ if (rc_type != HFI_RC_OFF) {
+ max_avg_slicesize =
+ ((bitrate / fps) / 8) / MAX_SLICES_PER_FRAME;
+ slice_val = max(slice_val, max_avg_slicesize);
+ }
+ }
+ adjusted_value = slice_val;
+
+exit:
+ inst->cap[update_cap].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value;
+ s32 profile = -1;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[TRANSFORM_8X8].value;
+
+ profile = inst->cap[PROFILE].value;
+ if (profile != V4L2_MPEG_VIDEO_H264_PROFILE_HIGH &&
+ profile != V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH)
+ adjusted_value = 0;
+
+ inst->cap[TRANSFORM_8X8].value = adjusted_value;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 28ce767..5421d9f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -31,5 +31,40 @@ int get_inst_capability(struct iris_inst *inst);
int set_v4l2_properties(struct iris_inst *inst);
int adjust_v4l2_properties(struct iris_inst *inst);
int ctrls_init(struct iris_inst *inst, bool init);
+int set_q16(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_flip(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_peak_bitrate(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+int set_use_and_mark_ltr(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+int set_ir_period(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 395a189..d9d22e2 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -40,12 +40,20 @@ bool res_is_less_than(u32 width, u32 height,
u32 get_port_info(struct iris_inst *inst,
enum plat_inst_cap_type cap_id)
{
+ if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT &&
+ inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) {
+ if (inst->vb2q_dst->streaming)
+ return get_hfi_port(INPUT_MPLANE);
+ else
+ return get_hfi_port(OUTPUT_MPLANE);
+ }
+
if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT)
- return HFI_PORT_BITSTREAM;
+ return get_hfi_port(INPUT_MPLANE);
else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT)
- return HFI_PORT_RAW;
-
- return HFI_PORT_NONE;
+ return get_hfi_port(OUTPUT_MPLANE);
+ else
+ return HFI_PORT_NONE;
}
enum iris_buffer_type v4l2_type_to_driver(u32 type)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 2850fd5..00e598d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -668,6 +668,59 @@ int iris_hfi_set_property(struct iris_inst *inst,
return ret;
}
+int iris_hfi_set_ir_period(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ u32 sync_frame_req = 0;
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+
+ ret = hfi_create_header(inst->packet, inst->packet_size,
+ inst->session_id, core->header_id++);
+ if (ret)
+ goto exit;
+
+ if (!inst->ir_enabled) {
+ inst->ir_enabled = ((*(u32 *)payload > 0) ? true : false);
+ if (inst->ir_enabled && inst->vb2q_dst->streaming) {
+ sync_frame_req = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR;
+ ret = hfi_create_packet(inst->packet, inst->packet_size,
+ HFI_PROP_REQUEST_SYNC_FRAME,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32_ENUM,
+ HFI_PORT_BITSTREAM,
+ core->packet_id++,
+ &sync_frame_req,
+ sizeof(u32));
+ if (ret)
+ goto exit;
+ }
+ }
+
+ ret = hfi_create_packet(inst->packet, inst->packet_size,
+ packet_type,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ plane,
+ core->packet_id++,
+ payload,
+ sizeof(u32));
+ if (ret)
+ goto exit;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+exit:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 465bfc5..95e0523 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -40,9 +40,8 @@ int prepare_pc(struct iris_core *core);
irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
-int iris_hfi_queue_buffer(struct iris_inst *inst,
- struct iris_buffer *buffer);
-int iris_hfi_release_buffer(struct iris_inst *inst,
- struct iris_buffer *buffer);
+int iris_hfi_set_ir_period(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index fab206cf..dd27fa4 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -364,8 +364,8 @@ int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
return 0;
}
-static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
- u32 header_id)
+int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
+ u32 header_id)
{
struct hfi_header *hdr = (struct hfi_header *)packet;
@@ -382,9 +382,9 @@ static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
return 0;
}
-static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
- u32 pkt_flags, u32 payload_type, u32 port,
- u32 packet_id, void *payload, u32 payload_size)
+int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
+ u32 pkt_flags, u32 payload_type, u32 port,
+ u32 packet_id, void *payload, u32 payload_size)
{
struct hfi_header *hdr;
struct hfi_packet *pkt;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 849b585..82148b7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -108,5 +108,10 @@ int hfi_packet_sys_interframe_powercollapse(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_sys_pc_prep(struct iris_core *core,
u8 *pkt, u32 pkt_size);
+int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
+ u32 header_id);
+int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
+ u32 pkt_flags, u32 payload_type, u32 port,
+ u32 packet_id, void *payload, u32 payload_size);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index a4d5d9c..1b667a5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -758,6 +758,12 @@ static int handle_session_property(struct iris_inst *inst,
case HFI_PROP_PICTURE_TYPE:
inst->hfi_frame_info.picture_type = payload_ptr[0];
break;
+ case HFI_PROP_CABAC_SESSION:
+ if (payload_ptr[0] == 1)
+ inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC;
+ else
+ inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC;
+ break;
case HFI_PROP_DPB_LIST:
ret = handle_dpb_list_property(inst, pkt);
if (ret)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 3c85e78..70f4c7d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -52,6 +52,9 @@
* @power: structure of power info
* @bus_data: structure of bus data
* @input_timer_list: list head of input timer
+ * @ir_enabled: boolean for intra refresh
+ * @hfi_rc_type: rate control type
+ * @hfi_layer_type: type of HFI layer encoding
*/
struct iris_inst {
@@ -90,6 +93,9 @@ struct iris_inst {
struct iris_inst_power power;
struct bus_vote_data bus_data;
struct list_head input_timer_list;
+ bool ir_enabled;
+ u32 hfi_rc_type;
+ u32 hfi_layer_type;
};
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 81de610..443894c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -20,9 +20,29 @@ struct iris_inst;
#define BIT_DEPTH_8 (8 << 16 | 8)
#define BIT_DEPTH_10 (10 << 16 | 10)
-#define CODED_FRAMES_PROGRESSIVE 0x0
-#define CODED_FRAMES_INTERLACE 0x1
-#define MAX_NUM_CHILD 10
+#define CODED_FRAMES_PROGRESSIVE 0x0
+#define CODED_FRAMES_INTERLACE 0x1
+#define MAX_NUM_CHILD 10
+#define MAX_ENH_LAYER_HB 3
+#define MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW 5
+#define MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW 3
+#define MAX_AVC_ENH_LAYER_SLIDING_WINDOW 3
+#define MAX_AVC_ENH_LAYER_HYBRID_HP 5
+#define MAX_SLICES_FRAME_RATE 60
+#define MAX_MB_SLICE_WIDTH 4096
+#define MAX_MB_SLICE_HEIGHT 2160
+#define MAX_BYTES_SLICE_WIDTH 1920
+#define MAX_BYTES_SLICE_HEIGHT 1088
+#define MIN_HEVC_SLICE_WIDTH 384
+#define MIN_AVC_SLICE_WIDTH 192
+#define MIN_SLICE_HEIGHT 128
+#define MAX_SLICES_PER_FRAME 10
+#define MIN_QP_8BIT 1
+#define MIN_SLICE_BYTE_SIZE 512
+#define MAX_SLICE_BYTE_SIZE ((MAX_BITRATE) >> 3)
+#define MAX_SLICE_MB_SIZE (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
+#define INVALID_DEFAULT_MARK_OR_USE_LTR -1
+#define MAX_SUPPORTED_MIN_QUALITY 70
#define DEFAULT_MAX_HOST_BUF_COUNT 64
#define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256
@@ -128,12 +148,17 @@ enum plat_inst_cap_type {
PIX_FMTS,
MBPF,
QUEUED_RATE,
+ FRAME_RATE,
+ OPERATING_RATE,
MB_CYCLES_VSP,
MB_CYCLES_VPP,
MB_CYCLES_LP,
MB_CYCLES_FW,
MB_CYCLES_FW_VPP,
NUM_COMV,
+ ENTROPY_MODE,
+ BASELAYER_PRIORITY,
+ IR_TYPE,
PROFILE,
LEVEL,
HEVC_TIER,
@@ -148,6 +173,48 @@ enum plat_inst_cap_type {
BIT_DEPTH,
DEFAULT_HEADER,
RAP_FRAME,
+ MIN_FRAME_QP,
+ MAX_FRAME_QP,
+ B_FRAME,
+ I_FRAME_QP,
+ P_FRAME_QP,
+ B_FRAME_QP,
+ BIT_RATE,
+ PEAK_BITRATE,
+ BITRATE_MODE,
+ FRAME_SKIP_MODE,
+ FRAME_RC_ENABLE,
+ GOP_SIZE,
+ MIN_QUALITY,
+ IR_PERIOD,
+ LTR_COUNT,
+ USE_LTR,
+ MARK_LTR,
+ I_FRAME_MIN_QP,
+ P_FRAME_MIN_QP,
+ B_FRAME_MIN_QP,
+ I_FRAME_MAX_QP,
+ P_FRAME_MAX_QP,
+ B_FRAME_MAX_QP,
+ LAYER_TYPE,
+ LAYER_ENABLE,
+ L0_BR,
+ L1_BR,
+ L2_BR,
+ L3_BR,
+ L4_BR,
+ L5_BR,
+ ENH_LAYER_COUNT,
+ TRANSFORM_8X8,
+ SLICE_MODE,
+ SLICE_MAX_MB,
+ SLICE_MAX_BYTES,
+ HFLIP,
+ VFLIP,
+ ROTATION,
+ HEADER_MODE,
+ PREPEND_SPSPPS_TO_IDR,
+ REQUEST_I_FRAME,
INST_CAP_MAX,
};
@@ -164,6 +231,7 @@ enum plat_inst_cap_flags {
struct plat_inst_cap {
enum plat_inst_cap_type cap_id;
+ enum domain_type domain;
enum codec_type codec;
s32 min;
s32 max;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 7ae9715..6d5192a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -13,11 +13,19 @@
#include "platform_common.h"
#include "resources.h"
-#define CODECS_ALL (H264 | HEVC | VP9)
-
-#define DEFAULT_FPS 30
-#define MINIMUM_FPS 1
-#define MAXIMUM_FPS 480
+#define CODECS_ALL (H264 | HEVC | VP9)
+#define ENC ENCODER
+#define DEC DECODER
+
+#define DEFAULT_FPS 30
+#define MINIMUM_FPS 1
+#define MAXIMUM_FPS 480
+#define MAX_BITRATE 245000000
+#define DEFAULT_BITRATE 20000000
+#define MAX_LTR_FRAME_COUNT 2
+#define MAX_BASE_LAYER_PRIORITY_ID 63
+#define MAX_QP 51
+#define DEFAULT_QP 20
static struct codec_info codec_data_sm8550[] = {
{
@@ -68,21 +76,37 @@ static struct plat_core_cap core_data_sm8550[] = {
};
static struct plat_inst_cap instance_cap_data_sm8550[] = {
- {FRAME_WIDTH, CODECS_ALL, 96, 8192, 1, 1920},
+ {FRAME_WIDTH, DEC, CODECS_ALL, 96, 8192, 1, 1920},
+
+ {FRAME_WIDTH, DEC, VP9, 96, 4096, 1, 1920},
+
+ {FRAME_WIDTH, ENC, CODECS_ALL, 128, 8192, 1, 1920},
- {FRAME_WIDTH, VP9, 96, 4096, 1, 1920},
+ {FRAME_WIDTH, ENC, HEVC, 96, 8192, 1, 1920},
- {FRAME_HEIGHT, CODECS_ALL, 96, 8192, 1, 1080},
+ {FRAME_HEIGHT, DEC, CODECS_ALL, 96, 8192, 1, 1080},
- {FRAME_HEIGHT, VP9, 96, 4096, 1, 1080},
+ {FRAME_HEIGHT, DEC, VP9, 96, 4096, 1, 1080},
- {PIX_FMTS, H264,
+ {FRAME_HEIGHT, ENC, CODECS_ALL, 128, 8192, 1, 1080},
+
+ {FRAME_HEIGHT, ENC, HEVC, 96, 8192, 1, 1080},
+
+ {PIX_FMTS, DEC, H264,
FMT_NV12,
FMT_NV12C,
FMT_NV12 | FMT_NV21 | FMT_NV12C,
FMT_NV12C},
- {PIX_FMTS, HEVC,
+ {PIX_FMTS, ENC, H264,
+ FMT_NV12,
+ FMT_NV12C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C,
+ FMT_NV12C,
+ 0, 0,
+ CAP_FLAG_NONE},
+
+ {PIX_FMTS, DEC, HEVC,
FMT_NV12,
FMT_TP10C,
FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
@@ -91,37 +115,73 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
CAP_FLAG_NONE,
{PROFILE}},
- {PIX_FMTS, VP9,
+ {PIX_FMTS, ENC, HEVC,
+ FMT_NV12,
+ FMT_TP10C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
+ FMT_NV12C,
+ 0, 0,
+ CAP_FLAG_NONE,
+ {PROFILE, MIN_QUALITY, IR_PERIOD, LTR_COUNT}},
+
+ {PIX_FMTS, DEC, VP9,
FMT_NV12,
FMT_TP10C,
FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
FMT_NV12C},
- {MBPF, CODECS_ALL, 36, 138240, 1, 138240},
+ {MBPF, DEC, CODECS_ALL, 36, 138240, 1, 138240},
/* (4096 * 2304) / 256 */
- {MBPF, VP9, 36, 36864, 1, 36864},
+ {MBPF, DEC, VP9, 36, 36864, 1, 36864},
+
+ /* (8192 * 4320) / 256 */
+ {MBPF, ENC, CODECS_ALL, 64, 138240, 1, 138240},
+
+ {MBPF, ENC, HEVC, 36, 138240, 1, 138240},
- {QUEUED_RATE, CODECS_ALL,
+ {QUEUED_RATE, DEC | ENC, CODECS_ALL,
(MINIMUM_FPS << 16), INT_MAX,
1, (DEFAULT_FPS << 16)},
- {MB_CYCLES_VSP, CODECS_ALL, 25, 25, 1, 25},
+ {FRAME_RATE, ENC, CODECS_ALL,
+ (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16),
+ 1, (DEFAULT_FPS << 16),
+ 0,
+ HFI_PROP_FRAME_RATE,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_q16},
+
+ {OPERATING_RATE, ENC, CODECS_ALL,
+ (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16),
+ 1, (DEFAULT_FPS << 16)},
+
+ {MB_CYCLES_VSP, DEC, CODECS_ALL, 25, 25, 1, 25},
+
+ {MB_CYCLES_VSP, DEC, VP9, 60, 60, 1, 60},
+
+ {MB_CYCLES_VSP, ENC, CODECS_ALL, 25, 25, 1, 25},
- {MB_CYCLES_VSP, VP9, 60, 60, 1, 60},
+ {MB_CYCLES_VPP, DEC, CODECS_ALL, 200, 200, 1, 200},
- {MB_CYCLES_VPP, CODECS_ALL, 200, 200, 1, 200},
+ {MB_CYCLES_VPP, ENC, CODECS_ALL, 675, 675, 1, 675},
- {MB_CYCLES_LP, CODECS_ALL, 200, 200, 1, 200},
+ {MB_CYCLES_LP, DEC, CODECS_ALL, 200, 200, 1, 200},
- {MB_CYCLES_FW, CODECS_ALL, 489583, 489583, 1, 489583},
+ {MB_CYCLES_LP, ENC, CODECS_ALL, 320, 320, 1, 320},
- {MB_CYCLES_FW_VPP, CODECS_ALL, 66234, 66234, 1, 66234},
+ {MB_CYCLES_FW, DEC | ENC, CODECS_ALL, 489583, 489583, 1, 489583},
- {NUM_COMV, CODECS_ALL,
+ {MB_CYCLES_FW_VPP, DEC, CODECS_ALL, 66234, 66234, 1, 66234},
+
+ {MB_CYCLES_FW_VPP, ENC, CODECS_ALL, 48405, 48405, 1, 48405},
+
+ {NUM_COMV, DEC, CODECS_ALL,
0, INT_MAX, 1, 0},
- {PROFILE, H264,
+ {PROFILE, DEC, H264,
V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH,
BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
@@ -137,7 +197,23 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},
- {PROFILE, HEVC,
+ {PROFILE, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH,
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH),
+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ HFI_PROP_PROFILE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {ENTROPY_MODE, TRANSFORM_8X8},
+ NULL,
+ set_u32_enum},
+
+ {PROFILE, ENC | DEC, HEVC,
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) |
@@ -151,7 +227,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
adjust_profile,
set_u32_enum},
- {PROFILE, VP9,
+ {PROFILE, DEC, VP9,
V4L2_MPEG_VIDEO_VP9_PROFILE_0,
V4L2_MPEG_VIDEO_VP9_PROFILE_2,
BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) |
@@ -164,7 +240,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},
- {LEVEL, H264,
+ {LEVEL, DEC, H264,
V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
V4L2_MPEG_VIDEO_H264_LEVEL_6_2,
BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) |
@@ -195,7 +271,36 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},
- {LEVEL, HEVC,
+ {LEVEL, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+ V4L2_MPEG_VIDEO_H264_LEVEL_6_0,
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0),
+ V4L2_MPEG_VIDEO_H264_LEVEL_5_0,
+ V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ HFI_PROP_LEVEL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_level},
+
+ {LEVEL, DEC, HEVC,
V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2,
BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) |
@@ -219,7 +324,31 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},
- {LEVEL, VP9,
+ {LEVEL, ENC, HEVC,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2,
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2),
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_5,
+ V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+ HFI_PROP_LEVEL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_level},
+
+ {LEVEL, DEC, VP9,
V4L2_MPEG_VIDEO_VP9_LEVEL_1_0,
V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) |
@@ -242,7 +371,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},
- {HEVC_TIER, HEVC,
+ {HEVC_TIER, DEC | ENC, HEVC,
V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) |
@@ -255,7 +384,566 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},
- {DISPLAY_DELAY_ENABLE, CODECS_ALL,
+ {HFLIP, ENC, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_HFLIP,
+ HFI_PROP_FLIP,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_flip},
+
+ {VFLIP, ENC, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_VFLIP,
+ HFI_PROP_FLIP,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_flip},
+
+ {ROTATION, ENC, CODECS_ALL,
+ 0, 270, 90, 0,
+ V4L2_CID_ROTATE,
+ HFI_PROP_ROTATION,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_rotation},
+
+ {HEADER_MODE, ENC, CODECS_ALL,
+ V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+ BIT(V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) |
+ BIT(V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME),
+ V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+ HFI_PROP_SEQ_HEADER_MODE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_header_mode},
+
+ {PREPEND_SPSPPS_TO_IDR, ENC, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR},
+
+ {REQUEST_I_FRAME, ENC, H264 | HEVC,
+ 0, 0, 0, 0,
+ V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
+ HFI_PROP_REQUEST_SYNC_FRAME,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_req_sync_frame},
+
+ {BIT_RATE, ENC, H264 | HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_BITRATE,
+ HFI_PROP_TOTAL_BITRATE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {PEAK_BITRATE, L0_BR},
+ adjust_bitrate,
+ set_bitrate},
+
+ {PEAK_BITRATE, ENC, H264 | HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_BITRATE_PEAK,
+ HFI_PROP_TOTAL_PEAK_BITRATE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_peak_bitrate,
+ set_peak_bitrate},
+
+ {BITRATE_MODE, ENC, H264 | HEVC,
+ V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+ V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+ BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) |
+ BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_CBR),
+ V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+ V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+ HFI_PROP_RATE_CONTROL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {LTR_COUNT, IR_PERIOD, I_FRAME_QP, P_FRAME_QP,
+ B_FRAME_QP, ENH_LAYER_COUNT, BIT_RATE,
+ MIN_QUALITY, PEAK_BITRATE, SLICE_MODE},
+ adjust_bitrate_mode,
+ set_u32_enum},
+
+ {FRAME_SKIP_MODE, ENC, H264 | HEVC,
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT,
+ BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED) |
+ BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_LEVEL_LIMIT) |
+ BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT),
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE,
+ 0,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {FRAME_RC_ENABLE, ENC, H264 | HEVC,
+ 0, 1, 1, 1,
+ V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE},
+
+ {GOP_SIZE, ENC, CODECS_ALL,
+ 0, INT_MAX, 1, 2 * DEFAULT_FPS - 1,
+ V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+ HFI_PROP_MAX_GOP_FRAMES,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_gop_size,
+ set_gop_size},
+
+ {B_FRAME, ENC, H264 | HEVC,
+ 0, 7, 1, 0,
+ V4L2_CID_MPEG_VIDEO_B_FRAMES,
+ HFI_PROP_MAX_B_FRAMES,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_b_frame,
+ set_u32},
+
+ {LTR_COUNT, ENC, H264 | HEVC,
+ 0, MAX_LTR_FRAME_COUNT, 1, 0,
+ V4L2_CID_MPEG_VIDEO_LTR_COUNT,
+ HFI_PROP_LTR_COUNT,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_ltr_count,
+ set_u32},
+
+ {USE_LTR, ENC, H264 | HEVC,
+ 0,
+ ((1 << MAX_LTR_FRAME_COUNT) - 1),
+ 0, 0,
+ V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES,
+ HFI_PROP_LTR_USE,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_use_ltr,
+ set_use_and_mark_ltr},
+
+ {MARK_LTR, ENC, H264 | HEVC,
+ INVALID_DEFAULT_MARK_OR_USE_LTR,
+ (MAX_LTR_FRAME_COUNT - 1),
+ 1, INVALID_DEFAULT_MARK_OR_USE_LTR,
+ V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX,
+ HFI_PROP_LTR_MARK,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_mark_ltr,
+ set_use_and_mark_ltr},
+
+ {BASELAYER_PRIORITY, ENC, H264,
+ 0, MAX_BASE_LAYER_PRIORITY_ID, 1, 0,
+ V4L2_CID_MPEG_VIDEO_BASELAYER_PRIORITY_ID,
+ HFI_PROP_BASELAYER_PRIORITYID,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {IR_TYPE, ENC, H264 | HEVC,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC,
+ BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) |
+ BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC),
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE,
+ 0,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {IR_PERIOD, ENC, H264 | HEVC,
+ 0, INT_MAX, 1, 0,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD,
+ 0,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_OUTPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_ir_period,
+ set_ir_period},
+
+ {MIN_QUALITY, ENC, H264 | HEVC,
+ 0, MAX_SUPPORTED_MIN_QUALITY, 70, MAX_SUPPORTED_MIN_QUALITY,
+ 0,
+ HFI_PROP_MAINTAIN_MIN_QUALITY,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_min_quality,
+ set_u32},
+
+ {MIN_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_MIN_QP,
+ HFI_PROP_MIN_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_min_qp},
+
+ {MIN_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP,
+ HFI_PROP_MIN_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_min_qp},
+
+ {MAX_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+ HFI_PROP_MAX_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_max_qp},
+
+ {MAX_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP,
+ HFI_PROP_MAX_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_max_qp},
+
+ {I_FRAME_MIN_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MIN_QP},
+
+ {I_FRAME_MIN_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MIN_QP},
+
+ {P_FRAME_MIN_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MIN_QP},
+
+ {P_FRAME_MIN_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MIN_QP},
+
+ {B_FRAME_MIN_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MIN_QP},
+
+ {B_FRAME_MIN_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MIN_QP},
+
+ {I_FRAME_MAX_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MAX_QP},
+
+ {I_FRAME_MAX_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MAX_QP},
+
+ {P_FRAME_MAX_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MAX_QP},
+
+ {P_FRAME_MAX_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MAX_QP},
+
+ {B_FRAME_MAX_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MAX_QP},
+
+ {B_FRAME_MAX_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MAX_QP},
+
+ {I_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {I_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {P_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {P_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {B_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {B_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {LAYER_TYPE, ENC, HEVC,
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B,
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P,
+ BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P),
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE,
+ HFI_PROP_LAYER_ENCODING_TYPE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {LTR_COUNT}},
+
+ {LAYER_TYPE, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B,
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P,
+ BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) |
+ BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P),
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P,
+ V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE,
+ HFI_PROP_LAYER_ENCODING_TYPE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {LTR_COUNT}},
+
+ {LAYER_ENABLE, ENC, H264,
+ 0, 1, 1, 0,
+ V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING,
+ HFI_PROP_LAYER_ENCODING_TYPE,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {LAYER_ENABLE, ENC, HEVC,
+ 0, 1, 1, 0,
+ 0,
+ 0,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {ENH_LAYER_COUNT, ENC, HEVC,
+ 0, 5, 1, 0,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER,
+ HFI_PROP_LAYER_COUNT,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE,
+ LTR_COUNT},
+ adjust_layer_count,
+ set_layer_count_and_type},
+
+ {ENH_LAYER_COUNT, ENC, H264,
+ 0, 5, 1, 0,
+ V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER,
+ HFI_PROP_LAYER_COUNT,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE,
+ LTR_COUNT},
+ adjust_layer_count,
+ set_layer_count_and_type},
+
+ {L0_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR,
+ HFI_PROP_BITRATE_LAYER1,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L1_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L0_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR,
+ HFI_PROP_BITRATE_LAYER1,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L1_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L1_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR,
+ HFI_PROP_BITRATE_LAYER2,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L2_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L1_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR,
+ HFI_PROP_BITRATE_LAYER2,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L2_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L2_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR,
+ HFI_PROP_BITRATE_LAYER3,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L3_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L2_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR,
+ HFI_PROP_BITRATE_LAYER3,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L3_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L3_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR,
+ HFI_PROP_BITRATE_LAYER4,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L4_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L3_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR,
+ HFI_PROP_BITRATE_LAYER4,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L4_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L4_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR,
+ HFI_PROP_BITRATE_LAYER5,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L5_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L4_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR,
+ HFI_PROP_BITRATE_LAYER5,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L5_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L5_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR,
+ HFI_PROP_BITRATE_LAYER6,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L5_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR,
+ HFI_PROP_BITRATE_LAYER6,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {ENTROPY_MODE, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC,
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC,
+ BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC) |
+ BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC),
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC,
+ V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE,
+ HFI_PROP_CABAC_SESSION,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {BIT_RATE},
+ adjust_entropy_mode,
+ set_u32},
+
+ {SLICE_MODE, ENC, H264 | HEVC,
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE,
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES,
+ BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) |
+ BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) |
+ BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES),
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE,
+ 0,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {STAGE},
+ adjust_slice_count,
+ set_slice_count},
+
+ {SLICE_MAX_BYTES, ENC, H264 | HEVC,
+ MIN_SLICE_BYTE_SIZE, MAX_SLICE_BYTE_SIZE,
+ 1, MIN_SLICE_BYTE_SIZE,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES,
+ HFI_PROP_MULTI_SLICE_BYTES_COUNT,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {SLICE_MAX_MB, ENC, H264 | HEVC,
+ 1, MAX_SLICE_MB_SIZE, 1, 1,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB,
+ HFI_PROP_MULTI_SLICE_MB_COUNT,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {TRANSFORM_8X8, ENC, H264,
+ 0, 1, 1, 1,
+ V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM,
+ HFI_PROP_8X8_TRANSFORM,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_transform_8x8,
+ set_u32},
+
+ {DISPLAY_DELAY_ENABLE, DEC, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE,
HFI_PROP_DECODE_ORDER_OUTPUT,
@@ -264,7 +952,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
NULL},
- {DISPLAY_DELAY, CODECS_ALL,
+ {DISPLAY_DELAY, DEC, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY,
HFI_PROP_DECODE_ORDER_OUTPUT,
@@ -273,7 +961,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
NULL},
- {OUTPUT_ORDER, CODECS_ALL,
+ {OUTPUT_ORDER, DEC, CODECS_ALL,
0, 1, 1, 0,
0,
HFI_PROP_DECODE_ORDER_OUTPUT,
@@ -282,7 +970,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
adjust_output_order,
set_u32},
- {INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL,
+ {INPUT_BUF_HOST_MAX_COUNT, ENC | DEC, CODECS_ALL,
DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT,
1, DEFAULT_MAX_HOST_BUF_COUNT,
0,
@@ -292,7 +980,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32},
- {STAGE, CODECS_ALL,
+ {STAGE, ENC | DEC, CODECS_ALL,
STAGE_1,
STAGE_2, 1,
STAGE_2,
@@ -303,7 +991,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_stage},
- {PIPE, CODECS_ALL,
+ {PIPE, ENC | DEC, CODECS_ALL,
PIPE_1,
PIPE_4, 1,
PIPE_4,
@@ -314,26 +1002,26 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_pipe},
- {POC, H264, 0, 2, 1, 1,
+ {POC, DEC, H264, 0, 2, 1, 1,
0,
HFI_PROP_PIC_ORDER_CNT_TYPE},
- {CODED_FRAMES, H264 | HEVC,
+ {CODED_FRAMES, DEC, H264 | HEVC,
CODED_FRAMES_PROGRESSIVE, CODED_FRAMES_PROGRESSIVE,
0, CODED_FRAMES_PROGRESSIVE,
0,
HFI_PROP_CODED_FRAMES},
- {BIT_DEPTH, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8,
+ {BIT_DEPTH, DEC, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8,
0,
HFI_PROP_LUMA_CHROMA_BIT_DEPTH},
- {DEFAULT_HEADER, CODECS_ALL,
+ {DEFAULT_HEADER, DEC, CODECS_ALL,
0, 1, 1, 0,
0,
HFI_PROP_DEC_DEFAULT_HEADER},
- {RAP_FRAME, CODECS_ALL,
+ {RAP_FRAME, DEC, CODECS_ALL,
0, 1, 1, 1,
0,
HFI_PROP_DEC_START_FROM_RAP_FRAME,
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
index 5a02f24..58498af 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -42,6 +42,9 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
if (inst->codec == VP9) {
vsp_cycles = div_u64(vsp_cycles * 170, 100);
+ } else if (inst->cap[ENTROPY_MODE].value ==
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
+ vsp_cycles = div_u64(vsp_cycles * 135, 100);
} else {
base_cycles = 0;
vsp_cycles = div_u64(vsp_cycles, 2);
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 32/34] media: iris: implement iris v4l2 ioctl ops supported by encoder
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (30 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 31/34] media: iris: add platform specific instance capabilities for encoder Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 33/34] media: iris: add vb2 streaming and buffer ops for encoder Dikshita Agarwal
` (8 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Implement all iris v4l2 ioctls ops supported by encoder.
Add state checks to ensure ioctl are allowed in valid
instance state only. Codec format can be changed by client
during s_fmt. Update the v4l2 control values according
to the updated codec format.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 8 +-
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 59 ++-
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 89 +++-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 3 +
.../platform/qcom/vcodec/iris/iris_instance.h | 5 +
.../media/platform/qcom/vcodec/iris/iris_probe.c | 3 +-
.../media/platform/qcom/vcodec/iris/iris_state.c | 5 +-
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 80 +---
.../media/platform/qcom/vcodec/iris/iris_venc.c | 525 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_venc.h | 24 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 415 ++++++++++++++--
.../platform/qcom/vcodec/iris/platform_common.h | 2 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 1 +
14 files changed, 1088 insertions(+), 132 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 4c8f8f6..b95d627 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -6,6 +6,7 @@ iris-objs += iris_probe.o \
iris_vidc.o \
iris_vb2.o \
iris_vdec.o \
+ iris_venc.o \
iris_state.o \
iris_ctrls.o \
iris_helpers.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index c56eb24..baced21 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -25,7 +25,8 @@
* @vdev_dec: iris video device structure for decoder
* @vdev_enc: iris video device structure for encoder
* @v4l2_file_ops: iris v4l2 file ops
- * @v4l2_ioctl_ops: iris v4l2 ioctl ops
+ * @v4l2_ioctl_ops_dec: iris v4l2 ioctl ops for decoder
+ * @v4l2_ioctl_ops_enc: iris v4l2 ioctl ops for encoder
* @bus_tbl: table of iris buses
* @bus_count: count of iris buses
* @power_domain_tbl: table of iris power domains
@@ -52,6 +53,7 @@
* @packet_id: id of packet
* @vpu_ops: a pointer to vpu ops
* @session_ops: a pointer to session level ops
+ * @enc_codecs_count: supported codec count for encoder
* @dec_codecs_count: supported codec count for decoder
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
@@ -69,7 +71,8 @@ struct iris_core {
struct video_device *vdev_dec;
struct video_device *vdev_enc;
const struct v4l2_file_operations *v4l2_file_ops;
- const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
+ const struct v4l2_ioctl_ops *v4l2_ioctl_ops_dec;
+ const struct v4l2_ioctl_ops *v4l2_ioctl_ops_enc;
struct bus_info *bus_tbl;
u32 bus_count;
struct power_domain_info *power_domain_tbl;
@@ -97,6 +100,7 @@ struct iris_core {
const struct vpu_ops *vpu_ops;
const struct vpu_session_ops *session_ops;
u32 dec_codecs_count;
+ u32 enc_codecs_count;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
struct plat_inst_caps *inst_caps;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index a648cc1..af99ac73 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -217,7 +217,7 @@ static int set_dynamic_property(struct iris_inst *inst,
return ret;
}
-static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+static int iris_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
{
enum plat_inst_cap_type cap_id;
struct iris_inst *inst = NULL;
@@ -242,7 +242,7 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
return ret;
}
-static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
+static int iris_op_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct cap_entry *entry = NULL, *temp = NULL;
struct list_head children_list, firmware_list;
@@ -273,7 +273,8 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
cap[cap_id].flags |= CAP_FLAG_CLIENT_SET;
- if (!inst->vb2q_src->streaming) {
+ if ((inst->domain == ENCODER && !inst->vb2q_dst->streaming) ||
+ (inst->domain == DECODER && !inst->vb2q_src->streaming)) {
inst->cap[cap_id].value = ctrl->val;
} else {
ret = adjust_dynamic_property(inst, cap_id, ctrl,
@@ -300,16 +301,16 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
}
static const struct v4l2_ctrl_ops ctrl_ops = {
- .s_ctrl = vdec_op_s_ctrl,
- .g_volatile_ctrl = vdec_op_g_volatile_ctrl,
+ .s_ctrl = iris_op_s_ctrl,
+ .g_volatile_ctrl = iris_op_g_volatile_ctrl,
};
int ctrls_init(struct iris_inst *inst, bool init)
{
int num_ctrls = 0, ctrl_idx = 0;
+ u64 codecs_count, step_or_mask;
struct plat_inst_cap *cap;
struct iris_core *core;
- u64 step_or_mask;
int idx = 0;
int ret = 0;
@@ -324,8 +325,11 @@ int ctrls_init(struct iris_inst *inst, bool init)
return -EINVAL;
if (init) {
+ codecs_count = inst->domain == ENCODER ?
+ core->enc_codecs_count :
+ core->dec_codecs_count;
ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
- INST_CAP_MAX * core->dec_codecs_count);
+ INST_CAP_MAX * codecs_count);
if (ret)
return ret;
}
@@ -451,29 +455,49 @@ static int update_inst_capability(struct plat_inst_cap *in,
int iris_init_instance_caps(struct iris_core *core)
{
struct plat_inst_cap *inst_plat_cap_data = NULL;
- u8 dec_codecs_count = 0;
- int num_inst_cap;
- u32 dec_valid_codecs;
+ u8 enc_codecs_count = 0, dec_codecs_count = 0;
+ u32 enc_valid_codecs, dec_valid_codecs;
int i, j, check_bit = 0;
+ u8 codecs_count = 0;
+ int num_inst_cap;
int ret = 0;
inst_plat_cap_data = core->platform_data->inst_cap_data;
if (!inst_plat_cap_data)
return -EINVAL;
+ enc_valid_codecs = core->cap[ENC_CODECS].value;
+ enc_codecs_count = hweight32(enc_valid_codecs);
+ core->enc_codecs_count = enc_codecs_count;
+
dec_valid_codecs = core->cap[DEC_CODECS].value;
dec_codecs_count = hweight32(dec_valid_codecs);
core->dec_codecs_count = dec_codecs_count;
+ codecs_count = enc_codecs_count + dec_codecs_count;
core->inst_caps = devm_kzalloc(core->dev,
- dec_codecs_count * sizeof(struct plat_inst_caps),
+ codecs_count * sizeof(struct plat_inst_caps),
GFP_KERNEL);
if (!core->inst_caps)
return -ENOMEM;
- for (i = 0; i < dec_codecs_count; i++) {
+ for (i = 0; i < enc_codecs_count; i++) {
+ while (check_bit < (sizeof(enc_valid_codecs) * 8)) {
+ if (enc_valid_codecs & BIT(check_bit)) {
+ core->inst_caps[i].domain = ENCODER;
+ core->inst_caps[i].codec = enc_valid_codecs &
+ BIT(check_bit);
+ check_bit++;
+ break;
+ }
+ check_bit++;
+ }
+ }
+
+ for (; i < codecs_count; i++) {
while (check_bit < (sizeof(dec_valid_codecs) * 8)) {
if (dec_valid_codecs & BIT(check_bit)) {
+ core->inst_caps[i].domain = DECODER;
core->inst_caps[i].codec = dec_valid_codecs &
BIT(check_bit);
check_bit++;
@@ -486,9 +510,9 @@ int iris_init_instance_caps(struct iris_core *core)
num_inst_cap = core->platform_data->inst_cap_data_size;
for (i = 0; i < num_inst_cap; i++) {
- for (j = 0; j < dec_codecs_count; j++) {
- if ((inst_plat_cap_data[i].codec &
- core->inst_caps[j].codec)) {
+ for (j = 0; j < codecs_count; j++) {
+ if ((inst_plat_cap_data[i].domain & core->inst_caps[j].domain) &&
+ (inst_plat_cap_data[i].codec & core->inst_caps[j].codec)) {
ret = update_inst_capability(&inst_plat_cap_data[i],
&core->inst_caps[j]);
if (ret)
@@ -503,11 +527,14 @@ int iris_init_instance_caps(struct iris_core *core)
int get_inst_capability(struct iris_inst *inst)
{
struct iris_core *core;
+ u32 codecs_count = 0;
int i;
core = inst->core;
- for (i = 0; i < core->dec_codecs_count; i++) {
+ codecs_count = core->enc_codecs_count + core->dec_codecs_count;
+
+ for (i = 0; i < codecs_count; i++) {
if (core->inst_caps[i].codec == inst->codec) {
memcpy(&inst->cap[0], &core->inst_caps[i].cap[0],
(INST_CAP_MAX + 1) * sizeof(struct plat_inst_cap));
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index d9d22e2..c84bb51 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -7,6 +7,7 @@
#include "hfi_defines.h"
#include "iris_core.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
@@ -273,7 +274,7 @@ int close_session(struct iris_inst *inst)
return ret;
}
-static int check_core_mbps_mbpf(struct iris_inst *inst)
+int check_core_mbps_mbpf(struct iris_inst *inst)
{
u32 mbpf = 0, mbps = 0, total_mbpf = 0, total_mbps = 0;
struct iris_core *core;
@@ -284,7 +285,9 @@ static int check_core_mbps_mbpf(struct iris_inst *inst)
mutex_lock(&core->lock);
list_for_each_entry(instance, &core->instances, list) {
- fps = inst->cap[QUEUED_RATE].value >> 16;
+ fps = max3(inst->cap[QUEUED_RATE].value >> 16,
+ inst->cap[FRAME_RATE].value >> 16,
+ inst->cap[OPERATING_RATE].value >> 16);
mbpf = get_mbpf(inst);
mbps = mbpf * fps;
total_mbpf += mbpf;
@@ -814,6 +817,88 @@ int session_streamoff(struct iris_inst *inst, u32 plane)
return ret;
}
+int process_resume(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ int ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ } else if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST;
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ }
+
+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
+ return ret;
+}
+
+int codec_change(struct iris_inst *inst, u32 v4l2_codec)
+{
+ bool session_init = false;
+ int ret;
+
+ if (!inst->codec)
+ session_init = true;
+
+ if (inst->codec &&
+ ((inst->domain == DECODER && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec) ||
+ (inst->domain == ENCODER && inst->fmt_dst->fmt.pix_mp.pixelformat == v4l2_codec)))
+ return 0;
+
+ inst->codec = v4l2_codec_to_driver(inst, v4l2_codec);
+ if (!inst->codec)
+ return -EINVAL;
+
+ if (inst->domain == DECODER)
+ inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec;
+ else if (inst->domain == ENCODER)
+ inst->fmt_dst->fmt.pix_mp.pixelformat = v4l2_codec;
+
+ ret = get_inst_capability(inst);
+ if (ret)
+ return ret;
+
+ ret = ctrls_init(inst, session_init);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
+
int iris_pm_get(struct iris_core *core)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index c628b2e..39cec8c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -41,6 +41,7 @@ enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat);
enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat);
struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type);
+int check_core_mbps_mbpf(struct iris_inst *inst);
int check_session_supported(struct iris_inst *inst);
struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index);
@@ -51,6 +52,8 @@ int iris_vb2_buffer_done(struct iris_inst *inst,
int iris_release_nonref_buffers(struct iris_inst *inst);
void iris_destroy_buffers(struct iris_inst *inst);
int session_streamoff(struct iris_inst *inst, u32 plane);
+int process_resume(struct iris_inst *inst);
+int codec_change(struct iris_inst *inst, u32 v4l2_codec);
int iris_pm_get(struct iris_core *core);
int iris_pm_put(struct iris_core *core, bool autosuspend);
int iris_pm_get_put(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 70f4c7d..a1547c5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -8,6 +8,7 @@
#include <media/v4l2-ctrls.h>
+#include "hfi_defines.h"
#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_core.h"
@@ -29,6 +30,7 @@
* @fmt_dst: structure of v4l2_format for destination
* @ctrl_handler: reference of v4l2 ctrl handler
* @crop: structure of crop info
+ * @compose: structure of compose info
* @packet: HFI packet
* @packet_size: HFI packet size
* @completions: structure of signal completions
@@ -36,6 +38,7 @@
* @num_ctrls: supported number of controls
* @caps_list: list head of capability
* @codec: codec type
+ * @domain: domain type: encoder or decoder
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
@@ -70,6 +73,7 @@ struct iris_inst {
struct v4l2_format *fmt_dst;
struct v4l2_ctrl_handler ctrl_handler;
struct rect_desc crop;
+ struct rect_desc compose;
void *packet;
u32 packet_size;
struct completion completions[MAX_SIGNAL];
@@ -77,6 +81,7 @@ struct iris_inst {
u32 num_ctrls;
struct list_head caps_list;
enum codec_type codec;
+ enum domain_type domain;
struct iris_buffers_info buffers;
u32 fw_min_count;
enum iris_inst_state state;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index b487e83..49d2701 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -42,13 +42,13 @@ static int iris_register_video_device(struct iris_core *core, enum domain_type t
vdev->release = video_device_release;
vdev->fops = core->v4l2_file_ops;
- vdev->ioctl_ops = core->v4l2_ioctl_ops;
vdev->vfl_dir = VFL_DIR_M2M;
vdev->v4l2_dev = &core->v4l2_dev;
vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
if (type == DECODER) {
strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
+ vdev->ioctl_ops = core->v4l2_ioctl_ops_dec;
ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
if (ret)
@@ -57,6 +57,7 @@ static int iris_register_video_device(struct iris_core *core, enum domain_type t
core->vdev_dec = vdev;
} else if (type == ENCODER) {
strscpy(vdev->name, "qcom-iris-encoder", sizeof(vdev->name));
+ vdev->ioctl_ops = core->v4l2_ioctl_ops_enc;
ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
if (ret)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 952ba2a..e853aec 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -198,8 +198,9 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id)
{
return ((inst->state == IRIS_INST_OPEN) ||
((inst->cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED) &&
- (inst->state == IRIS_INST_INPUT_STREAMING ||
- inst->state == IRIS_INST_STREAMING)));
+ ((inst->state == IRIS_INST_INPUT_STREAMING && inst->domain == DECODER) ||
+ (inst->state == IRIS_INST_OUTPUT_STREAMING && inst->domain == ENCODER) ||
+ (inst->state == IRIS_INST_STREAMING))));
}
int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 371615e..300d0e9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -22,39 +22,6 @@ struct vdec_prop_type_handle {
int (*handle)(struct iris_inst *inst);
};
-static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
-{
- bool session_init = false;
- int ret;
-
- if (!inst->codec)
- session_init = true;
-
- if (inst->codec && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec)
- return 0;
-
- inst->codec = v4l2_codec_to_driver(inst, v4l2_codec);
- if (!inst->codec)
- return -EINVAL;
-
- inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec;
- ret = get_inst_capability(inst);
- if (ret)
- return ret;
-
- ret = ctrls_init(inst, session_init);
- if (ret)
- return ret;
-
- ret = update_buffer_count(inst, INPUT_MPLANE);
- if (ret)
- return ret;
-
- ret = update_buffer_count(inst, OUTPUT_MPLANE);
-
- return ret;
-}
-
int vdec_inst_init(struct iris_inst *inst)
{
struct v4l2_format *f;
@@ -93,7 +60,7 @@ int vdec_inst_init(struct iris_inst *inst)
inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
inst->fw_min_count = 0;
- ret = vdec_codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat);
+ ret = codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat);
return ret;
}
@@ -233,7 +200,7 @@ int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
if (f->type == INPUT_MPLANE) {
if (inst->fmt_src->fmt.pix_mp.pixelformat !=
f->fmt.pix_mp.pixelformat) {
- ret = vdec_codec_change(inst, f->fmt.pix_mp.pixelformat);
+ ret = codec_change(inst, f->fmt.pix_mp.pixelformat);
if (ret)
return ret;
}
@@ -1304,49 +1271,6 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
return ret;
}
-static int process_resume(struct iris_inst *inst)
-{
- enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
- int ret;
-
- if (inst->sub_state & IRIS_INST_SUB_DRC &&
- inst->sub_state & IRIS_INST_SUB_DRC_LAST) {
- clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
-
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
- }
- if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
- ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
- }
- } else if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
- inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) {
- clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST;
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
- }
- if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
- ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
- }
- }
-
- ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
-
- return ret;
-}
-
int vdec_start_cmd(struct iris_inst *inst)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
new file mode 100644
index 0000000..802db40
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
@@ -0,0 +1,525 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "hfi_defines.h"
+#include "iris_buffer.h"
+#include "iris_common.h"
+#include "iris_ctrls.h"
+#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
+#include "iris_power.h"
+#include "iris_venc.h"
+
+int venc_inst_init(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ int ret;
+
+ inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL);
+ if (!inst->fmt_src)
+ return -ENOMEM;
+
+ inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL);
+ if (!inst->fmt_dst)
+ return -ENOMEM;
+
+ inst->vb2q_src = kzalloc(sizeof(*inst->vb2q_src), GFP_KERNEL);
+ if (!inst->vb2q_src)
+ return -ENOMEM;
+
+ inst->vb2q_dst = kzalloc(sizeof(*inst->vb2q_dst), GFP_KERNEL);
+ if (!inst->vb2q_dst)
+ return -ENOMEM;
+
+ f = inst->fmt_dst;
+ f->type = OUTPUT_MPLANE;
+ f->fmt.pix_mp.width = DEFAULT_WIDTH;
+ f->fmt.pix_mp.height = DEFAULT_HEIGHT;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ inst->buffers.output.actual_count = inst->buffers.output.min_count;
+ inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ inst->crop.left = 0;
+ inst->crop.top = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+
+ inst->compose.left = 0;
+ inst->compose.top = 0;
+ inst->compose.width = f->fmt.pix_mp.width;
+ inst->compose.height = f->fmt.pix_mp.height;
+
+ f = inst->fmt_src;
+ f->type = INPUT_MPLANE;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_QC08C;
+ f->fmt.pix_mp.width = DEFAULT_WIDTH;
+ f->fmt.pix_mp.height = DEFAULT_HEIGHT;
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128);
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT);
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT);
+ inst->buffers.input.actual_count = inst->buffers.input.min_count;
+ inst->buffers.input.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ inst->hfi_rc_type = HFI_RC_VBR_CFR;
+ inst->hfi_layer_type = HFI_HIER_P_SLIDING_WINDOW;
+
+ ret = codec_change(inst, inst->fmt_dst->fmt.pix_mp.pixelformat);
+
+ return ret;
+}
+
+void venc_inst_deinit(struct iris_inst *inst)
+{
+ kfree(inst->fmt_dst);
+ kfree(inst->fmt_src);
+}
+
+int venc_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f)
+{
+ struct iris_core *core;
+ u32 array[32] = {0};
+ u32 i = 0;
+
+ core = inst->core;
+
+ if (f->type == OUTPUT_MPLANE) {
+ u32 codecs = core->cap[ENC_CODECS].value;
+ u32 idx = 0;
+
+ for (i = 0; i <= 31; i++) {
+ if (codecs & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ array[idx] = codecs & BIT(i);
+ idx++;
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_codec_from_driver(inst, array[f->index]);
+ if (!f->pixelformat)
+ return -EINVAL;
+ f->flags = V4L2_FMT_FLAG_COMPRESSED;
+ strscpy(f->description, "codec", sizeof(f->description));
+ } else if (f->type == INPUT_MPLANE) {
+ u32 formats = inst->cap[PIX_FMTS].step_or_mask;
+ u32 idx = 0;
+
+ for (i = 0; i <= 31; i++) {
+ if (formats & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ array[idx] = formats & BIT(i);
+ idx++;
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_colorformat_from_driver(inst, array[f->index]);
+ if (!f->pixelformat)
+ return -EINVAL;
+ strscpy(f->description, "colorformat", sizeof(f->description));
+ }
+
+ memset(f->reserved, 0, sizeof(f->reserved));
+
+ return 0;
+}
+
+int venc_try_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ u32 pix_fmt;
+
+ memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+
+ if (f->type == INPUT_MPLANE) {
+ pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f->fmt.pix_mp.pixelformat = inst->fmt_src->fmt.pix_mp.pixelformat;
+ f->fmt.pix_mp.width = inst->fmt_src->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = inst->fmt_src->fmt.pix_mp.height;
+ }
+ } else if (f->type == OUTPUT_MPLANE) {
+ pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f->fmt.pix_mp.width = inst->fmt_dst->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = inst->fmt_dst->fmt.pix_mp.height;
+ f->fmt.pix_mp.pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ if (pixmp->field == V4L2_FIELD_ANY)
+ pixmp->field = V4L2_FIELD_NONE;
+ pixmp->num_planes = 1;
+
+ return 0;
+}
+
+static int venc_s_fmt_output(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_format *fmt;
+ enum codec_type codec;
+ u32 codec_align;
+ u32 width, height;
+ int ret = 0;
+
+ venc_try_fmt(inst, f);
+
+ fmt = inst->fmt_dst;
+ if (fmt->fmt.pix_mp.pixelformat != f->fmt.pix_mp.pixelformat) {
+ ret = codec_change(inst, f->fmt.pix_mp.pixelformat);
+ if (ret)
+ return ret;
+ }
+ fmt->type = OUTPUT_MPLANE;
+
+ codec = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+
+ codec_align = (codec == HEVC) ? 32 : 16;
+ width = inst->compose.width;
+ height = inst->compose.height;
+ if (inst->cap[ROTATION].value == 90 || inst->cap[ROTATION].value == 270) {
+ width = inst->compose.height;
+ height = inst->compose.width;
+ }
+ fmt->fmt.pix_mp.width = ALIGN(width, codec_align);
+ fmt->fmt.pix_mp.height = ALIGN(height, codec_align);
+ fmt->fmt.pix_mp.num_planes = 1;
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage =
+ iris_get_buffer_size(inst, BUF_OUTPUT);
+ if (f->fmt.pix_mp.colorspace != V4L2_COLORSPACE_DEFAULT &&
+ f->fmt.pix_mp.colorspace != V4L2_COLORSPACE_REC709)
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ if (inst->buffers.output.actual_count <
+ inst->buffers.output.min_count) {
+ inst->buffers.output.actual_count =
+ inst->buffers.output.min_count;
+ }
+ inst->buffers.output.size =
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ memcpy(f, fmt, sizeof(struct v4l2_format));
+
+ return ret;
+}
+
+static int venc_s_fmt_input(struct iris_inst *inst, struct v4l2_format *f)
+{
+ u32 pix_fmt, width, height, size, bytesperline;
+ struct v4l2_format *fmt, *output_fmt;
+ int ret = 0;
+
+ venc_try_fmt(inst, f);
+
+ pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ inst->cap[PIX_FMTS].value = pix_fmt;
+
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ bytesperline = pix_fmt == FMT_TP10C ?
+ ALIGN(ALIGN(f->fmt.pix_mp.width, 192) * 4 / 3, 256) :
+ ALIGN(f->fmt.pix_mp.width, 128);
+
+ fmt = inst->fmt_src;
+ fmt->type = INPUT_MPLANE;
+ fmt->fmt.pix_mp.width = width;
+ fmt->fmt.pix_mp.height = height;
+ fmt->fmt.pix_mp.num_planes = 1;
+ fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat;
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = bytesperline;
+ size = iris_get_buffer_size(inst, BUF_INPUT);
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage = size;
+ fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+
+ output_fmt = inst->fmt_dst;
+ output_fmt->fmt.pix_mp.colorspace = fmt->fmt.pix_mp.colorspace;
+ output_fmt->fmt.pix_mp.xfer_func = fmt->fmt.pix_mp.xfer_func;
+ output_fmt->fmt.pix_mp.ycbcr_enc = fmt->fmt.pix_mp.ycbcr_enc;
+ output_fmt->fmt.pix_mp.quantization = fmt->fmt.pix_mp.quantization;
+
+ inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT);
+ if (inst->buffers.input.actual_count <
+ inst->buffers.input.min_count) {
+ inst->buffers.input.actual_count =
+ inst->buffers.input.min_count;
+ }
+ inst->buffers.input.size = size;
+
+ if (f->fmt.pix_mp.width != inst->crop.width ||
+ f->fmt.pix_mp.height != inst->crop.height) {
+ inst->crop.top = 0;
+ inst->crop.left = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+
+ inst->compose.top = 0;
+ inst->compose.left = 0;
+ inst->compose.width = f->fmt.pix_mp.width;
+ inst->compose.height = f->fmt.pix_mp.height;
+
+ ret = venc_s_fmt_output(inst, output_fmt);
+ if (ret)
+ return ret;
+ }
+
+ memcpy(f, fmt, sizeof(struct v4l2_format));
+
+ return ret;
+}
+
+int venc_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ int ret;
+
+ if (f->type == INPUT_MPLANE)
+ ret = venc_s_fmt_input(inst, f);
+ else if (f->type == OUTPUT_MPLANE)
+ ret = venc_s_fmt_output(inst, f);
+ else
+ ret = -EINVAL;
+
+ return ret;
+}
+
+int venc_s_selection(struct iris_inst *inst, struct v4l2_selection *s)
+{
+ struct v4l2_format *output_fmt;
+ int ret;
+
+ if (s->type != INPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP:
+ if (s->r.left || s->r.top) {
+ s->r.left = 0;
+ s->r.top = 0;
+ }
+ if (s->r.width > inst->fmt_src->fmt.pix_mp.width)
+ s->r.width = inst->fmt_src->fmt.pix_mp.width;
+
+ if (s->r.height > inst->fmt_src->fmt.pix_mp.height)
+ s->r.height = inst->fmt_src->fmt.pix_mp.height;
+
+ inst->crop.left = s->r.left;
+ inst->crop.top = s->r.top;
+ inst->crop.width = s->r.width;
+ inst->crop.height = s->r.height;
+ inst->compose.left = inst->crop.left;
+ inst->compose.top = inst->crop.top;
+ inst->compose.width = inst->crop.width;
+ inst->compose.height = inst->crop.height;
+ output_fmt = inst->fmt_dst;
+ ret = venc_s_fmt_output(inst, output_fmt);
+ if (ret)
+ return ret;
+ break;
+ case V4L2_SEL_TGT_COMPOSE:
+ if (s->r.left < inst->crop.left)
+ s->r.left = inst->crop.left;
+
+ if (s->r.top < inst->crop.top)
+ s->r.top = inst->crop.top;
+
+ if (s->r.width > inst->crop.width)
+ s->r.width = inst->crop.width;
+
+ if (s->r.height > inst->crop.height)
+ s->r.height = inst->crop.height;
+ inst->compose.left = s->r.left;
+ inst->compose.top = s->r.top;
+ inst->compose.width = s->r.width;
+ inst->compose.height = s->r.height;
+
+ output_fmt = inst->fmt_dst;
+ ret = venc_s_fmt_output(inst, output_fmt);
+ if (ret)
+ return ret;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+int venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
+{
+ struct v4l2_fract *timeperframe = NULL;
+ u32 q16_rate, max_rate, default_rate;
+ u64 us_per_frame = 0, input_rate = 0;
+ bool is_frame_rate = false;
+ int ret = 0;
+
+ if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ timeperframe = &s_parm->parm.output.timeperframe;
+ max_rate = inst->cap[OPERATING_RATE].max >> 16;
+ default_rate = inst->cap[OPERATING_RATE].value >> 16;
+ s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ } else {
+ timeperframe = &s_parm->parm.capture.timeperframe;
+ is_frame_rate = true;
+ max_rate = inst->cap[FRAME_RATE].max >> 16;
+ default_rate = inst->cap[FRAME_RATE].value >> 16;
+ s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ }
+
+ if (!timeperframe->denominator || !timeperframe->numerator) {
+ if (!timeperframe->numerator)
+ timeperframe->numerator = 1;
+ if (!timeperframe->denominator)
+ timeperframe->denominator = default_rate;
+ }
+
+ us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
+ us_per_frame = div64_u64(us_per_frame, timeperframe->denominator);
+
+ if (!us_per_frame) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ input_rate = (u64)USEC_PER_SEC;
+ input_rate = div64_u64(input_rate, us_per_frame);
+
+ q16_rate = (u32)input_rate << 16;
+ if (is_frame_rate)
+ inst->cap[FRAME_RATE].value = q16_rate;
+ else
+ inst->cap[OPERATING_RATE].value = q16_rate;
+
+ if ((s_parm->type == INPUT_MPLANE && inst->vb2q_src->streaming) ||
+ (s_parm->type == OUTPUT_MPLANE && inst->vb2q_dst->streaming)) {
+ ret = check_core_mbps_mbpf(inst);
+ if (ret)
+ goto reset_rate;
+ ret = input_rate > max_rate;
+ if (ret) {
+ ret = -ENOMEM;
+ goto reset_rate;
+ }
+ }
+
+ if (is_frame_rate)
+ inst->cap[FRAME_RATE].flags |= CAP_FLAG_CLIENT_SET;
+ else
+ inst->cap[OPERATING_RATE].flags |= CAP_FLAG_CLIENT_SET;
+
+ if (inst->vb2q_dst->streaming) {
+ ret = iris_hfi_set_property(inst,
+ HFI_PROP_FRAME_RATE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_BITSTREAM,
+ HFI_PAYLOAD_Q16,
+ &q16_rate,
+ sizeof(u32));
+ if (ret)
+ goto exit;
+ }
+
+ return ret;
+
+reset_rate:
+ if (ret) {
+ if (is_frame_rate)
+ inst->cap[FRAME_RATE].value = default_rate << 16;
+ else
+ inst->cap[OPERATING_RATE].value = default_rate << 16;
+ }
+exit:
+ return ret;
+}
+
+int venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
+{
+ struct v4l2_fract *timeperframe = NULL;
+
+ if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ timeperframe = &s_parm->parm.output.timeperframe;
+ timeperframe->numerator = 1;
+ timeperframe->denominator =
+ inst->cap[OPERATING_RATE].value >> 16;
+ s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ } else {
+ timeperframe = &s_parm->parm.capture.timeperframe;
+ timeperframe->numerator = 1;
+ timeperframe->denominator =
+ inst->cap[FRAME_RATE].value >> 16;
+ s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ }
+
+ return 0;
+}
+
+int venc_subscribe_event(struct iris_inst *inst,
+ const struct v4l2_event_subscription *sub)
+{
+ int ret;
+
+ switch (sub->type) {
+ case V4L2_EVENT_EOS:
+ ret = v4l2_event_subscribe(&inst->fh, sub, MAX_EVENTS, NULL);
+ break;
+ case V4L2_EVENT_CTRL:
+ ret = v4l2_ctrl_subscribe_event(&inst->fh, sub);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return ret;
+}
+
+int venc_start_cmd(struct iris_inst *inst)
+{
+ vb2_clear_last_buffer_dequeued(inst->vb2q_dst);
+
+ return process_resume(inst);
+}
+
+int venc_stop_cmd(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_drain(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_DRAIN);
+
+ iris_scale_power(inst);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
new file mode 100644
index 0000000..24da63f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VENC_H_
+#define _IRIS_VENC_H_
+
+#include "iris_instance.h"
+
+int venc_inst_init(struct iris_inst *inst);
+void venc_inst_deinit(struct iris_inst *inst);
+int venc_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
+int venc_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int venc_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int venc_s_selection(struct iris_inst *inst, struct v4l2_selection *s);
+int venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm);
+int venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm);
+int venc_subscribe_event(struct iris_inst *inst,
+ const struct v4l2_event_subscription *sub);
+int venc_start_cmd(struct iris_inst *inst);
+int venc_stop_cmd(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 437d6b4..aa19978 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -15,6 +15,7 @@
#include "iris_instance.h"
#include "iris_power.h"
#include "iris_vdec.h"
+#include "iris_venc.h"
#include "iris_vidc.h"
#include "iris_vb2.h"
@@ -30,7 +31,11 @@ static int vidc_v4l2_fh_init(struct iris_inst *inst)
if (inst->fh.vdev)
return -EINVAL;
- v4l2_fh_init(&inst->fh, core->vdev_dec);
+ if (inst->domain == ENCODER)
+ v4l2_fh_init(&inst->fh, core->vdev_enc);
+ else if (inst->domain == DECODER)
+ v4l2_fh_init(&inst->fh, core->vdev_dec);
+
inst->fh.ctrl_handler = &inst->ctrl_handler;
v4l2_fh_add(&inst->fh);
@@ -160,9 +165,20 @@ int vidc_open(struct file *filp)
{
struct iris_core *core = video_drvdata(filp);
struct iris_inst *inst = NULL;
+ struct video_device *vdev;
+ u32 session_type = 0;
int i = 0;
int ret;
+ vdev = video_devdata(filp);
+ if (strcmp(vdev->name, "qcom-iris-decoder") == 0)
+ session_type = DECODER;
+ else if (strcmp(vdev->name, "qcom-iris-encoder") == 0)
+ session_type = ENCODER;
+
+ if (session_type != DECODER && session_type != ENCODER)
+ return -EINVAL;
+
ret = iris_pm_get(core);
if (ret)
return ret;
@@ -182,6 +198,7 @@ int vidc_open(struct file *filp)
}
inst->core = core;
+ inst->domain = session_type;
inst->session_id = hash32_ptr(inst);
inst->ipsc_properties_set = false;
inst->opsc_properties_set = false;
@@ -213,7 +230,12 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_remove_session;
- vdec_inst_init(inst);
+ if (inst->domain == DECODER)
+ ret = vdec_inst_init(inst);
+ else if (inst->domain == ENCODER)
+ ret = venc_inst_init(inst);
+ if (ret)
+ goto fail_fh_deinit;
ret = vidc_vb2_queue_init(inst);
if (ret)
@@ -238,7 +260,11 @@ int vidc_open(struct file *filp)
iris_core_deinit(core);
vidc_vb2_queue_deinit(inst);
fail_inst_deinit:
- vdec_inst_deinit(inst);
+ if (inst->domain == DECODER)
+ vdec_inst_deinit(inst);
+ else if (inst->domain == ENCODER)
+ venc_inst_deinit(inst);
+fail_fh_deinit:
vidc_v4l2_fh_deinit(inst);
fail_remove_session:
vidc_remove_session(inst);
@@ -264,7 +290,11 @@ int vidc_close(struct file *filp)
core = inst->core;
v4l2_ctrl_handler_free(&inst->ctrl_handler);
- vdec_inst_deinit(inst);
+ if (inst->domain == DECODER)
+ vdec_inst_deinit(inst);
+ else if (inst->domain == ENCODER)
+ venc_inst_deinit(inst);
+
mutex_lock(&inst->lock);
iris_pm_get(core);
close_session(inst);
@@ -342,7 +372,7 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;
inst = get_vidc_inst(filp, fh);
if (!inst)
@@ -354,7 +384,10 @@ static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
goto unlock;
}
- ret = vdec_enum_fmt(inst, f);
+ if (inst->domain == DECODER)
+ ret = vdec_enum_fmt(inst, f);
+ else if (inst->domain == ENCODER)
+ ret = venc_enum_fmt(inst, f);
unlock:
mutex_unlock(&inst->lock);
@@ -365,7 +398,7 @@ static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;
inst = get_vidc_inst(filp, fh);
if (!inst)
@@ -382,7 +415,10 @@ static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
goto unlock;
}
- ret = vdec_try_fmt(inst, f);
+ if (inst->domain == DECODER)
+ ret = vdec_try_fmt(inst, f);
+ else if (inst->domain == ENCODER)
+ ret = venc_try_fmt(inst, f);
unlock:
mutex_unlock(&inst->lock);
@@ -393,7 +429,7 @@ static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;
inst = get_vidc_inst(filp, fh);
if (!inst)
@@ -410,7 +446,10 @@ static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f)
goto unlock;
}
- ret = vdec_s_fmt(inst, f);
+ if (inst->domain == DECODER)
+ ret = vdec_s_fmt(inst, f);
+ else if (inst->domain == ENCODER)
+ ret = venc_s_fmt(inst, f);
unlock:
mutex_unlock(&inst->lock);
@@ -488,6 +527,70 @@ static int vidc_enum_framesizes(struct file *filp, void *fh,
return ret;
}
+static int vidc_enum_frameintervals(struct file *filp, void *fh,
+ struct v4l2_frmivalenum *fival)
+
+{
+ enum colorformat_type colorfmt;
+ struct iris_inst *inst;
+ struct iris_core *core;
+ u32 fps, mbpf;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !fival)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (inst->domain == DECODER) {
+ ret = -ENOTTY;
+ goto unlock;
+ }
+
+ core = inst->core;
+
+ if (fival->index) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ colorfmt = v4l2_colorformat_to_driver(inst, fival->pixel_format);
+ if (colorfmt == FMT_NONE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (fival->width > inst->cap[FRAME_WIDTH].max ||
+ fival->width < inst->cap[FRAME_WIDTH].min ||
+ fival->height > inst->cap[FRAME_HEIGHT].max ||
+ fival->height < inst->cap[FRAME_HEIGHT].min) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ mbpf = NUM_MBS_PER_FRAME(fival->height, fival->width);
+ fps = core->cap[MAX_MBPS].value / mbpf;
+
+ fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
+ fival->stepwise.min.numerator = 1;
+ fival->stepwise.min.denominator =
+ min_t(u32, fps, inst->cap[FRAME_RATE].max);
+ fival->stepwise.max.numerator = 1;
+ fival->stepwise.max.denominator = 1;
+ fival->stepwise.step.numerator = 1;
+ fival->stepwise.step.denominator = inst->cap[FRAME_RATE].max;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static int vidc_reqbufs(struct file *filp, void *fh, struct v4l2_requestbuffers *b)
{
struct vb2_queue *vb2q = NULL;
@@ -751,7 +854,11 @@ static int vidc_querycap(struct file *filp, void *fh, struct v4l2_capability *ca
strscpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
strscpy(cap->bus_info, VIDC_BUS_NAME, sizeof(cap->bus_info));
memset(cap->reserved, 0, sizeof(cap->reserved));
- strscpy(cap->card, "iris_decoder", sizeof(cap->card));
+
+ if (inst->domain == DECODER)
+ strscpy(cap->card, "iris_decoder", sizeof(cap->card));
+ else if (inst->domain == ENCODER)
+ strscpy(cap->card, "iris_encoder", sizeof(cap->card));
unlock:
mutex_unlock(&inst->lock);
@@ -839,7 +946,7 @@ static int vidc_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qm
static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;
inst = container_of(fh, struct iris_inst, fh);
@@ -849,7 +956,10 @@ static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subs
goto unlock;
}
- ret = vdec_subscribe_event(inst, sub);
+ if (inst->domain == DECODER)
+ ret = vdec_subscribe_event(inst, sub);
+ else if (inst->domain == ENCODER)
+ ret = venc_subscribe_event(inst, sub);
unlock:
mutex_unlock(&inst->lock);
@@ -893,28 +1003,152 @@ static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection *
goto unlock;
}
- if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) {
+ if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE &&
+ inst->domain == DECODER) {
ret = -EINVAL;
goto unlock;
}
- switch (s->target) {
- case V4L2_SEL_TGT_CROP_BOUNDS:
- case V4L2_SEL_TGT_CROP_DEFAULT:
- case V4L2_SEL_TGT_CROP:
- case V4L2_SEL_TGT_COMPOSE_BOUNDS:
- case V4L2_SEL_TGT_COMPOSE_PADDED:
- case V4L2_SEL_TGT_COMPOSE_DEFAULT:
- case V4L2_SEL_TGT_COMPOSE:
- s->r.left = inst->crop.left;
- s->r.top = inst->crop.top;
- s->r.width = inst->crop.width;
- s->r.height = inst->crop.height;
- break;
- default:
+ if (s->type != INPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT &&
+ inst->domain == ENCODER) {
ret = -EINVAL;
+ goto unlock;
}
+ if (inst->domain == DECODER) {
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP:
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE:
+ s->r.left = inst->crop.left;
+ s->r.top = inst->crop.top;
+ s->r.width = inst->crop.width;
+ s->r.height = inst->crop.height;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP:
+ s->r.left = inst->crop.left;
+ s->r.top = inst->crop.top;
+ s->r.width = inst->crop.width;
+ s->r.height = inst->crop.height;
+ break;
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE:
+ s->r.left = inst->compose.left;
+ s->r.top = inst->compose.top;
+ s->r.width = inst->compose.width;
+ s->r.height = inst->compose.height;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_s_selection(struct file *filp, void *fh, struct v4l2_selection *s)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !s)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+ if (inst->domain == DECODER) {
+ ret = -EINVAL;
+ goto unlock;
+ } else if (inst->domain == ENCODER) {
+ ret = venc_s_selection(inst, s);
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_s_parm(struct file *filp, void *fh, struct v4l2_streamparm *a)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !a)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (inst->domain == ENCODER)
+ ret = venc_s_param(inst, a);
+ else
+ ret = -EINVAL;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_g_parm(struct file *filp, void *fh, struct v4l2_streamparm *a)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !a)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (inst->domain == ENCODER)
+ ret = venc_g_param(inst, a);
+ else
+ ret = -EINVAL;
+
unlock:
mutex_unlock(&inst->lock);
@@ -955,6 +1189,39 @@ static int vidc_try_dec_cmd(struct file *filp, void *fh,
return ret;
}
+static int vidc_try_enc_cmd(struct file *filp, void *fh,
+ struct v4l2_encoder_cmd *enc)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !enc)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (inst->domain != ENCODER) {
+ ret = -ENOTTY;
+ goto unlock;
+ }
+
+ if (enc->cmd != V4L2_ENC_CMD_STOP && enc->cmd != V4L2_ENC_CMD_START) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ enc->flags = 0;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static int vidc_dec_cmd(struct file *filp, void *fh,
struct v4l2_decoder_cmd *dec)
{
@@ -1002,6 +1269,60 @@ static int vidc_dec_cmd(struct file *filp, void *fh,
return ret;
}
+static int vidc_enc_cmd(struct file *filp, void *fh,
+ struct v4l2_encoder_cmd *enc)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !enc)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (inst->domain != ENCODER) {
+ ret = -ENOTTY;
+ goto unlock;
+ }
+
+ if (enc->cmd != V4L2_ENC_CMD_START &&
+ enc->cmd != V4L2_ENC_CMD_STOP) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (enc->cmd == V4L2_ENC_CMD_STOP && inst->state == IRIS_INST_OPEN) {
+ ret = 0;
+ goto unlock;
+ }
+
+ if (!allow_cmd(inst, enc->cmd)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = iris_pm_get(inst->core);
+ if (ret)
+ goto unlock;
+
+ if (enc->cmd == V4L2_ENC_CMD_START)
+ ret = venc_start_cmd(inst);
+ else if (enc->cmd == V4L2_ENC_CMD_STOP)
+ ret = venc_stop_cmd(inst);
+
+ iris_pm_put(inst->core, true);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static struct v4l2_file_operations v4l2_file_ops = {
.owner = THIS_MODULE,
.open = vidc_open,
@@ -1027,7 +1348,7 @@ static struct vb2_mem_ops iris_vb2_mem_ops = {
.unmap_dmabuf = iris_vb2_unmap_dmabuf,
};
-static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
+static const struct v4l2_ioctl_ops v4l2_ioctl_ops_dec = {
.vidioc_enum_fmt_vid_cap = vidc_enum_fmt,
.vidioc_enum_fmt_vid_out = vidc_enum_fmt,
.vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt,
@@ -1055,12 +1376,44 @@ static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
.vidioc_decoder_cmd = vidc_dec_cmd,
};
+static const struct v4l2_ioctl_ops v4l2_ioctl_ops_enc = {
+ .vidioc_enum_fmt_vid_cap = vidc_enum_fmt,
+ .vidioc_enum_fmt_vid_out = vidc_enum_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt,
+ .vidioc_try_fmt_vid_out_mplane = vidc_try_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = vidc_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = vidc_s_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = vidc_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = vidc_g_fmt,
+ .vidioc_enum_framesizes = vidc_enum_framesizes,
+ .vidioc_enum_frameintervals = vidc_enum_frameintervals,
+ .vidioc_reqbufs = vidc_reqbufs,
+ .vidioc_querybuf = vidc_querybuf,
+ .vidioc_create_bufs = vidc_create_bufs,
+ .vidioc_prepare_buf = vidc_prepare_buf,
+ .vidioc_qbuf = vidc_qbuf,
+ .vidioc_dqbuf = vidc_dqbuf,
+ .vidioc_streamon = vidc_streamon,
+ .vidioc_streamoff = vidc_streamoff,
+ .vidioc_querycap = vidc_querycap,
+ .vidioc_queryctrl = vidc_queryctrl,
+ .vidioc_querymenu = vidc_querymenu,
+ .vidioc_subscribe_event = vidc_subscribe_event,
+ .vidioc_unsubscribe_event = vidc_unsubscribe_event,
+ .vidioc_g_selection = vidc_g_selection,
+ .vidioc_s_selection = vidc_s_selection,
+ .vidioc_s_parm = vidc_s_parm,
+ .vidioc_g_parm = vidc_g_parm,
+ .vidioc_try_encoder_cmd = vidc_try_enc_cmd,
+ .vidioc_encoder_cmd = vidc_enc_cmd,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
core->vb2_ops = &iris_vb2_ops;
core->vb2_mem_ops = &iris_vb2_mem_ops;
- core->v4l2_ioctl_ops = &v4l2_ioctl_ops;
-
+ core->v4l2_ioctl_ops_dec = &v4l2_ioctl_ops_dec;
+ core->v4l2_ioctl_ops_enc = &v4l2_ioctl_ops_enc;
return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 443894c..effecbb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -115,6 +115,7 @@ struct iris_inst_power {
enum plat_core_cap_type {
CORE_CAP_NONE = 0,
DEC_CODECS,
+ ENC_CODECS,
MAX_SESSION_COUNT,
MAX_MBPF,
MAX_MBPS,
@@ -249,6 +250,7 @@ struct plat_inst_cap {
struct plat_inst_caps {
enum codec_type codec;
+ enum domain_type domain;
struct plat_inst_cap cap[INST_CAP_MAX + 1];
};
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 6d5192a..ef0aad7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -63,6 +63,7 @@ static struct color_format_info color_format_data_sm8550[] = {
static struct plat_core_cap core_data_sm8550[] = {
{DEC_CODECS, H264 | HEVC | VP9},
+ {ENC_CODECS, H264 | HEVC},
{MAX_SESSION_COUNT, 16},
{MAX_MBPF, 278528}, /* ((8192x4352)/256) * 2 */
{MAX_MBPS, 7833600}, /* max_load 7680x4320@60fps */
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 33/34] media: iris: add vb2 streaming and buffer ops for encoder
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (31 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 32/34] media: iris: implement iris v4l2 ioctl ops supported by encoder Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-18 11:32 ` [PATCH v2 34/34] media: iris: add power management " Dikshita Agarwal
` (7 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
During stream on, set some mandatory properties
to firmware to start a session. Set all v4l2 properties set
by client, to firmware prepared with the dependency graph.
Also, configure the hardware internal buffers required
for frame processing. Send HFI_CMD_START on capture and
output planes to start the processing on respective planes.
The instance state is changed accordingly.
During stream off, send HFI_CMD_STOP to firmware which is
a synchronous command. After the response is received from
firmware, the session is closed on firmware and
instance state is changed accordingly.
Buffers are submitted and received via HFI_CMD_BUFFER.
Add Internal buffer size calculations for Encoder.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 11 +
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 296 ++++++++--
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 49 +-
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 169 +++++-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 6 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 22 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 137 +++--
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 9 +-
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 143 +++--
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 62 +-
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 137 +----
.../media/platform/qcom/vcodec/iris/iris_venc.c | 423 ++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_venc.h | 3 +
.../platform/qcom/vcodec/iris/platform_common.h | 5 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 2 +
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 198 ++++++-
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 624 +++++++++++++++++++++
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 2 +-
19 files changed, 1992 insertions(+), 307 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index bc32c99..59765de 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -284,6 +284,10 @@ enum hfi_picture_type {
#define HFI_PROP_NO_OUTPUT 0x0300016a
+#define HFI_PROP_BUFFER_MARK 0x0300016c
+
+#define HFI_PROP_RAW_RESOLUTION 0x03000178
+
#define HFI_PROP_DPB_LIST 0x0300017A
#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
@@ -372,4 +376,11 @@ enum hfi_buffer_firmware_flags {
HFI_BUF_FW_FLAG_PSC_LAST = 0x20000000,
};
+enum hfi_hevc_profile_type {
+ HFI_H265_PROFILE_MAIN = 0,
+ HFI_H265_PROFILE_MAIN_STILL_PICTURE = 1,
+ HFI_H265_PROFILE_MAIN_10 = 2,
+ HFI_H265_PROFILE_MAIN_10_STILL_PICTURE = 3,
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index f85b52c..18418e4e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -10,6 +10,7 @@
#include "hfi_defines.h"
#include "iris_hfi_packet.h"
#include "iris_instance.h"
+#include "vpu_iris3_buffer.h"
static const u32 dec_ip_int_buf_type[] = {
BUF_BIN,
@@ -22,15 +23,51 @@ static const u32 dec_op_int_buf_type[] = {
BUF_DPB,
};
+static const u32 enc_ip_int_buf_type[] = {
+ BUF_VPSS,
+};
+
+static const u32 enc_op_int_buf_type[] = {
+ BUF_BIN,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+ BUF_DPB,
+};
+
static int input_min_count(struct iris_inst *inst)
{
- return MIN_BUFFERS;
+ u32 input_min_count = 0;
+ u32 total_hb_layer = 0;
+
+ if (inst->domain == DECODER) {
+ input_min_count = MIN_BUFFERS;
+ } else if (inst->domain == ENCODER) {
+ total_hb_layer = is_hierb_type_requested(inst) ?
+ inst->cap[ENH_LAYER_COUNT].value + 1 : 0;
+ if (inst->codec == H264 &&
+ !inst->cap[LAYER_ENABLE].value) {
+ total_hb_layer = 0;
+ }
+ input_min_count =
+ hfi_iris3_enc_min_input_buf_count(total_hb_layer);
+ } else {
+ return 0;
+ }
+
+ return input_min_count;
}
static int output_min_count(struct iris_inst *inst)
{
int output_min_count;
+ if (inst->domain != DECODER && inst->domain != ENCODER)
+ return 0;
+
+ if (inst->domain == ENCODER)
+ return MIN_BUFFERS;
+
/* fw_min_count > 0 indicates reconfig event has already arrived */
if (inst->fw_min_count) {
if (is_split_mode_enabled(inst) && inst->codec == VP9)
@@ -83,16 +120,21 @@ static u32 internal_buffer_count(struct iris_inst *inst,
{
u32 count = 0;
- if (buffer_type == BUF_BIN || buffer_type == BUF_LINE ||
- buffer_type == BUF_PERSIST) {
- count = 1;
- } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
- if (inst->codec == H264 || inst->codec == HEVC)
+ if (inst->domain == ENCODER)
+ return 1;
+
+ if (inst->domain == DECODER) {
+ if (buffer_type == BUF_BIN || buffer_type == BUF_LINE ||
+ buffer_type == BUF_PERSIST) {
count = 1;
- else
+ } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
+ if (inst->codec == H264 || inst->codec == HEVC)
+ count = 1;
+ else
+ count = 0;
+ } else {
count = 0;
- } else {
- count = 0;
+ }
}
return count;
@@ -102,6 +144,9 @@ static int dpb_count(struct iris_inst *inst)
{
int count = 0;
+ if (inst->domain == ENCODER)
+ return get_recon_buf_count(inst);
+
if (is_split_mode_enabled(inst)) {
count = inst->fw_min_count ?
inst->fw_min_count : inst->buffers.output.min_count;
@@ -123,6 +168,7 @@ int iris_get_buf_min_count(struct iris_inst *inst,
case BUF_NON_COMV:
case BUF_LINE:
case BUF_PERSIST:
+ case BUF_ARP:
return internal_buffer_count(inst, buffer_type);
case BUF_DPB:
return dpb_count(inst);
@@ -131,7 +177,7 @@ int iris_get_buf_min_count(struct iris_inst *inst,
}
}
-static u32 input_buffer_size(struct iris_inst *inst)
+static u32 dec_input_buffer_size(struct iris_inst *inst)
{
u32 base_res_mbs = NUM_MBS_4k;
u32 frame_size, num_mbs;
@@ -163,7 +209,7 @@ static u32 input_buffer_size(struct iris_inst *inst)
return ALIGN(frame_size, SZ_4K);
}
-static u32 output_buffer_size(struct iris_inst *inst)
+static u32 dec_output_buffer_size(struct iris_inst *inst)
{
struct v4l2_format *f;
u32 size;
@@ -175,17 +221,97 @@ static u32 output_buffer_size(struct iris_inst *inst)
return size;
}
+static u32 enc_input_buffer_size(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ u32 size;
+
+ f = inst->fmt_src;
+
+ size = video_raw_buffer_size(f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
+ f->fmt.pix_mp.height);
+ return size;
+}
+
+static inline
+u32 hfi_buffer_bitstream_enc(u32 frame_width, u32 frame_height,
+ u32 rc_type, bool is_ten_bit)
+{
+ u32 aligned_width, aligned_height, bitstream_size, yuv_size;
+
+ aligned_width = ALIGN(frame_width, 32);
+ aligned_height = ALIGN(frame_height, 32);
+ bitstream_size = aligned_width * aligned_height * 3;
+ yuv_size = (aligned_width * aligned_height * 3) >> 1;
+ if (aligned_width * aligned_height > (4096 * 2176))
+ /* bitstream_size = 0.25 * yuv_size; */
+ bitstream_size = (bitstream_size >> 3);
+ else if (aligned_width * aligned_height > (1280 * 720))
+ /* bitstream_size = 0.5 * yuv_size; */
+ bitstream_size = (bitstream_size >> 2);
+
+ if ((rc_type == HFI_RC_CQ || rc_type == HFI_RC_OFF) &&
+ bitstream_size < yuv_size)
+ bitstream_size = (bitstream_size << 1);
+
+ if (is_ten_bit)
+ bitstream_size = (bitstream_size) + (bitstream_size >> 2);
+
+ return ALIGN(bitstream_size, HFI_ALIGNMENT_4096);
+}
+
+static u32 enc_output_buffer_size(struct iris_inst *inst)
+{
+ u32 hfi_rc_type = HFI_RC_VBR_CFR;
+ enum codec_type codec;
+ int bitrate_mode, frame_rc;
+ bool is_ten_bit = false;
+ struct v4l2_format *f;
+ u32 frame_size;
+
+ f = inst->fmt_dst;
+ codec = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (codec == HEVC)
+ is_ten_bit = true;
+
+ bitrate_mode = inst->cap[BITRATE_MODE].value;
+ frame_rc = inst->cap[FRAME_RC_ENABLE].value;
+ if (!frame_rc)
+ hfi_rc_type = HFI_RC_OFF;
+ else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ)
+ hfi_rc_type = HFI_RC_CQ;
+
+ frame_size = hfi_buffer_bitstream_enc(f->fmt.pix_mp.width,
+ f->fmt.pix_mp.height,
+ hfi_rc_type, is_ten_bit);
+
+ return frame_size;
+}
+
int iris_get_buffer_size(struct iris_inst *inst,
enum iris_buffer_type buffer_type)
{
- switch (buffer_type) {
- case BUF_INPUT:
- return input_buffer_size(inst);
- case BUF_OUTPUT:
- return output_buffer_size(inst);
- default:
- return 0;
+ if (inst->domain == DECODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return dec_input_buffer_size(inst);
+ case BUF_OUTPUT:
+ return dec_output_buffer_size(inst);
+ default:
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return enc_input_buffer_size(inst);
+ case BUF_OUTPUT:
+ return enc_output_buffer_size(inst);
+ default:
+ break;
+ }
}
+
+ return 0;
}
struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst,
@@ -300,14 +426,30 @@ int iris_get_internal_buffers(struct iris_inst *inst,
int ret = 0;
u32 i = 0;
- if (plane == INPUT_MPLANE) {
- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else {
+ return iris_get_internal_buf_info(inst, BUF_DPB);
+ }
+ } else if (inst->domain == ENCODER) {
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, enc_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else {
+ for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, enc_op_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}
- } else {
- return iris_get_internal_buf_info(inst, BUF_DPB);
}
return ret;
@@ -377,10 +519,18 @@ int iris_create_input_internal_buffers(struct iris_inst *inst)
int ret = 0;
u32 i = 0;
- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, enc_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}
return ret;
@@ -388,7 +538,20 @@ int iris_create_input_internal_buffers(struct iris_inst *inst)
int iris_create_output_internal_buffers(struct iris_inst *inst)
{
- return iris_create_internal_buffers(inst, BUF_DPB);
+ int ret = 0;
+ u32 i = 0;
+
+ if (inst->domain == DECODER) {
+ return iris_create_internal_buffers(inst, BUF_DPB);
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, enc_op_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
}
static int set_num_comv(struct iris_inst *inst)
@@ -413,7 +576,7 @@ static int iris_queue_internal_buffers(struct iris_inst *inst,
struct iris_buffers *buffers;
int ret = 0;
- if (buffer_type == BUF_COMV) {
+ if (inst->domain == DECODER && buffer_type == BUF_COMV) {
ret = set_num_comv(inst);
if (ret)
return ret;
@@ -442,10 +605,18 @@ int iris_queue_input_internal_buffers(struct iris_inst *inst)
int ret = 0;
u32 i;
- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, enc_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}
return ret;
@@ -453,7 +624,20 @@ int iris_queue_input_internal_buffers(struct iris_inst *inst)
int iris_queue_output_internal_buffers(struct iris_inst *inst)
{
- return iris_queue_internal_buffers(inst, BUF_DPB);
+ int ret = 0;
+ u32 i = 0;
+
+ if (inst->domain == DECODER) {
+ return iris_queue_internal_buffers(inst, BUF_DPB);
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, enc_op_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
}
int iris_destroy_internal_buffer(struct iris_inst *inst,
@@ -489,16 +673,26 @@ int iris_destroy_internal_buffers(struct iris_inst *inst,
{
struct iris_buffer *buf, *dummy;
struct iris_buffers *buffers;
- const u32 *internal_buf_type;
+ const u32 *internal_buf_type = NULL;
int ret = 0;
- u32 i, len;
-
- if (plane == INPUT_MPLANE) {
- internal_buf_type = dec_ip_int_buf_type;
- len = ARRAY_SIZE(dec_ip_int_buf_type);
- } else {
- internal_buf_type = dec_op_int_buf_type;
- len = ARRAY_SIZE(dec_op_int_buf_type);
+ u32 i, len = 0;
+
+ if (inst->domain == DECODER) {
+ if (plane == INPUT_MPLANE) {
+ internal_buf_type = dec_ip_int_buf_type;
+ len = ARRAY_SIZE(dec_ip_int_buf_type);
+ } else {
+ internal_buf_type = dec_op_int_buf_type;
+ len = ARRAY_SIZE(dec_op_int_buf_type);
+ }
+ } else if (inst->domain == ENCODER) {
+ if (plane == INPUT_MPLANE) {
+ internal_buf_type = enc_ip_int_buf_type;
+ len = ARRAY_SIZE(enc_ip_int_buf_type);
+ } else {
+ internal_buf_type = enc_op_int_buf_type;
+ len = ARRAY_SIZE(enc_op_int_buf_type);
+ }
}
for (i = 0; i < len; i++) {
@@ -552,10 +746,12 @@ static int iris_release_input_internal_buffers(struct iris_inst *inst)
int ret = 0;
u32 i = 0;
- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}
return ret;
@@ -566,7 +762,7 @@ int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
{
int ret;
- if (buffer_type != BUF_PERSIST)
+ if (buffer_type != BUF_ARP && buffer_type != BUF_PERSIST)
return -EINVAL;
ret = iris_get_internal_buf_info(inst, buffer_type);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index af99ac73..559b0dd 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -723,11 +723,20 @@ int set_stage(struct iris_inst *inst,
hfi_id = inst->cap[cap_id].hfi_id;
- inp_f = inst->fmt_src;
- height = inp_f->fmt.pix_mp.height;
- width = inp_f->fmt.pix_mp.width;
- if (res_is_less_than(width, height, 1280, 720))
- work_mode = STAGE_1;
+ if (inst->domain == DECODER) {
+ inp_f = inst->fmt_src;
+ height = inp_f->fmt.pix_mp.height;
+ width = inp_f->fmt.pix_mp.width;
+ if (res_is_less_than(width, height, 1280, 720))
+ work_mode = STAGE_1;
+ } else if (inst->domain == ENCODER) {
+ if (inst->cap[SLICE_MODE].value ==
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES)
+ work_mode = STAGE_1;
+
+ if (!inst->cap[GOP_SIZE].value)
+ work_mode = STAGE_2;
+ }
return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
get_port_info(inst, cap_id),
@@ -743,6 +752,12 @@ int set_pipe(struct iris_inst *inst,
work_route = inst->cap[cap_id].value;
hfi_id = inst->cap[cap_id].hfi_id;
+ if (inst->domain == ENCODER) {
+ if (inst->cap[SLICE_MODE].value ==
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES)
+ work_route = PIPE_1;
+ }
+
return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
get_port_info(inst, cap_id),
HFI_PAYLOAD_U32,
@@ -763,6 +778,30 @@ int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
&hfi_value, sizeof(u32));
}
+int decide_quality_mode(struct iris_inst *inst)
+{
+ u32 fps, mbpf, mbps, max_hq_mbpf, max_hq_mbps;
+ u32 mode = POWER_SAVE_MODE;
+ struct iris_core *core;
+
+ if (inst->domain != ENCODER)
+ return 0;
+
+ mbpf = NUM_MBS_PER_FRAME(inst->crop.height, inst->crop.width);
+ fps = max3(inst->cap[QUEUED_RATE].value >> 16,
+ inst->cap[FRAME_RATE].value >> 16,
+ inst->cap[OPERATING_RATE].value >> 16);
+ mbps = mbpf * fps;
+ core = inst->core;
+ max_hq_mbpf = core->cap[MAX_MBPF_HQ].value;
+ max_hq_mbps = core->cap[MAX_MBPS_HQ].value;
+
+ if (mbpf <= max_hq_mbpf && mbps <= max_hq_mbps)
+ mode = MAX_QUALITY_MODE;
+
+ return mode;
+}
+
int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
{
u32 hfi_id, hfi_val;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 5421d9f..404d98c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -33,6 +33,7 @@ int adjust_v4l2_properties(struct iris_inst *inst);
int ctrls_init(struct iris_inst *inst, bool init);
int set_q16(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int decide_quality_mode(struct iris_inst *inst);
int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_flip(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index c84bb51..a770157 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -44,15 +44,15 @@ u32 get_port_info(struct iris_inst *inst,
if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT &&
inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) {
if (inst->vb2q_dst->streaming)
- return get_hfi_port(INPUT_MPLANE);
+ return get_hfi_port(inst, INPUT_MPLANE);
else
- return get_hfi_port(OUTPUT_MPLANE);
+ return get_hfi_port(inst, OUTPUT_MPLANE);
}
if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT)
- return get_hfi_port(INPUT_MPLANE);
+ return get_hfi_port(inst, INPUT_MPLANE);
else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT)
- return get_hfi_port(OUTPUT_MPLANE);
+ return get_hfi_port(inst, OUTPUT_MPLANE);
else
return HFI_PORT_NONE;
}
@@ -86,9 +86,14 @@ int get_mbpf(struct iris_inst *inst)
int height = 0, width = 0;
struct v4l2_format *inp_f;
- inp_f = inst->fmt_src;
- width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
- height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+ if (inst->domain == DECODER) {
+ inp_f = inst->fmt_src;
+ width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
+ height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+ } else if (inst->domain == ENCODER) {
+ width = inst->crop.width;
+ height = inst->crop.height;
+ }
return NUM_MBS_PER_FRAME(height, width);
}
@@ -100,6 +105,9 @@ inline bool is_linear_colorformat(u32 colorformat)
bool is_split_mode_enabled(struct iris_inst *inst)
{
+ if (inst->domain != DECODER)
+ return false;
+
if (is_linear_colorformat(inst->fmt_dst->fmt.pix_mp.pixelformat))
return true;
@@ -118,6 +126,24 @@ inline bool is_8bit_colorformat(enum colorformat_type colorformat)
colorformat == FMT_NV21;
}
+inline bool is_scaling_enabled(struct iris_inst *inst)
+{
+ return inst->crop.left != inst->compose.left ||
+ inst->crop.top != inst->compose.top ||
+ inst->crop.width != inst->compose.width ||
+ inst->crop.height != inst->compose.height;
+}
+
+inline bool is_hierb_type_requested(struct iris_inst *inst)
+{
+ return (inst->codec == H264 &&
+ inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) ||
+ (inst->codec == HEVC &&
+ inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B);
+}
+
u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec)
{
const struct codec_info *codec_info;
@@ -319,17 +345,24 @@ static int check_resolution_supported(struct iris_inst *inst)
u32 width = 0, height = 0, min_width, min_height,
max_width, max_height;
- width = inst->fmt_src->fmt.pix_mp.width;
- height = inst->fmt_src->fmt.pix_mp.height;
+ if (inst->domain == DECODER) {
+ width = inst->fmt_src->fmt.pix_mp.width;
+ height = inst->fmt_src->fmt.pix_mp.height;
+ } else if (inst->domain == ENCODER) {
+ width = inst->crop.width;
+ height = inst->crop.height;
+ }
min_width = inst->cap[FRAME_WIDTH].min;
max_width = inst->cap[FRAME_WIDTH].max;
min_height = inst->cap[FRAME_HEIGHT].min;
max_height = inst->cap[FRAME_HEIGHT].max;
- if (!(min_width <= width && width <= max_width) ||
- !(min_height <= height && height <= max_height))
- return -EINVAL;
+ if (inst->domain == DECODER || inst->domain == ENCODER) {
+ if (!(min_width <= width && width <= max_width) ||
+ !(min_height <= height && height <= max_height))
+ return -EINVAL;
+ }
return 0;
}
@@ -470,7 +503,7 @@ int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf)
{
int ret;
- if (buf->type == BUF_OUTPUT)
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT)
process_requeued_readonly_buffers(inst, buf);
ret = iris_hfi_queue_buffer(inst, buf);
@@ -653,7 +686,7 @@ static int iris_flush_read_only_buffers(struct iris_inst *inst,
{
struct iris_buffer *ro_buf, *dummy;
- if (type != BUF_OUTPUT)
+ if (inst->domain != DECODER || type != BUF_OUTPUT)
return 0;
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
@@ -899,6 +932,114 @@ int codec_change(struct iris_inst *inst, u32 v4l2_codec)
return ret;
}
+int process_streamon_input(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+ int ret;
+
+ iris_scale_power(inst);
+
+ ret = iris_hfi_start(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0);
+ if (ret)
+ return ret;
+ }
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC ||
+ inst->sub_state & IRIS_INST_SUB_DRAIN) {
+ if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) {
+ ret = iris_hfi_pause(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
+ ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, 0, set_sub_state);
+
+ return ret;
+}
+
+int process_streamon_output(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ bool drain_pending = false;
+ int ret;
+
+ iris_scale_power(inst);
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST)
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->domain == DECODER && inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_alloc_and_queue_input_int_bufs(inst);
+ if (ret)
+ return ret;
+ ret = set_stage(inst, STAGE);
+ if (ret)
+ return ret;
+ ret = set_pipe(inst, PIPE);
+ if (ret)
+ return ret;
+ }
+
+ drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
+
+ if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) {
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
+ ret = iris_hfi_start(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE)
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+
+ ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
+ return ret;
+}
+
+int vb2_buffer_to_driver(struct vb2_buffer *vb2, struct iris_buffer *buf)
+{
+ struct vb2_v4l2_buffer *vbuf;
+
+ if (!vb2 || !buf)
+ return -EINVAL;
+
+ vbuf = to_vb2_v4l2_buffer(vb2);
+
+ buf->fd = vb2->planes[0].m.fd;
+ buf->data_offset = vb2->planes[0].data_offset;
+ buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset;
+ buf->buffer_size = vb2->planes[0].length;
+ buf->timestamp = vb2->timestamp;
+ buf->flags = vbuf->flags;
+ buf->attr = 0;
+
+ return 0;
+}
+
int iris_pm_get(struct iris_core *core)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 39cec8c..bf8448a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -31,7 +31,9 @@ int close_session(struct iris_inst *inst);
bool is_linear_colorformat(u32 colorformat);
bool is_10bit_colorformat(enum colorformat_type colorformat);
bool is_8bit_colorformat(enum colorformat_type colorformat);
+bool is_scaling_enabled(struct iris_inst *inst);
bool is_split_mode_enabled(struct iris_inst *inst);
+bool is_hierb_type_requested(struct iris_inst *inst);
int signal_session_msg_receipt(struct iris_inst *inst,
enum signal_session_response cmd);
struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
@@ -54,6 +56,10 @@ void iris_destroy_buffers(struct iris_inst *inst);
int session_streamoff(struct iris_inst *inst, u32 plane);
int process_resume(struct iris_inst *inst);
int codec_change(struct iris_inst *inst, u32 v4l2_codec);
+int process_streamon_input(struct iris_inst *inst);
+int process_streamon_output(struct iris_inst *inst);
+int vb2_buffer_to_driver(struct vb2_buffer *vb2,
+ struct iris_buffer *buf);
int iris_pm_get(struct iris_core *core);
int iris_pm_put(struct iris_core *core, bool autosuspend);
int iris_pm_get_put(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 00e598d..4a54293 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -356,7 +356,7 @@ int iris_hfi_session_set_default_header(struct iris_inst *inst)
ret = hfi_packet_session_property(inst,
HFI_PROP_DEC_DEFAULT_HEADER,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&default_header,
sizeof(u32));
@@ -394,7 +394,7 @@ int iris_hfi_start(struct iris_inst *inst, u32 plane)
HFI_CMD_START,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -434,7 +434,7 @@ int iris_hfi_stop(struct iris_inst *inst, u32 plane)
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED |
HFI_HOST_FLAGS_NON_DISCARDABLE),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -472,7 +472,7 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
cmd,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
payload_type,
payload,
@@ -511,7 +511,7 @@ int iris_hfi_pause(struct iris_inst *inst, u32 plane)
HFI_CMD_PAUSE,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -550,7 +550,7 @@ int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 payload)
HFI_CMD_RESUME,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_U32,
&payload,
@@ -590,7 +590,7 @@ int iris_hfi_drain(struct iris_inst *inst, u32 plane)
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED |
HFI_HOST_FLAGS_NON_DISCARDABLE),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -739,14 +739,14 @@ int iris_hfi_queue_buffer(struct iris_inst *inst,
goto unlock;
}
- ret = get_hfi_buffer(buffer, &hfi_buffer);
+ ret = get_hfi_buffer(inst, buffer, &hfi_buffer);
if (ret)
goto unlock;
ret = hfi_packet_session_command(inst,
HFI_CMD_BUFFER,
HFI_HOST_FLAGS_INTR_REQUIRED,
- get_hfi_port_from_buffer_type(buffer->type),
+ get_hfi_port_from_buffer_type(inst, buffer->type),
inst->session_id,
HFI_PAYLOAD_STRUCTURE,
&hfi_buffer,
@@ -780,7 +780,7 @@ int iris_hfi_release_buffer(struct iris_inst *inst,
goto unlock;
}
- ret = get_hfi_buffer(buffer, &hfi_buffer);
+ ret = get_hfi_buffer(inst, buffer, &hfi_buffer);
if (ret)
goto unlock;
@@ -790,7 +790,7 @@ int iris_hfi_release_buffer(struct iris_inst *inst,
HFI_CMD_BUFFER,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port_from_buffer_type(buffer->type),
+ get_hfi_port_from_buffer_type(inst, buffer->type),
inst->session_id,
HFI_PAYLOAD_STRUCTURE,
&hfi_buffer,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index dd27fa4..e9f3749 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -3,64 +3,108 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/
+#include "iris_common.h"
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "hfi_defines.h"
-u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type)
+u32 get_hfi_port_from_buffer_type(struct iris_inst *inst, enum iris_buffer_type buffer_type)
{
u32 hfi_port = HFI_PORT_NONE;
- switch (buffer_type) {
- case BUF_INPUT:
- case BUF_BIN:
- case BUF_COMV:
- case BUF_NON_COMV:
- case BUF_LINE:
- hfi_port = HFI_PORT_BITSTREAM;
- break;
- case BUF_OUTPUT:
- case BUF_DPB:
- hfi_port = HFI_PORT_RAW;
- break;
- case BUF_PERSIST:
- hfi_port = HFI_PORT_NONE;
- break;
- default:
- break;
+ if (inst->domain == DECODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case BUF_OUTPUT:
+ case BUF_DPB:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case BUF_PERSIST:
+ hfi_port = HFI_PORT_NONE;
+ break;
+ default:
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ case BUF_VPSS:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case BUF_OUTPUT:
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ case BUF_DPB:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case BUF_ARP:
+ hfi_port = HFI_PORT_NONE;
+ break;
+ default:
+ break;
+ }
}
return hfi_port;
}
-u32 get_hfi_port(u32 plane)
+u32 get_hfi_port(struct iris_inst *inst, u32 plane)
{
u32 hfi_port = HFI_PORT_NONE;
- switch (plane) {
- case INPUT_MPLANE:
- hfi_port = HFI_PORT_BITSTREAM;
- break;
- case OUTPUT_MPLANE:
- hfi_port = HFI_PORT_RAW;
- break;
- default:
- break;
+ if (inst->domain == DECODER) {
+ switch (plane) {
+ case INPUT_MPLANE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case OUTPUT_MPLANE:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ default:
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (plane) {
+ case INPUT_MPLANE:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case OUTPUT_MPLANE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ default:
+ break;
+ }
}
return hfi_port;
}
-static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
+static u32 hfi_buf_type_from_driver(enum domain_type domain, enum iris_buffer_type buffer_type)
{
switch (buffer_type) {
case BUF_INPUT:
- return HFI_BUFFER_BITSTREAM;
+ if (domain == DECODER)
+ return HFI_BUFFER_BITSTREAM;
+ else
+ return HFI_BUFFER_RAW;
case BUF_OUTPUT:
- return HFI_BUFFER_RAW;
+ if (domain == DECODER)
+ return HFI_BUFFER_RAW;
+ else
+ return HFI_BUFFER_BITSTREAM;
case BUF_BIN:
return HFI_BUFFER_BIN;
+ case BUF_ARP:
+ return HFI_BUFFER_ARP;
case BUF_COMV:
return HFI_BUFFER_COMV;
case BUF_NON_COMV:
@@ -76,13 +120,19 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}
-u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type)
+u32 hfi_buf_type_to_driver(enum domain_type domain, enum hfi_buffer_type buf_type)
{
switch (buf_type) {
case HFI_BUFFER_BITSTREAM:
- return BUF_INPUT;
+ if (domain == DECODER)
+ return BUF_INPUT;
+ else
+ return BUF_OUTPUT;
case HFI_BUFFER_RAW:
- return BUF_OUTPUT;
+ if (domain == DECODER)
+ return BUF_OUTPUT;
+ else
+ return BUF_INPUT;
case HFI_BUFFER_BIN:
return BUF_BIN;
case HFI_BUFFER_ARP:
@@ -108,9 +158,15 @@ u32 get_hfi_codec(struct iris_inst *inst)
{
switch (inst->codec) {
case H264:
- return HFI_CODEC_DECODE_AVC;
+ if (inst->domain == ENCODER)
+ return HFI_CODEC_ENCODE_AVC;
+ else
+ return HFI_CODEC_DECODE_AVC;
case HEVC:
- return HFI_CODEC_DECODE_HEVC;
+ if (inst->domain == ENCODER)
+ return HFI_CODEC_ENCODE_HEVC;
+ else
+ return HFI_CODEC_DECODE_HEVC;
case VP9:
return HFI_CODEC_DECODE_VP9;
default:
@@ -337,10 +393,11 @@ u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients)
return coefficients;
}
-int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
+int get_hfi_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer, struct hfi_buffer *buf)
{
- memset(buf, 0, sizeof(*buf));
- buf->type = hfi_buf_type_from_driver(buffer->type);
+ memset(buf, 0, sizeof(struct hfi_buffer));
+ buf->type = hfi_buf_type_from_driver(inst->domain, buffer->type);
buf->index = buffer->index;
buf->base_address = buffer->device_addr;
buf->addr_offset = 0;
@@ -350,7 +407,7 @@ int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
* buffer size otherwise it will truncate or ignore the data after 256
* aligned size which may lead to error concealment
*/
- if (buffer->type == BUF_INPUT)
+ if (inst->domain == DECODER && buffer->type == BUF_INPUT)
buf->buffer_size = ALIGN(buffer->buffer_size, 256);
buf->data_offset = buffer->data_offset;
buf->data_size = buffer->data_size;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 82148b7..734a070 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -80,8 +80,8 @@ enum hfi_packet_port_type {
HFI_PORT_RAW = 0x00000002,
};
-u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
-u32 get_hfi_port(u32 plane);
+u32 get_hfi_port_from_buffer_type(struct iris_inst *inst, enum iris_buffer_type buffer_type);
+u32 get_hfi_port(struct iris_inst *inst, u32 plane);
u32 get_hfi_colorformat(u32 colorformat);
u32 get_hfi_codec(struct iris_inst *inst);
u32 get_hfi_color_primaries(u32 primaries);
@@ -90,8 +90,9 @@ u32 get_hfi_matrix_coefficients(u32 coefficients);
u32 get_v4l2_color_primaries(u32 hfi_primaries);
u32 get_v4l2_transfer_char(u32 hfi_characterstics);
u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients);
-u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type);
-int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
+u32 hfi_buf_type_to_driver(enum domain_type domain, enum hfi_buffer_type buf_type);
+int get_hfi_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer, struct hfi_buffer *buf);
int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 1b667a5..08bdb8f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -71,7 +71,8 @@ static bool is_valid_hfi_buffer_type(u32 buffer_type)
static bool is_valid_hfi_port(u32 port, u32 buffer_type)
{
if (port == HFI_PORT_NONE &&
- buffer_type != HFI_BUFFER_PERSIST)
+ buffer_type != HFI_BUFFER_ARP &&
+ buffer_type != HFI_BUFFER_PERSIST)
return false;
if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW)
@@ -103,8 +104,9 @@ static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags)
if (inst->hfi_frame_info.overflow)
driver_flags |= BUF_FLAG_ERROR;
- if (hfi_flags & HFI_BUF_FW_FLAG_LAST ||
- hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST)
+ if ((inst->domain == ENCODER && (hfi_flags & HFI_BUF_FW_FLAG_LAST)) ||
+ (inst->domain == DECODER && (hfi_flags & HFI_BUF_FW_FLAG_LAST ||
+ hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST)))
driver_flags |= BUF_FLAG_LAST;
return driver_flags;
@@ -309,9 +311,12 @@ static int handle_session_close(struct iris_inst *inst,
static int handle_read_only_buffer(struct iris_inst *inst,
struct iris_buffer *buf)
{
- struct iris_buffer *ro_buf, *iter;
+ struct iris_buffer *ro_buf, *iter = NULL;
bool found = false;
+ if (inst->domain != DECODER && inst->domain != ENCODER)
+ return 0;
+
list_for_each_entry(iter, &inst->buffers.read_only.list, list) {
if (iter->device_addr == buf->device_addr) {
found = true;
@@ -344,6 +349,9 @@ static int handle_non_read_only_buffer(struct iris_inst *inst,
{
struct iris_buffer *ro_buf;
+ if (inst->domain != DECODER && inst->domain != ENCODER)
+ return 0;
+
list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
if (ro_buf->device_addr == buffer->base_address) {
ro_buf->attr &= ~BUF_ATTR_READ_ONLY;
@@ -414,9 +422,9 @@ static int handle_input_buffer(struct iris_inst *inst,
static int handle_output_buffer(struct iris_inst *inst,
struct hfi_buffer *hfi_buffer)
{
- struct iris_buffers *buffers;
struct iris_buffer *buf, *iter;
- bool found;
+ struct iris_buffers *buffers;
+ bool found, fatal = false;
int ret = 0;
if (hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) {
@@ -425,17 +433,19 @@ static int handle_output_buffer(struct iris_inst *inst,
return ret;
}
- if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
- return handle_release_output_buffer(inst, hfi_buffer);
+ if (inst->domain == DECODER) {
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
+ return handle_release_output_buffer(inst, hfi_buffer);
- if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) {
- ret = iris_inst_sub_state_change_drc_last(inst);
- if (ret)
- return ret;
- }
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) {
+ ret = iris_inst_sub_state_change_drc_last(inst);
+ if (ret)
+ return ret;
+ }
- if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
- ret = handle_non_read_only_buffer(inst, hfi_buffer);
+ if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
+ ret = handle_non_read_only_buffer(inst, hfi_buffer);
+ }
buffers = iris_get_buffer_list(inst, BUF_OUTPUT);
if (!buffers)
@@ -446,9 +456,12 @@ static int handle_output_buffer(struct iris_inst *inst,
if (!(iter->attr & BUF_ATTR_QUEUED))
continue;
- found = (iter->index == hfi_buffer->index &&
- iter->device_addr == hfi_buffer->base_address &&
- iter->data_offset == hfi_buffer->data_offset);
+ if (inst->domain == DECODER)
+ found = (iter->index == hfi_buffer->index &&
+ iter->device_addr == hfi_buffer->base_address &&
+ iter->data_offset == hfi_buffer->data_offset);
+ else
+ found = iter->index == hfi_buffer->index;
if (found) {
buf = iter;
@@ -465,14 +478,27 @@ static int handle_output_buffer(struct iris_inst *inst,
buf->attr &= ~BUF_ATTR_QUEUED;
buf->attr |= BUF_ATTR_DEQUEUED;
- if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)
- iris_inst_change_state(inst, IRIS_INST_ERROR);
+ if (inst->domain == ENCODER) {
+ if (inst->hfi_frame_info.data_corrupt)
+ fatal = true;
+ if (inst->hfi_frame_info.overflow) {
+ if (!hfi_buffer->data_size && inst->hfi_rc_type == HFI_RC_CBR_CFR)
+ fatal = true;
+ }
+ if (fatal)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ }
+
+ if (inst->domain == DECODER) {
+ if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
- if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) {
- buf->attr |= BUF_ATTR_READ_ONLY;
- ret = handle_read_only_buffer(inst, buf);
- } else {
- buf->attr &= ~BUF_ATTR_READ_ONLY;
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) {
+ buf->attr |= BUF_ATTR_READ_ONLY;
+ ret = handle_read_only_buffer(inst, buf);
+ } else {
+ buf->attr &= ~BUF_ATTR_READ_ONLY;
+ }
}
buf->flags = get_driver_buffer_flags(inst, hfi_buffer->flags);
@@ -521,7 +547,7 @@ static int handle_release_internal_buffer(struct iris_inst *inst,
int ret = 0;
bool found;
- buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(buffer->type));
+ buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(inst->domain, buffer->type));
if (!buffers)
return -EINVAL;
@@ -550,12 +576,22 @@ static int handle_session_stop(struct iris_inst *inst,
int ret = 0;
enum signal_session_response signal_type = -1;
- if (pkt->port == HFI_PORT_RAW) {
- signal_type = SIGNAL_CMD_STOP_OUTPUT;
- ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
- } else if (pkt->port == HFI_PORT_BITSTREAM) {
- signal_type = SIGNAL_CMD_STOP_INPUT;
- ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ if (inst->domain == DECODER) {
+ if (pkt->port == HFI_PORT_RAW) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ }
+ } else if (inst->domain == ENCODER) {
+ if (pkt->port == HFI_PORT_RAW) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
+ }
}
if (signal_type != -1)
@@ -571,7 +607,7 @@ static int handle_session_buffer(struct iris_inst *inst,
u32 hfi_handle_size = 0;
int i, ret = 0;
const struct iris_hfi_buffer_handle *hfi_handle_arr = NULL;
- static const struct iris_hfi_buffer_handle input_hfi_handle[] = {
+ static const struct iris_hfi_buffer_handle dec_input_hfi_handle[] = {
{HFI_BUFFER_BITSTREAM, handle_input_buffer },
{HFI_BUFFER_BIN, handle_release_internal_buffer },
{HFI_BUFFER_COMV, handle_release_internal_buffer },
@@ -579,10 +615,24 @@ static int handle_session_buffer(struct iris_inst *inst,
{HFI_BUFFER_LINE, handle_release_internal_buffer },
{HFI_BUFFER_PERSIST, handle_release_internal_buffer },
};
- static const struct iris_hfi_buffer_handle output_hfi_handle[] = {
+ static const struct iris_hfi_buffer_handle dec_output_hfi_handle[] = {
{HFI_BUFFER_RAW, handle_output_buffer },
{HFI_BUFFER_DPB, handle_release_internal_buffer },
};
+ static const struct iris_hfi_buffer_handle enc_input_hfi_handle[] = {
+ {HFI_BUFFER_RAW, handle_input_buffer },
+ {HFI_BUFFER_VPSS, handle_release_internal_buffer },
+ };
+ static const struct iris_hfi_buffer_handle enc_output_hfi_handle[] = {
+ {HFI_BUFFER_BITSTREAM, handle_output_buffer },
+ {HFI_BUFFER_BIN, handle_release_internal_buffer },
+ {HFI_BUFFER_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_NON_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_LINE, handle_release_internal_buffer },
+ {HFI_BUFFER_ARP, handle_release_internal_buffer },
+ {HFI_BUFFER_DPB, handle_release_internal_buffer },
+ };
+
if (pkt->payload_info == HFI_PAYLOAD_NONE)
return 0;
@@ -599,12 +649,22 @@ static int handle_session_buffer(struct iris_inst *inst,
if (!is_valid_hfi_port(pkt->port, buffer->type))
return 0;
- if (pkt->port == HFI_PORT_BITSTREAM) {
- hfi_handle_size = ARRAY_SIZE(input_hfi_handle);
- hfi_handle_arr = input_hfi_handle;
- } else if (pkt->port == HFI_PORT_RAW) {
- hfi_handle_size = ARRAY_SIZE(output_hfi_handle);
- hfi_handle_arr = output_hfi_handle;
+ if (inst->domain == DECODER) {
+ if (pkt->port == HFI_PORT_BITSTREAM) {
+ hfi_handle_size = ARRAY_SIZE(dec_input_hfi_handle);
+ hfi_handle_arr = dec_input_hfi_handle;
+ } else if (pkt->port == HFI_PORT_RAW) {
+ hfi_handle_size = ARRAY_SIZE(dec_output_hfi_handle);
+ hfi_handle_arr = dec_output_hfi_handle;
+ }
+ } else if (inst->domain == ENCODER) {
+ if (pkt->port == HFI_PORT_RAW) {
+ hfi_handle_size = ARRAY_SIZE(enc_input_hfi_handle);
+ hfi_handle_arr = enc_input_hfi_handle;
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ hfi_handle_size = ARRAY_SIZE(enc_output_hfi_handle);
+ hfi_handle_arr = enc_output_hfi_handle;
+ }
}
if (!hfi_handle_arr || !hfi_handle_size)
@@ -693,6 +753,9 @@ static int handle_dpb_list_property(struct iris_inst *inst,
u8 *payload_start;
u32 payload_size;
+ if (inst->domain != DECODER)
+ return -EINVAL;
+
payload_size = pkt->size - sizeof(*pkt);
payload_start = (u8 *)((u8 *)pkt + sizeof(*pkt));
memset(inst->dpb_list_payload, 0, MAX_DPB_LIST_ARRAY_SIZE);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index c0878f1..1a434ff 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -13,6 +13,7 @@
#include "iris_power.h"
#include "iris_vb2.h"
#include "iris_vdec.h"
+#include "iris_venc.h"
int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
@@ -58,7 +59,8 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
return ret;
}
- if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ if ((inst->domain == DECODER && q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) ||
+ (inst->domain == ENCODER && q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) {
ret = adjust_v4l2_properties(inst);
if (ret)
return ret;
@@ -112,6 +114,11 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
goto error;
}
+ if (inst->domain != DECODER && inst->domain != ENCODER) {
+ ret = -EINVAL;
+ goto error;
+ }
+
ret = iris_pm_get(inst->core);
if (ret)
goto error;
@@ -122,21 +129,34 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
if (ret)
goto err_pm_get;
- ret = iris_hfi_session_set_default_header(inst);
- if (ret)
- goto err_pm_get;
-
- ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
- if (ret)
- goto err_pm_get;
+ if (inst->domain == ENCODER) {
+ ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_ARP);
+ if (ret)
+ goto err_pm_get;
+ } else if (inst->domain == DECODER) {
+ ret = iris_hfi_session_set_default_header(inst);
+ if (ret)
+ goto err_pm_get;
+
+ ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
+ if (ret)
+ goto err_pm_get;
+ }
}
iris_scale_power(inst);
- if (q->type == INPUT_MPLANE)
- ret = vdec_streamon_input(inst);
- else if (q->type == OUTPUT_MPLANE)
- ret = vdec_streamon_output(inst);
+ if (q->type == INPUT_MPLANE) {
+ if (inst->domain == DECODER)
+ ret = vdec_streamon_input(inst);
+ else if (inst->domain == ENCODER)
+ ret = venc_streamon_input(inst);
+ } else if (q->type == OUTPUT_MPLANE) {
+ if (inst->domain == DECODER)
+ ret = vdec_streamon_output(inst);
+ else if (inst->domain == ENCODER)
+ ret = venc_streamon_output(inst);
+ }
if (ret)
goto err_pm_get;
@@ -179,6 +199,11 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)
if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE)
goto error;
+ if (inst->domain != DECODER && inst->domain != ENCODER) {
+ ret = -EINVAL;
+ goto error;
+ }
+
ret = iris_pm_get_put(inst->core);
if (ret)
goto error;
@@ -225,7 +250,10 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2)
if (ret)
goto exit;
- ret = vdec_qbuf(inst, vb2);
+ if (inst->domain == DECODER)
+ ret = vdec_qbuf(inst, vb2);
+ else if (inst->domain == ENCODER)
+ ret = venc_qbuf(inst, vb2);
exit:
if (ret) {
@@ -270,7 +298,7 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
buf->inst = inst;
buf->dmabuf = dbuf;
- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
@@ -307,7 +335,7 @@ int iris_vb2_map_dmabuf(void *buf_priv)
return -EINVAL;
}
- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
@@ -356,7 +384,7 @@ void iris_vb2_unmap_dmabuf(void *buf_priv)
return;
}
- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
@@ -393,7 +421,7 @@ void iris_vb2_detach_dmabuf(void *buf_priv)
buf->sg_table = NULL;
}
- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 300d0e9..b2d4739 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -362,7 +362,7 @@ static int vdec_set_bitstream_resolution(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_BITSTREAM_RESOLUTION,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&resolution,
sizeof(u32));
@@ -388,7 +388,7 @@ static int vdec_set_crop_offsets(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_CROP_OFFSETS,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_64_PACKED,
&payload,
sizeof(u64));
@@ -409,7 +409,7 @@ static int vdec_set_bit_depth(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&bitdepth,
sizeof(u32));
@@ -426,7 +426,7 @@ static int vdec_set_coded_frames(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_CODED_FRAMES,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&coded_frames,
sizeof(u32));
@@ -442,7 +442,7 @@ static int vdec_set_min_output_count(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&min_output,
sizeof(u32));
@@ -457,7 +457,7 @@ static int vdec_set_picture_order_count(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_PIC_ORDER_CNT_TYPE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&poc,
sizeof(u32));
@@ -509,7 +509,7 @@ static int vdec_set_colorspace(struct iris_inst *inst)
ret = iris_hfi_set_property(inst,
HFI_PROP_SIGNAL_COLOR_INFO,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_32_PACKED,
&color_info,
sizeof(u32));
@@ -527,7 +527,7 @@ static int vdec_set_profile(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_PROFILE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32_ENUM,
&profile,
sizeof(u32));
@@ -543,7 +543,7 @@ static int vdec_set_level(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_LEVEL,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32_ENUM,
&level,
sizeof(u32));
@@ -559,7 +559,7 @@ static int vdec_set_tier(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_TIER,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32_ENUM,
&tier,
sizeof(u32));
@@ -811,7 +811,7 @@ static int vdec_set_colorformat(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_COLOR_FORMAT,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
HFI_PAYLOAD_U32,
&hfi_colorformat,
sizeof(u32));
@@ -839,7 +839,7 @@ static int vdec_set_linear_stride_scanline(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_LINEAR_STRIDE_SCANLINE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
HFI_PAYLOAD_U64,
&payload,
sizeof(u64));
@@ -887,7 +887,7 @@ static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_UBWC_STRIDE_SCANLINE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
HFI_PAYLOAD_U32_ARRAY,
&payload[0],
sizeof(u32) * 4);
@@ -1020,7 +1020,7 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst)
ret = iris_hfi_set_property(inst,
prop_type,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
payload_type,
&payload,
payload_size);
@@ -1032,42 +1032,6 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst)
return ret;
}
-static int process_streamon_input(struct iris_inst *inst)
-{
- enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
- int ret;
-
- iris_scale_power(inst);
-
- ret = iris_hfi_start(inst, INPUT_MPLANE);
- if (ret)
- return ret;
-
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0);
- if (ret)
- return ret;
- }
-
- if (inst->sub_state & IRIS_INST_SUB_DRC ||
- inst->sub_state & IRIS_INST_SUB_DRAIN) {
- if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) {
- ret = iris_hfi_pause(inst, INPUT_MPLANE);
- if (ret)
- return ret;
- set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
- }
- }
-
- ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
- if (ret)
- return ret;
-
- ret = iris_inst_change_sub_state(inst, 0, set_sub_state);
-
- return ret;
-}
-
int vdec_streamon_input(struct iris_inst *inst)
{
int ret;
@@ -1114,58 +1078,6 @@ int vdec_streamon_input(struct iris_inst *inst)
return ret;
}
-static int process_streamon_output(struct iris_inst *inst)
-{
- enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
- bool drain_pending = false;
- int ret;
-
- iris_scale_power(inst);
-
- if (inst->sub_state & IRIS_INST_SUB_DRC &&
- inst->sub_state & IRIS_INST_SUB_DRC_LAST)
- clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
-
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_alloc_and_queue_input_int_bufs(inst);
- if (ret)
- return ret;
- ret = set_stage(inst, STAGE);
- if (ret)
- return ret;
- ret = set_pipe(inst, PIPE);
- if (ret)
- return ret;
- }
-
- drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN &&
- inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
-
- if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) {
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
- }
- }
-
- ret = iris_hfi_start(inst, OUTPUT_MPLANE);
- if (ret)
- return ret;
-
- if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE)
- clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
-
- ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
- if (ret)
- return ret;
-
- ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
-
- return ret;
-}
-
int vdec_streamon_output(struct iris_inst *inst)
{
int ret;
@@ -1220,27 +1132,6 @@ int vdec_streamon_output(struct iris_inst *inst)
return ret;
}
-static int vb2_buffer_to_driver(struct vb2_buffer *vb2,
- struct iris_buffer *buf)
-{
- struct vb2_v4l2_buffer *vbuf;
-
- if (!vb2 || !buf)
- return -EINVAL;
-
- vbuf = to_vb2_v4l2_buffer(vb2);
-
- buf->fd = vb2->planes[0].m.fd;
- buf->data_offset = vb2->planes[0].data_offset;
- buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset;
- buf->buffer_size = vb2->planes[0].length;
- buf->timestamp = vb2->timestamp;
- buf->flags = vbuf->flags;
- buf->attr = 0;
-
- return 0;
-}
-
int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
{
struct iris_buffer *buf = NULL;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
index 802db40..28c6b20 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
@@ -13,6 +13,23 @@
#include "iris_power.h"
#include "iris_venc.h"
+#define SCALE_FACTOR 8
+#define UNSPECIFIED_COLOR_FORMAT 5
+
+static const u32 enc_input_properties[] = {
+ HFI_PROP_NO_OUTPUT,
+};
+
+static const u32 enc_output_properties[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_BUFFER_MARK,
+};
+
+struct venc_prop_type_handle {
+ u32 type;
+ int (*handle)(struct iris_inst *inst);
+};
+
int venc_inst_init(struct iris_inst *inst)
{
struct v4l2_format *f;
@@ -523,3 +540,409 @@ int venc_stop_cmd(struct iris_inst *inst)
return ret;
}
+
+int venc_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
+{
+ struct iris_buffer *buf = NULL;
+ int ret;
+
+ buf = get_driver_buf(inst, vb2->type, vb2->index);
+ if (!buf)
+ return -EINVAL;
+
+ ret = vb2_buffer_to_driver(vb2, buf);
+ if (ret)
+ return ret;
+
+ if (!allow_qbuf(inst, vb2->type)) {
+ buf->attr |= BUF_ATTR_DEFERRED;
+ return 0;
+ }
+
+ iris_scale_power(inst);
+
+ return queue_buffer(inst, buf);
+}
+
+static int check_scaling_supported(struct iris_inst *inst)
+{
+ u32 iwidth, owidth, iheight, oheight;
+
+ if (!(inst->crop.left != inst->compose.left ||
+ inst->crop.top != inst->compose.top ||
+ inst->crop.width != inst->compose.width ||
+ inst->crop.height != inst->compose.height))
+ return 0;
+
+ iwidth = inst->crop.width;
+ iheight = inst->crop.height;
+ owidth = inst->compose.width;
+ oheight = inst->compose.height;
+
+ if (owidth > iwidth || oheight > iheight)
+ return -EINVAL;
+
+ if (iwidth > owidth * SCALE_FACTOR || iheight > oheight * SCALE_FACTOR)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int venc_set_colorformat(struct iris_inst *inst)
+{
+ u32 hfi_colorformat;
+ u32 pixelformat;
+
+ pixelformat = inst->fmt_src->fmt.pix_mp.pixelformat;
+ hfi_colorformat = get_hfi_colorformat(pixelformat);
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_COLOR_FORMAT,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_colorformat,
+ sizeof(u32));
+}
+
+static int venc_set_stride_scanline(struct iris_inst *inst)
+{
+ u32 color_format, stride_y, scanline_y;
+ u32 stride_uv = 0, scanline_uv = 0;
+ u32 payload[2];
+
+ color_format = inst->cap[PIX_FMTS].value;
+ if (!is_linear_colorformat(color_format))
+ return 0;
+
+ stride_y = color_format == FMT_TP10C ?
+ ALIGN(inst->fmt_src->fmt.pix_mp.width, 192) :
+ ALIGN(inst->fmt_src->fmt.pix_mp.width, 128);
+ scanline_y = color_format == FMT_TP10C ?
+ ALIGN(inst->fmt_src->fmt.pix_mp.height, 16) :
+ ALIGN(inst->fmt_src->fmt.pix_mp.height, 32);
+
+ if (color_format == FMT_NV12 ||
+ color_format == FMT_NV21) {
+ stride_uv = stride_y;
+ scanline_uv = scanline_y / 2;
+ }
+
+ payload[0] = stride_y << 16 | scanline_y;
+ payload[1] = stride_uv << 16 | scanline_uv;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LINEAR_STRIDE_SCANLINE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &payload,
+ sizeof(u64));
+}
+
+static int venc_set_raw_resolution(struct iris_inst *inst)
+{
+ u32 resolution;
+
+ resolution = (inst->fmt_src->fmt.pix_mp.width << 16) |
+ inst->fmt_src->fmt.pix_mp.height;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_RAW_RESOLUTION,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &resolution,
+ sizeof(u32));
+}
+
+static int venc_set_bitstream_resolution(struct iris_inst *inst)
+{
+ u32 resolution;
+
+ resolution = (inst->fmt_dst->fmt.pix_mp.width << 16) |
+ inst->fmt_dst->fmt.pix_mp.height;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, OUTPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &resolution,
+ sizeof(u32));
+}
+
+static int venc_set_inp_crop_offsets(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ u32 crop[2] = {0};
+ u32 width, height;
+
+ left_offset = inst->crop.left;
+ top_offset = inst->crop.top;
+ width = inst->crop.width;
+ height = inst->crop.height;
+
+ right_offset = (inst->fmt_src->fmt.pix_mp.width - width);
+ bottom_offset = (inst->fmt_src->fmt.pix_mp.height - height);
+
+ crop[0] = left_offset << 16 | top_offset;
+ crop[1] = right_offset << 16 | bottom_offset;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &crop,
+ sizeof(u64));
+}
+
+static int venc_set_out_crop_offsets(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ u32 crop[2] = {0};
+ u32 width, height;
+
+ left_offset = inst->compose.left;
+ top_offset = inst->compose.top;
+ width = inst->compose.width;
+ height = inst->compose.height;
+ if (inst->cap[ROTATION].value == 90 || inst->cap[ROTATION].value == 270) {
+ width = inst->compose.height;
+ height = inst->compose.width;
+ }
+
+ right_offset = (inst->fmt_dst->fmt.pix_mp.width - width);
+ bottom_offset = (inst->fmt_dst->fmt.pix_mp.height - height);
+
+ crop[0] = left_offset << 16 | top_offset;
+ crop[1] = right_offset << 16 | bottom_offset;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, OUTPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &crop,
+ sizeof(u64));
+}
+
+static int venc_set_colorspace(struct iris_inst *inst)
+{
+ u32 video_signal_type_present_flag = 0, payload = 0;
+ u32 matrix_coeff = HFI_MATRIX_COEFF_RESERVED;
+ u32 video_format = UNSPECIFIED_COLOR_FORMAT;
+ struct v4l2_pix_format_mplane *pixmp = NULL;
+ u32 transfer_char = HFI_TRANSFER_RESERVED;
+ u32 colour_description_present_flag = 0;
+ u32 primaries = HFI_PRIMARIES_RESERVED;
+ u32 full_range = 0;
+
+ pixmp = &inst->fmt_src->fmt.pix_mp;
+ if (pixmp->colorspace != V4L2_COLORSPACE_DEFAULT ||
+ pixmp->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT ||
+ pixmp->xfer_func != V4L2_XFER_FUNC_DEFAULT) {
+ colour_description_present_flag = 1;
+ video_signal_type_present_flag = 1;
+ primaries = get_hfi_color_primaries(pixmp->colorspace);
+ matrix_coeff = get_hfi_matrix_coefficients(pixmp->ycbcr_enc);
+ transfer_char = get_hfi_transfer_char(pixmp->xfer_func);
+ }
+
+ if (pixmp->quantization != V4L2_QUANTIZATION_DEFAULT) {
+ video_signal_type_present_flag = 1;
+ full_range = pixmp->quantization ==
+ V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0;
+ }
+
+ payload = (matrix_coeff & 0xFF) |
+ ((transfer_char << 8) & 0xFF00) |
+ ((primaries << 16) & 0xFF0000) |
+ ((colour_description_present_flag << 24) & 0x1000000) |
+ ((full_range << 25) & 0x2000000) |
+ ((video_format << 26) & 0x1C000000) |
+ ((video_signal_type_present_flag << 29) & 0x20000000);
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &payload,
+ sizeof(u32));
+}
+
+static int venc_set_quality_mode(struct iris_inst *inst)
+{
+ u32 mode;
+
+ mode = decide_quality_mode(inst);
+ return iris_hfi_set_property(inst,
+ HFI_PROP_QUALITY_MODE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_BITSTREAM,
+ HFI_PAYLOAD_U32_ENUM,
+ &mode,
+ sizeof(u32));
+}
+
+static int venc_set_input_properties(struct iris_inst *inst)
+{
+ int j, ret;
+ static const struct venc_prop_type_handle prop_type_handle_arr[] = {
+ {HFI_PROP_COLOR_FORMAT, venc_set_colorformat },
+ {HFI_PROP_RAW_RESOLUTION, venc_set_raw_resolution },
+ {HFI_PROP_CROP_OFFSETS, venc_set_inp_crop_offsets },
+ {HFI_PROP_LINEAR_STRIDE_SCANLINE, venc_set_stride_scanline },
+ {HFI_PROP_SIGNAL_COLOR_INFO, venc_set_colorspace },
+ };
+
+ for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) {
+ ret = prop_type_handle_arr[j].handle(inst);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+static int venc_property_subscription(struct iris_inst *inst, u32 plane)
+{
+ u32 payload[32] = {0};
+ u32 payload_size = 0;
+ u32 i;
+
+ payload[0] = HFI_MODE_PROPERTY;
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(enc_input_properties); i++)
+ payload[i + 1] = enc_input_properties[i];
+ payload_size = (ARRAY_SIZE(enc_input_properties) + 1) *
+ sizeof(u32);
+ } else if (plane == OUTPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(enc_output_properties); i++)
+ payload[i + 1] = enc_output_properties[i];
+ payload_size = (ARRAY_SIZE(enc_output_properties) + 1) *
+ sizeof(u32);
+ } else {
+ return -EINVAL;
+ }
+
+ return iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ plane,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ payload_size);
+}
+
+int venc_streamon_input(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_session_supported(inst);
+ if (ret)
+ goto error;
+
+ ret = check_scaling_supported(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_set_input_properties(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_get_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_destroy_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_create_input_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_queue_input_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_property_subscription(inst, INPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = process_streamon_input(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ return ret;
+}
+
+static int venc_set_output_properties(struct iris_inst *inst)
+{
+ int j, ret;
+ static const struct venc_prop_type_handle prop_type_handle_arr[] = {
+ {HFI_PROP_BITSTREAM_RESOLUTION, venc_set_bitstream_resolution },
+ {HFI_PROP_CROP_OFFSETS, venc_set_out_crop_offsets },
+ };
+
+ for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) {
+ ret = prop_type_handle_arr[j].handle(inst);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int venc_streamon_output(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = venc_set_output_properties(inst);
+ if (ret)
+ goto error;
+
+ ret = set_v4l2_properties(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_set_quality_mode(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_get_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_destroy_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_create_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_queue_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_property_subscription(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = process_streamon_output(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ session_streamoff(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
index 24da63f..b289750 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
@@ -20,5 +20,8 @@ int venc_subscribe_event(struct iris_inst *inst,
const struct v4l2_event_subscription *sub);
int venc_start_cmd(struct iris_inst *inst);
int venc_stop_cmd(struct iris_inst *inst);
+int venc_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2);
+int venc_streamon_input(struct iris_inst *inst);
+int venc_streamon_output(struct iris_inst *inst);
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index effecbb..42ac662 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -71,6 +71,11 @@ enum pipe_type {
PIPE_4 = 4,
};
+enum quality_mode {
+ MAX_QUALITY_MODE = 0x1,
+ POWER_SAVE_MODE = 0x2,
+};
+
extern struct platform_data sm8550_data;
struct bw_info {
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index ef0aad7..17c5856 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -74,6 +74,8 @@ static struct plat_core_cap core_data_sm8550[] = {
{CP_SIZE, 0x25800000},
{CP_NONPIXEL_START, 0x01000000},
{CP_NONPIXEL_SIZE, 0x24800000},
+ {MAX_MBPF_HQ, 8160}, /* ((1920x1088)/256) */
+ {MAX_MBPS_HQ, 489600}, /* ((1920x1088)/256)@60fps */
};
static struct plat_inst_cap instance_cap_data_sm8550[] = {
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
index 44f9342..9882f00 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
@@ -163,6 +163,186 @@ static u32 dec_dpb_size_iris3(struct iris_inst *inst)
return size;
}
+static u32 enc_bin_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes, stage, profile;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+ stage = inst->cap[STAGE].value;
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ profile = inst->cap[PROFILE].value;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_bin_h264e(inst->hfi_rc_type, width, height,
+ stage, num_vpp_pipes, profile);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_bin_h265e(inst->hfi_rc_type, width, height,
+ stage, num_vpp_pipes, profile);
+
+ return size;
+}
+
+u32 get_recon_buf_count(struct iris_inst *inst)
+{
+ s32 n_bframe, ltr_count, hp_layers = 0, hb_layers = 0;
+ bool is_hybrid_hp = false;
+ u32 num_buf_recon = 0;
+ u32 hfi_codec = 0;
+
+ n_bframe = inst->cap[B_FRAME].value;
+ ltr_count = inst->cap[LTR_COUNT].value;
+
+ if (inst->hfi_layer_type == HFI_HIER_B) {
+ hb_layers = inst->cap[ENH_LAYER_COUNT].value + 1;
+ } else {
+ hp_layers = inst->cap[ENH_LAYER_COUNT].value + 1;
+ if (inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR)
+ is_hybrid_hp = true;
+ }
+
+ if (inst->codec == H264)
+ hfi_codec = HFI_CODEC_ENCODE_AVC;
+ else if (inst->codec == HEVC)
+ hfi_codec = HFI_CODEC_ENCODE_HEVC;
+
+ num_buf_recon = hfi_iris3_enc_recon_buf_count(n_bframe, ltr_count,
+ hp_layers, hb_layers,
+ is_hybrid_hp, hfi_codec);
+
+ return num_buf_recon;
+}
+
+static u32 enc_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_recon = 0;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ num_recon = get_recon_buf_count(inst);
+ if (inst->codec == H264)
+ size = hfi_buffer_comv_h264e(width, height, num_recon);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_comv_h265e(width, height, num_recon);
+
+ return size;
+}
+
+static u32 enc_non_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_non_comv_h264e(width, height, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_non_comv_h265e(width, height, num_vpp_pipes);
+
+ return size;
+}
+
+static u32 enc_line_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, pixfmt, num_vpp_pipes;
+ struct iris_core *core;
+ bool is_tenbit = false;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+ pixfmt = inst->cap[PIX_FMTS].value;
+
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ is_tenbit = (pixfmt == FMT_TP10C);
+
+ if (inst->codec == H264)
+ size = hfi_buffer_line_h264e(width, height, is_tenbit, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_line_h265e(width, height, is_tenbit, num_vpp_pipes);
+
+ return size;
+}
+
+static u32 enc_dpb_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, pixfmt;
+ struct v4l2_format *f;
+ bool is_tenbit;
+ u32 size = 0;
+
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ pixfmt = inst->cap[PIX_FMTS].value;
+ is_tenbit = (pixfmt == FMT_TP10C);
+
+ if (inst->codec == H264)
+ size = hfi_buffer_dpb_h264e(width, height);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_dpb_h265e(width, height, is_tenbit);
+
+ return size;
+}
+
+static u32 enc_arp_size_iris3(struct iris_inst *inst)
+{
+ u32 size = 0;
+
+ HFI_BUFFER_ARP_ENC(size);
+
+ return size;
+}
+
+static u32 enc_vpss_size_iris3(struct iris_inst *inst)
+{
+ bool ds_enable = false, is_tenbit = false;
+ struct v4l2_format *f;
+ u32 width, height;
+ u32 size = 0;
+
+ ds_enable = is_scaling_enabled(inst);
+
+ f = inst->fmt_dst;
+ if (inst->cap[ROTATION].value == 90 ||
+ inst->cap[ROTATION].value == 270) {
+ width = f->fmt.pix_mp.height;
+ height = f->fmt.pix_mp.width;
+ } else {
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ }
+
+ f = inst->fmt_src;
+ is_tenbit = is_10bit_colorformat(f->fmt.pix_mp.pixelformat);
+
+ size = hfi_buffer_vpss_enc(width, height, ds_enable, 0, is_tenbit);
+
+ return size;
+}
+
struct iris_buf_type_handle {
enum iris_buffer_type type;
u32 (*handle)(struct iris_inst *inst);
@@ -183,9 +363,23 @@ int iris_int_buf_size_iris3(struct iris_inst *inst,
{BUF_PERSIST, dec_persist_size_iris3 },
{BUF_DPB, dec_dpb_size_iris3 },
};
+ static const struct iris_buf_type_handle enc_internal_buf_type_handle[] = {
+ {BUF_BIN, enc_bin_size_iris3 },
+ {BUF_COMV, enc_comv_size_iris3 },
+ {BUF_NON_COMV, enc_non_comv_size_iris3 },
+ {BUF_LINE, enc_line_size_iris3 },
+ {BUF_DPB, enc_dpb_size_iris3 },
+ {BUF_ARP, enc_arp_size_iris3 },
+ {BUF_VPSS, enc_vpss_size_iris3 },
+ };
- buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle);
- buf_type_handle_arr = dec_internal_buf_type_handle;
+ if (inst->domain == DECODER) {
+ buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle);
+ buf_type_handle_arr = dec_internal_buf_type_handle;
+ } else if (inst->domain == ENCODER) {
+ buf_type_handle_size = ARRAY_SIZE(enc_internal_buf_type_handle);
+ buf_type_handle_arr = enc_internal_buf_type_handle;
+ }
if (!buf_type_handle_arr || !buf_type_handle_size)
return size;
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
index b520c79..c63b3ac 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
@@ -14,8 +14,10 @@
#define DMA_ALIGNMENT 256
+#define BUFFER_ALIGNMENT_4096_BYTES 4096
#define BUFFER_ALIGNMENT_512_BYTES 512
#define BUFFER_ALIGNMENT_256_BYTES 256
+#define BUFFER_ALIGNMENT_128_BYTES 128
#define BUFFER_ALIGNMENT_64_BYTES 64
#define BUFFER_ALIGNMENT_32_BYTES 32
#define BUFFER_ALIGNMENT_16_BYTES 16
@@ -26,6 +28,8 @@
#define HFI_COL_FMT_NV12C_Y_TILE_WIDTH (32)
#define HFI_COL_FMT_NV12C_UV_TILE_HEIGHT (8)
#define HFI_COL_FMT_NV12C_UV_TILE_WIDTH (16)
+#define HFI_COL_FMT_TP10C_Y_TILE_HEIGHT (4)
+#define HFI_COL_FMT_TP10C_Y_TILE_WIDTH (48)
#define NUM_HW_PIC_BUF 32
#define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf))
@@ -102,6 +106,27 @@
#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE (64 * 2 * 3)
#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640
+#define HFI_BUFFER_ARP_ENC(size) (size = 204800)
+
+#define HFI_MAX_COL_FRAME 6
+#define HFI_VENUS_WIDTH_ALIGNMENT 128
+#define HFI_VENUS_HEIGHT_ALIGNMENT 32
+#define VENUS_METADATA_STRIDE_MULTIPLE 64
+#define VENUS_METADATA_HEIGHT_MULTIPLE 16
+
+#ifndef SYSTEM_LAL_TILE10
+#define SYSTEM_LAL_TILE10 192
+#endif
+
+#define SIZE_SLICE_CMD_BUFFER (ALIGN(20480, BUFFER_ALIGNMENT_256_BYTES))
+#define SIZE_SPS_PPS_SLICE_HDR (2048 + 4096)
+
+#define SIZE_BSE_SLICE_CMD_BUF ((((8192 << 2) + 7) & (~7)) * 3)
+#define SIZE_LAMBDA_LUT (256 * 11)
+
+#define HFI_WORKMODE_1 1
+#define HFI_WORKMODE_2 2
+
static inline
u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height,
u32 num_vpp_pipes)
@@ -842,4 +867,603 @@ u32 hfi_yuv420_tp10_ubwc_calc_buf_size(u32 y_stride, u32 y_buf_height,
int iris_int_buf_size_iris3(struct iris_inst *inst,
enum iris_buffer_type buffer_type);
+static inline
+u32 hfi_iris3_enc_recon_buf_count(s32 n_bframe, s32 ltr_count,
+ s32 _total_hp_layers,
+ s32 _total_hb_layers,
+ bool hybrid_hp,
+ u32 codec_standard)
+{
+ u32 num_ref = 1;
+
+ if (n_bframe)
+ num_ref = 2;
+
+ if (_total_hp_layers > 1) {
+ if (hybrid_hp)
+ num_ref = (_total_hp_layers + 1) >> 1;
+ else if (codec_standard == HFI_CODEC_ENCODE_HEVC)
+ num_ref = (_total_hp_layers + 1) >> 1;
+ else if (codec_standard == HFI_CODEC_ENCODE_AVC &&
+ _total_hp_layers < 4)
+ num_ref = (_total_hp_layers - 1);
+ else
+ num_ref = _total_hp_layers;
+ }
+
+ if (ltr_count)
+ num_ref = num_ref + ltr_count;
+
+ if (_total_hb_layers > 1) {
+ if (codec_standard == HFI_CODEC_ENCODE_HEVC)
+ num_ref = (_total_hb_layers);
+ else if (codec_standard == HFI_CODEC_ENCODE_AVC)
+ num_ref = (1 << (_total_hb_layers - 2)) + 1;
+ }
+
+ return num_ref + 1;
+}
+
+static inline
+u32 size_bin_bitstream_enc(u32 rc_type, u32 frame_width, u32 frame_height,
+ u32 work_mode, u32 lcu_size, u32 profile)
+{
+ u32 size_aligned_width = 0, size_aligned_height = 0;
+ u32 bitstream_size_eval = 0;
+
+ size_aligned_width = ALIGN((frame_width), lcu_size);
+ size_aligned_height = ALIGN((frame_height), lcu_size);
+ if (work_mode == HFI_WORKMODE_2) {
+ if (rc_type == HFI_RC_CQ || rc_type == HFI_RC_OFF) {
+ bitstream_size_eval = (((size_aligned_width) *
+ (size_aligned_height) * 3) >> 1);
+ } else {
+ bitstream_size_eval = ((size_aligned_width) *
+ (size_aligned_height) * 3);
+ if ((size_aligned_width * size_aligned_height) > (4096 * 2176))
+ bitstream_size_eval >>= 3;
+ else if ((size_aligned_width * size_aligned_height) > (480 * 320))
+ bitstream_size_eval >>= 2;
+
+ if (profile == HFI_H265_PROFILE_MAIN_10 ||
+ profile == HFI_H265_PROFILE_MAIN_10_STILL_PICTURE)
+ bitstream_size_eval = (bitstream_size_eval * 5 >> 2);
+ }
+ } else {
+ bitstream_size_eval = size_aligned_width * size_aligned_height * 3;
+ }
+
+ return ALIGN(bitstream_size_eval, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_enc_single_pipe(u32 rc_type, u32 bitbin_size, u32 num_vpp_pipes,
+ u32 frame_width, u32 frame_height, u32 lcu_size)
+{
+ u32 size_single_pipe_eval = 0, sao_bin_buffer_size = 0;
+ u32 size_aligned_width = 0, size_aligned_height = 0;
+ u32 _padded_bin_sz = 0;
+
+ size_aligned_width = ALIGN((frame_width), lcu_size);
+ size_aligned_height = ALIGN((frame_height), lcu_size);
+ if ((size_aligned_width * size_aligned_height) > (3840 * 2160))
+ size_single_pipe_eval = (bitbin_size / num_vpp_pipes);
+ else if (num_vpp_pipes > 2)
+ size_single_pipe_eval = bitbin_size / 2;
+ else
+ size_single_pipe_eval = bitbin_size;
+
+ sao_bin_buffer_size =
+ (64 * ((((frame_width) + BUFFER_ALIGNMENT_32_BYTES) *
+ ((frame_height) + BUFFER_ALIGNMENT_32_BYTES)) >> 10)) + 384;
+ _padded_bin_sz = ALIGN(size_single_pipe_eval, BUFFER_ALIGNMENT_256_BYTES);
+ size_single_pipe_eval = sao_bin_buffer_size + _padded_bin_sz;
+ size_single_pipe_eval = ALIGN(size_single_pipe_eval, BUFFER_ALIGNMENT_256_BYTES);
+
+ return size_single_pipe_eval;
+}
+
+static inline
+u32 hfi_buffer_bin_enc(u32 rc_type, u32 frame_width, u32 frame_height, u32 lcu_size,
+ u32 work_mode, u32 num_vpp_pipes, u32 profile)
+{
+ u32 bitstream_size = 0, total_bitbin_buffers = 0;
+ u32 size_single_pipe = 0, bitbin_size = 0;
+ u32 _size = 0;
+
+ bitstream_size = size_bin_bitstream_enc(rc_type, frame_width,
+ frame_height, work_mode,
+ lcu_size, profile);
+
+ if (work_mode == HFI_WORKMODE_2) {
+ total_bitbin_buffers = 3;
+ bitbin_size = bitstream_size * 12 / 10;
+ bitbin_size = ALIGN(bitbin_size, BUFFER_ALIGNMENT_256_BYTES);
+ } else if ((lcu_size == 16) || (num_vpp_pipes > 1)) {
+ total_bitbin_buffers = 1;
+ bitbin_size = bitstream_size;
+ }
+
+ if (total_bitbin_buffers > 0) {
+ size_single_pipe = size_enc_single_pipe(rc_type, bitbin_size,
+ num_vpp_pipes, frame_width,
+ frame_height, lcu_size);
+ bitbin_size = size_single_pipe * num_vpp_pipes;
+ _size = ALIGN(bitbin_size, BUFFER_ALIGNMENT_256_BYTES) *
+ total_bitbin_buffers + 512;
+ } else {
+ /* Avoid 512 Bytes allocation in case of 1Pipe HEVC Direct Mode*/
+ _size = 0;
+ }
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_bin_h264e(u32 rc_type, u32 frame_width, u32 frame_height,
+ u32 work_mode, u32 num_vpp_pipes, u32 profile)
+{
+ return hfi_buffer_bin_enc(rc_type, frame_width, frame_height, 16,
+ work_mode, num_vpp_pipes, profile);
+}
+
+static inline
+u32 hfi_buffer_bin_h265e(u32 rc_type, u32 frame_width, u32 frame_height,
+ u32 work_mode, u32 num_vpp_pipes, u32 profile)
+{
+ return hfi_buffer_bin_enc(rc_type, frame_width, frame_height, 32,
+ work_mode, num_vpp_pipes, profile);
+}
+
+static inline
+u32 size_enc_slice_info_buf(u32 num_lcu_in_frame)
+{
+ return ALIGN((256 + (num_lcu_in_frame << 4)), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_line_buf_ctrl(u32 frame_width_coded)
+{
+ return ALIGN(frame_width_coded, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_line_buf_ctrl_id2(u32 frame_width_coded)
+{
+ return ALIGN(frame_width_coded, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_linebuff_data(bool is_ten_bit, u32 frame_width_coded)
+{
+ u32 _size = 0;
+
+ _size = is_ten_bit ?
+ (((((10 * (frame_width_coded) + 1024) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 1) +
+ (((((10 * (frame_width_coded) + 1024) >> 1) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 2)) :
+ (((((8 * (frame_width_coded) + 1024) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 1) +
+ (((((8 * (frame_width_coded) + 1024) >> 1) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 2));
+
+ return _size;
+}
+
+static inline
+u32 size_left_linebuff_ctrl(u32 standard, u32 frame_height_coded,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = standard == HFI_CODEC_ENCODE_HEVC ?
+ (((frame_height_coded) +
+ (BUFFER_ALIGNMENT_32_BYTES)) / BUFFER_ALIGNMENT_32_BYTES * 4 * 16) :
+ (((frame_height_coded) + 15) / 16 * 5 * 16);
+
+ if ((num_vpp_pipes_enc) > 1) {
+ _size += BUFFER_ALIGNMENT_512_BYTES;
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_512_BYTES) *
+ num_vpp_pipes_enc;
+ }
+
+ return ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_left_linebuff_recon_pix(bool is_ten_bit, u32 frame_height_coded,
+ u32 num_vpp_pipes_enc)
+{
+ return (((is_ten_bit + 1) * 2 * (frame_height_coded) + BUFFER_ALIGNMENT_256_BYTES) +
+ (BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1) &
+ (~((BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1)) * 1;
+}
+
+static inline
+u32 size_top_linebuff_ctrl_fe(u32 frame_width_coded, u32 standard)
+{
+ u32 _size = 0;
+
+ _size = standard == HFI_CODEC_ENCODE_HEVC ?
+ (64 * ((frame_width_coded) >> 5)) :
+ (BUFFER_ALIGNMENT_256_BYTES + 16 * ((frame_width_coded) >> 4));
+
+ return ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_left_linebuff_ctrl_fe(u32 frame_height_coded, u32 num_vpp_pipes_enc)
+{
+ return (((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) >> 4)) +
+ (BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1) &
+ (~((BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1)) * 1) *
+ num_vpp_pipes_enc;
+}
+
+static inline
+u32 size_left_linebuff_metadata_recon_y(u32 frame_height_coded,
+ bool is_ten_bit,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = ((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) /
+ (8 * (is_ten_bit ? 4 : 8)))));
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+
+ return (_size * num_vpp_pipes_enc);
+}
+
+static inline
+u32 size_left_linebuff_metadata_recon_uv(u32 frame_height_coded,
+ bool is_ten_bit,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = ((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) /
+ (4 * (is_ten_bit ? 4 : 8)))));
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+
+ return (_size * num_vpp_pipes_enc);
+}
+
+static inline
+u32 size_linebuff_recon_pix(bool is_ten_bit, u32 frame_width_coded)
+{
+ return ALIGN(((is_ten_bit ? 3 : 2) * (frame_width_coded)),
+ BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_frame_rc_buf_size(u32 standard, u32 frame_height_coded,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = (standard == HFI_CODEC_ENCODE_HEVC) ?
+ (256 + 16 * (14 + ((((frame_height_coded) >> 5) + 7) >> 3))) :
+ (256 + 16 * (14 + ((((frame_height_coded) >> 4) + 7) >> 3)));
+ _size *= 11;
+ if (num_vpp_pipes_enc > 1)
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES) * num_vpp_pipes_enc;
+
+ return ALIGN(_size, BUFFER_ALIGNMENT_512_BYTES) * HFI_MAX_COL_FRAME;
+}
+
+static inline u32 enc_bitcnt_buf_size(u32 num_lcu_in_frame)
+{
+ return ALIGN((256 + (4 * (num_lcu_in_frame))), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline u32 enc_bitmap_buf_size(u32 num_lcu_in_frame)
+{
+ return ALIGN((256 + ((num_lcu_in_frame) >> 3)), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline u32 size_line_buf_sde(u32 frame_width_coded)
+{
+ return ALIGN((256 + (16 * ((frame_width_coded) >> 4))), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline u32 size_override_buf(u32 num_lcumb)
+{
+ return ALIGN(((16 * (((num_lcumb) + 7) >> 3))), BUFFER_ALIGNMENT_256_BYTES) * 2;
+}
+
+static inline u32 size_ir_buf(u32 num_lcu_in_frame)
+{
+ return ALIGN((((((num_lcu_in_frame) << 1) + 7) & (~7)) * 3), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_vpss_line_buf(u32 num_vpp_pipes_enc, u32 frame_height_coded,
+ u32 frame_width_coded)
+{
+ return ALIGN(((((((8192) >> 2) << 5) * (num_vpp_pipes_enc)) + 64) +
+ (((((max_t(u32, (frame_width_coded),
+ (frame_height_coded)) + 3) >> 2) << 5) + 256) * 16)),
+ BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_top_line_buf_first_stg_sao(u32 frame_width_coded)
+{
+ return ALIGN((16 * ((frame_width_coded) >> 5)), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 hfi_buffer_line_enc(u32 frame_width, u32 frame_height, bool is_ten_bit,
+ u32 num_vpp_pipes_enc, u32 lcu_size, u32 standard)
+{
+ u32 line_buff_data_size = 0, left_line_buff_ctrl_size = 0;
+ u32 frame_width_coded = 0, frame_height_coded = 0;
+ u32 left_line_buff_metadata_recon__uv__size = 0;
+ u32 left_line_buff_metadata_recon__y__size = 0;
+ u32 width_in_lcus = 0, height_in_lcus = 0;
+ u32 left_line_buff_recon_pix_size = 0;
+ u32 top_line_buff_ctrl_fe_size = 0;
+ u32 line_buff_recon_pix_size = 0;
+
+ width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size);
+ height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size);
+ frame_width_coded = width_in_lcus * (lcu_size);
+ frame_height_coded = height_in_lcus * (lcu_size);
+ line_buff_data_size = size_linebuff_data(is_ten_bit, frame_width_coded);
+ left_line_buff_ctrl_size =
+ size_left_linebuff_ctrl(standard, frame_height_coded, num_vpp_pipes_enc);
+ left_line_buff_recon_pix_size =
+ size_left_linebuff_recon_pix(is_ten_bit, frame_height_coded,
+ num_vpp_pipes_enc);
+ top_line_buff_ctrl_fe_size =
+ size_top_linebuff_ctrl_fe(frame_width_coded, standard);
+ left_line_buff_metadata_recon__y__size =
+ size_left_linebuff_metadata_recon_y(frame_height_coded, is_ten_bit,
+ num_vpp_pipes_enc);
+ left_line_buff_metadata_recon__uv__size =
+ size_left_linebuff_metadata_recon_uv(frame_height_coded, is_ten_bit,
+ num_vpp_pipes_enc);
+ line_buff_recon_pix_size = size_linebuff_recon_pix(is_ten_bit, frame_width_coded);
+
+ return size_line_buf_ctrl(frame_width_coded) +
+ size_line_buf_ctrl_id2(frame_width_coded) +
+ line_buff_data_size +
+ left_line_buff_ctrl_size +
+ left_line_buff_recon_pix_size +
+ top_line_buff_ctrl_fe_size +
+ left_line_buff_metadata_recon__y__size +
+ left_line_buff_metadata_recon__uv__size +
+ line_buff_recon_pix_size +
+ size_left_linebuff_ctrl_fe(frame_height_coded,
+ num_vpp_pipes_enc) +
+ size_line_buf_sde(frame_width_coded) +
+ size_vpss_line_buf(num_vpp_pipes_enc, frame_height_coded,
+ frame_width_coded) +
+ size_top_line_buf_first_stg_sao(frame_width_coded);
+}
+
+static inline
+u32 hfi_buffer_line_h264e(u32 frame_width, u32 frame_height, bool is_ten_bit,
+ u32 num_vpp_pipes)
+{
+ return hfi_buffer_line_enc(frame_width, frame_height, 0,
+ num_vpp_pipes, 16,
+ HFI_CODEC_ENCODE_AVC);
+}
+
+static inline
+u32 hfi_buffer_line_h265e(u32 frame_width, u32 frame_height, bool is_ten_bit,
+ u32 num_vpp_pipes)
+{
+ return hfi_buffer_line_enc(frame_width, frame_height,
+ is_ten_bit, num_vpp_pipes, 32,
+ HFI_CODEC_ENCODE_HEVC);
+}
+
+static inline
+u32 hfi_buffer_comv_enc(u32 frame_width, u32 frame_height, u32 lcu_size,
+ u32 num_recon, u32 standard)
+{
+ u32 width_in_lcus, height_in_lcus, num_lcu_in_frame;
+ u32 size_colloc_mv = 0, size_colloc_rc = 0;
+ u32 mb_height = ((frame_height) + 15) >> 4;
+ u32 mb_width = ((frame_width) + 15) >> 4;
+
+ width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size);
+ height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size);
+ num_lcu_in_frame = width_in_lcus * height_in_lcus;
+ size_colloc_mv = (standard == HFI_CODEC_ENCODE_HEVC) ?
+ (16 * ((num_lcu_in_frame << 2) + BUFFER_ALIGNMENT_32_BYTES)) :
+ (3 * 16 * (width_in_lcus * height_in_lcus + BUFFER_ALIGNMENT_32_BYTES));
+ size_colloc_mv = ALIGN(size_colloc_mv, BUFFER_ALIGNMENT_256_BYTES) * num_recon;
+ size_colloc_rc = (((mb_width + 7) >> 3) * 16 * 2 * mb_height);
+ size_colloc_rc = ALIGN(size_colloc_rc, BUFFER_ALIGNMENT_256_BYTES) * HFI_MAX_COL_FRAME;
+
+ return size_colloc_mv + size_colloc_rc;
+}
+
+static inline
+u32 hfi_buffer_comv_h264e(u32 frame_width, u32 frame_height, u32 num_recon)
+{
+ return hfi_buffer_comv_enc(frame_width, frame_height, 16,
+ num_recon, HFI_CODEC_ENCODE_AVC);
+}
+
+static inline
+u32 hfi_buffer_comv_h265e(u32 frame_width, u32 frame_height, u32 num_recon)
+{
+ return hfi_buffer_comv_enc(frame_width, frame_height, 32,
+ num_recon, HFI_CODEC_ENCODE_HEVC);
+}
+
+static inline
+u32 hfi_buffer_non_comv_enc(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes_enc, u32 lcu_size, u32 standard)
+{
+ u32 frame_width_coded = 0, frame_height_coded = 0;
+ u32 width_in_lcus = 0, height_in_lcus = 0;
+ u32 num_lcu_in_frame = 0, num_lcumb = 0;
+ u32 frame_rc_buf_size = 0;
+
+ width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size);
+ height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size);
+ num_lcu_in_frame = width_in_lcus * height_in_lcus;
+ frame_width_coded = width_in_lcus * (lcu_size);
+ frame_height_coded = height_in_lcus * (lcu_size);
+ num_lcumb = (frame_height_coded / lcu_size) *
+ ((frame_width_coded + lcu_size * 8) / lcu_size);
+ frame_rc_buf_size =
+ size_frame_rc_buf_size(standard, frame_height_coded,
+ num_vpp_pipes_enc);
+ return size_enc_slice_info_buf(num_lcu_in_frame) +
+ SIZE_SLICE_CMD_BUFFER +
+ SIZE_SPS_PPS_SLICE_HDR +
+ frame_rc_buf_size +
+ enc_bitcnt_buf_size(num_lcu_in_frame) +
+ enc_bitmap_buf_size(num_lcu_in_frame) +
+ SIZE_BSE_SLICE_CMD_BUF +
+ SIZE_LAMBDA_LUT +
+ size_override_buf(num_lcumb) +
+ size_ir_buf(num_lcu_in_frame);
+}
+
+static inline
+u32 hfi_buffer_non_comv_h264e(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes_enc)
+{
+ return hfi_buffer_non_comv_enc(frame_width, frame_height,
+ num_vpp_pipes_enc, 16,
+ HFI_CODEC_ENCODE_AVC);
+}
+
+static inline
+u32 hfi_buffer_non_comv_h265e(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes_enc)
+{
+ return hfi_buffer_non_comv_enc(frame_width, frame_height,
+ num_vpp_pipes_enc, 32,
+ HFI_CODEC_ENCODE_HEVC);
+}
+
+static inline
+u32 size_enc_ref_buffer(u32 frame_width, u32 frame_height)
+{
+ u32 u_buffer_width = 0, u_buffer_height = 0;
+ u32 u_chroma_buffer_height = 0;
+
+ u_buffer_height = ALIGN(frame_height, HFI_VENUS_HEIGHT_ALIGNMENT);
+ u_chroma_buffer_height = frame_height >> 1;
+ u_chroma_buffer_height = ALIGN(u_chroma_buffer_height, HFI_VENUS_HEIGHT_ALIGNMENT);
+ u_buffer_width = ALIGN(frame_width, HFI_VENUS_WIDTH_ALIGNMENT);
+
+ return (u_buffer_height + u_chroma_buffer_height) * u_buffer_width;
+}
+
+static inline
+u32 size_enc_ten_bit_ref_buffer(u32 frame_width, u32 frame_height)
+{
+ u32 ref_buf_height = 0, ref_luma_stride_in_bytes = 0;
+ u32 chroma_size = 0, ref_buf_size = 0;
+ u32 u_ref_stride = 0, luma_size = 0;
+ u32 ref_chrm_height_in_bytes = 0;
+
+ ref_buf_height = (frame_height + (HFI_VENUS_HEIGHT_ALIGNMENT - 1)) &
+ (~(HFI_VENUS_HEIGHT_ALIGNMENT - 1));
+ ref_luma_stride_in_bytes = ((frame_width + SYSTEM_LAL_TILE10 - 1) / SYSTEM_LAL_TILE10) *
+ SYSTEM_LAL_TILE10;
+ u_ref_stride = 4 * (ref_luma_stride_in_bytes / 3);
+ u_ref_stride = (u_ref_stride + (BUFFER_ALIGNMENT_128_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_128_BYTES - 1));
+ luma_size = ref_buf_height * u_ref_stride;
+ ref_chrm_height_in_bytes = (((frame_height + 1) >> 1) + (BUFFER_ALIGNMENT_32_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_32_BYTES - 1));
+ chroma_size = u_ref_stride * ref_chrm_height_in_bytes;
+ luma_size = (luma_size + (BUFFER_ALIGNMENT_4096_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_4096_BYTES - 1));
+ chroma_size = (chroma_size + (BUFFER_ALIGNMENT_4096_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_4096_BYTES - 1));
+ ref_buf_size = luma_size + chroma_size;
+
+ return ref_buf_size;
+}
+
+static inline
+u32 hfi_buffer_dpb_enc(u32 frame_width, u32 frame_height, bool is_ten_bit)
+{
+ u32 metadata_stride, metadata_buf_height, meta_size_y, meta_size_c;
+ u32 ten_bit_ref_buf_size = 0, ref_buf_size = 0;
+ u32 _size = 0;
+
+ if (!is_ten_bit) {
+ ref_buf_size = size_enc_ref_buffer(frame_width, frame_height);
+ metadata_stride =
+ hfi_ubwc_calc_metadata_plane_stride(frame_width, 64,
+ HFI_COL_FMT_NV12C_Y_TILE_WIDTH);
+ metadata_buf_height =
+ hfi_ubwc_metadata_plane_bufheight(frame_height, 16,
+ HFI_COL_FMT_NV12C_Y_TILE_HEIGHT);
+ meta_size_y =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ meta_size_c =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ _size = ref_buf_size + meta_size_y + meta_size_c;
+ } else {
+ ten_bit_ref_buf_size = size_enc_ten_bit_ref_buffer(frame_width, frame_height);
+ metadata_stride =
+ hfi_ubwc_calc_metadata_plane_stride(frame_width,
+ VENUS_METADATA_STRIDE_MULTIPLE,
+ HFI_COL_FMT_TP10C_Y_TILE_WIDTH);
+ metadata_buf_height =
+ hfi_ubwc_metadata_plane_bufheight(frame_height,
+ VENUS_METADATA_HEIGHT_MULTIPLE,
+ HFI_COL_FMT_TP10C_Y_TILE_HEIGHT);
+ meta_size_y =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ meta_size_c =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ _size = ten_bit_ref_buf_size + meta_size_y + meta_size_c;
+ }
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_dpb_h264e(u32 frame_width, u32 frame_height)
+{
+ return hfi_buffer_dpb_enc(frame_width, frame_height, 0);
+}
+
+static inline
+u32 hfi_buffer_dpb_h265e(u32 frame_width, u32 frame_height, bool is_ten_bit)
+{
+ return hfi_buffer_dpb_enc(frame_width, frame_height, is_ten_bit);
+}
+
+static inline
+u32 hfi_buffer_vpss_enc(u32 dswidth, u32 dsheight, bool ds_enable,
+ u32 blur, bool is_ten_bit)
+{
+ u32 vpss_size = 0;
+
+ if (ds_enable || blur)
+ vpss_size = hfi_buffer_dpb_enc(dswidth, dsheight, is_ten_bit);
+
+ return vpss_size;
+}
+
+static inline
+u32 hfi_iris3_enc_min_input_buf_count(u32 totalhblayers)
+{
+ u32 numinput = 3;
+
+ if (totalhblayers >= 2)
+ numinput = (1 << (totalhblayers - 1)) + 2;
+
+ return numinput;
+}
+
+u32 enc_output_buffer_size_iris3(struct iris_inst *inst);
+u32 get_recon_buf_count(struct iris_inst *inst);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
index 58498af..2828227 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -54,7 +54,7 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
if (inst->cap[STAGE].value == STAGE_1)
vsp_cycles = vsp_cycles * 3;
- vsp_cycles += mbs_per_second * base_cycles;
+ vsp_cycles += mbs_per_second * base_cycles;
freq = max3(vpp_cycles, vsp_cycles, fw_cycles);
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* [PATCH v2 34/34] media: iris: add power management for encoder
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (32 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 33/34] media: iris: add vb2 streaming and buffer ops for encoder Dikshita Agarwal
@ 2023-12-18 11:32 ` Dikshita Agarwal
2023-12-19 12:26 ` Konrad Dybcio
2023-12-18 12:09 ` [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (6 subsequent siblings)
40 siblings, 1 reply; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 11:32 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Hardware specific power sequence include programming specific
sequence of registers. Implements this sequence for iris3.
Also, implement iris3 Encoder specific calculation for clock
and bus bandwidth which depends on hardware configuration,
codec format, resolution and frame rate.
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 2 +
.../media/platform/qcom/vcodec/iris/iris_power.c | 15 ++--
.../platform/qcom/vcodec/iris/platform_common.h | 3 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 14 +++
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 100 +++++++++++++++------
5 files changed, 103 insertions(+), 31 deletions(-)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 559b0dd..1a2e305 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -799,6 +799,8 @@ int decide_quality_mode(struct iris_inst *inst)
if (mbpf <= max_hq_mbpf && mbps <= max_hq_mbps)
mode = MAX_QUALITY_MODE;
+ inst->cap[QUALITY_MODE].value = mode;
+
return mode;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.c b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
index 77439e3..be71751 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
@@ -48,11 +48,16 @@ static int iris_vote_buses(struct iris_inst *inst)
vote_data->height = inp_f->fmt.pix_mp.height;
vote_data->fps = inst->max_rate;
- if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) {
- vote_data->color_formats[0] = V4L2_PIX_FMT_NV12;
- vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat;
- } else {
- vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat;
+ if (inst->domain == ENCODER) {
+ vote_data->color_formats[0] =
+ v4l2_colorformat_to_driver(inst, inst->fmt_src->fmt.pix_mp.pixelformat);
+ } else if (inst->domain == DECODER) {
+ if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) {
+ vote_data->color_formats[0] = V4L2_PIX_FMT_NV12;
+ vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat;
+ } else {
+ vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat;
+ }
}
call_session_op(core, calc_bw, inst, vote_data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 42ac662..e9b5c99 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -174,6 +174,7 @@ enum plat_inst_cap_type {
INPUT_BUF_HOST_MAX_COUNT,
STAGE,
PIPE,
+ QUALITY_MODE,
POC,
CODED_FRAMES,
BIT_DEPTH,
@@ -279,6 +280,8 @@ struct format_capability {
struct platform_data {
const struct bus_info *bus_tbl;
unsigned int bus_tbl_size;
+ const struct bw_info *bw_tbl_enc;
+ unsigned int bw_tbl_enc_size;
const struct bw_info *bw_tbl_dec;
unsigned int bw_tbl_dec_size;
const char * const *pd_tbl;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 17c5856..d47c985 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -1005,6 +1005,11 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_pipe},
+ {QUALITY_MODE, ENC, CODECS_ALL,
+ MAX_QUALITY_MODE,
+ POWER_SAVE_MODE, 1,
+ POWER_SAVE_MODE},
+
{POC, DEC, H264, 0, 2, 1, 1,
0,
HFI_PROP_PIC_ORDER_CNT_TYPE},
@@ -1051,6 +1056,13 @@ static const char * const sm8550_pd_table[] = { "iris-ctl", "vcodec", NULL };
static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx", NULL };
+static const struct bw_info sm8550_bw_table_enc[] = {
+ { 1944000, 1491000, 2693000 }, /* 3840x2160@60 */
+ { 972000, 765000, 1366000 }, /* 3840x2160@30 */
+ { 489600, 557000, 780000 }, /* 1920x1080@60 */
+ { 244800, 288000, 399000 }, /* 1920x1080@30 */
+};
+
static const struct bw_info sm8550_bw_table_dec[] = {
{ 2073600, 1608000, 2742000 }, /* 4096x2160@60 */
{ 1036800, 826000, 1393000 }, /* 4096x2160@30 */
@@ -1133,6 +1145,8 @@ struct platform_data sm8550_data = {
.clk_rst_tbl = sm8550_clk_reset_table,
.clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table),
+ .bw_tbl_enc = sm8550_bw_table_enc,
+ .bw_tbl_enc_size = ARRAY_SIZE(sm8550_bw_table_enc),
.bw_tbl_dec = sm8550_bw_table_dec,
.bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec),
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
index 2828227..8497b84 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -11,12 +11,13 @@
u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
{
+ u32 operating_rate, vsp_factor_num = 1, vsp_factor_den = 1;
u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0;
u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0;
+ u32 vpp_cycles_per_mb, mbs_per_second;
u32 base_cycles = 0, fps, mbpf;
u32 height = 0, width = 0;
struct v4l2_format *inp_f;
- u32 mbs_per_second;
inp_f = inst->fmt_src;
width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
@@ -29,32 +30,74 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
fw_cycles = fps * inst->cap[MB_CYCLES_FW].value;
fw_vpp_cycles = fps * inst->cap[MB_CYCLES_FW_VPP].value;
- vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value /
- inst->cap[PIPE].value;
- vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles);
-
- if (inst->cap[PIPE].value > 1)
- vpp_cycles += div_u64(vpp_cycles * 59, 1000);
-
- base_cycles = inst->cap[MB_CYCLES_VSP].value;
- bitrate = fps * data_size * 8;
- vsp_cycles = bitrate;
-
- if (inst->codec == VP9) {
- vsp_cycles = div_u64(vsp_cycles * 170, 100);
- } else if (inst->cap[ENTROPY_MODE].value ==
- V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
- vsp_cycles = div_u64(vsp_cycles * 135, 100);
- } else {
- base_cycles = 0;
- vsp_cycles = div_u64(vsp_cycles, 2);
- }
- vsp_cycles = div_u64(vsp_cycles * 21, 20);
+ if (inst->domain == ENCODER) {
+ vpp_cycles_per_mb =
+ inst->cap[QUALITY_MODE].value == POWER_SAVE_MODE ?
+ inst->cap[MB_CYCLES_LP].value :
+ inst->cap[MB_CYCLES_VPP].value;
+
+ vpp_cycles = mbs_per_second * vpp_cycles_per_mb / inst->cap[PIPE].value;
+
+ if (inst->cap[B_FRAME].value > 1)
+ vpp_cycles += (vpp_cycles / 4) + (vpp_cycles / 8);
+ else if (inst->cap[B_FRAME].value)
+ vpp_cycles += vpp_cycles / 4;
+
+ vpp_cycles += max(div_u64(vpp_cycles, 20), fw_vpp_cycles);
+ if (inst->cap[PIPE].value > 1)
+ vpp_cycles += div_u64(vpp_cycles, 100);
+
+ operating_rate = inst->cap[OPERATING_RATE].value >> 16;
+ if (operating_rate > (inst->cap[FRAME_RATE].value >> 16) &&
+ (inst->cap[FRAME_RATE].value >> 16)) {
+ vsp_factor_num = operating_rate;
+ vsp_factor_den = inst->cap[FRAME_RATE].value >> 16;
+ }
+ vsp_cycles = div_u64(((u64)inst->cap[BIT_RATE].value * vsp_factor_num),
+ vsp_factor_den);
+
+ base_cycles = inst->cap[MB_CYCLES_VSP].value;
+ if (inst->cap[ENTROPY_MODE].value ==
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
+ vsp_cycles = div_u64(vsp_cycles * 135, 100);
+ } else {
+ base_cycles = 0;
+ vsp_cycles = div_u64(vsp_cycles, 2);
+ }
+ vsp_cycles = div_u64(vsp_cycles * 21, 20);
+
+ if (inst->cap[STAGE].value == STAGE_1)
+ vsp_cycles = vsp_cycles * 3;
- if (inst->cap[STAGE].value == STAGE_1)
- vsp_cycles = vsp_cycles * 3;
+ vsp_cycles += mbs_per_second * base_cycles;
+ } else if (inst->domain == DECODER) {
+ vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value /
+ inst->cap[PIPE].value;
+ vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles);
+
+ if (inst->cap[PIPE].value > 1)
+ vpp_cycles += div_u64(vpp_cycles * 59, 1000);
+
+ base_cycles = inst->cap[MB_CYCLES_VSP].value;
+ bitrate = fps * data_size * 8;
+ vsp_cycles = bitrate;
+
+ if (inst->codec == VP9) {
+ vsp_cycles = div_u64(vsp_cycles * 170, 100);
+ } else if (inst->cap[ENTROPY_MODE].value ==
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
+ vsp_cycles = div_u64(vsp_cycles * 135, 100);
+ } else {
+ base_cycles = 0;
+ vsp_cycles = div_u64(vsp_cycles, 2);
+ }
+ vsp_cycles = div_u64(vsp_cycles * 21, 20);
+
+ if (inst->cap[STAGE].value == STAGE_1)
+ vsp_cycles = vsp_cycles * 3;
vsp_cycles += mbs_per_second * base_cycles;
+ }
freq = max3(vpp_cycles, vsp_cycles, fw_cycles);
@@ -78,8 +121,13 @@ int iris_calc_bw_iris3(struct iris_inst *inst, struct bus_vote_data *data)
if (mbps == 0)
return 0;
- bw_tbl = core->platform_data->bw_tbl_dec;
- num_rows = core->platform_data->bw_tbl_dec_size;
+ if (inst->domain == DECODER) {
+ bw_tbl = core->platform_data->bw_tbl_dec;
+ num_rows = core->platform_data->bw_tbl_dec_size;
+ } else if (inst->domain == ENCODER) {
+ bw_tbl = core->platform_data->bw_tbl_enc;
+ num_rows = core->platform_data->bw_tbl_enc_size;
+ }
if (!bw_tbl || num_rows == 0)
return 0;
--
2.7.4
^ permalink raw reply related [flat|nested] 102+ messages in thread* Re: [PATCH v2 34/34] media: iris: add power management for encoder
2023-12-18 11:32 ` [PATCH v2 34/34] media: iris: add power management " Dikshita Agarwal
@ 2023-12-19 12:26 ` Konrad Dybcio
0 siblings, 0 replies; 102+ messages in thread
From: Konrad Dybcio @ 2023-12-19 12:26 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Hardware specific power sequence include programming specific
> sequence of registers. Implements this sequence for iris3.
>
> Also, implement iris3 Encoder specific calculation for clock
> and bus bandwidth which depends on hardware configuration,
> codec format, resolution and frame rate.
>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
[...]
> +static const struct bw_info sm8550_bw_table_enc[] = {
> + { 1944000, 1491000, 2693000 }, /* 3840x2160@60 */
> + { 972000, 765000, 1366000 }, /* 3840x2160@30 */
> + { 489600, 557000, 780000 }, /* 1920x1080@60 */
> + { 244800, 288000, 399000 }, /* 1920x1080@30 */
> +};
can this be calculated at runtime by chance?
Konrad
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (33 preceding siblings ...)
2023-12-18 11:32 ` [PATCH v2 34/34] media: iris: add power management " Dikshita Agarwal
@ 2023-12-18 12:09 ` Dikshita Agarwal
2023-12-18 14:36 ` Bryan O'Donoghue
` (5 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Dikshita Agarwal @ 2023-12-18 12:09 UTC (permalink / raw)
To: linux-media, linux-kernel, stanimir.k.varbanov, quic_vgarodia,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.
> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.
>
> Changes done since v1[1]:
> - Patches are created as logical split instead of file by file.
> - Extracted common functionality between venus and iris driver and placed them
> under common folder vcodec.
> - Flattened directory structure.
> - Restructured the code in a simplified layer architecture.
> - Implemented runtime PM, and dropped explicit power collapse thread in driver.
> - Moved to standard kernel log print api.
> - Simplified the bus and clock calculation logic.
> - Removed debug features like dma tracker, frame stats, debugfs.
> - Merged v4l2 and vidc layer as vidc in driver.
> - Removed separate probe for context bank.
> - Use of_device_get_match_data to get platform data.
> - Replaced complex macros with inline functions.
> - Migrated venus to iris names.
> - Addressed many other comments received on [1].
>
> [1]: https://patchwork.kernel.org/project/linux-media/list/?series=770581
>
> Patch 1, adds dt binding for iris driver.
>
> Patches 2-4, are introducing vcodec folder and moving common APIs like fw
> load/unload, buffer size calcualtions, read/write to shared queues to common
> files which are being used by both venus and iris drivers.
> This also moves the venus driver folder to vcodec folder.
>
> Patch 5, adds the maintainers for iris driver.
>
> Patch 6-29, adds core iris driver and enable decoder.
>
> Patch 30-34, enable encoder functionality in iris driver.
>
Made a mistake while explaining the patches.
Please find the correct order below:
Patches 1-3, are introducing vcodec folder and moving common APIs like fw
load/unload, buffer size calcualtions, read/write to shared queues to
common files which are being used by both venus and iris drivers.
This also moves the venus driver folder to vcodec folder.
Patch 4, adds dt binding for iris driver.
Patch 5, adds the maintainers for iris driver.
Patch 6-29, adds core iris driver and enable decoder.
Patch 30-34, enable encoder functionality in iris driver.
Thanks,
Dikshita
> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> testVIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video0: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Compliance test for iris_driver device /dev/video1:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video1: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Dikshita Agarwal (27):
> media: introduce common helpers for video firmware handling
> media: introduce common helpers for queues handling
> media: introduce common helpers for buffer size calculation
> dt-bindings: media: Add sm8550 dt schema
> media: MAINTAINERS: Add Qualcomm Iris video accelerator driver
> media: iris: register video device to platform driver
> media: iris: initialize power resources
> media: iris: introduce state machine for iris core
> media: iris: initialize shared queues for host and firmware
> communication
> media: iris: add PIL functionality for video firmware
> media: iris: introduce packetization layer for creating HFI packets
> media: iris: add video processing unit(VPU) specific register handling
> media: iris: introduce platform specific capabilities for core and
> instance
> media: iris: add handling for interrupt service routine(ISR) invoked
> by hardware
> media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
> media: iris: implement vb2_ops queue setup
> media: iris: implement HFI to queue and release buffers
> media: iris: add video hardware internal buffer count and size
> calculation
> media: iris: implement internal buffer management
> media: iris: introduce instance states
> media: iris: implement vb2 streaming ops on capture and output planes
> media: iris: implement vb2 ops for buf_queue and firmware response
> media: iris: register video encoder device to platform driver
> media: iris: add platform specific instance capabilities for encoder
> media: iris: implement iris v4l2 ioctl ops supported by encoder
> media: iris: add vb2 streaming and buffer ops for encoder
> media: iris: add power management for encoder
>
> Vikash Garodia (7):
> media: iris: implement iris v4l2 file ops
> media: iris: introduce and implement iris vb2 mem ops
> media: iris: implement iris v4l2 ioctl ops supported by decoder
> media: iris: subscribe input and output properties to firmware
> media: iris: subscribe src change and handle firmware responses
> media: iris: add instance sub states and implement DRC and Drain
> sequence
> media: iris: implement power management
>
> .../bindings/media/qcom,sm8550-iris.yaml | 177 ++
> MAINTAINERS | 11 +
> drivers/media/platform/qcom/Kconfig | 3 +-
> drivers/media/platform/qcom/Makefile | 3 +-
> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++
> drivers/media/platform/qcom/vcodec/buffers.h | 15 +
> drivers/media/platform/qcom/vcodec/firmware.c | 147 ++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 +
> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++
> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +
> drivers/media/platform/qcom/vcodec/iris/Makefile | 26 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 386 +++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 833 +++++++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.h | 65 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 147 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 110 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 121 ++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 1782 ++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 71 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 1099 ++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 68 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 917 ++++++++++
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 47 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 729 ++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 118 ++
> .../platform/qcom/vcodec/iris/iris_hfi_response.c | 1097 ++++++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 106 ++
> .../media/platform/qcom/vcodec/iris/iris_power.c | 178 ++
> .../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 357 ++++
> .../media/platform/qcom/vcodec/iris/iris_state.c | 453 +++++
> .../media/platform/qcom/vcodec/iris/iris_state.h | 78 +
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 457 +++++
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 28 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1211 +++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 25 +
> .../media/platform/qcom/vcodec/iris/iris_venc.c | 948 +++++++++++
> .../media/platform/qcom/vcodec/iris/iris_venc.h | 27 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 1419 ++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 +
> .../platform/qcom/vcodec/iris/platform_common.c | 29 +
> .../platform/qcom/vcodec/iris/platform_common.h | 323 ++++
> .../platform/qcom/vcodec/iris/platform_sm8550.c | 1190 +++++++++++++
> .../media/platform/qcom/vcodec/iris/resources.c | 506 ++++++
> .../media/platform/qcom/vcodec/iris/resources.h | 44 +
> .../media/platform/qcom/vcodec/iris/vpu_common.c | 158 ++
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 52 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 568 +++++++
> .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 395 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 1469 ++++++++++++++++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 148 ++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 +
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 172 +-
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 2 +-
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 4 +-
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 10 +-
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 521 +-----
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ----
> drivers/media/platform/qcom/venus/firmware.h | 26 -
> 93 files changed, 19175 insertions(+), 989 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (75%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (90%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (98%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (94%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (73%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (34 preceding siblings ...)
2023-12-18 12:09 ` [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
@ 2023-12-18 14:36 ` Bryan O'Donoghue
2023-12-18 18:38 ` Dmitry Baryshkov
` (4 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Bryan O'Donoghue @ 2023-12-18 14:36 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 11:31, Dikshita Agarwal wrote:
> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
Good to see some code sharing, is there no way to facilitate the new
code in existing venus ?
I applied your patches to -stable and was happy to find they didn't
break venus on rb5, at least for the superficial test I did [1]. The
names you export here look odd though.
"Driver name" and "Cart Type" should be something like "qcom-iris" and
"Qualcomm Iris video decoder"
v4l2-compliance -d /dev/video15
v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
Compliance test for qcom-venus device /dev/video15:
Driver Info:
Driver name : qcom-venus
Card type : Qualcomm Venus video encoder
Bus info : platform:qcom-venus
Driver version : 6.7.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Encoder
[1]
https://file-examples.com/index.php/sample-video-files/sample-mp4-files/
ffplay -loglevel debug -codec:video h264_v4l2m2m -i sample-30s.mp4
> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder
Same comment on normalising names wrt to venus.
---
bod
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (35 preceding siblings ...)
2023-12-18 14:36 ` Bryan O'Donoghue
@ 2023-12-18 18:38 ` Dmitry Baryshkov
2023-12-19 12:10 ` Bryan O'Donoghue
2023-12-20 6:32 ` Vikash Garodia
2023-12-20 20:51 ` Nicolas Dufresne
` (3 subsequent siblings)
40 siblings, 2 replies; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-18 18:38 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 13:31, Dikshita Agarwal wrote:
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.
This doesn't answer one important point, you have been asked for v1.
What is the actual change point between Venus and Iris? What has been
changed so much that it demands a separate driver. This is the main
question for the cover letter, which has not been answered so far.
From what I see from you bindings, the hardware is pretty close to what
we see in the latest venus generations. I asssme that there was a change
in the vcodec inteface to the firmware and other similar changes. Could
you please point out, which parts of Venus driver do no longer work or
are not applicable for sm8550?
> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.
>
> Changes done since v1[1]:
> - Patches are created as logical split instead of file by file.
> - Extracted common functionality between venus and iris driver and placed them
> under common folder vcodec.
> - Flattened directory structure.
> - Restructured the code in a simplified layer architecture.
> - Implemented runtime PM, and dropped explicit power collapse thread in driver.
> - Moved to standard kernel log print api.
> - Simplified the bus and clock calculation logic.
> - Removed debug features like dma tracker, frame stats, debugfs.
> - Merged v4l2 and vidc layer as vidc in driver.
> - Removed separate probe for context bank.
> - Use of_device_get_match_data to get platform data.
> - Replaced complex macros with inline functions.
> - Migrated venus to iris names.
> - Addressed many other comments received on [1].
>
> [1]: https://patchwork.kernel.org/project/linux-media/list/?series=770581
>
> Patch 1, adds dt binding for iris driver.
>
> Patches 2-4, are introducing vcodec folder and moving common APIs like fw
> load/unload, buffer size calcualtions, read/write to shared queues to common
> files which are being used by both venus and iris drivers.
> This also moves the venus driver folder to vcodec folder.
>
> Patch 5, adds the maintainers for iris driver.
>
> Patch 6-29, adds core iris driver and enable decoder.
>
> Patch 30-34, enable encoder functionality in iris driver.
>
> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> testVIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video0: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Compliance test for iris_driver device /dev/video1:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video1: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Dikshita Agarwal (27):
> media: introduce common helpers for video firmware handling
> media: introduce common helpers for queues handling
> media: introduce common helpers for buffer size calculation
> dt-bindings: media: Add sm8550 dt schema
> media: MAINTAINERS: Add Qualcomm Iris video accelerator driver
> media: iris: register video device to platform driver
> media: iris: initialize power resources
> media: iris: introduce state machine for iris core
> media: iris: initialize shared queues for host and firmware
> communication
> media: iris: add PIL functionality for video firmware
> media: iris: introduce packetization layer for creating HFI packets
> media: iris: add video processing unit(VPU) specific register handling
> media: iris: introduce platform specific capabilities for core and
> instance
> media: iris: add handling for interrupt service routine(ISR) invoked
> by hardware
> media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
> media: iris: implement vb2_ops queue setup
> media: iris: implement HFI to queue and release buffers
> media: iris: add video hardware internal buffer count and size
> calculation
> media: iris: implement internal buffer management
> media: iris: introduce instance states
> media: iris: implement vb2 streaming ops on capture and output planes
> media: iris: implement vb2 ops for buf_queue and firmware response
> media: iris: register video encoder device to platform driver
> media: iris: add platform specific instance capabilities for encoder
> media: iris: implement iris v4l2 ioctl ops supported by encoder
> media: iris: add vb2 streaming and buffer ops for encoder
> media: iris: add power management for encoder
>
> Vikash Garodia (7):
> media: iris: implement iris v4l2 file ops
> media: iris: introduce and implement iris vb2 mem ops
> media: iris: implement iris v4l2 ioctl ops supported by decoder
> media: iris: subscribe input and output properties to firmware
> media: iris: subscribe src change and handle firmware responses
> media: iris: add instance sub states and implement DRC and Drain
> sequence
> media: iris: implement power management
>
> .../bindings/media/qcom,sm8550-iris.yaml | 177 ++
> MAINTAINERS | 11 +
> drivers/media/platform/qcom/Kconfig | 3 +-
> drivers/media/platform/qcom/Makefile | 3 +-
> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++
> drivers/media/platform/qcom/vcodec/buffers.h | 15 +
> drivers/media/platform/qcom/vcodec/firmware.c | 147 ++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 +
> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++
> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +
> drivers/media/platform/qcom/vcodec/iris/Makefile | 26 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 386 +++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 833 +++++++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.h | 65 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 147 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 110 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 121 ++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 1782 ++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 71 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 1099 ++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 68 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 917 ++++++++++
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 47 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 729 ++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 118 ++
> .../platform/qcom/vcodec/iris/iris_hfi_response.c | 1097 ++++++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 106 ++
> .../media/platform/qcom/vcodec/iris/iris_power.c | 178 ++
> .../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 357 ++++
> .../media/platform/qcom/vcodec/iris/iris_state.c | 453 +++++
> .../media/platform/qcom/vcodec/iris/iris_state.h | 78 +
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 457 +++++
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 28 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1211 +++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 25 +
> .../media/platform/qcom/vcodec/iris/iris_venc.c | 948 +++++++++++
> .../media/platform/qcom/vcodec/iris/iris_venc.h | 27 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 1419 ++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 +
> .../platform/qcom/vcodec/iris/platform_common.c | 29 +
> .../platform/qcom/vcodec/iris/platform_common.h | 323 ++++
> .../platform/qcom/vcodec/iris/platform_sm8550.c | 1190 +++++++++++++
> .../media/platform/qcom/vcodec/iris/resources.c | 506 ++++++
> .../media/platform/qcom/vcodec/iris/resources.h | 44 +
> .../media/platform/qcom/vcodec/iris/vpu_common.c | 158 ++
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 52 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 568 +++++++
> .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 395 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 1469 ++++++++++++++++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 148 ++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 +
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 172 +-
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 2 +-
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 4 +-
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 10 +-
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 521 +-----
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ----
> drivers/media/platform/qcom/venus/firmware.h | 26 -
> 93 files changed, 19175 insertions(+), 989 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (75%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (90%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (98%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (94%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (73%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 18:38 ` Dmitry Baryshkov
@ 2023-12-19 12:10 ` Bryan O'Donoghue
2023-12-20 6:32 ` Vikash Garodia
1 sibling, 0 replies; 102+ messages in thread
From: Bryan O'Donoghue @ 2023-12-19 12:10 UTC (permalink / raw)
To: Dmitry Baryshkov, Dikshita Agarwal, linux-media, linux-kernel,
stanimir.k.varbanov, quic_vgarodia, agross, andersson,
konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 18/12/2023 18:38, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> This patch series introduces support for Qualcomm new video acceleration
>> hardware architecture, used for video stream decoding/encoding. This
>> driver
>> is based on new communication protocol between video hardware and
>> application
>> processor.
>
> This doesn't answer one important point, you have been asked for v1.
> What is the actual change point between Venus and Iris? What has been
> changed so much that it demands a separate driver. This is the main
> question for the cover letter, which has not been answered so far.
>
> From what I see from you bindings, the hardware is pretty close to what
> we see in the latest venus generations. I asssme that there was a change
> in the vcodec inteface to the firmware and other similar changes. Could
> you please point out, which parts of Venus driver do no longer work or
> are not applicable for sm8550?
I'd like to hear this detail too.
Grazing on some of the patches here I saw were we adding vb2 buffer ops,
which is already supported by venus.
Similar question on booting cores and doing a busy/wait instead of a
rendezvous based on an IRQ.
I feel there's a version of this series that can probably unify the
codebases but, am open to being told how that is incorrect.
---
bod
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 18:38 ` Dmitry Baryshkov
2023-12-19 12:10 ` Bryan O'Donoghue
@ 2023-12-20 6:32 ` Vikash Garodia
2023-12-20 7:37 ` Dmitry Baryshkov
2023-12-20 8:15 ` Krzysztof Kozlowski
1 sibling, 2 replies; 102+ messages in thread
From: Vikash Garodia @ 2023-12-20 6:32 UTC (permalink / raw)
To: Dmitry Baryshkov, Dikshita Agarwal, linux-media, linux-kernel,
stanimir.k.varbanov, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
Hi Dmitry,
On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> This patch series introduces support for Qualcomm new video acceleration
>> hardware architecture, used for video stream decoding/encoding. This driver
>> is based on new communication protocol between video hardware and application
>> processor.
>
> This doesn't answer one important point, you have been asked for v1. What is the
> actual change point between Venus and Iris? What has been changed so much that
> it demands a separate driver. This is the main question for the cover letter,
> which has not been answered so far.
>
> From what I see from you bindings, the hardware is pretty close to what we see
> in the latest venus generations. I asssme that there was a change in the vcodec
> inteface to the firmware and other similar changes. Could you please point out,
> which parts of Venus driver do no longer work or are not applicable for sm8550
The motivation behind having a separate IRIS driver was discussed earlier in [1]
In the same discussion, it was ellaborated on how the impact would be with
change in the new firmware interface and other video layers in the driver. I can
add this in cover letter in the next revision.
We see some duplication of code and to handle the same, the series brings in a
common code reusability between iris and venus. Aligning the common peices of
venus and iris will be a work in progress, once we land the base driver for iris.
Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
venus as the changes are too interleaved to absorb in venus driver. And there is
significant interest in community to start validating video driver on sm8550 or
x1e80100.
[1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
Regards,
Vikash
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 6:32 ` Vikash Garodia
@ 2023-12-20 7:37 ` Dmitry Baryshkov
2023-12-20 8:14 ` Vikash Garodia
2023-12-20 8:15 ` Krzysztof Kozlowski
1 sibling, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-20 7:37 UTC (permalink / raw)
To: Vikash Garodia
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm, quic_abhinavk
On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>
> Hi Dmitry,
>
> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> > On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >> This patch series introduces support for Qualcomm new video acceleration
> >> hardware architecture, used for video stream decoding/encoding. This driver
> >> is based on new communication protocol between video hardware and application
> >> processor.
> >
> > This doesn't answer one important point, you have been asked for v1. What is the
> > actual change point between Venus and Iris? What has been changed so much that
> > it demands a separate driver. This is the main question for the cover letter,
> > which has not been answered so far.
> >
> > From what I see from you bindings, the hardware is pretty close to what we see
> > in the latest venus generations. I asssme that there was a change in the vcodec
> > inteface to the firmware and other similar changes. Could you please point out,
> > which parts of Venus driver do no longer work or are not applicable for sm8550
>
> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> In the same discussion, it was ellaborated on how the impact would be with
> change in the new firmware interface and other video layers in the driver. I can
> add this in cover letter in the next revision.
Ok. So the changes cover the HFI interface. Is that correct?
> We see some duplication of code and to handle the same, the series brings in a
> common code reusability between iris and venus. Aligning the common peices of
> venus and iris will be a work in progress, once we land the base driver for iris.
This is not how it usually works. Especially not with the patches you
have posted.
I have the following suggestion how this story can continue:
You can _start_ by reworking venus driver, separating the HFI /
firmware / etc interface to an internal interface in the driver. Then
implement Iris as a plug in for that interface. I might be mistaken
here, but I think this is the way how this can be beneficial for both
the video en/decoding on both old and new platforms.
Short rationale:
The venus driver has a history of supported platforms. There is
already some kind of buffer management in place. Both developers and
testers have spent their effort on finding issues there. Sending new
driver means that we have to spend the same amount of efforts on this.
Moreover, even from the porter point of view. You are creating new
bindings for the new hardware. Which do not follow the
venus-common.yaml. And they do not follow the defined bindings for the
recent venus platforms. Which means that as a developer I have to care
about two different ways to describe nearly the same hardware.
> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> venus as the changes are too interleaved to absorb in venus driver. And there is
> significant interest in community to start validating video driver on sm8550 or
> x1e80100.
>
> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
>
> Regards,
> Vikash
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 7:37 ` Dmitry Baryshkov
@ 2023-12-20 8:14 ` Vikash Garodia
2023-12-20 8:39 ` Dmitry Baryshkov
0 siblings, 1 reply; 102+ messages in thread
From: Vikash Garodia @ 2023-12-20 8:14 UTC (permalink / raw)
To: Dmitry Baryshkov
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm, quic_abhinavk
Hi,
On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>>
>> Hi Dmitry,
>>
>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>> This patch series introduces support for Qualcomm new video acceleration
>>>> hardware architecture, used for video stream decoding/encoding. This driver
>>>> is based on new communication protocol between video hardware and application
>>>> processor.
>>>
>>> This doesn't answer one important point, you have been asked for v1. What is the
>>> actual change point between Venus and Iris? What has been changed so much that
>>> it demands a separate driver. This is the main question for the cover letter,
>>> which has not been answered so far.
>>>
>>> From what I see from you bindings, the hardware is pretty close to what we see
>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>> inteface to the firmware and other similar changes. Could you please point out,
>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>
>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>> In the same discussion, it was ellaborated on how the impact would be with
>> change in the new firmware interface and other video layers in the driver. I can
>> add this in cover letter in the next revision.
>
> Ok. So the changes cover the HFI interface. Is that correct?
Change wise, yes.
>> We see some duplication of code and to handle the same, the series brings in a
>> common code reusability between iris and venus. Aligning the common peices of
>> venus and iris will be a work in progress, once we land the base driver for iris.
>
> This is not how it usually works. Especially not with the patches you
> have posted.
>
> I have the following suggestion how this story can continue:
> You can _start_ by reworking venus driver, separating the HFI /
> firmware / etc interface to an internal interface in the driver. Then
> implement Iris as a plug in for that interface. I might be mistaken
> here, but I think this is the way how this can be beneficial for both
> the video en/decoding on both old and new platforms.
HFI/firmware interface is already confined to HFI layer in the existing venus
driver. We explained in the previous discussion [1], on how the HFI change
impacts the other layers by taking example of a DRC usecase. Please have a look
considering the usecase and the impact it brings to other layers in the driver.
[1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
> Short rationale:
> The venus driver has a history of supported platforms. There is
> already some kind of buffer management in place. Both developers and
> testers have spent their effort on finding issues there. Sending new
> driver means that we have to spend the same amount of efforts on this.
> Moreover, even from the porter point of view. You are creating new
> bindings for the new hardware. Which do not follow the
> venus-common.yaml. And they do not follow the defined bindings for the
> recent venus platforms. Which means that as a developer I have to care
> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>> venus as the changes are too interleaved to absorb in venus driver. And there is
>> significant interest in community to start validating video driver on sm8550 or
>> x1e80100.
>>
>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
>>
>> Regards,
>> Vikash
>
>
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 8:14 ` Vikash Garodia
@ 2023-12-20 8:39 ` Dmitry Baryshkov
2023-12-20 8:53 ` Vikash Garodia
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-20 8:39 UTC (permalink / raw)
To: Vikash Garodia
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm, quic_abhinavk
On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>
> Hi,
>
> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
> >>
> >> Hi Dmitry,
> >>
> >> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> >>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>> This patch series introduces support for Qualcomm new video acceleration
> >>>> hardware architecture, used for video stream decoding/encoding. This driver
> >>>> is based on new communication protocol between video hardware and application
> >>>> processor.
> >>>
> >>> This doesn't answer one important point, you have been asked for v1. What is the
> >>> actual change point between Venus and Iris? What has been changed so much that
> >>> it demands a separate driver. This is the main question for the cover letter,
> >>> which has not been answered so far.
> >>>
> >>> From what I see from you bindings, the hardware is pretty close to what we see
> >>> in the latest venus generations. I asssme that there was a change in the vcodec
> >>> inteface to the firmware and other similar changes. Could you please point out,
> >>> which parts of Venus driver do no longer work or are not applicable for sm8550
> >>
> >> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> >> In the same discussion, it was ellaborated on how the impact would be with
> >> change in the new firmware interface and other video layers in the driver. I can
> >> add this in cover letter in the next revision.
> >
> > Ok. So the changes cover the HFI interface. Is that correct?
> Change wise, yes.
>
> >> We see some duplication of code and to handle the same, the series brings in a
> >> common code reusability between iris and venus. Aligning the common peices of
> >> venus and iris will be a work in progress, once we land the base driver for iris.
> >
> > This is not how it usually works. Especially not with the patches you
> > have posted.
> >
> > I have the following suggestion how this story can continue:
> > You can _start_ by reworking venus driver, separating the HFI /
> > firmware / etc interface to an internal interface in the driver. Then
> > implement Iris as a plug in for that interface. I might be mistaken
> > here, but I think this is the way how this can be beneficial for both
> > the video en/decoding on both old and new platforms.
>
> HFI/firmware interface is already confined to HFI layer in the existing venus
> driver. We explained in the previous discussion [1], on how the HFI change
> impacts the other layers by taking example of a DRC usecase. Please have a look
> considering the usecase and the impact it brings to other layers in the driver.
I have looked at it. And I still see huge change in the interface
side, but it doesn't tell me about the hardware changes.
Have you evaluated the other opportunity?
To have a common platform interface and firmware-specific backend?
You have already done a part of it, but from a different perspective.
You have tried to move common code out of the driver. Instead we are
asking you to do a different thing. Move non-common code within the
driver. Then add your code on top of that.
>
> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
> > Short rationale:
> > The venus driver has a history of supported platforms. There is
> > already some kind of buffer management in place. Both developers and
> > testers have spent their effort on finding issues there. Sending new
> > driver means that we have to spend the same amount of efforts on this.
> > Moreover, even from the porter point of view. You are creating new
> > bindings for the new hardware. Which do not follow the
> > venus-common.yaml. And they do not follow the defined bindings for the
> > recent venus platforms. Which means that as a developer I have to care
> > about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> >> venus as the changes are too interleaved to absorb in venus driver. And there is
> >> significant interest in community to start validating video driver on sm8550 or
> >> x1e80100.
> >>
> >> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
> >>
> >> Regards,
> >> Vikash
> >
> >
> >
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 8:39 ` Dmitry Baryshkov
@ 2023-12-20 8:53 ` Vikash Garodia
2023-12-20 9:52 ` Dmitry Baryshkov
0 siblings, 1 reply; 102+ messages in thread
From: Vikash Garodia @ 2023-12-20 8:53 UTC (permalink / raw)
To: Dmitry Baryshkov
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm, quic_abhinavk
On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>>
>> Hi,
>>
>> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
>>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>>>>
>>>> Hi Dmitry,
>>>>
>>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>>>> This patch series introduces support for Qualcomm new video acceleration
>>>>>> hardware architecture, used for video stream decoding/encoding. This driver
>>>>>> is based on new communication protocol between video hardware and application
>>>>>> processor.
>>>>>
>>>>> This doesn't answer one important point, you have been asked for v1. What is the
>>>>> actual change point between Venus and Iris? What has been changed so much that
>>>>> it demands a separate driver. This is the main question for the cover letter,
>>>>> which has not been answered so far.
>>>>>
>>>>> From what I see from you bindings, the hardware is pretty close to what we see
>>>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>>>> inteface to the firmware and other similar changes. Could you please point out,
>>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>>>
>>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>>>> In the same discussion, it was ellaborated on how the impact would be with
>>>> change in the new firmware interface and other video layers in the driver. I can
>>>> add this in cover letter in the next revision.
>>>
>>> Ok. So the changes cover the HFI interface. Is that correct?
>> Change wise, yes.
>>
>>>> We see some duplication of code and to handle the same, the series brings in a
>>>> common code reusability between iris and venus. Aligning the common peices of
>>>> venus and iris will be a work in progress, once we land the base driver for iris.
>>>
>>> This is not how it usually works. Especially not with the patches you
>>> have posted.
>>>
>>> I have the following suggestion how this story can continue:
>>> You can _start_ by reworking venus driver, separating the HFI /
>>> firmware / etc interface to an internal interface in the driver. Then
>>> implement Iris as a plug in for that interface. I might be mistaken
>>> here, but I think this is the way how this can be beneficial for both
>>> the video en/decoding on both old and new platforms.
>>
>> HFI/firmware interface is already confined to HFI layer in the existing venus
>> driver. We explained in the previous discussion [1], on how the HFI change
>> impacts the other layers by taking example of a DRC usecase. Please have a look
>> considering the usecase and the impact it brings to other layers in the driver.
>
> I have looked at it. And I still see huge change in the interface
> side, but it doesn't tell me about the hardware changes.
I hope you noticed how the common layers like decoder, response, state layers
are impacted in handling one of usecase. Now add that to all the different
scenarios like seek, drain, DRC during seek, DRC during drain, etc.
> Have you evaluated the other opportunity?
>
> To have a common platform interface and firmware-specific backend?
>
> You have already done a part of it, but from a different perspective.
> You have tried to move common code out of the driver. Instead we are
> asking you to do a different thing. Move non-common code within the
> driver. Then add your code on top of that.
For common platform - yes, we are bringing in common stuff like PIL.
Other than that, abstraction to firmware interface is not that confined to one
layer. It spreads over decoder/encoder/common layers. Now when majority of the
layers/code is different, we planned to make it in parallel to venus and have a
common layer having common things to both iris and venus.
>>
>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
>>> Short rationale:
>>> The venus driver has a history of supported platforms. There is
>>> already some kind of buffer management in place. Both developers and
>>> testers have spent their effort on finding issues there. Sending new
>>> driver means that we have to spend the same amount of efforts on this.
>>> Moreover, even from the porter point of view. You are creating new
>>> bindings for the new hardware. Which do not follow the
>>> venus-common.yaml. And they do not follow the defined bindings for the
>>> recent venus platforms. Which means that as a developer I have to care
>>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>>>> venus as the changes are too interleaved to absorb in venus driver. And there is
>>>> significant interest in community to start validating video driver on sm8550 or
>>>> x1e80100.
>>>>
>>>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
>>>>
>>>> Regards,
>>>> Vikash
>>>
>>>
>>>
>
>
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 8:53 ` Vikash Garodia
@ 2023-12-20 9:52 ` Dmitry Baryshkov
2023-12-20 18:55 ` Abhinav Kumar
0 siblings, 1 reply; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-20 9:52 UTC (permalink / raw)
To: Vikash Garodia
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm, quic_abhinavk
On Wed, 20 Dec 2023 at 10:53, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>
> On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
> >>
> >> Hi,
> >>
> >> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> >>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
> >>>>
> >>>> Hi Dmitry,
> >>>>
> >>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> >>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>>>> This patch series introduces support for Qualcomm new video acceleration
> >>>>>> hardware architecture, used for video stream decoding/encoding. This driver
> >>>>>> is based on new communication protocol between video hardware and application
> >>>>>> processor.
> >>>>>
> >>>>> This doesn't answer one important point, you have been asked for v1. What is the
> >>>>> actual change point between Venus and Iris? What has been changed so much that
> >>>>> it demands a separate driver. This is the main question for the cover letter,
> >>>>> which has not been answered so far.
> >>>>>
> >>>>> From what I see from you bindings, the hardware is pretty close to what we see
> >>>>> in the latest venus generations. I asssme that there was a change in the vcodec
> >>>>> inteface to the firmware and other similar changes. Could you please point out,
> >>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
> >>>>
> >>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> >>>> In the same discussion, it was ellaborated on how the impact would be with
> >>>> change in the new firmware interface and other video layers in the driver. I can
> >>>> add this in cover letter in the next revision.
> >>>
> >>> Ok. So the changes cover the HFI interface. Is that correct?
> >> Change wise, yes.
> >>
> >>>> We see some duplication of code and to handle the same, the series brings in a
> >>>> common code reusability between iris and venus. Aligning the common peices of
> >>>> venus and iris will be a work in progress, once we land the base driver for iris.
> >>>
> >>> This is not how it usually works. Especially not with the patches you
> >>> have posted.
> >>>
> >>> I have the following suggestion how this story can continue:
> >>> You can _start_ by reworking venus driver, separating the HFI /
> >>> firmware / etc interface to an internal interface in the driver. Then
> >>> implement Iris as a plug in for that interface. I might be mistaken
> >>> here, but I think this is the way how this can be beneficial for both
> >>> the video en/decoding on both old and new platforms.
> >>
> >> HFI/firmware interface is already confined to HFI layer in the existing venus
> >> driver. We explained in the previous discussion [1], on how the HFI change
> >> impacts the other layers by taking example of a DRC usecase. Please have a look
> >> considering the usecase and the impact it brings to other layers in the driver.
> >
> > I have looked at it. And I still see huge change in the interface
> > side, but it doesn't tell me about the hardware changes.
>
> I hope you noticed how the common layers like decoder, response, state layers
> are impacted in handling one of usecase. Now add that to all the different
> scenarios like seek, drain, DRC during seek, DRC during drain, etc.
Yes, for sure.
>
> > Have you evaluated the other opportunity?
> >
> > To have a common platform interface and firmware-specific backend?
> >
> > You have already done a part of it, but from a different perspective.
> > You have tried to move common code out of the driver. Instead we are
> > asking you to do a different thing. Move non-common code within the
> > driver. Then add your code on top of that.
>
> For common platform - yes, we are bringing in common stuff like PIL.
> Other than that, abstraction to firmware interface is not that confined to one
> layer. It spreads over decoder/encoder/common layers. Now when majority of the
> layers/code is different, we planned to make it in parallel to venus and have a
> common layer having common things to both iris and venus.
My suggestion still holds. Start with this common platform code.
Rather than arguing back and forth, could you please perform an
experiment on the current venus driver and move firmware interface to
subdirs, leaving just the platform / PIL / formats / etc in place?
This will at least allow us to determine whether it is a feasible
concept or not.
>
> >>
> >> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
> >>> Short rationale:
> >>> The venus driver has a history of supported platforms. There is
> >>> already some kind of buffer management in place. Both developers and
> >>> testers have spent their effort on finding issues there. Sending new
> >>> driver means that we have to spend the same amount of efforts on this.
> >>> Moreover, even from the porter point of view. You are creating new
> >>> bindings for the new hardware. Which do not follow the
> >>> venus-common.yaml. And they do not follow the defined bindings for the
> >>> recent venus platforms. Which means that as a developer I have to care
> >>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> >>>> venus as the changes are too interleaved to absorb in venus driver. And there is
> >>>> significant interest in community to start validating video driver on sm8550 or
> >>>> x1e80100.
> >>>>
> >>>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
> >>>>
> >>>> Regards,
> >>>> Vikash
> >>>
> >>>
> >>>
> >
> >
> >
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 9:52 ` Dmitry Baryshkov
@ 2023-12-20 18:55 ` Abhinav Kumar
2023-12-20 21:24 ` Dmitry Baryshkov
0 siblings, 1 reply; 102+ messages in thread
From: Abhinav Kumar @ 2023-12-20 18:55 UTC (permalink / raw)
To: Dmitry Baryshkov, Vikash Garodia
Cc: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
agross, andersson, konrad.dybcio, mchehab, bryan.odonoghue,
linux-arm-msm
Hi Dmitry and Krzysztof
On 12/20/2023 1:52 AM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 10:53, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>>
>> On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
>>> On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
>>>>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
>>>>>>
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
>>>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>>>>>> This patch series introduces support for Qualcomm new video acceleration
>>>>>>>> hardware architecture, used for video stream decoding/encoding. This driver
>>>>>>>> is based on new communication protocol between video hardware and application
>>>>>>>> processor.
>>>>>>>
>>>>>>> This doesn't answer one important point, you have been asked for v1. What is the
>>>>>>> actual change point between Venus and Iris? What has been changed so much that
>>>>>>> it demands a separate driver. This is the main question for the cover letter,
>>>>>>> which has not been answered so far.
>>>>>>>
>>>>>>> From what I see from you bindings, the hardware is pretty close to what we see
>>>>>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>>>>>> inteface to the firmware and other similar changes. Could you please point out,
>>>>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>>>>>
>>>>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>>>>>> In the same discussion, it was ellaborated on how the impact would be with
>>>>>> change in the new firmware interface and other video layers in the driver. I can
>>>>>> add this in cover letter in the next revision.
>>>>>
>>>>> Ok. So the changes cover the HFI interface. Is that correct?
>>>> Change wise, yes.
>>>>
>>>>>> We see some duplication of code and to handle the same, the series brings in a
>>>>>> common code reusability between iris and venus. Aligning the common peices of
>>>>>> venus and iris will be a work in progress, once we land the base driver for iris.
>>>>>
>>>>> This is not how it usually works. Especially not with the patches you
>>>>> have posted.
>>>>>
>>>>> I have the following suggestion how this story can continue:
>>>>> You can _start_ by reworking venus driver, separating the HFI /
>>>>> firmware / etc interface to an internal interface in the driver. Then
>>>>> implement Iris as a plug in for that interface. I might be mistaken
>>>>> here, but I think this is the way how this can be beneficial for both
>>>>> the video en/decoding on both old and new platforms.
>>>>
>>>> HFI/firmware interface is already confined to HFI layer in the existing venus
>>>> driver. We explained in the previous discussion [1], on how the HFI change
>>>> impacts the other layers by taking example of a DRC usecase. Please have a look
>>>> considering the usecase and the impact it brings to other layers in the driver.
>>>
>>> I have looked at it. And I still see huge change in the interface
>>> side, but it doesn't tell me about the hardware changes.
>>
>> I hope you noticed how the common layers like decoder, response, state layers
>> are impacted in handling one of usecase. Now add that to all the different
>> scenarios like seek, drain, DRC during seek, DRC during drain, etc.
>
> Yes, for sure.
>
>>
>>> Have you evaluated the other opportunity?
>>>
>>> To have a common platform interface and firmware-specific backend?
>>>
>>> You have already done a part of it, but from a different perspective.
>>> You have tried to move common code out of the driver. Instead we are
>>> asking you to do a different thing. Move non-common code within the
>>> driver. Then add your code on top of that.
>>
>> For common platform - yes, we are bringing in common stuff like PIL.
>> Other than that, abstraction to firmware interface is not that confined to one
>> layer. It spreads over decoder/encoder/common layers. Now when majority of the
>> layers/code is different, we planned to make it in parallel to venus and have a
>> common layer having common things to both iris and venus.
>
> My suggestion still holds. Start with this common platform code.
> Rather than arguing back and forth, could you please perform an
> experiment on the current venus driver and move firmware interface to
> subdirs, leaving just the platform / PIL / formats / etc in place?
> This will at least allow us to determine whether it is a feasible
> concept or not.
>
Your comments are valid. The platform driver piece and some other pieces
still are common between venus and iris despite this initial effort of
moving common pieces out. I have also seen this from whatever I have
reviewed.
Video team also acknowledges this part and internally I think they did
evaluate this option and their feedback was, the more and more they
changed, they were touching pretty much every file of venus.
The missing piece i think in all this discussion is that in addition to
the forward channel, the reverse channel of HFI, based on which the rest
of the video state machine changes should also be considered.
So even with respect to the code layout, its not just the forward
communication but the backwards communication of fw--->hfi--->codec is
becoming a challenge as the venus layers seem to only work with the hfi
of venus.
For adding support for the new HFI events/communication, it was getting
harder to extend venus.
What I certainly acknowledge is now with iris whether this can be dealt
with better for future chipsets to avoid another re-write for another
HFI as that will be too much of maintenance cost.
I will let the video team comment on this part.
Thanks
Abhinav
>>
>>>>
>>>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
>>>>> Short rationale:
>>>>> The venus driver has a history of supported platforms. There is
>>>>> already some kind of buffer management in place. Both developers and
>>>>> testers have spent their effort on finding issues there. Sending new
>>>>> driver means that we have to spend the same amount of efforts on this.
>>>>> Moreover, even from the porter point of view. You are creating new
>>>>> bindings for the new hardware. Which do not follow the
>>>>> venus-common.yaml. And they do not follow the defined bindings for the
>>>>> recent venus platforms. Which means that as a developer I have to care
>>>>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>>>>>> venus as the changes are too interleaved to absorb in venus driver. And there is
>>>>>> significant interest in community to start validating video driver on sm8550 or
>>>>>> x1e80100.
>>>>>>
>>>>>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
>>>>>>
>>>>>> Regards,
>>>>>> Vikash
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>
>
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 18:55 ` Abhinav Kumar
@ 2023-12-20 21:24 ` Dmitry Baryshkov
0 siblings, 0 replies; 102+ messages in thread
From: Dmitry Baryshkov @ 2023-12-20 21:24 UTC (permalink / raw)
To: Abhinav Kumar
Cc: Vikash Garodia, Dikshita Agarwal, linux-media, linux-kernel,
stanimir.k.varbanov, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue, linux-arm-msm
Hi Abhinav,
On Wed, 20 Dec 2023 at 20:55, Abhinav Kumar <quic_abhinavk@quicinc.com> wrote:
>
> Hi Dmitry and Krzysztof
>
> On 12/20/2023 1:52 AM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 10:53, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
> >>
> >> On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
> >>> On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> >>>>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <quic_vgarodia@quicinc.com> wrote:
> >>>>>>
> >>>>>> Hi Dmitry,
> >>>>>>
> >>>>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> >>>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>>>>>> This patch series introduces support for Qualcomm new video acceleration
> >>>>>>>> hardware architecture, used for video stream decoding/encoding. This driver
> >>>>>>>> is based on new communication protocol between video hardware and application
> >>>>>>>> processor.
> >>>>>>>
> >>>>>>> This doesn't answer one important point, you have been asked for v1. What is the
> >>>>>>> actual change point between Venus and Iris? What has been changed so much that
> >>>>>>> it demands a separate driver. This is the main question for the cover letter,
> >>>>>>> which has not been answered so far.
> >>>>>>>
> >>>>>>> From what I see from you bindings, the hardware is pretty close to what we see
> >>>>>>> in the latest venus generations. I asssme that there was a change in the vcodec
> >>>>>>> inteface to the firmware and other similar changes. Could you please point out,
> >>>>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
> >>>>>>
> >>>>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> >>>>>> In the same discussion, it was ellaborated on how the impact would be with
> >>>>>> change in the new firmware interface and other video layers in the driver. I can
> >>>>>> add this in cover letter in the next revision.
> >>>>>
> >>>>> Ok. So the changes cover the HFI interface. Is that correct?
> >>>> Change wise, yes.
> >>>>
> >>>>>> We see some duplication of code and to handle the same, the series brings in a
> >>>>>> common code reusability between iris and venus. Aligning the common peices of
> >>>>>> venus and iris will be a work in progress, once we land the base driver for iris.
> >>>>>
> >>>>> This is not how it usually works. Especially not with the patches you
> >>>>> have posted.
> >>>>>
> >>>>> I have the following suggestion how this story can continue:
> >>>>> You can _start_ by reworking venus driver, separating the HFI /
> >>>>> firmware / etc interface to an internal interface in the driver. Then
> >>>>> implement Iris as a plug in for that interface. I might be mistaken
> >>>>> here, but I think this is the way how this can be beneficial for both
> >>>>> the video en/decoding on both old and new platforms.
> >>>>
> >>>> HFI/firmware interface is already confined to HFI layer in the existing venus
> >>>> driver. We explained in the previous discussion [1], on how the HFI change
> >>>> impacts the other layers by taking example of a DRC usecase. Please have a look
> >>>> considering the usecase and the impact it brings to other layers in the driver.
> >>>
> >>> I have looked at it. And I still see huge change in the interface
> >>> side, but it doesn't tell me about the hardware changes.
> >>
> >> I hope you noticed how the common layers like decoder, response, state layers
> >> are impacted in handling one of usecase. Now add that to all the different
> >> scenarios like seek, drain, DRC during seek, DRC during drain, etc.
> >
> > Yes, for sure.
> >
> >>
> >>> Have you evaluated the other opportunity?
> >>>
> >>> To have a common platform interface and firmware-specific backend?
> >>>
> >>> You have already done a part of it, but from a different perspective.
> >>> You have tried to move common code out of the driver. Instead we are
> >>> asking you to do a different thing. Move non-common code within the
> >>> driver. Then add your code on top of that.
> >>
> >> For common platform - yes, we are bringing in common stuff like PIL.
> >> Other than that, abstraction to firmware interface is not that confined to one
> >> layer. It spreads over decoder/encoder/common layers. Now when majority of the
> >> layers/code is different, we planned to make it in parallel to venus and have a
> >> common layer having common things to both iris and venus.
> >
> > My suggestion still holds. Start with this common platform code.
> > Rather than arguing back and forth, could you please perform an
> > experiment on the current venus driver and move firmware interface to
> > subdirs, leaving just the platform / PIL / formats / etc in place?
> > This will at least allow us to determine whether it is a feasible
> > concept or not.
> >
>
> Your comments are valid. The platform driver piece and some other pieces
> still are common between venus and iris despite this initial effort of
> moving common pieces out. I have also seen this from whatever I have
> reviewed.
>
> Video team also acknowledges this part and internally I think they did
> evaluate this option and their feedback was, the more and more they
> changed, they were touching pretty much every file of venus.
This is fine from my POV. Splitting the the common functionality also
touched a significant part of the venus driver in a pretty bad way.
> The missing piece i think in all this discussion is that in addition to
> the forward channel, the reverse channel of HFI, based on which the rest
> of the video state machine changes should also be considered.
>
> So even with respect to the code layout, its not just the forward
> communication but the backwards communication of fw--->hfi--->codec is
> becoming a challenge as the venus layers seem to only work with the hfi
> of venus.
Again, this is a question of the platform backend part. Nobody
questions that newer platforms have their own driver interface.
We are not asking your team to add if(new) { foo; } else { bar; }
conditions all over the code. Instead we have been asking to split all
platform specifics to the 'backend'. Compare this with mdp4, mdp5 and
dpu backends of the single drm/msm frontend. Or with a similar
approach found in other DRM drivers. Adding new driver is frequently
unjustified, instead the existing driver gets extended to support
different generations.
> For adding support for the new HFI events/communication, it was getting
> harder to extend venus.
>
> What I certainly acknowledge is now with iris whether this can be dealt
> with better for future chipsets to avoid another re-write for another
> HFI as that will be too much of maintenance cost.
Another approach, which might also be fine, if that looks better from
your point of view. The current venus driver might have issues with
the internal interfaces. Or it might be not suitable for the backends.
In this case, can we please get older platforms also supported by the
iris driver? This will be a very simple case then: the old deprecated
driver, being phased out and removed, and a new one, which is being
phased in.
The problem is very simple from my side. I do not want to end up in a
situation where we have to change platform code for both drivers if
there is any common issue. Neither would I like for venus and iris
bindings to diverge more than necessary. And as you have seen from my
comments, the iris bindings already do not follow the venus example.
> I will let the video team comment on this part.
>
> Thanks
>
> Abhinav
>
>
>
>
>
> >>
> >>>>
> >>>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
> >>>>> Short rationale:
> >>>>> The venus driver has a history of supported platforms. There is
> >>>>> already some kind of buffer management in place. Both developers and
> >>>>> testers have spent their effort on finding issues there. Sending new
> >>>>> driver means that we have to spend the same amount of efforts on this.
> >>>>> Moreover, even from the porter point of view. You are creating new
> >>>>> bindings for the new hardware. Which do not follow the
> >>>>> venus-common.yaml. And they do not follow the defined bindings for the
> >>>>> recent venus platforms. Which means that as a developer I have to care
> >>>>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> >>>>>> venus as the changes are too interleaved to absorb in venus driver. And there is
> >>>>>> significant interest in community to start validating video driver on sm8550 or
> >>>>>> x1e80100.
> >>>>>>
> >>>>>> [1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
> >>>>>>
> >>>>>> Regards,
> >>>>>> Vikash
--
With best wishes
Dmitry
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 6:32 ` Vikash Garodia
2023-12-20 7:37 ` Dmitry Baryshkov
@ 2023-12-20 8:15 ` Krzysztof Kozlowski
2023-12-20 8:32 ` Vikash Garodia
1 sibling, 1 reply; 102+ messages in thread
From: Krzysztof Kozlowski @ 2023-12-20 8:15 UTC (permalink / raw)
To: Vikash Garodia, Dmitry Baryshkov, Dikshita Agarwal, linux-media,
linux-kernel, stanimir.k.varbanov, agross, andersson,
konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
On 20/12/2023 07:32, Vikash Garodia wrote:
>> From what I see from you bindings, the hardware is pretty close to what we see
>> in the latest venus generations. I asssme that there was a change in the vcodec
>> inteface to the firmware and other similar changes. Could you please point out,
>> which parts of Venus driver do no longer work or are not applicable for sm8550
>
> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> In the same discussion, it was ellaborated on how the impact would be with
> change in the new firmware interface and other video layers in the driver. I can
> add this in cover letter in the next revision.
>
> We see some duplication of code and to handle the same, the series brings in a
> common code reusability between iris and venus. Aligning the common peices of
> venus and iris will be a work in progress, once we land the base driver for iris.
>
> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
If you want it to get merged, then create such plan, please.
> venus as the changes are too interleaved to absorb in venus driver. And there is
> significant interest in community to start validating video driver on sm8550 or
> x1e80100.
Community does not want duplicated drivers leading to maintenance costs.
Your approach to this is not how upstreaming process works.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-20 8:15 ` Krzysztof Kozlowski
@ 2023-12-20 8:32 ` Vikash Garodia
0 siblings, 0 replies; 102+ messages in thread
From: Vikash Garodia @ 2023-12-20 8:32 UTC (permalink / raw)
To: Krzysztof Kozlowski, Dmitry Baryshkov, Dikshita Agarwal,
linux-media, linux-kernel, stanimir.k.varbanov, agross, andersson,
konrad.dybcio, mchehab, bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
Hi Krzysztof,
On 12/20/2023 1:45 PM, Krzysztof Kozlowski wrote:
> On 20/12/2023 07:32, Vikash Garodia wrote:
>>> From what I see from you bindings, the hardware is pretty close to what we see
>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>> inteface to the firmware and other similar changes. Could you please point out,
>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>
>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>> In the same discussion, it was ellaborated on how the impact would be with
>> change in the new firmware interface and other video layers in the driver. I can
>> add this in cover letter in the next revision.
>>
>> We see some duplication of code and to handle the same, the series brings in a
>> common code reusability between iris and venus. Aligning the common peices of
>> venus and iris will be a work in progress, once we land the base driver for iris.
>>
>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>
> If you want it to get merged, then create such plan, please.
>
>> venus as the changes are too interleaved to absorb in venus driver. And there is
>> significant interest in community to start validating video driver on sm8550 or
>> x1e80100.
>
> Community does not want duplicated drivers leading to maintenance costs.
> Your approach to this is not how upstreaming process works.
If you go over the pseudo code which i explained in [1], you can see that the
maintenance cost would be higher if try to impose the new interface in existing
driver. Video team, alongwith Stan, have considered the aspect of maintenance
before starting on this new driver.
[1] https://lore.kernel.org/lkml/8c97d866-1cab-0106-4ab3-3ca070945ef7@quicinc.com/
Regards,
Vikash
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (36 preceding siblings ...)
2023-12-18 18:38 ` Dmitry Baryshkov
@ 2023-12-20 20:51 ` Nicolas Dufresne
2024-02-29 15:09 ` Vikash Garodia
` (2 subsequent siblings)
40 siblings, 0 replies; 102+ messages in thread
From: Nicolas Dufresne @ 2023-12-20 20:51 UTC (permalink / raw)
To: Dikshita Agarwal, linux-media, linux-kernel, stanimir.k.varbanov,
quic_vgarodia, agross, andersson, konrad.dybcio, mchehab,
bryan.odonoghue
Cc: linux-arm-msm, quic_abhinavk
Hi,
Le lundi 18 décembre 2023 à 17:01 +0530, Dikshita Agarwal a écrit :
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.
> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.
>
> Changes done since v1[1]:
> - Patches are created as logical split instead of file by file.
> - Extracted common functionality between venus and iris driver and placed them
> under common folder vcodec.
> - Flattened directory structure.
> - Restructured the code in a simplified layer architecture.
> - Implemented runtime PM, and dropped explicit power collapse thread in driver.
> - Moved to standard kernel log print api.
> - Simplified the bus and clock calculation logic.
> - Removed debug features like dma tracker, frame stats, debugfs.
> - Merged v4l2 and vidc layer as vidc in driver.
> - Removed separate probe for context bank.
> - Use of_device_get_match_data to get platform data.
> - Replaced complex macros with inline functions.
> - Migrated venus to iris names.
> - Addressed many other comments received on [1].
>
> [1]: https://patchwork.kernel.org/project/linux-media/list/?series=770581
>
> Patch 1, adds dt binding for iris driver.
>
> Patches 2-4, are introducing vcodec folder and moving common APIs like fw
> load/unload, buffer size calcualtions, read/write to shared queues to common
> files which are being used by both venus and iris drivers.
> This also moves the venus driver folder to vcodec folder.
>
> Patch 5, adds the maintainers for iris driver.
>
> Patch 6-29, adds core iris driver and enable decoder.
>
> Patch 30-34, enable encoder functionality in iris driver.
>
> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
The test says:
/*
* There should *not* be an artificial limit to the number
* of open()s you can do on a V4L2 device node. So test whether
* you can open a device node at least 100 times.
*
* And please don't start rejecting opens in your driver at 101!
* There really shouldn't be a limit in the driver.
*
* If there are resource limits, then check against those limits
* where they are actually needed.
*/
So might have been acceptable for old venus, but not for iris since its a new
driver. Discussion and solution is needed. This is, I suppose, limiting the
number of decode sessions ? I personally find that method easier for userspace
then let say failing at streamon or something, but clearly a discussion need to
take place here as this is failing the compliance.
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> testVIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video0: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Compliance test for iris_driver device /dev/video1:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video1: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Dikshita Agarwal (27):
> media: introduce common helpers for video firmware handling
> media: introduce common helpers for queues handling
> media: introduce common helpers for buffer size calculation
These helper make it possible to introduce regressions, did you run compliance
tests against a venus based system to make sure ?
On top of this, initial submission and core modification to CODEC drivers are
now expected to be accompanied with fluster [0] results. This will help later
with integration testing enablement into KernelCI and will show that you made
sure the new driver is performing fairly well and is comparable to venus
(results available in Kernel CI, let me know if you have a hard time finding the
results).
[0] https://github.com/fluendo/fluster kernel CI uses GStreamer decoder, but
ffmpeg support is also available in the test runner.
> dt-bindings: media: Add sm8550 dt schema
> media: MAINTAINERS: Add Qualcomm Iris video accelerator driver
> media: iris: register video device to platform driver
> media: iris: initialize power resources
> media: iris: introduce state machine for iris core
> media: iris: initialize shared queues for host and firmware
> communication
> media: iris: add PIL functionality for video firmware
> media: iris: introduce packetization layer for creating HFI packets
> media: iris: add video processing unit(VPU) specific register handling
> media: iris: introduce platform specific capabilities for core and
> instance
> media: iris: add handling for interrupt service routine(ISR) invoked
> by hardware
> media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
> media: iris: implement vb2_ops queue setup
> media: iris: implement HFI to queue and release buffers
> media: iris: add video hardware internal buffer count and size
> calculation
> media: iris: implement internal buffer management
> media: iris: introduce instance states
> media: iris: implement vb2 streaming ops on capture and output planes
> media: iris: implement vb2 ops for buf_queue and firmware response
> media: iris: register video encoder device to platform driver
> media: iris: add platform specific instance capabilities for encoder
> media: iris: implement iris v4l2 ioctl ops supported by encoder
> media: iris: add vb2 streaming and buffer ops for encoder
> media: iris: add power management for encoder
>
> Vikash Garodia (7):
> media: iris: implement iris v4l2 file ops
> media: iris: introduce and implement iris vb2 mem ops
> media: iris: implement iris v4l2 ioctl ops supported by decoder
> media: iris: subscribe input and output properties to firmware
> media: iris: subscribe src change and handle firmware responses
> media: iris: add instance sub states and implement DRC and Drain
> sequence
> media: iris: implement power management
>
> .../bindings/media/qcom,sm8550-iris.yaml | 177 ++
> MAINTAINERS | 11 +
> drivers/media/platform/qcom/Kconfig | 3 +-
> drivers/media/platform/qcom/Makefile | 3 +-
> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++
> drivers/media/platform/qcom/vcodec/buffers.h | 15 +
> drivers/media/platform/qcom/vcodec/firmware.c | 147 ++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 +
> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++
> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +
> drivers/media/platform/qcom/vcodec/iris/Makefile | 26 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 386 +++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 833 +++++++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.h | 65 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 147 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 110 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 121 ++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 1782 ++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 71 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 1099 ++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 68 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 917 ++++++++++
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 47 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 729 ++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 118 ++
> .../platform/qcom/vcodec/iris/iris_hfi_response.c | 1097 ++++++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 106 ++
> .../media/platform/qcom/vcodec/iris/iris_power.c | 178 ++
> .../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 357 ++++
> .../media/platform/qcom/vcodec/iris/iris_state.c | 453 +++++
> .../media/platform/qcom/vcodec/iris/iris_state.h | 78 +
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 457 +++++
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 28 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1211 +++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 25 +
> .../media/platform/qcom/vcodec/iris/iris_venc.c | 948 +++++++++++
> .../media/platform/qcom/vcodec/iris/iris_venc.h | 27 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 1419 ++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 +
> .../platform/qcom/vcodec/iris/platform_common.c | 29 +
> .../platform/qcom/vcodec/iris/platform_common.h | 323 ++++
> .../platform/qcom/vcodec/iris/platform_sm8550.c | 1190 +++++++++++++
> .../media/platform/qcom/vcodec/iris/resources.c | 506 ++++++
> .../media/platform/qcom/vcodec/iris/resources.h | 44 +
> .../media/platform/qcom/vcodec/iris/vpu_common.c | 158 ++
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 52 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 568 +++++++
> .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 395 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 1469 ++++++++++++++++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 148 ++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 +
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 172 +-
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 2 +-
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 4 +-
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 10 +-
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 521 +-----
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ----
> drivers/media/platform/qcom/venus/firmware.h | 26 -
> 93 files changed, 19175 insertions(+), 989 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (75%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (90%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (98%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (94%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (73%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (37 preceding siblings ...)
2023-12-20 20:51 ` Nicolas Dufresne
@ 2024-02-29 15:09 ` Vikash Garodia
2024-03-12 10:37 ` Hans Verkuil
2024-04-12 7:13 ` Hyunjun Ko
2024-05-16 7:57 ` Hyunjun Ko
40 siblings, 1 reply; 102+ messages in thread
From: Vikash Garodia @ 2024-02-29 15:09 UTC (permalink / raw)
To: Hans Verkuil, mchehab, stanimir.k.varbanov, andersson,
konrad.dybcio, bryan.odonoghue, agross, linux-media, linux-kernel
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Hello All,
On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.
> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.
1. We considered enhancing the venus driver to support iris functionality but
concluded that the result would essentially be two parallel drivers implemented
in the same folder.
2. Although the underlying hardware for venus and iris are quite similar, but
there are other factors which brings the need of new driver:
a. the host firmware interface (HFI) changed between the two. Venus supports
HFI protocol 1.0 while iris supports HFI protocol 2.0.
b. unlike HFI protocol 1.0, HFI protocol 2.0 is better aligned to V4L2 APIs.
c. iris driver modularize platforms, hardware variants, and states to extend
it for any upcoming SOC. Thereby more extendable to newer SOCs in future.
d. Iris also supports many advanced features.
3. Based on the comments received on posted iris driver [1], we evaluated it
further and to better align with the upstream standard and practices, we
acknowledge that even though iris driver is the way forward, it would be ideal
to bring in the Venus hardwares enabled in this driver.
4. Hence, we decided to rework iris driver to add support of HFI protocol 1.0 as
well.
5. Initially we would start with adding support for one HFI protocol 1.0 based
platform which would be SM8250.
6. SM8250 enablement on iris driver would be incremental, starting with basic
decode for H264 codec.
7. In long-term, all venus supported platforms would be migrated to iris.
8. Iris driver and venus driver will co-exist till remaining supported targets
also gets migrated to iris driver.
9. We would continue to support and maintain venus driver during this phased out
approach.
Please share your thoughts on the above proposal.
[1]
https://patchwork.kernel.org/project/linux-media/cover/1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com/#25643473
Regards,
Vikash
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2024-02-29 15:09 ` Vikash Garodia
@ 2024-03-12 10:37 ` Hans Verkuil
2024-03-15 13:51 ` Vikash Garodia
0 siblings, 1 reply; 102+ messages in thread
From: Hans Verkuil @ 2024-03-12 10:37 UTC (permalink / raw)
To: Vikash Garodia, Hans Verkuil, mchehab, stanimir.k.varbanov,
andersson, konrad.dybcio, bryan.odonoghue, agross, linux-media,
linux-kernel
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Hi Vikash,
On 2/29/24 16:09, Vikash Garodia wrote:
> Hello All,
>
> On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
>> This patch series introduces support for Qualcomm new video acceleration
>> hardware architecture, used for video stream decoding/encoding. This driver
>> is based on new communication protocol between video hardware and application
>> processor.
>> This driver comes with below capabilities:
>> - V4L2 complaint video driver with M2M and STREAMING capability.
>> - Supports H264, H265, VP9 decoders.
>> - Supports H264, H265 encoders.
>> This driver comes with below features:
>> - Centralized resource and memory management.
>> - Centralized management of core and instance states.
>> - Defines platform specific capabilities and features. As a results, it provides
>> a single point of control to enable/disable a given feature depending on
>> specific platform capabilities.
>> - Handles hardware interdependent configurations. For a given configuration from
>> client, the driver checks for hardware dependent configuration/s and updates
>> the same.
>> - Handles multiple complex video scenarios involving state transitions - DRC,
>> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
>> sequence.
>> - Introduces a flexible way for driver to subscribe for any property with
>> hardware. Hardware would inform driver with those subscribed property with any
>> change in value.
>> - Introduces performance (clock and bus) model based on new hardware
>> architecture.
>> - Introduces multi thread safe design to handle communication between client and
>> hardware.
>> - Adapts encoder quality improvements available in new hardware architecture.
>> - Implements asynchronous communication with hardware to achieve better
>> experience in low latency usecases.
>> - Supports multi stage hardware architecture for encode/decode.
>> - Output and capture planes are controlled independently. Thereby providing a
>> way to reconfigure individual plane.
>> - Hardware packetization layer supports synchronization between configuration
>> packet and data packet.
>> - Introduces a flexibility to receive a hardware response for a given command
>> packet.
>> - Native hardware support of LAST flag which is mandatory to align with port
>> reconfiguration and DRAIN sequence as per V4L guidelines.
>> - Native hardware support for drain sequence.
>
> 1. We considered enhancing the venus driver to support iris functionality but
> concluded that the result would essentially be two parallel drivers implemented
> in the same folder.
> 2. Although the underlying hardware for venus and iris are quite similar, but
> there are other factors which brings the need of new driver:
> a. the host firmware interface (HFI) changed between the two. Venus supports
> HFI protocol 1.0 while iris supports HFI protocol 2.0.
> b. unlike HFI protocol 1.0, HFI protocol 2.0 is better aligned to V4L2 APIs.
> c. iris driver modularize platforms, hardware variants, and states to extend
> it for any upcoming SOC. Thereby more extendable to newer SOCs in future.
> d. Iris also supports many advanced features.
> 3. Based on the comments received on posted iris driver [1], we evaluated it
> further and to better align with the upstream standard and practices, we
> acknowledge that even though iris driver is the way forward, it would be ideal
> to bring in the Venus hardwares enabled in this driver.
> 4. Hence, we decided to rework iris driver to add support of HFI protocol 1.0 as
> well.
> 5. Initially we would start with adding support for one HFI protocol 1.0 based
> platform which would be SM8250.
> 6. SM8250 enablement on iris driver would be incremental, starting with basic
> decode for H264 codec.
> 7. In long-term, all venus supported platforms would be migrated to iris.
> 8. Iris driver and venus driver will co-exist till remaining supported targets
> also gets migrated to iris driver.
> 9. We would continue to support and maintain venus driver during this phased out
> approach.
>
> Please share your thoughts on the above proposal.
I think this is a reasonable approach: the venus driver is quite old and it was
created when we were at more-or-less the same time also developing better codec
frameworks. So it is not quite up-to-date with all the latest requirements.
Starting with a clean slate for the iris driver and then add support for venus
platforms to the iris driver makes sense.
Just one serious concern: who will do the venus platform migration? Are there resources
available to do that work? Or is this just wishful thinking?
Regards,
Hans
>
> [1]
> https://patchwork.kernel.org/project/linux-media/cover/1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com/#25643473
>
> Regards,
> Vikash
>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2024-03-12 10:37 ` Hans Verkuil
@ 2024-03-15 13:51 ` Vikash Garodia
0 siblings, 0 replies; 102+ messages in thread
From: Vikash Garodia @ 2024-03-15 13:51 UTC (permalink / raw)
To: Hans Verkuil, Hans Verkuil, mchehab, stanimir.k.varbanov,
andersson, konrad.dybcio, bryan.odonoghue, agross, linux-media,
linux-kernel
Cc: linux-arm-msm, quic_abhinavk, Dikshita Agarwal
Hi Hans,
On 3/12/2024 4:07 PM, Hans Verkuil wrote:
> Hi Vikash,
>
> On 2/29/24 16:09, Vikash Garodia wrote:
>> Hello All,
>>
>> On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
>>> This patch series introduces support for Qualcomm new video acceleration
>>> hardware architecture, used for video stream decoding/encoding. This driver
>>> is based on new communication protocol between video hardware and application
>>> processor.
>>> This driver comes with below capabilities:
>>> - V4L2 complaint video driver with M2M and STREAMING capability.
>>> - Supports H264, H265, VP9 decoders.
>>> - Supports H264, H265 encoders.
>>> This driver comes with below features:
>>> - Centralized resource and memory management.
>>> - Centralized management of core and instance states.
>>> - Defines platform specific capabilities and features. As a results, it provides
>>> a single point of control to enable/disable a given feature depending on
>>> specific platform capabilities.
>>> - Handles hardware interdependent configurations. For a given configuration from
>>> client, the driver checks for hardware dependent configuration/s and updates
>>> the same.
>>> - Handles multiple complex video scenarios involving state transitions - DRC,
>>> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
>>> sequence.
>>> - Introduces a flexible way for driver to subscribe for any property with
>>> hardware. Hardware would inform driver with those subscribed property with any
>>> change in value.
>>> - Introduces performance (clock and bus) model based on new hardware
>>> architecture.
>>> - Introduces multi thread safe design to handle communication between client and
>>> hardware.
>>> - Adapts encoder quality improvements available in new hardware architecture.
>>> - Implements asynchronous communication with hardware to achieve better
>>> experience in low latency usecases.
>>> - Supports multi stage hardware architecture for encode/decode.
>>> - Output and capture planes are controlled independently. Thereby providing a
>>> way to reconfigure individual plane.
>>> - Hardware packetization layer supports synchronization between configuration
>>> packet and data packet.
>>> - Introduces a flexibility to receive a hardware response for a given command
>>> packet.
>>> - Native hardware support of LAST flag which is mandatory to align with port
>>> reconfiguration and DRAIN sequence as per V4L guidelines.
>>> - Native hardware support for drain sequence.
>>
>> 1. We considered enhancing the venus driver to support iris functionality but
>> concluded that the result would essentially be two parallel drivers implemented
>> in the same folder.
>> 2. Although the underlying hardware for venus and iris are quite similar, but
>> there are other factors which brings the need of new driver:
>> a. the host firmware interface (HFI) changed between the two. Venus supports
>> HFI protocol 1.0 while iris supports HFI protocol 2.0.
>> b. unlike HFI protocol 1.0, HFI protocol 2.0 is better aligned to V4L2 APIs.
>> c. iris driver modularize platforms, hardware variants, and states to extend
>> it for any upcoming SOC. Thereby more extendable to newer SOCs in future.
>> d. Iris also supports many advanced features.
>> 3. Based on the comments received on posted iris driver [1], we evaluated it
>> further and to better align with the upstream standard and practices, we
>> acknowledge that even though iris driver is the way forward, it would be ideal
>> to bring in the Venus hardwares enabled in this driver.
>> 4. Hence, we decided to rework iris driver to add support of HFI protocol 1.0 as
>> well.
>> 5. Initially we would start with adding support for one HFI protocol 1.0 based
>> platform which would be SM8250.
>> 6. SM8250 enablement on iris driver would be incremental, starting with basic
>> decode for H264 codec.
>> 7. In long-term, all venus supported platforms would be migrated to iris.
>> 8. Iris driver and venus driver will co-exist till remaining supported targets
>> also gets migrated to iris driver.
>> 9. We would continue to support and maintain venus driver during this phased out
>> approach.
>>
>> Please share your thoughts on the above proposal.
>
> I think this is a reasonable approach: the venus driver is quite old and it was
> created when we were at more-or-less the same time also developing better codec
> frameworks. So it is not quite up-to-date with all the latest requirements.
>
> Starting with a clean slate for the iris driver and then add support for venus
> platforms to the iris driver makes sense.
Appreciate your time to review the proposal.
>
> Just one serious concern: who will do the venus platform migration? Are there resources
> available to do that work? Or is this just wishful thinking?
To initiate the venus platform migration, iris driver would be enabled with all
SOCs under HFI_V6XX which are currently supported in venus driver. In doing so,
iris driver, posted earlier [1] is being reworked to have cleaner abstraction to
invoke HFI 1.0 protocol (venus variants) and would be applicable for all venus
platforms. With this, remaining platform migration would be done with some
incremental efforts (platform specific handling).
We would like to get help from community and co-work in enabling the migration
for remaining venus platforms in a phased out approach.
Regards,
Vikash
[1]
https://patchwork.kernel.org/project/linux-media/cover/1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com/#25643473
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (38 preceding siblings ...)
2024-02-29 15:09 ` Vikash Garodia
@ 2024-04-12 7:13 ` Hyunjun Ko
2024-04-12 13:52 ` Bryan O'Donoghue
2024-05-16 7:57 ` Hyunjun Ko
40 siblings, 1 reply; 102+ messages in thread
From: Hyunjun Ko @ 2024-04-12 7:13 UTC (permalink / raw)
To: quic_dikshita
Cc: agross, andersson, bryan.odonoghue, konrad.dybcio, linux-arm-msm,
linux-kernel, linux-media, mchehab, quic_abhinavk, quic_vgarodia,
stanimir.k.varbanov
Hi,
I'm trying this series of patches for enabling /dev/video0 on sm8650 hdk but failed.
After modprobing, lsmod says just like the follwoing:
root@hdk8650:/lib/modules/6.7.0-rc3+# lsmod
Module Size Used by
iris 110592 -2
v4l2_mem2mem 20480 -2
videobuf2_memops 12288 -2
videobuf2_v4l2 20480 -2
videobuf2_common 45056 -2
videodev 176128 -2
This series looks for sm8550 device though, my question is that this series have been tested on the device (sm8650 hdk)? or do you expect this should work on it?
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2024-04-12 7:13 ` Hyunjun Ko
@ 2024-04-12 13:52 ` Bryan O'Donoghue
0 siblings, 0 replies; 102+ messages in thread
From: Bryan O'Donoghue @ 2024-04-12 13:52 UTC (permalink / raw)
To: Hyunjun Ko, quic_dikshita
Cc: agross, andersson, konrad.dybcio, linux-arm-msm, linux-kernel,
linux-media, mchehab, quic_abhinavk, quic_vgarodia,
stanimir.k.varbanov
On 12/04/2024 08:13, Hyunjun Ko wrote:
> Hi,
>
> I'm trying this series of patches for enabling /dev/video0 on sm8650 hdk but failed.
> After modprobing, lsmod says just like the follwoing:
>
> root@hdk8650:/lib/modules/6.7.0-rc3+# lsmod
> Module Size Used by
> iris 110592 -2
> v4l2_mem2mem 20480 -2
> videobuf2_memops 12288 -2
> videobuf2_v4l2 20480 -2
> videobuf2_common 45056 -2
> videodev 176128 -2
>
>
> This series looks for sm8550 device though, my question is that this series have been tested on the device (sm8650 hdk)? or do you expect this should work on it?
>
Different SoCs so despite how close they look in version numbers, you'd
expect at a minimum some clock and/or power-domain churn - even if there
is a 1:1 mapping in register numbers and offsets.
We wouldn't ordinarily expect to be able to be able to move SoC versions
so easily - 8550 and 8650 have similar SoC version numbers but, this is
not necessarily an indicator of silicon IP block version reuse.
---
bod
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2023-12-18 11:31 [PATCH v2 00/34] Qualcomm video encoder and decoder driver Dikshita Agarwal
` (39 preceding siblings ...)
2024-04-12 7:13 ` Hyunjun Ko
@ 2024-05-16 7:57 ` Hyunjun Ko
2024-05-16 8:06 ` Vikash Garodia
40 siblings, 1 reply; 102+ messages in thread
From: Hyunjun Ko @ 2024-05-16 7:57 UTC (permalink / raw)
To: quic_dikshita
Cc: agross, andersson, bryan.odonoghue, konrad.dybcio, linux-arm-msm,
linux-kernel, linux-media, mchehab, quic_abhinavk, quic_vgarodia,
stanimir.k.varbanov
Hi,
Thanks for this series of patches. I successfully adjusted these patches and tried to test video features with gstreamer or ffmpeg.
But I found this provides staetful interfaces while I need stateless, which might cause an issue for my side..
My question is do you have any plan to implement stateless interfaces or already you have somewhere?
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver
2024-05-16 7:57 ` Hyunjun Ko
@ 2024-05-16 8:06 ` Vikash Garodia
0 siblings, 0 replies; 102+ messages in thread
From: Vikash Garodia @ 2024-05-16 8:06 UTC (permalink / raw)
To: Hyunjun Ko, quic_dikshita
Cc: agross, andersson, bryan.odonoghue, konrad.dybcio, linux-arm-msm,
linux-kernel, linux-media, mchehab, quic_abhinavk,
stanimir.k.varbanov
Hi,
On 5/16/2024 1:27 PM, Hyunjun Ko wrote:
> Hi,
>
> Thanks for this series of patches. I successfully adjusted these patches and tried to test video features with gstreamer or ffmpeg.
> But I found this provides staetful interfaces while I need stateless, which might cause an issue for my side..
>
> My question is do you have any plan to implement stateless interfaces or already you have somewhere?
There is no plan to implement stateless interfaces.
Regards,
Vikash
^ permalink raw reply [flat|nested] 102+ messages in thread