* [PATCH v2 0/5] Enable support for AV1 stateful decoder
@ 2025-10-17 18:35 Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 1/5] media: uapi: videodev2: Add " Deepa Guthyappa Madivalara
` (4 more replies)
0 siblings, 5 replies; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-17 18:35 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Deepa Guthyappa Madivalara, Bryan O'Donoghue
Hi all,
This patch series adds initial support for the AV1 stateful decoder
codecs in iris decoder. Also it adds support for AV1 stateful decoder
in V4l2. The objective of this work is to extend the Iris decoder's
capabilities to handle AV1 format codec streams, including necessary
format handling and buffer management.
These patches also address the comments and feedback received from the
RFC patches previously sent. I have made the necessary improvements
based on the community's suggestions.
Changes in v2:
- Updated documentation to target AV1 codec, not just AV1 decoder
(Nicolas)
- Updated description for V4L2_PIX_FMT_AV1 (Nicolas)
- Simplified buffer calculations and replaced numbers with relevant
enums (Bryan, Nicolas)
- Improved commit text for patch 5/5
- Fix for kernel test robot failure
Reported-by: kernel test robot <lkp@intel.com>
Closes:
https://lore.kernel.org/oe-kbuild-all/202510021620.4BVCZwgf-lkp@intel.com/
- Link to v1:
https://lore.kernel.org/r/20251001-av1_irisdecoder-v1-0-9fb08f3b96a0@oss.qualcomm.com
Changes since RFC:
- Addressed CRC issues seen during fluster testing which
are fixed with firmware fix [1]
- Added Documentation for AV1 stateful uapi [Nicholas]
- Resolved issues reported by static tool analyzers
- RFC:
https://lore.kernel.org/linux-media/20250902-rfc_split-v1-0-47307a70c061@oss.qualcomm.com/
[1]:
https://lore.kernel.org/linux-firmware/ff27f712-a96e-4fa6-7572-a0091537d8ac@oss.qualcomm.com/
These patches are tested on SM8550 for AV1 decoder while
ensuring other codecs are not affected.
Gstreamer testing:
Gstreamer MR for enabling AV1 stateful decoder:
https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1
Fluster testing:
Test suite: AV1-TEST-VECTORS
The result of fluster test on SM8550:
135/242 testcases passed while testing AV1-TEST-VECTORS with
GStreamer-AV1-V4L2-Gst1.0
failing tests:
unsupported content with bitdepth 10 (66 tests)
Iris decoder supports only 8bit NV12
av1-1-b10-00-quantizer-*
Unsupported resolution (36 tests).
Iris hardware decoder supports min resolution of 96x96
av1-1-b8-01-size-*
Unsupported colorformat (1 test)
av1-1-b8-24-monochrome
Unsupported SVC tests (2tests)
av1-1-b8-22-svc-L2T1
av1-1-b8-22-svc-L2T2
Bitstream corruption issue: (2tests)
av1-1-b8-03-sizeup
av1-1-b8-03-sizedown
Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS
13/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with
GStreamer-AV1-V4L2-Gst1.0
Following the RFC feedback, focused on only IVF/MKV content
as AV1 parser lacks support for below content
AV1-ARGON-PROFILE0-CORE-ANNEX-B
AV1-ARGON-PROFILE0-NON-ANNEX-B
AV1-ARGON-PROFILE0-NON-ANNEX-B
Unsupported test suites:
Iris Decoder supports only PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN
and 8 bit, 420 only
AV1-ARGON-PROFILE1-CORE-ANNEX-B
AV1-ARGON-PROFILE1-NON-ANNEX-B
AV1-ARGON-PROFILE1-STRESS-ANNEX-B
AV1-ARGON-PROFILE2-CORE-ANNEX-B
AV1-ARGON-PROFILE2-NON-ANNEX-B
AV1-ARGON-PROFILE2-STRESS-ANNEX-B
CHROMIUM-10bit-AV1-TEST-VECTORS
Compliance test for iris_driver device /dev/video0:
Driver Info:
Driver name : iris_driver
Card type : Iris Decoder
Bus info : platform:aa00000.video-codec
Driver version : 6.17.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 12 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
test blocking wait: OK
Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0
Thanks,
Deepa
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
To: Mauro Carvalho Chehab <mchehab@kernel.org>
To: Vikash Garodia <vikash.garodia@oss.qualcomm.com>
To: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
To: Abhinav Kumar <abhinav.kumar@linux.dev>
To: Bryan O'Donoghue <bod@kernel.org>
Cc: linux-media@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-arm-msm@vger.kernel.org
---
Deepa Guthyappa Madivalara (5):
media: uapi: videodev2: Add support for AV1 stateful decoder
media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
media: iris: Add support for AV1 format in iris decoder
media: iris: Define AV1-specific platform capabilities and properties
media: iris: Add internal buffer calculation for AV1 decoder
.../userspace-api/media/v4l/pixfmt-compressed.rst | 8 +
drivers/media/platform/qcom/iris/iris_buffer.h | 2 +
drivers/media/platform/qcom/iris/iris_ctrls.c | 8 +
drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
.../platform/qcom/iris/iris_hfi_gen2_command.c | 110 +++++++-
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 10 +
.../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++
drivers/media/platform/qcom/iris/iris_instance.h | 1 +
.../platform/qcom/iris/iris_platform_common.h | 15 ++
.../media/platform/qcom/iris/iris_platform_gen2.c | 156 ++++++++++-
.../platform/qcom/iris/iris_platform_sm8250.c | 18 ++
drivers/media/platform/qcom/iris/iris_vdec.c | 25 +-
drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 298 ++++++++++++++++++++-
drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 116 ++++++++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
17 files changed, 767 insertions(+), 28 deletions(-)
---
base-commit: afb100a5ea7a13d7e6937dcd3b36b19dc6cc9328
change-id: 20251001-av1_irisdecoder-846c31f70a51
Best regards,
--
Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2 1/5] media: uapi: videodev2: Add support for AV1 stateful decoder
2025-10-17 18:35 [PATCH v2 0/5] Enable support for AV1 stateful decoder Deepa Guthyappa Madivalara
@ 2025-10-17 18:35 ` Deepa Guthyappa Madivalara
2025-10-17 20:01 ` Nicolas Dufresne
2025-10-17 18:35 ` [PATCH v2 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() Deepa Guthyappa Madivalara
` (3 subsequent siblings)
4 siblings, 1 reply; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-17 18:35 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Deepa Guthyappa Madivalara
Introduce a new pixel format, V4L2_PIX_FMT_AV1, to the
Video4Linux2(V4L2) API. This format is intended for AV1
bitstreams in stateful decoding/encoding workflows.
The fourcc code 'AV10' is used to distinguish
this format from the existing V4L2_PIX_FMT_AV1_FRAME,
which is used for stateless AV1 decoder implementation.
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst | 8 ++++++++
include/uapi/linux/videodev2.h | 1 +
2 files changed, 9 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 806ed73ac474ce0e6df00f902850db9fd0db240e..d82557a6ac30851b4fd432a5ceab969abaed2e25 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -274,6 +274,14 @@ Compressed Formats
of macroblocks to decode a full corresponding frame to the matching
capture buffer.
+ * .. _V4L2-PIX-FMT-AV1:
+
+ - ``V4L2_PIX_FMT_AV1``
+ - 'AV10'
+ - AV1 compressed video frame. This format is adapted for implementing AV1
+ pipeline. The decoder implements stateful video decoder and expects one
+ Temporal Unit per buffer from OBU-stream or AnnexB.
+ The encoder generates one Temporal Unit per buffer.
.. raw:: latex
\normalsize
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index becd08fdbddb857f8f2bf205d2164dc6e20e80b2..e03cfe0a5ae830595f0bf69b951ca16beaedab72 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -775,6 +775,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
#define V4L2_PIX_FMT_AV1_FRAME v4l2_fourcc('A', 'V', '1', 'F') /* AV1 parsed frame */
+#define V4L2_PIX_FMT_AV1 v4l2_fourcc('A', 'V', '1', '0') /* AV1 */
#define V4L2_PIX_FMT_SPK v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
#define V4L2_PIX_FMT_RV30 v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
#define V4L2_PIX_FMT_RV40 v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
2025-10-17 18:35 [PATCH v2 0/5] Enable support for AV1 stateful decoder Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 1/5] media: uapi: videodev2: Add " Deepa Guthyappa Madivalara
@ 2025-10-17 18:35 ` Deepa Guthyappa Madivalara
2025-10-17 20:02 ` Nicolas Dufresne
2025-10-17 18:35 ` [PATCH v2 3/5] media: iris: Add support for AV1 format in iris decoder Deepa Guthyappa Madivalara
` (2 subsequent siblings)
4 siblings, 1 reply; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-17 18:35 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Deepa Guthyappa Madivalara
Add a descriptive string for the AV1 pixel format to v4l_fill_fmtdesc(),
enabling proper reporting of AV1 support via VIDIOC_ENUM_FMT.
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 01cf52c3ea33e1a01e1b306036ba4e57ef5c95d0..bdfdf45c5de2f2ce885f219007718a54b5c86251 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1542,6 +1542,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_QC10C: descr = "QCOM Compressed 10-bit Format"; break;
case V4L2_PIX_FMT_AJPG: descr = "Aspeed JPEG"; break;
case V4L2_PIX_FMT_AV1_FRAME: descr = "AV1 Frame"; break;
+ case V4L2_PIX_FMT_AV1: descr = "AV1 OBU stream"; break;
case V4L2_PIX_FMT_MT2110T: descr = "Mediatek 10bit Tile Mode"; break;
case V4L2_PIX_FMT_MT2110R: descr = "Mediatek 10bit Raster Mode"; break;
case V4L2_PIX_FMT_HEXTILE: descr = "Hextile Compressed Format"; break;
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 3/5] media: iris: Add support for AV1 format in iris decoder
2025-10-17 18:35 [PATCH v2 0/5] Enable support for AV1 stateful decoder Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 1/5] media: uapi: videodev2: Add " Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() Deepa Guthyappa Madivalara
@ 2025-10-17 18:35 ` Deepa Guthyappa Madivalara
2025-10-27 6:04 ` Dikshita Agarwal
2025-10-17 18:35 ` [PATCH v2 4/5] media: iris: Define AV1-specific platform capabilities and properties Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 5/5] media: iris: Add internal buffer calculation for AV1 decoder Deepa Guthyappa Madivalara
4 siblings, 1 reply; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-17 18:35 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Deepa Guthyappa Madivalara, Bryan O'Donoghue
Extend iris decoder driver to support format V4L2_PIX_FMT_AV1.
This change updates the format enumeration (VIDIOC_ENUM_FMT)
and allows setting AV1 format via VIDIOC_S_FMT for gen2 and beyond.
Gen1 iris hardware decoder does not support AV1 format.
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 1 +
drivers/media/platform/qcom/iris/iris_instance.h | 1 +
.../platform/qcom/iris/iris_platform_common.h | 2 ++
.../media/platform/qcom/iris/iris_platform_gen2.c | 23 ++++++++++++++++++++
.../platform/qcom/iris/iris_platform_sm8250.c | 17 +++++++++++++++
drivers/media/platform/qcom/iris/iris_vdec.c | 25 ++++++----------------
6 files changed, 50 insertions(+), 19 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index aa1f795f5626c1f76a32dd650302633877ce67be..bbfe7a0851ea94fb7041a868b4df8b2ec63bf427 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -138,6 +138,7 @@ enum hfi_codec_type {
HFI_CODEC_DECODE_HEVC = 3,
HFI_CODEC_ENCODE_HEVC = 4,
HFI_CODEC_DECODE_VP9 = 5,
+ HFI_CODEC_DECODE_AV1 = 7,
};
enum hfi_picture_type {
diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
index 5982d7adefeab80905478b32cddba7bd4651a691..f1883ffc138fd975fb76d4e45904ee04e196cd20 100644
--- a/drivers/media/platform/qcom/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/iris/iris_instance.h
@@ -19,6 +19,7 @@ enum iris_fmt_type {
IRIS_FMT_H264,
IRIS_FMT_HEVC,
IRIS_FMT_VP9,
+ IRIS_FMT_AV1,
};
struct iris_fmt {
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 58d05e0a112eed25faea027a34c719c89d6c3897..9aca70b4c0690f0d8d799e2a9976bd20d6bb8c94 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -214,6 +214,8 @@ struct iris_platform_data {
u64 dma_mask;
const char *fwname;
u32 pas_id;
+ struct iris_fmt *inst_iris_fmts;
+ u32 inst_iris_fmts_size;
struct platform_inst_caps *inst_caps;
struct platform_inst_fw_cap *inst_fw_caps_dec;
u32 inst_fw_caps_dec_size;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index 36d69cc73986b74534a2912524c8553970fd862e..15a38b7122af6d978fac433e8c02b4da5c141e1a 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -19,6 +19,25 @@
#define VIDEO_ARCH_LX 1
#define BITRATE_MAX 245000000
+static struct iris_fmt platform_fmts_sm8550_dec[] = {
+ [IRIS_FMT_H264] = {
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_HEVC] = {
+ .pixfmt = V4L2_PIX_FMT_HEVC,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_VP9] = {
+ .pixfmt = V4L2_PIX_FMT_VP9,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_AV1] = {
+ .pixfmt = V4L2_PIX_FMT_AV1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+};
+
static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
{
.cap_id = PROFILE_H264,
@@ -760,6 +779,8 @@ struct iris_platform_data sm8550_data = {
.dma_mask = 0xe0000000 - 1,
.fwname = "qcom/vpu/vpu30_p4.mbn",
.pas_id = IRIS_PAS_ID,
+ .inst_iris_fmts = platform_fmts_sm8550_dec,
+ .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
.inst_caps = &platform_inst_cap_sm8550,
.inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
@@ -852,6 +873,8 @@ struct iris_platform_data sm8650_data = {
.dma_mask = 0xe0000000 - 1,
.fwname = "qcom/vpu/vpu33_p4.mbn",
.pas_id = IRIS_PAS_ID,
+ .inst_iris_fmts = platform_fmts_sm8550_dec,
+ .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
.inst_caps = &platform_inst_cap_sm8550,
.inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
index 16486284f8acccf6a95a27f6003e885226e28f4d..8342a0b6f825b385192baedfc67c50bba7661470 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
@@ -17,6 +17,21 @@
#define BITRATE_PEAK_DEFAULT (BITRATE_DEFAULT * 2)
#define BITRATE_STEP 100
+static struct iris_fmt platform_fmts_sm8250_dec[] = {
+ [IRIS_FMT_H264] = {
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_HEVC] = {
+ .pixfmt = V4L2_PIX_FMT_HEVC,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_VP9] = {
+ .pixfmt = V4L2_PIX_FMT_VP9,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+};
+
static struct platform_inst_fw_cap inst_fw_cap_sm8250_dec[] = {
{
.cap_id = PIPE,
@@ -337,6 +352,8 @@ struct iris_platform_data sm8250_data = {
.dma_mask = 0xe0000000 - 1,
.fwname = "qcom/vpu-1.0/venus.mbn",
.pas_id = IRIS_PAS_ID,
+ .inst_iris_fmts = platform_fmts_sm8250_dec,
+ .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8250_dec),
.inst_caps = &platform_inst_cap_sm8250,
.inst_fw_caps_dec = inst_fw_cap_sm8250_dec,
.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8250_dec),
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index ae13c3e1b426bfd81a7b46dc6c3ff5eb5c4860cb..be8d2d48c82f385e4f46807f7e0dd52e469927cd 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -67,26 +67,12 @@ void iris_vdec_inst_deinit(struct iris_inst *inst)
kfree(inst->fmt_src);
}
-static const struct iris_fmt iris_vdec_formats[] = {
- [IRIS_FMT_H264] = {
- .pixfmt = V4L2_PIX_FMT_H264,
- .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
- },
- [IRIS_FMT_HEVC] = {
- .pixfmt = V4L2_PIX_FMT_HEVC,
- .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
- },
- [IRIS_FMT_VP9] = {
- .pixfmt = V4L2_PIX_FMT_VP9,
- .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
- },
-};
-
static const struct iris_fmt *
find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
{
- unsigned int size = ARRAY_SIZE(iris_vdec_formats);
- const struct iris_fmt *fmt = iris_vdec_formats;
+ unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
+ const struct iris_fmt *fmt =
+ inst->core->iris_platform_data->inst_iris_fmts;
unsigned int i;
for (i = 0; i < size; i++) {
@@ -103,8 +89,9 @@ find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
static const struct iris_fmt *
find_format_by_index(struct iris_inst *inst, u32 index, u32 type)
{
- const struct iris_fmt *fmt = iris_vdec_formats;
- unsigned int size = ARRAY_SIZE(iris_vdec_formats);
+ unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
+ const struct iris_fmt *fmt =
+ inst->core->iris_platform_data->inst_iris_fmts;
if (index >= size || fmt[index].type != type)
return NULL;
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 4/5] media: iris: Define AV1-specific platform capabilities and properties
2025-10-17 18:35 [PATCH v2 0/5] Enable support for AV1 stateful decoder Deepa Guthyappa Madivalara
` (2 preceding siblings ...)
2025-10-17 18:35 ` [PATCH v2 3/5] media: iris: Add support for AV1 format in iris decoder Deepa Guthyappa Madivalara
@ 2025-10-17 18:35 ` Deepa Guthyappa Madivalara
2025-10-27 9:07 ` Dikshita Agarwal
2025-10-17 18:35 ` [PATCH v2 5/5] media: iris: Add internal buffer calculation for AV1 decoder Deepa Guthyappa Madivalara
4 siblings, 1 reply; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-17 18:35 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Deepa Guthyappa Madivalara, Bryan O'Donoghue
Defining platform specific capabilities specific to AV1 decoder.
Set and subscribe to manadatory properties to firmware for AV1.
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
drivers/media/platform/qcom/iris/iris_buffer.h | 1 +
drivers/media/platform/qcom/iris/iris_ctrls.c | 8 ++
drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
.../platform/qcom/iris/iris_hfi_gen2_command.c | 110 ++++++++++++++++-
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 9 ++
.../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++++
.../platform/qcom/iris/iris_platform_common.h | 13 ++
.../media/platform/qcom/iris/iris_platform_gen2.c | 133 ++++++++++++++++++++-
.../platform/qcom/iris/iris_platform_sm8250.c | 1 +
drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
10 files changed, 296 insertions(+), 5 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
index 325d30fce5c99185b61ff989fbfd4de9a56762b2..5ef365d9236c7cbdee24a4614789b3191881968b 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_buffer.h
@@ -42,6 +42,7 @@ enum iris_buffer_type {
BUF_SCRATCH_1,
BUF_SCRATCH_2,
BUF_VPSS,
+ BUF_PARTIAL,
BUF_TYPE_MAX,
};
diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
index 754a5ad718bc37630bb861012301df7a2e7342a1..620c7e1bd273e25febd8ca70dd1dcfb0b862692b 100644
--- a/drivers/media/platform/qcom/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
@@ -98,6 +98,10 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
return B_FRAME_QP_H264;
case V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP:
return B_FRAME_QP_HEVC;
+ case V4L2_CID_MPEG_VIDEO_AV1_PROFILE:
+ return PROFILE_AV1;
+ case V4L2_CID_MPEG_VIDEO_AV1_LEVEL:
+ return LEVEL_AV1;
default:
return INST_FW_CAP_MAX;
}
@@ -185,6 +189,10 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
return V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP;
case B_FRAME_QP_HEVC:
return V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP;
+ case PROFILE_AV1:
+ return V4L2_CID_MPEG_VIDEO_AV1_PROFILE;
+ case LEVEL_AV1:
+ return V4L2_CID_MPEG_VIDEO_AV1_LEVEL;
default:
return 0;
}
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h
index b51471fb32c70acee44c37f8e9dce0c6bc0b6ccc..3edb5ae582b49bea2e2408c4a5cfc0a742adc05f 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_common.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h
@@ -141,6 +141,9 @@ struct hfi_subscription_params {
u32 profile;
u32 level;
u32 tier;
+ u32 drap;
+ u32 film_grain;
+ u32 super_block;
};
u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries);
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
index 4ce71a14250832440099e4cf3835b4aedfb749e8..86fa5ab02bfe3701e0c3dd38f1a9cadc88f5e4cd 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -10,6 +10,7 @@
#define UNSPECIFIED_COLOR_FORMAT 5
#define NUM_SYS_INIT_PACKETS 8
+#define NUM_COMV_AV1 18
#define SYS_INIT_PKT_SIZE (sizeof(struct iris_hfi_header) + \
NUM_SYS_INIT_PACKETS * (sizeof(struct iris_hfi_packet) + sizeof(u32)))
@@ -121,6 +122,7 @@ static u32 iris_hfi_gen2_get_port_from_buf_type(struct iris_inst *inst,
case BUF_COMV:
case BUF_NON_COMV:
case BUF_LINE:
+ case BUF_PARTIAL:
return HFI_PORT_BITSTREAM;
case BUF_OUTPUT:
case BUF_DPB:
@@ -380,6 +382,9 @@ static int iris_hfi_gen2_set_profile(struct iris_inst *inst, u32 plane)
case V4L2_PIX_FMT_H264:
profile = inst->fw_caps[PROFILE_H264].value;
break;
+ case V4L2_PIX_FMT_AV1:
+ profile = inst->fw_caps[PROFILE_AV1].value;
+ break;
}
inst_hfi_gen2->src_subcr_params.profile = profile;
@@ -409,6 +414,9 @@ static int iris_hfi_gen2_set_level(struct iris_inst *inst, u32 plane)
case V4L2_PIX_FMT_H264:
level = inst->fw_caps[LEVEL_H264].value;
break;
+ case V4L2_PIX_FMT_AV1:
+ level = inst->fw_caps[LEVEL_AV1].value;
+ break;
}
inst_hfi_gen2->src_subcr_params.level = level;
@@ -480,10 +488,12 @@ static int iris_hfi_gen2_set_linear_stride_scanline(struct iris_inst *inst, u32
static int iris_hfi_gen2_set_tier(struct iris_inst *inst, u32 plane)
{
- struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
u32 tier = inst->fw_caps[TIER].value;
+ tier = (inst->codec == V4L2_PIX_FMT_AV1) ? inst->fw_caps[TIER_AV1].value :
+ inst->fw_caps[TIER].value;
inst_hfi_gen2->src_subcr_params.tier = tier;
return iris_hfi_gen2_session_set_property(inst,
@@ -509,6 +519,56 @@ static int iris_hfi_gen2_set_frame_rate(struct iris_inst *inst, u32 plane)
sizeof(u32));
}
+static int iris_hfi_gen2_set_film_grain(struct iris_inst *inst, u32 plane)
+{
+ u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
+ u32 film_grain;
+
+ film_grain = inst->fw_caps[FILM_GRAIN].value;
+ inst_hfi_gen2->src_subcr_params.film_grain = film_grain;
+
+ return iris_hfi_gen2_session_set_property(inst,
+ HFI_PROP_AV1_FILM_GRAIN_PRESENT,
+ HFI_HOST_FLAGS_NONE,
+ port,
+ HFI_PAYLOAD_U32_ENUM,
+ &film_grain,
+ sizeof(u32));
+}
+
+static int iris_hfi_gen2_set_super_block(struct iris_inst *inst, u32 plane)
+{
+ u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
+ u32 super_block;
+
+ super_block = inst->fw_caps[SUPER_BLOCK].value;
+ inst_hfi_gen2->src_subcr_params.super_block = super_block;
+
+ return iris_hfi_gen2_session_set_property(inst,
+ HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
+ HFI_HOST_FLAGS_NONE,
+ port,
+ HFI_PAYLOAD_U32_ENUM,
+ &super_block,
+ sizeof(u32));
+}
+
+static int iris_hfi_gen2_set_opb_enable(struct iris_inst *inst, u32 plane)
+{
+ u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ u32 opb_enable = iris_split_mode_enabled(inst);
+
+ return iris_hfi_gen2_session_set_property(inst,
+ HFI_PROP_OPB_ENABLE,
+ HFI_HOST_FLAGS_NONE,
+ port,
+ HFI_PAYLOAD_U32,
+ &opb_enable,
+ sizeof(u32));
+}
+
static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 plane)
{
const struct iris_platform_data *pdata = inst->core->iris_platform_data;
@@ -531,6 +591,9 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
{HFI_PROP_LINEAR_STRIDE_SCANLINE, iris_hfi_gen2_set_linear_stride_scanline },
{HFI_PROP_TIER, iris_hfi_gen2_set_tier },
{HFI_PROP_FRAME_RATE, iris_hfi_gen2_set_frame_rate },
+ {HFI_PROP_AV1_FILM_GRAIN_PRESENT, iris_hfi_gen2_set_film_grain },
+ {HFI_PROP_AV1_SUPER_BLOCK_ENABLED, iris_hfi_gen2_set_super_block },
+ {HFI_PROP_OPB_ENABLE, iris_hfi_gen2_set_opb_enable },
};
if (inst->domain == DECODER) {
@@ -544,12 +607,20 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
} else if (inst->codec == V4L2_PIX_FMT_VP9) {
config_params = pdata->dec_input_config_params_vp9;
config_params_size = pdata->dec_input_config_params_vp9_size;
+ } else if (inst->codec == V4L2_PIX_FMT_AV1) {
+ config_params = pdata->dec_input_config_params_av1;
+ config_params_size = pdata->dec_input_config_params_av1_size;
} else {
return -EINVAL;
}
} else {
- config_params = pdata->dec_output_config_params;
- config_params_size = pdata->dec_output_config_params_size;
+ if (inst->codec == V4L2_PIX_FMT_AV1) {
+ config_params = pdata->dec_output_config_params_av1;
+ config_params_size = pdata->dec_output_config_params_av1_size;
+ } else {
+ config_params = pdata->dec_output_config_params;
+ config_params_size = pdata->dec_output_config_params_size;
+ }
}
} else {
if (V4L2_TYPE_IS_OUTPUT(plane)) {
@@ -598,6 +669,9 @@ static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst)
break;
case V4L2_PIX_FMT_VP9:
codec = HFI_CODEC_DECODE_VP9;
+ break;
+ case V4L2_PIX_FMT_AV1:
+ codec = HFI_CODEC_DECODE_AV1;
}
iris_hfi_gen2_packet_session_property(inst,
@@ -763,6 +837,11 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
change_param_size =
core->iris_platform_data->dec_input_config_params_vp9_size;
break;
+ case V4L2_PIX_FMT_AV1:
+ change_param = core->iris_platform_data->dec_input_config_params_av1;
+ change_param_size =
+ core->iris_platform_data->dec_input_config_params_av1_size;
+ break;
}
payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
@@ -845,6 +924,16 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
payload_size = sizeof(u32);
payload_type = HFI_PAYLOAD_U32;
break;
+ case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
+ payload[0] = subsc_params.film_grain;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
+ payload[0] = subsc_params.super_block;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
default:
prop_type = 0;
ret = -EINVAL;
@@ -900,6 +989,11 @@ static int iris_hfi_gen2_subscribe_property(struct iris_inst *inst, u32 plane)
subscribe_prop_size =
core->iris_platform_data->dec_output_prop_vp9_size;
break;
+ case V4L2_PIX_FMT_AV1:
+ subcribe_prop = core->iris_platform_data->dec_output_prop_av1;
+ subscribe_prop_size =
+ core->iris_platform_data->dec_output_prop_av1_size;
+ break;
}
}
@@ -1075,6 +1169,8 @@ static u32 iris_hfi_gen2_buf_type_from_driver(u32 domain, enum iris_buffer_type
return HFI_BUFFER_ARP;
case BUF_VPSS:
return HFI_BUFFER_VPSS;
+ case BUF_PARTIAL:
+ return HFI_BUFFER_PARTIAL_DATA;
default:
return 0;
}
@@ -1087,7 +1183,13 @@ static int iris_set_num_comv(struct iris_inst *inst)
u32 num_comv;
caps = core->iris_platform_data->inst_caps;
- num_comv = caps->num_comv;
+
+ /*
+ * AV1 needs more comv buffers than other codecs.
+ * Update accordingly.
+ */
+ num_comv = (inst->codec == V4L2_PIX_FMT_AV1) ?
+ NUM_COMV_AV1 : caps->num_comv;
return core->hfi_ops->session_set_property(inst,
HFI_PROP_COMV_BUFFER_COUNT,
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index bbfe7a0851ea94fb7041a868b4df8b2ec63bf427..9cc0989c67d74a9e051725e9ee571a2ab9160519 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -89,9 +89,18 @@ enum hfi_seq_header_mode {
#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
#define HFI_PROP_NO_OUTPUT 0x0300016a
#define HFI_PROP_BUFFER_MARK 0x0300016c
+#define HFI_PROP_WORST_COMPRESSION_RATIO 0x03000174
+#define HFI_PROP_WORST_COMPLEXITY_FACTOR 0x03000175
#define HFI_PROP_RAW_RESOLUTION 0x03000178
+#define HFI_PROP_AV1_FILM_GRAIN_PRESENT 0x03000180
+#define HFI_PROP_AV1_SUPER_BLOCK_ENABLED 0x03000181
+#define HFI_PROP_AV1_OP_POINT 0x03000182
+#define HFI_PROP_OPB_ENABLE 0x03000184
+#define HFI_PROP_AV1_TILE_ROWS_COLUMNS 0x03000187
+#define HFI_PROP_AV1_DRAP_CONFIG 0x03000189
#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
+#define HFI_PROP_AV1_UNIFORM_TILE_SPACING 0x03000197
#define HFI_PROP_END 0x03FFFFFF
#define HFI_SESSION_ERROR_BEGIN 0x04000000
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 2f1f118eae4f6462ab1aa1d16844b34e6e699f1e..eb3373f0ad4a1b26fb847db02449ec8d8cb3bdbb 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -54,6 +54,8 @@ static u32 iris_hfi_gen2_buf_type_to_driver(struct iris_inst *inst,
return BUF_SCRATCH_2;
case HFI_BUFFER_PERSIST:
return BUF_PERSIST;
+ case HFI_BUFFER_PARTIAL_DATA:
+ return BUF_PARTIAL;
default:
return 0;
}
@@ -72,6 +74,7 @@ static bool iris_hfi_gen2_is_valid_hfi_buffer_type(u32 buffer_type)
case HFI_BUFFER_DPB:
case HFI_BUFFER_PERSIST:
case HFI_BUFFER_VPSS:
+ case HFI_BUFFER_PARTIAL_DATA:
return true;
default:
return false;
@@ -596,6 +599,10 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
inst->fw_caps[PROFILE_H264].value = subsc_params.profile;
inst->fw_caps[LEVEL_H264].value = subsc_params.level;
break;
+ case V4L2_PIX_FMT_AV1:
+ inst->fw_caps[PROFILE_AV1].value = subsc_params.profile;
+ inst->fw_caps[LEVEL_AV1].value = subsc_params.level;
+ break;
}
inst->fw_caps[POC].value = subsc_params.pic_order_cnt;
@@ -608,6 +615,11 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
iris_inst_change_state(inst, IRIS_INST_ERROR);
}
+ if (inst->codec == V4L2_PIX_FMT_AV1) {
+ inst->fw_caps[FILM_GRAIN].value = subsc_params.film_grain;
+ inst->fw_caps[SUPER_BLOCK].value = subsc_params.super_block;
+ }
+
inst->fw_min_count = subsc_params.fw_min_count;
inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT);
inst->buffers[BUF_OUTPUT].size = pixmp_op->plane_fmt[0].sizeimage;
@@ -711,6 +723,12 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
case HFI_PROP_NO_OUTPUT:
inst_hfi_gen2->hfi_frame_info.no_output = 1;
break;
+ case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
+ inst_hfi_gen2->src_subcr_params.film_grain = pkt->payload[0];
+ break;
+ case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
+ inst_hfi_gen2->src_subcr_params.super_block = pkt->payload[0];
+ break;
case HFI_PROP_QUALITY_MODE:
case HFI_PROP_STAGE:
case HFI_PROP_PIPE:
@@ -841,6 +859,10 @@ static void iris_hfi_gen2_init_src_change_param(struct iris_inst *inst)
subsc_params->profile = inst->fw_caps[PROFILE_H264].value;
subsc_params->level = inst->fw_caps[LEVEL_H264].value;
break;
+ case V4L2_PIX_FMT_AV1:
+ subsc_params->profile = inst->fw_caps[PROFILE_AV1].value;
+ subsc_params->level = inst->fw_caps[LEVEL_AV1].value;
+ break;
}
subsc_params->pic_order_cnt = inst->fw_caps[POC].value;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 9aca70b4c0690f0d8d799e2a9976bd20d6bb8c94..19f0660f0388499ab71a1413b2c05f891bd5d511 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -100,6 +100,13 @@ enum platform_inst_fw_cap_type {
LEVEL_H264,
LEVEL_HEVC,
LEVEL_VP9,
+ PROFILE_AV1,
+ LEVEL_AV1,
+ TIER_AV1,
+ DRAP,
+ FILM_GRAIN,
+ SUPER_BLOCK,
+ ENH_LAYER_COUNT,
INPUT_BUF_HOST_MAX_COUNT,
OUTPUT_BUF_HOST_MAX_COUNT,
STAGE,
@@ -237,8 +244,12 @@ struct iris_platform_data {
unsigned int dec_input_config_params_hevc_size;
const u32 *dec_input_config_params_vp9;
unsigned int dec_input_config_params_vp9_size;
+ const u32 *dec_input_config_params_av1;
+ unsigned int dec_input_config_params_av1_size;
const u32 *dec_output_config_params;
unsigned int dec_output_config_params_size;
+ const u32 *dec_output_config_params_av1;
+ unsigned int dec_output_config_params_av1_size;
const u32 *enc_input_config_params;
unsigned int enc_input_config_params_size;
const u32 *enc_output_config_params;
@@ -251,6 +262,8 @@ struct iris_platform_data {
unsigned int dec_output_prop_hevc_size;
const u32 *dec_output_prop_vp9;
unsigned int dec_output_prop_vp9_size;
+ const u32 *dec_output_prop_av1;
+ unsigned int dec_output_prop_av1_size;
const u32 *dec_ip_int_buf_tbl;
unsigned int dec_ip_int_buf_tbl_size;
const u32 *dec_op_int_buf_tbl;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index 15a38b7122af6d978fac433e8c02b4da5c141e1a..600664b2aea88067a7f66a20beaacd42181f9532 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -64,6 +64,16 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
.set = iris_set_u32_enum,
},
+ {
+ .cap_id = PROFILE_AV1,
+ .min = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
+ .max = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
+ .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN),
+ .value = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
+ .hfi_id = HFI_PROP_PROFILE,
+ .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ .set = iris_set_u32_enum,
+ },
{
.cap_id = PROFILE_VP9,
.min = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
@@ -147,6 +157,33 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
.set = iris_set_u32_enum,
},
+ {
+ .cap_id = LEVEL_AV1,
+ .min = V4L2_MPEG_VIDEO_AV1_LEVEL_2_0,
+ .max = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
+ .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_1),
+ .value = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
+ .hfi_id = HFI_PROP_LEVEL,
+ .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ .set = iris_set_u32_enum,
+ },
{
.cap_id = TIER,
.min = V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
@@ -158,6 +195,53 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
.set = iris_set_u32_enum,
},
+ {
+ .cap_id = TIER_AV1,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_TIER,
+ .flags = CAP_FLAG_INPUT_PORT,
+ .set = iris_set_u32,
+ },
+ {
+ .cap_id = DRAP,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_DRAP_CONFIG,
+ .flags = CAP_FLAG_INPUT_PORT,
+ .set = iris_set_u32,
+ },
+ {
+ .cap_id = FILM_GRAIN,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_FILM_GRAIN_PRESENT,
+ .flags = CAP_FLAG_VOLATILE,
+ },
+ {
+ .cap_id = SUPER_BLOCK,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
+ },
+ {
+ .cap_id = ENH_LAYER_COUNT,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_OP_POINT,
+ .flags = CAP_FLAG_INPUT_PORT,
+ .set = iris_set_u32,
+ },
{
.cap_id = INPUT_BUF_HOST_MAX_COUNT,
.min = DEFAULT_MAX_HOST_BUF_COUNT,
@@ -701,6 +785,19 @@ static const u32 sm8550_vdec_input_config_param_vp9[] = {
HFI_PROP_LEVEL,
};
+static const u32 sm8550_vdec_input_config_param_av1[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+ HFI_PROP_TIER,
+ HFI_PROP_AV1_FILM_GRAIN_PRESENT,
+ HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+};
+
static const u32 sm8550_venc_input_config_params[] = {
HFI_PROP_COLOR_FORMAT,
HFI_PROP_RAW_RESOLUTION,
@@ -714,6 +811,12 @@ static const u32 sm8550_vdec_output_config_params[] = {
HFI_PROP_LINEAR_STRIDE_SCANLINE,
};
+static const u32 sm8550_vdec_output_config_param_av1[] = {
+ HFI_PROP_OPB_ENABLE,
+ HFI_PROP_COLOR_FORMAT,
+ HFI_PROP_LINEAR_STRIDE_SCANLINE,
+};
+
static const u32 sm8550_venc_output_config_params[] = {
HFI_PROP_BITSTREAM_RESOLUTION,
HFI_PROP_CROP_OFFSETS,
@@ -737,11 +840,18 @@ static const u32 sm8550_vdec_subscribe_output_properties_vp9[] = {
HFI_PROP_PICTURE_TYPE,
};
+static const u32 sm8550_vdec_subscribe_output_properties_av1[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_WORST_COMPRESSION_RATIO,
+ HFI_PROP_WORST_COMPLEXITY_FACTOR,
+};
+
static const u32 sm8550_dec_ip_int_buf_tbl[] = {
BUF_BIN,
BUF_COMV,
BUF_NON_COMV,
BUF_LINE,
+ BUF_PARTIAL,
};
static const u32 sm8550_dec_op_int_buf_tbl[] = {
@@ -806,11 +916,18 @@ struct iris_platform_data sm8550_data = {
sm8550_vdec_input_config_param_vp9,
.dec_input_config_params_vp9_size =
ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
+ .dec_input_config_params_av1 =
+ sm8550_vdec_input_config_param_av1,
+ .dec_input_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
.dec_output_config_params =
sm8550_vdec_output_config_params,
.dec_output_config_params_size =
ARRAY_SIZE(sm8550_vdec_output_config_params),
-
+ .dec_output_config_params_av1 =
+ sm8550_vdec_output_config_param_av1,
+ .dec_output_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
.enc_input_config_params =
sm8550_venc_input_config_params,
.enc_input_config_params_size =
@@ -831,6 +948,9 @@ struct iris_platform_data sm8550_data = {
.dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
.dec_output_prop_vp9_size =
ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
+ .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
+ .dec_output_prop_av1_size =
+ ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
.dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
.dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
@@ -900,10 +1020,18 @@ struct iris_platform_data sm8650_data = {
sm8550_vdec_input_config_param_vp9,
.dec_input_config_params_vp9_size =
ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
+ .dec_input_config_params_av1 =
+ sm8550_vdec_input_config_param_av1,
+ .dec_input_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
.dec_output_config_params =
sm8550_vdec_output_config_params,
.dec_output_config_params_size =
ARRAY_SIZE(sm8550_vdec_output_config_params),
+ .dec_output_config_params_av1 =
+ sm8550_vdec_output_config_param_av1,
+ .dec_output_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
.enc_input_config_params =
sm8550_venc_input_config_params,
@@ -925,6 +1053,9 @@ struct iris_platform_data sm8650_data = {
.dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
.dec_output_prop_vp9_size =
ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
+ .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
+ .dec_output_prop_av1_size =
+ ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
.dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
.dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
index 8342a0b6f825b385192baedfc67c50bba7661470..0297756a3724181498ed8e5776dd5b872f1d4304 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
@@ -11,6 +11,7 @@
#include "iris_hfi_gen1_defines.h"
#include "iris_vpu_buffer.h"
#include "iris_vpu_common.h"
+#include "iris_instance.h"
#define BITRATE_MIN 32000
#define BITRATE_MAX 160000000
diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
index d38d0f6961cd5cb9929e2aecbec7353dcc2d4a7d..05eec18f937a51d3c081d0f98a8ffa6bbd30d166 100644
--- a/drivers/media/platform/qcom/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/iris/iris_vidc.c
@@ -178,6 +178,7 @@ int iris_open(struct file *filp)
INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_1].list);
INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_2].list);
INIT_LIST_HEAD(&inst->buffers[BUF_VPSS].list);
+ INIT_LIST_HEAD(&inst->buffers[BUF_PARTIAL].list);
init_completion(&inst->completion);
init_completion(&inst->flush_completion);
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 5/5] media: iris: Add internal buffer calculation for AV1 decoder
2025-10-17 18:35 [PATCH v2 0/5] Enable support for AV1 stateful decoder Deepa Guthyappa Madivalara
` (3 preceding siblings ...)
2025-10-17 18:35 ` [PATCH v2 4/5] media: iris: Define AV1-specific platform capabilities and properties Deepa Guthyappa Madivalara
@ 2025-10-17 18:35 ` Deepa Guthyappa Madivalara
2025-10-27 9:07 ` Dikshita Agarwal
4 siblings, 1 reply; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-17 18:35 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Deepa Guthyappa Madivalara
Implement internal buffer count and size calculations for AV1 decoder
for all the buffer types required by the AV1 decoder, including BIN,
COMV, PERSIST, LINE, and PARTIAL.
This ensures the hardware decoder has properly allocated memory for AV1
decoding operations, enabling correct AV1 video playback.
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
drivers/media/platform/qcom/iris/iris_buffer.h | 1 +
drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 298 ++++++++++++++++++++-
drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 116 ++++++++
3 files changed, 411 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
index 5ef365d9236c7cbdee24a4614789b3191881968b..75bb767761824c4c02e0df9b765896cc093be333 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_buffer.h
@@ -27,6 +27,7 @@ struct iris_inst;
* @BUF_SCRATCH_1: buffer to store decoding/encoding context data for HW
* @BUF_SCRATCH_2: buffer to store encoding context data for HW
* @BUF_VPSS: buffer to store VPSS context data for HW
+ * @BUF_PARTIAL: buffer for AV1 IBC data
* @BUF_TYPE_MAX: max buffer types
*/
enum iris_buffer_type {
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
index 4463be05ce165adef6b152eb0c155d2e6a7b3c36..e03ae7cfc9551dd2450b27d5d19ef1d23bba4c99 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
@@ -9,6 +9,17 @@
#include "iris_hfi_gen2_defines.h"
#define HFI_MAX_COL_FRAME 6
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_HEIGHT (8)
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_WIDTH (32)
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_HEIGHT (8)
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_WIDTH (16)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_HEIGHT (4)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_WIDTH (48)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_HEIGHT (4)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_WIDTH (24)
+#define AV1D_SIZE_BSE_COL_MV_64x64 512
+#define AV1D_SIZE_BSE_COL_MV_128x128 2816
+
#ifndef SYSTEM_LAL_TILE10
#define SYSTEM_LAL_TILE10 192
@@ -39,6 +50,31 @@ static u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_p
return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h, num_vpp_pipes);
}
+static u32 size_av1d_hw_bin_buffer(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+ u32 size_yuv, size_bin_hdr, size_bin_res;
+
+ size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ?
+ ((BIN_BUFFER_THRESHOLD * 3) >> 1) :
+ ((frame_width * frame_height * 3) >> 1);
+ size_bin_hdr = size_yuv * AV1_CABAC_HDR_RATIO_HD_TOT;
+ size_bin_res = size_yuv * AV1_CABAC_RES_RATIO_HD_TOT;
+ size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+ size_bin_res = ALIGN(size_bin_res / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+
+ return size_bin_hdr + size_bin_res;
+}
+
+static u32 hfi_buffer_bin_av1d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+ u32 n_aligned_h = ALIGN(frame_height, 16);
+ u32 n_aligned_w = ALIGN(frame_width, 16);
+
+ return size_av1d_hw_bin_buffer(n_aligned_w, n_aligned_h, num_vpp_pipes);
+}
+
static u32 size_h265d_hw_bin_buffer(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
{
u32 product = frame_width * frame_height;
@@ -110,6 +146,26 @@ static u32 hfi_buffer_comv_h265d(u32 frame_width, u32 frame_height, u32 _comv_bu
return (_size * (_comv_bufcount)) + 512;
}
+static u32 num_lcu(u32 frame_width, u32 frame_height, u32 lcu_size)
+{
+ return ((frame_width + lcu_size - 1) / lcu_size) *
+ ((frame_height + lcu_size - 1) / lcu_size);
+}
+
+static u32 hfi_buffer_comv_av1d(u32 frame_width, u32 frame_height, u32 comv_bufcount)
+{
+ u32 size;
+
+ size = 2 * ALIGN(max(num_lcu(frame_width, frame_height, 64) *
+ AV1D_SIZE_BSE_COL_MV_64x64,
+ num_lcu(frame_width, frame_height, 128) *
+ AV1D_SIZE_BSE_COL_MV_128x128),
+ DMA_ALIGNMENT);
+ size *= comv_bufcount;
+
+ return size;
+}
+
static u32 size_h264d_bse_cmd_buf(u32 frame_height)
{
u32 height = ALIGN(frame_height, 32);
@@ -174,6 +230,20 @@ static u32 hfi_buffer_persist_h264d(void)
DMA_ALIGNMENT);
}
+static u32 hfi_buffer_persist_av1d(u32 max_width, u32 max_height, u32 total_ref_count)
+{
+ u32 comv_size, size;
+
+ comv_size = hfi_buffer_comv_av1d(max_width, max_height, total_ref_count);
+ size = ALIGN((SIZE_AV1D_SEQUENCE_HEADER * 2 + SIZE_AV1D_METADATA +
+ AV1D_NUM_HW_PIC_BUF * (SIZE_AV1D_TILE_OFFSET + SIZE_AV1D_QM) +
+ AV1D_NUM_FRAME_HEADERS * (SIZE_AV1D_FRAME_HEADER +
+ 2 * SIZE_AV1D_PROB_TABLE) + comv_size + HDR10_HIST_EXTRADATA_SIZE +
+ SIZE_AV1D_METADATA * AV1D_NUM_HW_PIC_BUF), DMA_ALIGNMENT);
+
+ return ALIGN(size, DMA_ALIGNMENT);
+}
+
static u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
{
u32 size_bse = size_h264d_bse_cmd_buf(frame_height);
@@ -459,6 +529,182 @@ static u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height,
return ALIGN((size + vpss_lb_size), DMA_ALIGNMENT);
}
+static u32 size_av1d_lb_opb_wr1_nv12_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 size, y_width, y_width_a = 128;
+
+ y_width = ALIGN(frame_width, y_width_a);
+
+ size = ((y_width + HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_WIDTH - 1) /
+ HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_WIDTH +
+ (AV1D_MAX_TILE_COLS - 1));
+ return size * 256;
+}
+
+static u32 size_av1d_lb_opb_wr1_tp10_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 size, y_width, y_width_a = 256;
+
+ y_width = ALIGN(frame_width, y_width_a);
+
+ size = ((y_width + HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_WIDTH - 1) /
+ HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_WIDTH +
+ (AV1D_MAX_TILE_COLS - 1));
+
+ return size * 256;
+}
+
+static u32 hfi_buffer_line_av1d(u32 frame_width, u32 frame_height,
+ bool is_opb, u32 num_vpp_pipes)
+{
+ u32 size, vpss_lb_size, opbwrbufsize, opbwr8, opbwr10;
+
+ size = ALIGN(size_av1d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_fe_left_data(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_av1d_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_av1d_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_av1d_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_recon_dma_metadata_wr
+ (frame_width, frame_height), DMA_ALIGNMENT) * 2 +
+ ALIGN(size_av1d_qp(frame_width, frame_height), DMA_ALIGNMENT);
+ opbwr8 = size_av1d_lb_opb_wr1_nv12_ubwc(frame_width, frame_height);
+ opbwr10 = size_av1d_lb_opb_wr1_tp10_ubwc(frame_width, frame_height);
+ opbwrbufsize = opbwr8 >= opbwr10 ? opbwr8 : opbwr10;
+ size = ALIGN((size + opbwrbufsize), DMA_ALIGNMENT);
+ if (is_opb) {
+ vpss_lb_size = size_vpss_lb(frame_width, frame_height);
+ size = ALIGN((size + vpss_lb_size) * 2, DMA_ALIGNMENT);
+ }
+
+ return size;
+}
+
+static u32 size_av1d_ibc_nv12_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 size;
+ u32 y_width_a = 128, y_height_a = 32;
+ u32 uv_width_a = 128, uv_height_a = 32;
+ u32 ybufsize, uvbufsize, y_width, y_height, uv_width, uv_height;
+ u32 y_meta_width_a = 64, y_meta_height_a = 16;
+ u32 uv_meta_width_a = 64, uv_meta_height_a = 16;
+ u32 meta_height, meta_stride, meta_size;
+ u32 tile_width_y = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_WIDTH;
+ u32 tile_height_y = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_HEIGHT;
+ u32 tile_width_uv = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_WIDTH;
+ u32 tile_height_uv = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_HEIGHT;
+
+ y_width = ALIGN(frame_width, y_width_a);
+ y_height = ALIGN(frame_height, y_height_a);
+ uv_width = ALIGN(frame_width, uv_width_a);
+ uv_height = ALIGN(((frame_height + 1) >> 1), uv_height_a);
+ ybufsize = ALIGN((y_width * y_height), HFI_ALIGNMENT_4096);
+ uvbufsize = ALIGN(uv_width * uv_height, HFI_ALIGNMENT_4096);
+ size = ybufsize + uvbufsize;
+ meta_stride = ALIGN(((frame_width + (tile_width_y - 1)) / tile_width_y),
+ y_meta_width_a);
+ meta_height = ALIGN(((frame_height + (tile_height_y - 1)) / tile_height_y),
+ y_meta_height_a);
+ meta_size = ALIGN(meta_stride * meta_height, HFI_ALIGNMENT_4096);
+ size += meta_size;
+ meta_stride = ALIGN(((((frame_width + 1) >> 1) + (tile_width_uv - 1)) /
+ tile_width_uv), uv_meta_width_a);
+ meta_height = ALIGN(((((frame_height + 1) >> 1) + (tile_height_uv - 1)) /
+ tile_height_uv), uv_meta_height_a);
+ meta_size = ALIGN(meta_stride * meta_height, HFI_ALIGNMENT_4096);
+ size += meta_size;
+
+ return size;
+}
+
+static u32 hfi_yuv420_tp10_calc_y_stride(u32 frame_width, u32 stride_multiple)
+{
+ u32 stride;
+
+ stride = ALIGN(frame_width, 192);
+ stride = ALIGN(stride * 4 / 3, stride_multiple);
+
+ return stride;
+}
+
+static u32 hfi_yuv420_tp10_calc_y_bufheight(u32 frame_height, u32 min_buf_height_multiple)
+{
+ return ALIGN(frame_height, min_buf_height_multiple);
+}
+
+static u32 hfi_yuv420_tp10_calc_uv_stride(u32 frame_width, u32 stride_multiple)
+{
+ u32 stride;
+
+ stride = ALIGN(frame_width, 192);
+ stride = ALIGN(stride * 4 / 3, stride_multiple);
+
+ return stride;
+}
+
+static u32 hfi_yuv420_tp10_calc_uv_bufheight(u32 frame_height, u32 min_buf_height_multiple)
+{
+ return ALIGN(((frame_height + 1) >> 1), min_buf_height_multiple);
+}
+
+static u32 size_av1d_ibc_tp10_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 size;
+ u32 y_width_a = 256, y_height_a = 16,
+ uv_width_a = 256, uv_height_a = 16;
+ u32 ybufsize, uvbufsize, y_width, y_height, uv_width, uv_height;
+ u32 y_meta_width_a = 64, y_meta_height_a = 16,
+ uv_meta_width_a = 64, uv_meta_height_a = 16;
+ u32 meta_height, meta_stride, meta_size;
+ u32 tile_width_y = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_WIDTH;
+ u32 tile_height_y = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_HEIGHT;
+ u32 tile_width_uv = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_WIDTH;
+ u32 tile_height_uv = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_HEIGHT;
+
+ y_width = hfi_yuv420_tp10_calc_y_stride(frame_width, y_width_a);
+ y_height = hfi_yuv420_tp10_calc_y_bufheight(frame_height, y_height_a);
+ uv_width = hfi_yuv420_tp10_calc_uv_stride(frame_width, uv_width_a);
+ uv_height = hfi_yuv420_tp10_calc_uv_bufheight(frame_height, uv_height_a);
+ ybufsize = ALIGN(y_width * y_height, HFI_ALIGNMENT_4096);
+ uvbufsize = ALIGN(uv_width * uv_height, HFI_ALIGNMENT_4096);
+ size = ybufsize + uvbufsize;
+ meta_stride = ALIGN(((frame_width + (tile_width_y - 1)) / tile_width_y),
+ y_meta_width_a);
+ meta_height = ALIGN(((frame_height + (tile_height_y - 1)) / tile_height_y),
+ y_meta_height_a);
+ meta_size = ALIGN(meta_stride * meta_height, HFI_ALIGNMENT_4096);
+ size += meta_size;
+ meta_stride = ALIGN(((((frame_width + 1) >> 1) + (tile_width_uv - 1)) /
+ tile_width_uv), uv_meta_width_a);
+ meta_height = ALIGN(((((frame_height + 1) >> 1) + (tile_height_uv - 1)) /
+ tile_height_uv), uv_meta_height_a);
+ meta_size = ALIGN(meta_stride * meta_height, HFI_ALIGNMENT_4096);
+ size += meta_size;
+
+ return size;
+}
+
+static u32 hfi_buffer_ibc_av1d(u32 frame_width, u32 frame_height)
+{
+ u32 size, ibc8, ibc10;
+
+ ibc8 = size_av1d_ibc_nv12_ubwc(frame_width, frame_height);
+ ibc10 = size_av1d_ibc_tp10_ubwc(frame_width, frame_height);
+ size = ibc8 >= ibc10 ? ibc8 : ibc10;
+
+ return ALIGN(size, DMA_ALIGNMENT);
+}
+
static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
{
u32 num_vpp_pipes = inst->core->iris_platform_data->num_vpp_pipe;
@@ -472,6 +718,8 @@ static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
return hfi_buffer_bin_h265d(width, height, num_vpp_pipes);
else if (inst->codec == V4L2_PIX_FMT_VP9)
return hfi_buffer_bin_vp9d(width, height, num_vpp_pipes);
+ else if (inst->codec == V4L2_PIX_FMT_AV1)
+ return hfi_buffer_bin_av1d(width, height, num_vpp_pipes);
return 0;
}
@@ -487,18 +735,33 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
return hfi_buffer_comv_h264d(width, height, num_comv);
else if (inst->codec == V4L2_PIX_FMT_HEVC)
return hfi_buffer_comv_h265d(width, height, num_comv);
-
+ else if (inst->codec == V4L2_PIX_FMT_AV1) {
+ if (inst->fw_caps[DRAP].value)
+ return 0;
+ else
+ return hfi_buffer_comv_av1d(width, height, num_comv);
+ }
return 0;
}
static u32 iris_vpu_dec_persist_size(struct iris_inst *inst)
{
+ struct platform_inst_caps *caps;
+
if (inst->codec == V4L2_PIX_FMT_H264)
return hfi_buffer_persist_h264d();
else if (inst->codec == V4L2_PIX_FMT_HEVC)
return hfi_buffer_persist_h265d(0);
else if (inst->codec == V4L2_PIX_FMT_VP9)
return hfi_buffer_persist_vp9d();
+ else if (inst->codec == V4L2_PIX_FMT_AV1) {
+ caps = inst->core->iris_platform_data->inst_caps;
+ if (inst->fw_caps[DRAP].value)
+ return hfi_buffer_persist_av1d(caps->max_frame_width,
+ caps->max_frame_height, 16);
+ else
+ return hfi_buffer_persist_av1d(0, 0, 0);
+ }
return 0;
}
@@ -545,6 +808,8 @@ static u32 iris_vpu_dec_line_size(struct iris_inst *inst)
else if (inst->codec == V4L2_PIX_FMT_VP9)
return hfi_buffer_line_vp9d(width, height, out_min_count, is_opb,
num_vpp_pipes);
+ else if (inst->codec == V4L2_PIX_FMT_AV1)
+ return hfi_buffer_line_av1d(width, height, is_opb, num_vpp_pipes);
return 0;
}
@@ -653,6 +918,15 @@ static u32 iris_vpu_enc_bin_size(struct iris_inst *inst)
num_vpp_pipes, inst->hfi_rc_type);
}
+static u32 iris_vpu_dec_partial_size(struct iris_inst *inst)
+{
+ struct v4l2_format *f = inst->fmt_src;
+ u32 height = f->fmt.pix_mp.height;
+ u32 width = f->fmt.pix_mp.width;
+
+ return hfi_buffer_ibc_av1d(width, height);
+}
+
static inline
u32 hfi_buffer_comv_enc(u32 frame_width, u32 frame_height, u32 lcu_size,
u32 num_recon, u32 standard)
@@ -1414,7 +1688,9 @@ static int output_min_count(struct iris_inst *inst)
/* fw_min_count > 0 indicates reconfig event has already arrived */
if (inst->fw_min_count) {
- if (iris_split_mode_enabled(inst) && inst->codec == V4L2_PIX_FMT_VP9)
+ if (iris_split_mode_enabled(inst) &&
+ (inst->codec == V4L2_PIX_FMT_VP9 ||
+ inst->codec == V4L2_PIX_FMT_VP9))
return min_t(u32, 4, inst->fw_min_count);
else
return inst->fw_min_count;
@@ -1422,6 +1698,8 @@ static int output_min_count(struct iris_inst *inst)
if (inst->codec == V4L2_PIX_FMT_VP9)
output_min_count = 9;
+ else if (inst->codec == V4L2_PIX_FMT_AV1)
+ output_min_count = 11;
return output_min_count;
}
@@ -1444,6 +1722,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)
{BUF_PERSIST, iris_vpu_dec_persist_size },
{BUF_DPB, iris_vpu_dec_dpb_size },
{BUF_SCRATCH_1, iris_vpu_dec_scratch1_size },
+ {BUF_PARTIAL, iris_vpu_dec_partial_size },
};
static const struct iris_vpu_buf_type_handle enc_internal_buf_type_handle[] = {
@@ -1510,14 +1789,20 @@ static u32 internal_buffer_count(struct iris_inst *inst,
buffer_type == BUF_PERSIST) {
return 1;
} else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
- if (inst->codec == V4L2_PIX_FMT_H264 || inst->codec == V4L2_PIX_FMT_HEVC)
+ if (inst->codec == V4L2_PIX_FMT_H264 ||
+ inst->codec == V4L2_PIX_FMT_HEVC ||
+ inst->codec == V4L2_PIX_FMT_AV1)
return 1;
}
+
return 0;
}
static inline int iris_vpu_dpb_count(struct iris_inst *inst)
{
+ if (inst->codec == V4L2_PIX_FMT_AV1)
+ return 11;
+
if (iris_split_mode_enabled(inst)) {
return inst->fw_min_count ?
inst->fw_min_count : inst->buffers[BUF_OUTPUT].min_count;
@@ -1536,9 +1821,13 @@ int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type
return MIN_BUFFERS;
else
return output_min_count(inst);
+ case BUF_NON_COMV:
+ if (inst->codec == V4L2_PIX_FMT_AV1)
+ return 0;
+ else
+ return 1;
case BUF_BIN:
case BUF_COMV:
- case BUF_NON_COMV:
case BUF_LINE:
case BUF_PERSIST:
return internal_buffer_count(inst, buffer_type);
@@ -1546,6 +1835,7 @@ int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type
case BUF_SCRATCH_2:
case BUF_VPSS:
case BUF_ARP:
+ case BUF_PARTIAL:
return 1; /* internal buffer count needed by firmware is 1 */
case BUF_DPB:
return iris_vpu_dpb_count(inst);
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
index 04f0b7400a1e4e1d274d690a2761b9e57778e8b7..a8ffc1286260bddbf49df18fba0a23f78056a484 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
@@ -11,6 +11,7 @@ struct iris_inst;
#define MIN_BUFFERS 4
#define DMA_ALIGNMENT 256
+#define HFI_ALIGNMENT_4096 4096
#define NUM_HW_PIC_BUF 32
#define LCU_MAX_SIZE_PELS 64
@@ -81,6 +82,22 @@ struct iris_inst;
#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE 384
#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640
+#define AV1_CABAC_HDR_RATIO_HD_TOT 2
+#define AV1_CABAC_RES_RATIO_HD_TOT 2
+#define AV1D_LCU_MAX_SIZE_PELS 128
+#define AV1D_LCU_MIN_SIZE_PELS 64
+#define AV1D_MAX_TILE_COLS 64
+#define MAX_PE_NBR_DATA_LCU32_LINE_BUFFER_SIZE 192
+#define MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE 96
+#define AV1D_NUM_HW_PIC_BUF 16
+#define AV1D_NUM_FRAME_HEADERS 16
+#define SIZE_AV1D_SEQUENCE_HEADER 768
+#define SIZE_AV1D_METADATA 512
+#define SIZE_AV1D_FRAME_HEADER 1280
+#define SIZE_AV1D_TILE_OFFSET 65536
+#define SIZE_AV1D_QM 3328
+#define SIZE_AV1D_PROB_TABLE 22784
+
#define SIZE_SLICE_CMD_BUFFER (ALIGN(20480, 256))
#define SIZE_SPS_PPS_SLICE_HDR (2048 + 4096)
#define SIZE_BSE_SLICE_CMD_BUF ((((8192 << 2) + 7) & (~7)) * 3)
@@ -101,6 +118,15 @@ struct iris_inst;
#define NUM_MBS_4K (DIV_ROUND_UP(MAX_WIDTH, 16) * DIV_ROUND_UP(MAX_HEIGHT, 16))
#define NUM_MBS_720P (((ALIGN(1280, 16)) >> 4) * ((ALIGN(736, 16)) >> 4))
+#define BITS_PER_PIX 16
+#define NUM_LINES_LUMA 10
+#define NUM_LINES_CHROMA 6
+#define AV1D_LCU_MAX_SIZE_PELS 128
+#define AV1D_LCU_MIN_SIZE_PELS 64
+#define AV1D_MAX_TILE_COLS 64
+#define BITS_PER_CTRL_PACK 128
+#define NUM_CTRL_PACK_LCU 10
+
static inline u32 size_h264d_lb_fe_top_data(u32 frame_width)
{
return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * ALIGN(frame_width, 16) * 3;
@@ -146,6 +172,96 @@ static inline u32 size_h264d_qp(u32 frame_width, u32 frame_height)
return DIV_ROUND_UP(frame_width, 64) * DIV_ROUND_UP(frame_height, 64) * 128;
}
+static inline u32 size_av1d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return (ALIGN(frame_width, AV1D_LCU_MAX_SIZE_PELS) *
+ ((BITS_PER_PIX * NUM_LINES_LUMA) >> 3) +
+ ALIGN(frame_width, AV1D_LCU_MAX_SIZE_PELS) / 2 *
+ ((BITS_PER_PIX * NUM_LINES_CHROMA) >> 3) * 2);
+}
+
+static inline u32 size_av1d_lb_fe_left_data(u32 frame_width, u32 frame_height)
+{
+ return (32 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 2 +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 8) * 2 +
+ 24 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 24 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 2 +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 12) * 2 +
+ 24 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 2 +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 12) * 2);
+}
+
+static inline u32 size_av1d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (NUM_CTRL_PACK_LCU * ((frame_width + AV1D_LCU_MIN_SIZE_PELS - 1) /
+ AV1D_LCU_MIN_SIZE_PELS) * BITS_PER_CTRL_PACK / 8);
+}
+
+static inline u32 size_av1d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (16 * ((ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 16) +
+ (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS)) +
+ 3 * 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS));
+}
+
+static inline u32 size_av1d_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (((frame_width + 7) / 8) * MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE);
+}
+
+static inline u32 size_av1d_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (max(((frame_height + 15) / 16) *
+ MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max(((frame_height + 31) / 32) *
+ MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 63) / 64) *
+ MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)));
+}
+
+static inline u32 size_av1d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return (max(((frame_width + 15) / 16) *
+ MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE,
+ max(((frame_width + 31) / 32) *
+ MAX_PE_NBR_DATA_LCU32_LINE_BUFFER_SIZE,
+ ((frame_width + 63) / 64) *
+ MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE)));
+}
+
+static inline u32 size_av1d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return (max(((frame_width + 63) / 64) * 1280,
+ ((frame_width + 127) / 128) * 2304));
+}
+
+static inline u32 size_av1d_lb_recon_dma_metadata_wr(u32 frame_width,
+ u32 frame_height)
+{
+ return ((ALIGN(frame_height, 8) / (4 / 2)) * 64);
+}
+
+static inline u32 size_av1d_qp(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_qp(frame_width, frame_height);
+}
+
u32 iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type);
u32 iris_vpu33_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type);
int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type);
--
2.34.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH v2 1/5] media: uapi: videodev2: Add support for AV1 stateful decoder
2025-10-17 18:35 ` [PATCH v2 1/5] media: uapi: videodev2: Add " Deepa Guthyappa Madivalara
@ 2025-10-17 20:01 ` Nicolas Dufresne
2025-10-20 21:55 ` Deepa Guthyappa Madivalara
0 siblings, 1 reply; 14+ messages in thread
From: Nicolas Dufresne @ 2025-10-17 20:01 UTC (permalink / raw)
To: Deepa Guthyappa Madivalara, Mauro Carvalho Chehab, Vikash Garodia,
Dikshita Agarwal, Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot
[-- Attachment #1: Type: text/plain, Size: 3180 bytes --]
Le vendredi 17 octobre 2025 à 11:35 -0700, Deepa Guthyappa Madivalara a écrit :
> Introduce a new pixel format, V4L2_PIX_FMT_AV1, to the
> Video4Linux2(V4L2) API. This format is intended for AV1
> bitstreams in stateful decoding/encoding workflows.
> The fourcc code 'AV10' is used to distinguish
> this format from the existing V4L2_PIX_FMT_AV1_FRAME,
> which is used for stateless AV1 decoder implementation.
>
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> ---
> Documentation/userspace-api/media/v4l/pixfmt-compressed.rst | 8 ++++++++
> include/uapi/linux/videodev2.h | 1 +
> 2 files changed, 9 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
> index 806ed73ac474ce0e6df00f902850db9fd0db240e..d82557a6ac30851b4fd432a5ceab969abaed2e25 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
> @@ -274,6 +274,14 @@ Compressed Formats
> of macroblocks to decode a full corresponding frame to the matching
> capture buffer.
>
> + * .. _V4L2-PIX-FMT-AV1:
> +
> + - ``V4L2_PIX_FMT_AV1``
> + - 'AV10'
> + - AV1 compressed video frame. This format is adapted for implementing AV1
> + pipeline. The decoder implements stateful video decoder and expects one
> + Temporal Unit per buffer from OBU-stream or AnnexB.
> + The encoder generates one Temporal Unit per buffer.
> .. raw:: latex
>
> \normalsize
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index becd08fdbddb857f8f2bf205d2164dc6e20e80b2..e03cfe0a5ae830595f0bf69b951ca16beaedab72 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -775,6 +775,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
> #define V4L2_PIX_FMT_AV1_FRAME v4l2_fourcc('A', 'V', '1', 'F') /* AV1 parsed frame */
> +#define V4L2_PIX_FMT_AV1 v4l2_fourcc('A', 'V', '1', '0') /* AV1 */
An alternative fourcc could be to match the one used in the ISO specification,
which is also what browsers and Android uses. It would be "av01", link to the
spec:
http://downloads.aomedia.org/assets/pdf/AV1-ISO-Base-Media-File-Format-Binding-Specification.pdf
Though, we never had any requirement in this direction, its just an idea. With
or without, I'm happy with the definition.
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
cheers,
Nicolas
> #define V4L2_PIX_FMT_SPK v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
> #define V4L2_PIX_FMT_RV30 v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
> #define V4L2_PIX_FMT_RV40 v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
2025-10-17 18:35 ` [PATCH v2 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() Deepa Guthyappa Madivalara
@ 2025-10-17 20:02 ` Nicolas Dufresne
0 siblings, 0 replies; 14+ messages in thread
From: Nicolas Dufresne @ 2025-10-17 20:02 UTC (permalink / raw)
To: Deepa Guthyappa Madivalara, Mauro Carvalho Chehab, Vikash Garodia,
Dikshita Agarwal, Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot
[-- Attachment #1: Type: text/plain, Size: 1369 bytes --]
Le vendredi 17 octobre 2025 à 11:35 -0700, Deepa Guthyappa Madivalara a écrit :
> Add a descriptive string for the AV1 pixel format to v4l_fill_fmtdesc(),
> enabling proper reporting of AV1 support via VIDIOC_ENUM_FMT.
>
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-
> core/v4l2-ioctl.c
> index
> 01cf52c3ea33e1a01e1b306036ba4e57ef5c95d0..bdfdf45c5de2f2ce885f219007718a54b5c8
> 6251 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1542,6 +1542,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_PIX_FMT_QC10C: descr = "QCOM Compressed 10-
> bit Format"; break;
> case V4L2_PIX_FMT_AJPG: descr = "Aspeed
> JPEG"; break;
> case V4L2_PIX_FMT_AV1_FRAME: descr = "AV1 Frame"; break;
> + case V4L2_PIX_FMT_AV1: descr = "AV1 OBU stream";
> break;
> case V4L2_PIX_FMT_MT2110T: descr = "Mediatek 10bit Tile
> Mode"; break;
> case V4L2_PIX_FMT_MT2110R: descr = "Mediatek 10bit
> Raster Mode"; break;
> case V4L2_PIX_FMT_HEXTILE: descr = "Hextile Compressed
> Format"; break;
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 1/5] media: uapi: videodev2: Add support for AV1 stateful decoder
2025-10-17 20:01 ` Nicolas Dufresne
@ 2025-10-20 21:55 ` Deepa Guthyappa Madivalara
0 siblings, 0 replies; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-20 21:55 UTC (permalink / raw)
To: Nicolas Dufresne, Mauro Carvalho Chehab, Vikash Garodia,
Dikshita Agarwal, Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot
On 10/17/2025 1:01 PM, Nicolas Dufresne wrote:
> Le vendredi 17 octobre 2025 à 11:35 -0700, Deepa Guthyappa Madivalara a écrit :
>> Introduce a new pixel format, V4L2_PIX_FMT_AV1, to the
>> Video4Linux2(V4L2) API. This format is intended for AV1
>> bitstreams in stateful decoding/encoding workflows.
>> The fourcc code 'AV10' is used to distinguish
>> this format from the existing V4L2_PIX_FMT_AV1_FRAME,
>> which is used for stateless AV1 decoder implementation.
>>
>> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
>> ---
>> Documentation/userspace-api/media/v4l/pixfmt-compressed.rst | 8 ++++++++
>> include/uapi/linux/videodev2.h | 1 +
>> 2 files changed, 9 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
>> index 806ed73ac474ce0e6df00f902850db9fd0db240e..d82557a6ac30851b4fd432a5ceab969abaed2e25 100644
>> --- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
>> @@ -274,6 +274,14 @@ Compressed Formats
>> of macroblocks to decode a full corresponding frame to the matching
>> capture buffer.
>>
>> + * .. _V4L2-PIX-FMT-AV1:
>> +
>> + - ``V4L2_PIX_FMT_AV1``
>> + - 'AV10'
>> + - AV1 compressed video frame. This format is adapted for implementing AV1
>> + pipeline. The decoder implements stateful video decoder and expects one
>> + Temporal Unit per buffer from OBU-stream or AnnexB.
>> + The encoder generates one Temporal Unit per buffer.
>> .. raw:: latex
>>
>> \normalsize
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index becd08fdbddb857f8f2bf205d2164dc6e20e80b2..e03cfe0a5ae830595f0bf69b951ca16beaedab72 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -775,6 +775,7 @@ struct v4l2_pix_format {
>> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
>> #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
>> #define V4L2_PIX_FMT_AV1_FRAME v4l2_fourcc('A', 'V', '1', 'F') /* AV1 parsed frame */
>> +#define V4L2_PIX_FMT_AV1 v4l2_fourcc('A', 'V', '1', '0') /* AV1 */
> An alternative fourcc could be to match the one used in the ISO specification,
> which is also what browsers and Android uses. It would be "av01", link to the
> spec:
>
> http://downloads.aomedia.org/assets/pdf/AV1-ISO-Base-Media-File-Format-Binding-Specification.pdf
>
> Though, we never had any requirement in this direction, its just an idea. With
> or without, I'm happy with the definition.
"av01" sounds good. It would keep it consistent with the spec.
I will update it in v3.
> Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
>
> cheers,
> Nicolas
>
>> #define V4L2_PIX_FMT_SPK v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
>> #define V4L2_PIX_FMT_RV30 v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
>> #define V4L2_PIX_FMT_RV40 v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 3/5] media: iris: Add support for AV1 format in iris decoder
2025-10-17 18:35 ` [PATCH v2 3/5] media: iris: Add support for AV1 format in iris decoder Deepa Guthyappa Madivalara
@ 2025-10-27 6:04 ` Dikshita Agarwal
0 siblings, 0 replies; 14+ messages in thread
From: Dikshita Agarwal @ 2025-10-27 6:04 UTC (permalink / raw)
To: Deepa Guthyappa Madivalara, Mauro Carvalho Chehab, Vikash Garodia,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Bryan O'Donoghue
On 10/18/2025 12:05 AM, Deepa Guthyappa Madivalara wrote:
> Extend iris decoder driver to support format V4L2_PIX_FMT_AV1.
> This change updates the format enumeration (VIDIOC_ENUM_FMT)
> and allows setting AV1 format via VIDIOC_S_FMT for gen2 and beyond.
> Gen1 iris hardware decoder does not support AV1 format.
>
> Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> ---
> .../platform/qcom/iris/iris_hfi_gen2_defines.h | 1 +
> drivers/media/platform/qcom/iris/iris_instance.h | 1 +
> .../platform/qcom/iris/iris_platform_common.h | 2 ++
> .../media/platform/qcom/iris/iris_platform_gen2.c | 23 ++++++++++++++++++++
> .../platform/qcom/iris/iris_platform_sm8250.c | 17 +++++++++++++++
> drivers/media/platform/qcom/iris/iris_vdec.c | 25 ++++++----------------
> 6 files changed, 50 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> index aa1f795f5626c1f76a32dd650302633877ce67be..bbfe7a0851ea94fb7041a868b4df8b2ec63bf427 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> @@ -138,6 +138,7 @@ enum hfi_codec_type {
> HFI_CODEC_DECODE_HEVC = 3,
> HFI_CODEC_ENCODE_HEVC = 4,
> HFI_CODEC_DECODE_VP9 = 5,
> + HFI_CODEC_DECODE_AV1 = 7,
> };
>
> enum hfi_picture_type {
> diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
> index 5982d7adefeab80905478b32cddba7bd4651a691..f1883ffc138fd975fb76d4e45904ee04e196cd20 100644
> --- a/drivers/media/platform/qcom/iris/iris_instance.h
> +++ b/drivers/media/platform/qcom/iris/iris_instance.h
> @@ -19,6 +19,7 @@ enum iris_fmt_type {
> IRIS_FMT_H264,
> IRIS_FMT_HEVC,
> IRIS_FMT_VP9,
> + IRIS_FMT_AV1,
> };
>
> struct iris_fmt {
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
> index 58d05e0a112eed25faea027a34c719c89d6c3897..9aca70b4c0690f0d8d799e2a9976bd20d6bb8c94 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_common.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
> @@ -214,6 +214,8 @@ struct iris_platform_data {
> u64 dma_mask;
> const char *fwname;
> u32 pas_id;
> + struct iris_fmt *inst_iris_fmts;
> + u32 inst_iris_fmts_size;
> struct platform_inst_caps *inst_caps;
> struct platform_inst_fw_cap *inst_fw_caps_dec;
> u32 inst_fw_caps_dec_size;
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> index 36d69cc73986b74534a2912524c8553970fd862e..15a38b7122af6d978fac433e8c02b4da5c141e1a 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> @@ -19,6 +19,25 @@
> #define VIDEO_ARCH_LX 1
> #define BITRATE_MAX 245000000
>
> +static struct iris_fmt platform_fmts_sm8550_dec[] = {
> + [IRIS_FMT_H264] = {
> + .pixfmt = V4L2_PIX_FMT_H264,
> + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + },
> + [IRIS_FMT_HEVC] = {
> + .pixfmt = V4L2_PIX_FMT_HEVC,
> + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + },
> + [IRIS_FMT_VP9] = {
> + .pixfmt = V4L2_PIX_FMT_VP9,
> + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + },
> + [IRIS_FMT_AV1] = {
> + .pixfmt = V4L2_PIX_FMT_AV1,
> + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + },
> +};
> +
> static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
> {
> .cap_id = PROFILE_H264,
> @@ -760,6 +779,8 @@ struct iris_platform_data sm8550_data = {
> .dma_mask = 0xe0000000 - 1,
> .fwname = "qcom/vpu/vpu30_p4.mbn",
> .pas_id = IRIS_PAS_ID,
> + .inst_iris_fmts = platform_fmts_sm8550_dec,
> + .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
> .inst_caps = &platform_inst_cap_sm8550,
> .inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
> .inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
> @@ -852,6 +873,8 @@ struct iris_platform_data sm8650_data = {
> .dma_mask = 0xe0000000 - 1,
> .fwname = "qcom/vpu/vpu33_p4.mbn",
> .pas_id = IRIS_PAS_ID,
> + .inst_iris_fmts = platform_fmts_sm8550_dec,
> + .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
> .inst_caps = &platform_inst_cap_sm8550,
> .inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
> .inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> index 16486284f8acccf6a95a27f6003e885226e28f4d..8342a0b6f825b385192baedfc67c50bba7661470 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> @@ -17,6 +17,21 @@
> #define BITRATE_PEAK_DEFAULT (BITRATE_DEFAULT * 2)
> #define BITRATE_STEP 100
>
> +static struct iris_fmt platform_fmts_sm8250_dec[] = {
> + [IRIS_FMT_H264] = {
> + .pixfmt = V4L2_PIX_FMT_H264,
> + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + },
> + [IRIS_FMT_HEVC] = {
> + .pixfmt = V4L2_PIX_FMT_HEVC,
> + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + },
> + [IRIS_FMT_VP9] = {
> + .pixfmt = V4L2_PIX_FMT_VP9,
> + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + },
> +};
> +
> static struct platform_inst_fw_cap inst_fw_cap_sm8250_dec[] = {
> {
> .cap_id = PIPE,
> @@ -337,6 +352,8 @@ struct iris_platform_data sm8250_data = {
> .dma_mask = 0xe0000000 - 1,
> .fwname = "qcom/vpu-1.0/venus.mbn",
> .pas_id = IRIS_PAS_ID,
> + .inst_iris_fmts = platform_fmts_sm8250_dec,
> + .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8250_dec),
> .inst_caps = &platform_inst_cap_sm8250,
> .inst_fw_caps_dec = inst_fw_cap_sm8250_dec,
> .inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8250_dec),
> diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
> index ae13c3e1b426bfd81a7b46dc6c3ff5eb5c4860cb..be8d2d48c82f385e4f46807f7e0dd52e469927cd 100644
> --- a/drivers/media/platform/qcom/iris/iris_vdec.c
> +++ b/drivers/media/platform/qcom/iris/iris_vdec.c
> @@ -67,26 +67,12 @@ void iris_vdec_inst_deinit(struct iris_inst *inst)
> kfree(inst->fmt_src);
> }
>
> -static const struct iris_fmt iris_vdec_formats[] = {
> - [IRIS_FMT_H264] = {
> - .pixfmt = V4L2_PIX_FMT_H264,
> - .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> - },
> - [IRIS_FMT_HEVC] = {
> - .pixfmt = V4L2_PIX_FMT_HEVC,
> - .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> - },
> - [IRIS_FMT_VP9] = {
> - .pixfmt = V4L2_PIX_FMT_VP9,
> - .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> - },
> -};
> -
> static const struct iris_fmt *
> find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
> {
> - unsigned int size = ARRAY_SIZE(iris_vdec_formats);
> - const struct iris_fmt *fmt = iris_vdec_formats;
> + unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
> + const struct iris_fmt *fmt =
> + inst->core->iris_platform_data->inst_iris_fmts;
> unsigned int i;
>
> for (i = 0; i < size; i++) {
> @@ -103,8 +89,9 @@ find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
> static const struct iris_fmt *
> find_format_by_index(struct iris_inst *inst, u32 index, u32 type)
> {
> - const struct iris_fmt *fmt = iris_vdec_formats;
> - unsigned int size = ARRAY_SIZE(iris_vdec_formats);
> + unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
> + const struct iris_fmt *fmt =
> + inst->core->iris_platform_data->inst_iris_fmts;
>
> if (index >= size || fmt[index].type != type)
> return NULL;
>
Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
Thanks,
Dikshita
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 4/5] media: iris: Define AV1-specific platform capabilities and properties
2025-10-17 18:35 ` [PATCH v2 4/5] media: iris: Define AV1-specific platform capabilities and properties Deepa Guthyappa Madivalara
@ 2025-10-27 9:07 ` Dikshita Agarwal
2025-10-29 2:46 ` Deepa Guthyappa Madivalara
0 siblings, 1 reply; 14+ messages in thread
From: Dikshita Agarwal @ 2025-10-27 9:07 UTC (permalink / raw)
To: Deepa Guthyappa Madivalara, Mauro Carvalho Chehab, Vikash Garodia,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Bryan O'Donoghue
On 10/18/2025 12:05 AM, Deepa Guthyappa Madivalara wrote:
> Defining platform specific capabilities specific to AV1 decoder.
> Set and subscribe to manadatory properties to firmware for AV1.
>
> Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> ---
> drivers/media/platform/qcom/iris/iris_buffer.h | 1 +
> drivers/media/platform/qcom/iris/iris_ctrls.c | 8 ++
> drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
> .../platform/qcom/iris/iris_hfi_gen2_command.c | 110 ++++++++++++++++-
> .../platform/qcom/iris/iris_hfi_gen2_defines.h | 9 ++
> .../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++++
> .../platform/qcom/iris/iris_platform_common.h | 13 ++
> .../media/platform/qcom/iris/iris_platform_gen2.c | 133 ++++++++++++++++++++-
> .../platform/qcom/iris/iris_platform_sm8250.c | 1 +
> drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
> 10 files changed, 296 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
> index 325d30fce5c99185b61ff989fbfd4de9a56762b2..5ef365d9236c7cbdee24a4614789b3191881968b 100644
> --- a/drivers/media/platform/qcom/iris/iris_buffer.h
> +++ b/drivers/media/platform/qcom/iris/iris_buffer.h
> @@ -42,6 +42,7 @@ enum iris_buffer_type {
> BUF_SCRATCH_1,
> BUF_SCRATCH_2,
> BUF_VPSS,
> + BUF_PARTIAL,
> BUF_TYPE_MAX,
> };
>
> diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
> index 754a5ad718bc37630bb861012301df7a2e7342a1..620c7e1bd273e25febd8ca70dd1dcfb0b862692b 100644
> --- a/drivers/media/platform/qcom/iris/iris_ctrls.c
> +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
> @@ -98,6 +98,10 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
> return B_FRAME_QP_H264;
> case V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP:
> return B_FRAME_QP_HEVC;
> + case V4L2_CID_MPEG_VIDEO_AV1_PROFILE:
> + return PROFILE_AV1;
> + case V4L2_CID_MPEG_VIDEO_AV1_LEVEL:
> + return LEVEL_AV1;
> default:
> return INST_FW_CAP_MAX;
> }
> @@ -185,6 +189,10 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
> return V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP;
> case B_FRAME_QP_HEVC:
> return V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP;
> + case PROFILE_AV1:
> + return V4L2_CID_MPEG_VIDEO_AV1_PROFILE;
> + case LEVEL_AV1:
> + return V4L2_CID_MPEG_VIDEO_AV1_LEVEL;
> default:
> return 0;
> }
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h
> index b51471fb32c70acee44c37f8e9dce0c6bc0b6ccc..3edb5ae582b49bea2e2408c4a5cfc0a742adc05f 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h
> @@ -141,6 +141,9 @@ struct hfi_subscription_params {
> u32 profile;
> u32 level;
> u32 tier;
> + u32 drap;
> + u32 film_grain;
> + u32 super_block;
> };
>
> u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries);
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
> index 4ce71a14250832440099e4cf3835b4aedfb749e8..86fa5ab02bfe3701e0c3dd38f1a9cadc88f5e4cd 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
> @@ -10,6 +10,7 @@
>
> #define UNSPECIFIED_COLOR_FORMAT 5
> #define NUM_SYS_INIT_PACKETS 8
> +#define NUM_COMV_AV1 18
>
> #define SYS_INIT_PKT_SIZE (sizeof(struct iris_hfi_header) + \
> NUM_SYS_INIT_PACKETS * (sizeof(struct iris_hfi_packet) + sizeof(u32)))
> @@ -121,6 +122,7 @@ static u32 iris_hfi_gen2_get_port_from_buf_type(struct iris_inst *inst,
> case BUF_COMV:
> case BUF_NON_COMV:
> case BUF_LINE:
> + case BUF_PARTIAL:
> return HFI_PORT_BITSTREAM;
> case BUF_OUTPUT:
> case BUF_DPB:
> @@ -380,6 +382,9 @@ static int iris_hfi_gen2_set_profile(struct iris_inst *inst, u32 plane)
> case V4L2_PIX_FMT_H264:
> profile = inst->fw_caps[PROFILE_H264].value;
> break;
> + case V4L2_PIX_FMT_AV1:
> + profile = inst->fw_caps[PROFILE_AV1].value;
> + break;
> }
>
> inst_hfi_gen2->src_subcr_params.profile = profile;
> @@ -409,6 +414,9 @@ static int iris_hfi_gen2_set_level(struct iris_inst *inst, u32 plane)
> case V4L2_PIX_FMT_H264:
> level = inst->fw_caps[LEVEL_H264].value;
> break;
> + case V4L2_PIX_FMT_AV1:
> + level = inst->fw_caps[LEVEL_AV1].value;
> + break;
> }
>
> inst_hfi_gen2->src_subcr_params.level = level;
> @@ -480,10 +488,12 @@ static int iris_hfi_gen2_set_linear_stride_scanline(struct iris_inst *inst, u32
>
> static int iris_hfi_gen2_set_tier(struct iris_inst *inst, u32 plane)
> {
> - struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
> u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
> u32 tier = inst->fw_caps[TIER].value;
>
> + tier = (inst->codec == V4L2_PIX_FMT_AV1) ? inst->fw_caps[TIER_AV1].value :
> + inst->fw_caps[TIER].value;
> inst_hfi_gen2->src_subcr_params.tier = tier;
>
> return iris_hfi_gen2_session_set_property(inst,
> @@ -509,6 +519,56 @@ static int iris_hfi_gen2_set_frame_rate(struct iris_inst *inst, u32 plane)
> sizeof(u32));
> }
>
> +static int iris_hfi_gen2_set_film_grain(struct iris_inst *inst, u32 plane)
> +{
> + u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
> + u32 film_grain;
u32 film_grain = inst->fw_caps[FILM_GRAIN].value;
> +
> + film_grain = inst->fw_caps[FILM_GRAIN].value;
> + inst_hfi_gen2->src_subcr_params.film_grain = film_grain;
> +
> + return iris_hfi_gen2_session_set_property(inst,
> + HFI_PROP_AV1_FILM_GRAIN_PRESENT,
> + HFI_HOST_FLAGS_NONE,
> + port,
> + HFI_PAYLOAD_U32_ENUM,
> + &film_grain,
> + sizeof(u32));
> +}
> +
> +static int iris_hfi_gen2_set_super_block(struct iris_inst *inst, u32 plane)
> +{
> + u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
> + u32 super_block;
> +
> + super_block = inst->fw_caps[SUPER_BLOCK].value;
u32 super_block = inst->fw_caps[SUPER_BLOCK].value;
> + inst_hfi_gen2->src_subcr_params.super_block = super_block;
> +
> + return iris_hfi_gen2_session_set_property(inst,
> + HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
> + HFI_HOST_FLAGS_NONE,
> + port,
> + HFI_PAYLOAD_U32_ENUM,
> + &super_block,
> + sizeof(u32));
> +}
> +
> +static int iris_hfi_gen2_set_opb_enable(struct iris_inst *inst, u32 plane)
> +{
> + u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> + u32 opb_enable = iris_split_mode_enabled(inst);
> +
> + return iris_hfi_gen2_session_set_property(inst,
> + HFI_PROP_OPB_ENABLE,
> + HFI_HOST_FLAGS_NONE,
> + port,
> + HFI_PAYLOAD_U32,
> + &opb_enable,
> + sizeof(u32));
> +}
> +
https://lore.kernel.org/linux-media/20251008-video-iris-ubwc-enable-v2-1-478ba2d96427@oss.qualcomm.com/
is adding the support for above.
> static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 plane)
> {
> const struct iris_platform_data *pdata = inst->core->iris_platform_data;
> @@ -531,6 +591,9 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
> {HFI_PROP_LINEAR_STRIDE_SCANLINE, iris_hfi_gen2_set_linear_stride_scanline },
> {HFI_PROP_TIER, iris_hfi_gen2_set_tier },
> {HFI_PROP_FRAME_RATE, iris_hfi_gen2_set_frame_rate },
> + {HFI_PROP_AV1_FILM_GRAIN_PRESENT, iris_hfi_gen2_set_film_grain },
> + {HFI_PROP_AV1_SUPER_BLOCK_ENABLED, iris_hfi_gen2_set_super_block },
> + {HFI_PROP_OPB_ENABLE, iris_hfi_gen2_set_opb_enable },
> };
>
> if (inst->domain == DECODER) {
> @@ -544,12 +607,20 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
> } else if (inst->codec == V4L2_PIX_FMT_VP9) {
> config_params = pdata->dec_input_config_params_vp9;
> config_params_size = pdata->dec_input_config_params_vp9_size;
> + } else if (inst->codec == V4L2_PIX_FMT_AV1) {
> + config_params = pdata->dec_input_config_params_av1;
> + config_params_size = pdata->dec_input_config_params_av1_size;
> } else {
> return -EINVAL;
> }
> } else {
> - config_params = pdata->dec_output_config_params;
> - config_params_size = pdata->dec_output_config_params_size;
> + if (inst->codec == V4L2_PIX_FMT_AV1) {
> + config_params = pdata->dec_output_config_params_av1;
> + config_params_size = pdata->dec_output_config_params_av1_size;
> + } else {
> + config_params = pdata->dec_output_config_params;
> + config_params_size = pdata->dec_output_config_params_size;
> + }
with
https://lore.kernel.org/linux-media/20251008-video-iris-ubwc-enable-v2-1-478ba2d96427@oss.qualcomm.com/,
this change will not be required.
> }
> } else {
> if (V4L2_TYPE_IS_OUTPUT(plane)) {
> @@ -598,6 +669,9 @@ static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst)
> break;
> case V4L2_PIX_FMT_VP9:
> codec = HFI_CODEC_DECODE_VP9;
> + break;
> + case V4L2_PIX_FMT_AV1:
> + codec = HFI_CODEC_DECODE_AV1;
> }
>
> iris_hfi_gen2_packet_session_property(inst,
> @@ -763,6 +837,11 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
> change_param_size =
> core->iris_platform_data->dec_input_config_params_vp9_size;
> break;
> + case V4L2_PIX_FMT_AV1:
> + change_param = core->iris_platform_data->dec_input_config_params_av1;
> + change_param_size =
> + core->iris_platform_data->dec_input_config_params_av1_size;
> + break;
> }
>
> payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
> @@ -845,6 +924,16 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
> payload_size = sizeof(u32);
> payload_type = HFI_PAYLOAD_U32;
> break;
> + case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
> + payload[0] = subsc_params.film_grain;
> + payload_size = sizeof(u32);
> + payload_type = HFI_PAYLOAD_U32;
> + break;
> + case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
> + payload[0] = subsc_params.super_block;
> + payload_size = sizeof(u32);
> + payload_type = HFI_PAYLOAD_U32;
> + break;
> default:
> prop_type = 0;
> ret = -EINVAL;
> @@ -900,6 +989,11 @@ static int iris_hfi_gen2_subscribe_property(struct iris_inst *inst, u32 plane)
> subscribe_prop_size =
> core->iris_platform_data->dec_output_prop_vp9_size;
> break;
> + case V4L2_PIX_FMT_AV1:
> + subcribe_prop = core->iris_platform_data->dec_output_prop_av1;
> + subscribe_prop_size =
> + core->iris_platform_data->dec_output_prop_av1_size;
> + break;
> }
> }
>
> @@ -1075,6 +1169,8 @@ static u32 iris_hfi_gen2_buf_type_from_driver(u32 domain, enum iris_buffer_type
> return HFI_BUFFER_ARP;
> case BUF_VPSS:
> return HFI_BUFFER_VPSS;
> + case BUF_PARTIAL:
> + return HFI_BUFFER_PARTIAL_DATA;
> default:
> return 0;
> }
> @@ -1087,7 +1183,13 @@ static int iris_set_num_comv(struct iris_inst *inst)
> u32 num_comv;
>
> caps = core->iris_platform_data->inst_caps;
> - num_comv = caps->num_comv;
> +
> + /*
> + * AV1 needs more comv buffers than other codecs.
> + * Update accordingly.
> + */
> + num_comv = (inst->codec == V4L2_PIX_FMT_AV1) ?
> + NUM_COMV_AV1 : caps->num_comv;
>
> return core->hfi_ops->session_set_property(inst,
> HFI_PROP_COMV_BUFFER_COUNT,
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> index bbfe7a0851ea94fb7041a868b4df8b2ec63bf427..9cc0989c67d74a9e051725e9ee571a2ab9160519 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> @@ -89,9 +89,18 @@ enum hfi_seq_header_mode {
> #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
> #define HFI_PROP_NO_OUTPUT 0x0300016a
> #define HFI_PROP_BUFFER_MARK 0x0300016c
> +#define HFI_PROP_WORST_COMPRESSION_RATIO 0x03000174
> +#define HFI_PROP_WORST_COMPLEXITY_FACTOR 0x03000175
> #define HFI_PROP_RAW_RESOLUTION 0x03000178
> +#define HFI_PROP_AV1_FILM_GRAIN_PRESENT 0x03000180
> +#define HFI_PROP_AV1_SUPER_BLOCK_ENABLED 0x03000181
> +#define HFI_PROP_AV1_OP_POINT 0x03000182
> +#define HFI_PROP_OPB_ENABLE 0x03000184
> +#define HFI_PROP_AV1_TILE_ROWS_COLUMNS 0x03000187
> +#define HFI_PROP_AV1_DRAP_CONFIG 0x03000189
> #define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
Pls maintain the order when adding new macro definitions.
This should go up.
> #define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
> +#define HFI_PROP_AV1_UNIFORM_TILE_SPACING 0x03000197
> #define HFI_PROP_END 0x03FFFFFF
>
> #define HFI_SESSION_ERROR_BEGIN 0x04000000
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> index 2f1f118eae4f6462ab1aa1d16844b34e6e699f1e..eb3373f0ad4a1b26fb847db02449ec8d8cb3bdbb 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> @@ -54,6 +54,8 @@ static u32 iris_hfi_gen2_buf_type_to_driver(struct iris_inst *inst,
> return BUF_SCRATCH_2;
> case HFI_BUFFER_PERSIST:
> return BUF_PERSIST;
> + case HFI_BUFFER_PARTIAL_DATA:
> + return BUF_PARTIAL;
> default:
> return 0;
> }
> @@ -72,6 +74,7 @@ static bool iris_hfi_gen2_is_valid_hfi_buffer_type(u32 buffer_type)
> case HFI_BUFFER_DPB:
> case HFI_BUFFER_PERSIST:
> case HFI_BUFFER_VPSS:
> + case HFI_BUFFER_PARTIAL_DATA:
> return true;
> default:
> return false;
> @@ -596,6 +599,10 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
> inst->fw_caps[PROFILE_H264].value = subsc_params.profile;
> inst->fw_caps[LEVEL_H264].value = subsc_params.level;
> break;
> + case V4L2_PIX_FMT_AV1:
> + inst->fw_caps[PROFILE_AV1].value = subsc_params.profile;
> + inst->fw_caps[LEVEL_AV1].value = subsc_params.level;
> + break;
> }
>
> inst->fw_caps[POC].value = subsc_params.pic_order_cnt;
> @@ -608,6 +615,11 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
> iris_inst_change_state(inst, IRIS_INST_ERROR);
> }
>
> + if (inst->codec == V4L2_PIX_FMT_AV1) {
> + inst->fw_caps[FILM_GRAIN].value = subsc_params.film_grain;
> + inst->fw_caps[SUPER_BLOCK].value = subsc_params.super_block;
> + }
> +
> inst->fw_min_count = subsc_params.fw_min_count;
> inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT);
> inst->buffers[BUF_OUTPUT].size = pixmp_op->plane_fmt[0].sizeimage;
> @@ -711,6 +723,12 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
> case HFI_PROP_NO_OUTPUT:
> inst_hfi_gen2->hfi_frame_info.no_output = 1;
> break;
> + case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
> + inst_hfi_gen2->src_subcr_params.film_grain = pkt->payload[0];
> + break;
> + case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
> + inst_hfi_gen2->src_subcr_params.super_block = pkt->payload[0];
> + break;
> case HFI_PROP_QUALITY_MODE:
> case HFI_PROP_STAGE:
> case HFI_PROP_PIPE:
> @@ -841,6 +859,10 @@ static void iris_hfi_gen2_init_src_change_param(struct iris_inst *inst)
> subsc_params->profile = inst->fw_caps[PROFILE_H264].value;
> subsc_params->level = inst->fw_caps[LEVEL_H264].value;
> break;
> + case V4L2_PIX_FMT_AV1:
> + subsc_params->profile = inst->fw_caps[PROFILE_AV1].value;
> + subsc_params->level = inst->fw_caps[LEVEL_AV1].value;
> + break;
> }
>
> subsc_params->pic_order_cnt = inst->fw_caps[POC].value;
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
> index 9aca70b4c0690f0d8d799e2a9976bd20d6bb8c94..19f0660f0388499ab71a1413b2c05f891bd5d511 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_common.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
> @@ -100,6 +100,13 @@ enum platform_inst_fw_cap_type {
> LEVEL_H264,
> LEVEL_HEVC,
> LEVEL_VP9,
> + PROFILE_AV1,
> + LEVEL_AV1,
> + TIER_AV1,
> + DRAP,
> + FILM_GRAIN,
> + SUPER_BLOCK,
> + ENH_LAYER_COUNT,
> INPUT_BUF_HOST_MAX_COUNT,
> OUTPUT_BUF_HOST_MAX_COUNT,
> STAGE,
> @@ -237,8 +244,12 @@ struct iris_platform_data {
> unsigned int dec_input_config_params_hevc_size;
> const u32 *dec_input_config_params_vp9;
> unsigned int dec_input_config_params_vp9_size;
> + const u32 *dec_input_config_params_av1;
> + unsigned int dec_input_config_params_av1_size;
> const u32 *dec_output_config_params;
> unsigned int dec_output_config_params_size;
> + const u32 *dec_output_config_params_av1;
> + unsigned int dec_output_config_params_av1_size;
> const u32 *enc_input_config_params;
> unsigned int enc_input_config_params_size;
> const u32 *enc_output_config_params;
> @@ -251,6 +262,8 @@ struct iris_platform_data {
> unsigned int dec_output_prop_hevc_size;
> const u32 *dec_output_prop_vp9;
> unsigned int dec_output_prop_vp9_size;
> + const u32 *dec_output_prop_av1;
> + unsigned int dec_output_prop_av1_size;
> const u32 *dec_ip_int_buf_tbl;
> unsigned int dec_ip_int_buf_tbl_size;
> const u32 *dec_op_int_buf_tbl;
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> index 15a38b7122af6d978fac433e8c02b4da5c141e1a..600664b2aea88067a7f66a20beaacd42181f9532 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> @@ -64,6 +64,16 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
> .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
> .set = iris_set_u32_enum,
> },
> + {
> + .cap_id = PROFILE_AV1,
> + .min = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
> + .max = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
> + .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN),
> + .value = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
> + .hfi_id = HFI_PROP_PROFILE,
> + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
> + .set = iris_set_u32_enum,
> + },
> {
> .cap_id = PROFILE_VP9,
> .min = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
> @@ -147,6 +157,33 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
> .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
> .set = iris_set_u32_enum,
> },
> + {
> + .cap_id = LEVEL_AV1,
> + .min = V4L2_MPEG_VIDEO_AV1_LEVEL_2_0,
> + .max = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
> + .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_0) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_1) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_2) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_3) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_0) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_1) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_2) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_3) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_0) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_1) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_2) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_3) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_0) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_1) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_2) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_3) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_0) |
> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_1),
> + .value = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
> + .hfi_id = HFI_PROP_LEVEL,
> + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
> + .set = iris_set_u32_enum,
> + },
> {
> .cap_id = TIER,
> .min = V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
> @@ -158,6 +195,53 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
> .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
> .set = iris_set_u32_enum,
> },
> + {
> + .cap_id = TIER_AV1,
> + .min = 0,
> + .max = 1,
> + .step_or_mask = 1,
> + .value = 0,
> + .hfi_id = HFI_PROP_TIER,
> + .flags = CAP_FLAG_INPUT_PORT,
> + .set = iris_set_u32,
> + },
> + {
> + .cap_id = DRAP,
> + .min = 0,
> + .max = 1,
> + .step_or_mask = 1,
> + .value = 0,
> + .hfi_id = HFI_PROP_AV1_DRAP_CONFIG,
> + .flags = CAP_FLAG_INPUT_PORT,
> + .set = iris_set_u32,
> + },
In the current design, this property would be set for even non-av1 case,
hope you checked that its OK and doesn't create any issue with other
codecs, same applies to TIER_AV1.
> + {
> + .cap_id = FILM_GRAIN,
> + .min = 0,
> + .max = 1,
> + .step_or_mask = 1,
> + .value = 0,
> + .hfi_id = HFI_PROP_AV1_FILM_GRAIN_PRESENT,
> + .flags = CAP_FLAG_VOLATILE,
> + },
> + {
> + .cap_id = SUPER_BLOCK,
> + .min = 0,
> + .max = 1,
> + .step_or_mask = 1,
> + .value = 0,
> + .hfi_id = HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
> + },
> + {
> + .cap_id = ENH_LAYER_COUNT,
> + .min = 0,
> + .max = 1,
> + .step_or_mask = 1,
> + .value = 0,
> + .hfi_id = HFI_PROP_AV1_OP_POINT,
> + .flags = CAP_FLAG_INPUT_PORT,
> + .set = iris_set_u32,
> + },
> {
> .cap_id = INPUT_BUF_HOST_MAX_COUNT,
> .min = DEFAULT_MAX_HOST_BUF_COUNT,
> @@ -701,6 +785,19 @@ static const u32 sm8550_vdec_input_config_param_vp9[] = {
> HFI_PROP_LEVEL,
> };
>
> +static const u32 sm8550_vdec_input_config_param_av1[] = {
> + HFI_PROP_BITSTREAM_RESOLUTION,
> + HFI_PROP_CROP_OFFSETS,
> + HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
> + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
> + HFI_PROP_PROFILE,
> + HFI_PROP_LEVEL,
> + HFI_PROP_TIER,
> + HFI_PROP_AV1_FILM_GRAIN_PRESENT,
> + HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
> + HFI_PROP_SIGNAL_COLOR_INFO,
> +};
> +
> static const u32 sm8550_venc_input_config_params[] = {
> HFI_PROP_COLOR_FORMAT,
> HFI_PROP_RAW_RESOLUTION,
> @@ -714,6 +811,12 @@ static const u32 sm8550_vdec_output_config_params[] = {
> HFI_PROP_LINEAR_STRIDE_SCANLINE,
> };
>
> +static const u32 sm8550_vdec_output_config_param_av1[] = {
> + HFI_PROP_OPB_ENABLE,
> + HFI_PROP_COLOR_FORMAT,
> + HFI_PROP_LINEAR_STRIDE_SCANLINE,
> +};
> +
> static const u32 sm8550_venc_output_config_params[] = {
> HFI_PROP_BITSTREAM_RESOLUTION,
> HFI_PROP_CROP_OFFSETS,
> @@ -737,11 +840,18 @@ static const u32 sm8550_vdec_subscribe_output_properties_vp9[] = {
> HFI_PROP_PICTURE_TYPE,
> };
>
> +static const u32 sm8550_vdec_subscribe_output_properties_av1[] = {
> + HFI_PROP_PICTURE_TYPE,
> + HFI_PROP_WORST_COMPRESSION_RATIO,
> + HFI_PROP_WORST_COMPLEXITY_FACTOR,
> +};
> +
> static const u32 sm8550_dec_ip_int_buf_tbl[] = {
> BUF_BIN,
> BUF_COMV,
> BUF_NON_COMV,
> BUF_LINE,
> + BUF_PARTIAL,
> };
>
> static const u32 sm8550_dec_op_int_buf_tbl[] = {
> @@ -806,11 +916,18 @@ struct iris_platform_data sm8550_data = {
> sm8550_vdec_input_config_param_vp9,
> .dec_input_config_params_vp9_size =
> ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
> + .dec_input_config_params_av1 =
> + sm8550_vdec_input_config_param_av1,
> + .dec_input_config_params_av1_size =
> + ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
> .dec_output_config_params =
> sm8550_vdec_output_config_params,
> .dec_output_config_params_size =
> ARRAY_SIZE(sm8550_vdec_output_config_params),
> -
> + .dec_output_config_params_av1 =
> + sm8550_vdec_output_config_param_av1,
> + .dec_output_config_params_av1_size =
> + ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
> .enc_input_config_params =
> sm8550_venc_input_config_params,
> .enc_input_config_params_size =
> @@ -831,6 +948,9 @@ struct iris_platform_data sm8550_data = {
> .dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
> .dec_output_prop_vp9_size =
> ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
> + .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
> + .dec_output_prop_av1_size =
> + ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
>
> .dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
> .dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
> @@ -900,10 +1020,18 @@ struct iris_platform_data sm8650_data = {
> sm8550_vdec_input_config_param_vp9,
> .dec_input_config_params_vp9_size =
> ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
> + .dec_input_config_params_av1 =
> + sm8550_vdec_input_config_param_av1,
> + .dec_input_config_params_av1_size =
> + ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
> .dec_output_config_params =
> sm8550_vdec_output_config_params,
> .dec_output_config_params_size =
> ARRAY_SIZE(sm8550_vdec_output_config_params),
> + .dec_output_config_params_av1 =
> + sm8550_vdec_output_config_param_av1,
> + .dec_output_config_params_av1_size =
> + ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
>
> .enc_input_config_params =
> sm8550_venc_input_config_params,
> @@ -925,6 +1053,9 @@ struct iris_platform_data sm8650_data = {
> .dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
> .dec_output_prop_vp9_size =
> ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
> + .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
> + .dec_output_prop_av1_size =
> + ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
>
> .dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
> .dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> index 8342a0b6f825b385192baedfc67c50bba7661470..0297756a3724181498ed8e5776dd5b872f1d4304 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> @@ -11,6 +11,7 @@
> #include "iris_hfi_gen1_defines.h"
> #include "iris_vpu_buffer.h"
> #include "iris_vpu_common.h"
> +#include "iris_instance.h"
why is this change needed here?
Thanks,
Dikshita
>
> #define BITRATE_MIN 32000
> #define BITRATE_MAX 160000000
> diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
> index d38d0f6961cd5cb9929e2aecbec7353dcc2d4a7d..05eec18f937a51d3c081d0f98a8ffa6bbd30d166 100644
> --- a/drivers/media/platform/qcom/iris/iris_vidc.c
> +++ b/drivers/media/platform/qcom/iris/iris_vidc.c
> @@ -178,6 +178,7 @@ int iris_open(struct file *filp)
> INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_1].list);
> INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_2].list);
> INIT_LIST_HEAD(&inst->buffers[BUF_VPSS].list);
> + INIT_LIST_HEAD(&inst->buffers[BUF_PARTIAL].list);
> init_completion(&inst->completion);
> init_completion(&inst->flush_completion);
>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 5/5] media: iris: Add internal buffer calculation for AV1 decoder
2025-10-17 18:35 ` [PATCH v2 5/5] media: iris: Add internal buffer calculation for AV1 decoder Deepa Guthyappa Madivalara
@ 2025-10-27 9:07 ` Dikshita Agarwal
2025-10-29 2:56 ` Deepa Guthyappa Madivalara
0 siblings, 1 reply; 14+ messages in thread
From: Dikshita Agarwal @ 2025-10-27 9:07 UTC (permalink / raw)
To: Deepa Guthyappa Madivalara, Mauro Carvalho Chehab, Vikash Garodia,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot
On 10/18/2025 12:05 AM, Deepa Guthyappa Madivalara wrote:
> Implement internal buffer count and size calculations for AV1 decoder
> for all the buffer types required by the AV1 decoder, including BIN,
> COMV, PERSIST, LINE, and PARTIAL.
>
> This ensures the hardware decoder has properly allocated memory for AV1
> decoding operations, enabling correct AV1 video playback.
>
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> ---
> drivers/media/platform/qcom/iris/iris_buffer.h | 1 +
> drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 298 ++++++++++++++++++++-
> drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 116 ++++++++
> 3 files changed, 411 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
> index 5ef365d9236c7cbdee24a4614789b3191881968b..75bb767761824c4c02e0df9b765896cc093be333 100644
> --- a/drivers/media/platform/qcom/iris/iris_buffer.h
> +++ b/drivers/media/platform/qcom/iris/iris_buffer.h
> @@ -27,6 +27,7 @@ struct iris_inst;
> * @BUF_SCRATCH_1: buffer to store decoding/encoding context data for HW
> * @BUF_SCRATCH_2: buffer to store encoding context data for HW
> * @BUF_VPSS: buffer to store VPSS context data for HW
> + * @BUF_PARTIAL: buffer for AV1 IBC data
> * @BUF_TYPE_MAX: max buffer types
> */
> enum iris_buffer_type {
> diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
> index 4463be05ce165adef6b152eb0c155d2e6a7b3c36..e03ae7cfc9551dd2450b27d5d19ef1d23bba4c99 100644
> --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
> +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
> @@ -9,6 +9,17 @@
> #include "iris_hfi_gen2_defines.h"
>
<snip>
> static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
> {
> u32 num_vpp_pipes = inst->core->iris_platform_data->num_vpp_pipe;
> @@ -472,6 +718,8 @@ static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
> return hfi_buffer_bin_h265d(width, height, num_vpp_pipes);
> else if (inst->codec == V4L2_PIX_FMT_VP9)
> return hfi_buffer_bin_vp9d(width, height, num_vpp_pipes);
> + else if (inst->codec == V4L2_PIX_FMT_AV1)
> + return hfi_buffer_bin_av1d(width, height, num_vpp_pipes);
>
> return 0;
> }
> @@ -487,18 +735,33 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
> return hfi_buffer_comv_h264d(width, height, num_comv);
> else if (inst->codec == V4L2_PIX_FMT_HEVC)
> return hfi_buffer_comv_h265d(width, height, num_comv);
> -
> + else if (inst->codec == V4L2_PIX_FMT_AV1) {
> + if (inst->fw_caps[DRAP].value)
> + return 0;
> + else
> + return hfi_buffer_comv_av1d(width, height, num_comv);
> + }
newline is required before return.
> return 0;
> }
>
<snip>
> +static u32 iris_vpu_dec_partial_size(struct iris_inst *inst)
> +{
> + struct v4l2_format *f = inst->fmt_src;
> + u32 height = f->fmt.pix_mp.height;
> + u32 width = f->fmt.pix_mp.width;
> +
> + return hfi_buffer_ibc_av1d(width, height);
> +}
> +
> static inline
> u32 hfi_buffer_comv_enc(u32 frame_width, u32 frame_height, u32 lcu_size,
> u32 num_recon, u32 standard)
> @@ -1414,7 +1688,9 @@ static int output_min_count(struct iris_inst *inst)
>
> /* fw_min_count > 0 indicates reconfig event has already arrived */
> if (inst->fw_min_count) {
> - if (iris_split_mode_enabled(inst) && inst->codec == V4L2_PIX_FMT_VP9)
> + if (iris_split_mode_enabled(inst) &&
> + (inst->codec == V4L2_PIX_FMT_VP9 ||
> + inst->codec == V4L2_PIX_FMT_VP9))
This change doesn't make any sense, do you mean V4L2_PIX_FMT_AV1?
> return min_t(u32, 4, inst->fw_min_count);
> else
> return inst->fw_min_count;
> @@ -1422,6 +1698,8 @@ static int output_min_count(struct iris_inst *inst)
>
> if (inst->codec == V4L2_PIX_FMT_VP9)
> output_min_count = 9;
> + else if (inst->codec == V4L2_PIX_FMT_AV1)
> + output_min_count = 11;
>
> return output_min_count;
> }
> @@ -1444,6 +1722,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)
> {BUF_PERSIST, iris_vpu_dec_persist_size },
> {BUF_DPB, iris_vpu_dec_dpb_size },
> {BUF_SCRATCH_1, iris_vpu_dec_scratch1_size },
> + {BUF_PARTIAL, iris_vpu_dec_partial_size },
> };
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 4/5] media: iris: Define AV1-specific platform capabilities and properties
2025-10-27 9:07 ` Dikshita Agarwal
@ 2025-10-29 2:46 ` Deepa Guthyappa Madivalara
0 siblings, 0 replies; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-29 2:46 UTC (permalink / raw)
To: Dikshita Agarwal, Mauro Carvalho Chehab, Vikash Garodia,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot,
Bryan O'Donoghue
On 10/27/2025 2:07 AM, Dikshita Agarwal wrote:
>
> On 10/18/2025 12:05 AM, Deepa Guthyappa Madivalara wrote:
>> Defining platform specific capabilities specific to AV1 decoder.
>> Set and subscribe to manadatory properties to firmware for AV1.
>>
>> Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
>> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
>> ---
>> drivers/media/platform/qcom/iris/iris_buffer.h | 1 +
>> drivers/media/platform/qcom/iris/iris_ctrls.c | 8 ++
>> drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
>> .../platform/qcom/iris/iris_hfi_gen2_command.c | 110 ++++++++++++++++-
>> .../platform/qcom/iris/iris_hfi_gen2_defines.h | 9 ++
>> .../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++++
>> .../platform/qcom/iris/iris_platform_common.h | 13 ++
>> .../media/platform/qcom/iris/iris_platform_gen2.c | 133 ++++++++++++++++++++-
>> .../platform/qcom/iris/iris_platform_sm8250.c | 1 +
>> drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
>> 10 files changed, 296 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
>> index 325d30fce5c99185b61ff989fbfd4de9a56762b2..5ef365d9236c7cbdee24a4614789b3191881968b 100644
>> --- a/drivers/media/platform/qcom/iris/iris_buffer.h
>> +++ b/drivers/media/platform/qcom/iris/iris_buffer.h
>> @@ -42,6 +42,7 @@ enum iris_buffer_type {
>> BUF_SCRATCH_1,
>> BUF_SCRATCH_2,
>> BUF_VPSS,
>> + BUF_PARTIAL,
>> BUF_TYPE_MAX,
>> };
>>
>> diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
>> index 754a5ad718bc37630bb861012301df7a2e7342a1..620c7e1bd273e25febd8ca70dd1dcfb0b862692b 100644
>> --- a/drivers/media/platform/qcom/iris/iris_ctrls.c
>> +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
>> @@ -98,6 +98,10 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
>> return B_FRAME_QP_H264;
>> case V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP:
>> return B_FRAME_QP_HEVC;
>> + case V4L2_CID_MPEG_VIDEO_AV1_PROFILE:
>> + return PROFILE_AV1;
>> + case V4L2_CID_MPEG_VIDEO_AV1_LEVEL:
>> + return LEVEL_AV1;
>> default:
>> return INST_FW_CAP_MAX;
>> }
>> @@ -185,6 +189,10 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
>> return V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP;
>> case B_FRAME_QP_HEVC:
>> return V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP;
>> + case PROFILE_AV1:
>> + return V4L2_CID_MPEG_VIDEO_AV1_PROFILE;
>> + case LEVEL_AV1:
>> + return V4L2_CID_MPEG_VIDEO_AV1_LEVEL;
>> default:
>> return 0;
>> }
>> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h
>> index b51471fb32c70acee44c37f8e9dce0c6bc0b6ccc..3edb5ae582b49bea2e2408c4a5cfc0a742adc05f 100644
>> --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h
>> +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h
>> @@ -141,6 +141,9 @@ struct hfi_subscription_params {
>> u32 profile;
>> u32 level;
>> u32 tier;
>> + u32 drap;
>> + u32 film_grain;
>> + u32 super_block;
>> };
>>
>> u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries);
>> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
>> index 4ce71a14250832440099e4cf3835b4aedfb749e8..86fa5ab02bfe3701e0c3dd38f1a9cadc88f5e4cd 100644
>> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
>> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
>> @@ -10,6 +10,7 @@
>>
>> #define UNSPECIFIED_COLOR_FORMAT 5
>> #define NUM_SYS_INIT_PACKETS 8
>> +#define NUM_COMV_AV1 18
>>
>> #define SYS_INIT_PKT_SIZE (sizeof(struct iris_hfi_header) + \
>> NUM_SYS_INIT_PACKETS * (sizeof(struct iris_hfi_packet) + sizeof(u32)))
>> @@ -121,6 +122,7 @@ static u32 iris_hfi_gen2_get_port_from_buf_type(struct iris_inst *inst,
>> case BUF_COMV:
>> case BUF_NON_COMV:
>> case BUF_LINE:
>> + case BUF_PARTIAL:
>> return HFI_PORT_BITSTREAM;
>> case BUF_OUTPUT:
>> case BUF_DPB:
>> @@ -380,6 +382,9 @@ static int iris_hfi_gen2_set_profile(struct iris_inst *inst, u32 plane)
>> case V4L2_PIX_FMT_H264:
>> profile = inst->fw_caps[PROFILE_H264].value;
>> break;
>> + case V4L2_PIX_FMT_AV1:
>> + profile = inst->fw_caps[PROFILE_AV1].value;
>> + break;
>> }
>>
>> inst_hfi_gen2->src_subcr_params.profile = profile;
>> @@ -409,6 +414,9 @@ static int iris_hfi_gen2_set_level(struct iris_inst *inst, u32 plane)
>> case V4L2_PIX_FMT_H264:
>> level = inst->fw_caps[LEVEL_H264].value;
>> break;
>> + case V4L2_PIX_FMT_AV1:
>> + level = inst->fw_caps[LEVEL_AV1].value;
>> + break;
>> }
>>
>> inst_hfi_gen2->src_subcr_params.level = level;
>> @@ -480,10 +488,12 @@ static int iris_hfi_gen2_set_linear_stride_scanline(struct iris_inst *inst, u32
>>
>> static int iris_hfi_gen2_set_tier(struct iris_inst *inst, u32 plane)
>> {
>> - struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
>> u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
>> u32 tier = inst->fw_caps[TIER].value;
>>
>> + tier = (inst->codec == V4L2_PIX_FMT_AV1) ? inst->fw_caps[TIER_AV1].value :
>> + inst->fw_caps[TIER].value;
>> inst_hfi_gen2->src_subcr_params.tier = tier;
>>
>> return iris_hfi_gen2_session_set_property(inst,
>> @@ -509,6 +519,56 @@ static int iris_hfi_gen2_set_frame_rate(struct iris_inst *inst, u32 plane)
>> sizeof(u32));
>> }
>>
>> +static int iris_hfi_gen2_set_film_grain(struct iris_inst *inst, u32 plane)
>> +{
>> + u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
>> + u32 film_grain;
> u32 film_grain = inst->fw_caps[FILM_GRAIN].value;
<ack>
>
>> +
>> + film_grain = inst->fw_caps[FILM_GRAIN].value;
>> + inst_hfi_gen2->src_subcr_params.film_grain = film_grain;
>> +
>> + return iris_hfi_gen2_session_set_property(inst,
>> + HFI_PROP_AV1_FILM_GRAIN_PRESENT,
>> + HFI_HOST_FLAGS_NONE,
>> + port,
>> + HFI_PAYLOAD_U32_ENUM,
>> + &film_grain,
>> + sizeof(u32));
>> +}
>> +
>> +static int iris_hfi_gen2_set_super_block(struct iris_inst *inst, u32 plane)
>> +{
>> + u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
>> + u32 super_block;
>> +
>> + super_block = inst->fw_caps[SUPER_BLOCK].value;
> u32 super_block = inst->fw_caps[SUPER_BLOCK].value;
<ack>
>> + inst_hfi_gen2->src_subcr_params.super_block = super_block;
>> +
>> + return iris_hfi_gen2_session_set_property(inst,
>> + HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
>> + HFI_HOST_FLAGS_NONE,
>> + port,
>> + HFI_PAYLOAD_U32_ENUM,
>> + &super_block,
>> + sizeof(u32));
>> +}
>> +
>> +static int iris_hfi_gen2_set_opb_enable(struct iris_inst *inst, u32 plane)
>> +{
>> + u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>> + u32 opb_enable = iris_split_mode_enabled(inst);
>> +
>> + return iris_hfi_gen2_session_set_property(inst,
>> + HFI_PROP_OPB_ENABLE,
>> + HFI_HOST_FLAGS_NONE,
>> + port,
>> + HFI_PAYLOAD_U32,
>> + &opb_enable,
>> + sizeof(u32));
>> +}
>> +
> https://lore.kernel.org/linux-media/20251008-video-iris-ubwc-enable-v2-1-478ba2d96427@oss.qualcomm.com/
> is adding the support for above.
<ack>
>> static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 plane)
>> {
>> const struct iris_platform_data *pdata = inst->core->iris_platform_data;
>> @@ -531,6 +591,9 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
>> {HFI_PROP_LINEAR_STRIDE_SCANLINE, iris_hfi_gen2_set_linear_stride_scanline },
>> {HFI_PROP_TIER, iris_hfi_gen2_set_tier },
>> {HFI_PROP_FRAME_RATE, iris_hfi_gen2_set_frame_rate },
>> + {HFI_PROP_AV1_FILM_GRAIN_PRESENT, iris_hfi_gen2_set_film_grain },
>> + {HFI_PROP_AV1_SUPER_BLOCK_ENABLED, iris_hfi_gen2_set_super_block },
>> + {HFI_PROP_OPB_ENABLE, iris_hfi_gen2_set_opb_enable },
>> };
>>
>> if (inst->domain == DECODER) {
>> @@ -544,12 +607,20 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
>> } else if (inst->codec == V4L2_PIX_FMT_VP9) {
>> config_params = pdata->dec_input_config_params_vp9;
>> config_params_size = pdata->dec_input_config_params_vp9_size;
>> + } else if (inst->codec == V4L2_PIX_FMT_AV1) {
>> + config_params = pdata->dec_input_config_params_av1;
>> + config_params_size = pdata->dec_input_config_params_av1_size;
>> } else {
>> return -EINVAL;
>> }
>> } else {
>> - config_params = pdata->dec_output_config_params;
>> - config_params_size = pdata->dec_output_config_params_size;
>> + if (inst->codec == V4L2_PIX_FMT_AV1) {
>> + config_params = pdata->dec_output_config_params_av1;
>> + config_params_size = pdata->dec_output_config_params_av1_size;
>> + } else {
>> + config_params = pdata->dec_output_config_params;
>> + config_params_size = pdata->dec_output_config_params_size;
>> + }
> with
> https://lore.kernel.org/linux-media/20251008-video-iris-ubwc-enable-v2-1-478ba2d96427@oss.qualcomm.com/,
> this change will not be required.
<ack>
>> }
>> } else {
>> if (V4L2_TYPE_IS_OUTPUT(plane)) {
>> @@ -598,6 +669,9 @@ static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst)
>> break;
>> case V4L2_PIX_FMT_VP9:
>> codec = HFI_CODEC_DECODE_VP9;
>> + break;
>> + case V4L2_PIX_FMT_AV1:
>> + codec = HFI_CODEC_DECODE_AV1;
>> }
>>
>> iris_hfi_gen2_packet_session_property(inst,
>> @@ -763,6 +837,11 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
>> change_param_size =
>> core->iris_platform_data->dec_input_config_params_vp9_size;
>> break;
>> + case V4L2_PIX_FMT_AV1:
>> + change_param = core->iris_platform_data->dec_input_config_params_av1;
>> + change_param_size =
>> + core->iris_platform_data->dec_input_config_params_av1_size;
>> + break;
>> }
>>
>> payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
>> @@ -845,6 +924,16 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
>> payload_size = sizeof(u32);
>> payload_type = HFI_PAYLOAD_U32;
>> break;
>> + case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
>> + payload[0] = subsc_params.film_grain;
>> + payload_size = sizeof(u32);
>> + payload_type = HFI_PAYLOAD_U32;
>> + break;
>> + case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
>> + payload[0] = subsc_params.super_block;
>> + payload_size = sizeof(u32);
>> + payload_type = HFI_PAYLOAD_U32;
>> + break;
>> default:
>> prop_type = 0;
>> ret = -EINVAL;
>> @@ -900,6 +989,11 @@ static int iris_hfi_gen2_subscribe_property(struct iris_inst *inst, u32 plane)
>> subscribe_prop_size =
>> core->iris_platform_data->dec_output_prop_vp9_size;
>> break;
>> + case V4L2_PIX_FMT_AV1:
>> + subcribe_prop = core->iris_platform_data->dec_output_prop_av1;
>> + subscribe_prop_size =
>> + core->iris_platform_data->dec_output_prop_av1_size;
>> + break;
>> }
>> }
>>
>> @@ -1075,6 +1169,8 @@ static u32 iris_hfi_gen2_buf_type_from_driver(u32 domain, enum iris_buffer_type
>> return HFI_BUFFER_ARP;
>> case BUF_VPSS:
>> return HFI_BUFFER_VPSS;
>> + case BUF_PARTIAL:
>> + return HFI_BUFFER_PARTIAL_DATA;
>> default:
>> return 0;
>> }
>> @@ -1087,7 +1183,13 @@ static int iris_set_num_comv(struct iris_inst *inst)
>> u32 num_comv;
>>
>> caps = core->iris_platform_data->inst_caps;
>> - num_comv = caps->num_comv;
>> +
>> + /*
>> + * AV1 needs more comv buffers than other codecs.
>> + * Update accordingly.
>> + */
>> + num_comv = (inst->codec == V4L2_PIX_FMT_AV1) ?
>> + NUM_COMV_AV1 : caps->num_comv;
>>
>> return core->hfi_ops->session_set_property(inst,
>> HFI_PROP_COMV_BUFFER_COUNT,
>> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
>> index bbfe7a0851ea94fb7041a868b4df8b2ec63bf427..9cc0989c67d74a9e051725e9ee571a2ab9160519 100644
>> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
>> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
>> @@ -89,9 +89,18 @@ enum hfi_seq_header_mode {
>> #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
>> #define HFI_PROP_NO_OUTPUT 0x0300016a
>> #define HFI_PROP_BUFFER_MARK 0x0300016c
>> +#define HFI_PROP_WORST_COMPRESSION_RATIO 0x03000174
>> +#define HFI_PROP_WORST_COMPLEXITY_FACTOR 0x03000175
>> #define HFI_PROP_RAW_RESOLUTION 0x03000178
>> +#define HFI_PROP_AV1_FILM_GRAIN_PRESENT 0x03000180
>> +#define HFI_PROP_AV1_SUPER_BLOCK_ENABLED 0x03000181
>> +#define HFI_PROP_AV1_OP_POINT 0x03000182
>> +#define HFI_PROP_OPB_ENABLE 0x03000184
>> +#define HFI_PROP_AV1_TILE_ROWS_COLUMNS 0x03000187
>> +#define HFI_PROP_AV1_DRAP_CONFIG 0x03000189
>> #define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
> Pls maintain the order when adding new macro definitions.
> This should go up.
<ack>
>> #define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
>> +#define HFI_PROP_AV1_UNIFORM_TILE_SPACING 0x03000197
>> #define HFI_PROP_END 0x03FFFFFF
>>
>> #define HFI_SESSION_ERROR_BEGIN 0x04000000
>> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> index 2f1f118eae4f6462ab1aa1d16844b34e6e699f1e..eb3373f0ad4a1b26fb847db02449ec8d8cb3bdbb 100644
>> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> @@ -54,6 +54,8 @@ static u32 iris_hfi_gen2_buf_type_to_driver(struct iris_inst *inst,
>> return BUF_SCRATCH_2;
>> case HFI_BUFFER_PERSIST:
>> return BUF_PERSIST;
>> + case HFI_BUFFER_PARTIAL_DATA:
>> + return BUF_PARTIAL;
>> default:
>> return 0;
>> }
>> @@ -72,6 +74,7 @@ static bool iris_hfi_gen2_is_valid_hfi_buffer_type(u32 buffer_type)
>> case HFI_BUFFER_DPB:
>> case HFI_BUFFER_PERSIST:
>> case HFI_BUFFER_VPSS:
>> + case HFI_BUFFER_PARTIAL_DATA:
>> return true;
>> default:
>> return false;
>> @@ -596,6 +599,10 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
>> inst->fw_caps[PROFILE_H264].value = subsc_params.profile;
>> inst->fw_caps[LEVEL_H264].value = subsc_params.level;
>> break;
>> + case V4L2_PIX_FMT_AV1:
>> + inst->fw_caps[PROFILE_AV1].value = subsc_params.profile;
>> + inst->fw_caps[LEVEL_AV1].value = subsc_params.level;
>> + break;
>> }
>>
>> inst->fw_caps[POC].value = subsc_params.pic_order_cnt;
>> @@ -608,6 +615,11 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
>> iris_inst_change_state(inst, IRIS_INST_ERROR);
>> }
>>
>> + if (inst->codec == V4L2_PIX_FMT_AV1) {
>> + inst->fw_caps[FILM_GRAIN].value = subsc_params.film_grain;
>> + inst->fw_caps[SUPER_BLOCK].value = subsc_params.super_block;
>> + }
>> +
>> inst->fw_min_count = subsc_params.fw_min_count;
>> inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT);
>> inst->buffers[BUF_OUTPUT].size = pixmp_op->plane_fmt[0].sizeimage;
>> @@ -711,6 +723,12 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
>> case HFI_PROP_NO_OUTPUT:
>> inst_hfi_gen2->hfi_frame_info.no_output = 1;
>> break;
>> + case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
>> + inst_hfi_gen2->src_subcr_params.film_grain = pkt->payload[0];
>> + break;
>> + case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
>> + inst_hfi_gen2->src_subcr_params.super_block = pkt->payload[0];
>> + break;
>> case HFI_PROP_QUALITY_MODE:
>> case HFI_PROP_STAGE:
>> case HFI_PROP_PIPE:
>> @@ -841,6 +859,10 @@ static void iris_hfi_gen2_init_src_change_param(struct iris_inst *inst)
>> subsc_params->profile = inst->fw_caps[PROFILE_H264].value;
>> subsc_params->level = inst->fw_caps[LEVEL_H264].value;
>> break;
>> + case V4L2_PIX_FMT_AV1:
>> + subsc_params->profile = inst->fw_caps[PROFILE_AV1].value;
>> + subsc_params->level = inst->fw_caps[LEVEL_AV1].value;
>> + break;
>> }
>>
>> subsc_params->pic_order_cnt = inst->fw_caps[POC].value;
>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
>> index 9aca70b4c0690f0d8d799e2a9976bd20d6bb8c94..19f0660f0388499ab71a1413b2c05f891bd5d511 100644
>> --- a/drivers/media/platform/qcom/iris/iris_platform_common.h
>> +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
>> @@ -100,6 +100,13 @@ enum platform_inst_fw_cap_type {
>> LEVEL_H264,
>> LEVEL_HEVC,
>> LEVEL_VP9,
>> + PROFILE_AV1,
>> + LEVEL_AV1,
>> + TIER_AV1,
>> + DRAP,
>> + FILM_GRAIN,
>> + SUPER_BLOCK,
>> + ENH_LAYER_COUNT,
>> INPUT_BUF_HOST_MAX_COUNT,
>> OUTPUT_BUF_HOST_MAX_COUNT,
>> STAGE,
>> @@ -237,8 +244,12 @@ struct iris_platform_data {
>> unsigned int dec_input_config_params_hevc_size;
>> const u32 *dec_input_config_params_vp9;
>> unsigned int dec_input_config_params_vp9_size;
>> + const u32 *dec_input_config_params_av1;
>> + unsigned int dec_input_config_params_av1_size;
>> const u32 *dec_output_config_params;
>> unsigned int dec_output_config_params_size;
>> + const u32 *dec_output_config_params_av1;
>> + unsigned int dec_output_config_params_av1_size;
>> const u32 *enc_input_config_params;
>> unsigned int enc_input_config_params_size;
>> const u32 *enc_output_config_params;
>> @@ -251,6 +262,8 @@ struct iris_platform_data {
>> unsigned int dec_output_prop_hevc_size;
>> const u32 *dec_output_prop_vp9;
>> unsigned int dec_output_prop_vp9_size;
>> + const u32 *dec_output_prop_av1;
>> + unsigned int dec_output_prop_av1_size;
>> const u32 *dec_ip_int_buf_tbl;
>> unsigned int dec_ip_int_buf_tbl_size;
>> const u32 *dec_op_int_buf_tbl;
>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> index 15a38b7122af6d978fac433e8c02b4da5c141e1a..600664b2aea88067a7f66a20beaacd42181f9532 100644
>> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> @@ -64,6 +64,16 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
>> .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
>> .set = iris_set_u32_enum,
>> },
>> + {
>> + .cap_id = PROFILE_AV1,
>> + .min = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
>> + .max = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
>> + .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN),
>> + .value = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
>> + .hfi_id = HFI_PROP_PROFILE,
>> + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
>> + .set = iris_set_u32_enum,
>> + },
>> {
>> .cap_id = PROFILE_VP9,
>> .min = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
>> @@ -147,6 +157,33 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
>> .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
>> .set = iris_set_u32_enum,
>> },
>> + {
>> + .cap_id = LEVEL_AV1,
>> + .min = V4L2_MPEG_VIDEO_AV1_LEVEL_2_0,
>> + .max = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
>> + .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_0) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_1) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_2) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_3) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_0) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_1) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_2) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_3) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_0) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_1) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_2) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_3) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_0) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_1) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_2) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_3) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_0) |
>> + BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_1),
>> + .value = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
>> + .hfi_id = HFI_PROP_LEVEL,
>> + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
>> + .set = iris_set_u32_enum,
>> + },
>> {
>> .cap_id = TIER,
>> .min = V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
>> @@ -158,6 +195,53 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
>> .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
>> .set = iris_set_u32_enum,
>> },
>> + {
>> + .cap_id = TIER_AV1,
>> + .min = 0,
>> + .max = 1,
>> + .step_or_mask = 1,
>> + .value = 0,
>> + .hfi_id = HFI_PROP_TIER,
>> + .flags = CAP_FLAG_INPUT_PORT,
>> + .set = iris_set_u32,
>> + },
>> + {
>> + .cap_id = DRAP,
>> + .min = 0,
>> + .max = 1,
>> + .step_or_mask = 1,
>> + .value = 0,
>> + .hfi_id = HFI_PROP_AV1_DRAP_CONFIG,
>> + .flags = CAP_FLAG_INPUT_PORT,
>> + .set = iris_set_u32,
>> + },
> In the current design, this property would be set for even non-av1 case,
> hope you checked that its OK and doesn't create any issue with other
> codecs, same applies to TIER_AV1.
I have run h264, hevc and vp9 gst test for sanity.
>> + {
>> + .cap_id = FILM_GRAIN,
>> + .min = 0,
>> + .max = 1,
>> + .step_or_mask = 1,
>> + .value = 0,
>> + .hfi_id = HFI_PROP_AV1_FILM_GRAIN_PRESENT,
>> + .flags = CAP_FLAG_VOLATILE,
>> + },
>> + {
>> + .cap_id = SUPER_BLOCK,
>> + .min = 0,
>> + .max = 1,
>> + .step_or_mask = 1,
>> + .value = 0,
>> + .hfi_id = HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
>> + },
>> + {
>> + .cap_id = ENH_LAYER_COUNT,
>> + .min = 0,
>> + .max = 1,
>> + .step_or_mask = 1,
>> + .value = 0,
>> + .hfi_id = HFI_PROP_AV1_OP_POINT,
>> + .flags = CAP_FLAG_INPUT_PORT,
>> + .set = iris_set_u32,
>> + },
>> {
>> .cap_id = INPUT_BUF_HOST_MAX_COUNT,
>> .min = DEFAULT_MAX_HOST_BUF_COUNT,
>> @@ -701,6 +785,19 @@ static const u32 sm8550_vdec_input_config_param_vp9[] = {
>> HFI_PROP_LEVEL,
>> };
>>
>> +static const u32 sm8550_vdec_input_config_param_av1[] = {
>> + HFI_PROP_BITSTREAM_RESOLUTION,
>> + HFI_PROP_CROP_OFFSETS,
>> + HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
>> + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
>> + HFI_PROP_PROFILE,
>> + HFI_PROP_LEVEL,
>> + HFI_PROP_TIER,
>> + HFI_PROP_AV1_FILM_GRAIN_PRESENT,
>> + HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
>> + HFI_PROP_SIGNAL_COLOR_INFO,
>> +};
>> +
>> static const u32 sm8550_venc_input_config_params[] = {
>> HFI_PROP_COLOR_FORMAT,
>> HFI_PROP_RAW_RESOLUTION,
>> @@ -714,6 +811,12 @@ static const u32 sm8550_vdec_output_config_params[] = {
>> HFI_PROP_LINEAR_STRIDE_SCANLINE,
>> };
>>
>> +static const u32 sm8550_vdec_output_config_param_av1[] = {
>> + HFI_PROP_OPB_ENABLE,
>> + HFI_PROP_COLOR_FORMAT,
>> + HFI_PROP_LINEAR_STRIDE_SCANLINE,
>> +};
>> +
>> static const u32 sm8550_venc_output_config_params[] = {
>> HFI_PROP_BITSTREAM_RESOLUTION,
>> HFI_PROP_CROP_OFFSETS,
>> @@ -737,11 +840,18 @@ static const u32 sm8550_vdec_subscribe_output_properties_vp9[] = {
>> HFI_PROP_PICTURE_TYPE,
>> };
>>
>> +static const u32 sm8550_vdec_subscribe_output_properties_av1[] = {
>> + HFI_PROP_PICTURE_TYPE,
>> + HFI_PROP_WORST_COMPRESSION_RATIO,
>> + HFI_PROP_WORST_COMPLEXITY_FACTOR,
>> +};
>> +
>> static const u32 sm8550_dec_ip_int_buf_tbl[] = {
>> BUF_BIN,
>> BUF_COMV,
>> BUF_NON_COMV,
>> BUF_LINE,
>> + BUF_PARTIAL,
>> };
>>
>> static const u32 sm8550_dec_op_int_buf_tbl[] = {
>> @@ -806,11 +916,18 @@ struct iris_platform_data sm8550_data = {
>> sm8550_vdec_input_config_param_vp9,
>> .dec_input_config_params_vp9_size =
>> ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
>> + .dec_input_config_params_av1 =
>> + sm8550_vdec_input_config_param_av1,
>> + .dec_input_config_params_av1_size =
>> + ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
>> .dec_output_config_params =
>> sm8550_vdec_output_config_params,
>> .dec_output_config_params_size =
>> ARRAY_SIZE(sm8550_vdec_output_config_params),
>> -
>> + .dec_output_config_params_av1 =
>> + sm8550_vdec_output_config_param_av1,
>> + .dec_output_config_params_av1_size =
>> + ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
>> .enc_input_config_params =
>> sm8550_venc_input_config_params,
>> .enc_input_config_params_size =
>> @@ -831,6 +948,9 @@ struct iris_platform_data sm8550_data = {
>> .dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
>> .dec_output_prop_vp9_size =
>> ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
>> + .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
>> + .dec_output_prop_av1_size =
>> + ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
>>
>> .dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
>> .dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
>> @@ -900,10 +1020,18 @@ struct iris_platform_data sm8650_data = {
>> sm8550_vdec_input_config_param_vp9,
>> .dec_input_config_params_vp9_size =
>> ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
>> + .dec_input_config_params_av1 =
>> + sm8550_vdec_input_config_param_av1,
>> + .dec_input_config_params_av1_size =
>> + ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
>> .dec_output_config_params =
>> sm8550_vdec_output_config_params,
>> .dec_output_config_params_size =
>> ARRAY_SIZE(sm8550_vdec_output_config_params),
>> + .dec_output_config_params_av1 =
>> + sm8550_vdec_output_config_param_av1,
>> + .dec_output_config_params_av1_size =
>> + ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
>>
>> .enc_input_config_params =
>> sm8550_venc_input_config_params,
>> @@ -925,6 +1053,9 @@ struct iris_platform_data sm8650_data = {
>> .dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
>> .dec_output_prop_vp9_size =
>> ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
>> + .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
>> + .dec_output_prop_av1_size =
>> + ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
>>
>> .dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
>> .dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
>> index 8342a0b6f825b385192baedfc67c50bba7661470..0297756a3724181498ed8e5776dd5b872f1d4304 100644
>> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
>> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
>> @@ -11,6 +11,7 @@
>> #include "iris_hfi_gen1_defines.h"
>> #include "iris_vpu_buffer.h"
>> #include "iris_vpu_common.h"
>> +#include "iris_instance.h"
> why is this change needed here?
Fixes kernel test robot compile failure
>
> Thanks,
> Dikshita
>
>>
>> #define BITRATE_MIN 32000
>> #define BITRATE_MAX 160000000
>> diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
>> index d38d0f6961cd5cb9929e2aecbec7353dcc2d4a7d..05eec18f937a51d3c081d0f98a8ffa6bbd30d166 100644
>> --- a/drivers/media/platform/qcom/iris/iris_vidc.c
>> +++ b/drivers/media/platform/qcom/iris/iris_vidc.c
>> @@ -178,6 +178,7 @@ int iris_open(struct file *filp)
>> INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_1].list);
>> INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_2].list);
>> INIT_LIST_HEAD(&inst->buffers[BUF_VPSS].list);
>> + INIT_LIST_HEAD(&inst->buffers[BUF_PARTIAL].list);
>> init_completion(&inst->completion);
>> init_completion(&inst->flush_completion);
>>
>>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 5/5] media: iris: Add internal buffer calculation for AV1 decoder
2025-10-27 9:07 ` Dikshita Agarwal
@ 2025-10-29 2:56 ` Deepa Guthyappa Madivalara
0 siblings, 0 replies; 14+ messages in thread
From: Deepa Guthyappa Madivalara @ 2025-10-29 2:56 UTC (permalink / raw)
To: Dikshita Agarwal, Mauro Carvalho Chehab, Vikash Garodia,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm, kernel test robot
On 10/27/2025 2:07 AM, Dikshita Agarwal wrote:
>
> On 10/18/2025 12:05 AM, Deepa Guthyappa Madivalara wrote:
>> Implement internal buffer count and size calculations for AV1 decoder
>> for all the buffer types required by the AV1 decoder, including BIN,
>> COMV, PERSIST, LINE, and PARTIAL.
>>
>> This ensures the hardware decoder has properly allocated memory for AV1
>> decoding operations, enabling correct AV1 video playback.
>>
>> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
>> ---
>> drivers/media/platform/qcom/iris/iris_buffer.h | 1 +
>> drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 298 ++++++++++++++++++++-
>> drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 116 ++++++++
>> 3 files changed, 411 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
>> index 5ef365d9236c7cbdee24a4614789b3191881968b..75bb767761824c4c02e0df9b765896cc093be333 100644
>> --- a/drivers/media/platform/qcom/iris/iris_buffer.h
>> +++ b/drivers/media/platform/qcom/iris/iris_buffer.h
>> @@ -27,6 +27,7 @@ struct iris_inst;
>> * @BUF_SCRATCH_1: buffer to store decoding/encoding context data for HW
>> * @BUF_SCRATCH_2: buffer to store encoding context data for HW
>> * @BUF_VPSS: buffer to store VPSS context data for HW
>> + * @BUF_PARTIAL: buffer for AV1 IBC data
>> * @BUF_TYPE_MAX: max buffer types
>> */
>> enum iris_buffer_type {
>> diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
>> index 4463be05ce165adef6b152eb0c155d2e6a7b3c36..e03ae7cfc9551dd2450b27d5d19ef1d23bba4c99 100644
>> --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
>> +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
>> @@ -9,6 +9,17 @@
>> #include "iris_hfi_gen2_defines.h"
>>
> <snip>
>
>> static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
>> {
>> u32 num_vpp_pipes = inst->core->iris_platform_data->num_vpp_pipe;
>> @@ -472,6 +718,8 @@ static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
>> return hfi_buffer_bin_h265d(width, height, num_vpp_pipes);
>> else if (inst->codec == V4L2_PIX_FMT_VP9)
>> return hfi_buffer_bin_vp9d(width, height, num_vpp_pipes);
>> + else if (inst->codec == V4L2_PIX_FMT_AV1)
>> + return hfi_buffer_bin_av1d(width, height, num_vpp_pipes);
>>
>> return 0;
>> }
>> @@ -487,18 +735,33 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
>> return hfi_buffer_comv_h264d(width, height, num_comv);
>> else if (inst->codec == V4L2_PIX_FMT_HEVC)
>> return hfi_buffer_comv_h265d(width, height, num_comv);
>> -
>> + else if (inst->codec == V4L2_PIX_FMT_AV1) {
>> + if (inst->fw_caps[DRAP].value)
>> + return 0;
>> + else
>> + return hfi_buffer_comv_av1d(width, height, num_comv);
>> + }
> newline is required before return.
<ack>
>
>> return 0;
>> }
>>
> <snip>
>
>> +static u32 iris_vpu_dec_partial_size(struct iris_inst *inst)
>> +{
>> + struct v4l2_format *f = inst->fmt_src;
>> + u32 height = f->fmt.pix_mp.height;
>> + u32 width = f->fmt.pix_mp.width;
>> +
>> + return hfi_buffer_ibc_av1d(width, height);
>> +}
>> +
>> static inline
>> u32 hfi_buffer_comv_enc(u32 frame_width, u32 frame_height, u32 lcu_size,
>> u32 num_recon, u32 standard)
>> @@ -1414,7 +1688,9 @@ static int output_min_count(struct iris_inst *inst)
>>
>> /* fw_min_count > 0 indicates reconfig event has already arrived */
>> if (inst->fw_min_count) {
>> - if (iris_split_mode_enabled(inst) && inst->codec == V4L2_PIX_FMT_VP9)
>> + if (iris_split_mode_enabled(inst) &&
>> + (inst->codec == V4L2_PIX_FMT_VP9 ||
>> + inst->codec == V4L2_PIX_FMT_VP9))
> This change doesn't make any sense, do you mean V4L2_PIX_FMT_AV1?
Yes, updated in v3
>
>> return min_t(u32, 4, inst->fw_min_count);
>> else
>> return inst->fw_min_count;
>> @@ -1422,6 +1698,8 @@ static int output_min_count(struct iris_inst *inst)
>>
>> if (inst->codec == V4L2_PIX_FMT_VP9)
>> output_min_count = 9;
>> + else if (inst->codec == V4L2_PIX_FMT_AV1)
>> + output_min_count = 11;
>>
>> return output_min_count;
>> }
>> @@ -1444,6 +1722,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)
>> {BUF_PERSIST, iris_vpu_dec_persist_size },
>> {BUF_DPB, iris_vpu_dec_dpb_size },
>> {BUF_SCRATCH_1, iris_vpu_dec_scratch1_size },
>> + {BUF_PARTIAL, iris_vpu_dec_partial_size },
>> };
>>
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2025-10-29 2:56 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-17 18:35 [PATCH v2 0/5] Enable support for AV1 stateful decoder Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 1/5] media: uapi: videodev2: Add " Deepa Guthyappa Madivalara
2025-10-17 20:01 ` Nicolas Dufresne
2025-10-20 21:55 ` Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() Deepa Guthyappa Madivalara
2025-10-17 20:02 ` Nicolas Dufresne
2025-10-17 18:35 ` [PATCH v2 3/5] media: iris: Add support for AV1 format in iris decoder Deepa Guthyappa Madivalara
2025-10-27 6:04 ` Dikshita Agarwal
2025-10-17 18:35 ` [PATCH v2 4/5] media: iris: Define AV1-specific platform capabilities and properties Deepa Guthyappa Madivalara
2025-10-27 9:07 ` Dikshita Agarwal
2025-10-29 2:46 ` Deepa Guthyappa Madivalara
2025-10-17 18:35 ` [PATCH v2 5/5] media: iris: Add internal buffer calculation for AV1 decoder Deepa Guthyappa Madivalara
2025-10-27 9:07 ` Dikshita Agarwal
2025-10-29 2:56 ` Deepa Guthyappa Madivalara
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).