* [PATCH RFC 0/5] Enable support for AV1 stateful decoder
@ 2025-09-02 23:00 DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 1/5] media: uapi: videodev2: Add " DEEPA GUTHYAPPA MADIVALARA
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: DEEPA GUTHYAPPA MADIVALARA @ 2025-09-02 23:00 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm,
Deepa Guthyappa Madivalara
Hi all,
This patch series adds initial support for the AV1 stateful
decoder codecs in iris decoder. Also it adds support for AV1
stateful decoder in V4l2. The objective of this work is to
extend the Iris decoder's capabilities to handle AV1 format
codec streams, including necessary format handling and buffer
management. I'm sharing this series as an RFC because conformance
testing and gstreamer testing are still in progress. While initial
functional tests show positive results, I would appreciate early
feedback on the design, implementation, and fixes before moving to
a formal submission. I plan to submit a formal patch series after
completing all the compliance checks. Meanwhile, any feedback or
suggestion to improve this work are very welcome and will be of
great help.
Gstreamer testing:
Gstreamer MR for enabling AV1 stateful decoder:
https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1
Thanks to Nicolas Dufresne for proving the MR
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469
However, Gst testing with parsebin is not resolving to av1parser as
below:
Ex: With the following command parsebin is unable to resolve to
av1parser.
GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault
filesrc
location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/
argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu
! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none !
video/x-raw,format=I420
! filesink location=gst_decoder_output.yuv
0:00:00.051674896 400 0xffff8c000b90 DEBUG
parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps
video/x-h263, variant=(string)itu
The same test with the av1parse command parses correctly:
GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault
filesrc
location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/
argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu
! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none !
video/x-raw,format=I420
! filesink location=/tmp/gst_decoder_output.yuv
Got EOS from element "pipeline0".
Execution ended after 0:00:01.599088176
Setting pipeline to NULL ...
0:00:03.580831249 1075 0x3354f960 DEBUG
av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop
Fluster testing:
As fluster.py is using parsebin for gstreamer, seeing the same issue as
described above for the following testsuites.
AV1-ARGON-PROFILE0-CORE-ANNEX-B
AV1-ARGON-PROFILE0-NON-ANNEX-B
AV1-ARGON-PROFILE0-NON-ANNEX-B
Test suite: AV1-TEST-VECTORS
The result of fluster test on SM8550:
134/242 testcases passed while testing AV1-TEST-VECTORS with
GStreamer-AV1-V4L2-Gst1.0
unsupported content, bit depth: a000a (66 tests)
Iris hardware decoder supports only 8bit NV12
av1-1-b10-00-quantizer-*
Unsupported resolution (36 tests).
Iris hardware decoder supports min resolution of 96x96
av1-1-b8-01-size-*
Unsupported colorformat (1 test)
av1-1-b8-24-monochrome
Crc mismatch: debug in progress (5tests)
av1-1-b8-03-sizeup
av1-1-b8-03-sizedown
av1-1-b8-16-intra_only-intrabc-extreme-dv
av1-1-b8-22-svc-L2T1
av1-1-b8-22-svc-L2T2
Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS
12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with
GStreamer-AV1-V4L2-Gst1.0
Crc mismatch: debug in progress
av1-1-b8-03-sizeup.ivf
Unsupported test suites:
Iris Hardware Decoder supports only
PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN
and 8 bit, 420 only
AV1-ARGON-PROFILE1-CORE-ANNEX-B
AV1-ARGON-PROFILE1-NON-ANNEX-B
AV1-ARGON-PROFILE1-STRESS-ANNEX-B
AV1-ARGON-PROFILE2-CORE-ANNEX-B
AV1-ARGON-PROFILE2-NON-ANNEX-B
AV1-ARGON-PROFILE2-STRESS-ANNEX-B
CHROMIUM-10bit-AV1-TEST-VECTORS
Compliance test for iris_driver device /dev/video0:
Driver Info:
Driver name : iris_driver
Card type : Iris Decoder
Bus info : platform:aa00000.video-codec
Driver version : 6.16.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 12 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
[65391.311675] qcom-iris aa00000.video-codec: invalid plane
[65395.340586] qcom-iris aa00000.video-codec: invalid plane
test blocking wait: OK
Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0
V4l2-ctl Test verified for 2 streams as well.
Thanks,
Deepa
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
DEEPA GUTHYAPPA MADIVALARA (5):
media: uapi: videodev2: Add support for AV1 stateful decoder
media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
media: iris: Add support for AV1 format in iris decoder
media: iris: Add internal buffer calculation for AV1 decoder
media: iris: Define AV1-specific platform capabilities and properties
drivers/media/platform/qcom/iris/iris_buffer.h | 2 +
drivers/media/platform/qcom/iris/iris_ctrls.c | 8 +
drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
.../platform/qcom/iris/iris_hfi_gen2_command.c | 109 ++++++++-
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 10 +
.../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++
drivers/media/platform/qcom/iris/iris_instance.h | 1 +
.../platform/qcom/iris/iris_platform_common.h | 15 ++
.../media/platform/qcom/iris/iris_platform_gen2.c | 156 ++++++++++++-
.../platform/qcom/iris/iris_platform_sm8250.c | 17 ++
drivers/media/platform/qcom/iris/iris_vdec.c | 25 +-
drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++-
drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
16 files changed, 703 insertions(+), 28 deletions(-)
---
base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f
change-id: 20250821-rfc_split-c3ff834bb2c9
prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3
prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c
prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e
prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4
prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3
prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2
prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce
prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0
prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca
prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e
prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec
prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759
prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7
prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1
prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0
prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc
prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5
prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c
prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3
prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3
prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5
prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968
prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5
prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423
prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd
prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa
prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83
Best regards,
--
Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH RFC 1/5] media: uapi: videodev2: Add support for AV1 stateful decoder
2025-09-02 23:00 [PATCH RFC 0/5] Enable support for AV1 stateful decoder DEEPA GUTHYAPPA MADIVALARA
@ 2025-09-02 23:00 ` DEEPA GUTHYAPPA MADIVALARA
2025-09-03 13:17 ` Nicolas Dufresne
2025-09-02 23:00 ` [PATCH RFC 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() DEEPA GUTHYAPPA MADIVALARA
` (4 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: DEEPA GUTHYAPPA MADIVALARA @ 2025-09-02 23:00 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm,
Deepa Guthyappa Madivalara
Introduce a new pixel format, V4L2_PIX_FMT_AV1, to the
Video4Linux2(V4L2) API. This format is intended for AV1
bitstreams in stateful decoding/encoding workflows.
The fourcc code 'AV10' is used to distinguish
this format from the existing V4L2_PIX_FMT_AV1_FRAME,
which is used for stateless AV1 decoder implementation.
Signed-off-by: DEEPA GUTHYAPPA MADIVALARA <deepa.madivalara@oss.qualcomm.com>
---
include/uapi/linux/videodev2.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3dd9fa45dde1066d52a68581625a39e7ec92c9b7..bff42a71c67b3f4b570dd6f3d250f1bb482ec8ae 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -775,6 +775,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
#define V4L2_PIX_FMT_AV1_FRAME v4l2_fourcc('A', 'V', '1', 'F') /* AV1 parsed frame */
+#define V4L2_PIX_FMT_AV1 v4l2_fourcc('A', 'V', '1', '0') /* AV1 (stateful) */
#define V4L2_PIX_FMT_SPK v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
#define V4L2_PIX_FMT_RV30 v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
#define V4L2_PIX_FMT_RV40 v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH RFC 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
2025-09-02 23:00 [PATCH RFC 0/5] Enable support for AV1 stateful decoder DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 1/5] media: uapi: videodev2: Add " DEEPA GUTHYAPPA MADIVALARA
@ 2025-09-02 23:00 ` DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 3/5] media: iris: Add support for AV1 format in iris decoder DEEPA GUTHYAPPA MADIVALARA
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: DEEPA GUTHYAPPA MADIVALARA @ 2025-09-02 23:00 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm,
Deepa Guthyappa Madivalara
Add a descriptive string for the AV1 pixel format to v4l_fill_fmtdesc(),
enabling proper reporting of AV1 support via VIDIOC_ENUM_FMT.
Signed-off-by: DEEPA GUTHYAPPA MADIVALARA <deepa.madivalara@oss.qualcomm.com>
---
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 01cf52c3ea33e1a01e1b306036ba4e57ef5c95d0..d3ee7736b74b0f277d3208782e3ac3282eca1e6b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1542,6 +1542,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_QC10C: descr = "QCOM Compressed 10-bit Format"; break;
case V4L2_PIX_FMT_AJPG: descr = "Aspeed JPEG"; break;
case V4L2_PIX_FMT_AV1_FRAME: descr = "AV1 Frame"; break;
+ case V4L2_PIX_FMT_AV1: descr = "AV1"; break;
case V4L2_PIX_FMT_MT2110T: descr = "Mediatek 10bit Tile Mode"; break;
case V4L2_PIX_FMT_MT2110R: descr = "Mediatek 10bit Raster Mode"; break;
case V4L2_PIX_FMT_HEXTILE: descr = "Hextile Compressed Format"; break;
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH RFC 3/5] media: iris: Add support for AV1 format in iris decoder
2025-09-02 23:00 [PATCH RFC 0/5] Enable support for AV1 stateful decoder DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 1/5] media: uapi: videodev2: Add " DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() DEEPA GUTHYAPPA MADIVALARA
@ 2025-09-02 23:00 ` DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 4/5] media: iris: Add internal buffer calculation for AV1 decoder DEEPA GUTHYAPPA MADIVALARA
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: DEEPA GUTHYAPPA MADIVALARA @ 2025-09-02 23:00 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm,
Deepa Guthyappa Madivalara
Extend iris decoder driver to support format V4L2_PIX_FMT_AV1.
This change updates the format enumeration (VIDIOC_ENUM_FMT)
and allows setting AV1 format via VIDIOC_S_FMT for gen2 and beyond.
Gen1 iris hardware decoder does not support AV1 format.
Signed-off-by: DEEPA GUTHYAPPA MADIVALARA <deepa.madivalara@oss.qualcomm.com>
---
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 1 +
drivers/media/platform/qcom/iris/iris_instance.h | 1 +
.../platform/qcom/iris/iris_platform_common.h | 2 ++
.../media/platform/qcom/iris/iris_platform_gen2.c | 23 ++++++++++++++++++++
.../platform/qcom/iris/iris_platform_sm8250.c | 17 +++++++++++++++
drivers/media/platform/qcom/iris/iris_vdec.c | 25 ++++++----------------
6 files changed, 50 insertions(+), 19 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index aa1f795f5626c1f76a32dd650302633877ce67be..bbfe7a0851ea94fb7041a868b4df8b2ec63bf427 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -138,6 +138,7 @@ enum hfi_codec_type {
HFI_CODEC_DECODE_HEVC = 3,
HFI_CODEC_ENCODE_HEVC = 4,
HFI_CODEC_DECODE_VP9 = 5,
+ HFI_CODEC_DECODE_AV1 = 7,
};
enum hfi_picture_type {
diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
index 5982d7adefeab80905478b32cddba7bd4651a691..f1883ffc138fd975fb76d4e45904ee04e196cd20 100644
--- a/drivers/media/platform/qcom/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/iris/iris_instance.h
@@ -19,6 +19,7 @@ enum iris_fmt_type {
IRIS_FMT_H264,
IRIS_FMT_HEVC,
IRIS_FMT_VP9,
+ IRIS_FMT_AV1,
};
struct iris_fmt {
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 96fa7b1bb592441e85664da408ea4ba42c9a15b5..99057a624cb976af68403ef042173b5ebefde8af 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -208,6 +208,8 @@ struct iris_platform_data {
u64 dma_mask;
const char *fwname;
u32 pas_id;
+ struct iris_fmt *inst_iris_fmts;
+ u32 inst_iris_fmts_size;
struct platform_inst_caps *inst_caps;
struct platform_inst_fw_cap *inst_fw_caps_dec;
u32 inst_fw_caps_dec_size;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index cf4b92f534b272a0a1ac2a0e7bb9316501374332..fecf2e06f19723f30777c04bc4b155954c2b746d 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -16,6 +16,25 @@
#define VIDEO_ARCH_LX 1
#define BITRATE_MAX 245000000
+static struct iris_fmt platform_fmts_sm8550_dec[] = {
+ [IRIS_FMT_H264] = {
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_HEVC] = {
+ .pixfmt = V4L2_PIX_FMT_HEVC,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_VP9] = {
+ .pixfmt = V4L2_PIX_FMT_VP9,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_AV1] = {
+ .pixfmt = V4L2_PIX_FMT_AV1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }
+};
+
static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
{
.cap_id = PROFILE_H264,
@@ -756,6 +775,8 @@ struct iris_platform_data sm8550_data = {
.dma_mask = 0xe0000000 - 1,
.fwname = "qcom/vpu/vpu30_p4.mbn",
.pas_id = IRIS_PAS_ID,
+ .inst_iris_fmts = platform_fmts_sm8550_dec,
+ .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
.inst_caps = &platform_inst_cap_sm8550,
.inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
@@ -847,6 +868,8 @@ struct iris_platform_data sm8650_data = {
.dma_mask = 0xe0000000 - 1,
.fwname = "qcom/vpu/vpu33_p4.mbn",
.pas_id = IRIS_PAS_ID,
+ .inst_iris_fmts = platform_fmts_sm8550_dec,
+ .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
.inst_caps = &platform_inst_cap_sm8550,
.inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
index 978d0130d43b5f6febb65430a9bbe3932e8f24df..947dd25a483a792681e34fbbf4396342db582203 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
@@ -16,6 +16,21 @@
#define BITRATE_PEAK_DEFAULT (BITRATE_DEFAULT * 2)
#define BITRATE_STEP 100
+static struct iris_fmt platform_fmts_sm8250_dec[] = {
+ [IRIS_FMT_H264] = {
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_HEVC] = {
+ .pixfmt = V4L2_PIX_FMT_HEVC,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+ [IRIS_FMT_VP9] = {
+ .pixfmt = V4L2_PIX_FMT_VP9,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }
+};
+
static struct platform_inst_fw_cap inst_fw_cap_sm8250_dec[] = {
{
.cap_id = PIPE,
@@ -335,6 +350,8 @@ struct iris_platform_data sm8250_data = {
.dma_mask = 0xe0000000 - 1,
.fwname = "qcom/vpu-1.0/venus.mbn",
.pas_id = IRIS_PAS_ID,
+ .inst_iris_fmts = platform_fmts_sm8250_dec,
+ .inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8250_dec),
.inst_caps = &platform_inst_cap_sm8250,
.inst_fw_caps_dec = inst_fw_cap_sm8250_dec,
.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8250_dec),
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index ae13c3e1b426bfd81a7b46dc6c3ff5eb5c4860cb..be8d2d48c82f385e4f46807f7e0dd52e469927cd 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -67,26 +67,12 @@ void iris_vdec_inst_deinit(struct iris_inst *inst)
kfree(inst->fmt_src);
}
-static const struct iris_fmt iris_vdec_formats[] = {
- [IRIS_FMT_H264] = {
- .pixfmt = V4L2_PIX_FMT_H264,
- .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
- },
- [IRIS_FMT_HEVC] = {
- .pixfmt = V4L2_PIX_FMT_HEVC,
- .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
- },
- [IRIS_FMT_VP9] = {
- .pixfmt = V4L2_PIX_FMT_VP9,
- .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
- },
-};
-
static const struct iris_fmt *
find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
{
- unsigned int size = ARRAY_SIZE(iris_vdec_formats);
- const struct iris_fmt *fmt = iris_vdec_formats;
+ unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
+ const struct iris_fmt *fmt =
+ inst->core->iris_platform_data->inst_iris_fmts;
unsigned int i;
for (i = 0; i < size; i++) {
@@ -103,8 +89,9 @@ find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
static const struct iris_fmt *
find_format_by_index(struct iris_inst *inst, u32 index, u32 type)
{
- const struct iris_fmt *fmt = iris_vdec_formats;
- unsigned int size = ARRAY_SIZE(iris_vdec_formats);
+ unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
+ const struct iris_fmt *fmt =
+ inst->core->iris_platform_data->inst_iris_fmts;
if (index >= size || fmt[index].type != type)
return NULL;
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH RFC 4/5] media: iris: Add internal buffer calculation for AV1 decoder
2025-09-02 23:00 [PATCH RFC 0/5] Enable support for AV1 stateful decoder DEEPA GUTHYAPPA MADIVALARA
` (2 preceding siblings ...)
2025-09-02 23:00 ` [PATCH RFC 3/5] media: iris: Add support for AV1 format in iris decoder DEEPA GUTHYAPPA MADIVALARA
@ 2025-09-02 23:00 ` DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 5/5] media: iris: Define AV1-specific platform capabilities and properties DEEPA GUTHYAPPA MADIVALARA
2025-09-03 13:07 ` [PATCH RFC 0/5] Enable support for AV1 stateful decoder Nicolas Dufresne
5 siblings, 0 replies; 8+ messages in thread
From: DEEPA GUTHYAPPA MADIVALARA @ 2025-09-02 23:00 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm,
Deepa Guthyappa Madivalara
Implement internal buffer count and size calculations for AV1 decoder.
Signed-off-by: DEEPA GUTHYAPPA MADIVALARA <deepa.madivalara@oss.qualcomm.com>
---
drivers/media/platform/qcom/iris/iris_buffer.h | 2 +
.../platform/qcom/iris/iris_hfi_gen2_command.c | 109 ++++++++-
.../platform/qcom/iris/iris_hfi_gen2_response.c | 3 +
.../media/platform/qcom/iris/iris_platform_gen2.c | 1 +
drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++-
drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++
7 files changed, 468 insertions(+), 8 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
index 325d30fce5c99185b61ff989fbfd4de9a56762b2..75bb767761824c4c02e0df9b765896cc093be333 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_buffer.h
@@ -27,6 +27,7 @@ struct iris_inst;
* @BUF_SCRATCH_1: buffer to store decoding/encoding context data for HW
* @BUF_SCRATCH_2: buffer to store encoding context data for HW
* @BUF_VPSS: buffer to store VPSS context data for HW
+ * @BUF_PARTIAL: buffer for AV1 IBC data
* @BUF_TYPE_MAX: max buffer types
*/
enum iris_buffer_type {
@@ -42,6 +43,7 @@ enum iris_buffer_type {
BUF_SCRATCH_1,
BUF_SCRATCH_2,
BUF_VPSS,
+ BUF_PARTIAL,
BUF_TYPE_MAX,
};
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
index 4ce71a14250832440099e4cf3835b4aedfb749e8..e3a8b031b3f191a6d18e1084db34804a8172439c 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -10,6 +10,7 @@
#define UNSPECIFIED_COLOR_FORMAT 5
#define NUM_SYS_INIT_PACKETS 8
+#define NUM_COMV_AV1 18
#define SYS_INIT_PKT_SIZE (sizeof(struct iris_hfi_header) + \
NUM_SYS_INIT_PACKETS * (sizeof(struct iris_hfi_packet) + sizeof(u32)))
@@ -121,6 +122,7 @@ static u32 iris_hfi_gen2_get_port_from_buf_type(struct iris_inst *inst,
case BUF_COMV:
case BUF_NON_COMV:
case BUF_LINE:
+ case BUF_PARTIAL:
return HFI_PORT_BITSTREAM;
case BUF_OUTPUT:
case BUF_DPB:
@@ -380,6 +382,9 @@ static int iris_hfi_gen2_set_profile(struct iris_inst *inst, u32 plane)
case V4L2_PIX_FMT_H264:
profile = inst->fw_caps[PROFILE_H264].value;
break;
+ case V4L2_PIX_FMT_AV1:
+ profile = inst->fw_caps[PROFILE_AV1].value;
+ break;
}
inst_hfi_gen2->src_subcr_params.profile = profile;
@@ -409,6 +414,9 @@ static int iris_hfi_gen2_set_level(struct iris_inst *inst, u32 plane)
case V4L2_PIX_FMT_H264:
level = inst->fw_caps[LEVEL_H264].value;
break;
+ case V4L2_PIX_FMT_AV1:
+ level = inst->fw_caps[LEVEL_AV1].value;
+ break;
}
inst_hfi_gen2->src_subcr_params.level = level;
@@ -484,7 +492,8 @@ static int iris_hfi_gen2_set_tier(struct iris_inst *inst, u32 plane)
u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
u32 tier = inst->fw_caps[TIER].value;
- inst_hfi_gen2->src_subcr_params.tier = tier;
+ tier = (inst->codec == V4L2_PIX_FMT_AV1) ? inst->fw_caps[TIER_AV1].value :
+ inst->fw_caps[TIER].value;
return iris_hfi_gen2_session_set_property(inst,
HFI_PROP_TIER,
@@ -509,6 +518,56 @@ static int iris_hfi_gen2_set_frame_rate(struct iris_inst *inst, u32 plane)
sizeof(u32));
}
+static int iris_hfi_gen2_set_film_grain(struct iris_inst *inst, u32 plane)
+{
+ u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
+ u32 film_grain;
+
+ film_grain = inst->fw_caps[FILM_GRAIN].value;
+ inst_hfi_gen2->src_subcr_params.film_grain = film_grain;
+
+ return iris_hfi_gen2_session_set_property(inst,
+ HFI_PROP_AV1_FILM_GRAIN_PRESENT,
+ HFI_HOST_FLAGS_NONE,
+ port,
+ HFI_PAYLOAD_U32_ENUM,
+ &film_grain,
+ sizeof(u32));
+}
+
+static int iris_hfi_gen2_set_super_block(struct iris_inst *inst, u32 plane)
+{
+ u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
+ u32 super_block;
+
+ super_block = inst->fw_caps[SUPER_BLOCK].value;
+ inst_hfi_gen2->src_subcr_params.super_block = super_block;
+
+ return iris_hfi_gen2_session_set_property(inst,
+ HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
+ HFI_HOST_FLAGS_NONE,
+ port,
+ HFI_PAYLOAD_U32_ENUM,
+ &super_block,
+ sizeof(u32));
+}
+
+static int iris_hfi_gen2_set_opb_enable(struct iris_inst *inst, u32 plane)
+{
+ u32 port = iris_hfi_gen2_get_port(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ u32 opb_enable = iris_split_mode_enabled(inst);
+
+ return iris_hfi_gen2_session_set_property(inst,
+ HFI_PROP_OPB_ENABLE,
+ HFI_HOST_FLAGS_NONE,
+ port,
+ HFI_PAYLOAD_U32,
+ &opb_enable,
+ sizeof(u32));
+}
+
static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 plane)
{
const struct iris_platform_data *pdata = inst->core->iris_platform_data;
@@ -531,6 +590,9 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
{HFI_PROP_LINEAR_STRIDE_SCANLINE, iris_hfi_gen2_set_linear_stride_scanline },
{HFI_PROP_TIER, iris_hfi_gen2_set_tier },
{HFI_PROP_FRAME_RATE, iris_hfi_gen2_set_frame_rate },
+ {HFI_PROP_AV1_FILM_GRAIN_PRESENT, iris_hfi_gen2_set_film_grain },
+ {HFI_PROP_AV1_SUPER_BLOCK_ENABLED, iris_hfi_gen2_set_super_block },
+ {HFI_PROP_OPB_ENABLE, iris_hfi_gen2_set_opb_enable },
};
if (inst->domain == DECODER) {
@@ -544,12 +606,20 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
} else if (inst->codec == V4L2_PIX_FMT_VP9) {
config_params = pdata->dec_input_config_params_vp9;
config_params_size = pdata->dec_input_config_params_vp9_size;
+ } else if (inst->codec == V4L2_PIX_FMT_AV1) {
+ config_params = pdata->dec_input_config_params_av1;
+ config_params_size = pdata->dec_input_config_params_av1_size;
} else {
return -EINVAL;
}
} else {
- config_params = pdata->dec_output_config_params;
- config_params_size = pdata->dec_output_config_params_size;
+ if (inst->codec == V4L2_PIX_FMT_AV1) {
+ config_params = pdata->dec_output_config_params_av1;
+ config_params_size = pdata->dec_output_config_params_av1_size;
+ } else {
+ config_params = pdata->dec_output_config_params;
+ config_params_size = pdata->dec_output_config_params_size;
+ }
}
} else {
if (V4L2_TYPE_IS_OUTPUT(plane)) {
@@ -598,6 +668,9 @@ static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst)
break;
case V4L2_PIX_FMT_VP9:
codec = HFI_CODEC_DECODE_VP9;
+ break;
+ case V4L2_PIX_FMT_AV1:
+ codec = HFI_CODEC_DECODE_AV1;
}
iris_hfi_gen2_packet_session_property(inst,
@@ -763,6 +836,11 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
change_param_size =
core->iris_platform_data->dec_input_config_params_vp9_size;
break;
+ case V4L2_PIX_FMT_AV1:
+ change_param = core->iris_platform_data->dec_input_config_params_av1;
+ change_param_size =
+ core->iris_platform_data->dec_input_config_params_av1_size;
+ break;
}
payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
@@ -845,6 +923,16 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
payload_size = sizeof(u32);
payload_type = HFI_PAYLOAD_U32;
break;
+ case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
+ payload[0] = subsc_params.film_grain;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
+ payload[0] = subsc_params.super_block;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
default:
prop_type = 0;
ret = -EINVAL;
@@ -900,6 +988,11 @@ static int iris_hfi_gen2_subscribe_property(struct iris_inst *inst, u32 plane)
subscribe_prop_size =
core->iris_platform_data->dec_output_prop_vp9_size;
break;
+ case V4L2_PIX_FMT_AV1:
+ subcribe_prop = core->iris_platform_data->dec_output_prop_av1;
+ subscribe_prop_size =
+ core->iris_platform_data->dec_output_prop_av1_size;
+ break;
}
}
@@ -1075,6 +1168,8 @@ static u32 iris_hfi_gen2_buf_type_from_driver(u32 domain, enum iris_buffer_type
return HFI_BUFFER_ARP;
case BUF_VPSS:
return HFI_BUFFER_VPSS;
+ case BUF_PARTIAL:
+ return HFI_BUFFER_PARTIAL_DATA;
default:
return 0;
}
@@ -1087,7 +1182,13 @@ static int iris_set_num_comv(struct iris_inst *inst)
u32 num_comv;
caps = core->iris_platform_data->inst_caps;
- num_comv = caps->num_comv;
+
+ /*
+ * If Host allocates less comv count for AV1 then
+ * FW can raise SESSION_ERROR
+ */
+ num_comv = (inst->codec == V4L2_PIX_FMT_AV1) ?
+ NUM_COMV_AV1 : caps->num_comv;
return core->hfi_ops->session_set_property(inst,
HFI_PROP_COMV_BUFFER_COUNT,
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 2f1f118eae4f6462ab1aa1d16844b34e6e699f1e..943d3884248fafccc3b8e944e455c96124914353 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -54,6 +54,8 @@ static u32 iris_hfi_gen2_buf_type_to_driver(struct iris_inst *inst,
return BUF_SCRATCH_2;
case HFI_BUFFER_PERSIST:
return BUF_PERSIST;
+ case HFI_BUFFER_PARTIAL_DATA:
+ return BUF_PARTIAL;
default:
return 0;
}
@@ -72,6 +74,7 @@ static bool iris_hfi_gen2_is_valid_hfi_buffer_type(u32 buffer_type)
case HFI_BUFFER_DPB:
case HFI_BUFFER_PERSIST:
case HFI_BUFFER_VPSS:
+ case HFI_BUFFER_PARTIAL_DATA:
return true;
default:
return false;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index fecf2e06f19723f30777c04bc4b155954c2b746d..c0a03c03bb9349a0e551ce6dc72b6f8b23ff696d 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -739,6 +739,7 @@ static const u32 sm8550_dec_ip_int_buf_tbl[] = {
BUF_COMV,
BUF_NON_COMV,
BUF_LINE,
+ BUF_PARTIAL,
};
static const u32 sm8550_dec_op_int_buf_tbl[] = {
diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
index 798c3613e57eac1742633d61c1482229dbc32562..43a77687a668423a5547c7f159622249634264e0 100644
--- a/drivers/media/platform/qcom/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/iris/iris_vidc.c
@@ -178,6 +178,7 @@ int iris_open(struct file *filp)
INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_1].list);
INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_2].list);
INIT_LIST_HEAD(&inst->buffers[BUF_VPSS].list);
+ INIT_LIST_HEAD(&inst->buffers[BUF_PARTIAL].list);
init_completion(&inst->completion);
init_completion(&inst->flush_completion);
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
index 7337d8d33715810669399d9f86b864d0eb002897..cfdd28f005986d5b14b18a97215933b171355f7a 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
@@ -9,6 +9,14 @@
#include "iris_hfi_gen2_defines.h"
#define HFI_MAX_COL_FRAME 6
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_HEIGHT (8)
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_WIDTH (32)
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_HEIGHT (8)
+#define HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_WIDTH (16)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_HEIGHT (4)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_WIDTH (48)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_HEIGHT (4)
+#define HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_WIDTH (24)
#ifndef SYSTEM_LAL_TILE10
#define SYSTEM_LAL_TILE10 192
@@ -39,6 +47,31 @@ static u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_p
return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h, num_vpp_pipes);
}
+static u32 size_av1d_hw_bin_buffer(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+ u32 size_yuv, size_bin_hdr, size_bin_res;
+
+ size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ?
+ ((BIN_BUFFER_THRESHOLD * 3) >> 1) :
+ ((frame_width * frame_height * 3) >> 1);
+ size_bin_hdr = size_yuv * AV1_CABAC_HDR_RATIO_HD_TOT;
+ size_bin_res = size_yuv * AV1_CABAC_RES_RATIO_HD_TOT;
+ size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+ size_bin_res = ALIGN(size_bin_res / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+
+ return size_bin_hdr + size_bin_res;
+}
+
+static u32 hfi_buffer_bin_av1d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+ u32 n_aligned_h = ALIGN(frame_height, 16);
+ u32 n_aligned_w = ALIGN(frame_width, 16);
+
+ return size_av1d_hw_bin_buffer(n_aligned_w, n_aligned_h, num_vpp_pipes);
+}
+
static u32 size_h265d_hw_bin_buffer(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
{
u32 product = frame_width * frame_height;
@@ -110,6 +143,20 @@ static u32 hfi_buffer_comv_h265d(u32 frame_width, u32 frame_height, u32 _comv_bu
return (_size * (_comv_bufcount)) + 512;
}
+static u32 hfi_buffer_comv_av1d(u32 frame_width, u32 frame_height, u32 comv_bufcount)
+{
+ u32 size;
+
+ size = 2 * ALIGN(MAX(((frame_width + 63) / 64) *
+ ((frame_height + 63) / 64) * 512,
+ ((frame_width + 127) / 128) *
+ ((frame_height + 127) / 128) * 2816),
+ DMA_ALIGNMENT);
+ size *= comv_bufcount;
+
+ return size;
+}
+
static u32 size_h264d_bse_cmd_buf(u32 frame_height)
{
u32 height = ALIGN(frame_height, 32);
@@ -174,6 +221,20 @@ static u32 hfi_buffer_persist_h264d(void)
DMA_ALIGNMENT);
}
+static u32 hfi_buffer_persist_av1d(u32 max_width, u32 max_height, u32 total_ref_count)
+{
+ u32 comv_size, size;
+
+ comv_size = hfi_buffer_comv_av1d(max_width, max_height, total_ref_count);
+ size = ALIGN((SIZE_AV1D_SEQUENCE_HEADER * 2 + SIZE_AV1D_METADATA +
+ AV1D_NUM_HW_PIC_BUF * (SIZE_AV1D_TILE_OFFSET + SIZE_AV1D_QM) +
+ AV1D_NUM_FRAME_HEADERS * (SIZE_AV1D_FRAME_HEADER +
+ 2 * SIZE_AV1D_PROB_TABLE) + comv_size + HDR10_HIST_EXTRADATA_SIZE +
+ SIZE_AV1D_METADATA * AV1D_NUM_HW_PIC_BUF), DMA_ALIGNMENT);
+
+ return ALIGN(size, DMA_ALIGNMENT);
+}
+
static u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
{
u32 size_bse = size_h264d_bse_cmd_buf(frame_height);
@@ -459,6 +520,148 @@ static u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height,
return ALIGN((size + vpss_lb_size), DMA_ALIGNMENT);
}
+static u32 size_av1d_lb_opb_wr1_nv12_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 y_width, y_width_a = 128;
+
+ y_width = ALIGN(frame_width, y_width_a);
+
+ return (256 * ((y_width + 31) / 32 + (AV1D_MAX_TILE_COLS - 1)));
+}
+
+static u32 size_av1d_lb_opb_wr1_tp10_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 y_width, y_width_a = 256;
+
+ y_width = ALIGN(frame_width, 192);
+ y_width = ALIGN(y_width * 4 / 3, y_width_a);
+
+ return (256 * ((y_width + 47) / 48 + (AV1D_MAX_TILE_COLS - 1)));
+}
+
+static u32 hfi_buffer_line_av1d(u32 frame_width, u32 frame_height,
+ bool is_opb, u32 num_vpp_pipes)
+{
+ u32 size, vpss_lb_size, opbwrbufsize, opbwr8, opbwr10;
+
+ size = ALIGN(size_av1d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_fe_left_data(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_av1d_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_av1d_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_av1d_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_av1d_lb_recon_dma_metadata_wr
+ (frame_width, frame_height), DMA_ALIGNMENT) * 2 +
+ ALIGN(size_av1d_qp(frame_width, frame_height), DMA_ALIGNMENT);
+ opbwr8 = size_av1d_lb_opb_wr1_nv12_ubwc(frame_width, frame_height);
+ opbwr10 = size_av1d_lb_opb_wr1_tp10_ubwc(frame_width, frame_height);
+ opbwrbufsize = opbwr8 >= opbwr10 ? opbwr8 : opbwr10;
+ size = ALIGN((size + opbwrbufsize), DMA_ALIGNMENT);
+ if (is_opb) {
+ size_vpss_lb(frame_width, frame_height);
+ size = ALIGN((size + vpss_lb_size) * 2, DMA_ALIGNMENT);
+ }
+
+ return size;
+}
+
+static u32 size_av1d_ibc_nv12_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 size;
+ u32 y_width_a = 128, y_height_a = 32;
+ u32 uv_width_a = 128, uv_height_a = 32;
+ u32 ybufsize, uvbufsize, y_width, y_height, uv_width, uv_height;
+ u32 y_meta_width_a = 64, y_meta_height_a = 16;
+ u32 uv_meta_width_a = 64, uv_meta_height_a = 16;
+ u32 meta_height, meta_stride, meta_size;
+ u32 tile_width_y = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_WIDTH;
+ u32 tile_height_y = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_Y_TILE_HEIGHT;
+ u32 tile_width_uv = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_WIDTH;
+ u32 tile_height_uv = HFI_COLOR_FORMAT_YUV420_NV12_UBWC_UV_TILE_HEIGHT;
+
+ y_width = ALIGN(frame_width, y_width_a);
+ y_height = ALIGN(frame_height, y_height_a);
+ uv_width = ALIGN(frame_width, uv_width_a);
+ uv_height = ALIGN(((frame_height + 1) >> 1), uv_height_a);
+ ybufsize = ALIGN((y_width * y_height), HFI_ALIGNMENT_4096);
+ uvbufsize = ALIGN(uv_width * uv_height, HFI_ALIGNMENT_4096);
+ size = ybufsize + uvbufsize;
+ meta_stride = ALIGN(((frame_width + (tile_width_y - 1)) / tile_width_y),
+ y_meta_width_a);
+ meta_height = ALIGN(((frame_height + (tile_height_y - 1)) / tile_height_y),
+ y_meta_height_a);
+ meta_size = ALIGN(meta_stride * meta_height, HFI_ALIGNMENT_4096);
+ size += meta_size;
+ meta_stride = ALIGN(((((frame_width + 1) >> 1) + (tile_width_uv - 1)) /
+ tile_width_uv), uv_meta_width_a);
+ meta_height = ALIGN(((((frame_height + 1) >> 1) + (tile_height_uv - 1)) /
+ tile_height_uv), uv_meta_height_a);
+ meta_size = ALIGN(meta_stride * meta_height, HFI_ALIGNMENT_4096);
+ size += meta_size;
+
+ return size;
+}
+
+static u32 size_av1d_ibc_tp10_ubwc(u32 frame_width, u32 frame_height)
+{
+ u32 size;
+ u32 y_width_a = 256, y_height_a = 16,
+ uv_width_a = 256, uv_height_a = 16;
+ u32 ybufsize, uvbufsize, y_width, y_height, uv_width, uv_height;
+ u32 y_meta_width_a = 64, y_meta_height_a = 16,
+ uv_meta_width_a = 64, uv_meta_height_a = 16;
+ u32 meta_height, meta_stride, meta_size;
+ u32 tile_width_y = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_WIDTH;
+ u32 tile_height_y = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_Y_TILE_HEIGHT;
+ u32 tile_width_uv = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_WIDTH;
+ u32 tile_height_uv = HFI_COLOR_FORMAT_YUV420_TP10_UBWC_UV_TILE_HEIGHT;
+
+ y_width = ALIGN(frame_width, 192);
+ y_width = ALIGN(y_width * 4 / 3, y_width_a);
+ y_height = ALIGN(frame_height, y_height_a);
+ uv_width = ALIGN(frame_width, 192);
+ uv_width = ALIGN(uv_width * 4 / 3, uv_width_a);
+ uv_height = ALIGN(((frame_height + 1) >> 1), uv_height_a);
+ ybufsize = ALIGN(y_width * y_height, HFI_ALIGNMENT_4096);
+ uvbufsize = ALIGN(uv_width * uv_height, HFI_ALIGNMENT_4096);
+ size = ybufsize + uvbufsize;
+ meta_stride = ALIGN(((frame_width + (tile_width_y - 1)) / tile_width_y),
+ y_meta_width_a);
+ meta_height = ALIGN(((frame_height + (tile_height_y - 1)) / tile_height_y),
+ y_meta_height_a);
+ meta_size = ALIGN(((frame_height + (meta_height - 1)) / meta_height), meta_stride);
+ size += meta_size;
+ meta_stride = ALIGN(((((frame_width + 1) >> 1) + (tile_width_uv - 1)) /
+ tile_width_uv), uv_meta_width_a);
+ meta_height = ALIGN(((((frame_height + 1) >> 1) + (tile_height_uv - 1)) /
+ tile_height_uv), uv_meta_height_a);
+ meta_size = ALIGN(meta_stride * meta_height, HFI_ALIGNMENT_4096);
+ size += meta_size;
+
+ return size;
+}
+
+static u32 hfi_buffer_ibc_av1d(u32 frame_width, u32 frame_height)
+{
+ u32 size, ibc8, ibc10;
+
+ ibc8 = size_av1d_ibc_nv12_ubwc(frame_width, frame_height);
+ ibc10 = size_av1d_ibc_tp10_ubwc(frame_width, frame_height);
+ size = ibc8 >= ibc10 ? ibc8 : ibc10;
+
+ return ALIGN(size, DMA_ALIGNMENT);
+}
+
static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
{
u32 num_vpp_pipes = inst->core->iris_platform_data->num_vpp_pipe;
@@ -472,6 +675,8 @@ static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
return hfi_buffer_bin_h265d(width, height, num_vpp_pipes);
else if (inst->codec == V4L2_PIX_FMT_VP9)
return hfi_buffer_bin_vp9d(width, height, num_vpp_pipes);
+ else if (inst->codec == V4L2_PIX_FMT_AV1)
+ return hfi_buffer_bin_av1d(width, height, num_vpp_pipes);
return 0;
}
@@ -487,18 +692,33 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
return hfi_buffer_comv_h264d(width, height, num_comv);
else if (inst->codec == V4L2_PIX_FMT_HEVC)
return hfi_buffer_comv_h265d(width, height, num_comv);
-
+ else if (inst->codec == V4L2_PIX_FMT_AV1) {
+ if (inst->fw_caps[DRAP].value)
+ return 0;
+ else
+ return hfi_buffer_comv_av1d(width, height, num_comv);
+ }
return 0;
}
static u32 iris_vpu_dec_persist_size(struct iris_inst *inst)
{
+ struct platform_inst_caps *caps;
+
if (inst->codec == V4L2_PIX_FMT_H264)
return hfi_buffer_persist_h264d();
else if (inst->codec == V4L2_PIX_FMT_HEVC)
return hfi_buffer_persist_h265d(0);
else if (inst->codec == V4L2_PIX_FMT_VP9)
return hfi_buffer_persist_vp9d();
+ else if (inst->codec == V4L2_PIX_FMT_AV1) {
+ caps = inst->core->iris_platform_data->inst_caps;
+ if (inst->fw_caps[DRAP].value)
+ return hfi_buffer_persist_av1d(caps->max_frame_width,
+ caps->max_frame_height, 16);
+ else
+ return hfi_buffer_persist_av1d(0, 0, 0);
+ }
return 0;
}
@@ -545,6 +765,8 @@ static u32 iris_vpu_dec_line_size(struct iris_inst *inst)
else if (inst->codec == V4L2_PIX_FMT_VP9)
return hfi_buffer_line_vp9d(width, height, out_min_count, is_opb,
num_vpp_pipes);
+ else if (inst->codec == V4L2_PIX_FMT_AV1)
+ return hfi_buffer_line_av1d(width, height, is_opb, num_vpp_pipes);
return 0;
}
@@ -556,6 +778,15 @@ static u32 iris_vpu_dec_scratch1_size(struct iris_inst *inst)
iris_vpu_dec_line_size(inst);
}
+static u32 iris_vpu_dec_partial_size(struct iris_inst *inst)
+{
+ struct v4l2_format *f = inst->fmt_src;
+ u32 height = f->fmt.pix_mp.height;
+ u32 width = f->fmt.pix_mp.width;
+
+ return hfi_buffer_ibc_av1d(width, height);
+}
+
static inline
u32 size_enc_single_pipe(u32 rc_type, u32 bitbin_size, u32 num_vpp_pipes,
u32 frame_width, u32 frame_height, u32 lcu_size)
@@ -1369,7 +1600,9 @@ static int output_min_count(struct iris_inst *inst)
/* fw_min_count > 0 indicates reconfig event has already arrived */
if (inst->fw_min_count) {
- if (iris_split_mode_enabled(inst) && inst->codec == V4L2_PIX_FMT_VP9)
+ if (iris_split_mode_enabled(inst) &&
+ (inst->codec == V4L2_PIX_FMT_VP9 ||
+ inst->codec == V4L2_PIX_FMT_VP9))
return min_t(u32, 4, inst->fw_min_count);
else
return inst->fw_min_count;
@@ -1377,6 +1610,8 @@ static int output_min_count(struct iris_inst *inst)
if (inst->codec == V4L2_PIX_FMT_VP9)
output_min_count = 9;
+ else if (inst->codec == V4L2_PIX_FMT_AV1)
+ output_min_count = 11;
return output_min_count;
}
@@ -1399,6 +1634,7 @@ int iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)
{BUF_PERSIST, iris_vpu_dec_persist_size },
{BUF_DPB, iris_vpu_dec_dpb_size },
{BUF_SCRATCH_1, iris_vpu_dec_scratch1_size },
+ {BUF_PARTIAL, iris_vpu_dec_partial_size },
};
static const struct iris_vpu_buf_type_handle enc_internal_buf_type_handle[] = {
@@ -1437,14 +1673,20 @@ static u32 internal_buffer_count(struct iris_inst *inst,
buffer_type == BUF_PERSIST) {
return 1;
} else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
- if (inst->codec == V4L2_PIX_FMT_H264 || inst->codec == V4L2_PIX_FMT_HEVC)
+ if (inst->codec == V4L2_PIX_FMT_H264 ||
+ inst->codec == V4L2_PIX_FMT_HEVC ||
+ inst->codec == V4L2_PIX_FMT_AV1)
return 1;
}
+
return 0;
}
static inline int iris_vpu_dpb_count(struct iris_inst *inst)
{
+ if (inst->codec == V4L2_PIX_FMT_AV1)
+ return 11;
+
if (iris_split_mode_enabled(inst)) {
return inst->fw_min_count ?
inst->fw_min_count : inst->buffers[BUF_OUTPUT].min_count;
@@ -1463,9 +1705,13 @@ int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type
return MIN_BUFFERS;
else
return output_min_count(inst);
+ case BUF_NON_COMV:
+ if (inst->codec == V4L2_PIX_FMT_AV1)
+ return 0;
+ else
+ return 1;
case BUF_BIN:
case BUF_COMV:
- case BUF_NON_COMV:
case BUF_LINE:
case BUF_PERSIST:
return internal_buffer_count(inst, buffer_type);
@@ -1473,6 +1719,7 @@ int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type
case BUF_SCRATCH_2:
case BUF_VPSS:
case BUF_ARP:
+ case BUF_PARTIAL:
return 1; /* internal buffer count needed by firmware is 1 */
case BUF_DPB:
return iris_vpu_dpb_count(inst);
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
index 1ff1b07ecbaa85345ca948affeb1c4a1c55e36b0..71fb2a408797c3b5e2eb25222b2dfd356f532d96 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
@@ -11,6 +11,7 @@ struct iris_inst;
#define MIN_BUFFERS 4
#define DMA_ALIGNMENT 256
+#define HFI_ALIGNMENT_4096 4096
#define NUM_HW_PIC_BUF 32
#define LCU_MAX_SIZE_PELS 64
@@ -80,6 +81,22 @@ struct iris_inst;
#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE 384
#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640
+#define AV1_CABAC_HDR_RATIO_HD_TOT 2
+#define AV1_CABAC_RES_RATIO_HD_TOT 2
+#define AV1D_LCU_MAX_SIZE_PELS 128
+#define AV1D_LCU_MIN_SIZE_PELS 64
+#define AV1D_MAX_TILE_COLS 64
+#define MAX_PE_NBR_DATA_LCU32_LINE_BUFFER_SIZE 192
+#define MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE 96
+#define AV1D_NUM_HW_PIC_BUF 16
+#define AV1D_NUM_FRAME_HEADERS 16
+#define SIZE_AV1D_SEQUENCE_HEADER 768
+#define SIZE_AV1D_METADATA 512
+#define SIZE_AV1D_FRAME_HEADER 1280
+#define SIZE_AV1D_TILE_OFFSET 65536
+#define SIZE_AV1D_QM 3328
+#define SIZE_AV1D_PROB_TABLE 22784
+
#define SIZE_SLICE_CMD_BUFFER (ALIGN(20480, 256))
#define SIZE_SPS_PPS_SLICE_HDR (2048 + 4096)
#define SIZE_BSE_SLICE_CMD_BUF ((((8192 << 2) + 7) & (~7)) * 3)
@@ -145,6 +162,94 @@ static inline u32 size_h264d_qp(u32 frame_width, u32 frame_height)
return DIV_ROUND_UP(frame_width, 64) * DIV_ROUND_UP(frame_height, 64) * 128;
}
+static inline u32 size_av1d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return (ALIGN(frame_width, AV1D_LCU_MAX_SIZE_PELS) * ((16 * 10) >> 3) +
+ ALIGN(frame_width, AV1D_LCU_MAX_SIZE_PELS) / 2 * ((16 * 6) >> 3) * 2);
+}
+
+static inline u32 size_av1d_lb_fe_left_data(u32 frame_width, u32 frame_height)
+{
+ return (32 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 2 +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 8) * 2 +
+ 24 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 24 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 2 +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 12) * 2 +
+ 24 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 16) +
+ 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 2 +
+ ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS * 12) * 2);
+}
+
+static inline u32 size_av1d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (10 * ((frame_width + AV1D_LCU_MIN_SIZE_PELS - 1) /
+ AV1D_LCU_MIN_SIZE_PELS) * 128 / 8);
+}
+
+static inline u32 size_av1d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (16 * ((ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) / 16) +
+ (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS)) +
+ 3 * 16 * (ALIGN(frame_height, AV1D_LCU_MAX_SIZE_PELS) /
+ AV1D_LCU_MIN_SIZE_PELS));
+}
+
+static inline u32 size_av1d_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (((frame_width + 7) / 8) * 16);
+}
+
+static inline u32 size_av1d_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (max(((frame_height + 15) / 16) *
+ MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max(((frame_height + 31) / 32) *
+ MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 63) / 64) *
+ MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)));
+}
+
+static inline u32 size_av1d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return (max(((frame_width + 15) / 16) *
+ MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE,
+ max(((frame_width + 31) / 32) *
+ MAX_PE_NBR_DATA_LCU32_LINE_BUFFER_SIZE,
+ ((frame_width + 63) / 64) *
+ MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE)));
+}
+
+static inline u32 size_av1d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return (max(((frame_width + 63) / 64) * 1280,
+ ((frame_width + 127) / 128) * 2304));
+}
+
+static inline u32 size_av1d_lb_recon_dma_metadata_wr(u32 frame_width,
+ u32 frame_height)
+{
+ return ((ALIGN(frame_height, 8) / (4 / 2)) * 64);
+}
+
+static inline u32 size_av1d_qp(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_qp(frame_width, frame_height);
+}
+
int iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type);
int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type);
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH RFC 5/5] media: iris: Define AV1-specific platform capabilities and properties
2025-09-02 23:00 [PATCH RFC 0/5] Enable support for AV1 stateful decoder DEEPA GUTHYAPPA MADIVALARA
` (3 preceding siblings ...)
2025-09-02 23:00 ` [PATCH RFC 4/5] media: iris: Add internal buffer calculation for AV1 decoder DEEPA GUTHYAPPA MADIVALARA
@ 2025-09-02 23:00 ` DEEPA GUTHYAPPA MADIVALARA
2025-09-03 13:07 ` [PATCH RFC 0/5] Enable support for AV1 stateful decoder Nicolas Dufresne
5 siblings, 0 replies; 8+ messages in thread
From: DEEPA GUTHYAPPA MADIVALARA @ 2025-09-02 23:00 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Vikash Garodia, Dikshita Agarwal,
Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm,
Deepa Guthyappa Madivalara
Defining platform specific capabilities specific to AV1 decoder.
Set and subscribe to manadatory properties to firmware for AV1.
Signed-off-by: DEEPA GUTHYAPPA MADIVALARA <deepa.madivalara@oss.qualcomm.com>
---
drivers/media/platform/qcom/iris/iris_ctrls.c | 8 ++
drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 9 ++
.../platform/qcom/iris/iris_hfi_gen2_response.c | 19 +++
.../platform/qcom/iris/iris_platform_common.h | 13 ++
.../media/platform/qcom/iris/iris_platform_gen2.c | 132 ++++++++++++++++++++-
6 files changed, 183 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
index 754a5ad718bc37630bb861012301df7a2e7342a1..620c7e1bd273e25febd8ca70dd1dcfb0b862692b 100644
--- a/drivers/media/platform/qcom/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
@@ -98,6 +98,10 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
return B_FRAME_QP_H264;
case V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP:
return B_FRAME_QP_HEVC;
+ case V4L2_CID_MPEG_VIDEO_AV1_PROFILE:
+ return PROFILE_AV1;
+ case V4L2_CID_MPEG_VIDEO_AV1_LEVEL:
+ return LEVEL_AV1;
default:
return INST_FW_CAP_MAX;
}
@@ -185,6 +189,10 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
return V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP;
case B_FRAME_QP_HEVC:
return V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP;
+ case PROFILE_AV1:
+ return V4L2_CID_MPEG_VIDEO_AV1_PROFILE;
+ case LEVEL_AV1:
+ return V4L2_CID_MPEG_VIDEO_AV1_LEVEL;
default:
return 0;
}
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h
index b51471fb32c70acee44c37f8e9dce0c6bc0b6ccc..3edb5ae582b49bea2e2408c4a5cfc0a742adc05f 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_common.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h
@@ -141,6 +141,9 @@ struct hfi_subscription_params {
u32 profile;
u32 level;
u32 tier;
+ u32 drap;
+ u32 film_grain;
+ u32 super_block;
};
u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries);
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index bbfe7a0851ea94fb7041a868b4df8b2ec63bf427..9cc0989c67d74a9e051725e9ee571a2ab9160519 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -89,9 +89,18 @@ enum hfi_seq_header_mode {
#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169
#define HFI_PROP_NO_OUTPUT 0x0300016a
#define HFI_PROP_BUFFER_MARK 0x0300016c
+#define HFI_PROP_WORST_COMPRESSION_RATIO 0x03000174
+#define HFI_PROP_WORST_COMPLEXITY_FACTOR 0x03000175
#define HFI_PROP_RAW_RESOLUTION 0x03000178
+#define HFI_PROP_AV1_FILM_GRAIN_PRESENT 0x03000180
+#define HFI_PROP_AV1_SUPER_BLOCK_ENABLED 0x03000181
+#define HFI_PROP_AV1_OP_POINT 0x03000182
+#define HFI_PROP_OPB_ENABLE 0x03000184
+#define HFI_PROP_AV1_TILE_ROWS_COLUMNS 0x03000187
+#define HFI_PROP_AV1_DRAP_CONFIG 0x03000189
#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
+#define HFI_PROP_AV1_UNIFORM_TILE_SPACING 0x03000197
#define HFI_PROP_END 0x03FFFFFF
#define HFI_SESSION_ERROR_BEGIN 0x04000000
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 943d3884248fafccc3b8e944e455c96124914353..eb3373f0ad4a1b26fb847db02449ec8d8cb3bdbb 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -599,6 +599,10 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
inst->fw_caps[PROFILE_H264].value = subsc_params.profile;
inst->fw_caps[LEVEL_H264].value = subsc_params.level;
break;
+ case V4L2_PIX_FMT_AV1:
+ inst->fw_caps[PROFILE_AV1].value = subsc_params.profile;
+ inst->fw_caps[LEVEL_AV1].value = subsc_params.level;
+ break;
}
inst->fw_caps[POC].value = subsc_params.pic_order_cnt;
@@ -611,6 +615,11 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
iris_inst_change_state(inst, IRIS_INST_ERROR);
}
+ if (inst->codec == V4L2_PIX_FMT_AV1) {
+ inst->fw_caps[FILM_GRAIN].value = subsc_params.film_grain;
+ inst->fw_caps[SUPER_BLOCK].value = subsc_params.super_block;
+ }
+
inst->fw_min_count = subsc_params.fw_min_count;
inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT);
inst->buffers[BUF_OUTPUT].size = pixmp_op->plane_fmt[0].sizeimage;
@@ -714,6 +723,12 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
case HFI_PROP_NO_OUTPUT:
inst_hfi_gen2->hfi_frame_info.no_output = 1;
break;
+ case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
+ inst_hfi_gen2->src_subcr_params.film_grain = pkt->payload[0];
+ break;
+ case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
+ inst_hfi_gen2->src_subcr_params.super_block = pkt->payload[0];
+ break;
case HFI_PROP_QUALITY_MODE:
case HFI_PROP_STAGE:
case HFI_PROP_PIPE:
@@ -844,6 +859,10 @@ static void iris_hfi_gen2_init_src_change_param(struct iris_inst *inst)
subsc_params->profile = inst->fw_caps[PROFILE_H264].value;
subsc_params->level = inst->fw_caps[LEVEL_H264].value;
break;
+ case V4L2_PIX_FMT_AV1:
+ subsc_params->profile = inst->fw_caps[PROFILE_AV1].value;
+ subsc_params->level = inst->fw_caps[LEVEL_AV1].value;
+ break;
}
subsc_params->pic_order_cnt = inst->fw_caps[POC].value;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 99057a624cb976af68403ef042173b5ebefde8af..e543a3cc142b15e50b12cfe672d00dd0acfdb4d1 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -95,6 +95,13 @@ enum platform_inst_fw_cap_type {
LEVEL_H264,
LEVEL_HEVC,
LEVEL_VP9,
+ PROFILE_AV1,
+ LEVEL_AV1,
+ TIER_AV1,
+ DRAP,
+ FILM_GRAIN,
+ SUPER_BLOCK,
+ ENH_LAYER_COUNT,
INPUT_BUF_HOST_MAX_COUNT,
OUTPUT_BUF_HOST_MAX_COUNT,
STAGE,
@@ -231,8 +238,12 @@ struct iris_platform_data {
unsigned int dec_input_config_params_hevc_size;
const u32 *dec_input_config_params_vp9;
unsigned int dec_input_config_params_vp9_size;
+ const u32 *dec_input_config_params_av1;
+ unsigned int dec_input_config_params_av1_size;
const u32 *dec_output_config_params;
unsigned int dec_output_config_params_size;
+ const u32 *dec_output_config_params_av1;
+ unsigned int dec_output_config_params_av1_size;
const u32 *enc_input_config_params;
unsigned int enc_input_config_params_size;
const u32 *enc_output_config_params;
@@ -245,6 +256,8 @@ struct iris_platform_data {
unsigned int dec_output_prop_hevc_size;
const u32 *dec_output_prop_vp9;
unsigned int dec_output_prop_vp9_size;
+ const u32 *dec_output_prop_av1;
+ unsigned int dec_output_prop_av1_size;
const u32 *dec_ip_int_buf_tbl;
unsigned int dec_ip_int_buf_tbl_size;
const u32 *dec_op_int_buf_tbl;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index c0a03c03bb9349a0e551ce6dc72b6f8b23ff696d..4c2ffe3ceecc88bb3c1830357ae27cdafb53791b 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -61,6 +61,16 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
.set = iris_set_u32_enum,
},
+ {
+ .cap_id = PROFILE_AV1,
+ .min = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
+ .max = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
+ .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN),
+ .value = V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN,
+ .hfi_id = HFI_PROP_PROFILE,
+ .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ .set = iris_set_u32_enum,
+ },
{
.cap_id = PROFILE_VP9,
.min = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
@@ -144,6 +154,33 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
.set = iris_set_u32_enum,
},
+ {
+ .cap_id = LEVEL_AV1,
+ .min = V4L2_MPEG_VIDEO_AV1_LEVEL_2_0,
+ .max = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
+ .step_or_mask = BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_2_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_3_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_4_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_5_3) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_0) |
+ BIT(V4L2_MPEG_VIDEO_AV1_LEVEL_6_1),
+ .value = V4L2_MPEG_VIDEO_AV1_LEVEL_6_1,
+ .hfi_id = HFI_PROP_LEVEL,
+ .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ .set = iris_set_u32_enum,
+ },
{
.cap_id = TIER,
.min = V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
@@ -155,6 +192,53 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
.set = iris_set_u32_enum,
},
+ {
+ .cap_id = TIER_AV1,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_TIER,
+ .flags = CAP_FLAG_INPUT_PORT,
+ .set = iris_set_u32,
+ },
+ {
+ .cap_id = DRAP,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_DRAP_CONFIG,
+ .flags = CAP_FLAG_INPUT_PORT,
+ .set = iris_set_u32,
+ },
+ {
+ .cap_id = FILM_GRAIN,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_FILM_GRAIN_PRESENT,
+ .flags = CAP_FLAG_VOLATILE,
+ },
+ {
+ .cap_id = SUPER_BLOCK,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
+ },
+ {
+ .cap_id = ENH_LAYER_COUNT,
+ .min = 0,
+ .max = 1,
+ .step_or_mask = 1,
+ .value = 0,
+ .hfi_id = HFI_PROP_AV1_OP_POINT,
+ .flags = CAP_FLAG_INPUT_PORT,
+ .set = iris_set_u32,
+ },
{
.cap_id = INPUT_BUF_HOST_MAX_COUNT,
.min = DEFAULT_MAX_HOST_BUF_COUNT,
@@ -698,6 +782,19 @@ static const u32 sm8550_vdec_input_config_param_vp9[] = {
HFI_PROP_LEVEL,
};
+static const u32 sm8550_vdec_input_config_param_av1[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+ HFI_PROP_TIER,
+ HFI_PROP_AV1_FILM_GRAIN_PRESENT,
+ HFI_PROP_AV1_SUPER_BLOCK_ENABLED,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+};
+
static const u32 sm8550_venc_input_config_params[] = {
HFI_PROP_COLOR_FORMAT,
HFI_PROP_RAW_RESOLUTION,
@@ -711,6 +808,12 @@ static const u32 sm8550_vdec_output_config_params[] = {
HFI_PROP_LINEAR_STRIDE_SCANLINE,
};
+static const u32 sm8550_vdec_output_config_param_av1[] = {
+ HFI_PROP_OPB_ENABLE,
+ HFI_PROP_COLOR_FORMAT,
+ HFI_PROP_LINEAR_STRIDE_SCANLINE,
+};
+
static const u32 sm8550_venc_output_config_params[] = {
HFI_PROP_BITSTREAM_RESOLUTION,
HFI_PROP_CROP_OFFSETS,
@@ -734,6 +837,12 @@ static const u32 sm8550_vdec_subscribe_output_properties_vp9[] = {
HFI_PROP_PICTURE_TYPE,
};
+static const u32 sm8550_vdec_subscribe_output_properties_av1[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_WORST_COMPRESSION_RATIO,
+ HFI_PROP_WORST_COMPLEXITY_FACTOR,
+};
+
static const u32 sm8550_dec_ip_int_buf_tbl[] = {
BUF_BIN,
BUF_COMV,
@@ -803,11 +912,18 @@ struct iris_platform_data sm8550_data = {
sm8550_vdec_input_config_param_vp9,
.dec_input_config_params_vp9_size =
ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
+ .dec_input_config_params_av1 =
+ sm8550_vdec_input_config_param_av1,
+ .dec_input_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
.dec_output_config_params =
sm8550_vdec_output_config_params,
.dec_output_config_params_size =
ARRAY_SIZE(sm8550_vdec_output_config_params),
-
+ .dec_output_config_params_av1 =
+ sm8550_vdec_output_config_param_av1,
+ .dec_output_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
.enc_input_config_params =
sm8550_venc_input_config_params,
.enc_input_config_params_size =
@@ -828,6 +944,9 @@ struct iris_platform_data sm8550_data = {
.dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
.dec_output_prop_vp9_size =
ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
+ .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
+ .dec_output_prop_av1_size =
+ ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
.dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
.dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
@@ -896,10 +1015,18 @@ struct iris_platform_data sm8650_data = {
sm8550_vdec_input_config_param_vp9,
.dec_input_config_params_vp9_size =
ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
+ .dec_input_config_params_av1 =
+ sm8550_vdec_input_config_param_av1,
+ .dec_input_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_input_config_param_av1),
.dec_output_config_params =
sm8550_vdec_output_config_params,
.dec_output_config_params_size =
ARRAY_SIZE(sm8550_vdec_output_config_params),
+ .dec_output_config_params_av1 =
+ sm8550_vdec_output_config_param_av1,
+ .dec_output_config_params_av1_size =
+ ARRAY_SIZE(sm8550_vdec_output_config_param_av1),
.enc_input_config_params =
sm8550_venc_input_config_params,
@@ -921,6 +1048,9 @@ struct iris_platform_data sm8650_data = {
.dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
.dec_output_prop_vp9_size =
ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
+ .dec_output_prop_av1 = sm8550_vdec_subscribe_output_properties_av1,
+ .dec_output_prop_av1_size =
+ ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_av1),
.dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
.dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH RFC 0/5] Enable support for AV1 stateful decoder
2025-09-02 23:00 [PATCH RFC 0/5] Enable support for AV1 stateful decoder DEEPA GUTHYAPPA MADIVALARA
` (4 preceding siblings ...)
2025-09-02 23:00 ` [PATCH RFC 5/5] media: iris: Define AV1-specific platform capabilities and properties DEEPA GUTHYAPPA MADIVALARA
@ 2025-09-03 13:07 ` Nicolas Dufresne
5 siblings, 0 replies; 8+ messages in thread
From: Nicolas Dufresne @ 2025-09-03 13:07 UTC (permalink / raw)
To: DEEPA GUTHYAPPA MADIVALARA, Mauro Carvalho Chehab, Vikash Garodia,
Dikshita Agarwal, Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 14144 bytes --]
Le mardi 02 septembre 2025 à 16:00 -0700, DEEPA GUTHYAPPA MADIVALARA a écrit :
> Hi all,
>
> This patch series adds initial support for the AV1 stateful
> decoder codecs in iris decoder. Also it adds support for AV1
> stateful decoder in V4l2. The objective of this work is to
> extend the Iris decoder's capabilities to handle AV1 format
> codec streams, including necessary format handling and buffer
> management. I'm sharing this series as an RFC because conformance
> testing and gstreamer testing are still in progress. While initial
> functional tests show positive results, I would appreciate early
> feedback on the design, implementation, and fixes before moving to
> a formal submission. I plan to submit a formal patch series after
> completing all the compliance checks. Meanwhile, any feedback or
> suggestion to improve this work are very welcome and will be of
> great help.
>
> Gstreamer testing:
> Gstreamer MR for enabling AV1 stateful decoder:
> https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1
>
> Thanks to Nicolas Dufresne for proving the MR
> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469
>
> However, Gst testing with parsebin is not resolving to av1parser as
> below:
> Ex: With the following command parsebin is unable to resolve to
> av1parser.
> GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault
> filesrc
> location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/
> argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu
> ! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none !
> video/x-raw,format=I420
> ! filesink location=gst_decoder_output.yuv
Be aware that Argon integration with fluster/GStreamer still needs some work,
see issue tracker:
https://github.com/fluendo/fluster/issues/222
It is fine to proceed with testing the other (IVF or MKV based) test suites.
Feel free to report all wrongly identified files onto that issue, this will be
addressed in GStreamer (and eventually in FFMPEG too if need be).
>
> 0:00:00.051674896 400 0xffff8c000b90 DEBUG
> parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps
> video/x-h263, variant=(string)itu
>
> The same test with the av1parse command parses correctly:
> GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault
> filesrc
> location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/
> argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu
> ! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none !
> video/x-raw,format=I420
> ! filesink location=/tmp/gst_decoder_output.yuv
>
> Got EOS from element "pipeline0".
> Execution ended after 0:00:01.599088176
> Setting pipeline to NULL ...
> 0:00:03.580831249 1075 0x3354f960 DEBUG
> av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop
>
> Fluster testing:
> As fluster.py is using parsebin for gstreamer, seeing the same issue as
> described above for the following testsuites.
> AV1-ARGON-PROFILE0-CORE-ANNEX-B
> AV1-ARGON-PROFILE0-NON-ANNEX-B
> AV1-ARGON-PROFILE0-NON-ANNEX-B
>
> Test suite: AV1-TEST-VECTORS
> The result of fluster test on SM8550:
> 134/242 testcases passed while testing AV1-TEST-VECTORS with
> GStreamer-AV1-V4L2-Gst1.0
> unsupported content, bit depth: a000a (66 tests)
> Iris hardware decoder supports only 8bit NV12
> av1-1-b10-00-quantizer-*
That is interesting, I believe there was no profile without 10bit, which would
mean this is non-compliant hardware. Not a blocker, just a remark, and me being
surprised 10bit isn't supported on modern codec in 2025.
>
> Unsupported resolution (36 tests).
> Iris hardware decoder supports min resolution of 96x96
> av1-1-b8-01-size-*
>
> Unsupported colorformat (1 test)
> av1-1-b8-24-monochrome
Fair enough.
>
> Crc mismatch: debug in progress (5tests)
> av1-1-b8-03-sizeup
> av1-1-b8-03-sizedown
These should be basic dynamic resolution changes cases, let me know what you
find.
> av1-1-b8-16-intra_only-intrabc-extreme-dv
No idea about this one.
> av1-1-b8-22-svc-L2T1
> av1-1-b8-22-svc-L2T2
These two are spatial SVC. That means that some decode only frames are going to
be smaller in dimensions. Only the last frame of a TU is displayed. Both do have
the optional sequence header announcing the maximum resolution though. On
stateless side, these usually works by simply decoding the hidden frames into
the much larger buffers. Dealing with firmware can be harder, since firmware may
silently drop the decode only frames, which leads to wrong timestamp matching,
which could have side effect in frameworks.
On stateful decoding, SVC have never been done, some documentation will likely
be needed. Strictly unsupported is the case the sequence header is not there,
which results in resolution change on non-keyframe. We now have the ability to
allocate frames of different dimension at run-time, and free old unused frames,
but in stateful, decoders streamoff the capture queue, which currently results
in lost of references.
Let us know your progress, inter-frame resolution change is not a strict
requirement, it is quite rare in real life.
looking forward, there is nothing particularly alarming in this report. Thanks
for the transparency.
Nicolas
>
> Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS
> 12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with
> GStreamer-AV1-V4L2-Gst1.0
> Crc mismatch: debug in progress
> av1-1-b8-03-sizeup.ivf
>
> Unsupported test suites:
> Iris Hardware Decoder supports only
> PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN
> and 8 bit, 420 only
> AV1-ARGON-PROFILE1-CORE-ANNEX-B
> AV1-ARGON-PROFILE1-NON-ANNEX-B
> AV1-ARGON-PROFILE1-STRESS-ANNEX-B
> AV1-ARGON-PROFILE2-CORE-ANNEX-B
> AV1-ARGON-PROFILE2-NON-ANNEX-B
> AV1-ARGON-PROFILE2-STRESS-ANNEX-B
> CHROMIUM-10bit-AV1-TEST-VECTORS
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : Iris Decoder
> Bus info : platform:aa00000.video-codec
> Driver version : 6.16.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Detected Stateful Decoder
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
> test invalid ioctls: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 12 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test CREATE_BUFS maximum buffers: OK
> test VIDIOC_REMOVE_BUFS: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
> [65391.311675] qcom-iris aa00000.video-codec: invalid plane
> [65395.340586] qcom-iris aa00000.video-codec: invalid plane
> test blocking wait: OK
>
> Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0,
> Warnings: 0
>
> V4l2-ctl Test verified for 2 streams as well.
>
> Thanks,
> Deepa
>
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> ---
> DEEPA GUTHYAPPA MADIVALARA (5):
> media: uapi: videodev2: Add support for AV1 stateful decoder
> media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
> media: iris: Add support for AV1 format in iris decoder
> media: iris: Add internal buffer calculation for AV1 decoder
> media: iris: Define AV1-specific platform capabilities and properties
>
> drivers/media/platform/qcom/iris/iris_buffer.h | 2 +
> drivers/media/platform/qcom/iris/iris_ctrls.c | 8 +
> drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
> .../platform/qcom/iris/iris_hfi_gen2_command.c | 109 ++++++++-
> .../platform/qcom/iris/iris_hfi_gen2_defines.h | 10 +
> .../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++
> drivers/media/platform/qcom/iris/iris_instance.h | 1 +
> .../platform/qcom/iris/iris_platform_common.h | 15 ++
> .../media/platform/qcom/iris/iris_platform_gen2.c | 156 ++++++++++++-
> .../platform/qcom/iris/iris_platform_sm8250.c | 17 ++
> drivers/media/platform/qcom/iris/iris_vdec.c | 25 +-
> drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
> drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++-
> drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/uapi/linux/videodev2.h | 1 +
> 16 files changed, 703 insertions(+), 28 deletions(-)
> ---
> base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f
> change-id: 20250821-rfc_split-c3ff834bb2c9
> prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3
> prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c
> prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e
> prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4
> prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3
> prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2
> prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce
> prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0
> prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca
> prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e
> prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec
> prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759
> prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7
> prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1
> prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0
> prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc
> prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5
> prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c
> prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3
> prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3
> prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5
> prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968
> prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5
> prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423
> prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd
> prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa
> prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83
>
> Best regards,
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH RFC 1/5] media: uapi: videodev2: Add support for AV1 stateful decoder
2025-09-02 23:00 ` [PATCH RFC 1/5] media: uapi: videodev2: Add " DEEPA GUTHYAPPA MADIVALARA
@ 2025-09-03 13:17 ` Nicolas Dufresne
0 siblings, 0 replies; 8+ messages in thread
From: Nicolas Dufresne @ 2025-09-03 13:17 UTC (permalink / raw)
To: DEEPA GUTHYAPPA MADIVALARA, Mauro Carvalho Chehab, Vikash Garodia,
Dikshita Agarwal, Abhinav Kumar, Bryan O'Donoghue
Cc: linux-media, linux-kernel, linux-arm-msm
[-- Attachment #1: Type: text/plain, Size: 2247 bytes --]
Le mardi 02 septembre 2025 à 16:00 -0700, DEEPA GUTHYAPPA MADIVALARA a écrit :
> Introduce a new pixel format, V4L2_PIX_FMT_AV1, to the
> Video4Linux2(V4L2) API. This format is intended for AV1
> bitstreams in stateful decoding/encoding workflows.
> The fourcc code 'AV10' is used to distinguish
> this format from the existing V4L2_PIX_FMT_AV1_FRAME,
> which is used for stateless AV1 decoder implementation.
>
> Signed-off-by: DEEPA GUTHYAPPA MADIVALARA <deepa.madivalara@oss.qualcomm.com>
> ---
> include/uapi/linux/videodev2.h | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 3dd9fa45dde1066d52a68581625a39e7ec92c9b7..bff42a71c67b3f4b570dd6f3d250f1bb482ec8ae 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -775,6 +775,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
> #define V4L2_PIX_FMT_AV1_FRAME v4l2_fourcc('A', 'V', '1', 'F') /* AV1 parsed frame */
> +#define V4L2_PIX_FMT_AV1 v4l2_fourcc('A', 'V', '1', '0') /* AV1 (stateful) */
You also need to update Documentation/userspace-api/media/v4l/pixfmt-
compressed.rst. Mistakes were made in the past leading to great confusion, AV1
can be wrapped in different forms and with different alignments. Here's few
question that should be answered in the RST documentation:
- Should we pass complete time units to the decoder ?
- If not
- How is the timestamp going to be interpreted and transferred ?
- Should userspace pass complete frame (with leading headers) ? or can it
pass OBUs ?
- Is that format meant to be OBU streams or Annex B (not to confused with H.264
annex B, it groups the TUs for faster skipping) ? (I bet the first :-D)
cheers,
Nicolas
> #define V4L2_PIX_FMT_SPK v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
> #define V4L2_PIX_FMT_RV30 v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
> #define V4L2_PIX_FMT_RV40 v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-09-03 13:17 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-02 23:00 [PATCH RFC 0/5] Enable support for AV1 stateful decoder DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 1/5] media: uapi: videodev2: Add " DEEPA GUTHYAPPA MADIVALARA
2025-09-03 13:17 ` Nicolas Dufresne
2025-09-02 23:00 ` [PATCH RFC 2/5] media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 3/5] media: iris: Add support for AV1 format in iris decoder DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 4/5] media: iris: Add internal buffer calculation for AV1 decoder DEEPA GUTHYAPPA MADIVALARA
2025-09-02 23:00 ` [PATCH RFC 5/5] media: iris: Define AV1-specific platform capabilities and properties DEEPA GUTHYAPPA MADIVALARA
2025-09-03 13:07 ` [PATCH RFC 0/5] Enable support for AV1 stateful decoder Nicolas Dufresne
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).