[PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[Dikshita Agarwal]

Hi All,

This patch series adds initial support for the HEVC(H.265) and VP9
codecs in iris decoder. The objective of this work is to extend the 
decoder's capabilities to handle HEVC and VP9 codec streams,
including necessary format handling and buffer management.
In addition, the series also includes a set of fixes to address issues
identified during testing of these additional codecs.

These patches also address the comments and feedback received from the 
RFC patches previously sent. I have made the necessary improvements 
based on the community's suggestions.

Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal 
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2: https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com

Changes in v2:
- Added Changes to make sure all buffers are released in session close 
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1: 
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com

Changes sinces RFC:
- Added additional fixes to address issues identified during further 
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil, 
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC: 
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/

This patch series depends on [1] & [2]
[1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
[2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/

These patches are tested on SM8250 and SM8550 with v4l2-ctl and 
Gstreamer for HEVC and VP9 decoders, at the same time ensured that 
the existing H264 decoder functionality remains uneffected.

Note: 1 of the fluster compliance test is fixed with firmware [3]
[3]: 
https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u 

The result of fluster test on SM8550:
 131/147 testcases passed while testing JCT-VC-HEVC_V1 with 
 GStreamer-H.265-V4L2-Gst1.0.
 The failing test case:
 - 10 testcases failed due to unsupported 10 bit format.
   - DBLK_A_MAIN10_VIXS_4
   - INITQP_B_Main10_Sony_1
   - TSUNEQBD_A_MAIN10_Technicolor_2
   - WP_A_MAIN10_Toshiba_3
   - WP_MAIN10_B_Toshiba_3
   - WPP_A_ericsson_MAIN10_2
   - WPP_B_ericsson_MAIN10_2
   - WPP_C_ericsson_MAIN10_2
   - WPP_E_ericsson_MAIN10_2
   - WPP_F_ericsson_MAIN10_2
 - 4 testcase failed due to unsupported resolution
   - PICSIZE_A_Bossen_1
   - PICSIZE_B_Bossen_1
   - WPP_D_ericsson_MAIN10_2
   - WPP_D_ericsson_MAIN_2 
 - 2 testcase failed due to CRC mismatch
   - RAP_A_docomo_6
   - RAP_B_Bossen_2
   - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
     Analysis - First few frames in this discarded by firmware and are 
     sent to driver with 0 filled length. Driver send such buffers to 
     client with timestamp 0 and payload set to 0 and 
     make buf state to VB2_BUF_STATE_ERROR. Such buffers should be 
     dropped by GST. But instead, the first frame displayed as green 
     frame and when a valid buffer is sent to client later with same 0 
     timestamp, its dropped, leading to CRC mismatch for first frame.

 235/305 testcases passed while testing VP9-TEST-VECTORS with 
 GStreamer-VP9-V4L2-Gst1.0.
 The failing test case:
 - 64 testcases failed due to unsupported resolution
   - vp90-2-02-size-08x08.webm
   - vp90-2-02-size-08x10.webm
   - vp90-2-02-size-08x16.webm
   - vp90-2-02-size-08x18.webm
   - vp90-2-02-size-08x32.webm
   - vp90-2-02-size-08x34.webm
   - vp90-2-02-size-08x64.webm
   - vp90-2-02-size-08x66.webm
   - vp90-2-02-size-10x08.webm
   - vp90-2-02-size-10x10.webm
   - vp90-2-02-size-10x16.webm
   - vp90-2-02-size-10x18.webm
   - vp90-2-02-size-10x32.webm
   - vp90-2-02-size-10x34.webm
   - vp90-2-02-size-10x64.webm
   - vp90-2-02-size-10x66.webm
   - vp90-2-02-size-16x08.webm
   - vp90-2-02-size-16x10.webm
   - vp90-2-02-size-16x16.webm
   - vp90-2-02-size-16x18.webm
   - vp90-2-02-size-16x32.webm
   - vp90-2-02-size-16x34.webm
   - vp90-2-02-size-16x64.webm
   - vp90-2-02-size-16x66.webm
   - vp90-2-02-size-18x08.webm
   - vp90-2-02-size-18x10.webm
   - vp90-2-02-size-18x16.webm
   - vp90-2-02-size-18x18.webm
   - vp90-2-02-size-18x32.webm
   - vp90-2-02-size-18x34.webm
   - vp90-2-02-size-18x64.webm
   - vp90-2-02-size-18x66.webm
   - vp90-2-02-size-32x08.webm
   - vp90-2-02-size-32x10.webm
   - vp90-2-02-size-32x16.webm
   - vp90-2-02-size-32x18.webm
   - vp90-2-02-size-32x32.webm
   - vp90-2-02-size-32x34.webm
   - vp90-2-02-size-32x64.webm
   - vp90-2-02-size-32x66.webm
   - vp90-2-02-size-34x08.webm
   - vp90-2-02-size-34x10.webm
   - vp90-2-02-size-34x16.webm
   - vp90-2-02-size-34x18.webm
   - vp90-2-02-size-34x32.webm
   - vp90-2-02-size-34x34.webm
   - vp90-2-02-size-34x64.webm
   - vp90-2-02-size-34x66.webm
   - vp90-2-02-size-64x08.webm
   - vp90-2-02-size-64x10.webm
   - vp90-2-02-size-64x16.webm
   - vp90-2-02-size-64x18.webm
   - vp90-2-02-size-64x32.webm
   - vp90-2-02-size-64x34.webm
   - vp90-2-02-size-64x64.webm
   - vp90-2-02-size-64x66.webm
   - vp90-2-02-size-66x08.webm
   - vp90-2-02-size-66x10.webm
   - vp90-2-02-size-66x16.webm
   - vp90-2-02-size-66x18.webm
   - vp90-2-02-size-66x32.webm
   - vp90-2-02-size-66x34.webm
   - vp90-2-02-size-66x64.webm
   - vp90-2-02-size-66x66.webm
 - 2 testcases failed due to unsupported format
   - vp91-2-04-yuv422.webm
   - vp91-2-04-yuv444.webm
 - 1 testcase failed with CRC mismatch
   - vp90-2-22-svc_1280x720_3.ivf
   - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
 - 2 testcase failed due to unsupported resolution after sequence change
   - vp90-2-21-resize_inter_320x180_5_1-2.webm
   - vp90-2-21-resize_inter_320x180_7_1-2.webm
 - 1 testcase failed due to unsupported stream
   - vp90-2-16-intra-only.webm

The result of fluster test on SM8250:
 133/147 testcases passed while testing JCT-VC-HEVC_V1 with
 GStreamer-H.265-V4L2-Gst1.0.
 The failing test case:
 - 10 testcases failed due to unsupported 10 bit format.
   - DBLK_A_MAIN10_VIXS_4
   - INITQP_B_Main10_Sony_1
   - TSUNEQBD_A_MAIN10_Technicolor_2
   - WP_A_MAIN10_Toshiba_3
   - WP_MAIN10_B_Toshiba_3
   - WPP_A_ericsson_MAIN10_2
   - WPP_B_ericsson_MAIN10_2
   - WPP_C_ericsson_MAIN10_2
   - WPP_E_ericsson_MAIN10_2
   - WPP_F_ericsson_MAIN10_2
 - 4 testcase failed due to unsupported resolution
   - PICSIZE_A_Bossen_1
   - PICSIZE_B_Bossen_1
   - WPP_D_ericsson_MAIN10_2
   - WPP_D_ericsson_MAIN_2

 232/305 testcases passed while testing VP9-TEST-VECTORS with
 GStreamer-VP9-V4L2-Gst1.0.
 The failing test case:
 - 64 testcases failed due to unsupported resolution
   - vp90-2-02-size-08x08.webm
   - vp90-2-02-size-08x10.webm
   - vp90-2-02-size-08x16.webm
   - vp90-2-02-size-08x18.webm
   - vp90-2-02-size-08x32.webm
   - vp90-2-02-size-08x34.webm
   - vp90-2-02-size-08x64.webm
   - vp90-2-02-size-08x66.webm
   - vp90-2-02-size-10x08.webm
   - vp90-2-02-size-10x10.webm
   - vp90-2-02-size-10x16.webm
   - vp90-2-02-size-10x18.webm
   - vp90-2-02-size-10x32.webm
   - vp90-2-02-size-10x34.webm
   - vp90-2-02-size-10x64.webm
   - vp90-2-02-size-10x66.webm
   - vp90-2-02-size-16x08.webm
   - vp90-2-02-size-16x10.webm
   - vp90-2-02-size-16x16.webm
   - vp90-2-02-size-16x18.webm
   - vp90-2-02-size-16x32.webm
   - vp90-2-02-size-16x34.webm
   - vp90-2-02-size-16x64.webm
   - vp90-2-02-size-16x66.webm
   - vp90-2-02-size-18x08.webm
   - vp90-2-02-size-18x10.webm
   - vp90-2-02-size-18x16.webm
   - vp90-2-02-size-18x18.webm
   - vp90-2-02-size-18x32.webm
   - vp90-2-02-size-18x34.webm
   - vp90-2-02-size-18x64.webm
   - vp90-2-02-size-18x66.webm
   - vp90-2-02-size-32x08.webm
   - vp90-2-02-size-32x10.webm
   - vp90-2-02-size-32x16.webm
   - vp90-2-02-size-32x18.webm
   - vp90-2-02-size-32x32.webm
   - vp90-2-02-size-32x34.webm
   - vp90-2-02-size-32x64.webm
   - vp90-2-02-size-32x66.webm
   - vp90-2-02-size-34x08.webm
   - vp90-2-02-size-34x10.webm
   - vp90-2-02-size-34x16.webm
   - vp90-2-02-size-34x18.webm
   - vp90-2-02-size-34x32.webm
   - vp90-2-02-size-34x34.webm
   - vp90-2-02-size-34x64.webm
   - vp90-2-02-size-34x66.webm
   - vp90-2-02-size-64x08.webm
   - vp90-2-02-size-64x10.webm
   - vp90-2-02-size-64x16.webm
   - vp90-2-02-size-64x18.webm
   - vp90-2-02-size-64x32.webm
   - vp90-2-02-size-64x34.webm
   - vp90-2-02-size-64x64.webm
   - vp90-2-02-size-64x66.webm
   - vp90-2-02-size-66x08.webm
   - vp90-2-02-size-66x10.webm
   - vp90-2-02-size-66x16.webm
   - vp90-2-02-size-66x18.webm
   - vp90-2-02-size-66x32.webm
   - vp90-2-02-size-66x34.webm
   - vp90-2-02-size-66x64.webm
   - vp90-2-02-size-66x66.webm
 - 2 testcases failed due to unsupported format
   - vp91-2-04-yuv422.webm
   - vp91-2-04-yuv444.webm
 - 1 testcase failed with CRC mismatch
   - vp90-2-22-svc_1280x720_3.ivf
   - Bug raised: 
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
 - 5 testcase failed due to unsupported resolution after sequence change
   - vp90-2-21-resize_inter_320x180_5_1-2.webm
   - vp90-2-21-resize_inter_320x180_7_1-2.webm
   - vp90-2-21-resize_inter_320x240_5_1-2.webm
   - vp90-2-21-resize_inter_320x240_7_1-2.webm
   - vp90-2-18-resize.ivf
 - 1 testcase failed with CRC mismatch
   - vp90-2-16-intra-only.webm
   Analysis: First few frames are marked by firmware as NO_SHOW frame.
   Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
   Such buffers should be dropped by GST. But instead, the first frame 
   is being displayed and when a valid buffer is sent to client later
   with same timestamp, its dropped, leading to CRC mismatch for first 
   frame.

Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
Dikshita Agarwal (23):
      media: iris: Skip destroying internal buffer if not dequeued
      media: iris: Update CAPTURE format info based on OUTPUT format
      media: iris: Avoid updating frame size to firmware during reconfig
      media: iris: Drop port check for session property response
      media: iris: Prevent HFI queue writes when core is in deinit state
      media: iris: Remove deprecated property setting to firmware
      media: iris: Fix missing function pointer initialization
      media: iris: Fix NULL pointer dereference
      media: iris: Fix typo in depth variable
      media: iris: Track flush responses to prevent premature completion
      media: iris: Fix buffer preparation failure during resolution change
      media: iris: Add handling for corrupt and drop frames
      media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length
      media: iris: Add handling for no show frames
      media: iris: Improve last flag handling
      media: iris: Skip flush on first sequence change
      media: iris: Remove redundant buffer count check in stream off
      media: iris: Add a comment to explain usage of MBPS
      media: iris: Add HEVC and VP9 formats for decoder
      media: iris: Add platform capabilities for HEVC and VP9 decoders
      media: iris: Set mandatory properties for HEVC and VP9 decoders.
      media: iris: Add internal buffer calculation for HEVC and VP9 decoders
      media: iris: Add codec specific check for VP9 decoder drain handling

 drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
 drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
 drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
 drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
 .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
 .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
 .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
 .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
 .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
 drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
 drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
 .../platform/qcom/iris/iris_platform_common.h      |  28 +-
 .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
 .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
 .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
 drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
 drivers/media/platform/qcom/iris/iris_state.h      |   1 +
 drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
 drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
 drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
 drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
 drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++-
 drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
 24 files changed, 1160 insertions(+), 211 deletions(-)
---
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
prerequisite-message-id: <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
prerequisite-message-id: <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea

Best regards,
-- 
Dikshita Agarwal <quic_dikshita@quicinc.com>

[PATCH v3 01/23] media: iris: Skip destroying internal buffer if not dequeued

[Dikshita Agarwal]

Firmware might hold the DPB buffers for reference in case of sequence
change, so skip destroying buffers for which QUEUED flag is not removed.
Also, make sure that all buffers are released during streamoff.

Cc: stable@vger.kernel.org
Fixes: 73702f45db81 ("media: iris: allocate, initialize and queue internal buffers")
Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_buffer.c | 20 +++++++++++++++-
 drivers/media/platform/qcom/iris/iris_buffer.h |  3 ++-
 drivers/media/platform/qcom/iris/iris_vdec.c   |  4 ++--
 drivers/media/platform/qcom/iris/iris_vidc.c   | 33 ++++++++++++++++++++++++--
 4 files changed, 54 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c
index e5c5a564fcb8..981fedb000ed 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_buffer.c
@@ -376,7 +376,7 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf
 	return 0;
 }
 
-int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane)
+static int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane, bool force)
 {
 	const struct iris_platform_data *platform_data = inst->core->iris_platform_data;
 	struct iris_buffer *buf, *next;
@@ -396,6 +396,14 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane)
 	for (i = 0; i < len; i++) {
 		buffers = &inst->buffers[internal_buf_type[i]];
 		list_for_each_entry_safe(buf, next, &buffers->list, list) {
+			/*
+			 * during stream on, skip destroying internal(DPB) buffer
+			 * if firmware did not return it.
+			 * during close, destroy all buffers irrespectively.
+			 */
+			if (!force && buf->attr & BUF_ATTR_QUEUED)
+				continue;
+
 			ret = iris_destroy_internal_buffer(inst, buf);
 			if (ret)
 				return ret;
@@ -405,6 +413,16 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane)
 	return 0;
 }
 
+int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane)
+{
+	return iris_destroy_internal_buffers(inst, plane, true);
+}
+
+int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane)
+{
+	return iris_destroy_internal_buffers(inst, plane, false);
+}
+
 static int iris_release_internal_buffers(struct iris_inst *inst,
 					 enum iris_buffer_type buffer_type)
 {
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
index c36b6347b077..00825ad2dc3a 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_buffer.h
@@ -106,7 +106,8 @@ void iris_get_internal_buffers(struct iris_inst *inst, u32 plane);
 int iris_create_internal_buffers(struct iris_inst *inst, u32 plane);
 int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane);
 int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer);
-int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane);
+int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane);
+int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane);
 int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst);
 int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst);
 int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf);
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index 4143acedfc57..9c049b9671cc 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -408,7 +408,7 @@ int iris_vdec_streamon_input(struct iris_inst *inst)
 
 	iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
 
-	ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+	ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
 	if (ret)
 		return ret;
 
@@ -496,7 +496,7 @@ int iris_vdec_streamon_output(struct iris_inst *inst)
 
 	iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
 
-	ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
index ca0f4e310f77..a8144595cc78 100644
--- a/drivers/media/platform/qcom/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/iris/iris_vidc.c
@@ -221,6 +221,33 @@ static void iris_session_close(struct iris_inst *inst)
 		iris_wait_for_session_response(inst, false);
 }
 
+static void iris_check_num_queued_internal_buffers(struct iris_inst *inst, u32 plane)
+{
+	const struct iris_platform_data *platform_data = inst->core->iris_platform_data;
+	struct iris_buffer *buf, *next;
+	struct iris_buffers *buffers;
+	const u32 *internal_buf_type;
+	u32 internal_buffer_count, i;
+	u32 count = 0;
+
+	if (V4L2_TYPE_IS_OUTPUT(plane)) {
+		internal_buf_type = platform_data->dec_ip_int_buf_tbl;
+		internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size;
+	} else {
+		internal_buf_type = platform_data->dec_op_int_buf_tbl;
+		internal_buffer_count = platform_data->dec_op_int_buf_tbl_size;
+	}
+
+	for (i = 0; i < internal_buffer_count; i++) {
+		buffers = &inst->buffers[internal_buf_type[i]];
+		list_for_each_entry_safe(buf, next, &buffers->list, list)
+			count++;
+		if (count)
+			dev_err(inst->core->dev, "%d buffer of type %d not released",
+				count, internal_buf_type[i]);
+	}
+}
+
 int iris_close(struct file *filp)
 {
 	struct iris_inst *inst = iris_get_inst(filp, NULL);
@@ -233,8 +260,10 @@ int iris_close(struct file *filp)
 	iris_session_close(inst);
 	iris_inst_change_state(inst, IRIS_INST_DEINIT);
 	iris_v4l2_fh_deinit(inst);
-	iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
-	iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+	iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+	iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
 	iris_remove_session(inst);
 	mutex_unlock(&inst->lock);
 	mutex_destroy(&inst->ctx_q_lock);

-- 
2.34.1

[PATCH v3 02/23] media: iris: Update CAPTURE format info based on OUTPUT format

[Dikshita Agarwal]

Update the width, height and buffer size of CAPTURE based on the
resolution set to OUTPUT via VIDIOC_S_FMT. This is required to set the
updated capture resolution to firmware when S_FMT is called only for
OUTPUT.

Cc: stable@vger.kernel.org
Fixes: b530b95de22c ("media: iris: implement s_fmt, g_fmt and try_fmt ioctls")
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_vdec.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index 9c049b9671cc..d342f733feb9 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -171,6 +171,11 @@ int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
 		output_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
 		output_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
 
+		/* Update capture format based on new ip w/h */
+		output_fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, 128);
+		output_fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, 32);
+		inst->buffers[BUF_OUTPUT].size = iris_get_buffer_size(inst, BUF_OUTPUT);
+
 		inst->crop.left = 0;
 		inst->crop.top = 0;
 		inst->crop.width = f->fmt.pix_mp.width;

-- 
2.34.1

[PATCH v3 03/23] media: iris: Avoid updating frame size to firmware during reconfig

[Dikshita Agarwal]

During reconfig, the firmware sends the resolution aligned to 8 bytes.
If the driver sends the same resolution back to the firmware the resolution
will be aligned to 16 bytes not 8.

The alignment mismatch would then subsequently cause the firmware to
send another redundant sequence change event.

Fix this by not setting the resolution property during reconfig.

Cc: stable@vger.kernel.org
Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon")
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c | 15 ++++++++-------
 drivers/media/platform/qcom/iris/iris_state.c            |  2 +-
 drivers/media/platform/qcom/iris/iris_state.h            |  1 +
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
index 64f887d9a17d..2a86c27443ea 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
@@ -546,14 +546,15 @@ static int iris_hfi_gen1_set_resolution(struct iris_inst *inst)
 	struct hfi_framesize fs;
 	int ret;
 
-	fs.buffer_type = HFI_BUFFER_INPUT;
-	fs.width = inst->fmt_src->fmt.pix_mp.width;
-	fs.height = inst->fmt_src->fmt.pix_mp.height;
-
-	ret = hfi_gen1_set_property(inst, ptype, &fs, sizeof(fs));
-	if (ret)
-		return ret;
+	if (!iris_drc_pending(inst)) {
+		fs.buffer_type = HFI_BUFFER_INPUT;
+		fs.width = inst->fmt_src->fmt.pix_mp.width;
+		fs.height = inst->fmt_src->fmt.pix_mp.height;
 
+		ret = hfi_gen1_set_property(inst, ptype, &fs, sizeof(fs));
+		if (ret)
+			return ret;
+	}
 	fs.buffer_type = HFI_BUFFER_OUTPUT2;
 	fs.width = inst->fmt_dst->fmt.pix_mp.width;
 	fs.height = inst->fmt_dst->fmt.pix_mp.height;
diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c
index 5976e926c83d..104e1687ad39 100644
--- a/drivers/media/platform/qcom/iris/iris_state.c
+++ b/drivers/media/platform/qcom/iris/iris_state.c
@@ -245,7 +245,7 @@ int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane)
 	return iris_inst_change_sub_state(inst, 0, set_sub_state);
 }
 
-static inline bool iris_drc_pending(struct iris_inst *inst)
+bool iris_drc_pending(struct iris_inst *inst)
 {
 	return inst->sub_state & IRIS_INST_SUB_DRC &&
 		inst->sub_state & IRIS_INST_SUB_DRC_LAST;
diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h
index 78c61aac5e7e..e718386dbe04 100644
--- a/drivers/media/platform/qcom/iris/iris_state.h
+++ b/drivers/media/platform/qcom/iris/iris_state.h
@@ -140,5 +140,6 @@ int iris_inst_sub_state_change_drain_last(struct iris_inst *inst);
 int iris_inst_sub_state_change_drc_last(struct iris_inst *inst);
 int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane);
 bool iris_allow_cmd(struct iris_inst *inst, u32 cmd);
+bool iris_drc_pending(struct iris_inst *inst);
 
 #endif

-- 
2.34.1

[PATCH v3 04/23] media: iris: Drop port check for session property response

[Dikshita Agarwal]

Currently, port check enforces that session property response must
arrive only on the BITSTREAM port. However, firmware can send some
responses on other port as well.

Remove the strict port validation to correctly handle session property
responses from the firmware.

Cc: stable@vger.kernel.org
Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon")
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index b75a01641d5d..d1a2a497a7b2 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -636,9 +636,6 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
 {
 	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
 
-	if (pkt->port != HFI_PORT_BITSTREAM)
-		return 0;
-
 	if (pkt->flags & HFI_FW_FLAGS_INFORMATION)
 		return 0;
 

-- 
2.34.1

[PATCH v3 05/23] media: iris: Prevent HFI queue writes when core is in deinit state

[Dikshita Agarwal]

The current check only considers the core error state before allowing
writes to the HFI queues. However, the core can also transition to the
deinit state due to a system error triggered by the response thread.
In such cases, writing to the HFI queues should not be allowed.

Fix this by adding a check for the core deinit state, ensuring that
writes are rejected when core is not in a valid state.

Cc: stable@vger.kernel.org
Fixes: fb583a214337 ("media: iris: introduce host firmware interface with necessary hooks")
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_queue.c b/drivers/media/platform/qcom/iris/iris_hfi_queue.c
index fac7df0c4d1a..221dcd09e1e1 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_queue.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_queue.c
@@ -113,7 +113,7 @@ int iris_hfi_queue_cmd_write_locked(struct iris_core *core, void *pkt, u32 pkt_s
 {
 	struct iris_iface_q_info *q_info = &core->command_queue;
 
-	if (core->state == IRIS_CORE_ERROR)
+	if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT)
 		return -EINVAL;
 
 	if (!iris_hfi_queue_write(q_info, pkt, pkt_size)) {

-- 
2.34.1

[PATCH v3 06/23] media: iris: Remove deprecated property setting to firmware

[Dikshita Agarwal]

HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER is deprecated and no longer
supported on current firmware, remove setting the same to firmware.

At the same time, remove the check for non-zero number of v4l2 controls
as some SOC might not expose any capability which requires v4l2 control.

Cc: stable@vger.kernel.org
Fixes: 79865252acb6 ("media: iris: enable video driver probe of SM8250 SoC")
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_ctrls.c            | 6 ------
 drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c | 8 --------
 drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h | 1 -
 drivers/media/platform/qcom/iris/iris_platform_common.h  | 2 +-
 drivers/media/platform/qcom/iris/iris_platform_sm8250.c  | 9 ---------
 5 files changed, 1 insertion(+), 25 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
index b690578256d5..915de101fcba 100644
--- a/drivers/media/platform/qcom/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
@@ -17,8 +17,6 @@ static inline bool iris_valid_cap_id(enum platform_inst_fw_cap_type cap_id)
 static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
 {
 	switch (id) {
-	case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
-		return DEBLOCK;
 	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
 		return PROFILE;
 	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
@@ -34,8 +32,6 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
 		return 0;
 
 	switch (cap_id) {
-	case DEBLOCK:
-		return V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER;
 	case PROFILE:
 		return V4L2_CID_MPEG_VIDEO_H264_PROFILE;
 	case LEVEL:
@@ -84,8 +80,6 @@ int iris_ctrls_init(struct iris_inst *inst)
 		if (iris_get_v4l2_id(cap[idx].cap_id))
 			num_ctrls++;
 	}
-	if (!num_ctrls)
-		return -EINVAL;
 
 	/* Adding 1 to num_ctrls to include V4L2_CID_MIN_BUFFERS_FOR_CAPTURE */
 
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
index 2a86c27443ea..ce855a20ce4b 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
@@ -490,14 +490,6 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p
 		packet->shdr.hdr.size += sizeof(u32) + sizeof(*wm);
 		break;
 	}
-	case HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER: {
-		struct hfi_enable *en = prop_data;
-		u32 *in = pdata;
-
-		en->enable = *in;
-		packet->shdr.hdr.size += sizeof(u32) + sizeof(*en);
-		break;
-	}
 	default:
 		return -EINVAL;
 	}
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
index 9f246816a286..e178604855c1 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
@@ -65,7 +65,6 @@
 
 #define HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS		0x202001
 
-#define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER	0x1200001
 #define HFI_PROPERTY_PARAM_VDEC_DPB_COUNTS		0x120300e
 #define HFI_PROPERTY_CONFIG_VDEC_ENTROPY		0x1204004
 
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index ac76d9e1ef9c..1dab276431c7 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -89,7 +89,7 @@ enum platform_inst_fw_cap_type {
 	CODED_FRAMES,
 	BIT_DEPTH,
 	RAP_FRAME,
-	DEBLOCK,
+	TIER,
 	INST_FW_CAP_MAX,
 };
 
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
index 5c86fd7b7b6f..543fa2661539 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
@@ -30,15 +30,6 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8250[] = {
 		.hfi_id = HFI_PROPERTY_PARAM_WORK_MODE,
 		.set = iris_set_stage,
 	},
-	{
-		.cap_id = DEBLOCK,
-		.min = 0,
-		.max = 1,
-		.step_or_mask = 1,
-		.value = 0,
-		.hfi_id = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER,
-		.set = iris_set_u32,
-	},
 };
 
 static struct platform_inst_caps platform_inst_cap_sm8250 = {

-- 
2.34.1

[PATCH v3 07/23] media: iris: Fix missing function pointer initialization

[Dikshita Agarwal]

The function pointers responsible for setting firmware properties were
never initialized in the instance capability structure, causing it to
remain NULL. As a result, the firmware properties were not being set
correctly.

Fix this by properly assigning the function pointers from the core
capability to the instance capability, ensuring that the properties are
correctly applied to the firmware.

Cc: stable@vger.kernel.org
Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon")
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_ctrls.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
index 915de101fcba..13f5cf0d0e8a 100644
--- a/drivers/media/platform/qcom/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
@@ -157,6 +157,7 @@ void iris_session_init_caps(struct iris_core *core)
 		core->inst_fw_caps[cap_id].value = caps[i].value;
 		core->inst_fw_caps[cap_id].flags = caps[i].flags;
 		core->inst_fw_caps[cap_id].hfi_id = caps[i].hfi_id;
+		core->inst_fw_caps[cap_id].set = caps[i].set;
 	}
 }
 

-- 
2.34.1

[PATCH v3 08/23] media: iris: Fix NULL pointer dereference

[Dikshita Agarwal]

A warning reported by smatch indicated a possible null pointer
dereference where one of the arguments to API
"iris_hfi_gen2_handle_system_error" could sometimes be null.

To fix this, add a check to validate that the argument passed is not
null before accessing its members.

Cc: stable@vger.kernel.org
Fixes: fb583a214337 ("media: iris: introduce host firmware interface with necessary hooks")
Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
Closes: https://lore.kernel.org/linux-media/634cc9b8-f099-4b54-8556-d879fb2b5169@stanley.mountain/
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index d1a2a497a7b2..4488540d1d41 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -265,7 +265,8 @@ static int iris_hfi_gen2_handle_system_error(struct iris_core *core,
 {
 	struct iris_inst *instance;
 
-	dev_err(core->dev, "received system error of type %#x\n", pkt->type);
+	if (pkt)
+		dev_err(core->dev, "received system error of type %#x\n", pkt->type);
 
 	core->state = IRIS_CORE_ERROR;
 

-- 
2.34.1

[PATCH v3 09/23] media: iris: Fix typo in depth variable

[Dikshita Agarwal]

Correct a typo from "dpeth" to "depth".

Cc: stable@vger.kernel.org
Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon")
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
index a908b41e2868..802fa62c26eb 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -178,7 +178,7 @@ static int iris_hfi_gen2_set_crop_offsets(struct iris_inst *inst)
 						  sizeof(u64));
 }
 
-static int iris_hfi_gen2_set_bit_dpeth(struct iris_inst *inst)
+static int iris_hfi_gen2_set_bit_depth(struct iris_inst *inst)
 {
 	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
 	u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
@@ -378,7 +378,7 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
 		{HFI_PROP_BITSTREAM_RESOLUTION,       iris_hfi_gen2_set_bitstream_resolution   },
 		{HFI_PROP_CROP_OFFSETS,               iris_hfi_gen2_set_crop_offsets           },
 		{HFI_PROP_CODED_FRAMES,               iris_hfi_gen2_set_coded_frames           },
-		{HFI_PROP_LUMA_CHROMA_BIT_DEPTH,      iris_hfi_gen2_set_bit_dpeth              },
+		{HFI_PROP_LUMA_CHROMA_BIT_DEPTH,      iris_hfi_gen2_set_bit_depth              },
 		{HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, iris_hfi_gen2_set_min_output_count       },
 		{HFI_PROP_PIC_ORDER_CNT_TYPE,         iris_hfi_gen2_set_picture_order_count    },
 		{HFI_PROP_SIGNAL_COLOR_INFO,          iris_hfi_gen2_set_colorspace             },

-- 
2.34.1

[PATCH v3 10/23] media: iris: Track flush responses to prevent premature completion

[Dikshita Agarwal]

Currently, two types of flush commands are queued to the firmware,
the first flush queued as part of sequence change, does not wait for a
response, while the second flush queued as part of stop, expects a
completion response before proceeding further.

Due to timing issue, the flush response corresponding to the first
command could arrive after the second flush is issued. This casuses the
driver to incorrectly assume that the second flush has completed,
leading to the premature signaling of flush_completion.

To address this, introduce a counter to track the number of pending
flush responses and signal flush completion only when all expected
responses are received.

Cc: stable@vger.kernel.org
Fixes: 11712ce70f8e ("media: iris: implement vb2 streaming ops")
Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 .../media/platform/qcom/iris/iris_hfi_gen1_command.c    |  4 +++-
 .../media/platform/qcom/iris/iris_hfi_gen1_response.c   | 17 +++++++++++------
 drivers/media/platform/qcom/iris/iris_instance.h        |  2 ++
 3 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
index ce855a20ce4b..bd9d86220e61 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
@@ -208,8 +208,10 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane)
 		flush_pkt.flush_type = flush_type;
 
 		ret = iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size);
-		if (!ret)
+		if (!ret) {
+			inst->flush_responses_pending++;
 			ret = iris_wait_for_session_response(inst, true);
+		}
 	}
 
 	return ret;
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
index b72d503dd740..271e14469223 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
@@ -207,7 +207,8 @@ static void iris_hfi_gen1_event_seq_changed(struct iris_inst *inst,
 		flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH;
 		flush_pkt.shdr.session_id = inst->session_id;
 		flush_pkt.flush_type = HFI_FLUSH_OUTPUT;
-		iris_hfi_queue_cmd_write(inst->core, &flush_pkt, flush_pkt.shdr.hdr.size);
+		if (!iris_hfi_queue_cmd_write(inst->core, &flush_pkt, flush_pkt.shdr.hdr.size))
+			inst->flush_responses_pending++;
 	}
 
 	iris_vdec_src_change(inst);
@@ -408,7 +409,9 @@ static void iris_hfi_gen1_session_ftb_done(struct iris_inst *inst, void *packet)
 		flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH;
 		flush_pkt.shdr.session_id = inst->session_id;
 		flush_pkt.flush_type = HFI_FLUSH_OUTPUT;
-		iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size);
+		if (!iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size))
+			inst->flush_responses_pending++;
+
 		iris_inst_sub_state_change_drain_last(inst);
 
 		return;
@@ -558,7 +561,6 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response
 	const struct iris_hfi_gen1_response_pkt_info *pkt_info;
 	struct device *dev = core->dev;
 	struct hfi_session_pkt *pkt;
-	struct completion *done;
 	struct iris_inst *inst;
 	bool found = false;
 	u32 i;
@@ -619,9 +621,12 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response
 			if (shdr->error_type != HFI_ERR_NONE)
 				iris_inst_change_state(inst, IRIS_INST_ERROR);
 
-			done = pkt_info->pkt == HFI_MSG_SESSION_FLUSH ?
-				&inst->flush_completion : &inst->completion;
-			complete(done);
+			if (pkt_info->pkt == HFI_MSG_SESSION_FLUSH) {
+				if (!(--inst->flush_responses_pending))
+					complete(&inst->flush_completion);
+			} else {
+				complete(&inst->completion);
+			}
 		}
 		mutex_unlock(&inst->lock);
 
diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
index caa3c6507006..06a7f1174ad5 100644
--- a/drivers/media/platform/qcom/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/iris/iris_instance.h
@@ -27,6 +27,7 @@
  * @crop: structure of crop info
  * @completion: structure of signal completions
  * @flush_completion: structure of signal completions for flush cmd
+ * @flush_responses_pending: counter to track number of pending flush responses
  * @fw_caps: array of supported instance firmware capabilities
  * @buffers: array of different iris buffers
  * @fw_min_count: minimnum count of buffers needed by fw
@@ -57,6 +58,7 @@ struct iris_inst {
 	struct iris_hfi_rect_desc	crop;
 	struct completion		completion;
 	struct completion		flush_completion;
+	u32				flush_responses_pending;
 	struct platform_inst_fw_cap	fw_caps[INST_FW_CAP_MAX];
 	struct iris_buffers		buffers[BUF_TYPE_MAX];
 	u32				fw_min_count;

-- 
2.34.1

[PATCH v3 11/23] media: iris: Fix buffer preparation failure during resolution change

[Dikshita Agarwal]

When the resolution changes, the driver internally updates the width and
height, but the client continue to queue buffers with the older
resolution until the last flag is received. This results in a mismatch
when the buffers are prepared, causing failure due to outdated size.

Introduce a check to prevent size validation during buffer preparation
if a resolution reconfiguration is in progress, to handle this.

Cc: stable@vger.kernel.org
Fixes: 17f2a485ca67 ("media: iris: implement vb2 ops for buf_queue and firmware response")
Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_vb2.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c
index cdf11feb590b..b3bde10eb6d2 100644
--- a/drivers/media/platform/qcom/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/iris/iris_vb2.c
@@ -259,13 +259,14 @@ int iris_vb2_buf_prepare(struct vb2_buffer *vb)
 			return -EINVAL;
 	}
 
-	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
-	    vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_OUTPUT))
-		return -EINVAL;
-	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
-	    vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_INPUT))
-		return -EINVAL;
-
+	if (!(inst->sub_state & IRIS_INST_SUB_DRC)) {
+		if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+		    vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_OUTPUT))
+			return -EINVAL;
+		if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+		    vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_INPUT))
+			return -EINVAL;
+	}
 	return 0;
 }
 

-- 
2.34.1

[PATCH v3 12/23] media: iris: Add handling for corrupt and drop frames

[Dikshita Agarwal]

Firmware attach DATACORRUPT/DROP buffer flags for the frames which
needs to be dropped, handle it by setting VB2_BUF_STATE_ERROR for these
buffers before calling buf_done.

Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_buffer.c            | 11 ++++++++---
 drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h  |  2 ++
 drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c |  6 ++++++
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c
index 981fedb000ed..018334512bae 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_buffer.c
@@ -611,10 +611,13 @@ int iris_vb2_buffer_done(struct iris_inst *inst, struct iris_buffer *buf)
 
 	vb2 = &vbuf->vb2_buf;
 
-	if (buf->flags & V4L2_BUF_FLAG_ERROR)
+	if (buf->flags & V4L2_BUF_FLAG_ERROR) {
 		state = VB2_BUF_STATE_ERROR;
-	else
-		state = VB2_BUF_STATE_DONE;
+		vb2_set_plane_payload(vb2, 0, 0);
+		vb2->timestamp = 0;
+		v4l2_m2m_buf_done(vbuf, state);
+		return 0;
+	}
 
 	vbuf->flags |= buf->flags;
 
@@ -634,6 +637,8 @@ int iris_vb2_buffer_done(struct iris_inst *inst, struct iris_buffer *buf)
 			v4l2_m2m_mark_stopped(m2m_ctx);
 		}
 	}
+
+	state = VB2_BUF_STATE_DONE;
 	vb2->timestamp = buf->timestamp;
 	v4l2_m2m_buf_done(vbuf, state);
 
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
index e178604855c1..adffcead58ea 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
@@ -116,6 +116,8 @@
 #define HFI_FRAME_NOTCODED				0x7f002000
 #define HFI_FRAME_YUV					0x7f004000
 #define HFI_UNUSED_PICT					0x10000000
+#define HFI_BUFFERFLAG_DATACORRUPT			0x00000008
+#define HFI_BUFFERFLAG_DROP_FRAME			0x20000000
 
 struct hfi_pkt_hdr {
 	u32 size;
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
index 271e14469223..d36e29c779db 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
@@ -484,6 +484,12 @@ static void iris_hfi_gen1_session_ftb_done(struct iris_inst *inst, void *packet)
 	buf->attr |= BUF_ATTR_DEQUEUED;
 	buf->attr |= BUF_ATTR_BUFFER_DONE;
 
+	if (hfi_flags & HFI_BUFFERFLAG_DATACORRUPT)
+		flags |= V4L2_BUF_FLAG_ERROR;
+
+	if (hfi_flags & HFI_BUFFERFLAG_DROP_FRAME)
+		flags |= V4L2_BUF_FLAG_ERROR;
+
 	buf->flags |= flags;
 
 	iris_vb2_buffer_done(inst, buf);

-- 
2.34.1

[PATCH v3 13/23] media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length

[Dikshita Agarwal]

Firmware sends buffers with 0 filled length which needs to be dropped,
to achieve the same, add V4L2_BUF_FLAG_ERROR to such buffers.
Also make sure:
- These 0 length buffers are not returned as result of flush.
- Its not a buffer with LAST flag enabled which will also have 0 filled
  length.

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 4488540d1d41..3bb326843a7b 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -378,6 +378,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst,
 
 	buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, hfi_buffer->flags);
 
+	if (!buf->data_size && inst->state == IRIS_INST_STREAMING &&
+	    !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) &&
+	    !(inst->sub_state & IRIS_INST_SUB_DRC)) {
+		buf->flags |= V4L2_BUF_FLAG_ERROR;
+	}
+
 	return 0;
 }
 

-- 
2.34.1

[PATCH v3 14/23] media: iris: Add handling for no show frames

[Dikshita Agarwal]

Firmware sends the picture type as NO_SHOW for frames which are not
supposed to be displayed, add handling for the same in driver to drop
them.

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h  | 1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index 806f8bb7f505..666061a612c3 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -113,6 +113,7 @@ enum hfi_picture_type {
 	HFI_PICTURE_I				= 0x00000008,
 	HFI_PICTURE_CRA				= 0x00000010,
 	HFI_PICTURE_BLA				= 0x00000020,
+	HFI_PICTURE_NOSHOW			= 0x00000040,
 };
 
 enum hfi_buffer_type {
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 3bb326843a7b..2267e220c9ea 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -91,7 +91,9 @@ static int iris_hfi_gen2_get_driver_buffer_flags(struct iris_inst *inst, u32 hfi
 	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
 	u32 driver_flags = 0;
 
-	if (inst_hfi_gen2->hfi_frame_info.picture_type & keyframe)
+	if (inst_hfi_gen2->hfi_frame_info.picture_type & HFI_PICTURE_NOSHOW)
+		driver_flags |= V4L2_BUF_FLAG_ERROR;
+	else if (inst_hfi_gen2->hfi_frame_info.picture_type & keyframe)
 		driver_flags |= V4L2_BUF_FLAG_KEYFRAME;
 	else if (inst_hfi_gen2->hfi_frame_info.picture_type & HFI_PICTURE_P)
 		driver_flags |= V4L2_BUF_FLAG_PFRAME;

-- 
2.34.1

[PATCH v3 15/23] media: iris: Improve last flag handling

[Dikshita Agarwal]

Improve the handling of the V4L2_BUF_FLAG_LAST flag in the driver:
- Ensure that the last flag is not sent multiple times.
- Attach the last flag to the first capture buffer returned during
  flush, triggered by a sequence change, addressing cases where the
  firmware does not set the last flag.

Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_buffer.c            | 1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c | 7 ++++++-
 drivers/media/platform/qcom/iris/iris_instance.h          | 2 ++
 drivers/media/platform/qcom/iris/iris_vb2.c               | 3 ++-
 drivers/media/platform/qcom/iris/iris_vdec.c              | 2 ++
 5 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c
index 018334512bae..7dbac74b1a8d 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_buffer.c
@@ -636,6 +636,7 @@ int iris_vb2_buffer_done(struct iris_inst *inst, struct iris_buffer *buf)
 			v4l2_event_queue_fh(&inst->fh, &ev);
 			v4l2_m2m_mark_stopped(m2m_ctx);
 		}
+		inst->last_buffer_dequeued = true;
 	}
 
 	state = VB2_BUF_STATE_DONE;
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
index d36e29c779db..19ced1b3ee74 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
@@ -458,7 +458,12 @@ static void iris_hfi_gen1_session_ftb_done(struct iris_inst *inst, void *packet)
 		timestamp_us = timestamp_hi;
 		timestamp_us = (timestamp_us << 32) | timestamp_lo;
 	} else {
-		flags |= V4L2_BUF_FLAG_LAST;
+		if (pkt->stream_id == 1 && !inst->last_buffer_dequeued) {
+			if (iris_drc_pending(inst)) {
+				flags |= V4L2_BUF_FLAG_LAST;
+				inst->last_buffer_dequeued = true;
+			}
+		}
 	}
 	buf->timestamp = timestamp_us;
 
diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
index 06a7f1174ad5..5ec6368b2af7 100644
--- a/drivers/media/platform/qcom/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/iris/iris_instance.h
@@ -43,6 +43,7 @@
  * @sequence_out: a sequence counter for output queue
  * @tss: timestamp metadata
  * @metadata_idx: index for metadata buffer
+ * @last_buffer_dequeued: a flag to indicate that last buffer is sent by driver
  */
 
 struct iris_inst {
@@ -74,6 +75,7 @@ struct iris_inst {
 	u32				sequence_out;
 	struct iris_ts_metadata		tss[VIDEO_MAX_FRAME];
 	u32				metadata_idx;
+	bool				last_buffer_dequeued;
 };
 
 #endif
diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c
index b3bde10eb6d2..8b17c7c39487 100644
--- a/drivers/media/platform/qcom/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/iris/iris_vb2.c
@@ -305,7 +305,7 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2)
 		goto exit;
 	}
 
-	if (V4L2_TYPE_IS_CAPTURE(vb2->vb2_queue->type)) {
+	if (!inst->last_buffer_dequeued && V4L2_TYPE_IS_CAPTURE(vb2->vb2_queue->type)) {
 		if ((inst->sub_state & IRIS_INST_SUB_DRC &&
 		     inst->sub_state & IRIS_INST_SUB_DRC_LAST) ||
 		    (inst->sub_state & IRIS_INST_SUB_DRAIN &&
@@ -319,6 +319,7 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2)
 				v4l2_event_queue_fh(&inst->fh, &eos);
 				v4l2_m2m_mark_stopped(m2m_ctx);
 			}
+			inst->last_buffer_dequeued = true;
 			goto exit;
 		}
 	}
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index d342f733feb9..de4e3fe8ed5a 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -487,6 +487,8 @@ static int iris_vdec_process_streamon_output(struct iris_inst *inst)
 	if (ret)
 		return ret;
 
+	inst->last_buffer_dequeued = false;
+
 	return iris_inst_change_sub_state(inst, clear_sub_state, 0);
 }
 

-- 
2.34.1

[PATCH v3 16/23] media: iris: Skip flush on first sequence change

[Dikshita Agarwal]

Add a condition to skip the flush operation during the first sequence
change event. At this point, the capture queue is not streaming, making
the flush unnecessary.

Additionally, remove the reinit_completion call for the flush completion
signal, as it is not needed. This simplifies the code and avoids
unnecessary reinitialization.

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
index 19ced1b3ee74..926acee1f48c 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
@@ -200,8 +200,7 @@ static void iris_hfi_gen1_event_seq_changed(struct iris_inst *inst,
 
 	iris_hfi_gen1_read_changed_params(inst, pkt);
 
-	if (inst->state != IRIS_INST_ERROR) {
-		reinit_completion(&inst->flush_completion);
+	if (inst->state != IRIS_INST_ERROR && !(inst->sub_state & IRIS_INST_SUB_FIRST_IPSC)) {
 
 		flush_pkt.shdr.hdr.size = sizeof(struct hfi_session_flush_pkt);
 		flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH;

-- 
2.34.1

[PATCH v3 17/23] media: iris: Remove redundant buffer count check in stream off

[Dikshita Agarwal]

Currently, the stream off process checks the count of buffers in
v4l2_m2m_queues using v4l2_m2m_for_each_src_buf_safe and
v4l2_m2m_for_each_dst_buf_safe APIs. If the count is non-zero, it
returns an error. This check is redundant as the V4L2 framework already
handles buffer management internally.

Remove the unnecessary buffer count check in stream off, simplifying the
process and relying on V4L2's internal mechanisms for buffer management.

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_vdec.c | 36 ----------------------------
 1 file changed, 36 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index de4e3fe8ed5a..ce97c555192a 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -244,35 +244,6 @@ void iris_vdec_src_change(struct iris_inst *inst)
 	v4l2_event_queue_fh(&inst->fh, &event);
 }
 
-static int iris_vdec_get_num_queued_buffers(struct iris_inst *inst,
-					    enum iris_buffer_type type)
-{
-	struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx;
-	struct v4l2_m2m_buffer *buffer, *n;
-	struct iris_buffer *buf;
-	u32 count = 0;
-
-	switch (type) {
-	case BUF_INPUT:
-		v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buffer, n) {
-			buf = to_iris_buffer(&buffer->vb);
-			if (!(buf->attr & BUF_ATTR_QUEUED))
-				continue;
-			count++;
-		}
-		return count;
-	case BUF_OUTPUT:
-		v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, buffer, n) {
-			buf = to_iris_buffer(&buffer->vb);
-			if (!(buf->attr & BUF_ATTR_QUEUED))
-				continue;
-			count++;
-		}
-		return count;
-	default:
-		return count;
-	}
-}
 
 static void iris_vdec_flush_deferred_buffers(struct iris_inst *inst,
 					     enum iris_buffer_type type)
@@ -321,7 +292,6 @@ int iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane)
 {
 	const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;
 	enum iris_buffer_type buffer_type;
-	u32 count;
 	int ret;
 
 	switch (plane) {
@@ -339,12 +309,6 @@ int iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane)
 	if (ret)
 		goto error;
 
-	count = iris_vdec_get_num_queued_buffers(inst, buffer_type);
-	if (count) {
-		ret = -EINVAL;
-		goto error;
-	}
-
 	ret = iris_inst_state_change_streamoff(inst, plane);
 	if (ret)
 		goto error;

-- 
2.34.1

[PATCH v3 18/23] media: iris: Add a comment to explain usage of MBPS

[Dikshita Agarwal]

Add a comment to explain usage of MBPS and define a macro for 8K
resolution for better readability

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_platform_common.h | 2 ++
 drivers/media/platform/qcom/iris/iris_platform_gen2.c   | 4 ++--
 drivers/media/platform/qcom/iris/iris_platform_sm8250.c | 2 +-
 3 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 1dab276431c7..3e0ae87526a0 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -21,6 +21,7 @@ struct iris_inst;
 #define DEFAULT_MAX_HOST_BUF_COUNT		64
 #define DEFAULT_MAX_HOST_BURST_BUF_COUNT	256
 #define DEFAULT_FPS				30
+#define NUM_MBS_8K				((8192 * 4352) / 256)
 
 enum stage_type {
 	STAGE_1 = 1,
@@ -172,6 +173,7 @@ struct iris_platform_data {
 	struct ubwc_config_data *ubwc_config;
 	u32 num_vpp_pipe;
 	u32 max_session_count;
+	/* max number of macroblocks per frame supported */
 	u32 max_core_mbpf;
 	const u32 *input_config_params;
 	unsigned int input_config_params_size;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index 1e69ba15db0f..deb7037e8e86 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -248,7 +248,7 @@ struct iris_platform_data sm8550_data = {
 	.ubwc_config = &ubwc_config_sm8550,
 	.num_vpp_pipe = 4,
 	.max_session_count = 16,
-	.max_core_mbpf = ((8192 * 4352) / 256) * 2,
+	.max_core_mbpf = NUM_MBS_8K * 2,
 	.input_config_params =
 		sm8550_vdec_input_config_params,
 	.input_config_params_size =
@@ -308,7 +308,7 @@ struct iris_platform_data sm8650_data = {
 	.ubwc_config = &ubwc_config_sm8550,
 	.num_vpp_pipe = 4,
 	.max_session_count = 16,
-	.max_core_mbpf = ((8192 * 4352) / 256) * 2,
+	.max_core_mbpf = NUM_MBS_8K * 2,
 	.input_config_params =
 		sm8550_vdec_input_config_params,
 	.input_config_params_size =
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
index 543fa2661539..8183e4e95fa4 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
@@ -127,7 +127,7 @@ struct iris_platform_data sm8250_data = {
 	.hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE,
 	.num_vpp_pipe = 4,
 	.max_session_count = 16,
-	.max_core_mbpf = (8192 * 4352) / 256,
+	.max_core_mbpf = NUM_MBS_8K,
 	.input_config_params =
 		sm8250_vdec_input_config_param_default,
 	.input_config_params_size =

-- 
2.34.1

[PATCH v3 19/23] media: iris: Add HEVC and VP9 formats for decoder

[Dikshita Agarwal]

Extend the decoder driver's supported formats to include HEVC (H.265)
and VP9. This change updates the format enumeration (VIDIOC_ENUM_FMT)
and allows setting these formats via VIDIOC_S_FMT.

Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 .../platform/qcom/iris/iris_hfi_gen1_command.c     | 15 ++++-
 .../platform/qcom/iris/iris_hfi_gen1_defines.h     |  2 +
 .../platform/qcom/iris/iris_hfi_gen2_command.c     | 14 ++++-
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  3 +
 drivers/media/platform/qcom/iris/iris_instance.h   |  2 +
 drivers/media/platform/qcom/iris/iris_vdec.c       | 69 ++++++++++++++++++++--
 drivers/media/platform/qcom/iris/iris_vdec.h       | 11 ++++
 drivers/media/platform/qcom/iris/iris_vidc.c       |  3 -
 8 files changed, 108 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
index bd9d86220e61..dbb1b1dab097 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
@@ -88,16 +88,29 @@ static int iris_hfi_gen1_sys_pc_prep(struct iris_core *core)
 static int iris_hfi_gen1_session_open(struct iris_inst *inst)
 {
 	struct hfi_session_open_pkt packet;
+	u32 codec = 0;
 	int ret;
 
 	if (inst->state != IRIS_INST_DEINIT)
 		return -EALREADY;
 
+	switch (inst->codec) {
+	case V4L2_PIX_FMT_H264:
+		codec = HFI_VIDEO_CODEC_H264;
+		break;
+	case V4L2_PIX_FMT_HEVC:
+		codec = HFI_VIDEO_CODEC_HEVC;
+		break;
+	case V4L2_PIX_FMT_VP9:
+		codec = HFI_VIDEO_CODEC_VP9;
+		break;
+	}
+
 	packet.shdr.hdr.size = sizeof(struct hfi_session_open_pkt);
 	packet.shdr.hdr.pkt_type = HFI_CMD_SYS_SESSION_INIT;
 	packet.shdr.session_id = inst->session_id;
 	packet.session_domain = HFI_SESSION_TYPE_DEC;
-	packet.session_codec = HFI_VIDEO_CODEC_H264;
+	packet.session_codec = codec;
 
 	reinit_completion(&inst->completion);
 
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
index adffcead58ea..d4d119ca98b0 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h
@@ -13,6 +13,8 @@
 #define HFI_SESSION_TYPE_DEC				2
 
 #define HFI_VIDEO_CODEC_H264				0x00000002
+#define HFI_VIDEO_CODEC_HEVC				0x00002000
+#define HFI_VIDEO_CODEC_VP9				0x00004000
 
 #define HFI_ERR_NONE					0x0
 
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
index 802fa62c26eb..f23be2340658 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -416,7 +416,19 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
 static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst)
 {
 	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
-	u32 codec = HFI_CODEC_DECODE_AVC;
+	u32 codec = 0;
+
+	switch (inst->codec) {
+	case V4L2_PIX_FMT_H264:
+		codec = HFI_CODEC_DECODE_AVC;
+		break;
+	case V4L2_PIX_FMT_HEVC:
+		codec = HFI_CODEC_DECODE_HEVC;
+		break;
+	case V4L2_PIX_FMT_VP9:
+		codec = HFI_CODEC_DECODE_VP9;
+		break;
+	}
 
 	iris_hfi_gen2_packet_session_property(inst,
 					      HFI_PROP_CODEC,
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index 666061a612c3..283d2f27e4c8 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -104,6 +104,9 @@ enum hfi_color_format {
 enum hfi_codec_type {
 	HFI_CODEC_DECODE_AVC			= 1,
 	HFI_CODEC_ENCODE_AVC			= 2,
+	HFI_CODEC_DECODE_HEVC			= 3,
+	HFI_CODEC_ENCODE_HEVC			= 4,
+	HFI_CODEC_DECODE_VP9			= 5,
 };
 
 enum hfi_picture_type {
diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
index 5ec6368b2af7..0e1f5799b72d 100644
--- a/drivers/media/platform/qcom/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/iris/iris_instance.h
@@ -43,6 +43,7 @@
  * @sequence_out: a sequence counter for output queue
  * @tss: timestamp metadata
  * @metadata_idx: index for metadata buffer
+ * @codec: codec type
  * @last_buffer_dequeued: a flag to indicate that last buffer is sent by driver
  */
 
@@ -75,6 +76,7 @@ struct iris_inst {
 	u32				sequence_out;
 	struct iris_ts_metadata		tss[VIDEO_MAX_FRAME];
 	u32				metadata_idx;
+	u32				codec;
 	bool				last_buffer_dequeued;
 };
 
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index ce97c555192a..d670b51c5839 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -32,6 +32,7 @@ int iris_vdec_inst_init(struct iris_inst *inst)
 	f->fmt.pix_mp.width = DEFAULT_WIDTH;
 	f->fmt.pix_mp.height = DEFAULT_HEIGHT;
 	f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;
+	inst->codec = f->fmt.pix_mp.pixelformat;
 	f->fmt.pix_mp.num_planes = 1;
 	f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
 	f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT);
@@ -67,14 +68,67 @@ void iris_vdec_inst_deinit(struct iris_inst *inst)
 	kfree(inst->fmt_src);
 }
 
+static const struct iris_fmt iris_vdec_formats[] = {
+	[IRIS_FMT_H264] = {
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+	[IRIS_FMT_HEVC] = {
+		.pixfmt = V4L2_PIX_FMT_HEVC,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+	[IRIS_FMT_VP9] = {
+		.pixfmt = V4L2_PIX_FMT_VP9,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+};
+
+static const struct iris_fmt *
+find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
+{
+	unsigned int size = ARRAY_SIZE(iris_vdec_formats);
+	const struct iris_fmt *fmt = iris_vdec_formats;
+	unsigned int i;
+
+	for (i = 0; i < size; i++) {
+		if (fmt[i].pixfmt == pixfmt)
+			break;
+	}
+
+	if (i == size || fmt[i].type != type)
+		return NULL;
+
+	return &fmt[i];
+}
+
+static const struct iris_fmt *
+find_format_by_index(struct iris_inst *inst, u32 index, u32 type)
+{
+	const struct iris_fmt *fmt = iris_vdec_formats;
+	unsigned int size = ARRAY_SIZE(iris_vdec_formats);
+
+	if (index >= size || fmt[index].type != type)
+		return NULL;
+
+	return &fmt[index];
+}
+
 int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f)
 {
+	const struct iris_fmt *fmt;
+
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
-		f->pixelformat = V4L2_PIX_FMT_H264;
+		fmt = find_format_by_index(inst, f->index, f->type);
+		if (!fmt)
+			return -EINVAL;
+
+		f->pixelformat = fmt->pixfmt;
 		f->flags = V4L2_FMT_FLAG_COMPRESSED | V4L2_FMT_FLAG_DYN_RESOLUTION;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		if (f->index)
+			return -EINVAL;
 		f->pixelformat = V4L2_PIX_FMT_NV12;
 		break;
 	default:
@@ -88,13 +142,15 @@ int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f)
 {
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
 	struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx;
+	const struct iris_fmt *fmt;
 	struct v4l2_format *f_inst;
 	struct vb2_queue *src_q;
 
 	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+	fmt = find_format(inst, pixmp->pixelformat, f->type);
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
-		if (f->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_H264) {
+		if (!fmt) {
 			f_inst = inst->fmt_src;
 			f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
 			f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height;
@@ -102,7 +158,7 @@ int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f)
 		}
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
-		if (f->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_NV12) {
+		if (!fmt) {
 			f_inst = inst->fmt_dst;
 			f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat;
 			f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
@@ -145,13 +201,14 @@ int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
 
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
-		if (f->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_H264)
+		if (!(find_format(inst, f->fmt.pix_mp.pixelformat, f->type)))
 			return -EINVAL;
 
 		fmt = inst->fmt_src;
 		fmt->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
-
-		codec_align = DEFAULT_CODEC_ALIGNMENT;
+		fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat;
+		inst->codec = fmt->fmt.pix_mp.pixelformat;
+		codec_align = inst->codec == V4L2_PIX_FMT_HEVC ? 32 : 16;
 		fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align);
 		fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align);
 		fmt->fmt.pix_mp.num_planes = 1;
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h
index b24932dc511a..cd7aab66dc7c 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/iris/iris_vdec.h
@@ -8,6 +8,17 @@
 
 struct iris_inst;
 
+enum iris_fmt_type {
+	IRIS_FMT_H264,
+	IRIS_FMT_HEVC,
+	IRIS_FMT_VP9,
+};
+
+struct iris_fmt {
+	u32 pixfmt;
+	u32 type;
+};
+
 int iris_vdec_inst_init(struct iris_inst *inst);
 void iris_vdec_inst_deinit(struct iris_inst *inst);
 int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
index a8144595cc78..c417e8c31f80 100644
--- a/drivers/media/platform/qcom/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/iris/iris_vidc.c
@@ -278,9 +278,6 @@ static int iris_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
 {
 	struct iris_inst *inst = iris_get_inst(filp, NULL);
 
-	if (f->index)
-		return -EINVAL;
-
 	return iris_vdec_enum_fmt(inst, f);
 }
 

-- 
2.34.1

[PATCH v3 20/23] media: iris: Add platform capabilities for HEVC and VP9 decoders

[Dikshita Agarwal]

Add platform capabilities for HEVC and VP9 codecs in decoder driver
with related hooks.

Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_ctrls.c      |  28 ++++-
 .../platform/qcom/iris/iris_hfi_gen2_command.c     |  28 ++++-
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   1 +
 .../platform/qcom/iris/iris_hfi_gen2_response.c    |  34 +++++-
 .../platform/qcom/iris/iris_platform_common.h      |   8 +-
 .../media/platform/qcom/iris/iris_platform_gen2.c  |  80 ++++++++++++-
 .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++++++++++++++----
 7 files changed, 266 insertions(+), 39 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
index 13f5cf0d0e8a..9136b723c0f2 100644
--- a/drivers/media/platform/qcom/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
@@ -18,9 +18,19 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
 {
 	switch (id) {
 	case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
-		return PROFILE;
+		return PROFILE_H264;
+	case V4L2_CID_MPEG_VIDEO_HEVC_PROFILE:
+		return PROFILE_HEVC;
+	case V4L2_CID_MPEG_VIDEO_VP9_PROFILE:
+		return PROFILE_VP9;
 	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
-		return LEVEL;
+		return LEVEL_H264;
+	case V4L2_CID_MPEG_VIDEO_HEVC_LEVEL:
+		return LEVEL_HEVC;
+	case V4L2_CID_MPEG_VIDEO_VP9_LEVEL:
+		return LEVEL_VP9;
+	case V4L2_CID_MPEG_VIDEO_HEVC_TIER:
+		return TIER;
 	default:
 		return INST_FW_CAP_MAX;
 	}
@@ -32,10 +42,20 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
 		return 0;
 
 	switch (cap_id) {
-	case PROFILE:
+	case PROFILE_H264:
 		return V4L2_CID_MPEG_VIDEO_H264_PROFILE;
-	case LEVEL:
+	case PROFILE_HEVC:
+		return V4L2_CID_MPEG_VIDEO_HEVC_PROFILE;
+	case PROFILE_VP9:
+		return V4L2_CID_MPEG_VIDEO_VP9_PROFILE;
+	case LEVEL_H264:
 		return V4L2_CID_MPEG_VIDEO_H264_LEVEL;
+	case LEVEL_HEVC:
+		return V4L2_CID_MPEG_VIDEO_HEVC_LEVEL;
+	case LEVEL_VP9:
+		return V4L2_CID_MPEG_VIDEO_VP9_LEVEL;
+	case TIER:
+		return V4L2_CID_MPEG_VIDEO_HEVC_TIER;
 	default:
 		return 0;
 	}
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
index f23be2340658..8c91d336ff7e 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -295,7 +295,19 @@ static int iris_hfi_gen2_set_profile(struct iris_inst *inst)
 {
 	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
 	u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
-	u32 profile = inst->fw_caps[PROFILE].value;
+	u32 profile = 0;
+
+	switch (inst->codec) {
+	case V4L2_PIX_FMT_HEVC:
+		profile = inst->fw_caps[PROFILE_HEVC].value;
+		break;
+	case V4L2_PIX_FMT_VP9:
+		profile = inst->fw_caps[PROFILE_VP9].value;
+		break;
+	case V4L2_PIX_FMT_H264:
+		profile = inst->fw_caps[PROFILE_H264].value;
+		break;
+	}
 
 	inst_hfi_gen2->src_subcr_params.profile = profile;
 
@@ -312,7 +324,19 @@ static int iris_hfi_gen2_set_level(struct iris_inst *inst)
 {
 	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
 	u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
-	u32 level = inst->fw_caps[LEVEL].value;
+	u32 level = 0;
+
+	switch (inst->codec) {
+	case V4L2_PIX_FMT_HEVC:
+		level = inst->fw_caps[LEVEL_HEVC].value;
+		break;
+	case V4L2_PIX_FMT_VP9:
+		level = inst->fw_caps[LEVEL_VP9].value;
+		break;
+	case V4L2_PIX_FMT_H264:
+		level = inst->fw_caps[LEVEL_H264].value;
+		break;
+	}
 
 	inst_hfi_gen2->src_subcr_params.level = level;
 
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index 283d2f27e4c8..5f13dc11bea5 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -46,6 +46,7 @@
 #define HFI_PROP_CROP_OFFSETS			0x03000105
 #define HFI_PROP_PROFILE			0x03000107
 #define HFI_PROP_LEVEL				0x03000108
+#define HFI_PROP_TIER				0x03000109
 #define HFI_PROP_STAGE				0x0300010a
 #define HFI_PROP_PIPE				0x0300010b
 #define HFI_PROP_LUMA_CHROMA_BIT_DEPTH		0x0300010f
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 2267e220c9ea..1b8787c0f6a7 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -572,8 +572,21 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
 	inst->crop.width = pixmp_ip->width -
 		((subsc_params.crop_offsets[1] >> 16) & 0xFFFF) - inst->crop.left;
 
-	inst->fw_caps[PROFILE].value = subsc_params.profile;
-	inst->fw_caps[LEVEL].value = subsc_params.level;
+	switch (inst->codec) {
+	case V4L2_PIX_FMT_HEVC:
+		inst->fw_caps[PROFILE_HEVC].value = subsc_params.profile;
+		inst->fw_caps[LEVEL_HEVC].value = subsc_params.level;
+		break;
+	case V4L2_PIX_FMT_VP9:
+		inst->fw_caps[PROFILE_VP9].value = subsc_params.profile;
+		inst->fw_caps[LEVEL_VP9].value = subsc_params.level;
+		break;
+	case V4L2_PIX_FMT_H264:
+		inst->fw_caps[PROFILE_H264].value = subsc_params.profile;
+		inst->fw_caps[LEVEL_H264].value = subsc_params.level;
+		break;
+	}
+
 	inst->fw_caps[POC].value = subsc_params.pic_order_cnt;
 
 	if (subsc_params.bit_depth != BIT_DEPTH_8 ||
@@ -797,8 +810,21 @@ static void iris_hfi_gen2_init_src_change_param(struct iris_inst *inst)
 					     full_range, video_format,
 					     video_signal_type_present_flag);
 
-	subsc_params->profile = inst->fw_caps[PROFILE].value;
-	subsc_params->level = inst->fw_caps[LEVEL].value;
+	switch (inst->codec) {
+	case V4L2_PIX_FMT_HEVC:
+		subsc_params->profile = inst->fw_caps[PROFILE_HEVC].value;
+		subsc_params->level = inst->fw_caps[LEVEL_HEVC].value;
+		break;
+	case V4L2_PIX_FMT_VP9:
+		subsc_params->profile = inst->fw_caps[PROFILE_VP9].value;
+		subsc_params->level = inst->fw_caps[LEVEL_VP9].value;
+		break;
+	case V4L2_PIX_FMT_H264:
+		subsc_params->profile = inst->fw_caps[PROFILE_H264].value;
+		subsc_params->level = inst->fw_caps[LEVEL_H264].value;
+		break;
+	}
+
 	subsc_params->pic_order_cnt = inst->fw_caps[POC].value;
 	subsc_params->bit_depth = inst->fw_caps[BIT_DEPTH].value;
 	if (inst->fw_caps[CODED_FRAMES].value ==
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 3e0ae87526a0..71d23214f224 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -81,8 +81,12 @@ struct platform_inst_caps {
 };
 
 enum platform_inst_fw_cap_type {
-	PROFILE = 1,
-	LEVEL,
+	PROFILE_H264 = 1,
+	PROFILE_HEVC,
+	PROFILE_VP9,
+	LEVEL_H264,
+	LEVEL_HEVC,
+	LEVEL_VP9,
 	INPUT_BUF_HOST_MAX_COUNT,
 	STAGE,
 	PIPE,
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index deb7037e8e86..c2cded2876b7 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -17,7 +17,7 @@
 
 static struct platform_inst_fw_cap inst_fw_cap_sm8550[] = {
 	{
-		.cap_id = PROFILE,
+		.cap_id = PROFILE_H264,
 		.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
 		.max = V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH,
 		.step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
@@ -31,7 +31,29 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550[] = {
 		.set = iris_set_u32_enum,
 	},
 	{
-		.cap_id = LEVEL,
+		.cap_id = PROFILE_HEVC,
+		.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+		.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE),
+		.value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+		.hfi_id = HFI_PROP_PROFILE,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = PROFILE_VP9,
+		.min = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+		.max = V4L2_MPEG_VIDEO_VP9_PROFILE_2,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_2),
+		.value = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+		.hfi_id = HFI_PROP_PROFILE,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = LEVEL_H264,
 		.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
 		.max = V4L2_MPEG_VIDEO_H264_LEVEL_6_2,
 		.step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) |
@@ -59,6 +81,60 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8550[] = {
 		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
 		.set = iris_set_u32_enum,
 	},
+	{
+		.cap_id = LEVEL_HEVC,
+		.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+		.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2),
+		.value = V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1,
+		.hfi_id = HFI_PROP_LEVEL,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = LEVEL_VP9,
+		.min = V4L2_MPEG_VIDEO_VP9_LEVEL_1_0,
+		.max = V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_2) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_6_0),
+		.value = V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
+		.hfi_id = HFI_PROP_LEVEL,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = TIER,
+		.min = V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
+		.max = V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_TIER_HIGH),
+		.value = V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
+		.hfi_id = HFI_PROP_TIER,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
 	{
 		.cap_id = INPUT_BUF_HOST_MAX_COUNT,
 		.min = DEFAULT_MAX_HOST_BUF_COUNT,
diff --git a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
index f82355d72fcf..a8d66ed388a3 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
@@ -5,48 +5,124 @@
 
 static struct platform_inst_fw_cap inst_fw_cap_qcs8300[] = {
 	{
-		.cap_id = PROFILE,
+		.cap_id = PROFILE_H264,
 		.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
 		.max = V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH,
 		.step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
-			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) |
-			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
-			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
-			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH),
+				BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+				BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+				BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) |
+				BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH),
 		.value = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
 		.hfi_id = HFI_PROP_PROFILE,
 		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
 		.set = iris_set_u32_enum,
 	},
 	{
-		.cap_id = LEVEL,
+		.cap_id = PROFILE_HEVC,
+		.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+		.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE),
+		.value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+		.hfi_id = HFI_PROP_PROFILE,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = PROFILE_VP9,
+		.min = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+		.max = V4L2_MPEG_VIDEO_VP9_PROFILE_2,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_2),
+		.value = V4L2_MPEG_VIDEO_VP9_PROFILE_0,
+		.hfi_id = HFI_PROP_PROFILE,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = LEVEL_H264,
 		.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
 		.max = V4L2_MPEG_VIDEO_H264_LEVEL_6_2,
 		.step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B)  |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_1) |
-			BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_2),
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_1) |
+				BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_2),
 		.value = V4L2_MPEG_VIDEO_H264_LEVEL_6_1,
 		.hfi_id = HFI_PROP_LEVEL,
 		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
 		.set = iris_set_u32_enum,
 	},
+	{
+		.cap_id = LEVEL_HEVC,
+		.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+		.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2),
+		.value = V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1,
+		.hfi_id = HFI_PROP_LEVEL,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = LEVEL_VP9,
+		.min = V4L2_MPEG_VIDEO_VP9_LEVEL_1_0,
+		.max = V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_0) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_1) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_2) |
+				BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_6_0),
+		.value = V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
+		.hfi_id = HFI_PROP_LEVEL,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
+	{
+		.cap_id = TIER,
+		.min = V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
+		.max = V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
+		.step_or_mask = BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) |
+				BIT(V4L2_MPEG_VIDEO_HEVC_TIER_HIGH),
+		.value = V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
+		.hfi_id = HFI_PROP_TIER,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+		.set = iris_set_u32_enum,
+	},
 	{
 		.cap_id = INPUT_BUF_HOST_MAX_COUNT,
 		.min = DEFAULT_MAX_HOST_BUF_COUNT,

-- 
2.34.1

[PATCH v3 21/23] media: iris: Set mandatory properties for HEVC and VP9 decoders.

[Dikshita Agarwal]

Subscribe and set mandatory properties to the firmware for HEVC and VP9
decoders.

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
 .../platform/qcom/iris/iris_hfi_gen1_command.c     |   4 +-
 .../platform/qcom/iris/iris_hfi_gen2_command.c     |  97 +++++++++++++++---
 .../platform/qcom/iris/iris_hfi_gen2_response.c    |   7 ++
 .../platform/qcom/iris/iris_platform_common.h      |  16 ++-
 .../media/platform/qcom/iris/iris_platform_gen2.c  | 114 +++++++++++++++++----
 .../platform/qcom/iris/iris_platform_sm8250.c      |   4 +-
 7 files changed, 203 insertions(+), 40 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h
index b2c541367fc6..9e6aadb83783 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_common.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h
@@ -140,6 +140,7 @@ struct hfi_subscription_params {
 	u32	color_info;
 	u32	profile;
 	u32	level;
+	u32	tier;
 };
 
 u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries);
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
index dbb1b1dab097..2e3f5a6b2ff1 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
@@ -776,8 +776,8 @@ static int iris_hfi_gen1_session_set_config_params(struct iris_inst *inst, u32 p
 			iris_hfi_gen1_set_bufsize},
 	};
 
-	config_params = core->iris_platform_data->input_config_params;
-	config_params_size = core->iris_platform_data->input_config_params_size;
+	config_params = core->iris_platform_data->input_config_params_default;
+	config_params_size = core->iris_platform_data->input_config_params_default_size;
 
 	if (V4L2_TYPE_IS_OUTPUT(plane)) {
 		for (i = 0; i < config_params_size; i++) {
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
index 8c91d336ff7e..7ca5ae13d62b 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -391,11 +391,28 @@ static int iris_hfi_gen2_set_linear_stride_scanline(struct iris_inst *inst)
 						  sizeof(u64));
 }
 
+static int iris_hfi_gen2_set_tier(struct iris_inst *inst)
+{
+	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
+	u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+	u32 tier = inst->fw_caps[TIER].value;
+
+	inst_hfi_gen2->src_subcr_params.tier = tier;
+
+	return iris_hfi_gen2_session_set_property(inst,
+						  HFI_PROP_TIER,
+						  HFI_HOST_FLAGS_NONE,
+						  port,
+						  HFI_PAYLOAD_U32_ENUM,
+						  &tier,
+						  sizeof(u32));
+}
+
 static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 plane)
 {
 	struct iris_core *core = inst->core;
-	u32 config_params_size, i, j;
-	const u32 *config_params;
+	u32 config_params_size = 0, i, j;
+	const u32 *config_params = NULL;
 	int ret;
 
 	static const struct iris_hfi_prop_type_handle prop_type_handle_arr[] = {
@@ -410,11 +427,27 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
 		{HFI_PROP_LEVEL,                      iris_hfi_gen2_set_level                  },
 		{HFI_PROP_COLOR_FORMAT,               iris_hfi_gen2_set_colorformat            },
 		{HFI_PROP_LINEAR_STRIDE_SCANLINE,     iris_hfi_gen2_set_linear_stride_scanline },
+		{HFI_PROP_TIER,                       iris_hfi_gen2_set_tier                   },
 	};
 
 	if (V4L2_TYPE_IS_OUTPUT(plane)) {
-		config_params = core->iris_platform_data->input_config_params;
-		config_params_size = core->iris_platform_data->input_config_params_size;
+		switch (inst->codec) {
+		case V4L2_PIX_FMT_H264:
+			config_params = core->iris_platform_data->input_config_params_default;
+			config_params_size =
+				core->iris_platform_data->input_config_params_default_size;
+			break;
+		case V4L2_PIX_FMT_HEVC:
+			config_params = core->iris_platform_data->input_config_params_hevc;
+			config_params_size =
+				core->iris_platform_data->input_config_params_hevc_size;
+			break;
+		case V4L2_PIX_FMT_VP9:
+			config_params = core->iris_platform_data->input_config_params_vp9;
+			config_params_size =
+				core->iris_platform_data->input_config_params_vp9_size;
+			break;
+		}
 	} else {
 		config_params = core->iris_platform_data->output_config_params;
 		config_params_size = core->iris_platform_data->output_config_params_size;
@@ -584,8 +617,8 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
 	struct hfi_subscription_params subsc_params;
 	u32 prop_type, payload_size, payload_type;
 	struct iris_core *core = inst->core;
-	const u32 *change_param;
-	u32 change_param_size;
+	const u32 *change_param = NULL;
+	u32 change_param_size = 0;
 	u32 payload[32] = {0};
 	u32 hfi_port = 0, i;
 	int ret;
@@ -596,8 +629,23 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
 		return 0;
 	}
 
-	change_param = core->iris_platform_data->input_config_params;
-	change_param_size = core->iris_platform_data->input_config_params_size;
+	switch (inst->codec) {
+	case V4L2_PIX_FMT_H264:
+		change_param = core->iris_platform_data->input_config_params_default;
+		change_param_size =
+			core->iris_platform_data->input_config_params_default_size;
+		break;
+	case V4L2_PIX_FMT_HEVC:
+		change_param = core->iris_platform_data->input_config_params_hevc;
+		change_param_size =
+			core->iris_platform_data->input_config_params_hevc_size;
+		break;
+	case V4L2_PIX_FMT_VP9:
+		change_param = core->iris_platform_data->input_config_params_vp9;
+		change_param_size =
+			core->iris_platform_data->input_config_params_vp9_size;
+		break;
+	}
 
 	payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
 
@@ -644,6 +692,11 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
 				payload_size = sizeof(u32);
 				payload_type = HFI_PAYLOAD_U32;
 				break;
+			case HFI_PROP_LUMA_CHROMA_BIT_DEPTH:
+				payload[0] = subsc_params.bit_depth;
+				payload_size = sizeof(u32);
+				payload_type = HFI_PAYLOAD_U32;
+				break;
 			case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT:
 				payload[0] = subsc_params.fw_min_count;
 				payload_size = sizeof(u32);
@@ -669,6 +722,11 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
 				payload_size = sizeof(u32);
 				payload_type = HFI_PAYLOAD_U32;
 				break;
+			case HFI_PROP_TIER:
+				payload[0] = subsc_params.tier;
+				payload_size = sizeof(u32);
+				payload_type = HFI_PAYLOAD_U32;
+				break;
 			default:
 				prop_type = 0;
 				ret = -EINVAL;
@@ -695,8 +753,8 @@ static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plan
 static int iris_hfi_gen2_subscribe_property(struct iris_inst *inst, u32 plane)
 {
 	struct iris_core *core = inst->core;
-	u32 subscribe_prop_size, i;
-	const u32 *subcribe_prop;
+	u32 subscribe_prop_size = 0, i;
+	const u32 *subcribe_prop = NULL;
 	u32 payload[32] = {0};
 
 	payload[0] = HFI_MODE_PROPERTY;
@@ -705,8 +763,23 @@ static int iris_hfi_gen2_subscribe_property(struct iris_inst *inst, u32 plane)
 		subscribe_prop_size = core->iris_platform_data->dec_input_prop_size;
 		subcribe_prop = core->iris_platform_data->dec_input_prop;
 	} else {
-		subscribe_prop_size = core->iris_platform_data->dec_output_prop_size;
-		subcribe_prop = core->iris_platform_data->dec_output_prop;
+		switch (inst->codec) {
+		case V4L2_PIX_FMT_H264:
+			subcribe_prop = core->iris_platform_data->dec_output_prop_avc;
+			subscribe_prop_size =
+				core->iris_platform_data->dec_output_prop_avc_size;
+			break;
+		case V4L2_PIX_FMT_HEVC:
+			subcribe_prop = core->iris_platform_data->dec_output_prop_hevc;
+			subscribe_prop_size =
+				core->iris_platform_data->dec_output_prop_hevc_size;
+			break;
+		case V4L2_PIX_FMT_VP9:
+			subcribe_prop = core->iris_platform_data->dec_output_prop_vp9;
+			subscribe_prop_size =
+				core->iris_platform_data->dec_output_prop_vp9_size;
+			break;
+		}
 	}
 
 	for (i = 0; i < subscribe_prop_size; i++)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 1b8787c0f6a7..221f84d98a48 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -588,6 +588,7 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
 	}
 
 	inst->fw_caps[POC].value = subsc_params.pic_order_cnt;
+	inst->fw_caps[TIER].value = subsc_params.tier;
 
 	if (subsc_params.bit_depth != BIT_DEPTH_8 ||
 	    !(subsc_params.coded_frames & HFI_BITMASK_FRAME_MBS_ONLY_FLAG)) {
@@ -669,6 +670,9 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
 		inst_hfi_gen2->src_subcr_params.crop_offsets[0] = pkt->payload[0];
 		inst_hfi_gen2->src_subcr_params.crop_offsets[1] = pkt->payload[1];
 		break;
+	case HFI_PROP_LUMA_CHROMA_BIT_DEPTH:
+		inst_hfi_gen2->src_subcr_params.bit_depth = pkt->payload[0];
+		break;
 	case HFI_PROP_CODED_FRAMES:
 		inst_hfi_gen2->src_subcr_params.coded_frames = pkt->payload[0];
 		break;
@@ -687,6 +691,9 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
 	case HFI_PROP_LEVEL:
 		inst_hfi_gen2->src_subcr_params.level = pkt->payload[0];
 		break;
+	case HFI_PROP_TIER:
+		inst_hfi_gen2->src_subcr_params.tier = pkt->payload[0];
+		break;
 	case HFI_PROP_PICTURE_TYPE:
 		inst_hfi_gen2->hfi_frame_info.picture_type = pkt->payload[0];
 		break;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 71d23214f224..adafdce8a856 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -179,14 +179,22 @@ struct iris_platform_data {
 	u32 max_session_count;
 	/* max number of macroblocks per frame supported */
 	u32 max_core_mbpf;
-	const u32 *input_config_params;
-	unsigned int input_config_params_size;
+	const u32 *input_config_params_default;
+	unsigned int input_config_params_default_size;
+	const u32 *input_config_params_hevc;
+	unsigned int input_config_params_hevc_size;
+	const u32 *input_config_params_vp9;
+	unsigned int input_config_params_vp9_size;
 	const u32 *output_config_params;
 	unsigned int output_config_params_size;
 	const u32 *dec_input_prop;
 	unsigned int dec_input_prop_size;
-	const u32 *dec_output_prop;
-	unsigned int dec_output_prop_size;
+	const u32 *dec_output_prop_avc;
+	unsigned int dec_output_prop_avc_size;
+	const u32 *dec_output_prop_hevc;
+	unsigned int dec_output_prop_hevc_size;
+	const u32 *dec_output_prop_vp9;
+	unsigned int dec_output_prop_vp9_size;
 	const u32 *dec_ip_int_buf_tbl;
 	unsigned int dec_ip_int_buf_tbl_size;
 	const u32 *dec_op_int_buf_tbl;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index c2cded2876b7..d3026b2bcb70 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -257,9 +257,10 @@ static struct tz_cp_config tz_cp_config_sm8550 = {
 	.cp_nonpixel_size = 0x24800000,
 };
 
-static const u32 sm8550_vdec_input_config_params[] = {
+static const u32 sm8550_vdec_input_config_params_default[] = {
 	HFI_PROP_BITSTREAM_RESOLUTION,
 	HFI_PROP_CROP_OFFSETS,
+	HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
 	HFI_PROP_CODED_FRAMES,
 	HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
 	HFI_PROP_PIC_ORDER_CNT_TYPE,
@@ -268,6 +269,26 @@ static const u32 sm8550_vdec_input_config_params[] = {
 	HFI_PROP_SIGNAL_COLOR_INFO,
 };
 
+static const u32 sm8550_vdec_input_config_param_hevc[] = {
+	HFI_PROP_BITSTREAM_RESOLUTION,
+	HFI_PROP_CROP_OFFSETS,
+	HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+	HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+	HFI_PROP_PROFILE,
+	HFI_PROP_LEVEL,
+	HFI_PROP_TIER,
+	HFI_PROP_SIGNAL_COLOR_INFO,
+};
+
+static const u32 sm8550_vdec_input_config_param_vp9[] = {
+	HFI_PROP_BITSTREAM_RESOLUTION,
+	HFI_PROP_CROP_OFFSETS,
+	HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+	HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+	HFI_PROP_PROFILE,
+	HFI_PROP_LEVEL,
+};
+
 static const u32 sm8550_vdec_output_config_params[] = {
 	HFI_PROP_COLOR_FORMAT,
 	HFI_PROP_LINEAR_STRIDE_SCANLINE,
@@ -277,11 +298,19 @@ static const u32 sm8550_vdec_subscribe_input_properties[] = {
 	HFI_PROP_NO_OUTPUT,
 };
 
-static const u32 sm8550_vdec_subscribe_output_properties[] = {
+static const u32 sm8550_vdec_subscribe_output_properties_avc[] = {
 	HFI_PROP_PICTURE_TYPE,
 	HFI_PROP_CABAC_SESSION,
 };
 
+static const u32 sm8550_vdec_subscribe_output_properties_hevc[] = {
+	HFI_PROP_PICTURE_TYPE,
+};
+
+static const u32 sm8550_vdec_subscribe_output_properties_vp9[] = {
+	HFI_PROP_PICTURE_TYPE,
+};
+
 static const u32 sm8550_dec_ip_int_buf_tbl[] = {
 	BUF_BIN,
 	BUF_COMV,
@@ -325,18 +354,33 @@ struct iris_platform_data sm8550_data = {
 	.num_vpp_pipe = 4,
 	.max_session_count = 16,
 	.max_core_mbpf = NUM_MBS_8K * 2,
-	.input_config_params =
-		sm8550_vdec_input_config_params,
-	.input_config_params_size =
-		ARRAY_SIZE(sm8550_vdec_input_config_params),
+	.input_config_params_default =
+		sm8550_vdec_input_config_params_default,
+	.input_config_params_default_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_params_default),
+	.input_config_params_hevc =
+		sm8550_vdec_input_config_param_hevc,
+	.input_config_params_hevc_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_param_hevc),
+	.input_config_params_vp9 =
+		sm8550_vdec_input_config_param_vp9,
+	.input_config_params_vp9_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
 	.output_config_params =
 		sm8550_vdec_output_config_params,
 	.output_config_params_size =
 		ARRAY_SIZE(sm8550_vdec_output_config_params),
 	.dec_input_prop = sm8550_vdec_subscribe_input_properties,
 	.dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_input_properties),
-	.dec_output_prop = sm8550_vdec_subscribe_output_properties,
-	.dec_output_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_output_properties),
+	.dec_output_prop_avc = sm8550_vdec_subscribe_output_properties_avc,
+	.dec_output_prop_avc_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_avc),
+	.dec_output_prop_hevc = sm8550_vdec_subscribe_output_properties_hevc,
+	.dec_output_prop_hevc_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_hevc),
+	.dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
+	.dec_output_prop_vp9_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
 
 	.dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
 	.dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
@@ -385,18 +429,33 @@ struct iris_platform_data sm8650_data = {
 	.num_vpp_pipe = 4,
 	.max_session_count = 16,
 	.max_core_mbpf = NUM_MBS_8K * 2,
-	.input_config_params =
-		sm8550_vdec_input_config_params,
-	.input_config_params_size =
-		ARRAY_SIZE(sm8550_vdec_input_config_params),
+	.input_config_params_default =
+		sm8550_vdec_input_config_params_default,
+	.input_config_params_default_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_params_default),
+	.input_config_params_hevc =
+		sm8550_vdec_input_config_param_hevc,
+	.input_config_params_hevc_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_param_hevc),
+	.input_config_params_vp9 =
+		sm8550_vdec_input_config_param_vp9,
+	.input_config_params_vp9_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
 	.output_config_params =
 		sm8550_vdec_output_config_params,
 	.output_config_params_size =
 		ARRAY_SIZE(sm8550_vdec_output_config_params),
 	.dec_input_prop = sm8550_vdec_subscribe_input_properties,
 	.dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_input_properties),
-	.dec_output_prop = sm8550_vdec_subscribe_output_properties,
-	.dec_output_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_output_properties),
+	.dec_output_prop_avc = sm8550_vdec_subscribe_output_properties_avc,
+	.dec_output_prop_avc_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_avc),
+	.dec_output_prop_hevc = sm8550_vdec_subscribe_output_properties_hevc,
+	.dec_output_prop_hevc_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_hevc),
+	.dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
+	.dec_output_prop_vp9_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
 
 	.dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
 	.dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
@@ -441,18 +500,33 @@ struct iris_platform_data qcs8300_data = {
 	.num_vpp_pipe = 2,
 	.max_session_count = 16,
 	.max_core_mbpf = ((4096 * 2176) / 256) * 4,
-	.input_config_params =
-		sm8550_vdec_input_config_params,
-	.input_config_params_size =
-		ARRAY_SIZE(sm8550_vdec_input_config_params),
+	.input_config_params_default =
+		sm8550_vdec_input_config_params_default,
+	.input_config_params_default_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_params_default),
+	.input_config_params_hevc =
+		sm8550_vdec_input_config_param_hevc,
+	.input_config_params_hevc_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_param_hevc),
+	.input_config_params_vp9 =
+		sm8550_vdec_input_config_param_vp9,
+	.input_config_params_vp9_size =
+		ARRAY_SIZE(sm8550_vdec_input_config_param_vp9),
 	.output_config_params =
 		sm8550_vdec_output_config_params,
 	.output_config_params_size =
 		ARRAY_SIZE(sm8550_vdec_output_config_params),
 	.dec_input_prop = sm8550_vdec_subscribe_input_properties,
 	.dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_input_properties),
-	.dec_output_prop = sm8550_vdec_subscribe_output_properties,
-	.dec_output_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_output_properties),
+	.dec_output_prop_avc = sm8550_vdec_subscribe_output_properties_avc,
+	.dec_output_prop_avc_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_avc),
+	.dec_output_prop_hevc = sm8550_vdec_subscribe_output_properties_hevc,
+	.dec_output_prop_hevc_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_hevc),
+	.dec_output_prop_vp9 = sm8550_vdec_subscribe_output_properties_vp9,
+	.dec_output_prop_vp9_size =
+		ARRAY_SIZE(sm8550_vdec_subscribe_output_properties_vp9),
 
 	.dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl,
 	.dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl),
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
index 8183e4e95fa4..8d0816a67ae0 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
@@ -128,9 +128,9 @@ struct iris_platform_data sm8250_data = {
 	.num_vpp_pipe = 4,
 	.max_session_count = 16,
 	.max_core_mbpf = NUM_MBS_8K,
-	.input_config_params =
+	.input_config_params_default =
 		sm8250_vdec_input_config_param_default,
-	.input_config_params_size =
+	.input_config_params_default_size =
 		ARRAY_SIZE(sm8250_vdec_input_config_param_default),
 
 	.dec_ip_int_buf_tbl = sm8250_dec_ip_int_buf_tbl,

-- 
2.34.1

[PATCH v3 22/23] media: iris: Add internal buffer calculation for HEVC and VP9 decoders

[Dikshita Agarwal]

Add internal buffer count and size calculations for HEVC and VP9
decoders.

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_buffer.c     |   3 +
 drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++-
 drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
 3 files changed, 432 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c
index 7dbac74b1a8d..6425e4919e3b 100644
--- a/drivers/media/platform/qcom/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_buffer.c
@@ -205,6 +205,9 @@ static u32 iris_bitstream_buffer_size(struct iris_inst *inst)
 	if (num_mbs > NUM_MBS_4K) {
 		div_factor = 4;
 		base_res_mbs = caps->max_mbpf;
+	} else {
+		if (inst->codec == V4L2_PIX_FMT_VP9)
+			div_factor = 1;
 	}
 
 	/*
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
index dce25e410d80..13ee93356bcb 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
@@ -31,6 +31,42 @@ static u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_p
 	return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h, num_vpp_pipes);
 }
 
+static u32 size_h265d_hw_bin_buffer(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+	u32 product = frame_width * frame_height;
+	u32 size_yuv, size_bin_hdr, size_bin_res;
+
+	size_yuv = (product <= BIN_BUFFER_THRESHOLD) ?
+		((BIN_BUFFER_THRESHOLD * 3) >> 1) : ((product * 3) >> 1);
+	size_bin_hdr = size_yuv * H265_CABAC_HDR_RATIO_HD_TOT;
+	size_bin_res = size_yuv * H265_CABAC_RES_RATIO_HD_TOT;
+	size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, DMA_ALIGNMENT) * num_vpp_pipes;
+	size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, DMA_ALIGNMENT) * num_vpp_pipes;
+
+	return size_bin_hdr + size_bin_res;
+}
+
+static u32 hfi_buffer_bin_vp9d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+	u32 _size_yuv = ALIGN(frame_width, 16) * ALIGN(frame_height, 16) * 3 / 2;
+	u32 _size = ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) *
+			VPX_DECODER_FRAME_BIN_HDR_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR *
+			VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes), DMA_ALIGNMENT) +
+			ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) *
+			VPX_DECODER_FRAME_BIN_RES_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR *
+			VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes), DMA_ALIGNMENT);
+
+	return _size * num_vpp_pipes;
+}
+
+static u32 hfi_buffer_bin_h265d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+	u32 n_aligned_w = ALIGN(frame_width, 16);
+	u32 n_aligned_h = ALIGN(frame_height, 16);
+
+	return size_h265d_hw_bin_buffer(n_aligned_w, n_aligned_h, num_vpp_pipes);
+}
+
 static u32 hfi_buffer_comv_h264d(u32 frame_width, u32 frame_height, u32 _comv_bufcount)
 {
 	u32 frame_height_in_mbs = DIV_ROUND_UP(frame_height, 16);
@@ -55,6 +91,17 @@ static u32 hfi_buffer_comv_h264d(u32 frame_width, u32 frame_height, u32 _comv_bu
 	return (size_colloc * (_comv_bufcount)) + 512;
 }
 
+static u32 hfi_buffer_comv_h265d(u32 frame_width, u32 frame_height, u32 _comv_bufcount)
+{
+	u32 frame_height_in_mbs = (frame_height + 15) >> 4;
+	u32 frame_width_in_mbs = (frame_width + 15) >> 4;
+	u32 _size;
+
+	_size = ALIGN(((frame_width_in_mbs * frame_height_in_mbs) << 8), 512);
+
+	return (_size * (_comv_bufcount)) + 512;
+}
+
 static u32 size_h264d_bse_cmd_buf(u32 frame_height)
 {
 	u32 height = ALIGN(frame_height, 32);
@@ -63,6 +110,44 @@ static u32 size_h264d_bse_cmd_buf(u32 frame_height)
 		SIZE_H264D_BSE_CMD_PER_BUF;
 }
 
+static u32 size_h265d_bse_cmd_buf(u32 frame_width, u32 frame_height)
+{
+	u32 _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+			   (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) *
+			  NUM_HW_PIC_BUF, DMA_ALIGNMENT);
+	_size = min_t(u32, _size, H265D_MAX_SLICE + 1);
+	_size = 2 * _size * SIZE_H265D_BSE_CMD_PER_BUF;
+
+	return _size;
+}
+
+static u32 hfi_buffer_persist_h265d(u32 rpu_enabled)
+{
+	return ALIGN((SIZE_SLIST_BUF_H265 * NUM_SLIST_BUF_H265 +
+		      H265_NUM_FRM_INFO * H265_DISPLAY_BUF_SIZE +
+		      H265_NUM_TILE * sizeof(u32) +
+		      NUM_HW_PIC_BUF * SIZE_SEI_USERDATA +
+		      rpu_enabled * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA),
+		     DMA_ALIGNMENT);
+}
+
+static inline
+u32 hfi_iris3_vp9d_comv_size(void)
+{
+	return (((8192 + 63) >> 6) * ((4320 + 63) >> 6) * 8 * 8 * 2 * 8);
+}
+
+static u32 hfi_buffer_persist_vp9d(void)
+{
+	return ALIGN(VP9_NUM_PROBABILITY_TABLE_BUF * VP9_PROB_TABLE_SIZE, DMA_ALIGNMENT) +
+		ALIGN(hfi_iris3_vp9d_comv_size(), DMA_ALIGNMENT) +
+		ALIGN(MAX_SUPERFRAME_HEADER_LEN, DMA_ALIGNMENT) +
+		ALIGN(VP9_UDC_HEADER_BUF_SIZE, DMA_ALIGNMENT) +
+		ALIGN(VP9_NUM_FRAME_INFO_BUF * CCE_TILE_OFFSET_SIZE, DMA_ALIGNMENT) +
+		ALIGN(VP9_NUM_FRAME_INFO_BUF * VP9_FRAME_INFO_BUF_SIZE, DMA_ALIGNMENT) +
+		HDR10_HIST_EXTRADATA_SIZE;
+}
+
 static u32 size_h264d_vpp_cmd_buf(u32 frame_height)
 {
 	u32 size, height = ALIGN(frame_height, 32);
@@ -83,17 +168,45 @@ static u32 hfi_buffer_persist_h264d(void)
 
 static u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
 {
-	u32 size_bse, size_vpp, size;
-
-	size_bse = size_h264d_bse_cmd_buf(frame_height);
-	size_vpp = size_h264d_vpp_cmd_buf(frame_height);
-	size = ALIGN(size_bse, DMA_ALIGNMENT) +
+	u32 size_bse = size_h264d_bse_cmd_buf(frame_height);
+	u32 size_vpp = size_h264d_vpp_cmd_buf(frame_height);
+	u32 size = ALIGN(size_bse, DMA_ALIGNMENT) +
 		ALIGN(size_vpp, DMA_ALIGNMENT) +
 		ALIGN(SIZE_HW_PIC(SIZE_H264D_HW_PIC_T), DMA_ALIGNMENT);
 
 	return ALIGN(size, DMA_ALIGNMENT);
 }
 
+static u32 size_h265d_vpp_cmd_buf(u32 frame_width, u32 frame_height)
+{
+	u32 _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+			   (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) *
+			  NUM_HW_PIC_BUF, DMA_ALIGNMENT);
+	_size = min_t(u32, _size, H265D_MAX_SLICE + 1);
+	_size = ALIGN(_size, 4);
+	_size = 2 * _size * SIZE_H265D_VPP_CMD_PER_BUF;
+	if (_size > VPP_CMD_MAX_SIZE)
+		_size = VPP_CMD_MAX_SIZE;
+
+	return _size;
+}
+
+static u32 hfi_buffer_non_comv_h265d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+	u32 _size_bse = size_h265d_bse_cmd_buf(frame_width, frame_height);
+	u32 _size_vpp = size_h265d_vpp_cmd_buf(frame_width, frame_height);
+	u32 _size = ALIGN(_size_bse, DMA_ALIGNMENT) +
+		ALIGN(_size_vpp, DMA_ALIGNMENT) +
+		ALIGN(NUM_HW_PIC_BUF * 20 * 22 * 4, DMA_ALIGNMENT) +
+		ALIGN(2 * sizeof(u16) *
+		(ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+		(ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS), DMA_ALIGNMENT) +
+		ALIGN(SIZE_HW_PIC(SIZE_H265D_HW_PIC_T), DMA_ALIGNMENT) +
+		HDR10_HIST_EXTRADATA_SIZE;
+
+	return ALIGN(_size, DMA_ALIGNMENT);
+}
+
 static u32 size_vpss_lb(u32 frame_width, u32 frame_height)
 {
 	u32 opb_lb_wr_llb_y_buffer_size, opb_lb_wr_llb_uv_buffer_size;
@@ -119,6 +232,203 @@ static u32 size_vpss_lb(u32 frame_width, u32 frame_height)
 		opb_lb_wr_llb_y_buffer_size;
 }
 
+static inline
+u32 size_h265d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+	return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE *
+		(ALIGN(frame_width, 64) + 8) * 2;
+}
+
+static inline
+u32 size_h265d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+	return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE *
+		(ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+	return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE *
+		(ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+	return (LCU_MAX_SIZE_PELS / 8 * (128 / 8)) * ((frame_width + 15) >> 4);
+}
+
+static inline
+u32 size_h265d_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+	return max_t(u32, ((frame_height + 16 - 1) / 8) *
+		MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+		max_t(u32, ((frame_height + 32 - 1) / 8) *
+		MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+		((frame_height + 64 - 1) / 8) *
+		MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_h265d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+	return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE *
+		(ALIGN(frame_width, LCU_MIN_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+	return ((frame_width + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_h265d_lb_vsp_left(u32 frame_width, u32 frame_height)
+{
+	return ((frame_height + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_h265d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+	return size_h264d_lb_recon_dma_metadata_wr(frame_height);
+}
+
+static inline
+u32 size_h265d_qp(u32 frame_width, u32 frame_height)
+{
+	return size_h264d_qp(frame_width, frame_height);
+}
+
+static inline
+u32 hfi_buffer_line_h265d(u32 frame_width, u32 frame_height, bool is_opb, u32 num_vpp_pipes)
+{
+	u32 vpss_lb_size = 0, _size;
+
+	_size = ALIGN(size_h265d_lb_fe_top_data(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_h265d_lb_fe_top_ctrl(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_h265d_lb_fe_left_ctrl(frame_width, frame_height),
+		      DMA_ALIGNMENT) * num_vpp_pipes +
+		ALIGN(size_h265d_lb_se_left_ctrl(frame_width, frame_height),
+		      DMA_ALIGNMENT) * num_vpp_pipes +
+		ALIGN(size_h265d_lb_se_top_ctrl(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_h265d_lb_pe_top_data(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_h265d_lb_vsp_top(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_h265d_lb_vsp_left(frame_width, frame_height),
+		      DMA_ALIGNMENT) * num_vpp_pipes +
+		ALIGN(size_h265d_lb_recon_dma_metadata_wr(frame_width, frame_height),
+		      DMA_ALIGNMENT) * 4 +
+		ALIGN(size_h265d_qp(frame_width, frame_height), DMA_ALIGNMENT);
+	if (is_opb)
+		vpss_lb_size = size_vpss_lb(frame_width, frame_height);
+
+	return ALIGN((_size + vpss_lb_size), DMA_ALIGNMENT);
+}
+
+static inline
+u32 size_vpxd_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+	return max_t(u32, ((frame_height + 15) >> 4) *
+		     MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+		     max_t(u32, ((frame_height + 31) >> 5) *
+			   MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+			   ((frame_height + 63) >> 6) *
+			   MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_vpxd_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+	return ((ALIGN(frame_width, 64) + 8) * 10 * 2);
+}
+
+static inline
+u32 size_vpxd_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+	return ((frame_width + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE;
+}
+
+static inline
+u32 size_vpxd_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+	return max_t(u32, ((frame_height + 15) >> 4) *
+		     MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+		     max_t(u32, ((frame_height + 31) >> 5) *
+			   MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+			   ((frame_height + 63) >> 6) *
+			   MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_vpxd_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+	return ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64,
+		BUFFER_ALIGNMENT_32_BYTES);
+}
+
+static inline
+u32 size_mp2d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+	return ((ALIGN(frame_width, 16) + 8) * 10 * 2);
+}
+
+static inline
+u32 size_vp9d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+	return (ALIGN(ALIGN(frame_width, 8), 64) + 8) * 10 * 2;
+}
+
+static inline
+u32 size_vp9d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+	return ((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 176);
+}
+
+static inline
+u32 size_vp9d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+	return (((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 64 * 8) + 256);
+}
+
+static inline
+u32 size_vp9d_qp(u32 frame_width, u32 frame_height)
+{
+	return size_h264d_qp(frame_width, frame_height);
+}
+
+static inline
+u32 hfi_iris3_vp9d_lb_size(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+	return ALIGN(size_vpxd_lb_fe_left_ctrl(frame_width, frame_height), DMA_ALIGNMENT) *
+		num_vpp_pipes +
+		ALIGN(size_vpxd_lb_se_left_ctrl(frame_width, frame_height), DMA_ALIGNMENT) *
+		num_vpp_pipes +
+		ALIGN(size_vp9d_lb_vsp_top(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_vpxd_lb_fe_top_ctrl(frame_width, frame_height), DMA_ALIGNMENT) +
+		2 * ALIGN(size_vpxd_lb_recon_dma_metadata_wr(frame_width, frame_height),
+			  DMA_ALIGNMENT) +
+		ALIGN(size_vpxd_lb_se_top_ctrl(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_vp9d_lb_pe_top_data(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_vp9d_lb_fe_top_data(frame_width, frame_height), DMA_ALIGNMENT) +
+		ALIGN(size_vp9d_qp(frame_width, frame_height), DMA_ALIGNMENT);
+}
+
+static inline
+u32 hfi_buffer_line_vp9d(u32 frame_width, u32 frame_height, u32 _yuv_bufcount_min, bool is_opb,
+			 u32 num_vpp_pipes)
+{
+	u32 vpss_lb_size = 0;
+	u32 _lb_size;
+
+	_lb_size = hfi_iris3_vp9d_lb_size(frame_width, frame_height, num_vpp_pipes);
+
+	if (is_opb)
+		vpss_lb_size = size_vpss_lb(frame_width, frame_height);
+
+	return _lb_size + vpss_lb_size + 4096;
+}
+
 static u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height,
 				 bool is_opb, u32 num_vpp_pipes)
 {
@@ -148,7 +458,14 @@ static u32 iris_vpu_dec_bin_size(struct iris_inst *inst)
 	u32 height = f->fmt.pix_mp.height;
 	u32 width = f->fmt.pix_mp.width;
 
-	return hfi_buffer_bin_h264d(width, height, num_vpp_pipes);
+	if (inst->codec == V4L2_PIX_FMT_H264)
+		return hfi_buffer_bin_h264d(width, height, num_vpp_pipes);
+	else if (inst->codec == V4L2_PIX_FMT_HEVC)
+		return hfi_buffer_bin_h265d(width, height, num_vpp_pipes);
+	else if (inst->codec == V4L2_PIX_FMT_VP9)
+		return hfi_buffer_bin_vp9d(width, height, num_vpp_pipes);
+
+	return 0;
 }
 
 static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
@@ -158,12 +475,24 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
 	u32 height = f->fmt.pix_mp.height;
 	u32 width = f->fmt.pix_mp.width;
 
-	return hfi_buffer_comv_h264d(width, height, num_comv);
+	if (inst->codec == V4L2_PIX_FMT_H264)
+		return hfi_buffer_comv_h264d(width, height, num_comv);
+	else if (inst->codec == V4L2_PIX_FMT_HEVC)
+		return hfi_buffer_comv_h265d(width, height, num_comv);
+
+	return 0;
 }
 
 static u32 iris_vpu_dec_persist_size(struct iris_inst *inst)
 {
-	return hfi_buffer_persist_h264d();
+	if (inst->codec == V4L2_PIX_FMT_H264)
+		return hfi_buffer_persist_h264d();
+	else if (inst->codec == V4L2_PIX_FMT_HEVC)
+		return hfi_buffer_persist_h265d(0);
+	else if (inst->codec == V4L2_PIX_FMT_VP9)
+		return hfi_buffer_persist_vp9d();
+
+	return 0;
 }
 
 static u32 iris_vpu_dec_dpb_size(struct iris_inst *inst)
@@ -181,7 +510,12 @@ static u32 iris_vpu_dec_non_comv_size(struct iris_inst *inst)
 	u32 height = f->fmt.pix_mp.height;
 	u32 width = f->fmt.pix_mp.width;
 
-	return hfi_buffer_non_comv_h264d(width, height, num_vpp_pipes);
+	if (inst->codec == V4L2_PIX_FMT_H264)
+		return hfi_buffer_non_comv_h264d(width, height, num_vpp_pipes);
+	else if (inst->codec == V4L2_PIX_FMT_HEVC)
+		return hfi_buffer_non_comv_h265d(width, height, num_vpp_pipes);
+
+	return 0;
 }
 
 static u32 iris_vpu_dec_line_size(struct iris_inst *inst)
@@ -191,11 +525,20 @@ static u32 iris_vpu_dec_line_size(struct iris_inst *inst)
 	u32 height = f->fmt.pix_mp.height;
 	u32 width = f->fmt.pix_mp.width;
 	bool is_opb = false;
+	u32 out_min_count = inst->buffers[BUF_OUTPUT].min_count;
 
 	if (iris_split_mode_enabled(inst))
 		is_opb = true;
 
-	return hfi_buffer_line_h264d(width, height, is_opb, num_vpp_pipes);
+	if (inst->codec == V4L2_PIX_FMT_H264)
+		return hfi_buffer_line_h264d(width, height, is_opb, num_vpp_pipes);
+	else if (inst->codec == V4L2_PIX_FMT_HEVC)
+		return hfi_buffer_line_h265d(width, height, is_opb, num_vpp_pipes);
+	else if (inst->codec == V4L2_PIX_FMT_VP9)
+		return hfi_buffer_line_vp9d(width, height, out_min_count, is_opb,
+			num_vpp_pipes);
+
+	return 0;
 }
 
 static u32 iris_vpu_dec_scratch1_size(struct iris_inst *inst)
@@ -205,6 +548,24 @@ static u32 iris_vpu_dec_scratch1_size(struct iris_inst *inst)
 		iris_vpu_dec_line_size(inst);
 }
 
+static int output_min_count(struct iris_inst *inst)
+{
+	int output_min_count = 4;
+
+	/* fw_min_count > 0 indicates reconfig event has already arrived */
+	if (inst->fw_min_count) {
+		if (iris_split_mode_enabled(inst) && inst->codec == V4L2_PIX_FMT_VP9)
+			return min_t(u32, 4, inst->fw_min_count);
+		else
+			return inst->fw_min_count;
+	}
+
+	if (inst->codec == V4L2_PIX_FMT_VP9)
+		output_min_count = 9;
+
+	return output_min_count;
+}
+
 struct iris_vpu_buf_type_handle {
 	enum iris_buffer_type type;
 	u32 (*handle)(struct iris_inst *inst);
@@ -238,6 +599,19 @@ int iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)
 	return size;
 }
 
+static u32 internal_buffer_count(struct iris_inst *inst,
+				 enum iris_buffer_type buffer_type)
+{
+	if (buffer_type == BUF_BIN || buffer_type == BUF_LINE ||
+	    buffer_type == BUF_PERSIST) {
+		return 1;
+	} else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
+		if (inst->codec == V4L2_PIX_FMT_H264 || inst->codec == V4L2_PIX_FMT_HEVC)
+			return 1;
+	}
+	return 0;
+}
+
 static inline int iris_vpu_dpb_count(struct iris_inst *inst)
 {
 	if (iris_split_mode_enabled(inst)) {
@@ -254,12 +628,13 @@ int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type
 	case BUF_INPUT:
 		return MIN_BUFFERS;
 	case BUF_OUTPUT:
-		return inst->fw_min_count;
+		return output_min_count(inst);
 	case BUF_BIN:
 	case BUF_COMV:
 	case BUF_NON_COMV:
 	case BUF_LINE:
 	case BUF_PERSIST:
+		return internal_buffer_count(inst, buffer_type);
 	case BUF_SCRATCH_1:
 		return 1; /* internal buffer count needed by firmware is 1 */
 	case BUF_DPB:
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
index 62af6ea6ba1f..2272f0c21683 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
@@ -13,6 +13,10 @@ struct iris_inst;
 #define DMA_ALIGNMENT			256
 
 #define NUM_HW_PIC_BUF			32
+#define LCU_MAX_SIZE_PELS 64
+#define LCU_MIN_SIZE_PELS 16
+#define HDR10_HIST_EXTRADATA_SIZE (4 * 1024)
+
 #define SIZE_HW_PIC(size_per_buf)	(NUM_HW_PIC_BUF * (size_per_buf))
 
 #define MAX_TILE_COLUMNS		32
@@ -28,11 +32,47 @@ struct iris_inst;
 #define SIZE_SLIST_BUF_H264		512
 #define H264_DISPLAY_BUF_SIZE		3328
 #define H264_NUM_FRM_INFO		66
-
-#define SIZE_SEI_USERDATA		4096
-
+#define H265_NUM_TILE_COL 32
+#define H265_NUM_TILE_ROW 12
+#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
+#define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32))
+
+#define NUM_SLIST_BUF_H265 (80 + 20)
+#define SIZE_SLIST_BUF_H265 (BIT(10))
+#define H265_DISPLAY_BUF_SIZE (3072)
+#define H265_NUM_FRM_INFO (48)
+
+#define VP9_NUM_FRAME_INFO_BUF 32
+#define VP9_NUM_PROBABILITY_TABLE_BUF (VP9_NUM_FRAME_INFO_BUF + 4)
+#define VP9_PROB_TABLE_SIZE (3840)
+#define VP9_FRAME_INFO_BUF_SIZE (6144)
+#define BUFFER_ALIGNMENT_32_BYTES 32
+#define CCE_TILE_OFFSET_SIZE ALIGN(32 * 4 * 4, BUFFER_ALIGNMENT_32_BYTES)
+#define MAX_SUPERFRAME_HEADER_LEN (34)
+#define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 64
+#define MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE 64
+#define MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE 64
+#define MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE (128 / 8)
+#define MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE (128 / 8)
+#define VP9_UDC_HEADER_BUF_SIZE	(3 * 128)
+
+#define SIZE_SEI_USERDATA			4096
+#define SIZE_DOLBY_RPU_METADATA (41 * 1024)
 #define H264_CABAC_HDR_RATIO_HD_TOT	1
 #define H264_CABAC_RES_RATIO_HD_TOT	3
+#define H265D_MAX_SLICE	1200
+#define SIZE_H265D_HW_PIC_T SIZE_H264D_HW_PIC_T
+#define H265_CABAC_HDR_RATIO_HD_TOT 2
+#define H265_CABAC_RES_RATIO_HD_TOT 2
+#define SIZE_H265D_VPP_CMD_PER_BUF (256)
+
+#define VPX_DECODER_FRAME_CONCURENCY_LVL (2)
+#define VPX_DECODER_FRAME_BIN_HDR_BUDGET 1
+#define VPX_DECODER_FRAME_BIN_RES_BUDGET 3
+#define VPX_DECODER_FRAME_BIN_DENOMINATOR 2
+
+#define VPX_DECODER_FRAME_BIN_RES_BUDGET_RATIO (3 / 2)
+
 #define SIZE_H264D_HW_PIC_T		(BIT(11))
 
 #define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE	64

-- 
2.34.1

[PATCH v3 23/23] media: iris: Add codec specific check for VP9 decoder drain handling

[Dikshita Agarwal]

Add a codec specific for the VP9 decoder to ensure that a non-null
buffer is sent to the firmware during drain. The firmware enforces a
check for VP9 decoder that the number of buffers queued and dequeued on
the output plane should match. When a null buffer is sent, the firmware
does not return a response for it, leading to a count mismatch and an
assertion failure from the firmware.

Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
 drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c  | 2 ++
 drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c | 4 ++++
 2 files changed, 6 insertions(+)

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
index 2e3f5a6b2ff1..5fc30d54af4d 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c
@@ -401,6 +401,8 @@ static int iris_hfi_gen1_session_drain(struct iris_inst *inst, u32 plane)
 	ip_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_EMPTY_BUFFER;
 	ip_pkt.shdr.session_id = inst->session_id;
 	ip_pkt.flags = HFI_BUFFERFLAG_EOS;
+	if (inst->codec == V4L2_PIX_FMT_VP9)
+		ip_pkt.packet_buffer = 0xdeadb000;
 
 	return iris_hfi_queue_cmd_write(inst->core, &ip_pkt, ip_pkt.shdr.hdr.size);
 }
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
index 926acee1f48c..8d1ce8a19a45 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c
@@ -348,6 +348,10 @@ static void iris_hfi_gen1_session_etb_done(struct iris_inst *inst, void *packet)
 	struct iris_buffer *buf = NULL;
 	bool found = false;
 
+	/* EOS buffer sent via drain won't be in v4l2 buffer list */
+	if (pkt->packet_buffer == 0xdeadb000)
+		return;
+
 	v4l2_m2m_for_each_src_buf_safe(m2m_ctx, m2m_buffer, n) {
 		buf = to_iris_buffer(&m2m_buffer->vb);
 		if (buf->index == pkt->input_tag) {

-- 
2.34.1

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[Neil Armstrong]

Hi,

On 01/05/2025 21:13, Dikshita Agarwal wrote:
> Hi All,
> 
> This patch series adds initial support for the HEVC(H.265) and VP9
> codecs in iris decoder. The objective of this work is to extend the
> decoder's capabilities to handle HEVC and VP9 codec streams,
> including necessary format handling and buffer management.
> In addition, the series also includes a set of fixes to address issues
> identified during testing of these additional codecs.
> 
> These patches also address the comments and feedback received from the
> RFC patches previously sent. I have made the necessary improvements
> based on the community's suggestions.
> 
> Changes in v3:
> - Introduced two wrappers with explicit names to handle destroy internal
> buffers (Nicolas)
> - Used sub state check instead of introducing new boolean (Vikash)
> - Addressed other comments (Vikash)
> - Reorderd patches to have all fixes patches first (Dmitry)
> - Link to v2: https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
> 
> Changes in v2:
> - Added Changes to make sure all buffers are released in session close
> (bryna)
> - Added tracking for flush responses to fix a timing issue.
> - Added a handling to fix timing issue in reconfig
> - Splitted patch 06/20 in two patches (Bryan)
> - Added missing fixes tag (bryan)
> - Updated fluster report (Nicolas)
> - Link to v1:
> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
> 
> Changes sinces RFC:
> - Added additional fixes to address issues identified during further
> testing.
> - Moved typo fix to a seperate patch [Neil]
> - Reordered the patches for better logical flow and clarity [Neil,
> Dmitry]
> - Added fixes tag wherever applicable [Neil, Dmitry]
> - Removed the default case in the switch statement for codecs [Bryan]
> - Replaced if-else statements with switch-case [Bryan]
> - Added comments for mbpf [Bryan]
> - RFC:
> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
> 
> This patch series depends on [1] & [2]
> [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
> [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
> 
> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
> the existing H264 decoder functionality remains uneffected.
> 
> Note: 1 of the fluster compliance test is fixed with firmware [3]
> [3]:
> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
> 
> The result of fluster test on SM8550:
>   131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>   GStreamer-H.265-V4L2-Gst1.0.
>   The failing test case:
>   - 10 testcases failed due to unsupported 10 bit format.
>     - DBLK_A_MAIN10_VIXS_4
>     - INITQP_B_Main10_Sony_1
>     - TSUNEQBD_A_MAIN10_Technicolor_2
>     - WP_A_MAIN10_Toshiba_3
>     - WP_MAIN10_B_Toshiba_3
>     - WPP_A_ericsson_MAIN10_2
>     - WPP_B_ericsson_MAIN10_2
>     - WPP_C_ericsson_MAIN10_2
>     - WPP_E_ericsson_MAIN10_2
>     - WPP_F_ericsson_MAIN10_2
>   - 4 testcase failed due to unsupported resolution
>     - PICSIZE_A_Bossen_1
>     - PICSIZE_B_Bossen_1
>     - WPP_D_ericsson_MAIN10_2
>     - WPP_D_ericsson_MAIN_2
>   - 2 testcase failed due to CRC mismatch
>     - RAP_A_docomo_6
>     - RAP_B_Bossen_2
>     - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>       Analysis - First few frames in this discarded by firmware and are
>       sent to driver with 0 filled length. Driver send such buffers to
>       client with timestamp 0 and payload set to 0 and
>       make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>       dropped by GST. But instead, the first frame displayed as green
>       frame and when a valid buffer is sent to client later with same 0
>       timestamp, its dropped, leading to CRC mismatch for first frame.
> 
>   235/305 testcases passed while testing VP9-TEST-VECTORS with
>   GStreamer-VP9-V4L2-Gst1.0.
>   The failing test case:
>   - 64 testcases failed due to unsupported resolution
>     - vp90-2-02-size-08x08.webm
>     - vp90-2-02-size-08x10.webm
>     - vp90-2-02-size-08x16.webm
>     - vp90-2-02-size-08x18.webm
>     - vp90-2-02-size-08x32.webm
>     - vp90-2-02-size-08x34.webm
>     - vp90-2-02-size-08x64.webm
>     - vp90-2-02-size-08x66.webm
>     - vp90-2-02-size-10x08.webm
>     - vp90-2-02-size-10x10.webm
>     - vp90-2-02-size-10x16.webm
>     - vp90-2-02-size-10x18.webm
>     - vp90-2-02-size-10x32.webm
>     - vp90-2-02-size-10x34.webm
>     - vp90-2-02-size-10x64.webm
>     - vp90-2-02-size-10x66.webm
>     - vp90-2-02-size-16x08.webm
>     - vp90-2-02-size-16x10.webm
>     - vp90-2-02-size-16x16.webm
>     - vp90-2-02-size-16x18.webm
>     - vp90-2-02-size-16x32.webm
>     - vp90-2-02-size-16x34.webm
>     - vp90-2-02-size-16x64.webm
>     - vp90-2-02-size-16x66.webm
>     - vp90-2-02-size-18x08.webm
>     - vp90-2-02-size-18x10.webm
>     - vp90-2-02-size-18x16.webm
>     - vp90-2-02-size-18x18.webm
>     - vp90-2-02-size-18x32.webm
>     - vp90-2-02-size-18x34.webm
>     - vp90-2-02-size-18x64.webm
>     - vp90-2-02-size-18x66.webm
>     - vp90-2-02-size-32x08.webm
>     - vp90-2-02-size-32x10.webm
>     - vp90-2-02-size-32x16.webm
>     - vp90-2-02-size-32x18.webm
>     - vp90-2-02-size-32x32.webm
>     - vp90-2-02-size-32x34.webm
>     - vp90-2-02-size-32x64.webm
>     - vp90-2-02-size-32x66.webm
>     - vp90-2-02-size-34x08.webm
>     - vp90-2-02-size-34x10.webm
>     - vp90-2-02-size-34x16.webm
>     - vp90-2-02-size-34x18.webm
>     - vp90-2-02-size-34x32.webm
>     - vp90-2-02-size-34x34.webm
>     - vp90-2-02-size-34x64.webm
>     - vp90-2-02-size-34x66.webm
>     - vp90-2-02-size-64x08.webm
>     - vp90-2-02-size-64x10.webm
>     - vp90-2-02-size-64x16.webm
>     - vp90-2-02-size-64x18.webm
>     - vp90-2-02-size-64x32.webm
>     - vp90-2-02-size-64x34.webm
>     - vp90-2-02-size-64x64.webm
>     - vp90-2-02-size-64x66.webm
>     - vp90-2-02-size-66x08.webm
>     - vp90-2-02-size-66x10.webm
>     - vp90-2-02-size-66x16.webm
>     - vp90-2-02-size-66x18.webm
>     - vp90-2-02-size-66x32.webm
>     - vp90-2-02-size-66x34.webm
>     - vp90-2-02-size-66x64.webm
>     - vp90-2-02-size-66x66.webm
>   - 2 testcases failed due to unsupported format
>     - vp91-2-04-yuv422.webm
>     - vp91-2-04-yuv444.webm
>   - 1 testcase failed with CRC mismatch
>     - vp90-2-22-svc_1280x720_3.ivf
>     - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>   - 2 testcase failed due to unsupported resolution after sequence change
>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>   - 1 testcase failed due to unsupported stream
>     - vp90-2-16-intra-only.webm
> 
> The result of fluster test on SM8250:
>   133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>   GStreamer-H.265-V4L2-Gst1.0.
>   The failing test case:
>   - 10 testcases failed due to unsupported 10 bit format.
>     - DBLK_A_MAIN10_VIXS_4
>     - INITQP_B_Main10_Sony_1
>     - TSUNEQBD_A_MAIN10_Technicolor_2
>     - WP_A_MAIN10_Toshiba_3
>     - WP_MAIN10_B_Toshiba_3
>     - WPP_A_ericsson_MAIN10_2
>     - WPP_B_ericsson_MAIN10_2
>     - WPP_C_ericsson_MAIN10_2
>     - WPP_E_ericsson_MAIN10_2
>     - WPP_F_ericsson_MAIN10_2
>   - 4 testcase failed due to unsupported resolution
>     - PICSIZE_A_Bossen_1
>     - PICSIZE_B_Bossen_1
>     - WPP_D_ericsson_MAIN10_2
>     - WPP_D_ericsson_MAIN_2
> 
>   232/305 testcases passed while testing VP9-TEST-VECTORS with
>   GStreamer-VP9-V4L2-Gst1.0.
>   The failing test case:
>   - 64 testcases failed due to unsupported resolution
>     - vp90-2-02-size-08x08.webm
>     - vp90-2-02-size-08x10.webm
>     - vp90-2-02-size-08x16.webm
>     - vp90-2-02-size-08x18.webm
>     - vp90-2-02-size-08x32.webm
>     - vp90-2-02-size-08x34.webm
>     - vp90-2-02-size-08x64.webm
>     - vp90-2-02-size-08x66.webm
>     - vp90-2-02-size-10x08.webm
>     - vp90-2-02-size-10x10.webm
>     - vp90-2-02-size-10x16.webm
>     - vp90-2-02-size-10x18.webm
>     - vp90-2-02-size-10x32.webm
>     - vp90-2-02-size-10x34.webm
>     - vp90-2-02-size-10x64.webm
>     - vp90-2-02-size-10x66.webm
>     - vp90-2-02-size-16x08.webm
>     - vp90-2-02-size-16x10.webm
>     - vp90-2-02-size-16x16.webm
>     - vp90-2-02-size-16x18.webm
>     - vp90-2-02-size-16x32.webm
>     - vp90-2-02-size-16x34.webm
>     - vp90-2-02-size-16x64.webm
>     - vp90-2-02-size-16x66.webm
>     - vp90-2-02-size-18x08.webm
>     - vp90-2-02-size-18x10.webm
>     - vp90-2-02-size-18x16.webm
>     - vp90-2-02-size-18x18.webm
>     - vp90-2-02-size-18x32.webm
>     - vp90-2-02-size-18x34.webm
>     - vp90-2-02-size-18x64.webm
>     - vp90-2-02-size-18x66.webm
>     - vp90-2-02-size-32x08.webm
>     - vp90-2-02-size-32x10.webm
>     - vp90-2-02-size-32x16.webm
>     - vp90-2-02-size-32x18.webm
>     - vp90-2-02-size-32x32.webm
>     - vp90-2-02-size-32x34.webm
>     - vp90-2-02-size-32x64.webm
>     - vp90-2-02-size-32x66.webm
>     - vp90-2-02-size-34x08.webm
>     - vp90-2-02-size-34x10.webm
>     - vp90-2-02-size-34x16.webm
>     - vp90-2-02-size-34x18.webm
>     - vp90-2-02-size-34x32.webm
>     - vp90-2-02-size-34x34.webm
>     - vp90-2-02-size-34x64.webm
>     - vp90-2-02-size-34x66.webm
>     - vp90-2-02-size-64x08.webm
>     - vp90-2-02-size-64x10.webm
>     - vp90-2-02-size-64x16.webm
>     - vp90-2-02-size-64x18.webm
>     - vp90-2-02-size-64x32.webm
>     - vp90-2-02-size-64x34.webm
>     - vp90-2-02-size-64x64.webm
>     - vp90-2-02-size-64x66.webm
>     - vp90-2-02-size-66x08.webm
>     - vp90-2-02-size-66x10.webm
>     - vp90-2-02-size-66x16.webm
>     - vp90-2-02-size-66x18.webm
>     - vp90-2-02-size-66x32.webm
>     - vp90-2-02-size-66x34.webm
>     - vp90-2-02-size-66x64.webm
>     - vp90-2-02-size-66x66.webm
>   - 2 testcases failed due to unsupported format
>     - vp91-2-04-yuv422.webm
>     - vp91-2-04-yuv444.webm
>   - 1 testcase failed with CRC mismatch
>     - vp90-2-22-svc_1280x720_3.ivf
>     - Bug raised:
> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>   - 5 testcase failed due to unsupported resolution after sequence change
>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>     - vp90-2-21-resize_inter_320x240_5_1-2.webm
>     - vp90-2-21-resize_inter_320x240_7_1-2.webm
>     - vp90-2-18-resize.ivf
>   - 1 testcase failed with CRC mismatch
>     - vp90-2-16-intra-only.webm
>     Analysis: First few frames are marked by firmware as NO_SHOW frame.
>     Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>     Such buffers should be dropped by GST. But instead, the first frame
>     is being displayed and when a valid buffer is sent to client later
>     with same timestamp, its dropped, leading to CRC mismatch for first
>     frame.
> 
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
> Dikshita Agarwal (23):
>        media: iris: Skip destroying internal buffer if not dequeued
>        media: iris: Update CAPTURE format info based on OUTPUT format
>        media: iris: Avoid updating frame size to firmware during reconfig
>        media: iris: Drop port check for session property response
>        media: iris: Prevent HFI queue writes when core is in deinit state
>        media: iris: Remove deprecated property setting to firmware
>        media: iris: Fix missing function pointer initialization
>        media: iris: Fix NULL pointer dereference
>        media: iris: Fix typo in depth variable
>        media: iris: Track flush responses to prevent premature completion
>        media: iris: Fix buffer preparation failure during resolution change
>        media: iris: Add handling for corrupt and drop frames
>        media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length
>        media: iris: Add handling for no show frames
>        media: iris: Improve last flag handling
>        media: iris: Skip flush on first sequence change
>        media: iris: Remove redundant buffer count check in stream off
>        media: iris: Add a comment to explain usage of MBPS
>        media: iris: Add HEVC and VP9 formats for decoder
>        media: iris: Add platform capabilities for HEVC and VP9 decoders
>        media: iris: Set mandatory properties for HEVC and VP9 decoders.
>        media: iris: Add internal buffer calculation for HEVC and VP9 decoders
>        media: iris: Add codec specific check for VP9 decoder drain handling
> 
>   drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>   drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>   drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>   drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>   .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>   .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>   .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>   .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>   .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>   .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>   drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>   drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>   .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>   .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>   .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>   .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>   drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>   drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>   drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>   drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>   drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>   drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>   drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++-
>   drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>   24 files changed, 1160 insertions(+), 211 deletions(-)
> ---
> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
> prerequisite-message-id: <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
> prerequisite-message-id: <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
> 
> Best regards,

HEVC & VP9 works fine on HDK8550.

But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
[   44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown
[   44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources
[   44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources
[   44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown
[   44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources
[   44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown
[   44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources
[   44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown
[   44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources
[   44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown
[   44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state
[   44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state
[   44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown
[   45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state
[   45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown
[   45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state
[   45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown
[   45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state
[   45.105459] ------------[ cut here ]------------
[   45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common]
while VP9 works fine.

Is it a firmware issue ?

I've added:
========================================><======================================
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index d3026b2bcb70..8c0ab00ab435 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
         .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
         .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
         .vpu_ops = &iris_vpu33_ops,
-       .set_preset_registers = iris_set_sm8550_preset_registers,
+       .set_preset_registers = iris_set_sm8650_preset_registers,
         .icc_tbl = sm8550_icc_table,
         .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
         .clk_rst_tbl = sm8650_clk_reset_table,
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
index 75e9d572e788..9e2d23f12f75 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
@@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };

  static const char * const sm8650_controller_reset_table[] = { "xo" };

+static void iris_set_sm8650_preset_registers(struct iris_core *core)
+{
+       writel(0x0, core->reg_base + 0xB0088);
+       writel(0x33332222, core->reg_base + 0x13030);
+       writel(0x44444444, core->reg_base + 0x13034);
+       writel(0x1022, core->reg_base + 0x13038);
+       writel(0x0, core->reg_base + 0x13040);
+       writel(0xFFFF, core->reg_base + 0x13048);
+       writel(0x33332222, core->reg_base + 0x13430);
+       writel(0x44444444, core->reg_base + 0x13434);
+       writel(0x1022, core->reg_base + 0x13438);
+       writel(0x0, core->reg_base + 0x13440);
+       writel(0xFFFF, core->reg_base + 0x13448);
+       writel(0x99, core->reg_base + 0xA013C);
+}
+
  #endif
========================================><======================================
and no change, error still occurs with HEVC decoding.

Thanks,
Neil

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[Dikshita Agarwal]

On 5/2/2025 12:55 PM, Neil Armstrong wrote:
> Hi,
> 
> On 01/05/2025 21:13, Dikshita Agarwal wrote:
>> Hi All,
>>
>> This patch series adds initial support for the HEVC(H.265) and VP9
>> codecs in iris decoder. The objective of this work is to extend the
>> decoder's capabilities to handle HEVC and VP9 codec streams,
>> including necessary format handling and buffer management.
>> In addition, the series also includes a set of fixes to address issues
>> identified during testing of these additional codecs.
>>
>> These patches also address the comments and feedback received from the
>> RFC patches previously sent. I have made the necessary improvements
>> based on the community's suggestions.
>>
>> Changes in v3:
>> - Introduced two wrappers with explicit names to handle destroy internal
>> buffers (Nicolas)
>> - Used sub state check instead of introducing new boolean (Vikash)
>> - Addressed other comments (Vikash)
>> - Reorderd patches to have all fixes patches first (Dmitry)
>> - Link to v2:
>> https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
>>
>> Changes in v2:
>> - Added Changes to make sure all buffers are released in session close
>> (bryna)
>> - Added tracking for flush responses to fix a timing issue.
>> - Added a handling to fix timing issue in reconfig
>> - Splitted patch 06/20 in two patches (Bryan)
>> - Added missing fixes tag (bryan)
>> - Updated fluster report (Nicolas)
>> - Link to v1:
>> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
>>
>> Changes sinces RFC:
>> - Added additional fixes to address issues identified during further
>> testing.
>> - Moved typo fix to a seperate patch [Neil]
>> - Reordered the patches for better logical flow and clarity [Neil,
>> Dmitry]
>> - Added fixes tag wherever applicable [Neil, Dmitry]
>> - Removed the default case in the switch statement for codecs [Bryan]
>> - Replaced if-else statements with switch-case [Bryan]
>> - Added comments for mbpf [Bryan]
>> - RFC:
>> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
>>
>> This patch series depends on [1] & [2]
>> [1]
>> https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
>> [2]
>> https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
>>
>> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
>> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
>> the existing H264 decoder functionality remains uneffected.
>>
>> Note: 1 of the fluster compliance test is fixed with firmware [3]
>> [3]:
>> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
>>
>> The result of fluster test on SM8550:
>>   131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>   GStreamer-H.265-V4L2-Gst1.0.
>>   The failing test case:
>>   - 10 testcases failed due to unsupported 10 bit format.
>>     - DBLK_A_MAIN10_VIXS_4
>>     - INITQP_B_Main10_Sony_1
>>     - TSUNEQBD_A_MAIN10_Technicolor_2
>>     - WP_A_MAIN10_Toshiba_3
>>     - WP_MAIN10_B_Toshiba_3
>>     - WPP_A_ericsson_MAIN10_2
>>     - WPP_B_ericsson_MAIN10_2
>>     - WPP_C_ericsson_MAIN10_2
>>     - WPP_E_ericsson_MAIN10_2
>>     - WPP_F_ericsson_MAIN10_2
>>   - 4 testcase failed due to unsupported resolution
>>     - PICSIZE_A_Bossen_1
>>     - PICSIZE_B_Bossen_1
>>     - WPP_D_ericsson_MAIN10_2
>>     - WPP_D_ericsson_MAIN_2
>>   - 2 testcase failed due to CRC mismatch
>>     - RAP_A_docomo_6
>>     - RAP_B_Bossen_2
>>     - BUG reported:
>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>>       Analysis - First few frames in this discarded by firmware and are
>>       sent to driver with 0 filled length. Driver send such buffers to
>>       client with timestamp 0 and payload set to 0 and
>>       make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>>       dropped by GST. But instead, the first frame displayed as green
>>       frame and when a valid buffer is sent to client later with same 0
>>       timestamp, its dropped, leading to CRC mismatch for first frame.
>>
>>   235/305 testcases passed while testing VP9-TEST-VECTORS with
>>   GStreamer-VP9-V4L2-Gst1.0.
>>   The failing test case:
>>   - 64 testcases failed due to unsupported resolution
>>     - vp90-2-02-size-08x08.webm
>>     - vp90-2-02-size-08x10.webm
>>     - vp90-2-02-size-08x16.webm
>>     - vp90-2-02-size-08x18.webm
>>     - vp90-2-02-size-08x32.webm
>>     - vp90-2-02-size-08x34.webm
>>     - vp90-2-02-size-08x64.webm
>>     - vp90-2-02-size-08x66.webm
>>     - vp90-2-02-size-10x08.webm
>>     - vp90-2-02-size-10x10.webm
>>     - vp90-2-02-size-10x16.webm
>>     - vp90-2-02-size-10x18.webm
>>     - vp90-2-02-size-10x32.webm
>>     - vp90-2-02-size-10x34.webm
>>     - vp90-2-02-size-10x64.webm
>>     - vp90-2-02-size-10x66.webm
>>     - vp90-2-02-size-16x08.webm
>>     - vp90-2-02-size-16x10.webm
>>     - vp90-2-02-size-16x16.webm
>>     - vp90-2-02-size-16x18.webm
>>     - vp90-2-02-size-16x32.webm
>>     - vp90-2-02-size-16x34.webm
>>     - vp90-2-02-size-16x64.webm
>>     - vp90-2-02-size-16x66.webm
>>     - vp90-2-02-size-18x08.webm
>>     - vp90-2-02-size-18x10.webm
>>     - vp90-2-02-size-18x16.webm
>>     - vp90-2-02-size-18x18.webm
>>     - vp90-2-02-size-18x32.webm
>>     - vp90-2-02-size-18x34.webm
>>     - vp90-2-02-size-18x64.webm
>>     - vp90-2-02-size-18x66.webm
>>     - vp90-2-02-size-32x08.webm
>>     - vp90-2-02-size-32x10.webm
>>     - vp90-2-02-size-32x16.webm
>>     - vp90-2-02-size-32x18.webm
>>     - vp90-2-02-size-32x32.webm
>>     - vp90-2-02-size-32x34.webm
>>     - vp90-2-02-size-32x64.webm
>>     - vp90-2-02-size-32x66.webm
>>     - vp90-2-02-size-34x08.webm
>>     - vp90-2-02-size-34x10.webm
>>     - vp90-2-02-size-34x16.webm
>>     - vp90-2-02-size-34x18.webm
>>     - vp90-2-02-size-34x32.webm
>>     - vp90-2-02-size-34x34.webm
>>     - vp90-2-02-size-34x64.webm
>>     - vp90-2-02-size-34x66.webm
>>     - vp90-2-02-size-64x08.webm
>>     - vp90-2-02-size-64x10.webm
>>     - vp90-2-02-size-64x16.webm
>>     - vp90-2-02-size-64x18.webm
>>     - vp90-2-02-size-64x32.webm
>>     - vp90-2-02-size-64x34.webm
>>     - vp90-2-02-size-64x64.webm
>>     - vp90-2-02-size-64x66.webm
>>     - vp90-2-02-size-66x08.webm
>>     - vp90-2-02-size-66x10.webm
>>     - vp90-2-02-size-66x16.webm
>>     - vp90-2-02-size-66x18.webm
>>     - vp90-2-02-size-66x32.webm
>>     - vp90-2-02-size-66x34.webm
>>     - vp90-2-02-size-66x64.webm
>>     - vp90-2-02-size-66x66.webm
>>   - 2 testcases failed due to unsupported format
>>     - vp91-2-04-yuv422.webm
>>     - vp91-2-04-yuv444.webm
>>   - 1 testcase failed with CRC mismatch
>>     - vp90-2-22-svc_1280x720_3.ivf
>>     - Bug reported:
>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>   - 2 testcase failed due to unsupported resolution after sequence change
>>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>   - 1 testcase failed due to unsupported stream
>>     - vp90-2-16-intra-only.webm
>>
>> The result of fluster test on SM8250:
>>   133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>   GStreamer-H.265-V4L2-Gst1.0.
>>   The failing test case:
>>   - 10 testcases failed due to unsupported 10 bit format.
>>     - DBLK_A_MAIN10_VIXS_4
>>     - INITQP_B_Main10_Sony_1
>>     - TSUNEQBD_A_MAIN10_Technicolor_2
>>     - WP_A_MAIN10_Toshiba_3
>>     - WP_MAIN10_B_Toshiba_3
>>     - WPP_A_ericsson_MAIN10_2
>>     - WPP_B_ericsson_MAIN10_2
>>     - WPP_C_ericsson_MAIN10_2
>>     - WPP_E_ericsson_MAIN10_2
>>     - WPP_F_ericsson_MAIN10_2
>>   - 4 testcase failed due to unsupported resolution
>>     - PICSIZE_A_Bossen_1
>>     - PICSIZE_B_Bossen_1
>>     - WPP_D_ericsson_MAIN10_2
>>     - WPP_D_ericsson_MAIN_2
>>
>>   232/305 testcases passed while testing VP9-TEST-VECTORS with
>>   GStreamer-VP9-V4L2-Gst1.0.
>>   The failing test case:
>>   - 64 testcases failed due to unsupported resolution
>>     - vp90-2-02-size-08x08.webm
>>     - vp90-2-02-size-08x10.webm
>>     - vp90-2-02-size-08x16.webm
>>     - vp90-2-02-size-08x18.webm
>>     - vp90-2-02-size-08x32.webm
>>     - vp90-2-02-size-08x34.webm
>>     - vp90-2-02-size-08x64.webm
>>     - vp90-2-02-size-08x66.webm
>>     - vp90-2-02-size-10x08.webm
>>     - vp90-2-02-size-10x10.webm
>>     - vp90-2-02-size-10x16.webm
>>     - vp90-2-02-size-10x18.webm
>>     - vp90-2-02-size-10x32.webm
>>     - vp90-2-02-size-10x34.webm
>>     - vp90-2-02-size-10x64.webm
>>     - vp90-2-02-size-10x66.webm
>>     - vp90-2-02-size-16x08.webm
>>     - vp90-2-02-size-16x10.webm
>>     - vp90-2-02-size-16x16.webm
>>     - vp90-2-02-size-16x18.webm
>>     - vp90-2-02-size-16x32.webm
>>     - vp90-2-02-size-16x34.webm
>>     - vp90-2-02-size-16x64.webm
>>     - vp90-2-02-size-16x66.webm
>>     - vp90-2-02-size-18x08.webm
>>     - vp90-2-02-size-18x10.webm
>>     - vp90-2-02-size-18x16.webm
>>     - vp90-2-02-size-18x18.webm
>>     - vp90-2-02-size-18x32.webm
>>     - vp90-2-02-size-18x34.webm
>>     - vp90-2-02-size-18x64.webm
>>     - vp90-2-02-size-18x66.webm
>>     - vp90-2-02-size-32x08.webm
>>     - vp90-2-02-size-32x10.webm
>>     - vp90-2-02-size-32x16.webm
>>     - vp90-2-02-size-32x18.webm
>>     - vp90-2-02-size-32x32.webm
>>     - vp90-2-02-size-32x34.webm
>>     - vp90-2-02-size-32x64.webm
>>     - vp90-2-02-size-32x66.webm
>>     - vp90-2-02-size-34x08.webm
>>     - vp90-2-02-size-34x10.webm
>>     - vp90-2-02-size-34x16.webm
>>     - vp90-2-02-size-34x18.webm
>>     - vp90-2-02-size-34x32.webm
>>     - vp90-2-02-size-34x34.webm
>>     - vp90-2-02-size-34x64.webm
>>     - vp90-2-02-size-34x66.webm
>>     - vp90-2-02-size-64x08.webm
>>     - vp90-2-02-size-64x10.webm
>>     - vp90-2-02-size-64x16.webm
>>     - vp90-2-02-size-64x18.webm
>>     - vp90-2-02-size-64x32.webm
>>     - vp90-2-02-size-64x34.webm
>>     - vp90-2-02-size-64x64.webm
>>     - vp90-2-02-size-64x66.webm
>>     - vp90-2-02-size-66x08.webm
>>     - vp90-2-02-size-66x10.webm
>>     - vp90-2-02-size-66x16.webm
>>     - vp90-2-02-size-66x18.webm
>>     - vp90-2-02-size-66x32.webm
>>     - vp90-2-02-size-66x34.webm
>>     - vp90-2-02-size-66x64.webm
>>     - vp90-2-02-size-66x66.webm
>>   - 2 testcases failed due to unsupported format
>>     - vp91-2-04-yuv422.webm
>>     - vp91-2-04-yuv444.webm
>>   - 1 testcase failed with CRC mismatch
>>     - vp90-2-22-svc_1280x720_3.ivf
>>     - Bug raised:
>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>   - 5 testcase failed due to unsupported resolution after sequence change
>>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>     - vp90-2-21-resize_inter_320x240_5_1-2.webm
>>     - vp90-2-21-resize_inter_320x240_7_1-2.webm
>>     - vp90-2-18-resize.ivf
>>   - 1 testcase failed with CRC mismatch
>>     - vp90-2-16-intra-only.webm
>>     Analysis: First few frames are marked by firmware as NO_SHOW frame.
>>     Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>>     Such buffers should be dropped by GST. But instead, the first frame
>>     is being displayed and when a valid buffer is sent to client later
>>     with same timestamp, its dropped, leading to CRC mismatch for first
>>     frame.
>>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>> Dikshita Agarwal (23):
>>        media: iris: Skip destroying internal buffer if not dequeued
>>        media: iris: Update CAPTURE format info based on OUTPUT format
>>        media: iris: Avoid updating frame size to firmware during reconfig
>>        media: iris: Drop port check for session property response
>>        media: iris: Prevent HFI queue writes when core is in deinit state
>>        media: iris: Remove deprecated property setting to firmware
>>        media: iris: Fix missing function pointer initialization
>>        media: iris: Fix NULL pointer dereference
>>        media: iris: Fix typo in depth variable
>>        media: iris: Track flush responses to prevent premature completion
>>        media: iris: Fix buffer preparation failure during resolution change
>>        media: iris: Add handling for corrupt and drop frames
>>        media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled
>> length
>>        media: iris: Add handling for no show frames
>>        media: iris: Improve last flag handling
>>        media: iris: Skip flush on first sequence change
>>        media: iris: Remove redundant buffer count check in stream off
>>        media: iris: Add a comment to explain usage of MBPS
>>        media: iris: Add HEVC and VP9 formats for decoder
>>        media: iris: Add platform capabilities for HEVC and VP9 decoders
>>        media: iris: Set mandatory properties for HEVC and VP9 decoders.
>>        media: iris: Add internal buffer calculation for HEVC and VP9
>> decoders
>>        media: iris: Add codec specific check for VP9 decoder drain handling
>>
>>   drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>>   drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>>   drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>>   drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>>   .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>>   .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>>   .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>>   .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>>   .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>>   .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>>   drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>>   drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>>   .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>>   .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>>   .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>>   .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>>   drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>>   drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>>   drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>>   drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>>   drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>>   drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>>   drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397
>> ++++++++++++++++++++-
>>   drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>>   24 files changed, 1160 insertions(+), 211 deletions(-)
>> ---
>> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
>> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
>> prerequisite-message-id:
>> <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
>> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
>> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
>> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
>> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
>> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
>> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
>> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
>> prerequisite-message-id:
>> <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
>> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
>> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
>> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
>> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
>> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
>>
>> Best regards,
> 
> HEVC & VP9 works fine on HDK8550.
> 
> But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
> [   44.741670] qcom-iris aa00000.video-codec: session error received
> 0x1000005: unknown
> [   44.755724] qcom-iris aa00000.video-codec: session error received
> 0x4000005: insufficient resources
> [   44.776462] qcom-iris aa00000.video-codec: session error received
> 0x4000005: insufficient resources
> [   44.797179] qcom-iris aa00000.video-codec: session error received
> 0x1000005: unknown
> [   44.816630] qcom-iris aa00000.video-codec: session error received
> 0x4000005: insufficient resources
> [   44.837387] qcom-iris aa00000.video-codec: session error received
> 0x1000005: unknown
> [   44.856812] qcom-iris aa00000.video-codec: session error received
> 0x4000005: insufficient resources
> [   44.877576] qcom-iris aa00000.video-codec: session error received
> 0x1000005: unknown
> [   44.897000] qcom-iris aa00000.video-codec: session error received
> 0x4000005: insufficient resources
> [   44.917801] qcom-iris aa00000.video-codec: session error received
> 0x1000009: unknown
> [   44.937254] qcom-iris aa00000.video-codec: session error received
> 0x4000004: invalid operation for current state
> [   44.959128] qcom-iris aa00000.video-codec: session error received
> 0x4000004: invalid operation for current state
> [   44.981025] qcom-iris aa00000.video-codec: session error received
> 0x1000009: unknown
> [   45.000459] qcom-iris aa00000.video-codec: session error received
> 0x4000004: invalid operation for current state
> [   45.022376] qcom-iris aa00000.video-codec: session error received
> 0x1000009: unknown
> [   45.041816] qcom-iris aa00000.video-codec: session error received
> 0x4000004: invalid operation for current state
> [   45.063736] qcom-iris aa00000.video-codec: session error received
> 0x1000009: unknown
> [   45.083167] qcom-iris aa00000.video-codec: session error received
> 0x4000004: invalid operation for current state
> [   45.105459] ------------[ cut here ]------------
> [   45.121152] WARNING: CPU: 6 PID: 573 at
> drivers/media/common/videobuf2/videobuf2-core.c:1827
> vb2_start_streaming+0x100/0x178 [videobuf2_common]
> while VP9 works fine.
> 
> Is it a firmware issue ?
> 
Looks like resources set to firmware are not sufficient.
I suspect, internal buffers set to firmware are less than what it requires,
this can change for different VPUs. Pls check if there is any difference in
internal buffer calculations between vpu3 and vpu33.

Thanks,
Dikshita
> I've added:
> ========================================><======================================
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> index d3026b2bcb70..8c0ab00ab435 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
>         .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
>         .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
>         .vpu_ops = &iris_vpu33_ops,
> -       .set_preset_registers = iris_set_sm8550_preset_registers,
> +       .set_preset_registers = iris_set_sm8650_preset_registers,
>         .icc_tbl = sm8550_icc_table,
>         .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
>         .clk_rst_tbl = sm8650_clk_reset_table,
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
> b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
> index 75e9d572e788..9e2d23f12f75 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
> @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = {
> "bus", "core" };
> 
>  static const char * const sm8650_controller_reset_table[] = { "xo" };
> 
> +static void iris_set_sm8650_preset_registers(struct iris_core *core)
> +{
> +       writel(0x0, core->reg_base + 0xB0088);
> +       writel(0x33332222, core->reg_base + 0x13030);
> +       writel(0x44444444, core->reg_base + 0x13034);
> +       writel(0x1022, core->reg_base + 0x13038);
> +       writel(0x0, core->reg_base + 0x13040);
> +       writel(0xFFFF, core->reg_base + 0x13048);
> +       writel(0x33332222, core->reg_base + 0x13430);
> +       writel(0x44444444, core->reg_base + 0x13434);
> +       writel(0x1022, core->reg_base + 0x13438);
> +       writel(0x0, core->reg_base + 0x13440);
> +       writel(0xFFFF, core->reg_base + 0x13448);
> +       writel(0x99, core->reg_base + 0xA013C);
> +}
> +
>  #endif
> ========================================><======================================
> and no change, error still occurs with HEVC decoding.
> 
> Thanks,
> Neil

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[Vikash Garodia]

On 5/2/2025 1:04 PM, Dikshita Agarwal wrote:
> 
> 
> On 5/2/2025 12:55 PM, Neil Armstrong wrote:
>> Hi,
>>
>> On 01/05/2025 21:13, Dikshita Agarwal wrote:
>>> Hi All,
>>>
>>> This patch series adds initial support for the HEVC(H.265) and VP9
>>> codecs in iris decoder. The objective of this work is to extend the
>>> decoder's capabilities to handle HEVC and VP9 codec streams,
>>> including necessary format handling and buffer management.
>>> In addition, the series also includes a set of fixes to address issues
>>> identified during testing of these additional codecs.
>>>
>>> These patches also address the comments and feedback received from the
>>> RFC patches previously sent. I have made the necessary improvements
>>> based on the community's suggestions.
>>>
>>> Changes in v3:
>>> - Introduced two wrappers with explicit names to handle destroy internal
>>> buffers (Nicolas)
>>> - Used sub state check instead of introducing new boolean (Vikash)
>>> - Addressed other comments (Vikash)
>>> - Reorderd patches to have all fixes patches first (Dmitry)
>>> - Link to v2:
>>> https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
>>>
>>> Changes in v2:
>>> - Added Changes to make sure all buffers are released in session close
>>> (bryna)
>>> - Added tracking for flush responses to fix a timing issue.
>>> - Added a handling to fix timing issue in reconfig
>>> - Splitted patch 06/20 in two patches (Bryan)
>>> - Added missing fixes tag (bryan)
>>> - Updated fluster report (Nicolas)
>>> - Link to v1:
>>> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
>>>
>>> Changes sinces RFC:
>>> - Added additional fixes to address issues identified during further
>>> testing.
>>> - Moved typo fix to a seperate patch [Neil]
>>> - Reordered the patches for better logical flow and clarity [Neil,
>>> Dmitry]
>>> - Added fixes tag wherever applicable [Neil, Dmitry]
>>> - Removed the default case in the switch statement for codecs [Bryan]
>>> - Replaced if-else statements with switch-case [Bryan]
>>> - Added comments for mbpf [Bryan]
>>> - RFC:
>>> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
>>>
>>> This patch series depends on [1] & [2]
>>> [1]
>>> https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
>>> [2]
>>> https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
>>>
>>> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
>>> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
>>> the existing H264 decoder functionality remains uneffected.
>>>
>>> Note: 1 of the fluster compliance test is fixed with firmware [3]
>>> [3]:
>>> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
>>>
>>> The result of fluster test on SM8550:
>>>   131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>   GStreamer-H.265-V4L2-Gst1.0.
>>>   The failing test case:
>>>   - 10 testcases failed due to unsupported 10 bit format.
>>>     - DBLK_A_MAIN10_VIXS_4
>>>     - INITQP_B_Main10_Sony_1
>>>     - TSUNEQBD_A_MAIN10_Technicolor_2
>>>     - WP_A_MAIN10_Toshiba_3
>>>     - WP_MAIN10_B_Toshiba_3
>>>     - WPP_A_ericsson_MAIN10_2
>>>     - WPP_B_ericsson_MAIN10_2
>>>     - WPP_C_ericsson_MAIN10_2
>>>     - WPP_E_ericsson_MAIN10_2
>>>     - WPP_F_ericsson_MAIN10_2
>>>   - 4 testcase failed due to unsupported resolution
>>>     - PICSIZE_A_Bossen_1
>>>     - PICSIZE_B_Bossen_1
>>>     - WPP_D_ericsson_MAIN10_2
>>>     - WPP_D_ericsson_MAIN_2
>>>   - 2 testcase failed due to CRC mismatch
>>>     - RAP_A_docomo_6
>>>     - RAP_B_Bossen_2
>>>     - BUG reported:
>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>>>       Analysis - First few frames in this discarded by firmware and are
>>>       sent to driver with 0 filled length. Driver send such buffers to
>>>       client with timestamp 0 and payload set to 0 and
>>>       make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>>>       dropped by GST. But instead, the first frame displayed as green
>>>       frame and when a valid buffer is sent to client later with same 0
>>>       timestamp, its dropped, leading to CRC mismatch for first frame.
>>>
>>>   235/305 testcases passed while testing VP9-TEST-VECTORS with
>>>   GStreamer-VP9-V4L2-Gst1.0.
>>>   The failing test case:
>>>   - 64 testcases failed due to unsupported resolution
>>>     - vp90-2-02-size-08x08.webm
>>>     - vp90-2-02-size-08x10.webm
>>>     - vp90-2-02-size-08x16.webm
>>>     - vp90-2-02-size-08x18.webm
>>>     - vp90-2-02-size-08x32.webm
>>>     - vp90-2-02-size-08x34.webm
>>>     - vp90-2-02-size-08x64.webm
>>>     - vp90-2-02-size-08x66.webm
>>>     - vp90-2-02-size-10x08.webm
>>>     - vp90-2-02-size-10x10.webm
>>>     - vp90-2-02-size-10x16.webm
>>>     - vp90-2-02-size-10x18.webm
>>>     - vp90-2-02-size-10x32.webm
>>>     - vp90-2-02-size-10x34.webm
>>>     - vp90-2-02-size-10x64.webm
>>>     - vp90-2-02-size-10x66.webm
>>>     - vp90-2-02-size-16x08.webm
>>>     - vp90-2-02-size-16x10.webm
>>>     - vp90-2-02-size-16x16.webm
>>>     - vp90-2-02-size-16x18.webm
>>>     - vp90-2-02-size-16x32.webm
>>>     - vp90-2-02-size-16x34.webm
>>>     - vp90-2-02-size-16x64.webm
>>>     - vp90-2-02-size-16x66.webm
>>>     - vp90-2-02-size-18x08.webm
>>>     - vp90-2-02-size-18x10.webm
>>>     - vp90-2-02-size-18x16.webm
>>>     - vp90-2-02-size-18x18.webm
>>>     - vp90-2-02-size-18x32.webm
>>>     - vp90-2-02-size-18x34.webm
>>>     - vp90-2-02-size-18x64.webm
>>>     - vp90-2-02-size-18x66.webm
>>>     - vp90-2-02-size-32x08.webm
>>>     - vp90-2-02-size-32x10.webm
>>>     - vp90-2-02-size-32x16.webm
>>>     - vp90-2-02-size-32x18.webm
>>>     - vp90-2-02-size-32x32.webm
>>>     - vp90-2-02-size-32x34.webm
>>>     - vp90-2-02-size-32x64.webm
>>>     - vp90-2-02-size-32x66.webm
>>>     - vp90-2-02-size-34x08.webm
>>>     - vp90-2-02-size-34x10.webm
>>>     - vp90-2-02-size-34x16.webm
>>>     - vp90-2-02-size-34x18.webm
>>>     - vp90-2-02-size-34x32.webm
>>>     - vp90-2-02-size-34x34.webm
>>>     - vp90-2-02-size-34x64.webm
>>>     - vp90-2-02-size-34x66.webm
>>>     - vp90-2-02-size-64x08.webm
>>>     - vp90-2-02-size-64x10.webm
>>>     - vp90-2-02-size-64x16.webm
>>>     - vp90-2-02-size-64x18.webm
>>>     - vp90-2-02-size-64x32.webm
>>>     - vp90-2-02-size-64x34.webm
>>>     - vp90-2-02-size-64x64.webm
>>>     - vp90-2-02-size-64x66.webm
>>>     - vp90-2-02-size-66x08.webm
>>>     - vp90-2-02-size-66x10.webm
>>>     - vp90-2-02-size-66x16.webm
>>>     - vp90-2-02-size-66x18.webm
>>>     - vp90-2-02-size-66x32.webm
>>>     - vp90-2-02-size-66x34.webm
>>>     - vp90-2-02-size-66x64.webm
>>>     - vp90-2-02-size-66x66.webm
>>>   - 2 testcases failed due to unsupported format
>>>     - vp91-2-04-yuv422.webm
>>>     - vp91-2-04-yuv444.webm
>>>   - 1 testcase failed with CRC mismatch
>>>     - vp90-2-22-svc_1280x720_3.ivf
>>>     - Bug reported:
>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>   - 2 testcase failed due to unsupported resolution after sequence change
>>>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>   - 1 testcase failed due to unsupported stream
>>>     - vp90-2-16-intra-only.webm
>>>
>>> The result of fluster test on SM8250:
>>>   133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>   GStreamer-H.265-V4L2-Gst1.0.
>>>   The failing test case:
>>>   - 10 testcases failed due to unsupported 10 bit format.
>>>     - DBLK_A_MAIN10_VIXS_4
>>>     - INITQP_B_Main10_Sony_1
>>>     - TSUNEQBD_A_MAIN10_Technicolor_2
>>>     - WP_A_MAIN10_Toshiba_3
>>>     - WP_MAIN10_B_Toshiba_3
>>>     - WPP_A_ericsson_MAIN10_2
>>>     - WPP_B_ericsson_MAIN10_2
>>>     - WPP_C_ericsson_MAIN10_2
>>>     - WPP_E_ericsson_MAIN10_2
>>>     - WPP_F_ericsson_MAIN10_2
>>>   - 4 testcase failed due to unsupported resolution
>>>     - PICSIZE_A_Bossen_1
>>>     - PICSIZE_B_Bossen_1
>>>     - WPP_D_ericsson_MAIN10_2
>>>     - WPP_D_ericsson_MAIN_2
>>>
>>>   232/305 testcases passed while testing VP9-TEST-VECTORS with
>>>   GStreamer-VP9-V4L2-Gst1.0.
>>>   The failing test case:
>>>   - 64 testcases failed due to unsupported resolution
>>>     - vp90-2-02-size-08x08.webm
>>>     - vp90-2-02-size-08x10.webm
>>>     - vp90-2-02-size-08x16.webm
>>>     - vp90-2-02-size-08x18.webm
>>>     - vp90-2-02-size-08x32.webm
>>>     - vp90-2-02-size-08x34.webm
>>>     - vp90-2-02-size-08x64.webm
>>>     - vp90-2-02-size-08x66.webm
>>>     - vp90-2-02-size-10x08.webm
>>>     - vp90-2-02-size-10x10.webm
>>>     - vp90-2-02-size-10x16.webm
>>>     - vp90-2-02-size-10x18.webm
>>>     - vp90-2-02-size-10x32.webm
>>>     - vp90-2-02-size-10x34.webm
>>>     - vp90-2-02-size-10x64.webm
>>>     - vp90-2-02-size-10x66.webm
>>>     - vp90-2-02-size-16x08.webm
>>>     - vp90-2-02-size-16x10.webm
>>>     - vp90-2-02-size-16x16.webm
>>>     - vp90-2-02-size-16x18.webm
>>>     - vp90-2-02-size-16x32.webm
>>>     - vp90-2-02-size-16x34.webm
>>>     - vp90-2-02-size-16x64.webm
>>>     - vp90-2-02-size-16x66.webm
>>>     - vp90-2-02-size-18x08.webm
>>>     - vp90-2-02-size-18x10.webm
>>>     - vp90-2-02-size-18x16.webm
>>>     - vp90-2-02-size-18x18.webm
>>>     - vp90-2-02-size-18x32.webm
>>>     - vp90-2-02-size-18x34.webm
>>>     - vp90-2-02-size-18x64.webm
>>>     - vp90-2-02-size-18x66.webm
>>>     - vp90-2-02-size-32x08.webm
>>>     - vp90-2-02-size-32x10.webm
>>>     - vp90-2-02-size-32x16.webm
>>>     - vp90-2-02-size-32x18.webm
>>>     - vp90-2-02-size-32x32.webm
>>>     - vp90-2-02-size-32x34.webm
>>>     - vp90-2-02-size-32x64.webm
>>>     - vp90-2-02-size-32x66.webm
>>>     - vp90-2-02-size-34x08.webm
>>>     - vp90-2-02-size-34x10.webm
>>>     - vp90-2-02-size-34x16.webm
>>>     - vp90-2-02-size-34x18.webm
>>>     - vp90-2-02-size-34x32.webm
>>>     - vp90-2-02-size-34x34.webm
>>>     - vp90-2-02-size-34x64.webm
>>>     - vp90-2-02-size-34x66.webm
>>>     - vp90-2-02-size-64x08.webm
>>>     - vp90-2-02-size-64x10.webm
>>>     - vp90-2-02-size-64x16.webm
>>>     - vp90-2-02-size-64x18.webm
>>>     - vp90-2-02-size-64x32.webm
>>>     - vp90-2-02-size-64x34.webm
>>>     - vp90-2-02-size-64x64.webm
>>>     - vp90-2-02-size-64x66.webm
>>>     - vp90-2-02-size-66x08.webm
>>>     - vp90-2-02-size-66x10.webm
>>>     - vp90-2-02-size-66x16.webm
>>>     - vp90-2-02-size-66x18.webm
>>>     - vp90-2-02-size-66x32.webm
>>>     - vp90-2-02-size-66x34.webm
>>>     - vp90-2-02-size-66x64.webm
>>>     - vp90-2-02-size-66x66.webm
>>>   - 2 testcases failed due to unsupported format
>>>     - vp91-2-04-yuv422.webm
>>>     - vp91-2-04-yuv444.webm
>>>   - 1 testcase failed with CRC mismatch
>>>     - vp90-2-22-svc_1280x720_3.ivf
>>>     - Bug raised:
>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>   - 5 testcase failed due to unsupported resolution after sequence change
>>>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>     - vp90-2-21-resize_inter_320x240_5_1-2.webm
>>>     - vp90-2-21-resize_inter_320x240_7_1-2.webm
>>>     - vp90-2-18-resize.ivf
>>>   - 1 testcase failed with CRC mismatch
>>>     - vp90-2-16-intra-only.webm
>>>     Analysis: First few frames are marked by firmware as NO_SHOW frame.
>>>     Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>>>     Such buffers should be dropped by GST. But instead, the first frame
>>>     is being displayed and when a valid buffer is sent to client later
>>>     with same timestamp, its dropped, leading to CRC mismatch for first
>>>     frame.
>>>
>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>> ---
>>> Dikshita Agarwal (23):
>>>        media: iris: Skip destroying internal buffer if not dequeued
>>>        media: iris: Update CAPTURE format info based on OUTPUT format
>>>        media: iris: Avoid updating frame size to firmware during reconfig
>>>        media: iris: Drop port check for session property response
>>>        media: iris: Prevent HFI queue writes when core is in deinit state
>>>        media: iris: Remove deprecated property setting to firmware
>>>        media: iris: Fix missing function pointer initialization
>>>        media: iris: Fix NULL pointer dereference
>>>        media: iris: Fix typo in depth variable
>>>        media: iris: Track flush responses to prevent premature completion
>>>        media: iris: Fix buffer preparation failure during resolution change
>>>        media: iris: Add handling for corrupt and drop frames
>>>        media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled
>>> length
>>>        media: iris: Add handling for no show frames
>>>        media: iris: Improve last flag handling
>>>        media: iris: Skip flush on first sequence change
>>>        media: iris: Remove redundant buffer count check in stream off
>>>        media: iris: Add a comment to explain usage of MBPS
>>>        media: iris: Add HEVC and VP9 formats for decoder
>>>        media: iris: Add platform capabilities for HEVC and VP9 decoders
>>>        media: iris: Set mandatory properties for HEVC and VP9 decoders.
>>>        media: iris: Add internal buffer calculation for HEVC and VP9
>>> decoders
>>>        media: iris: Add codec specific check for VP9 decoder drain handling
>>>
>>>   drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>>>   drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>>>   drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>>>   drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>>>   .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>>>   .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>>>   .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>>>   .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>>>   .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>>>   .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>>>   drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>>>   drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>>>   .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>>>   .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>>>   .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>>>   .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>>>   drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>>>   drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>>>   drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>>>   drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>>>   drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>>>   drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>>>   drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397
>>> ++++++++++++++++++++-
>>>   drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>>>   24 files changed, 1160 insertions(+), 211 deletions(-)
>>> ---
>>> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
>>> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
>>> prerequisite-message-id:
>>> <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
>>> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
>>> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
>>> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
>>> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
>>> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
>>> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
>>> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
>>> prerequisite-message-id:
>>> <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
>>> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
>>> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
>>> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
>>> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
>>> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
>>>
>>> Best regards,
>>
>> HEVC & VP9 works fine on HDK8550.
>>
>> But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
>> [   44.741670] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.755724] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.776462] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.797179] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.816630] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.837387] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.856812] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.877576] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.897000] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.917801] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   44.937254] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   44.959128] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   44.981025] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   45.000459] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   45.022376] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   45.041816] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   45.063736] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   45.083167] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   45.105459] ------------[ cut here ]------------
>> [   45.121152] WARNING: CPU: 6 PID: 573 at
>> drivers/media/common/videobuf2/videobuf2-core.c:1827
>> vb2_start_streaming+0x100/0x178 [videobuf2_common]
>> while VP9 works fine.
>>
>> Is it a firmware issue ?
>>
> Looks like resources set to firmware are not sufficient.
> I suspect, internal buffers set to firmware are less than what it requires,
> this can change for different VPUs. Pls check if there is any difference in
> internal buffer calculations between vpu3 and vpu33.
I would not recommend adding the HEVC and VP9 for 8650 unless it is functional.
Please drop the HEVC and VP9 from 8650 platform data, and once it is validated,
they can be added in a separate series.

Regards,
Vikash

> 
> Thanks,
> Dikshita
>> I've added:
>> ========================================><======================================
>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> index d3026b2bcb70..8c0ab00ab435 100644
>> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
>>         .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
>>         .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
>>         .vpu_ops = &iris_vpu33_ops,
>> -       .set_preset_registers = iris_set_sm8550_preset_registers,
>> +       .set_preset_registers = iris_set_sm8650_preset_registers,
>>         .icc_tbl = sm8550_icc_table,
>>         .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
>>         .clk_rst_tbl = sm8650_clk_reset_table,
>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> index 75e9d572e788..9e2d23f12f75 100644
>> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = {
>> "bus", "core" };
>>
>>  static const char * const sm8650_controller_reset_table[] = { "xo" };
>>
>> +static void iris_set_sm8650_preset_registers(struct iris_core *core)
>> +{
>> +       writel(0x0, core->reg_base + 0xB0088);
>> +       writel(0x33332222, core->reg_base + 0x13030);
>> +       writel(0x44444444, core->reg_base + 0x13034);
>> +       writel(0x1022, core->reg_base + 0x13038);
>> +       writel(0x0, core->reg_base + 0x13040);
>> +       writel(0xFFFF, core->reg_base + 0x13048);
>> +       writel(0x33332222, core->reg_base + 0x13430);
>> +       writel(0x44444444, core->reg_base + 0x13434);
>> +       writel(0x1022, core->reg_base + 0x13438);
>> +       writel(0x0, core->reg_base + 0x13440);
>> +       writel(0xFFFF, core->reg_base + 0x13448);
>> +       writel(0x99, core->reg_base + 0xA013C);
>> +}
>> +
>>  #endif
>> ========================================><======================================
>> and no change, error still occurs with HEVC decoding.
>>
>> Thanks,
>> Neil

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[Dikshita Agarwal]

On 5/2/2025 2:01 PM, Vikash Garodia wrote:
> 
> On 5/2/2025 1:04 PM, Dikshita Agarwal wrote:
>>
>>
>> On 5/2/2025 12:55 PM, Neil Armstrong wrote:
>>> Hi,
>>>
>>> On 01/05/2025 21:13, Dikshita Agarwal wrote:
>>>> Hi All,
>>>>
>>>> This patch series adds initial support for the HEVC(H.265) and VP9
>>>> codecs in iris decoder. The objective of this work is to extend the
>>>> decoder's capabilities to handle HEVC and VP9 codec streams,
>>>> including necessary format handling and buffer management.
>>>> In addition, the series also includes a set of fixes to address issues
>>>> identified during testing of these additional codecs.
>>>>
>>>> These patches also address the comments and feedback received from the
>>>> RFC patches previously sent. I have made the necessary improvements
>>>> based on the community's suggestions.
>>>>
>>>> Changes in v3:
>>>> - Introduced two wrappers with explicit names to handle destroy internal
>>>> buffers (Nicolas)
>>>> - Used sub state check instead of introducing new boolean (Vikash)
>>>> - Addressed other comments (Vikash)
>>>> - Reorderd patches to have all fixes patches first (Dmitry)
>>>> - Link to v2:
>>>> https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
>>>>
>>>> Changes in v2:
>>>> - Added Changes to make sure all buffers are released in session close
>>>> (bryna)
>>>> - Added tracking for flush responses to fix a timing issue.
>>>> - Added a handling to fix timing issue in reconfig
>>>> - Splitted patch 06/20 in two patches (Bryan)
>>>> - Added missing fixes tag (bryan)
>>>> - Updated fluster report (Nicolas)
>>>> - Link to v1:
>>>> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
>>>>
>>>> Changes sinces RFC:
>>>> - Added additional fixes to address issues identified during further
>>>> testing.
>>>> - Moved typo fix to a seperate patch [Neil]
>>>> - Reordered the patches for better logical flow and clarity [Neil,
>>>> Dmitry]
>>>> - Added fixes tag wherever applicable [Neil, Dmitry]
>>>> - Removed the default case in the switch statement for codecs [Bryan]
>>>> - Replaced if-else statements with switch-case [Bryan]
>>>> - Added comments for mbpf [Bryan]
>>>> - RFC:
>>>> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
>>>>
>>>> This patch series depends on [1] & [2]
>>>> [1]
>>>> https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
>>>> [2]
>>>> https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
>>>>
>>>> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
>>>> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
>>>> the existing H264 decoder functionality remains uneffected.
>>>>
>>>> Note: 1 of the fluster compliance test is fixed with firmware [3]
>>>> [3]:
>>>> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
>>>>
>>>> The result of fluster test on SM8550:
>>>>   131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>   GStreamer-H.265-V4L2-Gst1.0.
>>>>   The failing test case:
>>>>   - 10 testcases failed due to unsupported 10 bit format.
>>>>     - DBLK_A_MAIN10_VIXS_4
>>>>     - INITQP_B_Main10_Sony_1
>>>>     - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>     - WP_A_MAIN10_Toshiba_3
>>>>     - WP_MAIN10_B_Toshiba_3
>>>>     - WPP_A_ericsson_MAIN10_2
>>>>     - WPP_B_ericsson_MAIN10_2
>>>>     - WPP_C_ericsson_MAIN10_2
>>>>     - WPP_E_ericsson_MAIN10_2
>>>>     - WPP_F_ericsson_MAIN10_2
>>>>   - 4 testcase failed due to unsupported resolution
>>>>     - PICSIZE_A_Bossen_1
>>>>     - PICSIZE_B_Bossen_1
>>>>     - WPP_D_ericsson_MAIN10_2
>>>>     - WPP_D_ericsson_MAIN_2
>>>>   - 2 testcase failed due to CRC mismatch
>>>>     - RAP_A_docomo_6
>>>>     - RAP_B_Bossen_2
>>>>     - BUG reported:
>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>>>>       Analysis - First few frames in this discarded by firmware and are
>>>>       sent to driver with 0 filled length. Driver send such buffers to
>>>>       client with timestamp 0 and payload set to 0 and
>>>>       make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>>>>       dropped by GST. But instead, the first frame displayed as green
>>>>       frame and when a valid buffer is sent to client later with same 0
>>>>       timestamp, its dropped, leading to CRC mismatch for first frame.
>>>>
>>>>   235/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>   GStreamer-VP9-V4L2-Gst1.0.
>>>>   The failing test case:
>>>>   - 64 testcases failed due to unsupported resolution
>>>>     - vp90-2-02-size-08x08.webm
>>>>     - vp90-2-02-size-08x10.webm
>>>>     - vp90-2-02-size-08x16.webm
>>>>     - vp90-2-02-size-08x18.webm
>>>>     - vp90-2-02-size-08x32.webm
>>>>     - vp90-2-02-size-08x34.webm
>>>>     - vp90-2-02-size-08x64.webm
>>>>     - vp90-2-02-size-08x66.webm
>>>>     - vp90-2-02-size-10x08.webm
>>>>     - vp90-2-02-size-10x10.webm
>>>>     - vp90-2-02-size-10x16.webm
>>>>     - vp90-2-02-size-10x18.webm
>>>>     - vp90-2-02-size-10x32.webm
>>>>     - vp90-2-02-size-10x34.webm
>>>>     - vp90-2-02-size-10x64.webm
>>>>     - vp90-2-02-size-10x66.webm
>>>>     - vp90-2-02-size-16x08.webm
>>>>     - vp90-2-02-size-16x10.webm
>>>>     - vp90-2-02-size-16x16.webm
>>>>     - vp90-2-02-size-16x18.webm
>>>>     - vp90-2-02-size-16x32.webm
>>>>     - vp90-2-02-size-16x34.webm
>>>>     - vp90-2-02-size-16x64.webm
>>>>     - vp90-2-02-size-16x66.webm
>>>>     - vp90-2-02-size-18x08.webm
>>>>     - vp90-2-02-size-18x10.webm
>>>>     - vp90-2-02-size-18x16.webm
>>>>     - vp90-2-02-size-18x18.webm
>>>>     - vp90-2-02-size-18x32.webm
>>>>     - vp90-2-02-size-18x34.webm
>>>>     - vp90-2-02-size-18x64.webm
>>>>     - vp90-2-02-size-18x66.webm
>>>>     - vp90-2-02-size-32x08.webm
>>>>     - vp90-2-02-size-32x10.webm
>>>>     - vp90-2-02-size-32x16.webm
>>>>     - vp90-2-02-size-32x18.webm
>>>>     - vp90-2-02-size-32x32.webm
>>>>     - vp90-2-02-size-32x34.webm
>>>>     - vp90-2-02-size-32x64.webm
>>>>     - vp90-2-02-size-32x66.webm
>>>>     - vp90-2-02-size-34x08.webm
>>>>     - vp90-2-02-size-34x10.webm
>>>>     - vp90-2-02-size-34x16.webm
>>>>     - vp90-2-02-size-34x18.webm
>>>>     - vp90-2-02-size-34x32.webm
>>>>     - vp90-2-02-size-34x34.webm
>>>>     - vp90-2-02-size-34x64.webm
>>>>     - vp90-2-02-size-34x66.webm
>>>>     - vp90-2-02-size-64x08.webm
>>>>     - vp90-2-02-size-64x10.webm
>>>>     - vp90-2-02-size-64x16.webm
>>>>     - vp90-2-02-size-64x18.webm
>>>>     - vp90-2-02-size-64x32.webm
>>>>     - vp90-2-02-size-64x34.webm
>>>>     - vp90-2-02-size-64x64.webm
>>>>     - vp90-2-02-size-64x66.webm
>>>>     - vp90-2-02-size-66x08.webm
>>>>     - vp90-2-02-size-66x10.webm
>>>>     - vp90-2-02-size-66x16.webm
>>>>     - vp90-2-02-size-66x18.webm
>>>>     - vp90-2-02-size-66x32.webm
>>>>     - vp90-2-02-size-66x34.webm
>>>>     - vp90-2-02-size-66x64.webm
>>>>     - vp90-2-02-size-66x66.webm
>>>>   - 2 testcases failed due to unsupported format
>>>>     - vp91-2-04-yuv422.webm
>>>>     - vp91-2-04-yuv444.webm
>>>>   - 1 testcase failed with CRC mismatch
>>>>     - vp90-2-22-svc_1280x720_3.ivf
>>>>     - Bug reported:
>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>   - 2 testcase failed due to unsupported resolution after sequence change
>>>>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>   - 1 testcase failed due to unsupported stream
>>>>     - vp90-2-16-intra-only.webm
>>>>
>>>> The result of fluster test on SM8250:
>>>>   133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>   GStreamer-H.265-V4L2-Gst1.0.
>>>>   The failing test case:
>>>>   - 10 testcases failed due to unsupported 10 bit format.
>>>>     - DBLK_A_MAIN10_VIXS_4
>>>>     - INITQP_B_Main10_Sony_1
>>>>     - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>     - WP_A_MAIN10_Toshiba_3
>>>>     - WP_MAIN10_B_Toshiba_3
>>>>     - WPP_A_ericsson_MAIN10_2
>>>>     - WPP_B_ericsson_MAIN10_2
>>>>     - WPP_C_ericsson_MAIN10_2
>>>>     - WPP_E_ericsson_MAIN10_2
>>>>     - WPP_F_ericsson_MAIN10_2
>>>>   - 4 testcase failed due to unsupported resolution
>>>>     - PICSIZE_A_Bossen_1
>>>>     - PICSIZE_B_Bossen_1
>>>>     - WPP_D_ericsson_MAIN10_2
>>>>     - WPP_D_ericsson_MAIN_2
>>>>
>>>>   232/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>   GStreamer-VP9-V4L2-Gst1.0.
>>>>   The failing test case:
>>>>   - 64 testcases failed due to unsupported resolution
>>>>     - vp90-2-02-size-08x08.webm
>>>>     - vp90-2-02-size-08x10.webm
>>>>     - vp90-2-02-size-08x16.webm
>>>>     - vp90-2-02-size-08x18.webm
>>>>     - vp90-2-02-size-08x32.webm
>>>>     - vp90-2-02-size-08x34.webm
>>>>     - vp90-2-02-size-08x64.webm
>>>>     - vp90-2-02-size-08x66.webm
>>>>     - vp90-2-02-size-10x08.webm
>>>>     - vp90-2-02-size-10x10.webm
>>>>     - vp90-2-02-size-10x16.webm
>>>>     - vp90-2-02-size-10x18.webm
>>>>     - vp90-2-02-size-10x32.webm
>>>>     - vp90-2-02-size-10x34.webm
>>>>     - vp90-2-02-size-10x64.webm
>>>>     - vp90-2-02-size-10x66.webm
>>>>     - vp90-2-02-size-16x08.webm
>>>>     - vp90-2-02-size-16x10.webm
>>>>     - vp90-2-02-size-16x16.webm
>>>>     - vp90-2-02-size-16x18.webm
>>>>     - vp90-2-02-size-16x32.webm
>>>>     - vp90-2-02-size-16x34.webm
>>>>     - vp90-2-02-size-16x64.webm
>>>>     - vp90-2-02-size-16x66.webm
>>>>     - vp90-2-02-size-18x08.webm
>>>>     - vp90-2-02-size-18x10.webm
>>>>     - vp90-2-02-size-18x16.webm
>>>>     - vp90-2-02-size-18x18.webm
>>>>     - vp90-2-02-size-18x32.webm
>>>>     - vp90-2-02-size-18x34.webm
>>>>     - vp90-2-02-size-18x64.webm
>>>>     - vp90-2-02-size-18x66.webm
>>>>     - vp90-2-02-size-32x08.webm
>>>>     - vp90-2-02-size-32x10.webm
>>>>     - vp90-2-02-size-32x16.webm
>>>>     - vp90-2-02-size-32x18.webm
>>>>     - vp90-2-02-size-32x32.webm
>>>>     - vp90-2-02-size-32x34.webm
>>>>     - vp90-2-02-size-32x64.webm
>>>>     - vp90-2-02-size-32x66.webm
>>>>     - vp90-2-02-size-34x08.webm
>>>>     - vp90-2-02-size-34x10.webm
>>>>     - vp90-2-02-size-34x16.webm
>>>>     - vp90-2-02-size-34x18.webm
>>>>     - vp90-2-02-size-34x32.webm
>>>>     - vp90-2-02-size-34x34.webm
>>>>     - vp90-2-02-size-34x64.webm
>>>>     - vp90-2-02-size-34x66.webm
>>>>     - vp90-2-02-size-64x08.webm
>>>>     - vp90-2-02-size-64x10.webm
>>>>     - vp90-2-02-size-64x16.webm
>>>>     - vp90-2-02-size-64x18.webm
>>>>     - vp90-2-02-size-64x32.webm
>>>>     - vp90-2-02-size-64x34.webm
>>>>     - vp90-2-02-size-64x64.webm
>>>>     - vp90-2-02-size-64x66.webm
>>>>     - vp90-2-02-size-66x08.webm
>>>>     - vp90-2-02-size-66x10.webm
>>>>     - vp90-2-02-size-66x16.webm
>>>>     - vp90-2-02-size-66x18.webm
>>>>     - vp90-2-02-size-66x32.webm
>>>>     - vp90-2-02-size-66x34.webm
>>>>     - vp90-2-02-size-66x64.webm
>>>>     - vp90-2-02-size-66x66.webm
>>>>   - 2 testcases failed due to unsupported format
>>>>     - vp91-2-04-yuv422.webm
>>>>     - vp91-2-04-yuv444.webm
>>>>   - 1 testcase failed with CRC mismatch
>>>>     - vp90-2-22-svc_1280x720_3.ivf
>>>>     - Bug raised:
>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>   - 5 testcase failed due to unsupported resolution after sequence change
>>>>     - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>     - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>     - vp90-2-21-resize_inter_320x240_5_1-2.webm
>>>>     - vp90-2-21-resize_inter_320x240_7_1-2.webm
>>>>     - vp90-2-18-resize.ivf
>>>>   - 1 testcase failed with CRC mismatch
>>>>     - vp90-2-16-intra-only.webm
>>>>     Analysis: First few frames are marked by firmware as NO_SHOW frame.
>>>>     Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>>>>     Such buffers should be dropped by GST. But instead, the first frame
>>>>     is being displayed and when a valid buffer is sent to client later
>>>>     with same timestamp, its dropped, leading to CRC mismatch for first
>>>>     frame.
>>>>
>>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>>> ---
>>>> Dikshita Agarwal (23):
>>>>        media: iris: Skip destroying internal buffer if not dequeued
>>>>        media: iris: Update CAPTURE format info based on OUTPUT format
>>>>        media: iris: Avoid updating frame size to firmware during reconfig
>>>>        media: iris: Drop port check for session property response
>>>>        media: iris: Prevent HFI queue writes when core is in deinit state
>>>>        media: iris: Remove deprecated property setting to firmware
>>>>        media: iris: Fix missing function pointer initialization
>>>>        media: iris: Fix NULL pointer dereference
>>>>        media: iris: Fix typo in depth variable
>>>>        media: iris: Track flush responses to prevent premature completion
>>>>        media: iris: Fix buffer preparation failure during resolution change
>>>>        media: iris: Add handling for corrupt and drop frames
>>>>        media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled
>>>> length
>>>>        media: iris: Add handling for no show frames
>>>>        media: iris: Improve last flag handling
>>>>        media: iris: Skip flush on first sequence change
>>>>        media: iris: Remove redundant buffer count check in stream off
>>>>        media: iris: Add a comment to explain usage of MBPS
>>>>        media: iris: Add HEVC and VP9 formats for decoder
>>>>        media: iris: Add platform capabilities for HEVC and VP9 decoders
>>>>        media: iris: Set mandatory properties for HEVC and VP9 decoders.
>>>>        media: iris: Add internal buffer calculation for HEVC and VP9
>>>> decoders
>>>>        media: iris: Add codec specific check for VP9 decoder drain handling
>>>>
>>>>   drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>>>>   drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>>>>   drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>>>>   drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>>>>   .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>>>>   .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>>>>   .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>>>>   .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>>>>   .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>>>>   .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>>>>   drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>>>>   drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>>>>   .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>>>>   .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>>>>   .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>>>>   .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>>>>   drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>>>>   drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>>>>   drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>>>>   drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>>>>   drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>>>>   drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>>>>   drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397
>>>> ++++++++++++++++++++-
>>>>   drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>>>>   24 files changed, 1160 insertions(+), 211 deletions(-)
>>>> ---
>>>> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
>>>> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
>>>> prerequisite-message-id:
>>>> <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
>>>> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
>>>> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
>>>> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
>>>> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
>>>> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
>>>> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
>>>> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
>>>> prerequisite-message-id:
>>>> <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
>>>> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
>>>> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
>>>> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
>>>> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
>>>> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
>>>>
>>>> Best regards,
>>>
>>> HEVC & VP9 works fine on HDK8550.
>>>
>>> But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
>>> [   44.741670] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.755724] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.776462] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.797179] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.816630] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.837387] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.856812] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.877576] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.897000] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.917801] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   44.937254] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   44.959128] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   44.981025] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   45.000459] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   45.022376] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   45.041816] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   45.063736] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   45.083167] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   45.105459] ------------[ cut here ]------------
>>> [   45.121152] WARNING: CPU: 6 PID: 573 at
>>> drivers/media/common/videobuf2/videobuf2-core.c:1827
>>> vb2_start_streaming+0x100/0x178 [videobuf2_common]
>>> while VP9 works fine.
>>>
>>> Is it a firmware issue ?
>>>
>> Looks like resources set to firmware are not sufficient.
>> I suspect, internal buffers set to firmware are less than what it requires,
>> this can change for different VPUs. Pls check if there is any difference in
>> internal buffer calculations between vpu3 and vpu33.
> I would not recommend adding the HEVC and VP9 for 8650 unless it is functional.
> Please drop the HEVC and VP9 from 8650 platform data, and once it is validated,
> they can be added in a separate series.
> 
Agree, I would not want to hold the enablement of HEVC and VP9 codecs.
So, I will drop support for HEVC and VP9 codecs and retain only H.264
support in the 8650 caps in the next version.

Thanks,
Dikshita
> Regards,
> Vikash
> 
>>
>> Thanks,
>> Dikshita
>>> I've added:
>>> ========================================><======================================
>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> index d3026b2bcb70..8c0ab00ab435 100644
>>> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
>>>         .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
>>>         .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
>>>         .vpu_ops = &iris_vpu33_ops,
>>> -       .set_preset_registers = iris_set_sm8550_preset_registers,
>>> +       .set_preset_registers = iris_set_sm8650_preset_registers,
>>>         .icc_tbl = sm8550_icc_table,
>>>         .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
>>>         .clk_rst_tbl = sm8650_clk_reset_table,
>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> index 75e9d572e788..9e2d23f12f75 100644
>>> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = {
>>> "bus", "core" };
>>>
>>>  static const char * const sm8650_controller_reset_table[] = { "xo" };
>>>
>>> +static void iris_set_sm8650_preset_registers(struct iris_core *core)
>>> +{
>>> +       writel(0x0, core->reg_base + 0xB0088);
>>> +       writel(0x33332222, core->reg_base + 0x13030);
>>> +       writel(0x44444444, core->reg_base + 0x13034);
>>> +       writel(0x1022, core->reg_base + 0x13038);
>>> +       writel(0x0, core->reg_base + 0x13040);
>>> +       writel(0xFFFF, core->reg_base + 0x13048);
>>> +       writel(0x33332222, core->reg_base + 0x13430);
>>> +       writel(0x44444444, core->reg_base + 0x13434);
>>> +       writel(0x1022, core->reg_base + 0x13438);
>>> +       writel(0x0, core->reg_base + 0x13440);
>>> +       writel(0xFFFF, core->reg_base + 0x13448);
>>> +       writel(0x99, core->reg_base + 0xA013C);
>>> +}
>>> +
>>>  #endif
>>> ========================================><======================================
>>> and no change, error still occurs with HEVC decoding.
>>>
>>> Thanks,
>>> Neil

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[neil.armstrong]

On 02/05/2025 10:41, Dikshita Agarwal wrote:
> 
> 
> On 5/2/2025 2:01 PM, Vikash Garodia wrote:
>>
>> On 5/2/2025 1:04 PM, Dikshita Agarwal wrote:
>>>
>>>
>>> On 5/2/2025 12:55 PM, Neil Armstrong wrote:
>>>> Hi,
>>>>
>>>> On 01/05/2025 21:13, Dikshita Agarwal wrote:
>>>>> Hi All,
>>>>>
>>>>> This patch series adds initial support for the HEVC(H.265) and VP9
>>>>> codecs in iris decoder. The objective of this work is to extend the
>>>>> decoder's capabilities to handle HEVC and VP9 codec streams,
>>>>> including necessary format handling and buffer management.
>>>>> In addition, the series also includes a set of fixes to address issues
>>>>> identified during testing of these additional codecs.
>>>>>
>>>>> These patches also address the comments and feedback received from the
>>>>> RFC patches previously sent. I have made the necessary improvements
>>>>> based on the community's suggestions.
>>>>>
>>>>> Changes in v3:
>>>>> - Introduced two wrappers with explicit names to handle destroy internal
>>>>> buffers (Nicolas)
>>>>> - Used sub state check instead of introducing new boolean (Vikash)
>>>>> - Addressed other comments (Vikash)
>>>>> - Reorderd patches to have all fixes patches first (Dmitry)
>>>>> - Link to v2:
>>>>> https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
>>>>>
>>>>> Changes in v2:
>>>>> - Added Changes to make sure all buffers are released in session close
>>>>> (bryna)
>>>>> - Added tracking for flush responses to fix a timing issue.
>>>>> - Added a handling to fix timing issue in reconfig
>>>>> - Splitted patch 06/20 in two patches (Bryan)
>>>>> - Added missing fixes tag (bryan)
>>>>> - Updated fluster report (Nicolas)
>>>>> - Link to v1:
>>>>> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
>>>>>
>>>>> Changes sinces RFC:
>>>>> - Added additional fixes to address issues identified during further
>>>>> testing.
>>>>> - Moved typo fix to a seperate patch [Neil]
>>>>> - Reordered the patches for better logical flow and clarity [Neil,
>>>>> Dmitry]
>>>>> - Added fixes tag wherever applicable [Neil, Dmitry]
>>>>> - Removed the default case in the switch statement for codecs [Bryan]
>>>>> - Replaced if-else statements with switch-case [Bryan]
>>>>> - Added comments for mbpf [Bryan]
>>>>> - RFC:
>>>>> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
>>>>>
>>>>> This patch series depends on [1] & [2]
>>>>> [1]
>>>>> https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
>>>>> [2]
>>>>> https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
>>>>>
>>>>> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
>>>>> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
>>>>> the existing H264 decoder functionality remains uneffected.
>>>>>
>>>>> Note: 1 of the fluster compliance test is fixed with firmware [3]
>>>>> [3]:
>>>>> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
>>>>>
>>>>> The result of fluster test on SM8550:
>>>>>    131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>>    GStreamer-H.265-V4L2-Gst1.0.
>>>>>    The failing test case:
>>>>>    - 10 testcases failed due to unsupported 10 bit format.
>>>>>      - DBLK_A_MAIN10_VIXS_4
>>>>>      - INITQP_B_Main10_Sony_1
>>>>>      - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>>      - WP_A_MAIN10_Toshiba_3
>>>>>      - WP_MAIN10_B_Toshiba_3
>>>>>      - WPP_A_ericsson_MAIN10_2
>>>>>      - WPP_B_ericsson_MAIN10_2
>>>>>      - WPP_C_ericsson_MAIN10_2
>>>>>      - WPP_E_ericsson_MAIN10_2
>>>>>      - WPP_F_ericsson_MAIN10_2
>>>>>    - 4 testcase failed due to unsupported resolution
>>>>>      - PICSIZE_A_Bossen_1
>>>>>      - PICSIZE_B_Bossen_1
>>>>>      - WPP_D_ericsson_MAIN10_2
>>>>>      - WPP_D_ericsson_MAIN_2
>>>>>    - 2 testcase failed due to CRC mismatch
>>>>>      - RAP_A_docomo_6
>>>>>      - RAP_B_Bossen_2
>>>>>      - BUG reported:
>>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>>>>>        Analysis - First few frames in this discarded by firmware and are
>>>>>        sent to driver with 0 filled length. Driver send such buffers to
>>>>>        client with timestamp 0 and payload set to 0 and
>>>>>        make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>>>>>        dropped by GST. But instead, the first frame displayed as green
>>>>>        frame and when a valid buffer is sent to client later with same 0
>>>>>        timestamp, its dropped, leading to CRC mismatch for first frame.
>>>>>
>>>>>    235/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>>    GStreamer-VP9-V4L2-Gst1.0.
>>>>>    The failing test case:
>>>>>    - 64 testcases failed due to unsupported resolution
>>>>>      - vp90-2-02-size-08x08.webm
>>>>>      - vp90-2-02-size-08x10.webm
>>>>>      - vp90-2-02-size-08x16.webm
>>>>>      - vp90-2-02-size-08x18.webm
>>>>>      - vp90-2-02-size-08x32.webm
>>>>>      - vp90-2-02-size-08x34.webm
>>>>>      - vp90-2-02-size-08x64.webm
>>>>>      - vp90-2-02-size-08x66.webm
>>>>>      - vp90-2-02-size-10x08.webm
>>>>>      - vp90-2-02-size-10x10.webm
>>>>>      - vp90-2-02-size-10x16.webm
>>>>>      - vp90-2-02-size-10x18.webm
>>>>>      - vp90-2-02-size-10x32.webm
>>>>>      - vp90-2-02-size-10x34.webm
>>>>>      - vp90-2-02-size-10x64.webm
>>>>>      - vp90-2-02-size-10x66.webm
>>>>>      - vp90-2-02-size-16x08.webm
>>>>>      - vp90-2-02-size-16x10.webm
>>>>>      - vp90-2-02-size-16x16.webm
>>>>>      - vp90-2-02-size-16x18.webm
>>>>>      - vp90-2-02-size-16x32.webm
>>>>>      - vp90-2-02-size-16x34.webm
>>>>>      - vp90-2-02-size-16x64.webm
>>>>>      - vp90-2-02-size-16x66.webm
>>>>>      - vp90-2-02-size-18x08.webm
>>>>>      - vp90-2-02-size-18x10.webm
>>>>>      - vp90-2-02-size-18x16.webm
>>>>>      - vp90-2-02-size-18x18.webm
>>>>>      - vp90-2-02-size-18x32.webm
>>>>>      - vp90-2-02-size-18x34.webm
>>>>>      - vp90-2-02-size-18x64.webm
>>>>>      - vp90-2-02-size-18x66.webm
>>>>>      - vp90-2-02-size-32x08.webm
>>>>>      - vp90-2-02-size-32x10.webm
>>>>>      - vp90-2-02-size-32x16.webm
>>>>>      - vp90-2-02-size-32x18.webm
>>>>>      - vp90-2-02-size-32x32.webm
>>>>>      - vp90-2-02-size-32x34.webm
>>>>>      - vp90-2-02-size-32x64.webm
>>>>>      - vp90-2-02-size-32x66.webm
>>>>>      - vp90-2-02-size-34x08.webm
>>>>>      - vp90-2-02-size-34x10.webm
>>>>>      - vp90-2-02-size-34x16.webm
>>>>>      - vp90-2-02-size-34x18.webm
>>>>>      - vp90-2-02-size-34x32.webm
>>>>>      - vp90-2-02-size-34x34.webm
>>>>>      - vp90-2-02-size-34x64.webm
>>>>>      - vp90-2-02-size-34x66.webm
>>>>>      - vp90-2-02-size-64x08.webm
>>>>>      - vp90-2-02-size-64x10.webm
>>>>>      - vp90-2-02-size-64x16.webm
>>>>>      - vp90-2-02-size-64x18.webm
>>>>>      - vp90-2-02-size-64x32.webm
>>>>>      - vp90-2-02-size-64x34.webm
>>>>>      - vp90-2-02-size-64x64.webm
>>>>>      - vp90-2-02-size-64x66.webm
>>>>>      - vp90-2-02-size-66x08.webm
>>>>>      - vp90-2-02-size-66x10.webm
>>>>>      - vp90-2-02-size-66x16.webm
>>>>>      - vp90-2-02-size-66x18.webm
>>>>>      - vp90-2-02-size-66x32.webm
>>>>>      - vp90-2-02-size-66x34.webm
>>>>>      - vp90-2-02-size-66x64.webm
>>>>>      - vp90-2-02-size-66x66.webm
>>>>>    - 2 testcases failed due to unsupported format
>>>>>      - vp91-2-04-yuv422.webm
>>>>>      - vp91-2-04-yuv444.webm
>>>>>    - 1 testcase failed with CRC mismatch
>>>>>      - vp90-2-22-svc_1280x720_3.ivf
>>>>>      - Bug reported:
>>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>>    - 2 testcase failed due to unsupported resolution after sequence change
>>>>>      - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>>      - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>>    - 1 testcase failed due to unsupported stream
>>>>>      - vp90-2-16-intra-only.webm
>>>>>
>>>>> The result of fluster test on SM8250:
>>>>>    133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>>    GStreamer-H.265-V4L2-Gst1.0.
>>>>>    The failing test case:
>>>>>    - 10 testcases failed due to unsupported 10 bit format.
>>>>>      - DBLK_A_MAIN10_VIXS_4
>>>>>      - INITQP_B_Main10_Sony_1
>>>>>      - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>>      - WP_A_MAIN10_Toshiba_3
>>>>>      - WP_MAIN10_B_Toshiba_3
>>>>>      - WPP_A_ericsson_MAIN10_2
>>>>>      - WPP_B_ericsson_MAIN10_2
>>>>>      - WPP_C_ericsson_MAIN10_2
>>>>>      - WPP_E_ericsson_MAIN10_2
>>>>>      - WPP_F_ericsson_MAIN10_2
>>>>>    - 4 testcase failed due to unsupported resolution
>>>>>      - PICSIZE_A_Bossen_1
>>>>>      - PICSIZE_B_Bossen_1
>>>>>      - WPP_D_ericsson_MAIN10_2
>>>>>      - WPP_D_ericsson_MAIN_2
>>>>>
>>>>>    232/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>>    GStreamer-VP9-V4L2-Gst1.0.
>>>>>    The failing test case:
>>>>>    - 64 testcases failed due to unsupported resolution
>>>>>      - vp90-2-02-size-08x08.webm
>>>>>      - vp90-2-02-size-08x10.webm
>>>>>      - vp90-2-02-size-08x16.webm
>>>>>      - vp90-2-02-size-08x18.webm
>>>>>      - vp90-2-02-size-08x32.webm
>>>>>      - vp90-2-02-size-08x34.webm
>>>>>      - vp90-2-02-size-08x64.webm
>>>>>      - vp90-2-02-size-08x66.webm
>>>>>      - vp90-2-02-size-10x08.webm
>>>>>      - vp90-2-02-size-10x10.webm
>>>>>      - vp90-2-02-size-10x16.webm
>>>>>      - vp90-2-02-size-10x18.webm
>>>>>      - vp90-2-02-size-10x32.webm
>>>>>      - vp90-2-02-size-10x34.webm
>>>>>      - vp90-2-02-size-10x64.webm
>>>>>      - vp90-2-02-size-10x66.webm
>>>>>      - vp90-2-02-size-16x08.webm
>>>>>      - vp90-2-02-size-16x10.webm
>>>>>      - vp90-2-02-size-16x16.webm
>>>>>      - vp90-2-02-size-16x18.webm
>>>>>      - vp90-2-02-size-16x32.webm
>>>>>      - vp90-2-02-size-16x34.webm
>>>>>      - vp90-2-02-size-16x64.webm
>>>>>      - vp90-2-02-size-16x66.webm
>>>>>      - vp90-2-02-size-18x08.webm
>>>>>      - vp90-2-02-size-18x10.webm
>>>>>      - vp90-2-02-size-18x16.webm
>>>>>      - vp90-2-02-size-18x18.webm
>>>>>      - vp90-2-02-size-18x32.webm
>>>>>      - vp90-2-02-size-18x34.webm
>>>>>      - vp90-2-02-size-18x64.webm
>>>>>      - vp90-2-02-size-18x66.webm
>>>>>      - vp90-2-02-size-32x08.webm
>>>>>      - vp90-2-02-size-32x10.webm
>>>>>      - vp90-2-02-size-32x16.webm
>>>>>      - vp90-2-02-size-32x18.webm
>>>>>      - vp90-2-02-size-32x32.webm
>>>>>      - vp90-2-02-size-32x34.webm
>>>>>      - vp90-2-02-size-32x64.webm
>>>>>      - vp90-2-02-size-32x66.webm
>>>>>      - vp90-2-02-size-34x08.webm
>>>>>      - vp90-2-02-size-34x10.webm
>>>>>      - vp90-2-02-size-34x16.webm
>>>>>      - vp90-2-02-size-34x18.webm
>>>>>      - vp90-2-02-size-34x32.webm
>>>>>      - vp90-2-02-size-34x34.webm
>>>>>      - vp90-2-02-size-34x64.webm
>>>>>      - vp90-2-02-size-34x66.webm
>>>>>      - vp90-2-02-size-64x08.webm
>>>>>      - vp90-2-02-size-64x10.webm
>>>>>      - vp90-2-02-size-64x16.webm
>>>>>      - vp90-2-02-size-64x18.webm
>>>>>      - vp90-2-02-size-64x32.webm
>>>>>      - vp90-2-02-size-64x34.webm
>>>>>      - vp90-2-02-size-64x64.webm
>>>>>      - vp90-2-02-size-64x66.webm
>>>>>      - vp90-2-02-size-66x08.webm
>>>>>      - vp90-2-02-size-66x10.webm
>>>>>      - vp90-2-02-size-66x16.webm
>>>>>      - vp90-2-02-size-66x18.webm
>>>>>      - vp90-2-02-size-66x32.webm
>>>>>      - vp90-2-02-size-66x34.webm
>>>>>      - vp90-2-02-size-66x64.webm
>>>>>      - vp90-2-02-size-66x66.webm
>>>>>    - 2 testcases failed due to unsupported format
>>>>>      - vp91-2-04-yuv422.webm
>>>>>      - vp91-2-04-yuv444.webm
>>>>>    - 1 testcase failed with CRC mismatch
>>>>>      - vp90-2-22-svc_1280x720_3.ivf
>>>>>      - Bug raised:
>>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>>    - 5 testcase failed due to unsupported resolution after sequence change
>>>>>      - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>>      - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>>      - vp90-2-21-resize_inter_320x240_5_1-2.webm
>>>>>      - vp90-2-21-resize_inter_320x240_7_1-2.webm
>>>>>      - vp90-2-18-resize.ivf
>>>>>    - 1 testcase failed with CRC mismatch
>>>>>      - vp90-2-16-intra-only.webm
>>>>>      Analysis: First few frames are marked by firmware as NO_SHOW frame.
>>>>>      Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>>>>>      Such buffers should be dropped by GST. But instead, the first frame
>>>>>      is being displayed and when a valid buffer is sent to client later
>>>>>      with same timestamp, its dropped, leading to CRC mismatch for first
>>>>>      frame.
>>>>>
>>>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>>>> ---
>>>>> Dikshita Agarwal (23):
>>>>>         media: iris: Skip destroying internal buffer if not dequeued
>>>>>         media: iris: Update CAPTURE format info based on OUTPUT format
>>>>>         media: iris: Avoid updating frame size to firmware during reconfig
>>>>>         media: iris: Drop port check for session property response
>>>>>         media: iris: Prevent HFI queue writes when core is in deinit state
>>>>>         media: iris: Remove deprecated property setting to firmware
>>>>>         media: iris: Fix missing function pointer initialization
>>>>>         media: iris: Fix NULL pointer dereference
>>>>>         media: iris: Fix typo in depth variable
>>>>>         media: iris: Track flush responses to prevent premature completion
>>>>>         media: iris: Fix buffer preparation failure during resolution change
>>>>>         media: iris: Add handling for corrupt and drop frames
>>>>>         media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled
>>>>> length
>>>>>         media: iris: Add handling for no show frames
>>>>>         media: iris: Improve last flag handling
>>>>>         media: iris: Skip flush on first sequence change
>>>>>         media: iris: Remove redundant buffer count check in stream off
>>>>>         media: iris: Add a comment to explain usage of MBPS
>>>>>         media: iris: Add HEVC and VP9 formats for decoder
>>>>>         media: iris: Add platform capabilities for HEVC and VP9 decoders
>>>>>         media: iris: Set mandatory properties for HEVC and VP9 decoders.
>>>>>         media: iris: Add internal buffer calculation for HEVC and VP9
>>>>> decoders
>>>>>         media: iris: Add codec specific check for VP9 decoder drain handling
>>>>>
>>>>>    drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>>>>>    drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>>>>>    drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>>>>>    drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>>>>>    .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>>>>>    .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>>>>>    .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>>>>>    .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>>>>>    .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>>>>>    .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>>>>>    drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>>>>>    drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>>>>>    .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>>>>>    .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>>>>>    .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>>>>>    .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>>>>>    drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>>>>>    drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>>>>>    drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>>>>>    drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>>>>>    drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>>>>>    drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>>>>>    drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397
>>>>> ++++++++++++++++++++-
>>>>>    drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>>>>>    24 files changed, 1160 insertions(+), 211 deletions(-)
>>>>> ---
>>>>> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
>>>>> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
>>>>> prerequisite-message-id:
>>>>> <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
>>>>> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
>>>>> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
>>>>> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
>>>>> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
>>>>> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
>>>>> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
>>>>> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
>>>>> prerequisite-message-id:
>>>>> <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
>>>>> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
>>>>> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
>>>>> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
>>>>> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
>>>>> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
>>>>>
>>>>> Best regards,
>>>>
>>>> HEVC & VP9 works fine on HDK8550.
>>>>
>>>> But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
>>>> [   44.741670] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.755724] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.776462] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.797179] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.816630] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.837387] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.856812] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.877576] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.897000] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.917801] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   44.937254] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   44.959128] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   44.981025] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   45.000459] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   45.022376] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   45.041816] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   45.063736] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   45.083167] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   45.105459] ------------[ cut here ]------------
>>>> [   45.121152] WARNING: CPU: 6 PID: 573 at
>>>> drivers/media/common/videobuf2/videobuf2-core.c:1827
>>>> vb2_start_streaming+0x100/0x178 [videobuf2_common]
>>>> while VP9 works fine.
>>>>
>>>> Is it a firmware issue ?
>>>>
>>> Looks like resources set to firmware are not sufficient.
>>> I suspect, internal buffers set to firmware are less than what it requires,
>>> this can change for different VPUs. Pls check if there is any difference in
>>> internal buffer calculations between vpu3 and vpu33.
>> I would not recommend adding the HEVC and VP9 for 8650 unless it is functional.
>> Please drop the HEVC and VP9 from 8650 platform data, and once it is validated,
>> they can be added in a separate series.
>>
> Agree, I would not want to hold the enablement of HEVC and VP9 codecs.
> So, I will drop support for HEVC and VP9 codecs and retain only H.264
> support in the 8650 caps in the next version.

Sure, thx

Neil

> 
> Thanks,
> Dikshita
>> Regards,
>> Vikash
>>
>>>
>>> Thanks,
>>> Dikshita
>>>> I've added:
>>>> ========================================><======================================
>>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> index d3026b2bcb70..8c0ab00ab435 100644
>>>> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
>>>>          .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
>>>>          .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
>>>>          .vpu_ops = &iris_vpu33_ops,
>>>> -       .set_preset_registers = iris_set_sm8550_preset_registers,
>>>> +       .set_preset_registers = iris_set_sm8650_preset_registers,
>>>>          .icc_tbl = sm8550_icc_table,
>>>>          .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
>>>>          .clk_rst_tbl = sm8650_clk_reset_table,
>>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> index 75e9d572e788..9e2d23f12f75 100644
>>>> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = {
>>>> "bus", "core" };
>>>>
>>>>   static const char * const sm8650_controller_reset_table[] = { "xo" };
>>>>
>>>> +static void iris_set_sm8650_preset_registers(struct iris_core *core)
>>>> +{
>>>> +       writel(0x0, core->reg_base + 0xB0088);
>>>> +       writel(0x33332222, core->reg_base + 0x13030);
>>>> +       writel(0x44444444, core->reg_base + 0x13034);
>>>> +       writel(0x1022, core->reg_base + 0x13038);
>>>> +       writel(0x0, core->reg_base + 0x13040);
>>>> +       writel(0xFFFF, core->reg_base + 0x13048);
>>>> +       writel(0x33332222, core->reg_base + 0x13430);
>>>> +       writel(0x44444444, core->reg_base + 0x13434);
>>>> +       writel(0x1022, core->reg_base + 0x13438);
>>>> +       writel(0x0, core->reg_base + 0x13440);
>>>> +       writel(0xFFFF, core->reg_base + 0x13448);
>>>> +       writel(0x99, core->reg_base + 0xA013C);
>>>> +}
>>>> +
>>>>   #endif
>>>> ========================================><======================================
>>>> and no change, error still occurs with HEVC decoding.
>>>>
>>>> Thanks,
>>>> Neil

Re: [PATCH v3 01/23] media: iris: Skip destroying internal buffer if not dequeued

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> Firmware might hold the DPB buffers for reference in case of sequence
> change, so skip destroying buffers for which QUEUED flag is not removed.
> Also, make sure that all buffers are released during streamoff.
> 
> Cc: stable@vger.kernel.org
> Fixes: 73702f45db81 ("media: iris: allocate, initialize and queue internal buffers")
> Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_buffer.c | 20 +++++++++++++++-
>   drivers/media/platform/qcom/iris/iris_buffer.h |  3 ++-
>   drivers/media/platform/qcom/iris/iris_vdec.c   |  4 ++--
>   drivers/media/platform/qcom/iris/iris_vidc.c   | 33 ++++++++++++++++++++++++--
>   4 files changed, 54 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c
> index e5c5a564fcb8..981fedb000ed 100644
> --- a/drivers/media/platform/qcom/iris/iris_buffer.c
> +++ b/drivers/media/platform/qcom/iris/iris_buffer.c
> @@ -376,7 +376,7 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf
>   	return 0;
>   }
>   
> -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane)
> +static int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane, bool force)
>   {
>   	const struct iris_platform_data *platform_data = inst->core->iris_platform_data;
>   	struct iris_buffer *buf, *next;
> @@ -396,6 +396,14 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane)
>   	for (i = 0; i < len; i++) {
>   		buffers = &inst->buffers[internal_buf_type[i]];
>   		list_for_each_entry_safe(buf, next, &buffers->list, list) {
> +			/*
> +			 * during stream on, skip destroying internal(DPB) buffer
> +			 * if firmware did not return it.
> +			 * during close, destroy all buffers irrespectively.
> +			 */
> +			if (!force && buf->attr & BUF_ATTR_QUEUED)
> +				continue;
> +
>   			ret = iris_destroy_internal_buffer(inst, buf);
>   			if (ret)
>   				return ret;
> @@ -405,6 +413,16 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane)
>   	return 0;
>   }
>   
> +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane)
> +{
> +	return iris_destroy_internal_buffers(inst, plane, true);
> +}
> +
> +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane)
> +{
> +	return iris_destroy_internal_buffers(inst, plane, false);
> +}
> +
>   static int iris_release_internal_buffers(struct iris_inst *inst,
>   					 enum iris_buffer_type buffer_type)
>   {
> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h
> index c36b6347b077..00825ad2dc3a 100644
> --- a/drivers/media/platform/qcom/iris/iris_buffer.h
> +++ b/drivers/media/platform/qcom/iris/iris_buffer.h
> @@ -106,7 +106,8 @@ void iris_get_internal_buffers(struct iris_inst *inst, u32 plane);
>   int iris_create_internal_buffers(struct iris_inst *inst, u32 plane);
>   int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane);
>   int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer);
> -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane);
> +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane);
> +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane);
>   int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst);
>   int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst);
>   int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf);
> diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
> index 4143acedfc57..9c049b9671cc 100644
> --- a/drivers/media/platform/qcom/iris/iris_vdec.c
> +++ b/drivers/media/platform/qcom/iris/iris_vdec.c
> @@ -408,7 +408,7 @@ int iris_vdec_streamon_input(struct iris_inst *inst)
>   
>   	iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>   
> -	ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> +	ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>   	if (ret)
>   		return ret;
>   
> @@ -496,7 +496,7 @@ int iris_vdec_streamon_output(struct iris_inst *inst)
>   
>   	iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>   
> -	ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>   	if (ret)
>   		return ret;
>   
> diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
> index ca0f4e310f77..a8144595cc78 100644
> --- a/drivers/media/platform/qcom/iris/iris_vidc.c
> +++ b/drivers/media/platform/qcom/iris/iris_vidc.c
> @@ -221,6 +221,33 @@ static void iris_session_close(struct iris_inst *inst)
>   		iris_wait_for_session_response(inst, false);
>   }
>   
> +static void iris_check_num_queued_internal_buffers(struct iris_inst *inst, u32 plane)
> +{
> +	const struct iris_platform_data *platform_data = inst->core->iris_platform_data;
> +	struct iris_buffer *buf, *next;
> +	struct iris_buffers *buffers;
> +	const u32 *internal_buf_type;
> +	u32 internal_buffer_count, i;
> +	u32 count = 0;
> +
> +	if (V4L2_TYPE_IS_OUTPUT(plane)) {
> +		internal_buf_type = platform_data->dec_ip_int_buf_tbl;
> +		internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size;
> +	} else {
> +		internal_buf_type = platform_data->dec_op_int_buf_tbl;
> +		internal_buffer_count = platform_data->dec_op_int_buf_tbl_size;
> +	}
> +
> +	for (i = 0; i < internal_buffer_count; i++) {
> +		buffers = &inst->buffers[internal_buf_type[i]];
> +		list_for_each_entry_safe(buf, next, &buffers->list, list)
> +			count++;
> +		if (count)
> +			dev_err(inst->core->dev, "%d buffer of type %d not released",
> +				count, internal_buf_type[i]);
> +	}
> +}
> +
>   int iris_close(struct file *filp)
>   {
>   	struct iris_inst *inst = iris_get_inst(filp, NULL);
> @@ -233,8 +260,10 @@ int iris_close(struct file *filp)
>   	iris_session_close(inst);
>   	iris_inst_change_state(inst, IRIS_INST_DEINIT);
>   	iris_v4l2_fh_deinit(inst);
> -	iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> -	iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> +	iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> +	iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>   	iris_remove_session(inst);
>   	mutex_unlock(&inst->lock);
>   	mutex_destroy(&inst->ctx_q_lock);
> 

I left some comments in the previous cycle

https://lore.kernel.org/linux-arm-msm/a056266e-612d-4abf-916f-3db49b00dbde@linaro.org

I don't see those addressed in this cycle. Can you give some feedback 
from your POV.

---
bod

Re: [PATCH v3 03/23] media: iris: Avoid updating frame size to firmware during reconfig

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> During reconfig, the firmware sends the resolution aligned to 8 bytes.
> If the driver sends the same resolution back to the firmware the resolution
> will be aligned to 16 bytes not 8.
My question here is why there's an alignment mismatch between the APSS 
and firmware at all ?

---
bod

Re: [PATCH v3 05/23] media: iris: Prevent HFI queue writes when core is in deinit state

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> -	if (core->state == IRIS_CORE_ERROR)
> +	if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT)
>   		return -EINVAL;

Instead of checking for 2/3 of the states why not just check for the 1/3 ?

enum iris_core_state {
         IRIS_CORE_DEINIT,
         IRIS_CORE_INIT,
         IRIS_CORE_ERROR,
};

if (core->state != IRIS_CORE_INIT)
	return -EINVAL;

Cleaner and more explicit - declaring the state you must be in, as 
opposed to a list of states you should not be in.

Assuming you accept that suggested change:

Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

Re: [PATCH v3 06/23] media: iris: Remove deprecated property setting to firmware

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER is deprecated and no longer
> supported on current firmware, remove setting the same to firmware.

What about older firmware - what's the effect there ?

> At the same time, remove the check for non-zero number of v4l2 controls
> as some SOC might not expose any capability which requires v4l2 control.

Please break up patches like this one patch removing your legacy 
functionality - assuming that makes sense another patch amending your 
non-zero number of v4l2 controls.

Generally any commit log or patch title that requires "do this and do 
that" should be broken up @ the and.

---
bod

Re: [PATCH v3 07/23] media: iris: Fix missing function pointer initialization

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> The function pointers responsible for setting firmware properties were
> never initialized in the instance capability structure, causing it to
> remain NULL. As a result, the firmware properties were not being set
> correctly.
> 
> Fix this by properly assigning the function pointers from the core
> capability to the instance capability, ensuring that the properties are
> correctly applied to the firmware.
> 
> Cc: stable@vger.kernel.org
> Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon")
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_ctrls.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
> index 915de101fcba..13f5cf0d0e8a 100644
> --- a/drivers/media/platform/qcom/iris/iris_ctrls.c
> +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
> @@ -157,6 +157,7 @@ void iris_session_init_caps(struct iris_core *core)
>   		core->inst_fw_caps[cap_id].value = caps[i].value;
>   		core->inst_fw_caps[cap_id].flags = caps[i].flags;
>   		core->inst_fw_caps[cap_id].hfi_id = caps[i].hfi_id;
> +		core->inst_fw_caps[cap_id].set = caps[i].set;
>   	}
>   }
>   
> 
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

Re: [PATCH v3 08/23] media: iris: Fix NULL pointer dereference

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> A warning reported by smatch indicated a possible null pointer
> dereference where one of the arguments to API
> "iris_hfi_gen2_handle_system_error" could sometimes be null.
> 
> To fix this, add a check to validate that the argument passed is not
> null before accessing its members.
> 
> Cc: stable@vger.kernel.org
> Fixes: fb583a214337 ("media: iris: introduce host firmware interface with necessary hooks")
> Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
> Closes: https://lore.kernel.org/linux-media/634cc9b8-f099-4b54-8556-d879fb2b5169@stanley.mountain/
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> index d1a2a497a7b2..4488540d1d41 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> @@ -265,7 +265,8 @@ static int iris_hfi_gen2_handle_system_error(struct iris_core *core,
>   {
>   	struct iris_inst *instance;
>   
> -	dev_err(core->dev, "received system error of type %#x\n", pkt->type);
> +	if (pkt)
> +		dev_err(core->dev, "received system error of type %#x\n", pkt->type);
>   
>   	core->state = IRIS_CORE_ERROR;
>   
> 
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

Re: [PATCH v3 09/23] media: iris: Fix typo in depth variable

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> Correct a typo from "dpeth" to "depth".
> 
> Cc: stable@vger.kernel.org
> Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon")
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
> index a908b41e2868..802fa62c26eb 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
> @@ -178,7 +178,7 @@ static int iris_hfi_gen2_set_crop_offsets(struct iris_inst *inst)
>   						  sizeof(u64));
>   }
>   
> -static int iris_hfi_gen2_set_bit_dpeth(struct iris_inst *inst)
> +static int iris_hfi_gen2_set_bit_depth(struct iris_inst *inst)
>   {
>   	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
>   	u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
> @@ -378,7 +378,7 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p
>   		{HFI_PROP_BITSTREAM_RESOLUTION,       iris_hfi_gen2_set_bitstream_resolution   },
>   		{HFI_PROP_CROP_OFFSETS,               iris_hfi_gen2_set_crop_offsets           },
>   		{HFI_PROP_CODED_FRAMES,               iris_hfi_gen2_set_coded_frames           },
> -		{HFI_PROP_LUMA_CHROMA_BIT_DEPTH,      iris_hfi_gen2_set_bit_dpeth              },
> +		{HFI_PROP_LUMA_CHROMA_BIT_DEPTH,      iris_hfi_gen2_set_bit_depth              },
>   		{HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, iris_hfi_gen2_set_min_output_count       },
>   		{HFI_PROP_PIC_ORDER_CNT_TYPE,         iris_hfi_gen2_set_picture_order_count    },
>   		{HFI_PROP_SIGNAL_COLOR_INFO,          iris_hfi_gen2_set_colorspace             },
> 
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

Re: [PATCH v3 10/23] media: iris: Track flush responses to prevent premature completion

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> Currently, two types of flush commands are queued to the firmware,
> the first flush queued as part of sequence change, does not wait for a
> response, while the second flush queued as part of stop, expects a
> completion response before proceeding further.
> 
> Due to timing issue, the flush response corresponding to the first
> command could arrive after the second flush is issued. This casuses the
> driver to incorrectly assume that the second flush has completed,
> leading to the premature signaling of flush_completion.
> 
> To address this, introduce a counter to track the number of pending
> flush responses and signal flush completion only when all expected
> responses are received.

Is there no other way to correlate command/response than a counter on 
the APSS side ?

Usually command/response protocols have some kind of identifier field or 
sequence number embedded in the protocol headers.

---
bod

Re: [PATCH v3 13/23] media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> Firmware sends buffers with 0 filled length which needs to be dropped,
> to achieve the same, add V4L2_BUF_FLAG_ERROR to such buffers.
> Also make sure:
> - These 0 length buffers are not returned as result of flush.
> - Its not a buffer with LAST flag enabled which will also have 0 filled
>    length.
> 
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> index 4488540d1d41..3bb326843a7b 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> @@ -378,6 +378,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst,
>   
>   	buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, hfi_buffer->flags);
>   
> +	if (!buf->data_size && inst->state == IRIS_INST_STREAMING &&
> +	    !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) &&
> +	    !(inst->sub_state & IRIS_INST_SUB_DRC)) {
> +		buf->flags |= V4L2_BUF_FLAG_ERROR;
> +	}
> +
>   	return 0;
>   }
>   
> 

This is a pretty complex conjunctive clause.

Is it possible for say 1/3 of these logical criteria to be false ?

i.e. if you get to:

1. buf->data_size && inst->state == IRIS_INST_STREAMING
2. !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST)

is it possible to get to

3. !(inst->sub_state & IRIS_INST_SUB_DRC)

?

This also feels like a bugfix ?

---
bod

Re: [PATCH v3 14/23] media: iris: Add handling for no show frames

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> Firmware sends the picture type as NO_SHOW for frames which are not
> supposed to be displayed, add handling for the same in driver to drop
> them.
> 
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h  | 1 +
>   drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 4 +++-
>   2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> index 806f8bb7f505..666061a612c3 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
> @@ -113,6 +113,7 @@ enum hfi_picture_type {
>   	HFI_PICTURE_I				= 0x00000008,
>   	HFI_PICTURE_CRA				= 0x00000010,
>   	HFI_PICTURE_BLA				= 0x00000020,
> +	HFI_PICTURE_NOSHOW			= 0x00000040,
>   };
>   
>   enum hfi_buffer_type {
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> index 3bb326843a7b..2267e220c9ea 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> @@ -91,7 +91,9 @@ static int iris_hfi_gen2_get_driver_buffer_flags(struct iris_inst *inst, u32 hfi
>   	struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
>   	u32 driver_flags = 0;
>   
> -	if (inst_hfi_gen2->hfi_frame_info.picture_type & keyframe)
> +	if (inst_hfi_gen2->hfi_frame_info.picture_type & HFI_PICTURE_NOSHOW)
> +		driver_flags |= V4L2_BUF_FLAG_ERROR;
> +	else if (inst_hfi_gen2->hfi_frame_info.picture_type & keyframe)
>   		driver_flags |= V4L2_BUF_FLAG_KEYFRAME;
>   	else if (inst_hfi_gen2->hfi_frame_info.picture_type & HFI_PICTURE_P)
>   		driver_flags |= V4L2_BUF_FLAG_PFRAME;
> 
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

Re: [PATCH v3 01/23] media: iris: Skip destroying internal buffer if not dequeued

[Dikshita Agarwal]

On 5/2/2025 5:44 PM, Bryan O'Donoghue wrote:
> On 01/05/2025 20:13, Dikshita Agarwal wrote:
>> Firmware might hold the DPB buffers for reference in case of sequence
>> change, so skip destroying buffers for which QUEUED flag is not removed.
>> Also, make sure that all buffers are released during streamoff.
>>
>> Cc: stable@vger.kernel.org
>> Fixes: 73702f45db81 ("media: iris: allocate, initialize and queue
>> internal buffers")
>> Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>>   drivers/media/platform/qcom/iris/iris_buffer.c | 20 +++++++++++++++-
>>   drivers/media/platform/qcom/iris/iris_buffer.h |  3 ++-
>>   drivers/media/platform/qcom/iris/iris_vdec.c   |  4 ++--
>>   drivers/media/platform/qcom/iris/iris_vidc.c   | 33
>> ++++++++++++++++++++++++--
>>   4 files changed, 54 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c
>> b/drivers/media/platform/qcom/iris/iris_buffer.c
>> index e5c5a564fcb8..981fedb000ed 100644
>> --- a/drivers/media/platform/qcom/iris/iris_buffer.c
>> +++ b/drivers/media/platform/qcom/iris/iris_buffer.c
>> @@ -376,7 +376,7 @@ int iris_destroy_internal_buffer(struct iris_inst
>> *inst, struct iris_buffer *buf
>>       return 0;
>>   }
>>   -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane)
>> +static int iris_destroy_internal_buffers(struct iris_inst *inst, u32
>> plane, bool force)
>>   {
>>       const struct iris_platform_data *platform_data =
>> inst->core->iris_platform_data;
>>       struct iris_buffer *buf, *next;
>> @@ -396,6 +396,14 @@ int iris_destroy_internal_buffers(struct iris_inst
>> *inst, u32 plane)
>>       for (i = 0; i < len; i++) {
>>           buffers = &inst->buffers[internal_buf_type[i]];
>>           list_for_each_entry_safe(buf, next, &buffers->list, list) {
>> +            /*
>> +             * during stream on, skip destroying internal(DPB) buffer
>> +             * if firmware did not return it.
>> +             * during close, destroy all buffers irrespectively.
>> +             */
>> +            if (!force && buf->attr & BUF_ATTR_QUEUED)
>> +                continue;
>> +
>>               ret = iris_destroy_internal_buffer(inst, buf);
>>               if (ret)
>>                   return ret;
>> @@ -405,6 +413,16 @@ int iris_destroy_internal_buffers(struct iris_inst
>> *inst, u32 plane)
>>       return 0;
>>   }
>>   +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane)
>> +{
>> +    return iris_destroy_internal_buffers(inst, plane, true);
>> +}
>> +
>> +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32
>> plane)
>> +{
>> +    return iris_destroy_internal_buffers(inst, plane, false);
>> +}
>> +
>>   static int iris_release_internal_buffers(struct iris_inst *inst,
>>                        enum iris_buffer_type buffer_type)
>>   {
>> diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h
>> b/drivers/media/platform/qcom/iris/iris_buffer.h
>> index c36b6347b077..00825ad2dc3a 100644
>> --- a/drivers/media/platform/qcom/iris/iris_buffer.h
>> +++ b/drivers/media/platform/qcom/iris/iris_buffer.h
>> @@ -106,7 +106,8 @@ void iris_get_internal_buffers(struct iris_inst
>> *inst, u32 plane);
>>   int iris_create_internal_buffers(struct iris_inst *inst, u32 plane);
>>   int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane);
>>   int iris_destroy_internal_buffer(struct iris_inst *inst, struct
>> iris_buffer *buffer);
>> -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane);
>> +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane);
>> +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32
>> plane);
>>   int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst);
>>   int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst);
>>   int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf);
>> diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c
>> b/drivers/media/platform/qcom/iris/iris_vdec.c
>> index 4143acedfc57..9c049b9671cc 100644
>> --- a/drivers/media/platform/qcom/iris/iris_vdec.c
>> +++ b/drivers/media/platform/qcom/iris/iris_vdec.c
>> @@ -408,7 +408,7 @@ int iris_vdec_streamon_input(struct iris_inst *inst)
>>         iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>>   -    ret = iris_destroy_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> +    ret = iris_destroy_dequeued_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>>       if (ret)
>>           return ret;
>>   @@ -496,7 +496,7 @@ int iris_vdec_streamon_output(struct iris_inst *inst)
>>         iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>>   -    ret = iris_destroy_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>> +    ret = iris_destroy_dequeued_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>>       if (ret)
>>           return ret;
>>   diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c
>> b/drivers/media/platform/qcom/iris/iris_vidc.c
>> index ca0f4e310f77..a8144595cc78 100644
>> --- a/drivers/media/platform/qcom/iris/iris_vidc.c
>> +++ b/drivers/media/platform/qcom/iris/iris_vidc.c
>> @@ -221,6 +221,33 @@ static void iris_session_close(struct iris_inst *inst)
>>           iris_wait_for_session_response(inst, false);
>>   }
>>   +static void iris_check_num_queued_internal_buffers(struct iris_inst
>> *inst, u32 plane)
>> +{
>> +    const struct iris_platform_data *platform_data =
>> inst->core->iris_platform_data;
>> +    struct iris_buffer *buf, *next;
>> +    struct iris_buffers *buffers;
>> +    const u32 *internal_buf_type;
>> +    u32 internal_buffer_count, i;
>> +    u32 count = 0;
>> +
>> +    if (V4L2_TYPE_IS_OUTPUT(plane)) {
>> +        internal_buf_type = platform_data->dec_ip_int_buf_tbl;
>> +        internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size;
>> +    } else {
>> +        internal_buf_type = platform_data->dec_op_int_buf_tbl;
>> +        internal_buffer_count = platform_data->dec_op_int_buf_tbl_size;
>> +    }
>> +
>> +    for (i = 0; i < internal_buffer_count; i++) {
>> +        buffers = &inst->buffers[internal_buf_type[i]];
>> +        list_for_each_entry_safe(buf, next, &buffers->list, list)
>> +            count++;
>> +        if (count)
>> +            dev_err(inst->core->dev, "%d buffer of type %d not released",
>> +                count, internal_buf_type[i]);
>> +    }
>> +}
>> +
>>   int iris_close(struct file *filp)
>>   {
>>       struct iris_inst *inst = iris_get_inst(filp, NULL);
>> @@ -233,8 +260,10 @@ int iris_close(struct file *filp)
>>       iris_session_close(inst);
>>       iris_inst_change_state(inst, IRIS_INST_DEINIT);
>>       iris_v4l2_fh_deinit(inst);
>> -    iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> -    iris_destroy_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>> +    iris_destroy_all_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> +    iris_destroy_all_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>> +    iris_check_num_queued_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
>> +    iris_check_num_queued_internal_buffers(inst,
>> V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
>>       iris_remove_session(inst);
>>       mutex_unlock(&inst->lock);
>>       mutex_destroy(&inst->ctx_q_lock);
>>
> 
> I left some comments in the previous cycle
> 
> https://lore.kernel.org/linux-arm-msm/a056266e-612d-4abf-916f-3db49b00dbde@linaro.org
> 
> I don't see those addressed in this cycle. Can you give some feedback from
> your POV.
> 
I responded to your comments, pls check here
https://lore.kernel.org/linux-arm-msm/2c431232-e0d6-1e6c-cd22-a912b5f08f7a@quicinc.com/

Thanks,
Dikshita
> ---
> bod

Re: [PATCH v3 10/23] media: iris: Track flush responses to prevent premature completion

[Dikshita Agarwal]

On 5/2/2025 6:10 PM, Bryan O'Donoghue wrote:
> On 01/05/2025 20:13, Dikshita Agarwal wrote:
>> Currently, two types of flush commands are queued to the firmware,
>> the first flush queued as part of sequence change, does not wait for a
>> response, while the second flush queued as part of stop, expects a
>> completion response before proceeding further.
>>
>> Due to timing issue, the flush response corresponding to the first
>> command could arrive after the second flush is issued. This casuses the
>> driver to incorrectly assume that the second flush has completed,
>> leading to the premature signaling of flush_completion.
>>
>> To address this, introduce a counter to track the number of pending
>> flush responses and signal flush completion only when all expected
>> responses are received.
> 
> Is there no other way to correlate command/response than a counter on the
> APSS side ?
> 
> Usually command/response protocols have some kind of identifier field or
> sequence number embedded in the protocol headers.
> 
There is no such identifier field for flush command and response at the moment.

Thanks,
Dikshita
> ---
> bod

Re: [PATCH v3 06/23] media: iris: Remove deprecated property setting to firmware

[Dikshita Agarwal]

On 5/2/2025 5:57 PM, Bryan O'Donoghue wrote:
> On 01/05/2025 20:13, Dikshita Agarwal wrote:
>> HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER is deprecated and no longer
>> supported on current firmware, remove setting the same to firmware.
> 
> What about older firmware - what's the effect there ?
if you are referring to firmware being used in venus, its the same.
This property was never supported on SM8250 firmware.
> 
>> At the same time, remove the check for non-zero number of v4l2 controls
>> as some SOC might not expose any capability which requires v4l2 control.
> 
> Please break up patches like this one patch removing your legacy
> functionality - assuming that makes sense another patch amending your
> non-zero number of v4l2 controls.
> 
Ok, will split.

Thanks,
Dikshita
> Generally any commit log or patch title that requires "do this and do that"
> should be broken up @ the and.
> 
> ---
> bod

Re: [PATCH v3 03/23] media: iris: Avoid updating frame size to firmware during reconfig

[Dikshita Agarwal]

On 5/2/2025 5:48 PM, Bryan O'Donoghue wrote:
> On 01/05/2025 20:13, Dikshita Agarwal wrote:
>> During reconfig, the firmware sends the resolution aligned to 8 bytes.
>> If the driver sends the same resolution back to the firmware the resolution
>> will be aligned to 16 bytes not 8.
> My question here is why there's an alignment mismatch between the APSS and
> firmware at all ?
> 
It's not about the mismatch between APPS and firmware.
It's about how firmware handles this, it's not expecting the driver to set
the resolution after reconfig.

Thanks,
Dikshita
> ---
> bod

Re: [PATCH v3 05/23] media: iris: Prevent HFI queue writes when core is in deinit state

[Konrad Dybcio]

On 5/2/25 2:22 PM, Bryan O'Donoghue wrote:
> On 01/05/2025 20:13, Dikshita Agarwal wrote:
>> -    if (core->state == IRIS_CORE_ERROR)
>> +    if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT)
>>           return -EINVAL;
> 
> Instead of checking for 2/3 of the states why not just check for the 1/3 ?
> 
> enum iris_core_state {
>         IRIS_CORE_DEINIT,
>         IRIS_CORE_INIT,
>         IRIS_CORE_ERROR,
> };
> 
> if (core->state != IRIS_CORE_INIT)
>     return -EINVAL;
> 
> Cleaner and more explicit - declaring the state you must be in, as opposed to a list of states you should not be in.

Being explicit in state machines helps maintainability - if we get
e.g. IRIS_CORE_LIGHT_SLEEP down the line, this could easily fail

Konrad

Re: [PATCH v3 13/23] media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length

[Dikshita Agarwal]

On 5/2/2025 6:19 PM, Bryan O'Donoghue wrote:
> On 01/05/2025 20:13, Dikshita Agarwal wrote:
>> Firmware sends buffers with 0 filled length which needs to be dropped,
>> to achieve the same, add V4L2_BUF_FLAG_ERROR to such buffers.
>> Also make sure:
>> - These 0 length buffers are not returned as result of flush.
>> - Its not a buffer with LAST flag enabled which will also have 0 filled
>>    length.
>>
>> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>>   drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 6 ++++++
>>   1 file changed, 6 insertions(+)
>>
>> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> index 4488540d1d41..3bb326843a7b 100644
>> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> @@ -378,6 +378,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct
>> iris_inst *inst,
>>         buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst,
>> hfi_buffer->flags);
>>   +    if (!buf->data_size && inst->state == IRIS_INST_STREAMING &&
>> +        !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) &&
>> +        !(inst->sub_state & IRIS_INST_SUB_DRC)) {
>> +        buf->flags |= V4L2_BUF_FLAG_ERROR;
>> +    }
>> +
>>       return 0;
>>   }
>>  
> 
> This is a pretty complex conjunctive clause.
> 
> Is it possible for say 1/3 of these logical criteria to be false ?
> 
> i.e. if you get to:
> 
> 1. buf->data_size && inst->state == IRIS_INST_STREAMING
This makes sure that the 0 length buffers are not returned as response to
flush.
> 2. !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST)
This makes sure that we don't mark the buffer with last flag as error since
this will be also be a zero length buffer but a valid one.
> 
> is it possible to get to
> 
> 3. !(inst->sub_state & IRIS_INST_SUB_DRC)
I can remove this condition as first one captures the essence of this one.

Thanks,
Dikshita
> 
> ?
> 
> This also feels like a bugfix ?
> 
> ---
> bod

Re: [PATCH v3 15/23] media: iris: Improve last flag handling

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> Improve the handling of the V4L2_BUF_FLAG_LAST flag in the driver:
> - Ensure that the last flag is not sent multiple times.
> - Attach the last flag to the first capture buffer returned during
>    flush, triggered by a sequence change, addressing cases where the
>    firmware does not set the last flag.
> 
> Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
I'm sure this is logically correct, I just wonder if you could represent 
it as a inst->state instead of as a flag ?

Up to you to think about.

Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

Re: [PATCH v3 18/23] media: iris: Add a comment to explain usage of MBPS

[Bryan O'Donoghue]

On 01/05/2025 20:13, Dikshita Agarwal wrote:
> Add a comment to explain usage of MBPS and define a macro for 8K
> resolution for better readability
> 
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_platform_common.h | 2 ++
>   drivers/media/platform/qcom/iris/iris_platform_gen2.c   | 4 ++--
>   drivers/media/platform/qcom/iris/iris_platform_sm8250.c | 2 +-
>   3 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
> index 1dab276431c7..3e0ae87526a0 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_common.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
> @@ -21,6 +21,7 @@ struct iris_inst;
>   #define DEFAULT_MAX_HOST_BUF_COUNT		64
>   #define DEFAULT_MAX_HOST_BURST_BUF_COUNT	256
>   #define DEFAULT_FPS				30
> +#define NUM_MBS_8K				((8192 * 4352) / 256)
>   
>   enum stage_type {
>   	STAGE_1 = 1,
> @@ -172,6 +173,7 @@ struct iris_platform_data {
>   	struct ubwc_config_data *ubwc_config;
>   	u32 num_vpp_pipe;
>   	u32 max_session_count;
> +	/* max number of macroblocks per frame supported */
>   	u32 max_core_mbpf;
>   	const u32 *input_config_params;
>   	unsigned int input_config_params_size;
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> index 1e69ba15db0f..deb7037e8e86 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
> @@ -248,7 +248,7 @@ struct iris_platform_data sm8550_data = {
>   	.ubwc_config = &ubwc_config_sm8550,
>   	.num_vpp_pipe = 4,
>   	.max_session_count = 16,
> -	.max_core_mbpf = ((8192 * 4352) / 256) * 2,
> +	.max_core_mbpf = NUM_MBS_8K * 2,
>   	.input_config_params =
>   		sm8550_vdec_input_config_params,
>   	.input_config_params_size =
> @@ -308,7 +308,7 @@ struct iris_platform_data sm8650_data = {
>   	.ubwc_config = &ubwc_config_sm8550,
>   	.num_vpp_pipe = 4,
>   	.max_session_count = 16,
> -	.max_core_mbpf = ((8192 * 4352) / 256) * 2,
> +	.max_core_mbpf = NUM_MBS_8K * 2,
>   	.input_config_params =
>   		sm8550_vdec_input_config_params,
>   	.input_config_params_size =
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> index 543fa2661539..8183e4e95fa4 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c
> @@ -127,7 +127,7 @@ struct iris_platform_data sm8250_data = {
>   	.hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE,
>   	.num_vpp_pipe = 4,
>   	.max_session_count = 16,
> -	.max_core_mbpf = (8192 * 4352) / 256,
> +	.max_core_mbpf = NUM_MBS_8K,
>   	.input_config_params =
>   		sm8250_vdec_input_config_param_default,
>   	.input_config_params_size =
> 

Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

Re: [PATCH v3 13/23] media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length

[Nicolas Dufresne]

Hi Dikshita,

Le vendredi 02 mai 2025 à 00:43 +0530, Dikshita Agarwal a écrit :
> Firmware sends buffers with 0 filled length which needs to be dropped,
> to achieve the same, add V4L2_BUF_FLAG_ERROR to such buffers.
> Also make sure:
> - These 0 length buffers are not returned as result of flush.
> - Its not a buffer with LAST flag enabled which will also have 0 filled
>   length.

This message is quite vague, is this about capture or output buffers ?
If its output buffers that don't produce capture, I don't see why they
have to be flagged as errors, or why the payload size matter. Then, if
its about assigned capture buffers that did not get used in the end, you
should put them back in the queue instead of returning them to user
space.

Returning a capture buffers to userspace should only be used if a frame
could not be produced. That imply copying the cookie timestamp from the
src buffers into the capture buffer. Please make sure you don't endup
returning fake erorrs to userspace, which may lead to some frame
metadata being dropped erroneously.

Nicolas

> 
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>  drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> index 4488540d1d41..3bb326843a7b 100644
> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> @@ -378,6 +378,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst,
>  
>  	buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, hfi_buffer->flags);
>  
> +	if (!buf->data_size && inst->state == IRIS_INST_STREAMING &&
> +	    !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) &&
> +	    !(inst->sub_state & IRIS_INST_SUB_DRC)) {
> +		buf->flags |= V4L2_BUF_FLAG_ERROR;
> +	}
> +
>  	return 0;
>  }
>  

Re: [PATCH v3 13/23] media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length

[Dikshita Agarwal]

On 5/3/2025 9:39 PM, Nicolas Dufresne wrote:
> Hi Dikshita,
> 
> Le vendredi 02 mai 2025 à 00:43 +0530, Dikshita Agarwal a écrit :
>> Firmware sends buffers with 0 filled length which needs to be dropped,
>> to achieve the same, add V4L2_BUF_FLAG_ERROR to such buffers.
>> Also make sure:
>> - These 0 length buffers are not returned as result of flush.
>> - Its not a buffer with LAST flag enabled which will also have 0 filled
>>   length.
> 
> This message is quite vague, is this about capture or output buffers ?
> If its output buffers that don't produce capture, I don't see why they
> have to be flagged as errors, or why the payload size matter. Then, if
> its about assigned capture buffers that did not get used in the end, you
> should put them back in the queue instead of returning them to user
> space.
> 
> Returning a capture buffers to userspace should only be used if a frame
> could not be produced. That imply copying the cookie timestamp from the
> src buffers into the capture buffer. Please make sure you don't endup
> returning fake erorrs to userspace, which may lead to some frame
> metadata being dropped erroneously.
> 
The capture buffers which I am trying to handle here are of 0 byteused
which means they don't have any valid data and they have the timestamp
copied from src buffers.
How these buffers will be handled by client? if we don't associate error
flag to such buffers?

Thanks,
Dikshita
> Nicolas
> 
>>
>> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>> ---
>>  drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> index 4488540d1d41..3bb326843a7b 100644
>> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>> @@ -378,6 +378,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst,
>>  
>>  	buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, hfi_buffer->flags);
>>  
>> +	if (!buf->data_size && inst->state == IRIS_INST_STREAMING &&
>> +	    !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) &&
>> +	    !(inst->sub_state & IRIS_INST_SUB_DRC)) {
>> +		buf->flags |= V4L2_BUF_FLAG_ERROR;
>> +	}
>> +
>>  	return 0;
>>  }
>>  

Re: [PATCH v3 13/23] media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length

[Nicolas Dufresne]

Hi Dikshita,

Le dimanche 04 mai 2025 à 20:53 +0530, Dikshita Agarwal a écrit :
> 
> 
> On 5/3/2025 9:39 PM, Nicolas Dufresne wrote:
> > Hi Dikshita,
> > 
> > Le vendredi 02 mai 2025 à 00:43 +0530, Dikshita Agarwal a écrit :
> > > Firmware sends buffers with 0 filled length which needs to be dropped,
> > > to achieve the same, add V4L2_BUF_FLAG_ERROR to such buffers.
> > > Also make sure:
> > > - These 0 length buffers are not returned as result of flush.
> > > - Its not a buffer with LAST flag enabled which will also have 0 filled
> > >   length.
> > 
> > This message is quite vague, is this about capture or output buffers ?
> > If its output buffers that don't produce capture, I don't see why they
> > have to be flagged as errors, or why the payload size matter. Then, if
> > its about assigned capture buffers that did not get used in the end, you
> > should put them back in the queue instead of returning them to user
> > space.
> > 
> > Returning a capture buffers to userspace should only be used if a frame
> > could not be produced. That imply copying the cookie timestamp from the
> > src buffers into the capture buffer. Please make sure you don't endup
> > returning fake erorrs to userspace, which may lead to some frame
> > metadata being dropped erroneously.
> > 
> The capture buffers which I am trying to handle here are of 0 byteused
> which means they don't have any valid data and they have the timestamp
> copied from src buffers.
> How these buffers will be handled by client? if we don't associate error
> flag to such buffers?

Please share a link to the stream and specify which frames are handled
this way by your firmware. The answer to your question is entirely
dependent on the stream you are decoding.

Nicolas

> 
> Thanks,
> Dikshita
> > Nicolas
> > 
> > > 
> > > Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> > > Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> > > ---
> > >  drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 6 ++++++
> > >  1 file changed, 6 insertions(+)
> > > 
> > > diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> > > b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> > > index 4488540d1d41..3bb326843a7b 100644
> > > --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> > > +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
> > > @@ -378,6 +378,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst,
> > >  
> > >  	buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, hfi_buffer->flags);
> > >  
> > > +	if (!buf->data_size && inst->state == IRIS_INST_STREAMING &&
> > > +	    !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) &&
> > > +	    !(inst->sub_state & IRIS_INST_SUB_DRC)) {
> > > +		buf->flags |= V4L2_BUF_FLAG_ERROR;
> > > +	}
> > > +
> > >  	return 0;
> > >  }
> > >  

Re: [PATCH v3 22/23] media: iris: Add internal buffer calculation for HEVC and VP9 decoders

[Neil Armstrong]

On 01/05/2025 21:13, Dikshita Agarwal wrote:
> Add internal buffer count and size calculations for HEVC and VP9
> decoders.
> 
> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
> ---
>   drivers/media/platform/qcom/iris/iris_buffer.c     |   3 +
>   drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++-
>   drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>   3 files changed, 432 insertions(+), 14 deletions(-)
> 

<snip>

> diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
> index 62af6ea6ba1f..2272f0c21683 100644
> --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
> +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
> @@ -13,6 +13,10 @@ struct iris_inst;
>   #define DMA_ALIGNMENT			256
>   
>   #define NUM_HW_PIC_BUF			32
> +#define LCU_MAX_SIZE_PELS 64
> +#define LCU_MIN_SIZE_PELS 16
> +#define HDR10_HIST_EXTRADATA_SIZE (4 * 1024)
> +
>   #define SIZE_HW_PIC(size_per_buf)	(NUM_HW_PIC_BUF * (size_per_buf))
>   
>   #define MAX_TILE_COLUMNS		32
> @@ -28,11 +32,47 @@ struct iris_inst;
>   #define SIZE_SLIST_BUF_H264		512
>   #define H264_DISPLAY_BUF_SIZE		3328
>   #define H264_NUM_FRM_INFO		66
> -
> -#define SIZE_SEI_USERDATA		4096
> -
> +#define H265_NUM_TILE_COL 32
> +#define H265_NUM_TILE_ROW 12

This one should be 128, in fact it was right on v2 and seems itr got wrong on v3.

Grepping in downstream also shows 128:
$ grep H265_NUM_TILE_ROW ./ -R
./driver/variant/iris33/inc/hfi_buffer_iris33.h:#define H265_NUM_TILE_ROW 128
./driver/variant/iris33/inc/hfi_buffer_iris33.h:#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
./driver/variant/iris3/inc/hfi_buffer_iris3.h:#define H265_NUM_TILE_ROW 128
./driver/variant/iris3/inc/hfi_buffer_iris3.h:#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
./driver/variant/iris2/inc/hfi_buffer_iris2.h:#define H265_NUM_TILE_ROW 128
./driver/variant/iris2/inc/hfi_buffer_iris2.h:#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)

And fixing to to 128 makes HEVC works flawlessly on SM8650.

Neil

> +#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
> +#define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32))
> +
> +#define NUM_SLIST_BUF_H265 (80 + 20)
> +#define SIZE_SLIST_BUF_H265 (BIT(10))
> +#define H265_DISPLAY_BUF_SIZE (3072)
> +#define H265_NUM_FRM_INFO (48)
> +
> +#define VP9_NUM_FRAME_INFO_BUF 32
> +#define VP9_NUM_PROBABILITY_TABLE_BUF (VP9_NUM_FRAME_INFO_BUF + 4)
> +#define VP9_PROB_TABLE_SIZE (3840)
> +#define VP9_FRAME_INFO_BUF_SIZE (6144)
> +#define BUFFER_ALIGNMENT_32_BYTES 32
> +#define CCE_TILE_OFFSET_SIZE ALIGN(32 * 4 * 4, BUFFER_ALIGNMENT_32_BYTES)
> +#define MAX_SUPERFRAME_HEADER_LEN (34)
> +#define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 64
> +#define MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE 64
> +#define MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE 64
> +#define MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE (128 / 8)
> +#define MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE (128 / 8)
> +#define VP9_UDC_HEADER_BUF_SIZE	(3 * 128)
> +
> +#define SIZE_SEI_USERDATA			4096
> +#define SIZE_DOLBY_RPU_METADATA (41 * 1024)
>   #define H264_CABAC_HDR_RATIO_HD_TOT	1
>   #define H264_CABAC_RES_RATIO_HD_TOT	3
> +#define H265D_MAX_SLICE	1200
> +#define SIZE_H265D_HW_PIC_T SIZE_H264D_HW_PIC_T
> +#define H265_CABAC_HDR_RATIO_HD_TOT 2
> +#define H265_CABAC_RES_RATIO_HD_TOT 2
> +#define SIZE_H265D_VPP_CMD_PER_BUF (256)
> +
> +#define VPX_DECODER_FRAME_CONCURENCY_LVL (2)
> +#define VPX_DECODER_FRAME_BIN_HDR_BUDGET 1
> +#define VPX_DECODER_FRAME_BIN_RES_BUDGET 3
> +#define VPX_DECODER_FRAME_BIN_DENOMINATOR 2
> +
> +#define VPX_DECODER_FRAME_BIN_RES_BUDGET_RATIO (3 / 2)
> +
>   #define SIZE_H264D_HW_PIC_T		(BIT(11))
>   
>   #define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE	64
> 

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[neil.armstrong]

On 02/05/2025 09:34, Dikshita Agarwal wrote:
> 
> 
> On 5/2/2025 12:55 PM, Neil Armstrong wrote:
>> Hi,
>>
>> On 01/05/2025 21:13, Dikshita Agarwal wrote:
>>> Hi All,
>>>
>>> This patch series adds initial support for the HEVC(H.265) and VP9
>>> codecs in iris decoder. The objective of this work is to extend the
>>> decoder's capabilities to handle HEVC and VP9 codec streams,
>>> including necessary format handling and buffer management.
>>> In addition, the series also includes a set of fixes to address issues
>>> identified during testing of these additional codecs.
>>>
>>> These patches also address the comments and feedback received from the
>>> RFC patches previously sent. I have made the necessary improvements
>>> based on the community's suggestions.
>>>
>>> Changes in v3:
>>> - Introduced two wrappers with explicit names to handle destroy internal
>>> buffers (Nicolas)
>>> - Used sub state check instead of introducing new boolean (Vikash)
>>> - Addressed other comments (Vikash)
>>> - Reorderd patches to have all fixes patches first (Dmitry)
>>> - Link to v2:
>>> https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
>>>
>>> Changes in v2:
>>> - Added Changes to make sure all buffers are released in session close
>>> (bryna)
>>> - Added tracking for flush responses to fix a timing issue.
>>> - Added a handling to fix timing issue in reconfig
>>> - Splitted patch 06/20 in two patches (Bryan)
>>> - Added missing fixes tag (bryan)
>>> - Updated fluster report (Nicolas)
>>> - Link to v1:
>>> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
>>>
>>> Changes sinces RFC:
>>> - Added additional fixes to address issues identified during further
>>> testing.
>>> - Moved typo fix to a seperate patch [Neil]
>>> - Reordered the patches for better logical flow and clarity [Neil,
>>> Dmitry]
>>> - Added fixes tag wherever applicable [Neil, Dmitry]
>>> - Removed the default case in the switch statement for codecs [Bryan]
>>> - Replaced if-else statements with switch-case [Bryan]
>>> - Added comments for mbpf [Bryan]
>>> - RFC:
>>> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
>>>
>>> This patch series depends on [1] & [2]
>>> [1]
>>> https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
>>> [2]
>>> https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
>>>
>>> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
>>> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
>>> the existing H264 decoder functionality remains uneffected.
>>>
>>> Note: 1 of the fluster compliance test is fixed with firmware [3]
>>> [3]:
>>> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
>>>
>>> The result of fluster test on SM8550:
>>>    131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>    GStreamer-H.265-V4L2-Gst1.0.
>>>    The failing test case:
>>>    - 10 testcases failed due to unsupported 10 bit format.
>>>      - DBLK_A_MAIN10_VIXS_4
>>>      - INITQP_B_Main10_Sony_1
>>>      - TSUNEQBD_A_MAIN10_Technicolor_2
>>>      - WP_A_MAIN10_Toshiba_3
>>>      - WP_MAIN10_B_Toshiba_3
>>>      - WPP_A_ericsson_MAIN10_2
>>>      - WPP_B_ericsson_MAIN10_2
>>>      - WPP_C_ericsson_MAIN10_2
>>>      - WPP_E_ericsson_MAIN10_2
>>>      - WPP_F_ericsson_MAIN10_2
>>>    - 4 testcase failed due to unsupported resolution
>>>      - PICSIZE_A_Bossen_1
>>>      - PICSIZE_B_Bossen_1
>>>      - WPP_D_ericsson_MAIN10_2
>>>      - WPP_D_ericsson_MAIN_2
>>>    - 2 testcase failed due to CRC mismatch
>>>      - RAP_A_docomo_6
>>>      - RAP_B_Bossen_2
>>>      - BUG reported:
>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>>>        Analysis - First few frames in this discarded by firmware and are
>>>        sent to driver with 0 filled length. Driver send such buffers to
>>>        client with timestamp 0 and payload set to 0 and
>>>        make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>>>        dropped by GST. But instead, the first frame displayed as green
>>>        frame and when a valid buffer is sent to client later with same 0
>>>        timestamp, its dropped, leading to CRC mismatch for first frame.
>>>
>>>    235/305 testcases passed while testing VP9-TEST-VECTORS with
>>>    GStreamer-VP9-V4L2-Gst1.0.
>>>    The failing test case:
>>>    - 64 testcases failed due to unsupported resolution
>>>      - vp90-2-02-size-08x08.webm
>>>      - vp90-2-02-size-08x10.webm
>>>      - vp90-2-02-size-08x16.webm
>>>      - vp90-2-02-size-08x18.webm
>>>      - vp90-2-02-size-08x32.webm
>>>      - vp90-2-02-size-08x34.webm
>>>      - vp90-2-02-size-08x64.webm
>>>      - vp90-2-02-size-08x66.webm
>>>      - vp90-2-02-size-10x08.webm
>>>      - vp90-2-02-size-10x10.webm
>>>      - vp90-2-02-size-10x16.webm
>>>      - vp90-2-02-size-10x18.webm
>>>      - vp90-2-02-size-10x32.webm
>>>      - vp90-2-02-size-10x34.webm
>>>      - vp90-2-02-size-10x64.webm
>>>      - vp90-2-02-size-10x66.webm
>>>      - vp90-2-02-size-16x08.webm
>>>      - vp90-2-02-size-16x10.webm
>>>      - vp90-2-02-size-16x16.webm
>>>      - vp90-2-02-size-16x18.webm
>>>      - vp90-2-02-size-16x32.webm
>>>      - vp90-2-02-size-16x34.webm
>>>      - vp90-2-02-size-16x64.webm
>>>      - vp90-2-02-size-16x66.webm
>>>      - vp90-2-02-size-18x08.webm
>>>      - vp90-2-02-size-18x10.webm
>>>      - vp90-2-02-size-18x16.webm
>>>      - vp90-2-02-size-18x18.webm
>>>      - vp90-2-02-size-18x32.webm
>>>      - vp90-2-02-size-18x34.webm
>>>      - vp90-2-02-size-18x64.webm
>>>      - vp90-2-02-size-18x66.webm
>>>      - vp90-2-02-size-32x08.webm
>>>      - vp90-2-02-size-32x10.webm
>>>      - vp90-2-02-size-32x16.webm
>>>      - vp90-2-02-size-32x18.webm
>>>      - vp90-2-02-size-32x32.webm
>>>      - vp90-2-02-size-32x34.webm
>>>      - vp90-2-02-size-32x64.webm
>>>      - vp90-2-02-size-32x66.webm
>>>      - vp90-2-02-size-34x08.webm
>>>      - vp90-2-02-size-34x10.webm
>>>      - vp90-2-02-size-34x16.webm
>>>      - vp90-2-02-size-34x18.webm
>>>      - vp90-2-02-size-34x32.webm
>>>      - vp90-2-02-size-34x34.webm
>>>      - vp90-2-02-size-34x64.webm
>>>      - vp90-2-02-size-34x66.webm
>>>      - vp90-2-02-size-64x08.webm
>>>      - vp90-2-02-size-64x10.webm
>>>      - vp90-2-02-size-64x16.webm
>>>      - vp90-2-02-size-64x18.webm
>>>      - vp90-2-02-size-64x32.webm
>>>      - vp90-2-02-size-64x34.webm
>>>      - vp90-2-02-size-64x64.webm
>>>      - vp90-2-02-size-64x66.webm
>>>      - vp90-2-02-size-66x08.webm
>>>      - vp90-2-02-size-66x10.webm
>>>      - vp90-2-02-size-66x16.webm
>>>      - vp90-2-02-size-66x18.webm
>>>      - vp90-2-02-size-66x32.webm
>>>      - vp90-2-02-size-66x34.webm
>>>      - vp90-2-02-size-66x64.webm
>>>      - vp90-2-02-size-66x66.webm
>>>    - 2 testcases failed due to unsupported format
>>>      - vp91-2-04-yuv422.webm
>>>      - vp91-2-04-yuv444.webm
>>>    - 1 testcase failed with CRC mismatch
>>>      - vp90-2-22-svc_1280x720_3.ivf
>>>      - Bug reported:
>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>    - 2 testcase failed due to unsupported resolution after sequence change
>>>      - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>      - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>    - 1 testcase failed due to unsupported stream
>>>      - vp90-2-16-intra-only.webm
>>>
>>> The result of fluster test on SM8250:
>>>    133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>    GStreamer-H.265-V4L2-Gst1.0.
>>>    The failing test case:
>>>    - 10 testcases failed due to unsupported 10 bit format.
>>>      - DBLK_A_MAIN10_VIXS_4
>>>      - INITQP_B_Main10_Sony_1
>>>      - TSUNEQBD_A_MAIN10_Technicolor_2
>>>      - WP_A_MAIN10_Toshiba_3
>>>      - WP_MAIN10_B_Toshiba_3
>>>      - WPP_A_ericsson_MAIN10_2
>>>      - WPP_B_ericsson_MAIN10_2
>>>      - WPP_C_ericsson_MAIN10_2
>>>      - WPP_E_ericsson_MAIN10_2
>>>      - WPP_F_ericsson_MAIN10_2
>>>    - 4 testcase failed due to unsupported resolution
>>>      - PICSIZE_A_Bossen_1
>>>      - PICSIZE_B_Bossen_1
>>>      - WPP_D_ericsson_MAIN10_2
>>>      - WPP_D_ericsson_MAIN_2
>>>
>>>    232/305 testcases passed while testing VP9-TEST-VECTORS with
>>>    GStreamer-VP9-V4L2-Gst1.0.
>>>    The failing test case:
>>>    - 64 testcases failed due to unsupported resolution
>>>      - vp90-2-02-size-08x08.webm
>>>      - vp90-2-02-size-08x10.webm
>>>      - vp90-2-02-size-08x16.webm
>>>      - vp90-2-02-size-08x18.webm
>>>      - vp90-2-02-size-08x32.webm
>>>      - vp90-2-02-size-08x34.webm
>>>      - vp90-2-02-size-08x64.webm
>>>      - vp90-2-02-size-08x66.webm
>>>      - vp90-2-02-size-10x08.webm
>>>      - vp90-2-02-size-10x10.webm
>>>      - vp90-2-02-size-10x16.webm
>>>      - vp90-2-02-size-10x18.webm
>>>      - vp90-2-02-size-10x32.webm
>>>      - vp90-2-02-size-10x34.webm
>>>      - vp90-2-02-size-10x64.webm
>>>      - vp90-2-02-size-10x66.webm
>>>      - vp90-2-02-size-16x08.webm
>>>      - vp90-2-02-size-16x10.webm
>>>      - vp90-2-02-size-16x16.webm
>>>      - vp90-2-02-size-16x18.webm
>>>      - vp90-2-02-size-16x32.webm
>>>      - vp90-2-02-size-16x34.webm
>>>      - vp90-2-02-size-16x64.webm
>>>      - vp90-2-02-size-16x66.webm
>>>      - vp90-2-02-size-18x08.webm
>>>      - vp90-2-02-size-18x10.webm
>>>      - vp90-2-02-size-18x16.webm
>>>      - vp90-2-02-size-18x18.webm
>>>      - vp90-2-02-size-18x32.webm
>>>      - vp90-2-02-size-18x34.webm
>>>      - vp90-2-02-size-18x64.webm
>>>      - vp90-2-02-size-18x66.webm
>>>      - vp90-2-02-size-32x08.webm
>>>      - vp90-2-02-size-32x10.webm
>>>      - vp90-2-02-size-32x16.webm
>>>      - vp90-2-02-size-32x18.webm
>>>      - vp90-2-02-size-32x32.webm
>>>      - vp90-2-02-size-32x34.webm
>>>      - vp90-2-02-size-32x64.webm
>>>      - vp90-2-02-size-32x66.webm
>>>      - vp90-2-02-size-34x08.webm
>>>      - vp90-2-02-size-34x10.webm
>>>      - vp90-2-02-size-34x16.webm
>>>      - vp90-2-02-size-34x18.webm
>>>      - vp90-2-02-size-34x32.webm
>>>      - vp90-2-02-size-34x34.webm
>>>      - vp90-2-02-size-34x64.webm
>>>      - vp90-2-02-size-34x66.webm
>>>      - vp90-2-02-size-64x08.webm
>>>      - vp90-2-02-size-64x10.webm
>>>      - vp90-2-02-size-64x16.webm
>>>      - vp90-2-02-size-64x18.webm
>>>      - vp90-2-02-size-64x32.webm
>>>      - vp90-2-02-size-64x34.webm
>>>      - vp90-2-02-size-64x64.webm
>>>      - vp90-2-02-size-64x66.webm
>>>      - vp90-2-02-size-66x08.webm
>>>      - vp90-2-02-size-66x10.webm
>>>      - vp90-2-02-size-66x16.webm
>>>      - vp90-2-02-size-66x18.webm
>>>      - vp90-2-02-size-66x32.webm
>>>      - vp90-2-02-size-66x34.webm
>>>      - vp90-2-02-size-66x64.webm
>>>      - vp90-2-02-size-66x66.webm
>>>    - 2 testcases failed due to unsupported format
>>>      - vp91-2-04-yuv422.webm
>>>      - vp91-2-04-yuv444.webm
>>>    - 1 testcase failed with CRC mismatch
>>>      - vp90-2-22-svc_1280x720_3.ivf
>>>      - Bug raised:
>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>    - 5 testcase failed due to unsupported resolution after sequence change
>>>      - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>      - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>      - vp90-2-21-resize_inter_320x240_5_1-2.webm
>>>      - vp90-2-21-resize_inter_320x240_7_1-2.webm
>>>      - vp90-2-18-resize.ivf
>>>    - 1 testcase failed with CRC mismatch
>>>      - vp90-2-16-intra-only.webm
>>>      Analysis: First few frames are marked by firmware as NO_SHOW frame.
>>>      Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>>>      Such buffers should be dropped by GST. But instead, the first frame
>>>      is being displayed and when a valid buffer is sent to client later
>>>      with same timestamp, its dropped, leading to CRC mismatch for first
>>>      frame.
>>>
>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>> ---
>>> Dikshita Agarwal (23):
>>>         media: iris: Skip destroying internal buffer if not dequeued
>>>         media: iris: Update CAPTURE format info based on OUTPUT format
>>>         media: iris: Avoid updating frame size to firmware during reconfig
>>>         media: iris: Drop port check for session property response
>>>         media: iris: Prevent HFI queue writes when core is in deinit state
>>>         media: iris: Remove deprecated property setting to firmware
>>>         media: iris: Fix missing function pointer initialization
>>>         media: iris: Fix NULL pointer dereference
>>>         media: iris: Fix typo in depth variable
>>>         media: iris: Track flush responses to prevent premature completion
>>>         media: iris: Fix buffer preparation failure during resolution change
>>>         media: iris: Add handling for corrupt and drop frames
>>>         media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled
>>> length
>>>         media: iris: Add handling for no show frames
>>>         media: iris: Improve last flag handling
>>>         media: iris: Skip flush on first sequence change
>>>         media: iris: Remove redundant buffer count check in stream off
>>>         media: iris: Add a comment to explain usage of MBPS
>>>         media: iris: Add HEVC and VP9 formats for decoder
>>>         media: iris: Add platform capabilities for HEVC and VP9 decoders
>>>         media: iris: Set mandatory properties for HEVC and VP9 decoders.
>>>         media: iris: Add internal buffer calculation for HEVC and VP9
>>> decoders
>>>         media: iris: Add codec specific check for VP9 decoder drain handling
>>>
>>>    drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>>>    drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>>>    drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>>>    drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>>>    .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>>>    .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>>>    .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>>>    .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>>>    .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>>>    .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>>>    drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>>>    drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>>>    .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>>>    .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>>>    .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>>>    .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>>>    drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>>>    drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>>>    drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>>>    drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>>>    drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>>>    drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>>>    drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397
>>> ++++++++++++++++++++-
>>>    drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>>>    24 files changed, 1160 insertions(+), 211 deletions(-)
>>> ---
>>> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
>>> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
>>> prerequisite-message-id:
>>> <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
>>> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
>>> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
>>> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
>>> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
>>> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
>>> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
>>> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
>>> prerequisite-message-id:
>>> <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
>>> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
>>> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
>>> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
>>> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
>>> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
>>>
>>> Best regards,
>>
>> HEVC & VP9 works fine on HDK8550.
>>
>> But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
>> [   44.741670] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.755724] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.776462] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.797179] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.816630] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.837387] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.856812] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.877576] qcom-iris aa00000.video-codec: session error received
>> 0x1000005: unknown
>> [   44.897000] qcom-iris aa00000.video-codec: session error received
>> 0x4000005: insufficient resources
>> [   44.917801] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   44.937254] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   44.959128] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   44.981025] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   45.000459] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   45.022376] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   45.041816] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   45.063736] qcom-iris aa00000.video-codec: session error received
>> 0x1000009: unknown
>> [   45.083167] qcom-iris aa00000.video-codec: session error received
>> 0x4000004: invalid operation for current state
>> [   45.105459] ------------[ cut here ]------------
>> [   45.121152] WARNING: CPU: 6 PID: 573 at
>> drivers/media/common/videobuf2/videobuf2-core.c:1827
>> vb2_start_streaming+0x100/0x178 [videobuf2_common]
>> while VP9 works fine.
>>
>> Is it a firmware issue ?
>>
> Looks like resources set to firmware are not sufficient.
> I suspect, internal buffers set to firmware are less than what it requires,
> this can change for different VPUs. Pls check if there is any difference in
> internal buffer calculations between vpu3 and vpu33.

I found the fix, it was a difference in buffer calculation, but not for vpu33,
but a typo since v2 was right:

========================================><======================================
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
index 2272f0c21683..ee95fd20b794 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
@@ -33,7 +33,7 @@ struct iris_inst;
  #define H264_DISPLAY_BUF_SIZE          3328
  #define H264_NUM_FRM_INFO              66
  #define H265_NUM_TILE_COL 32
-#define H265_NUM_TILE_ROW 12
+#define H265_NUM_TILE_ROW 128
  #define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
  #define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32))
========================================><======================================

This fixes HEVC on SM8650, so with this change VP9 and HEVC works fine, I'm suprised
this still works on SM8550 !

So please keep VP9 and HEVC enabled on v4.

Thanks,
Neil
> 
> Thanks,
> Dikshita
>> I've added:
>> ========================================><======================================
>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> index d3026b2bcb70..8c0ab00ab435 100644
>> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>> @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
>>          .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
>>          .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
>>          .vpu_ops = &iris_vpu33_ops,
>> -       .set_preset_registers = iris_set_sm8550_preset_registers,
>> +       .set_preset_registers = iris_set_sm8650_preset_registers,
>>          .icc_tbl = sm8550_icc_table,
>>          .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
>>          .clk_rst_tbl = sm8650_clk_reset_table,
>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> index 75e9d572e788..9e2d23f12f75 100644
>> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>> @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = {
>> "bus", "core" };
>>
>>   static const char * const sm8650_controller_reset_table[] = { "xo" };
>>
>> +static void iris_set_sm8650_preset_registers(struct iris_core *core)
>> +{
>> +       writel(0x0, core->reg_base + 0xB0088);
>> +       writel(0x33332222, core->reg_base + 0x13030);
>> +       writel(0x44444444, core->reg_base + 0x13034);
>> +       writel(0x1022, core->reg_base + 0x13038);
>> +       writel(0x0, core->reg_base + 0x13040);
>> +       writel(0xFFFF, core->reg_base + 0x13048);
>> +       writel(0x33332222, core->reg_base + 0x13430);
>> +       writel(0x44444444, core->reg_base + 0x13434);
>> +       writel(0x1022, core->reg_base + 0x13438);
>> +       writel(0x0, core->reg_base + 0x13440);
>> +       writel(0xFFFF, core->reg_base + 0x13448);
>> +       writel(0x99, core->reg_base + 0xA013C);
>> +}
>> +
>>   #endif
>> ========================================><======================================
>> and no change, error still occurs with HEVC decoding.
>>
>> Thanks,
>> Neil

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[Dikshita Agarwal]

On 5/5/2025 10:10 PM, neil.armstrong@linaro.org wrote:
> On 02/05/2025 09:34, Dikshita Agarwal wrote:
>>
>>
>> On 5/2/2025 12:55 PM, Neil Armstrong wrote:
>>> Hi,
>>>
>>> On 01/05/2025 21:13, Dikshita Agarwal wrote:
>>>> Hi All,
>>>>
>>>> This patch series adds initial support for the HEVC(H.265) and VP9
>>>> codecs in iris decoder. The objective of this work is to extend the
>>>> decoder's capabilities to handle HEVC and VP9 codec streams,
>>>> including necessary format handling and buffer management.
>>>> In addition, the series also includes a set of fixes to address issues
>>>> identified during testing of these additional codecs.
>>>>
>>>> These patches also address the comments and feedback received from the
>>>> RFC patches previously sent. I have made the necessary improvements
>>>> based on the community's suggestions.
>>>>
>>>> Changes in v3:
>>>> - Introduced two wrappers with explicit names to handle destroy internal
>>>> buffers (Nicolas)
>>>> - Used sub state check instead of introducing new boolean (Vikash)
>>>> - Addressed other comments (Vikash)
>>>> - Reorderd patches to have all fixes patches first (Dmitry)
>>>> - Link to v2:
>>>> https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
>>>>
>>>> Changes in v2:
>>>> - Added Changes to make sure all buffers are released in session close
>>>> (bryna)
>>>> - Added tracking for flush responses to fix a timing issue.
>>>> - Added a handling to fix timing issue in reconfig
>>>> - Splitted patch 06/20 in two patches (Bryan)
>>>> - Added missing fixes tag (bryan)
>>>> - Updated fluster report (Nicolas)
>>>> - Link to v1:
>>>> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
>>>>
>>>> Changes sinces RFC:
>>>> - Added additional fixes to address issues identified during further
>>>> testing.
>>>> - Moved typo fix to a seperate patch [Neil]
>>>> - Reordered the patches for better logical flow and clarity [Neil,
>>>> Dmitry]
>>>> - Added fixes tag wherever applicable [Neil, Dmitry]
>>>> - Removed the default case in the switch statement for codecs [Bryan]
>>>> - Replaced if-else statements with switch-case [Bryan]
>>>> - Added comments for mbpf [Bryan]
>>>> - RFC:
>>>> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
>>>>
>>>> This patch series depends on [1] & [2]
>>>> [1]
>>>> https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
>>>> [2]
>>>> https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
>>>>
>>>> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
>>>> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
>>>> the existing H264 decoder functionality remains uneffected.
>>>>
>>>> Note: 1 of the fluster compliance test is fixed with firmware [3]
>>>> [3]:
>>>> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
>>>>
>>>> The result of fluster test on SM8550:
>>>>    131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>    GStreamer-H.265-V4L2-Gst1.0.
>>>>    The failing test case:
>>>>    - 10 testcases failed due to unsupported 10 bit format.
>>>>      - DBLK_A_MAIN10_VIXS_4
>>>>      - INITQP_B_Main10_Sony_1
>>>>      - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>      - WP_A_MAIN10_Toshiba_3
>>>>      - WP_MAIN10_B_Toshiba_3
>>>>      - WPP_A_ericsson_MAIN10_2
>>>>      - WPP_B_ericsson_MAIN10_2
>>>>      - WPP_C_ericsson_MAIN10_2
>>>>      - WPP_E_ericsson_MAIN10_2
>>>>      - WPP_F_ericsson_MAIN10_2
>>>>    - 4 testcase failed due to unsupported resolution
>>>>      - PICSIZE_A_Bossen_1
>>>>      - PICSIZE_B_Bossen_1
>>>>      - WPP_D_ericsson_MAIN10_2
>>>>      - WPP_D_ericsson_MAIN_2
>>>>    - 2 testcase failed due to CRC mismatch
>>>>      - RAP_A_docomo_6
>>>>      - RAP_B_Bossen_2
>>>>      - BUG reported:
>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>>>>        Analysis - First few frames in this discarded by firmware and are
>>>>        sent to driver with 0 filled length. Driver send such buffers to
>>>>        client with timestamp 0 and payload set to 0 and
>>>>        make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>>>>        dropped by GST. But instead, the first frame displayed as green
>>>>        frame and when a valid buffer is sent to client later with same 0
>>>>        timestamp, its dropped, leading to CRC mismatch for first frame.
>>>>
>>>>    235/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>    GStreamer-VP9-V4L2-Gst1.0.
>>>>    The failing test case:
>>>>    - 64 testcases failed due to unsupported resolution
>>>>      - vp90-2-02-size-08x08.webm
>>>>      - vp90-2-02-size-08x10.webm
>>>>      - vp90-2-02-size-08x16.webm
>>>>      - vp90-2-02-size-08x18.webm
>>>>      - vp90-2-02-size-08x32.webm
>>>>      - vp90-2-02-size-08x34.webm
>>>>      - vp90-2-02-size-08x64.webm
>>>>      - vp90-2-02-size-08x66.webm
>>>>      - vp90-2-02-size-10x08.webm
>>>>      - vp90-2-02-size-10x10.webm
>>>>      - vp90-2-02-size-10x16.webm
>>>>      - vp90-2-02-size-10x18.webm
>>>>      - vp90-2-02-size-10x32.webm
>>>>      - vp90-2-02-size-10x34.webm
>>>>      - vp90-2-02-size-10x64.webm
>>>>      - vp90-2-02-size-10x66.webm
>>>>      - vp90-2-02-size-16x08.webm
>>>>      - vp90-2-02-size-16x10.webm
>>>>      - vp90-2-02-size-16x16.webm
>>>>      - vp90-2-02-size-16x18.webm
>>>>      - vp90-2-02-size-16x32.webm
>>>>      - vp90-2-02-size-16x34.webm
>>>>      - vp90-2-02-size-16x64.webm
>>>>      - vp90-2-02-size-16x66.webm
>>>>      - vp90-2-02-size-18x08.webm
>>>>      - vp90-2-02-size-18x10.webm
>>>>      - vp90-2-02-size-18x16.webm
>>>>      - vp90-2-02-size-18x18.webm
>>>>      - vp90-2-02-size-18x32.webm
>>>>      - vp90-2-02-size-18x34.webm
>>>>      - vp90-2-02-size-18x64.webm
>>>>      - vp90-2-02-size-18x66.webm
>>>>      - vp90-2-02-size-32x08.webm
>>>>      - vp90-2-02-size-32x10.webm
>>>>      - vp90-2-02-size-32x16.webm
>>>>      - vp90-2-02-size-32x18.webm
>>>>      - vp90-2-02-size-32x32.webm
>>>>      - vp90-2-02-size-32x34.webm
>>>>      - vp90-2-02-size-32x64.webm
>>>>      - vp90-2-02-size-32x66.webm
>>>>      - vp90-2-02-size-34x08.webm
>>>>      - vp90-2-02-size-34x10.webm
>>>>      - vp90-2-02-size-34x16.webm
>>>>      - vp90-2-02-size-34x18.webm
>>>>      - vp90-2-02-size-34x32.webm
>>>>      - vp90-2-02-size-34x34.webm
>>>>      - vp90-2-02-size-34x64.webm
>>>>      - vp90-2-02-size-34x66.webm
>>>>      - vp90-2-02-size-64x08.webm
>>>>      - vp90-2-02-size-64x10.webm
>>>>      - vp90-2-02-size-64x16.webm
>>>>      - vp90-2-02-size-64x18.webm
>>>>      - vp90-2-02-size-64x32.webm
>>>>      - vp90-2-02-size-64x34.webm
>>>>      - vp90-2-02-size-64x64.webm
>>>>      - vp90-2-02-size-64x66.webm
>>>>      - vp90-2-02-size-66x08.webm
>>>>      - vp90-2-02-size-66x10.webm
>>>>      - vp90-2-02-size-66x16.webm
>>>>      - vp90-2-02-size-66x18.webm
>>>>      - vp90-2-02-size-66x32.webm
>>>>      - vp90-2-02-size-66x34.webm
>>>>      - vp90-2-02-size-66x64.webm
>>>>      - vp90-2-02-size-66x66.webm
>>>>    - 2 testcases failed due to unsupported format
>>>>      - vp91-2-04-yuv422.webm
>>>>      - vp91-2-04-yuv444.webm
>>>>    - 1 testcase failed with CRC mismatch
>>>>      - vp90-2-22-svc_1280x720_3.ivf
>>>>      - Bug reported:
>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>    - 2 testcase failed due to unsupported resolution after sequence change
>>>>      - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>      - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>    - 1 testcase failed due to unsupported stream
>>>>      - vp90-2-16-intra-only.webm
>>>>
>>>> The result of fluster test on SM8250:
>>>>    133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>    GStreamer-H.265-V4L2-Gst1.0.
>>>>    The failing test case:
>>>>    - 10 testcases failed due to unsupported 10 bit format.
>>>>      - DBLK_A_MAIN10_VIXS_4
>>>>      - INITQP_B_Main10_Sony_1
>>>>      - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>      - WP_A_MAIN10_Toshiba_3
>>>>      - WP_MAIN10_B_Toshiba_3
>>>>      - WPP_A_ericsson_MAIN10_2
>>>>      - WPP_B_ericsson_MAIN10_2
>>>>      - WPP_C_ericsson_MAIN10_2
>>>>      - WPP_E_ericsson_MAIN10_2
>>>>      - WPP_F_ericsson_MAIN10_2
>>>>    - 4 testcase failed due to unsupported resolution
>>>>      - PICSIZE_A_Bossen_1
>>>>      - PICSIZE_B_Bossen_1
>>>>      - WPP_D_ericsson_MAIN10_2
>>>>      - WPP_D_ericsson_MAIN_2
>>>>
>>>>    232/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>    GStreamer-VP9-V4L2-Gst1.0.
>>>>    The failing test case:
>>>>    - 64 testcases failed due to unsupported resolution
>>>>      - vp90-2-02-size-08x08.webm
>>>>      - vp90-2-02-size-08x10.webm
>>>>      - vp90-2-02-size-08x16.webm
>>>>      - vp90-2-02-size-08x18.webm
>>>>      - vp90-2-02-size-08x32.webm
>>>>      - vp90-2-02-size-08x34.webm
>>>>      - vp90-2-02-size-08x64.webm
>>>>      - vp90-2-02-size-08x66.webm
>>>>      - vp90-2-02-size-10x08.webm
>>>>      - vp90-2-02-size-10x10.webm
>>>>      - vp90-2-02-size-10x16.webm
>>>>      - vp90-2-02-size-10x18.webm
>>>>      - vp90-2-02-size-10x32.webm
>>>>      - vp90-2-02-size-10x34.webm
>>>>      - vp90-2-02-size-10x64.webm
>>>>      - vp90-2-02-size-10x66.webm
>>>>      - vp90-2-02-size-16x08.webm
>>>>      - vp90-2-02-size-16x10.webm
>>>>      - vp90-2-02-size-16x16.webm
>>>>      - vp90-2-02-size-16x18.webm
>>>>      - vp90-2-02-size-16x32.webm
>>>>      - vp90-2-02-size-16x34.webm
>>>>      - vp90-2-02-size-16x64.webm
>>>>      - vp90-2-02-size-16x66.webm
>>>>      - vp90-2-02-size-18x08.webm
>>>>      - vp90-2-02-size-18x10.webm
>>>>      - vp90-2-02-size-18x16.webm
>>>>      - vp90-2-02-size-18x18.webm
>>>>      - vp90-2-02-size-18x32.webm
>>>>      - vp90-2-02-size-18x34.webm
>>>>      - vp90-2-02-size-18x64.webm
>>>>      - vp90-2-02-size-18x66.webm
>>>>      - vp90-2-02-size-32x08.webm
>>>>      - vp90-2-02-size-32x10.webm
>>>>      - vp90-2-02-size-32x16.webm
>>>>      - vp90-2-02-size-32x18.webm
>>>>      - vp90-2-02-size-32x32.webm
>>>>      - vp90-2-02-size-32x34.webm
>>>>      - vp90-2-02-size-32x64.webm
>>>>      - vp90-2-02-size-32x66.webm
>>>>      - vp90-2-02-size-34x08.webm
>>>>      - vp90-2-02-size-34x10.webm
>>>>      - vp90-2-02-size-34x16.webm
>>>>      - vp90-2-02-size-34x18.webm
>>>>      - vp90-2-02-size-34x32.webm
>>>>      - vp90-2-02-size-34x34.webm
>>>>      - vp90-2-02-size-34x64.webm
>>>>      - vp90-2-02-size-34x66.webm
>>>>      - vp90-2-02-size-64x08.webm
>>>>      - vp90-2-02-size-64x10.webm
>>>>      - vp90-2-02-size-64x16.webm
>>>>      - vp90-2-02-size-64x18.webm
>>>>      - vp90-2-02-size-64x32.webm
>>>>      - vp90-2-02-size-64x34.webm
>>>>      - vp90-2-02-size-64x64.webm
>>>>      - vp90-2-02-size-64x66.webm
>>>>      - vp90-2-02-size-66x08.webm
>>>>      - vp90-2-02-size-66x10.webm
>>>>      - vp90-2-02-size-66x16.webm
>>>>      - vp90-2-02-size-66x18.webm
>>>>      - vp90-2-02-size-66x32.webm
>>>>      - vp90-2-02-size-66x34.webm
>>>>      - vp90-2-02-size-66x64.webm
>>>>      - vp90-2-02-size-66x66.webm
>>>>    - 2 testcases failed due to unsupported format
>>>>      - vp91-2-04-yuv422.webm
>>>>      - vp91-2-04-yuv444.webm
>>>>    - 1 testcase failed with CRC mismatch
>>>>      - vp90-2-22-svc_1280x720_3.ivf
>>>>      - Bug raised:
>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>    - 5 testcase failed due to unsupported resolution after sequence change
>>>>      - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>      - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>      - vp90-2-21-resize_inter_320x240_5_1-2.webm
>>>>      - vp90-2-21-resize_inter_320x240_7_1-2.webm
>>>>      - vp90-2-18-resize.ivf
>>>>    - 1 testcase failed with CRC mismatch
>>>>      - vp90-2-16-intra-only.webm
>>>>      Analysis: First few frames are marked by firmware as NO_SHOW frame.
>>>>      Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>>>>      Such buffers should be dropped by GST. But instead, the first frame
>>>>      is being displayed and when a valid buffer is sent to client later
>>>>      with same timestamp, its dropped, leading to CRC mismatch for first
>>>>      frame.
>>>>
>>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>>> ---
>>>> Dikshita Agarwal (23):
>>>>         media: iris: Skip destroying internal buffer if not dequeued
>>>>         media: iris: Update CAPTURE format info based on OUTPUT format
>>>>         media: iris: Avoid updating frame size to firmware during reconfig
>>>>         media: iris: Drop port check for session property response
>>>>         media: iris: Prevent HFI queue writes when core is in deinit state
>>>>         media: iris: Remove deprecated property setting to firmware
>>>>         media: iris: Fix missing function pointer initialization
>>>>         media: iris: Fix NULL pointer dereference
>>>>         media: iris: Fix typo in depth variable
>>>>         media: iris: Track flush responses to prevent premature completion
>>>>         media: iris: Fix buffer preparation failure during resolution
>>>> change
>>>>         media: iris: Add handling for corrupt and drop frames
>>>>         media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled
>>>> length
>>>>         media: iris: Add handling for no show frames
>>>>         media: iris: Improve last flag handling
>>>>         media: iris: Skip flush on first sequence change
>>>>         media: iris: Remove redundant buffer count check in stream off
>>>>         media: iris: Add a comment to explain usage of MBPS
>>>>         media: iris: Add HEVC and VP9 formats for decoder
>>>>         media: iris: Add platform capabilities for HEVC and VP9 decoders
>>>>         media: iris: Set mandatory properties for HEVC and VP9 decoders.
>>>>         media: iris: Add internal buffer calculation for HEVC and VP9
>>>> decoders
>>>>         media: iris: Add codec specific check for VP9 decoder drain
>>>> handling
>>>>
>>>>    drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>>>>    drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>>>>    drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>>>>    drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>>>>    .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>>>>    .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>>>>    .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>>>>    .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>>>>    .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>>>>    .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>>>>    drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>>>>    drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>>>>    .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>>>>    .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>>>>    .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>>>>    .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>>>>    drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>>>>    drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>>>>    drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>>>>    drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>>>>    drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>>>>    drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>>>>    drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397
>>>> ++++++++++++++++++++-
>>>>    drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>>>>    24 files changed, 1160 insertions(+), 211 deletions(-)
>>>> ---
>>>> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
>>>> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
>>>> prerequisite-message-id:
>>>> <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
>>>> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
>>>> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
>>>> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
>>>> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
>>>> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
>>>> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
>>>> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
>>>> prerequisite-message-id:
>>>> <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
>>>> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
>>>> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
>>>> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
>>>> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
>>>> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
>>>>
>>>> Best regards,
>>>
>>> HEVC & VP9 works fine on HDK8550.
>>>
>>> But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
>>> [   44.741670] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.755724] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.776462] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.797179] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.816630] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.837387] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.856812] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.877576] qcom-iris aa00000.video-codec: session error received
>>> 0x1000005: unknown
>>> [   44.897000] qcom-iris aa00000.video-codec: session error received
>>> 0x4000005: insufficient resources
>>> [   44.917801] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   44.937254] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   44.959128] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   44.981025] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   45.000459] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   45.022376] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   45.041816] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   45.063736] qcom-iris aa00000.video-codec: session error received
>>> 0x1000009: unknown
>>> [   45.083167] qcom-iris aa00000.video-codec: session error received
>>> 0x4000004: invalid operation for current state
>>> [   45.105459] ------------[ cut here ]------------
>>> [   45.121152] WARNING: CPU: 6 PID: 573 at
>>> drivers/media/common/videobuf2/videobuf2-core.c:1827
>>> vb2_start_streaming+0x100/0x178 [videobuf2_common]
>>> while VP9 works fine.
>>>
>>> Is it a firmware issue ?
>>>
>> Looks like resources set to firmware are not sufficient.
>> I suspect, internal buffers set to firmware are less than what it requires,
>> this can change for different VPUs. Pls check if there is any difference in
>> internal buffer calculations between vpu3 and vpu33.
> 
> I found the fix, it was a difference in buffer calculation, but not for vpu33,
> but a typo since v2 was right:
> 
> ========================================><======================================
> diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
> b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
> index 2272f0c21683..ee95fd20b794 100644
> --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
> +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
> @@ -33,7 +33,7 @@ struct iris_inst;
>  #define H264_DISPLAY_BUF_SIZE          3328
>  #define H264_NUM_FRM_INFO              66
>  #define H265_NUM_TILE_COL 32
> -#define H265_NUM_TILE_ROW 12
> +#define H265_NUM_TILE_ROW 128
>  #define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
>  #define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32))
> ========================================><======================================
> 
> This fixes HEVC on SM8650, so with this change VP9 and HEVC works fine, I'm
> suprised
> this still works on SM8550 !
> 
Thanks for checking this.
but this value has been the same since v1
https://lore.kernel.org/linux-media/20250408-iris-dec-hevc-vp9-v1-19-acd258778bd6@quicinc.com/
And yes, it works on SM8550, I believe you also tested v3 and confirmed the
same.
But anyways, the correct value should be 128 and I confirmed that SM8550
works with the fix as well.

> So please keep VP9 and HEVC enabled on v4.
Ok

Thanks,
Dikshita
> 
> Thanks,
> Neil
>>
>> Thanks,
>> Dikshita
>>> I've added:
>>> ========================================><======================================
>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> index d3026b2bcb70..8c0ab00ab435 100644
>>> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>> @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
>>>          .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
>>>          .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
>>>          .vpu_ops = &iris_vpu33_ops,
>>> -       .set_preset_registers = iris_set_sm8550_preset_registers,
>>> +       .set_preset_registers = iris_set_sm8650_preset_registers,
>>>          .icc_tbl = sm8550_icc_table,
>>>          .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
>>>          .clk_rst_tbl = sm8650_clk_reset_table,
>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> index 75e9d572e788..9e2d23f12f75 100644
>>> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>> @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = {
>>> "bus", "core" };
>>>
>>>   static const char * const sm8650_controller_reset_table[] = { "xo" };
>>>
>>> +static void iris_set_sm8650_preset_registers(struct iris_core *core)
>>> +{
>>> +       writel(0x0, core->reg_base + 0xB0088);
>>> +       writel(0x33332222, core->reg_base + 0x13030);
>>> +       writel(0x44444444, core->reg_base + 0x13034);
>>> +       writel(0x1022, core->reg_base + 0x13038);
>>> +       writel(0x0, core->reg_base + 0x13040);
>>> +       writel(0xFFFF, core->reg_base + 0x13048);
>>> +       writel(0x33332222, core->reg_base + 0x13430);
>>> +       writel(0x44444444, core->reg_base + 0x13434);
>>> +       writel(0x1022, core->reg_base + 0x13438);
>>> +       writel(0x0, core->reg_base + 0x13440);
>>> +       writel(0xFFFF, core->reg_base + 0x13448);
>>> +       writel(0x99, core->reg_base + 0xA013C);
>>> +}
>>> +
>>>   #endif
>>> ========================================><======================================
>>> and no change, error still occurs with HEVC decoding.
>>>
>>> Thanks,
>>> Neil
> 

Re: [PATCH v3 00/23] Add support for HEVC and VP9 codecs in decoder

[Neil Armstrong]

On 06/05/2025 10:06, Dikshita Agarwal wrote:
> 
> 
> On 5/5/2025 10:10 PM, neil.armstrong@linaro.org wrote:
>> On 02/05/2025 09:34, Dikshita Agarwal wrote:
>>>
>>>
>>> On 5/2/2025 12:55 PM, Neil Armstrong wrote:
>>>> Hi,
>>>>
>>>> On 01/05/2025 21:13, Dikshita Agarwal wrote:
>>>>> Hi All,
>>>>>
>>>>> This patch series adds initial support for the HEVC(H.265) and VP9
>>>>> codecs in iris decoder. The objective of this work is to extend the
>>>>> decoder's capabilities to handle HEVC and VP9 codec streams,
>>>>> including necessary format handling and buffer management.
>>>>> In addition, the series also includes a set of fixes to address issues
>>>>> identified during testing of these additional codecs.
>>>>>
>>>>> These patches also address the comments and feedback received from the
>>>>> RFC patches previously sent. I have made the necessary improvements
>>>>> based on the community's suggestions.
>>>>>
>>>>> Changes in v3:
>>>>> - Introduced two wrappers with explicit names to handle destroy internal
>>>>> buffers (Nicolas)
>>>>> - Used sub state check instead of introducing new boolean (Vikash)
>>>>> - Addressed other comments (Vikash)
>>>>> - Reorderd patches to have all fixes patches first (Dmitry)
>>>>> - Link to v2:
>>>>> https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com
>>>>>
>>>>> Changes in v2:
>>>>> - Added Changes to make sure all buffers are released in session close
>>>>> (bryna)
>>>>> - Added tracking for flush responses to fix a timing issue.
>>>>> - Added a handling to fix timing issue in reconfig
>>>>> - Splitted patch 06/20 in two patches (Bryan)
>>>>> - Added missing fixes tag (bryan)
>>>>> - Updated fluster report (Nicolas)
>>>>> - Link to v1:
>>>>> https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com
>>>>>
>>>>> Changes sinces RFC:
>>>>> - Added additional fixes to address issues identified during further
>>>>> testing.
>>>>> - Moved typo fix to a seperate patch [Neil]
>>>>> - Reordered the patches for better logical flow and clarity [Neil,
>>>>> Dmitry]
>>>>> - Added fixes tag wherever applicable [Neil, Dmitry]
>>>>> - Removed the default case in the switch statement for codecs [Bryan]
>>>>> - Replaced if-else statements with switch-case [Bryan]
>>>>> - Added comments for mbpf [Bryan]
>>>>> - RFC:
>>>>> https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/
>>>>>
>>>>> This patch series depends on [1] & [2]
>>>>> [1]
>>>>> https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/
>>>>> [2]
>>>>> https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/
>>>>>
>>>>> These patches are tested on SM8250 and SM8550 with v4l2-ctl and
>>>>> Gstreamer for HEVC and VP9 decoders, at the same time ensured that
>>>>> the existing H264 decoder functionality remains uneffected.
>>>>>
>>>>> Note: 1 of the fluster compliance test is fixed with firmware [3]
>>>>> [3]:
>>>>> https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u
>>>>>
>>>>> The result of fluster test on SM8550:
>>>>>     131/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>>     GStreamer-H.265-V4L2-Gst1.0.
>>>>>     The failing test case:
>>>>>     - 10 testcases failed due to unsupported 10 bit format.
>>>>>       - DBLK_A_MAIN10_VIXS_4
>>>>>       - INITQP_B_Main10_Sony_1
>>>>>       - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>>       - WP_A_MAIN10_Toshiba_3
>>>>>       - WP_MAIN10_B_Toshiba_3
>>>>>       - WPP_A_ericsson_MAIN10_2
>>>>>       - WPP_B_ericsson_MAIN10_2
>>>>>       - WPP_C_ericsson_MAIN10_2
>>>>>       - WPP_E_ericsson_MAIN10_2
>>>>>       - WPP_F_ericsson_MAIN10_2
>>>>>     - 4 testcase failed due to unsupported resolution
>>>>>       - PICSIZE_A_Bossen_1
>>>>>       - PICSIZE_B_Bossen_1
>>>>>       - WPP_D_ericsson_MAIN10_2
>>>>>       - WPP_D_ericsson_MAIN_2
>>>>>     - 2 testcase failed due to CRC mismatch
>>>>>       - RAP_A_docomo_6
>>>>>       - RAP_B_Bossen_2
>>>>>       - BUG reported:
>>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392
>>>>>         Analysis - First few frames in this discarded by firmware and are
>>>>>         sent to driver with 0 filled length. Driver send such buffers to
>>>>>         client with timestamp 0 and payload set to 0 and
>>>>>         make buf state to VB2_BUF_STATE_ERROR. Such buffers should be
>>>>>         dropped by GST. But instead, the first frame displayed as green
>>>>>         frame and when a valid buffer is sent to client later with same 0
>>>>>         timestamp, its dropped, leading to CRC mismatch for first frame.
>>>>>
>>>>>     235/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>>     GStreamer-VP9-V4L2-Gst1.0.
>>>>>     The failing test case:
>>>>>     - 64 testcases failed due to unsupported resolution
>>>>>       - vp90-2-02-size-08x08.webm
>>>>>       - vp90-2-02-size-08x10.webm
>>>>>       - vp90-2-02-size-08x16.webm
>>>>>       - vp90-2-02-size-08x18.webm
>>>>>       - vp90-2-02-size-08x32.webm
>>>>>       - vp90-2-02-size-08x34.webm
>>>>>       - vp90-2-02-size-08x64.webm
>>>>>       - vp90-2-02-size-08x66.webm
>>>>>       - vp90-2-02-size-10x08.webm
>>>>>       - vp90-2-02-size-10x10.webm
>>>>>       - vp90-2-02-size-10x16.webm
>>>>>       - vp90-2-02-size-10x18.webm
>>>>>       - vp90-2-02-size-10x32.webm
>>>>>       - vp90-2-02-size-10x34.webm
>>>>>       - vp90-2-02-size-10x64.webm
>>>>>       - vp90-2-02-size-10x66.webm
>>>>>       - vp90-2-02-size-16x08.webm
>>>>>       - vp90-2-02-size-16x10.webm
>>>>>       - vp90-2-02-size-16x16.webm
>>>>>       - vp90-2-02-size-16x18.webm
>>>>>       - vp90-2-02-size-16x32.webm
>>>>>       - vp90-2-02-size-16x34.webm
>>>>>       - vp90-2-02-size-16x64.webm
>>>>>       - vp90-2-02-size-16x66.webm
>>>>>       - vp90-2-02-size-18x08.webm
>>>>>       - vp90-2-02-size-18x10.webm
>>>>>       - vp90-2-02-size-18x16.webm
>>>>>       - vp90-2-02-size-18x18.webm
>>>>>       - vp90-2-02-size-18x32.webm
>>>>>       - vp90-2-02-size-18x34.webm
>>>>>       - vp90-2-02-size-18x64.webm
>>>>>       - vp90-2-02-size-18x66.webm
>>>>>       - vp90-2-02-size-32x08.webm
>>>>>       - vp90-2-02-size-32x10.webm
>>>>>       - vp90-2-02-size-32x16.webm
>>>>>       - vp90-2-02-size-32x18.webm
>>>>>       - vp90-2-02-size-32x32.webm
>>>>>       - vp90-2-02-size-32x34.webm
>>>>>       - vp90-2-02-size-32x64.webm
>>>>>       - vp90-2-02-size-32x66.webm
>>>>>       - vp90-2-02-size-34x08.webm
>>>>>       - vp90-2-02-size-34x10.webm
>>>>>       - vp90-2-02-size-34x16.webm
>>>>>       - vp90-2-02-size-34x18.webm
>>>>>       - vp90-2-02-size-34x32.webm
>>>>>       - vp90-2-02-size-34x34.webm
>>>>>       - vp90-2-02-size-34x64.webm
>>>>>       - vp90-2-02-size-34x66.webm
>>>>>       - vp90-2-02-size-64x08.webm
>>>>>       - vp90-2-02-size-64x10.webm
>>>>>       - vp90-2-02-size-64x16.webm
>>>>>       - vp90-2-02-size-64x18.webm
>>>>>       - vp90-2-02-size-64x32.webm
>>>>>       - vp90-2-02-size-64x34.webm
>>>>>       - vp90-2-02-size-64x64.webm
>>>>>       - vp90-2-02-size-64x66.webm
>>>>>       - vp90-2-02-size-66x08.webm
>>>>>       - vp90-2-02-size-66x10.webm
>>>>>       - vp90-2-02-size-66x16.webm
>>>>>       - vp90-2-02-size-66x18.webm
>>>>>       - vp90-2-02-size-66x32.webm
>>>>>       - vp90-2-02-size-66x34.webm
>>>>>       - vp90-2-02-size-66x64.webm
>>>>>       - vp90-2-02-size-66x66.webm
>>>>>     - 2 testcases failed due to unsupported format
>>>>>       - vp91-2-04-yuv422.webm
>>>>>       - vp91-2-04-yuv444.webm
>>>>>     - 1 testcase failed with CRC mismatch
>>>>>       - vp90-2-22-svc_1280x720_3.ivf
>>>>>       - Bug reported:
>>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>>     - 2 testcase failed due to unsupported resolution after sequence change
>>>>>       - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>>       - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>>     - 1 testcase failed due to unsupported stream
>>>>>       - vp90-2-16-intra-only.webm
>>>>>
>>>>> The result of fluster test on SM8250:
>>>>>     133/147 testcases passed while testing JCT-VC-HEVC_V1 with
>>>>>     GStreamer-H.265-V4L2-Gst1.0.
>>>>>     The failing test case:
>>>>>     - 10 testcases failed due to unsupported 10 bit format.
>>>>>       - DBLK_A_MAIN10_VIXS_4
>>>>>       - INITQP_B_Main10_Sony_1
>>>>>       - TSUNEQBD_A_MAIN10_Technicolor_2
>>>>>       - WP_A_MAIN10_Toshiba_3
>>>>>       - WP_MAIN10_B_Toshiba_3
>>>>>       - WPP_A_ericsson_MAIN10_2
>>>>>       - WPP_B_ericsson_MAIN10_2
>>>>>       - WPP_C_ericsson_MAIN10_2
>>>>>       - WPP_E_ericsson_MAIN10_2
>>>>>       - WPP_F_ericsson_MAIN10_2
>>>>>     - 4 testcase failed due to unsupported resolution
>>>>>       - PICSIZE_A_Bossen_1
>>>>>       - PICSIZE_B_Bossen_1
>>>>>       - WPP_D_ericsson_MAIN10_2
>>>>>       - WPP_D_ericsson_MAIN_2
>>>>>
>>>>>     232/305 testcases passed while testing VP9-TEST-VECTORS with
>>>>>     GStreamer-VP9-V4L2-Gst1.0.
>>>>>     The failing test case:
>>>>>     - 64 testcases failed due to unsupported resolution
>>>>>       - vp90-2-02-size-08x08.webm
>>>>>       - vp90-2-02-size-08x10.webm
>>>>>       - vp90-2-02-size-08x16.webm
>>>>>       - vp90-2-02-size-08x18.webm
>>>>>       - vp90-2-02-size-08x32.webm
>>>>>       - vp90-2-02-size-08x34.webm
>>>>>       - vp90-2-02-size-08x64.webm
>>>>>       - vp90-2-02-size-08x66.webm
>>>>>       - vp90-2-02-size-10x08.webm
>>>>>       - vp90-2-02-size-10x10.webm
>>>>>       - vp90-2-02-size-10x16.webm
>>>>>       - vp90-2-02-size-10x18.webm
>>>>>       - vp90-2-02-size-10x32.webm
>>>>>       - vp90-2-02-size-10x34.webm
>>>>>       - vp90-2-02-size-10x64.webm
>>>>>       - vp90-2-02-size-10x66.webm
>>>>>       - vp90-2-02-size-16x08.webm
>>>>>       - vp90-2-02-size-16x10.webm
>>>>>       - vp90-2-02-size-16x16.webm
>>>>>       - vp90-2-02-size-16x18.webm
>>>>>       - vp90-2-02-size-16x32.webm
>>>>>       - vp90-2-02-size-16x34.webm
>>>>>       - vp90-2-02-size-16x64.webm
>>>>>       - vp90-2-02-size-16x66.webm
>>>>>       - vp90-2-02-size-18x08.webm
>>>>>       - vp90-2-02-size-18x10.webm
>>>>>       - vp90-2-02-size-18x16.webm
>>>>>       - vp90-2-02-size-18x18.webm
>>>>>       - vp90-2-02-size-18x32.webm
>>>>>       - vp90-2-02-size-18x34.webm
>>>>>       - vp90-2-02-size-18x64.webm
>>>>>       - vp90-2-02-size-18x66.webm
>>>>>       - vp90-2-02-size-32x08.webm
>>>>>       - vp90-2-02-size-32x10.webm
>>>>>       - vp90-2-02-size-32x16.webm
>>>>>       - vp90-2-02-size-32x18.webm
>>>>>       - vp90-2-02-size-32x32.webm
>>>>>       - vp90-2-02-size-32x34.webm
>>>>>       - vp90-2-02-size-32x64.webm
>>>>>       - vp90-2-02-size-32x66.webm
>>>>>       - vp90-2-02-size-34x08.webm
>>>>>       - vp90-2-02-size-34x10.webm
>>>>>       - vp90-2-02-size-34x16.webm
>>>>>       - vp90-2-02-size-34x18.webm
>>>>>       - vp90-2-02-size-34x32.webm
>>>>>       - vp90-2-02-size-34x34.webm
>>>>>       - vp90-2-02-size-34x64.webm
>>>>>       - vp90-2-02-size-34x66.webm
>>>>>       - vp90-2-02-size-64x08.webm
>>>>>       - vp90-2-02-size-64x10.webm
>>>>>       - vp90-2-02-size-64x16.webm
>>>>>       - vp90-2-02-size-64x18.webm
>>>>>       - vp90-2-02-size-64x32.webm
>>>>>       - vp90-2-02-size-64x34.webm
>>>>>       - vp90-2-02-size-64x64.webm
>>>>>       - vp90-2-02-size-64x66.webm
>>>>>       - vp90-2-02-size-66x08.webm
>>>>>       - vp90-2-02-size-66x10.webm
>>>>>       - vp90-2-02-size-66x16.webm
>>>>>       - vp90-2-02-size-66x18.webm
>>>>>       - vp90-2-02-size-66x32.webm
>>>>>       - vp90-2-02-size-66x34.webm
>>>>>       - vp90-2-02-size-66x64.webm
>>>>>       - vp90-2-02-size-66x66.webm
>>>>>     - 2 testcases failed due to unsupported format
>>>>>       - vp91-2-04-yuv422.webm
>>>>>       - vp91-2-04-yuv444.webm
>>>>>     - 1 testcase failed with CRC mismatch
>>>>>       - vp90-2-22-svc_1280x720_3.ivf
>>>>>       - Bug raised:
>>>>> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
>>>>>     - 5 testcase failed due to unsupported resolution after sequence change
>>>>>       - vp90-2-21-resize_inter_320x180_5_1-2.webm
>>>>>       - vp90-2-21-resize_inter_320x180_7_1-2.webm
>>>>>       - vp90-2-21-resize_inter_320x240_5_1-2.webm
>>>>>       - vp90-2-21-resize_inter_320x240_7_1-2.webm
>>>>>       - vp90-2-18-resize.ivf
>>>>>     - 1 testcase failed with CRC mismatch
>>>>>       - vp90-2-16-intra-only.webm
>>>>>       Analysis: First few frames are marked by firmware as NO_SHOW frame.
>>>>>       Driver make buf state to VB2_BUF_STATE_ERROR for such frames.
>>>>>       Such buffers should be dropped by GST. But instead, the first frame
>>>>>       is being displayed and when a valid buffer is sent to client later
>>>>>       with same timestamp, its dropped, leading to CRC mismatch for first
>>>>>       frame.
>>>>>
>>>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>>>> ---
>>>>> Dikshita Agarwal (23):
>>>>>          media: iris: Skip destroying internal buffer if not dequeued
>>>>>          media: iris: Update CAPTURE format info based on OUTPUT format
>>>>>          media: iris: Avoid updating frame size to firmware during reconfig
>>>>>          media: iris: Drop port check for session property response
>>>>>          media: iris: Prevent HFI queue writes when core is in deinit state
>>>>>          media: iris: Remove deprecated property setting to firmware
>>>>>          media: iris: Fix missing function pointer initialization
>>>>>          media: iris: Fix NULL pointer dereference
>>>>>          media: iris: Fix typo in depth variable
>>>>>          media: iris: Track flush responses to prevent premature completion
>>>>>          media: iris: Fix buffer preparation failure during resolution
>>>>> change
>>>>>          media: iris: Add handling for corrupt and drop frames
>>>>>          media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled
>>>>> length
>>>>>          media: iris: Add handling for no show frames
>>>>>          media: iris: Improve last flag handling
>>>>>          media: iris: Skip flush on first sequence change
>>>>>          media: iris: Remove redundant buffer count check in stream off
>>>>>          media: iris: Add a comment to explain usage of MBPS
>>>>>          media: iris: Add HEVC and VP9 formats for decoder
>>>>>          media: iris: Add platform capabilities for HEVC and VP9 decoders
>>>>>          media: iris: Set mandatory properties for HEVC and VP9 decoders.
>>>>>          media: iris: Add internal buffer calculation for HEVC and VP9
>>>>> decoders
>>>>>          media: iris: Add codec specific check for VP9 decoder drain
>>>>> handling
>>>>>
>>>>>     drivers/media/platform/qcom/iris/iris_buffer.c     |  35 +-
>>>>>     drivers/media/platform/qcom/iris/iris_buffer.h     |   3 +-
>>>>>     drivers/media/platform/qcom/iris/iris_ctrls.c      |  35 +-
>>>>>     drivers/media/platform/qcom/iris/iris_hfi_common.h |   1 +
>>>>>     .../platform/qcom/iris/iris_hfi_gen1_command.c     |  48 ++-
>>>>>     .../platform/qcom/iris/iris_hfi_gen1_defines.h     |   5 +-
>>>>>     .../platform/qcom/iris/iris_hfi_gen1_response.c    |  37 +-
>>>>>     .../platform/qcom/iris/iris_hfi_gen2_command.c     | 143 +++++++-
>>>>>     .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   5 +
>>>>>     .../platform/qcom/iris/iris_hfi_gen2_response.c    |  57 ++-
>>>>>     drivers/media/platform/qcom/iris/iris_hfi_queue.c  |   2 +-
>>>>>     drivers/media/platform/qcom/iris/iris_instance.h   |   6 +
>>>>>     .../platform/qcom/iris/iris_platform_common.h      |  28 +-
>>>>>     .../media/platform/qcom/iris/iris_platform_gen2.c  | 198 ++++++++--
>>>>>     .../platform/qcom/iris/iris_platform_qcs8300.h     | 126 +++++--
>>>>>     .../platform/qcom/iris/iris_platform_sm8250.c      |  15 +-
>>>>>     drivers/media/platform/qcom/iris/iris_state.c      |   2 +-
>>>>>     drivers/media/platform/qcom/iris/iris_state.h      |   1 +
>>>>>     drivers/media/platform/qcom/iris/iris_vb2.c        |  18 +-
>>>>>     drivers/media/platform/qcom/iris/iris_vdec.c       | 116 +++---
>>>>>     drivers/media/platform/qcom/iris/iris_vdec.h       |  11 +
>>>>>     drivers/media/platform/qcom/iris/iris_vidc.c       |  36 +-
>>>>>     drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397
>>>>> ++++++++++++++++++++-
>>>>>     drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  46 ++-
>>>>>     24 files changed, 1160 insertions(+), 211 deletions(-)
>>>>> ---
>>>>> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
>>>>> change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390
>>>>> prerequisite-message-id:
>>>>> <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org>
>>>>> prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e
>>>>> prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868
>>>>> prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0
>>>>> prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e
>>>>> prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add
>>>>> prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17
>>>>> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578
>>>>> prerequisite-message-id:
>>>>> <20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com>
>>>>> prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f
>>>>> prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132
>>>>> prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac
>>>>> prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046
>>>>> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
>>>>>
>>>>> Best regards,
>>>>
>>>> HEVC & VP9 works fine on HDK8550.
>>>>
>>>> But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get:
>>>> [   44.741670] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.755724] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.776462] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.797179] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.816630] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.837387] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.856812] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.877576] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000005: unknown
>>>> [   44.897000] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000005: insufficient resources
>>>> [   44.917801] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   44.937254] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   44.959128] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   44.981025] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   45.000459] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   45.022376] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   45.041816] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   45.063736] qcom-iris aa00000.video-codec: session error received
>>>> 0x1000009: unknown
>>>> [   45.083167] qcom-iris aa00000.video-codec: session error received
>>>> 0x4000004: invalid operation for current state
>>>> [   45.105459] ------------[ cut here ]------------
>>>> [   45.121152] WARNING: CPU: 6 PID: 573 at
>>>> drivers/media/common/videobuf2/videobuf2-core.c:1827
>>>> vb2_start_streaming+0x100/0x178 [videobuf2_common]
>>>> while VP9 works fine.
>>>>
>>>> Is it a firmware issue ?
>>>>
>>> Looks like resources set to firmware are not sufficient.
>>> I suspect, internal buffers set to firmware are less than what it requires,
>>> this can change for different VPUs. Pls check if there is any difference in
>>> internal buffer calculations between vpu3 and vpu33.
>>
>> I found the fix, it was a difference in buffer calculation, but not for vpu33,
>> but a typo since v2 was right:
>>
>> ========================================><======================================
>> diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
>> b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
>> index 2272f0c21683..ee95fd20b794 100644
>> --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
>> +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
>> @@ -33,7 +33,7 @@ struct iris_inst;
>>   #define H264_DISPLAY_BUF_SIZE          3328
>>   #define H264_NUM_FRM_INFO              66
>>   #define H265_NUM_TILE_COL 32
>> -#define H265_NUM_TILE_ROW 12
>> +#define H265_NUM_TILE_ROW 128
>>   #define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
>>   #define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32))
>> ========================================><======================================
>>
>> This fixes HEVC on SM8650, so with this change VP9 and HEVC works fine, I'm
>> suprised
>> this still works on SM8550 !
>>
> Thanks for checking this.
> but this value has been the same since v1
> https://lore.kernel.org/linux-media/20250408-iris-dec-hevc-vp9-v1-19-acd258778bd6@quicinc.com/\

Yeah indeed I found that in v2 of iris patchset patch 20 the value is right:
https://lore.kernel.org/all/1702899149-21321-21-git-send-email-quic_dikshita@quicinc.com/

Seems the typo appeared on the vp9 & hevc patchset.

> And yes, it works on SM8550, I believe you also tested v3 and confirmed the
> same.
> But anyways, the correct value should be 128 and I confirmed that SM8550
> works with the fix as well.

Good !

> 
>> So please keep VP9 and HEVC enabled on v4.
> Ok


Thanks!
Neil
> 
> Thanks,
> Dikshita
>>
>> Thanks,
>> Neil
>>>
>>> Thanks,
>>> Dikshita
>>>> I've added:
>>>> ========================================><======================================
>>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> index d3026b2bcb70..8c0ab00ab435 100644
>>>> --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
>>>> @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = {
>>>>           .init_hfi_command_ops = iris_hfi_gen2_command_ops_init,
>>>>           .init_hfi_response_ops = iris_hfi_gen2_response_ops_init,
>>>>           .vpu_ops = &iris_vpu33_ops,
>>>> -       .set_preset_registers = iris_set_sm8550_preset_registers,
>>>> +       .set_preset_registers = iris_set_sm8650_preset_registers,
>>>>           .icc_tbl = sm8550_icc_table,
>>>>           .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table),
>>>>           .clk_rst_tbl = sm8650_clk_reset_table,
>>>> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> index 75e9d572e788..9e2d23f12f75 100644
>>>> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h
>>>> @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = {
>>>> "bus", "core" };
>>>>
>>>>    static const char * const sm8650_controller_reset_table[] = { "xo" };
>>>>
>>>> +static void iris_set_sm8650_preset_registers(struct iris_core *core)
>>>> +{
>>>> +       writel(0x0, core->reg_base + 0xB0088);
>>>> +       writel(0x33332222, core->reg_base + 0x13030);
>>>> +       writel(0x44444444, core->reg_base + 0x13034);
>>>> +       writel(0x1022, core->reg_base + 0x13038);
>>>> +       writel(0x0, core->reg_base + 0x13040);
>>>> +       writel(0xFFFF, core->reg_base + 0x13048);
>>>> +       writel(0x33332222, core->reg_base + 0x13430);
>>>> +       writel(0x44444444, core->reg_base + 0x13434);
>>>> +       writel(0x1022, core->reg_base + 0x13438);
>>>> +       writel(0x0, core->reg_base + 0x13440);
>>>> +       writel(0xFFFF, core->reg_base + 0x13448);
>>>> +       writel(0x99, core->reg_base + 0xA013C);
>>>> +}
>>>> +
>>>>    #endif
>>>> ========================================><======================================
>>>> and no change, error still occurs with HEVC decoding.
>>>>
>>>> Thanks,
>>>> Neil
>>

Re: [PATCH v3 05/23] media: iris: Prevent HFI queue writes when core is in deinit state

[Dikshita Agarwal]

On 5/2/2025 7:24 PM, Konrad Dybcio wrote:
> On 5/2/25 2:22 PM, Bryan O'Donoghue wrote:
>> On 01/05/2025 20:13, Dikshita Agarwal wrote:
>>> -    if (core->state == IRIS_CORE_ERROR)
>>> +    if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT)
>>>           return -EINVAL;
>>
>> Instead of checking for 2/3 of the states why not just check for the 1/3 ?
>>
>> enum iris_core_state {
>>         IRIS_CORE_DEINIT,
>>         IRIS_CORE_INIT,
>>         IRIS_CORE_ERROR,
>> };
>>
>> if (core->state != IRIS_CORE_INIT)
>>     return -EINVAL;
>>
>> Cleaner and more explicit - declaring the state you must be in, as opposed to a list of states you should not be in.
> 
> Being explicit in state machines helps maintainability - if we get
> e.g. IRIS_CORE_LIGHT_SLEEP down the line, this could easily fail
> 
Agree, would prefer to keep the check as is
if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT)

Thanks,
Dikshita

> Konrad

Re: [PATCH v3 13/23] media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length

[Dikshita Agarwal]

On 5/5/2025 6:07 PM, Nicolas Dufresne wrote:
> Hi Dikshita,
> 
> Le dimanche 04 mai 2025 à 20:53 +0530, Dikshita Agarwal a écrit :
>>
>>
>> On 5/3/2025 9:39 PM, Nicolas Dufresne wrote:
>>> Hi Dikshita,
>>>
>>> Le vendredi 02 mai 2025 à 00:43 +0530, Dikshita Agarwal a écrit :
>>>> Firmware sends buffers with 0 filled length which needs to be dropped,
>>>> to achieve the same, add V4L2_BUF_FLAG_ERROR to such buffers.
>>>> Also make sure:
>>>> - These 0 length buffers are not returned as result of flush.
>>>> - Its not a buffer with LAST flag enabled which will also have 0 filled
>>>>   length.
>>>
>>> This message is quite vague, is this about capture or output buffers ?
>>> If its output buffers that don't produce capture, I don't see why they
>>> have to be flagged as errors, or why the payload size matter. Then, if
>>> its about assigned capture buffers that did not get used in the end, you
>>> should put them back in the queue instead of returning them to user
>>> space.
>>>
>>> Returning a capture buffers to userspace should only be used if a frame
>>> could not be produced. That imply copying the cookie timestamp from the
>>> src buffers into the capture buffer. Please make sure you don't endup
>>> returning fake erorrs to userspace, which may lead to some frame
>>> metadata being dropped erroneously.
>>>
>> The capture buffers which I am trying to handle here are of 0 byteused
>> which means they don't have any valid data and they have the timestamp
>> copied from src buffers.
>> How these buffers will be handled by client? if we don't associate error
>> flag to such buffers?
> 
> Please share a link to the stream and specify which frames are handled
> this way by your firmware. The answer to your question is entirely
> dependent on the stream you are decoding.
> 
one such stream is
JCT-VC-HEVC_V1 -tv NoOutPrior_A_Qualcomm_1 (fluster stream)

for this stream, POC 17, 22 and 24 are not supposed to be displayed (output
is false), the corresponding capture buffers are returned with 0 filled
length by firmware to driver and expected to be dropped by the client.

Thanks,
Dikshita

> Nicolas
> 
>>
>> Thanks,
>> Dikshita
>>> Nicolas
>>>
>>>>
>>>> Acked-by: Vikash Garodia <quic_vgarodia@quicinc.com>
>>>> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
>>>> ---
>>>>  drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 6 ++++++
>>>>  1 file changed, 6 insertions(+)
>>>>
>>>> diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>>>> b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>>>> index 4488540d1d41..3bb326843a7b 100644
>>>> --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>>>> +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
>>>> @@ -378,6 +378,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst,
>>>>  
>>>>  	buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, hfi_buffer->flags);
>>>>  
>>>> +	if (!buf->data_size && inst->state == IRIS_INST_STREAMING &&
>>>> +	    !(hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) &&
>>>> +	    !(inst->sub_state & IRIS_INST_SUB_DRC)) {
>>>> +		buf->flags |= V4L2_BUF_FLAG_ERROR;
>>>> +	}
>>>> +
>>>>  	return 0;
>>>>  }
>>>>