Re: [PATCH v4] media: iris: add FPS calculation and VPP FW overhead in frequency formula

[Vishnu Reddy <busanna.reddy@oss.qualcomm.com>]

On 3/31/2026 8:29 PM, Vishnu Reddy wrote:
>
> On 3/31/2026 7:45 PM, Bryan O'Donoghue wrote:
>> On 25/03/2026 16:35, Vishnu Reddy wrote:
>>> The driver was using a fixed default FPS value when calculating the VPU
>>> frequency. This caused wrong frequency requests for high‑frame‑rate
>>> streams, for example 4K at 240 FPS. Because of this, the hardware was
>>> running at a lower frequency than needed.
>>>
>>> Add the FPS measurement based on the decoder input buffer arrival rate.
>>> The measured FPS is stored per instance and used in frequency 
>>> calculation
>>> instead of the fixed default FPS. The value is clamped so that it does
>>> not exceed platform limits. Add a VPP firmware overhead when running in
>>> STAGE_2.
>>>
>>> Signed-off-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com>
>>> ---
>>> Changes in v4:
>>> - Renamed the variable names (Vikash Garodia)
>>> - Updated the fps calculation logic (Vikash Garodia)
>>> - Link to v3: 
>>> https://lore.kernel.org/r/20260325-update_fps_calculation-v3-1-b2ec654f7e4d@oss.qualcomm.com
>>>
>>> Changes in v3:
>>> - Updated the frame_count condition check (Vikash Garodia)
>>> - Link to v2: 
>>> https://lore.kernel.org/r/20260305-update_fps_calculation-v2-1-e3b5cccb1246@oss.qualcomm.com
>>>
>>> Changes in v2:
>>> - Replaced div_u64 with mult_frac (Konrad Dybcio)
>>> - Link to v1: 
>>> https://lore.kernel.org/r/20260304-update_fps_calculation-v1-1-4eeac373a504@oss.qualcomm.com
>>> ---
>>>   drivers/media/platform/qcom/iris/iris_instance.h   |  2 ++
>>>   drivers/media/platform/qcom/iris/iris_vdec.c       | 20 
>>> ++++++++++++++++++++
>>>   drivers/media/platform/qcom/iris/iris_vpu_common.c |  6 +++++-
>>>   3 files changed, 27 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/media/platform/qcom/iris/iris_instance.h 
>>> b/drivers/media/platform/qcom/iris/iris_instance.h
>>> index 16965150f427..a47cd949645f 100644
>>> --- a/drivers/media/platform/qcom/iris/iris_instance.h
>>> +++ b/drivers/media/platform/qcom/iris/iris_instance.h
>>> @@ -109,6 +109,8 @@ struct iris_inst {
>>>       u32                metadata_idx;
>>>       u32                codec;
>>>       bool                last_buffer_dequeued;
>>> +    u64                last_buf_ns;
>>> +    u32                frame_counter;
>>>       u32                frame_rate;
>>>       u32                operating_rate;
>>>       u32                hfi_rc_type;
>>> diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c 
>>> b/drivers/media/platform/qcom/iris/iris_vdec.c
>>> index 719217399a30..7fb45df37db6 100644
>>> --- a/drivers/media/platform/qcom/iris/iris_vdec.c
>>> +++ b/drivers/media/platform/qcom/iris/iris_vdec.c
>>> @@ -54,6 +54,7 @@ int iris_vdec_inst_init(struct iris_inst *inst)
>>>       f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>       inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, 
>>> BUF_OUTPUT);
>>>       inst->buffers[BUF_OUTPUT].size = 
>>> f->fmt.pix_mp.plane_fmt[0].sizeimage;
>>> +    inst->frame_rate = MAXIMUM_FPS;
>>>
>>>       memcpy(&inst->fw_caps[0], &core->inst_fw_caps_dec[0],
>>>              INST_FW_CAP_MAX * sizeof(struct platform_inst_fw_cap));
>>> @@ -369,6 +370,8 @@ int iris_vdec_streamon_input(struct iris_inst 
>>> *inst)
>>>       if (ret)
>>>           return ret;
>>>
>>> +    inst->frame_counter = 0;
>>> +
>>>       return iris_process_streamon_input(inst);
>>>   }
>>>
>>> @@ -411,6 +414,7 @@ int iris_vdec_qbuf(struct iris_inst *inst, 
>>> struct vb2_v4l2_buffer *vbuf)
>>>   {
>>>       struct iris_buffer *buf = to_iris_buffer(vbuf);
>>>       struct vb2_buffer *vb2 = &vbuf->vb2_buf;
>>> +    u64 cur_buf_ns, delta_ns;
>>>       struct vb2_queue *q;
>>>       int ret;
>>>
>>> @@ -427,6 +431,22 @@ int iris_vdec_qbuf(struct iris_inst *inst, 
>>> struct vb2_v4l2_buffer *vbuf)
>>>           return 0;
>>>       }
>>>
>>> +    if (buf->type == BUF_INPUT) {
>>> +        cur_buf_ns = ktime_get_ns();
>>> +
>>> +        if (!inst->frame_counter)
>>> +            inst->last_buf_ns = cur_buf_ns;
>>> +
>>> +        inst->frame_counter++;
>>> +        delta_ns = cur_buf_ns - inst->last_buf_ns;
>>> +
>>> +        if (delta_ns >= NSEC_PER_SEC) {
>>> +            inst->frame_rate = clamp_t(u32, inst->frame_counter, 
>>> DEFAULT_FPS,
>>> +                           MAXIMUM_FPS);
>>> +            inst->frame_counter = 0;
>>> +        }
>>> +    }
>>> +
>>>       iris_scale_power(inst);
>>>
>>>       return iris_queue_buffer(inst, buf);
>>> diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.c 
>>> b/drivers/media/platform/qcom/iris/iris_vpu_common.c
>>> index 548e5f1727fd..d621ccffa868 100644
>>> --- a/drivers/media/platform/qcom/iris/iris_vpu_common.c
>>> +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.c
>>> @@ -416,7 +416,7 @@ u64 iris_vpu3x_vpu4x_calculate_frequency(struct 
>>> iris_inst *inst, size_t data_siz
>>>       u32 height, width, mbs_per_second, mbpf;
>>>       u64 fw_cycles, fw_vpp_cycles;
>>>       u64 vsp_cycles, vpp_cycles;
>>> -    u32 fps = DEFAULT_FPS;
>>> +    u32 fps = inst->frame_rate;
>>>
>>>       width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
>>>       height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
>>> @@ -435,6 +435,10 @@ u64 iris_vpu3x_vpu4x_calculate_frequency(struct 
>>> iris_inst *inst, size_t data_siz
>>>       if (inst->fw_caps[PIPE].value > 1)
>>>           vpp_cycles += div_u64(vpp_cycles * 59, 1000);
>>>
>>> +    /* 1.05 is VPP FW overhead */
>>> +    if (inst->fw_caps[STAGE].value == STAGE_2)
>>> +        vpp_cycles += mult_frac(vpp_cycles, 5, 100);
>>> +
>>>       vsp_cycles = fps * data_size * 8;
>>>       vsp_cycles = div_u64(vsp_cycles, 2);
>>>       /* VSP FW overhead 1.05 */
>>>
>>> ---
>>> base-commit: f505e978d1a0442adbbde48aed38c084ddea6d6e
>>> change-id: 20260304-update_fps_calculation-98ee7f7507b1
>>>
>>> Best regards,
>>> -- 
>>> Vishnu Reddy <busanna.reddy@oss.qualcomm.com>
>>>
>>>
>>
>> Please fix your doxygen.
>>
>> [[ATTACHMENT|junit/test-kernel-doc.log.txt]]
>> 4 warnings as errors
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'last_buf_ns' not described in 'iris_inst'
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'frame_counter' not described in 'iris_inst'
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'last_buf_ns' not described in 'iris_inst'
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'frame_counter' not described in 'iris_inst'
>>
>>
>> System Err:
>>
>> [[ATTACHMENT|junit/test-kernel-doc.err.txt]]
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'last_buf_ns' not described in 'iris_inst'
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'frame_counter' not described in 'iris_inst'
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'last_buf_ns' not described in 'iris_inst'
>> Warning: drivers/media/platform/qcom/iris/iris_instance.h:124 struct 
>> member 'frame_counter' not described in 'iris_inst'
>>
> Ack
> Vishnu Reddy.

v5 addressing above, please apply.
Vishnu Reddy.

>> ---
>> bod