From: sashiko-bot@kernel.org
To: "Charvi Mehta" <charvi.mehta@arm.com>, devicetree@vger.kernel.org
Cc: robh@kernel.org, conor+dt@kernel.org, media-ci@linuxtv.org
Subject: Re: [RFC PATCH 27/28] media: mali-vpu: add stateful decoder state machine
Date: Thu, 19 Nov 2026 16:10:10 +0000 [thread overview]
Message-ID: <sashiko-outbox-223481@kernel.org> (raw)
In-Reply-To: <20261119153758.1426673-28-charvi.mehta@arm.com>
Thank you for your contribution! Sashiko AI review found 6 potential
issues to consider:
- [High] media: mali-vpu: allow STREAMING to STOPPED on CAPTURE
streamoff
- [High] media: mali-vpu: missing DRAIN to STOPPED transition breaks
CMD_START
- [High] media: mali-vpu: handle OUTPUT streamoff in source-change
states
- [High] media: mali-vpu: handle MALI_VPU_FW_CODE_ALLOC_PARAM during
DRAIN state
- [Medium] media: mali-vpu: validate queue state on V4L2_DEC_CMD_START
in decoder_cmd()
- [Medium] media: mali-vpu: avoid premature state transition in
session_streamoff()
--
commit 97c2307773c544ec658afbe30255359b9cb184c5
Author: Shobhit Pandey <shobhit.pandey@arm.com>
media: mali-vpu: add stateful decoder state machine
This commit replaces the decoder drc_pending flag with an explicit session
state machine tracking queue streaming states, drain handling, and
source-change events.
> diff --git a/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c b/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c
> index 7e34c24ae050f..6ce61de22a802 100644
> --- a/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c
> +++ b/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c
[ ... ]
> @@ -79,11 +81,31 @@ int mali_vpu_decoder_cmd(struct file *filp, void *priv, struct v4l2_decoder_cmd
>
> switch (cmd->cmd) {
> case V4L2_DEC_CMD_STOP:
> - ret = mali_vpu_session_fw_send_eos(sess);
> + if (sess->sess_state == MALI_VPU_STATE_DRAIN) {
> + ret = -EBUSY;
> + break;
> + } else if (sess->sess_state == MALI_VPU_STATE_STREAMING) {
> + ret = mali_vpu_session_fw_send_eos(sess);
> + if (!ret)
> + ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_DRAIN);
> + }
> break;
> case V4L2_DEC_CMD_START:
> - vb2_clear_last_buffer_dequeued(v4l2_m2m_get_dst_vq(sess->m2m_ctx));
> - ret = 0;
> + if (sess->sess_state == MALI_VPU_STATE_DRAIN &&
> + !dst_q->last_buffer_dequeued) {
> + ret = -EBUSY;
> + break;
> + }
> +
> + ret = mali_vpu_submit_deferred_bufs(sess);
> + if (ret)
> + break;
> +
> + ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_STREAMING);
> + if (ret)
> + break;
> +
> + vb2_clear_last_buffer_dequeued(dst_q);
> break;
[Severity: Medium]
Can V4L2_DEC_CMD_START transition sess->sess_state to
MALI_VPU_STATE_STREAMING when only one queue has been started?
Unlike V4L2_DEC_CMD_STOP, V4L2_DEC_CMD_START does not check whether both
queues are streaming before calling mali_vpu_submit_deferred_bufs() and
mali_vpu_dec_update_state(). Because mali_vpu_dec_allow_state_change()
allows MALI_VPU_STATE_STOPPED, MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE, and
MALI_VPU_STATE_RESET to enter MALI_VPU_STATE_STREAMING, calling
V4L2_DEC_CMD_START while only OUTPUT is streaming moves sess->sess_state
to MALI_VPU_STATE_STREAMING while CAPTURE is still off. When
MALI_VPU_FW_CODE_ALLOC_PARAM later arrives during initial header parsing,
mali_vpu_session_handle_fw_msg() then enters
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE instead of
MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE, and subsequent CAPTURE STREAMON
cannot exit that state, causing mali_vpu_session_qbuf() to immediately
return every queued CAPTURE buffer with V4L2_BUF_FLAG_LAST.
In addition, calling V4L2_DEC_CMD_START in MALI_VPU_STATE_INITIALIZATION
runs mali_vpu_submit_deferred_bufs() and then fails in
mali_vpu_dec_update_state() with -EINVAL.
> default:
> ret = -EINVAL;
> @@ -93,3 +115,112 @@ int mali_vpu_decoder_cmd(struct file *filp, void *priv, struct v4l2_decoder_cmd
> mutex_unlock(&sess->vq_lock);
> return ret;
> }
> +
> +static bool mali_vpu_dec_allow_state_change(struct mali_vpu_session *sess,
> + enum mali_vpu_sess_state new_state)
> +{
> + switch (sess->sess_state) {
> + case MALI_VPU_STATE_INITIALIZATION:
> + return (new_state == MALI_VPU_STATE_RESET ||
> + new_state == MALI_VPU_STATE_STOPPED);
> + case MALI_VPU_STATE_STREAMING:
> + return (new_state == MALI_VPU_STATE_RESET ||
> + new_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE ||
> + new_state == MALI_VPU_STATE_DRAIN);
[Severity: High]
Does MALI_VPU_STATE_STREAMING also need to allow transitioning to
MALI_VPU_STATE_STOPPED here?
In mali_vpu_dec_state_change_streamoff(), stopping the CAPTURE queue while
in MALI_VPU_STATE_STREAMING requests a transition to
MALI_VPU_STATE_STOPPED. Because MALI_VPU_STATE_STOPPED is omitted here,
mali_vpu_dec_update_state() returns -EINVAL, causing
mali_vpu_session_streamoff() to jump to err_session, skip stopping and
flushing the firmware, and put the session into a fatal error state via
mali_vpu_session_set_error().
[ ... ]
> +int mali_vpu_dec_state_change_streamoff(struct mali_vpu_session *sess, enum v4l2_buf_type type)
> +{
> + enum mali_vpu_sess_state new_state;
> +
> + if (V4L2_TYPE_IS_OUTPUT(type)) {
> + switch (sess->sess_state) {
> + case MALI_VPU_STATE_STOPPED:
> + new_state = MALI_VPU_STATE_INITIALIZATION;
> + break;
> + case MALI_VPU_STATE_STREAMING:
> + case MALI_VPU_STATE_DRAIN:
> + new_state = MALI_VPU_STATE_RESET;
> + break;
> + default:
> + return 0;
> + }
> + } else {
[Severity: High]
What happens when VIDIOC_STREAMOFF is called on the OUTPUT queue while in
MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE or
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE?
Both states hit the default branch and return 0 without updating
sess->sess_state (and mali_vpu_dec_allow_state_change() does not allow
either state to transition to MALI_VPU_STATE_INITIALIZATION or
MALI_VPU_STATE_RESET). If OUTPUT is streamed off during
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE (for example to seek or tear
down the stream), sess->sess_state remains
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE; a subsequent OUTPUT STREAMON
stays trapped in that state and bounces every queued CAPTURE buffer with
V4L2_BUF_FLAG_LAST, while a subsequent CAPTURE STREAMOFF sees drc_capture
as true, skips stopping and flushing the firmware, and leaves
sess->sess_state in MALI_VPU_STATE_STOPPED when both queues are off.
Similarly, OUTPUT STREAMOFF during MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE
leaves sess->sess_state unchanged when neither queue is streaming.
> diff --git a/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c b/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c
> index 500d80788d0eb..3ff4635e682eb 100644
> --- a/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c
> +++ b/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c
[ ... ]
> @@ -513,7 +513,14 @@ static int mali_vpu_session_handle_fw_msg(struct mali_vpu_session *sess,
> state = mali_vpu_buffer_update_info_to_v4l2(vpu_buf);
>
> mali_vpu_session_buffer_done(sess, vpu_buf, state);
> -
> + if (vpu_buf->flags & MALI_VPU_BUFFER_EOS) {
> + if (V4L2_TYPE_IS_OUTPUT(vpu_buf->type) &&
> + sess->sess_state == MALI_VPU_STATE_STREAMING) {
> + ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_DRAIN);
> + if (ret)
> + return ret;
> + }
> + }
> return 0;
> }
[Severity: High]
Where does sess->sess_state transition from MALI_VPU_STATE_DRAIN to
MALI_VPU_STATE_STOPPED when the final CAPTURE buffer is returned with
MALI_VPU_BUFFER_EOS?
Only OUTPUT buffers with MALI_VPU_BUFFER_EOS update sess->sess_state here,
so sess->sess_state remains MALI_VPU_STATE_DRAIN after the final CAPTURE
buffer is returned and dequeued. Because mali_vpu_dec_allow_state_change()
does not allow MALI_VPU_STATE_DRAIN to transition directly to
MALI_VPU_STATE_STREAMING, a subsequent V4L2_DEC_CMD_START in
mali_vpu_decoder_cmd() submits deferred buffers and then fails in
mali_vpu_dec_update_state(sess, MALI_VPU_STATE_STREAMING) with -EINVAL
before clearing dst_q->last_buffer_dequeued.
Note that if sess->sess_state moves to MALI_VPU_STATE_STOPPED as soon as
the firmware returns the CAPTURE EOS buffer while CAPTURE is still
streaming, V4L2_DEC_CMD_START and mali_vpu_session_qbuf() also need to
preserve drain gating until dst_q->last_buffer_dequeued is set, and OUTPUT
STREAMOFF in mali_vpu_dec_state_change_streamoff() needs to transition to
MALI_VPU_STATE_RESET rather than MALI_VPU_STATE_INITIALIZATION when
CAPTURE is still streaming.
> @@ -569,17 +576,31 @@ static int mali_vpu_session_handle_fw_msg(struct mali_vpu_session *sess,
> fw_msg->data.dpb_held_frames);
> return 0;
> case MALI_VPU_FW_CODE_OUTPUT_FLUSH:
> - dev_dbg(sess->vpu_dev->dev, "fw rsp: Dst queue flushed\n");
> - if (sess->role == MALI_VPU_DECODER && sess->drc_pending)
> + if (sess->sess_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE ||
> + sess->sess_state == MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE) {
> return mali_vpu_session_send_drc_event(sess);
> + }
>
> return 0;
> case MALI_VPU_FW_CODE_ALLOC_PARAM:
> ret = mali_vpu_session_update_capture_format(sess, fw_msg->data.alloc_params);
> - if (!ret)
> - sess->drc_pending = true;
> + if (ret)
> + return ret;
>
> - return ret;
> + if (sess->sess_state == MALI_VPU_STATE_STREAMING) {
> + ret =
> + mali_vpu_dec_update_state(sess,
> + MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE);
> + if (ret)
> + return ret;
> + } else if (sess->sess_state == MALI_VPU_STATE_STOPPED) {
> + ret = mali_vpu_dec_update_state(sess,
> + MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
[Severity: High]
Can MALI_VPU_FW_CODE_ALLOC_PARAM arrive while sess->sess_state is already
MALI_VPU_STATE_DRAIN?
If userspace queues bitstream buffers containing headers or a resolution
change and immediately initiates a drain via V4L2_DEC_CMD_STOP (or an EOS
OUTPUT buffer), sess->sess_state enters MALI_VPU_STATE_DRAIN before the
firmware finishes parsing the queued buffers. When
MALI_VPU_FW_CODE_ALLOC_PARAM then arrives, sess->sess_state is neither
MALI_VPU_STATE_STREAMING nor MALI_VPU_STATE_STOPPED, so it stays in
MALI_VPU_STATE_DRAIN. The CAPTURE flush triggered by
MALI_VPU_FW_CODE_SEQUENCE_PARAMS then returns a 0-byte buffer with
V4L2_BUF_FLAG_LAST that queues a premature V4L2_EVENT_EOS in
mali_vpu_session_buffer_done(), and MALI_VPU_FW_CODE_OUTPUT_FLUSH drops
V4L2_EVENT_SOURCE_CHANGE.
[ ... ]
> @@ -1003,12 +1033,19 @@ int mali_vpu_session_streamoff(struct mali_vpu_session *sess, enum v4l2_buf_type
> if (sess->error)
> return 0;
>
> + drc_capture =
> + sess->sess_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE &&
> + mali_vpu_is_raw_q(sess, type);
> +
> + ret = mali_vpu_dec_state_change_streamoff(sess, type);
> + if (ret)
> + goto err_session;
> +
> if (sess->fw_bin_state != MALI_VPU_SESSION_FW_BIN_READY ||
> !sess->sched_sess.lsid)
> return 0;
>
> - if (sess->drc_pending && sess->role == MALI_VPU_DECODER &&
> - mali_vpu_is_raw_q(sess, type)) {
> + if (drc_capture) {
> ret = mali_vpu_session_wait_pending(sess);
[Severity: Medium]
Does updating sess->sess_state in mali_vpu_dec_state_change_streamoff()
before mali_vpu_session_wait_pending() cause in-flight firmware flush
responses to see the wrong state?
In the drc_capture path, mali_vpu_dec_state_change_streamoff() immediately
changes sess->sess_state from MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE
to MALI_VPU_STATE_STOPPED before mali_vpu_session_wait_pending() drops
sess->vq_lock to wait for pending firmware responses. If
MALI_VPU_FW_CODE_OUTPUT_FLUSH or flushed 0-byte CAPTURE buffers (marked
with V4L2_BUF_FLAG_LAST) are processed during that wait,
mali_vpu_session_handle_fw_msg() sees MALI_VPU_STATE_STOPPED, which drops
V4L2_EVENT_SOURCE_CHANGE and causes mali_vpu_session_buffer_done() to emit
a spurious V4L2_EVENT_EOS. Similarly, during a normal CAPTURE STREAMOFF,
mali_vpu_session_fw_send_flush() returns active CAPTURE buffers with 0
payload size while sess->sess_state is already MALI_VPU_STATE_STOPPED or
MALI_VPU_STATE_INITIALIZATION, also triggering V4L2_EVENT_EOS in
mali_vpu_session_buffer_done().
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119153758.1426673-1-charvi.mehta@arm.com?part=27
next prev parent reply other threads:[~2026-11-19 16:10 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 15:37 [RFC PATCH 00/28] media: add arm Mali-V76 video decoder driver Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 01/28] dt-bindings: media: add Arm Mali-V76 VPU Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 02/28] media: mali-vpu: add Mali VPU platform driver Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 03/28] media: mali-vpu: register V4L2 decoder device and add querycap ioctl Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 04/28] media: mali-vpu: initialize decoder format state Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 05/28] media: mali-vpu: add V4L2 mem2mem and VB2 queue support Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 06/28] media: mali-vpu: implement format query ioctls Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 07/28] media: mali-vpu: implement TRY_FMT and S_FMT ioctls Charvi Mehta
2026-11-19 15:55 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 08/28] media: mali-vpu: register V76 decoder controls and introduce UAPI controls Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 09/28] media: mali-vpu: add MMU software page table manager Charvi Mehta
2026-11-19 15:59 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 10/28] media: mali-vpu: add firmware cache infrastructure Charvi Mehta
2026-11-19 15:50 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 11/28] media: mali-vpu: load firmware for decoder sessions Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 12/28] media: mali-vpu: prepare firmware address space Charvi Mehta
2026-11-19 15:57 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 13/28] media: mali-vpu: add firmware message transport Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 14/28] media: mali-vpu: route decoder controls to firmware Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 15/28] media: mali-vpu: handle firmware RPC requests Charvi Mehta
2026-11-19 15:51 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 16/28] media: mali-vpu: add LSID hardware support Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 17/28] media: mali-vpu: add single session scheduler Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 18/28] media: mali-vpu: handle firmware interrupts Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 19/28] media: mali-vpu: start decoder firmware through LSID scheduling Charvi Mehta
2026-11-19 16:03 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 20/28] media: mali-vpu: implement vb2 buffer queue handling Charvi Mehta
2026-11-19 16:06 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 21/28] media: mali-vpu: handle firmware buffer exchange Charvi Mehta
2026-11-19 15:58 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 22/28] media: mali-vpu: handle fatal firmware error Charvi Mehta
2026-11-19 16:07 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 23/28] media: mali-vpu: propagate buffer metadata Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 24/28] media: mali-vpu: add decoder EOS and DRC handling Charvi Mehta
2026-11-19 16:18 ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 25/28] media: mali-vpu: handle firmware buffer parameters Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 26/28] media: mali-vpu: handle firmware event responses Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 27/28] media: mali-vpu: add stateful decoder state machine Charvi Mehta
2026-11-19 16:10 ` sashiko-bot [this message]
2026-11-19 15:37 ` [RFC PATCH 28/28] media: MAINTAINERS: add Arm Mali-V76 VPU driver Charvi Mehta
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-223481@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=charvi.mehta@arm.com \
--cc=conor+dt@kernel.org \
--cc=devicetree@vger.kernel.org \
--cc=media-ci@linuxtv.org \
--cc=robh@kernel.org \
--cc=sashiko-reviews@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox