Devicetree
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Charvi Mehta" <charvi.mehta@arm.com>, devicetree@vger.kernel.org
Cc: robh@kernel.org, conor+dt@kernel.org, media-ci@linuxtv.org
Subject: Re: [RFC PATCH 27/28] media: mali-vpu: add stateful decoder state machine
Date: Thu, 19 Nov 2026 16:10:10 +0000	[thread overview]
Message-ID: <sashiko-outbox-223481@kernel.org> (raw)
In-Reply-To: <20261119153758.1426673-28-charvi.mehta@arm.com>

Thank you for your contribution! Sashiko AI review found 6 potential
issues to consider:
- [High] media: mali-vpu: allow STREAMING to STOPPED on CAPTURE
  streamoff
- [High] media: mali-vpu: missing DRAIN to STOPPED transition breaks
  CMD_START
- [High] media: mali-vpu: handle OUTPUT streamoff in source-change
  states
- [High] media: mali-vpu: handle MALI_VPU_FW_CODE_ALLOC_PARAM during
  DRAIN state
- [Medium] media: mali-vpu: validate queue state on V4L2_DEC_CMD_START
  in decoder_cmd()
- [Medium] media: mali-vpu: avoid premature state transition in
  session_streamoff()
--

commit 97c2307773c544ec658afbe30255359b9cb184c5
Author: Shobhit Pandey <shobhit.pandey@arm.com>

media: mali-vpu: add stateful decoder state machine

This commit replaces the decoder drc_pending flag with an explicit session
state machine tracking queue streaming states, drain handling, and
source-change events.

> diff --git a/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c b/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c
> index 7e34c24ae050f..6ce61de22a802 100644
> --- a/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c
> +++ b/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c

[ ... ]

> @@ -79,11 +81,31 @@ int mali_vpu_decoder_cmd(struct file *filp, void *priv, struct v4l2_decoder_cmd
>  
>  	switch (cmd->cmd) {
>  	case V4L2_DEC_CMD_STOP:
> -		ret = mali_vpu_session_fw_send_eos(sess);
> +		if (sess->sess_state == MALI_VPU_STATE_DRAIN) {
> +			ret = -EBUSY;
> +			break;
> +		} else if (sess->sess_state == MALI_VPU_STATE_STREAMING) {
> +			ret = mali_vpu_session_fw_send_eos(sess);
> +			if (!ret)
> +				ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_DRAIN);
> +		}
>  		break;
>  	case V4L2_DEC_CMD_START:
> -		vb2_clear_last_buffer_dequeued(v4l2_m2m_get_dst_vq(sess->m2m_ctx));
> -		ret = 0;
> +		if (sess->sess_state == MALI_VPU_STATE_DRAIN &&
> +		    !dst_q->last_buffer_dequeued) {
> +			ret = -EBUSY;
> +			break;
> +		}
> +
> +		ret = mali_vpu_submit_deferred_bufs(sess);
> +		if (ret)
> +			break;
> +
> +		ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_STREAMING);
> +		if (ret)
> +			break;
> +
> +		vb2_clear_last_buffer_dequeued(dst_q);
>  		break;

[Severity: Medium]
Can V4L2_DEC_CMD_START transition sess->sess_state to
MALI_VPU_STATE_STREAMING when only one queue has been started?

Unlike V4L2_DEC_CMD_STOP, V4L2_DEC_CMD_START does not check whether both
queues are streaming before calling mali_vpu_submit_deferred_bufs() and
mali_vpu_dec_update_state().  Because mali_vpu_dec_allow_state_change()
allows MALI_VPU_STATE_STOPPED, MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE, and
MALI_VPU_STATE_RESET to enter MALI_VPU_STATE_STREAMING, calling
V4L2_DEC_CMD_START while only OUTPUT is streaming moves sess->sess_state
to MALI_VPU_STATE_STREAMING while CAPTURE is still off.  When
MALI_VPU_FW_CODE_ALLOC_PARAM later arrives during initial header parsing,
mali_vpu_session_handle_fw_msg() then enters
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE instead of
MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE, and subsequent CAPTURE STREAMON
cannot exit that state, causing mali_vpu_session_qbuf() to immediately
return every queued CAPTURE buffer with V4L2_BUF_FLAG_LAST.

In addition, calling V4L2_DEC_CMD_START in MALI_VPU_STATE_INITIALIZATION
runs mali_vpu_submit_deferred_bufs() and then fails in
mali_vpu_dec_update_state() with -EINVAL.

>  	default:
>  		ret = -EINVAL;
> @@ -93,3 +115,112 @@ int mali_vpu_decoder_cmd(struct file *filp, void *priv, struct v4l2_decoder_cmd
>  	mutex_unlock(&sess->vq_lock);
>  	return ret;
>  }
> +
> +static bool mali_vpu_dec_allow_state_change(struct mali_vpu_session *sess,
> +					    enum mali_vpu_sess_state new_state)
> +{
> +	switch (sess->sess_state) {
> +	case MALI_VPU_STATE_INITIALIZATION:
> +		return (new_state == MALI_VPU_STATE_RESET ||
> +			new_state == MALI_VPU_STATE_STOPPED);
> +	case MALI_VPU_STATE_STREAMING:
> +		return (new_state == MALI_VPU_STATE_RESET ||
> +			new_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE ||
> +			new_state == MALI_VPU_STATE_DRAIN);

[Severity: High]
Does MALI_VPU_STATE_STREAMING also need to allow transitioning to
MALI_VPU_STATE_STOPPED here?

In mali_vpu_dec_state_change_streamoff(), stopping the CAPTURE queue while
in MALI_VPU_STATE_STREAMING requests a transition to
MALI_VPU_STATE_STOPPED.  Because MALI_VPU_STATE_STOPPED is omitted here,
mali_vpu_dec_update_state() returns -EINVAL, causing
mali_vpu_session_streamoff() to jump to err_session, skip stopping and
flushing the firmware, and put the session into a fatal error state via
mali_vpu_session_set_error().

[ ... ]

> +int mali_vpu_dec_state_change_streamoff(struct mali_vpu_session *sess, enum v4l2_buf_type type)
> +{
> +	enum mali_vpu_sess_state new_state;
> +
> +	if (V4L2_TYPE_IS_OUTPUT(type)) {
> +		switch (sess->sess_state) {
> +		case MALI_VPU_STATE_STOPPED:
> +			new_state = MALI_VPU_STATE_INITIALIZATION;
> +			break;
> +		case MALI_VPU_STATE_STREAMING:
> +		case MALI_VPU_STATE_DRAIN:
> +			new_state = MALI_VPU_STATE_RESET;
> +			break;
> +		default:
> +			return 0;
> +		}
> +	} else {

[Severity: High]
What happens when VIDIOC_STREAMOFF is called on the OUTPUT queue while in
MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE or
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE?

Both states hit the default branch and return 0 without updating
sess->sess_state (and mali_vpu_dec_allow_state_change() does not allow
either state to transition to MALI_VPU_STATE_INITIALIZATION or
MALI_VPU_STATE_RESET).  If OUTPUT is streamed off during
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE (for example to seek or tear
down the stream), sess->sess_state remains
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE; a subsequent OUTPUT STREAMON
stays trapped in that state and bounces every queued CAPTURE buffer with
V4L2_BUF_FLAG_LAST, while a subsequent CAPTURE STREAMOFF sees drc_capture
as true, skips stopping and flushing the firmware, and leaves
sess->sess_state in MALI_VPU_STATE_STOPPED when both queues are off.
Similarly, OUTPUT STREAMOFF during MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE
leaves sess->sess_state unchanged when neither queue is streaming.

> diff --git a/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c b/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c
> index 500d80788d0eb..3ff4635e682eb 100644
> --- a/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c
> +++ b/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c

[ ... ]

> @@ -513,7 +513,14 @@ static int mali_vpu_session_handle_fw_msg(struct mali_vpu_session *sess,
>  		state = mali_vpu_buffer_update_info_to_v4l2(vpu_buf);
>  
>  		mali_vpu_session_buffer_done(sess, vpu_buf, state);
> -
> +		if (vpu_buf->flags & MALI_VPU_BUFFER_EOS) {
> +			if (V4L2_TYPE_IS_OUTPUT(vpu_buf->type) &&
> +			    sess->sess_state == MALI_VPU_STATE_STREAMING) {
> +				ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_DRAIN);
> +				if (ret)
> +					return ret;
> +			}
> +		}
>  		return 0;
>  	}

[Severity: High]
Where does sess->sess_state transition from MALI_VPU_STATE_DRAIN to
MALI_VPU_STATE_STOPPED when the final CAPTURE buffer is returned with
MALI_VPU_BUFFER_EOS?

Only OUTPUT buffers with MALI_VPU_BUFFER_EOS update sess->sess_state here,
so sess->sess_state remains MALI_VPU_STATE_DRAIN after the final CAPTURE
buffer is returned and dequeued.  Because mali_vpu_dec_allow_state_change()
does not allow MALI_VPU_STATE_DRAIN to transition directly to
MALI_VPU_STATE_STREAMING, a subsequent V4L2_DEC_CMD_START in
mali_vpu_decoder_cmd() submits deferred buffers and then fails in
mali_vpu_dec_update_state(sess, MALI_VPU_STATE_STREAMING) with -EINVAL
before clearing dst_q->last_buffer_dequeued.

Note that if sess->sess_state moves to MALI_VPU_STATE_STOPPED as soon as
the firmware returns the CAPTURE EOS buffer while CAPTURE is still
streaming, V4L2_DEC_CMD_START and mali_vpu_session_qbuf() also need to
preserve drain gating until dst_q->last_buffer_dequeued is set, and OUTPUT
STREAMOFF in mali_vpu_dec_state_change_streamoff() needs to transition to
MALI_VPU_STATE_RESET rather than MALI_VPU_STATE_INITIALIZATION when
CAPTURE is still streaming.

> @@ -569,17 +576,31 @@ static int mali_vpu_session_handle_fw_msg(struct mali_vpu_session *sess,
>  			fw_msg->data.dpb_held_frames);
>  		return 0;
>  	case MALI_VPU_FW_CODE_OUTPUT_FLUSH:
> -		dev_dbg(sess->vpu_dev->dev, "fw rsp: Dst queue flushed\n");
> -		if (sess->role == MALI_VPU_DECODER && sess->drc_pending)
> +		if (sess->sess_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE ||
> +		    sess->sess_state == MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE) {
>  			return mali_vpu_session_send_drc_event(sess);
> +		}
>  
>  		return 0;
>  	case MALI_VPU_FW_CODE_ALLOC_PARAM:
>  		ret = mali_vpu_session_update_capture_format(sess, fw_msg->data.alloc_params);
> -		if (!ret)
> -			sess->drc_pending = true;
> +		if (ret)
> +			return ret;
>  
> -		return ret;
> +		if (sess->sess_state == MALI_VPU_STATE_STREAMING) {
> +			ret =
> +			mali_vpu_dec_update_state(sess,
> +						  MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE);
> +			if (ret)
> +				return ret;
> +		} else if (sess->sess_state == MALI_VPU_STATE_STOPPED) {
> +			ret = mali_vpu_dec_update_state(sess,
> +							MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE);
> +			if (ret)
> +				return ret;
> +		}
> +
> +		return 0;

[Severity: High]
Can MALI_VPU_FW_CODE_ALLOC_PARAM arrive while sess->sess_state is already
MALI_VPU_STATE_DRAIN?

If userspace queues bitstream buffers containing headers or a resolution
change and immediately initiates a drain via V4L2_DEC_CMD_STOP (or an EOS
OUTPUT buffer), sess->sess_state enters MALI_VPU_STATE_DRAIN before the
firmware finishes parsing the queued buffers.  When
MALI_VPU_FW_CODE_ALLOC_PARAM then arrives, sess->sess_state is neither
MALI_VPU_STATE_STREAMING nor MALI_VPU_STATE_STOPPED, so it stays in
MALI_VPU_STATE_DRAIN.  The CAPTURE flush triggered by
MALI_VPU_FW_CODE_SEQUENCE_PARAMS then returns a 0-byte buffer with
V4L2_BUF_FLAG_LAST that queues a premature V4L2_EVENT_EOS in
mali_vpu_session_buffer_done(), and MALI_VPU_FW_CODE_OUTPUT_FLUSH drops
V4L2_EVENT_SOURCE_CHANGE.

[ ... ]

> @@ -1003,12 +1033,19 @@ int mali_vpu_session_streamoff(struct mali_vpu_session *sess, enum v4l2_buf_type
>  	if (sess->error)
>  		return 0;
>  
> +	drc_capture =
> +		sess->sess_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE &&
> +		mali_vpu_is_raw_q(sess, type);
> +
> +	ret = mali_vpu_dec_state_change_streamoff(sess, type);
> +	if (ret)
> +		goto err_session;
> +
>  	if (sess->fw_bin_state != MALI_VPU_SESSION_FW_BIN_READY ||
>  	    !sess->sched_sess.lsid)
>  		return 0;
>  
> -	if (sess->drc_pending && sess->role == MALI_VPU_DECODER &&
> -	    mali_vpu_is_raw_q(sess, type)) {
> +	if (drc_capture) {
>  		ret = mali_vpu_session_wait_pending(sess);

[Severity: Medium]
Does updating sess->sess_state in mali_vpu_dec_state_change_streamoff()
before mali_vpu_session_wait_pending() cause in-flight firmware flush
responses to see the wrong state?

In the drc_capture path, mali_vpu_dec_state_change_streamoff() immediately
changes sess->sess_state from MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE
to MALI_VPU_STATE_STOPPED before mali_vpu_session_wait_pending() drops
sess->vq_lock to wait for pending firmware responses.  If
MALI_VPU_FW_CODE_OUTPUT_FLUSH or flushed 0-byte CAPTURE buffers (marked
with V4L2_BUF_FLAG_LAST) are processed during that wait,
mali_vpu_session_handle_fw_msg() sees MALI_VPU_STATE_STOPPED, which drops
V4L2_EVENT_SOURCE_CHANGE and causes mali_vpu_session_buffer_done() to emit
a spurious V4L2_EVENT_EOS.  Similarly, during a normal CAPTURE STREAMOFF,
mali_vpu_session_fw_send_flush() returns active CAPTURE buffers with 0
payload size while sess->sess_state is already MALI_VPU_STATE_STOPPED or
MALI_VPU_STATE_INITIALIZATION, also triggering V4L2_EVENT_EOS in
mali_vpu_session_buffer_done().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119153758.1426673-1-charvi.mehta@arm.com?part=27

  reply	other threads:[~2026-11-19 16:10 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 15:37 [RFC PATCH 00/28] media: add arm Mali-V76 video decoder driver Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 01/28] dt-bindings: media: add Arm Mali-V76 VPU Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 02/28] media: mali-vpu: add Mali VPU platform driver Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 03/28] media: mali-vpu: register V4L2 decoder device and add querycap ioctl Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 04/28] media: mali-vpu: initialize decoder format state Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 05/28] media: mali-vpu: add V4L2 mem2mem and VB2 queue support Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 06/28] media: mali-vpu: implement format query ioctls Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 07/28] media: mali-vpu: implement TRY_FMT and S_FMT ioctls Charvi Mehta
2026-11-19 15:55   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 08/28] media: mali-vpu: register V76 decoder controls and introduce UAPI controls Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 09/28] media: mali-vpu: add MMU software page table manager Charvi Mehta
2026-11-19 15:59   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 10/28] media: mali-vpu: add firmware cache infrastructure Charvi Mehta
2026-11-19 15:50   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 11/28] media: mali-vpu: load firmware for decoder sessions Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 12/28] media: mali-vpu: prepare firmware address space Charvi Mehta
2026-11-19 15:57   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 13/28] media: mali-vpu: add firmware message transport Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 14/28] media: mali-vpu: route decoder controls to firmware Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 15/28] media: mali-vpu: handle firmware RPC requests Charvi Mehta
2026-11-19 15:51   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 16/28] media: mali-vpu: add LSID hardware support Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 17/28] media: mali-vpu: add single session scheduler Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 18/28] media: mali-vpu: handle firmware interrupts Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 19/28] media: mali-vpu: start decoder firmware through LSID scheduling Charvi Mehta
2026-11-19 16:03   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 20/28] media: mali-vpu: implement vb2 buffer queue handling Charvi Mehta
2026-11-19 16:06   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 21/28] media: mali-vpu: handle firmware buffer exchange Charvi Mehta
2026-11-19 15:58   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 22/28] media: mali-vpu: handle fatal firmware error Charvi Mehta
2026-11-19 16:07   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 23/28] media: mali-vpu: propagate buffer metadata Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 24/28] media: mali-vpu: add decoder EOS and DRC handling Charvi Mehta
2026-11-19 16:18   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 25/28] media: mali-vpu: handle firmware buffer parameters Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 26/28] media: mali-vpu: handle firmware event responses Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 27/28] media: mali-vpu: add stateful decoder state machine Charvi Mehta
2026-11-19 16:10   ` sashiko-bot [this message]
2026-11-19 15:37 ` [RFC PATCH 28/28] media: MAINTAINERS: add Arm Mali-V76 VPU driver Charvi Mehta

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-223481@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=charvi.mehta@arm.com \
    --cc=conor+dt@kernel.org \
    --cc=devicetree@vger.kernel.org \
    --cc=media-ci@linuxtv.org \
    --cc=robh@kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox