All of lore.kernel.org
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Charvi Mehta" <charvi.mehta@arm.com>, devicetree@vger.kernel.org
Cc: robh@kernel.org, conor+dt@kernel.org, media-ci@linuxtv.org
Subject: Re: [RFC PATCH 27/28] media: mali-vpu: add stateful decoder state machine
Date: Thu, 19 Nov 2026 16:10:10 +0000	[thread overview]
Message-ID: <sashiko-outbox-223481@kernel.org> (raw)
In-Reply-To: <20261119153758.1426673-28-charvi.mehta@arm.com>

Thank you for your contribution! Sashiko AI review found 6 potential
issues to consider:
- [High] media: mali-vpu: allow STREAMING to STOPPED on CAPTURE
  streamoff
- [High] media: mali-vpu: missing DRAIN to STOPPED transition breaks
  CMD_START
- [High] media: mali-vpu: handle OUTPUT streamoff in source-change
  states
- [High] media: mali-vpu: handle MALI_VPU_FW_CODE_ALLOC_PARAM during
  DRAIN state
- [Medium] media: mali-vpu: validate queue state on V4L2_DEC_CMD_START
  in decoder_cmd()
- [Medium] media: mali-vpu: avoid premature state transition in
  session_streamoff()
--

commit 97c2307773c544ec658afbe30255359b9cb184c5
Author: Shobhit Pandey <shobhit.pandey@arm.com>

media: mali-vpu: add stateful decoder state machine

This commit replaces the decoder drc_pending flag with an explicit session
state machine tracking queue streaming states, drain handling, and
source-change events.

> diff --git a/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c b/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c
> index 7e34c24ae050f..6ce61de22a802 100644
> --- a/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c
> +++ b/drivers/media/platform/arm/mali-vpu/mali-vpu-dec.c

[ ... ]

> @@ -79,11 +81,31 @@ int mali_vpu_decoder_cmd(struct file *filp, void *priv, struct v4l2_decoder_cmd
>  
>  	switch (cmd->cmd) {
>  	case V4L2_DEC_CMD_STOP:
> -		ret = mali_vpu_session_fw_send_eos(sess);
> +		if (sess->sess_state == MALI_VPU_STATE_DRAIN) {
> +			ret = -EBUSY;
> +			break;
> +		} else if (sess->sess_state == MALI_VPU_STATE_STREAMING) {
> +			ret = mali_vpu_session_fw_send_eos(sess);
> +			if (!ret)
> +				ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_DRAIN);
> +		}
>  		break;
>  	case V4L2_DEC_CMD_START:
> -		vb2_clear_last_buffer_dequeued(v4l2_m2m_get_dst_vq(sess->m2m_ctx));
> -		ret = 0;
> +		if (sess->sess_state == MALI_VPU_STATE_DRAIN &&
> +		    !dst_q->last_buffer_dequeued) {
> +			ret = -EBUSY;
> +			break;
> +		}
> +
> +		ret = mali_vpu_submit_deferred_bufs(sess);
> +		if (ret)
> +			break;
> +
> +		ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_STREAMING);
> +		if (ret)
> +			break;
> +
> +		vb2_clear_last_buffer_dequeued(dst_q);
>  		break;

[Severity: Medium]
Can V4L2_DEC_CMD_START transition sess->sess_state to
MALI_VPU_STATE_STREAMING when only one queue has been started?

Unlike V4L2_DEC_CMD_STOP, V4L2_DEC_CMD_START does not check whether both
queues are streaming before calling mali_vpu_submit_deferred_bufs() and
mali_vpu_dec_update_state().  Because mali_vpu_dec_allow_state_change()
allows MALI_VPU_STATE_STOPPED, MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE, and
MALI_VPU_STATE_RESET to enter MALI_VPU_STATE_STREAMING, calling
V4L2_DEC_CMD_START while only OUTPUT is streaming moves sess->sess_state
to MALI_VPU_STATE_STREAMING while CAPTURE is still off.  When
MALI_VPU_FW_CODE_ALLOC_PARAM later arrives during initial header parsing,
mali_vpu_session_handle_fw_msg() then enters
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE instead of
MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE, and subsequent CAPTURE STREAMON
cannot exit that state, causing mali_vpu_session_qbuf() to immediately
return every queued CAPTURE buffer with V4L2_BUF_FLAG_LAST.

In addition, calling V4L2_DEC_CMD_START in MALI_VPU_STATE_INITIALIZATION
runs mali_vpu_submit_deferred_bufs() and then fails in
mali_vpu_dec_update_state() with -EINVAL.

>  	default:
>  		ret = -EINVAL;
> @@ -93,3 +115,112 @@ int mali_vpu_decoder_cmd(struct file *filp, void *priv, struct v4l2_decoder_cmd
>  	mutex_unlock(&sess->vq_lock);
>  	return ret;
>  }
> +
> +static bool mali_vpu_dec_allow_state_change(struct mali_vpu_session *sess,
> +					    enum mali_vpu_sess_state new_state)
> +{
> +	switch (sess->sess_state) {
> +	case MALI_VPU_STATE_INITIALIZATION:
> +		return (new_state == MALI_VPU_STATE_RESET ||
> +			new_state == MALI_VPU_STATE_STOPPED);
> +	case MALI_VPU_STATE_STREAMING:
> +		return (new_state == MALI_VPU_STATE_RESET ||
> +			new_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE ||
> +			new_state == MALI_VPU_STATE_DRAIN);

[Severity: High]
Does MALI_VPU_STATE_STREAMING also need to allow transitioning to
MALI_VPU_STATE_STOPPED here?

In mali_vpu_dec_state_change_streamoff(), stopping the CAPTURE queue while
in MALI_VPU_STATE_STREAMING requests a transition to
MALI_VPU_STATE_STOPPED.  Because MALI_VPU_STATE_STOPPED is omitted here,
mali_vpu_dec_update_state() returns -EINVAL, causing
mali_vpu_session_streamoff() to jump to err_session, skip stopping and
flushing the firmware, and put the session into a fatal error state via
mali_vpu_session_set_error().

[ ... ]

> +int mali_vpu_dec_state_change_streamoff(struct mali_vpu_session *sess, enum v4l2_buf_type type)
> +{
> +	enum mali_vpu_sess_state new_state;
> +
> +	if (V4L2_TYPE_IS_OUTPUT(type)) {
> +		switch (sess->sess_state) {
> +		case MALI_VPU_STATE_STOPPED:
> +			new_state = MALI_VPU_STATE_INITIALIZATION;
> +			break;
> +		case MALI_VPU_STATE_STREAMING:
> +		case MALI_VPU_STATE_DRAIN:
> +			new_state = MALI_VPU_STATE_RESET;
> +			break;
> +		default:
> +			return 0;
> +		}
> +	} else {

[Severity: High]
What happens when VIDIOC_STREAMOFF is called on the OUTPUT queue while in
MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE or
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE?

Both states hit the default branch and return 0 without updating
sess->sess_state (and mali_vpu_dec_allow_state_change() does not allow
either state to transition to MALI_VPU_STATE_INITIALIZATION or
MALI_VPU_STATE_RESET).  If OUTPUT is streamed off during
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE (for example to seek or tear
down the stream), sess->sess_state remains
MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE; a subsequent OUTPUT STREAMON
stays trapped in that state and bounces every queued CAPTURE buffer with
V4L2_BUF_FLAG_LAST, while a subsequent CAPTURE STREAMOFF sees drc_capture
as true, skips stopping and flushing the firmware, and leaves
sess->sess_state in MALI_VPU_STATE_STOPPED when both queues are off.
Similarly, OUTPUT STREAMOFF during MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE
leaves sess->sess_state unchanged when neither queue is streaming.

> diff --git a/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c b/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c
> index 500d80788d0eb..3ff4635e682eb 100644
> --- a/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c
> +++ b/drivers/media/platform/arm/mali-vpu/mali-vpu-session.c

[ ... ]

> @@ -513,7 +513,14 @@ static int mali_vpu_session_handle_fw_msg(struct mali_vpu_session *sess,
>  		state = mali_vpu_buffer_update_info_to_v4l2(vpu_buf);
>  
>  		mali_vpu_session_buffer_done(sess, vpu_buf, state);
> -
> +		if (vpu_buf->flags & MALI_VPU_BUFFER_EOS) {
> +			if (V4L2_TYPE_IS_OUTPUT(vpu_buf->type) &&
> +			    sess->sess_state == MALI_VPU_STATE_STREAMING) {
> +				ret = mali_vpu_dec_update_state(sess, MALI_VPU_STATE_DRAIN);
> +				if (ret)
> +					return ret;
> +			}
> +		}
>  		return 0;
>  	}

[Severity: High]
Where does sess->sess_state transition from MALI_VPU_STATE_DRAIN to
MALI_VPU_STATE_STOPPED when the final CAPTURE buffer is returned with
MALI_VPU_BUFFER_EOS?

Only OUTPUT buffers with MALI_VPU_BUFFER_EOS update sess->sess_state here,
so sess->sess_state remains MALI_VPU_STATE_DRAIN after the final CAPTURE
buffer is returned and dequeued.  Because mali_vpu_dec_allow_state_change()
does not allow MALI_VPU_STATE_DRAIN to transition directly to
MALI_VPU_STATE_STREAMING, a subsequent V4L2_DEC_CMD_START in
mali_vpu_decoder_cmd() submits deferred buffers and then fails in
mali_vpu_dec_update_state(sess, MALI_VPU_STATE_STREAMING) with -EINVAL
before clearing dst_q->last_buffer_dequeued.

Note that if sess->sess_state moves to MALI_VPU_STATE_STOPPED as soon as
the firmware returns the CAPTURE EOS buffer while CAPTURE is still
streaming, V4L2_DEC_CMD_START and mali_vpu_session_qbuf() also need to
preserve drain gating until dst_q->last_buffer_dequeued is set, and OUTPUT
STREAMOFF in mali_vpu_dec_state_change_streamoff() needs to transition to
MALI_VPU_STATE_RESET rather than MALI_VPU_STATE_INITIALIZATION when
CAPTURE is still streaming.

> @@ -569,17 +576,31 @@ static int mali_vpu_session_handle_fw_msg(struct mali_vpu_session *sess,
>  			fw_msg->data.dpb_held_frames);
>  		return 0;
>  	case MALI_VPU_FW_CODE_OUTPUT_FLUSH:
> -		dev_dbg(sess->vpu_dev->dev, "fw rsp: Dst queue flushed\n");
> -		if (sess->role == MALI_VPU_DECODER && sess->drc_pending)
> +		if (sess->sess_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE ||
> +		    sess->sess_state == MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE) {
>  			return mali_vpu_session_send_drc_event(sess);
> +		}
>  
>  		return 0;
>  	case MALI_VPU_FW_CODE_ALLOC_PARAM:
>  		ret = mali_vpu_session_update_capture_format(sess, fw_msg->data.alloc_params);
> -		if (!ret)
> -			sess->drc_pending = true;
> +		if (ret)
> +			return ret;
>  
> -		return ret;
> +		if (sess->sess_state == MALI_VPU_STATE_STREAMING) {
> +			ret =
> +			mali_vpu_dec_update_state(sess,
> +						  MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE);
> +			if (ret)
> +				return ret;
> +		} else if (sess->sess_state == MALI_VPU_STATE_STOPPED) {
> +			ret = mali_vpu_dec_update_state(sess,
> +							MALI_VPU_DEC_STATE_FIRST_SOURCE_CHANGE);
> +			if (ret)
> +				return ret;
> +		}
> +
> +		return 0;

[Severity: High]
Can MALI_VPU_FW_CODE_ALLOC_PARAM arrive while sess->sess_state is already
MALI_VPU_STATE_DRAIN?

If userspace queues bitstream buffers containing headers or a resolution
change and immediately initiates a drain via V4L2_DEC_CMD_STOP (or an EOS
OUTPUT buffer), sess->sess_state enters MALI_VPU_STATE_DRAIN before the
firmware finishes parsing the queued buffers.  When
MALI_VPU_FW_CODE_ALLOC_PARAM then arrives, sess->sess_state is neither
MALI_VPU_STATE_STREAMING nor MALI_VPU_STATE_STOPPED, so it stays in
MALI_VPU_STATE_DRAIN.  The CAPTURE flush triggered by
MALI_VPU_FW_CODE_SEQUENCE_PARAMS then returns a 0-byte buffer with
V4L2_BUF_FLAG_LAST that queues a premature V4L2_EVENT_EOS in
mali_vpu_session_buffer_done(), and MALI_VPU_FW_CODE_OUTPUT_FLUSH drops
V4L2_EVENT_SOURCE_CHANGE.

[ ... ]

> @@ -1003,12 +1033,19 @@ int mali_vpu_session_streamoff(struct mali_vpu_session *sess, enum v4l2_buf_type
>  	if (sess->error)
>  		return 0;
>  
> +	drc_capture =
> +		sess->sess_state == MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE &&
> +		mali_vpu_is_raw_q(sess, type);
> +
> +	ret = mali_vpu_dec_state_change_streamoff(sess, type);
> +	if (ret)
> +		goto err_session;
> +
>  	if (sess->fw_bin_state != MALI_VPU_SESSION_FW_BIN_READY ||
>  	    !sess->sched_sess.lsid)
>  		return 0;
>  
> -	if (sess->drc_pending && sess->role == MALI_VPU_DECODER &&
> -	    mali_vpu_is_raw_q(sess, type)) {
> +	if (drc_capture) {
>  		ret = mali_vpu_session_wait_pending(sess);

[Severity: Medium]
Does updating sess->sess_state in mali_vpu_dec_state_change_streamoff()
before mali_vpu_session_wait_pending() cause in-flight firmware flush
responses to see the wrong state?

In the drc_capture path, mali_vpu_dec_state_change_streamoff() immediately
changes sess->sess_state from MALI_VPU_DEC_STATE_DYNAMIC_RESOLUTION_CHANGE
to MALI_VPU_STATE_STOPPED before mali_vpu_session_wait_pending() drops
sess->vq_lock to wait for pending firmware responses.  If
MALI_VPU_FW_CODE_OUTPUT_FLUSH or flushed 0-byte CAPTURE buffers (marked
with V4L2_BUF_FLAG_LAST) are processed during that wait,
mali_vpu_session_handle_fw_msg() sees MALI_VPU_STATE_STOPPED, which drops
V4L2_EVENT_SOURCE_CHANGE and causes mali_vpu_session_buffer_done() to emit
a spurious V4L2_EVENT_EOS.  Similarly, during a normal CAPTURE STREAMOFF,
mali_vpu_session_fw_send_flush() returns active CAPTURE buffers with 0
payload size while sess->sess_state is already MALI_VPU_STATE_STOPPED or
MALI_VPU_STATE_INITIALIZATION, also triggering V4L2_EVENT_EOS in
mali_vpu_session_buffer_done().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119153758.1426673-1-charvi.mehta@arm.com?part=27

  reply	other threads:[~2026-11-19 16:10 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 15:37 [RFC PATCH 00/28] media: add arm Mali-V76 video decoder driver Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 01/28] dt-bindings: media: add Arm Mali-V76 VPU Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 02/28] media: mali-vpu: add Mali VPU platform driver Charvi Mehta
2026-11-19 20:04   ` Nicolas Dufresne
2026-11-20 11:48   ` Uwe Kleine-König
2026-11-19 15:37 ` [RFC PATCH 03/28] media: mali-vpu: register V4L2 decoder device and add querycap ioctl Charvi Mehta
2026-11-19 20:10   ` Nicolas Dufresne
2026-11-19 15:37 ` [RFC PATCH 04/28] media: mali-vpu: initialize decoder format state Charvi Mehta
2026-11-19 20:30   ` Nicolas Dufresne
2026-11-19 15:37 ` [RFC PATCH 05/28] media: mali-vpu: add V4L2 mem2mem and VB2 queue support Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 06/28] media: mali-vpu: implement format query ioctls Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 07/28] media: mali-vpu: implement TRY_FMT and S_FMT ioctls Charvi Mehta
2026-11-19 15:55   ` sashiko-bot
2026-11-19 20:44   ` Nicolas Dufresne
2026-11-19 15:37 ` [RFC PATCH 08/28] media: mali-vpu: register V76 decoder controls and introduce UAPI controls Charvi Mehta
2026-11-19 21:06   ` Nicolas Dufresne
2026-11-19 15:37 ` [RFC PATCH 09/28] media: mali-vpu: add MMU software page table manager Charvi Mehta
2026-11-19 15:59   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 10/28] media: mali-vpu: add firmware cache infrastructure Charvi Mehta
2026-11-19 15:50   ` sashiko-bot
2026-11-19 21:34   ` Nicolas Dufresne
2026-11-19 15:37 ` [RFC PATCH 11/28] media: mali-vpu: load firmware for decoder sessions Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 12/28] media: mali-vpu: prepare firmware address space Charvi Mehta
2026-11-19 15:57   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 13/28] media: mali-vpu: add firmware message transport Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 14/28] media: mali-vpu: route decoder controls to firmware Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 15/28] media: mali-vpu: handle firmware RPC requests Charvi Mehta
2026-11-19 15:51   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 16/28] media: mali-vpu: add LSID hardware support Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 17/28] media: mali-vpu: add single session scheduler Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 18/28] media: mali-vpu: handle firmware interrupts Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 19/28] media: mali-vpu: start decoder firmware through LSID scheduling Charvi Mehta
2026-11-19 16:03   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 20/28] media: mali-vpu: implement vb2 buffer queue handling Charvi Mehta
2026-11-19 16:06   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 21/28] media: mali-vpu: handle firmware buffer exchange Charvi Mehta
2026-11-19 15:58   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 22/28] media: mali-vpu: handle fatal firmware error Charvi Mehta
2026-11-19 16:07   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 23/28] media: mali-vpu: propagate buffer metadata Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 24/28] media: mali-vpu: add decoder EOS and DRC handling Charvi Mehta
2026-11-19 16:18   ` sashiko-bot
2026-11-19 15:37 ` [RFC PATCH 25/28] media: mali-vpu: handle firmware buffer parameters Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 26/28] media: mali-vpu: handle firmware event responses Charvi Mehta
2026-11-19 15:37 ` [RFC PATCH 27/28] media: mali-vpu: add stateful decoder state machine Charvi Mehta
2026-11-19 16:10   ` sashiko-bot [this message]
2026-11-19 15:37 ` [RFC PATCH 28/28] media: MAINTAINERS: add Arm Mali-V76 VPU driver Charvi Mehta
2026-11-19 19:44 ` [RFC PATCH 00/28] media: add arm Mali-V76 video decoder driver Nicolas Dufresne

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-223481@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=charvi.mehta@arm.com \
    --cc=conor+dt@kernel.org \
    --cc=devicetree@vger.kernel.org \
    --cc=media-ci@linuxtv.org \
    --cc=robh@kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.