[PATCH v1 0/4] media: rkvdec: Fix H.264 error resilience

[PATCH v1 0/4] media: rkvdec: Fix H.264 error resilience

[Nicolas Dufresne]

This patch serie changes the decoding mode from "exit on error"
to "keep decoding". Using this mode and re-enabling error detection
allow getting error resilience without loosing the ability to report
errors to userland. This have showed great results, but might be a
little more risky since this is not the mode that the reference code
uses and the documentation is very brief. With this in place,
userspace can chose to skip or display corrupted frames depending
on its application requirement. Previsouly, applicaiton would have
had no choice but to present the corrupted frames.

Changes since V1:
	- Removed merged patch
	- Changed usage of pr_debug into v4l2_dbg
	- Fix typos in commit messages and comments

Nicolas Dufresne (5):
  media: rkvdec: Disable H.264 error detection
  media: rkvdec: Add an ops to check for decode errors
  media: rkvdec: Fix RKVDEC_ERR_PKT_NUM macro
  media: rkvdec: Re-enable H.264 error detection
  rkvdec: Improve error handling

 drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++--
 drivers/staging/media/rkvdec/rkvdec-regs.h |  2 +-
 drivers/staging/media/rkvdec/rkvdec.c      | 34 ++++++++++++++++++----
 drivers/staging/media/rkvdec/rkvdec.h      |  2 ++
 4 files changed, 51 insertions(+), 10 deletions(-)

-- 
2.38.1

[PATCH v2 1/4] media: rkvdec: Add an ops to check for decode errors

[Nicolas Dufresne]

This optional internal ops allow each codec to do their own
error status checking. The presence of an error is reported
using the ERROR buffer state. This patch have no functional
changes.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
 drivers/staging/media/rkvdec/rkvdec.c | 10 ++++++----
 drivers/staging/media/rkvdec/rkvdec.h |  2 ++
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 7bab7586918c1..7e76f8b728854 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -950,6 +950,7 @@ static void rkvdec_v4l2_cleanup(struct rkvdec_dev *rkvdec)
 static irqreturn_t rkvdec_irq_handler(int irq, void *priv)
 {
 	struct rkvdec_dev *rkvdec = priv;
+	struct rkvdec_ctx *ctx;
 	enum vb2_buffer_state state;
 	u32 status;
 
@@ -958,12 +959,13 @@ static irqreturn_t rkvdec_irq_handler(int irq, void *priv)
 		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
 
 	writel(0, rkvdec->regs + RKVDEC_REG_INTERRUPT);
-	if (cancel_delayed_work(&rkvdec->watchdog_work)) {
-		struct rkvdec_ctx *ctx;
+	ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
 
-		ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
+	if (ctx->coded_fmt_desc->ops->check_error_info)
+		state = ctx->coded_fmt_desc->ops->check_error_info(ctx);
+
+	if (cancel_delayed_work(&rkvdec->watchdog_work))
 		rkvdec_job_finish(ctx, state);
-	}
 
 	return IRQ_HANDLED;
 }
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 633335ebb9c49..4ae8e6c6b03c9 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -73,6 +73,8 @@ struct rkvdec_coded_fmt_ops {
 		     struct vb2_v4l2_buffer *dst_buf,
 		     enum vb2_buffer_state result);
 	int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
+	/* called from IRQ handler */
+	int (*check_error_info)(struct rkvdec_ctx *ctx);
 };
 
 struct rkvdec_coded_fmt_desc {
-- 
2.38.1

Re: [PATCH v2 1/4] media: rkvdec: Add an ops to check for decode errors

[Ezequiel Garcia]

Hi Nicolas,

Thanks for the patch.

On Fri, Dec 23, 2022 at 02:38:03PM -0500, Nicolas Dufresne wrote:
> This optional internal ops allow each codec to do their own
> error status checking. The presence of an error is reported
> using the ERROR buffer state. This patch have no functional
> changes.
> 
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
>  drivers/staging/media/rkvdec/rkvdec.c | 10 ++++++----
>  drivers/staging/media/rkvdec/rkvdec.h |  2 ++
>  2 files changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 7bab7586918c1..7e76f8b728854 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -950,6 +950,7 @@ static void rkvdec_v4l2_cleanup(struct rkvdec_dev *rkvdec)
>  static irqreturn_t rkvdec_irq_handler(int irq, void *priv)
>  {
>  	struct rkvdec_dev *rkvdec = priv;
> +	struct rkvdec_ctx *ctx;
>  	enum vb2_buffer_state state;
>  	u32 status;
>  
> @@ -958,12 +959,13 @@ static irqreturn_t rkvdec_irq_handler(int irq, void *priv)
>  		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
>  
>  	writel(0, rkvdec->regs + RKVDEC_REG_INTERRUPT);
> -	if (cancel_delayed_work(&rkvdec->watchdog_work)) {
> -		struct rkvdec_ctx *ctx;
> +	ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
>  
> -		ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
> +	if (ctx->coded_fmt_desc->ops->check_error_info)
> +		state = ctx->coded_fmt_desc->ops->check_error_info(ctx);
> +
> +	if (cancel_delayed_work(&rkvdec->watchdog_work))
>  		rkvdec_job_finish(ctx, state);
> -	}
>  
>  	return IRQ_HANDLED;
>  }
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index 633335ebb9c49..4ae8e6c6b03c9 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -73,6 +73,8 @@ struct rkvdec_coded_fmt_ops {
>  		     struct vb2_v4l2_buffer *dst_buf,
>  		     enum vb2_buffer_state result);
>  	int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
> +	/* called from IRQ handler */
> +	int (*check_error_info)(struct rkvdec_ctx *ctx);

I don't think it's a good idea to mix the return of this internal API
with enum vb2_buffer_state.

Please make the return type of this function a boolean or an integer
type that is decoupled from the VB2 buffer state.

Thanks!
Ezequiel

>  };
>  
>  struct rkvdec_coded_fmt_desc {
> -- 
> 2.38.1
> 

[PATCH v2 2/4] media: rkvdec: Fix RKVDEC_ERR_PKT_NUM macro

[Nicolas Dufresne]

This information is expressed by bits [29:16], but the actual
implementation was reading bits [13:0] and shifting that 16
bits to the left.

Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
 drivers/staging/media/rkvdec/rkvdec-regs.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec-regs.h b/drivers/staging/media/rkvdec/rkvdec-regs.h
index 15b9bee92016c..14530b81560ed 100644
--- a/drivers/staging/media/rkvdec/rkvdec-regs.h
+++ b/drivers/staging/media/rkvdec/rkvdec-regs.h
@@ -212,7 +212,7 @@
 #define RKVDEC_REG_H264_ERRINFO_NUM			0x130
 #define RKVDEC_SLICEDEC_NUM(x)				((x) & 0x3fff)
 #define RKVDEC_STRMD_DECT_ERR_FLAG			BIT(15)
-#define RKVDEC_ERR_PKT_NUM(x)				(((x) & 0x3fff) << 16)
+#define RKVDEC_ERR_PKT_NUM(x)				((x >> 16) & 0x3fff)
 
 #define RKVDEC_REG_H264_ERR_E				0x134
 #define RKVDEC_H264_ERR_EN_HIGHBITS(x)			((x) & 0x3fffffff)
-- 
2.38.1

[PATCH v2 3/4] media: rkvdec: Re-enable H.264 error detection

[Nicolas Dufresne]

This re-enable H.264 error detection, but using the other error mode.
In that mode, the decoder will skip over the error macro-block or
slices and complete the decoding. As a side effect, the error status
is not set in the interrupt status register, and instead errors are
detected per format. Using this mode workaround the issue that the
HW get stuck in error state, and allow reporting that some corruption
may be present in the buffer to userland.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
 drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 4fc167b42cf0c..dfe3e235f099a 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -1162,14 +1162,15 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
 
 	schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000));
 
-	writel(0, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
-	writel(0, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
+	writel(0xffffffff, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
+	writel(0xffffffff, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
 	writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND);
 	writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND);
 
 	/* Start decoding! */
 	writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E |
-	       RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E,
+	       RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E |
+	       RKVDEC_H264ORVP9_ERR_MODE,
 	       rkvdec->regs + RKVDEC_REG_INTERRUPT);
 
 	return 0;
@@ -1183,10 +1184,26 @@ static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
 	return 0;
 }
 
+static int rkvdec_h264_check_error_info(struct rkvdec_ctx *ctx)
+{
+	struct rkvdec_dev *rkvdec = ctx->dev;
+	int err;
+
+	err = readl(rkvdec->regs + RKVDEC_REG_H264_ERRINFO_NUM);
+	if (err & RKVDEC_STRMD_DECT_ERR_FLAG) {
+		pr_debug("Decoded picture have %i/%i slices with errors.\n",
+			 RKVDEC_ERR_PKT_NUM(err), RKVDEC_SLICEDEC_NUM(err));
+		return VB2_BUF_STATE_ERROR;
+	}
+
+	return VB2_BUF_STATE_DONE;
+}
+
 const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
 	.adjust_fmt = rkvdec_h264_adjust_fmt,
 	.start = rkvdec_h264_start,
 	.stop = rkvdec_h264_stop,
 	.run = rkvdec_h264_run,
 	.try_ctrl = rkvdec_h264_try_ctrl,
+	.check_error_info = rkvdec_h264_check_error_info,
 };
-- 
2.38.1

Re: [PATCH v2 3/4] media: rkvdec: Re-enable H.264 error detection

[Chen-Yu Tsai]

On Sat, Dec 24, 2022 at 3:39 AM Nicolas Dufresne
<nicolas.dufresne@collabora.com> wrote:
>
> This re-enable H.264 error detection, but using the other error mode.
> In that mode, the decoder will skip over the error macro-block or
> slices and complete the decoding. As a side effect, the error status
> is not set in the interrupt status register, and instead errors are
> detected per format. Using this mode workaround the issue that the
> HW get stuck in error state, and allow reporting that some corruption
> may be present in the buffer to userland.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
>  drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++++++++---
>  1 file changed, 20 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 4fc167b42cf0c..dfe3e235f099a 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -1162,14 +1162,15 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>
>         schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000));
>
> -       writel(0, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> -       writel(0, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
> +       writel(0xffffffff, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> +       writel(0xffffffff, rkvdec->regs + RKVDEC_REG_H264_ERR_E);

Use RKVDEC_H264_ERR_EN_HIGHBITS() here? Only lower 30 bits are valid.

>         writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND);
>         writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND);
>
>         /* Start decoding! */
>         writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E |
> -              RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E,
> +              RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E |
> +              RKVDEC_H264ORVP9_ERR_MODE,
>                rkvdec->regs + RKVDEC_REG_INTERRUPT);
>
>         return 0;
> @@ -1183,10 +1184,26 @@ static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
>         return 0;
>  }
>
> +static int rkvdec_h264_check_error_info(struct rkvdec_ctx *ctx)
> +{
> +       struct rkvdec_dev *rkvdec = ctx->dev;
> +       int err;

u32?

Otherwise,

Reviewed-by: Chen-Yu Tsai <wenst@chromium.org>

> +
> +       err = readl(rkvdec->regs + RKVDEC_REG_H264_ERRINFO_NUM);
> +       if (err & RKVDEC_STRMD_DECT_ERR_FLAG) {
> +               pr_debug("Decoded picture have %i/%i slices with errors.\n",
> +                        RKVDEC_ERR_PKT_NUM(err), RKVDEC_SLICEDEC_NUM(err));
> +               return VB2_BUF_STATE_ERROR;
> +       }
> +
> +       return VB2_BUF_STATE_DONE;
> +}
> +
>  const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
>         .adjust_fmt = rkvdec_h264_adjust_fmt,
>         .start = rkvdec_h264_start,
>         .stop = rkvdec_h264_stop,
>         .run = rkvdec_h264_run,
>         .try_ctrl = rkvdec_h264_try_ctrl,
> +       .check_error_info = rkvdec_h264_check_error_info,
>  };
> --
> 2.38.1
>

Re: [PATCH v2 3/4] media: rkvdec: Re-enable H.264 error detection

[Nicolas Dufresne]

Le lundi 26 décembre 2022 à 12:17 +0800, Chen-Yu Tsai a écrit :
> On Sat, Dec 24, 2022 at 3:39 AM Nicolas Dufresne
> <nicolas.dufresne@collabora.com> wrote:
> > 
> > This re-enable H.264 error detection, but using the other error mode.
> > In that mode, the decoder will skip over the error macro-block or
> > slices and complete the decoding. As a side effect, the error status
> > is not set in the interrupt status register, and instead errors are
> > detected per format. Using this mode workaround the issue that the
> > HW get stuck in error state, and allow reporting that some corruption
> > may be present in the buffer to userland.
> > 
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > ---
> >  drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++++++++---
> >  1 file changed, 20 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 4fc167b42cf0c..dfe3e235f099a 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -1162,14 +1162,15 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
> > 
> >         schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000));
> > 
> > -       writel(0, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> > -       writel(0, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
> > +       writel(0xffffffff, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> > +       writel(0xffffffff, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
> 
> Use RKVDEC_H264_ERR_EN_HIGHBITS() here? Only lower 30 bits are valid.

I was simply reverting back to the value it was before, I can try that too, you
are likely right.

> 
> >         writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND);
> >         writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND);
> > 
> >         /* Start decoding! */
> >         writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E |
> > -              RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E,
> > +              RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E |
> > +              RKVDEC_H264ORVP9_ERR_MODE,
> >                rkvdec->regs + RKVDEC_REG_INTERRUPT);
> > 
> >         return 0;
> > @@ -1183,10 +1184,26 @@ static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> >         return 0;
> >  }
> > 
> > +static int rkvdec_h264_check_error_info(struct rkvdec_ctx *ctx)
> > +{
> > +       struct rkvdec_dev *rkvdec = ctx->dev;
> > +       int err;
> 
> u32?

Will do.

> 
> Otherwise,
> 
> Reviewed-by: Chen-Yu Tsai <wenst@chromium.org>
> 
> > +
> > +       err = readl(rkvdec->regs + RKVDEC_REG_H264_ERRINFO_NUM);
> > +       if (err & RKVDEC_STRMD_DECT_ERR_FLAG) {
> > +               pr_debug("Decoded picture have %i/%i slices with errors.\n",
> > +                        RKVDEC_ERR_PKT_NUM(err), RKVDEC_SLICEDEC_NUM(err));
> > +               return VB2_BUF_STATE_ERROR;
> > +       }
> > +
> > +       return VB2_BUF_STATE_DONE;
> > +}
> > +
> >  const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
> >         .adjust_fmt = rkvdec_h264_adjust_fmt,
> >         .start = rkvdec_h264_start,
> >         .stop = rkvdec_h264_stop,
> >         .run = rkvdec_h264_run,
> >         .try_ctrl = rkvdec_h264_try_ctrl,
> > +       .check_error_info = rkvdec_h264_check_error_info,
> >  };
> > --
> > 2.38.1
> > 

Re: [PATCH v2 3/4] media: rkvdec: Re-enable H.264 error detection

[Ezequiel Garcia]

Hi Nicolas,

I'm still unsure about this patchset.
It sounds like a good approach and a nice
improvement, but I want to make sure I think through it.

Meanwhile, a small comment...

On Fri, Dec 23, 2022 at 02:38:05PM -0500, Nicolas Dufresne wrote:
> This re-enable H.264 error detection, but using the other error mode.
> In that mode, the decoder will skip over the error macro-block or
> slices and complete the decoding. As a side effect, the error status
> is not set in the interrupt status register, and instead errors are
> detected per format. Using this mode workaround the issue that the
> HW get stuck in error state, and allow reporting that some corruption
> may be present in the buffer to userland.
> 
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
>  drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++++++++---
>  1 file changed, 20 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 4fc167b42cf0c..dfe3e235f099a 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -1162,14 +1162,15 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>  
>  	schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000));
>  
> -	writel(0, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> -	writel(0, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
> +	writel(0xffffffff, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> +	writel(0xffffffff, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
>  	writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND);
>  	writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND);
>  
>  	/* Start decoding! */
>  	writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E |
> -	       RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E,
> +	       RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E |
> +	       RKVDEC_H264ORVP9_ERR_MODE,
>  	       rkvdec->regs + RKVDEC_REG_INTERRUPT);
>  
>  	return 0;
> @@ -1183,10 +1184,26 @@ static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
>  	return 0;
>  }
>  
> +static int rkvdec_h264_check_error_info(struct rkvdec_ctx *ctx)
> +{
> +	struct rkvdec_dev *rkvdec = ctx->dev;
> +	int err;
> +
> +	err = readl(rkvdec->regs + RKVDEC_REG_H264_ERRINFO_NUM);
> +	if (err & RKVDEC_STRMD_DECT_ERR_FLAG) {
> +		pr_debug("Decoded picture have %i/%i slices with errors.\n",

... still uses pr_debug. I would change it so it uses v4l2_dbg,
and can be controlled using the same debug parameter
as you use in patch 4/4.

> +			 RKVDEC_ERR_PKT_NUM(err), RKVDEC_SLICEDEC_NUM(err));
> +		return VB2_BUF_STATE_ERROR;
> +	}
> +
> +	return VB2_BUF_STATE_DONE;
> +}
> +
>  const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
>  	.adjust_fmt = rkvdec_h264_adjust_fmt,
>  	.start = rkvdec_h264_start,
>  	.stop = rkvdec_h264_stop,
>  	.run = rkvdec_h264_run,
>  	.try_ctrl = rkvdec_h264_try_ctrl,
> +	.check_error_info = rkvdec_h264_check_error_info,
>  };
> -- 
> 2.38.1
> 

Re: [PATCH v2 3/4] media: rkvdec: Re-enable H.264 error detection

[Nicolas Dufresne]

Le lundi 26 décembre 2022 à 18:33 -0300, Ezequiel Garcia a écrit :
> Hi Nicolas,
> 
> I'm still unsure about this patchset.
> It sounds like a good approach and a nice
> improvement, but I want to make sure I think through it.
> 
> Meanwhile, a small comment...
> 
> On Fri, Dec 23, 2022 at 02:38:05PM -0500, Nicolas Dufresne wrote:
> > This re-enable H.264 error detection, but using the other error mode.
> > In that mode, the decoder will skip over the error macro-block or
> > slices and complete the decoding. As a side effect, the error status
> > is not set in the interrupt status register, and instead errors are
> > detected per format. Using this mode workaround the issue that the
> > HW get stuck in error state, and allow reporting that some corruption
> > may be present in the buffer to userland.
> > 
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > ---
> >  drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++++++++---
> >  1 file changed, 20 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 4fc167b42cf0c..dfe3e235f099a 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -1162,14 +1162,15 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
> >  
> >  	schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000));
> >  
> > -	writel(0, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> > -	writel(0, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
> > +	writel(0xffffffff, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN);
> > +	writel(0xffffffff, rkvdec->regs + RKVDEC_REG_H264_ERR_E);
> >  	writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND);
> >  	writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND);
> >  
> >  	/* Start decoding! */
> >  	writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E |
> > -	       RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E,
> > +	       RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E |
> > +	       RKVDEC_H264ORVP9_ERR_MODE,
> >  	       rkvdec->regs + RKVDEC_REG_INTERRUPT);
> >  
> >  	return 0;
> > @@ -1183,10 +1184,26 @@ static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> >  	return 0;
> >  }
> >  
> > +static int rkvdec_h264_check_error_info(struct rkvdec_ctx *ctx)
> > +{
> > +	struct rkvdec_dev *rkvdec = ctx->dev;
> > +	int err;
> > +
> > +	err = readl(rkvdec->regs + RKVDEC_REG_H264_ERRINFO_NUM);
> > +	if (err & RKVDEC_STRMD_DECT_ERR_FLAG) {
> > +		pr_debug("Decoded picture have %i/%i slices with errors.\n",
> 
> ... still uses pr_debug. I would change it so it uses v4l2_dbg,
> and can be controlled using the same debug parameter
> as you use in patch 4/4.

Good catch, I missed that one, will fix.

> 
> > +			 RKVDEC_ERR_PKT_NUM(err), RKVDEC_SLICEDEC_NUM(err));
> > +		return VB2_BUF_STATE_ERROR;
> > +	}
> > +
> > +	return VB2_BUF_STATE_DONE;
> > +}
> > +
> >  const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
> >  	.adjust_fmt = rkvdec_h264_adjust_fmt,
> >  	.start = rkvdec_h264_start,
> >  	.stop = rkvdec_h264_stop,
> >  	.run = rkvdec_h264_run,
> >  	.try_ctrl = rkvdec_h264_try_ctrl,
> > +	.check_error_info = rkvdec_h264_check_error_info,
> >  };
> > -- 
> > 2.38.1
> > 

[PATCH v2 4/4] rkvdec: Improve error handling

[Nicolas Dufresne]

There are two ways decoding errors can occure. In one case, the ready
status is not set and nothing has been written into the destination,
while in the other case, the buffer is written but may contain a
certain amount of errors. In order to differentiate these, we set
the payload for the first case to 0.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
 drivers/staging/media/rkvdec/rkvdec.c | 31 +++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 7e76f8b728854..11e2bbb20aea1 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,6 +27,9 @@
 #include "rkvdec.h"
 #include "rkvdec-regs.h"
 
+static int debug;
+module_param(debug, int, 0644);
+
 static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
@@ -954,14 +957,34 @@ static irqreturn_t rkvdec_irq_handler(int irq, void *priv)
 	enum vb2_buffer_state state;
 	u32 status;
 
+	ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
 	status = readl(rkvdec->regs + RKVDEC_REG_INTERRUPT);
-	state = (status & RKVDEC_RDY_STA) ?
-		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+	if (!(status & RKVDEC_RDY_STA)) {
+		struct vb2_v4l2_buffer *dst_buf = NULL;
+
+		if (status & RKVDEC_TIMEOUT_STA)
+			v4l2_dbg(debug, 1, &rkvdec->v4l2_dev,
+				 "Decoder stopped due to an internal timeout.");
+		else
+			v4l2_dbg(debug, 1, &rkvdec->v4l2_dev,
+				 "Decoder stopped due to an internal error.");
+
+		/*
+		 * When this happens, the buffer is left unmodified. As it
+		 * contains no meaningful data we mark is as empty.
+		 */
+		dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
+		state = VB2_BUF_STATE_ERROR;
+	} else {
+		state = VB2_BUF_STATE_DONE;
+	}
 
 	writel(0, rkvdec->regs + RKVDEC_REG_INTERRUPT);
-	ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
 
-	if (ctx->coded_fmt_desc->ops->check_error_info)
+	if (ctx->coded_fmt_desc->ops->check_error_info &&
+	    state == VB2_BUF_STATE_DONE)
 		state = ctx->coded_fmt_desc->ops->check_error_info(ctx);
 
 	if (cancel_delayed_work(&rkvdec->watchdog_work))
-- 
2.38.1

Re: [PATCH v2 4/4] rkvdec: Improve error handling

[Ezequiel Garcia]

Hi Nicolas,

Thanks a lot for the patchset. I have just some style feedback.

On Fri, Dec 23, 2022 at 02:38:06PM -0500, Nicolas Dufresne wrote:
> There are two ways decoding errors can occure. In one case, the ready
> status is not set and nothing has been written into the destination,
> while in the other case, the buffer is written but may contain a
> certain amount of errors. In order to differentiate these, we set
> the payload for the first case to 0.
> 
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
>  drivers/staging/media/rkvdec/rkvdec.c | 31 +++++++++++++++++++++++----
>  1 file changed, 27 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 7e76f8b728854..11e2bbb20aea1 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -27,6 +27,9 @@
>  #include "rkvdec.h"
>  #include "rkvdec-regs.h"
>  
> +static int debug;
> +module_param(debug, int, 0644);
> +
>  static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
>  {
>  	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
> @@ -954,14 +957,34 @@ static irqreturn_t rkvdec_irq_handler(int irq, void *priv)
>  	enum vb2_buffer_state state;
>  	u32 status;
>  
> +	ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
>  	status = readl(rkvdec->regs + RKVDEC_REG_INTERRUPT);

Maybe group the I/O together, i.e. the writel would
be right after this readl:

    writel(0, rkvdec->regs + RKVDEC_REG_INTERRUPT);

> -	state = (status & RKVDEC_RDY_STA) ?
> -		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
> +
> +	if (!(status & RKVDEC_RDY_STA)) {
> +		struct vb2_v4l2_buffer *dst_buf = NULL;
> +
> +		if (status & RKVDEC_TIMEOUT_STA)
> +			v4l2_dbg(debug, 1, &rkvdec->v4l2_dev,
> +				 "Decoder stopped due to an internal timeout.");
> +		else
> +			v4l2_dbg(debug, 1, &rkvdec->v4l2_dev,
> +				 "Decoder stopped due to an internal error.");

Unless you really want to ensure this string is greppable,
you can do something like:

v4l2_dbg(debug, 1, &rkvdec->v4l2_dev, "Decoder stopped due to an internal %s.", .... ? "error" : "timeout");

> +
> +		/*
> +		 * When this happens, the buffer is left unmodified. As it
> +		 * contains no meaningful data we mark is as empty.
> +		 */
> +		dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
> +		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);

Perhaps we can avoid this vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
if we instead set the payload in rkvdec_job_finish_no_pm().
It would change the behavior, as we would be setting payload
only when state is _DONE, so maybe that's not what you want.

> +		state = VB2_BUF_STATE_ERROR;
> +	} else {
> +		state = VB2_BUF_STATE_DONE;
> +	}
>  
>  	writel(0, rkvdec->regs + RKVDEC_REG_INTERRUPT);
> -	ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
>  
> -	if (ctx->coded_fmt_desc->ops->check_error_info)
> +	if (ctx->coded_fmt_desc->ops->check_error_info &&
> +	    state == VB2_BUF_STATE_DONE)
>  		state = ctx->coded_fmt_desc->ops->check_error_info(ctx);
>  

How about this:

static irqreturn_t rkvdec_irq_handler(int irq, void *priv)
{
    struct rkvdec_dev *rkvdec = priv;
    struct rkvdec_ctx *ctx;
    enum vb2_buffer_state state = VB2_BUF_STATE_DONE;
    u32 status;

    ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
    status = readl(rkvdec->regs + RKVDEC_REG_INTERRUPT);
    writel(0, rkvdec->regs + RKVDEC_REG_INTERRUPT);

    if (!(status & RKVDEC_RDY_STA)) {
        ...
        state = VB2_BUF_STATE_ERROR;
    } else {
        if (ctx->coded_fmt_desc->ops->check_error_info &&
            ctx->coded_fmt_desc->ops->check_error_info(ctx))
            state = VB2_BUF_STATE_ERROR;
    }

    if (cancel_delayed_work(&rkvdec->watchdog_work))
        rkvdec_job_finish(ctx, state);

    return IRQ_HANDLED;
}

So it's clear which paths lead to VB2_BUF_STATE_ERROR.

Thanks!
Ezequiel

>  	if (cancel_delayed_work(&rkvdec->watchdog_work))
> -- 
> 2.38.1
> 

Re: [PATCH v2 0/4] media: rkvdec: Fix H.264 error resilience

[Nicolas Dufresne]

fyi, this cover letter was v2.

Le vendredi 23 décembre 2022 à 14:38 -0500, Nicolas Dufresne a écrit :
> This patch serie changes the decoding mode from "exit on error"
> to "keep decoding". Using this mode and re-enabling error detection
> allow getting error resilience without loosing the ability to report
> errors to userland. This have showed great results, but might be a
> little more risky since this is not the mode that the reference code
> uses and the documentation is very brief. With this in place,
> userspace can chose to skip or display corrupted frames depending
> on its application requirement. Previsouly, applicaiton would have
> had no choice but to present the corrupted frames.
> 
> Changes since V1:
> 	- Removed merged patch
> 	- Changed usage of pr_debug into v4l2_dbg
> 	- Fix typos in commit messages and comments
> 
> Nicolas Dufresne (5):
>   media: rkvdec: Disable H.264 error detection
>   media: rkvdec: Add an ops to check for decode errors
>   media: rkvdec: Fix RKVDEC_ERR_PKT_NUM macro
>   media: rkvdec: Re-enable H.264 error detection
>   rkvdec: Improve error handling
> 
>  drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++--
>  drivers/staging/media/rkvdec/rkvdec-regs.h |  2 +-
>  drivers/staging/media/rkvdec/rkvdec.c      | 34 ++++++++++++++++++----
>  drivers/staging/media/rkvdec/rkvdec.h      |  2 ++
>  4 files changed, 51 insertions(+), 10 deletions(-)
> 

Re: [PATCH v1 0/4] media: rkvdec: Fix H.264 error resilience

[Chen-Yu Tsai]

On Sat, Dec 24, 2022 at 3:38 AM Nicolas Dufresne
<nicolas.dufresne@collabora.com> wrote:
>
> This patch serie changes the decoding mode from "exit on error"
> to "keep decoding". Using this mode and re-enabling error detection
> allow getting error resilience without loosing the ability to report
> errors to userland. This have showed great results, but might be a
> little more risky since this is not the mode that the reference code
> uses and the documentation is very brief. With this in place,
> userspace can chose to skip or display corrupted frames depending
> on its application requirement. Previsouly, applicaiton would have
> had no choice but to present the corrupted frames.
>
> Changes since V1:
>         - Removed merged patch
>         - Changed usage of pr_debug into v4l2_dbg
>         - Fix typos in commit messages and comments
>
> Nicolas Dufresne (5):
>   media: rkvdec: Disable H.264 error detection
>   media: rkvdec: Add an ops to check for decode errors
>   media: rkvdec: Fix RKVDEC_ERR_PKT_NUM macro
>   media: rkvdec: Re-enable H.264 error detection
>   rkvdec: Improve error handling

Apart from the minor comments in patch 3, the series is

Reviewed-by: Chen-Yu Tsai <wenst@chromium.org>

Do we need to add the check_error_info op for VP9?

>  drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++--
>  drivers/staging/media/rkvdec/rkvdec-regs.h |  2 +-
>  drivers/staging/media/rkvdec/rkvdec.c      | 34 ++++++++++++++++++----
>  drivers/staging/media/rkvdec/rkvdec.h      |  2 ++
>  4 files changed, 51 insertions(+), 10 deletions(-)
>
> --
> 2.38.1
>

Re: [PATCH v1 0/4] media: rkvdec: Fix H.264 error resilience

[Nicolas Dufresne]

Le lundi 26 décembre 2022 à 12:20 +0800, Chen-Yu Tsai a écrit :
> On Sat, Dec 24, 2022 at 3:38 AM Nicolas Dufresne
> <nicolas.dufresne@collabora.com> wrote:
> > 
> > This patch serie changes the decoding mode from "exit on error"
> > to "keep decoding". Using this mode and re-enabling error detection
> > allow getting error resilience without loosing the ability to report
> > errors to userland. This have showed great results, but might be a
> > little more risky since this is not the mode that the reference code
> > uses and the documentation is very brief. With this in place,
> > userspace can chose to skip or display corrupted frames depending
> > on its application requirement. Previsouly, applicaiton would have
> > had no choice but to present the corrupted frames.
> > 
> > Changes since V1:
> >         - Removed merged patch
> >         - Changed usage of pr_debug into v4l2_dbg
> >         - Fix typos in commit messages and comments
> > 
> > Nicolas Dufresne (5):
> >   media: rkvdec: Disable H.264 error detection
> >   media: rkvdec: Add an ops to check for decode errors
> >   media: rkvdec: Fix RKVDEC_ERR_PKT_NUM macro
> >   media: rkvdec: Re-enable H.264 error detection
> >   rkvdec: Improve error handling
> 
> Apart from the minor comments in patch 3, the series is
> 
> Reviewed-by: Chen-Yu Tsai <wenst@chromium.org>
> 
> Do we need to add the check_error_info op for VP9?

In general, I try to avoid changes I cannot test. If you happen to have VP9 with
errors let me know. Looking quickly at the TRM, some part seems shared between
H.264 and VP9 indeed. Though, the TRM discourage from enabling the "keep
decoding on error" mode we use here.

> 
> >  drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++--
> >  drivers/staging/media/rkvdec/rkvdec-regs.h |  2 +-
> >  drivers/staging/media/rkvdec/rkvdec.c      | 34 ++++++++++++++++++----
> >  drivers/staging/media/rkvdec/rkvdec.h      |  2 ++
> >  4 files changed, 51 insertions(+), 10 deletions(-)
> > 
> > --
> > 2.38.1
> >