[PATCH v6 00/16] Add MPEG-2 decoding to Rockchip VPU

[PATCH v6 00/16] Add MPEG-2 decoding to Rockchip VPU

[Ezequiel Garcia]

This series introduces the decoding infrastructure that will be
used to add support for other codecs such as VP8, VP9 and H.264.

The driver is exposes two video device nodes, the encoder being
/dev/video0, and decoder /dev/video1. Both devices are associated
to a single memory-to-memory queue, and same media device /dev/media0.

This work has been possible thanks to Jonas Karlman, who did
the initial MPEG-2 decoding work and also got mpv+ffmpeg working using
the Request API.

Patchset applies cleanly on top of:

git://linuxtv.org/hverkuil/media_tree.git tags/br-v5.3c

For those wanting to test, please find a branch here:

http://git.infradead.org/users/ezequielg/linux/shortlog/refs/heads/vpu-mpeg2-v6

Future plans
------------

Once this series is merged the plan is to rename the driver to Hantro
VPU, and add support to i.MX8, and then add VP8 and H264 decoding
support.

v6:
 * Change fill_fmt() helpers to accept only user-defined sizeimage,
   but not bytesperline.
 * Add RK3328 support, contributed by Jonas Karlman.

v5:
 * Add helper to clamp height/width based on frmsize
 * Modify fill_fmt() helpers to keep user-defined sizeimage/bytesperline
   when they're big enough
 * Fixed a bunch of problems reported by Hans (see per-patch changelog
   for more details)

v4:
 * Fix issues in the probe/remove path preventing the driver from being
   unloaded/reloaded (Jonas)
 * Rework the media controller registration code to simplify the
   error/remove path (Boris)
 * Split the MPEG2 patches (one adding the generic code and one patch
   per platform)

v3:
  * Share more code between the encoder/decoder logic (VB2/V4L2
    interface implementation)
  * Add a patch to support MPEG2 decoding on RK3288 (Jonas)
  * Several fixes/improvements all over the place (see changelog
    in each patch for more details)

v2:
  * Fixed some minor issues brought up by v4l2-compliance.
  * Fixed bytesused wrongly assigned 0 on the MPEG-2 decoder.
  * Addressed comments from Hans and Tomasz on the pixel format
    helpers.

Boris Brezillon (8):
  media: v4l2-common: Fix v4l2_fill_pixfmt[_mp]() prototypes
  media: v4l2-common: Add an helper to apply frmsize constraints
  media: v4l2-common: Support custom imagesize in fill_pixfmt()
  rockchip/vpu: Use v4l2_apply_frmsize_constraints() where appropriate
  rockchip/vpu: Rename rockchip_vpu_common.h into rockchip_vpu_v4l2.h
  rockchip/vpu: Move encoder logic to a common place
  rockchip/vpu: Provide a helper to reset both src and dst formats
  rockchip/vpu: Prepare things to support decoders

Ezequiel Garcia (4):
  rockchip/vpu: Open-code media controller register
  rockchip/vpu: Support the Request API
  rockchip/vpu: Add decoder boilerplate
  rockchip/vpu: Add support for non-standard controls

Jonas Karlman (4):
  rockchip/vpu: Add infra to support MPEG-2 decoding
  rockchip/vpu: Add MPEG2 decoding support to RK3399
  rockchip/vpu: Add support for MPEG-2 decoding on RK3288
  rockchip/vpu: Add support for MPEG-2 decoding on RK3328

 drivers/media/v4l2-core/v4l2-common.c         |  90 ++-
 drivers/staging/media/rockchip/vpu/Kconfig    |   1 +
 drivers/staging/media/rockchip/vpu/Makefile   |   7 +-
 .../media/rockchip/vpu/rk3288_vpu_hw.c        |  59 +-
 .../rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c     |   2 +-
 .../rockchip/vpu/rk3288_vpu_hw_mpeg2_dec.c    | 261 +++++++
 .../media/rockchip/vpu/rk3288_vpu_regs.h      |   1 +
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  71 +-
 .../rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c     |   8 +-
 .../rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c    | 267 +++++++
 .../staging/media/rockchip/vpu/rockchip_vpu.h | 110 ++-
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 466 ++++++++++--
 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 571 ---------------
 .../media/rockchip/vpu/rockchip_vpu_hw.h      |  17 +
 .../media/rockchip/vpu/rockchip_vpu_mpeg2.c   |  61 ++
 .../media/rockchip/vpu/rockchip_vpu_v4l2.c    | 692 ++++++++++++++++++
 ...kchip_vpu_common.h => rockchip_vpu_v4l2.h} |  15 +-
 include/media/v4l2-common.h                   |  10 +-
 18 files changed, 2026 insertions(+), 683 deletions(-)
 create mode 100644 drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_mpeg2_dec.c
 create mode 100644 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c
 delete mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
 create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c
 create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
 rename drivers/staging/media/rockchip/vpu/{rockchip_vpu_common.h => rockchip_vpu_v4l2.h} (50%)

-- 
2.20.1

[PATCH v6 01/16] media: v4l2-common: Fix v4l2_fill_pixfmt[_mp]() prototypes

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

Width/height and 4CC formats are expressed using u32 types everywhere,
let's fix the v4l2_fill_pixfmt[_mp]() prototypes to do the same.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v5:
* None

Changes from v4:
* New patch

 drivers/media/v4l2-core/v4l2-common.c | 5 +++--
 include/media/v4l2-common.h           | 8 ++++----
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index b5778b2ffa27..9417a883ebe3 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -542,7 +542,7 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
 }
 
 int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
-			 int pixelformat, int width, int height)
+			u32 pixelformat, u32 width, u32 height)
 {
 	const struct v4l2_format_info *info;
 	struct v4l2_plane_pix_format *plane;
@@ -596,7 +596,8 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
 }
 EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt_mp);
 
-int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat, int width, int height)
+int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
+		     u32 width, u32 height)
 {
 	const struct v4l2_format_info *info;
 	int i;
diff --git a/include/media/v4l2-common.h b/include/media/v4l2-common.h
index 0a41bbecf3d3..3226bc8107cc 100644
--- a/include/media/v4l2-common.h
+++ b/include/media/v4l2-common.h
@@ -420,9 +420,9 @@ struct v4l2_format_info {
 
 const struct v4l2_format_info *v4l2_format_info(u32 format);
 
-int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat,
-		     int width, int height);
-int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, int pixelformat,
-			int width, int height);
+int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
+		     u32 width, u32 height);
+int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, u32 pixelformat,
+			u32 width, u32 height);
 
 #endif /* V4L2_COMMON_H_ */
-- 
2.20.1

[PATCH v6 02/16] media: v4l2-common: Add an helper to apply frmsize constraints

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

The rockchip VPU driver is open-coding this logic which seems pretty
generic. Let's provide an helper to apply the min/max and alignment
constraints on width/height.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v5:
* None

Changes from v4:
* New patch

 drivers/media/v4l2-core/v4l2-common.c | 27 +++++++++++++++++++++++++++
 include/media/v4l2-common.h           |  2 ++
 2 files changed, 29 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 9417a883ebe3..b2d1e55d9561 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -331,6 +331,16 @@ static unsigned int clamp_align(unsigned int x, unsigned int min,
 	return x;
 }
 
+static unsigned int clamp_roundup(unsigned int x, unsigned int min,
+				   unsigned int max, unsigned int alignment)
+{
+	x = clamp(x, min, max);
+	if (alignment)
+		x = round_up(x, alignment);
+
+	return x;
+}
+
 void v4l_bound_align_image(u32 *w, unsigned int wmin, unsigned int wmax,
 			   unsigned int walign,
 			   u32 *h, unsigned int hmin, unsigned int hmax,
@@ -541,6 +551,23 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
 	return info->block_h[plane];
 }
 
+void v4l2_apply_frmsize_constraints(u32 *width, u32 *height,
+				    const struct v4l2_frmsize_stepwise *frmsize)
+{
+	if (!frmsize)
+		return;
+
+	/*
+	 * Clamp width/height to meet min/max constraints and round it up to
+	 * macroblock alignment.
+	 */
+	*width = clamp_roundup(*width, frmsize->min_width, frmsize->max_width,
+			       frmsize->step_width);
+        *height = clamp_roundup(*height, frmsize->min_height, frmsize->max_height,
+				frmsize->step_height);
+}
+EXPORT_SYMBOL_GPL(v4l2_apply_frmsize_constraints);
+
 int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
 			u32 pixelformat, u32 width, u32 height)
 {
diff --git a/include/media/v4l2-common.h b/include/media/v4l2-common.h
index 3226bc8107cc..e826b154bc35 100644
--- a/include/media/v4l2-common.h
+++ b/include/media/v4l2-common.h
@@ -420,6 +420,8 @@ struct v4l2_format_info {
 
 const struct v4l2_format_info *v4l2_format_info(u32 format);
 
+void v4l2_apply_frmsize_constraints(u32 *width, u32 *height,
+				    const struct v4l2_frmsize_stepwise *frmsize);
 int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
 		     u32 width, u32 height);
 int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, u32 pixelformat,
-- 
2.20.1

[PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

Users can define custom sizeimage as long as they're big enough to
store the amount of pixels required for a specific width/height under a
specific format. Avoid overriding those fields in this case.

We could possibly do the same for bytesperline, but it gets tricky when
dealing with !MPLANE definitions, so this case is omitted for now and
->bytesperline is always overwritten with the value calculated in
fill_pixfmt().

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v5:
* Overwrite bytesperline with the value calculated in fill_pixfmt()

Changes from v4:
* New patch

 drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
 1 file changed, 43 insertions(+), 15 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index b2d1e55d9561..fd286f6e17d7 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
 	pixfmt->num_planes = info->mem_planes;
 
 	if (info->mem_planes == 1) {
+		u32 sizeimage = 0;
+
 		plane = &pixfmt->plane_fmt[0];
-		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
-		plane->sizeimage = 0;
 
 		for (i = 0; i < info->comp_planes; i++) {
 			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
@@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
 			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
 			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
 
-			plane->sizeimage += info->bpp[i] *
-				DIV_ROUND_UP(aligned_width, hdiv) *
-				DIV_ROUND_UP(aligned_height, vdiv);
+			sizeimage += info->bpp[i] *
+				     DIV_ROUND_UP(aligned_width, hdiv) *
+				     DIV_ROUND_UP(aligned_height, vdiv);
 		}
+
+		/* Custom bytesperline value is not supported yet. */
+		plane->bytesperline = ALIGN(width,
+					    v4l2_format_block_width(info, 0)) *
+				      info->bpp[0];
+
+		/*
+		 * The user might have specified a custom sizeimage, only
+		 * override it if it's not big enough.
+		 */
+		plane->sizeimage = max(sizeimage, plane->sizeimage);
 	} else {
 		for (i = 0; i < info->comp_planes; i++) {
 			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
@@ -613,10 +624,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
 			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
 
 			plane = &pixfmt->plane_fmt[i];
-			plane->bytesperline =
-				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv);
-			plane->sizeimage =
-				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
+
+			/* Custom bytesperline value is not supported yet. */
+			plane->bytesperline = info->bpp[i] *
+					      DIV_ROUND_UP(aligned_width, hdiv);
+
+			/*
+			 * The user might have specified a custom sizeimage,
+			 * only override it if it's not big enough.
+			 */
+			plane->sizeimage = max_t(u32,
+						 plane->bytesperline *
+						 DIV_ROUND_UP(aligned_height, vdiv),
+						 plane->sizeimage);
 		}
 	}
 	return 0;
@@ -627,6 +647,7 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
 		     u32 width, u32 height)
 {
 	const struct v4l2_format_info *info;
+	u32 sizeimage = 0;
 	int i;
 
 	info = v4l2_format_info(pixelformat);
@@ -640,8 +661,6 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
 	pixfmt->width = width;
 	pixfmt->height = height;
 	pixfmt->pixelformat = pixelformat;
-	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
-	pixfmt->sizeimage = 0;
 
 	for (i = 0; i < info->comp_planes; i++) {
 		unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
@@ -651,11 +670,20 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
 
 		aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
 		aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
-		pixfmt->sizeimage += info->bpp[i] *
-			DIV_ROUND_UP(aligned_width, hdiv) *
-			DIV_ROUND_UP(aligned_height, vdiv);
+		sizeimage += info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) *
+			     DIV_ROUND_UP(aligned_height, vdiv);
 	}
+
+	/* Custom bytesperline value is not supported yet. */
+	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) *
+			       info->bpp[0];
+
+	/*
+	 * The user might have specified its own sizeimage value, only override
+	 * it if it's not big enough.
+	 */
+	pixfmt->sizeimage = max(sizeimage, pixfmt->sizeimage);
+
 	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
-- 
2.20.1

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Mauro Carvalho Chehab]

Em Tue, 28 May 2019 14:02:19 -0300
Ezequiel Garcia <ezequiel@collabora.com> escreveu:

> From: Boris Brezillon <boris.brezillon@collabora.com>
> 
> Users can define custom sizeimage as long as they're big enough to
> store the amount of pixels required for a specific width/height under a
> specific format. Avoid overriding those fields in this case.
> 
> We could possibly do the same for bytesperline, but it gets tricky when
> dealing with !MPLANE definitions, so this case is omitted for now and
> ->bytesperline is always overwritten with the value calculated in
> fill_pixfmt().
> 
> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> ---
> Changes from v5:
> * Overwrite bytesperline with the value calculated in fill_pixfmt()
> 
> Changes from v4:
> * New patch
> 
>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
>  1 file changed, 43 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> index b2d1e55d9561..fd286f6e17d7 100644
> --- a/drivers/media/v4l2-core/v4l2-common.c
> +++ b/drivers/media/v4l2-core/v4l2-common.c
> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>  	pixfmt->num_planes = info->mem_planes;
>  
>  	if (info->mem_planes == 1) {
> +		u32 sizeimage = 0;
> +
>  		plane = &pixfmt->plane_fmt[0];
> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> -		plane->sizeimage = 0;
>  
>  		for (i = 0; i < info->comp_planes; i++) {
>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
>  
> -			plane->sizeimage += info->bpp[i] *
> -				DIV_ROUND_UP(aligned_width, hdiv) *
> -				DIV_ROUND_UP(aligned_height, vdiv);
> +			sizeimage += info->bpp[i] *
> +				     DIV_ROUND_UP(aligned_width, hdiv) *
> +				     DIV_ROUND_UP(aligned_height, vdiv);
>  		}
> +
> +		/* Custom bytesperline value is not supported yet. */
> +		plane->bytesperline = ALIGN(width,
> +					    v4l2_format_block_width(info, 0)) *
> +				      info->bpp[0];
> +
> +		/*
> +		 * The user might have specified a custom sizeimage, only
> +		 * override it if it's not big enough.
> +		 */
> +		plane->sizeimage = max(sizeimage, plane->sizeimage);

No upper limit? That doesn't sound a good idea to me, specially since some
(broken) app might not be memset the format to zero before filling the ioctl
structure.

Perhaps we could do something like:

		sizeimage = min (sizeimage, 2 * plane->sizeimage)

or something similar that would be reasonable.

>  	} else {
>  		for (i = 0; i < info->comp_planes; i++) {
>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> @@ -613,10 +624,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
>  
>  			plane = &pixfmt->plane_fmt[i];
> -			plane->bytesperline =
> -				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv);
> -			plane->sizeimage =
> -				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);

> +
> +			/* Custom bytesperline value is not supported yet. */

Supporting custom bytesperline seems too risky of breaking apps. 
So, I would drop this comment.


> +			plane->bytesperline = info->bpp[i] *
> +					      DIV_ROUND_UP(aligned_width, hdiv);

> +
> +			/*
> +			 * The user might have specified a custom sizeimage,
> +			 * only override it if it's not big enough.
> +			 */
> +			plane->sizeimage = max_t(u32,
> +						 plane->bytesperline *
> +						 DIV_ROUND_UP(aligned_height, vdiv),
> +						 plane->sizeimage);
>  		}
>  	}
>  	return 0;
> @@ -627,6 +647,7 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
>  		     u32 width, u32 height)
>  {
>  	const struct v4l2_format_info *info;
> +	u32 sizeimage = 0;
>  	int i;
>  
>  	info = v4l2_format_info(pixelformat);
> @@ -640,8 +661,6 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
>  	pixfmt->width = width;
>  	pixfmt->height = height;
>  	pixfmt->pixelformat = pixelformat;
> -	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> -	pixfmt->sizeimage = 0;
>  
>  	for (i = 0; i < info->comp_planes; i++) {
>  		unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> @@ -651,11 +670,20 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
>  
>  		aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
>  		aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> -
> -		pixfmt->sizeimage += info->bpp[i] *
> -			DIV_ROUND_UP(aligned_width, hdiv) *
> -			DIV_ROUND_UP(aligned_height, vdiv);
> +		sizeimage += info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) *
> +			     DIV_ROUND_UP(aligned_height, vdiv);
>  	}
> +
> +	/* Custom bytesperline value is not supported yet. */
> +	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) *
> +			       info->bpp[0];
> +
> +	/*
> +	 * The user might have specified its own sizeimage value, only override
> +	 * it if it's not big enough.
> +	 */
> +	pixfmt->sizeimage = max(sizeimage, pixfmt->sizeimage);
> +

Same comment applies here: We need to sanitize it from too big sizeimages.

>  	return 0;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);



Thanks,
Mauro

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Hans Verkuil]

On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:
> Em Tue, 28 May 2019 14:02:19 -0300
> Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> 
>> From: Boris Brezillon <boris.brezillon@collabora.com>
>>
>> Users can define custom sizeimage as long as they're big enough to
>> store the amount of pixels required for a specific width/height under a
>> specific format. Avoid overriding those fields in this case.
>>
>> We could possibly do the same for bytesperline, but it gets tricky when
>> dealing with !MPLANE definitions, so this case is omitted for now and
>> ->bytesperline is always overwritten with the value calculated in
>> fill_pixfmt().
>>
>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
>> ---
>> Changes from v5:
>> * Overwrite bytesperline with the value calculated in fill_pixfmt()
>>
>> Changes from v4:
>> * New patch
>>
>>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
>>  1 file changed, 43 insertions(+), 15 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
>> index b2d1e55d9561..fd286f6e17d7 100644
>> --- a/drivers/media/v4l2-core/v4l2-common.c
>> +++ b/drivers/media/v4l2-core/v4l2-common.c
>> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>>  	pixfmt->num_planes = info->mem_planes;
>>  
>>  	if (info->mem_planes == 1) {
>> +		u32 sizeimage = 0;
>> +
>>  		plane = &pixfmt->plane_fmt[0];
>> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
>> -		plane->sizeimage = 0;
>>  
>>  		for (i = 0; i < info->comp_planes; i++) {
>>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
>> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
>>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
>>  
>> -			plane->sizeimage += info->bpp[i] *
>> -				DIV_ROUND_UP(aligned_width, hdiv) *
>> -				DIV_ROUND_UP(aligned_height, vdiv);
>> +			sizeimage += info->bpp[i] *
>> +				     DIV_ROUND_UP(aligned_width, hdiv) *
>> +				     DIV_ROUND_UP(aligned_height, vdiv);
>>  		}
>> +
>> +		/* Custom bytesperline value is not supported yet. */
>> +		plane->bytesperline = ALIGN(width,
>> +					    v4l2_format_block_width(info, 0)) *
>> +				      info->bpp[0];
>> +
>> +		/*
>> +		 * The user might have specified a custom sizeimage, only
>> +		 * override it if it's not big enough.
>> +		 */
>> +		plane->sizeimage = max(sizeimage, plane->sizeimage);
> 
> No upper limit? That doesn't sound a good idea to me, specially since some
> (broken) app might not be memset the format to zero before filling the ioctl
> structure.
> 
> Perhaps we could do something like:
> 
> 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> 
> or something similar that would be reasonable.

I've no idea what's sane.

Buffers can be really large. The largest video resolution defined by CTA-861-G
is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
instead of min/max.

Regards,

	Hans

> 
>>  	} else {
>>  		for (i = 0; i < info->comp_planes; i++) {
>>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
>> @@ -613,10 +624,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
>>  
>>  			plane = &pixfmt->plane_fmt[i];
>> -			plane->bytesperline =
>> -				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv);
>> -			plane->sizeimage =
>> -				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
> 
>> +
>> +			/* Custom bytesperline value is not supported yet. */
> 
> Supporting custom bytesperline seems too risky of breaking apps. 
> So, I would drop this comment.
> 
> 
>> +			plane->bytesperline = info->bpp[i] *
>> +					      DIV_ROUND_UP(aligned_width, hdiv);
> 
>> +
>> +			/*
>> +			 * The user might have specified a custom sizeimage,
>> +			 * only override it if it's not big enough.
>> +			 */
>> +			plane->sizeimage = max_t(u32,
>> +						 plane->bytesperline *
>> +						 DIV_ROUND_UP(aligned_height, vdiv),
>> +						 plane->sizeimage);
>>  		}
>>  	}
>>  	return 0;
>> @@ -627,6 +647,7 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
>>  		     u32 width, u32 height)
>>  {
>>  	const struct v4l2_format_info *info;
>> +	u32 sizeimage = 0;
>>  	int i;
>>  
>>  	info = v4l2_format_info(pixelformat);
>> @@ -640,8 +661,6 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
>>  	pixfmt->width = width;
>>  	pixfmt->height = height;
>>  	pixfmt->pixelformat = pixelformat;
>> -	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
>> -	pixfmt->sizeimage = 0;
>>  
>>  	for (i = 0; i < info->comp_planes; i++) {
>>  		unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
>> @@ -651,11 +670,20 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
>>  
>>  		aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
>>  		aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
>> -
>> -		pixfmt->sizeimage += info->bpp[i] *
>> -			DIV_ROUND_UP(aligned_width, hdiv) *
>> -			DIV_ROUND_UP(aligned_height, vdiv);
>> +		sizeimage += info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) *
>> +			     DIV_ROUND_UP(aligned_height, vdiv);
>>  	}
>> +
>> +	/* Custom bytesperline value is not supported yet. */
>> +	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) *
>> +			       info->bpp[0];
>> +
>> +	/*
>> +	 * The user might have specified its own sizeimage value, only override
>> +	 * it if it's not big enough.
>> +	 */
>> +	pixfmt->sizeimage = max(sizeimage, pixfmt->sizeimage);
>> +
> 
> Same comment applies here: We need to sanitize it from too big sizeimages.
> 
>>  	return 0;
>>  }
>>  EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
> 
> 
> 
> Thanks,
> Mauro
> 

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Mauro Carvalho Chehab]

Em Wed, 29 May 2019 13:43:20 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:
> > Em Tue, 28 May 2019 14:02:19 -0300
> > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> >   
> >> From: Boris Brezillon <boris.brezillon@collabora.com>
> >>
> >> Users can define custom sizeimage as long as they're big enough to
> >> store the amount of pixels required for a specific width/height under a
> >> specific format. Avoid overriding those fields in this case.
> >>
> >> We could possibly do the same for bytesperline, but it gets tricky when
> >> dealing with !MPLANE definitions, so this case is omitted for now and  
> >> ->bytesperline is always overwritten with the value calculated in  
> >> fill_pixfmt().
> >>
> >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> >> ---
> >> Changes from v5:
> >> * Overwrite bytesperline with the value calculated in fill_pixfmt()
> >>
> >> Changes from v4:
> >> * New patch
> >>
> >>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> >>  1 file changed, 43 insertions(+), 15 deletions(-)
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> >> index b2d1e55d9561..fd286f6e17d7 100644
> >> --- a/drivers/media/v4l2-core/v4l2-common.c
> >> +++ b/drivers/media/v4l2-core/v4l2-common.c
> >> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >>  	pixfmt->num_planes = info->mem_planes;
> >>  
> >>  	if (info->mem_planes == 1) {
> >> +		u32 sizeimage = 0;
> >> +
> >>  		plane = &pixfmt->plane_fmt[0];
> >> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> >> -		plane->sizeimage = 0;
> >>  
> >>  		for (i = 0; i < info->comp_planes; i++) {
> >>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> >> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> >>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> >>  
> >> -			plane->sizeimage += info->bpp[i] *
> >> -				DIV_ROUND_UP(aligned_width, hdiv) *
> >> -				DIV_ROUND_UP(aligned_height, vdiv);
> >> +			sizeimage += info->bpp[i] *
> >> +				     DIV_ROUND_UP(aligned_width, hdiv) *
> >> +				     DIV_ROUND_UP(aligned_height, vdiv);
> >>  		}
> >> +
> >> +		/* Custom bytesperline value is not supported yet. */
> >> +		plane->bytesperline = ALIGN(width,
> >> +					    v4l2_format_block_width(info, 0)) *
> >> +				      info->bpp[0];
> >> +
> >> +		/*
> >> +		 * The user might have specified a custom sizeimage, only
> >> +		 * override it if it's not big enough.
> >> +		 */
> >> +		plane->sizeimage = max(sizeimage, plane->sizeimage);  
> > 
> > No upper limit? That doesn't sound a good idea to me, specially since some
> > (broken) app might not be memset the format to zero before filling the ioctl
> > structure.
> > 
> > Perhaps we could do something like:
> > 
> > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > 
> > or something similar that would be reasonable.  
> 
> I've no idea what's sane.
> 
> Buffers can be really large. The largest video resolution defined by CTA-861-G
> is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> instead of min/max.

Well, the max is driver-specific. 

For example, for a camera with a max resolution of 640x480 with 2 bytes
per pixel as the max format can only be

	max_size = 640*480*2 (plus some alignment value if pertinent)

It sounds to me that the best would be to have a callback function
or value filled by the drivers that would support custom sizeimage.

The core could actually calculate during init (by asking the driver
to a very big resolution and getting the returned value), but
it sounds better to let the drivers to explicitly calculate it.

> 
> Regards,
> 
> 	Hans
> 
> >   
> >>  	} else {
> >>  		for (i = 0; i < info->comp_planes; i++) {
> >>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> >> @@ -613,10 +624,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> >>  
> >>  			plane = &pixfmt->plane_fmt[i];
> >> -			plane->bytesperline =
> >> -				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv);
> >> -			plane->sizeimage =
> >> -				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);  
> >   
> >> +
> >> +			/* Custom bytesperline value is not supported yet. */  
> > 
> > Supporting custom bytesperline seems too risky of breaking apps. 
> > So, I would drop this comment.
> > 
> >   
> >> +			plane->bytesperline = info->bpp[i] *
> >> +					      DIV_ROUND_UP(aligned_width, hdiv);  
> >   
> >> +
> >> +			/*
> >> +			 * The user might have specified a custom sizeimage,
> >> +			 * only override it if it's not big enough.
> >> +			 */
> >> +			plane->sizeimage = max_t(u32,
> >> +						 plane->bytesperline *
> >> +						 DIV_ROUND_UP(aligned_height, vdiv),
> >> +						 plane->sizeimage);
> >>  		}
> >>  	}
> >>  	return 0;
> >> @@ -627,6 +647,7 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
> >>  		     u32 width, u32 height)
> >>  {
> >>  	const struct v4l2_format_info *info;
> >> +	u32 sizeimage = 0;
> >>  	int i;
> >>  
> >>  	info = v4l2_format_info(pixelformat);
> >> @@ -640,8 +661,6 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
> >>  	pixfmt->width = width;
> >>  	pixfmt->height = height;
> >>  	pixfmt->pixelformat = pixelformat;
> >> -	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> >> -	pixfmt->sizeimage = 0;
> >>  
> >>  	for (i = 0; i < info->comp_planes; i++) {
> >>  		unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> >> @@ -651,11 +670,20 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
> >>  
> >>  		aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> >>  		aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> >> -
> >> -		pixfmt->sizeimage += info->bpp[i] *
> >> -			DIV_ROUND_UP(aligned_width, hdiv) *
> >> -			DIV_ROUND_UP(aligned_height, vdiv);
> >> +		sizeimage += info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) *
> >> +			     DIV_ROUND_UP(aligned_height, vdiv);
> >>  	}
> >> +
> >> +	/* Custom bytesperline value is not supported yet. */
> >> +	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) *
> >> +			       info->bpp[0];
> >> +
> >> +	/*
> >> +	 * The user might have specified its own sizeimage value, only override
> >> +	 * it if it's not big enough.
> >> +	 */
> >> +	pixfmt->sizeimage = max(sizeimage, pixfmt->sizeimage);
> >> +  
> > 
> > Same comment applies here: We need to sanitize it from too big sizeimages.
> >   
> >>  	return 0;
> >>  }
> >>  EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);  
> > 
> > 
> > 
> > Thanks,
> > Mauro
> >   
> 



Thanks,
Mauro

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Boris Brezillon]

On Wed, 29 May 2019 08:58:54 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:

> Em Wed, 29 May 2019 13:43:20 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> 
> > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:  
> > > Em Tue, 28 May 2019 14:02:19 -0300
> > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > >     
> > >> From: Boris Brezillon <boris.brezillon@collabora.com>
> > >>
> > >> Users can define custom sizeimage as long as they're big enough to
> > >> store the amount of pixels required for a specific width/height under a
> > >> specific format. Avoid overriding those fields in this case.
> > >>
> > >> We could possibly do the same for bytesperline, but it gets tricky when
> > >> dealing with !MPLANE definitions, so this case is omitted for now and    
> > >> ->bytesperline is always overwritten with the value calculated in    
> > >> fill_pixfmt().
> > >>
> > >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > >> ---
> > >> Changes from v5:
> > >> * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > >>
> > >> Changes from v4:
> > >> * New patch
> > >>
> > >>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > >>  1 file changed, 43 insertions(+), 15 deletions(-)
> > >>
> > >> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > >> index b2d1e55d9561..fd286f6e17d7 100644
> > >> --- a/drivers/media/v4l2-core/v4l2-common.c
> > >> +++ b/drivers/media/v4l2-core/v4l2-common.c
> > >> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > >>  	pixfmt->num_planes = info->mem_planes;
> > >>  
> > >>  	if (info->mem_planes == 1) {
> > >> +		u32 sizeimage = 0;
> > >> +
> > >>  		plane = &pixfmt->plane_fmt[0];
> > >> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > >> -		plane->sizeimage = 0;
> > >>  
> > >>  		for (i = 0; i < info->comp_planes; i++) {
> > >>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > >> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > >>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > >>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > >>  
> > >> -			plane->sizeimage += info->bpp[i] *
> > >> -				DIV_ROUND_UP(aligned_width, hdiv) *
> > >> -				DIV_ROUND_UP(aligned_height, vdiv);
> > >> +			sizeimage += info->bpp[i] *
> > >> +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > >> +				     DIV_ROUND_UP(aligned_height, vdiv);
> > >>  		}
> > >> +
> > >> +		/* Custom bytesperline value is not supported yet. */
> > >> +		plane->bytesperline = ALIGN(width,
> > >> +					    v4l2_format_block_width(info, 0)) *
> > >> +				      info->bpp[0];
> > >> +
> > >> +		/*
> > >> +		 * The user might have specified a custom sizeimage, only
> > >> +		 * override it if it's not big enough.
> > >> +		 */
> > >> +		plane->sizeimage = max(sizeimage, plane->sizeimage);    
> > > 
> > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > (broken) app might not be memset the format to zero before filling the ioctl
> > > structure.
> > > 
> > > Perhaps we could do something like:
> > > 
> > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > 
> > > or something similar that would be reasonable.    
> > 
> > I've no idea what's sane.
> > 
> > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > instead of min/max.  
> 
> Well, the max is driver-specific. 
> 
> For example, for a camera with a max resolution of 640x480 with 2 bytes
> per pixel as the max format can only be
> 
> 	max_size = 640*480*2 (plus some alignment value if pertinent)
> 
> It sounds to me that the best would be to have a callback function
> or value filled by the drivers that would support custom sizeimage.
> 
> The core could actually calculate during init (by asking the driver
> to a very big resolution and getting the returned value), but
> it sounds better to let the drivers to explicitly calculate it.

If we want max_sizeimage to be driver specific I can add it as an extra
arg to the fill_pixfmt() funcs.

If that works for you, we'll send a new version of this patch alone
(unless you want us to send the whole series again).

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Mauro Carvalho Chehab]

Em Wed, 29 May 2019 14:16:33 +0200
Boris Brezillon <boris.brezillon@collabora.com> escreveu:

> On Wed, 29 May 2019 08:58:54 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> 
> > Em Wed, 29 May 2019 13:43:20 +0200
> > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> >   
> > > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:    
> > > > Em Tue, 28 May 2019 14:02:19 -0300
> > > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > > >       
> > > >> From: Boris Brezillon <boris.brezillon@collabora.com>
> > > >>
> > > >> Users can define custom sizeimage as long as they're big enough to
> > > >> store the amount of pixels required for a specific width/height under a
> > > >> specific format. Avoid overriding those fields in this case.
> > > >>
> > > >> We could possibly do the same for bytesperline, but it gets tricky when
> > > >> dealing with !MPLANE definitions, so this case is omitted for now and      
> > > >> ->bytesperline is always overwritten with the value calculated in      
> > > >> fill_pixfmt().
> > > >>
> > > >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > >> ---
> > > >> Changes from v5:
> > > >> * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > >>
> > > >> Changes from v4:
> > > >> * New patch
> > > >>
> > > >>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > > >>  1 file changed, 43 insertions(+), 15 deletions(-)
> > > >>
> > > >> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > >> index b2d1e55d9561..fd286f6e17d7 100644
> > > >> --- a/drivers/media/v4l2-core/v4l2-common.c
> > > >> +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > >> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > >>  	pixfmt->num_planes = info->mem_planes;
> > > >>  
> > > >>  	if (info->mem_planes == 1) {
> > > >> +		u32 sizeimage = 0;
> > > >> +
> > > >>  		plane = &pixfmt->plane_fmt[0];
> > > >> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > >> -		plane->sizeimage = 0;
> > > >>  
> > > >>  		for (i = 0; i < info->comp_planes; i++) {
> > > >>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > >> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > >>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > > >>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > > >>  
> > > >> -			plane->sizeimage += info->bpp[i] *
> > > >> -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > >> -				DIV_ROUND_UP(aligned_height, vdiv);
> > > >> +			sizeimage += info->bpp[i] *
> > > >> +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > >> +				     DIV_ROUND_UP(aligned_height, vdiv);
> > > >>  		}
> > > >> +
> > > >> +		/* Custom bytesperline value is not supported yet. */
> > > >> +		plane->bytesperline = ALIGN(width,
> > > >> +					    v4l2_format_block_width(info, 0)) *
> > > >> +				      info->bpp[0];
> > > >> +
> > > >> +		/*
> > > >> +		 * The user might have specified a custom sizeimage, only
> > > >> +		 * override it if it's not big enough.
> > > >> +		 */
> > > >> +		plane->sizeimage = max(sizeimage, plane->sizeimage);      
> > > > 
> > > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > > (broken) app might not be memset the format to zero before filling the ioctl
> > > > structure.
> > > > 
> > > > Perhaps we could do something like:
> > > > 
> > > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > > 
> > > > or something similar that would be reasonable.      
> > > 
> > > I've no idea what's sane.
> > > 
> > > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > > instead of min/max.    
> > 
> > Well, the max is driver-specific. 
> > 
> > For example, for a camera with a max resolution of 640x480 with 2 bytes
> > per pixel as the max format can only be
> > 
> > 	max_size = 640*480*2 (plus some alignment value if pertinent)
> > 
> > It sounds to me that the best would be to have a callback function
> > or value filled by the drivers that would support custom sizeimage.
> > 
> > The core could actually calculate during init (by asking the driver
> > to a very big resolution and getting the returned value), but
> > it sounds better to let the drivers to explicitly calculate it.  
> 
> If we want max_sizeimage to be driver specific I can add it as an extra
> arg to the fill_pixfmt() funcs.

Works for me.

> If that works for you, we'll send a new version of this patch alone
> (unless you want us to send the whole series again).

If the other patches on this series don't depend on this, I can
apply the PR just skipping this one, applying your patch afterwards.

Thanks,
Mauro

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Mauro Carvalho Chehab]

Em Wed, 29 May 2019 09:26:32 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:

> Em Wed, 29 May 2019 14:16:33 +0200
> Boris Brezillon <boris.brezillon@collabora.com> escreveu:
> 
> > On Wed, 29 May 2019 08:58:54 -0300
> > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> >   
> > > Em Wed, 29 May 2019 13:43:20 +0200
> > > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> > >     
> > > > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:      
> > > > > Em Tue, 28 May 2019 14:02:19 -0300
> > > > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > > > >         
> > > > >> From: Boris Brezillon <boris.brezillon@collabora.com>
> > > > >>
> > > > >> Users can define custom sizeimage as long as they're big enough to
> > > > >> store the amount of pixels required for a specific width/height under a
> > > > >> specific format. Avoid overriding those fields in this case.
> > > > >>
> > > > >> We could possibly do the same for bytesperline, but it gets tricky when
> > > > >> dealing with !MPLANE definitions, so this case is omitted for now and        
> > > > >> ->bytesperline is always overwritten with the value calculated in        
> > > > >> fill_pixfmt().
> > > > >>
> > > > >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > > >> ---
> > > > >> Changes from v5:
> > > > >> * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > > >>
> > > > >> Changes from v4:
> > > > >> * New patch
> > > > >>
> > > > >>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > > > >>  1 file changed, 43 insertions(+), 15 deletions(-)
> > > > >>
> > > > >> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > > >> index b2d1e55d9561..fd286f6e17d7 100644
> > > > >> --- a/drivers/media/v4l2-core/v4l2-common.c
> > > > >> +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > > >> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > >>  	pixfmt->num_planes = info->mem_planes;
> > > > >>  
> > > > >>  	if (info->mem_planes == 1) {
> > > > >> +		u32 sizeimage = 0;
> > > > >> +
> > > > >>  		plane = &pixfmt->plane_fmt[0];
> > > > >> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > > >> -		plane->sizeimage = 0;
> > > > >>  
> > > > >>  		for (i = 0; i < info->comp_planes; i++) {
> > > > >>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > > >> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > >>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > > > >>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > > > >>  
> > > > >> -			plane->sizeimage += info->bpp[i] *
> > > > >> -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > > >> -				DIV_ROUND_UP(aligned_height, vdiv);
> > > > >> +			sizeimage += info->bpp[i] *
> > > > >> +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > > >> +				     DIV_ROUND_UP(aligned_height, vdiv);
> > > > >>  		}
> > > > >> +
> > > > >> +		/* Custom bytesperline value is not supported yet. */
> > > > >> +		plane->bytesperline = ALIGN(width,
> > > > >> +					    v4l2_format_block_width(info, 0)) *
> > > > >> +				      info->bpp[0];
> > > > >> +
> > > > >> +		/*
> > > > >> +		 * The user might have specified a custom sizeimage, only
> > > > >> +		 * override it if it's not big enough.
> > > > >> +		 */
> > > > >> +		plane->sizeimage = max(sizeimage, plane->sizeimage);        
> > > > > 
> > > > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > > > (broken) app might not be memset the format to zero before filling the ioctl
> > > > > structure.
> > > > > 
> > > > > Perhaps we could do something like:
> > > > > 
> > > > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > > > 
> > > > > or something similar that would be reasonable.        
> > > > 
> > > > I've no idea what's sane.
> > > > 
> > > > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > > > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > > > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > > > instead of min/max.      
> > > 
> > > Well, the max is driver-specific. 
> > > 
> > > For example, for a camera with a max resolution of 640x480 with 2 bytes
> > > per pixel as the max format can only be
> > > 
> > > 	max_size = 640*480*2 (plus some alignment value if pertinent)
> > > 
> > > It sounds to me that the best would be to have a callback function
> > > or value filled by the drivers that would support custom sizeimage.
> > > 
> > > The core could actually calculate during init (by asking the driver
> > > to a very big resolution and getting the returned value), but
> > > it sounds better to let the drivers to explicitly calculate it.    
> > 
> > If we want max_sizeimage to be driver specific I can add it as an extra
> > arg to the fill_pixfmt() funcs.  
> 
> Works for me.
> 
> > If that works for you, we'll send a new version of this patch alone
> > (unless you want us to send the whole series again).  
> 
> If the other patches on this series don't depend on this, I can
> apply the PR just skipping this one, applying your patch afterwards.

Well, I can also apply the PR as-is and apply an extra patch at the
end.

> 
> Thanks,
> Mauro



Thanks,
Mauro

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Boris Brezillon]

On Wed, 29 May 2019 09:26:32 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:

> Em Wed, 29 May 2019 14:16:33 +0200
> Boris Brezillon <boris.brezillon@collabora.com> escreveu:
> 
> > On Wed, 29 May 2019 08:58:54 -0300
> > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> >   
> > > Em Wed, 29 May 2019 13:43:20 +0200
> > > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> > >     
> > > > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:      
> > > > > Em Tue, 28 May 2019 14:02:19 -0300
> > > > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > > > >         
> > > > >> From: Boris Brezillon <boris.brezillon@collabora.com>
> > > > >>
> > > > >> Users can define custom sizeimage as long as they're big enough to
> > > > >> store the amount of pixels required for a specific width/height under a
> > > > >> specific format. Avoid overriding those fields in this case.
> > > > >>
> > > > >> We could possibly do the same for bytesperline, but it gets tricky when
> > > > >> dealing with !MPLANE definitions, so this case is omitted for now and        
> > > > >> ->bytesperline is always overwritten with the value calculated in        
> > > > >> fill_pixfmt().
> > > > >>
> > > > >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > > >> ---
> > > > >> Changes from v5:
> > > > >> * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > > >>
> > > > >> Changes from v4:
> > > > >> * New patch
> > > > >>
> > > > >>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > > > >>  1 file changed, 43 insertions(+), 15 deletions(-)
> > > > >>
> > > > >> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > > >> index b2d1e55d9561..fd286f6e17d7 100644
> > > > >> --- a/drivers/media/v4l2-core/v4l2-common.c
> > > > >> +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > > >> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > >>  	pixfmt->num_planes = info->mem_planes;
> > > > >>  
> > > > >>  	if (info->mem_planes == 1) {
> > > > >> +		u32 sizeimage = 0;
> > > > >> +
> > > > >>  		plane = &pixfmt->plane_fmt[0];
> > > > >> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > > >> -		plane->sizeimage = 0;
> > > > >>  
> > > > >>  		for (i = 0; i < info->comp_planes; i++) {
> > > > >>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > > >> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > >>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > > > >>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > > > >>  
> > > > >> -			plane->sizeimage += info->bpp[i] *
> > > > >> -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > > >> -				DIV_ROUND_UP(aligned_height, vdiv);
> > > > >> +			sizeimage += info->bpp[i] *
> > > > >> +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > > >> +				     DIV_ROUND_UP(aligned_height, vdiv);
> > > > >>  		}
> > > > >> +
> > > > >> +		/* Custom bytesperline value is not supported yet. */
> > > > >> +		plane->bytesperline = ALIGN(width,
> > > > >> +					    v4l2_format_block_width(info, 0)) *
> > > > >> +				      info->bpp[0];
> > > > >> +
> > > > >> +		/*
> > > > >> +		 * The user might have specified a custom sizeimage, only
> > > > >> +		 * override it if it's not big enough.
> > > > >> +		 */
> > > > >> +		plane->sizeimage = max(sizeimage, plane->sizeimage);        
> > > > > 
> > > > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > > > (broken) app might not be memset the format to zero before filling the ioctl
> > > > > structure.
> > > > > 
> > > > > Perhaps we could do something like:
> > > > > 
> > > > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > > > 
> > > > > or something similar that would be reasonable.        
> > > > 
> > > > I've no idea what's sane.
> > > > 
> > > > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > > > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > > > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > > > instead of min/max.      
> > > 
> > > Well, the max is driver-specific. 
> > > 
> > > For example, for a camera with a max resolution of 640x480 with 2 bytes
> > > per pixel as the max format can only be
> > > 
> > > 	max_size = 640*480*2 (plus some alignment value if pertinent)
> > > 
> > > It sounds to me that the best would be to have a callback function
> > > or value filled by the drivers that would support custom sizeimage.
> > > 
> > > The core could actually calculate during init (by asking the driver
> > > to a very big resolution and getting the returned value), but
> > > it sounds better to let the drivers to explicitly calculate it.    
> > 
> > If we want max_sizeimage to be driver specific I can add it as an extra
> > arg to the fill_pixfmt() funcs.  
> 
> Works for me.
> 
> > If that works for you, we'll send a new version of this patch alone
> > (unless you want us to send the whole series again).  
> 
> If the other patches on this series don't depend on this, I can
> apply the PR just skipping this one, applying your patch afterwards.

Unfortunately they do. I was just asking if we could send a new version
of this patch and let you or Hans apply the other ones from this series
on top. If that's too complicated, we'll submit a v7 containing all
patches. BTW, any concerns on other patches in this series before we
send a new version?

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Hans Verkuil]

On 5/29/19 2:16 PM, Boris Brezillon wrote:
> On Wed, 29 May 2019 08:58:54 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> 
>> Em Wed, 29 May 2019 13:43:20 +0200
>> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
>>
>>> On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:  
>>>> Em Tue, 28 May 2019 14:02:19 -0300
>>>> Ezequiel Garcia <ezequiel@collabora.com> escreveu:
>>>>     
>>>>> From: Boris Brezillon <boris.brezillon@collabora.com>
>>>>>
>>>>> Users can define custom sizeimage as long as they're big enough to
>>>>> store the amount of pixels required for a specific width/height under a
>>>>> specific format. Avoid overriding those fields in this case.
>>>>>
>>>>> We could possibly do the same for bytesperline, but it gets tricky when
>>>>> dealing with !MPLANE definitions, so this case is omitted for now and    
>>>>> ->bytesperline is always overwritten with the value calculated in    
>>>>> fill_pixfmt().
>>>>>
>>>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
>>>>> ---
>>>>> Changes from v5:
>>>>> * Overwrite bytesperline with the value calculated in fill_pixfmt()
>>>>>
>>>>> Changes from v4:
>>>>> * New patch
>>>>>
>>>>>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
>>>>>  1 file changed, 43 insertions(+), 15 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
>>>>> index b2d1e55d9561..fd286f6e17d7 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-common.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-common.c
>>>>> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>>>>>  	pixfmt->num_planes = info->mem_planes;
>>>>>  
>>>>>  	if (info->mem_planes == 1) {
>>>>> +		u32 sizeimage = 0;
>>>>> +
>>>>>  		plane = &pixfmt->plane_fmt[0];
>>>>> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
>>>>> -		plane->sizeimage = 0;
>>>>>  
>>>>>  		for (i = 0; i < info->comp_planes; i++) {
>>>>>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
>>>>> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>>>>>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
>>>>>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
>>>>>  
>>>>> -			plane->sizeimage += info->bpp[i] *
>>>>> -				DIV_ROUND_UP(aligned_width, hdiv) *
>>>>> -				DIV_ROUND_UP(aligned_height, vdiv);
>>>>> +			sizeimage += info->bpp[i] *
>>>>> +				     DIV_ROUND_UP(aligned_width, hdiv) *
>>>>> +				     DIV_ROUND_UP(aligned_height, vdiv);
>>>>>  		}
>>>>> +
>>>>> +		/* Custom bytesperline value is not supported yet. */
>>>>> +		plane->bytesperline = ALIGN(width,
>>>>> +					    v4l2_format_block_width(info, 0)) *
>>>>> +				      info->bpp[0];
>>>>> +
>>>>> +		/*
>>>>> +		 * The user might have specified a custom sizeimage, only
>>>>> +		 * override it if it's not big enough.
>>>>> +		 */
>>>>> +		plane->sizeimage = max(sizeimage, plane->sizeimage);    
>>>>
>>>> No upper limit? That doesn't sound a good idea to me, specially since some
>>>> (broken) app might not be memset the format to zero before filling the ioctl
>>>> structure.
>>>>
>>>> Perhaps we could do something like:
>>>>
>>>> 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
>>>>
>>>> or something similar that would be reasonable.    
>>>
>>> I've no idea what's sane.
>>>
>>> Buffers can be really large. The largest video resolution defined by CTA-861-G
>>> is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
>>> use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
>>> instead of min/max.  
>>
>> Well, the max is driver-specific. 
>>
>> For example, for a camera with a max resolution of 640x480 with 2 bytes
>> per pixel as the max format can only be
>>
>> 	max_size = 640*480*2 (plus some alignment value if pertinent)
>>
>> It sounds to me that the best would be to have a callback function
>> or value filled by the drivers that would support custom sizeimage.
>>
>> The core could actually calculate during init (by asking the driver
>> to a very big resolution and getting the returned value), but
>> it sounds better to let the drivers to explicitly calculate it.
> 
> If we want max_sizeimage to be driver specific I can add it as an extra
> arg to the fill_pixfmt() funcs.

Looking more closely, only compressed formats can accept a user-specified
sizeimage value, and this function is only called for uncompressed formats.

So doesn't that mean that this sizeimage override code can be dropped?

Regards,

	Hans

> 
> If that works for you, we'll send a new version of this patch alone
> (unless you want us to send the whole series again).
> 

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Boris Brezillon]

On Wed, 29 May 2019 14:31:03 +0200
Hans Verkuil <hverkuil@xs4all.nl> wrote:

> On 5/29/19 2:16 PM, Boris Brezillon wrote:
> > On Wed, 29 May 2019 08:58:54 -0300
> > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> >   
> >> Em Wed, 29 May 2019 13:43:20 +0200
> >> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> >>  
> >>> On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:    
> >>>> Em Tue, 28 May 2019 14:02:19 -0300
> >>>> Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> >>>>       
> >>>>> From: Boris Brezillon <boris.brezillon@collabora.com>
> >>>>>
> >>>>> Users can define custom sizeimage as long as they're big enough to
> >>>>> store the amount of pixels required for a specific width/height under a
> >>>>> specific format. Avoid overriding those fields in this case.
> >>>>>
> >>>>> We could possibly do the same for bytesperline, but it gets tricky when
> >>>>> dealing with !MPLANE definitions, so this case is omitted for now and      
> >>>>> ->bytesperline is always overwritten with the value calculated in      
> >>>>> fill_pixfmt().
> >>>>>
> >>>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> >>>>> ---
> >>>>> Changes from v5:
> >>>>> * Overwrite bytesperline with the value calculated in fill_pixfmt()
> >>>>>
> >>>>> Changes from v4:
> >>>>> * New patch
> >>>>>
> >>>>>  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> >>>>>  1 file changed, 43 insertions(+), 15 deletions(-)
> >>>>>
> >>>>> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> >>>>> index b2d1e55d9561..fd286f6e17d7 100644
> >>>>> --- a/drivers/media/v4l2-core/v4l2-common.c
> >>>>> +++ b/drivers/media/v4l2-core/v4l2-common.c
> >>>>> @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >>>>>  	pixfmt->num_planes = info->mem_planes;
> >>>>>  
> >>>>>  	if (info->mem_planes == 1) {
> >>>>> +		u32 sizeimage = 0;
> >>>>> +
> >>>>>  		plane = &pixfmt->plane_fmt[0];
> >>>>> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> >>>>> -		plane->sizeimage = 0;
> >>>>>  
> >>>>>  		for (i = 0; i < info->comp_planes; i++) {
> >>>>>  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> >>>>> @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >>>>>  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> >>>>>  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> >>>>>  
> >>>>> -			plane->sizeimage += info->bpp[i] *
> >>>>> -				DIV_ROUND_UP(aligned_width, hdiv) *
> >>>>> -				DIV_ROUND_UP(aligned_height, vdiv);
> >>>>> +			sizeimage += info->bpp[i] *
> >>>>> +				     DIV_ROUND_UP(aligned_width, hdiv) *
> >>>>> +				     DIV_ROUND_UP(aligned_height, vdiv);
> >>>>>  		}
> >>>>> +
> >>>>> +		/* Custom bytesperline value is not supported yet. */
> >>>>> +		plane->bytesperline = ALIGN(width,
> >>>>> +					    v4l2_format_block_width(info, 0)) *
> >>>>> +				      info->bpp[0];
> >>>>> +
> >>>>> +		/*
> >>>>> +		 * The user might have specified a custom sizeimage, only
> >>>>> +		 * override it if it's not big enough.
> >>>>> +		 */
> >>>>> +		plane->sizeimage = max(sizeimage, plane->sizeimage);      
> >>>>
> >>>> No upper limit? That doesn't sound a good idea to me, specially since some
> >>>> (broken) app might not be memset the format to zero before filling the ioctl
> >>>> structure.
> >>>>
> >>>> Perhaps we could do something like:
> >>>>
> >>>> 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> >>>>
> >>>> or something similar that would be reasonable.      
> >>>
> >>> I've no idea what's sane.
> >>>
> >>> Buffers can be really large. The largest video resolution defined by CTA-861-G
> >>> is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> >>> use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> >>> instead of min/max.    
> >>
> >> Well, the max is driver-specific. 
> >>
> >> For example, for a camera with a max resolution of 640x480 with 2 bytes
> >> per pixel as the max format can only be
> >>
> >> 	max_size = 640*480*2 (plus some alignment value if pertinent)
> >>
> >> It sounds to me that the best would be to have a callback function
> >> or value filled by the drivers that would support custom sizeimage.
> >>
> >> The core could actually calculate during init (by asking the driver
> >> to a very big resolution and getting the returned value), but
> >> it sounds better to let the drivers to explicitly calculate it.  
> > 
> > If we want max_sizeimage to be driver specific I can add it as an extra
> > arg to the fill_pixfmt() funcs.  
> 
> Looking more closely, only compressed formats can accept a user-specified
> sizeimage value, and this function is only called for uncompressed formats.
> 
> So doesn't that mean that this sizeimage override code can be dropped?

Hehe, IIRC, you were the one asking for this  :P (or maybe I
misunderstood what you suggested). I don't think we need to preserve
user-defined ->sizeimage for uncompressed fmt in the VPU driver, so I'm
perfectly fine implementing a version that overrides ->sizeimage
unconditionally.

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Ezequiel Garcia]

On Wed, 2019-05-29 at 14:31 +0200, Hans Verkuil wrote:
> On 5/29/19 2:16 PM, Boris Brezillon wrote:
> > On Wed, 29 May 2019 08:58:54 -0300
> > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > 
> > > Em Wed, 29 May 2019 13:43:20 +0200
> > > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> > > 
> > > > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:  
> > > > > Em Tue, 28 May 2019 14:02:19 -0300
> > > > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > > > >     
> > > > > > From: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > 
> > > > > > Users can define custom sizeimage as long as they're big enough to
> > > > > > store the amount of pixels required for a specific width/height under a
> > > > > > specific format. Avoid overriding those fields in this case.
> > > > > > 
> > > > > > We could possibly do the same for bytesperline, but it gets tricky when
> > > > > > dealing with !MPLANE definitions, so this case is omitted for now and    
> > > > > > ->bytesperline is always overwritten with the value calculated in    
> > > > > > fill_pixfmt().
> > > > > > 
> > > > > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > ---
> > > > > > Changes from v5:
> > > > > > * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > > > > 
> > > > > > Changes from v4:
> > > > > > * New patch
> > > > > > 
> > > > > >  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > > > > >  1 file changed, 43 insertions(+), 15 deletions(-)
> > > > > > 
> > > > > > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > index b2d1e55d9561..fd286f6e17d7 100644
> > > > > > --- a/drivers/media/v4l2-core/v4l2-common.c
> > > > > > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > >  	pixfmt->num_planes = info->mem_planes;
> > > > > >  
> > > > > >  	if (info->mem_planes == 1) {
> > > > > > +		u32 sizeimage = 0;
> > > > > > +
> > > > > >  		plane = &pixfmt->plane_fmt[0];
> > > > > > -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > > > > -		plane->sizeimage = 0;
> > > > > >  
> > > > > >  		for (i = 0; i < info->comp_planes; i++) {
> > > > > >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > > > > @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > >  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > > > > >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > > > > >  
> > > > > > -			plane->sizeimage += info->bpp[i] *
> > > > > > -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > -				DIV_ROUND_UP(aligned_height, vdiv);
> > > > > > +			sizeimage += info->bpp[i] *
> > > > > > +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > +				     DIV_ROUND_UP(aligned_height, vdiv);
> > > > > >  		}
> > > > > > +
> > > > > > +		/* Custom bytesperline value is not supported yet. */
> > > > > > +		plane->bytesperline = ALIGN(width,
> > > > > > +					    v4l2_format_block_width(info, 0)) *
> > > > > > +				      info->bpp[0];
> > > > > > +
> > > > > > +		/*
> > > > > > +		 * The user might have specified a custom sizeimage, only
> > > > > > +		 * override it if it's not big enough.
> > > > > > +		 */
> > > > > > +		plane->sizeimage = max(sizeimage, plane->sizeimage);    
> > > > > 
> > > > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > > > (broken) app might not be memset the format to zero before filling the ioctl
> > > > > structure.
> > > > > 
> > > > > Perhaps we could do something like:
> > > > > 
> > > > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > > > 
> > > > > or something similar that would be reasonable.    
> > > > 
> > > > I've no idea what's sane.
> > > > 
> > > > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > > > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > > > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > > > instead of min/max.  
> > > 
> > > Well, the max is driver-specific. 
> > > 
> > > For example, for a camera with a max resolution of 640x480 with 2 bytes
> > > per pixel as the max format can only be
> > > 
> > > 	max_size = 640*480*2 (plus some alignment value if pertinent)
> > > 
> > > It sounds to me that the best would be to have a callback function
> > > or value filled by the drivers that would support custom sizeimage.
> > > 
> > > The core could actually calculate during init (by asking the driver
> > > to a very big resolution and getting the returned value), but
> > > it sounds better to let the drivers to explicitly calculate it.
> > 
> > If we want max_sizeimage to be driver specific I can add it as an extra
> > arg to the fill_pixfmt() funcs.
> 
> Looking more closely, only compressed formats can accept a user-specified
> sizeimage value, and this function is only called for uncompressed formats.
> 
> So doesn't that mean that this sizeimage override code can be dropped?
> 

I think this is a good idea, which means just picking all patches except this one
(and the RK3328 one).

So it would be patches: 1,2 and 4 to 15.

Boris, Hans: are you OK with this?

Thanks,
Ezequiel

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Boris Brezillon]

On Wed, 29 May 2019 11:04:35 -0300
Ezequiel Garcia <ezequiel@collabora.com> wrote:

> On Wed, 2019-05-29 at 14:31 +0200, Hans Verkuil wrote:
> > On 5/29/19 2:16 PM, Boris Brezillon wrote:  
> > > On Wed, 29 May 2019 08:58:54 -0300
> > > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > >   
> > > > Em Wed, 29 May 2019 13:43:20 +0200
> > > > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> > > >   
> > > > > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:    
> > > > > > Em Tue, 28 May 2019 14:02:19 -0300
> > > > > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > > > > >       
> > > > > > > From: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > > 
> > > > > > > Users can define custom sizeimage as long as they're big enough to
> > > > > > > store the amount of pixels required for a specific width/height under a
> > > > > > > specific format. Avoid overriding those fields in this case.
> > > > > > > 
> > > > > > > We could possibly do the same for bytesperline, but it gets tricky when
> > > > > > > dealing with !MPLANE definitions, so this case is omitted for now and      
> > > > > > > ->bytesperline is always overwritten with the value calculated in      
> > > > > > > fill_pixfmt().
> > > > > > > 
> > > > > > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > > ---
> > > > > > > Changes from v5:
> > > > > > > * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > > > > > 
> > > > > > > Changes from v4:
> > > > > > > * New patch
> > > > > > > 
> > > > > > >  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > > > > > >  1 file changed, 43 insertions(+), 15 deletions(-)
> > > > > > > 
> > > > > > > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > index b2d1e55d9561..fd286f6e17d7 100644
> > > > > > > --- a/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > > >  	pixfmt->num_planes = info->mem_planes;
> > > > > > >  
> > > > > > >  	if (info->mem_planes == 1) {
> > > > > > > +		u32 sizeimage = 0;
> > > > > > > +
> > > > > > >  		plane = &pixfmt->plane_fmt[0];
> > > > > > > -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > > > > > -		plane->sizeimage = 0;
> > > > > > >  
> > > > > > >  		for (i = 0; i < info->comp_planes; i++) {
> > > > > > >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > > > > > @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > > >  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > > > > > >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > > > > > >  
> > > > > > > -			plane->sizeimage += info->bpp[i] *
> > > > > > > -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > > -				DIV_ROUND_UP(aligned_height, vdiv);
> > > > > > > +			sizeimage += info->bpp[i] *
> > > > > > > +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > > +				     DIV_ROUND_UP(aligned_height, vdiv);
> > > > > > >  		}
> > > > > > > +
> > > > > > > +		/* Custom bytesperline value is not supported yet. */
> > > > > > > +		plane->bytesperline = ALIGN(width,
> > > > > > > +					    v4l2_format_block_width(info, 0)) *
> > > > > > > +				      info->bpp[0];
> > > > > > > +
> > > > > > > +		/*
> > > > > > > +		 * The user might have specified a custom sizeimage, only
> > > > > > > +		 * override it if it's not big enough.
> > > > > > > +		 */
> > > > > > > +		plane->sizeimage = max(sizeimage, plane->sizeimage);      
> > > > > > 
> > > > > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > > > > (broken) app might not be memset the format to zero before filling the ioctl
> > > > > > structure.
> > > > > > 
> > > > > > Perhaps we could do something like:
> > > > > > 
> > > > > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > > > > 
> > > > > > or something similar that would be reasonable.      
> > > > > 
> > > > > I've no idea what's sane.
> > > > > 
> > > > > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > > > > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > > > > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > > > > instead of min/max.    
> > > > 
> > > > Well, the max is driver-specific. 
> > > > 
> > > > For example, for a camera with a max resolution of 640x480 with 2 bytes
> > > > per pixel as the max format can only be
> > > > 
> > > > 	max_size = 640*480*2 (plus some alignment value if pertinent)
> > > > 
> > > > It sounds to me that the best would be to have a callback function
> > > > or value filled by the drivers that would support custom sizeimage.
> > > > 
> > > > The core could actually calculate during init (by asking the driver
> > > > to a very big resolution and getting the returned value), but
> > > > it sounds better to let the drivers to explicitly calculate it.  
> > > 
> > > If we want max_sizeimage to be driver specific I can add it as an extra
> > > arg to the fill_pixfmt() funcs.  
> > 
> > Looking more closely, only compressed formats can accept a user-specified
> > sizeimage value, and this function is only called for uncompressed formats.
> > 
> > So doesn't that mean that this sizeimage override code can be dropped?
> >   
> 
> I think this is a good idea, which means just picking all patches except this one
> (and the RK3328 one).
> 
> So it would be patches: 1,2 and 4 to 15.
> 
> Boris, Hans: are you OK with this?

I doesn't work => the prototype of the fill_pixfmt() changed.

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Boris Brezillon]

On Wed, 29 May 2019 16:06:10 +0200
Boris Brezillon <boris.brezillon@collabora.com> wrote:

> On Wed, 29 May 2019 11:04:35 -0300
> Ezequiel Garcia <ezequiel@collabora.com> wrote:
> 
> > On Wed, 2019-05-29 at 14:31 +0200, Hans Verkuil wrote:  
> > > On 5/29/19 2:16 PM, Boris Brezillon wrote:    
> > > > On Wed, 29 May 2019 08:58:54 -0300
> > > > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > > >     
> > > > > Em Wed, 29 May 2019 13:43:20 +0200
> > > > > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> > > > >     
> > > > > > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:      
> > > > > > > Em Tue, 28 May 2019 14:02:19 -0300
> > > > > > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > > > > > >         
> > > > > > > > From: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > > > 
> > > > > > > > Users can define custom sizeimage as long as they're big enough to
> > > > > > > > store the amount of pixels required for a specific width/height under a
> > > > > > > > specific format. Avoid overriding those fields in this case.
> > > > > > > > 
> > > > > > > > We could possibly do the same for bytesperline, but it gets tricky when
> > > > > > > > dealing with !MPLANE definitions, so this case is omitted for now and        
> > > > > > > > ->bytesperline is always overwritten with the value calculated in        
> > > > > > > > fill_pixfmt().
> > > > > > > > 
> > > > > > > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > > > ---
> > > > > > > > Changes from v5:
> > > > > > > > * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > > > > > > 
> > > > > > > > Changes from v4:
> > > > > > > > * New patch
> > > > > > > > 
> > > > > > > >  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > > > > > > >  1 file changed, 43 insertions(+), 15 deletions(-)
> > > > > > > > 
> > > > > > > > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > > index b2d1e55d9561..fd286f6e17d7 100644
> > > > > > > > --- a/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > > @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > > > >  	pixfmt->num_planes = info->mem_planes;
> > > > > > > >  
> > > > > > > >  	if (info->mem_planes == 1) {
> > > > > > > > +		u32 sizeimage = 0;
> > > > > > > > +
> > > > > > > >  		plane = &pixfmt->plane_fmt[0];
> > > > > > > > -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > > > > > > -		plane->sizeimage = 0;
> > > > > > > >  
> > > > > > > >  		for (i = 0; i < info->comp_planes; i++) {
> > > > > > > >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > > > > > > @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > > > >  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > > > > > > >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > > > > > > >  
> > > > > > > > -			plane->sizeimage += info->bpp[i] *
> > > > > > > > -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > > > -				DIV_ROUND_UP(aligned_height, vdiv);
> > > > > > > > +			sizeimage += info->bpp[i] *
> > > > > > > > +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > > > +				     DIV_ROUND_UP(aligned_height, vdiv);
> > > > > > > >  		}
> > > > > > > > +
> > > > > > > > +		/* Custom bytesperline value is not supported yet. */
> > > > > > > > +		plane->bytesperline = ALIGN(width,
> > > > > > > > +					    v4l2_format_block_width(info, 0)) *
> > > > > > > > +				      info->bpp[0];
> > > > > > > > +
> > > > > > > > +		/*
> > > > > > > > +		 * The user might have specified a custom sizeimage, only
> > > > > > > > +		 * override it if it's not big enough.
> > > > > > > > +		 */
> > > > > > > > +		plane->sizeimage = max(sizeimage, plane->sizeimage);        
> > > > > > > 
> > > > > > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > > > > > (broken) app might not be memset the format to zero before filling the ioctl
> > > > > > > structure.
> > > > > > > 
> > > > > > > Perhaps we could do something like:
> > > > > > > 
> > > > > > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > > > > > 
> > > > > > > or something similar that would be reasonable.        
> > > > > > 
> > > > > > I've no idea what's sane.
> > > > > > 
> > > > > > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > > > > > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > > > > > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > > > > > instead of min/max.      
> > > > > 
> > > > > Well, the max is driver-specific. 
> > > > > 
> > > > > For example, for a camera with a max resolution of 640x480 with 2 bytes
> > > > > per pixel as the max format can only be
> > > > > 
> > > > > 	max_size = 640*480*2 (plus some alignment value if pertinent)
> > > > > 
> > > > > It sounds to me that the best would be to have a callback function
> > > > > or value filled by the drivers that would support custom sizeimage.
> > > > > 
> > > > > The core could actually calculate during init (by asking the driver
> > > > > to a very big resolution and getting the returned value), but
> > > > > it sounds better to let the drivers to explicitly calculate it.    
> > > > 
> > > > If we want max_sizeimage to be driver specific I can add it as an extra
> > > > arg to the fill_pixfmt() funcs.    
> > > 
> > > Looking more closely, only compressed formats can accept a user-specified
> > > sizeimage value, and this function is only called for uncompressed formats.
> > > 
> > > So doesn't that mean that this sizeimage override code can be dropped?
> > >     
> > 
> > I think this is a good idea, which means just picking all patches except this one
> > (and the RK3328 one).
> > 
> > So it would be patches: 1,2 and 4 to 15.
> > 
> > Boris, Hans: are you OK with this?  
> 
> I doesn't work => the prototype of the fill_pixfmt() changed.

Forget what I said, dropping patch 3 should work just fine.

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Ezequiel Garcia]

On Wed, 2019-05-29 at 16:17 +0200, Boris Brezillon wrote:
> On Wed, 29 May 2019 16:06:10 +0200
> Boris Brezillon <boris.brezillon@collabora.com> wrote:
> 
> > On Wed, 29 May 2019 11:04:35 -0300
> > Ezequiel Garcia <ezequiel@collabora.com> wrote:
> > 
> > > On Wed, 2019-05-29 at 14:31 +0200, Hans Verkuil wrote:  
> > > > On 5/29/19 2:16 PM, Boris Brezillon wrote:    
> > > > > On Wed, 29 May 2019 08:58:54 -0300
> > > > > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > > > >     
> > > > > > Em Wed, 29 May 2019 13:43:20 +0200
> > > > > > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> > > > > >     
> > > > > > > On 5/29/19 1:28 PM, Mauro Carvalho Chehab wrote:      
> > > > > > > > Em Tue, 28 May 2019 14:02:19 -0300
> > > > > > > > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > > > > > > >         
> > > > > > > > > From: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > > > > 
> > > > > > > > > Users can define custom sizeimage as long as they're big enough to
> > > > > > > > > store the amount of pixels required for a specific width/height under a
> > > > > > > > > specific format. Avoid overriding those fields in this case.
> > > > > > > > > 
> > > > > > > > > We could possibly do the same for bytesperline, but it gets tricky when
> > > > > > > > > dealing with !MPLANE definitions, so this case is omitted for now and        
> > > > > > > > > ->bytesperline is always overwritten with the value calculated in        
> > > > > > > > > fill_pixfmt().
> > > > > > > > > 
> > > > > > > > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > > > > > > > ---
> > > > > > > > > Changes from v5:
> > > > > > > > > * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > > > > > > > 
> > > > > > > > > Changes from v4:
> > > > > > > > > * New patch
> > > > > > > > > 
> > > > > > > > >  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > > > > > > > >  1 file changed, 43 insertions(+), 15 deletions(-)
> > > > > > > > > 
> > > > > > > > > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > > > index b2d1e55d9561..fd286f6e17d7 100644
> > > > > > > > > --- a/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > > > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > > > > > > > @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > > > > >  	pixfmt->num_planes = info->mem_planes;
> > > > > > > > >  
> > > > > > > > >  	if (info->mem_planes == 1) {
> > > > > > > > > +		u32 sizeimage = 0;
> > > > > > > > > +
> > > > > > > > >  		plane = &pixfmt->plane_fmt[0];
> > > > > > > > > -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > > > > > > > -		plane->sizeimage = 0;
> > > > > > > > >  
> > > > > > > > >  		for (i = 0; i < info->comp_planes; i++) {
> > > > > > > > >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > > > > > > > @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > > > > > > > >  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > > > > > > > >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > > > > > > > >  
> > > > > > > > > -			plane->sizeimage += info->bpp[i] *
> > > > > > > > > -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > > > > -				DIV_ROUND_UP(aligned_height, vdiv);
> > > > > > > > > +			sizeimage += info->bpp[i] *
> > > > > > > > > +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > > > > > > > +				     DIV_ROUND_UP(aligned_height, vdiv);
> > > > > > > > >  		}
> > > > > > > > > +
> > > > > > > > > +		/* Custom bytesperline value is not supported yet. */
> > > > > > > > > +		plane->bytesperline = ALIGN(width,
> > > > > > > > > +					    v4l2_format_block_width(info, 0)) *
> > > > > > > > > +				      info->bpp[0];
> > > > > > > > > +
> > > > > > > > > +		/*
> > > > > > > > > +		 * The user might have specified a custom sizeimage, only
> > > > > > > > > +		 * override it if it's not big enough.
> > > > > > > > > +		 */
> > > > > > > > > +		plane->sizeimage = max(sizeimage, plane->sizeimage);        
> > > > > > > > 
> > > > > > > > No upper limit? That doesn't sound a good idea to me, specially since some
> > > > > > > > (broken) app might not be memset the format to zero before filling the ioctl
> > > > > > > > structure.
> > > > > > > > 
> > > > > > > > Perhaps we could do something like:
> > > > > > > > 
> > > > > > > > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > > > > > > > 
> > > > > > > > or something similar that would be reasonable.        
> > > > > > > 
> > > > > > > I've no idea what's sane.
> > > > > > > 
> > > > > > > Buffers can be really large. The largest video resolution defined by CTA-861-G
> > > > > > > is 10240x4320, so at 4 bytes per pixel that's 0x0a8c0000. So perhaps we can
> > > > > > > use min(sizeimage, 0x10000000)? Although we should probably use the clamp function
> > > > > > > instead of min/max.      
> > > > > > 
> > > > > > Well, the max is driver-specific. 
> > > > > > 
> > > > > > For example, for a camera with a max resolution of 640x480 with 2 bytes
> > > > > > per pixel as the max format can only be
> > > > > > 
> > > > > > 	max_size = 640*480*2 (plus some alignment value if pertinent)
> > > > > > 
> > > > > > It sounds to me that the best would be to have a callback function
> > > > > > or value filled by the drivers that would support custom sizeimage.
> > > > > > 
> > > > > > The core could actually calculate during init (by asking the driver
> > > > > > to a very big resolution and getting the returned value), but
> > > > > > it sounds better to let the drivers to explicitly calculate it.    
> > > > > 
> > > > > If we want max_sizeimage to be driver specific I can add it as an extra
> > > > > arg to the fill_pixfmt() funcs.    
> > > > 
> > > > Looking more closely, only compressed formats can accept a user-specified
> > > > sizeimage value, and this function is only called for uncompressed formats.
> > > > 
> > > > So doesn't that mean that this sizeimage override code can be dropped?
> > > >     
> > > 
> > > I think this is a good idea, which means just picking all patches except this one
> > > (and the RK3328 one).
> > > 
> > > So it would be patches: 1,2 and 4 to 15.
> > > 
> > > Boris, Hans: are you OK with this?  
> > 
> > I doesn't work => the prototype of the fill_pixfmt() changed.
> 
> Forget what I said, dropping patch 3 should work just fine.

Yeah, I have reviewed the code and it seems OK. The prototype is not changed,
and the driver is not relying on the sizeimage override feature.

Also, tested on my RK3399 Rockpi board, and MPEG-2 decoding works fine.

v4l2-compliance also passes with no failures.

Seems OK to just drop patch 3.

Thanks,
Ezequiel

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 6802 bytes --]

Hi Mauro,

Le mercredi 29 mai 2019 à 08:28 -0300, Mauro Carvalho Chehab a écrit :
> Em Tue, 28 May 2019 14:02:19 -0300
> Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> 
> > From: Boris Brezillon <boris.brezillon@collabora.com>
> > 
> > Users can define custom sizeimage as long as they're big enough to
> > store the amount of pixels required for a specific width/height under a
> > specific format. Avoid overriding those fields in this case.
> > 
> > We could possibly do the same for bytesperline, but it gets tricky when
> > dealing with !MPLANE definitions, so this case is omitted for now and
> > ->bytesperline is always overwritten with the value calculated in
> > fill_pixfmt().
> > 
> > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > ---
> > Changes from v5:
> > * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > 
> > Changes from v4:
> > * New patch
> > 
> >  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> >  1 file changed, 43 insertions(+), 15 deletions(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > index b2d1e55d9561..fd286f6e17d7 100644
> > --- a/drivers/media/v4l2-core/v4l2-common.c
> > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >  	pixfmt->num_planes = info->mem_planes;
> >  
> >  	if (info->mem_planes == 1) {
> > +		u32 sizeimage = 0;
> > +
> >  		plane = &pixfmt->plane_fmt[0];
> > -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > -		plane->sizeimage = 0;
> >  
> >  		for (i = 0; i < info->comp_planes; i++) {
> >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> >  
> > -			plane->sizeimage += info->bpp[i] *
> > -				DIV_ROUND_UP(aligned_width, hdiv) *
> > -				DIV_ROUND_UP(aligned_height, vdiv);
> > +			sizeimage += info->bpp[i] *
> > +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > +				     DIV_ROUND_UP(aligned_height, vdiv);
> >  		}
> > +
> > +		/* Custom bytesperline value is not supported yet. */
> > +		plane->bytesperline = ALIGN(width,
> > +					    v4l2_format_block_width(info, 0)) *
> > +				      info->bpp[0];
> > +
> > +		/*
> > +		 * The user might have specified a custom sizeimage, only
> > +		 * override it if it's not big enough.
> > +		 */
> > +		plane->sizeimage = max(sizeimage, plane->sizeimage);
> 
> No upper limit? That doesn't sound a good idea to me, specially since some
> (broken) app might not be memset the format to zero before filling the ioctl
> structure.
> 
> Perhaps we could do something like:
> 
> 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> 
> or something similar that would be reasonable.
> 
> >  	} else {
> >  		for (i = 0; i < info->comp_planes; i++) {
> >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > @@ -613,10 +624,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> >  
> >  			plane = &pixfmt->plane_fmt[i];
> > -			plane->bytesperline =
> > -				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv);
> > -			plane->sizeimage =
> > -				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
> > +
> > +			/* Custom bytesperline value is not supported yet. */
> 
> Supporting custom bytesperline seems too risky of breaking apps. 
> So, I would drop this comment.

We will really need this in the long run in many drivers in order to
allow import/export of DMABuf. Without such adaptive feature, we have a
software limitation that forces bouncing memory. I have already
discussed about adding this feature notably in vivid and uvcvideo on
IRC and in conference, which both have no restriction the memory
alignment, so should allow importing any kind of video layout.

We already have a partial userspace implementation for this in
GStreamer and upstream driver submission should come when the IP is
considered stable enough.

Why I think it won't break userspace is that the correct way to use
these read-only members of V4L2 struct is to set these to 0, which is
also documented.

Adding upper bound seems like a good idea though.

> 
> 
> > +			plane->bytesperline = info->bpp[i] *
> > +					      DIV_ROUND_UP(aligned_width, hdiv);
> > +
> > +			/*
> > +			 * The user might have specified a custom sizeimage,
> > +			 * only override it if it's not big enough.
> > +			 */
> > +			plane->sizeimage = max_t(u32,
> > +						 plane->bytesperline *
> > +						 DIV_ROUND_UP(aligned_height, vdiv),
> > +						 plane->sizeimage);
> >  		}
> >  	}
> >  	return 0;
> > @@ -627,6 +647,7 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
> >  		     u32 width, u32 height)
> >  {
> >  	const struct v4l2_format_info *info;
> > +	u32 sizeimage = 0;
> >  	int i;
> >  
> >  	info = v4l2_format_info(pixelformat);
> > @@ -640,8 +661,6 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
> >  	pixfmt->width = width;
> >  	pixfmt->height = height;
> >  	pixfmt->pixelformat = pixelformat;
> > -	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > -	pixfmt->sizeimage = 0;
> >  
> >  	for (i = 0; i < info->comp_planes; i++) {
> >  		unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > @@ -651,11 +670,20 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
> >  
> >  		aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> >  		aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > -
> > -		pixfmt->sizeimage += info->bpp[i] *
> > -			DIV_ROUND_UP(aligned_width, hdiv) *
> > -			DIV_ROUND_UP(aligned_height, vdiv);
> > +		sizeimage += info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) *
> > +			     DIV_ROUND_UP(aligned_height, vdiv);
> >  	}
> > +
> > +	/* Custom bytesperline value is not supported yet. */
> > +	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) *
> > +			       info->bpp[0];
> > +
> > +	/*
> > +	 * The user might have specified its own sizeimage value, only override
> > +	 * it if it's not big enough.
> > +	 */
> > +	pixfmt->sizeimage = max(sizeimage, pixfmt->sizeimage);
> > +
> 
> Same comment applies here: We need to sanitize it from too big sizeimages.
> 
> >  	return 0;
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
> 
> 
> Thanks,
> Mauro

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH v6 03/16] media: v4l2-common: Support custom imagesize in fill_pixfmt()

[Mauro Carvalho Chehab]

Em Wed, 29 May 2019 09:54:09 -0400
Nicolas Dufresne <nicolas.dufresne@collabora.com> escreveu:

> Hi Mauro,
> 
> Le mercredi 29 mai 2019 à 08:28 -0300, Mauro Carvalho Chehab a écrit :
> > Em Tue, 28 May 2019 14:02:19 -0300
> > Ezequiel Garcia <ezequiel@collabora.com> escreveu:
> > 
> > > From: Boris Brezillon <boris.brezillon@collabora.com>
> > > 
> > > Users can define custom sizeimage as long as they're big enough to
> > > store the amount of pixels required for a specific width/height under a
> > > specific format. Avoid overriding those fields in this case.
> > > 
> > > We could possibly do the same for bytesperline, but it gets tricky when
> > > dealing with !MPLANE definitions, so this case is omitted for now and
> > > ->bytesperline is always overwritten with the value calculated in
> > > fill_pixfmt().
> > > 
> > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > ---
> > > Changes from v5:
> > > * Overwrite bytesperline with the value calculated in fill_pixfmt()
> > > 
> > > Changes from v4:
> > > * New patch
> > > 
> > >  drivers/media/v4l2-core/v4l2-common.c | 58 ++++++++++++++++++++-------
> > >  1 file changed, 43 insertions(+), 15 deletions(-)
> > > 
> > > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> > > index b2d1e55d9561..fd286f6e17d7 100644
> > > --- a/drivers/media/v4l2-core/v4l2-common.c
> > > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > > @@ -585,9 +585,9 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > >  	pixfmt->num_planes = info->mem_planes;
> > >  
> > >  	if (info->mem_planes == 1) {
> > > +		u32 sizeimage = 0;
> > > +
> > >  		plane = &pixfmt->plane_fmt[0];
> > > -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0];
> > > -		plane->sizeimage = 0;
> > >  
> > >  		for (i = 0; i < info->comp_planes; i++) {
> > >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > @@ -598,10 +598,21 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > >  			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> > >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > >  
> > > -			plane->sizeimage += info->bpp[i] *
> > > -				DIV_ROUND_UP(aligned_width, hdiv) *
> > > -				DIV_ROUND_UP(aligned_height, vdiv);
> > > +			sizeimage += info->bpp[i] *
> > > +				     DIV_ROUND_UP(aligned_width, hdiv) *
> > > +				     DIV_ROUND_UP(aligned_height, vdiv);
> > >  		}
> > > +
> > > +		/* Custom bytesperline value is not supported yet. */
> > > +		plane->bytesperline = ALIGN(width,
> > > +					    v4l2_format_block_width(info, 0)) *
> > > +				      info->bpp[0];
> > > +
> > > +		/*
> > > +		 * The user might have specified a custom sizeimage, only
> > > +		 * override it if it's not big enough.
> > > +		 */
> > > +		plane->sizeimage = max(sizeimage, plane->sizeimage);
> > 
> > No upper limit? That doesn't sound a good idea to me, specially since some
> > (broken) app might not be memset the format to zero before filling the ioctl
> > structure.
> > 
> > Perhaps we could do something like:
> > 
> > 		sizeimage = min (sizeimage, 2 * plane->sizeimage)
> > 
> > or something similar that would be reasonable.
> > 
> > >  	} else {
> > >  		for (i = 0; i < info->comp_planes; i++) {
> > >  			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> > > @@ -613,10 +624,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> > >  			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> > >  
> > >  			plane = &pixfmt->plane_fmt[i];
> > > -			plane->bytesperline =
> > > -				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv);
> > > -			plane->sizeimage =
> > > -				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
> > > +
> > > +			/* Custom bytesperline value is not supported yet. */
> > 
> > Supporting custom bytesperline seems too risky of breaking apps. 
> > So, I would drop this comment.
> 
> We will really need this in the long run in many drivers in order to
> allow import/export of DMABuf. Without such adaptive feature, we have a
> software limitation that forces bouncing memory. I have already
> discussed about adding this feature notably in vivid and uvcvideo on
> IRC and in conference, which both have no restriction the memory
> alignment, so should allow importing any kind of video layout.
> 
> We already have a partial userspace implementation for this in
> GStreamer and upstream driver submission should come when the IP is
> considered stable enough.

I understand the need. I'm not against an implementation for such
feature, provided that it won't break anything.

I guess one of the things we miss at V4L2 API is an indication from
userspace about what it supports. I mean, just like we have the
caps flags where the Kernel reports what it supports, we could have
a similar "userspace caps"  field.

> Why I think it won't break userspace is that the correct way to use
> these read-only members of V4L2 struct is to set these to 0, which is
> also documented.

Yeah, the apps I'm aware of usually call memset() before filling
V4L2 structs. On those, adding this behavior would be ok. Yet,
I'm not sure if 100% of (open source) apps do that.

> Adding upper bound seems like a good idea though.

Agreed.

> 
>
Thanks,
Mauro

[PATCH v6 04/16] rockchip/vpu: Use v4l2_apply_frmsize_constraints() where appropriate

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

Use the v4l2_apply_frmsize_constraints() helper instead of open-coding
it.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v5:
* None

Changes from v4:
* New patch

 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 44 ++++++-------------
 1 file changed, 14 insertions(+), 30 deletions(-)

diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
index 7c7c20ab2733..aa00df9a7ecb 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
@@ -197,15 +197,9 @@ vidioc_try_fmt_cap_mplane(struct file *file, void *priv, struct v4l2_format *f)
 
 	pix_mp->num_planes = 1;
 	pix_mp->field = V4L2_FIELD_NONE;
-	pix_mp->width = clamp(pix_mp->width,
-			      fmt->frmsize.min_width,
-			      fmt->frmsize.max_width);
-	pix_mp->height = clamp(pix_mp->height,
-			       fmt->frmsize.min_height,
-			       fmt->frmsize.max_height);
-	/* Round up to macroblocks. */
-	pix_mp->width = round_up(pix_mp->width, fmt->frmsize.step_width);
-	pix_mp->height = round_up(pix_mp->height, fmt->frmsize.step_height);
+
+	v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height,
+				       &fmt->frmsize);
 
 	/*
 	 * For compressed formats the application can specify
@@ -226,7 +220,6 @@ vidioc_try_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	struct rockchip_vpu_ctx *ctx = fh_to_ctx(priv);
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	const struct rockchip_vpu_fmt *fmt;
-	unsigned int width, height;
 	int i;
 
 	vpu_debug(4, "%c%c%c%c\n",
@@ -242,18 +235,13 @@ vidioc_try_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	}
 
 	pix_mp->field = V4L2_FIELD_NONE;
-	width = clamp(pix_mp->width,
-		      ctx->vpu_dst_fmt->frmsize.min_width,
-		      ctx->vpu_dst_fmt->frmsize.max_width);
-	height = clamp(pix_mp->height,
-		       ctx->vpu_dst_fmt->frmsize.min_height,
-		       ctx->vpu_dst_fmt->frmsize.max_height);
-	/* Round up to macroblocks. */
-	width = round_up(width, ctx->vpu_dst_fmt->frmsize.step_width);
-	height = round_up(height, ctx->vpu_dst_fmt->frmsize.step_height);
+
+	v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height,
+				       &ctx->vpu_dst_fmt->frmsize);
 
 	/* Fill remaining fields */
-	v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, width, height);
+	v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, pix_mp->width,
+			    pix_mp->height);
 
 	for (i = 0; i < pix_mp->num_planes; i++) {
 		memset(pix_mp->plane_fmt[i].reserved, 0,
@@ -272,10 +260,8 @@ void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
 	memset(fmt, 0, sizeof(*fmt));
 
 	fmt->num_planes = 1;
-	fmt->width = clamp(fmt->width, ctx->vpu_dst_fmt->frmsize.min_width,
-			   ctx->vpu_dst_fmt->frmsize.max_width);
-	fmt->height = clamp(fmt->height, ctx->vpu_dst_fmt->frmsize.min_height,
-			    ctx->vpu_dst_fmt->frmsize.max_height);
+	v4l2_apply_frmsize_constraints(&fmt->width, &fmt->height,
+				       &ctx->vpu_dst_fmt->frmsize);
 	fmt->pixelformat = ctx->vpu_dst_fmt->fourcc;
 	fmt->field = V4L2_FIELD_NONE;
 	fmt->colorspace = V4L2_COLORSPACE_JPEG,
@@ -291,23 +277,21 @@ void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx)
 {
 	struct v4l2_pix_format_mplane *fmt = &ctx->src_fmt;
-	unsigned int width, height;
 
 	ctx->vpu_src_fmt = rockchip_vpu_get_default_fmt(ctx, false);
 
 	memset(fmt, 0, sizeof(*fmt));
 
-	width = clamp(fmt->width, ctx->vpu_dst_fmt->frmsize.min_width,
-		      ctx->vpu_dst_fmt->frmsize.max_width);
-	height = clamp(fmt->height, ctx->vpu_dst_fmt->frmsize.min_height,
-		       ctx->vpu_dst_fmt->frmsize.max_height);
+	v4l2_apply_frmsize_constraints(&fmt->width, &fmt->height,
+				       &ctx->vpu_src_fmt->frmsize);
 	fmt->field = V4L2_FIELD_NONE;
 	fmt->colorspace = V4L2_COLORSPACE_JPEG,
 	fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
 	fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
-	v4l2_fill_pixfmt_mp(fmt, ctx->vpu_src_fmt->fourcc, width, height);
+	v4l2_fill_pixfmt_mp(fmt, ctx->vpu_src_fmt->fourcc, fmt->width,
+			    fmt->height);
 }
 
 static int
-- 
2.20.1

[PATCH v6 05/16] rockchip/vpu: Open-code media controller register

[Ezequiel Garcia]

In preparation to support decoders, using a single memory-to-memory
device, we need to roll our own media controller entities registration.

To do that, we define a rockchip_vpu_func object that embeds the
video_device object plus all the elements that are needed to attach this
vdev to the media device.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
--
Changes from v3:
* Rework the media controller registration logic (Boris)
* Fix media controller deregistration (Jonas)

Changes from v2:
* Use kvasprintf instead of kmalloc and snprintf.
* Fix missing kfree in error paths.
* Remove unneeded media_remove_intf_links on error paths.

 .../staging/media/rockchip/vpu/rockchip_vpu.h |  39 +++-
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 207 +++++++++++++++---
 2 files changed, 215 insertions(+), 31 deletions(-)

diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
index b15c02333a70..aba257c663a7 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
@@ -71,12 +71,47 @@ enum rockchip_vpu_codec_mode {
 	RK_VPU_MODE_JPEG_ENC,
 };
 
+/*
+ * struct rockchip_vpu_func - rockchip VPU functionality
+ *
+ * @id:			processing functionality ID (can be
+ *			%MEDIA_ENT_F_PROC_VIDEO_ENCODER or
+ *			%MEDIA_ENT_F_PROC_VIDEO_DECODER)
+ * @vdev:		&struct video_device that exposes the encoder or
+ *			decoder functionality
+ * @source_pad:		&struct media_pad with the source pad.
+ * @sink:		&struct media_entity pointer with the sink entity
+ * @sink_pad:		&struct media_pad with the sink pad.
+ * @proc:		&struct media_entity pointer with the M2M device itself.
+ * @proc_pads:		&struct media_pad with the @proc pads.
+ * @intf_devnode:	&struct media_intf devnode pointer with the interface
+ *			with controls the M2M device.
+ *
+ * Contains everything needed to attach the video device to the media device.
+ */
+struct rockchip_vpu_func {
+	unsigned int id;
+	struct video_device vdev;
+	struct media_pad source_pad;
+	struct media_entity sink;
+	struct media_pad sink_pad;
+	struct media_entity proc;
+	struct media_pad proc_pads[2];
+	struct media_intf_devnode *intf_devnode;
+};
+
+static inline struct rockchip_vpu_func *
+rockchip_vpu_vdev_to_func(struct video_device *vdev)
+{
+	return container_of(vdev, struct rockchip_vpu_func, vdev);
+}
+
 /**
  * struct rockchip_vpu_dev - driver data
  * @v4l2_dev:		V4L2 device to register video devices for.
  * @m2m_dev:		mem2mem device associated to this device.
  * @mdev:		media device associated to this device.
- * @vfd_enc:		Video device for encoder.
+ * @encoder:		encoder functionality.
  * @pdev:		Pointer to VPU platform device.
  * @dev:		Pointer to device for convenient logging using
  *			dev_ macros.
@@ -93,7 +128,7 @@ struct rockchip_vpu_dev {
 	struct v4l2_device v4l2_dev;
 	struct v4l2_m2m_dev *m2m_dev;
 	struct media_device mdev;
-	struct video_device *vfd_enc;
+	struct rockchip_vpu_func *encoder;
 	struct platform_device *pdev;
 	struct device *dev;
 	struct clk_bulk_data clocks[ROCKCHIP_VPU_MAX_CLOCKS];
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index 3c3ce3baeb6d..fa02354a0f8a 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -239,6 +239,7 @@ static int rockchip_vpu_open(struct file *filp)
 {
 	struct rockchip_vpu_dev *vpu = video_drvdata(filp);
 	struct video_device *vdev = video_devdata(filp);
+	struct rockchip_vpu_func *func = rockchip_vpu_vdev_to_func(vdev);
 	struct rockchip_vpu_ctx *ctx;
 	int ret;
 
@@ -256,7 +257,7 @@ static int rockchip_vpu_open(struct file *filp)
 		return -ENOMEM;
 
 	ctx->dev = vpu;
-	if (vdev == vpu->vfd_enc)
+	if (func->id == MEDIA_ENT_F_PROC_VIDEO_ENCODER)
 		ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(vpu->m2m_dev, ctx,
 						    &enc_queue_init);
 	else
@@ -324,52 +325,206 @@ static const struct of_device_id of_rockchip_vpu_match[] = {
 };
 MODULE_DEVICE_TABLE(of, of_rockchip_vpu_match);
 
-static int rockchip_vpu_video_device_register(struct rockchip_vpu_dev *vpu)
+static int rockchip_vpu_register_entity(struct media_device *mdev,
+					struct media_entity *entity,
+					const char *entity_name,
+					struct media_pad *pads, int num_pads,
+					int function,
+					struct video_device *vdev)
+{
+	char *name;
+	int ret;
+
+	entity->obj_type = MEDIA_ENTITY_TYPE_BASE;
+	if (function == MEDIA_ENT_F_IO_V4L) {
+		entity->info.dev.major = VIDEO_MAJOR;
+		entity->info.dev.minor = vdev->minor;
+	}
+
+	name = devm_kasprintf(mdev->dev, GFP_KERNEL, "%s-%s", vdev->name,
+			      entity_name);
+	if (!name)
+		return -ENOMEM;
+
+	entity->name = name;
+	entity->function = function;
+
+	ret = media_entity_pads_init(entity, num_pads, pads);
+	if (ret)
+		return ret;
+
+	ret = media_device_register_entity(mdev, entity);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int rockchip_attach_func(struct rockchip_vpu_dev *vpu,
+				struct rockchip_vpu_func *func)
+{
+	struct media_device *mdev = &vpu->mdev;
+	struct media_link *link;
+	int ret;
+
+	/* Create the three encoder entities with their pads */
+	func->source_pad.flags = MEDIA_PAD_FL_SOURCE;
+	ret = rockchip_vpu_register_entity(mdev, &func->vdev.entity,
+					   "source", &func->source_pad, 1,
+					   MEDIA_ENT_F_IO_V4L, &func->vdev);
+	if (ret)
+		return ret;
+
+	func->proc_pads[0].flags = MEDIA_PAD_FL_SINK;
+	func->proc_pads[1].flags = MEDIA_PAD_FL_SOURCE;
+	ret = rockchip_vpu_register_entity(mdev, &func->proc, "proc",
+					   func->proc_pads, 2, func->id,
+					   &func->vdev);
+	if (ret)
+		goto err_rel_entity0;
+
+	func->sink_pad.flags = MEDIA_PAD_FL_SINK;
+	ret = rockchip_vpu_register_entity(mdev, &func->sink, "sink",
+					   &func->sink_pad, 1,
+					   MEDIA_ENT_F_IO_V4L, &func->vdev);
+	if (ret)
+		goto err_rel_entity1;
+
+	/* Connect the three entities */
+	ret = media_create_pad_link(&func->vdev.entity, 0, &func->proc, 1,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		goto err_rel_entity2;
+
+	ret = media_create_pad_link(&func->proc, 0, &func->sink, 0,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		goto err_rm_links0;
+
+	/* Create video interface */
+	func->intf_devnode = media_devnode_create(mdev, MEDIA_INTF_T_V4L_VIDEO,
+						  0, VIDEO_MAJOR,
+						  func->vdev.minor);
+	if (!func->intf_devnode) {
+		ret = -ENOMEM;
+		goto err_rm_links1;
+	}
+
+	/* Connect the two DMA engines to the interface */
+	link = media_create_intf_link(&func->vdev.entity,
+				      &func->intf_devnode->intf,
+				      MEDIA_LNK_FL_IMMUTABLE |
+				      MEDIA_LNK_FL_ENABLED);
+	if (!link) {
+		ret = -ENOMEM;
+		goto err_rm_devnode;
+	}
+
+	link = media_create_intf_link(&func->sink, &func->intf_devnode->intf,
+				      MEDIA_LNK_FL_IMMUTABLE |
+				      MEDIA_LNK_FL_ENABLED);
+	if (!link) {
+		ret = -ENOMEM;
+		goto err_rm_devnode;
+	}
+	return 0;
+
+err_rm_devnode:
+	media_devnode_remove(func->intf_devnode);
+
+err_rm_links1:
+	media_entity_remove_links(&func->sink);
+
+err_rm_links0:
+	media_entity_remove_links(&func->proc);
+	media_entity_remove_links(&func->vdev.entity);
+
+err_rel_entity2:
+	media_device_unregister_entity(&func->sink);
+
+err_rel_entity1:
+	media_device_unregister_entity(&func->proc);
+
+err_rel_entity0:
+	media_device_unregister_entity(&func->vdev.entity);
+	return ret;
+}
+
+static void rockchip_detach_func(struct rockchip_vpu_func *func)
+{
+	media_devnode_remove(func->intf_devnode);
+	media_entity_remove_links(&func->sink);
+	media_entity_remove_links(&func->proc);
+	media_entity_remove_links(&func->vdev.entity);
+	media_device_unregister_entity(&func->sink);
+	media_device_unregister_entity(&func->proc);
+	media_device_unregister_entity(&func->vdev.entity);
+}
+
+static int rockchip_vpu_add_enc_func(struct rockchip_vpu_dev *vpu)
 {
 	const struct of_device_id *match;
+	struct rockchip_vpu_func *func;
 	struct video_device *vfd;
-	int function, ret;
+	int ret;
 
 	match = of_match_node(of_rockchip_vpu_match, vpu->dev->of_node);
-	vfd = video_device_alloc();
-	if (!vfd) {
+	func = devm_kzalloc(vpu->dev, sizeof(*func), GFP_KERNEL);
+	if (!func) {
 		v4l2_err(&vpu->v4l2_dev, "Failed to allocate video device\n");
 		return -ENOMEM;
 	}
 
+	func->id = MEDIA_ENT_F_PROC_VIDEO_ENCODER;
+
+	vfd = &func->vdev;
 	vfd->fops = &rockchip_vpu_fops;
-	vfd->release = video_device_release;
+	vfd->release = video_device_release_empty;
 	vfd->lock = &vpu->vpu_mutex;
 	vfd->v4l2_dev = &vpu->v4l2_dev;
 	vfd->vfl_dir = VFL_DIR_M2M;
 	vfd->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE;
 	vfd->ioctl_ops = &rockchip_vpu_enc_ioctl_ops;
 	snprintf(vfd->name, sizeof(vfd->name), "%s-enc", match->compatible);
-	vpu->vfd_enc = vfd;
+
+	vpu->encoder = func;
 	video_set_drvdata(vfd, vpu);
 
 	ret = video_register_device(vfd, VFL_TYPE_GRABBER, -1);
 	if (ret) {
 		v4l2_err(&vpu->v4l2_dev, "Failed to register video device\n");
-		goto err_free_dev;
+		return ret;
 	}
-	v4l2_info(&vpu->v4l2_dev, "registered as /dev/video%d\n", vfd->num);
 
-	function = MEDIA_ENT_F_PROC_VIDEO_ENCODER;
-	ret = v4l2_m2m_register_media_controller(vpu->m2m_dev, vfd, function);
+	ret = rockchip_attach_func(vpu, func);
 	if (ret) {
-		v4l2_err(&vpu->v4l2_dev, "Failed to init mem2mem media controller\n");
-		goto err_unreg_video;
+		v4l2_err(&vpu->v4l2_dev,
+			 "Failed to attach functionality to the media device\n");
+		goto err_unreg_dev;
 	}
+
+	v4l2_info(&vpu->v4l2_dev, "registered as /dev/video%d\n", vfd->num);
+
 	return 0;
 
-err_unreg_video:
+err_unreg_dev:
 	video_unregister_device(vfd);
-err_free_dev:
-	video_device_release(vfd);
 	return ret;
 }
 
+static void rockchip_vpu_remove_enc_func(struct rockchip_vpu_dev *vpu)
+{
+	struct rockchip_vpu_func *func = vpu->encoder;
+
+	if (!func)
+		return;
+
+	rockchip_detach_func(func);
+	video_unregister_device(&func->vdev);
+}
+
 static int rockchip_vpu_probe(struct platform_device *pdev)
 {
 	const struct of_device_id *match;
@@ -464,7 +619,7 @@ static int rockchip_vpu_probe(struct platform_device *pdev)
 	media_device_init(&vpu->mdev);
 	vpu->v4l2_dev.mdev = &vpu->mdev;
 
-	ret = rockchip_vpu_video_device_register(vpu);
+	ret = rockchip_vpu_add_enc_func(vpu);
 	if (ret) {
 		dev_err(&pdev->dev, "Failed to register encoder\n");
 		goto err_m2m_rel;
@@ -473,15 +628,13 @@ static int rockchip_vpu_probe(struct platform_device *pdev)
 	ret = media_device_register(&vpu->mdev);
 	if (ret) {
 		v4l2_err(&vpu->v4l2_dev, "Failed to register mem2mem media device\n");
-		goto err_video_dev_unreg;
+		goto err_rm_enc_func;
 	}
+
 	return 0;
-err_video_dev_unreg:
-	if (vpu->vfd_enc) {
-		v4l2_m2m_unregister_media_controller(vpu->m2m_dev);
-		video_unregister_device(vpu->vfd_enc);
-		video_device_release(vpu->vfd_enc);
-	}
+
+err_rm_enc_func:
+	rockchip_vpu_remove_enc_func(vpu);
 err_m2m_rel:
 	media_device_cleanup(&vpu->mdev);
 	v4l2_m2m_release(vpu->m2m_dev);
@@ -501,11 +654,7 @@ static int rockchip_vpu_remove(struct platform_device *pdev)
 	v4l2_info(&vpu->v4l2_dev, "Removing %s\n", pdev->name);
 
 	media_device_unregister(&vpu->mdev);
-	if (vpu->vfd_enc) {
-		v4l2_m2m_unregister_media_controller(vpu->m2m_dev);
-		video_unregister_device(vpu->vfd_enc);
-		video_device_release(vpu->vfd_enc);
-	}
+	rockchip_vpu_remove_enc_func(vpu);
 	media_device_cleanup(&vpu->mdev);
 	v4l2_m2m_release(vpu->m2m_dev);
 	v4l2_device_unregister(&vpu->v4l2_dev);
-- 
2.20.1

[PATCH v6 06/16] rockchip/vpu: Support the Request API

[Ezequiel Garcia]

Introduce support for the Request API. Although the JPEG encoder
does not mandate using the Request API, it's perfectly possible to
use it, if the application wants to.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
--
Changes from v3:
* None

Changes from v2:
* Drop unrelated mdev.bus_info changes.
* Remove the get_ref and get_ctrl helpers as they are not used yet.

 .../rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c     |  6 ++++++
 .../media/rockchip/vpu/rockchip_vpu_drv.c     |  7 +++++++
 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 19 +++++++++++++++++++
 3 files changed, 32 insertions(+)

diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c
index 6f9f5158d193..74823d25cd8d 100644
--- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c
+++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c
@@ -113,11 +113,15 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx)
 	struct rockchip_vpu_dev *vpu = ctx->dev;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
 	struct rockchip_vpu_jpeg_ctx jpeg_ctx;
+	struct media_request *src_req;
 	u32 reg;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 
+	src_req = src_buf->vb2_buf.req_obj.req;
+	v4l2_ctrl_request_setup(src_req, &ctx->ctrl_handler);
+
 	memset(&jpeg_ctx, 0, sizeof(jpeg_ctx));
 	jpeg_ctx.buffer = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
 	jpeg_ctx.width = ctx->dst_fmt.width;
@@ -153,6 +157,8 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx)
 		| VEPU_REG_ENCODE_FORMAT_JPEG
 		| VEPU_REG_ENCODE_ENABLE;
 
+	v4l2_ctrl_request_complete(src_req, &ctx->ctrl_handler);
+
 	/* Kick the watchdog and start encoding */
 	schedule_delayed_work(&vpu->watchdog_work, msecs_to_jiffies(2000));
 	vepu_write(vpu, reg, VEPU_REG_ENCODE_START);
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index fa02354a0f8a..f47fbd0f9545 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -162,6 +162,7 @@ enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
 	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	src_vq->lock = &ctx->dev->vpu_mutex;
 	src_vq->dev = ctx->dev->v4l2_dev.dev;
+	src_vq->supports_requests = true;
 
 	ret = vb2_queue_init(src_vq);
 	if (ret)
@@ -525,6 +526,11 @@ static void rockchip_vpu_remove_enc_func(struct rockchip_vpu_dev *vpu)
 	video_unregister_device(&func->vdev);
 }
 
+static const struct media_device_ops rockchip_m2m_media_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = v4l2_m2m_request_queue,
+};
+
 static int rockchip_vpu_probe(struct platform_device *pdev)
 {
 	const struct of_device_id *match;
@@ -617,6 +623,7 @@ static int rockchip_vpu_probe(struct platform_device *pdev)
 	strscpy(vpu->mdev.bus_info, "platform: " DRIVER_NAME,
 		sizeof(vpu->mdev.model));
 	media_device_init(&vpu->mdev);
+	vpu->mdev.ops = &rockchip_m2m_media_ops;
 	vpu->v4l2_dev.mdev = &vpu->mdev;
 
 	ret = rockchip_vpu_add_enc_func(vpu);
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
index aa00df9a7ecb..4512e94c3f32 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
@@ -540,14 +540,33 @@ static void rockchip_vpu_stop_streaming(struct vb2_queue *q)
 			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 		if (!vbuf)
 			break;
+		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
+					   &ctx->ctrl_handler);
 		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
 	}
 }
 
+static void rockchip_vpu_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req, &ctx->ctrl_handler);
+}
+
+static int rockchip_vpu_buf_out_validate(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+	vbuf->field = V4L2_FIELD_NONE;
+	return 0;
+}
+
 const struct vb2_ops rockchip_vpu_enc_queue_ops = {
 	.queue_setup = rockchip_vpu_queue_setup,
 	.buf_prepare = rockchip_vpu_buf_prepare,
 	.buf_queue = rockchip_vpu_buf_queue,
+	.buf_out_validate = rockchip_vpu_buf_out_validate,
+	.buf_request_complete = rockchip_vpu_buf_request_complete,
 	.start_streaming = rockchip_vpu_start_streaming,
 	.stop_streaming = rockchip_vpu_stop_streaming,
 	.wait_prepare = vb2_ops_wait_prepare,
-- 
2.20.1

[PATCH v6 07/16] rockchip/vpu: Rename rockchip_vpu_common.h into rockchip_vpu_v4l2.h

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

We're about to add prototypes for the vb2/v4l2 helpers shared by the
encoder/decoder logic in this file, so let's pick a name that reflects
that (rockchip_vpu_common.h was a bit to generic).

Suggested-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v3:
* None

Changes from v2:
* New patch

 drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c | 2 +-
 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c | 2 +-
 drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c       | 2 +-
 drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c       | 2 +-
 .../vpu/{rockchip_vpu_common.h => rockchip_vpu_v4l2.h}      | 6 +++---
 5 files changed, 7 insertions(+), 7 deletions(-)
 rename drivers/staging/media/rockchip/vpu/{rockchip_vpu_common.h => rockchip_vpu_v4l2.h} (88%)

diff --git a/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c
index 791353ae01e7..68176e91330a 100644
--- a/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c
+++ b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c
@@ -9,7 +9,7 @@
 #include <media/v4l2-mem2mem.h>
 #include "rockchip_vpu_jpeg.h"
 #include "rockchip_vpu.h"
-#include "rockchip_vpu_common.h"
+#include "rockchip_vpu_v4l2.h"
 #include "rockchip_vpu_hw.h"
 #include "rk3288_vpu_regs.h"
 
diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c
index 74823d25cd8d..460edc5ebe4d 100644
--- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c
+++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c
@@ -27,7 +27,7 @@
 #include <media/v4l2-mem2mem.h>
 #include "rockchip_vpu_jpeg.h"
 #include "rockchip_vpu.h"
-#include "rockchip_vpu_common.h"
+#include "rockchip_vpu_v4l2.h"
 #include "rockchip_vpu_hw.h"
 #include "rk3399_vpu_regs.h"
 
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index f47fbd0f9545..59b72245fb07 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -24,7 +24,7 @@
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>
 
-#include "rockchip_vpu_common.h"
+#include "rockchip_vpu_v4l2.h"
 #include "rockchip_vpu.h"
 #include "rockchip_vpu_hw.h"
 
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
index 4512e94c3f32..d2b4225516b5 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
@@ -28,7 +28,7 @@
 
 #include "rockchip_vpu.h"
 #include "rockchip_vpu_hw.h"
-#include "rockchip_vpu_common.h"
+#include "rockchip_vpu_v4l2.h"
 
 static const struct rockchip_vpu_fmt *
 rockchip_vpu_find_format(struct rockchip_vpu_ctx *ctx, u32 fourcc)
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_common.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
similarity index 88%
rename from drivers/staging/media/rockchip/vpu/rockchip_vpu_common.h
rename to drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
index ca77668d9579..50ad40dfb4f4 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_common.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
@@ -13,8 +13,8 @@
  * Copyright (C) 2011 Samsung Electronics Co., Ltd.
  */
 
-#ifndef ROCKCHIP_VPU_COMMON_H_
-#define ROCKCHIP_VPU_COMMON_H_
+#ifndef ROCKCHIP_VPU_V4L2_H_
+#define ROCKCHIP_VPU_V4L2_H_
 
 #include "rockchip_vpu.h"
 
@@ -26,4 +26,4 @@ void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
 void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx);
 
-#endif /* ROCKCHIP_VPU_COMMON_H_ */
+#endif /* ROCKCHIP_VPU_V4L2_H_ */
-- 
2.20.1

[PATCH v6 08/16] rockchip/vpu: Move encoder logic to a common place

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

The V4L2/VB2 implementation for the encoder and decoder logic are very
similar, so let's rename rockchip_vpu_enc.c file into
rockchip_vpu_v4l2.c and remove the _enc_ part in objects/functions
exposed in rockchip_vpu_v4l2.h. We also rename the enc_queue_init()
function (in rockchip_vpu_drv.c) queue_init() since it will be used
to initialize both type of queues.

The implementation itself will be patched to support the decoding case
when decoder support is added.

Suggested-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v3:
* None

Changes from v2:
* New patch

 drivers/staging/media/rockchip/vpu/Makefile        |  2 +-
 .../staging/media/rockchip/vpu/rockchip_vpu_drv.c  | 14 +++++++-------
 .../{rockchip_vpu_enc.c => rockchip_vpu_v4l2.c}    | 14 +++++++-------
 .../staging/media/rockchip/vpu/rockchip_vpu_v4l2.h | 12 ++++++------
 4 files changed, 21 insertions(+), 21 deletions(-)
 rename drivers/staging/media/rockchip/vpu/{rockchip_vpu_enc.c => rockchip_vpu_v4l2.c} (97%)

diff --git a/drivers/staging/media/rockchip/vpu/Makefile b/drivers/staging/media/rockchip/vpu/Makefile
index ae5d143a0bfa..33606391e910 100644
--- a/drivers/staging/media/rockchip/vpu/Makefile
+++ b/drivers/staging/media/rockchip/vpu/Makefile
@@ -3,7 +3,7 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VPU) += rockchip-vpu.o
 
 rockchip-vpu-y += \
 		rockchip_vpu_drv.o \
-		rockchip_vpu_enc.o \
+		rockchip_vpu_v4l2.o \
 		rk3288_vpu_hw.o \
 		rk3288_vpu_hw_jpeg_enc.o \
 		rk3399_vpu_hw.o \
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index 59b72245fb07..ec18578d55d7 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -140,7 +140,7 @@ static struct v4l2_m2m_ops vpu_m2m_ops = {
 };
 
 static int
-enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
+queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
 {
 	struct rockchip_vpu_ctx *ctx = priv;
 	int ret;
@@ -148,7 +148,7 @@ enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
 	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
 	src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
 	src_vq->drv_priv = ctx;
-	src_vq->ops = &rockchip_vpu_enc_queue_ops;
+	src_vq->ops = &rockchip_vpu_queue_ops;
 	src_vq->mem_ops = &vb2_dma_contig_memops;
 
 	/*
@@ -179,7 +179,7 @@ enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
 	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 	dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
 	dst_vq->drv_priv = ctx;
-	dst_vq->ops = &rockchip_vpu_enc_queue_ops;
+	dst_vq->ops = &rockchip_vpu_queue_ops;
 	dst_vq->mem_ops = &vb2_vmalloc_memops;
 	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
@@ -260,7 +260,7 @@ static int rockchip_vpu_open(struct file *filp)
 	ctx->dev = vpu;
 	if (func->id == MEDIA_ENT_F_PROC_VIDEO_ENCODER)
 		ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(vpu->m2m_dev, ctx,
-						    &enc_queue_init);
+						    queue_init);
 	else
 		ctx->fh.m2m_ctx = ERR_PTR(-ENODEV);
 	if (IS_ERR(ctx->fh.m2m_ctx)) {
@@ -273,8 +273,8 @@ static int rockchip_vpu_open(struct file *filp)
 	filp->private_data = &ctx->fh;
 	v4l2_fh_add(&ctx->fh);
 
-	rockchip_vpu_enc_reset_dst_fmt(vpu, ctx);
-	rockchip_vpu_enc_reset_src_fmt(vpu, ctx);
+	rockchip_vpu_reset_dst_fmt(vpu, ctx);
+	rockchip_vpu_reset_src_fmt(vpu, ctx);
 
 	ret = rockchip_vpu_ctrls_setup(vpu, ctx);
 	if (ret) {
@@ -487,7 +487,7 @@ static int rockchip_vpu_add_enc_func(struct rockchip_vpu_dev *vpu)
 	vfd->v4l2_dev = &vpu->v4l2_dev;
 	vfd->vfl_dir = VFL_DIR_M2M;
 	vfd->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE;
-	vfd->ioctl_ops = &rockchip_vpu_enc_ioctl_ops;
+	vfd->ioctl_ops = &rockchip_vpu_ioctl_ops;
 	snprintf(vfd->name, sizeof(vfd->name), "%s-enc", match->compatible);
 
 	vpu->encoder = func;
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
similarity index 97%
rename from drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
rename to drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
index d2b4225516b5..3e8f6256e0ed 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
@@ -250,8 +250,8 @@ vidioc_try_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	return 0;
 }
 
-void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
-				    struct rockchip_vpu_ctx *ctx)
+void rockchip_vpu_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
+				struct rockchip_vpu_ctx *ctx)
 {
 	struct v4l2_pix_format_mplane *fmt = &ctx->dst_fmt;
 
@@ -273,8 +273,8 @@ void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
 		fmt->width * fmt->height * ctx->vpu_dst_fmt->max_depth;
 }
 
-void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
-				    struct rockchip_vpu_ctx *ctx)
+void rockchip_vpu_reset_src_fmt(struct rockchip_vpu_dev *vpu,
+				struct rockchip_vpu_ctx *ctx)
 {
 	struct v4l2_pix_format_mplane *fmt = &ctx->src_fmt;
 
@@ -373,11 +373,11 @@ vidioc_s_fmt_cap_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	 * the raw format again after we return, so we don't need
 	 * anything smarter.
 	 */
-	rockchip_vpu_enc_reset_src_fmt(vpu, ctx);
+	rockchip_vpu_reset_src_fmt(vpu, ctx);
 	return 0;
 }
 
-const struct v4l2_ioctl_ops rockchip_vpu_enc_ioctl_ops = {
+const struct v4l2_ioctl_ops rockchip_vpu_ioctl_ops = {
 	.vidioc_querycap = vidioc_querycap,
 	.vidioc_enum_framesizes = vidioc_enum_framesizes,
 
@@ -561,7 +561,7 @@ static int rockchip_vpu_buf_out_validate(struct vb2_buffer *vb)
 	return 0;
 }
 
-const struct vb2_ops rockchip_vpu_enc_queue_ops = {
+const struct vb2_ops rockchip_vpu_queue_ops = {
 	.queue_setup = rockchip_vpu_queue_setup,
 	.buf_prepare = rockchip_vpu_buf_prepare,
 	.buf_queue = rockchip_vpu_buf_queue,
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
index 50ad40dfb4f4..816bd3988218 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
@@ -18,12 +18,12 @@
 
 #include "rockchip_vpu.h"
 
-extern const struct v4l2_ioctl_ops rockchip_vpu_enc_ioctl_ops;
-extern const struct vb2_ops rockchip_vpu_enc_queue_ops;
+extern const struct v4l2_ioctl_ops rockchip_vpu_ioctl_ops;
+extern const struct vb2_ops rockchip_vpu_queue_ops;
 
-void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
-				    struct rockchip_vpu_ctx *ctx);
-void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
-				    struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_reset_src_fmt(struct rockchip_vpu_dev *vpu,
+				struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
+				struct rockchip_vpu_ctx *ctx);
 
 #endif /* ROCKCHIP_VPU_V4L2_H_ */
-- 
2.20.1

[PATCH v6 09/16] rockchip/vpu: Provide a helper to reset both src and dst formats

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

When initializing a context, the core wants to reset both src and dst
formats. Right now the order doesn't matter, but if we want to have a
valid default width/height on the non-coded/raw format side (src in
case of encoders, dst in case of decoders), we need to reset those
formats in the right order: first the coded-format side, then the other,
such that width and height on the raw format side can be taken from
the coded format.

Let's provide a helper that will reset both formats and make sure this
is done in the right order.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v2:
* None

Changes from v2:
* New patch

 .../staging/media/rockchip/vpu/rockchip_vpu_drv.c  |  3 +--
 .../staging/media/rockchip/vpu/rockchip_vpu_v4l2.c | 14 ++++++++++----
 .../staging/media/rockchip/vpu/rockchip_vpu_v4l2.h |  5 +----
 3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index ec18578d55d7..d85b88067b03 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -273,8 +273,7 @@ static int rockchip_vpu_open(struct file *filp)
 	filp->private_data = &ctx->fh;
 	v4l2_fh_add(&ctx->fh);
 
-	rockchip_vpu_reset_dst_fmt(vpu, ctx);
-	rockchip_vpu_reset_src_fmt(vpu, ctx);
+	rockchip_vpu_reset_fmts(ctx);
 
 	ret = rockchip_vpu_ctrls_setup(vpu, ctx);
 	if (ret) {
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
index 3e8f6256e0ed..e30056dc6758 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
@@ -250,8 +250,8 @@ vidioc_try_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	return 0;
 }
 
-void rockchip_vpu_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
-				struct rockchip_vpu_ctx *ctx)
+static void rockchip_vpu_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
+				       struct rockchip_vpu_ctx *ctx)
 {
 	struct v4l2_pix_format_mplane *fmt = &ctx->dst_fmt;
 
@@ -273,8 +273,8 @@ void rockchip_vpu_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
 		fmt->width * fmt->height * ctx->vpu_dst_fmt->max_depth;
 }
 
-void rockchip_vpu_reset_src_fmt(struct rockchip_vpu_dev *vpu,
-				struct rockchip_vpu_ctx *ctx)
+static void rockchip_vpu_reset_src_fmt(struct rockchip_vpu_dev *vpu,
+				       struct rockchip_vpu_ctx *ctx)
 {
 	struct v4l2_pix_format_mplane *fmt = &ctx->src_fmt;
 
@@ -294,6 +294,12 @@ void rockchip_vpu_reset_src_fmt(struct rockchip_vpu_dev *vpu,
 			    fmt->height);
 }
 
+void rockchip_vpu_reset_fmts(struct rockchip_vpu_ctx *ctx)
+{
+	rockchip_vpu_reset_dst_fmt(ctx->dev, ctx);
+	rockchip_vpu_reset_src_fmt(ctx->dev, ctx);
+}
+
 static int
 vidioc_s_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 {
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
index 816bd3988218..493e8751d22d 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.h
@@ -21,9 +21,6 @@
 extern const struct v4l2_ioctl_ops rockchip_vpu_ioctl_ops;
 extern const struct vb2_ops rockchip_vpu_queue_ops;
 
-void rockchip_vpu_reset_src_fmt(struct rockchip_vpu_dev *vpu,
-				struct rockchip_vpu_ctx *ctx);
-void rockchip_vpu_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
-				struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_reset_fmts(struct rockchip_vpu_ctx *ctx);
 
 #endif /* ROCKCHIP_VPU_V4L2_H_ */
-- 
2.20.1

[PATCH v6 10/16] rockchip/vpu: Prepare things to support decoders

[Ezequiel Garcia]

From: Boris Brezillon <boris.brezillon@collabora.com>

The code in rockchip_vpu_v4l2 was hardcoded for encoder support.
Modify it more generic to support the decoder case so that we can
re-use the same vb2/v4l2 ops for both devices.

Co-developed-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v4:
* Fix s_fmt() implems WRT to format properties propagation
* s/fmt->pix/fmt->pix_mp/ in try_fmt() (Reported by Hans)

Changes from v3:
* None

Changes from v2:
* New patch

 .../staging/media/rockchip/vpu/rockchip_vpu.h |   6 +
 .../media/rockchip/vpu/rockchip_vpu_v4l2.c    | 491 +++++++++++-------
 2 files changed, 295 insertions(+), 202 deletions(-)

diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
index aba257c663a7..0d24fd257a2b 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
@@ -262,4 +262,10 @@ static inline u32 vepu_read(struct rockchip_vpu_dev *vpu, u32 reg)
 	return val;
 }
 
+static inline bool
+rockchip_vpu_is_encoder_ctx(const struct rockchip_vpu_ctx *ctx)
+{
+	return true;
+}
+
 #endif /* ROCKCHIP_VPU_H_ */
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
index e30056dc6758..1ab558d6492d 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
@@ -31,14 +31,23 @@
 #include "rockchip_vpu_v4l2.h"
 
 static const struct rockchip_vpu_fmt *
-rockchip_vpu_find_format(struct rockchip_vpu_ctx *ctx, u32 fourcc)
+rockchip_vpu_get_formats(const struct rockchip_vpu_ctx *ctx,
+			 unsigned int *num_fmts)
 {
-	struct rockchip_vpu_dev *dev = ctx->dev;
 	const struct rockchip_vpu_fmt *formats;
-	unsigned int num_fmts, i;
 
-	formats = dev->variant->enc_fmts;
-	num_fmts = dev->variant->num_enc_fmts;
+	formats = ctx->dev->variant->enc_fmts;
+	*num_fmts = ctx->dev->variant->num_enc_fmts;
+
+	return formats;
+}
+
+static const struct rockchip_vpu_fmt *
+rockchip_vpu_find_format(const struct rockchip_vpu_fmt *formats,
+			 unsigned int num_fmts, u32 fourcc)
+{
+	unsigned int i;
+
 	for (i = 0; i < num_fmts; i++)
 		if (formats[i].fourcc == fourcc)
 			return &formats[i];
@@ -46,14 +55,11 @@ rockchip_vpu_find_format(struct rockchip_vpu_ctx *ctx, u32 fourcc)
 }
 
 static const struct rockchip_vpu_fmt *
-rockchip_vpu_get_default_fmt(struct rockchip_vpu_ctx *ctx, bool bitstream)
+rockchip_vpu_get_default_fmt(const struct rockchip_vpu_fmt *formats,
+			     unsigned int num_fmts, bool bitstream)
 {
-	struct rockchip_vpu_dev *dev = ctx->dev;
-	const struct rockchip_vpu_fmt *formats;
-	unsigned int num_fmts, i;
+	unsigned int i;
 
-	formats = dev->variant->enc_fmts;
-	num_fmts = dev->variant->num_enc_fmts;
 	for (i = 0; i < num_fmts; i++) {
 		if (bitstream == (formats[i].codec_mode != RK_VPU_MODE_NONE))
 			return &formats[i];
@@ -78,7 +84,8 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
 				  struct v4l2_frmsizeenum *fsize)
 {
 	struct rockchip_vpu_ctx *ctx = fh_to_ctx(priv);
-	const struct rockchip_vpu_fmt *fmt;
+	const struct rockchip_vpu_fmt *formats, *fmt;
+	unsigned int num_fmts;
 
 	if (fsize->index != 0) {
 		vpu_debug(0, "invalid frame size index (expected 0, got %d)\n",
@@ -86,7 +93,8 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
 		return -EINVAL;
 	}
 
-	fmt = rockchip_vpu_find_format(ctx, fsize->pixel_format);
+	formats = rockchip_vpu_get_formats(ctx, &num_fmts);
+	fmt = rockchip_vpu_find_format(formats, num_fmts, fsize->pixel_format);
 	if (!fmt) {
 		vpu_debug(0, "unsupported bitstream format (%08x)\n",
 			  fsize->pixel_format);
@@ -103,19 +111,32 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
 	return 0;
 }
 
-static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *priv,
-					  struct v4l2_fmtdesc *f)
+static int vidioc_enum_fmt(struct file *file, void *priv,
+			   struct v4l2_fmtdesc *f, bool capture)
+
 {
-	struct rockchip_vpu_dev *dev = video_drvdata(file);
-	const struct rockchip_vpu_fmt *fmt;
-	const struct rockchip_vpu_fmt *formats;
-	int num_fmts, i, j = 0;
+	struct rockchip_vpu_ctx *ctx = fh_to_ctx(priv);
+	const struct rockchip_vpu_fmt *fmt, *formats;
+	unsigned int num_fmts, i, j = 0;
+	bool skip_mode_none;
 
-	formats = dev->variant->enc_fmts;
-	num_fmts = dev->variant->num_enc_fmts;
+	/*
+	 * When dealing with an encoder:
+	 *  - on the capture side we want to filter out all MODE_NONE formats.
+	 *  - on the output side we want to filter out all formats that are
+	 *    not MODE_NONE.
+	 * When dealing with a decoder:
+	 *  - on the capture side we want to filter out all formats that are
+	 *    not MODE_NONE.
+	 *  - on the output side we want to filter out all MODE_NONE formats.
+	 */
+	skip_mode_none = capture == rockchip_vpu_is_encoder_ctx(ctx);
+
+	formats = rockchip_vpu_get_formats(ctx, &num_fmts);
 	for (i = 0; i < num_fmts; i++) {
-		/* Skip uncompressed formats */
-		if (formats[i].codec_mode == RK_VPU_MODE_NONE)
+		bool mode_none = formats[i].codec_mode == RK_VPU_MODE_NONE;
+
+		if (skip_mode_none == mode_none)
 			continue;
 		if (j == f->index) {
 			fmt = &formats[i];
@@ -127,27 +148,16 @@ static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *priv,
 	return -EINVAL;
 }
 
-static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *priv,
+static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *priv,
 					  struct v4l2_fmtdesc *f)
 {
-	struct rockchip_vpu_dev *dev = video_drvdata(file);
-	const struct rockchip_vpu_fmt *formats;
-	const struct rockchip_vpu_fmt *fmt;
-	int num_fmts, i, j = 0;
+	return vidioc_enum_fmt(file, priv, f, true);
+}
 
-	formats = dev->variant->enc_fmts;
-	num_fmts = dev->variant->num_enc_fmts;
-	for (i = 0; i < num_fmts; i++) {
-		if (formats[i].codec_mode != RK_VPU_MODE_NONE)
-			continue;
-		if (j == f->index) {
-			fmt = &formats[i];
-			f->pixelformat = fmt->fourcc;
-			return 0;
-		}
-		++j;
-	}
-	return -EINVAL;
+static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *priv,
+					  struct v4l2_fmtdesc *f)
+{
+	return vidioc_enum_fmt(file, priv, f, false);
 }
 
 static int vidioc_g_fmt_out_mplane(struct file *file, void *priv,
@@ -176,128 +186,149 @@ static int vidioc_g_fmt_cap_mplane(struct file *file, void *priv,
 	return 0;
 }
 
-static int
-vidioc_try_fmt_cap_mplane(struct file *file, void *priv, struct v4l2_format *f)
+static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f,
+			  bool capture)
 {
 	struct rockchip_vpu_ctx *ctx = fh_to_ctx(priv);
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	const struct rockchip_vpu_fmt *fmt;
+	const struct rockchip_vpu_fmt *formats, *fmt, *vpu_fmt;
+	unsigned int num_fmts;
+	bool coded;
 
-	vpu_debug(4, "%c%c%c%c\n",
+	coded = capture == rockchip_vpu_is_encoder_ctx(ctx);
+
+	vpu_debug(4, "trying format %c%c%c%c\n",
 		  (pix_mp->pixelformat & 0x7f),
 		  (pix_mp->pixelformat >> 8) & 0x7f,
 		  (pix_mp->pixelformat >> 16) & 0x7f,
 		  (pix_mp->pixelformat >> 24) & 0x7f);
 
-	fmt = rockchip_vpu_find_format(ctx, pix_mp->pixelformat);
+	formats = rockchip_vpu_get_formats(ctx, &num_fmts);
+	fmt = rockchip_vpu_find_format(formats, num_fmts, pix_mp->pixelformat);
 	if (!fmt) {
-		fmt = rockchip_vpu_get_default_fmt(ctx, true);
-		f->fmt.pix.pixelformat = fmt->fourcc;
+		fmt = rockchip_vpu_get_default_fmt(formats, num_fmts, coded);
+		f->fmt.pix_mp.pixelformat = fmt->fourcc;
+	}
+
+	if (coded) {
+		pix_mp->num_planes = 1;
+		vpu_fmt = fmt;
+	} else if (rockchip_vpu_is_encoder_ctx(ctx)) {
+		vpu_fmt = ctx->vpu_dst_fmt;
+	} else {
+		vpu_fmt = ctx->vpu_src_fmt;
+		/*
+		 * Width/height on the CAPTURE end of a decoder are ignored and
+		 * replaced by the OUTPUT ones.
+		 */
+		pix_mp->width = ctx->src_fmt.width;
+		pix_mp->height = ctx->src_fmt.height;
 	}
 
-	pix_mp->num_planes = 1;
 	pix_mp->field = V4L2_FIELD_NONE;
 
 	v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height,
-				       &fmt->frmsize);
-
-	/*
-	 * For compressed formats the application can specify
-	 * sizeimage. If the application passes a zero sizeimage,
-	 * let's default to the maximum frame size.
-	 */
-	if (!pix_mp->plane_fmt[0].sizeimage)
+				       &vpu_fmt->frmsize);
+
+	if (!coded) {
+		/* Fill remaining fields */
+		v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, pix_mp->width,
+				    pix_mp->height);
+	} else if (!pix_mp->plane_fmt[0].sizeimage) {
+		/*
+		 * For coded formats the application can specify
+		 * sizeimage. If the application passes a zero sizeimage,
+		 * let's default to the maximum frame size.
+		 */
 		pix_mp->plane_fmt[0].sizeimage = fmt->header_size +
 			pix_mp->width * pix_mp->height * fmt->max_depth;
-	memset(pix_mp->plane_fmt[0].reserved, 0,
-	       sizeof(pix_mp->plane_fmt[0].reserved));
-	return 0;
-}
-
-static int
-vidioc_try_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
-{
-	struct rockchip_vpu_ctx *ctx = fh_to_ctx(priv);
-	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
-	const struct rockchip_vpu_fmt *fmt;
-	int i;
-
-	vpu_debug(4, "%c%c%c%c\n",
-		  (pix_mp->pixelformat & 0x7f),
-		  (pix_mp->pixelformat >> 8) & 0x7f,
-		  (pix_mp->pixelformat >> 16) & 0x7f,
-		  (pix_mp->pixelformat >> 24) & 0x7f);
-
-	fmt = rockchip_vpu_find_format(ctx, pix_mp->pixelformat);
-	if (!fmt) {
-		fmt = rockchip_vpu_get_default_fmt(ctx, false);
-		f->fmt.pix.pixelformat = fmt->fourcc;
 	}
 
-	pix_mp->field = V4L2_FIELD_NONE;
-
-	v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height,
-				       &ctx->vpu_dst_fmt->frmsize);
-
-	/* Fill remaining fields */
-	v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, pix_mp->width,
-			    pix_mp->height);
-
-	for (i = 0; i < pix_mp->num_planes; i++) {
-		memset(pix_mp->plane_fmt[i].reserved, 0,
-		       sizeof(pix_mp->plane_fmt[i].reserved));
-	}
 	return 0;
 }
 
-static void rockchip_vpu_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
-				       struct rockchip_vpu_ctx *ctx)
+static int vidioc_try_fmt_cap_mplane(struct file *file, void *priv,
+				     struct v4l2_format *f)
 {
-	struct v4l2_pix_format_mplane *fmt = &ctx->dst_fmt;
+	return vidioc_try_fmt(file, priv, f, true);
+}
 
-	ctx->vpu_dst_fmt = rockchip_vpu_get_default_fmt(ctx, true);
+static int vidioc_try_fmt_out_mplane(struct file *file, void *priv,
+				     struct v4l2_format *f)
+{
+	return vidioc_try_fmt(file, priv, f, false);
+}
 
+static void
+rockchip_vpu_reset_fmt(struct v4l2_pix_format_mplane *fmt,
+		       const struct rockchip_vpu_fmt *vpu_fmt)
+{
 	memset(fmt, 0, sizeof(*fmt));
 
-	fmt->num_planes = 1;
-	v4l2_apply_frmsize_constraints(&fmt->width, &fmt->height,
-				       &ctx->vpu_dst_fmt->frmsize);
-	fmt->pixelformat = ctx->vpu_dst_fmt->fourcc;
+	fmt->pixelformat = vpu_fmt->fourcc;
 	fmt->field = V4L2_FIELD_NONE;
 	fmt->colorspace = V4L2_COLORSPACE_JPEG,
 	fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
 	fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
-
-	fmt->plane_fmt[0].sizeimage = ctx->vpu_dst_fmt->header_size +
-		fmt->width * fmt->height * ctx->vpu_dst_fmt->max_depth;
 }
 
-static void rockchip_vpu_reset_src_fmt(struct rockchip_vpu_dev *vpu,
-				       struct rockchip_vpu_ctx *ctx)
+static void
+rockchip_vpu_reset_encoded_fmt(struct rockchip_vpu_ctx *ctx)
 {
-	struct v4l2_pix_format_mplane *fmt = &ctx->src_fmt;
-
-	ctx->vpu_src_fmt = rockchip_vpu_get_default_fmt(ctx, false);
+	const struct rockchip_vpu_fmt *vpu_fmt, *formats;
+	struct v4l2_pix_format_mplane *fmt;
+	unsigned int num_fmts;
+
+	formats = rockchip_vpu_get_formats(ctx, &num_fmts);
+	vpu_fmt = rockchip_vpu_get_default_fmt(formats, num_fmts, true);
+
+	if (rockchip_vpu_is_encoder_ctx(ctx)) {
+		ctx->vpu_dst_fmt = vpu_fmt;
+		fmt = &ctx->dst_fmt;
+	} else {
+		ctx->vpu_src_fmt = vpu_fmt;
+		fmt = &ctx->src_fmt;
+	}
 
-	memset(fmt, 0, sizeof(*fmt));
+	rockchip_vpu_reset_fmt(fmt, vpu_fmt);
+	fmt->num_planes = 1;
+	fmt->width = vpu_fmt->frmsize.min_width;
+	fmt->height = vpu_fmt->frmsize.min_height;
+	fmt->plane_fmt[0].sizeimage = vpu_fmt->header_size +
+				fmt->width * fmt->height * vpu_fmt->max_depth;
+}
 
-	v4l2_apply_frmsize_constraints(&fmt->width, &fmt->height,
-				       &ctx->vpu_src_fmt->frmsize);
-	fmt->field = V4L2_FIELD_NONE;
-	fmt->colorspace = V4L2_COLORSPACE_JPEG,
-	fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
-	fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
-	fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+static void
+rockchip_vpu_reset_raw_fmt(struct rockchip_vpu_ctx *ctx)
+{
+	const struct rockchip_vpu_fmt *raw_vpu_fmt, *formats;
+	struct v4l2_pix_format_mplane *raw_fmt, *encoded_fmt;
+	unsigned int num_fmts;
+
+	formats = rockchip_vpu_get_formats(ctx, &num_fmts);
+	raw_vpu_fmt = rockchip_vpu_get_default_fmt(formats, num_fmts, false);
+
+	if (rockchip_vpu_is_encoder_ctx(ctx)) {
+		ctx->vpu_src_fmt = raw_vpu_fmt;
+		raw_fmt = &ctx->src_fmt;
+		encoded_fmt = &ctx->dst_fmt;
+	} else {
+		ctx->vpu_dst_fmt = raw_vpu_fmt;
+		raw_fmt = &ctx->dst_fmt;
+		encoded_fmt = &ctx->src_fmt;
+	}
 
-	v4l2_fill_pixfmt_mp(fmt, ctx->vpu_src_fmt->fourcc, fmt->width,
-			    fmt->height);
+	rockchip_vpu_reset_fmt(raw_fmt, raw_vpu_fmt);
+	v4l2_fill_pixfmt_mp(raw_fmt, raw_vpu_fmt->fourcc,
+			    encoded_fmt->width,
+			    encoded_fmt->height);
 }
 
 void rockchip_vpu_reset_fmts(struct rockchip_vpu_ctx *ctx)
 {
-	rockchip_vpu_reset_dst_fmt(ctx->dev, ctx);
-	rockchip_vpu_reset_src_fmt(ctx->dev, ctx);
+	rockchip_vpu_reset_encoded_fmt(ctx);
+	rockchip_vpu_reset_raw_fmt(ctx);
 }
 
 static int
@@ -305,28 +336,56 @@ vidioc_s_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct rockchip_vpu_ctx *ctx = fh_to_ctx(priv);
+	const struct rockchip_vpu_fmt *formats;
+	unsigned int num_fmts;
 	struct vb2_queue *vq;
 	int ret;
 
-	/* Change not allowed if queue is streaming. */
+	/* Change not allowed if queue is busy. */
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
-	if (vb2_is_streaming(vq))
+	if (vb2_is_busy(vq))
 		return -EBUSY;
 
+	if (!rockchip_vpu_is_encoder_ctx(ctx)) {
+		struct vb2_queue *peer_vq;
+
+		/*
+		 * Since format change on the OUTPUT queue will reset
+		 * the CAPTURE queue, we can't allow doing so
+		 * when the CAPTURE queue has buffers allocated.
+		 */
+		peer_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+					  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+		if (vb2_is_busy(peer_vq))
+			return -EBUSY;
+	}
+
 	ret = vidioc_try_fmt_out_mplane(file, priv, f);
 	if (ret)
 		return ret;
 
-	ctx->vpu_src_fmt = rockchip_vpu_find_format(ctx, pix_mp->pixelformat);
+	formats = rockchip_vpu_get_formats(ctx, &num_fmts);
+	ctx->vpu_src_fmt = rockchip_vpu_find_format(formats, num_fmts,
+						    pix_mp->pixelformat);
 	ctx->src_fmt = *pix_mp;
 
-	/* Propagate to the CAPTURE format */
+	/*
+	 * Current raw format might have become invalid with newly
+	 * selected codec, so reset it to default just to be safe and
+	 * keep internal driver state sane. User is mandated to set
+	 * the raw format again after we return, so we don't need
+	 * anything smarter.
+	 * Note that rockchip_vpu_reset_raw_fmt() also propagates size
+	 * changes to the raw format.
+	 */
+	if (!rockchip_vpu_is_encoder_ctx(ctx))
+		rockchip_vpu_reset_raw_fmt(ctx);
+
+	/* Colorimetry information are always propagated. */
 	ctx->dst_fmt.colorspace = pix_mp->colorspace;
 	ctx->dst_fmt.ycbcr_enc = pix_mp->ycbcr_enc;
 	ctx->dst_fmt.xfer_func = pix_mp->xfer_func;
 	ctx->dst_fmt.quantization = pix_mp->quantization;
-	ctx->dst_fmt.width = pix_mp->width;
-	ctx->dst_fmt.height = pix_mp->height;
 
 	vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode);
 	vpu_debug(0, "fmt - w: %d, h: %d\n",
@@ -334,52 +393,69 @@ vidioc_s_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	return 0;
 }
 
-static int
-vidioc_s_fmt_cap_mplane(struct file *file, void *priv, struct v4l2_format *f)
+static int vidioc_s_fmt_cap_mplane(struct file *file, void *priv,
+				   struct v4l2_format *f)
 {
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct rockchip_vpu_ctx *ctx = fh_to_ctx(priv);
-	struct rockchip_vpu_dev *vpu = ctx->dev;
-	struct vb2_queue *vq, *peer_vq;
+	const struct rockchip_vpu_fmt *formats;
+	struct vb2_queue *vq;
+	unsigned int num_fmts;
 	int ret;
 
-	/* Change not allowed if queue is streaming. */
+	/* Change not allowed if queue is busy. */
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
-	if (vb2_is_streaming(vq))
+	if (vb2_is_busy(vq))
 		return -EBUSY;
 
-	/*
-	 * Since format change on the CAPTURE queue will reset
-	 * the OUTPUT queue, we can't allow doing so
-	 * when the OUTPUT queue has buffers allocated.
-	 */
-	peer_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
-				  V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
-	if (vb2_is_busy(peer_vq) &&
-	    (pix_mp->pixelformat != ctx->dst_fmt.pixelformat ||
-	     pix_mp->height != ctx->dst_fmt.height ||
-	     pix_mp->width != ctx->dst_fmt.width))
-		return -EBUSY;
+	if (rockchip_vpu_is_encoder_ctx(ctx)) {
+		struct vb2_queue *peer_vq;
+
+		/*
+		 * Since format change on the CAPTURE queue will reset
+		 * the OUTPUT queue, we can't allow doing so
+		 * when the OUTPUT queue has buffers allocated.
+		 */
+		peer_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+					  V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+		if (vb2_is_busy(peer_vq) &&
+		    (pix_mp->pixelformat != ctx->dst_fmt.pixelformat ||
+		     pix_mp->height != ctx->dst_fmt.height ||
+		     pix_mp->width != ctx->dst_fmt.width))
+			return -EBUSY;
+	}
 
 	ret = vidioc_try_fmt_cap_mplane(file, priv, f);
 	if (ret)
 		return ret;
 
-	ctx->vpu_dst_fmt = rockchip_vpu_find_format(ctx, pix_mp->pixelformat);
+	formats = rockchip_vpu_get_formats(ctx, &num_fmts);
+	ctx->vpu_dst_fmt = rockchip_vpu_find_format(formats, num_fmts,
+						    pix_mp->pixelformat);
 	ctx->dst_fmt = *pix_mp;
 
-	vpu_debug(0, "CAPTURE codec mode: %d\n", ctx->vpu_dst_fmt->codec_mode);
-	vpu_debug(0, "fmt - w: %d, h: %d\n",
-		  pix_mp->width, pix_mp->height);
-
 	/*
 	 * Current raw format might have become invalid with newly
 	 * selected codec, so reset it to default just to be safe and
 	 * keep internal driver state sane. User is mandated to set
 	 * the raw format again after we return, so we don't need
 	 * anything smarter.
+	 * Note that rockchip_vpu_reset_raw_fmt() also propagates size
+	 * changes to the raw format.
 	 */
-	rockchip_vpu_reset_src_fmt(vpu, ctx);
+	if (rockchip_vpu_is_encoder_ctx(ctx))
+		rockchip_vpu_reset_raw_fmt(ctx);
+
+	/* Colorimetry information are always propagated. */
+	ctx->src_fmt.colorspace = pix_mp->colorspace;
+	ctx->src_fmt.ycbcr_enc = pix_mp->ycbcr_enc;
+	ctx->src_fmt.xfer_func = pix_mp->xfer_func;
+	ctx->src_fmt.quantization = pix_mp->quantization;
+
+	vpu_debug(0, "CAPTURE codec mode: %d\n", ctx->vpu_dst_fmt->codec_mode);
+	vpu_debug(0, "fmt - w: %d, h: %d\n",
+		  pix_mp->width, pix_mp->height);
+
 	return 0;
 }
 
@@ -449,48 +525,37 @@ rockchip_vpu_queue_setup(struct vb2_queue *vq,
 	return 0;
 }
 
-static int rockchip_vpu_buf_prepare(struct vb2_buffer *vb)
+static int
+rockchip_vpu_buf_plane_check(struct vb2_buffer *vb,
+			     const struct rockchip_vpu_fmt *vpu_fmt,
+			     struct v4l2_pix_format_mplane *pixfmt)
 {
-	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
-	struct vb2_queue *vq = vb->vb2_queue;
-	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(vq);
-	struct v4l2_pix_format_mplane *pixfmt;
 	unsigned int sz;
-	int ret = 0;
 	int i;
 
-	switch (vq->type) {
-	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
-		pixfmt = &ctx->dst_fmt;
-		break;
-	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
-		pixfmt = &ctx->src_fmt;
-
-		if (vbuf->field == V4L2_FIELD_ANY)
-			vbuf->field = V4L2_FIELD_NONE;
-		if (vbuf->field != V4L2_FIELD_NONE) {
-			vpu_debug(4, "field %d not supported\n",
-				  vbuf->field);
-			return -EINVAL;
-		}
-		break;
-	default:
-		vpu_err("invalid queue type: %d\n", vq->type);
-		return -EINVAL;
-	}
-
 	for (i = 0; i < pixfmt->num_planes; ++i) {
 		sz = pixfmt->plane_fmt[i].sizeimage;
 		vpu_debug(4, "plane %d size: %ld, sizeimage: %u\n",
 			  i, vb2_plane_size(vb, i), sz);
 		if (vb2_plane_size(vb, i) < sz) {
-			vpu_err("plane %d is too small\n", i);
-			ret = -EINVAL;
-			break;
+			vpu_err("plane %d is too small for output\n", i);
+			return -EINVAL;
 		}
 	}
+	return 0;
+}
 
-	return ret;
+static int rockchip_vpu_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(vq);
+
+	if (V4L2_TYPE_IS_OUTPUT(vq->type))
+		return rockchip_vpu_buf_plane_check(vb, ctx->vpu_src_fmt,
+						    &ctx->src_fmt);
+
+	return rockchip_vpu_buf_plane_check(vb, ctx->vpu_dst_fmt,
+					    &ctx->dst_fmt);
 }
 
 static void rockchip_vpu_buf_queue(struct vb2_buffer *vb)
@@ -501,10 +566,17 @@ static void rockchip_vpu_buf_queue(struct vb2_buffer *vb)
 	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
 }
 
-static int rockchip_vpu_start_streaming(struct vb2_queue *q, unsigned int count)
+static bool rockchip_vpu_vq_is_coded(struct vb2_queue *q)
+{
+	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(q);
+
+	return rockchip_vpu_is_encoder_ctx(ctx) != V4L2_TYPE_IS_OUTPUT(q->type);
+}
+
+static int rockchip_vpu_start_streaming(struct vb2_queue *q,
+					unsigned int count)
 {
 	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(q);
-	enum rockchip_vpu_codec_mode codec_mode;
 	int ret = 0;
 
 	if (V4L2_TYPE_IS_OUTPUT(q->type))
@@ -512,38 +584,33 @@ static int rockchip_vpu_start_streaming(struct vb2_queue *q, unsigned int count)
 	else
 		ctx->sequence_cap = 0;
 
-	/* Set codec_ops for the chosen destination format */
-	codec_mode = ctx->vpu_dst_fmt->codec_mode;
+	if (rockchip_vpu_vq_is_coded(q)) {
+		enum rockchip_vpu_codec_mode codec_mode;
 
-	vpu_debug(4, "Codec mode = %d\n", codec_mode);
-	ctx->codec_ops = &ctx->dev->variant->codec_ops[codec_mode];
+		if (V4L2_TYPE_IS_OUTPUT(q->type))
+			codec_mode = ctx->vpu_src_fmt->codec_mode;
+		else
+			codec_mode = ctx->vpu_dst_fmt->codec_mode;
 
-	if (!V4L2_TYPE_IS_OUTPUT(q->type))
+		vpu_debug(4, "Codec mode = %d\n", codec_mode);
+		ctx->codec_ops = &ctx->dev->variant->codec_ops[codec_mode];
 		if (ctx->codec_ops && ctx->codec_ops->init)
 			ret = ctx->codec_ops->init(ctx);
+	}
+
 	return ret;
 }
 
-static void rockchip_vpu_stop_streaming(struct vb2_queue *q)
+static void
+rockchip_vpu_return_bufs(struct vb2_queue *q,
+			 struct vb2_v4l2_buffer *(*buf_remove)(struct v4l2_m2m_ctx *))
 {
 	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(q);
 
-	if (!V4L2_TYPE_IS_OUTPUT(q->type))
-		if (ctx->codec_ops && ctx->codec_ops->exit)
-			ctx->codec_ops->exit(ctx);
-
-	/*
-	 * The mem2mem framework calls v4l2_m2m_cancel_job before
-	 * .stop_streaming, so there isn't any job running and
-	 * it is safe to return all the buffers.
-	 */
 	for (;;) {
 		struct vb2_v4l2_buffer *vbuf;
 
-		if (V4L2_TYPE_IS_OUTPUT(q->type))
-			vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
-		else
-			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+		vbuf = buf_remove(ctx->fh.m2m_ctx);
 		if (!vbuf)
 			break;
 		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
@@ -552,6 +619,26 @@ static void rockchip_vpu_stop_streaming(struct vb2_queue *q)
 	}
 }
 
+static void rockchip_vpu_stop_streaming(struct vb2_queue *q)
+{
+	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(q);
+
+	if (rockchip_vpu_vq_is_coded(q)) {
+		if (ctx->codec_ops && ctx->codec_ops->exit)
+			ctx->codec_ops->exit(ctx);
+	}
+
+	/*
+	 * The mem2mem framework calls v4l2_m2m_cancel_job before
+	 * .stop_streaming, so there isn't any job running and
+	 * it is safe to return all the buffers.
+	 */
+	if (V4L2_TYPE_IS_OUTPUT(q->type))
+		rockchip_vpu_return_bufs(q, v4l2_m2m_src_buf_remove);
+	else
+		rockchip_vpu_return_bufs(q, v4l2_m2m_dst_buf_remove);
+}
+
 static void rockchip_vpu_buf_request_complete(struct vb2_buffer *vb)
 {
 	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
-- 
2.20.1

[PATCH v6 11/16] rockchip/vpu: Add decoder boilerplate

[Ezequiel Garcia]

This commit adds the needed boilerplate code to support the VPU
in decoding operation. Two v4l2 interfaces are exposed, one for
encoding and one for decoding, but a single m2m device is shared
by them, so jobs are properly serialized.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
--
Changes from v4:
* Remove vpu_common.h inclusion
* Set ->requires_requests from s_fmt() instead of fixing it at queue
  creation time (suggested by Hans)

Changes from v3:
* Update things according to changes done in the patch introducing
  custom media controller support (Boris)
* Move the rk3399 vdpu irq definition in the commit introducing
  MPGEG2 support on rk3399
* Fix media controller deregistration code (Jonas)

Changes from v2:
* Use the common vb2/v4l2 implementation
* Use strscpy instead of strlcpy.
* Abstract vidioc v4l2 api implementations into generic code, creating
  helpers that can be used by the encoder and the decoder.
* Only prevent S_FMT on the coded format queue, if the peer queue has buffers allocated.
* Refactor the code, adding a buf_finish callback to rockchip_vpu_ctx.
  With this change, is_enc field is not needed.
* Separate OUTPUT and CAPTURE queue ops (vb2_ops), and create common helpers that
  can be used by both.
* Pass a no kernel mapping attribute on both ends of the decoder.
---
 drivers/staging/media/rockchip/vpu/Kconfig    |   1 +
 .../staging/media/rockchip/vpu/rockchip_vpu.h |  42 ++++-
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 169 ++++++++++++++----
 .../media/rockchip/vpu/rockchip_vpu_v4l2.c    |  29 ++-
 4 files changed, 203 insertions(+), 38 deletions(-)

diff --git a/drivers/staging/media/rockchip/vpu/Kconfig b/drivers/staging/media/rockchip/vpu/Kconfig
index fc54bbf6753d..842b003e08b8 100644
--- a/drivers/staging/media/rockchip/vpu/Kconfig
+++ b/drivers/staging/media/rockchip/vpu/Kconfig
@@ -3,6 +3,7 @@ config VIDEO_ROCKCHIP_VPU
 	tristate "Rockchip VPU driver"
 	depends on ARCH_ROCKCHIP || COMPILE_TEST
 	depends on VIDEO_DEV && VIDEO_V4L2 && MEDIA_CONTROLLER
+	depends on MEDIA_CONTROLLER_REQUEST_API
 	select VIDEOBUF2_DMA_CONTIG
 	select VIDEOBUF2_VMALLOC
 	select V4L2_MEM2MEM_DEV
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
index 0d24fd257a2b..3d64f3e95c9b 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
@@ -40,23 +40,31 @@ struct rockchip_vpu_codec_ops;
  * struct rockchip_vpu_variant - information about VPU hardware variant
  *
  * @enc_offset:			Offset from VPU base to encoder registers.
+ * @dec_offset:			Offset from VPU base to decoder registers.
  * @enc_fmts:			Encoder formats.
  * @num_enc_fmts:		Number of encoder formats.
+ * @dec_fmts:			Decoder formats.
+ * @num_dec_fmts:		Number of decoder formats.
  * @codec:			Supported codecs
  * @codec_ops:			Codec ops.
  * @init:			Initialize hardware.
  * @vepu_irq:			encoder interrupt handler
+ * @vdpu_irq:			decoder interrupt handler
  * @clk_names:			array of clock names
  * @num_clocks:			number of clocks in the array
  */
 struct rockchip_vpu_variant {
 	unsigned int enc_offset;
+	unsigned int dec_offset;
 	const struct rockchip_vpu_fmt *enc_fmts;
 	unsigned int num_enc_fmts;
+	const struct rockchip_vpu_fmt *dec_fmts;
+	unsigned int num_dec_fmts;
 	unsigned int codec;
 	const struct rockchip_vpu_codec_ops *codec_ops;
 	int (*init)(struct rockchip_vpu_dev *vpu);
 	irqreturn_t (*vepu_irq)(int irq, void *priv);
+	irqreturn_t (*vdpu_irq)(int irq, void *priv);
 	const char *clk_names[ROCKCHIP_VPU_MAX_CLOCKS];
 	int num_clocks;
 };
@@ -112,12 +120,14 @@ rockchip_vpu_vdev_to_func(struct video_device *vdev)
  * @m2m_dev:		mem2mem device associated to this device.
  * @mdev:		media device associated to this device.
  * @encoder:		encoder functionality.
+ * @decoder:		decoder functionality.
  * @pdev:		Pointer to VPU platform device.
  * @dev:		Pointer to device for convenient logging using
  *			dev_ macros.
  * @clocks:		Array of clock handles.
  * @base:		Mapped address of VPU registers.
  * @enc_base:		Mapped address of VPU encoder register for convenience.
+ * @dec_base:		Mapped address of VPU decoder register for convenience.
  * @vpu_mutex:		Mutex to synchronize V4L2 calls.
  * @irqlock:		Spinlock to synchronize access to data structures
  *			shared with interrupt handlers.
@@ -129,11 +139,13 @@ struct rockchip_vpu_dev {
 	struct v4l2_m2m_dev *m2m_dev;
 	struct media_device mdev;
 	struct rockchip_vpu_func *encoder;
+	struct rockchip_vpu_func *decoder;
 	struct platform_device *pdev;
 	struct device *dev;
 	struct clk_bulk_data clocks[ROCKCHIP_VPU_MAX_CLOCKS];
 	void __iomem *base;
 	void __iomem *enc_base;
+	void __iomem *dec_base;
 
 	struct mutex vpu_mutex;	/* video_device lock */
 	spinlock_t irqlock;
@@ -158,6 +170,9 @@ struct rockchip_vpu_dev {
  * @ctrl_handler:	Control handler used to register controls.
  * @jpeg_quality:	User-specified JPEG compression quality.
  *
+ * @buf_finish:		Buffer finish. This depends on encoder or decoder
+ *			context, and it's called right before
+ *			calling v4l2_m2m_job_finish.
  * @codec_ops:		Set of operations related to codec mode.
  * @jpeg_enc:		JPEG-encoding context.
  */
@@ -176,6 +191,10 @@ struct rockchip_vpu_ctx {
 	struct v4l2_ctrl_handler ctrl_handler;
 	int jpeg_quality;
 
+	int (*buf_finish)(struct rockchip_vpu_ctx *ctx,
+			  struct vb2_buffer *buf,
+			  unsigned int bytesused);
+
 	const struct rockchip_vpu_codec_ops *codec_ops;
 
 	/* Specific for particular codec modes. */
@@ -262,10 +281,27 @@ static inline u32 vepu_read(struct rockchip_vpu_dev *vpu, u32 reg)
 	return val;
 }
 
-static inline bool
-rockchip_vpu_is_encoder_ctx(const struct rockchip_vpu_ctx *ctx)
+static inline void vdpu_write_relaxed(struct rockchip_vpu_dev *vpu,
+				      u32 val, u32 reg)
+{
+	vpu_debug(6, "0x%04x = 0x%08x\n", reg / 4, val);
+	writel_relaxed(val, vpu->dec_base + reg);
+}
+
+static inline void vdpu_write(struct rockchip_vpu_dev *vpu, u32 val, u32 reg)
+{
+	vpu_debug(6, "0x%04x = 0x%08x\n", reg / 4, val);
+	writel(val, vpu->dec_base + reg);
+}
+
+static inline u32 vdpu_read(struct rockchip_vpu_dev *vpu, u32 reg)
 {
-	return true;
+	u32 val = readl(vpu->dec_base + reg);
+
+	vpu_debug(6, "0x%04x = 0x%08x\n", reg / 4, val);
+	return val;
 }
 
+bool rockchip_vpu_is_encoder_ctx(const struct rockchip_vpu_ctx *ctx);
+
 #endif /* ROCKCHIP_VPU_H_ */
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index d85b88067b03..0a8d7fb8903a 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -35,13 +35,48 @@ module_param_named(debug, rockchip_vpu_debug, int, 0644);
 MODULE_PARM_DESC(debug,
 		 "Debug level - higher value produces more verbose messages");
 
+static int
+rockchip_vpu_enc_buf_finish(struct rockchip_vpu_ctx *ctx,
+			    struct vb2_buffer *buf,
+			    unsigned int bytesused)
+{
+	size_t avail_size;
+
+	avail_size = vb2_plane_size(buf, 0) - ctx->vpu_dst_fmt->header_size;
+	if (bytesused > avail_size)
+		return -EINVAL;
+	/*
+	 * The bounce buffer is only for the JPEG encoder.
+	 * TODO: Rework the JPEG encoder to eliminate the need
+	 * for a bounce buffer.
+	 */
+	if (ctx->jpeg_enc.bounce_buffer.cpu) {
+		memcpy(vb2_plane_vaddr(buf, 0) +
+		       ctx->vpu_dst_fmt->header_size,
+		       ctx->jpeg_enc.bounce_buffer.cpu, bytesused);
+	}
+	buf->planes[0].bytesused =
+		ctx->vpu_dst_fmt->header_size + bytesused;
+	return 0;
+}
+
+static int
+rockchip_vpu_dec_buf_finish(struct rockchip_vpu_ctx *ctx,
+			    struct vb2_buffer *buf,
+			    unsigned int bytesused)
+{
+	/* For decoders set bytesused as per the output picture. */
+	buf->planes[0].bytesused = ctx->dst_fmt.plane_fmt[0].sizeimage;
+	return 0;
+}
+
 static void rockchip_vpu_job_finish(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx,
 				    unsigned int bytesused,
 				    enum vb2_buffer_state result)
 {
 	struct vb2_v4l2_buffer *src, *dst;
-	size_t avail_size;
+	int ret;
 
 	pm_runtime_mark_last_busy(vpu->dev);
 	pm_runtime_put_autosuspend(vpu->dev);
@@ -60,24 +95,9 @@ static void rockchip_vpu_job_finish(struct rockchip_vpu_dev *vpu,
 
 	v4l2_m2m_buf_copy_metadata(src, dst, true);
 
-	avail_size = vb2_plane_size(&dst->vb2_buf, 0) -
-		     ctx->vpu_dst_fmt->header_size;
-	if (bytesused <= avail_size) {
-		/*
-		 * The bounce buffer is only for the JPEG encoder.
-		 * TODO: Rework the JPEG encoder to eliminate the need
-		 * for a bounce buffer.
-		 */
-		if (ctx->jpeg_enc.bounce_buffer.cpu) {
-			memcpy(vb2_plane_vaddr(&dst->vb2_buf, 0) +
-			       ctx->vpu_dst_fmt->header_size,
-			       ctx->jpeg_enc.bounce_buffer.cpu, bytesused);
-		}
-		dst->vb2_buf.planes[0].bytesused =
-			ctx->vpu_dst_fmt->header_size + bytesused;
-	} else {
+	ret = ctx->buf_finish(ctx, &dst->vb2_buf, bytesused);
+	if (ret)
 		result = VB2_BUF_STATE_ERROR;
-	}
 
 	v4l2_m2m_buf_done(src, result);
 	v4l2_m2m_buf_done(dst, result);
@@ -135,6 +155,11 @@ static void device_run(void *priv)
 	rockchip_vpu_job_finish(ctx->dev, ctx, 0, VB2_BUF_STATE_ERROR);
 }
 
+bool rockchip_vpu_is_encoder_ctx(const struct rockchip_vpu_ctx *ctx)
+{
+	return ctx->buf_finish == rockchip_vpu_enc_buf_finish;
+}
+
 static struct v4l2_m2m_ops vpu_m2m_ops = {
 	.device_run = device_run,
 };
@@ -169,18 +194,25 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
 		return ret;
 
 	/*
-	 * The CAPTURE queue doesn't need dma memory,
-	 * as the CPU needs to create the JPEG frames,
-	 * from the hardware-produced JPEG payload.
+	 * When encoding, the CAPTURE queue doesn't need dma memory,
+	 * as the CPU needs to create the JPEG frames, from the
+	 * hardware-produced JPEG payload.
 	 *
-	 * For the DMA destination buffer, we use
-	 * a bounce buffer.
+	 * For the DMA destination buffer, we use a bounce buffer.
 	 */
+	if (rockchip_vpu_is_encoder_ctx(ctx)) {
+		dst_vq->mem_ops = &vb2_vmalloc_memops;
+	} else {
+		dst_vq->bidirectional = true;
+		dst_vq->mem_ops = &vb2_dma_contig_memops;
+		dst_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES |
+				    DMA_ATTR_NO_KERNEL_MAPPING;
+	}
+
 	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 	dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
 	dst_vq->drv_priv = ctx;
 	dst_vq->ops = &rockchip_vpu_queue_ops;
-	dst_vq->mem_ops = &vb2_vmalloc_memops;
 	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	dst_vq->lock = &ctx->dev->vpu_mutex;
@@ -258,11 +290,17 @@ static int rockchip_vpu_open(struct file *filp)
 		return -ENOMEM;
 
 	ctx->dev = vpu;
-	if (func->id == MEDIA_ENT_F_PROC_VIDEO_ENCODER)
+	if (func->id == MEDIA_ENT_F_PROC_VIDEO_ENCODER) {
+		ctx->buf_finish = rockchip_vpu_enc_buf_finish;
 		ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(vpu->m2m_dev, ctx,
 						    queue_init);
-	else
+	} else if (func->id == MEDIA_ENT_F_PROC_VIDEO_DECODER) {
+		ctx->buf_finish = rockchip_vpu_dec_buf_finish;
+		ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(vpu->m2m_dev, ctx,
+						    queue_init);
+	} else {
 		ctx->fh.m2m_ctx = ERR_PTR(-ENODEV);
+	}
 	if (IS_ERR(ctx->fh.m2m_ctx)) {
 		ret = PTR_ERR(ctx->fh.m2m_ctx);
 		kfree(ctx);
@@ -463,7 +501,8 @@ static void rockchip_detach_func(struct rockchip_vpu_func *func)
 	media_device_unregister_entity(&func->vdev.entity);
 }
 
-static int rockchip_vpu_add_enc_func(struct rockchip_vpu_dev *vpu)
+static int rockchip_vpu_add_func(struct rockchip_vpu_dev *vpu,
+				 unsigned int funcid)
 {
 	const struct of_device_id *match;
 	struct rockchip_vpu_func *func;
@@ -477,7 +516,7 @@ static int rockchip_vpu_add_enc_func(struct rockchip_vpu_dev *vpu)
 		return -ENOMEM;
 	}
 
-	func->id = MEDIA_ENT_F_PROC_VIDEO_ENCODER;
+	func->id = funcid;
 
 	vfd = &func->vdev;
 	vfd->fops = &rockchip_vpu_fops;
@@ -487,9 +526,14 @@ static int rockchip_vpu_add_enc_func(struct rockchip_vpu_dev *vpu)
 	vfd->vfl_dir = VFL_DIR_M2M;
 	vfd->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE;
 	vfd->ioctl_ops = &rockchip_vpu_ioctl_ops;
-	snprintf(vfd->name, sizeof(vfd->name), "%s-enc", match->compatible);
+	snprintf(vfd->name, sizeof(vfd->name), "%s-%s", match->compatible,
+		 funcid == MEDIA_ENT_F_PROC_VIDEO_ENCODER ? "enc" : "dec");
+
+	if (funcid == MEDIA_ENT_F_PROC_VIDEO_ENCODER)
+		vpu->encoder = func;
+	else
+		vpu->decoder = func;
 
-	vpu->encoder = func;
 	video_set_drvdata(vfd, vpu);
 
 	ret = video_register_device(vfd, VFL_TYPE_GRABBER, -1);
@@ -514,9 +558,31 @@ static int rockchip_vpu_add_enc_func(struct rockchip_vpu_dev *vpu)
 	return ret;
 }
 
-static void rockchip_vpu_remove_enc_func(struct rockchip_vpu_dev *vpu)
+static int rockchip_vpu_add_enc_func(struct rockchip_vpu_dev *vpu)
+{
+	if (!vpu->variant->enc_fmts)
+		return 0;
+
+	return rockchip_vpu_add_func(vpu, MEDIA_ENT_F_PROC_VIDEO_ENCODER);
+}
+
+static int rockchip_vpu_add_dec_func(struct rockchip_vpu_dev *vpu)
+{
+	if (!vpu->variant->dec_fmts)
+		return 0;
+
+	return rockchip_vpu_add_func(vpu, MEDIA_ENT_F_PROC_VIDEO_DECODER);
+}
+
+static void rockchip_vpu_remove_func(struct rockchip_vpu_dev *vpu,
+				     unsigned int funcid)
 {
-	struct rockchip_vpu_func *func = vpu->encoder;
+	struct rockchip_vpu_func *func;
+
+	if (funcid == MEDIA_ENT_F_PROC_VIDEO_ENCODER)
+		func = vpu->encoder;
+	else
+		func = vpu->decoder;
 
 	if (!func)
 		return;
@@ -525,6 +591,16 @@ static void rockchip_vpu_remove_enc_func(struct rockchip_vpu_dev *vpu)
 	video_unregister_device(&func->vdev);
 }
 
+static void rockchip_vpu_remove_enc_func(struct rockchip_vpu_dev *vpu)
+{
+	rockchip_vpu_remove_func(vpu, MEDIA_ENT_F_PROC_VIDEO_ENCODER);
+}
+
+static void rockchip_vpu_remove_dec_func(struct rockchip_vpu_dev *vpu)
+{
+	rockchip_vpu_remove_func(vpu, MEDIA_ENT_F_PROC_VIDEO_DECODER);
+}
+
 static const struct media_device_ops rockchip_m2m_media_ops = {
 	.req_validate = vb2_request_validate,
 	.req_queue = v4l2_m2m_request_queue,
@@ -563,6 +639,7 @@ static int rockchip_vpu_probe(struct platform_device *pdev)
 	if (IS_ERR(vpu->base))
 		return PTR_ERR(vpu->base);
 	vpu->enc_base = vpu->base + vpu->variant->enc_offset;
+	vpu->dec_base = vpu->base + vpu->variant->dec_offset;
 
 	ret = dma_set_coherent_mask(vpu->dev, DMA_BIT_MASK(32));
 	if (ret) {
@@ -570,6 +647,23 @@ static int rockchip_vpu_probe(struct platform_device *pdev)
 		return ret;
 	}
 
+	if (vpu->variant->vdpu_irq) {
+		int irq;
+
+		irq = platform_get_irq_byname(vpu->pdev, "vdpu");
+		if (irq <= 0) {
+			dev_err(vpu->dev, "Could not get vdpu IRQ.\n");
+			return -ENXIO;
+		}
+
+		ret = devm_request_irq(vpu->dev, irq, vpu->variant->vdpu_irq,
+				       0, dev_name(vpu->dev), vpu);
+		if (ret) {
+			dev_err(vpu->dev, "Could not request vdpu IRQ.\n");
+			return ret;
+		}
+	}
+
 	if (vpu->variant->vepu_irq) {
 		int irq;
 
@@ -631,14 +725,22 @@ static int rockchip_vpu_probe(struct platform_device *pdev)
 		goto err_m2m_rel;
 	}
 
+	ret = rockchip_vpu_add_dec_func(vpu);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to register decoder\n");
+		goto err_rm_enc_func;
+	}
+
 	ret = media_device_register(&vpu->mdev);
 	if (ret) {
 		v4l2_err(&vpu->v4l2_dev, "Failed to register mem2mem media device\n");
-		goto err_rm_enc_func;
+		goto err_rm_dec_func;
 	}
 
 	return 0;
 
+err_rm_dec_func:
+	rockchip_vpu_remove_dec_func(vpu);
 err_rm_enc_func:
 	rockchip_vpu_remove_enc_func(vpu);
 err_m2m_rel:
@@ -660,6 +762,7 @@ static int rockchip_vpu_remove(struct platform_device *pdev)
 	v4l2_info(&vpu->v4l2_dev, "Removing %s\n", pdev->name);
 
 	media_device_unregister(&vpu->mdev);
+	rockchip_vpu_remove_dec_func(vpu);
 	rockchip_vpu_remove_enc_func(vpu);
 	media_device_cleanup(&vpu->mdev);
 	v4l2_m2m_release(vpu->m2m_dev);
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
index 1ab558d6492d..1b80a45df8fe 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_v4l2.c
@@ -36,8 +36,13 @@ rockchip_vpu_get_formats(const struct rockchip_vpu_ctx *ctx,
 {
 	const struct rockchip_vpu_fmt *formats;
 
-	formats = ctx->dev->variant->enc_fmts;
-	*num_fmts = ctx->dev->variant->num_enc_fmts;
+	if (rockchip_vpu_is_encoder_ctx(ctx)) {
+		formats = ctx->dev->variant->enc_fmts;
+		*num_fmts = ctx->dev->variant->num_enc_fmts;
+	} else {
+		formats = ctx->dev->variant->dec_fmts;
+		*num_fmts = ctx->dev->variant->num_dec_fmts;
+	}
 
 	return formats;
 }
@@ -331,6 +336,22 @@ void rockchip_vpu_reset_fmts(struct rockchip_vpu_ctx *ctx)
 	rockchip_vpu_reset_raw_fmt(ctx);
 }
 
+static void
+rockchip_vpu_update_requires_request(struct rockchip_vpu_ctx *ctx,
+				     u32 fourcc)
+{
+	switch (fourcc) {
+	case V4L2_PIX_FMT_JPEG:
+		ctx->fh.m2m_ctx->out_q_ctx.q.requires_requests = false;
+		break;
+	case V4L2_PIX_FMT_MPEG2_SLICE:
+		ctx->fh.m2m_ctx->out_q_ctx.q.requires_requests = true;
+		break;
+	default:
+		break;
+	}
+}
+
 static int
 vidioc_s_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 {
@@ -387,6 +408,8 @@ vidioc_s_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	ctx->dst_fmt.xfer_func = pix_mp->xfer_func;
 	ctx->dst_fmt.quantization = pix_mp->quantization;
 
+	rockchip_vpu_update_requires_request(ctx, pix_mp->pixelformat);
+
 	vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode);
 	vpu_debug(0, "fmt - w: %d, h: %d\n",
 		  pix_mp->width, pix_mp->height);
@@ -456,6 +479,8 @@ static int vidioc_s_fmt_cap_mplane(struct file *file, void *priv,
 	vpu_debug(0, "fmt - w: %d, h: %d\n",
 		  pix_mp->width, pix_mp->height);
 
+	rockchip_vpu_update_requires_request(ctx, pix_mp->pixelformat);
+
 	return 0;
 }
 
-- 
2.20.1

[PATCH v6 12/16] rockchip/vpu: Add support for non-standard controls

[Ezequiel Garcia]

Rework the way controls are registered by the driver,
so it can support non-standard controls, such as those
used by stateless codecs.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
--
Changes from v4:
* Remove unused var in rockchip_vpu_ctrls_setup()

Changes from v3:
* None

Changes from v2:
* Got rid of unused ctrls related fields.
* Removed unndeeded is_std internal field.

 .../media/rockchip/vpu/rk3288_vpu_hw.c        |  2 +-
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  2 +-
 .../staging/media/rockchip/vpu/rockchip_vpu.h | 17 ++++++-
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 51 +++++++++++++++----
 4 files changed, 59 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c
index a874a0d83c2d..da19f1cad957 100644
--- a/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c
+++ b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c
@@ -112,7 +112,7 @@ const struct rockchip_vpu_variant rk3288_vpu_variant = {
 	.enc_fmts = rk3288_vpu_enc_fmts,
 	.num_enc_fmts = ARRAY_SIZE(rk3288_vpu_enc_fmts),
 	.codec_ops = rk3288_vpu_codec_ops,
-	.codec = RK_VPU_CODEC_JPEG,
+	.codec = RK_VPU_JPEG_ENCODER,
 	.vepu_irq = rk3288_vepu_irq,
 	.init = rk3288_vpu_hw_init,
 	.clk_names = {"aclk", "hclk"},
diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
index f4effad00605..5c6c1e8b36d6 100644
--- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
+++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
@@ -111,7 +111,7 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
 	.enc_offset = 0x0,
 	.enc_fmts = rk3399_vpu_enc_fmts,
 	.num_enc_fmts = ARRAY_SIZE(rk3399_vpu_enc_fmts),
-	.codec = RK_VPU_CODEC_JPEG,
+	.codec = RK_VPU_JPEG_ENCODER,
 	.codec_ops = rk3399_vpu_codec_ops,
 	.vepu_irq = rk3399_vepu_irq,
 	.init = rk3399_vpu_hw_init,
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
index 3d64f3e95c9b..ec7557a98583 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
@@ -34,7 +34,10 @@
 struct rockchip_vpu_ctx;
 struct rockchip_vpu_codec_ops;
 
-#define RK_VPU_CODEC_JPEG BIT(0)
+#define RK_VPU_JPEG_ENCODER	BIT(0)
+#define RK_VPU_ENCODERS		0x0000ffff
+
+#define RK_VPU_DECODERS		0xffff0000
 
 /**
  * struct rockchip_vpu_variant - information about VPU hardware variant
@@ -79,6 +82,18 @@ enum rockchip_vpu_codec_mode {
 	RK_VPU_MODE_JPEG_ENC,
 };
 
+/*
+ * struct rockchip_vpu_ctrl - helper type to declare supported controls
+ * @id:		V4L2 control ID (V4L2_CID_xxx)
+ * @codec:	codec id this control belong to (RK_VPU_JPEG_ENCODER, etc.)
+ * @cfg:	control configuration
+ */
+struct rockchip_vpu_ctrl {
+	unsigned int id;
+	unsigned int codec;
+	struct v4l2_ctrl_config cfg;
+};
+
 /*
  * struct rockchip_vpu_func - rockchip VPU functionality
  *
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index 0a8d7fb8903a..75104174640b 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -245,22 +245,51 @@ static const struct v4l2_ctrl_ops rockchip_vpu_ctrl_ops = {
 	.s_ctrl = rockchip_vpu_s_ctrl,
 };
 
+static struct rockchip_vpu_ctrl controls[] = {
+	{
+		.id = V4L2_CID_JPEG_COMPRESSION_QUALITY,
+		.codec = RK_VPU_JPEG_ENCODER,
+		.cfg = {
+			.min = 5,
+			.max = 100,
+			.step = 1,
+			.def = 50,
+		},
+	},
+};
+
 static int rockchip_vpu_ctrls_setup(struct rockchip_vpu_dev *vpu,
-				    struct rockchip_vpu_ctx *ctx)
+				    struct rockchip_vpu_ctx *ctx,
+				    int allowed_codecs)
 {
-	v4l2_ctrl_handler_init(&ctx->ctrl_handler, 1);
-	if (vpu->variant->codec & RK_VPU_CODEC_JPEG) {
-		v4l2_ctrl_new_std(&ctx->ctrl_handler, &rockchip_vpu_ctrl_ops,
-				  V4L2_CID_JPEG_COMPRESSION_QUALITY,
-				  5, 100, 1, 50);
+	int i, num_ctrls = ARRAY_SIZE(controls);
+
+	v4l2_ctrl_handler_init(&ctx->ctrl_handler, num_ctrls);
+
+	for (i = 0; i < num_ctrls; i++) {
+		if (!(allowed_codecs & controls[i].codec))
+			continue;
+		if (!controls[i].cfg.elem_size) {
+			v4l2_ctrl_new_std(&ctx->ctrl_handler,
+					  &rockchip_vpu_ctrl_ops,
+					  controls[i].id, controls[i].cfg.min,
+					  controls[i].cfg.max,
+					  controls[i].cfg.step,
+					  controls[i].cfg.def);
+		} else {
+			controls[i].cfg.id = controls[i].id;
+			v4l2_ctrl_new_custom(&ctx->ctrl_handler,
+					     &controls[i].cfg, NULL);
+		}
+
 		if (ctx->ctrl_handler.error) {
-			vpu_err("Adding JPEG control failed %d\n",
+			vpu_err("Adding control (%d) failed %d\n",
+				controls[i].id,
 				ctx->ctrl_handler.error);
 			v4l2_ctrl_handler_free(&ctx->ctrl_handler);
 			return ctx->ctrl_handler.error;
 		}
 	}
-
 	return v4l2_ctrl_handler_setup(&ctx->ctrl_handler);
 }
 
@@ -274,7 +303,7 @@ static int rockchip_vpu_open(struct file *filp)
 	struct video_device *vdev = video_devdata(filp);
 	struct rockchip_vpu_func *func = rockchip_vpu_vdev_to_func(vdev);
 	struct rockchip_vpu_ctx *ctx;
-	int ret;
+	int allowed_codecs, ret;
 
 	/*
 	 * We do not need any extra locking here, because we operate only
@@ -291,10 +320,12 @@ static int rockchip_vpu_open(struct file *filp)
 
 	ctx->dev = vpu;
 	if (func->id == MEDIA_ENT_F_PROC_VIDEO_ENCODER) {
+		allowed_codecs = vpu->variant->codec & RK_VPU_ENCODERS;
 		ctx->buf_finish = rockchip_vpu_enc_buf_finish;
 		ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(vpu->m2m_dev, ctx,
 						    queue_init);
 	} else if (func->id == MEDIA_ENT_F_PROC_VIDEO_DECODER) {
+		allowed_codecs = vpu->variant->codec & RK_VPU_DECODERS;
 		ctx->buf_finish = rockchip_vpu_dec_buf_finish;
 		ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(vpu->m2m_dev, ctx,
 						    queue_init);
@@ -313,7 +344,7 @@ static int rockchip_vpu_open(struct file *filp)
 
 	rockchip_vpu_reset_fmts(ctx);
 
-	ret = rockchip_vpu_ctrls_setup(vpu, ctx);
+	ret = rockchip_vpu_ctrls_setup(vpu, ctx, allowed_codecs);
 	if (ret) {
 		vpu_err("Failed to set up controls\n");
 		goto err_fh_free;
-- 
2.20.1

[PATCH v6 13/16] rockchip/vpu: Add infra to support MPEG-2 decoding

[Ezequiel Garcia]

From: Jonas Karlman <jonas@kwiboo.se>

Only adds structs and helpers to allow supporting MPEG-2 decoding on
rockchip SoCs. Support for RK3399 and RK3288 will be added in separate
commits

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
--
Changes from v4:
* Define rockchip_vpu_get_ctrl() and rockchip_vpu_get_ref() prototypes
  in rockchip_vpu.h

Changes from v3:
* Coding style improvements (Jonas)
* Move RK3399 support in a separate commit

Changes from v2:
* Remove uneeded check from rockchip_vpu_get_ctrl.
* Use user negotiated resolution when programming the hardware.

 drivers/staging/media/rockchip/vpu/Makefile   |  3 +-
 .../staging/media/rockchip/vpu/rockchip_vpu.h | 12 ++++
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 30 +++++++++
 .../media/rockchip/vpu/rockchip_vpu_hw.h      | 14 +++++
 .../media/rockchip/vpu/rockchip_vpu_mpeg2.c   | 61 +++++++++++++++++++
 5 files changed, 119 insertions(+), 1 deletion(-)
 create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c

diff --git a/drivers/staging/media/rockchip/vpu/Makefile b/drivers/staging/media/rockchip/vpu/Makefile
index 33606391e910..b3144e4f9d33 100644
--- a/drivers/staging/media/rockchip/vpu/Makefile
+++ b/drivers/staging/media/rockchip/vpu/Makefile
@@ -8,4 +8,5 @@ rockchip-vpu-y += \
 		rk3288_vpu_hw_jpeg_enc.o \
 		rk3399_vpu_hw.o \
 		rk3399_vpu_hw_jpeg_enc.o \
-		rockchip_vpu_jpeg.o
+		rockchip_vpu_jpeg.o \
+		rockchip_vpu_mpeg2.o
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
index ec7557a98583..3093821440c0 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu.h
@@ -27,6 +27,10 @@
 
 #define ROCKCHIP_VPU_MAX_CLOCKS		4
 
+#define MPEG2_MB_DIM			16
+#define MPEG2_MB_WIDTH(w)		DIV_ROUND_UP(w, MPEG2_MB_DIM)
+#define MPEG2_MB_HEIGHT(h)		DIV_ROUND_UP(h, MPEG2_MB_DIM)
+
 #define JPEG_MB_DIM			16
 #define JPEG_MB_WIDTH(w)		DIV_ROUND_UP(w, JPEG_MB_DIM)
 #define JPEG_MB_HEIGHT(h)		DIV_ROUND_UP(h, JPEG_MB_DIM)
@@ -37,6 +41,7 @@ struct rockchip_vpu_codec_ops;
 #define RK_VPU_JPEG_ENCODER	BIT(0)
 #define RK_VPU_ENCODERS		0x0000ffff
 
+#define RK_VPU_MPEG2_DECODER	BIT(16)
 #define RK_VPU_DECODERS		0xffff0000
 
 /**
@@ -76,10 +81,12 @@ struct rockchip_vpu_variant {
  * enum rockchip_vpu_codec_mode - codec operating mode.
  * @RK_VPU_MODE_NONE:  No operating mode. Used for RAW video formats.
  * @RK_VPU_MODE_JPEG_ENC: JPEG encoder.
+ * @RK_VPU_MODE_MPEG2_DEC: MPEG-2 decoder.
  */
 enum rockchip_vpu_codec_mode {
 	RK_VPU_MODE_NONE = -1,
 	RK_VPU_MODE_JPEG_ENC,
+	RK_VPU_MODE_MPEG2_DEC,
 };
 
 /*
@@ -190,6 +197,7 @@ struct rockchip_vpu_dev {
  *			calling v4l2_m2m_job_finish.
  * @codec_ops:		Set of operations related to codec mode.
  * @jpeg_enc:		JPEG-encoding context.
+ * @mpeg2_dec:		MPEG-2-decoding context.
  */
 struct rockchip_vpu_ctx {
 	struct rockchip_vpu_dev *dev;
@@ -215,6 +223,7 @@ struct rockchip_vpu_ctx {
 	/* Specific for particular codec modes. */
 	union {
 		struct rockchip_vpu_jpeg_enc_hw_ctx jpeg_enc;
+		struct rockchip_vpu_mpeg2_dec_hw_ctx mpeg2_dec;
 	};
 };
 
@@ -319,4 +328,7 @@ static inline u32 vdpu_read(struct rockchip_vpu_dev *vpu, u32 reg)
 
 bool rockchip_vpu_is_encoder_ctx(const struct rockchip_vpu_ctx *ctx);
 
+void *rockchip_vpu_get_ctrl(struct rockchip_vpu_ctx *ctx, u32 id);
+dma_addr_t rockchip_vpu_get_ref(struct vb2_queue *q, u64 ts);
+
 #endif /* ROCKCHIP_VPU_H_ */
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index 75104174640b..b94ff97451db 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -35,6 +35,24 @@ module_param_named(debug, rockchip_vpu_debug, int, 0644);
 MODULE_PARM_DESC(debug,
 		 "Debug level - higher value produces more verbose messages");
 
+void *rockchip_vpu_get_ctrl(struct rockchip_vpu_ctx *ctx, u32 id)
+{
+	struct v4l2_ctrl *ctrl;
+
+	ctrl = v4l2_ctrl_find(&ctx->ctrl_handler, id);
+	return ctrl ? ctrl->p_cur.p : NULL;
+}
+
+dma_addr_t rockchip_vpu_get_ref(struct vb2_queue *q, u64 ts)
+{
+	int index;
+
+	index = vb2_find_timestamp(q, ts, 0);
+	if (index >= 0)
+		return vb2_dma_contig_plane_dma_addr(q->bufs[index], 0);
+	return 0;
+}
+
 static int
 rockchip_vpu_enc_buf_finish(struct rockchip_vpu_ctx *ctx,
 			    struct vb2_buffer *buf,
@@ -255,6 +273,18 @@ static struct rockchip_vpu_ctrl controls[] = {
 			.step = 1,
 			.def = 50,
 		},
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS,
+		.codec = RK_VPU_MPEG2_DECODER,
+		.cfg = {
+			.elem_size = sizeof(struct v4l2_ctrl_mpeg2_slice_params),
+		},
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION,
+		.codec = RK_VPU_MPEG2_DECODER,
+		.cfg = {
+			.elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization),
+		},
 	},
 };
 
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
index 46716d121538..1bdc5ceb956f 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
@@ -11,6 +11,7 @@
 
 #include <linux/interrupt.h>
 #include <linux/v4l2-controls.h>
+#include <media/mpeg2-ctrls.h>
 #include <media/videobuf2-core.h>
 
 struct rockchip_vpu_dev;
@@ -38,6 +39,14 @@ struct rockchip_vpu_jpeg_enc_hw_ctx {
 	struct rockchip_vpu_aux_buf bounce_buffer;
 };
 
+/**
+ * struct rockchip_vpu_mpeg2_dec_hw_ctx
+ * @qtable:		Quantization table
+ */
+struct rockchip_vpu_mpeg2_dec_hw_ctx {
+	struct rockchip_vpu_aux_buf qtable;
+};
+
 /**
  * struct rockchip_vpu_codec_ops - codec mode specific operations
  *
@@ -83,4 +92,9 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx);
 int rockchip_vpu_jpeg_enc_init(struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_jpeg_enc_exit(struct rockchip_vpu_ctx *ctx);
 
+void rockchip_vpu_mpeg2_dec_copy_qtable(u8 *qtable,
+	const struct v4l2_ctrl_mpeg2_quantization *ctrl);
+int rockchip_vpu_mpeg2_dec_init(struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_mpeg2_dec_exit(struct rockchip_vpu_ctx *ctx);
+
 #endif /* ROCKCHIP_VPU_HW_H_ */
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c
new file mode 100644
index 000000000000..5a5b9ea1f6b5
--- /dev/null
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip VPU codec driver
+ *
+ * Copyright (C) 2018 Rockchip Electronics Co., Ltd.
+ */
+
+#include "rockchip_vpu.h"
+
+static const u8 zigzag[64] = {
+	0,   1,  8, 16,  9,  2,  3, 10,
+	17, 24, 32, 25, 18, 11,  4,  5,
+	12, 19, 26, 33, 40, 48, 41, 34,
+	27, 20, 13,  6,  7, 14, 21, 28,
+	35, 42, 49, 56, 57, 50, 43, 36,
+	29, 22, 15, 23, 30, 37, 44, 51,
+	58, 59, 52, 45, 38, 31, 39, 46,
+	53, 60, 61, 54, 47, 55, 62, 63
+};
+
+void rockchip_vpu_mpeg2_dec_copy_qtable(u8 *qtable,
+	const struct v4l2_ctrl_mpeg2_quantization *ctrl)
+{
+	int i, n;
+
+	if (!qtable || !ctrl)
+		return;
+
+	for (i = 0; i < ARRAY_SIZE(zigzag); i++) {
+		n = zigzag[i];
+		qtable[n + 0] = ctrl->intra_quantiser_matrix[i];
+		qtable[n + 64] = ctrl->non_intra_quantiser_matrix[i];
+		qtable[n + 128] = ctrl->chroma_intra_quantiser_matrix[i];
+		qtable[n + 192] = ctrl->chroma_non_intra_quantiser_matrix[i];
+	}
+}
+
+int rockchip_vpu_mpeg2_dec_init(struct rockchip_vpu_ctx *ctx)
+{
+	struct rockchip_vpu_dev *vpu = ctx->dev;
+
+	ctx->mpeg2_dec.qtable.size = ARRAY_SIZE(zigzag) * 4;
+	ctx->mpeg2_dec.qtable.cpu =
+		dma_alloc_coherent(vpu->dev,
+				   ctx->mpeg2_dec.qtable.size,
+				   &ctx->mpeg2_dec.qtable.dma,
+				   GFP_KERNEL);
+	if (!ctx->mpeg2_dec.qtable.cpu)
+		return -ENOMEM;
+	return 0;
+}
+
+void rockchip_vpu_mpeg2_dec_exit(struct rockchip_vpu_ctx *ctx)
+{
+	struct rockchip_vpu_dev *vpu = ctx->dev;
+
+	dma_free_coherent(vpu->dev,
+			  ctx->mpeg2_dec.qtable.size,
+			  ctx->mpeg2_dec.qtable.cpu,
+			  ctx->mpeg2_dec.qtable.dma);
+}
-- 
2.20.1

[PATCH v6 14/16] rockchip/vpu: Add MPEG2 decoding support to RK3399

[Ezequiel Garcia]

From: Jonas Karlman <jonas@kwiboo.se>

Add the necessary bits to support MPEG2 decoding on RK3399.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v4:
* Stop including rockchip_vpu_common.h

Changes from v3:
* New patch (was part of another patch before)

 drivers/staging/media/rockchip/vpu/Makefile   |   1 +
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  59 +++-
 .../rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c    | 267 ++++++++++++++++++
 .../media/rockchip/vpu/rockchip_vpu_hw.h      |   1 +
 4 files changed, 327 insertions(+), 1 deletion(-)
 create mode 100644 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c

diff --git a/drivers/staging/media/rockchip/vpu/Makefile b/drivers/staging/media/rockchip/vpu/Makefile
index b3144e4f9d33..c742754a89a5 100644
--- a/drivers/staging/media/rockchip/vpu/Makefile
+++ b/drivers/staging/media/rockchip/vpu/Makefile
@@ -8,5 +8,6 @@ rockchip-vpu-y += \
 		rk3288_vpu_hw_jpeg_enc.o \
 		rk3399_vpu_hw.o \
 		rk3399_vpu_hw_jpeg_enc.o \
+		rk3399_vpu_hw_mpeg2_dec.o \
 		rockchip_vpu_jpeg.o \
 		rockchip_vpu_mpeg2.o
diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
index 5c6c1e8b36d6..2b3689968ef4 100644
--- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
+++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
@@ -55,6 +55,26 @@ static const struct rockchip_vpu_fmt rk3399_vpu_enc_fmts[] = {
 	},
 };
 
+static const struct rockchip_vpu_fmt rk3399_vpu_dec_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.codec_mode = RK_VPU_MODE_NONE,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_MPEG2_SLICE,
+		.codec_mode = RK_VPU_MODE_MPEG2_DEC,
+		.max_depth = 2,
+		.frmsize = {
+			.min_width = 48,
+			.max_width = 1920,
+			.step_width = MPEG2_MB_DIM,
+			.min_height = 48,
+			.max_height = 1088,
+			.step_height = MPEG2_MB_DIM,
+		},
+	},
+};
+
 static irqreturn_t rk3399_vepu_irq(int irq, void *dev_id)
 {
 	struct rockchip_vpu_dev *vpu = dev_id;
@@ -74,6 +94,24 @@ static irqreturn_t rk3399_vepu_irq(int irq, void *dev_id)
 	return IRQ_HANDLED;
 }
 
+static irqreturn_t rk3399_vdpu_irq(int irq, void *dev_id)
+{
+	struct rockchip_vpu_dev *vpu = dev_id;
+	enum vb2_buffer_state state;
+	u32 status;
+
+	status = vdpu_read(vpu, VDPU_REG_INTERRUPT);
+	state = (status & VDPU_REG_INTERRUPT_DEC_IRQ) ?
+		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+	vdpu_write(vpu, 0, VDPU_REG_INTERRUPT);
+	vdpu_write(vpu, 0, VDPU_REG_AXI_CTRL);
+
+	rockchip_vpu_irq_done(vpu, 0, state);
+
+	return IRQ_HANDLED;
+}
+
 static int rk3399_vpu_hw_init(struct rockchip_vpu_dev *vpu)
 {
 	/* Bump ACLK to max. possible freq. to improve performance. */
@@ -90,6 +128,15 @@ static void rk3399_vpu_enc_reset(struct rockchip_vpu_ctx *ctx)
 	vepu_write(vpu, 0, VEPU_REG_AXI_CTRL);
 }
 
+static void rk3399_vpu_dec_reset(struct rockchip_vpu_ctx *ctx)
+{
+	struct rockchip_vpu_dev *vpu = ctx->dev;
+
+	vdpu_write(vpu, VDPU_REG_INTERRUPT_DEC_IRQ_DIS, VDPU_REG_INTERRUPT);
+	vdpu_write(vpu, 0, VDPU_REG_EN_FLAGS);
+	vdpu_write(vpu, 1, VDPU_REG_SOFT_RESET);
+}
+
 /*
  * Supported codec ops.
  */
@@ -101,6 +148,12 @@ static const struct rockchip_vpu_codec_ops rk3399_vpu_codec_ops[] = {
 		.init = rockchip_vpu_jpeg_enc_init,
 		.exit = rockchip_vpu_jpeg_enc_exit,
 	},
+	[RK_VPU_MODE_MPEG2_DEC] = {
+		.run = rk3399_vpu_mpeg2_dec_run,
+		.reset = rk3399_vpu_dec_reset,
+		.init = rockchip_vpu_mpeg2_dec_init,
+		.exit = rockchip_vpu_mpeg2_dec_exit,
+	},
 };
 
 /*
@@ -111,9 +164,13 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
 	.enc_offset = 0x0,
 	.enc_fmts = rk3399_vpu_enc_fmts,
 	.num_enc_fmts = ARRAY_SIZE(rk3399_vpu_enc_fmts),
-	.codec = RK_VPU_JPEG_ENCODER,
+	.dec_offset = 0x400,
+	.dec_fmts = rk3399_vpu_dec_fmts,
+	.num_dec_fmts = ARRAY_SIZE(rk3399_vpu_dec_fmts),
+	.codec = RK_VPU_JPEG_ENCODER | RK_VPU_MPEG2_DECODER,
 	.codec_ops = rk3399_vpu_codec_ops,
 	.vepu_irq = rk3399_vepu_irq,
+	.vdpu_irq = rk3399_vdpu_irq,
 	.init = rk3399_vpu_hw_init,
 	.clk_names = {"aclk", "hclk"},
 	.num_clocks = 2
diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c
new file mode 100644
index 000000000000..c4c092c2004a
--- /dev/null
+++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c
@@ -0,0 +1,267 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip VPU codec driver
+ *
+ * Copyright (C) 2018 Rockchip Electronics Co., Ltd.
+ */
+
+#include <asm/unaligned.h>
+#include <linux/bitfield.h>
+#include <media/v4l2-mem2mem.h>
+#include "rockchip_vpu.h"
+#include "rockchip_vpu_hw.h"
+
+#define VDPU_SWREG(nr)			((nr) * 4)
+
+#define VDPU_REG_DEC_OUT_BASE		VDPU_SWREG(63)
+#define VDPU_REG_RLC_VLC_BASE		VDPU_SWREG(64)
+#define VDPU_REG_QTABLE_BASE		VDPU_SWREG(61)
+#define VDPU_REG_REFER0_BASE		VDPU_SWREG(131)
+#define VDPU_REG_REFER2_BASE		VDPU_SWREG(134)
+#define VDPU_REG_REFER3_BASE		VDPU_SWREG(135)
+#define VDPU_REG_REFER1_BASE		VDPU_SWREG(148)
+#define VDPU_REG_DEC_E(v)		((v) ? BIT(0) : 0)
+
+#define VDPU_REG_DEC_ADV_PRE_DIS(v)	((v) ? BIT(11) : 0)
+#define VDPU_REG_DEC_SCMD_DIS(v)	((v) ? BIT(10) : 0)
+#define VDPU_REG_FILTERING_DIS(v)	((v) ? BIT(8) : 0)
+#define VDPU_REG_DEC_LATENCY(v)		(((v) << 1) & GENMASK(6, 1))
+
+#define VDPU_REG_INIT_QP(v)		(((v) << 25) & GENMASK(30, 25))
+#define VDPU_REG_STREAM_LEN(v)		(((v) << 0) & GENMASK(23, 0))
+
+#define VDPU_REG_APF_THRESHOLD(v)	(((v) << 17) & GENMASK(30, 17))
+#define VDPU_REG_STARTMB_X(v)		(((v) << 8) & GENMASK(16, 8))
+#define VDPU_REG_STARTMB_Y(v)		(((v) << 0) & GENMASK(7, 0))
+
+#define VDPU_REG_DEC_MODE(v)		(((v) << 0) & GENMASK(3, 0))
+
+#define VDPU_REG_DEC_STRENDIAN_E(v)	((v) ? BIT(5) : 0)
+#define VDPU_REG_DEC_STRSWAP32_E(v)	((v) ? BIT(4) : 0)
+#define VDPU_REG_DEC_OUTSWAP32_E(v)	((v) ? BIT(3) : 0)
+#define VDPU_REG_DEC_INSWAP32_E(v)	((v) ? BIT(2) : 0)
+#define VDPU_REG_DEC_OUT_ENDIAN(v)	((v) ? BIT(1) : 0)
+#define VDPU_REG_DEC_IN_ENDIAN(v)	((v) ? BIT(0) : 0)
+
+#define VDPU_REG_DEC_DATA_DISC_E(v)	((v) ? BIT(22) : 0)
+#define VDPU_REG_DEC_MAX_BURST(v)	(((v) << 16) & GENMASK(20, 16))
+#define VDPU_REG_DEC_AXI_WR_ID(v)	(((v) << 8) & GENMASK(15, 8))
+#define VDPU_REG_DEC_AXI_RD_ID(v)	(((v) << 0) & GENMASK(7, 0))
+
+#define VDPU_REG_RLC_MODE_E(v)		((v) ? BIT(20) : 0)
+#define VDPU_REG_PIC_INTERLACE_E(v)	((v) ? BIT(17) : 0)
+#define VDPU_REG_PIC_FIELDMODE_E(v)	((v) ? BIT(16) : 0)
+#define VDPU_REG_PIC_B_E(v)		((v) ? BIT(15) : 0)
+#define VDPU_REG_PIC_INTER_E(v)		((v) ? BIT(14) : 0)
+#define VDPU_REG_PIC_TOPFIELD_E(v)	((v) ? BIT(13) : 0)
+#define VDPU_REG_FWD_INTERLACE_E(v)	((v) ? BIT(12) : 0)
+#define VDPU_REG_WRITE_MVS_E(v)		((v) ? BIT(10) : 0)
+#define VDPU_REG_DEC_TIMEOUT_E(v)	((v) ? BIT(5) : 0)
+#define VDPU_REG_DEC_CLK_GATE_E(v)	((v) ? BIT(4) : 0)
+
+#define VDPU_REG_PIC_MB_WIDTH(v)	(((v) << 23) & GENMASK(31, 23))
+#define VDPU_REG_PIC_MB_HEIGHT_P(v)	(((v) << 11) & GENMASK(18, 11))
+#define VDPU_REG_ALT_SCAN_E(v)		((v) ? BIT(6) : 0)
+#define VDPU_REG_TOPFIELDFIRST_E(v)	((v) ? BIT(5) : 0)
+
+#define VDPU_REG_STRM_START_BIT(v)	(((v) << 26) & GENMASK(31, 26))
+#define VDPU_REG_QSCALE_TYPE(v)		((v) ? BIT(24) : 0)
+#define VDPU_REG_CON_MV_E(v)		((v) ? BIT(4) : 0)
+#define VDPU_REG_INTRA_DC_PREC(v)	(((v) << 2) & GENMASK(3, 2))
+#define VDPU_REG_INTRA_VLC_TAB(v)	((v) ? BIT(1) : 0)
+#define VDPU_REG_FRAME_PRED_DCT(v)	((v) ? BIT(0) : 0)
+
+#define VDPU_REG_ALT_SCAN_FLAG_E(v)	((v) ? BIT(19) : 0)
+#define VDPU_REG_FCODE_FWD_HOR(v)	(((v) << 15) & GENMASK(18, 15))
+#define VDPU_REG_FCODE_FWD_VER(v)	(((v) << 11) & GENMASK(14, 11))
+#define VDPU_REG_FCODE_BWD_HOR(v)	(((v) << 7) & GENMASK(10, 7))
+#define VDPU_REG_FCODE_BWD_VER(v)	(((v) << 3) & GENMASK(6, 3))
+#define VDPU_REG_MV_ACCURACY_FWD(v)	((v) ? BIT(2) : 0)
+#define VDPU_REG_MV_ACCURACY_BWD(v)	((v) ? BIT(1) : 0)
+
+#define PICT_TOP_FIELD     1
+#define PICT_BOTTOM_FIELD  2
+#define PICT_FRAME         3
+
+static void
+rk3399_vpu_mpeg2_dec_set_quantization(struct rockchip_vpu_dev *vpu,
+				      struct rockchip_vpu_ctx *ctx)
+{
+	struct v4l2_ctrl_mpeg2_quantization *quantization;
+
+	quantization = rockchip_vpu_get_ctrl(ctx,
+				V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
+	rockchip_vpu_mpeg2_dec_copy_qtable(ctx->mpeg2_dec.qtable.cpu,
+					   quantization);
+	vdpu_write_relaxed(vpu, ctx->mpeg2_dec.qtable.dma,
+			   VDPU_REG_QTABLE_BASE);
+}
+
+static void
+rk3399_vpu_mpeg2_dec_set_buffers(struct rockchip_vpu_dev *vpu,
+				 struct rockchip_vpu_ctx *ctx,
+				 struct vb2_buffer *src_buf,
+				 struct vb2_buffer *dst_buf,
+				 const struct v4l2_mpeg2_sequence *sequence,
+				 const struct v4l2_mpeg2_picture *picture,
+				 const struct v4l2_ctrl_mpeg2_slice_params *slice_params)
+{
+	dma_addr_t forward_addr = 0, backward_addr = 0;
+	dma_addr_t current_addr, addr;
+	struct vb2_queue *vq;
+
+	vq = v4l2_m2m_get_dst_vq(ctx->fh.m2m_ctx);
+
+	switch (picture->picture_coding_type) {
+	case V4L2_MPEG2_PICTURE_CODING_TYPE_B:
+		backward_addr = rockchip_vpu_get_ref(vq,
+						slice_params->backward_ref_ts);
+		/* fall-through */
+	case V4L2_MPEG2_PICTURE_CODING_TYPE_P:
+		forward_addr = rockchip_vpu_get_ref(vq,
+						slice_params->forward_ref_ts);
+	}
+
+	/* Source bitstream buffer */
+	addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+	vdpu_write_relaxed(vpu, addr, VDPU_REG_RLC_VLC_BASE);
+
+	/* Destination frame buffer */
+	addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	current_addr = addr;
+
+	if (picture->picture_structure == PICT_BOTTOM_FIELD)
+		addr += ALIGN(ctx->dst_fmt.width, 16);
+	vdpu_write_relaxed(vpu, addr, VDPU_REG_DEC_OUT_BASE);
+
+	if (!forward_addr)
+		forward_addr = current_addr;
+	if (!backward_addr)
+		backward_addr = current_addr;
+
+	/* Set forward ref frame (top/bottom field) */
+	if (picture->picture_structure == PICT_FRAME ||
+	    picture->picture_coding_type == V4L2_MPEG2_PICTURE_CODING_TYPE_B ||
+	    (picture->picture_structure == PICT_TOP_FIELD &&
+	     picture->top_field_first) ||
+	    (picture->picture_structure == PICT_BOTTOM_FIELD &&
+	     !picture->top_field_first)) {
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER0_BASE);
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER1_BASE);
+	} else if (picture->picture_structure == PICT_TOP_FIELD) {
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER0_BASE);
+		vdpu_write_relaxed(vpu, current_addr, VDPU_REG_REFER1_BASE);
+	} else if (picture->picture_structure == PICT_BOTTOM_FIELD) {
+		vdpu_write_relaxed(vpu, current_addr, VDPU_REG_REFER0_BASE);
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER1_BASE);
+	}
+
+	/* Set backward ref frame (top/bottom field) */
+	vdpu_write_relaxed(vpu, backward_addr, VDPU_REG_REFER2_BASE);
+	vdpu_write_relaxed(vpu, backward_addr, VDPU_REG_REFER3_BASE);
+}
+
+void rk3399_vpu_mpeg2_dec_run(struct rockchip_vpu_ctx *ctx)
+{
+	struct rockchip_vpu_dev *vpu = ctx->dev;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	const struct v4l2_ctrl_mpeg2_slice_params *slice_params;
+	const struct v4l2_mpeg2_sequence *sequence;
+	const struct v4l2_mpeg2_picture *picture;
+	u32 reg;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+
+	/* Apply request controls if any */
+	v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
+				&ctx->ctrl_handler);
+
+	slice_params = rockchip_vpu_get_ctrl(ctx,
+				V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS);
+	sequence = &slice_params->sequence;
+	picture = &slice_params->picture;
+
+	reg = VDPU_REG_DEC_ADV_PRE_DIS(0) |
+	      VDPU_REG_DEC_SCMD_DIS(0) |
+	      VDPU_REG_FILTERING_DIS(1) |
+	      VDPU_REG_DEC_LATENCY(0);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(50));
+
+	reg = VDPU_REG_INIT_QP(1) |
+	      VDPU_REG_STREAM_LEN(slice_params->bit_size >> 3);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(51));
+
+	reg = VDPU_REG_APF_THRESHOLD(8) |
+	      VDPU_REG_STARTMB_X(0) |
+	      VDPU_REG_STARTMB_Y(0);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(52));
+
+	reg = VDPU_REG_DEC_MODE(5);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(53));
+
+	reg = VDPU_REG_DEC_STRENDIAN_E(1) |
+	      VDPU_REG_DEC_STRSWAP32_E(1) |
+	      VDPU_REG_DEC_OUTSWAP32_E(1) |
+	      VDPU_REG_DEC_INSWAP32_E(1) |
+	      VDPU_REG_DEC_OUT_ENDIAN(1) |
+	      VDPU_REG_DEC_IN_ENDIAN(1);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(54));
+
+	reg = VDPU_REG_DEC_DATA_DISC_E(0) |
+	      VDPU_REG_DEC_MAX_BURST(16) |
+	      VDPU_REG_DEC_AXI_WR_ID(0) |
+	      VDPU_REG_DEC_AXI_RD_ID(0);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(56));
+
+	reg = VDPU_REG_RLC_MODE_E(0) |
+	      VDPU_REG_PIC_INTERLACE_E(!sequence->progressive_sequence) |
+	      VDPU_REG_PIC_FIELDMODE_E(picture->picture_structure != PICT_FRAME) |
+	      VDPU_REG_PIC_B_E(picture->picture_coding_type == V4L2_MPEG2_PICTURE_CODING_TYPE_B) |
+	      VDPU_REG_PIC_INTER_E(picture->picture_coding_type != V4L2_MPEG2_PICTURE_CODING_TYPE_I) |
+	      VDPU_REG_PIC_TOPFIELD_E(picture->picture_structure == PICT_TOP_FIELD) |
+	      VDPU_REG_FWD_INTERLACE_E(0) |
+	      VDPU_REG_WRITE_MVS_E(0) |
+	      VDPU_REG_DEC_TIMEOUT_E(1) |
+	      VDPU_REG_DEC_CLK_GATE_E(1);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(57));
+
+	reg = VDPU_REG_PIC_MB_WIDTH(MPEG2_MB_WIDTH(ctx->dst_fmt.width)) |
+	      VDPU_REG_PIC_MB_HEIGHT_P(MPEG2_MB_HEIGHT(ctx->dst_fmt.height)) |
+	      VDPU_REG_ALT_SCAN_E(picture->alternate_scan) |
+	      VDPU_REG_TOPFIELDFIRST_E(picture->top_field_first);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(120));
+
+	reg = VDPU_REG_STRM_START_BIT(slice_params->data_bit_offset) |
+	      VDPU_REG_QSCALE_TYPE(picture->q_scale_type) |
+	      VDPU_REG_CON_MV_E(picture->concealment_motion_vectors) |
+	      VDPU_REG_INTRA_DC_PREC(picture->intra_dc_precision) |
+	      VDPU_REG_INTRA_VLC_TAB(picture->intra_vlc_format) |
+	      VDPU_REG_FRAME_PRED_DCT(picture->frame_pred_frame_dct);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(122));
+
+	reg = VDPU_REG_ALT_SCAN_FLAG_E(picture->alternate_scan) |
+	      VDPU_REG_FCODE_FWD_HOR(picture->f_code[0][0]) |
+	      VDPU_REG_FCODE_FWD_VER(picture->f_code[0][1]) |
+	      VDPU_REG_FCODE_BWD_HOR(picture->f_code[1][0]) |
+	      VDPU_REG_FCODE_BWD_VER(picture->f_code[1][1]) |
+	      VDPU_REG_MV_ACCURACY_FWD(1) |
+	      VDPU_REG_MV_ACCURACY_BWD(1);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(136));
+
+	rk3399_vpu_mpeg2_dec_set_quantization(vpu, ctx);
+
+	rk3399_vpu_mpeg2_dec_set_buffers(vpu, ctx, &src_buf->vb2_buf,
+					 &dst_buf->vb2_buf,
+					 sequence, picture, slice_params);
+
+	/* Controls no longer in-use, we can complete them */
+	v4l2_ctrl_request_complete(src_buf->vb2_buf.req_obj.req,
+				   &ctx->ctrl_handler);
+
+	/* Kick the watchdog and start decoding */
+	schedule_delayed_work(&vpu->watchdog_work, msecs_to_jiffies(2000));
+
+	reg = vdpu_read(vpu, VDPU_SWREG(57)) | VDPU_REG_DEC_E(1);
+	vdpu_write(vpu, reg, VDPU_SWREG(57));
+}
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
index 1bdc5ceb956f..4268c3a8b924 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
@@ -92,6 +92,7 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx);
 int rockchip_vpu_jpeg_enc_init(struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_jpeg_enc_exit(struct rockchip_vpu_ctx *ctx);
 
+void rk3399_vpu_mpeg2_dec_run(struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_mpeg2_dec_copy_qtable(u8 *qtable,
 	const struct v4l2_ctrl_mpeg2_quantization *ctrl);
 int rockchip_vpu_mpeg2_dec_init(struct rockchip_vpu_ctx *ctx);
-- 
2.20.1

[PATCH v6 15/16] rockchip/vpu: Add support for MPEG-2 decoding on RK3288

[Ezequiel Garcia]

From: Jonas Karlman <jonas@kwiboo.se>

Add necessary bits to support MPEG2 decoding on RK3288.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes from v4:
* Stop including rockchip_vpu_common.h

Changes from v3:
* Coding style improvements (Jonas)
* Use ALIGN() instead of div+shift (Jonas)

Changes from v2:
* New patch

 drivers/staging/media/rockchip/vpu/Makefile   |   1 +
 .../media/rockchip/vpu/rk3288_vpu_hw.c        |  59 +++-
 .../rockchip/vpu/rk3288_vpu_hw_mpeg2_dec.c    | 261 ++++++++++++++++++
 .../media/rockchip/vpu/rk3288_vpu_regs.h      |   1 +
 .../media/rockchip/vpu/rockchip_vpu_hw.h      |   1 +
 5 files changed, 322 insertions(+), 1 deletion(-)
 create mode 100644 drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_mpeg2_dec.c

diff --git a/drivers/staging/media/rockchip/vpu/Makefile b/drivers/staging/media/rockchip/vpu/Makefile
index c742754a89a5..be278157d196 100644
--- a/drivers/staging/media/rockchip/vpu/Makefile
+++ b/drivers/staging/media/rockchip/vpu/Makefile
@@ -6,6 +6,7 @@ rockchip-vpu-y += \
 		rockchip_vpu_v4l2.o \
 		rk3288_vpu_hw.o \
 		rk3288_vpu_hw_jpeg_enc.o \
+		rk3288_vpu_hw_mpeg2_dec.o \
 		rk3399_vpu_hw.o \
 		rk3399_vpu_hw_jpeg_enc.o \
 		rk3399_vpu_hw_mpeg2_dec.o \
diff --git a/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c
index da19f1cad957..003143c77d37 100644
--- a/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c
+++ b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c
@@ -55,6 +55,26 @@ static const struct rockchip_vpu_fmt rk3288_vpu_enc_fmts[] = {
 	},
 };
 
+static const struct rockchip_vpu_fmt rk3288_vpu_dec_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.codec_mode = RK_VPU_MODE_NONE,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_MPEG2_SLICE,
+		.codec_mode = RK_VPU_MODE_MPEG2_DEC,
+		.max_depth = 2,
+		.frmsize = {
+			.min_width = 48,
+			.max_width = 1920,
+			.step_width = MPEG2_MB_DIM,
+			.min_height = 48,
+			.max_height = 1088,
+			.step_height = MPEG2_MB_DIM,
+		},
+	},
+};
+
 static irqreturn_t rk3288_vepu_irq(int irq, void *dev_id)
 {
 	struct rockchip_vpu_dev *vpu = dev_id;
@@ -74,6 +94,24 @@ static irqreturn_t rk3288_vepu_irq(int irq, void *dev_id)
 	return IRQ_HANDLED;
 }
 
+static irqreturn_t rk3288_vdpu_irq(int irq, void *dev_id)
+{
+	struct rockchip_vpu_dev *vpu = dev_id;
+	enum vb2_buffer_state state;
+	u32 status;
+
+	status = vdpu_read(vpu, VDPU_REG_INTERRUPT);
+	state = (status & VDPU_REG_INTERRUPT_DEC_RDY_INT) ?
+		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+	vdpu_write(vpu, 0, VDPU_REG_INTERRUPT);
+	vdpu_write(vpu, VDPU_REG_CONFIG_DEC_CLK_GATE_E, VDPU_REG_CONFIG);
+
+	rockchip_vpu_irq_done(vpu, 0, state);
+
+	return IRQ_HANDLED;
+}
+
 static int rk3288_vpu_hw_init(struct rockchip_vpu_dev *vpu)
 {
 	/* Bump ACLK to max. possible freq. to improve performance. */
@@ -90,6 +128,15 @@ static void rk3288_vpu_enc_reset(struct rockchip_vpu_ctx *ctx)
 	vepu_write(vpu, 0, VEPU_REG_AXI_CTRL);
 }
 
+static void rk3288_vpu_dec_reset(struct rockchip_vpu_ctx *ctx)
+{
+	struct rockchip_vpu_dev *vpu = ctx->dev;
+
+	vdpu_write(vpu, VDPU_REG_INTERRUPT_DEC_IRQ_DIS, VDPU_REG_INTERRUPT);
+	vdpu_write(vpu, VDPU_REG_CONFIG_DEC_CLK_GATE_E, VDPU_REG_CONFIG);
+	vdpu_write(vpu, 1, VDPU_REG_SOFT_RESET);
+}
+
 /*
  * Supported codec ops.
  */
@@ -101,6 +148,12 @@ static const struct rockchip_vpu_codec_ops rk3288_vpu_codec_ops[] = {
 		.init = rockchip_vpu_jpeg_enc_init,
 		.exit = rockchip_vpu_jpeg_enc_exit,
 	},
+	[RK_VPU_MODE_MPEG2_DEC] = {
+		.run = rk3288_vpu_mpeg2_dec_run,
+		.reset = rk3288_vpu_dec_reset,
+		.init = rockchip_vpu_mpeg2_dec_init,
+		.exit = rockchip_vpu_mpeg2_dec_exit,
+	},
 };
 
 /*
@@ -111,9 +164,13 @@ const struct rockchip_vpu_variant rk3288_vpu_variant = {
 	.enc_offset = 0x0,
 	.enc_fmts = rk3288_vpu_enc_fmts,
 	.num_enc_fmts = ARRAY_SIZE(rk3288_vpu_enc_fmts),
+	.dec_offset = 0x400,
+	.dec_fmts = rk3288_vpu_dec_fmts,
+	.num_dec_fmts = ARRAY_SIZE(rk3288_vpu_dec_fmts),
+	.codec = RK_VPU_JPEG_ENCODER | RK_VPU_MPEG2_DECODER,
 	.codec_ops = rk3288_vpu_codec_ops,
-	.codec = RK_VPU_JPEG_ENCODER,
 	.vepu_irq = rk3288_vepu_irq,
+	.vdpu_irq = rk3288_vdpu_irq,
 	.init = rk3288_vpu_hw_init,
 	.clk_names = {"aclk", "hclk"},
 	.num_clocks = 2
diff --git a/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_mpeg2_dec.c b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_mpeg2_dec.c
new file mode 100644
index 000000000000..e9eee47fcea1
--- /dev/null
+++ b/drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_mpeg2_dec.c
@@ -0,0 +1,261 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip VPU codec driver
+ *
+ * Copyright (C) 2018 Rockchip Electronics Co., Ltd.
+ */
+
+#include <asm/unaligned.h>
+#include <linux/bitfield.h>
+#include <media/v4l2-mem2mem.h>
+#include "rockchip_vpu.h"
+#include "rockchip_vpu_hw.h"
+
+#define VDPU_SWREG(nr)			((nr) * 4)
+
+#define VDPU_REG_RLC_VLC_BASE		VDPU_SWREG(12)
+#define VDPU_REG_DEC_OUT_BASE		VDPU_SWREG(13)
+#define VDPU_REG_REFER0_BASE		VDPU_SWREG(14)
+#define VDPU_REG_REFER1_BASE		VDPU_SWREG(15)
+#define VDPU_REG_REFER2_BASE		VDPU_SWREG(16)
+#define VDPU_REG_REFER3_BASE		VDPU_SWREG(17)
+#define VDPU_REG_QTABLE_BASE		VDPU_SWREG(40)
+#define VDPU_REG_DEC_E(v)		((v) ? BIT(0) : 0)
+
+#define VDPU_REG_DEC_AXI_RD_ID(v)	(((v) << 24) & GENMASK(31, 24))
+#define VDPU_REG_DEC_TIMEOUT_E(v)	((v) ? BIT(23) : 0)
+#define VDPU_REG_DEC_STRSWAP32_E(v)	((v) ? BIT(22) : 0)
+#define VDPU_REG_DEC_STRENDIAN_E(v)	((v) ? BIT(21) : 0)
+#define VDPU_REG_DEC_INSWAP32_E(v)	((v) ? BIT(20) : 0)
+#define VDPU_REG_DEC_OUTSWAP32_E(v)	((v) ? BIT(19) : 0)
+#define VDPU_REG_DEC_DATA_DISC_E(v)	((v) ? BIT(18) : 0)
+#define VDPU_REG_DEC_LATENCY(v)		(((v) << 11) & GENMASK(16, 11))
+#define VDPU_REG_DEC_CLK_GATE_E(v)	((v) ? BIT(10) : 0)
+#define VDPU_REG_DEC_IN_ENDIAN(v)	((v) ? BIT(9) : 0)
+#define VDPU_REG_DEC_OUT_ENDIAN(v)	((v) ? BIT(8) : 0)
+#define VDPU_REG_DEC_ADV_PRE_DIS(v)	((v) ? BIT(6) : 0)
+#define VDPU_REG_DEC_SCMD_DIS(v)	((v) ? BIT(5) : 0)
+#define VDPU_REG_DEC_MAX_BURST(v)	(((v) << 0) & GENMASK(4, 0))
+
+#define VDPU_REG_DEC_MODE(v)		(((v) << 28) & GENMASK(31, 28))
+#define VDPU_REG_RLC_MODE_E(v)		((v) ? BIT(27) : 0)
+#define VDPU_REG_PIC_INTERLACE_E(v)	((v) ? BIT(23) : 0)
+#define VDPU_REG_PIC_FIELDMODE_E(v)	((v) ? BIT(22) : 0)
+#define VDPU_REG_PIC_B_E(v)		((v) ? BIT(21) : 0)
+#define VDPU_REG_PIC_INTER_E(v)		((v) ? BIT(20) : 0)
+#define VDPU_REG_PIC_TOPFIELD_E(v)	((v) ? BIT(19) : 0)
+#define VDPU_REG_FWD_INTERLACE_E(v)	((v) ? BIT(18) : 0)
+#define VDPU_REG_FILTERING_DIS(v)	((v) ? BIT(14) : 0)
+#define VDPU_REG_WRITE_MVS_E(v)		((v) ? BIT(12) : 0)
+#define VDPU_REG_DEC_AXI_WR_ID(v)	(((v) << 0) & GENMASK(7, 0))
+
+#define VDPU_REG_PIC_MB_WIDTH(v)	(((v) << 23) & GENMASK(31, 23))
+#define VDPU_REG_PIC_MB_HEIGHT_P(v)	(((v) << 11) & GENMASK(18, 11))
+#define VDPU_REG_ALT_SCAN_E(v)		((v) ? BIT(6) : 0)
+#define VDPU_REG_TOPFIELDFIRST_E(v)	((v) ? BIT(5) : 0)
+
+#define VDPU_REG_STRM_START_BIT(v)	(((v) << 26) & GENMASK(31, 26))
+#define VDPU_REG_QSCALE_TYPE(v)		((v) ? BIT(24) : 0)
+#define VDPU_REG_CON_MV_E(v)		((v) ? BIT(4) : 0)
+#define VDPU_REG_INTRA_DC_PREC(v)	(((v) << 2) & GENMASK(3, 2))
+#define VDPU_REG_INTRA_VLC_TAB(v)	((v) ? BIT(1) : 0)
+#define VDPU_REG_FRAME_PRED_DCT(v)	((v) ? BIT(0) : 0)
+
+#define VDPU_REG_INIT_QP(v)		(((v) << 25) & GENMASK(30, 25))
+#define VDPU_REG_STREAM_LEN(v)		(((v) << 0) & GENMASK(23, 0))
+
+#define VDPU_REG_ALT_SCAN_FLAG_E(v)	((v) ? BIT(19) : 0)
+#define VDPU_REG_FCODE_FWD_HOR(v)	(((v) << 15) & GENMASK(18, 15))
+#define VDPU_REG_FCODE_FWD_VER(v)	(((v) << 11) & GENMASK(14, 11))
+#define VDPU_REG_FCODE_BWD_HOR(v)	(((v) << 7) & GENMASK(10, 7))
+#define VDPU_REG_FCODE_BWD_VER(v)	(((v) << 3) & GENMASK(6, 3))
+#define VDPU_REG_MV_ACCURACY_FWD(v)	((v) ? BIT(2) : 0)
+#define VDPU_REG_MV_ACCURACY_BWD(v)	((v) ? BIT(1) : 0)
+
+#define VDPU_REG_STARTMB_X(v)		(((v) << 23) & GENMASK(31, 23))
+#define VDPU_REG_STARTMB_Y(v)		(((v) << 15) & GENMASK(22, 15))
+
+#define VDPU_REG_APF_THRESHOLD(v)	(((v) << 0) & GENMASK(13, 0))
+
+#define PICT_TOP_FIELD     1
+#define PICT_BOTTOM_FIELD  2
+#define PICT_FRAME         3
+
+static void
+rk3288_vpu_mpeg2_dec_set_quantization(struct rockchip_vpu_dev *vpu,
+				      struct rockchip_vpu_ctx *ctx)
+{
+	struct v4l2_ctrl_mpeg2_quantization *quantization;
+
+	quantization = rockchip_vpu_get_ctrl(ctx,
+				V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
+	rockchip_vpu_mpeg2_dec_copy_qtable(ctx->mpeg2_dec.qtable.cpu,
+					   quantization);
+	vdpu_write_relaxed(vpu, ctx->mpeg2_dec.qtable.dma,
+			   VDPU_REG_QTABLE_BASE);
+}
+
+static void
+rk3288_vpu_mpeg2_dec_set_buffers(struct rockchip_vpu_dev *vpu,
+				 struct rockchip_vpu_ctx *ctx,
+				 struct vb2_buffer *src_buf,
+				 struct vb2_buffer *dst_buf,
+				 const struct v4l2_mpeg2_sequence *sequence,
+				 const struct v4l2_mpeg2_picture *picture,
+				 const struct v4l2_ctrl_mpeg2_slice_params *slice_params)
+{
+	dma_addr_t forward_addr = 0, backward_addr = 0;
+	dma_addr_t current_addr, addr;
+	struct vb2_queue *vq;
+
+	vq = v4l2_m2m_get_dst_vq(ctx->fh.m2m_ctx);
+
+	switch (picture->picture_coding_type) {
+	case V4L2_MPEG2_PICTURE_CODING_TYPE_B:
+		backward_addr = rockchip_vpu_get_ref(vq,
+						slice_params->backward_ref_ts);
+		/* fall-through */
+	case V4L2_MPEG2_PICTURE_CODING_TYPE_P:
+		forward_addr = rockchip_vpu_get_ref(vq,
+						slice_params->forward_ref_ts);
+	}
+
+	/* Source bitstream buffer */
+	addr = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+	vdpu_write_relaxed(vpu, addr, VDPU_REG_RLC_VLC_BASE);
+
+	/* Destination frame buffer */
+	addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+	current_addr = addr;
+
+	if (picture->picture_structure == PICT_BOTTOM_FIELD)
+		addr += ALIGN(ctx->dst_fmt.width, 16);
+	vdpu_write_relaxed(vpu, addr, VDPU_REG_DEC_OUT_BASE);
+
+	if (!forward_addr)
+		forward_addr = current_addr;
+	if (!backward_addr)
+		backward_addr = current_addr;
+
+	/* Set forward ref frame (top/bottom field) */
+	if (picture->picture_structure == PICT_FRAME ||
+	    picture->picture_coding_type == V4L2_MPEG2_PICTURE_CODING_TYPE_B ||
+	    (picture->picture_structure == PICT_TOP_FIELD &&
+	     picture->top_field_first) ||
+	    (picture->picture_structure == PICT_BOTTOM_FIELD &&
+	     !picture->top_field_first)) {
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER0_BASE);
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER1_BASE);
+	} else if (picture->picture_structure == PICT_TOP_FIELD) {
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER0_BASE);
+		vdpu_write_relaxed(vpu, current_addr, VDPU_REG_REFER1_BASE);
+	} else if (picture->picture_structure == PICT_BOTTOM_FIELD) {
+		vdpu_write_relaxed(vpu, current_addr, VDPU_REG_REFER0_BASE);
+		vdpu_write_relaxed(vpu, forward_addr, VDPU_REG_REFER1_BASE);
+	}
+
+	/* Set backward ref frame (top/bottom field) */
+	vdpu_write_relaxed(vpu, backward_addr, VDPU_REG_REFER2_BASE);
+	vdpu_write_relaxed(vpu, backward_addr, VDPU_REG_REFER3_BASE);
+}
+
+void rk3288_vpu_mpeg2_dec_run(struct rockchip_vpu_ctx *ctx)
+{
+	struct rockchip_vpu_dev *vpu = ctx->dev;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	const struct v4l2_ctrl_mpeg2_slice_params *slice_params;
+	const struct v4l2_mpeg2_sequence *sequence;
+	const struct v4l2_mpeg2_picture *picture;
+	u32 reg;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+
+	/* Apply request controls if any */
+	v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
+				&ctx->ctrl_handler);
+
+	slice_params = rockchip_vpu_get_ctrl(ctx,
+				V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS);
+	sequence = &slice_params->sequence;
+	picture = &slice_params->picture;
+
+	reg = VDPU_REG_DEC_AXI_RD_ID(0) |
+	      VDPU_REG_DEC_TIMEOUT_E(1) |
+	      VDPU_REG_DEC_STRSWAP32_E(1) |
+	      VDPU_REG_DEC_STRENDIAN_E(1) |
+	      VDPU_REG_DEC_INSWAP32_E(1) |
+	      VDPU_REG_DEC_OUTSWAP32_E(1) |
+	      VDPU_REG_DEC_DATA_DISC_E(0) |
+	      VDPU_REG_DEC_LATENCY(0) |
+	      VDPU_REG_DEC_CLK_GATE_E(1) |
+	      VDPU_REG_DEC_IN_ENDIAN(1) |
+	      VDPU_REG_DEC_OUT_ENDIAN(1) |
+	      VDPU_REG_DEC_ADV_PRE_DIS(0) |
+	      VDPU_REG_DEC_SCMD_DIS(0) |
+	      VDPU_REG_DEC_MAX_BURST(16);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(2));
+
+	reg = VDPU_REG_DEC_MODE(5) |
+	      VDPU_REG_RLC_MODE_E(0) |
+	      VDPU_REG_PIC_INTERLACE_E(!sequence->progressive_sequence) |
+	      VDPU_REG_PIC_FIELDMODE_E(picture->picture_structure != PICT_FRAME) |
+	      VDPU_REG_PIC_B_E(picture->picture_coding_type == V4L2_MPEG2_PICTURE_CODING_TYPE_B) |
+	      VDPU_REG_PIC_INTER_E(picture->picture_coding_type != V4L2_MPEG2_PICTURE_CODING_TYPE_I) |
+	      VDPU_REG_PIC_TOPFIELD_E(picture->picture_structure == PICT_TOP_FIELD) |
+	      VDPU_REG_FWD_INTERLACE_E(0) |
+	      VDPU_REG_FILTERING_DIS(1) |
+	      VDPU_REG_WRITE_MVS_E(0) |
+	      VDPU_REG_DEC_AXI_WR_ID(0);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(3));
+
+	reg = VDPU_REG_PIC_MB_WIDTH(MPEG2_MB_WIDTH(ctx->dst_fmt.width)) |
+	      VDPU_REG_PIC_MB_HEIGHT_P(MPEG2_MB_HEIGHT(ctx->dst_fmt.height)) |
+	      VDPU_REG_ALT_SCAN_E(picture->alternate_scan) |
+	      VDPU_REG_TOPFIELDFIRST_E(picture->top_field_first);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(4));
+
+	reg = VDPU_REG_STRM_START_BIT(slice_params->data_bit_offset) |
+	      VDPU_REG_QSCALE_TYPE(picture->q_scale_type) |
+	      VDPU_REG_CON_MV_E(picture->concealment_motion_vectors) |
+	      VDPU_REG_INTRA_DC_PREC(picture->intra_dc_precision) |
+	      VDPU_REG_INTRA_VLC_TAB(picture->intra_vlc_format) |
+	      VDPU_REG_FRAME_PRED_DCT(picture->frame_pred_frame_dct);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(5));
+
+	reg = VDPU_REG_INIT_QP(1) |
+	      VDPU_REG_STREAM_LEN(slice_params->bit_size >> 3);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(6));
+
+	reg = VDPU_REG_ALT_SCAN_FLAG_E(picture->alternate_scan) |
+	      VDPU_REG_FCODE_FWD_HOR(picture->f_code[0][0]) |
+	      VDPU_REG_FCODE_FWD_VER(picture->f_code[0][1]) |
+	      VDPU_REG_FCODE_BWD_HOR(picture->f_code[1][0]) |
+	      VDPU_REG_FCODE_BWD_VER(picture->f_code[1][1]) |
+	      VDPU_REG_MV_ACCURACY_FWD(1) |
+	      VDPU_REG_MV_ACCURACY_BWD(1);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(18));
+
+	reg = VDPU_REG_STARTMB_X(0) |
+	      VDPU_REG_STARTMB_Y(0);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(48));
+
+	reg = VDPU_REG_APF_THRESHOLD(8);
+	vdpu_write_relaxed(vpu, reg, VDPU_SWREG(55));
+
+	rk3288_vpu_mpeg2_dec_set_quantization(vpu, ctx);
+
+	rk3288_vpu_mpeg2_dec_set_buffers(vpu, ctx, &src_buf->vb2_buf,
+					 &dst_buf->vb2_buf,
+					 sequence, picture, slice_params);
+
+	/* Controls no longer in-use, we can complete them */
+	v4l2_ctrl_request_complete(src_buf->vb2_buf.req_obj.req,
+				   &ctx->ctrl_handler);
+
+	/* Kick the watchdog and start decoding */
+	schedule_delayed_work(&vpu->watchdog_work, msecs_to_jiffies(2000));
+
+	reg = VDPU_REG_DEC_E(1);
+	vdpu_write(vpu, reg, VDPU_SWREG(1));
+}
diff --git a/drivers/staging/media/rockchip/vpu/rk3288_vpu_regs.h b/drivers/staging/media/rockchip/vpu/rk3288_vpu_regs.h
index 9d0b9bdf3297..c9631b713804 100644
--- a/drivers/staging/media/rockchip/vpu/rk3288_vpu_regs.h
+++ b/drivers/staging/media/rockchip/vpu/rk3288_vpu_regs.h
@@ -438,5 +438,6 @@
 #define     VDPU_REG_REF_BUF_CTRL2_REFBU2_THR(x)	(((x) & 0xfff) << 19)
 #define     VDPU_REG_REF_BUF_CTRL2_REFBU2_PICID(x)	(((x) & 0x1f) << 14)
 #define     VDPU_REG_REF_BUF_CTRL2_APF_THRESHOLD(x)	(((x) & 0x3fff) << 0)
+#define VDPU_REG_SOFT_RESET			0x194
 
 #endif /* RK3288_VPU_REGS_H_ */
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
index 4268c3a8b924..6cecb528f994 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
@@ -92,6 +92,7 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx);
 int rockchip_vpu_jpeg_enc_init(struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_jpeg_enc_exit(struct rockchip_vpu_ctx *ctx);
 
+void rk3288_vpu_mpeg2_dec_run(struct rockchip_vpu_ctx *ctx);
 void rk3399_vpu_mpeg2_dec_run(struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_mpeg2_dec_copy_qtable(u8 *qtable,
 	const struct v4l2_ctrl_mpeg2_quantization *ctrl);
-- 
2.20.1

[PATCH v6 16/16] rockchip/vpu: Add support for MPEG-2 decoding on RK3328

[Ezequiel Garcia]

From: Jonas Karlman <jonas@kwiboo.se>

Add necessary bits to support MPEG2 decoding on RK3328.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
--
Changes from v5:
* New patch.

 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c   | 12 ++++++++++++
 .../staging/media/rockchip/vpu/rockchip_vpu_drv.c    |  1 +
 drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h |  1 +
 3 files changed, 14 insertions(+)

diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
index 2b3689968ef4..341f8d69c33d 100644
--- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
+++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
@@ -175,3 +175,15 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
 	.clk_names = {"aclk", "hclk"},
 	.num_clocks = 2
 };
+
+const struct rockchip_vpu_variant rk3328_vpu_variant = {
+	.dec_offset = 0x400,
+	.dec_fmts = rk3399_vpu_dec_fmts,
+	.num_dec_fmts = ARRAY_SIZE(rk3399_vpu_dec_fmts),
+	.codec = RK_VPU_MPEG2_DECODER,
+	.codec_ops = rk3399_vpu_codec_ops,
+	.vdpu_irq = rk3399_vdpu_irq,
+	.init = rk3399_vpu_hw_init,
+	.clk_names = {"aclk", "hclk"},
+	.num_clocks = 2
+};
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
index b94ff97451db..2e22009b6583 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
@@ -419,6 +419,7 @@ static const struct v4l2_file_operations rockchip_vpu_fops = {
 
 static const struct of_device_id of_rockchip_vpu_match[] = {
 	{ .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
+	{ .compatible = "rockchip,rk3328-vpu", .data = &rk3328_vpu_variant, },
 	{ .compatible = "rockchip,rk3288-vpu", .data = &rk3288_vpu_variant, },
 	{ /* sentinel */ }
 };
diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
index 6cecb528f994..3d6b97af90fb 100644
--- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
+++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
@@ -79,6 +79,7 @@ enum rockchip_vpu_enc_fmt {
 };
 
 extern const struct rockchip_vpu_variant rk3399_vpu_variant;
+extern const struct rockchip_vpu_variant rk3328_vpu_variant;
 extern const struct rockchip_vpu_variant rk3288_vpu_variant;
 
 void rockchip_vpu_watchdog(struct work_struct *work);
-- 
2.20.1

Re: [PATCH v6 16/16] rockchip/vpu: Add support for MPEG-2 decoding on RK3328

[Hans Verkuil]

On 5/28/19 7:02 PM, Ezequiel Garcia wrote:
> From: Jonas Karlman <jonas@kwiboo.se>
> 
> Add necessary bits to support MPEG2 decoding on RK3328.
> 
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
> --
> Changes from v5:
> * New patch.
> 
>  drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c   | 12 ++++++++++++
>  .../staging/media/rockchip/vpu/rockchip_vpu_drv.c    |  1 +
>  drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h |  1 +
>  3 files changed, 14 insertions(+)
> 
> diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
> index 2b3689968ef4..341f8d69c33d 100644
> --- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
> +++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
> @@ -175,3 +175,15 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
>  	.clk_names = {"aclk", "hclk"},
>  	.num_clocks = 2
>  };
> +
> +const struct rockchip_vpu_variant rk3328_vpu_variant = {
> +	.dec_offset = 0x400,
> +	.dec_fmts = rk3399_vpu_dec_fmts,
> +	.num_dec_fmts = ARRAY_SIZE(rk3399_vpu_dec_fmts),
> +	.codec = RK_VPU_MPEG2_DECODER,
> +	.codec_ops = rk3399_vpu_codec_ops,
> +	.vdpu_irq = rk3399_vdpu_irq,
> +	.init = rk3399_vpu_hw_init,
> +	.clk_names = {"aclk", "hclk"},
> +	.num_clocks = 2
> +};
> diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
> index b94ff97451db..2e22009b6583 100644
> --- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
> +++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
> @@ -419,6 +419,7 @@ static const struct v4l2_file_operations rockchip_vpu_fops = {
>  
>  static const struct of_device_id of_rockchip_vpu_match[] = {
>  	{ .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
> +	{ .compatible = "rockchip,rk3328-vpu", .data = &rk3328_vpu_variant, },

This new compatible string should be documented in
Documentation/devicetree/bindings/media/rockchip-vpu.txt as well.

I'll take patches 1-15 and drop this one. This patch can be merged once
the bindings file is updated as well.

Regards,

	Hans

>  	{ .compatible = "rockchip,rk3288-vpu", .data = &rk3288_vpu_variant, },
>  	{ /* sentinel */ }
>  };
> diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
> index 6cecb528f994..3d6b97af90fb 100644
> --- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
> +++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
> @@ -79,6 +79,7 @@ enum rockchip_vpu_enc_fmt {
>  };
>  
>  extern const struct rockchip_vpu_variant rk3399_vpu_variant;
> +extern const struct rockchip_vpu_variant rk3328_vpu_variant;
>  extern const struct rockchip_vpu_variant rk3288_vpu_variant;
>  
>  void rockchip_vpu_watchdog(struct work_struct *work);
> 

Re: [PATCH v6 16/16] rockchip/vpu: Add support for MPEG-2 decoding on RK3328

[Jonas Karlman]

On 2019-05-29 10:11, Hans Verkuil wrote:
> On 5/28/19 7:02 PM, Ezequiel Garcia wrote:
>> From: Jonas Karlman <jonas@kwiboo.se>
>>
>> Add necessary bits to support MPEG2 decoding on RK3328.
>>
>> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
>> Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
>> --
>> Changes from v5:
>> * New patch.
>>
>>  drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c   | 12 ++++++++++++
>>  .../staging/media/rockchip/vpu/rockchip_vpu_drv.c    |  1 +
>>  drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h |  1 +
>>  3 files changed, 14 insertions(+)
>>
>> diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
>> index 2b3689968ef4..341f8d69c33d 100644
>> --- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
>> +++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
>> @@ -175,3 +175,15 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
>>  	.clk_names = {"aclk", "hclk"},
>>  	.num_clocks = 2
>>  };
>> +
>> +const struct rockchip_vpu_variant rk3328_vpu_variant = {
>> +	.dec_offset = 0x400,
>> +	.dec_fmts = rk3399_vpu_dec_fmts,
>> +	.num_dec_fmts = ARRAY_SIZE(rk3399_vpu_dec_fmts),
>> +	.codec = RK_VPU_MPEG2_DECODER,
>> +	.codec_ops = rk3399_vpu_codec_ops,
>> +	.vdpu_irq = rk3399_vdpu_irq,
>> +	.init = rk3399_vpu_hw_init,
>> +	.clk_names = {"aclk", "hclk"},
>> +	.num_clocks = 2
>> +};
>> diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
>> index b94ff97451db..2e22009b6583 100644
>> --- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
>> +++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
>> @@ -419,6 +419,7 @@ static const struct v4l2_file_operations rockchip_vpu_fops = {
>>  
>>  static const struct of_device_id of_rockchip_vpu_match[] = {
>>  	{ .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
>> +	{ .compatible = "rockchip,rk3328-vpu", .data = &rk3328_vpu_variant, },
> This new compatible string should be documented in
> Documentation/devicetree/bindings/media/rockchip-vpu.txt as well.
>
> I'll take patches 1-15 and drop this one. This patch can be merged once
> the bindings file is updated as well.

Looks like the bindings patch did not get picked for v6 :-)

I am a bit unclear on how to handle patch submission that covers multiple subtrees.
Should I send a single series including three patches: bindings update, this patch and device tree update?
Or is a series with only bindings update and this patch preferred?

Regards,
Jonas

>
> Regards,
>
> 	Hans
>
>>  	{ .compatible = "rockchip,rk3288-vpu", .data = &rk3288_vpu_variant, },
>>  	{ /* sentinel */ }
>>  };
>> diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
>> index 6cecb528f994..3d6b97af90fb 100644
>> --- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
>> +++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
>> @@ -79,6 +79,7 @@ enum rockchip_vpu_enc_fmt {
>>  };
>>  
>>  extern const struct rockchip_vpu_variant rk3399_vpu_variant;
>> +extern const struct rockchip_vpu_variant rk3328_vpu_variant;
>>  extern const struct rockchip_vpu_variant rk3288_vpu_variant;
>>  
>>  void rockchip_vpu_watchdog(struct work_struct *work);
>>

Re: [PATCH v6 16/16] rockchip/vpu: Add support for MPEG-2 decoding on RK3328

[Hans Verkuil]

On 5/29/19 10:50 AM, Jonas Karlman wrote:
> On 2019-05-29 10:11, Hans Verkuil wrote:
>> On 5/28/19 7:02 PM, Ezequiel Garcia wrote:
>>> From: Jonas Karlman <jonas@kwiboo.se>
>>>
>>> Add necessary bits to support MPEG2 decoding on RK3328.
>>>
>>> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
>>> Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
>>> --
>>> Changes from v5:
>>> * New patch.
>>>
>>>  drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c   | 12 ++++++++++++
>>>  .../staging/media/rockchip/vpu/rockchip_vpu_drv.c    |  1 +
>>>  drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h |  1 +
>>>  3 files changed, 14 insertions(+)
>>>
>>> diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
>>> index 2b3689968ef4..341f8d69c33d 100644
>>> --- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
>>> +++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
>>> @@ -175,3 +175,15 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
>>>  	.clk_names = {"aclk", "hclk"},
>>>  	.num_clocks = 2
>>>  };
>>> +
>>> +const struct rockchip_vpu_variant rk3328_vpu_variant = {
>>> +	.dec_offset = 0x400,
>>> +	.dec_fmts = rk3399_vpu_dec_fmts,
>>> +	.num_dec_fmts = ARRAY_SIZE(rk3399_vpu_dec_fmts),
>>> +	.codec = RK_VPU_MPEG2_DECODER,
>>> +	.codec_ops = rk3399_vpu_codec_ops,
>>> +	.vdpu_irq = rk3399_vdpu_irq,
>>> +	.init = rk3399_vpu_hw_init,
>>> +	.clk_names = {"aclk", "hclk"},
>>> +	.num_clocks = 2
>>> +};
>>> diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
>>> index b94ff97451db..2e22009b6583 100644
>>> --- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
>>> +++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
>>> @@ -419,6 +419,7 @@ static const struct v4l2_file_operations rockchip_vpu_fops = {
>>>  
>>>  static const struct of_device_id of_rockchip_vpu_match[] = {
>>>  	{ .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
>>> +	{ .compatible = "rockchip,rk3328-vpu", .data = &rk3328_vpu_variant, },
>> This new compatible string should be documented in
>> Documentation/devicetree/bindings/media/rockchip-vpu.txt as well.
>>
>> I'll take patches 1-15 and drop this one. This patch can be merged once
>> the bindings file is updated as well.
> 
> Looks like the bindings patch did not get picked for v6 :-)
> 
> I am a bit unclear on how to handle patch submission that covers multiple subtrees.
> Should I send a single series including three patches: bindings update, this patch and device tree update?
> Or is a series with only bindings update and this patch preferred?

In this case just post a patch updating rockchip-vpu.txt, CC-ing devicetree@vger.kernel.org and
Rob Herring <robh+dt@kernel.org>.

Rob will Ack it, and then I can make a pull request for that bindings patch and this
16/16 patch. There is nothing wrong with this patch, so no need to repost. I just
need to have an updated rockchip-vpu.txt patch as well.

Regards,

	Hans

> 
> Regards,
> Jonas
> 
>>
>> Regards,
>>
>> 	Hans
>>
>>>  	{ .compatible = "rockchip,rk3288-vpu", .data = &rk3288_vpu_variant, },
>>>  	{ /* sentinel */ }
>>>  };
>>> diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
>>> index 6cecb528f994..3d6b97af90fb 100644
>>> --- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
>>> +++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h
>>> @@ -79,6 +79,7 @@ enum rockchip_vpu_enc_fmt {
>>>  };
>>>  
>>>  extern const struct rockchip_vpu_variant rk3399_vpu_variant;
>>> +extern const struct rockchip_vpu_variant rk3328_vpu_variant;
>>>  extern const struct rockchip_vpu_variant rk3288_vpu_variant;
>>>  
>>>  void rockchip_vpu_watchdog(struct work_struct *work);
>>>
> 

Re: [PATCH v6 16/16] rockchip/vpu: Add support for MPEG-2 decoding on RK3328

[Ezequiel Garcia]

On Wed, 2019-05-29 at 08:50 +0000, Jonas Karlman wrote:
> On 2019-05-29 10:11, Hans Verkuil wrote:
> > On 5/28/19 7:02 PM, Ezequiel Garcia wrote:
> > > From: Jonas Karlman <jonas@kwiboo.se>
> > > 
> > > Add necessary bits to support MPEG2 decoding on RK3328.
> > > 
> > > Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> > > Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
> > > --
> > > Changes from v5:
> > > * New patch.
> > > 
> > >  drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c   | 12 ++++++++++++
> > >  .../staging/media/rockchip/vpu/rockchip_vpu_drv.c    |  1 +
> > >  drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h |  1 +
> > >  3 files changed, 14 insertions(+)
> > > 
> > > diff --git a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
> > > index 2b3689968ef4..341f8d69c33d 100644
> > > --- a/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
> > > +++ b/drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c
> > > @@ -175,3 +175,15 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
> > >  	.clk_names = {"aclk", "hclk"},
> > >  	.num_clocks = 2
> > >  };
> > > +
> > > +const struct rockchip_vpu_variant rk3328_vpu_variant = {
> > > +	.dec_offset = 0x400,
> > > +	.dec_fmts = rk3399_vpu_dec_fmts,
> > > +	.num_dec_fmts = ARRAY_SIZE(rk3399_vpu_dec_fmts),
> > > +	.codec = RK_VPU_MPEG2_DECODER,
> > > +	.codec_ops = rk3399_vpu_codec_ops,
> > > +	.vdpu_irq = rk3399_vdpu_irq,
> > > +	.init = rk3399_vpu_hw_init,
> > > +	.clk_names = {"aclk", "hclk"},
> > > +	.num_clocks = 2
> > > +};
> > > diff --git a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
> > > index b94ff97451db..2e22009b6583 100644
> > > --- a/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
> > > +++ b/drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c
> > > @@ -419,6 +419,7 @@ static const struct v4l2_file_operations rockchip_vpu_fops = {
> > >  
> > >  static const struct of_device_id of_rockchip_vpu_match[] = {
> > >  	{ .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
> > > +	{ .compatible = "rockchip,rk3328-vpu", .data = &rk3328_vpu_variant, },
> > This new compatible string should be documented in
> > Documentation/devicetree/bindings/media/rockchip-vpu.txt as well.
> > 
> > I'll take patches 1-15 and drop this one. This patch can be merged once
> > the bindings file is updated as well.
> 
> Looks like the bindings patch did not get picked for v6 :-)
> 

Oh, that's my bad. I picked the support and left the binding out.

> I am a bit unclear on how to handle patch submission that covers multiple subtrees.
> Should I send a single series including three patches: bindings update, this patch and device tree update?
> Or is a series with only bindings update and this patch preferred?
> 

Thanks both for sorting this one out,
Ezequiel