[PATCH v2 0/2] rcar-du, vsp1: rcar-gen3: Add support for colorkey alpha blending

[PATCH v2 0/2] rcar-du, vsp1: rcar-gen3: Add support for colorkey alpha blending

[Alexandru Gheorghe]

Currently, rcar-du supports colorkeying  only for rcar-gen2 and it uses 
some hw capability of the display unit(DU) which is not available on gen3.
In order to implement colorkeying for gen3 we need to use the colorkey
capability of the VSPD, hence the need to change both drivers rcar-du and
vsp1.

This patchset had been developed and tested on top of v4.9/rcar-3.5.1 from
git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git

Changes since v1:
- group boolean variables to reduce object size
- fixed signoff

Alexandru Gheorghe (2):
  v4l: vsp1: Add support for colorkey alpha blending
  drm: rcar-du: Add support for colorkey alpha blending

 drivers/gpu/drm/rcar-du/rcar_du_drv.h   |  1 +
 drivers/gpu/drm/rcar-du/rcar_du_kms.c   |  8 ++++++++
 drivers/gpu/drm/rcar-du/rcar_du_plane.c |  3 ---
 drivers/gpu/drm/rcar-du/rcar_du_plane.h |  6 ++++++
 drivers/gpu/drm/rcar-du/rcar_du_vsp.c   | 22 ++++++++++++++++++++++
 drivers/gpu/drm/rcar-du/rcar_du_vsp.h   |  5 +++++
 drivers/media/platform/vsp1/vsp1_drm.c  |  3 +++
 drivers/media/platform/vsp1/vsp1_rpf.c  | 10 ++++++++--
 drivers/media/platform/vsp1/vsp1_rwpf.h |  3 +++
 include/media/vsp1.h                    |  3 +++
 10 files changed, 59 insertions(+), 5 deletions(-)

-- 
1.9.1

[PATCH v2 1/2] v4l: vsp1: Add support for colorkey alpha blending

[Alexandru Gheorghe]

The vsp2 hw supports changing of the alpha of pixels that match a color
key, this patch adds support for this feature in order to be used by
the rcar-du driver.
The colorkey is interpreted different depending of the pixel format:
	* RGB   - all color components have to match.
	* YCbCr - only the Y component has to match.

Signed-off-by: Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com>
---
 drivers/media/platform/vsp1/vsp1_drm.c  |  3 +++
 drivers/media/platform/vsp1/vsp1_rpf.c  | 10 ++++++++--
 drivers/media/platform/vsp1/vsp1_rwpf.h |  3 +++
 include/media/vsp1.h                    |  3 +++
 4 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 3627f08..a4d0aee 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -393,6 +393,9 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 	else
 		rpf->format.plane_fmt[1].bytesperline = cfg->pitch;
 	rpf->alpha = cfg->alpha;
+	rpf->colorkey = cfg->colorkey;
+	rpf->colorkey_en = cfg->colorkey_en;
+	rpf->colorkey_alpha = cfg->colorkey_alpha;
 	rpf->interlaced = cfg->interlaced;
 
 	if (soc_device_match(r8a7795es1) && rpf->interlaced) {
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index a12d6f9..91f2a9f 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -356,8 +356,14 @@ static void rpf_configure(struct vsp1_entity *entity,
 	}
 
 	vsp1_rpf_write(rpf, dl, VI6_RPF_MSK_CTRL, 0);
-	vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL, 0);
-
+	if (rpf->colorkey_en) {
+		vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_SET0,
+			       (rpf->colorkey_alpha << 24) | rpf->colorkey);
+		vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL,
+			       VI6_RPF_CKEY_CTRL_SAPE0);
+	} else {
+		vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL, 0);
+	}
 }
 
 static const struct vsp1_entity_operations rpf_entity_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index fbe6aa6..2d7f4b9 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -51,6 +51,9 @@ struct vsp1_rwpf {
 	unsigned int brs_input;
 
 	unsigned int alpha;
+	u32 colorkey;
+	bool colorkey_en;
+	u32 colorkey_alpha;
 
 	u32 mult_alpha;
 	u32 outfmt;
diff --git a/include/media/vsp1.h b/include/media/vsp1.h
index 97265f7..65e3934 100644
--- a/include/media/vsp1.h
+++ b/include/media/vsp1.h
@@ -32,6 +32,9 @@ struct vsp1_du_atomic_config {
 	struct v4l2_rect dst;
 	unsigned int alpha;
 	unsigned int zpos;
+	u32 colorkey;
+	u32 colorkey_alpha;
+	bool colorkey_en;
 	bool interlaced;
 };
 
-- 
1.9.1

Re: [PATCH v2 1/2] v4l: vsp1: Add support for colorkey alpha blending

[Mauro Carvalho Chehab]

Em Sun, 7 May 2017 13:13:26 +0300
Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com> escreveu:

> The vsp2 hw supports changing of the alpha of pixels that match a color
> key, this patch adds support for this feature in order to be used by
> the rcar-du driver.
> The colorkey is interpreted different depending of the pixel format:
> 	* RGB   - all color components have to match.
> 	* YCbCr - only the Y component has to match.
> 
> Signed-off-by: Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com>

As most of the changes on this series are for DRM, from my side,
feel free to merge this via DRM tree.

Acked-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

> ---
>  drivers/media/platform/vsp1/vsp1_drm.c  |  3 +++
>  drivers/media/platform/vsp1/vsp1_rpf.c  | 10 ++++++++--
>  drivers/media/platform/vsp1/vsp1_rwpf.h |  3 +++
>  include/media/vsp1.h                    |  3 +++
>  4 files changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
> index 3627f08..a4d0aee 100644
> --- a/drivers/media/platform/vsp1/vsp1_drm.c
> +++ b/drivers/media/platform/vsp1/vsp1_drm.c
> @@ -393,6 +393,9 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
>  	else
>  		rpf->format.plane_fmt[1].bytesperline = cfg->pitch;
>  	rpf->alpha = cfg->alpha;
> +	rpf->colorkey = cfg->colorkey;
> +	rpf->colorkey_en = cfg->colorkey_en;
> +	rpf->colorkey_alpha = cfg->colorkey_alpha;
>  	rpf->interlaced = cfg->interlaced;
>  
>  	if (soc_device_match(r8a7795es1) && rpf->interlaced) {
> diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
> index a12d6f9..91f2a9f 100644
> --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> @@ -356,8 +356,14 @@ static void rpf_configure(struct vsp1_entity *entity,
>  	}
>  
>  	vsp1_rpf_write(rpf, dl, VI6_RPF_MSK_CTRL, 0);
> -	vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL, 0);
> -
> +	if (rpf->colorkey_en) {
> +		vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_SET0,
> +			       (rpf->colorkey_alpha << 24) | rpf->colorkey);
> +		vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL,
> +			       VI6_RPF_CKEY_CTRL_SAPE0);
> +	} else {
> +		vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL, 0);
> +	}
>  }
>  
>  static const struct vsp1_entity_operations rpf_entity_ops = {
> diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
> index fbe6aa6..2d7f4b9 100644
> --- a/drivers/media/platform/vsp1/vsp1_rwpf.h
> +++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
> @@ -51,6 +51,9 @@ struct vsp1_rwpf {
>  	unsigned int brs_input;
>  
>  	unsigned int alpha;
> +	u32 colorkey;
> +	bool colorkey_en;
> +	u32 colorkey_alpha;
>  
>  	u32 mult_alpha;
>  	u32 outfmt;
> diff --git a/include/media/vsp1.h b/include/media/vsp1.h
> index 97265f7..65e3934 100644
> --- a/include/media/vsp1.h
> +++ b/include/media/vsp1.h
> @@ -32,6 +32,9 @@ struct vsp1_du_atomic_config {
>  	struct v4l2_rect dst;
>  	unsigned int alpha;
>  	unsigned int zpos;
> +	u32 colorkey;
> +	u32 colorkey_alpha;
> +	bool colorkey_en;
>  	bool interlaced;
>  };
>  



Thanks,
Mauro

[PATCH v2 2/2] drm: rcar-du: Add support for colorkey alpha blending

[Alexandru Gheorghe]

Add two new plane properties colorkey and colorkey_alpha for rcar gen3.
* colorkey:
	- used for specifying the color on which the filtering is done.
	- bits 0 to 23 are interpreted as RGB888 format, in case we are
	  dealing with an YCbCr format, only the Y componenet is
	  compared and it is represented by the G bits from RGB888
	  format.
	- bit 24 tells if it is enabled or not.
* colorkey_alpha:
	- the alpha to be set for matching pixels, in case it is
	  missing the pixels will be made transparent

Signed-off-by: Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com>
---
 drivers/gpu/drm/rcar-du/rcar_du_drv.h   |  1 +
 drivers/gpu/drm/rcar-du/rcar_du_kms.c   |  8 ++++++++
 drivers/gpu/drm/rcar-du/rcar_du_plane.c |  3 ---
 drivers/gpu/drm/rcar-du/rcar_du_plane.h |  6 ++++++
 drivers/gpu/drm/rcar-du/rcar_du_vsp.c   | 22 ++++++++++++++++++++++
 drivers/gpu/drm/rcar-du/rcar_du_vsp.h   |  5 +++++
 6 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.h b/drivers/gpu/drm/rcar-du/rcar_du_drv.h
index 91e8fc5..1cb92e3 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_drv.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.h
@@ -98,6 +98,7 @@ struct rcar_du_device {
 	struct {
 		struct drm_property *alpha;
 		struct drm_property *colorkey;
+		struct drm_property *colorkey_alpha;
 	} props;
 
 	unsigned int dpad0_source;
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_kms.c b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
index 1cc88ed..a733fa2 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_kms.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
@@ -630,6 +630,14 @@ static int rcar_du_properties_init(struct rcar_du_device *rcdu)
 	if (rcdu->props.colorkey == NULL)
 		return -ENOMEM;
 
+	if (rcdu->info->gen == 3) {
+		rcdu->props.colorkey_alpha =
+			drm_property_create_range(rcdu->ddev, 0,
+						  "colorkey_alpha", 0, 255);
+		if (!rcdu->props.colorkey_alpha)
+			return -ENOMEM;
+	}
+
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.c b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
index e408aa3..df689c4 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
@@ -307,9 +307,6 @@ int rcar_du_atomic_check_planes(struct drm_device *dev,
  * Plane Setup
  */
 
-#define RCAR_DU_COLORKEY_NONE		(0 << 24)
-#define RCAR_DU_COLORKEY_SOURCE		(1 << 24)
-#define RCAR_DU_COLORKEY_MASK		(1 << 24)
 
 static void rcar_du_plane_write(struct rcar_du_group *rgrp,
 				unsigned int index, u32 reg, u32 data)
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.h b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
index c1de338..9e7c3b6 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
@@ -49,6 +49,12 @@ static inline struct rcar_du_plane *to_rcar_plane(struct drm_plane *plane)
 	return container_of(plane, struct rcar_du_plane, plane);
 }
 
+#define RCAR_DU_COLORKEY_NONE		(0 << 24)
+#define RCAR_DU_COLORKEY_MASK		BIT(24)
+#define RCAR_DU_COLORKEY_EN_MASK	RCAR_DU_COLORKEY_MASK
+#define RCAR_DU_COLORKEY_COLOR_MASK	0xFFFFFF
+#define RCAR_DU_COLORKEY_ALPHA_MASK	0xFF
+
 /**
  * struct rcar_du_plane_state - Driver-specific plane state
  * @state: base DRM plane state
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c
index 4b460d4..b223be1 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c
@@ -180,6 +180,11 @@ static void rcar_du_vsp_plane_setup(struct rcar_du_vsp_plane *plane)
 		.pitch = fb->pitches[0],
 		.alpha = state->alpha,
 		.zpos = state->state.zpos,
+		.colorkey = state->colorkey & RCAR_DU_COLORKEY_COLOR_MASK,
+		.colorkey_en =
+			((state->colorkey & RCAR_DU_COLORKEY_EN_MASK) != 0),
+		.colorkey_alpha =
+			(state->colorkey_alpha & RCAR_DU_COLORKEY_ALPHA_MASK),
 	};
 	unsigned int i;
 
@@ -379,6 +384,8 @@ static void rcar_du_vsp_plane_reset(struct drm_plane *plane)
 		return;
 
 	state->alpha = 255;
+	state->colorkey = RCAR_DU_COLORKEY_NONE;
+	state->colorkey_alpha = 0;
 	state->state.zpos = plane->type == DRM_PLANE_TYPE_PRIMARY ? 0 : 1;
 
 	plane->state = &state->state;
@@ -394,6 +401,10 @@ static int rcar_du_vsp_plane_atomic_set_property(struct drm_plane *plane,
 
 	if (property == rcdu->props.alpha)
 		rstate->alpha = val;
+	else if (property == rcdu->props.colorkey)
+		rstate->colorkey = val;
+	else if (property == rcdu->props.colorkey_alpha)
+		rstate->colorkey_alpha = val;
 	else
 		return -EINVAL;
 
@@ -410,6 +421,10 @@ static int rcar_du_vsp_plane_atomic_get_property(struct drm_plane *plane,
 
 	if (property == rcdu->props.alpha)
 		*val = rstate->alpha;
+	else if (property == rcdu->props.colorkey)
+		*val = rstate->colorkey;
+	else if (property == rcdu->props.colorkey_alpha)
+		*val = rstate->colorkey_alpha;
 	else
 		return -EINVAL;
 
@@ -633,6 +648,13 @@ int rcar_du_vsp_init(struct rcar_du_vsp *vsp)
 
 		drm_object_attach_property(&plane->plane.base,
 					   rcdu->props.alpha, 255);
+		drm_object_attach_property(&plane->plane.base,
+					   rcdu->props.colorkey,
+					   RCAR_DU_COLORKEY_NONE);
+		if (rcdu->props.colorkey_alpha)
+			drm_object_attach_property(&plane->plane.base,
+						   rcdu->props.colorkey_alpha,
+						   0);
 		drm_plane_create_zpos_property(&plane->plane, 1, 1,
 					       vsp->num_planes - 1);
 	}
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_vsp.h b/drivers/gpu/drm/rcar-du/rcar_du_vsp.h
index 3fd9cef..1543503 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_vsp.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_vsp.h
@@ -47,6 +47,9 @@ static inline struct rcar_du_vsp_plane *to_rcar_vsp_plane(struct drm_plane *p)
  * @sg_tables: scatter-gather tables for the frame buffer memory
  * @alpha: value of the plane alpha property
  * @zpos: value of the plane zpos property
+ * @colorkey: value of the color for which to apply colorkey_alpha, bit 24
+ * tells if it is enabled or not
+ * @colorkey_alpha: alpha to be used for pixels with color equal to colorkey
  */
 struct rcar_du_vsp_plane_state {
 	struct drm_plane_state state;
@@ -56,6 +59,8 @@ struct rcar_du_vsp_plane_state {
 
 	unsigned int alpha;
 	unsigned int zpos;
+	u32 colorkey;
+	u32 colorkey_alpha;
 };
 
 static inline struct rcar_du_vsp_plane_state *
-- 
1.9.1

Re: [PATCH v2 0/2] rcar-du, vsp1: rcar-gen3: Add support for colorkey alpha blending

[Eric Anholt]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com> writes:

> Currently, rcar-du supports colorkeying  only for rcar-gen2 and it uses 
> some hw capability of the display unit(DU) which is not available on gen3.
> In order to implement colorkeying for gen3 we need to use the colorkey
> capability of the VSPD, hence the need to change both drivers rcar-du and
> vsp1.
>
> This patchset had been developed and tested on top of v4.9/rcar-3.5.1 from
> git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git

A few questions:

Are other drivers interested in supporting this property?  VC4 has the
24-bit RGB colorkey, but I don't see YCBCR support.  Should it be
documented in a generic location?

Does your colorkey end up forcing alpha to 1 for the plane when it's not
matched?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [PATCH v2 0/2] rcar-du, vsp1: rcar-gen3: Add support for colorkey alpha blending

[Daniel Vetter]

On Mon, May 08, 2017 at 09:33:37AM -0700, Eric Anholt wrote:
> Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com> writes:
> 
> > Currently, rcar-du supports colorkeying  only for rcar-gen2 and it uses 
> > some hw capability of the display unit(DU) which is not available on gen3.
> > In order to implement colorkeying for gen3 we need to use the colorkey
> > capability of the VSPD, hence the need to change both drivers rcar-du and
> > vsp1.
> >
> > This patchset had been developed and tested on top of v4.9/rcar-3.5.1 from
> > git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
> 
> A few questions:
> 
> Are other drivers interested in supporting this property?  VC4 has the
> 24-bit RGB colorkey, but I don't see YCBCR support.  Should it be
> documented in a generic location?
> 
> Does your colorkey end up forcing alpha to 1 for the plane when it's not
> matched?

I think generic color-key for plane compositioning would be nice, but I'm
not sure that's possible due to differences in how the key works.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

RE: [PATCH v2 0/2] rcar-du, vsp1: rcar-gen3: Add support for colorkey alpha blending

[Gheorghe, Alexandru]

Hello Daniel, Eric,

On Mon, Monday, May 8, 2017 9:29 PM +0200, Daniel Vetter wrote:
> -----Original Message-----
> From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of Daniel
> Vetter
> Sent: 8 mai 2017 21:30
> To: Eric Anholt <eric@anholt.net>
> Cc: Gheorghe, Alexandru <Alexandru_Gheorghe@mentor.com>;
> laurent.pinchart@ideasonboard.com; linux-renesas-soc@vger.kernel.org;
> dri-devel@lists.freedesktop.org; linux-media@vger.kernel.org; geert@linux-
> m68k.org; sergei.shtylyov@cogentembedded.com
> Subject: Re: [PATCH v2 0/2] rcar-du, vsp1: rcar-gen3: Add support for
> colorkey alpha blending
> 
> On Mon, May 08, 2017 at 09:33:37AM -0700, Eric Anholt wrote:
> > Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com> writes:
> >
> > > Currently, rcar-du supports colorkeying  only for rcar-gen2 and it
> > > uses some hw capability of the display unit(DU) which is not available on
> gen3.
> > > In order to implement colorkeying for gen3 we need to use the
> > > colorkey capability of the VSPD, hence the need to change both
> > > drivers rcar-du and vsp1.
> > >
> > > This patchset had been developed and tested on top of
> > > v4.9/rcar-3.5.1 from
> > > git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
> >
> > A few questions:
> >
> > Are other drivers interested in supporting this property?  VC4 has the
> > 24-bit RGB colorkey, but I don't see YCBCR support.  Should it be
> > documented in a generic location?


As far as I identified  armada, i815, nouveau, rcar-du, vmwgfx drivers have this notion of colorkey. 
There could be other HW which don't have this implemented yet.

> >
> > Does your colorkey end up forcing alpha to 1 for the plane when it's
> > not matched?

I think this  behavior is HW dependent, on R-CAR Gen3, the alpha for pixel that don't match is not touch. 

> 
> I think generic color-key for plane compositioning would be nice, but I'm not
> sure that's possible due to differences in how the key works.

I'm thinking of starting from the drivers that do have this property and see if there is any common ground for a generic color-key property/ies

> -Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

Alex Gheorghe

Re: [PATCH v2 0/2] rcar-du, vsp1: rcar-gen3: Add support for colorkey alpha blending

[Laurent Pinchart]

Hi Alex et all,

On Tuesday, 9 May 2017 10:12:31 EET Gheorghe, Alexandru wrote:
> On Mon, Monday, May 8, 2017 9:29 PM +0200, Daniel Vetter wrote:
> > On Mon, May 08, 2017 at 09:33:37AM -0700, Eric Anholt wrote:
> >> Alexandru Gheorghe <Alexandru_Gheorghe@mentor.com> writes:
> >>> Currently, rcar-du supports colorkeying  only for rcar-gen2 and it
> >>> uses some hw capability of the display unit(DU) which is not available
> >>> on gen3.
> >>> 
> >>> In order to implement colorkeying for gen3 we need to use the
> >>> colorkey capability of the VSPD, hence the need to change both
> >>> drivers rcar-du and vsp1.
> >>> 
> >>> This patchset had been developed and tested on top of
> >>> v4.9/rcar-3.5.1 from
> >>> git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
> >> 
> >> A few questions:
> >> 
> >> Are other drivers interested in supporting this property?  VC4 has the
> >> 24-bit RGB colorkey, but I don't see YCBCR support.  Should it be
> >> documented in a generic location?
> 
> As far as I identified  armada, i815, nouveau, rcar-du, vmwgfx drivers have
> this notion of colorkey. There could be other HW which don't have this
> implemented yet.

Among those drivers only armada, nouveau and rcar-du expose the color key 
through DRM properties. The i915 and vmwgfx drivers use custom ioctls. Here is 
what is currently implemented.

- armada

"colorkey" range  0x00000000 0x00ffffff
"colorkey_min" range  0x00000000 0x00ffffff
"colorkey_max" range  0x00000000 0x00ffffff
"colorkey_val" range  0x00000000 0x00ffffff
"colorkey_alpha" range  0x00000000 0x00ffffff
"colorkey_mode" enum "disable", "Y component", "U component", "V component", 
"RGB", "R component", "G component", "B component"

All range properties store a RGB888 or YUV888 triplet.

The min and max properties store the comparison ranges. When a match occurs 
for one of the components, the value and alpha from the val and alpha 
properties replace the pixel. It's not clear which of the alpha "components" 
is used when a match occurs in RGB mode.

The colorkey property is a shortcut that stores identical values in min, max 
and val and 0 in alpha.

- i915

#define I915_SET_COLORKEY_NONE          (1<<0)
#define I915_SET_COLORKEY_DESTINATION   (1<<1)
#define I915_SET_COLORKEY_SOURCE        (1<<2)

struct drm_intel_sprite_colorkey {
        __u32 plane_id;
        __u32 min_value;
        __u32 channel_mask;
        __u32 max_value;
        __u32 flags;
};

- nouveau

"colorkey" range 0x00000000 0x01ffffff

The format isn't documented but it seems from the source code that bits 23:0 
store the color key value (written directly to a register, so possibly in a 
pixel format-dependent format) and bit 24 enables color keying.

- rcar-du

"colorkey" range 0x00000000 0x01ffffff

Bits 23:0 store the color key value in RGB888 (regardless of the pixel format 
of the plane) and bit 24 enables color keying. This supports Gen2 hardware 
only, where the only available more is  exact match. The pixel then becomes 
fully transparent (the hardware doesn't support custom target alpha values).

On Gen3 hardware color keying can be performed in exact RGB match, exact Y 
match or range Y match (only the max value is programmable, the min value is 
always 0). Furthermore in exact match modes the hardware can operate with a 
single match value, in which case it can then override the full ARGB or AYUV 
pixel, or double match value, in which case it can then override the alpha 
component only, but with two distinct match values each associated with a 
target alpha.

- vmwgfx

struct drm_vmw_control_stream_arg {
        __u32 stream_id;
        __u32 enabled;

        __u32 flags;
        __u32 color_key;

        __u32 handle;
        __u32 offset;
        __s32 format;
        __u32 size;
        __u32 width;
        __u32 height;
        __u32 pitch[3];

        __u32 pad64;
        struct drm_vmw_rect src;
        struct drm_vmw_rect dst;
};

The color_key field isn't documented, but the following (unused) macros hint 
that it could store an RGB888, with the color key feature enabled through the 
flags field.

#define SVGA_VIDEO_FLAG_COLORKEY        0x0001
#define SVGA_VIDEO_COLORKEY_MASK             0x00ffffff

> >> Does your colorkey end up forcing alpha to 1 for the plane when it's
> >> not matched?
> 
> I think this  behavior is HW dependent, on R-CAR Gen3, the alpha for pixel
> that don't match is not touch.
> 
> > I think generic color-key for plane compositioning would be nice, but I'm
> > not sure that's possible due to differences in how the key works.
> 
> I'm thinking of starting from the drivers that do have this property and see
> if there is any common ground for a generic color-key property/ies

Looking at the 5 drivers we have in mainline that support color keying one way 
or another, we can already see that the hardware implementations differ quite 
widely. There are however similarities, and we could express most of the above 
features through a set of generic properties similar to the ones already 
implemented by the armada driver.

- The match range could be set through minimum and maximum properties. Drivers 
that support exact match only would report an error with minimum != maximum.

- The replacement value could be set through a value property. The property 
could store both the pixel value (RGB or YUV) and the alpha value, or we could 
separate those in two properties. With a single property bits that are not 
applicable would be ignored (for instance RGB/YUV bits when the driver 
supports alpha replacement only). If programmable color replacement isn't 
supported (as in the R-Car Gen2 example above) the property could be omitted.

- The mode could be set through a mode property. Enabling color keying through 
one bit in a color property (like done by the nouveau and rcar-du drivers) is 
a hack and I don't think we should carry it forward. A mode property would 
also allow configuring source or destination color keying.

- Part of the mode information could be deduced automatically without a need 
to specify it explicitly. For instance RGB/YUV mode can be configured based on 
the pixel format of the plane. Similarly, exact match vs. range match can be 
configured based on whether the minimum and maximum value differ.

- Properties that store pixel values could either contain a value in a fixed 
format (e.g. RGB888) or in a format that varies depending on the pixel format 
(e.g. encoded in the same pixel format as the plane). I have a preference for 
the former but I'm open for discussions.

- We need to keep the existing "colorkey" properties implemented by armada, 
nouveau and rcar-du for backward compatibility reasons, but this proposed API 
doesn't require a "colorkey" property. We could however decide to standardize 
a "colorkey" property as a shortcut similar to what the armada driver does.


Some of the R-Car Gen3 hardware features wouldn't be supported by the above 
proposal. Selecting exact match vs. range match should be fine, with atomic 
commit returning an error if minimum != 0 && minimum != maximum. The dual 
target mode, however, can't really fit the properties proposed here. I don't 
have a use case for that mode at the moment but I'm fairly certain that 
someone will come up with one, at least if not for the R-Car for a different 
display engine that provides similarly exotic features.

We could also decide to standardize the color keying properties in a way that 
would balance complexity vs. hardware features support, and let drivers that 
need exotic features implement additional properties.

Ideas and comments are welcome, I'd like to start working on this fairly soon.

-- 
Regards,

Laurent Pinchart