[RFC/PATCH 0/5] Add live source objects to DRM

[RFC/PATCH 0/5] Add live source objects to DRM

[Laurent Pinchart]

Hello,

I have a feeling that RFC/PATCH will work better than just RFC, so here's a
patch set that adds a new object named live source to DRM.

The need comes from the Renesas R-Car SoCs in which a video processing engine
(named VSP1) that operates from memory to memory has one output directly
connected to a plane of the display engine (DU) without going through memory.

The VSP1 is supported by a V4L2 driver. While it could be argued that it
should instead be supported directly by the DRM rcar-du driver, this wouldn't
be a good idea for at least two reasons. First, the R-Car SoCs contain several
VSP1 instances, of which only a subset have a direct DU connection. The only
other instances operate solely in memory to memory mode. Then, the VSP1 is a
video processing engine and not a display engine. Its features are easily
supported by the V4L2 API, but don't map to the DRM/KMS API. Significant
changes to DRM/KMS would be required, beyond what is in my opinion an
acceptable scope for a display API.

Now that the need to interface two separate devices supported by two different
drivers in two separate subsystems has been established, we need an API to do
so. It should be noted that while that API doesn't exist in the mainline
kernel, the need isn't limited to Renesas SoCs.

This patch set proposes one possible solution for the problem in the form of a
new DRM object named live source. Live sources are created by drivers to model
hardware connections between a plane input and an external source, and are
attached to planes through the KMS userspace API.

Patch 1/5 adds live source objects to DRM, with an in-kernel API for drivers
to register the sources, and a userspace API to enumerate and configure them.
Configuring a live source sets the width, height and pixel format of the
video stream. This should ideally be queried directly from the driver that
supports the live source device, but I've decided not to implement such
communication between V4L2 and DRM/KMS at the moment to keep the proposal
simple.

Patch 2/2 implements connection between live sources and planes. This takes
different forms depending on whether drivers use the setplane or the atomic
updates API:

- For setplane, the fb field can now contain a live source ID in addition to
  a framebuffer ID. As DRM allocates object IDs from a single ID space, the
  type can be inferred from the ID. This makes specifying both a framebuffer
  and a live source impossible, which isn't an issue given that such a
  configuration would be invalid.

  The live source is looked up by the DRM core and passed as a pointer to the
  .update_plane() operation. Unlike framebuffers live sources are not
  refcounted as they're created statically at driver initialization time.

- For atomic update, a new SRC_ID property has been added to planes. The live
  source is looked up from the source ID and stored into the plane state.

Patches 3/5 to 5/5 then implement support for live sources in the R-Car DU
driver.

Nothing here is set in stone. One point I'm not sure about is whether live
sources support should be enabled for setplane or only for atomic updates.
Other parts of the API can certainly be modified as well, and I'm open for
totally different implementations as well.

Laurent Pinchart (5):
  drm: Add live source object
  drm: Connect live source to plane
  drm/rcar-du: Add VSP1 support to the planes allocator
  drm/rcar-du: Add VSP1 live source support
  drm/rcar-du: Restart the DU group when a plane source changes

 drivers/gpu/drm/armada/armada_overlay.c     |   2 +-
 drivers/gpu/drm/drm_atomic.c                |   7 +
 drivers/gpu/drm/drm_atomic_helper.c         |   4 +
 drivers/gpu/drm/drm_crtc.c                  | 365 ++++++++++++++++++++++++++--
 drivers/gpu/drm/drm_fops.c                  |   6 +-
 drivers/gpu/drm/drm_ioctl.c                 |   3 +
 drivers/gpu/drm/drm_plane_helper.c          |   1 +
 drivers/gpu/drm/exynos/exynos_drm_crtc.c    |   4 +-
 drivers/gpu/drm/exynos/exynos_drm_plane.c   |   3 +-
 drivers/gpu/drm/exynos/exynos_drm_plane.h   |   3 +-
 drivers/gpu/drm/i915/intel_display.c        |   4 +-
 drivers/gpu/drm/i915/intel_sprite.c         |   2 +-
 drivers/gpu/drm/imx/ipuv3-plane.c           |   3 +-
 drivers/gpu/drm/nouveau/dispnv04/overlay.c  |   6 +-
 drivers/gpu/drm/omapdrm/omap_plane.c        |   1 +
 drivers/gpu/drm/rcar-du/rcar_du_crtc.c      |  10 +-
 drivers/gpu/drm/rcar-du/rcar_du_drv.c       |   6 +-
 drivers/gpu/drm/rcar-du/rcar_du_drv.h       |   3 +
 drivers/gpu/drm/rcar-du/rcar_du_group.c     |  21 +-
 drivers/gpu/drm/rcar-du/rcar_du_group.h     |   2 +
 drivers/gpu/drm/rcar-du/rcar_du_kms.c       |  62 ++++-
 drivers/gpu/drm/rcar-du/rcar_du_plane.c     | 196 ++++++++++++---
 drivers/gpu/drm/rcar-du/rcar_du_plane.h     |  11 +
 drivers/gpu/drm/rcar-du/rcar_du_regs.h      |   1 +
 drivers/gpu/drm/rockchip/rockchip_drm_vop.c |   5 +-
 drivers/gpu/drm/shmobile/shmob_drm_plane.c  |   3 +-
 drivers/gpu/drm/sti/sti_drm_plane.c         |   3 +-
 drivers/gpu/drm/sti/sti_hqvdp.c             |   2 +-
 include/drm/drmP.h                          |   3 +
 include/drm/drm_atomic_helper.h             |   1 +
 include/drm/drm_crtc.h                      |  48 ++++
 include/drm/drm_plane_helper.h              |   1 +
 include/uapi/drm/drm.h                      |   4 +
 include/uapi/drm/drm_mode.h                 |  32 +++
 34 files changed, 728 insertions(+), 100 deletions(-)

-- 
Regards,

Laurent Pinchart

[RFC/PATCH 1/5] drm: Add live source object

[Laurent Pinchart]

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/gpu/drm/drm_crtc.c  | 231 ++++++++++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/drm_ioctl.c |   3 +
 include/drm/drm_crtc.h      |  42 ++++++++
 include/uapi/drm/drm.h      |   4 +
 include/uapi/drm/drm_mode.h |  32 ++++++
 5 files changed, 312 insertions(+)

diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index 5785336695ca..a510c9742a16 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -1261,6 +1261,60 @@ void drm_plane_cleanup(struct drm_plane *plane)
 }
 EXPORT_SYMBOL(drm_plane_cleanup);
 
+int drm_live_source_init(struct drm_device *dev, struct drm_live_source *src,
+			 const char *name, unsigned long possible_planes,
+			 const uint32_t *formats, uint32_t format_count,
+			 const struct drm_live_source_funcs *funcs)
+{
+	int ret;
+
+	drm_modeset_lock_all(dev);
+
+	ret = drm_mode_object_get(dev, &src->base, DRM_MODE_OBJECT_LIVE_SOURCE);
+	if (ret)
+		goto out;
+
+	src->dev = dev;
+	src->funcs = funcs;
+	if (name)
+		strlcpy(src->name, name, DRM_SOURCE_NAME_LEN);
+	src->possible_planes = possible_planes;
+
+	src->format_types = kmalloc(format_count * sizeof(*src->format_types),
+				    GFP_KERNEL);
+	if (!src->format_types) {
+		DRM_DEBUG_KMS("out of memory when allocating source foramts\n");
+		drm_mode_object_put(dev, &src->base);
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	memcpy(src->format_types, formats,
+	       format_count * sizeof(*src->format_types));
+	src->format_count = format_count;
+
+	list_add_tail(&src->head, &dev->mode_config.live_source_list);
+	dev->mode_config.num_live_source++;
+
+ out:
+	drm_modeset_unlock_all(dev);
+
+	return ret;
+}
+EXPORT_SYMBOL(drm_live_source_init);
+
+void drm_live_source_cleanup(struct drm_live_source *src)
+{
+	struct drm_device *dev = src->dev;
+
+	drm_modeset_lock_all(dev);
+	drm_mode_object_put(dev, &src->base);
+	list_del(&src->head);
+	dev->mode_config.num_live_source--;
+	drm_modeset_unlock_all(dev);
+}
+EXPORT_SYMBOL(drm_live_source_cleanup);
+
 /**
  * drm_plane_index - find the index of a registered plane
  * @plane: plane to find index for
@@ -2612,6 +2666,176 @@ int drm_mode_setplane(struct drm_device *dev, void *data,
 }
 
 /**
+ * drm_mode_getsource_res - get live source info
+ * @dev: DRM device
+ * @data: ioctl data
+ * @file_priv: DRM file info
+ *
+ * Return a live source and set of IDs.
+ */
+int drm_mode_getsource_res(struct drm_device *dev, void *data,
+			   struct drm_file *file_priv)
+{
+	struct drm_mode_get_live_source_res *src_resp = data;
+	struct drm_mode_config *config;
+	struct drm_live_source *src;
+	uint32_t __user *src_ptr;
+	int copied = 0, ret = 0;
+
+	if (!drm_core_check_feature(dev, DRIVER_MODESET))
+		return -EINVAL;
+
+	drm_modeset_lock_all(dev);
+	config = &dev->mode_config;
+
+	/*
+	 * This ioctl is called twice, once to determine how much space is
+	 * needed, and the 2nd time to fill it.
+	 */
+	if (config->num_live_source &&
+	    (src_resp->count_sources >= config->num_live_source)) {
+		src_ptr = (uint32_t __user *)(unsigned long)src_resp->source_id_ptr;
+
+		list_for_each_entry(src, &config->live_source_list, head) {
+			if (put_user(src->base.id, src_ptr + copied)) {
+				ret = -EFAULT;
+				goto out;
+			}
+			copied++;
+		}
+	}
+	src_resp->count_sources = config->num_live_source;
+
+out:
+	drm_modeset_unlock_all(dev);
+	return ret;
+}
+
+/**
+ * drm_mode_getsource - get live source info
+ * @dev: DRM device
+ * @data: ioctl data
+ * @file_priv: DRM file info
+ *
+ * Return live source info, including formats supported, ...
+ */
+int drm_mode_getsource(struct drm_device *dev, void *data,
+		       struct drm_file *file_priv)
+{
+	struct drm_mode_get_live_source *src_resp = data;
+	struct drm_mode_object *obj;
+	struct drm_live_source *src;
+	int ret = 0;
+
+	if (!drm_core_check_feature(dev, DRIVER_MODESET))
+		return -EINVAL;
+
+	obj = drm_mode_object_find(dev, src_resp->source_id,
+				   DRM_MODE_OBJECT_LIVE_SOURCE);
+	if (!obj)
+		return -ENOENT;
+	src = obj_to_live_source(obj);
+
+	src_resp->source_id = src->base.id;
+	strlcpy(src_resp->name, src->name, DRM_SOURCE_NAME_LEN);
+	src_resp->possible_planes = src->possible_planes;
+
+	drm_modeset_lock_all(dev);
+
+	if (src->plane)
+		src_resp->plane_id = src->plane->base.id;
+	else
+		src_resp->plane_id = 0;
+
+	src_resp->width = src->width;
+	src_resp->height = src->height;
+	src_resp->pixel_format = src->pixel_format;
+
+	/*
+	 * This ioctl is called twice, once to determine how much space is
+	 * needed, and the 2nd time to fill it.
+	 */
+	if (src->format_count &&
+	    (src_resp->count_format_types >= src->format_count)) {
+		uint32_t __user *format_ptr;
+
+		format_ptr = (uint32_t __user *)(unsigned long)src_resp->format_type_ptr;
+		if (copy_to_user(format_ptr, src->format_types,
+				 sizeof(uint32_t) * src->format_count)) {
+			ret = -EFAULT;
+			goto out;
+		}
+	}
+	src_resp->count_format_types = src->format_count;
+
+out:
+	drm_modeset_unlock_all(dev);
+	return ret;
+}
+
+/**
+ * drm_mode_setsource - set up or tear down a live source
+ * @dev: DRM device
+ * @data: ioctl data*
+ * @file_priv: DRM file info
+ *
+ * Set live source info, including plane, and other factors.
+ * Or pass a NULL plane to disable.
+ */
+int drm_mode_setsource(struct drm_device *dev, void *data,
+		       struct drm_file *file_priv)
+{
+	struct drm_mode_set_live_source *src_req = data;
+	struct drm_mode_object *obj;
+	struct drm_live_source *src;
+	unsigned int i;
+	int ret = 0;
+
+	if (!drm_core_check_feature(dev, DRIVER_MODESET))
+		return -EINVAL;
+
+	/*
+	 * First, find the live source and plane objects. If not available, we
+	 * don't bother to call the driver.
+	 */
+	obj = drm_mode_object_find(dev, src_req->source_id,
+				   DRM_MODE_OBJECT_LIVE_SOURCE);
+	if (!obj) {
+		DRM_DEBUG_KMS("Unknown live source ID %d\n",
+			      src_req->source_id);
+		return -ENOENT;
+	}
+	src = obj_to_live_source(obj);
+
+	drm_modeset_lock_all(dev);
+
+	if (src->plane) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	/* Check whether this source supports the pixel format. */
+	for (i = 0; i < src->format_count; i++)
+		if (src_req->pixel_format == src->format_types[i])
+			break;
+
+	if (i == src->format_count) {
+		DRM_DEBUG_KMS("Invalid pixel format 0x%08x for source %u\n",
+			      src_req->pixel_format, src->base.id);
+		ret = -EINVAL;
+		goto out;
+	}
+
+	src->width = src_req->width;
+	src->height = src_req->height;
+	src->pixel_format = src_req->pixel_format;
+
+out:
+	drm_modeset_unlock_all(dev);
+	return ret;
+}
+
+/**
  * drm_mode_set_config_internal - helper to call ->set_config
  * @set: modeset config to set
  *
@@ -5429,6 +5653,7 @@ void drm_mode_config_init(struct drm_device *dev)
 	INIT_LIST_HEAD(&dev->mode_config.property_list);
 	INIT_LIST_HEAD(&dev->mode_config.property_blob_list);
 	INIT_LIST_HEAD(&dev->mode_config.plane_list);
+	INIT_LIST_HEAD(&dev->mode_config.live_source_list);
 	idr_init(&dev->mode_config.crtc_idr);
 	idr_init(&dev->mode_config.tile_idr);
 
@@ -5468,6 +5693,7 @@ void drm_mode_config_cleanup(struct drm_device *dev)
 	struct drm_property *property, *pt;
 	struct drm_property_blob *blob, *bt;
 	struct drm_plane *plane, *plt;
+	struct drm_live_source *src, *psrc;
 
 	list_for_each_entry_safe(encoder, enct, &dev->mode_config.encoder_list,
 				 head) {
@@ -5507,6 +5733,11 @@ void drm_mode_config_cleanup(struct drm_device *dev)
 		plane->funcs->destroy(plane);
 	}
 
+	list_for_each_entry_safe(src, psrc, &dev->mode_config.live_source_list,
+				 head) {
+		src->funcs->destroy(src);
+	}
+
 	list_for_each_entry_safe(crtc, ct, &dev->mode_config.crtc_list, head) {
 		crtc->funcs->destroy(crtc);
 	}
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
index a6d773a61c2d..2cb400a8bc8d 100644
--- a/drivers/gpu/drm/drm_ioctl.c
+++ b/drivers/gpu/drm/drm_ioctl.c
@@ -609,10 +609,13 @@ static const struct drm_ioctl_desc drm_ioctls[] = {
 	DRM_IOCTL_DEF(DRM_IOCTL_PRIME_FD_TO_HANDLE, drm_prime_fd_to_handle_ioctl, DRM_AUTH|DRM_UNLOCKED|DRM_RENDER_ALLOW),
 
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETPLANERESOURCES, drm_mode_getplane_res, DRM_CONTROL_ALLOW|DRM_UNLOCKED),
+	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETSOURCERESOURCES, drm_mode_getsource_res, DRM_CONTROL_ALLOW|DRM_UNLOCKED),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETCRTC, drm_mode_getcrtc, DRM_CONTROL_ALLOW|DRM_UNLOCKED),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_SETCRTC, drm_mode_setcrtc, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETPLANE, drm_mode_getplane, DRM_CONTROL_ALLOW|DRM_UNLOCKED),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_SETPLANE, drm_mode_setplane, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
+	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETSOURCE, drm_mode_getsource, DRM_CONTROL_ALLOW|DRM_UNLOCKED),
+	DRM_IOCTL_DEF(DRM_IOCTL_MODE_SETSOURCE, drm_mode_setsource, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_CURSOR, drm_mode_cursor_ioctl, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_GETGAMMA, drm_mode_gamma_get_ioctl, DRM_UNLOCKED),
 	DRM_IOCTL_DEF(DRM_IOCTL_MODE_SETGAMMA, drm_mode_gamma_set_ioctl, DRM_MASTER|DRM_UNLOCKED),
diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h
index adc9ea5acf02..775a9a84c5bf 100644
--- a/include/drm/drm_crtc.h
+++ b/include/drm/drm_crtc.h
@@ -54,6 +54,7 @@ struct fence;
 #define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb
 #define DRM_MODE_OBJECT_PLANE 0xeeeeeeee
 #define DRM_MODE_OBJECT_BRIDGE 0xbdbdbdbd
+#define DRM_MODE_OBJECT_LIVE_SOURCE 0xe1e1e1e1
 #define DRM_MODE_OBJECT_ANY 0
 
 struct drm_mode_object {
@@ -247,6 +248,7 @@ struct drm_pending_vblank_event;
 struct drm_plane;
 struct drm_bridge;
 struct drm_atomic_state;
+struct drm_live_source;
 
 /**
  * struct drm_crtc_state - mutable CRTC state
@@ -869,6 +871,30 @@ struct drm_plane {
 	struct drm_plane_state *state;
 };
 
+struct drm_live_source_funcs {
+	void (*destroy)(struct drm_live_source *src);
+};
+
+struct drm_live_source {
+	struct drm_device *dev;
+	struct list_head head;
+
+	struct drm_mode_object base;
+
+	char name[DRM_SOURCE_NAME_LEN];
+
+	uint32_t possible_planes;
+	uint32_t *format_types;
+	uint32_t format_count;
+
+	struct drm_plane *plane;
+	unsigned int width;
+	unsigned int height;
+	uint32_t pixel_format;
+
+	const struct drm_live_source_funcs *funcs;
+};
+
 /**
  * struct drm_bridge_funcs - drm_bridge control functions
  * @attach: Called during drm_bridge_attach
@@ -1087,6 +1113,8 @@ struct drm_mode_config {
 	int num_overlay_plane;
 	int num_total_plane;
 	struct list_head plane_list;
+	int num_live_source;
+	struct list_head live_source_list;
 
 	int num_crtc;
 	struct list_head crtc_list;
@@ -1186,6 +1214,7 @@ struct drm_mode_config {
 #define obj_to_property(x) container_of(x, struct drm_property, base)
 #define obj_to_blob(x) container_of(x, struct drm_property_blob, base)
 #define obj_to_plane(x) container_of(x, struct drm_plane, base)
+#define obj_to_live_source(x) container_of(x, struct drm_live_source, base)
 
 struct drm_prop_enum_list {
 	int type;
@@ -1276,6 +1305,13 @@ extern int drm_crtc_check_viewport(const struct drm_crtc *crtc,
 
 extern void drm_encoder_cleanup(struct drm_encoder *encoder);
 
+extern int drm_live_source_init(struct drm_device *dev,
+				struct drm_live_source *src, const char *name,
+				unsigned long possible_planes,
+				const uint32_t *formats, uint32_t format_count,
+				const struct drm_live_source_funcs *funcs);
+extern void drm_live_source_cleanup(struct drm_live_source *src);
+
 extern const char *drm_get_connector_status_name(enum drm_connector_status status);
 extern const char *drm_get_subpixel_order_name(enum subpixel_order order);
 extern const char *drm_get_dpms_name(int val);
@@ -1391,6 +1427,8 @@ extern int drm_mode_getresources(struct drm_device *dev,
 				 void *data, struct drm_file *file_priv);
 extern int drm_mode_getplane_res(struct drm_device *dev, void *data,
 				   struct drm_file *file_priv);
+extern int drm_mode_getsource_res(struct drm_device *dev, void *data,
+				  struct drm_file *file_priv);
 extern int drm_mode_getcrtc(struct drm_device *dev,
 			    void *data, struct drm_file *file_priv);
 extern int drm_mode_getconnector(struct drm_device *dev,
@@ -1402,6 +1440,10 @@ extern int drm_mode_getplane(struct drm_device *dev,
 			       void *data, struct drm_file *file_priv);
 extern int drm_mode_setplane(struct drm_device *dev,
 			       void *data, struct drm_file *file_priv);
+extern int drm_mode_getsource(struct drm_device *dev,
+			      void *data, struct drm_file *file_priv);
+extern int drm_mode_setsource(struct drm_device *dev,
+			      void *data, struct drm_file *file_priv);
 extern int drm_mode_cursor_ioctl(struct drm_device *dev,
 				void *data, struct drm_file *file_priv);
 extern int drm_mode_cursor2_ioctl(struct drm_device *dev,
diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h
index ff6ef62d084b..d60cb8fa034c 100644
--- a/include/uapi/drm/drm.h
+++ b/include/uapi/drm/drm.h
@@ -787,6 +787,10 @@ struct drm_prime_handle {
 #define DRM_IOCTL_MODE_CURSOR2		DRM_IOWR(0xBB, struct drm_mode_cursor2)
 #define DRM_IOCTL_MODE_ATOMIC		DRM_IOWR(0xBC, struct drm_mode_atomic)
 
+#define DRM_IOCTL_MODE_GETSOURCERESOURCES DRM_IOWR(0xBC, struct drm_mode_get_live_source_res)
+#define DRM_IOCTL_MODE_GETSOURCE	DRM_IOWR(0xBD, struct drm_mode_get_live_source)
+#define DRM_IOCTL_MODE_SETSOURCE	DRM_IOWR(0xBE, struct drm_mode_set_live_source)
+
 /**
  * Device specific ioctls should only be in their respective headers
  * The device specific ioctl range is from 0x40 to 0x9f.
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index dbeba949462a..ecc1fa3000f4 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -33,6 +33,7 @@
 #define DRM_CONNECTOR_NAME_LEN	32
 #define DRM_DISPLAY_MODE_LEN	32
 #define DRM_PROP_NAME_LEN	32
+#define DRM_SOURCE_NAME_LEN	32
 
 #define DRM_MODE_TYPE_BUILTIN	(1<<0)
 #define DRM_MODE_TYPE_CLOCK_C	((1<<1) | DRM_MODE_TYPE_BUILTIN)
@@ -179,6 +180,37 @@ struct drm_mode_get_plane_res {
 	__u32 count_planes;
 };
 
+struct drm_mode_set_live_source {
+	__u32 source_id;
+
+	__u32 plane_id;
+
+	__u32 width;
+	__u32 height;
+	__u32 pixel_format;
+};
+
+struct drm_mode_get_live_source {
+	__u32 source_id;
+	char name[DRM_SOURCE_NAME_LEN];
+
+	__u32 plane_id;
+
+	__u32 possible_planes;
+
+	__u32 count_format_types;
+	__u64 format_type_ptr;
+
+	__u32 width;
+	__u32 height;
+	__u32 pixel_format;
+};
+
+struct drm_mode_get_live_source_res {
+	__u64 source_id_ptr;
+	__u32 count_sources;
+};
+
 #define DRM_MODE_ENCODER_NONE	0
 #define DRM_MODE_ENCODER_DAC	1
 #define DRM_MODE_ENCODER_TMDS	2
-- 
2.0.5

[RFC/PATCH 2/5] drm: Connect live source to plane

[Laurent Pinchart]

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/gpu/drm/armada/armada_overlay.c     |   2 +-
 drivers/gpu/drm/drm_atomic.c                |   7 ++
 drivers/gpu/drm/drm_atomic_helper.c         |   4 +
 drivers/gpu/drm/drm_crtc.c                  | 134 +++++++++++++++++++++++-----
 drivers/gpu/drm/drm_fops.c                  |   6 +-
 drivers/gpu/drm/drm_plane_helper.c          |   1 +
 drivers/gpu/drm/exynos/exynos_drm_crtc.c    |   4 +-
 drivers/gpu/drm/exynos/exynos_drm_plane.c   |   3 +-
 drivers/gpu/drm/exynos/exynos_drm_plane.h   |   3 +-
 drivers/gpu/drm/i915/intel_display.c        |   4 +-
 drivers/gpu/drm/i915/intel_sprite.c         |   2 +-
 drivers/gpu/drm/imx/ipuv3-plane.c           |   3 +-
 drivers/gpu/drm/nouveau/dispnv04/overlay.c  |   6 +-
 drivers/gpu/drm/omapdrm/omap_plane.c        |   1 +
 drivers/gpu/drm/rockchip/rockchip_drm_vop.c |   5 +-
 drivers/gpu/drm/shmobile/shmob_drm_plane.c  |   3 +-
 drivers/gpu/drm/sti/sti_drm_plane.c         |   3 +-
 drivers/gpu/drm/sti/sti_hqvdp.c             |   2 +-
 include/drm/drmP.h                          |   3 +
 include/drm/drm_atomic_helper.h             |   1 +
 include/drm/drm_crtc.h                      |   6 ++
 include/drm/drm_plane_helper.h              |   1 +
 22 files changed, 165 insertions(+), 39 deletions(-)

diff --git a/drivers/gpu/drm/armada/armada_overlay.c b/drivers/gpu/drm/armada/armada_overlay.c
index c5b06fdb459c..4d5708bbee6e 100644
--- a/drivers/gpu/drm/armada/armada_overlay.c
+++ b/drivers/gpu/drm/armada/armada_overlay.c
@@ -99,7 +99,7 @@ static unsigned armada_limit(int start, unsigned size, unsigned max)
 
 static int
 armada_plane_update(struct drm_plane *plane, struct drm_crtc *crtc,
-	struct drm_framebuffer *fb,
+	struct drm_framebuffer *fb, struct drm_live_source *src,
 	int crtc_x, int crtc_y, unsigned crtc_w, unsigned crtc_h,
 	uint32_t src_x, uint32_t src_y, uint32_t src_w, uint32_t src_h)
 {
diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index a6caaae40b9e..0be059be2d58 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -384,6 +384,11 @@ int drm_atomic_plane_set_property(struct drm_plane *plane,
 		drm_atomic_set_fb_for_plane(state, fb);
 		if (fb)
 			drm_framebuffer_unreference(fb);
+	} else if (property == config->prop_src_id) {
+		struct drm_mode_object *obj;
+		obj = drm_mode_object_find(dev, val,
+					   DRM_MODE_OBJECT_LIVE_SOURCE);
+		state->src = obj ? obj_to_live_source(obj) : NULL;
 	} else if (property == config->prop_crtc_id) {
 		struct drm_crtc *crtc = drm_crtc_find(dev, val);
 		return drm_atomic_set_crtc_for_plane(state, crtc);
@@ -432,6 +437,8 @@ drm_atomic_plane_get_property(struct drm_plane *plane,
 
 	if (property == config->prop_fb_id) {
 		*val = (state->fb) ? state->fb->base.id : 0;
+	} else if (property == config->prop_src_id) {
+		*val = (state->src) ? state->src->base.id : 0;
 	} else if (property == config->prop_crtc_id) {
 		*val = (state->crtc) ? state->crtc->base.id : 0;
 	} else if (property == config->prop_crtc_x) {
diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
index a7458813af2b..b182d9e6abba 100644
--- a/drivers/gpu/drm/drm_atomic_helper.c
+++ b/drivers/gpu/drm/drm_atomic_helper.c
@@ -1342,6 +1342,7 @@ EXPORT_SYMBOL(drm_atomic_helper_swap_state);
  * @plane: plane object to update
  * @crtc: owning CRTC of owning plane
  * @fb: framebuffer to flip onto plane
+ * @src: live source to connect to the plane
  * @crtc_x: x offset of primary plane on crtc
  * @crtc_y: y offset of primary plane on crtc
  * @crtc_w: width of primary plane rectangle on crtc
@@ -1359,6 +1360,7 @@ EXPORT_SYMBOL(drm_atomic_helper_swap_state);
 int drm_atomic_helper_update_plane(struct drm_plane *plane,
 				   struct drm_crtc *crtc,
 				   struct drm_framebuffer *fb,
+				   struct drm_live_source *src,
 				   int crtc_x, int crtc_y,
 				   unsigned int crtc_w, unsigned int crtc_h,
 				   uint32_t src_x, uint32_t src_y,
@@ -1384,6 +1386,7 @@ retry:
 	if (ret != 0)
 		goto fail;
 	drm_atomic_set_fb_for_plane(plane_state, fb);
+	plane_state->src = src;
 	plane_state->crtc_x = crtc_x;
 	plane_state->crtc_y = crtc_y;
 	plane_state->crtc_h = crtc_h;
@@ -1466,6 +1469,7 @@ retry:
 	if (ret != 0)
 		goto fail;
 	drm_atomic_set_fb_for_plane(plane_state, NULL);
+	plane_state->src = NULL;
 	plane_state->crtc_x = 0;
 	plane_state->crtc_y = 0;
 	plane_state->crtc_h = 0;
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index a510c9742a16..f11960372307 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -1183,6 +1183,7 @@ int drm_universal_plane_init(struct drm_device *dev, struct drm_plane *plane,
 
 	if (drm_core_check_feature(dev, DRIVER_ATOMIC)) {
 		drm_object_attach_property(&plane->base, config->prop_fb_id, 0);
+		drm_object_attach_property(&plane->base, config->prop_src_id, 0);
 		drm_object_attach_property(&plane->base, config->prop_crtc_id, 0);
 		drm_object_attach_property(&plane->base, config->prop_crtc_x, 0);
 		drm_object_attach_property(&plane->base, config->prop_crtc_y, 0);
@@ -1315,6 +1316,31 @@ void drm_live_source_cleanup(struct drm_live_source *src)
 }
 EXPORT_SYMBOL(drm_live_source_cleanup);
 
+void drm_live_sources_release(struct drm_file *priv)
+{
+	struct drm_device *dev = priv->minor->dev;
+	struct drm_live_source *src, *next;
+	int ret;
+
+	drm_modeset_lock_all(dev);
+	mutex_lock(&priv->sources_lock);
+
+	list_for_each_entry_safe(src, next, &priv->sources, filp_head) {
+		list_del_init(&src->filp_head);
+		if (src->plane) {
+			ret = src->plane->funcs->disable_plane(src->plane);
+			if (ret)
+				DRM_ERROR("failed to disable plane with busy source\n");
+			src->plane->src = NULL;
+			src->plane->crtc = NULL;
+			src->plane = NULL;
+		}
+	}
+
+	mutex_unlock(&priv->sources_lock);
+	drm_modeset_unlock_all(dev);
+}
+
 /**
  * drm_plane_index - find the index of a registered plane
  * @plane: plane to find index for
@@ -1468,6 +1494,12 @@ static int drm_mode_create_standard_properties(struct drm_device *dev)
 	dev->mode_config.prop_fb_id = prop;
 
 	prop = drm_property_create_object(dev, DRM_MODE_PROP_ATOMIC,
+			"SRC_ID", DRM_MODE_OBJECT_FB);
+	if (!prop)
+		return -ENOMEM;
+	dev->mode_config.prop_src_id = prop;
+
+	prop = drm_property_create_object(dev, DRM_MODE_PROP_ATOMIC,
 			"CRTC_ID", DRM_MODE_OBJECT_CRTC);
 	if (!prop)
 		return -ENOMEM;
@@ -2436,6 +2468,8 @@ int drm_mode_getplane(struct drm_device *dev, void *data,
 
 	if (plane->fb)
 		plane_resp->fb_id = plane->fb->base.id;
+	else if (plane->src)
+		plane_resp->fb_id = plane->src->base.id;
 	else
 		plane_resp->fb_id = 0;
 	drm_modeset_unlock(&plane->mutex);
@@ -2492,25 +2526,29 @@ int drm_plane_check_pixel_format(const struct drm_plane *plane, u32 format)
  *
  * src_{x,y,w,h} are provided in 16.16 fixed point format
  */
-static int __setplane_internal(struct drm_plane *plane,
+static int __setplane_internal(struct drm_file *file_priv,
+			       struct drm_plane *plane,
 			       struct drm_crtc *crtc,
 			       struct drm_framebuffer *fb,
+			       struct drm_live_source *src,
 			       int32_t crtc_x, int32_t crtc_y,
 			       uint32_t crtc_w, uint32_t crtc_h,
 			       /* src_{x,y,w,h} values are 16.16 fixed point */
 			       uint32_t src_x, uint32_t src_y,
 			       uint32_t src_w, uint32_t src_h)
 {
+	unsigned int width, height;
+	uint32_t pixel_format;
 	int ret = 0;
-	unsigned int fb_width, fb_height;
 
-	/* No fb means shut it down */
-	if (!fb) {
+	/* No fb and src means shut it down */
+	if (!fb && !src) {
 		plane->old_fb = plane->fb;
 		ret = plane->funcs->disable_plane(plane);
 		if (!ret) {
 			plane->crtc = NULL;
 			plane->fb = NULL;
+			plane->src = NULL;
 		} else {
 			plane->old_fb = NULL;
 		}
@@ -2524,22 +2562,50 @@ static int __setplane_internal(struct drm_plane *plane,
 		goto out;
 	}
 
-	/* Check whether this plane supports the fb pixel format. */
-	ret = drm_plane_check_pixel_format(plane, fb->pixel_format);
+	/* Check whether this plane supports the pixel format. */
+	pixel_format = fb ? fb->pixel_format : src->pixel_format;
+	ret = drm_plane_check_pixel_format(plane, pixel_format);
 	if (ret) {
 		DRM_DEBUG_KMS("Invalid pixel format %s\n",
-			      drm_get_format_name(fb->pixel_format));
+			      drm_get_format_name(pixel_format));
 		goto out;
 	}
 
-	fb_width = fb->width << 16;
-	fb_height = fb->height << 16;
+	/* Check whether the source can be associated with the plane. */
+	if (src) {
+		struct drm_device *dev = plane->dev;
+		unsigned int plane_mask = 1;
+		struct drm_plane *p;
+
+		if (src->plane && src->plane != plane) {
+			ret = -EBUSY;
+			goto out;
+		}
+
+		list_for_each_entry(p, &dev->mode_config.plane_list, head) {
+			if (p == plane)
+				break;
+			plane_mask <<= 1;
+		}
+
+		if (!(src->possible_planes & plane_mask)) {
+			DRM_DEBUG_KMS("Invalid source for plane\n");
+			ret = -EINVAL;
+			goto out;
+		}
+	}
+
+	/* Make sure source coordinates are inside the source. */
+	if (fb) {
+		width = fb->width << 16;
+		height = fb->height << 16;
+	} else {
+		width = src->width << 16;
+		height = src->height << 16;
+	}
 
-	/* Make sure source coordinates are inside the fb. */
-	if (src_w > fb_width ||
-	    src_x > fb_width - src_w ||
-	    src_h > fb_height ||
-	    src_y > fb_height - src_h) {
+	if (src_w > width || src_x > width - src_w ||
+	    src_h > height || src_y > height - src_h) {
 		DRM_DEBUG_KMS("Invalid source coordinates "
 			      "%u.%06ux%u.%06u+%u.%06u+%u.%06u\n",
 			      src_w >> 16, ((src_w & 0xffff) * 15625) >> 10,
@@ -2551,12 +2617,24 @@ static int __setplane_internal(struct drm_plane *plane,
 	}
 
 	plane->old_fb = plane->fb;
-	ret = plane->funcs->update_plane(plane, crtc, fb,
+	ret = plane->funcs->update_plane(plane, crtc, fb, src,
 					 crtc_x, crtc_y, crtc_w, crtc_h,
 					 src_x, src_y, src_w, src_h);
 	if (!ret) {
+		mutex_lock(&file_priv->sources_lock);
+		if (src) {
+			list_add_tail(&src->filp_head, &file_priv->sources);
+			src->plane = plane;
+		}
+		if (plane->src) {
+			list_del(&plane->src->filp_head);
+			plane->src->plane = NULL;
+		}
+		mutex_unlock(&file_priv->sources_lock);
+
 		plane->crtc = crtc;
 		plane->fb = fb;
+		plane->src = src;
 		fb = NULL;
 	} else {
 		plane->old_fb = NULL;
@@ -2572,9 +2650,11 @@ out:
 	return ret;
 }
 
-static int setplane_internal(struct drm_plane *plane,
+static int setplane_internal(struct drm_file *file_priv,
+			     struct drm_plane *plane,
 			     struct drm_crtc *crtc,
 			     struct drm_framebuffer *fb,
+			     struct drm_live_source *src,
 			     int32_t crtc_x, int32_t crtc_y,
 			     uint32_t crtc_w, uint32_t crtc_h,
 			     /* src_{x,y,w,h} values are 16.16 fixed point */
@@ -2584,7 +2664,7 @@ static int setplane_internal(struct drm_plane *plane,
 	int ret;
 
 	drm_modeset_lock_all(plane->dev);
-	ret = __setplane_internal(plane, crtc, fb,
+	ret = __setplane_internal(file_priv, plane, crtc, fb, src,
 				  crtc_x, crtc_y, crtc_w, crtc_h,
 				  src_x, src_y, src_w, src_h);
 	drm_modeset_unlock_all(plane->dev);
@@ -2612,6 +2692,7 @@ int drm_mode_setplane(struct drm_device *dev, void *data,
 	struct drm_plane *plane;
 	struct drm_crtc *crtc = NULL;
 	struct drm_framebuffer *fb = NULL;
+	struct drm_live_source *src = NULL;
 
 	if (!drm_core_check_feature(dev, DRIVER_MODESET))
 		return -EINVAL;
@@ -2628,8 +2709,8 @@ int drm_mode_setplane(struct drm_device *dev, void *data,
 	}
 
 	/*
-	 * First, find the plane, crtc, and fb objects.  If not available,
-	 * we don't bother to call the driver.
+	 * First, find the plane, crtc, and fb or source objects. If not
+	 * available, we don't bother to call the driver.
 	 */
 	plane = drm_plane_find(dev, plane_req->plane_id);
 	if (!plane) {
@@ -2641,7 +2722,16 @@ int drm_mode_setplane(struct drm_device *dev, void *data,
 	if (plane_req->fb_id) {
 		fb = drm_framebuffer_lookup(dev, plane_req->fb_id);
 		if (!fb) {
-			DRM_DEBUG_KMS("Unknown framebuffer ID %d\n",
+			struct drm_mode_object *obj;
+
+			obj = drm_mode_object_find(dev, plane_req->fb_id,
+						   DRM_MODE_OBJECT_LIVE_SOURCE);
+			if (obj)
+				src = obj_to_live_source(obj);
+		}
+
+		if (!fb && !src) {
+			DRM_DEBUG_KMS("Unknown framebuffer or live source ID %d\n",
 				      plane_req->fb_id);
 			return -ENOENT;
 		}
@@ -2658,7 +2748,7 @@ int drm_mode_setplane(struct drm_device *dev, void *data,
 	 * setplane_internal will take care of deref'ing either the old or new
 	 * framebuffer depending on success.
 	 */
-	return setplane_internal(plane, crtc, fb,
+	return setplane_internal(file_priv, plane, crtc, fb, src,
 				 plane_req->crtc_x, plane_req->crtc_y,
 				 plane_req->crtc_w, plane_req->crtc_h,
 				 plane_req->src_x, plane_req->src_y,
@@ -3199,7 +3289,7 @@ static int drm_mode_cursor_universal(struct drm_crtc *crtc,
 	 * setplane_internal will take care of deref'ing either the old or new
 	 * framebuffer depending on success.
 	 */
-	ret = __setplane_internal(crtc->cursor, crtc, fb,
+	ret = __setplane_internal(file_priv, crtc->cursor, crtc, fb, NULL,
 				crtc_x, crtc_y, crtc_w, crtc_h,
 				0, 0, src_w, src_h);
 
diff --git a/drivers/gpu/drm/drm_fops.c b/drivers/gpu/drm/drm_fops.c
index 076dd606b580..28591acf7e7a 100644
--- a/drivers/gpu/drm/drm_fops.c
+++ b/drivers/gpu/drm/drm_fops.c
@@ -167,6 +167,8 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor)
 	INIT_LIST_HEAD(&priv->lhead);
 	INIT_LIST_HEAD(&priv->fbs);
 	mutex_init(&priv->fbs_lock);
+	INIT_LIST_HEAD(&priv->sources);
+	mutex_init(&priv->sources_lock);
 	INIT_LIST_HEAD(&priv->event_list);
 	init_waitqueue_head(&priv->event_wait);
 	priv->event_space = 4096; /* set aside 4k for event buffer */
@@ -408,8 +410,10 @@ int drm_release(struct inode *inode, struct file *filp)
 
 	drm_events_release(file_priv);
 
-	if (drm_core_check_feature(dev, DRIVER_MODESET))
+	if (drm_core_check_feature(dev, DRIVER_MODESET)) {
 		drm_fb_release(file_priv);
+		drm_live_sources_release(file_priv);
+	}
 
 	if (drm_core_check_feature(dev, DRIVER_GEM))
 		drm_gem_release(dev, file_priv);
diff --git a/drivers/gpu/drm/drm_plane_helper.c b/drivers/gpu/drm/drm_plane_helper.c
index b62b03635050..09bf8d9ba580 100644
--- a/drivers/gpu/drm/drm_plane_helper.c
+++ b/drivers/gpu/drm/drm_plane_helper.c
@@ -222,6 +222,7 @@ EXPORT_SYMBOL(drm_plane_helper_check_update);
  */
 int drm_primary_helper_update(struct drm_plane *plane, struct drm_crtc *crtc,
 			      struct drm_framebuffer *fb,
+			      struct drm_live_source *lsrc,
 			      int crtc_x, int crtc_y,
 			      unsigned int crtc_w, unsigned int crtc_h,
 			      uint32_t src_x, uint32_t src_y,
diff --git a/drivers/gpu/drm/exynos/exynos_drm_crtc.c b/drivers/gpu/drm/exynos/exynos_drm_crtc.c
index 48ccab7fdf63..95ded0286eb9 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_crtc.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_crtc.c
@@ -127,7 +127,7 @@ static int exynos_drm_crtc_mode_set_base(struct drm_crtc *crtc, int x, int y,
 	crtc_w = fb->width - x;
 	crtc_h = fb->height - y;
 
-	return exynos_update_plane(crtc->primary, crtc, fb, 0, 0,
+	return exynos_update_plane(crtc->primary, crtc, fb, NULL, 0, 0,
 				   crtc_w, crtc_h, x, y, crtc_w, crtc_h);
 }
 
@@ -201,7 +201,7 @@ static int exynos_drm_crtc_page_flip(struct drm_crtc *crtc,
 		crtc->primary->fb = fb;
 		crtc_w = fb->width - crtc->x;
 		crtc_h = fb->height - crtc->y;
-		ret = exynos_update_plane(crtc->primary, crtc, fb, 0, 0,
+		ret = exynos_update_plane(crtc->primary, crtc, fb, NULL, 0, 0,
 					  crtc_w, crtc_h, crtc->x, crtc->y,
 					  crtc_w, crtc_h);
 		if (ret) {
diff --git a/drivers/gpu/drm/exynos/exynos_drm_plane.c b/drivers/gpu/drm/exynos/exynos_drm_plane.c
index a5616872eee7..693bfbf6b868 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_plane.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_plane.c
@@ -146,7 +146,8 @@ void exynos_plane_mode_set(struct drm_plane *plane, struct drm_crtc *crtc,
 
 int
 exynos_update_plane(struct drm_plane *plane, struct drm_crtc *crtc,
-		     struct drm_framebuffer *fb, int crtc_x, int crtc_y,
+		     struct drm_framebuffer *fb, struct drm_live_source *src,
+		     int crtc_x, int crtc_y,
 		     unsigned int crtc_w, unsigned int crtc_h,
 		     uint32_t src_x, uint32_t src_y,
 		     uint32_t src_w, uint32_t src_h)
diff --git a/drivers/gpu/drm/exynos/exynos_drm_plane.h b/drivers/gpu/drm/exynos/exynos_drm_plane.h
index 9d3c374e7b3e..bffb282c4860 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_plane.h
+++ b/drivers/gpu/drm/exynos/exynos_drm_plane.h
@@ -16,7 +16,8 @@ void exynos_plane_mode_set(struct drm_plane *plane, struct drm_crtc *crtc,
 			   uint32_t src_x, uint32_t src_y,
 			   uint32_t src_w, uint32_t src_h);
 int exynos_update_plane(struct drm_plane *plane, struct drm_crtc *crtc,
-			struct drm_framebuffer *fb, int crtc_x, int crtc_y,
+			struct drm_framebuffer *fb, struct drm_live_source *src,
+			int crtc_x, int crtc_y,
 			unsigned int crtc_w, unsigned int crtc_h,
 			uint32_t src_x, uint32_t src_y,
 			uint32_t src_w, uint32_t src_h);
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 1aa1cbd16c19..3b72dc9a5676 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11242,7 +11242,7 @@ static int __intel_set_mode(struct drm_crtc *crtc,
 
 		drm_crtc_get_hv_timing(mode, &hdisplay, &vdisplay);
 		ret = primary->funcs->update_plane(primary, &intel_crtc->base,
-						   fb, 0, 0,
+						   fb, NULL, 0, 0,
 						   hdisplay, vdisplay,
 						   x << 16, y << 16,
 						   hdisplay << 16, vdisplay << 16);
@@ -11717,7 +11717,7 @@ static int intel_crtc_set_config(struct drm_mode_set *set)
 
 		drm_crtc_get_hv_timing(set->mode, &hdisplay, &vdisplay);
 		ret = primary->funcs->update_plane(primary, set->crtc, set->fb,
-						   0, 0, hdisplay, vdisplay,
+						   NULL, 0, 0, hdisplay, vdisplay,
 						   set->x << 16, set->y << 16,
 						   hdisplay << 16, vdisplay << 16);
 
diff --git a/drivers/gpu/drm/i915/intel_sprite.c b/drivers/gpu/drm/i915/intel_sprite.c
index 7051da7015d3..be382d02ce14 100644
--- a/drivers/gpu/drm/i915/intel_sprite.c
+++ b/drivers/gpu/drm/i915/intel_sprite.c
@@ -1364,7 +1364,7 @@ int intel_plane_restore(struct drm_plane *plane)
 	if (!plane->crtc || !plane->fb)
 		return 0;
 
-	return plane->funcs->update_plane(plane, plane->crtc, plane->fb,
+	return plane->funcs->update_plane(plane, plane->crtc, plane->fb, NULL,
 				  plane->state->crtc_x, plane->state->crtc_y,
 				  plane->state->crtc_w, plane->state->crtc_h,
 				  plane->state->src_x, plane->state->src_y,
diff --git a/drivers/gpu/drm/imx/ipuv3-plane.c b/drivers/gpu/drm/imx/ipuv3-plane.c
index 6987e16fe99b..a26d969fab0e 100644
--- a/drivers/gpu/drm/imx/ipuv3-plane.c
+++ b/drivers/gpu/drm/imx/ipuv3-plane.c
@@ -295,7 +295,8 @@ void ipu_plane_disable(struct ipu_plane *ipu_plane)
  */
 
 static int ipu_update_plane(struct drm_plane *plane, struct drm_crtc *crtc,
-			    struct drm_framebuffer *fb, int crtc_x, int crtc_y,
+			    struct drm_framebuffer *fb,
+			    struct drm_live_source *src, int crtc_x, int crtc_y,
 			    unsigned int crtc_w, unsigned int crtc_h,
 			    uint32_t src_x, uint32_t src_y,
 			    uint32_t src_w, uint32_t src_h)
diff --git a/drivers/gpu/drm/nouveau/dispnv04/overlay.c b/drivers/gpu/drm/nouveau/dispnv04/overlay.c
index 9f2498571d09..485d1de72460 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/overlay.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/overlay.c
@@ -91,7 +91,8 @@ cos_mul(int degrees, int factor)
 
 static int
 nv10_update_plane(struct drm_plane *plane, struct drm_crtc *crtc,
-		  struct drm_framebuffer *fb, int crtc_x, int crtc_y,
+		  struct drm_framebuffer *fb, struct drm_live_source *src,
+		  int crtc_x, int crtc_y,
 		  unsigned int crtc_w, unsigned int crtc_h,
 		  uint32_t src_x, uint32_t src_y,
 		  uint32_t src_w, uint32_t src_h)
@@ -341,7 +342,8 @@ err:
 
 static int
 nv04_update_plane(struct drm_plane *plane, struct drm_crtc *crtc,
-		  struct drm_framebuffer *fb, int crtc_x, int crtc_y,
+		  struct drm_framebuffer *fb, struct drm_live_source *src,
+		  int crtc_x, int crtc_y,
 		  unsigned int crtc_w, unsigned int crtc_h,
 		  uint32_t src_x, uint32_t src_y,
 		  uint32_t src_w, uint32_t src_h)
diff --git a/drivers/gpu/drm/omapdrm/omap_plane.c b/drivers/gpu/drm/omapdrm/omap_plane.c
index ee8e2b3a117e..953707c8a795 100644
--- a/drivers/gpu/drm/omapdrm/omap_plane.c
+++ b/drivers/gpu/drm/omapdrm/omap_plane.c
@@ -238,6 +238,7 @@ int omap_plane_mode_set(struct drm_plane *plane,
 
 static int omap_plane_update(struct drm_plane *plane,
 		struct drm_crtc *crtc, struct drm_framebuffer *fb,
+		struct drm_live_source *src,
 		int crtc_x, int crtc_y,
 		unsigned int crtc_w, unsigned int crtc_h,
 		uint32_t src_x, uint32_t src_y,
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
index 9a5c571b95fc..6c0767a03ad8 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
@@ -504,8 +504,9 @@ static int vop_win_queue_fb(struct vop_win *vop_win,
 
 static int vop_update_plane_event(struct drm_plane *plane,
 				  struct drm_crtc *crtc,
-				  struct drm_framebuffer *fb, int crtc_x,
-				  int crtc_y, unsigned int crtc_w,
+				  struct drm_framebuffer *fb,
+				  struct drm_live_source *src,
+				  int crtc_x, int crtc_y, unsigned int crtc_w,
 				  unsigned int crtc_h, uint32_t src_x,
 				  uint32_t src_y, uint32_t src_w,
 				  uint32_t src_h,
diff --git a/drivers/gpu/drm/shmobile/shmob_drm_plane.c b/drivers/gpu/drm/shmobile/shmob_drm_plane.c
index 1805bb23b113..236613bcd571 100644
--- a/drivers/gpu/drm/shmobile/shmob_drm_plane.c
+++ b/drivers/gpu/drm/shmobile/shmob_drm_plane.c
@@ -174,7 +174,8 @@ void shmob_drm_plane_setup(struct drm_plane *plane)
 
 static int
 shmob_drm_plane_update(struct drm_plane *plane, struct drm_crtc *crtc,
-		       struct drm_framebuffer *fb, int crtc_x, int crtc_y,
+		       struct drm_framebuffer *fb, struct drm_live_source *src,
+		       int crtc_x, int crtc_y,
 		       unsigned int crtc_w, unsigned int crtc_h,
 		       uint32_t src_x, uint32_t src_y,
 		       uint32_t src_w, uint32_t src_h)
diff --git a/drivers/gpu/drm/sti/sti_drm_plane.c b/drivers/gpu/drm/sti/sti_drm_plane.c
index bb6a29339e10..f43a4341e59c 100644
--- a/drivers/gpu/drm/sti/sti_drm_plane.c
+++ b/drivers/gpu/drm/sti/sti_drm_plane.c
@@ -24,7 +24,8 @@ enum sti_layer_desc sti_layer_default_zorder[] = {
 
 static int
 sti_drm_update_plane(struct drm_plane *plane, struct drm_crtc *crtc,
-		     struct drm_framebuffer *fb, int crtc_x, int crtc_y,
+		     struct drm_framebuffer *fb, struct drm_live_source *src,
+		     int crtc_x, int crtc_y,
 		     unsigned int crtc_w, unsigned int crtc_h,
 		     uint32_t src_x, uint32_t src_y,
 		     uint32_t src_w, uint32_t src_h)
diff --git a/drivers/gpu/drm/sti/sti_hqvdp.c b/drivers/gpu/drm/sti/sti_hqvdp.c
index b0eb62de1b2e..4ec096e62d9e 100644
--- a/drivers/gpu/drm/sti/sti_hqvdp.c
+++ b/drivers/gpu/drm/sti/sti_hqvdp.c
@@ -533,7 +533,7 @@ static int sti_hqvdp_prepare_layer(struct sti_layer *layer, bool first_prepare)
 
 	/* prepare and commit VID plane */
 	hqvdp->vid_plane->funcs->update_plane(hqvdp->vid_plane,
-					layer->crtc, layer->fb,
+					layer->crtc, layer->fb, NULL,
 					layer->dst_x, layer->dst_y,
 					layer->dst_w, layer->dst_h,
 					layer->src_x, layer->src_y,
diff --git a/include/drm/drmP.h b/include/drm/drmP.h
index 63c0b0131f61..6adcdb99482a 100644
--- a/include/drm/drmP.h
+++ b/include/drm/drmP.h
@@ -322,6 +322,9 @@ struct drm_file {
 	struct list_head fbs;
 	struct mutex fbs_lock;
 
+	struct list_head sources;
+	struct mutex sources_lock;
+
 	wait_queue_head_t event_wait;
 	struct list_head event_list;
 	int event_space;
diff --git a/include/drm/drm_atomic_helper.h b/include/drm/drm_atomic_helper.h
index 829280b56874..68b9bb68d015 100644
--- a/include/drm/drm_atomic_helper.h
+++ b/include/drm/drm_atomic_helper.h
@@ -62,6 +62,7 @@ void drm_atomic_helper_swap_state(struct drm_device *dev,
 int drm_atomic_helper_update_plane(struct drm_plane *plane,
 				   struct drm_crtc *crtc,
 				   struct drm_framebuffer *fb,
+				   struct drm_live_source *src,
 				   int crtc_x, int crtc_y,
 				   unsigned int crtc_w, unsigned int crtc_h,
 				   uint32_t src_x, uint32_t src_y,
diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h
index 775a9a84c5bf..c66e23a60f75 100644
--- a/include/drm/drm_crtc.h
+++ b/include/drm/drm_crtc.h
@@ -759,6 +759,7 @@ struct drm_plane_state {
 
 	struct drm_crtc *crtc;   /* do not write directly, use drm_atomic_set_crtc_for_plane() */
 	struct drm_framebuffer *fb;  /* do not write directly, use drm_atomic_set_fb_for_plane() */
+	struct drm_live_source *src;
 	struct fence *fence;
 
 	/* Signed dest location allows it to be partially off screen */
@@ -793,6 +794,7 @@ struct drm_plane_state {
 struct drm_plane_funcs {
 	int (*update_plane)(struct drm_plane *plane,
 			    struct drm_crtc *crtc, struct drm_framebuffer *fb,
+			    struct drm_live_source *src,
 			    int crtc_x, int crtc_y,
 			    unsigned int crtc_w, unsigned int crtc_h,
 			    uint32_t src_x, uint32_t src_y,
@@ -857,6 +859,7 @@ struct drm_plane {
 
 	struct drm_crtc *crtc;
 	struct drm_framebuffer *fb;
+	struct drm_live_source *src;
 
 	struct drm_framebuffer *old_fb;
 
@@ -878,6 +881,7 @@ struct drm_live_source_funcs {
 struct drm_live_source {
 	struct drm_device *dev;
 	struct list_head head;
+	struct list_head filp_head;
 
 	struct drm_mode_object base;
 
@@ -1149,6 +1153,7 @@ struct drm_mode_config {
 	struct drm_property *prop_crtc_w;
 	struct drm_property *prop_crtc_h;
 	struct drm_property *prop_fb_id;
+	struct drm_property *prop_src_id;
 	struct drm_property *prop_crtc_id;
 	struct drm_property *prop_active;
 
@@ -1311,6 +1316,7 @@ extern int drm_live_source_init(struct drm_device *dev,
 				const uint32_t *formats, uint32_t format_count,
 				const struct drm_live_source_funcs *funcs);
 extern void drm_live_source_cleanup(struct drm_live_source *src);
+extern void drm_live_sources_release(struct drm_file *priv);
 
 extern const char *drm_get_connector_status_name(enum drm_connector_status status);
 extern const char *drm_get_subpixel_order_name(enum subpixel_order order);
diff --git a/include/drm/drm_plane_helper.h b/include/drm/drm_plane_helper.h
index e48157a5a59c..441cd55e094c 100644
--- a/include/drm/drm_plane_helper.h
+++ b/include/drm/drm_plane_helper.h
@@ -93,6 +93,7 @@ extern int drm_plane_helper_check_update(struct drm_plane *plane,
 extern int drm_primary_helper_update(struct drm_plane *plane,
 				     struct drm_crtc *crtc,
 				     struct drm_framebuffer *fb,
+				     struct drm_live_source *src,
 				     int crtc_x, int crtc_y,
 				     unsigned int crtc_w, unsigned int crtc_h,
 				     uint32_t src_x, uint32_t src_y,
-- 
2.0.5

[RFC/PATCH 3/5] drm/rcar-du: Add VSP1 support to the planes allocator

[Laurent Pinchart]

The R8A7790 DU can source frames directly from the VSP1 devices VSPD0
and VSPD1. VSPD0 feeds DU0/1 plane 0, and VSPD1 feeds either DU2 plane 0
or DU0/1 plane 1.

Allocate the correct fixed plane when sourcing frames from VSPD0 or
VSPD1, and allocate planes in reverse index order otherwise to ensure
maximum availability of planes 0 and 1.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/gpu/drm/rcar-du/rcar_du_kms.c   | 40 ++++++++++++++++++++++++++++-----
 drivers/gpu/drm/rcar-du/rcar_du_plane.c |  1 +
 drivers/gpu/drm/rcar-du/rcar_du_plane.h |  7 ++++++
 3 files changed, 43 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/rcar-du/rcar_du_kms.c b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
index fb052bca574f..17f89bfca8f8 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_kms.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
@@ -244,11 +244,41 @@ static unsigned int rcar_du_plane_hwmask(struct rcar_du_plane_state *state)
 	return mask;
 }
 
-static int rcar_du_plane_hwalloc(unsigned int num_planes, unsigned int free)
+/*
+ * The R8A7790 DU can source frames directly from the VSP1 devices VSPD0 and
+ * VSPD1. VSPD0 feeds DU0/1 plane 0, and VSPD1 feeds either DU2 plane 0 or
+ * DU0/1 plane 1.
+ *
+ * Allocate the correct fixed plane when sourcing frames from VSPD0 or VSPD1,
+ * and allocate planes in reverse index order otherwise to ensure maximum
+ * availability of planes 0 and 1.
+ *
+ * The caller is responsible for ensuring that the requested source is
+ * compatible with the DU revision.
+ */
+static int rcar_du_plane_hwalloc(struct rcar_du_plane *plane,
+				 struct rcar_du_plane_state *state,
+				 unsigned int free)
 {
-	unsigned int i;
+	unsigned int num_planes = state->format->planes;
+	int fixed = -1;
+	int i;
+
+	if (state->source == RCAR_DU_PLANE_VSPD0) {
+		/* VSPD0 feeds plane 0 on DU0/1. */
+		if (plane->group->index != 0)
+			return -EINVAL;
+
+		fixed = 0;
+	} else if (state->source == RCAR_DU_PLANE_VSPD1) {
+		/* VSPD1 feeds plane 1 on DU0/1 or plane 0 on DU2. */
+		fixed = plane->group->index == 0 ? 1 : 0;
+	}
+
+	if (fixed >= 0)
+		return free & (1 << fixed) ? fixed : -EBUSY;
 
-	for (i = 0; i < RCAR_DU_NUM_HW_PLANES; ++i) {
+	for (i = RCAR_DU_NUM_HW_PLANES - 1; i >= 0; --i) {
 		if (!(free & (1 << i)))
 			continue;
 
@@ -256,7 +286,7 @@ static int rcar_du_plane_hwalloc(unsigned int num_planes, unsigned int free)
 			break;
 	}
 
-	return i == RCAR_DU_NUM_HW_PLANES ? -EBUSY : i;
+	return i < 0 ? -EBUSY : i;
 }
 
 static int rcar_du_atomic_check(struct drm_device *dev,
@@ -372,7 +402,7 @@ static int rcar_du_atomic_check(struct drm_device *dev,
 		    !rcar_du_plane_needs_realloc(plane, plane_state))
 			continue;
 
-		idx = rcar_du_plane_hwalloc(plane_state->format->planes,
+		idx = rcar_du_plane_hwalloc(plane, plane_state,
 					group_free_planes[plane->group->index]);
 		if (idx < 0) {
 			dev_dbg(rcdu->dev, "%s: no available hardware plane\n",
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.c b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
index 210e5c3fd982..80e4dba78aef 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
@@ -288,6 +288,7 @@ static void rcar_du_plane_reset(struct drm_plane *plane)
 		return;
 
 	state->hwindex = -1;
+	state->source = RCAR_DU_PLANE_MEMORY;
 	state->alpha = 255;
 	state->colorkey = RCAR_DU_COLORKEY_NONE;
 	state->zpos = plane->type == DRM_PLANE_TYPE_PRIMARY ? 0 : 1;
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.h b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
index abff0ebeb195..81c2d361a94f 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
@@ -28,6 +28,12 @@ struct rcar_du_group;
 #define RCAR_DU_NUM_KMS_PLANES		9
 #define RCAR_DU_NUM_HW_PLANES		8
 
+enum rcar_du_plane_source {
+	RCAR_DU_PLANE_MEMORY,
+	RCAR_DU_PLANE_VSPD0,
+	RCAR_DU_PLANE_VSPD1,
+};
+
 struct rcar_du_plane {
 	struct drm_plane plane;
 	struct rcar_du_group *group;
@@ -51,6 +57,7 @@ struct rcar_du_plane_state {
 
 	const struct rcar_du_format_info *format;
 	int hwindex;		/* 0-based, -1 means unused */
+	enum rcar_du_plane_source source;
 
 	unsigned int alpha;
 	unsigned int colorkey;
-- 
2.0.5

[RFC/PATCH 4/5] drm/rcar-du: Add VSP1 live source support

[Laurent Pinchart]

Register live sources for VSPD0 and VSPD1 and configure the plane source
at plane setup time to source frames from memory or from the VSP1.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/gpu/drm/rcar-du/rcar_du_drv.c   |   6 +-
 drivers/gpu/drm/rcar-du/rcar_du_drv.h   |   3 +
 drivers/gpu/drm/rcar-du/rcar_du_group.c |  19 ++--
 drivers/gpu/drm/rcar-du/rcar_du_group.h |   2 +
 drivers/gpu/drm/rcar-du/rcar_du_kms.c   |  22 +++-
 drivers/gpu/drm/rcar-du/rcar_du_plane.c | 173 +++++++++++++++++++++++++-------
 drivers/gpu/drm/rcar-du/rcar_du_plane.h |   3 +
 drivers/gpu/drm/rcar-du/rcar_du_regs.h  |   1 +
 8 files changed, 177 insertions(+), 52 deletions(-)

diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.c b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
index da1216a73969..e16a912327b7 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_drv.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
@@ -58,7 +58,8 @@ static const struct rcar_du_device_info rcar_du_r8a7779_info = {
 
 static const struct rcar_du_device_info rcar_du_r8a7790_info = {
 	.features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK
-		  | RCAR_DU_FEATURE_EXT_CTRL_REGS,
+		  | RCAR_DU_FEATURE_EXT_CTRL_REGS
+		  | RCAR_DU_FEATURE_VSP1_SOURCE,
 	.quirks = RCAR_DU_QUIRK_ALIGN_128B | RCAR_DU_QUIRK_LVDS_LANES,
 	.num_crtcs = 3,
 	.routes = {
@@ -86,7 +87,8 @@ static const struct rcar_du_device_info rcar_du_r8a7790_info = {
 
 static const struct rcar_du_device_info rcar_du_r8a7791_info = {
 	.features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK
-		  | RCAR_DU_FEATURE_EXT_CTRL_REGS,
+		  | RCAR_DU_FEATURE_EXT_CTRL_REGS
+		  | RCAR_DU_FEATURE_VSP1_SOURCE,
 	.num_crtcs = 2,
 	.routes = {
 		/* R8A7791 has one RGB output, one LVDS output and one
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.h b/drivers/gpu/drm/rcar-du/rcar_du_drv.h
index c7c538dd2e68..b5be16053e71 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_drv.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.h
@@ -29,6 +29,7 @@ struct rcar_du_lvdsenc;
 
 #define RCAR_DU_FEATURE_CRTC_IRQ_CLOCK	(1 << 0)	/* Per-CRTC IRQ and clock */
 #define RCAR_DU_FEATURE_EXT_CTRL_REGS	(1 << 1)	/* Has extended control registers */
+#define RCAR_DU_FEATURE_VSP1_SOURCE	(1 << 2)	/* Has inputs from VSP1 */
 
 #define RCAR_DU_QUIRK_ALIGN_128B	(1 << 0)	/* Align pitches to 128 bytes */
 #define RCAR_DU_QUIRK_LVDS_LANES	(1 << 1)	/* LVDS lanes 1 and 3 inverted */
@@ -84,6 +85,8 @@ struct rcar_du_device {
 	struct rcar_du_group groups[RCAR_DU_MAX_GROUPS];
 
 	unsigned int dpad0_source;
+	unsigned int vspd1_sink;
+
 	struct rcar_du_lvdsenc *lvds[RCAR_DU_MAX_LVDS];
 
 	struct {
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_group.c b/drivers/gpu/drm/rcar-du/rcar_du_group.c
index 1bdc0ee0c248..71f50bf45581 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_group.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_group.c
@@ -49,10 +49,13 @@ static void rcar_du_group_setup_defr8(struct rcar_du_group *rgrp)
 	u32 defr8 = DEFR8_CODE | DEFR8_DEFE8;
 
 	/* The DEFR8 register for the first group also controls RGB output
-	 * routing to DPAD0
+	 * routing to DPAD0 and VSPD1 routing to DU0/1/2.
 	 */
-	if (rgrp->index == 0)
+	if (rgrp->index == 0) {
 		defr8 |= DEFR8_DRGBS_DU(rgrp->dev->dpad0_source);
+		if (rgrp->dev->vspd1_sink == 2)
+			defr8 |= DEFR8_VSCS;
+	}
 
 	rcar_du_group_write(rgrp, DEFR8, defr8);
 }
@@ -155,17 +158,17 @@ void rcar_du_group_restart(struct rcar_du_group *rgrp)
 	__rcar_du_group_start_stop(rgrp, true);
 }
 
-static int rcar_du_set_dpad0_routing(struct rcar_du_device *rcdu)
+int rcar_du_set_dpad0_vsp1_routing(struct rcar_du_device *rcdu)
 {
 	int ret;
 
 	if (!rcar_du_has(rcdu, RCAR_DU_FEATURE_EXT_CTRL_REGS))
 		return 0;
 
-	/* RGB output routing to DPAD0 is configured in the DEFR8 register of
-	 * the first group. As this function can be called with the DU0 and DU1
-	 * CRTCs disabled, we need to enable the first group clock before
-	 * accessing the register.
+	/* RGB output routing to DPAD0 and VSP1D routing to DU0/1/2 are
+	 * configured in the DEFR8 register of the first group. As this function
+	 * can be called with the DU0 and DU1 CRTCs disabled, we need to enable
+	 * the first group clock before accessing the register.
 	 */
 	ret = clk_prepare_enable(rcdu->crtcs[0].clock);
 	if (ret < 0)
@@ -196,5 +199,5 @@ int rcar_du_group_set_routing(struct rcar_du_group *rgrp)
 
 	rcar_du_group_write(rgrp, DORCR, dorcr);
 
-	return rcar_du_set_dpad0_routing(rgrp->dev);
+	return rcar_du_set_dpad0_vsp1_routing(rgrp->dev);
 }
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_group.h b/drivers/gpu/drm/rcar-du/rcar_du_group.h
index ed36433fbe84..3fdf77171034 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_group.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_group.h
@@ -52,4 +52,6 @@ void rcar_du_group_start_stop(struct rcar_du_group *rgrp, bool start);
 void rcar_du_group_restart(struct rcar_du_group *rgrp);
 int rcar_du_group_set_routing(struct rcar_du_group *rgrp);
 
+int rcar_du_set_dpad0_vsp1_routing(struct rcar_du_device *rcdu);
+
 #endif /* __RCAR_DU_GROUP_H__ */
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_kms.c b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
index 17f89bfca8f8..b78ced38b696 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_kms.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
@@ -217,17 +217,25 @@ static void rcar_du_output_poll_changed(struct drm_device *dev)
  */
 
 static bool rcar_du_plane_needs_realloc(struct rcar_du_plane *plane,
-					struct rcar_du_plane_state *state)
+					struct rcar_du_plane_state *new_state)
 {
-	const struct rcar_du_format_info *cur_format;
+	struct rcar_du_plane_state *cur_state;
 
-	cur_format = to_rcar_du_plane_state(plane->plane.state)->format;
+	cur_state = to_rcar_du_plane_state(plane->plane.state);
 
 	/* Lowering the number of planes doesn't strictly require reallocation
 	 * as the extra hardware plane will be freed when committing, but doing
 	 * so could lead to more fragmentation.
 	 */
-	return !cur_format || cur_format->planes != state->format->planes;
+	if (!cur_state->format ||
+	    cur_state->format->planes != new_state->format->planes)
+		return true;
+
+	/* Reallocate hardware planes if the source has changed. */
+	if (cur_state->source != new_state->source)
+		return true;
+
+	return false;
 }
 
 static unsigned int rcar_du_plane_hwmask(struct rcar_du_plane_state *state)
@@ -776,6 +784,12 @@ int rcar_du_modeset_init(struct rcar_du_device *rcdu)
 
 	drm_mode_config_reset(dev);
 
+	if (rcar_du_has(rcdu, RCAR_DU_FEATURE_VSP1_SOURCE)) {
+		ret = rcar_du_vsp1_sources_init(rcdu);
+		if (ret < 0)
+			return ret;
+	}
+
 	drm_kms_helper_poll_init(dev);
 
 	if (dev->mode_config.num_connector) {
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.c b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
index 80e4dba78aef..e77f9d93c1c5 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
@@ -20,21 +20,84 @@
 #include <drm/drm_plane_helper.h>
 
 #include "rcar_du_drv.h"
+#include "rcar_du_group.h"
 #include "rcar_du_kms.h"
 #include "rcar_du_plane.h"
 #include "rcar_du_regs.h"
 
-#define RCAR_DU_COLORKEY_NONE		(0 << 24)
-#define RCAR_DU_COLORKEY_SOURCE		(1 << 24)
-#define RCAR_DU_COLORKEY_MASK		(1 << 24)
+/* -----------------------------------------------------------------------------
+ * Live Sources
+ */
+
+struct rcar_du_vsp1_source {
+	struct drm_live_source base;
+
+	enum rcar_du_plane_source source;
+};
 
-static u32 rcar_du_plane_read(struct rcar_du_group *rgrp,
-			      unsigned int index, u32 reg)
+static inline struct rcar_du_vsp1_source *
+to_rcar_vsp1_source(struct drm_live_source *src)
 {
-	return rcar_du_read(rgrp->dev,
-			    rgrp->mmio_offset + index * PLANE_OFF + reg);
+	return container_of(src, struct rcar_du_vsp1_source, base);
+}
+
+static const struct drm_live_source_funcs rcar_du_live_source_funcs = {
+	.destroy = drm_live_source_cleanup,
+};
+
+static const uint32_t source_formats[] = {
+	DRM_FORMAT_XRGB8888,
+};
+
+int rcar_du_vsp1_sources_init(struct rcar_du_device *rcdu)
+{
+	static const struct {
+		enum rcar_du_plane_source source;
+		unsigned int planes;
+	} sources[] = {
+		{ RCAR_DU_PLANE_VSPD0, BIT(RCAR_DU_NUM_KMS_PLANES - 1) },
+		{ RCAR_DU_PLANE_VSPD1, BIT(RCAR_DU_NUM_KMS_PLANES - 2) |
+				       BIT(2 * RCAR_DU_NUM_KMS_PLANES - 1) },
+	};
+	unsigned int planes_mask;
+	unsigned int num_planes;
+	unsigned int i;
+
+	num_planes = RCAR_DU_NUM_KMS_PLANES * DIV_ROUND_UP(rcdu->num_crtcs, 2);
+	planes_mask = (1 << num_planes) - 1;
+
+	for (i = 0; i < ARRAY_SIZE(sources); ++i) {
+		struct rcar_du_vsp1_source *src;
+		char name[6];
+		int ret;
+
+		src = devm_kzalloc(rcdu->dev, sizeof(*src), GFP_KERNEL);
+		if (src == NULL)
+			return -ENOMEM;
+
+		src->source = sources[i].source;
+
+		sprintf(name, "vspd%u", i);
+		ret = drm_live_source_init(rcdu->ddev, &src->base, name,
+					   sources[i].planes & planes_mask,
+					   source_formats,
+					   ARRAY_SIZE(source_formats),
+					   &rcar_du_live_source_funcs);
+		if (ret < 0)
+			return ret;
+	}
+
+	return 0;
 }
 
+/* -----------------------------------------------------------------------------
+ * Planes
+ */
+
+#define RCAR_DU_COLORKEY_NONE		(0 << 24)
+#define RCAR_DU_COLORKEY_SOURCE		(1 << 24)
+#define RCAR_DU_COLORKEY_MASK		(1 << 24)
+
 static void rcar_du_plane_write(struct rcar_du_group *rgrp,
 				unsigned int index, u32 reg, u32 data)
 {
@@ -42,34 +105,47 @@ static void rcar_du_plane_write(struct rcar_du_group *rgrp,
 		      data);
 }
 
-static void rcar_du_plane_setup_fb(struct rcar_du_plane *plane)
+static void rcar_du_plane_setup_scanout(struct rcar_du_plane *plane)
 {
 	struct rcar_du_plane_state *state =
 		to_rcar_du_plane_state(plane->plane.state);
-	struct drm_framebuffer *fb = plane->plane.state->fb;
 	struct rcar_du_group *rgrp = plane->group;
 	unsigned int src_x = state->state.src_x >> 16;
 	unsigned int src_y = state->state.src_y >> 16;
 	unsigned int index = state->hwindex;
-	struct drm_gem_cma_object *gem;
+	unsigned int pitch;
 	bool interlaced;
-	u32 mwr;
+	u32 dma[2];
 
 	interlaced = state->state.crtc->state->adjusted_mode.flags
 		   & DRM_MODE_FLAG_INTERLACE;
 
+	if (plane->plane.state->fb) {
+		struct drm_framebuffer *fb = state->state.fb;
+		struct drm_gem_cma_object *gem;
+		unsigned int i;
+
+		if (state->format->planes == 2)
+			pitch = fb->pitches[0];
+		else
+			pitch = fb->pitches[0] * 8 / state->format->bpp;
+
+		for (i = 0; i < state->format->planes; ++i) {
+			gem = drm_fb_cma_get_gem_obj(fb, i);
+			dma[i] = gem->paddr + fb->offsets[i];
+		}
+	} else {
+		pitch = state->state.src_w >> 16;
+		dma[0] = 0;
+		dma[1] = 0;
+	}
+
 	/* Memory pitch (expressed in pixels). Must be doubled for interlaced
 	 * operation with 32bpp formats.
 	 */
-	if (state->format->planes == 2)
-		mwr = fb->pitches[0];
-	else
-		mwr = fb->pitches[0] * 8 / state->format->bpp;
-
-	if (interlaced && state->format->bpp == 32)
-		mwr *= 2;
-
-	rcar_du_plane_write(rgrp, index, PnMWR, mwr);
+	rcar_du_plane_write(rgrp, index, PnMWR,
+			    (interlaced && state->format->bpp == 32) ?
+			    pitch * 2 : pitch);
 
 	/* The Y position is expressed in raster line units and must be doubled
 	 * for 32bpp formats, according to the R8A7790 datasheet. No mention of
@@ -87,21 +163,18 @@ static void rcar_du_plane_setup_fb(struct rcar_du_plane *plane)
 	rcar_du_plane_write(rgrp, index, PnSPYR, src_y *
 			    (!interlaced && state->format->bpp == 32 ? 2 : 1));
 
-	gem = drm_fb_cma_get_gem_obj(fb, 0);
-	rcar_du_plane_write(rgrp, index, PnDSA0R, gem->paddr + fb->offsets[0]);
+	rcar_du_plane_write(rgrp, index, PnDSA0R, dma[0]);
 
 	if (state->format->planes == 2) {
 		index = (index + 1) % 8;
 
-		rcar_du_plane_write(rgrp, index, PnMWR, fb->pitches[0]);
+		rcar_du_plane_write(rgrp, index, PnMWR, pitch);
 
 		rcar_du_plane_write(rgrp, index, PnSPXR, src_x);
 		rcar_du_plane_write(rgrp, index, PnSPYR, src_y *
 				    (state->format->bpp == 16 ? 2 : 1) / 2);
 
-		gem = drm_fb_cma_get_gem_obj(fb, 1);
-		rcar_du_plane_write(rgrp, index, PnDSA0R,
-				    gem->paddr + fb->offsets[1]);
+		rcar_du_plane_write(rgrp, index, PnDSA0R, dma[1]);
 	}
 }
 
@@ -168,8 +241,8 @@ static void rcar_du_plane_setup_mode(struct rcar_du_plane *plane,
 	}
 }
 
-static void __rcar_du_plane_setup(struct rcar_du_plane *plane,
-				  unsigned int index)
+static void rcar_du_plane_setup_format(struct rcar_du_plane *plane,
+				       unsigned int index)
 {
 	struct rcar_du_plane_state *state =
 		to_rcar_du_plane_state(plane->plane.state);
@@ -182,10 +255,6 @@ static void __rcar_du_plane_setup(struct rcar_du_plane *plane,
 	 * The data format is selected by the DDDF field in PnMR and the EDF
 	 * field in DDCR4.
 	 */
-	ddcr4 = rcar_du_plane_read(rgrp, index, PnDDCR4);
-	ddcr4 &= ~PnDDCR4_EDF_MASK;
-	ddcr4 |= state->format->edf | PnDDCR4_CODE;
-
 	rcar_du_plane_setup_mode(plane, index);
 
 	if (state->format->planes == 2) {
@@ -204,6 +273,11 @@ static void __rcar_du_plane_setup(struct rcar_du_plane *plane,
 	}
 
 	rcar_du_plane_write(rgrp, index, PnDDCR2, ddcr2);
+
+	ddcr4 = state->format->edf | PnDDCR4_CODE;
+	if (state->source != RCAR_DU_PLANE_MEMORY)
+		ddcr4 |= PnDDCR4_VSPS;
+
 	rcar_du_plane_write(rgrp, index, PnDDCR4, ddcr4);
 
 	/* Destination position and size */
@@ -224,11 +298,21 @@ void rcar_du_plane_setup(struct rcar_du_plane *plane)
 	struct rcar_du_plane_state *state =
 		to_rcar_du_plane_state(plane->plane.state);
 
-	__rcar_du_plane_setup(plane, state->hwindex);
+	rcar_du_plane_setup_format(plane, state->hwindex);
 	if (state->format->planes == 2)
-		__rcar_du_plane_setup(plane, (state->hwindex + 1) % 8);
+		rcar_du_plane_setup_format(plane, (state->hwindex + 1) % 8);
+
+	rcar_du_plane_setup_scanout(plane);
+
+	if (state->source == RCAR_DU_PLANE_VSPD1) {
+		unsigned int vspd1_sink = plane->group->index ? 2 : 0;
+		struct rcar_du_device *rcdu = plane->group->dev;
 
-	rcar_du_plane_setup_fb(plane);
+		if (rcdu->vspd1_sink != vspd1_sink) {
+			rcdu->vspd1_sink = vspd1_sink;
+			rcar_du_set_dpad0_vsp1_routing(rcdu);
+		}
+	}
 }
 
 static int rcar_du_plane_atomic_check(struct drm_plane *plane,
@@ -237,8 +321,9 @@ static int rcar_du_plane_atomic_check(struct drm_plane *plane,
 	struct rcar_du_plane_state *rstate = to_rcar_du_plane_state(state);
 	struct rcar_du_plane *rplane = to_rcar_plane(plane);
 	struct rcar_du_device *rcdu = rplane->group->dev;
+	uint32_t pixel_format;
 
-	if (!state->fb || !state->crtc) {
+	if ((!state->fb && !state->src) || !state->crtc) {
 		rstate->format = NULL;
 		return 0;
 	}
@@ -249,13 +334,25 @@ static int rcar_du_plane_atomic_check(struct drm_plane *plane,
 		return -EINVAL;
 	}
 
-	rstate->format = rcar_du_format_info(state->fb->pixel_format);
+	pixel_format = state->fb ? state->fb->pixel_format
+				 : state->src->pixel_format;
+	rstate->format = rcar_du_format_info(pixel_format);
 	if (rstate->format == NULL) {
 		dev_dbg(rcdu->dev, "%s: unsupported format %08x\n", __func__,
-			state->fb->pixel_format);
+			pixel_format);
+		return -EINVAL;
+	}
+
+	if (state->src && rstate->format->planes > 1) {
+		dev_dbg(rcdu->dev,
+			"%s: unsupported format %08x for live source\n",
+			__func__, pixel_format);
 		return -EINVAL;
 	}
 
+	rstate->source = state->fb ? RCAR_DU_PLANE_MEMORY
+		       : to_rcar_vsp1_source(state->src)->source;
+
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.h b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
index 81c2d361a94f..9a6132899d59 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
@@ -17,6 +17,7 @@
 #include <drm/drmP.h>
 #include <drm/drm_crtc.h>
 
+struct rcar_du_device;
 struct rcar_du_format_info;
 struct rcar_du_group;
 
@@ -70,6 +71,8 @@ to_rcar_du_plane_state(struct drm_plane_state *state)
 	return container_of(state, struct rcar_du_plane_state, state);
 }
 
+int rcar_du_vsp1_sources_init(struct rcar_du_device *rcdu);
+
 int rcar_du_planes_init(struct rcar_du_group *rgrp);
 
 void rcar_du_plane_setup(struct rcar_du_plane *plane);
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_regs.h b/drivers/gpu/drm/rcar-du/rcar_du_regs.h
index 70fcbc471ebd..ac9c3e511e79 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_regs.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_regs.h
@@ -389,6 +389,7 @@
 
 #define PnDDCR4			0x00190
 #define PnDDCR4_CODE		(0x7766 << 16)
+#define PnDDCR4_VSPS		(1 << 13)
 #define PnDDCR4_SDFS_RGB	(0 << 4)
 #define PnDDCR4_SDFS_YC		(5 << 4)
 #define PnDDCR4_SDFS_MASK	(7 << 4)
-- 
2.0.5

[RFC/PATCH 5/5] drm/rcar-du: Restart the DU group when a plane source changes

[Laurent Pinchart]

Plane sources are configured by the VSPS bit in the PnDDCR4 register.
Although the datasheet states that the bit is updated during vertical
blanking, it seems that updates only occur when the DU group is held in
reset through the DSYSR.DRES bit. Restart the group if the source
changes.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/gpu/drm/rcar-du/rcar_du_crtc.c  | 10 ++++++++--
 drivers/gpu/drm/rcar-du/rcar_du_group.c |  2 ++
 drivers/gpu/drm/rcar-du/rcar_du_plane.c | 22 ++++++++++++++++++++--
 drivers/gpu/drm/rcar-du/rcar_du_plane.h |  1 +
 4 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
index 7d0b8ef9bea2..969d49ab0d09 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
@@ -249,6 +249,8 @@ static void rcar_du_crtc_update_planes(struct rcar_du_crtc *rcrtc)
 		}
 	}
 
+	mutex_lock(&rcrtc->group->lock);
+
 	/* Select display timing and dot clock generator 2 for planes associated
 	 * with superposition controller 2.
 	 */
@@ -260,15 +262,19 @@ static void rcar_du_crtc_update_planes(struct rcar_du_crtc *rcrtc)
 		 * split, or through a module parameter). Flicker would then
 		 * occur only if we need to break the pre-association.
 		 */
-		mutex_lock(&rcrtc->group->lock);
 		if (rcar_du_group_read(rcrtc->group, DPTSR) != dptsr) {
 			rcar_du_group_write(rcrtc->group, DPTSR, dptsr);
 			if (rcrtc->group->used_crtcs)
 				rcar_du_group_restart(rcrtc->group);
 		}
-		mutex_unlock(&rcrtc->group->lock);
 	}
 
+	/* Restart the group if plane sources have changed. */
+	if (rcrtc->group->planes.need_restart)
+		rcar_du_group_restart(rcrtc->group);
+
+	mutex_unlock(&rcrtc->group->lock);
+
 	rcar_du_group_write(rcrtc->group, rcrtc->index % 2 ? DS2PR : DS1PR,
 			    dspr);
 }
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_group.c b/drivers/gpu/drm/rcar-du/rcar_du_group.c
index 71f50bf45581..101997e6e531 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_group.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_group.c
@@ -154,6 +154,8 @@ void rcar_du_group_start_stop(struct rcar_du_group *rgrp, bool start)
 
 void rcar_du_group_restart(struct rcar_du_group *rgrp)
 {
+	rgrp->planes.need_restart = false;
+
 	__rcar_du_group_start_stop(rgrp, false);
 	__rcar_du_group_start_stop(rgrp, true);
 }
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.c b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
index e77f9d93c1c5..51a9f9d5fa46 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
@@ -360,9 +360,27 @@ static void rcar_du_plane_atomic_update(struct drm_plane *plane,
 					struct drm_plane_state *old_state)
 {
 	struct rcar_du_plane *rplane = to_rcar_plane(plane);
+	struct rcar_du_plane_state *old_rstate;
+	struct rcar_du_plane_state *new_rstate;
 
-	if (plane->state->crtc)
-		rcar_du_plane_setup(rplane);
+	if (!plane->state->crtc)
+		return;
+
+	rcar_du_plane_setup(rplane);
+
+	/* Check whether the source has changed from memory to live source or
+	 * from live source to memory. The source has been configured by the
+	 * VSPS bit in the PnDDCR4 register. Although the datasheet states that
+	 * the bit is updated during vertical blanking, it seems that updates
+	 * only occur when the DU group is held in reset through the DSYSR.DRES
+	 * bit. We thus need to restart the group if the source changes.
+	 */
+	old_rstate = to_rcar_du_plane_state(old_state);
+	new_rstate = to_rcar_du_plane_state(plane->state);
+
+	if ((old_rstate->source == RCAR_DU_PLANE_MEMORY) !=
+	    (new_rstate->source == RCAR_DU_PLANE_MEMORY))
+		rplane->group->planes.need_restart = true;
 }
 
 static const struct drm_plane_helper_funcs rcar_du_plane_helper_funcs = {
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.h b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
index 9a6132899d59..694b44c151b6 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.h
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.h
@@ -47,6 +47,7 @@ static inline struct rcar_du_plane *to_rcar_plane(struct drm_plane *plane)
 
 struct rcar_du_planes {
 	struct rcar_du_plane planes[RCAR_DU_NUM_KMS_PLANES];
+	bool need_restart;
 
 	struct drm_property *alpha;
 	struct drm_property *colorkey;
-- 
2.0.5

Re: [RFC/PATCH 0/5] Add live source objects to DRM

[Daniel Vetter]

On Sun, Mar 15, 2015 at 11:55:35PM +0200, Laurent Pinchart wrote:
> Hello,
> 
> I have a feeling that RFC/PATCH will work better than just RFC, so here's a
> patch set that adds a new object named live source to DRM.
> 
> The need comes from the Renesas R-Car SoCs in which a video processing engine
> (named VSP1) that operates from memory to memory has one output directly
> connected to a plane of the display engine (DU) without going through memory.
> 
> The VSP1 is supported by a V4L2 driver. While it could be argued that it
> should instead be supported directly by the DRM rcar-du driver, this wouldn't
> be a good idea for at least two reasons. First, the R-Car SoCs contain several
> VSP1 instances, of which only a subset have a direct DU connection. The only
> other instances operate solely in memory to memory mode. Then, the VSP1 is a
> video processing engine and not a display engine. Its features are easily
> supported by the V4L2 API, but don't map to the DRM/KMS API. Significant
> changes to DRM/KMS would be required, beyond what is in my opinion an
> acceptable scope for a display API.
> 
> Now that the need to interface two separate devices supported by two different
> drivers in two separate subsystems has been established, we need an API to do
> so. It should be noted that while that API doesn't exist in the mainline
> kernel, the need isn't limited to Renesas SoCs.
> 
> This patch set proposes one possible solution for the problem in the form of a
> new DRM object named live source. Live sources are created by drivers to model
> hardware connections between a plane input and an external source, and are
> attached to planes through the KMS userspace API.
> 
> Patch 1/5 adds live source objects to DRM, with an in-kernel API for drivers
> to register the sources, and a userspace API to enumerate and configure them.
> Configuring a live source sets the width, height and pixel format of the
> video stream. This should ideally be queried directly from the driver that
> supports the live source device, but I've decided not to implement such
> communication between V4L2 and DRM/KMS at the moment to keep the proposal
> simple.
> 
> Patch 2/2 implements connection between live sources and planes. This takes
> different forms depending on whether drivers use the setplane or the atomic
> updates API:
> 
> - For setplane, the fb field can now contain a live source ID in addition to
>   a framebuffer ID. As DRM allocates object IDs from a single ID space, the
>   type can be inferred from the ID. This makes specifying both a framebuffer
>   and a live source impossible, which isn't an issue given that such a
>   configuration would be invalid.
> 
>   The live source is looked up by the DRM core and passed as a pointer to the
>   .update_plane() operation. Unlike framebuffers live sources are not
>   refcounted as they're created statically at driver initialization time.
> 
> - For atomic update, a new SRC_ID property has been added to planes. The live
>   source is looked up from the source ID and stored into the plane state.

What about directly treating live sources as (very) special framebuffers?
A bunch of reasons:
- All the abi fu above gets resolved naturally.
- You have lot of duplication between fb and live source wrt size,
  possible/selected pixel format and other stuff.
- The backing storage of framebuffers is fully opaque anyway ...

I think we still need separate live source objects though for things like
telling userspace what v4l thing it corresponds to, and for getting at the
pixel format. But connecting the live source with the plane could still be
done with a framebuffer and a special flag in the addfb2 ioctl to use
live sources as backing storage ids instead of gem/ttm handles.

That would also give you a good point to enforce pixel format
compatibility: As soon as someone created a framebuffer for a live source
you disallow pixel format changes in the v4l pipeline to make sure things
will fit.

> Patches 3/5 to 5/5 then implement support for live sources in the R-Car DU
> driver.
> 
> Nothing here is set in stone. One point I'm not sure about is whether live
> sources support should be enabled for setplane or only for atomic updates.
> Other parts of the API can certainly be modified as well, and I'm open for
> totally different implementations as well.

Imo this should be irrespective of atomic or not really. And by using
magic framebuffers as the link we'd get that for free.

Cheers, Daniel

> 
> Laurent Pinchart (5):
>   drm: Add live source object
>   drm: Connect live source to plane
>   drm/rcar-du: Add VSP1 support to the planes allocator
>   drm/rcar-du: Add VSP1 live source support
>   drm/rcar-du: Restart the DU group when a plane source changes
> 
>  drivers/gpu/drm/armada/armada_overlay.c     |   2 +-
>  drivers/gpu/drm/drm_atomic.c                |   7 +
>  drivers/gpu/drm/drm_atomic_helper.c         |   4 +
>  drivers/gpu/drm/drm_crtc.c                  | 365 ++++++++++++++++++++++++++--
>  drivers/gpu/drm/drm_fops.c                  |   6 +-
>  drivers/gpu/drm/drm_ioctl.c                 |   3 +
>  drivers/gpu/drm/drm_plane_helper.c          |   1 +
>  drivers/gpu/drm/exynos/exynos_drm_crtc.c    |   4 +-
>  drivers/gpu/drm/exynos/exynos_drm_plane.c   |   3 +-
>  drivers/gpu/drm/exynos/exynos_drm_plane.h   |   3 +-
>  drivers/gpu/drm/i915/intel_display.c        |   4 +-
>  drivers/gpu/drm/i915/intel_sprite.c         |   2 +-
>  drivers/gpu/drm/imx/ipuv3-plane.c           |   3 +-
>  drivers/gpu/drm/nouveau/dispnv04/overlay.c  |   6 +-
>  drivers/gpu/drm/omapdrm/omap_plane.c        |   1 +
>  drivers/gpu/drm/rcar-du/rcar_du_crtc.c      |  10 +-
>  drivers/gpu/drm/rcar-du/rcar_du_drv.c       |   6 +-
>  drivers/gpu/drm/rcar-du/rcar_du_drv.h       |   3 +
>  drivers/gpu/drm/rcar-du/rcar_du_group.c     |  21 +-
>  drivers/gpu/drm/rcar-du/rcar_du_group.h     |   2 +
>  drivers/gpu/drm/rcar-du/rcar_du_kms.c       |  62 ++++-
>  drivers/gpu/drm/rcar-du/rcar_du_plane.c     | 196 ++++++++++++---
>  drivers/gpu/drm/rcar-du/rcar_du_plane.h     |  11 +
>  drivers/gpu/drm/rcar-du/rcar_du_regs.h      |   1 +
>  drivers/gpu/drm/rockchip/rockchip_drm_vop.c |   5 +-
>  drivers/gpu/drm/shmobile/shmob_drm_plane.c  |   3 +-
>  drivers/gpu/drm/sti/sti_drm_plane.c         |   3 +-
>  drivers/gpu/drm/sti/sti_hqvdp.c             |   2 +-
>  include/drm/drmP.h                          |   3 +
>  include/drm/drm_atomic_helper.h             |   1 +
>  include/drm/drm_crtc.h                      |  48 ++++
>  include/drm/drm_plane_helper.h              |   1 +
>  include/uapi/drm/drm.h                      |   4 +
>  include/uapi/drm/drm_mode.h                 |  32 +++
>  34 files changed, 728 insertions(+), 100 deletions(-)
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [RFC/PATCH 0/5] Add live source objects to DRM

[Laurent Pinchart]

Hi Daniel,

On Monday 16 March 2015 09:35:22 Daniel Vetter wrote:
> On Sun, Mar 15, 2015 at 11:55:35PM +0200, Laurent Pinchart wrote:
> > Hello,
> > 
> > I have a feeling that RFC/PATCH will work better than just RFC, so here's
> > a patch set that adds a new object named live source to DRM.
> > 
> > The need comes from the Renesas R-Car SoCs in which a video processing
> > engine (named VSP1) that operates from memory to memory has one output
> > directly connected to a plane of the display engine (DU) without going
> > through memory.
> > 
> > The VSP1 is supported by a V4L2 driver. While it could be argued that it
> > should instead be supported directly by the DRM rcar-du driver, this
> > wouldn't be a good idea for at least two reasons. First, the R-Car SoCs
> > contain several VSP1 instances, of which only a subset have a direct DU
> > connection. The only other instances operate solely in memory to memory
> > mode. Then, the VSP1 is a video processing engine and not a display
> > engine. Its features are easily supported by the V4L2 API, but don't map
> > to the DRM/KMS API. Significant changes to DRM/KMS would be required,
> > beyond what is in my opinion an acceptable scope for a display API.
> > 
> > Now that the need to interface two separate devices supported by two
> > different drivers in two separate subsystems has been established, we
> > need an API to do so. It should be noted that while that API doesn't
> > exist in the mainline kernel, the need isn't limited to Renesas SoCs.
> > 
> > This patch set proposes one possible solution for the problem in the form
> > of a new DRM object named live source. Live sources are created by
> > drivers to model hardware connections between a plane input and an
> > external source, and are attached to planes through the KMS userspace
> > API.
> > 
> > Patch 1/5 adds live source objects to DRM, with an in-kernel API for
> > drivers to register the sources, and a userspace API to enumerate and
> > configure them. Configuring a live source sets the width, height and
> > pixel format of the video stream. This should ideally be queried directly
> > from the driver that supports the live source device, but I've decided
> > not to implement such communication between V4L2 and DRM/KMS at the
> > moment to keep the proposal simple.
> > 
> > Patch 2/2 implements connection between live sources and planes. This
> > takes different forms depending on whether drivers use the setplane or the
> > atomic updates API:
> > 
> > - For setplane, the fb field can now contain a live source ID in addition
> >   to a framebuffer ID. As DRM allocates object IDs from a single ID space,
> >   the type can be inferred from the ID. This makes specifying both a
> >   framebuffer and a live source impossible, which isn't an issue given
> >   that such a configuration would be invalid.
> >   
> >   The live source is looked up by the DRM core and passed as a pointer to
> >   the .update_plane() operation. Unlike framebuffers live sources are not
> >   refcounted as they're created statically at driver initialization time.
> > 
> > - For atomic update, a new SRC_ID property has been added to planes. The
> >   live source is looked up from the source ID and stored into the plane
> >   state.
> 
> What about directly treating live sources as (very) special framebuffers?
> A bunch of reasons:
> - All the abi fu above gets resolved naturally.
> - You have lot of duplication between fb and live source wrt size,
>   possible/selected pixel format and other stuff.
> - The backing storage of framebuffers is fully opaque anyway ...
> 
> I think we still need separate live source objects though for things like
> telling userspace what v4l thing it corresponds to, and for getting at the
> pixel format. But connecting the live source with the plane could still be
> done with a framebuffer and a special flag in the addfb2 ioctl to use
> live sources as backing storage ids instead of gem/ttm handles.
> 
> That would also give you a good point to enforce pixel format
> compatibility: As soon as someone created a framebuffer for a live source
> you disallow pixel format changes in the v4l pipeline to make sure things
> will fit.

That's an interesting idea, I'll give it a try. This will have to wait a 
little bit though, I'll be busy with other tasks this week, and I'll be 
attending ELC next week.

Thank you for the quick review and the proposal, it's very appreciated.

> > Patches 3/5 to 5/5 then implement support for live sources in the R-Car DU
> > driver.
> > 
> > Nothing here is set in stone. One point I'm not sure about is whether live
> > sources support should be enabled for setplane or only for atomic updates.
> > Other parts of the API can certainly be modified as well, and I'm open for
> > totally different implementations as well.
> 
> Imo this should be irrespective of atomic or not really. And by using
> magic framebuffers as the link we'd get that for free.
> 
> Cheers, Daniel
> 
> > Laurent Pinchart (5):
> >   drm: Add live source object
> >   drm: Connect live source to plane
> >   drm/rcar-du: Add VSP1 support to the planes allocator
> >   drm/rcar-du: Add VSP1 live source support
> >   drm/rcar-du: Restart the DU group when a plane source changes
> >  
> >  drivers/gpu/drm/armada/armada_overlay.c     |   2 +-
> >  drivers/gpu/drm/drm_atomic.c                |   7 +
> >  drivers/gpu/drm/drm_atomic_helper.c         |   4 +
> >  drivers/gpu/drm/drm_crtc.c                  | 365 +++++++++++++++++++++--
> >  drivers/gpu/drm/drm_fops.c                  |   6 +-
> >  drivers/gpu/drm/drm_ioctl.c                 |   3 +
> >  drivers/gpu/drm/drm_plane_helper.c          |   1 +
> >  drivers/gpu/drm/exynos/exynos_drm_crtc.c    |   4 +-
> >  drivers/gpu/drm/exynos/exynos_drm_plane.c   |   3 +-
> >  drivers/gpu/drm/exynos/exynos_drm_plane.h   |   3 +-
> >  drivers/gpu/drm/i915/intel_display.c        |   4 +-
> >  drivers/gpu/drm/i915/intel_sprite.c         |   2 +-
> >  drivers/gpu/drm/imx/ipuv3-plane.c           |   3 +-
> >  drivers/gpu/drm/nouveau/dispnv04/overlay.c  |   6 +-
> >  drivers/gpu/drm/omapdrm/omap_plane.c        |   1 +
> >  drivers/gpu/drm/rcar-du/rcar_du_crtc.c      |  10 +-
> >  drivers/gpu/drm/rcar-du/rcar_du_drv.c       |   6 +-
> >  drivers/gpu/drm/rcar-du/rcar_du_drv.h       |   3 +
> >  drivers/gpu/drm/rcar-du/rcar_du_group.c     |  21 +-
> >  drivers/gpu/drm/rcar-du/rcar_du_group.h     |   2 +
> >  drivers/gpu/drm/rcar-du/rcar_du_kms.c       |  62 ++++-
> >  drivers/gpu/drm/rcar-du/rcar_du_plane.c     | 196 ++++++++++++---
> >  drivers/gpu/drm/rcar-du/rcar_du_plane.h     |  11 +
> >  drivers/gpu/drm/rcar-du/rcar_du_regs.h      |   1 +
> >  drivers/gpu/drm/rockchip/rockchip_drm_vop.c |   5 +-
> >  drivers/gpu/drm/shmobile/shmob_drm_plane.c  |   3 +-
> >  drivers/gpu/drm/sti/sti_drm_plane.c         |   3 +-
> >  drivers/gpu/drm/sti/sti_hqvdp.c             |   2 +-
> >  include/drm/drmP.h                          |   3 +
> >  include/drm/drm_atomic_helper.h             |   1 +
> >  include/drm/drm_crtc.h                      |  48 ++++
> >  include/drm/drm_plane_helper.h              |   1 +
> >  include/uapi/drm/drm.h                      |   4 +
> >  include/uapi/drm/drm_mode.h                 |  32 +++
> >  34 files changed, 728 insertions(+), 100 deletions(-)

-- 
Regards,

Laurent Pinchart