[RFC 2.6.28 1/2] fbdev: add damage support

[RFC 2.6.28 1/2] fbdev: add damage support

[Jaya Kumar]

Hi Geert, Krzysztof, Magnus, fbdev friends,

This is version 2 of an idea about allowing userspace to provide damage
information to drivers. Thanks to Magnus, Tomi, and others for discussion
and ideas.

This is just a first pass implementation. The primary change from before
is that we no longer do any processing of the damage within fbdev.
Instead, we pass it straight through to the driver for handling. Please
note I have not added a get_damage in order to postpone the question of how
best to expose optimization information to userspace. I felt that this
would be a good way for us to determine if we have consensus on the basic
part of userspace API first. If we agree on that, then later we can work
out how (and if) we should abstract the kernel side further. That is, if
we can abstract some of the processing within fbdev itself and simplify
the driver handlers. Please let me know your thoughts. 

Thanks,
jaya

This patch adds the ability for userspace applications to provide damage
information to the underlying driver. This is useful in scenarios where the
underlying driver can perform transfer optimizations based on knowing
exactly which framebuffer areas that were updated. This functionality is
exposed by using a simple x,y,w,h bounding box structure. Userspace is
expected to perform its damage and then perform the ioctl. The underlying
driver is free to use this information as it sees fit including ignoring it
if it chooses to. An example use case will be provided in the case of
broadsheetfb.c where the damage information is aggregated for deferred use.

Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Krzysztof Helt <krzysztof.h1@poczta.fm>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Tomi Valkeinen <tomi.valkeinen@nokia.com>
Cc: armbru@redhat.com
Cc: lethal@linux-sh.org
Cc: adaplas@gmail.com
Cc: linux-fbdev-devel@lists.sourceforge.net
Signed-off-by: Jaya Kumar <jayakumar.lkml@gmail.com>
---
 drivers/video/fbmem.c |   15 +++++++++++++++
 include/linux/fb.h    |   19 ++++++++++++++++++-
 2 files changed, 33 insertions(+), 1 deletions(-)

diff --git a/drivers/video/fbmem.c b/drivers/video/fbmem.c
index 756efeb..11a0e05 100644
--- a/drivers/video/fbmem.c
+++ b/drivers/video/fbmem.c
@@ -1006,6 +1006,14 @@ fb_blank(struct fb_info *info, int blank)
  	return ret;
 }
 
+static int fb_set_damage(struct fb_info *info, struct fb_damage_user *udamage)
+{
+	if (info->fbops->fb_set_damage)
+		return info->fbops->fb_set_damage(info, udamage);
+
+	return -ENOTTY;
+}
+
 static long do_fb_ioctl(struct fb_info *info, unsigned int cmd,
 			unsigned long arg)
 {
@@ -1015,6 +1023,7 @@ static long do_fb_ioctl(struct fb_info *info, unsigned int cmd,
 	struct fb_con2fbmap con2fb;
 	struct fb_cmap_user cmap;
 	struct fb_event event;
+	struct fb_damage_user udamage;
 	void __user *argp = (void __user *)arg;
 	long ret = 0;
 
@@ -1116,6 +1125,12 @@ static long do_fb_ioctl(struct fb_info *info, unsigned int cmd,
 		info->flags &= ~FBINFO_MISC_USEREVENT;
 		release_console_sem();
 		break;;
+	case FBIOPUT_DAMAGE:
+		if (copy_from_user(&udamage, argp, sizeof(udamage)))
+			ret = -EFAULT;
+		else
+			ret = fb_set_damage(info, &udamage);
+		break;
 	default:
 		if (fb->fb_ioctl == NULL)
 			ret = -ENOTTY;
diff --git a/include/linux/fb.h b/include/linux/fb.h
index 1ee63df..ac9e224 100644
--- a/include/linux/fb.h
+++ b/include/linux/fb.h
@@ -37,7 +37,8 @@ struct dentry;
 #define FBIOGET_HWCINFO         0x4616
 #define FBIOPUT_MODEINFO        0x4617
 #define FBIOGET_DISPINFO        0x4618
-
+/* to allow userspace to provide screen damage information to drivers */
+#define FBIOPUT_DAMAGE          0x4619
 
 #define FB_TYPE_PACKED_PIXELS		0	/* Packed Pixels	*/
 #define FB_TYPE_PLANES			1	/* Non interleaved planes */
@@ -357,6 +358,18 @@ struct fb_image {
 	struct fb_cmap cmap;	/* color map info */
 };
 
+struct fb_damage_rect {
+	__u16 x;
+	__u16 y;
+	__u16 w;
+	__u16 h;
+};
+
+struct fb_damage_user {
+	__u32 len;		/* Number of entries */
+	struct fb_damage_rect __user *rects; /* array of damage rectangles */
+};
+
 /*
  * hardware cursor control
  */
@@ -672,6 +685,10 @@ struct fb_ops {
 	/* get capability given var */
 	void (*fb_get_caps)(struct fb_info *info, struct fb_blit_caps *caps,
 			    struct fb_var_screeninfo *var);
+
+	/* provide damage information */
+	int (*fb_set_damage)(struct fb_info *info,
+				struct fb_damage_user *damage);
 };
 
 #ifdef CONFIG_FB_TILEBLITTING
-- 
1.5.2.3


------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword

[RFC 2.6.28 2/2] broadsheetfb: add damage handling

[Jaya Kumar]

This patch adds support within broadsheetfb to process damage information
provided by userspace in order to perform more accurate partial updates. It
does not yet handle the scenario where the user may be switching from a
damage providing client to a non-damage providing client within a single
transfer interval.

Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Krzysztof Helt <krzysztof.h1@poczta.fm>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: armbru@redhat.com
Cc: lethal@linux-sh.org
Cc: adaplas@gmail.com
Cc: linux-fbdev-devel@lists.sourceforge.net
Signed-off-by: Jaya Kumar <jayakumar.lkml@gmail.com>
---
 drivers/video/broadsheetfb.c |  165 +++++++++++++++++++++++++++++++++++++++++-
 include/video/broadsheetfb.h |   10 +++
 2 files changed, 174 insertions(+), 1 deletions(-)

diff --git a/drivers/video/broadsheetfb.c b/drivers/video/broadsheetfb.c
index 509cb92..5654376 100644
--- a/drivers/video/broadsheetfb.c
+++ b/drivers/video/broadsheetfb.c
@@ -293,6 +293,83 @@ static void broadsheetfb_dpy_update(struct broadsheetfb_par *par)
 
 }
 
+static void broadsheetfb_upd_full(struct broadsheetfb_par *par)
+{
+	u16 args[5];
+	args[0] = 0x4300;
+	broadsheet_send_cmdargs(par, BS_CMD_UPD_FULL, 1, args);
+
+	broadsheet_send_command(par, BS_CMD_WAIT_DSPE_TRG);
+
+	broadsheet_send_command(par, BS_CMD_WAIT_DSPE_FREND);
+
+	par->board->wait_for_rdy(par);
+}
+
+static void broadsheetfb_load_image_area(struct broadsheetfb_par *par, u16 x,
+						u16 y, u16 w, u16 h)
+{
+	u16 args[5];
+	unsigned char *sbuf = (unsigned char *)par->info->screen_base;
+	unsigned char *buf;
+	int j;
+
+	/* x must be a multiple of 4 so drop the lower bits */
+	x &= 0xFFFC;
+
+	/* y must be a multiple of 4 so drop the lower bits */
+	y &= 0xFFFC;
+
+	args[0] = 0x3 << 4;
+	args[1] = x;
+	args[2] = y;
+	args[3] = w;
+	args[4] = h;
+	broadsheet_send_cmdargs(par, BS_CMD_LD_IMG_AREA, 5, args);
+
+	args[0] = 0x154;
+	broadsheet_send_cmdargs(par, BS_CMD_WR_REG, 1, args);
+
+	for (j = y; j < y + h; j++) {
+		buf = sbuf + x + (j * par->info->var.xres);
+		broadsheet_burst_write(par, (w+1)/2, (u16 *) buf);
+	}
+	broadsheet_send_command(par, BS_CMD_LD_IMG_END);
+}
+
+static int broadsheetfb_process_damage(struct fb_info *info)
+{
+	struct broadsheetfb_par *par = info->par;
+	struct broadsheetfb_damage *cur, *next;
+	int ret = -EINVAL;
+	int i;
+	struct fb_damage_rect *rect;
+
+	mutex_lock(&par->damage_lock);
+	/* if there is no damage, then caller has to do work to figure out
+	 * the changes on its own */
+	if (list_empty(&par->damagelist))
+		goto finish;
+
+	list_for_each_entry_safe(cur, next, &par->damagelist, list) {
+		for (i = 0; i < cur->len; i++) {
+			rect = &(cur->rects[i]);
+			broadsheetfb_load_image_area(par, rect->x, rect->y,
+							rect->w, rect->h);
+		}
+		list_del(&cur->list);
+		kfree(cur->rects);
+		kfree(cur);
+	}
+
+	broadsheetfb_upd_full(par);
+	ret = 0;
+finish:
+	mutex_unlock(&par->damage_lock);
+	return ret;
+}
+
+
 /* this is called back from the deferred io workqueue */
 static void broadsheetfb_dpy_deferred_io(struct fb_info *info,
 				struct list_head *pagelist)
@@ -304,6 +381,14 @@ static void broadsheetfb_dpy_deferred_io(struct fb_info *info,
 	int h_inc;
 	u16 yres = info->var.yres;
 	u16 xres = info->var.xres;
+	int ret;
+
+	/* if we have damage data then use it exclusively */
+	ret = broadsheetfb_process_damage(info);
+	if (!ret)
+		return;
+
+	/* if no damage then rely on page information */
 
 	/* height increment is fixed per page */
 	h_inc = DIV_ROUND_UP(PAGE_SIZE , xres);
@@ -413,6 +498,67 @@ static ssize_t broadsheetfb_write(struct fb_info *info, const char __user *buf,
 	return (err) ? err : count;
 }
 
+static int broadsheetfb_alloc_damage(struct broadsheetfb_damage **damagep,
+					int len)
+{
+	struct broadsheetfb_damage *damage;
+
+	damage = kzalloc(sizeof(struct broadsheetfb_damage), GFP_KERNEL);
+	if (!damage)
+		return -ENOMEM;
+
+	damage->len = len;
+	damage->rects = kzalloc(sizeof(struct fb_damage_rect) * len,
+				GFP_KERNEL);
+	if (!damage->rects) {
+		kfree(damage);
+		return -ENOMEM;
+	}
+
+	*damagep = damage;
+	return 0;
+}
+
+static void broadsheetfb_free_damage(struct broadsheetfb_damage *damage)
+{
+	if (damage)
+		kfree(damage->rects);
+	kfree(damage);
+}
+
+static int broadsheetfb_set_damage(struct fb_info *info,
+				struct fb_damage_user *udamage)
+{
+	struct broadsheetfb_par *par = info->par;
+	struct broadsheetfb_damage *damage;
+	int ret = -EINVAL;
+	int size = udamage->len;
+
+	if (size > BROADSHEETFB_DAMAGE_COUNT_MAX)
+		goto fail;
+
+	ret = broadsheetfb_alloc_damage(&damage, size);
+	if (ret)
+		goto fail;
+
+	if (copy_from_user(damage->rects, udamage->rects,
+				sizeof(struct fb_damage_rect)*size)) {
+		ret = -EFAULT;
+		goto fail2;
+	}
+
+	mutex_lock(&par->damage_lock);
+	list_add_tail(&damage->list, &par->damagelist);
+	mutex_unlock(&par->damage_lock);
+
+	return 0;
+
+fail2:
+	broadsheetfb_free_damage(damage);
+fail:
+	return ret;
+}
+
 static struct fb_ops broadsheetfb_ops = {
 	.owner		= THIS_MODULE,
 	.fb_read        = fb_sys_read,
@@ -420,6 +566,7 @@ static struct fb_ops broadsheetfb_ops = {
 	.fb_fillrect	= broadsheetfb_fillrect,
 	.fb_copyarea	= broadsheetfb_copyarea,
 	.fb_imageblit	= broadsheetfb_imageblit,
+	.fb_set_damage	= broadsheetfb_set_damage,
 };
 
 static struct fb_deferred_io broadsheetfb_defio = {
@@ -498,6 +645,9 @@ static int __devinit broadsheetfb_probe(struct platform_device *dev)
 
 	broadsheet_init(par);
 
+	INIT_LIST_HEAD(&par->damagelist);
+	mutex_init(&par->damage_lock);
+
 	retval = register_framebuffer(info);
 	if (retval < 0)
 		goto err_free_irq;
@@ -512,8 +662,10 @@ static int __devinit broadsheetfb_probe(struct platform_device *dev)
 
 err_free_irq:
 	board->cleanup(par);
+	mutex_destroy(&par->damage_lock);
 err_cmap:
 	fb_dealloc_cmap(&info->cmap);
+	fb_deferred_io_cleanup(info);
 err_vfree:
 	vfree(videomemory);
 err_fb_rel:
@@ -524,16 +676,27 @@ err:
 
 }
 
+static void __devexit broadsheetfb_cleanup_damage(struct broadsheetfb_par *par)
+{
+	broadsheetfb_process_damage(par->info);
+	mutex_destroy(&par->damage_lock);
+}
+
 static int __devexit broadsheetfb_remove(struct platform_device *dev)
 {
 	struct fb_info *info = platform_get_drvdata(dev);
 
 	if (info) {
 		struct broadsheetfb_par *par = info->par;
+
 		unregister_framebuffer(info);
-		fb_deferred_io_cleanup(info);
+
+		/* cleanup any pending damage */
+		broadsheetfb_cleanup_damage(par);
+
 		par->board->cleanup(par);
 		fb_dealloc_cmap(&info->cmap);
+		fb_deferred_io_cleanup(info);
 		vfree((void *)info->screen_base);
 		module_put(par->board->owner);
 		framebuffer_release(info);
diff --git a/include/video/broadsheetfb.h b/include/video/broadsheetfb.h
index a758534..14f4594 100644
--- a/include/video/broadsheetfb.h
+++ b/include/video/broadsheetfb.h
@@ -34,6 +34,8 @@
 #define BS_DC 	0x02
 #define BS_WR 	0x03
 
+#define BROADSHEETFB_DAMAGE_COUNT_MAX 255
+
 /* struct used by broadsheet. board specific stuff comes from *board */
 struct broadsheetfb_par {
 	struct fb_info *info;
@@ -41,6 +43,8 @@ struct broadsheetfb_par {
 	void (*write_reg)(struct broadsheetfb_par *, u16 reg, u16 val);
 	u16 (*read_reg)(struct broadsheetfb_par *, u16 reg);
 	wait_queue_head_t waitq;
+	struct mutex damage_lock;
+	struct list_head damagelist;
 };
 
 /* board specific routines */
@@ -56,4 +60,10 @@ struct broadsheet_board {
 	int (*setup_irq)(struct fb_info *);
 };
 
+struct broadsheetfb_damage {
+	struct list_head list;
+	__u32 len;		/* Number of entries */
+	struct fb_damage_rect *rects; /* array of damage rectangles */
+};
+
 #endif
-- 
1.5.2.3


------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword

Re: [RFC 2.6.28 1/2] fbdev: add damage support

[Magnus Damm]

On Sun, Jan 25, 2009 at 9:15 PM, Jaya Kumar <jayakumar.lkml@gmail.com> wrote:
> This is version 2 of an idea about allowing userspace to provide damage
> information to drivers. Thanks to Magnus, Tomi, and others for discussion
> and ideas.

...

> This patch adds the ability for userspace applications to provide damage
> information to the underlying driver. This is useful in scenarios where the
> underlying driver can perform transfer optimizations based on knowing
> exactly which framebuffer areas that were updated. This functionality is
> exposed by using a simple x,y,w,h bounding box structure. Userspace is
> expected to perform its damage and then perform the ioctl. The underlying
> driver is free to use this information as it sees fit including ignoring it
> if it chooses to. An example use case will be provided in the case of
> broadsheetfb.c where the damage information is aggregated for deferred use.
>
> Cc: Geert Uytterhoeven <geert@linux-m68k.org>
> Cc: Krzysztof Helt <krzysztof.h1@poczta.fm>
> Cc: Magnus Damm <magnus.damm@gmail.com>
> Cc: Tomi Valkeinen <tomi.valkeinen@nokia.com>
> Cc: armbru@redhat.com
> Cc: lethal@linux-sh.org
> Cc: adaplas@gmail.com
> Cc: linux-fbdev-devel@lists.sourceforge.net
> Signed-off-by: Jaya Kumar <jayakumar.lkml@gmail.com>

Looking good except that the patch doesn't apply on new kernels.

Acked-by: Magnus Damm <damm@igel.co.jp>

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword