[PATCH 0/2] video: s3c-fb: Add window positioning support

[PATCH 0/2] video: s3c-fb: Add window positioning support

[Ajay Kumar]

These patches are created against "for-next" branch of Kukjin Kim's tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/kgene/linux-samsung.git

This patch adds support for positioning of the FB windows on the LCD screen.

This patchset creates an ioctl and defines a data structure which are
specific to samsung SOCs, to hold the window position.

Just as a note, there are many drivers like mx3fb.c, au1200fb.c and OMAP
seem to be doing window/plane positioning in their driver code.
Is it possible to have this window positioning support at a common place?

For instance, we can have a common struture and ioctl number in 
include/linux/fb.h as below:

	#define FBIOPOS_OVERLAY_WIN    _IOW('F', 0x21, struct fb_overlay_win_pos)

	struct fb_overlay_win_pos {
		__u32 win_pos_x;  /* x-offset of window from LCD(0,0) */
		__u32 win_pos_y;  /* y-offset of window from LCD(0,0) */
	};

where LCD(0,0) means the first pixel of the LCD screen.
Individual drivers can have implementation for this ioctl.

To Kukjin Kim,
  [PATCH 1/2] ARM: SAMSUNG: Add Window Positioning Support for s3c-fb driver

To Paul Mundt, Florian Tobias Schandinat
  [PATCH 2/2] video: s3c-fb: Modify s3c-fb driver to support window positioning

 arch/arm/plat-samsung/include/plat/fb.h |   14 +++++++++++
 drivers/video/s3c-fb.c                  |   37 ++++++++++++++++++++++++++----
 2 files changed, 46 insertions(+), 5 deletions(-)

[PATCH 1/2] ARM: SAMSUNG: Add Window Positioning Support for s3c-fb

[Ajay Kumar]

This patch:
	--adds a data-structure to hold the current position of windows.
	--adds an ioctl number to support dynamic positioning the windows.

Signed-off-by: Ajay Kumar <ajaykumar.rs@samsung.com>
Signed-off-by: Banajit Goswami <banajit.g@samsung.com>
---
 arch/arm/plat-samsung/include/plat/fb.h |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/arch/arm/plat-samsung/include/plat/fb.h b/arch/arm/plat-samsung/include/plat/fb.h
index bd79c0a..77ed75c 100644
--- a/arch/arm/plat-samsung/include/plat/fb.h
+++ b/arch/arm/plat-samsung/include/plat/fb.h
@@ -22,6 +22,18 @@
  */
 #define S3C_FB_MAX_WIN	(5)
 
+/* struct s3cfb_window_pos
+ * @win_pos_x: X-coordinate of window from the left.
+ * @win_pos_y: Y-coordinate of window from the top.
+ */
+struct s3cfb_window_pos {
+	int	win_pos_x;
+	int	win_pos_y;
+};
+
+/* Custom ioctl */
+#define S3CFB_WIN_POSITION	_IOW('F', 1, struct s3cfb_window_pos)
+
 /**
  * struct s3c_fb_pd_win - per window setup data
  * @win_mode: The display parameters to initialise (not for window 0)
@@ -35,6 +47,8 @@ struct s3c_fb_pd_win {
 	unsigned short		max_bpp;
 	unsigned short		virtual_x;
 	unsigned short		virtual_y;
+
+	struct s3cfb_window_pos		winpos;
 };
 
 /**
-- 
1.7.0.4

RE: [PATCH 1/2] ARM: SAMSUNG: Add Window Positioning Support for

[Jingoo Han]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 2118 bytes --]


Hi, Ajay.
> -----Original Message-----
> From: Ajay Kumar [mailto:ajaykumar.rs@samsung.com]
> Sent: Friday, August 26, 2011 4:52 AM
> To: linux-samsung-soc@vger.kernel.org; linux-fbdev@vger.kernel.org; linux-
> arm-kernel@lists.infradead.org
> Cc: FlorianSchandinat@gmx.de; lethal@linux-sh.org; jg1.han@samsung.com;
> m.szyprowski@samsung.com; ben-linux@fluff.org; banajit.g@samsung.com
> Subject: [PATCH 1/2] ARM: SAMSUNG: Add Window Positioning Support for s3c-
> fb driver
> 
> This patch:
> 	--adds a data-structure to hold the current position of windows.
> 	--adds an ioctl number to support dynamic positioning the windows.
> 
> Signed-off-by: Ajay Kumar <ajaykumar.rs@samsung.com>
> Signed-off-by: Banajit Goswami <banajit.g@samsung.com>
> ---
>  arch/arm/plat-samsung/include/plat/fb.h |   14 ++++++++++++++
>  1 files changed, 14 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/arm/plat-samsung/include/plat/fb.h b/arch/arm/plat-
> samsung/include/plat/fb.h
> index bd79c0a..77ed75c 100644
> --- a/arch/arm/plat-samsung/include/plat/fb.h
> +++ b/arch/arm/plat-samsung/include/plat/fb.h
> @@ -22,6 +22,18 @@
>   */
>  #define S3C_FB_MAX_WIN	(5)
> 
> +/* struct s3cfb_window_pos
> + * @win_pos_x: X-coordinate of window from the left.
> + * @win_pos_y: Y-coordinate of window from the top.
> + */
> +struct s3cfb_window_pos {
> +	int	win_pos_x;
> +	int	win_pos_y;
> +};
> +
> +/* Custom ioctl */
> +#define S3CFB_WIN_POSITION	_IOW('F', 1, struct s3cfb_window_pos)
Can you change the number from 1 to 203 as follows?
+#define S3CFB_WIN_POSITION	_IOW('F', 203, struct s3cfb_window_pos)
We already use this number as S3CFB_WIN_POSITION.
> +
>  /**
>   * struct s3c_fb_pd_win - per window setup data
>   * @win_mode: The display parameters to initialise (not for window 0)
> @@ -35,6 +47,8 @@ struct s3c_fb_pd_win {
>  	unsigned short		max_bpp;
>  	unsigned short		virtual_x;
>  	unsigned short		virtual_y;
> +
> +	struct s3cfb_window_pos		winpos;
>  };
> 
>  /**
> --
> 1.7.0.4


ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ýöÝzÿâžØ^n‡r¡ö¦zË\x1aëh™¨è­Ú&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~†­†Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm

[PATCH 2/2] video: s3c-fb: Modify s3c-fb driver to support window

[Ajay Kumar]

This patch modifies the existing s3c-fb driver to provide
flexibility to the user to reposition the framebuffer windows.

Signed-off-by: Ajay Kumar <ajaykumar.rs@samsung.com>
Signed-off-by: Banajit Goswami <banajit.g@samsung.com>
---
 drivers/video/s3c-fb.c |   37 ++++++++++++++++++++++++++++++++-----
 1 files changed, 32 insertions(+), 5 deletions(-)

diff --git a/drivers/video/s3c-fb.c b/drivers/video/s3c-fb.c
index 0fda252..41179d7 100644
--- a/drivers/video/s3c-fb.c
+++ b/drivers/video/s3c-fb.c
@@ -442,6 +442,7 @@ static int s3c_fb_set_par(struct fb_info *info)
 	struct fb_var_screeninfo *var = &info->var;
 	struct s3c_fb_win *win = info->par;
 	struct s3c_fb *sfb = win->parent;
+	struct s3cfb_window_pos *winpos = &win->windata->winpos;
 	void __iomem *regs = sfb->regs;
 	void __iomem *buf = regs;
 	int win_no = win->index;
@@ -539,12 +540,13 @@ static int s3c_fb_set_par(struct fb_info *info)
 
 	/* write 'OSD' registers to control position of framebuffer */
 
-	data = VIDOSDxA_TOPLEFT_X(0) | VIDOSDxA_TOPLEFT_Y(0);
+	data = VIDOSDxA_TOPLEFT_X(winpos->win_pos_x) |
+	       VIDOSDxA_TOPLEFT_Y(winpos->win_pos_y);
 	writel(data, regs + VIDOSD_A(win_no, sfb->variant));
 
-	data = VIDOSDxB_BOTRIGHT_X(s3c_fb_align_word(var->bits_per_pixel,
-						     var->xres - 1)) |
-	       VIDOSDxB_BOTRIGHT_Y(var->yres - 1);
+	data = VIDOSDxB_BOTRIGHT_X((s3c_fb_align_word(var->bits_per_pixel,
+	       (winpos->win_pos_x + var->xres - 1)))) |
+	       VIDOSDxB_BOTRIGHT_Y((winpos->win_pos_y + var->yres - 1));
 
 	writel(data, regs + VIDOSD_B(win_no, sfb->variant));
 
@@ -999,8 +1001,10 @@ static int s3c_fb_ioctl(struct fb_info *info, unsigned int cmd,
 {
 	struct s3c_fb_win *win = info->par;
 	struct s3c_fb *sfb = win->parent;
-	int ret;
+	struct s3cfb_window_pos *winpos = &win->windata->winpos;
+	int ret = 0;
 	u32 crtc;
+	u32 data;
 
 	switch (cmd) {
 	case FBIO_WAITFORVSYNC:
@@ -1011,6 +1015,29 @@ static int s3c_fb_ioctl(struct fb_info *info, unsigned int cmd,
 
 		ret = s3c_fb_wait_for_vsync(sfb, crtc);
 		break;
+	case S3CFB_WIN_POSITION:
+		if (copy_from_user(winpos, (u32 __user *)arg,
+					sizeof(struct s3cfb_window_pos))) {
+			ret = -EFAULT;
+			break;
+		}
+
+		shadow_protect_win(win, 1);
+
+		/* write 'OSD' registers to set position of the window */
+		data = VIDOSDxA_TOPLEFT_X(winpos->win_pos_x) |
+		       VIDOSDxA_TOPLEFT_Y(winpos->win_pos_y);
+		writel(data, sfb->regs + VIDOSD_A(win->index, sfb->variant));
+
+		data = VIDOSDxB_BOTRIGHT_X(
+				s3c_fb_align_word(info->var.bits_per_pixel,
+				(winpos->win_pos_x + info->var.xres - 1)));
+		data |=	VIDOSDxB_BOTRIGHT_Y(winpos->win_pos_y +
+				info->var.yres - 1);
+		writel(data, sfb->regs + VIDOSD_B(win->index, sfb->variant));
+
+		shadow_protect_win(win, 0);
+		break;
 	default:
 		ret = -ENOTTY;
 	}
-- 
1.7.0.4

RE: [PATCH 2/2] video: s3c-fb: Modify s3c-fb driver to support window

[JinGoo Han]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 4182 bytes --]

Hi, Ajay.
> -----Original Message-----
> From: Ajay Kumar [mailto:ajaykumar.rs@samsung.com]
> Sent: Friday, August 26, 2011 4:52 AM
> To: linux-samsung-soc@vger.kernel.org; linux-fbdev@vger.kernel.org; linux-
> arm-kernel@lists.infradead.org
> Cc: FlorianSchandinat@gmx.de; lethal@linux-sh.org; jg1.han@samsung.com;
> m.szyprowski@samsung.com; ben-linux@fluff.org; banajit.g@samsung.com
> Subject: [PATCH 2/2] video: s3c-fb: Modify s3c-fb driver to support window
> positioning
> 
> This patch modifies the existing s3c-fb driver to provide
> flexibility to the user to reposition the framebuffer windows.
> 
> Signed-off-by: Ajay Kumar <ajaykumar.rs@samsung.com>
> Signed-off-by: Banajit Goswami <banajit.g@samsung.com>
> ---
>  drivers/video/s3c-fb.c |   37 ++++++++++++++++++++++++++++++++-----
>  1 files changed, 32 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/video/s3c-fb.c b/drivers/video/s3c-fb.c
> index 0fda252..41179d7 100644
> --- a/drivers/video/s3c-fb.c
> +++ b/drivers/video/s3c-fb.c
> @@ -442,6 +442,7 @@ static int s3c_fb_set_par(struct fb_info *info)
>  	struct fb_var_screeninfo *var = &info->var;
>  	struct s3c_fb_win *win = info->par;
>  	struct s3c_fb *sfb = win->parent;
> +	struct s3cfb_window_pos *winpos = &win->windata->winpos;
>  	void __iomem *regs = sfb->regs;
>  	void __iomem *buf = regs;
>  	int win_no = win->index;
> @@ -539,12 +540,13 @@ static int s3c_fb_set_par(struct fb_info *info)
> 
>  	/* write 'OSD' registers to control position of framebuffer */
> 
> -	data = VIDOSDxA_TOPLEFT_X(0) | VIDOSDxA_TOPLEFT_Y(0);
> +	data = VIDOSDxA_TOPLEFT_X(winpos->win_pos_x) |
> +	       VIDOSDxA_TOPLEFT_Y(winpos->win_pos_y);
>  	writel(data, regs + VIDOSD_A(win_no, sfb->variant));
> 
> -	data = VIDOSDxB_BOTRIGHT_X(s3c_fb_align_word(var->bits_per_pixel,
> -						     var->xres - 1)) |
> -	       VIDOSDxB_BOTRIGHT_Y(var->yres - 1);
> +	data = VIDOSDxB_BOTRIGHT_X((s3c_fb_align_word(var->bits_per_pixel,
> +	       (winpos->win_pos_x + var->xres - 1)))) |
> +	       VIDOSDxB_BOTRIGHT_Y((winpos->win_pos_y + var->yres - 1));
> 
>  	writel(data, regs + VIDOSD_B(win_no, sfb->variant));
> 
> @@ -999,8 +1001,10 @@ static int s3c_fb_ioctl(struct fb_info *info,
> unsigned int cmd,
>  {
>  	struct s3c_fb_win *win = info->par;
>  	struct s3c_fb *sfb = win->parent;
> -	int ret;
> +	struct s3cfb_window_pos *winpos = &win->windata->winpos;
> +	int ret = 0;
>  	u32 crtc;
> +	u32 data;
> 
>  	switch (cmd) {
>  	case FBIO_WAITFORVSYNC:
> @@ -1011,6 +1015,29 @@ static int s3c_fb_ioctl(struct fb_info *info,
> unsigned int cmd,
> 
>  		ret = s3c_fb_wait_for_vsync(sfb, crtc);
>  		break;
> +	case S3CFB_WIN_POSITION:
> +		if (copy_from_user(winpos, (u32 __user *)arg,
> +					sizeof(struct s3cfb_window_pos))) {
> +			ret = -EFAULT;
> +			break;
> +		}
> +
> +		shadow_protect_win(win, 1);
> +
> +		/* write 'OSD' registers to set position of the window */
> +		data = VIDOSDxA_TOPLEFT_X(winpos->win_pos_x) |
> +		       VIDOSDxA_TOPLEFT_Y(winpos->win_pos_y);
> +		writel(data, sfb->regs + VIDOSD_A(win->index, sfb->variant));
> +
> +		data = VIDOSDxB_BOTRIGHT_X(
> +				s3c_fb_align_word(info->var.bits_per_pixel,
> +				(winpos->win_pos_x + info->var.xres - 1)));
> +		data |=	VIDOSDxB_BOTRIGHT_Y(winpos->win_pos_y +
> +				info->var.yres - 1);
> +		writel(data, sfb->regs + VIDOSD_B(win->index, sfb->variant));
> +
> +		shadow_protect_win(win, 0);
> +		break;
Can you move this to separate function?
In my opinion, it would be better for readability and consistency.
Also, FBIO_WAITFORVSYNC has been implemented in this way.
		ret = s3c_fb_wait_for_vsync(sfb, crtc);
How about adding a function 's3c_fb_set_window_position' for this as belows?
+int s3c_fb_set_window_position(struct fb_info *info,
+				struct s3cfb_window_pos winpos)
		........
+	case S3CFB_WIN_POSITION:
+		if (copy_from_user(winpos, (u32 __user *)arg,
+					sizeof(struct s3cfb_window_pos))) {
+			ret = -EFAULT;
+			break;
+		}
+		ret = s3c_fb_set_window_position(info, winpos);
+		break;
>  	default:
>  		ret = -ENOTTY;
>  	}
> --
> 1.7.0.4


ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ýöÝzÿâžØ^n‡r¡ö¦zË\x1aëh™¨è­Ú&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~†­†Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm

Re: [PATCH 2/2] video: s3c-fb: Modify s3c-fb driver to support window positioning

[Ajay kumar]

Hi Jingoo,

2011/8/26 JinGoo Han <jg1.han@samsung.com>:
> Hi, Ajay.
>> -----Original Message-----
>> From: Ajay Kumar [mailto:ajaykumar.rs@samsung.com]
>> Sent: Friday, August 26, 2011 4:52 AM
>> To: linux-samsung-soc@vger.kernel.org; linux-fbdev@vger.kernel.org; linux-
>> arm-kernel@lists.infradead.org
>> Cc: FlorianSchandinat@gmx.de; lethal@linux-sh.org; jg1.han@samsung.com;
>> m.szyprowski@samsung.com; ben-linux@fluff.org; banajit.g@samsung.com
>> Subject: [PATCH 2/2] video: s3c-fb: Modify s3c-fb driver to support window
>> positioning
>>
>> This patch modifies the existing s3c-fb driver to provide
>> flexibility to the user to reposition the framebuffer windows.
>>
>> Signed-off-by: Ajay Kumar <ajaykumar.rs@samsung.com>
>> Signed-off-by: Banajit Goswami <banajit.g@samsung.com>
>> ---
>>  drivers/video/s3c-fb.c |   37 ++++++++++++++++++++++++++++++++-----
>>  1 files changed, 32 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/video/s3c-fb.c b/drivers/video/s3c-fb.c
>> index 0fda252..41179d7 100644
>> --- a/drivers/video/s3c-fb.c
>> +++ b/drivers/video/s3c-fb.c
>> @@ -442,6 +442,7 @@ static int s3c_fb_set_par(struct fb_info *info)
>>       struct fb_var_screeninfo *var = &info->var;
>>       struct s3c_fb_win *win = info->par;
>>       struct s3c_fb *sfb = win->parent;
>> +     struct s3cfb_window_pos *winpos = &win->windata->winpos;
>>       void __iomem *regs = sfb->regs;
>>       void __iomem *buf = regs;
>>       int win_no = win->index;
>> @@ -539,12 +540,13 @@ static int s3c_fb_set_par(struct fb_info *info)
>>
>>       /* write 'OSD' registers to control position of framebuffer */
>>
>> -     data = VIDOSDxA_TOPLEFT_X(0) | VIDOSDxA_TOPLEFT_Y(0);
>> +     data = VIDOSDxA_TOPLEFT_X(winpos->win_pos_x) |
>> +            VIDOSDxA_TOPLEFT_Y(winpos->win_pos_y);
>>       writel(data, regs + VIDOSD_A(win_no, sfb->variant));
>>
>> -     data = VIDOSDxB_BOTRIGHT_X(s3c_fb_align_word(var->bits_per_pixel,
>> -                                                  var->xres - 1)) |
>> -            VIDOSDxB_BOTRIGHT_Y(var->yres - 1);
>> +     data = VIDOSDxB_BOTRIGHT_X((s3c_fb_align_word(var->bits_per_pixel,
>> +            (winpos->win_pos_x + var->xres - 1)))) |
>> +            VIDOSDxB_BOTRIGHT_Y((winpos->win_pos_y + var->yres - 1));
>>
>>       writel(data, regs + VIDOSD_B(win_no, sfb->variant));
>>
>> @@ -999,8 +1001,10 @@ static int s3c_fb_ioctl(struct fb_info *info,
>> unsigned int cmd,
>>  {
>>       struct s3c_fb_win *win = info->par;
>>       struct s3c_fb *sfb = win->parent;
>> -     int ret;
>> +     struct s3cfb_window_pos *winpos = &win->windata->winpos;
>> +     int ret = 0;
>>       u32 crtc;
>> +     u32 data;
>>
>>       switch (cmd) {
>>       case FBIO_WAITFORVSYNC:
>> @@ -1011,6 +1015,29 @@ static int s3c_fb_ioctl(struct fb_info *info,
>> unsigned int cmd,
>>
>>               ret = s3c_fb_wait_for_vsync(sfb, crtc);
>>               break;
>> +     case S3CFB_WIN_POSITION:
>> +             if (copy_from_user(winpos, (u32 __user *)arg,
>> +                                     sizeof(struct s3cfb_window_pos))) {
>> +                     ret = -EFAULT;
>> +                     break;
>> +             }
>> +
>> +             shadow_protect_win(win, 1);
>> +
>> +             /* write 'OSD' registers to set position of the window */
>> +             data = VIDOSDxA_TOPLEFT_X(winpos->win_pos_x) |
>> +                    VIDOSDxA_TOPLEFT_Y(winpos->win_pos_y);
>> +             writel(data, sfb->regs + VIDOSD_A(win->index, sfb->variant));
>> +
>> +             data = VIDOSDxB_BOTRIGHT_X(
>> +                             s3c_fb_align_word(info->var.bits_per_pixel,
>> +                             (winpos->win_pos_x + info->var.xres - 1)));
>> +             data |= VIDOSDxB_BOTRIGHT_Y(winpos->win_pos_y +
>> +                             info->var.yres - 1);
>> +             writel(data, sfb->regs + VIDOSD_B(win->index, sfb->variant));
>> +
>> +             shadow_protect_win(win, 0);
>> +             break;
> Can you move this to separate function?
> In my opinion, it would be better for readability and consistency.
> Also, FBIO_WAITFORVSYNC has been implemented in this way.
>                ret = s3c_fb_wait_for_vsync(sfb, crtc);
> How about adding a function 's3c_fb_set_window_position' for this as belows?
> +int s3c_fb_set_window_position(struct fb_info *info,
> +                               struct s3cfb_window_pos winpos)

Thanks for your suggestion.
I have considered it, and now I have written a separate function for this.
I will be sending it in V2 version.
>                ........
> +       case S3CFB_WIN_POSITION:
> +               if (copy_from_user(winpos, (u32 __user *)arg,
> +                                       sizeof(struct s3cfb_window_pos))) {
> +                       ret = -EFAULT;
> +                       break;
> +               }
> +               ret = s3c_fb_set_window_position(info, winpos);
> +               break;
>>       default:
>>               ret = -ENOTTY;
>>       }
>> --
>> 1.7.0.4
>
>
>

Re: [PATCH 0/2] video: s3c-fb: Add window positioning support

[Florian Tobias Schandinat]

Hi all,

On 08/25/2011 07:51 PM, Ajay Kumar wrote:
> Just as a note, there are many drivers like mx3fb.c, au1200fb.c and OMAP
> seem to be doing window/plane positioning in their driver code.
> Is it possible to have this window positioning support at a common place?

Good point. Congratulations for figuring out that I like to standardize things.
But I think your suggestion is far from being enough to be useful for userspace
(which is our goal so that applications can be reused along drivers and don't
need to know about individual drivers).

So let me at first summarize how I understand you implemented those things after
having a brief look at some of the drivers:
Windows are rectangular screen areas whose pixel data come from other locations.
The other locations are accessible via other framebuffer devices (e.g. fb1). So
in this area the data of fb1 is shown and not the data of fb0 that would be
normally shown.

So in addition to your proposed positioning I think we should also have the
following to give userspace a useful set of functionality:

- a way to discover how the screen is composited (how many windows are there,
how they are stacked and how to access those)

- a way to enable/disable windows (make them (in)visible)

- reporting and selecting how the window content can be mixed with the root
screen (overwrite, source or destination color keying)

- things like window size and color format could be handled by the usual fb API
used on the window. However there might be restrictions which cause them to be
not 100% API compatible (for example when changing the color format if the
windows are required to have the same format as the root screen)

- do we need to worry about hardware (up/down) scaling of the window content?


So is that what you need for a standardized window implementation?
Any additional things that were useful/needed in this context?
Would you consider adding support for this API in your drivers? (as
standardizing wouldn't be useful if nobody would implement it)


Best regards,

Florian Tobias Schandinat


> 
> For instance, we can have a common struture and ioctl number in 
> include/linux/fb.h as below:
> 
> 	#define FBIOPOS_OVERLAY_WIN    _IOW('F', 0x21, struct fb_overlay_win_pos)
> 
> 	struct fb_overlay_win_pos {
> 		__u32 win_pos_x;  /* x-offset of window from LCD(0,0) */
> 		__u32 win_pos_y;  /* y-offset of window from LCD(0,0) */
> 	};
> 
> where LCD(0,0) means the first pixel of the LCD screen.
> Individual drivers can have implementation for this ioctl.
> 
> To Kukjin Kim,
>   [PATCH 1/2] ARM: SAMSUNG: Add Window Positioning Support for s3c-fb driver
> 
> To Paul Mundt, Florian Tobias Schandinat
>   [PATCH 2/2] video: s3c-fb: Modify s3c-fb driver to support window positioning
> 
>  arch/arm/plat-samsung/include/plat/fb.h |   14 +++++++++++
>  drivers/video/s3c-fb.c                  |   37 ++++++++++++++++++++++++++----
>  2 files changed, 46 insertions(+), 5 deletions(-)
> 
> 

Re: [PATCH 0/2] video: s3c-fb: Add window positioning support

[Tomi Valkeinen]

Hi,

On Thu, 2011-09-01 at 16:45 +0000, Florian Tobias Schandinat wrote:
> Hi all,
> 
> On 08/25/2011 07:51 PM, Ajay Kumar wrote:
> > Just as a note, there are many drivers like mx3fb.c, au1200fb.c and OMAP
> > seem to be doing window/plane positioning in their driver code.
> > Is it possible to have this window positioning support at a common place?
> 
> Good point. Congratulations for figuring out that I like to standardize things.
> But I think your suggestion is far from being enough to be useful for userspace
> (which is our goal so that applications can be reused along drivers and don't
> need to know about individual drivers).
> 
> So let me at first summarize how I understand you implemented those things after
> having a brief look at some of the drivers:
> Windows are rectangular screen areas whose pixel data come from other locations.
> The other locations are accessible via other framebuffer devices (e.g. fb1). So
> in this area the data of fb1 is shown and not the data of fb0 that would be
> normally shown.

Here's what we have on OMAP:

We have a bunch of hardware overlays, each of which can go to one
output. When using fbdev, the pixel data for overlays comes from the
framebuffers. One fb can be used as a  pixel source for multiple
overlays.

So, for example, the "connections" can be like these:

           Initial configuration

 .-----.         .------.           .------.
 | fb0 |-------->| ovl0 |-----.---->| LCD  |
 '-----'         '------'     |     '------'
 .-----.         .------.     |
 | fb1 |-------->| ovl1 |-----|
 '-----'         '------'     |
 .-----.         .------.     |     .------.
 | fb2 |-------->| ovl2 |-----'     |  TV  |
 '-----'         '------'           '------'


      Video on fb1, shown on LCD and TV

 .-----.         .------.           .------.
 | fb0 |-------->| ovl0 |-----.---->| LCD  |
 '-----'         '------'     |     '------'
 .-----.         .------.     |
 | fb1 |----.--->| ovl1 |-----'
 '-----'    |    '------'
 .-----.    |    .------.           .------.
 | fb2 |    '--->| ovl2 |---------->|  TV  |
 '-----'         '------'           '------'


And how the actual image is composited on the display, we have first the
fb side (no news there):

 .---------------------fb-vxres---------------------.
 |                                                  |
 |                                                  |
 |   (xoffset,yoffset)                              |
 |           .-------fb-xres------------.           |
 f           |                          |           |
 b           f                          |           |
 |           b                          |           |
 v           |                          |           |
 y           y                          |           |
 r           r                          |           |
 e           e                          |           |
 s           s                          |           |
 |           |                          |           |
 |           '--------------------------'           |
 |                                                  |
 |                                                  |
 '--------------------------------------------------'

The area marked by x/yoffset and x/yres is used as the pixel source for
the overlay.

On the display we have something like this:

 .----------------disp-xres----------------.
 |                                         |
 | (xpos,ypos)                             |
 |      .-----ovl-xres-----.               |
 d      |                  |               |
 i      o                  |               |
 s      v                  |               |
 p      l                  |               |
 |      |                  |               |
 y      y                  |               |
 r      r                  |               |
 e      e                  |               |
 s      s                  |               |
 |      |                  |               |
 |      '------------------'               |
 |                                         |
 '-----------------------------------------'

The x/ypos of the overlay does not have any relation to the x/yoffset of
the framebuffer. The overlay's x/yres is the same as the fb's x/yres in
case the overlay doesn't support scaling (OMAP's case this is true for
overlay0). Otherwise the overlay's x/yres is only limited by the HW's
scaling capabilities.

The overlays have a priority order, and where there's no overlay, a
background color is used. So like this:

 .-----------------------------------------.
 |            background color             |
 | .-----------------------.               |
 | |         ovl0          |               |
 | |                       |               |
 | | .--------------.      |               |
 | | |     ovl1     |      |               |
 | | |              |      |               |
 | | |        .-------------------.        |
 | | |        |       ovl2        |        |
 | | '--------|                   |        |
 | '----------|                   |        |
 |            |                   |        |
 |            |                   |        |
 |            |                   |        |
 |            '-------------------'        |
 '-----------------------------------------'

As you may have noticed, this raises in annoying problem: the
framebuffer does not represent the display, and thus the display related
information in the fb, like video timings, are difficult or impossible
to use. I haven't found any solution to this.

> So in addition to your proposed positioning I think we should also have the
> following to give userspace a useful set of functionality:
> 
> - a way to discover how the screen is composited (how many windows are there,
> how they are stacked and how to access those)
> 
> - a way to enable/disable windows (make them (in)visible)
> 
> - reporting and selecting how the window content can be mixed with the root
> screen (overwrite, source or destination color keying)

Right, and there may not be a root screen at all, like on OMAP. (if the
background color doesn't count as a root screen).

> - things like window size and color format could be handled by the usual fb API
> used on the window. However there might be restrictions which cause them to be
> not 100% API compatible (for example when changing the color format if the
> windows are required to have the same format as the root screen)
> 
> - do we need to worry about hardware (up/down) scaling of the window content?
> 
> 
> So is that what you need for a standardized window implementation?
> Any additional things that were useful/needed in this context?
> Would you consider adding support for this API in your drivers? (as
> standardizing wouldn't be useful if nobody would implement it)

We already have the current implementation in omapfb, so while a
standard API would be nice, we still need to keep the old custom API for
compatibility.

Another thing is that I'm not sure how much features we should try to
add to fbdev. The fbdev model is a bit limited (for example, the timings
problem above, or double/triple buffering), and at least we are
currently looking at using DRM. There will still be a framebuffer, but
that's mostly for fbconsole and such, and doesn't need to support
overlays, scaling, and other more complex features.

 Tomi

Re: [PATCH 0/2] video: s3c-fb: Add window positioning support

[Ajay kumar]

Hi Florian,

On Thu, Sep 1, 2011 at 10:15 PM, Florian Tobias Schandinat
<FlorianSchandinat@gmx.de> wrote:
> Hi all,
>
> On 08/25/2011 07:51 PM, Ajay Kumar wrote:
>> Just as a note, there are many drivers like mx3fb.c, au1200fb.c and OMAP
>> seem to be doing window/plane positioning in their driver code.
>> Is it possible to have this window positioning support at a common place?
>
> Good point. Congratulations for figuring out that I like to standardize things.
> But I think your suggestion is far from being enough to be useful for userspace
> (which is our goal so that applications can be reused along drivers and don't
> need to know about individual drivers).

Thanks for considering the scenerio. Yes, you are right and we need to
start somewhere.

> So let me at first summarize how I understand you implemented those things after
> having a brief look at some of the drivers:
> Windows are rectangular screen areas whose pixel data come from other locations.
> The other locations are accessible via other framebuffer devices (e.g. fb1). So
> in this area the data of fb1 is shown and not the data of fb0 that would be
> normally shown.

Yes, thats right! The area not covered by the other framebuffer
device/window (e.g. fb1) will be transparent and the framebuffer
device/window below (e.g. fb0) will be visible.

> So in addition to your proposed positioning I think we should also have the
> following to give userspace a useful set of functionality:
>
> - a way to discover how the screen is composited (how many windows are there,
> how they are stacked and how to access those)

This will need more discussions, as the amount of information to be
exposed to the user-space might vary from controller-to-controller.

> - a way to enable/disable windows (make them (in)visible)

This is already present. Switching on/off any particular framebuffer
device/window can be done by calling ioctl with FBIOBLANK, from user
space.

> - reporting and selecting how the window content can be mixed with the root
> screen (overwrite, source or destination color keying)

Yes, thats a good point. This can be achieved by implementing ioctl call

> - things like window size and color format could be handled by the usual fb API
> used on the window. However there might be restrictions which cause them to be
> not 100% API compatible (for example when changing the color format if the
> windows are required to have the same format as the root screen)

This seems to be more to do with the property of the FB controller.
Moreover, the driver can always enforce checks on such hardware
restrictions.

> - do we need to worry about hardware (up/down) scaling of the window content?

The user application should take care of these functions, as this is
directly related to the window size. (Usually these operations are
handled by some separate hardware modules (post-processors) )

>
> So is that what you need for a standardized window implementation?
> Any additional things that were useful/needed in this context?
> Would you consider adding support for this API in your drivers? (as
> standardizing wouldn't be useful if nobody would implement it)

Yes, having these missing features will definitely give the user much
more control over the various available framebuffer devices/windows.
We would like to start with the fb window re-positioning feature and
add more features in coming days.

>
> Best regards,
>
> Florian Tobias Schandinat
>

Thanks for your review and suggestions. It was very helpful.

Best Regards,
Ajay Kumar

Re: [PATCH 0/2] video: s3c-fb: Add window positioning support

[Laurent Pinchart]

Hi Florian,

On Thursday 01 September 2011 18:45:18 Florian Tobias Schandinat wrote:
> Hi all,
> 
> On 08/25/2011 07:51 PM, Ajay Kumar wrote:
> > Just as a note, there are many drivers like mx3fb.c, au1200fb.c and OMAP
> > seem to be doing window/plane positioning in their driver code.
> > Is it possible to have this window positioning support at a common place?
> 
> Good point. Congratulations for figuring out that I like to standardize
> things. But I think your suggestion is far from being enough to be useful
> for userspace (which is our goal so that applications can be reused along
> drivers and don't need to know about individual drivers).

Beside standardizing things, do you also like to take them one level higher to 
solve challenging issues ? I know the answer must be yes :-)

The problem at hand here is something we have solved in V4L2 (theoretically 
only for part of it) with the media controller API, the V4L2 subdevs and their 
pad-level format API.

In a nutshell, the media controller lets drivers model hardware as a graph of 
buliding blocks connected through their pads and expose that description to 
userspace applications. In V4L2 most of those blocks are V4L2 subdevs, which 
are abstract building blocks that implement sets of standard operations. Those 
operations are exposed to userspace through the V4L2 subdevs pad-level format 
API, allowing application to configure sizes and selection rectangles at all 
pads in the graph. Selection rectangles can be used to configure cropping and 
composing, which is exactly what the window positioning API needs to do.

Instead of creating a new fbdev-specific API to do the same, shouldn't we try 
to join forces ?

> So let me at first summarize how I understand you implemented those things
> after having a brief look at some of the drivers:
> Windows are rectangular screen areas whose pixel data come from other
> locations. The other locations are accessible via other framebuffer
> devices (e.g. fb1). So in this area the data of fb1 is shown and not the
> data of fb0 that would be normally shown.
> 
> So in addition to your proposed positioning I think we should also have the
> following to give userspace a useful set of functionality:
> 
> - a way to discover how the screen is composited (how many windows are
> there, how they are stacked and how to access those)
> 
> - a way to enable/disable windows (make them (in)visible)
> 
> - reporting and selecting how the window content can be mixed with the root
> screen (overwrite, source or destination color keying)
> 
> - things like window size and color format could be handled by the usual fb
> API used on the window. However there might be restrictions which cause
> them to be not 100% API compatible (for example when changing the color
> format if the windows are required to have the same format as the root
> screen)
> 
> - do we need to worry about hardware (up/down) scaling of the window
> content?
> 
> 
> So is that what you need for a standardized window implementation?
> Any additional things that were useful/needed in this context?
> Would you consider adding support for this API in your drivers? (as
> standardizing wouldn't be useful if nobody would implement it)

-- 
Regards,

Laurent Pinchart

Re: [PATCH 0/2] video: s3c-fb: Add window positioning support

[Florian Tobias Schandinat]

Hi Laurent,

On 09/07/2011 03:31 PM, Laurent Pinchart wrote:
> Hi Florian,
> 
> On Thursday 01 September 2011 18:45:18 Florian Tobias Schandinat wrote:
>> Hi all,
>>
>> On 08/25/2011 07:51 PM, Ajay Kumar wrote:
>>> Just as a note, there are many drivers like mx3fb.c, au1200fb.c and OMAP
>>> seem to be doing window/plane positioning in their driver code.
>>> Is it possible to have this window positioning support at a common place?
>>
>> Good point. Congratulations for figuring out that I like to standardize
>> things. But I think your suggestion is far from being enough to be useful
>> for userspace (which is our goal so that applications can be reused along
>> drivers and don't need to know about individual drivers).
> 
> Beside standardizing things, do you also like to take them one level higher to 
> solve challenging issues ? I know the answer must be yes :-)
> 
> The problem at hand here is something we have solved in V4L2 (theoretically 
> only for part of it) with the media controller API, the V4L2 subdevs and their 
> pad-level format API.
> 
> In a nutshell, the media controller lets drivers model hardware as a graph of 
> buliding blocks connected through their pads and expose that description to 
> userspace applications. In V4L2 most of those blocks are V4L2 subdevs, which 
> are abstract building blocks that implement sets of standard operations. Those 
> operations are exposed to userspace through the V4L2 subdevs pad-level format 
> API, allowing application to configure sizes and selection rectangles at all 
> pads in the graph. Selection rectangles can be used to configure cropping and 
> composing, which is exactly what the window positioning API needs to do.
> 
> Instead of creating a new fbdev-specific API to do the same, shouldn't we try 
> to join forces ?

Okay, thanks for the pointer. After having a look at your API I understand that
it would solve the problem to discover how many windows (in this case) are there
and how they can be accessed. It looks fine for this purpose, powerful enough
and not too complex. So if I get it correct we still need at least a way to
configure the position of the windows/overlays/sink pads similar to what Ajay
proposed. Additionally a way to get and/or set the z-position of the overlays if
multiple overlays overlap and set/get how the overlays work (overdraw, constant
alpha, source/destination color keying). Normally I'd consider these link
properties but I think implementing them as properties of the source framebuffer
or sink pad would work as well.
Is this correct or did I miss something?


Best regards,

Florian Tobias Schandinat

Re: [PATCH 0/2] video: s3c-fb: Add window positioning support

[Laurent Pinchart]

Hi Florian,

On Sunday 18 September 2011 21:29:57 Florian Tobias Schandinat wrote:
> On 09/07/2011 03:31 PM, Laurent Pinchart wrote:
> > On Thursday 01 September 2011 18:45:18 Florian Tobias Schandinat wrote:
> >> On 08/25/2011 07:51 PM, Ajay Kumar wrote:
> >>> Just as a note, there are many drivers like mx3fb.c, au1200fb.c and
> >>> OMAP seem to be doing window/plane positioning in their driver code.
> >>> Is it possible to have this window positioning support at a common
> >>> place?
> >> 
> >> Good point. Congratulations for figuring out that I like to standardize
> >> things. But I think your suggestion is far from being enough to be
> >> useful for userspace (which is our goal so that applications can be
> >> reused along drivers and don't need to know about individual drivers).
> > 
> > Beside standardizing things, do you also like to take them one level
> > higher to solve challenging issues ? I know the answer must be yes :-)
> > 
> > The problem at hand here is something we have solved in V4L2
> > (theoretically only for part of it) with the media controller API, the
> > V4L2 subdevs and their pad-level format API.
> > 
> > In a nutshell, the media controller lets drivers model hardware as a
> > graph of buliding blocks connected through their pads and expose that
> > description to userspace applications. In V4L2 most of those blocks are
> > V4L2 subdevs, which are abstract building blocks that implement sets of
> > standard operations. Those operations are exposed to userspace through
> > the V4L2 subdevs pad-level format API, allowing application to configure
> > sizes and selection rectangles at all pads in the graph. Selection
> > rectangles can be used to configure cropping and composing, which is
> > exactly what the window positioning API needs to do.
> > 
> > Instead of creating a new fbdev-specific API to do the same, shouldn't we
> > try to join forces ?
> 
> Okay, thanks for the pointer. After having a look at your API I understand
> that it would solve the problem to discover how many windows (in this
> case) are there and how they can be accessed. It looks fine for this
> purpose, powerful enough and not too complex. So if I get it correct we
> still need at least a way to configure the position of the
> windows/overlays/sink pads similar to what Ajay proposed.

Yes, the media controller API can only expose the topology to userspace, it 
can't be used to configure FB-specific parameters on the pipeline.

> Additionally a way to get and/or set the z-position of the overlays if
> multiple overlays overlap and set/get how the overlays work (overdraw,
> constant alpha, source/destination color keying). Normally I'd consider
> these link properties but I think implementing them as properties of the
> source framebuffer or sink pad would work as well.
> Is this correct or did I miss something?

That's correct.

What bothers me is that both V4L2 and DRM/KMS have the exact same needs. I 
don't think it makes sense to implement three different solutions to the same 
problem in our three video-related APIs. What's your opinion about that ?

I've tried to raise the issue on the dri-devel mailing list ("Proposal for a 
low-level Linux display framework"), but there's still a long way to go before 
convincing everybody. Feel free to help me :-)

-- 
Regards,

Laurent Pinchart