Re: fbcmap: non working tests on unsigned cmap->start

["Ville Syrjälä" <syrjala@sci.fi>]

On Sun, Apr 26, 2009 at 02:20:47PM +0200, Roel Kluin wrote:
> cmap->start is unsigned, A negative start could result in incorrect
> colors. `start' is used as the starting index for the hardware palette,
> 'start+len-1' is the last index.
> 
> Signed-off-by: Roel Kluin <roel.kluin@gmail.com>
> ---
> >>> vi include/linux/fb.h +478
> >>>
> >>> Note that struct fb_cmap_user cmap->start in fb_set_user_cmap() is unsigned.
> >>> Should there maybe be a test:
> >>>
> >>> if (cmap->start > MAX || ...)
> >>>
> >>> (and what should MAX be then?)
> >>>
> 
> > On Thu, Apr 23, 2009 at 01:41:10PM -0700, Andrew Morton wrote:
> >> argh.
> >>
> >> - Perhaps userspace can kill the kernel by sending a "negative"
> >>   `start'.  Removing the test will make it even less likely that we'll
> >>   fix this bug.
> > 
> > Shouldn't happen. 'start' is used as the starting index for the hardware
> > palette, 'start+len-1' is the last index. All drivers should already check
> > the passed values since the maximum index depends on the display mode.
> > And I suppose the worst thing that could happen if the driver fails to
> > check the values would be incorrect colors.
> 
> Thanks for your explanation, I think this should fix it?
> 
> diff --git a/drivers/video/fbcmap.c b/drivers/video/fbcmap.c
> index f53b9f1..ea62d87 100644
> --- a/drivers/video/fbcmap.c
> +++ b/drivers/video/fbcmap.c
> @@ -266,7 +266,7 @@ int fb_set_user_cmap(struct fb_cmap_user *cmap, struct fb_info *info)
>  		rc = -ENODEV;
>  		goto out;
>  	}
> -	if (cmap->start < 0 || (!info->fbops->fb_setcolreg &&
> +	if (cmap->start >= cmap->len || (!info->fbops->fb_setcolreg &&
>  				!info->fbops->fb_setcmap)) {
>  		rc = -EINVAL;
>  		goto out1;

That's not correct. There's nothing wrong with having 'start' >= 'len'.

You would rather need something like
'if (start+len > 1 << max(red.len, green.len, blue.len, transp.len))'
and a check to make sure that start+len doesn't overflow.

Oh and I guess it should also check that the visual is pseudocolor or
directcolor.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

------------------------------------------------------------------------------
Crystal Reports &#45; New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty&#45;free distribution of the report engine for externally facing 
server and web deployment.
http://p.sf.net/sfu/businessobjects