[PATCH 12/13] drm/i2c/tda998x: Fix signed overflow issue

[PATCH 12/13] drm/i2c/tda998x: Fix signed overflow issue

[Daniel Vetter]

This is C standard hair-splitting, but afaict
- sum will be promoted to signed int in computation since
  uint8_t fits
- signed overflow is undefined.

No we need to add up an awful lot of bytes to actually make it
overflow. But I guess the real risk is gcc spotting this and going
bananas. Fix this by simply using unsigned in to force all computations
to use the well-defined unsigned behaviour.

Spotted by coverity.

Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Jean-Francois Moine <moinejf@free.fr>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---
 drivers/gpu/drm/i2c/tda998x_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i2c/tda998x_drv.c b/drivers/gpu/drm/i2c/tda998x_drv.c
index 48af5cac1902..ae2754760d77 100644
--- a/drivers/gpu/drm/i2c/tda998x_drv.c
+++ b/drivers/gpu/drm/i2c/tda998x_drv.c
@@ -568,7 +568,7 @@ static irqreturn_t tda998x_irq_thread(int irq, void *data)
 
 static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
 {
-	uint8_t sum = 0;
+	unsigned sum = 0;
 
 	while (bytes--)
 		sum += *buf++;
-- 
1.8.5.2

Re: [PATCH 12/13] drm/i2c/tda998x: Fix signed overflow issue

[Jean-Francois Moine]

On Sat Apr 5 02:45:01 PDT 2014,
Daniel Vetter <daniel.vetter@ffwll.ch> wrote:

> This is C standard hair-splitting, but afaict
> - sum will be promoted to signed int in computation since
>   uint8_t fits
> - signed overflow is undefined.
	[snip]
>  drivers/gpu/drm/i2c/tda998x_drv.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i2c/tda998x_drv.c b/drivers/gpu/drm/i2c/tda998x_drv.c
> index 48af5cac1902..ae2754760d77 100644
> --- a/drivers/gpu/drm/i2c/tda998x_drv.c
> +++ b/drivers/gpu/drm/i2c/tda998x_drv.c
> @@ -568,7 +568,7 @@ static irqreturn_t tda998x_irq_thread(int irq, void *data)
>  
>  static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
>  {
> -	uint8_t sum = 0;
> +	unsigned sum = 0;
>  
>  	while (bytes--)
>  		sum += *buf++;

This function may be simplified by:

--- tda998x_drv.c~
+++ tda998x_drv.c
@@ -568,11 +568,11 @@
 
 static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
 {
-	uint8_t sum = 0;
+	int sum = 0;
 
 	while (bytes--)
-		sum += *buf++;
-	return (255 - sum) + 1;
+		sum -= *buf++;
+	return sum;
 }
 
 #define HB(x) (x)


and the same may be done in hdmi.c:

diff --git a/drivers/video/hdmi.c b/drivers/video/hdmi.c
index 9e758a8..b6c9030 100644
--- a/drivers/video/hdmi.c
+++ b/drivers/video/hdmi.c
@@ -31,14 +31,14 @@
 static void hdmi_infoframe_checksum(void *buffer, size_t size)
 {
 	u8 *ptr = buffer;
-	u8 csum = 0;
+	int csum = 0;
 	size_t i;
 
 	/* compute checksum */
 	for (i = 0; i < size; i++)
-		csum += ptr[i];
+		csum -= ptr[i];
 
-	ptr[3] = 256 - csum;
+	ptr[3] = csum;
 }
 
 /**


-- 
Ken ar c'hentañ	|	      ** Breizh ha Linux atav! **
Jef		|		http://moinejf.free.fr/
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH] drm/i2c/tda998x: Fix signed overflow issue

[Daniel Vetter]

This is C standard hair-splitting, but afaict
- sum will be promoted to signed int in computation since
  uint8_t fits
- signed overflow is undefined.

No we need to add up an awful lot of bytes to actually make it
overflow. But I guess the real risk is gcc spotting this and going
bananas. Fix this by simply using unsigned in to force all computations
to use the well-defined unsigned behaviour.

Spotted by coverity.

v2: Simplify the entire computation as suggested by Jean.

Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Jean-Francois Moine <moinejf@free.fr>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---
 drivers/gpu/drm/i2c/tda998x_drv.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/i2c/tda998x_drv.c b/drivers/gpu/drm/i2c/tda998x_drv.c
index 48af5cac1902..240c331405b9 100644
--- a/drivers/gpu/drm/i2c/tda998x_drv.c
+++ b/drivers/gpu/drm/i2c/tda998x_drv.c
@@ -568,11 +568,11 @@ static irqreturn_t tda998x_irq_thread(int irq, void *data)
 
 static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
 {
-	uint8_t sum = 0;
+	int sum = 0;
 
 	while (bytes--)
-		sum += *buf++;
-	return (255 - sum) + 1;
+		sum -= *buf++;
+	return sum;
 }
 
 #define HB(x) (x)
-- 
1.8.5.2

Re: [PATCH 12/13] drm/i2c/tda998x: Fix signed overflow issue

[Ian Romanick]

On 04/05/2014 02:45 AM, Daniel Vetter wrote:
> This is C standard hair-splitting, but afaict
> - sum will be promoted to signed int in computation since
>   uint8_t fits
> - signed overflow is undefined.
> 
> No we need to add up an awful lot of bytes to actually make it
  ^^
Now

> overflow. But I guess the real risk is gcc spotting this and going
> bananas. Fix this by simply using unsigned in to force all computations
> to use the well-defined unsigned behaviour.

Seems reasonable... it also seems impossible (ha!) to break anything.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>

> Spotted by coverity.
> 
> Cc: Russell King <rmk+kernel@arm.linux.org.uk>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Jean-Francois Moine <moinejf@free.fr>
> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
>  drivers/gpu/drm/i2c/tda998x_drv.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i2c/tda998x_drv.c b/drivers/gpu/drm/i2c/tda998x_drv.c
> index 48af5cac1902..ae2754760d77 100644
> --- a/drivers/gpu/drm/i2c/tda998x_drv.c
> +++ b/drivers/gpu/drm/i2c/tda998x_drv.c
> @@ -568,7 +568,7 @@ static irqreturn_t tda998x_irq_thread(int irq, void *data)
>  
>  static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
>  {
> -	uint8_t sum = 0;
> +	unsigned sum = 0;
>  
>  	while (bytes--)
>  		sum += *buf++;
>