[Qemu-devel] [PATCH] fix Coverity scan SIGN_EXTENSION error

[Qemu-devel] [PATCH] fix Coverity scan SIGN_EXTENSION error

[Gerd Hoffmann]

Cc: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/display/qxl-render.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/display/qxl-render.c b/hw/display/qxl-render.c
index f511a62..269b1a7 100644
--- a/hw/display/qxl-render.c
+++ b/hw/display/qxl-render.c
@@ -199,7 +199,7 @@ static QEMUCursor *qxl_cursor(PCIQXLDevice *qxl, QXLCursor *cursor)
     c->hot_y = cursor->header.hot_spot_y;
     switch (cursor->header.type) {
     case SPICE_CURSOR_TYPE_ALPHA:
-        size = cursor->header.width * cursor->header.height * sizeof(uint32_t);
+        size = sizeof(uint32_t) * cursor->header.width * cursor->header.height;
         memcpy(c->data, cursor->chunk.data, size);
         if (qxl->debug > 2) {
             cursor_print_ascii_art(c, "qxl/alpha");
-- 
1.7.9.7

Re: [Qemu-devel] [PATCH] fix Coverity scan SIGN_EXTENSION error

[Laszlo Ersek]

On 06/06/13 17:12, Gerd Hoffmann wrote:
> Cc: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  hw/display/qxl-render.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/display/qxl-render.c b/hw/display/qxl-render.c
> index f511a62..269b1a7 100644
> --- a/hw/display/qxl-render.c
> +++ b/hw/display/qxl-render.c
> @@ -199,7 +199,7 @@ static QEMUCursor *qxl_cursor(PCIQXLDevice *qxl, QXLCursor *cursor)
>      c->hot_y = cursor->header.hot_spot_y;
>      switch (cursor->header.type) {
>      case SPICE_CURSOR_TYPE_ALPHA:
> -        size = cursor->header.width * cursor->header.height * sizeof(uint32_t);
> +        size = sizeof(uint32_t) * cursor->header.width * cursor->header.height;
>          memcpy(c->data, cursor->chunk.data, size);
>          if (qxl->debug > 2) {
>              cursor_print_ascii_art(c, "qxl/alpha");
> 

Right. "width" and "height" are both "uint16_t" (short unsigned) and
they are promoted to "int". The binding is then

  (int * int) * size_t
  int         * size_t
  size_t      * size_t
  size_t

By the time the product would be converted to size_t ("unsigned" or
"unsigned long" in practice, dependent on ILP32 or LP64), it may have
overflowed (undefined behavior for signed int).

The above changes it to

  (size_t * int)    * int
  (size_t * size_t) * int
  size_t            * int
  size_t            * size_t
  size_t

(BTW this is not order of evaluation, just how the types are determined
by operator binding.)

Reviewed-by: Laszlo Ersek <lersek@redhat.com>

Re: [Qemu-devel] [PATCH] fix Coverity scan SIGN_EXTENSION error

[Andreas Färber]

Am 06.06.2013 17:12, schrieb Gerd Hoffmann:
> Cc: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

Please indicate in the subject that this is about qxl when queuing. :)

Cheers,
Andreas

> ---
>  hw/display/qxl-render.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/display/qxl-render.c b/hw/display/qxl-render.c
> index f511a62..269b1a7 100644
> --- a/hw/display/qxl-render.c
> +++ b/hw/display/qxl-render.c
> @@ -199,7 +199,7 @@ static QEMUCursor *qxl_cursor(PCIQXLDevice *qxl, QXLCursor *cursor)
>      c->hot_y = cursor->header.hot_spot_y;
>      switch (cursor->header.type) {
>      case SPICE_CURSOR_TYPE_ALPHA:
> -        size = cursor->header.width * cursor->header.height * sizeof(uint32_t);
> +        size = sizeof(uint32_t) * cursor->header.width * cursor->header.height;
>          memcpy(c->data, cursor->chunk.data, size);
>          if (qxl->debug > 2) {
>              cursor_print_ascii_art(c, "qxl/alpha");

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg