From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37719) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WrTJ0-00019B-Hv for qemu-devel@nongnu.org; Mon, 02 Jun 2014 10:33:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WrTIv-0007Pg-IC for qemu-devel@nongnu.org; Mon, 02 Jun 2014 10:33:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60572) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WrTIv-0007PU-4p for qemu-devel@nongnu.org; Mon, 02 Jun 2014 10:33:37 -0400 From: Gerd Hoffmann Date: Mon, 2 Jun 2014 16:33:12 +0200 Message-Id: <1401719592-26362-4-git-send-email-kraxel@redhat.com> In-Reply-To: <1401719592-26362-1-git-send-email-kraxel@redhat.com> References: <1401719592-26362-1-git-send-email-kraxel@redhat.com> Subject: [Qemu-devel] [PULL 3/3] vnc-enc-tight: Fix divide-by-zero in tight_detect_smooth_image{16, 24, 32} List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Gonglei , Gerd Hoffmann , Anthony Liguori From: Gonglei Spotted by Coverity: (1) Event assignment: Assigning: "pixels" = "0". (2) Event cond_true: Condition "y < h", taking true branch (3) Event cond_false: Condition "x < w", taking false branch (4) Event loop_end: Reached end of loop (5) Event divide_by_zero: In expression "(stats[0] + stats[1]) * 100U / pixels", division by expression "pixels" which may be zero has undefined behavior. 290 DEFINE_DETECT_FUNCTION(16) 291 DEFINE_DETECT_FUNCTION(32) Signed-off-by: Gonglei Signed-off-by: Gerd Hoffmann --- ui/vnc-enc-tight.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/ui/vnc-enc-tight.c b/ui/vnc-enc-tight.c index 59b59c0..f02352c 100644 --- a/ui/vnc-enc-tight.c +++ b/ui/vnc-enc-tight.c @@ -181,6 +181,10 @@ tight_detect_smooth_image24(VncState *vs, int w, int h) } } + if (pixels == 0) { + return 0; + } + /* 95% smooth or more ... */ if (stats[0] * 33 / pixels >= 95) { return 0; @@ -267,7 +271,9 @@ tight_detect_smooth_image24(VncState *vs, int w, int h) y += w; \ } \ } \ - \ + if (pixels == 0) { \ + return 0; \ + } \ if ((stats[0] + stats[1]) * 100 / pixels >= 90) { \ return 0; \ } \ -- 1.8.3.1