qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH to consider for 0.12] vmware_vga: Don't crash on too-big DEFINE_CURSOR command
@ 2009-12-17 22:27 Roland Dreier
  2009-12-17 22:34 ` [Qemu-devel] " Anthony Liguori
  0 siblings, 1 reply; 9+ messages in thread
From: Roland Dreier @ 2009-12-17 22:27 UTC (permalink / raw)
  To: anthony; +Cc: qemu-devel

Hi Anthony -- just sent this patch to qemu-devel (although I don't see
it in archives yet).  Anyway I realize it is really really late given
your release timeframe but I think the risk of this pretty minimal, and
the patch fixes a crash in a pretty reasonable config (running a modern
Linux distro with the fastest guest video adapter).  So please consider
this for 0.12.

Another possibility would be to just take the part of the patch that
bumps the array size in the structure, since that seems to have
essentially 0 risk and fixes the crash in the case I've seen.

Thanks,
  Roland

===

QEMU crashes with vmware_vga when running a Linux guest with the latest
X.org vmware video driver if QEMU is using SDL for video output.  In
this case, vmware_vga advertises cursor acceleration to the guest, and
the crash comes when the guest does a DEFINE_CURSOR command with a 64x64
32bpp cursor.  This request overruns the image[] array in struct
vmsvga_cursor_definition_s and QEMU ends up segfaulting because of
memory corruption caused by writing past the end of the array.

Fix this by enlarging the image[] array to be able to hold 4096 32-bit
pixels so we don't fail for the case of 64*64*32bpp, and also add error
checking to avoid a crash if an even bigger request is sent by a guest.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
---
 hw/vmware_vga.c |   11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/hw/vmware_vga.c b/hw/vmware_vga.c
index f3e3749..d253a2e 100644
--- a/hw/vmware_vga.c
+++ b/hw/vmware_vga.c
@@ -462,7 +462,7 @@ struct vmsvga_cursor_definition_s {
     int hot_x;
     int hot_y;
     uint32_t mask[1024];
-    uint32_t image[1024];
+    uint32_t image[4096];	/* allow for 64x64 32bpp cursor */
 };
 
 #define SVGA_BITMAP_SIZE(w, h)		((((w) + 31) >> 5) * (h))
@@ -557,6 +557,15 @@ static void vmsvga_fifo_run(struct vmsvga_state_s *s)
             cursor.height = y = vmsvga_fifo_read(s);
             vmsvga_fifo_read(s);
             cursor.bpp = vmsvga_fifo_read(s);
+
+	    if (SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask / sizeof (uint32_t) ||
+		SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image / sizeof (uint32_t)) {
+		    fprintf(stderr, "%s: DEFINE_CSURSOR too large x: %d, y: %d bpp: %d\n",
+			    __FUNCTION__, x, y, cursor.bpp);
+		    args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, cursor.bpp);
+		    goto badcmd;
+	    }
+
             for (args = 0; args < SVGA_BITMAP_SIZE(x, y); args ++)
                 cursor.mask[args] = vmsvga_fifo_read_raw(s);
             for (args = 0; args < SVGA_PIXMAP_SIZE(x, y, cursor.bpp); args ++)

^ permalink raw reply related	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2010-01-11 16:01 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-12-17 22:27 [Qemu-devel] [PATCH to consider for 0.12] vmware_vga: Don't crash on too-big DEFINE_CURSOR command Roland Dreier
2009-12-17 22:34 ` [Qemu-devel] " Anthony Liguori
2009-12-17 22:38   ` Roland Dreier
2009-12-17 22:49     ` Anthony Liguori
2009-12-17 22:41   ` Roland Dreier
2009-12-17 22:48     ` Anthony Liguori
2009-12-20 18:06       ` Roland Dreier
2010-01-06  4:43   ` [Qemu-devel] [PATCH resend] vmware_vga: Check cursor dimensions passed from guest to avoid buffer overflow Roland Dreier
2010-01-11 16:01     ` Anthony Liguori

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).