From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from [140.186.70.92] (port=59626 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Puu7O-0001y3-G0
	for qemu-devel@nongnu.org; Wed, 02 Mar 2011 17:02:03 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <weil@mail.berlios.de>) id 1Puu7N-0007dh-CQ
	for qemu-devel@nongnu.org; Wed, 02 Mar 2011 17:02:02 -0500
Received: from moutng.kundenserver.de ([212.227.126.171]:61083)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <weil@mail.berlios.de>) id 1Puu7M-0007dY-Sj
	for qemu-devel@nongnu.org; Wed, 02 Mar 2011 17:02:01 -0500
Message-ID: <4D6EBE4B.8070705@mail.berlios.de>
Date: Wed, 02 Mar 2011 23:01:47 +0100
From: Stefan Weil <weil@mail.berlios.de>
MIME-Version: 1.0
Subject: Re: [Qemu-devel] Re: [PATCH RESEND 2/2] vnc: Fix heap corruption
References: <4D6DBDA4.3050909@cn.fujitsu.com>	<4D6DC06B.6070308@cn.fujitsu.com>	<AANLkTi=sjn2vHo7B-1yii4ofX5cseLei0u-BbWugdwt6@mail.gmail.com>	<4D6E8E3A.50106@mail.berlios.de>
	<AANLkTi=29R6oLQQyumP2du8Bm-bZji98_Ah5M5UnHkns@mail.gmail.com>
In-Reply-To: <AANLkTi=29R6oLQQyumP2du8Bm-bZji98_Ah5M5UnHkns@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
List-Id: qemu-devel.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Peter Maydell <peter.maydell@linaro.org>
Cc: Anthony Liguori <aliguori@us.ibm.com>, qemu-devel <qemu-devel@nongnu.org>, Corentin Chary <corentin.chary@gmail.com>

Am 02.03.2011 19:47, schrieb Peter Maydell:
> On 2 March 2011 18:36, Stefan Weil <weil@mail.berlios.de> wrote:
>> No. I dont't think that the third parameter of bitmap_clear is
>> ok like that. See my patch for the correct value.
>
> Wen's patch:
>
> + const size_t width = ds_get_width(vd->ds) / 16;
> [...]
> -    bitmap_set(width_mask, 0, (ds_get_width(vd->ds) / 16));
> -    bitmap_clear(width_mask, (ds_get_width(vd->ds) / 16),
> -                 VNC_DIRTY_WORDS * BITS_PER_LONG);
> +    bitmap_set(width_mask, 0, width);
> +    bitmap_clear(width_mask, width, VNC_DIRTY_WORDS * BITS_PER_LONG - 
> width);
>
> Your patch:
>
> bitmap_clear(width_mask, (ds_get_width(vd->ds) / 16),
> - VNC_DIRTY_WORDS * BITS_PER_LONG);
> + (VNC_MAX_WIDTH - ds_get_width(vd->ds)) / 16);
>
> Since ui/vnc.h has:
>
> #define VNC_DIRTY_WORDS (VNC_MAX_WIDTH / (16 * BITS_PER_LONG))
>
> the third parameter to bitmap_clear is the same value in
> both cases, isn't it? Or is this a rounding bug?
>
> -- PMM

Because of rounding effects, both values can be different.

The part missing in my patch is correct handling of another
rounding effect:

VNC_DIRTY_WORDS is exact for 32 bit long values (and the
"old" code which used uint32_t until some weeks ago), where
VNC_DIRTY_WORDS = 2560/16/32 = 5.

For 64 bit values, VNC_DIRTY_WORDS = 2560/16/64 = 2 (rounded)!

Stefan W.