From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from [140.186.70.92] (port=41742 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Ovaso-0003jH-CH
	for qemu-devel@nongnu.org; Tue, 14 Sep 2010 15:09:36 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69)
	(envelope-from <anthony@codemonkey.ws>) id 1Ovai3-0002YN-LO
	for qemu-devel@nongnu.org; Tue, 14 Sep 2010 14:58:28 -0400
Received: from mail-gw0-f45.google.com ([74.125.83.45]:55773)
	by eggs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <anthony@codemonkey.ws>) id 1Ovai3-0002YH-FO
	for qemu-devel@nongnu.org; Tue, 14 Sep 2010 14:58:27 -0400
Received: by gwb11 with SMTP id 11so2936451gwb.4
	for <qemu-devel@nongnu.org>; Tue, 14 Sep 2010 11:58:26 -0700 (PDT)
Message-ID: <4C8FC5CF.3080906@codemonkey.ws>
Date: Tue, 14 Sep 2010 13:58:23 -0500
From: Anthony Liguori <anthony@codemonkey.ws>
MIME-Version: 1.0
Subject: Re: [Qemu-devel] qcow2 performance plan
References: <4C8F7394.8060802@redhat.com>
	<4C8F7BE4.5010102@codemonkey.ws>	<4C8F9087.2050005@redhat.com>
	<4C8F92D9.2000908@codemonkey.ws>	<4C8F9920.7070908@redhat.com>
	<AANLkTimtONED4Spt-i1A4bHdAfivxo3MhuECk-NAHtk3@mail.gmail.com>
	<4C8F9FDE.8050004@codemonkey.ws> <4C8FA297.2020807@redhat.com>
	<4C8FAC15.1020500@codemonkey.ws> <4C8FAF92.3090503@redhat.com>
In-Reply-To: <4C8FAF92.3090503@redhat.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
List-Id: qemu-devel.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Avi Kivity <avi@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>, Stefan Hajnoczi <stefanha@gmail.com>, qemu-devel <qemu-devel@nongnu.org>

On 09/14/2010 12:23 PM, Avi Kivity wrote:
>  On 09/14/2010 07:08 PM, Anthony Liguori wrote:
>>> Yes, I hit this too.  So without this patch, it does serialize all 
>>> allocating writes?
>>
>>
>> Yes, but my patch is not enough as it turns out.
>>
>> When dealing with O_DIRECT, we have to handle RMW on our own which 
>> means we need to serialize access to the same sector.
>>
>> The way we're planning on addressing this in the short term is to 
>> break the single allocator queue into a per-L2 table queue.  So 
>> writes to the same L2 would be serialized but writes to different L2s 
>> would not be serialized.
>>
>
> So at least I read the code correctly.
>
> The next step (also addressed in the qcow2 performance plan) is to 
> batch writes to L2.  You'd actually expect to have many concurrent 
> allocating writes to one L2.  The first is sent to disk, but the 
> following ones just mark the L2 dirty.  When the write returns, it 
> sees it's still dirty and goes back to disk again.

Yeah, I have to think through batching quite a bit more but I agree that 
batching should be a natural next step and can further reduce the cost 
of updating metadata in a streaming workload.

Regards,

Anthony Liguori