From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1JfIvP-0006nt-B8
	for qemu-devel@nongnu.org; Fri, 28 Mar 2008 14:03:35 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1JfIvN-0006ne-FG
	for qemu-devel@nongnu.org; Fri, 28 Mar 2008 14:03:34 -0400
Received: from [199.232.76.173] (helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1JfIvN-0006nb-9Y
	for qemu-devel@nongnu.org; Fri, 28 Mar 2008 14:03:33 -0400
Received: from mail2.shareable.org ([80.68.89.115])
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <jamie@shareable.org>) id 1JfIvM-0004y9-QN
	for qemu-devel@nongnu.org; Fri, 28 Mar 2008 14:03:33 -0400
Date: Fri, 28 Mar 2008 18:03:25 +0000
From: Jamie Lokier <jamie@shareable.org>
Subject: Re: [kvm-devel] [Qemu-devel] [PATCH] QEMU: fsync AIO writes on flush
	request
Message-ID: <20080328180324.GA22555@shareable.org>
References: <20080328150517.GA18077@dmt> <20080328150703.GA19624@shareable.org>
	<20080328163116.GA18853@dmt>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20080328163116.GA18853@dmt>
Reply-To: qemu-devel@nongnu.org
List-Id: qemu-devel.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Marcelo Tosatti <marcelo@kvack.org>
Cc: kvm-devel <kvm-devel@lists.sourceforge.net>, qemu-devel@nongnu.org

Marcelo Tosatti wrote:
> On Fri, Mar 28, 2008 at 03:07:03PM +0000, Jamie Lokier wrote:
> > Marcelo Tosatti wrote:
> > > Its necessary to guarantee that pending AIO writes have reached stable
> > > storage when the flush request returns.
> > > 
> > > Also change fsync() to fdatasync(), since the modification time is not
> > > critical data.
> > > +    if (aio_fsync(O_DSYNC, &acb->aiocb) < 0) {
> > 
> > >      BDRVRawState *s = bs->opaque;
> > > -    fsync(s->fd);
> > > +    raw_aio_flush(bs);
> > > +    fdatasync(s->fd);
> > > +
> > > +    /* We rely on the fact that no other AIO will be submitted
> > > +     * in parallel, but this should be fixed by per-device
> > > +     * AIO queues when allowing multiple CPU's to process IO
> > > +     * in QEMU.
> > > +     */
> > > +    qemu_aio_flush();
> > 
> > I'm a bit confused by this.  Why do you need aio_fsync(O_DSYNC) _and_
> > synchronous fdatasync() calls?  Aren't they equivalent?
> 
> fdatasync() will write and wait for completion of dirty file data
> present in memory.
> 
> aio_write() only queues data for submission:
> 
>        The "asynchronous" means that this call returns as soon as the  request
>        has  been  enqueued;  the  write may or may not have completed when the
>        call returns. One tests for completion using aio_error(3).
> 

> So fdatasync() is not enough because data written via AIO may not
> have been reflected as "dirty file data" through write() by the time
> raw_flush() is called.

Sure.  But why isn't the aio_fsync(O_DSYNC) enough by itself?

It seems to me you should have something like this:

    /* Flush pending aio_writes until they are dirty data,
       and wait before the aio_fsync. */
    qemu_aio_flush();

    /* Call aio_fsync(O_DSYNC). */
    raw_aio_flush(bs);

    /* Wait for the aio_fsync to complete. */
    qemu_aio_flush();

What am I missing?

-- Jamie