Re: [kvm-devel] [Qemu-devel] [PATCH] QEMU: fsync AIO writes on flush request

[Jamie Lokier <jamie@shareable.org>]

Marcelo Tosatti wrote:
> I don't think the first qemu_aio_flush() is necessary because the fsync
> request will be enqueued after pending ones: 
> 
>        aio_fsync() function does a sync on all outstanding
>        asynchronous I/O operations associated with
>        aiocbp->aio_fildes.
>
>        More precisely, if op is O_SYNC, then all currently queued
>        I/O operations shall be completed as if by a call of
>        fsync(2), and if op is O_DSYNC, this call is the asynchronous
>        analog of fdatasync(2).  Note that this is a request only —
>        this call does not wait for I/O completion.
> 
> glibc sets the priority for fsync as 0, which is the same priority AIO
> reads and writes are submitted by QEMU.

Do AIO operations always get executed in the order they are submitted?

I was under the impression this is not guaranteed, as relaxed ordering
permits better I/O scheduling (e.g. to reduce disk seeks) - which is
one of the most useful points of AIO.  (Otherwise you might as well
just have one worker thread doing synchronous IO in order).

And because of that, I was under the impression the only way to
implement a write barrier+flush in AIO was (1) wait for pending writes
to complete, then (2) aio_fsync, then (3) wait for the aio_fsync.

I could be wrong, but I haven't seen any documentation which says
otherwise, and it's what I'd expect of an implementation.  I.e. it's
just an asynchronous version of fsync().

The quoted man page doesn't convince me.  It says "all currently
queued I/O operations shall be completed" which _could_ mean that
aio_fsync is an AIO barrier too.

But then "if by a call of fsync(2)" implies that aio_fsync+aio_suspend
could just be replaced by fsync() with no change of semantics.  So
"queued I/O operations" means what fsync() handles: dirty file data,
not in-flight AIO writes.

And you already noticed that fsync() is _not_ guaranteed to flush
in-flight AIO writes.  Being the asynchronous analog, aio_fsync()
would not either.

-- Jamie