Re: [Qemu-devel] Re: [kvm-devel] [PATCH 1/3] Refactor AIO interface to allow other AIO implementations

[Jamie Lokier <jamie@shareable.org>]

Daniel P. Berrange wrote:
> > Those cases aren't always discoverable.  Linux-aio just falls back to 
> > using synchronous IO.  It's pretty terrible.  We need a new AIO 
> > interface for Linux (and yes, we're working on this).  Once we have 
> > something better, we'll change that to be the default and things will 
> > Just Work for most users.
> 
> If QEMU can't discover cases where it won't work, what criteria should
> the end user use to decide between the impls, or for that matter, what
> criteria should a management api/app like libvirt use ? If the only decision
> logic is  'try it & benchmark your VM' then its not a particularly useful
> option.

Good use of Linux-AIO requires that you basically "know" which cases
it handles well, and which ones it doesn't.  Falling back to
synchronous I/O with no indication (except speed) is a pretty
atrocious API imho.  But that's what the Linux folks decided to do.

I suspect what you have to do is:

    1. Try opening the file with O_DIRECT.
    2. Use fstat to check the filesystem type and block device type.
    3. If it's on a whitelist of filesystem types,
    4. and a whitelist of block device types,
    5. and the kernel version is later than an fs+bd-dependent value,
    6. then select an alignment size (kernel version dependent)
       and use Linux-AIO with it.

Otherwise don't use Linux-AIO.  You may then decide to use Glibc's
POSIX-AIO (which uses threads), or use threads for I/O yourself.

In future, the above recipe will be more complicated, in that you have
to use the same decision tree to decide between:

    - Synchronous IO.
    - Your own thread based IO.
    - Glibc POSIX-AIO using threads.
    - Linux-AIO.
    - Virtio thing or whatever is based around vringfd.
    - Syslets if they gain traction and perform well.

> I've basically got a choice of making libvirt always ad '-aio linux'
> or never add it at all. My inclination is to the latter since it is
> compatible with existing QEMU which has no -aio option. Presumably
> '-aio linux' is intended to provide some performance benefit so it'd
> be nice to use it. If we can't express some criteria under which it
> should be turned on, I can't enable it; where as if you can express
> some criteria, then QEMU should apply them automatically.

I'm of the view that '-aio auto' would be a really good option - and
when it's proven itself, it should be the default.  It could work on
all QEMU hosts: it would pick synchronous IO when there is nothing else.

The criteria for selecting a good AIO strategy on Linux are quite
complex, and might be worth hard coding.  In that case, putting that
into QEMU itself would be much better than every program which
launches QEMU having it's own implementation of the criteria.

> Pushing this choice of AIO impls to the app or user invoking QEMU just
> does not seem like a win here.

I think having the choice is very good, because whatever the hard
coded selection criteria, there will be times when it's wrong (ideally
in conservative ways - it should always be functional, just suboptimal).

So I do support this patch to add the switch.

But _forcing_ the user to decide is not good, since the criteria are
rather obscure and change with things like filesystem.  At least, a
set of command line options to QEMU ought to work when you copy a VM
to another machine!

So I think '-aio auto', which invokes the selection criteria of the
day and is guaranteed to work (conservatively picking a slower method
if it cannot be sure a faster one will work) would be the most useful
option of all.

-- Jamie