Re: [kvm-devel] Re: [PATCH 1/3] Refactor AIO interface to allow other AIO implementations

[Jamie Lokier <jamie@shareable.org>]

Avi Kivity wrote:
> >Perhaps.  This raises another point about AIO vs. threads:
> >
> >If I submit sequential O_DIRECT reads with aio_read(), will they enter
> >the device read queue in the same order, and reach the disk in that
> >order (allowing for reordering when worthwhile by the elevator)?
> 
> Yes, unless the implementation in the kernel (or glibc) is threaded.

> 
> >With threads this isn't guaranteed and scheduling makes it quite
> >likely to issue the parallel synchronous reads out of order, and for
> >them to reach the disk out of order because the elevator doesn't see
> >them simultaneously.
> 
> If the disk is busy, it doesn't matter.  The requests will queue and the 
> elevator will sort them out.  So it's just the first few requests that 
> may get to disk out of order.

There's two cases where it matters to a read-streaming app:

    1. Disk isn't busy with anything else, maximum streaming
       performance is desired.

    2. Disk is busy with unrelated things, but you're using I/O
       priorities to give the streaming app near-absolute priority.
       Then you need to maintain overlapped streaming requests,
       otherwise disk is given to a lower priority I/O.  If that
       happens often, you lose, priority is ineffective.  Because one
       of the streaming requests is usually being serviced, elevator
       has similar limitations as for a disk which is not busy with
       anything else.

> I haven't considered tape, but this is a good point indeed.  I expect it 
> doesn't make much of a difference for a loaded disk.

Yes, as long as it's loaded with unrelated requests at the same I/O
priority, the elevator has time to sort requests and hide thread
scheduling artifacts.

Btw, regarding QEMU: QEMU gets requests _after_ sorting by the guest's
elevator, then submits them to the host's elevator.  If the guest and
host elevators are both configured 'anticipatory', do the anticipatory
delays add up?

-- Jamie