Re: [kvm-devel] Re: [PATCH 1/3] Refactor AIO interface to allow other AIO implementations

Jamie Lokier
2008-04-22T15:36:20Z

Avi Kivity wrote:
> >Perhaps. This raises another point about AIO vs. threads:
> >
> >If I submit sequential O_DIRECT reads with aio_read(), will they enter
> >the device read queue in the same order, and reach the disk in that
> >order (allowing for reordering when worthwhile by the elevator)?
>
> Yes, unless the implementation in the kernel (or glibc) is threaded.

>
> >With threads this isn't guaranteed and scheduling makes it quite
> >likely to issue the parallel synchronous reads out of order, and for
> >them to reach the disk out of order because the elevator doesn't see
> >them simultaneously.
>
> If the disk is busy, it doesn't matter. The requests will queue and the
> elevator will sort them out. So it's just the first few requests that
> may get to disk out of order.

There's two cases where it matters to a read-streaming app:

1. Disk isn't busy with anything else, maximum streaming
performance is desired.

2. Disk is busy with unrelated things, but you're using I/O
priorities to give the streaming app near-absolute priority.
Then you need to maintain overlapped streaming requests,
otherwise disk is given to a lower priority I/O. If that
happens often, you lose, priority is ineffective. Because one
of the streaming requests is usually being serviced, elevator
has similar limitations as for a disk which is not busy with
anything else.

> I haven't considered tape, but this is a good point indeed. I expect it
> doesn't make much of a difference for a loaded disk.

Yes, as long as it's loaded with unrelated requests at the same I/O
priority, the elevator has time to sort requests and hide thread
scheduling artifacts.

Btw, regarding QEMU: QEMU gets requests _after_ sorting by the guest's
elevator, then submits them to the host's elevator. If the guest and
host elevators are both configured 'anticipatory', do the anticipatory
delays add up?

-- Jamie