Re: [ANNOUNCE] Native Linux KVM tool v2

[Ingo Molnar <mingo@elte.hu>]

* Pekka Enberg <penberg@kernel.org> wrote:

> On Thu, Jun 16, 2011 at 1:07 AM, Alexander Graf <agraf@suse.de> wrote:
> >> qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio
> >
> > Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO?
> 
> We don't use AIO at all. It's just normal read()/write() with a 
> thread pool. I actually looked at AIO but didn't really see why 
> we'd want to use it.

We could certainly try kernel AIO, it would allow us to do all the 
virtio-blk logic from the vcpu thread, without single threading it - 
turning the QCOW2 logic into an AIO driven state machine in essence.

Advantages:

 - we wouldnt do context-switching between the vcpu thread and the 
   helper threads

 - it would potentially provide tighter caching and potentially would 
   allow higher scalability.

Disadvantages:

 - the kaio codepaths are actually *more* complex than the regular 
   read()/write() IO codepaths - they keep track of an 'IO context', 
   so part of the efficiency advantages are spent on AIO tracking.

 - executing AIO in the vcpu thread eats up precious vcpu execution 
   time: combined QCOW2 throughput would be limited by a single 
   core's performance, and any time spent on QCOW2 processing would 
   not be spent running the guest CPU. (In such a model we certainly 
   couldnt do more intelligent, CPU-intense storage solutions like on 
   the fly compress/decompress of QCOW2 data.)

 - state machines are also fragile in the sense that any 
   unintentional blocking of the vcpu context will kill the 
   performance and latencies of *all* processing in certain 
   circumstances. So we generally strive to keep the vcpu demux path 
   obvious, simple and atomic.

 - more advanced security models go out the window as well: we 
   couldnt isolate drivers from each other if all of them execute in 
   the same vcpu context ...

 - state machines are also notoriously difficult to develop,
   debug and maintain.

So careful performance, scalability, IO delay and maintainability 
measurements have to accompany such a model switch, because the 
disadvantages are numerous.

I'd only consider KAIO it if it provides some *real* measurable 
performance advantage of at least 10% in some important usecase.
A few percent probably wouldnt be worth it.

Thanks,

	Ingo