Re: KVM call minutes for Sept 14

[Anthony Liguori <anthony@codemonkey.ws>]

On 09/14/2010 09:47 AM, Chris Wright wrote:
> 0.13
> - if all goes well...tomorrow
>    

To tag, it may be thursday for announcement.  I need to run a regression 
run tonight.

> qed/qcow2
> - increase concurrency, performance
>    

To achieve performance, a block driver must: 1) support concurrent 
request handling 2) not hold the qemu_mutex for prolonged periods of time.

QED never does (2) and supports (1) in all circumstances except cluster 
allocation today.

qcow2 can do (1) for the data read/write portions of an I/O request.  
All metadata read/write is serialized.  It also does (2) for all 
metadata operations and for CoW operations.

These are implementation details though.  The real claim of QED is that 
by having fewer IO ops required to satisfy a request, it achieves better 
performance especially since it achieves zero syncs in the cluster 
allocation path.  qcow2 has two syncs in the cluster allocation path 
today.  One sync is due to the refcount table.  Another sync is due to 
the fact that it doesn't require fsck support.

We could sync() on cluster allocations in QED and we'd still have better 
performance than qcow2 on paper because we have less IO ops and fewer 
sync()s.  That would eliminate fsck.

However, since the design target is to have no sync()s in the fast path, 
we're starting with fsck.

> - threading vs state machine
> - avi doesn't like qed reliance on fsck
>    - undermines value of error checking (errors become normal)
>    - prefer preallocation and fsck just checks for leaked blocks
>    

We will provide performance data on fsck.  That's the next step.

> - just load and validate metadata
> - options for correctness are
>    - fsync at every data allocation
>    - leak data blocks
>    

I contend that leaking data blocks is incorrect and potentially guest 
exploitable so it's not an option IMHO.

>    - scan
> - qed is pure statemachine
>    - state on stack, control flow vs function call
> - common need to separate handle requests concurrently, issue async i/o
> - most disk formats have similar metadata and methods
>    - lookup cluster, read/write data
>    - qed could be a path to cleaning up other formats (reusing)
> - need an incremental way to improve qcow2 performance
>    - threading doesn't seem to be the way to achieve this (incrementally)
>    

Because qcow2 already implements a state machine and the qemu 
infrastructure is based on events.  We can incrementally split states in 
qcow2.  Once you've got explicit states, it's trivial to compact those 
states into control flow using coroutines.

OTOH, threading would probably require a full rewrite of qcow2 and a lot 
of the block layer.

Regards,

Anthony Liguori

> - coroutines vs. traditional threads discussion
>    - parallel (and locking) vs few well-defined preemption points
> - plan for qed...attempt to merge in 0.14
>    - online fsck support is all that's missing
>    - add bdrv check callback, look for new patch series over the next week
> - back to list with discussion...
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>