Re: [Qemu-devel] [PATCH 8/8] docs: document how to configure the qcow2 L2/refcount caches

[Max Reitz <mreitz@redhat.com>]

On 11.05.2015 14:55, Alberto Garcia wrote:
> QEMU has options to configure the size of the L2 and refcount caches
> for the qcow2 format. However, choosing the right sizes for a
> particular disk image is not a straightforward operation since the
> ratio between the cache size and the allocated disk space is not
> obvious and depends on the size of the cluster.
>
> This document attempts to give an overview of both caches and how to
> configure their sizes.
>
> Signed-off-by: Alberto Garcia <berto@igalia.com>
> ---
>   docs/qcow2-cache.txt | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 117 insertions(+)
>   create mode 100644 docs/qcow2-cache.txt
>
> diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt
> new file mode 100644
> index 0000000..2570172
> --- /dev/null
> +++ b/docs/qcow2-cache.txt
> @@ -0,0 +1,117 @@
> +qcow2 L2/refcount cache configuration
> +=====================================
> +The QEMU qcow2 driver has two caches that can improve the I/O
> +performance significantly. However, setting the right cache sizes is
> +not a straightforward operation.
> +
> +This document attempts to give an overview of the L2 and refcount
> +caches, and how to configure them.
> +
> +Please refer to the docs/specs/qcow2.txt file for an in-depth
> +technical description of the qcow2 file format.
> +
> +
> +Clusters
> +--------
> +A qcow2 file is organized in units of constant size called clusters.
> +
> +The cluster size is configurable, but it must be a power of two and
> +its value 512 bytes or higher. QEMU currently defaults to 64 KB
> +clusters, and it does not support sizes larger than 2MB.
> +
> +The 'qemu-img create' command supports specifying the size using the
> +cluster_size option:
> +
> +   qemu-img create -f qcow2 -o cluster_size=128K hd.img 4G
> +
> +
> +The L2 tables
> +-------------
> +The qcow2 format uses a two-level structure to map the virtual disk as
> +seen by the guest to the disk image in the host. These structures are
> +called the L1 and L2 tables.
> +
> +There is one single L1 table per disk image. The table is small and is
> +always kept in memory.
> +
> +There can be many L2 tables, depending on how much space has been
> +allocated in the image. Each table is one cluster in size. In order to
> +read or write data from the virtual disk, QEMU needs to read its
> +corresponding L2 table to find out where that data is located. Since
> +reading the table for each I/O operation can be expensive, QEMU keeps
> +an L2 cache in memory to speed up disk access.
> +
> +The size of the L2 cache can be configured, and setting the right
> +value can improve the I/O performance significantly.
> +
> +
> +The refcount blocks
> +-------------------
> +The qcow2 format also mantains a reference count for each cluster. The
> +data is stored in a two-level structure similar to the L1/L2 tables
> +described above.
> +
> +The second level structures are called refcount blocks, are also one
> +cluster in size and the number is also variable and dependent on the
> +amount of allocated space.
> +
> +QEMU keeps a refcount cache to speed up I/O much like the
> +aforementioned L2 cache, and its size can also be configured.
> +
> +
> +Choosing the right cache sizes
> +------------------------------
> +In order to choose the cache sizes we need to know how they relate to
> +the amount of allocated space.
> +
> +The amount of virtual disk that can be mapped by the L2 and refcount
> +caches (in bytes) is:
> +
> +   disk_size = l2_cache_size * cluster_size / 8
> +   disk_size = refcount_cache_size * cluster_size / 2

Only with the default of refcount_bits=16. In the general case, it's 
refcount_cache_size * cluster_size * 8 / refcount_bits.

> +
> +With the default cluster size (64KB), that is
> +
> +   disk_size = l2_cache_size * 8192
> +   disk_size = refcount_cache_size * 32768
> +
> +So in order to cover n GB of disk space with the default cluster size
> +we need:
> +
> +   l2_cache_size = disk_size_GB * 131072
> +   refcount_cache_size = disk_size_GB * 32768
> +
> +QEMU has a default L2 cache of 1MB (1048576 bytes) and a refcount
> +cache of 256KB (262144 bytes), so using the formulas we've just seen
> +we have
> +
> +   1048576 / 131072 = 8 GB of virtual disk covered by that cache
> +    262144 /  32768 = 8 GB
> +
> +
> +How to configure the cache sizes
> +--------------------------------
> +Cache sizes can be configured using the -drive option in the
> +command-line, or the 'blockdev-add' QMP command.
> +
> +There are three options available, and all of them take bytes:
> +
> +"l2-cache-size":         maximum size of the L2 table cache
> +"refcount-cache-size":   maximum size of the refcount block cache
> +"cache-size":            maximum size of both caches combined
> +
> +There are two things that need to be taken into account:
> +
> + - Both caches must have a size that is a multiple of the cluster
> +   size.
> +
> + - The L2 cache should be 4 times bigger than the refcount cache in
> +   order to cover the same amount of disk space.

Again, only for refcount_bits=16. For the general case, it's "8 * 8 / 
refcount_bits = 64 / refcount_bits times as big".

> +
> +However, if you only set one of the options above, QEMU will
> +automatically adjust the others so that the 1/4 ratio is maintained.
> +This means that these options are equivalent:
> +
> +-drive file=hd.img,l2-cache-size=2097152
> +-drive file=hd.img,refcount-cache-size=524288
> +-drive file=hd.img,cache-size=2621440

Apart from this minor thing, looks good. Well, except that maybe it 
should be preferred to use .qcow2 as the extension for qcow2 files 
(instead of .img).

Max