Re: Block alignment of qcow2 compress driver

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

28.01.2022 14:07, Richard W.M. Jones wrote:
> The commands below set up a sparse RAM disk, with an allocated block
> at offset 32K and another one at offset 1M-32K.  Then it tries to copy
> this to a compressed qcow2 file using qemu-nbd + the qemu compress
> filter:
> 
>    $ qemu-img create -f qcow2 output.qcow2 1M
>    $ qemu-nbd -t --image-opts driver=compress,file.driver=qcow2,file.file.driver=file,file.file.filename=output.qcow2 & sleep 1
>    $ nbdkit -U - \
>             data '@32768 1*32768 @1015808 1*32768' \
>             --run 'nbdcopy $uri nbd://localhost -p'
> 
> The nbdcopy command fails when zeroing the first 32K with:
> 
>    nbd://localhost: nbd_aio_zero: request is unaligned: Invalid argument
> 
> This is a bug in nbdcopy because it ignores the minimum block size
> being correctly declared by the compress filter:
> 
>    $ nbdinfo nbd://localhost
>    protocol: newstyle-fixed without TLS
>    export="":
> 	export-size: 1048576 (1M)
> 	uri: nbd://localhost:10809/
> 	contexts:
>    ...
> 		block_size_minimum: 65536          <----
> 		block_size_preferred: 65536
> 		block_size_maximum: 33554432
> 
> The compress filter sets the minimum block size to the the same as the
> qcow2 cluster size here:
> 
>    https://gitlab.com/qemu-project/qemu/-/blob/cfe63e46be0a1f8a7fd2fd5547222f8344a43279/block/filter-compress.c#L117
> 
> I patched qemu to force this to 4K:
> 
> -    bs->bl.request_alignment = bdi.cluster_size;
> +    //bs->bl.request_alignment = bdi.cluster_size;
> +    bs->bl.request_alignment = 4096;
> 
> and the copy above works, and the output file is compressed!
> 
> So my question is, does the compress filter in qemu really need to
> declare the large minimum block size?  I'm not especially concerned
> about efficiency, I'd prefer it just worked, and changing nbdcopy to
> understand block sizes is painful.
> 
> Is it already adjustable at run time?  (I tried using --image-opts
> like compress.request_alignment=4096 but it seems like the filter
> doesn't support anything I could think of, and I don't know how to
> list the supported options.)
> 


Hi!

I didn't read the whole thread, so in case it was not mentioned:

There is a limitation about compressed writes in qcow2 driver in Qemu: you can't do compressed write to the same cluster twice, the second write will fail. So, when we do some copying (or backup) and write cluster by cluster (or at least cluster-aligend) everything works. If you write partial cluster (with compress filter) it may work due to automatic RMW, but when you than try to write second part of same cluster it will fail.


-- 
Best regards,
Vladimir