Re: Question on Compression for Raw Image

[Eric Blake <eblake@redhat.com>]

[-- Attachment #1.1.1: Type: text/plain, Size: 2059 bytes --]

On 10/20/20 9:22 AM, Wang, Wei W wrote:
> On Tuesday, October 20, 2020 4:01 PM, Kevin Wolf wrote:
>> Am 20.10.2020 um 03:31 hat Wang, Wei W geschrieben:
>>> Hi,
>>>
>>> Does anyone know the reason why raw-format.c doesn't have
>> compression
>>> support (but qcow has the supported added)?  For example, raw image
>>> backup with compression, "qemu-img convert -c -O raw origin.img
>>> dist.img", doesn't work.
>>
>> A raw image is by definition a file that contains the exact same sequence of
>> bytes as the guest sees, without any additional information or encoding. If
>> you compress a raw file, the guest will see compressed data on its hard disk
>> instead of the real data.
> 
> Ok, thanks. I'm thinking QEMU could do decompression of the compressed data in raw.img when guest reads data.
> 
>>
>> Anything you could do to add transparent compression to it would mean that
>> it's not a raw image any more, but a new image format.
>>
> Yes, decompression makes it transparent to the guest. Would you think it's good to reuse the raw image implementation, just add the compress/decompress option?

My recommendation would be implementing a new BDS filter that does
uncompression.  Then, you could do things like:

raw -> decompress -> file.xz

or even

qcow2 -> decompress -> file.qcow2.xz

By the way, the notion of filters is already possible in other ways.
For example, you can point qemu to read from an NBD server, and then use
nbdkit to do the decompression with its filters:

nbdkit --filter=xz file file.xz
raw -> nbd://localhost:10809

Also note that serving a decompressed view of a compressed image tends
to be a read-only proposition (you really CAN'T write to the image
without recompressing, but even if recompression has been blocked for
parallelism, you would end up writing far more of the file after
recompression than the amount of data written by a guest).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

[-- Attachment #1.1.2: OpenPGP_0xA7A16B4A2527436A_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 103475 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]