[Qemu-devel] Using TRIM to shrink qcow2 images

[Qemu-devel] Using TRIM to shrink qcow2 images

[Ralf Ramsauer]

Hi,

QCOW2 uses a similar idea like file holes (sparse files) on filesystems [1].
RAW Images also may use file holes.

If qemu would support TRIM, then the guest could easily discard and zero
all unused blocks.
The host system could detect all unused blocks in a (e.g.) qcow2 image
and shrink it down to its minimum size.

Did anyone already think about that?

[1] : https://people.gnome.org/~markmc/qcow-image-format.html

Regards,

-- 
Ralf Ramsauer

PGP: 0x8F10049B

Re: [Qemu-devel] Using TRIM to shrink qcow2 images

[François Revol]

On 08/08/2013 12:34, Ralf Ramsauer wrote:
> Hi,
> 
> QCOW2 uses a similar idea like file holes (sparse files) on filesystems [1].
> RAW Images also may use file holes.
> 
> If qemu would support TRIM, then the guest could easily discard and zero
> all unused blocks.
> The host system could detect all unused blocks in a (e.g.) qcow2 image
> and shrink it down to its minimum size.
> 
> Did anyone already think about that?
> 

Yes but no time to implement.


I believe VirtualBox has some support for TRIM already btw.

François.

Re: [Qemu-devel] Using TRIM to shrink qcow2 images

[Paolo Bonzini]

On 08/08/2013 12:34 PM, Ralf Ramsauer wrote:
> Hi,
>
> QCOW2 uses a similar idea like file holes (sparse files) on filesystems [1].
> RAW Images also may use file holes.
>
> If qemu would support TRIM, then the guest could easily discard and zero
> all unused blocks.

QEMU supports trim.  1.5 supports it with raw images only, 1.6 will add 
qcow2 support.  Because it has the potential to cause fragmentation, it 
needs to be enabled explicitly.  Just add "discard=on" to the -drive option:

     qemu-system-x86_64 -enable-kvm -m 2048 ...\
        -drive if=virtio,discard=on,file=$HOME/foo.qcow2

It's probably always a good idea if the image is a block device on an 
SSD, but not necessarily on an image that is backed (for example) by a 
file or by a thin-provisioned logical volume.

> The host system could detect all unused blocks in a (e.g.) qcow2 image
> and shrink it down to its minimum size.

This depends on the host support for discard (on block device-backed 
qcow2 images) or hole punching (for file-backed qcow2 images).  For 
files, running fstrim in the guest will shrink down the on-disk 
footprint of a qcow2 image.

Paolo

> Did anyone already think about that?
>
> [1] : https://people.gnome.org/~markmc/qcow-image-format.html
>
> Regards,
>

Re: [Qemu-devel] Using TRIM to shrink qcow2 images

[Alexey Kardashevskiy]

[-- Attachment #1: Type: text/plain, Size: 1887 bytes --]

Hi Paolo!




On Thu, Aug 8, 2013 at 10:00 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 08/08/2013 12:34 PM, Ralf Ramsauer wrote:
>
>> Hi,
>>
>> QCOW2 uses a similar idea like file holes (sparse files) on filesystems
>> [1].
>> RAW Images also may use file holes.
>>
>> If qemu would support TRIM, then the guest could easily discard and zero
>> all unused blocks.
>>
>
> QEMU supports trim.  1.5 supports it with raw images only, 1.6 will add
> qcow2 support.  Because it has the potential to cause fragmentation, it
> needs to be enabled explicitly.  Just add "discard=on" to the -drive option:
>
>     qemu-system-x86_64 -enable-kvm -m 2048 ...\
>        -drive if=virtio,discard=on,file=$**HOME/foo.qcow2
>
>

btw I tried "discard=on" as you described (with if=virtio) and received an
error:
[root@localhost ~]# fstrim -v /
fstrim: /: FITRIM ioctl failed: Operation not supported

In my case if=virtio means virtio-blk. If I use virtio-scsi or ibmvscsi,
fstrim works just fine but failing virtio-blk bothers me a bit as we might
have a bug in ppc64, do not we? (we == ppc64 folks, of course :) )




> It's probably always a good idea if the image is a block device on an SSD,
> but not necessarily on an image that is backed (for example) by a file or
> by a thin-provisioned logical volume.
>
>
>  The host system could detect all unused blocks in a (e.g.) qcow2 image
>> and shrink it down to its minimum size.
>>
>
> This depends on the host support for discard (on block device-backed qcow2
> images) or hole punching (for file-backed qcow2 images).  For files,
> running fstrim in the guest will shrink down the on-disk footprint of a
> qcow2 image.
>
> Paolo
>
>
>  Did anyone already think about that?
>>
>> [1] : https://people.gnome.org/~**markmc/qcow-image-format.html<https://people.gnome.org/~markmc/qcow-image-format.html>
>>
>> Regards,
>>
>>
>
>


-- 
Alexey

[-- Attachment #2: Type: text/html, Size: 3166 bytes --]

Re: [Qemu-devel] Using TRIM to shrink qcow2 images

[Paolo Bonzini]

Il 04/10/2013 08:22, Alexey Kardashevskiy ha scritto:
> 
> btw I tried "discard=on" as you described (with if=virtio) and received
> an error:
> [root@localhost ~]# fstrim -v /
> fstrim: /: FITRIM ioctl failed: Operation not supported
> 
> In my case if=virtio means virtio-blk. If I use virtio-scsi or ibmvscsi,
> fstrim works just fine but failing virtio-blk bothers me a bit as we
> might have a bug in ppc64, do not we? (we == ppc64 folks, of course :) )
> 

No, virtio-blk does not support trim.

Paolo

Re: [Qemu-devel] Using TRIM to shrink qcow2 images

[Alexey Kardashevskiy]

On 10/04/2013 05:05 PM, Paolo Bonzini wrote:
> Il 04/10/2013 08:22, Alexey Kardashevskiy ha scritto:
>>
>> btw I tried "discard=on" as you described (with if=virtio) and received
>> an error:
>> [root@localhost ~]# fstrim -v /
>> fstrim: /: FITRIM ioctl failed: Operation not supported
>>
>> In my case if=virtio means virtio-blk. If I use virtio-scsi or ibmvscsi,
>> fstrim works just fine but failing virtio-blk bothers me a bit as we
>> might have a bug in ppc64, do not we? (we == ppc64 folks, of course :) )
>>
> 
> No, virtio-blk does not support trim.

So the example was broken or if=virtio uses virtio-scsi (instead of
virtio-blk) on x86?


-- 
Alexey

Re: [Qemu-devel] Using TRIM to shrink qcow2 images

[Paolo Bonzini]

Il 04/10/2013 09:07, Alexey Kardashevskiy ha scritto:
> On 10/04/2013 05:05 PM, Paolo Bonzini wrote:
>> Il 04/10/2013 08:22, Alexey Kardashevskiy ha scritto:
>>>
>>> btw I tried "discard=on" as you described (with if=virtio) and received
>>> an error:
>>> [root@localhost ~]# fstrim -v /
>>> fstrim: /: FITRIM ioctl failed: Operation not supported
>>>
>>> In my case if=virtio means virtio-blk. If I use virtio-scsi or ibmvscsi,
>>> fstrim works just fine but failing virtio-blk bothers me a bit as we
>>> might have a bug in ppc64, do not we? (we == ppc64 folks, of course :) )
>>>
>>
>> No, virtio-blk does not support trim.
> 
> So the example was broken or if=virtio uses virtio-scsi (instead of
> virtio-blk) on x86?

It was broken.  IDE works, as well as any SCSI HBA (including USB storage).

Paolo