Compact backing chain (sparsify + compress)

Compact backing chain (sparsify + compress)

[Christopher Pereira]

Hi,

I would like to revisit this old thread from 2016 with a special use 
case that I believe should be a standard `virsh` command:
https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg03571.html

**Summary:**

Given this QEMU backing chain:
`base <- snap1 <- snap2 <- snap3 (active)`

We want to merge `base <- snap1 <- snap2` into a new snapshot 
`collapsed-base` that is:
1. Sparsified (`virt-sparsify`)
2. Compressed

The resulting backing chain would be:
`collapsed-base <- snap3 (active)`

**Motivation:**

- We perform daily backup snapshots and never modify existing files (too 
dangerous). We only rebase.
- We collapse older chains into a new `collapsed-base` snapshot to limit 
chain size and avoid performance degradation.

We have been doing this successfully for over 10 years using:

- `qemu-img convert`
- `virt-sparsify`
- `virsh save`
- `qemu-img rebase`
- `virsh resume`

**Problems:**

- There is a small downtime due to `virsh save`/`resume`.
- In recent QEMU versions, `virsh` adds a `backingStore` tag to the XML 
even when using the `--no-metadata` option. This causes inconsistencies 
after `qemu-img rebase`.

We didn’t use QMP because it didn’t support sparsify + compression in 
the past.

**Questions:**

- Is there now a better way to achieve this?
- Could this feature be implemented or supported directly in `virsh`?

In my opinion, this would be the ideal backup solution: we could travel 
in time, sync immutable snapshots to a remote backup server, and 
maintain performance.

Re: Compact backing chain (sparsify + compress)

[Markus Armbruster]

Adding cc:s.

Christopher Pereira <kripper@imatronix.cl> writes:

> Hi,
>
> I would like to revisit this old thread from 2016 with a special use case that I believe should be a standard `virsh` command:
> https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg03571.html
>
> **Summary:**
>
> Given this QEMU backing chain:
> `base <- snap1 <- snap2 <- snap3 (active)`
>
> We want to merge `base <- snap1 <- snap2` into a new snapshot `collapsed-base` that is:
> 1. Sparsified (`virt-sparsify`)
> 2. Compressed
>
> The resulting backing chain would be:
> `collapsed-base <- snap3 (active)`
>
> **Motivation:**
>
> - We perform daily backup snapshots and never modify existing files (too dangerous). We only rebase.
> - We collapse older chains into a new `collapsed-base` snapshot to limit chain size and avoid performance degradation.
>
> We have been doing this successfully for over 10 years using:
>
> - `qemu-img convert`
> - `virt-sparsify`
> - `virsh save`
> - `qemu-img rebase`
> - `virsh resume`
>
> **Problems:**
>
> - There is a small downtime due to `virsh save`/`resume`.
> - In recent QEMU versions, `virsh` adds a `backingStore` tag to the XML even when using the `--no-metadata` option. This causes inconsistencies after `qemu-img rebase`.
>
> We didn’t use QMP because it didn’t support sparsify + compression in the past.
>
> **Questions:**
>
> - Is there now a better way to achieve this?
> - Could this feature be implemented or supported directly in `virsh`?
>
> In my opinion, this would be the ideal backup solution: we could travel in time, sync immutable snapshots to a remote backup server, and maintain performance.

Re: Compact backing chain (sparsify + compress)

[Peter Krempa]

On Mon, Nov 10, 2025 at 07:07:54 +0100, Markus Armbruster via Users wrote:
> Adding cc:s.
> 
> Christopher Pereira <kripper@imatronix.cl> writes:
> 
> > Hi,
> >
> > I would like to revisit this old thread from 2016 with a special use case that I believe should be a standard `virsh` command:
> > https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg03571.html
> >
> > **Summary:**
> >
> > Given this QEMU backing chain:
> > `base <- snap1 <- snap2 <- snap3 (active)`
> >
> > We want to merge `base <- snap1 <- snap2` into a new snapshot `collapsed-base` that is:
> > 1. Sparsified (`virt-sparsify`)
> > 2. Compressed
> >
> > The resulting backing chain would be:
> > `collapsed-base <- snap3 (active)`

So collapsing 'snap1' and 'snap2' into 'base' is currently possible
directly via libvirt's 'virsh blockcommit'. Select 'snap2' as 'top'
argument.

Unfornunately 'virt-sparsify' can be used only on inactive VMs as it
modifies the disk. You could sparisify the VM directly from within the
guest OS if you enable discard propagation.

Regarding qcow2 compression, libvirt currently doesn't support adding
'compress' filters to the backing chain which could theoretically make
the base to be compressed. Compression has performance impact also on
reads and thus users will not normally want to use it, especially if the
majority of the image (for reads) is still in the backing image.

> > **Motivation:**
> >
> > - We perform daily backup snapshots and never modify existing files (too dangerous). We only rebase.

For backups we now have specific APIs to perform block-level
incremental backups via qemu's support to track block changes.

See

https://www.libvirt.org/kbase/live_full_disk_backup.html
https://www.libvirt.org/formatbackup.html

> > - We collapse older chains into a new `collapsed-base` snapshot to limit chain size and avoid performance degradation.
> >
> > We have been doing this successfully for over 10 years using:
> >
> > - `qemu-img convert`
> > - `virt-sparsify`
> > - `virsh save`
> > - `qemu-img rebase`
> > - `virsh resume`
> >
> > **Problems:**
> >
> > - There is a small downtime due to `virsh save`/`resume`.
> > - In recent QEMU versions, `virsh` adds a `backingStore` tag to the XML even when using the `--no-metadata` option. This causes inconsistencies after `qemu-img rebase`.

That's a libvirt change that added backing store tracking. When you
collapse the backing chain via libvirt it modifies the <backingStore>
elements approrpiately.

> >
> > We didn’t use QMP because it didn’t support sparsify + compression in the past.
> >
> > **Questions:**
> >
> > - Is there now a better way to achieve this?

Consider using the backup APIs; see above.

> > - Could this feature be implemented or supported directly in `virsh`?
> >
> > In my opinion, this would be the ideal backup solution: we could travel in time, sync immutable snapshots to a remote backup server, and maintain performance.

So current state:
- for backups libvirt now supports the backup APIs to use qemu's changed
  block tracking
- backing chani merging is already implemented
- sparsification for a live VM needs to be driven via the VM itself
  (virt-sparsify can't be used while the VM is running or saved)
- compression support (while technically feasible) is unlikely to be ever
  implemented in libvirt