Re: [Qemu-devel] Loading snapshot with readonly qcow2 image

[Eric Blake <eblake@redhat.com>]

On 12/14/18 10:03 AM, Michael Spradling wrote:

>> Can you combine -s (create a writable temp file) with -l to get what you
>> want?
>>
>> /me tries:

>>
>> I can confirm that 'qemu-nbd -s a' lets me write data that is discarded on
>> disconnect (lsof says a temp file in /var/tmp/vl.XXXXXX was created); and
>> that 'qemu-nbd -l snap a' lets me read the snapshot data. But mixing the two
>> fails, and it would be a nice bug to fix.
> 
> I briefly looked at the code and is seams to be using the same base
> functions as qemu does.  So, if I get this working for the model it
> might also start working for qemu-nbd.

>>
> Ideally, I want to not modify old images or create new images with
> qemu-img, so I have been not modifing qemu-img, but qemu directly
> itself.  My use case will have several snapshots in an image.(say
> 100).  I will then later resume each of these snapshots in a qemu
> session in parallel.  This is why I have gone done the route of modifying
> the temp snapshots file /var/tmp/vl.XXXXX L1 and l2 tables.  My
> understanding is if these are updated and the cluster doesn't exists in
> the temp file the code will then look for it in the backing file.  Still
> researching this area.

Right now, the only thing that qemu reads from a backing file is a guest 
cluster. L1/L2 clusters have to be local to the file that they are 
describing (there is no way to make an L2 table fall back to the 
contents of a different cluster in the backing file).  It boils down to:

Reads:
Does the active layer have an L2 mapping for the current cluster being 
read?  Yes - read that cluster. No - ask the backing layer to provide 
the contents of that cluster (and if copy-on-read is enabled, also write 
those contents in a fresh allocation so that the current layer no longer 
has to defer to the backing).

Writes:
Does the active layer have an L2 mapping for the current cluster 
containing the data being written? Yes - modify that cluster in place. 
No - allocate an new cluster, and if the write was for less than a full 
cluster, also ask the backing layer to provide the contents of the rest 
of the cluster for a copy-on-write action. After the write, the current 
layer no longer has to defer to the backing.

Creating an arbitrary qcow2 file on top of any arbitrary read-only 
backing layer (including 'qemu-nbd -l snap image) should be doable, even 
if verbose (since the "backing file" of a qcow2 BDS node can be any 
other BDS).  Providing some shorter command lines, like making 'qemu-nbd 
-s -l snap image' work so that you don't have to provide your own manual 
overlay, is thus not a high priority.

> 
>>>
>>> I still don't have this working yet and I believe my area of problems is
>>> qcow2_update_snapshot_refcount.  Can anyone explain what this does
>>> exactly.  It seems the function does three different things based on the
>>> value of addend, either -1, 0, 1, but its somewhat unclear.
>>
>> Every cluster of qcow2 is reference-counted, to track which portions of the
>> file are (supposed to be) in use according to following the metadata trails.
>> When internal snapshots are used, this is implemented by incrementing the
>> refcount for each cluster that is reachable both from the snapshot and from
>> the current L1 table (update_snapshot_refcount +1), then when writing to the
>> cluster we break the reference count by writing the new data to a new
>> allocation and decrementing the reference count of the old cluster. When
>> trimming clusters, we decrement the refcount, and if it goes to 0 the
>> cluster can be reused for something else.
> 
> I think I understand this.  That would satifys addend being a -1 or 1.
> I am still unclear why you would call the fuction with addend being 0.

An addend of 0 allows a couple of callers to temporarily have an 
inconsistent image for the sake of optimizing a bulk allocation/freeing, 
followed by informing the refcount table to match, with fewer changes to 
the cluster containing the refcounts than if the algorithm had to 
accurately use -1/+1 on a per-cluster basis.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org