Re: [Qemu-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

Re: [Qemu-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

[Chun Yan Liu]

>>> On 12/19/2014 at 06:38 PM, in message <1418985490.20028.27.camel@citrix.com>,
Ian Campbell <Ian.Campbell@citrix.com> wrote: 
> On Thu, 2014-12-18 at 23:58 -0700, Chun Yan Liu wrote: 
> >  
> > >>> On 12/18/2014 at 11:27 PM, in message  
> <1418916436.11882.101.camel@citrix.com>, 
> > Ian Campbell <Ian.Campbell@citrix.com> wrote:  
> > > On Tue, 2014-12-16 at 14:32 +0800, Chunyan Liu wrote:  
> > > > Changes to V8:  
> > > >   * remove libxl_domain_snapshot_create/delete/revert API  
> > > >   * export disk snapshot functionality for both xl and libvirt usage  
> > > >   
> > > >  
> ===========================================================================  
> > > > Libxl/libxlu Design  
> > > >   
> > > > 1. New Structures  
> > > >   
> > > > libxl_disk_snapshot = Struct("disk_snapshot",[  
> > > >     # target disk  
> > > >     ("disk",            libxl_device_disk),  
> > > >   
> > > >     # disk snapshot name  
> > > >     ("name",            string),  
> > > >   
> > > >     # internal/external disk snapshot?  
> > > >     ("external",        bool),  
> > > >   
> > > >     # for external disk snapshot, specify following two field  
> > > >     ("external_format", string),  
> > > >     ("external_path",   string),  
> > >   
> > > Should this be a KeyedUnion over a new LIBXL_DISK_SNAPSHOT_KIND enum  
> > > (with values INTERNAL and EXTERNAL)? 
> > The KeyedUnion seems to be unnecessary. Only EXTERNAL has data items, 
> > INTERNAL doesn't, and no third types. 
> >  
> > > This would automatically make the  
> > > binding between external==true and the fields which depend on that.  
> > >   
> > > external_format should be of type libxl_disk_format, unless it is  
> > > referring to something else?  
> >  
> > Yes. That's right. I'll update. 
> >  
> > >   
> > > Is it possible for format to differ from the format of the underlying  
> > > disk? Perhaps taking a snapshot of a raw disk as a qcow? 
> >  
> > This is related to implementation details. As I understand qemu's 
> > implementation, taking an external disk snapshot is actually a way: 
> > origin domain disk: a raw disk 
> > external= true, external_format: qcow2, external_path: test 
> > a), create a qcow2 file (test.qcow2) with  backing file (the raw disk) 
> > b), replace domain disk, now domain uses test.qcow2 (the raw disk 
> >      is actually to be the snapshot) 
> >  
> > So, I think the external_format can only be those supporting backing file. 
>  
> Not sure what you mean here. 
>  
> What about a phy snapshot via lvm snapshotting? 
>  
> > > In any case  
> > > passing in UNKNOWN and letting libxl choose (probably by picking the  
> > > same as the underlying disk) should be supported. 
> >  
> > If external_format is not passed (NULL), by default, we will use qcow2. 
>  
> I think you need to base this on the type of the original disk, if it is 
> e.g. vhd then making a qcow snapshot seems a bit odd. 
>  
> >  
> > >   
> > > > /*  This API might not be used by xl, since xl won't take care of  
> deleting  
> > > >  *  snapshots. But for libvirt, since libvirt manages snapshots and will  
> > > >  *  delete snapshot, this API will be used.  
> > > >  */  
> > > > int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid,  
> > > >                                libxl_disk_snapshot *snapshot, int nb);  
> > >   
> > > The three usecases I mentioned in the previous mail are important here,  
> > > because depending on which usecases you are considering there maybe a  
> > > many to one relationship between domains and a given snapshot (gold  
> > > image case). This interface cannot support that I think. 
> >  
> > I'm not quite clear about the three usecases, especially the 3rd usercase, 
> > so really not sure what's the requirement towards deleting disk snapshot. 
>  
> I hope my reply to the previous mail helped clear this up a bit. The 
> reason deleting a disk is interesting is because that is what you would 
> do after the backup was finished. 
>  
> > > When we discussed this in previous iterations I suggested a libxl  
> > > command to tell a VM that it needed to reexamine its disks to see if any  
> > > of the chains had changed. I'm sure that's not the only potential answer  
> > > though. 
> >   
> > About delete disk snapshot in a snapshot chain, whether we need to do 
> > extra work to avoid data break, it can be discussed: 
> > a). For external snapshots, usually it's based on backing file chain, qemu 
> > does this, vhd-util does this. In this case, to delete a domain snapshot, 
> > one doesn't need to do anything to disk (no need to delete disk snapshot 
> > at all). Downside is, there might be a long backing chain. 
>  
> I'm not sure what you mean here I'm afraid. If you are deleting a domain 
> snapshot why do you not want to delete the disk snapshots associated 
> with it? 
>  
> > b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't support 
> > snapshot of snapshot, so out of scope. For qcow2, delete any disk snapshot 
> > won't affect others. 
>  
> For either internal or external if you are removing a snapshot from the 
> middle of a chain which ends in one or more active disks, then surely 
> the disk backend associated with those domains need to get some sort of 
> notification, otherwise they would need to be written *very* carefully 
> in order to be able to cope with disk metadata changing under their 
> feet. 
>  
> Are you saying that the qemu/qcow implementation has indeed been written 
> with this in mind and can cope with arbitrary other processes modifying 
> the qcow metadata under their feet? 

Yes.

I add qemu-devel Kevin and Stefan in this thread in case my understanding
has somewhere wrong.

Kevin & Stefan,

About the qcow2 snapshot implementation,  in following snapshot chain case,
if we delete SNAPSHOT A, will it affect domain 1 and domain 2 which uses
SNAPSHOT B and SNAPSHOT C?

>From my understanding, creating a snapshot will increases refcount of original data,
deleting a snapshot only descreases the refcount (won't delete data until the refcount
becomes 0), so I think it won't affect domain 1 and domain 2.
Is that right?

Thanks,
Chunyan

>  
> e.g.  
> BASE---SNAPSHOT A---SNAPSHOT B --- domain 1 
>                  `--SNAPSHOT C --- domain 2 
>  
> If SNAPSHOT B and C are in active use then I would expect the deletion 
> of SNAPSHOT A would need to notify the backends associated with domain 1 
> and domain 2 somehow, so they don't get very confused. 
>  
> It's possible that this relates to a use case which you aren't intending 
> to address (e.g. the gold image use case), in which case it might be out 
> of scope here. 
>    
> Ian. 
>  
>  
>  

Re: [Qemu-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

[Ian Campbell]

On Mon, 2014-12-22 at 02:36 -0700, Chun Yan Liu wrote:
> > > b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't support 
> > > snapshot of snapshot, so out of scope. For qcow2, delete any disk snapshot 
> > > won't affect others. 
> >  
> > For either internal or external if you are removing a snapshot from the 
> > middle of a chain which ends in one or more active disks, then surely 
> > the disk backend associated with those domains need to get some sort of 
> > notification, otherwise they would need to be written *very* carefully 
> > in order to be able to cope with disk metadata changing under their 
> > feet. 
> >  
> > Are you saying that the qemu/qcow implementation has indeed been written 
> > with this in mind and can cope with arbitrary other processes modifying 
> > the qcow metadata under their feet? 
> 
> Yes.
> 
> I add qemu-devel Kevin and Stefan in this thread in case my understanding
> has somewhere wrong.
> 
> Kevin & Stefan,
> 
> About the qcow2 snapshot implementation,  in following snapshot chain case,
> if we delete SNAPSHOT A, will it affect domain 1 and domain 2 which uses
> SNAPSHOT B and SNAPSHOT C?
> 
> From my understanding, creating a snapshot will increases refcount of original data,
> deleting a snapshot only descreases the refcount (won't delete data until the refcount
> becomes 0), so I think it won't affect domain 1 and domain 2.
> Is that right?

I'm not worried about the data being deleted (I'm sure qcow2 will get
that right), but rather about the snapshot chain being collapsed and
therefore the metadata (e.g. the tables of which block is in which
backing file, and perhaps the location of the data itself) changing
while a domain is running, e.g.

BASE---SNAPSHOT A---SNAPSHOT B --- domain 1 
                 `--SNAPSHOT C --- domain 2 

becoming 

BASE----------------SNAPSHOT B --- domain 1 
                 `--SNAPSHOT C --- domain 2 

(essentially changing B and C's tables to accommodate the lack of A)

For an internal snapshot I can see that it would be sensible (and easy)
to keep A around as a "ghost", to avoid this case, and the need to
perhaps move data around or duplicate it.

If A were external though an admin might think they could delete the
file...

Ian.

Re: [Qemu-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

[Chun Yan Liu]

>>> On 1/8/2015 at 08:11 PM, in message <1420719107.19787.53.camel@citrix.com>, Ian
Campbell <Ian.Campbell@citrix.com> wrote: 
> On Mon, 2014-12-22 at 02:36 -0700, Chun Yan Liu wrote: 
> > > > b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't  
> support  
> > > > snapshot of snapshot, so out of scope. For qcow2, delete any disk  
> snapshot  
> > > > won't affect others.  
> > >   
> > > For either internal or external if you are removing a snapshot from the  
> > > middle of a chain which ends in one or more active disks, then surely  
> > > the disk backend associated with those domains need to get some sort of  
> > > notification, otherwise they would need to be written *very* carefully  
> > > in order to be able to cope with disk metadata changing under their  
> > > feet.  
> > >   
> > > Are you saying that the qemu/qcow implementation has indeed been written  
> > > with this in mind and can cope with arbitrary other processes modifying  
> > > the qcow metadata under their feet?  
> >  
> > Yes. 
> >  
> > I add qemu-devel Kevin and Stefan in this thread in case my understanding 
> > has somewhere wrong. 
> >  
> > Kevin & Stefan, 
> >  
> > About the qcow2 snapshot implementation,  in following snapshot chain case, 
> > if we delete SNAPSHOT A, will it affect domain 1 and domain 2 which uses 
> > SNAPSHOT B and SNAPSHOT C? 
> >  
> > From my understanding, creating a snapshot will increases refcount of  
> original data, 
> > deleting a snapshot only descreases the refcount (won't delete data until  
> the refcount 
> > becomes 0), so I think it won't affect domain 1 and domain 2. 
> > Is that right? 
>  
> I'm not worried about the data being deleted (I'm sure qcow2 will get 
> that right), but rather about the snapshot chain being collapsed and 
> therefore the metadata (e.g. the tables of which block is in which 
> backing file, and perhaps the location of the data itself) changing 
> while a domain is running, e.g. 
>  
> BASE---SNAPSHOT A---SNAPSHOT B --- domain 1  
>                  `--SNAPSHOT C --- domain 2  
>  
> becoming  
>  
> BASE----------------SNAPSHOT B --- domain 1  
>                  `--SNAPSHOT C --- domain 2  
>  
> (essentially changing B and C's tables to accommodate the lack of A) 
>  
> For an internal snapshot I can see that it would be sensible (and easy) 
> to keep A around as a "ghost", to avoid this case, and the need to 
> perhaps move data around or duplicate it. 

This is what we talked about a lot :-)

If I understand correctly, qcow2 internal snapshot implementation
should be like this:
Taking a snapshot is increasing a new table which records the data
clusters index in this snapshot, at the same time increase the refcount
of those data clusters;
Teleting a snapshot only decrease the refcount (not deleting the data,
only when refcount=0 the data will be deleted.) and remove the map table.

So, I think deleting a snapshot even within a snapshot chain, won't
affect other snapshots. ( Otherwise, even if that's not the case, how can
we move internal data around or duplicate it? )

>  
> If A were external though an admin might think they could delete the 
> file...

About external, generally, external disk snapshot are made by backing file,
qemu, vhd-util and lvm are all done that way. Like:
    snapshot A -> snapshot B -> Current image
A is B's backing file, B is current image's backing file, as backing file, it
should not be changed, otherwise if A is changed or deleted, B is corrupted.
So, a delete operation is actually a merge process, like 'virsh blockcommit':
shorten disk image chain by live merging the current active disk content.

To external disk snapshot: revert (may need to clone the snapshot out)
and delete (merge in fact as mentioned above). The complexity is
internal/external mixing case (take internal snapshot, revert, take
external snapshot, ...)

Chunyan

>
> Ian. 
>  
>  
>