[linux-lvm] Snapshots of snapshots are not supported yet

[linux-lvm] Snapshots of snapshots are not supported yet

[markw]

I'm trying to get a snapshot of a snapshot. I know the new thin
provisioning stuff supports this, but I'm not sure the release state is
something I can use at a customer.

Using LVM2, the snapshot format is tried and true, but inefficient if you
want to take multiple snapshots.

Looking at dmsetup, there doesn't seem like there would be an issue taking
a snapshot of a snapshot, but it is disabled in the tools.

I guess my question is this: Can it be done? Is there any reason why it
should not work?

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[Zdenek Kabelac]

Dne 8.7.2013 18:05, markw@mohawksoft.com napsal(a):
> I'm trying to get a snapshot of a snapshot. I know the new thin
> provisioning stuff supports this, but I'm not sure the release state is
> something I can use at a customer.
>
> Using LVM2, the snapshot format is tried and true, but inefficient if you
> want to take multiple snapshots.
>
> Looking at dmsetup, there doesn't seem like there would be an issue taking
> a snapshot of a snapshot, but it is disabled in the tools.
>
> I guess my question is this: Can it be done? Is there any reason why it
> should not work?
>

1. Noone has written support for it in lvm2. And AFAICK noone is working on this.

2. Since the activation order of old snapshot is not as simple as with other 
targets, it's currently not so easy to add support for into lvm deptree
resolver.

Zdenek

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[markw]

> Dne 8.7.2013 18:05, markw@mohawksoft.com napsal(a):
>> I'm trying to get a snapshot of a snapshot. I know the new thin
>> provisioning stuff supports this, but I'm not sure the release state is
>> something I can use at a customer.
>>
>> Using LVM2, the snapshot format is tried and true, but inefficient if
>> you
>> want to take multiple snapshots.
>>
>> Looking at dmsetup, there doesn't seem like there would be an issue
>> taking
>> a snapshot of a snapshot, but it is disabled in the tools.
>>
>> I guess my question is this: Can it be done? Is there any reason why it
>> should not work?
>>
>
> 1. Noone has written support for it in lvm2. And AFAICK noone is working
> on this.

I thought I saw a patch for this in the 2010 time frame.

>
> 2. Since the activation order of old snapshot is not as simple as with
> other
> targets, it's currently not so easy to add support for into lvm deptree
> resolver.

In my case I would, more or less, be maintaining a lot of the
interdependency information for the chain of shapshots and it would
managed by a daemon being controlled by an application, not really subject
to the whims of a sys admin.


Is there a stable and supported way of getting this functionality? FYI:
I'm trying to use RHEL as the basis for this and I think that's an old
2.6.x kernel.


>
> Zdenek
>
>
>
>

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[Zdenek Kabelac]

Dne 8.7.2013 19:57, markw@mohawksoft.com napsal(a):
>> Dne 8.7.2013 18:05, markw@mohawksoft.com napsal(a):
>>> I'm trying to get a snapshot of a snapshot. I know the new thin
>>> provisioning stuff supports this, but I'm not sure the release state is
>>> something I can use at a customer.
>>>
>>> Using LVM2, the snapshot format is tried and true, but inefficient if
>>> you
>>> want to take multiple snapshots.
>>>
>>> Looking at dmsetup, there doesn't seem like there would be an issue
>>> taking
>>> a snapshot of a snapshot, but it is disabled in the tools.
>>>
>>> I guess my question is this: Can it be done? Is there any reason why it
>>> should not work?
>>>
>>
>> 1. Noone has written support for it in lvm2. And AFAICK noone is working
>> on this.
>
> I thought I saw a patch for this in the 2010 time frame.
>

I'm not aware of any such support

>>
>> 2. Since the activation order of old snapshot is not as simple as with
>> other
>> targets, it's currently not so easy to add support for into lvm deptree
>> resolver.
>
> In my case I would, more or less, be maintaining a lot of the
> interdependency information for the chain of shapshots and it would
> managed by a daemon being controlled by an application, not really subject
> to the whims of a sys admin.


Beware the old snaps are meant to be used as a 'temporary  objects'
not something you want to keep for a long time - it's degrading write 
performance of origin device and there is also no safe resize support -
i.e. when you overfill snapshot - it's invalidated.  If you would
have used such snapshot as a base origin for a chain with not enough space - 
then whole chain is lost.

> Is there a stable and supported way of getting this functionality? FYI:
> I'm trying to use RHEL as the basis for this and I think that's an old
> 2.6.x kernel.

Kernel RHEL6 2.6 is a heavily modified version - so it has a lot
of features of 3.X kernel and dm support is basically back ported to very 
latest version - so you have there thin provisioning support.

If you want to use snaps of snaps - I'd really recommend to use
thins and report problems rather there - then trying to old snaps.
They were not designed for this use-case.

Zdenek

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[markw]

One more annoying question, if you have the patience.

Suppose I create a thin provisioned volume, say disk0.

I take a snapshot of disk0 and name it disk0_snap0
After a number of changes I then take another snapshot, and call it
disk0_snap1

Is the thin provisioning stuff smart enough to realize that disk0_snap0
should be rewired to reference disk0_snap1 as its origin? It would look
like this:

disk0 -> disk0_snap1 -> disk0_snap0

What I would like to see is something like this:

disk0 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0

Create disk0_snap3

disk0 -> disk0_snap3 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0

Where all writes to disk0 result in CoW to disk0_snap3 and the remaining
snaps remain unchanged.

Here is the kicker, I want to perform a differential backup on disk0_snap3
and disk0_snap2.  Suppose I already recreated disk0_snap2 on some server.
I now want to update it to disk0_snap3. I need to get a block list from
disk0_snap2 and disk0_snap3, then generate a list of blocks needed to
permute snap2 to snap3.

Any info?


> Dne 8.7.2013 19:57, markw@mohawksoft.com napsal(a):
>>> Dne 8.7.2013 18:05, markw@mohawksoft.com napsal(a):
>>>> I'm trying to get a snapshot of a snapshot. I know the new thin
>>>> provisioning stuff supports this, but I'm not sure the release state
>>>> is
>>>> something I can use at a customer.
>>>>
>>>> Using LVM2, the snapshot format is tried and true, but inefficient if
>>>> you
>>>> want to take multiple snapshots.
>>>>
>>>> Looking at dmsetup, there doesn't seem like there would be an issue
>>>> taking
>>>> a snapshot of a snapshot, but it is disabled in the tools.
>>>>
>>>> I guess my question is this: Can it be done? Is there any reason why
>>>> it
>>>> should not work?
>>>>
>>>
>>> 1. Noone has written support for it in lvm2. And AFAICK noone is
>>> working
>>> on this.
>>
>> I thought I saw a patch for this in the 2010 time frame.
>>
>
> I'm not aware of any such support
>
>>>
>>> 2. Since the activation order of old snapshot is not as simple as with
>>> other
>>> targets, it's currently not so easy to add support for into lvm deptree
>>> resolver.
>>
>> In my case I would, more or less, be maintaining a lot of the
>> interdependency information for the chain of shapshots and it would
>> managed by a daemon being controlled by an application, not really
>> subject
>> to the whims of a sys admin.
>
>
> Beware the old snaps are meant to be used as a 'temporary  objects'
> not something you want to keep for a long time - it's degrading write
> performance of origin device and there is also no safe resize support -
> i.e. when you overfill snapshot - it's invalidated.  If you would
> have used such snapshot as a base origin for a chain with not enough space
> -
> then whole chain is lost.
>
>> Is there a stable and supported way of getting this functionality? FYI:
>> I'm trying to use RHEL as the basis for this and I think that's an old
>> 2.6.x kernel.
>
> Kernel RHEL6 2.6 is a heavily modified version - so it has a lot
> of features of 3.X kernel and dm support is basically back ported to very
> latest version - so you have there thin provisioning support.
>
> If you want to use snaps of snaps - I'd really recommend to use
> thins and report problems rather there - then trying to old snaps.
> They were not designed for this use-case.
>
> Zdenek
>
>
>

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[Zdenek Kabelac]

Dne 10.7.2013 05:45, markw@mohawksoft.com napsal(a):
> One more annoying question, if you have the patience.
>
> Suppose I create a thin provisioned volume, say disk0.
>
> I take a snapshot of disk0 and name it disk0_snap0
> After a number of changes I then take another snapshot, and call it
> disk0_snap1
>
> Is the thin provisioning stuff smart enough to realize that disk0_snap0
> should be rewired to reference disk0_snap1 as its origin? It would look
> like this:
>
> disk0 -> disk0_snap1 -> disk0_snap0
>
> What I would like to see is something like this:
>
> disk0 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0
>
> Create disk0_snap3
>
> disk0 -> disk0_snap3 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0
>
> Where all writes to disk0 result in CoW to disk0_snap3 and the remaining
> snaps remain unchanged.
>
> Here is the kicker, I want to perform a differential backup on disk0_snap3
> and disk0_snap2.  Suppose I already recreated disk0_snap2 on some server.
> I now want to update it to disk0_snap3. I need to get a block list from
> disk0_snap2 and disk0_snap3, then generate a list of blocks needed to
> permute snap2 to snap3.
>
> Any info?
>

Yes - differential snapshot will be supported through thin provisioning
target - where you will be able to make a simple diff just by reading
metadata - it will be essential piece of replication.

AFAIK Joe has this in plan for some time - and there was even some 
announcement from some third-party developer to support this.

There is not going to be any upstream support for doing this with
old-snaps in foreseeable future.

Also keep in mind your idea of using old-snap of snap of snap would be very 
slow and fragile to use.

Zdenek

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[markw]

> Dne 10.7.2013 05:45, markw@mohawksoft.com napsal(a):
>> One more annoying question, if you have the patience.
>>
>> Suppose I create a thin provisioned volume, say disk0.
>>
>> I take a snapshot of disk0 and name it disk0_snap0
>> After a number of changes I then take another snapshot, and call it
>> disk0_snap1
>>
>> Is the thin provisioning stuff smart enough to realize that disk0_snap0
>> should be rewired to reference disk0_snap1 as its origin? It would look
>> like this:
>>
>> disk0 -> disk0_snap1 -> disk0_snap0
>>
>> What I would like to see is something like this:
>>
>> disk0 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0
>>
>> Create disk0_snap3
>>
>> disk0 -> disk0_snap3 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0
>>
>> Where all writes to disk0 result in CoW to disk0_snap3 and the remaining
>> snaps remain unchanged.
>>
>> Here is the kicker, I want to perform a differential backup on
>> disk0_snap3
>> and disk0_snap2.  Suppose I already recreated disk0_snap2 on some
>> server.
>> I now want to update it to disk0_snap3. I need to get a block list from
>> disk0_snap2 and disk0_snap3, then generate a list of blocks needed to
>> permute snap2 to snap3.
>>
>> Any info?
>>
>
> Yes - differential snapshot will be supported through thin provisioning
> target - where you will be able to make a simple diff just by reading
> metadata - it will be essential piece of replication.

Is any of this in place now?

>
> AFAIK Joe has this in plan for some time - and there was even some
> announcement from some third-party developer to support this.

Do you have a link?

>
> There is not going to be any upstream support for doing this with
> old-snaps in foreseeable future.
>
> Also keep in mind your idea of using old-snap of snap of snap would be
> very
> slow and fragile to use.

Well, the reason I am pursuing this.....

At a previous employment a few years back, I created a system using LVM2,
old style snapshots, and FUSE to create this functionality.

Using my previous example as a reference

disk0 would always have one live snapshot to maintain diffs, call it
disk0_snap_root.

When I wanted to save a point in time, I would take a new snapshot, say
disk0_snap_tmp

I would deactivate disk0_snap_root and save it off. It is important to
create disk0_snap_tmp BEFORE you deactivate disk0_snap_root, of course.

Then rename disk0_snap_tmp to disk0_snap_root.

The saved off snap file has all the changes from the time it was created
until the time the new snapshot is created.

I implemented all the volume chaining in FUSE and could present versions
of the device. The system even had its own virtual device management
system so that read/write copies could be made by looking back at the snap
files. Essentially, I had to write a lot of what device mapper already
does in FUSE.

You know the obvious limitations to this strategy, but still, it worked
pretty well.

I need to implement similar behavior at a new employer and really really
want not to use FUSE and rewrite all that crap again and am trying to see
how to get it done using the device mapper layer. Also this is going to be
for a product that is expected to ship to customers in the relatively near
term.

Any information or insight you can share would be much appreciated. I am
beginning to suspect that LVM will not be usable for the project.

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[Zdenek Kabelac]

Dne 10.7.2013 16:45, markw@mohawksoft.com napsal(a):
>> Dne 10.7.2013 05:45, markw@mohawksoft.com napsal(a):
>>> One more annoying question, if you have the patience.
>>>
>>> Suppose I create a thin provisioned volume, say disk0.
>>>
>>> I take a snapshot of disk0 and name it disk0_snap0
>>> After a number of changes I then take another snapshot, and call it
>>> disk0_snap1
>>>
>>> Is the thin provisioning stuff smart enough to realize that disk0_snap0
>>> should be rewired to reference disk0_snap1 as its origin? It would look
>>> like this:
>>>
>>> disk0 -> disk0_snap1 -> disk0_snap0
>>>
>>> What I would like to see is something like this:
>>>
>>> disk0 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0
>>>
>>> Create disk0_snap3
>>>
>>> disk0 -> disk0_snap3 -> disk0_snap2 -> disk0_snap1 -> disk0_snap0
>>>
>>> Where all writes to disk0 result in CoW to disk0_snap3 and the remaining
>>> snaps remain unchanged.
>>>
>>> Here is the kicker, I want to perform a differential backup on
>>> disk0_snap3
>>> and disk0_snap2.  Suppose I already recreated disk0_snap2 on some
>>> server.
>>> I now want to update it to disk0_snap3. I need to get a block list from
>>> disk0_snap2 and disk0_snap3, then generate a list of blocks needed to
>>> permute snap2 to snap3.
>>>
>>> Any info?
>>>
>>
>> Yes - differential snapshot will be supported through thin provisioning
>> target - where you will be able to make a simple diff just by reading
>> metadata - it will be essential piece of replication.
>
> Is any of this in place now?

Surely not - if it would be in place - I'd suggest which command you should use :)

>
>>
>> AFAIK Joe has this in plan for some time - and there was even some
>> announcement from some third-party developer to support this.
>
> Do you have a link?


https://www.redhat.com/archives/dm-devel/2013-July/msg00005.html

>>
>> There is not going to be any upstream support for doing this with
>> old-snaps in foreseeable future.
>>
>> Also keep in mind your idea of using old-snap of snap of snap would be
>> very
>> slow and fragile to use.
>
> Well, the reason I am pursuing this.....
>
> At a previous employment a few years back, I created a system using LVM2,
> old style snapshots, and FUSE to create this functionality.
>
> Using my previous example as a reference
>
> disk0 would always have one live snapshot to maintain diffs, call it
> disk0_snap_root.

Using chain of old-snaps is just way too heavy for anything - you really need 
to use system which is not copying blocks


> I need to implement similar behavior at a new employer and really really
> want not to use FUSE and rewrite all that crap again and am trying to see
> how to get it done using the device mapper layer. Also this is going to be
> for a product that is expected to ship to customers in the relatively near
> term.
>
> Any information or insight you can share would be much appreciated. I am
> beginning to suspect that LVM will not be usable for the project.

As I said -  btrfs has some kind of functionality you are looking for,
Or you may start to help with lvm project...
(Or thin-provisioning tools in this case)

Zdenek

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[markw]

> Dne 10.7.2013 16:45, markw@mohawksoft.com napsal(a):
>>> Dne 10.7.2013 05:45, markw@mohawksoft.com napsal(a):
>>> Yes - differential snapshot will be supported through thin provisioning
>>> target - where you will be able to make a simple diff just by reading
>>> metadata - it will be essential piece of replication.
>>
>> Is any of this in place now?
>
> Surely not - if it would be in place - I'd suggest which command you
> should use :)

Ahh, ok.

>
>>
>>>
>>> AFAIK Joe has this in plan for some time - and there was even some
>>> announcement from some third-party developer to support this.
>>
>> Do you have a link?
>
>
> https://www.redhat.com/archives/dm-devel/2013-July/msg00005.html
>
>>>
>>> There is not going to be any upstream support for doing this with
>>> old-snaps in foreseeable future.
>>>
>>> Also keep in mind your idea of using old-snap of snap of snap would be
>>> very
>>> slow and fragile to use.
>>
>> Well, the reason I am pursuing this.....
>>
>> At a previous employment a few years back, I created a system using
>> LVM2,
>> old style snapshots, and FUSE to create this functionality.
>>
>> Using my previous example as a reference
>>
>> disk0 would always have one live snapshot to maintain diffs, call it
>> disk0_snap_root.
>
> Using chain of old-snaps is just way too heavy for anything - you really
> need
> to use system which is not copying blocks

Well, yes, grabbing a new block and using it in the original volume would
be more efficient than copying old data to (n) snapshots. However, in the
system I wrote I only did the copy to one snapshot. All previous snaps
referred to the next newer snap in the chain.

>
>
>> I need to implement similar behavior at a new employer and really really
>> want not to use FUSE and rewrite all that crap again and am trying to
>> see
>> how to get it done using the device mapper layer. Also this is going to
>> be
>> for a product that is expected to ship to customers in the relatively
>> near
>> term.
>>
>> Any information or insight you can share would be much appreciated. I am
>> beginning to suspect that LVM will not be usable for the project.
>
> As I said -  btrfs has some kind of functionality you are looking for,
> Or you may start to help with lvm project...
> (Or thin-provisioning tools in this case)

Where's a good place to look to start?

>
> Zdenek
>
>

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[markw]

> Dne 10.7.2013 16:45, markw@mohawksoft.com napsal(a):

> As I said -  btrfs has some kind of functionality you are looking for,
> Or you may start to help with lvm project...
> (Or thin-provisioning tools in this case)
>
> Zdenek

That link you sent was good, I am looking in to it, it may provide a large
amount of help.

One last thing.... :-)

Now imagine this:

create disk0

Activity on disk0

Take a snapshot
disk0 -> disk0_snap0

Activity on disk0

take a snapshot
disk0 -> disk0_snap1

(We will now use the differential between snap1 and snap0 to represent
what needs to be backed up from snap1)

This leaves us with something that looks like this:

disk0 --> disk0_snap0
       -> disk0_snap1

Will we need to remap the parentage of disk0_snap0 so it looks something
like this?

disk0 -> disk0_snap1 -> disk0_snap0
(Is this possible with dmsetup?)

Or could we have many (order of hundreds or thousands) of snaps directly
on disk0 and not impact performance?

Re: [linux-lvm] Snapshots of snapshots are not supported yet

[Zdenek Kabelac]

Dne 10.7.2013 22:12, markw@mohawksoft.com napsal(a):
>> Dne 10.7.2013 16:45, markw@mohawksoft.com napsal(a):
>
>> As I said -  btrfs has some kind of functionality you are looking for,
>> Or you may start to help with lvm project...
>> (Or thin-provisioning tools in this case)
>>
>> Zdenek
>
> That link you sent was good, I am looking in to it, it may provide a large
> amount of help.
>
> One last thing.... :-)
>
> Now imagine this:
>
> create disk0
>
> Activity on disk0
>
> Take a snapshot
> disk0 -> disk0_snap0
>
> Activity on disk0
>
> take a snapshot
> disk0 -> disk0_snap1
>
> (We will now use the differential between snap1 and snap0 to represent
> what needs to be backed up from snap1)
>
> This leaves us with something that looks like this:
>
> disk0 --> disk0_snap0
>         -> disk0_snap1
>
> Will we need to remap the parentage of disk0_snap0 so it looks something
> like this?
>
> disk0 -> disk0_snap1 -> disk0_snap0
> (Is this possible with dmsetup?)
>
> Or could we have many (order of hundreds or thousands) of snaps directly
> on disk0 and not impact performance?
>

Thin provisioning is written to support this. Note - you do not need to have 
snapshots present as active devices - i.e. you could snapshot your device 
every hour.

If you want to discover diff between _snap0 and _snap1 - you have number of 
choices - if you want to know 'filesystem' diff or block device diff.
Block device  diff could be detected by doing a diff in metadata between
btree for _snap0 and _snap1.

Also there are projects like 'snapper' for filesystem diffs.

Zdenek