[linux-lvm] 'snapshot' target still experimental :-(

[linux-lvm] 'snapshot' target still experimental :-(

[Kevin P. Fleming]

I noticed that this is still the case... but I want to use it :-)

I need to set up simple snapshot/rsync backups of XFS filesystems, so 
I'm wondering what the current thinking in the community is. Is it 
reliable enough to trust not to crash my server or do other nasty things?

RE: [linux-lvm] 'snapshot' target still experimental :-(

[Steffen Plotner]

Hi,

With FC4 for example I cannot remove a snapshot while the underlying
logical volume is under heavy I/O.  Once all I/O stops I can remove the
snapshot. Doing a lvremove will hang.

Steffen 

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
On Behalf Of Kevin P. Fleming
Sent: Wednesday, February 01, 2006 11:38 AM
To: LVM general discussion and development
Subject: [linux-lvm] 'snapshot' target still experimental :-(

I noticed that this is still the case... but I want to use it :-)

I need to set up simple snapshot/rsync backups of XFS filesystems, so
I'm wondering what the current thinking in the community is. Is it
reliable enough to trust not to crash my server or do other nasty
things?

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] 'snapshot' target still experimental :-(

[Kevin P. Fleming]

Steffen Plotner wrote:

> With FC4 for example I cannot remove a snapshot while the underlying
> logical volume is under heavy I/O.  Once all I/O stops I can remove the
> snapshot. Doing a lvremove will hang.

Interesting.

I am using this on a Gentoo system using vanilla kernel 2.6.15.1, and my 
volumes don't experience 'heavy I/O' at the times I make backups, so it 
shouldn't be too bad.

Thanks for the info :-)

RE: [linux-lvm] 'snapshot' target still experimental :-(

[Steffen Plotner]

Hi,

Ah, I see, reading is never the problem. Do a simple

cd to a directory on an lv that has an active snapshot

dd if=/dev/zero of=test bs=1M count=10000

And see how hard it is to remove a live active snapshot. You will find
that lvremove hangs, you can press Ctrl-C, then try lvscan and it hangs
- maybe 2.6.15.1 has fixed this?  Would you let us know?

Thanks.

Steffen

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
On Behalf Of Kevin P. Fleming
Sent: Wednesday, February 01, 2006 12:56 PM
To: LVM general discussion and development
Subject: Re: [linux-lvm] 'snapshot' target still experimental :-(

Steffen Plotner wrote:

> With FC4 for example I cannot remove a snapshot while the underlying 
> logical volume is under heavy I/O.  Once all I/O stops I can remove 
> the snapshot. Doing a lvremove will hang.

Interesting.

I am using this on a Gentoo system using vanilla kernel 2.6.15.1, and my
volumes don't experience 'heavy I/O' at the times I make backups, so it
shouldn't be too bad.

Thanks for the info :-)

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] 'snapshot' target still experimental :-(

[Alasdair G Kergon]

On Wed, Feb 01, 2006 at 01:07:20PM -0500, Steffen Plotner wrote:
> cd to a directory on an lv that has an active snapshot
> dd if=/dev/zero of=test bs=1M count=10000
 
> And see how hard it is to remove a live active snapshot. 

Strange - I don't recall anyone reporting that problem before.

I've just tried it here and the machine locked up:-(

Alasdair
-- 
agk@redhat.com

RE: [linux-lvm] 'snapshot' target still experimental :-(

[Steffen Plotner]

Hi,

I apologize for looking up your machine. I have reported this problem
sometime ago in November of last year.  

https://www.redhat.com/archives/linux-lvm/2005-November/msg00044.html

We currently deployed an iscsi-target (from sourceforge) server on FC4
which functions well, until we utilize snapshots. The way I currently
remove snapshots is to stop the iscsi-target daemon for 5 seconds, to
let everything settle, then remove the snap, the restart the
iscsi-target and all is ok (of course this is not how I would like to do
things.. :).

I have tried several patches dm and lvm in late December and they did
not help. I have specifics at my house. I will email those later on.
Thank you for responding to this issue.

Steffen

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
On Behalf Of Alasdair G Kergon
Sent: Wednesday, February 01, 2006 1:31 PM
To: LVM general discussion and development
Subject: Re: [linux-lvm] 'snapshot' target still experimental :-(

On Wed, Feb 01, 2006 at 01:07:20PM -0500, Steffen Plotner wrote:
> cd to a directory on an lv that has an active snapshot dd if=/dev/zero

> of=test bs=1M count=10000
 
> And see how hard it is to remove a live active snapshot. 

Strange - I don't recall anyone reporting that problem before.

I've just tried it here and the machine locked up:-(

Alasdair
--
agk@redhat.com

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

RE: [linux-lvm] 'snapshot' target still experimental :-(

[Ming Zhang]

i can confirm this. iet runs well with heavy io until you create many
snapshots. lvresize will be slow as well, though not hang. i think that
is because the outstanding bio? not quite since i faced that several
months ago.

ps, not lvm related, but if u stop iet for 5 seconds. i guess iscsi ini
will be unhappy? :P we can discuss in iet list.

ming

 
On Wed, 2006-02-01 at 13:39 -0500, Steffen Plotner wrote:
> Hi,
> 
> I apologize for looking up your machine. I have reported this problem
> sometime ago in November of last year.  
> 
> https://www.redhat.com/archives/linux-lvm/2005-November/msg00044.html
> 
> We currently deployed an iscsi-target (from sourceforge) server on FC4
> which functions well, until we utilize snapshots. The way I currently
> remove snapshots is to stop the iscsi-target daemon for 5 seconds, to
> let everything settle, then remove the snap, the restart the
> iscsi-target and all is ok (of course this is not how I would like to do
> things.. :).
> 
> I have tried several patches dm and lvm in late December and they did
> not help. I have specifics at my house. I will email those later on.
> Thank you for responding to this issue.
> 
> Steffen
> 
> -----Original Message-----
> From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
> On Behalf Of Alasdair G Kergon
> Sent: Wednesday, February 01, 2006 1:31 PM
> To: LVM general discussion and development
> Subject: Re: [linux-lvm] 'snapshot' target still experimental :-(
> 
> On Wed, Feb 01, 2006 at 01:07:20PM -0500, Steffen Plotner wrote:
> > cd to a directory on an lv that has an active snapshot dd if=/dev/zero
> 
> > of=test bs=1M count=10000
>  
> > And see how hard it is to remove a live active snapshot. 
> 
> Strange - I don't recall anyone reporting that problem before.
> 
> I've just tried it here and the machine locked up:-(
> 
> Alasdair
> --
> agk@redhat.com
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] 'snapshot' target still experimental :-(

[Dan Stromberg]

On Wed, 2006-02-01 at 10:37 -0600, Kevin P. Fleming wrote:
> I noticed that this is still the case... but I want to use it :-)
> 
> I need to set up simple snapshot/rsync backups of XFS filesystems, so 
> I'm wondering what the current thinking in the community is. Is it 
> reliable enough to trust not to crash my server or do other nasty things?
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

I've been doing my home backups via rsync to an xfs via cron, and using
rsync from whatever (mostly ext3 and UFS) to ext3 for one-off backups
prior to making large administrative changes to systems.  It seems to be
working well so far, and also seems to give a very nice combination of
single-file recovery ease and disaster recovery ease, which isn't
astonishingly common.

It just maintains a series of hardlink trees, so any time a file is
deleted or added, the actual disk usage increase is only due to the
changes.

And, every backup after the first is both a fullsave and an incremental
in a sense - but the network usage is far, far less than your typical
fullsave.

http://dcs.nac.uci.edu/~strombrg/Backup.remote.html

Re: [linux-lvm] 'snapshot' target still experimental :-(

[Brian J. Murrell]

[-- Attachment #1: Type: text/plain, Size: 992 bytes --]

On Wed, 2006-02-01 at 13:29 -0800, Dan Stromberg wrote:
> It just maintains a series of hardlink trees, so any time a file is
> deleted or added, the actual disk usage increase is only due to the
> changes.

Yes, this is quite a popular technique.  To be clear though, a change in
a file does not consume just the amount of the change in the file on the
backup target, but it consumes the entire size of the new file.  A
not-so-insignificant amount for very large files.  This is where a
(block level or filesystem level) snapshotting scheme would excel as it
would likely only consume an amount of space rounded up to the next
"unit" size more even for changes within a file.

Indeed (and to keep quite on topic), perhaps rather than hardlink trees,
LVM snapshots would be even more space efficient.  Maybe that is what
this thread has been about.  I just jumped in.  Apologies if it was.

b.

-- 
My other computer is your Microsoft Windows server.

Brian J. Murrell

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] 'snapshot' target still experimental :-(

[Kevin P. Fleming]

Brian J. Murrell wrote:

> Indeed (and to keep quite on topic), perhaps rather than hardlink trees,
> LVM snapshots would be even more space efficient.  Maybe that is what
> this thread has been about.  I just jumped in.  Apologies if it was.

No, I'm using snapshots to make rsync backups onto removable drives (so 
the hardlink tree method won't work for me either).

Re: [linux-lvm] 'snapshot' target still experimental :-(

[Ming Zhang]

On Wed, 2006-02-01 at 16:51 -0500, Brian J. Murrell wrote:
> On Wed, 2006-02-01 at 13:29 -0800, Dan Stromberg wrote:
> > It just maintains a series of hardlink trees, so any time a file is
> > deleted or added, the actual disk usage increase is only due to the
> > changes.
> 
> Yes, this is quite a popular technique.  To be clear though, a change in
> a file does not consume just the amount of the change in the file on the
> backup target, but it consumes the entire size of the new file.  A
> not-so-insignificant amount for very large files.  This is where a
> (block level or filesystem level) snapshotting scheme would excel as it
> would likely only consume an amount of space rounded up to the next
> "unit" size more even for changes within a file.

it depends.

assume u have a huge text file. u change 1 char, the snapshot is useful.
u add one line at the beginning, the snapshot is useless here.

the offset shift.

some delta tech will be useful here.


> 
> Indeed (and to keep quite on topic), perhaps rather than hardlink trees,
> LVM snapshots would be even more space efficient.  Maybe that is what
> this thread has been about.  I just jumped in.  Apologies if it was.
> 
> b.
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

RE: [linux-lvm] 'snapshot' target still experimental :-(

[Steffen Plotner]

[-- Attachment #1: Type: text/plain, Size: 3711 bytes --]

Hello,
 
On an experimental system I changed the device mapper from 1.01.02 to 1.02.02 and lvm from 2.01.08 to 2.02.01 and upgraded on FC4 from 2.6.13 to 2.6.14-1.1653_FC4. Create a LV, create a snapshot, dd if=/dev/zero of=test bs=1M count=10000 on the LV (not the snapshot), then try to remove the snapshot and the kernel faults. 
 
I suspect that the outstanding block I/Os are probably the issue. You ask about the 5 second outage of ietd? the iscsi initiators see the event and reconnect - as long as the outage is not more than 1 minute (windows does not like it, linux is very resilient about it).
 
Steffen
 
________________________________ 
Steffen Plotner 
Systems Administrator/Programmer 
Systems & Networking 
Amherst College 
PO BOX 5000 
Amherst, MA 01002-5000 
Tel (413) 542-2348 
Fax (413) 542-2626 
Email: swplotner@amherst.edu 
________________________________ 

________________________________

From: linux-lvm-bounces@redhat.com on behalf of Ming Zhang
Sent: Wed 2/1/2006 3:54 PM
To: LVM general discussion and development
Subject: RE: [linux-lvm] 'snapshot' target still experimental :-(



i can confirm this. iet runs well with heavy io until you create many
snapshots. lvresize will be slow as well, though not hang. i think that
is because the outstanding bio? not quite since i faced that several
months ago.

ps, not lvm related, but if u stop iet for 5 seconds. i guess iscsi ini
will be unhappy? :P we can discuss in iet list.

ming


On Wed, 2006-02-01 at 13:39 -0500, Steffen Plotner wrote:
> Hi,
>
> I apologize for looking up your machine. I have reported this problem
> sometime ago in November of last year. 
>
> https://www.redhat.com/archives/linux-lvm/2005-November/msg00044.html
>
> We currently deployed an iscsi-target (from sourceforge) server on FC4
> which functions well, until we utilize snapshots. The way I currently
> remove snapshots is to stop the iscsi-target daemon for 5 seconds, to
> let everything settle, then remove the snap, the restart the
> iscsi-target and all is ok (of course this is not how I would like to do
> things.. :).
>
> I have tried several patches dm and lvm in late December and they did
> not help. I have specifics at my house. I will email those later on.
> Thank you for responding to this issue.
>
> Steffen
>
> -----Original Message-----
> From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
> On Behalf Of Alasdair G Kergon
> Sent: Wednesday, February 01, 2006 1:31 PM
> To: LVM general discussion and development
> Subject: Re: [linux-lvm] 'snapshot' target still experimental :-(
>
> On Wed, Feb 01, 2006 at 01:07:20PM -0500, Steffen Plotner wrote:
> > cd to a directory on an lv that has an active snapshot dd if=/dev/zero
>
> > of=test bs=1M count=10000
> 
> > And see how hard it is to remove a live active snapshot.
>
> Strange - I don't recall anyone reporting that problem before.
>
> I've just tried it here and the machine locked up:-(
>
> Alasdair
> --
> agk@redhat.com
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/



[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 7433 bytes --]

Re: [linux-lvm] 'snapshot' target still experimental :-(

[Imre Gergely]

hi

i'm currently doing the following: every night (when there's not much i/o on
the server), i create a snapshot of one LV. the next day i remove this
snapshot, and create another one.
this worked without problems so far. a couple of months back i tried to create
more than one snapshot of one LV, and it didn't work out. it hang, got a kernel
panic, whatever.

i would like to try again with multiple snapshots (i don't plan to use those
snapshots, it's only for backup, if something gets deleted from the original
LV, i mount the last snapshot read-only, and get the files from it). has there
anything changed since?

Steffen Plotner wrote:
> Hello,
>  
> On an experimental system I changed the device mapper from 1.01.02 to 1.02.02 and lvm from 2.01.08 to 2.02.01 and upgraded on FC4 from 2.6.13 to 2.6.14-1.1653_FC4. Create a LV, create a snapshot, dd if=/dev/zero of=test bs=1M count=10000 on the LV (not the snapshot), then try to remove the snapshot and the kernel faults. 
>  
> I suspect that the outstanding block I/Os are probably the issue. You ask about the 5 second outage of ietd? the iscsi initiators see the event and reconnect - as long as the outage is not more than 1 minute (windows does not like it, linux is very resilient about it).
>  
> Steffen
>  
> ________________________________ 
> Steffen Plotner 
> Systems Administrator/Programmer 
> Systems & Networking 
> Amherst College 
> PO BOX 5000 
> Amherst, MA 01002-5000 
> Tel (413) 542-2348 
> Fax (413) 542-2626 
> Email: swplotner@amherst.edu 
> ________________________________ 
> 
> ________________________________
> 
> From: linux-lvm-bounces@redhat.com on behalf of Ming Zhang
> Sent: Wed 2/1/2006 3:54 PM
> To: LVM general discussion and development
> Subject: RE: [linux-lvm] 'snapshot' target still experimental :-(
> 
> 
> 
> i can confirm this. iet runs well with heavy io until you create many
> snapshots. lvresize will be slow as well, though not hang. i think that
> is because the outstanding bio? not quite since i faced that several
> months ago.
> 
> ps, not lvm related, but if u stop iet for 5 seconds. i guess iscsi ini
> will be unhappy? :P we can discuss in iet list.
> 
> ming
> 
> 
> On Wed, 2006-02-01 at 13:39 -0500, Steffen Plotner wrote:
>> Hi,
>>
>> I apologize for looking up your machine. I have reported this problem
>> sometime ago in November of last year. 
>>
>> https://www.redhat.com/archives/linux-lvm/2005-November/msg00044.html
>>
>> We currently deployed an iscsi-target (from sourceforge) server on FC4
>> which functions well, until we utilize snapshots. The way I currently
>> remove snapshots is to stop the iscsi-target daemon for 5 seconds, to
>> let everything settle, then remove the snap, the restart the
>> iscsi-target and all is ok (of course this is not how I would like to do
>> things.. :).
>>
>> I have tried several patches dm and lvm in late December and they did
>> not help. I have specifics at my house. I will email those later on.
>> Thank you for responding to this issue.
>>
>> Steffen
>>
>> -----Original Message-----
>> From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
>> On Behalf Of Alasdair G Kergon
>> Sent: Wednesday, February 01, 2006 1:31 PM
>> To: LVM general discussion and development
>> Subject: Re: [linux-lvm] 'snapshot' target still experimental :-(
>>
>> On Wed, Feb 01, 2006 at 01:07:20PM -0500, Steffen Plotner wrote:
>>> cd to a directory on an lv that has an active snapshot dd if=/dev/zero
>>> of=test bs=1M count=10000
>>> And see how hard it is to remove a live active snapshot.
>> Strange - I don't recall anyone reporting that problem before.
>>
>> I've just tried it here and the machine locked up:-(
>>
>> Alasdair
>> --
>> agk@redhat.com
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/