[linux-lvm] What happens with full snapshots

[linux-lvm] What happens with full snapshots

[Harald Heigl]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

Hello,

I'm relatively new to lvm, I'm using Fedora for almost 3 years, but up to
now I just set up lvm on install, formatted my ext3/ext4 and it just works.

Now I want to go somehow deeper playing around with snapshots.

 

I know you should size your snapshot volume to a size that can hold all the
changes. As I see a snap lv only contains the "original" datachunks and the
original lv is in a normal state as if there were no snapshot. 

 

My questions:

1.       What happens if the snap gets full (e.g. someone copies large data
from one lv to a snapshotted lv or something other you didn't expect), some
sites say it original and snapshot gets deactivated, some sites say the snap
lv gets invalidated. What does this mean? Is my original volume just crap
afterwards or can I work on as usual (with the difference that nothing is
written on the snap anymore), or just need to remove the snap ..

2.       What happens if there is a problem with a snap? I've heard on
reboot the snapshots are lost. What if there is a power shortage, so the
snap is lost, you boot and the original lv just seeks for the snap? Are
there implications on this. Can you just mount such a original lv or do you
have to remove the (non existent) snap lv before (How can you remove a
snapshot that doesn't exist anymore)

 

That would be nice to know before I use snaps.

Thanks in advance,

Harald (Harry) Heigl

 


[-- Attachment #2: Type: text/html, Size: 4932 bytes --]

Re: [linux-lvm] What happens with full snapshots

[Ray Morris]

   The summary answer to the list of questions below
is "it works well".


> 1.       What happens if the snap gets full (e.g. someone copies  
> large data
> from one lv to a snapshotted lv or something other you didn't  
> expect), some
> sites say it original and snapshot gets deactivated, some sites say  
> the snap
> lv gets invalidated. What does this mean? Is my original volume just  
> crap
> afterwards or can I work on as usual (with the difference that  
> nothing is
> written on the snap anymore), or just need to remove the snap ..

    The original LV is unaffected.  The snapshot stops being valid.
This is because, as you said:

> I know you should size your snapshot volume to a size that can hold  
> all the
> changes. As I see a snap lv only contains the "original" datachunks  
> and the
> original lv is in a normal state as if there were no snapshot.

    That is correct, the snapshot contains the old data, while
the LV holds the current data.  Overfilling the snapshot will
simply mean no more old data is saved as it's changed, so the
snapshot is therefore invalid.


> 2.       What happens if there is a problem with a snap? I've heard on
> reboot the snapshots are lost. What if there is a power shortage, so  
> the
> snap is lost, you boot and the original lv just seeks for the snap?

    LVM snapshots are NOT lost on reboot.  That information may apply
to some other type of snapshot mechanism unrelated to LVM, or some
system could potentially be built on top of LVM which holds snapshots
in RAM, but in the normal use case snapshots are written to disk
just like any other data you might write to disk.

> What if there is a power shortage, so the snap is lost, you boot
> and the original lv just seeks for the snap?

    Just like anything else written to disk, a sudden loss of power
could result in some lost data.  Snapshots are not different from
your normal "file system on a partition" in this respect.  With LVM
or without, battery backups systems (UPS) are advised for critical
systems.

    LVM can be set to maintain a log of backups of the LVM metadata
in case you had a power failure or other major "oops" in the middle
of creating a snapshot or something like that.  The tool "vgcfgrestore"
can restore the LVM configuration to any of your last X valid
configurations.  I've used that several times when I've done
something stupid like deleting an important LVM, as I'm doing a
lot of weird stuff with LVM.
--
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php


On 02/12/2010 08:56:11 AM, Harald Heigl wrote:
> Hello,
> 
> I'm relatively new to lvm, I'm using Fedora for almost 3 years, but  
> up to
> now I just set up lvm on install, formatted my ext3/ext4 and it just  
> works.
> 
> Now I want to go somehow deeper playing around with snapshots.
> 
> 
> 
> I know you should size your snapshot volume to a size that can hold  
> all the
> changes. As I see a snap lv only contains the "original" datachunks  
> and the
> original lv is in a normal state as if there were no snapshot.
> 
> 
> 
> My questions:
> 
> 1.       What happens if the snap gets full (e.g. someone copies  
> large data
> from one lv to a snapshotted lv or something other you didn't  
> expect), some
> sites say it original and snapshot gets deactivated, some sites say  
> the snap
> lv gets invalidated. What does this mean? Is my original volume just  
> crap
> afterwards or can I work on as usual (with the difference that  
> nothing is
> written on the snap anymore), or just need to remove the snap ..
> 
> 2.       What happens if there is a problem with a snap? I've heard on
> reboot the snapshots are lost. What if there is a power shortage, so  
> the
> snap is lost, you boot and the original lv just seeks for the snap?  
> Are
> there implications on this. Can you just mount such a original lv or  
> do you
> have to remove the (non existent) snap lv before (How can you remove a
> snapshot that doesn't exist anymore)
> 
> 
> 
> That would be nice to know before I use snaps.
> 
> Thanks in advance,
> 
> Harald (Harry) Heigl
> 
> 
> 
> 

------quoted attachment------
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] What happens with full snapshots

[Harald Heigl]

>    The summary answer to the list of questions below
> is "it works well".

Thanks a lot for this very fast answer!

 
> > 2.       What happens if there is a problem with a snap? I've heard
> on
> > reboot the snapshots are lost. What if there is a power shortage, so
> > the
> > snap is lost, you boot and the original lv just seeks for the snap?
> 
>     LVM snapshots are NOT lost on reboot.  That information may apply
> to some other type of snapshot mechanism unrelated to LVM, or some
> system could potentially be built on top of LVM which holds snapshots
> in RAM, but in the normal use case snapshots are written to disk
> just like any other data you might write to disk.

Ok I just assume I read something wrong it was about lvm, can't find it
anymore, perhaps someone had a problem on reboot (what could happen when
working and experimenting with computers :-) )
But I also think about trying this:
http://linuxsoftware.co.nz/blog/2008/03/11/lvm-snapshot-with-no-free-diskspa
ce
Do you see any problems here (besides that this snapshot will be definitely
lost)? As said the "real changed data" will be on the disk, just the
snapshot is lost ...

>     LVM can be set to maintain a log of backups of the LVM metadata
> in case you had a power failure or other major "oops" in the middle
> of creating a snapshot or something like that.  The tool "vgcfgrestore"
> can restore the LVM configuration to any of your last X valid
> configurations.  I've used that several times when I've done
> something stupid like deleting an important LVM, as I'm doing a
> lot of weird stuff with LVM.

This point is interesting, fortunately I had no problem with lvm. I use
lvmdump -amd [file] to "backup" my lvm (with metadata), never tried to
restore. Not knowing vgcfgrestore before, from the manpage I assume using
vgcfgbackup is the right tool to backup your metadata, right? I thought of
using lvmdump on a regularly basis to get some dumps, I don't know If they
are useful if something happens. 

You said you deleted some lv? Then a vgcfgrestore will restore the vg and
all data (which was there before) will be accessible again?

Thanks again,
Harald

Re: [linux-lvm] What happens with full snapshots

[Ray Morris]

> But I also think about trying this:
> http://linuxsoftware.co.nz/blog/2008/03/11/lvm-snapshot-with-no-free-diskspa
> ce
> Do you see any problems here (besides that this snapshot will be  
> definitely
> lost)? As said the "real changed data" will be on the disk, just the
> snapshot is lost ...

   That's an interesting idea, to use ramdisk to hold the
snapshot while taking a backup if no disk space is available.
Indeed it seems to me that just the snapshot would be lost.
A little reconfiguring or vgcfgrestore might be needed to
activate the volume group without the missing PV if power
failed during that procedure.  As noted by someone who replied
there, using a loopback file on the filesystem within the LV
as temporary storage will be problematic a reboots because
it's a chickien and egg problem.  I did that one last week.
(I screw up often, and in varied ways, since I try all kinds
of wacky to things to see what's possible.)

> I use lvmdump -amd [file] to "backup" my lvm (with metadata),
> never tried to restore. Not knowing vgcfgrestore before, from
> the manpage I assume using vgcfgbackup is the right tool to
> backup your metadata, right?


yes.

# man lvmdump
        lvmdump - create lvm2 information dumps for diagnostic purposes

# man vgcfgbackup
        vgcfgbackup - backup volume group descriptor area

    It seems that vgcfgbackup is for creating backups and lvmdump
is for diagnostic purposes.  However, lvm automatically creates
backups before and after every change (by default?).  See:

man lvm.conf | grep -A 15 backup
and:

man vgcfgrestore

> You said you deleted some lv? Then a vgcfgrestore will restore
> the vg and all data (which was there before) will be accessible again?

    Yes, assuming those disk blocks haven't been allocated to
something else and overwritten.  I've used vgcfgrestore a number
of times since as I said I push the limits, try things without
exactly knowing what I'm doing, then writes scripts that will
hopefully automatically do the things I have no business doing.

    vgcfgrestore can also revert several changes back.

    Note - I am not an LVM developer or expert.  I've only used it
extensively for about a year, so anything I say could be slightly
off or even plain wrong.  I have, however, done everything I could
to break my LVM.  Let's shrink some LVs while their filesystems
are mounted, shall we?!  ;)  In fact, let's shrink the LV which
is serving as the disk for a running qemu guest, so that the system
using the LV doesn't even know that half of it has dissapeared!
I found it to be pretty idiot proof, provided that moments of doing
stupid, crazy things to break it are followed by intelligent attempts
to recover rather than panic.
--
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php


On 02/12/2010 10:41:21 AM, Harald Heigl wrote:
> >    The summary answer to the list of questions below
> > is "it works well".
> 
> Thanks a lot for this very fast answer!
> 
> 
> > > 2.       What happens if there is a problem with a snap? I've  
> heard
> > on
> > > reboot the snapshots are lost. What if there is a power shortage,  
> so
> > > the
> > > snap is lost, you boot and the original lv just seeks for the  
> snap?
> >
> >     LVM snapshots are NOT lost on reboot.  That information may  
> apply
> > to some other type of snapshot mechanism unrelated to LVM, or some
> > system could potentially be built on top of LVM which holds  
> snapshots
> > in RAM, but in the normal use case snapshots are written to disk
> > just like any other data you might write to disk.
> 
> Ok I just assume I read something wrong it was about lvm, can't find  
> it
> anymore, perhaps someone had a problem on reboot (what could happen  
> when
> working and experimenting with computers :-) )
> But I also think about trying this:
> http://linuxsoftware.co.nz/blog/2008/03/11/lvm-snapshot-with-no-free-diskspa
> ce
> Do you see any problems here (besides that this snapshot will be  
> definitely
> lost)? As said the "real changed data" will be on the disk, just the
> snapshot is lost ...
> 
> >     LVM can be set to maintain a log of backups of the LVM metadata
> > in case you had a power failure or other major "oops" in the middle
> > of creating a snapshot or something like that.  The tool  
> "vgcfgrestore"
> > can restore the LVM configuration to any of your last X valid
> > configurations.  I've used that several times when I've done
> > something stupid like deleting an important LVM, as I'm doing a
> > lot of weird stuff with LVM.
> 
> This point is interesting, fortunately I had no problem with lvm. I  
> use
> lvmdump -amd [file] to "backup" my lvm (with metadata), never tried to
> restore. Not knowing vgcfgrestore before, from the manpage I assume  
> using
> vgcfgbackup is the right tool to backup your metadata, right? I  
> thought of
> using lvmdump on a regularly basis to get some dumps, I don't know If  
> they
> are useful if something happens.
> 
> You said you deleted some lv? Then a vgcfgrestore will restore the vg  
> and
> all data (which was there before) will be accessible again?
> 
> Thanks again,
> Harald
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 
> 

Re: [linux-lvm] What happens with full snapshots

[Harald Heigl]

> > But I also think about trying this:
> > http://linuxsoftware.co.nz/blog/2008/03/11/lvm-snapshot-with-no-free-
> diskspa
> > ce
> > Do you see any problems here (besides that this snapshot will be
> > definitely
> > lost)? As said the "real changed data" will be on the disk, just the
> > snapshot is lost ...
> 
>    That's an interesting idea, to use ramdisk to hold the
> snapshot while taking a backup if no disk space is available....

Perhaps I give it a try, vgcfgrestore should do the trick or reactivating
the ramdisk should do it too. Without it the extended vg isn't complete that
seems logical to me. Depending on the type of data you will need a
rescuesystem, but I hold an pxeboot "rescuecd" on my router, so this is no
great problem.

>     It seems that vgcfgbackup is for creating backups and lvmdump
> is for diagnostic purposes.  However, lvm automatically creates
> backups before and after every change (by default?).  See:

I looked throught the lvm.conf and also the respecting man page:
archive - Whether or not tools  automatically  archive  existing
              metadata  into archive_dir before making changes to it.
backup - Whether or not tools  make  an  automatic  backup  into
              backup_dir  after  changing  metadata.
So for my fedora the default is 10 archives (minimum 30 days), archive and
backup activated.
So before a change an archive of the metadata seems to go into
/etc/lvm/archive and after the change new metadata goes into /etc/lvm/backup
A vgcfgbackup saves the files in /etc/lvm/backup too, if no parameter is
given.
So It seems a backup of /etc (what I just do regularly) should be fine.

>     Note - I am not an LVM developer or expert.
No problem, I use lvm just normal since fedora 8 (must be two years or so)
and never had a problem, did some little resizes of lv before, but nothing
more. Did understand the general concept of lvm but for know I wanted to dig
somehow deeper and explore all the possibilities I have. Doing also
sometimes weird stuff :-), not only with lvm. An extensive user like you is
just welcome. I think I have to do a little more tests. Thanks again.