[linux-lvm] LVM snap size weirdness...

[linux-lvm] LVM snap size weirdness...

[Barry, Christopher]

All,
	See output of lvdisplay below...

lvm> lvdisplay
  --- Logical volume ---
  LV Name                /dev/vg0/data
  VG Name                vg0
  LV UUID                fox8qK-dePL-xTi1-zfBO-xQzu-R5Ls-yZ5bgQ
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                200.00 GB
  Current LE             51200
  Segments               1
  Allocation             next free (default)
  Read ahead sectors     0
  Block device           254:0

  --- Logical volume ---
  LV Name                /dev/vg0/home
  VG Name                vg0
  LV UUID                C748g0-PYnB-zdtB-p1Eh-kYIU-1zXN-n1yKsb
  LV Write Access        read/write
  LV snapshot status     source of
                         /dev/vg0/homeSnap [active]
  LV Status              available
  # open                 1
  LV Size                200.00 GB
  Current LE             51200
  Segments               1
  Allocation             next free (default)
  Read ahead sectors     0
  Block device           254:1

  --- Logical volume ---
  LV Name                /dev/vg0/installs
  VG Name                vg0
  LV UUID                yKoPPg-BM0J-245e-BgCZ-kj3m-DVSo-5edFbH
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                200.00 GB
  Current LE             51200
  Segments               1
  Allocation             next free (default)
  Read ahead sectors     0
  Block device           254:2

  --- Logical volume ---
  LV Name                /dev/vg0/vortex
  VG Name                vg0
  LV UUID                slyYlF-tU41-rLmt-y7eC-GU41-iWD3-bsnFil
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                800.00 GB
  Current LE             204800
  Segments               1
  Allocation             next free (default)
  Read ahead sectors     0
  Block device           254:3

  --- Logical volume ---
  LV Name                /dev/vg0/homeSnap
  VG Name                vg0
  LV UUID                1eaqQh-DtCf-1FD7-iUwx-UHzR-aH0G-4X2BQv
  LV Write Access        read only
  LV snapshot status     active destination for /dev/vg0/home
  LV Status              available
  # open                 1
  LV Size                200.00 GB
  Current LE             51200
  Segments               1
  Snapshot chunk size    8.00 KB
  Allocated to snapshot  0.00%
  Allocation             next free (default)
  Read ahead sectors     0
  Block device           254:10

lvm>


After creating the snapshot vg0/homeSnap, mounting it, and viewing it, everything looked fine. But a df (below) shows something is amiss. Look at the size on the homeSnap vs the size of the DATA lv. For some reason they are identical, yet the snap contains the data from the HOME lv.


debian:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda2             942M  688M  207M  77% /
/dev/hda1              46M  4.6M   39M  11% /boot
/dev/hda5             9.2G  875M  7.9G  10% /usr
/dev/hda6             1.9G  155M  1.6G   9% /var
/dev/hda7             5.4G   33M  5.1G   1% /sys_data
/dev/vg0/data         200G   32G  169G  16% /mnt/lvm/data
/dev/vg0/home         200G  102G   99G  51% /mnt/lvm/home
/dev/vg0/installs     200G  5.9G  195G   3% /mnt/lvm/installs
/dev/vg0/vortex       800G  3.2M  800G   1% /mnt/lvm/vortex
/dev/vg0/homeSnap     200G   32G  169G  16% /mnt/lvm/home-snap


I have dumped an ls of both the snap lv and the home lv and diffed them - no differences.
Why/how can df be confused?




Thanks for your help,
Christopher

Re: [linux-lvm] LVM snap size weirdness...

[Joe Thornber]

On Thursday, March 20, 2003, at 08:00 PM, Barry, Christopher wrote:

> I have dumped an ls of both the snap lv and the home lv and diffed 
> them - no differences.
> Why/how can df be confused?

It's magic ;)  Think of the filesystem that df is looking at as a 
combination of the original volume with some modifications held in the 
snapshot volume.

- Joe

RE: [linux-lvm] LVM snap size weirdness...

[Barry, Christopher]

Joe (the Magician),
	Thanks, for the insight on exactly what snapshots are doing. I was unaware that snapshots were updated by deltas (or, more precisely, only are the deltas) - I thought they were a static view of the volume at 'x' point in time. Guess that's why it only took a few secs to make!

Another question:

	When a backup program grabs files from the snapshot volume, is it subject to open file issues that are normally a problem during backups?

--
Christopher Barry
Manager of Information Systems
InfiniCon Systems
http://www.infiniconsys.com


-----Original Message-----
From: Joe Thornber [mailto:joe@fib011235813.fsnet.co.uk]
Sent: Thursday, March 20, 2003 3:31 PM
To: linux-lvm@sistina.com
Subject: Re: [linux-lvm] LVM snap size weirdness...



On Thursday, March 20, 2003, at 08:00 PM, Barry, Christopher wrote:

> I have dumped an ls of both the snap lv and the home lv and diffed 
> them - no differences.
> Why/how can df be confused?

It's magic ;)  Think of the filesystem that df is looking at as a 
combination of the original volume with some modifications held in the 
snapshot volume.

- Joe


_______________________________________________
linux-lvm mailing list
linux-lvm@sistina.com
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] LVM snap size weirdness...

[Joe Thornber]

On Thursday, March 20, 2003, at 09:59 PM, Barry, Christopher wrote:

> 	When a backup program grabs files from the snapshot volume, is it 
> subject to open file issues that are normally a problem during > backups?

If are taking a backup from the snapshot, you needed worry about 
applications that have files open on the origin.

- Joe

Re: [linux-lvm] LVM snap size weirdness...

[Christopher Barry]

On Fri, 2003-03-21 at 03:42, Joe Thornber wrote:
> 
> On Thursday, March 20, 2003, at 09:59 PM, Barry, Christopher wrote:
> 
> > 	When a backup program grabs files from the snapshot volume, is it 
> > subject to open file issues that are normally a problem during 
> > backups?
> 
> If are taking a backup from the snapshot, you needed worry about 
Thanks Joe. Is this a typo?---------------------^^^^^^

I assume you meant "needn't". Correct?

-C

> applications that have files open on the origin.
> 
> - Joe
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] LVM snap size weirdness...

[Alasdair G Kergon]

You're quite right - you must *always* worry about applications 
changing the filesystem when taking a backup.

Backing up from a filesystem snapshot means your backup will at
least contain the complete filesystem as it looked at the moment in
time when the snapshot was created.
(Consider instead if you were to run 'tar' on a changing filesystem:
if a process was moving directories at the same time, your tar
backup might completely miss them e.g. if they were moved from a 
place 'tar' hadn't backed up yet into a place it already had.
If you backup from a snapshot, that problem disappears.)

If you backup from a snapshot, you want to ensure that your
on-disk application data is consistent (i.e. checkpointed)
at the time you create the snapshot.  So you have to worry
about applications that are running at the point at which 
you take the snapshot: e.g. quiesce them, take the snapshot,
then set them off again.

Alasdair
-- 
agk@uk.sistina.com

RE: [linux-lvm] LVM snap size weirdness...

[Barry, Christopher]

Ok. Cool. Thanks very much for the knowledge. I have so much more to learn about LVM.

So, by quiesce, would you suggest then, that I stop samba, nfs, apache - take a snapshot and immediately restart them? Then, fire off my backups? Stopping/starting nfs can really wreak havoc on connections (e.g. 'stale file handles' and all). 

For my clarification, and possibly others as well, what happens if, during a snapshot, someone is writing a file to the server via nfs? Is this file snapshotted is an unstable state? Is it skipped? Should I issue a sync immediately prior to snapshotting? How does snapshotting handle this case? 

Regards and thanks,
--
Christopher Barry
Manager of Information Systems
InfiniCon Systems
http://www.infiniconsys.com

-----Original Message-----
From: Alasdair G Kergon [mailto:agk@uk.sistina.com]
Sent: Friday, March 21, 2003 10:25 AM
To: linux-lvm@sistina.com
Subject: Re: [linux-lvm] LVM snap size weirdness...


You're quite right - you must *always* worry about applications 
changing the filesystem when taking a backup.

Backing up from a filesystem snapshot means your backup will at
least contain the complete filesystem as it looked at the moment in
time when the snapshot was created.
(Consider instead if you were to run 'tar' on a changing filesystem:
if a process was moving directories at the same time, your tar
backup might completely miss them e.g. if they were moved from a 
place 'tar' hadn't backed up yet into a place it already had.
If you backup from a snapshot, that problem disappears.)

If you backup from a snapshot, you want to ensure that your
on-disk application data is consistent (i.e. checkpointed)
at the time you create the snapshot.  So you have to worry
about applications that are running at the point at which 
you take the snapshot: e.g. quiesce them, take the snapshot,
then set them off again.

Alasdair
-- 
agk@uk.sistina.com

_______________________________________________
linux-lvm mailing list
linux-lvm@sistina.com
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] LVM snap size weirdness...

[Alasdair G Kergon]

On Fri, Mar 21, 2003 at 12:19:34PM -0500, Barry, Christopher wrote:
> So, by quiesce, would you suggest then, that I stop samba, nfs, apache - 
> take a snapshot and immediately restart them? Then, fire off my backups? 

The answer is going to be different for each application!

In general for an application that modifies files, yes, stop it, 
take the snapshot, then restart it.  

But, for example, if the application only reads the data, or only 
appends to files, or has its own internal checkpointing you shouldn't
need to.

It's not really NFS that needs stopping: it's any applications 
on your remote machines that are writing to the NFS-mounted volumes.

It's good that you're thinking about this: many people don't, and 
subsequently discover when they restore data from their backups
that their applications think it's corrupt.

Alasdair