[linux-lvm] Enterprise Backup Software with good support for LVM

[linux-lvm] Enterprise Backup Software with good support for LVM

[Mark F]

Could someone suggest an Enterprise Backup package that has good support 
for LVM (LVM2 in this case) on Linux (Specifically Suse Linux Enterprise 
9) Most products I've seen say they support block level backups but 
don't specifically say they support LVM.

Thanks,
-Mark

Re: [linux-lvm] Enterprise Backup Software with good support for LVM

[Michael Loftis]

--On November 29, 2005 12:03:39 PM -0600 Mark F <mfaine@knology.net> wrote:

> Could someone suggest an Enterprise Backup package that has good support
> for LVM (LVM2 in this case) on Linux (Specifically Suse Linux Enterprise
> 9) Most products I've seen say they support block level backups but don't
> specifically say they support LVM.

LVM provides a block level device.  However you don't want to back up there 
anyway, you'll just end up with corrupt garbage.  You need to back up at 
the filesystem level unless you're using your LVM LVs for Oracle or 
something similar in which case the backup software needs to support that 
by putting Oracle or the related application into the correct hot backup 
mode before attempting to directly read the LVM data, as well as backing up 
the transaction/redo logs.

[linux-lvm] Re: Enterprise Backup Software with good support for LVM

[Mark F]

Michael Loftis wrote:
> 
> 
> --On November 29, 2005 12:03:39 PM -0600 Mark F <mfaine@knology.net> wrote:
> 
>> Could someone suggest an Enterprise Backup package that has good support
>> for LVM (LVM2 in this case) on Linux (Specifically Suse Linux Enterprise
>> 9) Most products I've seen say they support block level backups but don't
>> specifically say they support LVM.
> 
> 
> LVM provides a block level device.  However you don't want to back up 
> there anyway, you'll just end up with corrupt garbage.  You need to back 
> up at the filesystem level unless you're using your LVM LVs for Oracle 
> or something similar in which case the backup software needs to support 
> that by putting Oracle or the related application into the correct hot 
> backup mode before attempting to directly read the LVM data, as well as 
> backing up the transaction/redo logs.
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 
Our data center is using Veritas Netbackup (currently 4.5 but soon to be 
5.1).  We are preparing some new servers for production and would like 
to use LVM2 but I'm not sure if this will present a problem for the 
Veritas Netbackup agent. I don't know if the backup agent software will 
support it or if it will even know the difference.

Is anyone using it?  Or does anyone have any reason to believe it 
will/won't work?

Thanks,
-Mark

Re: [linux-lvm] Re: Enterprise Backup Software with good support for LVM

[Graham Wood]

On Tue, Nov 29, 2005 at 12:40:02PM -0600, Mark F wrote:
> Michael Loftis wrote:
> Is anyone using it?  Or does anyone have any reason to believe it 
> will/won't work?
Netbackup does a file level backup, rather than block level.  This means 
that the filesystem used is not relevant, and the underlying structure
even more so.

E.g. it works at the directory/file level - so it backs up the files
that it finds and doesn't care what sort of block device is under that.

(That's assuming you're doing file backups, and not raw devices for DBs)

Re: [linux-lvm] Enterprise Backup Software with good support for LVM

[Graham Wood]

On Tue, Nov 29, 2005 at 11:29:19AM -0700, Michael Loftis wrote:
> LVM provides a block level device.  However you don't want to back up there 
> anyway, you'll just end up with corrupt garbage.  You need to back up at 
> the filesystem level unless you're using your LVM LVs for Oracle or 
> something similar in which case the backup software needs to support that 
> by putting Oracle or the related application into the correct hot backup 
> mode before attempting to directly read the LVM data, as well as backing up 
> the transaction/redo logs.

Surely by using snapshots, a block level backup could be "OK".  You'd
need to (ideally) tell the file-system layer to put the machine into a
fit state to be snapshotted (quick & dirty, obviously issues with it,
you could remount it readonly, snapshot, and then mount read-write
again).  It's also going to backup dead space, which is going to waste
backup media (e.g. tape) at best, and leak information at worst.

If you have a journalling filesystem, then taking a snapshot of the
active device would probably be acceptable...  Not ideal, but depending
on the relative importance of getting a perfect backup/restore and
reducing the load on the system while the backup is being done, it might
be acceptable to most people.

Obviously if your backup solution can tie into the filesystem's own
backup utilities then you're going to get a better result, but for some
people that's not the important thing (and not an option in backup
utilities for others).

I know that dump/restore on solaris is unsafe to use on live partitions,
I can't see any reference within the dump man pages to see if the same
applies to linux, and have never used it in anger on an active
filesystem - old habits die hard, so I do my best to keep it inactive
when using it.

Graham

RE: [linux-lvm] Enterprise Backup Software with good support for LVM

[Steffen Plotner]

Hello,

I would like to add that creating backups of LVM snapshots is sensible,
as the data is in a crash-consistent state. Today's operating systems
can definitely handle this type of state. I have developed scripts that
effectively netcat contents from LVM snapshots to a remote machine
(either into another logical volume or simply a file). 

Steffen 

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
On Behalf Of Michael Loftis
Sent: Tuesday, November 29, 2005 1:29 PM
To: LVM general discussion and development
Subject: Re: [linux-lvm] Enterprise Backup Software with good support
for LVM



--On November 29, 2005 12:03:39 PM -0600 Mark F <mfaine@knology.net>
wrote:

> Could someone suggest an Enterprise Backup package that has good 
> support for LVM (LVM2 in this case) on Linux (Specifically Suse Linux 
> Enterprise
> 9) Most products I've seen say they support block level backups but 
> don't specifically say they support LVM.

LVM provides a block level device.  However you don't want to back up
there anyway, you'll just end up with corrupt garbage.  You need to back
up at the filesystem level unless you're using your LVM LVs for Oracle
or something similar in which case the backup software needs to support
that by putting Oracle or the related application into the correct hot
backup mode before attempting to directly read the LVM data, as well as
backing up the transaction/redo logs.


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Enterprise Backup Software with good support for LVM

[David Brown]

On Tue, Nov 29, 2005 at 01:43:17PM -0500, Steffen Plotner wrote:

> I would like to add that creating backups of LVM snapshots is sensible,
> as the data is in a crash-consistent state. Today's operating systems
> can definitely handle this type of state. I have developed scripts that
> effectively netcat contents from LVM snapshots to a remote machine
> (either into another logical volume or simply a file). 

If you are using a journalling filesystem, then the snapshot should be
consistent.  XFS and LVM2 work together to try and make the snapshot even
more consistent.

The question, then would be can you integrate the LVM2 snapshot
creation/removal with the backup software.  I backup my snapshot volumes
using custom backup software (file based) and things work fine.

I have discovered a minor problem with snapshots, but haven't figured out
exactly what is happening.  It seems that with some types of databases,
modifications made to the database happen across the snapshots (happen to
both images).  I suspect it has something to do with mmapping a file, but I
can't reproduce it with my own code.  For my instances, the changes don't
hurt anything, since they are databases that are essentially caches of
other derrived information, but a real database file could be a problem.

Dave

Re: [linux-lvm] Enterprise Backup Software with good support for LVM

[David Brown]

On Tue, Nov 29, 2005 at 01:43:17PM -0500, Steffen Plotner wrote:

> I would like to add that creating backups of LVM snapshots is sensible,
> as the data is in a crash-consistent state. Today's operating systems
> can definitely handle this type of state. I have developed scripts that
> effectively netcat contents from LVM snapshots to a remote machine
> (either into another logical volume or simply a file). 

One other thing to point out.  Any backup utility that is based on
timestamps, as opposed to a file database, is going to occasionally miss
changes made between the creation of the snapshot and the invocation of the
backup.

For example.  I perform the following.

   - Make snapshot of /foo.
   - Create file /foo/bar.
   - Backup the /foo snapshot.

At this point, the backup of /foo won't contain the file 'bar', since it
was created after the snapshot was made.

However, if the backup software uses the time that the backup was created,
rather than the time the snapshot was created, it will think that /foo/bar
would have been included in the backup, and won't catch it for future
incrementals.

If the backup software knows about LVM (creating the snapshots itself, for
example), then it _should_ know to use the correct timestamp.  My personal
experience is that it is very rare to find backup software that will
restore to the same filesystem at the time of backup.  Even the very
expensive ones don't always do the right thing.

As an example, GNU tar can be coerced into doing the right thing with LVM
snapshots.  Something like this.

  # touch /tmp/snapstamp
  # lvcreate -s ...
  # mount ...
  # tar ... -g $lfile ...

  Then fix up the timestamp on the backup.
  # stat -c %Y /tmp/snapstamp > $lfile.bak
  # tail -n +2 $lfile >> $lfile.bak
  # mv $lfile.bak $lfile

Star accepts a 'dumpdate=/tmp/snapstamp' option to use instead of current
time.

Dave

RE: [linux-lvm] Enterprise Backup Software with good support for LVM

[Allen Bolderoff]

> -----Original Message-----
> From: linux-lvm-bounces@redhat.com
[mailto:linux-lvm-bounces@redhat.com]
> On Behalf Of David Brown
> Sent: Wednesday, 30 November 2005 5:49 AM
> To: LVM general discussion and development
> Subject: Re: [linux-lvm] Enterprise Backup Software with good support
for
> LVM
> 
> On Tue, Nov 29, 2005 at 01:43:17PM -0500, Steffen Plotner wrote:
> 
> > I would like to add that creating backups of LVM snapshots is
sensible,
> > as the data is in a crash-consistent state. Today's operating
systems
> > can definitely handle this type of state. I have developed scripts
that
> > effectively netcat contents from LVM snapshots to a remote machine
> > (either into another logical volume or simply a file).
> 
> One other thing to point out.  Any backup utility that is based on
> timestamps, as opposed to a file database, is going to occasionally
miss
> changes made between the creation of the snapshot and the invocation
of
> the
> backup.

Not necessarily LVM specific, However, I am sure it will work with
snapshots, we have great success with backuppc - uses tar and/or rsync
and some clever compression/hardlinking to create easy to restore
backups with archiving all onto one partition.

If you haven't already, then check it out.
http://backuppc.sourceforge.net/

Allen Bolderoff

RE: [linux-lvm] Enterprise Backup Software with good support for LVM

[Allen Bolderoff]

> 
> Could someone suggest an Enterprise Backup package that has good
support
> for LVM (LVM2 in this case) on Linux (Specifically Suse Linux
Enterprise
> 


Speaking of LVM2, what is the state of snapshots? (specifically on
Debian if possible) - Where can I read more about *current* status?