Re: [linux-lvm] LVM & snapshots

[Les Mikesell <lesmikesell@gmail.com>]

Gordon Henderson wrote:
> 
> I tried LVM a few years back and "burnt my fingers" as it were - I found 
> it slow and unstable, but times are changing, progress being made, etc. 
> so does anyone have any input on it's current stability and usability?
> 
> Basically, I'm running out of steam on some backup servers I have - 
> trying to copy several TB of data using cp -al, then rsyncing on top of 
> it (and then copying the copy, etc. to give me 30 days of backups) has 
> been working well for some time, but the data-set is increasing all the 
> time, and the cp -al is currently taking 4-5 hours, so snapshotting via 
> LVM is looking like it's something I think I ought to be looking into 
> again.
> 
> The current data-set is 5TB, and having 150TB of storage isn't an option 
> right now... Essentially I need to be able to keep 30 cycles (days) of 
> snapshots on a server that is not a live server (ie. it's a huge disk 
> array, in a secure bunker backing up several remote servers every day, 
> and dumping the occasional snapshot to tape for archive. Data comes into 
> the server via a 100Mb line & rsync from the remote servers).
> 
> So is LVM(2) up to this task these days, or should I really be looking 
> at something else?

You might want to look at backuppc (http://backuppc.sourceforge.net/) to 
manage the on-line backups.  It uses a scheme of compression and 
hard-linking duplicate files to greatly reduce the storage space you 
need when keeping a long history - especially if many files are 
unchanged over that time.   With something that size you might still 
have to play some tricks with snapshots or break it down into smaller 
directory runs.  It uses tar, smb, or rsync to do the actual copies, 
then has some extra overhead for the compression and linking jobs.  If 
it is too slow, you might run it against your existing uncompressed copy 
so it has all day to make the long-term archive version.

-- 
   Les Mikesell
    lesmikesell@gmail.com