birgers RFE #2

birgers RFE #2

[birger]

I understand that there is resistance to adding dependencies to the
tools package, so how about a contrib package for contributed tools?

I have a neat snapshot script for my laptop that I wouldn't mind
sharing. Works mostly like the NetApp snapshots. Set up a cron job with
a prefix for the snapshot name and the number of snapshots to keep. It's
very easy to set up for schemes like 24 hourly snapshots, 7 daily and 4
weekly.

The only dependency as far as I remember now is bash.

I would also be willing to try setting up a btrfs-to-btrfs incremental
snapshot system given that my RFE #3 also gets done. :-)

-- 
birger

Re: birgers RFE #2

[Xavier Nicollet]

Le 03 ao=FBt 2010 =E0 14:44, birger a =E9crit:
> I have a neat snapshot script for my laptop that I wouldn't mind
> sharing. Works mostly like the NetApp snapshots. Set up a cron job wi=
th
> a prefix for the snapshot name and the number of snapshots to keep. I=
t's
> very easy to set up for schemes like 24 hourly snapshots, 7 daily and=
 4
> weekly.

I have roughly the same thing in python. I can post it if someone is
interested.

Cheers,

--=20
Xavier Nicollet
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: birgers RFE #2

[Benjamin Griese]

Hello guys,
I would be interested in that script, maybe in both of yours,
Thank you! :)

On Tue, Aug 3, 2010 at 15:48, Xavier Nicollet <nicollet@jeru.org> wrote=
:
>
> Le 03 ao=FBt 2010 =E0 14:44, birger a =E9crit:
> > I have a neat snapshot script for my laptop that I wouldn't mind
> > sharing. Works mostly like the NetApp snapshots. Set up a cron job =
with
> > a prefix for the snapshot name and the number of snapshots to keep.=
 It's
> > very easy to set up for schemes like 24 hourly snapshots, 7 daily a=
nd 4
> > weekly.
>
> I have roughly the same thing in python. I can post it if someone is
> interested.
>
> Cheers,
>
> --
> Xavier Nicollet
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html



--
To be or not to be -- Shakespeare | To do is to be -- Nietzsche | To
be is to do -- Sartre | Do be do be do -- Sinatra




--
To be or not to be -- Shakespeare | To do is to be -- Nietzsche | To
be is to do -- Sartre | Do be do be do -- Sinatra
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: birgers RFE #2

[Xavier Nicollet]

Le 03 ao=FBt 2010 =E0 15:42, Benjamin Griese a =E9crit:
> Hello guys,
> I would be interested in that script, maybe in both of yours,

It is not supposedly bulletproof. Use at your own risks.
backup1:/usr/local/bin# crontab -l
# m h  dom mon dow   command
# hourly backup
0 * * * *  /usr/local/bin/snapshots.py -h     =20
# daily backup
10 6 * * *  /usr/local/bin/snapshots.py -d
# weekly backup
#20 6 * * 0 /usr/local/bin/snapshots.py -w
# monthly backup
#30 6 1 * * /usr/local/bin/snapshots.py -m
# yearly backup
#40 6 1 1 * /usr/local/bin/snapshots.py -y


#! /usr/bin/env python

""" Daily snapshots for all our backups """

import os
import os.path
import glob
import sys
import subprocess
import time
import re



def usage():
        """ Print usage and exit """
        print "usage: %s [-y|-m|-d|-h]" % sys.argv[0]
        sys.exit(2)

def chdir_force(path):
        """ Chdir even if we have to make the path """
        try:
                os.makedirs(path)
        except OSError:
                pass
        os.chdir(path)

class Tuning:
        tuning =3D {
                '-y' : [ 'yearly', 1 ],   # we keep 1 snapshots
                '-m' : [ 'monthly', 1 ],  # we keep 1 snapshots
                '-w' : [ 'weekly', 1 ],   # we keep 1 weekly snapshots
                '-d' : [ 'daily', 4 ],
                '-h' : [ 'hourly', 2 ],
        }

        def __init__(self):
                try:
                        option =3D sys.argv[1]
                        (self.mode, self.keep) =3D Tuning.tuning[option=
]
                except (IndexError, KeyError):
                        usage()

def get_backup_name():
        """ Return the backup dir name.=20
                It depends on the current location and time. """
        name =3D time.strftime("backup-%y-%m-%d-%H:%M:%S")
        if os.path.exists(name):
                print "name already exists"
                sys.exit(2)
        return name

def main():
        tuning =3D Tuning()

        backup_path =3D os.path.join("/mnt/btrfs/history", tuning.mode)
        chdir_force(backup_path)

        # Make the new snapshot
        cmd =3D "/usr/local/bin/btrfs subvolume snapshot".split()
        backup_name =3D get_backup_name()
        cmd.extend( ["/backup/", backup_name] )
        ret =3D subprocess.call(cmd)
        if ret !=3D 0:
                print "return from btrfs subvol snap: " + str(ret)

        # Filter the number of snapshots
        files =3D sorted( (file for file in glob.glob( "backup-*") if n=
ot file.endswith(".old")) )
        if len(files) > tuning.keep:
                for file in files[:-tuning.keep]:
                        cmd =3D "/usr/local/bin/btrfs subvolume delete =
".split()
                        cmd.append( file )  =20
                        ret =3D subprocess.call(cmd)
                        if ret !=3D 0:
                                print "return from btrfs subvol del: " =
+ ret

if __name__ =3D=3D '__main__':
        main()




--=20
Xavier Nicollet
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: birgers RFE #2

[birger]

[-- Attachment #1: Type: text/plain, Size: 16 bytes --]

Here is mine...

[-- Attachment #2: btrfs-snap --]
[-- Type: application/x-shellscript, Size: 2899 bytes --]

Re: birgers RFE #2

[cwillu]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

And none of them are what I would consider remotely useful for
backups.  (What, you guys don't backup to external media?)  :)

Attached is my nightly backup script, edit it to suit, and dump a
symlink into /etc/cron.daily or whatever.  It requires a btrfs target
(rather than source).

[-- Attachment #2: backup --]
[-- Type: application/octet-stream, Size: 2785 bytes --]

#!/bin/bash

renice 20 $$
ionice -c3 -p$$

# export DISPLAY=:0

echo $(date) Starting backups >> /var/log/backup

# some day, make this find suitable drives automatically
BACKUP_TO_A="/media/drive_a/"  
BACKUP_TO_B="/media/drive_b/"

# abort if there's no backup drive connected
mountpoint "$BACKUP_TO_A" && BACKUP_TO="$BACKUP_TO_A"
mountpoint "$BACKUP_TO_B" && BACKUP_TO="$BACKUP_TO_B"
mountpoint "$BACKUP_TO"  || exit

YESTERDAY="$(ls -1t ${BACKUP_TO} | head -n 1)"
TODAY="$(date +%F)"

echo Will snapshot ${YESTERDAY} to ${TODAY} >> /var/log/backup

btrfs subvolume snapshot "${BACKUP_TO}/${YESTERDAY}" "${BACKUP_TO}/${TODAY}" || exit 1

BACKUP_TO="${BACKUP_TO}/${TODAY}"

echo $(date) Backing up to $BACKUP_TO >> /var/log/backup

eval `ssh-agent`
ssh-add

# Delete files which no longer exist in the destination, 
# even if errors occur during the transfer.  Exclude a handful
# of potentially troublesome directories (even though we use -x,
# .gvfs is often not readable by root)
STANDARD_PARMS="--delete --ignore-errors --exclude .local/share/Trash/ --exclude .gvfs/ --exclude /media/"

# List copied files, do the typical backup thing (attributes, 
# mod-dates, etc), stay on the source filesystem, and use 
# relative pathnames so that we can keep backup explicitely 
# listed sub-mounts filesystems to the 'right' place.
PARMS="-vaxR $STANDARD_PARMS"
MOUNTS="-vaR $STANDARD_PARMS" 

do_backup ()
{
  TARGET_USER="$1"
  TARGET_HOST="$2"
  TARGET_PATH="$3"
  
  echo $TARGET_HOST:$TARGET_PATH Try >> /var/log/backup
  
  STORE_PATH="$BACKUP_TO/$TARGET_HOST"
  ping -c 1 $TARGET_HOST && {
    echo $(date) $TARGET_HOST:$TARGET_PATH Start >> /var/log/backup
  	
    mkdir -p $STORE_PATH
    rsync $PARMS $TARGET_USER@$TARGET_HOST:$TARGET_PATH $STORE_PATH
    echo $(date) $TARGET_HOST@$TARGET_PATH Complete >> /var/log/backup
  }
}

do_local_backup ()
{
  TARGET_PATH="$1"

  STORE_PATH="$BACKUP_TO/local"

  echo local:$TARGET_PATH Try >> /var/log/backup

  # mountpoint $TARGET_PATH is not used because the mount point will
  # often exist even if though the device isn't there.  That would be
  # a problem given that we delete files that are missing from the
  # source; the --max-delete option is not a sane alternative solution.
  find $TARGET_PATH -maxdepth 0 -not -empty -type d | grep / && {
    echo $(date) local:$TARGET_PATH Start >> /var/log/backup

    mkdir -p $STORE_PATH
    rsync $PARMS $TARGET_PATH $STORE_PATH
    echo $(date) local:$TARGET_PATH Complete >> /var/log/backup
  }
}

do_backup root some-hostname-here / &
PARMS="$PARMS --exclude foo/ --exclude Music/" do_backup root some-other-hostname / &

do_local_backup / &
PARMS="$MOUNTS" do_local_backup /media/silly-windows-machine &

wait
ssh-agent -k
echo $(date) Backups completed  >> /var/log/backup