RFD Use git for off-site backups

RFD Use git for off-site backups

[Thomas Glanzmann]

Hello,
I am thinking about using git for doing differential backups of a whole
machines. And I would like to know what is necessary to obtain that. In
such a scenario the machine to be backed up should have a local index
file but sends its objects directly to the remote machine via git-via-ssh
or git protocol. Is it possible to store extra information like extended
attributes / acls in git? I don't care about named pipes, sockets and
device files for the moment.

        Thomas

Re: RFD Use git for off-site backups

[Johannes Schindelin]

Hi,

On Sun, 13 May 2007, Thomas Glanzmann wrote:

> I am thinking about using git for doing differential backups of a whole
> machines. And I would like to know what is necessary to obtain that. In
> such a scenario the machine to be backed up should have a local index
> file but sends its objects directly to the remote machine via git-via-ssh
> or git protocol.

I think that this setup (have objects remotely) will kill performance 
pretty quickly. You have to compare to other blobs in order to find 
reasonable deltas...

> Is it possible to store extra information like extended attributes / 
> acls in git?

There has been talk about pre-commit scripts which put this information 
into a file, and a post-checkout script to resurrect these metadata, but I 
haven't seen functional versions of these scripts yet.

Ciao,
Dscho

Re: RFD Use git for off-site backups

[Thomas Glanzmann]

Hello,

> I think that this setup (have objects remotely) will kill performance
> pretty quickly. You have to compare to other blobs in order to find
> reasonable deltas...

so true. I thought about that myself just after I sent the eMail.

> There has been talk about pre-commit scripts which put this
> information into a file, and a post-checkout script to resurrect these
> metadata, but I haven't seen functional versions of these scripts yet.

I thought there is a way to add information without loosing compability
with current git versions. But maybe I am just mistaken.

        Thomas

Re: RFD Use git for off-site backups

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1931 bytes --]

On Sun, May 13, 2007 at 11:34:17 +0200, Thomas Glanzmann wrote:
> I am thinking about using git for doing differential backups of a whole
> machines. And I would like to know what is necessary to obtain that. In
> such a scenario the machine to be backed up should have a local index
> file but sends its objects directly to the remote machine via git-via-ssh
> or git protocol. Is it possible to store extra information like extended
> attributes / acls in git? I don't care about named pipes, sockets and
> device files for the moment.

I don't actually think git is a good match for whole-machine backups. It's
strenghts are in distributedness and branching, but you don't need either of
those for large-scale backups (monitoring changes say in /etc is a different
thing -- git is sensible match there -- see http://www.isisetup.ch/).

I can suggest two solutions:

 - Use CVS on the client (machine to back up) and git-cvsserver on the
   server. You will probably be able to find some scripts that do backups to
   CVS. On server you will still have git for digging in history. Git is also
   very space efficient.

   CVS does not cache any data on the client. It only keeps metadata to
   detect which files were modified. Therefore it will have minimal overhead
   in storage, but needs to transfer the modified files in their entirety.

 - Use Subversion. There is summer of code project to create git-svnserver,
   so you might be able to use git on the server-side in future with
   subversion client too. But for backups you should not need any of the
   version control features subversion does not have, so subversion should be
   suitable.

   Subversion stores copy of the previous revision on the client, so all data
   under it's control will take twice as much space on client, but it will
   only transfer deltas over the network.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFD Use git for off-site backups

[Shawn O. Pearce]

Jan Hudec <bulb@ucw.cz> wrote:
>  - Use Subversion. There is summer of code project to create git-svnserver,
>    so you might be able to use git on the server-side in future with
>    subversion client too. But for backups you should not need any of the
>    version control features subversion does not have, so subversion should be
>    suitable.

Unfortunately Google is not sponsering the git-svnserver summer of
code project.  We didn't get enough slots for us to get Google to
pay for that one.  At least one of the students has still expressed
interest in working on it, but it won't be their full-time job this
summer. :-(

-- 
Shawn.