ok to delete tmp_pack file(s)? also, backing up a git repository

ok to delete tmp_pack file(s)? also, backing up a git repository

[Adam Monsen]

I backup my remote git repositories on sf.net using "rsync -a REMOTE
LOCAL". I chose rsync instead of clone --mirror/fetch because I wanted
hook scripts, "description", and "config" backed up.

I got the following error from my backup cron job:

        rsync: send_files failed to open
        "/mifos/head/objects/pack/tmp_pack_IcK2CZ" (in gitroot):
        Permission denied (13)

So, fine, I can't reach that file. Looks like a temp file--I think I did
a CTRL-C during a push, so maybe that's what caused it to be left
around. A few questions:

     1. Should I just ignore "objects/pack/tmp_pack_*" file in my rsync
        backup?
     2. Should I shell into sf.net and run "git gc" on the repository?
        or "git repack"? or something else?
     3. Is rsync a bad idea? For instance, is the backup useful even if
        rsync was running while someone was doing a "git push" into the
        repository being backed up?
     4. Anyone have different rsync options to recommend for optimal
        backup of a git repository?

Thank you!
-Adam

PS - git rocks... our benefits since migrating from svn are numerous:
* remote folks (UK, India, Africa, etc.) are quite happy with the speed
increase
* branching and merging just works
* examining logs/forensics is much easier and faster
* lots more, but I gotta grab me some lunch

Re: ok to delete tmp_pack file(s)? also, backing up a git repository

[Nicolas Pitre]

On Wed, 19 May 2010, Adam Monsen wrote:

> I backup my remote git repositories on sf.net using "rsync -a REMOTE
> LOCAL". I chose rsync instead of clone --mirror/fetch because I wanted
> hook scripts, "description", and "config" backed up.
> 
> I got the following error from my backup cron job:
> 
>         rsync: send_files failed to open
>         "/mifos/head/objects/pack/tmp_pack_IcK2CZ" (in gitroot):
>         Permission denied (13)
> 
> So, fine, I can't reach that file. Looks like a temp file--I think I did
> a CTRL-C during a push, so maybe that's what caused it to be left
> around. A few questions:
> 
>      1. Should I just ignore "objects/pack/tmp_pack_*" file in my rsync
>         backup?

Yes.

>      2. Should I shell into sf.net and run "git gc" on the repository?
>         or "git repack"? or something else?

The command which removes stall tmp files after a grace period is 'git 
prune'.  This is normally run amongst other things by 'git gc'.

>      3. Is rsync a bad idea? For instance, is the backup useful even if
>         rsync was running while someone was doing a "git push" into the
>         repository being backed up?

If you want your rsync backup to be fine, you need to follow some 
ordering.  You need to copy the refs first (.git/packed-refs and 
.git/refs/), then the loose objects (.git/objects/??/*), and then all 
the rest.  If files are copied in a different order while some write 
operations are performed on the source repository then you may end up 
with an incoherent repository.

>      4. Anyone have different rsync options to recommend for optimal
>         backup of a git repository?

The optimal backup of a git repo is performed by git itself.  Due to 
repacking, the pack files are likely to change over time which will 
completely defeat rsync's ability to transfer only differences to a file 
when updating your backup.  Git will always transfer the minimum of 
information to replicate a repository's _content_ even if the actual 
repository files aren't identical on both ends, plus you get some SHA1 
validation of the transferred data for free.  But if your repository is 
quiet and you don't mind the extra transfer size then rsync should work 
just fine.

> Thank you!
> -Adam
> 
> PS - git rocks... our benefits since migrating from svn are numerous:
> * remote folks (UK, India, Africa, etc.) are quite happy with the speed
> increase
> * branching and merging just works
> * examining logs/forensics is much easier and faster
> * lots more, but I gotta grab me some lunch
> 
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: ok to delete tmp_pack file(s)? also, backing up a git repository

[Adam Monsen]

Nicolas Pitre writes:
> If you want your rsync backup to be fine, you need to follow some 
> ordering.

Ah, good to know! I don't want to complicate my backup job too much, so
I'll probably switch to a job that does the following for each repository,
daily (or so):

1) "git fetch": update bare clone
2) rsync backup "hooks/", "config", and "description".
I think that's all I've customized that I'd miss in a catastrophic failure.

Thanks for the tips.