mirror git repos: rsync vs. --mirror

mirror git repos: rsync vs. --mirror

[Robin H. Johnson]

I'm trying to see about the state of automated mirroring git
repositories between different systems, eg. a private backend and a
public frontend, and ran into a catch between two possible options, each
with a downside. If there are other means besides these, I'm all ears
for them.

rsync:
------
How:
- Simply rsync the git repository, atomically, and use --delete-after.
Benefits:
- Guaranteed to match between master/slaves.
Caveats:
- Need to know that the master repo is in a good state.
- If a repo is repacked, there is a lot more traffic in the next sync.

git clone --mirror && git fetch:
--------------------------------
How:
- Setup with: "git clone --mirror URL"
- Cronjob with: "git fetch"
Benefits:
- Extremely fast
- Light on traffic
Caveats:
- Branches and tags deleted in master persist in slaves :-(
- Seperate gc/repack needed on slaves.

The traffic vs. deleted branches/tags is the main tradeoff here, esp. as
repos get larger.

Can git-fetch perhaps be made to delete the branches/tags?

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

Re: mirror git repos: rsync vs. --mirror

[Julian Phillips]

On Tue, 30 Mar 2010 20:54:09 +0000, "Robin H. Johnson"
<robbat2@gentoo.org>
wrote:
> I'm trying to see about the state of automated mirroring git
> repositories between different systems, eg. a private backend and a
> public frontend, and ran into a catch between two possible options, each
> with a downside. If there are other means besides these, I'm all ears
> for them.

You can mitigate the git caveats somewhat ...

> git clone --mirror && git fetch:
> --------------------------------
> How:
> - Setup with: "git clone --mirror URL"
> - Cronjob with: "git fetch"
> Benefits:
> - Extremely fast
> - Light on traffic
> Caveats:
> - Branches and tags deleted in master persist in slaves :-(
> - Seperate gc/repack needed on slaves.
> 
> The traffic vs. deleted branches/tags is the main tradeoff here, esp. as
> repos get larger.
> 
> Can git-fetch perhaps be made to delete the branches/tags?

I don't think that fetch can do it, but "git remote prune origin" will.
You could also add "git gc --auto" to your cron script to deal with the
repacking - I don't think fetch already does that ...

-- 
Julian

Re: mirror git repos: rsync vs. --mirror

[Sitaram Chamarty]

On Wed, Mar 31, 2010 at 3:30 AM, Julian Phillips
<julian@quantumfyre.co.uk> wrote:
> On Tue, 30 Mar 2010 20:54:09 +0000, "Robin H. Johnson"
> <robbat2@gentoo.org>

>> git clone --mirror && git fetch:

>> Caveats:
>> - Branches and tags deleted in master persist in slaves :-(

> I don't think that fetch can do it, but "git remote prune origin" will.

as of 1.7 at least, fetch has --prune, and I tested it yesterday when
someone (same person?) asked this on #git

Re: mirror git repos: rsync vs. --mirror

[Robin H. Johnson]

On Wed, Mar 31, 2010 at 07:14:46AM +0530, Sitaram Chamarty wrote:
> On Wed, Mar 31, 2010 at 3:30 AM, Julian Phillips
> <julian@quantumfyre.co.uk> wrote:
> > On Tue, 30 Mar 2010 20:54:09 +0000, "Robin H. Johnson"
> > <robbat2@gentoo.org>
> 
> >> git clone --mirror && git fetch:
> 
> >> Caveats:
> >> - Branches and tags deleted in master persist in slaves :-(
> 
> > I don't think that fetch can do it, but "git remote prune origin" will.
> as of 1.7 at least, fetch has --prune, and I tested it yesterday when
> someone (same person?) asked this on #git
Wasn't me asking, but thanks, that helps lots!

Looks like it was added for v1.6.6.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85