need help with syncing two bare repos

need help with syncing two bare repos

[Eugene Sajine]

Hi,

Could somebody please advise about how to address the following:

I have a bare repo (bareA) on one server in network1 and i have a
mirror of it on another server (bareB) in network2
BareB is updated periodically - no problem here
If bareA dies users are supposed to move seamlessly to bareB.
When bareA goes back up users are moved back but before it starts
serving repos (before git-daemon starts) it updates from bareB.

Now the problem i have is if bareA doesn't actually die, but the
connection between two networks drops.
In this case users from network2 will stop seeing bareA, they will
start working with bareB, while users in netwrok1 will continue
to work with bareA.

What would be the best way of syncing the bareB back to bareA when
connection is restored?

I think the best variant would be to do something like:

$ git pull --rebase /refs/heads/*:/refs/heads/*
$ git push origin /refs/heads/*:/refs/heads/*

but pull will not work on bare repos as i understand and there might
be conflicts that will lead to unknown (for me) state of bare repos

May be I'm looking into wrong direction? May be simple two way rsync
will do the job? But I'm a bit reluctant to rely on rsync because I'm
afraid it may screw up the repository information.

Any ideas are much appreciated!

Thanks,
Eugene

Re: need help with syncing two bare repos

[Sitaram Chamarty]

On Fri, Aug 3, 2012 at 11:59 PM, Eugene Sajine <euguess@gmail.com> wrote:
> Hi,
>
> Could somebody please advise about how to address the following:
>
> I have a bare repo (bareA) on one server in network1 and i have a
> mirror of it on another server (bareB) in network2
> BareB is updated periodically - no problem here
> If bareA dies users are supposed to move seamlessly to bareB.
> When bareA goes back up users are moved back but before it starts
> serving repos (before git-daemon starts) it updates from bareB.
>
> Now the problem i have is if bareA doesn't actually die, but the
> connection between two networks drops.
> In this case users from network2 will stop seeing bareA, they will
> start working with bareB, while users in netwrok1 will continue
> to work with bareA.
>
> What would be the best way of syncing the bareB back to bareA when
> connection is restored?
>
> I think the best variant would be to do something like:
>
> $ git pull --rebase /refs/heads/*:/refs/heads/*
> $ git push origin /refs/heads/*:/refs/heads/*
>
> but pull will not work on bare repos as i understand and there might
> be conflicts that will lead to unknown (for me) state of bare repos
>
> May be I'm looking into wrong direction? May be simple two way rsync
> will do the job? But I'm a bit reluctant to rely on rsync because I'm
> afraid it may screw up the repository information.

The way I solve this is to insist that I *manually* specify which is
the push destination and not make it automated.  The other one is a
read-only mirror until something is manually switched.

Might not be ideal for every situation; your call...

Re: need help with syncing two bare repos

[Junio C Hamano]

Eugene Sajine <euguess@gmail.com> writes:

> I think the best variant would be to do something like:
>
> $ git pull --rebase /refs/heads/*:/refs/heads/*
> $ git push origin /refs/heads/*:/refs/heads/*

You perhaps meant "worst" not "best" here.  From the point of view
of people who have pushed into the "origin" repository we see above,
their history is rewritten; you are screwing half the population by
doing this.

Not allowing B to accept pushes while it is not positively sure that
A has gone down would of course be the best solution (in your scenario,
B started serving when it merely found out that *it* cannot contact
A), but barring that, the recovery for two histories at A and B,
once they diverged, should be to "merge" corresponding branches.

Re: need help with syncing two bare repos

[Eugene Sajine]

On Fri, Aug 3, 2012 at 4:00 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Eugene Sajine <euguess@gmail.com> writes:
>
>> I think the best variant would be to do something like:
>>
>> $ git pull --rebase /refs/heads/*:/refs/heads/*
>> $ git push origin /refs/heads/*:/refs/heads/*
>
> You perhaps meant "worst" not "best" here.  From the point of view
> of people who have pushed into the "origin" repository we see above,
> their history is rewritten; you are screwing half the population by
> doing this.
>
> Not allowing B to accept pushes while it is not positively sure that
> A has gone down would of course be the best solution (in your scenario,
> B started serving when it merely found out that *it* cannot contact
> A), but barring that, the recovery for two histories at A and B,
> once they diverged, should be to "merge" corresponding branches.

Junio,

Thanks for chipping in!
Please correct me if I'm wrong, but wouldn't merge result in the same
history changes for people pushing to the origin (bareA)?
Still all their consequent changes will not be fast-forwardable
because of merge commit. Or am i missing something?
It might be that the command i specified is incorrect.

I meant that I would execute "pull --rebase" (or proper analogue for
bare repo) for each branch on B side and then push those refs to A.
This way it would pretend that B is another "dev repo".
May be you can advise on correct commands to do that properly?

I'll think about not allowing pushes to B while not sure it is really
time, so it is not reacting on irrelevant connectivity issues or bareA
downtime due to the maintenance.

Thanks,
Eugene

Re: need help with syncing two bare repos

[Junio C Hamano]

Eugene Sajine <euguess@gmail.com> writes:

> On Fri, Aug 3, 2012 at 4:00 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Eugene Sajine <euguess@gmail.com> writes:
>>
>>> I think the best variant would be to do something like:
>>>
>>> $ git pull --rebase /refs/heads/*:/refs/heads/*
>>> $ git push origin /refs/heads/*:/refs/heads/*
>>
>> You perhaps meant "worst" not "best" here.  From the point of view
>> of people who have pushed into the "origin" repository we see above,
>> their history is rewritten; you are screwing half the population by
>> doing this.
>>
>> Not allowing B to accept pushes while it is not positively sure that
>> A has gone down would of course be the best solution (in your scenario,
>> B started serving when it merely found out that *it* cannot contact
>> A), but barring that, the recovery for two histories at A and B,
>> once they diverged, should be to "merge" corresponding branches.
>
> ... but wouldn't merge result in the same
> history changes for people pushing to the origin (bareA)?
> Still all their consequent changes will not be fast-forwardable
> because of merge commit. Or am i missing something?

Welcome to the world of distributed version control.

Even with a single server, it always is possible that other people
have been working on their changes and pushed their results out
while you are working on your changes when you are working with
others.  It is perfectly expected that your changes may not be
fast-forwardable.  In such a case, you can, and you are expected to,
fetch the updated tip from the public meeting point, integrate your
work with it and then push the result back.

That is not an issue.

If the project has "strictly linear history only" policy [*1*], then
the "fetch the updated tip, integrate your work and push back" would
involve a rebase on the participant's part, and only in that very
narrow context, it wouldn't make a difference if you rebase at the
server end while doing your recovery procedure.  In a way, the
rebase during your recovery procedure is doing the rebase the
particupants would have to perform anyway for them.

But in a more general workflow where project participants are either
allowed or encouraged to merge (as opposed to rebase and immediately
push the untested results out), the history resulting from a merge
by a participant, when he finds that what he has is not a strict
descendant of the recovered A's history, will contain the commits he
made originally to A when it was half-alive and their doppelgänger
commits your recovery procedure produced by rebasing, which is a
disaster.


[Footnote]

*1* Projects can enforce strictly linear history by rejecting push
of a history with any merges in their pre-receive hooks.  Such a
practice may be denying its participants major benefit of Git,
namely, distributedness, but it is probably easier to explain to
people coming from CVS or systems where branching and merging are
impossibly hard.