Interleaved remote branch update problems

Interleaved remote branch update problems

[Martin Fick]

I have noticed that git push --force does not reliably force 
a push to remote branches.  In particular, it will not 
update a remote branch if that branch has been updated since 
the beginning of the push.  Is this normal, is this 
expected?  It also fails to fast forward in a similar 
situation without the -f flag.  This seems counter intuitive 
from the perspective of a git push user.  Are there good 
reasons for this?

-Martin

(Using git 1.7.8 or 1.7.10-rc3)

-- 
Employee of Qualcomm Innovation Center, Inc. which is a 
member of Code Aurora Forum

Re: Interleaved remote branch update problems

[Jeff King]

On Thu, Apr 05, 2012 at 03:49:14PM -0600, Martin Fick wrote:

> I have noticed that git push --force does not reliably force 
> a push to remote branches.  In particular, it will not 
> update a remote branch if that branch has been updated since 
> the beginning of the push.  Is this normal, is this 
> expected?

Yes, that's expected. --force means "it's OK to push something that will
rewind history", not "it's OK to clobber somebody else who is pushing at
the same time". The determination for the former happens on the client
side, and for the latter on the server side.

If you want the latter, you would need a protocol extension, I think;
I'm pretty sure the client doesn't transmit the force flag at all to the
server (and I don't think this should be tied to the force flag
automatically; they are two different cases, and you may be in a
situation where it is safe to do the former but not necessarily the
latter).

-Peff

Re: Interleaved remote branch update problems

[Martin Fick]

Jeff King <peff@peff.net> wrote:
>On Thu, Apr 05, 2012 at 03:49:14PM -0600, Martin Fick wrote:
>
>> I have noticed that git push --force does not reliably force 
>> a push to remote branches.  In particular, it will not 
>> update a remote branch if that branch has been updated since 
>> the beginning of the push.  Is this normal, is this 
>> expected?
>
>Yes, that's expected. --force means "it's OK to push something that
> will rewind history", not "it's OK to clobber somebody else who is
> pushing at the same time".

But why is it ok to clobber somebody else right after they pushed, this seems like splitting hairs?  Why force me to wait to clobber?  Afterall, git goes through great pains to only lock a ref for the small duration of time when it is updated.  But this behavior means that it effectively is actually locked for the entire duration of a push except that it teases a client into thinking it is not locked until then end of a potential costly transfer.

And what about the fast forward case?  If client A is pushing change a and then client B starts pushing b which depends on a, why shouldn't B succeed as long as A succeeds first, even without the force flag?  But it does not work this way even with the force flag.


> The determination for the former happens on the client
>side, and for the latter on the server side.
>
>If you want the latter, you would need a protocol extension, I think;
>I'm pretty sure the client doesn't transmit the force flag at all to
>the
>server (and I don't think this should be tied to the force flag
>automatically; they are two different cases, and you may be in a
>situation where it is safe to do the former but not necessarily the
>latter).

I think this use case distinction is irrelevant to users on a shared server unless you could somehow tell git to "only clobber the branch if it currently points to shax".  Which is sort of what git is pretending to do, but it isn't really since the user isn't specifying shax, it is determined when the push starts and is inherently subject to a race.  It's not like the user can safely determine what the remote branch points to before they update it to see if it is ok to clobber it.

Just a user's perspective...

-Martin

Employee of Qualcomm Innovation Center,Inc. which is a member of Code Aurora Forum

Re: Interleaved remote branch update problems

[Jeff King]

On Thu, Apr 05, 2012 at 11:32:40PM -0600, Martin Fick wrote:

> >Yes, that's expected. --force means "it's OK to push something that
> > will rewind history", not "it's OK to clobber somebody else who is
> > pushing at the same time".
> 
> But why is it ok to clobber somebody else right after they pushed,
> this seems like splitting hairs?  Why force me to wait to clobber?

Because it gives you a chance to take it back. You have told git "it is
OK to move from A to B, even if that is a rewind". You did not tell it
"it is OK to move C to B". Imagine your workflow is:

  1. Try to push B, and see that the remote is at A; your push is
     rejected.

  2. Examine "git log --graph A...B" and see that yes, your B is an
     amendment over A, and so it is OK to push. Or maybe you just happen
     to know what B and A are, because you just pushed a broken A and
     are amending it, in which case you can skip the log. But the point
     is that the user makes a decision about the case.

  3. Do "git push -f" to approve writing B over A.

If somebody else has built C on top of A, and pushed it during (3), then
you don't necessarily want to clobber it; you should repeat your
analysis from (2) and make a new decision.

The astute reader will notice that somebody might have pushed C between
steps (1) and (3), and therefore the second "push -f" may overwrite
their C, even though that was not what was intended by the user. The git
protocol already says "I am expecting to move ref R from A to B". The
problem is that the interface is mis-designed for this workflow. The
first push should note "A" somewhere on disk, and the second "git push
-f" should really be "git push --retry-with-force", and check to make
sure that our starting point really is "A".

I suspect nobody has complained so far because it is a race condition
that just doesn't come up unless you have an extremely busy repository
with lots of force pushes.

> And what about the fast forward case?  If client A is pushing change a
> and then client B starts pushing b which depends on a, why shouldn't B
> succeed as long as A succeeds first, even without the force flag?  But
> it does not work this way even with the force flag.

Right. Because the force flag is about history rewinds, not about
clobbering simultaneous pushes.

Don't get me wrong; I can see the use for an option to clobber a
simultaneous push in some workflows. I just think that there is value in
distinguishing it from what "force" does now.

> I think this use case distinction is irrelevant to users on a shared
> server unless you could somehow tell git to "only clobber the branch
> if it currently points to shax".  Which is sort of what git is
> pretending to do, but it isn't really since the user isn't specifying
> shax, it is determined when the push starts and is inherently subject
> to a race.  It's not like the user can safely determine what the
> remote branch points to before they update it to see if it is ok to
> clobber it.

Right. And I think that is a bug (or an interface misdesign), albeit one
that doesn't come up much. But I'd rather not make it worse by losing
what safety we have.

-Peff