Working copy revision and push pain

Working copy revision and push pain

[Jonathan Watt]

Hi,

There seems to be a problem with git-push when the working copy of the directory
being pushed to came from the magic revision HEAD, but not when the working copy
came from some other revision.

Say I've got a repository called a:

  mkdir a
  cd a
  git-init
  echo "line 1" > file
  git-add file
  git-commit -m "commit 1" file
  echo "line 2" >> file
  git-commit -m "commit 2" file
  echo "line 3" >> file
  git-commit -m "commit 3" file

Now let's say there's a clone of repository a called b, and b has HEAD checked
out. If I make changes in a and push them to b, there's a problem:

  git-clone . ../b
  echo "line 4" >> file
  git-commit -m "commit 4" file
  git-push ../b
  cd ../b
  git-log file # says the working copy came from commit 4, but...
  cat file # only lists up to line 3 :-/

If on the other hand I have a clone of repository a called c that has a non-HEAD
revision checked out, there's no problem:

  git-clone ../a ../c
  cd ../c
  git-checkout HEAD^
  cd ../a
  echo "line 5" >> file
  git-commit -m "commit 5" file
  git-push ../c
  cd ../c
  git-log file # says the working copy came from commit 3, and...
  cat file # only lists up to line 3. great!

I don't know anything about the implementation of git, but could this mean that
when the working copy comes from HEAD git tracks it as coming from the revision
"HEAD" instead of the underlying revision's sha1? If so, this seems like an
unnecessary gotcha/pain point. Couldn't git be changed to always track the sha1
of the revision it really came from? That would seem like the correct thing to
do, and it would be nice to avoid unnecessarily breaking working copies.

Regards,
Jonathan

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Jonathan Watt wrote:

> There seems to be a problem with git-push when the working copy of the 
> directory being pushed to came from the magic revision HEAD, but not 
> when the working copy came from some other revision.

http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73

Sidenote: I am constantly amazed how people have no problem accepting that 
a CVS-Server has no working directory, but all of a sudden think that a 
Git-server should have one, and auto-update it.

Ciao,
Dscho

Re: Working copy revision and push pain

[Jonathan Watt]

Johannes Schindelin wrote:
> Hi,
> 
> On Sun, 23 Mar 2008, Jonathan Watt wrote:
> 
>> There seems to be a problem with git-push when the working copy of the 
>> directory being pushed to came from the magic revision HEAD, but not 
>> when the working copy came from some other revision.
> 
> http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73
> 
> Sidenote: I am constantly amazed how people have no problem accepting that 
> a CVS-Server has no working directory, but all of a sudden think that a 
> Git-server should have one, and auto-update it.

Hi Dscho. I think you've misread my email. (Or not read it. ;-)) I do not expect
git-push to update the working copy of the repository being pushed to. In fact
my complaint would be more that it *does* appear to modify the working copy
(well, not so much modify the working copy as get confused about which revision
the working copy came from) when the working copy came from HEAD.

Jonathan

Re: Working copy revision and push pain

[Elijah Newren]

On Sun, Mar 23, 2008 at 7:19 AM, Jonathan Watt <jwatt@jwatt.org> wrote:
> Johannes Schindelin wrote:
>  > Hi,
>  >
>  > On Sun, 23 Mar 2008, Jonathan Watt wrote:
>  >
>  >> There seems to be a problem with git-push when the working copy of the
>  >> directory being pushed to came from the magic revision HEAD, but not
>  >> when the working copy came from some other revision.
>  >
>  > http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73
>  >
>  > Sidenote: I am constantly amazed how people have no problem accepting that
>  > a CVS-Server has no working directory, but all of a sudden think that a
>  > Git-server should have one, and auto-update it.
>
>  Hi Dscho. I think you've misread my email. (Or not read it. ;-)) I do not expect
>  git-push to update the working copy of the repository being pushed to. In fact
>  my complaint would be more that it *does* appear to modify the working copy
>  (well, not so much modify the working copy as get confused about which revision
>  the working copy came from) when the working copy came from HEAD.

Ah, I hadn't thought of it that way before.  I think you are
suggesting that pushing to the active branch of a repository with an
associated working copy should cause the HEAD to become detached.  Is
that right?  That's an interesting new twist to this old problem that
I personally haven't seen before (though I am kind of new and have
only had a little time to dig up archives about this problem).  Hmm...

Re: Working copy revision and push pain

[Jonathan Watt]

Elijah Newren wrote:
> On Sun, Mar 23, 2008 at 7:19 AM, Jonathan Watt <jwatt@jwatt.org> wrote:
>> Johannes Schindelin wrote:
>>  > Hi,
>>  >
>>  > On Sun, 23 Mar 2008, Jonathan Watt wrote:
>>  >
>>  >> There seems to be a problem with git-push when the working copy of the
>>  >> directory being pushed to came from the magic revision HEAD, but not
>>  >> when the working copy came from some other revision.
>>  >
>>  > http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73
>>  >
>>  > Sidenote: I am constantly amazed how people have no problem accepting that
>>  > a CVS-Server has no working directory, but all of a sudden think that a
>>  > Git-server should have one, and auto-update it.
>>
>>  Hi Dscho. I think you've misread my email. (Or not read it. ;-)) I do not expect
>>  git-push to update the working copy of the repository being pushed to. In fact
>>  my complaint would be more that it *does* appear to modify the working copy
>>  (well, not so much modify the working copy as get confused about which revision
>>  the working copy came from) when the working copy came from HEAD.
> 
> Ah, I hadn't thought of it that way before.  I think you are
> suggesting that pushing to the active branch of a repository with an
> associated working copy should cause the HEAD to become detached.  Is
> that right?

To be honest, I'm not sure what you mean by "HEAD to become detached". If you
mean that the git-push should, if necessary, stop associating the working copy
with HEAD if it's going to change HEAD, then absolutely. It wasn't the same
solution as I was thinking of (stop associating the working copy with HEAD and
instead associate it with the sha1 HEAD currently points to), but I guess it's
the same result. :-)

Jonathan

Re: Working copy revision and push pain

[Elijah Newren]

On Sun, Mar 23, 2008 at 7:54 AM, Jonathan Watt <jwatt@jwatt.org> wrote:
> Elijah Newren wrote:
>  > On Sun, Mar 23, 2008 at 7:19 AM, Jonathan Watt <jwatt@jwatt.org> wrote:
>  >> Johannes Schindelin wrote:
>  >>  > Hi,
>  >>  >
>  >>  > On Sun, 23 Mar 2008, Jonathan Watt wrote:
>  >>  >
>  >>  >> There seems to be a problem with git-push when the working copy of the
>  >>  >> directory being pushed to came from the magic revision HEAD, but not
>  >>  >> when the working copy came from some other revision.
>  >>  >
>  >>  > http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73
>  >>  >
>  >>  > Sidenote: I am constantly amazed how people have no problem accepting that
>  >>  > a CVS-Server has no working directory, but all of a sudden think that a
>  >>  > Git-server should have one, and auto-update it.
>  >>
>  >>  Hi Dscho. I think you've misread my email. (Or not read it. ;-)) I do not expect
>  >>  git-push to update the working copy of the repository being pushed to. In fact
>  >>  my complaint would be more that it *does* appear to modify the working copy
>  >>  (well, not so much modify the working copy as get confused about which revision
>  >>  the working copy came from) when the working copy came from HEAD.
>  >
>  > Ah, I hadn't thought of it that way before.  I think you are
>  > suggesting that pushing to the active branch of a repository with an
>  > associated working copy should cause the HEAD to become detached.  Is
>  > that right?
>
>  To be honest, I'm not sure what you mean by "HEAD to become detached". If you
>  mean that the git-push should, if necessary, stop associating the working copy
>  with HEAD if it's going to change HEAD, then absolutely. It wasn't the same
>  solution as I was thinking of (stop associating the working copy with HEAD and
>  instead associate it with the sha1 HEAD currently points to), but I guess it's
>  the same result. :-)
>
>  Jonathan

In git, HEAD always refers to the currently active branch...if there
is one.  (Also note that each branch tracks its most recent commit.)
If there is no currently active branch because you checked out a tag
or some arbitrary commit, then HEAD is said to be detached, and HEAD
will track the particular commit you checked out.  The end result is
that HEAD is always the most recent commit to which your working copy
is relative to.  See also
http://www.kernel.org/pub/software/scm/git/docs/glossary.html

So, it sounds like we're both saying that in your case, you'd like the
HEAD become detached and track the sha1 that it previously pointed to
before your push rather than continuing to track the updated branch.

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Elijah Newren wrote:

> If there is no currently active branch because you checked out a tag
> or some arbitrary commit, then HEAD is said to be detached, and HEAD
> will track the particular commit you checked out.

I'd say that "track" is the wrong verb here.

> The end result is that HEAD is always the most recent commit to which 
> your working copy is relative to.  See also 
> http://www.kernel.org/pub/software/scm/git/docs/glossary.html
> 
> So, it sounds like we're both saying that in your case, you'd like the
> HEAD become detached and track the sha1 that it previously pointed to
> before your push rather than continuing to track the updated branch.

If you ever propose to detach the HEAD in a remote repository when 
somebody pushed into the referenced branch, I am totally opposed to that.

You can play your own games with the update and post-update hooks as much 
as you want, but please leave official Git alone.  Thankyouverymuch.

Ciao,
Dscho

Re: Working copy revision and push pain

[Jonathan Watt]

Johannes Schindelin wrote:
>> The end result is that HEAD is always the most recent commit to which 
>> your working copy is relative to.  See also 
>> http://www.kernel.org/pub/software/scm/git/docs/glossary.html
>>
>> So, it sounds like we're both saying that in your case, you'd like the
>> HEAD become detached and track the sha1 that it previously pointed to
>> before your push rather than continuing to track the updated branch.
> 
> If you ever propose to detach the HEAD in a remote repository when 
> somebody pushed into the referenced branch, I am totally opposed to that.

The alternative is to leave the working copy in a totally broken state. I'm not
sure why that would be preferable.

> You can play your own games with the update and post-update hooks as much 
> as you want, but please leave official Git alone.  Thankyouverymuch.

My hope here is for a reduction in the pain/time taken out from real work in the
OSS developer world while people are moving to DSCMs - particularly for those
that have to learn/move between both Mercurial and git. Requiring people to be
aware of this gotcha and to use solutions like hooks seems to raise barriers
rather than eliminate them.

Cloning and working with multiple repositories locally is a particularly
Mercurial workflow. I see this issue as a barrier to those who've cut their DSCM
teeth on Mercurial and are trying to use git. Would detaching HEAD really be a
problem in git workflows?

Jonathan

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Jonathan Watt wrote:

> Would detaching HEAD really be a problem in git workflows?

If recovering is painful?  Yes.

If it is not what you want?  Yes.

Just to give you a small clue what other people would like: 
http://utsl.gen.nz/git/post-update

BTW that was in the link I sent you earlier.

Ciao,
Dscho

Re: Working copy revision and push pain

[Jonathan Watt]

Johannes Schindelin wrote:
> Hi,
> 
> On Sun, 23 Mar 2008, Jonathan Watt wrote:
> 
>> Would detaching HEAD really be a problem in git workflows?
> 
> If recovering is painful?  Yes.
> 
> If it is not what you want?  Yes.

If you push to the current branch of a non-bare repository, then as things
stand, whatever was pushed to the current branch will be reverted on the next
commit(!). Push hasn't just pushed changesets to the repository, it's
essentially screwed up the working copy. I personally can't see how this
behavior is useful or in any way "right", or why you'd want it that way. On the
other hand detaching HEAD makes sure that all push has done is push changesets,
and you've essentially created a branch. To me that seems like the only correct
thing to do. (It also happens to be the way Mercurial behaves.)

I'd venture that if you're pushing into a non-bare repository then you know the
state of the working copy, and you know if you're going to cause pain. (And it's
probably local.) If it's a shared repository, why is there a working copy? If
it's someone else's repository, the two of you probably know what you're doing.
These two seem like relative edge cases though, and even then, detaching HEAD
seems like the right think to me.

> Just to give you a small clue what other people would like: 
> http://utsl.gen.nz/git/post-update
> 
> BTW that was in the link I sent you earlier.

As I said, I don't want to update the files in the working copy. Seems like a
different issue to me.

Jonathan

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Jonathan Watt wrote:

> Johannes Schindelin wrote:
> 
> > Just to give you a small clue what other people would like: 
> > http://utsl.gen.nz/git/post-update
> > 
> > BTW that was in the link I sent you earlier.
> 
> As I said, I don't want to update the files in the working copy. Seems 
> like a different issue to me.

I understood that.  *yawns*

But I tried to tell you (unsuccessfully, it seems) that this is not 
desirable to all users, and therefore it would be a horrible, horrible 
idea to force the behaviour on other people you seemed to try to force on 
other people.

'nuff said,
Dscho

Auto detaching head options (Re: Working copy revision and push pain)

[Jan Hudec]

On Sun, Mar 23, 2008 at 17:00:12 +0100, Johannes Schindelin wrote:
> On Sun, 23 Mar 2008, Jonathan Watt wrote:
> > As I said, I don't want to update the files in the working copy. Seems 
> > like a different issue to me.
> 
> I understood that.  *yawns*
> 
> But I tried to tell you (unsuccessfully, it seems) that this is not 
> desirable to all users, and therefore it would be a horrible, horrible 
> idea to force the behaviour on other people you seemed to try to force on 
> other people.

The proponents of this (and I also) think, that meaning of HEAD is, or rather
should be, "the revision your work tree is derived from". If you think it
should be defined otherwise, can you please say how, so the merits of the
definitions can be discussed?

Note, that that definition implies that HEAD makes no sense for bare
repository. I think HEAD indeed does not make sense there, but you may of
course provide different definition where it does.

Also I believe that it would be very useful to have a ref defined "the
revision your work tree is derived from", be it HEAD or not. It would:
 - make the behaviour of push to non-bare repository defined.
 - make additional work trees for a repository (created by
   contrib/workdir/git-new-workdir) safe (the mechanizm could than be an
   option for submodules).

I see two ways to add something to keep track of work tree base version:

1. Make HEAD a special kind of ref, that would contain *both* the symbolic
   name *and* sha1. It would behave as symbolic as long as the sha1 values
   match and as detached when they don't.

   This should actually be relatively small code change, because it would be
   confined to update-ref, symbolic-ref and revparse. It would however be
   quite significant change to the repository format.

2. Create a new special ref BASE? (WORKTREE?), with the right semantics.

   The code change would be a bit more invasive here, as more commands would
   be affected (though I hope the commands share internal code so it would
   only be one checkout and one commit path). It would be smaller change to
   the repository format. It would, however, more stick out, because to find
   out which branch you will commit to would require looking at both HEAD and
   BASE.

   It would really be similar to the revision number in index proposal,
   except less invasive and I actually believe there is a case (some form of
   checkout or reset), where we want to read-tree, but not change this ref.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: Auto detaching head options (Re: Working copy revision and push pain)

[Johannes Schindelin]

Hi,

On Tue, 25 Mar 2008, Jan Hudec wrote:

> The proponents of this (and I also) think, that meaning of HEAD is, or 
> rather should be, "the revision your work tree is derived from".

By far the most operation I do with HEAD is to _advance_ it.  So for me, 
HEAD must be a pointer to the ref which is to be advanced.

I don't care about detaching the HEAD when pushing into the current 
branch, which you are not supposed to do anyway.  Really.  I could not 
care less about that.

You can do it -- I already pointed out that you can do that with hooks -- 
but please do not bother me with it.

Hth,
Dscho

Re: Auto detaching head options (Re: Working copy revision and push pain)

[Jeff King]

On Tue, Mar 25, 2008 at 08:25:52PM +0100, Jan Hudec wrote:

> The proponents of this (and I also) think, that meaning of HEAD is, or rather
> should be, "the revision your work tree is derived from". If you think it
> should be defined otherwise, can you please say how, so the merits of the
> definitions can be discussed?

I agree with Johannes. It has traditionally been considered "the ref
that will be advanced by new commits". And that is why I think detaching
the HEAD is just trading one problem for another: your push invisibly
mucks with state that the non-bare repository user relies on. When they
commit on a detached HEAD, their commits suddenly start going "nowhere"
instead of to the branch they thought.

FWIW, I also initially thought this was only a "HEAD" problem, but I
think Junio's recent argument makes a lot of sense: the problem is not
one of working tree and HEAD sync, nor even of detached versus ref HEAD.
The problem is that somebody is using the non-bare repository to do
stuff (why else would it be non-bare), and you are changing the state
behind their back.

> Note, that that definition implies that HEAD makes no sense for bare
> repository. I think HEAD indeed does not make sense there, but you may of
> course provide different definition where it does.

I think HEAD can be more generally considered the context of the
"current" branch. It is the default branch for cloning from bare
repositories, and it is the default branch for most operations (logging,
starting new branches, etc). So it really is a bit overloaded in that
sense.

> Also I believe that it would be very useful to have a ref defined "the
> revision your work tree is derived from", be it HEAD or not. It would:

Isn't this essentially the 'base' index extension that Junio did a
while back? It was eventually reverted (or perhaps never merged, I don't
recall). You should be able to find discussion in the list archives.

But maybe you are referencing it here:

>    It would really be similar to the revision number in index proposal,
>    except less invasive and I actually believe there is a case (some form of
>    checkout or reset), where we want to read-tree, but not change this ref.

I don't recall the reasons the base extension was not accepted, but I
think it would make sense to frame your argument as "this is like X;
people didn't like X for reason Y, but my proposal fixes this by..."

-Peff

Re: Auto detaching head options (Re: Working copy revision and push pain)

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> FWIW, I also initially thought this was only a "HEAD" problem, but I
> think Junio's recent argument makes a lot of sense: the problem is not
> one of working tree and HEAD sync, nor even of detached versus ref HEAD.
> The problem is that somebody is using the non-bare repository to do
> stuff (why else would it be non-bare), and you are changing the state
> behind their back.

For people who are overwhelmed by the volume of the list traffic, the
relevant thread is:

  http://thread.gmane.org/gmane.comp.version-control.git/77955/focus=78062

> Isn't this essentially the 'base' index extension that Junio did a
> while back? It was eventually reverted (or perhaps never merged, I don't
> recall).

It was in 'next' for a while but was reverted before it hit 'master':

  http://thread.gmane.org/gmane.comp.version-control.git/44360/focus=44508

> But maybe you are referencing it here:
>
>>    It would really be similar to the revision number in index proposal,
>>    except less invasive and I actually believe there is a case (some form of
>>    checkout or reset), where we want to read-tree, but not change this ref.
>
> I don't recall the reasons the base extension was not accepted, but I
> think it would make sense to frame your argument as "this is like X;
> people didn't like X for reason Y, but my proposal fixes this by..."

But if you take the position "work tree belongs to the repository owner
and nobody has any business pushing into it sideways; push into acceptance
branch and let the work tree owner merge it when able", the history of the
failed index base experiment becomes irrelevant.

Auto detaching head options (update) (Re: Working copy revision and push pain)

[Jan Hudec]

Hello All,

Since the original proposal is not quoted here, I should repeat it:
On Tue, Mar 25, Jan Hudec wrote:
>     Also I believe that it would be very useful to have a ref defined "the
>     revision your work tree is derived from", be it HEAD or not. It would:
>      - make the behaviour of push to non-bare repository defined.
>      - make additional work trees for a repository (created by
>        contrib/workdir/git-new-workdir) safe (the mechanizm could than be an
>        option for submodules).

First of all, I should apologise for proposing to introduce something that
already exists. It's been resting in the reflog all the time.

Excluding push to non-bare repository and extra work trees, the local
branches are only ever updated through the HEAD. And as long as they are
updated with update-ref, it writes the value to the reflog, so HEAD^{0}
should always equal HEAD.

Therefore the option to prevent users shooting themselves in the foot when
using multiple worktrees for the same repository (the more useful case) or
pushing to a non-bare repo (which is usually just shooting oneselves in the
feet because of lack of knowledge) would be to introduce an option to commit
to verify, that rev-parse HEAD and rev-parse HEAD^{0} give the same revision.

That would have only one casuality -- currently to create a new parentless
commit in non-new repository, one has to modify the HEAD manually. But that
just needs a command-line override to disable the check for cases when one
knows what he's doing.

On Wed, Mar 26, 2008 at 11:49:44 -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
> > FWIW, I also initially thought this was only a "HEAD" problem, but I
> > think Junio's recent argument makes a lot of sense: the problem is not
> > one of working tree and HEAD sync, nor even of detached versus ref HEAD.
> > The problem is that somebody is using the non-bare repository to do
> > stuff (why else would it be non-bare), and you are changing the state
> > behind their back.
> 
> For people who are overwhelmed by the volume of the list traffic, the
> relevant thread is:
> 
>   http://thread.gmane.org/gmane.comp.version-control.git/77955/focus=78062

Thanks for the pointer.

There are actually two use-cases for this. One is this pushing to non-bare
repo, which is indeed just shooting oneself in the foot out of lack of
knowledge most of the time. Though it would still be nice to ask user "do you
really want to shoot yourself in the foot?" if we can find out.

But the other usecase is using additional work trees
(contrib/workdir/git-new-workdir), which is rather useful for many people.
There it again does not make sense to commit to the same branch from two
work-trees, but it may easily happen by accident. Being able to check and
prevent such accidents would improve support for extra work-trees a lot,
maybe even to a point it could be made official.

> > Isn't this essentially the 'base' index extension that Junio did a
> > while back? It was eventually reverted (or perhaps never merged, I don't
> > recall).
> 
> It was in 'next' for a while but was reverted before it hit 'master':
> 
>   http://thread.gmane.org/gmane.comp.version-control.git/44360/focus=44508

Again, thanks for the pointer.

> > But maybe you are referencing it here:
> >
> >>    It would really be similar to the revision number in index proposal,
> >>    except less invasive and I actually believe there is a case (some form of
> >>    checkout or reset), where we want to read-tree, but not change this ref.
> >
> > I don't recall the reasons the base extension was not accepted, but I
> > think it would make sense to frame your argument as "this is like X;
> > people didn't like X for reason Y, but my proposal fixes this by..."
> 
> But if you take the position "work tree belongs to the repository owner
> and nobody has any business pushing into it sideways; push into acceptance
> branch and let the work tree owner merge it when able", the history of the
> failed index base experiment becomes irrelevant.

Ok. This proposal is indeed similar to the base-in-index one, but has some
significant differences. See below.

Besides whith the update above, I no longer propose to keep any new
information -- just to add a safety-check.

On 2007-04-15 19:56:55 GMT, Junio C Hamano wrote:
> [...]
> This is primarily because updating the branch tip is not tied
> closely to writing out a commit, and writing out a commit is not
> tied closely to writing out a tree (to be contained in that
> commit) out of the index.
> [...]

This is a big difference between the base-in-index and
base-somewhere-separately. The reflog is indeed tied to updating of the
branch tip, which is what we need here. It also does not add any new logic to
the low-level plumbing (read-tree, write-tree), that the base-in-index
proposal did.

For the BASE ref proposal, it would be equally tied to update of the branch
tip, except that all porcelain that does it would have to be changed, while
the reflog it already updates.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: Auto detaching head options (update) (Re: Working copy revision and push pain)

[Jeff King]

On Sat, Mar 29, 2008 at 09:27:57AM +0100, Jan Hudec wrote:

> Excluding push to non-bare repository and extra work trees, the local
> branches are only ever updated through the HEAD. And as long as they are
> updated with update-ref, it writes the value to the reflog, so HEAD^{0}
> should always equal HEAD.

I assume by HEAD^{0} you mean HEAD@{0}? The first is for peeling tags,
while the latter is "the latest reflog entry."

There are a few other ways I can think of to manipulate branches besides
through HEAD:

  - git branch -f existing new_ref (though I'm not sure if this works if
    the branch is HEAD)

  - git update-ref refs/heads/branch_that_is_HEAD_but_we_call_it_by_name

However, in both cases the HEAD ref is being munged behind the user's
back to some degree, and it can result in a situation where the index
doesn't match HEAD anymore. So in both cases, I think the safety check
you propose would actually be a benefit.

But I'm not sure if there aren't some "innocent" uses of the second form
in people's scripts. For example, a script that operates on HEAD but
resolves it internally to "refs/heads/master" and begins using that (and
takes care itself to keep the index in sync with the new value written
into the ref).

> There are actually two use-cases for this. One is this pushing to non-bare
> repo, which is indeed just shooting oneself in the foot out of lack of
> [...]
> But the other usecase is using additional work trees

I like the fact that it deals with both these cases, and I think it can
cover even more. It is a general safety valve for "somebody changed this
ref while you weren't looking."

> Besides whith the update above, I no longer propose to keep any new
> information -- just to add a safety-check.

And I like that, too. But I am a little worried that the information
will not be sufficient. You are asking for a consistency guarantee of
HEAD and HEAD@{0} that comes from using current porcelain. But there are
many uses of plumbing that would violate that consistency guarantee. And
some of those are exactly what you're trying to catch and warn about.
But I suspect there are some false positives (either in people's
plumbing-oriented workflows, in other scripts outside of core git, or
even in core git itself (I didn't carefully audit how each script
works)). And maybe it is OK to warn and force a "-f" there, or cause
those scripts to be updated.

By all of these "maybes" what I am saying is "I think this is a
promising approach but I am not entirely convinced it won't run into
implementation problems." And of course producing a patch is a good
first step in figuring it out. :)

-Peff

Re: Auto detaching head options (update) (Re: Working copy revision and push pain)

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Sat, Mar 29, 2008 at 09:27:57AM +0100, Jan Hudec wrote:
>
> I like the fact that it deals with both these cases, and I think it can
> cover even more. It is a general safety valve for "somebody changed this
> ref while you weren't looking."
>
>> Besides whith the update above, I no longer propose to keep any new
>> information -- just to add a safety-check.
>
> And I like that, too. But I am a little worried that the information
> will not be sufficient. You are asking for a consistency guarantee of
> HEAD and HEAD@{0} that comes from using current porcelain.

While I too find the proposal attractive as a concept, I am sorry to say
that it would not work even with the current set of Porcelain (fwiw, when
the index-base extension was done, the approach based on HEAD reflog would
not have worked either).  This is because 605fac8 (update HEAD reflog when
branch pointed to by HEAD is directly modified, 2007-03-21) made gremlin
updates to the current branch tip to also update the reflog of HEAD.

If we revert that commit, we may be able to.

Re: Auto detaching head options (update) (Re: Working copy revision and push pain)

[Jeff King]

On Sun, Mar 30, 2008 at 06:53:56PM -0700, Junio C Hamano wrote:

> While I too find the proposal attractive as a concept, I am sorry to say
> that it would not work even with the current set of Porcelain (fwiw, when
> the index-base extension was done, the approach based on HEAD reflog would
> not have worked either).  This is because 605fac8 (update HEAD reflog when
> branch pointed to by HEAD is directly modified, 2007-03-21) made gremlin
> updates to the current branch tip to also update the reflog of HEAD.
> 
> If we revert that commit, we may be able to.

Eww. I didn't know we were doing that. In fact, that was one of the
cases I mentioned as "this would cause the reflog to get out of sync,
but that is a good thing because it would trigger your safety valve."
But obviously I was wrong about how we behave now.

Why is 605fac8 beneficial? What common workflow is updating the HEAD
branch through its actual name? If it is manipulating the index and
worktree at the same time, then it should be using the name HEAD. If it
isn't, then shouldn't the HEAD and index get out of sync (exactly the
thing Jan's proposal is meant to protect against)?

-Peff

Re: Auto detaching head options (update) (Re: Working copy revision and push pain)

[Jeff King]

On Sun, Mar 30, 2008 at 09:59:37PM -0400, Jeff King wrote:

> Why is 605fac8 beneficial? What common workflow is updating the HEAD
> branch through its actual name? If it is manipulating the index and

Apparently my reading skills are not what they used to be. In the
comment from 605fac8, it specifically mentions pushing directly into a
branch that is the same as HEAD. But that is one of the cases that we
Jan is trying to improve, so I don't see a problem with reverting in
favor of his proposal.

-Peff

Re: Working copy revision and push pain

[Elijah Newren]

Hi,

On Sun, Mar 23, 2008 at 8:22 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>  On Sun, 23 Mar 2008, Elijah Newren wrote:
>
>  > If there is no currently active branch because you checked out a tag
>  > or some arbitrary commit, then HEAD is said to be detached, and HEAD
>  > will track the particular commit you checked out.
>
>  I'd say that "track" is the wrong verb here.

Good point.

>  > The end result is that HEAD is always the most recent commit to which
>  > your working copy is relative to.  See also
>  > http://www.kernel.org/pub/software/scm/git/docs/glossary.html
>  >
>  > So, it sounds like we're both saying that in your case, you'd like the
>  > HEAD become detached and track the sha1 that it previously pointed to
>  > before your push rather than continuing to track the updated branch.
>
>  If you ever propose to detach the HEAD in a remote repository when
>  somebody pushed into the referenced branch, I am totally opposed to that.
>
>  You can play your own games with the update and post-update hooks as much
>  as you want, but please leave official Git alone.  Thankyouverymuch.

Obviously such a change in isolation would merely shift to a new kind
of gotcha for users; I was thinking that I'd be able to figure out an
accompanying change that would make things work nicely (or that
someone else on the list might think of something)...but no such luck.
 Further, I think Junio's latest post describes why I likely couldn't
come up with anything else and the idea is just plain bad.

Elijah

Re: Working copy revision and push pain

[Jonathan Watt]

Elijah Newren wrote:
> On Sun, Mar 23, 2008 at 7:54 AM, Jonathan Watt <jwatt@jwatt.org> wrote:
>> Elijah Newren wrote:
>>  > On Sun, Mar 23, 2008 at 7:19 AM, Jonathan Watt <jwatt@jwatt.org> wrote:
>>  >>  Hi Dscho. I think you've misread my email. (Or not read it. ;-)) I do not expect
>>  >>  git-push to update the working copy of the repository being pushed to. In fact
>>  >>  my complaint would be more that it *does* appear to modify the working copy
>>  >>  (well, not so much modify the working copy as get confused about which revision
>>  >>  the working copy came from) when the working copy came from HEAD.
>>  >
>>  > Ah, I hadn't thought of it that way before.  I think you are
>>  > suggesting that pushing to the active branch of a repository with an
>>  > associated working copy should cause the HEAD to become detached.  Is
>>  > that right?
>>
>>  To be honest, I'm not sure what you mean by "HEAD to become detached". If you
>>  mean that the git-push should, if necessary, stop associating the working copy
>>  with HEAD if it's going to change HEAD, then absolutely. It wasn't the same
>>  solution as I was thinking of (stop associating the working copy with HEAD and
>>  instead associate it with the sha1 HEAD currently points to), but I guess it's
>>  the same result. :-)
> 
> In git, HEAD always refers to the currently active branch...if there
> is one.  (Also note that each branch tracks its most recent commit.)
> If there is no currently active branch because you checked out a tag
> or some arbitrary commit, then HEAD is said to be detached, and HEAD
> will track the particular commit you checked out.  The end result is
> that HEAD is always the most recent commit to which your working copy
> is relative to.  See also
> http://www.kernel.org/pub/software/scm/git/docs/glossary.html

I see. Thanks.

> So, it sounds like we're both saying that in your case, you'd like the
> HEAD become detached and track the sha1 that it previously pointed to
> before your push rather than continuing to track the updated branch.

Yes, indeed. From my novice perspective it seems like that's absolutely what
should happen, since that's where the working copy came from. Certainly I see no
reason for git-push to leave the working copy/revision relationship in a bad state.

Jonathan

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Jonathan Watt wrote:

> Elijah Newren wrote:
>
> > So, it sounds like we're both saying that in your case, you'd like the 
> > HEAD become detached and track the sha1 that it previously pointed to 
> > before your push rather than continuing to track the updated branch.
> 
> Yes, indeed. From my novice perspective it seems like that's absolutely 
> what should happen, since that's where the working copy came from. 
> Certainly I see no reason for git-push to leave the working 
> copy/revision relationship in a bad state.

This is a horrible idea.

If you want to push into non-bare repositories, then please, please, 
please read the tutorials on that subject.

Thanks,
Dscho

Re: Working copy revision and push pain

[Johan Herland]

On Sunday 23 March 2008, Johannes Schindelin wrote:
> On Sun, 23 Mar 2008, Jonathan Watt wrote:
> > Elijah Newren wrote:
> If you want to push into non-bare repositories, then please, please, 
> please read the tutorials on that subject.

I have personally no problem with the current default behaviour of push (in the few cases I push into non-bare repos, I've set up the proper hooks to do what I want), but I frequently get questions from colleagues about why Git behaves like this.

I'm starting to think it's worth changing the default behaviour of push as follows:

Upon receiving a push into a non-bare repository, if the working copy is on the same branch as is being pushed, then refuse the push with a helpful message describing why the push was refused, and how to resolve this issue (i.e. referring to the tutorials you mention).

This would:
- Not clobber the working copy
- Tell newbies what happened and why
- Hopefully make this issue pop up less frequently
- Not affect you if you only push into bare repos
- Not affect you if you take care to never push into a checked-out branch

Of course, you should be able to set a config option to get the old behaviour, and from there you can write hooks to either update the working copy, or detach HEAD, or whatever you please.


Is this acceptable to people?

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: Working copy revision and push pain

[Jonathan Watt]

Johan Herland wrote:
> I'm starting to think it's worth changing the default behaviour of push as follows:
> 
> Upon receiving a push into a non-bare repository, if the working copy is on the same branch as is being pushed, then refuse the push with a helpful message describing why the push was refused, and how to resolve this issue (i.e. referring to the tutorials you mention).
> 
> This would:
> - Not clobber the working copy
> - Tell newbies what happened and why
> - Hopefully make this issue pop up less frequently
> - Not affect you if you only push into bare repos
> - Not affect you if you take care to never push into a checked-out branch

The detach-HEAD idea does all these things, but rather:

- There's no need to tell newbies anything
- It don't just reduce the frequency of the problem, it eliminates it

:-) Also,

- You eliminate the problem of git thinking the working copy came from a
  revision it didn't come from, and thus eliminate the "any commit will
  now overwrite the push" problem
- You can still write hooks to update the working copy if you like
- It's completely intuitive to anyone coming from Mercurial (and it's these
  people who are going to be doing the pushing into non-bare repositories,
  because that's the workflow they're familiar with)

It might also be a good idea to print a warning about the working copy needing
to be updated or else committing changes will create a branch. That seems
obvious, and if you're pushing into a checked out branch, you probably know that
though.

> Of course, you should be able to set a config option to get the old behaviour, and from there you can write hooks to either update the working copy, or detach HEAD, or whatever you please.
> 
> Is this acceptable to people?

>From my perspective it makes things more complicated rather than simpler I'm afraid.

Johannes is apparently fed up of trying to explain to me why people want their
working copy associated with the wrong revision so that their commits will
overwrite the pull. ;-) If anyone else cares to give it ago I'd appreciate it,
since I still don't see the utility.

Jonathan
Hoping Johannes has a sense of humour...

Re: Working copy revision and push pain

[Junio C Hamano]

Jonathan Watt <jwatt@jwatt.org> writes:

> Johan Herland wrote:
>> I'm starting to think it's worth changing the default behaviour of push as follows:
>> 
>> Upon receiving a push into a non-bare repository, if the working copy is on the same branch as is being pushed, then refuse the push with a helpful message describing why the push was refused, and how to resolve this issue (i.e. referring to the tutorials you mention).
>> 
>> This would:
>> - Not clobber the working copy
>> - Tell newbies what happened and why
>> - Hopefully make this issue pop up less frequently
>> - Not affect you if you only push into bare repos
>> - Not affect you if you take care to never push into a checked-out branch
>
> The detach-HEAD idea does all these things, but rather:
>
> - There's no need to tell newbies anything
> - It don't just reduce the frequency of the problem, it eliminates it
>
> :-) Also,
>
> - You eliminate the problem of git thinking the working copy came from a
>   revision it didn't come from, and thus eliminate the "any commit will
>   now overwrite the push" problem
> - You can still write hooks to update the working copy if you like
> - It's completely intuitive to anyone coming from Mercurial (and it's these
>   people who are going to be doing the pushing into non-bare repositories,
>   because that's the workflow they're familiar with)

I am obviously sympathetic to people who are in the detox program from
Mercurial poisoning ;-), but seriously, I have to wonder if the detached
HEAD is the other way around from what merc does?

My understanding is if you push into a non-bare tree in merc, they make
the branch head into "unmerged" state, and as far as the work tree is
concerned you are still on the previous commit, and you merge "the other
commit" that was pushed into from sideways to update the branch tip with
pushed-into commit.

In git, if you detach HEAD when you push into a checked-out 'master', your
work tree won't be associated with 'master' anymore, and to merge the
pushed-into state into the history of 'master', you would need:

	# push into 'master' from elsewhere, which by your magic detaches
        # the HEAD in the pushed-into repository.
	elsewhere$ git push $non_bare_repo master

	# merge pushed-into change into old 'master' where your work tree
        # started from by merging master into detached HEAD.  Somehow you
        # need to know you were on master to be able to do this.  Also
        # because you magically detached HEAD, anything else you do after
        # such a push from sideways in this repository are not protected
        # by reflog for 'master' but only by reflog for 'HEAD'.
	here$ git merge master

	# update the master with the result of the merge
	here$ git push . HEAD:master

        # check out master again to get back to your original state.
        here$ git checkout master

The second point bothers me quite a lot, as you are assuming that the
user at the repository that was pushed into does _not_ know what is going
on, and may keep working on a detached HEAD _without knowing_ what is
going on.

There is an alternative, though.

Git does not use merc style "a single branch can have unmerged multiple
heads" paradigm, but it can be implemented more explicitly (look for
"mothership satellite" in the FAQ you were referred to earlier).

People in the Mercurial detox program may want to use the pattern of
pushing into acceptance branch and then merging from it when it is
convenient, like this:

	# push from sideways go to 'remotes/origin/master', and never to
        # 'master'.  Your 'master' that was checked out is not molested
        # by this push.
	elsewhere$ git push $non_bare_repo master:refs/remotes/origin/master

	# you can keep working on what you were doing without getting
	# molested by the above push.
	here$ hack, hack, commit, commit, all still on 'master'

        # when you are done with the work you were doing in the pushed-into
        # tree, and notice somebody pushed into your acceptance branch,
        # you can _choose to_ merge it in.
	here$ git merge remotes/origin/master

Re: Working copy revision and push pain

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jonathan Watt <jwatt@jwatt.org> writes:
>
> My understanding is if you push into a non-bare tree in merc, they make
> the branch head into "unmerged" state, and as far as the work tree is
> concerned you are still on the previous commit, and you merge "the other
> commit" that was pushed into from sideways to update the branch tip with
> pushed-into commit.
>
> In git, if you detach HEAD when you push into a checked-out 'master', your
> work tree won't be associated with 'master' anymore, and to merge the
> pushed-into state into the history of 'master', you would need:
> ...
> The second point bothers me quite a lot, as you are assuming that the
> user at the repository that was pushed into does _not_ know what is going
> on, and may keep working on a detached HEAD _without knowing_ what is
> going on.

Let's back up a bit and try again.

I was trying to say what Mercurial presents at the UI level may be much
nicer with respect to pushing into a checked out repository.  Their notion
of 'head' (and 'branch' in general) is quite different from ours in that
their branch can have more than one 'head's that the user can choose to
merge into its 'tip', and 'push' does not update the 'tip'.  It instead
adds a new (unmerged) 'head' or fast-forwards an existing 'head', and the
user can choose to merge them into 'tip' when convenient (please correct
me here, since I am talking from my reading their documentation and I may
have misread it).

In git, a branch is just a ref and points at a single commit, and push and
fetch updates a single ref.  There is no direct notion of "these other
refs are related to this branch ref and you may want to merge them" at the
low level plumbing layer, so if you push to update a ref, git does what it
is told to do.  The branch ref is directly updated, and won't get split to
be merged back later like Mercurial does.

My objection to the "always detach HEAD automatically" is because that
solution, while it may solve the issue of work tree and HEAD getting
inconsistent silently behind user's back, would introduce more confusion.

 - The repository loses where we were (so we would add a hack to record
   which branch you were on), and operations after that silent detaching
   will _not_ advance your local branch anymore, because the push took
   over the branch 'tip'.  If you switch to some other branch, you lose.

 - The final merge happens in the wrong direction.  The other commit is
   already called the official 'tip' of the branch because push took over
   the branch 'tip', and after merging it into your stale HEAD, you have
   to update 'master' with the merge made on the detached HEAD.

 - It's your repository and the commit you were on when this push from
   sideways happened should _stay_ as the official 'tip' of the branch,
   until you tell git so.  In that sense, pushing into a non-bare
   repository's refs/heads/ hierarchy directly is conceptually wrong,
   whether the branch in question is currently checked out or not.

Although we do not have their split branch heads, we do have a similar
concept by the name of "branch tracking".  Your 'master' is marked as
being related to 'remotes/origin/master' by "git clone" and by "git
checkout -b foo remotes/origin/foo", and recent "git checkout" to switch
to such a branch will report how diverged you are from the other party.

The mothership-satellite configuration described in FAQ is about using
this facility, because "git push" into a non-bare repository is exactly a
"git fetch" run in the reverse direction (i.e. fetching to pushed-into
repository from pushed-from repository).

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Jonathan Watt wrote:

> Hoping Johannes has a sense of humour...

I do.  However, I have to admit that I hate being ignored.  But then, I 
can always return the favour.

Ciao,
Dscho

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Johan Herland wrote:

> I'm starting to think it's worth changing the default behaviour of push 
> as follows:
> 
> Upon receiving a push into a non-bare repository, if the working copy is 
> on the same branch as is being pushed, then refuse the push with a 
> helpful message describing why the push was refused, and how to resolve 
> this issue (i.e. referring to the tutorials you mention).
> 
> This would:
> - Not clobber the working copy
> - Tell newbies what happened and why
> - Hopefully make this issue pop up less frequently
> - Not affect you if you only push into bare repos
> - Not affect you if you take care to never push into a checked-out 
>   branch
> 
> Of course, you should be able to set a config option to get the old 
> behaviour, and from there you can write hooks to either update the 
> working copy, or detach HEAD, or whatever you please.

I think I sent out a sensible patch, which does not change the behaviour 
in existing repositories.  This should be safer (read: nicer) for 
Git old-timers.

Ciao,
Dscho

Re: Working copy revision and push pain

[Johan Herland]

On Monday 24 March 2008, Johannes Schindelin wrote:
> On Sun, 23 Mar 2008, Johan Herland wrote:
> > I'm starting to think it's worth changing the default behaviour of push 
> > as follows:
> > 
> > Upon receiving a push into a non-bare repository, if the working copy is 
> > on the same branch as is being pushed, then refuse the push with a 
> > helpful message describing why the push was refused, and how to resolve 
> > this issue (i.e. referring to the tutorials you mention).
> > 
> > This would:
> > - Not clobber the working copy
> > - Tell newbies what happened and why
> > - Hopefully make this issue pop up less frequently
> > - Not affect you if you only push into bare repos
> > - Not affect you if you take care to never push into a checked-out 
> >   branch
> > 
> > Of course, you should be able to set a config option to get the old 
> > behaviour, and from there you can write hooks to either update the 
> > working copy, or detach HEAD, or whatever you please.
> 
> I think I sent out a sensible patch, which does not change the behaviour 
> in existing repositories.  This should be safer (read: nicer) for 
> Git old-timers.

Yes, I'm following the discussion between you and Junio with great interest. Will post if I have anything worthwhile to contribute. So far you both make a lot of sense.

Thanks a lot for the patch.


Have fun! :)

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: Working copy revision and push pain

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Jonathan Watt wrote:

> Johannes Schindelin wrote:
> 
> > On Sun, 23 Mar 2008, Jonathan Watt wrote:
> > 
> >> There seems to be a problem with git-push when the working copy of 
> >> the directory being pushed to came from the magic revision HEAD, but 
> >> not when the working copy came from some other revision.
> > 
> > http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73
> > 
> > Sidenote: I am constantly amazed how people have no problem accepting 
> > that a CVS-Server has no working directory, but all of a sudden think 
> > that a Git-server should have one, and auto-update it.
> 
> I think you've misread my email. (Or not read it. ;-))

Well, I did.  Thank you very much.

> I do not expect git-push to update the working copy of the repository 
> being pushed to. In fact my complaint would be more that it *does* 
> appear to modify the working copy (well, not so much modify the working 
> copy as get confused about which revision the working copy came from) 
> when the working copy came from HEAD.

You changed the current branch via push.  HEAD is the tip of the current 
branch.  <rhetoric>What else should happen?</rhetoric>

Your notion that the working tree has somehow a magic state that is not 
dependent on the current branch is somewhat strange to me.

Ciao,
Dscho

Re: Working copy revision and push pain

[Jonathan Watt]

Johannes Schindelin wrote:
> Hi,
>
> On Sun, 23 Mar 2008, Jonathan Watt wrote:
>
>> Johannes Schindelin wrote:
>>
>>> On Sun, 23 Mar 2008, Jonathan Watt wrote:
>>>
>>>> There seems to be a problem with git-push when the working copy of
>>>> the directory being pushed to came from the magic revision HEAD, but
>>>> not when the working copy came from some other revision.
>>> http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73
>>>
>>> Sidenote: I am constantly amazed how people have no problem accepting
>>> that a CVS-Server has no working directory, but all of a sudden think
>>> that a Git-server should have one, and auto-update it.
>> I think you've misread my email. (Or not read it. ;-))
>
> Well, I did.  Thank you very much.

Then my apologies. It just seems that you answered a different question.

>> I do not expect git-push to update the working copy of the repository
>> being pushed to. In fact my complaint would be more that it *does*
>> appear to modify the working copy (well, not so much modify the working
>> copy as get confused about which revision the working copy came from)
>> when the working copy came from HEAD.
>
> You changed the current branch via push.  HEAD is the tip of the current
> branch.  <rhetoric>What else should happen?</rhetoric>

I would expect it to do that, *and* note that the working copy didn't come from
what is the new head of HEAD.

> Your notion that the working tree has somehow a magic state that is not
> dependent on the current branch is somewhat strange to me.

I don't believe that's a notion that I have. I simply have a notion that git
should know which revision the working copy was checked out from, and not get
confused and think it came from a different revision. ;-)

Jonathan