git push to a non-bare repository

git push to a non-bare repository

[Matthieu Moy]

Hi,

I have a repository with a working tree on a machine A, did a clone to
another machine B and commited there locally.

I want my changes to get back into the first repository, so I did a
"push". The new commit is in the history, I can see it with "git log",
but the modifications are not in the working tree.

This time, it's OK: I didn't have any uncommited modifications on A,
so I just did a "git reset --hard HEAD" there.

But if I had some uncommited changes, "git reset --hard HEAD" means
data loss, which is precisely what I want to avoid by using a VCS. It
seems a solution is to do:

$ git reset --soft <commit-id-before-the-push>
$ git merge <commit-id-after-the-push>

But it means I have to remember <commit-id-before-the-push>.


I don't understand the design choice here: git had two options to
avoid this scenario:

1) update the working tree while doing the push. That's feasible with
   good performance since git is present on the server, but leaves the
   problem of possible conflicts.

2) let git remember what the local tree points to (not just the branch
   name, but the commit id itself, stored in a place that "git push"
   won't modify). Then, provide me a way to "update" to the latest
   revision.

Fyi, bzr does this. Indeed, in bzr, a branch (let's say "repository"
in the git vocabulary) with a working tree just means a working tree
(AKA lightweight checkout) located in the same directory as a branch.
The working tree knows which revision it corresponds to, and where to
find its branch. There's a "bzr update" command to get my working tree
to the head of the branch, keeping the uncommited changes.

I believe this idea is very much linked to the "Lightweight Checkout"
idea (listed on the SoC ideas), since, in the case of multiple working
directories sharing the same .git, you don't want a commit in one tree
to affect the others.

So, did I miss something? Is there anything on the todo-list?

Thanks,

-- 
Matthieu

Re: git push to a non-bare repository

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> I don't understand the design choice here: git had two options to
> avoid this scenario:

Actually, there are no such "design choices".  That's entirely
up to the repository owners to arrange post-update hook, to
allow you to do anything you want.  

The default is not to encourage people (who do not know what
they are doing anyway) to push into non-bare repository.

Re: git push to a non-bare repository

[Sam Vilain]

Junio C Hamano wrote:
>> I don't understand the design choice here: git had two options to
>> avoid this scenario:
>>     
>
> Actually, there are no such "design choices".  That's entirely
> up to the repository owners to arrange post-update hook, to
> allow you to do anything you want.  
>
> The default is not to encourage people (who do not know what
> they are doing anyway) to push into non-bare repository.
>   

Maybe it's worth making it an error (that can be forced) if you're
pushing to the head that's checked out in a non-bare repository ?

It's pretty nasty behaviour for people used to darcs / bzr et al.

Sam.

Re: git push to a non-bare repository

[Jakub Narebski]

Sam Vilain wrote:

> Junio C Hamano wrote:

>>> I don't understand the design choice here: git had two options to
>>> avoid this scenario:
>>
>> Actually, there are no such "design choices".  That's entirely
>> up to the repository owners to arrange post-update hook, to
>> allow you to do anything you want.  
>>
>> The default is not to encourage people (who do not know what
>> they are doing anyway) to push into non-bare repository.
>>   
> 
> Maybe it's worth making it an error (that can be forced) if you're
> pushing to the head that's checked out in a non-bare repository ?
> 
> It's pretty nasty behaviour for people used to darcs / bzr et al.

Perhaps it would be for the best.

BUT unless you arrange some fancy post-update hook you have two
sane choices:
 * push to bare repository, with 1:1 refs mapping
 * push to non-bare repository, but with mapping pushed refs on
   pushee to remotes refs (remote / tracking branches) on remote
   side.

In all other choices there madness lies... ;-)

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: git push to a non-bare repository

[Junio C Hamano]

Sam Vilain <sam@vilain.net> writes:

> Junio C Hamano wrote:
> ...
> Maybe it's worth making it an error (that can be forced) if you're
> pushing to the head that's checked out in a non-bare repository ?

We talked about that in the past on the list.  No.

Re: git push to a non-bare repository

[Theodore Tso]

On Sun, Mar 18, 2007 at 06:31:21PM +0100, Matthieu Moy wrote:
> I have a repository with a working tree on a machine A, did a clone to
> another machine B and commited there locally.
> 
> I want my changes to get back into the first repository, so I did a
> "push". The new commit is in the history, I can see it with "git log",
> but the modifications are not in the working tree.

The general answer (which you've already received) is to tell folks is
to simply don't use "git push" to remote trees; basically, if you ever
have a non-bare repository, it doesn't do what you expect, and it will
leave the novice user horribly confused.  A much better answer is to
simply go back to machine A, and pull from machine B.

I was exploring though to see if there was anything we could do
better, and so I used my standard test repository of the GNU Hello,
world program, and did the following:

	git clone hello r1
	git clone r1 r2
	cd r1
	<edit hello.c's headers to be GPL v2 only>
	git commit -a -m "GPL v2 only"
	cd ../r2
	<edit hello.c so that the message printed is "Hello, world!" 
		instead of "hello, world">
	cd ..

OK, so this sets up the standard test setup of repositories r1 and r2.
r1 contains a committed change so that hello.c is GPLv2 only.  r2
contains an uncommitted change to the actual text printed by hello.c.
The changes are nicely seprated in distaince by over 100 lines, so
there should be no problems with merges.  Let's play...

Experiment #1.  Let's try pushing from r1 to r2.

	cd r1
	git push ../r2

This pushes the change GPLv2 change from r1 to r2.  However, it leaves
the working tree and the index untouched, which leads to some very
unexpected and surprising behavior:

  a) If you do a "git commit" you will commit the current contents of
     the index, which is usually the contents of the head of r2 before 
     the push.
  b) If you do a "git commit -a" you will commit the modified changes to 
     the working directory --- based off of the state of r2 before the
     push.  What will therefore show up in the revision log is something
     which appears to be based off of the more recent change in r1, but 
     which is really based off of the old history as of r2 before the push.

All of this is bad, which is why "git push" to a non-bare repository
is extremely surprising.  (As an aside, what Bitkeeper would do is to
update the working tree, but it added the constraint that it would
only allow the "bk push" if the push resulted in a fast-forward merge,
thus guaranteeing no conflicts, and if the none of the files that
would need to be updated in the working tree had been locally
modified; if either constraint were modified, the push would be
aborted, and the remote repository not modified at all.  It would be
nice if we could enforce these constraints using the appropriate
hooks, but from what I can tell the hooks aren't in the right place
for us to be able to do that, or to be able to undo or recover from a
bad push.  More on that in a bit.)

OK, so suppose we do the push anyway.  As you suggested, one possible
solution is:

> $ git reset --soft <commit-id-before-the-push>
> $ git merge <commit-id-after-the-push>
> 
> But it means I have to remember <commit-id-before-the-push>.

Is it at all possible to figure out <commit-id-before-the-push>?  It
seems the answer is no, and I suspect that's a bug.  Maybe it doesn't
make sense to save the original HEAD in ORIG_HEAD in r2, but surely
the original HEAD should be saved in the reflog, right?  Well, at the
moment it saves it in neither case.

The problem is without doing this, it is as far as I can tell
impossible to determine what revision the index is currently
corresponding to, and without this information, it's very limited in
what you can do.

What possible solution which you *can* do is:

	git diff > /tmp/stage-patch
	git reset --hard
	patch -p1 < /tmp/stage-patch

But that seems a bit manual and somewhat kludgy.  If we had the
revision of r2 before the push, it would be possible to do a 3-way
merge, which in some cases might result in a cleaner merge of the
modified files in the working tree.

Experiment #2.  Let's try pulling from r2 to r1

So this is what we tell people they should do; so how well does it
work in this case?

	<do the above experimental setup>
	cd r2
	git pull ../r1

What do we get?

	Updating f2e3cc0..37508dc
	hello.c: needs update
	fatal: Entry 'hello.c' not uptodate. Cannot merge.
	 hello.c |    7 +++----
	 1 files changed, 3 insertions(+), 4 deletions(-)

Oh, dear.  Since hello.c was locally modified, git-merge refused to do
a 3-way merge to the local file.  That's unfortunate, since if it had,
it would have succeeded, and in this case it would have done the right
thing.  git-checkout will do something similar, but it at has an -m
option which will do a 3-way merge to update the local working file.
Unfortunately git-merge and git-pull do not have such an option.
'twould be nice if it did, but that's going to have to wait someone
wanting to scratch that particular itch...

The failure leaves the index and the working tree in the same confused
state as the "git push" scenario, though --- the index is still
referring to original state of the tree before the pull, but the HEAD
has been updated to after the pull, so "git commit" will lead to the
same confusing behavior.  Fortunately, though, when we do a pull,
ORIG_HEAD and the reflog are updated, so we can get back to our
original state via a command like this:

	git update-ref HEAD ORIG_HEAD

So ok, let's break down the git pull into its two constiuent parts.
The git-fetch and the git-merge.  

	git fetch
	git merge FETCH_HEAD

This fails in the same way:

   Updating f2e3cc0..37508dc
   hello.c: needs update
   fatal: Entry 'hello.c' not uptodate. Cannot merge.
    hello.c |    7 +++----
    1 files changed, 3 insertions(+), 4 deletions(-)

And just as before, it leaves HEAD updated to FETCH_HEAD, even though
it failed.  So it's consistent, but that's not what the documentation
for git-merge states:

      You  may  have  local modifications in the working tree files. In other
      words, git-diff is allowed to report changes. However, the  merge  uses
      your  working  tree  as  the  working area, and in order to prevent the
      merge operation from losing such changes, it makes sure  that  they  do
      not  interfere  with the merge. Those complex tables in read-tree docu-
      mentation define what it means  for  a  path  to  "interfere  with  the
      merge".  And  if  your  local  modifications  interfere with the merge,
      again, it stops before touching anything.
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Well, it certainly stops before touching any local files, but it has
already updated HEAD, which leaves things in a very confusion
situation.  Maybe it would be better if git-merge atomically failed
and left HEAD back pointing at the original revision?   

						- Ted

Re: git push to a non-bare repository

[Junio C Hamano]

Theodore Tso <tytso@mit.edu> writes:

> Is it at all possible to figure out <commit-id-before-the-push>?  It
> seems the answer is no, and I suspect that's a bug.

Doesn't update hook get pre- and post- commit object name?

Re: git push to a non-bare repository

[Shawn O. Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Theodore Tso <tytso@mit.edu> writes:
> 
> > Is it at all possible to figure out <commit-id-before-the-push>?  It
> > seems the answer is no, and I suspect that's a bug.
> 
> Doesn't update hook get pre- and post- commit object name?

Yes, and the same is true in the new post-receive hook.

-- 
Shawn.

Re: git push to a non-bare repository

[Theodore Tso]

On Sun, Mar 18, 2007 at 10:21:43PM -0400, Shawn O. Pearce wrote:
> Junio C Hamano <junkio@cox.net> wrote:
> > Theodore Tso <tytso@mit.edu> writes:
> > 
> > > Is it at all possible to figure out <commit-id-before-the-push>?  It
> > > seems the answer is no, and I suspect that's a bug.
> > 
> > Doesn't update hook get pre- and post- commit object name?
> 
> Yes, and the same is true in the new post-receive hook.

In my comments, I was observing that *after* the push had succeeded,
there was no way to find the commit-id-before-the-push, since neither
the reflog nor ORIG_HEAD is getting updated.  Is there a good reason
why not?  Would you accept a patch which caused the reflog and
possibly ORIG_HEAD to be updated on the remote side of the push?


When I was talking about a hook to enforce the BitKeeper semantics,
the question is whether we have enough to enforce the following:

	* Only accept the push if it will result in a fast-forward
		merge (and if not, tell the user to do a git pull, merge
		locally, and then redo the git push)
	* Only accept the push if there are no locally modified files
		that would be affected when the working directory is
		updated to reflect the new HEAD

I don't think there's any easy way to determine if these two criteria
would be met besides trying to actually do the merge, and if it fails
atomically back out to the original starting point, right?  Or am I
missing something painfully obvious?

Since one of the applications where I might want to do something like
this is a push a web site being maintained by git (where I don't want
any the result of the interim attempted to merge to accidentally get
seen by the web server), probably in order to do this right I'd have
to have the hook script do a cp -rl of the repository+working tree to
some scratch space, try to do the merge and update of the working
tree, and if it succeeds, allow it to happen for real in the "live"
tree, and if not, fail the merge.  This seems awfully kludgy; is there
some other way?

						- Ted

Re: git push to a non-bare repository

[Shawn O. Pearce]

Theodore Tso <tytso@mit.edu> wrote:
> On Sun, Mar 18, 2007 at 10:21:43PM -0400, Shawn O. Pearce wrote:
> > Junio C Hamano <junkio@cox.net> wrote:
> > > Theodore Tso <tytso@mit.edu> writes:
> > > 
> > > > Is it at all possible to figure out <commit-id-before-the-push>?  It
> > > > seems the answer is no, and I suspect that's a bug.
> > > 
> > > Doesn't update hook get pre- and post- commit object name?
> > 
> > Yes, and the same is true in the new post-receive hook.
> 
> In my comments, I was observing that *after* the push had succeeded,
> there was no way to find the commit-id-before-the-push, since neither
> the reflog nor ORIG_HEAD is getting updated.  Is there a good reason
> why not?  Would you accept a patch which caused the reflog and
> possibly ORIG_HEAD to be updated on the remote side of the push?

The reflog does update if the log file exists during a push (err,
actually during receive-pack).  Or if core.logAllRefUpdates is set
to true.  Now this isn't the default in a bare repository, but it
should be the default in a repository with a working directory.
So the case we are talking about should be seeing the reflog update.
 
> When I was talking about a hook to enforce the BitKeeper semantics,
> the question is whether we have enough to enforce the following:
> 
> 	* Only accept the push if it will result in a fast-forward
> 		merge (and if not, tell the user to do a git pull, merge
> 		locally, and then redo the git push)

Yes, the update hook can detect this.  Actually receive-pack by
default rejects *all* non-fast-forward pushes, even if the client
side uses --force.

> 	* Only accept the push if there are no locally modified files
> 		that would be affected when the working directory is
> 		updated to reflect the new HEAD

The update hook could also perform this check; test if the ref
being updated is the current branch, and if so, verify the index and
working directory is clean.  That's a simple run of git-symbolic-ref
(to get the current branch) and git-runstatus (to check the index
and working directory), is it not?

If git-runstatus exits to indicate the tree is clean (nothing to
commit) then a simple `read-tree -m -u HEAD $new` should update
the working directory and index, right?

-- 
Shawn.

Re: git push to a non-bare repository

[Theodore Tso]

On Sun, Mar 18, 2007 at 10:56:03PM -0400, Shawn O. Pearce wrote:
> The reflog does update if the log file exists during a push (err,
> actually during receive-pack).  Or if core.logAllRefUpdates is set
> to true.  Now this isn't the default in a bare repository, but it
> should be the default in a repository with a working directory.
> So the case we are talking about should be seeing the reflog update.

So I dug a little more deeply, and the problem is that the reflog for
master was getting updated, but not the reflog for HEAD, and that's
what "git reflog" was showing --- hence my confusion.

What are the rules for when HEAD's reflog should get updated, and is
this documented anywhere in the man pages?

						- Ted

Script started on Sun 18 Mar 2007 11:11:59 PM EDT
Top-level shell (parent script)
Using ssh-agent pid 7679
<tytso@candygram> {/home/tytso/talks/dscm/git}  
1% cp -r test1 test2 ; cd test2
<tytso@candygram> {/home/tytso/talks/dscm/git/test2}  
2% (cd r2; git-config core.logallrefupdates)
true
<tytso@candygram> {/home/tytso/talks/dscm/git/test2}  
3% cat r2/.git/refs/heads/master
f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2
<tytso@candygram> {/home/tytso/talks/dscm/git/test2}  
4% cat r2/.git/logs/HEAD
0000000000000000000000000000000000000000 f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2 Theodore Ts'o <tytso@mit.edu> 1174266825 -0400
<tytso@candygram> {/home/tytso/talks/dscm/git/test2}  
5% cat r2/.git/logs/refs/heads/master
0000000000000000000000000000000000000000 f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2 Theodore Ts'o <tytso@mit.edu> 1174266825 -0400
<tytso@candygram> {/home/tytso/talks/dscm/git/test2}  
6% (cd r1 ; git push ../r2)
updating 'refs/heads/master'
  from f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2
  to   37508dc11dbe274d021124057fd2d027f6ce9d17
Generating pack...
Done counting 5 objects.
Result has 3 objects.
Deltifying 3 objects.
 100% (3/3) done
Writing 3 objects.
 100% (3/3) done
Total 3 (delta 2), reused 0 (delta 0)
Unpacking 3 objects
  100% (3/3) done
refs/heads/master: f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2 -> 37508dc11dbe274d021124057fd2d027f6ce9d17
<tytso@candygram> {/home/tytso/talks/dscm/git/test2}  
7% cd r2
<tytso@candygram> {/home/tytso/talks/dscm/git/test2/r2}  [master]
8% cat .git/refs/heads/master
37508dc11dbe274d021124057fd2d027f6ce9d17
<tytso@candygram> {/home/tytso/talks/dscm/git/test2/r2}  [master]
9% cat .git/logs/HEAD
0000000000000000000000000000000000000000 f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2 Theodore Ts'o <tytso@mit.edu> 1174266825 -0400
<tytso@candygram> {/home/tytso/talks/dscm/git/test2/r2}  [master]
10% cat .git/logs/refs/heads/master
0000000000000000000000000000000000000000 f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2 Theodore Ts'o <tytso@mit.edu> 1174266825 -0400
f2e3cc0bb64c8c94b89ba07bfbdd1653584586f2 37508dc11dbe274d021124057fd2d027f6ce9d17 Theodore Ts'o <tytso@mit.edu> 1174274004 -0400	push
<tytso@candygram> {/home/tytso/talks/dscm/git/test2/r2}  [master]
11% git reflog
37508dc... HEAD@{0}: 
<tytso@candygram> {/home/tytso/talks/dscm/git/test2/r2}  [master]
13% git version
git version 1.5.0.5.425.g9cec6-dirty
<tytso@candygram> {/home/tytso/talks/dscm/git/test2/r2}  [master]
14% exit

Script done on Sun 18 Mar 2007 11:14:28 PM EDT

Re: git push to a non-bare repository

[Shawn O. Pearce]

Theodore Tso <tytso@mit.edu> wrote:
> So I dug a little more deeply, and the problem is that the reflog for
> master was getting updated, but not the reflog for HEAD, and that's
> what "git reflog" was showing --- hence my confusion.
> 
> What are the rules for when HEAD's reflog should get updated, and is
> this documented anywhere in the man pages?

It is buried down in write_ref_sha1 (in refs.c).  The rule is if the
name of the ref given to us for update does not match the actual
ref we are about to change, we log to both the original ref name
given and the actual ref name.

This handles the case of HEAD being a symref to some actual branch;
we update the HEAD reflog and the actual branch reflog whenever
someone updates HEAD.  Which is what we are usually doing from
tools like git-checkout.

receive-pack isn't updating the HEAD reflog as its updating the
actual branch, not HEAD.  If you pushed instead to HEAD you should
see the HEAD reflog entry too.

Its a little ugly here as I'm not sure we should always update
HEAD's reflog if HEAD points at a branch we are actually updating.
Maybe we should though in receive-pack ?

-- 
Shawn.

Re: git push to a non-bare repository

[Nicolas Pitre]

On Sun, 18 Mar 2007, Shawn O. Pearce wrote:

> Theodore Tso <tytso@mit.edu> wrote:
> > So I dug a little more deeply, and the problem is that the reflog for
> > master was getting updated, but not the reflog for HEAD, and that's
> > what "git reflog" was showing --- hence my confusion.
> > 
> > What are the rules for when HEAD's reflog should get updated, and is
> > this documented anywhere in the man pages?
> 
> It is buried down in write_ref_sha1 (in refs.c).  The rule is if the
> name of the ref given to us for update does not match the actual
> ref we are about to change, we log to both the original ref name
> given and the actual ref name.
> 
> This handles the case of HEAD being a symref to some actual branch;
> we update the HEAD reflog and the actual branch reflog whenever
> someone updates HEAD.  Which is what we are usually doing from
> tools like git-checkout.
> 
> receive-pack isn't updating the HEAD reflog as its updating the
> actual branch, not HEAD.  If you pushed instead to HEAD you should
> see the HEAD reflog entry too.

This is indeed a corner case.  And it was never considered before as 
great care was made at the time to be sure pushes wouldn't create any 
reflogs on the remote side, which is effectively done by not 
automatically enabling reflogs on bare repos.

> Its a little ugly here as I'm not sure we should always update
> HEAD's reflog if HEAD points at a branch we are actually updating.
> Maybe we should though in receive-pack ?

If the meaning of HEAD changed (although indirectly) because HEAD 
happens to point to the branch that just got updated then logically the 
HEAD reflog should be updated too.  On the other hand the HEAD reflog 
should reflect operations performed on HEAD.  Since the push updates the 
branch directly it is not exactly performing some operation on HEAD 
since HEAD could point anywhere and that wouldn't change the push at 
all.

Meaning that for the discussion of pushing to a non-bare repository with 
a dirty working tree... If the branch being pushed into is not pointed 
to by HEAD then no consideration what so ever about the working tree 
should be made, and no update to the HEAD reflog made of course.


Nicolas

Re: git push to a non-bare repository

[Theodore Tso]

On Mon, Mar 19, 2007 at 12:08:47AM -0400, Nicolas Pitre wrote:
> > Its a little ugly here as I'm not sure we should always update
> > HEAD's reflog if HEAD points at a branch we are actually updating.
> > Maybe we should though in receive-pack ?
> 
> If the meaning of HEAD changed (although indirectly) because HEAD 
> happens to point to the branch that just got updated then logically the 
> HEAD reflog should be updated too.  On the other hand the HEAD reflog 
> should reflect operations performed on HEAD.  Since the push updates the 
> branch directly it is not exactly performing some operation on HEAD 
> since HEAD could point anywhere and that wouldn't change the push at 
> all.
> 
> Meaning that for the discussion of pushing to a non-bare repository with 
> a dirty working tree... If the branch being pushed into is not pointed 
> to by HEAD then no consideration what so ever about the working tree 
> should be made, and no update to the HEAD reflog made of course.

Right, but if the branch being pointed to is pointed to by HEAD I
would argue that the reflog for HEAD should be updated, since
operations that reference HEAD will see a new commit, and and it will
be confusing when "git reflog" shows no hint of the change.

Of couse, if the branch being pushed to isn't one which is pointed by
HEAD, of course HEAD's reflog shouldn't be updated.

						- Ted

Re: git push to a non-bare repository

[Junio C Hamano]

Theodore Tso <tytso@mit.edu> writes:

> Right, but if the branch being pointed to is pointed to by HEAD I
> would argue that the reflog for HEAD should be updated, since
> operations that reference HEAD will see a new commit, and and it will
> be confusing when "git reflog" shows no hint of the change.
>
> Of couse, if the branch being pushed to isn't one which is pointed by
> HEAD, of course HEAD's reflog shouldn't be updated.

If we were to do this properly, we probably would need to
restructure the reflog update code for the HEAD in a major way.
"git-update-ref refs/heads/foo $newvalue" when HEAD points at
branch 'foo' currently does not update HEAD reflog because the
current definition of HEAD reflog is (as Nico mentioned) log of
changes made through HEAD symref.  Instead, we would need a
reverse lookup every time any ref is updated to see if that ref
is pointed by any symbolics ref and update the reflogs of those
symbolic refs.  This is expensive to do in general, though,
because there is no backpointer to list of symbolic refs that
point at a non-symbolic ref.

Re: git push to a non-bare repository

[Nicolas Pitre]

On Sun, 18 Mar 2007, Junio C Hamano wrote:

> Theodore Tso <tytso@mit.edu> writes:
> 
> > Right, but if the branch being pointed to is pointed to by HEAD I
> > would argue that the reflog for HEAD should be updated, since
> > operations that reference HEAD will see a new commit, and and it will
> > be confusing when "git reflog" shows no hint of the change.
> >
> > Of couse, if the branch being pushed to isn't one which is pointed by
> > HEAD, of course HEAD's reflog shouldn't be updated.
> 
> If we were to do this properly, we probably would need to
> restructure the reflog update code for the HEAD in a major way.
> "git-update-ref refs/heads/foo $newvalue" when HEAD points at
> branch 'foo' currently does not update HEAD reflog because the
> current definition of HEAD reflog is (as Nico mentioned) log of
> changes made through HEAD symref.  Instead, we would need a
> reverse lookup every time any ref is updated to see if that ref
> is pointed by any symbolics ref and update the reflogs of those
> symbolic refs.  This is expensive to do in general, though,
> because there is no backpointer to list of symbolic refs that
> point at a non-symbolic ref.

But practically speaking... is there that many cases where a branch is 
updated directly instead of the operation performed through HEAD?

We identified one case which is a push to a non bare repo.

If those cases are very few (and they _should_ be very few) then we 
might simply cheat a little and update HEAD separately in those cases.


Nicolas

Re: git push to a non-bare repository

[Nicolas Pitre]

On Mon, 19 Mar 2007, Theodore Tso wrote:

> On Mon, Mar 19, 2007 at 12:08:47AM -0400, Nicolas Pitre wrote:
> > If the meaning of HEAD changed (although indirectly) because HEAD 
> > happens to point to the branch that just got updated then logically the 
> > HEAD reflog should be updated too.  On the other hand the HEAD reflog 
> > should reflect operations performed on HEAD.  Since the push updates the 
> > branch directly it is not exactly performing some operation on HEAD 
> > since HEAD could point anywhere and that wouldn't change the push at 
> > all.
> > 
> > Meaning that for the discussion of pushing to a non-bare repository with 
> > a dirty working tree... If the branch being pushed into is not pointed 
> > to by HEAD then no consideration what so ever about the working tree 
> > should be made, and no update to the HEAD reflog made of course.
> 
> Right, but if the branch being pointed to is pointed to by HEAD I
> would argue that the reflog for HEAD should be updated, since
> operations that reference HEAD will see a new commit, and and it will
> be confusing when "git reflog" shows no hint of the change.
> 
> Of couse, if the branch being pushed to isn't one which is pointed by
> HEAD, of course HEAD's reflog shouldn't be updated.

I think we're saying the exact same thing.


Nicolas

Re: git push to a non-bare repository

[Sam Vilain]

Shawn O. Pearce wrote:
> receive-pack isn't updating the HEAD reflog as its updating the
> actual branch, not HEAD.  If you pushed instead to HEAD you should
> see the HEAD reflog entry too.
>   

What about splitting HEAD when you push to the underlying branch, and
making HEAD a non-symref?

Sam.

Re: git push to a non-bare repository

[Junio C Hamano]

Sam Vilain <sam@vilain.net> writes:

> Shawn O. Pearce wrote:
>> receive-pack isn't updating the HEAD reflog as its updating the
>> actual branch, not HEAD.  If you pushed instead to HEAD you should
>> see the HEAD reflog entry too.
>
> What about splitting HEAD when you push to the underlying branch, and
> making HEAD a non-symref?

I do not think any of the complication is needed, and I think
somebody mentioned a good example, which is a firewalled host
that can only be pushed into.  In that example, even though he
knows he could fetch in reverse direction in the ideal world,
the network configuration does not let him do so, hence need for
a push.

To deal with that sanely, people who push between non bare
repositories can just forget about pushing into branch heads.

Instead, they can arrange their pushes to be a true mirror image
of their fetch that they wish could do.  To illustrate:

On repo A that can only be pushed into, if you _could_ fetch
from repo B, you would:

	$ git fetch B

with something like this:

	[remote "B"] fetch = refs/heads/*:refs/remotes/B/*

But unfortunately because you can only push into A from B, you would
run this on B instead:

	$ git push A

with:

	[remote "A"] push = refs/heads/*:refs/remotes/B/*

And after you perform your push, you come to the machine with
repo A on it, and remembering that what you did was a mirror
image of "git fetch B", you would:

	$ git merge remotes/B/master

and you are done.

In other words, don't think of refs/remotes/B as something that
is for the use of "git fetch".  Its purpose is to track the
remote repository B's heads.  You maintain that hierarchy by
issuing fetch in repository A.  You can issue push in repository
B to do so as well.

I push into a live repository almost every day.  My typical day
concludes like this:

	gitster$ git push kernel-org-private
        gitster$ ssh kernel.org
        kernel.org$ git merge origin
        kernel.org$ Meta/Doit -pedantic &
        kernel.org$ exit
        ... go drink my tea ...

where

 (1) gitster is my private development machine
 (2) kernel.org is a machine made available to me by friendly
     k.org folks
 (3) Meta is a checkout of my 'todo' branch and 
 (4) Doit is a script to build all four public branches.

I always leave 'master' checked out on my kernel.org repository,
and the push from my private machine is done with (I still use
the non separate-remote layout):

	Push: refs/heads/master:refs/heads/origin
	Push: refs/heads/next:refs/heads/next
	Push: +refs/heads/pu:refs/heads/pu
	Push: refs/heads/maint:refs/heads/maint

So the first thing I do after logging in to kernel.org machine
is to run "git merge origin" to bring the 'master' up-to-date.
If you think of 'push' being mirror image of 'fetch' you would
understand why.  It is like issuing "git fetch" on kernel.org
machine to retrive the hot-off-the-press from my private machine
and then "git merge" it (usually "git pull" would do that as a
single step).

However, sometimes I accidentally leave 'next' checked out.  If
I find out that I left non 'master' checked out, I would do "git
reset --hard HEAD" before doing anything else, and I do not want
my push to sometimes result in detached HEAD and sometimes not.
I do not want to lose the information which branch I was on last
(because the next thing I would do is on which branch Meta/Doit
failed).  If I _were_ annoyed enough by sometimes mistakenly
pushing into the live branch, I would switch to separate remote
layout and push into remotes/origin/* hierarchy, and there will
be truly nothing to worry about after that point.

Re: git push to a non-bare repository

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> However, sometimes I accidentally leave 'next' checked out.  If
> I find out that I left non 'master' checked out, I would do "git
> reset --hard HEAD" before doing anything else, and I do not want
> my push to sometimes result in detached HEAD and sometimes not.
> I do not want to lose the information which branch I was on last
> (because the next thing I would do is on which branch Meta/Doit

s/do is on/do is to find out on/;

> failed).  If I _were_ annoyed enough by sometimes mistakenly
> pushing into the live branch, I would switch to separate remote
> layout and push into remotes/origin/* hierarchy, and there will
> be truly nothing to worry about after that point.

Re: git push to a non-bare repository

[Theodore Tso]

On Sun, Mar 18, 2007 at 10:56:03PM -0400, Shawn O. Pearce wrote:
> > When I was talking about a hook to enforce the BitKeeper semantics,
> > the question is whether we have enough to enforce the following:
> > 
> > 	* Only accept the push if it will result in a fast-forward
> > 		merge (and if not, tell the user to do a git pull, merge
> > 		locally, and then redo the git push)
> 
> Yes, the update hook can detect this.  Actually receive-pack by
> default rejects *all* non-fast-forward pushes, even if the client
> side uses --force.

Ah, so that's controlled by receive.denyNonFastForwards, right?  Cool,
I missed that.  Thanks!!

Documentation/config.txt doesn't say it defaults to true, but from
your comments that is the default?

> > 	* Only accept the push if there are no locally modified files
> > 		that would be affected when the working directory is
> > 		updated to reflect the new HEAD
> 
> The update hook could also perform this check; test if the ref
> being updated is the current branch, and if so, verify the index and
> working directory is clean.  That's a simple run of git-symbolic-ref
> (to get the current branch) and git-runstatus (to check the index
> and working directory), is it not?
> 
> If git-runstatus exits to indicate the tree is clean (nothing to
> commit) then a simple `read-tree -m -u HEAD $new` should update
> the working directory and index, right?

What git-runstatus will allow me to do is to abort if there are any
local modifications, regardless of whether or not they would conflict
with the working tree update.  The key phrase in my criteria was no
locally modified files "THAT WOULD BE AFFECTED".

What I could do with BitKeeper is that I could modify some file like
schedule.html on my webserver, and then push a changeset from my
laptop to would update sermons.html, and it would allow the push ---
since it would change the file sermons.html, and not touch
schedule.html.

But if I modified schedule.html on my laptop and then committed it,
and *then* try to push that changeset to the webserver, it would abort
since in order to accept the changeset, it would have to update the
working tree, and that would clash with the locally modified
schedule.html file.  At thta point I'd have to login to the webserver,
revert the local modification and bring it back down my laptop and
include it in a proper changeset.

Yeah, I probably shouldn't have ever modified the file locally on the
webserver, but that would sometimes happen when I was in a rush, and
it was nice when it Just Worked.

						- Ted

Re: git push to a non-bare repository

[Shawn O. Pearce]

Theodore Tso <tytso@mit.edu> wrote:
> Ah, so that's controlled by receive.denyNonFastForwards, right?  Cool,
> I missed that.  Thanks!!
> 
> Documentation/config.txt doesn't say it defaults to true, but from
> your comments that is the default?

Ah, my bad, it defaults to false:

  static int deny_non_fast_forwards = 0;

I should have known better, as I run a 1.5.x (aka 'next') server
for a workgroup and I never have that set, but use instead a complex
update hook that decides if a fast-forward is required or not.
 
> > > 	* Only accept the push if there are no locally modified files
> > > 		that would be affected when the working directory is
> > > 		updated to reflect the new HEAD
> > 
> > If git-runstatus exits to indicate the tree is clean (nothing to
> > commit) then a simple `read-tree -m -u HEAD $new` should update
> > the working directory and index, right?
> 
> What git-runstatus will allow me to do is to abort if there are any
> local modifications, regardless of whether or not they would conflict
> with the working tree update.  The key phrase in my criteria was no
> locally modified files "THAT WOULD BE AFFECTED".

  git-diff $old $new | git-apply --index ?

If the patch does not apply, nothing gets updated.  If it does apply,
the index is also updated and stat data updated.

OK, it doesn't quite handle every case, as sometimes a patch will
reject but the internal 3-way merge from xdiff that is called by
merge-recursive will succeed, but this does protect your working
tree and doesn't require making a temporary copy.


Of course another possible approach is to stuff the entire working
directory into a temporary tree, and then merge.  If the merge
doesn't work, you can reset to the temporary tree.  Unfortunately the
working directory is "in flux" during that process... its not atomic.

-- 
Shawn.

Re: git push to a non-bare repository

[Junio C Hamano]

"Shawn O. Pearce" <spearce@spearce.org> writes:

>   git-diff $old $new | git-apply --index ?
>
> If the patch does not apply, nothing gets updated.  If it does apply,
> the index is also updated and stat data updated.

And if you are only checking then perhaps instead of --index
use --check, so that the actual updates are deferred?

Re: git push to a non-bare repository

[Matthieu Moy]

Theodore Tso <tytso@mit.edu> writes:

> On Sun, Mar 18, 2007 at 06:31:21PM +0100, Matthieu Moy wrote:
>> I have a repository with a working tree on a machine A, did a clone to
>> another machine B and commited there locally.
>> 
>> I want my changes to get back into the first repository, so I did a
>> "push". The new commit is in the history, I can see it with "git log",
>> but the modifications are not in the working tree.
>
> The general answer (which you've already received) is to tell folks is
> to simply don't use "git push" to remote trees; basically, if you ever
> have a non-bare repository, it doesn't do what you expect, and it will
> leave the novice user horribly confused.  A much better answer is to
> simply go back to machine A, and pull from machine B.

It's not really an option in my case. A is a fixe-IP/fixe-DNS machine,
while B is my home machine, behind a NAT modem-router. So, I'd have to
figure out my home IP, port-forward the ssh port from the modem to my
machine, ...

If I understand correctly the other answers, I have two options:

* Git doesn't manage this case, and doesn't care about me loosing data
  if they're not commited, I'll have to do it myself with hooks.

* Create a bare repository on machine A, and clone it to a non-bare
  repo on which I'll work. But that means duplicating the repository
  on the same filesystem of the same machine. Not really satisfactory
  either. The "light checkout" feature would make it better, but I'm
  still worried about what will happen to my light checkout when
  someone pushes to the repository.

-- 
Matthieu

Re: git push to a non-bare repository

[Jakub Narebski]

Matthieu Moy wrote:

> * Create a bare repository on machine A, and clone it to a non-bare
>   repo on which I'll work. But that means duplicating the repository
>   on the same filesystem of the same machine. Not really satisfactory
>   either. The "light checkout" feature would make it better, but I'm
>   still worried about what will happen to my light checkout when
>   someone pushes to the repository.

You can share repository object database using alternates.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: git push to a non-bare repository

[Neil Schemenauer]

Matthieu Moy <Matthieu.Moy@imag.fr> wrote:
> It's not really an option in my case. A is a fixe-IP/fixe-DNS machine,
> while B is my home machine, behind a NAT modem-router. So, I'd have to
> figure out my home IP, port-forward the ssh port from the modem to my
> machine, ...

I think this is a pretty common scenario.  I have a central server
that I would like to push to and a bunch of other machines behind
firewalls.  Pulling from the central machine is not practical.

  Neil

Re: git push to a non-bare repository

[Sergio Callegari]

Hi,

Matthieu Moy <Matthieu.Moy <at> imag.fr> writes:

> 
> Hi,
> 
> I have a repository with a working tree on a machine A, did a clone to
> another machine B and commited there locally.
> 
> I want my changes to get back into the first repository, so I did a
> "push". The new commit is in the history, I can see it with "git log",
> but the modifications are not in the working tree.
> 
> This time, it's OK: I didn't have any uncommited modifications on A,
> so I just did a "git reset --hard HEAD" there.

At times I have a similar scenario, working with both an office-pc (attached to
the internet with a public static address) and a laptop (either detached from
the internet or attached to it under a dynamic address and NAT) 

> But if I had some uncommited changes, "git reset --hard HEAD" means
> data loss, which is precisely what I want to avoid by using a VCS. It
> seems a solution is to do:
> 
> $ git reset --soft <commit-id-before-the-push>
> $ git merge <commit-id-after-the-push>

I think that there might also be another option...
perhaps in some cases you might want to commit the local changes by branching
at <commit-id-before-the-push>...

For instance:
* you work on the office-pc: work work work (you do not commit yet, as you
expect you will be able to finish your stuff and commit after that)
* but for some reason you do not finish, you go away and you start working on
the laptop...
* you cannot pull the changes from the office-pc (as they are not committed)
and assume you cannot or don't want to retrieve them otherwise... so you start
working on what you have... work work work ... commit. You do not branch before
committing as this is a finished thing and you now believe it deserves going on
your current branch... (or you just have forgotten about the hanging stuff on
the office-pc)...
* As soon as you can, you push to the office-pc
* Since the local changes on the office-pc are relative to something that has
suddendly got older than the current branch-tip, you might want to merge, but
you also might not want to... you might prefer to complete the thing and first
test it as is... in this case you might want to branch, rather than merging.

So when you push to a non-bare repo, it might probably be nice to be able to
select between different behaviors:
A) Try to auto merge and refuse the push if merging is impossible (as it is
currently suggested)
B) Accept the push anyway (in the end if one machine is under NAT, pushing is
the only way to propagate changes), update the branch, loose the head and store
somewhere some information like "I used to be in branch so and so, but I am not
anymore at the branch tip since that has been changed, local changes are against
commit ..."  In this way the non-bare repo can start working as if it has become
a headless checkout with local changes, explaining the situation and suggesting
appropriate action (either merging or branching) to the user...
something like:
git status (or git commit)
No current branch since the repository has been changed from outside this
working tree... Cannot commit... Your last branch was ...
Possible options
< suggestion for merging >
< suggestion for branching >

I do not know if this makes any sense at all, since my usage of git is somehow
limited and atypical (e.g. due to the 2 machines) and I might be missing many
many implications...

However B looks like a behavior that could be applied also in the case of
multiple working trees insisting on a single repo, like in Junio's recen post
where he wrote:

 > These days I use a few working trees that are connected to my
> primary repository (which also has a working tree).  The primary
> repository is in /src/git, and other ones look like this:
> 
> : gitster git.wk0; ls -l .git/
> total 120
> drwxrwsr-x  3 junio src  4096 Mar  5 16:22 ./
> drwxrwsr-x 15 junio src 16384 Mar  5 16:23 ../
> -rw-rw-r--  1 junio src    41 Mar  5 16:22 HEAD
> lrwxrwxrwx  1 junio src    27 Mar  3 22:53 config -> /src/git/.git/config
> lrwxrwxrwx  1 junio src    26 Mar  3 22:53 hooks -> /src/git/.git/hooks/
> -rw-rw-r--  1 junio src 82455 Mar  5 16:22 index
> lrwxrwxrwx  1 junio src    25 Mar  3 22:53 info -> /src/git/.git/info/
> drwxrwsr-x  3 junio src  4096 Mar  3 22:59 logs/
> lrwxrwxrwx  1 junio src    28 Mar  3 22:53 objects -> /src/git/.git/objects/
> lrwxrwxrwx  1 junio src    32 Mar  3 22:53 packed-refs ->
/src/git/.git/packed-refs
> lrwxrwxrwx  1 junio src    25 Mar  3 22:53 refs -> /src/git/.git/refs/
> lrwxrwxrwx  1 junio src    28 Mar  3 22:53 remotes -> /src/git/.git/remotes/
> lrwxrwxrwx  1 junio src    29 Mar  3 22:53 rr-cache -> /src/git/.git/rr-cache/
> 
> It shares everything other than HEAD and the index (the reflog
> for branches are also shared by a symlink .git/logs/refs
> pointing at the one in the primary repository).
> 
> This risks confusion for an uninitiated if you update a ref that
> is checked out in another working tree, but modulo that caveat
> it works reasonably well.
> 
> We might want to add an option to 'git-clone' to create
> something like this, but I am somewhat worried about the newbie
> confusion factor.  Perhaps...
> 
> $ git clone --i-know-what-i-am-doing-give-me-an-alternate-working-tree \
>   /src/git /src/git.wk0


So, did I miss something?
 
Many thanks,

Sergio