[RFD] remembering hand resolve...

[RFD] remembering hand resolve...

[Junio C Hamano]

As people on the list may know, I keep many mini topic-branches
and keep combining them on top of then-current master to publish
"pu".  This involves resolving merge conflicts by hand, when the
areas topic-branches touch overlap.

The thing is, I find myself resolving the same conflicts over
and over.  This is because the master branch tends to advance
faster than topic branches that touch an overlapping area.  I'd
take more time than I usually do to decide what to do with them;
as a result, overlapping topic branches tend to stay unmerged
into "master" longer than other topic branches.

If I linearize topic-branches that conflict with each other in
some way, say base topic B on top of topic A, I would not have
problem merging them into "pu" as long as I do not change my
mind later and try to merge only topic B without topic A.  But
that defeats the whole point of having independent topic
branches.

I would imagine that people who use StGIT or quilt would have
similar issues.  If they are in the same series, then inside of
that queue the patches are already ordered to be in some way,
probably conflict is resolved once when the patch is refreshed
and they stay applicable as long as the base part cleanly
applies to the updated base version, but patches in the queue
then depend on the earlier ones in the same series, and
extracting and applying only the later parts of the queue would
need you to manually un-resolve the conflict you earlier
resolved.  If you keep different topics in separate queues, on
the other hand, I would imagine you would have exactly the same
"oh, I know this and that patch conflict with each other and I
recall I resolved that last time I merged everything up" issue.

How do people on patch-queue based systems like StGIT and quilt
deal with this?  I am wondering if somebody have a clever idea
to record and reuse an earlier conflict resolution.

A trivial solution would be to save the diff between conflicted
automerge result before hand resolving, and the result of my
hand resolve, and apply with "patch" when I see a conflicted
automerge the next time.  I've tried this by hand and it worked
quite well tonight, but I felt it was somewhat kludgy.  We
should be able to do better than that, with some tool support.

Another obvious way is to avoid rebuilding "pu"; instead I could
pull "master" into "pu" every time I have added something new to
"master".  That would work most of the time, until I decide to
change the order the topic branches are merged into "pu" (or
drop one of them).

Re: [RFD] remembering hand resolve...

[Andreas Ericsson]

Junio C Hamano wrote:
> As people on the list may know, I keep many mini topic-branches
> and keep combining them on top of then-current master to publish
> "pu".  This involves resolving merge conflicts by hand, when the
> areas topic-branches touch overlap.
> 
> The thing is, I find myself resolving the same conflicts over
> and over.  This is because the master branch tends to advance
> faster than topic branches that touch an overlapping area.


How can that be possible? If the area of code in master is modified 
beyond simple merging from the topic-branches, the resulting diff (which 
has to be against master's HEAD) should apply cleanly so long as there 
aren't *new* changes to master, in which case the pu commit needs 
adaptation again, so it wouldn't really be the same conflict (or even 
necessarily a similar one), would it?


> If I linearize topic-branches that conflict with each other in
> some way, say base topic B on top of topic A, I would not have
> problem merging them into "pu" as long as I do not change my
> mind later and try to merge only topic B without topic A.  But
> that defeats the whole point of having independent topic
> branches.
> 

Wouldn't cherry-pick be useful here? If it isn't, I fail to understand 
how a merge can solve it for you. Admittedly, I know little of the inner 
workings of git.

> 
> A trivial solution would be to save the diff between conflicted
> automerge result before hand resolving, and the result of my
> hand resolve, and apply with "patch" when I see a conflicted
> automerge the next time.  I've tried this by hand and it worked
> quite well tonight, but I felt it was somewhat kludgy.  We
> should be able to do better than that, with some tool support.
> 

> Another obvious way is to avoid rebuilding "pu"; instead I could
> pull "master" into "pu" every time I have added something new to
> "master".  That would work most of the time, until I decide to
> change the order the topic branches are merged into "pu" (or
> drop one of them).
> 

Couldn't you do something like this:

for t in $topic_branches; do
	checkout $t
	git rebase master
done
git checkout master
git branch -D pu
git checkout -b pu
git pull . $topic_branches

That's what I thought you were doing right now, but I can't imagine that 
flow leading to the same conflicts over and over, so I must be mistaken 
somehow.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [RFD] remembering hand resolve...

[Junio C Hamano]

Andreas Ericsson <ae@op5.se> writes:

> Junio C Hamano wrote:
>> The thing is, I find myself resolving the same conflicts over
>> and over.  This is because the master branch tends to advance
>> faster than topic branches that touch an overlapping area.
>
> How can that be possible? If the area of code in master is modified
> beyond simple merging from the topic-branches, the resulting diff
> (which has to be against master's HEAD) should apply cleanly so long
> as there aren't *new* changes to master, in which case the pu commit
> needs adaptation again, so it wouldn't really be the same conflict (or
> even necessarily a similar one), would it?

In general, you are right.  When the change that advances
"master" touches the same area as those held-back topic branches
touch, the earlier resolution between topic branches may not help.

However, the change to the "master" that leap-frogs these
held-back topic branches is often independent from the changes
in these held-back topic branches.

For example, as of this morning, my "master" and "pu" were like
this:

        $ git show-branch master pu
        * [master] Merge branches 'jc/clone', 'md/env' and 'mo/path'
         ! [pu] Merge jc/revparse,jc/abbrev
        --
         - [pu] Merge jc/revparse,jc/abbrev
         - [pu^] Merge lt/revlist,jc/diff,jc/bind
         + [pu^^3] combine-diff: fix appending at the tail of a list.
         + [pu^2] rev-parse --flags/--no-flags usability fix.
         + [pu^3] diff-tree: abbreviate merge parent object names wit...
         + [pu^^2] rev-list: stop when the file disappears
         + [pu^^3^] diff-tree --cc: denser combined diff output for a...
         + [pu^^3~2] diff-tree -c: show a merge commit a bit more sen...
         + [pu^^4] fsck-objects, convert-objects: understand bound co...
         + [pu^^4^] rev-list: understand bound commits.
         + [pu^^4~2] rev-list: simplify --object list generation.
         + [pu^^4~3] commit-tree: --bind <sha1> <path>/ option.
         + [pu^^4~4] write-tree: --prefix=<path>/ and --exclude=<pref...
         + [pu^^4~5] read-tree: --prefix=<path>/ option.
         + [pu^3^] rev-parse: --abbrev option.
         + [pu^3~2] abbrev cleanup: use symbolic constants
        -- [master] Merge branches 'jc/clone', 'md/env' and 'mo/path'

The two merges at the tip of "pu" shows that I merged lt/revlist
(Linus' "stop at disappearance"), jc/diff ("combined diff") and
jc/bind ("bound commit') first because I felt they are more or
less ready, and then jc/revparse and jc/abbrev on top of it.

Two topic branches, jc/abbrev and jc/bind, touch the same area
in commit.c.  The former changes the formatting of "Merge:" line
from prettyprinted commit (we did not honor --abbrev option to
diff-tree when we used that function).  The latter changes the
same function to include a call to add the new "Bind: " lines.
When I made the last merge for the tip of "pu", I had to hand
resolve conflicts in the file.

Now suppose I feel topic branches lt/revlist, jc/diff and
jc/revparse are ready, and want to have them graduate to the
"master" branch.  Also I may get a patch from you fixing
Documentation/Makefile, which none of my topic branches touch,
so I would advance "master" this way:

	$ git checkout master
        $ git pull . lt/revlist
        $ git pull . jc/diff
        $ git pull . jc/revparse
	$ git am -3 -s ./+ae-doc-make.email

What I would do after this step to rebuild "pu" would be to
merge jc/bind and jc/abbrev branches, in some order.  But in
whatever order I merge, I would get the conflict in commit.c
as before:

	$ git checkout pu
        $ git reset --hard master
        $ git pull . jc/bind
        $ git pull . jc/abbrev

If I swap the order I pull in jc/bind and jc/abbrev, the
sections marked with conflict markers <<< === >>> would be
swapped, so the conflict would not be exactly the same, but
the result of the resolution is the same.  I end up making the
result logically the same by hand resolving.

Although I happen to do Octopus when building "pu" but the issue
does not change if the merges were made one branch at a time.

>> If I linearize topic-branches that conflict with each other in
>> some way, say base topic B on top of topic A, I would not have
>> problem merging them into "pu" as long as I do not change my
>> mind later and try to merge only topic B without topic A.  But
>> that defeats the whole point of having independent topic
>> branches.
>
> Wouldn't cherry-pick be useful here? If it isn't, I fail to understand
> how a merge can solve it for you....

Suppose instead of having

              A---B---C jc/bind
             /    
  master ---O
             \    
              D---E---F jc/abbrev

I made this:

              A'--B'--C'--D---E---F
             /        ^jc/bind    ^jc/abbrev
  master ---O

When converting the two topic branches into this sequence,
obviously I have to hand merge the changes A-B-C makes and the
changes D-E-F makes to commit.c file.

I can later attempt to cherry-pick D-E-F on top of updated
"master".  I think that is what you mean by "cherry-picking",
but that would have the same "I've resolved this before" issue.
Because the change D-E-F brings in now depends on the change
A'-B'-C' brings in. But on the tip of the master branch, that
change is not there yet. The cherry-picked changes on top of O
would not cleanly apply.  Even with the 3-way merge fallback,
this is a merge of F and O using C' as the merge base, and we
will see conflicts.

However, as long as I do not have to merge the contents of
jc/abbrev branch in without the contents of jc/bind branch, a
merge to an updated master would not have any conflict between
A'-B'-C' chain and D-E-F chain, and often even after jc/bind and
jc/abbrev advances:

                        G---H jc/bind
                       /
              A'--B'--C'--D---E---F
             /                    ^jc/abbrev
  master ---O---I---J

Pulling jc/bind to master and then jc/abbrev on top it may have
conflicts between the changes O-I-J and C'-G-H bring in, but at
least I would not have to re-resolve the conflicts between
A'-B'-C' and D-E-F chains.  That is what I meant to say.  The
merge is helped by making jc/abbrev topic branch dependent on
the earlier part of jc/bind topic branch.

> Couldn't you do something like this:
>
> for t in $topic_branches; do
> 	checkout $t
> 	git rebase master
> done
> git checkout master
> git branch -D pu
> git checkout -b pu
> git pull . $topic_branches
>
> That's what I thought you were doing right now, but I can't imagine
> that flow leading to the same conflicts over and over, so I must be
> mistaken somehow.

That is exactly what I am doing [*1*]


[Footnote]

*1* In my primary development working tree, I have a checked-out
tree of "todo" branch at Meta/ directory.  The "pu" is built by
running Meta/PU (that is "PU" file in the "todo" branch) from
the toplevel and picking topic branches to merge on top of the
master branch, which essentially does what you just described.

To take a peek at how my primary development working tree looks
like, you could do this:

	$ git clone git://git.kernel.org/pub/scm/git/git GIT
        $ cd GIT
        $ git clone -l -s -n .git Meta
        $ cd Meta
        $ git checkout todo
        $ cd ..
	$ rm Meta/.git/refs/heads/[a-p]* ;# leave only "todo"
        $ rm -f .git/refs/heads/{html,man,todo}

This would give a rough approximation of where I work.

The above clone would not have any of the topic branches; but
their heads appear on merge commits between master and pu.  For
example, in the earlier show-branch output, the tip of "pu" says
"Merge jc/revparse,jc/abbrev", which means pu^1 was the tip of
pu when this merge was made, pu^2 is the tip of jc/revparse
topic, and pu^3 is jc/abbrev.  You can take them apart like so:

	$ git branch jc/revparse pu^2
	$ git branch jc/abbrev pu^3

How to create and keep up to date a naked/bare repository?

[Mathieu Chouquet-Stringer]

        Hello,

I've got a couple of questions and reading (some of the files in) the
Documentation directory didn't really enlighten me...

Let's say I want to maintain a private (read-only) copy of a git repository
(eg git.git or linus-2.6.git). Because this repository is shared
internally (used by a bunch of boxes), I'd like to clone it as a bare
repo, no need to have the files checked out on my local master.

Cloning is done by running the following:
git clone --naked git://www.kernel.org/pub/scm/git/git.git git.git

Now I've got my repository. Because I'll share it with git-daemon, I touch
the git-daemon-export-ok file to export it.

The first problem arises when I try to keep this repo up to date.

If I try a git fetch or pull (more on that later), git complains because it
can't find a .git directory (which is correct as in a bare repository the
repository directory is the .git directory).

I can do a: GIT_DIR=. git fetch and it works but that's kind of ugly. I
can also link . to .git but it's as disgusting as the previous hack.

Maybe I'm missing something obvious here, is there an option to tell
fetch/pull that it's working with a bare repository?

The second question is I'd think I would have to use git fetch instead of
git pull as a pull is fetch + merge and merging wouldn't make much sense as
I didn't check out any files. Does it make any sense?

The third thing (can't reproduce ATM as I just got a fresh new clone) is
that git fetch was just fetching over and over the same objects...

Can anyone tell me what I'm doing wrong?

-- 
Mathieu Chouquet-Stringer
    "Le disparu, si l'on vénère sa mémoire, est plus présent et
                 plus puissant que le vivant".
           -- Antoine de Saint-Exupéry, Citadelle --

Re: How to create and keep up to date a naked/bare repository?

[Junio C Hamano]

Mathieu Chouquet-Stringer <ml2news@free.fr> writes:

> I can do a: GIT_DIR=. git fetch and it works.

That is the right way of doing it.

> The second question is I'd think I would have to use git fetch instead of
> git pull as a pull is fetch + merge and merging wouldn't make much sense as
> I didn't check out any files. Does it make any sense?

Absolutely.  Bare repositories are either to be pushed into or
fetched into.

> The third thing (can't reproduce ATM as I just got a fresh new clone) is
> that git fetch was just fetching over and over the same objects...

It sounds like you are not telling fetch to update the refs you
track from outside.  Perhaps, this may help (untested):

	$ GIT_DIR=/pub/scm/git/git.git git fetch \
        	--update-head-ok $remote master:master

Re: How to create and keep up to date a naked/bare repository?

[Mathieu Chouquet-Stringer]

junkio@cox.net (Junio C Hamano) writes:
> Mathieu Chouquet-Stringer <ml2news@free.fr> writes:
> > I can do a: GIT_DIR=. git fetch and it works.
> 
> That is the right way of doing it.

Ok.
 
> > The second question is I'd think I would have to use git fetch instead of
> > git pull as a pull is fetch + merge and merging wouldn't make much sense as
> > I didn't check out any files. Does it make any sense?
> 
> Absolutely.  Bare repositories are either to be pushed into or
> fetched into.

Great I ain't totally lost then!
 
> > The third thing (can't reproduce ATM as I just got a fresh new clone) is
> > that git fetch was just fetching over and over the same objects...
> 
> It sounds like you are not telling fetch to update the refs you
> track from outside.  Perhaps, this may help (untested):
> 
> 	$ GIT_DIR=/pub/scm/git/git.git git fetch \
>         	--update-head-ok $remote master:master

I'll try that and will let you know...

Thanks Junio for getting back to me.
-- 
Mathieu Chouquet-Stringer
    "Le disparu, si l'on vénère sa mémoire, est plus présent et
                 plus puissant que le vivant".
           -- Antoine de Saint-Exupéry, Citadelle --

Re: How to create and keep up to date a naked/bare repository?

[Mathieu Chouquet-Stringer]

junkio@cox.net (Junio C Hamano) writes:
> It sounds like you are not telling fetch to update the refs you
> track from outside.  Perhaps, this may help (untested):
> 
> 	$ GIT_DIR=/pub/scm/git/git.git git fetch \
>         	--update-head-ok $remote master:master

I guess I'm a little bit confused here as I'm not quite sure what I'm
trying to accomplish...

Here's what I ran:
% GIT_DIR=. git fetch \
--update-head-ok git://www.kernel.org/pub/scm/git/git master:master

Isn't git supposed to fetch master:origin as my remotes/origin file
contains (I actually tried both and refs/heads/origin and refs/heads/master
have the same value):

URL: git://www.kernel.org/pub/scm/git/git.git
Pull: master:origin
Pull: todo:todo
Pull: html:html
Pull: maint:maint
Pull: man:man
Pull: pu:pu

So it knows about the repository and the ref spec so all I need would be
the --update-head-ok flag then, no?

-- 
Mathieu Chouquet-Stringer
    "Le disparu, si l'on vénère sa mémoire, est plus présent et
                 plus puissant que le vivant".
           -- Antoine de Saint-Exupéry, Citadelle --

Re: How to create and keep up to date a naked/bare repository?

[Junio C Hamano]

Mathieu Chouquet-Stringer <ml2news@free.fr> writes:

> junkio@cox.net (Junio C Hamano) writes:
>> It sounds like you are not telling fetch to update the refs you
>> track from outside.  Perhaps, this may help (untested):
>> 
>> 	$ GIT_DIR=/pub/scm/git/git.git git fetch \
>>         	--update-head-ok $remote master:master
>
> I guess I'm a little bit confused here as I'm not quite sure what I'm
> trying to accomplish...
>
> Here's what I ran:
> % GIT_DIR=. git fetch \
> --update-head-ok git://www.kernel.org/pub/scm/git/git master:master
>
> Isn't git supposed to fetch master:origin as my remotes/origin file
> contains (I actually tried both and refs/heads/origin and refs/heads/master
> have the same value):
>
> URL: git://www.kernel.org/pub/scm/git/git.git
> Pull: master:origin
> Pull: todo:todo
> Pull: html:html
> Pull: maint:maint
> Pull: man:man
> Pull: pu:pu
>
> So it knows about the repository and the ref spec so all I need would be
> the --update-head-ok flag then, no?

True.

The latest git-clone.sh stopped creating both refs/heads/origin
and remotes/origin.  The former was to prevent common mistake
when using it as a shared repository, but removal of the latter
might have been a mistake.  I dunno.


Let's step back a bit and think about the modes of usage for a
bare cloned repository.

 - A central distribution point for a (sub)project.

   This is the original mode of operation bare repositories were
   to be used.  Think of bare repositories on kernel.org public
   machines.  Subsystem maintainers never work in them; the
   maintainers push into them from their private development
   trees.  Most importantly, fetching into them is not usually
   done.  Fetching from upstream or sibling is done first into
   maintainers' private development trees, and then either
   vanilla upstream tip and/or merge result with it are pushed
   into this bare repository.

 - A shared repository, CVS style.

   This is a natural extension of the above, but instead of
   having one person (the owner of the central distribution
   point for the subproject, aka subsystem maintainer), members
   of a group push into it.  Other aspects of the repository are
   not different from the above.  For example, if the project
   has an upstream, the changes from the upstream is fetched by
   people into their own development tree, merged appropriately
   and pushed back to the shared repository.

 - A mirror of somebody else's work.

   I suspect this is what you are doing.  Instead of individual
   developers pulling directly from upstream, you would want to
   fetch and update the bare cloned repository and have your
   developers fetch from it.

The first two forms are equivalent.  The first one is a special
case of having only one user of the shared repository.  The
mirror usage can be combined with the shared repository style.

Let's outline how.

What I would suggest is to arrange things like this:

   +-------------------------------------------+
   | Your upstream repository                  |
   | git://git.kernel.org/pub/scm/git/git.git/ |
   +-------------------------------------------+ 
      |                                      ^
      | (1)                                  | (5)
      |                                      |
      v                                      |
   +-----------------+                     +----------------+
   | Your central    |        (4)          | My development |
   | bare repository |-------------------->| tree           |
   +-----------------+                     +----------------+
      |      ^
      | (2)  | (3)
      |      |
      v      |
   +------------------+
   | Your developers' |+
   | trees            ||+
   +------------------+||
     +-----------------+|
      +-----------------+

(1) is initially done with "clone --bare".  You have already
    done this.  To let your developers fetch the upstream
    without directly going to kernel.org/, you would need a
    mirror, which means you would want to keep all branches from
    the upstream that your developers might be interested in.

    So I would suggest to have something like this in this
    repository:

        $ cat remotes/origin
        URL: git://git.kernel.org/pub/scm/git/git.git/
        Pull: master:mirror
        Pull: maint:maint
        Pull: +pu:pu
        Pull: todo:todo
        Pull: html:html
        Pull: man:man
	$ mv refs/heads/origin refs/heads/mirror

    You may not particularly interested in the last three, in
    which case you can safely omit them.  Also note that I
    renamed "origin" to "mirror" here.

    I imagine that your developers would be making updates to
    git, and wish to share the changes with outside world (the
    repository is not strict a mirror in that sense), so you
    would need your own branch or two.  Your "master" branch is
    used for that.

    Periodically, you could mirror me with:

	$ GIT_DIR=. git fetch

    Which would update the branch heads listed on the RHS of
    Pull: lines above.  Note that the branch HEAD points at is
    refs/heads/master and is not listed on the RHS of Pull:
    lines, so --update-ok is not needed.  My "master" will be
    mirrored as "mirror" in this repository.

(2) is created by "git clone" from your "central bare repository", and
    maintained by subsequent "git pull".  I am not offhand certain
    what remotes/origin file they would get by default, but I would
    recommend having something like this in their trees:

	$ cat .git/remotes/origin
        URL: git://your.host/your/git.git
	Push: master:master
        Pull: master:public
        Pull: mirror:mirror
	Pull: +pu:pu

    Again, you may not particularly interested in all my branches, so
    I've omitted most from the above.  Your developers should never
    touch branches that are used to keep track of outside branches in
    their repositories (they are "public", "mirror" and "pu" in the
    above example).  Have them work on their "master" branch.

	$ git pull

    would fetch from the "master" of your central bare repository and
    keep track of it as the "public" branch [*1*][*2], and merge the
    result of other developers in your group into the "master" branch.
    If you would want to see how well your work works with the latest
    upstream:

	$ git branch -f test mirror
    	$ git checkout test
        $ git pull . master

    would create a "test" branch based on my tip of "master", and
    merge your development histories into it.  Substitute "mirror"
    with "pu" if you want to see how well your development works with
    the tip of "pu" branch.

(3) Your developers will make commits to their own "master" branch,
    and when things are ready, push the branch back:

	$ git push origin

    which would attempt to update "master" in your central bare
    repository with developer's "master".  This will fail if the
    developer is not up to date --- in which case another git
    pull and a merge would be needed to incorporate other developer's
    changes.

(4) If you got fixes and enhancements for public consumption,
    drop me a "please pull from here" note.  I'll pull your
    changes into my development tree [*3*].

(5) And I would push them out to the public repository.  The
    next fetch from me by your central bare repository and then
    your developer's pull from it would complete the full
    circle.


Does this make sense now?


[Footnotes]

*1* The "public" branch represents what other developers in your
    group have done.

*2* At the same time, it would fetch "mirror", which is my
    "master", and keep track of it as the "mirror" branch head,
    and "pu", which is also my "pu", and stores it as "pu".

    So if you are interested in what I've done in the past
    couple of days, you could:

	$ git whatchanged --since=3.days pu ;# everything
	$ git whatchanged --since=3.days mirror ;# my "master"

*3* This is one reason I did not recommend frequent merging from
    "mirror" to "master" in workflow (2).  If you do not expect me to
    pull from you, merging "mirror" into "master" to update the
    development base might be easier to handle for your developers.

Refs naming after clone (was: Re: How to create and keep up to date a naked/bare repository?)

[Josef Weidendorfer]

Hi,

On Friday 27 January 2006 04:36, Junio wrote:
> The latest git-clone.sh stopped creating both refs/heads/origin
> and remotes/origin.  The former was to prevent common mistake
> when using it as a shared repository, but removal of the latter
> might have been a mistake.  I dunno.

IMHO it is convenient if git-clone remembers the origin repository
in any case.
 
> Let's step back a bit and think about the modes of usage for a
> bare cloned repository.

It would be nice to have a default which is working nice in all cases
(not only bare ones as discussed here).

IMHO the main problem with the current default (creating master and origin
heads after cloning) is that two namespaces are mixed up:
(1) local (possible development) heads
(2) and heads tracking remote heads

Any compatibility issues aside, wouldn't it be better to have another
namespace for (2), similar to the proposed ref namespaces for subprojects,
like refs/remote/<remote-shortcut>/heads/<remote-head> ?
Clone would create a head refs/remote/origin/heads/master to track the
master of the origin remote repo.

In the case of a the mirror discussed here, you would default to
quite cumbersome long head names:
Tracking master of git://git.kernel.org/pub/scm/git/git.git in
the central mirroring repository would be done on "remote/origin/heads/master",
and tracking this mirror head in developer repos would give a head
name "remote/origin/remote/origin/heads/master".

Still, I think this is nice because everybody can see that above head
is tracking a remote head which is tracking itself another remote
head. gitk could give remote tracking heads another color.

Note that I talk only about the default setup, which of course can be
changed by changing .git/remotes/origin.
Eg. in the mirror case, you probably want local and remote namespace
being the same, ie. .git/refs/heads/* -> .git/refs/remote/origin/heads/*

Perhaps a command to rename head names, which automatically updates
pull/push lines of .git/remotes/ accordingly would be nice here.
Also a command to rename remote shortcuts (e.g. origin to gitmain),
renaming above proposed head names accordingly would be nice.

Josef

Re: How to create and keep up to date a naked/bare repository?

[Mathieu Chouquet-Stringer]

Thanks and sorry for making you write this long message...

junkio@cox.net (Junio C Hamano) writes:
> [...]
>  - A mirror of somebody else's work.
> 
>    I suspect this is what you are doing.  Instead of individual
>    developers pulling directly from upstream, you would want to
>    fetch and update the bare cloned repository and have your
>    developers fetch from it.

Correct.
 
> Let's outline how.
> 
> What I would suggest is to arrange things like this:
> 
>    +-------------------------------------------+
>    | Your upstream repository                  |
>    | git://git.kernel.org/pub/scm/git/git.git/ |
>    +-------------------------------------------+ 
>       |                                      ^
>       | (1)                                  | (5)
>       |                                      |
>       v                                      |
>    +-----------------+                     +----------------+
>    | Your central    |        (4)          | My development |
>    | bare repository |-------------------->| tree           |
>    +-----------------+                     +----------------+
>       |      ^
>       | (2)  | (3)
>       |      |
>       v      |
>    +------------------+
>    | Your developers' |+
>    | trees            ||+
>    +------------------+||
>      +-----------------+|
>       +-----------------+
> [...]
> Does this make sense now?

It totally does. I guess this email could end up in the Documentation
folder in some way!?

-- 
Mathieu Chouquet-Stringer
    "Le disparu, si l'on vénère sa mémoire, est plus présent et
                 plus puissant que le vivant".
           -- Antoine de Saint-Exupéry, Citadelle --

Re: How to create and keep up to date a naked/bare repository?

[Junio C Hamano]

Mathieu Chouquet-Stringer <ml2news@free.fr> writes:

> Thanks and sorry for making you write this long message...

Hey, I do not mind "I appreciate", but don't be sorry.  I
actually enjoyed drawing the ASCII art.

>...
> It totally does. I guess this email could end up in the Documentation
> folder in some way!?

Maybe somebody so inclined could copyedit it, find an
appropriate place for it somewhere in the tutorial.txt or
core-tutorial.txt and feed that back to me in a patch form.
Hint, hint,... ;-).

Re: How to create and keep up to date a naked/bare repository?

[J. Bruce Fields]

On Fri, Jan 27, 2006 at 10:41:19AM -0800, Junio C Hamano wrote:
> Mathieu Chouquet-Stringer <ml2news@free.fr> writes:
> 
> > Thanks and sorry for making you write this long message...
> 
> Hey, I do not mind "I appreciate", but don't be sorry.  I
> actually enjoyed drawing the ASCII art.
> 
> >...
> > It totally does. I guess this email could end up in the Documentation
> > folder in some way!?
> 
> Maybe somebody so inclined could copyedit it, find an
> appropriate place for it somewhere in the tutorial.txt or
> core-tutorial.txt and feed that back to me in a patch form.
> Hint, hint,... ;-).

It might make sense just to dump it in Documentation/howto/ for now.

--b.

Re: How to create and keep up to date a naked/bare repository?

[Junio C Hamano]

"J. Bruce Fields" <bfields@fieldses.org> writes:

>> Maybe somebody so inclined could copyedit it, find an
>> appropriate place for it somewhere in the tutorial.txt or
>> core-tutorial.txt and feed that back to me in a patch form.
>> Hint, hint,... ;-).
>
> It might make sense just to dump it in Documentation/howto/ for now.

Perhaps, but I forgot to say "after some discussion".

I think people on the list have better suggestion than what I
outlined, and I suspect I probably have some details wrong in
that message.

[PATCH/RFC] git-rerere: reuse recorded resolve.

[Junio C Hamano]

In a workflow that employs relatively long lived topic branches,
the developer sometimes needs to resolve the same conflict over
and over again until the topic branches are done (either merged
to the "release" branch, or sent out and accepted upstream).

This commit introduces a new command, "git rerere", to help this
process by recording the conflicted automerge results and
corresponding hand-resolve results on the initial manual merge,
and later by noticing the same conflicted automerge and applying
the previously recorded hand resolution using three-way merge.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---

 * With my admittedly limited tests, this seems to work rather
   well.  I do a lot of "same" merges, because I rebuild "pu"
   every time I have something new in the "master" branch, and
   often what conflict are the things in my own topic branches,
   none of which has been advanced meanwhile.

   The feature is not enabled until you manually create
   $GIT_DIR/rr-cache directory, at least for now.

 Documentation/git-rerere.txt |  177 +++++++++++++++++++++++++++++++++
 Documentation/git.txt        |    3 +
 Makefile                     |    2 
 git-am.sh                    |    1 
 git-commit.sh                |    1 
 git-merge.sh                 |    1 
 git-rerere.perl              |  223 ++++++++++++++++++++++++++++++++++++++++++
 git-reset.sh                 |    2 
 8 files changed, 408 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/git-rerere.txt
 create mode 100755 git-rerere.perl

9843f047463a7b0b993c4bd80d34934618d5d622
diff --git a/Documentation/git-rerere.txt b/Documentation/git-rerere.txt
new file mode 100644
index 0000000..8b6b651
--- /dev/null
+++ b/Documentation/git-rerere.txt
@@ -0,0 +1,177 @@
+git-rerere(1)
+=============
+
+NAME
+----
+git-rerere - Reuse recorded resolve
+
+SYNOPSIS
+--------
+'git-rerere'
+
+
+DESCRIPTION
+-----------
+
+In a workflow that employs relatively long lived topic branches,
+the developer sometimes needs to resolve the same conflict over
+and over again until the topic branches are done (either merged
+to the "release" branch, or sent out and accepted upstream).
+
+This command helps this process by recording conflicted
+automerge results and corresponding hand-resolve results on the
+initial manual merge, and later by noticing the same automerge
+results and applying the previously recorded hand resolution.
+
+[NOTE]
+You need to create `$GIT_DIR/rr-cache` directory to enable this
+command.
+
+DISCUSSION
+----------
+
+When your topic branch modifies overlapping area that your
+master branch (or upstream) touched since your topic branch
+forked from it, you may want to test it with the latest master,
+even before your topic branch is ready to be pushed upstream:
+
+------------
+              o---*---o topic
+             /
+    o---o---o---*---o---o master
+------------
+
+For such a test, you need to merge master and topic somehow.
+One way to do it is to pull master into the topic branch:
+
+------------
+	$ git checkout topic
+	$ git pull . master
+
+              o---*---o---+ topic
+             /           /
+    o---o---o---*---o---o master
+------------
+
+The commits marked with `*` touch the same area in the same
+file; you need to resolve the conflicts when creating the commit
+marked with `+`.  Then you can test the result to make sure your
+work-in-progress still works with what is in the latest master.
+
+After this test merge, there are two ways to continue your work
+on the topic.  The easiest is to build on top of the test merge
+commit `+`, and when your work in the topic branch is finally
+ready, pull the topic branch into master, and/or ask the
+upstream to pull from you.  By that time, however, the master or
+the upstream might have been advanced since the test merge `+`,
+in which case the final commit graph would look like this:
+
+------------
+	$ git checkout topic
+	$ git pull . master
+	$ ... work on both topic and master branches
+	$ git checkout master
+	$ git pull . topic
+
+              o---*---o---+---o---o topic
+             /           /         \
+    o---o---o---*---o---o---o---o---+ master
+------------
+
+When your topic branch is long-lived, however, your topic branch
+would end up having many such "Merge from master" commits on it,
+which would unnecessarily clutter the development history.
+Readers of the Linux kernel mailing list may remember that Linus
+complained about such too frequent test merges when a subsystem
+maintainer asked to pull from a branch full of "useless merges".
+
+As an alternative, to keep the topic branch clean of test
+merges, you could blow away the test merge, and keep building on
+top of the tip before the test merge:
+
+------------
+	$ git checkout topic
+	$ git pull . master
+	$ git reset --hard HEAD^ ;# rewind the test merge
+	$ ... work on both topic and master branches
+	$ git checkout master
+	$ git pull . topic
+
+              o---*---o-------o---o topic
+             /                     \
+    o---o---o---*---o---o---o---o---+ master
+------------
+
+This would leave only one merge commit when your topic branch is
+finally ready and merged into the master branch.  This merge
+would require you to resolve the conflict, introduced by the
+commits marked with `*`.  However, often this conflict is the
+same conflict you resolved when you created the test merge you
+blew away.  `git-rerere` command helps you to resolve this final
+conflicted merge using the information from your earlier hand
+resolve.
+
+Running `git-rerere` command immediately after a conflicted
+automerge records the conflicted working tree files, with the
+usual conflict markers `<<<<<<<`, `=======`, and `>>>>>>>` in
+them.  Later, after you are done resolving the conflicts,
+running `git-rerere` again records the resolved state of these
+files.  Suppose you did this when you created the test merge of
+master into the topic branch.
+
+Next time, running `git-rerere` after seeing a conflicted
+automerge, if the conflict is the same as the earlier one
+recorded, it is noticed and a three-way merge between the
+earlier conflicted automerge, the earlier manual resolution, and
+the current conflicted automerge is performed by the command.
+If this three-way merge resolves cleanly, the result is written
+out to your working tree file, so you would not have to manually
+resolve it.  Note that `git-rerere` leaves the index file alone,
+so you still need to do the final sanity checks with `git diff`
+(or `git diff -c`) and `git update-index` when you are
+satisfied.
+
+As a convenience measure, `git-merge` automatically invokes
+`git-rerere` when it exits with a failed automerge, which
+records it if it is a new conflict, or reuses the earlier hand
+resolve when it is not.  `git-commit` also invokes `git-rerere`
+when recording a merge result.  What this means is that you do
+not have to do anything special yourself (Note: you still have
+to create `$GIT_DIR/rr-cache` directory to enable this command).
+
+In our example, when you did the test merge, the manual
+resolution is recorded, and it will be reused when you do the
+actual merge later with updated master and topic branch, as long
+as the earlier resolution is still applicable.
+
+The information `git-rerere` records is also used when running
+`git-rebase`.  After blowing away the test merge and continuing
+development on the topic branch:
+
+------------
+              o---*---o-------o---o topic
+             /
+    o---o---o---*---o---o---o---o   master
+
+	$ git rebase master topic
+
+				  o---*---o-------o---o topic
+				 /
+    o---o---o---*---o---o---o---o   master
+------------
+
+you could run `git rebase master topic`, to keep yourself
+up-to-date even before your topic is ready to be sent upstream.
+This would result in falling back to three-way merge, and it
+would conflict the same way the test merge you resolved earlier.
+`git-rerere` is run by `git rebase` to help you resolve this
+conflict.
+
+
+Author
+------
+Written by Junio C Hamano <junkio@cox.net>
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Documentation/git.txt b/Documentation/git.txt
index e8ef3ef..2d0ca9d 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -313,6 +313,9 @@ gitlink:git-rebase[1]::
 gitlink:git-repack[1]::
 	Pack unpacked objects in a repository.
 
+gitlink:git-rerere[1]::
+	Reuse recorded resolution of conflicted merges.
+
 gitlink:git-reset[1]::
 	Reset current HEAD to the specified state.
 
diff --git a/Makefile b/Makefile
index 2aa2385..a566951 100644
--- a/Makefile
+++ b/Makefile
@@ -116,7 +116,7 @@ SCRIPT_SH = \
 
 SCRIPT_PERL = \
 	git-archimport.perl git-cvsimport.perl git-relink.perl \
-	git-shortlog.perl git-fmt-merge-msg.perl \
+	git-shortlog.perl git-fmt-merge-msg.perl git-rerere.perl \
 	git-svnimport.perl git-mv.perl git-cvsexportcommit.perl
 
 SCRIPT_PYTHON = \
diff --git a/git-am.sh b/git-am.sh
index 731ab1f..ee6886f 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -88,6 +88,7 @@ fall_back_3way () {
     # saying that we reverted all those changes.
 
     git-merge-resolve $orig_tree -- HEAD $his_tree || {
+	    git-rerere
 	    echo Failed to merge in the changes.
 	    exit 1
     }
diff --git a/git-commit.sh b/git-commit.sh
index 193feeb..bfcb0f2 100755
--- a/git-commit.sh
+++ b/git-commit.sh
@@ -237,6 +237,7 @@ else
 fi
 ret="$?"
 rm -f "$GIT_DIR/COMMIT_MSG" "$GIT_DIR/COMMIT_EDITMSG"
+git-rerere
 
 if test -x "$GIT_DIR"/hooks/post-commit && test "$ret" = 0
 then
diff --git a/git-merge.sh b/git-merge.sh
index 92e5a65..586ac08 100755
--- a/git-merge.sh
+++ b/git-merge.sh
@@ -309,5 +309,6 @@ Conflicts:
 		sed -e 's/^[^	]*	/	/' |
 		uniq
 	} >>"$GIT_DIR/MERGE_MSG"
+	git rerere
 	die "Automatic merge failed; fix up by hand"
 fi
diff --git a/git-rerere.perl b/git-rerere.perl
new file mode 100755
index 0000000..df11951
--- /dev/null
+++ b/git-rerere.perl
@@ -0,0 +1,223 @@
+#!/usr/bin/perl
+#
+# REuse REcorded REsolve.  This tool records a conflicted automerge
+# result and its hand resolution, and helps to resolve future
+# automerge that results in the same conflict.
+#
+# To enable this feature, create a directory 'rr-cache' under your
+# .git/ directory.
+
+use Digest;
+use File::Path;
+use File::Copy;
+
+my $git_dir = $::ENV{GIT_DIR} || ".git";
+my $rr_dir = "$git_dir/rr-cache";
+my $merge_rr = "$git_dir/rr-cache/MERGE_RR";
+
+my %merge_rr = ();
+
+sub read_rr {
+	if (!-f $merge_rr) {
+		%merge_rr = ();
+		return;
+	}
+	my $in;
+	local $/ = "\0";
+	open $in, "<$merge_rr" or die "$!: $merge_rr";
+	while (<$in>) {
+		chomp;
+		my ($name, $path) = /^([0-9a-f]{40})\t(.*)$/s;
+		$merge_rr{$path} = $name;
+	}
+	close $in;
+}
+
+sub write_rr {
+	my $out;
+	open $out, ">$merge_rr" or die "$!: $merge_rr";
+	for my $path (sort keys %merge_rr) {
+		my $name = $merge_rr{$path};
+		print $out "$name\t$path\0";
+	}
+	close $out;
+}
+
+sub compute_conflict_name {
+	my ($path) = @_;
+	my @side = ();
+	my $in;
+	open $in, "<$path"  or die "$!: $path";
+
+	my $sha1 = Digest->new("SHA-1");
+	my $hunk = 0;
+	while (<$in>) {
+		if (/^<<<<<<< .*/) {
+			$hunk++;
+			@side = ([], undef);
+		}
+		elsif (/^=======$/) {
+			$side[1] = [];
+		}
+		elsif (/^>>>>>>> .*/) {
+			my ($one, $two);
+			$one = join('', @{$side[0]});
+			$two = join('', @{$side[1]});
+			if ($two le $one) {
+				($one, $two) = ($two, $one);
+			}
+			$sha1->add($one);
+			$sha1->add("\0");
+			$sha1->add($two);
+			$sha1->add("\0");
+			@side = ();
+		}
+		elsif (@side == 0) {
+			next;
+		}
+		elsif (defined $side[1]) {
+			push @{$side[1]}, $_;
+		}
+		else {
+			push @{$side[0]}, $_;
+		}
+	}
+	close $in;
+	return ($sha1->hexdigest, $hunk);
+}
+
+sub record_preimage {
+	my ($path, $name) = @_;
+	my @side = ();
+	my ($in, $out);
+	open $in, "<$path"  or die "$!: $path";
+	open $out, ">$name" or die "$!: $name";
+
+	while (<$in>) {
+		if (/^<<<<<<< .*/) {
+			@side = ([], undef);
+		}
+		elsif (/^=======$/) {
+			$side[1] = [];
+		}
+		elsif (/^>>>>>>> .*/) {
+			my ($one, $two);
+			$one = join('', @{$side[0]});
+			$two = join('', @{$side[1]});
+			if ($two le $one) {
+				($one, $two) = ($two, $one);
+			}
+			print $out "<<<<<<<\n";
+			print $out $one;
+			print $out "=======\n";
+			print $out $two;
+			print $out ">>>>>>>\n";
+			@side = ();
+		}
+		elsif (@side == 0) {
+			print $out $_;
+		}
+		elsif (defined $side[1]) {
+			push @{$side[1]}, $_;
+		}
+		else {
+			push @{$side[0]}, $_;
+		}
+	}
+	close $out;
+	close $in;
+}
+
+sub find_conflict {
+	my $in;
+	local $/ = "\0";
+	open $in, '-|', qw(git ls-files -z -u) or die "$!: ls-files";
+	my %path = ();
+	my @path = ();
+	while (<$in>) {
+		chomp;
+		my ($mode, $sha1, $stage, $path) =
+		    /^([0-7]+) ([0-9a-f]{40}) ([123])\t(.*)$/s;
+		$path{$path} |= (1 << $stage);
+	}
+	close $in;
+	while (my ($path, $status) = each %path) {
+		if ($status == 14) { push @path, $path; }
+	}
+	return @path;
+}
+
+sub merge {
+	my ($name, $path) = @_;
+	record_preimage($path, "$rr_dir/$name/thisimage");
+	unless (system('merge', map { "$rr_dir/$name/${_}image" }
+		       qw(this pre post))) {
+		my $in;
+		open $in, "<$rr_dir/$name/thisimage" or
+		    die "$!: $name/thisimage";
+		my $out;
+		open $out, ">$path" or die "$!: $path";
+		while (<$in>) { print $out $_; }
+		close $in;
+		close $out;
+		return 1;
+	}
+	return 0;
+}
+
+-d "$rr_dir" || exit(0);
+
+read_rr();
+my %conflict = map { $_ => 1 } find_conflict();
+
+# MERGE_RR records paths with conflicts immediately after merge
+# failed.  Some of the conflicted paths might have been hand resolved
+# in the working tree since then, but the initial run would catch all
+# and register their preimages.
+
+for my $path (keys %conflict) {
+	# This path has conflict.  If it is not recorded yet,
+	# record the pre-image.
+	if (!exists $merge_rr{$path}) {
+		my ($name, $hunk) = compute_conflict_name($path);
+		next unless ($hunk);
+		$merge_rr{$path} = $name;
+		if (! -d "$rr_dir/$name") {
+			mkpath("$rr_dir/$name", 0, 0777);
+			print STDERR "Recorded preimage for '$path'\n";
+			record_preimage($path, "$rr_dir/$name/preimage");
+		}
+	}
+}
+
+# Now some of the paths that had conflicts earlier might have been
+# hand resolved.  Others may be similar to a conflict already that
+# was resolved before.
+
+for my $path (keys %merge_rr) {
+	my $name = $merge_rr{$path};
+
+	# We could resolve this automatically if we have images.
+	if (-f "$rr_dir/$name/preimage" &&
+	    -f "$rr_dir/$name/postimage") {
+		if (merge($name, $path)) {
+			print STDERR "Resolved '$path' using previous resolution.\n";
+			# Then we do not have to worry about this path
+			# anymore.
+			delete $merge_rr{$path};
+			next;
+		}
+	}
+
+	# Let's see if we have resolved it.
+	(undef, my $hunk) = compute_conflict_name($path);
+	next if ($hunk);
+
+	print STDERR "Recorded resolution for '$path'.\n";
+	copy($path, "$rr_dir/$name/postimage");
+	# And we do not have to worry about this path anymore.
+	delete $merge_rr{$path};
+}
+
+# Write out the rest.
+write_rr();
diff --git a/git-reset.sh b/git-reset.sh
index 6c9e58a..fe53fc8 100755
--- a/git-reset.sh
+++ b/git-reset.sh
@@ -100,4 +100,4 @@ case "$reset_type" in
 	;;
 esac
 
-rm -f "$GIT_DIR/MERGE_HEAD"
+rm -f "$GIT_DIR/MERGE_HEAD" "$GIT_DIR/rr-cache/MERGE_RR"
-- 
1.1.5.g3480-dirty