Merge with staged and unstaged changes

Merge with staged and unstaged changes

[Edward Thomson]

Hi-

I've been investigating the cases where merge is allowed to proceed when
there are staged changes in the index or unstaged files in the working
directory.  There are cases where I find the behavior surprising and I
hope I can get clarification.  There are also two cases that I will report
as bugs, where it appears that the unstaged file contents are deleted.

For these cases below, please consider the contents of a single path.
In the tables below, we will show the contents of a file across each input
and output of the merge - consider that we're merging a single file in
some branch "theirs" into the current branch "ours" and that these two
branches have a common ancestor "anc".  The state of that file in our
index and workdir are represented by "idx" and "wd", respectively.
Unless otherwise noted, these cases are true for both git-merge-resolve
and git-merge-recursive.


For completeness and illustration purposes, I'll included the cases where
there are no changes staged or unstaged.  These succeed, as expected:

   input                                 result
   anc ours theirs idx wd  merge result  idx wd
1  A   A    B      A   A   take B        B   B
2  A   B    A      B   B   take A        A   A

Merge is also expected to proceed if the contents of our branch are the
merge result, and there are unstaged changes for that file in the workdir.
In this case, the file remains unstaged:

   input                                 result
   anc ours theirs idx wd  merge result  idx wd
3  A   B    A      B   C   take B        B   C


What was surprising to me was that my merge can proceed if I stage a change
that is identical to the merge result.  That is, if my merge result would
be to take the contents from "theirs", then my merge can proceed if I've
already staged the same contents:

   input                                 result
   anc ours theirs idx wd  merge result  idx wd
4  A   A    B      B   B   take B        B   B
5  A   A    B      B   C   take B        B   C

This seems unexpected - is there a use-case that this enables or is
this accidental?


Another surprising result was that if I have deleted a file (and staged
the deletion or not) then the merge will proceed and the file in question
will be recreated.  Consider "X" to be a missing file:

   input                                 result
   anc ours theirs idx wd  merge result  idx wd
6  A   A    B      A   X   take B        B   B
7  A   A    B      X   X   take B        B   B

I wouldn't have expected a file I deleted to be recreated with the other
branch's contents.  Is this behavior also intentional?


Finally, there are cases when you have staged a deletion of the file and
you have unstaged changes in your workdir where the merge will silently
delete the unstaged data.  If there is a conflict, the xdiff output will
overwrite the unstaged file:

   input                                 result
   anc ours theirs idx wd  merge result  idx wd
8  A   B    C      X   D   conflict      X   diff3_file

And similarly, while git-merge-recursive (only) will also remove my
untracked file when there are no changes in our branch but the file was
deleted in their branch:

   input                                 result
   anc ours theirs idx wd  merge result  idx wd
9  A   A    X      X   B   delete file   X   X


I trust the last two cases, where data is lost, are bugs to report, but
could I get clarification on the other situations?

Thanks-

-ed

Re: Merge with staged and unstaged changes

[Junio C Hamano]

Edward Thomson <ethomson@microsoft.com> writes:

> What was surprising to me was that my merge can proceed if I stage a change
> that is identical to the merge result.  That is, if my merge result would
> be to take the contents from "theirs", then my merge can proceed if I've
> already staged the same contents:
>
>    input                                 result
>    anc ours theirs idx wd  merge result  idx wd
> 4  A   A    B      B   B   take B        B   B
> 5  A   A    B      B   C   take B        B   C
>
> This seems unexpected - is there a use-case that this enables or is
> this accidental?

Both are very much on purpose. The integrator may have seen the
patch on the list, ran "git apply [--index]" on it to contemplate on
it, and before commiting the result, saw a pull request for a branch
that contains the change.  The above two allow the pull from such a
state to succeed without losing any information.

I think we have a similar table in Documentation/technical area that
explains these things, by the way.

> Another surprising result was that if I have deleted a file (and staged
> the deletion or not) then the merge will proceed and the file in question
> will be recreated.  Consider "X" to be a missing file:
>
>    input                                 result
>    anc ours theirs idx wd  merge result  idx wd
> 6  A   A    B      A   X   take B        B   B
> 7  A   A    B      X   X   take B        B   B

I am not sure about #7, but #6 is done very much on purpose.  The
lower level merge machinery deliberately equates "in index but not
checked out to the working tree" state and "in index and not
modified in the working tree" state; this is to support a merge done
in a temporary working area that starts out empty.

This was designed really long time ago (read: during the first two
weeks of Git development, back when there were no "stash" nor
"remote tracking branches"), with a vision to make this scenario
work nicely:

    You have checked out 'master' branch and you are working on it.
    You see a more urgent pull request on 'maint' branch.  But your
    working tree for 'master' is no shape to commit, yet.

Without disturbing the working area you are using to advance the
'master' branch, you could do the merge "only in the index" by doing:

 * Fetch the requested commit:
 
	$ git fetch $over_there refs/heads/master

 * Populate a temporary index with the contents of your 'maint':

	$ GIT_INDEX_FILE=,,temp-index GIT_DIR=$(pwd)/.git
        $ export GIT_INDEX_FILE GIT_DIR
        $ git read-tree refs/heads/maint

 * Create an empty temporary working area and go there:

	$ mkdir ,,temp-merge
	$ cd ,,temp-merge

 * Run the three-way merge between your 'maint' and the requested
   commit:

	$ MB=$(git merge-base FETCH_HEAD maint)
	$ git read-tree -m -u $MB maint FETCH_HEAD

Notice that we start without _any_ file in that temporary working
area (,,temp-merge directory).  In the last step, because the merge
machinery (read-tree -m -u) treats missing files as "they are
unmodified by us; we didn't even bother checking them out of the
index", we will see _only_ the files that are different from our
'maint' and files that needs our help to get conflicts resolved
in the temporary working area after the command finishes.  You
inspect them, resolve conflicts and run "git update-index" on them
(remember, there weren't "git add" or "git commit -a"), write the
resulting index as a tree with "git write-tree" and record the tree
with "git commit-tree -p maint -p FETCH_HEAD" (actually, back then
"commit-tree -p" insisted on getting raw tree object names, so you
had to do the equivalent of "git rev-parse maint^{tree}" there) and
then update your 'maint' branch with the resulting commit.

If there is no conflict to be resolved, then you essentially can run
a script that does all of the above (and cleans up the temporary
directory ,,temp-merge and the temporary index) to implement "Can
perform a merge into a branch that is not currently checked out,
without disturbing my current work" feature.  And this "missing
files are unmodified---I didn't even bother to check them out of the
index" was done as an integral part of it.

Back when we (IIRC, mostly between Linus and I) were discussing this
design, it did come up that in the normal case of "I am merging in
my working tree", this behaviour would lose information, but "I'm
planning to remove this" is only a single bit and lossage of that
was judged to be trivially small and easily recoverable compared to
the benefit of being able to do a merge in an empty working tree,
and that is where this behaviour comes from.

We could certainly revisit this design and make the behaviour
optional.  When somebody wants to do the "Can perform a merge into a
branch that is not currently checked out, without disturbing my
current work" feature, its implementation needs to be able to turn
it back on, but for doing everything else we do not have to treat a
missing file as unmodified.

Re: Merge with staged and unstaged changes

[Edward Thomson]

On 2/20/13 2:21 PM, "Junio C Hamano" <gitster@pobox.com> wrote:

>Both are very much on purpose. The integrator may have seen the
>patch on the list, ran "git apply [--index]" on it to contemplate on
>it, and before commiting the result, saw a pull request for a branch
>that contains the change.  The above two allow the pull from such a
>state to succeed without losing any information.

>I think we have a similar table in Documentation/technical area that
>explains these things, by the way.

I believe you are referring to trivial-merge.txt which has been
exceptionally helpful in understanding "what" unpack trees does.
I appreciate this detailed explanation in providing the "why".

I also appreciate your explanation of the affect of the workdir,
and that makes sense.  I would have expected that the default was
to presume the workdir files were existent, rather than the
other way around, but we can agree that is an implementation detail.

My biggest concern, of course, was having the unstaged files in my
workdir overwritten or deleted.

Thanks again-

-ed

Re: Merge with staged and unstaged changes

[Junio C Hamano]

Edward Thomson <ethomson@microsoft.com> writes:

> I also appreciate your explanation of the affect of the workdir,
> and that makes sense.  I would have expected that the default was
> to presume the workdir files were existent, rather than the
> other way around, but we can agree that is an implementation detail.
>
> My biggest concern, of course, was having the unstaged files in my
> workdir overwritten or deleted.

Oh, no question about that part.  You concluded your original
message with:

>> I trust the last two cases, where data is lost, are bugs to
>> report, but could I get clarification on the other situations?

and I was responding to the part after the "but could I get...".

I am fairly familiar with the "read-tree -m -u O A B" three-way
merge codepath (after all I designed that with Linus in the very
early days of Git), but I am not as familar with the merge-recursive
backend as merge-resolve, and I was hoping to see the "bug" part
triaged by other people.