git-apply that handles rejects like merge conflicts

git-apply that handles rejects like merge conflicts

[Ori Avtalion]

Hey,

I'm trying to get git-apply to apply patches, and let me handle the
conflicts in a way I'm comfortable with -- by staging the "successful"
hunks and leaving conflict markers in the working tree.

With the available flags, I seem to only be able to have successful
hunks in the index, and rejected ones in patch-like .rej files.

Is there a way to accomplish this? If not, does anyone think it's a good
idea?

-Ori

Re: git-apply that handles rejects like merge conflicts

[Jeff King]

On Tue, Nov 08, 2011 at 12:10:48AM +0200, Ori Avtalion wrote:

> I'm trying to get git-apply to apply patches, and let me handle the
> conflicts in a way I'm comfortable with -- by staging the "successful"
> hunks and leaving conflict markers in the working tree.
> 
> With the available flags, I seem to only be able to have successful
> hunks in the index, and rejected ones in patch-like .rej files.
> 
> Is there a way to accomplish this? If not, does anyone think it's a good
> idea?

No, I don't think there's a way to do it now. I do find the conflict
markers the easiest way to mark something up. But most of my "git apply"
use is through "git am", which knows the trick of falling back to a
3-way merge (see fall_back_3way in git-am.sh).

If it's an actual git diff, the same 3-way trick will yield good
results, and it would be nice if it were easier to do that trick without
calling "git am". But if it's not a git diff (i.e., missing the original
blob information), then you won't be able to do that.

In the general case, you can't represent all failed hunks with conflict
markers, can you? I'm thinking something where we couldn't find any
relevant context. You know the lines from the original patch from the
hunk header, so you can drop the failed content from the patch in the
right spot. But how do you know how big a conflict marker to make for
the "current" side? The same number of lines as were in the hunk?
I think you'd end up with confusing conflict markers.

-Peff

Re: git-apply that handles rejects like merge conflicts

[Ori Avtalion]

On 11/08/2011 12:55 AM, Jeff King wrote:
> If it's an actual git diff, the same 3-way trick will yield good
> results, and it would be nice if it were easier to do that trick without
> calling "git am". But if it's not a git diff (i.e., missing the original
> blob information), then you won't be able to do that.

I'm dealing with two codebases that have branched in the past, before
any VCS was used, and now I'm tracking both separately with git. I'm
trying to  apply changes from one to the other with format-patch and
git-am/apply. So yeah, no blob info.

> In the general case, you can't represent all failed hunks with conflict
> markers, can you? I'm thinking something where we couldn't find any
> relevant context. You know the lines from the original patch from the
> hunk header, so you can drop the failed content from the patch in the
> right spot. But how do you know how big a conflict marker to make for
> the "current" side? The same number of lines as were in the hunk?
> I think you'd end up with confusing conflict markers.

Personally, I wouldn't object to having both "computable" conflicts, and
the .rej files for hunks that lack context, but I see how that would be
very confusing. :)

-Ori

Re: git-apply that handles rejects like merge conflicts

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> In the general case, you can't represent all failed hunks with conflict
> markers, can you?

Conflict markers come from the use of a 3-way merge, and if you were to do
a 3-way merge, by definition, you would need some way to tell where the
preimage of the patch and the target tree you are attempting to apply the
patch forked from. That's done by fall-back-3way in "am -3".

You _could_ lift that logic out of "am -3", but I do not think it is worth
the effort to do so (IOW, I do not see a reason to avoid "am -3").

If you do not want to create a commit for whatever reason, then you can
"reset --soft" back.

Re: git-apply that handles rejects like merge conflicts

[Jeff King]

On Mon, Nov 07, 2011 at 03:45:48PM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > In the general case, you can't represent all failed hunks with conflict
> > markers, can you?
> 
> Conflict markers come from the use of a 3-way merge, and if you were to do
> a 3-way merge, by definition, you would need some way to tell where the
> preimage of the patch and the target tree you are attempting to apply the
> patch forked from. That's done by fall-back-3way in "am -3".
> 
> You _could_ lift that logic out of "am -3", but I do not think it is worth
> the effort to do so (IOW, I do not see a reason to avoid "am -3").

I think it would purely be "I have a patch produced by git diff, not by
git format-patch". If you want to use "am -3", you would have to dress
up your patch with mail headers.

In practice, this doesn't come up much for me. I think I was using "git
diff >patch" as a poor-man's stash (and I did just stick some fake
headers in, and "git reset HEAD^" afterwards). But maybe other workflows
deal with this more.

But I think there are two questions:

  1. Should am's 3-way fallback be made more easily available to users
     of regular "apply"?

  2. Short of doing a 3-way merge, are there better ways to represent
     failed hunks in the patch target itself, rather than saving ".rej"
     files?

I'm actually not sure which one Ori was asking about.

-Peff

Re: git-apply that handles rejects like merge conflicts

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> But I think there are two questions:
>
>   1. Should am's 3-way fallback be made more easily available to users
>      of regular "apply"?
>
>   2. Short of doing a 3-way merge, are there better ways to represent
>      failed hunks in the patch target itself, rather than saving ".rej"
>      files?
>
> I'm actually not sure which one Ori was asking about.

Me neither, but if I have to guess it would be the former. If there were a
solution better than ".rej" in 2-way context, surely "patch" would have
implemented it 10 years before we started ;-).

Re: git-apply that handles rejects like merge conflicts

[Ori Avtalion]

On 11/08/2011 07:46 AM, Jeff King wrote:
> On Mon, Nov 07, 2011 at 03:45:48PM -0800, Junio C Hamano wrote:
> But I think there are two questions:
> 
[ snip ]
> 
> I'm actually not sure which one Ori was asking about.
> 

I'm actually interested in both :)

Here's a copy of the description of my problem from another reply:

> I'm dealing with two codebases that have branched in the past, before
> any VCS was used, and now I'm tracking both separately with git. I'm
> trying to  apply changes from one to the other with format-patch and
> git-am/apply.

In answer to your first question
>   1. Should am's 3-way fallback be made more easily available to users
>      of regular "apply"?

git-am is never part of this workflow as I'm trying to move patches
between separate repositories with no shared root.

<rant>
And, personally, I don't think git-am is named correctly as the only
use-case I have for it is applying+committing single patches produced by
format-patch and sent as individual files over some medium which isn't
mboxes (I'm not that old-school). I never understood why git-apply can't
do the commit and I have to instead use a tool with 'mail' in its name
(Let's ignore the historical reasons) -- Shouldn't git-am be an
mbox-reading wrapper around some more basic patch-applying tool?
</rant>

>>   2. Short of doing a 3-way merge, are there better ways to represent
>>      failed hunks in the patch target itself, rather than saving ".rej"
>>      files?

I really want this as .rej files feel very un-git-like. However, after
understanding the problems raised in this thread, I'm a bit more
realistic :)

-Ori

Re: git-apply that handles rejects like merge conflicts

[Jeff King]

On Tue, Nov 08, 2011 at 10:50:51PM +0200, Ori Avtalion wrote:

> In answer to your first question
> >   1. Should am's 3-way fallback be made more easily available to users
> >      of regular "apply"?
> 
> git-am is never part of this workflow as I'm trying to move patches
> between separate repositories with no shared root.

Isn't git-am the right tool for that? You format-patch out your commits
in one repo, and then apply them in the other. No shared history is
required; just the ability of the patches to actually be applied.

It _helps_ if you have the common base objects (the actual files, not
the commits), since the git diffs carry the pre- and post-image file
sha1s, which is what allows us to do a real 3-way merge. In that case,
it is just a matter of making the objects from the first repo available
to git during the moment you are applying in the second repo. You could
do it by fetching the history of the first into a side-branch of the
second, or even just by sharing object databases via the "alternates"
mechanism.

> <rant>
> And, personally, I don't think git-am is named correctly as the only
> use-case I have for it is applying+committing single patches produced by
> format-patch and sent as individual files over some medium which isn't
> mboxes (I'm not that old-school). I never understood why git-apply can't
> do the commit and I have to instead use a tool with 'mail' in its name
> (Let's ignore the historical reasons) -- Shouldn't git-am be an
> mbox-reading wrapper around some more basic patch-applying tool?
> </rant>

git-am _is_ an mbox-reading wrapper around some more basic
patch-applying tool. That tool is "git apply". I think what you are
missing is that a single patch (or multiple patches) produced by
format-patch _is_ an mbox. There is nothing wrong with:

  cd repo1 &&
  git format-patch -1 --stdout >../my.patch &&
  cd ../repo2 &&
  git am ../my.patch

There is no standard for representing commit metadata in the diff
format. So git had to invent its own. It used rfc822 messages and
mailboxes because it was simple and convenient, it mapped to what some
people were already doing, and it means we don't need a separate tool
for applying local commits versus ones that were emailed.

So the "m" is really for mbox, which happens to be git's format for
storing one or more commits, including metadata. If you just forget that
it's associated with mail, then I think you will be happy. :)

-Peff

Re: git-apply that handles rejects like merge conflicts

[Bert Wesarg]

On Mon, Nov 7, 2011 at 23:55, Jeff King <peff@peff.net> wrote:
> In the general case, you can't represent all failed hunks with conflict
> markers, can you? I'm thinking something where we couldn't find any
> relevant context. You know the lines from the original patch from the
> hunk header, so you can drop the failed content from the patch in the
> right spot. But how do you know how big a conflict marker to make for
> the "current" side? The same number of lines as were in the hunk?
> I think you'd end up with confusing conflict markers.

GNU patch can produce conflict markers with the --merge option.

Bert

>
> -Peff

Re: git-apply that handles rejects like merge conflicts

[Jeff King]

On Tue, Nov 08, 2011 at 09:52:26AM +0100, Bert Wesarg wrote:

> On Mon, Nov 7, 2011 at 23:55, Jeff King <peff@peff.net> wrote:
> > In the general case, you can't represent all failed hunks with conflict
> > markers, can you? I'm thinking something where we couldn't find any
> > relevant context. You know the lines from the original patch from the
> > hunk header, so you can drop the failed content from the patch in the
> > right spot. But how do you know how big a conflict marker to make for
> > the "current" side? The same number of lines as were in the hunk?
> > I think you'd end up with confusing conflict markers.
> 
> GNU patch can produce conflict markers with the --merge option.

Hmm. Yeah, it does work, but as I feared, it can produce pretty awful
conflicts. Try this fairly straightforward setup (3 lines changed in the
middle of a file):

  git init &&
  seq 1 10 >file && git add file && git commit -m base &&
  sed -i '3,5s/$/ master/' file && git commit -a -m master &&
  git checkout -b other HEAD^ &&
  sed -i '3,5s/$/ other/' file && git commit -a -m other

You can see what a real merge looks like:

  git merge master &&
  $EDITOR file

which is:

  1
  2
  <<<<<<< HEAD
  3 other
  4 other
  5 other
  =======
  3 master
  4 master
  5 master
  >>>>>>> master
  6
  7
  8
  9
  10

If you use "patch --merge", you get the same thing. Which is good. But
now try it with 10 lines changed out of 100:

  rm -rf .git
  git init &&
  seq 1 100 >file && git add file && git commit -m base &&
  sed -i '50,60s/$/ master/' file && git commit -a -m master &&
  git checkout -b other HEAD^ &&
  sed -i '50,60s/$/ other/' file && git commit -a -m other

Doing a merge will get you the same sensible results. But "patch
--merge" produces:

 ...
 45
 46
 <<<<<<<
 =======
 47
 48
 49
 50 master
 ...
 60 master
 61
 62
 63
 >>>>>>>
 47
 48
 49
 50 other
 51 other
 52 other
 53 other
 ...

which is not that helpful. Interestingly, I think it _should_ be able to
do the same thing here as it did on the 3-line case. So I'm not sure
why it doesn't.

But there are even more complex cases, like say "other" had added new
lines of new content at the beginning of the file, and messed up the
context lines that the patch was using. So I think in the general case,
you will end up with patches like the latter one. Just shoving the patch
hunk into the file with an empty preimage section. And that can even
still be useful, but you are relying on line counts then. If they're
off, it's going ot be quite confusing.

-Peff