git-am failed, what's next ?

git-am failed, what's next ?

[Francis Moreau]

Hi,

I'm a bit clueless when git-am failed to apply a patch. I dunno what I
should do at this point since errors reported by git-am are not
usefull for me. For example I got:

----
error: patch failed: foo:1
error: foo: patch does not apply

Patch failed at 0001.
When you have resolved this problem run "git-am --resolved".
If you would prefer to skip this patch, instead run "git-am --skip".
---

I know that git-am let some information in '.dotest' directory and
that I can find in it the plain patch file. I can use 'patch' command
to apply it and see which part of the patch is conflicting. But I
would like to know if there are other ways to do it specially by using
git ?

BTW, would it be possible to be more verbose when describing the error ?

thanks
-- 
Francis

PS: git rocks !

Re: git-am failed, what's next ?

[Andy Parkins]

On Thursday 2007 February 22 08:22, Francis Moreau wrote:

> ----
> error: patch failed: foo:1
> error: foo: patch does not apply
>
> Patch failed at 0001.
> When you have resolved this problem run "git-am --resolved".
> If you would prefer to skip this patch, instead run "git-am --skip".
> ---

I've often wished for more information from git-am (or more correctly, 
git-apply).  However, you can sometimes get what you need to know another 
way.

 * First, make sure that your current tree is checked in so you can get back
   to it easily.  Maybe switch to a new temporary branch to make it easy to
   return to your current point.
 * Then run git-am to get the number of the failing patch, in your example 
   it's "0001"
 * Now, try and apply the patch manually, but turn on verbose and reject in 
   git-apply
    $ git-apply --verbose --reject .dotest/0001
   This is the only way I've found to get git to tell you which hunk of the
   patch is being rejected.  Unfortunately, it will also leave you with that
   patch partially applied.
 * Sometimes the partially applied patch will be enough.  Have a look at the
   *.rej files that have been created and see if you can resolve the conflict
   by hand - you could apply the patch by hand if it's only a small hunk.
   If you do this, you can then continue with the git-am by first updating the 
   index (git-add somefile.c), then running git-am --resolved
 * If you wanted to you could just skip that patch with "git-am --skip"
 * If you wish you hadn't started any of this you can obviously use git-reset 
   to get you back to the start point.

Hope that helps.


Andy

-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: git-am failed, what's next ?

[Junio C Hamano]

"git-am" deserves a section in the user-manual on its own.
Perhaps the following can be massaged into JBF's User Manual,
but probably with copyediting and spellfixes.

First, the basics.
------------------

When "git am ./mbox" stops in the middle, you will find
these files in the .dotest/ directory:

0002 0003 0004...::

        Numbered files are split e-mails, yet to be applied, and
        the first one is the one that failed to apply.  Already
        applied ones are removed so they may not begin with
        0001.

next, last::

        These files are used to control sequencing.  'next' is
        the patch you are applying, and 'last' is the number of
        the last patch in the series.

info msg msg-clean final-commit patch::

        These files record information of the patch you are
        applying.  'info' lists the metainformation for the
        commit to be made, 'msg' is the raw message extracted
        from e-mail, 'msg-clean' is a copy of it after stripping
        excess whitespaces, and 'final-commit' is after adding
        sign-off lines (when -s is given).  'patch' is the patch
        text itself.

There are two strategies to resume an "am" that stopped because
a patch does not apply.

One is to fix the patch into applicable shape in whatever way
you can think of, and attempt to apply it again, with "git am".

The other is to update your index to the desired state after
applying the given patch in whatever way you can think of, and
continue with "git am --resolved".

You can also choose not to continue, but skip one patch.  You
can say:

	$ git reset --hard
        $ git am --skip

to skip that particular patch.  This advances 'next' and
continues with the remainder.

[jc: I think that is not what you want in your RFH, but I am
spelling it out for inclusion in the User Manual -- this comment
itself should be removed if somebody wants to include this
message to the manual.]

Now, let's go on to "whatever way you can think of".


Fixing patch text
-----------------

Linus's way is to visit the ./mbox file in the editor, remove
the messages that have already been applied cleanly, edit the
offending patch text to make it applicable.  Then:

	$ git reset --hard
        $ rm -fr .dotest
        $ git am edited-mailbox-file

A variant of this approach is to edit .dotest/patch to make it
applicable, and then say (without removing .dotest/ directory,
of course):

	$ git am

But these are probably reserved for people who are comfortable
editing the patch text.


Applying by hand
----------------

If you (as most people) are not comfortable editing the patch
text you can update your working tree to pretend that the patch
applied cleanly.  You can do number of things:

 * Use "git apply --index -C<n> .dotest/patch" to attempt
   applying the patch with reduced context.

 * Use "git apply --reject .dotest/patch" to get apply only
   hunks that apply cleanly, while getting *.rej files.

 * Use "GNU patch" with less strict options, perhaps like

	$ patch -p1 --fuzz=<n> --ignore-whitespace

Whatever you do, do not forget that your goal with this strategy
is to prepare your index into a shape that the patch should have
made it into, if it applied cleanly.  In other words, after you
are done, "git diff --cached HEAD" should produce what the
original patch in .dotest/patch should have contained.  Also,
typically, "git diff" at that point should say nothing (unless
you know what you are doing).

So, if you used "git apply" without --index (or plain "GNU
patch"), do not forget to "git add" to register the result in
the index.  Especially, if the patch adds a new file, do not
forget to include it in the index!

After you are done, you would continue with:

	$ git am --resolved


Letting git work harder
-----------------------

It _could_ be that the original submitter prepared the patch to
a different branch of your repository than the branch you are
trying to apply the patch to.  Sometimes I get a patch that is
against my 'next' but the change does not depend on what is only
in 'next' and should be applied to 'master' (or even 'maint').
If you have the pre-image blobs the patch was created against,
and the patch was created with git and records "index" lines
like these correctly:

        diff --git a/builtin-config.c b/builtin-config.c
        index 0f9051d..f1433a4 100644

then you can tell git to fall back on 3-way merge to apply the
patch.  After seeing "git am ./mbox" fail, you could try
(without doing anything else):

	$ git am -3

This can result in one of three things:

 * It can complain, with "Patch does not record usable index
   information".  When this happens, you are back to square-one.
   Your repository does not have enough blobs for it to fall
   back on 3-way merge.  The operation did not change anything
   in your working tree nor what are in .dotest/ directory, so
   it did not do any further harm.  You have to use one of the
   two strategies described above instead.

 * It can successfully fall back on 3-way merge and cleanly
   apply the patch.  Lucky you.

 * It can fall back on 3-way merge but the merge can conflict.

In the third case, the usual conflict resolution techniques you
would use during a conflicted merge applies, except that you do
NOT conclude it with "git commit".

This "3-way" is in fact a variant of "Applying by hand"
strategy, and your goal is still to update the index into the
shape the original patch should have made into.  After you
resolve the conflict, you do "git add" the resolved path (or if
the patch is about removing a path and if you do want to remove
that path, then "git rm").  Make sure that "git diff --cached
HEAD" output matches what you think .dotest/patch should have
contained.  Then:

	$ git am --resolved

to continue.

Re: git-am failed, what's next ?

[Francis Moreau]

Hi,

On 2/22/07, Andy Parkins <andyparkins@gmail.com> wrote:
> I've often wished for more information from git-am (or more correctly,
> git-apply).  However, you can sometimes get what you need to know another
> way.
>

it seems pretty the same flow that I'm used to doing except that I
used 'patch' instead of 'git-apply'...

>  * First, make sure that your current tree is checked in so you can get back
>    to it easily.  Maybe switch to a new temporary branch to make it easy to
>    return to your current point.
>  * Then run git-am to get the number of the failing patch, in your example
>    it's "0001"
>  * Now, try and apply the patch manually, but turn on verbose and reject in
>    git-apply
>     $ git-apply --verbose --reject .dotest/0001
>    This is the only way I've found to get git to tell you which hunk of the
>    patch is being rejected.  Unfortunately, it will also leave you with that
>    patch partially applied.

hm, I actually used 'patch' instead because I wasn't aware of
'--reject' option. The default behaviour is different, and I'm not
sure to know why...

Obviously I really should play with it.

> Hope that helps.

thanks
-- 
Francis

Re: git-am failed, what's next ?

[Francis Moreau]

Hi,

Junio C Hamano wrote:
> "git-am" deserves a section in the user-manual on its own.
> Perhaps the following can be massaged into JBF's User Manual,
> but probably with copyediting and spellfixes.
>

yes it's definitly something useful for me. Thanks for writing it !

> First, the basics.
> ------------------
>
> When "git am ./mbox" stops in the middle, you will find
> these files in the .dotest/ directory:
>

[snip]

> There are two strategies to resume an "am" that stopped because
> a patch does not apply.
>
> One is to fix the patch into applicable shape in whatever way
> you can think of, and attempt to apply it again, with "git am".
>
> The other is to update your index to the desired state after
> applying the given patch in whatever way you can think of, and
> continue with "git am --resolved".
>
> You can also choose not to continue, but skip one patch.  You
> can say:
>
> 	$ git reset --hard
>         $ git am --skip
>
> to skip that particular patch.  This advances 'next' and
> continues with the remainder.
>
> [jc: I think that is not what you want in your RFH, but I am
> spelling it out for inclusion in the User Manual -- this comment
> itself should be removed if somebody wants to include this
> message to the manual.]
>

nope, and this is quite well explained in the man page.

> Now, let's go on to "whatever way you can think of".
>
>
> Fixing patch text
> -----------------
>
> Linus's way is to visit the ./mbox file in the editor, remove
> the messages that have already been applied cleanly, edit the
> offending patch text to make it applicable.  Then:
>
> 	$ git reset --hard
>         $ rm -fr .dotest
>         $ git am edited-mailbox-file
>
> A variant of this approach is to edit .dotest/patch to make it
> applicable, and then say (without removing .dotest/ directory,
> of course):
>
> 	$ git am
>
> But these are probably reserved for people who are comfortable
> editing the patch text.
>

definitely not my case...

>
> Applying by hand
> ----------------
>
> If you (as most people) are not comfortable editing the patch
> text you can update your working tree to pretend that the patch
> applied cleanly.  You can do number of things:
>
>  * Use "git apply --index -C<n> .dotest/patch" to attempt
>    applying the patch with reduced context.
>
>  * Use "git apply --reject .dotest/patch" to get apply only
>    hunks that apply cleanly, while getting *.rej files.
>
>  * Use "GNU patch" with less strict options, perhaps like
>
> 	$ patch -p1 --fuzz=<n> --ignore-whitespace
>

why using 'patch' ? couldn't this be done by using 'git-apply' ?

Maybe these several choices could be proposed to the user when using
the interactive mode ?

>
>
> Letting git work harder
> -----------------------
>

[snip]

> This "3-way" is in fact a variant of "Applying by hand"
> strategy, and your goal is still to update the index into the
> shape the original patch should have made into.  After you
> resolve the conflict, you do "git add" the resolved path (or if
> the patch is about removing a path and if you do want to remove
> that path, then "git rm").  Make sure that "git diff --cached
> HEAD" output matches what you think .dotest/patch should have
> contained.  Then:
>
> 	$ git am --resolved
>
> to continue.
>

This part is really interesting.
-- 
Francis

Re: git-am failed, what's next ?

[Francis Moreau]

Junio C Hamano wrote:
> 	$ git am -3
>
> This can result in one of three things:
>
>  * It can complain, with "Patch does not record usable index
>    information".  When this happens, you are back to square-one.
>    Your repository does not have enough blobs for it to fall
>    back on 3-way merge.  The operation did not change anything
>    in your working tree nor what are in .dotest/ directory, so
>    it did not do any further harm.  You have to use one of the
>    two strategies described above instead.
>

BTW, in this case I got the following error message:

fatal: sha1 information is lacking or useless (foo).
Patch does not record usable index information.
Cannot fall back to three-way merge.
Patch failed at 0001.

which is hard to understand, whereas your explanation:

Your repository does not have enough blobs for it to fall
back on 3-way merge.

sounds better and easier to understand to me...
-- 
Francis

Re: git-am failed, what's next ?

[Andy Parkins]

On Thursday 2007 February 22 09:47, Junio C Hamano wrote:

> If you have the pre-image blobs the patch was created against,
> and the patch was created with git and records "index" lines
> like these correctly:
>
>         diff --git a/builtin-config.c b/builtin-config.c
>         index 0f9051d..f1433a4 100644
>
> then you can tell git to fall back on 3-way merge to apply the
> patch.  After seeing "git am ./mbox" fail, you could try

Magic.  I had no idea about this.  Could it be made even more magical by doing 
this for you?

git-am would know if the patch that failed is a git-generated patch, so when 
things go wrong, before bombing out it could try a 3-way merge.  It's much 
easier to resolve faults with the usual conflict markers than reading 
patches.  It's also possible that the 3-way merge will succeed and git-am 
will continue without ever needing to worry the user.


Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: git-am failed, what's next ?

[Johannes Schindelin]

Hi,

On Thu, 22 Feb 2007, Francis Moreau wrote:

> I'm a bit clueless when git-am failed to apply a patch. I dunno what I
> should do at this point since errors reported by git-am are not
> usefull for me. For example I got:
> 
> ----
> error: patch failed: foo:1
> error: foo: patch does not apply

I had that quite often. I then ran

	git apply --verbose < .dotest/patch

to find out why it fails, but alas it did not tell me more. That's why I 
wrote this patch:

-- snipsnap --
[PATCH] apply: make --verbose a little more useful

When a patch fails, I automatically add '-v' to the command line
to see what fails.

This patch makes -v a synonym to --verbose, and actually tells
the user which text was not found.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
---
 builtin-apply.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/builtin-apply.c b/builtin-apply.c
index 630d8fd..2dde341 100644
--- a/builtin-apply.c
+++ b/builtin-apply.c
@@ -1723,6 +1723,8 @@ static int apply_one_fragment(struct buffer_desc *desc, struct fragment *frag, i
 			/* Ignore it, we already handled it */
 			break;
 		default:
+			if (apply_verbosely)
+				error("invalid start of line: '%c'", first);
 			return -1;
 		}
 		patch += len;
@@ -1820,6 +1822,9 @@ static int apply_one_fragment(struct buffer_desc *desc, struct fragment *frag, i
 		}
 	}
 
+	if (offset && apply_verbosely)
+		error("while searching for:\n%.*s", oldsize, oldlines);
+
 	free(old);
 	free(new);
 	return offset;
@@ -2811,7 +2816,7 @@ int cmd_apply(int argc, const char **argv, const char *unused_prefix)
 			apply = apply_with_reject = apply_verbosely = 1;
 			continue;
 		}
-		if (!strcmp(arg, "--verbose")) {
+		if (!strcmp(arg, "-v") || !strcmp(arg, "--verbose")) {
 			apply_verbosely = 1;
 			continue;
 		}
-- 
1.5.0.51.ge5582-dirty

Re: git-am failed, what's next ?

[Junio C Hamano]

"Francis Moreau" <francis.moro@gmail.com> writes:

>> Applying by hand
>> ----------------
>>
>> If you (as most people) are not comfortable editing the patch
>> text you can update your working tree to pretend that the patch
>> applied cleanly.  You can do number of things:
>>
>>  * Use "git apply --index -C<n> .dotest/patch" to attempt
>>    applying the patch with reduced context.
>>
>>  * Use "git apply --reject .dotest/patch" to get apply only
>>    hunks that apply cleanly, while getting *.rej files.
>>
>>  * Use "GNU patch" with less strict options, perhaps like
>>
>> 	$ patch -p1 --fuzz=<n> --ignore-whitespace
>
> why using 'patch' ? couldn't this be done by using 'git-apply' ?

The default operationg mode of 'patch' is less strict than
git-apply is, and it can be told to be much less strict, so some
people seem to like it.  You do not have to use it unless your
patch text is based on too far away version.  I usually don't.

Also the above is not an exhaustive list.  If you are handy with
other tools to make patches that do not cleanly apply apply
(e.g. wiggle), they can be used instead.

> Maybe these several choices could be proposed to the user when using
> the interactive mode ?

I do not think so.  As far as I remember, interactive mode
helps the "Fixing patch text" strategy but not "Applying by
hand" strategy.  When a patch does not apply, you fix the text
and re-apply, and [v]iew command is to review how you fixed the
patch, for example.

Re: git-am failed, what's next ?

[Junio C Hamano]

Andy Parkins <andyparkins@gmail.com> writes:

> On Thursday 2007 February 22 09:47, Junio C Hamano wrote:
>
>> If you have the pre-image blobs the patch was created against,
>> and the patch was created with git and records "index" lines
>> like these correctly:
>>
>>         diff --git a/builtin-config.c b/builtin-config.c
>>         index 0f9051d..f1433a4 100644
>>
>> then you can tell git to fall back on 3-way merge to apply the
>> patch.  After seeing "git am ./mbox" fail, you could try
>
> Magic.  I had no idea about this.  Could it be made even more
> magical by doing this for you?

You can start with --3way from the beginning, not just after
seeing it fail.

In the earlier description, I made it sound as if "-3" is an
option for recovery, but that is not the case.

Running "git am" without mbox parameter and an existing .dotest/
is the instruction for "git am" to continue, and the first thing
it does when told to continue is different depending on if
either --skip or --resolved is given.  --skip makes it skip the
patch.  --resolved takes the index, uses the metainfo to make a
commit.  Lack of these options makes it re-try the one recorded
in .dotest/next file.  After that, if it fails (and --skip would
not fail), it stops.  Otherwise it goes on to the next patch.

The example you referred to as "magic" is just a normal "retry it"
codepath but running with the "--3way" option.

The 3-way fallback is not enabled by default.  A minor reason
for this is that it did not exist in the original and is an
optional feature that was added later on.

But the real reason is that as a principle, git tools err on the
safe side by default, just like our use of "git-apply" in
"git-am" does not allow fuzz in the patch nor reducing context
by default for strictness.  That way, the patch application
process is stopped and gives the non-interactive user a chance
to inspect _why_ the patch does not apply, before deciding to
continue, either with fuzz'ed patch application, fixing the
patch text, or falling back on 3-way.

But these days, I almost always run:

	$ git am -3 -s ./mbox

and inspect the ones that actually used 3-way fallback after the
fact.  It _might_ make sense to add a configuration option to
default to -3 (or -s for that matter), like:

	[am]
		signoff
                threeway

and do something like this:

---
 git-am.sh |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/git-am.sh b/git-am.sh
index 6db9cb5..c0395f5 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -109,6 +109,7 @@ prec=4
 dotest=.dotest sign= utf8=t keep= skip= interactive= resolved= binary= resolvemsg=
 git_apply_opt=
 
+has_opt=
 while case "$#" in 0) break;; esac
 do
 	case "$1" in
@@ -155,8 +156,21 @@ do
 	*)
 	break ;;
 	esac
+	has_opt=t
 done
 
+if test -z "$has_opt"
+then
+	if test true = "$(git config --bool am.signoff)"
+	then
+		sign=t
+	fi
+	if test true = "$(git config --bool am.threeway)"
+	then
+		threeway=t
+	fi
+fi
+
 # If the dotest directory exists, but we have finished applying all the
 # patches in them, clear it out.
 if test -d "$dotest" &&