serious problem with `git format-patch' & `git am'

serious problem with `git format-patch' & `git am'

[Werner LEMBERG]

[both version 1.6.0.2 and git from 2009-01-24]

Folks,


I'm going to convert the FreeType CVS repository to git, using Keith
Packard's `parsecvs'.  It sometimes happens that there are git entries
titled

  *** empty log message ***

I want to massage the git repository before publishing it, replacing
those entries with something more meaningful.

The last time I tested this (using git 1.5.something, I no longer
remember the exact version) I did the following:

  . git format-patch <start>..HEAD
    git reset --hard <start>

  . Edit the `Subject:' field in the created *.patch files where
    necessary.

  . git am --whitespace=nowarn *.patch

Unfortunately, this no longer works as before, and since I can't find
a hint in the docs about this change I wonder whether it is a bug.

With git 1.5.something, if the first paragraph of the commit message
(as output by parsecvs) looks like this

   foo foo foo foo
   bar bar bar bar
   baz baz baz baz

it is emitted exactly as-is (after `git format-patch' & `git am');
gitk shows `foo foo foo foo' as the first line.  However, git version
1.6.0.2 and the current git both convert newlines to spaces in the
first paragraph!  I now get

  foo foo foo foo bar bar bar bar baz baz baz baz

as the beginning of the commit message which is VERY bad since
CVS-style entries often have a long first paragraph, causing
ugly-looking, overlong lines.

In case this is an intended change I ask you urgently to provide an
option to get back the old behaviour of `git format-patch' & `git am'.


    Werner

Re: serious problem with `git format-patch' & `git am'

[Johannes Schindelin]

Hi,

[welcome on the Git list, Werner!]

On Wed, 25 Feb 2009, Werner LEMBERG wrote:

> 
> [both version 1.6.0.2 and git from 2009-01-24]
> 
> Folks,
> 
> 
> I'm going to convert the FreeType CVS repository to git, using Keith
> Packard's `parsecvs'.  It sometimes happens that there are git entries
> titled
> 
>   *** empty log message ***
> 
> I want to massage the git repository before publishing it, replacing
> those entries with something more meaningful.
> 
> The last time I tested this (using git 1.5.something, I no longer
> remember the exact version) I did the following:
> 
>   . git format-patch <start>..HEAD
>     git reset --hard <start>
> 
>   . Edit the `Subject:' field in the created *.patch files where
>     necessary.
> 
>   . git am --whitespace=nowarn *.patch
> 
> Unfortunately, this no longer works as before, and since I can't find
> a hint in the docs about this change I wonder whether it is a bug.
> 
> With git 1.5.something, if the first paragraph of the commit message
> (as output by parsecvs) looks like this
> 
>    foo foo foo foo
>    bar bar bar bar
>    baz baz baz baz
> 
> it is emitted exactly as-is (after `git format-patch' & `git am');
> gitk shows `foo foo foo foo' as the first line.  However, git version
> 1.6.0.2 and the current git both convert newlines to spaces in the
> first paragraph!  I now get
> 
>   foo foo foo foo bar bar bar bar baz baz baz baz
> 
> as the beginning of the commit message which is VERY bad since
> CVS-style entries often have a long first paragraph, causing
> ugly-looking, overlong lines.
> 
> In case this is an intended change I ask you urgently to provide an
> option to get back the old behaviour of `git format-patch' & `git am'.

It is an intended change, for the _opposite_ case: some people thrive on 
overly-long subject lines, and the change was supposed to help them.

I was not sure if it would be a good change back then, but I did not have 
anything concrete I could present as an example where the behavior hurts.

In your case, I suggest using something different, though.  Either use 
"rebase -i" and mark the commits you want to massage as "edit", or use 
"fast-export > dump", edit "dump" by hand, and then reimport the whole 
thing using fast-import.

IMHO your use case is not a good candidate for format-patch, as that 
program was really meant for patch submission (even if it also serves 
other purposes pretty well).

Ciao,
Dscho

Re: serious problem with `git format-patch' & `git am'

[Brandon Casey]

Werner LEMBERG wrote:
> [both version 1.6.0.2 and git from 2009-01-24]
> 
> Folks,
> 
> 
> I'm going to convert the FreeType CVS repository to git, using Keith
> Packard's `parsecvs'.  It sometimes happens that there are git entries
> titled
> 
>   *** empty log message ***
> 
> I want to massage the git repository before publishing it, replacing
> those entries with something more meaningful.
> 
> The last time I tested this (using git 1.5.something, I no longer
> remember the exact version) I did the following:
> 
>   . git format-patch <start>..HEAD
>     git reset --hard <start>
> 
>   . Edit the `Subject:' field in the created *.patch files where
>     necessary.
> 
>   . git am --whitespace=nowarn *.patch
> 
> Unfortunately, this no longer works as before, and since I can't find
> a hint in the docs about this change I wonder whether it is a bug.
> 
> With git 1.5.something, if the first paragraph of the commit message
> (as output by parsecvs) looks like this
> 
>    foo foo foo foo
>    bar bar bar bar
>    baz baz baz baz
> 
> it is emitted exactly as-is (after `git format-patch' & `git am');
> gitk shows `foo foo foo foo' as the first line.  However, git version
> 1.6.0.2 and the current git both convert newlines to spaces in the
> first paragraph!  I now get
> 
>   foo foo foo foo bar bar bar bar baz baz baz baz
> 
> as the beginning of the commit message which is VERY bad since
> CVS-style entries often have a long first paragraph, causing
> ugly-looking, overlong lines.
> 
> In case this is an intended change I ask you urgently to provide an
> option to get back the old behaviour of `git format-patch' & `git am'.

I was not aware of this behavior, but it looks like the '-k' option for
both format-patch and git-am is what you want to use.  It will prevent
mailinfo from munging the commit message.

-brandon

Re: serious problem with `git format-patch' & `git am'

[Jakub Narebski]

Brandon Casey <casey@nrlssc.navy.mil> writes:
> Werner LEMBERG wrote:
> > [both version 1.6.0.2 and git from 2009-01-24]
> >
> > I'm going to convert the FreeType CVS repository to git, using Keith
> > Packard's `parsecvs'.

>From what I understand it is not maintained, is not incremental, and
requires access to *,v files.  Why not use more modern and actively
developed cvs2git tool from cvs2svn project[1]?

[1] http://cvs2svn.tigris.org
[2] http://git.or.cz/gitwiki/InterfacesFrontendsAndTools
    "Interaction with other Revision Control Systems" section

> >  It sometimes happens that there are git entries
> > titled
> > 
> >   *** empty log message ***
> > 
> > I want to massage the git repository before publishing it, replacing
> > those entries with something more meaningful.
> > 
> > The last time I tested this (using git 1.5.something, I no longer
> > remember the exact version) I did the following:
> > 
> >   . git format-patch <start>..HEAD
> >     git reset --hard <start>
> > 
> >   . Edit the `Subject:' field in the created *.patch files where
> >     necessary.
> > 
> >   . git am --whitespace=nowarn *.patch

You can use the following alternate solutions:

 * git rebase --interactive (perhaps with --root option if your git
   supports it, or cherry-pick root commit by hand if it does not),
   changing 'pick' to 'edit' for patches (commits) that you want to
   modify commit message.

 * if you can automate the changes, i.e. if you can script it, the
   best solution would be to use git-filter-branch instead; or use
   fast-export / fast-import pair with your filter in the middle.

> > 
> > Unfortunately, this no longer works as before, and since I can't find
> > a hint in the docs about this change I wonder whether it is a bug.
> > 
> > With git 1.5.something, if the first paragraph of the commit message
> > (as output by parsecvs) looks like this
> > 
> >    foo foo foo foo
> >    bar bar bar bar
> >    baz baz baz baz
> > 
> > it is emitted exactly as-is (after `git format-patch' & `git am');
> > gitk shows `foo foo foo foo' as the first line.  However, git version
> > 1.6.0.2 and the current git both convert newlines to spaces in the
> > first paragraph!  I now get
> > 
> >   foo foo foo foo bar bar bar bar baz baz baz baz
> > 
> > as the beginning of the commit message which is VERY bad since
> > CVS-style entries often have a long first paragraph, causing
> > ugly-looking, overlong lines.

This is caused by the fact that git-am assumes that subject line goes
as first line of commit message, with empty line separating it from
the rest of commit message which comes from the body of email.

It used to preserve line breaking in Subject: line, if it was broken
into lines using continuation style for email headers, but that was
wrong solution in the intended use case (applying patches from email,
patches conforming to git commit message conventions), as Subject:
line could have been broken into multiple lines by the _transport_
layer, and not intended to be broken.

> > 
> > In case this is an intended change I ask you urgently to provide an
> > option to get back the old behaviour of `git format-patch' & `git am'.
> 
> I was not aware of this behavior, but it looks like the '-k' option for
> both format-patch and git-am is what you want to use.  It will prevent
> mailinfo from munging the commit message.

I think you want undocumented but for the git-am usage (e.g. 'git am
-h') --rebasing option to git-am, which makes git-am preserve Subject:
line formatting, _including_ line breaking (I think that -k/--keep
does preserve subject, but does not preserve line breaking; I haven't
checked this, though).  Note that in this case you would have to make
git-format-patch to not output subject prefix ([PATCH]).

See how git-rebase.sh does it.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: serious problem with `git format-patch' & `git am'

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> I think you want undocumented but for the git-am usage (e.g. 'git am
> -h') --rebasing option to git-am, which makes git-am preserve Subject:
> line formatting,...

I have to warn that that one requires you to have proper commit object
to begin with.  IOW, the first line "From [0-9a-f]{40} date" must record
an existing object name.

As you said, "am" is for applying e-mailed patches and filter-branch would
be the best fit for fixing imported history, but whichever tool is used, I
think it is a mistake to try preserving a "one-line-per-paragraph" format
the original poster seems to have, i.e.:

   foo foo foo foo
   bar bar bar bar
   baz baz baz baz

Yes, the low-level could create a commit object with a message like that,
but all the Porcelains rely on "paragraphs are separated with a blank
line" format.  So no matter how the conversion is done, I think it would
be a good idea to turn the above into:

   foo foo foo foo

   bar bar bar bar
   baz baz baz baz

if the user wants to keep the resulting history any useful.

To do it with "format-patch | am", you would format it like:

	Subject: foo foo foo foo
        From: author name <author.name@domain>
        Date: author date

        bar bar bar bar
        baz baz baz baz

        

Re: serious problem with `git format-patch' & `git am'

[Werner LEMBERG]

> I was not aware of this behavior, but it looks like the '-k' option
> for both format-patch and git-am is what you want to use.  It will
> prevent mailinfo from munging the commit message.

Tried it already.  It doesn't help.


    Werner

Re: serious problem with `git format-patch' & `git am'

[Brandon Casey]

Junio,

I think you are the author of git-am.sh.

What does this line do?

git-am.sh line 375:

    case "$keep_subject" in -k)  SUBJECT="[PATCH] $SUBJECT" ;; esac

It appears to do nothing.  It has existed since the beginning of git-am.sh,
and there has never been another reference to $keep_subject in this script.
Is it possible that $keep_subject is supposed to be $keep?

-brandon

Re: serious problem with `git format-patch' & `git am'

[Junio C Hamano]

Brandon Casey <casey@nrlssc.navy.mil> writes:

> What does this line do?
>
> git-am.sh line 375:
>
>     case "$keep_subject" in -k)  SUBJECT="[PATCH] $SUBJECT" ;; esac
>
> It appears to do nothing.  It has existed since the beginning of git-am.sh,
> and there has never been another reference to $keep_subject in this script.
> Is it possible that $keep_subject is supposed to be $keep?

I do not think it is doing anything, and I do not think it makes sense to
"fix" it to allow adding "[PATCH] " in front either.  I'd vote for
removing the entire thing.

The $keep_subject variable and "add [PATCH] only to later strip" logic was
borrowed from git-applymbox and git-applypatch pair, see 6bff6a6 (Teach
applymbox to keep the Subject: line., 2005-08-16) especially for the
change that affects tools/git-applypatch in the commit.  I think there was
a reason that -k refrained from adding the prefix in git-applypatch but
the option to git-am needed to add the prefix which seems inconsistent but
I do not recall the details.  Maybe the list archive may know.