Bug? diff.submodule=log adds text to commit -v message

Bug? diff.submodule=log adds text to commit -v message

[Ari Pollak]

Hi,

I'm using git 1.8.4.2, and I've set the "diff.submodule = log" option 
globally. If I change the revision that a submodule is set to, then run
"git commit -av", The submodule shortlog is appended to the log message without 
any #s before it, so the log messages get included in my own log message. 
This seems like a bug and not a feature, as diffs aren't normally included in 
the commit message with -v.

Cheers,
Ari

Re: Bug? diff.submodule=log adds text to commit -v message

[Jens Lehmann]

Hi Ari,

Am 10.11.2013 22:49, schrieb Ari Pollak:
> I'm using git 1.8.4.2, and I've set the "diff.submodule = log" option 
> globally. If I change the revision that a submodule is set to, then run
> "git commit -av", The submodule shortlog is appended to the log message without 
> any #s before it, so the log messages get included in my own log message. 
> This seems like a bug and not a feature, as diffs aren't normally included in 
> the commit message with -v.

Thanks for your report, I can reproduce that here. But first I think
this is unrelated to the "diff.submodule = log" setting, as without it
you'll just see the submodule commit hash diff instead of the shortlog
(which is perfectly consistent with what I'd expect from this setting).
And secondly what you describe looks like documented behavior, the man
page of "git commit" states:

 -v, --verbose
     Show unified diff between the HEAD commit and what would be
     committed at the bottom of the commit message template. Note that
     this diff output doesn't have its lines prefixed with #.

And after adding a modified file the log message also shows the diff of
that file (and without leading "# "s too), so I doubt that diffs aren't
normally included in the commit message with -v. What am I missing?


Thanks
Jens

Re: Bug? diff.submodule=log adds text to commit -v message

[Ari Pollak]

Jens Lehmann writes:
> And after adding a modified file the log message also shows the diff of
> that file (and without leading "# "s too), so I doubt that diffs aren't
> normally included in the commit message with -v. What am I missing?

Ah, it is true that -v normally does not prefix the diffs with #, but there 
must be some filter in place after I save & quit my editor when a normal file 
diff is present, since that does not get included in the final commit 
message. But the submodule log does get included, which does not seem 
intentional.

Cheers,
Ari

Re: Bug? diff.submodule=log adds text to commit -v message

[Jens Lehmann]

Am 11.11.2013 21:48, schrieb Ari Pollak:
> Jens Lehmann writes:
>> And after adding a modified file the log message also shows the diff of
>> that file (and without leading "# "s too), so I doubt that diffs aren't
>> normally included in the commit message with -v. What am I missing?
> 
> Ah, it is true that -v normally does not prefix the diffs with #, but there 
> must be some filter in place after I save & quit my editor when a normal file 
> diff is present, since that does not get included in the final commit 
> message. But the submodule log does get included, which does not seem 
> intentional.

Ok, now this makes sense. "git commit" strips off the diff added by
-v by skipping everything starting with "\ndiff --git ". But that
logic fails when the "diff.submodule = log" setting adds a shortlog
instead of a regular diff, as that starts with "\nSubmodule ".

The diff below fixes the problem you describe for me. (But I do not
consider it a worthwhile fix in its current form because a line
starting with "Submodule " might appear in a perfectly normal commit
message, while "diff --git " most probably won't).

And while testing this issue I noticed another problem: When using
"git commit -a" not only the staged commits of a submodule get
committed, but also the unstaged commits. Will look into that too.

-------------------------8<---------------------
diff --git a/builtin/commit.c b/builtin/commit.c
index 6ab4605..ff6e171 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1600,11 +1600,14 @@ int cmd_commit(int argc, const char **argv, const char *
                die(_("could not read commit message: %s"), strerror(saved_errno
        }

-       /* Truncate the message just before the diff, if any. */
+       /* Truncate the message just before the diff or submodule shortlog */
        if (verbose) {
                p = strstr(sb.buf, "\ndiff --git ");
                if (p != NULL)
                        strbuf_setlen(&sb, p - sb.buf + 1);
+               p = strstr(sb.buf, "\nSubmodule ");
+               if (p != NULL)
+                       strbuf_setlen(&sb, p - sb.buf + 1);
        }

        if (cleanup_mode != CLEANUP_NONE)

Re: Bug? diff.submodule=log adds text to commit -v message

[Jens Lehmann]

Am 11.11.2013 22:29, schrieb Jens Lehmann:
> And while testing this issue I noticed another problem: When using
> "git commit -a" not only the staged commits of a submodule get
> committed, but also the unstaged commits. Will look into that too.

Ok, scrap that. This is exactly what is expected.

Re: Bug? diff.submodule=log adds text to commit -v message

[Jeff King]

On Mon, Nov 11, 2013 at 10:29:25PM +0100, Jens Lehmann wrote:

> The diff below fixes the problem you describe for me. (But I do not
> consider it a worthwhile fix in its current form because a line
> starting with "Submodule " might appear in a perfectly normal commit
> message, while "diff --git " most probably won't).

Yeah, this fix makes me nervous for that reason. "commit -v" has always
been a little bit flaky in that respect, as it is simply guessing at
the beginning of the diff text it added earlier. In addition to false
negatives, it also has false positives, stripping out people's diffs
that they meant to include in the commit message.

The "right" way to fix this is to change the format to use some more
robust marker, like:

  # Everything below this line is a diff that will be removed.

I do not know offhand if anybody's commit-template generating or parsing
scripts would be broken, but I doubt the fallout would be that big. When
last we discussed this (AFAICT), we did not yet have 0b38227 (commit:
Fix stripping of patch in verbose mode., 2008-11-12), which meant that
it would affect _everybody_. Nowadays it would only affect users of
"-v", which is presumably a much smaller population.

-Peff

Re: Bug? diff.submodule=log adds text to commit -v message

[Johannes Sixt]

Am 11.11.2013 22:29, schrieb Jens Lehmann:
> The diff below fixes the problem you describe for me. (But I do not
> consider it a worthwhile fix in its current form because a line
> starting with "Submodule " might appear in a perfectly normal commit
> message, while "diff --git " most probably won't).

And on top of that, "Submodule " originates from a translatable string,
doesn't it?

-- Hannes

Re: Bug? diff.submodule=log adds text to commit -v message

[Jens Lehmann]

Am 12.11.2013 08:46, schrieb Johannes Sixt:
> Am 11.11.2013 22:29, schrieb Jens Lehmann:
>> The diff below fixes the problem you describe for me. (But I do not
>> consider it a worthwhile fix in its current form because a line
>> starting with "Submodule " might appear in a perfectly normal commit
>> message, while "diff --git " most probably won't).
> 
> And on top of that, "Submodule " originates from a translatable string,
> doesn't it?

This would also be true for the marker line that Peff proposed:

  # Everything below this line is a diff that will be removed.

But I suspect that would be ok if the marker would be both added
and searched for in its translated form. Or is it possible that
the locale changes between those two steps?

Re: Bug? diff.submodule=log adds text to commit -v message

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Am 12.11.2013 08:46, schrieb Johannes Sixt:
>> Am 11.11.2013 22:29, schrieb Jens Lehmann:
>>> The diff below fixes the problem you describe for me. (But I do not
>>> consider it a worthwhile fix in its current form because a line
>>> starting with "Submodule " might appear in a perfectly normal commit
>>> message, while "diff --git " most probably won't).
>> 
>> And on top of that, "Submodule " originates from a translatable string,
>> doesn't it?
>
> This would also be true for the marker line that Peff proposed:
>
>   # Everything below this line is a diff that will be removed.
>
> But I suspect that would be ok if the marker would be both added
> and searched for in its translated form. Or is it possible that
> the locale changes between those two steps?

If we were introducing a divider line for machine consumption, I do
not think it is wise to let that line even translated...

Re: Bug? diff.submodule=log adds text to commit -v message

[Jens Lehmann]

Am 12.11.2013 23:20, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> Am 12.11.2013 08:46, schrieb Johannes Sixt:
>>> Am 11.11.2013 22:29, schrieb Jens Lehmann:
>>>> The diff below fixes the problem you describe for me. (But I do not
>>>> consider it a worthwhile fix in its current form because a line
>>>> starting with "Submodule " might appear in a perfectly normal commit
>>>> message, while "diff --git " most probably won't).
>>>
>>> And on top of that, "Submodule " originates from a translatable string,
>>> doesn't it?
>>
>> This would also be true for the marker line that Peff proposed:
>>
>>   # Everything below this line is a diff that will be removed.
>>
>> But I suspect that would be ok if the marker would be both added
>> and searched for in its translated form. Or is it possible that
>> the locale changes between those two steps?
> 
> If we were introducing a divider line for machine consumption, I do
> not think it is wise to let that line even translated...

Ok, but then it won't mean much to readers who don't understand
English. I assume prefixing all diff lines with "# " is out of
the question because of backwards compatibility, so what about
using a descriptive text together with a scissor line? The former
can be be translated (and won't make it into the commit message
because it starts with a "#") while the latter serves as a robust
divider line:

# Everything below the following line is a diff that will be removed.
# --------------------------------8<--------------------------------

Re: Bug? diff.submodule=log adds text to commit -v message

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

>> If we were introducing a divider line for machine consumption, I do
>> not think it is wise to let that line even translated...
>
> Ok, but then it won't mean much to readers who don't understand
> English. I assume prefixing all diff lines with "# " is out of
> the question because of backwards compatibility, so what about
> using a descriptive text together with a scissor line? The former
> can be be translated (and won't make it into the commit message
> because it starts with a "#") while the latter serves as a robust
> divider line:
>
> # Everything below the following line is a diff that will be removed.
> # --------------------------------8<--------------------------------

Yeah, or swap them around if you are trying to protect the part
above the divider from getting contaminated by the noise.

[RFC PATCH] commit -v: strip diffs and submodule shortlogs from the commit message

[Jens Lehmann]

When using the '-v' option of "git commit" the diff added to the commit
message temporarily for editing is stripped off after the user exited the
editor by searching for "\ndiff --git " and truncating the commmit message
there if it is found. But this approach has two problems: when the commit
message itself contains a line starting with "diff --git" it will be
truncated there prematurely. And when the "diff.submodule" setting is set
to "log", the diff may start with "Submodule <hash1>..<hash2>", which will
be left in the commit message while it shouldn't.

Fix that by introducing a special scissor separator line starting with the
comment character '#' followed by a line describing what it is for. The
scissor line is used to reliably detect the start of the diff so it can be
chopped off from the commit message, no matter what the user enters there.
Turn a known test failure fixed by this change into a successful test and
add another one for a diff starting with a submodule log.

Reported-by: Ari Pollak <ari@debian.org>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---


Am 13.11.2013 21:04, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>>> If we were introducing a divider line for machine consumption, I do
>>> not think it is wise to let that line even translated...
>>
>> Ok, but then it won't mean much to readers who don't understand
>> English. I assume prefixing all diff lines with "# " is out of
>> the question because of backwards compatibility, so what about
>> using a descriptive text together with a scissor line? The former
>> can be be translated (and won't make it into the commit message
>> because it starts with a "#") while the latter serves as a robust
>> divider line:
>>
>> # Everything below the following line is a diff that will be removed.
>> # --------------------------------8<--------------------------------
> 
> Yeah, or swap them around if you are trying to protect the part
> above the divider from getting contaminated by the noise.

Ok, did that. I couldn't find another user of the verbose setting,
so emitting the scissor line together with the description inside
wt_status_print_verbose() should be ok (or did I miss something
here?). But currently these lines use the hardcoded '#' instead of
the status.displaycommentprefix configuration and ignores coloring,
looks like I need to fix this in the next iteration.


 builtin/commit.c          |  6 +++---
 t/t7507-commit-verbose.sh | 15 ++++++++++++++-
 wt-status.c               |  4 ++++
 wt-status.h               |  2 ++
 4 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 6ab4605..091a6e7 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1602,9 +1602,9 @@ int cmd_commit(int argc, const char **argv, const char *prefix)

 	/* Truncate the message just before the diff, if any. */
 	if (verbose) {
-		p = strstr(sb.buf, "\ndiff --git ");
-		if (p != NULL)
-			strbuf_setlen(&sb, p - sb.buf + 1);
+		p = strstr(sb.buf, wt_status_diff_divider);
+		if ((p != NULL) && (p > sb.buf) && (p[-1] == '\n'))
+			strbuf_setlen(&sb, p - sb.buf);
 	}

 	if (cleanup_mode != CLEANUP_NONE)
diff --git a/t/t7507-commit-verbose.sh b/t/t7507-commit-verbose.sh
index da5bd3b..09c1150 100755
--- a/t/t7507-commit-verbose.sh
+++ b/t/t7507-commit-verbose.sh
@@ -65,9 +65,22 @@ test_expect_success 'diff in message is retained without -v' '
 	check_message diff
 '

-test_expect_failure 'diff in message is retained with -v' '
+test_expect_success 'diff in message is retained with -v' '
 	git commit --amend -F diff -v &&
 	check_message diff
 '

+test_expect_success 'submodule log is stripped out too with -v' '
+	git config diff.submodule log &&
+	git submodule add ./. sub &&
+	git commit -m "sub added" &&
+	(
+		cd sub &&
+		echo "more" >>file &&
+		git commit -a -m "submodule commit"
+	) &&
+	GIT_EDITOR=cat test_must_fail git commit -a -v 2>err &&
+	test_i18ngrep "Aborting commit due to empty commit message." err
+'
+
 test_done
diff --git a/wt-status.c b/wt-status.c
index b4e44ba..a3f7115 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -16,6 +16,8 @@
 #include "column.h"
 #include "strbuf.h"

+const char wt_status_diff_divider[] = "# -----------------------------------8<-----------------------------------\n";
+
 static char default_wt_status_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_NORMAL, /* WT_STATUS_HEADER */
 	GIT_COLOR_GREEN,  /* WT_STATUS_UPDATED */
@@ -791,6 +793,8 @@ static void wt_status_print_verbose(struct wt_status *s)
 	 */
 	if (s->fp != stdout)
 		rev.diffopt.use_color = 0;
+	fprintf(s->fp, wt_status_diff_divider);
+	fprintf(s->fp, _("# The diff below will be removed when keeping the previous line.\n"));
 	run_diff_index(&rev, 1);
 }

diff --git a/wt-status.h b/wt-status.h
index 6c29e6f..cd2709f 100644
--- a/wt-status.h
+++ b/wt-status.h
@@ -91,6 +91,8 @@ struct wt_status_state {
 	unsigned char cherry_pick_head_sha1[20];
 };

+const char wt_status_diff_divider[];
+
 void wt_status_prepare(struct wt_status *s);
 void wt_status_print(struct wt_status *s);
 void wt_status_collect(struct wt_status *s);
-- 
1.8.5.rc2.1.g537c6ea

Re: [RFC PATCH] commit -v: strip diffs and submodule shortlogs from the commit message

[Eric Sunshine]

On Sat, Nov 16, 2013 at 5:52 PM, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> When using the '-v' option of "git commit" the diff added to the commit
> message temporarily for editing is stripped off after the user exited the
> editor by searching for "\ndiff --git " and truncating the commmit message
> there if it is found. But this approach has two problems: when the commit
> message itself contains a line starting with "diff --git" it will be
> truncated there prematurely. And when the "diff.submodule" setting is set
> to "log", the diff may start with "Submodule <hash1>..<hash2>", which will
> be left in the commit message while it shouldn't.
>
> Fix that by introducing a special scissor separator line starting with the
> comment character '#' followed by a line describing what it is for. The
> scissor line is used to reliably detect the start of the diff so it can be
> chopped off from the commit message, no matter what the user enters there.
> Turn a known test failure fixed by this change into a successful test and
> add another one for a diff starting with a submodule log.
>
> Reported-by: Ari Pollak <ari@debian.org>
> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
> ---
>
>  builtin/commit.c          |  6 +++---
>  t/t7507-commit-verbose.sh | 15 ++++++++++++++-
>  wt-status.c               |  4 ++++
>  wt-status.h               |  2 ++
>  4 files changed, 23 insertions(+), 4 deletions(-)
>
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 6ab4605..091a6e7 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1602,9 +1602,9 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
>
>         /* Truncate the message just before the diff, if any. */
>         if (verbose) {
> -               p = strstr(sb.buf, "\ndiff --git ");
> -               if (p != NULL)
> -                       strbuf_setlen(&sb, p - sb.buf + 1);
> +               p = strstr(sb.buf, wt_status_diff_divider);

Would it make sense to use the more flexible is_scissors_line() from
builtin/mailinfo.c here?

> +               if ((p != NULL) && (p > sb.buf) && (p[-1] == '\n'))
> +                       strbuf_setlen(&sb, p - sb.buf);
>         }
>
>         if (cleanup_mode != CLEANUP_NONE)
> diff --git a/wt-status.c b/wt-status.c
> index b4e44ba..a3f7115 100644
> --- a/wt-status.c
> +++ b/wt-status.c
> @@ -16,6 +16,8 @@
>  #include "column.h"
>  #include "strbuf.h"
>
> +const char wt_status_diff_divider[] = "# -----------------------------------8<-----------------------------------\n";
> +
>  static char default_wt_status_colors[][COLOR_MAXLEN] = {
>         GIT_COLOR_NORMAL, /* WT_STATUS_HEADER */
>         GIT_COLOR_GREEN,  /* WT_STATUS_UPDATED */
> @@ -791,6 +793,8 @@ static void wt_status_print_verbose(struct wt_status *s)
>          */
>         if (s->fp != stdout)
>                 rev.diffopt.use_color = 0;
> +       fprintf(s->fp, wt_status_diff_divider);
> +       fprintf(s->fp, _("# The diff below will be removed when keeping the previous line.\n"));
>         run_diff_index(&rev, 1);
>  }
>
> diff --git a/wt-status.h b/wt-status.h
> index 6c29e6f..cd2709f 100644
> --- a/wt-status.h
> +++ b/wt-status.h
> @@ -91,6 +91,8 @@ struct wt_status_state {
>         unsigned char cherry_pick_head_sha1[20];
>  };
>
> +const char wt_status_diff_divider[];
> +
>  void wt_status_prepare(struct wt_status *s);
>  void wt_status_print(struct wt_status *s);
>  void wt_status_collect(struct wt_status *s);
> --

Re: [RFC PATCH] commit -v: strip diffs and submodule shortlogs from the commit message

[Jeff King]

On Sat, Nov 16, 2013 at 07:22:29PM -0500, Eric Sunshine wrote:

> >         /* Truncate the message just before the diff, if any. */
> >         if (verbose) {
> > -               p = strstr(sb.buf, "\ndiff --git ");
> > -               if (p != NULL)
> > -                       strbuf_setlen(&sb, p - sb.buf + 1);
> > +               p = strstr(sb.buf, wt_status_diff_divider);
> 
> Would it make sense to use the more flexible is_scissors_line() from
> builtin/mailinfo.c here?

I don't think so. We are not trying to be friendly to a remote source
which has given us an arbitrarily-written scissor line. Rather the
opposite: we are trying to be very strict only to break on the line we
have included ourselves.

-Peff

Re: [RFC PATCH] commit -v: strip diffs and submodule shortlogs from the commit message

[Jeff King]

> diff --git a/builtin/commit.c b/builtin/commit.c
> index 6ab4605..091a6e7 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1602,9 +1602,9 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
> 
>  	/* Truncate the message just before the diff, if any. */
>  	if (verbose) {
> -		p = strstr(sb.buf, "\ndiff --git ");
> -		if (p != NULL)
> -			strbuf_setlen(&sb, p - sb.buf + 1);
> +		p = strstr(sb.buf, wt_status_diff_divider);
> +		if ((p != NULL) && (p > sb.buf) && (p[-1] == '\n'))
> +			strbuf_setlen(&sb, p - sb.buf);

I think your check for a preceding newline is too strict. If I delete
everything before the scissor line (e.g., because I am trying to abort
the commit), we should still remove the diff. With your patch, we do
not, and a commit message consisting solely of the diff.

So I think you want:

  if (p && (p == sb.buf || p[-1] == '\n'))

> +	fprintf(s->fp, _("# The diff below will be removed when keeping the previous line.\n"));

I found this hard to parse, I think because of the "keeping" (why would
I not keep it?), and because you are talking about lines above and
below. It is not as accurate to say:

  # ------------------ >8 --------------------
  # Everything below this line will be removed.

because it is technically the line above that is the cutoff. But I think
the meaning is clear, and it is simpler to parse.

I do think it would be simpler with a single line. I know handling the
i18n was a question there, but I think we should be fine as long as we
check for the exact bytes we wrote. Surely gettext can do something
like:

  magic = _("# Everything below this line will be removed");
  fprintf(fh, "%s", magic);
  ...
  p = strstr(magic);

I don't know what guarantees on string lifetime gettext gives us, but
the worst case is that we simply strdup the result.

I suppose it's possible that the translated string could have utf8 with
multiple representations, and the user's editor normalizes the text in a
different way than we wrote it when it saves the result. I don't know if
that is worth caring about or not; it seems kind of insane.

-Peff

Re: [RFC PATCH] commit -v: strip diffs and submodule shortlogs from the commit message

[Jens Lehmann]

Am 17.11.2013 10:09, schrieb Jeff King:
> 
>> diff --git a/builtin/commit.c b/builtin/commit.c
>> index 6ab4605..091a6e7 100644
>> --- a/builtin/commit.c
>> +++ b/builtin/commit.c
>> @@ -1602,9 +1602,9 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
>>
>>  	/* Truncate the message just before the diff, if any. */
>>  	if (verbose) {
>> -		p = strstr(sb.buf, "\ndiff --git ");
>> -		if (p != NULL)
>> -			strbuf_setlen(&sb, p - sb.buf + 1);
>> +		p = strstr(sb.buf, wt_status_diff_divider);
>> +		if ((p != NULL) && (p > sb.buf) && (p[-1] == '\n'))
>> +			strbuf_setlen(&sb, p - sb.buf);
> 
> I think your check for a preceding newline is too strict. If I delete
> everything before the scissor line (e.g., because I am trying to abort
> the commit), we should still remove the diff. With your patch, we do
> not, and a commit message consisting solely of the diff.
> 
> So I think you want:
> 
>   if (p && (p == sb.buf || p[-1] == '\n'))

Thanks for catching this, will do so in v2.

>> +	fprintf(s->fp, _("# The diff below will be removed when keeping the previous line.\n"));
> 
> I found this hard to parse, I think because of the "keeping" (why would
> I not keep it?), and because you are talking about lines above and
> below. It is not as accurate to say:
> 
>   # ------------------ >8 --------------------
>   # Everything below this line will be removed.
> 
> because it is technically the line above that is the cutoff. But I think
> the meaning is clear, and it is simpler to parse.

Ok.

> I do think it would be simpler with a single line. I know handling the
> i18n was a question there, but I think we should be fine as long as we
> check for the exact bytes we wrote. Surely gettext can do something
> like:
> 
>   magic = _("# Everything below this line will be removed");
>   fprintf(fh, "%s", magic);
>   ...
>   p = strstr(magic);
> 
> I don't know what guarantees on string lifetime gettext gives us, but
> the worst case is that we simply strdup the result.
> 
> I suppose it's possible that the translated string could have utf8 with
> multiple representations, and the user's editor normalizes the text in a
> different way than we wrote it when it saves the result. I don't know if
> that is worth caring about or not; it seems kind of insane.

I don't have any strong feelings about this one. I'd be fine with
dropping the scissor line and taking the translated string as divider
line. What do others think?

Re: [RFC PATCH] commit -v: strip diffs and submodule shortlogs from the commit message

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I found this hard to parse, I think because of the "keeping" (why would
> I not keep it?), and because you are talking about lines above and
> below. It is not as accurate to say:
>
>   # ------------------ >8 --------------------
>   # Everything below this line will be removed.
>
> because it is technically the line above that is the cutoff. But I think
> the meaning is clear, and it is simpler to parse.
>
> I do think it would be simpler with a single line. I know handling the
> i18n was a question there, but I think we should be fine as long as we
> check for the exact bytes we wrote. Surely gettext can do something
> like:
>
>   magic = _("# Everything below this line will be removed");
>   fprintf(fh, "%s", magic);
>   ...
>   p = strstr(magic);
>
> I don't know what guarantees on string lifetime gettext gives us, but
> the worst case is that we simply strdup the result.
>
> I suppose it's possible that the translated string could have utf8 with
> multiple representations, and the user's editor normalizes the text in a
> different way than we wrote it when it saves the result. I don't know if
> that is worth caring about or not; it seems kind of insane.

I agree with your rewording suggestion.  It might make it even more
robust to do something like

    const char cut_here[] = "# --- cut here --- >8 --- cut here ---";

    fprintf(fh, "%s\n", cut_here);
    fputs(_("# Everything below this line will be removed\n"), fh);
    ...
    p = strstr(cut_here);

i.e. a real marker line that will never be translated, with an
explanation immediately below that can be translated.

[PATCH v2 ] commit -v: strip diffs and submodule shortlogs from the commit message

[Jens Lehmann]

When using the '-v' option of "git commit" the diff added to the commit
message temporarily for editing is stripped off after the user exited the
editor by searching for "\ndiff --git " and truncating the commmit message
there if it is found. But this approach has two problems: when the commit
message itself contains a line starting with "diff --git" it will be
truncated there prematurely. And when the "diff.submodule" setting is set
to "log", the diff may start with "Submodule <hash1>..<hash2>", which will
be left in the commit message while it shouldn't.

Fix that by introducing a special scissor separator line starting with the
comment character '#' followed by a line describing what it is for. The
scissor line - which will not be translated - is used to reliably detect
the start of the diff so it can be chopped off from the commit message, no
matter what the user enters there. Turn a known test failure fixed by this
change into a successful test and add another one for a diff starting with
a submodule log.

Reported-by: Ari Pollak <ari@debian.org>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---

Am 18.11.2013 17:01, schrieb Junio C Hamano:
> Jeff King <peff@peff.net> writes:
> 
>> I found this hard to parse, I think because of the "keeping" (why would
>> I not keep it?), and because you are talking about lines above and
>> below. It is not as accurate to say:
>>
>>   # ------------------ >8 --------------------
>>   # Everything below this line will be removed.
>>
>> because it is technically the line above that is the cutoff. But I think
>> the meaning is clear, and it is simpler to parse.

Ok, changed in this version.

> I agree with your rewording suggestion.  It might make it even more
> robust to do something like
> 
>     const char cut_here[] = "# --- cut here --- >8 --- cut here ---";
> 
>     fprintf(fh, "%s\n", cut_here);
>     fputs(_("# Everything below this line will be removed\n"), fh);
>     ...
>     p = strstr(cut_here);
> 
> i.e. a real marker line that will never be translated, with an
> explanation immediately below that can be translated.

Which is what my last version already did :-)  But it didn't mention
that in the commit message, which it does now).

Also the too strict check for the scissor line is fixed according to
Peff's proposal, it will now handle an empty commit message correctly
too.


 builtin/commit.c          |  6 +++---
 t/t7507-commit-verbose.sh | 15 ++++++++++++++-
 wt-status.c               |  4 ++++
 wt-status.h               |  2 ++
 4 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 6ab4605..b6d6655 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1602,9 +1602,9 @@ int cmd_commit(int argc, const char **argv, const char *prefix)

 	/* Truncate the message just before the diff, if any. */
 	if (verbose) {
-		p = strstr(sb.buf, "\ndiff --git ");
-		if (p != NULL)
-			strbuf_setlen(&sb, p - sb.buf + 1);
+		p = strstr(sb.buf, wt_status_diff_divider);
+		if (p && (p == sb.buf || p[-1] == '\n'))
+			strbuf_setlen(&sb, p - sb.buf);
 	}

 	if (cleanup_mode != CLEANUP_NONE)
diff --git a/t/t7507-commit-verbose.sh b/t/t7507-commit-verbose.sh
index da5bd3b..09c1150 100755
--- a/t/t7507-commit-verbose.sh
+++ b/t/t7507-commit-verbose.sh
@@ -65,9 +65,22 @@ test_expect_success 'diff in message is retained without -v' '
 	check_message diff
 '

-test_expect_failure 'diff in message is retained with -v' '
+test_expect_success 'diff in message is retained with -v' '
 	git commit --amend -F diff -v &&
 	check_message diff
 '

+test_expect_success 'submodule log is stripped out too with -v' '
+	git config diff.submodule log &&
+	git submodule add ./. sub &&
+	git commit -m "sub added" &&
+	(
+		cd sub &&
+		echo "more" >>file &&
+		git commit -a -m "submodule commit"
+	) &&
+	GIT_EDITOR=cat test_must_fail git commit -a -v 2>err &&
+	test_i18ngrep "Aborting commit due to empty commit message." err
+'
+
 test_done
diff --git a/wt-status.c b/wt-status.c
index b4e44ba..a499bd2 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -16,6 +16,8 @@
 #include "column.h"
 #include "strbuf.h"

+const char wt_status_diff_divider[] = "# -----------------------------------8<-----------------------------------\n";
+
 static char default_wt_status_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_NORMAL, /* WT_STATUS_HEADER */
 	GIT_COLOR_GREEN,  /* WT_STATUS_UPDATED */
@@ -791,6 +793,8 @@ static void wt_status_print_verbose(struct wt_status *s)
 	 */
 	if (s->fp != stdout)
 		rev.diffopt.use_color = 0;
+	fprintf(s->fp, wt_status_diff_divider);
+	fprintf(s->fp, _("# Everything below this line will be removed.\n"));
 	run_diff_index(&rev, 1);
 }

diff --git a/wt-status.h b/wt-status.h
index 6c29e6f..cd2709f 100644
--- a/wt-status.h
+++ b/wt-status.h
@@ -91,6 +91,8 @@ struct wt_status_state {
 	unsigned char cherry_pick_head_sha1[20];
 };

+const char wt_status_diff_divider[];
+
 void wt_status_prepare(struct wt_status *s);
 void wt_status_print(struct wt_status *s);
 void wt_status_collect(struct wt_status *s);
-- 
1.8.5.rc2.7.gb1d0fd2

Re: [PATCH v2 ] commit -v: strip diffs and submodule shortlogs from the commit message

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> When using the '-v' option of "git commit" the diff added to the commit
> message temporarily for editing is stripped off after the user exited the
> editor by searching for "\ndiff --git " and truncating the commmit message
> there if it is found. But this approach has two problems: when the commit
> message itself contains a line starting with "diff --git" it will be
> truncated there prematurely. And when the "diff.submodule" setting is set
> to "log", the diff may start with "Submodule <hash1>..<hash2>", which will
> be left in the commit message while it shouldn't.
>
> Fix that by introducing a special scissor separator line starting with the
> comment character '#' followed by a line describing what it is for. The
> scissor line - which will not be translated - is used to reliably detect
> the start of the diff so it can be chopped off from the commit message, no
> matter what the user enters there. Turn a known test failure fixed by this
> change into a successful test and add another one for a diff starting with
> a submodule log.
>
> Reported-by: Ari Pollak <ari@debian.org>
> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
> ---
>
> Am 18.11.2013 17:01, schrieb Junio C Hamano:
>> Jeff King <peff@peff.net> writes:
>> 
>>> I found this hard to parse, I think because of the "keeping" (why would
>>> I not keep it?), and because you are talking about lines above and
>>> below. It is not as accurate to say:
>>>
>>>   # ------------------ >8 --------------------
>>>   # Everything below this line will be removed.
>>>
>>> because it is technically the line above that is the cutoff. But I think
>>> the meaning is clear, and it is simpler to parse.
>
> Ok, changed in this version.
>
>> I agree with your rewording suggestion.  It might make it even more
>> robust to do something like
>> 
>>     const char cut_here[] = "# --- cut here --- >8 --- cut here ---";
>> 
>>     fprintf(fh, "%s\n", cut_here);
>>     fputs(_("# Everything below this line will be removed\n"), fh);
>>     ...
>>     p = strstr(cut_here);
>> 
>> i.e. a real marker line that will never be translated, with an
>> explanation immediately below that can be translated.
>
> Which is what my last version already did :-)  But it didn't mention
> that in the commit message, which it does now).
>
> Also the too strict check for the scissor line is fixed according to
> Peff's proposal, it will now handle an empty commit message correctly
> too.

Thanks.

I have a feeling that the translatable message has to say two
things, though.

 - Any junk below that cut marker will be removed;

 - Do not touch the cut marker, or you will mess up the commit log
   message.

Nobody seems to have brought up the latter in the discussion, but I
have a feeling that the users can do almost anything to break us and
it is better to be defensive for the users.

Perhaps phrase it something like this (the message is to be
translated, but not the marker)?

	# ------------------------ >8 ------------------------
        # Do not touch the cut-marker line above; everything below
        # will be removed.

>
>  builtin/commit.c          |  6 +++---
>  t/t7507-commit-verbose.sh | 15 ++++++++++++++-
>  wt-status.c               |  4 ++++
>  wt-status.h               |  2 ++
>  4 files changed, 23 insertions(+), 4 deletions(-)
>
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 6ab4605..b6d6655 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1602,9 +1602,9 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
>
>  	/* Truncate the message just before the diff, if any. */
>  	if (verbose) {
> -		p = strstr(sb.buf, "\ndiff --git ");
> -		if (p != NULL)
> -			strbuf_setlen(&sb, p - sb.buf + 1);
> +		p = strstr(sb.buf, wt_status_diff_divider);
> +		if (p && (p == sb.buf || p[-1] == '\n'))
> +			strbuf_setlen(&sb, p - sb.buf);
>  	}
>
>  	if (cleanup_mode != CLEANUP_NONE)
> diff --git a/t/t7507-commit-verbose.sh b/t/t7507-commit-verbose.sh
> index da5bd3b..09c1150 100755
> --- a/t/t7507-commit-verbose.sh
> +++ b/t/t7507-commit-verbose.sh
> @@ -65,9 +65,22 @@ test_expect_success 'diff in message is retained without -v' '
>  	check_message diff
>  '
>
> -test_expect_failure 'diff in message is retained with -v' '
> +test_expect_success 'diff in message is retained with -v' '
>  	git commit --amend -F diff -v &&
>  	check_message diff
>  '
>
> +test_expect_success 'submodule log is stripped out too with -v' '
> +	git config diff.submodule log &&
> +	git submodule add ./. sub &&
> +	git commit -m "sub added" &&
> +	(
> +		cd sub &&
> +		echo "more" >>file &&
> +		git commit -a -m "submodule commit"
> +	) &&
> +	GIT_EDITOR=cat test_must_fail git commit -a -v 2>err &&
> +	test_i18ngrep "Aborting commit due to empty commit message." err
> +'
> +
>  test_done
> diff --git a/wt-status.c b/wt-status.c
> index b4e44ba..a499bd2 100644
> --- a/wt-status.c
> +++ b/wt-status.c
> @@ -16,6 +16,8 @@
>  #include "column.h"
>  #include "strbuf.h"
>
> +const char wt_status_diff_divider[] = "# -----------------------------------8<-----------------------------------\n";
> +
>  static char default_wt_status_colors[][COLOR_MAXLEN] = {
>  	GIT_COLOR_NORMAL, /* WT_STATUS_HEADER */
>  	GIT_COLOR_GREEN,  /* WT_STATUS_UPDATED */
> @@ -791,6 +793,8 @@ static void wt_status_print_verbose(struct wt_status *s)
>  	 */
>  	if (s->fp != stdout)
>  		rev.diffopt.use_color = 0;
> +	fprintf(s->fp, wt_status_diff_divider);
> +	fprintf(s->fp, _("# Everything below this line will be removed.\n"));
>  	run_diff_index(&rev, 1);
>  }
>
> diff --git a/wt-status.h b/wt-status.h
> index 6c29e6f..cd2709f 100644
> --- a/wt-status.h
> +++ b/wt-status.h
> @@ -91,6 +91,8 @@ struct wt_status_state {
>  	unsigned char cherry_pick_head_sha1[20];
>  };
>
> +const char wt_status_diff_divider[];
> +
>  void wt_status_prepare(struct wt_status *s);
>  void wt_status_print(struct wt_status *s);
>  void wt_status_collect(struct wt_status *s);

Re: [PATCH v2 ] commit -v: strip diffs and submodule shortlogs from the commit message

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> diff --git a/wt-status.h b/wt-status.h
> index 6c29e6f..cd2709f 100644
> --- a/wt-status.h
> +++ b/wt-status.h
> @@ -91,6 +91,8 @@ struct wt_status_state {
>  	unsigned char cherry_pick_head_sha1[20];
>  };
>
> +const char wt_status_diff_divider[];

This gives me:

./wt-status.h:94:12: error: array 'wt_status_diff_divider' assumed to have one element [-Werror]
cc1: all warnings being treated as errors

which is a bit unfortunate.

Regardless of that, from the API design standpoint, I think it may
be much better not to expose this particular implementation element
(i.e. the array) to the caller, but instead to export a helper
function that takes a pointer to a piece of memory and let callers
ask an "I have this line---is it the status cut mark?" question.

Re: [PATCH v2 ] commit -v: strip diffs and submodule shortlogs from the commit message

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jens Lehmann <Jens.Lehmann@web.de> writes:
>
>> diff --git a/wt-status.h b/wt-status.h
>> index 6c29e6f..cd2709f 100644
>> --- a/wt-status.h
>> +++ b/wt-status.h
>> @@ -91,6 +91,8 @@ struct wt_status_state {
>>  	unsigned char cherry_pick_head_sha1[20];
>>  };
>>
>> +const char wt_status_diff_divider[];
>
> This gives me:
>
> ./wt-status.h:94:12: error: array 'wt_status_diff_divider' assumed to have one element [-Werror]
> cc1: all warnings being treated as errors
>
> which is a bit unfortunate.
>
> Regardless of that, from the API design standpoint, I think it may
> be much better not to expose this particular implementation element
> (i.e. the array) to the caller, but instead to export a helper
> function that takes a pointer to a piece of memory and let callers
> ask an "I have this line---is it the status cut mark?" question.

That is, something like this, perhaps.

 builtin/commit.c          |  9 +++------
 t/t7507-commit-verbose.sh | 15 ++++++++++++++-
 wt-status.c               | 15 +++++++++++++++
 wt-status.h               |  1 +
 4 files changed, 33 insertions(+), 7 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 6ab4605..fedb45a 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1505,7 +1505,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf author_ident = STRBUF_INIT;
 	const char *index_file, *reflog_msg;
-	char *nl, *p;
+	char *nl;
 	unsigned char sha1[20];
 	struct ref_lock *ref_lock;
 	struct commit_list *parents = NULL, **pptr = &parents;
@@ -1601,11 +1601,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	}
 
 	/* Truncate the message just before the diff, if any. */
-	if (verbose) {
-		p = strstr(sb.buf, "\ndiff --git ");
-		if (p != NULL)
-			strbuf_setlen(&sb, p - sb.buf + 1);
-	}
+	if (verbose)
+		wt_status_truncate_message_at_cut_line(&sb);
 
 	if (cleanup_mode != CLEANUP_NONE)
 		stripspace(&sb, cleanup_mode == CLEANUP_ALL);
diff --git a/t/t7507-commit-verbose.sh b/t/t7507-commit-verbose.sh
index da5bd3b..09c1150 100755
--- a/t/t7507-commit-verbose.sh
+++ b/t/t7507-commit-verbose.sh
@@ -65,9 +65,22 @@ test_expect_success 'diff in message is retained without -v' '
 	check_message diff
 '
 
-test_expect_failure 'diff in message is retained with -v' '
+test_expect_success 'diff in message is retained with -v' '
 	git commit --amend -F diff -v &&
 	check_message diff
 '
 
+test_expect_success 'submodule log is stripped out too with -v' '
+	git config diff.submodule log &&
+	git submodule add ./. sub &&
+	git commit -m "sub added" &&
+	(
+		cd sub &&
+		echo "more" >>file &&
+		git commit -a -m "submodule commit"
+	) &&
+	GIT_EDITOR=cat test_must_fail git commit -a -v 2>err &&
+	test_i18ngrep "Aborting commit due to empty commit message." err
+'
+
 test_done
diff --git a/wt-status.c b/wt-status.c
index b4e44ba..492506a 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -16,6 +16,9 @@
 #include "column.h"
 #include "strbuf.h"
 
+static const char wt_status_cut_line[] =
+"# ------------------------ >8 ------------------------\n";
+
 static char default_wt_status_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_NORMAL, /* WT_STATUS_HEADER */
 	GIT_COLOR_GREEN,  /* WT_STATUS_UPDATED */
@@ -767,6 +770,15 @@ conclude:
 	status_printf_ln(s, GIT_COLOR_NORMAL, "");
 }
 
+void wt_status_truncate_message_at_cut_line(struct strbuf *buf)
+{
+	const char *p;
+
+	p = strstr(buf->buf, wt_status_cut_line);
+	if (p && (p == buf->buf || p[-1] == '\n'))
+		strbuf_setlen(buf, p - buf->buf);
+}
+
 static void wt_status_print_verbose(struct wt_status *s)
 {
 	struct rev_info rev;
@@ -791,6 +803,9 @@ static void wt_status_print_verbose(struct wt_status *s)
 	 */
 	if (s->fp != stdout)
 		rev.diffopt.use_color = 0;
+	fprintf(s->fp, wt_status_cut_line);
+	fprintf(s->fp, _("# Do not touch the line above.\n"));
+	fprintf(s->fp, _("# Everything below will be removed.\n"));
 	run_diff_index(&rev, 1);
 }
 
diff --git a/wt-status.h b/wt-status.h
index 6c29e6f..30a4812 100644
--- a/wt-status.h
+++ b/wt-status.h
@@ -91,6 +91,7 @@ struct wt_status_state {
 	unsigned char cherry_pick_head_sha1[20];
 };
 
+void wt_status_truncate_message_at_cut_line(struct strbuf *);
 void wt_status_prepare(struct wt_status *s);
 void wt_status_print(struct wt_status *s);
 void wt_status_collect(struct wt_status *s);

Re: [PATCH v2 ] commit -v: strip diffs and submodule shortlogs from the commit message

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Am 18.11.2013 17:01, schrieb Junio C Hamano:
>> Jeff King <peff@peff.net> writes:
>> 
>>> I found this hard to parse, I think because of the "keeping" (why would
>>> I not keep it?), and because you are talking about lines above and
>>> below. It is not as accurate to say:
>>>
>>>   # ------------------ >8 --------------------
>>>   # Everything below this line will be removed.
>>>
>>> because it is technically the line above that is the cutoff. But I think
>>> the meaning is clear, and it is simpler to parse.
>
> Ok, changed in this version.
>
>> I agree with your rewording suggestion....
>
> Which is what my last version already did :-)  But it didn't mention
> that in the commit message, which it does now).

Oh, another thing. Does this interact with the core.commentChar in
any way, and if so how?

Re: [PATCH v2 ] commit -v: strip diffs and submodule shortlogs from the commit message

[Jens Lehmann]

Am 19.11.2013 21:34, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> Am 18.11.2013 17:01, schrieb Junio C Hamano:
>>> Jeff King <peff@peff.net> writes:
>>>
>>>> I found this hard to parse, I think because of the "keeping" (why would
>>>> I not keep it?), and because you are talking about lines above and
>>>> below. It is not as accurate to say:
>>>>
>>>>   # ------------------ >8 --------------------
>>>>   # Everything below this line will be removed.
>>>>
>>>> because it is technically the line above that is the cutoff. But I think
>>>> the meaning is clear, and it is simpler to parse.
>>
>> Ok, changed in this version.
>>
>>> I agree with your rewording suggestion....
>>
>> Which is what my last version already did :-)  But it didn't mention
>> that in the commit message, which it does now).
> 
> Oh, another thing. Does this interact with the core.commentChar in
> any way, and if so how?

If I understand that setting correctly the hardcoded '#' characters
must be replaced with the core.commentChar when set. Will tackle that
in the next iteration.