[PATCHv3 0/3] Using git-mailsplit in mixed line ending environment

[PATCHv3 0/3] Using git-mailsplit in mixed line ending environment

[Stefan-W. Hahn]

Hello,

I'm using git in environments with files having dos or unix line
ending. I apply patches using 'git format-patch ... | git am ...'.
A change in git-mailsplit in commit c2ca1d79 introduced a change in
the default behaviour of git-mailsplit when splitting mbox patches. It
makes dos line endings to unix line endings. With this behaviour it is
impossible to apply patches.

The following patches introduce the '--kepp-cr' parameter to git-am an
an additional possibility to set '--keep-cr' via configuration for
git-am. Also I added missing description for '--keep-cr' of
git-mailsplit.

Second round:
I changed 'mailsplit.keep-cr' to 'mailsplit.keepcr' as suggested by Jakub
and comment in the testcase.

Third round:
I moved configuration 'mailsplit.keepcr' to 'am.keepcr' because
git-mailsplit can be used outside a git repository (thx Junio).

Stefan

Stefan-W. Hahn (3):
      git-mailsplit: Show parameter '--keep-cr' in usage and documentation
      git-am: Add command line parameter `--keep-cr` passing it to git-mailsplit.
      Adding test for `--keep-cr` for git-am.

 Documentation/config.txt        |    6 +++
 Documentation/git-mailsplit.txt |    5 ++-
 builtin-mailsplit.c             |    2 +-
 git-am.sh                       |   27 +++++++++++----
 t/t4253-am-keep-cr-dos.sh       |   68 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 99 insertions(+), 9 deletions(-)

[PATCH 1/3] git-mailsplit: Show parameter '--keep-cr' in usage and documentation

[Stefan-W. Hahn]

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 Documentation/git-mailsplit.txt |    5 ++++-
 builtin-mailsplit.c             |    2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-mailsplit.txt b/Documentation/git-mailsplit.txt
index 5cc94ec..a634485 100644
--- a/Documentation/git-mailsplit.txt
+++ b/Documentation/git-mailsplit.txt
@@ -7,7 +7,7 @@ git-mailsplit - Simple UNIX mbox splitter program
 
 SYNOPSIS
 --------
-'git mailsplit' [-b] [-f<nn>] [-d<prec>] -o<directory> [--] [<mbox>|<Maildir>...]
+'git mailsplit' [-b] [-f<nn>] [-d<prec>] [--keep-cr] -o<directory> [--] [<mbox>|<Maildir>...]
 
 DESCRIPTION
 -----------
@@ -43,6 +43,9 @@ OPTIONS
 	Skip the first <nn> numbers, for example if -f3 is specified,
 	start the numbering with 0004.
 
+--keep-cr::
+	Do not remove `\r` from lines ending with `\r\n`.
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>
diff --git a/builtin-mailsplit.c b/builtin-mailsplit.c
index 207e358..cdfc1b7 100644
--- a/builtin-mailsplit.c
+++ b/builtin-mailsplit.c
@@ -10,7 +10,7 @@
 #include "strbuf.h"
 
 static const char git_mailsplit_usage[] =
-"git mailsplit [-d<prec>] [-f<n>] [-b] -o<directory> [<mbox>|<Maildir>...]";
+"git mailsplit [-d<prec>] [-f<n>] [-b] [--keep-cr] -o<directory> [<mbox>|<Maildir>...]";
 
 static int is_from_line(const char *line, int len)
 {
-- 
1.7.0.rc2.31.g49e2a

[PATCH 2/3] git-am: Add command line parameter `--keep-cr` passing it to git-mailsplit.

[Stefan-W. Hahn]

The behaviour of git-mailsplit, which is called from git-am for
patches in mbox format, has been changed in commit c2ca1d79. The new
default behaviour will remove `\r` from line endings with `\r\n`.

If applying patches with the following command sequence

   git format-patch --stdout ... | git am ...

in repositories having files with dos and unix line endings,
git-mailsplit must be called with `--keep-cr` parameter.

This patch adds the command line parameter `--keep-cr` for git-am and
the configuration `am.keepcr`.

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 Documentation/config.txt |    6 ++++++
 git-am.sh                |   27 ++++++++++++++++++++-------
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 4c36aa9..aa452f3 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -550,6 +550,12 @@ it will be treated as a shell command.  For example, defining
 executed from the top-level directory of a repository, which may
 not necessarily be the current directory.
 
+am.keepcr::
+	If true, git-am will call git-mailsplit for patches in mbox format 
+	with parameter '--keep-cr'. In this case git-mailsplit will
+	not remove `\r` from lines ending with `\r\n`. 
+	See linkgit:git-am[1], linkgit:git-mailsplit[1].
+
 apply.ignorewhitespace::
 	When set to 'change', tells 'git apply' to ignore changes in
 	whitespace, in the same way as the '--ignore-space-change'
diff --git a/git-am.sh b/git-am.sh
index c8b9cbb..3057a83 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -15,6 +15,7 @@ q,quiet         be quiet
 s,signoff       add a Signed-off-by line to the commit message
 u,utf8          recode into utf8 (default)
 k,keep          pass -k flag to git-mailinfo
+keep-cr         pass --keep-cr flag to git-mailsplit for mbox format
 c,scissors      strip everything before a scissors line
 whitespace=     pass it through git-apply
 ignore-space-change pass it through git-apply
@@ -216,12 +217,12 @@ check_patch_format () {
 split_patches () {
 	case "$patch_format" in
 	mbox)
-		case "$rebasing" in
-		'')
-			keep_cr= ;;
-		?*)
-			keep_cr=--keep-cr ;;
-		esac
+		if test -n "$rebasing$keepcr"
+		then
+                    keep_cr=--keep-cr
+		else
+                    keep_cr=
+		fi
 		git mailsplit -d"$prec" -o"$dotest" -b $keep_cr -- "$@" > "$dotest/last" ||
 		clean_abort
 		;;
@@ -290,13 +291,18 @@ split_patches () {
 
 prec=4
 dotest="$GIT_DIR/rebase-apply"
-sign= utf8=t keep= skip= interactive= resolved= rebasing= abort=
+sign= utf8=t keep= keepcr= skip= interactive= resolved= rebasing= abort=
 resolvemsg= resume= scissors= no_inbody_headers=
 git_apply_opt=
 committer_date_is_author_date=
 ignore_date=
 allow_rerere_autoupdate=
 
+if test "$(git config --bool --get am.keepcr)" = true
+then 
+    keepcr=t
+fi
+
 while test $# != 0
 do
 	case "$1" in
@@ -347,6 +353,8 @@ do
 		allow_rerere_autoupdate="$1" ;;
 	-q|--quiet)
 		GIT_QUIET=t ;;
+	--keep-cr)
+		keepcr=t ;;
 	--)
 		shift; break ;;
 	*)
@@ -452,6 +460,7 @@ else
 	echo "$sign" >"$dotest/sign"
 	echo "$utf8" >"$dotest/utf8"
 	echo "$keep" >"$dotest/keep"
+	echo "$keepcr" >"$dotest/keepcr"
 	echo "$scissors" >"$dotest/scissors"
 	echo "$no_inbody_headers" >"$dotest/no_inbody_headers"
 	echo "$GIT_QUIET" >"$dotest/quiet"
@@ -495,6 +504,10 @@ if test "$(cat "$dotest/keep")" = t
 then
 	keep=-k
 fi
+if test "$(cat "$dotest/keepcr")" = t
+then
+	keepcr=--keep-cr
+fi
 case "$(cat "$dotest/scissors")" in
 t)
 	scissors=--scissors ;;
-- 
1.7.0.rc2.31.g49e2a

[PATCH 3/3] Adding test for `--keep-cr` for git-am.

[Stefan-W. Hahn]

This test adds test for git-am dos line endings, the command sequence
'git format-patch ... | git am ...' and the configuration variable
`am.keepcr`.

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 t/t4253-am-keep-cr-dos.sh |   68 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 68 insertions(+), 0 deletions(-)
 create mode 100644 t/t4253-am-keep-cr-dos.sh

diff --git a/t/t4253-am-keep-cr-dos.sh b/t/t4253-am-keep-cr-dos.sh
new file mode 100644
index 0000000..a4f5f80
--- /dev/null
+++ b/t/t4253-am-keep-cr-dos.sh
@@ -0,0 +1,68 @@
+#!/bin/sh
+#
+# Copyright (c) 2010 Stefan-W. Hahn
+#
+
+test_description='git-am mbox with dos line ending.
+
+'
+. ./test-lib.sh
+
+# Three patches which will be added as files with dos line ending.
+
+cat > file1 <<\EOF
+line 1
+EOF
+
+cat > file2 <<\EOF
+line 1
+line 2
+EOF
+
+cat > file3 <<\EOF
+line 1
+line 2
+line 3
+EOF
+
+test_expect_success 'setup repository with dos files' '
+        append_cr <file1 >file
+        git add file &&
+        git commit -m Initial &&
+        git tag start &&
+        append_cr <file2 >file
+        git commit -a -m Second &&
+        git tag start2 &&
+        append_cr <file3 >file
+        git commit -a -m Third &&
+	git format-patch -k --stdout start.. > format-patch.diff
+'
+
+test_expect_success 'mailsplit format-patch of dos files' '
+        mkdir split &&
+        git mailsplit -osplit format-patch.diff &&
+        cat split/0001 split/0002 > mailsplit.diff &&
+        test_must_fail test_cmp format-patch.diff mailsplit.diff
+'
+
+test_expect_success 'mailsplit --keep-cr format-patch of dos files' '
+        mkdir split2 &&
+        git mailsplit --keep-cr -osplit2 format-patch.diff &&
+        cat split2/0001 split2/0002 > mailsplit2.diff &&
+        test_cmp format-patch.diff mailsplit2.diff
+'
+
+test_expect_success 'format-patch with dos files --keep-cr' '
+        git checkout -b new start &&
+	git format-patch -k --stdout start..master | git am --keep-cr -k -3 &&
+        git diff master
+'
+
+test_expect_success 'format-patch with dos files config.mailsplit' '
+        git config am.keepcr 1 &&
+        git checkout -b new3 start &&
+	git format-patch -k --stdout start..master | git am -k -3 &&
+        git diff master
+'
+
+test_done
-- 
1.7.0.rc2.31.g49e2a

[PATCH 1/3] git-mailsplit: Show parameter '--keep-cr' in usage and documentation

[Stefan-W. Hahn]

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 Documentation/git-mailsplit.txt |    5 ++++-
 builtin-mailsplit.c             |    2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-mailsplit.txt b/Documentation/git-mailsplit.txt
index 5cc94ec..a634485 100644
--- a/Documentation/git-mailsplit.txt
+++ b/Documentation/git-mailsplit.txt
@@ -7,7 +7,7 @@ git-mailsplit - Simple UNIX mbox splitter program
 
 SYNOPSIS
 --------
-'git mailsplit' [-b] [-f<nn>] [-d<prec>] -o<directory> [--] [<mbox>|<Maildir>...]
+'git mailsplit' [-b] [-f<nn>] [-d<prec>] [--keep-cr] -o<directory> [--] [<mbox>|<Maildir>...]
 
 DESCRIPTION
 -----------
@@ -43,6 +43,9 @@ OPTIONS
 	Skip the first <nn> numbers, for example if -f3 is specified,
 	start the numbering with 0004.
 
+--keep-cr::
+	Do not remove `\r` from lines ending with `\r\n`.
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>
diff --git a/builtin-mailsplit.c b/builtin-mailsplit.c
index 207e358..cdfc1b7 100644
--- a/builtin-mailsplit.c
+++ b/builtin-mailsplit.c
@@ -10,7 +10,7 @@
 #include "strbuf.h"
 
 static const char git_mailsplit_usage[] =
-"git mailsplit [-d<prec>] [-f<n>] [-b] -o<directory> [<mbox>|<Maildir>...]";
+"git mailsplit [-d<prec>] [-f<n>] [-b] [--keep-cr] -o<directory> [<mbox>|<Maildir>...]";
 
 static int is_from_line(const char *line, int len)
 {
-- 
1.7.0.rc2.31.g49e2a

[PATCH 2/3] git-am: Add command line parameter `--keep-cr` passing it to git-mailsplit.

[Stefan-W. Hahn]

The behaviour of git-mailsplit, which is called from git-am for
patches in mbox format, has been changed in commit c2ca1d79. The new
default behaviour will remove `\r` from line endings with `\r\n`.

If applying patches with the following command sequence

   git format-patch --stdout ... | git am ...

in repositories having files with dos and unix line endings,
git-mailsplit must be called with `--keep-cr` parameter.

This patch adds the command line parameter `--keep-cr` for git-am and
the configuration `am.keepcr`.

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 Documentation/config.txt |    6 ++++++
 git-am.sh                |   27 ++++++++++++++++++++-------
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 4c36aa9..aa452f3 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -550,6 +550,12 @@ it will be treated as a shell command.  For example, defining
 executed from the top-level directory of a repository, which may
 not necessarily be the current directory.
 
+am.keepcr::
+	If true, git-am will call git-mailsplit for patches in mbox format 
+	with parameter '--keep-cr'. In this case git-mailsplit will
+	not remove `\r` from lines ending with `\r\n`. 
+	See linkgit:git-am[1], linkgit:git-mailsplit[1].
+
 apply.ignorewhitespace::
 	When set to 'change', tells 'git apply' to ignore changes in
 	whitespace, in the same way as the '--ignore-space-change'
diff --git a/git-am.sh b/git-am.sh
index c8b9cbb..3057a83 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -15,6 +15,7 @@ q,quiet         be quiet
 s,signoff       add a Signed-off-by line to the commit message
 u,utf8          recode into utf8 (default)
 k,keep          pass -k flag to git-mailinfo
+keep-cr         pass --keep-cr flag to git-mailsplit for mbox format
 c,scissors      strip everything before a scissors line
 whitespace=     pass it through git-apply
 ignore-space-change pass it through git-apply
@@ -216,12 +217,12 @@ check_patch_format () {
 split_patches () {
 	case "$patch_format" in
 	mbox)
-		case "$rebasing" in
-		'')
-			keep_cr= ;;
-		?*)
-			keep_cr=--keep-cr ;;
-		esac
+		if test -n "$rebasing$keepcr"
+		then
+                    keep_cr=--keep-cr
+		else
+                    keep_cr=
+		fi
 		git mailsplit -d"$prec" -o"$dotest" -b $keep_cr -- "$@" > "$dotest/last" ||
 		clean_abort
 		;;
@@ -290,13 +291,18 @@ split_patches () {
 
 prec=4
 dotest="$GIT_DIR/rebase-apply"
-sign= utf8=t keep= skip= interactive= resolved= rebasing= abort=
+sign= utf8=t keep= keepcr= skip= interactive= resolved= rebasing= abort=
 resolvemsg= resume= scissors= no_inbody_headers=
 git_apply_opt=
 committer_date_is_author_date=
 ignore_date=
 allow_rerere_autoupdate=
 
+if test "$(git config --bool --get am.keepcr)" = true
+then 
+    keepcr=t
+fi
+
 while test $# != 0
 do
 	case "$1" in
@@ -347,6 +353,8 @@ do
 		allow_rerere_autoupdate="$1" ;;
 	-q|--quiet)
 		GIT_QUIET=t ;;
+	--keep-cr)
+		keepcr=t ;;
 	--)
 		shift; break ;;
 	*)
@@ -452,6 +460,7 @@ else
 	echo "$sign" >"$dotest/sign"
 	echo "$utf8" >"$dotest/utf8"
 	echo "$keep" >"$dotest/keep"
+	echo "$keepcr" >"$dotest/keepcr"
 	echo "$scissors" >"$dotest/scissors"
 	echo "$no_inbody_headers" >"$dotest/no_inbody_headers"
 	echo "$GIT_QUIET" >"$dotest/quiet"
@@ -495,6 +504,10 @@ if test "$(cat "$dotest/keep")" = t
 then
 	keep=-k
 fi
+if test "$(cat "$dotest/keepcr")" = t
+then
+	keepcr=--keep-cr
+fi
 case "$(cat "$dotest/scissors")" in
 t)
 	scissors=--scissors ;;
-- 
1.7.0.rc2.31.g49e2a

Re: [PATCH 2/3] git-am: Add command line parameter `--keep-cr` passing it to git-mailsplit.

[Junio C Hamano]

"Stefan-W. Hahn" <stefan.hahn@s-hahn.de> writes:

> The behaviour of git-mailsplit, which is called from git-am for
> patches in mbox format, has been changed in commit c2ca1d79. The new
> default behaviour will remove `\r` from line endings with `\r\n`.

This might offend people who caused c2ca1d7 (Allow mailsplit (and hence
git-am) to handle mails with CRLF line-endings, 2009-08-04) to come into
existence in the first place, as their argument was that "git am" not
reading from the output from their MUA's save-as (Thunderbird I think it
was but I may be mistaken) was a _bug_.  I personally didn't like that
bugfix very much and we could have added --strip-cr to help them back
then, but that is not what happened.

Perhaps this would be a more agreeable description of the backstory?

    c2ca1d7 (Allow mailsplit (and hence git-am) to handle mails with CRLF
    line-endings, 2009-08-04) fixed "git mailsplit" to help people with
    MUA whose output from save-as command uses CRLF as line terminators by
    stripping CR at the end of lines.

    However, when you know you are feeding output from "git format-patch"
    directly to "git am", and especially when your contents have CR at the
    end of line, such stripping is undesirable.  To help such a use case,
    teach --keep-cr option to "git am" and pass that to "git mailinfo".

> This patch adds the command line parameter `--keep-cr` for git-am and
> the configuration `am.keepcr`.

If one sets am.keepcr (because he regularly runs format-patch piped to am
by hand), but occasionally wants to apply an e-mailed patch out of his MUA
that happens to write things out with CRLF, how would one do so, without
touching the configuration (and not forgetting to revert the change after
doing so)?

As a general rule, if you introduce a new configuration, you need to make
sure that the configuration can be overriden per invocation if necessary,
and it is usually done from the command line, i.e. "--no-keep-cr".

I have to warn you that it would be a lot more work that needs careful
thinking than adding a command line option alone, so you may want to split
this [PATCH 2/3] into two, one to add command line option, and the other
to add configuration.

> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 4c36aa9..aa452f3 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -550,6 +550,12 @@ it will be treated as a shell command.  For example, defining
>  executed from the top-level directory of a repository, which may
>  not necessarily be the current directory.
>  
> +am.keepcr::
> +	If true, git-am will call git-mailsplit for patches in mbox format 
> +	with parameter '--keep-cr'. In this case git-mailsplit will
> +	not remove `\r` from lines ending with `\r\n`. 

Hence you would need something like:

    s/$/  Can be overriden by giving --no-keep-cr from the command line./

Also Documentation/git-am.txt would need something like:

--keep-cr::
--no-keep-cr::
	With --keep-cr, call git-mailsplit with the same option, to
        prevent it from stripping CR at the end of lines.  `am.keepcr`
        configuration variable can be used to specify the default
	behaviour.  --no-keep-cr is useful to override `am.keepcr`.

> diff --git a/git-am.sh b/git-am.sh
> index c8b9cbb..3057a83 100755
> --- a/git-am.sh
> +++ b/git-am.sh
> @@ -347,6 +353,8 @@ do
>  		allow_rerere_autoupdate="$1" ;;
>  	-q|--quiet)
>  		GIT_QUIET=t ;;
> +	--keep-cr)
> +		keepcr=t ;;
>  	--)
>  		shift; break ;;
>  	*)

And you obviously need to have "--no-keep-cr" here...

> @@ -452,6 +460,7 @@ else
>  	echo "$sign" >"$dotest/sign"
>  	echo "$utf8" >"$dotest/utf8"
>  	echo "$keep" >"$dotest/keep"
> +	echo "$keepcr" >"$dotest/keepcr"
>  	echo "$scissors" >"$dotest/scissors"
>  	echo "$no_inbody_headers" >"$dotest/no_inbody_headers"
>  	echo "$GIT_QUIET" >"$dotest/quiet"
> @@ -495,6 +504,10 @@ if test "$(cat "$dotest/keep")" = t
>  then
>  	keep=-k
>  fi
> +if test "$(cat "$dotest/keepcr")" = t
> +then
> +	keepcr=--keep-cr
> +fi

Also you may have to set keepcr to --no-keep-cr or something (I won't do
the necessary thinking for you while writing this message), to deal with a
case where:

 - The user has am.keepcr set to true;

 - This particular invocation was made with --no-keep-cr from the command
   line;

 - It stopped due to unappliable patch in the series and $dotest/keepcr
   became empty;

 - The user dealt with the stoppage and restarted the command; we read
   empty from $dotest/keepcr.

If I am reading your patch correctly, I think the restarted command will
use keepcr=t that was set by reading from the configuration at the
beginning?

[PATCH 3/3] Adding test for `--keep-cr` for git-am.

[Stefan-W. Hahn]

This test adds test for git-am dos line endings, the command sequence
'git format-patch ... | git am ...' and the configuration variable
`am.keepcr`.

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 t/t4253-am-keep-cr-dos.sh |   68 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 68 insertions(+), 0 deletions(-)
 create mode 100644 t/t4253-am-keep-cr-dos.sh

diff --git a/t/t4253-am-keep-cr-dos.sh b/t/t4253-am-keep-cr-dos.sh
new file mode 100644
index 0000000..a4f5f80
--- /dev/null
+++ b/t/t4253-am-keep-cr-dos.sh
@@ -0,0 +1,68 @@
+#!/bin/sh
+#
+# Copyright (c) 2010 Stefan-W. Hahn
+#
+
+test_description='git-am mbox with dos line ending.
+
+'
+. ./test-lib.sh
+
+# Three patches which will be added as files with dos line ending.
+
+cat > file1 <<\EOF
+line 1
+EOF
+
+cat > file2 <<\EOF
+line 1
+line 2
+EOF
+
+cat > file3 <<\EOF
+line 1
+line 2
+line 3
+EOF
+
+test_expect_success 'setup repository with dos files' '
+        append_cr <file1 >file
+        git add file &&
+        git commit -m Initial &&
+        git tag start &&
+        append_cr <file2 >file
+        git commit -a -m Second &&
+        git tag start2 &&
+        append_cr <file3 >file
+        git commit -a -m Third &&
+	git format-patch -k --stdout start.. > format-patch.diff
+'
+
+test_expect_success 'mailsplit format-patch of dos files' '
+        mkdir split &&
+        git mailsplit -osplit format-patch.diff &&
+        cat split/0001 split/0002 > mailsplit.diff &&
+        test_must_fail test_cmp format-patch.diff mailsplit.diff
+'
+
+test_expect_success 'mailsplit --keep-cr format-patch of dos files' '
+        mkdir split2 &&
+        git mailsplit --keep-cr -osplit2 format-patch.diff &&
+        cat split2/0001 split2/0002 > mailsplit2.diff &&
+        test_cmp format-patch.diff mailsplit2.diff
+'
+
+test_expect_success 'format-patch with dos files --keep-cr' '
+        git checkout -b new start &&
+	git format-patch -k --stdout start..master | git am --keep-cr -k -3 &&
+        git diff master
+'
+
+test_expect_success 'format-patch with dos files config.mailsplit' '
+        git config am.keepcr 1 &&
+        git checkout -b new3 start &&
+	git format-patch -k --stdout start..master | git am -k -3 &&
+        git diff master
+'
+
+test_done
-- 
1.7.0.rc2.31.g49e2a

Re: [PATCH 3/3] Adding test for `--keep-cr` for git-am.

[Junio C Hamano]

"Stefan-W. Hahn" <stefan.hahn@s-hahn.de> writes:

> This test adds test for git-am dos line endings, the command sequence
> 'git format-patch ... | git am ...' and the configuration variable
> `am.keepcr`.
>
> Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
> ---
>  t/t4253-am-keep-cr-dos.sh |   68 +++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 68 insertions(+), 0 deletions(-)
>  create mode 100644 t/t4253-am-keep-cr-dos.sh
>
> diff --git a/t/t4253-am-keep-cr-dos.sh b/t/t4253-am-keep-cr-dos.sh
> new file mode 100644

This should be 100755, if this need to be a new separate test.

> index 0000000..a4f5f80
> --- /dev/null
> +++ b/t/t4253-am-keep-cr-dos.sh
> @@ -0,0 +1,68 @@
> +#!/bin/sh
> +#
> +# Copyright (c) 2010 Stefan-W. Hahn
> +#
> +
> +test_description='git-am mbox with dos line ending.
> +
> +'
> +. ./test-lib.sh
> +
> +# Three patches which will be added as files with dos line ending.
> +
> +cat > file1 <<\EOF
> +line 1
> +EOF
> +
> +cat > file2 <<\EOF
> +line 1
> +line 2
> +EOF
> +
> +cat > file3 <<\EOF
> +line 1
> +line 2
> +line 3
> +EOF
> +
> +test_expect_success 'setup repository with dos files' '
> +        append_cr <file1 >file
> +        git add file &&
> +        git commit -m Initial &&
> +        git tag start &&
> +        append_cr <file2 >file
> +        git commit -a -m Second &&
> +        git tag start2 &&
> +        append_cr <file3 >file
> +        git commit -a -m Third &&
> +	git format-patch -k --stdout start.. > format-patch.diff
> +'

Hmm, do I see a mixed indentation here?

> +test_expect_success 'mailsplit format-patch of dos files' '
> +        mkdir split &&
> +        git mailsplit -osplit format-patch.diff &&
> +        cat split/0001 split/0002 > mailsplit.diff &&
> +        test_must_fail test_cmp format-patch.diff mailsplit.diff
> +'
> +
> +test_expect_success 'mailsplit --keep-cr format-patch of dos files' '
> +        mkdir split2 &&
> +        git mailsplit --keep-cr -osplit2 format-patch.diff &&
> +        cat split2/0001 split2/0002 > mailsplit2.diff &&
> +        test_cmp format-patch.diff mailsplit2.diff
> +'

These seem to be mailsplit test; are they necessary?  mailsplit is an
internal implementation detail of am, and we might later want to change
what it does as long as the change does not affect what the calling am
ends up doing, but even with benign change the above two test would fail.

> +test_expect_success 'format-patch with dos files --keep-cr' '
> +        git checkout -b new start &&
> +	git format-patch -k --stdout start..master | git am --keep-cr -k -3 &&

If a long line bothers you, you can split the line at pipe like this:

	git format-patch -k --stdout start..master |
	git am --keep-cr -k -3 &&

Because the shell knows that you haven't finished your sentence yet when
it sees the pipe at the end of line, you do not need a backslash at the
end (the same goes for the && at the end).

> +        git diff master
> +'

"git diff" by default does not report presense or absense of difference
with its exit code.  A traditional way to check this is

	test -z "$(git diff master)"

You can choose to use a more modern

	git diff --exit-code master

> +test_expect_success 'format-patch with dos files config.mailsplit' '
> +        git config am.keepcr 1 &&
> +        git checkout -b new3 start &&
> +	git format-patch -k --stdout start..master | git am -k -3 &&
> +        git diff master
> +'

Two tests are lacking.

 - test that "git am" fails without --keep-cr nor configuration when fed a
   history with CRLF.

 - test that "am.keepcr" can be countermanded per "git am" invocation;

Also I think t4252 needs to be taught about this new option, as I think
you would want the second ('please continue') invocation in this sequence

	$ git am --keep-cr mbox
        ... oops, one does not apply
        ... goes to fix
        $ git am

to keep your CR in the payload.

[PATCHv4 0/4] Using git-mailsplit in mixed line ending environment

[Stefan-W. Hahn]

Hello,

I'm using git in environments with files having dos or unix line
ending. I apply patches using 'git format-patch ... | git am ...'.
A change in git-mailsplit in commit c2ca1d79 introduced a change in
the default behaviour of git-mailsplit when splitting mbox patches. It
makes dos line endings to unix line endings. With this behaviour it is
impossible to apply patches.

The following patches introduce the '--kepp-cr', '--no-keep-cr'
parameter to git-am an an additional possibility to set '--keep-cr'
via configuration for git-am. Also I added missing description for
'--keep-cr' of git-mailsplit.

Second round:
I changed 'mailsplit.keep-cr' to 'mailsplit.keepcr' as suggested by Jakub
and comment in the testcase.

Third round:
I moved configuration 'mailsplit.keepcr' to 'am.keepcr' because
git-mailsplit can be used outside a git repository (thx Junio).

Fourth round:
I considered the replies from Junio and split up the patch introducing 
configuration am.keepcr in two. I introduced an '--no-keep-cr' to overwrite
configuration. I removed git-mailsplit tests from the test suite, they only 
showed the behaviour when using dos line ending.
I added some tests to show the correct behaviour in different situations
using the new parameters and configuration.


Stefan

[PATCH 1/4] git-mailsplit: Show parameter '--keep-cr' in usage and documentation
[PATCH 2/4] git-am: Add command line parameter `--keep-cr` passing it to git-mailsplit.
[PATCH 3/4] git-am: Add configuration am.keepcr and parameter --no-keep-cr to override configuration.
[PATCH 4/4] git-am: Adding tests for `--keep-cr`, `--no-keep-cr` and `am.keepcr`.
 
Documentation/config.txt        |    7 +++
Documentation/git-am.txt        |    9 +++-
Documentation/git-mailsplit.txt |    5 ++-
builtin-mailsplit.c             |    2 +-
git-am.sh                       |   32 ++++++++++---
t/t4253-am-keep-cr-dos.sh       |   96 +++++++++++++++++++++++++++++++++++++++
6 files changed, 141 insertions(+), 10 deletions(-)

Re: [PATCHv4 0/4] Using git-mailsplit in mixed line ending environment

[Junio C Hamano]

Thanks, will queue.

[PATCH 1/4] git-mailsplit: Show parameter '--keep-cr' in usage and documentation

[Stefan-W. Hahn]

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 Documentation/git-mailsplit.txt |    5 ++++-
 builtin-mailsplit.c             |    2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-mailsplit.txt b/Documentation/git-mailsplit.txt
index 5cc94ec..a634485 100644
--- a/Documentation/git-mailsplit.txt
+++ b/Documentation/git-mailsplit.txt
@@ -7,7 +7,7 @@ git-mailsplit - Simple UNIX mbox splitter program
 
 SYNOPSIS
 --------
-'git mailsplit' [-b] [-f<nn>] [-d<prec>] -o<directory> [--] [<mbox>|<Maildir>...]
+'git mailsplit' [-b] [-f<nn>] [-d<prec>] [--keep-cr] -o<directory> [--] [<mbox>|<Maildir>...]
 
 DESCRIPTION
 -----------
@@ -43,6 +43,9 @@ OPTIONS
 	Skip the first <nn> numbers, for example if -f3 is specified,
 	start the numbering with 0004.
 
+--keep-cr::
+	Do not remove `\r` from lines ending with `\r\n`.
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>
diff --git a/builtin-mailsplit.c b/builtin-mailsplit.c
index 207e358..cdfc1b7 100644
--- a/builtin-mailsplit.c
+++ b/builtin-mailsplit.c
@@ -10,7 +10,7 @@
 #include "strbuf.h"
 
 static const char git_mailsplit_usage[] =
-"git mailsplit [-d<prec>] [-f<n>] [-b] -o<directory> [<mbox>|<Maildir>...]";
+"git mailsplit [-d<prec>] [-f<n>] [-b] [--keep-cr] -o<directory> [<mbox>|<Maildir>...]";
 
 static int is_from_line(const char *line, int len)
 {
-- 
1.7.0.98.g42448

[PATCH 2/4] git-am: Add command line parameter `--keep-cr` passing it to git-mailsplit.

[Stefan-W. Hahn]

c2ca1d7 (Allow mailsplit (and hence git-am) to handle mails with CRLF
line-endings, 2009-08-04) fixed "git mailsplit" to help people with
MUA whose output from save-as command uses CRLF as line terminators by
stripping CR at the end of lines.

However, when you know you are feeding output from "git format-patch"
directly to "git am", and especially when your contents have CR at the
end of line, such stripping is undesirable.  To help such a use case,
teach --keep-cr option to "git am" and pass that to "git mailinfo".

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 Documentation/git-am.txt |    7 ++++++-
 git-am.sh                |   22 +++++++++++++++-------
 2 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-am.txt b/Documentation/git-am.txt
index c66c565..df3c6d6 100644
--- a/Documentation/git-am.txt
+++ b/Documentation/git-am.txt
@@ -9,7 +9,7 @@ git-am - Apply a series of patches from a mailbox
 SYNOPSIS
 --------
 [verse]
-'git am' [--signoff] [--keep] [--utf8 | --no-utf8]
+'git am' [--signoff] [--keep] [--keep-cr] [--utf8 | --no-utf8]
 	 [--3way] [--interactive] [--committer-date-is-author-date]
 	 [--ignore-date] [--ignore-space-change | --ignore-whitespace]
 	 [--whitespace=<option>] [-C<n>] [-p<n>] [--directory=<dir>]
@@ -39,6 +39,11 @@ OPTIONS
 --keep::
 	Pass `-k` flag to 'git mailinfo' (see linkgit:git-mailinfo[1]).
 
+--keep-cr::
+	With `--keep-cr`, call 'git mailsplit' (see linkgit:git-mailsplit[1])
+	with the same option, to prevent it from stripping CR at the end of 
+	lines.
+
 -c::
 --scissors::
 	Remove everything in body before a scissors line (see
diff --git a/git-am.sh b/git-am.sh
index ebfbee5..7563609 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -15,6 +15,7 @@ q,quiet         be quiet
 s,signoff       add a Signed-off-by line to the commit message
 u,utf8          recode into utf8 (default)
 k,keep          pass -k flag to git-mailinfo
+keep-cr         pass --keep-cr flag to git-mailsplit for mbox format
 c,scissors      strip everything before a scissors line
 whitespace=     pass it through git-apply
 ignore-space-change pass it through git-apply
@@ -217,12 +218,12 @@ check_patch_format () {
 split_patches () {
 	case "$patch_format" in
 	mbox)
-		case "$rebasing" in
-		'')
-			keep_cr= ;;
-		?*)
-			keep_cr=--keep-cr ;;
-		esac
+		if test -n "$rebasing$keepcr"
+		then
+                    keep_cr=--keep-cr
+		else
+                    keep_cr=
+		fi
 		git mailsplit -d"$prec" -o"$dotest" -b $keep_cr -- "$@" > "$dotest/last" ||
 		clean_abort
 		;;
@@ -291,7 +292,7 @@ split_patches () {
 
 prec=4
 dotest="$GIT_DIR/rebase-apply"
-sign= utf8=t keep= skip= interactive= resolved= rebasing= abort=
+sign= utf8=t keep= keepcr= skip= interactive= resolved= rebasing= abort=
 resolvemsg= resume= scissors= no_inbody_headers=
 git_apply_opt=
 committer_date_is_author_date=
@@ -348,6 +349,8 @@ do
 		allow_rerere_autoupdate="$1" ;;
 	-q|--quiet)
 		GIT_QUIET=t ;;
+	--keep-cr)
+		keepcr=t ;;
 	--)
 		shift; break ;;
 	*)
@@ -453,6 +456,7 @@ else
 	echo "$sign" >"$dotest/sign"
 	echo "$utf8" >"$dotest/utf8"
 	echo "$keep" >"$dotest/keep"
+	echo "$keepcr" >"$dotest/keepcr"
 	echo "$scissors" >"$dotest/scissors"
 	echo "$no_inbody_headers" >"$dotest/no_inbody_headers"
 	echo "$GIT_QUIET" >"$dotest/quiet"
@@ -496,6 +500,10 @@ if test "$(cat "$dotest/keep")" = t
 then
 	keep=-k
 fi
+if test "$(cat "$dotest/keepcr")" = t
+then
+	keepcr=--keep-cr
+fi
 case "$(cat "$dotest/scissors")" in
 t)
 	scissors=--scissors ;;
-- 
1.7.0.98.g42448

[PATCH 3/4] git-am: Add configuration am.keepcr and parameter --no-keep-cr to override configuration.

[Stefan-W. Hahn]

This patch adds the configuration `am.keepcr` for git-am. It also adds
`--no-keep-cr` parameter for git-am to give the possibility to
override configuration from command line.

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 Documentation/config.txt |    7 +++++++
 Documentation/git-am.txt |    6 ++++--
 git-am.sh                |   20 +++++++++++++++-----
 3 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 664de6b..403c392 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -555,6 +555,13 @@ it will be treated as a shell command.  For example, defining
 executed from the top-level directory of a repository, which may
 not necessarily be the current directory.
 
+am.keepcr::
+	If true, git-am will call git-mailsplit for patches in mbox format 
+	with parameter '--keep-cr'. In this case git-mailsplit will
+	not remove `\r` from lines ending with `\r\n`. Can be overrriden
+	by giving '--no-keep-cr' from the command line.
+	See linkgit:git-am[1], linkgit:git-mailsplit[1].
+
 apply.ignorewhitespace::
 	When set to 'change', tells 'git apply' to ignore changes in
 	whitespace, in the same way as the '--ignore-space-change'
diff --git a/Documentation/git-am.txt b/Documentation/git-am.txt
index df3c6d6..aef2b86 100644
--- a/Documentation/git-am.txt
+++ b/Documentation/git-am.txt
@@ -9,7 +9,7 @@ git-am - Apply a series of patches from a mailbox
 SYNOPSIS
 --------
 [verse]
-'git am' [--signoff] [--keep] [--keep-cr] [--utf8 | --no-utf8]
+'git am' [--signoff] [--keep] [--keep-cr | --no-keep-cr] [--utf8 | --no-utf8]
 	 [--3way] [--interactive] [--committer-date-is-author-date]
 	 [--ignore-date] [--ignore-space-change | --ignore-whitespace]
 	 [--whitespace=<option>] [-C<n>] [-p<n>] [--directory=<dir>]
@@ -40,9 +40,11 @@ OPTIONS
 	Pass `-k` flag to 'git mailinfo' (see linkgit:git-mailinfo[1]).
 
 --keep-cr::
+--no-keep-cr::
 	With `--keep-cr`, call 'git mailsplit' (see linkgit:git-mailsplit[1])
 	with the same option, to prevent it from stripping CR at the end of 
-	lines.
+	lines. `am.keepcr` configuration variable can be used to specify the 
+	default behaviour.  `--no-keep-cr` is useful to override `am.keepcr`.
 
 -c::
 --scissors::
diff --git a/git-am.sh b/git-am.sh
index 7563609..ed42df0 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -16,6 +16,7 @@ s,signoff       add a Signed-off-by line to the commit message
 u,utf8          recode into utf8 (default)
 k,keep          pass -k flag to git-mailinfo
 keep-cr         pass --keep-cr flag to git-mailsplit for mbox format
+no-keep-cr      do not pass --keep-cr flag to git-mailsplit independent of am.keepcr
 c,scissors      strip everything before a scissors line
 whitespace=     pass it through git-apply
 ignore-space-change pass it through git-apply
@@ -218,7 +219,7 @@ check_patch_format () {
 split_patches () {
 	case "$patch_format" in
 	mbox)
-		if test -n "$rebasing$keepcr"
+		if test -n "$rebasing" -o "$keepcr" = t
 		then
                     keep_cr=--keep-cr
 		else
@@ -299,6 +300,11 @@ committer_date_is_author_date=
 ignore_date=
 allow_rerere_autoupdate=
 
+if test "$(git config --bool --get am.keepcr)" = true
+then 
+    keepcr=t
+fi
+
 while test $# != 0
 do
 	case "$1" in
@@ -351,6 +357,8 @@ do
 		GIT_QUIET=t ;;
 	--keep-cr)
 		keepcr=t ;;
+	--no-keep-cr)
+		keepcr=f ;;
 	--)
 		shift; break ;;
 	*)
@@ -500,10 +508,12 @@ if test "$(cat "$dotest/keep")" = t
 then
 	keep=-k
 fi
-if test "$(cat "$dotest/keepcr")" = t
-then
-	keepcr=--keep-cr
-fi
+case "$(cat "$dotest/keepcr")" in
+t)
+	keepcr=--keep-cr ;;
+f)
+	keepcr=--no-keep-cr ;;
+esac
 case "$(cat "$dotest/scissors")" in
 t)
 	scissors=--scissors ;;
-- 
1.7.0.98.g42448

[PATCH 4/4] git-am: Adding tests for `--keep-cr`, `--no-keep-cr` and `am.keepcr`.

[Stefan-W. Hahn]

This test adds tests for git-am using files with dos line endings for
various combinations of `--keep-cr`, `--no-keep-cr` and `am.keepcr`.

Signed-off-by: Stefan-W. Hahn <stefan.hahn@s-hahn.de>
---
 t/t4253-am-keep-cr-dos.sh |   96 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 96 insertions(+), 0 deletions(-)
 create mode 100755 t/t4253-am-keep-cr-dos.sh

diff --git a/t/t4253-am-keep-cr-dos.sh b/t/t4253-am-keep-cr-dos.sh
new file mode 100755
index 0000000..6c4cb35
--- /dev/null
+++ b/t/t4253-am-keep-cr-dos.sh
@@ -0,0 +1,96 @@
+#!/bin/sh
+#
+# Copyright (c) 2010 Stefan-W. Hahn
+#
+
+test_description='git-am mbox with dos line ending.
+
+'
+. ./test-lib.sh
+
+# Three patches which will be added as files with dos line ending.
+
+cat > file1 <<\EOF
+line 1
+EOF
+
+cat > file1a <<\EOF
+line 1
+line 4
+EOF
+
+cat > file2 <<\EOF
+line 1
+line 2
+EOF
+
+cat > file3 <<\EOF
+line 1
+line 2
+line 3
+EOF
+
+test_expect_success 'setup repository with dos files' '
+        append_cr <file1 >file &&
+        git add file &&
+        git commit -m Initial &&
+        git tag initial &&
+        append_cr <file2 >file &&
+        git commit -a -m Second &&
+        append_cr <file3 >file &&
+        git commit -a -m Third
+'
+
+test_expect_success 'am with dos files without --keep-cr' '
+        git checkout -b dosfiles initial &&
+        git format-patch -k initial..master &&
+        test_must_fail git am -k -3 000*.patch &&
+        git am --abort &&
+        rm -rf .git/rebase-apply 000*.patch
+'
+
+test_expect_success 'am with dos files with --keep-cr' '
+        git checkout -b dosfiles-keep-cr initial &&
+        git format-patch -k --stdout initial..master | git am --keep-cr -k -3 &&
+        git diff --exit-code master
+'
+
+test_expect_success 'am with dos files config am.keepcr' '
+        git config am.keepcr 1 &&
+        git checkout -b dosfiles-conf-keepcr initial &&
+        git format-patch -k --stdout initial..master | git am -k -3 &&
+        git diff --exit-code master
+'
+
+test_expect_success 'am with dos files config am.keepcr overriden by --no-keep-cr' '
+        git config am.keepcr 1 &&
+        git checkout -b dosfiles-conf-keepcr-override initial &&
+        git format-patch -k initial..master &&
+        test_must_fail git am -k -3 --no-keep-cr 000*.patch &&
+        git am --abort &&
+        rm -rf .git/rebase-apply 000*.patch
+'
+
+test_expect_success 'am with dos files with --keep-cr continue' '
+        git checkout -b dosfiles-keep-cr-continue initial &&
+        git format-patch -k initial..master &&
+        append_cr <file1a >file &&
+        git commit -m "different patch" file &&
+        test_must_fail git am --keep-cr -k -3 000*.patch &&
+        append_cr <file2 >file &&
+        git add file &&
+        git am -3 --resolved &&
+        git diff --exit-code master
+'
+
+test_expect_success 'am with unix files config am.keepcr overriden by --no-keep-cr' '
+        git config am.keepcr 1 &&
+        git checkout -b unixfiles-conf-keepcr-override initial &&
+        cp -f file1 file &&
+        git commit -m "line ending to unix" file &&
+        git format-patch -k initial..master &&
+        git am -k -3 --no-keep-cr 000*.patch &&
+        git diff --exit-code -w master
+'
+
+test_done
-- 
1.7.0.98.g42448