[PATCHv2 0/4] git-am support for foreign patches

[PATCHv2 0/4] git-am support for foreign patches

[Giuseppe Bilotta]

This patch series introduces a framework to allow git-am to support
patches that are not in mailbox format.

The first patch introduces the command-line settable patch_format
variable, describing the patch format.

The second patch introduces some patch format autodetection heuristics,
for mercurial and StGIT.

The third patch introduces actual support for StGIT patches (both series
and single patches).

The last one does some refactoring for the 'remove $dotest and exit 1'.

Giuseppe Bilotta (4):
  git-am foreign patch support: introduce patch_format
  git-am foreign patch support: autodetect some patch formats
  git-am foreign patch support: StGIT support
  git-am: refactor 'cleaning up and aborting'

 git-am.sh |  148 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 144 insertions(+), 4 deletions(-)

[PATCHv2 1/4] git-am foreign patch support: introduce patch_format

[Giuseppe Bilotta]

Set up a framework to allow git-am to support patches which are not in
mailbox format. Introduce a patch_format variable that presently can
only be set from the command line, defaulting to 'mbox' (the only
supported format) if not specified.
---
 git-am.sh |   37 +++++++++++++++++++++++++++++++++----
 1 files changed, 33 insertions(+), 4 deletions(-)

diff --git a/git-am.sh b/git-am.sh
index 578780b..da160de 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -18,6 +18,7 @@ whitespace=     pass it through git-apply
 directory=      pass it through git-apply
 C=              pass it through git-apply
 p=              pass it through git-apply
+patch-format=   format the patch(es) are in
 reject          pass it through git-apply
 resolvemsg=     override error message when patch failure occurs
 r,resolved      to be used after a patch failure
@@ -133,6 +134,32 @@ It does not apply to blobs recorded in its index."
     unset GITHEAD_$his_tree
 }
 
+patch_format=
+
+check_patch_format () {
+	# early return if patch_format was set from the command line
+	if test -n "$patch_format"
+	then
+		return 0
+	fi
+	patch_format=mbox
+}
+
+split_patches () {
+	case "$patch_format" in
+	mbox)
+		git mailsplit -d"$prec" -o"$dotest" -b -- "$@" > "$dotest/last" ||  {
+			rm -fr "$dotest"
+			exit 1
+		}
+		;;
+	*)
+		echo "Patch format $patch_format is not supported."
+		exit 1
+		;;
+	esac
+}
+
 prec=4
 dotest="$GIT_DIR/rebase-apply"
 sign= utf8=t keep= skip= interactive= resolved= rebasing= abort=
@@ -175,6 +202,8 @@ do
 		git_apply_opt="$git_apply_opt $(sq "$1=$2")"; shift ;;
 	-C|-p)
 		git_apply_opt="$git_apply_opt $(sq "$1$2")"; shift ;;
+	--patch-format)
+		shift ; patch_format="$1" ;;
 	--reject)
 		git_apply_opt="$git_apply_opt $1" ;;
 	--committer-date-is-author-date)
@@ -274,10 +303,10 @@ else
 		done
 		shift
 	fi
-	git mailsplit -d"$prec" -o"$dotest" -b -- "$@" > "$dotest/last" ||  {
-		rm -fr "$dotest"
-		exit 1
-	}
+
+	check_patch_format "$@"
+
+	split_patches "$@"
 
 	# -s, -u, -k, --whitespace, -3, -C and -p flags are kept
 	# for the resuming session after a patch failure.
-- 
1.6.3.1.248.gb44be

[PATCHv2 2/4] git-am foreign patch support: autodetect some patch formats

[Giuseppe Bilotta]

Default to mbox format if input is from stdin. Otherwise, look at the
first few lines of the first patch to try to guess its format.

Include checks for mailboxes, stgit patch series, stgit single patches
and hg patches.
---
 git-am.sh |   52 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 51 insertions(+), 1 deletions(-)

diff --git a/git-am.sh b/git-am.sh
index da160de..4cf66aa 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -142,7 +142,57 @@ check_patch_format () {
 	then
 		return 0
 	fi
-	patch_format=mbox
+
+	# we default to mbox format if input is from stdin and for
+	# directories
+	if test $# = 0 || test "x$1" = "x-" || test -d "$1"
+	then
+		patch_format=mbox
+		return 0
+	fi
+
+	# otherwise, check the first few lines of the first patch to try
+	# to detect its format
+	{
+		read l1
+		read l2
+		read l3
+		case "$l1" in
+			From\ *)
+				patch_format=mbox
+				;;
+			From:\ *)
+				patch_format=mbox
+				;;
+			'# This series applies on GIT commit'*)
+				patch_format=stgit-series
+				;;
+			"# HG changeset patch")
+				patch_format=hg
+				;;
+			*)
+				# if the second line is empty and the third is
+				# a From, Author or Date entry, this is very
+				# likely an StGIT patch
+				if test "x$l2" = x
+				then
+					case "$l3" in
+						From:\ *)
+							patch_format=stgit
+							;;
+						Author:\ *)
+							patch_format=stgit
+							;;
+						Date:\ *)
+							patch_format=stgit
+							;;
+						*)
+							;;
+					esac
+				fi
+				;;
+		esac
+	} < "$1"
 }
 
 split_patches () {
-- 
1.6.3.1.248.gb44be

[PATCHv2 3/4] git-am foreign patch support: StGIT support

[Giuseppe Bilotta]

Support StGIT patches by implementing a simple awk-based converter
mimicking StGIT's own parse_patch. Also support StGIT patch series by
'exploding' the index into a lif of files and re-running the mail
splitting with patch_format set to stgit.
---
 git-am.sh |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 60 insertions(+), 0 deletions(-)

diff --git a/git-am.sh b/git-am.sh
index 4cf66aa..1a00830 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -203,6 +203,66 @@ split_patches () {
 			exit 1
 		}
 		;;
+	stgit-series)
+		if test $# -ne 1
+		then
+			echo "Only one StGIT patch series can be applied at once"
+			exit 1
+		fi
+		series_dir=`dirname "$1"`
+		series_file="$1"
+		shift
+		{
+			set x
+			while read filename
+			do
+				set "$@" "$series_dir/$filename"
+			done
+			# remove the safety x
+			shift
+			# remove the arg coming from the first-line comment
+			shift
+		} < "$series_file"
+		# set the patch format appropriately
+		patch_format=stgit
+		# now handle the actual StGIT patches
+		split_patches "$@"
+		;;
+	stgit)
+		this=0
+		for stgit in "$@"
+		do
+			this=`expr "$this" + 1`
+			msgnum=`printf "%0${prec}d" $this`
+			touch "$dotest/$msgnum"
+			# Awk version of StGIT parse_patch. The first nonemptyline
+			# not starting with Author, From or Date is the
+			# subject, and the body starts with the next nonempty
+			# line not starting with Author, From or Date
+			awk 'BEGIN { subject=0 }
+			{
+				if (subject > 1)
+					print ;
+				else if (/^$/) next ;
+				else if (/^Author:/) print sub("Author", "From"), $ORS ;
+				else if (/^(From|Date)/) print ;
+				else if (subject) {
+					subject = 2 ;
+					print "" ;
+					print ;
+				} else {
+					print "Subject:", $0 ;
+					subject = 1;
+				}
+			}' "$stgit" > "$dotest/$msgnum" || {
+				echo "Failed to import $patch_format patch $stgit"
+				exit 1
+			}
+		done
+		echo "$this" > "$dotest/last"
+		this=
+		msgnum=
+		;;
 	*)
 		echo "Patch format $patch_format is not supported."
 		exit 1
-- 
1.6.3.1.248.gb44be

[PATCHv2 4/4] git-am: refactor 'cleaning up and aborting'

[Giuseppe Bilotta]

Introduce a clean_abort function that echoes an optional error message
to standard error, removes the dotest directory and exits with status 1.

Use it when patch format detection or patch splitting fails early.
---
 git-am.sh |   29 +++++++++++++++--------------
 1 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/git-am.sh b/git-am.sh
index 1a00830..057acfe 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -134,6 +134,15 @@ It does not apply to blobs recorded in its index."
     unset GITHEAD_$his_tree
 }
 
+clean_abort () {
+	if test $# -gt 0
+	then
+		echo "$@" > /dev/stderr
+	fi
+	rm -fr "$dotest"
+	exit 1
+}
+
 patch_format=
 
 check_patch_format () {
@@ -192,22 +201,18 @@ check_patch_format () {
 				fi
 				;;
 		esac
-	} < "$1"
+	} < "$1" || clean_abort
 }
 
 split_patches () {
 	case "$patch_format" in
 	mbox)
-		git mailsplit -d"$prec" -o"$dotest" -b -- "$@" > "$dotest/last" ||  {
-			rm -fr "$dotest"
-			exit 1
-		}
+		git mailsplit -d"$prec" -o"$dotest" -b -- "$@" > "$dotest/last" || clean_abort
 		;;
 	stgit-series)
 		if test $# -ne 1
 		then
-			echo "Only one StGIT patch series can be applied at once"
-			exit 1
+			clean_abort "Only one StGIT patch series can be applied at once"
 		fi
 		series_dir=`dirname "$1"`
 		series_file="$1"
@@ -222,7 +227,7 @@ split_patches () {
 			shift
 			# remove the arg coming from the first-line comment
 			shift
-		} < "$series_file"
+		} < "$series_file" || clean_abort
 		# set the patch format appropriately
 		patch_format=stgit
 		# now handle the actual StGIT patches
@@ -254,18 +259,14 @@ split_patches () {
 					print "Subject:", $0 ;
 					subject = 1;
 				}
-			}' "$stgit" > "$dotest/$msgnum" || {
-				echo "Failed to import $patch_format patch $stgit"
-				exit 1
-			}
+			}' "$stgit" > "$dotest/$msgnum" || clean_abort
 		done
 		echo "$this" > "$dotest/last"
 		this=
 		msgnum=
 		;;
 	*)
-		echo "Patch format $patch_format is not supported."
-		exit 1
+		clean_abort "Patch format $patch_format is not supported."
 		;;
 	esac
 }
-- 
1.6.3.1.248.gb44be

Re: [PATCHv2 4/4] git-am: refactor 'cleaning up and aborting'

[Junio C Hamano]

Giuseppe Bilotta <giuseppe.bilotta@gmail.com> writes:

> Introduce a clean_abort function that echoes an optional error message
> to standard error, removes the dotest directory and exits with status 1.
>
> Use it when patch format detection or patch splitting fails early.
> ---

S-o-b;

>  git-am.sh |   29 +++++++++++++++--------------
>  1 files changed, 15 insertions(+), 14 deletions(-)
>
> diff --git a/git-am.sh b/git-am.sh
> index 1a00830..057acfe 100755
> --- a/git-am.sh
> +++ b/git-am.sh
> @@ -134,6 +134,15 @@ It does not apply to blobs recorded in its index."
>      unset GITHEAD_$his_tree
>  }
>  
> +clean_abort () {
> +	if test $# -gt 0
> +	then
> +		echo "$@" > /dev/stderr

	echo >&2 "$@"

> +	fi
> +	rm -fr "$dotest"
> +	exit 1
> +}
> +
>  patch_format=
>  
>  check_patch_format () {
> @@ -192,22 +201,18 @@ check_patch_format () {
>  				fi
>  				;;
>  		esac
> -	} < "$1"
> +	} < "$1" || clean_abort
>  }
>  
>  split_patches () {
>  	case "$patch_format" in
>  	mbox)
> -		git mailsplit -d"$prec" -o"$dotest" -b -- "$@" > "$dotest/last" ||  {
> -			rm -fr "$dotest"
> -			exit 1
> -		}
> +		git mailsplit -d"$prec" -o"$dotest" -b -- "$@" > "$dotest/last" || clean_abort
>  		;;
>  	stgit-series)
>  		if test $# -ne 1
>  		then
> -			echo "Only one StGIT patch series can be applied at once"
> -			exit 1
> +			clean_abort "Only one StGIT patch series can be applied at once"
>  		fi
>  		series_dir=`dirname "$1"`
>  		series_file="$1"
> @@ -222,7 +227,7 @@ split_patches () {
>  			shift
>  			# remove the arg coming from the first-line comment
>  			shift
> -		} < "$series_file"
> +		} < "$series_file" || clean_abort
>  		# set the patch format appropriately
>  		patch_format=stgit
>  		# now handle the actual StGIT patches
> @@ -254,18 +259,14 @@ split_patches () {
>  					print "Subject:", $0 ;
>  					subject = 1;
>  				}
> -			}' "$stgit" > "$dotest/$msgnum" || {
> -				echo "Failed to import $patch_format patch $stgit"
> -				exit 1
> -			}
> +			}' "$stgit" > "$dotest/$msgnum" || clean_abort
>  		done
>  		echo "$this" > "$dotest/last"
>  		this=
>  		msgnum=
>  		;;
>  	*)
> -		echo "Patch format $patch_format is not supported."
> -		exit 1
> +		clean_abort "Patch format $patch_format is not supported."
>  		;;
>  	esac
>  }
> -- 
> 1.6.3.1.248.gb44be

Re: [PATCHv2 3/4] git-am foreign patch support: StGIT support

[Junio C Hamano]

Giuseppe Bilotta <giuseppe.bilotta@gmail.com> writes:

> Support StGIT patches by implementing a simple awk-based converter
> mimicking StGIT's own parse_patch. Also support StGIT patch series by
> 'exploding' the index into a lif of files and re-running the mail
> splitting with patch_format set to stgit.
> ---
>  git-am.sh |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 60 insertions(+), 0 deletions(-)
>
> diff --git a/git-am.sh b/git-am.sh
> index 4cf66aa..1a00830 100755
> --- a/git-am.sh
> +++ b/git-am.sh
> @@ -203,6 +203,66 @@ split_patches () {
>  			exit 1
>  		}
>  		;;
> +	stgit-series)
> +		if test $# -ne 1
> +		then
> +			echo "Only one StGIT patch series can be applied at once"
> +			exit 1
> +		fi
> +		series_dir=`dirname "$1"`
> +		series_file="$1"
> +		shift
> +		{
> +			set x
> +			while read filename
> +			do
> +				set "$@" "$series_dir/$filename"
> +			done
> +			# remove the safety x
> +			shift
> +			# remove the arg coming from the first-line comment
> +			shift
> +		} < "$series_file"
> +		# set the patch format appropriately
> +		patch_format=stgit
> +		# now handle the actual StGIT patches
> +		split_patches "$@"

Can an stgit patch file (or the leading pathname for that matter) have IFS
character in its name?

> +		;;
> +	stgit)
> +		this=0
> +		for stgit in "$@"
> +		do
> +			this=`expr "$this" + 1`
> +			msgnum=`printf "%0${prec}d" $this`
> +			touch "$dotest/$msgnum"

Portability tip from an old timer: do not create a new empty file with
"touch" (only use that command to update the timestamp of an existing
file).

Instead say

	>"$dotest/$msgnum"

> +			# Awk version of StGIT parse_patch. The first nonemptyline
> +			# not starting with Author, From or Date is the
> +			# subject, and the body starts with the next nonempty
> +			# line not starting with Author, From or Date
> +			awk 'BEGIN { subject=0 }
> +			{
> +				if (subject > 1)
> +					print ;
> +				else if (/^$/) next ;
> +				else if (/^Author:/) print sub("Author", "From"), $ORS ;

Can any token that match Author other than the initial "Author: " appear
on this line?

Since we rely on Perl but not awk in core-ish part of the scripted
Porcelains, it might be a good idea to write this in Perl as well.

> +				else if (/^(From|Date)/) print ;
> +				else if (subject) {
> +					subject = 2 ;
> +					print "" ;
> +					print ;
> +				} else {
> +					print "Subject:", $0 ;
> +					subject = 1;
> +				}
> +			}' "$stgit" > "$dotest/$msgnum" || {
> +				echo "Failed to import $patch_format patch $stgit"
> +				exit 1
> +			}
> +		done
> +		echo "$this" > "$dotest/last"
> +		this=
> +		msgnum=
> +		;;
>  	*)
>  		echo "Patch format $patch_format is not supported."
>  		exit 1
> -- 
> 1.6.3.1.248.gb44be

Re: [PATCHv2 3/4] git-am foreign patch support: StGIT support

[Giuseppe Bilotta]

On Wed, May 27, 2009 at 9:19 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Giuseppe Bilotta <giuseppe.bilotta@gmail.com> writes:
>
>> +             # set the patch format appropriately
>> +             patch_format=stgit
>> +             # now handle the actual StGIT patches
>> +             split_patches "$@"
>
> Can an stgit patch file (or the leading pathname for that matter) have IFS
> character in its name?

StGIT sanitizes filenames in a way similar to what format-patch does.

>> +             ;;
>> +     stgit)
>> +             this=0
>> +             for stgit in "$@"
>> +             do
>> +                     this=`expr "$this" + 1`
>> +                     msgnum=`printf "%0${prec}d" $this`
>> +                     touch "$dotest/$msgnum"
>
> Portability tip from an old timer: do not create a new empty file with
> "touch" (only use that command to update the timestamp of an existing
> file).
>
> Instead say
>
>        >"$dotest/$msgnum"
>
>> +                     # Awk version of StGIT parse_patch. The first nonemptyline
>> +                     # not starting with Author, From or Date is the
>> +                     # subject, and the body starts with the next nonempty
>> +                     # line not starting with Author, From or Date
>> +                     awk 'BEGIN { subject=0 }
>> +                     {
>> +                             if (subject > 1)
>> +                                     print ;
>> +                             else if (/^$/) next ;
>> +                             else if (/^Author:/) print sub("Author", "From"), $ORS ;
>
> Can any token that match Author other than the initial "Author: " appear
> on this line?

Irrelevant, sub only substitutes the first hit so only the initial
Author: is replaced.

> Since we rely on Perl but not awk in core-ish part of the scripted
> Porcelains, it might be a good idea to write this in Perl as well.

By 'as well' you mean in case awk is missing, and have both, or by
only keeping a perl version?

-- 
Giuseppe "Oblomov" Bilotta

Re: [PATCHv2 3/4] git-am foreign patch support: StGIT support

[Junio C Hamano]

Giuseppe Bilotta <giuseppe.bilotta@gmail.com> writes:

> On Wed, May 27, 2009 at 9:19 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Giuseppe Bilotta <giuseppe.bilotta@gmail.com> writes:
>>
>>> +             # set the patch format appropriately
>>> +             patch_format=stgit
>>> +             # now handle the actual StGIT patches
>>> +             split_patches "$@"
>>
>> Can an stgit patch file (or the leading pathname for that matter) have IFS
>> character in its name?
>
> StGIT sanitizes filenames in a way similar to what format-patch does.

Good ;-)

>> Since we rely on Perl but not awk in core-ish part of the scripted
>> Porcelains, it might be a good idea to write this in Perl as well.
>
> By 'as well' you mean in case awk is missing, and have both, or by
> only keeping a perl version?

"As well" I meant "Using Perl instead of awk, just like other scripted
Porcelains."

Re: [PATCHv2 2/4] git-am foreign patch support: autodetect some patch formats

[Junio C Hamano]

Giuseppe Bilotta <giuseppe.bilotta@gmail.com> writes:

> +	# otherwise, check the first few lines of the first patch to try
> +	# to detect its format
> +	{
> +		read l1
> +		read l2
> +		read l3
> +		case "$l1" in
> +			From\ *)
> +				patch_format=mbox
> +				;;
> +			From:\ *)
> +				patch_format=mbox
> +				;;
> +			'# This series applies on GIT commit'*)
> +				patch_format=stgit-series
> +				;;
> +			"# HG changeset patch")
> +				patch_format=hg
> +				;;
> +			*)
> +				# if the second line is empty and the third is
> +				# a From, Author or Date entry, this is very
> +				# likely an StGIT patch
> +				if test "x$l2" = x
> +				then
> +					case "$l3" in
> +						From:\ *)
> +							patch_format=stgit
> +							;;
> +						Author:\ *)
> +							patch_format=stgit
> +							;;
> +						Date:\ *)
> +							patch_format=stgit
> +							;;
> +						*)
> +							;;
> +					esac
> +				fi
> +				;;
> +		esac

Don't make the case unnecessary too deep nor redundant.

	...
	read l3
	case "$l1" in
	"From "* | "From: "* )
		patch_format=mbox
		;;
	"# This ...")
		...
		;;
	*)
		case "$l2,$l3" in
		,"From: "* | ,"Author: "* | ,"Date: "*)
			patch_format=stgit
			;;
		esac
		;;
	esac

> +	} < "$1"
>  }
>  
>  split_patches () {
> -- 
> 1.6.3.1.248.gb44be

Re: [PATCHv2 2/4] git-am foreign patch support: autodetect some patch formats

[Giuseppe Bilotta]

On Wed, May 27, 2009 at 9:19 AM, Junio C Hamano <gitster@pobox.com> wrote:

> Don't make the case unnecessary too deep nor redundant.
>
>        ...
>        read l3
>        case "$l1" in
>        "From "* | "From: "* )
>                patch_format=mbox
>                ;;

Oh thanks. I think I had tried the construct this way but failed to
make * and | work together. Looks like I was doing it wrong.

-- 
Giuseppe "Oblomov" Bilotta