How to split a patch

How to split a patch

[Paolo Ciarrocchi]

Hi all,
I have a big patch that affects a single file that needs to be split in
a few logical patches.

I know ho to do the opposite process, rebase -i and squash is something
I'm really used to do but this time that trick is not going to help me.

What is the preferred way to split a big patch in a series of smaller patches?

Thanks.

Ciao,
-- 
Paolo
http://paolo.ciarrocchi.googlepages.com/

Re: How to split a patch

[Junio C Hamano]

"Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:

> I know ho to do the opposite process, rebase -i and squash is something
> I'm really used to do but this time that trick is not going to help me.
>
> What is the preferred way to split a big patch in a series of
> smaller patches?


I personally found the procedure described there a bit on the
sketchy side, but does "SPLITTING COMMITS" section of git-rebase
manual help?

Re: How to split a patch

[Paolo Ciarrocchi]

On Jan 28, 2008 10:25 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:
>
> > I know ho to do the opposite process, rebase -i and squash is something
> > I'm really used to do but this time that trick is not going to help me.
> >
> > What is the preferred way to split a big patch in a series of
> > smaller patches?
>
>
> I personally found the procedure described there a bit on the
> sketchy side, but does "SPLITTING COMMITS" section of git-rebase
> manual help?

Yes it helps but I still wonder whether thereis a "simpler" way to achive that.
Is it possible to split a patch selecting the hunk in git gui or any
other graphical
tool?

That would be a good starting point for a newbie (like me).

Thanks!

Ciao,
-- 
Paolo
http://paolo.ciarrocchi.googlepages.com/

Re: How to split a patch

[Matthieu Moy]

"Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:

> Yes it helps but I still wonder whether thereis a "simpler" way to achive that.
> Is it possible to split a patch selecting the hunk in git gui or any
> other graphical
> tool?

You can apply the patch without commiting it, and them make several
partial commits, by right-click "stage hunk for commit" in git-gui.

-- 
Matthieu

Re: How to split a patch

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> "Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:
>
>> Yes it helps but I still wonder whether thereis a "simpler" way to achive that.
>> Is it possible to split a patch selecting the hunk in git gui or any
>> other graphical
>> tool?
>
> You can apply the patch without commiting it, and them make several
> partial commits, by right-click "stage hunk for commit" in git-gui.

Yes, and you can do the same with "git add -i".  These tools are
not quite nice, as they encourage a wrong workflow of committing
what you haven't had as a whole in the work tree.  By
definition, you are making untested commits between your base
commit (that presumably was tested well) and your final commit
(that would also be tested well).

Ideally, once you have a perfect state (i.e. the shape of the
tree recorded by the last commit in your "split" series), you
should be able to make a commit, and walk backwards, removing
the fanciest "finishing touches" changes from the work tree
files, test it, and record it as a new commit between where you
are and one commit before it.  A possible workflow could be:

	$ work work work, now it is perfect.
	$ git commit -a -m 'Now it is perfect'
        $ edit away the fanciest bits
        $ test test, the basics still work.
        $ git commit --splice -a -m 'Almost there'

And the last "git commit --splice" would record the tree object
as a commit, whose parent is the parent of the current HEAD, and
record the tree of the current HEAD as a (rewritten) commit that
is a child of that commit.

Graphically, you would start from (X is "the perfect commit"):

	---o-------X

then "--splice" would create a new commit "W" and record the
same tree as X as a new commit X' that is a child of W:

             .--W---X'
            /
	---o-------X

There is no such tool yet, though.

The splitting you can do with "rebase -i" instead walks
forwards.  That also lets you test before you make commits in
each step.

Re: How to split a patch

[Wincent Colaiuta]

El 28/1/2008, a las 10:47, Matthieu Moy escribió:

> "Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:
>
>> Yes it helps but I still wonder whether thereis a "simpler" way to  
>> achive that.
>> Is it possible to split a patch selecting the hunk in git gui or any
>> other graphical
>> tool?
>
> You can apply the patch without commiting it, and them make several
> partial commits, by right-click "stage hunk for commit" in git-gui.

Or "git add --interactive", which isn't GUI but is a similar idea in  
command line form.

And in case you've already applied the patch you can do a "git reset  
HEAD^"; this moves your HEAD back by one commit, puts the index back  
the way it was, but leaves the changes in your working tree. Then you  
stage your hunks and commit them like Matthieu says.

Cheers,
Wincent

Re: How to split a patch

[Boaz Harrosh]

[-- Attachment #1: Type: text/plain, Size: 2039 bytes --]

On Mon, Jan 28 2008 at 11:32 +0200, "Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> wrote:
> On Jan 28, 2008 10:25 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> "Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:
>>
>>> I know ho to do the opposite process, rebase -i and squash is something
>>> I'm really used to do but this time that trick is not going to help me.
>>>
>>> What is the preferred way to split a big patch in a series of
>>> smaller patches?
>>
>> I personally found the procedure described there a bit on the
>> sketchy side, but does "SPLITTING COMMITS" section of git-rebase
>> manual help?
> 
> Yes it helps but I still wonder whether thereis a "simpler" way to achive that.
> Is it possible to split a patch selecting the hunk in git gui or any
> other graphical
> tool?
> 
> That would be a good starting point for a newbie (like me).
> 
> Thanks!
> 
> Ciao,
What I do is edit the patch, let me explain.
- I create a new branch based at the base of the big patch. Now
  I have the original branch for reference. Lets say the original
  branch is called ALL, and this new branch SPLIT. and BASE is the base.
- # git-checkout -b SPLIT BASE
- # git-diff -R ALL >edit-the-patch-001.patch
- now I open the patch file with an editor. I use kwrite because of
  the good color highlighting.
- I remove all the hunks that I don't need, and am left with only the
  parts that I want to be in the first patch.
- I save the patch.
- If at the edit stage I have changed a big hunk and only used part
  of it I need to run fixpatch script attached below. If I only removed
  complete hunks then I'm good to go.
- I now use:
  # patch -p 1 < edit-the-patch-001.patch
- # git-commit -a
- and git-commit --amend until the first patch is good
- now do the 
  # git-diff -R ALL >edit-the-patch-002.patch
  and so on until there is nothing left. It is nice how
  every time it gets smaller and smaller.

[fixpatch command line]
usage: fixpatch inputfile outputfile
what I do is just
# fixpatch edit-the-patch-001.patch{,}

Boaz


[-- Attachment #2: fixpatch --]
[-- Type: text/plain, Size: 2455 bytes --]

#!/bin/sh

src="$1"
dest="$2"
tmp=/tmp/`basename $0`.tmp.$$

if [ -z "$dest" ]; then
	dest="/dev/fd/1"
elif [ "$src" == "$dest" ]; then
	cp "$src" "$src.orig" || exit 1
fi

awk '
function dump_hunk()
{
	if (!hunk) {
		return;
	}

	if (!hsrc_line)
		foffset = 1;

#	if (hsuffix !~ /\n$/) {
#		hsuffix = hsuffix "\n";
#	}
	printf("@@ -%d,%d +%d,%d %s\n", -hsrc_line, hsrc_count,
		-hsrc_line + foffset, hdest_count, hsuffix);

	if (hunchanged + hdel + hadd != hlines) {
		fflush();
		printf("%s %d: Warning: Unchanged %d deleted %d added %d != lines %d\n",
			FILENAME, FNR, hunchanged, hdel, hadd, hlines) > "/dev/stderr";
	}

	if (hunchanged + hdel != hsrc_count) {
		fflush();
		printf("%s %d: Warning: Unchanged %d deleted %d != hsrc_count %d\n",
			FILENAME, FNR, hunchanged, hdel, hsrc_count) > "/dev/stderr";
	}

	if (hunchanged + hadd != hdest_count) {
		fflush();
		printf("%s %d: Warning: Unchanged %d added %d != hdest_count %d\n",
			FILENAME, FNR, hunchanged, hadd, hdest_count) > "/dev/stderr";
	}

	for (i = 0; i < hlines; i++) {
		print hline[i];
		delete hline[i];
	}

	foffset += hdest_count - hsrc_count;
	hsrc_line = 0;
	hsrc_count = 0;
	hdest_line = 0;
	hdest_count = 0;
	hsuffix = "";

	hunchanged = hadd = hdel = 0;
	hlines = 0;
	hunk = 0;
}

/^(diff|====)/ {
	dump_hunk();
	print;
	foffset = 0;
	next;
}

/^@@/ {
	dump_hunk();
	n = split($2, a, /,/);
	hsrc_line = a[1]+0;
	hsrc_oldcount = a[2]+0;
	hsrc_count = 0;
	if (n != 2 || hsrc_line > 0 || hsrc_oldcount < 0) {
		fflush();
		printf("%s %d: Error: Corrupt hunk header (n %d hsrc_line %d hsrc_oldcount %d)\n",
			FILENAME, FNR, n, hsrc_line, hsrc_count) > "/dev/stderr";
	}

	n = split($3, a, /,/);
	hdest_line = a[1]+0;
	hdest_oldcount = a[2]+0;
	hdest_count = 0;
	if (n != 2 || hdest_line < 0 || hdest_oldcount < 0) {
		fflush();
		printf("%s %d: Error: Corrupt hunk header (hdest_line %d hdest_oldcount %d)\n",
			FILENAME, FNR, hdest_line, hdest_oldcount) > "/dev/stderr";
	}

	hsuffix = $0;
	sub(/^@@.*@@/, "@@", hsuffix);

	hlines = 0;
	hunk = 1;
	next;
}

(!hunk) || /^#/ {
	print;
	next;
}

/^-/ {
	hdel++;
	hsrc_count++;
}

/^+/ {
	hadd++;
	hdest_count++;
}

/^ / {
	hunchanged++;
	hsrc_count++;
	hdest_count++;
}

/^$/ || /^[^-+ ]/ {
	fflush();
	printf("%s %d: Warning: badly formatted line\n",
		FILENAME, FNR) > "/dev/stderr";
	exit(255);
##	hadd++;
##	hdest_count++;
#	next;
}

{
	hline[hlines++] = $0;
}

END {
	dump_hunk()
}
' "$src" > "$tmp" && mv "$tmp" "$dest"

Re: How to split a patch

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Yes, and you can do the same with "git add -i".  These tools are
> not quite nice, as they encourage a wrong workflow of committing
> what you haven't had as a whole in the work tree.  By
> definition, you are making untested commits between your base
> commit (that presumably was tested well) and your final commit
> (that would also be tested well).
> ...
> There is no such tool yet, though.
> 
> The splitting you can do with "rebase -i" instead walks
> forwards.  That also lets you test before you make commits in
> each step.

Having said all that, what I tend to do in practice is something
like this:

 (1) Prepare a perfect state (i.e. what to become the final
     commit in the series) in work tree.  Make a commit.  Let's
     say I was working on 'master' branch.

 (2) Checkout the state before the series.  If I currently have
     a three commit series that I want to split further, that
     would be HEAD~3.  If I just made a huge change without
     making intermediate commits, that would be HEAD~1:

     $ git checkout HEAD~3

 (3) Get the final state in the work tree, to decide what to put
     in and what to omit from the first in the series:

     $ git diff -R master | git apply --index

 (4) Use "git-add -i" to prepare what I want in the first
     commit.

     $ git add -i

 (5) When the index is in a good shape, drop all other changes
     from the work tree.  This is what I thought would be a good
     first commit:

     $ git diff -R | git apply

 (6) Test it.  Make corrections.  Perfect it.  Then commit.

     $ git commit

 (7) Go back to step (3), until the commit that I make in step
     (6) matches 'master'.

 (8) Reset 'master' with HEAD:

    $ git branch -f master HEAD
    $ git checkout master

Re: How to split a patch

[Wincent Colaiuta]

El 28/1/2008, a las 11:27, Junio C Hamano escribió:

> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>
>> "Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:
>>
>>> Yes it helps but I still wonder whether thereis a "simpler" way to  
>>> achive that.
>>> Is it possible to split a patch selecting the hunk in git gui or any
>>> other graphical
>>> tool?
>>
>> You can apply the patch without commiting it, and them make several
>> partial commits, by right-click "stage hunk for commit" in git-gui.
>
> Yes, and you can do the same with "git add -i".  These tools are
> not quite nice, as they encourage a wrong workflow of committing
> what you haven't had as a whole in the work tree.  By
> definition, you are making untested commits between your base
> commit (that presumably was tested well) and your final commit
> (that would also be tested well).

And as another alternative, seeing as Git is so easy to script, it is  
trivial to make a script that checks out all the commits in a series  
and runs the test suite against them. This of course assumes that you  
have a test suite!

For example, this is the script that I use on a topic branch before  
submitting the corresponding patch series. It's one of the first  
scripts for Git that I wrote, so it is probably a bit naïve, but it  
works in the basic case. Basically bails if "make clean && make test"  
doesn't work for any commit in the series.

I've used this on the series that I've sent to the Git mailing list,  
but you can use it internally on your local projects too before  
merging topic branches.

Cheers,
Wincent

#!/bin/sh -e
#
# check-series.sh
# Check all commits in a topic branch before submission
#
# Created by Wincent Colaiuta on 23 November 2007.
# Copyright 2007 Wincent Colaiuta.

#
# Functions
#

die () {
   echo "$1"
   exit 1
}

#
# Main
#

git diff --quiet || die "Unstaged changes; won't start"
git diff --cached --quiet || die "Staged changes; won't start"

# parse HEAD (something like "ref: refs/heads/my_local_branch") to get  
remote and merge
GIT_DIR=$(git rev-parse --git-dir)
TOPIC="$GIT_DIR/HEAD"
test -f "$TOPIC" || die "No HEAD found at '$TOPIC'"
BRANCH_REF=$(< "$TOPIC")
if [ "${BRANCH_REF%%: */*}" != "ref" ]; then
   die "expected HEAD '$BRANCH_REF' to be of the form 'ref: .../...'"
fi
BRANCH=${BRANCH_REF##*/}
REMOTE=$(git config branch.$BRANCH.remote) || die "failed to get  
branch.$BRANCH.remote"
MERGE=$(git config branch.$BRANCH.merge) || die "failed to get branch. 
$BRANCH.merge"
MERGE=${MERGE##*/}

# remember which branch we were on prior to starting
trap 'git checkout $BRANCH' exit

# will check all commits in topic branch not present in origin
echo "On branch $BRANCH"
echo "Commits to be checked:"
git rev-list --pretty=oneline HEAD ^$REMOTE/$MERGE
for COMMIT in $(git rev-list HEAD ^$REMOTE/$MERGE); do
   echo "Checking $COMMIT"
   git checkout $COMMIT
   make clean
   make test || die "Commit $COMMIT is bad"
done

echo "All revisions passed!"

Re: How to split a patch

[Mike Hommey]

On Mon, Jan 28, 2008 at 02:40:38AM -0800, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Yes, and you can do the same with "git add -i".  These tools are
> > not quite nice, as they encourage a wrong workflow of committing
> > what you haven't had as a whole in the work tree.  By
> > definition, you are making untested commits between your base
> > commit (that presumably was tested well) and your final commit
> > (that would also be tested well).
> > ...
> > There is no such tool yet, though.
> > 
> > The splitting you can do with "rebase -i" instead walks
> > forwards.  That also lets you test before you make commits in
> > each step.
> 
> Having said all that, what I tend to do in practice is something
> like this:
(...)

I just git commit the hunks, then git stash the rest of the changes, and
if I need some more changes to make the hunks only work, I commit --amend.

Once the commit is stable, I stash apply, resolving conflicts if any arose.

Mike

Re: How to split a patch

[Johannes Schindelin]

Hi,

On Mon, 28 Jan 2008, Junio C Hamano wrote:

> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
> 
> > "Paolo Ciarrocchi" <paolo.ciarrocchi@gmail.com> writes:
> >
> >> Yes it helps but I still wonder whether thereis a "simpler" way to 
> >> achive that. Is it possible to split a patch selecting the hunk in 
> >> git gui or any other graphical tool?
> >
> > You can apply the patch without commiting it, and them make several 
> > partial commits, by right-click "stage hunk for commit" in git-gui.
> 
> Yes, and you can do the same with "git add -i".  These tools are
> not quite nice, as they encourage a wrong workflow of committing
> what you haven't had as a whole in the work tree.

FWIW I have a preliminary patch for "git stash apply -i" in my personal 
next branch.  It does not quite work yet, and it is stalled, of course, 
since I am working on master until 1.5.4 is released.

Ciao,
Dscho