Volume of commits

Volume of commits

[Fredrik Tolf]

Hi List,

I was wondering -- whenever I see Git patches committed to projects
like the Linux kernel or Git itself, the commits always seems to be
committing rather large changes and be rather well-defined in terms of
what they change.

When I develop for myself, I usually commit incrementally quite a
bit, if for no other reason because Git won't let me switch between
branches if I don't commit first. I usually try to keep my commits
well-defined, but I don't manage to get anywhere close to what I see
when I look at the history of Linux or Git.

So what I'm wondering is how you people manage to do this? Do you
actually always commit changes this way (and, in that case, how do you
switch between branches)? Or do you somehow aggregate the smaller
commits into larger patches and recommit them? Or is there some third
possibility that I'm missing?

Fredrik Tolf

Re: Volume of commits

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 576 bytes --]

Hello,

Na Thu, Jul 12, 2007 at 03:16:47PM +0200, Fredrik Tolf <fredrik@dolda2000.com> pisal(a):
> So what I'm wondering is how you people manage to do this? Do you
> actually always commit changes this way (and, in that case, how do you
> switch between branches)? Or do you somehow aggregate the smaller
> commits into larger patches and recommit them? Or is there some third
> possibility that I'm missing?

you can cherry-pick the relevan patches to a separate branch and commit
then at once (cherry-pick -n), or can merge --squash to archive
something similar

- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Volume of commits

[Karl Hasselström]

On 2007-07-12 15:16:47 +0200, Fredrik Tolf wrote:

> When I develop for myself, I usually commit incrementally quite a
> bit, if for no other reason because Git won't let me switch between
> branches if I don't commit first.

You can switch branches even if you have local modifications if you
use the -m flag to git-checkout.

> I usually try to keep my commits well-defined, but I don't manage to
> get anywhere close to what I see when I look at the history of Linux
> or Git.
>
> So what I'm wondering is how you people manage to do this? Do you
> actually always commit changes this way (and, in that case, how do
> you switch between branches)? Or do you somehow aggregate the
> smaller commits into larger patches and recommit them? Or is there
> some third possibility that I'm missing?

When it's time to post patches for review on the mailing list, people
clean up the history so that it'll be easier for the reviewers to read
(and to make the permanent history easier to read in case the patches
are accepted). Of course, with practice one can write clean patches to
begin with, at least for simpler changes, but the conceptual workflow
is along the lines of:

  1. Do the changes, and make sure everything works.

  2. Rewrite your changes as a series of easy-to-read and mostly
     independent patches, and make sure that everything compiles and
     just generally makes sense at all intermediate steps. The
     maintainer might very well accept some of your patches but not
     all!

  3. Repeat until you're satisfied with the result.

The rationale for spending time making history legible is the same as
for making the end result code legible: stuff is written once by one
person, and read many times by many people.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Volume of commits

[Johannes Schindelin]

Hi,

On Thu, 12 Jul 2007, VMiklos wrote:

> Na Thu, Jul 12, 2007 at 03:16:47PM +0200, Fredrik Tolf <fredrik@dolda2000.com> pisal(a):
> > So what I'm wondering is how you people manage to do this? Do you 
> > actually always commit changes this way (and, in that case, how do you 
> > switch between branches)? Or do you somehow aggregate the smaller 
> > commits into larger patches and recommit them? Or is there some third 
> > possibility that I'm missing?
> 
> you can cherry-pick the relevan patches to a separate branch and commit 
> then at once (cherry-pick -n), or can merge --squash to archive 
> something similar

What I do these days is committing early, and often, and then rearrange 
with "git rebase -i".  Beware: this is a 1.5.3 feature!  But IMHO it is 
really something you can look forward to.

A little diagram hopefully explains what you can do with it:

- upstream
 \
   A - B - C - D - E - F - G - HEAD with a messy history

In this case, "messy history" means that there are tiny patches which are 
often in the wrong order, or should be squashed into one commit.  "git 
rebase -i upstream" presents you with the list of A - HEAD, and you can 
reorder the patches.  If you want to, you can combine ("squash") some 
into one commit, or you can skip it, by removing the corresponding line.

The result can look like this:

- upstream
 \
   B+C+F - A - G+E

Note that D is missing, which can be desirable, for example when you made 
a commit only introducing lots and lots of debug output.  See, nobody has 
to know what you did.  The end result will look elegant.

This demonstration of why distributed SCM is good ("it lowers the 
embarrasment factor") was brought to you by Git.
 
Ciao,
Dscho

Re: Volume of commits

[Karl Hasselström]

On 2007-07-12 15:16:47 +0200, Fredrik Tolf wrote:

> Or do you somehow aggregate the smaller commits into larger patches
> and recommit them? Or is there some third possibility that I'm
> missing?

Aggregating commits and recommitting is easy, so that's a common tool,
I'd say. But it's equally possible to take a large commit and pick it
apart, for example by editing the patch by hand and reapplying it, or
by using the hunk selection feature of git-gui.

For example, if you have just committed several changes as one big
commit, you can do

  $ git reset HEAD^

to undo the commit but retain the changes in your working tree, and
then use git-gui to select a subset of the changes and commit them,
then select another subset and commit that, and so on.

If you need to edit a commit that isn't HEAD, you can use git-reset to
go back to the commit you want to edit, do the editing, and then use
git-rebase to reapply the other commits on top of the changed commit.

In general, there are a thousand ways to use git to rewrite history,
either "by hand" or by using tools such as StGIT. (StGIT is what I
personally use most of the time. I find it a good tool for producing
readable patch series.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Volume of commits

[Fredrik Tolf]

Karl Hasselström <kha@treskal.com> writes:

> On 2007-07-12 15:16:47 +0200, Fredrik Tolf wrote:
>
>> Or do you somehow aggregate the smaller commits into larger patches
>> and recommit them? Or is there some third possibility that I'm
>> missing?
>
> Aggregating commits and recommitting is easy, so that's a common tool,
> I'd say. But it's equally possible to take a large commit and pick it
> apart, for example by editing the patch by hand and reapplying it, or
> by using the hunk selection feature of git-gui.

I see. Initially, it sounds like a lot of work, but the more I think
about it, the more I realize that it probably isn't that bad.

> [...]
> If you need to edit a commit that isn't HEAD, you can use git-reset to
> go back to the commit you want to edit, do the editing, and then use
> git-rebase to reapply the other commits on top of the changed commit.

git-rebase is one of those tools I haven't been looking at so far (I'm
still rather new to Git), so I should probably read through its
manpage.

> In general, there are a thousand ways to use git to rewrite history,
> either "by hand" or by using tools such as StGIT. (StGIT is what I
> personally use most of the time. I find it a good tool for producing
> readable patch series.)

I hadn't heard of StGIT, but it looks interesting.

Thanks for all the suggestions! I'll be needing some time to look them
through. :)

Fredrik Tolf

Re: Volume of commits

[Joshua N Pritikin]

On Thu, Jul 12, 2007 at 04:51:44PM +0200, Fredrik Tolf wrote:
> git-rebase is one of those tools I haven't been looking at so far (I'm
> still rather new to Git), so I should probably read through its
> manpage.

I was in your shoes but I recently learned rebase. If you want to edit a 
commit then find the SHA-1 in git log. Then assuming you are on master:

1. git checkout -b tmp SHA-1
2. git commit --amend
3. git checkout master
4. git rebase --onto tmp SHA-1

Both times, use the same SHA-1.

Re: Volume of commits

[Karl Hasselström]

On 2007-07-12 16:51:44 +0200, Fredrik Tolf wrote:

> I see. Initially, it sounds like a lot of work, but the more I think
> about it, the more I realize that it probably isn't that bad.

You're right that it isn't that much work, but it still is work. The
point is that just like investing time in making code nice and
readable, investing time in making the history nice and readable is a
net win. Others can follow what you do, which makes reviewing the
changes much cheaper, and you can look back six months and actually
understand what you were thinking back then.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Volume of commits

[Linus Torvalds]

On Thu, 12 Jul 2007, Fredrik Tolf wrote:
> 
> I was wondering -- whenever I see Git patches committed to projects
> like the Linux kernel or Git itself, the commits always seems to be
> committing rather large changes and be rather well-defined in terms of
> what they change.

Well, people already talked about how to do it using git, but I'd like to 
point out that one of the reasons you see this kind of pattern is that 
when you push patches around by email for commentary (which is what both 
git and the kernel do a _lot_), that very much inherently encourages a 
setup where each patch makes sense on its own, and I think it causes 
*much* better code cleanliness behaviour.

When people make changes that they know will be shown as patches, they 
just tend to make more sure that the changes make logical sense. Part of 
it is cleanups after-the-fact, but part of it is that when you get used to 
doing it, after a while you start _thinking_ in those terms when you make 
the changes, and that's also a good thing!

When I do any bigger changes, I usually tend to commit at points where it 
starts working, but then before I actually would send it to Junio, I'd go 
back and clean up the series (by creating a new branch, and 
cherry-picking, and doing diffs between the branch and applying the parts 
I want to).

But I do that only for stuff where I can't see the end result as a clean 
series of steps from the beginning. If I know exactly what I'm doing, I'll 
just do it the clean way from the get-go, and I don't need to clean up the 
series after-the-fact.

(Most of the time I actually try to get it right the first time. It's 
actually become a challenge to me to notice when some change needs a 
cleanup first in order to make the later changes much easier, so I really 
*like* trying to actually do the actual development in a logical order: 
first re-organize the code, and verify that the re-organized code works 
identically to the old one, then commit that, then start actually working 
on the new feature with the now cleaner code-base).

And no, I didn't start out programming that way. But when you get used to 
looking at changes as a nice series of independent commits in emails, you 
really start _working_ that way yourself. And I'm 100% convinced that it 
actually makes you a better programmer too.

> When I develop for myself, I usually commit incrementally quite a
> bit, if for no other reason because Git won't let me switch between
> branches if I don't commit first. I usually try to keep my commits
> well-defined, but I don't manage to get anywhere close to what I see
> when I look at the history of Linux or Git.

The stuff you see in git or the kernel has mostly been discussed as 
emails, or at least been sent out that way (and if it didn't cause any 
discussion, it was probably "obviously clean and correct"). And that whole 
flow really *does* end up causing people to write cleaner patches.

It's absolutely worth emulating it, but in some respect, if it's just your 
own project, I suspect you'll just never have the incentive to have your 
history be quite as clean as the kernel/git development itself has.

		Linus

Re: Volume of commits

[Jakub Narebski]

Fredrik Tolf wrote:


> When I develop for myself, I usually commit incrementally quite a
> bit, if for no other reason because Git won't let me switch between
> branches if I don't commit first. I usually try to keep my commits
> well-defined, but I don't manage to get anywhere close to what I see
> when I look at the history of Linux or Git.

First, if you commit only to switch branches you can always instead
of adding commit on top of whis WIP commit, just --amend it.

Second, there is git-stash just created for saving state to go back
to it.

Third, I guess that the neat patch series are result of reworking
existing series using tools like StGIT (which I use and find very
nice to work with, going and correcting back and forth between patches
in series), or guilt (similar to StGIT), or git-gui, or new interactive mode
of git-rebase, or git-cherry-pick...

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Volume of commits

[Sven Verdoolaege]

On Thu, Jul 12, 2007 at 02:59:53PM +0100, Johannes Schindelin wrote:
> In this case, "messy history" means that there are tiny patches which are 
> often in the wrong order, or should be squashed into one commit.  "git 
> rebase -i upstream" presents you with the list of A - HEAD, and you can 
> reorder the patches.  If you want to, you can combine ("squash") some 
> into one commit, or you can skip it, by removing the corresponding line.

If I squash a whole series of commits, how do I prevent git-rebase -i
from firing up an editor after every single commit in the series?

Also, if I do the following:

bash-3.00$ git init
Initialized empty Git repository in .git/
bash-3.00$ for i in a b c; do touch $i; git add $i; git commit -m $i -a; done
Created initial commit 19a8485: a
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 a
Created commit 4a00f85: b
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 b
Created commit defe3b5: c
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 c
bash-3.00$ git rebase -i  HEAD~2

and then replace the second "pick" by "squash", then I get presented
a commit message that contains the commit message of "c" twice and
after the rebase there are still three commits in the history.
This is with git version 1.5.3.rc1.10.gae1ae
(on top of v1.5.3-rc1-4-gaf83bed).

skimo

Re: Volume of commits

[Alex Riesen]

On 7/13/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> If I squash a whole series of commits, how do I prevent git-rebase -i
> from firing up an editor after every single commit in the series?

It is started only for squashed commits. But you can set VISUAL to true or ":".

> a commit message that contains the commit message of "c" twice and
> after the rebase there are still three commits in the history.

Known, fixed (but not yet approved) in
[PATCH] Fix git-rebase -i to allow squashing of fast-forwardable commits

[PATCH] rebase -i: call editor just once for a multi-squash

[Johannes Schindelin]

Sometimes you want to squash more than two commits.  Before this patch,
the editor was fired up for each squash command.  Now the editor is
started only with the last squash command.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---

	On Fri, 13 Jul 2007, Sven Verdoolaege wrote:

	> If I squash a whole series of commits, how do I prevent 
	> git-rebase -i from firing up an editor after every single commit 
	> in the series?

	By applying this patch ;-)

 git-rebase--interactive.sh    |   56 +++++++++++++++++++++++++++++++++-------
 t/t3404-rebase-interactive.sh |    9 ++++++
 2 files changed, 55 insertions(+), 10 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index a2d4d09..579a45e 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -19,6 +19,8 @@ require_work_tree
 DOTEST="$GIT_DIR/.dotest-merge"
 TODO="$DOTEST"/todo
 DONE="$DOTEST"/done
+MSG="$DOTEST"/message
+SQUASH_MSG="$DOTEST"/message-squash
 REWRITTEN="$DOTEST"/rewritten
 PRESERVE_MERGES=
 STRATEGY=
@@ -158,6 +160,38 @@ pick_one_preserving_merges () {
 	esac
 }
 
+nth_string () {
+	case "$1" in
+	*1[0-9]|*[04-9]) echo "$1"th;;
+	*1) echo "$1"st;;
+	*2) echo "$1"nd;;
+	*3) echo "$1"rd;;
+	esac
+}
+
+make_squash_message () {
+	if [ -f "$SQUASH_MSG" ]; then
+		COUNT=$(($(sed -n "s/^# This is [^0-9]*\([0-9]\+\).*/\1/p" \
+			< "$SQUASH_MSG" | tail -n 1)+1))
+		echo "# This is a combination of $COUNT commits."
+		sed -n "2,\$p" < "$SQUASH_MSG"
+	else
+		COUNT=2
+		echo "# This is a combination of two commits."
+		echo "# The first commit's message is:"
+		echo
+		git cat-file commit HEAD | sed -e '1,/^$/d'
+		echo
+	fi
+	echo "# This is the $(nth_string $COUNT) commit message:"
+	echo
+	git cat-file commit $1 | sed -e '1,/^$/d'
+}
+
+peek_next_command () {
+	sed -n "1s/ .*$//p" < "$TODO"
+}
+
 do_next () {
 	test -f "$DOTEST"/message && rm "$DOTEST"/message
 	test -f "$DOTEST"/author-script && rm "$DOTEST"/author-script
@@ -194,17 +228,19 @@ do_next () {
 			die "Cannot 'squash' without a previous commit"
 
 		mark_action_done
-		MSG="$DOTEST"/message
-		echo "# This is a combination of two commits." > "$MSG"
-		echo "# The first commit's message is:" >> "$MSG"
-		echo >> "$MSG"
-		git cat-file commit HEAD | sed -e '1,/^$/d' >> "$MSG"
-		echo >> "$MSG"
+		make_squash_message $sha1 > "$MSG"
+		case "$(peek_next_command)" in
+		squash)
+			EDIT_COMMIT=
+			cp "$MSG" "$SQUASH_MSG"
+		;;
+		*)
+			EDIT_COMMIT=-e
+			test -f "$SQUASH_MSG" && rm "$SQUASH_MSG"
+		esac
+
 		failed=f
 		pick_one -n $sha1 || failed=t
-		echo "# And this is the 2nd commit message:" >> "$MSG"
-		echo >> "$MSG"
-		git cat-file commit $sha1 | sed -e '1,/^$/d' >> "$MSG"
 		git reset --soft HEAD^
 		author_script=$(get_author_ident_from_commit $sha1)
 		echo "$author_script" > "$DOTEST"/author-script
@@ -213,7 +249,7 @@ do_next () {
 			# This is like --amend, but with a different message
 			eval "$author_script"
 			export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_AUTHOR_DATE
-			git commit -F "$MSG" -e
+			git commit -F "$MSG" $EDIT_COMMIT
 			;;
 		t)
 			cp "$MSG" "$GIT_DIR"/MERGE_MSG
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 43a6675..8206436 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -65,6 +65,7 @@ cat > fake-editor.sh << EOF
 #!/bin/sh
 test "\$1" = .git/COMMIT_EDITMSG && {
 	test -z "\$FAKE_COMMIT_MESSAGE" || echo "\$FAKE_COMMIT_MESSAGE" > "\$1"
+	test -z "\$FAKE_COMMIT_AMEND" || echo "\$FAKE_COMMIT_AMEND" >> "\$1"
 	exit
 }
 test -z "\$FAKE_LINES" && exit
@@ -212,4 +213,12 @@ test_expect_success 'verbose flag is heeded, even after --continue' '
 	grep "^ file1 |    2 +-$" output
 '
 
+test_expect_success 'multi-squash only fires up editor once' '
+	base=$(git rev-parse HEAD~4) &&
+	FAKE_COMMIT_AMEND="ONCE" FAKE_LINES="1 squash 2 squash 3 squash 4" \
+		git rebase -i $base &&
+	test $base = $(git rev-parse HEAD^) &&
+	test 1 = $(git show | grep ONCE | wc -l)
+'
+
 test_done
-- 
1.5.3.rc1.16.g9d6f-dirty

Re: Volume of commits

[Johannes Schindelin]

Hi,

On Fri, 13 Jul 2007, Sven Verdoolaege wrote:

> [...] if I do the following:
> 
> bash-3.00$ git init
> Initialized empty Git repository in .git/
> bash-3.00$ for i in a b c; do touch $i; git add $i; git commit -m $i -a; done
> Created initial commit 19a8485: a
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 a
> Created commit 4a00f85: b
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 b
> Created commit defe3b5: c
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 c
> bash-3.00$ git rebase -i  HEAD~2
> 
> and then replace the second "pick" by "squash", then I get presented
> a commit message that contains the commit message of "c" twice and
> after the rebase there are still three commits in the history.
> This is with git version 1.5.3.rc1.10.gae1ae
> (on top of v1.5.3-rc1-4-gaf83bed).

It no longer reproduces, which probably means that I inadvertently fixed 
the bug ;-)

Ciao,
Dscho