[PATCH] git-subtree: Add prune mode

[PATCH] git-subtree: Add prune mode

[Santi Béjar]

Add prune mode (flag --prune) with the following properties:

* The history must be as clean as possible
* The directory content must be equal to the external module,
  at least when you add/update it[b]
* The subproject should be able to switch back and forth between
  different versions.

[b] A consequence of this is that it loses all changes
    made in the subtree. If they are important you have to extract
    them, apply them and add the subproject back.

As all the history is lost and you never merge commits
'split' is not necessary, but it is basically:

$ git filter-branch --subdirectory-filter $prefix

And normally you will just want to extract the changes with format-patch
and apply them with:

git am -p2

Signed-off-by: Santi Béjar <santi@agolina.net>
---
Hi *,

   here it is a the same patch I send some days before but with documentation.

Santi

 git-subtree.sh  |   29 +++++++++++++++++++++++++++--
 git-subtree.txt |   21 ++++++++++++++++++++-
 2 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/git-subtree.sh b/git-subtree.sh
index 781eef3..766b68b 100755
--- a/git-subtree.sh
+++ b/git-subtree.sh
@@ -27,6 +27,7 @@ onto=         try connecting new tree to an existing one
 rejoin        merge the new branch back into HEAD
  options for 'add', 'merge', 'pull' and 'push'
 squash        merge subtree changes as a single commit
+prune         prune history
 "
 eval $(echo "$OPTS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?)
 
@@ -44,6 +45,7 @@ rejoin=
 ignore_joins=
 annotate=
 squash=
+prune=
 message=
 
 debug()
@@ -92,6 +94,8 @@ while [ $# -gt 0 ]; do
 		--no-ignore-joins) ignore_joins= ;;
 		--squash) squash=1 ;;
 		--no-squash) squash= ;;
+		--prune) prune=1;;
+		--no-prune) prune=;;
 		--) break ;;
 		*) die "Unexpected option: $opt" ;;
 	esac
@@ -110,12 +114,16 @@ if [ -z "$prefix" ]; then
 fi
 
 case "$command" in
-	add) [ -e "$prefix" ] && 
+	add) [ -e "$prefix" -a -z "$prune" ] &&
 		die "prefix '$prefix' already exists." ;;
 	*)   [ -e "$prefix" ] || 
 		die "'$prefix' does not exist; use 'git subtree add'" ;;
 esac
 
+if [ -n "$squash" -a -n "$prune" ] ; then
+	die "Flags --prune and --squash are incompatible"
+fi
+
 dir="$(dirname "$prefix/.")"
 
 if [ "$command" != "pull" -a "$command" != "add" -a "$command" != "push" ]; then
@@ -359,6 +367,17 @@ squash_msg()
 	echo "git-subtree-split: $newsub"
 }
 
+prune_msg()
+{
+	dir="$1"
+	newsub="$2"
+	
+	git show -s --pretty="tformat:Subtree '$dir/': %h %s" $newsub
+	echo
+	echo "git-subtree-dir: $dir"
+	echo "git-subtree-split: $newsub"
+}
+
 toptree_for_commit()
 {
 	commit="$1"
@@ -464,7 +483,7 @@ ensure_clean()
 
 cmd_add()
 {
-	if [ -e "$dir" ]; then
+	if [ -e "$dir" -a -z "$prune" ]; then
 		die "'$dir' already exists.  Cannot add."
 	fi
 
@@ -498,6 +517,9 @@ cmd_add_commit()
 	rev="$1"
 	
 	debug "Adding $dir as '$rev'..."
+	if [ -d "$dir" ]; then
+	    git rm -r -q $dir
+	fi
 	git read-tree --prefix="$dir" $rev || exit $?
 	git checkout -- "$dir" || exit $?
 	tree=$(git write-tree) || exit $?
@@ -513,6 +535,9 @@ cmd_add_commit()
 		rev=$(new_squash_commit "" "" "$rev") || exit $?
 		commit=$(add_squashed_msg "$rev" "$dir" |
 			 git commit-tree $tree $headp -p "$rev") || exit $?
+	elif [ -n "$prune" ]; then
+		commit=$(prune_msg "$dir" "$rev" |
+			 git commit-tree $tree -p $headrev) || exit $?
 	else
 		commit=$(add_msg "$dir" "$headrev" "$rev" |
 			 git commit-tree $tree $headp -p "$rev") || exit $?
diff --git a/git-subtree.txt b/git-subtree.txt
index dbcba31..5a91147 100644
--- a/git-subtree.txt
+++ b/git-subtree.txt
@@ -66,7 +66,8 @@ add::
 	A new commit is created	automatically, joining the imported
 	project's history with your own.  With '--squash', imports
 	only a single commit from the subproject, rather than its
-	entire history.
+	entire history. With '--prune', imports only the contents of
+	the commit from the subproject without any history.
 
 merge::
 	Merge recent changes up to <commit> into the <prefix>
@@ -165,6 +166,24 @@ OPTIONS FOR add, merge, AND pull
 	local repository remain intact and can be later split
 	and send upstream to the subproject.
 
+--prune::
+	Instead of merging the history (full or squashed) from the
+	subtree project, produce only a single commit that
+	reproduce the exact content in the preffix as in the
+	subtree.
+
+	It has similar features as the --squash option, namely
+	reduces the clutter (althougth --prune reduce it even
+	more), helps avoiding problems when the same subproject is
+	include multiple time and can switch back and forth
+	between different version of a subtree.
+
+	The main difference is that with --prune the content of
+	the prefix always matches the content of the subproject,
+	while with --squash it merges you changes with the changes
+	from the subtree. If you want to keep your changes you
+	have to extract them, apply them in the external
+	repository and add the subproject back.
 
 OPTIONS FOR split
 -----------------
-- 
1.7.1.765.gb5868

Re: [PATCH] git-subtree: Add prune mode

[Avery Pennarun]

On Sun, Aug 1, 2010 at 5:25 AM, Santi Béjar <santi@agolina.net> wrote:
> Add prune mode (flag --prune) with the following properties:
>
> * The history must be as clean as possible
> * The directory content must be equal to the external module,
>  at least when you add/update it[b]
> * The subproject should be able to switch back and forth between
>  different versions.
>
> [b] A consequence of this is that it loses all changes
>    made in the subtree. If they are important you have to extract
>    them, apply them and add the subproject back.

I think I started to reply to this before but I can't quite remember
what happened.  Anyway, I have several concerns with this patch:

- calling it "prune" is pretty incorrect.  It doesn't remove anything
from your history.  It silently loses patches from your tree, but
that's not "pruning" really.  I suggest "--squash
--discard-local-changes" or something.  ie. it's a variant of squash,
and it throws things away.  We want both of those to be clear.

- I'm not convinced this concept is even a good idea.  Who on earth
wants to silently lose changes?  Default --squash behaviour is to
merge the old branch with the new branch.  If you want to throw stuff
away, that should probably be an entirely separate operation from the
merging.  ie. "git subtree discard-local-changes --prefix=whatever" to
create a new patch that undoes all the local changes to this branch;
then "git subtree merge --squash" to update to a different upstream.

- In what sense is the history from this any "cleaner" (or any
different) from --squash?  It seems like, in fact, the history will be
a *lie* since it talks about merging but actually reverts changes.

> As all the history is lost and you never merge commits
> 'split' is not necessary, but it is basically:
>
> $ git filter-branch --subdirectory-filter $prefix

Are you saying that 'git subtree split' doesn't work after the
discard-local-changes operation?  If so, we should fix that, I think.
Otherwise it's very confusing.

> +prune_msg()
> +{
> +       dir="$1"
> +       newsub="$2"
> +
> +       git show -s --pretty="tformat:Subtree '$dir/': %h %s" $newsub
> +       echo
> +       echo "git-subtree-dir: $dir"
> +       echo "git-subtree-split: $newsub"
> +}
> +

Hmm.  I'm rather concerned about this one.  What's the top line of the
commit message?  add_msg, add_squashed_msg, rejoin_msg, and squash_msg
are all much clearer than this.  It also seems that it doesn't honour
the -m flag here.

>  cmd_add()
>  {
> -       if [ -e "$dir" ]; then
> +       if [ -e "$dir" -a -z "$prune" ]; then
>                die "'$dir' already exists.  Cannot add."
>        fi

This is the 'add' command.  I'm not sure it should have special
behaviour with prune.

(Arguably 'add' should let you optionally overwrite an existing
directory if it already exists.  But if so, I don't think this should
only happen with --prune; it should probably be a special --overwrite
or --force option.)

>        debug "Adding $dir as '$rev'..."
> +       if [ -d "$dir" ]; then
> +           git rm -r -q $dir
> +       fi
>        git read-tree --prefix="$dir" $rev || exit $?

Isn't there some plumbing command we can use instead of 'git rm'?  I
don't really know what.  Does anyone have any suggestions?

>        A new commit is created automatically, joining the imported
>        project's history with your own.  With '--squash', imports
>        only a single commit from the subproject, rather than its
> -       entire history.
> +       entire history. With '--prune', imports only the contents of
> +       the commit from the subproject without any history.

This is unclear.  "Without any history" isn't really accurate, since a
new commit message is generated; actually the history importing is
precisely the same as what happens with --squash.

> +--prune::
> +       Instead of merging the history (full or squashed) from the
> +       subtree project, produce only a single commit that
> +       reproduce the exact content in the preffix as in the
> +       subtree.

s/preffix/prefix/

> +       It has similar features as the --squash option, namely
> +       reduces the clutter (althougth --prune reduce it even
> +       more), helps avoiding problems when the same subproject is
> +       include multiple time and can switch back and forth
> +       between different version of a subtree.

s/althougth/although/
s/include multiple/included multiple/

I don't understand how --prune supposedly "reduces it even more" than
--squash.  What clutter?  How does it reduce it *more*?  How does it
avoid problems when the same subproject is included multiple times?
What problems are those?

You can switch back and forth between different versions of a subtree,
but this is true of --squash as well - that's why I think this
operation should be a subcommand of --squash instead.

> +       The main difference is that with --prune the content of
> +       the prefix always matches the content of the subproject,
> +       while with --squash it merges you changes with the changes
> +       from the subtree. If you want to keep your changes you
> +       have to extract them, apply them in the external
> +       repository and add the subproject back.

s/you changes/your changes/

This last part is especially strange: if you want to keep your own
changes, why do you want to use --prune at all?  --squash does that
automatically.

Have fun,

Avery

Re: [PATCH] git-subtree: Add prune mode

[Santi Béjar]

On Mon, Aug 2, 2010 at 4:54 AM, Avery Pennarun <apenwarr@gmail.com> wrote:
> On Sun, Aug 1, 2010 at 5:25 AM, Santi Béjar <santi@agolina.net> wrote:
>> Add prune mode (flag --prune) with the following properties:
>>
>> * The history must be as clean as possible
>> * The directory content must be equal to the external module,
>>  at least when you add/update it[b]
>> * The subproject should be able to switch back and forth between
>>  different versions.
>>
>> [b] A consequence of this is that it loses all changes
>>    made in the subtree. If they are important you have to extract
>>    them, apply them and add the subproject back.
>
> I think I started to reply to this before but I can't quite remember
> what happened.  Anyway, I have several concerns with this patch:
>
> - calling it "prune" is pretty incorrect.  It doesn't remove anything
> from your history.

It removes the history of the subproject.
And --squash does not squash your history. We are talking about the
subproject, not the superproject.

>  It silently loses patches from your tree, but
> that's not "pruning" really.  I suggest "--squash
> --discard-local-changes" or something.  ie. it's a variant of squash,
> and it throws things away.  We want both of those to be clear.

It's not a variant of squash, it is completely different, but it
shares some properties. One tries to keep your local changes, the
other no.

>
> - I'm not convinced this concept is even a good idea.  Who on earth
> wants to silently lose changes?

I want. I very much prefer do all the changes upstream than in the
subproject. I never modify the subtree, so it is not a issue for me,
but I really prefer to know that every time I add the subproject I get
exactly the content as the other project.

> Default --squash behaviour is to
> merge the old branch with the new branch.  If you want to throw stuff
> away, that should probably be an entirely separate operation from the
> merging.  ie. "git subtree discard-local-changes --prefix=whatever" to
> create a new patch that undoes all the local changes to this branch;
> then "git subtree merge --squash" to update to a different upstream.

Maybe it is not clear from the docs/code, but with --prune you always
have to "git subtree add", so it is pretty clear that you discard all
the local changes.

I should add a check that --prune is only compatible with "git subtree add".

> - In what sense is the history from this any "cleaner" (or any
> different) from --squash?

With git-subtree you always have the subtree history (even if it is
squashed). So when you merge a second time the submodule you get the always
the history of the subtree (even with --squash). So you basically always have
at least two branches while examining the history. Compare this
squashed history:

$ git log --graph --oneline
*   bb2dc25 (HEAD, master) Merge commit
'08b917ee90ecfd7b666364fe4ebb92aee5cdd2f7'
|\
| * 08b917e Squashed 'latex/' changes from ea35faf..895916a
* |   9de91f1 Merge commit 'b1b4c36bb8358582a6a20bb500bf98421428e2ca' as 'latex'
|\|
| * b1b4c36 Squashed 'latex/' content from commit ea35faf
* ea35faf Indent, whitespaces,...

with this pruned history:

$ git log --graph --oneline
* 8703aec (HEAD, master) Subtree 'latex/': 895916a Add files subcommand
* a942284 Subtree 'latex/': ea35faf Indent, whitespaces,...
* ea35faf Indent, whitespaces,...

>  It seems like, in fact, the history will be
> a *lie* since it talks about merging but actually reverts changes.

There is *no* merging with --prune, you get a new commit with a single
parent that changes the contents of the subtree in such a way that
reproduce the contents of the subproject.

>> As all the history is lost and you never merge commits
>> 'split' is not necessary, but it is basically:
>>
>> $ git filter-branch --subdirectory-filter $prefix
>
> Are you saying that 'git subtree split' doesn't work after the
> discard-local-changes operation?  If so, we should fix that, I think.
> Otherwise it's very confusing.

I tried but did not succeed.

But what I was trying to say is that I don´t see the necessity of it,
as you never merge the subproject (with history).

>
>> +prune_msg()
>> +{
>> +       dir="$1"
>> +       newsub="$2"
>> +
>> +       git show -s --pretty="tformat:Subtree '$dir/': %h %s" $newsub
>> +       echo
>> +       echo "git-subtree-dir: $dir"
>> +       echo "git-subtree-split: $newsub"
>> +}
>> +
>
> Hmm.  I'm rather concerned about this one.  What's the top line of the
> commit message?  add_msg, add_squashed_msg, rejoin_msg, and squash_msg
> are all much clearer than this.

I see it is very different from all the others. It is:

Subtree '$dir/': $hash $subject

I´ll change it to match what add_msg does:

Add '$dir/' from commit '$latest_new'


>  It also seems that it doesn't honour
> the -m flag here.

I´ll fix it. I took squash_msg as model...

>
>>  cmd_add()
>>  {
>> -       if [ -e "$dir" ]; then
>> +       if [ -e "$dir" -a -z "$prune" ]; then
>>                die "'$dir' already exists.  Cannot add."
>>        fi
>
> This is the 'add' command.  I'm not sure it should have special
> behaviour with prune.

With prune mode you don´t merge new commits but add them.

>
> (Arguably 'add' should let you optionally overwrite an existing
> directory if it already exists.  But if so, I don't think this should
> only happen with --prune; it should probably be a special --overwrite
> or --force option.)

This is sensible, and prune mode should use it when available.

>
>>        debug "Adding $dir as '$rev'..."
>> +       if [ -d "$dir" ]; then
>> +           git rm -r -q $dir
>> +       fi
>>        git read-tree --prefix="$dir" $rev || exit $?
>
> Isn't there some plumbing command we can use instead of 'git rm'?  I
> don't really know what.  Does anyone have any suggestions?

I tried it, but I didn´t found a command to remove from the index and
the working dir at the same time.

And I found "git checkout --" used after the "git read-tree".

>
>>        A new commit is created automatically, joining the imported
>>        project's history with your own.  With '--squash', imports
>>        only a single commit from the subproject, rather than its
>> -       entire history.
>> +       entire history. With '--prune', imports only the contents of
>> +       the commit from the subproject without any history.
>
> This is unclear.  "Without any history" isn't really accurate, since a
> new commit message is generated; actually the history importing is
> precisely the same as what happens with --squash.

No, with --squash you always have a side branch with the history of
the subtree as viewed from the superproject.

With --prune the only history you get is a new commit in the superproject.

See above for a "git log --graph" example of each.

Would it be clearer if "without any history" is gone?

>
>> +--prune::
>> +       Instead of merging the history (full or squashed) from the
>> +       subtree project, produce only a single commit that
>> +       reproduce the exact content in the preffix as in the
>> +       subtree.
>
> s/preffix/prefix/

OK

>
>> +       It has similar features as the --squash option, namely
>> +       reduces the clutter (althougth --prune reduce it even
>> +       more), helps avoiding problems when the same subproject is
>> +       include multiple time and can switch back and forth
>> +       between different version of a subtree.
>
> s/althougth/although/
> s/include multiple/included multiple/
>

OK.

> I don't understand how --prune supposedly "reduces it even more" than
> --squash.  What clutter?  How does it reduce it *more*?

See above for a "git log --graph" example.

>  How does it
> avoid problems when the same subproject is included multiple times?

You just add/have the content in different subdirectories. You don´t
have problems with the history/merging/... because it does keep the
history, does not merge,...

> What problems are those?

I suppose it is about history and merging, the same problems with
plain "git subtree" without --squash.

>
> You can switch back and forth between different versions of a subtree,
> but this is true of --squash as well - that's why I think this
> operation should be a subcommand of --squash instead.

The fact that it shares some properties with --squash does not make it
a subcommand of it. But you cannot merge, you don´t have a side branch
with the subproject "history",...

>
>> +       The main difference is that with --prune the content of
>> +       the prefix always matches the content of the subproject,
>> +       while with --squash it merges you changes with the changes
>> +       from the subtree. If you want to keep your changes you
>> +       have to extract them, apply them in the external
>> +       repository and add the subproject back.
>
> s/you changes/your changes/
>
> This last part is especially strange: if you want to keep your own
> changes, why do you want to use --prune at all?  --squash does that
> automatically.

If you want to keep your own changes you better upstream them, like
you would do with "git submodules".

If the changes are upstream, all the superprojects using the subtree
will benefit from it.

I try to "push" everything (changes, merging,whatever) upstream (or a
forked upstream).

Have fun,
Santi

Re: [PATCH] git-subtree: Add prune mode

[Santi Béjar]

On Mon, Aug 2, 2010 at 12:41 PM, Santi Béjar <santi@agolina.net> wrote:
> On Mon, Aug 2, 2010 at 4:54 AM, Avery Pennarun <apenwarr@gmail.com> wrote:
>> On Sun, Aug 1, 2010 at 5:25 AM, Santi Béjar <santi@agolina.net> wrote:
>>> Add prune mode (flag --prune) with the following properties:
>>>
>>> * The history must be as clean as possible
>>> * The directory content must be equal to the external module,
>>>  at least when you add/update it[b]
>>> * The subproject should be able to switch back and forth between
>>>  different versions.
>>>
>>> [b] A consequence of this is that it loses all changes
>>>    made in the subtree. If they are important you have to extract
>>>    them, apply them and add the subproject back.
>>
>> I think I started to reply to this before but I can't quite remember
>> what happened.  Anyway, I have several concerns with this patch:
>>
>> - calling it "prune" is pretty incorrect.  It doesn't remove anything
>> from your history.
>
> It removes the history of the subproject.
> And --squash does not squash your history. We are talking about the
> subproject, not the superproject.
>
>>  It silently loses patches from your tree, but
>> that's not "pruning" really.  I suggest "--squash
>> --discard-local-changes" or something.  ie. it's a variant of squash,
>> and it throws things away.  We want both of those to be clear.
>
> It's not a variant of squash, it is completely different, but it
> shares some properties. One tries to keep your local changes, the
> other no.

I just wanted to add one thing. For me "--squash
--discard-local-changes" would be something like --squash but with
"git merge -s theirs", so totally different from the --prune mode. I
cannot be a flag to --squash.

I'm open to discuss the name of the flag, but for now I found --prune
the best one. And as it is only used with "git subtree add" I don't
think it should be about "discard local changes".

Have fun,
Santi