[ANNOUNCE] Stacked GIT 0.7

[ANNOUNCE] Stacked GIT 0.7

[Catalin Marinas]

Stacked GIT 0.7 release is available from http://www.procode.org/stgit/

StGIT is a Python application providing similar functionality to Quilt
(i.e. pushing/popping patches to/from a stack) on top of GIT. These
operations are performed using GIT commands and the patches are stored
as GIT commit objects, allowing easy merging of the StGIT patches into
other repositories using standard GIT functionality.

What's new in this release (the full ChangeLog is in the archive):

      * Uses the renamed GIT commands (requires GIT >= 0.99.7) 
      * 'pick' command to cherry-pick a commit object as an StGIT patch
        using a three-way merge. It also allows reverse-applying 
      * 'commit' command to permanently store the applied patches into
        the repository 
      * 'push' can now fast-forward the patches if the base tree object
        has not changed 
      * The configuration file supports the 'editor' field 
      * '--version' option for the 'mail' command to generate the e-mail
        subject in the form '[PATCH version nr/total]' 
      * '--showpatch' option for 'refresh' to show the patch content
        when editing the patch description 
      * '--branch' option for several commands to work on a given branch
        instead of the current one without switching 
      * the 'pull' command was updated to follow the 'git pull' API 
      * '--base' option for 'import' to import a diff file on a
        different base and perform a three-way merge with the HEAD
        afterwards 
      * Support for the '.gitignore' file 
      * Several other optimisations 
      * Bug fixes

Many thanks to Paolo 'Blaisorblade' Giarrusso, Chuck Lever, Pierre
Ossman and Junio C Hamano for contributing patches to this release.

-- 
Catalin

Re: [ANNOUNCE] Stacked GIT 0.7

[Ingo Molnar]

* Catalin Marinas <catalin.marinas@gmail.com> wrote:

> Stacked GIT 0.7 release is available from 
> http://www.procode.org/stgit/
> 
> StGIT is a Python application providing similar functionality to Quilt 
> (i.e. pushing/popping patches to/from a stack) on top of GIT. These 
> operations are performed using GIT commands and the patches are stored 
> as GIT commit objects, allowing easy merging of the StGIT patches into 
> other repositories using standard GIT functionality.

i'm wondering - have you (or anyone else) done performance comparisons 
of quilt vs. stgit, using the same stack of patches? One of the most 
important features of quilt (for me) is that it's very fast at 
popping/pushing through hundreds of patches.

	Ingo

Re: [ANNOUNCE] Stacked GIT 0.7

[Catalin Marinas]

On 02/10/05, Ingo Molnar <mingo@elte.hu> wrote:
> i'm wondering - have you (or anyone else) done performance comparisons
> of quilt vs. stgit, using the same stack of patches? One of the most
> important features of quilt (for me) is that it's very fast at
> popping/pushing through hundreds of patches.

I've done some basic comparison and I must admit that Quilt is much
faster than StGIT. Popping is done in O(1) and it is equivalent to a
switch between two branches with 'git checkout' (can be less than 2
seconds with warm caches). Pushing without merging required (i.e.
fast-forwarding) is the same as popping.

What's time consuming is pushing when the base of the stack was
changed and merging is required. If the base changes do not involve
file removals/additions, a push operations takes ~1.5s per patch with
warm caches. Most of the time is spent in "git-read-tree -m" (StGIT
doesn't consume resources since it mainly calls GIT tools). If the
base changes involve file removals/additions, it can take considerably
longer since for each file GIT needs to call an external merger. This
could be solved by providing a smarter git-merge-index to deal with
file removals/additions but the time won't be shorter than ~1.5s per
patch.

An optimisation would be to keep the patches in diff format and apply
them but this would mean losing the advantages of the three-way
merging. Another issue is that git-apply doesn't support fuzzy
patches, though I could emulate it using patch. But I'm not sure
that's a good approach since I don't intend to re-write Quilt in
Python. My hope is that git-read-tree will get faster :-).

--
Catalin

Re: [ANNOUNCE] Stacked GIT 0.7

[Catalin Marinas]

On 02/10/05, Catalin Marinas <catalin.marinas@gmail.com> wrote:
> An optimisation would be to keep the patches in diff format and apply
> them but this would mean losing the advantages of the three-way
> merging.

After some tests, it looks like 'git-diff-tree -p ... | git-apply
--index' is about 3 times faster than 'git-read-tree -m' (in the best
case for git-read-tree with minor modifications of the base). It now
takes ~0.5s to push a single patch (compared to ~1.5s).

Since most of the patches, for routine updates of the base, don't
generate conflicts, git-apply should work fine. I'll modify StGIT to
use this method by default and fall back to the three-way merge in
case it fails. This will also solve the problem with complex
modifications of the base of the stack (i.e. file removals/additions)
when there won't be any need to call the external index merger if
git-apply succeeds.

--
Catalin

Re: [ANNOUNCE] Stacked GIT 0.7

[Catalin Marinas]

Catalin Marinas <catalin.marinas@gmail.com> wrote:
> After some tests, it looks like 'git-diff-tree -p ... | git-apply
> --index' is about 3 times faster than 'git-read-tree -m' (in the best
> case for git-read-tree with minor modifications of the base). It now
> takes ~0.5s to push a single patch (compared to ~1.5s).

And that's the patch for whoever wants to try. I will also add it to
the repository tonight:


Optimise 'push' to use git-apply instead of git-read-tree

With this patch, 'push' will use 'git-diff-tree | git-apply' first. If this
operation fails, it will fall back to the three-way merge with
git-read-tree.

Signed-off-by: Catalin Marinas <catalin.marinas@gmail.com>
---

 stgit/git.py   |    9 +++++++++
 stgit/stack.py |   19 +++++++++++--------
 2 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/stgit/git.py b/stgit/git.py
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -355,6 +355,15 @@ def commit(message, files = [], parents 
 
     return commit_id
 
+def apply_diff(rev1, rev2):
+    """Apply the diff between rev1 and rev2 onto the current
+    index. This function doesn't need to raise an exception since it
+    is only used for fast-pushing a patch. If this operation fails,
+    the pushing would fall back to the three-way merge.
+    """
+    return os.system('git-diff-tree -p %s %s | git-apply --index 2> /dev/null'
+                     % (rev1, rev2)) == 0
+
 def merge(base, head1, head2):
     """Perform a 3-way merge between base, head1 and head2 into the
     local tree
diff --git a/stgit/stack.py b/stgit/stack.py
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -598,14 +598,17 @@ class Series:
             # The current patch is empty after merge.
             patch.set_bottom(head, backup = True)
             patch.set_top(head, backup = True)
-            # merge/refresh can fail but the patch needs to be pushed
-            try:
-                git.merge(bottom, head, top)
-            except git.GitException, ex:
-                print >> sys.stderr, \
-                      'The merge failed during "push". ' \
-                      'Use "refresh" after fixing the conflicts'
-                pass
+
+            # Try the fast applying first. If this fails, fall back to the
+            # three-way merge
+            if not git.apply_diff(bottom, top):
+                # merge can fail but the patch needs to be pushed
+                try:
+                    git.merge(bottom, head, top)
+                except git.GitException, ex:
+                    print >> sys.stderr, \
+                          'The merge failed during "push". ' \
+                          'Use "refresh" after fixing the conflicts'
 
         append_string(self.__applied_file, name)
 


-- 
Catalin