[StGit PATCH 0/2] Patch stack logging

[StGit PATCH 0/2] Patch stack logging

[Karl Hasselström]

So here's what I have this far. It writes a log that's close to the
format I want, but doesn't do anything with it yet.

It's not in kha/experimental yet; it seems to work OK, but it leaves a
log behind that'll have to be cleaned up manually when the format
changes, which it will.

---

Karl Hasselström (2):
      Write to a stack log when stack is modified
      Library functions for tree and blob manipulation


 stgit/commands/branch.py   |   18 ++++--
 stgit/commands/clone.py    |    3 +
 stgit/commands/float.py    |    3 +
 stgit/commands/fold.py     |    3 +
 stgit/commands/imprt.py    |    3 +
 stgit/commands/init.py     |    3 +
 stgit/commands/pick.py     |    3 +
 stgit/commands/pop.py      |    3 +
 stgit/commands/pull.py     |    3 +
 stgit/commands/push.py     |    6 +-
 stgit/commands/rebase.py   |    4 +
 stgit/commands/refresh.py  |    4 +
 stgit/commands/rename.py   |    3 +
 stgit/commands/resolved.py |    4 +
 stgit/commands/sink.py     |    3 +
 stgit/commands/sync.py     |    5 +-
 stgit/lib/git.py           |  139 +++++++++++++++++++++++++++++++++++++-------
 stgit/lib/log.py           |   60 +++++++++++++++++++
 stgit/lib/transaction.py   |    3 +
 19 files changed, 229 insertions(+), 44 deletions(-)
 create mode 100644 stgit/lib/log.py

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

[StGit PATCH 1/2] Library functions for tree and blob manipulation

[Karl Hasselström]

Wrap trees and blobs in Python objects (just like commits were already
wrapped), so that StGit code can read and write them.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

I'm quite pleased with the Foo/Foodata distinction (the former being a
blob, tree, or commit in the git repository, the latter an immutable
specification for such an object). It's convenient to work with,
without loss of efficiency.

 stgit/lib/git.py |  139 ++++++++++++++++++++++++++++++++++++++++++++++--------
 1 files changed, 118 insertions(+), 21 deletions(-)


diff --git a/stgit/lib/git.py b/stgit/lib/git.py
index 6ee8a71..8184fec 100644
--- a/stgit/lib/git.py
+++ b/stgit/lib/git.py
@@ -146,13 +146,100 @@ class Person(Repr):
                 defaults = cls.user())
         return cls.__committer
 
-class Tree(Repr):
+class GitObject(Repr):
+    """Base class for all git objects."""
+
+class Blobdata(Repr):
+    def __init__(self, string):
+        self.__string = str(string)
+    str = property(lambda self: self.__string)
+    def commit(self, repository):
+        sha1 = repository.run(['git', 'hash-object', '-w', '--stdin']
+                              ).raw_input(self.str).output_one_line()
+        return repository.get_blob(sha1)
+
+class Blob(GitObject):
     """Immutable."""
-    def __init__(self, sha1):
+    typename = 'blob'
+    default_perm = '100644'
+    def __init__(self, repository, sha1):
+        self.__repository = repository
         self.__sha1 = sha1
     sha1 = property(lambda self: self.__sha1)
     def __str__(self):
-        return 'Tree<%s>' % self.sha1
+        return 'Blob<%s>' % self.sha1
+    @property
+    def data(self):
+        return Blobdata(self.__repository.cat_object(self.sha1))
+
+class ImmutableDict(dict):
+    def error(*args, **kwargs):
+        raise TypeError('Cannot modify immutable dict')
+    __delitem__ = error
+    __setitem__ = error
+    clear = error
+    pop = error
+    popitem = error
+    setdefault = error
+    update = error
+
+class Treedata(Repr):
+    """Immutable."""
+    @staticmethod
+    def __x(po):
+        if isinstance(po, GitObject):
+            perm, object = po.default_perm, po
+        else:
+            perm, object = po
+        return perm, object
+    def __init__(self, entries):
+        self.__entries = ImmutableDict((name, self.__x(po))
+                                       for (name, po) in entries.iteritems())
+    entries = property(lambda self: self.__entries)
+    def set_entry(self, name, po):
+        e = dict(self.entries)
+        e[name] = self.__x(po)
+        return type(self)(e)
+    def del_entry(self, name):
+        e = dict(self.entries)
+        del e[name]
+        return type(self)(e)
+    def commit(self, repository):
+        listing = ''.join(
+            '%s %s %s\t%s\0' % (mode, obj.typename, obj.sha1, name)
+            for (name, (mode, obj)) in self.entries.iteritems())
+        sha1 = repository.run(['git', 'mktree', '-z']
+                              ).raw_input(listing).output_one_line()
+        return repository.get_tree(sha1)
+    @classmethod
+    def parse(cls, repository, s):
+        entries = {}
+        for line in s.split('\0')[:-1]:
+            m = re.match(r'^([0-7]{6}) ([a-z]) ([0-9a-f]{40})\t(.*)$', line)
+            assert m
+            perm, type, sha1, name = m.groups()
+            entries[name] = (perm, repository.get_object(type, sha1))
+        return cls(entries)
+
+class Tree(GitObject):
+    """Immutable."""
+    typename = 'tree'
+    default_perm = '040000'
+    def __init__(self, repository, sha1):
+        self.__sha1 = sha1
+        self.__repository = repository
+        self.__data = None
+    sha1 = property(lambda self: self.__sha1)
+    @property
+    def data(self):
+        if self.__data == None:
+            self.__data = Treedata.parse(
+                self.__repository,
+                self.__repository.run(['git', 'ls-tree', '-z', self.sha1]
+                                      ).raw_output())
+        return self.__data
+    def __str__(self):
+        return 'Tree<sha1: %s>' % self.sha1
 
 class Commitdata(Repr):
     """Immutable."""
@@ -202,6 +289,22 @@ class Commitdata(Repr):
         return ('Commitdata<tree: %s, parents: %s, author: %s,'
                 ' committer: %s, message: "%s">'
                 ) % (tree, parents, self.author, self.committer, self.message)
+    def commit(self, repository):
+        c = ['git', 'commit-tree', self.tree.sha1]
+        for p in self.parents:
+            c.append('-p')
+            c.append(p.sha1)
+        env = {}
+        for p, v1 in ((self.author, 'AUTHOR'),
+                       (self.committer, 'COMMITTER')):
+            if p != None:
+                for attr, v2 in (('name', 'NAME'), ('email', 'EMAIL'),
+                                 ('date', 'DATE')):
+                    if getattr(p, attr) != None:
+                        env['GIT_%s_%s' % (v1, v2)] = str(getattr(p, attr))
+        sha1 = repository.run(c, env = env).raw_input(self.message
+                                                      ).output_one_line()
+        return repository.get_commit(sha1)
     @classmethod
     def parse(cls, repository, s):
         cd = cls()
@@ -223,8 +326,9 @@ class Commitdata(Repr):
                 assert False
         assert False
 
-class Commit(Repr):
+class Commit(GitObject):
     """Immutable."""
+    typename = 'commit'
     def __init__(self, repository, sha1):
         self.__sha1 = sha1
         self.__repository = repository
@@ -302,7 +406,8 @@ class Repository(RunWithEnv):
     def __init__(self, directory):
         self.__git_dir = directory
         self.__refs = Refs(self)
-        self.__trees = ObjectCache(lambda sha1: Tree(sha1))
+        self.__blobs = ObjectCache(lambda sha1: Blob(self, sha1))
+        self.__trees = ObjectCache(lambda sha1: Tree(self, sha1))
         self.__commits = ObjectCache(lambda sha1: Commit(self, sha1))
         self.__default_index = None
         self.__default_worktree = None
@@ -353,26 +458,18 @@ class Repository(RunWithEnv):
                     ).output_one_line())
         except run.RunException:
             raise RepositoryException('%s: No such revision' % rev)
+    def get_blob(self, sha1):
+        return self.__blobs[sha1]
     def get_tree(self, sha1):
         return self.__trees[sha1]
     def get_commit(self, sha1):
         return self.__commits[sha1]
-    def commit(self, commitdata):
-        c = ['git', 'commit-tree', commitdata.tree.sha1]
-        for p in commitdata.parents:
-            c.append('-p')
-            c.append(p.sha1)
-        env = {}
-        for p, v1 in ((commitdata.author, 'AUTHOR'),
-                       (commitdata.committer, 'COMMITTER')):
-            if p != None:
-                for attr, v2 in (('name', 'NAME'), ('email', 'EMAIL'),
-                                 ('date', 'DATE')):
-                    if getattr(p, attr) != None:
-                        env['GIT_%s_%s' % (v1, v2)] = str(getattr(p, attr))
-        sha1 = self.run(c, env = env).raw_input(commitdata.message
-                                                ).output_one_line()
-        return self.get_commit(sha1)
+    def get_object(self, type, sha1):
+        return { Blob.typename: self.get_blob,
+                 Tree.typename: self.get_tree,
+                 Commit.typename: self.get_commit }[type](sha1)
+    def commit(self, objectdata):
+        return objectdata.commit(self)
     @property
     def head(self):
         try:

[StGit PATCH 2/2] Write to a stack log when stack is modified

[Karl Hasselström]

Create a log branch (called <branchname>.stgit) for each StGit branch,
and write to it whenever the stack is modified.

As of yet, nothing can be done with this log.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

Lots of noise here since all the old-infrastructure commands update
the log manually. But note how few lines it takes to actually write
the blobs, trees, and commits (in log.py, near the end).

 stgit/commands/branch.py   |   18 ++++++++++---
 stgit/commands/clone.py    |    3 +-
 stgit/commands/float.py    |    3 ++
 stgit/commands/fold.py     |    3 +-
 stgit/commands/imprt.py    |    3 +-
 stgit/commands/init.py     |    3 +-
 stgit/commands/pick.py     |    3 +-
 stgit/commands/pop.py      |    3 +-
 stgit/commands/pull.py     |    3 +-
 stgit/commands/push.py     |    6 ++--
 stgit/commands/rebase.py   |    4 +--
 stgit/commands/refresh.py  |    4 ++-
 stgit/commands/rename.py   |    3 +-
 stgit/commands/resolved.py |    4 ++-
 stgit/commands/sink.py     |    3 +-
 stgit/commands/sync.py     |    5 +++-
 stgit/lib/log.py           |   60 ++++++++++++++++++++++++++++++++++++++++++++
 stgit/lib/transaction.py   |    3 +-
 18 files changed, 111 insertions(+), 23 deletions(-)
 create mode 100644 stgit/lib/log.py


diff --git a/stgit/commands/branch.py b/stgit/commands/branch.py
index 50684bb..131dfea 100644
--- a/stgit/commands/branch.py
+++ b/stgit/commands/branch.py
@@ -25,7 +25,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git, basedir
-
+from stgit.lib import log
 
 help = 'manage patch stacks'
 usage = """%prog [options] branch-name [commit-id]
@@ -161,6 +161,7 @@ def func(parser, options, args):
                                    parent_branch = parentbranch)
 
         out.info('Branch "%s" created' % args[0])
+        log.log_entry(log.default_stack(), 'stg branch --create')
         return
 
     elif options.clone:
@@ -181,6 +182,8 @@ def func(parser, options, args):
         crt_series.clone(clone)
         out.done()
 
+        log.copy_log(log.default_repo(), crt_series.get_name(), clone,
+                     'stg branch --clone')
         return
 
     elif options.delete:
@@ -188,6 +191,7 @@ def func(parser, options, args):
         if len(args) != 1:
             parser.error('incorrect number of arguments')
         __delete_branch(args[0], options.force)
+        log.delete_log(log.default_repo(), args[0])
         return
 
     elif options.list:
@@ -195,13 +199,16 @@ def func(parser, options, args):
         if len(args) != 0:
             parser.error('incorrect number of arguments')
 
-        branches = git.get_heads()
-        branches.sort()
+        branches = set(git.get_heads())
+        for br in set(branches):
+            m = re.match(r'^(.*)\.stgit$', br)
+            if m and m.group(1) in branches:
+                branches.remove(br)
 
         if branches:
             out.info('Available branches:')
             max_len = max([len(i) for i in branches])
-            for i in branches:
+            for i in sorted(branches):
                 __print_branch(i, max_len)
         else:
             out.info('No branches')
@@ -238,7 +245,8 @@ def func(parser, options, args):
         stack.Series(args[0]).rename(args[1])
 
         out.info('Renamed branch "%s" to "%s"' % (args[0], args[1]))
-
+        log.rename_log(log.default_repo(), args[0], args[1],
+                       'stg branch --rename')
         return
 
     elif options.unprotect:
diff --git a/stgit/commands/clone.py b/stgit/commands/clone.py
index c3b0bbe..8c3cccd 100644
--- a/stgit/commands/clone.py
+++ b/stgit/commands/clone.py
@@ -21,7 +21,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'make a local clone of a remote repository'
 usage = """%prog [options] <repository> <dir>
@@ -56,5 +56,6 @@ def func(parser, options, args):
     # to work on a brand new repository
     basedir.clear_cache()
     stack.Series().init()
+    log.log_entry(log.default_stack(), 'stg clone')
 
     print 'done'
diff --git a/stgit/commands/float.py b/stgit/commands/float.py
index 0ba4446..5fec87e 100644
--- a/stgit/commands/float.py
+++ b/stgit/commands/float.py
@@ -22,6 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
+from stgit.lib import log
 
 help = 'push patches to the top, even if applied'
 usage = """%prog [<patches> | -s [<series>] ]
@@ -89,3 +90,5 @@ def func(parser, options, args):
         if topush:
             topush.reverse()
             push_patches(crt_series, topush)
+
+    log.log_entry(log.default_stack(), 'stg float')
diff --git a/stgit/commands/fold.py b/stgit/commands/fold.py
index 3930a1f..105f54a 100644
--- a/stgit/commands/fold.py
+++ b/stgit/commands/fold.py
@@ -22,7 +22,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'integrate a GNU diff patch into the current patch'
 usage = """%prog [options] [<file>]
@@ -80,3 +80,4 @@ def func(parser, options, args):
         git.apply_patch(filename = filename)
 
     out.done()
+    log.log_entry(log.default_stack(), 'stg fold')
diff --git a/stgit/commands/imprt.py b/stgit/commands/imprt.py
index 4a4b792..d90e45d 100644
--- a/stgit/commands/imprt.py
+++ b/stgit/commands/imprt.py
@@ -24,7 +24,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'import a GNU diff file as a new patch'
 usage = """%prog [options] [<file>|<url>]
@@ -289,3 +289,4 @@ def func(parser, options, args):
         __import_file(filename, options)
 
     print_crt_patch(crt_series)
+    log.log_entry(log.default_stack(), 'stg import')
diff --git a/stgit/commands/init.py b/stgit/commands/init.py
index 475a4ce..b49b7b2 100644
--- a/stgit/commands/init.py
+++ b/stgit/commands/init.py
@@ -22,7 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'initialise the current branch for use with StGIT'
 usage = """%prog [options]
@@ -42,3 +42,4 @@ def func(parser, options, args):
         parser.error('incorrect number of arguments')
 
     crt_series.init()
+    log.log_entry(log.default_stack(), 'stg init')
diff --git a/stgit/commands/pick.py b/stgit/commands/pick.py
index e011a84..4cc8944 100644
--- a/stgit/commands/pick.py
+++ b/stgit/commands/pick.py
@@ -23,7 +23,7 @@ from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
 from stgit.stack import Series
-
+from stgit.lib import log
 
 help = 'import a patch from a different branch or a commit object'
 usage = """%prog [options] [<patch@branch>|<commit>]
@@ -163,4 +163,5 @@ def func(parser, options, args):
         else:
             out.done()
 
+    log.log_entry(log.default_stack(), 'stg pick')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/pop.py b/stgit/commands/pop.py
index dedf41b..e60361e 100644
--- a/stgit/commands/pop.py
+++ b/stgit/commands/pop.py
@@ -22,7 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'pop one or more patches from the stack'
 usage = """%prog [options] [<patch1>] [<patch2>] [<patch3>..<patch4>]
@@ -87,4 +87,5 @@ def func(parser, options, args):
     if topush:
         push_patches(crt_series, topush)
 
+    log.log_entry(log.default_stack(), 'stg pop')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/pull.py b/stgit/commands/pull.py
index bec7fd7..8be5a14 100644
--- a/stgit/commands/pull.py
+++ b/stgit/commands/pull.py
@@ -23,7 +23,7 @@ from stgit.utils import *
 from stgit.out import *
 from stgit.config import GitConfigException
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'pull the changes from the remote repository'
 usage = """%prog [options] [<repository>]
@@ -106,4 +106,5 @@ def func(parser, options, args):
     if config.get('stgit.keepoptimized') == 'yes':
         git.repack()
 
+    log.log_entry(log.default_stack(), 'stg pull')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/push.py b/stgit/commands/push.py
index 979835b..148a0b5 100644
--- a/stgit/commands/push.py
+++ b/stgit/commands/push.py
@@ -23,7 +23,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'push one or more patches onto of the stack'
 usage = """%prog [options] [<patch1>] [<patch2>] [<patch3>..<patch4>]
@@ -69,11 +69,11 @@ def func(parser, options, args):
         out.start('Undoing push of "%s"' % patch)
         resolved_all()
         if crt_series.undo_push():
+            log.log_entry(log.default_stack(), 'stg push --undo')
             out.done()
         else:
             out.done('patch unchanged')
         print_crt_patch(crt_series)
-
         return
 
     check_local_changes()
@@ -100,5 +100,5 @@ def func(parser, options, args):
         patches.reverse()
 
     push_patches(crt_series, patches, options.merged)
-
+    log.log_entry(log.default_stack(), 'stg push')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/rebase.py b/stgit/commands/rebase.py
index 12faaf8..41e93ce 100644
--- a/stgit/commands/rebase.py
+++ b/stgit/commands/rebase.py
@@ -21,7 +21,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'move the stack base to another point in history'
 usage = """%prog [options] <new-base-id>
@@ -57,5 +57,5 @@ def func(parser, options, args):
     applied = prepare_rebase(crt_series)
     rebase(crt_series, args[0])
     post_rebase(crt_series, applied, options.nopush, options.merged)
-
+    log.log_entry(log.default_stack(), 'stg rebase')
     print_crt_patch(crt_series)
diff --git a/stgit/commands/refresh.py b/stgit/commands/refresh.py
index 4695c62..017ec5b 100644
--- a/stgit/commands/refresh.py
+++ b/stgit/commands/refresh.py
@@ -24,7 +24,7 @@ from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
 from stgit.config import config
-
+from stgit.lib import log
 
 help = 'generate a new commit for the current patch'
 usage = """%prog [options] [<files or dirs>]
@@ -91,6 +91,7 @@ def func(parser, options, args):
     if options.undo:
         out.start('Undoing the refresh of "%s"' % patch)
         crt_series.undo_refresh()
+        log.log_entry(log.default_stack(), 'stg refresh --undo')
         out.done()
         return
 
@@ -131,6 +132,7 @@ def func(parser, options, args):
         if options.patch:
             between.reverse()
             push_patches(crt_series, between)
+        log.log_entry(log.default_stack(), 'stg refresh')
     elif options.annotate:
         # only annotate the top log entry as there is no need to
         # refresh the patch and generate a full commit
diff --git a/stgit/commands/rename.py b/stgit/commands/rename.py
index e2b0fa4..4e9aac0 100644
--- a/stgit/commands/rename.py
+++ b/stgit/commands/rename.py
@@ -22,7 +22,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'rename a patch in the series'
 usage = """%prog [options] <oldpatch> <newpatch>
@@ -42,4 +42,5 @@ def func(parser, options, args):
 
     out.start('Renaming patch "%s" to "%s"' % (args[0], args[1]))
     crt_series.rename_patch(args[0], args[1])
+    log.log_entry(log.default_stack(), 'stg rename')
     out.done()
diff --git a/stgit/commands/resolved.py b/stgit/commands/resolved.py
index 4ee75b8..cb21d84 100644
--- a/stgit/commands/resolved.py
+++ b/stgit/commands/resolved.py
@@ -24,7 +24,7 @@ from stgit.utils import *
 from stgit import stack, git, basedir
 from stgit.config import config, file_extensions
 from stgit.gitmergeonefile import interactive_merge
-
+from stgit.lib import log
 
 help = 'mark a file conflict as solved'
 usage = """%prog [options] [<files...>]
@@ -55,6 +55,7 @@ def func(parser, options, args):
 
     if options.all and not options.interactive:
         resolved_all(options.reset)
+        log.log_entry(log.default_stack(), 'stg resolved')
         return
 
     conflicts = git.get_conflicts()
@@ -82,3 +83,4 @@ def func(parser, options, args):
             git.resolved([filename])
     else:
         git.resolved(files, options.reset)
+    log.log_entry(log.default_stack(), 'stg resolved')
diff --git a/stgit/commands/sink.py b/stgit/commands/sink.py
index 2167d87..f3a9ada 100644
--- a/stgit/commands/sink.py
+++ b/stgit/commands/sink.py
@@ -22,7 +22,7 @@ from optparse import OptionParser, make_option
 from stgit.commands.common import *
 from stgit.utils import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'send patches deeper down the stack'
 usage = """%prog [-t <target patch>] [-n] [<patches>]
@@ -68,3 +68,4 @@ def func(parser, options, args):
         def not_reapplied_yet(p):
             return not p in newapplied
         push_patches(crt_series, filter(not_reapplied_yet, oldapplied))
+    log.log_entry(log.default_stack(), 'stg sink')
diff --git a/stgit/commands/sync.py b/stgit/commands/sync.py
index 99ab7de..168199d 100644
--- a/stgit/commands/sync.py
+++ b/stgit/commands/sync.py
@@ -23,7 +23,7 @@ from stgit.commands.common import *
 from stgit.utils import *
 from stgit.out import *
 from stgit import stack, git
-
+from stgit.lib import log
 
 help = 'synchronise patches with a branch or a series'
 usage = """%prog [options] [<patch1>] [<patch2>] [<patch3>..<patch4>]
@@ -77,6 +77,7 @@ def func(parser, options, args):
         out.start('Undoing the sync of "%s"' % crt_series.get_current())
         crt_series.undo_refresh()
         git.reset()
+        log.log_entry(log.default_stack(), 'stg sync --undo')
         out.done()
         return
 
@@ -166,3 +167,5 @@ def func(parser, options, args):
     # push the remaining patches
     if popped:
         push_patches(crt_series, popped)
+
+    log.log_entry(log.default_stack(), 'stg sync')
diff --git a/stgit/lib/log.py b/stgit/lib/log.py
new file mode 100644
index 0000000..1a9a326
--- /dev/null
+++ b/stgit/lib/log.py
@@ -0,0 +1,60 @@
+from stgit.lib import git, stack
+
+def patch_tree(repository, cd):
+    return repository.commit(git.Treedata({ 'a': cd.parent.data.tree,
+                                            'b': cd.tree }))
+
+def order_blob(repository, stack, kind):
+    return repository.commit(git.Blobdata(''.join(
+                '%s\n' % pn for pn in getattr(stack.patchorder, kind))))
+
+def log_entry_tree(repository, stack):
+    patches = repository.commit(
+        git.Treedata(dict((pn, patch_tree(repository,
+                                          stack.patches.get(pn).commit.data))
+                          for pn in stack.patchorder.all)))
+    return repository.commit(git.Treedata({
+                'version': repository.commit(git.Blobdata('0')),
+                'patches': patches,
+                'applied': order_blob(repository, stack, 'applied'),
+                'unapplied': order_blob(repository, stack, 'unapplied'),
+                }))
+
+def log_ref(branch):
+    return 'refs/heads/%s.stgit' % branch
+
+def log_entry(stack, msg):
+    ref = log_ref(stack.name)
+    try:
+        last = [stack.repository.refs.get(ref)]
+    except KeyError:
+        last = []
+    commit = stack.repository.commit(
+        git.Commitdata(tree = log_entry_tree(stack.repository, stack),
+                       parents = last, message = msg))
+    stack.repository.refs.set(ref, commit, msg)
+
+def delete_log(repo, branch):
+    ref = log_ref(branch)
+    if repo.refs.exists(ref):
+        repo.refs.delete(ref)
+
+def rename_log(repo, old_branch, new_branch, msg):
+    old_ref = log_ref(old_branch)
+    new_ref = log_ref(new_branch)
+    if repo.refs.exists(old_ref):
+        repo.refs.set(new_ref, repo.refs.get(old_ref), msg)
+        repo.refs.delete(old_ref)
+
+def copy_log(repo, src_branch, dst_branch, msg):
+    src_ref = log_ref(src_branch)
+    dst_ref = log_ref(dst_branch)
+    if repo.refs.exists(src_ref):
+        repo.refs.set(dst_ref, repo.refs.get(src_ref), msg)
+
+def default_repo():
+    return stack.Repository.default()
+
+def default_stack(branch = None):
+    repo = default_repo()
+    return repo.get_stack(branch or repo.current_branch)
diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index 3613b15..0b0b52d 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -1,6 +1,6 @@
 from stgit import exception, utils
 from stgit.out import *
-from stgit.lib import git
+from stgit.lib import git, log
 
 class TransactionException(exception.StgException):
     pass
@@ -117,6 +117,7 @@ class StackTransaction(object):
         _print_current_patch(self.__stack.patchorder.applied, self.__applied)
         self.__stack.patchorder.applied = self.__applied
         self.__stack.patchorder.unapplied = self.__unapplied
+        log.log_entry(self.__stack, self.__msg)
 
         if self.__error:
             return utils.STGIT_CONFLICT

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

[Catalin Marinas]

On 14/02/2008, Karl Hasselström <kha@treskal.com> wrote:
> Create a log branch (called <branchname>.stgit) for each StGit branch,
>  and write to it whenever the stack is modified.

The abstractions are really nice (and I still wonder how StGIT
codebase increased that much when all I needed two years ago was a
simple script-like application to reorder commits :-)).

Anyway, I don't really like the idea of an additional commit (I don't
even like the old patch log implementation) when the stack is
modified. It needs some profiling but it has a visible impact on
stacks with a big number of patches (my last kernel release at
www.linux-arm.org/git had 80 patches and it takes a lot of time to
push them).

Can we not use some of the automatic reflog recording that GIT does
instead of writing a commit? It's cheaper to write a text file than
generating a commit. In my kernel repository I have several branches
with many patches and, even after "git gc" and repacking, it is still
slow (mainly because of git-read-tree but I'd like to reduce the
number of calls to GIT).

-- 
Catalin

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

[Karl Hasselström]

On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:

> On 14/02/2008, Karl Hasselström <kha@treskal.com> wrote:
>
> > Create a log branch (called <branchname>.stgit) for each StGit
> > branch, and write to it whenever the stack is modified.
>
> The abstractions are really nice (and I still wonder how StGIT
> codebase increased that much when all I needed two years ago was a
> simple script-like application to reorder commits :-)).

:-) I'll take some of the blame, but StGit was quite large already
when I started submitting patches to it.

> Anyway, I don't really like the idea of an additional commit (I
> don't even like the old patch log implementation) when the stack is
> modified. It needs some profiling but it has a visible impact on
> stacks with a big number of patches (my last kernel release at
> www.linux-arm.org/git had 80 patches and it takes a lot of time to
> push them).

One thing to remember is that my patch stack log gets one new commit
per StGit operation, not one per changed patch. So if you push 80
patches with one command, that will give you just one commit on the
log. (If someone makes a scripts that calls push 80 times this won't
help, of course.)

> Can we not use some of the automatic reflog recording that GIT does
> instead of writing a commit? It's cheaper to write a text file than
> generating a commit.

As I said, I'm not convinced that it's a huge difference when you have
just one log commit per operation, not one per changed patch. But
let's stop arguing about it until I have some numbers to back my claim
up. ;-)

> In my kernel repository I have several branches with many patches
> and, even after "git gc" and repacking, it is still slow (mainly
> because of git-read-tree but I'd like to reduce the number of calls
> to GIT).

When you say "it's still slow", are you referring to the existing
per-patch log, my per-branch log, or just StGit in general?

Have you noticed any difference between commands using the old and new
infrastructure (say, stg push vs. stg goto)? The latter should be
taking less time, due to touching the worktree only when necessary.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

[Catalin Marinas]

On 21/02/2008, Karl Hasselström <kha@treskal.com> wrote:
> On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:
>
>  > The abstractions are really nice (and I still wonder how StGIT
>  > codebase increased that much when all I needed two years ago was a
>  > simple script-like application to reorder commits :-)).
>
>
> :-) I'll take some of the blame, but StGit was quite large already
>  when I started submitting patches to it.

Anyway, the new restructuring is much cleaner, though heavily OO and
some people might not like it (me not included).

>
>
>  > Anyway, I don't really like the idea of an additional commit (I
>  > don't even like the old patch log implementation) when the stack is
>  > modified. It needs some profiling but it has a visible impact on
>  > stacks with a big number of patches (my last kernel release at
>  > www.linux-arm.org/git had 80 patches and it takes a lot of time to
>  > push them).
>
>
> One thing to remember is that my patch stack log gets one new commit
>  per StGit operation, not one per changed patch. So if you push 80
>  patches with one command, that will give you just one commit on the
>  log. (If someone makes a scripts that calls push 80 times this won't
>  help, of course.)

Ah, I got it now. Indeed, it would be faster than the current
per-patch log. I no longer that worried :-)

>  > In my kernel repository I have several branches with many patches
>  > and, even after "git gc" and repacking, it is still slow (mainly
>  > because of git-read-tree but I'd like to reduce the number of calls
>  > to GIT).
>
>
> When you say "it's still slow", are you referring to the existing
>  per-patch log, my per-branch log, or just StGit in general?

I think it's more GIT in general. Checking the tree status takes some
time and a 3-way merge on a 512MB RAM machine with GIT using over
200MB gets a bit slow.

>  Have you noticed any difference between commands using the old and new
>  infrastructure (say, stg push vs. stg goto)? The latter should be
>  taking less time, due to touching the worktree only when necessary.

In the patch pushing functions, it now first calls simple_merge()
which is still a 3-way merge but without rename detection. The old
StGIT implementation was using "git-diff | git-apply" and falling back
to the recursive merge. Most of the patches apply cleanly and we don't
need the three-way merge which uses more RAM.

The "(modified)" information was also displayed for a three-way merge
(either read-tree or recursive-merge) but now it is only shown for the
recursive one. Correctly, merging git-read-tree modifies a patch as
well since it no longer applied cleanly. We could use this "modified"
feature to automatically export patches (some people asked for this in
the past, as means for backup in case of StGIT failures).

-- 
Catalin

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

[Karl Hasselström]

On 2008-02-22 14:05:55 +0000, Catalin Marinas wrote:

> On 21/02/2008, Karl Hasselström <kha@treskal.com> wrote:
>
> > On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:
> >
> >  > The abstractions are really nice (and I still wonder how StGIT
> >  > codebase increased that much when all I needed two years ago
> >  > was a simple script-like application to reorder commits :-)).
> >
> > :-) I'll take some of the blame, but StGit was quite large already
> >  when I started submitting patches to it.
>
> Anyway, the new restructuring is much cleaner, though heavily OO and
> some people might not like it (me not included).

That it's conceptually OO is unavoidable, I think (the only way to
avoid that would be through obfuscation). And using Python's OO
features to write code for such a model is the least bad you can do in
Python. IMHO.

> > When you say "it's still slow", are you referring to the existing
> > per-patch log, my per-branch log, or just StGit in general?
>
> I think it's more GIT in general. Checking the tree status takes
> some time and a 3-way merge on a 512MB RAM machine with GIT using
> over 200MB gets a bit slow.

I just upgraded my laptop from 512 MB to 2 GB, so I'll start
neglecting this kind of problem, I fear. :-/

> >  Have you noticed any difference between commands using the old
> >  and new infrastructure (say, stg push vs. stg goto)? The latter
> >  should be taking less time, due to touching the worktree only
> >  when necessary.
>
> In the patch pushing functions, it now first calls simple_merge()
> which is still a 3-way merge but without rename detection. The old
> StGIT implementation was using "git-diff | git-apply" and falling
> back to the recursive merge. Most of the patches apply cleanly and
> we don't need the three-way merge which uses more RAM.

I didn't include patch application because it wasn't necessary to get
things working, and is easy to add at any later point.

I'd be curious to see how much of a difference it makes.

> The "(modified)" information was also displayed for a three-way
> merge (either read-tree or recursive-merge) but now it is only shown
> for the recursive one. Correctly, merging git-read-tree modifies a
> patch as well since it no longer applied cleanly.

Ah, so the precise meaning of a patch being "modified" is that git
apply failed to apply it. I didn't know that.

> We could use this "modified" feature to automatically export patches
> (some people asked for this in the past, as means for backup in case
> of StGIT failures).

You mean automatically export only the changed patches? (Otherwise stg
export would do the job -- I used to use that before I got confident
enough with StGit.)

One cool thing we could do is export the patches as files in a git
branch -- the log machinery I'm building should make it trivial. That
way, you'd get the history of the patches too.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [StGit PATCH 2/2] Write to a stack log when stack is modified

[Catalin Marinas]

On 22/02/2008, Karl Hasselström <kha@treskal.com> wrote:
> On 2008-02-22 14:05:55 +0000, Catalin Marinas wrote:
>
>  > On 21/02/2008, Karl Hasselström <kha@treskal.com> wrote:
>  >
>  > > On 2008-02-20 22:46:48 +0000, Catalin Marinas wrote:
>  > >
>  > >  > The abstractions are really nice (and I still wonder how StGIT
>  > >  > codebase increased that much when all I needed two years ago
>  > >  > was a simple script-like application to reorder commits :-)).
>  > >
>  > > :-) I'll take some of the blame, but StGit was quite large already
>  > >  when I started submitting patches to it.
>  >
>  > Anyway, the new restructuring is much cleaner, though heavily OO and
>  > some people might not like it (me not included).
>
>
> That it's conceptually OO is unavoidable, I think (the only way to
>  avoid that would be through obfuscation). And using Python's OO
>  features to write code for such a model is the least bad you can do in
>  Python. IMHO.

I agree.

>  > > When you say "it's still slow", are you referring to the existing
>  > > per-patch log, my per-branch log, or just StGit in general?
>  >
>  > I think it's more GIT in general. Checking the tree status takes
>  > some time and a 3-way merge on a 512MB RAM machine with GIT using
>  > over 200MB gets a bit slow.
>
>
> I just upgraded my laptop from 512 MB to 2 GB, so I'll start
>  neglecting this kind of problem, I fear. :-/

You can boot with mem=512MB (or even less) :-).

>  > >  Have you noticed any difference between commands using the old
>  > >  and new infrastructure (say, stg push vs. stg goto)? The latter
>  > >  should be taking less time, due to touching the worktree only
>  > >  when necessary.
>  >
>  > In the patch pushing functions, it now first calls simple_merge()
>  > which is still a 3-way merge but without rename detection. The old
>  > StGIT implementation was using "git-diff | git-apply" and falling
>  > back to the recursive merge. Most of the patches apply cleanly and
>  > we don't need the three-way merge which uses more RAM.
>
>
> I didn't include patch application because it wasn't necessary to get
>  things working, and is easy to add at any later point.
>
>  I'd be curious to see how much of a difference it makes.

It makes a difference if you don't have much RAM available. I think
once a patch falls back to three-way merge, Linux already makes room
available for the subsequent merges. But, as I said, most of the time
the patches just apply cleanly.

>  > We could use this "modified" feature to automatically export patches
>  > (some people asked for this in the past, as means for backup in case
>  > of StGIT failures).
>
>
> You mean automatically export only the changed patches?

Yes, at refresh and pushing, but for the latter, only if a patch was modified.

>  One cool thing we could do is export the patches as files in a git
>  branch -- the log machinery I'm building should make it trivial. That
>  way, you'd get the history of the patches too.

Yes, I don't think it is difficult to do (it just need to be optional
as some people might not need it).

-- 
Catalin