[StGIT RFC PATCH] Don't use refs/bases/<branchname>

[StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Karl Hasselström]

From: Karl Hasselström <kha@treskal.com>

It's silly to save the stack base in a ref when it can trivially be
computed from the bottommost applied patch, if any. (If there are no
applied patches, it's simply equal to HEAD.)

Signed-off-by: Karl Hasselström <kha@treskal.com>
---

StGIT maintains too much metadata that can be inferred by asking git
the right questions. There are two downsides with this:

  * With more metadata, the design gets more complicated and harder to
    understand.

  * The duplicated metadata can get out of sync, and that situation
    has to be taken care of (otherwise, StGIT breaks). For example,
    there was code to set the base to HEAD whenever there were no
    patches applied and base != HEAD. With this patch, the data is
    only stored in one place and can never get stale in the first
    place.

If there are no objections, I'll probably send more patches to
eliminate more redundant metadata.

 Documentation/tutorial.txt |    3 --
 stgit/commands/uncommit.py |    5 +---
 stgit/stack.py             |   53 ++++++--------------------------------------
 t/t1000-branch-create.sh   |   14 ------------
 t/t1200-push-modified.sh   |    7 +++---
 t/t1201-pull-trailing.sh   |    7 +++---
 t/t2200-rebase.sh          |    2 +-
 7 files changed, 17 insertions(+), 74 deletions(-)

diff --git a/Documentation/tutorial.txt b/Documentation/tutorial.txt
index 5899c38..2b8e4e7 100644
--- a/Documentation/tutorial.txt
+++ b/Documentation/tutorial.txt
@@ -387,9 +387,6 @@ Layout of the .git directory
     heads/
       master		- the master commit id
       ...
-    bases/
-      master		- the bottom id of the stack (to get a big diff)
-      ...
     tags/
       ...
     branches/
diff --git a/stgit/commands/uncommit.py b/stgit/commands/uncommit.py
index 0ee1585..462846c 100644
--- a/stgit/commands/uncommit.py
+++ b/stgit/commands/uncommit.py
@@ -80,11 +80,9 @@ def func(parser, options, args):
     print 'Uncommitting %d patches...' % patch_nr,
     sys.stdout.flush()
 
-    base_file = crt_series.get_base_file()
-
     for n in xrange(0, patch_nr):
         # retrieve the commit (only commits with a single parent are allowed)
-        commit_id = read_string(base_file)
+        commit_id = crt_series.get_base()
         commit = git.Commit(commit_id)
         try:
             parent, = commit.get_parents()
@@ -107,6 +105,5 @@ def func(parser, options, args):
                              author_name = author_name,
                              author_email = author_email,
                              author_date = author_date)
-        write_string(base_file, parent)
 
     print 'done'
diff --git a/stgit/stack.py b/stgit/stack.py
index b0a01dd..2477ac6 100644
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -291,8 +291,6 @@ class Series(StgitObject):
         self._set_dir(os.path.join(self.__base_dir, 'patches', self.__name))
         self.__refs_dir = os.path.join(self.__base_dir, 'refs', 'patches',
                                        self.__name)
-        self.__base_file = os.path.join(self.__base_dir, 'refs', 'bases',
-                                        self.__name)
 
         self.__applied_file = os.path.join(self._dir(), 'applied')
         self.__unapplied_file = os.path.join(self._dir(), 'unapplied')
@@ -378,12 +376,14 @@ class Series(StgitObject):
         f.close()
         return names
 
-    def get_base_file(self):
-        self.__begin_stack_check()
-        return self.__base_file
-
     def get_base(self):
-        return read_string(self.get_base_file())
+        # Return the parent of the bottommost patch, if there is one.
+        if os.path.isfile(self.__applied_file):
+            bottommost = file(self.__applied_file).readline().strip()
+            if bottommost:
+                return self.get_patch(bottommost).get_bottom()
+        # No bottommost patch, so just return HEAD
+        return git.get_head()
 
     def get_head(self):
         """Return the head of the branch
@@ -482,22 +482,6 @@ class Series(StgitObject):
         otherwise."""
         return self.patch_applied(name) or self.patch_unapplied(name)
 
-    def __begin_stack_check(self):
-        """Save the current HEAD into .git/refs/heads/base if the stack
-        is empty
-        """
-        if len(self.get_applied()) == 0:
-            head = git.get_head()
-            write_string(self.__base_file, head)
-
-    def __end_stack_check(self):
-        """Remove .git/refs/heads/base if the stack is empty.
-        This warning should never happen
-        """
-        if len(self.get_applied()) == 0 \
-           and read_string(self.__base_file) != git.get_head():
-            print 'Warning: stack empty but the HEAD and base are different'
-
     def head_top_equal(self):
         """Return true if the head and the top are the same
         """
@@ -519,8 +503,6 @@ class Series(StgitObject):
             raise StackException, self.__patch_dir + ' already exists'
         if os.path.exists(self.__refs_dir):
             raise StackException, self.__refs_dir + ' already exists'
-        if os.path.exists(self.__base_file):
-            raise StackException, self.__base_file + ' already exists'
 
         if (create_at!=False):
             git.create_branch(self.__name, create_at)
@@ -528,15 +510,12 @@ class Series(StgitObject):
         os.makedirs(self.__patch_dir)
 
         self.set_parent(parent_remote, parent_branch)
-        
-        create_dirs(os.path.join(self.__base_dir, 'refs', 'bases'))
 
         self.create_empty_field('applied')
         self.create_empty_field('unapplied')
         self.create_empty_field('description')
         os.makedirs(os.path.join(self._dir(), 'patches'))
         os.makedirs(self.__refs_dir)
-        self.__begin_stack_check()
         self._set_field('orig-base', git.get_head())
 
     def convert(self):
@@ -582,17 +561,12 @@ class Series(StgitObject):
 
         if to_stack.is_initialised():
             raise StackException, '"%s" already exists' % to_stack.get_branch()
-        if os.path.exists(to_stack.__base_file):
-            os.remove(to_stack.__base_file)
 
         git.rename_branch(self.__name, to_name)
 
         if os.path.isdir(self._dir()):
             rename(os.path.join(self.__base_dir, 'patches'),
                    self.__name, to_stack.__name)
-        if os.path.exists(self.__base_file):
-            rename(os.path.join(self.__base_dir, 'refs', 'bases'),
-                   self.__name, to_stack.__name)
         if os.path.exists(self.__refs_dir):
             rename(os.path.join(self.__base_dir, 'refs', 'patches'),
                    self.__name, to_stack.__name)
@@ -698,10 +672,6 @@ class Series(StgitObject):
             except OSError:
                 print 'Refs directory %s is not empty.' % self.__refs_dir
 
-        if os.path.exists(self.__base_file):
-            remove_file_and_dirs(
-                os.path.join(self.__base_dir, 'refs', 'bases'), self.__name)
-
         # Cleanup parent informations
         # FIXME: should one day make use of git-config --section-remove,
         # scheduled for 1.5.1
@@ -824,8 +794,6 @@ class Series(StgitObject):
 
         head = git.get_head()
 
-        self.__begin_stack_check()
-
         patch = Patch(name, self.__patch_dir, self.__refs_dir)
         patch.create()
 
@@ -893,8 +861,6 @@ class Series(StgitObject):
         if self.patch_hidden(name):
             self.unhide_patch(name)
 
-        self.__begin_stack_check()
-
     def forward_patches(self, names):
         """Try to fast-forward an array of patches.
 
@@ -902,7 +868,6 @@ class Series(StgitObject):
         stack. Apply the rest with push_patch
         """
         unapplied = self.get_unapplied()
-        self.__begin_stack_check()
 
         forwarded = 0
         top = git.get_head()
@@ -1001,8 +966,6 @@ class Series(StgitObject):
         unapplied = self.get_unapplied()
         assert(name in unapplied)
 
-        self.__begin_stack_check()
-
         patch = Patch(name, self.__patch_dir, self.__refs_dir)
 
         head = git.get_head()
@@ -1140,8 +1103,6 @@ class Series(StgitObject):
         else:
             self.__set_current(applied[-1])
 
-        self.__end_stack_check()
-
     def empty_patch(self, name):
         """Returns True if the patch is empty
         """
diff --git a/t/t1000-branch-create.sh b/t/t1000-branch-create.sh
index f0c2367..58209e7 100755
--- a/t/t1000-branch-create.sh
+++ b/t/t1000-branch-create.sh
@@ -39,20 +39,6 @@ test_expect_success \
 '
 
 test_expect_failure \
-    'Try to create an stgit branch with a spurious refs/bases/ entry' \
-    'find .git -name foo | xargs rm -rf &&
-     touch .git/refs/bases/foo &&
-     stg branch -c foo
-'
-
-test_expect_success \
-    'Check no part of the branch was created' \
-    'test "`find .git -name foo | tee /dev/stderr`" = ".git/refs/bases/foo" &&
-     ( grep foo .git/HEAD; test $? = 1 )
-'
-
-
-test_expect_failure \
     'Try to create an stgit branch with an existing git branch by that name' \
     'find .git -name foo | xargs rm -rf &&
      cp .git/refs/heads/master .git/refs/heads/foo &&
diff --git a/t/t1200-push-modified.sh b/t/t1200-push-modified.sh
index 7847a38..6769667 100755
--- a/t/t1200-push-modified.sh
+++ b/t/t1200-push-modified.sh
@@ -30,11 +30,12 @@ test_expect_success \
 
 test_expect_success \
     'Port those patches to orig tree' \
-    "(cd foo &&
-      GIT_DIR=../bar/.git git-format-patch --stdout bases/master..HEAD |
+    '(cd foo &&
+      GIT_DIR=../bar/.git git-format-patch --stdout \
+          $(cd ../bar && stg id base@master)..HEAD |
       git-am -3 -k
      )
-"
+    '
 
 test_expect_success \
     'Pull to sync with parent, preparing for the problem' \
diff --git a/t/t1201-pull-trailing.sh b/t/t1201-pull-trailing.sh
index 42c6619..46d9f82 100755
--- a/t/t1201-pull-trailing.sh
+++ b/t/t1201-pull-trailing.sh
@@ -28,11 +28,12 @@ test_expect_success \
 
 test_expect_success \
     'Port those patches to orig tree' \
-    "(cd foo &&
-      GIT_DIR=../bar/.git git-format-patch --stdout bases/master..HEAD |
+    '(cd foo &&
+      GIT_DIR=../bar/.git git-format-patch --stdout \
+          $(cd ../bar && stg id base@master)..HEAD |
       git-am -3 -k
      )
-"
+    '
 
 test_expect_success \
     'Pull those patches applied upstream, without pushing' \
diff --git a/t/t2200-rebase.sh b/t/t2200-rebase.sh
index e2d9d9a..52462dd 100755
--- a/t/t2200-rebase.sh
+++ b/t/t2200-rebase.sh
@@ -27,7 +27,7 @@ test_expect_success \
 	'Rebase to previous commit' \
 	'
 	stg rebase master~1 &&
-	test `git rev-parse bases/stack` = `git rev-parse master~1`
+	test `stg id base@stack` = `git rev-parse master~1`
 	'
 
 test_done

Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Catalin Marinas]

On 30/04/07, Karl Hasselström <kha@treskal.com> wrote:
> It's silly to save the stack base in a ref when it can trivially be
> computed from the bottommost applied patch, if any. (If there are no
> applied patches, it's simply equal to HEAD.)

The reason I initially had the base ref was to see what's on top of
the stack when using gitk. I later added refs/patches/<branch>/...
which are shown by gitk and the base ref would be redundant.

I'm OK with this patch as long as tools like qgit don't rely on this ref.

-- 
Catalin

Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Marco Costalba]

On 5/1/07, Catalin Marinas <catalin.marinas@gmail.com> wrote:
> On 30/04/07, Karl Hasselström <kha@treskal.com> wrote:
> > It's silly to save the stack base in a ref when it can trivially be
> > computed from the bottommost applied patch, if any. (If there are no
> > applied patches, it's simply equal to HEAD.)
>
> The reason I initially had the base ref was to see what's on top of
> the stack when using gitk. I later added refs/patches/<branch>/...
> which are shown by gitk and the base ref would be redundant.
>
> I'm OK with this patch as long as tools like qgit don't rely on this ref.
>

It's OK for me. A recent qgit already filters out content of
refs/bases to reduce visual 'noise'.

The only StGit data read directly are patches sha's; qgit walks
recursively all the files called "top" under directory tree

           <git dir>/patches/<current branch>

to get sha list of each applied and unapplied patch in one go.

This is much faster then calling "stg id <patch name>" for all the patches.

   Marco

Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Karl Hasselström]

On 2007-05-01 11:10:47 +0200, Marco Costalba wrote:

> On 5/1/07, Catalin Marinas <catalin.marinas@gmail.com> wrote:
>
> > I'm OK with this patch as long as tools like qgit don't rely on
> > this ref.
>
> It's OK for me. A recent qgit already filters out content of
> refs/bases to reduce visual 'noise'.

Good.

> The only StGit data read directly are patches sha's; qgit walks
> recursively all the files called "top" under directory tree
>
>           <git dir>/patches/<current branch>
>
> to get sha list of each applied and unapplied patch in one go. This
> is much faster then calling "stg id <patch name>" for all the
> patches.

Hmm. These are on my kill list too. :-)

The patch tops are already recorded in
refs/patches/<branch>/<patchname> to keep them from being garbage
collected, so these top files are redundant. But it isn't _that_ bad,
so if removing them would break qgit, I guess I could try to restrain
myself. At least all the other metadata is fair game. :-)

(But if I were you, I'd look for the patches under patches/refs
anyway; they _have_ to be there to survive garbage collection, so no
amount of stgit refactoring will break qgit.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Marco Costalba]

On 5/1/07, Karl Hasselström <kha@treskal.com> wrote:
> On 2007-05-01 11:10:47 +0200, Marco Costalba wrote:
>
> > The only StGit data read directly are patches sha's; qgit walks
> > recursively all the files called "top" under directory tree
> >
> >           <git dir>/patches/<current branch>
> >
> > to get sha list of each applied and unapplied patch in one go. This
> > is much faster then calling "stg id <patch name>" for all the
> > patches.
>
> Hmm. These are on my kill list too. :-)
>
> The patch tops are already recorded in
> refs/patches/<branch>/<patchname> to keep them from being garbage
> collected, so these top files are redundant. But it isn't _that_ bad,
> so if removing them would break qgit, I guess I could try to restrain
> myself. At least all the other metadata is fair game. :-)
>
> (But if I were you, I'd look for the patches under patches/refs
> anyway; they _have_ to be there to survive garbage collection, so no
> amount of stgit refactoring will break qgit.)
>

Well, I did. ;-)

Actually I pushed a patch few hours  ago to read patches sha under refs/patches.

The problem is that the patch (for now) is pushed only for the new
development version of qgit, not the stable one and, worse, all the
currently released versions will break if you remove  <git
dir>/patches/ directory.

So please, if possible deprecate  <git dir>/patches/ directory but do
not remove for a while, so to let users to upgrade gracefully.

Thanks
Marco

Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Karl Hasselström]

On 2007-05-01 21:59:34 +0200, Marco Costalba wrote:

> On 5/1/07, Karl Hasselström <kha@treskal.com> wrote:
>
> > (But if I were you, I'd look for the patches under patches/refs
> > anyway; they _have_ to be there to survive garbage collection, so
> > no amount of stgit refactoring will break qgit.)
>
> Well, I did. ;-)
>
> Actually I pushed a patch few hours ago to read patches sha under
> refs/patches.

Good. This was precisely what I was hoping to scare you into doing!
:-)

> The problem is that the patch (for now) is pushed only for the new
> development version of qgit, not the stable one and, worse, all the
> currently released versions will break if you remove
> <git dir>/patches/ directory.
>
> So please, if possible deprecate <git dir>/patches/ directory but do
> not remove for a while, so to let users to upgrade gracefully.

It's not near the top of my kill list by any stretch of the
imagination, so no need to worry. And even if it were, Catalin would
certainly stand as a wall of sanity between qgit and my chain saw. :-)

Thanks.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Marco Costalba]

On 5/2/07, Karl Hasselström <kha@treskal.com> wrote:
>
> It's not near the top of my kill list by any stretch of the
> imagination, so no need to worry. And even if it were, Catalin would
> certainly stand as a wall of sanity between qgit and my chain saw. :-)
>

Currently I check for the existence of <git dir>/patches directory as
a quick exit in case a repository does NOT have a StGIT repo on it
(the common case).

This avoids a costly and 99% of cases not needed 'stg <something>' call.

I ask if it will be still a safe check in the long period or it is
better to change the check to something else (as existence of <git
dir>/refs/patches) instead ?

Thanks
Marco

Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>

[Karl Hasselström]

On 2007-05-02 13:17:01 +0200, Marco Costalba wrote:

> On 5/2/07, Karl Hasselström <kha@treskal.com> wrote:
>
> > It's not near the top of my kill list by any stretch of the
> > imagination, so no need to worry. And even if it were, Catalin
> > would certainly stand as a wall of sanity between qgit and my
> > chain saw. :-)
>
> Currently I check for the existence of <git dir>/patches directory
> as a quick exit in case a repository does NOT have a StGIT repo on
> it (the common case).
>
> This avoids a costly and 99% of cases not needed 'stg <something>'
> call.
>
> I ask if it will be still a safe check in the long period or it is
> better to change the check to something else (as existence of
> <git dir>/refs/patches) instead ?

I personally have no plan to attempt to remove all of .git/patches --
a lot of data in there is redundant, but certainly not all of it, and
there's no compelling reason to move it. But Catalin has the final
word, of course.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle