[PATCH 0/5] [DRAFT] StGIT, remotes, and parent information

[PATCH 0/5] [DRAFT] StGIT, remotes, and parent information

[Yann Dirson]

Although they are not finished, it may be useful to give some context
to current threads to publish a snapshot of what I'm working on, so
here it is.

1/5 Move identification of parent branch's remote def up into stack class.

Mostly code reorg.

2/5 Allows extraction of information about remotes.

Introduces a handful of functions to get informations about remotes in
the repository.  Makes use of remote.<branch>.fetch multi-valued
config entry.

3/5 Basic support for keeping a ref to the parent branch.

Currently relies on the failed assumption that config.set() does what
it sounds, so rely instead on a rewritten config.py.

That's why I'd rather rewrite it rapidly without a dictionnary, at
least as a first step - we can make the implementation better
afterwards, but first make it work.  Indeed I was originally only
working on the 5th patch of this stack, and all this is already a bit
beyond of what I was envisonning...

4/5 Have 'stg branch --create' record parent information.

Putting this all together.  Would also need to do something similar in
'stg clone' and possibly other places, so some code surely needs to be
factorized.

5/5 Make 'stg pull' use git-fetch and not git-pull.

Unfinished.  Git-fetch takes care of using the remote information.
Still have to use the previous work to find out the parent branch name
to rebase onto.

Best regards,
-- 
Yann Dirson    <ydirson@altern.org> |
Debian-related: <dirson@debian.org> |   Support Debian GNU/Linux:
                                    |  Freedom, Power, Stability, Gratis
     http://ydirson.free.fr/        | Check <http://www.debian.org/>

[PATCH 1/5] Move identification of parent branch's remote def up into stack class.

[Yann Dirson]

Also adds the __set_parent_remote() counterpart method, but only
private since it is expected to be called only through set_parent, to
be introduced in subsequent patch.

Signed-off-by: Yann Dirson <ydirson@altern.org>
---

 stgit/commands/pull.py |    6 +-----
 stgit/stack.py         |   13 +++++++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/stgit/commands/pull.py b/stgit/commands/pull.py
index 7c5db22..1a948be 100644
--- a/stgit/commands/pull.py
+++ b/stgit/commands/pull.py
@@ -53,11 +53,7 @@ def func(parser, options, args):
     if len(args) >= 1:
         repository = args[0]
     else:
-        section = 'branch "%s"' % git.get_head_file()
-        if config.has_option(section, 'remote'):
-            repository = config.get(section, 'remote')
-        else:
-            repository = 'origin'
+        repository = crt_series.get_parent_remote()
 
     refspec = None
     if len(args) == 2:
diff --git a/stgit/stack.py b/stgit/stack.py
index 2ae4dd5..e801f42 100644
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -397,6 +397,19 @@ class Series(StgitObject):
     def set_description(self, line):
         self._set_field('description', line)
 
+    def get_parent_remote(self):
+        section = 'branch "%s"' % self.__name
+        if config.has_option(section, 'remote'):
+            return config.get(section, 'remote')
+        else:
+            return 'origin'
+
+    def __set_parent_remote(self, remote):
+        section = 'branch "%s"' % self.__name
+        if not config.has_section(section):
+            config.add_section(section)
+        config.set(section, 'remote', remote)
+
     def __patch_is_current(self, patch):
         return patch.get_name() == self.get_current()
 

[PATCH 2/5] Allows extraction of information about remotes.

[Yann Dirson]

We want to know the list of declared remotes, the local branches they
hold, and which remotes holds a given branch.  All this regardless of
where the information is stored.

If there are any git-1.5 remotes declared in .git/config, we suppose
you know what you're doing and they will take precedence on any
pre-1.5 remotes.

This does not use git-remote for now, since it is 1.5 material not
released yet, does not support legacy branches/ remotes, and does not
allow yet to query all of the information we need.

Signed-off-by: Yann Dirson <ydirson@altern.org>
---

 stgit/git.py |   75 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 75 insertions(+), 0 deletions(-)

diff --git a/stgit/git.py b/stgit/git.py
index 4c5685a..3c2c237 100644
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -879,3 +879,78 @@ def modifying_revs(files, base_rev):
     revs = [line.strip() for line in _output_lines(cmd + files)]
 
     return revs
+
+
+def __remotes_from_config():
+    configremotes = []
+    stream = os.popen('git repo-config --get-regexp "^remote\..*\.url$"', 'r')
+    for line in stream:
+        m = re.match('^remote\.(.*)\.url ', line)
+        if m:
+            configremotes.append(m.group(1))
+    stream.close()
+    return configremotes
+
+def __remotes_from_dir(dir):
+    return os.listdir(os.path.join(basedir.get(), dir))
+
+def remotes_list():
+    """Return the list of remotes in the repository
+    """
+
+    return set(__remotes_from_config()) | \
+           set(__remotes_from_dir('remotes')) | \
+           set(__remotes_from_dir('branches'))
+
+def remotes_local_branches(remote):
+    """Returns the list of local branches fetched from given remote
+    """
+
+    branches = []
+    if remote in __remotes_from_config():
+        stream = os.popen('git repo-config --get-all "remote.%s.fetch"' % remote, 'r')
+        for line in stream:
+            # FIXME: should factorize refspec handling
+            m = re.match('^[^:]*:([^:]*)\n$', line)
+            if m:
+                branches.append(m.group(1))
+            else:
+                raise GitException, 'Cannot parse refspec "%s"' % line
+        stream.close()
+    elif remote in __remotes_from_dir('remotes'):
+        stream = open(os.path.join(basedir.get(), 'remotes', remote), 'r')
+        for line in stream:
+            # Only consider Pull lines
+            m = re.match('^Pull: (.*)\n$', line)
+            if m:
+                refspec = m.group(1)
+                m = re.match('^[^:]*:([^:]*)$', refspec)
+                if m:
+                    branches.append(m.group(1))
+                else:
+                    raise GitException, 'Cannot parse refspec "%s"' % refspec
+        stream.close()
+        pass
+    elif remote in __remotes_from_dir('branches'):
+        # old-style branches only declare one branch
+        branches.append('refs/heads/'+remote);
+        pass
+    else:
+        raise GitException, 'Unknown remote "%s"' % remote
+
+    return branches
+
+def identify_remote(branchname):
+    """Return the name for the remote to pull the given branchname
+    from, or None if we believe it is a local branch.
+    """
+
+    for remote in remotes_list():
+        if branchname in remotes_local_branches(remote):
+            return remote
+
+    # FIXME: in the case of local branch we should maybe set remote to
+    # "." but are we even sure it is the only case left ?
+
+    # if we get here we've found nothing
+    return None

[PATCH 3/5] Basic support for keeping a ref to the parent branch.

[Yann Dirson]

This adds a framework to handle the parent branch of a stack, in
addition to the parent remote, and to set them when creating a stack.

Signed-off-by: Yann Dirson <ydirson@altern.org>
---

 TODO           |    2 --
 stgit/stack.py |   27 ++++++++++++++++++++++++++-
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/TODO b/TODO
index 549bc9d..884b831 100644
--- a/TODO
+++ b/TODO
@@ -20,8 +20,6 @@ The future, when time allows or if someone else does them:
 - multiple heads in a patch - useful for forking a patch,
   synchronising with other patches (diff format or in other
   repositories)
-- "pull" argument should default to a sane value, "origin" is wrong in
-  many cases
 - commit directly to a patch which is not top
 - patch synchronisation between between branches (as some people,
   including me have the same patches based on different branches and
diff --git a/stgit/stack.py b/stgit/stack.py
index e801f42..9d4f881 100644
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -410,6 +410,29 @@ class Series(StgitObject):
             config.add_section(section)
         config.set(section, 'remote', remote)
 
+    def get_parent_branch(self):
+        section = 'branch "%s"' % self.__name
+        if config.has_option(section, 'merge'):
+            return config.get(section, 'merge')
+        elif rev_parse('heads/origin'):
+            return 'heads/origin'
+        else:
+            raise StackException, 'Cannot find a parent branch for "%s"' % self.__name
+
+    def __set_parent_branch(self, name):
+        section = 'branch "%s"' % self.__name
+        if not config.has_section(section):
+            config.add_section(section)
+        config.set(section, 'merge', name)
+
+    def set_parent(self, remote, localbranch):
+        if localbranch:
+            self.__set_parent_branch(localbranch)
+            if remote:
+                self.__set_parent_remote(remote)
+        elif remote:
+            raise StackException, 'Remote "%s" without a branch cannot be used as parent' % remote
+
     def __patch_is_current(self, patch):
         return patch.get_name() == self.get_current()
 
@@ -458,7 +481,7 @@ class Series(StgitObject):
         """
         return os.path.isdir(self.__patch_dir)
 
-    def init(self, create_at=False):
+    def init(self, create_at=False, parent_remote=None, parent_branch=None):
         """Initialises the stgit series
         """
         bases_dir = os.path.join(self.__base_dir, 'refs', 'bases')
@@ -475,6 +498,8 @@ class Series(StgitObject):
 
         os.makedirs(self.__patch_dir)
 
+        self.set_parent(parent_remote, parent_branch)
+        
         create_dirs(bases_dir)
 
         self.create_empty_field('applied')

[PATCH 4/5] Have 'stg branch --create' record parent information.

[Yann Dirson]

Signed-off-by: Yann Dirson <ydirson@altern.org>
---

 stgit/commands/branch.py |   37 ++++++++++++++++++++++++++++++++++---
 1 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/stgit/commands/branch.py b/stgit/commands/branch.py
index d348409..f074d47 100644
--- a/stgit/commands/branch.py
+++ b/stgit/commands/branch.py
@@ -123,10 +123,41 @@ def func(parser, options, args):
         check_head_top_equal()
 
         tree_id = None
-        if len(args) == 2:
+        if len(args) >= 2:
+            try:
+                if git.rev_parse(args[1]) == git.rev_parse('refs/heads/' + args[1]):
+                    # we are for sure refering to a branch
+                    parentbranch = 'refs/heads/' + args[1]
+                    print 'Recording "%s" as parent branch.' % parentbranch
+                elif git.rev_parse(args[1]) and re.search('/', args[1]):
+                    # FIXME: should the test be more strict ?
+                    parentbranch = args[1]
+                else:
+                    # Note: this includes refs to StGIT patches
+                    print 'Don\'t know how to determine parent branch from "%s".' % args[1]
+                    parentbranch = None
+            except git.GitException:
+                # should use a more specific exception to catch only non-git refs ?
+                print 'Don\'t know how to determine parent branch from "%s".' % args[1]
+                parentbranch = None
+
             tree_id = git_id(args[1])
-        
-        stack.Series(args[0]).init(create_at = tree_id)
+        else:
+            # branch stack off current branch
+            parentbranch = git.get_head_file()
+
+        if parentbranch:
+            parentremote = git.identify_remote(parentbranch)
+            if parentremote:
+                print 'Using "%s" remote to pull parent from.' % parentremote
+            else:
+                print 'Not identified a remote to pull parent from.'
+        else:
+            parentremote = None
+
+        stack.Series(args[0]).init(create_at = tree_id,
+                                   parent_remote = parentremote,
+                                   parent_branch = parentbranch)
 
         print 'Branch "%s" created.' % args[0]
         return

[PATCH 5/5] Make 'stg pull' use git-fetch and not git-pull.

[Yann Dirson]

We introduce a new pull-does-rebase setting, as companion to pullcmd.
This allows to use both pullcmd's that need a rebase (like git-fetch)
and pullcmd's that do not (like git-pull).

To be able to rebase, we rely on parent information being available
for the stack.

Signed-off-by: Yann Dirson <ydirson@altern.org>
---

 examples/gitconfig |    8 +++++++-
 stgit/config.py    |    3 ++-
 stgit/git.py       |   15 ++++++++++-----
 3 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/examples/gitconfig b/examples/gitconfig
index 5e7b240..e3d9889 100644
--- a/examples/gitconfig
+++ b/examples/gitconfig
@@ -33,8 +33,14 @@
 	#pager = ~/share/stgit/contrib/diffcol.sh
 	#pager = filterdiff --annotate | colordiff | less -FRX
 
-	# GIT pull command (should take the same arguments as git-pull)
+	# GIT pull command (should take the same arguments as
+	# git-fetch or git-pull).  By default:
+	#pullcmd = git-fetch
+	#pull-does-rebase = yes
+	# Alternative (old behaviour), less intuitive but maybe useful
+	# for some workflows:
 	#pullcmd = git-pull
+	#pull-does-rebase = no
 
 	# The three-way merge tool. Note that the 'output' file contains the
 	# same data as 'branch1'. This is useful for tools that do not take an
diff --git a/stgit/config.py b/stgit/config.py
index f5fbdab..0d4da06 100644
--- a/stgit/config.py
+++ b/stgit/config.py
@@ -65,7 +65,8 @@ def config_setup():
     config.set('stgit', 'autoresolved', 'no')
     config.set('stgit', 'smtpserver', 'localhost:25')
     config.set('stgit', 'smtpdelay', '5')
-    config.set('stgit', 'pullcmd', 'git-pull')
+    config.set('stgit', 'pullcmd', 'git-fetch')
+    config.set('stgit', 'pull-does-rebase', 'yes')
     config.set('stgit', 'merger',
                'diff3 -L current -L ancestor -L patched -m -E ' \
                '"%(branch1)s" "%(ancestor)s" "%(branch2)s" > "%(output)s"')
diff --git a/stgit/git.py b/stgit/git.py
index 3c2c237..82b5bf5 100644
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -806,18 +806,23 @@ def reset(files = None, tree_id = None, check_out = True):
         __set_head(tree_id)
 
 def pull(repository = 'origin', refspec = None):
-    """Pull changes from the remote repository. At the moment, just
-    use the 'git-pull' command
+    """Pull changes from the remote repository. Uses 'git-fetch'
+    and moves the stack base.
     """
-    # 'git-pull' updates the HEAD
+    # we update the HEAD
     __clear_head_cache()
 
     args = [repository]
     if refspec:
         args.append(refspec)
 
-    if __run(config.get('stgit', 'pullcmd'), args) != 0:
-        raise GitException, 'Failed "git-pull %s"' % repository
+    command = config.get('stgit', 'pullcmd')
+    if __run(command, args) != 0:
+        raise GitException, 'Failed "%s %s"' % (command, repository)
+
+    if (config.get('stgit', 'pull-does-rebase')):
+        # FIXME!
+        reset(tree_id = rev_parse(repository))
 
 def repack():
     """Repack all objects into a single pack