[PATCH v3 0/9] remote-helpers: fixes and cleanups

[PATCH v3 0/9] remote-helpers: fixes and cleanups

[Felipe Contreras]

Updated the commit messages, so we say Bazaar instead of Mercurial, and stuff.

Here's a bunch of cleanups mostly to synchronize remote-bzr and remote-hg.

Felipe Contreras (9):
  remote-helpers: trivial cleanups
  remote-hg: remove extra check
  remote-bzr: fix bad state issue
  remote-bzr: add support to push URLs
  remote-hg: use hashlib instead of hg sha1 util
  remote-bzr: store converted URL
  remote-bzr: tell bazaar to be quiet
  remote-bzr: strip extra newline
  remote-bzr: use proper push method

 contrib/remote-helpers/git-remote-bzr | 55 ++++++++++++++++++++++++++++-------
 contrib/remote-helpers/git-remote-hg  | 10 ++-----
 2 files changed, 48 insertions(+), 17 deletions(-)

-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 1/9] remote-helpers: trivial cleanups

[Felipe Contreras]

No functional changes. Typos, unused variables, redundant operations,
and white-spaces.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 7 ++++---
 contrib/remote-helpers/git-remote-hg  | 2 +-
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index cc6609b..87c54bf 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -94,7 +94,7 @@ class Marks:
         return self.last_mark
 
     def is_marked(self, rev):
-        return str(rev) in self.marks
+        return rev in self.marks
 
     def new_mark(self, rev, mark):
         self.marks[rev] = mark
@@ -224,7 +224,7 @@ def export_files(tree, files):
             else:
                 mode = '100644'
 
-            # is the blog already exported?
+            # is the blob already exported?
             if h in filenodes:
                 mark = filenodes[h]
                 final.append((mode, mark, path))
@@ -521,7 +521,7 @@ def c_style_unescape(string):
     return string
 
 def parse_commit(parser):
-    global marks, blob_marks, bmarks, parsed_refs
+    global marks, blob_marks, parsed_refs
     global mode
 
     parents = []
@@ -643,6 +643,7 @@ def do_export(parser):
                 wt = repo.bzrdir.open_workingtree()
                 wt.update()
         print "ok %s" % ref
+
     print
 
 def do_capabilities(parser):
diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index fda4199..cfa96c1 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -12,7 +12,7 @@
 # For remote repositories a local clone is stored in
 # "$GIT_DIR/hg/origin/clone/.hg/".
 
-from mercurial import hg, ui, bookmarks, context, util, encoding, node, error, extensions
+from mercurial import hg, ui, bookmarks, context, encoding, node, error, extensions
 
 import re
 import sys
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 2/9] remote-hg: remove extra check

[Felipe Contreras]

Not needed since we use xrange ourselves.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index cfa96c1..80b3606 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -373,10 +373,6 @@ def export_ref(repo, name, kind, head):
     ename = '%s/%s' % (kind, name)
     tip = marks.get_tip(ename)
 
-    # mercurial takes too much time checking this
-    if tip and tip == head.rev():
-        # nothing to do
-        return
     revs = xrange(tip, head.rev() + 1)
     count = 0
 
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 3/9] remote-bzr: fix bad state issue

[Felipe Contreras]

Carried from remote-hg.

The problem reportedly happened after doing a push that fails, the abort
causes the state of remote-hg to go bad, this happens because
remote-hg's marks are not stored, but 'git fast-export' marks are.

Ensure that the marks are _always_ stored.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 87c54bf..7b6584e 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -32,6 +32,7 @@ import os
 import json
 import re
 import StringIO
+import atexit
 
 NAME_RE = re.compile('^([^<>]+)')
 AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]*)>$')
@@ -728,6 +729,7 @@ def main(args):
     blob_marks = {}
     parsed_refs = {}
     files_cache = {}
+    marks = None
 
     gitdir = os.environ['GIT_DIR']
     dirname = os.path.join(gitdir, 'bzr', alias)
@@ -754,6 +756,10 @@ def main(args):
             die('unhandled command: %s' % line)
         sys.stdout.flush()
 
+def bye():
+    if not marks:
+        return
     marks.store()
 
+atexit.register(bye)
 sys.exit(main(sys.argv))
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 4/9] remote-bzr: add support to push URLs

[Felipe Contreras]

Just like in remote-hg.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 7b6584e..f1d6d5e 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -32,7 +32,7 @@ import os
 import json
 import re
 import StringIO
-import atexit
+import atexit, shutil, hashlib
 
 NAME_RE = re.compile('^([^<>]+)')
 AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]*)>$')
@@ -719,11 +719,11 @@ def main(args):
     global blob_marks
     global parsed_refs
     global files_cache
+    global is_tmp
 
     alias = args[1]
     url = args[2]
 
-    prefix = 'refs/bzr/%s' % alias
     tags = {}
     filenodes = {}
     blob_marks = {}
@@ -731,6 +731,13 @@ def main(args):
     files_cache = {}
     marks = None
 
+    if alias[5:] == url:
+        is_tmp = True
+        alias = hashlib.sha1(alias).hexdigest()
+    else:
+        is_tmp = False
+
+    prefix = 'refs/bzr/%s' % alias
     gitdir = os.environ['GIT_DIR']
     dirname = os.path.join(gitdir, 'bzr', alias)
 
@@ -759,7 +766,10 @@ def main(args):
 def bye():
     if not marks:
         return
-    marks.store()
+    if not is_tmp:
+        marks.store()
+    else:
+        shutil.rmtree(dirname)
 
 atexit.register(bye)
 sys.exit(main(sys.argv))
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 5/9] remote-hg: use hashlib instead of hg sha1 util

[Felipe Contreras]

To be in sync with remote-bzr.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 80b3606..06920f2 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -22,7 +22,7 @@ import shutil
 import subprocess
 import urllib
 import atexit
-import urlparse
+import urlparse, hashlib
 
 #
 # If you want to switch to hg-git compatibility mode:
@@ -933,7 +933,7 @@ def main(args):
 
     if alias[4:] == url:
         is_tmp = True
-        alias = util.sha1(alias).hexdigest()
+        alias = hashlib.sha1(alias).hexdigest()
     else:
         is_tmp = False
 
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 6/9] remote-bzr: store converted URL

[Felipe Contreras]

Bazaar might convert the URL to something more appropriate, like an
absolute path. Lets store that instead of the original URL, which won't
work from a different working directory if it's relative.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index f1d6d5e..dda2932 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -32,7 +32,7 @@ import os
 import json
 import re
 import StringIO
-import atexit, shutil, hashlib
+import atexit, shutil, hashlib, urlparse, subprocess
 
 NAME_RE = re.compile('^([^<>]+)')
 AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]*)>$')
@@ -713,6 +713,14 @@ def get_repo(url, alias):
 
     return branch
 
+def fix_path(alias, orig_url):
+    url = urlparse.urlparse(orig_url, 'file')
+    if url.scheme != 'file' or os.path.isabs(url.path):
+        return
+    abs_url = urlparse.urljoin("%s/" % os.getcwd(), orig_url)
+    cmd = ['git', 'config', 'remote.%s.url' % alias, "bzr::%s" % abs_url]
+    subprocess.call(cmd)
+
 def main(args):
     global marks, prefix, dirname
     global tags, filenodes
@@ -741,6 +749,9 @@ def main(args):
     gitdir = os.environ['GIT_DIR']
     dirname = os.path.join(gitdir, 'bzr', alias)
 
+    if not is_tmp:
+        fix_path(alias, url)
+
     if not os.path.exists(dirname):
         os.makedirs(dirname)
 
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 7/9] remote-bzr: tell bazaar to be quiet

[Felipe Contreras]

Otherwise we get notification, progress bars, and what not.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index dda2932..8617e25 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -26,6 +26,7 @@ bzrlib.plugin.load_plugins()
 import bzrlib.generate_ids
 import bzrlib.transport
 import bzrlib.errors
+import bzrlib.ui
 
 import sys
 import os
@@ -755,6 +756,8 @@ def main(args):
     if not os.path.exists(dirname):
         os.makedirs(dirname)
 
+    bzrlib.ui.ui_factory.be_quiet(True)
+
     repo = get_repo(url, alias)
 
     marks_path = os.path.join(dirname, 'marks-int')
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 8/9] remote-bzr: strip extra newline

[Felipe Contreras]

It's added by fast-export, the user didn't type it.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 8617e25..9f56297 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -549,6 +549,10 @@ def parse_commit(parser):
         parents.append(parser.get_mark())
         parser.next()
 
+    # fast-export adds an extra newline
+    if data[-1] == '\n':
+        data = data[:-1]
+
     files = {}
 
     for line in parser:
-- 
1.8.2.1.1031.g2ee5873

[PATCH v3 9/9] remote-bzr: use proper push method

[Felipe Contreras]

Not just randomly synchronize the revisions with no checks at all. This
is the way bazaar's UI does it.

Also, add a non-ff check.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 9f56297..c19ed0e 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -639,12 +639,12 @@ def do_export(parser):
     for ref, revid in parsed_refs.iteritems():
         if ref == 'refs/heads/master':
             repo.generate_revision_history(revid, marks.get_tip('master'))
-            revno, revid = repo.last_revision_info()
             if peer:
-                if hasattr(peer, "import_last_revision_info_and_tags"):
-                    peer.import_last_revision_info_and_tags(repo, revno, revid)
-                else:
-                    peer.import_last_revision_info(repo.repository, revno, revid)
+                try:
+                    repo.push(peer, stop_revision=revid)
+                except bzrlib.errors.DivergedBranches:
+                    print "error %s non-fast forward" % ref
+                    continue
             else:
                 wt = repo.bzrdir.open_workingtree()
                 wt.update()
-- 
1.8.2.1.1031.g2ee5873

Re: [PATCH v3 0/9] remote-helpers: fixes and cleanups

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> Updated the commit messages, so we say Bazaar instead of Mercurial, and stuff.
>
> Here's a bunch of cleanups mostly to synchronize remote-bzr and remote-hg.

Thanks.  Will queue on 'pu' without looking.

Re: [PATCH v3 0/9] remote-helpers: fixes and cleanups

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> Updated the commit messages, so we say Bazaar instead of Mercurial, and stuff.
>>
>> Here's a bunch of cleanups mostly to synchronize remote-bzr and remote-hg.
>
> Thanks.  Will queue on 'pu' without looking.

Actually, I was going to merge fc/remote-hg and fc/remote-bzr down
to master anyway, so I'll just apply them directly on 'master'.

By the way, I personally do not think the quality of the changes to
remote-bzr matters all that much at this point in its history.  It's
not like millions of people use it heavily from the v1.8.2 release.
A huge patch series from its original author and nobody else, either
reviewed or unreviwed, would not hurt them more than the one in the
v1.8.2 version anyway. And it is also not like bzr-to-git population
will grow significantly in the future to require us to pile a lot of
features on remote-bzr that makes the maintenance burden of it
becomes an issue.

Am I underestimating the pain of potentially breaking existing
remote-bzr userbase?

Re: [PATCH v3 0/9] remote-helpers: fixes and cleanups

[Felipe Contreras]

On Fri, Apr 26, 2013 at 5:23 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Felipe Contreras <felipe.contreras@gmail.com> writes:
>>
>>> Updated the commit messages, so we say Bazaar instead of Mercurial, and stuff.
>>>
>>> Here's a bunch of cleanups mostly to synchronize remote-bzr and remote-hg.
>>
>> Thanks.  Will queue on 'pu' without looking.
>
> Actually, I was going to merge fc/remote-hg and fc/remote-bzr down
> to master anyway, so I'll just apply them directly on 'master'.
>
> By the way, I personally do not think the quality of the changes to
> remote-bzr matters all that much at this point in its history.  It's
> not like millions of people use it heavily from the v1.8.2 release.
> A huge patch series from its original author and nobody else, either
> reviewed or unreviwed, would not hurt them more than the one in the
> v1.8.2 version anyway. And it is also not like bzr-to-git population
> will grow significantly in the future to require us to pile a lot of
> features on remote-bzr that makes the maintenance burden of it
> becomes an issue.
>
> Am I underestimating the pain of potentially breaking existing
> remote-bzr userbase?

No, I think it's reasonable. 1.8.2 was better because 1.8.1 didn't ave
any support whatsoever, and 1.8.3 will be better, because 1.8.2 was
barely usable for any real-world project. Will there be any
regressions? I doubt it, and if there are, it's unlikely that they
would be found in the review process, in 'pu', or in 'next', specially
since very few people actually use remote-bzr, in part because it
wasn't very useful, and in part because few people use bzr in the
first place.

Of course, I would prefer if people reviewed the patches for the
massive changes, the _important_ patches, and I would gladly explain
the reasoning and update the commit messages if needed. But nobody is
volunteering to review that. The only exception was for a few
irrelevant patches that could be easily dropped.

remote-hg is a different story, and so I'm being more careful with the
changes there, and I actually think the current patches are more than
enough for 1.8.3, they should be tested in an actual release, and the
rest would have to wait. For the rest, I'm still juggling in which
order I should send them, and I want to send first the ones that
should maximize the benefits for the users, with minimum possibility
of breakage, but even then I wouldn't be confident with those landing
in 1.8.3, so 1.8.4 it is.

Cheers.

-- 
Felipe Contreras

Re: [PATCH v3 0/9] remote-helpers: fixes and cleanups

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> Felipe Contreras <felipe.contreras@gmail.com> writes:
>>
>>> Updated the commit messages, so we say Bazaar instead of Mercurial, and stuff.
>>>
>>> Here's a bunch of cleanups mostly to synchronize remote-bzr and remote-hg.
>>
>> Thanks.  Will queue on 'pu' without looking.
>
> Actually, I was going to merge fc/remote-hg and fc/remote-bzr down
> to master anyway, so I'll just apply them directly on 'master'.
> 
> By the way, I personally do not think the quality of the changes to
> remote-bzr matters all that much ...

Just in case anybody get the wrong impression from the above quoted
lines...

The "without looking" was a plan to "keep them on 'pu' for later
inspection so that I do not have to go back to list archive when I
declare patch review bankruptcy after the final 1.8.3 release".

I did read these patches I applied to 'master' and they all looked
sensible.

Re: [PATCH v3 0/9] remote-helpers: fixes and cleanups

[Ramkumar Ramachandra]

Felipe Contreras wrote:
> Updated the commit messages, so we say Bazaar instead of Mercurial, and stuff.
>
> Here's a bunch of cleanups mostly to synchronize remote-bzr and remote-hg.

Looks reasonable.  Thanks.