[PATCH v2 0/7] remote-bzr: generic updates

[PATCH v2 0/7] remote-bzr: generic updates

[Felipe Contreras]

Hi,

Basically the same couple of fixes, plus a fixed fix.

Christophe Simonis (3):
  remote-bzr: fix directory renaming
  remote-bzr: remove files before modifications
  remote-bzr: add utf-8 support for fetching

David Engster (1):
  remote-bzr: set author if available

Felipe Contreras (3):
  remote-bzr: only update workingtree on local repos
  remote-bzr: improve tag handling
  remote-bzr: add utf-8 support for pushing

 contrib/remote-helpers/git-remote-bzr |  64 +++++++++++++++------
 contrib/remote-helpers/test-bzr.sh    | 102 ++++++++++++++++++++++++++++++++++
 2 files changed, 149 insertions(+), 17 deletions(-)

-- 
1.8.2

[PATCH v2 1/7] remote-bzr: fix directory renaming

[Felipe Contreras]

From: Christophe Simonis <christophe@kn.gl>

Git does not handle directories, renaming a directory is renaming every
files in this directory.

[fc: added tests]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr |  8 +++++++-
 contrib/remote-helpers/test-bzr.sh    | 24 ++++++++++++++++++++++++
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index c5822e4..a7d041b 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -191,7 +191,13 @@ def get_filechanges(cur, prev):
         modified[path] = fid
     for oldpath, newpath, fid, kind, mod, _ in changes.renamed:
         removed[oldpath] = None
-        modified[newpath] = fid
+        if kind == 'directory':
+            lst = cur.list_files(from_dir=newpath, recursive=True)
+            for path, file_class, kind, fid, entry in lst:
+                if kind != 'directory':
+                    modified[newpath + '/' + path] = fid
+        else:
+            modified[newpath] = fid
 
     return modified, removed
 
diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index 8450432..d26e5c7 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -126,4 +126,28 @@ test_expect_success 'special modes' '
   test_cmp expected actual
 '
 
+cat > expected <<EOF
+100644 blob 54f9d6da5c91d556e6b54340b1327573073030af	content
+100755 blob 68769579c3eaadbe555379b9c3538e6628bae1eb	executable
+120000 blob 6b584e8ece562ebffc15d38808cd6b98fc3d97ea	link
+040000 tree 35c0caa46693cef62247ac89a680f0c5ce32b37b	movedir-new
+EOF
+
+test_expect_success 'moving directory' '
+  (cd bzrrepo &&
+  mkdir movedir &&
+  echo one > movedir/one &&
+  echo two > movedir/two &&
+  bzr add movedir &&
+  bzr commit -m movedir &&
+  bzr mv movedir movedir-new &&
+  bzr commit -m movedir-new) &&
+
+  (cd gitrepo &&
+  git pull &&
+  git ls-tree HEAD > ../actual) &&
+
+  test_cmp expected actual
+'
+
 test_done
-- 
1.8.2

[PATCH v2 2/7] remote-bzr: remove files before modifications

[Felipe Contreras]

From: Christophe Simonis <christophe@kn.gl>

Allow re-add of a deleted file in the same commit.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index a7d041b..f818e93 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -303,10 +303,10 @@ def export_branch(branch, name):
             else:
                 print "merge :%s" % m
 
+        for f in removed:
+            print "D %s" % (f,)
         for f in modified_final:
             print "M %s :%u %s" % f
-        for f in removed:
-            print "D %s" % (f)
         print
 
         count += 1
-- 
1.8.2

[PATCH v2 3/7] remote-bzr: set author if available

[Felipe Contreras]

From: David Engster <deng@randomsample.de>

[fc: added tests]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr |  7 ++++++-
 contrib/remote-helpers/test-bzr.sh    | 15 +++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index f818e93..a99a924 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -266,7 +266,12 @@ def export_branch(branch, name):
         tz = rev.timezone
         committer = rev.committer.encode('utf-8')
         committer = "%s %u %s" % (fixup_user(committer), time, gittz(tz))
-        author = committer
+        authors = rev.get_apparent_authors()
+        if authors:
+            author = authors[0].encode('utf-8')
+            author = "%s %u %s" % (fixup_user(author), time, gittz(tz))
+        else:
+            author = committer
         msg = rev.message.encode('utf-8')
 
         msg += '\n'
diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index d26e5c7..68105fc 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -150,4 +150,19 @@ test_expect_success 'moving directory' '
   test_cmp expected actual
 '
 
+test_expect_success 'different authors' '
+  (cd bzrrepo &&
+  echo john >> content &&
+  bzr commit -m john \
+    --author "Jane Rey <jrey@example.com>" \
+    --author "John Doe <jdoe@example.com>") &&
+
+  (cd gitrepo &&
+  git pull &&
+  git show --format="%an <%ae>, %cn <%ce>" --quiet > ../actual) &&
+
+  echo "Jane Rey <jrey@example.com>, A U Thor <author@example.com>" > expected &&
+  test_cmp expected actual
+'
+
 test_done
-- 
1.8.2

[PATCH v2 4/7] remote-bzr: only update workingtree on local repos

[Felipe Contreras]

Apparently, that's the only way it's possible.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index a99a924..9466cb9 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -630,10 +630,9 @@ def do_export(parser):
                     peer.import_last_revision_info_and_tags(repo, revno, revid)
                 else:
                     peer.import_last_revision_info(repo.repository, revno, revid)
-                wt = peer.bzrdir.open_workingtree()
             else:
                 wt = repo.bzrdir.open_workingtree()
-            wt.update()
+                wt.update()
         print "ok %s" % ref
     print
 
-- 
1.8.2

[PATCH v2 5/7] remote-bzr: improve tag handling

[Felipe Contreras]

If the tags don't have an associated revision, there's nothing we can do
with them, skip them. Also, let's skip the ones git can't handle.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 9466cb9..dc0b757 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -25,6 +25,7 @@ bzrlib.plugin.load_plugins()
 
 import bzrlib.generate_ids
 import bzrlib.transport
+import bzrlib.errors
 
 import sys
 import os
@@ -332,12 +333,9 @@ def export_branch(branch, name):
 
 def export_tag(repo, name):
     global tags
-    try:
-        print "reset refs/tags/%s" % name
-        print "from :%u" % rev_to_mark(tags[name])
-        print
-    except KeyError:
-        warn("TODO: fetch tag '%s'" % name)
+    print "reset refs/tags/%s" % name
+    print "from :%u" % rev_to_mark(tags[name])
+    print
 
 def do_import(parser):
     global dirname
@@ -651,12 +649,25 @@ def do_capabilities(parser):
 
     print
 
+def ref_is_valid(name):
+    return not True in [c in name for c in '~^: \\']
+
 def do_list(parser):
     global tags
     print "? refs/heads/%s" % 'master'
-    for tag, revid in parser.repo.tags.get_tag_dict().items():
+
+    branch = parser.repo
+    branch.lock_read()
+    for tag, revid in branch.tags.get_tag_dict().items():
+        try:
+            branch.revision_id_to_dotted_revno(revid)
+        except bzrlib.errors.NoSuchRevision:
+            continue
+        if not ref_is_valid(tag):
+            continue
         print "? refs/tags/%s" % tag
         tags[tag] = revid
+    branch.unlock()
     print "@refs/heads/%s HEAD" % 'master'
     print
 
-- 
1.8.2

[PATCH v2 6/7] remote-bzr: add utf-8 support for fetching

[Felipe Contreras]

From: Christophe Simonis <christophe@kn.gl>

Initial patch by Timotheus Pokorra.

[fc: added tests]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 15 +++++++++------
 contrib/remote-helpers/test-bzr.sh    | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 6 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index dc0b757..64b35c7 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -184,21 +184,24 @@ def get_filechanges(cur, prev):
 
     changes = cur.changes_from(prev)
 
+    def u(s):
+        return s.encode('utf-8')
+
     for path, fid, kind in changes.added:
-        modified[path] = fid
+        modified[u(path)] = fid
     for path, fid, kind in changes.removed:
-        removed[path] = None
+        removed[u(path)] = None
     for path, fid, kind, mod, _ in changes.modified:
-        modified[path] = fid
+        modified[u(path)] = fid
     for oldpath, newpath, fid, kind, mod, _ in changes.renamed:
-        removed[oldpath] = None
+        removed[u(oldpath)] = None
         if kind == 'directory':
             lst = cur.list_files(from_dir=newpath, recursive=True)
             for path, file_class, kind, fid, entry in lst:
                 if kind != 'directory':
-                    modified[newpath + '/' + path] = fid
+                    modified[u(newpath + '/' + path)] = fid
         else:
-            modified[newpath] = fid
+            modified[u(newpath)] = fid
 
     return modified, removed
 
diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index 68105fc..9a510a8 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -165,4 +165,36 @@ test_expect_success 'different authors' '
   test_cmp expected actual
 '
 
+test_expect_success 'fetch utf-8 filenames' '
+  mkdir -p tmp && cd tmp &&
+  test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
+
+  export LC_ALL=en_US.UTF-8
+
+  (
+  bzr init bzrrepo &&
+  cd bzrrepo &&
+
+  echo test >> "áéíóú" &&
+  bzr add "áéíóú" &&
+  echo test >> "îø∫∆" &&
+  bzr add "îø∫∆" &&
+  bzr commit -m utf-8 &&
+  echo test >> "áéíóú" &&
+  bzr commit -m utf-8 &&
+  bzr rm "îø∫∆" &&
+  bzr mv "áéíóú" "åß∂" &&
+  bzr commit -m utf-8
+  ) &&
+
+  (
+  git clone "bzr::$PWD/bzrrepo" gitrepo &&
+  cd gitrepo &&
+  git ls-files > ../actual
+  ) &&
+
+  echo "\"\\303\\245\\303\\237\\342\\210\\202\"" > expected &&
+  test_cmp expected actual
+'
+
 test_done
-- 
1.8.2

[PATCH v2 7/7] remote-bzr: add utf-8 support for pushing

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr |  6 ++++++
 contrib/remote-helpers/test-bzr.sh    | 31 +++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 64b35c7..bd25e08 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -513,6 +513,11 @@ class CustomTree():
     def get_symlink_target(self, file_id):
         return self.updates[file_id]['data']
 
+def c_style_unescape(string):
+    if string[0] == string[-1] == '"':
+        return string.decode('string-escape')[1:-1]
+    return string
+
 def parse_commit(parser):
     global marks, blob_marks, bmarks, parsed_refs
     global mode
@@ -552,6 +557,7 @@ def parse_commit(parser):
             f = { 'deleted' : True }
         else:
             die('Unknown file command: %s' % line)
+        path = c_style_unescape(path).decode('utf-8')
         files[path] = f
 
     repo = parser.repo
diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index 9a510a8..e800c97 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -197,4 +197,35 @@ test_expect_success 'fetch utf-8 filenames' '
   test_cmp expected actual
 '
 
+test_expect_success 'push utf-8 filenames' '
+  mkdir -p tmp && cd tmp &&
+  test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
+
+  export LC_ALL=en_US.UTF-8
+
+  (
+  bzr init bzrrepo &&
+  cd bzrrepo &&
+
+  echo one >> content &&
+  bzr add content &&
+  bzr commit -m one
+  ) &&
+
+  (
+  git clone "bzr::$PWD/bzrrepo" gitrepo &&
+  cd gitrepo &&
+
+  echo test >> "áéíóú" &&
+  git add "áéíóú" &&
+  git commit -m utf-8 &&
+
+  git push
+  ) &&
+
+  (cd bzrrepo && bzr ls > ../actual) &&
+  echo -e "content\náéíóú" > expected &&
+  test_cmp expected actual
+'
+
 test_done
-- 
1.8.2

Re: [PATCH v2 0/7] remote-bzr: generic updates

[Junio C Hamano]

Oops.  The previous round is already now part of 'master'.

Re: [PATCH v2 0/7] remote-bzr: generic updates

[Felipe Contreras]

On Mon, Apr 8, 2013 at 1:26 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Oops.  The previous round is already now part of 'master'.

All right, rebased and resent.

--
Felipe Contreras

Re: [PATCH v2 0/7] remote-bzr: generic updates

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> On Mon, Apr 8, 2013 at 1:26 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Oops.  The previous round is already now part of 'master'.
>
> All right, rebased and resent.

Thanks.

Re: [PATCH v2 6/7] remote-bzr: add utf-8 support for fetching

[Torsten Bögershausen]

On 08.04.13 19:27, Felipe Contreras wrote:
> From: Christophe Simonis <christophe@kn.gl>
> 
> Initial patch by Timotheus Pokorra.
> 
> [fc: added tests]
> 
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
>  contrib/remote-helpers/git-remote-bzr | 15 +++++++++------
>  contrib/remote-helpers/test-bzr.sh    | 32 ++++++++++++++++++++++++++++++++
>  2 files changed, 41 insertions(+), 6 deletions(-)
> 
> +
> +  export LC_ALL=en_US.UTF-8
> +
> +  (
> +  bzr init bzrrepo &&
> +  cd bzrrepo &&
> +
> +  echo test >> "áéíóú" &&
> +  bzr add "áéíóú" &&

Just a whish from my side:
bzr under MacOS is not able to handle some unicode characters correctly.
(Those we can decompose)
https://bugs.launchpad.net/bzr/+bug/172383

I just added a fix on the test-bzr script we have on git.git/next,
and the short version is to avoid all code points like áéíóú.
The greek stuff is probably OK, but the accents are not.
I choose a danish/norwegian "æ", which is not decomposable.
(And the non-portable echo -e could be avoided by using printf)
/Torsten
 

git diff git.git/next
diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index f4c7768..2150e7c 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -175,8 +175,8 @@ test_expect_success 'fetch utf-8 filenames' '
   bzr init bzrrepo &&
   cd bzrrepo &&
 
-  echo test >> "áéíóú" &&
-  bzr add "áéíóú" &&
+  echo test >> "æ" &&
+  bzr add "æ" &&
   bzr commit -m utf-8
   ) &&
 
@@ -185,8 +185,7 @@ test_expect_success 'fetch utf-8 filenames' '
   cd gitrepo &&
   git ls-files > ../actual
   ) &&
-  echo "\"\\303\\241\\303\\251\\303\\255\\303\\263\\303\\272\"" > expected &&
+  echo "\"\\303\\246\"" > expected &&
   test_cmp expected actual
 '
 
@@ -209,15 +208,15 @@ test_expect_success 'push utf-8 filenames' '
   git clone "bzr::$PWD/bzrrepo" gitrepo &&
   cd gitrepo &&
 
-  echo test >> "áéíóú" &&
-  git add "áéíóú" &&
+  echo test >> "æ" &&
+  git add "æ" &&
   git commit -m utf-8 &&
 
   git push
   ) &&
 
   (cd bzrrepo && bzr ls > ../actual) &&
-  echo -e "content\náéíóú" > expected &&
+  printf "content\n\303\246\n" > expected &&
   test_cmp expected actual
 '