[PATCH 2/3] git-p4: support exclude paths

[PATCH 2/3] git-p4: support exclude paths

[Tommy Thorn]

Teach git-p4 about the -/ option which adds depot paths to the exclude
list, used when cloning. The option is chosen such that the natural
Perforce syntax works, eg:

  git p4 clone //branch/path/... -//branch/path/{large,old}/...

Trailing ... on exclude paths are optional.

This is a generalization of a change by Dmitry Kakurin (thanks).

Signed-off-by: Tommy Thorn <tommy-git@thorn.ws>
---
 contrib/fast-import/git-p4 |   26 ++++++++++++++++++++++----
 1 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 553e237..2340876 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -876,18 +876,25 @@ class P4Sync(Command):
         self.keepRepoPath = False
         self.depotPaths = None
         self.p4BranchesInGit = []
+        self.cloneExclude = []
 
         if gitConfig("git-p4.syncFromOrigin") == "false":
             self.syncWithOrigin = False
 
     def extractFilesFromCommit(self, commit):
+        self.cloneExclude = [re.sub(r"\.\.\.$", "", path)
+                             for path in self.cloneExclude]
         files = []
         fnum = 0
         while commit.has_key("depotFile%s" % fnum):
             path =  commit["depotFile%s" % fnum]
 
-            found = [p for p in self.depotPaths
-                     if path.startswith (p)]
+            if [p for p in self.cloneExclude
+                if path.startswith (p)]:
+                found = False
+            else:
+                found = [p for p in self.depotPaths
+                         if path.startswith (p)]
             if not found:
                 fnum = fnum + 1
                 continue
@@ -1658,13 +1665,23 @@ class P4Clone(P4Sync):
         P4Sync.__init__(self)
         self.description = "Creates a new git repository and imports from Perforce into it"
         self.usage = "usage: %prog [options] //depot/path[@revRange]"
-        self.options.append(
+        self.options += [
             optparse.make_option("--destination", dest="cloneDestination",
                                  action='store', default=None,
-                                 help="where to leave result of the clone"))
+                                 help="where to leave result of the clone"),
+            optparse.make_option("-/", dest="cloneExclude",
+                                 action="append", type="string",
+                                 help="exclude depot path")
+        ]
         self.cloneDestination = None
         self.needsGit = False
 
+    # This is required for the "append" cloneExclude action
+    def ensure_value(self, attr, value):
+        if not hasattr(self, attr) or getattr(self, attr) is None:
+            setattr(self, attr, value)
+        return getattr(self, attr)
+
     def defaultDestination(self, args):
         ## TODO: use common prefix of args?
         depotPath = args[0]
@@ -1688,6 +1705,7 @@ class P4Clone(P4Sync):
             self.cloneDestination = depotPaths[-1]
             depotPaths = depotPaths[:-1]
 
+        self.cloneExclude = ["/"+p for p in self.cloneExclude]
         for p in depotPaths:
             if not p.startswith("//"):
                 return False
-- 
1.5.4.rc5.17.g22b645

Re: [PATCH 2/3] git-p4: support exclude paths

[Simon Hausmann]

[-- Attachment #1: Type: text/plain, Size: 3529 bytes --]

On Sunday 03 February 2008 10:21:05 Tommy Thorn wrote:
> Teach git-p4 about the -/ option which adds depot paths to the exclude
> list, used when cloning. The option is chosen such that the natural
> Perforce syntax works, eg:
>
>   git p4 clone //branch/path/... -//branch/path/{large,old}/...
>
> Trailing ... on exclude paths are optional.
>
> This is a generalization of a change by Dmitry Kakurin (thanks).
>
> Signed-off-by: Tommy Thorn <tommy-git@thorn.ws>

Acked-By: Simon Hausmann <simon@lst.de>

I like it, Perforce'ish syntax. (Not that I like p4 though ;)


Simon

> ---
>  contrib/fast-import/git-p4 |   26 ++++++++++++++++++++++----
>  1 files changed, 22 insertions(+), 4 deletions(-)
>
> diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
> index 553e237..2340876 100755
> --- a/contrib/fast-import/git-p4
> +++ b/contrib/fast-import/git-p4
> @@ -876,18 +876,25 @@ class P4Sync(Command):
>          self.keepRepoPath = False
>          self.depotPaths = None
>          self.p4BranchesInGit = []
> +        self.cloneExclude = []
>
>          if gitConfig("git-p4.syncFromOrigin") == "false":
>              self.syncWithOrigin = False
>
>      def extractFilesFromCommit(self, commit):
> +        self.cloneExclude = [re.sub(r"\.\.\.$", "", path)
> +                             for path in self.cloneExclude]
>          files = []
>          fnum = 0
>          while commit.has_key("depotFile%s" % fnum):
>              path =  commit["depotFile%s" % fnum]
>
> -            found = [p for p in self.depotPaths
> -                     if path.startswith (p)]
> +            if [p for p in self.cloneExclude
> +                if path.startswith (p)]:
> +                found = False
> +            else:
> +                found = [p for p in self.depotPaths
> +                         if path.startswith (p)]
>              if not found:
>                  fnum = fnum + 1
>                  continue
> @@ -1658,13 +1665,23 @@ class P4Clone(P4Sync):
>          P4Sync.__init__(self)
>          self.description = "Creates a new git repository and imports from
> Perforce into it" self.usage = "usage: %prog [options]
> //depot/path[@revRange]" -        self.options.append(
> +        self.options += [
>              optparse.make_option("--destination", dest="cloneDestination",
>                                   action='store', default=None,
> -                                 help="where to leave result of the
> clone")) +                                 help="where to leave result of
> the clone"), +            optparse.make_option("-/", dest="cloneExclude",
> +                                 action="append", type="string",
> +                                 help="exclude depot path")
> +        ]
>          self.cloneDestination = None
>          self.needsGit = False
>
> +    # This is required for the "append" cloneExclude action
> +    def ensure_value(self, attr, value):
> +        if not hasattr(self, attr) or getattr(self, attr) is None:
> +            setattr(self, attr, value)
> +        return getattr(self, attr)
> +
>      def defaultDestination(self, args):
>          ## TODO: use common prefix of args?
>          depotPath = args[0]
> @@ -1688,6 +1705,7 @@ class P4Clone(P4Sync):
>              self.cloneDestination = depotPaths[-1]
>              depotPaths = depotPaths[:-1]
>
> +        self.cloneExclude = ["/"+p for p in self.cloneExclude]
>          for p in depotPaths:
>              if not p.startswith("//"):
>                  return False

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH 2/3] git-p4: support exclude paths

[Tommy Thorn]

Simon Hausmann wrote:
> On Sunday 03 February 2008 10:21:05 Tommy Thorn wrote:
>   
>> Teach git-p4 about the -/ option which adds depot paths to the exclude
>> list, used when cloning. The option is chosen such that the natural
>> Perforce syntax works, eg:
>>
>>   git p4 clone //branch/path/... -//branch/path/{large,old}/...
>>
>> Trailing ... on exclude paths are optional.
>>
>> This is a generalization of a change by Dmitry Kakurin (thanks).
>>
>> Signed-off-by: Tommy Thorn <tommy-git@thorn.ws>
>>     
>
> Acked-By: Simon Hausmann <simon@lst.de>
>
> I like it, Perforce'ish syntax. (Not that I like p4 though ;)
>   
Thank you.

I would appear that I have some mail server problems, so apologies if 
you get multiple copies.

Also, this is the first Python hacking I've tried, so it's likely that 
my changes needs improvement.

With these two patches, I can now use git-p4. However in a perfect 
world, git-p4 would:

- include support everything that a (ugly) Perforce client can do, 
including naming individual files
  and remapping things around (a sick feature that never should be used 
IMO), and

- not consume memory proportional to the imported files.

The former would require pervasive changes and likely break some 
assumptions currently made.

The latter is easy enough.


Regards
Tommy

Re: [PATCH 2/3] git-p4: support exclude paths

[Tommy Thorn]

On Sunday 03 February 2008 10:21:05 I wrote:
> Teach git-p4 about the -/ option which adds depot paths to the exclude
> list, used when cloning. The option is chosen such that the natural
> Perforce syntax works, eg:
>
>   git p4 clone //branch/path/... -//branch/path/{large,old}/...
>
> Trailing ... on exclude paths are optional.
>
> This is a generalization of a change by Dmitry Kakurin (thanks).
>
> Signed-off-by: Tommy Thorn <tommy-git@thorn.ws>

.. to which Simon replied:
> Acked-By: Simon Hausmann <simon@lst.de>
>
> I like it, Perforce'ish syntax. (Not that I like p4 though ;)

Alas, this change needs more work - the exclude paths needs to 
be maintained in the commit messages as otherwise we pull in new
files in the excluded path. I haven't done this yet.

However, the other patch (git-p4: no longer keep all file contents while cloning)
is IMO critical. You simply cannot clone a non-trivial Perforce repository 
without it. Why is this being ignored? Are there no users of git-p4?


Tommy

**

Re: [PATCH 2/3] git-p4: support exclude paths

[Simon Hausmann]

[-- Attachment #1: Type: text/plain, Size: 1434 bytes --]

On Tuesday 12 February 2008 16:53:38 Tommy Thorn wrote:
> On Sunday 03 February 2008 10:21:05 I wrote:
> > Teach git-p4 about the -/ option which adds depot paths to the exclude
> > list, used when cloning. The option is chosen such that the natural
> > Perforce syntax works, eg:
> >
> >   git p4 clone //branch/path/... -//branch/path/{large,old}/...
> >
> > Trailing ... on exclude paths are optional.
> >
> > This is a generalization of a change by Dmitry Kakurin (thanks).
> >
> > Signed-off-by: Tommy Thorn <tommy-git@thorn.ws>
>
> .. to which Simon replied:
> > Acked-By: Simon Hausmann <simon@lst.de>
> >
> > I like it, Perforce'ish syntax. (Not that I like p4 though ;)
>
> Alas, this change needs more work - the exclude paths needs to
> be maintained in the commit messages as otherwise we pull in new
> files in the excluded path. I haven't done this yet.
>
> However, the other patch (git-p4: no longer keep all file contents while
> cloning) is IMO critical. You simply cannot clone a non-trivial Perforce
> repository without it. Why is this being ignored? Are there no users of
> git-p4?

Sorry for the delay, my real life has kept be busy :)

I now looked at the patch and I'm all in favour of applying it. However it 
doesn't seem to apply against the current git-p4. Can you re-send the patch 
to me?

Thanks,
Simon

P.S.: I have applied the other patch (exclude path support)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]