[PATCH 00/12] Allow users to access Perforce repositories requiring authentication

[PATCH 00/12] Allow users to access Perforce repositories requiring authentication

[Anand Kumria]

Hi,

This series of patches refactors a few function calls into git-p4
so that they all go via the same function to build up the command
line.

It also then allows users to specify any particular user / password
required to access their Perforce repository (plus a few other parameters).

I have specifically tested this agains the public Zimbra repository and
it works for me fine. Any feedback would be welcomed.

Thanks,
Anand

Anand Kumria (12):
  Create a specific version of the read_pipe_lines command for p4
    invocations
  Utilise the new 'p4_read_pipe_lines' command
  Have a command that specifically invokes 'p4' (via system)
  Utilise the new 'p4_system' function.
  Add a single command that will be used to construct the 'p4' command
  If we are in verbose mode, output what we are about to run (or
    return)
  Switch to using 'p4_build_cmd'
  If the user has configured various parameters, use them.
  Consistently use 'git-p4' for the configuration entries
  Move git-p4.syncFromOrigin into a configuration parameters section
  Put some documentation in about the parameters that have been added
  Put in the two other configuration elements found in the source

 contrib/fast-import/git-p4     |   71 ++++++++++++++++++++++++++++++++-------
 contrib/fast-import/git-p4.txt |   68 +++++++++++++++++++++++++++++++-------
 2 files changed, 114 insertions(+), 25 deletions(-)

[PATCH 01/12] Create a specific version of the read_pipe_lines command for p4 invocations

[Anand Kumria]

 This will make it easier to isolate changes to how 'p4' is invoked
 (whether with parameters or not, etc.).

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 6ae0429..fc2a60d 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -57,6 +57,13 @@ def read_pipe_lines(c):
 
     return val
 
+def p4_read_pipe_lines(c):
+    """Specifically invoke p4 on the command supplied. """
+    real_cmd = "%s %s" % ("p4", c)
+    if verbose:
+        print real_cmd
+    return read_pipe_lines(real_cmd)
+
 def system(cmd):
     if verbose:
         sys.stderr.write("executing %s\n" % cmd)
-- 
1.5.6.3

[PATCH 02/12] Utilise the new 'p4_read_pipe_lines' command

[Anand Kumria]

 Now that we have the new command, we can utilise it and then
 eventually, isolate any changes required to the one place.

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index fc2a60d..3deaa42 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -371,7 +371,7 @@ def originP4BranchesExist():
 
 def p4ChangesForPaths(depotPaths, changeRange):
     assert depotPaths
-    output = read_pipe_lines("p4 changes " + ' '.join (["%s...%s" % (p, changeRange)
+    output = p4_read_pipe_lines("changes " + ' '.join (["%s...%s" % (p, changeRange)
                                                         for p in depotPaths]))
 
     changes = []
@@ -519,7 +519,7 @@ class P4Submit(Command):
         # remove lines in the Files section that show changes to files outside the depot path we're committing into
         template = ""
         inFilesSection = False
-        for line in read_pipe_lines("p4 change -o"):
+        for line in p4_read_pipe_lines("change -o"):
             if line.endswith("\r\n"):
                 line = line[:-2] + "\n"
             if inFilesSection:
-- 
1.5.6.3

[PATCH 03/12] Have a command that specifically invokes 'p4' (via system)

[Anand Kumria]

 Similiar to our 'p4_read_pipe_lines' command, we can isolate
 specific changes to the invocation method in the one location
 with this change.

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 3deaa42..08acd51 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -70,6 +70,13 @@ def system(cmd):
     if os.system(cmd) != 0:
         die("command failed: %s" % cmd)
 
+def p4_system(cmd):
+    """Specifically invoke p4 as the system command. """
+    real_cmd = "%s %s" % ("p4", cmd)
+    if verbose:
+        print real_cmd
+    return system(real_cmd)
+
 def isP4Exec(kind):
     """Determine if a Perforce 'kind' should have execute permission
 
-- 
1.5.6.3

[PATCH 04/12] Utilise the new 'p4_system' function.

[Anand Kumria]

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |   18 +++++++++---------
 1 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 08acd51..2ed36ec 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -98,7 +98,7 @@ def setP4ExecBit(file, mode):
         if p4Type[-1] == "+":
             p4Type = p4Type[0:-1]
 
-    system("p4 reopen -t %s %s" % (p4Type, file))
+    p4_system("reopen -t %s %s" % (p4Type, file))
 
 def getP4OpenedType(file):
     # Returns the perforce file type for the given file.
@@ -561,7 +561,7 @@ class P4Submit(Command):
             modifier = diff['status']
             path = diff['src']
             if modifier == "M":
-                system("p4 edit \"%s\"" % path)
+                p4_system("edit \"%s\"" % path)
                 if isModeExecChanged(diff['src_mode'], diff['dst_mode']):
                     filesToChangeExecBit[path] = diff['dst_mode']
                 editedFiles.add(path)
@@ -576,8 +576,8 @@ class P4Submit(Command):
                     filesToAdd.remove(path)
             elif modifier == "R":
                 src, dest = diff['src'], diff['dst']
-                system("p4 integrate -Dt \"%s\" \"%s\"" % (src, dest))
-                system("p4 edit \"%s\"" % (dest))
+                p4_system("integrate -Dt \"%s\" \"%s\"" % (src, dest))
+                p4_system("edit \"%s\"" % (dest))
                 if isModeExecChanged(diff['src_mode'], diff['dst_mode']):
                     filesToChangeExecBit[dest] = diff['dst_mode']
                 os.unlink(dest)
@@ -601,7 +601,7 @@ class P4Submit(Command):
             if response == "s":
                 print "Skipping! Good luck with the next patches..."
                 for f in editedFiles:
-                    system("p4 revert \"%s\"" % f);
+                    p4_system("revert \"%s\"" % f);
                 for f in filesToAdd:
                     system("rm %s" %f)
                 return
@@ -624,10 +624,10 @@ class P4Submit(Command):
         system(applyPatchCmd)
 
         for f in filesToAdd:
-            system("p4 add \"%s\"" % f)
+            p4_system("add \"%s\"" % f)
         for f in filesToDelete:
-            system("p4 revert \"%s\"" % f)
-            system("p4 delete \"%s\"" % f)
+            p4_system("revert \"%s\"" % f)
+            p4_system("delete \"%s\"" % f)
 
         # Set/clear executable bits
         for f in filesToChangeExecBit.keys():
@@ -728,7 +728,7 @@ class P4Submit(Command):
 
         os.chdir(self.clientPath)
         print "Syncronizing p4 checkout..."
-        system("p4 sync ...")
+        p4_system("sync ...")
 
         self.check()
 
-- 
1.5.6.3

[PATCH 05/12] Add a single command that will be used to construct the 'p4' command

[Anand Kumria]

 Rather than having three locations where the 'p4' command is built up,
 refactor this into the one place. This will, eventually, allow us to
 have one place where we modify the evironment or pass extra
 command-line options to the 'p4' binary.

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 2ed36ec..b4acf76 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -16,6 +16,17 @@ from sets import Set;
 
 verbose = False
 
+
+def p4_build_cmd(cmd):
+    """Build a suitable p4 command line.
+
+    This consolidates building and returning a p4 command line into one
+    location. It means that hooking into the environment, or other configuration
+    can be done more easily.
+    """
+    real_cmd = "%s %s" % ("p4", cmd)
+    return real_cmd
+
 def die(msg):
     if verbose:
         raise Exception(msg)
-- 
1.5.6.3

[PATCH 06/12] If we are in verbose mode, output what we are about to run (or return)

[Anand Kumria]

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index b4acf76..d36b0c6 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -25,6 +25,8 @@ def p4_build_cmd(cmd):
     can be done more easily.
     """
     real_cmd = "%s %s" % ("p4", cmd)
+    if verbose:
+        print real_cmd
     return real_cmd
 
 def die(msg):
-- 
1.5.6.3

[PATCH 07/12] Switch to using 'p4_build_cmd'

[Anand Kumria]

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |   10 +++-------
 1 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index d36b0c6..2b6ea74 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -72,9 +72,7 @@ def read_pipe_lines(c):
 
 def p4_read_pipe_lines(c):
     """Specifically invoke p4 on the command supplied. """
-    real_cmd = "%s %s" % ("p4", c)
-    if verbose:
-        print real_cmd
+    real_cmd = p4_build_cmd(c)
     return read_pipe_lines(real_cmd)
 
 def system(cmd):
@@ -85,9 +83,7 @@ def system(cmd):
 
 def p4_system(cmd):
     """Specifically invoke p4 as the system command. """
-    real_cmd = "%s %s" % ("p4", cmd)
-    if verbose:
-        print real_cmd
+    real_cmd = p4_build_cmd(cmd)
     return system(real_cmd)
 
 def isP4Exec(kind):
@@ -172,7 +168,7 @@ def isModeExecChanged(src_mode, dst_mode):
     return isModeExec(src_mode) != isModeExec(dst_mode)
 
 def p4CmdList(cmd, stdin=None, stdin_mode='w+b'):
-    cmd = "p4 -G %s" % cmd
+    cmd = p4_build_cmd("-G %s" % (cmd))
     if verbose:
         sys.stderr.write("Opening pipe: %s\n" % cmd)
 
-- 
1.5.6.3

[PATCH 08/12] If the user has configured various parameters, use them.

[Anand Kumria]

 Some repositories require authentication and access to certain
 hosts. Allow git-p4 to pull this information from the configuration

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |   24 +++++++++++++++++++++++-
 1 files changed, 23 insertions(+), 1 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 2b6ea74..a927e50 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -24,7 +24,29 @@ def p4_build_cmd(cmd):
     location. It means that hooking into the environment, or other configuration
     can be done more easily.
     """
-    real_cmd = "%s %s" % ("p4", cmd)
+    real_cmd = "%s " % "p4"
+
+    user = gitConfig("git-p4.user")
+    if len(user) > 0:
+        real_cmd += "-u %s " % user
+
+    password = gitConfig("git-p4.password")
+    if len(password) > 0:
+        real_cmd += "-P %s " % password
+
+    port = gitConfig("git-p4.port")
+    if len(port) > 0:
+        real_cmd += "-p %s " % port
+
+    host = gitConfig("git-p4.host")
+    if len(host) > 0:
+        real_cmd += "-h %s " % host
+
+    client = gitConfig("git-p4.client")
+    if len(client) > 0:
+        real_cmd += "-c %s " % client
+
+    real_cmd += "%s" % (cmd)
     if verbose:
         print real_cmd
     return real_cmd
-- 
1.5.6.3

[PATCH 09/12] Consistently use 'git-p4' for the configuration entries

[Anand Kumria]

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4 |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index a927e50..6c64224 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -1444,7 +1444,7 @@ class P4Sync(Command):
             if not gitBranchExists(self.refPrefix + "HEAD") and self.importIntoRemotes and gitBranchExists(self.branch):
                 system("git symbolic-ref %sHEAD %s" % (self.refPrefix, self.branch))
 
-        if self.useClientSpec or gitConfig("p4.useclientspec") == "true":
+        if self.useClientSpec or gitConfig("git-p4.useclientspec") == "true":
             self.getClientSpec()
 
         # TODO: should always look at previous commits,
-- 
1.5.6.3

[PATCH 10/12] Move git-p4.syncFromOrigin into a configuration parameters section

[Anand Kumria]

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4.txt |   28 ++++++++++++++++------------
 1 files changed, 16 insertions(+), 12 deletions(-)

diff --git a/contrib/fast-import/git-p4.txt b/contrib/fast-import/git-p4.txt
index b16a838..0896abb 100644
--- a/contrib/fast-import/git-p4.txt
+++ b/contrib/fast-import/git-p4.txt
@@ -63,18 +63,6 @@ It is recommended to run 'git repack -a -d -f' from time to time when using
 incremental imports to optimally combine the individual git packs that each
 incremental import creates through the use of git-fast-import.
 
-
-A useful setup may be that you have a periodically updated git repository
-somewhere that contains a complete import of a Perforce project. That git
-repository can be used to clone the working repository from and one would
-import from Perforce directly after cloning using git-p4. If the connection to
-the Perforce server is slow and the working repository hasn't been synced for a
-while it may be desirable to fetch changes from the origin git repository using
-the efficient git protocol. git-p4 supports this setup by calling "git fetch origin"
-by default if there is an origin branch. You can disable this using
-
-  git config git-p4.syncFromOrigin false
-
 Updating
 ========
 
@@ -140,6 +128,22 @@ Example
   git-p4 rebase
 
 
+Configuration parameters
+========================
+
+git-p4.syncFromOrigin
+
+A useful setup may be that you have a periodically updated git repository
+somewhere that contains a complete import of a Perforce project. That git
+repository can be used to clone the working repository from and one would
+import from Perforce directly after cloning using git-p4. If the connection to
+the Perforce server is slow and the working repository hasn't been synced for a
+while it may be desirable to fetch changes from the origin git repository using
+the efficient git protocol. git-p4 supports this setup by calling "git fetch origin"
+by default if there is an origin branch. You can disable this using:
+
+  git config [--global] git-p4.syncFromOrigin false
+
 Implementation Details...
 =========================
 
-- 
1.5.6.3

[PATCH 11/12] Put some documentation in about the parameters that have been added

[Anand Kumria]

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4.txt |   32 ++++++++++++++++++++++++++++++++
 1 files changed, 32 insertions(+), 0 deletions(-)

diff --git a/contrib/fast-import/git-p4.txt b/contrib/fast-import/git-p4.txt
index 0896abb..79a22e9 100644
--- a/contrib/fast-import/git-p4.txt
+++ b/contrib/fast-import/git-p4.txt
@@ -131,6 +131,38 @@ Example
 Configuration parameters
 ========================
 
+git-p4.user ($P4USER)
+
+Allows you to specify the username to use to connect to the Perforce repository.
+
+  git config [--global] git-p4.user public
+
+git-p4.password ($P4PASS)
+
+Allows you to specify the password to use to connect to the Perforce repository.
+Warning this password will be visible on the command-line invocation of the p4 binary.
+
+  git config [--global] git-p4.password public1234
+
+git-p4.port ($P4PORT)
+
+Specify the port to be used to contact the Perforce server. As this will be passed
+directly to the p4 binary, it may be in the format host:port as well.
+
+  git config [--global] git-p4.port codes.zimbra.com:2666
+
+git-p4.host ($P4HOST)
+
+Specify the host to contact for a Perforce repository.
+
+  git config [--global] git-p4.host perforce.example.com
+
+git-p4.client ($P4CLIENT)
+
+Specify the client name to use
+
+  git config [--global] git-p4.client public-view
+
 git-p4.syncFromOrigin
 
 A useful setup may be that you have a periodically updated git repository
-- 
1.5.6.3

[PATCH 12/12] Put in the two other configuration elements found in the source

[Anand Kumria]

 I am not entirely clear what these parameters do but felt it
 useful to call them out in the documentation.

Signed-off-by: Anand Kumria <wildfire@progsoc.org>
---
 contrib/fast-import/git-p4.txt |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/contrib/fast-import/git-p4.txt b/contrib/fast-import/git-p4.txt
index 79a22e9..ac551d4 100644
--- a/contrib/fast-import/git-p4.txt
+++ b/contrib/fast-import/git-p4.txt
@@ -163,6 +163,10 @@ Specify the client name to use
 
   git config [--global] git-p4.client public-view
 
+git-p4.allowSubmit
+
+  git config [--global] git-p4.allowSubmit false
+
 git-p4.syncFromOrigin
 
 A useful setup may be that you have a periodically updated git repository
@@ -176,6 +180,10 @@ by default if there is an origin branch. You can disable this using:
 
   git config [--global] git-p4.syncFromOrigin false
 
+git-p4.useclientspec
+
+  git config [--global] git-p4.useclientspec false
+
 Implementation Details...
 =========================
 
-- 
1.5.6.3

Re: [PATCH 00/12] Allow users to access Perforce repositories requiring authentication

[Simon Hausmann]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

Hi,

On Sunday 10 August 2008 20:26:23 Anand Kumria wrote:
> Hi,
>
> This series of patches refactors a few function calls into git-p4
> so that they all go via the same function to build up the command
> line.
>
> It also then allows users to specify any particular user / password
> required to access their Perforce repository (plus a few other parameters).
>
> I have specifically tested this agains the public Zimbra repository and
> it works for me fine. Any feedback would be welcomed.

Junio, I saw that you queued up this series in pu as ak/p4. I looked through 
the patches and I think they look great. I suggest to include them into master 
after 1.6.0.

Anand, great work :)


Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH 00/12] Allow users to access Perforce repositories requiring authentication

[Tor Arvid Lund]

Hi,

On Sun, Aug 10, 2008 at 8:26 PM, Anand Kumria <wildfire@progsoc.org> wrote:
> Hi,
>
> This series of patches refactors a few function calls into git-p4
> so that they all go via the same function to build up the command
> line.
>
> It also then allows users to specify any particular user / password
> required to access their Perforce repository (plus a few other parameters).
>
> I have specifically tested this agains the public Zimbra repository and
> it works for me fine. Any feedback would be welcomed.

I had an issue when using git p4 submit, and I had newly added files
in the changelist. This, I think, is caused by a call to "p4 opened"
that was not converted to use your new p4_build_cmd function. So I
don't think your patches should be merged to master before this is
fixed. However, when applying the simple diff below, this works for
me.

Otherwise, I think your patch series were very nice, since I often use
different perforce servers. Btw, my diff works correctly, I think, but
it is probably nicer to wrap read_pipe inside a p4_read_pipe - similar
to what you did with p4_read_pipe_lines. All in all: good work! :)

-Tor Arvid-

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 6c64224..8705ec9 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -134,7 +134,8 @@ def setP4ExecBit(file, mode):
 def getP4OpenedType(file):
     # Returns the perforce file type for the given file.

-    result = read_pipe("p4 opened %s" % file)
+    real_cmd = p4_build_cmd("opened %s" % file)
+    result = read_pipe(real_cmd)
     match = re.match(".*\((.+)\)\r?$", result)
     if match:
         return match.group(1)

Re: [PATCH 00/12] Allow users to access Perforce repositories requiring authentication

[Anand Kumria]

Hi Tor,

On Thu, 14 Aug 2008 19:00:12 +0200, Tor Arvid Lund wrote:

> I had an issue when using git p4 submit, and I had newly added files in
> the changelist. This, I think, is caused by a call to "p4 opened" that
> was not converted to use your new p4_build_cmd function. So I don't
> think your patches should be merged to master before this is fixed.
> However, when applying the simple diff below, this works for me.

Thanks for catching this. I found a few other occurances similar to the 
one you mentioned. I will send out an updated patch momentarily.

Thanks,
Anand