[PATCH v4 0/3] git-p4: fix Git LFS pointer parsing

[PATCH v4 0/3] git-p4: fix Git LFS pointer parsing

[larsxschneider]

From: Lars Schneider <larsxschneider@gmail.com>

v1: $gmane/291917
v2: $gmane/291991
v3: $gmane/292425

diff to v3:
* fix missing assignment of pointerFile variable
  ($gmane/292454, thanks Sebastian for making me aware)
* fix s/brake/break/ in commit message
  ($gmane/292451, thanks Eric)

Thanks,
Lars

Lars Schneider (3):
  travis-ci: update Git-LFS and P4 to the latest version
  travis-ci: express Linux/OS X dependency versions more clearly
  git-p4: fix Git LFS pointer parsing

 .travis.yml               | 17 ++++++++++-------
 git-p4.py                 | 13 ++++++++++---
 t/t9824-git-p4-git-lfs.sh |  4 ++++
 3 files changed, 24 insertions(+), 10 deletions(-)

--
2.5.1

[PATCH v4 1/3] travis-ci: update Git-LFS and P4 to the latest version

[larsxschneider]

From: Lars Schneider <larsxschneider@gmail.com>

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 .travis.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 78e433b..4acf617 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -22,8 +22,8 @@ addons:
 env:
   global:
     - DEVELOPER=1
-    - P4_VERSION="15.2"
-    - GIT_LFS_VERSION="1.1.0"
+    - P4_VERSION="16.1"
+    - GIT_LFS_VERSION="1.2.0"
     - DEFAULT_TEST_TARGET=prove
     - GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
     - GIT_TEST_OPTS="--verbose --tee"
-- 
2.5.1

[PATCH v4 2/3] travis-ci: express Linux/OS X dependency versions more clearly

[larsxschneider]

From: Lars Schneider <larsxschneider@gmail.com>

The Git Travis CI OSX build always installs the latest versions of Git LFS and
Perforce via brew and the Linux build installs fixed versions. Consequently new
LFS/Perforce versions can break the OS X build even if there is no change in
Git.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 .travis.yml | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 4acf617..1fdcec8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -22,8 +22,11 @@ addons:
 env:
   global:
     - DEVELOPER=1
-    - P4_VERSION="16.1"
-    - GIT_LFS_VERSION="1.2.0"
+    # The Linux build installs the defined dependency versions below.
+    # The OS X build installs the latest available versions. Keep that
+    # in mind when you encounter a broken OS X build!
+    - LINUX_P4_VERSION="16.1"
+    - LINUX_GIT_LFS_VERSION="1.2.0"
     - DEFAULT_TEST_TARGET=prove
     - GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
     - GIT_TEST_OPTS="--verbose --tee"
@@ -38,17 +41,17 @@ before_install:
     linux)
       mkdir --parents custom/p4
       pushd custom/p4
-        wget --quiet http://filehost.perforce.com/perforce/r$P4_VERSION/bin.linux26x86_64/p4d
-        wget --quiet http://filehost.perforce.com/perforce/r$P4_VERSION/bin.linux26x86_64/p4
+        wget --quiet http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION/bin.linux26x86_64/p4d
+        wget --quiet http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION/bin.linux26x86_64/p4
         chmod u+x p4d
         chmod u+x p4
         export PATH="$(pwd):$PATH"
       popd
       mkdir --parents custom/git-lfs
       pushd custom/git-lfs
-        wget --quiet https://github.com/github/git-lfs/releases/download/v$GIT_LFS_VERSION/git-lfs-linux-amd64-$GIT_LFS_VERSION.tar.gz
-        tar --extract --gunzip --file "git-lfs-linux-amd64-$GIT_LFS_VERSION.tar.gz"
-        cp git-lfs-$GIT_LFS_VERSION/git-lfs .
+        wget --quiet https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz
+        tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
+        cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
         export PATH="$(pwd):$PATH"
       popd
       ;;
-- 
2.5.1

[PATCH v4 3/3] git-p4: fix Git LFS pointer parsing

[larsxschneider]

From: Lars Schneider <larsxschneider@gmail.com>

Git LFS 1.2.0 removed a preamble from the output of the 'git lfs pointer'
command [1] which broke the parsing of this output. Adjust the parser
to support the old and the new format.

Please note that this patch slightly changes the second return parameter
from a list of LF terminated strings to a single string that contains
a number of LF characters.

[1] https://github.com/github/git-lfs/commit/da2935d9a739592bc775c98d8ef4df9c72ea3b43

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Helped-by: Sebastian Schuberth <sschuberth@gmail.com>
Helped-by: Ben Woosley <ben.woosley@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 git-p4.py                 | 13 ++++++++++---
 t/t9824-git-p4-git-lfs.sh |  4 ++++
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 527d44b..8ab48f2 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -1064,8 +1064,15 @@ class GitLFS(LargeFileSystem):
         if pointerProcess.wait():
             os.remove(contentFile)
             die('git-lfs pointer command failed. Did you install the extension?')
-        pointerContents = [i+'\n' for i in pointerFile.split('\n')[2:][:-1]]
-        oid = pointerContents[1].split(' ')[1].split(':')[1][:-1]
+
+        # Git LFS removed the preamble in the output of the 'pointer' command
+        # starting from version 1.2.0. Check for the preamble here to support
+        # earlier versions.
+        # c.f. https://github.com/github/git-lfs/commit/da2935d9a739592bc775c98d8ef4df9c72ea3b43
+        if pointerFile.startswith('Git LFS pointer for'):
+            pointerFile = re.sub(r'Git LFS pointer for.*\n\n', '', pointerFile)
+
+        oid = re.search(r'^oid \w+:(\w+)', pointerFile, re.MULTILINE).group(1)
         localLargeFile = os.path.join(
             os.getcwd(),
             '.git', 'lfs', 'objects', oid[:2], oid[2:4],
@@ -1073,7 +1080,7 @@ class GitLFS(LargeFileSystem):
         )
         # LFS Spec states that pointer files should not have the executable bit set.
         gitMode = '100644'
-        return (gitMode, pointerContents, localLargeFile)
+        return (gitMode, pointerFile, localLargeFile)
 
     def pushFile(self, localLargeFile):
         uploadProcess = subprocess.Popen(
diff --git a/t/t9824-git-p4-git-lfs.sh b/t/t9824-git-p4-git-lfs.sh
index 0b664a3..ca93ac8 100755
--- a/t/t9824-git-p4-git-lfs.sh
+++ b/t/t9824-git-p4-git-lfs.sh
@@ -13,6 +13,10 @@ test_file_in_lfs () {
 	FILE="$1" &&
 	SIZE="$2" &&
 	EXPECTED_CONTENT="$3" &&
+	sed -n '1,1 p' "$FILE" | grep "^version " &&
+	sed -n '2,2 p' "$FILE" | grep "^oid " &&
+	sed -n '3,3 p' "$FILE" | grep "^size " &&
+	test_line_count = 3 "$FILE" &&
 	cat "$FILE" | grep "size $SIZE" &&
 	HASH=$(cat "$FILE" | grep "oid sha256:" | sed -e "s/oid sha256://g") &&
 	LFS_FILE=".git/lfs/objects/$(echo "$HASH" | cut -c1-2)/$(echo "$HASH" | cut -c3-4)/$HASH" &&
-- 
2.5.1

Re: [PATCH v4 0/3] git-p4: fix Git LFS pointer parsing

[Sebastian Schuberth]

On Thu, Apr 28, 2016 at 8:26 AM,  <larsxschneider@gmail.com> wrote:

> diff to v3:
> * fix missing assignment of pointerFile variable
>   ($gmane/292454, thanks Sebastian for making me aware)
> * fix s/brake/break/ in commit message
>   ($gmane/292451, thanks Eric)

The series looks good to me now.

Regards,
Sebastian