[PATCH v1 0/2] git-p4: handle "Translation of file content failed"

[PATCH v1 0/2] git-p4: handle "Translation of file content failed"

[larsxschneider]

From: Lars Schneider <larsxschneider@gmail.com>

Hi,

this patch fixes the P4 "Translation of file content failed" error. Unfortuantly
I was not able to generate a P4 test repository to reproduce the error with a
test case.

An Internet search shows that this error happens in the wild:
https://stackoverflow.com/questions/5156909/translation-of-file-content-failed-error-in-perforce
https://stackoverflow.com/questions/887006/perforce-translation-of-file-content-failed-error

Thanks,
Lars

Lars Schneider (2):
  git-p4: print stderr if P4 read_pipe operation fails
  git-p4: handle "Translation of file content failed"

 git-p4.py | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

--
2.5.1

[PATCH v1 1/2] git-p4: print stderr if P4 read_pipe operation fails

[larsxschneider]

From: Lars Schneider <larsxschneider@gmail.com>

If read_pipe crashes then the caller can inspect the error and handle
it appropriately.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 git-p4.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 073f87b..36a4bcb 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -134,11 +134,11 @@ def read_pipe(c, ignore_error=False):
         sys.stderr.write('Reading pipe: %s\n' % str(c))
 
     expand = isinstance(c,basestring)
-    p = subprocess.Popen(c, stdout=subprocess.PIPE, shell=expand)
+    p = subprocess.Popen(c, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=expand)
     pipe = p.stdout
     val = pipe.read()
     if p.wait() and not ignore_error:
-        die('Command failed: %s' % str(c))
+        die('Command failed: %s\nError: %s' % (str(c), p.stderr.read()))
 
     return val
 
-- 
2.5.1

[PATCH v1 2/2] git-p4: handle "Translation of file content failed"

[larsxschneider]

From: Lars Schneider <larsxschneider@gmail.com>

A P4 repository can get into a state where it contains a file with file
type UTF16 that does not not contain valid UTF16 characters. If git-p4
attempts to retrieve the file as UTF16 from P4 then the process crashes
with a "Translation of file content failed" error.

Fix this by detecting this error and retrieving the file as binary
instead. The result in Git is the same.
---
 git-p4.py | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 36a4bcb..aaa0ad9 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2186,10 +2186,17 @@ class P4Sync(Command, P4UserMap):
             # them back too.  This is not needed to the cygwin windows version,
             # just the native "NT" type.
             #
-            text = p4_read_pipe(['print', '-q', '-o', '-', "%s@%s" % (file['depotFile'], file['change']) ])
-            if p4_version_string().find("/NT") >= 0:
-                text = text.replace("\r\n", "\n")
-            contents = [ text ]
+            try:
+                text = p4_read_pipe(['print', '-q', '-o', '-', '%s@%s' % (file['depotFile'], file['change'])])
+            except Exception as e:
+                if 'Translation of file content failed' in str(e):
+                    type_base = 'binary'
+                else:
+                    raise e
+            else:
+                if p4_version_string().find('/NT') >= 0:
+                    text = text.replace('\r\n', '\n')
+                contents = [ text ]
 
         if type_base == "apple":
             # Apple filetype files will be streamed as a concatenation of
-- 
2.5.1

Re: [PATCH v1 1/2] git-p4: print stderr if P4 read_pipe operation fails

[Junio C Hamano]

larsxschneider@gmail.com writes:

> From: Lars Schneider <larsxschneider@gmail.com>
>
> If read_pipe crashes then the caller can inspect the error and handle
> it appropriately.
>
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---
>  git-p4.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/git-p4.py b/git-p4.py
> index 073f87b..36a4bcb 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -134,11 +134,11 @@ def read_pipe(c, ignore_error=False):
>          sys.stderr.write('Reading pipe: %s\n' % str(c))
>  
>      expand = isinstance(c,basestring)
> -    p = subprocess.Popen(c, stdout=subprocess.PIPE, shell=expand)
> +    p = subprocess.Popen(c, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=expand)
>      pipe = p.stdout
>      val = pipe.read()
>      if p.wait() and not ignore_error:
> -        die('Command failed: %s' % str(c))
> +        die('Command failed: %s\nError: %s' % (str(c), p.stderr.read()))

I don't know enough about the callers of this helper function to
tell offhand if that is an issue, but this looks unsafe depending on
what the process on the other side of these pipes are doing.

If it attempts to spew a lot on its standard error stream first and
then write some to its standard output, I would not be surprised it
would get stuck waiting for us to read and drain its standard error
before it can proceed to write to its standard output, and in the
meantime we would be waiting for it to say something on its standard
output, no?

Re: [PATCH v1 1/2] git-p4: print stderr if P4 read_pipe operation fails

[Lars Schneider]

On 09 Sep 2015, at 18:00, Junio C Hamano <gitster@pobox.com> wrote:

> larsxschneider@gmail.com writes:
> 
>> From: Lars Schneider <larsxschneider@gmail.com>
>> 
>> If read_pipe crashes then the caller can inspect the error and handle
>> it appropriately.
>> 
>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
>> ---
>> git-p4.py | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>> 
>> diff --git a/git-p4.py b/git-p4.py
>> index 073f87b..36a4bcb 100755
>> --- a/git-p4.py
>> +++ b/git-p4.py
>> @@ -134,11 +134,11 @@ def read_pipe(c, ignore_error=False):
>>         sys.stderr.write('Reading pipe: %s\n' % str(c))
>> 
>>     expand = isinstance(c,basestring)
>> -    p = subprocess.Popen(c, stdout=subprocess.PIPE, shell=expand)
>> +    p = subprocess.Popen(c, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=expand)
>>     pipe = p.stdout
>>     val = pipe.read()
>>     if p.wait() and not ignore_error:
>> -        die('Command failed: %s' % str(c))
>> +        die('Command failed: %s\nError: %s' % (str(c), p.stderr.read()))
> 
> I don't know enough about the callers of this helper function to
> tell offhand if that is an issue, but this looks unsafe depending on
> what the process on the other side of these pipes are doing.
> 
> If it attempts to spew a lot on its standard error stream first and
> then write some to its standard output, I would not be surprised it
> would get stuck waiting for us to read and drain its standard error
> before it can proceed to write to its standard output, and in the
> meantime we would be waiting for it to say something on its standard
> output, no?
> 
You are right. I will use the “communicate” function here as recommended in the Python docs:
https://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait

Thanks!