is git-p4 compatible with p4/linux?

is git-p4 compatible with p4/linux?

[Alexander Tomlinson]

git-p4.py (1.8.2.1.418.gaec3f77) has at least two behaviors that
seem to be incompatible with the version of p4 that I recently
downloaded from perforce.com (P4/LINUX26X86_64/2013.1/610569).

TLDR: Is git-p4 written for an old version of p4 CLI with different
behavior?  Or for a windows or mac release of p4?  Or am I missing
something?

First issue
-----------

git-p4 assumes the output of 'p4 print' adds a newline to the
target.  To work around this, git-p4.py strips the last char from
symlinks as shown in the following snippet:

    if type_base =3D=3D "symlink":
        git_mode =3D "120000"
        # p4 print on a symlink contains "target\n"; remove the newline
        data =3D ''.join(contents)
        contents =3D [data[:-1]]

But my 'p4 print' does not output the newline:

    $ ls -l pcre
    lrwxrwxrwx 1 user group 12 Apr 16 10:27 pcre -> ../libs/pcre

    $ p4 print -q pcre | od -t x1a
    0000000  2e  2e  2f  6c  69  62  73  2f  70  63  72  65
             .   .   /   l   i   b   s   /   p   c   r   e
    0000014

If I use 'git p4 clone' the above file shows up in git as a
symlink to '../libs/pcr'.  I had another symlink whose target had
a strlen of 1 and the 'git p4 clone' failed b/c after stripping
the last char the result was an empty string.

Second issue
------------

git-p4 uses 'p4 print -q -o o FILE' to print a file to stdout.
At least that is how I interpret this snippet:

    text = p4_read_pipe(['print', '-q', '-o', '-', file['depotFile']])

However, p4/Linux prints to stdout by default and '-o -' will save
the output in a file named '-'.

My git and p4 versions:

    $ git --version
    git version 1.8.2.1.418.gaec3f77

    $ p4 -V
    Perforce - The Fast Software Configuration Management System.
    Copyright 1995-2013 Perforce Software.  All rights reserved.
    This product includes software developed by the OpenSSL Project
    for use in the OpenSSL Toolkit (http://www.openssl.org/)
    See 'p4 help legal' for full OpenSSL license information
    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).

--Alex

Re: is git-p4 compatible with p4/linux?

[Alexander Tomlinson]

Sorry about the =3D escapes.  My first attempt to submit bounced b/c it was html and I missed
cleaning some escapes.  The snippet should be:

    if type_base == "symlink":
        git_mode = "120000"
        # p4 print on a symlink contains "target\n"; remove the newline
        data = ''.join(contents)
        contents = [data[:-1]]


--Alex

Re: is git-p4 compatible with p4/linux?

[Pete Wyckoff]

alex@aivor.com wrote on Tue, 16 Apr 2013 23:31 -0500:
> git-p4.py (1.8.2.1.418.gaec3f77) has at least two behaviors that
> seem to be incompatible with the version of p4 that I recently
> downloaded from perforce.com (P4/LINUX26X86_64/2013.1/610569).
> 
> TLDR: Is git-p4 written for an old version of p4 CLI with different
> behavior?  Or for a windows or mac release of p4?  Or am I missing
> something?

I had not done any testing beyond p4 12.2 (linux).  But running
the unit tests through 13.1 just now, they all pass.

    $ p4 -V
    Perforce - The Fast Software Configuration Management System.
    Copyright 1995-2013 Perforce Software.  All rights reserved.
    This product includes software developed by the OpenSSL Project
    for use in the OpenSSL Toolkit (http://www.openssl.org/)
    See 'p4 help legal' for full OpenSSL license information
    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).

I'm using python 2.7.3.

> First issue
> -----------
> 
> git-p4 assumes the output of 'p4 print' adds a newline to the
> target.  To work around this, git-p4.py strips the last char from
> symlinks as shown in the following snippet:
> 
>     if type_base =3D=3D "symlink":
>         git_mode =3D "120000"
>         # p4 print on a symlink contains "target\n"; remove the newline
>         data =3D ''.join(contents)
>         contents =3D [data[:-1]]
> 
> But my 'p4 print' does not output the newline:
> 
>     $ ls -l pcre
>     lrwxrwxrwx 1 user group 12 Apr 16 10:27 pcre -> ../libs/pcre
> 
>     $ p4 print -q pcre | od -t x1a
>     0000000  2e  2e  2f  6c  69  62  73  2f  70  63  72  65
>              .   .   /   l   i   b   s   /   p   c   r   e
>     0000014
> 
> If I use 'git p4 clone' the above file shows up in git as a
> symlink to '../libs/pcr'.  I had another symlink whose target had
> a strlen of 1 and the 'git p4 clone' failed b/c after stripping
> the last char the result was an empty string.

There wasn't an explict test for symlinks, but I threw
one together quickly and it seems to work.  Can you show
a bit more information about anything that potentially might
be odd with your install?

    arf-git-test$ ls -l symlink
    lrwxrwxrwx 1 pw pw 14 Apr 18 20:02 symlink -> symlink-target

    $ p4 fstat symlink
    ... depotFile //depot/symlink
    ... clientFile /run/shm/trash directory.t9802-git-p4-filetype/cli/symlink
    ... isMapped 
    ... headAction add
    ... headType symlink
    ... headTime 1366329740
    ... headRev 1
    ... headChange 6
    ... headModTime 1366329740
    ... haveRev 1

    $ p4 print -q symlink | od -t x1a
    0000000  73  79  6d  6c  69  6e  6b  2d  74  61  72  67  65  74  0a
	      s   y   m   l   i   n   k   -   t   a   r   g   e   t  nl
    0000017

No idea why I get an "nl" but you do not.  If you run _without_
the "| od ...", then the shell prompt ends up on the same line
as the output?  Any interesting shell or shell settings?

> Second issue
> ------------
> 
> git-p4 uses 'p4 print -q -o o FILE' to print a file to stdout.
> At least that is how I interpret this snippet:
> 
>     text = p4_read_pipe(['print', '-q', '-o', '-', file['depotFile']])
> 
> However, p4/Linux prints to stdout by default and '-o -' will save
> the output in a file named '-'.
> 
> My git and p4 versions:
> 
>     $ git --version
>     git version 1.8.2.1.418.gaec3f77
> 
>     $ p4 -V
>     Perforce - The Fast Software Configuration Management System.
>     Copyright 1995-2013 Perforce Software.  All rights reserved.
>     This product includes software developed by the OpenSSL Project
>     for use in the OpenSSL Toolkit (http://www.openssl.org/)
>     See 'p4 help legal' for full OpenSSL license information
>     Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
>     Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).

This code only happens on utf16 files.  But running it by hand,
I cannot reproduce the different behavior:

    $ p4 print -q //depot/f-ascii
    three
    line
    text

    $ p4 print -o - -q //depot/f-ascii
    three
    line

    $ ls ./-
    ls: cannot access ./-: No such file or directory

I'm again confused.  Any hints you can give would be helpful.

		-- Pete

Re: is git-p4 compatible with p4/linux?

[Alexander Tomlinson]

On Apr 18, 2013, at 7:09 PM, Pete Wyckoff <pw@padd.com> wrote:

> alex@aivor.com wrote on Tue, 16 Apr 2013 23:31 -0500:
>> git-p4.py (1.8.2.1.418.gaec3f77) has at least two behaviors that
>> seem to be incompatible with the version of p4 that I recently
>> downloaded from perforce.com (P4/LINUX26X86_64/2013.1/610569).
>> 
>> TLDR: Is git-p4 written for an old version of p4 CLI with different
>> behavior?  Or for a windows or mac release of p4?  Or am I missing
>> something?
> 
> I had not done any testing beyond p4 12.2 (linux).  But running
> the unit tests through 13.1 just now, they all pass.
> 
>    $ p4 -V
>    Perforce - The Fast Software Configuration Management System.
>    Copyright 1995-2013 Perforce Software.  All rights reserved.
>    This product includes software developed by the OpenSSL Project
>    for use in the OpenSSL Toolkit (http://www.openssl.org/)
>    See 'p4 help legal' for full OpenSSL license information
>    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
>    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).
> 
> I'm using python 2.7.3.
> 
>> First issue
>> -----------
>> 
>> git-p4 assumes the output of 'p4 print' adds a newline to the
>> target.  To work around this, git-p4.py strips the last char from
>> symlinks as shown in the following snippet:
>> 
>>    if type_base =3D=3D "symlink":
>>        git_mode =3D "120000"
>>        # p4 print on a symlink contains "target\n"; remove the newline
>>        data =3D ''.join(contents)
>>        contents =3D [data[:-1]]
>> 
>> But my 'p4 print' does not output the newline:
>> 
>>    $ ls -l pcre
>>    lrwxrwxrwx 1 user group 12 Apr 16 10:27 pcre -> ../libs/pcre
>> 
>>    $ p4 print -q pcre | od -t x1a
>>    0000000  2e  2e  2f  6c  69  62  73  2f  70  63  72  65
>>             .   .   /   l   i   b   s   /   p   c   r   e
>>    0000014
>> 
>> If I use 'git p4 clone' the above file shows up in git as a
>> symlink to '../libs/pcr'.  I had another symlink whose target had
>> a strlen of 1 and the 'git p4 clone' failed b/c after stripping
>> the last char the result was an empty string.
> 
> There wasn't an explict test for symlinks, but I threw
> one together quickly and it seems to work.  Can you show
> a bit more information about anything that potentially might
> be odd with your install?
> 
>    arf-git-test$ ls -l symlink
>    lrwxrwxrwx 1 pw pw 14 Apr 18 20:02 symlink -> symlink-target
> 
>    $ p4 fstat symlink
>    ... depotFile //depot/symlink
>    ... clientFile /run/shm/trash directory.t9802-git-p4-filetype/cli/symlink
>    ... isMapped 
>    ... headAction add
>    ... headType symlink
>    ... headTime 1366329740
>    ... headRev 1
>    ... headChange 6
>    ... headModTime 1366329740
>    ... haveRev 1
> 
>    $ p4 print -q symlink | od -t x1a
>    0000000  73  79  6d  6c  69  6e  6b  2d  74  61  72  67  65  74  0a
> 	      s   y   m   l   i   n   k   -   t   a   r   g   e   t  nl
>    0000017
> 
> No idea why I get an "nl" but you do not.  If you run _without_
> the "| od ...", then the shell prompt ends up on the same line
> as the output?  Any interesting shell or shell settings?
> 

Yes.  The shell prompt is on the same line as the output:

$ PS1="FOOBAR$"
FOOBAR$p4 print -q pcre
../libs/pcreFOOBAR$

Perhaps it is a configuration item on the server and/or client.   It seems we
are running the same version of p4.  But just to be sure, check yours against
mine:

$ cksum $(which p4)
3254530484 2420552 /usr/bin/p4

If yours if different, can you email a copy of your p4 executable to me
so I can check if it works differently than mine?

I will also check with coworkers here to see how their client behaves.

>> Second issue
>> ------------
>> 
>> git-p4 uses 'p4 print -q -o o FILE' to print a file to stdout.
>> At least that is how I interpret this snippet:
>> 
>>    text = p4_read_pipe(['print', '-q', '-o', '-', file['depotFile']])
>> 
>> However, p4/Linux prints to stdout by default and '-o -' will save
>> the output in a file named '-'.
>> 
>> My git and p4 versions:
>> 
>>    $ git --version
>>    git version 1.8.2.1.418.gaec3f77
>> 
>>    $ p4 -V
>>    Perforce - The Fast Software Configuration Management System.
>>    Copyright 1995-2013 Perforce Software.  All rights reserved.
>>    This product includes software developed by the OpenSSL Project
>>    for use in the OpenSSL Toolkit (http://www.openssl.org/)
>>    See 'p4 help legal' for full OpenSSL license information
>>    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
>>    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).
> 
> This code only happens on utf16 files.  But running it by hand,
> I cannot reproduce the different behavior:
> 
>    $ p4 print -q //depot/f-ascii
>    three
>    line
>    text
> 
>    $ p4 print -o - -q //depot/f-ascii
>    three
>    line
> 
>    $ ls ./-
>    ls: cannot access ./-: No such file or directory
> 
> I'm again confused.  Any hints you can give would be helpful.

This "second issue" is a non-issue.  It seems "-o -" does send to
stdout for files.  For symlinks, it creates a symlink named "-".
Example:

$ ls -l pcre
lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 17:17 pcre -> ../libs/pcre/
$ ls -l ./- ./xxx
/bin/ls: cannot access ./-: No such file or directory
/bin/ls: cannot access ./xxx: No such file or directory
$ p4 print -q -o - pcre
$ p4 print -q -o xxx pcre
$ ls -l ./- ./xxx
lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./- -> ../libs/pcre/
lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./xxx -> ../libs/pcre/


> 
> 		-- Pete
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thanks,
Alex

Re: is git-p4 compatible with p4/linux?

[Pete Wyckoff]

alex@aivor.com wrote on Thu, 18 Apr 2013 20:34 -0500:
> Perhaps it is a configuration item on the server and/or client.   It seems we
> are running the same version of p4.  But just to be sure, check yours against
> mine:
> 
> $ cksum $(which p4)
> 3254530484 2420552 /usr/bin/p4
> 
> If yours if different, can you email a copy of your p4 executable to me
> so I can check if it works differently than mine?

It is the same binary.  Assuming you're running p4d 2013.1 too?

    $ p4 info
    [..]
    Server version: P4D/LINUX26X86_64/2013.1/610569 (2013/03/19)

I'm using these p4 client settings:

    Options:        noallwrite noclobber nocompress unlocked nomodtime normdir
    SubmitOptions:  submitunchanged
    LineEnd:        unix

Running "strace -vf -s 2000 p4 print -q symlink" shows that the
newline is embedded in the response from p4d:

    read(7, "...\0\0\0symlink-target\n\0func\0\23...", 4096) = 277

Also the file depot/symlink,v in the p4d depot area includes the
\n in the RCS file too, somewhat surprisingly:

    $ cat depot/symlink,v
    head     1.6;
    [..]
    @@
    text
    @symlink-target
    @

> I will also check with coworkers here to see how their client behaves.
> > This code only happens on utf16 files.  But running it by hand,
> > I cannot reproduce the different behavior:
> > 
> >    $ p4 print -q //depot/f-ascii
> >    three
> >    line
> >    text
> > 
> >    $ p4 print -o - -q //depot/f-ascii
> >    three
> >    line
> > 
> >    $ ls ./-
> >    ls: cannot access ./-: No such file or directory
> > 
> > I'm again confused.  Any hints you can give would be helpful.
> 
> This "second issue" is a non-issue.  It seems "-o -" does send to
> stdout for files.  For symlinks, it creates a symlink named "-".
> Example:
> 
> $ ls -l pcre
> lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 17:17 pcre -> ../libs/pcre/
> $ ls -l ./- ./xxx
> /bin/ls: cannot access ./-: No such file or directory
> /bin/ls: cannot access ./xxx: No such file or directory
> $ p4 print -q -o - pcre
> $ p4 print -q -o xxx pcre
> $ ls -l ./- ./xxx
> lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./- -> ../libs/pcre/
> lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./xxx -> ../libs/pcre/

Me too.  That's annoying behavior, but not used by git-p4 fortunately.
The "-o -" option is only used for odd utf16 files where "p4 print"
generates invalid output.

		-- Pete

Re: is git-p4 compatible with p4/linux?

[Alex Tomlinson]

I found some symlinks in our depot that have the newline and
others that do not.  It looks like p4 print just dumps whatever
is in the depot file, which of course is what you would expect
it to do.  So the question (for me) is, how did some of my symlink
targets end up without the newline?  Note: that's not a question
for git-p4 because git-p4 was not involved in the creation of
this p4 depot (or the symlink).

git-p4 is off the hook since it appears to be an issue with
the depot itself.

Thanks for your time!
--Alex

Re: is git-p4 compatible with p4/linux?

[David Aguilar]

On Thu, Apr 18, 2013 at 5:09 PM, Pete Wyckoff <pw@padd.com> wrote:
>> First issue
>> -----------
>>
>> git-p4 assumes the output of 'p4 print' adds a newline to the
>> target.  To work around this, git-p4.py strips the last char from
>> symlinks as shown in the following snippet:
>>
>>     if type_base == "symlink":
>>         git_mode = "120000"
>>         # p4 print on a symlink contains "target\n"; remove the newline
>>         data = ''.join(contents)
>>         contents = [data[:-1]]

This line could be made more robust by changing it to:

    contents = [data.rstrip('\n')]

That way it only strips off newlines if they exist, which essentially
papers over these rogue depot files.
Alternatively, it could use rstrip() with no arguments to cast a wider
net and catch all whitespace.
-- 
David

Re: is git-p4 compatible with p4/linux?

[Pete Wyckoff]

davvid@gmail.com wrote on Sat, 20 Apr 2013 03:50 -0700:
> On Thu, Apr 18, 2013 at 5:09 PM, Pete Wyckoff <pw@padd.com> wrote:
> >> First issue
> >> -----------
> >>
> >> git-p4 assumes the output of 'p4 print' adds a newline to the
> >> target.  To work around this, git-p4.py strips the last char from
> >> symlinks as shown in the following snippet:
> >>
> >>     if type_base == "symlink":
> >>         git_mode = "120000"
> >>         # p4 print on a symlink contains "target\n"; remove the newline
> >>         data = ''.join(contents)
> >>         contents = [data[:-1]]
> 
> This line could be made more robust by changing it to:
> 
>     contents = [data.rstrip('\n')]
> 
> That way it only strips off newlines if they exist, which essentially
> papers over these rogue depot files.
> Alternatively, it could use rstrip() with no arguments to cast a wider
> net and catch all whitespace.

I was tempted to do that, but it is possible to put \n and other
space characters in the target of symlinks.  It's unfortunate
that p4 always appears to tack on a newline itself.

We'll see if Alex comes up with a pattern that shows how he ended
up with the odd symlinks.

		-- Pete