git-p4: exception when cloning a perforce repository

git-p4: exception when cloning a perforce repository

[Damien Gérard]

Hi !

I am trying to clone a perforce repository via git and I am having the following backtrace :

{14:20}~/projects/####:master ✗ ➭ git p4 clone //depot/@all .
Importing revision …
[...]
Importing revision 59702 (45%)Traceback (most recent call last):
  File "/opt/git/libexec/git-core/git-p4", line 3287, in <module>
    main()
  File "/opt/git/libexec/git-core/git-p4", line 3281, in main
    if not cmd.run(args):
  File "/opt/git/libexec/git-core/git-p4", line 3155, in run
    if not P4Sync.run(self, depotPaths):
  File "/opt/git/libexec/git-core/git-p4", line 3008, in run
    self.importChanges(changes)
  File "/opt/git/libexec/git-core/git-p4", line 2680, in importChanges
    self.initialParent)
  File "/opt/git/libexec/git-core/git-p4", line 2304, in commit
    self.streamP4Files(new_files)
  File "/opt/git/libexec/git-core/git-p4", line 2218, in streamP4Files
    cb=streamP4FilesCbSelf)
  File "/opt/git/libexec/git-core/git-p4", line 482, in p4CmdList
    cb(entry)
  File "/opt/git/libexec/git-core/git-p4", line 2212, in streamP4FilesCbSelf
    self.streamP4FilesCb(entry)
  File "/opt/git/libexec/git-core/git-p4", line 2167, in streamP4FilesCb
    self.streamOneP4File(self.stream_file, self.stream_contents)
  File "/opt/git/libexec/git-core/git-p4", line 2078, in streamOneP4File
    if data[-1] == '\n':
IndexError: string index out of range


git —version: git version 1.8.5.2.309.ga25014b [last commit from master from github.com/git/git]
os : ubuntu 13.10


Any ideas ? :)

Best regards,
Damien

Re: git-p4: exception when cloning a perforce repository

[Pete Wyckoff]

damien@iwi.me wrote on Mon, 13 Jan 2014 14:37 +0100:
> I am trying to clone a perforce repository via git and I am having the following backtrace :
> 
> {14:20}~/projects/####:master ✗ ➭ git p4 clone //depot/@all .
> Importing revision …
> [...]
> Importing revision 59702 (45%)Traceback (most recent call last):
[..]
>   File "/opt/git/libexec/git-core/git-p4", line 2078, in streamOneP4File
>     if data[-1] == '\n':
> IndexError: string index out of range
> 
> git —version: git version 1.8.5.2.309.ga25014b [last commit from master from github.com/git/git]
> os : ubuntu 13.10

This code:

        if type_base == "symlink":
            git_mode = "120000"
            # p4 print on a symlink sometimes contains "target\n";
            # if it does, remove the newline
            data = ''.join(contents)
  ==>       if data[-1] == '\n':
                contents = [data[:-1]]
            else:
                contents = [data]

means that data is an empty string.  Implies you've got a
symlink pointing to nothing.  Is that even possible?

It could be this is a regression introduced at 1292df1 (git-p4:
Fix occasional truncation of symlink contents., 2013-08-08).  The
old way of doing data[:-1] unconditionally would have worked but
was broken for other reasons.

Could you investigate the symlink a bit?  We're looking for
one in change 59702 that points to nowhere.  Maybe do:

    $ p4 describe -s 59702

and see if you can figure out which of those could be a symlink, then
inspect it:

    $ p4 fstat //depot/symlink@59702
    (probably shows it is "headRev 1")

    $ p4 print -q //depot/symlink#1

    $ p4 print -q //depot/symlink#1 | od -c

Thanks for checking this depot info first.

		-- Pete

Re: git-p4: exception when cloning a perforce repository

[Pete Wyckoff]

pw@padd.com wrote on Mon, 13 Jan 2014 19:18 -0500:
> damien@iwi.me wrote on Mon, 13 Jan 2014 14:37 +0100:
> > I am trying to clone a perforce repository via git and I am having the following backtrace :
> > 
> > {14:20}~/projects/####:master ✗ ➭ git p4 clone //depot/@all .
> > Importing revision …
> > [...]
> > Importing revision 59702 (45%)Traceback (most recent call last):
> [..]
> >   File "/opt/git/libexec/git-core/git-p4", line 2078, in streamOneP4File
> >     if data[-1] == '\n':
> > IndexError: string index out of range
> > 
> > git —version: git version 1.8.5.2.309.ga25014b [last commit from master from github.com/git/git]
> > os : ubuntu 13.10
> 
> This code:
> 
>         if type_base == "symlink":
>             git_mode = "120000"
>             # p4 print on a symlink sometimes contains "target\n";
>             # if it does, remove the newline
>             data = ''.join(contents)
>   ==>       if data[-1] == '\n':
>                 contents = [data[:-1]]
>             else:
>                 contents = [data]
> 
> means that data is an empty string.  Implies you've got a
> symlink pointing to nothing.  Is that even possible?
> 
> It could be this is a regression introduced at 1292df1 (git-p4:
> Fix occasional truncation of symlink contents., 2013-08-08).  The
> old way of doing data[:-1] unconditionally would have worked but
> was broken for other reasons.
> 
> Could you investigate the symlink a bit?  We're looking for
> one in change 59702 that points to nowhere.  Maybe do:
> 
>     $ p4 describe -s 59702
> 
> and see if you can figure out which of those could be a symlink, then
> inspect it:
> 
>     $ p4 fstat //depot/symlink@59702
>     (probably shows it is "headRev 1")
> 
>     $ p4 print -q //depot/symlink#1
> 
>     $ p4 print -q //depot/symlink#1 | od -c
> 
> Thanks for checking this depot info first.

I've tried to hack a test that produces a null symlink,
and having done so, find an error later on trying to
generate a symlink that points to "".  So the "easy"
fix of checking for an empty string is unlikely to work
for your repo.

Curious as to how you managed to generate such a thing.
If you find the file, and can get at the p4 depot, the
full ,v file would be interesting too.

		-- Pete

Re: git-p4: exception when cloning a perforce repository

[Damien Gérard]

On 15 Jan 2014, at 00:24, Pete Wyckoff <pw@padd.com> wrote:

> pw@padd.com wrote on Mon, 13 Jan 2014 19:18 -0500:
>> damien@iwi.me wrote on Mon, 13 Jan 2014 14:37 +0100:
>>> I am trying to clone a perforce repository via git and I am having the following backtrace :
>>> 
>>> {14:20}~/projects/####:master ✗ ➭ git p4 clone //depot/@all .
>>> Importing revision …
>>> [...]
>>> Importing revision 59702 (45%)Traceback (most recent call last):
>> [..]
>>>  File "/opt/git/libexec/git-core/git-p4", line 2078, in streamOneP4File
>>>    if data[-1] == '\n':
>>> IndexError: string index out of range
>>> 
>>> git —version: git version 1.8.5.2.309.ga25014b [last commit from master from github.com/git/git]
>>> os : ubuntu 13.10
>> 
>> This code:
>> 
>>        if type_base == "symlink":
>>            git_mode = "120000"
>>            # p4 print on a symlink sometimes contains "target\n";
>>            # if it does, remove the newline
>>            data = ''.join(contents)
>>  ==>       if data[-1] == '\n':
>>                contents = [data[:-1]]
>>            else:
>>                contents = [data]
>> 
>> means that data is an empty string.  Implies you've got a
>> symlink pointing to nothing.  Is that even possible?

It does not seem so but I am so sure.


>> It could be this is a regression introduced at 1292df1 (git-p4:
>> Fix occasional truncation of symlink contents., 2013-08-08).  The
>> old way of doing data[:-1] unconditionally would have worked but
>> was broken for other reasons.
>> 
>> Could you investigate the symlink a bit?  We're looking for
>> one in change 59702 that points to nowhere.  Maybe do:
>> 
>>    $ p4 describe -s 59702
>> 
>> and see if you can figure out which of those could be a symlink, then
>> inspect it:
>> 
>>    $ p4 fstat //depot/symlink@59702
>>    (probably shows it is "headRev 1")
>> 
>>    $ p4 print -q //depot/symlink#1
>> 
>>    $ p4 print -q //depot/symlink#1 | od -c
>> 
>> Thanks for checking this depot info first.
> 
> I've tried to hack a test that produces a null symlink,
> and having done so, find an error later on trying to
> generate a symlink that points to "".  So the "easy"
> fix of checking for an empty string is unlikely to work
> for your repo.
> 
> Curious as to how you managed to generate such a thing.
> If you find the file, and can get at the p4 depot, the
> full ,v file would be interesting too.
> 



Indeed, those files are symlinks actually.  But it sems they are all valid.


Here is what I can get :

Change 59702 by ##############@VS9 on 2009/03/24 15:53:39

	OpenSSL 0.9.8j

Affected files ...

... //depot/openssl/0.9.8j/openssl/include/openssl/aes.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/asn1.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/asn1_mac.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/asn1t.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/bio.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/blowfish.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/buffer.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/cast.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/comp.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/conf.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/conf_api.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/crypto.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/des.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/des_old.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/dh.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/dsa.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/dso.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/dtls1.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/e_os2.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ebcdic.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ec.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ecdh.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ecdsa.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/engine.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/err.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/evp.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/fips.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/fips_rand.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/hmac.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/idea.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/krb5_asn.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/kssl.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/lhash.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/md2.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/md4.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/md5.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/obj_mac.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/objects.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ocsp.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/opensslconf.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/opensslv.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ossl_typ.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/pem.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/pem2.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/pkcs12.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/pkcs7.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/pq_compat.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/pqueue.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/rand.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/rc2.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/rc4.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ripemd.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/rsa.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/safestack.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/sha.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ssl.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ssl2.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ssl23.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ssl3.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/stack.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/store.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/symhacks.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/tls1.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/tmdiff.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/txt_db.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ui.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/ui_compat.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/x509.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/x509_vfy.h#2 edit
... //depot/openssl/0.9.8j/openssl/include/openssl/x509v3.h#2 edit


Just in case :

$ p4 describe -s 59700
59700 - no such changelist.


p4 fstat  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h@59702 
... depotFile //depot/openssl/0.9.8j/openssl/include/openssl/bn.h
... headAction edit
... headType symlink
... headTime 1237906419
... headRev 2
... headChange 59702
... headModTime 1231329423


p4 print -q //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 | od -c
0000000

p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1        
//depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1 - add change 59574 (text)

 p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2
//depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - edit change 59702 (symlink)

Re: git-p4: exception when cloning a perforce repository

[Pete Wyckoff]

damien@iwi.me wrote on Wed, 15 Jan 2014 09:56 +0100:
> p4 fstat  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h@59702 
> ... depotFile //depot/openssl/0.9.8j/openssl/include/openssl/bn.h
> ... headAction edit
> ... headType symlink
> ... headTime 1237906419
> ... headRev 2
> ... headChange 59702
> ... headModTime 1231329423
> 
> p4 print -q //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 | od -c
> 0000000
> 
> p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1        
> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1 - add change 59574 (text)
>  p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2
> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - edit change 59702 (symlink)

That's interesting.  When I do the equivalent "p4 print" commands
it shows something like this.

arf-git-test$ p4 fstat //depot/bn.h
... depotFile //depot/bn.h
... clientFile /dev/shm/trash directory.t9802-git-p4-filetype/cli/bn.h
... isMapped 
... headAction edit
... headType symlink
... headTime 1389876870
... headRev 2
... headChange 8
... headModTime 1389876870
... haveRev 2

arf-git-test$ p4 print //depot/bn.h#1
//depot/bn.h#1 - add change 7 (text)
file-text

arf-git-test$ p4 print //depot/bn.h#2
//depot/bn.h#2 - edit change 8 (symlink)
/elsewhere/bn.h

I don't know how you manage to get a symlink with an empty
destination like that.

I'll work on a way to hack around this failure.  In the mean time,
if you're game, it might be fun to see what p4 does with such a
repository.  You could make a client for just that little subdir,
check out at 59702 and see what is there:

mkdir testmess
cd testmess
cat <<EOF | p4 client -i
Client: testmess
Description: testmess
Root: $(pwd)
View: //depot/openssl/0.9.8j/openssl/include/openssl/... //testmess/...
EOF

then take a look at how p4 represents the "empty" symlink
in the filesystem:

p4 sync @59702
ls -la bn.h

		-- Pete

Re: git-p4: exception when cloning a perforce repository

[Damien Gérard]

On 16 Jan 2014, at 14:08, Pete Wyckoff <pw@padd.com> wrote:

> damien@iwi.me wrote on Wed, 15 Jan 2014 09:56 +0100:
>> p4 fstat  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h@59702 
>> ... depotFile //depot/openssl/0.9.8j/openssl/include/openssl/bn.h
>> ... headAction edit
>> ... headType symlink
>> ... headTime 1237906419
>> ... headRev 2
>> ... headChange 59702
>> ... headModTime 1231329423
>> 
>> p4 print -q //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 | od -c
>> 0000000
>> 
>> p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1        
>> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1 - add change 59574 (text)
>> p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2
>> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - edit change 59702 (symlink)
> 
> That's interesting.  When I do the equivalent "p4 print" commands
> it shows something like this.
> 
> arf-git-test$ p4 fstat //depot/bn.h
> ... depotFile //depot/bn.h
> ... clientFile /dev/shm/trash directory.t9802-git-p4-filetype/cli/bn.h
> ... isMapped 
> ... headAction edit
> ... headType symlink
> ... headTime 1389876870
> ... headRev 2
> ... headChange 8
> ... headModTime 1389876870
> ... haveRev 2
> 
> arf-git-test$ p4 print //depot/bn.h#1
> //depot/bn.h#1 - add change 7 (text)
> file-text
> 
> arf-git-test$ p4 print //depot/bn.h#2
> //depot/bn.h#2 - edit change 8 (symlink)
> /elsewhere/bn.h
> 
> I don't know how you manage to get a symlink with an empty
> destination like that.
> 
> I'll work on a way to hack around this failure.  In the mean time,
> if you're game, it might be fun to see what p4 does with such a
> repository.  You could make a client for just that little subdir,
> check out at 59702 and see what is there:
> 
> mkdir testmess
> cd testmess
> cat <<EOF | p4 client -i
> Client: testmess
> Description: testmess
> Root: $(pwd)
> View: //depot/openssl/0.9.8j/openssl/include/openssl/... //testmess/...
> EOF
> 
> then take a look at how p4 represents the "empty" symlink
> in the filesystem:
> 
> p4 sync @59702
> ls -la bn.h

I’ve tried exactly your commands, and I’ve got an empty folder..

{14:38}~/p4/testmess ➭ p4 sync @59702
//depot/openssl/0.9.8j/openssl/include/openssl/aes.h#2 - refreshing /home/dgerard/p4/testmess/aes.h
//depot/openssl/0.9.8j/openssl/include/openssl/asn1.h#2 - refreshing /home/dgerard/p4/testmess/asn1.h
//depot/openssl/0.9.8j/openssl/include/openssl/asn1_mac.h#2 - refreshing /home/dgerard/p4/testmess/asn1_mac.h
//depot/openssl/0.9.8j/openssl/include/openssl/asn1t.h#2 - refreshing /home/dgerard/p4/testmess/asn1t.h
//depot/openssl/0.9.8j/openssl/include/openssl/bio.h#2 - refreshing /home/dgerard/p4/testmess/bio.h
//depot/openssl/0.9.8j/openssl/include/openssl/blowfish.h#2 - refreshing /home/dgerard/p4/testmess/blowfish.h
//depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - refreshing /home/dgerard/p4/testmess/bn.h
//depot/openssl/0.9.8j/openssl/include/openssl/buffer.h#2 - refreshing /home/dgerard/p4/testmess/buffer.h
[…]


{14:39}~/p4/testmess ➭ ls -la
total 12
drwxr-xr-x 2 dgerard dgerard 4096 janv. 16 14:37 .
drwxr-xr-x 4 dgerard dgerard 4096 janv. 16 14:34 ..
-rw-r--r-- 1 dgerard dgerard   93 janv. 16 14:37 .perforce	


Then I tried to sync the previous changeset, which is ok :

{14:44}~/p4/testmess ➭ p4 sync -f @59701
//depot/openssl/0.9.8j/openssl/include/openssl/aes.h#1 - updating /home/dgerard/p4/testmess/aes.h
[…]

{14:44}~/p4/testmess ➭ l
total 0
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 aes.h
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 asn1.h
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 asn1_mac.h
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 asn1t.h
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 bio.h
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 blowfish.h
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 bn.h
-r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 buffer.h
[…]



However, when trying to sync to the appropriate changeset :

{14:44}~/p4/testmess ➭ p4 sync -f @59702
//depot/openssl/0.9.8j/openssl/include/openssl/aes.h#2 - updating /home/dgerard/p4/testmess/aes.h
rename: /home/dgerard/p4/testmess/aes.h: No such file or directory
//depot/openssl/0.9.8j/openssl/include/openssl/asn1.h#2 - updating /home/dgerard/p4/testmess/asn1.h
rename: /home/dgerard/p4/testmess/asn1.h: No such file or directory
//depot/openssl/0.9.8j/openssl/include/openssl/asn1_mac.h#2 - updating /home/dgerard/p4/testmess/asn1_mac.h
rename: /home/dgerard/p4/testmess/asn1_mac.h: No such file or directory
//depot/openssl/0.9.8j/openssl/include/openssl/asn1t.h#2 - updating /home/dgerard/p4/testmess/asn1t.h
rename: /home/dgerard/p4/testmess/asn1t.h: No such file or directory
//depot/openssl/0.9.8j/openssl/include/openssl/bio.h#2 - updating /home/dgerard/p4/testmess/bio.h
rename: /home/dgerard/p4/testmess/bio.h: No such file or directory
//depot/openssl/0.9.8j/openssl/include/openssl/blowfish.h#2 - updating /home/dgerard/p4/testmess/blowfish.h
rename: /home/dgerard/p4/testmess/blowfish.h: No such file or directory
//depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - updating /home/dgerard/p4/testmess/bn.h
rename: /home/dgerard/p4/testmess/bn.h: No such file or directory

And the folder remains untouched.
Quite strange for me...

Re: git-p4: exception when cloning a perforce repository

[Pete Wyckoff]

damien@iwi.me wrote on Thu, 16 Jan 2014 14:46 +0100:
> 
> On 16 Jan 2014, at 14:08, Pete Wyckoff <pw@padd.com> wrote:
> 
> > damien@iwi.me wrote on Wed, 15 Jan 2014 09:56 +0100:
> >> p4 fstat  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h@59702 
> >> ... depotFile //depot/openssl/0.9.8j/openssl/include/openssl/bn.h
> >> ... headAction edit
> >> ... headType symlink
> >> ... headTime 1237906419
> >> ... headRev 2
> >> ... headChange 59702
> >> ... headModTime 1231329423
> >> 
> >> p4 print -q //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 | od -c
> >> 0000000
> >> 
> >> p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1        
> >> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#1 - add change 59574 (text)
> >> p4 print  //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2
> >> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - edit change 59702 (symlink)
> > 
> > That's interesting.  When I do the equivalent "p4 print" commands
> > it shows something like this.
> > 
> > arf-git-test$ p4 fstat //depot/bn.h
> > ... depotFile //depot/bn.h
> > ... clientFile /dev/shm/trash directory.t9802-git-p4-filetype/cli/bn.h
> > ... isMapped 
> > ... headAction edit
> > ... headType symlink
> > ... headTime 1389876870
> > ... headRev 2
> > ... headChange 8
> > ... headModTime 1389876870
> > ... haveRev 2
> > 
> > arf-git-test$ p4 print //depot/bn.h#1
> > //depot/bn.h#1 - add change 7 (text)
> > file-text
> > 
> > arf-git-test$ p4 print //depot/bn.h#2
> > //depot/bn.h#2 - edit change 8 (symlink)
> > /elsewhere/bn.h
> > 
> > I don't know how you manage to get a symlink with an empty
> > destination like that.
> > 
> > I'll work on a way to hack around this failure.  In the mean time,
> > if you're game, it might be fun to see what p4 does with such a
> > repository.  You could make a client for just that little subdir,
> > check out at 59702 and see what is there:
> > 
> > mkdir testmess
> > cd testmess
> > cat <<EOF | p4 client -i
> > Client: testmess
> > Description: testmess
> > Root: $(pwd)
> > View: //depot/openssl/0.9.8j/openssl/include/openssl/... //testmess/...
> > EOF
> > 
> > then take a look at how p4 represents the "empty" symlink
> > in the filesystem:
> > 
> > p4 sync @59702
> > ls -la bn.h
> 
> I’ve tried exactly your commands, and I’ve got an empty folder..
> 
> {14:38}~/p4/testmess ➭ p4 sync @59702
> //depot/openssl/0.9.8j/openssl/include/openssl/aes.h#2 - refreshing /home/dgerard/p4/testmess/aes.h
> //depot/openssl/0.9.8j/openssl/include/openssl/asn1.h#2 - refreshing /home/dgerard/p4/testmess/asn1.h
> //depot/openssl/0.9.8j/openssl/include/openssl/asn1_mac.h#2 - refreshing /home/dgerard/p4/testmess/asn1_mac.h
> //depot/openssl/0.9.8j/openssl/include/openssl/asn1t.h#2 - refreshing /home/dgerard/p4/testmess/asn1t.h
> //depot/openssl/0.9.8j/openssl/include/openssl/bio.h#2 - refreshing /home/dgerard/p4/testmess/bio.h
> //depot/openssl/0.9.8j/openssl/include/openssl/blowfish.h#2 - refreshing /home/dgerard/p4/testmess/blowfish.h
> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - refreshing /home/dgerard/p4/testmess/bn.h
> //depot/openssl/0.9.8j/openssl/include/openssl/buffer.h#2 - refreshing /home/dgerard/p4/testmess/buffer.h
> […]
> 
> 
> {14:39}~/p4/testmess ➭ ls -la
> total 12
> drwxr-xr-x 2 dgerard dgerard 4096 janv. 16 14:37 .
> drwxr-xr-x 4 dgerard dgerard 4096 janv. 16 14:34 ..
> -rw-r--r-- 1 dgerard dgerard   93 janv. 16 14:37 .perforce	
> 
> 
> Then I tried to sync the previous changeset, which is ok :
> 
> {14:44}~/p4/testmess ➭ p4 sync -f @59701
> //depot/openssl/0.9.8j/openssl/include/openssl/aes.h#1 - updating /home/dgerard/p4/testmess/aes.h
> […]
> 
> {14:44}~/p4/testmess ➭ l
> total 0
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 aes.h
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 asn1.h
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 asn1_mac.h
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 asn1t.h
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 bio.h
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 blowfish.h
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 bn.h
> -r--r--r-- 1 dgerard dgerard 0 janv. 16 14:44 buffer.h
> […]
> 
> 
> 
> However, when trying to sync to the appropriate changeset :
> 
> {14:44}~/p4/testmess ➭ p4 sync -f @59702
> //depot/openssl/0.9.8j/openssl/include/openssl/aes.h#2 - updating /home/dgerard/p4/testmess/aes.h
> rename: /home/dgerard/p4/testmess/aes.h: No such file or directory
> //depot/openssl/0.9.8j/openssl/include/openssl/asn1.h#2 - updating /home/dgerard/p4/testmess/asn1.h
> rename: /home/dgerard/p4/testmess/asn1.h: No such file or directory
> //depot/openssl/0.9.8j/openssl/include/openssl/asn1_mac.h#2 - updating /home/dgerard/p4/testmess/asn1_mac.h
> rename: /home/dgerard/p4/testmess/asn1_mac.h: No such file or directory
> //depot/openssl/0.9.8j/openssl/include/openssl/asn1t.h#2 - updating /home/dgerard/p4/testmess/asn1t.h
> rename: /home/dgerard/p4/testmess/asn1t.h: No such file or directory
> //depot/openssl/0.9.8j/openssl/include/openssl/bio.h#2 - updating /home/dgerard/p4/testmess/bio.h
> rename: /home/dgerard/p4/testmess/bio.h: No such file or directory
> //depot/openssl/0.9.8j/openssl/include/openssl/blowfish.h#2 - updating /home/dgerard/p4/testmess/blowfish.h
> rename: /home/dgerard/p4/testmess/blowfish.h: No such file or directory
> //depot/openssl/0.9.8j/openssl/include/openssl/bn.h#2 - updating /home/dgerard/p4/testmess/bn.h
> rename: /home/dgerard/p4/testmess/bn.h: No such file or directory
> 
> And the folder remains untouched.
> Quite strange for me...

Oh cool, that helps a lot.  P4 is just broken here, so we can get
away with being a bit sloppy in git.  I'll try just pretending
"empty symlinks" are not in the repo.  Hopefully you'll have a
future commit in your p4 repo that brings back bn.h properly.

Still not sure about how I'll test this.

Thanks,

		-- Pete

Re: git-p4: exception when cloning a perforce repository

[Damien Gérard]

On 16 Jan 2014, at 15:45, Pete Wyckoff <pw@padd.com> wrote:

> Oh cool, that helps a lot.  P4 is just broken here, so we can get
> away with being a bit sloppy in git.  I'll try just pretending
> "empty symlinks" are not in the repo.  Hopefully you'll have a
> future commit in your p4 repo that brings back bn.h properly.

Thanks !
I would love to use git instead of perforce if possible :)

> Still not sure about how I'll test this.

I can test for you, no probleme with that.


> 
> Thanks,
> 
> 		-- Pete
> 

Re: git-p4: exception when cloning a perforce repository

[Pete Wyckoff]

damien@iwi.me wrote on Thu, 16 Jan 2014 17:02 +0100:
> 
> On 16 Jan 2014, at 15:45, Pete Wyckoff <pw@padd.com> wrote:
> 
> > Oh cool, that helps a lot.  P4 is just broken here, so we can get
> > away with being a bit sloppy in git.  I'll try just pretending
> > "empty symlinks" are not in the repo.  Hopefully you'll have a
> > future commit in your p4 repo that brings back bn.h properly.
> 
> Thanks !
> I would love to use git instead of perforce if possible :)
> 
> > Still not sure about how I'll test this.
> 
> I can test for you, no probleme with that.

Any chance you can give this a go?  I've a bigger patch in
a longer series, but this should be the minimal fix.  If it
works, I'll ship it to Junio.

Thanks,

		-- Pete

----8<--------

From 8556ab04dd126184e26a380b7ed08998fd33debe Mon Sep 17 00:00:00 2001
From: Pete Wyckoff <pw@padd.com>
Date: Thu, 16 Jan 2014 18:34:09 -0500
Subject: [PATCH] git p4: work around p4 bug that causes empty symlinks
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Damien Gérard highlights an interesting problem.  Some p4
repositories end up with symlinks that have an empty target.  It
is not possible to create this with current p4, but they do
indeed exist.

The effect in git p4 is that "p4 print" on the symlink returns an
empty string, confusing the curret symlink-handling code.

In p4, syncing to a change that includes such a bogus symlink
creates errors:

    //depot/empty-symlink - updating /home/me/p4/empty-symlink
    rename: /home/me/p4/empty-symlink: No such file or directory

and leaves no symlink.

Replicate the p4 behavior by ignoring these bogus symlinks.  If
they are fixed in later revisions, the symlink will be replaced
properly.

Reported-by: Damien Gérard <damien@iwi.me>
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/git-p4.py b/git-p4.py
index 5ea8bb8..e798ecf 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2075,7 +2075,14 @@ class P4Sync(Command, P4UserMap):
             # p4 print on a symlink sometimes contains "target\n";
             # if it does, remove the newline
             data = ''.join(contents)
-            if data[-1] == '\n':
+            if not data:
+                # Some version of p4 allowed creating a symlink that pointed
+                # to nothing.  This causes p4 errors when checking out such
+                # a change, and errors here too.  Work around it by ignoring
+                # the bad symlink; hopefully a future change fixes it.
+                print "\nIgnoring empty symlink in %s" % file['depotFile']
+                return
+            elif data[-1] == '\n':
                 contents = [data[:-1]]
             else:
                 contents = [data]
-- 
1.8.5.2.320.g99957e5

Re: git-p4: exception when cloning a perforce repository

[Damien Gérard]

On 18 Jan 2014, at 19:22, Pete Wyckoff <pw@padd.com> wrote:

> damien@iwi.me wrote on Thu, 16 Jan 2014 17:02 +0100:
>> 
>> On 16 Jan 2014, at 15:45, Pete Wyckoff <pw@padd.com> wrote:
>> 
>>> Oh cool, that helps a lot.  P4 is just broken here, so we can get
>>> away with being a bit sloppy in git.  I'll try just pretending
>>> "empty symlinks" are not in the repo.  Hopefully you'll have a
>>> future commit in your p4 repo that brings back bn.h properly.
>> 
>> Thanks !
>> I would love to use git instead of perforce if possible :)
>> 
>>> Still not sure about how I'll test this.
>> 
>> I can test for you, no probleme with that.
> 
> Any chance you can give this a go?  I've a bigger patch in
> a longer series, but this should be the minimal fix.  If it
> works, I'll ship it to Junio.

Yeah it seems to work fine !
I’ve finally imported all 152620 changesets :)

Thanks !

(git pull from this morning + patch)


> 
> Thanks,
> 
> 		-- Pete
> 
> ----8<--------
> 
> From 8556ab04dd126184e26a380b7ed08998fd33debe Mon Sep 17 00:00:00 2001
> From: Pete Wyckoff <pw@padd.com>
> Date: Thu, 16 Jan 2014 18:34:09 -0500
> Subject: [PATCH] git p4: work around p4 bug that causes empty symlinks
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> 
> Damien Gérard highlights an interesting problem.  Some p4
> repositories end up with symlinks that have an empty target.  It
> is not possible to create this with current p4, but they do
> indeed exist.
> 
> The effect in git p4 is that "p4 print" on the symlink returns an
> empty string, confusing the curret symlink-handling code.
> 
> In p4, syncing to a change that includes such a bogus symlink
> creates errors:
> 
>    //depot/empty-symlink - updating /home/me/p4/empty-symlink
>    rename: /home/me/p4/empty-symlink: No such file or directory
> 
> and leaves no symlink.
> 
> Replicate the p4 behavior by ignoring these bogus symlinks.  If
> they are fixed in later revisions, the symlink will be replaced
> properly.
> 
> Reported-by: Damien Gérard <damien@iwi.me>
> Signed-off-by: Pete Wyckoff <pw@padd.com>
> ---
> git-p4.py | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/git-p4.py b/git-p4.py
> index 5ea8bb8..e798ecf 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -2075,7 +2075,14 @@ class P4Sync(Command, P4UserMap):
>             # p4 print on a symlink sometimes contains "target\n";
>             # if it does, remove the newline
>             data = ''.join(contents)
> -            if data[-1] == '\n':
> +            if not data:
> +                # Some version of p4 allowed creating a symlink that pointed
> +                # to nothing.  This causes p4 errors when checking out such
> +                # a change, and errors here too.  Work around it by ignoring
> +                # the bad symlink; hopefully a future change fixes it.
> +                print "\nIgnoring empty symlink in %s" % file['depotFile']
> +                return
> +            elif data[-1] == '\n':
>                 contents = [data[:-1]]
>             else:
>                 contents = [data]
> -- 
> 1.8.5.2.320.g99957e5
> 
> 

Re: git-p4: exception when cloning a perforce repository

[Damien Gérard]

On 18 Jan 2014, at 19:22, Pete Wyckoff <pw@padd.com> wrote:

> damien@iwi.me wrote on Thu, 16 Jan 2014 17:02 +0100:
>> 
>> On 16 Jan 2014, at 15:45, Pete Wyckoff <pw@padd.com> wrote:
>> 
>>> Oh cool, that helps a lot.  P4 is just broken here, so we can get
>>> away with being a bit sloppy in git.  I'll try just pretending
>>> "empty symlinks" are not in the repo.  Hopefully you'll have a
>>> future commit in your p4 repo that brings back bn.h properly.
>> 
>> Thanks !
>> I would love to use git instead of perforce if possible :)
>> 
>>> Still not sure about how I'll test this.
>> 
>> I can test for you, no probleme with that.
> 
> Any chance you can give this a go?  I've a bigger patch in
> a longer series, but this should be the minimal fix.  If it
> works, I'll ship it to Junio.
> 

Just for info, it works but it seems there are still some issues when a git repository is present within the perforce repo :

error: Invalid path 'Tools/Doc/bin/yuidoc/.git/FETCH_HEAD'
error: Invalid path 'Tools/Doc/bin/yuidoc/.git/HEAD'
error: Invalid path 'Tools/Doc/bin/yuidoc/.git/ORIG_HEAD’
[...]

Those files have been added then removed in another commit

I’ve have to make git reset —hard ‘HEAD^’ && git p4 sync to a clean staging area right after the "clone".




> Thanks,
> 
> 		-- Pete
> 
> ----8<--------
> 
> From 8556ab04dd126184e26a380b7ed08998fd33debe Mon Sep 17 00:00:00 2001
> From: Pete Wyckoff <pw@padd.com>
> Date: Thu, 16 Jan 2014 18:34:09 -0500
> Subject: [PATCH] git p4: work around p4 bug that causes empty symlinks
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> 
> Damien Gérard highlights an interesting problem.  Some p4
> repositories end up with symlinks that have an empty target.  It
> is not possible to create this with current p4, but they do
> indeed exist.
> 
> The effect in git p4 is that "p4 print" on the symlink returns an
> empty string, confusing the curret symlink-handling code.
> 
> In p4, syncing to a change that includes such a bogus symlink
> creates errors:
> 
>    //depot/empty-symlink - updating /home/me/p4/empty-symlink
>    rename: /home/me/p4/empty-symlink: No such file or directory
> 
> and leaves no symlink.
> 
> Replicate the p4 behavior by ignoring these bogus symlinks.  If
> they are fixed in later revisions, the symlink will be replaced
> properly.
> 
> Reported-by: Damien Gérard <damien@iwi.me>
> Signed-off-by: Pete Wyckoff <pw@padd.com>
> ---
> git-p4.py | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/git-p4.py b/git-p4.py
> index 5ea8bb8..e798ecf 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -2075,7 +2075,14 @@ class P4Sync(Command, P4UserMap):
>             # p4 print on a symlink sometimes contains "target\n";
>             # if it does, remove the newline
>             data = ''.join(contents)
> -            if data[-1] == '\n':
> +            if not data:
> +                # Some version of p4 allowed creating a symlink that pointed
> +                # to nothing.  This causes p4 errors when checking out such
> +                # a change, and errors here too.  Work around it by ignoring
> +                # the bad symlink; hopefully a future change fixes it.
> +                print "\nIgnoring empty symlink in %s" % file['depotFile']
> +                return
> +            elif data[-1] == '\n':
>                 contents = [data[:-1]]
>             else:
>                 contents = [data]
> -- 
> 1.8.5.2.320.g99957e5
> 
>