Git clone stalls at a read(3, ...) saw using strace

Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

Hi, all!

    Please CC me as I'm not subscribed.

    Using  GIT 1.4.1, I tried to get this repository:

# git clone git://source.mvista.com/git/linux-davinci-2.6.git

    After some time the download stalled completely. I used ps and
saw two git processes.

    Using strace, I saw that the first one, "git clone", was waiting
with waitpid()
for its child termination.
    Its child, "git pack --all -k", was stalled at a "read(3, ...)"

    I went to /proc and saw that fd 3 was a pipe. Anyone know why this
is happening?

    I found an issue somewhat related in google (not sure that is the
same problem):

http://groups.google.com/group/fa.linux.kernel/browse_thread/thread/e64a92f28def32a7/93bd8bf5df1fcc66?lnk=st&q=&rnum=1#93bd8bf5df1fcc66

    It stalled at the first 60 Mb transfered. After CTRL+C it deletes
and I have to start again from scratch.

Thank you for the great alternative to bitkeeper and CVS,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[Pavel Roskin]

On Thu, 2006-07-27 at 08:16 -0400, André Goddard Rosa wrote:
> Hi, all!
> 
>     Please CC me as I'm not subscribed.
> 
>     Using  GIT 1.4.1, I tried to get this repository:

Please try git 1.4.1.1, it fixes git-clone.

-- 
Regards,
Pavel Roskin

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> On Thu, 2006-07-27 at 08:16 -0400, André Goddard Rosa wrote:
> > Hi, all!
> >
> >     Please CC me as I'm not subscribed.
> >
> >     Using  GIT 1.4.1, I tried to get this repository:
>
> Please try git 1.4.1.1, it fixes git-clone.

Thank you so much, Pavel!

Downloading it right now!

Thanks a lot,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[Linus Torvalds]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 890 bytes --]



On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> > 
> > Please try git 1.4.1.1, it fixes git-clone.
> 
> Thank you so much, Pavel!
> 
> Downloading it right now!

The problem is that at least the timeout-fix is a server-side thing, it 
won't fix the client. So if a server is running the broken 1.4.1 release, 
it will time out the clients after 10 minutes. It wasn't obvious breakage, 
because most people do smaller fetches, and never even got close to that 
timeout (most of my connections tend to be in the seconds ;).

I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this 
week, so I'm bcc'ing the other admins here to ask them to upgrade to 
1.4.1.1. That won't fix the montavista site, though.

That said, I don't think the timeout problem should have caused a _stall_ 
of the connection, it should have caused an early EOF and thus a unpack 
error. 

		Linus

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/27/06, Linus Torvalds <torvalds@osdl.org> wrote:
> On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> > >
> > > Please try git 1.4.1.1, it fixes git-clone.
> >
> > Thank you so much, Pavel!
> >
> > Downloading it right now!
>
> The problem is that at least the timeout-fix is a server-side thing, it
> won't fix the client. So if a server is running the broken 1.4.1 release,
> it will time out the clients after 10 minutes. It wasn't obvious breakage,
> because most people do smaller fetches, and never even got close to that
> timeout (most of my connections tend to be in the seconds ;).
>
> I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this
> week, so I'm bcc'ing the other admins here to ask them to upgrade to
> 1.4.1.1. That won't fix the montavista site, though.
>
> That said, I don't think the timeout problem should have caused a _stall_
> of the connection, it should have caused an early EOF and thus a unpack
> error.

Great! Thank you so much, I will try to contact Montavista.

Best regards,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> On 7/27/06, Linus Torvalds <torvalds@osdl.org> wrote:
> > On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> > > >
> > > > Please try git 1.4.1.1, it fixes git-clone.
> > >
> > > Thank you so much, Pavel!
> > >
> > > Downloading it right now!
> >
> > The problem is that at least the timeout-fix is a server-side thing, it
> > won't fix the client. So if a server is running the broken 1.4.1 release,
> > it will time out the clients after 10 minutes. It wasn't obvious breakage,
> > because most people do smaller fetches, and never even got close to that
> > timeout (most of my connections tend to be in the seconds ;).
> >
> > I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this
> > week, so I'm bcc'ing the other admins here to ask them to upgrade to
> > 1.4.1.1. That won't fix the montavista site, though.
> >
> > That said, I don't think the timeout problem should have caused a _stall_
> > of the connection, it should have caused an early EOF and thus a unpack
> > error.
>
> Great! Thank you so much, I will try to contact Montavista.

Hi, Linus!

The problem happened again, and it stalls AFAICS.
Can you reproduce there?

# git clone git://source.mvista.com/git/linux-davinci-2.6.git

Here I have a constant 55 kbps and it stalls with about 117Mb downloaded.

I will try to contact Montavista.

Thanks you so much for the support,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[Linus Torvalds]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 391 bytes --]



On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> 
> The problem happened again, and it stalls AFAICS.
> Can you reproduce there?
> 
> # git clone git://source.mvista.com/git/linux-davinci-2.6.git

Nope. I have a fairly constant 120kbps, and:

[torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
Checking files out...)      
 100% (19754/19754) done

here

		Linus

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/27/06, Linus Torvalds <torvalds@osdl.org> wrote:
>
>
> On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> >
> > The problem happened again, and it stalls AFAICS.
> > Can you reproduce there?
> >
> > # git clone git://source.mvista.com/git/linux-davinci-2.6.git
>
> Nope. I have a fairly constant 120kbps, and:
>
> [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> Checking files out...)
>  100% (19754/19754) done
> here

Do you think you can reproduce the server timeout by CTRL + Z and
stopping the process for a while?

I sent an email to the last one who commited in that tree asking for
updating the git version there. I also sent and email to the maillist
owner asking this.

Thanks again,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[Pavel Roskin]

On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> Nope. I have a fairly constant 120kbps, and:
> 
> [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> Checking files out...)      
>  100% (19754/19754) done

Same thing here.  Current git from the master branch.

-- 
Regards,
Pavel Roskin

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > Nope. I have a fairly constant 120kbps, and:
> >
> > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > Checking files out...)
> >  100% (19754/19754) done
>
> Same thing here.  Current git from the master branch.

Forgot to say that we are using this script in GIT_PROXY_COMMAND
environment variable:

(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
| (read a; read a; cat )

The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and
the second removes an empty line as described here:

http://www.gelato.unsw.edu.au/archives/git/0605/20664.html

I will try from home later again.

Thanks,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > > Nope. I have a fairly constant 120kbps, and:
> > >
> > > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > > Checking files out...)
> > >  100% (19754/19754) done
> >
> > Same thing here.  Current git from the master branch.
>
> Forgot to say that we are using this script in GIT_PROXY_COMMAND
> environment variable:
>
> (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
> | (read a; read a; cat )
>
> The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and
> the second removes an empty line as described here:
>
> http://www.gelato.unsw.edu.au/archives/git/0605/20664.html
>
> I will try from home later again.

Okey, I tried from home (without the proxy trick) and it behaved a lot
better but my disc went full in the process and I got these messages:
...
...
...
error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c
error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h
error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c
error: git-checkout-index: unable to create file
drivers/scsi/mac_scsi.c (No space left on device)
error: git-checkout-index: unable to create file
drivers/scsi/mac_scsi.h (No space left on device)
error: git-checkout-index: unable to create file
drivers/scsi/mca_53c9x.c (No space left on device)
error: git-checkout-index: unable to create file
drivers/scsi/megaraid.c (No space left on device)
error: git-checkout-index: unable to create file
drivers/scsi/megaraid.h (No space left on device)
fatal: cannot create directory at drivers/scsi/megaraid

And it finished keeping the downloaded files, but I still cannot see
these files listed above.
I tried to pull but it says that I'm up-to-date:

doctorture:/opt/downloads/mvista/linux-mvista # git-pull
Already up-to-date.

I remember that using CVS I just used 'cvs update' after checkout and
it would bring the missing files to me.

What I'm doing wrong here?

Thank you so much,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/28/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> > On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> > > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > > > Nope. I have a fairly constant 120kbps, and:
> > > >
> > > > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > > > Checking files out...)
> > > >  100% (19754/19754) done
> > >
> > > Same thing here.  Current git from the master branch.
> >
> > Forgot to say that we are using this script in GIT_PROXY_COMMAND
> > environment variable:
> >
> > (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
> > | (read a; read a; cat )
> >
> > The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and
> > the second removes an empty line as described here:
> >
> > http://www.gelato.unsw.edu.au/archives/git/0605/20664.html
> >
> > I will try from home later again.
>
> Okey, I tried from home (without the proxy trick) and it behaved a lot
> better but my disc went full in the process and I got these messages:
> ...
> ...
> ...
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h
> error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.h (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mca_53c9x.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.h (No space left on device)
> fatal: cannot create directory at drivers/scsi/megaraid
>
> And it finished keeping the downloaded files, but I still cannot see
> these files listed above.
> I tried to pull but it says that I'm up-to-date:
>
> doctorture:/opt/downloads/mvista/linux-mvista # git-pull
> Already up-to-date.
>
> I remember that using CVS I just used 'cvs update' after checkout and
> it would bring the missing files to me.
>
> What I'm doing wrong here?

I'm also receiving these messages when trying to change branches:

doctorture:/opt/downloads/mvista/linux-mvista # git-checkout origin
fatal: Untracked working tree file '.gitignore' would be overwritten by merge.

Perhaps I will need to download using git-clone again?

Thanks,
-- 
[]s,
André Goddard

RE: Git clone stalls at a read(3, ...) saw using strace

[Ribeiro, Humberto Plinio]

Hi, Andre.

After a "ps -ef" I saw two instances of the script defined by GIT_PROXY_COMMAND. Strangely one of those instances was child of the other. A strace showed the scripts were blocked in a "waitpid(-1,". I've killed the child script and the git clone resumed the process. 

I didn't understand why this blocking on git happened though. The creation of these two instances (one child of the other) was also strange. 

Regards,
Humberto 

---
Obrigado/Best regards/Mit freundlichen Grüßen/Pozdrawiam
Humberto Ribeiro, MSc.
SIEMENS Home and Office Communication
Phone: +55 92 2129 6205
Mobile: +55 92 8146 3147

-----Original Message-----
From: André Goddard Rosa [mailto:andre.goddard@gmail.com] 
Sent: sexta-feira, 28 de julho de 2006 05:58
To: Pavel Roskin
Cc: Git Mailing List; Linus Torvalds; Ribeiro, Humberto Plinio
Subject: Re: Git clone stalls at a read(3, ...) saw using strace

On 7/28/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> > On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> > > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > > > Nope. I have a fairly constant 120kbps, and:
> > > >
> > > > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > > > Checking files out...)
> > > >  100% (19754/19754) done
> > >
> > > Same thing here.  Current git from the master branch.
> >
> > Forgot to say that we are using this script in GIT_PROXY_COMMAND
> > environment variable:
> >
> > (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
> > | (read a; read a; cat )
> >
> > The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and
> > the second removes an empty line as described here:
> >
> > http://www.gelato.unsw.edu.au/archives/git/0605/20664.html
> >
> > I will try from home later again.
>
> Okey, I tried from home (without the proxy trick) and it behaved a lot
> better but my disc went full in the process and I got these messages:
> ...
> ...
> ...
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h
> error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.h (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mca_53c9x.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.h (No space left on device)
> fatal: cannot create directory at drivers/scsi/megaraid
>
> And it finished keeping the downloaded files, but I still cannot see
> these files listed above.
> I tried to pull but it says that I'm up-to-date:
>
> doctorture:/opt/downloads/mvista/linux-mvista # git-pull
> Already up-to-date.
>
> I remember that using CVS I just used 'cvs update' after checkout and
> it would bring the missing files to me.
>
> What I'm doing wrong here?

I'm also receiving these messages when trying to change branches:

doctorture:/opt/downloads/mvista/linux-mvista # git-checkout origin
fatal: Untracked working tree file '.gitignore' would be overwritten by merge.

Perhaps I will need to download using git-clone again?

Thanks,
-- 
[]s,
André Goddard

Re: Git clone stalls at a read(3, ...) saw using strace

[André Goddard Rosa]

On 7/28/06, Ribeiro, Humberto Plinio <humberto.ribeiro@siemens.com> wrote:
> Hi, Andre.
>
> After a "ps -ef" I saw two instances of the script defined by
> GIT_PROXY_COMMAND. Strangely one of those instances was child
> of the other. A strace showed the scripts were blocked in a
>"waitpid(-1,". I've killed the child script and the git clone resumed the
>process.
>
> I didn't understand why this blocking on git happened though. The
> creation of these two instances (one child of the other) was also
> strange.

Yes,  it worked to me too, but I also don't know why.

Let me explain exactly what we had to do so someone perhaps can
explain why we had to do this:

I configured GIT_PROXY_COMMAND pointing to a script with this content:

(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc 172.29.0.6 3128 |
(read a; read a; cat )

Later:

git clone git://source.mvista.com/git/linux-davinci-2.6.git

After some time (downloading 117 Mb) it stalled (git-fetch on a pipe
read) and we had to do this to resume and let it finish:

[root@mao2wx23 tmp]# ps -ef | grep git | grep -v grep
opb694    9975  1451  0 01:39 pts/11   00:00:00 /bin/sh
/usr/bin/git-clone git://source.mvista.com/git/linux-davinci-2.6.git
opb694    9985  9975  3 01:39 pts/11   00:01:56 git-fetch-pack --all
-k git //source.mvista.com git/linux-davinci-2.6.git
[root@mao2wx23 tmp]# ps -ef | grep proxy | grep -v grep
opb694    9986  9985  0 01:39 pts/11   00:00:00 /bin/sh
/home/opb694/proxy-cmd.sh source.mvista.com 9418
opb694    9987  9986  0 01:39 pts/11   00:00:00 /bin/sh
/home/opb694/proxy-cmd.sh source.mvista.com 9418
[root@mao2wx23 tmp]# kill 9987

Thanks again for the patience,
-- 
[]s,
André Goddard