HELP: GIT Cloning failed

HELP: GIT Cloning failed

[Ju, Seokmann]

Hi,

Recently, I found that cloning from GIT server has been failed.
I'm using following script for it.
---
...
rm -r /home/git/kernels/2.4/linux-2.4.git
cg-clone
http://www.kernel.org/pub/scm/linux/kernel/git/marcelo/linux-2.4.git/
/home/git/kernels/2.4/linux-2.4.git/
sync
rm -r /home/git/kernels/2.4/linux-2.6.git
cg-clone
http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
/home/git/kernels/2.4/linux-2.6.git/
sync
rm -r /home/git/kernels/2.4/scsi-misc-2.6.git
cg-clone
http://www.kernel.org/pub/scm/linux/kernel/git/marcelo/scsi-misc-2.6.git
/home/git/kernels/2.4/scsi-misc-2.6.git
sync
...
---

In the script, I'm cloning 3 different sources. First two sources
getting successfully cloned, however, last one is getting failed with
following error messages,
---
Fetching head...
Fetching objects...
Getting alternates list for
http://www.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6.git
Also look at http://www.kernel.or
Error: The requested URL returned error: 502 (curl_result = 22,
http_code = 502, sha1 = 1039f0760e...)
Getting pack list for
http://www.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6.git/
Getting pack list for http://www.kernel.or
Error: The requested URL returned error: 502
Error: Unable to find 27fd37621... Under
http://www.kernel.org/pub/scsm/linux/kernel/git/jejb/scsi-misc-2.6.git/
Cannot obtain needed blob 27fd37621...
While processing commit 4041b9cd87...
Progress: 8 objects, 13120 bytes
Cg-fetch: objects fetch failed
---

Above script worked without any problem when I started several months
ago and I'm not sure when did it stop working.
I'm using _cron_ utility on my Linux box for scheduled execution of the
script.

Any comment would be appreciated.

Thank you,

Seokmann

Re: HELP: GIT Cloning failed

[Sean]

On Tue, 22 Aug 2006 13:25:23 -0600
"Ju, Seokmann" <Seokmann.Ju@lsil.com> wrote:

Ju,

> Above script worked without any problem when I started several months
> ago and I'm not sure when did it stop working.
> I'm using _cron_ utility on my Linux box for scheduled execution of the
> script.
> 
> Any comment would be appreciated.

It looks like the jejb scsi-misc archive might not have been
configured properly for http transfers.  There's a script below
which just uses git commands (not cogito) and the native git
protocol which tests out okay here.

A few comments though:

 - Apparently the scsi misc tree you want is actually under "jejb"
   rather than "marcelo" which you had in your script.

 - It's better to use the native git protocol when possible
   (well, the benefits are less on initial clone, but it's still
   a good practice)

 - You're causing a lot of unnecessary traffic for kernel.org by
   cloning a fresh copy of all these trees, it would be much better
   to clone just _once_ and then simply "git pull" in any updates.

 - Two of the trees you're cloning are very close in content to
   each other (linux-2.6 & scsi-misc-2.6) so you can use the git
   "--reference" option to share local objects saving disk space
   (and reducing bandwidth needs even further)

Cheers,
Sean

#!/bin/sh
cd /home/git/kernels/2.4 || exit
BASE="git://www.kernel.org/pub/scm/linux/kernel/git"
rm -rf linux-2.4 linux-2.6 scsi-misc-2.6
git clone $BASE/marcelo/linux-2.4
git clone $BASE/torvalds/linux-2.6
git clone --reference linux-2.6 $BASE/jejb/scsi-misc-2.6

Re: HELP: GIT Cloning failed

[Sean]

On Tue, 22 Aug 2006 13:25:23 -0600
"Ju, Seokmann" <Seokmann.Ju@lsil.com> wrote:

Ju,

> Above script worked without any problem when I started several months
> ago and I'm not sure when did it stop working.
> I'm using _cron_ utility on my Linux box for scheduled execution of the
> script.
> 
> Any comment would be appreciated.

It looks like the jejb scsi-misc archive might not have been
configured properly for http transfers.  There's a script below
which just uses git commands (not cogito) and the native git
protocol which tests out okay here.

A few comments though:

 - Apparently the scsi misc tree you want is actually under "jejb"
   rather than "marcelo" which you had in your script.

 - It's better to use the native git protocol when possible
   (well, the benefits are less on initial clone, but it's still
   a good practice)

 - You're causing a lot of unnecessary traffic for kernel.org by
   cloning a fresh copy of all these trees, it would be much better
   to clone just _once_ and then simply "git pull" in any updates.

 - Two of the trees you're cloning are very close in content to
   each other (linux-2.6 & scsi-misc-2.6) so you can use the git
   "--reference" option to share local objects saving disk space
   (and reducing bandwidth needs even further)

Cheers,
Sean

#!/bin/sh
cd /home/git/kernels/2.4 || exit
BASE="git://www.kernel.org/pub/scm/linux/kernel/git"
rm -rf linux-2.4 linux-2.6 scsi-misc-2.6
git clone $BASE/marcelo/linux-2.4
git clone $BASE/torvalds/linux-2.6
git clone --reference linux-2.6 $BASE/jejb/scsi-misc-2.6

Re: HELP: GIT Cloning failed

[Phillip Susi]

Is there a reason that you regularly delete and re-download the entire 
repository, rather than simply doing a git pull to update your existing 
clone?

Ju, Seokmann wrote:
> Hi,
> 
> Recently, I found that cloning from GIT server has been failed.
> I'm using following script for it.
> ---
> ...
> rm -r /home/git/kernels/2.4/linux-2.4.git
> cg-clone
> http://www.kernel.org/pub/scm/linux/kernel/git/marcelo/linux-2.4.git/
> /home/git/kernels/2.4/linux-2.4.git/
> sync
> rm -r /home/git/kernels/2.4/linux-2.6.git
> cg-clone
> http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> /home/git/kernels/2.4/linux-2.6.git/
> sync
> rm -r /home/git/kernels/2.4/scsi-misc-2.6.git
> cg-clone
> http://www.kernel.org/pub/scm/linux/kernel/git/marcelo/scsi-misc-2.6.git
> /home/git/kernels/2.4/scsi-misc-2.6.git
> sync
> ...
> ---
> 
> In the script, I'm cloning 3 different sources. First two sources
> getting successfully cloned, however, last one is getting failed with
> following error messages,
> ---
> Fetching head...
> Fetching objects...
> Getting alternates list for
> http://www.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6.git
> Also look at http://www.kernel.or
> Error: The requested URL returned error: 502 (curl_result = 22,
> http_code = 502, sha1 = 1039f0760e...)
> Getting pack list for
> http://www.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6.git/
> Getting pack list for http://www.kernel.or
> Error: The requested URL returned error: 502
> Error: Unable to find 27fd37621... Under
> http://www.kernel.org/pub/scsm/linux/kernel/git/jejb/scsi-misc-2.6.git/
> Cannot obtain needed blob 27fd37621...
> While processing commit 4041b9cd87...
> Progress: 8 objects, 13120 bytes
> Cg-fetch: objects fetch failed
> ---
> 
> Above script worked without any problem when I started several months
> ago and I'm not sure when did it stop working.
> I'm using _cron_ utility on my Linux box for scheduled execution of the
> script.
> 
> Any comment would be appreciated.
> 
> Thank you,
> 
> Seokmann

RE: HELP: GIT Cloning failed

[Ju, Seokmann]

Hi,
On Tuesday, August 22, 2006 4:08 PM, Sean  wrote:
> It looks like the jejb scsi-misc archive might not have been
> configured properly for http transfers.  There's a script below
> which just uses git commands (not cogito) and the native git
> protocol which tests out okay here.
I will try with the script and rest of your comments as well.
Thank you for your guidence.

Regards,

Seokmann


> -----Original Message-----
> From: Sean [mailto:seanlkml@sympatico.ca] 
> Sent: Tuesday, August 22, 2006 4:08 PM
> To: Ju, Seokmann
> Cc: git@vger.kernel.org; Patro, Sumant; linux-kernel@vger.kernel.org
> Subject: Re: HELP: GIT Cloning failed
> 
> On Tue, 22 Aug 2006 13:25:23 -0600
> "Ju, Seokmann" <Seokmann.Ju@lsil.com> wrote:
> 
> Ju,
> 
> > Above script worked without any problem when I started 
> several months
> > ago and I'm not sure when did it stop working.
> > I'm using _cron_ utility on my Linux box for scheduled 
> execution of the
> > script.
> > 
> > Any comment would be appreciated.
> 
> It looks like the jejb scsi-misc archive might not have been
> configured properly for http transfers.  There's a script below
> which just uses git commands (not cogito) and the native git
> protocol which tests out okay here.
> 
> A few comments though:
> 
>  - Apparently the scsi misc tree you want is actually under "jejb"
>    rather than "marcelo" which you had in your script.
> 
>  - It's better to use the native git protocol when possible
>    (well, the benefits are less on initial clone, but it's still
>    a good practice)
> 
>  - You're causing a lot of unnecessary traffic for kernel.org by
>    cloning a fresh copy of all these trees, it would be much better
>    to clone just _once_ and then simply "git pull" in any updates.
> 
>  - Two of the trees you're cloning are very close in content to
>    each other (linux-2.6 & scsi-misc-2.6) so you can use the git
>    "--reference" option to share local objects saving disk space
>    (and reducing bandwidth needs even further)
> 
> Cheers,
> Sean
> 
> #!/bin/sh
> cd /home/git/kernels/2.4 || exit
> BASE="git://www.kernel.org/pub/scm/linux/kernel/git"
> rm -rf linux-2.4 linux-2.6 scsi-misc-2.6
> git clone $BASE/marcelo/linux-2.4
> git clone $BASE/torvalds/linux-2.6
> git clone --reference linux-2.6 $BASE/jejb/scsi-misc-2.6
> 
> 
> 
>