Am I doing something wrong?

Am I doing something wrong?

[Randal L. Schwartz]

localhost:~/MIRROR/git-GIT % cg-fetch
Recovering from a previously interrupted fetch...
Fetching head...
Fetching objects...
Getting alternates list for http://www.kernel.org/pub/scm/git/git.git/
Getting pack list for http://www.kernel.org/pub/scm/git/git.git/
error: Unable to find c2f3bf071ee90b01f2d629921bb04c4f798f02fa under http://www.kernel.org/pub/scm/git/git.git/

Cannot obtain needed object c2f3bf071ee90b01f2d629921bb04c4f798f02fa
while processing commit 0000000000000000000000000000000000000000.
cg-fetch: objects fetch failed

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: Am I doing something wrong?

[Junio C Hamano]

merlyn@stonehenge.com (Randal L. Schwartz) writes:

> localhost:~/MIRROR/git-GIT % cg-fetch
> Recovering from a previously interrupted fetch...

Sorry, I do not know what Cogito is doing there (the message is
from an "echo" in cg-fetch).  git-clone over HTTP seems to work
for me, and I just tried git-fetch over HTTP in my random
back-up repo that happened to be at around 0.99.9g without
problems, so I am reasonably sure the public repository is not
corrupt.

Re: Am I doing something wrong?

[Randal L. Schwartz]

>>>>> "Junio" == Junio C Hamano <junkio@cox.net> writes:

Junio> merlyn@stonehenge.com (Randal L. Schwartz) writes:
>> localhost:~/MIRROR/git-GIT % cg-fetch
>> Recovering from a previously interrupted fetch...

Junio> Sorry, I do not know what Cogito is doing there (the message is
Junio> from an "echo" in cg-fetch).  git-clone over HTTP seems to work
Junio> for me, and I just tried git-fetch over HTTP in my random
Junio> back-up repo that happened to be at around 0.99.9g without
Junio> problems, so I am reasonably sure the public repository is not
Junio> corrupt.

It worked once I changed the URL from http://www to git://git

Perhaps something is wrong with the archive when viewed with HTTP protocol?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: Am I doing something wrong?

[Junio C Hamano]

merlyn@stonehenge.com (Randal L. Schwartz) writes:

>>>>>> "Junio" == Junio C Hamano <junkio@cox.net> writes:
>
> Junio> Sorry, I do not know what Cogito is doing there (the message is
> Junio> from an "echo" in cg-fetch).  git-clone over HTTP seems to work
> Junio> for me, and I just tried git-fetch over HTTP in my random
> Junio> back-up repo that happened to be at around 0.99.9g without
> Junio> problems, so I am reasonably sure the public repository is not
> Junio> corrupt.
>
> It worked once I changed the URL from http://www to git://git
>
> Perhaps something is wrong with the archive when viewed with HTTP protocol?

I said I tried cloning from scratch and fetching into an old one
over HTTP and succeeded, did I not?  So it does not seem to be
the case either.

My initial reaction after seeing the two other messages that
report failure to grab the tip of the master head was "Perhaps,
the usual mirroring delay".  But considering that your failure
happened way later than when the release material was placed on
the server that public servers mirror from (which was around
01:00 US/Pacific), I do not know what to think (especially
before my caffeine ;-).

Re: Am I doing something wrong?

[Randal L. Schwartz]

>>>>> "Junio" == Junio C Hamano <junkio@cox.net> writes:

Junio> I said I tried cloning from scratch and fetching into an old one
Junio> over HTTP and succeeded, did I not?  So it does not seem to be
Junio> the case either.

Cloning from scratch probably worked differently than starting
from wherever I was.

Sorry, I didn't preserve the exact state from where I started... I
think my last update was yesterday afternoon.  Would have been a good
test case.  However, I'm very clear on the symptoms... trying to start
from wherever I had last updated, to update to the current release,
failed when sucking from HTTP, but worked when sucking from GIT.

In fact, I got the same exact behavior both on my Darwin laptop and my
OpenBSD box, so I had to edit them both to get away from HTTP
protocol.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: Am I doing something wrong?

[Junio C Hamano]

merlyn@stonehenge.com (Randal L. Schwartz) writes:

>>>>>> "Junio" == Junio C Hamano <junkio@cox.net> writes:
>
> Junio> I said I tried cloning from scratch and fetching into an old one
> Junio> over HTTP and succeeded, did I not?  So it does not seem to be
> Junio> the case either.
>
> Cloning from scratch probably worked differently than starting
> from wherever I was.

Yup, that's why I tried both.

> ... However, I'm very clear on the symptoms... trying to start
> from wherever I had last updated, to update to the current release,
> failed when sucking from HTTP, but worked when sucking from GIT.

I do not doubt you got a failure.  I just couldn't reproduce it
easily by fetching into a copy of a stale repository.  Earlier I
was suspecting mirroring lags (my successful trial was done
within the last hour) but I do not think that is the case
because your failure was also way after the mirroring should
have happened.

Will dig a bit more when able.

Re: Am I doing something wrong?

[Peter Baumann]

2005/12/21, Junio C Hamano <junkio@cox.net>:
> merlyn@stonehenge.com (Randal L. Schwartz) writes:
>
> >>>>>> "Junio" == Junio C Hamano <junkio@cox.net> writes:
> >
> > Junio> I said I tried cloning from scratch and fetching into an old one
> > Junio> over HTTP and succeeded, did I not?  So it does not seem to be
> > Junio> the case either.
> >
> > Cloning from scratch probably worked differently than starting
> > from wherever I was.
>
> Yup, that's why I tried both.
>
> > ... However, I'm very clear on the symptoms... trying to start
> > from wherever I had last updated, to update to the current release,
> > failed when sucking from HTTP, but worked when sucking from GIT.
>
> I do not doubt you got a failure.  I just couldn't reproduce it
> easily by fetching into a copy of a stale repository.  Earlier I
> was suspecting mirroring lags (my successful trial was done
> within the last hour) but I do not think that is the case
> because your failure was also way after the mirroring should
> have happened.
>
> Will dig a bit more when able.
>

Sorry , but I have seen this too. I deleted my git directory and
cloned it again and it worked, but the original pull didn't work.
(Same error as above.) I'll look if I can find a backup of my old git
directory and look if there is the same error for you to reproduce. If
I can find something, I'll get back to you.

Peter Baumann

Re: Am I doing something wrong?

[Peter Baumann]

2005/12/21, Peter Baumann <peter.baumann@gmail.com>:
> 2005/12/21, Junio C Hamano <junkio@cox.net>:
> > merlyn@stonehenge.com (Randal L. Schwartz) writes:
> >
> > >>>>>> "Junio" == Junio C Hamano <junkio@cox.net> writes:
> > >
> > > Junio> I said I tried cloning from scratch and fetching into an old one
> > > Junio> over HTTP and succeeded, did I not?  So it does not seem to be
> > > Junio> the case either.
> > >
> > > Cloning from scratch probably worked differently than starting
> > > from wherever I was.
> >
> > Yup, that's why I tried both.
> >
> > > ... However, I'm very clear on the symptoms... trying to start
> > > from wherever I had last updated, to update to the current release,
> > > failed when sucking from HTTP, but worked when sucking from GIT.
> >
> > I do not doubt you got a failure.  I just couldn't reproduce it
> > easily by fetching into a copy of a stale repository.  Earlier I
> > was suspecting mirroring lags (my successful trial was done
> > within the last hour) but I do not think that is the case
> > because your failure was also way after the mirroring should
> > have happened.
> >
> > Will dig a bit more when able.
> >
>
> Sorry , but I have seen this too. I deleted my git directory and
> cloned it again and it worked, but the original pull didn't work.
> (Same error as above.) I'll look if I can find a backup of my old git
> directory and look if there is the same error for you to reproduce. If
> I can find something, I'll get back to you.
>
> Peter Baumann
>

Ok. I digged a bit around and found the backup of my old git version repository.
xp:/var/tmp/git cat .git/HEAD
112d0bafd620c0e0f10614a3ba021d4de4fae331

xp:/var/tmp/git git --version
git version 0.99.9i

But I wouldn't take this for serious if I were you. If the process for
building the debian package isnt't able to set the exact version the
above is definitly not true. Can't remember what version it was :-(

xp:/var/tmp/git git pull
Fetching refs/heads/master from
http://www.kernel.org/pub/scm/git/git.git using http
Getting alternates list for http://www.kernel.org/pub/scm/git/git.git/
Getting pack list for http://www.kernel.org/pub/scm/git/git.git/
error: Unable to find c2f3bf071ee90b01f2d629921bb04c4f798f02fa under
http://www.kernel.org/pub/scm/git/git.git/

Cannot obtain needed object c2f3bf071ee90b01f2d629921bb04c4f798f02fa
while processing commit 0000000000000000000000000000000000000000.


Peter

Re: Am I doing something wrong?

[Junio C Hamano]

It turns out to be a stupid off-by-one error in http-fetch.c.
The objects/info/packs file has the following:

        $ cat objects/info/packs
        P pack-46ff81b11ed16ed38caa4aada913cb08c00185b2.pack
        P pack-05f611b3b8198b262acdf678584d365f8e879aec.pack
        P pack-740c99c0be6734adbd130737dec2608dc4682761.pack
        $

The code that parses this in http-fetch had an off-by-one, and
incorrectly thought that the last entry was incomplete.  This
problem did not surface earlier, because objects/info/packs file
used to have other information after all the P lines, but
http-fetch was not interested in was skipping them.  The latest
update-server-info stopped producing those extra lines.

As a band-aid, I just ran this command at the server that public
ones mirror from:

	$ echo >>objects/info/packs

The change seems to have mirrored out already, and my "broken"
client before the attached patch successfully fetches from
there, so hopefully things would work OK for you as well.

I also have to add some code to server-info.c, to append an
empty after objects/info/packs file to work around this bug in
the deployed http clients.

Thanks for your help in diagnosing and fixing this problem.

-- >8 --
[PATCH] http-fetch.c: fix objects/info/pack parsing.

It failed to register the last pack mentioned in the
objects/info/packs file.  Also it had an independent overrun
error.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---
git diff
diff --git a/http-fetch.c b/http-fetch.c
index ad59f1c..3cd6ef9 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -658,7 +658,7 @@ static int fetch_indices(struct alt_base
 		switch (data[i]) {
 		case 'P':
 			i++;
-			if (i + 52 < buffer.posn &&
+			if (i + 52 <= buffer.posn &&
 			    !strncmp(data + i, " pack-", 6) &&
 			    !strncmp(data + i + 46, ".pack\n", 6)) {
 				get_sha1_hex(data + i + 6, sha1);
@@ -667,7 +667,7 @@ static int fetch_indices(struct alt_base
 				break;
 			}
 		default:
-			while (data[i] != '\n')
+			while (i < buffer.posn && data[i] != '\n')
 				i++;
 		}
 		i++;

Compilation finished at Wed Dec 21 11:40:06

Re: Am I doing something wrong?

[Chuck Lever]

[-- Attachment #1: Type: text/plain, Size: 2597 bytes --]

maybe the Linux kernel repository needs the same treatment.  i'm getting 
the "all zeros" commit error message when trying to pull from it.


Junio C Hamano wrote:
> It turns out to be a stupid off-by-one error in http-fetch.c.
> The objects/info/packs file has the following:
> 
>         $ cat objects/info/packs
>         P pack-46ff81b11ed16ed38caa4aada913cb08c00185b2.pack
>         P pack-05f611b3b8198b262acdf678584d365f8e879aec.pack
>         P pack-740c99c0be6734adbd130737dec2608dc4682761.pack
>         $
> 
> The code that parses this in http-fetch had an off-by-one, and
> incorrectly thought that the last entry was incomplete.  This
> problem did not surface earlier, because objects/info/packs file
> used to have other information after all the P lines, but
> http-fetch was not interested in was skipping them.  The latest
> update-server-info stopped producing those extra lines.
> 
> As a band-aid, I just ran this command at the server that public
> ones mirror from:
> 
> 	$ echo >>objects/info/packs
> 
> The change seems to have mirrored out already, and my "broken"
> client before the attached patch successfully fetches from
> there, so hopefully things would work OK for you as well.
> 
> I also have to add some code to server-info.c, to append an
> empty after objects/info/packs file to work around this bug in
> the deployed http clients.
> 
> Thanks for your help in diagnosing and fixing this problem.
> 
> -- >8 --
> [PATCH] http-fetch.c: fix objects/info/pack parsing.
> 
> It failed to register the last pack mentioned in the
> objects/info/packs file.  Also it had an independent overrun
> error.
> 
> Signed-off-by: Junio C Hamano <junkio@cox.net>
> 
> ---
> git diff
> diff --git a/http-fetch.c b/http-fetch.c
> index ad59f1c..3cd6ef9 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -658,7 +658,7 @@ static int fetch_indices(struct alt_base
>  		switch (data[i]) {
>  		case 'P':
>  			i++;
> -			if (i + 52 < buffer.posn &&
> +			if (i + 52 <= buffer.posn &&
>  			    !strncmp(data + i, " pack-", 6) &&
>  			    !strncmp(data + i + 46, ".pack\n", 6)) {
>  				get_sha1_hex(data + i + 6, sha1);
> @@ -667,7 +667,7 @@ static int fetch_indices(struct alt_base
>  				break;
>  			}
>  		default:
> -			while (data[i] != '\n')
> +			while (i < buffer.posn && data[i] != '\n')
>  				i++;
>  		}
>  		i++;
> 
> Compilation finished at Wed Dec 21 11:40:06
> 
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: cel.vcf --]
[-- Type: text/x-vcard, Size: 451 bytes --]

begin:vcard
fn:Chuck Lever
n:Lever;Charles
org:Network Appliance, Incorporated;Open Source NFS Client Development
adr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA
email;internet:cel@citi.umich.edu
title:Member of Technical Staff
tel;work:+1 734 763 4415
tel;fax:+1 734 763 4434
tel;home:+1 734 668 1089
x-mozilla-html:FALSE
url:http://troy.citi.umich.edu/u/cel/
version:2.1
end:vcard

Re: Am I doing something wrong?

[Junio C Hamano]

Chuck Lever <cel@citi.umich.edu> writes:

> maybe the Linux kernel repository needs the same treatment.  i'm getting 
> the "all zeros" commit error message when trying to pull from it.

Indeed.

Linus, sorry about bothering you, but could you do the same
band-aid for now and also apply the attached to your
server-info.c, please?

> Junio C Hamano wrote:
>>
>> As a band-aid, I just ran this command at the server that public
>> ones mirror from:
>>
>> 	$ echo >>objects/info/packs
>>

GIT 1.0.0a needs to follow soonish X-<.

-- >8 --
[PATCH] objects/info/packs: work around bug in http-fetch.c::fetch_indices()

The code to fetch pack index files in deployed clients have a
bug that causes it to ignore the pack file on the last line of
objects/info/packs file, so append an empty line to work it
around.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---

 server-info.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

21b1aced83195af50fd8bc9a88e7734c8ee77c0e
diff --git a/server-info.c b/server-info.c
index df19e49..6089765 100644
--- a/server-info.c
+++ b/server-info.c
@@ -200,6 +200,7 @@ static void write_pack_info_file(FILE *f
 	int i;
 	for (i = 0; i < num_pack; i++)
 		fprintf(fp, "P %s\n", info[i]->p->pack_name + objdirlen + 6);
+	fputc('\n', fp);
 }
 
 static int update_info_packs(int force)
-- 
1.0.0