* Partial clone fails with file:/// URI
@ 2020-10-14 16:38 Nipunn Koorapati
2020-10-14 16:56 ` Taylor Blau
0 siblings, 1 reply; 6+ messages in thread
From: Nipunn Koorapati @ 2020-10-14 16:38 UTC (permalink / raw)
To: git
Hi friends,
When I run a git clone with --filter and a file:/// URI - it fails.
Repro steps:
I have the `git` repo cloned and built into the CWD at v2.28.0
git/git clone --filter=blob:none file:///home/nipunn/src/git git2
nipunn@nipunn-dbx:~/src$ git/git clone --filter=blob:none
file:///home/nipunn/src/git git2
Cloning into 'git2'...
warning: templates not found in /home/nipunn/share/git-core/templates
remote: Enumerating objects: 177110, done.
remote: Counting objects: 100% (177110/177110), done.
remote: Compressing objects: 100% (61436/61436), done.
Receiving objects: 100% (177110/177110), 62.58 MiB | 43.95 MiB/s, done.
remote: Total 177110 (delta 114305), reused 176904 (delta 114102), pack-reused 0
Resolving deltas: 100% (114305/114305), done.
fatal: the remote end hung up unexpectedly
fatal: protocol error: bad pack header
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'
It seems to work ok with a remote URI or w/o a --filter arg.
I was able to confirm that the "bad pack header" is a "-1" response
code PH_ERROR_EOF
Perhaps the `fatal: protocol error: bad pack header` - should include
the error code as well.
Thanks
--Nipunn
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Partial clone fails with file:/// URI
2020-10-14 16:38 Partial clone fails with file:/// URI Nipunn Koorapati
@ 2020-10-14 16:56 ` Taylor Blau
2020-10-14 17:04 ` Nipunn Koorapati
0 siblings, 1 reply; 6+ messages in thread
From: Taylor Blau @ 2020-10-14 16:56 UTC (permalink / raw)
To: Nipunn Koorapati; +Cc: git
Hi Nipunn,
On Wed, Oct 14, 2020 at 05:38:15PM +0100, Nipunn Koorapati wrote:
> Hi friends,
>
> When I run a git clone with --filter and a file:/// URI - it fails.
>
> Repro steps:
>
> I have the `git` repo cloned and built into the CWD at v2.28.0
> git/git clone --filter=blob:none file:///home/nipunn/src/git git2
I couldn't quite reproduce this. Here's my reproduction:
#!/bin/sh
rm -fr repo clone
git init repo
(
cd repo
git config uploadpack.allowfilter true
git config uploadpack.allowanysha1inwant true
for f in a b; do
echo $f > $f
git add $f
git commit -m "$f: initial commit"
done
)
git clone --filter=blob:none file://$(pwd)/repo clone
That works on v2.28.0, and v2.29.0 for me. Is git installed properly on
your system?
Thanks,
Taylor
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Partial clone fails with file:/// URI
2020-10-14 16:56 ` Taylor Blau
@ 2020-10-14 17:04 ` Nipunn Koorapati
2020-10-14 17:27 ` Taylor Blau
2020-10-15 1:20 ` Jeff King
0 siblings, 2 replies; 6+ messages in thread
From: Nipunn Koorapati @ 2020-10-14 17:04 UTC (permalink / raw)
To: Taylor Blau; +Cc: git
Hi,
I've been building git from source and directly using the executable. Roughly
```
cd git
make
cd ..
git/git clone --filter=blob:none file://$(pwd)/git a_clone_of_git
```
This is what I was missing:
git config uploadpack.allowanysha1inwant true
If this is not set, then the checkout fails w/ a
fatal: protocol error: bad pack header
Perhaps the error message and/or docs could be improved here to
something with more detail than "bad pack header"?
Not seeing anything to suggest this in the docs
https://git-scm.com/docs/partial-clone
Thanks for the prompt response. Appreciated!
--Nipunn
On Wed, Oct 14, 2020 at 5:56 PM Taylor Blau <me@ttaylorr.com> wrote:
>
> Hi Nipunn,
>
> On Wed, Oct 14, 2020 at 05:38:15PM +0100, Nipunn Koorapati wrote:
> > Hi friends,
> >
> > When I run a git clone with --filter and a file:/// URI - it fails.
> >
> > Repro steps:
> >
> > I have the `git` repo cloned and built into the CWD at v2.28.0
> > git/git clone --filter=blob:none file:///home/nipunn/src/git git2
>
> I couldn't quite reproduce this. Here's my reproduction:
>
> #!/bin/sh
>
> rm -fr repo clone
> git init repo
>
> (
> cd repo
>
> git config uploadpack.allowfilter true
> git config uploadpack.allowanysha1inwant true
>
> for f in a b; do
> echo $f > $f
> git add $f
> git commit -m "$f: initial commit"
> done
> )
>
> git clone --filter=blob:none file://$(pwd)/repo clone
>
> That works on v2.28.0, and v2.29.0 for me. Is git installed properly on
> your system?
>
> Thanks,
> Taylor
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Partial clone fails with file:/// URI
2020-10-14 17:04 ` Nipunn Koorapati
@ 2020-10-14 17:27 ` Taylor Blau
2020-10-14 23:41 ` Nipunn Koorapati
2020-10-15 1:20 ` Jeff King
1 sibling, 1 reply; 6+ messages in thread
From: Taylor Blau @ 2020-10-14 17:27 UTC (permalink / raw)
To: Nipunn Koorapati; +Cc: Taylor Blau, git
On Wed, Oct 14, 2020 at 06:04:18PM +0100, Nipunn Koorapati wrote:
> Hi,
>
> I've been building git from source and directly using the executable. Roughly
> ```
> cd git
> make
> cd ..
> git/git clone --filter=blob:none file://$(pwd)/git a_clone_of_git
> ```
>
> This is what I was missing:
> git config uploadpack.allowanysha1inwant true
>
> If this is not set, then the checkout fails w/ a
>
> fatal: protocol error: bad pack header
>
> Perhaps the error message and/or docs could be improved here to
> something with more detail than "bad pack header"?
> Not seeing anything to suggest this in the docs
> https://git-scm.com/docs/partial-clone
Ah. OK; I feel much better knowing what the difference between our two
setups is. This is a long-standing issue (I remember seeing it when I
was experimenting with partial clones several releases ago) so I'm glad
that we _don't_ have to treat this as a regression in 2.29.
> Thanks for the prompt response. Appreciated!
> --Nipunn
Thanks,
Taylor
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Partial clone fails with file:/// URI
2020-10-14 17:27 ` Taylor Blau
@ 2020-10-14 23:41 ` Nipunn Koorapati
0 siblings, 0 replies; 6+ messages in thread
From: Nipunn Koorapati @ 2020-10-14 23:41 UTC (permalink / raw)
To: Taylor Blau; +Cc: git
I improved the "bad pack header" error message in a patch off of
v2.28.0 (https://github.com/gitgitgadget/git/pull/755)
However, rebasing onto v2.29.0, it seems like the error no longer
occurs during checkout.
Was there some recent improvement? Looking through the changelog, I'm
not finding anything obvious.
It appears that regardless of
git config uploadpack.allowanysha1inwant true
we find `git checkout` succeeds on v2.29.0
There is a test case in the PR above
Also a manual repro:
cd git
git checkout v2.29.0-rc1
make
cd ..
git/git clone --filter=blob:none file://$(pwd)/git a_clone_of_git
Thanks
--Nipunn
On Wed, Oct 14, 2020 at 6:27 PM Taylor Blau <me@ttaylorr.com> wrote:
>
> On Wed, Oct 14, 2020 at 06:04:18PM +0100, Nipunn Koorapati wrote:
> > Hi,
> >
> > I've been building git from source and directly using the executable. Roughly
> > ```
> > cd git
> > make
> > cd ..
> > git/git clone --filter=blob:none file://$(pwd)/git a_clone_of_git
> > ```
> >
> > This is what I was missing:
> > git config uploadpack.allowanysha1inwant true
> >
> > If this is not set, then the checkout fails w/ a
> >
> > fatal: protocol error: bad pack header
> >
> > Perhaps the error message and/or docs could be improved here to
> > something with more detail than "bad pack header"?
> > Not seeing anything to suggest this in the docs
> > https://git-scm.com/docs/partial-clone
>
> Ah. OK; I feel much better knowing what the difference between our two
> setups is. This is a long-standing issue (I remember seeing it when I
> was experimenting with partial clones several releases ago) so I'm glad
> that we _don't_ have to treat this as a regression in 2.29.
>
> > Thanks for the prompt response. Appreciated!
> > --Nipunn
>
> Thanks,
> Taylor
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Partial clone fails with file:/// URI
2020-10-14 17:04 ` Nipunn Koorapati
2020-10-14 17:27 ` Taylor Blau
@ 2020-10-15 1:20 ` Jeff King
1 sibling, 0 replies; 6+ messages in thread
From: Jeff King @ 2020-10-15 1:20 UTC (permalink / raw)
To: Nipunn Koorapati; +Cc: Taylor Blau, git
On Wed, Oct 14, 2020 at 06:04:18PM +0100, Nipunn Koorapati wrote:
> I've been building git from source and directly using the executable. Roughly
> ```
> cd git
> make
> cd ..
> git/git clone --filter=blob:none file://$(pwd)/git a_clone_of_git
> ```
Note that running a built copy directly without installing may end up
running a different version of Git under the hood. Your initial
"git/git" will run the newly built version, but it will put the
to-be-installed exec-path at the front of $PATH (probably
~/libexec/git-core if you haven't configured anything). But since you
haven't installed anything there, that won't do anything, and instead
any git subprocesses it runs (including the server-side upload-pack and
any child fetches it runs to get the blobs) will be done with whatever
version of git is in your $PATH already.
Try:
git/bin-wrappers/git clone ...
which is a shell script that sets up the exec-path properly for running
out of the build directory.
When I run your commands, I get:
error: Server does not allow request for unadvertised object d16e5f041f5d3ad91129c4fe6713c71727895e9c
error: unable to read sha1 file of Documentation/RelNotes/2.1.4.txt (d16e5f041f5d3ad91129c4fe6713c71727895e9c)
and so on for each object. Some older versions of Git did not correctly
handle this situation (they asked for the object even though the server
did not claim to allow it). It was fixed in 2b69d3116f (Merge branch
'mm/fetch-show-error-message-on-unadvertised-object' into maint,
2017-03-28), I think, in v2.21.3. If your system Git is older than that,
this might be the culprit.
> This is what I was missing:
> git config uploadpack.allowanysha1inwant true
Note that you shouldn't need to do that starting with v2.29, because the
v2 protocol is the default (which always allows any sha1 to be
requested).
> If this is not set, then the checkout fails w/ a
>
> fatal: protocol error: bad pack header
>
> Perhaps the error message and/or docs could be improved here to
> something with more detail than "bad pack header"?
The problem is that the client is hanging up when we are trying to read
the pack header. I think in newer versions of Git this is better, as
well (the server is better about sending an over-the-wire ERR packet,
and the client is better about reading and relaying it, though there are
still some races around write() errors here).
-Peff
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-10-15 2:09 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-10-14 16:38 Partial clone fails with file:/// URI Nipunn Koorapati
2020-10-14 16:56 ` Taylor Blau
2020-10-14 17:04 ` Nipunn Koorapati
2020-10-14 17:27 ` Taylor Blau
2020-10-14 23:41 ` Nipunn Koorapati
2020-10-15 1:20 ` Jeff King
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).