git fetch origin $SHA1 doesn't work as often as it could

git fetch origin $SHA1 doesn't work as often as it could

[JP Sugarbroad]

So I'm trying to make cloning the rust repo a little easier, and I
noticed there's a new feature to `git submodule update --depth=1` --
but it doesn't work for this, even though it could.

Reason is that fetch-pack will only fetch by sha1 if the remote
specifies allow_tip/unreachable_sha1, even if the sha1 specified
matches an advertised ref.

My thought is to rework filter_refs() to match on oid if matching on
name fails. Thoughts?

- JP

Re: git fetch origin $SHA1 doesn't work as often as it could

[Junio C Hamano]

JP Sugarbroad <jpsugar@google.com> writes:

> So I'm trying to make cloning the rust repo a little easier, and I
> noticed there's a new feature to `git submodule update --depth=1` --
> but it doesn't work for this, even though it could.
>
> Reason is that fetch-pack will only fetch by sha1 if the remote
> specifies allow_tip/unreachable_sha1, even if the sha1 specified
> matches an advertised ref.

If a project wants its users to do the 'update --depth=1' thing, it
should allow fetching not just from the tip but also reachable.

The commit from the submodule that is bound to the superproject may
no longer be sitting at the tip of any branch in the submodule.  The
hack you are outlining here would not help, and invites "the feature
works some of the time (i.e. the commit happens to be at the tip of
one of the refs) but fails most of the time" bug reports.

So I am not enthused, even though at the technical level, I agree
that it is a good "solution" to pretend as if one of the refs were
requested after the fetch-pack discovers the refs advertised by the
upload-pack.

I just fear that it is a wrong approach to solve the real issue and
instead make it worse by relieving the pressure from the project and
hosting site to configure their repository to support their users
properly.

Re: git fetch origin $SHA1 doesn't work as often as it could

[JP Sugarbroad]

On Wed, May 18, 2016 at 5:59 PM, Junio C Hamano <gitster@pobox.com> wrote:
> The commit from the submodule that is bound to the superproject may
> no longer be sitting at the tip of any branch in the submodule.  The
> hack you are outlining here would not help, and invites "the feature
> works some of the time (i.e. the commit happens to be at the tip of
> one of the refs) but fails most of the time" bug reports.

Sure, that's a reasonable fear. In this case the branches don't move,
but if one were concerned about that one could use tags instead. It is
quite reasonable to point a submodule to a tag and not expect that to
go away.

> So I am not enthused, even though at the technical level, I agree
> that it is a good "solution" to pretend as if one of the refs were
> requested after the fetch-pack discovers the refs advertised by the
> upload-pack.

It could also lead to allowing "git clone -b $SHA".

It's possible to implement this today by manually doing the conversion
via git ls-remote. This is a significant streamlining that would
interact well with the recent change to submodules to attempt "git
fetch origin $SHA1".

> I just fear that it is a wrong approach to solve the real issue and
> instead make it worse by relieving the pressure from the project and
> hosting site to configure their repository to support their users
> properly.

Unfortunately I don't see the likes of github or bitbucket adding
allow-reachable-sha1-in-want any time soon, as it is an expensive
operation server-side.

This is a significant boost to utility of shallow submodules (a flag
for which was just recently added to clone), without requiring any
significant changes to repository configuration.

- JP

Re: git fetch origin $SHA1 doesn't work as often as it could

[Jeff King]

On Wed, May 18, 2016 at 06:40:37PM -0400, JP Sugarbroad wrote:

> > I just fear that it is a wrong approach to solve the real issue and
> > instead make it worse by relieving the pressure from the project and
> > hosting site to configure their repository to support their users
> > properly.
> 
> Unfortunately I don't see the likes of github or bitbucket adding
> allow-reachable-sha1-in-want any time soon, as it is an expensive
> operation server-side.

I don't think it's _too_ bad, at least compared to some of the nastier
things one can convince the server side of a git fetch to do. It at
least looks at just the commits; the bad cases usually involve
traversing the whole object graph.

It's also something that's easily sped up with reachability bitmaps.

But I haven't done any real analysis on the kind of load it would cause.
So no promises, but I don't think it's outside the realm of possibility
for GitHub to enable, if it's something that would be useful to a lot of
people. I can't speak for Bitbucket. :)

-Peff