Supporting --depth when cloning from a bundle

Supporting --depth when cloning from a bundle

[Konstantin Ryabitsev]

Hello, all:

It would be pretty handy if it was possible to do shallow clones from a bundle
file. For example, this would be nice for CI environments that first download
a clone.bundle file from a locally cached location, clone from it, and then
fetch any remaining objects from remote.

E.g., in my mind this outta work:

$ wget https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/clone.bundle
$ git clone --depth 1 clone.bundle linux -b master

Right now, this operation succeeds as-is without any errors, but the resulting
git repository isn't actually shallow.

Oh, it would be even awesomer if the bundle files could be used directly with
--reference, e.g.:

$ git clone --reference clone.bundle --dissociate \
  https://git.kernel.org/.../torvalds/linux.git linux

Any reason that can't work?

-K

Re: Supporting --depth when cloning from a bundle

[Junio C Hamano]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

> Hello, all:
>
> It would be pretty handy if it was possible to do shallow clones from a bundle
> file. For example, this would be nice for CI environments that first download
> a clone.bundle file from a locally cached location, clone from it, and then
> fetch any remaining objects from remote.
>
> E.g., in my mind this outta work:
>
> $ wget https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/clone.bundle
> $ git clone --depth 1 clone.bundle linux -b master
>
> Right now, this operation succeeds as-is without any errors, but the resulting
> git repository isn't actually shallow.
>
> Oh, it would be even awesomer if the bundle files could be used directly with
> --reference, e.g.:
>
> $ git clone --reference clone.bundle --dissociate \
>   https://git.kernel.org/.../torvalds/linux.git linux
>
> Any reason that can't work?

The "depth" is used to negitiate what objects to include in the
packfile that is transmit from the other side.  That would not
fundamentally work on bundles, because a bundle is pregenerated and
there is nobody on the other side for you to negotiate with to
affect what objects are included in the packfile which is contained
in the bundle file.

You _should_ be able to make the resulting full clone into a shallow
one, though.

Re: Supporting --depth when cloning from a bundle

[Konstantin Ryabitsev]

On Wed, Feb 16, 2022 at 02:37:20PM -0800, Junio C Hamano wrote:
> The "depth" is used to negitiate what objects to include in the
> packfile that is transmit from the other side.  That would not
> fundamentally work on bundles, because a bundle is pregenerated and
> there is nobody on the other side for you to negotiate with to
> affect what objects are included in the packfile which is contained
> in the bundle file.

Junio:

Sorry for the potentially stupid follow-up question. I know that git makes a
distinction when cloning locally between a direct path and a file:// URL (e.g.
it's only possible to do a shallow clone from a local repository when using
the file:// scheme). When that happens, doesn't that same negotiation happen
locally (perhaps in a simulated way)? If that so, then theoretically the same
approach could be taken with the bundle file?

(I'm not actually that interested in that feature, to be honest -- I'd much
rather see further work on git-native CDN offload of packfiles, so we stop
using kludges like bundle files for this purpose.)

-K

Re: Supporting --depth when cloning from a bundle

[Junio C Hamano]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

> Sorry for the potentially stupid follow-up question. I know that git makes a
> distinction when cloning locally between a direct path and a file:// URL (e.g.
> it's only possible to do a shallow clone from a local repository when using
> the file:// scheme).

The local optimization uses hardlinks into an existing repository
bypassing most of what makes Git Git, and there is no negotiation.

"clone --no-local $PATH" or "clone file://$PATH" runs the "server
side" inside $PATH and talks with it, which is ONLY possible because
$PATH is a real repository.  A bundle file is not a repository, and
nobody has written the "server side" to workin inside it, so...