Clones with fetch.bundleURI slower than standard, full clone?

Clones with fetch.bundleURI slower than standard, full clone?

[Knop, Ryszard]

Hey all,

I'm working on a Linux kernel-related CI system where we need to
perform full clones of the kernel repo in most jobs. Because of some
jobs in the pipeline, it usually cannot be a shallow clone :( Since the
kernel repo is large and slow to clone, and I don't want to put undue
stress on the remote host, I used git bundles, where CI clones the repo
over a weekend, packages that as a bundle, then in jobs it gets used
like this (weird, but works for <REF> being a branch, tag or a specific
commit hash):

git init
git remote add origin <REPO-URL>
git config set fetch.bundleURI <BUNDLE-URL>
git fetch origin <REF>
git reset --hard FETCH_HEAD

Cloning a repo this way takes 4-5mins. Unpacking a bundle appears to be
super slow. Not even faster than just running a full, normal clone from
the remote server, actually (~3-4mins for a single branch).

On one of the build VMs, with Git 2.53 (stock Ubuntu 26.04), GIT_TRACE
suggests most of the time is spent in some variation of `/usr/lib/git-
core/git index-pack --stdin -v --fix-thin '--keep=fetch-pack 39430 on
build-server' --check-self-contained-and-connected`, and indeed that
process burns 100% of its single thread for most of that time.

Is it expected that doing it this way is so slow? The alternative is to
just package and work with the whole bare repo, but bundles appear to
be an elegant way of dealing with exactly this scenario.

Thanks, Ryszard

Re: Clones with fetch.bundleURI slower than standard, full clone?

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 3555 bytes --]

On 2026-07-28 at 23:42:03, Knop, Ryszard wrote:
> Hey all,
> 
> I'm working on a Linux kernel-related CI system where we need to
> perform full clones of the kernel repo in most jobs. Because of some
> jobs in the pipeline, it usually cannot be a shallow clone :( Since the
> kernel repo is large and slow to clone, and I don't want to put undue
> stress on the remote host, I used git bundles, where CI clones the repo
> over a weekend, packages that as a bundle, then in jobs it gets used
> like this (weird, but works for <REF> being a branch, tag or a specific
> commit hash):
> 
> git init
> git remote add origin <REPO-URL>
> git config set fetch.bundleURI <BUNDLE-URL>
> git fetch origin <REF>
> git reset --hard FETCH_HEAD
> 
> Cloning a repo this way takes 4-5mins. Unpacking a bundle appears to be
> super slow. Not even faster than just running a full, normal clone from
> the remote server, actually (~3-4mins for a single branch).

You're comparing apples to oranges here.  A clone of a single branch
includes only that line of history and only those objects, but when you
use a bundle with multiple refs, Git has to handle all of the objects in
the bundle's entire pack, not just the ref you've specified.  In order
to compare adequately, you'd have to compare a bundle containing only
that one ref with the single-branch clone or a regular clone of the full
repository with your full bundles.

> On one of the build VMs, with Git 2.53 (stock Ubuntu 26.04), GIT_TRACE
> suggests most of the time is spent in some variation of `/usr/lib/git-
> core/git index-pack --stdin -v --fix-thin '--keep=fetch-pack 39430 on
> build-server' --check-self-contained-and-connected`, and indeed that
> process burns 100% of its single thread for most of that time.

As far as I can tell, the unbundling code just calls index-pack, so it
should honour pack.threads.  Setting that value to 0 causes this code to
be executed:

		/*
		 * Experiments show that going above 20 threads doesn't help,
		 * no matter how many cores you have. Below that, we tend to
		 * max at half the number of online_cpus(), presumably because
		 * half of those are hyperthreads rather than full cores. We'll
		 * never reduce the level below "3", though, to match a
		 * historical value that nobody complained about.
		 */
		if (nr_threads < 4)
			; /* too few cores to consider capping */
		else if (nr_threads < 6)
			nr_threads = 3; /* historic cap */
		else if (nr_threads < 40)
			nr_threads /= 2;
		else
			nr_threads = 20; /* hard cap */

So I would expect this to not be single threaded unless Git was compiled
without pthreads, run on a machine with few cores, or configured to use
only a single thread.  If you can get threading to work here, I expect
it will perform better, although I don't have any experience with bundle
URIs so I can't really say for certain.

> Is it expected that doing it this way is so slow? The alternative is to
> just package and work with the whole bare repo, but bundles appear to
> be an elegant way of dealing with exactly this scenario.

To be clear, it is insecure to do anything with an untrusted repo except
clone or fetch from it, so you will almost certainly not want to
distribute bare repos, since that will encourage people to use them
as-is (which is insecure).  Even if these are internal users who can
trust you, it encourages an anti-pattern which has security problems in
the general case.
-- 
brian m. carlson (they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 325 bytes --]

Re: Clones with fetch.bundleURI slower than standard, full clone?

[Knop, Ryszard]

On Wed, 2026-07-29 at 00:14 +0000, brian m. carlson wrote:
> On 2026-07-28 at 23:42:03, Knop, Ryszard wrote:
> > Hey all,
> > 
> > I'm working on a Linux kernel-related CI system where we need to
> > perform full clones of the kernel repo in most jobs. Because of some
> > jobs in the pipeline, it usually cannot be a shallow clone :( Since the
> > kernel repo is large and slow to clone, and I don't want to put undue
> > stress on the remote host, I used git bundles, where CI clones the repo
> > over a weekend, packages that as a bundle, then in jobs it gets used
> > like this (weird, but works for <REF> being a branch, tag or a specific
> > commit hash):
> > 
> > git init
> > git remote add origin <REPO-URL>
> > git config set fetch.bundleURI <BUNDLE-URL>
> > git fetch origin <REF>
> > git reset --hard FETCH_HEAD
> > 
> > Cloning a repo this way takes 4-5mins. Unpacking a bundle appears to be
> > super slow. Not even faster than just running a full, normal clone from
> > the remote server, actually (~3-4mins for a single branch).
> 
> You're comparing apples to oranges here.  A clone of a single branch
> includes only that line of history and only those objects, but when you
> use a bundle with multiple refs, Git has to handle all of the objects in
> the bundle's entire pack, not just the ref you've specified.  In order
> to compare adequately, you'd have to compare a bundle containing only
> that one ref with the single-branch clone or a regular clone of the full
> repository with your full bundles.

I thought that due to changes mentioned in this post, Git 2.50 and
newer should just take in all the bundle changes and there should not
be much of a difference during index packs:

https://blog.gitbutler.com/going-down-the-rabbit-hole-of-gits-new-bundle-uri

> 
> > On one of the build VMs, with Git 2.53 (stock Ubuntu 26.04), GIT_TRACE
> > suggests most of the time is spent in some variation of `/usr/lib/git-
> > core/git index-pack --stdin -v --fix-thin '--keep=fetch-pack 39430 on
> > build-server' --check-self-contained-and-connected`, and indeed that
> > process burns 100% of its single thread for most of that time.
> 
> As far as I can tell, the unbundling code just calls index-pack, so it
> should honour pack.threads.  Setting that value to 0 causes this code to
> be executed:
> 
> 		/*
> 		 * Experiments show that going above 20 threads doesn't help,
> 		 * no matter how many cores you have. Below that, we tend to
> 		 * max at half the number of online_cpus(), presumably because
> 		 * half of those are hyperthreads rather than full cores. We'll
> 		 * never reduce the level below "3", though, to match a
> 		 * historical value that nobody complained about.
> 		 */
> 		if (nr_threads < 4)
> 			; /* too few cores to consider capping */
> 		else if (nr_threads < 6)
> 			nr_threads = 3; /* historic cap */
> 		else if (nr_threads < 40)
> 			nr_threads /= 2;
> 		else
> 			nr_threads = 20; /* hard cap */
> 
> So I would expect this to not be single threaded unless Git was compiled
> without pthreads, run on a machine with few cores, or configured to use
> only a single thread.  If you can get threading to work here, I expect
> it will perform better, although I don't have any experience with bundle
> URIs so I can't really say for certain.

Ohhh, this is way better, that halved the checkout time. Thank you! It
seems like the process still has some single-threaded sections during
packing, but it's a problem for another day.

> > Is it expected that doing it this way is so slow? The alternative is to
> > just package and work with the whole bare repo, but bundles appear to
> > be an elegant way of dealing with exactly this scenario.
> 
> To be clear, it is insecure to do anything with an untrusted repo except
> clone or fetch from it, so you will almost certainly not want to
> distribute bare repos, since that will encourage people to use them
> as-is (which is insecure).  Even if these are internal users who can
> trust you, it encourages an anti-pattern which has security problems in
> the general case.

I know it's not ideal, but our reference repos are trusted and the
cache files are used in CI only, so it would probably be fine(tm). With
that config change above I don't have to fall back to this though.

Thanks again, Ryszard