Git development
 help / color / mirror / Atom feed
From: Aleksei Sviridkin <f@lex.la>
To: git@vger.kernel.org
Cc: Aleksei Sviridkin <f@lex.la>, Junio C Hamano <gitster@pobox.com>,
	Elijah Newren <newren@gmail.com>
Subject: [PATCH] technical/partial-clone: narrow the one-at-a-time fetch limitation
Date: Mon, 19 Oct 2026 19:03:00 +0300	[thread overview]
Message-ID: <20261019160300.15789-1-f@lex.la> (raw)

The design doc says twice that dynamic object fetching is slow because
objects are fetched one at a time, without saying which code path that
describes.

Only the object lookup fallback in odb.c works that way. The other ten
call sites of promisor_remote_get_direct() collect the missing objects
first and hand the whole array to one fetch subprocess per promisor
remote.

The bullet also still names fetch-pack, which 7ca3c0ac37
(promisor-remote: lazy-fetch objects in subprocess, 2020-08-17)
replaced with a "git fetch" subprocess.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
---
Repost of the patch from September 3. Only the log message changed; the
diff is the same and still applies to master.

I am very sorry about the unreadable message. Your guess is right: it
was machine-written and I signed it off without reading it as a reader
would.

Unrelated to this patch, since the subject of messages going unanswered
came up. I sent a security report to git-security on September 3, a
follow-up on September 16 with you in To, and one more to you directly
on September 26. The same day as the follow-up I also filed it through
GitHub's private vulnerability reporting, where it still sits in triage,
untouched. Nothing back on any of them, 46 days now. The documented
process says the list aims for an initial assessment within a few days,
and that a report judged not to need an embargo comes back as a redirect
to this list, so silence does not look like one of the expected answers.
It is not a contrived unreachable path, it is something I ran into doing
ordinary work, which is why I would like to know if anyone is looking.
Is that channel working, and if not, where should this go instead.

 Documentation/technical/partial-clone.adoc | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/Documentation/technical/partial-clone.adoc b/Documentation/technical/partial-clone.adoc
index 18718a3840..d5fd1420e5 100644
--- a/Documentation/technical/partial-clone.adoc
+++ b/Documentation/technical/partial-clone.adoc
@@ -141,8 +141,10 @@ remote and then retry the object lookup.  This allows objects to be
 For efficiency reasons, no check as to whether the missing object is
 actually a promisor object is performed.
 +
-Dynamic object fetching tends to be slow as objects are fetched one at
-a time.
+This fallback asks for a single object per request, so it is slow when
+many objects are missing.  Code that can name several missing objects at
+once passes the whole list to promisor_remote_get_direct(), which
+requests the set rather than one object at a time.
 
 - `checkout` (and any other command using `unpack-trees`) has been taught
   to bulk pre-fetch all required missing blobs in a single batch.
@@ -247,11 +249,11 @@ remote in a specific order.
 - Repack essentially treats promisor and non-promisor packfiles as 2
   distinct partitions and does not mix them.
 
-- Dynamic object fetching invokes fetch-pack once *for each item*
-  because most algorithms stumble upon a missing object and need to have
-  it resolved before continuing their work.  This may incur significant
-  overhead -- and multiple authentication requests -- if many objects are
-  needed.
+- Dynamic object fetching invokes a "git fetch" subprocess once *for
+  each item* when it goes through the object lookup fallback, because
+  the object has to be resolved before the work can continue.  This may
+  incur significant overhead -- and multiple authentication requests --
+  if many objects are needed.
 
 - Dynamic object fetching currently uses the existing pack protocol V0
   which means that each object is requested via fetch-pack.  The server
-- 
2.56.0


             reply	other threads:[~2026-10-19 16:03 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-19 16:03 Aleksei Sviridkin [this message]
  -- strict thread matches above, loose matches on Subject: below --
2026-09-03 12:55 [PATCH] technical/partial-clone: narrow the one-at-a-time fetch limitation Aleksei Sviridkin
2026-09-03 16:12 ` Elijah Newren
2026-09-03 20:00   ` Aleksei Sviridkin
2026-09-03 23:03     ` Elijah Newren
2026-10-19  2:34 ` Aleksei Sviridkin
2026-10-19 15:37   ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261019160300.15789-1-f@lex.la \
    --to=f@lex.la \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=newren@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox