From: Christian Couder <christian.couder@gmail.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>,
Patrick Steinhardt <ps@pks.im>, Matt Cote <mcote@nvidia.com>,
Christian Couder <christian.couder@gmail.com>
Subject: [PATCH v2 7/8] fetch-pack: ensure fetch sends a promisor-remote reply
Date: Fri, 20 Nov 2026 17:48:20 +0100 [thread overview]
Message-ID: <20261120164812.707946-17-christian.couder@gmail.com> (raw)
In-Reply-To: <20261120164812.707946-10-christian.couder@gmail.com>
In protocol v2, when the client accepts promisor remotes, the
`promisor-remote=...` reply is only sent with the `ls-refs` and
`bundle-uri` protocol commands, by get_remote_refs() and
get_remote_bundle_uri() in "connect.c".
The request for the `fetch` protocol command is built by
send_fetch_request() in "fetch-pack.c" which doesn't send that reply.
do_fetch_pack_v2() calls promisor_remote_reply(), but only to mark the
accepted remotes, as e15a6b2f8b (promisor-remote: change
promisor_remote_reply()'s signature, 2026-02-16) assumed that
fetch-pack would not need to send a reply.
With file:// and ssh:// URLs on a Git server, one upload-pack process
handles both `ls-refs` and `fetch`. The acceptance recorded during
`ls-refs` by promisor_remote_receive() in "serve.c" is still there at
`fetch` time, so the clone succeeds. But it fails when each request is
handled on its own, for example with HTTP where different upload-pack
processes are used.
It also fails when only object IDs are fetched, as no `ls-refs`
command is then sent. In these cases, the server doesn't know that the
client accepted a promisor remote, so it doesn't allow promisor
objects to be missing from the pack it sends, and it fails with "could
not fetch <oid> from promisor remote".
Let's fix that by building the list of capabilities for the `fetch`
requests once in do_fetch_pack_v2(), using add_server_options() and
add_promisor_remote_reply(), and by passing that list to
send_fetch_request(). This way the `promisor-remote=...` reply is
always properly sent with the `fetch` requests, and the accepted
remotes are still marked at the same point as before.
The `fetch` requests sent by negotiate_using_fetch() don't ask for a
pack, so they don't need that reply.
Let's also add tests in t5710 to check that the reply is sent with the
`fetch` request, and that fetching only an object ID works. For HTTP,
let's add a new t5711 test script, copying the setup from t5710.
Reported-by: Matt Cote <mcote@nvidia.com>
Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
fetch-pack.c | 18 +--
t/meson.build | 1 +
t/t5710-promisor-remote-capability.sh | 40 +++++++
t/t5711-promisor-remote-capability-http.sh | 126 +++++++++++++++++++++
4 files changed, 176 insertions(+), 9 deletions(-)
create mode 100755 t/t5711-promisor-remote-capability-http.sh
diff --git a/fetch-pack.c b/fetch-pack.c
index 176d625936..3d6d3e33a2 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1380,16 +1380,15 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out,
const struct ref *wants, struct oidset *common,
int *haves_to_send, int *in_vain,
int sideband_all, int seen_ack,
- struct oidset *negotiation_include_oids)
+ struct oidset *negotiation_include_oids,
+ const struct strvec *capabilities)
{
int haves_added;
int done_sent = 0;
struct strbuf req_buf = STRBUF_INIT;
- struct strvec capabilities = STRVEC_INIT;
- add_server_options(&capabilities, args->server_options);
write_command_and_capabilities(&req_buf, "fetch",
- &capabilities, NULL);
+ capabilities, NULL);
if (args->use_thin_pack)
packet_buf_write(&req_buf, "thin-pack");
@@ -1454,7 +1453,6 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out,
if (write_in_full(fd_out, req_buf.buf, req_buf.len) < 0)
die_errno(_("unable to write request to remote"));
- strvec_clear(&capabilities);
strbuf_release(&req_buf);
return done_sent;
}
@@ -1698,12 +1696,12 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
struct string_list packfile_uris = STRING_LIST_INIT_DUP;
int i;
struct strvec index_pack_args = STRVEC_INIT;
- const char *promisor_remote_config;
+ struct strvec capabilities = STRVEC_INIT;
fsck_options_init(&fsck_options, the_repository, FSCK_OPTIONS_MISSING_GITMODULES);
- if (server_feature_v2("promisor-remote", &promisor_remote_config))
- promisor_remote_reply(promisor_remote_config, NULL);
+ add_server_options(&capabilities, args->server_options);
+ add_promisor_remote_reply(&capabilities);
if (args->filter_options.choice == LOFC_AUTO) {
struct strbuf errbuf = STRBUF_INIT;
@@ -1781,7 +1779,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
&haves_to_send, &in_vain,
reader.use_sideband,
seen_ack,
- &negotiation_include_oids)) {
+ &negotiation_include_oids,
+ &capabilities)) {
trace2_region_leave_printf("negotiation_v2", "round",
the_repository, "%d",
negotiation_round);
@@ -1924,6 +1923,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
string_list_clear(&packfile_uris, 0);
strvec_clear(&index_pack_args);
+ strvec_clear(&capabilities);
if (fsck_finish(&fsck_options))
die("fsck failed");
diff --git a/t/meson.build b/t/meson.build
index 1cf1d95f70..84644703f1 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -766,6 +766,7 @@ integration_tests = [
't5704-protocol-violations.sh',
't5705-session-id-in-capabilities.sh',
't5710-promisor-remote-capability.sh',
+ 't5711-promisor-remote-capability-http.sh',
't5730-protocol-v2-bundle-uri-file.sh',
't5731-protocol-v2-bundle-uri-git.sh',
't5732-protocol-v2-bundle-uri-http.sh',
diff --git a/t/t5710-promisor-remote-capability.sh b/t/t5710-promisor-remote-capability.sh
index 463e9e00b0..e5d6f9d1c6 100755
--- a/t/t5710-promisor-remote-capability.sh
+++ b/t/t5710-promisor-remote-capability.sh
@@ -999,6 +999,46 @@ test_expect_success "clone and fetch with --filter=auto" '
check_missing_objects server 1 "$oid"
'
+test_expect_success "clone sends the promisor-remote reply with fetch" '
+ git -C server config promisor.advertise true &&
+ test_when_finished "rm -rf client trace" &&
+
+ GIT_TRACE_PACKET="$(pwd)/trace" git clone \
+ -c remote.lop.promisor=true \
+ -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \
+ -c promisor.acceptfromserver=All \
+ --no-local --filter="blob:limit=5k" server client &&
+
+ # Check that the reply is part of the fetch request
+ sed -n "/clone> command=fetch/,/clone> 0001/p" trace >request &&
+ test_grep "clone> promisor-remote=lop" request &&
+
+ # Check that the largest object is still missing on the server
+ check_missing_objects server 1 "$oid"
+'
+
+test_expect_success "fetch by object ID sends the promisor-remote reply" '
+ git -C server config promisor.advertise true &&
+ test_when_finished "rm -rf client trace" &&
+
+ git init client &&
+ git -C client remote add origin "$TRASH_DIRECTORY_URL/server" &&
+ git -C client config remote.lop.promisor true &&
+ git -C client config remote.lop.url "$TRASH_DIRECTORY_URL/lop" &&
+ git -C client config promisor.acceptfromserver All &&
+
+ # Fetching only an object ID skips the "ls-refs" command, so the
+ # reply can only be sent with "fetch"
+ head=$(git -C server rev-parse HEAD) &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch --no-tags \
+ --filter="blob:limit=5k" origin "$head" &&
+ test_grep ! "fetch> command=ls-refs" trace &&
+ test_grep "fetch> promisor-remote=lop" trace &&
+
+ # Check that the largest object is still missing on the server
+ check_missing_objects server 1 "$oid"
+'
+
test_expect_success "clone with promisor.advertise set to 'true' but don't delete the client" '
git -C server config promisor.advertise true &&
diff --git a/t/t5711-promisor-remote-capability-http.sh b/t/t5711-promisor-remote-capability-http.sh
new file mode 100755
index 0000000000..c08cca3e51
--- /dev/null
+++ b/t/t5711-promisor-remote-capability-http.sh
@@ -0,0 +1,126 @@
+#!/bin/sh
+
+test_description='handling of promisor remote advertisement over HTTP'
+
+. ./test-lib.sh
+
+if ! test_have_prereq PERL_TEST_HELPERS
+then
+ skip_all='skipping promisor remote capabilities tests; Perl not available'
+ test_done
+fi
+
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+GIT_TEST_MULTI_PACK_INDEX=0
+GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL=0
+
+# The setup and helpers below are copied from t5710, except that the
+# server repository is in the HTTP document root.
+
+SERVER="$HTTPD_DOCUMENT_ROOT_PATH/server.git"
+
+check_missing_objects () {
+ git -C "$1" rev-list --objects --all --missing=print > all.txt &&
+ perl -ne 'print if s/^[?]//' all.txt >missing.txt &&
+ test_line_count = "$2" missing.txt &&
+ test "$3" = "$(cat missing.txt)"
+}
+
+initialize_server () {
+ # Repack everything first
+ git -C "$SERVER" -c repack.writebitmaps=false repack -a -d &&
+
+ # Repack without the largest object and create a promisor pack on server
+ git -C "$SERVER" -c repack.writebitmaps=false repack -a -d \
+ --filter=blob:limit=5k --filter-to="$(pwd)/pack" &&
+ promisor_file=$(ls "$SERVER"/objects/pack/*.pack | sed "s/\.pack/.promisor/") &&
+ >"$promisor_file" &&
+
+ # Check objects missing on the server
+ check_missing_objects "$SERVER" 1 "$1"
+}
+
+copy_to_lop () {
+ oid_path="$(test_oid_to_path $1)" &&
+ path="$SERVER/objects/$oid_path" &&
+ path2="lop/objects/$oid_path" &&
+ mkdir -p $(dirname "$path2") &&
+ cp "$path" "$path2"
+}
+
+# On Windows, `pwd` returns a path like 'D:/foo/bar'. Prepend '/' to turn
+# it into '/D:/foo/bar', which is what git expects in file:// URLs on Windows.
+# On Unix, the path already starts with '/', so this is a no-op.
+pwd_path=$(pwd)
+case "$pwd_path" in
+[a-zA-Z]:*) pwd_path="/$pwd_path" ;;
+esac
+
+TRASH_DIRECTORY_URL="file://$pwd_path"
+
+test_expect_success 'setup: create "template" repository' '
+ git init template &&
+ test_commit -C template 1 &&
+ test_commit -C template 2 &&
+ test_commit -C template 3 &&
+ test-tool genrandom foo 10k >template/foo &&
+ git -C template add foo &&
+ git -C template commit -m foo
+'
+
+# A bare repo will act as a server repo with unpacked objects.
+test_expect_success 'setup: create bare "server" repository' '
+ git clone --bare --no-local template "$SERVER" &&
+ mv "$SERVER"/objects/pack/pack-* . &&
+ packfile=$(ls pack-*.pack) &&
+ git -C "$SERVER" unpack-objects --strict <"$packfile"
+'
+
+test_expect_success "setup for testing promisor remote advertisement" '
+ # Create another bare repo called "lop" (for Large Object Promisor)
+ git init --bare lop &&
+
+ # Copy the largest object from server to lop
+ obj="HEAD:foo" &&
+ oid="$(git -C "$SERVER" rev-parse $obj)" &&
+ copy_to_lop "$oid" &&
+
+ initialize_server "$oid" &&
+
+ # Configure lop as promisor remote for server
+ git -C "$SERVER" remote add lop "$TRASH_DIRECTORY_URL/lop" &&
+ git -C "$SERVER" config remote.lop.promisor true &&
+
+ git -C lop config uploadpack.allowFilter true &&
+ git -C lop config uploadpack.allowAnySHA1InWant true &&
+ git -C "$SERVER" config uploadpack.allowFilter true &&
+ git -C "$SERVER" config uploadpack.allowAnySHA1InWant true &&
+ git -C "$SERVER" config promisor.advertise true
+'
+
+# Over HTTP, each request is handled by a separate upload-pack process,
+# so the server knows that the client accepted "lop" only if the reply
+# is sent with the "fetch" request itself.
+test_expect_success "clone over HTTP with an accepted promisor remote" '
+ test_when_finished "rm -rf client trace" &&
+
+ GIT_TRACE_PACKET="$(pwd)/trace" git -c protocol.version=2 clone \
+ -c remote.lop.promisor=true \
+ -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \
+ -c promisor.acceptfromserver=All \
+ --filter="blob:limit=5k" "$HTTPD_URL/smart/server.git" client &&
+
+ # Check that the reply is part of the fetch request
+ sed -n "/clone> command=fetch/,/clone> 0001/p" trace >request &&
+ test_grep "clone> promisor-remote=lop" request &&
+
+ # Check that the largest object was lazily fetched from lop
+ test_cmp template/foo client/foo &&
+
+ # Check that the largest object is still missing on the server
+ check_missing_objects "$SERVER" 1 "$oid"
+'
+
+test_done
--
2.56.0
next prev parent reply other threads:[~2026-11-20 16:48 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-12 19:58 [PATCH 0/3] Fix promisor-remote reply missing from fetch Christian Couder
2026-11-12 19:59 ` [PATCH 1/3] connect: let write_command_and_capabilities() send a promisor-remote reply Christian Couder
2026-11-12 22:18 ` Junio C Hamano
2026-11-12 22:34 ` Junio C Hamano
2026-11-13 13:10 ` Christian Couder
2026-11-13 19:11 ` Junio C Hamano
2026-11-20 16:57 ` Christian Couder
2026-11-12 19:59 ` [PATCH 2/3] fetch-pack: ensure fetch sends " Christian Couder
2026-11-12 19:59 ` [PATCH 3/3] doc: gitprotocol-v2: send the promisor-remote reply with fetch Christian Couder
2026-11-20 16:48 ` [PATCH v2 0/8] Fix promisor-remote reply missing from fetch Christian Couder
2026-11-20 16:48 ` [PATCH v2 1/8] connect: pass capability lines to write_command_and_capabilities() Christian Couder
2026-11-20 16:48 ` [PATCH v2 2/8] connect: send session-id with ls-refs and bundle-uri Christian Couder
2026-11-20 16:48 ` [PATCH v2 3/8] connect: pass a reader to write_command_and_capabilities() Christian Couder
2026-11-20 16:48 ` [PATCH v2 4/8] connect: buffer the ls-refs and bundle-uri requests Christian Couder
2026-11-20 16:48 ` [PATCH v2 5/8] connect: use write_command_and_capabilities() in get_remote_refs() Christian Couder
2026-11-20 16:48 ` [PATCH v2 6/8] connect: use write_command_and_capabilities() in get_remote_bundle_uri() Christian Couder
2026-11-20 16:48 ` Christian Couder [this message]
2026-11-20 16:48 ` [PATCH v2 8/8] doc: gitprotocol-v2: send the promisor-remote reply with fetch Christian Couder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120164812.707946-17-christian.couder@gmail.com \
--to=christian.couder@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=mcote@nvidia.com \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox