[PATCH] fetch-pack: trace packfile URI downloads

[PATCH] fetch-pack: trace packfile URI downloads

[Ted Nyman]

When a protocol v2 fetch includes packfile URIs, the client downloads
each advertised pack in a separate http-fetch process. Existing Trace2
regions cover negotiation, but not the time spent downloading these
packs or the number of advertised URIs.

Add a Trace2 region around the packfile URI download loop and record the
number of URIs. This makes the cost of downloading external packs
visible without emitting an event for each pack.

Extend the existing packfile URI test to verify the region and count.

Signed-off-by: Ted Nyman <tnyman@openai.com>
---
 fetch-pack.c           | 12 ++++++++++++
 t/t5702-protocol-v2.sh |  7 ++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/fetch-pack.c b/fetch-pack.c
index 29c41132ee..701a23f808 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1886,6 +1886,13 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
 		}
 	}
 
+	if (packfile_uris.nr) {
+		trace2_region_enter("fetch-pack", "packfile-uris",
+				    the_repository);
+		trace2_data_intmax("fetch-pack", the_repository,
+				   "packfile-uris/count", packfile_uris.nr);
+	}
+
 	for (i = 0; i < packfile_uris.nr; i++) {
 		int j;
 		struct child_process cmd = CHILD_PROCESS_INIT;
@@ -1936,6 +1943,11 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
 						 repo_get_object_directory(the_repository),
 						 packname));
 	}
+
+	if (packfile_uris.nr)
+		trace2_region_leave("fetch-pack", "packfile-uris",
+				    the_repository);
+
 	string_list_clear(&packfile_uris, 0);
 	strvec_clear(&index_pack_args);
 
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 74a2b7730b..537deff7b3 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -1223,7 +1223,7 @@ configure_exclusion () {
 
 test_expect_success 'part of packfile response provided as URI' '
 	P="$HTTPD_DOCUMENT_ROOT_PATH/http_parent" &&
-	rm -rf "$P" http_child log &&
+	rm -rf "$P" http_child log trace2 &&
 
 	git init "$P" &&
 	git -C "$P" config "uploadpack.allowsidebandall" "true" &&
@@ -1238,10 +1238,15 @@ test_expect_success 'part of packfile response provided as URI' '
 	configure_exclusion "$P" other-blob >h2 &&
 
 	GIT_TRACE=1 GIT_TRACE_PACKET="$(pwd)/log" GIT_TEST_SIDEBAND_ALL=1 \
+	GIT_TRACE2_EVENT="$(pwd)/trace2" \
 	git -c protocol.version=2 \
 		-c fetch.uriprotocols=http,https \
 		clone "$HTTPD_URL/smart/http_parent" http_child &&
 
+	test_grep \"event\":\"region_enter\".*\"label\":\"packfile-uris\" trace2 &&
+	test_grep \"key\":\"packfile-uris/count\",\"value\":\"2\" trace2 &&
+	test_grep \"event\":\"region_leave\".*\"label\":\"packfile-uris\" trace2 &&
+
 	# Ensure that my-blob and other-blob are in separate packfiles.
 	for idx in http_child/.git/objects/pack/*.idx
 	do

Re: [PATCH] fetch-pack: trace packfile URI downloads

[Patrick Steinhardt]

On Sun, Jul 26, 2026 at 01:33:11AM -0700, Ted Nyman wrote:
> When a protocol v2 fetch includes packfile URIs, the client downloads
> each advertised pack in a separate http-fetch process. Existing Trace2
> regions cover negotiation, but not the time spent downloading these
> packs or the number of advertised URIs.
> 
> Add a Trace2 region around the packfile URI download loop and record the
> number of URIs. This makes the cost of downloading external packs
> visible without emitting an event for each pack.

Right, by having a region we can verify how long downloading the
packfiles took, and by tracking the number of packfiles we know how many
we fetched. What we don't know is how long fetching each of the
individual packs took, but I think that omission makes sense. After all,
we can reasonably expect all packs to be served by the same infra, and
as such they should usually have similar download speeds.

> diff --git a/fetch-pack.c b/fetch-pack.c
> index 29c41132ee..701a23f808 100644
> --- a/fetch-pack.c
> +++ b/fetch-pack.c
> @@ -1886,6 +1886,13 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
>  		}
>  	}
>  
> +	if (packfile_uris.nr) {
> +		trace2_region_enter("fetch-pack", "packfile-uris",
> +				    the_repository);
> +		trace2_data_intmax("fetch-pack", the_repository,
> +				   "packfile-uris/count", packfile_uris.nr);

We don't have a repository available in our context, so we have to use
`the_repository`.

> +	}
> +
>  	for (i = 0; i < packfile_uris.nr; i++) {
>  		int j;
>  		struct child_process cmd = CHILD_PROCESS_INIT;

Sensible. We don't need to track fetching if we don't have any packfiles
at all.

> @@ -1936,6 +1943,11 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
>  						 repo_get_object_directory(the_repository),
>  						 packname));
>  	}
> +
> +	if (packfile_uris.nr)
> +		trace2_region_leave("fetch-pack", "packfile-uris",
> +				    the_repository);
> +
>  	string_list_clear(&packfile_uris, 0);
>  	strvec_clear(&index_pack_args);
>  

And likewise, we don't have to leave the region, either in that case.

> diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
> index 74a2b7730b..537deff7b3 100755
> --- a/t/t5702-protocol-v2.sh
> +++ b/t/t5702-protocol-v2.sh
> @@ -1223,7 +1223,7 @@ configure_exclusion () {
>  
>  test_expect_success 'part of packfile response provided as URI' '
>  	P="$HTTPD_DOCUMENT_ROOT_PATH/http_parent" &&
> -	rm -rf "$P" http_child log &&
> +	rm -rf "$P" http_child log trace2 &&
>  
>  	git init "$P" &&
>  	git -C "$P" config "uploadpack.allowsidebandall" "true" &&
> @@ -1238,10 +1238,15 @@ test_expect_success 'part of packfile response provided as URI' '
>  	configure_exclusion "$P" other-blob >h2 &&
>  
>  	GIT_TRACE=1 GIT_TRACE_PACKET="$(pwd)/log" GIT_TEST_SIDEBAND_ALL=1 \
> +	GIT_TRACE2_EVENT="$(pwd)/trace2" \
>  	git -c protocol.version=2 \
>  		-c fetch.uriprotocols=http,https \
>  		clone "$HTTPD_URL/smart/http_parent" http_child &&
>  
> +	test_grep \"event\":\"region_enter\".*\"label\":\"packfile-uris\" trace2 &&
> +	test_grep \"key\":\"packfile-uris/count\",\"value\":\"2\" trace2 &&
> +	test_grep \"event\":\"region_leave\".*\"label\":\"packfile-uris\" trace2 &&
> +
>  	# Ensure that my-blob and other-blob are in separate packfiles.
>  	for idx in http_child/.git/objects/pack/*.idx
>  	do

It does feel a tiny bit off to piggy-back on an existing test that has
nothing to do with tracing except that it requires the traces to... I
dunno, what does the existing test even do with the written logfile?
Doesn't seem like it's using it at all.

Anyway, having this in a separate test would've been nice, but that
doesn't warrant a reroll in my eyes. So overall, this patch looks good
to me, thanks!

Patrick