From: Jeff King <peff@peff.net>
To: Michael Montalbo <mmontalbo@gmail.com>
Cc: Patrick Steinhardt <ps@pks.im>,
git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>
Subject: [PATCH 3/3] t5551: pack refs after creating many tags
Date: Sun, 28 Jun 2026 04:07:10 -0400 [thread overview]
Message-ID: <20260628080710.GC107826@coredump.intra.peff.net> (raw)
In-Reply-To: <20260628075716.GA3525066@coredump.intra.peff.net>
We have two tests that create 2,000 and 100,000 tags respectively.
After doing so, the resulting state can be a bit slow to work with when
using the "files" ref backend, as each of those refs is in its own file.
This isn't a very realistic scenario, as we'd expect most of those refs
to be packed. If they accrue over time along with objects, they'd get
packed by maintenance/gc runs. And if you have a process that creates a
ton of refs at once (like a big fast-import), the usual recommendation
is to run maintenance afterwards.
So let's follow that recommendation and pack the refs ourselves.
Unfortunately, this does not seem to produce an improvement to the
run-time of the test script! That's because after producing this state,
we perform only a few fetches of it. And packing the refs costs at least
as much as serving a ref advertisement (both have to iterate the refs,
but packing additionally must write .lock files as we pack).
My wall-clock time was slightly improved (but within the noise) with
this patch, but my user and system CPU time were slightly worse!
However, on a loaded system with I/O bottlenecks, it may be a net win.
That's somewhat of a guess, though.
It would be nice if we had a way to generate all of these refs without
writing so many individual files. But even if we taught the ref code to
write large cases directly to the packed-refs file, we'd still need to
take individual locks. The real solution is a backend like reftable,
which shaves ~30% off of the test runtime.
Signed-off-by: Jeff King <peff@peff.net>
---
I'm iffy on whether this one is worth it.
If you apply just this patch without patch 2, then the run-time does
improve quite a bit. The cost of packing is amortized by the improved
performance for all of those subsequent tests (but after patch 2, they
never even see the unpacked state).
Likewise, I suspect this would make our timeout problems go away even
without patch 1.
So the whole series _could_ be reduced to just this one patch. But
hopefully the reasoning given in the earlier patches makes sense, at
which point this one is kind of superfluous.
t/t5551-http-fetch-smart.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh
index cd851f24b8..e2e729216f 100755
--- a/t/t5551-http-fetch-smart.sh
+++ b/t/t5551-http-fetch-smart.sh
@@ -393,6 +393,7 @@ create_tags () {
tag=$(perl -e "print \"bla\" x 30") &&
sed -e "s|^:\([^ ]*\) \(.*\)$|create refs/tags/$tag-\1 \2|" <marks >input &&
git update-ref --stdin <input &&
+ git pack-refs --all &&
rm input
}
--
2.55.0.rc2.353.gf769b6597e
next prev parent reply other threads:[~2026-06-28 8:07 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-20 15:33 [RFH] Why do osx CI jobs so unreliable? Michael Montalbo
2026-06-21 21:34 ` Jeff King
2026-06-22 4:42 ` Patrick Steinhardt
2026-06-22 9:47 ` Patrick Steinhardt
2026-06-22 9:55 ` Patrick Steinhardt
2026-06-22 10:29 ` Patrick Steinhardt
2026-06-26 3:27 ` Michael Montalbo
2026-06-26 5:16 ` Jeff King
2026-06-26 10:50 ` Patrick Steinhardt
2026-06-26 13:45 ` Junio C Hamano
2026-06-26 23:26 ` Michael Montalbo
2026-06-28 7:57 ` [PATCH 0/3] fixing expensive http test timeouts Jeff King
2026-06-28 8:00 ` [PATCH 1/3] t/lib-httpd: bump apache timeout Jeff King
2026-06-28 8:03 ` [PATCH 2/3] t5551: put many-tags case into its own repo Jeff King
2026-06-28 21:44 ` Junio C Hamano
2026-06-29 0:34 ` Jeff King
2026-06-28 8:07 ` Jeff King [this message]
2026-06-28 21:25 ` [PATCH 3/3] t5551: pack refs after creating many tags Junio C Hamano
2026-06-26 23:43 ` [RFH] Why do osx CI jobs so unreliable? Jeff King
2026-06-22 5:05 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260628080710.GC107826@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=mmontalbo@gmail.com \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox