From: Jon Simons <jon@jonsimons.org>
To: git@vger.kernel.org
Cc: Jon Simons <jon@jonsimons.org>
Subject: [PATCH 00/15] push: speed up client-side refspec matching
Date: Fri, 9 Oct 2026 15:29:38 -0400 [thread overview]
Message-ID: <20261009192953.81794-1-jon@jonsimons.org> (raw)
This series speeds up client-side refspec matching in push, focused on
pushing lots of branch deletes to a server that advertises lots of refs
(100k in the benchmark below).
Before this series, a 'git push --dry-run' of 100 delete refspecs to a
server with 100k refs takes ~2.5s from an empty client and ~6.6s from a
mirror; using one --force-with-lease per refspec takes ~4.8s and ~8.9s
respectively. After, the same pushes take ~0.15s to ~0.19s.
refname_match() is updated to reuse the existing match_parse_rule(),
and a few client-side linear ref traversals are converted to
strmap lookups in remote.c.
Guide to changes:
- Commits are ordered to introduce failing tests for behavioral
changes and bugfixes, before the code change that fixes them.
- Prep refname_match() call sites for behavioral change of mkpath()
removal, and introduce benchmark:
(1) remote: validate --force-with-lease <refname> argument
(2) t5516: demonstrate push with "./"-prefixed source
(3) t5510: document fetch with "./"-prefixed branch.<name>.merge
(4) t/perf: add explicit delete refspec matching test
- The first speedup, best viewed with `--color-moved`, is where
refname_match() changes behavior: mkpath() stripped a leading "./"
from the name being matched, so "./refs/heads/foo" used to match
"refs/heads/foo". It no longer does.
(5) refs: stop using mkpath() in refname_match()
- The next speedup establishes the conversion pattern of using
expand_ref_prefix() with strmap lookups (the same pattern is used
in two more spots that follow):
(6) remote: use strmap for check_push_refs()
- Fix a long-standing buglet where pushing two refspecs with the same
destination to an empty remote is not rejected client-side, as it is
when the remote has any ref.
t5408 and t5410 now cover server-side duplicate rejection and
send-pack's handling of a ref reported twice, without relying
on the client missing the duplicates.
Fixing this separately keeps the behavior change out of the strmap
conversion that immediately follows, which would otherwise fix it
as a side-effect:
(7) t5516: test pushing two refspecs creating the same new branch
(8) t5408, t5410: test duplicate updates without relying on the client
(9) t5408: check refspec order with distinct destinations
(10) t5408: expect client-side error for duplicate destinations
(11) remote: reject duplicate destinations on an empty remote
- Convert match_explicit_refs():
(12) remote: use strmap for match_explicit_refs()
- Measure and convert --force-with-lease paths:
(13) t/perf: measure --force-with-lease in p5516
(14) remote: restructure apply_push_cas() loops
(15) remote: use strmap for apply_push_cas()
Numbers:
A new benchmark demonstrates each performance change. These numbers
were gathered running the final form of p5516 against the earlier
trees on my machine:
BASELINE origin/master
REFNAME_MATCH (5) refs: stop using mkpath() in refname_match()
MAP_CHECK_PUSH_REFS (6) remote: use strmap for check_push_refs()
MAP_EXPLICIT_REFS (12) remote: use strmap for match_explicit_refs()
MAP_APPLY_PUSH_CAS (15) remote: use strmap for apply_push_cas()
Test BASELINE REFNAME_MATCH MAP_CHECK_PUSH_REFS MAP_EXPLICIT_REFS MAP_APPLY_PUSH_CAS
--------------------------------------------------------------------------------------------------------------------------------------------------
5516.3: empty:refspecs:1 0.16(0.09+0.11) 0.14(0.07+0.11) -12.5% 0.14(0.07+0.11) -12.5% 0.14(0.07+0.11) -12.5% 0.14(0.08+0.11) -12.5%
5516.5: empty:refspecs:10 0.37(0.30+0.11) 0.17(0.10+0.11) -54.1% 0.17(0.11+0.11) -54.1% 0.13(0.07+0.11) -64.9% 0.14(0.07+0.11) -62.2%
5516.7: empty:refspecs:100 2.48(2.41+0.11) 0.49(0.43+0.11) -80.2% 0.47(0.40+0.11) -81.0% 0.15(0.08+0.11) -94.0% 0.15(0.09+0.11) -94.0%
5516.9: mirror:refspecs:1 0.22(0.15+0.11) 0.16(0.09+0.11) -27.3% 0.16(0.09+0.11) -27.3% 0.16(0.09+0.11) -27.3% 0.17(0.10+0.11) -22.7%
5516.11: mirror:refspecs:10 0.81(0.74+0.11) 0.26(0.19+0.11) -67.9% 0.23(0.16+0.11) -71.6% 0.16(0.09+0.11) -80.2% 0.17(0.10+0.12) -79.0%
5516.13: mirror:refspecs:100 6.64(6.57+0.12) 1.21(1.13+0.12) -81.8% 0.86(0.79+0.12) -87.0% 0.18(0.11+0.11) -97.3% 0.18(0.11+0.11) -97.3%
5516.16: empty:lease:1 0.17(0.11+0.11) 0.14(0.07+0.11) -17.6% 0.14(0.07+0.11) -17.6% 0.14(0.07+0.11) -17.6% 0.15(0.08+0.11) -11.8%
5516.19: empty:lease:10 0.71(0.65+0.11) 0.20(0.14+0.11) -71.8% 0.20(0.13+0.11) -71.8% 0.17(0.10+0.11) -76.1% 0.15(0.08+0.11) -78.9%
5516.22: empty:lease:100 4.79(4.71+0.12) 0.78(0.71+0.12) -83.7% 0.80(0.73+0.12) -83.3% 0.45(0.38+0.11) -90.6% 0.16(0.09+0.11) -96.7%
5516.25: mirror:lease:1 0.24(0.17+0.11) 0.17(0.10+0.11) -29.2% 0.17(0.10+0.12) -29.2% 0.17(0.10+0.12) -29.2% 0.18(0.11+0.12) -25.0%
5516.28: mirror:lease:10 1.04(0.97+0.11) 0.29(0.22+0.11) -72.1% 0.26(0.19+0.12) -75.0% 0.20(0.13+0.11) -80.8% 0.18(0.11+0.12) -82.7%
5516.31: mirror:lease:100 8.93(8.85+0.12) 1.47(1.40+0.12) -83.5% 1.11(1.04+0.12) -87.6% 0.49(0.42+0.12) -94.5% 0.19(0.12+0.12) -97.9%
Jon Simons (15):
remote: validate --force-with-lease <refname> argument
t5516: demonstrate push with "./"-prefixed source
t5510: document fetch with "./"-prefixed branch.<name>.merge
t/perf: add explicit delete refspec matching test
refs: stop using mkpath() in refname_match()
remote: use strmap for check_push_refs()
t5516: test pushing two refspecs creating the same new branch
t5408, t5410: test duplicate updates without relying on the client
t5408: check refspec order with distinct destinations
t5408: expect client-side error for duplicate destinations
remote: reject duplicate destinations on an empty remote
remote: use strmap for match_explicit_refs()
t/perf: measure --force-with-lease in p5516
remote: restructure apply_push_cas() loops
remote: use strmap for apply_push_cas()
refs.c | 88 +++++-----
remote.c | 257 +++++++++++++++++++---------
t/perf/p5516-push-delete-refspec.sh | 67 ++++++++
t/t5408-send-pack-stdin.sh | 56 +++++-
t/t5410-receive-pack.sh | 35 ++++
t/t5510-fetch.sh | 44 +++++
t/t5516-fetch-push.sh | 42 +++++
t/t5533-push-cas.sh | 9 +
8 files changed, 465 insertions(+), 133 deletions(-)
create mode 100755 t/perf/p5516-push-delete-refspec.sh
--
2.55.0
next reply other threads:[~2026-10-09 19:37 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-09 19:29 Jon Simons [this message]
2026-10-09 19:29 ` [PATCH 01/15] remote: validate --force-with-lease <refname> argument Jon Simons
2026-10-09 19:29 ` [PATCH 02/15] t5516: demonstrate push with "./"-prefixed source Jon Simons
2026-10-09 19:29 ` [PATCH 03/15] t5510: document fetch with "./"-prefixed branch.<name>.merge Jon Simons
2026-10-09 19:29 ` [PATCH 04/15] t/perf: add explicit delete refspec matching test Jon Simons
2026-10-09 19:29 ` [PATCH 05/15] refs: stop using mkpath() in refname_match() Jon Simons
2026-10-09 19:29 ` [PATCH 06/15] remote: use strmap for check_push_refs() Jon Simons
2026-10-09 19:29 ` [PATCH 07/15] t5516: test pushing two refspecs creating the same new branch Jon Simons
2026-10-09 19:29 ` [PATCH 08/15] t5408, t5410: test duplicate updates without relying on the client Jon Simons
2026-10-09 19:29 ` [PATCH 09/15] t5408: check refspec order with distinct destinations Jon Simons
2026-10-09 19:29 ` [PATCH 10/15] t5408: expect client-side error for duplicate destinations Jon Simons
2026-10-09 19:29 ` [PATCH 11/15] remote: reject duplicate destinations on an empty remote Jon Simons
2026-10-09 19:29 ` [PATCH 12/15] remote: use strmap for match_explicit_refs() Jon Simons
2026-10-09 19:29 ` [PATCH 13/15] t/perf: measure --force-with-lease in p5516 Jon Simons
2026-10-09 19:29 ` [PATCH 14/15] remote: restructure apply_push_cas() loops Jon Simons
2026-10-09 19:29 ` [PATCH 15/15] remote: use strmap for apply_push_cas() Jon Simons
2026-10-09 20:18 ` [PATCH 00/15] push: speed up client-side refspec matching Kristoffer Haugsbakk
2026-10-11 2:35 ` Jon Simons
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261009192953.81794-1-jon@jonsimons.org \
--to=jon@jonsimons.org \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox