Git development
 help / color / mirror / Atom feed
* [PATCH 00/15] push: speed up client-side refspec matching
@ 2026-10-09 19:29 Jon Simons
  2026-10-09 19:29 ` [PATCH 01/15] remote: validate --force-with-lease <refname> argument Jon Simons
                   ` (15 more replies)
  0 siblings, 16 replies; 18+ messages in thread
From: Jon Simons @ 2026-10-09 19:29 UTC (permalink / raw)
  To: git; +Cc: Jon Simons

This series speeds up client-side refspec matching in push, focused on
pushing lots of branch deletes to a server that advertises lots of refs
(100k in the benchmark below).

Before this series, a 'git push --dry-run' of 100 delete refspecs to a
server with 100k refs takes ~2.5s from an empty client and ~6.6s from a
mirror; using one --force-with-lease per refspec takes ~4.8s and ~8.9s
respectively.  After, the same pushes take ~0.15s to ~0.19s.

refname_match() is updated to reuse the existing match_parse_rule(),
and a few client-side linear ref traversals are converted to
strmap lookups in remote.c.

Guide to changes:

 - Commits are ordered to introduce failing tests for behavioral
   changes and bugfixes, before the code change that fixes them.

 - Prep refname_match() call sites for behavioral change of mkpath()
   removal, and introduce benchmark:

   (1) remote: validate --force-with-lease <refname> argument
   (2) t5516: demonstrate push with "./"-prefixed source
   (3) t5510: document fetch with "./"-prefixed branch.<name>.merge
   (4) t/perf: add explicit delete refspec matching test

 - The first speedup, best viewed with `--color-moved`, is where
   refname_match() changes behavior: mkpath() stripped a leading "./"
   from the name being matched, so "./refs/heads/foo" used to match
   "refs/heads/foo".  It no longer does.

   (5) refs: stop using mkpath() in refname_match()

 - The next speedup establishes the conversion pattern of using
   expand_ref_prefix() with strmap lookups (the same pattern is used
   in two more spots that follow):

   (6) remote: use strmap for check_push_refs()

 - Fix a long-standing buglet where pushing two refspecs with the same
   destination to an empty remote is not rejected client-side, as it is
   when the remote has any ref.

   t5408 and t5410 now cover server-side duplicate rejection and
   send-pack's handling of a ref reported twice, without relying
   on the client missing the duplicates.

   Fixing this separately keeps the behavior change out of the strmap
   conversion that immediately follows, which would otherwise fix it
   as a side-effect:

   (7) t5516: test pushing two refspecs creating the same new branch
   (8) t5408, t5410: test duplicate updates without relying on the client
   (9) t5408: check refspec order with distinct destinations
   (10) t5408: expect client-side error for duplicate destinations
   (11) remote: reject duplicate destinations on an empty remote

 - Convert match_explicit_refs():

   (12) remote: use strmap for match_explicit_refs()

 - Measure and convert --force-with-lease paths:

   (13) t/perf: measure --force-with-lease in p5516
   (14) remote: restructure apply_push_cas() loops
   (15) remote: use strmap for apply_push_cas()

Numbers:

A new benchmark demonstrates each performance change.  These numbers
were gathered running the final form of p5516 against the earlier
trees on my machine:

  BASELINE             origin/master
  REFNAME_MATCH        (5)  refs: stop using mkpath() in refname_match()
  MAP_CHECK_PUSH_REFS  (6)  remote: use strmap for check_push_refs()
  MAP_EXPLICIT_REFS    (12) remote: use strmap for match_explicit_refs()
  MAP_APPLY_PUSH_CAS   (15) remote: use strmap for apply_push_cas()

  Test                           BASELINE          REFNAME_MATCH            MAP_CHECK_PUSH_REFS      MAP_EXPLICIT_REFS        MAP_APPLY_PUSH_CAS
  --------------------------------------------------------------------------------------------------------------------------------------------------
  5516.3: empty:refspecs:1       0.16(0.09+0.11)   0.14(0.07+0.11) -12.5%   0.14(0.07+0.11) -12.5%   0.14(0.07+0.11) -12.5%   0.14(0.08+0.11) -12.5%
  5516.5: empty:refspecs:10      0.37(0.30+0.11)   0.17(0.10+0.11) -54.1%   0.17(0.11+0.11) -54.1%   0.13(0.07+0.11) -64.9%   0.14(0.07+0.11) -62.2%
  5516.7: empty:refspecs:100     2.48(2.41+0.11)   0.49(0.43+0.11) -80.2%   0.47(0.40+0.11) -81.0%   0.15(0.08+0.11) -94.0%   0.15(0.09+0.11) -94.0%
  5516.9: mirror:refspecs:1      0.22(0.15+0.11)   0.16(0.09+0.11) -27.3%   0.16(0.09+0.11) -27.3%   0.16(0.09+0.11) -27.3%   0.17(0.10+0.11) -22.7%
  5516.11: mirror:refspecs:10    0.81(0.74+0.11)   0.26(0.19+0.11) -67.9%   0.23(0.16+0.11) -71.6%   0.16(0.09+0.11) -80.2%   0.17(0.10+0.12) -79.0%
  5516.13: mirror:refspecs:100   6.64(6.57+0.12)   1.21(1.13+0.12) -81.8%   0.86(0.79+0.12) -87.0%   0.18(0.11+0.11) -97.3%   0.18(0.11+0.11) -97.3%
  5516.16: empty:lease:1         0.17(0.11+0.11)   0.14(0.07+0.11) -17.6%   0.14(0.07+0.11) -17.6%   0.14(0.07+0.11) -17.6%   0.15(0.08+0.11) -11.8%
  5516.19: empty:lease:10        0.71(0.65+0.11)   0.20(0.14+0.11) -71.8%   0.20(0.13+0.11) -71.8%   0.17(0.10+0.11) -76.1%   0.15(0.08+0.11) -78.9%
  5516.22: empty:lease:100       4.79(4.71+0.12)   0.78(0.71+0.12) -83.7%   0.80(0.73+0.12) -83.3%   0.45(0.38+0.11) -90.6%   0.16(0.09+0.11) -96.7%
  5516.25: mirror:lease:1        0.24(0.17+0.11)   0.17(0.10+0.11) -29.2%   0.17(0.10+0.12) -29.2%   0.17(0.10+0.12) -29.2%   0.18(0.11+0.12) -25.0%
  5516.28: mirror:lease:10       1.04(0.97+0.11)   0.29(0.22+0.11) -72.1%   0.26(0.19+0.12) -75.0%   0.20(0.13+0.11) -80.8%   0.18(0.11+0.12) -82.7%
  5516.31: mirror:lease:100      8.93(8.85+0.12)   1.47(1.40+0.12) -83.5%   1.11(1.04+0.12) -87.6%   0.49(0.42+0.12) -94.5%   0.19(0.12+0.12) -97.9%

Jon Simons (15):
  remote: validate --force-with-lease <refname> argument
  t5516: demonstrate push with "./"-prefixed source
  t5510: document fetch with "./"-prefixed branch.<name>.merge
  t/perf: add explicit delete refspec matching test
  refs: stop using mkpath() in refname_match()
  remote: use strmap for check_push_refs()
  t5516: test pushing two refspecs creating the same new branch
  t5408, t5410: test duplicate updates without relying on the client
  t5408: check refspec order with distinct destinations
  t5408: expect client-side error for duplicate destinations
  remote: reject duplicate destinations on an empty remote
  remote: use strmap for match_explicit_refs()
  t/perf: measure --force-with-lease in p5516
  remote: restructure apply_push_cas() loops
  remote: use strmap for apply_push_cas()

 refs.c                              |  88 +++++-----
 remote.c                            | 257 +++++++++++++++++++---------
 t/perf/p5516-push-delete-refspec.sh |  67 ++++++++
 t/t5408-send-pack-stdin.sh          |  56 +++++-
 t/t5410-receive-pack.sh             |  35 ++++
 t/t5510-fetch.sh                    |  44 +++++
 t/t5516-fetch-push.sh               |  42 +++++
 t/t5533-push-cas.sh                 |   9 +
 8 files changed, 465 insertions(+), 133 deletions(-)
 create mode 100755 t/perf/p5516-push-delete-refspec.sh

-- 
2.55.0


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2026-10-11  2:41 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-09 19:29 [PATCH 00/15] push: speed up client-side refspec matching Jon Simons
2026-10-09 19:29 ` [PATCH 01/15] remote: validate --force-with-lease <refname> argument Jon Simons
2026-10-09 19:29 ` [PATCH 02/15] t5516: demonstrate push with "./"-prefixed source Jon Simons
2026-10-09 19:29 ` [PATCH 03/15] t5510: document fetch with "./"-prefixed branch.<name>.merge Jon Simons
2026-10-09 19:29 ` [PATCH 04/15] t/perf: add explicit delete refspec matching test Jon Simons
2026-10-09 19:29 ` [PATCH 05/15] refs: stop using mkpath() in refname_match() Jon Simons
2026-10-09 19:29 ` [PATCH 06/15] remote: use strmap for check_push_refs() Jon Simons
2026-10-09 19:29 ` [PATCH 07/15] t5516: test pushing two refspecs creating the same new branch Jon Simons
2026-10-09 19:29 ` [PATCH 08/15] t5408, t5410: test duplicate updates without relying on the client Jon Simons
2026-10-09 19:29 ` [PATCH 09/15] t5408: check refspec order with distinct destinations Jon Simons
2026-10-09 19:29 ` [PATCH 10/15] t5408: expect client-side error for duplicate destinations Jon Simons
2026-10-09 19:29 ` [PATCH 11/15] remote: reject duplicate destinations on an empty remote Jon Simons
2026-10-09 19:29 ` [PATCH 12/15] remote: use strmap for match_explicit_refs() Jon Simons
2026-10-09 19:29 ` [PATCH 13/15] t/perf: measure --force-with-lease in p5516 Jon Simons
2026-10-09 19:29 ` [PATCH 14/15] remote: restructure apply_push_cas() loops Jon Simons
2026-10-09 19:29 ` [PATCH 15/15] remote: use strmap for apply_push_cas() Jon Simons
2026-10-09 20:18 ` [PATCH 00/15] push: speed up client-side refspec matching Kristoffer Haugsbakk
2026-10-11  2:35   ` Jon Simons

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox