[PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing

["Marc Strapetz via GitGitGadget" <gitgitgadget@gmail.com>]

A common feature-branch workflow is to push a branch, rebase it onto an
updated upstream, and force-push the rewritten commits. Large blobs added or
changed on the feature branch may survive the rebase unchanged. Although the
receiver already has those blobs through the old branch tip, Git can pack
and send them again. This can turn a small history rewrite into a large
upload.

This series teaches send-pack to request inspection of the old tip's tree
before packing, so more receiver-known objects can be excluded.

The additional inspection is enabled only for forced non-fast-forward
updates of existing branches with locally available old tips. Ordinary
fast-forward pushes do not enable it, including ones made with --force. No
server or wire-protocol change is needed.

The old tip is already a negative revision, but it may lie outside the
boundary visited by the ordinary revision walk. Its tree is then not
necessarily inspected. Passing it to pack-objects as --inspect-have lets us
compare its root tree with the outgoing trees using the existing sparse tree
marking. This excludes additional receiver-known objects without walking the
old tip's commit history, and prunes identical and receiver-only subtrees.

Shallow pushes keep their existing aggressive edge handling. Partial clones
skip the extra tree inspection to avoid fetching missing objects, and
successful bitmap enumeration keeps its existing path. Old tips are resolved
in pack-objects, where replacement refs are already disabled.

I compared upstream 7e34d8174dcda3435d1404c9a30bef46afe6c13e with the
patched source using trees from Chromium commit
5b05cc16e960a021389ef9b543ab38a037c38d60. A depth-two clone supplies the
objects; the measured fixture has a complete, non-shallow history with a
common base and sibling receiver/outgoing commits.

Both builds used GCC 15.2.0 and -g -O2 -Wall on an Intel i7-13700H under
WSL2. Measurements used a warm cache, sparse enumeration, no bitmaps, one
packing thread, two warm-up pairs and 15 measured pairs with alternating
build order. Values below are upstream/patched; total wall times are medians
in milliseconds, including process startup.

Case Objects Pack bytes Total wall ms Reword 3 / 1 421 / 215 2.776 / 1.338
Single file 6 / 5 667 / 553 3.542 / 2.744 ~99% of files 552,724 / 552,723
58,082,988 / 58,082,873 4,525.094 / 4,883.623

In the single-file case, enumeration increases from 0.758 to 0.982 ms, but
pack preparation decreases from 1.637 to 0.458 ms. In the last case, 506,468
regular files receive distinct small replacement blobs; total packing time
increases by about 8%, saving only 115 pack bytes.

These are isolated local pack-objects measurements, not end-to-end pushes:
network transfer and receiving-server work are excluded. Where redundant
large blobs materially inflate the pack, avoiding their transfer should
further reduce end-to-end push time, especially on slower connections; that
benefit was not measured here.

Most objects in this fixture are already excluded through the common base,
so the pack-size savings are small. The widespread-change case tests
inspection overhead on Chromium's directory structure, rather than realistic
compression of a whole-project rewrite. Feedback on this cost and the scope
of the inspection would be particularly useful.

Local verification passed t5322-pack-objects-sparse.sh (19 tests) and
t5516-fetch-push.sh (132 tests). Coverage includes identical trees, shared
blobs under different trees, retained changed objects, subtree pruning,
partial clones and local replacement refs.

Marc Strapetz (3):
  t5516: record redundant blobs after rebasing
  pack-objects: inspect selected haves for shared objects
  send-pack: avoid resending unchanged blobs after rebasing

 Documentation/git-pack-objects.adoc |   7 ++
 builtin/pack-objects.c              |  81 ++++++++++++++-
 odb.c                               |   1 +
 odb.h                               |   8 ++
 odb/source-files.c                  |   3 +
 send-pack.c                         |  30 ++++--
 t/t5322-pack-objects-sparse.sh      | 148 ++++++++++++++++++++++++++++
 t/t5516-fetch-push.sh               |  43 ++++++++
 8 files changed, 314 insertions(+), 7 deletions(-)


base-commit: 7e34d8174dcda3435d1404c9a30bef46afe6c13e
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2281%2Fmstrap%2Fsend-pack-inspect-haves-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2281/mstrap/send-pack-inspect-haves-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2281
-- 
gitgitgadget