[PATCH v3 0/4] refs: run copy and rename through transactions

[Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com>]

Reference copy and rename bypass the transaction API. With files, the
reference-transaction hook sees only the source deletion; with reftable,
it sees neither endpoint. This series puts the logical ref updates and
the reflog history into ordinary transactions so hooks can observe and
reject the complete operation.

Thanks to Patrick, Junio and Karthik for their feedback. Following the
review of v2, this version uses ref_transaction_update_reflog(), the API
used by backend migration, to replay history. A new
ref_transaction_replace_reflog() operation supplies the missing ability
to discard destination history before installing the queued entries.

Changes since v2:

* Split the work into four patches: internal hook suppression, reflog
  replacement, copy/rename integration, and removal of the old callbacks.
* Remove the special copy/rename dispatch from backend prepare, finish
  and abort. Each destination points to its source update, so multiple
  copies, renames and ordinary updates can share one transaction.
* Keep the hook-suppression flag private and check it centrally in
  run_transaction_hook(). It is used for the physical packed-refs child
  transaction, whose changes the parent already reports.
* Drop the pre-lock snapshot revalidation. A preparing hook may change
  the source; prepared and committed report the value read under lock.
  Copy sources are locked but are not reported as changes to the hook.
* Stage reflog replacements during prepare. A prepared veto discards
  staging files without restoring old values over live refs or changing
  the source, destination or HEAD history.
* Add coverage for complete reflog contents, mixed transactions, hook
  vetoes, source races and failure paths, and measure the performance
  cost of replaying history.

Backend-specific work remains necessary to implement the transaction
primitives. Files stages logs beside logs/refs using unique temporary
files. For directory/file conflicts and case-only renames, it queues
the destination in packed-refs: the loose source cannot remain visible
while also making room for a loose destination lock. During finish, a
backup preserves the source log until the destination log is installed;
an installation error restores that backup. Reftable writes tombstones
for old destination entries in the same table as the replacement.

The backends' final reflog records remain distinct: files appends
old->old; reftable appends old->zero and zero->old for rename, or
zero->old for copy. Forced reftable copies do change behavior: unrelated
destination history is replaced by source history, matching files when
the source has a reflog.

Replaying history requires O(N) time and memory. Buffered staging avoids
a write system call per entry, but files rename loses its constant-time
reflog move. On macOS/arm64, with no hook configured, median milliseconds
per operation over seven samples of ten operations were:

                    entries       base        v3
  files copy             10       5.05      5.08
  files rename           10       4.88      5.26
  files copy          10000      13.20     16.11
  files rename        10000       5.09     17.09
  reftable copy          10       5.25      6.11
  reftable rename        10       5.81      6.50
  reftable copy       10000      53.39     68.06
  reftable rename     10000      43.44     54.06

Process startup is included. Copy overwrites the same destination;
rename alternates between two names. Reftable starts from a migration
of the same files fixture. Repeated operations add normal log entries
and incur normal reftable compaction. Patch 3 adds
t/perf/p1424-ref-copy-rename.sh; these measurements used a separate
monotonic-clock driver because GNU time is not installed here. The
files rename regression is a cost of this design, not just hook overhead.

Validation covered 15 relevant suites with files and reftable, including
branch, update-ref, hooks, migration, reflogs and worktree refs. The two
new suites contain 34 tests, with backend-specific skips; they also
passed with SHA-256 and AddressSanitizer/UndefinedBehaviorSanitizer.
The reflog prerequisite and main change were tested as intermediate
trees. This was not a full test-suite run or a Linux/Windows run.

Files transactions are still not crash-atomic. A later failure in
finish may leave partial ref changes, as with ordinary transactions.
The source-log backup covers an installation error, not a process crash.
Unrelated nested renames can also contend for the global packed-refs
lock.

The base is 0f8e75abeb. This series is independent of the separate
batched-deletion old-OID fix discussed elsewhere in the thread.

Original patch:
https://lore.kernel.org/git/20260920165037.88524-1-maciej.ciemborowicz@gmail.com/

The range-diff below treats the rewritten and split implementation as
four new commits, so the changes above describe the mapping from v2.

Maciej Ciemborowicz (4):
  refs: distinguish internal transactions from logical updates
  refs: support replacing reflogs in a transaction
  refs: run copy and rename through ordinary transactions
  refs: remove backend-specific copy and rename callbacks

 Documentation/githooks.adoc     |  10 +
 refs.c                          | 200 ++++++++-
 refs.h                          |  25 ++
 refs/debug.c                    |  24 --
 refs/files-backend.c            | 739 +++++++++++++++++---------------
 refs/packed-backend.c           |   2 -
 refs/refs-internal.h            |  28 +-
 refs/reftable-backend.c         | 334 ++-------------
 t/helper/test-ref-store.c       |  76 ++++
 t/perf/p1424-ref-copy-rename.sh |  48 +++
 t/t1424-ref-copy-transaction.sh | 352 +++++++++++++++
 t/t1425-reflog-transaction.sh   |  59 +++
 12 files changed, 1206 insertions(+), 691 deletions(-)
 create mode 100755 t/perf/p1424-ref-copy-rename.sh
 create mode 100755 t/t1424-ref-copy-transaction.sh
 create mode 100755 t/t1425-reflog-transaction.sh

Range-diff against v2:
1:  d852537d8c < -:  ---------- refs: run copy and rename through transactions
-:  ---------- > 1:  6d7c146e57 refs: distinguish internal transactions from logical updates
-:  ---------- > 2:  5c3ec4eb49 refs: support replacing reflogs in a transaction
-:  ---------- > 3:  77af4e809c refs: run copy and rename through ordinary transactions
-:  ---------- > 4:  83fa644fb3 refs: remove backend-specific copy and rename callbacks

base-commit: 0f8e75abebff0877cae681a3d5ff31ac47f54220
-- 
2.39.3 (Apple Git-146)