* [BUG] reference-transaction hook misses destination of git branch -m
@ 2026-09-19 13:33 Maciej Ciemborowicz
2026-09-19 20:52 ` Karthik Nayak
0 siblings, 1 reply; 35+ messages in thread
From: Maciej Ciemborowicz @ 2026-09-19 13:33 UTC (permalink / raw)
To: git
Hello,
`git branch -m old new` changes refs/heads/old into refs/heads/new,
but the reference-transaction hook does not report both ref changes.
Observed with Git 2.55:
* files backend: the hook reports deletion of refs/heads/old, but
does not report creation of refs/heads/new;
* reftable backend: the hook reports no usable rename payload.
The equivalent atomic operation performed with `git update-ref --stdin`
reports both updates correctly.
I expected the transaction to include:
<oid> <zero> refs/heads/old
<zero> <oid> refs/heads/new
Minimal reproducer:
#!/bin/sh
set -eu
format=${1:-files}
root=$(mktemp -d)
trap 'rm -rf "$root"' EXIT
repo=$root/repo
hooks=$root/hooks
log=$root/transactions
git init -q --ref-format="$format" "$repo"
git -C "$repo" config user.name Reproducer
git -C "$repo" config user.email repro@example.com
git -C "$repo" commit --allow-empty -qm initial
git -C "$repo" branch old
mkdir "$hooks"
cat >"$hooks/reference-transaction" <<'HOOK'
#!/bin/sh
printf '%s\n' "--- $1" >>"$HOOK_LOG"
cat >>"$HOOK_LOG"
HOOK
chmod +x "$hooks/reference-transaction"
git -C "$repo" config core.hooksPath "$hooks"
export HOOK_LOG=$log
: >"$log"
git -C "$repo" branch -m old new
cat "$log"
The behavior was also tested across Git 2.28–2.55:
https://github.com/ciembor/git-hooks-ext/blob/v0.2.0/tests/compat/README.md
The reference-transaction documentation says that the hook is invoked by
Git commands performing reference updates. A branch rename changes two refs,
but the destination update is not visible to the hook.
Thanks,
Maciej Ciemborowicz
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [BUG] reference-transaction hook misses destination of git branch -m 2026-09-19 13:33 [BUG] reference-transaction hook misses destination of git branch -m Maciej Ciemborowicz @ 2026-09-19 20:52 ` Karthik Nayak 2026-09-20 16:50 ` [PATCH] refs: run copy and rename through transactions Maciej Ciemborowicz 2026-09-23 12:49 ` [PATCH v4 0/3] refs: report old OIDs for batched deletions Maciej Ciemborowicz 0 siblings, 2 replies; 35+ messages in thread From: Karthik Nayak @ 2026-09-19 20:52 UTC (permalink / raw) To: Maciej Ciemborowicz, git [-- Attachment #1: Type: text/plain, Size: 2640 bytes --] Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> writes: > Hello, > > `git branch -m old new` changes refs/heads/old into refs/heads/new, > but the reference-transaction hook does not report both ref changes. > My understanding without digging too deep is that both 'copy' and 'rename' do not use a transaction, as such they don't trigger the 'reference-transaction' hook. This is definitely absurd, since we would expect that all reference operations should use the transaction and trigger the hook. > Observed with Git 2.55: > > * files backend: the hook reports deletion of refs/heads/old, but > does not report creation of refs/heads/new; Yeah, because the files backend calls `refs_delete_ref()` on the old reference which is done within a transaction. > * reftable backend: the hook reports no usable rename payload. Reftable doesn't call `refs_delete_ref()` and simply writes a TOMBSTONE entry to delete the old reference. > The equivalent atomic operation performed with `git update-ref --stdin` > reports both updates correctly. > By equivalent atomic operation do you a 'delete' and 'create' command? > I expected the transaction to include: > > <oid> <zero> refs/heads/old > <zero> <oid> refs/heads/new > > Minimal reproducer: > > #!/bin/sh > set -eu > > format=${1:-files} > root=$(mktemp -d) > trap 'rm -rf "$root"' EXIT > > repo=$root/repo > hooks=$root/hooks > log=$root/transactions > > git init -q --ref-format="$format" "$repo" > git -C "$repo" config user.name Reproducer > git -C "$repo" config user.email repro@example.com > git -C "$repo" commit --allow-empty -qm initial > git -C "$repo" branch old > > mkdir "$hooks" > cat >"$hooks/reference-transaction" <<'HOOK' > #!/bin/sh > printf '%s\n' "--- $1" >>"$HOOK_LOG" > cat >>"$HOOK_LOG" > HOOK > chmod +x "$hooks/reference-transaction" > > git -C "$repo" config core.hooksPath "$hooks" > export HOOK_LOG=$log > : >"$log" > > git -C "$repo" branch -m old new > cat "$log" > > The behavior was also tested across Git 2.28–2.55: > https://github.com/ciembor/git-hooks-ext/blob/v0.2.0/tests/compat/README.md > > The reference-transaction documentation says that the hook is invoked by > Git commands performing reference updates. A branch rename changes two refs, > but the destination update is not visible to the hook. > Either ways, I think we should fix this, I could have a look if you aren't submitting a patch yourself :) > Thanks, > Maciej Ciemborowicz [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 690 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH] refs: run copy and rename through transactions 2026-09-19 20:52 ` Karthik Nayak @ 2026-09-20 16:50 ` Maciej Ciemborowicz 2026-09-21 17:54 ` Junio C Hamano ` (3 more replies) 2026-09-23 12:49 ` [PATCH v4 0/3] refs: report old OIDs for batched deletions Maciej Ciemborowicz 1 sibling, 4 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-09-20 16:50 UTC (permalink / raw) To: git; +Cc: Maciej Ciemborowicz, Karthik Nayak Reference copy and rename operations currently bypass the transaction API. Consequently, the reference-transaction hook sees only the source deletion with the files backend and no useful update with the reftable backend. Represent both operations as reference transactions containing their logical updates. A rename is a deletion of the old reference and creation of the new reference in the same transaction. Retain backend-specific reflog handling: the files backend stages its existing rename procedure across prepare, finish and abort, while reftable stages an addition while holding the stack lock. Suppress hooks for the files backend's nested deletion transactions so that callers observe one logical transaction. Record and verify the source and destination values after taking backend locks. This rejects concurrent changes instead of applying a rename or copy that differs from the payload shown to the preparing hook. Preserve D/F renames and restore overwritten references and reflogs when a prepared hook rejects the operation. Add coverage for rename, copy, forced updates, both directions of D/F conflicts, concurrent updates and prepared-hook rollback. Helped-by: Karthik Nayak <karthik.188@gmail.com> Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- refs.c | 137 ++++++++++++--- refs.h | 3 + refs/debug.c | 25 --- refs/files-backend.c | 276 +++++++++++++++++++++++++++---- refs/packed-backend.c | 2 - refs/refs-internal.h | 38 +++-- refs/reftable-backend.c | 194 ++++++++++++++++------ t/t1416-ref-transaction-hooks.sh | 142 ++++++++++++++++ 8 files changed, 679 insertions(+), 138 deletions(-) diff --git a/refs.c b/refs.c index 92d5df5b7..22c000f7f 100644 --- a/refs.c +++ b/refs.c @@ -1004,15 +1004,17 @@ long get_files_ref_lock_timeout_ms(struct repository *repo) return timeout_ms; } -int refs_delete_ref(struct ref_store *refs, const char *msg, - const char *refname, - const struct object_id *old_oid, - unsigned int flags) +int refs_delete_ref_with_transaction_flags(struct ref_store *refs, + const char *msg, + const char *refname, + const struct object_id *old_oid, + unsigned int flags, + unsigned int transaction_flags) { struct ref_transaction *transaction; struct strbuf err = STRBUF_INIT; - transaction = ref_store_transaction_begin(refs, 0, &err); + transaction = ref_store_transaction_begin(refs, transaction_flags, &err); if (!transaction || ref_transaction_delete(transaction, refname, old_oid, NULL, flags, msg, &err) || @@ -1027,6 +1029,15 @@ int refs_delete_ref(struct ref_store *refs, const char *msg, return 0; } +int refs_delete_ref(struct ref_store *refs, const char *msg, + const char *refname, + const struct object_id *old_oid, + unsigned int flags) +{ + return refs_delete_ref_with_transaction_flags(refs, msg, refname, + old_oid, flags, 0); +} + static void copy_reflog_msg(struct strbuf *sb, const char *msg) { char c; @@ -1270,6 +1281,10 @@ void ref_transaction_free(struct ref_transaction *transaction) string_list_clear(&transaction->refnames, 0); free(transaction->updates); + free(transaction->old_refname); + free(transaction->new_refname); + free(transaction->logmsg); + free(transaction->destination_target); free(transaction); } @@ -2710,7 +2725,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, return REF_TRANSACTION_ERROR_GENERIC; /* Preparing checks before locking references */ - ret = run_transaction_hook(transaction, "preparing"); + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : + run_transaction_hook(transaction, "preparing"); if (ret) { ref_transaction_abort(transaction, err); die(_(abort_by_ref_transaction_hook), "preparing"); @@ -2720,7 +2736,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, if (ret) return ret; - ret = run_transaction_hook(transaction, "prepared"); + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : + run_transaction_hook(transaction, "prepared"); if (ret) { ref_transaction_abort(transaction, err); die(_(abort_by_ref_transaction_hook), "prepared"); @@ -2750,7 +2767,8 @@ int ref_transaction_abort(struct ref_transaction *transaction, break; } - run_transaction_hook(transaction, "aborted"); + if (!(transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK)) + run_transaction_hook(transaction, "aborted"); ref_transaction_free(transaction); return ret; @@ -2781,7 +2799,8 @@ int ref_transaction_commit(struct ref_transaction *transaction, } ret = refs->be->transaction_finish(refs, transaction, err); - if (!ret && !(transaction->flags & REF_TRANSACTION_FLAG_INITIAL)) + if (!ret && !(transaction->flags & (REF_TRANSACTION_FLAG_INITIAL | + REF_TRANSACTION_FLAG_SKIP_HOOK))) run_transaction_hook(transaction, "committed"); return ret; } @@ -3123,28 +3142,100 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg, return ret; } -int refs_rename_ref(struct ref_store *refs, const char *oldref, - const char *newref, const char *logmsg) +static int refs_copy_or_rename_ref(struct ref_store *refs, const char *oldref, + const char *newref, const char *logmsg, + int copy) { - char *msg; - int retval; + struct ref_transaction *transaction = NULL; + struct object_id old_oid, new_oid; + struct strbuf new_target = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; + char *msg = normalize_reflog_message(logmsg); + int old_flags, new_flags = 0, new_exists = 0, ret = 1; - msg = normalize_reflog_message(logmsg); - retval = refs->be->rename_ref(refs, oldref, newref, msg); + if (!strcmp(oldref, newref)) { + ret = 0; + goto out; + } + + if (!refs_resolve_ref_unsafe(refs, oldref, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, + &old_oid, &old_flags)) { + error("refname %s not found", oldref); + goto out; + } + if (old_flags & REF_ISSYMREF) { + error("refname %s is a symbolic ref, %s it is not supported", + oldref, copy ? "copying" : "renaming"); + goto out; + } + + transaction = ref_store_transaction_begin(refs, 0, &err); + if (!transaction) + goto error; + transaction->type = copy ? REF_TRANSACTION_TYPE_COPY : + REF_TRANSACTION_TYPE_RENAME; + transaction->old_refname = xstrdup(oldref); + transaction->new_refname = xstrdup(newref); + transaction->logmsg = xstrdup(msg); + oidcpy(&transaction->source_oid, &old_oid); + + if (!copy && ref_transaction_delete(transaction, oldref, &old_oid, NULL, + REF_NO_DEREF, msg, &err)) + goto error; + + if (refs_resolve_ref_unsafe(refs, newref, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, + &new_oid, &new_flags)) { + new_exists = 1; + if ((new_flags & REF_ISSYMREF) && + refs_read_symbolic_ref(refs, newref, &new_target) < 0) { + strbuf_addf(&err, "unable to read symbolic ref %s", newref); + goto error; + } + } else { + oidclr(&new_oid, refs->repo->hash_algo); + } + transaction->destination_exists = new_exists; + if (new_flags & REF_ISSYMREF) + transaction->destination_target = xstrdup(new_target.buf); + else if (transaction->destination_exists) + oidcpy(&transaction->destination_oid, &new_oid); + + if (ref_transaction_update(transaction, newref, &old_oid, + (new_flags & REF_ISSYMREF) ? NULL : &new_oid, + NULL, + (new_flags & REF_ISSYMREF) ? new_target.buf : NULL, + REF_NO_DEREF | REF_SKIP_CREATE_REFLOG, + NULL, &err)) + goto error; + + if (ref_transaction_commit(transaction, &err)) + goto error; + + ret = 0; + goto out; + +error: + error("%s", err.buf); +out: + ref_transaction_free(transaction); + strbuf_release(&new_target); + strbuf_release(&err); free(msg); - return retval; + return ret; } -int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, +int refs_rename_ref(struct ref_store *refs, const char *oldref, const char *newref, const char *logmsg) { - char *msg; - int retval; + return refs_copy_or_rename_ref(refs, oldref, newref, logmsg, 0); +} - msg = normalize_reflog_message(logmsg); - retval = refs->be->copy_ref(refs, oldref, newref, msg); - free(msg); - return retval; +int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, + const char *newref, const char *logmsg) +{ + return refs_copy_or_rename_ref(refs, oldref, newref, logmsg, 1); } const char *ref_update_original_update_refname(struct ref_update *update) diff --git a/refs.h b/refs.h index 9979446d1..25bb8720f 100644 --- a/refs.h +++ b/refs.h @@ -784,6 +784,9 @@ enum ref_transaction_flag { * while rejecting updates which do not match the expected state. */ REF_TRANSACTION_ALLOW_FAILURE = (1 << 1), + + /* Suppress hooks for an update nested in another transaction. */ + REF_TRANSACTION_FLAG_SKIP_HOOK = (1 << 2), }; /* diff --git a/refs/debug.c b/refs/debug.c index 639db0f26..87b84e767 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -143,28 +143,6 @@ static int debug_optimize_required(struct ref_store *ref_store, return res; } -static int debug_rename_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = drefs->refs->be->rename_ref(drefs->refs, oldref, newref, - logmsg); - trace_printf_key(&trace_refs, "rename_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - -static int debug_copy_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = - drefs->refs->be->copy_ref(drefs->refs, oldref, newref, logmsg); - trace_printf_key(&trace_refs, "copy_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - struct debug_ref_iterator { struct ref_iterator base; struct ref_iterator *iter; @@ -453,9 +431,6 @@ struct ref_storage_be refs_be_debug = { .optimize = debug_optimize, .optimize_required = debug_optimize_required, - .rename_ref = debug_rename_ref, - .copy_ref = debug_copy_ref, - .iterator_begin = debug_ref_iterator_begin, .read_raw_ref = debug_read_raw_ref, .read_symbolic_ref = debug_read_symbolic_ref, diff --git a/refs/files-backend.c b/refs/files-backend.c index 71628550f..92601b5e9 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1594,6 +1594,7 @@ static int files_optimize_required(struct ref_store *ref_store, * live into logs/refs. */ #define TMP_RENAMED_LOG "refs/.tmp-renamed-log" +#define TMP_RENAMED_LOG_DESTINATION "refs/.tmp-renamed-log-destination" struct rename_cb { const char *tmp_renamed_log; @@ -1685,12 +1686,24 @@ static int refs_rename_ref_available(struct ref_store *refs, return ok; } +struct files_copy_or_rename_transaction_data { + struct ref_lock *lock; + struct object_id orig_oid; + struct object_id destination_oid; + char *destination_target; + int logmoved; + int destination_exists; + int destination_log_backed_up; +}; + static int files_copy_or_rename_ref(struct ref_store *ref_store, const char *oldrefname, const char *newrefname, - const char *logmsg, int copy) + const char *logmsg, int copy, + struct ref_transaction *transaction) { struct files_ref_store *refs = - files_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); + files_downcast(ref_store, REF_STORE_WRITE, + "ref_transaction_prepare"); struct object_id orig_oid; int flag = 0, logmoved = 0; struct ref_lock *lock; @@ -1698,12 +1711,19 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, struct strbuf sb_oldref = STRBUF_INIT; struct strbuf sb_newref = STRBUF_INIT; struct strbuf tmp_renamed_log = STRBUF_INIT; + struct strbuf tmp_destination_log = STRBUF_INIT; + struct strbuf destination_target = STRBUF_INIT; int log, ret; + int destination_exists = 0, destination_flags = 0; + int destination_log_backed_up = 0; + struct object_id destination_oid; + struct files_copy_or_rename_transaction_data *data; struct strbuf err = STRBUF_INIT; files_reflog_path(refs, &sb_oldref, oldrefname); files_reflog_path(refs, &sb_newref, newrefname); files_reflog_path(refs, &tmp_renamed_log, TMP_RENAMED_LOG); + files_reflog_path(refs, &tmp_destination_log, TMP_RENAMED_LOG_DESTINATION); log = !lstat(sb_oldref.buf, &loginfo); if (log && S_ISLNK(loginfo.st_mode)) { @@ -1727,11 +1747,67 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, oldrefname); goto out; } + if (!oideq(&orig_oid, &transaction->source_oid)) { + ret = error("refname %s is at %s but expected %s", + oldrefname, oid_to_hex(&orig_oid), + oid_to_hex(&transaction->source_oid)); + goto out; + } if (!refs_rename_ref_available(&refs->base, oldrefname, newrefname)) { ret = 1; goto out; } + if (refs_resolve_ref_unsafe(&refs->base, newrefname, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, + &destination_oid, &destination_flags)) { + destination_exists = 1; + if ((destination_flags & REF_ISSYMREF) && + refs_read_symbolic_ref(&refs->base, newrefname, + &destination_target) < 0) { + ret = error("unable to read symbolic ref %s", newrefname); + goto out; + } + } + if (destination_exists != transaction->destination_exists) { + ret = error("refname %s changed while renaming", newrefname); + goto out; + } + if (destination_exists) { + if (destination_flags & REF_ISSYMREF) { + if (!transaction->destination_target || + strcmp(destination_target.buf, + transaction->destination_target)) { + ret = error("refname %s changed while renaming", + newrefname); + goto out; + } + } else if (transaction->destination_target || + !oideq(&destination_oid, + &transaction->destination_oid)) { + ret = error("refname %s changed while renaming", newrefname); + goto out; + } + } + + if (!lstat(sb_newref.buf, &loginfo)) { + if (S_ISLNK(loginfo.st_mode)) { + ret = error("reflog for %s is a symlink", newrefname); + goto out; + } + if (S_ISREG(loginfo.st_mode)) { + if (copy_file(refs->base.repo, tmp_destination_log.buf, + sb_newref.buf, 0644)) { + if (errno != EEXIST) + unlink(tmp_destination_log.buf); + ret = error("unable to back up logfile logs/%s: %s", + newrefname, strerror(errno)); + goto out; + } + destination_log_backed_up = 1; + } + } + if (!copy && log && rename(sb_oldref.buf, tmp_renamed_log.buf)) { ret = error("unable to move logfile logs/%s to logs/"TMP_RENAMED_LOG": %s", oldrefname, strerror(errno)); @@ -1744,8 +1820,10 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, goto out; } - if (!copy && refs_delete_ref(&refs->base, logmsg, oldrefname, - &orig_oid, REF_NO_DEREF)) { + if (!copy && refs_delete_ref_with_transaction_flags(&refs->base, logmsg, + oldrefname, &orig_oid, + REF_NO_DEREF, + REF_TRANSACTION_FLAG_SKIP_HOOK)) { error("unable to delete old %s", oldrefname); goto rollback; } @@ -1760,8 +1838,9 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, if (!copy && refs_resolve_ref_unsafe(&refs->base, newrefname, RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, NULL, NULL) && - refs_delete_ref(&refs->base, NULL, newrefname, - NULL, REF_NO_DEREF)) { + refs_delete_ref_with_transaction_flags(&refs->base, NULL, newrefname, + NULL, REF_NO_DEREF, + REF_TRANSACTION_FLAG_SKIP_HOOK)) { if (errno == EISDIR) { struct strbuf path = STRBUF_INIT; int result; @@ -1796,13 +1875,25 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, } oidcpy(&lock->old_oid, &orig_oid); - if (write_ref_to_lockfile(refs, lock, &orig_oid, &err) || - commit_ref_update(refs, lock, &orig_oid, logmsg, 0, &err)) { + if (write_ref_to_lockfile(refs, lock, &orig_oid, &err)) { error("unable to write current sha1 into %s: %s", newrefname, err.buf); strbuf_release(&err); goto rollback; } + CALLOC_ARRAY(data, 1); + data->lock = lock; + oidcpy(&data->orig_oid, &orig_oid); + data->logmoved = logmoved; + data->destination_exists = destination_exists; + data->destination_log_backed_up = destination_log_backed_up; + if (destination_exists && !(destination_flags & REF_ISSYMREF)) + oidcpy(&data->destination_oid, &destination_oid); + if (destination_flags & REF_ISSYMREF) + data->destination_target = strbuf_detach(&destination_target, NULL); + transaction->backend_data = data; + transaction->state = REF_TRANSACTION_PREPARED; + ret = 0; goto out; @@ -1821,38 +1912,40 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, } rollbacklog: - if (logmoved && rename(sb_newref.buf, sb_oldref.buf)) - error("unable to restore logfile %s from %s: %s", - oldrefname, newrefname, strerror(errno)); + if (logmoved) { + if (rename(sb_newref.buf, tmp_renamed_log.buf)) { + error("unable to restore logfile %s from %s: %s", + oldrefname, newrefname, strerror(errno)); + } else { + try_remove_empty_parents(refs, newrefname, + REMOVE_EMPTY_PARENTS_REFLOG); + if (rename_tmp_log(refs, oldrefname)) + error("unable to restore logfile %s from logs/" + TMP_RENAMED_LOG ": %s", + oldrefname, strerror(errno)); + } + } if (!logmoved && log && rename(tmp_renamed_log.buf, sb_oldref.buf)) error("unable to restore logfile %s from logs/"TMP_RENAMED_LOG": %s", oldrefname, strerror(errno)); + if (destination_log_backed_up && + rename(tmp_destination_log.buf, sb_newref.buf)) + error("unable to restore logfile %s: %s", + newrefname, strerror(errno)); ret = 1; out: + if (ret && destination_log_backed_up) + unlink(tmp_destination_log.buf); strbuf_release(&sb_newref); strbuf_release(&sb_oldref); strbuf_release(&tmp_renamed_log); + strbuf_release(&tmp_destination_log); + strbuf_release(&destination_target); return ret; } -static int files_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 0); -} - -static int files_copy_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 1); -} - static int close_ref_gently(struct ref_lock *lock) { if (close_lock_file_gently(&lock->lk)) @@ -2962,6 +3055,13 @@ static int files_transaction_prepare(struct ref_store *ref_store, struct ref_transaction *packed_transaction = NULL; assert(err); + if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) + return files_copy_or_rename_ref(ref_store, + transaction->old_refname, + transaction->new_refname, + transaction->logmsg, + transaction->type == REF_TRANSACTION_TYPE_COPY, + transaction); if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) goto cleanup; @@ -3318,6 +3418,10 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, return ret; } +static int files_transaction_abort(struct ref_store *ref_store, + struct ref_transaction *transaction, + struct strbuf *err); + static int files_transaction_finish(struct ref_store *ref_store, struct ref_transaction *transaction, struct strbuf *err) @@ -3333,6 +3437,33 @@ static int files_transaction_finish(struct ref_store *ref_store, assert(err); + if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { + struct files_copy_or_rename_transaction_data *data = + transaction->backend_data; + int special_ret; + + special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, + transaction->logmsg, 0, err); + if (special_ret) { + error("unable to write current sha1 into %s: %s", + transaction->new_refname, err->buf); + data->lock = NULL; + files_transaction_abort(ref_store, transaction, err); + return special_ret; + } else if (data->destination_log_backed_up) { + struct strbuf path = STRBUF_INIT; + + files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); + if (unlink(path.buf) < 0 && errno != ENOENT) + warning_errno("unable to remove '%s'", path.buf); + strbuf_release(&path); + } + free(data->destination_target); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + return special_ret; + } if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) return files_transaction_finish_initial(refs, transaction, err); @@ -3476,11 +3607,98 @@ static int files_transaction_finish(struct ref_store *ref_store, static int files_transaction_abort(struct ref_store *ref_store, struct ref_transaction *transaction, - struct strbuf *err UNUSED) + struct strbuf *err) { struct files_ref_store *refs = files_downcast(ref_store, 0, "ref_transaction_abort"); + if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { + struct files_copy_or_rename_transaction_data *data = + transaction->backend_data; + struct strbuf new_log = STRBUF_INIT; + struct strbuf destination_log = STRBUF_INIT; + struct strbuf temporary_log = STRBUF_INIT; + struct ref_transaction *restore_transaction = NULL; + struct ref_lock *lock; + int ret = 0; + + if (data->lock) + unlock_ref(data->lock); + if (transaction->type == REF_TRANSACTION_TYPE_RENAME) { + lock = lock_ref_oid_basic(refs, transaction->old_refname, err); + if (!lock || + write_ref_to_lockfile(refs, lock, &data->orig_oid, err) || + commit_ref_update(refs, lock, &data->orig_oid, NULL, + REF_SKIP_CREATE_REFLOG, err)) + ret = -1; + } + + if (data->logmoved) { + files_reflog_path(refs, &new_log, transaction->new_refname); + if (transaction->type == REF_TRANSACTION_TYPE_RENAME) { + files_reflog_path(refs, &temporary_log, TMP_RENAMED_LOG); + if (rename(new_log.buf, temporary_log.buf) < 0) { + strbuf_addf(err, "unable to restore logfile %s: %s", + transaction->old_refname, strerror(errno)); + ret = -1; + } else { + try_remove_empty_parents(refs, + transaction->new_refname, + REMOVE_EMPTY_PARENTS_REFLOG); + if (rename_tmp_log(refs, + transaction->old_refname)) { + strbuf_addf(err, "unable to restore logfile %s: %s", + transaction->old_refname, + strerror(errno)); + ret = -1; + } + } + } else if (unlink(new_log.buf) < 0 && errno != ENOENT) { + strbuf_addf(err, "unable to remove logfile %s: %s", + transaction->new_refname, strerror(errno)); + ret = -1; + } + } + if (data->destination_log_backed_up) { + files_reflog_path(refs, &destination_log, + TMP_RENAMED_LOG_DESTINATION); + if (rename(destination_log.buf, new_log.buf) < 0) { + strbuf_addf(err, "unable to restore logfile %s: %s", + transaction->new_refname, strerror(errno)); + ret = -1; + } + } + + if (transaction->type == REF_TRANSACTION_TYPE_RENAME && + data->destination_exists) { + restore_transaction = ref_store_transaction_begin( + &refs->base, REF_TRANSACTION_FLAG_SKIP_HOOK, err); + if (!restore_transaction || + ref_transaction_update(restore_transaction, + transaction->new_refname, + data->destination_target ? NULL : + &data->destination_oid, + NULL, + data->destination_target, + NULL, + REF_NO_DEREF | + REF_SKIP_CREATE_REFLOG, + NULL, err) || + ref_transaction_commit(restore_transaction, err)) + ret = -1; + ref_transaction_free(restore_transaction); + } + + strbuf_release(&destination_log); + strbuf_release(&temporary_log); + strbuf_release(&new_log); + free(data->destination_target); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + return ret; + } + files_transaction_cleanup(refs, transaction); return 0; } @@ -4095,8 +4313,6 @@ struct ref_storage_be refs_be_files = { .optimize = files_optimize, .optimize_required = files_optimize_required, - .rename_ref = files_rename_ref, - .copy_ref = files_copy_ref, .iterator_begin = files_ref_iterator_begin, .read_raw_ref = files_read_raw_ref, diff --git a/refs/packed-backend.c b/refs/packed-backend.c index a73fc6aca..364a91291 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -2164,8 +2164,6 @@ struct ref_storage_be refs_be_packed = { .optimize = packed_optimize, .optimize_required = packed_optimize_required, - .rename_ref = NULL, - .copy_ref = NULL, .iterator_begin = packed_ref_iterator_begin, .read_raw_ref = packed_read_raw_ref, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index c3ac7b556..406b54b65 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -187,6 +187,13 @@ struct ref_update *ref_transaction_add_update( const char *committer_info, const char *msg); +int refs_delete_ref_with_transaction_flags(struct ref_store *refs, + const char *msg, + const char *refname, + const struct object_id *old_oid, + unsigned int flags, + unsigned int transaction_flags); + /* * Transaction states. * @@ -212,6 +219,12 @@ enum ref_transaction_state { REF_TRANSACTION_CLOSED = 2 }; +enum ref_transaction_type { + REF_TRANSACTION_TYPE_NORMAL = 0, + REF_TRANSACTION_TYPE_RENAME, + REF_TRANSACTION_TYPE_COPY, +}; + /* * Data structure to hold indices of updates which were rejected, for batched * reference updates. While the updates themselves hold the rejection error, @@ -240,6 +253,21 @@ struct ref_transaction { void *backend_data; unsigned int flags; uint64_t max_index; + + /* + * Rename and copy operations need backend-specific reflog handling. + * Their logical updates still live in `updates`, so hooks see the + * operation like any other reference transaction. The fields below + * retain the state that backends verify after taking their locks. + */ + enum ref_transaction_type type; + char *old_refname; + char *new_refname; + char *logmsg; + struct object_id source_oid; + struct object_id destination_oid; + char *destination_target; + unsigned int destination_exists:1; }; /* @@ -451,13 +479,6 @@ typedef int optimize_required_fn(struct ref_store *ref_store, struct refs_optimize_opts *opts, bool *required); -typedef int rename_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); -typedef int copy_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); - /* * Iterate over the references in `ref_store` whose names start with * `prefix`. `prefix` is matched as a literal string, without regard @@ -577,9 +598,6 @@ struct ref_storage_be { optimize_fn *optimize; optimize_required_fn *optimize_required; - rename_ref_fn *rename_ref; - copy_ref_fn *copy_ref; - ref_iterator_begin_fn *iterator_begin; read_raw_ref_fn *read_raw_ref; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 10db03991..cf8a16af5 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -953,6 +953,14 @@ struct reftable_transaction_data { size_t args_nr, args_alloc; }; +struct reftable_copy_or_rename_transaction_data { + struct reftable_addition *addition; +}; + +static int reftable_be_copy_or_rename_prepare(struct ref_store *ref_store, + struct ref_transaction *transaction, + struct strbuf *err); + static void free_transaction_data(struct reftable_transaction_data *tx_data) { if (!tx_data) @@ -1326,6 +1334,10 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, size_t i; int ret; + if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) + return reftable_be_copy_or_rename_prepare(ref_store, transaction, + err); + ret = refs->err; if (ret < 0) goto done; @@ -1419,7 +1431,20 @@ static int reftable_be_transaction_abort(struct ref_store *ref_store UNUSED, struct ref_transaction *transaction, struct strbuf *err UNUSED) { - struct reftable_transaction_data *tx_data = transaction->backend_data; + struct reftable_transaction_data *tx_data; + + if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { + struct reftable_copy_or_rename_transaction_data *data = + transaction->backend_data; + + reftable_addition_destroy(data->addition); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + return 0; + } + + tx_data = transaction->backend_data; free_transaction_data(tx_data); transaction->state = REF_TRANSACTION_CLOSED; return 0; @@ -1667,9 +1692,28 @@ static int reftable_be_transaction_finish(struct ref_store *ref_store UNUSED, struct ref_transaction *transaction, struct strbuf *err) { - struct reftable_transaction_data *tx_data = transaction->backend_data; + struct reftable_transaction_data *tx_data; int ret = 0; + if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { + struct reftable_copy_or_rename_transaction_data *data = + transaction->backend_data; + int special_ret = reftable_addition_commit(data->addition); + + reftable_addition_destroy(data->addition); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + if (special_ret < 0) { + strbuf_addf(err, _("reftable: transaction failure: %s"), + reftable_error_str(special_ret)); + return -1; + } + return 0; + } + + tx_data = transaction->backend_data; + for (size_t i = 0; i < tx_data->args_nr; i++) { tx_data->args[i].max_index = transaction->max_index; @@ -1764,17 +1808,20 @@ struct write_create_symref_arg { struct write_copy_arg { struct reftable_ref_store *refs; struct reftable_backend *be; + struct strbuf *err; const char *oldname; const char *newname; const char *logmsg; int delete_old; + struct ref_transaction *transaction; }; static int write_copy_table(struct reftable_writer *writer, void *cb_data) { struct write_copy_arg *arg = cb_data; uint64_t deletion_ts, creation_ts; - struct reftable_ref_record old_ref = {0}, refs[2] = {0}; + struct reftable_ref_record old_ref = {0}, destination_ref = {0}; + struct reftable_ref_record refs[2] = {0}; struct reftable_log_record old_log = {0}, *logs = NULL; struct reftable_iterator it = {0}; struct string_list skip = STRING_LIST_INIT_NODUP; @@ -1789,14 +1836,75 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) BUG("failed splitting committer info"); if (reftable_stack_read_ref(arg->be->stack, arg->oldname, &old_ref)) { - ret = error(_("refname %s not found"), arg->oldname); + strbuf_addf(arg->err, _("refname %s not found"), arg->oldname); + ret = -1; goto done; } if (old_ref.value_type == REFTABLE_REF_SYMREF) { - ret = error(_("refname %s is a symbolic ref, copying it is not supported"), + strbuf_addf(arg->err, + _("refname %s is a symbolic ref, copying it is not supported"), arg->oldname); + ret = -1; goto done; } + if (arg->transaction) { + struct object_id oid; + + if (old_ref.value_type == REFTABLE_REF_VAL2) + oidread(&oid, old_ref.value.val2.value, + arg->refs->base.repo->hash_algo); + else + oidread(&oid, old_ref.value.val1, + arg->refs->base.repo->hash_algo); + if (!oideq(&oid, &arg->transaction->source_oid)) { + strbuf_addf(arg->err, + _("refname %s is at %s but expected %s"), + arg->oldname, oid_to_hex(&oid), + oid_to_hex(&arg->transaction->source_oid)); + ret = -1; + goto done; + } + + ret = reftable_stack_read_ref(arg->be->stack, arg->newname, + &destination_ref); + if (ret < 0) + goto done; + if (arg->transaction->destination_exists != !ret) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); + ret = -1; + goto done; + } + if (!ret) { + if (destination_ref.value_type == REFTABLE_REF_SYMREF) { + if (!arg->transaction->destination_target || + strcmp(destination_ref.value.symref, + arg->transaction->destination_target)) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); + ret = -1; + goto done; + } + } else { + if (destination_ref.value_type == REFTABLE_REF_VAL2) + oidread(&oid, destination_ref.value.val2.value, + arg->refs->base.repo->hash_algo); + else + oidread(&oid, destination_ref.value.val1, + arg->refs->base.repo->hash_algo); + if (arg->transaction->destination_target || + !oideq(&oid, &arg->transaction->destination_oid)) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); + ret = -1; + goto done; + } + } + } + } /* * There's nothing to do in case the old and new name are the same, so @@ -1815,7 +1923,7 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) ret = refs_verify_refname_available(&arg->refs->base, arg->newname, NULL, &skip, 0, &errbuf); if (ret < 0) { - error("%s", errbuf.buf); + strbuf_addbuf(arg->err, &errbuf); goto done; } @@ -1980,68 +2088,60 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) for (i = 0; i < ARRAY_SIZE(refs); i++) reftable_ref_record_release(&refs[i]); reftable_ref_record_release(&old_ref); + reftable_ref_record_release(&destination_ref); reftable_log_record_release(&old_log); return ret; } -static int reftable_be_rename_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) +static int reftable_be_copy_or_rename_prepare(struct ref_store *ref_store, + struct ref_transaction *transaction, + struct strbuf *err) { struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); + reftable_be_downcast(ref_store, REF_STORE_WRITE, + "ref_transaction_prepare"); + struct reftable_copy_or_rename_transaction_data *data = NULL; struct write_copy_arg arg = { .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - .delete_old = 1, + .err = err, + .oldname = transaction->old_refname, + .newname = transaction->new_refname, + .logmsg = transaction->logmsg, + .delete_old = transaction->type == REF_TRANSACTION_TYPE_RENAME, + .transaction = transaction, }; int ret; + CALLOC_ARRAY(data, 1); ret = refs->err; if (ret < 0) goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); + ret = backend_for(&arg.be, refs, transaction->new_refname, + &arg.newname, 1); if (ret) goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); - -done: - assert(ret != REFTABLE_API_ERROR); - return ret; -} - -static int reftable_be_copy_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) -{ - struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); - struct write_copy_arg arg = { - .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - }; - int ret; - - ret = refs->err; - if (ret < 0) + ret = reftable_stack_addition_new(&data->addition, arg.be->stack, + &reftable_be_write_options(refs)->opts); + if (ret) goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); + ret = reftable_addition_add(data->addition, &write_copy_table, &arg); if (ret) goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); + + transaction->backend_data = data; + transaction->state = REF_TRANSACTION_PREPARED; + return 0; done: assert(ret != REFTABLE_API_ERROR); + if (data) { + reftable_addition_destroy(data->addition); + free(data); + } + transaction->state = REF_TRANSACTION_CLOSED; + if (ret && !err->len) + strbuf_addf(err, _("reftable: transaction prepare: %s"), + reftable_error_str(ret)); return ret; } @@ -2872,8 +2972,6 @@ struct ref_storage_be refs_be_reftable = { .optimize = reftable_be_optimize, .optimize_required = reftable_be_optimize_required, - .rename_ref = reftable_be_rename_ref, - .copy_ref = reftable_be_copy_ref, .iterator_begin = reftable_be_iterator_begin, .read_raw_ref = reftable_be_read_raw_ref, diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh index 4fe9d9b23..116b2ff07 100755 --- a/t/t1416-ref-transaction-hooks.sh +++ b/t/t1416-ref-transaction-hooks.sh @@ -93,6 +93,148 @@ test_expect_success 'hook gets all queued updates in committed state' ' test_cmp expect actual ' +test_expect_success 'hook gets both updates when renaming a branch' ' + test_when_finished "rm -f actual" && + git branch old PRE && + test_hook reference-transaction <<-\EOF && + echo "$1" >>actual && + cat >>actual + EOF + cat >expect <<-EOF && + preparing + $PRE_OID $ZERO_OID refs/heads/old + $ZERO_OID $PRE_OID refs/heads/new + prepared + $PRE_OID $ZERO_OID refs/heads/old + $ZERO_OID $PRE_OID refs/heads/new + committed + $PRE_OID $ZERO_OID refs/heads/old + $ZERO_OID $PRE_OID refs/heads/new + EOF + git branch -m old new && + test_cmp expect actual && + test_must_fail git rev-parse --verify refs/heads/old && + test_cmp_rev PRE refs/heads/new +' + +test_expect_success 'hook gets destination update when copying a branch' ' + test_when_finished "rm -f actual" && + git branch copy-source PRE && + test_hook reference-transaction <<-\EOF && + echo "$1" >>actual && + cat >>actual + EOF + cat >expect <<-EOF && + preparing + $ZERO_OID $PRE_OID refs/heads/copy-destination + prepared + $ZERO_OID $PRE_OID refs/heads/copy-destination + committed + $ZERO_OID $PRE_OID refs/heads/copy-destination + EOF + git branch -c copy-source copy-destination && + test_cmp expect actual && + test_cmp_rev PRE refs/heads/copy-source && + test_cmp_rev PRE refs/heads/copy-destination +' + +test_expect_success 'hook gets overwritten values for forced rename and copy' ' + git branch force-old PRE && + git branch force-new POST && + git branch force-copy-source PRE && + git branch force-copy-destination POST && + test_hook reference-transaction <<-\EOF && + if test "$1" = committed + then + cat >>actual + fi + EOF + git branch -M force-old force-new && + git branch -C force-copy-source force-copy-destination && + cat >expect <<-EOF && + $PRE_OID $ZERO_OID refs/heads/force-old + $POST_OID $PRE_OID refs/heads/force-new + $POST_OID $PRE_OID refs/heads/force-copy-destination + EOF + test_cmp expect actual +' + +test_expect_success 'hook can abort a branch rename after preparation' ' + git branch abort-old PRE && + git branch abort-new POST && + git reflog show --format=%gs abort-old >old-log && + git reflog show --format=%gs abort-new >new-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -M abort-old abort-new && + test_cmp_rev PRE refs/heads/abort-old && + test_cmp_rev POST refs/heads/abort-new && + git reflog show --format=%gs abort-old >old-log-after && + git reflog show --format=%gs abort-new >new-log-after && + test_cmp old-log old-log-after && + test_cmp new-log new-log-after +' + +test_expect_success 'hook can abort a D/F branch rename after preparation' ' + git branch df-old PRE && + git reflog show --format=%gs df-old >df-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -m df-old df-old/child && + test_cmp_rev PRE refs/heads/df-old && + test_must_fail git rev-parse --verify refs/heads/df-old/child && + git reflog show --format=%gs df-old >df-log-after && + test_cmp df-log df-log-after +' + +test_expect_success 'hook can abort a reverse D/F rename after preparation' ' + git branch reverse/old PRE && + git reflog show --format=%gs reverse/old >reverse-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -m reverse/old reverse && + test_cmp_rev PRE refs/heads/reverse/old && + test_must_fail git rev-parse --verify refs/heads/reverse && + git reflog show --format=%gs reverse/old >reverse-log-after && + test_cmp reverse-log reverse-log-after +' + +test_expect_success 'hook can abort a forced branch copy after preparation' ' + git branch copy-abort-old PRE && + git branch copy-abort-new POST && + git reflog show --format=%gs copy-abort-old >copy-old-log && + git reflog show --format=%gs copy-abort-new >copy-new-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -C copy-abort-old copy-abort-new && + test_cmp_rev PRE refs/heads/copy-abort-old && + test_cmp_rev POST refs/heads/copy-abort-new && + git reflog show --format=%gs copy-abort-old >copy-old-log-after && + git reflog show --format=%gs copy-abort-new >copy-new-log-after && + test_cmp copy-old-log copy-old-log-after && + test_cmp copy-new-log copy-new-log-after +' + +test_expect_success 'branch rename detects an update during preparing hook' ' + git branch race-old PRE && + git branch race-new POST && + test_hook reference-transaction <<-\EOF && + marker=$(git rev-parse --git-path rename-race-once) + if test "$1" = preparing && test ! -e "$marker" + then + >"$marker" && + git update-ref refs/heads/race-old POST + fi + EOF + test_must_fail git branch -M race-old race-new && + test_cmp_rev POST refs/heads/race-old && + test_cmp_rev POST refs/heads/race-new +' + test_expect_success 'hook gets all queued updates in aborted state' ' test_when_finished "rm actual" && git reset --hard PRE && base-commit: d38352cd43ab9745686d697872408bc3249a153f -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [PATCH] refs: run copy and rename through transactions 2026-09-20 16:50 ` [PATCH] refs: run copy and rename through transactions Maciej Ciemborowicz @ 2026-09-21 17:54 ` Junio C Hamano 2026-09-21 23:28 ` Junio C Hamano 2026-09-23 13:36 ` [PATCH v2] " Maciej Ciemborowicz ` (2 subsequent siblings) 3 siblings, 1 reply; 35+ messages in thread From: Junio C Hamano @ 2026-09-21 17:54 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: git, Karthik Nayak Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> writes: > Reference copy and rename operations currently bypass the transaction API. > Consequently, the reference-transaction hook sees only the source deletion > with the files backend and no useful update with the reftable backend. > > Represent both operations as reference transactions containing their > logical updates. A rename is a deletion of the old reference and creation > of the new reference in the same transaction. Retain backend-specific > reflog handling: the files backend stages its existing rename procedure > across prepare, finish and abort, while reftable stages an addition while > holding the stack lock. Suppress hooks for the files backend's nested > deletion transactions so that callers observe one logical transaction. > > Record and verify the source and destination values after taking backend > locks. This rejects concurrent changes instead of applying a rename or copy > that differs from the payload shown to the preparing hook. Preserve D/F > renames and restore overwritten references and reflogs when a prepared hook > rejects the operation. > > Add coverage for rename, copy, forced updates, both directions of D/F > conflicts, concurrent updates and prepared-hook rollback. > > Helped-by: Karthik Nayak <karthik.188@gmail.com> > Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> > --- Drop unnecessary "currently" to the first sentence, and add "test" to the laste sentence somewhere, and this would be perfect. Very pleasing to see an exceptionally well-written proposed commit log message by a new contributor. > refs.c | 137 ++++++++++++--- > refs.h | 3 + > refs/debug.c | 25 --- > refs/files-backend.c | 276 +++++++++++++++++++++++++++---- > refs/packed-backend.c | 2 - > refs/refs-internal.h | 38 +++-- > refs/reftable-backend.c | 194 ++++++++++++++++------ > t/t1416-ref-transaction-hooks.sh | 142 ++++++++++++++++ > 8 files changed, 679 insertions(+), 138 deletions(-) > > diff --git a/refs.c b/refs.c > index 92d5df5b7..22c000f7f 100644 > --- a/refs.c > +++ b/refs.c > @@ -1004,15 +1004,17 @@ long get_files_ref_lock_timeout_ms(struct repository *repo) > return timeout_ms; > } > > -int refs_delete_ref(struct ref_store *refs, const char *msg, > - const char *refname, > - const struct object_id *old_oid, > - unsigned int flags) > +int refs_delete_ref_with_transaction_flags(struct ref_store *refs, > + const char *msg, > + const char *refname, > + const struct object_id *old_oid, > + unsigned int flags, > + unsigned int transaction_flags) > { > struct ref_transaction *transaction; > struct strbuf err = STRBUF_INIT; > > - transaction = ref_store_transaction_begin(refs, 0, &err); > + transaction = ref_store_transaction_begin(refs, transaction_flags, &err); > if (!transaction || > ref_transaction_delete(transaction, refname, old_oid, > NULL, flags, msg, &err) || > @@ -1027,6 +1029,15 @@ int refs_delete_ref(struct ref_store *refs, const char *msg, > return 0; > } > > +int refs_delete_ref(struct ref_store *refs, const char *msg, > + const char *refname, > + const struct object_id *old_oid, > + unsigned int flags) > +{ > + return refs_delete_ref_with_transaction_flags(refs, msg, refname, > + old_oid, flags, 0); > +} > + > static void copy_reflog_msg(struct strbuf *sb, const char *msg) > { > char c; > @@ -1270,6 +1281,10 @@ void ref_transaction_free(struct ref_transaction *transaction) > > string_list_clear(&transaction->refnames, 0); > free(transaction->updates); > + free(transaction->old_refname); > + free(transaction->new_refname); > + free(transaction->logmsg); > + free(transaction->destination_target); > free(transaction); > } > > @@ -2710,7 +2725,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, > return REF_TRANSACTION_ERROR_GENERIC; > > /* Preparing checks before locking references */ > - ret = run_transaction_hook(transaction, "preparing"); > + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : > + run_transaction_hook(transaction, "preparing"); > if (ret) { > ref_transaction_abort(transaction, err); > die(_(abort_by_ref_transaction_hook), "preparing"); > @@ -2720,7 +2736,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, > if (ret) > return ret; > > - ret = run_transaction_hook(transaction, "prepared"); > + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : > + run_transaction_hook(transaction, "prepared"); > if (ret) { > ref_transaction_abort(transaction, err); > die(_(abort_by_ref_transaction_hook), "prepared"); > @@ -2750,7 +2767,8 @@ int ref_transaction_abort(struct ref_transaction *transaction, > break; > } > > - run_transaction_hook(transaction, "aborted"); > + if (!(transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK)) > + run_transaction_hook(transaction, "aborted"); > > ref_transaction_free(transaction); > return ret; > @@ -2781,7 +2799,8 @@ int ref_transaction_commit(struct ref_transaction *transaction, > } > > ret = refs->be->transaction_finish(refs, transaction, err); > - if (!ret && !(transaction->flags & REF_TRANSACTION_FLAG_INITIAL)) > + if (!ret && !(transaction->flags & (REF_TRANSACTION_FLAG_INITIAL | > + REF_TRANSACTION_FLAG_SKIP_HOOK))) > run_transaction_hook(transaction, "committed"); > return ret; > } > @@ -3123,28 +3142,100 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg, > return ret; > } > > -int refs_rename_ref(struct ref_store *refs, const char *oldref, > - const char *newref, const char *logmsg) It is annoying that we have to give random callers an unrestricted way to skip calling hooks. I suspect it may come from "this function should call hook when invoked as the top-level operation, but when it is used as a subroutine for a different top-level operation, we want to skip hooks" kind of reasoning, but is this something we can avoid by rearranging the call chain? > +static int refs_copy_or_rename_ref(struct ref_store *refs, const char *oldref, > + const char *newref, const char *logmsg, > + int copy) Will this function ever gain a third mode of operation other than copy or rename? If not, perhaps "bool copy"? > { > - char *msg; > - int retval; > + struct ref_transaction *transaction = NULL; > + struct object_id old_oid, new_oid; > + struct strbuf new_target = STRBUF_INIT; > + struct strbuf err = STRBUF_INIT; > + char *msg = normalize_reflog_message(logmsg); > + int old_flags, new_flags = 0, new_exists = 0, ret = 1; > > - msg = normalize_reflog_message(logmsg); > - retval = refs->be->rename_ref(refs, oldref, newref, msg); > + if (!strcmp(oldref, newref)) { > + ret = 0; > + goto out; > + } > + > + if (!refs_resolve_ref_unsafe(refs, oldref, > + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, > + &old_oid, &old_flags)) { > + error("refname %s not found", oldref); > + goto out; > + } > + if (old_flags & REF_ISSYMREF) { > + error("refname %s is a symbolic ref, %s it is not supported", > + oldref, copy ? "copying" : "renaming"); > + goto out; > + } > + > + transaction = ref_store_transaction_begin(refs, 0, &err); > + if (!transaction) > + goto error; > + transaction->type = copy ? REF_TRANSACTION_TYPE_COPY : > + REF_TRANSACTION_TYPE_RENAME; > + transaction->old_refname = xstrdup(oldref); > + transaction->new_refname = xstrdup(newref); > + transaction->logmsg = xstrdup(msg); > + oidcpy(&transaction->source_oid, &old_oid); > + > + if (!copy && ref_transaction_delete(transaction, oldref, &old_oid, NULL, > + REF_NO_DEREF, msg, &err)) > + goto error; > + > + if (refs_resolve_ref_unsafe(refs, newref, > + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, > + &new_oid, &new_flags)) { > + new_exists = 1; > + if ((new_flags & REF_ISSYMREF) && > + refs_read_symbolic_ref(refs, newref, &new_target) < 0) { > + strbuf_addf(&err, "unable to read symbolic ref %s", newref); > + goto error; > + } > + } else { > + oidclr(&new_oid, refs->repo->hash_algo); > + } > + transaction->destination_exists = new_exists; > + if (new_flags & REF_ISSYMREF) > + transaction->destination_target = xstrdup(new_target.buf); > + else if (transaction->destination_exists) > + oidcpy(&transaction->destination_oid, &new_oid); > + > + if (ref_transaction_update(transaction, newref, &old_oid, > + (new_flags & REF_ISSYMREF) ? NULL : &new_oid, > + NULL, > + (new_flags & REF_ISSYMREF) ? new_target.buf : NULL, > + REF_NO_DEREF | REF_SKIP_CREATE_REFLOG, > + NULL, &err)) > + goto error; > + > + if (ref_transaction_commit(transaction, &err)) > + goto error; > + > + ret = 0; > + goto out; > + > +error: > + error("%s", err.buf); > +out: > + ref_transaction_free(transaction); > + strbuf_release(&new_target); > + strbuf_release(&err); > free(msg); > - return retval; > + return ret; > } That's quite a lot of new code. I see ref_transaction_delete(), ref_transaction_update() and others are already reused from existing code paths, which is good. > +struct files_copy_or_rename_transaction_data { > + struct ref_lock *lock; > + struct object_id orig_oid; > + struct object_id destination_oid; > + char *destination_target; > + int logmoved; > + int destination_exists; > + int destination_log_backed_up; > +}; Good to have a type that can be used to hold pieces of information specific to the operation. Can't we do without rename/copy specific addition to the generic ref_transaction struct by following the same principle? The comment above the members does make it understandable, but ... > @@ -240,6 +253,21 @@ struct ref_transaction { > void *backend_data; > unsigned int flags; > uint64_t max_index; > + > + /* > + * Rename and copy operations need backend-specific reflog handling. > + * Their logical updates still live in `updates`, so hooks see the > + * operation like any other reference transaction. The fields below > + * retain the state that backends verify after taking their locks. > + */ > + enum ref_transaction_type type; > + char *old_refname; > + char *new_refname; > + char *logmsg; > + struct object_id source_oid; > + struct object_id destination_oid; > + char *destination_target; > + unsigned int destination_exists:1; > }; ... is it the best we can do to contaminate a rather generic data structure for such a details relevant only to one specific operation? Thanks. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH] refs: run copy and rename through transactions 2026-09-21 17:54 ` Junio C Hamano @ 2026-09-21 23:28 ` Junio C Hamano 2026-09-22 13:08 ` Maciej Ciemborowicz 0 siblings, 1 reply; 35+ messages in thread From: Junio C Hamano @ 2026-09-21 23:28 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: git, Karthik Nayak Junio C Hamano <gitster@pobox.com> writes: >> +struct files_copy_or_rename_transaction_data { >> + struct ref_lock *lock; >> + struct object_id orig_oid; >> + struct object_id destination_oid; >> + char *destination_target; >> + int logmoved; >> + int destination_exists; >> + int destination_log_backed_up; >> +}; > > Good to have a type that can be used to hold pieces of information > specific to the operation. Can't we do without rename/copy specific > addition to the generic ref_transaction struct by following the same > principle? > > The comment above the members does make it understandable, but ... > >> @@ -240,6 +253,21 @@ struct ref_transaction { >> void *backend_data; >> unsigned int flags; >> uint64_t max_index; >> + >> + /* >> + * Rename and copy operations need backend-specific reflog handling. >> + * Their logical updates still live in `updates`, so hooks see the >> + * operation like any other reference transaction. The fields below >> + * retain the state that backends verify after taking their locks. >> + */ >> + enum ref_transaction_type type; >> + char *old_refname; >> + char *new_refname; >> + char *logmsg; >> + struct object_id source_oid; >> + struct object_id destination_oid; >> + char *destination_target; >> + unsigned int destination_exists:1; >> }; > > ... is it the best we can do to contaminate a rather generic data > structure for such a details relevant only to one specific > operation? More importantly, this structure suggests to me that you can have a single rename (or copy) from one source to one destination in a single transaction. Is that correct or am I misunderstanding the way this data structure is used? How would one rename A, B and C to X, Y and Z in a single transaction? Or perhaps rename A to B and copy C to D in a single transaction? ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH] refs: run copy and rename through transactions 2026-09-21 23:28 ` Junio C Hamano @ 2026-09-22 13:08 ` Maciej Ciemborowicz 0 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-09-22 13:08 UTC (permalink / raw) To: Junio C Hamano; +Cc: git, Karthik Nayak On Tue, Sep 22, 2026 at 1:28 AM Junio C Hamano <gitster@pobox.com> wrote: > More importantly, this structure suggests to me that you can have a > single rename (or copy) from one source to one destination in a > single transaction. Is that correct or am I misunderstanding the > way this data structure is used? How would one rename A, B and C > to X, Y and Z in a single transaction? Or perhaps rename A to B > and copy C to D in a single transaction? Good point. The current design indeed makes copy/rename a property of the whole transacton, so it does not compose with multiple such operations. I'll rework it so that state belongs to individual updates instead. Unfortunately, I'll have to put this aside for a few days until I finish this first: https://lore.kernel.org/git/CACQ=SRHthWOLVXmY6wgknOPgpQ+oB1vV-Q0AL=mK9mXb2Xy9Nw@mail.gmail.com/T/#t Thanks, Maciej Ciemborowicz ^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2] refs: run copy and rename through transactions 2026-09-20 16:50 ` [PATCH] refs: run copy and rename through transactions Maciej Ciemborowicz 2026-09-21 17:54 ` Junio C Hamano @ 2026-09-23 13:36 ` Maciej Ciemborowicz 2026-09-30 3:32 ` Maciej Ciemborowicz 2026-10-02 10:56 ` Patrick Steinhardt 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz 3 siblings, 2 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-09-23 13:36 UTC (permalink / raw) To: git; +Cc: Maciej Ciemborowicz, gitster, karthik.188 Reference copy and rename operations bypass the transaction API. Consequently, the reference-transaction hook sees only the source deletion with the files backend and no useful update with the reftable backend. Represent both operations as reference transactions containing their logical updates. A rename is a deletion of the old reference and creation of the new reference in the same transaction. Attach operation-specific state to the destination update instead of making copy or rename a property of the entire transaction. Retain backend-specific reflog handling: the files backend stages its existing rename procedure across prepare, finish and abort, while reftable stages an addition while holding the stack lock. Suppress hooks for the files backend's nested deletion transactions so that callers observe one logical transaction. Record and verify the source and destination values after taking backend locks. This rejects concurrent changes instead of applying a rename or copy that differs from the payload shown to the preparing hook. Preserve D/F renames and restore overwritten references and reflogs when a prepared hook rejects the operation. Add tests covering rename, copy, forced updates, both directions of D/F conflicts, concurrent updates and prepared-hook rollback. Helped-by: Karthik Nayak <karthik.188@gmail.com> Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- Apologies for the unrelated Subject header on my earlier reply. This reroll incorporates the points discussed there. Changes since v1: * Keep copy/rename state on the destination ref update instead of the generic transaction, so the operation is no longer a transaction-wide property. * Keep REF_TRANSACTION_FLAG_SKIP_HOOK private to the refs implementation. * Use bool for the two-state copy parameter. * Apply Junio's commit-message wording suggestions. Range-diff against v1: 1: de0a5a9f7 ! 1: d852537d8 refs: run copy and rename through transactions @@ Metadata ## Commit message ## refs: run copy and rename through transactions - Reference copy and rename operations currently bypass the transaction API. + Reference copy and rename operations bypass the transaction API. Consequently, the reference-transaction hook sees only the source deletion with the files backend and no useful update with the reftable backend. Represent both operations as reference transactions containing their logical updates. A rename is a deletion of the old reference and creation - of the new reference in the same transaction. Retain backend-specific - reflog handling: the files backend stages its existing rename procedure - across prepare, finish and abort, while reftable stages an addition while - holding the stack lock. Suppress hooks for the files backend's nested - deletion transactions so that callers observe one logical transaction. + of the new reference in the same transaction. Attach operation-specific + state to the destination update instead of making copy or rename a property + of the entire transaction. + + Retain backend-specific reflog handling: the files backend stages its + existing rename procedure across prepare, finish and abort, while reftable + stages an addition while holding the stack lock. Suppress hooks for the + files backend's nested deletion transactions so that callers observe one + logical transaction. Record and verify the source and destination values after taking backend locks. This rejects concurrent changes instead of applying a rename or copy @@ Commit message renames and restore overwritten references and reflogs when a prepared hook rejects the operation. - Add coverage for rename, copy, forced updates, both directions of D/F + Add tests covering rename, copy, forced updates, both directions of D/F conflicts, concurrent updates and prepared-hook rollback. Helped-by: Karthik Nayak <karthik.188@gmail.com> @@ refs.c: int refs_delete_ref(struct ref_store *refs, const char *msg, { char c; @@ refs.c: void ref_transaction_free(struct ref_transaction *transaction) + } + + for (i = 0; i < transaction->nr; i++) { ++ struct ref_copy_or_rename_update *operation = ++ transaction->updates[i]->copy_or_rename; ++ + free(transaction->updates[i]->msg); + free(transaction->updates[i]->committer_info); + free((char *)transaction->updates[i]->new_target); + free((char *)transaction->updates[i]->old_target); + free((char *)transaction->updates[i]->rejection_details); ++ if (operation) { ++ free(operation->old_refname); ++ free(operation->logmsg); ++ free(operation->destination_target); ++ free(operation); ++ } + free(transaction->updates[i]); + } - string_list_clear(&transaction->refnames, 0); - free(transaction->updates); -+ free(transaction->old_refname); -+ free(transaction->new_refname); -+ free(transaction->logmsg); -+ free(transaction->destination_target); +@@ refs.c: void ref_transaction_free(struct ref_transaction *transaction) free(transaction); } ++struct ref_update *ref_transaction_copy_or_rename_update( ++ struct ref_transaction *transaction) ++{ ++ struct ref_update *operation = NULL; ++ size_t i; ++ ++ for (i = 0; i < transaction->nr; i++) { ++ if (!transaction->updates[i]->copy_or_rename) ++ continue; ++ if (operation) ++ BUG("multiple copy or rename updates in one transaction"); ++ operation = transaction->updates[i]; ++ } ++ ++ return operation; ++} ++ + int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction, + size_t update_idx, + enum ref_transaction_error err, @@ refs.c: int ref_transaction_prepare(struct ref_transaction *transaction, return REF_TRANSACTION_ERROR_GENERIC; @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, - const char *newref, const char *logmsg) +static int refs_copy_or_rename_ref(struct ref_store *refs, const char *oldref, + const char *newref, const char *logmsg, -+ int copy) ++ bool copy) { - char *msg; - int retval; + struct ref_transaction *transaction = NULL; ++ struct ref_copy_or_rename_update *operation = NULL; ++ struct ref_update *destination_update; + struct object_id old_oid, new_oid; + struct strbuf new_target = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, + transaction = ref_store_transaction_begin(refs, 0, &err); + if (!transaction) + goto error; -+ transaction->type = copy ? REF_TRANSACTION_TYPE_COPY : -+ REF_TRANSACTION_TYPE_RENAME; -+ transaction->old_refname = xstrdup(oldref); -+ transaction->new_refname = xstrdup(newref); -+ transaction->logmsg = xstrdup(msg); -+ oidcpy(&transaction->source_oid, &old_oid); -+ + if (!copy && ref_transaction_delete(transaction, oldref, &old_oid, NULL, + REF_NO_DEREF, msg, &err)) + goto error; @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, + } else { + oidclr(&new_oid, refs->repo->hash_algo); + } -+ transaction->destination_exists = new_exists; -+ if (new_flags & REF_ISSYMREF) -+ transaction->destination_target = xstrdup(new_target.buf); -+ else if (transaction->destination_exists) -+ oidcpy(&transaction->destination_oid, &new_oid); -+ + if (ref_transaction_update(transaction, newref, &old_oid, + (new_flags & REF_ISSYMREF) ? NULL : &new_oid, + NULL, @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, + NULL, &err)) + goto error; + ++ destination_update = transaction->updates[transaction->nr - 1]; ++ CALLOC_ARRAY(operation, 1); ++ operation->type = copy ? REF_UPDATE_COPY : REF_UPDATE_RENAME; ++ operation->old_refname = xstrdup(oldref); ++ operation->logmsg = xstrdup(msg); ++ oidcpy(&operation->source_oid, &old_oid); ++ operation->destination_exists = new_exists; ++ if (new_flags & REF_ISSYMREF) ++ operation->destination_target = xstrdup(new_target.buf); ++ else if (operation->destination_exists) ++ oidcpy(&operation->destination_oid, &new_oid); ++ destination_update->copy_or_rename = operation; ++ + if (ref_transaction_commit(transaction, &err)) + goto error; + @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, const char *ref_update_original_update_refname(struct ref_update *update) - ## refs.h ## -@@ refs.h: enum ref_transaction_flag { - * while rejecting updates which do not match the expected state. - */ - REF_TRANSACTION_ALLOW_FAILURE = (1 << 1), -+ -+ /* Suppress hooks for an update nested in another transaction. */ -+ REF_TRANSACTION_FLAG_SKIP_HOOK = (1 << 2), - }; - - /* - ## refs/debug.c ## @@ refs/debug.c: static int debug_optimize_required(struct ref_store *ref_store, return res; @@ refs/files-backend.c: static int refs_rename_ref_available(struct ref_store *ref +}; + static int files_copy_or_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, +- const char *oldrefname, const char *newrefname, - const char *logmsg, int copy) -+ const char *logmsg, int copy, -+ struct ref_transaction *transaction) ++ struct ref_update *update, ++ struct ref_transaction *transaction) { struct files_ref_store *refs = - files_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); + files_downcast(ref_store, REF_STORE_WRITE, + "ref_transaction_prepare"); ++ struct ref_copy_or_rename_update *operation = update->copy_or_rename; ++ const char *oldrefname = operation->old_refname; ++ const char *newrefname = update->refname; ++ const char *logmsg = operation->logmsg; ++ bool copy = operation->type == REF_UPDATE_COPY; struct object_id orig_oid; int flag = 0, logmoved = 0; struct ref_lock *lock; @@ refs/files-backend.c: static int files_copy_or_rename_ref(struct ref_store *ref_ oldrefname); goto out; } -+ if (!oideq(&orig_oid, &transaction->source_oid)) { ++ if (!oideq(&orig_oid, &operation->source_oid)) { + ret = error("refname %s is at %s but expected %s", + oldrefname, oid_to_hex(&orig_oid), -+ oid_to_hex(&transaction->source_oid)); ++ oid_to_hex(&operation->source_oid)); + goto out; + } if (!refs_rename_ref_available(&refs->base, oldrefname, newrefname)) { @@ refs/files-backend.c: static int files_copy_or_rename_ref(struct ref_store *ref_ + goto out; + } + } -+ if (destination_exists != transaction->destination_exists) { ++ if (destination_exists != operation->destination_exists) { + ret = error("refname %s changed while renaming", newrefname); + goto out; + } + if (destination_exists) { + if (destination_flags & REF_ISSYMREF) { -+ if (!transaction->destination_target || ++ if (!operation->destination_target || + strcmp(destination_target.buf, -+ transaction->destination_target)) { ++ operation->destination_target)) { + ret = error("refname %s changed while renaming", + newrefname); + goto out; + } -+ } else if (transaction->destination_target || ++ } else if (operation->destination_target || + !oideq(&destination_oid, -+ &transaction->destination_oid)) { ++ &operation->destination_oid)) { + ret = error("refname %s changed while renaming", newrefname); + goto out; + } @@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref struct ref_transaction *packed_transaction = NULL; assert(err); -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) -+ return files_copy_or_rename_ref(ref_store, -+ transaction->old_refname, -+ transaction->new_refname, -+ transaction->logmsg, -+ transaction->type == REF_TRANSACTION_TYPE_COPY, -+ transaction); ++ { ++ struct ref_update *operation = ++ ref_transaction_copy_or_rename_update(transaction); ++ ++ if (operation) ++ return files_copy_or_rename_ref(ref_store, operation, ++ transaction); ++ } if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) goto cleanup; @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_ assert(err); -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { -+ struct files_copy_or_rename_transaction_data *data = -+ transaction->backend_data; -+ int special_ret; ++ { ++ struct ref_update *update = ++ ref_transaction_copy_or_rename_update(transaction); + -+ special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, -+ transaction->logmsg, 0, err); -+ if (special_ret) { -+ error("unable to write current sha1 into %s: %s", -+ transaction->new_refname, err->buf); -+ data->lock = NULL; -+ files_transaction_abort(ref_store, transaction, err); -+ return special_ret; -+ } else if (data->destination_log_backed_up) { -+ struct strbuf path = STRBUF_INIT; ++ if (update) { ++ struct ref_copy_or_rename_update *operation = ++ update->copy_or_rename; ++ struct files_copy_or_rename_transaction_data *data = ++ transaction->backend_data; ++ int special_ret; ++ ++ special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, ++ operation->logmsg, 0, err); ++ if (special_ret) { ++ error("unable to write current sha1 into %s: %s", ++ update->refname, err->buf); ++ data->lock = NULL; ++ files_transaction_abort(ref_store, transaction, err); ++ return special_ret; ++ } else if (data->destination_log_backed_up) { ++ struct strbuf path = STRBUF_INIT; + -+ files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); -+ if (unlink(path.buf) < 0 && errno != ENOENT) -+ warning_errno("unable to remove '%s'", path.buf); -+ strbuf_release(&path); ++ files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); ++ if (unlink(path.buf) < 0 && errno != ENOENT) ++ warning_errno("unable to remove '%s'", path.buf); ++ strbuf_release(&path); ++ } ++ free(data->destination_target); ++ free(data); ++ transaction->backend_data = NULL; ++ transaction->state = REF_TRANSACTION_CLOSED; ++ return special_ret; + } -+ free(data->destination_target); -+ free(data); -+ transaction->backend_data = NULL; -+ transaction->state = REF_TRANSACTION_CLOSED; -+ return special_ret; + } if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_ struct files_ref_store *refs = files_downcast(ref_store, 0, "ref_transaction_abort"); -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { -+ struct files_copy_or_rename_transaction_data *data = -+ transaction->backend_data; -+ struct strbuf new_log = STRBUF_INIT; -+ struct strbuf destination_log = STRBUF_INIT; -+ struct strbuf temporary_log = STRBUF_INIT; -+ struct ref_transaction *restore_transaction = NULL; -+ struct ref_lock *lock; -+ int ret = 0; ++ { ++ struct ref_update *update = ++ ref_transaction_copy_or_rename_update(transaction); + -+ if (data->lock) -+ unlock_ref(data->lock); -+ if (transaction->type == REF_TRANSACTION_TYPE_RENAME) { -+ lock = lock_ref_oid_basic(refs, transaction->old_refname, err); -+ if (!lock || -+ write_ref_to_lockfile(refs, lock, &data->orig_oid, err) || -+ commit_ref_update(refs, lock, &data->orig_oid, NULL, -+ REF_SKIP_CREATE_REFLOG, err)) -+ ret = -1; -+ } ++ if (update) { ++ struct ref_copy_or_rename_update *operation = ++ update->copy_or_rename; ++ struct files_copy_or_rename_transaction_data *data = ++ transaction->backend_data; ++ struct strbuf new_log = STRBUF_INIT; ++ struct strbuf destination_log = STRBUF_INIT; ++ struct strbuf temporary_log = STRBUF_INIT; ++ struct ref_transaction *restore_transaction = NULL; ++ struct ref_lock *lock; ++ int ret = 0; + -+ if (data->logmoved) { -+ files_reflog_path(refs, &new_log, transaction->new_refname); -+ if (transaction->type == REF_TRANSACTION_TYPE_RENAME) { -+ files_reflog_path(refs, &temporary_log, TMP_RENAMED_LOG); -+ if (rename(new_log.buf, temporary_log.buf) < 0) { -+ strbuf_addf(err, "unable to restore logfile %s: %s", -+ transaction->old_refname, strerror(errno)); ++ if (data->lock) ++ unlock_ref(data->lock); ++ if (operation->type == REF_UPDATE_RENAME) { ++ lock = lock_ref_oid_basic(refs, operation->old_refname, err); ++ if (!lock || ++ write_ref_to_lockfile(refs, lock, &data->orig_oid, err) || ++ commit_ref_update(refs, lock, &data->orig_oid, NULL, ++ REF_SKIP_CREATE_REFLOG, err)) + ret = -1; -+ } else { -+ try_remove_empty_parents(refs, -+ transaction->new_refname, -+ REMOVE_EMPTY_PARENTS_REFLOG); -+ if (rename_tmp_log(refs, -+ transaction->old_refname)) { ++ } ++ ++ if (data->logmoved) { ++ files_reflog_path(refs, &new_log, update->refname); ++ if (operation->type == REF_UPDATE_RENAME) { ++ files_reflog_path(refs, &temporary_log, TMP_RENAMED_LOG); ++ if (rename(new_log.buf, temporary_log.buf) < 0) { + strbuf_addf(err, "unable to restore logfile %s: %s", -+ transaction->old_refname, -+ strerror(errno)); ++ operation->old_refname, strerror(errno)); + ret = -1; ++ } else { ++ try_remove_empty_parents(refs, ++ update->refname, ++ REMOVE_EMPTY_PARENTS_REFLOG); ++ if (rename_tmp_log(refs, ++ operation->old_refname)) { ++ strbuf_addf(err, "unable to restore logfile %s: %s", ++ operation->old_refname, ++ strerror(errno)); ++ ret = -1; ++ } + } ++ } else if (unlink(new_log.buf) < 0 && errno != ENOENT) { ++ strbuf_addf(err, "unable to remove logfile %s: %s", ++ update->refname, strerror(errno)); ++ ret = -1; + } -+ } else if (unlink(new_log.buf) < 0 && errno != ENOENT) { -+ strbuf_addf(err, "unable to remove logfile %s: %s", -+ transaction->new_refname, strerror(errno)); -+ ret = -1; + } -+ } -+ if (data->destination_log_backed_up) { -+ files_reflog_path(refs, &destination_log, -+ TMP_RENAMED_LOG_DESTINATION); -+ if (rename(destination_log.buf, new_log.buf) < 0) { -+ strbuf_addf(err, "unable to restore logfile %s: %s", -+ transaction->new_refname, strerror(errno)); -+ ret = -1; ++ if (data->destination_log_backed_up) { ++ files_reflog_path(refs, &destination_log, ++ TMP_RENAMED_LOG_DESTINATION); ++ if (rename(destination_log.buf, new_log.buf) < 0) { ++ strbuf_addf(err, "unable to restore logfile %s: %s", ++ update->refname, strerror(errno)); ++ ret = -1; ++ } + } -+ } + -+ if (transaction->type == REF_TRANSACTION_TYPE_RENAME && -+ data->destination_exists) { -+ restore_transaction = ref_store_transaction_begin( ++ if (operation->type == REF_UPDATE_RENAME && ++ data->destination_exists) { ++ restore_transaction = ref_store_transaction_begin( + &refs->base, REF_TRANSACTION_FLAG_SKIP_HOOK, err); -+ if (!restore_transaction || -+ ref_transaction_update(restore_transaction, -+ transaction->new_refname, -+ data->destination_target ? NULL : -+ &data->destination_oid, -+ NULL, -+ data->destination_target, -+ NULL, -+ REF_NO_DEREF | -+ REF_SKIP_CREATE_REFLOG, -+ NULL, err) || -+ ref_transaction_commit(restore_transaction, err)) -+ ret = -1; -+ ref_transaction_free(restore_transaction); -+ } ++ if (!restore_transaction || ++ ref_transaction_update(restore_transaction, ++ update->refname, ++ data->destination_target ? NULL : ++ &data->destination_oid, ++ NULL, ++ data->destination_target, ++ NULL, ++ REF_NO_DEREF | ++ REF_SKIP_CREATE_REFLOG, ++ NULL, err) || ++ ref_transaction_commit(restore_transaction, err)) ++ ret = -1; ++ ref_transaction_free(restore_transaction); ++ } + -+ strbuf_release(&destination_log); -+ strbuf_release(&temporary_log); -+ strbuf_release(&new_log); -+ free(data->destination_target); -+ free(data); -+ transaction->backend_data = NULL; -+ transaction->state = REF_TRANSACTION_CLOSED; -+ return ret; ++ strbuf_release(&destination_log); ++ strbuf_release(&temporary_log); ++ strbuf_release(&new_log); ++ free(data->destination_target); ++ free(data); ++ transaction->backend_data = NULL; ++ transaction->state = REF_TRANSACTION_CLOSED; ++ return ret; ++ } + } + files_transaction_cleanup(refs, transaction); @@ refs/packed-backend.c: struct ref_storage_be refs_be_packed = { .read_raw_ref = packed_read_raw_ref, ## refs/refs-internal.h ## +@@ refs/refs-internal.h: struct ref_update { + */ + struct ref_update *parent_update; + ++ /* ++ * Copy and rename operations require backend-specific handling while ++ * still exposing their logical updates to transaction hooks. Keep that ++ * state on the destination update so it composes with other updates in ++ * the transaction instead of making copy or rename a transaction-wide ++ * property. ++ */ ++ struct ref_copy_or_rename_update *copy_or_rename; ++ + const char refname[FLEX_ARRAY]; + }; + ++enum ref_copy_or_rename_type { ++ REF_UPDATE_RENAME, ++ REF_UPDATE_COPY, ++}; ++ ++struct ref_copy_or_rename_update { ++ enum ref_copy_or_rename_type type; ++ char *old_refname; ++ char *logmsg; ++ struct object_id source_oid; ++ struct object_id destination_oid; ++ char *destination_target; ++ unsigned int destination_exists:1; ++}; ++ + int refs_read_raw_ref(struct ref_store *ref_store, const char *refname, + struct object_id *oid, struct strbuf *referent, + unsigned int *type, int *failure_errno); @@ refs/refs-internal.h: struct ref_update *ref_transaction_add_update( const char *committer_info, const char *msg); @@ refs/refs-internal.h: struct ref_update *ref_transaction_add_update( /* * Transaction states. * -@@ refs/refs-internal.h: enum ref_transaction_state { - REF_TRANSACTION_CLOSED = 2 - }; - -+enum ref_transaction_type { -+ REF_TRANSACTION_TYPE_NORMAL = 0, -+ REF_TRANSACTION_TYPE_RENAME, -+ REF_TRANSACTION_TYPE_COPY, -+}; -+ - /* - * Data structure to hold indices of updates which were rejected, for batched - * reference updates. While the updates themselves hold the rejection error, @@ refs/refs-internal.h: struct ref_transaction { - void *backend_data; - unsigned int flags; uint64_t max_index; -+ -+ /* -+ * Rename and copy operations need backend-specific reflog handling. -+ * Their logical updates still live in `updates`, so hooks see the -+ * operation like any other reference transaction. The fields below -+ * retain the state that backends verify after taking their locks. -+ */ -+ enum ref_transaction_type type; -+ char *old_refname; -+ char *new_refname; -+ char *logmsg; -+ struct object_id source_oid; -+ struct object_id destination_oid; -+ char *destination_target; -+ unsigned int destination_exists:1; }; ++/* Suppress hooks for a transaction nested inside another refs operation. */ ++#define REF_TRANSACTION_FLAG_SKIP_HOOK (1 << 2) ++ ++struct ref_update *ref_transaction_copy_or_rename_update( ++ struct ref_transaction *transaction); ++ /* + * Check for entries in extras that are within the specified + * directory, where dirname is a reference directory name including @@ refs/refs-internal.h: typedef int optimize_required_fn(struct ref_store *ref_store, struct refs_optimize_opts *opts, bool *required); @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s size_t i; int ret; -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) ++ if (ref_transaction_copy_or_rename_update(transaction)) + return reftable_be_copy_or_rename_prepare(ref_store, transaction, + err); + @@ refs/reftable-backend.c: static int reftable_be_transaction_abort(struct ref_sto - struct reftable_transaction_data *tx_data = transaction->backend_data; + struct reftable_transaction_data *tx_data; + -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { ++ if (ref_transaction_copy_or_rename_update(transaction)) { + struct reftable_copy_or_rename_transaction_data *data = + transaction->backend_data; + @@ refs/reftable-backend.c: static int reftable_be_transaction_finish(struct ref_st + struct reftable_transaction_data *tx_data; int ret = 0; -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { ++ if (ref_transaction_copy_or_rename_update(transaction)) { + struct reftable_copy_or_rename_transaction_data *data = + transaction->backend_data; + int special_ret = reftable_addition_commit(data->addition); @@ refs/reftable-backend.c: struct write_create_symref_arg { const char *newname; const char *logmsg; int delete_old; -+ struct ref_transaction *transaction; ++ struct ref_copy_or_rename_update *operation; }; static int write_copy_table(struct reftable_writer *writer, void *cb_data) @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri + ret = -1; goto done; } -+ if (arg->transaction) { ++ if (arg->operation) { + struct object_id oid; + + if (old_ref.value_type == REFTABLE_REF_VAL2) @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri + else + oidread(&oid, old_ref.value.val1, + arg->refs->base.repo->hash_algo); -+ if (!oideq(&oid, &arg->transaction->source_oid)) { ++ if (!oideq(&oid, &arg->operation->source_oid)) { + strbuf_addf(arg->err, + _("refname %s is at %s but expected %s"), + arg->oldname, oid_to_hex(&oid), -+ oid_to_hex(&arg->transaction->source_oid)); ++ oid_to_hex(&arg->operation->source_oid)); + ret = -1; + goto done; + } @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri + &destination_ref); + if (ret < 0) + goto done; -+ if (arg->transaction->destination_exists != !ret) { ++ if (arg->operation->destination_exists != !ret) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri + } + if (!ret) { + if (destination_ref.value_type == REFTABLE_REF_SYMREF) { -+ if (!arg->transaction->destination_target || ++ if (!arg->operation->destination_target || + strcmp(destination_ref.value.symref, -+ arg->transaction->destination_target)) { ++ arg->operation->destination_target)) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri + else + oidread(&oid, destination_ref.value.val1, + arg->refs->base.repo->hash_algo); -+ if (arg->transaction->destination_target || -+ !oideq(&oid, &arg->transaction->destination_oid)) { ++ if (arg->operation->destination_target || ++ !oideq(&oid, &arg->operation->destination_oid)) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri + reftable_be_downcast(ref_store, REF_STORE_WRITE, + "ref_transaction_prepare"); + struct reftable_copy_or_rename_transaction_data *data = NULL; ++ struct ref_update *update = ++ ref_transaction_copy_or_rename_update(transaction); ++ struct ref_copy_or_rename_update *operation = update->copy_or_rename; struct write_copy_arg arg = { .refs = refs, - .oldname = oldrefname, @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri - .logmsg = logmsg, - .delete_old = 1, + .err = err, -+ .oldname = transaction->old_refname, -+ .newname = transaction->new_refname, -+ .logmsg = transaction->logmsg, -+ .delete_old = transaction->type == REF_TRANSACTION_TYPE_RENAME, -+ .transaction = transaction, ++ .oldname = operation->old_refname, ++ .newname = update->refname, ++ .logmsg = operation->logmsg, ++ .delete_old = operation->type == REF_UPDATE_RENAME, ++ .operation = operation, }; int ret; @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); -+ ret = backend_for(&arg.be, refs, transaction->new_refname, ++ ret = backend_for(&arg.be, refs, update->refname, + &arg.newname, 1); if (ret) goto done; refs.c | 161 ++++++++++++++--- refs/debug.c | 25 --- refs/files-backend.c | 297 +++++++++++++++++++++++++++---- refs/packed-backend.c | 2 - refs/refs-internal.h | 47 +++-- refs/reftable-backend.c | 197 +++++++++++++++----- t/t1416-ref-transaction-hooks.sh | 142 +++++++++++++++ 7 files changed, 732 insertions(+), 139 deletions(-) diff --git a/refs.c b/refs.c index 92d5df5b7..f036ae4b9 100644 --- a/refs.c +++ b/refs.c @@ -1004,15 +1004,17 @@ long get_files_ref_lock_timeout_ms(struct repository *repo) return timeout_ms; } -int refs_delete_ref(struct ref_store *refs, const char *msg, - const char *refname, - const struct object_id *old_oid, - unsigned int flags) +int refs_delete_ref_with_transaction_flags(struct ref_store *refs, + const char *msg, + const char *refname, + const struct object_id *old_oid, + unsigned int flags, + unsigned int transaction_flags) { struct ref_transaction *transaction; struct strbuf err = STRBUF_INIT; - transaction = ref_store_transaction_begin(refs, 0, &err); + transaction = ref_store_transaction_begin(refs, transaction_flags, &err); if (!transaction || ref_transaction_delete(transaction, refname, old_oid, NULL, flags, msg, &err) || @@ -1027,6 +1029,15 @@ int refs_delete_ref(struct ref_store *refs, const char *msg, return 0; } +int refs_delete_ref(struct ref_store *refs, const char *msg, + const char *refname, + const struct object_id *old_oid, + unsigned int flags) +{ + return refs_delete_ref_with_transaction_flags(refs, msg, refname, + old_oid, flags, 0); +} + static void copy_reflog_msg(struct strbuf *sb, const char *msg) { char c; @@ -1256,11 +1267,20 @@ void ref_transaction_free(struct ref_transaction *transaction) } for (i = 0; i < transaction->nr; i++) { + struct ref_copy_or_rename_update *operation = + transaction->updates[i]->copy_or_rename; + free(transaction->updates[i]->msg); free(transaction->updates[i]->committer_info); free((char *)transaction->updates[i]->new_target); free((char *)transaction->updates[i]->old_target); free((char *)transaction->updates[i]->rejection_details); + if (operation) { + free(operation->old_refname); + free(operation->logmsg); + free(operation->destination_target); + free(operation); + } free(transaction->updates[i]); } @@ -1273,6 +1293,23 @@ void ref_transaction_free(struct ref_transaction *transaction) free(transaction); } +struct ref_update *ref_transaction_copy_or_rename_update( + struct ref_transaction *transaction) +{ + struct ref_update *operation = NULL; + size_t i; + + for (i = 0; i < transaction->nr; i++) { + if (!transaction->updates[i]->copy_or_rename) + continue; + if (operation) + BUG("multiple copy or rename updates in one transaction"); + operation = transaction->updates[i]; + } + + return operation; +} + int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction, size_t update_idx, enum ref_transaction_error err, @@ -2710,7 +2747,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, return REF_TRANSACTION_ERROR_GENERIC; /* Preparing checks before locking references */ - ret = run_transaction_hook(transaction, "preparing"); + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : + run_transaction_hook(transaction, "preparing"); if (ret) { ref_transaction_abort(transaction, err); die(_(abort_by_ref_transaction_hook), "preparing"); @@ -2720,7 +2758,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, if (ret) return ret; - ret = run_transaction_hook(transaction, "prepared"); + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : + run_transaction_hook(transaction, "prepared"); if (ret) { ref_transaction_abort(transaction, err); die(_(abort_by_ref_transaction_hook), "prepared"); @@ -2750,7 +2789,8 @@ int ref_transaction_abort(struct ref_transaction *transaction, break; } - run_transaction_hook(transaction, "aborted"); + if (!(transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK)) + run_transaction_hook(transaction, "aborted"); ref_transaction_free(transaction); return ret; @@ -2781,7 +2821,8 @@ int ref_transaction_commit(struct ref_transaction *transaction, } ret = refs->be->transaction_finish(refs, transaction, err); - if (!ret && !(transaction->flags & REF_TRANSACTION_FLAG_INITIAL)) + if (!ret && !(transaction->flags & (REF_TRANSACTION_FLAG_INITIAL | + REF_TRANSACTION_FLAG_SKIP_HOOK))) run_transaction_hook(transaction, "committed"); return ret; } @@ -3123,28 +3164,102 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg, return ret; } -int refs_rename_ref(struct ref_store *refs, const char *oldref, - const char *newref, const char *logmsg) +static int refs_copy_or_rename_ref(struct ref_store *refs, const char *oldref, + const char *newref, const char *logmsg, + bool copy) { - char *msg; - int retval; + struct ref_transaction *transaction = NULL; + struct ref_copy_or_rename_update *operation = NULL; + struct ref_update *destination_update; + struct object_id old_oid, new_oid; + struct strbuf new_target = STRBUF_INIT; + struct strbuf err = STRBUF_INIT; + char *msg = normalize_reflog_message(logmsg); + int old_flags, new_flags = 0, new_exists = 0, ret = 1; - msg = normalize_reflog_message(logmsg); - retval = refs->be->rename_ref(refs, oldref, newref, msg); + if (!strcmp(oldref, newref)) { + ret = 0; + goto out; + } + + if (!refs_resolve_ref_unsafe(refs, oldref, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, + &old_oid, &old_flags)) { + error("refname %s not found", oldref); + goto out; + } + if (old_flags & REF_ISSYMREF) { + error("refname %s is a symbolic ref, %s it is not supported", + oldref, copy ? "copying" : "renaming"); + goto out; + } + + transaction = ref_store_transaction_begin(refs, 0, &err); + if (!transaction) + goto error; + if (!copy && ref_transaction_delete(transaction, oldref, &old_oid, NULL, + REF_NO_DEREF, msg, &err)) + goto error; + + if (refs_resolve_ref_unsafe(refs, newref, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, + &new_oid, &new_flags)) { + new_exists = 1; + if ((new_flags & REF_ISSYMREF) && + refs_read_symbolic_ref(refs, newref, &new_target) < 0) { + strbuf_addf(&err, "unable to read symbolic ref %s", newref); + goto error; + } + } else { + oidclr(&new_oid, refs->repo->hash_algo); + } + if (ref_transaction_update(transaction, newref, &old_oid, + (new_flags & REF_ISSYMREF) ? NULL : &new_oid, + NULL, + (new_flags & REF_ISSYMREF) ? new_target.buf : NULL, + REF_NO_DEREF | REF_SKIP_CREATE_REFLOG, + NULL, &err)) + goto error; + + destination_update = transaction->updates[transaction->nr - 1]; + CALLOC_ARRAY(operation, 1); + operation->type = copy ? REF_UPDATE_COPY : REF_UPDATE_RENAME; + operation->old_refname = xstrdup(oldref); + operation->logmsg = xstrdup(msg); + oidcpy(&operation->source_oid, &old_oid); + operation->destination_exists = new_exists; + if (new_flags & REF_ISSYMREF) + operation->destination_target = xstrdup(new_target.buf); + else if (operation->destination_exists) + oidcpy(&operation->destination_oid, &new_oid); + destination_update->copy_or_rename = operation; + + if (ref_transaction_commit(transaction, &err)) + goto error; + + ret = 0; + goto out; + +error: + error("%s", err.buf); +out: + ref_transaction_free(transaction); + strbuf_release(&new_target); + strbuf_release(&err); free(msg); - return retval; + return ret; } -int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, +int refs_rename_ref(struct ref_store *refs, const char *oldref, const char *newref, const char *logmsg) { - char *msg; - int retval; + return refs_copy_or_rename_ref(refs, oldref, newref, logmsg, 0); +} - msg = normalize_reflog_message(logmsg); - retval = refs->be->copy_ref(refs, oldref, newref, msg); - free(msg); - return retval; +int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, + const char *newref, const char *logmsg) +{ + return refs_copy_or_rename_ref(refs, oldref, newref, logmsg, 1); } const char *ref_update_original_update_refname(struct ref_update *update) diff --git a/refs/debug.c b/refs/debug.c index 639db0f26..87b84e767 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -143,28 +143,6 @@ static int debug_optimize_required(struct ref_store *ref_store, return res; } -static int debug_rename_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = drefs->refs->be->rename_ref(drefs->refs, oldref, newref, - logmsg); - trace_printf_key(&trace_refs, "rename_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - -static int debug_copy_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = - drefs->refs->be->copy_ref(drefs->refs, oldref, newref, logmsg); - trace_printf_key(&trace_refs, "copy_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - struct debug_ref_iterator { struct ref_iterator base; struct ref_iterator *iter; @@ -453,9 +431,6 @@ struct ref_storage_be refs_be_debug = { .optimize = debug_optimize, .optimize_required = debug_optimize_required, - .rename_ref = debug_rename_ref, - .copy_ref = debug_copy_ref, - .iterator_begin = debug_ref_iterator_begin, .read_raw_ref = debug_read_raw_ref, .read_symbolic_ref = debug_read_symbolic_ref, diff --git a/refs/files-backend.c b/refs/files-backend.c index 71628550f..c28228116 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1594,6 +1594,7 @@ static int files_optimize_required(struct ref_store *ref_store, * live into logs/refs. */ #define TMP_RENAMED_LOG "refs/.tmp-renamed-log" +#define TMP_RENAMED_LOG_DESTINATION "refs/.tmp-renamed-log-destination" struct rename_cb { const char *tmp_renamed_log; @@ -1685,12 +1686,28 @@ static int refs_rename_ref_available(struct ref_store *refs, return ok; } +struct files_copy_or_rename_transaction_data { + struct ref_lock *lock; + struct object_id orig_oid; + struct object_id destination_oid; + char *destination_target; + int logmoved; + int destination_exists; + int destination_log_backed_up; +}; + static int files_copy_or_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg, int copy) + struct ref_update *update, + struct ref_transaction *transaction) { struct files_ref_store *refs = - files_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); + files_downcast(ref_store, REF_STORE_WRITE, + "ref_transaction_prepare"); + struct ref_copy_or_rename_update *operation = update->copy_or_rename; + const char *oldrefname = operation->old_refname; + const char *newrefname = update->refname; + const char *logmsg = operation->logmsg; + bool copy = operation->type == REF_UPDATE_COPY; struct object_id orig_oid; int flag = 0, logmoved = 0; struct ref_lock *lock; @@ -1698,12 +1715,19 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, struct strbuf sb_oldref = STRBUF_INIT; struct strbuf sb_newref = STRBUF_INIT; struct strbuf tmp_renamed_log = STRBUF_INIT; + struct strbuf tmp_destination_log = STRBUF_INIT; + struct strbuf destination_target = STRBUF_INIT; int log, ret; + int destination_exists = 0, destination_flags = 0; + int destination_log_backed_up = 0; + struct object_id destination_oid; + struct files_copy_or_rename_transaction_data *data; struct strbuf err = STRBUF_INIT; files_reflog_path(refs, &sb_oldref, oldrefname); files_reflog_path(refs, &sb_newref, newrefname); files_reflog_path(refs, &tmp_renamed_log, TMP_RENAMED_LOG); + files_reflog_path(refs, &tmp_destination_log, TMP_RENAMED_LOG_DESTINATION); log = !lstat(sb_oldref.buf, &loginfo); if (log && S_ISLNK(loginfo.st_mode)) { @@ -1727,11 +1751,67 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, oldrefname); goto out; } + if (!oideq(&orig_oid, &operation->source_oid)) { + ret = error("refname %s is at %s but expected %s", + oldrefname, oid_to_hex(&orig_oid), + oid_to_hex(&operation->source_oid)); + goto out; + } if (!refs_rename_ref_available(&refs->base, oldrefname, newrefname)) { ret = 1; goto out; } + if (refs_resolve_ref_unsafe(&refs->base, newrefname, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, + &destination_oid, &destination_flags)) { + destination_exists = 1; + if ((destination_flags & REF_ISSYMREF) && + refs_read_symbolic_ref(&refs->base, newrefname, + &destination_target) < 0) { + ret = error("unable to read symbolic ref %s", newrefname); + goto out; + } + } + if (destination_exists != operation->destination_exists) { + ret = error("refname %s changed while renaming", newrefname); + goto out; + } + if (destination_exists) { + if (destination_flags & REF_ISSYMREF) { + if (!operation->destination_target || + strcmp(destination_target.buf, + operation->destination_target)) { + ret = error("refname %s changed while renaming", + newrefname); + goto out; + } + } else if (operation->destination_target || + !oideq(&destination_oid, + &operation->destination_oid)) { + ret = error("refname %s changed while renaming", newrefname); + goto out; + } + } + + if (!lstat(sb_newref.buf, &loginfo)) { + if (S_ISLNK(loginfo.st_mode)) { + ret = error("reflog for %s is a symlink", newrefname); + goto out; + } + if (S_ISREG(loginfo.st_mode)) { + if (copy_file(refs->base.repo, tmp_destination_log.buf, + sb_newref.buf, 0644)) { + if (errno != EEXIST) + unlink(tmp_destination_log.buf); + ret = error("unable to back up logfile logs/%s: %s", + newrefname, strerror(errno)); + goto out; + } + destination_log_backed_up = 1; + } + } + if (!copy && log && rename(sb_oldref.buf, tmp_renamed_log.buf)) { ret = error("unable to move logfile logs/%s to logs/"TMP_RENAMED_LOG": %s", oldrefname, strerror(errno)); @@ -1744,8 +1824,10 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, goto out; } - if (!copy && refs_delete_ref(&refs->base, logmsg, oldrefname, - &orig_oid, REF_NO_DEREF)) { + if (!copy && refs_delete_ref_with_transaction_flags(&refs->base, logmsg, + oldrefname, &orig_oid, + REF_NO_DEREF, + REF_TRANSACTION_FLAG_SKIP_HOOK)) { error("unable to delete old %s", oldrefname); goto rollback; } @@ -1760,8 +1842,9 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, if (!copy && refs_resolve_ref_unsafe(&refs->base, newrefname, RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, NULL, NULL) && - refs_delete_ref(&refs->base, NULL, newrefname, - NULL, REF_NO_DEREF)) { + refs_delete_ref_with_transaction_flags(&refs->base, NULL, newrefname, + NULL, REF_NO_DEREF, + REF_TRANSACTION_FLAG_SKIP_HOOK)) { if (errno == EISDIR) { struct strbuf path = STRBUF_INIT; int result; @@ -1796,13 +1879,25 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, } oidcpy(&lock->old_oid, &orig_oid); - if (write_ref_to_lockfile(refs, lock, &orig_oid, &err) || - commit_ref_update(refs, lock, &orig_oid, logmsg, 0, &err)) { + if (write_ref_to_lockfile(refs, lock, &orig_oid, &err)) { error("unable to write current sha1 into %s: %s", newrefname, err.buf); strbuf_release(&err); goto rollback; } + CALLOC_ARRAY(data, 1); + data->lock = lock; + oidcpy(&data->orig_oid, &orig_oid); + data->logmoved = logmoved; + data->destination_exists = destination_exists; + data->destination_log_backed_up = destination_log_backed_up; + if (destination_exists && !(destination_flags & REF_ISSYMREF)) + oidcpy(&data->destination_oid, &destination_oid); + if (destination_flags & REF_ISSYMREF) + data->destination_target = strbuf_detach(&destination_target, NULL); + transaction->backend_data = data; + transaction->state = REF_TRANSACTION_PREPARED; + ret = 0; goto out; @@ -1821,38 +1916,40 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, } rollbacklog: - if (logmoved && rename(sb_newref.buf, sb_oldref.buf)) - error("unable to restore logfile %s from %s: %s", - oldrefname, newrefname, strerror(errno)); + if (logmoved) { + if (rename(sb_newref.buf, tmp_renamed_log.buf)) { + error("unable to restore logfile %s from %s: %s", + oldrefname, newrefname, strerror(errno)); + } else { + try_remove_empty_parents(refs, newrefname, + REMOVE_EMPTY_PARENTS_REFLOG); + if (rename_tmp_log(refs, oldrefname)) + error("unable to restore logfile %s from logs/" + TMP_RENAMED_LOG ": %s", + oldrefname, strerror(errno)); + } + } if (!logmoved && log && rename(tmp_renamed_log.buf, sb_oldref.buf)) error("unable to restore logfile %s from logs/"TMP_RENAMED_LOG": %s", oldrefname, strerror(errno)); + if (destination_log_backed_up && + rename(tmp_destination_log.buf, sb_newref.buf)) + error("unable to restore logfile %s: %s", + newrefname, strerror(errno)); ret = 1; out: + if (ret && destination_log_backed_up) + unlink(tmp_destination_log.buf); strbuf_release(&sb_newref); strbuf_release(&sb_oldref); strbuf_release(&tmp_renamed_log); + strbuf_release(&tmp_destination_log); + strbuf_release(&destination_target); return ret; } -static int files_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 0); -} - -static int files_copy_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 1); -} - static int close_ref_gently(struct ref_lock *lock) { if (close_lock_file_gently(&lock->lk)) @@ -2962,6 +3059,14 @@ static int files_transaction_prepare(struct ref_store *ref_store, struct ref_transaction *packed_transaction = NULL; assert(err); + { + struct ref_update *operation = + ref_transaction_copy_or_rename_update(transaction); + + if (operation) + return files_copy_or_rename_ref(ref_store, operation, + transaction); + } if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) goto cleanup; @@ -3318,6 +3423,10 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, return ret; } +static int files_transaction_abort(struct ref_store *ref_store, + struct ref_transaction *transaction, + struct strbuf *err); + static int files_transaction_finish(struct ref_store *ref_store, struct ref_transaction *transaction, struct strbuf *err) @@ -3333,6 +3442,40 @@ static int files_transaction_finish(struct ref_store *ref_store, assert(err); + { + struct ref_update *update = + ref_transaction_copy_or_rename_update(transaction); + + if (update) { + struct ref_copy_or_rename_update *operation = + update->copy_or_rename; + struct files_copy_or_rename_transaction_data *data = + transaction->backend_data; + int special_ret; + + special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, + operation->logmsg, 0, err); + if (special_ret) { + error("unable to write current sha1 into %s: %s", + update->refname, err->buf); + data->lock = NULL; + files_transaction_abort(ref_store, transaction, err); + return special_ret; + } else if (data->destination_log_backed_up) { + struct strbuf path = STRBUF_INIT; + + files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); + if (unlink(path.buf) < 0 && errno != ENOENT) + warning_errno("unable to remove '%s'", path.buf); + strbuf_release(&path); + } + free(data->destination_target); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + return special_ret; + } + } if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) return files_transaction_finish_initial(refs, transaction, err); @@ -3476,11 +3619,105 @@ static int files_transaction_finish(struct ref_store *ref_store, static int files_transaction_abort(struct ref_store *ref_store, struct ref_transaction *transaction, - struct strbuf *err UNUSED) + struct strbuf *err) { struct files_ref_store *refs = files_downcast(ref_store, 0, "ref_transaction_abort"); + { + struct ref_update *update = + ref_transaction_copy_or_rename_update(transaction); + + if (update) { + struct ref_copy_or_rename_update *operation = + update->copy_or_rename; + struct files_copy_or_rename_transaction_data *data = + transaction->backend_data; + struct strbuf new_log = STRBUF_INIT; + struct strbuf destination_log = STRBUF_INIT; + struct strbuf temporary_log = STRBUF_INIT; + struct ref_transaction *restore_transaction = NULL; + struct ref_lock *lock; + int ret = 0; + + if (data->lock) + unlock_ref(data->lock); + if (operation->type == REF_UPDATE_RENAME) { + lock = lock_ref_oid_basic(refs, operation->old_refname, err); + if (!lock || + write_ref_to_lockfile(refs, lock, &data->orig_oid, err) || + commit_ref_update(refs, lock, &data->orig_oid, NULL, + REF_SKIP_CREATE_REFLOG, err)) + ret = -1; + } + + if (data->logmoved) { + files_reflog_path(refs, &new_log, update->refname); + if (operation->type == REF_UPDATE_RENAME) { + files_reflog_path(refs, &temporary_log, TMP_RENAMED_LOG); + if (rename(new_log.buf, temporary_log.buf) < 0) { + strbuf_addf(err, "unable to restore logfile %s: %s", + operation->old_refname, strerror(errno)); + ret = -1; + } else { + try_remove_empty_parents(refs, + update->refname, + REMOVE_EMPTY_PARENTS_REFLOG); + if (rename_tmp_log(refs, + operation->old_refname)) { + strbuf_addf(err, "unable to restore logfile %s: %s", + operation->old_refname, + strerror(errno)); + ret = -1; + } + } + } else if (unlink(new_log.buf) < 0 && errno != ENOENT) { + strbuf_addf(err, "unable to remove logfile %s: %s", + update->refname, strerror(errno)); + ret = -1; + } + } + if (data->destination_log_backed_up) { + files_reflog_path(refs, &destination_log, + TMP_RENAMED_LOG_DESTINATION); + if (rename(destination_log.buf, new_log.buf) < 0) { + strbuf_addf(err, "unable to restore logfile %s: %s", + update->refname, strerror(errno)); + ret = -1; + } + } + + if (operation->type == REF_UPDATE_RENAME && + data->destination_exists) { + restore_transaction = ref_store_transaction_begin( + &refs->base, REF_TRANSACTION_FLAG_SKIP_HOOK, err); + if (!restore_transaction || + ref_transaction_update(restore_transaction, + update->refname, + data->destination_target ? NULL : + &data->destination_oid, + NULL, + data->destination_target, + NULL, + REF_NO_DEREF | + REF_SKIP_CREATE_REFLOG, + NULL, err) || + ref_transaction_commit(restore_transaction, err)) + ret = -1; + ref_transaction_free(restore_transaction); + } + + strbuf_release(&destination_log); + strbuf_release(&temporary_log); + strbuf_release(&new_log); + free(data->destination_target); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + return ret; + } + } + files_transaction_cleanup(refs, transaction); return 0; } @@ -4095,8 +4332,6 @@ struct ref_storage_be refs_be_files = { .optimize = files_optimize, .optimize_required = files_optimize_required, - .rename_ref = files_rename_ref, - .copy_ref = files_copy_ref, .iterator_begin = files_ref_iterator_begin, .read_raw_ref = files_read_raw_ref, diff --git a/refs/packed-backend.c b/refs/packed-backend.c index a73fc6aca..364a91291 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -2164,8 +2164,6 @@ struct ref_storage_be refs_be_packed = { .optimize = packed_optimize, .optimize_required = packed_optimize_required, - .rename_ref = NULL, - .copy_ref = NULL, .iterator_begin = packed_ref_iterator_begin, .read_raw_ref = packed_read_raw_ref, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index c3ac7b556..5d4dc0171 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -155,9 +155,33 @@ struct ref_update { */ struct ref_update *parent_update; + /* + * Copy and rename operations require backend-specific handling while + * still exposing their logical updates to transaction hooks. Keep that + * state on the destination update so it composes with other updates in + * the transaction instead of making copy or rename a transaction-wide + * property. + */ + struct ref_copy_or_rename_update *copy_or_rename; + const char refname[FLEX_ARRAY]; }; +enum ref_copy_or_rename_type { + REF_UPDATE_RENAME, + REF_UPDATE_COPY, +}; + +struct ref_copy_or_rename_update { + enum ref_copy_or_rename_type type; + char *old_refname; + char *logmsg; + struct object_id source_oid; + struct object_id destination_oid; + char *destination_target; + unsigned int destination_exists:1; +}; + int refs_read_raw_ref(struct ref_store *ref_store, const char *refname, struct object_id *oid, struct strbuf *referent, unsigned int *type, int *failure_errno); @@ -187,6 +211,13 @@ struct ref_update *ref_transaction_add_update( const char *committer_info, const char *msg); +int refs_delete_ref_with_transaction_flags(struct ref_store *refs, + const char *msg, + const char *refname, + const struct object_id *old_oid, + unsigned int flags, + unsigned int transaction_flags); + /* * Transaction states. * @@ -242,6 +273,12 @@ struct ref_transaction { uint64_t max_index; }; +/* Suppress hooks for a transaction nested inside another refs operation. */ +#define REF_TRANSACTION_FLAG_SKIP_HOOK (1 << 2) + +struct ref_update *ref_transaction_copy_or_rename_update( + struct ref_transaction *transaction); + /* * Check for entries in extras that are within the specified * directory, where dirname is a reference directory name including @@ -451,13 +488,6 @@ typedef int optimize_required_fn(struct ref_store *ref_store, struct refs_optimize_opts *opts, bool *required); -typedef int rename_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); -typedef int copy_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); - /* * Iterate over the references in `ref_store` whose names start with * `prefix`. `prefix` is matched as a literal string, without regard @@ -577,9 +607,6 @@ struct ref_storage_be { optimize_fn *optimize; optimize_required_fn *optimize_required; - rename_ref_fn *rename_ref; - copy_ref_fn *copy_ref; - ref_iterator_begin_fn *iterator_begin; read_raw_ref_fn *read_raw_ref; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 10db03991..589fcc998 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -953,6 +953,14 @@ struct reftable_transaction_data { size_t args_nr, args_alloc; }; +struct reftable_copy_or_rename_transaction_data { + struct reftable_addition *addition; +}; + +static int reftable_be_copy_or_rename_prepare(struct ref_store *ref_store, + struct ref_transaction *transaction, + struct strbuf *err); + static void free_transaction_data(struct reftable_transaction_data *tx_data) { if (!tx_data) @@ -1326,6 +1334,10 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, size_t i; int ret; + if (ref_transaction_copy_or_rename_update(transaction)) + return reftable_be_copy_or_rename_prepare(ref_store, transaction, + err); + ret = refs->err; if (ret < 0) goto done; @@ -1419,7 +1431,20 @@ static int reftable_be_transaction_abort(struct ref_store *ref_store UNUSED, struct ref_transaction *transaction, struct strbuf *err UNUSED) { - struct reftable_transaction_data *tx_data = transaction->backend_data; + struct reftable_transaction_data *tx_data; + + if (ref_transaction_copy_or_rename_update(transaction)) { + struct reftable_copy_or_rename_transaction_data *data = + transaction->backend_data; + + reftable_addition_destroy(data->addition); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + return 0; + } + + tx_data = transaction->backend_data; free_transaction_data(tx_data); transaction->state = REF_TRANSACTION_CLOSED; return 0; @@ -1667,9 +1692,28 @@ static int reftable_be_transaction_finish(struct ref_store *ref_store UNUSED, struct ref_transaction *transaction, struct strbuf *err) { - struct reftable_transaction_data *tx_data = transaction->backend_data; + struct reftable_transaction_data *tx_data; int ret = 0; + if (ref_transaction_copy_or_rename_update(transaction)) { + struct reftable_copy_or_rename_transaction_data *data = + transaction->backend_data; + int special_ret = reftable_addition_commit(data->addition); + + reftable_addition_destroy(data->addition); + free(data); + transaction->backend_data = NULL; + transaction->state = REF_TRANSACTION_CLOSED; + if (special_ret < 0) { + strbuf_addf(err, _("reftable: transaction failure: %s"), + reftable_error_str(special_ret)); + return -1; + } + return 0; + } + + tx_data = transaction->backend_data; + for (size_t i = 0; i < tx_data->args_nr; i++) { tx_data->args[i].max_index = transaction->max_index; @@ -1764,17 +1808,20 @@ struct write_create_symref_arg { struct write_copy_arg { struct reftable_ref_store *refs; struct reftable_backend *be; + struct strbuf *err; const char *oldname; const char *newname; const char *logmsg; int delete_old; + struct ref_copy_or_rename_update *operation; }; static int write_copy_table(struct reftable_writer *writer, void *cb_data) { struct write_copy_arg *arg = cb_data; uint64_t deletion_ts, creation_ts; - struct reftable_ref_record old_ref = {0}, refs[2] = {0}; + struct reftable_ref_record old_ref = {0}, destination_ref = {0}; + struct reftable_ref_record refs[2] = {0}; struct reftable_log_record old_log = {0}, *logs = NULL; struct reftable_iterator it = {0}; struct string_list skip = STRING_LIST_INIT_NODUP; @@ -1789,14 +1836,75 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) BUG("failed splitting committer info"); if (reftable_stack_read_ref(arg->be->stack, arg->oldname, &old_ref)) { - ret = error(_("refname %s not found"), arg->oldname); + strbuf_addf(arg->err, _("refname %s not found"), arg->oldname); + ret = -1; goto done; } if (old_ref.value_type == REFTABLE_REF_SYMREF) { - ret = error(_("refname %s is a symbolic ref, copying it is not supported"), + strbuf_addf(arg->err, + _("refname %s is a symbolic ref, copying it is not supported"), arg->oldname); + ret = -1; goto done; } + if (arg->operation) { + struct object_id oid; + + if (old_ref.value_type == REFTABLE_REF_VAL2) + oidread(&oid, old_ref.value.val2.value, + arg->refs->base.repo->hash_algo); + else + oidread(&oid, old_ref.value.val1, + arg->refs->base.repo->hash_algo); + if (!oideq(&oid, &arg->operation->source_oid)) { + strbuf_addf(arg->err, + _("refname %s is at %s but expected %s"), + arg->oldname, oid_to_hex(&oid), + oid_to_hex(&arg->operation->source_oid)); + ret = -1; + goto done; + } + + ret = reftable_stack_read_ref(arg->be->stack, arg->newname, + &destination_ref); + if (ret < 0) + goto done; + if (arg->operation->destination_exists != !ret) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); + ret = -1; + goto done; + } + if (!ret) { + if (destination_ref.value_type == REFTABLE_REF_SYMREF) { + if (!arg->operation->destination_target || + strcmp(destination_ref.value.symref, + arg->operation->destination_target)) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); + ret = -1; + goto done; + } + } else { + if (destination_ref.value_type == REFTABLE_REF_VAL2) + oidread(&oid, destination_ref.value.val2.value, + arg->refs->base.repo->hash_algo); + else + oidread(&oid, destination_ref.value.val1, + arg->refs->base.repo->hash_algo); + if (arg->operation->destination_target || + !oideq(&oid, &arg->operation->destination_oid)) { + strbuf_addf(arg->err, + _("refname %s changed while renaming"), + arg->newname); + ret = -1; + goto done; + } + } + } + } /* * There's nothing to do in case the old and new name are the same, so @@ -1815,7 +1923,7 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) ret = refs_verify_refname_available(&arg->refs->base, arg->newname, NULL, &skip, 0, &errbuf); if (ret < 0) { - error("%s", errbuf.buf); + strbuf_addbuf(arg->err, &errbuf); goto done; } @@ -1980,68 +2088,63 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) for (i = 0; i < ARRAY_SIZE(refs); i++) reftable_ref_record_release(&refs[i]); reftable_ref_record_release(&old_ref); + reftable_ref_record_release(&destination_ref); reftable_log_record_release(&old_log); return ret; } -static int reftable_be_rename_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) +static int reftable_be_copy_or_rename_prepare(struct ref_store *ref_store, + struct ref_transaction *transaction, + struct strbuf *err) { struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); + reftable_be_downcast(ref_store, REF_STORE_WRITE, + "ref_transaction_prepare"); + struct reftable_copy_or_rename_transaction_data *data = NULL; + struct ref_update *update = + ref_transaction_copy_or_rename_update(transaction); + struct ref_copy_or_rename_update *operation = update->copy_or_rename; struct write_copy_arg arg = { .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - .delete_old = 1, + .err = err, + .oldname = operation->old_refname, + .newname = update->refname, + .logmsg = operation->logmsg, + .delete_old = operation->type == REF_UPDATE_RENAME, + .operation = operation, }; int ret; + CALLOC_ARRAY(data, 1); ret = refs->err; if (ret < 0) goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); + ret = backend_for(&arg.be, refs, update->refname, + &arg.newname, 1); if (ret) goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); - -done: - assert(ret != REFTABLE_API_ERROR); - return ret; -} - -static int reftable_be_copy_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) -{ - struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); - struct write_copy_arg arg = { - .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - }; - int ret; - - ret = refs->err; - if (ret < 0) + ret = reftable_stack_addition_new(&data->addition, arg.be->stack, + &reftable_be_write_options(refs)->opts); + if (ret) goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); + ret = reftable_addition_add(data->addition, &write_copy_table, &arg); if (ret) goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); + + transaction->backend_data = data; + transaction->state = REF_TRANSACTION_PREPARED; + return 0; done: assert(ret != REFTABLE_API_ERROR); + if (data) { + reftable_addition_destroy(data->addition); + free(data); + } + transaction->state = REF_TRANSACTION_CLOSED; + if (ret && !err->len) + strbuf_addf(err, _("reftable: transaction prepare: %s"), + reftable_error_str(ret)); return ret; } @@ -2872,8 +2975,6 @@ struct ref_storage_be refs_be_reftable = { .optimize = reftable_be_optimize, .optimize_required = reftable_be_optimize_required, - .rename_ref = reftable_be_rename_ref, - .copy_ref = reftable_be_copy_ref, .iterator_begin = reftable_be_iterator_begin, .read_raw_ref = reftable_be_read_raw_ref, diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh index 4fe9d9b23..116b2ff07 100755 --- a/t/t1416-ref-transaction-hooks.sh +++ b/t/t1416-ref-transaction-hooks.sh @@ -93,6 +93,148 @@ test_expect_success 'hook gets all queued updates in committed state' ' test_cmp expect actual ' +test_expect_success 'hook gets both updates when renaming a branch' ' + test_when_finished "rm -f actual" && + git branch old PRE && + test_hook reference-transaction <<-\EOF && + echo "$1" >>actual && + cat >>actual + EOF + cat >expect <<-EOF && + preparing + $PRE_OID $ZERO_OID refs/heads/old + $ZERO_OID $PRE_OID refs/heads/new + prepared + $PRE_OID $ZERO_OID refs/heads/old + $ZERO_OID $PRE_OID refs/heads/new + committed + $PRE_OID $ZERO_OID refs/heads/old + $ZERO_OID $PRE_OID refs/heads/new + EOF + git branch -m old new && + test_cmp expect actual && + test_must_fail git rev-parse --verify refs/heads/old && + test_cmp_rev PRE refs/heads/new +' + +test_expect_success 'hook gets destination update when copying a branch' ' + test_when_finished "rm -f actual" && + git branch copy-source PRE && + test_hook reference-transaction <<-\EOF && + echo "$1" >>actual && + cat >>actual + EOF + cat >expect <<-EOF && + preparing + $ZERO_OID $PRE_OID refs/heads/copy-destination + prepared + $ZERO_OID $PRE_OID refs/heads/copy-destination + committed + $ZERO_OID $PRE_OID refs/heads/copy-destination + EOF + git branch -c copy-source copy-destination && + test_cmp expect actual && + test_cmp_rev PRE refs/heads/copy-source && + test_cmp_rev PRE refs/heads/copy-destination +' + +test_expect_success 'hook gets overwritten values for forced rename and copy' ' + git branch force-old PRE && + git branch force-new POST && + git branch force-copy-source PRE && + git branch force-copy-destination POST && + test_hook reference-transaction <<-\EOF && + if test "$1" = committed + then + cat >>actual + fi + EOF + git branch -M force-old force-new && + git branch -C force-copy-source force-copy-destination && + cat >expect <<-EOF && + $PRE_OID $ZERO_OID refs/heads/force-old + $POST_OID $PRE_OID refs/heads/force-new + $POST_OID $PRE_OID refs/heads/force-copy-destination + EOF + test_cmp expect actual +' + +test_expect_success 'hook can abort a branch rename after preparation' ' + git branch abort-old PRE && + git branch abort-new POST && + git reflog show --format=%gs abort-old >old-log && + git reflog show --format=%gs abort-new >new-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -M abort-old abort-new && + test_cmp_rev PRE refs/heads/abort-old && + test_cmp_rev POST refs/heads/abort-new && + git reflog show --format=%gs abort-old >old-log-after && + git reflog show --format=%gs abort-new >new-log-after && + test_cmp old-log old-log-after && + test_cmp new-log new-log-after +' + +test_expect_success 'hook can abort a D/F branch rename after preparation' ' + git branch df-old PRE && + git reflog show --format=%gs df-old >df-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -m df-old df-old/child && + test_cmp_rev PRE refs/heads/df-old && + test_must_fail git rev-parse --verify refs/heads/df-old/child && + git reflog show --format=%gs df-old >df-log-after && + test_cmp df-log df-log-after +' + +test_expect_success 'hook can abort a reverse D/F rename after preparation' ' + git branch reverse/old PRE && + git reflog show --format=%gs reverse/old >reverse-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -m reverse/old reverse && + test_cmp_rev PRE refs/heads/reverse/old && + test_must_fail git rev-parse --verify refs/heads/reverse && + git reflog show --format=%gs reverse/old >reverse-log-after && + test_cmp reverse-log reverse-log-after +' + +test_expect_success 'hook can abort a forced branch copy after preparation' ' + git branch copy-abort-old PRE && + git branch copy-abort-new POST && + git reflog show --format=%gs copy-abort-old >copy-old-log && + git reflog show --format=%gs copy-abort-new >copy-new-log && + test_hook reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail git branch -C copy-abort-old copy-abort-new && + test_cmp_rev PRE refs/heads/copy-abort-old && + test_cmp_rev POST refs/heads/copy-abort-new && + git reflog show --format=%gs copy-abort-old >copy-old-log-after && + git reflog show --format=%gs copy-abort-new >copy-new-log-after && + test_cmp copy-old-log copy-old-log-after && + test_cmp copy-new-log copy-new-log-after +' + +test_expect_success 'branch rename detects an update during preparing hook' ' + git branch race-old PRE && + git branch race-new POST && + test_hook reference-transaction <<-\EOF && + marker=$(git rev-parse --git-path rename-race-once) + if test "$1" = preparing && test ! -e "$marker" + then + >"$marker" && + git update-ref refs/heads/race-old POST + fi + EOF + test_must_fail git branch -M race-old race-new && + test_cmp_rev POST refs/heads/race-old && + test_cmp_rev POST refs/heads/race-new +' + test_expect_success 'hook gets all queued updates in aborted state' ' test_when_finished "rm actual" && git reset --hard PRE && -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [PATCH v2] refs: run copy and rename through transactions 2026-09-23 13:36 ` [PATCH v2] " Maciej Ciemborowicz @ 2026-09-30 3:32 ` Maciej Ciemborowicz 2026-10-02 10:56 ` Patrick Steinhardt 1 sibling, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-09-30 3:32 UTC (permalink / raw) To: git; +Cc: gitster, karthik.188 I'd appreciate a code review of v2. Thanks, - Maciej Ciemborowicz On Wed, Sep 23, 2026 at 3:36 PM Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> wrote: > > Reference copy and rename operations bypass the transaction API. > Consequently, the reference-transaction hook sees only the source deletion > with the files backend and no useful update with the reftable backend. > > Represent both operations as reference transactions containing their > logical updates. A rename is a deletion of the old reference and creation > of the new reference in the same transaction. Attach operation-specific > state to the destination update instead of making copy or rename a property > of the entire transaction. > > Retain backend-specific reflog handling: the files backend stages its > existing rename procedure across prepare, finish and abort, while reftable > stages an addition while holding the stack lock. Suppress hooks for the > files backend's nested deletion transactions so that callers observe one > logical transaction. > > Record and verify the source and destination values after taking backend > locks. This rejects concurrent changes instead of applying a rename or copy > that differs from the payload shown to the preparing hook. Preserve D/F > renames and restore overwritten references and reflogs when a prepared hook > rejects the operation. > > Add tests covering rename, copy, forced updates, both directions of D/F > conflicts, concurrent updates and prepared-hook rollback. > > Helped-by: Karthik Nayak <karthik.188@gmail.com> > Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> > --- > Apologies for the unrelated Subject header on my earlier reply. This > reroll incorporates the points discussed there. > > Changes since v1: > > * Keep copy/rename state on the destination ref update instead of the > generic transaction, so the operation is no longer a transaction-wide > property. > * Keep REF_TRANSACTION_FLAG_SKIP_HOOK private to the refs implementation. > * Use bool for the two-state copy parameter. > * Apply Junio's commit-message wording suggestions. > > Range-diff against v1: > 1: de0a5a9f7 ! 1: d852537d8 refs: run copy and rename through transactions > @@ Metadata > ## Commit message ## > refs: run copy and rename through transactions > > - Reference copy and rename operations currently bypass the transaction API. > + Reference copy and rename operations bypass the transaction API. > Consequently, the reference-transaction hook sees only the source deletion > with the files backend and no useful update with the reftable backend. > > Represent both operations as reference transactions containing their > logical updates. A rename is a deletion of the old reference and creation > - of the new reference in the same transaction. Retain backend-specific > - reflog handling: the files backend stages its existing rename procedure > - across prepare, finish and abort, while reftable stages an addition while > - holding the stack lock. Suppress hooks for the files backend's nested > - deletion transactions so that callers observe one logical transaction. > + of the new reference in the same transaction. Attach operation-specific > + state to the destination update instead of making copy or rename a property > + of the entire transaction. > + > + Retain backend-specific reflog handling: the files backend stages its > + existing rename procedure across prepare, finish and abort, while reftable > + stages an addition while holding the stack lock. Suppress hooks for the > + files backend's nested deletion transactions so that callers observe one > + logical transaction. > > Record and verify the source and destination values after taking backend > locks. This rejects concurrent changes instead of applying a rename or copy > @@ Commit message > renames and restore overwritten references and reflogs when a prepared hook > rejects the operation. > > - Add coverage for rename, copy, forced updates, both directions of D/F > + Add tests covering rename, copy, forced updates, both directions of D/F > conflicts, concurrent updates and prepared-hook rollback. > > Helped-by: Karthik Nayak <karthik.188@gmail.com> > @@ refs.c: int refs_delete_ref(struct ref_store *refs, const char *msg, > { > char c; > @@ refs.c: void ref_transaction_free(struct ref_transaction *transaction) > + } > + > + for (i = 0; i < transaction->nr; i++) { > ++ struct ref_copy_or_rename_update *operation = > ++ transaction->updates[i]->copy_or_rename; > ++ > + free(transaction->updates[i]->msg); > + free(transaction->updates[i]->committer_info); > + free((char *)transaction->updates[i]->new_target); > + free((char *)transaction->updates[i]->old_target); > + free((char *)transaction->updates[i]->rejection_details); > ++ if (operation) { > ++ free(operation->old_refname); > ++ free(operation->logmsg); > ++ free(operation->destination_target); > ++ free(operation); > ++ } > + free(transaction->updates[i]); > + } > > - string_list_clear(&transaction->refnames, 0); > - free(transaction->updates); > -+ free(transaction->old_refname); > -+ free(transaction->new_refname); > -+ free(transaction->logmsg); > -+ free(transaction->destination_target); > +@@ refs.c: void ref_transaction_free(struct ref_transaction *transaction) > free(transaction); > } > > ++struct ref_update *ref_transaction_copy_or_rename_update( > ++ struct ref_transaction *transaction) > ++{ > ++ struct ref_update *operation = NULL; > ++ size_t i; > ++ > ++ for (i = 0; i < transaction->nr; i++) { > ++ if (!transaction->updates[i]->copy_or_rename) > ++ continue; > ++ if (operation) > ++ BUG("multiple copy or rename updates in one transaction"); > ++ operation = transaction->updates[i]; > ++ } > ++ > ++ return operation; > ++} > ++ > + int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction, > + size_t update_idx, > + enum ref_transaction_error err, > @@ refs.c: int ref_transaction_prepare(struct ref_transaction *transaction, > return REF_TRANSACTION_ERROR_GENERIC; > > @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, > - const char *newref, const char *logmsg) > +static int refs_copy_or_rename_ref(struct ref_store *refs, const char *oldref, > + const char *newref, const char *logmsg, > -+ int copy) > ++ bool copy) > { > - char *msg; > - int retval; > + struct ref_transaction *transaction = NULL; > ++ struct ref_copy_or_rename_update *operation = NULL; > ++ struct ref_update *destination_update; > + struct object_id old_oid, new_oid; > + struct strbuf new_target = STRBUF_INIT; > + struct strbuf err = STRBUF_INIT; > @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, > + transaction = ref_store_transaction_begin(refs, 0, &err); > + if (!transaction) > + goto error; > -+ transaction->type = copy ? REF_TRANSACTION_TYPE_COPY : > -+ REF_TRANSACTION_TYPE_RENAME; > -+ transaction->old_refname = xstrdup(oldref); > -+ transaction->new_refname = xstrdup(newref); > -+ transaction->logmsg = xstrdup(msg); > -+ oidcpy(&transaction->source_oid, &old_oid); > -+ > + if (!copy && ref_transaction_delete(transaction, oldref, &old_oid, NULL, > + REF_NO_DEREF, msg, &err)) > + goto error; > @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, > + } else { > + oidclr(&new_oid, refs->repo->hash_algo); > + } > -+ transaction->destination_exists = new_exists; > -+ if (new_flags & REF_ISSYMREF) > -+ transaction->destination_target = xstrdup(new_target.buf); > -+ else if (transaction->destination_exists) > -+ oidcpy(&transaction->destination_oid, &new_oid); > -+ > + if (ref_transaction_update(transaction, newref, &old_oid, > + (new_flags & REF_ISSYMREF) ? NULL : &new_oid, > + NULL, > @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, > + NULL, &err)) > + goto error; > + > ++ destination_update = transaction->updates[transaction->nr - 1]; > ++ CALLOC_ARRAY(operation, 1); > ++ operation->type = copy ? REF_UPDATE_COPY : REF_UPDATE_RENAME; > ++ operation->old_refname = xstrdup(oldref); > ++ operation->logmsg = xstrdup(msg); > ++ oidcpy(&operation->source_oid, &old_oid); > ++ operation->destination_exists = new_exists; > ++ if (new_flags & REF_ISSYMREF) > ++ operation->destination_target = xstrdup(new_target.buf); > ++ else if (operation->destination_exists) > ++ oidcpy(&operation->destination_oid, &new_oid); > ++ destination_update->copy_or_rename = operation; > ++ > + if (ref_transaction_commit(transaction, &err)) > + goto error; > + > @@ refs.c: int refs_delete_refs(struct ref_store *refs, const char *logmsg, > > const char *ref_update_original_update_refname(struct ref_update *update) > > - ## refs.h ## > -@@ refs.h: enum ref_transaction_flag { > - * while rejecting updates which do not match the expected state. > - */ > - REF_TRANSACTION_ALLOW_FAILURE = (1 << 1), > -+ > -+ /* Suppress hooks for an update nested in another transaction. */ > -+ REF_TRANSACTION_FLAG_SKIP_HOOK = (1 << 2), > - }; > - > - /* > - > ## refs/debug.c ## > @@ refs/debug.c: static int debug_optimize_required(struct ref_store *ref_store, > return res; > @@ refs/files-backend.c: static int refs_rename_ref_available(struct ref_store *ref > +}; > + > static int files_copy_or_rename_ref(struct ref_store *ref_store, > - const char *oldrefname, const char *newrefname, > +- const char *oldrefname, const char *newrefname, > - const char *logmsg, int copy) > -+ const char *logmsg, int copy, > -+ struct ref_transaction *transaction) > ++ struct ref_update *update, > ++ struct ref_transaction *transaction) > { > struct files_ref_store *refs = > - files_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); > + files_downcast(ref_store, REF_STORE_WRITE, > + "ref_transaction_prepare"); > ++ struct ref_copy_or_rename_update *operation = update->copy_or_rename; > ++ const char *oldrefname = operation->old_refname; > ++ const char *newrefname = update->refname; > ++ const char *logmsg = operation->logmsg; > ++ bool copy = operation->type == REF_UPDATE_COPY; > struct object_id orig_oid; > int flag = 0, logmoved = 0; > struct ref_lock *lock; > @@ refs/files-backend.c: static int files_copy_or_rename_ref(struct ref_store *ref_ > oldrefname); > goto out; > } > -+ if (!oideq(&orig_oid, &transaction->source_oid)) { > ++ if (!oideq(&orig_oid, &operation->source_oid)) { > + ret = error("refname %s is at %s but expected %s", > + oldrefname, oid_to_hex(&orig_oid), > -+ oid_to_hex(&transaction->source_oid)); > ++ oid_to_hex(&operation->source_oid)); > + goto out; > + } > if (!refs_rename_ref_available(&refs->base, oldrefname, newrefname)) { > @@ refs/files-backend.c: static int files_copy_or_rename_ref(struct ref_store *ref_ > + goto out; > + } > + } > -+ if (destination_exists != transaction->destination_exists) { > ++ if (destination_exists != operation->destination_exists) { > + ret = error("refname %s changed while renaming", newrefname); > + goto out; > + } > + if (destination_exists) { > + if (destination_flags & REF_ISSYMREF) { > -+ if (!transaction->destination_target || > ++ if (!operation->destination_target || > + strcmp(destination_target.buf, > -+ transaction->destination_target)) { > ++ operation->destination_target)) { > + ret = error("refname %s changed while renaming", > + newrefname); > + goto out; > + } > -+ } else if (transaction->destination_target || > ++ } else if (operation->destination_target || > + !oideq(&destination_oid, > -+ &transaction->destination_oid)) { > ++ &operation->destination_oid)) { > + ret = error("refname %s changed while renaming", newrefname); > + goto out; > + } > @@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref > struct ref_transaction *packed_transaction = NULL; > > assert(err); > -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) > -+ return files_copy_or_rename_ref(ref_store, > -+ transaction->old_refname, > -+ transaction->new_refname, > -+ transaction->logmsg, > -+ transaction->type == REF_TRANSACTION_TYPE_COPY, > -+ transaction); > ++ { > ++ struct ref_update *operation = > ++ ref_transaction_copy_or_rename_update(transaction); > ++ > ++ if (operation) > ++ return files_copy_or_rename_ref(ref_store, operation, > ++ transaction); > ++ } > > if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) > goto cleanup; > @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_ > > > assert(err); > -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { > -+ struct files_copy_or_rename_transaction_data *data = > -+ transaction->backend_data; > -+ int special_ret; > ++ { > ++ struct ref_update *update = > ++ ref_transaction_copy_or_rename_update(transaction); > + > -+ special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, > -+ transaction->logmsg, 0, err); > -+ if (special_ret) { > -+ error("unable to write current sha1 into %s: %s", > -+ transaction->new_refname, err->buf); > -+ data->lock = NULL; > -+ files_transaction_abort(ref_store, transaction, err); > -+ return special_ret; > -+ } else if (data->destination_log_backed_up) { > -+ struct strbuf path = STRBUF_INIT; > ++ if (update) { > ++ struct ref_copy_or_rename_update *operation = > ++ update->copy_or_rename; > ++ struct files_copy_or_rename_transaction_data *data = > ++ transaction->backend_data; > ++ int special_ret; > ++ > ++ special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, > ++ operation->logmsg, 0, err); > ++ if (special_ret) { > ++ error("unable to write current sha1 into %s: %s", > ++ update->refname, err->buf); > ++ data->lock = NULL; > ++ files_transaction_abort(ref_store, transaction, err); > ++ return special_ret; > ++ } else if (data->destination_log_backed_up) { > ++ struct strbuf path = STRBUF_INIT; > + > -+ files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); > -+ if (unlink(path.buf) < 0 && errno != ENOENT) > -+ warning_errno("unable to remove '%s'", path.buf); > -+ strbuf_release(&path); > ++ files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); > ++ if (unlink(path.buf) < 0 && errno != ENOENT) > ++ warning_errno("unable to remove '%s'", path.buf); > ++ strbuf_release(&path); > ++ } > ++ free(data->destination_target); > ++ free(data); > ++ transaction->backend_data = NULL; > ++ transaction->state = REF_TRANSACTION_CLOSED; > ++ return special_ret; > + } > -+ free(data->destination_target); > -+ free(data); > -+ transaction->backend_data = NULL; > -+ transaction->state = REF_TRANSACTION_CLOSED; > -+ return special_ret; > + } > > if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) > @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_ > struct files_ref_store *refs = > files_downcast(ref_store, 0, "ref_transaction_abort"); > > -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { > -+ struct files_copy_or_rename_transaction_data *data = > -+ transaction->backend_data; > -+ struct strbuf new_log = STRBUF_INIT; > -+ struct strbuf destination_log = STRBUF_INIT; > -+ struct strbuf temporary_log = STRBUF_INIT; > -+ struct ref_transaction *restore_transaction = NULL; > -+ struct ref_lock *lock; > -+ int ret = 0; > ++ { > ++ struct ref_update *update = > ++ ref_transaction_copy_or_rename_update(transaction); > + > -+ if (data->lock) > -+ unlock_ref(data->lock); > -+ if (transaction->type == REF_TRANSACTION_TYPE_RENAME) { > -+ lock = lock_ref_oid_basic(refs, transaction->old_refname, err); > -+ if (!lock || > -+ write_ref_to_lockfile(refs, lock, &data->orig_oid, err) || > -+ commit_ref_update(refs, lock, &data->orig_oid, NULL, > -+ REF_SKIP_CREATE_REFLOG, err)) > -+ ret = -1; > -+ } > ++ if (update) { > ++ struct ref_copy_or_rename_update *operation = > ++ update->copy_or_rename; > ++ struct files_copy_or_rename_transaction_data *data = > ++ transaction->backend_data; > ++ struct strbuf new_log = STRBUF_INIT; > ++ struct strbuf destination_log = STRBUF_INIT; > ++ struct strbuf temporary_log = STRBUF_INIT; > ++ struct ref_transaction *restore_transaction = NULL; > ++ struct ref_lock *lock; > ++ int ret = 0; > + > -+ if (data->logmoved) { > -+ files_reflog_path(refs, &new_log, transaction->new_refname); > -+ if (transaction->type == REF_TRANSACTION_TYPE_RENAME) { > -+ files_reflog_path(refs, &temporary_log, TMP_RENAMED_LOG); > -+ if (rename(new_log.buf, temporary_log.buf) < 0) { > -+ strbuf_addf(err, "unable to restore logfile %s: %s", > -+ transaction->old_refname, strerror(errno)); > ++ if (data->lock) > ++ unlock_ref(data->lock); > ++ if (operation->type == REF_UPDATE_RENAME) { > ++ lock = lock_ref_oid_basic(refs, operation->old_refname, err); > ++ if (!lock || > ++ write_ref_to_lockfile(refs, lock, &data->orig_oid, err) || > ++ commit_ref_update(refs, lock, &data->orig_oid, NULL, > ++ REF_SKIP_CREATE_REFLOG, err)) > + ret = -1; > -+ } else { > -+ try_remove_empty_parents(refs, > -+ transaction->new_refname, > -+ REMOVE_EMPTY_PARENTS_REFLOG); > -+ if (rename_tmp_log(refs, > -+ transaction->old_refname)) { > ++ } > ++ > ++ if (data->logmoved) { > ++ files_reflog_path(refs, &new_log, update->refname); > ++ if (operation->type == REF_UPDATE_RENAME) { > ++ files_reflog_path(refs, &temporary_log, TMP_RENAMED_LOG); > ++ if (rename(new_log.buf, temporary_log.buf) < 0) { > + strbuf_addf(err, "unable to restore logfile %s: %s", > -+ transaction->old_refname, > -+ strerror(errno)); > ++ operation->old_refname, strerror(errno)); > + ret = -1; > ++ } else { > ++ try_remove_empty_parents(refs, > ++ update->refname, > ++ REMOVE_EMPTY_PARENTS_REFLOG); > ++ if (rename_tmp_log(refs, > ++ operation->old_refname)) { > ++ strbuf_addf(err, "unable to restore logfile %s: %s", > ++ operation->old_refname, > ++ strerror(errno)); > ++ ret = -1; > ++ } > + } > ++ } else if (unlink(new_log.buf) < 0 && errno != ENOENT) { > ++ strbuf_addf(err, "unable to remove logfile %s: %s", > ++ update->refname, strerror(errno)); > ++ ret = -1; > + } > -+ } else if (unlink(new_log.buf) < 0 && errno != ENOENT) { > -+ strbuf_addf(err, "unable to remove logfile %s: %s", > -+ transaction->new_refname, strerror(errno)); > -+ ret = -1; > + } > -+ } > -+ if (data->destination_log_backed_up) { > -+ files_reflog_path(refs, &destination_log, > -+ TMP_RENAMED_LOG_DESTINATION); > -+ if (rename(destination_log.buf, new_log.buf) < 0) { > -+ strbuf_addf(err, "unable to restore logfile %s: %s", > -+ transaction->new_refname, strerror(errno)); > -+ ret = -1; > ++ if (data->destination_log_backed_up) { > ++ files_reflog_path(refs, &destination_log, > ++ TMP_RENAMED_LOG_DESTINATION); > ++ if (rename(destination_log.buf, new_log.buf) < 0) { > ++ strbuf_addf(err, "unable to restore logfile %s: %s", > ++ update->refname, strerror(errno)); > ++ ret = -1; > ++ } > + } > -+ } > + > -+ if (transaction->type == REF_TRANSACTION_TYPE_RENAME && > -+ data->destination_exists) { > -+ restore_transaction = ref_store_transaction_begin( > ++ if (operation->type == REF_UPDATE_RENAME && > ++ data->destination_exists) { > ++ restore_transaction = ref_store_transaction_begin( > + &refs->base, REF_TRANSACTION_FLAG_SKIP_HOOK, err); > -+ if (!restore_transaction || > -+ ref_transaction_update(restore_transaction, > -+ transaction->new_refname, > -+ data->destination_target ? NULL : > -+ &data->destination_oid, > -+ NULL, > -+ data->destination_target, > -+ NULL, > -+ REF_NO_DEREF | > -+ REF_SKIP_CREATE_REFLOG, > -+ NULL, err) || > -+ ref_transaction_commit(restore_transaction, err)) > -+ ret = -1; > -+ ref_transaction_free(restore_transaction); > -+ } > ++ if (!restore_transaction || > ++ ref_transaction_update(restore_transaction, > ++ update->refname, > ++ data->destination_target ? NULL : > ++ &data->destination_oid, > ++ NULL, > ++ data->destination_target, > ++ NULL, > ++ REF_NO_DEREF | > ++ REF_SKIP_CREATE_REFLOG, > ++ NULL, err) || > ++ ref_transaction_commit(restore_transaction, err)) > ++ ret = -1; > ++ ref_transaction_free(restore_transaction); > ++ } > + > -+ strbuf_release(&destination_log); > -+ strbuf_release(&temporary_log); > -+ strbuf_release(&new_log); > -+ free(data->destination_target); > -+ free(data); > -+ transaction->backend_data = NULL; > -+ transaction->state = REF_TRANSACTION_CLOSED; > -+ return ret; > ++ strbuf_release(&destination_log); > ++ strbuf_release(&temporary_log); > ++ strbuf_release(&new_log); > ++ free(data->destination_target); > ++ free(data); > ++ transaction->backend_data = NULL; > ++ transaction->state = REF_TRANSACTION_CLOSED; > ++ return ret; > ++ } > + } > + > files_transaction_cleanup(refs, transaction); > @@ refs/packed-backend.c: struct ref_storage_be refs_be_packed = { > .read_raw_ref = packed_read_raw_ref, > > ## refs/refs-internal.h ## > +@@ refs/refs-internal.h: struct ref_update { > + */ > + struct ref_update *parent_update; > + > ++ /* > ++ * Copy and rename operations require backend-specific handling while > ++ * still exposing their logical updates to transaction hooks. Keep that > ++ * state on the destination update so it composes with other updates in > ++ * the transaction instead of making copy or rename a transaction-wide > ++ * property. > ++ */ > ++ struct ref_copy_or_rename_update *copy_or_rename; > ++ > + const char refname[FLEX_ARRAY]; > + }; > + > ++enum ref_copy_or_rename_type { > ++ REF_UPDATE_RENAME, > ++ REF_UPDATE_COPY, > ++}; > ++ > ++struct ref_copy_or_rename_update { > ++ enum ref_copy_or_rename_type type; > ++ char *old_refname; > ++ char *logmsg; > ++ struct object_id source_oid; > ++ struct object_id destination_oid; > ++ char *destination_target; > ++ unsigned int destination_exists:1; > ++}; > ++ > + int refs_read_raw_ref(struct ref_store *ref_store, const char *refname, > + struct object_id *oid, struct strbuf *referent, > + unsigned int *type, int *failure_errno); > @@ refs/refs-internal.h: struct ref_update *ref_transaction_add_update( > const char *committer_info, > const char *msg); > @@ refs/refs-internal.h: struct ref_update *ref_transaction_add_update( > /* > * Transaction states. > * > -@@ refs/refs-internal.h: enum ref_transaction_state { > - REF_TRANSACTION_CLOSED = 2 > - }; > - > -+enum ref_transaction_type { > -+ REF_TRANSACTION_TYPE_NORMAL = 0, > -+ REF_TRANSACTION_TYPE_RENAME, > -+ REF_TRANSACTION_TYPE_COPY, > -+}; > -+ > - /* > - * Data structure to hold indices of updates which were rejected, for batched > - * reference updates. While the updates themselves hold the rejection error, > @@ refs/refs-internal.h: struct ref_transaction { > - void *backend_data; > - unsigned int flags; > uint64_t max_index; > -+ > -+ /* > -+ * Rename and copy operations need backend-specific reflog handling. > -+ * Their logical updates still live in `updates`, so hooks see the > -+ * operation like any other reference transaction. The fields below > -+ * retain the state that backends verify after taking their locks. > -+ */ > -+ enum ref_transaction_type type; > -+ char *old_refname; > -+ char *new_refname; > -+ char *logmsg; > -+ struct object_id source_oid; > -+ struct object_id destination_oid; > -+ char *destination_target; > -+ unsigned int destination_exists:1; > }; > > ++/* Suppress hooks for a transaction nested inside another refs operation. */ > ++#define REF_TRANSACTION_FLAG_SKIP_HOOK (1 << 2) > ++ > ++struct ref_update *ref_transaction_copy_or_rename_update( > ++ struct ref_transaction *transaction); > ++ > /* > + * Check for entries in extras that are within the specified > + * directory, where dirname is a reference directory name including > @@ refs/refs-internal.h: typedef int optimize_required_fn(struct ref_store *ref_store, > struct refs_optimize_opts *opts, > bool *required); > @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s > size_t i; > int ret; > > -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) > ++ if (ref_transaction_copy_or_rename_update(transaction)) > + return reftable_be_copy_or_rename_prepare(ref_store, transaction, > + err); > + > @@ refs/reftable-backend.c: static int reftable_be_transaction_abort(struct ref_sto > - struct reftable_transaction_data *tx_data = transaction->backend_data; > + struct reftable_transaction_data *tx_data; > + > -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { > ++ if (ref_transaction_copy_or_rename_update(transaction)) { > + struct reftable_copy_or_rename_transaction_data *data = > + transaction->backend_data; > + > @@ refs/reftable-backend.c: static int reftable_be_transaction_finish(struct ref_st > + struct reftable_transaction_data *tx_data; > int ret = 0; > > -+ if (transaction->type != REF_TRANSACTION_TYPE_NORMAL) { > ++ if (ref_transaction_copy_or_rename_update(transaction)) { > + struct reftable_copy_or_rename_transaction_data *data = > + transaction->backend_data; > + int special_ret = reftable_addition_commit(data->addition); > @@ refs/reftable-backend.c: struct write_create_symref_arg { > const char *newname; > const char *logmsg; > int delete_old; > -+ struct ref_transaction *transaction; > ++ struct ref_copy_or_rename_update *operation; > }; > > static int write_copy_table(struct reftable_writer *writer, void *cb_data) > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > + ret = -1; > goto done; > } > -+ if (arg->transaction) { > ++ if (arg->operation) { > + struct object_id oid; > + > + if (old_ref.value_type == REFTABLE_REF_VAL2) > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > + else > + oidread(&oid, old_ref.value.val1, > + arg->refs->base.repo->hash_algo); > -+ if (!oideq(&oid, &arg->transaction->source_oid)) { > ++ if (!oideq(&oid, &arg->operation->source_oid)) { > + strbuf_addf(arg->err, > + _("refname %s is at %s but expected %s"), > + arg->oldname, oid_to_hex(&oid), > -+ oid_to_hex(&arg->transaction->source_oid)); > ++ oid_to_hex(&arg->operation->source_oid)); > + ret = -1; > + goto done; > + } > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > + &destination_ref); > + if (ret < 0) > + goto done; > -+ if (arg->transaction->destination_exists != !ret) { > ++ if (arg->operation->destination_exists != !ret) { > + strbuf_addf(arg->err, > + _("refname %s changed while renaming"), > + arg->newname); > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > + } > + if (!ret) { > + if (destination_ref.value_type == REFTABLE_REF_SYMREF) { > -+ if (!arg->transaction->destination_target || > ++ if (!arg->operation->destination_target || > + strcmp(destination_ref.value.symref, > -+ arg->transaction->destination_target)) { > ++ arg->operation->destination_target)) { > + strbuf_addf(arg->err, > + _("refname %s changed while renaming"), > + arg->newname); > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > + else > + oidread(&oid, destination_ref.value.val1, > + arg->refs->base.repo->hash_algo); > -+ if (arg->transaction->destination_target || > -+ !oideq(&oid, &arg->transaction->destination_oid)) { > ++ if (arg->operation->destination_target || > ++ !oideq(&oid, &arg->operation->destination_oid)) { > + strbuf_addf(arg->err, > + _("refname %s changed while renaming"), > + arg->newname); > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > + reftable_be_downcast(ref_store, REF_STORE_WRITE, > + "ref_transaction_prepare"); > + struct reftable_copy_or_rename_transaction_data *data = NULL; > ++ struct ref_update *update = > ++ ref_transaction_copy_or_rename_update(transaction); > ++ struct ref_copy_or_rename_update *operation = update->copy_or_rename; > struct write_copy_arg arg = { > .refs = refs, > - .oldname = oldrefname, > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > - .logmsg = logmsg, > - .delete_old = 1, > + .err = err, > -+ .oldname = transaction->old_refname, > -+ .newname = transaction->new_refname, > -+ .logmsg = transaction->logmsg, > -+ .delete_old = transaction->type == REF_TRANSACTION_TYPE_RENAME, > -+ .transaction = transaction, > ++ .oldname = operation->old_refname, > ++ .newname = update->refname, > ++ .logmsg = operation->logmsg, > ++ .delete_old = operation->type == REF_UPDATE_RENAME, > ++ .operation = operation, > }; > int ret; > > @@ refs/reftable-backend.c: static int write_copy_table(struct reftable_writer *wri > goto done; > - > - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); > -+ ret = backend_for(&arg.be, refs, transaction->new_refname, > ++ ret = backend_for(&arg.be, refs, update->refname, > + &arg.newname, 1); > if (ret) > goto done; > > refs.c | 161 ++++++++++++++--- > refs/debug.c | 25 --- > refs/files-backend.c | 297 +++++++++++++++++++++++++++---- > refs/packed-backend.c | 2 - > refs/refs-internal.h | 47 +++-- > refs/reftable-backend.c | 197 +++++++++++++++----- > t/t1416-ref-transaction-hooks.sh | 142 +++++++++++++++ > 7 files changed, 732 insertions(+), 139 deletions(-) > > diff --git a/refs.c b/refs.c > index 92d5df5b7..f036ae4b9 100644 > --- a/refs.c > +++ b/refs.c > @@ -1004,15 +1004,17 @@ long get_files_ref_lock_timeout_ms(struct repository *repo) > return timeout_ms; > } > > -int refs_delete_ref(struct ref_store *refs, const char *msg, > - const char *refname, > - const struct object_id *old_oid, > - unsigned int flags) > +int refs_delete_ref_with_transaction_flags(struct ref_store *refs, > + const char *msg, > + const char *refname, > + const struct object_id *old_oid, > + unsigned int flags, > + unsigned int transaction_flags) > { > struct ref_transaction *transaction; > struct strbuf err = STRBUF_INIT; > > - transaction = ref_store_transaction_begin(refs, 0, &err); > + transaction = ref_store_transaction_begin(refs, transaction_flags, &err); > if (!transaction || > ref_transaction_delete(transaction, refname, old_oid, > NULL, flags, msg, &err) || > @@ -1027,6 +1029,15 @@ int refs_delete_ref(struct ref_store *refs, const char *msg, > return 0; > } > > +int refs_delete_ref(struct ref_store *refs, const char *msg, > + const char *refname, > + const struct object_id *old_oid, > + unsigned int flags) > +{ > + return refs_delete_ref_with_transaction_flags(refs, msg, refname, > + old_oid, flags, 0); > +} > + > static void copy_reflog_msg(struct strbuf *sb, const char *msg) > { > char c; > @@ -1256,11 +1267,20 @@ void ref_transaction_free(struct ref_transaction *transaction) > } > > for (i = 0; i < transaction->nr; i++) { > + struct ref_copy_or_rename_update *operation = > + transaction->updates[i]->copy_or_rename; > + > free(transaction->updates[i]->msg); > free(transaction->updates[i]->committer_info); > free((char *)transaction->updates[i]->new_target); > free((char *)transaction->updates[i]->old_target); > free((char *)transaction->updates[i]->rejection_details); > + if (operation) { > + free(operation->old_refname); > + free(operation->logmsg); > + free(operation->destination_target); > + free(operation); > + } > free(transaction->updates[i]); > } > > @@ -1273,6 +1293,23 @@ void ref_transaction_free(struct ref_transaction *transaction) > free(transaction); > } > > +struct ref_update *ref_transaction_copy_or_rename_update( > + struct ref_transaction *transaction) > +{ > + struct ref_update *operation = NULL; > + size_t i; > + > + for (i = 0; i < transaction->nr; i++) { > + if (!transaction->updates[i]->copy_or_rename) > + continue; > + if (operation) > + BUG("multiple copy or rename updates in one transaction"); > + operation = transaction->updates[i]; > + } > + > + return operation; > +} > + > int ref_transaction_maybe_set_rejected(struct ref_transaction *transaction, > size_t update_idx, > enum ref_transaction_error err, > @@ -2710,7 +2747,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, > return REF_TRANSACTION_ERROR_GENERIC; > > /* Preparing checks before locking references */ > - ret = run_transaction_hook(transaction, "preparing"); > + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : > + run_transaction_hook(transaction, "preparing"); > if (ret) { > ref_transaction_abort(transaction, err); > die(_(abort_by_ref_transaction_hook), "preparing"); > @@ -2720,7 +2758,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, > if (ret) > return ret; > > - ret = run_transaction_hook(transaction, "prepared"); > + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : > + run_transaction_hook(transaction, "prepared"); > if (ret) { > ref_transaction_abort(transaction, err); > die(_(abort_by_ref_transaction_hook), "prepared"); > @@ -2750,7 +2789,8 @@ int ref_transaction_abort(struct ref_transaction *transaction, > break; > } > > - run_transaction_hook(transaction, "aborted"); > + if (!(transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK)) > + run_transaction_hook(transaction, "aborted"); > > ref_transaction_free(transaction); > return ret; > @@ -2781,7 +2821,8 @@ int ref_transaction_commit(struct ref_transaction *transaction, > } > > ret = refs->be->transaction_finish(refs, transaction, err); > - if (!ret && !(transaction->flags & REF_TRANSACTION_FLAG_INITIAL)) > + if (!ret && !(transaction->flags & (REF_TRANSACTION_FLAG_INITIAL | > + REF_TRANSACTION_FLAG_SKIP_HOOK))) > run_transaction_hook(transaction, "committed"); > return ret; > } > @@ -3123,28 +3164,102 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg, > return ret; > } > > -int refs_rename_ref(struct ref_store *refs, const char *oldref, > - const char *newref, const char *logmsg) > +static int refs_copy_or_rename_ref(struct ref_store *refs, const char *oldref, > + const char *newref, const char *logmsg, > + bool copy) > { > - char *msg; > - int retval; > + struct ref_transaction *transaction = NULL; > + struct ref_copy_or_rename_update *operation = NULL; > + struct ref_update *destination_update; > + struct object_id old_oid, new_oid; > + struct strbuf new_target = STRBUF_INIT; > + struct strbuf err = STRBUF_INIT; > + char *msg = normalize_reflog_message(logmsg); > + int old_flags, new_flags = 0, new_exists = 0, ret = 1; > > - msg = normalize_reflog_message(logmsg); > - retval = refs->be->rename_ref(refs, oldref, newref, msg); > + if (!strcmp(oldref, newref)) { > + ret = 0; > + goto out; > + } > + > + if (!refs_resolve_ref_unsafe(refs, oldref, > + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, > + &old_oid, &old_flags)) { > + error("refname %s not found", oldref); > + goto out; > + } > + if (old_flags & REF_ISSYMREF) { > + error("refname %s is a symbolic ref, %s it is not supported", > + oldref, copy ? "copying" : "renaming"); > + goto out; > + } > + > + transaction = ref_store_transaction_begin(refs, 0, &err); > + if (!transaction) > + goto error; > + if (!copy && ref_transaction_delete(transaction, oldref, &old_oid, NULL, > + REF_NO_DEREF, msg, &err)) > + goto error; > + > + if (refs_resolve_ref_unsafe(refs, newref, > + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, > + &new_oid, &new_flags)) { > + new_exists = 1; > + if ((new_flags & REF_ISSYMREF) && > + refs_read_symbolic_ref(refs, newref, &new_target) < 0) { > + strbuf_addf(&err, "unable to read symbolic ref %s", newref); > + goto error; > + } > + } else { > + oidclr(&new_oid, refs->repo->hash_algo); > + } > + if (ref_transaction_update(transaction, newref, &old_oid, > + (new_flags & REF_ISSYMREF) ? NULL : &new_oid, > + NULL, > + (new_flags & REF_ISSYMREF) ? new_target.buf : NULL, > + REF_NO_DEREF | REF_SKIP_CREATE_REFLOG, > + NULL, &err)) > + goto error; > + > + destination_update = transaction->updates[transaction->nr - 1]; > + CALLOC_ARRAY(operation, 1); > + operation->type = copy ? REF_UPDATE_COPY : REF_UPDATE_RENAME; > + operation->old_refname = xstrdup(oldref); > + operation->logmsg = xstrdup(msg); > + oidcpy(&operation->source_oid, &old_oid); > + operation->destination_exists = new_exists; > + if (new_flags & REF_ISSYMREF) > + operation->destination_target = xstrdup(new_target.buf); > + else if (operation->destination_exists) > + oidcpy(&operation->destination_oid, &new_oid); > + destination_update->copy_or_rename = operation; > + > + if (ref_transaction_commit(transaction, &err)) > + goto error; > + > + ret = 0; > + goto out; > + > +error: > + error("%s", err.buf); > +out: > + ref_transaction_free(transaction); > + strbuf_release(&new_target); > + strbuf_release(&err); > free(msg); > - return retval; > + return ret; > } > > -int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, > +int refs_rename_ref(struct ref_store *refs, const char *oldref, > const char *newref, const char *logmsg) > { > - char *msg; > - int retval; > + return refs_copy_or_rename_ref(refs, oldref, newref, logmsg, 0); > +} > > - msg = normalize_reflog_message(logmsg); > - retval = refs->be->copy_ref(refs, oldref, newref, msg); > - free(msg); > - return retval; > +int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, > + const char *newref, const char *logmsg) > +{ > + return refs_copy_or_rename_ref(refs, oldref, newref, logmsg, 1); > } > > const char *ref_update_original_update_refname(struct ref_update *update) > diff --git a/refs/debug.c b/refs/debug.c > index 639db0f26..87b84e767 100644 > --- a/refs/debug.c > +++ b/refs/debug.c > @@ -143,28 +143,6 @@ static int debug_optimize_required(struct ref_store *ref_store, > return res; > } > > -static int debug_rename_ref(struct ref_store *ref_store, const char *oldref, > - const char *newref, const char *logmsg) > -{ > - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; > - int res = drefs->refs->be->rename_ref(drefs->refs, oldref, newref, > - logmsg); > - trace_printf_key(&trace_refs, "rename_ref: %s -> %s \"%s\": %d\n", oldref, newref, > - logmsg, res); > - return res; > -} > - > -static int debug_copy_ref(struct ref_store *ref_store, const char *oldref, > - const char *newref, const char *logmsg) > -{ > - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; > - int res = > - drefs->refs->be->copy_ref(drefs->refs, oldref, newref, logmsg); > - trace_printf_key(&trace_refs, "copy_ref: %s -> %s \"%s\": %d\n", oldref, newref, > - logmsg, res); > - return res; > -} > - > struct debug_ref_iterator { > struct ref_iterator base; > struct ref_iterator *iter; > @@ -453,9 +431,6 @@ struct ref_storage_be refs_be_debug = { > .optimize = debug_optimize, > .optimize_required = debug_optimize_required, > > - .rename_ref = debug_rename_ref, > - .copy_ref = debug_copy_ref, > - > .iterator_begin = debug_ref_iterator_begin, > .read_raw_ref = debug_read_raw_ref, > .read_symbolic_ref = debug_read_symbolic_ref, > diff --git a/refs/files-backend.c b/refs/files-backend.c > index 71628550f..c28228116 100644 > --- a/refs/files-backend.c > +++ b/refs/files-backend.c > @@ -1594,6 +1594,7 @@ static int files_optimize_required(struct ref_store *ref_store, > * live into logs/refs. > */ > #define TMP_RENAMED_LOG "refs/.tmp-renamed-log" > +#define TMP_RENAMED_LOG_DESTINATION "refs/.tmp-renamed-log-destination" > > struct rename_cb { > const char *tmp_renamed_log; > @@ -1685,12 +1686,28 @@ static int refs_rename_ref_available(struct ref_store *refs, > return ok; > } > > +struct files_copy_or_rename_transaction_data { > + struct ref_lock *lock; > + struct object_id orig_oid; > + struct object_id destination_oid; > + char *destination_target; > + int logmoved; > + int destination_exists; > + int destination_log_backed_up; > +}; > + > static int files_copy_or_rename_ref(struct ref_store *ref_store, > - const char *oldrefname, const char *newrefname, > - const char *logmsg, int copy) > + struct ref_update *update, > + struct ref_transaction *transaction) > { > struct files_ref_store *refs = > - files_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); > + files_downcast(ref_store, REF_STORE_WRITE, > + "ref_transaction_prepare"); > + struct ref_copy_or_rename_update *operation = update->copy_or_rename; > + const char *oldrefname = operation->old_refname; > + const char *newrefname = update->refname; > + const char *logmsg = operation->logmsg; > + bool copy = operation->type == REF_UPDATE_COPY; > struct object_id orig_oid; > int flag = 0, logmoved = 0; > struct ref_lock *lock; > @@ -1698,12 +1715,19 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, > struct strbuf sb_oldref = STRBUF_INIT; > struct strbuf sb_newref = STRBUF_INIT; > struct strbuf tmp_renamed_log = STRBUF_INIT; > + struct strbuf tmp_destination_log = STRBUF_INIT; > + struct strbuf destination_target = STRBUF_INIT; > int log, ret; > + int destination_exists = 0, destination_flags = 0; > + int destination_log_backed_up = 0; > + struct object_id destination_oid; > + struct files_copy_or_rename_transaction_data *data; > struct strbuf err = STRBUF_INIT; > > files_reflog_path(refs, &sb_oldref, oldrefname); > files_reflog_path(refs, &sb_newref, newrefname); > files_reflog_path(refs, &tmp_renamed_log, TMP_RENAMED_LOG); > + files_reflog_path(refs, &tmp_destination_log, TMP_RENAMED_LOG_DESTINATION); > > log = !lstat(sb_oldref.buf, &loginfo); > if (log && S_ISLNK(loginfo.st_mode)) { > @@ -1727,11 +1751,67 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, > oldrefname); > goto out; > } > + if (!oideq(&orig_oid, &operation->source_oid)) { > + ret = error("refname %s is at %s but expected %s", > + oldrefname, oid_to_hex(&orig_oid), > + oid_to_hex(&operation->source_oid)); > + goto out; > + } > if (!refs_rename_ref_available(&refs->base, oldrefname, newrefname)) { > ret = 1; > goto out; > } > > + if (refs_resolve_ref_unsafe(&refs->base, newrefname, > + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, > + &destination_oid, &destination_flags)) { > + destination_exists = 1; > + if ((destination_flags & REF_ISSYMREF) && > + refs_read_symbolic_ref(&refs->base, newrefname, > + &destination_target) < 0) { > + ret = error("unable to read symbolic ref %s", newrefname); > + goto out; > + } > + } > + if (destination_exists != operation->destination_exists) { > + ret = error("refname %s changed while renaming", newrefname); > + goto out; > + } > + if (destination_exists) { > + if (destination_flags & REF_ISSYMREF) { > + if (!operation->destination_target || > + strcmp(destination_target.buf, > + operation->destination_target)) { > + ret = error("refname %s changed while renaming", > + newrefname); > + goto out; > + } > + } else if (operation->destination_target || > + !oideq(&destination_oid, > + &operation->destination_oid)) { > + ret = error("refname %s changed while renaming", newrefname); > + goto out; > + } > + } > + > + if (!lstat(sb_newref.buf, &loginfo)) { > + if (S_ISLNK(loginfo.st_mode)) { > + ret = error("reflog for %s is a symlink", newrefname); > + goto out; > + } > + if (S_ISREG(loginfo.st_mode)) { > + if (copy_file(refs->base.repo, tmp_destination_log.buf, > + sb_newref.buf, 0644)) { > + if (errno != EEXIST) > + unlink(tmp_destination_log.buf); > + ret = error("unable to back up logfile logs/%s: %s", > + newrefname, strerror(errno)); > + goto out; > + } > + destination_log_backed_up = 1; > + } > + } > + > if (!copy && log && rename(sb_oldref.buf, tmp_renamed_log.buf)) { > ret = error("unable to move logfile logs/%s to logs/"TMP_RENAMED_LOG": %s", > oldrefname, strerror(errno)); > @@ -1744,8 +1824,10 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, > goto out; > } > > - if (!copy && refs_delete_ref(&refs->base, logmsg, oldrefname, > - &orig_oid, REF_NO_DEREF)) { > + if (!copy && refs_delete_ref_with_transaction_flags(&refs->base, logmsg, > + oldrefname, &orig_oid, > + REF_NO_DEREF, > + REF_TRANSACTION_FLAG_SKIP_HOOK)) { > error("unable to delete old %s", oldrefname); > goto rollback; > } > @@ -1760,8 +1842,9 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, > if (!copy && refs_resolve_ref_unsafe(&refs->base, newrefname, > RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, > NULL, NULL) && > - refs_delete_ref(&refs->base, NULL, newrefname, > - NULL, REF_NO_DEREF)) { > + refs_delete_ref_with_transaction_flags(&refs->base, NULL, newrefname, > + NULL, REF_NO_DEREF, > + REF_TRANSACTION_FLAG_SKIP_HOOK)) { > if (errno == EISDIR) { > struct strbuf path = STRBUF_INIT; > int result; > @@ -1796,13 +1879,25 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, > } > oidcpy(&lock->old_oid, &orig_oid); > > - if (write_ref_to_lockfile(refs, lock, &orig_oid, &err) || > - commit_ref_update(refs, lock, &orig_oid, logmsg, 0, &err)) { > + if (write_ref_to_lockfile(refs, lock, &orig_oid, &err)) { > error("unable to write current sha1 into %s: %s", newrefname, err.buf); > strbuf_release(&err); > goto rollback; > } > > + CALLOC_ARRAY(data, 1); > + data->lock = lock; > + oidcpy(&data->orig_oid, &orig_oid); > + data->logmoved = logmoved; > + data->destination_exists = destination_exists; > + data->destination_log_backed_up = destination_log_backed_up; > + if (destination_exists && !(destination_flags & REF_ISSYMREF)) > + oidcpy(&data->destination_oid, &destination_oid); > + if (destination_flags & REF_ISSYMREF) > + data->destination_target = strbuf_detach(&destination_target, NULL); > + transaction->backend_data = data; > + transaction->state = REF_TRANSACTION_PREPARED; > + > ret = 0; > goto out; > > @@ -1821,38 +1916,40 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store, > } > > rollbacklog: > - if (logmoved && rename(sb_newref.buf, sb_oldref.buf)) > - error("unable to restore logfile %s from %s: %s", > - oldrefname, newrefname, strerror(errno)); > + if (logmoved) { > + if (rename(sb_newref.buf, tmp_renamed_log.buf)) { > + error("unable to restore logfile %s from %s: %s", > + oldrefname, newrefname, strerror(errno)); > + } else { > + try_remove_empty_parents(refs, newrefname, > + REMOVE_EMPTY_PARENTS_REFLOG); > + if (rename_tmp_log(refs, oldrefname)) > + error("unable to restore logfile %s from logs/" > + TMP_RENAMED_LOG ": %s", > + oldrefname, strerror(errno)); > + } > + } > if (!logmoved && log && > rename(tmp_renamed_log.buf, sb_oldref.buf)) > error("unable to restore logfile %s from logs/"TMP_RENAMED_LOG": %s", > oldrefname, strerror(errno)); > + if (destination_log_backed_up && > + rename(tmp_destination_log.buf, sb_newref.buf)) > + error("unable to restore logfile %s: %s", > + newrefname, strerror(errno)); > ret = 1; > out: > + if (ret && destination_log_backed_up) > + unlink(tmp_destination_log.buf); > strbuf_release(&sb_newref); > strbuf_release(&sb_oldref); > strbuf_release(&tmp_renamed_log); > + strbuf_release(&tmp_destination_log); > + strbuf_release(&destination_target); > > return ret; > } > > -static int files_rename_ref(struct ref_store *ref_store, > - const char *oldrefname, const char *newrefname, > - const char *logmsg) > -{ > - return files_copy_or_rename_ref(ref_store, oldrefname, > - newrefname, logmsg, 0); > -} > - > -static int files_copy_ref(struct ref_store *ref_store, > - const char *oldrefname, const char *newrefname, > - const char *logmsg) > -{ > - return files_copy_or_rename_ref(ref_store, oldrefname, > - newrefname, logmsg, 1); > -} > - > static int close_ref_gently(struct ref_lock *lock) > { > if (close_lock_file_gently(&lock->lk)) > @@ -2962,6 +3059,14 @@ static int files_transaction_prepare(struct ref_store *ref_store, > struct ref_transaction *packed_transaction = NULL; > > assert(err); > + { > + struct ref_update *operation = > + ref_transaction_copy_or_rename_update(transaction); > + > + if (operation) > + return files_copy_or_rename_ref(ref_store, operation, > + transaction); > + } > > if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) > goto cleanup; > @@ -3318,6 +3423,10 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, > return ret; > } > > +static int files_transaction_abort(struct ref_store *ref_store, > + struct ref_transaction *transaction, > + struct strbuf *err); > + > static int files_transaction_finish(struct ref_store *ref_store, > struct ref_transaction *transaction, > struct strbuf *err) > @@ -3333,6 +3442,40 @@ static int files_transaction_finish(struct ref_store *ref_store, > > > assert(err); > + { > + struct ref_update *update = > + ref_transaction_copy_or_rename_update(transaction); > + > + if (update) { > + struct ref_copy_or_rename_update *operation = > + update->copy_or_rename; > + struct files_copy_or_rename_transaction_data *data = > + transaction->backend_data; > + int special_ret; > + > + special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, > + operation->logmsg, 0, err); > + if (special_ret) { > + error("unable to write current sha1 into %s: %s", > + update->refname, err->buf); > + data->lock = NULL; > + files_transaction_abort(ref_store, transaction, err); > + return special_ret; > + } else if (data->destination_log_backed_up) { > + struct strbuf path = STRBUF_INIT; > + > + files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); > + if (unlink(path.buf) < 0 && errno != ENOENT) > + warning_errno("unable to remove '%s'", path.buf); > + strbuf_release(&path); > + } > + free(data->destination_target); > + free(data); > + transaction->backend_data = NULL; > + transaction->state = REF_TRANSACTION_CLOSED; > + return special_ret; > + } > + } > > if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) > return files_transaction_finish_initial(refs, transaction, err); > @@ -3476,11 +3619,105 @@ static int files_transaction_finish(struct ref_store *ref_store, > > static int files_transaction_abort(struct ref_store *ref_store, > struct ref_transaction *transaction, > - struct strbuf *err UNUSED) > + struct strbuf *err) > { > struct files_ref_store *refs = > files_downcast(ref_store, 0, "ref_transaction_abort"); > > + { > + struct ref_update *update = > + ref_transaction_copy_or_rename_update(transaction); > + > + if (update) { > + struct ref_copy_or_rename_update *operation = > + update->copy_or_rename; > + struct files_copy_or_rename_transaction_data *data = > + transaction->backend_data; > + struct strbuf new_log = STRBUF_INIT; > + struct strbuf destination_log = STRBUF_INIT; > + struct strbuf temporary_log = STRBUF_INIT; > + struct ref_transaction *restore_transaction = NULL; > + struct ref_lock *lock; > + int ret = 0; > + > + if (data->lock) > + unlock_ref(data->lock); > + if (operation->type == REF_UPDATE_RENAME) { > + lock = lock_ref_oid_basic(refs, operation->old_refname, err); > + if (!lock || > + write_ref_to_lockfile(refs, lock, &data->orig_oid, err) || > + commit_ref_update(refs, lock, &data->orig_oid, NULL, > + REF_SKIP_CREATE_REFLOG, err)) > + ret = -1; > + } > + > + if (data->logmoved) { > + files_reflog_path(refs, &new_log, update->refname); > + if (operation->type == REF_UPDATE_RENAME) { > + files_reflog_path(refs, &temporary_log, TMP_RENAMED_LOG); > + if (rename(new_log.buf, temporary_log.buf) < 0) { > + strbuf_addf(err, "unable to restore logfile %s: %s", > + operation->old_refname, strerror(errno)); > + ret = -1; > + } else { > + try_remove_empty_parents(refs, > + update->refname, > + REMOVE_EMPTY_PARENTS_REFLOG); > + if (rename_tmp_log(refs, > + operation->old_refname)) { > + strbuf_addf(err, "unable to restore logfile %s: %s", > + operation->old_refname, > + strerror(errno)); > + ret = -1; > + } > + } > + } else if (unlink(new_log.buf) < 0 && errno != ENOENT) { > + strbuf_addf(err, "unable to remove logfile %s: %s", > + update->refname, strerror(errno)); > + ret = -1; > + } > + } > + if (data->destination_log_backed_up) { > + files_reflog_path(refs, &destination_log, > + TMP_RENAMED_LOG_DESTINATION); > + if (rename(destination_log.buf, new_log.buf) < 0) { > + strbuf_addf(err, "unable to restore logfile %s: %s", > + update->refname, strerror(errno)); > + ret = -1; > + } > + } > + > + if (operation->type == REF_UPDATE_RENAME && > + data->destination_exists) { > + restore_transaction = ref_store_transaction_begin( > + &refs->base, REF_TRANSACTION_FLAG_SKIP_HOOK, err); > + if (!restore_transaction || > + ref_transaction_update(restore_transaction, > + update->refname, > + data->destination_target ? NULL : > + &data->destination_oid, > + NULL, > + data->destination_target, > + NULL, > + REF_NO_DEREF | > + REF_SKIP_CREATE_REFLOG, > + NULL, err) || > + ref_transaction_commit(restore_transaction, err)) > + ret = -1; > + ref_transaction_free(restore_transaction); > + } > + > + strbuf_release(&destination_log); > + strbuf_release(&temporary_log); > + strbuf_release(&new_log); > + free(data->destination_target); > + free(data); > + transaction->backend_data = NULL; > + transaction->state = REF_TRANSACTION_CLOSED; > + return ret; > + } > + } > + > files_transaction_cleanup(refs, transaction); > return 0; > } > @@ -4095,8 +4332,6 @@ struct ref_storage_be refs_be_files = { > > .optimize = files_optimize, > .optimize_required = files_optimize_required, > - .rename_ref = files_rename_ref, > - .copy_ref = files_copy_ref, > > .iterator_begin = files_ref_iterator_begin, > .read_raw_ref = files_read_raw_ref, > diff --git a/refs/packed-backend.c b/refs/packed-backend.c > index a73fc6aca..364a91291 100644 > --- a/refs/packed-backend.c > +++ b/refs/packed-backend.c > @@ -2164,8 +2164,6 @@ struct ref_storage_be refs_be_packed = { > .optimize = packed_optimize, > .optimize_required = packed_optimize_required, > > - .rename_ref = NULL, > - .copy_ref = NULL, > > .iterator_begin = packed_ref_iterator_begin, > .read_raw_ref = packed_read_raw_ref, > diff --git a/refs/refs-internal.h b/refs/refs-internal.h > index c3ac7b556..5d4dc0171 100644 > --- a/refs/refs-internal.h > +++ b/refs/refs-internal.h > @@ -155,9 +155,33 @@ struct ref_update { > */ > struct ref_update *parent_update; > > + /* > + * Copy and rename operations require backend-specific handling while > + * still exposing their logical updates to transaction hooks. Keep that > + * state on the destination update so it composes with other updates in > + * the transaction instead of making copy or rename a transaction-wide > + * property. > + */ > + struct ref_copy_or_rename_update *copy_or_rename; > + > const char refname[FLEX_ARRAY]; > }; > > +enum ref_copy_or_rename_type { > + REF_UPDATE_RENAME, > + REF_UPDATE_COPY, > +}; > + > +struct ref_copy_or_rename_update { > + enum ref_copy_or_rename_type type; > + char *old_refname; > + char *logmsg; > + struct object_id source_oid; > + struct object_id destination_oid; > + char *destination_target; > + unsigned int destination_exists:1; > +}; > + > int refs_read_raw_ref(struct ref_store *ref_store, const char *refname, > struct object_id *oid, struct strbuf *referent, > unsigned int *type, int *failure_errno); > @@ -187,6 +211,13 @@ struct ref_update *ref_transaction_add_update( > const char *committer_info, > const char *msg); > > +int refs_delete_ref_with_transaction_flags(struct ref_store *refs, > + const char *msg, > + const char *refname, > + const struct object_id *old_oid, > + unsigned int flags, > + unsigned int transaction_flags); > + > /* > * Transaction states. > * > @@ -242,6 +273,12 @@ struct ref_transaction { > uint64_t max_index; > }; > > +/* Suppress hooks for a transaction nested inside another refs operation. */ > +#define REF_TRANSACTION_FLAG_SKIP_HOOK (1 << 2) > + > +struct ref_update *ref_transaction_copy_or_rename_update( > + struct ref_transaction *transaction); > + > /* > * Check for entries in extras that are within the specified > * directory, where dirname is a reference directory name including > @@ -451,13 +488,6 @@ typedef int optimize_required_fn(struct ref_store *ref_store, > struct refs_optimize_opts *opts, > bool *required); > > -typedef int rename_ref_fn(struct ref_store *ref_store, > - const char *oldref, const char *newref, > - const char *logmsg); > -typedef int copy_ref_fn(struct ref_store *ref_store, > - const char *oldref, const char *newref, > - const char *logmsg); > - > /* > * Iterate over the references in `ref_store` whose names start with > * `prefix`. `prefix` is matched as a literal string, without regard > @@ -577,9 +607,6 @@ struct ref_storage_be { > > optimize_fn *optimize; > optimize_required_fn *optimize_required; > - rename_ref_fn *rename_ref; > - copy_ref_fn *copy_ref; > - > ref_iterator_begin_fn *iterator_begin; > read_raw_ref_fn *read_raw_ref; > > diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c > index 10db03991..589fcc998 100644 > --- a/refs/reftable-backend.c > +++ b/refs/reftable-backend.c > @@ -953,6 +953,14 @@ struct reftable_transaction_data { > size_t args_nr, args_alloc; > }; > > +struct reftable_copy_or_rename_transaction_data { > + struct reftable_addition *addition; > +}; > + > +static int reftable_be_copy_or_rename_prepare(struct ref_store *ref_store, > + struct ref_transaction *transaction, > + struct strbuf *err); > + > static void free_transaction_data(struct reftable_transaction_data *tx_data) > { > if (!tx_data) > @@ -1326,6 +1334,10 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, > size_t i; > int ret; > > + if (ref_transaction_copy_or_rename_update(transaction)) > + return reftable_be_copy_or_rename_prepare(ref_store, transaction, > + err); > + > ret = refs->err; > if (ret < 0) > goto done; > @@ -1419,7 +1431,20 @@ static int reftable_be_transaction_abort(struct ref_store *ref_store UNUSED, > struct ref_transaction *transaction, > struct strbuf *err UNUSED) > { > - struct reftable_transaction_data *tx_data = transaction->backend_data; > + struct reftable_transaction_data *tx_data; > + > + if (ref_transaction_copy_or_rename_update(transaction)) { > + struct reftable_copy_or_rename_transaction_data *data = > + transaction->backend_data; > + > + reftable_addition_destroy(data->addition); > + free(data); > + transaction->backend_data = NULL; > + transaction->state = REF_TRANSACTION_CLOSED; > + return 0; > + } > + > + tx_data = transaction->backend_data; > free_transaction_data(tx_data); > transaction->state = REF_TRANSACTION_CLOSED; > return 0; > @@ -1667,9 +1692,28 @@ static int reftable_be_transaction_finish(struct ref_store *ref_store UNUSED, > struct ref_transaction *transaction, > struct strbuf *err) > { > - struct reftable_transaction_data *tx_data = transaction->backend_data; > + struct reftable_transaction_data *tx_data; > int ret = 0; > > + if (ref_transaction_copy_or_rename_update(transaction)) { > + struct reftable_copy_or_rename_transaction_data *data = > + transaction->backend_data; > + int special_ret = reftable_addition_commit(data->addition); > + > + reftable_addition_destroy(data->addition); > + free(data); > + transaction->backend_data = NULL; > + transaction->state = REF_TRANSACTION_CLOSED; > + if (special_ret < 0) { > + strbuf_addf(err, _("reftable: transaction failure: %s"), > + reftable_error_str(special_ret)); > + return -1; > + } > + return 0; > + } > + > + tx_data = transaction->backend_data; > + > for (size_t i = 0; i < tx_data->args_nr; i++) { > tx_data->args[i].max_index = transaction->max_index; > > @@ -1764,17 +1808,20 @@ struct write_create_symref_arg { > struct write_copy_arg { > struct reftable_ref_store *refs; > struct reftable_backend *be; > + struct strbuf *err; > const char *oldname; > const char *newname; > const char *logmsg; > int delete_old; > + struct ref_copy_or_rename_update *operation; > }; > > static int write_copy_table(struct reftable_writer *writer, void *cb_data) > { > struct write_copy_arg *arg = cb_data; > uint64_t deletion_ts, creation_ts; > - struct reftable_ref_record old_ref = {0}, refs[2] = {0}; > + struct reftable_ref_record old_ref = {0}, destination_ref = {0}; > + struct reftable_ref_record refs[2] = {0}; > struct reftable_log_record old_log = {0}, *logs = NULL; > struct reftable_iterator it = {0}; > struct string_list skip = STRING_LIST_INIT_NODUP; > @@ -1789,14 +1836,75 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) > BUG("failed splitting committer info"); > > if (reftable_stack_read_ref(arg->be->stack, arg->oldname, &old_ref)) { > - ret = error(_("refname %s not found"), arg->oldname); > + strbuf_addf(arg->err, _("refname %s not found"), arg->oldname); > + ret = -1; > goto done; > } > if (old_ref.value_type == REFTABLE_REF_SYMREF) { > - ret = error(_("refname %s is a symbolic ref, copying it is not supported"), > + strbuf_addf(arg->err, > + _("refname %s is a symbolic ref, copying it is not supported"), > arg->oldname); > + ret = -1; > goto done; > } > + if (arg->operation) { > + struct object_id oid; > + > + if (old_ref.value_type == REFTABLE_REF_VAL2) > + oidread(&oid, old_ref.value.val2.value, > + arg->refs->base.repo->hash_algo); > + else > + oidread(&oid, old_ref.value.val1, > + arg->refs->base.repo->hash_algo); > + if (!oideq(&oid, &arg->operation->source_oid)) { > + strbuf_addf(arg->err, > + _("refname %s is at %s but expected %s"), > + arg->oldname, oid_to_hex(&oid), > + oid_to_hex(&arg->operation->source_oid)); > + ret = -1; > + goto done; > + } > + > + ret = reftable_stack_read_ref(arg->be->stack, arg->newname, > + &destination_ref); > + if (ret < 0) > + goto done; > + if (arg->operation->destination_exists != !ret) { > + strbuf_addf(arg->err, > + _("refname %s changed while renaming"), > + arg->newname); > + ret = -1; > + goto done; > + } > + if (!ret) { > + if (destination_ref.value_type == REFTABLE_REF_SYMREF) { > + if (!arg->operation->destination_target || > + strcmp(destination_ref.value.symref, > + arg->operation->destination_target)) { > + strbuf_addf(arg->err, > + _("refname %s changed while renaming"), > + arg->newname); > + ret = -1; > + goto done; > + } > + } else { > + if (destination_ref.value_type == REFTABLE_REF_VAL2) > + oidread(&oid, destination_ref.value.val2.value, > + arg->refs->base.repo->hash_algo); > + else > + oidread(&oid, destination_ref.value.val1, > + arg->refs->base.repo->hash_algo); > + if (arg->operation->destination_target || > + !oideq(&oid, &arg->operation->destination_oid)) { > + strbuf_addf(arg->err, > + _("refname %s changed while renaming"), > + arg->newname); > + ret = -1; > + goto done; > + } > + } > + } > + } > > /* > * There's nothing to do in case the old and new name are the same, so > @@ -1815,7 +1923,7 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) > ret = refs_verify_refname_available(&arg->refs->base, arg->newname, > NULL, &skip, 0, &errbuf); > if (ret < 0) { > - error("%s", errbuf.buf); > + strbuf_addbuf(arg->err, &errbuf); > goto done; > } > > @@ -1980,68 +2088,63 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) > for (i = 0; i < ARRAY_SIZE(refs); i++) > reftable_ref_record_release(&refs[i]); > reftable_ref_record_release(&old_ref); > + reftable_ref_record_release(&destination_ref); > reftable_log_record_release(&old_log); > return ret; > } > > -static int reftable_be_rename_ref(struct ref_store *ref_store, > - const char *oldrefname, > - const char *newrefname, > - const char *logmsg) > +static int reftable_be_copy_or_rename_prepare(struct ref_store *ref_store, > + struct ref_transaction *transaction, > + struct strbuf *err) > { > struct reftable_ref_store *refs = > - reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); > + reftable_be_downcast(ref_store, REF_STORE_WRITE, > + "ref_transaction_prepare"); > + struct reftable_copy_or_rename_transaction_data *data = NULL; > + struct ref_update *update = > + ref_transaction_copy_or_rename_update(transaction); > + struct ref_copy_or_rename_update *operation = update->copy_or_rename; > struct write_copy_arg arg = { > .refs = refs, > - .oldname = oldrefname, > - .newname = newrefname, > - .logmsg = logmsg, > - .delete_old = 1, > + .err = err, > + .oldname = operation->old_refname, > + .newname = update->refname, > + .logmsg = operation->logmsg, > + .delete_old = operation->type == REF_UPDATE_RENAME, > + .operation = operation, > }; > int ret; > > + CALLOC_ARRAY(data, 1); > ret = refs->err; > if (ret < 0) > goto done; > - > - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); > + ret = backend_for(&arg.be, refs, update->refname, > + &arg.newname, 1); > if (ret) > goto done; > - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, > - &reftable_be_write_options(refs)->opts); > - > -done: > - assert(ret != REFTABLE_API_ERROR); > - return ret; > -} > - > -static int reftable_be_copy_ref(struct ref_store *ref_store, > - const char *oldrefname, > - const char *newrefname, > - const char *logmsg) > -{ > - struct reftable_ref_store *refs = > - reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); > - struct write_copy_arg arg = { > - .refs = refs, > - .oldname = oldrefname, > - .newname = newrefname, > - .logmsg = logmsg, > - }; > - int ret; > - > - ret = refs->err; > - if (ret < 0) > + ret = reftable_stack_addition_new(&data->addition, arg.be->stack, > + &reftable_be_write_options(refs)->opts); > + if (ret) > goto done; > - > - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); > + ret = reftable_addition_add(data->addition, &write_copy_table, &arg); > if (ret) > goto done; > - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, > - &reftable_be_write_options(refs)->opts); > + > + transaction->backend_data = data; > + transaction->state = REF_TRANSACTION_PREPARED; > + return 0; > > done: > assert(ret != REFTABLE_API_ERROR); > + if (data) { > + reftable_addition_destroy(data->addition); > + free(data); > + } > + transaction->state = REF_TRANSACTION_CLOSED; > + if (ret && !err->len) > + strbuf_addf(err, _("reftable: transaction prepare: %s"), > + reftable_error_str(ret)); > return ret; > } > > @@ -2872,8 +2975,6 @@ struct ref_storage_be refs_be_reftable = { > .optimize = reftable_be_optimize, > .optimize_required = reftable_be_optimize_required, > > - .rename_ref = reftable_be_rename_ref, > - .copy_ref = reftable_be_copy_ref, > > .iterator_begin = reftable_be_iterator_begin, > .read_raw_ref = reftable_be_read_raw_ref, > diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh > index 4fe9d9b23..116b2ff07 100755 > --- a/t/t1416-ref-transaction-hooks.sh > +++ b/t/t1416-ref-transaction-hooks.sh > @@ -93,6 +93,148 @@ test_expect_success 'hook gets all queued updates in committed state' ' > test_cmp expect actual > ' > > +test_expect_success 'hook gets both updates when renaming a branch' ' > + test_when_finished "rm -f actual" && > + git branch old PRE && > + test_hook reference-transaction <<-\EOF && > + echo "$1" >>actual && > + cat >>actual > + EOF > + cat >expect <<-EOF && > + preparing > + $PRE_OID $ZERO_OID refs/heads/old > + $ZERO_OID $PRE_OID refs/heads/new > + prepared > + $PRE_OID $ZERO_OID refs/heads/old > + $ZERO_OID $PRE_OID refs/heads/new > + committed > + $PRE_OID $ZERO_OID refs/heads/old > + $ZERO_OID $PRE_OID refs/heads/new > + EOF > + git branch -m old new && > + test_cmp expect actual && > + test_must_fail git rev-parse --verify refs/heads/old && > + test_cmp_rev PRE refs/heads/new > +' > + > +test_expect_success 'hook gets destination update when copying a branch' ' > + test_when_finished "rm -f actual" && > + git branch copy-source PRE && > + test_hook reference-transaction <<-\EOF && > + echo "$1" >>actual && > + cat >>actual > + EOF > + cat >expect <<-EOF && > + preparing > + $ZERO_OID $PRE_OID refs/heads/copy-destination > + prepared > + $ZERO_OID $PRE_OID refs/heads/copy-destination > + committed > + $ZERO_OID $PRE_OID refs/heads/copy-destination > + EOF > + git branch -c copy-source copy-destination && > + test_cmp expect actual && > + test_cmp_rev PRE refs/heads/copy-source && > + test_cmp_rev PRE refs/heads/copy-destination > +' > + > +test_expect_success 'hook gets overwritten values for forced rename and copy' ' > + git branch force-old PRE && > + git branch force-new POST && > + git branch force-copy-source PRE && > + git branch force-copy-destination POST && > + test_hook reference-transaction <<-\EOF && > + if test "$1" = committed > + then > + cat >>actual > + fi > + EOF > + git branch -M force-old force-new && > + git branch -C force-copy-source force-copy-destination && > + cat >expect <<-EOF && > + $PRE_OID $ZERO_OID refs/heads/force-old > + $POST_OID $PRE_OID refs/heads/force-new > + $POST_OID $PRE_OID refs/heads/force-copy-destination > + EOF > + test_cmp expect actual > +' > + > +test_expect_success 'hook can abort a branch rename after preparation' ' > + git branch abort-old PRE && > + git branch abort-new POST && > + git reflog show --format=%gs abort-old >old-log && > + git reflog show --format=%gs abort-new >new-log && > + test_hook reference-transaction <<-\EOF && > + test "$1" != prepared > + EOF > + test_must_fail git branch -M abort-old abort-new && > + test_cmp_rev PRE refs/heads/abort-old && > + test_cmp_rev POST refs/heads/abort-new && > + git reflog show --format=%gs abort-old >old-log-after && > + git reflog show --format=%gs abort-new >new-log-after && > + test_cmp old-log old-log-after && > + test_cmp new-log new-log-after > +' > + > +test_expect_success 'hook can abort a D/F branch rename after preparation' ' > + git branch df-old PRE && > + git reflog show --format=%gs df-old >df-log && > + test_hook reference-transaction <<-\EOF && > + test "$1" != prepared > + EOF > + test_must_fail git branch -m df-old df-old/child && > + test_cmp_rev PRE refs/heads/df-old && > + test_must_fail git rev-parse --verify refs/heads/df-old/child && > + git reflog show --format=%gs df-old >df-log-after && > + test_cmp df-log df-log-after > +' > + > +test_expect_success 'hook can abort a reverse D/F rename after preparation' ' > + git branch reverse/old PRE && > + git reflog show --format=%gs reverse/old >reverse-log && > + test_hook reference-transaction <<-\EOF && > + test "$1" != prepared > + EOF > + test_must_fail git branch -m reverse/old reverse && > + test_cmp_rev PRE refs/heads/reverse/old && > + test_must_fail git rev-parse --verify refs/heads/reverse && > + git reflog show --format=%gs reverse/old >reverse-log-after && > + test_cmp reverse-log reverse-log-after > +' > + > +test_expect_success 'hook can abort a forced branch copy after preparation' ' > + git branch copy-abort-old PRE && > + git branch copy-abort-new POST && > + git reflog show --format=%gs copy-abort-old >copy-old-log && > + git reflog show --format=%gs copy-abort-new >copy-new-log && > + test_hook reference-transaction <<-\EOF && > + test "$1" != prepared > + EOF > + test_must_fail git branch -C copy-abort-old copy-abort-new && > + test_cmp_rev PRE refs/heads/copy-abort-old && > + test_cmp_rev POST refs/heads/copy-abort-new && > + git reflog show --format=%gs copy-abort-old >copy-old-log-after && > + git reflog show --format=%gs copy-abort-new >copy-new-log-after && > + test_cmp copy-old-log copy-old-log-after && > + test_cmp copy-new-log copy-new-log-after > +' > + > +test_expect_success 'branch rename detects an update during preparing hook' ' > + git branch race-old PRE && > + git branch race-new POST && > + test_hook reference-transaction <<-\EOF && > + marker=$(git rev-parse --git-path rename-race-once) > + if test "$1" = preparing && test ! -e "$marker" > + then > + >"$marker" && > + git update-ref refs/heads/race-old POST > + fi > + EOF > + test_must_fail git branch -M race-old race-new && > + test_cmp_rev POST refs/heads/race-old && > + test_cmp_rev POST refs/heads/race-new > +' > + > test_expect_success 'hook gets all queued updates in aborted state' ' > test_when_finished "rm actual" && > git reset --hard PRE && > -- > 2.39.3 (Apple Git-146) ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2] refs: run copy and rename through transactions 2026-09-23 13:36 ` [PATCH v2] " Maciej Ciemborowicz 2026-09-30 3:32 ` Maciej Ciemborowicz @ 2026-10-02 10:56 ` Patrick Steinhardt 2026-10-02 14:16 ` Maciej Ciemborowicz 1 sibling, 1 reply; 35+ messages in thread From: Patrick Steinhardt @ 2026-10-02 10:56 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: git, gitster, karthik.188 On Wed, Sep 23, 2026 at 03:36:51PM +0200, Maciej Ciemborowicz wrote: > Reference copy and rename operations bypass the transaction API. > Consequently, the reference-transaction hook sees only the source deletion > with the files backend and no useful update with the reftable backend. > > Represent both operations as reference transactions containing their > logical updates. A rename is a deletion of the old reference and creation > of the new reference in the same transaction. Attach operation-specific > state to the destination update instead of making copy or rename a property > of the entire transaction. Sorry, but what does this last sentence mean? What is the consequence of it? > Retain backend-specific reflog handling: the files backend stages its > existing rename procedure across prepare, finish and abort, while reftable > stages an addition while holding the stack lock. Suppress hooks for the > files backend's nested deletion transactions so that callers observe one > logical transaction. The fact that we retain the backend-specific logic is not really interesting by itself. The way more interesting question is _why_ we retain it. Or asked differently, why can't we make this whole mechanism completely agnostic of the backend and implement this via pure transactions? > Record and verify the source and destination values after taking backend > locks. This rejects concurrent changes instead of applying a rename or copy > that differs from the payload shown to the preparing hook. Preserve D/F > renames and restore overwritten references and reflogs when a prepared hook > rejects the operation. Is this new behaviour? Is this retaining old behaviour? I have no clue. > Add tests covering rename, copy, forced updates, both directions of D/F > conflicts, concurrent updates and prepared-hook rollback. This sentence doesn't really add much value to the message. How does all of this impact performance? > diff --git a/refs.c b/refs.c > index 92d5df5b7..f036ae4b9 100644 > --- a/refs.c > +++ b/refs.c > @@ -1027,6 +1029,15 @@ int refs_delete_ref(struct ref_store *refs, const char *msg, > return 0; > } > > +int refs_delete_ref(struct ref_store *refs, const char *msg, > + const char *refname, > + const struct object_id *old_oid, > + unsigned int flags) > +{ > + return refs_delete_ref_with_transaction_flags(refs, msg, refname, > + old_oid, flags, 0); > +} > + > static void copy_reflog_msg(struct strbuf *sb, const char *msg) > { > char c; Refactorings like these could easily go into a separate commit to make this easier to review. > @@ -2710,7 +2747,8 @@ int ref_transaction_prepare(struct ref_transaction *transaction, > return REF_TRANSACTION_ERROR_GENERIC; > > /* Preparing checks before locking references */ > - ret = run_transaction_hook(transaction, "preparing"); > + ret = transaction->flags & REF_TRANSACTION_FLAG_SKIP_HOOK ? 0 : > + run_transaction_hook(transaction, "preparing"); > if (ret) { > ref_transaction_abort(transaction, err); > die(_(abort_by_ref_transaction_hook), "preparing"); Instead of teaching every site to conditionally call `run_transaction_hook()` only when the flag is not set, can't we adapt the function itself to skip? In any case, this is another change that could easily be split out into a separate commit. > diff --git a/refs/files-backend.c b/refs/files-backend.c > index 71628550f..c28228116 100644 > --- a/refs/files-backend.c > +++ b/refs/files-backend.c > @@ -2962,6 +3059,14 @@ static int files_transaction_prepare(struct ref_store *ref_store, > struct ref_transaction *packed_transaction = NULL; > > assert(err); > + { > + struct ref_update *operation = > + ref_transaction_copy_or_rename_update(transaction); > + > + if (operation) > + return files_copy_or_rename_ref(ref_store, operation, > + transaction); > + } > > if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) > goto cleanup; I know this is a construct that AI loves, but that's not following our coding style. > @@ -3333,6 +3442,40 @@ static int files_transaction_finish(struct ref_store *ref_store, > > > assert(err); > + { > + struct ref_update *update = > + ref_transaction_copy_or_rename_update(transaction); > + > + if (update) { > + struct ref_copy_or_rename_update *operation = > + update->copy_or_rename; > + struct files_copy_or_rename_transaction_data *data = > + transaction->backend_data; > + int special_ret; > + > + special_ret = commit_ref_update(refs, data->lock, &data->orig_oid, > + operation->logmsg, 0, err); > + if (special_ret) { > + error("unable to write current sha1 into %s: %s", > + update->refname, err->buf); > + data->lock = NULL; > + files_transaction_abort(ref_store, transaction, err); > + return special_ret; > + } else if (data->destination_log_backed_up) { > + struct strbuf path = STRBUF_INIT; > + > + files_reflog_path(refs, &path, TMP_RENAMED_LOG_DESTINATION); > + if (unlink(path.buf) < 0 && errno != ENOENT) > + warning_errno("unable to remove '%s'", path.buf); > + strbuf_release(&path); > + } > + free(data->destination_target); > + free(data); > + transaction->backend_data = NULL; > + transaction->state = REF_TRANSACTION_CLOSED; > + return special_ret; > + } > + } > > if (transaction->flags & REF_TRANSACTION_FLAG_INITIAL) > return files_transaction_finish_initial(refs, transaction, err); Yeah... > @@ -3476,11 +3619,105 @@ static int files_transaction_finish(struct ref_store *ref_store, > > static int files_transaction_abort(struct ref_store *ref_store, > struct ref_transaction *transaction, > - struct strbuf *err UNUSED) > + struct strbuf *err) > { > struct files_ref_store *refs = > files_downcast(ref_store, 0, "ref_transaction_abort"); > > + { > + struct ref_update *update = > + ref_transaction_copy_or_rename_update(transaction); ... really? Sorry, but I'm going to stop reading here. This is not in a state that is reviewable and has way too much stuff that is obviously generated by an AI without much thought being put into it by the author. I don't want to invest my time into a topic where the author has obviously not spent their time thinking about it, either. Patrick ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2] refs: run copy and rename through transactions 2026-10-02 10:56 ` Patrick Steinhardt @ 2026-10-02 14:16 ` Maciej Ciemborowicz 2026-10-05 6:03 ` Patrick Steinhardt 0 siblings, 1 reply; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-02 14:16 UTC (permalink / raw) To: Patrick Steinhardt; +Cc: git, gitster, karthik.188 On Fri, Oct 2, 2026 at 12:56 PM Patrick Steinhardt <ps@pks.im> wrote: > Sorry, but what does this last sentence mean? What is the consequence > of it? The intent was to address Junio's comment about making copy/rename a property of the whole ref_transaction. In v2 the extra state is attached to the destination ref_update instead. > why can't we make this whole mechanism completely agnostic of the > backend and implement this via pure transactions? The part I was trying to preserve is the existing reflog semantics. A normal ref transaction can express the logical ref updates. In example deleting the old ref and creating/updating the destination. But branch rename/copy also moves or copies the existing reflog history. For the files backend that currently involves filesystem-level reflog rename/copy and D/F handling, while reftable represents the same operation differently. So my assumption was that the logical ref updates could go through the generic transaction API, while the reflog-history operation would remain backend-specific. > Is this new behaviour? Is this retaining old behaviour? The source/destination revalidation is new validation required by introducing the preparing hook before the backend locks are taken. The hook can itself change one of the refs. Without revalidation, the hook payload could describe one state while the rename/copy later operates on another state. The intention is therefore to reject an operation when the state observed by the preparing hook is no longer the state being committed. > How does all of this impact performance? Enabling reference-transaction for rename/copy naturally adds the cost of invoking the hook when one is installed. I measured `git branch -m` and `git branch -c`. Each result is the median of five blocks of 40 commands per version: Hook Before After Change files branch -m no 5.089 ms 5.234 ms +3.8% files branch -m yes 12.608 ms 11.134 ms -10.8% files branch -c no 4.753 ms 4.916 ms +3.4% files branch -c yes 5.509 ms 12.732 ms +137.7% reftable branch -m no 6.478 ms 6.998 ms +7.8% reftable branch -m yes 5.918 ms 13.229 ms +114.6% reftable branch -c no 5.620 ms 6.368 ms +10.9% reftable branch -c yes 6.261 ms 12.354 ms +97.0% > Sorry, but I'm going to stop reading here. This is not in a state that > is reviewable and has way too much stuff that is obviously generated by > an AI without much thought being put into it by the author. I don't want > to invest my time into a topic where the author has obviously not spent > their time thinking about it, either. I'm really sorry to hear that. Yes, the patches I prepared were AI-assisted, but I do feel that I understand what I am doing. I would appreciate some understanding, though, as I do not work with C on a daily basis. The bug report and my attempt to fix it came from the fact that I am working on a Ruby gem for per-branch and per-worktree containerization. That is why I had to write git-hooks-ext, which is how I ended up running into this bug in the first place. I am not insisting that my patch should be merged. I simply thought that submitting a patch might help get the bug fixed faster, and getting the bug fixed is what I care about most. Karthik Nayak offered to help fix it, so perhaps it would be better for someone who works with C on a daily basis to take it over. I can, of course, also prepare a v3, split it into more commits, and explain my reasoning more clearly. But I cannot guarantee that it will meet your standards, simply because I am not yet familiar with them. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2] refs: run copy and rename through transactions 2026-10-02 14:16 ` Maciej Ciemborowicz @ 2026-10-05 6:03 ` Patrick Steinhardt 0 siblings, 0 replies; 35+ messages in thread From: Patrick Steinhardt @ 2026-10-05 6:03 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: git, gitster, karthik.188 On Fri, Oct 02, 2026 at 04:16:08PM +0200, Maciej Ciemborowicz wrote: > On Fri, Oct 2, 2026 at 12:56 PM Patrick Steinhardt <ps@pks.im> wrote: > > why can't we make this whole mechanism completely agnostic of the > > backend and implement this via pure transactions? > > The part I was trying to preserve is the existing reflog semantics. A > normal ref transaction can express the logical ref updates. In example > deleting the old ref and creating/updating the destination. But branch > rename/copy also moves or copies the existing reflog history. For the > files backend that currently involves filesystem-level reflog > rename/copy and D/F handling, while reftable represents the same > operation differently. So my assumption was that the logical ref > updates could go through the generic transaction API, while the > reflog-history operation would remain backend-specific. Yes, the reflog semantics should of course stay the same. But nowadays, this would also be achievable with only backend-agnostic logic as the reference transactions have learned to write many reflog entries for a single reference. This was added back when we introduced the migration logic to convert between two different backends. Now there's potentially two caveats: - I don't think we have a way to delete many old reflog entries yet. - There may be a significant impact on performance. The question thus is whether we can avoid or fix those caveats somehow and thus arrive at a more future-proof mechanism. > > Is this new behaviour? Is this retaining old behaviour? > > The source/destination revalidation is new validation required by > introducing the preparing hook before the backend locks are taken. The > hook can itself change one of the refs. Without revalidation, the hook > payload could describe one state while the rename/copy later operates > on another state. The intention is therefore to reject an operation > when the state observed by the preparing hook is no longer the state > being committed. I don't feel like that's sensible. The "preparing" hook is explicitly run before we perform locking and is documented as such. So it is fully expected that the on-disk state may still change between executing this and the "prepared" phase. It is the responsibility of the hook author to handle such cases, we shouldn't do this ourselves as we're now starting to assume semantics of the hook itself. > > Sorry, but I'm going to stop reading here. This is not in a state that > > is reviewable and has way too much stuff that is obviously generated by > > an AI without much thought being put into it by the author. I don't want > > to invest my time into a topic where the author has obviously not spent > > their time thinking about it, either. > > I'm really sorry to hear that. Yes, the patches I prepared were > AI-assisted, but I do feel that I understand what I am doing. I would > appreciate some understanding, though, as I do not work with C on a > daily basis. The bug report and my attempt to fix it came from the > fact that I am working on a Ruby gem for per-branch and per-worktree > containerization. That is why I had to write git-hooks-ext, which is > how I ended up running into this bug in the first place. > > I am not insisting that my patch should be merged. I simply thought > that submitting a patch might help get the bug fixed faster, and > getting the bug fixed is what I care about most. Karthik Nayak offered > to help fix it, so perhaps it would be better for someone who works > with C on a daily basis to take it over. > > I can, of course, also prepare a v3, split it into more commits, and > explain my reasoning more clearly. But I cannot guarantee that it will > meet your standards, simply because I am not yet familiar with them. I'd suggest to iterate then. In the current version this patch is not in a shape that is ready for review. The patch needs to be split up, and there are a lot of gaps in the commit message. Taken together that gives the signal that you don't really understand what you are doing. That doesn't mean that you cannot fix that with another iteration though. But I'd suggest to take your time prepping the next iteration to read through the code, understand the concepts and doubt what AI spits out. Thanks! Patrick ^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v3 0/4] refs: run copy and rename through transactions 2026-09-20 16:50 ` [PATCH] refs: run copy and rename through transactions Maciej Ciemborowicz 2026-09-21 17:54 ` Junio C Hamano 2026-09-23 13:36 ` [PATCH v2] " Maciej Ciemborowicz @ 2026-10-07 18:05 ` Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz ` (4 more replies) 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz 3 siblings, 5 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-07 18:05 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Reference copy and rename bypass the transaction API. With files, the reference-transaction hook sees only the source deletion; with reftable, it sees neither endpoint. This series puts the logical ref updates and the reflog history into ordinary transactions so hooks can observe and reject the complete operation. Thanks to Patrick, Junio and Karthik for their feedback. Following the review of v2, this version uses ref_transaction_update_reflog(), the API used by backend migration, to replay history. A new ref_transaction_replace_reflog() operation supplies the missing ability to discard destination history before installing the queued entries. Changes since v2: * Split the work into four patches: internal hook suppression, reflog replacement, copy/rename integration, and removal of the old callbacks. * Remove the special copy/rename dispatch from backend prepare, finish and abort. Each destination points to its source update, so multiple copies, renames and ordinary updates can share one transaction. * Keep the hook-suppression flag private and check it centrally in run_transaction_hook(). It is used for the physical packed-refs child transaction, whose changes the parent already reports. * Drop the pre-lock snapshot revalidation. A preparing hook may change the source; prepared and committed report the value read under lock. Copy sources are locked but are not reported as changes to the hook. * Stage reflog replacements during prepare. A prepared veto discards staging files without restoring old values over live refs or changing the source, destination or HEAD history. * Add coverage for complete reflog contents, mixed transactions, hook vetoes, source races and failure paths, and measure the performance cost of replaying history. Backend-specific work remains necessary to implement the transaction primitives. Files stages logs beside logs/refs using unique temporary files. For directory/file conflicts and case-only renames, it queues the destination in packed-refs: the loose source cannot remain visible while also making room for a loose destination lock. During finish, a backup preserves the source log until the destination log is installed; an installation error restores that backup. Reftable writes tombstones for old destination entries in the same table as the replacement. The backends' final reflog records remain distinct: files appends old->old; reftable appends old->zero and zero->old for rename, or zero->old for copy. Forced reftable copies do change behavior: unrelated destination history is replaced by source history, matching files when the source has a reflog. Replaying history requires O(N) time and memory. Buffered staging avoids a write system call per entry, but files rename loses its constant-time reflog move. On macOS/arm64, with no hook configured, median milliseconds per operation over seven samples of ten operations were: entries base v3 files copy 10 5.05 5.08 files rename 10 4.88 5.26 files copy 10000 13.20 16.11 files rename 10000 5.09 17.09 reftable copy 10 5.25 6.11 reftable rename 10 5.81 6.50 reftable copy 10000 53.39 68.06 reftable rename 10000 43.44 54.06 Process startup is included. Copy overwrites the same destination; rename alternates between two names. Reftable starts from a migration of the same files fixture. Repeated operations add normal log entries and incur normal reftable compaction. Patch 3 adds t/perf/p1424-ref-copy-rename.sh; these measurements used a separate monotonic-clock driver because GNU time is not installed here. The files rename regression is a cost of this design, not just hook overhead. Validation covered 15 relevant suites with files and reftable, including branch, update-ref, hooks, migration, reflogs and worktree refs. The two new suites contain 34 tests, with backend-specific skips; they also passed with SHA-256 and AddressSanitizer/UndefinedBehaviorSanitizer. The reflog prerequisite and main change were tested as intermediate trees. This was not a full test-suite run or a Linux/Windows run. Files transactions are still not crash-atomic. A later failure in finish may leave partial ref changes, as with ordinary transactions. The source-log backup covers an installation error, not a process crash. Unrelated nested renames can also contend for the global packed-refs lock. The base is 0f8e75abeb. This series is independent of the separate batched-deletion old-OID fix discussed elsewhere in the thread. Original patch: https://lore.kernel.org/git/20260920165037.88524-1-maciej.ciemborowicz@gmail.com/ The range-diff below treats the rewritten and split implementation as four new commits, so the changes above describe the mapping from v2. Maciej Ciemborowicz (4): refs: distinguish internal transactions from logical updates refs: support replacing reflogs in a transaction refs: run copy and rename through ordinary transactions refs: remove backend-specific copy and rename callbacks Documentation/githooks.adoc | 10 + refs.c | 200 ++++++++- refs.h | 25 ++ refs/debug.c | 24 -- refs/files-backend.c | 739 +++++++++++++++++--------------- refs/packed-backend.c | 2 - refs/refs-internal.h | 28 +- refs/reftable-backend.c | 334 ++------------- t/helper/test-ref-store.c | 76 ++++ t/perf/p1424-ref-copy-rename.sh | 48 +++ t/t1424-ref-copy-transaction.sh | 352 +++++++++++++++ t/t1425-reflog-transaction.sh | 59 +++ 12 files changed, 1206 insertions(+), 691 deletions(-) create mode 100755 t/perf/p1424-ref-copy-rename.sh create mode 100755 t/t1424-ref-copy-transaction.sh create mode 100755 t/t1425-reflog-transaction.sh Range-diff against v2: 1: d852537d8c < -: ---------- refs: run copy and rename through transactions -: ---------- > 1: 6d7c146e57 refs: distinguish internal transactions from logical updates -: ---------- > 2: 5c3ec4eb49 refs: support replacing reflogs in a transaction -: ---------- > 3: 77af4e809c refs: run copy and rename through ordinary transactions -: ---------- > 4: 83fa644fb3 refs: remove backend-specific copy and rename callbacks base-commit: 0f8e75abebff0877cae681a3d5ff31ac47f54220 -- 2.39.3 (Apple Git-146) ^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v3 1/4] refs: distinguish internal transactions from logical updates 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz @ 2026-10-07 18:05 ` Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 2/4] refs: support replacing reflogs in a transaction Maciej Ciemborowicz ` (3 subsequent siblings) 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-07 18:05 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak A ref backend may use a nested transaction to persist part of a logical update. Provide an internal-only transaction flag so that these physical updates can avoid reporting the same operation to reference-transaction hooks again. Keep the check in run_transaction_hook(), covering every hook state in one place. The flag is deliberately not part of the public refs API. Subsequent changes use it for packed-refs updates belonging to a copy or rename. Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- refs.c | 3 +++ refs/refs-internal.h | 3 +++ 2 files changed, 6 insertions(+) diff --git a/refs.c b/refs.c index 92d5df5b71..00b1b51280 100644 --- a/refs.c +++ b/refs.c @@ -2666,6 +2666,9 @@ static int run_transaction_hook(struct ref_transaction *transaction, struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; int ret = 0; + if (transaction->flags & REF_TRANSACTION_FLAG_INTERNAL) + return 0; + strvec_push(&opt.args, state); opt.feed_pipe = transaction_hook_feed_stdin; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index c3ac7b556f..0b41f5fa4b 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -8,6 +8,9 @@ struct fsck_options; struct ref_transaction; +/* Physical updates nested in a transaction already reported to hooks. */ +#define REF_TRANSACTION_FLAG_INTERNAL (1 << 2) + /* * Data structures and functions for the internal use of the refs * module. Code outside of the refs module should use only the public -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v3 2/4] refs: support replacing reflogs in a transaction 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz @ 2026-10-07 18:05 ` Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 3/4] refs: run copy and rename through ordinary transactions Maciej Ciemborowicz ` (2 subsequent siblings) 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-07 18:05 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Explicit log-only updates can already recreate a reflog during migration. Add ref_transaction_replace_reflog() to replace existing history with these entries without changing the reference itself. Keep its marker flag private to the refs implementation so callers cannot attach this cross-update behavior to an ordinary ref update accidentally. Stage files-backend replacements in unique temporary files beside the logs and install them only during finish. Buffer writes and order entries by their requested index. Aborting prepare only removes staging files. In reftable, tombstone the previous entries in the same table as the replacement. Do not treat a log-only entry with a null new OID as a reference deletion: it is part of the history being replayed. Propagate read errors instead of silently copying a partial log. Test preservation of null-OID entries, ordering, removal, hook rejection and duplicate replacement. This is preparation for copying and renaming through the ordinary transaction path. Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- refs.c | 34 ++++++- refs.h | 9 ++ refs/files-backend.c | 180 +++++++++++++++++++++++++++++++--- refs/refs-internal.h | 3 + refs/reftable-backend.c | 10 +- t/helper/test-ref-store.c | 40 ++++++++ t/t1425-reflog-transaction.sh | 59 +++++++++++ 7 files changed, 320 insertions(+), 15 deletions(-) create mode 100755 t/t1425-reflog-transaction.sh diff --git a/refs.c b/refs.c index 00b1b51280..bf219573cd 100644 --- a/refs.c +++ b/refs.c @@ -1413,7 +1413,6 @@ enum ref_transaction_error ref_transaction_update(struct ref_transaction *transa strbuf_addstr(err, _("refusing to force and skip creation of reflog")); return REF_TRANSACTION_ERROR_GENERIC; } - if (!transaction_refname_valid(refname, new_oid, flags, err)) return REF_TRANSACTION_ERROR_GENERIC; @@ -1496,6 +1495,39 @@ int ref_transaction_update_reflog(struct ref_transaction *transaction, return 0; } +int ref_transaction_replace_reflog(struct ref_transaction *transaction, + const char *refname, + struct strbuf *err) +{ + size_t i; + unsigned int flags = REF_LOG_ONLY | REF_NO_DEREF | + REF_SKIP_CREATE_REFLOG | REF_REPLACE_REFLOG; + + assert(err); + if (transaction->flags & + (REF_TRANSACTION_FLAG_INITIAL | REF_TRANSACTION_ALLOW_FAILURE)) { + strbuf_addstr(err, _("reflog replacement requires an " + "all-or-nothing, non-initial transaction")); + return REF_TRANSACTION_ERROR_GENERIC; + } + if (!transaction_refname_valid(refname, NULL, flags, err)) + return REF_TRANSACTION_ERROR_GENERIC; + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + + if ((update->flags & REF_REPLACE_REFLOG) && + !strcmp(update->refname, refname)) { + strbuf_addf(err, _("reflog replacement for '%s' already queued"), + refname); + return REF_TRANSACTION_ERROR_NAME_CONFLICT; + } + } + + ref_transaction_add_update(transaction, refname, flags, + NULL, NULL, NULL, NULL, NULL, NULL, NULL); + return 0; +} + int ref_transaction_create(struct ref_transaction *transaction, const char *refname, const struct object_id *new_oid, diff --git a/refs.h b/refs.h index 9979446d15..7e80d0bdfb 100644 --- a/refs.h +++ b/refs.h @@ -939,6 +939,15 @@ int ref_transaction_update_reflog(struct ref_transaction *transaction, uint64_t index, struct strbuf *err); +/* + * Replace a reference's reflog with the explicit entries added to the same + * transaction via ref_transaction_update_reflog(). Without such entries, + * remove the reflog. This operation does not change the reference itself. + */ +int ref_transaction_replace_reflog(struct ref_transaction *transaction, + const char *refname, + struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 71628550f2..70d1ff80c4 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1983,6 +1983,22 @@ static int files_create_reflog(struct ref_store *ref_store, const char *refname, return 0; } +static void format_reflog_entry(struct strbuf *sb, + const struct object_id *old_oid, + const struct object_id *new_oid, + const char *committer, const char *msg) +{ + if (!committer) + committer = git_committer_info(0); + + strbuf_addf(sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); + if (msg && *msg) { + strbuf_addch(sb, '\t'); + strbuf_addstr(sb, msg); + } + strbuf_addch(sb, '\n'); +} + static int log_ref_write_fd(int fd, const struct object_id *old_oid, const struct object_id *new_oid, const char *committer, const char *msg) @@ -1990,15 +2006,7 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, struct strbuf sb = STRBUF_INIT; int ret = 0; - if (!committer) - committer = git_committer_info(0); - - strbuf_addf(&sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); - if (msg && *msg) { - strbuf_addch(&sb, '\t'); - strbuf_addstr(&sb, msg); - } - strbuf_addch(&sb, '\n'); + format_reflog_entry(&sb, old_oid, new_oid, committer, msg); if (write_in_full(fd, sb.buf, sb.len) < 0) ret = -1; strbuf_release(&sb); @@ -2397,6 +2405,8 @@ static int files_for_each_reflog_ent(struct ref_store *ref_store, while (!ret && !strbuf_getwholeline(&sb, logfp, '\n')) ret = show_one_reflog_ent(refs, refname, &sb, fn, cb_data); + if (ferror(logfp)) + ret = -1; fclose(logfp); strbuf_release(&sb); return ret; @@ -2665,12 +2675,120 @@ static enum ref_transaction_error check_old_oid(struct ref_update *update, return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE; } +struct staged_reflog { + struct tempfile *file; + struct ref_update **updates; + size_t nr, alloc; +}; + struct files_transaction_backend_data { struct ref_transaction *packed_transaction; int packed_refs_locked; struct strmap ref_locks; + struct strmap reflog_files; }; +static int reflog_update_cmp(const void *a, const void *b) +{ + const struct ref_update *one = *(struct ref_update * const *)a; + const struct ref_update *two = *(struct ref_update * const *)b; + + return (one->index > two->index) - (one->index < two->index); +} + +static int prepare_reflog_replacements(struct files_ref_store *refs, + struct ref_transaction *transaction, + struct strbuf *err) +{ + struct files_transaction_backend_data *data = transaction->backend_data; + struct strbuf path = STRBUF_INIT; + struct strbuf contents = STRBUF_INIT; + struct hashmap_iter iter; + struct strmap_entry *entry; + size_t i; + int ret = -1; + + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + struct staged_reflog *log; + + if (!(update->flags & REF_REPLACE_REFLOG)) + continue; + CALLOC_ARRAY(log, 1); + strmap_put(&data->reflog_files, update->refname, log); + } + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + struct staged_reflog *log = strmap_get(&data->reflog_files, update->refname); + + if (!log || !(update->flags & REF_LOG_ONLY) || + !(update->flags & REF_HAVE_NEW)) + continue; + if (!(update->flags & REF_LOG_USE_PROVIDED_OIDS)) { + strbuf_addf(err, "replacing reflog '%s' requires explicit OIDs", + update->refname); + goto out; + } + ALLOC_GROW(log->updates, log->nr + 1, log->alloc); + log->updates[log->nr++] = update; + } + strmap_for_each_entry(&data->reflog_files, &iter, entry) { + struct staged_reflog *log = entry->value; + int fd; + + if (!log->nr) + continue; + QSORT(log->updates, log->nr, reflog_update_cmp); + /* Keep the staging file beside the logs, which may be on another device. */ + strbuf_reset(&path); + files_reflog_path(refs, &path, is_root_ref(entry->key) ? + ".tmp-reflog-XXXXXX" : "refs/.tmp-reflog-XXXXXX"); + if (safe_create_leading_directories(refs->base.repo, path.buf)) + goto write_error; + log->file = mks_tempfile_m(path.buf, 0666); + if (!log->file) + goto write_error; + fd = get_tempfile_fd(log->file); + for (i = 0; i < log->nr; i++) { + struct ref_update *update = log->updates[i]; + + format_reflog_entry(&contents, &update->old_oid, &update->new_oid, + update->committer_info, update->msg); + if (contents.len >= 65536) { + if (write_in_full(fd, contents.buf, contents.len) < 0) + goto write_error; + strbuf_reset(&contents); + } + } + if (write_in_full(fd, contents.buf, contents.len) < 0) + goto write_error; + strbuf_reset(&contents); + if (adjust_shared_perm(refs->base.repo, get_tempfile_path(log->file)) || + close_tempfile_gently(log->file)) + goto write_error; + } + ret = 0; + goto out; + +write_error: + strbuf_addf(err, "cannot write staged reflog: %s", strerror(errno)); +out: + strbuf_release(&path); + strbuf_release(&contents); + return ret; +} + +static int install_reflog(const char *path, void *data) +{ + struct staged_reflog *log = data; + int ret = rename(get_tempfile_path(log->file), path); + + /* Some systems report ENOTDIR when the destination is a directory. */ + if (ret && errno == ENOTDIR) + errno = EISDIR; + return ret; +} + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2929,6 +3047,17 @@ static void files_transaction_cleanup(struct files_ref_store *refs, } if (backend_data) { + struct hashmap_iter iter; + struct strmap_entry *entry; + + strmap_for_each_entry(&backend_data->reflog_files, &iter, entry) { + struct staged_reflog *log = entry->value; + + delete_tempfile(&log->file); + free(log->updates); + free(log); + } + strmap_clear(&backend_data->reflog_files, 0); if (backend_data->packed_transaction && ref_transaction_abort(backend_data->packed_transaction, &err)) { error("error aborting transaction: %s", err.buf); @@ -2970,6 +3099,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, CALLOC_ARRAY(backend_data, 1); strmap_init(&backend_data->ref_locks); + strmap_init(&backend_data->reflog_files); transaction->backend_data = backend_data; /* @@ -3120,6 +3250,8 @@ static int files_transaction_prepare(struct ref_store *ref_store, } } + ret = prepare_reflog_replacements(refs, transaction, err); + cleanup: free(head_ref); string_list_clear(&refnames_to_check, 1); @@ -3344,6 +3476,31 @@ static int files_transaction_finish(struct ref_store *ref_store, backend_data = transaction->backend_data; packed_transaction = backend_data->packed_transaction; + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + struct staged_reflog *log; + + if (!(update->flags & REF_REPLACE_REFLOG)) + continue; + log = strmap_get(&backend_data->reflog_files, update->refname); + strbuf_reset(&sb); + files_reflog_path(refs, &sb, update->refname); + if (!log->file) { + if (unlink(sb.buf) && errno != ENOENT && errno != EISDIR) + ret = -1; + } else if (raceproof_create_file(refs, sb.buf, install_reflog, log)) { + ret = -1; + } else { + delete_tempfile(&log->file); + } + if (ret) { + strbuf_addf(err, "cannot replace reflog '%s': %s", + update->refname, strerror(errno)); + } + if (ret) + goto cleanup; + } + /* Perform updates first so live commits remain referenced */ for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; @@ -3352,8 +3509,9 @@ static int files_transaction_finish(struct ref_store *ref_store, if (update->rejection_err) continue; - if (update->flags & REF_NEEDS_COMMIT || - update->flags & REF_LOG_ONLY) { + if ((update->flags & REF_NEEDS_COMMIT || + update->flags & REF_LOG_ONLY) && + !strmap_contains(&backend_data->reflog_files, update->refname)) { if (parse_and_write_reflog(refs, update, lock, err)) { ret = REF_TRANSACTION_ERROR_GENERIC; goto cleanup; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 0b41f5fa4b..19fdf39d1d 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -49,6 +49,9 @@ struct ref_transaction; */ #define REF_HAVE_PEELED (1 << 15) +/* Replace a reflog with the explicit log-only updates in the transaction. */ +#define REF_REPLACE_REFLOG (1 << 16) + /* * Return the length of time to retry acquiring a loose reference lock * before giving up, in milliseconds: diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 10db03991e..203b111f3d 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1503,9 +1503,11 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data * - `core.logAllRefUpdates` tells us to create the reflog for * the given ref. */ - if ((u->flags & REF_HAVE_NEW) && + if ((u->flags & REF_REPLACE_REFLOG) || + ((u->flags & REF_HAVE_NEW) && + !(u->flags & REF_LOG_ONLY) && !(u->type & REF_ISSYMREF) && - ref_update_has_null_new_value(u)) { + ref_update_has_null_new_value(u))) { struct reftable_log_record log = {0}; struct reftable_iterator it = {0}; @@ -1548,8 +1550,10 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data if (ret) goto done; - } else if (!(u->flags & REF_SKIP_CREATE_REFLOG) && + } + if (!(u->flags & REF_SKIP_CREATE_REFLOG) && (u->flags & REF_HAVE_NEW) && + (!ref_update_has_null_new_value(u) || (u->flags & REF_LOG_ONLY)) && (u->flags & REF_FORCE_CREATE_REFLOG || should_write_log(arg->refs, u->refname))) { struct reftable_log_record *log; diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index db58f00589..7ce10c28af 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -146,6 +146,45 @@ static int cmd_rename_ref(struct ref_store *refs, const char **argv) return refs_rename_ref(refs, oldref, newref, logmsg); } +static int cmd_reflog_transaction(struct ref_store *refs, const char **argv) +{ + const char *refname = notnull(*argv++, "refname"); + const char *mode = notnull(*argv++, "mode"); + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = !transaction; + + if (!ret) { + if (!strcmp(mode, "replace")) { + ret = ref_transaction_replace_reflog(transaction, refname, &err); + } else if (!strcmp(mode, "replace-twice")) { + ret = ref_transaction_replace_reflog(transaction, refname, &err) || + ref_transaction_replace_reflog(transaction, refname, &err); + } else if (strcmp(mode, "append")) { + die("unknown reflog transaction mode: %s", mode); + } + } + while (!ret && *argv) { + uint64_t index = strtoumax(notnull(*argv++, "index"), NULL, 10); + struct object_id old_oid, new_oid; + const char *message; + + if (get_oid_hex(notnull(*argv++, "old-oid"), &old_oid) || + get_oid_hex(notnull(*argv++, "new-oid"), &new_oid)) + die("invalid object ID"); + message = notnull(*argv++, "message"); + ret = ref_transaction_update_reflog(transaction, refname, + &new_oid, &old_oid, NULL, message, index, &err); + } + if (!ret) + ret = ref_transaction_commit(transaction, &err); + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; +} + static int each_ref(const struct reference *ref, void *cb_data UNUSED) { printf("%s %s 0x%x\n", oid_to_hex(ref->oid), ref->name, ref->flags); @@ -308,6 +347,7 @@ static struct command commands[] = { { "create-symref", cmd_create_symref }, { "delete-refs", cmd_delete_refs }, { "rename-ref", cmd_rename_ref }, + { "reflog-transaction", cmd_reflog_transaction }, { "for-each-ref", cmd_for_each_ref }, { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, diff --git a/t/t1425-reflog-transaction.sh b/t/t1425-reflog-transaction.sh new file mode 100755 index 0000000000..ee5278045e --- /dev/null +++ b/t/t1425-reflog-transaction.sh @@ -0,0 +1,59 @@ +#!/bin/sh + +test_description='explicit reflog entries in reference transactions' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit A && + test_commit B && + A=$(git rev-parse A) && + B=$(git rev-parse B) +' + +test_expect_success 'log-only entry with null new OID does not delete existing history' ' + git branch source "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/source >before && + test-tool ref-store main reflog-transaction refs/heads/source append \ + 1 "$A" "$ZERO_OID" deletion && + test-tool ref-store main for-each-reflog-ent refs/heads/source >after && + sed "$ d" after >history && + test_cmp before history && + test_grep "$A $ZERO_OID .*deletion" after && + test_cmp_rev "$A" source +' + +test_expect_success 'replace reflog with entries ordered by index' ' + test-tool ref-store main reflog-transaction refs/heads/source replace \ + 2 "$A" "$B" second \ + 1 "$ZERO_OID" "$A" first && + git reflog show --format=%gs source >actual && + printf "%s\n" second first >expect && + test_cmp expect actual && + test_cmp_rev "$A" source +' + +test_expect_success 'prepared hook can reject replacement without losing history' ' + test_when_finished "rm -f .git/hooks/reference-transaction" && + test-tool ref-store main for-each-reflog-ent refs/heads/source >before && + write_script .git/hooks/reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail test-tool ref-store main reflog-transaction refs/heads/source replace \ + 1 "$ZERO_OID" "$B" replacement && + test-tool ref-store main for-each-reflog-ent refs/heads/source >after && + test_cmp before after +' + +test_expect_success 'replacement with no entries removes only the reflog' ' + test-tool ref-store main reflog-transaction refs/heads/source replace && + test_must_fail git reflog exists refs/heads/source && + test_cmp_rev "$A" source +' + +test_expect_success 'duplicate replacement is rejected' ' + test_must_fail test-tool ref-store main reflog-transaction \ + refs/heads/source replace-twice +' + +test_done -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v3 3/4] refs: run copy and rename through ordinary transactions 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 2/4] refs: support replacing reflogs in a transaction Maciej Ciemborowicz @ 2026-10-07 18:05 ` Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 4/4] refs: remove backend-specific copy and rename callbacks Maciej Ciemborowicz 2026-10-07 19:55 ` [PATCH v3 0/4] refs: run copy and rename through transactions Junio C Hamano 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-07 18:05 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Copy and rename currently bypass reference transactions. A hook sees the source deletion with files and sees neither endpoint with reftable. Queue the source and destination as ordinary updates instead, with the destination referring to its source update. Read and lock the source during prepare, then replay its history through the explicit reflog API. Keep this state per update so multiple independent operations and ordinary updates can share a transaction. Do not reject source changes made before locking: preparing is advisory, while prepared sees the values read under lock and unchanged on-disk refs and reflogs. For files D/F and case-only renames, the source prevents locking a loose destination. Queue the destination in packed-refs, protected by the source lock (and an ancestor lock when needed). Suppress hooks for this physical child transaction only. Keep source logs until finish and use the staged replacements so a veto needs no rollback writes. During finish, retain a unique backup of a D/F source log until its destination log is installed, and restore it if installation fails. Preserve the backends' distinct final reflog records. Forced reftable copies now replace destination history instead of retaining unrelated destination entries. Add regression tests for hooks, full histories, HEAD, source races, mixed transactions, lock and installation errors, restoration and cleanup, and a reproducible performance test. Replaying history costs O(N) time and memory. In a local no-hook benchmark with 10,000 entries, files rename takes about 17 ms versus 5 ms before, and reftable rename about 54 ms versus 44 ms. Short-history operations remain within roughly 1 ms. This trades the files constant-time rename for staging that does not expose partial changes to a prepared hook. Helped-by: Karthik Nayak <karthik.188@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- Documentation/githooks.adoc | 10 + refs.c | 163 +++++++++++++-- refs.h | 16 ++ refs/files-backend.c | 225 +++++++++++++++++++- refs/refs-internal.h | 13 ++ refs/reftable-backend.c | 37 +++- t/helper/test-ref-store.c | 36 ++++ t/perf/p1424-ref-copy-rename.sh | 48 +++++ t/t1424-ref-copy-transaction.sh | 352 ++++++++++++++++++++++++++++++++ 9 files changed, 880 insertions(+), 20 deletions(-) create mode 100755 t/perf/p1424-ref-copy-rename.sh create mode 100755 t/t1424-ref-copy-transaction.sh diff --git a/Documentation/githooks.adoc b/Documentation/githooks.adoc index ed045940d1..629dd3e4d1 100644 --- a/Documentation/githooks.adoc +++ b/Documentation/githooks.adoc @@ -518,6 +518,16 @@ distinguish these cases, you can inspect the current value of references are not resolved: `<ref-name>` will reflect the symbolic reference itself rather than the object it points to. +Copying or renaming a reference is reported as one transaction. A rename +reports the deletion of the source and the update of the destination; +a copy reports only the destination update. Copying the reflog does not +produce additional hook input. In the "preparing" state, the old values +are unspecified (all zeroes), and the destination's new value is the +source value read before locking. In subsequent states, these operations +report the old values and the source value read under lock. Thus, a +"preparing" hook may change the source before it is copied or renamed. +A "prepared" hook still sees the original references and reflogs on disk. + For symbolic reference updates the `<old_value>` and `<new-value>` fields could denote references instead of objects. A reference will be denoted with a 'ref:' prefix, like `ref:<ref-target>`. diff --git a/refs.c b/refs.c index bf219573cd..eb3e58c789 100644 --- a/refs.c +++ b/refs.c @@ -3158,28 +3158,165 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg, return ret; } +static int transaction_copy_ref(struct ref_transaction *transaction, + const char *oldref, const char *newref, + unsigned int source_flags, const char *logmsg, + struct strbuf *err) +{ + struct ref_update *source, *destination; + struct object_id oid; + int type; + + if (transaction->state != REF_TRANSACTION_OPEN) + BUG("copy added to a prepared transaction"); + if (!refs_resolve_ref_unsafe(transaction->ref_store, oldref, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, &oid, &type)) { + strbuf_addf(err, _("refname %s not found"), oldref); + return -1; + } + if (type & REF_ISSYMREF) { + strbuf_addf(err, _("refname %s is a symbolic ref, copying or renaming it is not supported"), oldref); + return -1; + } + if (!strcmp(oldref, newref)) + return 0; + if (!transaction_refname_valid(newref, &oid, REF_NO_DEREF, err)) + return -1; + if (transaction->flags & (REF_TRANSACTION_FLAG_INITIAL | REF_TRANSACTION_ALLOW_FAILURE)) { + strbuf_addstr(err, _("copy and rename require an all-or-nothing, non-initial transaction")); + return -1; + } + + /* This value describes the request to 'preparing'; locks decide the value used. */ + source = ref_transaction_add_update(transaction, oldref, + REF_NO_DEREF | REF_COPY_SOURCE | source_flags, + (source_flags & REF_HAVE_NEW) ? null_oid(transaction->ref_store->repo->hash_algo) : NULL, + NULL, NULL, NULL, NULL, NULL, logmsg); + destination = ref_transaction_add_update(transaction, newref, + REF_NO_DEREF | REF_HAVE_NEW, + &oid, NULL, NULL, NULL, NULL, NULL, logmsg); + destination->copy_from = source; + return ref_transaction_replace_reflog(transaction, newref, err); +} + +int ref_transaction_copy(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err) +{ + return transaction_copy_ref(transaction, oldref, newref, + REF_LOG_ONLY | REF_SKIP_CREATE_REFLOG, logmsg, err); +} + +int ref_transaction_rename(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err) +{ + return transaction_copy_ref(transaction, oldref, newref, + REF_HAVE_NEW, logmsg, err); +} + +int ref_update_record_copy_source(struct ref_update *update, + const struct object_id *oid, + struct strbuf *err) +{ + if ((update->type & REF_ISSYMREF) || is_null_oid(oid)) { + strbuf_addf(err, _("'%s' is not an existing direct reference"), update->refname); + return -1; + } + oidcpy(&update->old_oid, oid); + update->flags |= REF_HAVE_OLD; + return 0; +} + +struct copy_reflog_data { + struct ref_transaction *transaction; + const char *refname; + struct strbuf *err; +}; + +static int copy_reflog_entry(const char *refname UNUSED, + struct object_id *old_oid, struct object_id *new_oid, + const char *committer, timestamp_t timestamp, int tz, + const char *message, void *cb_data) +{ + struct copy_reflog_data *data = cb_data; + struct strbuf ident = STRBUF_INIT; + int ret; + + strbuf_addf(&ident, "%s %" PRItime " %+05d", committer, timestamp, tz); + ret = ref_transaction_update_reflog(data->transaction, data->refname, + new_oid, old_oid, ident.buf, message, + data->transaction->max_index + 1, data->err); + strbuf_release(&ident); + return ret; +} + +int ref_transaction_prepare_copy(struct ref_transaction *transaction, + struct ref_update *update, + struct strbuf *err) +{ + struct ref_update *source = update->copy_from; + struct copy_reflog_data data = { transaction, update->refname, err }; + struct object *object; + int ret; + + oidcpy(&update->new_oid, &source->old_oid); + object = parse_object(transaction->ref_store->repo, &update->new_oid); + if (!object || (is_branch(update->refname) && object->type != OBJ_COMMIT)) { + strbuf_addf(err, _("cannot copy object %s to '%s'"), + oid_to_hex(&update->new_oid), update->refname); + return -1; + } + if (object->type == OBJ_TAG && + !peel_object(transaction->ref_store->repo, &update->new_oid, + &update->peeled, PEEL_OBJECT_VERIFY_TAGGED_OBJECT_TYPE)) + update->flags |= REF_HAVE_PEELED; + + if (!refs_reflog_exists(transaction->ref_store, source->refname)) + return 0; + ret = refs_for_each_reflog_ent(transaction->ref_store, source->refname, + copy_reflog_entry, &data); + if (ret && !err->len) + strbuf_addf(err, _("cannot read reflog for '%s'"), source->refname); + return ret; +} + int refs_rename_ref(struct ref_store *refs, const char *oldref, const char *newref, const char *logmsg) { - char *msg; - int retval; + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = -1; - msg = normalize_reflog_message(logmsg); - retval = refs->be->rename_ref(refs, oldref, newref, msg); - free(msg); - return retval; + if (!transaction || + ref_transaction_rename(transaction, oldref, newref, logmsg, &err)) + goto out; + ret = transaction->nr ? ref_transaction_commit(transaction, &err) : 0; +out: + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; } int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, - const char *newref, const char *logmsg) + const char *newref, const char *logmsg) { - char *msg; - int retval; + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = -1; - msg = normalize_reflog_message(logmsg); - retval = refs->be->copy_ref(refs, oldref, newref, msg); - free(msg); - return retval; + if (!transaction || + ref_transaction_copy(transaction, oldref, newref, logmsg, &err)) + goto out; + ret = transaction->nr ? ref_transaction_commit(transaction, &err) : 0; +out: + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; } const char *ref_update_original_update_refname(struct ref_update *update) diff --git a/refs.h b/refs.h index 7e80d0bdfb..2538c611cd 100644 --- a/refs.h +++ b/refs.h @@ -948,6 +948,22 @@ int ref_transaction_replace_reflog(struct ref_transaction *transaction, const char *refname, struct strbuf *err); +/* + * Queue a copy or rename of a direct reference and its reflog. The source is + * read again under lock during prepare; changes made by the preparing hook + * are included. An existing destination is overwritten without dereferencing + * it. Several independent copies, renames and ordinary updates may be queued + * together. Each destination (and each source being renamed) must be unique. + * Initial transactions and transactions allowing individual failures are not + * supported. On failure, discard the transaction. + */ +int ref_transaction_copy(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err); +int ref_transaction_rename(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 70d1ff80c4..9e22f62d3f 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -74,6 +74,12 @@ extern int ignore_case; */ #define REF_LOG_VIA_SPLIT (1 << 14) +/* A D/F or case-only rename cannot lock a loose destination beside its source. */ +#define REF_NEEDS_PACK (1 << 18) + +/* The source reflog was removed to make room for its rename destination. */ +#define REF_RENAMED_LOG (1 << 19) + struct ref_lock { char *ref_name; struct lock_file lk; @@ -788,6 +794,37 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs, lock->count = 1; files_ref_path(refs, &ref_file, refname); + if (update->copy_from && (update->copy_from->flags & REF_HAVE_NEW)) { + const char *source = update->copy_from->refname; + size_t source_len = strlen(source), dest_len = strlen(refname); + int source_is_parent = starts_with(refname, source) && refname[source_len] == '/'; + int dest_is_parent = starts_with(source, refname) && source[dest_len] == '/'; + int same_path = repo_ignore_case(refs->base.repo) && !strcasecmp(source, refname); + + if (source_is_parent || dest_is_parent || same_path) { + struct string_list skip = STRING_LIST_INIT_NODUP; + + string_list_insert(&skip, source); + ret = refs_verify_refname_available(&refs->base, refname, + extras, &skip, 0, err); + string_list_clear(&skip, 0); + if (ret) + goto error_return; + /* The existing source and its lock protect a child or case-only destination. */ + if (dest_is_parent && repo_hold_lock_file_for_update_timeout( + refs->base.repo, &lock->lk, ref_file.buf, LOCK_NO_DEREF, + get_files_ref_lock_timeout_ms(refs->base.repo)) < 0) { + unable_to_lock_message(ref_file.buf, errno, err); + ret = REF_TRANSACTION_ERROR_GENERIC; + goto error_return; + } + oidclr(&lock->old_oid, refs->base.repo->hash_algo); + update->flags |= REF_NEEDS_PACK; + ret = 0; + goto out; + } + } + retry: switch (safe_create_leading_directories(refs->base.repo, ref_file.buf)) { case SCLD_OK: @@ -2538,7 +2575,8 @@ static enum ref_transaction_error split_head_update(struct ref_update *update, new_update = ref_transaction_add_update( transaction, "HEAD", - update->flags | REF_LOG_ONLY | REF_NO_DEREF | REF_LOG_VIA_SPLIT, + (update->flags & ~REF_COPY_SOURCE) | + REF_LOG_ONLY | REF_NO_DEREF | REF_LOG_VIA_SPLIT, &update->new_oid, &update->old_oid, &update->peeled, NULL, NULL, update->committer_info, update->msg); new_update->parent_update = update; @@ -2677,6 +2715,7 @@ static enum ref_transaction_error check_old_oid(struct ref_update *update, struct staged_reflog { struct tempfile *file; + struct tempfile *source_log; struct ref_update **updates; size_t nr, alloc; }; @@ -2789,6 +2828,155 @@ static int install_reflog(const char *path, void *data) return ret; } +static int move_reflog(const char *path, void *data) +{ + struct tempfile *tempfile = data; + int ret = rename(get_tempfile_path(tempfile), path); + + if (ret && errno == ENOTDIR) + errno = EISDIR; + return ret; +} + +static int stage_source_reflog(struct files_ref_store *refs, + struct ref_update *update, + struct staged_reflog *log, + struct strbuf *err) +{ + struct strbuf source = STRBUF_INIT; + struct strbuf path = STRBUF_INIT; + int ret = -1; + + files_reflog_path(refs, &source, update->copy_from->refname); + if (!refs_reflog_exists(&refs->base, update->copy_from->refname)) { + ret = 0; + goto out; + } + files_reflog_path(refs, &path, "refs/.tmp-reflog-source-XXXXXX"); + log->source_log = mks_tempfile_m(path.buf, 0666); + if (!log->source_log || close_tempfile_gently(log->source_log) || + rename(source.buf, get_tempfile_path(log->source_log))) { + strbuf_addf(err, "cannot stage source reflog '%s': %s", + update->copy_from->refname, strerror(errno)); + goto out; + } + try_remove_empty_parents(refs, update->copy_from->refname, + REMOVE_EMPTY_PARENTS_REFLOG); + update->copy_from->flags |= REF_RENAMED_LOG; + ret = 0; + +out: + if (ret) + delete_tempfile(&log->source_log); + strbuf_release(&source); + strbuf_release(&path); + return ret; +} + +static int restore_source_reflog(struct files_ref_store *refs, + struct ref_update *update, + struct staged_reflog *log, + struct strbuf *err) +{ + struct strbuf source = STRBUF_INIT; + int ret; + + if (!log->source_log) + return 0; + files_reflog_path(refs, &source, update->copy_from->refname); + ret = raceproof_create_file(refs, source.buf, move_reflog, + log->source_log); + if (ret) { + struct strbuf recovery = STRBUF_INIT; + + strbuf_addf(err, "; cannot restore source reflog '%s': %s", + update->copy_from->refname, strerror(errno)); + strbuf_addf(&recovery, "%s.recovery", + get_tempfile_path(log->source_log)); + if (rename_tempfile(&log->source_log, recovery.buf)) + strbuf_addf(err, "; cannot preserve its backup: %s", + strerror(errno)); + else + strbuf_addf(err, "; backup saved as '%s'", recovery.buf); + strbuf_release(&recovery); + } else { + delete_tempfile(&log->source_log); + } + strbuf_release(&source); + return ret; +} + +static int check_reflog_symlink(struct files_ref_store *refs, + const char *refname, struct strbuf *err) +{ + struct strbuf path = STRBUF_INIT; + struct stat st; + int ret = 0; + + files_reflog_path(refs, &path, refname); + if (!lstat(path.buf, &st)) { + if (S_ISLNK(st.st_mode)) { + strbuf_addf(err, "reflog for %s is a symlink", refname); + ret = -1; + } + } else if (errno != ENOENT && errno != ENOTDIR) { + strbuf_addf(err, "cannot stat reflog for %s: %s", refname, strerror(errno)); + ret = -1; + } + strbuf_release(&path); + return ret; +} + +static struct ref_update *find_update_with_flag(struct ref_transaction *transaction, + const char *refname, + unsigned int flag) +{ + size_t i; + + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + + if ((update->flags & flag) && !strcmp(update->refname, refname)) + return update; + } + return NULL; +} + +static int prepare_copy_reflog(struct files_ref_store *refs, + struct ref_transaction *transaction, + struct ref_update *update, struct strbuf *err) +{ + enum log_refs_config config = files_ref_store_write_options(refs)->log_all_ref_updates; + int source_has_log; + int write_log; + + if (check_reflog_symlink(refs, update->copy_from->refname, err) || + check_reflog_symlink(refs, update->refname, err)) + return -1; + source_has_log = refs_reflog_exists(&refs->base, update->copy_from->refname); + /* A copy without source history only appends to an existing destination log. */ + if (!source_has_log && !(update->copy_from->flags & REF_HAVE_NEW)) { + struct ref_update *replacement = find_update_with_flag( + transaction, update->refname, REF_REPLACE_REFLOG); + + if (!replacement) + BUG("copy destination without a reflog replacement"); + replacement->flags &= ~REF_REPLACE_REFLOG; + } + if (config == LOG_REFS_UNSET) + config = is_bare_repository(refs->base.repo) ? LOG_REFS_NONE : LOG_REFS_NORMAL; + write_log = source_has_log || should_autocreate_reflog(config, update->refname) || + (!(update->copy_from->flags & REF_HAVE_NEW) && + refs_reflog_exists(&refs->base, update->refname)); + if (ref_transaction_prepare_copy(transaction, update, err)) + return -1; + if (!write_log) + return 0; + return ref_transaction_update_reflog(transaction, update->refname, + &update->new_oid, &update->new_oid, NULL, update->msg, + transaction->max_index + 1, err); +} + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2819,6 +3007,8 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re files_assert_main_repository(refs, "lock_ref_for_update"); backend_data = transaction->backend_data; + if (update->copy_from && prepare_copy_reflog(refs, transaction, update, err)) + return REF_TRANSACTION_ERROR_GENERIC; if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update)) update->flags |= REF_DELETING; @@ -2828,6 +3018,8 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re if (ret) goto out; } + if (update->copy_from) + update->flags |= REF_SKIP_CREATE_REFLOG; lock = strmap_get(&backend_data->ref_locks, update->refname); if (lock) { @@ -2849,6 +3041,17 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re } update->backend_data = lock; + if (update->copy_from) { + oidcpy(&update->old_oid, &lock->old_oid); + if (update->type & REF_ISSYMREF) + update->old_target = xstrdup(referent.buf); + update->flags |= REF_HAVE_OLD; + } + if ((update->flags & REF_COPY_SOURCE) && + ref_update_record_copy_source(update, &lock->old_oid, err)) { + ret = REF_TRANSACTION_ERROR_GENERIC; + goto out; + } if (update->flags & REF_LOG_VIA_SPLIT) { struct ref_lock *parent_lock; @@ -2974,7 +3177,7 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re update->flags |= REF_NEEDS_COMMIT; } else if ((update->flags & REF_HAVE_NEW) && !(update->flags & REF_DELETING) && - !(update->flags & REF_LOG_ONLY)) { + !(update->flags & (REF_LOG_ONLY | REF_NEEDS_PACK))) { if (!(update->type & REF_ISSYMREF) && oideq(&lock->old_oid, &update->new_oid)) { /* @@ -3003,7 +3206,7 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re } } } - if (!(update->flags & REF_NEEDS_COMMIT)) { + if (!(update->flags & (REF_NEEDS_COMMIT | REF_NEEDS_PACK))) { /* * We didn't call write_ref_to_lockfile(), so * the lockfile is still open. Close it to @@ -3054,6 +3257,7 @@ static void files_transaction_cleanup(struct files_ref_store *refs, struct staged_reflog *log = entry->value; delete_tempfile(&log->file); + delete_tempfile(&log->source_log); free(log->updates); free(log); } @@ -3162,7 +3366,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, goto cleanup; } - if (update->flags & REF_DELETING && + if (update->flags & (REF_DELETING | REF_NEEDS_PACK) && !(update->flags & REF_LOG_ONLY) && !(update->flags & REF_IS_PRUNING)) { /* @@ -3187,6 +3391,8 @@ static int files_transaction_prepare(struct ref_store *ref_store, REF_HAVE_NEW | REF_NO_DEREF, &update->new_oid, NULL, NULL, NULL, NULL, NULL, NULL); + if (update->copy_from || (update->flags & REF_COPY_SOURCE)) + packed_transaction->flags |= REF_TRANSACTION_FLAG_INTERNAL; } } @@ -3478,11 +3684,18 @@ static int files_transaction_finish(struct ref_store *ref_store, for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; + struct ref_update *operation; struct staged_reflog *log; if (!(update->flags & REF_REPLACE_REFLOG)) continue; log = strmap_get(&backend_data->reflog_files, update->refname); + operation = find_update_with_flag(transaction, update->refname, + REF_NEEDS_PACK); + if (operation && stage_source_reflog(refs, operation, log, err)) { + ret = -1; + goto cleanup; + } strbuf_reset(&sb); files_reflog_path(refs, &sb, update->refname); if (!log->file) { @@ -3496,6 +3709,8 @@ static int files_transaction_finish(struct ref_store *ref_store, if (ret) { strbuf_addf(err, "cannot replace reflog '%s': %s", update->refname, strerror(errno)); + if (operation) + restore_source_reflog(refs, operation, log, err); } if (ret) goto cleanup; @@ -3560,7 +3775,7 @@ static int files_transaction_finish(struct ref_store *ref_store, if (update->flags & REF_DELETING && !(update->flags & REF_LOG_ONLY) && - !(update->flags & REF_IS_PRUNING)) { + !(update->flags & (REF_IS_PRUNING | REF_RENAMED_LOG))) { strbuf_reset(&sb); files_reflog_path(refs, &sb, update->refname); if (!unlink_or_warn(sb.buf)) diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 19fdf39d1d..2746ce5b49 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -52,6 +52,9 @@ struct ref_transaction; /* Replace a reflog with the explicit log-only updates in the transaction. */ #define REF_REPLACE_REFLOG (1 << 16) +/* A source whose value and reflog are consumed by a later update. */ +#define REF_COPY_SOURCE (1 << 17) + /* * Return the length of time to retry acquiring a loose reference lock * before giving up, in milliseconds: @@ -161,6 +164,9 @@ struct ref_update { */ struct ref_update *parent_update; + /* The source is queued first, so backends lock it before the destination. */ + struct ref_update *copy_from; + const char refname[FLEX_ARRAY]; }; @@ -193,6 +199,13 @@ struct ref_update *ref_transaction_add_update( const char *committer_info, const char *msg); +int ref_update_record_copy_source(struct ref_update *update, + const struct object_id *oid, + struct strbuf *err); +int ref_transaction_prepare_copy(struct ref_transaction *transaction, + struct ref_update *update, + struct strbuf *err); + /* * Transaction states. * diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 203b111f3d..72c1596c11 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1076,6 +1076,22 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor struct object_id current_oid = {0}; const char *rewritten_ref; + if (u->copy_from) { + const struct object_id *zero = null_oid(refs->base.repo->hash_algo); + + if (ref_transaction_prepare_copy(transaction, u, err)) + return REF_TRANSACTION_ERROR_GENERIC; + if ((u->copy_from->flags & REF_HAVE_NEW) && + ref_transaction_update_reflog(transaction, u->refname, + zero, &u->new_oid, NULL, u->msg, + transaction->max_index + 1, err)) + return REF_TRANSACTION_ERROR_GENERIC; + if (ref_transaction_update_reflog(transaction, u->refname, + &u->new_oid, zero, NULL, u->msg, + transaction->max_index + 1, err)) + return REF_TRANSACTION_ERROR_GENERIC; + } + /* * There is no need to reload the respective backends here as * we have already reloaded them when preparing the transaction @@ -1126,15 +1142,27 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor ref_transaction_add_update( transaction, "HEAD", - u->flags | REF_LOG_ONLY | REF_NO_DEREF, + (u->flags & ~REF_COPY_SOURCE) | + REF_LOG_ONLY | REF_NO_DEREF, &u->new_oid, &u->old_oid, &u->peeled, NULL, NULL, NULL, u->msg); } + if (u->copy_from) + u->flags |= REF_SKIP_CREATE_REFLOG; ret = reftable_backend_read_ref(be, rewritten_ref, ¤t_oid, referent, &u->type); if (ret < 0) return REF_TRANSACTION_ERROR_GENERIC; + if (u->copy_from) { + oidcpy(&u->old_oid, ¤t_oid); + if (u->type & REF_ISSYMREF) + u->old_target = xstrdup(referent->buf); + u->flags |= REF_HAVE_OLD; + } + if ((u->flags & REF_COPY_SOURCE) && + ref_update_record_copy_source(u, ¤t_oid, err)) + return REF_TRANSACTION_ERROR_GENERIC; if (ret > 0 && !ref_update_expects_existing_old_ref(u)) { struct string_list_item *item; /* @@ -1319,6 +1347,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare"); struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT; struct string_list refnames_to_check = STRING_LIST_INIT_NODUP; + struct string_list renamed_sources = STRING_LIST_INIT_NODUP; struct reftable_transaction_data *tx_data = NULL; struct reftable_backend *be; struct object_id head_oid; @@ -1338,6 +1367,9 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, * that will be modified during the transaction. */ for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + if (update->copy_from && (update->copy_from->flags & REF_HAVE_NEW)) + string_list_insert(&renamed_sources, update->copy_from->refname); ret = prepare_transaction_update(NULL, refs, tx_data, transaction->updates[i], err); if (ret) @@ -1390,7 +1422,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, } ret = refs_verify_refnames_available(ref_store, &refnames_to_check, - &transaction->refnames, NULL, + &transaction->refnames, &renamed_sources, transaction, transaction->flags & REF_TRANSACTION_FLAG_INITIAL, err); @@ -1411,6 +1443,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, strbuf_release(&referent); strbuf_release(&head_referent); string_list_clear(&refnames_to_check, 1); + string_list_clear(&renamed_sources, 0); return ret; } diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index 7ce10c28af..ec7728334c 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -185,6 +185,41 @@ static int cmd_reflog_transaction(struct ref_store *refs, const char **argv) return ret; } +static int cmd_copy_transaction(struct ref_store *refs, const char **argv) +{ + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = !transaction; + + while (!ret && *argv) { + const char *operation = *argv++; + const char *source = notnull(*argv++, "source"); + const char *destination = notnull(*argv++, "destination"); + + if (!strcmp(operation, "rename")) + ret = ref_transaction_rename(transaction, source, destination, "rename", &err); + else if (!strcmp(operation, "copy")) + ret = ref_transaction_copy(transaction, source, destination, "copy", &err); + else if (!strcmp(operation, "update")) { + struct object_id oid; + + if (get_oid_hex(destination, &oid)) + die("invalid object ID: %s", destination); + ret = ref_transaction_update(transaction, source, &oid, + NULL, NULL, NULL, 0, "update", &err); + } else { + die("unknown operation: %s", operation); + } + } + if (!ret) + ret = ref_transaction_commit(transaction, &err); + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; +} + static int each_ref(const struct reference *ref, void *cb_data UNUSED) { printf("%s %s 0x%x\n", oid_to_hex(ref->oid), ref->name, ref->flags); @@ -348,6 +383,7 @@ static struct command commands[] = { { "delete-refs", cmd_delete_refs }, { "rename-ref", cmd_rename_ref }, { "reflog-transaction", cmd_reflog_transaction }, + { "copy-transaction", cmd_copy_transaction }, { "for-each-ref", cmd_for_each_ref }, { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, diff --git a/t/perf/p1424-ref-copy-rename.sh b/t/perf/p1424-ref-copy-rename.sh new file mode 100755 index 0000000000..a901129b00 --- /dev/null +++ b/t/perf/p1424-ref-copy-rename.sh @@ -0,0 +1,48 @@ +#!/bin/sh + +test_description='Copy and rename references with short and long reflogs' + +. ./perf-lib.sh + +test_perf_fresh_repo + +for entries in 10 10000 +do + export entries + test_expect_success "setup $entries reflog entries" ' + git init --ref-format=files "repo-$entries" && + ( + cd "repo-$entries" && + test_commit A && + git branch source && + oid=$(git rev-parse HEAD) && + awk -v oid="$oid" -v count="$entries" '\'' + BEGIN { + for (i = 0; i < count; i++) + printf "%s %s A <a@example.com> %d +0000\tentry %d\n", oid, oid, 1700000000 + i, i + } + '\'' >.git/logs/refs/heads/source && + if test "$GIT_TEST_DEFAULT_REF_FORMAT" = reftable + then + git refs migrate --ref-format=reftable + fi + ) + ' + + test_perf "copy $entries reflog entries (10 operations)" ' + for i in $(test_seq 10) + do + git -C "repo-$entries" branch -C source destination || return 1 + done + ' + + test_perf "rename $entries reflog entries (10 operations)" ' + for i in $(test_seq 5) + do + git -C "repo-$entries" branch -m source renamed && + git -C "repo-$entries" branch -m renamed source || return 1 + done + ' +done + +test_done diff --git a/t/t1424-ref-copy-transaction.sh b/t/t1424-ref-copy-transaction.sh new file mode 100755 index 0000000000..4c7592795f --- /dev/null +++ b/t/t1424-ref-copy-transaction.sh @@ -0,0 +1,352 @@ +#!/bin/sh + +test_description='reference transactions for copy and rename' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +. ./test-lib.sh + +snapshot () { + git for-each-ref --format="%(refname) %(objectname) %(symref)" >"$1.refs" && + for ref in HEAD refs/heads/source refs/heads/destination + do + if git reflog exists "$ref" + then + test-tool ref-store main for-each-reflog-ent "$ref" + else + echo missing + fi >"$1.$(basename "$ref").log" || return 1 + done +} + +compare_snapshot () { + for suffix in refs HEAD.log source.log destination.log + do + test_cmp "$1.$suffix" "$2.$suffix" || return 1 + done +} + +test_expect_success 'setup' ' + test_commit A && + test_commit B && + A=$(git rev-parse A) && + B=$(git rev-parse B) +' + +for operation in rename copy +do + test_expect_success "$operation reports one logical transaction" ' + test_when_finished "git config --unset core.hooksPath" && + git branch -f source "$A" && + git branch -f destination "$B" && + mkdir -p hooks && + write_script hooks/reference-transaction <<-\EOF && + echo "$1" >>actual && + cat >>actual + EOF + git config core.hooksPath hooks && + >actual && + test-tool ref-store main copy-transaction "$operation" refs/heads/source refs/heads/destination && + { + echo preparing && + if test "$operation" = rename + then + echo "$ZERO_OID $ZERO_OID refs/heads/source" + fi && + echo "$ZERO_OID $A refs/heads/destination" && + for state in prepared committed + do + echo "$state" && + if test "$operation" = rename + then + echo "$A $ZERO_OID refs/heads/source" + fi && + echo "$B $A refs/heads/destination" || return 1 + done + } >expect && + test_cmp expect actual + ' + + for state in preparing prepared + do + test_expect_success "$operation rejected at $state leaves refs and full reflogs unchanged" ' + test_when_finished "git config --unset core.hooksPath" && + git branch -f source "$A" && + git branch -f destination "$B" && + git symbolic-ref HEAD refs/heads/source && + test_when_finished "git -c core.hooksPath=/dev/null symbolic-ref HEAD refs/heads/main" && + snapshot before && + write_script hooks/reference-transaction <<-EOF && + test "\$1" != "$state" + EOF + git config core.hooksPath hooks && + test_must_fail test-tool ref-store main copy-transaction "$operation" refs/heads/source refs/heads/destination && + snapshot after && + compare_snapshot before after + ' + done +done + +test_expect_success 'prepared hook sees old refs and cannot modify the source' ' + test_when_finished "git config --unset core.hooksPath" && + git branch -f source "$A" && + git branch -f destination "$B" && + write_script hooks/reference-transaction <<-EOF && + test "\$1" = prepared || exit 0 + git rev-parse refs/heads/source >source-seen && + git rev-parse refs/heads/destination >destination-seen && + if git -c core.hooksPath=/dev/null -c core.filesRefLockTimeout=0 \ + update-ref refs/heads/source $B + then + exit 1 + fi + EOF + git config core.hooksPath hooks && + git branch -M source destination && + echo "$A" >expect && + test_cmp expect source-seen && + echo "$B" >expect && + test_cmp expect destination-seen && + test_cmp_rev "$A" destination +' + +test_expect_success 'preparing may update source and its reflog' ' + test_when_finished "git config --unset core.hooksPath" && + git branch source "$A" && + write_script hooks/reference-transaction <<-EOF && + test "\$1" = preparing || exit 0 + git -c core.hooksPath=/dev/null update-ref -m concurrent refs/heads/source $B + EOF + git config core.hooksPath hooks && + git branch -M source destination && + test_cmp_rev "$B" destination && + git reflog show --format=%gs destination >actual && + test_grep concurrent actual +' + +test_expect_success 'rename without a source reflog can be rejected without losing destination history' ' + test_when_finished "git config --unset core.hooksPath" && + git -c core.logAllRefUpdates=false branch source "$A" && + test_must_fail git reflog exists refs/heads/source && + test-tool ref-store main for-each-reflog-ent refs/heads/destination >before && + write_script hooks/reference-transaction <<-\EOF && + test "$1" != prepared + EOF + git config core.hooksPath hooks && + test_must_fail git branch -M source destination && + test-tool ref-store main for-each-reflog-ent refs/heads/destination >after && + test_cmp before after +' + +for names in "parent parent/child" "child/branch child" +do + set -- $names + from=$1 to=$2 + test_expect_success "D/F rename $from to $to can be aborted and committed" ' + test_when_finished "git config --unset core.hooksPath" && + git branch "$from" "$A" && + test-tool ref-store main for-each-reflog-ent "refs/heads/$from" >before && + write_script hooks/reference-transaction <<-EOF && + test "\$1" = prepared || exit 0 + git rev-parse refs/heads/$from >seen + exit 1 + EOF + git config core.hooksPath hooks && + test_must_fail git branch -m "$from" "$to" && + echo "$A" >expect && + test_cmp expect seen && + test_cmp_rev "$A" "$from" && + test-tool ref-store main for-each-reflog-ent "refs/heads/$from" >after && + test_cmp before after && + git -c core.hooksPath=/dev/null branch -m "$from" "$to" && + test_cmp_rev "$A" "$to" + ' +done + +test_expect_success REFFILES,POSIXPERM 'D/F reflog installation failure restores source history' ' + test_when_finished "chmod u+w .git/logs/refs/heads" && + test_when_finished "git config --unset core.hooksPath" && + git branch rollback/branch "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/rollback/branch >before && + write_script hooks/reference-transaction <<-\EOF && + test "$1" = prepared || exit 0 + chmod a-w .git/logs/refs/heads + EOF + git config core.hooksPath hooks && + test_must_fail git branch -m rollback/branch rollback && + chmod u+w .git/logs/refs/heads && + test_cmp_rev "$A" rollback/branch && + test_must_fail git show-ref --verify refs/heads/rollback && + test-tool ref-store main for-each-reflog-ent refs/heads/rollback/branch >after && + test_cmp before after +' + +test_expect_success 'multiple renames, a copy and an ordinary update compose' ' + git branch first "$A" && + git branch second "$B" && + git branch third "$A" && + test-tool ref-store main copy-transaction \ + rename refs/heads/first refs/heads/first-new \ + rename refs/heads/second refs/heads/second-new \ + copy refs/heads/third refs/heads/third-new \ + update refs/heads/fourth "$B" && + test_must_fail git show-ref --verify refs/heads/first && + test_must_fail git show-ref --verify refs/heads/second && + test_cmp_rev "$A" first-new && + test_cmp_rev "$B" second-new && + test_cmp_rev "$A" third-new && + test_cmp_rev "$A" third && + test_cmp_rev "$B" fourth +' + +test_expect_success 'copy preserves complete source history and replaces destination history' ' + git branch history-source "$A" && + git update-ref -m history-step refs/heads/history-source "$B" && + git branch history-destination "$B" && + test-tool ref-store main for-each-reflog-ent refs/heads/history-source >before && + git branch -C history-source history-destination && + test-tool ref-store main for-each-reflog-ent refs/heads/history-source >after && + test_cmp before after && + test-tool ref-store main for-each-reflog-ent refs/heads/history-destination >copied && + sed "$ d" copied >history && + test_cmp before history && + test_grep "Branch: copied refs/heads/history-source to refs/heads/history-destination" copied +' + +test_expect_success 'packed rename reports no internal transactions' ' + test_when_finished "git config --unset core.hooksPath" && + git branch packed-source "$A" && + git pack-refs --all && + write_script hooks/reference-transaction <<-\EOF && + echo "$1" >>states + cat >/dev/null + EOF + git config core.hooksPath hooks && + git branch -m packed-source packed-destination && + printf "%s\n" preparing prepared committed >expect && + test_cmp expect states +' + +test_expect_success REFFILES 'unrelated forced copies use independent reflog staging files' ' + test_when_finished "git config --unset core.hooksPath" && + git branch nested-a "$A" && + git branch nested-b "$B" && + git branch outer-a "$A" && + git branch outer-b "$B" && + write_script hooks/reference-transaction <<-\EOF && + test "$1" = prepared || exit 0 + git -c core.hooksPath=/dev/null branch -C nested-a nested-b + EOF + git config core.hooksPath hooks && + git branch -C outer-a outer-b && + test_cmp_rev "$A" outer-b && + test_cmp_rev "$A" nested-b +' + +test_expect_success REFFILES 'rename with reflogs disabled does not create a reflog' ' + git -c core.logAllRefUpdates=false branch no-log "$A" && + git -c core.logAllRefUpdates=false branch -m no-log still-no-log && + test_must_fail git reflog exists refs/heads/still-no-log +' + +test_expect_success REFFILES 'copy without source history retains existing destination history' ' + git -c core.logAllRefUpdates=false branch no-copy-log "$A" && + git branch existing-copy-log "$B" && + test-tool ref-store main for-each-reflog-ent refs/heads/existing-copy-log >before && + git branch -C no-copy-log existing-copy-log && + test-tool ref-store main for-each-reflog-ent refs/heads/existing-copy-log >after && + sed "$ d" after >history && + test_cmp before history +' + +test_expect_success REFFILES,CASE_INSENSITIVE_FS 'case-only rename retains history and supports abort' ' + test_when_finished "git config --unset core.hooksPath" && + git branch case-source "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/case-source >before && + write_script hooks/reference-transaction <<-\EOF && + test "$1" != prepared + EOF + git config core.hooksPath hooks && + test_must_fail git branch -M case-source CASE-SOURCE && + test-tool ref-store main for-each-reflog-ent refs/heads/case-source >after && + test_cmp before after && + git -c core.hooksPath=/dev/null branch -M case-source CASE-SOURCE && + git for-each-ref --format="%(refname)" refs/heads/CASE-SOURCE >actual && + echo refs/heads/CASE-SOURCE >expect && + test_cmp expect actual +' + +test_expect_success REFFILES,SYMLINKS 'symlink source reflog is rejected without changing refs' ' + git branch symlink-source "$A" && + mv .git/logs/refs/heads/symlink-source saved-log && + ln -s "$PWD/saved-log" .git/logs/refs/heads/symlink-source && + test_must_fail git branch -m symlink-source symlink-destination && + test_cmp_rev "$A" symlink-source && + test_must_fail git show-ref --verify refs/heads/symlink-destination +' + +for operation in copy rename +do + test_expect_success "$operation over HEAD referent preserves HEAD history" ' + git branch -f head-source "$A" && + git update-ref refs/heads/main "$B" && + test-tool ref-store main for-each-reflog-ent HEAD >before && + test-tool ref-store main copy-transaction "$operation" refs/heads/head-source refs/heads/main && + test-tool ref-store main for-each-reflog-ent HEAD >after && + sed "$ d" after >history && + test_cmp before history && + test_cmp_rev "$A" HEAD + ' +done + +test_expect_success 'preparing may delete source but cannot cause a partial rename' ' + test_when_finished "git config --unset core.hooksPath" && + git branch race-source "$A" && + write_script hooks/reference-transaction <<-\EOF && + test "$1" = preparing || exit 0 + git -c core.hooksPath=/dev/null update-ref -d refs/heads/race-source + EOF + git config core.hooksPath hooks && + test_must_fail git branch -m race-source race-destination && + test_must_fail git show-ref --verify refs/heads/race-source && + test_must_fail git show-ref --verify refs/heads/race-destination +' + +test_expect_success 'copy cannot replace a D/F-conflicting source' ' + git branch conflict-source "$A" && + test_must_fail git branch -c conflict-source conflict-source/child && + test_cmp_rev "$A" conflict-source && + test_must_fail git show-ref --verify refs/heads/conflict-source/child +' + +test_expect_success REFFILES,POSIXPERM 'unreadable source reflog leaves refs and history unchanged' ' + git branch unreadable "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/unreadable >before && + test_when_finished "chmod u+r .git/logs/refs/heads/unreadable" && + chmod a-r .git/logs/refs/heads/unreadable && + test_must_fail git branch -m unreadable readable && + chmod u+r .git/logs/refs/heads/unreadable && + test_cmp_rev "$A" unreadable && + test_must_fail git show-ref --verify refs/heads/readable && + test-tool ref-store main for-each-reflog-ent refs/heads/unreadable >after && + test_cmp before after +' + +test_expect_success REFFILES 'destination lock failure preserves source and destination history' ' + git branch -f source "$A" && + git branch -f destination "$B" && + snapshot before && + test_when_finished "rm -f .git/refs/heads/destination.lock" && + >.git/refs/heads/destination.lock && + test_must_fail git -c core.filesRefLockTimeout=0 branch -M source destination && + snapshot after && + compare_snapshot before after +' + +test_expect_success REFFILES 'staged reflog files are cleaned up after success and abort' ' + find .git/logs -name ".tmp-reflog-*" >actual && + test_must_be_empty actual +' + +test_done -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v3 4/4] refs: remove backend-specific copy and rename callbacks 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz ` (2 preceding siblings ...) 2026-10-07 18:05 ` [PATCH v3 3/4] refs: run copy and rename through ordinary transactions Maciej Ciemborowicz @ 2026-10-07 18:05 ` Maciej Ciemborowicz 2026-10-07 19:55 ` [PATCH v3 0/4] refs: run copy and rename through transactions Junio C Hamano 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-07 18:05 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Copy and rename now use the transaction API for refs and reflogs. Remove the unused callbacks, backend implementations, and their private helpers. The files backend no longer needs a shared temporary reflog name or rollback writes that reconstruct previously visible refs and logs. Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- refs/debug.c | 24 --- refs/files-backend.c | 334 ---------------------------------------- refs/packed-backend.c | 2 - refs/refs-internal.h | 9 -- refs/reftable-backend.c | 287 ---------------------------------- 5 files changed, 656 deletions(-) diff --git a/refs/debug.c b/refs/debug.c index 639db0f26e..b4991f71ae 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -143,28 +143,6 @@ static int debug_optimize_required(struct ref_store *ref_store, return res; } -static int debug_rename_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = drefs->refs->be->rename_ref(drefs->refs, oldref, newref, - logmsg); - trace_printf_key(&trace_refs, "rename_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - -static int debug_copy_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = - drefs->refs->be->copy_ref(drefs->refs, oldref, newref, logmsg); - trace_printf_key(&trace_refs, "copy_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - struct debug_ref_iterator { struct ref_iterator base; struct ref_iterator *iter; @@ -453,8 +431,6 @@ struct ref_storage_be refs_be_debug = { .optimize = debug_optimize, .optimize_required = debug_optimize_required, - .rename_ref = debug_rename_ref, - .copy_ref = debug_copy_ref, .iterator_begin = debug_ref_iterator_begin, .read_raw_ref = debug_read_raw_ref, diff --git a/refs/files-backend.c b/refs/files-backend.c index 9e22f62d3f..a00cfc6be5 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1623,273 +1623,6 @@ static int files_optimize_required(struct ref_store *ref_store, return 0; } -/* - * People using contrib's git-new-workdir have .git/logs/refs -> - * /some/other/path/.git/logs/refs, and that may live on another device. - * - * IOW, to avoid cross device rename errors, the temporary renamed log must - * live into logs/refs. - */ -#define TMP_RENAMED_LOG "refs/.tmp-renamed-log" - -struct rename_cb { - const char *tmp_renamed_log; - int true_errno; -}; - -static int rename_tmp_log_callback(const char *path, void *cb_data) -{ - struct rename_cb *cb = cb_data; - - if (rename(cb->tmp_renamed_log, path)) { - /* - * rename(a, b) when b is an existing directory ought - * to result in ISDIR, but Solaris 5.8 gives ENOTDIR. - * Sheesh. Record the true errno for error reporting, - * but report EISDIR to raceproof_create_file() so - * that it knows to retry. - */ - cb->true_errno = errno; - if (errno == ENOTDIR) - errno = EISDIR; - return -1; - } else { - return 0; - } -} - -static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname) -{ - struct strbuf path = STRBUF_INIT; - struct strbuf tmp = STRBUF_INIT; - struct rename_cb cb; - int ret; - - files_reflog_path(refs, &path, newrefname); - files_reflog_path(refs, &tmp, TMP_RENAMED_LOG); - cb.tmp_renamed_log = tmp.buf; - ret = raceproof_create_file(refs, path.buf, rename_tmp_log_callback, &cb); - if (ret) { - if (errno == EISDIR) - error("directory not empty: %s", path.buf); - else - error("unable to move logfile %s to %s: %s", - tmp.buf, path.buf, - strerror(cb.true_errno)); - } - - strbuf_release(&path); - strbuf_release(&tmp); - return ret; -} - -static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs, - struct ref_lock *lock, - const struct object_id *oid, - struct strbuf *err); -static int commit_ref_update(struct files_ref_store *refs, - struct ref_lock *lock, - const struct object_id *oid, const char *logmsg, - int flags, - struct strbuf *err); - -/* - * Emit a better error message than lockfile.c's - * unable_to_lock_message() would in case there is a D/F conflict with - * another existing reference. If there would be a conflict, emit an error - * message and return false; otherwise, return true. - * - * Note that this function is not safe against all races with other - * processes, and that's not its job. We'll emit a more verbose error on D/f - * conflicts if we get past it into lock_ref_oid_basic(). - */ -static int refs_rename_ref_available(struct ref_store *refs, - const char *old_refname, - const char *new_refname) -{ - struct string_list skip = STRING_LIST_INIT_NODUP; - struct strbuf err = STRBUF_INIT; - int ok; - - string_list_insert(&skip, old_refname); - ok = !refs_verify_refname_available(refs, new_refname, - NULL, &skip, 0, &err); - if (!ok) - error("%s", err.buf); - - string_list_clear(&skip, 0); - strbuf_release(&err); - return ok; -} - -static int files_copy_or_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg, int copy) -{ - struct files_ref_store *refs = - files_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); - struct object_id orig_oid; - int flag = 0, logmoved = 0; - struct ref_lock *lock; - struct stat loginfo; - struct strbuf sb_oldref = STRBUF_INIT; - struct strbuf sb_newref = STRBUF_INIT; - struct strbuf tmp_renamed_log = STRBUF_INIT; - int log, ret; - struct strbuf err = STRBUF_INIT; - - files_reflog_path(refs, &sb_oldref, oldrefname); - files_reflog_path(refs, &sb_newref, newrefname); - files_reflog_path(refs, &tmp_renamed_log, TMP_RENAMED_LOG); - - log = !lstat(sb_oldref.buf, &loginfo); - if (log && S_ISLNK(loginfo.st_mode)) { - ret = error("reflog for %s is a symlink", oldrefname); - goto out; - } - - if (!refs_resolve_ref_unsafe(&refs->base, oldrefname, - RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, - &orig_oid, &flag)) { - ret = error("refname %s not found", oldrefname); - goto out; - } - - if (flag & REF_ISSYMREF) { - if (copy) - ret = error("refname %s is a symbolic ref, copying it is not supported", - oldrefname); - else - ret = error("refname %s is a symbolic ref, renaming it is not supported", - oldrefname); - goto out; - } - if (!refs_rename_ref_available(&refs->base, oldrefname, newrefname)) { - ret = 1; - goto out; - } - - if (!copy && log && rename(sb_oldref.buf, tmp_renamed_log.buf)) { - ret = error("unable to move logfile logs/%s to logs/"TMP_RENAMED_LOG": %s", - oldrefname, strerror(errno)); - goto out; - } - - if (copy && log && copy_file(refs->base.repo, tmp_renamed_log.buf, sb_oldref.buf, 0644)) { - ret = error("unable to copy logfile logs/%s to logs/"TMP_RENAMED_LOG": %s", - oldrefname, strerror(errno)); - goto out; - } - - if (!copy && refs_delete_ref(&refs->base, logmsg, oldrefname, - &orig_oid, REF_NO_DEREF)) { - error("unable to delete old %s", oldrefname); - goto rollback; - } - - /* - * Since we are doing a shallow lookup, oid is not the - * correct value to pass to delete_ref as old_oid. But that - * doesn't matter, because an old_oid check wouldn't add to - * the safety anyway; we want to delete the reference whatever - * its current value. - */ - if (!copy && refs_resolve_ref_unsafe(&refs->base, newrefname, - RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, - NULL, NULL) && - refs_delete_ref(&refs->base, NULL, newrefname, - NULL, REF_NO_DEREF)) { - if (errno == EISDIR) { - struct strbuf path = STRBUF_INIT; - int result; - - files_ref_path(refs, &path, newrefname); - result = remove_empty_directories(&path); - strbuf_release(&path); - - if (result) { - error("Directory not empty: %s", newrefname); - goto rollback; - } - } else { - error("unable to delete existing %s", newrefname); - goto rollback; - } - } - - if (log && rename_tmp_log(refs, newrefname)) - goto rollback; - - logmoved = log; - - lock = lock_ref_oid_basic(refs, newrefname, &err); - if (!lock) { - if (copy) - error("unable to copy '%s' to '%s': %s", oldrefname, newrefname, err.buf); - else - error("unable to rename '%s' to '%s': %s", oldrefname, newrefname, err.buf); - strbuf_release(&err); - goto rollback; - } - oidcpy(&lock->old_oid, &orig_oid); - - if (write_ref_to_lockfile(refs, lock, &orig_oid, &err) || - commit_ref_update(refs, lock, &orig_oid, logmsg, 0, &err)) { - error("unable to write current sha1 into %s: %s", newrefname, err.buf); - strbuf_release(&err); - goto rollback; - } - - ret = 0; - goto out; - - rollback: - lock = lock_ref_oid_basic(refs, oldrefname, &err); - if (!lock) { - error("unable to lock %s for rollback: %s", oldrefname, err.buf); - strbuf_release(&err); - goto rollbacklog; - } - - if (write_ref_to_lockfile(refs, lock, &orig_oid, &err) || - commit_ref_update(refs, lock, &orig_oid, NULL, REF_SKIP_CREATE_REFLOG, &err)) { - error("unable to write current sha1 into %s: %s", oldrefname, err.buf); - strbuf_release(&err); - } - - rollbacklog: - if (logmoved && rename(sb_newref.buf, sb_oldref.buf)) - error("unable to restore logfile %s from %s: %s", - oldrefname, newrefname, strerror(errno)); - if (!logmoved && log && - rename(tmp_renamed_log.buf, sb_oldref.buf)) - error("unable to restore logfile %s from logs/"TMP_RENAMED_LOG": %s", - oldrefname, strerror(errno)); - ret = 1; - out: - strbuf_release(&sb_newref); - strbuf_release(&sb_oldref); - strbuf_release(&tmp_renamed_log); - - return ret; -} - -static int files_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 0); -} - -static int files_copy_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 1); -} - static int close_ref_gently(struct ref_lock *lock) { if (close_lock_file_gently(&lock->lk)) @@ -2121,71 +1854,6 @@ static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store * return 0; } -/* - * Commit a change to a loose reference that has already been written - * to the loose reference lockfile. Also update the reflogs if - * necessary, using the specified lockmsg (which can be NULL). - */ -static int commit_ref_update(struct files_ref_store *refs, - struct ref_lock *lock, - const struct object_id *oid, const char *logmsg, - int flags, - struct strbuf *err) -{ - files_assert_main_repository(refs, "commit_ref_update"); - - clear_loose_ref_cache(refs); - if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, oid, NULL, - logmsg, flags, err)) { - char *old_msg = strbuf_detach(err, NULL); - strbuf_addf(err, "cannot update the ref '%s': %s", - lock->ref_name, old_msg); - free(old_msg); - unlock_ref(lock); - return -1; - } - - if (strcmp(lock->ref_name, "HEAD") != 0) { - /* - * Special hack: If a branch is updated directly and HEAD - * points to it (may happen on the remote side of a push - * for example) then logically the HEAD reflog should be - * updated too. - * A generic solution implies reverse symref information, - * but finding all symrefs pointing to the given branch - * would be rather costly for this rare event (the direct - * update of a branch) to be worth it. So let's cheat and - * check with HEAD only which should cover 99% of all usage - * scenarios (even 100% of the default ones). - */ - int head_flag; - const char *head_ref; - - head_ref = refs_resolve_ref_unsafe(&refs->base, "HEAD", - RESOLVE_REF_READING, - NULL, &head_flag); - if (head_ref && (head_flag & REF_ISSYMREF) && - !strcmp(head_ref, lock->ref_name)) { - struct strbuf log_err = STRBUF_INIT; - if (files_log_ref_write(refs, "HEAD", &lock->old_oid, - oid, NULL, logmsg, flags, - &log_err)) { - error("%s", log_err.buf); - strbuf_release(&log_err); - } - } - } - - if (commit_ref(lock)) { - strbuf_addf(err, "couldn't set '%s'", lock->ref_name); - unlock_ref(lock); - return -1; - } - - unlock_ref(lock); - return 0; -} - #if defined(NO_SYMLINK_HEAD) || defined(WITH_BREAKING_CHANGES) #define create_ref_symlink(a, b) (-1) #else @@ -4468,8 +4136,6 @@ struct ref_storage_be refs_be_files = { .optimize = files_optimize, .optimize_required = files_optimize_required, - .rename_ref = files_rename_ref, - .copy_ref = files_copy_ref, .iterator_begin = files_ref_iterator_begin, .read_raw_ref = files_read_raw_ref, diff --git a/refs/packed-backend.c b/refs/packed-backend.c index a73fc6aca7..364a912912 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -2164,8 +2164,6 @@ struct ref_storage_be refs_be_packed = { .optimize = packed_optimize, .optimize_required = packed_optimize_required, - .rename_ref = NULL, - .copy_ref = NULL, .iterator_begin = packed_ref_iterator_begin, .read_raw_ref = packed_read_raw_ref, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 2746ce5b49..c415ba0a4e 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -470,13 +470,6 @@ typedef int optimize_required_fn(struct ref_store *ref_store, struct refs_optimize_opts *opts, bool *required); -typedef int rename_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); -typedef int copy_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); - /* * Iterate over the references in `ref_store` whose names start with * `prefix`. `prefix` is matched as a literal string, without regard @@ -596,8 +589,6 @@ struct ref_storage_be { optimize_fn *optimize; optimize_required_fn *optimize_required; - rename_ref_fn *rename_ref; - copy_ref_fn *copy_ref; ref_iterator_begin_fn *iterator_begin; read_raw_ref_fn *read_raw_ref; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 72c1596c11..ef3bab69ee 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1798,290 +1798,6 @@ struct write_create_symref_arg { const char *logmsg; }; -struct write_copy_arg { - struct reftable_ref_store *refs; - struct reftable_backend *be; - const char *oldname; - const char *newname; - const char *logmsg; - int delete_old; -}; - -static int write_copy_table(struct reftable_writer *writer, void *cb_data) -{ - struct write_copy_arg *arg = cb_data; - uint64_t deletion_ts, creation_ts; - struct reftable_ref_record old_ref = {0}, refs[2] = {0}; - struct reftable_log_record old_log = {0}, *logs = NULL; - struct reftable_iterator it = {0}; - struct string_list skip = STRING_LIST_INIT_NODUP; - struct ident_split committer_ident = {0}; - struct strbuf errbuf = STRBUF_INIT; - size_t logs_nr = 0, logs_alloc = 0, i; - const char *committer_info; - int ret; - - committer_info = git_committer_info(0); - if (split_ident_line(&committer_ident, committer_info, strlen(committer_info))) - BUG("failed splitting committer info"); - - if (reftable_stack_read_ref(arg->be->stack, arg->oldname, &old_ref)) { - ret = error(_("refname %s not found"), arg->oldname); - goto done; - } - if (old_ref.value_type == REFTABLE_REF_SYMREF) { - ret = error(_("refname %s is a symbolic ref, copying it is not supported"), - arg->oldname); - goto done; - } - - /* - * There's nothing to do in case the old and new name are the same, so - * we exit early in that case. - */ - if (!strcmp(arg->oldname, arg->newname)) { - ret = 0; - goto done; - } - - /* - * Verify that the new refname is available. - */ - if (arg->delete_old) - string_list_insert(&skip, arg->oldname); - ret = refs_verify_refname_available(&arg->refs->base, arg->newname, - NULL, &skip, 0, &errbuf); - if (ret < 0) { - error("%s", errbuf.buf); - goto done; - } - - /* - * When deleting the old reference we have to use two update indices: - * once to delete the old ref and its reflog, and once to create the - * new ref and its reflog. They need to be staged with two separate - * indices because the new reflog needs to encode both the deletion of - * the old branch and the creation of the new branch, and we cannot do - * two changes to a reflog in a single update. - */ - deletion_ts = creation_ts = reftable_stack_next_update_index(arg->be->stack); - if (arg->delete_old) - creation_ts++; - ret = reftable_writer_set_limits(writer, deletion_ts, creation_ts); - if (ret < 0) - goto done; - - /* - * Add the new reference. If this is a rename then we also delete the - * old reference. - */ - refs[0] = old_ref; - refs[0].refname = xstrdup(arg->newname); - refs[0].update_index = creation_ts; - if (arg->delete_old) { - refs[1].refname = xstrdup(arg->oldname); - refs[1].value_type = REFTABLE_REF_DELETION; - refs[1].update_index = deletion_ts; - } - ret = reftable_writer_add_refs(writer, refs, arg->delete_old ? 2 : 1); - if (ret < 0) - goto done; - - /* - * When deleting the old branch we need to create a reflog entry on the - * new branch name that indicates that the old branch has been deleted - * and then recreated. This is a tad weird, but matches what the files - * backend does. - */ - if (arg->delete_old) { - struct strbuf head_referent = STRBUF_INIT; - struct object_id head_oid; - int append_head_reflog; - unsigned head_type = 0; - - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - memset(&logs[logs_nr], 0, sizeof(logs[logs_nr])); - fill_reftable_log_record(&logs[logs_nr], &committer_ident); - logs[logs_nr].refname = xstrdup(arg->newname); - logs[logs_nr].update_index = deletion_ts; - logs[logs_nr].value.update.message = - xstrndup(arg->logmsg, reftable_be_write_options(arg->refs)->opts.block_size / 2); - memcpy(logs[logs_nr].value.update.old_hash, old_ref.value.val1, GIT_MAX_RAWSZ); - logs_nr++; - - ret = reftable_backend_read_ref(arg->be, "HEAD", &head_oid, - &head_referent, &head_type); - if (ret < 0) - goto done; - append_head_reflog = (head_type & REF_ISSYMREF) && !strcmp(head_referent.buf, arg->oldname); - strbuf_release(&head_referent); - - /* - * The files backend uses `refs_delete_ref()` to delete the old - * branch name, which will append a reflog entry for HEAD in - * case it points to the old branch. - */ - if (append_head_reflog) { - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - logs[logs_nr] = logs[logs_nr - 1]; - logs[logs_nr].refname = xstrdup("HEAD"); - logs[logs_nr].value.update.name = - xstrdup(logs[logs_nr].value.update.name); - logs[logs_nr].value.update.email = - xstrdup(logs[logs_nr].value.update.email); - logs[logs_nr].value.update.message = - xstrdup(logs[logs_nr].value.update.message); - logs_nr++; - } - } - - /* - * Create the reflog entry for the newly created branch. - */ - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - memset(&logs[logs_nr], 0, sizeof(logs[logs_nr])); - fill_reftable_log_record(&logs[logs_nr], &committer_ident); - logs[logs_nr].refname = xstrdup(arg->newname); - logs[logs_nr].update_index = creation_ts; - logs[logs_nr].value.update.message = - xstrndup(arg->logmsg, reftable_be_write_options(arg->refs)->opts.block_size / 2); - memcpy(logs[logs_nr].value.update.new_hash, old_ref.value.val1, GIT_MAX_RAWSZ); - logs_nr++; - - /* - * In addition to writing the reflog entry for the new branch, we also - * copy over all log entries from the old reflog. Last but not least, - * when renaming we also have to delete all the old reflog entries. - */ - ret = reftable_stack_init_log_iterator(arg->be->stack, &it); - if (ret < 0) - goto done; - - ret = reftable_iterator_seek_log(&it, arg->oldname); - if (ret < 0) - goto done; - - while (1) { - ret = reftable_iterator_next_log(&it, &old_log); - if (ret < 0) - goto done; - if (ret > 0 || strcmp(old_log.refname, arg->oldname)) { - ret = 0; - break; - } - if (reftable_log_record_is_deletion(&old_log)) - continue; - - free(old_log.refname); - - /* - * Copy over the old reflog entry with the new refname. - */ - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - logs[logs_nr] = old_log; - logs[logs_nr].refname = xstrdup(arg->newname); - logs_nr++; - - /* - * Delete the old reflog entry in case we are renaming. - */ - if (arg->delete_old) { - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - memset(&logs[logs_nr], 0, sizeof(logs[logs_nr])); - logs[logs_nr].refname = xstrdup(arg->oldname); - logs[logs_nr].value_type = REFTABLE_LOG_DELETION; - logs[logs_nr].update_index = old_log.update_index; - logs_nr++; - } - - /* - * Transfer ownership of the log record we're iterating over to - * the array of log records. Otherwise, the pointers would get - * free'd or reallocated by the iterator. - */ - memset(&old_log, 0, sizeof(old_log)); - } - - ret = reftable_writer_add_logs(writer, logs, logs_nr); - if (ret < 0) - goto done; - -done: - assert(ret != REFTABLE_API_ERROR); - reftable_iterator_destroy(&it); - string_list_clear(&skip, 0); - strbuf_release(&errbuf); - for (i = 0; i < logs_nr; i++) - reftable_log_record_release(&logs[i]); - free(logs); - for (i = 0; i < ARRAY_SIZE(refs); i++) - reftable_ref_record_release(&refs[i]); - reftable_ref_record_release(&old_ref); - reftable_log_record_release(&old_log); - return ret; -} - -static int reftable_be_rename_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) -{ - struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); - struct write_copy_arg arg = { - .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - .delete_old = 1, - }; - int ret; - - ret = refs->err; - if (ret < 0) - goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); - if (ret) - goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); - -done: - assert(ret != REFTABLE_API_ERROR); - return ret; -} - -static int reftable_be_copy_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) -{ - struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); - struct write_copy_arg arg = { - .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - }; - int ret; - - ret = refs->err; - if (ret < 0) - goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); - if (ret) - goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); - -done: - assert(ret != REFTABLE_API_ERROR); - return ret; -} - struct reftable_reflog_iterator { struct ref_iterator base; struct reftable_ref_store *refs; @@ -2909,9 +2625,6 @@ struct ref_storage_be refs_be_reftable = { .optimize = reftable_be_optimize, .optimize_required = reftable_be_optimize_required, - .rename_ref = reftable_be_rename_ref, - .copy_ref = reftable_be_copy_ref, - .iterator_begin = reftable_be_iterator_begin, .read_raw_ref = reftable_be_read_raw_ref, .read_symbolic_ref = reftable_be_read_symbolic_ref, -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [PATCH v3 0/4] refs: run copy and rename through transactions 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz ` (3 preceding siblings ...) 2026-10-07 18:05 ` [PATCH v3 4/4] refs: remove backend-specific copy and rename callbacks Maciej Ciemborowicz @ 2026-10-07 19:55 ` Junio C Hamano 4 siblings, 0 replies; 35+ messages in thread From: Junio C Hamano @ 2026-10-07 19:55 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: git, Patrick Steinhardt, Karthik Nayak Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> writes: > Reference copy and rename bypass the transaction API. With files, the > reference-transaction hook sees only the source deletion; with reftable, > it sees neither endpoint. This series puts the logical ref updates and > the reflog history into ordinary transactions so hooks can observe and > reject the complete operation. > ... > Documentation/githooks.adoc | 10 + > refs.c | 200 ++++++++- > refs.h | 25 ++ > refs/debug.c | 24 -- > refs/files-backend.c | 739 +++++++++++++++++--------------- > refs/packed-backend.c | 2 - > refs/refs-internal.h | 28 +- > refs/reftable-backend.c | 334 ++------------- > t/helper/test-ref-store.c | 76 ++++ > t/perf/p1424-ref-copy-rename.sh | 48 +++ > t/t1424-ref-copy-transaction.sh | 352 +++++++++++++++ > t/t1425-reflog-transaction.sh | 59 +++ > 12 files changed, 1206 insertions(+), 691 deletions(-) > create mode 100755 t/perf/p1424-ref-copy-rename.sh > create mode 100755 t/t1424-ref-copy-transaction.sh > create mode 100755 t/t1425-reflog-transaction.sh > > Range-diff against v2: > 1: d852537d8c < -: ---------- refs: run copy and rename through transactions > -: ---------- > 1: 6d7c146e57 refs: distinguish internal transactions from logical updates > -: ---------- > 2: 5c3ec4eb49 refs: support replacing reflogs in a transaction > -: ---------- > 3: 77af4e809c refs: run copy and rename through ordinary transactions > -: ---------- > 4: 83fa644fb3 refs: remove backend-specific copy and rename callbacks I do not think I have time to read these humoungous patches, some of which weigh more than 1000+ lines, with fine toothed comb any time soon, but when applied to the recent tip of master 6de20f6092 (The 4th batch, 2026-10-06), it seems to break t0600 and t5510. The failing tests do not seem to be anything so system specific (and I am on a GNU/Linux platform that is not anything remarkable). I wonder what I am doing differently. Did they pass for you? Here are the first failure from these test scripts. Thanks. expecting success of 0600.16 'delete fails cleanly if packed-refs.new write fails': # Setup and expectations are similar to the test above. prefix=refs/failed-packed-refs && git update-ref $prefix/foo $C && git pack-refs --all && git update-ref $prefix/foo $D && git for-each-ref $prefix >unchanged && # This should not happen in practice, but it is an easy way to get a # reliable error (we open with create_tempfile(), which uses O_EXCL). : >.git/packed-refs.new && test_when_finished "rm -f .git/packed-refs.new" && test_must_fail git update-ref -d $prefix/foo && git for-each-ref $prefix >actual && test_cmp unchanged actual test_must_fail: command succeeded: git update-ref -d refs/failed-packed-refs/foo not ok 16 - delete fails cleanly if packed-refs.new write fails expecting success of 5510.35 'fetch --prune fails to delete branches': git clone . prune-fail && ( cd prune-fail && git update-ref refs/remotes/origin/extrabranch main && git pack-refs --all && : this will prevent --prune from locking packed-refs for deleting refs, but adding loose refs still succeeds && >.git/packed-refs.new && test_must_fail git fetch --prune origin ) Cloning into 'prune-fail'... done. From /usr/local/google/home/jch/w/git.git/t/trash directory.t5510-fetch/. - [deleted] (none) -> origin/extrabranch test_must_fail: command succeeded: git fetch --prune origin not ok 35 - fetch --prune fails to delete branches ^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v4 0/4] refs: run copy and rename through transactions 2026-09-20 16:50 ` [PATCH] refs: run copy and rename through transactions Maciej Ciemborowicz ` (2 preceding siblings ...) 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz @ 2026-10-08 9:44 ` Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz ` (4 more replies) 3 siblings, 5 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 9:44 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Reference copy and rename bypass the transaction API. With files, the reference-transaction hook sees only the source deletion; with reftable, it sees neither endpoint. This series represents the logical ref updates and reflog history through ordinary transactions, allowing hooks to observe and reject the operation. Junio, thank you for reporting the t0600 and t5510 failures in v3. I reproduced both on macOS with the original base as well. I had not run those suites before submitting v3; they were outside the test selection reported in that cover letter. That was a gap in my validation, and I am sorry for sending the regression. Patch 2 added a call to prepare_reflog_replacements() after preparing the packed transaction. If packed preparation failed, its error was saved in ret, but the new call overwrote it with success. The files transaction could then proceed despite the failure to prepare packed-refs. The fix jumps to cleanup after freeing the failed packed transaction, preserving the error and releasing the remaining resources. The existing t0600.16 and t5510.35 cover this path; their expectations have not been changed. Changes since v3: * Rebase onto 6de20f6092 (The 4th batch, 2026-10-06), the master commit used in Junio's report. * Preserve the packed preparation error in patch 2 as described above. * Register t1425 and t1424 in t/meson.build in the commits adding them. The four-patch structure and implementation otherwise remain unchanged. This iteration does not address the concern about the size of the main patch; the range-diff below isolates the correction and registrations. I ran the complete standard core test collection, including the unit-test executable, on macOS/arm64 with each default ref backend: files: Files=1064, Tests=34710, Result: PASS reftable: Files=1064, Tests=34712, Result: PASS Both full runs include successful t0600, t5510, t1424 and t1425 runs. The build enables Perl, Python, cURL and gettext. The files run used: LC_ALL=C make -j8 PYTHON_PATH=/opt/homebrew/bin/python3 \ GNU_GETTEXT_PATH=/opt/homebrew/opt/gettext \ DEFAULT_TEST_TARGET=prove GIT_PROVE_OPTS='--jobs 8' test The reftable run used GIT_TEST_DEFAULT_REF_FORMAT=reftable and ran every t[0-9][0-9][0-9][0-9]-*.sh plus unit-tests/bin/unit-tests through prove, with four jobs and a separate TEST_OUTPUT_DIRECTORY. Backend-specific suites can override the default. The Meson registration check and git diff --check also pass. The contrib test target completed with no unexpected failures; diff-highlight retains two known TODO failures. The harness totals include skips and expected failures. The files run skipped 126 whole scripts, mostly for unavailable SVN, Perforce and CVS tools. Other skips include GPG, JGit, Windows-specific tests, tests requiring sudo or writable /, case-sensitive filesystem tests, the opt-in 2GB clone test, and t5564 because its web-server setup failed. There are also individual prerequisite-based skips within suites. I have not validated this iteration on Linux or Windows. The performance trade-off described in v3 remains: replaying history uses O(N) time and memory, replacing the files backend's constant-time reflog rename. The measurements in patch 3 are from the v3 base, not a new benchmark on this base. Files transactions also remain non-atomic with respect to process crashes and failures late in finish. Previous iteration: https://lore.kernel.org/git/cover.1791395643.git.maciej.ciemborowicz@gmail.com/ Maciej Ciemborowicz (4): refs: distinguish internal transactions from logical updates refs: support replacing reflogs in a transaction refs: run copy and rename through ordinary transactions refs: remove backend-specific copy and rename callbacks Documentation/githooks.adoc | 10 + refs.c | 200 ++++++++- refs.h | 25 ++ refs/debug.c | 24 -- refs/files-backend.c | 740 +++++++++++++++++--------------- refs/packed-backend.c | 2 - refs/refs-internal.h | 28 +- refs/reftable-backend.c | 334 ++------------ t/helper/test-ref-store.c | 76 ++++ t/meson.build | 2 + t/perf/p1424-ref-copy-rename.sh | 48 +++ t/t1424-ref-copy-transaction.sh | 352 +++++++++++++++ t/t1425-reflog-transaction.sh | 59 +++ 13 files changed, 1209 insertions(+), 691 deletions(-) create mode 100755 t/perf/p1424-ref-copy-rename.sh create mode 100755 t/t1424-ref-copy-transaction.sh create mode 100755 t/t1425-reflog-transaction.sh Range-diff against v3: 1: 6d7c146e57 = 1: 948927d8fb refs: distinguish internal transactions from logical updates 2: 5c3ec4eb49 ! 2: d023da3c09 refs: support replacing reflogs in a transaction @@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref transaction->backend_data = backend_data; /* +@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store, + if (ret) { + ref_transaction_free(packed_transaction); + backend_data->packed_transaction = NULL; ++ goto cleanup; + } + } else { + /* @@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store, } } @@ t/helper/test-ref-store.c: static struct command commands[] = { { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, + ## t/meson.build ## +@@ t/meson.build: integration_tests = [ + 't1421-reflog-write.sh', + 't1422-show-ref-exists.sh', + 't1423-ref-backend.sh', ++ 't1425-reflog-transaction.sh', + 't1430-bad-ref-name.sh', + 't1450-fsck.sh', + 't1451-fsck-buffer.sh', + ## t/t1425-reflog-transaction.sh (new) ## @@ +#!/bin/sh 3: 77af4e809c ! 3: 150349f9d0 refs: run copy and rename through ordinary transactions @@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref - if (update->flags & REF_DELETING && + if (update->flags & (REF_DELETING | REF_NEEDS_PACK) && !(update->flags & REF_LOG_ONLY) && - !(update->flags & REF_IS_PRUNING)) { - /* + !(update->flags & REF_IS_PRUNING) && + !is_root_ref(update->refname)) { @@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store, REF_HAVE_NEW | REF_NO_DEREF, &update->new_oid, NULL, NULL, @@ t/helper/test-ref-store.c: static struct command commands[] = { { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, + ## t/meson.build ## +@@ t/meson.build: integration_tests = [ + 't1421-reflog-write.sh', + 't1422-show-ref-exists.sh', + 't1423-ref-backend.sh', ++ 't1424-ref-copy-transaction.sh', + 't1425-reflog-transaction.sh', + 't1430-bad-ref-name.sh', + 't1450-fsck.sh', + ## t/perf/p1424-ref-copy-rename.sh (new) ## @@ +#!/bin/sh 4: 83fa644fb3 = 4: f3f2c7ee11 refs: remove backend-specific copy and rename callbacks base-commit: 6de20f6092dcf9bdb1c8efe03db4b70c82b423dd -- 2.39.3 (Apple Git-146) ^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v4 1/4] refs: distinguish internal transactions from logical updates 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz @ 2026-10-08 9:44 ` Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 2/4] refs: support replacing reflogs in a transaction Maciej Ciemborowicz ` (3 subsequent siblings) 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 9:44 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak A ref backend may use a nested transaction to persist part of a logical update. Provide an internal-only transaction flag so that these physical updates can avoid reporting the same operation to reference-transaction hooks again. Keep the check in run_transaction_hook(), covering every hook state in one place. The flag is deliberately not part of the public refs API. Subsequent changes use it for packed-refs updates belonging to a copy or rename. Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- refs.c | 3 +++ refs/refs-internal.h | 3 +++ 2 files changed, 6 insertions(+) diff --git a/refs.c b/refs.c index 951db56113..642f895b71 100644 --- a/refs.c +++ b/refs.c @@ -2689,6 +2689,9 @@ static int run_transaction_hook(struct ref_transaction *transaction, struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; int ret = 0; + if (transaction->flags & REF_TRANSACTION_FLAG_INTERNAL) + return 0; + strvec_push(&opt.args, state); opt.feed_pipe = transaction_hook_feed_stdin; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index c3ac7b556f..0b41f5fa4b 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -8,6 +8,9 @@ struct fsck_options; struct ref_transaction; +/* Physical updates nested in a transaction already reported to hooks. */ +#define REF_TRANSACTION_FLAG_INTERNAL (1 << 2) + /* * Data structures and functions for the internal use of the refs * module. Code outside of the refs module should use only the public -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 2/4] refs: support replacing reflogs in a transaction 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz @ 2026-10-08 9:44 ` Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 3/4] refs: run copy and rename through ordinary transactions Maciej Ciemborowicz ` (2 subsequent siblings) 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 9:44 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Explicit log-only updates can already recreate a reflog during migration. Add ref_transaction_replace_reflog() to replace existing history with these entries without changing the reference itself. Keep its marker flag private to the refs implementation so callers cannot attach this cross-update behavior to an ordinary ref update accidentally. Stage files-backend replacements in unique temporary files beside the logs and install them only during finish. Buffer writes and order entries by their requested index. Aborting prepare only removes staging files. In reftable, tombstone the previous entries in the same table as the replacement. Do not treat a log-only entry with a null new OID as a reference deletion: it is part of the history being replayed. Propagate read errors instead of silently copying a partial log. Test preservation of null-OID entries, ordering, removal, hook rejection and duplicate replacement. This is preparation for copying and renaming through the ordinary transaction path. Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- refs.c | 34 ++++++- refs.h | 9 ++ refs/files-backend.c | 181 +++++++++++++++++++++++++++++++--- refs/refs-internal.h | 3 + refs/reftable-backend.c | 10 +- t/helper/test-ref-store.c | 40 ++++++++ t/meson.build | 1 + t/t1425-reflog-transaction.sh | 59 +++++++++++ 8 files changed, 322 insertions(+), 15 deletions(-) create mode 100755 t/t1425-reflog-transaction.sh diff --git a/refs.c b/refs.c index 642f895b71..c02da9d966 100644 --- a/refs.c +++ b/refs.c @@ -1436,7 +1436,6 @@ enum ref_transaction_error ref_transaction_update(struct ref_transaction *transa strbuf_addstr(err, _("refusing to force and skip creation of reflog")); return REF_TRANSACTION_ERROR_GENERIC; } - if (!transaction_refname_valid(refname, new_oid, flags, err)) return REF_TRANSACTION_ERROR_GENERIC; @@ -1519,6 +1518,39 @@ int ref_transaction_update_reflog(struct ref_transaction *transaction, return 0; } +int ref_transaction_replace_reflog(struct ref_transaction *transaction, + const char *refname, + struct strbuf *err) +{ + size_t i; + unsigned int flags = REF_LOG_ONLY | REF_NO_DEREF | + REF_SKIP_CREATE_REFLOG | REF_REPLACE_REFLOG; + + assert(err); + if (transaction->flags & + (REF_TRANSACTION_FLAG_INITIAL | REF_TRANSACTION_ALLOW_FAILURE)) { + strbuf_addstr(err, _("reflog replacement requires an " + "all-or-nothing, non-initial transaction")); + return REF_TRANSACTION_ERROR_GENERIC; + } + if (!transaction_refname_valid(refname, NULL, flags, err)) + return REF_TRANSACTION_ERROR_GENERIC; + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + + if ((update->flags & REF_REPLACE_REFLOG) && + !strcmp(update->refname, refname)) { + strbuf_addf(err, _("reflog replacement for '%s' already queued"), + refname); + return REF_TRANSACTION_ERROR_NAME_CONFLICT; + } + } + + ref_transaction_add_update(transaction, refname, flags, + NULL, NULL, NULL, NULL, NULL, NULL, NULL); + return 0; +} + int ref_transaction_create(struct ref_transaction *transaction, const char *refname, const struct object_id *new_oid, diff --git a/refs.h b/refs.h index ee3b8a62ef..b0b2b0e4fd 100644 --- a/refs.h +++ b/refs.h @@ -943,6 +943,15 @@ int ref_transaction_update_reflog(struct ref_transaction *transaction, uint64_t index, struct strbuf *err); +/* + * Replace a reference's reflog with the explicit entries added to the same + * transaction via ref_transaction_update_reflog(). Without such entries, + * remove the reflog. This operation does not change the reference itself. + */ +int ref_transaction_replace_reflog(struct ref_transaction *transaction, + const char *refname, + struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 7ddf0bef7d..36ecd21ed7 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1983,6 +1983,22 @@ static int files_create_reflog(struct ref_store *ref_store, const char *refname, return 0; } +static void format_reflog_entry(struct strbuf *sb, + const struct object_id *old_oid, + const struct object_id *new_oid, + const char *committer, const char *msg) +{ + if (!committer) + committer = git_committer_info(0); + + strbuf_addf(sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); + if (msg && *msg) { + strbuf_addch(sb, '\t'); + strbuf_addstr(sb, msg); + } + strbuf_addch(sb, '\n'); +} + static int log_ref_write_fd(int fd, const struct object_id *old_oid, const struct object_id *new_oid, const char *committer, const char *msg) @@ -1990,15 +2006,7 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, struct strbuf sb = STRBUF_INIT; int ret = 0; - if (!committer) - committer = git_committer_info(0); - - strbuf_addf(&sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); - if (msg && *msg) { - strbuf_addch(&sb, '\t'); - strbuf_addstr(&sb, msg); - } - strbuf_addch(&sb, '\n'); + format_reflog_entry(&sb, old_oid, new_oid, committer, msg); if (write_in_full(fd, sb.buf, sb.len) < 0) ret = -1; strbuf_release(&sb); @@ -2397,6 +2405,8 @@ static int files_for_each_reflog_ent(struct ref_store *ref_store, while (!ret && !strbuf_getwholeline(&sb, logfp, '\n')) ret = show_one_reflog_ent(refs, refname, &sb, fn, cb_data); + if (ferror(logfp)) + ret = -1; fclose(logfp); strbuf_release(&sb); return ret; @@ -2665,12 +2675,120 @@ static enum ref_transaction_error check_old_oid(struct ref_update *update, return REF_TRANSACTION_ERROR_INCORRECT_OLD_VALUE; } +struct staged_reflog { + struct tempfile *file; + struct ref_update **updates; + size_t nr, alloc; +}; + struct files_transaction_backend_data { struct ref_transaction *packed_transaction; int packed_refs_locked; struct strmap ref_locks; + struct strmap reflog_files; }; +static int reflog_update_cmp(const void *a, const void *b) +{ + const struct ref_update *one = *(struct ref_update * const *)a; + const struct ref_update *two = *(struct ref_update * const *)b; + + return (one->index > two->index) - (one->index < two->index); +} + +static int prepare_reflog_replacements(struct files_ref_store *refs, + struct ref_transaction *transaction, + struct strbuf *err) +{ + struct files_transaction_backend_data *data = transaction->backend_data; + struct strbuf path = STRBUF_INIT; + struct strbuf contents = STRBUF_INIT; + struct hashmap_iter iter; + struct strmap_entry *entry; + size_t i; + int ret = -1; + + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + struct staged_reflog *log; + + if (!(update->flags & REF_REPLACE_REFLOG)) + continue; + CALLOC_ARRAY(log, 1); + strmap_put(&data->reflog_files, update->refname, log); + } + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + struct staged_reflog *log = strmap_get(&data->reflog_files, update->refname); + + if (!log || !(update->flags & REF_LOG_ONLY) || + !(update->flags & REF_HAVE_NEW)) + continue; + if (!(update->flags & REF_LOG_USE_PROVIDED_OIDS)) { + strbuf_addf(err, "replacing reflog '%s' requires explicit OIDs", + update->refname); + goto out; + } + ALLOC_GROW(log->updates, log->nr + 1, log->alloc); + log->updates[log->nr++] = update; + } + strmap_for_each_entry(&data->reflog_files, &iter, entry) { + struct staged_reflog *log = entry->value; + int fd; + + if (!log->nr) + continue; + QSORT(log->updates, log->nr, reflog_update_cmp); + /* Keep the staging file beside the logs, which may be on another device. */ + strbuf_reset(&path); + files_reflog_path(refs, &path, is_root_ref(entry->key) ? + ".tmp-reflog-XXXXXX" : "refs/.tmp-reflog-XXXXXX"); + if (safe_create_leading_directories(refs->base.repo, path.buf)) + goto write_error; + log->file = mks_tempfile_m(path.buf, 0666); + if (!log->file) + goto write_error; + fd = get_tempfile_fd(log->file); + for (i = 0; i < log->nr; i++) { + struct ref_update *update = log->updates[i]; + + format_reflog_entry(&contents, &update->old_oid, &update->new_oid, + update->committer_info, update->msg); + if (contents.len >= 65536) { + if (write_in_full(fd, contents.buf, contents.len) < 0) + goto write_error; + strbuf_reset(&contents); + } + } + if (write_in_full(fd, contents.buf, contents.len) < 0) + goto write_error; + strbuf_reset(&contents); + if (adjust_shared_perm(refs->base.repo, get_tempfile_path(log->file)) || + close_tempfile_gently(log->file)) + goto write_error; + } + ret = 0; + goto out; + +write_error: + strbuf_addf(err, "cannot write staged reflog: %s", strerror(errno)); +out: + strbuf_release(&path); + strbuf_release(&contents); + return ret; +} + +static int install_reflog(const char *path, void *data) +{ + struct staged_reflog *log = data; + int ret = rename(get_tempfile_path(log->file), path); + + /* Some systems report ENOTDIR when the destination is a directory. */ + if (ret && errno == ENOTDIR) + errno = EISDIR; + return ret; +} + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2929,6 +3047,17 @@ static void files_transaction_cleanup(struct files_ref_store *refs, } if (backend_data) { + struct hashmap_iter iter; + struct strmap_entry *entry; + + strmap_for_each_entry(&backend_data->reflog_files, &iter, entry) { + struct staged_reflog *log = entry->value; + + delete_tempfile(&log->file); + free(log->updates); + free(log); + } + strmap_clear(&backend_data->reflog_files, 0); if (backend_data->packed_transaction && ref_transaction_abort(backend_data->packed_transaction, &err)) { error("error aborting transaction: %s", err.buf); @@ -2970,6 +3099,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, CALLOC_ARRAY(backend_data, 1); strmap_init(&backend_data->ref_locks); + strmap_init(&backend_data->reflog_files); transaction->backend_data = backend_data; /* @@ -3102,6 +3232,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, if (ret) { ref_transaction_free(packed_transaction); backend_data->packed_transaction = NULL; + goto cleanup; } } else { /* @@ -3122,6 +3253,8 @@ static int files_transaction_prepare(struct ref_store *ref_store, } } + ret = prepare_reflog_replacements(refs, transaction, err); + cleanup: free(head_ref); string_list_clear(&refnames_to_check, 1); @@ -3346,6 +3479,31 @@ static int files_transaction_finish(struct ref_store *ref_store, backend_data = transaction->backend_data; packed_transaction = backend_data->packed_transaction; + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + struct staged_reflog *log; + + if (!(update->flags & REF_REPLACE_REFLOG)) + continue; + log = strmap_get(&backend_data->reflog_files, update->refname); + strbuf_reset(&sb); + files_reflog_path(refs, &sb, update->refname); + if (!log->file) { + if (unlink(sb.buf) && errno != ENOENT && errno != EISDIR) + ret = -1; + } else if (raceproof_create_file(refs, sb.buf, install_reflog, log)) { + ret = -1; + } else { + delete_tempfile(&log->file); + } + if (ret) { + strbuf_addf(err, "cannot replace reflog '%s': %s", + update->refname, strerror(errno)); + } + if (ret) + goto cleanup; + } + /* Perform updates first so live commits remain referenced */ for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; @@ -3354,8 +3512,9 @@ static int files_transaction_finish(struct ref_store *ref_store, if (update->rejection_err) continue; - if (update->flags & REF_NEEDS_COMMIT || - update->flags & REF_LOG_ONLY) { + if ((update->flags & REF_NEEDS_COMMIT || + update->flags & REF_LOG_ONLY) && + !strmap_contains(&backend_data->reflog_files, update->refname)) { if (parse_and_write_reflog(refs, update, lock, err)) { ret = REF_TRANSACTION_ERROR_GENERIC; goto cleanup; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 0b41f5fa4b..19fdf39d1d 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -49,6 +49,9 @@ struct ref_transaction; */ #define REF_HAVE_PEELED (1 << 15) +/* Replace a reflog with the explicit log-only updates in the transaction. */ +#define REF_REPLACE_REFLOG (1 << 16) + /* * Return the length of time to retry acquiring a loose reference lock * before giving up, in milliseconds: diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 10db03991e..203b111f3d 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1503,9 +1503,11 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data * - `core.logAllRefUpdates` tells us to create the reflog for * the given ref. */ - if ((u->flags & REF_HAVE_NEW) && + if ((u->flags & REF_REPLACE_REFLOG) || + ((u->flags & REF_HAVE_NEW) && + !(u->flags & REF_LOG_ONLY) && !(u->type & REF_ISSYMREF) && - ref_update_has_null_new_value(u)) { + ref_update_has_null_new_value(u))) { struct reftable_log_record log = {0}; struct reftable_iterator it = {0}; @@ -1548,8 +1550,10 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data if (ret) goto done; - } else if (!(u->flags & REF_SKIP_CREATE_REFLOG) && + } + if (!(u->flags & REF_SKIP_CREATE_REFLOG) && (u->flags & REF_HAVE_NEW) && + (!ref_update_has_null_new_value(u) || (u->flags & REF_LOG_ONLY)) && (u->flags & REF_FORCE_CREATE_REFLOG || should_write_log(arg->refs, u->refname))) { struct reftable_log_record *log; diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index db58f00589..7ce10c28af 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -146,6 +146,45 @@ static int cmd_rename_ref(struct ref_store *refs, const char **argv) return refs_rename_ref(refs, oldref, newref, logmsg); } +static int cmd_reflog_transaction(struct ref_store *refs, const char **argv) +{ + const char *refname = notnull(*argv++, "refname"); + const char *mode = notnull(*argv++, "mode"); + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = !transaction; + + if (!ret) { + if (!strcmp(mode, "replace")) { + ret = ref_transaction_replace_reflog(transaction, refname, &err); + } else if (!strcmp(mode, "replace-twice")) { + ret = ref_transaction_replace_reflog(transaction, refname, &err) || + ref_transaction_replace_reflog(transaction, refname, &err); + } else if (strcmp(mode, "append")) { + die("unknown reflog transaction mode: %s", mode); + } + } + while (!ret && *argv) { + uint64_t index = strtoumax(notnull(*argv++, "index"), NULL, 10); + struct object_id old_oid, new_oid; + const char *message; + + if (get_oid_hex(notnull(*argv++, "old-oid"), &old_oid) || + get_oid_hex(notnull(*argv++, "new-oid"), &new_oid)) + die("invalid object ID"); + message = notnull(*argv++, "message"); + ret = ref_transaction_update_reflog(transaction, refname, + &new_oid, &old_oid, NULL, message, index, &err); + } + if (!ret) + ret = ref_transaction_commit(transaction, &err); + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; +} + static int each_ref(const struct reference *ref, void *cb_data UNUSED) { printf("%s %s 0x%x\n", oid_to_hex(ref->oid), ref->name, ref->flags); @@ -308,6 +347,7 @@ static struct command commands[] = { { "create-symref", cmd_create_symref }, { "delete-refs", cmd_delete_refs }, { "rename-ref", cmd_rename_ref }, + { "reflog-transaction", cmd_reflog_transaction }, { "for-each-ref", cmd_for_each_ref }, { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, diff --git a/t/meson.build b/t/meson.build index f65eb04684..fb5266cd0d 100644 --- a/t/meson.build +++ b/t/meson.build @@ -218,6 +218,7 @@ integration_tests = [ 't1421-reflog-write.sh', 't1422-show-ref-exists.sh', 't1423-ref-backend.sh', + 't1425-reflog-transaction.sh', 't1430-bad-ref-name.sh', 't1450-fsck.sh', 't1451-fsck-buffer.sh', diff --git a/t/t1425-reflog-transaction.sh b/t/t1425-reflog-transaction.sh new file mode 100755 index 0000000000..ee5278045e --- /dev/null +++ b/t/t1425-reflog-transaction.sh @@ -0,0 +1,59 @@ +#!/bin/sh + +test_description='explicit reflog entries in reference transactions' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit A && + test_commit B && + A=$(git rev-parse A) && + B=$(git rev-parse B) +' + +test_expect_success 'log-only entry with null new OID does not delete existing history' ' + git branch source "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/source >before && + test-tool ref-store main reflog-transaction refs/heads/source append \ + 1 "$A" "$ZERO_OID" deletion && + test-tool ref-store main for-each-reflog-ent refs/heads/source >after && + sed "$ d" after >history && + test_cmp before history && + test_grep "$A $ZERO_OID .*deletion" after && + test_cmp_rev "$A" source +' + +test_expect_success 'replace reflog with entries ordered by index' ' + test-tool ref-store main reflog-transaction refs/heads/source replace \ + 2 "$A" "$B" second \ + 1 "$ZERO_OID" "$A" first && + git reflog show --format=%gs source >actual && + printf "%s\n" second first >expect && + test_cmp expect actual && + test_cmp_rev "$A" source +' + +test_expect_success 'prepared hook can reject replacement without losing history' ' + test_when_finished "rm -f .git/hooks/reference-transaction" && + test-tool ref-store main for-each-reflog-ent refs/heads/source >before && + write_script .git/hooks/reference-transaction <<-\EOF && + test "$1" != prepared + EOF + test_must_fail test-tool ref-store main reflog-transaction refs/heads/source replace \ + 1 "$ZERO_OID" "$B" replacement && + test-tool ref-store main for-each-reflog-ent refs/heads/source >after && + test_cmp before after +' + +test_expect_success 'replacement with no entries removes only the reflog' ' + test-tool ref-store main reflog-transaction refs/heads/source replace && + test_must_fail git reflog exists refs/heads/source && + test_cmp_rev "$A" source +' + +test_expect_success 'duplicate replacement is rejected' ' + test_must_fail test-tool ref-store main reflog-transaction \ + refs/heads/source replace-twice +' + +test_done -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 3/4] refs: run copy and rename through ordinary transactions 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 2/4] refs: support replacing reflogs in a transaction Maciej Ciemborowicz @ 2026-10-08 9:44 ` Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 4/4] refs: remove backend-specific copy and rename callbacks Maciej Ciemborowicz 2026-10-08 10:10 ` [PATCH v4 0/4] refs: run copy and rename through transactions Patrick Steinhardt 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 9:44 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Copy and rename currently bypass reference transactions. A hook sees the source deletion with files and sees neither endpoint with reftable. Queue the source and destination as ordinary updates instead, with the destination referring to its source update. Read and lock the source during prepare, then replay its history through the explicit reflog API. Keep this state per update so multiple independent operations and ordinary updates can share a transaction. Do not reject source changes made before locking: preparing is advisory, while prepared sees the values read under lock and unchanged on-disk refs and reflogs. For files D/F and case-only renames, the source prevents locking a loose destination. Queue the destination in packed-refs, protected by the source lock (and an ancestor lock when needed). Suppress hooks for this physical child transaction only. Keep source logs until finish and use the staged replacements so a veto needs no rollback writes. During finish, retain a unique backup of a D/F source log until its destination log is installed, and restore it if installation fails. Preserve the backends' distinct final reflog records. Forced reftable copies now replace destination history instead of retaining unrelated destination entries. Add regression tests for hooks, full histories, HEAD, source races, mixed transactions, lock and installation errors, restoration and cleanup, and a reproducible performance test. Replaying history costs O(N) time and memory. In a local no-hook benchmark with 10,000 entries, files rename takes about 17 ms versus 5 ms before, and reftable rename about 54 ms versus 44 ms. Short-history operations remain within roughly 1 ms. This trades the files constant-time rename for staging that does not expose partial changes to a prepared hook. Helped-by: Karthik Nayak <karthik.188@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- Documentation/githooks.adoc | 10 + refs.c | 163 +++++++++++++-- refs.h | 16 ++ refs/files-backend.c | 225 +++++++++++++++++++- refs/refs-internal.h | 13 ++ refs/reftable-backend.c | 37 +++- t/helper/test-ref-store.c | 36 ++++ t/meson.build | 1 + t/perf/p1424-ref-copy-rename.sh | 48 +++++ t/t1424-ref-copy-transaction.sh | 352 ++++++++++++++++++++++++++++++++ 10 files changed, 881 insertions(+), 20 deletions(-) create mode 100755 t/perf/p1424-ref-copy-rename.sh create mode 100755 t/t1424-ref-copy-transaction.sh diff --git a/Documentation/githooks.adoc b/Documentation/githooks.adoc index 145642bf05..245fec3b70 100644 --- a/Documentation/githooks.adoc +++ b/Documentation/githooks.adoc @@ -518,6 +518,16 @@ distinguish these cases, you can inspect the current value of references are not resolved: `<ref-name>` will reflect the symbolic reference itself rather than the object it points to. +Copying or renaming a reference is reported as one transaction. A rename +reports the deletion of the source and the update of the destination; +a copy reports only the destination update. Copying the reflog does not +produce additional hook input. In the "preparing" state, the old values +are unspecified (all zeroes), and the destination's new value is the +source value read before locking. In subsequent states, these operations +report the old values and the source value read under lock. Thus, a +"preparing" hook may change the source before it is copied or renamed. +A "prepared" hook still sees the original references and reflogs on disk. + For symbolic reference updates the `<old_value>` and `<new-value>` fields could denote references instead of objects. A reference will be denoted with a 'ref:' prefix, like `ref:<ref-target>`. diff --git a/refs.c b/refs.c index c02da9d966..df682e0e26 100644 --- a/refs.c +++ b/refs.c @@ -3181,28 +3181,165 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg, return ret; } +static int transaction_copy_ref(struct ref_transaction *transaction, + const char *oldref, const char *newref, + unsigned int source_flags, const char *logmsg, + struct strbuf *err) +{ + struct ref_update *source, *destination; + struct object_id oid; + int type; + + if (transaction->state != REF_TRANSACTION_OPEN) + BUG("copy added to a prepared transaction"); + if (!refs_resolve_ref_unsafe(transaction->ref_store, oldref, + RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, &oid, &type)) { + strbuf_addf(err, _("refname %s not found"), oldref); + return -1; + } + if (type & REF_ISSYMREF) { + strbuf_addf(err, _("refname %s is a symbolic ref, copying or renaming it is not supported"), oldref); + return -1; + } + if (!strcmp(oldref, newref)) + return 0; + if (!transaction_refname_valid(newref, &oid, REF_NO_DEREF, err)) + return -1; + if (transaction->flags & (REF_TRANSACTION_FLAG_INITIAL | REF_TRANSACTION_ALLOW_FAILURE)) { + strbuf_addstr(err, _("copy and rename require an all-or-nothing, non-initial transaction")); + return -1; + } + + /* This value describes the request to 'preparing'; locks decide the value used. */ + source = ref_transaction_add_update(transaction, oldref, + REF_NO_DEREF | REF_COPY_SOURCE | source_flags, + (source_flags & REF_HAVE_NEW) ? null_oid(transaction->ref_store->repo->hash_algo) : NULL, + NULL, NULL, NULL, NULL, NULL, logmsg); + destination = ref_transaction_add_update(transaction, newref, + REF_NO_DEREF | REF_HAVE_NEW, + &oid, NULL, NULL, NULL, NULL, NULL, logmsg); + destination->copy_from = source; + return ref_transaction_replace_reflog(transaction, newref, err); +} + +int ref_transaction_copy(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err) +{ + return transaction_copy_ref(transaction, oldref, newref, + REF_LOG_ONLY | REF_SKIP_CREATE_REFLOG, logmsg, err); +} + +int ref_transaction_rename(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err) +{ + return transaction_copy_ref(transaction, oldref, newref, + REF_HAVE_NEW, logmsg, err); +} + +int ref_update_record_copy_source(struct ref_update *update, + const struct object_id *oid, + struct strbuf *err) +{ + if ((update->type & REF_ISSYMREF) || is_null_oid(oid)) { + strbuf_addf(err, _("'%s' is not an existing direct reference"), update->refname); + return -1; + } + oidcpy(&update->old_oid, oid); + update->flags |= REF_HAVE_OLD; + return 0; +} + +struct copy_reflog_data { + struct ref_transaction *transaction; + const char *refname; + struct strbuf *err; +}; + +static int copy_reflog_entry(const char *refname UNUSED, + struct object_id *old_oid, struct object_id *new_oid, + const char *committer, timestamp_t timestamp, int tz, + const char *message, void *cb_data) +{ + struct copy_reflog_data *data = cb_data; + struct strbuf ident = STRBUF_INIT; + int ret; + + strbuf_addf(&ident, "%s %" PRItime " %+05d", committer, timestamp, tz); + ret = ref_transaction_update_reflog(data->transaction, data->refname, + new_oid, old_oid, ident.buf, message, + data->transaction->max_index + 1, data->err); + strbuf_release(&ident); + return ret; +} + +int ref_transaction_prepare_copy(struct ref_transaction *transaction, + struct ref_update *update, + struct strbuf *err) +{ + struct ref_update *source = update->copy_from; + struct copy_reflog_data data = { transaction, update->refname, err }; + struct object *object; + int ret; + + oidcpy(&update->new_oid, &source->old_oid); + object = parse_object(transaction->ref_store->repo, &update->new_oid); + if (!object || (is_branch(update->refname) && object->type != OBJ_COMMIT)) { + strbuf_addf(err, _("cannot copy object %s to '%s'"), + oid_to_hex(&update->new_oid), update->refname); + return -1; + } + if (object->type == OBJ_TAG && + !peel_object(transaction->ref_store->repo, &update->new_oid, + &update->peeled, PEEL_OBJECT_VERIFY_TAGGED_OBJECT_TYPE)) + update->flags |= REF_HAVE_PEELED; + + if (!refs_reflog_exists(transaction->ref_store, source->refname)) + return 0; + ret = refs_for_each_reflog_ent(transaction->ref_store, source->refname, + copy_reflog_entry, &data); + if (ret && !err->len) + strbuf_addf(err, _("cannot read reflog for '%s'"), source->refname); + return ret; +} + int refs_rename_ref(struct ref_store *refs, const char *oldref, const char *newref, const char *logmsg) { - char *msg; - int retval; + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = -1; - msg = normalize_reflog_message(logmsg); - retval = refs->be->rename_ref(refs, oldref, newref, msg); - free(msg); - return retval; + if (!transaction || + ref_transaction_rename(transaction, oldref, newref, logmsg, &err)) + goto out; + ret = transaction->nr ? ref_transaction_commit(transaction, &err) : 0; +out: + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; } int refs_copy_existing_ref(struct ref_store *refs, const char *oldref, - const char *newref, const char *logmsg) + const char *newref, const char *logmsg) { - char *msg; - int retval; + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = -1; - msg = normalize_reflog_message(logmsg); - retval = refs->be->copy_ref(refs, oldref, newref, msg); - free(msg); - return retval; + if (!transaction || + ref_transaction_copy(transaction, oldref, newref, logmsg, &err)) + goto out; + ret = transaction->nr ? ref_transaction_commit(transaction, &err) : 0; +out: + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; } const char *ref_update_original_update_refname(struct ref_update *update) diff --git a/refs.h b/refs.h index b0b2b0e4fd..eba5025351 100644 --- a/refs.h +++ b/refs.h @@ -952,6 +952,22 @@ int ref_transaction_replace_reflog(struct ref_transaction *transaction, const char *refname, struct strbuf *err); +/* + * Queue a copy or rename of a direct reference and its reflog. The source is + * read again under lock during prepare; changes made by the preparing hook + * are included. An existing destination is overwritten without dereferencing + * it. Several independent copies, renames and ordinary updates may be queued + * together. Each destination (and each source being renamed) must be unique. + * Initial transactions and transactions allowing individual failures are not + * supported. On failure, discard the transaction. + */ +int ref_transaction_copy(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err); +int ref_transaction_rename(struct ref_transaction *transaction, + const char *oldref, const char *newref, + const char *logmsg, struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 36ecd21ed7..ffdc112f9a 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -74,6 +74,12 @@ extern int ignore_case; */ #define REF_LOG_VIA_SPLIT (1 << 14) +/* A D/F or case-only rename cannot lock a loose destination beside its source. */ +#define REF_NEEDS_PACK (1 << 18) + +/* The source reflog was removed to make room for its rename destination. */ +#define REF_RENAMED_LOG (1 << 19) + struct ref_lock { char *ref_name; struct lock_file lk; @@ -788,6 +794,37 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs, lock->count = 1; files_ref_path(refs, &ref_file, refname); + if (update->copy_from && (update->copy_from->flags & REF_HAVE_NEW)) { + const char *source = update->copy_from->refname; + size_t source_len = strlen(source), dest_len = strlen(refname); + int source_is_parent = starts_with(refname, source) && refname[source_len] == '/'; + int dest_is_parent = starts_with(source, refname) && source[dest_len] == '/'; + int same_path = repo_ignore_case(refs->base.repo) && !strcasecmp(source, refname); + + if (source_is_parent || dest_is_parent || same_path) { + struct string_list skip = STRING_LIST_INIT_NODUP; + + string_list_insert(&skip, source); + ret = refs_verify_refname_available(&refs->base, refname, + extras, &skip, 0, err); + string_list_clear(&skip, 0); + if (ret) + goto error_return; + /* The existing source and its lock protect a child or case-only destination. */ + if (dest_is_parent && repo_hold_lock_file_for_update_timeout( + refs->base.repo, &lock->lk, ref_file.buf, LOCK_NO_DEREF, + get_files_ref_lock_timeout_ms(refs->base.repo)) < 0) { + unable_to_lock_message(ref_file.buf, errno, err); + ret = REF_TRANSACTION_ERROR_GENERIC; + goto error_return; + } + oidclr(&lock->old_oid, refs->base.repo->hash_algo); + update->flags |= REF_NEEDS_PACK; + ret = 0; + goto out; + } + } + retry: switch (safe_create_leading_directories(refs->base.repo, ref_file.buf)) { case SCLD_OK: @@ -2538,7 +2575,8 @@ static enum ref_transaction_error split_head_update(struct ref_update *update, new_update = ref_transaction_add_update( transaction, "HEAD", - update->flags | REF_LOG_ONLY | REF_NO_DEREF | REF_LOG_VIA_SPLIT, + (update->flags & ~REF_COPY_SOURCE) | + REF_LOG_ONLY | REF_NO_DEREF | REF_LOG_VIA_SPLIT, &update->new_oid, &update->old_oid, &update->peeled, NULL, NULL, update->committer_info, update->msg); new_update->parent_update = update; @@ -2677,6 +2715,7 @@ static enum ref_transaction_error check_old_oid(struct ref_update *update, struct staged_reflog { struct tempfile *file; + struct tempfile *source_log; struct ref_update **updates; size_t nr, alloc; }; @@ -2789,6 +2828,155 @@ static int install_reflog(const char *path, void *data) return ret; } +static int move_reflog(const char *path, void *data) +{ + struct tempfile *tempfile = data; + int ret = rename(get_tempfile_path(tempfile), path); + + if (ret && errno == ENOTDIR) + errno = EISDIR; + return ret; +} + +static int stage_source_reflog(struct files_ref_store *refs, + struct ref_update *update, + struct staged_reflog *log, + struct strbuf *err) +{ + struct strbuf source = STRBUF_INIT; + struct strbuf path = STRBUF_INIT; + int ret = -1; + + files_reflog_path(refs, &source, update->copy_from->refname); + if (!refs_reflog_exists(&refs->base, update->copy_from->refname)) { + ret = 0; + goto out; + } + files_reflog_path(refs, &path, "refs/.tmp-reflog-source-XXXXXX"); + log->source_log = mks_tempfile_m(path.buf, 0666); + if (!log->source_log || close_tempfile_gently(log->source_log) || + rename(source.buf, get_tempfile_path(log->source_log))) { + strbuf_addf(err, "cannot stage source reflog '%s': %s", + update->copy_from->refname, strerror(errno)); + goto out; + } + try_remove_empty_parents(refs, update->copy_from->refname, + REMOVE_EMPTY_PARENTS_REFLOG); + update->copy_from->flags |= REF_RENAMED_LOG; + ret = 0; + +out: + if (ret) + delete_tempfile(&log->source_log); + strbuf_release(&source); + strbuf_release(&path); + return ret; +} + +static int restore_source_reflog(struct files_ref_store *refs, + struct ref_update *update, + struct staged_reflog *log, + struct strbuf *err) +{ + struct strbuf source = STRBUF_INIT; + int ret; + + if (!log->source_log) + return 0; + files_reflog_path(refs, &source, update->copy_from->refname); + ret = raceproof_create_file(refs, source.buf, move_reflog, + log->source_log); + if (ret) { + struct strbuf recovery = STRBUF_INIT; + + strbuf_addf(err, "; cannot restore source reflog '%s': %s", + update->copy_from->refname, strerror(errno)); + strbuf_addf(&recovery, "%s.recovery", + get_tempfile_path(log->source_log)); + if (rename_tempfile(&log->source_log, recovery.buf)) + strbuf_addf(err, "; cannot preserve its backup: %s", + strerror(errno)); + else + strbuf_addf(err, "; backup saved as '%s'", recovery.buf); + strbuf_release(&recovery); + } else { + delete_tempfile(&log->source_log); + } + strbuf_release(&source); + return ret; +} + +static int check_reflog_symlink(struct files_ref_store *refs, + const char *refname, struct strbuf *err) +{ + struct strbuf path = STRBUF_INIT; + struct stat st; + int ret = 0; + + files_reflog_path(refs, &path, refname); + if (!lstat(path.buf, &st)) { + if (S_ISLNK(st.st_mode)) { + strbuf_addf(err, "reflog for %s is a symlink", refname); + ret = -1; + } + } else if (errno != ENOENT && errno != ENOTDIR) { + strbuf_addf(err, "cannot stat reflog for %s: %s", refname, strerror(errno)); + ret = -1; + } + strbuf_release(&path); + return ret; +} + +static struct ref_update *find_update_with_flag(struct ref_transaction *transaction, + const char *refname, + unsigned int flag) +{ + size_t i; + + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + + if ((update->flags & flag) && !strcmp(update->refname, refname)) + return update; + } + return NULL; +} + +static int prepare_copy_reflog(struct files_ref_store *refs, + struct ref_transaction *transaction, + struct ref_update *update, struct strbuf *err) +{ + enum log_refs_config config = files_ref_store_write_options(refs)->log_all_ref_updates; + int source_has_log; + int write_log; + + if (check_reflog_symlink(refs, update->copy_from->refname, err) || + check_reflog_symlink(refs, update->refname, err)) + return -1; + source_has_log = refs_reflog_exists(&refs->base, update->copy_from->refname); + /* A copy without source history only appends to an existing destination log. */ + if (!source_has_log && !(update->copy_from->flags & REF_HAVE_NEW)) { + struct ref_update *replacement = find_update_with_flag( + transaction, update->refname, REF_REPLACE_REFLOG); + + if (!replacement) + BUG("copy destination without a reflog replacement"); + replacement->flags &= ~REF_REPLACE_REFLOG; + } + if (config == LOG_REFS_UNSET) + config = is_bare_repository(refs->base.repo) ? LOG_REFS_NONE : LOG_REFS_NORMAL; + write_log = source_has_log || should_autocreate_reflog(config, update->refname) || + (!(update->copy_from->flags & REF_HAVE_NEW) && + refs_reflog_exists(&refs->base, update->refname)); + if (ref_transaction_prepare_copy(transaction, update, err)) + return -1; + if (!write_log) + return 0; + return ref_transaction_update_reflog(transaction, update->refname, + &update->new_oid, &update->new_oid, NULL, update->msg, + transaction->max_index + 1, err); +} + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2819,6 +3007,8 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re files_assert_main_repository(refs, "lock_ref_for_update"); backend_data = transaction->backend_data; + if (update->copy_from && prepare_copy_reflog(refs, transaction, update, err)) + return REF_TRANSACTION_ERROR_GENERIC; if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update)) update->flags |= REF_DELETING; @@ -2828,6 +3018,8 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re if (ret) goto out; } + if (update->copy_from) + update->flags |= REF_SKIP_CREATE_REFLOG; lock = strmap_get(&backend_data->ref_locks, update->refname); if (lock) { @@ -2849,6 +3041,17 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re } update->backend_data = lock; + if (update->copy_from) { + oidcpy(&update->old_oid, &lock->old_oid); + if (update->type & REF_ISSYMREF) + update->old_target = xstrdup(referent.buf); + update->flags |= REF_HAVE_OLD; + } + if ((update->flags & REF_COPY_SOURCE) && + ref_update_record_copy_source(update, &lock->old_oid, err)) { + ret = REF_TRANSACTION_ERROR_GENERIC; + goto out; + } if (update->flags & REF_LOG_VIA_SPLIT) { struct ref_lock *parent_lock; @@ -2974,7 +3177,7 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re update->flags |= REF_NEEDS_COMMIT; } else if ((update->flags & REF_HAVE_NEW) && !(update->flags & REF_DELETING) && - !(update->flags & REF_LOG_ONLY)) { + !(update->flags & (REF_LOG_ONLY | REF_NEEDS_PACK))) { if (!(update->type & REF_ISSYMREF) && oideq(&lock->old_oid, &update->new_oid)) { /* @@ -3003,7 +3206,7 @@ static enum ref_transaction_error lock_ref_for_update(struct files_ref_store *re } } } - if (!(update->flags & REF_NEEDS_COMMIT)) { + if (!(update->flags & (REF_NEEDS_COMMIT | REF_NEEDS_PACK))) { /* * We didn't call write_ref_to_lockfile(), so * the lockfile is still open. Close it to @@ -3054,6 +3257,7 @@ static void files_transaction_cleanup(struct files_ref_store *refs, struct staged_reflog *log = entry->value; delete_tempfile(&log->file); + delete_tempfile(&log->source_log); free(log->updates); free(log); } @@ -3162,7 +3366,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, goto cleanup; } - if (update->flags & REF_DELETING && + if (update->flags & (REF_DELETING | REF_NEEDS_PACK) && !(update->flags & REF_LOG_ONLY) && !(update->flags & REF_IS_PRUNING) && !is_root_ref(update->refname)) { @@ -3189,6 +3393,8 @@ static int files_transaction_prepare(struct ref_store *ref_store, REF_HAVE_NEW | REF_NO_DEREF, &update->new_oid, NULL, NULL, NULL, NULL, NULL, NULL); + if (update->copy_from || (update->flags & REF_COPY_SOURCE)) + packed_transaction->flags |= REF_TRANSACTION_FLAG_INTERNAL; } } @@ -3481,11 +3687,18 @@ static int files_transaction_finish(struct ref_store *ref_store, for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; + struct ref_update *operation; struct staged_reflog *log; if (!(update->flags & REF_REPLACE_REFLOG)) continue; log = strmap_get(&backend_data->reflog_files, update->refname); + operation = find_update_with_flag(transaction, update->refname, + REF_NEEDS_PACK); + if (operation && stage_source_reflog(refs, operation, log, err)) { + ret = -1; + goto cleanup; + } strbuf_reset(&sb); files_reflog_path(refs, &sb, update->refname); if (!log->file) { @@ -3499,6 +3712,8 @@ static int files_transaction_finish(struct ref_store *ref_store, if (ret) { strbuf_addf(err, "cannot replace reflog '%s': %s", update->refname, strerror(errno)); + if (operation) + restore_source_reflog(refs, operation, log, err); } if (ret) goto cleanup; @@ -3563,7 +3778,7 @@ static int files_transaction_finish(struct ref_store *ref_store, if (update->flags & REF_DELETING && !(update->flags & REF_LOG_ONLY) && - !(update->flags & REF_IS_PRUNING)) { + !(update->flags & (REF_IS_PRUNING | REF_RENAMED_LOG))) { strbuf_reset(&sb); files_reflog_path(refs, &sb, update->refname); if (!unlink_or_warn(sb.buf)) diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 19fdf39d1d..2746ce5b49 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -52,6 +52,9 @@ struct ref_transaction; /* Replace a reflog with the explicit log-only updates in the transaction. */ #define REF_REPLACE_REFLOG (1 << 16) +/* A source whose value and reflog are consumed by a later update. */ +#define REF_COPY_SOURCE (1 << 17) + /* * Return the length of time to retry acquiring a loose reference lock * before giving up, in milliseconds: @@ -161,6 +164,9 @@ struct ref_update { */ struct ref_update *parent_update; + /* The source is queued first, so backends lock it before the destination. */ + struct ref_update *copy_from; + const char refname[FLEX_ARRAY]; }; @@ -193,6 +199,13 @@ struct ref_update *ref_transaction_add_update( const char *committer_info, const char *msg); +int ref_update_record_copy_source(struct ref_update *update, + const struct object_id *oid, + struct strbuf *err); +int ref_transaction_prepare_copy(struct ref_transaction *transaction, + struct ref_update *update, + struct strbuf *err); + /* * Transaction states. * diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 203b111f3d..72c1596c11 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1076,6 +1076,22 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor struct object_id current_oid = {0}; const char *rewritten_ref; + if (u->copy_from) { + const struct object_id *zero = null_oid(refs->base.repo->hash_algo); + + if (ref_transaction_prepare_copy(transaction, u, err)) + return REF_TRANSACTION_ERROR_GENERIC; + if ((u->copy_from->flags & REF_HAVE_NEW) && + ref_transaction_update_reflog(transaction, u->refname, + zero, &u->new_oid, NULL, u->msg, + transaction->max_index + 1, err)) + return REF_TRANSACTION_ERROR_GENERIC; + if (ref_transaction_update_reflog(transaction, u->refname, + &u->new_oid, zero, NULL, u->msg, + transaction->max_index + 1, err)) + return REF_TRANSACTION_ERROR_GENERIC; + } + /* * There is no need to reload the respective backends here as * we have already reloaded them when preparing the transaction @@ -1126,15 +1142,27 @@ static enum ref_transaction_error prepare_single_update(struct reftable_ref_stor ref_transaction_add_update( transaction, "HEAD", - u->flags | REF_LOG_ONLY | REF_NO_DEREF, + (u->flags & ~REF_COPY_SOURCE) | + REF_LOG_ONLY | REF_NO_DEREF, &u->new_oid, &u->old_oid, &u->peeled, NULL, NULL, NULL, u->msg); } + if (u->copy_from) + u->flags |= REF_SKIP_CREATE_REFLOG; ret = reftable_backend_read_ref(be, rewritten_ref, ¤t_oid, referent, &u->type); if (ret < 0) return REF_TRANSACTION_ERROR_GENERIC; + if (u->copy_from) { + oidcpy(&u->old_oid, ¤t_oid); + if (u->type & REF_ISSYMREF) + u->old_target = xstrdup(referent->buf); + u->flags |= REF_HAVE_OLD; + } + if ((u->flags & REF_COPY_SOURCE) && + ref_update_record_copy_source(u, ¤t_oid, err)) + return REF_TRANSACTION_ERROR_GENERIC; if (ret > 0 && !ref_update_expects_existing_old_ref(u)) { struct string_list_item *item; /* @@ -1319,6 +1347,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare"); struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT; struct string_list refnames_to_check = STRING_LIST_INIT_NODUP; + struct string_list renamed_sources = STRING_LIST_INIT_NODUP; struct reftable_transaction_data *tx_data = NULL; struct reftable_backend *be; struct object_id head_oid; @@ -1338,6 +1367,9 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, * that will be modified during the transaction. */ for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + if (update->copy_from && (update->copy_from->flags & REF_HAVE_NEW)) + string_list_insert(&renamed_sources, update->copy_from->refname); ret = prepare_transaction_update(NULL, refs, tx_data, transaction->updates[i], err); if (ret) @@ -1390,7 +1422,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, } ret = refs_verify_refnames_available(ref_store, &refnames_to_check, - &transaction->refnames, NULL, + &transaction->refnames, &renamed_sources, transaction, transaction->flags & REF_TRANSACTION_FLAG_INITIAL, err); @@ -1411,6 +1443,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, strbuf_release(&referent); strbuf_release(&head_referent); string_list_clear(&refnames_to_check, 1); + string_list_clear(&renamed_sources, 0); return ret; } diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index 7ce10c28af..ec7728334c 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -185,6 +185,41 @@ static int cmd_reflog_transaction(struct ref_store *refs, const char **argv) return ret; } +static int cmd_copy_transaction(struct ref_store *refs, const char **argv) +{ + struct strbuf err = STRBUF_INIT; + struct ref_transaction *transaction = ref_store_transaction_begin(refs, 0, &err); + int ret = !transaction; + + while (!ret && *argv) { + const char *operation = *argv++; + const char *source = notnull(*argv++, "source"); + const char *destination = notnull(*argv++, "destination"); + + if (!strcmp(operation, "rename")) + ret = ref_transaction_rename(transaction, source, destination, "rename", &err); + else if (!strcmp(operation, "copy")) + ret = ref_transaction_copy(transaction, source, destination, "copy", &err); + else if (!strcmp(operation, "update")) { + struct object_id oid; + + if (get_oid_hex(destination, &oid)) + die("invalid object ID: %s", destination); + ret = ref_transaction_update(transaction, source, &oid, + NULL, NULL, NULL, 0, "update", &err); + } else { + die("unknown operation: %s", operation); + } + } + if (!ret) + ret = ref_transaction_commit(transaction, &err); + if (ret) + error("%s", err.buf); + ref_transaction_free(transaction); + strbuf_release(&err); + return ret; +} + static int each_ref(const struct reference *ref, void *cb_data UNUSED) { printf("%s %s 0x%x\n", oid_to_hex(ref->oid), ref->name, ref->flags); @@ -348,6 +383,7 @@ static struct command commands[] = { { "delete-refs", cmd_delete_refs }, { "rename-ref", cmd_rename_ref }, { "reflog-transaction", cmd_reflog_transaction }, + { "copy-transaction", cmd_copy_transaction }, { "for-each-ref", cmd_for_each_ref }, { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, diff --git a/t/meson.build b/t/meson.build index fb5266cd0d..36d47c92c4 100644 --- a/t/meson.build +++ b/t/meson.build @@ -218,6 +218,7 @@ integration_tests = [ 't1421-reflog-write.sh', 't1422-show-ref-exists.sh', 't1423-ref-backend.sh', + 't1424-ref-copy-transaction.sh', 't1425-reflog-transaction.sh', 't1430-bad-ref-name.sh', 't1450-fsck.sh', diff --git a/t/perf/p1424-ref-copy-rename.sh b/t/perf/p1424-ref-copy-rename.sh new file mode 100755 index 0000000000..a901129b00 --- /dev/null +++ b/t/perf/p1424-ref-copy-rename.sh @@ -0,0 +1,48 @@ +#!/bin/sh + +test_description='Copy and rename references with short and long reflogs' + +. ./perf-lib.sh + +test_perf_fresh_repo + +for entries in 10 10000 +do + export entries + test_expect_success "setup $entries reflog entries" ' + git init --ref-format=files "repo-$entries" && + ( + cd "repo-$entries" && + test_commit A && + git branch source && + oid=$(git rev-parse HEAD) && + awk -v oid="$oid" -v count="$entries" '\'' + BEGIN { + for (i = 0; i < count; i++) + printf "%s %s A <a@example.com> %d +0000\tentry %d\n", oid, oid, 1700000000 + i, i + } + '\'' >.git/logs/refs/heads/source && + if test "$GIT_TEST_DEFAULT_REF_FORMAT" = reftable + then + git refs migrate --ref-format=reftable + fi + ) + ' + + test_perf "copy $entries reflog entries (10 operations)" ' + for i in $(test_seq 10) + do + git -C "repo-$entries" branch -C source destination || return 1 + done + ' + + test_perf "rename $entries reflog entries (10 operations)" ' + for i in $(test_seq 5) + do + git -C "repo-$entries" branch -m source renamed && + git -C "repo-$entries" branch -m renamed source || return 1 + done + ' +done + +test_done diff --git a/t/t1424-ref-copy-transaction.sh b/t/t1424-ref-copy-transaction.sh new file mode 100755 index 0000000000..4c7592795f --- /dev/null +++ b/t/t1424-ref-copy-transaction.sh @@ -0,0 +1,352 @@ +#!/bin/sh + +test_description='reference transactions for copy and rename' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +. ./test-lib.sh + +snapshot () { + git for-each-ref --format="%(refname) %(objectname) %(symref)" >"$1.refs" && + for ref in HEAD refs/heads/source refs/heads/destination + do + if git reflog exists "$ref" + then + test-tool ref-store main for-each-reflog-ent "$ref" + else + echo missing + fi >"$1.$(basename "$ref").log" || return 1 + done +} + +compare_snapshot () { + for suffix in refs HEAD.log source.log destination.log + do + test_cmp "$1.$suffix" "$2.$suffix" || return 1 + done +} + +test_expect_success 'setup' ' + test_commit A && + test_commit B && + A=$(git rev-parse A) && + B=$(git rev-parse B) +' + +for operation in rename copy +do + test_expect_success "$operation reports one logical transaction" ' + test_when_finished "git config --unset core.hooksPath" && + git branch -f source "$A" && + git branch -f destination "$B" && + mkdir -p hooks && + write_script hooks/reference-transaction <<-\EOF && + echo "$1" >>actual && + cat >>actual + EOF + git config core.hooksPath hooks && + >actual && + test-tool ref-store main copy-transaction "$operation" refs/heads/source refs/heads/destination && + { + echo preparing && + if test "$operation" = rename + then + echo "$ZERO_OID $ZERO_OID refs/heads/source" + fi && + echo "$ZERO_OID $A refs/heads/destination" && + for state in prepared committed + do + echo "$state" && + if test "$operation" = rename + then + echo "$A $ZERO_OID refs/heads/source" + fi && + echo "$B $A refs/heads/destination" || return 1 + done + } >expect && + test_cmp expect actual + ' + + for state in preparing prepared + do + test_expect_success "$operation rejected at $state leaves refs and full reflogs unchanged" ' + test_when_finished "git config --unset core.hooksPath" && + git branch -f source "$A" && + git branch -f destination "$B" && + git symbolic-ref HEAD refs/heads/source && + test_when_finished "git -c core.hooksPath=/dev/null symbolic-ref HEAD refs/heads/main" && + snapshot before && + write_script hooks/reference-transaction <<-EOF && + test "\$1" != "$state" + EOF + git config core.hooksPath hooks && + test_must_fail test-tool ref-store main copy-transaction "$operation" refs/heads/source refs/heads/destination && + snapshot after && + compare_snapshot before after + ' + done +done + +test_expect_success 'prepared hook sees old refs and cannot modify the source' ' + test_when_finished "git config --unset core.hooksPath" && + git branch -f source "$A" && + git branch -f destination "$B" && + write_script hooks/reference-transaction <<-EOF && + test "\$1" = prepared || exit 0 + git rev-parse refs/heads/source >source-seen && + git rev-parse refs/heads/destination >destination-seen && + if git -c core.hooksPath=/dev/null -c core.filesRefLockTimeout=0 \ + update-ref refs/heads/source $B + then + exit 1 + fi + EOF + git config core.hooksPath hooks && + git branch -M source destination && + echo "$A" >expect && + test_cmp expect source-seen && + echo "$B" >expect && + test_cmp expect destination-seen && + test_cmp_rev "$A" destination +' + +test_expect_success 'preparing may update source and its reflog' ' + test_when_finished "git config --unset core.hooksPath" && + git branch source "$A" && + write_script hooks/reference-transaction <<-EOF && + test "\$1" = preparing || exit 0 + git -c core.hooksPath=/dev/null update-ref -m concurrent refs/heads/source $B + EOF + git config core.hooksPath hooks && + git branch -M source destination && + test_cmp_rev "$B" destination && + git reflog show --format=%gs destination >actual && + test_grep concurrent actual +' + +test_expect_success 'rename without a source reflog can be rejected without losing destination history' ' + test_when_finished "git config --unset core.hooksPath" && + git -c core.logAllRefUpdates=false branch source "$A" && + test_must_fail git reflog exists refs/heads/source && + test-tool ref-store main for-each-reflog-ent refs/heads/destination >before && + write_script hooks/reference-transaction <<-\EOF && + test "$1" != prepared + EOF + git config core.hooksPath hooks && + test_must_fail git branch -M source destination && + test-tool ref-store main for-each-reflog-ent refs/heads/destination >after && + test_cmp before after +' + +for names in "parent parent/child" "child/branch child" +do + set -- $names + from=$1 to=$2 + test_expect_success "D/F rename $from to $to can be aborted and committed" ' + test_when_finished "git config --unset core.hooksPath" && + git branch "$from" "$A" && + test-tool ref-store main for-each-reflog-ent "refs/heads/$from" >before && + write_script hooks/reference-transaction <<-EOF && + test "\$1" = prepared || exit 0 + git rev-parse refs/heads/$from >seen + exit 1 + EOF + git config core.hooksPath hooks && + test_must_fail git branch -m "$from" "$to" && + echo "$A" >expect && + test_cmp expect seen && + test_cmp_rev "$A" "$from" && + test-tool ref-store main for-each-reflog-ent "refs/heads/$from" >after && + test_cmp before after && + git -c core.hooksPath=/dev/null branch -m "$from" "$to" && + test_cmp_rev "$A" "$to" + ' +done + +test_expect_success REFFILES,POSIXPERM 'D/F reflog installation failure restores source history' ' + test_when_finished "chmod u+w .git/logs/refs/heads" && + test_when_finished "git config --unset core.hooksPath" && + git branch rollback/branch "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/rollback/branch >before && + write_script hooks/reference-transaction <<-\EOF && + test "$1" = prepared || exit 0 + chmod a-w .git/logs/refs/heads + EOF + git config core.hooksPath hooks && + test_must_fail git branch -m rollback/branch rollback && + chmod u+w .git/logs/refs/heads && + test_cmp_rev "$A" rollback/branch && + test_must_fail git show-ref --verify refs/heads/rollback && + test-tool ref-store main for-each-reflog-ent refs/heads/rollback/branch >after && + test_cmp before after +' + +test_expect_success 'multiple renames, a copy and an ordinary update compose' ' + git branch first "$A" && + git branch second "$B" && + git branch third "$A" && + test-tool ref-store main copy-transaction \ + rename refs/heads/first refs/heads/first-new \ + rename refs/heads/second refs/heads/second-new \ + copy refs/heads/third refs/heads/third-new \ + update refs/heads/fourth "$B" && + test_must_fail git show-ref --verify refs/heads/first && + test_must_fail git show-ref --verify refs/heads/second && + test_cmp_rev "$A" first-new && + test_cmp_rev "$B" second-new && + test_cmp_rev "$A" third-new && + test_cmp_rev "$A" third && + test_cmp_rev "$B" fourth +' + +test_expect_success 'copy preserves complete source history and replaces destination history' ' + git branch history-source "$A" && + git update-ref -m history-step refs/heads/history-source "$B" && + git branch history-destination "$B" && + test-tool ref-store main for-each-reflog-ent refs/heads/history-source >before && + git branch -C history-source history-destination && + test-tool ref-store main for-each-reflog-ent refs/heads/history-source >after && + test_cmp before after && + test-tool ref-store main for-each-reflog-ent refs/heads/history-destination >copied && + sed "$ d" copied >history && + test_cmp before history && + test_grep "Branch: copied refs/heads/history-source to refs/heads/history-destination" copied +' + +test_expect_success 'packed rename reports no internal transactions' ' + test_when_finished "git config --unset core.hooksPath" && + git branch packed-source "$A" && + git pack-refs --all && + write_script hooks/reference-transaction <<-\EOF && + echo "$1" >>states + cat >/dev/null + EOF + git config core.hooksPath hooks && + git branch -m packed-source packed-destination && + printf "%s\n" preparing prepared committed >expect && + test_cmp expect states +' + +test_expect_success REFFILES 'unrelated forced copies use independent reflog staging files' ' + test_when_finished "git config --unset core.hooksPath" && + git branch nested-a "$A" && + git branch nested-b "$B" && + git branch outer-a "$A" && + git branch outer-b "$B" && + write_script hooks/reference-transaction <<-\EOF && + test "$1" = prepared || exit 0 + git -c core.hooksPath=/dev/null branch -C nested-a nested-b + EOF + git config core.hooksPath hooks && + git branch -C outer-a outer-b && + test_cmp_rev "$A" outer-b && + test_cmp_rev "$A" nested-b +' + +test_expect_success REFFILES 'rename with reflogs disabled does not create a reflog' ' + git -c core.logAllRefUpdates=false branch no-log "$A" && + git -c core.logAllRefUpdates=false branch -m no-log still-no-log && + test_must_fail git reflog exists refs/heads/still-no-log +' + +test_expect_success REFFILES 'copy without source history retains existing destination history' ' + git -c core.logAllRefUpdates=false branch no-copy-log "$A" && + git branch existing-copy-log "$B" && + test-tool ref-store main for-each-reflog-ent refs/heads/existing-copy-log >before && + git branch -C no-copy-log existing-copy-log && + test-tool ref-store main for-each-reflog-ent refs/heads/existing-copy-log >after && + sed "$ d" after >history && + test_cmp before history +' + +test_expect_success REFFILES,CASE_INSENSITIVE_FS 'case-only rename retains history and supports abort' ' + test_when_finished "git config --unset core.hooksPath" && + git branch case-source "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/case-source >before && + write_script hooks/reference-transaction <<-\EOF && + test "$1" != prepared + EOF + git config core.hooksPath hooks && + test_must_fail git branch -M case-source CASE-SOURCE && + test-tool ref-store main for-each-reflog-ent refs/heads/case-source >after && + test_cmp before after && + git -c core.hooksPath=/dev/null branch -M case-source CASE-SOURCE && + git for-each-ref --format="%(refname)" refs/heads/CASE-SOURCE >actual && + echo refs/heads/CASE-SOURCE >expect && + test_cmp expect actual +' + +test_expect_success REFFILES,SYMLINKS 'symlink source reflog is rejected without changing refs' ' + git branch symlink-source "$A" && + mv .git/logs/refs/heads/symlink-source saved-log && + ln -s "$PWD/saved-log" .git/logs/refs/heads/symlink-source && + test_must_fail git branch -m symlink-source symlink-destination && + test_cmp_rev "$A" symlink-source && + test_must_fail git show-ref --verify refs/heads/symlink-destination +' + +for operation in copy rename +do + test_expect_success "$operation over HEAD referent preserves HEAD history" ' + git branch -f head-source "$A" && + git update-ref refs/heads/main "$B" && + test-tool ref-store main for-each-reflog-ent HEAD >before && + test-tool ref-store main copy-transaction "$operation" refs/heads/head-source refs/heads/main && + test-tool ref-store main for-each-reflog-ent HEAD >after && + sed "$ d" after >history && + test_cmp before history && + test_cmp_rev "$A" HEAD + ' +done + +test_expect_success 'preparing may delete source but cannot cause a partial rename' ' + test_when_finished "git config --unset core.hooksPath" && + git branch race-source "$A" && + write_script hooks/reference-transaction <<-\EOF && + test "$1" = preparing || exit 0 + git -c core.hooksPath=/dev/null update-ref -d refs/heads/race-source + EOF + git config core.hooksPath hooks && + test_must_fail git branch -m race-source race-destination && + test_must_fail git show-ref --verify refs/heads/race-source && + test_must_fail git show-ref --verify refs/heads/race-destination +' + +test_expect_success 'copy cannot replace a D/F-conflicting source' ' + git branch conflict-source "$A" && + test_must_fail git branch -c conflict-source conflict-source/child && + test_cmp_rev "$A" conflict-source && + test_must_fail git show-ref --verify refs/heads/conflict-source/child +' + +test_expect_success REFFILES,POSIXPERM 'unreadable source reflog leaves refs and history unchanged' ' + git branch unreadable "$A" && + test-tool ref-store main for-each-reflog-ent refs/heads/unreadable >before && + test_when_finished "chmod u+r .git/logs/refs/heads/unreadable" && + chmod a-r .git/logs/refs/heads/unreadable && + test_must_fail git branch -m unreadable readable && + chmod u+r .git/logs/refs/heads/unreadable && + test_cmp_rev "$A" unreadable && + test_must_fail git show-ref --verify refs/heads/readable && + test-tool ref-store main for-each-reflog-ent refs/heads/unreadable >after && + test_cmp before after +' + +test_expect_success REFFILES 'destination lock failure preserves source and destination history' ' + git branch -f source "$A" && + git branch -f destination "$B" && + snapshot before && + test_when_finished "rm -f .git/refs/heads/destination.lock" && + >.git/refs/heads/destination.lock && + test_must_fail git -c core.filesRefLockTimeout=0 branch -M source destination && + snapshot after && + compare_snapshot before after +' + +test_expect_success REFFILES 'staged reflog files are cleaned up after success and abort' ' + find .git/logs -name ".tmp-reflog-*" >actual && + test_must_be_empty actual +' + +test_done -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 4/4] refs: remove backend-specific copy and rename callbacks 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz ` (2 preceding siblings ...) 2026-10-08 9:44 ` [PATCH v4 3/4] refs: run copy and rename through ordinary transactions Maciej Ciemborowicz @ 2026-10-08 9:44 ` Maciej Ciemborowicz 2026-10-08 10:10 ` [PATCH v4 0/4] refs: run copy and rename through transactions Patrick Steinhardt 4 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 9:44 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Junio C Hamano, Karthik Nayak Copy and rename now use the transaction API for refs and reflogs. Remove the unused callbacks, backend implementations, and their private helpers. The files backend no longer needs a shared temporary reflog name or rollback writes that reconstruct previously visible refs and logs. Signed-off-by: Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> --- refs/debug.c | 24 --- refs/files-backend.c | 334 ---------------------------------------- refs/packed-backend.c | 2 - refs/refs-internal.h | 9 -- refs/reftable-backend.c | 287 ---------------------------------- 5 files changed, 656 deletions(-) diff --git a/refs/debug.c b/refs/debug.c index 639db0f26e..b4991f71ae 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -143,28 +143,6 @@ static int debug_optimize_required(struct ref_store *ref_store, return res; } -static int debug_rename_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = drefs->refs->be->rename_ref(drefs->refs, oldref, newref, - logmsg); - trace_printf_key(&trace_refs, "rename_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - -static int debug_copy_ref(struct ref_store *ref_store, const char *oldref, - const char *newref, const char *logmsg) -{ - struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; - int res = - drefs->refs->be->copy_ref(drefs->refs, oldref, newref, logmsg); - trace_printf_key(&trace_refs, "copy_ref: %s -> %s \"%s\": %d\n", oldref, newref, - logmsg, res); - return res; -} - struct debug_ref_iterator { struct ref_iterator base; struct ref_iterator *iter; @@ -453,8 +431,6 @@ struct ref_storage_be refs_be_debug = { .optimize = debug_optimize, .optimize_required = debug_optimize_required, - .rename_ref = debug_rename_ref, - .copy_ref = debug_copy_ref, .iterator_begin = debug_ref_iterator_begin, .read_raw_ref = debug_read_raw_ref, diff --git a/refs/files-backend.c b/refs/files-backend.c index ffdc112f9a..589e501050 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1623,273 +1623,6 @@ static int files_optimize_required(struct ref_store *ref_store, return 0; } -/* - * People using contrib's git-new-workdir have .git/logs/refs -> - * /some/other/path/.git/logs/refs, and that may live on another device. - * - * IOW, to avoid cross device rename errors, the temporary renamed log must - * live into logs/refs. - */ -#define TMP_RENAMED_LOG "refs/.tmp-renamed-log" - -struct rename_cb { - const char *tmp_renamed_log; - int true_errno; -}; - -static int rename_tmp_log_callback(const char *path, void *cb_data) -{ - struct rename_cb *cb = cb_data; - - if (rename(cb->tmp_renamed_log, path)) { - /* - * rename(a, b) when b is an existing directory ought - * to result in ISDIR, but Solaris 5.8 gives ENOTDIR. - * Sheesh. Record the true errno for error reporting, - * but report EISDIR to raceproof_create_file() so - * that it knows to retry. - */ - cb->true_errno = errno; - if (errno == ENOTDIR) - errno = EISDIR; - return -1; - } else { - return 0; - } -} - -static int rename_tmp_log(struct files_ref_store *refs, const char *newrefname) -{ - struct strbuf path = STRBUF_INIT; - struct strbuf tmp = STRBUF_INIT; - struct rename_cb cb; - int ret; - - files_reflog_path(refs, &path, newrefname); - files_reflog_path(refs, &tmp, TMP_RENAMED_LOG); - cb.tmp_renamed_log = tmp.buf; - ret = raceproof_create_file(refs, path.buf, rename_tmp_log_callback, &cb); - if (ret) { - if (errno == EISDIR) - error("directory not empty: %s", path.buf); - else - error("unable to move logfile %s to %s: %s", - tmp.buf, path.buf, - strerror(cb.true_errno)); - } - - strbuf_release(&path); - strbuf_release(&tmp); - return ret; -} - -static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store *refs, - struct ref_lock *lock, - const struct object_id *oid, - struct strbuf *err); -static int commit_ref_update(struct files_ref_store *refs, - struct ref_lock *lock, - const struct object_id *oid, const char *logmsg, - int flags, - struct strbuf *err); - -/* - * Emit a better error message than lockfile.c's - * unable_to_lock_message() would in case there is a D/F conflict with - * another existing reference. If there would be a conflict, emit an error - * message and return false; otherwise, return true. - * - * Note that this function is not safe against all races with other - * processes, and that's not its job. We'll emit a more verbose error on D/f - * conflicts if we get past it into lock_ref_oid_basic(). - */ -static int refs_rename_ref_available(struct ref_store *refs, - const char *old_refname, - const char *new_refname) -{ - struct string_list skip = STRING_LIST_INIT_NODUP; - struct strbuf err = STRBUF_INIT; - int ok; - - string_list_insert(&skip, old_refname); - ok = !refs_verify_refname_available(refs, new_refname, - NULL, &skip, 0, &err); - if (!ok) - error("%s", err.buf); - - string_list_clear(&skip, 0); - strbuf_release(&err); - return ok; -} - -static int files_copy_or_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg, int copy) -{ - struct files_ref_store *refs = - files_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); - struct object_id orig_oid; - int flag = 0, logmoved = 0; - struct ref_lock *lock; - struct stat loginfo; - struct strbuf sb_oldref = STRBUF_INIT; - struct strbuf sb_newref = STRBUF_INIT; - struct strbuf tmp_renamed_log = STRBUF_INIT; - int log, ret; - struct strbuf err = STRBUF_INIT; - - files_reflog_path(refs, &sb_oldref, oldrefname); - files_reflog_path(refs, &sb_newref, newrefname); - files_reflog_path(refs, &tmp_renamed_log, TMP_RENAMED_LOG); - - log = !lstat(sb_oldref.buf, &loginfo); - if (log && S_ISLNK(loginfo.st_mode)) { - ret = error("reflog for %s is a symlink", oldrefname); - goto out; - } - - if (!refs_resolve_ref_unsafe(&refs->base, oldrefname, - RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, - &orig_oid, &flag)) { - ret = error("refname %s not found", oldrefname); - goto out; - } - - if (flag & REF_ISSYMREF) { - if (copy) - ret = error("refname %s is a symbolic ref, copying it is not supported", - oldrefname); - else - ret = error("refname %s is a symbolic ref, renaming it is not supported", - oldrefname); - goto out; - } - if (!refs_rename_ref_available(&refs->base, oldrefname, newrefname)) { - ret = 1; - goto out; - } - - if (!copy && log && rename(sb_oldref.buf, tmp_renamed_log.buf)) { - ret = error("unable to move logfile logs/%s to logs/"TMP_RENAMED_LOG": %s", - oldrefname, strerror(errno)); - goto out; - } - - if (copy && log && copy_file(refs->base.repo, tmp_renamed_log.buf, sb_oldref.buf, 0644)) { - ret = error("unable to copy logfile logs/%s to logs/"TMP_RENAMED_LOG": %s", - oldrefname, strerror(errno)); - goto out; - } - - if (!copy && refs_delete_ref(&refs->base, logmsg, oldrefname, - &orig_oid, REF_NO_DEREF)) { - error("unable to delete old %s", oldrefname); - goto rollback; - } - - /* - * Since we are doing a shallow lookup, oid is not the - * correct value to pass to delete_ref as old_oid. But that - * doesn't matter, because an old_oid check wouldn't add to - * the safety anyway; we want to delete the reference whatever - * its current value. - */ - if (!copy && refs_resolve_ref_unsafe(&refs->base, newrefname, - RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE, - NULL, NULL) && - refs_delete_ref(&refs->base, NULL, newrefname, - NULL, REF_NO_DEREF)) { - if (errno == EISDIR) { - struct strbuf path = STRBUF_INIT; - int result; - - files_ref_path(refs, &path, newrefname); - result = remove_empty_directories(&path); - strbuf_release(&path); - - if (result) { - error("Directory not empty: %s", newrefname); - goto rollback; - } - } else { - error("unable to delete existing %s", newrefname); - goto rollback; - } - } - - if (log && rename_tmp_log(refs, newrefname)) - goto rollback; - - logmoved = log; - - lock = lock_ref_oid_basic(refs, newrefname, &err); - if (!lock) { - if (copy) - error("unable to copy '%s' to '%s': %s", oldrefname, newrefname, err.buf); - else - error("unable to rename '%s' to '%s': %s", oldrefname, newrefname, err.buf); - strbuf_release(&err); - goto rollback; - } - oidcpy(&lock->old_oid, &orig_oid); - - if (write_ref_to_lockfile(refs, lock, &orig_oid, &err) || - commit_ref_update(refs, lock, &orig_oid, logmsg, 0, &err)) { - error("unable to write current sha1 into %s: %s", newrefname, err.buf); - strbuf_release(&err); - goto rollback; - } - - ret = 0; - goto out; - - rollback: - lock = lock_ref_oid_basic(refs, oldrefname, &err); - if (!lock) { - error("unable to lock %s for rollback: %s", oldrefname, err.buf); - strbuf_release(&err); - goto rollbacklog; - } - - if (write_ref_to_lockfile(refs, lock, &orig_oid, &err) || - commit_ref_update(refs, lock, &orig_oid, NULL, REF_SKIP_CREATE_REFLOG, &err)) { - error("unable to write current sha1 into %s: %s", oldrefname, err.buf); - strbuf_release(&err); - } - - rollbacklog: - if (logmoved && rename(sb_newref.buf, sb_oldref.buf)) - error("unable to restore logfile %s from %s: %s", - oldrefname, newrefname, strerror(errno)); - if (!logmoved && log && - rename(tmp_renamed_log.buf, sb_oldref.buf)) - error("unable to restore logfile %s from logs/"TMP_RENAMED_LOG": %s", - oldrefname, strerror(errno)); - ret = 1; - out: - strbuf_release(&sb_newref); - strbuf_release(&sb_oldref); - strbuf_release(&tmp_renamed_log); - - return ret; -} - -static int files_rename_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 0); -} - -static int files_copy_ref(struct ref_store *ref_store, - const char *oldrefname, const char *newrefname, - const char *logmsg) -{ - return files_copy_or_rename_ref(ref_store, oldrefname, - newrefname, logmsg, 1); -} - static int close_ref_gently(struct ref_lock *lock) { if (close_lock_file_gently(&lock->lk)) @@ -2121,71 +1854,6 @@ static enum ref_transaction_error write_ref_to_lockfile(struct files_ref_store * return 0; } -/* - * Commit a change to a loose reference that has already been written - * to the loose reference lockfile. Also update the reflogs if - * necessary, using the specified lockmsg (which can be NULL). - */ -static int commit_ref_update(struct files_ref_store *refs, - struct ref_lock *lock, - const struct object_id *oid, const char *logmsg, - int flags, - struct strbuf *err) -{ - files_assert_main_repository(refs, "commit_ref_update"); - - clear_loose_ref_cache(refs); - if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, oid, NULL, - logmsg, flags, err)) { - char *old_msg = strbuf_detach(err, NULL); - strbuf_addf(err, "cannot update the ref '%s': %s", - lock->ref_name, old_msg); - free(old_msg); - unlock_ref(lock); - return -1; - } - - if (strcmp(lock->ref_name, "HEAD") != 0) { - /* - * Special hack: If a branch is updated directly and HEAD - * points to it (may happen on the remote side of a push - * for example) then logically the HEAD reflog should be - * updated too. - * A generic solution implies reverse symref information, - * but finding all symrefs pointing to the given branch - * would be rather costly for this rare event (the direct - * update of a branch) to be worth it. So let's cheat and - * check with HEAD only which should cover 99% of all usage - * scenarios (even 100% of the default ones). - */ - int head_flag; - const char *head_ref; - - head_ref = refs_resolve_ref_unsafe(&refs->base, "HEAD", - RESOLVE_REF_READING, - NULL, &head_flag); - if (head_ref && (head_flag & REF_ISSYMREF) && - !strcmp(head_ref, lock->ref_name)) { - struct strbuf log_err = STRBUF_INIT; - if (files_log_ref_write(refs, "HEAD", &lock->old_oid, - oid, NULL, logmsg, flags, - &log_err)) { - error("%s", log_err.buf); - strbuf_release(&log_err); - } - } - } - - if (commit_ref(lock)) { - strbuf_addf(err, "couldn't set '%s'", lock->ref_name); - unlock_ref(lock); - return -1; - } - - unlock_ref(lock); - return 0; -} - #if defined(NO_SYMLINK_HEAD) || defined(WITH_BREAKING_CHANGES) #define create_ref_symlink(a, b) (-1) #else @@ -4471,8 +4139,6 @@ struct ref_storage_be refs_be_files = { .optimize = files_optimize, .optimize_required = files_optimize_required, - .rename_ref = files_rename_ref, - .copy_ref = files_copy_ref, .iterator_begin = files_ref_iterator_begin, .read_raw_ref = files_read_raw_ref, diff --git a/refs/packed-backend.c b/refs/packed-backend.c index a73fc6aca7..364a912912 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -2164,8 +2164,6 @@ struct ref_storage_be refs_be_packed = { .optimize = packed_optimize, .optimize_required = packed_optimize_required, - .rename_ref = NULL, - .copy_ref = NULL, .iterator_begin = packed_ref_iterator_begin, .read_raw_ref = packed_read_raw_ref, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 2746ce5b49..c415ba0a4e 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -470,13 +470,6 @@ typedef int optimize_required_fn(struct ref_store *ref_store, struct refs_optimize_opts *opts, bool *required); -typedef int rename_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); -typedef int copy_ref_fn(struct ref_store *ref_store, - const char *oldref, const char *newref, - const char *logmsg); - /* * Iterate over the references in `ref_store` whose names start with * `prefix`. `prefix` is matched as a literal string, without regard @@ -596,8 +589,6 @@ struct ref_storage_be { optimize_fn *optimize; optimize_required_fn *optimize_required; - rename_ref_fn *rename_ref; - copy_ref_fn *copy_ref; ref_iterator_begin_fn *iterator_begin; read_raw_ref_fn *read_raw_ref; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 72c1596c11..ef3bab69ee 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1798,290 +1798,6 @@ struct write_create_symref_arg { const char *logmsg; }; -struct write_copy_arg { - struct reftable_ref_store *refs; - struct reftable_backend *be; - const char *oldname; - const char *newname; - const char *logmsg; - int delete_old; -}; - -static int write_copy_table(struct reftable_writer *writer, void *cb_data) -{ - struct write_copy_arg *arg = cb_data; - uint64_t deletion_ts, creation_ts; - struct reftable_ref_record old_ref = {0}, refs[2] = {0}; - struct reftable_log_record old_log = {0}, *logs = NULL; - struct reftable_iterator it = {0}; - struct string_list skip = STRING_LIST_INIT_NODUP; - struct ident_split committer_ident = {0}; - struct strbuf errbuf = STRBUF_INIT; - size_t logs_nr = 0, logs_alloc = 0, i; - const char *committer_info; - int ret; - - committer_info = git_committer_info(0); - if (split_ident_line(&committer_ident, committer_info, strlen(committer_info))) - BUG("failed splitting committer info"); - - if (reftable_stack_read_ref(arg->be->stack, arg->oldname, &old_ref)) { - ret = error(_("refname %s not found"), arg->oldname); - goto done; - } - if (old_ref.value_type == REFTABLE_REF_SYMREF) { - ret = error(_("refname %s is a symbolic ref, copying it is not supported"), - arg->oldname); - goto done; - } - - /* - * There's nothing to do in case the old and new name are the same, so - * we exit early in that case. - */ - if (!strcmp(arg->oldname, arg->newname)) { - ret = 0; - goto done; - } - - /* - * Verify that the new refname is available. - */ - if (arg->delete_old) - string_list_insert(&skip, arg->oldname); - ret = refs_verify_refname_available(&arg->refs->base, arg->newname, - NULL, &skip, 0, &errbuf); - if (ret < 0) { - error("%s", errbuf.buf); - goto done; - } - - /* - * When deleting the old reference we have to use two update indices: - * once to delete the old ref and its reflog, and once to create the - * new ref and its reflog. They need to be staged with two separate - * indices because the new reflog needs to encode both the deletion of - * the old branch and the creation of the new branch, and we cannot do - * two changes to a reflog in a single update. - */ - deletion_ts = creation_ts = reftable_stack_next_update_index(arg->be->stack); - if (arg->delete_old) - creation_ts++; - ret = reftable_writer_set_limits(writer, deletion_ts, creation_ts); - if (ret < 0) - goto done; - - /* - * Add the new reference. If this is a rename then we also delete the - * old reference. - */ - refs[0] = old_ref; - refs[0].refname = xstrdup(arg->newname); - refs[0].update_index = creation_ts; - if (arg->delete_old) { - refs[1].refname = xstrdup(arg->oldname); - refs[1].value_type = REFTABLE_REF_DELETION; - refs[1].update_index = deletion_ts; - } - ret = reftable_writer_add_refs(writer, refs, arg->delete_old ? 2 : 1); - if (ret < 0) - goto done; - - /* - * When deleting the old branch we need to create a reflog entry on the - * new branch name that indicates that the old branch has been deleted - * and then recreated. This is a tad weird, but matches what the files - * backend does. - */ - if (arg->delete_old) { - struct strbuf head_referent = STRBUF_INIT; - struct object_id head_oid; - int append_head_reflog; - unsigned head_type = 0; - - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - memset(&logs[logs_nr], 0, sizeof(logs[logs_nr])); - fill_reftable_log_record(&logs[logs_nr], &committer_ident); - logs[logs_nr].refname = xstrdup(arg->newname); - logs[logs_nr].update_index = deletion_ts; - logs[logs_nr].value.update.message = - xstrndup(arg->logmsg, reftable_be_write_options(arg->refs)->opts.block_size / 2); - memcpy(logs[logs_nr].value.update.old_hash, old_ref.value.val1, GIT_MAX_RAWSZ); - logs_nr++; - - ret = reftable_backend_read_ref(arg->be, "HEAD", &head_oid, - &head_referent, &head_type); - if (ret < 0) - goto done; - append_head_reflog = (head_type & REF_ISSYMREF) && !strcmp(head_referent.buf, arg->oldname); - strbuf_release(&head_referent); - - /* - * The files backend uses `refs_delete_ref()` to delete the old - * branch name, which will append a reflog entry for HEAD in - * case it points to the old branch. - */ - if (append_head_reflog) { - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - logs[logs_nr] = logs[logs_nr - 1]; - logs[logs_nr].refname = xstrdup("HEAD"); - logs[logs_nr].value.update.name = - xstrdup(logs[logs_nr].value.update.name); - logs[logs_nr].value.update.email = - xstrdup(logs[logs_nr].value.update.email); - logs[logs_nr].value.update.message = - xstrdup(logs[logs_nr].value.update.message); - logs_nr++; - } - } - - /* - * Create the reflog entry for the newly created branch. - */ - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - memset(&logs[logs_nr], 0, sizeof(logs[logs_nr])); - fill_reftable_log_record(&logs[logs_nr], &committer_ident); - logs[logs_nr].refname = xstrdup(arg->newname); - logs[logs_nr].update_index = creation_ts; - logs[logs_nr].value.update.message = - xstrndup(arg->logmsg, reftable_be_write_options(arg->refs)->opts.block_size / 2); - memcpy(logs[logs_nr].value.update.new_hash, old_ref.value.val1, GIT_MAX_RAWSZ); - logs_nr++; - - /* - * In addition to writing the reflog entry for the new branch, we also - * copy over all log entries from the old reflog. Last but not least, - * when renaming we also have to delete all the old reflog entries. - */ - ret = reftable_stack_init_log_iterator(arg->be->stack, &it); - if (ret < 0) - goto done; - - ret = reftable_iterator_seek_log(&it, arg->oldname); - if (ret < 0) - goto done; - - while (1) { - ret = reftable_iterator_next_log(&it, &old_log); - if (ret < 0) - goto done; - if (ret > 0 || strcmp(old_log.refname, arg->oldname)) { - ret = 0; - break; - } - if (reftable_log_record_is_deletion(&old_log)) - continue; - - free(old_log.refname); - - /* - * Copy over the old reflog entry with the new refname. - */ - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - logs[logs_nr] = old_log; - logs[logs_nr].refname = xstrdup(arg->newname); - logs_nr++; - - /* - * Delete the old reflog entry in case we are renaming. - */ - if (arg->delete_old) { - ALLOC_GROW(logs, logs_nr + 1, logs_alloc); - memset(&logs[logs_nr], 0, sizeof(logs[logs_nr])); - logs[logs_nr].refname = xstrdup(arg->oldname); - logs[logs_nr].value_type = REFTABLE_LOG_DELETION; - logs[logs_nr].update_index = old_log.update_index; - logs_nr++; - } - - /* - * Transfer ownership of the log record we're iterating over to - * the array of log records. Otherwise, the pointers would get - * free'd or reallocated by the iterator. - */ - memset(&old_log, 0, sizeof(old_log)); - } - - ret = reftable_writer_add_logs(writer, logs, logs_nr); - if (ret < 0) - goto done; - -done: - assert(ret != REFTABLE_API_ERROR); - reftable_iterator_destroy(&it); - string_list_clear(&skip, 0); - strbuf_release(&errbuf); - for (i = 0; i < logs_nr; i++) - reftable_log_record_release(&logs[i]); - free(logs); - for (i = 0; i < ARRAY_SIZE(refs); i++) - reftable_ref_record_release(&refs[i]); - reftable_ref_record_release(&old_ref); - reftable_log_record_release(&old_log); - return ret; -} - -static int reftable_be_rename_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) -{ - struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); - struct write_copy_arg arg = { - .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - .delete_old = 1, - }; - int ret; - - ret = refs->err; - if (ret < 0) - goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); - if (ret) - goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); - -done: - assert(ret != REFTABLE_API_ERROR); - return ret; -} - -static int reftable_be_copy_ref(struct ref_store *ref_store, - const char *oldrefname, - const char *newrefname, - const char *logmsg) -{ - struct reftable_ref_store *refs = - reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); - struct write_copy_arg arg = { - .refs = refs, - .oldname = oldrefname, - .newname = newrefname, - .logmsg = logmsg, - }; - int ret; - - ret = refs->err; - if (ret < 0) - goto done; - - ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); - if (ret) - goto done; - ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg, - &reftable_be_write_options(refs)->opts); - -done: - assert(ret != REFTABLE_API_ERROR); - return ret; -} - struct reftable_reflog_iterator { struct ref_iterator base; struct reftable_ref_store *refs; @@ -2909,9 +2625,6 @@ struct ref_storage_be refs_be_reftable = { .optimize = reftable_be_optimize, .optimize_required = reftable_be_optimize_required, - .rename_ref = reftable_be_rename_ref, - .copy_ref = reftable_be_copy_ref, - .iterator_begin = reftable_be_iterator_begin, .read_raw_ref = reftable_be_read_raw_ref, .read_symbolic_ref = reftable_be_read_symbolic_ref, -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz ` (3 preceding siblings ...) 2026-10-08 9:44 ` [PATCH v4 4/4] refs: remove backend-specific copy and rename callbacks Maciej Ciemborowicz @ 2026-10-08 10:10 ` Patrick Steinhardt 2026-10-08 10:43 ` Maciej Ciemborowicz 2026-10-08 15:54 ` Junio C Hamano 4 siblings, 2 replies; 35+ messages in thread From: Patrick Steinhardt @ 2026-10-08 10:10 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: git, Junio C Hamano, Karthik Nayak On Thu, Oct 08, 2026 at 11:44:15AM +0200, Maciej Ciemborowicz wrote: > Changes since v3: > > * Rebase onto 6de20f6092 (The 4th batch, 2026-10-06), the master commit > used in Junio's report. > * Preserve the packed preparation error in patch 2 as described above. > * Register t1425 and t1424 in t/meson.build in the commits adding them. Please engage with the reviewers. Just posting new versions without replying to them at all will very likely not get you anywhere. This kind of behaviour is nowadays a red flag and often hints at contributors who are basically just a meat proxy. And as a consequence, reviewers are very likely to disengage and stop reviewing your patch series altogether, which is frustrating to everyone involved. Patrick ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 10:10 ` [PATCH v4 0/4] refs: run copy and rename through transactions Patrick Steinhardt @ 2026-10-08 10:43 ` Maciej Ciemborowicz 2026-10-08 11:01 ` Maciej Ciemborowicz 2026-10-08 15:54 ` Junio C Hamano 1 sibling, 1 reply; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 10:43 UTC (permalink / raw) To: Patrick Steinhardt; +Cc: git, Junio C Hamano, Karthik Nayak On Thu, Oct 8, 2026 at 12:10 PM Patrick Steinhardt <ps@pks.im> wrote: > Please engage with the reviewers. Just posting new versions without > replying to them at all will very likely not get you anywhere. This kind > of behaviour is nowadays a red flag and often hints at contributors who > are basically just a meat proxy. And as a consequence, reviewers are > very likely to disengage and stop reviewing your patch series > altogether, which is frustrating to everyone involved. I reproduced both failures. In v3, preparing the reflogs overwrote an earlier packed-refs preparation error with success. V4 jumps to cleanup on that error. I had not run t0600 or t5510 before submitting v3. Both now pass unchanged, and the full standard test suite passes on macOS with both files and reftable, based on 6de20f6092. Sorry for missing this before submission. Thanks, Maciej ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 10:43 ` Maciej Ciemborowicz @ 2026-10-08 11:01 ` Maciej Ciemborowicz 2026-10-08 15:45 ` Junio C Hamano 0 siblings, 1 reply; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 11:01 UTC (permalink / raw) To: Patrick Steinhardt; +Cc: git, Junio C Hamano, Karthik Nayak On Thu, Oct 8, 2026 at 12:10 PM Patrick Steinhardt <ps@pks.im> wrote: > And as a consequence, reviewers are very likely to disengage and stop reviewing your patch series altogether, which is frustrating to everyone involved. I also read Notes from the Git Contributor's Summit yesterday, so I understand the problem, and I'm curious to see how things develop. I try to respect the people reviewing code and avoid being just a "meat proxy", but I increasingly feel like I'm becoming one myself. Month after month, AI is doing more and more of my work, and it's simply better at it than I am. If I had written all of this by hand, first, it would have taken me a month, and second, I would have made far more mistakes than AI does. I understand that the patch is large, and I'm concerned that this could be a barrier to code review and discourage people from reading it and eventually merging it. After all, the more code there is, the harder it becomes to maintain. If you don't have the time to review it, I completely understand. In that case, I'll wait for someone else to solve the problem, as I suspect it will come up again on the mailing list at some point in the future. Thanks, Maciej ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 11:01 ` Maciej Ciemborowicz @ 2026-10-08 15:45 ` Junio C Hamano 2026-10-08 19:06 ` Maciej Ciemborowicz 0 siblings, 1 reply; 35+ messages in thread From: Junio C Hamano @ 2026-10-08 15:45 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: Patrick Steinhardt, git, Karthik Nayak Maciej Ciemborowicz <maciej.ciemborowicz@gmail.com> writes: > better at it than I am. If I had written all of this by hand, first, > it would have taken me a month, and second, I would have made far more > mistakes than AI does. And thanks to your learning, the next patch series you will write would be of much higher quality. > I understand that the patch is large, and I'm concerned that this > could be a barrier to code review and discourage people from reading Large is not a problem; unnecessarily large is. Code generated by an LLM tends to be unnecessarily large, replete with poor refactorings and outright repetitions of existing code. This often shows that insufficient thought went into crafting the final result. Perhaps this is unavoidable, since they do not really think. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 15:45 ` Junio C Hamano @ 2026-10-08 19:06 ` Maciej Ciemborowicz 2026-10-08 19:19 ` Kristoffer Haugsbakk ` (2 more replies) 0 siblings, 3 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 19:06 UTC (permalink / raw) To: Junio C Hamano; +Cc: Patrick Steinhardt, git, Karthik Nayak On Thu, Oct 8, 2026 at 5:46 PM Junio C Hamano <gitster@pobox.com> wrote: > A response to reviews on the N-th round must come long before > sending the v(N+1) round of patches. Some contributors send them > after v(N+1), or immediately before, but the proper time to respond > is soon after receiving the reviews on vN and having had enough time > to understand the comments, before starting work on v(N+1). Only > after that work is complete would you send the new patches. Hence, > we expect the time between vN and v(N+1) from real contributors to > be measured in days, not hours. Whenever I see vN responses arrive > after or immediately before the v(N+1) patches, or worse, no > response at all but just the new patches, it smells fishy. I wouldn't rely on that. It's easy to fake. I'd rather ask for transparency. > And thanks to your learning, the next patch series you will write would be of much higher quality. TL;DR: I'm facing a moral dilemma: should I refrain from submitting a patch, or submit one written with the help of AI? If what I'm submitting isn't useful, feel free to say so or simply ignore the patch. If it has value and people are willing to accept it, I'm happy to continue. My current workflow looks like this: 1. I give an AI agent a task. 2. I check whether the solution works. If it doesn't, I refine the instructions until it does. 3. I check for edge cases. 4. I do a code review, ask the AI agent to explain what it wrote, and have it make corrections. My own code review is pretty weak, though, because I haven't spent (and won't spend) thousands of hours working with C and Git internals. 5. I ask another AI agent to review the code and think through its findings. I usually conclude that its review is better than mine. 6. I go back to "writing" code, and the agent implements the changes suggested during the review. 7. GOTO 4 if I'm still not satisfied. At this point, I face a moral dilemma, because the next step is to hand the patch over to other people to read, which means asking them to spend their time on it. I have the following options: 1. Report the bug without submitting a patch. 2. Report the bug and spend a month writing the patch myself, learning Git internals and C along the way. Unfortunately, I'm not planning a career in C. I've been programming for well over a decade in a completely different stack, and I simply can't afford to devote that much time to it. It might be worthwhile if I intended to spend the next several years working with Git and C, but unfortunately, that's not an option for me. On top of that, I can't shake the feeling that AI is already learning faster than I am. 3. Report the bug and solve the problem as well as I can with the help of an AI agent. So realistically, my choice comes down to options 1 and 3. And that's my moral dilemma: I don't know whether it's better not to submit a patch at all, or to do the best I can with the time I have, using AI. Choosing option 3, however, creates another dilemma before submitting the patch: Have I reviewed the code thoroughly enough myself? Do I understand what I'm doing well enough? I chose option 3, and out of respect for other people's time, I'm doing my best to understand what I'm submitting. And the only thing I can do to feel that I'm being honest about it is to be transparent and explain exactly what my workflow for developing this patch looks like. Thanks, Maciej ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 19:06 ` Maciej Ciemborowicz @ 2026-10-08 19:19 ` Kristoffer Haugsbakk 2026-10-08 21:11 ` Maciej Ciemborowicz 2026-10-09 5:41 ` Patrick Steinhardt 2026-10-10 0:22 ` brian m. carlson 2 siblings, 1 reply; 35+ messages in thread From: Kristoffer Haugsbakk @ 2026-10-08 19:19 UTC (permalink / raw) To: Maciej Ciemborowicz, Junio C Hamano Cc: Patrick Steinhardt, git, Karthik Nayak On Thu, Oct 8, 2026, at 21:06, Maciej Ciemborowicz wrote: >[snip] > And the only thing I can do to feel that I'm being honest about it is > to be transparent and explain exactly what my workflow for developing > this patch looks like. Augment option #1 with stating upfront that one is using a coding agents and to what degree. That’s full transparancy and people can then choose to engage or not. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 19:19 ` Kristoffer Haugsbakk @ 2026-10-08 21:11 ` Maciej Ciemborowicz 0 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-08 21:11 UTC (permalink / raw) To: Kristoffer Haugsbakk Cc: Junio C Hamano, Patrick Steinhardt, git, Karthik Nayak On Thu, Oct 8, 2026 at 9:20 PM Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com> wrote: > Augment option #1 with stating upfront that one is using a coding agents > and to what degree. That’s full transparancy and people can then choose > to engage or not. Yes, that could be a rule. Thanks, Maciej ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 19:06 ` Maciej Ciemborowicz 2026-10-08 19:19 ` Kristoffer Haugsbakk @ 2026-10-09 5:41 ` Patrick Steinhardt 2026-10-10 0:22 ` brian m. carlson 2 siblings, 0 replies; 35+ messages in thread From: Patrick Steinhardt @ 2026-10-09 5:41 UTC (permalink / raw) To: Maciej Ciemborowicz; +Cc: Junio C Hamano, git, Karthik Nayak On Thu, Oct 08, 2026 at 09:06:32PM +0200, Maciej Ciemborowicz wrote: > On Thu, Oct 8, 2026 at 5:46 PM Junio C Hamano <gitster@pobox.com> wrote: [snip] > So realistically, my choice comes down to options 1 and 3. And that's > my moral dilemma: I don't know whether it's better not to submit a > patch at all, or to do the best I can with the time I have, using AI. I think one important factor here is the size and complexity of the changes that you are proposing. If the change you're about to submit is smallish and can be reasoned through quite easily then I'd always be in favor of submitting such a patch. But if it's large and complex I'd refrain from doing so because chances are high that there are lots of nuances that the AI will get wrong and that you cannot vet because you lack the context. And I think that's something that was true even before AI. People that want to contributo to Git have a bit of a ramp-up time, and the guidance has always been to start with microprojects and then work your way up. And I think that guidance continues to apply even when these newcomers are equipped with AI. Patrick ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 19:06 ` Maciej Ciemborowicz 2026-10-08 19:19 ` Kristoffer Haugsbakk 2026-10-09 5:41 ` Patrick Steinhardt @ 2026-10-10 0:22 ` brian m. carlson 2 siblings, 0 replies; 35+ messages in thread From: brian m. carlson @ 2026-10-10 0:22 UTC (permalink / raw) To: Maciej Ciemborowicz Cc: Junio C Hamano, Patrick Steinhardt, git, Karthik Nayak [-- Attachment #1: Type: text/plain, Size: 2434 bytes --] On 2026-10-08 at 19:06:32, Maciej Ciemborowicz wrote: > At this point, I face a moral dilemma, because the next step is to > hand the patch over to other people to read, which means asking them > to spend their time on it. I have the following options: > > 1. Report the bug without submitting a patch. > 2. Report the bug and spend a month writing the patch myself, learning > Git internals and C along the way. Unfortunately, I'm not planning a > career in C. I've been programming for well over a decade in a > completely different stack, and I simply can't afford to devote that > much time to it. It might be worthwhile if I intended to spend the > next several years working with Git and C, but unfortunately, that's > not an option for me. On top of that, I can't shake the feeling that > AI is already learning faster than I am. > 3. Report the bug and solve the problem as well as I can with the help > of an AI agent. > > So realistically, my choice comes down to options 1 and 3. And that's > my moral dilemma: I don't know whether it's better not to submit a > patch at all, or to do the best I can with the time I have, using AI. It's better not to submit a patch at all if you can't submit something of reasonably good quality or can't satisfy the guidelines in SubmittingPatches (either with regard to AI usage or in any other way). I've been contributing for probably a decade and there are lots of times I submit a bug report or notice a problem and _don't_ submit a patch, either because I'm limited in time or because it's something that I'm just not familiar with (like the internals of the pack bitmap code). Reporting bugs is valuable to us even without a patch and nobody is expected to magically understand all the code. The problem you reported has been noticed by other people, so it may be that someone who _is_ familiar with the code or confidence in C picks it up. That happened with a change that went into 2.56 about an issue I reported around the 2.51 time frame. I agree this is something that should ideally be fixed and a fix would be valuable for another problem that I've also reported (`git remote rename` is very expensive with many remote-tracking refs), so it's likely someone will pick it up eventually and that could end up being me if nobody else gets to it first (no promises, though). -- brian m. carlson (they/them) Toronto, Ontario, CA [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 325 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 10:10 ` [PATCH v4 0/4] refs: run copy and rename through transactions Patrick Steinhardt 2026-10-08 10:43 ` Maciej Ciemborowicz @ 2026-10-08 15:54 ` Junio C Hamano 2026-10-09 21:21 ` Karthik Nayak 1 sibling, 1 reply; 35+ messages in thread From: Junio C Hamano @ 2026-10-08 15:54 UTC (permalink / raw) To: Patrick Steinhardt; +Cc: Maciej Ciemborowicz, git, Karthik Nayak Patrick Steinhardt <ps@pks.im> writes: > On Thu, Oct 08, 2026 at 11:44:15AM +0200, Maciej Ciemborowicz wrote: >> Changes since v3: >> >> * Rebase onto 6de20f6092 (The 4th batch, 2026-10-06), the master commit >> used in Junio's report. >> * Preserve the packed preparation error in patch 2 as described above. >> * Register t1425 and t1424 in t/meson.build in the commits adding them. > > Please engage with the reviewers. Just posting new versions without > replying to them at all will very likely not get you anywhere. This kind > of behaviour is nowadays a red flag and often hints at contributors who > are basically just a meat proxy. And as a consequence, reviewers are > very likely to disengage and stop reviewing your patch series > altogether, which is frustrating to everyone involved. Thanks for bringing this up. A response to reviews on the N-th round must come long before sending the v(N+1) round of patches. Some contributors send them after v(N+1), or immediately before, but the proper time to respond is soon after receiving the reviews on vN and having had enough time to understand the comments, before starting work on v(N+1). Only after that work is complete would you send the new patches. Hence, we expect the time between vN and v(N+1) from real contributors to be measured in days, not hours. Whenever I see vN responses arrive after or immediately before the v(N+1) patches, or worse, no response at all but just the new patches, it smells fishy. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-08 15:54 ` Junio C Hamano @ 2026-10-09 21:21 ` Karthik Nayak 2026-10-10 0:06 ` Maciej Ciemborowicz 0 siblings, 1 reply; 35+ messages in thread From: Karthik Nayak @ 2026-10-09 21:21 UTC (permalink / raw) To: Junio C Hamano, Patrick Steinhardt; +Cc: Maciej Ciemborowicz, git [-- Attachment #1: Type: text/plain, Size: 1928 bytes --] Junio C Hamano <gitster@pobox.com> writes: > Patrick Steinhardt <ps@pks.im> writes: > >> On Thu, Oct 08, 2026 at 11:44:15AM +0200, Maciej Ciemborowicz wrote: >>> Changes since v3: >>> >>> * Rebase onto 6de20f6092 (The 4th batch, 2026-10-06), the master commit >>> used in Junio's report. >>> * Preserve the packed preparation error in patch 2 as described above. >>> * Register t1425 and t1424 in t/meson.build in the commits adding them. >> >> Please engage with the reviewers. Just posting new versions without >> replying to them at all will very likely not get you anywhere. This kind >> of behaviour is nowadays a red flag and often hints at contributors who >> are basically just a meat proxy. And as a consequence, reviewers are >> very likely to disengage and stop reviewing your patch series >> altogether, which is frustrating to everyone involved. > > Thanks for bringing this up. > > A response to reviews on the N-th round must come long before > sending the v(N+1) round of patches. Some contributors send them > after v(N+1), or immediately before, but the proper time to respond > is soon after receiving the reviews on vN and having had enough time > to understand the comments, before starting work on v(N+1). Only > after that work is complete would you send the new patches. Hence, > we expect the time between vN and v(N+1) from real contributors to > be measured in days, not hours. Whenever I see vN responses arrive > after or immediately before the v(N+1) patches, or worse, no > response at all but just the new patches, it smells fishy. This is kinda why I stopped reviewing the other patches [1] from the author, my reviews were simply met with N+1 version of the series. At some point I felt it would've been faster if I used a LLM locally for the same task and reviewed its code instead. [1]: https://lore.kernel.org/git/CACQ=SRGTTdQ+dHXhN6F52dBv5KxZBRfk_Em2fvmEmGJDoB6oTg@mail.gmail.com/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 690 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/4] refs: run copy and rename through transactions 2026-10-09 21:21 ` Karthik Nayak @ 2026-10-10 0:06 ` Maciej Ciemborowicz 0 siblings, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-10-10 0:06 UTC (permalink / raw) To: Karthik Nayak; +Cc: Junio C Hamano, Patrick Steinhardt, git > At some point I felt it would've been faster if I used a LLM locally for the same task and reviewed its code instead. If writing the patches on your end would be more efficient than reviewing mine, I'd be more than happy to go with that approach. Ultimately, my main goal is simply to get both issues resolved, the one discussed here and the one in the thread you referenced. I don't want to hold you back. Thanks, Maciej On Fri, Oct 9, 2026 at 11:21 PM Karthik Nayak <karthik.188@gmail.com> wrote: > > Junio C Hamano <gitster@pobox.com> writes: > > > Patrick Steinhardt <ps@pks.im> writes: > > > >> On Thu, Oct 08, 2026 at 11:44:15AM +0200, Maciej Ciemborowicz wrote: > >>> Changes since v3: > >>> > >>> * Rebase onto 6de20f6092 (The 4th batch, 2026-10-06), the master commit > >>> used in Junio's report. > >>> * Preserve the packed preparation error in patch 2 as described above. > >>> * Register t1425 and t1424 in t/meson.build in the commits adding them. > >> > >> Please engage with the reviewers. Just posting new versions without > >> replying to them at all will very likely not get you anywhere. This kind > >> of behaviour is nowadays a red flag and often hints at contributors who > >> are basically just a meat proxy. And as a consequence, reviewers are > >> very likely to disengage and stop reviewing your patch series > >> altogether, which is frustrating to everyone involved. > > > > Thanks for bringing this up. > > > > A response to reviews on the N-th round must come long before > > sending the v(N+1) round of patches. Some contributors send them > > after v(N+1), or immediately before, but the proper time to respond > > is soon after receiving the reviews on vN and having had enough time > > to understand the comments, before starting work on v(N+1). Only > > after that work is complete would you send the new patches. Hence, > > we expect the time between vN and v(N+1) from real contributors to > > be measured in days, not hours. Whenever I see vN responses arrive > > after or immediately before the v(N+1) patches, or worse, no > > response at all but just the new patches, it smells fishy. > > This is kinda why I stopped reviewing the other patches [1] from the > author, my reviews were simply met with N+1 version of the series. At > some point I felt it would've been faster if I used a LLM locally for > the same task and reviewed its code instead. > > [1]: https://lore.kernel.org/git/CACQ=SRGTTdQ+dHXhN6F52dBv5KxZBRfk_Em2fvmEmGJDoB6oTg@mail.gmail.com/ ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/3] refs: report old OIDs for batched deletions 2026-09-19 20:52 ` Karthik Nayak 2026-09-20 16:50 ` [PATCH] refs: run copy and rename through transactions Maciej Ciemborowicz @ 2026-09-23 12:49 ` Maciej Ciemborowicz 1 sibling, 0 replies; 35+ messages in thread From: Maciej Ciemborowicz @ 2026-09-23 12:49 UTC (permalink / raw) To: git; +Cc: Maciej Ciemborowicz Hi Junio, > Can we avoid REF_TRANSACTION_FLAG_SKIP_HOOK by changing the call chain > instead of exposing a general mechanism for skipping hooks? I looked into that, but I do not think it removes the distinction cleanly. The packed-refs transaction is an internal, physical part of one logical files-backend update: it needs the normal transaction machinery, while the reference-transaction hook must observe only the outer logical update. Restructuring the call chain would either duplicate the prepare/finish/abort lifecycle or hide the same hook-suppression decision in a less explicit helper. I do agree that this should not be a public escape hatch, though. I will keep it internal to the refs implementation and limit its use to these internal transactions. > Should the int copy parameter simply be bool copy, given that the > function has only two modes? Yes, agreed. It is a two-state mode, so bool copy expresses the contract more clearly. I will change it in the next reroll. Thanks, Maciej ^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2026-10-10 0:22 UTC | newest] Thread overview: 35+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-09-19 13:33 [BUG] reference-transaction hook misses destination of git branch -m Maciej Ciemborowicz 2026-09-19 20:52 ` Karthik Nayak 2026-09-20 16:50 ` [PATCH] refs: run copy and rename through transactions Maciej Ciemborowicz 2026-09-21 17:54 ` Junio C Hamano 2026-09-21 23:28 ` Junio C Hamano 2026-09-22 13:08 ` Maciej Ciemborowicz 2026-09-23 13:36 ` [PATCH v2] " Maciej Ciemborowicz 2026-09-30 3:32 ` Maciej Ciemborowicz 2026-10-02 10:56 ` Patrick Steinhardt 2026-10-02 14:16 ` Maciej Ciemborowicz 2026-10-05 6:03 ` Patrick Steinhardt 2026-10-07 18:05 ` [PATCH v3 0/4] " Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 2/4] refs: support replacing reflogs in a transaction Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 3/4] refs: run copy and rename through ordinary transactions Maciej Ciemborowicz 2026-10-07 18:05 ` [PATCH v3 4/4] refs: remove backend-specific copy and rename callbacks Maciej Ciemborowicz 2026-10-07 19:55 ` [PATCH v3 0/4] refs: run copy and rename through transactions Junio C Hamano 2026-10-08 9:44 ` [PATCH v4 " Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 1/4] refs: distinguish internal transactions from logical updates Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 2/4] refs: support replacing reflogs in a transaction Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 3/4] refs: run copy and rename through ordinary transactions Maciej Ciemborowicz 2026-10-08 9:44 ` [PATCH v4 4/4] refs: remove backend-specific copy and rename callbacks Maciej Ciemborowicz 2026-10-08 10:10 ` [PATCH v4 0/4] refs: run copy and rename through transactions Patrick Steinhardt 2026-10-08 10:43 ` Maciej Ciemborowicz 2026-10-08 11:01 ` Maciej Ciemborowicz 2026-10-08 15:45 ` Junio C Hamano 2026-10-08 19:06 ` Maciej Ciemborowicz 2026-10-08 19:19 ` Kristoffer Haugsbakk 2026-10-08 21:11 ` Maciej Ciemborowicz 2026-10-09 5:41 ` Patrick Steinhardt 2026-10-10 0:22 ` brian m. carlson 2026-10-08 15:54 ` Junio C Hamano 2026-10-09 21:21 ` Karthik Nayak 2026-10-10 0:06 ` Maciej Ciemborowicz 2026-09-23 12:49 ` [PATCH v4 0/3] refs: report old OIDs for batched deletions Maciej Ciemborowicz
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.