From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: karthik nayak <karthik.188@gmail.com>,
Junio C Hamano <gitster@pobox.com>,
Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com>,
Christian Couder <chriscool@tuxfamily.org>
Subject: [PATCH v3 04/10] refs/files: support symbolic and root refs in initial transaction
Date: Mon, 25 Nov 2024 07:27:09 +0100 [thread overview]
Message-ID: <20241125-pks-refs-optimize-migrations-v3-4-17bc85e33ad7@pks.im> (raw)
In-Reply-To: <20241125-pks-refs-optimize-migrations-v3-0-17bc85e33ad7@pks.im>
The "files" backend has implemented special logic when committing
the first transactions in an otherwise empty ref store: instead of
writing all refs as separate loose files, it instead knows to write them
all into a "packed-refs" file directly. This is significantly more
efficient than having to write each of the refs as separate "loose" ref.
The only user of this optimization is git-clone(1), which only uses this
mechanism to write regular refs. Consequently, the implementation does
not know how to handle both symbolic and root refs. While fine in the
context of git-clone(1), this keeps us from using the mechanism in more
cases.
Adapt the logic to also support symbolic and root refs by using a second
transaction that we use for all of the refs that need to be written as
loose refs.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
refs/files-backend.c | 44 ++++++++++++++++++++++++++++++++++----------
1 file changed, 34 insertions(+), 10 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3ed18475a72aa4798d15b2912c37b4caabd47aac..116d4259697b20583cb2db34ed47025e8781cd42 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2995,6 +2995,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
int ret = 0;
struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
struct ref_transaction *packed_transaction = NULL;
+ struct ref_transaction *loose_transaction = NULL;
assert(err);
@@ -3040,6 +3041,7 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
if ((update->flags & REF_HAVE_OLD) &&
!is_null_oid(&update->old_oid))
BUG("initial ref transaction with old_sha1 set");
+
if (refs_verify_refname_available(&refs->base, update->refname,
&affected_refnames, NULL,
err)) {
@@ -3048,26 +3050,48 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,
}
/*
- * Add a reference creation for this reference to the
- * packed-refs transaction:
+ * packed-refs don't support symbolic refs and root refs, so we
+ * have to queue these references via the loose transaction.
*/
- ref_transaction_add_update(packed_transaction, update->refname,
- update->flags & ~REF_HAVE_OLD,
- &update->new_oid, &update->old_oid,
- NULL, NULL, NULL);
+ if (update->new_target || is_root_ref(update->refname)) {
+ if (!loose_transaction) {
+ loose_transaction = ref_store_transaction_begin(&refs->base, 0, err);
+ if (!loose_transaction) {
+ ret = TRANSACTION_GENERIC_ERROR;
+ goto cleanup;
+ }
+ }
+
+ ref_transaction_add_update(loose_transaction, update->refname,
+ update->flags & ~REF_HAVE_OLD,
+ update->new_target ? NULL : &update->new_oid, NULL,
+ update->new_target, NULL, NULL);
+ } else {
+ ref_transaction_add_update(packed_transaction, update->refname,
+ update->flags & ~REF_HAVE_OLD,
+ &update->new_oid, &update->old_oid,
+ NULL, NULL, NULL);
+ }
}
- if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
+ if (packed_refs_lock(refs->packed_ref_store, 0, err) ||
+ ref_transaction_commit(packed_transaction, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
+ packed_refs_unlock(refs->packed_ref_store);
- if (ref_transaction_commit(packed_transaction, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
+ if (loose_transaction) {
+ if (ref_transaction_prepare(loose_transaction, err) ||
+ ref_transaction_commit(loose_transaction, err)) {
+ ret = TRANSACTION_GENERIC_ERROR;
+ goto cleanup;
+ }
}
- packed_refs_unlock(refs->packed_ref_store);
cleanup:
+ if (loose_transaction)
+ ref_transaction_free(loose_transaction);
if (packed_transaction)
ref_transaction_free(packed_transaction);
transaction->state = REF_TRANSACTION_CLOSED;
--
2.47.0.274.g962d0b743d.dirty
next prev parent reply other threads:[~2024-11-25 6:27 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-08 9:34 [PATCH 00/10] refs: optimize ref format migrations Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 01/10] refs: allow passing flags when setting up a transaction Patrick Steinhardt
2024-11-11 10:30 ` karthik nayak
2024-11-11 12:53 ` Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 02/10] refs/files: move logic to commit initial transaction Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 03/10] refs: introduce "initial" transaction flag Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 04/10] refs/files: support symbolic and root refs in initial transaction Patrick Steinhardt
2024-11-11 10:42 ` karthik nayak
2024-11-11 12:53 ` Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 05/10] refs: use "initial" transaction semantics to migrate refs Patrick Steinhardt
2024-11-11 10:43 ` karthik nayak
2024-11-08 9:34 ` [PATCH 06/10] refs: skip collision checks in initial transactions Patrick Steinhardt
2024-11-11 10:53 ` karthik nayak
2024-11-08 9:34 ` [PATCH 07/10] refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG` Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 08/10] reftable/writer: optimize allocations by using a scratch buffer Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 09/10] reftable/block: rename `block_writer::buf` variable Patrick Steinhardt
2024-11-08 9:34 ` [PATCH 10/10] reftable/block: optimize allocations by using scratch buffer Patrick Steinhardt
2024-11-11 10:57 ` [PATCH 00/10] refs: optimize ref format migrations karthik nayak
2024-11-11 12:53 ` Patrick Steinhardt
2024-11-20 7:04 ` Junio C Hamano
2024-11-20 7:50 ` Patrick Steinhardt
2024-11-20 10:25 ` Christian Couder
2024-11-25 5:52 ` Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 " Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 01/10] refs: allow passing flags when setting up a transaction Patrick Steinhardt
2024-11-20 10:19 ` Christian Couder
2024-11-20 7:51 ` [PATCH v2 02/10] refs/files: move logic to commit initial transaction Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 03/10] refs: introduce "initial" transaction flag Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 04/10] refs/files: support symbolic and root refs in initial transaction Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 05/10] refs: use "initial" transaction semantics to migrate refs Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 06/10] refs: skip collision checks in initial transactions Patrick Steinhardt
2024-11-20 10:21 ` Christian Couder
2024-11-25 5:52 ` Patrick Steinhardt
2024-11-20 10:42 ` Kristoffer Haugsbakk
2024-11-25 5:52 ` Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 07/10] refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG` Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 08/10] reftable/writer: optimize allocations by using a scratch buffer Patrick Steinhardt
2024-11-20 10:21 ` Christian Couder
2024-11-25 5:52 ` Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 09/10] reftable/block: rename `block_writer::buf` variable Patrick Steinhardt
2024-11-20 7:51 ` [PATCH v2 10/10] reftable/block: optimize allocations by using scratch buffer Patrick Steinhardt
2024-11-20 10:22 ` Christian Couder
2024-11-25 6:27 ` [PATCH v3 00/10] refs: optimize ref format migrations Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 01/10] refs: allow passing flags when setting up a transaction Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 02/10] refs/files: move logic to commit initial transaction Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 03/10] refs: introduce "initial" transaction flag Patrick Steinhardt
2024-11-25 6:27 ` Patrick Steinhardt [this message]
2024-11-25 6:27 ` [PATCH v3 05/10] refs: use "initial" transaction semantics to migrate refs Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 06/10] refs: skip collision checks in initial transactions Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 07/10] refs: don't normalize log messages with `REF_SKIP_CREATE_REFLOG` Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 08/10] reftable/writer: optimize allocations by using a scratch buffer Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 09/10] reftable/block: rename `block_writer::buf` variable Patrick Steinhardt
2024-11-25 6:27 ` [PATCH v3 10/10] reftable/block: optimize allocations by using scratch buffer Patrick Steinhardt
2024-11-25 6:29 ` [PATCH v3 00/10] refs: optimize ref format migrations Patrick Steinhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241125-pks-refs-optimize-migrations-v3-4-17bc85e33ad7@pks.im \
--to=ps@pks.im \
--cc=chriscool@tuxfamily.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=karthik.188@gmail.com \
--cc=kristofferhaugsbakk@fastmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).