From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Karthik Nayak <karthik.188@gmail.com>,
"brian m. carlson" <sandals@crustytoothpaste.net>,
Jeff King <peff@peff.net>, Junio C Hamano <gitster@pobox.com>,
shejialuo <shejialuo@gmail.com>,
Christian Couder <chriscool@tuxfamily.org>
Subject: [PATCH v4 05/16] refs/reftable: batch refname availability checks
Date: Fri, 28 Feb 2025 10:26:42 +0100 [thread overview]
Message-ID: <20250228-pks-update-ref-optimization-v4-5-6425c04268b5@pks.im> (raw)
In-Reply-To: <20250228-pks-update-ref-optimization-v4-0-6425c04268b5@pks.im>
Refactor the "reftable" backend to batch the availability check for
refnames. This does not yet have an effect on performance as we
essentially still call `refs_verify_refname_available()` in a loop, but
this will change in subsequent commits.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
refs/reftable-backend.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index d39a14c5a46..2a90e7cb391 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1069,6 +1069,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
reftable_be_downcast(ref_store, REF_STORE_WRITE|REF_STORE_MAIN, "ref_transaction_prepare");
struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT;
struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
+ struct string_list refnames_to_check = STRING_LIST_INIT_NODUP;
struct reftable_transaction_data *tx_data = NULL;
struct reftable_backend *be;
struct object_id head_oid;
@@ -1224,12 +1225,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
* can output a proper error message instead of failing
* at a later point.
*/
- ret = refs_verify_refname_available(ref_store, u->refname,
- &affected_refnames, NULL,
- transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
- err);
- if (ret < 0)
- goto done;
+ string_list_append(&refnames_to_check, u->refname);
/*
* There is no need to write the reference deletion
@@ -1379,6 +1375,13 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
}
}
+ string_list_sort(&refnames_to_check);
+ ret = refs_verify_refnames_available(ref_store, &refnames_to_check, &affected_refnames, NULL,
+ transaction->flags & REF_TRANSACTION_FLAG_INITIAL,
+ err);
+ if (ret < 0)
+ goto done;
+
transaction->backend_data = tx_data;
transaction->state = REF_TRANSACTION_PREPARED;
@@ -1394,6 +1397,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
string_list_clear(&affected_refnames, 0);
strbuf_release(&referent);
strbuf_release(&head_referent);
+ string_list_clear(&refnames_to_check, 0);
return ret;
}
--
2.49.0.rc0.375.gae4b89d849.dirty
next prev parent reply other threads:[~2025-02-28 9:26 UTC|newest]
Thread overview: 169+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-17 15:50 [PATCH 00/14] refs: batch refname availability checks Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 01/14] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 02/14] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 03/14] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-18 16:04 ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 04/14] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 05/14] refs/reftable: start using `refs_verify_refnames_available()` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 06/14] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-18 16:12 ` Karthik Nayak
2025-02-19 11:52 ` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 07/14] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-18 16:52 ` shejialuo
2025-02-19 11:52 ` Patrick Steinhardt
2025-02-19 12:41 ` shejialuo
2025-02-19 12:59 ` Patrick Steinhardt
2025-02-19 13:06 ` shejialuo
2025-02-19 13:17 ` Patrick Steinhardt
2025-02-19 13:20 ` Patrick Steinhardt
2025-02-19 13:23 ` shejialuo
2025-02-18 17:13 ` Karthik Nayak
2025-02-19 11:52 ` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 08/14] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 09/14] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-19 20:10 ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 10/14] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-19 20:13 ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 11/14] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 12/14] refs/iterator: implement seeking for `packed-ref` iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 13/14] refs/iterator: implement seeking for "files" iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 14/14] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-18 17:10 ` [PATCH 00/14] refs: batch refname availability checks brian m. carlson
2025-02-19 13:23 ` [PATCH v2 00/16] " Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-19 17:02 ` Justin Tobler
2025-02-19 13:23 ` [PATCH v2 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-21 8:00 ` Jeff King
2025-02-21 8:36 ` Patrick Steinhardt
2025-02-21 9:06 ` Jeff King
2025-02-19 13:23 ` [PATCH v2 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-19 18:21 ` Justin Tobler
2025-02-20 8:05 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 05/16] refs/reftable: " Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-24 13:08 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-24 13:37 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-24 14:00 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-24 14:49 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 14/16] refs/iterator: implement seeking for `packed-ref` iterators Patrick Steinhardt
2025-02-24 15:09 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-25 12:07 ` shejialuo
2025-02-19 13:23 ` [PATCH v2 15/16] refs/iterator: implement seeking for "files" iterators Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-24 15:14 ` shejialuo
2025-02-24 15:18 ` [PATCH v2 00/16] refs: batch refname availability checks shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 " Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-26 22:26 ` Junio C Hamano
2025-02-27 11:57 ` Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 05/16] refs/reftable: " Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-25 8:56 ` [PATCH v3 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-02-25 8:56 ` [PATCH v3 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-02-25 8:56 ` [PATCH v3 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 00/16] refs: batch refname availability checks Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-06 13:21 ` Karthik Nayak
2025-02-28 9:26 ` [PATCH v4 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-06 13:47 ` Karthik Nayak
2025-02-28 9:26 ` Patrick Steinhardt [this message]
2025-03-06 14:00 ` [PATCH v4 05/16] refs/reftable: " Karthik Nayak
2025-03-06 14:12 ` Karthik Nayak
2025-03-06 15:13 ` Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-06 14:10 ` Karthik Nayak
2025-02-28 9:26 ` [PATCH v4 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-06 14:16 ` Karthik Nayak
2025-02-28 9:26 ` [PATCH v4 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-06 14:20 ` [PATCH v4 00/16] refs: batch refname availability checks Karthik Nayak
2025-03-06 15:08 ` [PATCH v5 " Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-12 12:12 ` shejialuo
2025-03-06 15:08 ` [PATCH v5 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-12 12:36 ` shejialuo
2025-03-12 12:44 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 05/16] refs/reftable: " Patrick Steinhardt
2025-03-12 12:54 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-03-12 12:58 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-12 13:06 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-03-12 13:22 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-03-12 13:45 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-06 15:32 ` [PATCH v5 00/16] refs: batch refname availability checks Karthik Nayak
2025-03-12 14:03 ` shejialuo
2025-03-12 15:56 ` [PATCH v6 " Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 05/16] refs/reftable: " Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-04-03 19:56 ` Elijah Newren
2025-04-03 22:18 ` brian m. carlson
2025-04-04 7:18 ` shejialuo
2025-04-04 10:00 ` Patrick Steinhardt
2025-04-04 10:05 ` Patrick Steinhardt
2025-04-04 10:59 ` Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-13 2:57 ` [PATCH v6 00/16] refs: batch refname availability checks shejialuo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250228-pks-update-ref-optimization-v4-5-6425c04268b5@pks.im \
--to=ps@pks.im \
--cc=chriscool@tuxfamily.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=karthik.188@gmail.com \
--cc=peff@peff.net \
--cc=sandals@crustytoothpaste.net \
--cc=shejialuo@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).