git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Karthik Nayak <karthik.188@gmail.com>,
	 "brian m. carlson" <sandals@crustytoothpaste.net>,
	 Jeff King <peff@peff.net>, Junio C Hamano <gitster@pobox.com>,
	 Christian Couder <chriscool@tuxfamily.org>
Subject: [PATCH 03/14] builtin/update-ref: skip ambiguity checks when parsing object IDs
Date: Mon, 17 Feb 2025 16:50:17 +0100	[thread overview]
Message-ID: <20250217-pks-update-ref-optimization-v1-3-a2b6d87a24af@pks.im> (raw)
In-Reply-To: <20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im>

Most of the commands in git-update-ref(1) accept an old and/or new
object ID to update a specific reference to. These object IDs get parsed
via `repo_get_oid()`, which not only handles plain object IDs, but also
those that have a suffix like "~" or "^2". More surprisingly though, it
even knows to resolve references, despite the fact that its manpage does
not mention this fact even once.

One consequence of this is that we also check for ambiguous references:
when parsing a full object ID where the DWIM mechanism would also cause
us to resolve it as a branch, we'd end up printing a warning. While this
check makes sense to have in general, it is arguably less useful in the
context of git-update-ref(1). This is out of two reasons:

  - The manpage is explicitly structured around object IDs. So if we see
    a fully blown object ID, the intent should be quite clear in
    general.

  - The command is part of our plumbing layer and not a tool that users
    would generally use in interactive workflows. As such, the warning
    will likely not be visible to anybody in the first place.

Furthermore, this check can be quite expensive when updating lots of
references via `--stdin`, because we try to read multiple references per
object ID that we parse according to the DWIM rules. This effect can be
seen both with the "files" and "reftable" backend.

Disable the warning in git-update-ref(1), which provides a significant
speedup with both backends. The following benchmark creates 10000 new
references with a 100000 preexisting refs with the "files" backend:

    Benchmark 1: update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD~)
      Time (mean ± σ):     467.3 ms ±   5.1 ms    [User: 100.0 ms, System: 365.1 ms]
      Range (min … max):   461.9 ms … 479.3 ms    10 runs

    Benchmark 2: update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD)
      Time (mean ± σ):     394.1 ms ±   5.8 ms    [User: 63.3 ms, System: 327.6 ms]
      Range (min … max):   384.9 ms … 405.7 ms    10 runs

    Summary
      update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD) ran
        1.19 ± 0.02 times faster than update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD~)

And with the "reftable" backend:

    Benchmark 1: update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD~)
      Time (mean ± σ):     146.9 ms ±   2.2 ms    [User: 90.4 ms, System: 56.0 ms]
      Range (min … max):   142.7 ms … 150.8 ms    19 runs

    Benchmark 2: update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD)
      Time (mean ± σ):      63.2 ms ±   1.1 ms    [User: 41.0 ms, System: 21.8 ms]
      Range (min … max):    61.1 ms …  66.6 ms    41 runs

    Summary
      update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD) ran
        2.32 ± 0.05 times faster than update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD~)

Note that the absolute improvement with both backends is roughly in the
same ballpark, but the relative improvement for the "reftable" backend
is more significant because writing the new table to disk is faster in
the first place.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/update-ref.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 4d35bdc4b4b..ac330748244 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -179,7 +179,8 @@ static int parse_next_oid(const char **next, const char *end,
 		(*next)++;
 		*next = parse_arg(*next, &arg);
 		if (arg.len) {
-			if (repo_get_oid(the_repository, arg.buf, oid))
+			if (repo_get_oid_with_flags(the_repository, arg.buf, oid,
+						    GET_OID_HASH_SKIP_AMBIGUITY_CHECK))
 				goto invalid;
 		} else {
 			/* Without -z, an empty value means all zeros: */
@@ -197,7 +198,8 @@ static int parse_next_oid(const char **next, const char *end,
 		*next += arg.len;
 
 		if (arg.len) {
-			if (repo_get_oid(the_repository, arg.buf, oid))
+			if (repo_get_oid_with_flags(the_repository, arg.buf, oid,
+						    GET_OID_HASH_SKIP_AMBIGUITY_CHECK))
 				goto invalid;
 		} else if (flags & PARSE_SHA1_ALLOW_EMPTY) {
 			/* With -z, treat an empty value as all zeros: */
@@ -772,7 +774,8 @@ int cmd_update_ref(int argc,
 		refname = argv[0];
 		value = argv[1];
 		oldval = argv[2];
-		if (repo_get_oid(the_repository, value, &oid))
+		if (repo_get_oid_with_flags(the_repository, value, &oid,
+					    GET_OID_HASH_SKIP_AMBIGUITY_CHECK))
 			die("%s: not a valid SHA1", value);
 	}
 
@@ -783,7 +786,8 @@ int cmd_update_ref(int argc,
 			 * must not already exist:
 			 */
 			oidclr(&oldoid, the_repository->hash_algo);
-		else if (repo_get_oid(the_repository, oldval, &oldoid))
+		else if (repo_get_oid_with_flags(the_repository, oldval, &oldoid,
+						 GET_OID_HASH_SKIP_AMBIGUITY_CHECK))
 			die("%s: not a valid old SHA1", oldval);
 	}
 

-- 
2.48.1.666.gff9fcf71b7.dirty


  parent reply	other threads:[~2025-02-17 15:50 UTC|newest]

Thread overview: 169+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-02-17 15:50 [PATCH 00/14] refs: batch refname availability checks Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 01/14] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 02/14] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-17 15:50 ` Patrick Steinhardt [this message]
2025-02-18 16:04   ` [PATCH 03/14] builtin/update-ref: skip ambiguity checks when parsing object IDs Karthik Nayak
2025-02-17 15:50 ` [PATCH 04/14] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 05/14] refs/reftable: start using `refs_verify_refnames_available()` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 06/14] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-18 16:12   ` Karthik Nayak
2025-02-19 11:52     ` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 07/14] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-18 16:52   ` shejialuo
2025-02-19 11:52     ` Patrick Steinhardt
2025-02-19 12:41       ` shejialuo
2025-02-19 12:59         ` Patrick Steinhardt
2025-02-19 13:06           ` shejialuo
2025-02-19 13:17             ` Patrick Steinhardt
2025-02-19 13:20               ` Patrick Steinhardt
2025-02-19 13:23                 ` shejialuo
2025-02-18 17:13   ` Karthik Nayak
2025-02-19 11:52     ` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 08/14] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 09/14] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-19 20:10   ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 10/14] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-19 20:13   ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 11/14] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 12/14] refs/iterator: implement seeking for `packed-ref` iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 13/14] refs/iterator: implement seeking for "files" iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 14/14] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-18 17:10 ` [PATCH 00/14] refs: batch refname availability checks brian m. carlson
2025-02-19 13:23 ` [PATCH v2 00/16] " Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-19 17:02     ` Justin Tobler
2025-02-19 13:23   ` [PATCH v2 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-21  8:00     ` Jeff King
2025-02-21  8:36       ` Patrick Steinhardt
2025-02-21  9:06         ` Jeff King
2025-02-19 13:23   ` [PATCH v2 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-19 18:21     ` Justin Tobler
2025-02-20  8:05       ` Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 05/16] refs/reftable: " Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-24 13:08     ` shejialuo
2025-02-25  7:39       ` Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-24 13:37     ` shejialuo
2025-02-25  7:39       ` Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-24 14:00     ` shejialuo
2025-02-25  7:39       ` Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-24 14:49     ` shejialuo
2025-02-25  7:39       ` Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 14/16] refs/iterator: implement seeking for `packed-ref` iterators Patrick Steinhardt
2025-02-24 15:09     ` shejialuo
2025-02-25  7:39       ` Patrick Steinhardt
2025-02-25 12:07         ` shejialuo
2025-02-19 13:23   ` [PATCH v2 15/16] refs/iterator: implement seeking for "files" iterators Patrick Steinhardt
2025-02-19 13:23   ` [PATCH v2 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-24 15:14     ` shejialuo
2025-02-24 15:18   ` [PATCH v2 00/16] refs: batch refname availability checks shejialuo
2025-02-25  7:39     ` Patrick Steinhardt
2025-02-25  8:55 ` [PATCH v3 " Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-26 22:26     ` Junio C Hamano
2025-02-27 11:57       ` Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 05/16] refs/reftable: " Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-25  8:55   ` [PATCH v3 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-25  8:56   ` [PATCH v3 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-02-25  8:56   ` [PATCH v3 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-02-25  8:56   ` [PATCH v3 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-28  9:26 ` [PATCH v4 00/16] refs: batch refname availability checks Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-06 13:21     ` Karthik Nayak
2025-02-28  9:26   ` [PATCH v4 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-06 13:47     ` Karthik Nayak
2025-02-28  9:26   ` [PATCH v4 05/16] refs/reftable: " Patrick Steinhardt
2025-03-06 14:00     ` Karthik Nayak
2025-03-06 14:12       ` Karthik Nayak
2025-03-06 15:13         ` Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-06 14:10     ` Karthik Nayak
2025-02-28  9:26   ` [PATCH v4 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-06 14:16     ` Karthik Nayak
2025-02-28  9:26   ` [PATCH v4 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-02-28  9:26   ` [PATCH v4 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-06 14:20   ` [PATCH v4 00/16] refs: batch refname availability checks Karthik Nayak
2025-03-06 15:08 ` [PATCH v5 " Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-12 12:12     ` shejialuo
2025-03-06 15:08   ` [PATCH v5 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-12 12:36     ` shejialuo
2025-03-12 12:44       ` shejialuo
2025-03-12 15:36       ` Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 05/16] refs/reftable: " Patrick Steinhardt
2025-03-12 12:54     ` shejialuo
2025-03-12 15:36       ` Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-03-12 12:58     ` shejialuo
2025-03-12 15:36       ` Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-12 13:06     ` shejialuo
2025-03-12 15:36       ` Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-03-12 13:22     ` shejialuo
2025-03-12 15:36       ` Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-03-12 13:45     ` shejialuo
2025-03-12 15:36       ` Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-03-06 15:08   ` [PATCH v5 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-06 15:32   ` [PATCH v5 00/16] refs: batch refname availability checks Karthik Nayak
2025-03-12 14:03   ` shejialuo
2025-03-12 15:56 ` [PATCH v6 " Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 05/16] refs/reftable: " Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-04-03 19:56     ` Elijah Newren
2025-04-03 22:18       ` brian m. carlson
2025-04-04  7:18         ` shejialuo
2025-04-04 10:00       ` Patrick Steinhardt
2025-04-04 10:05         ` Patrick Steinhardt
2025-04-04 10:59           ` Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-03-12 15:56   ` [PATCH v6 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-13  2:57   ` [PATCH v6 00/16] refs: batch refname availability checks shejialuo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250217-pks-update-ref-optimization-v1-3-a2b6d87a24af@pks.im \
    --to=ps@pks.im \
    --cc=chriscool@tuxfamily.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=karthik.188@gmail.com \
    --cc=peff@peff.net \
    --cc=sandals@crustytoothpaste.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).