From: Karthik Nayak <karthik.188@gmail.com>
To: git@vger.kernel.org
Cc: Patrick Steinhardt <ps@pks.im>,
Junio C Hamano <gitster@pobox.com>, Toon Claes <toon@iotcl.com>,
Karthik Nayak <karthik.188@gmail.com>
Subject: [PATCH v4] packed-refs: use `fwrite()` when passing refs verbatim
Date: Wed, 07 Oct 2026 13:20:16 +0200 [thread overview]
Message-ID: <20261007-kn-speedup-packed-refs-v4-1-79a411026596@gmail.com> (raw)
In-Reply-To: <20260930-kn-speedup-packed-refs-v1-1-111cd03d9b0e@gmail.com>
The `write_with_updates()` function uses a `struct ref_iterator` to
iterate over all refs to write to the temporary packed-refs file. It
receives the iterator from `packed_ref_iterator_begin()` which takes a
snapshot of the 'packed-refs' file.
While writing to the new packed-refs file, writes are routed via
`write_packed_entry()` which uses `fprintf()`. Even for references which
haven't changed, we use the same mechanism. This is slow since we need
to do string parsing and formatting. Instead, let's track the position
of unchanged references in the snapshot iterator and directly write
verbatim via `fwrite()`.
With this, any sanitization which was happening as a side effect of
reformatting is now lost. But that was never the job of this section of
the code, since the main intention is to simply rewrite the remaining
refs post deletion of the selective few.
This removes the unnecessary formatting operation involved. We can see a
consistent ~20% performance improvement when deleting from packed
references.
Benchmark 1: update-ref: delete ref (refcount = 100000, revision = master)
Time (mean ± σ): 28.7 ms ± 1.7 ms [User: 22.5 ms, System: 5.9 ms]
Range (min … max): 26.7 ms … 33.3 ms 46 runs
Benchmark 2: update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs)
Time (mean ± σ): 23.8 ms ± 1.2 ms [User: 17.5 ms, System: 6.0 ms]
Range (min … max): 22.1 ms … 27.7 ms 56 runs
Summary
update-ref: delete ref (refcount = 100000, revision = b4/kn-speedup-packed-refs) ran
1.21 ± 0.09 times faster than update-ref: delete ref (refcount = 100000, revision = master)
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
Changes in v4:
- Modify the commit message to state the issue with the previous
approach.
- Modify the comment for `record_start` to remove ambiguity around its
setting.
- Remove `write_packed_entry_raw()` and inline the call to `fwrite()`.
- Link to v3: https://patch.msgid.link/20261006-kn-speedup-packed-refs-v3-1-a1c76b1df9e0@gmail.com
Changes in v3:
- Fixed a typo in the commit message.
- Link to v2: https://patch.msgid.link/20261002-kn-speedup-packed-refs-v2-1-2ae75772ebc1@gmail.com
Changes in v2:
- Instead of using the existing function, introduce a new
`write_packed_entry_raw()`.
- Modify the commit to also note that we lose sanitization.
- Link to v1: https://patch.msgid.link/20260930-kn-speedup-packed-refs-v1-1-111cd03d9b0e@gmail.com
---
refs/packed-backend.c | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a73fc6aca7..53511f2d91 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -879,6 +879,9 @@ struct packed_ref_iterator {
/* The current position in the snapshot's buffer: */
const char *pos;
+ /* The starting position of the current ref record. */
+ const char *record_start;
+
/* The end of the part of the buffer that will be iterated over: */
const char *eof;
@@ -933,6 +936,7 @@ static int next_record(struct packed_ref_iterator *iter)
if (iter->pos == iter->eof)
return ITER_DONE;
+ iter->record_start = iter->pos;
iter->base.ref.flags = REF_ISPACKED;
p = iter->pos;
@@ -1530,9 +1534,15 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re
}
if (cmp < 0) {
- /* Pass the old reference through. */
- if (write_packed_entry(out, iter->ref.name,
- iter->ref.oid, iter->ref.peeled_oid))
+ const struct packed_ref_iterator *packed_iter =
+ (const struct packed_ref_iterator *)iter;
+ size_t len = packed_iter->pos - packed_iter->record_start;
+
+ /*
+ * Skip any formatting and directly write to the packed-refs file
+ * when deleting references and writing the remaining refs verbatim.
+ */
+ if (fwrite(packed_iter->record_start, len, 1, out) != 1)
goto write_error;
if ((ok = ref_iterator_advance(iter)) != ITER_OK) {
---
base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
change-id: 20260930-kn-speedup-packed-refs-9868f5d0abe9
Thanks
- Karthik
next prev parent reply other threads:[~2026-10-07 11:20 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-30 15:15 [PATCH] packed-refs: use `fwrite()` when passing refs verbatim Karthik Nayak
2026-10-01 20:30 ` Toon Claes
2026-10-02 0:16 ` Karthik Nayak
2026-10-02 13:03 ` [PATCH v2] " Karthik Nayak
2026-10-05 18:34 ` Toon Claes
2026-10-06 9:36 ` Karthik Nayak
2026-10-06 9:18 ` [PATCH v3] " Karthik Nayak
2026-10-06 12:31 ` Patrick Steinhardt
2026-10-06 21:11 ` Karthik Nayak
2026-10-07 5:12 ` Patrick Steinhardt
2026-10-07 9:25 ` Karthik Nayak
2026-10-06 16:19 ` Junio C Hamano
2026-10-06 20:59 ` Karthik Nayak
2026-10-07 11:20 ` Karthik Nayak [this message]
2026-10-07 11:52 ` [PATCH v4] " Patrick Steinhardt
2026-10-08 11:41 ` Toon Claes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261007-kn-speedup-packed-refs-v4-1-79a411026596@gmail.com \
--to=karthik.188@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=ps@pks.im \
--cc=toon@iotcl.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox