From: shejialuo <shejialuo@gmail.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>,
Patrick Steinhardt <ps@pks.im>
Subject: [PATCH v3 3/3] packed-backend: mmap large "packed-refs" file during fsck
Date: Sun, 11 May 2025 22:01:59 +0800 [thread overview]
Message-ID: <aCCt1zJ2yviOz--l@ArchLinux> (raw)
In-Reply-To: <aCCtQDnWII-knmEc@ArchLinux>
During fsck, we use "strbuf_read" to read the content of "packed-refs"
without using mmap mechanism. This is a bad practice which would consume
more memory than using mmap mechanism. Besides, as all code paths in
"packed-backend.c" use this way, we should make "fsck" align with the
current codebase.
As we have introduced the helper function "allocate_snapshot_buffer", we
could simple use this function to use mmap mechanism.
Suggested-by: Jeff King <peff@peff.net>
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: shejialuo <shejialuo@gmail.com>
---
refs/packed-backend.c | 19 +++++++------------
1 file changed, 7 insertions(+), 12 deletions(-)
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index e582227772..85f5a45160 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -2069,7 +2069,7 @@ static int packed_fsck(struct ref_store *ref_store,
{
struct packed_ref_store *refs = packed_downcast(ref_store,
REF_STORE_READ, "fsck");
- struct strbuf packed_ref_content = STRBUF_INIT;
+ struct snapshot snapshot = { 0 };
unsigned int sorted = 0;
struct stat st;
int ret = 0;
@@ -2113,24 +2113,19 @@ static int packed_fsck(struct ref_store *ref_store,
goto cleanup;
}
- if (!st.st_size)
+ if (!allocate_snapshot_buffer(&snapshot, fd, &st))
goto cleanup;
- if (strbuf_read(&packed_ref_content, fd, 0) < 0) {
- ret = error_errno(_("unable to read '%s'"), refs->path);
- goto cleanup;
- }
-
- ret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,
- packed_ref_content.buf + packed_ref_content.len);
+ ret = packed_fsck_ref_content(o, ref_store, &sorted, snapshot.start,
+ snapshot.eof);
if (!ret && sorted)
- ret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,
- packed_ref_content.buf + packed_ref_content.len);
+ ret = packed_fsck_ref_sorted(o, ref_store, snapshot.start,
+ snapshot.eof);
cleanup:
if (fd >= 0)
close(fd);
- strbuf_release(&packed_ref_content);
+ clear_snapshot_buffer(&snapshot);
return ret;
}
--
2.49.0
next prev parent reply other threads:[~2025-05-11 14:02 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-05-06 16:39 [PATCH 0/4] align the behavior when opening "packed-refs" shejialuo
2025-05-06 16:41 ` [PATCH 1/4] packed-backend: skip checking consistency of empty packed-refs file shejialuo
2025-05-06 18:42 ` Junio C Hamano
2025-05-07 12:09 ` shejialuo
2025-05-06 19:14 ` Junio C Hamano
2025-05-07 12:10 ` shejialuo
2025-05-06 16:41 ` [PATCH 2/4] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-06 19:16 ` Junio C Hamano
2025-05-06 16:41 ` [PATCH 3/4] packed-backend: extract munmap operation for `MMAP_TEMPORARY` shejialuo
2025-05-06 18:52 ` Junio C Hamano
2025-05-06 22:17 ` Junio C Hamano
2025-05-07 12:21 ` shejialuo
2025-05-06 16:41 ` [PATCH 4/4] packed-backend: use mmap when opening large "packed-refs" file shejialuo
2025-05-06 19:00 ` Junio C Hamano
2025-05-06 22:18 ` Junio C Hamano
2025-05-07 12:34 ` shejialuo
2025-05-07 14:52 ` [PATCH v2 0/4] align the behavior when opening "packed-refs" shejialuo
2025-05-07 14:53 ` [PATCH v2 1/4] packed-backend: fsck should allow an empty "packed-refs" file shejialuo
2025-05-07 14:53 ` [PATCH v2 2/4] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-07 14:53 ` [PATCH v2 3/4] packed-backend: extract munmap operation for `MMAP_TEMPORARY` shejialuo
2025-05-08 19:57 ` Jeff King
2025-05-08 20:05 ` Junio C Hamano
2025-05-09 15:03 ` shejialuo
2025-05-07 14:54 ` [PATCH v2 4/4] packed-backend: mmap large "packed-refs" file during fsck shejialuo
2025-05-08 20:07 ` Jeff King
2025-05-09 15:21 ` shejialuo
2025-05-09 15:59 ` Jeff King
2025-05-09 16:40 ` shejialuo
2025-05-07 22:51 ` [PATCH v2 0/4] align the behavior when opening "packed-refs" Junio C Hamano
2025-05-08 20:08 ` Jeff King
2025-05-08 20:20 ` Junio C Hamano
2025-05-08 20:33 ` Jeff King
2025-05-09 15:26 ` shejialuo
2025-05-11 13:59 ` [PATCH v3 0/3] " shejialuo
2025-05-11 14:01 ` [PATCH v3 1/3] packed-backend: fsck should allow an empty "packed-refs" file shejialuo
2025-05-12 8:36 ` Patrick Steinhardt
2025-05-12 12:25 ` shejialuo
2025-05-12 14:39 ` Patrick Steinhardt
2025-05-12 15:56 ` Jeff King
2025-05-12 17:18 ` Junio C Hamano
2025-05-13 5:08 ` Patrick Steinhardt
2025-05-13 7:06 ` shejialuo
2025-05-11 14:01 ` [PATCH v3 2/3] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-12 8:37 ` Patrick Steinhardt
2025-05-12 10:35 ` shejialuo
2025-05-12 14:41 ` Patrick Steinhardt
2025-05-12 13:06 ` Jeff King
2025-05-13 6:55 ` shejialuo
2025-05-11 14:01 ` shejialuo [this message]
2025-05-12 13:08 ` [PATCH v3 3/3] packed-backend: mmap large "packed-refs" file during fsck Jeff King
2025-05-13 11:06 ` [PATCH v4 0/3] align the behavior when opening "packed-refs" shejialuo
2025-05-13 11:07 ` [PATCH v4 1/3] packed-backend: fsck should warn when "packed-refs" file is empty shejialuo
2025-05-13 16:30 ` Junio C Hamano
2025-05-14 12:51 ` shejialuo
2025-05-13 11:07 ` [PATCH v4 2/3] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-13 11:07 ` [PATCH v4 3/3] packed-backend: mmap large "packed-refs" file during fsck shejialuo
2025-05-13 16:51 ` Junio C Hamano
2025-05-14 13:05 ` shejialuo
2025-05-14 15:48 ` [PATCH v5 0/3] align the behavior when opening "packed-refs" shejialuo
2025-05-14 15:50 ` [PATCH v5 1/3] packed-backend: fsck should warn when "packed-refs" file is empty shejialuo
2025-05-14 15:50 ` [PATCH v5 2/3] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-14 15:50 ` [PATCH v5 3/3] packed-backend: mmap large "packed-refs" file during fsck shejialuo
2025-05-15 12:57 ` [PATCH v5 0/3] align the behavior when opening "packed-refs" Junio C Hamano
2025-05-21 16:31 ` Junio C Hamano
2025-05-22 5:50 ` Jeff King
2025-05-23 9:40 ` Patrick Steinhardt
2025-05-23 15:58 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aCCt1zJ2yviOz--l@ArchLinux \
--to=shejialuo@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).