git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: shejialuo <shejialuo@gmail.com>
Cc: git@vger.kernel.org,  Jeff King <peff@peff.net>,
	 Patrick Steinhardt <ps@pks.im>
Subject: Re: [PATCH v4 3/3] packed-backend: mmap large "packed-refs" file during fsck
Date: Tue, 13 May 2025 09:51:20 -0700	[thread overview]
Message-ID: <xmqqcycch2t3.fsf@gitster.g> (raw)
In-Reply-To: <aCMoD-c_oHlu0c5c@ArchLinux> (shejialuo@gmail.com's message of "Tue, 13 May 2025 19:07:59 +0800")

shejialuo <shejialuo@gmail.com> writes:

> During fsck, we use "strbuf_read" to read the content of "packed-refs"
> without using mmap mechanism. This is a bad practice which would consume
> more memory than using mmap mechanism. Besides, as all code paths in
> "packed-backend.c" use this way, we should make "fsck" align with the
> current codebase.
>
> As we have introduced the helper function "allocate_snapshot_buffer", we
> could simple use this function to use mmap mechanism.

"could simple" -> "can simply".

> Suggested-by: Jeff King <peff@peff.net>
> Suggested-by: Patrick Steinhardt <ps@pks.im>
> Signed-off-by: shejialuo <shejialuo@gmail.com>
> ---
>  refs/packed-backend.c | 19 +++++++------------
>  1 file changed, 7 insertions(+), 12 deletions(-)

Nice loss of line count ;-)

> -	if (!st.st_size) {
> +	if (!allocate_snapshot_buffer(&snapshot, fd, &st)) {

It is a bit funny to see that a helper function that works at a much
higher conceptual level treat an empty file so specially (namely,
should it be different from a header-only packed-refs file?).  If I
were doing this refactoring in 2 & 3, I would probalby have made the
helper return "void", and have callers who do care about st.st_size
check that themselves.

But I'll let it pass.

> @@ -2121,21 +2121,16 @@ static int packed_fsck(struct ref_store *ref_store,
>  		goto cleanup;
>  	}
>  
> -	if (strbuf_read(&packed_ref_content, fd, 0) < 0) {
> -		ret = error_errno(_("unable to read '%s'"), refs->path);
> -		goto cleanup;
> -	}
> -
> -	ret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,
> -				      packed_ref_content.buf + packed_ref_content.len);
> +	ret = packed_fsck_ref_content(o, ref_store, &sorted, snapshot.start,
> +				      snapshot.eof);
>  	if (!ret && sorted)
> -		ret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,
> -					     packed_ref_content.buf + packed_ref_content.len);
> +		ret = packed_fsck_ref_sorted(o, ref_store, snapshot.start,
> +					     snapshot.eof);
>  
>  cleanup:
>  	if (fd >= 0)
>  		close(fd);
> -	strbuf_release(&packed_ref_content);
> +	clear_snapshot_buffer(&snapshot);
>  	return ret;
>  }

  reply	other threads:[~2025-05-13 16:51 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-05-06 16:39 [PATCH 0/4] align the behavior when opening "packed-refs" shejialuo
2025-05-06 16:41 ` [PATCH 1/4] packed-backend: skip checking consistency of empty packed-refs file shejialuo
2025-05-06 18:42   ` Junio C Hamano
2025-05-07 12:09     ` shejialuo
2025-05-06 19:14   ` Junio C Hamano
2025-05-07 12:10     ` shejialuo
2025-05-06 16:41 ` [PATCH 2/4] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-06 19:16   ` Junio C Hamano
2025-05-06 16:41 ` [PATCH 3/4] packed-backend: extract munmap operation for `MMAP_TEMPORARY` shejialuo
2025-05-06 18:52   ` Junio C Hamano
2025-05-06 22:17     ` Junio C Hamano
2025-05-07 12:21     ` shejialuo
2025-05-06 16:41 ` [PATCH 4/4] packed-backend: use mmap when opening large "packed-refs" file shejialuo
2025-05-06 19:00   ` Junio C Hamano
2025-05-06 22:18     ` Junio C Hamano
2025-05-07 12:34     ` shejialuo
2025-05-07 14:52 ` [PATCH v2 0/4] align the behavior when opening "packed-refs" shejialuo
2025-05-07 14:53   ` [PATCH v2 1/4] packed-backend: fsck should allow an empty "packed-refs" file shejialuo
2025-05-07 14:53   ` [PATCH v2 2/4] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-07 14:53   ` [PATCH v2 3/4] packed-backend: extract munmap operation for `MMAP_TEMPORARY` shejialuo
2025-05-08 19:57     ` Jeff King
2025-05-08 20:05       ` Junio C Hamano
2025-05-09 15:03         ` shejialuo
2025-05-07 14:54   ` [PATCH v2 4/4] packed-backend: mmap large "packed-refs" file during fsck shejialuo
2025-05-08 20:07     ` Jeff King
2025-05-09 15:21       ` shejialuo
2025-05-09 15:59         ` Jeff King
2025-05-09 16:40           ` shejialuo
2025-05-07 22:51   ` [PATCH v2 0/4] align the behavior when opening "packed-refs" Junio C Hamano
2025-05-08 20:08     ` Jeff King
2025-05-08 20:20       ` Junio C Hamano
2025-05-08 20:33         ` Jeff King
2025-05-09 15:26           ` shejialuo
2025-05-11 13:59   ` [PATCH v3 0/3] " shejialuo
2025-05-11 14:01     ` [PATCH v3 1/3] packed-backend: fsck should allow an empty "packed-refs" file shejialuo
2025-05-12  8:36       ` Patrick Steinhardt
2025-05-12 12:25         ` shejialuo
2025-05-12 14:39           ` Patrick Steinhardt
2025-05-12 15:56             ` Jeff King
2025-05-12 17:18               ` Junio C Hamano
2025-05-13  5:08                 ` Patrick Steinhardt
2025-05-13  7:06                   ` shejialuo
2025-05-11 14:01     ` [PATCH v3 2/3] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-12  8:37       ` Patrick Steinhardt
2025-05-12 10:35         ` shejialuo
2025-05-12 14:41           ` Patrick Steinhardt
2025-05-12 13:06       ` Jeff King
2025-05-13  6:55         ` shejialuo
2025-05-11 14:01     ` [PATCH v3 3/3] packed-backend: mmap large "packed-refs" file during fsck shejialuo
2025-05-12 13:08       ` Jeff King
2025-05-13 11:06     ` [PATCH v4 0/3] align the behavior when opening "packed-refs" shejialuo
2025-05-13 11:07       ` [PATCH v4 1/3] packed-backend: fsck should warn when "packed-refs" file is empty shejialuo
2025-05-13 16:30         ` Junio C Hamano
2025-05-14 12:51           ` shejialuo
2025-05-13 11:07       ` [PATCH v4 2/3] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-13 11:07       ` [PATCH v4 3/3] packed-backend: mmap large "packed-refs" file during fsck shejialuo
2025-05-13 16:51         ` Junio C Hamano [this message]
2025-05-14 13:05           ` shejialuo
2025-05-14 15:48       ` [PATCH v5 0/3] align the behavior when opening "packed-refs" shejialuo
2025-05-14 15:50         ` [PATCH v5 1/3] packed-backend: fsck should warn when "packed-refs" file is empty shejialuo
2025-05-14 15:50         ` [PATCH v5 2/3] packed-backend: extract snapshot allocation in `load_contents` shejialuo
2025-05-14 15:50         ` [PATCH v5 3/3] packed-backend: mmap large "packed-refs" file during fsck shejialuo
2025-05-15 12:57         ` [PATCH v5 0/3] align the behavior when opening "packed-refs" Junio C Hamano
2025-05-21 16:31         ` Junio C Hamano
2025-05-22  5:50           ` Jeff King
2025-05-23  9:40             ` Patrick Steinhardt
2025-05-23 15:58               ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqcycch2t3.fsf@gitster.g \
    --to=gitster@pobox.com \
    --cc=git@vger.kernel.org \
    --cc=peff@peff.net \
    --cc=ps@pks.im \
    --cc=shejialuo@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).