All of lore.kernel.org
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: Jeff King <peff@peff.net>
Cc: "brian m. carlson" <sandals@crustytoothpaste.net>,
	Junio C Hamano <gitster@pobox.com>,
	git@vger.kernel.org
Subject: Re: [PATCH] fsck: do not loop infinitely when processing packs
Date: Mon, 23 Feb 2026 09:46:00 +0100	[thread overview]
Message-ID: <aZwTyLMWbcXWnYhQ@pks.im> (raw)
In-Reply-To: <20260223071215.GA136463@coredump.intra.peff.net>

On Mon, Feb 23, 2026 at 02:12:15AM -0500, Jeff King wrote:
> On Sun, Feb 22, 2026 at 11:07:41PM +0000, brian m. carlson wrote:
> 
> > I noticed that the code here seems to have come in with the 2.53 cycle,
> > so we may want to cherry-pick it to `maint` at some point if it seems
> > like the problem occurs often.  From what I can tell, it only occurs
> > when one explicitly invokes `git fsck`[0] and not on transfer, so it
> > shouldn't cause a DoS against server implementations.
> > 
> > Of course, we should wait for Patrick, who authored this code, to chime
> > in and lend his expertise here.  I must admit I'm not very familiar with
> > this area, although I had recently seen the MRU code when working on
> > pack index v3 (and then I thought, "is this actually the problem?").
> 
> The problem seems to bisect to c31bad4f7d (packfile: track packs via the
> MRU list exclusively, 2025-10-30), which is not terribly surprising, as
> it was one of the known risks of collapsing the two lists into one.
> 
> Your solution is using the tool provided by that commit for its edge
> case:
> 
>     Note that there is one important edge case: `for_each_packed_object()`
>     uses the MRU list to iterate through packs, and then it lists each
>     object in those packs. This would have the effect that we now sort the
>     current pack towards the front, thus modifying the list of packfiles we
>     are iterating over, with the consequence that we'll see an infinite
>     loop. This edge case is worked around by introducing a new field that
>     allows us to skip updating the MRU.
> 
> So in that sense it is the right thing. But it really makes me wonder if
> we are going back to keeping two lists (one MRU and one in some stable
> order). Or at the very least providing _some_ iteration method that is
> guaranteed to be stable (whether a linked list or a function), so that
> iterating code is not subject to this subtle dependency by default.
> 
> Having to identify each potential spot and set a "btw, don't switch the
> pack list order!" flag seems error-prone. And also loses efficiency when
> you are iterating a pack and accessing objects in it (since we can't
> push that pack to the front of the MRU then, even though we'd expect
> there to be high locality with our iteration).

As pointed out in [1] the root cause is actually something different,
and we merely expose this now with the MRU-based iteration. But I
wouldn't mind if we eventually switched back to maintaining two lists,
or finding a different way for how to maintain the iteration order.

Patrick

[1]: <aZwTPfmyrFp-QAPq@pks.im>

  reply	other threads:[~2026-02-23  8:46 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-02-22 18:37 [PATCH] fsck: do not loop infinitely when processing packs brian m. carlson
2026-02-22 22:39 ` Junio C Hamano
2026-02-22 23:07   ` brian m. carlson
2026-02-23  7:12     ` Jeff King
2026-02-23  8:46       ` Patrick Steinhardt [this message]
2026-02-23  9:25         ` Jeff King
2026-02-23  9:36           ` Patrick Steinhardt
2026-02-23  9:46             ` Jeff King
2026-02-23 15:49       ` Junio C Hamano
2026-02-23  8:43 ` Patrick Steinhardt
2026-02-23  9:27   ` Jeff King
2026-02-23  9:53   ` Patrick Steinhardt
2026-02-24 22:23   ` brian m. carlson
2026-02-24 22:32     ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aZwTyLMWbcXWnYhQ@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    --cc=sandals@crustytoothpaste.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.