All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Theodore Tso" <tytso@mit.edu>
To: NeilBrown <neil@brown.name>
Cc: Matthew Wilcox <willy@infradead.org>,
	fstests@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	Chuck Lever <cel@kernel.org>
Subject: Re: [PATCH] fstests: generic: Add test of seek in directories
Date: Sun, 30 Aug 2026 21:37:33 -0400	[thread overview]
Message-ID: <apTQqcuR7bI6qY54@mit.edu> (raw)
In-Reply-To: <178812852153.3510150.11411436066299613954@noble.neil.brown.name>

On Mon, Aug 31, 2026 at 08:22:01AM -0500, NeilBrown wrote:
> Given that the requirements predates the htree design, isn't this "pain
> and performance hit" a deliberate design choice?  Or maybe an
> unfortunate design omission?  Such things can be fixed.

The requirements assumed that the directory was a linear structure,
much like the V7 and BSD FFS structure.  That's why
telldir()/seekdir() interface used a 32-biit cookie --- e.g., a seek
offset.  This doesn't deal well with a b-tree or equivalent structure
where when you insert an item into the tree, the tree might require a
node to be split or the tree to be rebalanced.

> ext4 already has two indexing schemes - indirect and extent-based.  How
> hard would it be to add a third that effectively moved the current
> directory index blocks out of the "file" and into the "file-index"?
> Obviously the 32 block address in dx_entry would need to become a 48 bit
> device address etc, and there are bound to be details I've glossed over,
> but this approach would amortise all metadata access of a lot data
> access, and would allow read-ahead.

The primary detail that you've glossed over is that the tree would
have to be a B-tree.  Ext4 currently uses a b+tree which means that we
don't need to store the file name (key) in the index nodes.  The index
node in the htree is a fixed size, and compact, and this allows for a
much greater fanout (340 nodes in each node).  This makes the tree
much shallower, and reduces the number of seeks when doing lookups.

Switching to a b-tree would almost double the size of the overhead of
the tree, and the tree would be deeper, and thus require more seeks
when doing a lookup.

So it's a tradeoff, and it depends on whether you think readdir or
directory lookups are more frequent.

Cheers,

						- Ted

  reply	other threads:[~2026-08-31  1:44 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-27 23:36 fstests: Verify behaviour of "seek" in a directory NeilBrown
2026-08-27 23:36 ` [PATCH] fstests: generic: Add test of seek in directories NeilBrown
2026-08-28  4:13   ` Matthew Wilcox
2026-08-28 11:12     ` NeilBrown
2026-08-28 14:55       ` Theodore Tso
2026-08-30 22:22         ` NeilBrown
2026-08-31  1:37           ` Theodore Tso [this message]
2026-08-31 13:16       ` Christoph Hellwig
2026-09-01  1:05         ` NeilBrown
2026-09-01  9:18           ` Christoph Hellwig
2026-08-30 23:13   ` Chuck Lever
2026-08-31 12:59   ` Christoph Hellwig
2026-08-31 22:47     ` NeilBrown
2026-09-01  9:09       ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=apTQqcuR7bI6qY54@mit.edu \
    --to=tytso@mit.edu \
    --cc=cel@kernel.org \
    --cc=fstests@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=neil@brown.name \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.