All of lore.kernel.org
 help / color / mirror / Atom feed
From: Nikolaus Rath <Nikolaus@rath.org>
To: Miklos Szeredi <miklos@szeredi.hu>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	fuse-devel <fuse-devel@lists.sourceforge.net>
Subject: [fuse] Speeding up readdir()
Date: Thu, 06 Dec 2018 20:00:57 +0000	[thread overview]
Message-ID: <87va46pgqu.fsf@vostro.rath.org> (raw)

Hello,

I am trying to improve the performance of readdir() requests. I have a
client application that issues a lot of readdir() requests, and a FUSE
filesystem that makes extensive use of kernel caching for inode entry
attributes because retrieving the attributes from the backend is
relatively expensive.

Unfortunately, it seems to me that currently there is no way to avoid
having to retrieve the attributes from the backend for every entry that
is returned by readdir - on every call:

If I am using readdirplus, I have to include the full attributes even if
the kernel already has them cached.

If I disable readdirplus, I can return just the entry name and its inode
- but I believe because this doesn't result in a lookup count increase
of the inode, the kernel can't match this with the existing cached data
for the inode (is that correct?) and I'm getting a separate lookup()
request for each entry that I've returned.


I could implement a readdir cache in the filesystem, but that means I
have to take care of cache invalidation and I'm basically wasting
memory.

Is there a reason why readdirplus() couldn't return just the name and
inode, together with a special flag that tells the kernel to "just use
the attributes that are already cached"?


Best,
-Nikolaus

-- 
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

             reply	other threads:[~2018-12-06 20:07 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-06 20:00 Nikolaus Rath [this message]
2018-12-07  9:21 ` [fuse] Speeding up readdir() Miklos Szeredi
2018-12-07 12:55   ` [fuse-devel] " Nikolaus Rath
2018-12-07 13:04     ` Miklos Szeredi
2018-12-07 13:13       ` Nikolaus Rath
2018-12-07 13:18         ` Miklos Szeredi
2018-12-07 13:39       ` Nikolaus Rath
2018-12-10  9:31         ` Miklos Szeredi
2018-12-07 13:38   ` Nikolaus Rath

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87va46pgqu.fsf@vostro.rath.org \
    --to=nikolaus@rath.org \
    --cc=fuse-devel@lists.sourceforge.net \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.