Re: directory caching & negative file lookups?

[Trond Myklebust <trondmy@hammerspace.com>]

On Thu, 2022-09-01 at 14:32 +0100, Daire Byrne wrote:
> Hi,
> 
> So I have a bit of a newbie question (apologies) that came to me
> while
> debugging some code that was spamming our NFS servers with lookups
> for
> nonexistent files.
> 
> If we can cache directory entries (readdir) and even all their
> attributes (readdirplus) for some specified period of time (actimeo,
> nocto) on a client, then why can't we use that data to serve negative
> lookups for files in that directory too (if we so choose)?
> 
> There are probably very good reasons you always need to do a
> (negative) file lookup, like being able to read files recently
> created
> on another client (despite your local cache for that directory), but
> I'm just curious what the official reasons are. If we could choose to
> serve negative lookups using the directory entries cache for a
> read-only or unchanging filesystem, would that still be bad? We
> already choose to use nocto for some workloads...
> 
> In our case we see these kinds of heavy negative lookup workloads for
> network installed software (100 entries in PYTHONPATH is bad) and in
> buggy software (randomly generated filename lookups are really bad!).
> Of course, this overhead gets really bad as you add latency between
> the client and server.
> 
> Daire

man 5 nfs

Look for the section on the 'lookupcache=mode' mount option.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com