Re: directory caching & negative file lookups?

[Trond Myklebust <trondmy@hammerspace.com>]

On Fri, 2024-04-05 at 15:47 +0100, Daire Byrne wrote:
> Apologies for dragging up an old thread, but I've had to tackle
> wayward negative lookup storms again and I have obviously half
> forgotten what I learned in this thread last time (even after
> re-reading it!).
> 
> Can I just ask if I understand correctly and that there was an
> intention a long time ago to be able to serve negative dentries from
> a
> "complete" READDIRPLUS result?
> 
> https://www.cs.helsinki.fi/linux/linux-kernel/2002-30/0108.html
> 
> So if we did a readdirplus on a directory then immediately fired
> random non existent lookups at the directory, it could be served from
> the readdirplus result? i.e. not in readdir result, then return
> ENOENT
> without needing to ask server?
> 
> But that is not the case today because you can't track the
> "completeness" of a READDIRPLUS result for a directory over time (in
> page cache)? Or is it all due to needing to deal with case
> insensitive
> filesystems (which I would think effects positive lookups too)?
> 
> I did try to decipher the v6.6 fs/nfs/dir.c READDIR bits but I
> quickly
> got lost...
> 
> Cheers,
> 
> Daire

If the question is whether the client trusts that a READDIR call to the
server returns all the names that can be successfully looked up, then
the answer is "no".
It's not even a question of case sensitivity. There are plenty of
servers out there that will allow you to look up names that won't ever
appear in the results of a READDIR (or READDIRPLUS) call. Having a
hidden ".snapshot" directory is, for instance, a popular way to present
snapshots.

So no, we're not ever going to implement any negative dentry cache
scheme that relies on READDIR/READDIRPLUS.
-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com