Negative dentries on Linux SMB filesystems

Negative dentries on Linux SMB filesystems

[Shyam Prasad N]

Hi,

Negative dentry is a VFS concept where it maintains a cache of
dentries even for files that do not exist. This can come in handy for
situations where they are looked up. For example, many applications
first check the existence of a file before creating it.

Ideally, negative dentries should allow a filename lookup to happen
entirely from the dentry cache if the lookup had happened once
already. But I noticed that the SMB client goes to the server every
time we do a stat of a file that does not exist.

I investigated this more and it looks like vfs_getattr does make use
of negative dentry, but the revalidate always comes to
cifs_d_revalidate even for negative dentries. And we do not have the
code necessary to deal with it. We do use d_really_is_positive before
we do the dentry validation, but it looks like that comes to us as
success, even in case of non-existent dentries. Is this expected?

I compared the revalidate code on the NFS client, and it does look
like nfs_neg_need_reval deals with this situation based on some
heuristics.

-- 
Regards,
Shyam

Re: Negative dentries on Linux SMB filesystems

[Paulo Alcantara]

Shyam Prasad N <nspmangalore@gmail.com> writes:

> Ideally, negative dentries should allow a filename lookup to happen
> entirely from the dentry cache if the lookup had happened once
> already. But I noticed that the SMB client goes to the server every
> time we do a stat of a file that does not exist.

This is a network filesystem.  If the last lookup ended up with a
negative dentry in dcache, that doesn't mean the file won't exist the
next time we look it up again.  The file could have been created by a
different client, so we need to query it on server.

> I investigated this more and it looks like vfs_getattr does make use
> of negative dentry, but the revalidate always comes to
> cifs_d_revalidate even for negative dentries. And we do not have the
> code necessary to deal with it.

I think we do.  Check the places where we return 0 from
cifs_d_revalidate(), meaning that the dentry will need to be looked up
again and hopefully instantiated (e.g. file was created on server).

> We do use d_really_is_positive before we do the dentry validation, but
> it looks like that comes to us as success, even in case of
> non-existent dentries. Is this expected?

I don't think so.

Al, am I missing someting?

Re: Negative dentries on Linux SMB filesystems

[Shyam Prasad N]

Hi Paulo,

Thanks for your replies.

On Mon, Jan 13, 2025 at 8:55 PM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Shyam Prasad N <nspmangalore@gmail.com> writes:
>
> > Ideally, negative dentries should allow a filename lookup to happen
> > entirely from the dentry cache if the lookup had happened once
> > already. But I noticed that the SMB client goes to the server every
> > time we do a stat of a file that does not exist.
>
> This is a network filesystem.  If the last lookup ended up with a
> negative dentry in dcache, that doesn't mean the file won't exist the
> next time we look it up again.  The file could have been created by a
> different client, so we need to query it on server.

I agree. But we do have tools to trade performance for accuracy using
parameters like actimeo/acdirmax/acregmax.
So we can avoid going to the server each time if it's within some interval.
If the server gives us dir leases, we can be sure that the dentries
have not changed without us knowing. So we can definitely cache the
negative dentries till as long as we have the lease.

>
> > I investigated this more and it looks like vfs_getattr does make use
> > of negative dentry, but the revalidate always comes to
> > cifs_d_revalidate even for negative dentries. And we do not have the
> > code necessary to deal with it.
>
> I think we do.  Check the places where we return 0 from
> cifs_d_revalidate(), meaning that the dentry will need to be looked up
> again and hopefully instantiated (e.g. file was created on server).

I was thinking so too. But I could see that a revalidate loop that
repeats every second is calling cifs_revalidate_dentry in
cifs_d_revalidate, which can only happen if d_really_is_positive
returns success.
        if (d_really_is_positive(direntry)) {
                inode = d_inode(direntry);
                if ((flags & LOOKUP_REVAL) && !CIFS_CACHE_READ(CIFS_I(inode)))
                        CIFS_I(inode)->time = 0; /* force reval */

                rc = cifs_revalidate_dentry(direntry);    <<<<<<<<<<<<<<<<<<<
                if (rc) {
                        cifs_dbg(FYI, "cifs_revalidate_dentry failed
with rc=%d", rc);
....
                 }
                else {
....
                        return 1;
                }

>
> > We do use d_really_is_positive before we do the dentry validation, but
> > it looks like that comes to us as success, even in case of
> > non-existent dentries. Is this expected?
>
> I don't think so.
>
> Al, am I missing someting?

-- 
Regards,
Shyam

Re: Negative dentries on Linux SMB filesystems

[Paulo Alcantara]

Shyam Prasad N <nspmangalore@gmail.com> writes:

> Hi Paulo,
>
> Thanks for your replies.
>
> On Mon, Jan 13, 2025 at 8:55 PM Paulo Alcantara <pc@manguebit.com> wrote:
>>
>> Shyam Prasad N <nspmangalore@gmail.com> writes:
>>
>> > Ideally, negative dentries should allow a filename lookup to happen
>> > entirely from the dentry cache if the lookup had happened once
>> > already. But I noticed that the SMB client goes to the server every
>> > time we do a stat of a file that does not exist.
>>
>> This is a network filesystem.  If the last lookup ended up with a
>> negative dentry in dcache, that doesn't mean the file won't exist the
>> next time we look it up again.  The file could have been created by a
>> different client, so we need to query it on server.
>
> I agree. But we do have tools to trade performance for accuracy using
> parameters like actimeo/acdirmax/acregmax.

Do you mean using these parameters for negative dentries?  These are
used for caching file attributes of files and directories, which means
they are all positive dentries.

> So we can avoid going to the server each time if it's within some interval.
> If the server gives us dir leases, we can be sure that the dentries
> have not changed without us knowing. So we can definitely cache the
> negative dentries till as long as we have the lease.

Yes, that could be done with directory leases.

Note that negative dentries are also cached when @lookupCacheEnabled is
set.

Re: Negative dentries on Linux SMB filesystems

[Shyam Prasad N]

On Mon, Jan 13, 2025 at 9:59 PM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Shyam Prasad N <nspmangalore@gmail.com> writes:
>
> > Hi Paulo,
> >
> > Thanks for your replies.
> >
> > On Mon, Jan 13, 2025 at 8:55 PM Paulo Alcantara <pc@manguebit.com> wrote:
> >>
> >> Shyam Prasad N <nspmangalore@gmail.com> writes:
> >>
> >> > Ideally, negative dentries should allow a filename lookup to happen
> >> > entirely from the dentry cache if the lookup had happened once
> >> > already. But I noticed that the SMB client goes to the server every
> >> > time we do a stat of a file that does not exist.
> >>
> >> This is a network filesystem.  If the last lookup ended up with a
> >> negative dentry in dcache, that doesn't mean the file won't exist the
> >> next time we look it up again.  The file could have been created by a
> >> different client, so we need to query it on server.
> >
> > I agree. But we do have tools to trade performance for accuracy using
> > parameters like actimeo/acdirmax/acregmax.
>
> Do you mean using these parameters for negative dentries?  These are
> used for caching file attributes of files and directories, which means
> they are all positive dentries.
Yes. Precisely.

>
> > So we can avoid going to the server each time if it's within some interval.
> > If the server gives us dir leases, we can be sure that the dentries
> > have not changed without us knowing. So we can definitely cache the
> > negative dentries till as long as we have the lease.
>
> Yes, that could be done with directory leases.
>
> Note that negative dentries are also cached when @lookupCacheEnabled is
> set.
Ah ok. I need to check about this.

My point was that today, if we keep doing a stat for a non existent
file in 1 second loop, each call translates to a server QueryInfo,
unless dir leases are enabled.


-- 
Regards,
Shyam