Re: [PATCH 0/2] NFS: limit use of ACCESS cache for negative responses

[Trond Myklebust <trondmy@hammerspace.com>]

On Sat, 2022-08-27 at 09:39 +1000, NeilBrown wrote:
> On Sat, 27 Aug 2022, Trond Myklebust wrote:
> > On Fri, 2022-08-26 at 10:59 -0400, Benjamin Coddington wrote:
> > > On 16 May 2022, at 21:36, Trond Myklebust wrote:
> > > > So until you have a different solution that doesn't impact the
> > > > client's
> > > > ability to cache permissions, then the answer is going to be
> > > > "no"
> > > > to
> > > > these patches.
> > > 
> > > Hi Trond,
> > > 
> > > We have some folks negatively impacted by this issue as well. 
> > > Are
> > > you
> > > willing to consider this via a mount option?
> > > 
> > > Ben
> > > 
> > 
> > I don't see how that answers my concern.
> 
> Could you please spell out again what your concerns are?  I still
> don't
> understand. 
> The only performance impact is when a permission test fails.  In what
> circumstance is permission failure expected on a fast-path?
> 

You're treating the problem as if it were a timeout issue, when clearly
it has nothing at all to do with timeouts. There is no problem of
'group membership changes on a regular basis' to be solved.

The problem to be solved is that on the very rare occasion when a group
membership does change, then the server and the client may update their
view of that membership at completely different times. In the
particular case when the client updates its view of the group
membership before the server does, then the access cache is polluted,
and there is no remedy.

So my concerns are around the mismatch of problem and solution. I see
multiple issues.

   1. Your timeouts are per inode. That means that if inode A sees the
      problem being solved, then there is no guarantee that inode B
      sees the same problem as being solved (and the converse is true
      as well).
   2. There is no quick on-the-spot solution. If your admin updates the
      group membership, then you are only guaranteed that the client
      and server are in sync once the server has picked up the solution
      (however you arrange that), and the client cache has expired.
      IOW: your only solution is to wait 1 client cache expiration
      period after the server is known to be fixed (or to reboot the
      client).
   3. There is no solution at all for the positive cache case. If your
      sysadmin is trying to revoke an access due to a group membership
      change, their only solution is to reboot the client.
   4. You are tying the access cache timeout to the completely
      unrelated 'acregmin' and 'acdirmin' values. Not only does that
      mean that the default values for regular files are extremely
      small (3 seconds), meaning that we have to refresh extremely
      often. However it also means that you have to explain why
      directories behave differently (longer default timeouts) despite
      the fact that the group membership changed at exactly the same
      time for both types of object.
         1. Bonus points for explaining why our default values are designed
            for a group membership that changes every 3 seconds.
   5. 'noac' suddenly now turns off access caching, but only for
      negative cached values.


-- 
Trond Myklebust Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com