Re: 2.6.38.6 - state manager constantly respawns

["Dr. J. Bruce Fields" <bfields@fieldses.org>]

On Fri, May 20, 2011 at 01:52:43PM -0400, Trond Myklebust wrote:
> On Fri, 2011-05-20 at 13:26 -0400, Dr. J. Bruce Fields wrote: 
> > On Fri, May 20, 2011 at 09:20:47AM -0700, Harry Edmon wrote:
> > > On 05/16/11 13:53, Dr. J. Bruce Fields wrote:
> > > >Hm, so the renews all have clid 465ccc4d09000000, and the reads all have
> > > >a stateid (0, 465ccc4dc24c0a0000000000).
> > > >
> > > >So the first 4 bytes matching just tells me both were handed out by the
> > > >same server instance (so there was no server reboot in between); there's
> > > >no way for me to tell whether they really belong to the same client.
> > > >
> > > >The server does assume that any stateid from the current server instance
> > > >that no longer exists in its table is expired.  I believe that's
> > > >correct, given a correctly functioning client, but perhaps I'm missing a
> > > >case.
> > > >
> > > >--b.
> > > I am very appreciative of the quick initial comments I receive from
> > > all of you on my NFS problem.   I notice that there has been silence
> > > on the problem since the 16th, so I assume that either this is a
> > > hard bug to track down or you have been busy with higher priority
> > > tasks.  Is there anything I can do to help develop a solution to
> > > this problem?
> > 
> > Well, the only candidate explanation for the problem is that my
> > assumption--that any time the server gets a stateid from the current
> > boot instance that it doesn't recognize as an active stateid, it is safe
> > for the server to return EXPIRED--is wrong.
> > 
> > I don't immediately see why it's wrong, and based on the silence nobody
> > else does either, but I'm not 100% convinced I'm right either.
> > 
> > So one approach might be to add server code that makes a better effort
> > to return EXPIRED only when we're sure it's a stateid from an expired
> > client, and see if that solves your problem.
> > 
> > Remind me, did you have an easy way to reproduce your problem?
> 
> My silence is simply because I'm mystified as to how this can happen.

So since the client's sending it with a READ, the client thinks that the
stateid is still a valid open, lock, or delegation stateid, while the
server thinks it's not.  Hm.

--b.

> Patching for it is trivial (see below).
> 
> When the server tells us that our lease is expired, the normal behaviour
> for the client is to re-establish the lease, and then proceed to recover
> all known stateids. I don't see how we can 'miss' a stateid that then
> needs to be recovered afterwards...