Re: 2.6.38.6 - state manager constantly respawns

[Bryan Schumaker <bjschuma@netapp.com>]

On 05/20/2011 02:47 PM, Dr. J. Bruce Fields wrote:
> On Fri, May 20, 2011 at 01:52:43PM -0400, Trond Myklebust wrote:
>> On Fri, 2011-05-20 at 13:26 -0400, Dr. J. Bruce Fields wrote: 
>>> On Fri, May 20, 2011 at 09:20:47AM -0700, Harry Edmon wrote:
>>>> On 05/16/11 13:53, Dr. J. Bruce Fields wrote:
>>>>> Hm, so the renews all have clid 465ccc4d09000000, and the reads all have
>>>>> a stateid (0, 465ccc4dc24c0a0000000000).
>>>>>
>>>>> So the first 4 bytes matching just tells me both were handed out by the
>>>>> same server instance (so there was no server reboot in between); there's
>>>>> no way for me to tell whether they really belong to the same client.
>>>>>
>>>>> The server does assume that any stateid from the current server instance
>>>>> that no longer exists in its table is expired.  I believe that's
>>>>> correct, given a correctly functioning client, but perhaps I'm missing a
>>>>> case.
>>>>>
>>>>> --b.
>>>> I am very appreciative of the quick initial comments I receive from
>>>> all of you on my NFS problem.   I notice that there has been silence
>>>> on the problem since the 16th, so I assume that either this is a
>>>> hard bug to track down or you have been busy with higher priority
>>>> tasks.  Is there anything I can do to help develop a solution to
>>>> this problem?
>>>
>>> Well, the only candidate explanation for the problem is that my
>>> assumption--that any time the server gets a stateid from the current
>>> boot instance that it doesn't recognize as an active stateid, it is safe
>>> for the server to return EXPIRED--is wrong.
>>>
>>> I don't immediately see why it's wrong, and based on the silence nobody
>>> else does either, but I'm not 100% convinced I'm right either.
>>>
>>> So one approach might be to add server code that makes a better effort
>>> to return EXPIRED only when we're sure it's a stateid from an expired
>>> client, and see if that solves your problem.
>>>
>>> Remind me, did you have an easy way to reproduce your problem?
>>
>> My silence is simply because I'm mystified as to how this can happen.
> 
> So since the client's sending it with a READ, the client thinks that the
> stateid is still a valid open, lock, or delegation stateid, while the
> server thinks it's not.  Hm.

I found this bug when I used "forget all locks" in the fault injection code I recently posted.  Trond's fix works for me.

- Bryan

> 
> --b.
> 
>> Patching for it is trivial (see below).
>>
>> When the server tells us that our lease is expired, the normal behaviour
>> for the client is to re-establish the lease, and then proceed to recover
>> all known stateids. I don't see how we can 'miss' a stateid that then
>> needs to be recovered afterwards...
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html