Re: NFS client behavior on WRITE_DELEGATION

Soumya Koduri
2018-12-07T15:07:03Z

On 12/6/18 8:40 PM, Soumya Koduri wrote:
> Hi Anna,
>
> Thanks a lot for the response. To understand client behavior under
> various scenarios, I started with a simple "C" program which does below
> operations in a loop on a nfsv4.1 mount point-
>
> while(..)
> {
> OPEN,
> LOCK,
> READ/WRITE,
> UNLCK,
> CLOSE
> }
>
> While examining the pkt traces, I observed that even when granted
> WRITE_DELEGATION, there are WRITEs (though asynchronous & followed by
> COMMIT) sent to the server for each of those iterations. So could it be
> because of the kernel flushing out the data?
>
> Also I noticed that the behavior of WRITE_DELEGATION granted for
> OPEN(O_RDWR) is different when compared to the one granted to
> OPEN(O_WRONLY)
>
> 1) When the file is opened with OPEN4_SHARE_ACCESS_WRITE and a
> OPEN_DELEGATE_WRITE delegation is granted, subsequent OPENs are avoided
> and not sent to the server as expected, whereas
> If the file is opened with OPEN4_SHARE_ACCESS_BOTH and granted
> OPEN_DELEGATE_WRITE, the next open is sent to server resulting in server
> recalling the delegation it granted to the same client.
>
> 2) And to that recall request, client responded back with EBADHANDLE error.
>
> This doesn't seem right. Could you please confirm.
>
> (Maybe the server also can avoid conflicting the OPENfrom the same
> client to which it granted the delegation)

Apologies. This seem to have resulted from a server-side issue. On
further debugging, I found that server rejected READ op with
ERR_BAD_STATEID if there is WRITE_delegation granted to that file. This
may have led nfs client to discard the delegation granted and send
subsequent OPEN to server.

I think the server should allow READs if the access is
OPEN4_SHARE_ACCESS_BOTH and its the same client which holds the write
delegation. Will send out a patch to fix it.

So the only question I have for now is if/how client can save WRITEs
from being sent to the server while holding write delegation. Please
share your thoughts.

Thanks,
Soumya

>
> Thanks,
> Soumya
>
> On 12/6/18 1:52 AM, Anna Schumaker wrote:
>> Hi Soumya,
>>
>> What conditions are you seeing the writes?  It could be the kernel
>> trying to flush out data to free up memory for other processes.
>>
>> I don't think preemptively writing back data is necessarily a bad
>> thing though, even with a write delegation.  If the server decides to
>> recall the delegation, we wouldn't keep it waiting as long while we
>> write back local data.  Additionally, if the client crashes then any
>> updates might have already had a chance to sync to the server.
>>
>> I hope this helps!
>> Anna
>>
>> On Mon, Dec 3, 2018 at 2:55 AM Soumya Koduri <skoduri@redhat.com
>> <mailto:skoduri@redhat.com>> wrote:
>>
>>     Hi,
>>
>>     In our NFSv4.1 Delegations testing we observed that NFS client sends
>>     WRITE requests to the server even when granted WRITE_DELEGATION. I
>> see
>>     that only OPEN/CLOSE and LOCK operations are served locally by the
>>     client.
>>
>>     Please confirm if this behavior is expected and if yes, can it be
>>     improved to avoid sending WRITEs to the server until the application
>>     does fsync.
>>
>>     Thanks,
>>     Soumya
>>