Chuck Lever wrote:
> Wim Colgate wrote:
>> If I have a soft mount, and open a file with O_DIRECT and O_SYNC, 
>> should I ever expect a callback (nfs_writeback_done) with a successful 
>> task->tk_status (i.e >= 0) with the committed state 
>> (resp->verf->committed) set to NFS_UNSTABLE?
> 
> Yes, this can happen if the server decides to return NFS_UNSTABLE. Rare, 
> but possible.

Let me be more clear about this.

O_DIRECT and O_SYNC determine client behavior only.  However, they don't 
necessarily force NFS_FILE_SYNC writes all the time.  For example, if an 
application issues a direct write request that is much larger than the 
current wsize, the Linux NFS client will send the write using 
NFS_UNSTABLE requests followed by a COMMIT.  That makes it easier for 
the server to schedule disk writes more efficiently.

>> A secondary question: if the above is expected, does this occur 
>> because someone is caching the write and is there a mechanism to 
>> disable this effect?
> 
> Servers can return NFS_UNSTABLE to any WRITE request, so I can't think 
> of a way this might be disabled.

Even though an NFS client requests an NFS_FILE_SYNC write, the server 
still has the choice of returning something less, even NFS_UNSTABLE.  In 
general that's a rare occurrence, but is something I've seen in practice.