Re: Server bottleneck(?) due to large record write() buffer size from client app

[Shehjar Tikoo <shehjart-YbfuJp6tym7X/JP9YwkgDA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 1512 bytes --]

Trond Myklebust wrote:
> On Thu, 2008-08-21 at 12:05 +1000, Shehjar Tikoo wrote:
>> By changing pdflush settings on the client, I'd be changing the 
>> settings for the whole system. Is there a proc FS entry or any 
>> other config param that lets me lower the number of write 
>> requests buffered at client before the commit request is sent?
> 
> Write buffering is not something which is under the control of the 
> NFS filesystem: it is entirely managed by the VM. The filesystem 
> only enforces the close-to-open cache consistency requirements, 
> which of course are specific to NFS.

True, but I see that there are atleast two cases where NFS client will
send out the outstanding requests:

1. In nfs_flush_incompatible and through the balanced_rate_limited
code, which is essentially an interface into the VM.

2. In a few other places in VFS, where it sends the requests after
checking whether the file is opened O_SYNC or if the inode is IS_SYNC.

WRT point 2, if its possible to send the requests out explicitly for
these two flags, i.e. O_SYNC or S_SYNC, it might be possible to
optimize the transmit behaviour for large write() record sizes, which
are left uncommitted either:

1. till the full record has been transmitted as UNSTABLE.

2. or till the file is closed.

...resulting in a big flush at the server when the commit does come
in.

See the attached file for the plot of what I am observing. The plot is
based on data for 2.6.26 but I see the same behaviour on 2.6.27-rc3.

Thanks
Shehjar

[-- Attachment #2: nfs_xfs_recsize_write_128slots_1mwsize.png --]
[-- Type: image/png, Size: 4893 bytes --]