[NFS] Advise on NFS performance tunning.

[NFS] Advise on NFS performance tunning.

[howard chen]

Hello, I have a busy web server, in which all HTML files are store in
a centralized NFS server.

I experienced the NFS is slow, i.e.  %wa is high in top command


My client's uptime is 144 days, has the following info by nfsstat:

==================================
calls      retrans    authrefrsh
1084310678   2347363    0


Client nfs v3:
null         getattr      setattr      lookup       access       readlink
0         0% 512606060 47% 26314540  2% 5826894   0% 143270988 13% 15        0%
read         write        create       mkdir        symlink      mknod
379396607 34% 15794239  1% 496509    0% 415960    0% 0         0% 0         0%
remove       rmdir        rename       link         readdir      readdirplus
25743     0% 153       0% 321       0% 0         0% 543       0% 697       0%
fsstat       fsinfo       pathconf     commit
6935      0% 4         0% 0         0% 154469    0%

==================================

As you can see the getattr is quite high, but my HTML files will
change every minutes.
And there are so many HTML files in NFS server in fact.


Are there any way I can tune my NFS setup?

e.g.

1. adding more memory to server?
2. adding more memory to client?
3. upgrade to NFS v4?
...

NFS mount option:
async,noatime,noexec,nosuid,hard,intr,udp,retry=3,rsize=32768,wsize=32768
NFS version: 3

Thanks.

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that nfs@lists.sourceforge.net is being discontinued.
Please subscribe to linux-nfs@vger.kernel.org instead.
    http://vger.kernel.org/vger-lists.html#linux-nfs

Re: [NFS] Advise on NFS performance tunning.

[Greg Banks]

howard chen wrote:
> Are there any way I can tune my NFS setup?
>
> e.g.
>
> 1. adding more memory to server?
> 2. adding more memory to client?
> 3. upgrade to NFS v4?
> ...
>
> NFS mount option:
> async,noatime,noexec,nosuid,hard,intr,udp,retry=3,rsize=32768,wsize=32768
> NFS version: 3
>
>   
http://mirror.linux.org.au/pub/linux.conf.au/2008/slides/130-lca2008-nfs-tuning-secrets-d7.odp

See in particular slides 27-30, 71-74, 86-88.

You mention increasing the server's RAM; this might help if you're
thrashing the server's dentry or pages caches.  Unfortunately I don't
know of a sensible or easy way to check this, except by comparing the
working set size to RAM.

Bumping up actimeo on the client could be a useful way to reduce the
amount of GETATTR traffic.

-- 
Greg Banks, P.Engineer, SGI Australian Software Group.
the brightly coloured sporks of revolution.
I don't speak for SGI.


------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that nfs@lists.sourceforge.net is being discontinued.
Please subscribe to linux-nfs@vger.kernel.org instead.
    http://vger.kernel.org/vger-lists.html#linux-nfs

Re: [NFS] Advise on NFS performance tunning.

[Chuck Lever]

On Feb 8, 2009, at 11:57 PM, Greg Banks wrote:

> howard chen wrote:
>> Are there any way I can tune my NFS setup?
>>
>> e.g.
>>
>> 1. adding more memory to server?
>> 2. adding more memory to client?
>> 3. upgrade to NFS v4?
>> ...
>>
>> NFS mount option:
>> async
>> ,noatime,noexec,nosuid,hard,intr,udp,retry=3,rsize=32768,wsize=32768
>> NFS version: 3
>>
>>
> http://mirror.linux.org.au/pub/linux.conf.au/2008/slides/130-lca2008-nfs-tuning-secrets-d7.odp
>
> See in particular slides 27-30, 71-74, 86-88.
>
> You mention increasing the server's RAM; this might help if you're
> thrashing the server's dentry or pages caches.  Unfortunately I don't
> know of a sensible or easy way to check this, except by comparing the
> working set size to RAM.
>
> Bumping up actimeo on the client could be a useful way to reduce the
> amount of GETATTR traffic.

I wonder if the GETATTR/ACCESS traffic is due to the web server  
calling open(3) on each file it serves.  Each open(3) will result in  
at least one synchronous NFS request.

You can use the "nocto" mount option plus a short "acreg{min,max}="  
setting to mitigate this.  Set acreg{min,max} to a number which  
reflects your tolerance for the web server detecting changed files on  
the NFS server (could be a minute or more).

This assumes you have only web server _read_ traffic on this mount  
point.

If the GETATTR/ACCESS traffic drops so much that the majority of your  
NFS traffic becomes READ requests, you can switch to NFS over TCP and  
use a large rsize to improve the efficiency of the READ operations.   
Increasing the client's RAM will also allow it to cache more web page  
file data, but that may not help if your web files are changing often  
(in which case Greg's server-side suggestions would be beneficial).

Naturally you should test such changes thoroughly before deploying them.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com