Re: NFS tuning - high performance throughput.

[Chris Penney <cpenney@gmail.com>]

>  Local RW is ~135Mbytes/sec

That seems slow for LSI disks.  I use LSI disk (STK rebrand) and have
one HBA connected to controller A and the other HBA to controller B.=20
I then have two 1TB luns primary on A and B (so a total of 4 TB).  I
get ~300MB/s.  I'm wondering if you are striping across controllers
and what file system you use (I use JFS).

How are you chaining the luns into one volume?  I use the device
mapper (4 multipath devices =3D> 1 linear).

> I have read most of the tuning guides I can find on the net and
> attempted just about everything I can get my hands on (I have not tried
> jumbo frames yet, still waiting for some downtime to attempt that).  My
> problem is that no matter how I tune the machines I can get at max
> 45Mb/ps throughput on NFS.=20

That is pretty low, esp. if you are talking about reads, which I can
get >100MB/s with only a single e1000 card.  I use the following mount
options:

Client: nosuid,rw,bg,hard,intr,vers=3D3,proto=3Dtcp,rsize=3D32768,wsize=3D3=
2768
Server: rw,sync,no_subtree_check,no_root_squash

I use 128 NFS threads (which in SuSE is set in /etc/sysconfig/nfs).

In /etc/sysctl.conf I have (can't say how well tuned these are):
net.core.rmem_default =3D 262144
net.core.wmem_default =3D 262144
net.core.rmem_max =3D 8388608
net.core.wmem_max =3D 8388608
net.ipv4.tcp_rmem =3D 4096 87380 8388608
net.ipv4.tcp_wmem =3D 4096 65536 8388608
net.ipv4.tcp_mem =3D 8388608 8388608 8388608

I've also found that enabling hyperthreading is a good thing for NFS.=20
Under load using etheral I show an improvement in write/read/commit
latency using HT.  I'm also using SLES 9.

   Chris


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs