Re: Performance Diagnosis

["Chuck Lever" <chuck.lever@oracle.com>]

On Tue, Jul 15, 2008 at 11:34 AM, Andrew Bell <andrew.bell.ia@gmail.com> wrote:
> Hi,
>
> I have a RHEL 5 system that exhibits less than wonderful performance
> when copying large files from/to an NFS filesystem.  When the copy is
> taking place, other access to the filesystem is painfully slow.  I
> would like to have the filesystem react well to small requests while a
> large request is taking place.
>
> A couple of questions:
>
> Is this a reasonable expectation?

Yes, but Linux NFS can't fulfill it.  :-)

There is currently only one RPC transport socket between client and
server for each mount point.  Large file copies (or similar
operations) will queue a lot of I/O, so your small requests will take
a while to get through the queued up writes or reads ahead of them.

> Is this perhaps an I/O scheduling issue that isn't specific to NFS,
> but shows up there because of the latency of my NFS setup?
>
> Is this most likely a client issue, a server issue or a combination?

Well, if your server or network is slow, this kind of thing is more
likely to happen.

> Do you have recomendations on the best way to determine what is
> happening?  Are there existing tools to monitor active IO/NFS
> requests/responses and any relevant queues?

Yes, I wrote some Python tools that are still undocumented (ie you
will likely have to read the Python source to figure out what they
do).  They were recently included in nfs-utils, but you can download
them from:

  http://oss.oracle.com/~cel/linux-2.6/2.6.25/nfs-iostat

and

  http://oss.oracle.com/~cel/linux-2.6/2.6.25/mountstats

>
>
> Thanks for any info/ideas before I get in too deep :)
>
> --
> Andrew Bell
> andrew.bell.ia@gmail.com

-- 
Chuck Lever