Re: Very Slow Sequential Reads over NFS from an XFS disk in Amazon EC2

[Ric Wheeler <ricwheeler@gmail.com>]

On 03/12/2010 01:22 PM, Brandon Simmons wrote:
> I am using tiobench to test performance of an NFS mounted volume, and
> notice that Sequential Reads are much slower than Random Reads. This
> isn't the behavior when I run the same test on the disk mounted
> locally.
>
> For random reads I'm getting:
>
>     50 MB/s  over NFS
>
> v.s
>
>     384 MB/s  when mounted locally
>
> This is in comparison to the benchmark for _Random Reads_, in which I get:
>
>     288 MB/s both over NFS _and_ when directly mounted
>
> The other benchmarks seem to be in line with what I would expect, but
> I'm fairly new to NFS. Why would sequential reads over NFS be sooo
> much slower than random reads over NFS?
>
> I am exporting the volume on the server like this
>
> /export *.internal(no_subtree_check,rw,no_root_squash)
>
> and mounting with this:
>
> mount -o hard,intr,async,noatime,nodiratime,noacl $NFS_SERVER:/export /nfs
>
> Additionally I am doing all this in amazon EC2, exporting an EBS
> volume with the XFS file system (redundant, I know).
>
> I have tried using jumbo frames and various other mount options, but
> none seem to have much effect.
>
> Thanks for any clues.
>    

Not sure what kind of network you are running the NFS test over so it is 
quite hard to figure out why your performance varies so wildly.

Normal NFS testing with a gigabit network between the client and server 
would be much closer to 50MB/sec than your 288MB/sec.

Can you try to reproduce this locally with known client and server hardware?

ric