All of lore.kernel.org
 help / color / mirror / Atom feed
* NFS performance issue - P3 vs P4
@ 2002-10-11 19:43 Andrei Maslennikov
  2002-10-11 21:01 ` Trond Myklebust
  0 siblings, 1 reply; 4+ messages in thread
From: Andrei Maslennikov @ 2002-10-11 19:43 UTC (permalink / raw)
  To: nfs


We've ran into a funny problem with NFS performance and 
since a couple of days are trying to overcome it. We 
will be digging further in, but maybe somebody could give
us a valid advice..

     
               NetApp F760 (NFS server 1)

             /       |         |          \   
            /        |         |           \
        18 MB/s    18 MB/s    18 MB/s      18 MB/s 
           |         |         |            |
           |         |         |            |
Clients>> P3        P4       ALPHA         IBM
           |         |         |            |
           |         |         |            |
         34 MB/s   2 MB/s   22 MB/s      22 MB/s   
            \        |         |           /
             \       |         |          /
                      
            Linux NFS Server on P3 (server 2)


As you may see here, we are looking at the performance of two
different NFS servers. One is a 3-year old commercial NAS system
(Network Appliance Filer F760), another is a P3 machine
(biprocessor Pentium III at 1 Ghz).

Then, we have 4 NFS clients (one ALPHA quadriprocessor at 600
Mhz, one IBM quadriprocessor at 375 Mhz, one P3 biprocessor
at 1 Ghz and one P4 biprocessor at 2 Ghz). Alpha client is
running Tru 64 5.1, IBM client - AIX 5.1D. All Pentiums
including the P3 server run exactly the same version of OS:
Red Hat Linux 7.3 with all the recent patches. On clients, we
use RH kernel 2.4.18-10smp. And on server - 2.4.18 from
kernel.org + Alan Cox patch 3 + EVMS suport patches for
volume handling and snapshotting. 

All machines are sitting on the same switched non-blocking
Gigabit network. 

What we observe, is *very* strange. NetApp is uniformly
delivering 18 MB/sec write speed to all 4 different clients.
P3 server does visibly better for ALPHA, IBM and P3 clients
(in case of P3 client, we observe almost a factor of two in
performance, 34 MB/s compared with the 18 MB/sec of NetApp). 
However, when the same P3 server is used from P4 client, we 
see an enormous drop in performance. We can write basically 
only at 2 MB/s! 

We first suspected the network was responsible for this
behaviour. In fact, the retrans rate on P4 as seen with 
"nfsstat -r" between P4 client and P3 server was ridicolously 
high. However, when we checked the P3srv-P4cli link with ttcp, 
we have seen a perfect speed of memory-to-memory transfer of
more than 100 MB/s (both udp and tcp). Ftp disk-to-disk between
these two machines runs at more than 30 MB/s, exactly as one 
would expect. Server runs with 64 nfsd's, and we played a lot 
with timeo, rsize and wsize, nfsvers etc to arrive at most to 
5 MB/s. Never we got more than that.. 
 
At this point, we have checked the cables: used on
P4 client the cable that was connecting before the
extra-performant P3 client. Exactly the same behaviour. We
then added another Gigabit card of different make to the P4
client - same lousy performance. We then jumped to yet
another P4 machine with exactly the same hw and sw
configuration, and obtained the same identical result...

The worst point about it all is the fact that the same P4 client
nicely operates at 18 MB/s when NetApp is used. If this would be
a P4 client issue, than its performance should be equally bad for
both P3 and NetApp servers. And if it would be a P3 server issue,
then why the latter is able to deliver a good performance to 
ALPHA, IBM and P3 clients? And this is definitively *not* a 
hardware problem, as we are able to reproduce this behaviour on
another equivalent hardware.

What else we could look at? If anybody has any idea, please let
us know.

   
Thanks ahead - Andrei.




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2002-10-11 23:40 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2002-10-11 19:43 NFS performance issue - P3 vs P4 Andrei Maslennikov
2002-10-11 21:01 ` Trond Myklebust
2002-10-11 22:42   ` Andrei Maslennikov
2002-10-11 23:40     ` Trond Myklebust

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.