From: Emmanuel Florac <eflorac@intellique.com>
To: xfs@oss.sgi.com
Subject: nfs performance delta between filesystems
Date: Fri, 22 Jan 2010 18:54:19 +0100 [thread overview]
Message-ID: <20100122185419.63ae6430@harpe.intellique.com> (raw)
Here is an interesting mystery to solve : XFS is always much faster
locally than all other FSs, but quite slower through NFS.
I'm benchmarking a RAID-10 setup with various filesystems, ext3, reiser
(3.6), jfs and xfs. I'm strictly and only interested in continuous
sequential write (data recording) through NFS (basic 1G Eth), so I'm
benchmarking with
dd if=/dev/zero of=/mnt/whatever/file bs=1M
following the throughput with iostat.
Setup : linux 2.6.27.21, software RAID 10 (64K stripe), 1GB RAM, 64
bits kernel, 4 Savvio 500GB disks, Phenom II, Intel Pro1000 network.
All the following tests have been run several times each, to even
results.
Locally, xfs mounted this way
/dev/md2 on /mnt/raid type xfs (rw,noatime,nobarrier)
is wayyyyy better than all other filesystems :
~# dd if=/dev/zero of=/mnt/raid/testdd bs=1M count=4096
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB) copied, 23.0655 s, 186 MB/s
ext3, jfs and reiser score the same around 150 MB/s.
I'm exporting the mountpoint through NFS :
/mnt/raid *(fsid=0,rw,async,no_root_squash,no_subtree_check)
And mount with no options (or just "tcp") on the client. Then the
results are quite different. XFS gives a bit less than 90 MB/s (not
bad):
~$ sudo dd if=/dev/zero of=/mnt/temp/testnfsdd1 bs=1M count=2048
2048+0 enregistrements lus
2048+0 enregistrements écrits
2147483648 octets (2,1 GB) copiés, 24,1755 s, 88,8 MB/s
But... reiserfs gives ~ 92 MB/s, and ext3 95 MB/s. Results are
consistently better with both through NFS, while they're constantly
much worse locally... How come???
--
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <eflorac@intellique.com>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next reply other threads:[~2010-01-22 17:47 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-01-22 17:54 Emmanuel Florac [this message]
2010-01-22 18:38 ` nfs performance delta between filesystems bpm
2010-01-22 20:46 ` Emmanuel Florac
2010-01-23 12:30 ` Dave Chinner
2010-01-25 15:04 ` Christoph Hellwig
2010-01-25 20:28 ` bpm
2010-01-25 20:40 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100122185419.63ae6430@harpe.intellique.com \
--to=eflorac@intellique.com \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox