From: Xupeng Yun <xupeng@xupeng.me>
To: "Ted Ts'o" <tytso@mit.edu>
Cc: Ext4 development <linux-ext4@vger.kernel.org>
Subject: Re: Bad performance of ext4 with kernel 3.0.17
Date: Fri, 2 Mar 2012 08:50:55 +0800 [thread overview]
Message-ID: <CACaf2aacsH-hd9YmXff+DX8qiDjNGeUv6kNe9JamPH6OpaN1Sw@mail.gmail.com> (raw)
In-Reply-To: <20120301194735.GD32588@thunk.org>
On Fri, Mar 2, 2012 at 03:47, Ted Ts'o <tytso@mit.edu> wrote:
> Two things I'd try:
>
> #1) If this is a freshly created file system, the kernel may be
> initializing the inode table in the background, and this could be
> interfering with your benchmark workload. To address this, you can
> either (a) add the mount option noinititable, (b) add the mke2fs
> option "-E lazy_itable_init=0" --- but this will cause the mke2fs to
> take a lot longer, or (c) mount the file system and wait until
> "dumpe2fs /dev/md3 | tail" shows that the last block group has the
> ITABLE_ZEROED flag set. For benchmarking purposes on a scratch
> workload, option (a) above is the fast thing to do.
>
Thank you Ted, I followed this and got the same result (read IOPS ~950
/ write IOPS ~100)
> #2) It could be that the file system is choosing blocks farther away
> from the beginning of the disk, which is slower, whereas the fio on
> the raw disk will use the blocks closest to the beginning of the disk,
> which are the fastest one. You could try creating the file system so
> it is only 10GB, and then try running fio on that small, truncated
> file system, and see if that makes a difference.
I created LVM on top of the RAID10 device, and then created a smaller LV(20GB),
after that I took benchmarks against the very same LV with different
filesystems, the
results are interesting:
xfs (read IOPS ~1700 / write IOPS ~200)
ext4 (read IOPS ~950 / write IOPS ~100)
ext3( read IOPS ~900 / write IOPS ~100)
reisferfs (read IOPS ~930 / write IOPS ~100)
btrfs (read IOPS ~1200 / write IOPS ~120)
I got very bad performance from XFS
(http://www.spinics.net/lists/xfs/msg08688.html) about
two months ago, which was caused by known bugs of XFS, then I tried
ext4 on some of
my servers, it works very well until I got a new server set up with soft RAID10.
What should I learn to understand what's happening? any suggestion is
appreciated.
--
Xupeng Yun
http://about.me/xupeng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2012-03-02 0:51 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-03-01 5:31 Bad performance of ext4 with kernel 3.0.17 Xupeng Yun
2012-03-01 19:47 ` Ted Ts'o
2012-03-02 0:50 ` Xupeng Yun [this message]
2012-03-02 2:45 ` Ted Ts'o
2012-03-02 7:06 ` Xupeng Yun
2012-03-03 3:56 ` Xupeng Yun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACaf2aacsH-hd9YmXff+DX8qiDjNGeUv6kNe9JamPH6OpaN1Sw@mail.gmail.com \
--to=xupeng@xupeng.me \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).