From: Dave Chinner <david@fromorbit.com>
To: Christian Affolter <c.affolter@purplehaze.ch>
Cc: xfs@oss.sgi.com
Subject: Re: Random write result differences between RAID device and XFS
Date: Mon, 1 Feb 2016 16:46:39 +1100 [thread overview]
Message-ID: <20160201054639.GU6033@dastard> (raw)
In-Reply-To: <56AC93EC.80202@purplehaze.ch>
On Sat, Jan 30, 2016 at 11:43:56AM +0100, Christian Affolter wrote:
> Hi Dave,
>
> On 29.01.2016 23:25, Dave Chinner wrote:
> > On Fri, Jan 29, 2016 at 11:53:35AM +0100, Christian Affolter wrote:
> >> Hi everyone,
> >>
> >> I'm trying to understand the differences of some bandwidth and IOPs test
> >> results I see while running a random-write full-stripe-width aligned fio
> >> test (using libaio with direct IO) on a hardware RAID 6 raw device
> >> versus on the same device with the XFS file system on top of it.
> >>
> >> On the raw device I get:
> >> write: io=24828MB, bw=423132KB/s, iops=137, runt= 60085msec
> >>
> >> With XFS on top of it:
> >> write: io=14658MB, bw=249407KB/s, iops=81, runt= 60182msec
> >
> > Now repeat with a file that is contiguously allocated before you
> > start. And also perhaps with the "swalloc" mount option.
>
> Wow, thanks! After specifying --fallocate=none (instead of the default
> fallocate=posix), bandwidth and iops increases and are even higher than
> on the raw device:
>
> write: io=30720MB, bw=599232KB/s, iops=195, runt= 52496msec
>
> I'm eager to learn what's going on behind the scenes, can you give a
> short explanation?
Usually when concurrent direct IO writes are slower than the raw
device it's because something is causing IO submission
serialisation. Usually that's to do with writes that extend the
file because that can require the inode to be locked exclusively.
Whatever behaviour the fio configuration change modifed, it removed
the IO submission serialisation and so it's now running at full disk
speed.
As to why XFS is faster than the raw block device, the XFS file
is only 30GB, so the random writes are only seeking a short
distance compared to the block device test which is seeking across
the whole device.
> Btw. mounting the volume with "swalloc" didn't make any change.
Which means there is no performance differential between stripe unit
and stripe width aligned writes in this test on your hardware.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2016-02-01 5:46 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-29 10:53 Random write result differences between RAID device and XFS Christian Affolter
2016-01-29 22:25 ` Dave Chinner
2016-01-30 10:43 ` Christian Affolter
2016-02-01 5:46 ` Dave Chinner [this message]
2016-02-01 8:59 ` Christian Affolter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160201054639.GU6033@dastard \
--to=david@fromorbit.com \
--cc=c.affolter@purplehaze.ch \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox