All of lore.kernel.org
 help / color / mirror / Atom feed
From: Stan Hoeppner <stan@hardwarefreak.com>
To: Dave Chinner <david@fromorbit.com>
Cc: Jimmy Thrasibule <thrasibule.jimmy@gmail.com>,
	Linux RAID <linux-raid@vger.kernel.org>,
	"xfs@oss.sgi.com" <xfs@oss.sgi.com>
Subject: Re: ARC-1120 and MD very sloooow
Date: Mon, 25 Nov 2013 21:58:21 -0600	[thread overview]
Message-ID: <52941C5D.1000305@hardwarefreak.com> (raw)
In-Reply-To: <20131126025210.GL8803@dastard>

On 11/25/2013 8:52 PM, Dave Chinner wrote:
...
> sunit/swidth is in filesystem blocks, not sectors. Hence
> sunit is 1MB, swidth = 2MB. While it's not quite correct
> (su=512k,sw=1m), it's not actually a problem...

Well that's what I thought as well, and I was puzzled by the 8 blocks
value for the log sunit.  So I double checked before posting, and 'man
mkfs.xfs' told me

	sunit=value
              This is used to specify the stripe unit for a RAID device
              or a logical volume. The  value  has  to  be specified in
              512-byte block units.

So apparently the units of 'sunit' are different depending on which XFS
tool one is using.  That's a bit confusing.  And 'man xfs_info'
(xfs_growfs) doesn't tell us that sunit is given in filesystem blocks.
I'm using xfsprogs 3.1.4 so maybe these have been corrected since.

> Well, mkfs.xfs just uses what it gets from the kernel, so it
> might have been told the wrong thing by MD itself.  However, you can
> modify sunit/swidth by mount options, so you can't directly trust
> what is reported from xfs_info to be what mkfs actually set
> originally.

Got it.

> Again, lsunit is in filesystem blocks, so it is 32k, not 4k. And
> yes, the default lsunit when the sunit > 256k is 32k. So, nothing
> wrong there, either.

So where should I have looked to confirm sunit reported by xfs_info is
in fs block (4KB) multiples, not the in the 512B multiples of mkfs.xfs?

> The usual: "iostat -x -d -m 5" output while the test is running.
> Also, you are using buffered IO, so changing it to use direct IO
> will tell us exactly what the disks are doing when Io is issued.
> blktrace is your friend here....

It'll be interesting to see where this troubleshooting leads.  Buffered
single stream write speed is ~6x slower than read w/RAID10.  That makes
me wonder if the controller and drive write caches have been disabled.
That could explain this.

-- 
Stan

WARNING: multiple messages have this Message-ID (diff)
From: Stan Hoeppner <stan@hardwarefreak.com>
To: Dave Chinner <david@fromorbit.com>
Cc: Jimmy Thrasibule <thrasibule.jimmy@gmail.com>,
	Linux RAID <linux-raid@vger.kernel.org>,
	"xfs@oss.sgi.com" <xfs@oss.sgi.com>
Subject: Re: ARC-1120 and MD very sloooow
Date: Mon, 25 Nov 2013 21:58:21 -0600	[thread overview]
Message-ID: <52941C5D.1000305@hardwarefreak.com> (raw)
In-Reply-To: <20131126025210.GL8803@dastard>

On 11/25/2013 8:52 PM, Dave Chinner wrote:
...
> sunit/swidth is in filesystem blocks, not sectors. Hence
> sunit is 1MB, swidth = 2MB. While it's not quite correct
> (su=512k,sw=1m), it's not actually a problem...

Well that's what I thought as well, and I was puzzled by the 8 blocks
value for the log sunit.  So I double checked before posting, and 'man
mkfs.xfs' told me

	sunit=value
              This is used to specify the stripe unit for a RAID device
              or a logical volume. The  value  has  to  be specified in
              512-byte block units.

So apparently the units of 'sunit' are different depending on which XFS
tool one is using.  That's a bit confusing.  And 'man xfs_info'
(xfs_growfs) doesn't tell us that sunit is given in filesystem blocks.
I'm using xfsprogs 3.1.4 so maybe these have been corrected since.

> Well, mkfs.xfs just uses what it gets from the kernel, so it
> might have been told the wrong thing by MD itself.  However, you can
> modify sunit/swidth by mount options, so you can't directly trust
> what is reported from xfs_info to be what mkfs actually set
> originally.

Got it.

> Again, lsunit is in filesystem blocks, so it is 32k, not 4k. And
> yes, the default lsunit when the sunit > 256k is 32k. So, nothing
> wrong there, either.

So where should I have looked to confirm sunit reported by xfs_info is
in fs block (4KB) multiples, not the in the 512B multiples of mkfs.xfs?

> The usual: "iostat -x -d -m 5" output while the test is running.
> Also, you are using buffered IO, so changing it to use direct IO
> will tell us exactly what the disks are doing when Io is issued.
> blktrace is your friend here....

It'll be interesting to see where this troubleshooting leads.  Buffered
single stream write speed is ~6x slower than read w/RAID10.  That makes
me wonder if the controller and drive write caches have been disabled.
That could explain this.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  reply	other threads:[~2013-11-26  3:58 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-11-22 11:13 ARC-1120 and MD very sloooow Jimmy Thrasibule
2013-11-22 11:17 ` Mikael Abrahamsson
2013-11-22 20:17 ` Stan Hoeppner
2013-11-22 20:17   ` Stan Hoeppner
2013-11-25  8:56   ` Jimmy Thrasibule
2013-11-26  0:45     ` Stan Hoeppner
2013-11-26  0:45       ` Stan Hoeppner
2013-11-26  2:52       ` Dave Chinner
2013-11-26  2:52         ` Dave Chinner
2013-11-26  3:58         ` Stan Hoeppner [this message]
2013-11-26  3:58           ` Stan Hoeppner
2013-11-26  6:14           ` Dave Chinner
2013-11-26  8:03             ` Stan Hoeppner
2013-11-26  8:03               ` Stan Hoeppner
2013-11-28 15:59               ` Jimmy Thrasibule
2013-11-28 15:59                 ` Jimmy Thrasibule
2013-11-28 19:59                 ` Stan Hoeppner
2013-11-27 13:48             ` md raid5 performace 6x SSD RAID5 lilofile
2013-11-27 13:51             ` 答复：md " lilofile
2013-11-28  4:41               ` Stan Hoeppner
2013-11-28  4:46                 ` Roman Mamedov
2013-11-28  6:24                   ` Stan Hoeppner
2013-11-28 10:02               ` 答复：答复：md " lilofile
2013-11-29  2:38                 ` Stan Hoeppner
2013-11-29  6:23                   ` Stan Hoeppner
2013-11-30 14:12                 ` 答复：答复：答复：md raid5 random " lilofile
2013-12-01 14:14                   ` Stan Hoeppner
2013-12-01 16:33                   ` md " lilofile
2013-12-02  2:37                     ` Stan Hoeppner
2013-11-28 11:54               ` 答复：答复：md raid5 " lilofile
2013-12-02  3:48               ` md " lilofile
2013-12-02  5:51                 ` Stan Hoeppner
2014-09-23  3:34               ` raid sync speed lilofile
2014-09-23  5:11               ` behind_writes lilofile

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=52941C5D.1000305@hardwarefreak.com \
    --to=stan@hardwarefreak.com \
    --cc=david@fromorbit.com \
    --cc=linux-raid@vger.kernel.org \
    --cc=thrasibule.jimmy@gmail.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.