* Re: mkfs options for a 16x hw raid5 and xfs (mostly large files)
[not found] ` <20070926084924.GB30287@p15145560.pureserver.info>
@ 2007-09-26 9:52 ` Justin Piszcz
2007-09-26 15:03 ` Bryan J Smith
0 siblings, 1 reply; 4+ messages in thread
From: Justin Piszcz @ 2007-09-26 9:52 UTC (permalink / raw)
To: Ralf Gross; +Cc: linux-xfs, linux-raid
On Wed, 26 Sep 2007, Ralf Gross wrote:
> Justin Piszcz schrieb:
>> What was the command line you used for that output?
>> tiobench.. ?
>
> tiobench --numruns 3 --threads 1 --threads 2 --block 4096 --size 20000
>
> --size 20000 because the server has 16 GB RAM.
>
> Ralf
>
>
Here is my output on my SW RAID5 keep in mind it is currently being used so the numbers are a little slower than they probably should be:
My machine only has 8 GiB of memory but I used the same command you did:
This is with the 2.6.22.6 kernel, the 2.6.23-rcX/final when released is supposed to have the SW RAID5 accelerator code, correct?
Unit information
================
File size = megabytes
Blk Size = bytes
Rate = megabytes per second
CPU% = percentage of CPU used during the test
Latency = milliseconds
Lat% = percent of requests that took longer than X seconds
CPU Eff = Rate divided by CPU% - throughput per cpu load
Sequential Reads
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 523.01 45.79% 0.022 510.77 0.00000 0.00000 1142
2.6.22.6 20000 4096 2 501.29 85.84% 0.046 855.59 0.00000 0.00000 584
Random Reads
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 0.90 0.276% 13.003 74.41 0.00000 0.00000 326
2.6.22.6 20000 4096 2 1.61 1.167% 14.443 126.43 0.00000 0.00000 137
Sequential Writes
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 363.46 75.72% 0.030 2757.45 0.00000 0.00000 480
2.6.22.6 20000 4096 2 394.45 287.9% 0.056 2798.92 0.00000 0.00000 137
Random Writes
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 3.16 1.752% 0.011 1.02 0.00000 0.00000 180
2.6.22.6 20000 4096 2 3.07 3.769% 0.013 0.10 0.00000 0.00000 82
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: mkfs options for a 16x hw raid5 and xfs (mostly large files)
2007-09-26 9:52 ` mkfs options for a 16x hw raid5 and xfs (mostly large files) Justin Piszcz
@ 2007-09-26 15:03 ` Bryan J Smith
2007-09-26 16:24 ` Justin Piszcz
0 siblings, 1 reply; 4+ messages in thread
From: Bryan J Smith @ 2007-09-26 15:03 UTC (permalink / raw)
To: Justin Piszcz, xfs-bounce, Ralf Gross; +Cc: linux-xfs, linux-raid
Everyone can play local benchmarking games all they want,
and software RAID will almost always be faster, significantly at times.
What matters is actual, multiple client performance under full load.
Anything less is a completely irrelevant.
--
Bryan J Smith - mailto:b.j.smith@ieee.org
http://thebs413.blogspot.com
Sent via BlackBerry from T-Mobile
-----Original Message-----
From: Justin Piszcz <jpiszcz@lucidpixels.com>
Date: Wed, 26 Sep 2007 05:52:39
To:Ralf Gross <Ralf-Lists@ralfgross.de>
Cc:linux-xfs@oss.sgi.com, linux-raid@vger.kernel.org
Subject: Re: mkfs options for a 16x hw raid5 and xfs (mostly large files)
On Wed, 26 Sep 2007, Ralf Gross wrote:
> Justin Piszcz schrieb:
>> What was the command line you used for that output?
>> tiobench.. ?
>
> tiobench --numruns 3 --threads 1 --threads 2 --block 4096 --size 20000
>
> --size 20000 because the server has 16 GB RAM.
>
> Ralf
>
>
Here is my output on my SW RAID5 keep in mind it is currently being used so the numbers are a little slower than they probably should be:
My machine only has 8 GiB of memory but I used the same command you did:
This is with the 2.6.22.6 kernel, the 2.6.23-rcX/final when released is supposed to have the SW RAID5 accelerator code, correct?
Unit information
================
File size = megabytes
Blk Size = bytes
Rate = megabytes per second
CPU% = percentage of CPU used during the test
Latency = milliseconds
Lat% = percent of requests that took longer than X seconds
CPU Eff = Rate divided by CPU% - throughput per cpu load
Sequential Reads
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 523.01 45.79% 0.022 510.77 0.00000 0.00000 1142
2.6.22.6 20000 4096 2 501.29 85.84% 0.046 855.59 0.00000 0.00000 584
Random Reads
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 0.90 0.276% 13.003 74.41 0.00000 0.00000 326
2.6.22.6 20000 4096 2 1.61 1.167% 14.443 126.43 0.00000 0.00000 137
Sequential Writes
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 363.46 75.72% 0.030 2757.45 0.00000 0.00000 480
2.6.22.6 20000 4096 2 394.45 287.9% 0.056 2798.92 0.00000 0.00000 137
Random Writes
File Blk Num Avg Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.22.6 20000 4096 1 3.16 1.752% 0.011 1.02 0.00000 0.00000 180
2.6.22.6 20000 4096 2 3.07 3.769% 0.013 0.10 0.00000 0.00000 82
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: mkfs options for a 16x hw raid5 and xfs (mostly large files)
2007-09-26 15:03 ` Bryan J Smith
@ 2007-09-26 16:24 ` Justin Piszcz
2007-09-26 17:11 ` Bryan J. Smith
0 siblings, 1 reply; 4+ messages in thread
From: Justin Piszcz @ 2007-09-26 16:24 UTC (permalink / raw)
To: Bryan J Smith; +Cc: xfs-bounce, Ralf Gross, linux-xfs, linux-raid
I have a question, when I use multiple writer threads (2 or 3) I see
550-600 MiB/s write speed (vmstat) but when using only 1 thread, ~420-430
MiB/s... Also without tweaking, SW RAID is very slow (180-200 MiB/s) using
the same disks.
Justin.
On Wed, 26 Sep 2007, Bryan J Smith wrote:
> Everyone can play local benchmarking games all they want,
> and software RAID will almost always be faster, significantly at times.
>
> What matters is actual, multiple client performance under full load.
> Anything less is a completely irrelevant.
> --
> Bryan J Smith - mailto:b.j.smith@ieee.org
> http://thebs413.blogspot.com
> Sent via BlackBerry from T-Mobile
>
>
> -----Original Message-----
> From: Justin Piszcz <jpiszcz@lucidpixels.com>
>
> Date: Wed, 26 Sep 2007 05:52:39
> To:Ralf Gross <Ralf-Lists@ralfgross.de>
> Cc:linux-xfs@oss.sgi.com, linux-raid@vger.kernel.org
> Subject: Re: mkfs options for a 16x hw raid5 and xfs (mostly large files)
>
>
>
>
> On Wed, 26 Sep 2007, Ralf Gross wrote:
>
>> Justin Piszcz schrieb:
>>> What was the command line you used for that output?
>>> tiobench.. ?
>>
>> tiobench --numruns 3 --threads 1 --threads 2 --block 4096 --size 20000
>>
>> --size 20000 because the server has 16 GB RAM.
>>
>> Ralf
>>
>>
>
> Here is my output on my SW RAID5 keep in mind it is currently being used so the numbers are a little slower than they probably should be:
>
> My machine only has 8 GiB of memory but I used the same command you did:
>
> This is with the 2.6.22.6 kernel, the 2.6.23-rcX/final when released is supposed to have the SW RAID5 accelerator code, correct?
>
> Unit information
> ================
> File size = megabytes
> Blk Size = bytes
> Rate = megabytes per second
> CPU% = percentage of CPU used during the test
> Latency = milliseconds
> Lat% = percent of requests that took longer than X seconds
> CPU Eff = Rate divided by CPU% - throughput per cpu load
>
> Sequential Reads
> File Blk Num Avg Maximum Lat% Lat% CPU
> Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
> ---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
> 2.6.22.6 20000 4096 1 523.01 45.79% 0.022 510.77 0.00000 0.00000 1142
> 2.6.22.6 20000 4096 2 501.29 85.84% 0.046 855.59 0.00000 0.00000 584
>
> Random Reads
> File Blk Num Avg Maximum Lat% Lat% CPU
> Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
> ---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
> 2.6.22.6 20000 4096 1 0.90 0.276% 13.003 74.41 0.00000 0.00000 326
> 2.6.22.6 20000 4096 2 1.61 1.167% 14.443 126.43 0.00000 0.00000 137
>
> Sequential Writes
> File Blk Num Avg Maximum Lat% Lat% CPU
> Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
> ---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
> 2.6.22.6 20000 4096 1 363.46 75.72% 0.030 2757.45 0.00000 0.00000 480
> 2.6.22.6 20000 4096 2 394.45 287.9% 0.056 2798.92 0.00000 0.00000 137
>
> Random Writes
> File Blk Num Avg Maximum Lat% Lat% CPU
> Identifier Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff
> ---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
> 2.6.22.6 20000 4096 1 3.16 1.752% 0.011 1.02 0.00000 0.00000 180
> 2.6.22.6 20000 4096 2 3.07 3.769% 0.013 0.10 0.00000 0.00000 82
>
>
>
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: mkfs options for a 16x hw raid5 and xfs (mostly large files)
2007-09-26 16:24 ` Justin Piszcz
@ 2007-09-26 17:11 ` Bryan J. Smith
0 siblings, 0 replies; 4+ messages in thread
From: Bryan J. Smith @ 2007-09-26 17:11 UTC (permalink / raw)
To: Justin Piszcz, Bryan J Smith
Cc: xfs-bounce, Ralf Gross, linux-xfs, linux-raid
Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
> I have a question, when I use multiple writer threads (2 or 3) I
> see 550-600 MiB/s write speed (vmstat) but when using only 1
thread,
> ~420-430 MiB/s...
It's called scheduling buffer flushes, as well as the buffering
itself.
> Also without tweaking, SW RAID is very slow (180-200
> MiB/s) using the same disks.
But how much of that tweaking is actually just buffering?
That's a continued theme (and issue).
Unless you can force completely synchronous writes, you honestly
don't know. Using a larger size than memory is not anywhere near the
same.
Plus it makes software RAID utterly n/a in comparison to hardware
RAID, where the driver is waiting until the commit to actual NVRAM or
disc is complete.
--
Bryan J. Smith Professional, Technical Annoyance
b.j.smith@ieee.org http://thebs413.blogspot.com
--------------------------------------------------
Fission Power: An Inconvenient Solution
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2007-09-26 17:11 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <498689.78850.qm@web32907.mail.mud.yahoo.com>
[not found] ` <Pine.LNX.4.64.0709251938400.7763@p34.internal.lan>
[not found] ` <20070926082322.GA30287@p15145560.pureserver.info>
[not found] ` <Pine.LNX.4.64.0709260442070.31289@p34.internal.lan>
[not found] ` <20070926084924.GB30287@p15145560.pureserver.info>
2007-09-26 9:52 ` mkfs options for a 16x hw raid5 and xfs (mostly large files) Justin Piszcz
2007-09-26 15:03 ` Bryan J Smith
2007-09-26 16:24 ` Justin Piszcz
2007-09-26 17:11 ` Bryan J. Smith
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).