linux-raid.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Joe Landman <joe.landman@gmail.com>
To: Ole Tange <ole@tange.dk>
Cc: linux-raid@vger.kernel.org
Subject: Re: Software RAID checksum performance on 24 disks not even close to kernel reported
Date: Mon, 04 Jun 2012 21:26:05 -0400	[thread overview]
Message-ID: <4FCD602D.9080005@gmail.com> (raw)
In-Reply-To: <CA+4vN7wPd0Av_sbtA0JD5y=fGcsuq20SOQVHWnE8BUFpDxOEXQ@mail.gmail.com>

On 06/04/2012 07:14 PM, Ole Tange wrote:

> But I cannot explain why even the best performance (4600 MB/11s = 420
> MB/s) is not even close to the checksum performance reported by the
> kernel at boot (6196 MB/s):
>
>      Mar 13 16:02:42 server kernel: [   35.120035] raid6: using
> algorithm sse2x4 (6196 MB/s)
>
> Can you explain why I only get 420 MB/s of real world checksumming
> instead of 6196 MB/s?

In the best possible case, you would get 22x 1-disk bandwidth, which 
would be ~120MB/s (assuming RAID6, and "infinite" speed of checksum 
computation).  This is your "theoretical" upper bound on performance.

Your pragmatic upper bound on performance is reduced from the 
theoretical by many issues, including various hardware issues 
(controller, PCIe lanes, memory, disk, ...), as well as software (IO 
stack traversal, elevators, buffers/cache fills, ... etc.).

Aside from this, it is very rare that you will have a single application 
reading and writing at full stripe width all the time, which would be 
the optimal case for you.  There are some, and we've play with a number 
for our customers.  But they are the exception and not the rule.

Your real world IO performance for 24 disks is 420MB/s.  So this 
particular setup is, from your numbers, operating at about 16% peak 
efficiency per disk +/- some.  This is not uncommon for people's 
self-built systems.  The checksumming isn't your rate limiting feature. 
  Other things are.

Joe

>
>
> /Ole

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman@scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615


  reply	other threads:[~2012-06-05  1:26 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-06-04 23:14 Software RAID checksum performance on 24 disks not even close to kernel reported Ole Tange
2012-06-05  1:26 ` Joe Landman [this message]
2012-06-05  3:36 ` Igor M Podlesny
2012-06-05  7:47   ` Ole Tange
2012-06-05 11:25     ` Peter Grandi
2012-06-05 20:57       ` Ole Tange
2012-06-06 17:37         ` Peter Grandi
2012-06-05 14:15     ` Stan Hoeppner
2012-06-05 20:45       ` Ole Tange
2012-06-05  3:39 ` Igor M Podlesny
2012-06-05  7:47   ` Ole Tange
2012-06-05 11:29     ` Igor M Podlesny
2012-06-05 13:09       ` Peter Grandi
2012-06-05 21:17         ` Ole Tange
2012-06-06  1:38           ` Stan Hoeppner
2012-06-05 18:44       ` Ole Tange
2012-06-06  1:40         ` Brad Campbell
2012-06-06  3:48           ` Marcus Sorensen
2012-06-06 11:21             ` Ole Tange
2012-06-06 11:17           ` Ole Tange
2012-06-06 12:58             ` Brad Campbell
2012-06-06 14:11 ` Ole Tange
2012-06-06 16:05   ` Igor M Podlesny
2012-06-06 19:51     ` Ole Tange
2012-06-06 22:21       ` Igor M Podlesny
2012-06-06 22:53         ` Peter Grandi
2012-06-07  3:41           ` Igor M Podlesny
2012-06-07  4:59             ` Stan Hoeppner
2012-06-07  5:22               ` Igor M Podlesny
2012-06-07  9:03                 ` Stan Hoeppner
2012-06-07  9:22                   ` Igor M Podlesny
2012-06-06 16:09   ` Dan Williams
2012-06-06 19:19     ` Ole Tange
2012-06-06 19:24       ` Dan Williams
2012-06-06 19:26         ` Ole Tange
2012-06-07  4:06     ` Stan Hoeppner
2012-06-07 14:40       ` Joe Landman
2012-06-08  1:23         ` Stan Hoeppner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4FCD602D.9080005@gmail.com \
    --to=joe.landman@gmail.com \
    --cc=linux-raid@vger.kernel.org \
    --cc=ole@tange.dk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).