Re: RAID-10 initial sync is CPU-limited

[Jan Kasprzak <kas@fi.muni.cz>]

John Robinson wrote:
: On 03/01/2011 16:32, Jan Kasprzak wrote:
: [...]
: >	My hardware is apparently able to keep all the disks busy most
: >of the time (verified by running dd if=/dev/sd$i bs=1M of=/dev/null
: >in parallel - iostat reports 99-100 % utilization of each disk and about
: >55 MB/s read per disk). All the disks are connected by a four-lane
: >SAS controller, so the maximum theoretical throughput is 4x 3 Gbit/s
: >= 12 Gbit/s = 0.5 Gbit/s per disk = 62.5 MByte/s per disk).
: 
: This is part of your limit.

	Yes, I am aware of this. A single disk is able to do about
147 MB/s according to hdparm -t. However (a big "however"),
my usage pattern rarely issues big/sequential requests, and for more
random load the total throughput generated by all disks will be
much lower and the disks themselves become the bottleneck.

	I have just been suriprised that for initial RAID-10 resync
the bottleneck is in the (single) CPU.

: but because you're throttled by the PCIe x4 interface, you're only 
: getting about half of what your discs could do.

	I have not talked about PCIe x4, but SAS 4-way multichannel.
Anyway, my SAS controller is connected by PCIe 2.0 x8, which equals
to (if I read Wikipedia correctly :-) 32 Gbit/s, i.e. 2 GByte/s.
So PCIe is not a bottleneck here. SAS is, and I am aware of that.

-Yenya

-- 
| Jan "Yenya" Kasprzak  <kas at {fi.muni.cz - work | yenya.net - private}> |
| GPG: ID 1024/D3498839      Fingerprint 0D99A7FB206605D7 8B35FCDE05B18A5E |
| http://www.fi.muni.cz/~kas/    Journal: http://www.fi.muni.cz/~kas/blog/ |
Please don't top post and in particular don't attach entire digests to your
mail or we'll all soon be using bittorrent to read the list.     --Alan Cox