From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kasprzak Subject: Re: RAID-10 initial sync is CPU-limited Date: Tue, 4 Jan 2011 17:41:23 +0100 Message-ID: <20110104164123.GT17455@fi.muni.cz> References: <20110103163213.GC17455@fi.muni.cz> <4D2334A4.4040109@anonymous.org.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <4D2334A4.4040109@anonymous.org.uk> Sender: linux-raid-owner@vger.kernel.org To: John Robinson Cc: linux-raid@vger.kernel.org List-Id: linux-raid.ids John Robinson wrote: : On 03/01/2011 16:32, Jan Kasprzak wrote: : [...] : > My hardware is apparently able to keep all the disks busy most : >of the time (verified by running dd if=/dev/sd$i bs=1M of=/dev/null : >in parallel - iostat reports 99-100 % utilization of each disk and about : >55 MB/s read per disk). All the disks are connected by a four-lane : >SAS controller, so the maximum theoretical throughput is 4x 3 Gbit/s : >= 12 Gbit/s = 0.5 Gbit/s per disk = 62.5 MByte/s per disk). : : This is part of your limit. Yes, I am aware of this. A single disk is able to do about 147 MB/s according to hdparm -t. However (a big "however"), my usage pattern rarely issues big/sequential requests, and for more random load the total throughput generated by all disks will be much lower and the disks themselves become the bottleneck. I have just been suriprised that for initial RAID-10 resync the bottleneck is in the (single) CPU. : but because you're throttled by the PCIe x4 interface, you're only : getting about half of what your discs could do. I have not talked about PCIe x4, but SAS 4-way multichannel. Anyway, my SAS controller is connected by PCIe 2.0 x8, which equals to (if I read Wikipedia correctly :-) 32 Gbit/s, i.e. 2 GByte/s. So PCIe is not a bottleneck here. SAS is, and I am aware of that. -Yenya -- | Jan "Yenya" Kasprzak | | GPG: ID 1024/D3498839 Fingerprint 0D99A7FB206605D7 8B35FCDE05B18A5E | | http://www.fi.muni.cz/~kas/ Journal: http://www.fi.muni.cz/~kas/blog/ | Please don't top post and in particular don't attach entire digests to your mail or we'll all soon be using bittorrent to read the list. --Alan Cox