From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Robinson Subject: Re: RAID-10 initial sync is CPU-limited Date: Tue, 04 Jan 2011 17:05:34 +0000 Message-ID: <4D23535E.5060607@anonymous.org.uk> References: <20110103163213.GC17455@fi.muni.cz> <4D2334A4.4040109@anonymous.org.uk> <20110104164123.GT17455@fi.muni.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20110104164123.GT17455@fi.muni.cz> Sender: linux-raid-owner@vger.kernel.org To: Jan Kasprzak Cc: linux-raid@vger.kernel.org List-Id: linux-raid.ids On 04/01/2011 16:41, Jan Kasprzak wrote: > John Robinson wrote: [...] > Yes, I am aware of this. A single disk is able to do about > 147 MB/s according to hdparm -t. However (a big "however"), > my usage pattern rarely issues big/sequential requests, and for more > random load the total throughput generated by all disks will be > much lower and the disks themselves become the bottleneck. Sure, but doing a resync does require huge sequential reads and writes. > I have just been suriprised that for initial RAID-10 resync > the bottleneck is in the (single) CPU. > > : but because you're throttled by the PCIe x4 interface, you're only > : getting about half of what your discs could do. > > I have not talked about PCIe x4, but SAS 4-way multichannel. My bad. Same effect in this situation though. > Anyway, my SAS controller is connected by PCIe 2.0 x8, which equals > to (if I read Wikipedia correctly :-) 32 Gbit/s, i.e. 2 GByte/s. > So PCIe is not a bottleneck here. SAS is, and I am aware of that. Which is why the the md kernel threads appear to be using 100% of CPU, they're blocked waiting for I/O. (And possibly RAM, per my other reply to this thread.) Cheers, John.