From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bill Davidsen Subject: Re: Awful RAID5 random read performance Date: Tue, 02 Jun 2009 15:47:40 -0400 Message-ID: <4A2581DC.3060005@tmr.com> References: <20090601045710141.GMHE24335@cdptpa-omta04.mail.rr.com> <200905312339.12234.tfjellstrom@shaw.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <200905312339.12234.tfjellstrom@shaw.ca> Sender: linux-raid-owner@vger.kernel.org To: tfjellstrom@shaw.ca Cc: linux-raid@vger.kernel.org List-Id: linux-raid.ids Thomas Fjellstrom wrote: > On Sun May 31 2009, Leslie Rhorer wrote: > > >>> Unfortunately it doesn't seem to be. Take a well-considered drive such >>> as the WD RE3; it's spec for average latency is 4.2ms. However does it >>> include the rotational latency (the time the head takes to reach the >>> sector once it's on the track)? I bet it doesn't. Taking it to be only >>> the average seek time, this drive is still among the fastest. For a >>> 7200rpm drive this latency is just 4.2ms, so we'd have for this fast >>> drive an average total latency of 8.4ms. >>> >> That's an average. For a random seek to exceed that, it's going to have to >> span many cylinders. Give the container size of a modern cylinder, that's >> a pretty big jump. Single applications will tend to have their data lumped >> somewhat together on the drive. >> >> > > >>> with >>> >>> > > >>> No, random I/O is the most common case for busy servers, when there >>> are lots of processes doing uncorrelated reads and writes. Even if a >>> >> Yes, exactly. By definition, such a scenario represents a multithreaded >> set of seeks, and as we already established, multithreaded seeks are vastly >> more efficient than serial random seeks. The 400 seeks per second number >> for 4 drives applies. I don't know the details of the Linux schedulers, >> but most schedulers employ some variation of an elevator seek to maximize >> seek efficiency. The brings the average latency way down and brings the >> seek frequency way up. >> > > Ah, I never really understood how adding more random load could increase > performance. Now I get it :) > > >>> single application does sequential access the head will likely have >>> moved between them. The only solution is to have lots of ram for >>> cache, and/or lots of disks. It'd be better if they were connected to >>> several controllers... >>> >> A large RAM cache will help, but as I already pointed out, the increases in >> returns for increasing cache size diminish rapidly past a certain point. >> Most quality drives these days have a 32MB cache, or 128M for a 4 drive >> array. Add the Linux cache on top of that, and it should be sufficient for >> most purposes. Remember, random seeks implies small data extents. Lots of >> disks will bring the biggest benefit, and disks are cheap. Multiple >> controllers really are not necessary, especially if the controller and >> drives support NCQ , but having multiple controllers certainly doesn't >> hurt. >> > > Yet I've heard NCQ makes some things worse. Some raid tweaking pages tell you > to try disabling NCQ. > > I've actually been thinking about trying md-cache with an SSD on top of my new > raid and see how that works long term. But I can't really think of a good > benchmark that actually imitates my particular use cases well enough to show > me if it'd help me at all ::) > > I doubt my punny little 30G OCZ Vertex would really help all that much any > how. > For ext[34] you might want to put the journal on SSD, if you are doing any significant write that will help. Mounting data=journal may also help write, supposedly the write will complete when the data hits the journal, and not wait for the platter. -- Bill Davidsen Even purely technical things can appear to be magic, if the documentation is obscure enough. For example, PulseAudio is configured by dancing naked around a fire at midnight, shaking a rattle with one hand and a LISP manual with the other, while reciting the GNU manifesto in hexadecimal. The documentation fails to note that you must circle the fire counter-clockwise in the southern hemisphere.