Re: Bad sequential performance of RAID5 with a lot of disk seeks

[Robin Hill <robin@robinhill.me.uk>]

[-- Attachment #1: Type: text/plain, Size: 3074 bytes --]

On Tue Oct 07, 2014 at 09:54:22AM +0200, P. Gautschi wrote:

> Quoting Robin Hill <robin@robinhill.me.uk>:
> > On Tue Oct 07, 2014 at 06:44:49AM +0200, P. Gautschi wrote:
> >
> >> I've created a RAID5 on 5 identical SATA disks. Doing some  
> >> performance measurements
> >> with dd I get a disappointing performance.
> >> A dd with bs=1M on a btrfs created on md0 transfers about 110 MB/s.  
> >> (both read and write)
> >> A dd on md0 has the same write speed but only about 20 MB/s on read.
> >> In all of the tests I hear the disk constantly seeking. This was  
> >> also the case
> >> during creation of the array.
> >> I also created a RAID4 to make sure that I doesn't get fooled by  
> >> the stripe layout of RAID5.
> >> Now I get about 110 MB/s for write and 230 MB/s for read on md0.  
> >> But the constant
> >> seeking is still present for both read and write and during  
> >> creation of the array.
> >>
> >> Why are the disk perform so many seek operations? I think a  
> >> sequential access on md0 should
> >> cause a sequential access on the individual disk.
> >>
> >> I have to add that I did something unusual: I created the RAID4/5  
> >> with a chunk size of 4KiB.
> >> The idea of this was that when I'm going to use btrfs with the  
> >> default nodesize of 16KiB
> >> all node write will fill a full stripe and there won't be any RMW  
> >> at all. (both fortunate
> >> for performance and integrity in a power loss situation.)
> >> Nevertheless I think a sequential access on the array should cause  
> >> a sequential access on the
> >> disks for any chunk size if the read/write block size is a exact multiple of
> >> the (numdisks-1)*chunk size.
> >>
> >> Is there any explanation for the seeks and how do I get rid of them?
> >>
> > After creating the arrays did you wait for them to finish syncing? The
> > array is created in degraded mode initially and then rebuilds onto the
> > additional disk (this is the fastest way to do things, unless you know
> > the disks are all zeroed initially). Until this rebuild is complete then
> > it'll be competing with any other disk activity.
> >
> Yes I did wait for the syncing to complete before accessing md0.
> I had the seeks during the syncing and afterwards when reading or  
> writing to the array.
> 
Hmm, shouldn't be seeking then.

What does the SMART info show for the drives - are there any reallocated
blocks? A large number of those scattered over the disk would certainly
cause seeking for both reads and writes.

It's also worth checking whether there's anything else that would be
accessing the disks in the background (monitoring/indexing/etc).

I can't think of anything else that would be causing reads to seek - SMR
disks or write-intent bitmaps would only affect writes.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]