Re: very strange (maybe) raid1 testing results

[Bill Davidsen <davidsen@tmr.com>]

Neil Brown wrote:
> On Wednesday May 30, jnelson-linux-raid@jamponi.net wrote:
>   
>> On Wed, 30 May 2007, Jon Nelson wrote:
>>
>>     
>>> On Thu, 31 May 2007, Richard Scobie wrote:
>>>
>>>       
>>>> Jon Nelson wrote:
>>>>
>>>>         
>>>>> I am getting 70-80MB/s read rates as reported via dstat, and 60-80MB/s as
>>>>> reported by dd. What I don't understand is why just one disk is being used
>>>>> here, instead of two or more. I tried different versions of metadata, and
>>>>> using a bitmap makes no difference. I created the array with (allowing for
>>>>> variations of bitmap and metadata version):
>>>>>           
>>>> This is normal for md RAID1. What you should find is that for 
>>>> concurrent reads, each read will be serviced by a different disk, 
>>>> until no. of reads = no. of drives.
>>>>         
>>> Alright. To clarify, let's assume some process (like a single-threaded 
>>> webserver) using a raid1 to store content (who knows why, let's just say 
>>> it is), and also assume that the I/O load is 100% reads. Given that the 
>>> server does not fork (or create a thread) for each request, does that 
>>> mean that every single web request is essentially serviced from one 
>>> disk, always? What mechanism determines which disk actually services the 
>>> request?
>>>       
>> It's probably bad form to reply to one's own posts, but I just found
>>
>> static int read_balance(conf_t *conf, r1bio_t *r1_bio)
>>
>> in raid1.c which, if I'm reading the rest of the source correctly, 
>> basically says "pick the disk whose current head position is closest". 
>> This *could* explain the behavior I was seeing. Is that not correct?
>>     
>
> Yes, that is correct.  
> md/raid1 will send a completely sequential read request to just one
> device.  There is not much to be gained by doing anything else.
> md/raid10 in 'far' or 'offset' mode lays the data out differently and
> will issue read requests to all devices and often get better read
> throughput at some cost in write throughput.
>   
The whole "single process" thing may be a distraction rather than a 
solution, as well. I wrote a small program using pthreads which shared 
reads of a file between N threads in 1k blocks, such that each read was 
preceded by a seek. It *seemed* that these were being combined in the 
block layer before being passed on to the md logic, and treated as a 
single read as nearly as I could tell.

I did NOT look at actually disk i/o (didn't care), but rather only at 
the transfer rate from the file to memory, which did not change 
significantly from 1..N threads active, where N was the number of 
mirrors. And RAID-10 did as well with one thread as several.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979