raid 5 read performance

raid 5 read performance

[Raz Ben-Jehuda(caro)]

Neil hello

Sorry for the delay. too many things to do.

I have implemented all said in :
http://www.spinics.net/lists/raid/msg11838.html

As always I have some questions:

1.  mergeable_bvec
     I did not understand first i must admit. now i do not see how it
differs from the
     one of raid0.  so i  actually copied it and renamed it.

2. statistics.
    i have added md statistics since the code does not reach the
statics in make_request.
    it returns from make_request before that.

3. i have added the new retry list called toread_aligned to raid5_conf_t .
    hope this is correct.

4.  your instructions are to add a failed bio to sh, but it does not
say to handle it directly.
    i have tried it and something is missing here. raid5d handle
stripes only if  conf->handle_list is not empty. i added handle_stripe
and and release_stripe of my own.
   this way i managed to get from the completion routine:
   "R5: read error corrected!! " message . ( i have tested by failing
a ram disk ).


5. I am going to test the non common path heavily before submitting
you the patch ( on real disks  and use  several file systems and
several chunk sizes).
 It is quite a big patch so I need to know which kernel do you want me
to use ? i am using poor 2.6.15.

I thank you
-- 
Raz

Re: raid 5 read performance

[Neil Brown]

On Friday June 9, raziebe@gmail.com wrote:
> Neil hello
> 
> Sorry for the delay. too many things to do.

You aren't alone there!

> 
> I have implemented all said in :
> http://www.spinics.net/lists/raid/msg11838.html
> 
> As always I have some questions:
> 
> 1.  mergeable_bvec
>      I did not understand first i must admit. now i do not see how it
> differs from the
>      one of raid0.  so i  actually copied it and renamed it.

Sounds fine.  For raid5 there is no need to force write requests to be
split up, but that's a minor difference.

> 
> 2. statistics.
>     i have added md statistics since the code does not reach the
> statics in make_request.
>     it returns from make_request before that.

Why not put the code *after* that?  Not that it matters a great deal.
I'll comment more when I see the code I expect.

> 
> 3. i have added the new retry list called toread_aligned to raid5_conf_t .
>     hope this is correct.
> 

Sounds good.


> 4.  your instructions are to add a failed bio to sh, but it does not
> say to handle it directly.
>     i have tried it and something is missing here. raid5d handle
> stripes only if  conf->handle_list is not empty. i added handle_stripe
> and and release_stripe of my own.
>    this way i managed to get from the completion routine:
>    "R5: read error corrected!! " message . ( i have tested by failing
> a ram disk ).
> 

Sounds right, but I'd need to see the code to be sure.

> 
> 5. I am going to test the non common path heavily before submitting
> you the patch ( on real disks  and use  several file systems and
> several chunk sizes).

I'd rather see the patch earlier, even if it isn't fully tested.

>  It is quite a big patch so I need to know which kernel do you want me
> to use ? i am using poor 2.6.15.

A patch against the latest -mm would be best, but I'm happy to take it
against anything even vaguely recent.

However, it needs to be multiple patches, not just one.
This is a *very* important point.  As that original email said:

  This should be developed and eventually presented as a sequence of
  patches.

There are several distinct steps in this change and they need to be
reviewed separately or it is just too hard.
So I would really like it if you could separate out the changes into
logically distinct patches.
If you can't or won't, then still send the patch, but I'll have to
break it up so it'll probably take longer to process.


Thanks for your efforts,

NeilBrown

Re: raid 5 read performance

[Dan Williams]

> Neil hello
>
> Sorry for the delay. too many things to do.
>
> I have implemented all said in :
> http://www.spinics.net/lists/raid/msg11838.html
>
> As always I have some questions:
>
> 1.  mergeable_bvec
>      I did not understand first i must admit. now i do not see how it
> differs from the
>      one of raid0.  so i  actually copied it and renamed it.
>
> 2. statistics.
>     i have added md statistics since the code does not reach the
> statics in make_request.
>     it returns from make_request before that.
>
> 3. i have added the new retry list called toread_aligned to raid5_conf_t .
>     hope this is correct.
>
> 4.  your instructions are to add a failed bio to sh, but it does not
> say to handle it directly.
>     i have tried it and something is missing here. raid5d handle
> stripes only if  conf->handle_list is not empty. i added handle_stripe
> and and release_stripe of my own.
>    this way i managed to get from the completion routine:
>    "R5: read error corrected!! " message . ( i have tested by failing
> a ram disk ).
>
>
> 5. I am going to test the non common path heavily before submitting
> you the patch ( on real disks  and use  several file systems and
> several chunk sizes).
>  It is quite a big patch so I need to know which kernel do you want me
> to use ? i am using poor 2.6.15.
>
> I thank you
> --
> Raz
Hi Raz,

What is the status of this patch?  Anything I can help out with, like
testing or forward porting to the latest kernel?

Thanks,

Dan

Re: raid 5 read performance

[Dan Williams]

On 8/5/06, Raz Ben-Jehuda(caro) <raziebe@gmail.com> wrote:
> patch is applied by Neil.
> I do not know when he going to apply it.
> i have applied it on my systems ( on 2.6.15 )  but they are currenly in the
> lab and not in production.
> Raz.
> PS
> I must say that it saves lots of cpu cycles.
>
Did you send the 2.6.15 patch in a private message I can't find it in
the archives?

Thanks,

Dan

Raid 5 Read performance

[Koti]

Hi All

    I am trying measure the RAID 5 Read numbers on linux kernel-2.6.21 and 
kernel 2.6.27 the numbers are lower on later kernel (2.6.27). Iam measuring 
the numbers using "time dd" command (4GB ). Is any valid reason for this, or 
Am missing anything in the process. Can anybody help me in analyze this 
problem.

Thanks,
Koti

Re: Raid 5 Read performance

[Majed B.]

Someone else has already emailed the mailing list about this. Are you
two running an assignment or something?!

Check the mailing list archive for whatever people have already replied with.

On Thu, Sep 10, 2009 at 11:07 PM, Koti<satha_koti@yahoo.co.in> wrote:
> Hi All
>
>    I am trying measure the RAID 5 Read numbers on linux kernel-2.6.21 and
> kernel 2.6.27 the numbers are lower on later kernel (2.6.27). Iam measuring
> the numbers using "time dd" command (4GB ). Is any valid reason for this, or
> Am missing anything in the process. Can anybody help me in analyze this
> problem.
>
> Thanks,
> Koti
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
       Majed B.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Raid 5 Read performance

[Majed B.]

All kernel changes are mentioned in the changelogs, so I think you can
find what you want at kernel.org -- it also mentions who did the
change, so you can contact him/her directly.

I'm not saying you won't find help here, but the aforementioned is a
better chance :)

On Fri, Sep 11, 2009 at 6:48 PM, satha kottidi<satha_koti@yahoo.co.in> wrote:
> Hi Majed & All,
>
> Yes we two are working together. Actually we found a change in md/raid5.c
> file in the function sync_request() it restricts the numbers, the kernel
> 2.6.27 has code like this
>
> while(unlikely(!handle_stripe(sh, NULL)))
>                 ;
>  But in kernel 2.6.21 it simply the handle_stripe(sh, NULL) call.
>
> Can anybody tell me the necessity of the change, and why it reduces the
> numbers. Any comments is very helpful for me to analyze the driver further.
>
>
> Thanks,
> Satha.
>
> --- On Fri, 11/9/09, Majed B. <majedb@gmail.com> wrote:
>
> From: Majed B. <majedb@gmail.com>
> Subject: Re: Raid 5 Read performance
> To: "Koti" <satha_koti@yahoo.co.in>
> Cc: linux-raid@vger.kernel.org
> Date: Friday, 11 September, 2009, 1:47 AM
>
> Someone else has already emailed the mailing list about this. Are you
> two running an assignment or something?!
>
> Check the mailing list archive for whatever people have already replied
> with.
>
> On Thu, Sep 10, 2009 at 11:07 PM, Koti<satha_koti@yahoo.co.in> wrote:
>> Hi All
>>
>> I am trying measure the RAID 5 Read numbers on linux kernel-2.6.21 and
>> kernel 2.6.27 the numbers are lower on later kernel (2.6.27). Iam
>> measuring
>> the numbers using "time dd" command (4GB ). Is any valid reason for this,
>> or
>> Am missing anything in the process. Can anybody help me in analyze this
>> problem.
>>
>> Thanks,
>> Koti
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
>
>
> --
>        Majed B.
>
> ________________________________
> Connect more, do more and share more with Yahoo! India Mail. Learn more.



-- 
       Majed B.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

raid 5 read performance

[Raz Ben-Jehuda(caro)]

Neil hello
I am measuring read performance of two raid5 with 7 sata disks, chunk size 1MB.
when i set the stripe_cache_size to 4096 i get 240 MB/s. IO'ing from
the two raids ended with 270 MB/s.

i have added a code in make_request which passes the raid5 logic in
the case of read.

it looks like this :
static int make_request (request_queue_t *q, struct bio * bi)
{
.....
	if ( conf->raid5_bypass_read  &&  bio_data_dir(bi) == READ )
	{	
		new_sector = raid5_compute_sector(bi->bi_sector,
						  raid_disks,
						  data_disks,
						  &dd_idx,
						  &pd_idx,
						  conf);

		bi->bi_sector = new_sector;
		bi->bi_bdev =  conf->disks[dd_idx].rdev->bdev;
		//
		//	do some statics
		//
		disk_stat_inc(mddev->gendisk, ios[rw]);
		disk_stat_add(mddev->gendisk, sectors[rw], bio_sectors(bi));

		//
		// make upper level to the work for me
		//
		return 1;
	}
...
}

it increased the performance to 440 MB/s.

Question :
       What is the cost of not walking trough the raid5 code in the
case of READ ?
       if i add and error handling code will it be suffice ?

thank you
-- 
Raz

Re: raid 5 read performance

[Neil Brown]

On Sunday May 21, raziebe@gmail.com wrote:
> 
> Question :
>        What is the cost of not walking trough the raid5 code in the
> case of READ ?
>        if i add and error handling code will it be suffice ?
> 

Please read

http://www.spinics.net/lists/raid/msg11838.html

and ask if you have further questions.

NeilBrown

Re: raid 5 read performance

[Dan Williams]

> Please read
>
> http://www.spinics.net/lists/raid/msg11838.html
>
> and ask if you have further questions.
>
Does this implementation also need to do delayed updates to the stripe
cache?  I.e. we bypass the cache and get the requester the data it
needs but then schedule that data to also be copied into the cache by
the work queue.  The thinking being to reduce the read penalties
associated with subsequent writes to the stripe.

Dan

Re: raid 5 read performance

[Neil Brown]

On Sunday May 21, dan.j.williams@gmail.com wrote:
> > Please read
> >
> > http://www.spinics.net/lists/raid/msg11838.html
> >
> > and ask if you have further questions.
> >
> Does this implementation also need to do delayed updates to the stripe
> cache?  I.e. we bypass the cache and get the requester the data it
> needs but then schedule that data to also be copied into the cache by
> the work queue.  The thinking being to reduce the read penalties
> associated with subsequent writes to the stripe.
> 

I doubt that would be worth the effort.  I don't think data lives very
long in the stripe cache, and so a lot of the copying would be wasted.
Also, I gather that modern drives have a reasonable cache and if data
was read recently for a read, then reading it again for a write might
be very quick.

I think the gains you get on the read side will more than offset any
small loss on the write side.

NeilBrown

Re: raid 5 read performance

[Stephan van Hienen]

On Sun, 21 May 2006, Neil Brown wrote:

> Please read
> 
> http://www.spinics.net/lists/raid/msg11838.html
> 
> and ask if you have further questions.

Neil,

what is the current status on the slow read performance with 2.6 ?

Regards,

Stephan 

Re: raid 5 read performance

[Neil Brown]

On Saturday October 21, raid@a2000.nu wrote:
> On Sun, 21 May 2006, Neil Brown wrote:
> 
> > Please read
> > 
> > http://www.spinics.net/lists/raid/msg11838.html
> > 
> > and ask if you have further questions.
> 
> Neil,
> 
> what is the current status on the slow read performance with 2.6 ?
> 

You mean by-passing the cache to avoid extra copy and synchronisation
overheads?
I have the patches.  I tried them once and couldn't measure any
speedup (I didn't try very hard) and put them aside think "I have to
look into this properly" and haven't got back to it yet.

Thanks for the reminder.  Hope to do some performance measurements
soon.  I'll make sure I include these patches when I do.

NeilBrown