raid and elevator

raid and elevator

[Raz Ben-Jehuda(caro)]

It is my belief that the linux elevators are not optimized for
raid throughout.

Basically when IO initiator generates IO over a single disk he can control
the amount of IOs and this way control IOs latencies.

This is not the case in raided volumes.

Example:

An IO initiator intiates bulks of asynchronous IOs over a raided
volume. since the user does not know the target destination of each IO
he
might cause imbalance over the disks. one disk may become overflooded
with IOs while the other is may starve. So eventually we are ending with
a non balaned raided volume in terms of bad latencies.

I am trying to deal with this problem by fixing the deadline elevator code
to batch IOs , meaning , when n IOs are reaching the disk, each m deadlined
IOs are sorted and then dispatced.

I would appreciate any coments in this matter.

thank you
Raz Ben Yehuda

Re: raid and elevator

[Jens Axboe]

On Sun, Mar 26 2006, Raz Ben-Jehuda(caro) wrote:
> It is my belief that the linux elevators are not optimized for
> raid throughout.
> 
> Basically when IO initiator generates IO over a single disk he can control
> the amount of IOs and this way control IOs latencies.
> 
> This is not the case in raided volumes.
> 
> Example:
> 
> An IO initiator intiates bulks of asynchronous IOs over a raided
> volume. since the user does not know the target destination of each IO
> he
> might cause imbalance over the disks. one disk may become overflooded
> with IOs while the other is may starve. So eventually we are ending with
> a non balaned raided volume in terms of bad latencies.
> 
> I am trying to deal with this problem by fixing the deadline elevator code
> to batch IOs , meaning , when n IOs are reaching the disk, each m deadlined
> IOs are sorted and then dispatced.

You do realize that the io scheduler resides _below_ the raid
personality? So if you want to balance what goes to what io scheduler
(and thus, disk), you'd want to mess with the raid personality.

-- 
Jens Axboe

Re: raid and elevator

[Raz Ben-Jehuda(caro)]

i am sending 1MB buffer to a raid of 1MB chunk size.
i know that each offset is aligned by the chunk size.


On 3/27/06, Jens Axboe <axboe@suse.de> wrote:
>
> don't top-post!
>
> On Mon, Mar 27 2006, Raz Ben-Jehuda(caro) wrote:
> > I know that.
> > Currently i did some stupid stuff in the deadline.
> > I am thinking of creating some sort of awaiting list in the raid
> > layer by hooking into the make_request.
> >
> > Let me give u an example:
> > when issue 15 iops over a single disk i have an average latency of 580
> > ms ( 1MB IO buffers).
> > when i 60 iops over  raid5 of 4 disks i am getting 750 ms average latency.
> > I believe this is due to non balancing hence non fairness.
> > i need some sort of strict batching.
> > Do u agree ?
>
> That raid setup will always get you a little higher latency, simply
> because you will be waiting for more disks. As an example - submitting
> two ios to separate drives will always get higher latency than just one
> of the ios, since you want to wait for both to complete. Throughput may
> be better, but that doesn't say anything about the average or max
> experienced latency.
>
> --
> Jens Axboe
>
>


--
Raz

Re: raid and elevator

[Jens Axboe]

Once again, don't top post. Please!

On Mon, Mar 27 2006, Raz Ben-Jehuda(caro) wrote:
> i am sending 1MB buffer to a raid of 1MB chunk size.
> i know that each offset is aligned by the chunk size.

The raid is limited by the hardware storage, as the bio units are
essentially (usually) passed further down the stack untouched.

-- 
Jens Axboe