Re: elevator priorities vs. full request queues

[Jens Axboe <axboe@suse.de>]

On Tue, Jun 22 2004, Werner Almesberger wrote:
> Jens Axboe wrote:
> > Well yes, different parties have worked on io priorities in the past.
> > Request allocation is a important part of that.
> 
> I only found your "cfq + io priorities" posting from November
> 2003. Are there any others that are publicly available ?

Don't think I posted any newer ones, I'll see if I can get something
posted today/tomorrow (or at least before going on vacation thursday).

> > #4 doesn't work for various reasons, the obvious being that the request
> > is already allocated at this point.
> 
> The idea would be to refuse/delay low-priority requests before
> reaching the queue limit. But yes, having a possibly large
> number of allocated but not enqueuable requests sit around,
> or being forced to discard requests after building them, may
> not be so great.

That's pushing the problem somewhere it doesn't belong, imho. Let the
process block in getting a request instead, it doesn't make sense to let
it allocate a request and then refuse to queue it. Plus it raises all
sorts of handling in the lower layers to cope with this.

> > CFQ with priorities used a combination of 2+3. It seems you haven't
> > looked any of this up, maybe that would be a good starting point.
> 
> I had to have a second look :-) I didn't quite realize that
> your changes actually solved that. Nice and simple. Are there
> any newer versions of it ? What are your plans for IO priorities
> anyway ?

It's basically just deciding whether a process can allocate a request in
elv_may_queue(). Christoffer Hall implemented split allocation lists as
well.

My general plans for IO priorities basically involve rewriting lots of
ll_rw_blk, I'm afraid. Not fundamentally, but there are some areas that
don't work well if you assign different rights to different processes.
One being the queue congested bit - the queue may be congested for
processs A, but not for B. And I'd like to pass in priority through the
bio, like described further down.

> > CFQ used per-process (-group) priorities,
> 
> In my case, they're per open file or per memory page. The latter
> works as follows: pages are explicitly prefetched at the
> application's declared read rate. When doing this, the prefetcher
> sets the page's priority, which the elevator retrieves. Sound
> complicated but actually doesn't cause all that much overhead. As
> a welcome side-effect, prefetching also limits the high-priority
> IO activity an application can generate.

It doesn't sound too complicated :)

> > A bio has room for priority info, so you could also pass it in
> > from submit_bio() -> __make_request() -> get_request() -> may_queue().
> 
> Hmm yes, that's the large set of little changes I was hoping to
> avoid. But I guess there's no way around this.

I'd much rather do it this way, since I think it's the right way. It
might make sense to just do this change right away and merge it, just
using a static io priority. Makes it easier to build and work on an
external scheduler using priorities without throwing lots of rejects in
the core. It's not really a lot of work, either...

-- 
Jens Axboe