From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: elevator priorities vs. full request queues Date: Tue, 22 Jun 2004 09:48:52 +0200 Sender: linux-fsdevel-owner@vger.kernel.org Message-ID: <20040622074852.GW12881@suse.de> References: <20040622012502.B1325@almesberger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: linux-fsdevel@vger.kernel.org Return-path: Received: from ns.virtualhost.dk ([195.184.98.160]:12472 "EHLO virtualhost.dk") by vger.kernel.org with ESMTP id S261205AbUFVHsz (ORCPT ); Tue, 22 Jun 2004 03:48:55 -0400 To: Werner Almesberger Content-Disposition: inline In-Reply-To: <20040622012502.B1325@almesberger.net> List-Id: linux-fsdevel.vger.kernel.org On Tue, Jun 22 2004, Werner Almesberger wrote: > Another issue regarding priorities: let's suppose we have "high" > and "low" priority requests. Those priorities are indicated by > some means, and do not depend on CPU scheduler priorities. Let's > say that the elevator always handles high priority requests > first, unless barriers dictate it to do otherwise. > > So far so good. Next, let's suppose that the request queue is > filling up. Now all the processes doing IO will compete for > request slots, and the outcome of this competition is > determined by the CPU scheduler - which is blissfully unaware > of those request priorities. > > Has anybody thought of a solution for this problem ? Well yes, different parties have worked on io priorities in the past. Request allocation is a important part of that. > I can think of a few possible approaches, but they're all > relatively intrusive or plain ugly. A few of them: > > 1) temporarily raise the CPU priority of processes waiting > for request queue slots > 2) implement some priority scheme in ll_rw_blk.c > 3) give elv_may_queue more information about what the request > will look like, so that it can decide to accept "high" > priority requests even if the queue is officially full > 4) allow elv_add_request to refuse requests, e.g. to keep > some reserve for "high" priority requests > > 1) and 2) would have the advantage that they also work if > requests are delayed due to a lack of memory. However, in this > case, all bets are off anyway, so I wouldn't worry too much > about it. > > 3) and 4) have the nice property of keeping all the priority > logic entirely in the elevator. For 4), one would also have to > force an interface change (besides giving elv_add_request a > return value), to make sure there are no enqueuers left in the > kernel or elsewhere which don't know that elv_add_request may > now fail. #4 doesn't work for various reasons, the obvious being that the request is already allocated at this point. Or request could be coming from outside get_request(). CFQ with priorities used a combination of 2+3. It seems you haven't looked any of this up, maybe that would be a good starting point. You can do this internally in your io scheduler in several ways. CFQ used per-process (-group) priorities, so it would get the priority from there. A bio has room for priority info, so you could also pass it in from submit_bio() -> __make_request() -> get_request() -> may_queue(). > Opinions ? #1 can complement a solution of 2+3, it's not a solution in itself imo. You absolutely need some way of reserving requests for some processes, or denying requests for other and priorities alone does not accomplish that. #4 is a definite no-go. Maybe you meant to specify a different function, I don't think your suggestion makes any sense. -- Jens Axboe