Re: [PATCH 0/3] BFQ I/O Scheduler (second take)

Re: [PATCH 0/3] BFQ I/O Scheduler (second take)

[Li Zefan]

CC: Linux Containers <containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org>
    Vivek Goyal <vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

Fabio Checconi wrote:
> Hi Jens, hi all,
> 

Hi Fabio,

>   this is an updated version of the patchset introducing the BFQ (Budget
> Fair Queueing) I/O scheduler.  It is based on CFQ and it has been
> developed trying to improve the predictability and the fairness of the
> service prvided, without penalizing the aggregate throughput.
> 
> This new version hopefully addresses the issues pointed out after the
> first submission[1]:
> 
>   1) it introduces a bitmap to notify priority changes to the bfq/cfq;
>   2) it introduces a timeout to put an higher limit to the time a
>       task can take to consume its budget;
>   3) it supports hierarchical scheduling.
> 

So this is another cgroup-aware block i/o controller. ;)

People are sending different proposals for bio controller. A summary
on this can be found here:
	http://marc.info/?l=linux-kernel&m=121798534716673&w=2

And a new proposal sent by Vivek several days ago:
	http://lkml.org/lkml/2008/11/6/227

And there are some discussions about can we modify the elevator layer
and create a common layer so we can provide group control for all
the 4 io schedulers. (in the above mail thread)

You may share some ideas with us.

> With the introduction of a budget timeout, each task is still assigned
> a budget measured in number of sectors instead of amount of time,
> but a budget is not served to its exhaustion if it's taking too much
> to complete.  Budgets are still scheduled using a modified version
> of WF2Q+.
> 
> Using huge values for max_budget turns the scheduler into a pure
> time-domain proportional share one, and the timeout controls the duration
> of the time quanta.
> 
> WRT the support for hierarchical scheduling, that is being discussed
> so much in these days, this scheduler does not add anything new, it
> supports proportional share distribution of disk bandwidth among
> full cgroup hierarchies; the mapping from requests to cgroups is
> correct only in case of synchronous reads.
> 
> The overall diffstat is the following:
> 
>  block/Kconfig.iosched         |   25 +
>  block/Makefile                |    1 +
>  block/bfq-cgroup.c            |  743 +++++++++++++++
>  block/bfq-ioc.c               |  375 ++++++++
>  block/bfq-iosched.c           | 2010 +++++++++++++++++++++++++++++++++++++++++
>  block/bfq-sched.c             |  950 +++++++++++++++++++
>  block/bfq.h                   |  514 +++++++++++
>  block/blk-ioc.c               |   27 +-
>  block/cfq-iosched.c           |   10 +-
>  fs/ioprio.c                   |    9 +-
>  include/linux/cgroup_subsys.h |    6 +
>  include/linux/iocontext.h     |   18 +-
>  12 files changed, 4669 insertions(+), 19 deletions(-)
> 
> For a complete description of the algorithm and of its properties
> please refer to [2], (where extensive experiments on older hw can
> also be found), and for some performance measurements and implementation
> details, to [3].
> 
> [1] http://lkml.org/lkml/2008/4/1/234
> [2] http://algo.ing.unimo.it/people/paolo/disk_sched/
> [3] http://feanor.sssup.it/~fabio/linux/bfq/
> 
> As usual, any kind of feedback will be greatly appreciated, thanks
> in advance.
> 
> Fabio Checconi, Paolo Valente

Re: [PATCH 0/3] BFQ I/O Scheduler (second take)

[Fabio Checconi]

Hi,

> From: Li Zefan <lizf-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
> Date: Wed, Nov 12, 2008 10:48:00AM +0800
>
> CC: Linux Containers <containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org>
>     Vivek Goyal <vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> 
...
> So this is another cgroup-aware block i/o controller. ;)
> 
> People are sending different proposals for bio controller. A summary
> on this can be found here:
> 	http://marc.info/?l=linux-kernel&m=121798534716673&w=2
> 
> And a new proposal sent by Vivek several days ago:
> 	http://lkml.org/lkml/2008/11/6/227
> 
> And there are some discussions about can we modify the elevator layer
> and create a common layer so we can provide group control for all
> the 4 io schedulers. (in the above mail thread)
> 
> You may share some ideas with us.
> 

  yes, BFQ is another block I/O controller, and the ongoing discussion
is very interesting, as the other patchsets proposed.

BFQ deals with hierarchical scheduling in the same way as the proposed
CFQ extensions do, and it does not handle the I/O traking issue, as we
considered it not belonging directly to the elevator layer.

Talking about what may be interesting from the controller POV, it supports
arbitrary hierarchies (not only two layers) and ioprio classes for cgroups.
It can be easily extended to support any I/O tracking mechanism and
per-device ioprio_class/ioprio (it's just about choosing a sane interface
for that).

The main reasons why we turned BFQ into a cgroup controller were:
  - the algorithm was quite easy to adapt, and in a hierarchical
    context the advantages of using WF2Q+ over RR are more relevant.
  - Being a new scheduler, we had some freedom experimenting with it.
  - Avi asked for it :)

I'd like to stress the fact that the I/O controller thing is _not_ the
main feature of BFQ.  At least at this point of its development, we are
more interested in understanding if the differences it introduces WRT
CFQ (namely, the mixed time-service domain approach, WF2Q+ scheduling,
the feedback on budget lengths, etc.) are of interest or not for the
comminity and/or are worth the effort of a new I/O scheduler.