[RFC] Workload type Vs Groups (Was: Re: [PATCH 02/20] blkio: Change CFQ to use CFS like queue time stamps)

[Vivek Goyal <vgoyal@redhat.com>]

On Wed, Nov 04, 2009 at 10:18:15PM +0100, Corrado Zoccolo wrote:
> Hi Vivek,
> On Wed, Nov 4, 2009 at 12:43 AM, Vivek Goyal <vgoyal@redhat.com> wrote:
> > o Previously CFQ had one service tree where queues of all theree prio classes
> >  were being queued. One side affect of this time stamping approach is that
> >  now single tree approach might not work and we need to keep separate service
> >  trees for three prio classes.
> >
> Single service tree is no longer true in cfq for-2.6.33.
> Now we have a matrix of service trees, with first dimension being the
> priority class, and second dimension being the workload type
> (synchronous idle, synchronous no-idle, async).
> You can have a look at the series: http://lkml.org/lkml/2009/10/26/482 .
> It may have other interesting influences on your work, as the idle
> introduced at the end of the synchronous no-idle tree, that provides
> fairness also for seeky or high-think-time queues.
> 

Hi All,

I am now rebasing my patches to for-2.6.33 branch. There are significant
number of changes in that branch, especially changes from corrado bring
in an interesting question.

Currently corrado has introduced the functinality of kind of grouping the
cfq queues based on workload type and gives the time slots to these sub
groups (sync-idle, sync-noidle, async).

I was thinking of placing groups on top of this model, so that we select
the group first and then select the type of workload and then finally 
the queue to run.

Corrodo came up with an interesting suggestion (in a private mail), that
what if we implement workload type at top and divide the share among
groups with-in workoad type.

So one would first select the workload to run and then select group
with-in workload and then cfq queue with-in group.

The advantage of this approach are.

- for sync-noidle group, we will not idle per group. We will idle only
  only at root level. (Well if we don't idle on the group once it becomes
  empty, we will not see fairness for group. So it will be fairness vs
  throughput call).

- It allows us to limit system wide share of workload type. So for
  example, one can kind of fix system wide share of async queues.
  Generally it might not be very prudent to allocate a group 50% of
  disk share and then that group decides to just do async IO and sync
  IO in rest of the groups suffer.

Disadvantage

- The definition of fairness becomes bit murkier. Now fairness will be 
  achieved for a group with-in the workload type. So if a group is doing
  IO of type sync-idle as well as sync-noidle and other group is doing
  IO of type only sync-noidle, then first group will get overall more
  disk time even if both the groups have same weight.

Looking for some feedback about which appraoch makes more sense before I 
write patches.

Thanks
Vivek