Re: [RFC PATCHSET] workqueue: implement concurrency managed workqueue

[Tejun Heo <tj@kernel.org>]

Hello,

Avi Kivity wrote:
> On 10/01/2009 10:40 AM, Ingo Molnar wrote:
>> Ok, this looks fairly interesting - and the way you reused scheduler
>> classes to auto-regulate with no impact on regular performance is quite
>> an ingenious idea as well. (KVM's preempt notifiers should probably use
>> this trick too, instead of an ugly notifier in the scheduler hotpath)
>>
>> This mechanism could be used to implement threadlets/syslets too btw.,
>> and other forms of asynchronous IO.
>>    
> 
> In fact I've thought of implementing threadlets and concurrency-managed
> workqueues with preempt notifiers ;)

:-)

> Isn't a scheduling class overkill for two existing callbacks?  Note we
> can easily use a thread flag and __switch_to_xtra() to avoid the overhead.

The partial overriding only takes about seventy lines of code and is
conceptually trivial.  I don't think it's an overkill.

> For kvm, we don't want to force a specific scheduling class for vcpu
> threads, so we'd need infrastructure to create a new scheduling class
> out of an existing one to hook the two callbacks.  Seems like quite a
> lot of work, for something that is orthogonal to scheduling.
> 
> Tejun, would preempt notifiers work for your workqueues?  see bottom of
> include/linux/preempt.h.

I considered that but the thing is workqueue needs to know when a
thread wakes up not when it gets scheduled.  Of course we can add
another notifier op and call it from try_to_wake_up() but I really
didn't want to add yet another hook in a very hot path which will only
be useful for very small number of tasks but yet has to be called for
every operation and the sched_class mechanism means that we already
have hooks at all the interesting spots, so I think it's better to
make use of them instead of adding another set of callbacks.

Thanks.

-- 
tejun