Re: [Ksummit-2008-discuss] Delayed interrupt work, thread pools

[Steven Rostedt <rostedt@goodmis.org>]

On Wed, Jul 02, 2008 at 09:11:36AM -0500, James Bottomley wrote:
> 
> If you really need the full scheduling capabilities of threads, then it
> sounds like a threadpool is all you need (and we should just provide a
> unified interface).

Something like this may also be useful for the RT kernel as well. Being
able to push off tasks that we could prioritize would be greatly
beneficial.

Too bad we don't have a lighter task. Looking at the task_struct it
looks quite heavy, to be storing lots of threads. Perhaps we can clean
it up some time and remove out anything that would only be useful for
userspace threads. Not sure how much that would save us.

As for interrupt threads, those would help for some non-RT issues
(having a better desktop feel) but not for the issue that Ben has been
stating. I would be interested in knowing exactly what is needing to
handle a page fault inside the kernel.  If we need to do something for a
user space task, as soon as that task is found the work should be passed
to that thread.

> 
> Initially you were implying you'd prefer some type of non blockable
> workqueue (i.e. a workqueue that shifts to the next work item when and
> earlier item blocks).   I can see this construct being useful because it
> would have easier to use semantics and be more lightweight than a full
> thread spawn.  It strikes me we could use some of the syslets work to do
> this ... all the queue needs is an "next activation head", which will be
> the next job in the queue in the absence of blocking.  When a job
> blocks, syslets informs the workqueue and it moves on to the work on the
> "next activation head".  If a prior job unblocks, syslets informs the
> queue and it moves the "next activation head" to the unblocked job.
> What this is doing is implementing a really simple scheduler within a
> single workqueue, which I'm unsure is actually a good idea since
> schedulers are complex and tricky things, but it is probably worthy of
> discussion.

I think doing a "mini scheduler" inside a workgroup thread would be a
major hack.  We would have to have hooks into the normal scheduler to
let the mini-scheduler know something is blocking, and then have that
scheduler do some work. Not to mention that we need to handle
preemption.

Having a thread pool sounds much more reasonable and easier to
implement.

BTW, if something like this is implemented, I think that it should be a
replacement for softirqs and tasklets.

-- Steve