Re: [Ksummit-2008-discuss] Delayed interrupt work, thread pools

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Wed, 2008-07-02 at 16:00 -0400, Steven Rostedt wrote:
> On Wed, Jul 02, 2008 at 09:11:36AM -0500, James Bottomley wrote:
> > 
> > If you really need the full scheduling capabilities of threads, then it
> > sounds like a threadpool is all you need (and we should just provide a
> > unified interface).
> 
> Something like this may also be useful for the RT kernel as well. Being
> able to push off tasks that we could prioritize would be greatly
> beneficial.
> 
> Too bad we don't have a lighter task. Looking at the task_struct it
> looks quite heavy, to be storing lots of threads. Perhaps we can clean
> it up some time and remove out anything that would only be useful for
> userspace threads. Not sure how much that would save us.
> 
> As for interrupt threads, those would help for some non-RT issues
> (having a better desktop feel) but not for the issue that Ben has been
> stating. I would be interested in knowing exactly what is needing to
> handle a page fault inside the kernel.  If we need to do something for a
> user space task, as soon as that task is found the work should be passed
> to that thread.
> 
> > 
> > Initially you were implying you'd prefer some type of non blockable
> > workqueue (i.e. a workqueue that shifts to the next work item when and
> > earlier item blocks).   I can see this construct being useful because it
> > would have easier to use semantics and be more lightweight than a full
> > thread spawn.  It strikes me we could use some of the syslets work to do
> > this ... all the queue needs is an "next activation head", which will be
> > the next job in the queue in the absence of blocking.  When a job
> > blocks, syslets informs the workqueue and it moves on to the work on the
> > "next activation head".  If a prior job unblocks, syslets informs the
> > queue and it moves the "next activation head" to the unblocked job.
> > What this is doing is implementing a really simple scheduler within a
> > single workqueue, which I'm unsure is actually a good idea since
> > schedulers are complex and tricky things, but it is probably worthy of
> > discussion.
> 
> I think doing a "mini scheduler" inside a workgroup thread would be a
> major hack.  We would have to have hooks into the normal scheduler to
> let the mini-scheduler know something is blocking, and then have that
> scheduler do some work. Not to mention that we need to handle
> preemption.

Not necessarly ... a simplistic round robin is fine.

The work to detect the "am I being blocked" has already been done for
some of the aio patches, so I'm merely suggesting another use for it.

Isn't preemption an orthogonal problem ... it will surely exist even in
the threadpool approach?

> Having a thread pool sounds much more reasonable and easier to
> implement.

Easier to implement, yes.  Easier to program, unlikely, and coming with
a large amount of overhead, definitely.

> BTW, if something like this is implemented, I think that it should be a
> replacement for softirqs and tasklets.

James