calling kthread_create() from interrupt thread

calling kthread_create() from interrupt thread

[Dean Nelson]

I'm working on a driver that needs to create threads that can sleep/block
for an indefinite period of time.

    . Can kthread_create() be called from an interrupt handler?

    . Is the cost of a kthread's creation/demise low enough so that one
      can, as often as needed, create a kthread that performs a simple
      function and exits?  Or is the cost too high for this?

Thanks,
Dean

Re: calling kthread_create() from interrupt thread

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 516 bytes --]

On Tue, 2004-06-15 at 19:42, Dean Nelson wrote:
> I'm working on a driver that needs to create threads that can sleep/block
> for an indefinite period of time.
> 
>     . Can kthread_create() be called from an interrupt handler?

no

> 
>     . Is the cost of a kthread's creation/demise low enough so that one
>       can, as often as needed, create a kthread that performs a simple
>       function and exits?  Or is the cost too high for this?

for that we have keventd in 2.4, work queues in 2.6


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: calling kthread_create() from interrupt thread

[Dean Nelson]

On Tue, Jun 15, 2004 at 07:49:37PM +0200, Arjan van de Ven wrote:
> On Tue, 2004-06-15 at 19:42, Dean Nelson wrote:
> > I'm working on a driver that needs to create threads that can sleep/block
> > for an indefinite period of time.
> > 
> >     . Can kthread_create() be called from an interrupt handler?
> 
> no
> 
> > 
> >     . Is the cost of a kthread's creation/demise low enough so that one
> >       can, as often as needed, create a kthread that performs a simple
> >       function and exits?  Or is the cost too high for this?
> 
> for that we have keventd in 2.4, work queues in 2.6

As mentioned above, it is possible for this "simple" function to sleep/block
for an indefinite period of time. I was under the impression that one
couldn't block a work queue thread for an indefinite period of time. Am
I mistaken?

Thanks,
Dean

Re: calling kthread_create() from interrupt thread

[Jesse Barnes]

On Tuesday, June 15, 2004 2:05 pm, Dean Nelson wrote:
> As mentioned above, it is possible for this "simple" function to
> sleep/block for an indefinite period of time. I was under the impression
> that one couldn't block a work queue thread for an indefinite period of
> time. Am I mistaken?

For tasklets and softirqs you're not allowed to sleep, but I think it's ok for 
work queues.

Jesse

Re: calling kthread_create() from interrupt thread

[Richard B. Johnson]

On Tue, 15 Jun 2004, Dean Nelson wrote:

> On Tue, Jun 15, 2004 at 07:49:37PM +0200, Arjan van de Ven wrote:
> > On Tue, 2004-06-15 at 19:42, Dean Nelson wrote:
> > > I'm working on a driver that needs to create threads that can sleep/block
> > > for an indefinite period of time.
> > >
> > >     . Can kthread_create() be called from an interrupt handler?
> >
> > no
> >
> > >
> > >     . Is the cost of a kthread's creation/demise low enough so that one
> > >       can, as often as needed, create a kthread that performs a simple
> > >       function and exits?  Or is the cost too high for this?
> >
> > for that we have keventd in 2.4, work queues in 2.6
>
> As mentioned above, it is possible for this "simple" function to sleep/block
> for an indefinite period of time. I was under the impression that one
> couldn't block a work queue thread for an indefinite period of time. Am
> I mistaken?
>
> Thanks,
> Dean
>

If you make a kernel thread, it can sleep forever if it wants, you
can wake it up with wake_up_interruptible() from an interrupt after
you have laid out the work you want it to do. That kernel thread
has access to all your kernel data space, plus can spin-lock to
prevent an interrupt from changing things in critical sections,
etc.

It's the greatest thing since sliced bread.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.26 on an i686 machine (5570.56 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: calling kthread_create() from interrupt thread

[Robin Holt]

On Tue, Jun 15, 2004 at 02:15:34PM -0400, Richard B. Johnson wrote:
> On Tue, 15 Jun 2004, Dean Nelson wrote:
> 
> > On Tue, Jun 15, 2004 at 07:49:37PM +0200, Arjan van de Ven wrote:
> > > On Tue, 2004-06-15 at 19:42, Dean Nelson wrote:
> > > > I'm working on a driver that needs to create threads that can sleep/block
> > > > for an indefinite period of time.
> > > >
> > > >     . Can kthread_create() be called from an interrupt handler?
> > >
> > > no
> > >
> > > >
> > > >     . Is the cost of a kthread's creation/demise low enough so that one
> > > >       can, as often as needed, create a kthread that performs a simple
> > > >       function and exits?  Or is the cost too high for this?
> > >
> > > for that we have keventd in 2.4, work queues in 2.6
> >
> > As mentioned above, it is possible for this "simple" function to sleep/block
> > for an indefinite period of time. I was under the impression that one
> > couldn't block a work queue thread for an indefinite period of time. Am
> > I mistaken?
> >
> > Thanks,
> > Dean
> >
> 
> If you make a kernel thread, it can sleep forever if it wants, you
> can wake it up with wake_up_interruptible() from an interrupt after
> you have laid out the work you want it to do. That kernel thread
> has access to all your kernel data space, plus can spin-lock to
> prevent an interrupt from changing things in critical sections,
> etc.
> 
> It's the greatest thing since sliced bread.
> 

The problem Dean is trying to address is as follows:

We receive an interrupt.  The interrupt handler determines that some work
needs to be done.  Part of that work to be done may result in the process
needing to go to sleep waiting for a resource to become available.

Currently, the interrupt handler wakes a thread sleeping on a
wait_event_interruptible().  This wakeup is taking approx 35uSec.  Dean
is looking for a lower latency means of doing the wakeup.

Thanks,
Robin

Re: calling kthread_create() from interrupt thread

[Dean Nelson]

On Tue, Jun 15, 2004 at 07:49:37PM +0200, Arjan van de Ven wrote:
> On Tue, 2004-06-15 at 19:42, Dean Nelson wrote:
> > I'm working on a driver that needs to create threads that can sleep/block
> > for an indefinite period of time.
> > 
> >     . Can kthread_create() be called from an interrupt handler?
> 
> no
> 
> > 
> >     . Is the cost of a kthread's creation/demise low enough so that one
> >       can, as often as needed, create a kthread that performs a simple
> >       function and exits?  Or is the cost too high for this?
> 
> for that we have keventd in 2.4, work queues in 2.6

Can an interrupt handler setup a work_struct structure, call schedule_work()
and then simply return, not waiting around for the work queue event to
complete?

Thanks,
Dean

Re: calling kthread_create() from interrupt thread

[Roland Dreier]

    Dean> Can an interrupt handler setup a work_struct structure, call
    Dean> schedule_work() and then simply return, not waiting around
    Dean> for the work queue event to complete?

Yes (as long as the work_struct structure is not freed at the end of
the interrupt handler or something like that).

 - Roland

Re: calling kthread_create() from interrupt thread

[Roland Dreier]

    Robin> The problem Dean is trying to address is as follows:

    Robin> We receive an interrupt.  The interrupt handler determines
    Robin> that some work needs to be done.  Part of that work to be
    Robin> done may result in the process needing to go to sleep
    Robin> waiting for a resource to become available.

    Robin> Currently, the interrupt handler wakes a thread sleeping on
    Robin> a wait_event_interruptible().  This wakeup is taking approx
    Robin> 35uSec.  Dean is looking for a lower latency means of doing
    Robin> the wakeup.

Could the interrupt handler attempt to do the work directly, and only
defer things if it determines it needs to sleep for the resource?  It
seems if your are sleeping waiting for something to free up, then
your latency is shot anyway.

 - Roland

Re: calling kthread_create() from interrupt thread

[Chris Wright]

* Robin Holt (holt@sgi.com) wrote:
> Currently, the interrupt handler wakes a thread sleeping on a
> wait_event_interruptible().  This wakeup is taking approx 35uSec.  Dean
> is looking for a lower latency means of doing the wakeup.

I can't imagine adding thread creation, etc to the mix is going to improve
latency.

thanks,
-chris
-- 
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net

Re: calling kthread_create() from interrupt thread

[Chris Wright]

* Dean Nelson (dcn@sgi.com) wrote:
> Can an interrupt handler setup a work_struct structure, call schedule_work()
> and then simply return, not waiting around for the work queue event to
> complete?

Yes, that's a fundamental feature.  However, it's not so nice to use the
generic events workqueue for really long sleeping work since it can stall
the entire queue.  You might consider your own queue.

thanks,
-chris
-- 
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net

Re: calling kthread_create() from interrupt thread

[Andi Kleen]

Dean Nelson <dcn@sgi.com> writes:
>
> As mentioned above, it is possible for this "simple" function to sleep/block
> for an indefinite period of time. I was under the impression that one
> couldn't block a work queue thread for an indefinite period of time. Am
> I mistaken?

You could create your own work queue upfront. Blocking an generic workqueue 
for a long time is indeed nasty.

-Andi

Re: calling kthread_create() from interrupt thread

[Jeff Garzik]

Jesse Barnes wrote:
> On Tuesday, June 15, 2004 2:05 pm, Dean Nelson wrote:
> 
>>As mentioned above, it is possible for this "simple" function to
>>sleep/block for an indefinite period of time. I was under the impression
>>that one couldn't block a work queue thread for an indefinite period of
>>time. Am I mistaken?
> 
> 
> For tasklets and softirqs you're not allowed to sleep, but I think it's ok for 
> work queues.


Dean is correct and incorrect ;-)

If you are using schedule_work() or schedule_task(), blocking for 
extended periods of time would be very undesirable.  We see this on 
occasion in 2.6 uniprocessor, where a long-running keventd task may 
block a console or tty update.

In 2.6, the solution is easy... create your own private workqueue.  No 
such solution in 2.4 (though I would argue that workqueues would help 
drivers, if accepted into 2.4 at this late stage).

	Jeff

Re: calling kthread_create() from interrupt thread

[Rusty Russell]

On Wed, 2004-06-16 at 05:01, Robin Holt wrote:
> We receive an interrupt.  The interrupt handler determines that some work
> needs to be done.  Part of that work to be done may result in the process
> needing to go to sleep waiting for a resource to become available.
> 
> Currently, the interrupt handler wakes a thread sleeping on a
> wait_event_interruptible().  This wakeup is taking approx 35uSec.  Dean
> is looking for a lower latency means of doing the wakeup.

The best approach is, as suggested in this thread, to have a fastpath
which is called from interrupt handler, which fails if it needs to
sleep; in that case you back off to your own workqueue.  It'd look
something like:

	static DEFINE_PER_CPU(struct work_struct slow_work);
	static workqueue_struct *wq;

	irqreturn_t do_interrupt(...)
	{
		if (!fast_irq_handle())
			queue_work(wq, &__get_cpu_var(slow_work));
	}

	static void do_slow_work(void *unused)
	{
		...
	}

	static int __init init(void)
	{
		int cpu;
		wq = create_workqueue("drivername", 0);
		for_each_cpu(cpu)
			PREPARE_WORK(&per_cpu(slow_work, cpu),
				     do_slow_work, NULL);
	}

You need to come up with a mechanism to pass details from the interrupt
handler to the do_slow_work() fn, probably a separate queue or array
which do_slow_work() will need to disable irqs to access.  queue_work
will not requeue the work_struct if it's already pending, your
do_slow_work() needs to handle all the requests which are waiting.

Hope that helps,
Rusty.
-- 
Anyone who quotes me in their signature is an idiot -- Rusty Russell