Any lightweight way for one thread to force another thread to suspend execution?

Any lightweight way for one thread to force another thread to suspend execution?

[Eric Smith]

Is there any method for one thread to force another thread to suspend
execution, then later let it resume, that is lighter weight than using
signals?  I don't need it to be portable to anything other than Linux.
The suspend needs to be synchronous, but resume doesn't.

What I've thought about so far (but haven't tested) for thread A
to suspend thread B is to use a signal and two pthread conditions:

    Thread A                   Thread B
    -----------------------    -----------------------
    send signal to B

    wait on a cond 1

                               enter signal handler

                               signal cond 1

                               wait on cond 2

    wake up
    do stuff that needs
    thread B suspended

    signal cond 2

    continue                   wake up


Thanks for any suggestions!
Eric Smith

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Andi Kleen]

"Eric Smith" <eric@brouhaha.com> writes:
>
> What I've thought about so far (but haven't tested) for thread A
> to suspend thread B is to use a signal and two pthread conditions:

Any such mechanism will need a syscall, and it's unlikely that
any syscall will get much cheaper than a kill(SIGSTOP) 

-Andi

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Eric Smith]

Andi wrote:
> Any such mechanism will need a syscall, and it's unlikely that
> any syscall will get much cheaper than a kill(SIGSTOP)

But is there a way for the process sending the SIGSTOP to wait until it
has taken effect?  I need a method to *synchronously* stop another
thread.  That's why I thought I probably needed something more
elaborate than SIGSTOP, though I'd like to minimize the number of
system calls required.

Thanks!
Eric Smith

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Bart Van Assche]

On Sun, Jun 22, 2008 at 3:55 AM, Eric Smith <eric@brouhaha.com> wrote:
> Andi wrote:
>> Any such mechanism will need a syscall, and it's unlikely that
>> any syscall will get much cheaper than a kill(SIGSTOP)
>
> But is there a way for the process sending the SIGSTOP to wait until it
> has taken effect?  I need a method to *synchronously* stop another
> thread.  That's why I thought I probably needed something more
> elaborate than SIGSTOP, though I'd like to minimize the number of
> system calls required.

Are you aware that SIGSTOP suspends all the threads in a process
instead of a single thread ? (Note: this is how the NPTL behaves. With
LinuxThreads it was possible to suspend a single thread via SIGSTOP,
but this was a violation of the POSIX standards.)

Bart.

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Jan Engelhardt]

On Sunday 2008-06-22 03:55, Eric Smith wrote:
>Andi wrote:
>> Any such mechanism will need a syscall, and it's unlikely that
>> any syscall will get much cheaper than a kill(SIGSTOP)
>
>But is there a way for the process sending the SIGSTOP to wait until it
>has taken effect?  I need a method to *synchronously* stop another
>thread.  That's why I thought I probably needed something more
>elaborate than SIGSTOP, though I'd like to minimize the number of
>system calls required.

When it is stopped, the process state changes to "T" (in ps and /proc).
Note that debugging a program with gdb or ptrace also puts it in the T state.

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Eric Smith]

Andi wrote about my inquire regarding a way to synchronously stop
another thread:
> Any such mechanism will need a syscall, and it's unlikely that
> any syscall will get much cheaper than a kill(SIGSTOP)

Unfortunately, as Bart pointed out, this stops ALL the threads
in a process (required for POSIX conformance).  However, it appears that
I may be able to do it by creating my threads using clone() with appropriate
parameters rather than pthread_create().

The next question, which I haven't yet been able to answer for myself
from staring at the kernel sources, is whether sending SIGSTOP is
synchronous.  In other words, when the kill() returns, is the target
thread already stopped?  Even on an SMP system where the thread may have
been executing concurrently on another processor?

If not, the next question will be whether there's a more efficient way
to wait for a process to enter the "T" state than busy-polling a file
in /proc.

Thanks,
Eric

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Mikael Pettersson]

Eric Smith writes:
 > Andi wrote about my inquire regarding a way to synchronously stop
 > another thread:
 > > Any such mechanism will need a syscall, and it's unlikely that
 > > any syscall will get much cheaper than a kill(SIGSTOP)
 > 
 > Unfortunately, as Bart pointed out, this stops ALL the threads
 > in a process (required for POSIX conformance).  However, it appears that
 > I may be able to do it by creating my threads using clone() with appropriate
 > parameters rather than pthread_create().
 > 
 > The next question, which I haven't yet been able to answer for myself
 > from staring at the kernel sources, is whether sending SIGSTOP is
 > synchronous.  In other words, when the kill() returns, is the target
 > thread already stopped?  Even on an SMP system where the thread may have
 > been executing concurrently on another processor?
 > 
 > If not, the next question will be whether there's a more efficient way
 > to wait for a process to enter the "T" state than busy-polling a file
 > in /proc.

SIGSTOP is unreliable since some other process may wake the
target up with a SIGCONT.

man ptrace(), see the PTRACE_ATTACH request.

(Presumably it will DTRT if you give it a tid not just a pid,
but I haven't tested that.)

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Bart Van Assche]

On Sat, Jun 21, 2008 at 1:54 AM, Eric Smith <eric@brouhaha.com> wrote:
> Is there any method for one thread to force another thread to suspend
> execution, then later let it resume, that is lighter weight than using
> signals?  I don't need it to be portable to anything other than Linux.
> The suspend needs to be synchronous, but resume doesn't.

Trying to suspend another thread synchronously is a bad idea because
this can easily trigger deadlocks. E.g. if you suspend a thread while
that thread holds a lock on a mutex, your application will deadlock.

Bart.

Re: Any lightweight way for one thread to force another thread to suspend execution?

[Eric Smith]

Bart wrote:
> Trying to suspend another thread synchronously is a bad idea because
> this can easily trigger deadlocks. E.g. if you suspend a thread while
> that thread holds a lock on a mutex, your application will deadlock.

I understand that and am willing to take responsibility for ensuring
that my application doesn't get into that situation.  Basically I'm
trying to port something that ran on a strict priority preemptive
kernel (on a single processor) to run on a Linux SMP system, so I
need a higher priority thread to completely preempt a lower priority
thread, even though the lower priority thread might be running on
another processor.

As far as I can tell, pthreads doesn't solve this problem, which is
unfortunately, since in other respects pthreads looks like it is
mostly suitable for use in porting code from other real-time kernels
to Linux.

It appears that some pthreads implementations add a nonstandard
pthread_suspend() that can do this.  What I'm looking for is a way
to efficiently implement pthrad_suspend() on Linux, ideally with
a minimum of system calls.

It seems to me that in principle, if the kernel provided a way to do
this, it should be able to suspend a thread that wasn't currently
executing merely by changing some of that thread's metadata, while
suspending a thread that is executing on another processor would be
more expensive since it would be necessary to force an interrupt
on that processor, etc.

Thanks,
Eric