Two questions about scheduling and threading.

Two questions about scheduling and threading.

[Brian McGrew]

We're using the 2.6.16.16 kernel in an almost real time, multi processor and
multi threaded environment.

When I start my system for the first time, I start one thread for each
processor/core in the machine (is this the correct thing to do)?  These
threads set a busy flag, go to work and then go to sleep.  I put everything
to sleep as opposed to killing the threads because it saves me on average of
about 400ms each time around.

My problem is, and, it is very reproducable, that if CPU0 is at 100%, none
of my threads see the wakeup!  It doesn't matter what the other CPU's are
doing, if they're all at 0 or 100%, but if CPU0 is 100, I'm toast.  Is there
anyway around this?

Also, I know that we're supposed to sit back and let the scheduler do all
the work for us; but, in the 2.6.16.16 kernel, is there a way to assign a
specific thread and/or process to a designated processor???  I really need
to be able to do this because even with the preemptive scheduling, I'm still
real-time and it's not quite real-time enough!

Thanks!

-brian

Re: Two questions about scheduling and threading.

[Dan Noe]

On 5/8/2008 11:15, Brian McGrew wrote:
> Also, I know that we're supposed to sit back and let the scheduler do all
> the work for us; but, in the 2.6.16.16 kernel, is there a way to assign a
> specific thread and/or process to a designated processor???  I really need
> to be able to do this because even with the preemptive scheduling, I'm still
> real-time and it's not quite real-time enough!

There are some functions to set CPU affinity from user space:

sched_setaffinity()
sched_getaffinity()

And a user space program:

taskset

All have decent man pages.  They'll allow you to pin a process to a 
particular CPU, or set the normal behavior, which is to try and keep a 
process on the same CPU as long as possible.  There are some other 
utilities mentioned in the related section of the manpages - they allow 
you to tweak scheduler behavior.

Hope that helps.

Cheers,
Dan

-- 
                     /--------------- - -  -  -   -   -
                     |  Dan Noe
                     |  http://isomerica.net/~dpn/

RE: Two questions about scheduling and threading.

[David Schwartz]

> When I start my system for the first time, I start one thread for each
> processor/core in the machine (is this the correct thing to do)?  These
> threads set a busy flag, go to work and then go to sleep.  I put
> everything
> to sleep as opposed to killing the threads because it saves me on
> average of
> about 400ms each time around.

That's not unreasonable. However, you may wish to create a few extra
threads. Otherwise, if one thread is blocked on I/O, you can't use all the
CPUs.

> My problem is, and, it is very reproducable, that if CPU0 is at 100%, none
> of my threads see the wakeup!  It doesn't matter what the other CPU's are
> doing, if they're all at 0 or 100%, but if CPU0 is 100, I'm
> toast.  Is there
> anyway around this?

Sounds like a bug. What does your wakeup code look like? Do you put your
threads to sleep blocked on a condition variable? Is the c.v. code correctly
using a predicate, that's something easy to screw up. It's like 'select',
there's a dozen classic mistakes and someone often makes one or two of them
and their code still soemtimes works.

> Also, I know that we're supposed to sit back and let the scheduler do all
> the work for us; but, in the 2.6.16.16 kernel, is there a way to assign a
> specific thread and/or process to a designated processor???  I really need
> to be able to do this because even with the preemptive
> scheduling, I'm still
> real-time and it's not quite real-time enough!

Big mistake. If you bind threads to CPUs and the thread that gets a wakeup
is assigned to a CPU that's busy, the job that thread was going to do will
have to wait, while other CPUs sit idle.

DS