Embedded Linux, pthreads and scheduling

Embedded Linux, pthreads and scheduling

[Stephen Williams]

[This is probably not unique to ppc linux, but it is imbedded
and I am running linux on an embedded pc in this case...]

I have a multi-threaded (pthreads) application running on an
embedded PPC. One of the threads operates a scanner video input,
and I want to give it (and only it) high priority, so that if
a device driver wakes it up, it is scheduled as close to "now"
as possible.

I code the thread in question to capp pthread_setschedparam
to set its policy to SCHED_FIFO and the priority to some
number >0. I'm using glibc-ppc_4xx-2.3.1 as packaged in the
Denx ELDK-3.0. Can I expect this to work? Is any priority >0
enough to get it priority over other threads, or do I need
to give it a high (i.e. 99) priority to get past the sliding
priorities of threads with default scheduling?

For the particular case I'm seeing, it seems to *not* have
any effect. My interrupt handler is activated (I see on the
scope) and in the first few cases the response is immediate,
but sometimes the response is delayed significantly. Where
I look for the problem may depend on whether the collective
understanding is that the SCHED_FIFO has the effect I expect.

-- 
Steve Williams                "The woods are lovely, dark and deep.
steve at icarus.com           But I have promises to keep,
http://www.icarus.com         and lines to code before I sleep,
http://www.picturel.com       And lines to code before I sleep."

Re: Embedded Linux, pthreads and scheduling

[Jaap-Jan Boor]

Steve,

On 30-sep-04, at 19:39, Stephen Williams wrote:

>
> [This is probably not unique to ppc linux, but it is imbedded
> and I am running linux on an embedded pc in this case...]
>
> I have a multi-threaded (pthreads) application running on an
> embedded PPC. One of the threads operates a scanner video input,
> and I want to give it (and only it) high priority, so that if
> a device driver wakes it up, it is scheduled as close to "now"
> as possible.
>
> I code the thread in question to capp pthread_setschedparam
> to set its policy to SCHED_FIFO and the priority to some
> number >0. I'm using glibc-ppc_4xx-2.3.1 as packaged in the
> Denx ELDK-3.0. Can I expect this to work? Is any priority >0
> enough to get it priority over other threads, or do I need
> to give it a high (i.e. 99) priority to get past the sliding
> priorities of threads with default scheduling?

yes, if you run this program as root a priority >1 and class
SCHED_FIFO should work as expected.

>
> For the particular case I'm seeing, it seems to *not* have
> any effect. My interrupt handler is activated (I see on the
> scope) and in the first few cases the response is immediate,
> but sometimes the response is delayed significantly.

Possible. What kernel version do you use? In my experience
a 2.4.x kernel with O(1) scheduler, preemptible kernel patch and low
latency patch still have significant delays (> 5 ms) sometimes.
A 2.4 kernel without these patches can have much longer response times.
I didn't experiment with the 2.6 kernel on this particular
system yet but 2.6 includes the O(1) scheduler and preemtible
kernel patch.

By the way, do you not share a lock with a lower priority thread?
If the lower priority thread has the lock, your high priority
thread needs to wait until it's finished (unlocks).

Jaap-Jan

> Where
> I look for the problem may depend on whether the collective
> understanding is that the SCHED_FIFO has the effect I expect.
>
> -- 
> Steve Williams                "The woods are lovely, dark and deep.
> steve at icarus.com           But I have promises to keep,
> http://www.icarus.com         and lines to code before I sleep,
> http://www.picturel.com       And lines to code before I sleep."
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded

Re: Embedded Linux, pthreads and scheduling

[Stephen Williams]

Jaap-Jan Boor wrote:
> Steve,
> 
> On 30-sep-04, at 19:39, Stephen Williams wrote:
>> I have a multi-threaded (pthreads) application running on an
>> embedded PPC. One of the threads operates a scanner video input,
>> and I want to give it (and only it) high priority, so that if
>> a device driver wakes it up, it is scheduled as close to "now"
>> as possible.

> yes, if you run this program as root a priority >1 and class
> SCHED_FIFO should work as expected.

The program is run by root and I'm using the pthread interface
to set the thread policy.

>>
>> For the particular case I'm seeing, it seems to *not* have
>> any effect. My interrupt handler is activated (I see on the
>> scope) and in the first few cases the response is immediate,
>> but sometimes the response is delayed significantly.
> 
> 
> Possible. What kernel version do you use? In my experience
> a 2.4.x kernel with O(1) scheduler, preemptible kernel patch and low
> latency patch still have significant delays (> 5 ms) sometimes.
> A 2.4 kernel without these patches can have much longer response times.
> I didn't experiment with the 2.6 kernel on this particular
> system yet but 2.6 includes the O(1) scheduler and preemtible
> kernel patch.

This is linuxppc-2.4 from bitkeeper. The last pull I did was a
few months ago, I think.

The process environment should be under control. There are a
half dozen idle daemons (init, boa, syslogd, mostly busybox)
and about a dozen threads in the application at hand. The
network and serial console should be idle during the test,
only my hardware should be going.

> By the way, do you not share a lock with a lower priority thread?
> If the lower priority thread has the lock, your high priority
> thread needs to wait until it's finished (unlocks).

The program is working with in-house devices, I've written the
drivers. The device driver can be used in a pipeline-like fashion,
so the driver itself can be used for most synchronization. In
fact, The thread in question synchronizes entirely through the
driver. I guess that means its a candidate for a seperate process,
eh?

But that means, for this thread, all the synchronization is inside
the kernel. I can also see that there *should* be no cause to block
on anything at the instant I see the delay, hence my puzzlement.

Of course, I'm processing 24bit color images at 30MPixels/sec,
so I might well be hitting *PCI* scheduling issues as easily as
anything else.
-- 
Steve Williams                "The woods are lovely, dark and deep.
steve at icarus.com           But I have promises to keep,
http://www.icarus.com         and lines to code before I sleep,
http://www.picturel.com       And lines to code before I sleep."