RE: [Xenomai-help] number-crunching execution jitter

RE: [Xenomai-help] number-crunching execution jitter

[Dominique.RAGOT]

Stephane Fillod wrote:
> -----Message d'origine-----
> De : Fillod Stephane [mailto:stephane.fillod@domain.hid]
> Envoye : vendredi 4 novembre 2005 13:30
> A : Dominique.RAGOT@fr.thalesgroup.com; xenomai@xenomai.org
> Objet : RE: [Xenomai-help] number-crunching execution jitter
> 
> 
> Dominique.RAGOT wrote:
> >I am using Xenomai 2.0 for a number-crunching application on
> bi-processor x86 SMP and using the Posix skin.
> >
> >There are 2 tasks on each processor that are awaken every 20ms and
> execute for 16ms.
> >When there are no perturbations on the machine, the jitter on tasks
> wakeup and execution time is acceptable (approx. 5%).
> >
> >However, when adding a simple load such as find / -print 
> redirected to
> /dev/null, the execution jitter is much worse (more than 100% in some
> cases), whereas wakeup jitter remains good. My conclusion is that the
> tasks are preempted when running. Is it right ?
> 
> IMHO, there are plenty of other possible reasons.
> What *measurement* result makes you conclude it is preemption which
> induces
> execution jitter?

The system is not connected to anything else than keyboard, mouse and display screen and is booted in single-user mode. The only visible userland tasks are init, udevd and the current shell. When the tests are running I do not touch anything and display output is redirected to /dev/null. What are the other possible reasons remaining in this case ?

In the meantime I modified the benchmark with a call to the routine pthread_set_mode_np(0, PTHREAD_SHIELD) immediately after thread is created, and observes a huge reduction in execution jitter, but at the expense of wakeup jitter. So I concluded the interrupt shield was definitely not doing much without this call. Am I wrong ?

> 
> Rem: I assume your RT tasks are free of non-RT Linux syscalls.
> 

RT tasks are doing mathematical operations and calls to Xenomai POSIX skin routines. How can I make sure no non-RT Linux syscall is ever called ?

> >I suspect the interrupt shield to be inactive (although I see it in
> /proc/adeos). How could it be activated with Posix skin ? How could it
> be monitored ?
> 
> Subtract the number of interrupts handlers processed when 
> leaving your 
> RT task by the number when entering. This measurement will 
> tell you if 
> the interrupt shield works as expected or not.

How do you obtain these numbers ? Is there any system call usable or shall I use the information in /proc.

> 
> A tool like LTT (not available now with Xenomai?) would have been able
> to tell what's happening during the 20ms period.
> 
> My random guess is you're maybe in the need of a "cache-shield", 
> a "tlb-shield" and/or a "DMA-shield". All those shields (tm) :-).

Sorry, I do not understand your point here. If this stuff is trademark, (tm) as you write it, I doubt it has ever been incorporated in any Xenomai or Fusion release. Where can I get it anyway ? Google was not my friend on this !

> 
> Measurements can be done using OProfile, with appropriate event
> selection.
> 

OK, I will try it.

> 
> >PS: too bad the cruncher will not survive 2.0. Would you consider
> including POSIX tests in the new 2.1 release ? I did not see 
> any in the
> 2.0.
> 
> I'm sure the Xenomai committer team will enjoy your patches ;-)
> 

This is why I asked. If number crunching is no longer in the scope it is worth knowing. Are you a member of this team ?

> 
> Best Regards,
> -- 
> Stephane
>

Best Regards,
--
Dominique

Re: [Xenomai-help] number-crunching execution jitter

[Philippe Gerum]

Dominique.RAGOT@fr.thalesgroup.com wrote:
> Stephane Fillod wrote:
> 
>>-----Message d'origine-----
>>De : Fillod Stephane [mailto:stephane.fillod@domain.hid]
>>Envoye : vendredi 4 novembre 2005 13:30
>>A : Dominique.RAGOT@fr.thalesgroup.com; xenomai@xenomai.org
>>Objet : RE: [Xenomai-help] number-crunching execution jitter
>>
>>
>>Dominique.RAGOT wrote:
>>
>>>I am using Xenomai 2.0 for a number-crunching application on
>>
>>bi-processor x86 SMP and using the Posix skin.
>>
>>>There are 2 tasks on each processor that are awaken every 20ms and
>>
>>execute for 16ms.
>>
>>>When there are no perturbations on the machine, the jitter on tasks
>>
>>wakeup and execution time is acceptable (approx. 5%).
>>
>>>However, when adding a simple load such as find / -print 
>>
>>redirected to
>>/dev/null, the execution jitter is much worse (more than 100% in some
>>cases), whereas wakeup jitter remains good. My conclusion is that the
>>tasks are preempted when running. Is it right ?
>>
>>IMHO, there are plenty of other possible reasons.
>>What *measurement* result makes you conclude it is preemption which
>>induces
>>execution jitter?
> 
> 
> The system is not connected to anything else than keyboard, mouse and display screen and is booted in single-user mode. The only visible userland tasks are init, udevd and the current shell. When the tests are running I do not touch anything and display output is redirected to /dev/null. What are the other possible reasons remaining in this case ?
> 
> In the meantime I modified the benchmark with a call to the routine pthread_set_mode_np(0, PTHREAD_SHIELD) immediately after thread is created, and observes a huge reduction in execution jitter, but at the expense of wakeup jitter. So I concluded the interrupt shield was definitely not doing much without this call. Am I wrong ?
> 

You've basically demonstrated that the interrupt shield works by preventing 
Linux-targeted interrupts from preempting your number crunching thread. However, 
interrupt shielding induces the overhead of filtering events through an 
additional intermediate Adeos domain. The tradeoff between shielding overhead 
and execution jitter might be in favour of the latter for intensive computing 
tasks that need deterministic execution times, but it's clearly the other way 
around for event-driven tasks that are bound to the shortest possible wake up 
latencies.

Since Ice Nine (v0.9), latency-bound tasks are explicitely favoured by default, 
which means that the interrupt shield support is off at build-time unless 
otherwise specified, and threads need to explicitely activate it when such 
support is compiled in. It's quite not like saying that the support is not there 
anymore, it just happens to not be the default.

> 
>>Rem: I assume your RT tasks are free of non-RT Linux syscalls.
>>
> 
> 
> RT tasks are doing mathematical operations and calls to Xenomai POSIX skin routines. How can I make sure no non-RT Linux syscall is ever called ?
> 

Search the various list archives for SIGXCPU and the T_WARNSW bit of the native 
interface. The POSIX skin exports the same support called PTHREAD_WARNSW.

> 
>>>I suspect the interrupt shield to be inactive (although I see it in
>>
>>/proc/adeos). How could it be activated with Posix skin ? How could it
>>be monitored ?
>>
>>Subtract the number of interrupts handlers processed when 
>>leaving your 
>>RT task by the number when entering. This measurement will 
>>tell you if 
>>the interrupt shield works as expected or not.
> 
> 
> How do you obtain these numbers ? Is there any system call usable or shall I use the information in /proc.
> 

/proc/xenomai/sched. A 's' bit in the status field of a thread tells you that 
the shield is operating for such thread.

> 
>>A tool like LTT (not available now with Xenomai?) would have been able
>>to tell what's happening during the 20ms period.
>>
>>My random guess is you're maybe in the need of a "cache-shield", 
>>a "tlb-shield" and/or a "DMA-shield". All those shields (tm) :-).
> 
> 
> Sorry, I do not understand your point here. If this stuff is trademark, (tm) as you write it, I doubt it has ever been incorporated in any Xenomai or Fusion release. Where can I get it anyway ? Google was not my friend on this !
> 
> 
>>Measurements can be done using OProfile, with appropriate event
>>selection.
>>
> 
> 
> OK, I will try it.
> 
> 
>>>PS: too bad the cruncher will not survive 2.0. Would you consider
>>
>>including POSIX tests in the new 2.1 release ? I did not see 
>>any in the
>>2.0.
>>
>>I'm sure the Xenomai committer team will enjoy your patches ;-)
>>
> 
> 
> This is why I asked. If number crunching is no longer in the scope it is worth knowing. Are you a member of this team ?
> 

Why would you want to limit the scope of a real-time framework by one of its 
possible applications? Not everyone is interested in number crunching, which 
does not mean that Xeno does not provide support for it, it still does. The 
point is that it's not the defaut setup because it's by far an uncommon use 
which induces a performance tradeoff, that's all.

> 
>>Best Regards,
>>-- 
>>Stephane
>>
> 
> 
> Best Regards,
> --
> Dominique
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
> 


-- 

Philippe.

RE: [Xenomai-help] number-crunching execution jitter

[Gilles Chanteperdrix]

Dominique.RAGOT@fr.thalesgroup.com wrote:
 > In the meantime I modified the benchmark with a call to the routine
 > pthread_set_mode_np(0, PTHREAD_SHIELD) immediately after thread is
 > created, and observes a huge reduction in execution jitter, but at
 > the expense of wakeup jitter. So I concluded the interrupt shield was
 > definitely not doing much without this call. Am I wrong ? 

I guess Philippe and Stephane gave you answers regarding the way to
activate the shield (I did not check if pthread_set_mode fails when the
shield was disabled at compilation time and is used at run time, if not,
maybe it would be a good idea ?).

Now about the wakeup jitter, if you are still using the "cruncher" as in
RTAI/Fusion, what is called "nanosleep jitter" may not be a precise
measurement of the wakeup latency. A better measurement of the
scheduling latency using the POSIX skin is obtained using the
clock_nanosleep call with the TIMER_ABSTIME flag, pass an absolute time,
and measure the difference between this time and the wakeup time.

Also note that CLOCK_MONOTONIC will give better accuracy than
CLOCK_REALTIME, since it is using the tsc even when system timer is in
periodic mode.

-- 


					    Gilles Chanteperdrix.

RE: [Xenomai-help] number-crunching execution jitter

[Fillod Stephane]

Dominique.RAGOT wrote:
>I am using Xenomai 2.0 for a number-crunching application on
bi-processor x86 SMP and using the Posix skin.
>
>There are 2 tasks on each processor that are awaken every 20ms and
execute for 16ms.
>When there are no perturbations on the machine, the jitter on tasks
wakeup and execution time is acceptable (approx. 5%).
>
>However, when adding a simple load such as find / -print redirected to
/dev/null, the execution jitter is much worse (more than 100% in some
cases), whereas wakeup jitter remains good. My conclusion is that the
tasks are preempted when running. Is it right ?

IMHO, there are plenty of other possible reasons.
What *measurement* result makes you conclude it is preemption which
induces
execution jitter?

Rem: I assume your RT tasks are free of non-RT Linux syscalls.

>I suspect the interrupt shield to be inactive (although I see it in
/proc/adeos). How could it be activated with Posix skin ? How could it
be monitored ?

Subtract the number of interrupts handlers processed when leaving your 
RT task by the number when entering. This measurement will tell you if 
the interrupt shield works as expected or not.

A tool like LTT (not available now with Xenomai?) would have been able
to tell what's happening during the 20ms period.

My random guess is you're maybe in the need of a "cache-shield", 
a "tlb-shield" and/or a "DMA-shield". All those shields (tm) :-).

Measurements can be done using OProfile, with appropriate event
selection.


>PS: too bad the cruncher will not survive 2.0. Would you consider
including POSIX tests in the new 2.1 release ? I did not see any in the
2.0.

I'm sure the Xenomai committer team will enjoy your patches ;-)


Best Regards,
-- 
Stephane

[Xenomai-help] number-crunching execution jitter

[Dominique.RAGOT]

Hello,

I am using Xenomai 2.0 for a number-crunching application on bi-processor x86 SMP and using the Posix skin.

There are 2 tasks on each processor that are awaken every 20ms and execute for 16ms.
When there are no perturbations on the machine, the jitter on tasks wakeup and execution time is acceptable (approx. 5%).

However, when adding a simple load such as find / -print redirected to /dev/null, the execution jitter is much worse (more than 100% in some cases), whereas wakeup jitter remains good. My conclusion is that the tasks are preempted when running. Is it right ?

I suspect the interrupt shield to be inactive (although I see it in /proc/adeos). How could it be activated with Posix skin ? How could it be monitored ?

PS: too bad the cruncher will not survive 2.0. Would you consider including POSIX tests in the new 2.1 release ? I did not see any in the 2.0.

Sincerely,

---------- 
Dominique RAGOT