[Xenomai-help] Results of xenomai's latency test vs. RTAI's latency test

[Xenomai-help] Results of xenomai's latency test vs. RTAI's latency test

[M. Koehrer]

Hi everybody,

I am currently checking XENOMAI (V2.2.3 on a 2.6.17.7 kernel P4) to see if I can use it
as replacement for a RTAI 3.3-cv application.
The first thing I did was to run the latency test in the the xenomai's testsuite directory.
The results of the worst time latency are really ugly - about 40µs!
On the very same PC I got a value of about 5µs using RTAI 3.3-cv running the RTAI's
user/latency test.

My question is now: Why can there be such a huge difference between the two systems on the very same
hardware??
Is there a way to improve this value?

The RTAI system uses a 2.4.33 kernel, the XENOMAI uses the 2.6.17.7 kernel. Could this
be an issue?

Thanks for any feedback on that issue!

Mathias


-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's latency test

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1590 bytes --]

M. Koehrer wrote:
> Hi everybody,
> 
> I am currently checking XENOMAI (V2.2.3 on a 2.6.17.7 kernel P4) to see if I can use it
> as replacement for a RTAI 3.3-cv application.
> The first thing I did was to run the latency test in the the xenomai's testsuite directory.
> The results of the worst time latency are really ugly - about 40µs!
> On the very same PC I got a value of about 5µs using RTAI 3.3-cv running the RTAI's
> user/latency test.

I'm _very_ sceptical about your 5 us. Could you elaborate on how you
load your box and how long those tests ran? See also TROUBLESHOOTING in
the Xenomai source tree on appropriate load for triggering the worst case.

The worst difference I once measured on a Pentium 133 MHz was 10% better
maximum latency for a timed task under RTAI. IIRC, it took longer for
RTAI to expose this than for Xenomai.

> 
> My question is now: Why can there be such a huge difference between the two systems on the very same
> hardware??
> Is there a way to improve this value?

You may what to have a look at the I-pipe tracer to analyse the worst
case - also under RTAI (requires a bit hacking to obtain the same
freezing feature with its latency test). It is helpful to see what
scenario both system faced when running on the maximum delay.

> 
> The RTAI system uses a 2.4.33 kernel, the XENOMAI uses the 2.6.17.7 kernel. Could this
> be an issue?

Nope, the latency differences between 2.4 and 2.6 are insignificant on
x86 last time I checked.

> 
> Thanks for any feedback on that issue!
> 
> Mathias
> 
> 

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's latency

[M. Koehrer]

Hi Jan,

thanks for your reply.

> > I am currently checking XENOMAI (V2.2.3 on a 2.6.17.7 kernel P4) to see if
> I can use it
> > as replacement for a RTAI 3.3-cv application.
> > The first thing I did was to run the latency test in the the xenomai's
> testsuite directory.
> > The results of the worst time latency are really ugly - about 40µs!
> > On the very same PC I got a value of about 5µs using RTAI 3.3-cv running
> the RTAI's
> > user/latency test.
> 
> I'm _very_ sceptical about your 5 us. Could you elaborate on how you
> load your box and how long those tests ran? See also TROUBLESHOOTING in
> the Xenomai source tree on appropriate load for triggering the worst case.

My test setup was the same in both cases.
I connect to the RTAI/Xenomai PC via telnet and let the latency test run for about 5 minutes.
No other activity happens (well, there are a couple of services running like
apache, ... - but they should not cause any load).
The strange thing, is that in Xenomai even after a few seconds a latency in the range of the
maximum latency is shown.

I can retry the tests using a background stress situation.



-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's latency test

[Philippe Gerum]

On Mon, 2006-10-30 at 11:05 +0100, M. Koehrer wrote:
> Hi everybody,
> 
> I am currently checking XENOMAI (V2.2.3 on a 2.6.17.7 kernel P4) to see if I can use it
> as replacement for a RTAI 3.3-cv application.
> The first thing I did was to run the latency test in the the xenomai's testsuite directory.
> The results of the worst time latency are really ugly - about 40µs!
> On the very same PC I got a value of about 5µs using RTAI 3.3-cv running the RTAI's
> user/latency test.
> 
> My question is now: Why can there be such a huge difference between the two systems on the very same
> hardware??
> Is there a way to improve this value?
> 
> The RTAI system uses a 2.4.33 kernel, the XENOMAI uses the 2.6.17.7 kernel. Could this
> be an issue?
> 

Several issues there:

- are you 100% sure that your kernel config file for 2.4.33 is perfectly
recycled for 2.6.17, e.g. are all latency killer options as listed in
the TROUBLESHOOTING file really disabled? P4 configurations are
jitter-prone; some options are know to induce bad latencies.

- worst-case figures do not depend on how fast you get them, the latter
information gives you nothing to interpret from. You may want to stress
test the box for a longer period of time, using things like e.g. a dd
loop, and a compilation in the background (dd if=/dev/zero of=/somefile
count=500 bs=1M). "ping" network test is not the worst latency raiser,
actually, under some circumstances, it could even hide some latency
issues, basically because it favours the code locality in i-cache. Also
make sure to measure without X-window interaction in both cases; some
graphic card drivers induce latencies, some don't. YMMV.

This said, 40us on a P4 class machine is not an expected value for
Xenomai/x86. To give you some reference figures, I have a uniprocessor
2.8Ghz Xeon box (no HT) performing at 15 us worst-case, and an older
dual 2.4Ghz Xeon SMP which honours 25 us worst-case in SMP mode. Fact is
that to get that, I need to disable a number of ACPI options (except
those which are needed to boot properly in SMP mode), and activate the
SMI work-around.

And above all, a good starting point would be to send your kernel
configuration file, so that we could discuss about facts. Additionally,
you may want to upgrade to 2.2.4; 2.2.3 has a FPU issue on some hw.

> Thanks for any feedback on that issue!
> 
> Mathias
> 
> 
-- 
Philippe.

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's latency

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1727 bytes --]

M. Koehrer wrote:
> Hi Jan,
> 
> thanks for your reply.
> 
>>> I am currently checking XENOMAI (V2.2.3 on a 2.6.17.7 kernel P4) to see if
>> I can use it
>>> as replacement for a RTAI 3.3-cv application.
>>> The first thing I did was to run the latency test in the the xenomai's
>> testsuite directory.
>>> The results of the worst time latency are really ugly - about 40µs!
>>> On the very same PC I got a value of about 5µs using RTAI 3.3-cv running
>> the RTAI's
>>> user/latency test.
>> I'm _very_ sceptical about your 5 us. Could you elaborate on how you
>> load your box and how long those tests ran? See also TROUBLESHOOTING in
>> the Xenomai source tree on appropriate load for triggering the worst case.
> 
> My test setup was the same in both cases.
> I connect to the RTAI/Xenomai PC via telnet and let the latency test run for about 5 minutes.

5 min. are far too short.

It takes a while to "arrange" the worst case, i.e. a dirty cache, one
unrelated IRQ or atomic section running while the timer (or any other
event of interest) occurs, and then one or more further IRQs right after
the woken-up task re-enabled interrupts. If you hit such scenario can be
observed via the latency tracer (which has some impact on the numbers,
but not on the structuring).

> No other activity happens (well, there are a couple of services running like
> apache, ... - but they should not cause any load).
> The strange thing, is that in Xenomai even after a few seconds a latency in the range of the
> maximum latency is shown.

Such a test on an unloaded system says almost nothing about a hard
real-time system.

> 
> I can retry the tests using a background stress situation.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's

[M. Koehrer]

[-- Attachment #1.1: Type: text/plain, Size: 3379 bytes --]

Hi Philippe,

I have rebuild my system using V2.2.4 of xenomai.
Also, I had a look to reuse the kernel configuration from the 2.4.33 RTAI system.
However, the results are more or less the same.
The only kernel module I have loaded is the "tg3" network driver, the chipset of the 
PC (ICH6) is supported by the SMI detection and it is activated.

I have enclosed my kernel configuration, perhaps this can help to identify an issue.
I do not have X11 running, it is a text-mode only system that is controlled via telnet from
another PC.

I'll try to do a stress test on the RTAI system as well.

Regards

Mathias

> > I am currently checking XENOMAI (V2.2.3 on a 2.6.17.7 kernel P4) to see if
> I can use it
> > as replacement for a RTAI 3.3-cv application.
> > The first thing I did was to run the latency test in the the xenomai's
> testsuite directory.
> > The results of the worst time latency are really ugly - about 40µs!
> > On the very same PC I got a value of about 5µs using RTAI 3.3-cv running
> the RTAI's
> > user/latency test.
> > 
> > My question is now: Why can there be such a huge difference between the
> two systems on the very same
> > hardware??
> > Is there a way to improve this value?
> > 
> > The RTAI system uses a 2.4.33 kernel, the XENOMAI uses the 2.6.17.7
> kernel. Could this
> > be an issue?
> > 
> 
> Several issues there:
> 
> - are you 100% sure that your kernel config file for 2.4.33 is perfectly
> recycled for 2.6.17, e.g. are all latency killer options as listed in
> the TROUBLESHOOTING file really disabled? P4 configurations are
> jitter-prone; some options are know to induce bad latencies.
> 
> - worst-case figures do not depend on how fast you get them, the latter
> information gives you nothing to interpret from. You may want to stress
> test the box for a longer period of time, using things like e.g. a dd
> loop, and a compilation in the background (dd if=/dev/zero of=/somefile
> count=500 bs=1M). "ping" network test is not the worst latency raiser,
> actually, under some circumstances, it could even hide some latency
> issues, basically because it favours the code locality in i-cache. Also
> make sure to measure without X-window interaction in both cases; some
> graphic card drivers induce latencies, some don't. YMMV.
> 
> This said, 40us on a P4 class machine is not an expected value for
> Xenomai/x86. To give you some reference figures, I have a uniprocessor
> 2.8Ghz Xeon box (no HT) performing at 15 us worst-case, and an older
> dual 2.4Ghz Xeon SMP which honours 25 us worst-case in SMP mode. Fact is
> that to get that, I need to disable a number of ACPI options (except
> those which are needed to boot properly in SMP mode), and activate the
> SMI work-around.
> 
> And above all, a good starting point would be to send your kernel
> configuration file, so that we could discuss about facts. Additionally,
> you may want to upgrade to 2.2.4; 2.2.3 has a FPU issue on some hw.
> 


-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

[-- Attachment #2: config_linux_2.6.17.7.gz --]
[-- Type: application/octetstream, Size: 7228 bytes --]

Re: Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's

[Philippe Gerum]

On Mon, 2006-10-30 at 15:15 +0100, M. Koehrer wrote:
> Hi Philippe,
> 
> I have rebuild my system using V2.2.4 of xenomai.
> Also, I had a look to reuse the kernel configuration from the 2.4.33 RTAI system.
> However, the results are more or less the same.
> The only kernel module I have loaded is the "tg3" network driver, the chipset of the 
> PC (ICH6) is supported by the SMI detection and it is activated.
> 

On such hw, you may want to enable the following:
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_BLK_DEV_IDEDMA=y
CONFIG_IDEDMA_AUTO=y

> I have enclosed my kernel configuration, perhaps this can help to identify an issue.
> I do not have X11 running, it is a text-mode only system that is controlled via telnet from
> another PC.
> 
> I'll try to do a stress test on the RTAI system as well.
> 

You could try running RTAI 3.3 over 2.6.17 with the very same config
file too (Xenomai opts aside, of course), so you could rule out any
configuration problem if the gap still exists.

> Regards
> 
> Mathias
> 
> > > I am currently checking XENOMAI (V2.2.3 on a 2.6.17.7 kernel P4) to see if
> > I can use it
> > > as replacement for a RTAI 3.3-cv application.
> > > The first thing I did was to run the latency test in the the xenomai's
> > testsuite directory.
> > > The results of the worst time latency are really ugly - about 40µs!
> > > On the very same PC I got a value of about 5µs using RTAI 3.3-cv running
> > the RTAI's
> > > user/latency test.
> > > 
> > > My question is now: Why can there be such a huge difference between the
> > two systems on the very same
> > > hardware??
> > > Is there a way to improve this value?
> > > 
> > > The RTAI system uses a 2.4.33 kernel, the XENOMAI uses the 2.6.17.7
> > kernel. Could this
> > > be an issue?
> > > 
> > 
> > Several issues there:
> > 
> > - are you 100% sure that your kernel config file for 2.4.33 is perfectly
> > recycled for 2.6.17, e.g. are all latency killer options as listed in
> > the TROUBLESHOOTING file really disabled? P4 configurations are
> > jitter-prone; some options are know to induce bad latencies.
> > 
> > - worst-case figures do not depend on how fast you get them, the latter
> > information gives you nothing to interpret from. You may want to stress
> > test the box for a longer period of time, using things like e.g. a dd
> > loop, and a compilation in the background (dd if=/dev/zero of=/somefile
> > count=500 bs=1M). "ping" network test is not the worst latency raiser,
> > actually, under some circumstances, it could even hide some latency
> > issues, basically because it favours the code locality in i-cache. Also
> > make sure to measure without X-window interaction in both cases; some
> > graphic card drivers induce latencies, some don't. YMMV.
> > 
> > This said, 40us on a P4 class machine is not an expected value for
> > Xenomai/x86. To give you some reference figures, I have a uniprocessor
> > 2.8Ghz Xeon box (no HT) performing at 15 us worst-case, and an older
> > dual 2.4Ghz Xeon SMP which honours 25 us worst-case in SMP mode. Fact is
> > that to get that, I need to disable a number of ACPI options (except
> > those which are needed to boot properly in SMP mode), and activate the
> > SMI work-around.
> > 
> > And above all, a good starting point would be to send your kernel
> > configuration file, so that we could discuss about facts. Additionally,
> > you may want to upgrade to 2.2.4; 2.2.3 has a FPU issue on some hw.
> > 
> 
> 
> -- 
> Mathias Koehrer
> mathias_koehrer@domain.hid
> 
> 
> Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
> ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
> und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
> nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
> http://www.arcor.de/rd/emf-dsl-2
-- 
Philippe.

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's

[M. Koehrer]

Hi everybody,

I have repeated the test using a parallel stress test (dd if=/dev/sda of=/dev/null, I have
a SATA harddisk).
The results are that my RTAI latency of 4µs is not valid, here the latency is about 26µs.
However, xenomai shows a higher latency of about 40µs.
The interesting thing is, that without load RTAI seems to run very long in a very 
low latency region, at the same setup xenomai shows large latencies earlier.


Well, I have repeated these tests on a different PC that has a server mainboard and 
a Pentium D 4 (real dual core) CPU.
The results (measured under load) are much better here:
RTAI shows a latency of about 6µs
Xenomai shows a latency of about 10µs.
Both values are acceptable, however I wonder where the difference comes from?

Thanks for all the feedback on that issue!

Regards

Mathias
> > I have rebuild my system using V2.2.4 of xenomai.
> > Also, I had a look to reuse the kernel configuration from the 2.4.33 RTAI
> system.
> > However, the results are more or less the same.
> > The only kernel module I have loaded is the "tg3" network driver, the
> chipset of the 
> > PC (ICH6) is supported by the SMI detection and it is activated.
> > 
> 
> On such hw, you may want to enable the following:
> CONFIG_BLK_DEV_IDEDMA_PCI=y
> CONFIG_BLK_DEV_IDEDMA=y
> CONFIG_IDEDMA_AUTO=y
> 
> > I have enclosed my kernel configuration, perhaps this can help to identify
> an issue.
> > I do not have X11 running, it is a text-mode only system that is
> controlled via telnet from
> > another PC.
> > 
> > I'll try to do a stress test on the RTAI system as well.
> > 
> 
> You could try running RTAI 3.3 over 2.6.17 with the very same config
> file too (Xenomai opts aside, of course), so you could rule out any
> configuration problem if the gap still exists.


-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1972 bytes --]

M. Koehrer wrote:
> Hi everybody,
> 
> I have repeated the test using a parallel stress test (dd if=/dev/sda of=/dev/null, I have
> a SATA harddisk).
> The results are that my RTAI latency of 4µs is not valid, here the latency is about 26µs.
> However, xenomai shows a higher latency of about 40µs.

How long did the tests run this time? To my experience anything below 30
min. is not meaningful. And the faster the machines get, the lower the
probability for certain constellations becomes. So each additional hour
runtime gives a little bit more confidence.

> The interesting thing is, that without load RTAI seems to run very long in a very 
> low latency region, at the same setup xenomai shows large latencies earlier.
> 
> 
> Well, I have repeated these tests on a different PC that has a server mainboard and 
> a Pentium D 4 (real dual core) CPU.
> The results (measured under load) are much better here:
> RTAI shows a latency of about 6µs
> Xenomai shows a latency of about 10µs.
> Both values are acceptable, however I wonder where the difference comes from?

RTAI contains a few shortcuts that may explain parts of this gap, but
not all.

One of these shortcuts is the immediate IRQ dispatching mode (the reason
why they have their own patch...). In that mode the registered timer IRQ
handler gets called without caring about any I-pipe abstraction.

Some others I recall are a missing clean timer subsystem abstraction
(tasked are timed, and then there are also some kind of timer tasklets)
and the immediate reschedule from IRQ context instead of doing this on
IRQ return. Some of those features should be controllable via magic
knobs, maybe it's worth a try what they actually buy you here.

In any case, to really understand the differences, it takes the tracer.
I will check if I can find my old RTAI hack to instrument their latency
tool - if you are still interested in digging deeper on your hardware.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's

[Gilles Chanteperdrix]

M. Koehrer wrote:
> Hi everybody,
> 
> I have repeated the test using a parallel stress test (dd if=/dev/sda of=/dev/null, I have
> a SATA harddisk).
> The results are that my RTAI latency of 4µs is not valid, here the latency is about 26µs.
> However, xenomai shows a higher latency of about 40µs.

Please show us your .config.

-- 
                                                  Gilles Chanteperdrix

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 528 bytes --]

M. Koehrer wrote:
> ...
> I have enclosed my kernel configuration, perhaps this can help to identify an issue.

CONFIG_XENO_OPT_STATS is on which can degrade the performance slightly.
Try to turn this off. I would be interested in the difference on your
boxes (a few micros should be noticeable on the slower one).

I don't know if RTAI has comparable accounting. If yes, check if it's
off as well to keep the conditions fair. I guess CONFIG_REGPARM is not
enabled for RTAI then (LXRT should break, right?).

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-help] Results of xenomai's latency test vs. RTAI's

[Philippe Gerum]

On Mon, 2006-10-30 at 16:34 +0100, M. Koehrer wrote:
> Hi everybody,
> 
> I have repeated the test using a parallel stress test (dd if=/dev/sda of=/dev/null, I have
> a SATA harddisk).
> The results are that my RTAI latency of 4µs is not valid, here the latency is about 26µs.
> However, xenomai shows a higher latency of about 40µs.
> The interesting thing is, that without load RTAI seems to run very long in a very 
> low latency region, at the same setup xenomai shows large latencies earlier.
> 

It has always been the case. A possible explanation could be the
immediate impact of the full interrupt/event virtualization Adeos
performs for Xenomai. The first contribution I once made to the RTAI
project back in late 2002, was to port the 24.1.x release over
Adeos/x86, in order to replace the former RTHAL. Doing so, I noticed
that when running over Adeos, RTAI quickly reached its latency
peak/worst-case after some time, but usually exhibited a lower variance
of the mean latency. The RTHAL had shorter latencies (5-10%) which
tended to converge close to the Adeos figures after a longer time (and
hours of debugging), but had higher variance figures on average.

Since the RTAI project has decided to go back to the RTHAL approach
circa 3.2 instead of using the vanilla Adeos support anymore, this
phenomenon is likely to have re-appeared when comparing a pure
Adeos-based implementation (i.e. Xenomai), and a raw/direct interrupt
handling from the x86 IDT to the RTOS innards.

> 
> Well, I have repeated these tests on a different PC that has a server mainboard and 
> a Pentium D 4 (real dual core) CPU.
> The results (measured under load) are much better here:
> RTAI shows a latency of about 6µs
> Xenomai shows a latency of about 10µs.
> Both values are acceptable, however I wonder where the difference comes from?
> 

Again, it's difficult to discuss this matter without figures and
instrumentation at hand, but some facts are known. Among those, there is
more code traversed in the Xenomai case, from the receipt of an IRQ, to
the actual scheduling of the processing task, and the I-cache penalty is
larger for Xenomai, which benefits less from code locality than RTAI
does when it comes to measuring the performances using a basic latency
test.

Basically, the code in question implements the "abstract RTOS", Xenomai
APIs run over. For instance, each Xenomai API having to block a thread
calls a particular function into the abstract core to do so (i.e.
xnpod_suspend_thread()), and this is the one and only way to suspend a
thread (i.e. the API never accesses the scheduler innards directly). By
contrast, each synchronization/scheduling primitive RTAI defines
directly manages the scheduler internals on its own (e.g. sema4, queue,
fifos, any IPC in general, and the task management services, all fiddle
with the scheduler's data structures locally in the body of their own
routines).
A possible explanation for the (reasonably small) slowdown is that
Xenomai's abstract RTOS code:

- 1) needs to handle more situations since it could be called from many
more contexts, than a particular RTAI service could expect; this
directly translates into more complexity and CPU cycles. The good news
is that we don't have to fix core issues into each and every API when
the weather gets rainy, but only once in the abstract RTOS if needed
(e.g. think of something weird happening in the priority inheritance
code).
- 2) relies on a strict functional layering, which must have some impact
on the code locality at the very least, as more internal interfaces are
involved. On the other hand, the advantage is that Xenomai has been
successfully ported over five new architectures in less than two years,
all being maintained at the very same functional level (i.e. full kernel
and user-space support, tight integration within the Linux programming
model).
- 3) deals with the vanilla Linux kernel as a partner, not as an enemy.
For instance, an additional level of complexity is induced in order to
properly manage regular Linux signals. This is what it takes to be able
to gracefully end any RT application running in user-space through ^C,
or debugging it over GDB, without the box jumping of out the window,
screaming. This "integration with Linux" part does have an indirect cost
in terms of footprints, but from the user POV, it's worth the handful of
microseconds spent ensuring this. And to achieve that reliably over a
co-kernel technology, we need the Adeos model. Deeply.

Said differently, both projects have radically different views on key
key aspects:

- RTAI wants to eliminate Linux from the picture (so it uses radical
sideways to do so), Xenomai wants to share the picture with it (so it
conforms to all Linux semantics when integration is an issue, e.g.
signals).
- RTAI promotes its own API, Xenomai is API-agnostic because it was born
as a portability platform from traditional RTOSes to Linux (so we must
rely on some RTOS abstraction, not to reinvent the wheel each time we
add a new interface/emulator).
- RTAI is fundamentally tied to a co-kernel technology, because of #1.
Xenomai is not, because of #1, since the real-time enabling layer is
isolated from the RTOS that needs its services to operate the various
APIs properly.

All this translates in microseconds, because nothing is for free,
especially a design choice. This said, a margin of improvement still
exists, but reliability will always have precedence over raw latency
figures.

> Thanks for all the feedback on that issue!
> 
> Regards
> 
> Mathias
> > > I have rebuild my system using V2.2.4 of xenomai.
> > > Also, I had a look to reuse the kernel configuration from the 2.4.33 RTAI
> > system.
> > > However, the results are more or less the same.
> > > The only kernel module I have loaded is the "tg3" network driver, the
> > chipset of the 
> > > PC (ICH6) is supported by the SMI detection and it is activated.
> > > 
> > 
> > On such hw, you may want to enable the following:
> > CONFIG_BLK_DEV_IDEDMA_PCI=y
> > CONFIG_BLK_DEV_IDEDMA=y
> > CONFIG_IDEDMA_AUTO=y
> > 
> > > I have enclosed my kernel configuration, perhaps this can help to identify
> > an issue.
> > > I do not have X11 running, it is a text-mode only system that is
> > controlled via telnet from
> > > another PC.
> > > 
> > > I'll try to do a stress test on the RTAI system as well.
> > > 
> > 
> > You could try running RTAI 3.3 over 2.6.17 with the very same config
> > file too (Xenomai opts aside, of course), so you could rule out any
> > configuration problem if the gap still exists.
> 
> 
-- 
Philippe.