Re: PREEMPT_RT vs ADEOS: the numbers, part 1

[Karim Yaghmour <karim@opersys.com>]

James R Bruce wrote:
> It seems that running lmbench improves the maximum response time
> considerably compared to an idle system, unless you touch the
> hard drive.  That sort of thing makes very little sense though,
> and thus is likely an artifact of the testing.  Maybe the test
> needs to be run for longer, or maybe each test should be
> duplicated a few times?  I realize the max is always going to be
> pretty noisy, but we can't really compare approaches much if it
> jumps around by a factor of 2.5.  Then again, maybe lmbench *does*
> improve latency and that would definitely be a bug somewhere that
> you've uncovered :)

Actually I personally read these numbers as being very good. What
I see here is that there were exactly two maximums on 5 different
configs and that standard deviation was always close to 0. What that
means is that Adeos' performance degradation is stepwise and can be
studied (i.e. in order to obtain things like: 60% of the time your
maximum will be 53us and 40% of the time, it'll be 22us.) I don't
think there's any correlation between the setup and the maximum
observed. Instead, it's more like ints were generated by the logger
every 1ms and 1ms is an eternity, so on every odd moon, a combination
of factors resulted in the 53 us actually occuring, but on other
setups, with luck, the maximum was less.

The real remedy to this would be to certainly run longer tests, but
more importantly, it would be to generate a lot more interrupts from
the logger at random times instead of just every 1ms. This would
avoid any sort of artificial sync that may occur between the logger
and the target by virtue of having the logger generate interrupts at
exactly every 1ms. This type of test, though, would be more
complicated and it would require very careful design on the logger
side to avoid introducing any sort of articial latency into the
measurement process.

> The nicest results would be CDFs or histograms of the response
> times, plotted againts each other for east comparison.  Obviously
> that makes more work for you, however.  If we can get full traces
> from the logger as text, then its easy for us to make such graphs,
> or add some scripts to your testbed once its released to generate
> them automatically with gnuplot/etc.

We will be providing full traces, amongst other things. And
getting additions/modifications allowing the automatic generation
of graphs, and other stuff would be great.

Karim
-- 
Author, Speaker, Developer, Consultant
Pushing Embedded and Real-Time Linux Systems Beyond the Limits
http://www.opersys.com || karim@opersys.com || 1-866-677-4546