Re: PREEMPT_RT vs ADEOS: the numbers, part 1

[Kristian Benoit <kbenoit@opersys.com>]

On Sun, 2005-06-12 at 15:49 -0400, Karim Yaghmour wrote:
> James R Bruce wrote:
> > It seems that running lmbench improves the maximum response time
> > considerably compared to an idle system, unless you touch the
> > hard drive.  That sort of thing makes very little sense though,
> > and thus is likely an artifact of the testing.  Maybe the test
> > needs to be run for longer, or maybe each test should be
> > duplicated a few times?  I realize the max is always going to be
> > pretty noisy, but we can't really compare approaches much if it
> > jumps around by a factor of 2.5.  Then again, maybe lmbench *does*
> > improve latency and that would definitely be a bug somewhere that
> > you've uncovered :)
> 
> Actually I personally read these numbers as being very good. What
> I see here is that there were exactly two maximums on 5 different
> configs and that standard deviation was always close to 0. What that
> means is that Adeos' performance degradation is stepwise and can be
> studied (i.e. in order to obtain things like: 60% of the time your
> maximum will be 53us and 40% of the time, it'll be 22us.) I don't
> think there's any correlation between the setup and the maximum
> observed. Instead, it's more like ints were generated by the logger
> every 1ms and 1ms is an eternity, so on every odd moon, a combination
> of factors resulted in the 53 us actually occuring, but on other
> setups, with luck, the maximum was less.
> 
> The real remedy to this would be to certainly run longer tests, but
> more importantly, it would be to generate a lot more interrupts from
> the logger at random times instead of just every 1ms. This would
> avoid any sort of artificial sync that may occur between the logger
> and the target by virtue of having the logger generate interrupts at
> exactly every 1ms. This type of test, though, would be more
> complicated and it would require very careful design on the logger
> side to avoid introducing any sort of articial latency into the
> measurement process.
> 
> > The nicest results would be CDFs or histograms of the response
> > times, plotted againts each other for east comparison.  Obviously
> > that makes more work for you, however.  If we can get full traces
> > from the logger as text, then its easy for us to make such graphs,
> > or add some scripts to your testbed once its released to generate
> > them automatically with gnuplot/etc.
> 
> We will be providing full traces, amongst other things. And
> getting additions/modifications allowing the automatic generation
> of graphs, and other stuff would be great.
> 
> Karim

All that sounds like lots of job and fun tomorrow morning. I better go
to sleep !!!