Re: PREEMPT_RT vs I-PIPE: the numbers, part 2

[Karim Yaghmour <karim@opersys.com>]

Ingo Molnar wrote:
> with lpptest (PREEMPT_RT's built-in parallel-port latency driver) that's 
> possible, as it polls the target with interrupts disabled, eliminating 
> much of the logger-side latencies. The main effect is that it's now only 
> a single worst-case latency that is measured, instead of having to have 
> two worst-cases meet.
> 
> Here's a rough calculation to show what the stakes are: if there's a 
> 1:100000 chance to trigger a worst-case irq handling latency, and you 
> have 600000 samples, then with lpptest you'll see an average of 6 events 
> during the measurement. With lrtfb (the one Karim used) the chance to 
> see both of these worst-case latencies on both sides of the measurement 
> is 1:10000000000, and you'd see 0.00006 of them during the measurement.  
> I.e. the chances of seeing the true max latency are pretty slim.

If indeed there are 6 events on a single-side which are worst-case,
then you would have to also factor in the probability of obtaining
an average or below average result on the other side. So again, if
all runs were measuring average on each side, one would expect that
at least one of the runs would have a bump over the 55us mark. Yet,
they all have the same maximum.

Here's an overview of the results spread in the case of IRQ latency
measurements in the HD case (this is just a view of one case, a
true study would require drawing graphs showing the spread for all
tests):
Of 833,000 results for PREEMPT_RT:
	-  36 values are above 50us (0.0045% or 4.5/100,000)
	- 860 values are 19us and above
Of 781,000 results for IPIPE:
	- 213 values are above 50us (0.0273% or 27/100,000)
	- 311 values are 19us and above

Contrary to the illustration you make above, it would seem that the
fact that both machines are running the same mechanism, the
blue-moon effect multiplies upward instead of downward. This,
though, is but a preliminary analysis.

Notes:
- Below 19us, the number of measurement points increases for both
setups as we get closer to the 14us average mark.
- There are more data points for PREEMPT_RT than ipipe because
LMbench takes much more time to complete on the former.

> So if you want to reliably measure worst-case latencies in your expected 
> lifetime, you truly never want to serially couple the probabilities of 
> worst-case latencies on the target and the logger side.

Like I said, we're going to settle this one to avoid any further
doubts.

Karim
-- 
Author, Speaker, Developer, Consultant
Pushing Embedded and Real-Time Linux Systems Beyond the Limits
http://www.opersys.com || karim@opersys.com || 1-866-677-4546