Re: [patch] Real-Time Preemption, -RT-2.6.10-rc2-mm3-V0.7.32-6

[Ingo Molnar <mingo@elte.hu>]

on SMP, latencytest + all IRQ threads (and ksoftirqd) at prio 99 +
PREEMPT_RT is not comparable to PREEMPT_DESKTOP (with no IRQ threading).

The -RT kernel will 'split' hardirq and softirq workloads and migrate
them to different CPUs - giving them a higher total throughput. Also, on
PREEMPT_DESKTOP the IRQs will most likely go to one CPU only, and most
softirq processing will be concentrated on that CPU too. Furthermore, 
the -RT kernel will agressively distribute highprio RT tasks.

latencytest under your priority setup measures an _inverse_ scenario. (a
CPU hog executing at a lower priority than all IRQ traffic) I'd not be
surprised at all if it had higher latencies under -RT than under
PREEMPT_DESKTOP. It's not clear-cut which one 'wins' though: because
even this inverse scenario will have benefits in the -RT case: due to
SCHED_OTHER workloads not interfering with this lower-prio RT task as
much. But i'd expect there to be a constant moving of the 'benchmark
result' forward and backwards, even if -RT only improves things - this
is the nature of such an inverse priority setup.

so this setup generates two conflicting parameters which are inverse to
each other, and the 'sum' of these two parameters ends up fluctuating
wildly. Pretty much like the results you are getting. The two parameters
are: latency of the prio 30 task, and latency of the highprio tasks. The
better the -RT kernel gets, the better the prio 30 tasks's priorities
get relative to SCHED_OTHER tasks - but the worse they also get, due to
the better handling of higher-prio tasks. Where the sum ends, whether
it's a "win" or a "loss" depends on the workload, how much highprio
activity the lowprio threads generate, etc.

if you really want to put all IRQ traffic on the same priority level
then a fairer comparison would be to bind all IRQ (via smp_affinity) and
ksoftirq (via taskset) threads to CPU#0, and to bind latencytest's
CPU-loop to CPU#1. (and do the same in the PREEMPT_DESKTOP case)

	Ingo