Q6600 + low frequency periodic timer + CPU_IDLE = much jitter

Q6600 + low frequency periodic timer + CPU_IDLE = much jitter

[Mike Galbraith]

Greetings,

It's looking like my Q6600 is kinda slow at switching gears, and I'd
just like to verify that it is the hardware, not something odd going on
in cpuidle territory.

If I boot idle=poll and set scaling_governor to performance, the CPU is
running full bore (2.4GHz) per both cpufreq and turbostat, and jitter is
~10 usecs (~5 usecs on non-rt kernel)

If I boot processor.max_cstate=1 instead, to have _some_ power saving,
set the governor to performance and measure, I notice that while cpufreq
swears my CPU is locked in at 2.4GHz, turbostat says cpufreq is lying to
me, that my CPUs _are_ changing frequency, bouncing around a bit.

If I switch to powersave, there is no frequency shifting, and little
jitter.  Switch to performance and there's much jitter, until I add a
cpu hog, then CPU goes to full throttle, and jitter goes away.

This large jitter differential does not happen on a L5520 box, or a few
others I've tested with much nicer CPUs than mine, which leads me to
suspect that my Q6600's oldish two speed gearbox has straight cut gears
and no synchros, it's double-clutching, and that takes a while ;-)

Kernel: v2.6.33.6-rt26-17-g911d28c processor.max_cstate=1
Box: Q6600

cset shield --cpu 1-3 -k on

marge:/root/tmp # ./periodic_timer -p40 -c3 -f60 -t1 -s -T -F2392.594 -C1000              
Pinned to CPU3 priority: 40 timer freq: 60 Hz tolerance: 1 usecs, stats interval: 1000 samples
                         measurement clock; TSC @2392.594MHz

period: 16666.67 usecs          min: 16552.33 max: 16772.62 mean: 16667.12 stddev:   5.72
        585 > 1 usec errors     min: -105.95 max: 114.34 mean:  -0.48 stddev:  28.12

period: 16666.67 usecs          min: 16549.42 max: 16783.26 mean: 16667.13 stddev:   6.30
        589 > 1 usec errors     min: -116.59 max: 117.25 mean:  -0.51 stddev:  36.58

start SCHED_OTHER cpu hog on CPU3 (or even better boot idle=poll)

period: 16666.67 usecs          min: 16662.84 max: 16676.36 mean: 16667.10 stddev:   0.37
        15 > 1 usec errors      min:  -9.69 max:   3.83 mean:  -0.93 stddev:   3.75

period: 16666.67 usecs          min: 16663.89 max: 16674.49 mean: 16667.10 stddev:   0.30
        11 > 1 usec errors      min:  -7.82 max:   2.78 mean:  -1.02 stddev:   3.40

Re: Q6600 + low frequency periodic timer + CPU_IDLE = much jitter

[Carsten Emde]

Mike,

> It's looking like my Q6600 is kinda slow at switching gears, and I'd
> just like to verify that it is the hardware, not something odd going on
> in cpuidle territory.
Yes, it's the CPU. It is a known restriction of this processor and has
been verified on a number of different systems. It appears that the
processor is halted while the PLL is stabilizing. However, there is no
formal confirmation from Intel AFAIK.

It happens that one of the test systems that we regularly use in the
quality assessment of the PREEMPT_RT patch is a Q6600:
# grep CPU /proc/cpuinfo
model name	: Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz
model name	: Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz

The recommendation is to add
-=--------------------------------------=-
cd /sys/devices/system/cpu
for i in cpu?/cpufreq/scaling_governor
do
  echo performance >$i
done
cd -
-=--------------------------------------=-
to /etc/rc.d/rc.local and to abandon dynamic frequency scaling on such
systems.


Carsten.

Re: Q6600 + low frequency periodic timer + CPU_IDLE = much jitter

[Mike Galbraith]

On Thu, 2010-07-29 at 12:35 +0200, Carsten Emde wrote:
> Mike,
> 
> > It's looking like my Q6600 is kinda slow at switching gears, and I'd
> > just like to verify that it is the hardware, not something odd going on
> > in cpuidle territory.
> Yes, it's the CPU. It is a known restriction of this processor and has
> been verified on a number of different systems. It appears that the
> processor is halted while the PLL is stabilizing. However, there is no
> formal confirmation from Intel AFAIK.

Thanks for the confirmation.

	-Mike