Re: [Xenomai] latency spikes under load

[Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>]

On 12/03/2013 04:31 PM, Kurijn Buys wrote:
> Op 3-dec.-2013, om 13:23 heeft Gilles Chanteperdrix het volgende
> geschreven:
>
>> On 12/03/2013 02:07 PM, Kurijn Buys wrote:
>>> Thanks for the quick response, ACPI is enabled, I only disabled
>>> "Processor" in there... -1 was a typo indeed, it is at 1... I
>>> see SCHED_SMT [=y] in my kernel config... shall I recompile the
>>> kernel with this disabled then... no other things to try first/at
>>> the same time?
>>
>> To remove hyperthreading, either: - disable it in the BIOS
>> configuration; - or disable CONFIG_SMP (not SCHED_SMPT) in the
>> kernel configuration.
>>
> Ah I see, CONFIG_SMP is also enabled... I've disabled it in BIOS, but
> no success (tell me if it is worth trying to disable it in the kernel
> config in stead).

When you say "no success", you mean you still have 2 cpus ? Or you still
have latency pikes? If the former, then yes, try without CONFIG_SMP, or
pass nr_cpus=1 on the command line. If the latter, then no, testing
without CONFIG_SMP is useless.

>
>>>
>>> I realized that the test with sched_rt_runtime_us on -1 I
>>> performed was with an earlier set-up. When I set it now to -1, I
>>> have better performance, but: 1) still spikes of up to 87us under
>>> load with ./latency 2) still some completely shifted occurrences
>>> with the other latency test, with a 1000µs period (but now only 2
>>> out of 890814), and the rest of the distribution lies in
>>> [861-1139]µs, which is also rather large I suppose.
>>
>> sched_rt_runtime_us should not make any difference.
>>
>> Something else you should try is to disable root thread priority
>> coupling.
>>
> I have tried a config with priority coupling support disabled before,
> but then the system was even more vulnerable for such latency peaks
> (however the mean latency was a little lower!) (I still have the
> kernel, but unfortunately the I-pipe tracer isn't installed there)

Please keep priority coupling disabled in further tests.

>
>>>
>>> The ipipe trace after test (1) was similar to the one I posted,
>>> where this line seems to be the problem I suppose: :|   #end
>>> 0x80000001 -179! 149.235  ipipe_check_context+0x87
>>> (add_preempt_count+0x15)
>>>
> ...I hoped the I-pipe trace would help..?

Unfortunately the trace is not helping much. Either something happens on
the other CPU, or you have an SMI. Disabling SMP should avoid the first
issue, and for the SMI, I may have broke the SMI workaround code when
replacing the compile-time option with the run-time option, so, you may
want to try Xenomai 2.6.2.1 to see if the SMI workaround really works in
that case.

Are you still running a 2.6 kernel, or did you upgrade to 3.8.13?

Regards.

-- 
					    Gilles.