Re: [EVL] Unstable with kernel command line nohz=on nohz_full=2-3

[Philippe Gerum <rpm@xenomai.org>]

Hannes Diethelm <hannes.diethelm@gmail.com> writes:

> Hello
>
> Due to I am switching back and forwards between PREEMT_RT and Xenomai4, I just left
> the kernel command line with the best PREEMT_RT performance in for Xenomai4. This is:
>
> isolcpus=managed_irq,domain,2-3 irqaffinity=0-1 rcu_nocbs=2-3 rcu_nocb_poll nohz=on nohz_full=2-3 intel_idle.max_cstate=1 cpufreq.off=1 nosoftlockup nowatchdog mce=off
>
> I had strange behavior when running an EVL task on CPU 2 or 3. Without network, it was fine. But as soon as I run my application with network enabled,
> it just freeze the threads from time to time. It looks like evl_sleep_until() just never wakes up sometimes. The reason might be the active network but also
> somewhere else in the code. Without network, there are also no 10'000 ns sleeps.
>
> The code is:
> evl_read_clock(EVL_CLOCK_MONOTONIC, &ts); -> ts: 1682 s 249982420 ns
> rtapi_timespec_advance(ts, ts, ns); -> ts: 1682 s 249992420 ns
> evl_sleep_until(EVL_CLOCK_MONOTONIC, &ts); -> Never returns
>
> As soon as I remove "nohz=on nohz_full=2-3", everything works fine. Might be evl relies on scheduling-clock ticks?
>

No, it can't and doesn't, that would inject in-band generated latency
into the oob timeline.

> After that resolved, I changed the command line to:
> isolcpus=managed_irq,domain,2-3 irqaffinity=0-1 rcu_nocbs=2-3 rcu_nocb_poll intel_idle.max_cstate=1 cpufreq.off=1 nosoftlockup nowatchdog mce=off evl.oobcpus=3
> so the Ethernet and also the main task run all on CPU3.
>
> After setting also the IRQ affinity for the Ethernet card to CPU3, the max latency is now quite good, max jitter 4us for the task entry, 15us at the end.
>
> I just report this anyway just in case it is a bug or others have the same issue.
>

Thanks for the heads up. I'll have a look.

-- 
Philippe.