[Xenomai-core] [RFC] Fix excessive host tick latencies

[Xenomai-core] [RFC] Fix excessive host tick latencies

[Jan Kiszka]

Hi Philippe,

I'm not 100% sure if this plugs all remaining wholes in the deferred
host tick processing, but at least the most easiest reproducible one is
cured now for me (Linux latency peak after termination of 'latency').
Please let me know if you see more potential issues, otherwise I would
include this in my for-upstream queue.

Found while instrumenting the timer core to find RT timer latencies.
Hope they become visible now...

Jan

-------

If a reschedule is pending, we won't allow the host timer to decide
about the next hardware tick. But if that next reschedule will simply
keep ROOT (which, e.g., happens during thread deletion), this deferral
can impose significant unwanted timer latencies on Linux.

diff --git a/ksrc/nucleus/pod.c b/ksrc/nucleus/pod.c
index 3ffd548..fb63f35 100644
--- a/ksrc/nucleus/pod.c
+++ b/ksrc/nucleus/pod.c
@@ -2184,9 +2184,16 @@ void __xnpod_schedule(struct xnsched *sched)
 	zombie = xnthread_test_state(curr, XNZOMBIE);
 
 	next = xnsched_pick_next(sched);
-	if (next == curr && !xnthread_test_state(curr, XNRESTART))
+	if (next == curr && !xnthread_test_state(curr, XNRESTART)) {
 		/* Note: the root thread never restarts. */
+		if (unlikely(xnthread_test_state(next, XNROOT))) {
+			if (testbits(sched->status, XNHTICK))
+				xnintr_host_tick(sched);
+			if (testbits(sched->status, XNHDEFER))
+				xntimer_next_local_shot(sched);
+		}
 		goto signal_unlock_and_exit;
+	}
 
 	XENO_BUGON(NUCLEUS, need_resched == 0);
 

Re: [Xenomai-core] [RFC] Fix excessive host tick latencies

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Hi Philippe,
> 
> I'm not 100% sure if this plugs all remaining wholes in the deferred
> host tick processing, but at least the most easiest reproducible one is
> cured now for me (Linux latency peak after termination of 'latency').
> Please let me know if you see more potential issues, otherwise I would
> include this in my for-upstream queue.
> 
> Found while instrumenting the timer core to find RT timer latencies.
> Hope they become visible now...

Note that the behaviour of xnpod_schedule may have changed since the fix
of the XENO_OPT_DEBUG_NUCLEUS issue. Notably, some calls to
xnpod_schedule should now be skipped if nucleus debug is turned off.

Are you running the tests with or without nucleus debug?

-- 
					    Gilles.

Re: [Xenomai-core] [RFC] Fix excessive host tick latencies

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Hi Philippe,
>>
>> I'm not 100% sure if this plugs all remaining wholes in the deferred
>> host tick processing, but at least the most easiest reproducible one is
>> cured now for me (Linux latency peak after termination of 'latency').
>> Please let me know if you see more potential issues, otherwise I would
>> include this in my for-upstream queue.
>>
>> Found while instrumenting the timer core to find RT timer latencies.
>> Hope they become visible now...
> 
> Note that the behaviour of xnpod_schedule may have changed since the fix
> of the XENO_OPT_DEBUG_NUCLEUS issue. Notably, some calls to
> xnpod_schedule should now be skipped if nucleus debug is turned off.
> 
> Are you running the tests with or without nucleus debug?
> 

Without CONFIG_XENO_OPT_DEBUG_NUCLEUS and with the debug fix applied.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC] Fix excessive host tick latencies

[Philippe Gerum]

On Fri, 2010-04-30 at 15:33 +0200, Jan Kiszka wrote:
> Hi Philippe,
> 
> I'm not 100% sure if this plugs all remaining wholes in the deferred
> host tick processing, but at least the most easiest reproducible one is
> cured now for me (Linux latency peak after termination of 'latency').
> Please let me know if you see more potential issues, otherwise I would
> include this in my for-upstream queue.

That patch is correct, please queue it. Anything that breaks the
assumption described in the following comment from
xntimer_next_local_shot() is wrong wrt tick deferral: "The host tick
deferral is cleared whenever Xenomai is about to yield control to the
host kernel".

In short, when the code will match the comments and documentation, we
will be done with debugging.

> 
> Found while instrumenting the timer core to find RT timer latencies.
> Hope they become visible now...
> 
> Jan
> 
> -------
> 
> If a reschedule is pending, we won't allow the host timer to decide
> about the next hardware tick. But if that next reschedule will simply
> keep ROOT (which, e.g., happens during thread deletion), this deferral
> can impose significant unwanted timer latencies on Linux.
> 
> diff --git a/ksrc/nucleus/pod.c b/ksrc/nucleus/pod.c
> index 3ffd548..fb63f35 100644
> --- a/ksrc/nucleus/pod.c
> +++ b/ksrc/nucleus/pod.c
> @@ -2184,9 +2184,16 @@ void __xnpod_schedule(struct xnsched *sched)
>  	zombie = xnthread_test_state(curr, XNZOMBIE);
>  
>  	next = xnsched_pick_next(sched);
> -	if (next == curr && !xnthread_test_state(curr, XNRESTART))
> +	if (next == curr && !xnthread_test_state(curr, XNRESTART)) {
>  		/* Note: the root thread never restarts. */
> +		if (unlikely(xnthread_test_state(next, XNROOT))) {
> +			if (testbits(sched->status, XNHTICK))
> +				xnintr_host_tick(sched);
> +			if (testbits(sched->status, XNHDEFER))
> +				xntimer_next_local_shot(sched);
> +		}
>  		goto signal_unlock_and_exit;
> +	}
>  
>  	XENO_BUGON(NUCLEUS, need_resched == 0);
>  
> 


-- 
Philippe.

Re: [Xenomai-core] [RFC] Fix excessive host tick latencies

[Gilles Chanteperdrix]

Philippe Gerum wrote:
> On Fri, 2010-04-30 at 15:33 +0200, Jan Kiszka wrote:
>> Hi Philippe,
>>
>> I'm not 100% sure if this plugs all remaining wholes in the deferred
>> host tick processing, but at least the most easiest reproducible one is
>> cured now for me (Linux latency peak after termination of 'latency').
>> Please let me know if you see more potential issues, otherwise I would
>> include this in my for-upstream queue.
> 
> That patch is correct, please queue it. Anything that breaks the
> assumption described in the following comment from
> xntimer_next_local_shot() is wrong wrt tick deferral: "The host tick
> deferral is cleared whenever Xenomai is about to yield control to the
> host kernel".
> 
> In short, when the code will match the comments and documentation, we
> will be done with debugging.

What I meant is that with the nucleus debugging fixed, we may not even
enter __xnpod_schedule often enough to handle the host tick propagation.
So, maybe we should make xnpod_schedule call __xnpod_schedule if XNHTICK
or XNHDEFER are set.

-- 
					    Gilles.

Re: [Xenomai-core] [RFC] Fix excessive host tick latencies

[Philippe Gerum]

On Fri, 2010-04-30 at 23:01 +0200, Gilles Chanteperdrix wrote:
> Philippe Gerum wrote:
> > On Fri, 2010-04-30 at 15:33 +0200, Jan Kiszka wrote:
> >> Hi Philippe,
> >>
> >> I'm not 100% sure if this plugs all remaining wholes in the deferred
> >> host tick processing, but at least the most easiest reproducible one is
> >> cured now for me (Linux latency peak after termination of 'latency').
> >> Please let me know if you see more potential issues, otherwise I would
> >> include this in my for-upstream queue.
> > 
> > That patch is correct, please queue it. Anything that breaks the
> > assumption described in the following comment from
> > xntimer_next_local_shot() is wrong wrt tick deferral: "The host tick
> > deferral is cleared whenever Xenomai is about to yield control to the
> > host kernel".
> > 
> > In short, when the code will match the comments and documentation, we
> > will be done with debugging.
> 
> What I meant is that with the nucleus debugging fixed, we may not even
> enter __xnpod_schedule often enough to handle the host tick propagation.
> So, maybe we should make xnpod_schedule call __xnpod_schedule if XNHTICK
> or XNHDEFER are set.
> 

I got that, but this is ok. Deferral is marked whenever the resched bit
is already set, or we don't run over the root thread, which means that
we will raise the resched bit to switch back to root at some point, and
clear the deferral there. IOW, xnpod_schedule() cannot filter out a
valid condition for clearing the host timer deferral, in any of its
debug/non-debug forms.

-- 
Philippe.