Re: [Xenomai-help] Unexpected switch to secondary mode

[Philippe Gerum <rpm@xenomai.org>]

On Fri, 2007-08-03 at 10:05 +0200, Johan Borkhuis wrote:
> Philippe,
> 
> (BTW: is this something that should be discussed in the Xenomai group, 
> or would it be better to move this discussion to the Adeos mailing list?)
> 

This list is ok; we are discussing about how RT threads are coping with
the underlying MMU, which is first and foremost a Xenomai issue (despite
Adeos must be the one to provide some support to fix the latter issue
though).

> I do have question on this. We already discussed this problem earlier, 
> and I managed to find a very dirty work around for my application: I 
> added a lot of "dummy" functions to my application, which are spread 
> over the whole application. By calling these after the mlockall, but 
> before I switch to RT-mode I manage to eliminate most of the switches.
> 

Yeah, not pretty, but clearly illustrates the point.

> Philippe Gerum wrote:
> > So it's unfortunately possible to experiment such switches on ppc hw for
> > now. It is likely happening when the ns value is copied back to a user
> > variable for which there is a PTE miss to handle in Linux kernel space
> > first. Check the mail archive about "pte", "cow" (copy-on-write) and
> > such things.
> >   
> As far as I know (but I can be incorrect here) an interrupt is generated 
> due to a PTE-miss. This causes the Nucleus to switch to secondary mode, 
> to allow Linux to process the PTE's. The total latency caused by this 
> would not be very bad, as you also indicated earlier.
> 

Correct, as the experience shows. Still, I don't like the idea of
leaving this window open for latency.

>  From the earlier discussion:
> 
>     In such a case, you have likely hit an illustration of the latter issue
>     which the I-pipe/ppc implementation still suffers from: some page table
>     entries are missed during real-time operations. As a consequence of
>     this, the nucleus catches page faults on behalf of RT threads in primary
>     mode, then switches these threads back to secondary in order to process
>     the faults, and eventually wire the missing PTEs in. This is something
>     calling mlockall() does not prevent the application from (like COW).
> 
> 
> Now for my question.
> When looking at a situation where you have a system running multiple 
> RT-tasks, when one of them hits an (unexpected) switch, it is possible 
> that this task will never be switched in again, as the other RT-tasks 
> might consume all the processor time.

Correct, if not the most probable since threads usually manage to call a
blocking Xenomai service which switches them back to primary mode.
Still, the switch-delayed-by-preemption scenario is perfectly possible.

> Would it be possible to have the Nucleus, after the page-fault is 
> processed and the "problem" is fixed, automagically switch the system 
> back to primary mode?

Yes, I think so. This would be restricted to minor VM faults in order to
prevent any misuse of this feature, but I'm going to implement this.

> It is not the real solution to the problem, but it would be a acceptable 
> workaround for this moment, until a real solution is available.
> 

Ack.

> Also another question on this issue: at this moment I am using kernel 
> 2.6.14(ppc), but I saw that there is now an adeos-patch for 
> 2.6.20(powerpc). Could using this version give an improvement in this 
> area, or does this not make a difference?
> 

No difference, yet.

> Kind regards,
>     Johan Borkhuis
-- 
Philippe.