[Xenomai-core] Fast mutexes vs. automatic mode switch

[Xenomai-core] Fast mutexes vs. automatic mode switch

[Jan Kiszka]

Hi,

while preparing my reworked fast mutex patches for submission, reviewing
them once again, I realized a conception problem that the fast path can
introduce: So far every pthread_mutex_lock or rt_mutex_acquire forced
the caller into primary mode in case it was in secondary before. Now
this will only happen if the mutex is contended!

Let's consider the fairly typical use case of a two threads
synchronizing a critical section with the help of a mutex. One thread is
high-prio, always in primary mode, the other is low-prio, constantly
transiting between primary (while holding the lock) and secondary (while
interacting with Linux). If the low-prio acquires the uncontended lock,
it will now remain in secondary mode thanks to the new mutex fast path.
That means if the high-prio requests the lock as well, prio-inheritance
will no longer work as the owner is not in the right mode!

I guess what we need is mode detection for the caller in the mutex fast
path. If the possible new owner is in secondary mode, the syscall path
needs to be taken to trigger a migration reliably. That in turn means we
need a syscall-less detection of the current execution mode. Any
spontaneous ideas on this?

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Fast mutexes vs. automatic mode switch

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Hi,
> 
> while preparing my reworked fast mutex patches for submission, reviewing
> them once again, I realized a conception problem that the fast path can
> introduce: So far every pthread_mutex_lock or rt_mutex_acquire forced
> the caller into primary mode in case it was in secondary before. Now
> this will only happen if the mutex is contended!

Yes, I had already thought about that, and even mentioned it on the
mailing list I believe.

> 
> Let's consider the fairly typical use case of a two threads
> synchronizing a critical section with the help of a mutex. One thread is
> high-prio, always in primary mode, the other is low-prio, constantly
> transiting between primary (while holding the lock) and secondary (while
> interacting with Linux). If the low-prio acquires the uncontended lock,
> it will now remain in secondary mode thanks to the new mutex fast path.
> That means if the high-prio requests the lock as well, prio-inheritance
> will no longer work as the owner is not in the right mode!
> 
> I guess what we need is mode detection for the caller in the mutex fast
> path. If the possible new owner is in secondary mode, the syscall path
> needs to be taken to trigger a migration reliably. That in turn means we
> need a syscall-less detection of the current execution mode. Any
> spontaneous ideas on this?

Is not it even simpler for the oscillating thread to switch to primary
mode when it accesses such a mutex? Otherwise we have kernel/user shared
 maps, so allocating a few bytes there should be enough.

-- 
                                                 Gilles.

Re: [Xenomai-core] Fast mutexes vs. automatic mode switch

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Hi,
>>
>> while preparing my reworked fast mutex patches for submission, reviewing
>> them once again, I realized a conception problem that the fast path can
>> introduce: So far every pthread_mutex_lock or rt_mutex_acquire forced
>> the caller into primary mode in case it was in secondary before. Now
>> this will only happen if the mutex is contended!
> 
> Yes, I had already thought about that, and even mentioned it on the
> mailing list I believe.

Sorry, /me probably missed it.

> 
>> Let's consider the fairly typical use case of a two threads
>> synchronizing a critical section with the help of a mutex. One thread is
>> high-prio, always in primary mode, the other is low-prio, constantly
>> transiting between primary (while holding the lock) and secondary (while
>> interacting with Linux). If the low-prio acquires the uncontended lock,
>> it will now remain in secondary mode thanks to the new mutex fast path.
>> That means if the high-prio requests the lock as well, prio-inheritance
>> will no longer work as the owner is not in the right mode!
>>
>> I guess what we need is mode detection for the caller in the mutex fast
>> path. If the possible new owner is in secondary mode, the syscall path
>> needs to be taken to trigger a migration reliably. That in turn means we
>> need a syscall-less detection of the current execution mode. Any
>> spontaneous ideas on this?
> 
> Is not it even simpler for the oscillating thread to switch to primary
> mode when it accesses such a mutex? Otherwise we have kernel/user shared
>  maps, so allocating a few bytes there should be enough.

As we are already fighting hard to avoid new explicit mode-switch use
cases, rather get rid of old ones, I thought it would be better to keep
existing semantic across the fast mutex changes.

Regarding those shared maps: they are per process, aren't they? But here
we need per thread memory. I'm currently thinking of a memory piece that
the caller of xeno_set_current provides (or some service that is called
together with the latter one, or some extended services - to keep APIs
clean). Then, on mode switches, the kernel will carefully
(xn_copy_to_user) update it.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Fast mutexes vs. automatic mode switch

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> As we are already fighting hard to avoid new explicit mode-switch use
> cases, rather get rid of old ones, I thought it would be better to keep
> existing semantic across the fast mutex changes.
> 
> Regarding those shared maps: they are per process, aren't they? But here
> we need per thread memory. I'm currently thinking of a memory piece that
> the caller of xeno_set_current provides (or some service that is called
> together with the latter one, or some extended services - to keep APIs
> clean). Then, on mode switches, the kernel will carefully
> (xn_copy_to_user) update it.

We can allocate a piece of memory on the shared heap, and attach it to a
thread using a pthread_key.

-- 
                                                 Gilles.

Re: [Xenomai-core] Fast mutexes vs. automatic mode switch

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> As we are already fighting hard to avoid new explicit mode-switch use
>> cases, rather get rid of old ones, I thought it would be better to keep
>> existing semantic across the fast mutex changes.
>>
>> Regarding those shared maps: they are per process, aren't they? But here
>> we need per thread memory. I'm currently thinking of a memory piece that
>> the caller of xeno_set_current provides (or some service that is called
>> together with the latter one, or some extended services - to keep APIs
>> clean). Then, on mode switches, the kernel will carefully
>> (xn_copy_to_user) update it.
> 
> We can allocate a piece of memory on the shared heap, and attach it to a
> thread using a pthread_key.

What would be the advantage of using a heap chunk over doing xn_copy_*?
I haven't looked into details yet but, in contrast to the mutex stuff, I
don't expect that we will have to access that data from within critical
regions.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Fast mutexes vs. automatic mode switch

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> As we are already fighting hard to avoid new explicit mode-switch use
>>> cases, rather get rid of old ones, I thought it would be better to keep
>>> existing semantic across the fast mutex changes.
>>>
>>> Regarding those shared maps: they are per process, aren't they? But here
>>> we need per thread memory. I'm currently thinking of a memory piece that
>>> the caller of xeno_set_current provides (or some service that is called
>>> together with the latter one, or some extended services - to keep APIs
>>> clean). Then, on mode switches, the kernel will carefully
>>> (xn_copy_to_user) update it.
>> We can allocate a piece of memory on the shared heap, and attach it to a
>> thread using a pthread_key.
> 
> What would be the advantage of using a heap chunk over doing xn_copy_*?
> I haven't looked into details yet but, in contrast to the mutex stuff, I
> don't expect that we will have to access that data from within critical
> regions.

Well, it is a more general solution, which should allow to access many
details in user-space. But I find it no real use right now.

-- 
					    Gilles.