[Xenomai] Mutex enhancement for Auto relaxed threads

[Xenomai] Mutex enhancement for Auto relaxed threads

[Makarand Pradhan]

Greetings,

I have a proposal for enhancing the process of locking a real-time mutex in auto-relaxed Xenomai threads. It will  save a lot of CPU cycles in some scenarios, and I would like to get your expert opinion about it (I have coded the enhancement for native skin and changes are attached to this email).

Xenomai mutex is designed in such a way that trapping to the kernel is not needed.This is a very efficient design which saves a great deal of CPU cycles.
However, there is an exception to this efficient behavior: if the Xenomai thread is an auto-relaxed thread (i.e. running with priority 0), locking a mutex always traps to the kernel because the current implementation handles auto-relaxing upon exiting the system call.

Look at this scenario:

A Xenomai thread
1) locks a mutex (which is an OUTER mutex in this scenario)
2) then loops multiple times through some operation that requires locking an unlocking some other mutex (INNER mutex)
3) unlocks the OUTER mutex

If the thread is a real-time thread, locking mutexes is done efficiently. If the thread is auto-relaxed though, both OUTER and INNER mutexes cause trapping to the kernel which is not really needed.
Please note that locking the OUTER mutex already moves the thread to the primary domain, so handling the INNER mutex could be done as if the thread was a real-time (non-auto-relaxed) thread.

Enhancements:
The mutex code was modified in such a way that priority 0 (auto-relaxed) thread will do a system call only once when grabbing and releasing the outtermost mutex. The inner mutexes will not jump into the kernel. This change would significantly save CPU cycles by using the user space fast mutexes when the thread is not relaxed after grabbing the outtermost mutex.


I would highly appreciate your comments.

Best Regards,
Makarand
Software Engineer
RuggedCom Inc.
www.ruggedcom.com
300 Applewood Crescent
Concord, Ontario, L4K 5C7
Tel:  (905)482-4523
Fax: (905)856-1995





-- 
___________________________________________________________________________
NOTICE OF CONFIDENTIALITY:
This e-mail and any attachments may contain confidential and privileged information.  If you are
not the intended recipient, please notify the sender immediately by return e-mail and delete this
e-mail and any copies.  Any dissemination or use of this information by a person other than the
intended recipient is unauthorized and may be illegal.
_____________________________________________________________________

  

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: xenomai_changes
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20121122/fc104aa6/attachment.ksh>

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Gilles Chanteperdrix]

On 11/22/2012 04:51 PM, Makarand Pradhan wrote:
> Greetings,
> 
> I have a proposal for enhancing the process of locking a real-time mutex in auto-relaxed Xenomai threads. It will  save a lot of CPU cycles in some scenarios, and I would like to get your expert opinion about it (I have coded the enhancement for native skin and changes are attached to this email).
> 
> Xenomai mutex is designed in such a way that trapping to the kernel is not needed.This is a very efficient design which saves a great deal of CPU cycles.
> However, there is an exception to this efficient behavior: if the Xenomai thread is an auto-relaxed thread (i.e. running with priority 0), locking a mutex always traps to the kernel because the current implementation handles auto-relaxing upon exiting the system call.
> 
> Look at this scenario:
> 
> A Xenomai thread
> 1) locks a mutex (which is an OUTER mutex in this scenario)
> 2) then loops multiple times through some operation that requires locking an unlocking some other mutex (INNER mutex)
> 3) unlocks the OUTER mutex

I did not look in details at your code, but what if I do

lock mutex1
lock mutex2
unlock mutex1
unlock mutex2

?

-- 
					    Gilles.

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Makarand Pradhan]

Hi Gilles,

lock mutex1
lock mutex2
unlock mutex1
unlock mutex2

As per my current implementation, I expect the thread to relax after 
unlocking mutex1. If another RT thread starts waiting on mutex2, I 
expect the thread holding the mutex will harden and start running in 
primary again. Will quickly test this and get back.

I did consider this scenario but since we never use such a behaviour in 
our system, did not pursue it further.

This brings up another question that I was contemplating. Do we want to 
harden unless there is another RT thread wanting this mutex? If not then 
the implementation becomes simpler and the scenario mentioned would also 
get addressed automatically.

Rgds,
Mak.


> I did not look in details at your code, but what if I do
>
> lock mutex1
> lock mutex2
> unlock mutex1
> unlock mutex2
>
> ?
>


-- 
___________________________________________________________________________
NOTICE OF CONFIDENTIALITY:
This e-mail and any attachments may contain confidential and privileged information.  If you are
not the intended recipient, please notify the sender immediately by return e-mail and delete this
e-mail and any copies.  Any dissemination or use of this information by a person other than the
intended recipient is unauthorized and may be illegal.
_____________________________________________________________________

  

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Makarand Pradhan]

>  Will quickly test this and get back.

Yes. That's pretty much how the system behaves.

Rgds,
Makarand.

Test program attached:
Results:
task1: Prio 0 Auto relaxed
task2: Prio 10. RT.
root@ruggedcom:~# ./tc5 0 10 1
Format: tc5 prio1 prio2 mux_iterations.
Spawning: task1
task2: Mutexes created
task2: Mutexes in memory
Running task1: status: b00380
task1: Global test: 25
task1: Acquiring mux1
rt_mutex_acquire_inner: Did system call. lockcnt: 1, dsof: 1
task1: Acquired mux1. rc: 0
task1: status: b00180
task1: Acquiring mux2
task1: Acquired mux2. rc: 0
task1: status: b00180
task1: Releasing mux1
rt_mutex_release: Doing system call. lockcnt: 0, dsof: 1
task1: Released mux1. rc: 0
task1: status: b00380 <- Relaxed task1 while holding mutex 2.
task1: bprio: 0 cprio: 0 status: b00380 info.status: b00380
task2: Acquiring mux2.
task1: bprio: 0 cprio: 10 status: b01180 info.status: b01180 <- task1 
jumps to primary again..
task1: Releasing mux2
rt_mutex_release: Doing system call. lockcnt: 0, dsof: 1
rt_mutex_acquire_inner: Did system call. lockcnt: 1, dsof: 1
task2: Acquired mux2.
task2: Releasing mux2
rt_mutex_release: Doing system call. lockcnt: 0, dsof: 1
task2: Released mux2. rc: 0
task2: status: 300180
task1: Released mux2. rc: 0
task1: status: b00380


On 22/11/12 11:47 AM, Makarand Pradhan wrote:
> Hi Gilles,
>
> lock mutex1
> lock mutex2
> unlock mutex1
> unlock mutex2
>
> As per my current implementation, I expect the thread to relax after
> unlocking mutex1. If another RT thread starts waiting on mutex2, I
> expect the thread holding the mutex will harden and start running in
> primary again. Will quickly test this and get back.
>
> I did consider this scenario but since we never use such a behaviour in
> our system, did not pursue it further.
>
> This brings up another question that I was contemplating. Do we want to
> harden unless there is another RT thread wanting this mutex? If not then
> the implementation becomes simpler and the scenario mentioned would also
> get addressed automatically.
>
> Rgds,
> Mak.
>
>
>> I did not look in details at your code, but what if I do
>>
>> lock mutex1
>> lock mutex2
>> unlock mutex1
>> unlock mutex2
>>
>> ?
>>
>


-- 
___________________________________________________________________________
NOTICE OF CONFIDENTIALITY:
This e-mail and any attachments may contain confidential and privileged information.  If you are
not the intended recipient, please notify the sender immediately by return e-mail and delete this
e-mail and any copies.  Any dissemination or use of this information by a person other than the
intended recipient is unauthorized and may be illegal.
_____________________________________________________________________

  

-------------- next part --------------
A non-text attachment was scrubbed...
Name: tc5.c
Type: text/x-csrc
Size: 4314 bytes
Desc: not available
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20121122/9631fb18/attachment.c>

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Makarand Pradhan]

Hi,

Made a small change.

So re sending changes.

Rgds,
Makarand.

On 22/11/12 02:24 PM, Makarand Pradhan wrote:
>>   Will quickly test this and get back.
> Yes. That's pretty much how the system behaves.
>
> Rgds,
> Makarand.
>
> Test program attached:
> Results:
> task1: Prio 0 Auto relaxed
> task2: Prio 10. RT.
> root@ruggedcom:~# ./tc5 0 10 1
> Format: tc5 prio1 prio2 mux_iterations.
> Spawning: task1
> task2: Mutexes created
> task2: Mutexes in memory
> Running task1: status: b00380
> task1: Global test: 25
> task1: Acquiring mux1
> rt_mutex_acquire_inner: Did system call. lockcnt: 1, dsof: 1
> task1: Acquired mux1. rc: 0
> task1: status: b00180
> task1: Acquiring mux2
> task1: Acquired mux2. rc: 0
> task1: status: b00180
> task1: Releasing mux1
> rt_mutex_release: Doing system call. lockcnt: 0, dsof: 1
> task1: Released mux1. rc: 0
> task1: status: b00380<- Relaxed task1 while holding mutex 2.
> task1: bprio: 0 cprio: 0 status: b00380 info.status: b00380
> task2: Acquiring mux2.
> task1: bprio: 0 cprio: 10 status: b01180 info.status: b01180<- task1
> jumps to primary again..
> task1: Releasing mux2
> rt_mutex_release: Doing system call. lockcnt: 0, dsof: 1
> rt_mutex_acquire_inner: Did system call. lockcnt: 1, dsof: 1
> task2: Acquired mux2.
> task2: Releasing mux2
> rt_mutex_release: Doing system call. lockcnt: 0, dsof: 1
> task2: Released mux2. rc: 0
> task2: status: 300180
> task1: Released mux2. rc: 0
> task1: status: b00380
>
>
> On 22/11/12 11:47 AM, Makarand Pradhan wrote:
>> Hi Gilles,
>>
>> lock mutex1
>> lock mutex2
>> unlock mutex1
>> unlock mutex2
>>
>> As per my current implementation, I expect the thread to relax after
>> unlocking mutex1. If another RT thread starts waiting on mutex2, I
>> expect the thread holding the mutex will harden and start running in
>> primary again. Will quickly test this and get back.
>>
>> I did consider this scenario but since we never use such a behaviour in
>> our system, did not pursue it further.
>>
>> This brings up another question that I was contemplating. Do we want to
>> harden unless there is another RT thread wanting this mutex? If not then
>> the implementation becomes simpler and the scenario mentioned would also
>> get addressed automatically.
>>
>> Rgds,
>> Mak.
>>
>>
>>> I did not look in details at your code, but what if I do
>>>
>>> lock mutex1
>>> lock mutex2
>>> unlock mutex1
>>> unlock mutex2
>>>
>>> ?
>>>
>


-- 
___________________________________________________________________________
NOTICE OF CONFIDENTIALITY:
This e-mail and any attachments may contain confidential and privileged information.  If you are
not the intended recipient, please notify the sender immediately by return e-mail and delete this
e-mail and any copies.  Any dissemination or use of this information by a person other than the
intended recipient is unauthorized and may be illegal.
_____________________________________________________________________

  

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: xenomai_changes
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20121122/bd0c4072/attachment.ksh>

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Michael Pustylnik]

Although you can use mutexes in this order, why would you want to do that? Logically, unlocking mutex1 would mean finishing some operation started when mutex1 was locked. If as a part of the operation you locked mutex2 and haven't unlocked it yet, that would mean you haven't completed your operation yet, and unlocking mutex1 at this point would probably be a bug. Am I wrong?

Michael Pustylnik, P.Eng.
Senior Software Engineer
RuggedCom Inc.
www.ruggedcom.com
300 Applewood Crescent
Concord, Ontario, L4K 5C7
Tel:  (905)482-4523
Fax: (905)856-1995

 


-----Original Message-----
From: xenomai-bounces@xenomai.org [mailto:xenomai-bounces@xenomai.org] On Behalf Of Gilles Chanteperdrix
Sent: Thursday, November 22, 2012 11:15 AM
To: Makarand Pradhan
Cc: xenomai@xenomai.org
Subject: Re: [Xenomai] Mutex enhancement for Auto relaxed threads

On 11/22/2012 04:51 PM, Makarand Pradhan wrote:
> Greetings,
> 
> I have a proposal for enhancing the process of locking a real-time mutex in auto-relaxed Xenomai threads. It will  save a lot of CPU cycles in some scenarios, and I would like to get your expert opinion about it (I have coded the enhancement for native skin and changes are attached to this email).
> 
> Xenomai mutex is designed in such a way that trapping to the kernel is not needed.This is a very efficient design which saves a great deal of CPU cycles.
> However, there is an exception to this efficient behavior: if the Xenomai thread is an auto-relaxed thread (i.e. running with priority 0), locking a mutex always traps to the kernel because the current implementation handles auto-relaxing upon exiting the system call.
> 
> Look at this scenario:
> 
> A Xenomai thread
> 1) locks a mutex (which is an OUTER mutex in this scenario)
> 2) then loops multiple times through some operation that requires locking an unlocking some other mutex (INNER mutex)
> 3) unlocks the OUTER mutex

I did not look in details at your code, but what if I do

lock mutex1
lock mutex2
unlock mutex1
unlock mutex2

?

-- 
					    Gilles.

_______________________________________________
Xenomai mailing list
Xenomai@xenomai.org
http://www.xenomai.org/mailman/listinfo/xenomai

No virus found in this incoming message.
Checked by AVG - www.avg.com 
Version: 8.5.455 / Virus Database: 271.1.1/5393 - Release Date: 11/21/12 20:05:00

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Jan Kiszka]

On 2012-11-22 17:55, Michael Pustylnik wrote:
> Although you can use mutexes in this order, why would you want to do that? Logically, unlocking mutex1 would mean finishing some operation started when mutex1 was locked. If as a part of the operation you locked mutex2 and haven't unlocked it yet, that would mean you haven't completed your operation yet, and unlocking mutex1 at this point would probably be a bug. Am I wrong?

I'm not aware that POSIX or some other APIs Xenomai implements forbid
such an ordering. And you never know if there is weird convoluted legacy
code out there that depends on exactly this.

However, I think you can avoid the limitation by attaching the property
"trap to kernel on mutex lock" to the thread instead of some mutex,
analogously to xeno_current_mode.

Jan

PS: Please no top postings!

> -----Original Message-----
> From: xenomai-bounces@xenomai.org [mailto:xenomai-bounces@xenomai.org] On Behalf Of Gilles Chanteperdrix
> Sent: Thursday, November 22, 2012 11:15 AM
> To: Makarand Pradhan
> Cc: xenomai@xenomai.org
> Subject: Re: [Xenomai] Mutex enhancement for Auto relaxed threads
> 
> On 11/22/2012 04:51 PM, Makarand Pradhan wrote:
>> Greetings,
>>
>> I have a proposal for enhancing the process of locking a real-time mutex in auto-relaxed Xenomai threads. It will  save a lot of CPU cycles in some scenarios, and I would like to get your expert opinion about it (I have coded the enhancement for native skin and changes are attached to this email).
>>
>> Xenomai mutex is designed in such a way that trapping to the kernel is not needed.This is a very efficient design which saves a great deal of CPU cycles.
>> However, there is an exception to this efficient behavior: if the Xenomai thread is an auto-relaxed thread (i.e. running with priority 0), locking a mutex always traps to the kernel because the current implementation handles auto-relaxing upon exiting the system call.
>>
>> Look at this scenario:
>>
>> A Xenomai thread
>> 1) locks a mutex (which is an OUTER mutex in this scenario)
>> 2) then loops multiple times through some operation that requires locking an unlocking some other mutex (INNER mutex)
>> 3) unlocks the OUTER mutex
> 
> I did not look in details at your code, but what if I do
> 
> lock mutex1
> lock mutex2
> unlock mutex1
> unlock mutex2
> 
> ?
> 

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Gilles Chanteperdrix]

On 11/23/2012 12:15 PM, Jan Kiszka wrote:
> On 2012-11-22 17:55, Michael Pustylnik wrote:
>> Although you can use mutexes in this order, why would you want to do that? Logically, unlocking mutex1 would mean finishing some operation started when mutex1 was locked. If as a part of the operation you locked mutex2 and haven't unlocked it yet, that would mean you haven't completed your operation yet, and unlocking mutex1 at this point would probably be a bug. Am I wrong?
> 
> I'm not aware that POSIX or some other APIs Xenomai implements forbid
> such an ordering. And you never know if there is weird convoluted legacy
> code out there that depends on exactly this.
> 
> However, I think you can avoid the limitation by attaching the property
> "trap to kernel on mutex lock" to the thread instead of some mutex,
> analogously to xeno_current_mode.

I think the simplest solution to implement this is to share the
resources count between user and kernel, using a pthread_key in
user-space. However, I really do not see how to do this without breaking
the ABI, so, this really is material for xenomai-forge, not for 2.6.

-- 
					    Gilles.

Re: [Xenomai] Mutex enhancement for Auto relaxed threads

[Jan Kiszka]

On 2012-11-27 14:46, Gilles Chanteperdrix wrote:
> On 11/23/2012 12:15 PM, Jan Kiszka wrote:
>> On 2012-11-22 17:55, Michael Pustylnik wrote:
>>> Although you can use mutexes in this order, why would you want to do that? Logically, unlocking mutex1 would mean finishing some operation started when mutex1 was locked. If as a part of the operation you locked mutex2 and haven't unlocked it yet, that would mean you haven't completed your operation yet, and unlocking mutex1 at this point would probably be a bug. Am I wrong?
>>
>> I'm not aware that POSIX or some other APIs Xenomai implements forbid
>> such an ordering. And you never know if there is weird convoluted legacy
>> code out there that depends on exactly this.
>>
>> However, I think you can avoid the limitation by attaching the property
>> "trap to kernel on mutex lock" to the thread instead of some mutex,
>> analogously to xeno_current_mode.
> 
> I think the simplest solution to implement this is to share the
> resources count between user and kernel, using a pthread_key in
> user-space. However, I really do not see how to do this without breaking
> the ABI, so, this really is material for xenomai-forge, not for 2.6.

For sure.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux