priority based thread wakeup

priority based thread wakeup

[Mark Hansen]

Hello,

Firstly, may I apologise as I am not a member of the LKML, and ask that 
I be CC'd in any responses that may be forthcoming.

My question concerns the following patch which was incorporated into the 
2.6.22 kernel (quoted from that change log):

>Today, all threads waiting for a given futex are woken in FIFO 
>order (first waiter woken first) instead of priority order.
>
>This patch makes use of plist (pirotity ordered lists) instead 
>of simple list in futex_hash_bucket.
>
>All non-RT threads are stored with priority MAX_RT_PRIO, causing
>them to be woken last, in FIFO order (RT-threads are woken first, 
>in priority order).
>
>Signed-off-by: Sebastien Dugue <sebastien.du...@bull.net>
>Signed-off-by: Pierre Peiffer <pierre.peif...@bull.net>

After updating to this version of the kernel, I was able to observe the 
above fix, where multiple RT threads invoking pthread_cond_wait(), and 
the highest priority thread will acquire the mutex first, after the 
thread holding the mutex calls pthread_cond_signal(); 
pthread_mutex_unlock()

However, since kernel 2.6.23, it seems that the functionality relating 
to this "priority based wakeup" has disappeared. 

I understand there have been significant changes in this kernel 
concerning the "Completely Fair Scheduler" replacing the "mainline" 
scheduler; however my understanding is that the RT functionality would 
be preserved. This does not appear to be the case based on repeating the 
experiment described above.

I was wondering if this functionality is considered no longer 
desirable/necessary?

If not, is it anticipated that this functionality could/would be 
included in a later kernel?

Regards,
Mark Hansen

Re: priority based thread wakeup

[Gregory Haskins]

Mark Hansen wrote:
> Hello,
> 
> Firstly, may I apologise as I am not a member of the LKML, and ask that 
> I be CC'd in any responses that may be forthcoming.
> 
> My question concerns the following patch which was incorporated into the 
> 2.6.22 kernel (quoted from that change log):
> 
>> Today, all threads waiting for a given futex are woken in FIFO 
>> order (first waiter woken first) instead of priority order.
>>
>> This patch makes use of plist (pirotity ordered lists) instead 
>> of simple list in futex_hash_bucket.
>>
>> All non-RT threads are stored with priority MAX_RT_PRIO, causing
>> them to be woken last, in FIFO order (RT-threads are woken first, 
>> in priority order).
>>
>> Signed-off-by: Sebastien Dugue <sebastien.du...@bull.net>
>> Signed-off-by: Pierre Peiffer <pierre.peif...@bull.net>
> 
> After updating to this version of the kernel, I was able to observe the 
> above fix, where multiple RT threads invoking pthread_cond_wait(), and 
> the highest priority thread will acquire the mutex first, after the 
> thread holding the mutex calls pthread_cond_signal(); 
> pthread_mutex_unlock()
> 
> However, since kernel 2.6.23, it seems that the functionality relating 
> to this "priority based wakeup" has disappeared. 
> 
> I understand there have been significant changes in this kernel 
> concerning the "Completely Fair Scheduler" replacing the "mainline" 
> scheduler; however my understanding is that the RT functionality would 
> be preserved. This does not appear to be the case based on repeating the 
> experiment described above.
> 
> I was wondering if this functionality is considered no longer 
> desirable/necessary?
> 
> If not, is it anticipated that this functionality could/would be 
> included in a later kernel?
>

Hi Mark,
   In a coffee deprived stupor ;), I responded to this mail in the wrong 
thread, here:

http://lkml.org/lkml/2008/1/18/207

Sorry for the confusion
HTH

-Greg


> Regards,
> Mark Hansen
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

RE: priority based thread wakeup

[Mark Hansen]

>>My question concerns the following patch which was incorporated into 
the >>2.6.22 kernel (quoted from that change log):
... "futex priority based wakeup"
>>After updating to this version of the kernel, I was able to observe 
the >>above fix, where multiple RT threads invoking pthread_cond_wait(), 
and the >>highest priority thread will acquire the mutex first, after 
the thread >>holding the mutex calls pthread_cond_signal(); 
pthread_mutex_unlock()

>>However, since kernel 2.6.23, it seems that the functionality relating 
to >>this "priority based wakeup" has disappeared. 

>>I understand there have been significant changes in this kernel 
concerning >>the "Completely Fair Scheduler" replacing the "mainline" 
scheduler; >>however my understanding is that the RT functionality would 
be preserved. >>This does not appear to be the case based on repeating 
the experiment >>described above.

>Hi,
>   I am not sure if this is whats happening to you (*), but there is a 
>known problem in the mainline tree related to the system allowing lower 
>priority threads to stay on the CPU under certain circumstances.
>
>You can see the problem happening with the following test:
>
>http://rt.wiki.kernel.org/index.php/Preemption_Test
>
>and I know the problem goes back to at least 2.6.16.
>
>Steven Rostedt and I submitted some patches to address the issue, and 
>they have been accepted into Ingo's sched-devel tree.  I would suggest 
>trying your code against Ingo's tree and see if the problem persists. 
>You can get it from:
>
>git://git.kernel.org/pub/scm/linux/kernel/git/mingo/linux-2.6-sched->de
vel.git

Thanks for the suggestion, however I've just tried this kernel, but the 
pthread_cond_wait() behaviour still remains as per 2.6.23 described 
above (i.e. highest prio thread doesnt get woken first).

>>(*) I have no information on whether the futex-plist implemetation was 

>>pulled from the tree to cause your regression.  

I've just had a look at the code from the 2.6.24 kernel and it seems as 
though the futex priority list patch code is still included. 

>>It is possible that the changes between 22 and 23 are just tickling 
>>your environment enough to bring out this RT-preempt issue.

Or that the changes between 22 and 23 are bringing out another issue 
whereby the desired behaviour of futex-plist is somehow being 
circumvented.

If so, my questions still stand to anyone who might know what has caused 
the change in pthread_cond_wait() behaviour for RT threads between the 
22 and 23 kernels.

Cheers,
Mark