[PATCH] sched: Optimize branch hint in context_switch()

[PATCH] sched: Optimize branch hint in context_switch()

[Tim Blechmann]

[-- Attachment #1.1: Type: text/plain, Size: 385 bytes --]


Branch hint profiling on my nehalem machine showed 88%
incorrect branch hints:

42017484 326957902  88 context_switch                 sched.c              3043
42038493 326953687  88 context_switch                 sched.c              3050

Signed-off-by: Tim Blechmann <tim@klingt.org>
---
 kernel/sched.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-sched-Optimize-branch-hint-in-context_switch.patch --]
[-- Type: text/x-patch; name="0001-sched-Optimize-branch-hint-in-context_switch.patch", Size: 553 bytes --]

diff --git a/kernel/sched.c b/kernel/sched.c
index 2a78b38..c78dcfe 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,
 	 */
 	arch_start_context_switch(prev);
 
-	if (likely(!mm)) {
+	if (unlikely(!mm)) {
 		next->active_mm = oldmm;
 		atomic_inc(&oldmm->mm_count);
 		enter_lazy_tlb(oldmm, next);
 	} else
 		switch_mm(oldmm, mm, next);
 
-	if (likely(!prev->mm)) {
+	if (unlikely(!prev->mm)) {
 		prev->active_mm = NULL;
 		rq->prev_mm = oldmm;
 	}


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH] sched: Optimize branch hint in context_switch()

[Avi Kivity]

On 11/29/2009 02:01 PM, Tim Blechmann wrote:
> Branch hint profiling on my nehalem machine showed 88%
> incorrect branch hints:
>
> 42017484 326957902  88 context_switch                 sched.c              3043
> 42038493 326953687  88 context_switch                 sched.c              3050
>
> @@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,
>   	 */
>   	arch_start_context_switch(prev);
>
> -	if (likely(!mm)) {
> +	if (unlikely(!mm)) {
>   		next->active_mm = oldmm;
>   		atomic_inc(&oldmm->mm_count);
>   		enter_lazy_tlb(oldmm, next);
>   	} else
>   		switch_mm(oldmm, mm, next);
>
> -	if (likely(!prev->mm)) {
> +	if (unlikely(!prev->mm)) {
>   		prev->active_mm = NULL;
>   		rq->prev_mm = oldmm;
>   	}
>    

I don't think either the original or the patch is correct.  Whether or 
not a task has an mm is entirely workload dependent, we shouldn't be 
giving hints here.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH] sched: Optimize branch hint in context_switch()

[Peter Zijlstra]

On Sun, 2009-11-29 at 17:12 +0200, Avi Kivity wrote:
> On 11/29/2009 02:01 PM, Tim Blechmann wrote:
> > Branch hint profiling on my nehalem machine showed 88%
> > incorrect branch hints:
> >
> > 42017484 326957902  88 context_switch                 sched.c              3043
> > 42038493 326953687  88 context_switch                 sched.c              3050
> >
> > @@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,
> >   	 */
> >   	arch_start_context_switch(prev);
> >
> > -	if (likely(!mm)) {
> > +	if (unlikely(!mm)) {
> >   		next->active_mm = oldmm;
> >   		atomic_inc(&oldmm->mm_count);
> >   		enter_lazy_tlb(oldmm, next);
> >   	} else
> >   		switch_mm(oldmm, mm, next);
> >
> > -	if (likely(!prev->mm)) {
> > +	if (unlikely(!prev->mm)) {
> >   		prev->active_mm = NULL;
> >   		rq->prev_mm = oldmm;
> >   	}
> >    
> 
> I don't think either the original or the patch is correct.  Whether or 
> not a task has an mm is entirely workload dependent, we shouldn't be 
> giving hints here.

There are reasons to still use branch hints, for example if the unlikely
branch is very expensive anyway and it pays to have the likely branch be
ever so slightly less expensive.

Now I don't think that applies here, but there are cases where such code
generation issues are the main motivator not the actual usage patterns.

Re: [PATCH] sched: Optimize branch hint in context_switch()

[Avi Kivity]

On 11/29/2009 05:20 PM, Peter Zijlstra wrote:
> On Sun, 2009-11-29 at 17:12 +0200, Avi Kivity wrote:
>    
>> On 11/29/2009 02:01 PM, Tim Blechmann wrote:
>>      
>>> Branch hint profiling on my nehalem machine showed 88%
>>> incorrect branch hints:
>>>
>>> 42017484 326957902  88 context_switch                 sched.c              3043
>>> 42038493 326953687  88 context_switch                 sched.c              3050
>>>
>>> @@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,
>>>    	 */
>>>    	arch_start_context_switch(prev);
>>>
>>> -	if (likely(!mm)) {
>>> +	if (unlikely(!mm)) {
>>>    		next->active_mm = oldmm;
>>>    		atomic_inc(&oldmm->mm_count);
>>>    		enter_lazy_tlb(oldmm, next);
>>>    	} else
>>>    		switch_mm(oldmm, mm, next);
>>>
>>> -	if (likely(!prev->mm)) {
>>> +	if (unlikely(!prev->mm)) {
>>>    		prev->active_mm = NULL;
>>>    		rq->prev_mm = oldmm;
>>>    	}
>>>
>>>        
>> I don't think either the original or the patch is correct.  Whether or
>> not a task has an mm is entirely workload dependent, we shouldn't be
>> giving hints here.
>>      
> There are reasons to still use branch hints, for example if the unlikely
> branch is very expensive anyway and it pays to have the likely branch be
> ever so slightly less expensive.
>
> Now I don't think that applies here, but there are cases where such code
> generation issues are the main motivator not the actual usage patterns.
>    

These should be documented then to avoid patches removing them:

      #define slowpath(x) unlikely(x)

      if (slowpath(condition))
            expensive_operation();

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH] sched: Optimize branch hint in context_switch()

[Tim Blechmann]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

On 11/29/2009 04:25 PM, Avi Kivity wrote:
> On 11/29/2009 05:20 PM, Peter Zijlstra wrote:
>> On Sun, 2009-11-29 at 17:12 +0200, Avi Kivity wrote:
>>    
>>> On 11/29/2009 02:01 PM, Tim Blechmann wrote:
>>>      
>>>> Branch hint profiling on my nehalem machine showed 88%
>>>> incorrect branch hints:
>>>>
>>>> 42017484 326957902  88 context_switch                 sched.c              3043
>>>> 42038493 326953687  88 context_switch                 sched.c              3050
>>>>
>>>> @@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,
>>>>    	 */
>>>>    	arch_start_context_switch(prev);
>>>>
>>>> -	if (likely(!mm)) {
>>>> +	if (unlikely(!mm)) {
>>>>    		next->active_mm = oldmm;
>>>>    		atomic_inc(&oldmm->mm_count);
>>>>    		enter_lazy_tlb(oldmm, next);
>>>>    	} else
>>>>    		switch_mm(oldmm, mm, next);
>>>>
>>>> -	if (likely(!prev->mm)) {
>>>> +	if (unlikely(!prev->mm)) {
>>>>    		prev->active_mm = NULL;
>>>>    		rq->prev_mm = oldmm;
>>>>    	}
>>>>
>>>>        
>>> I don't think either the original or the patch is correct.  Whether or
>>> not a task has an mm is entirely workload dependent, we shouldn't be
>>> giving hints here.
>>>      
>> There are reasons to still use branch hints, for example if the unlikely
>> branch is very expensive anyway and it pays to have the likely branch be
>> ever so slightly less expensive.
>>
>> Now I don't think that applies here, but there are cases where such code
>> generation issues are the main motivator not the actual usage patterns.

would be nice, if you commit a patch, removing this hint

> These should be documented then to avoid patches removing them:
> 
>       #define slowpath(x) unlikely(x)
> 
>       if (slowpath(condition))
>             expensive_operation();

this would definitely improve the expressive power ...

thnx, tim

-- 
tim@klingt.org
http://tim.klingt.org

Only very good and very bad programmers use goto in C


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH] sched: Optimize branch hint in context_switch()

[Christian Borntraeger]

Am Sonntag 29 November 2009 16:25:43 schrieb Avi Kivity:
> These should be documented then to avoid patches removing them:
> 
>       #define slowpath(x) unlikely(x)
> 
>       if (slowpath(condition))
>             expensive_operation();

Neat. If we also modify the likelyhood tracer to __not__ check the
slowpath annotation by default this really looks like a good idea.

Christian