[PATCH 0/2] sched: misc fixes for stable-25.y and 25-rt

[PATCH 0/2] sched: misc fixes for stable-25.y and 25-rt

[Gregory Haskins]

Hi Ingo, Steven, Peter,
  I found a few minor issues w.r.t. the CFS load values during a few
  dequeue/enqueue operations in the 2.6.25 kernel.  I found these via
  code review as opposed to observing a bug in the field.  I believe
  at least the first issue is already fixed in 26-rcx.  I have
  confirmed that the issue is present in both 25.8-rt7 as well as
  stable-2.6.25.10.  These patches are against 25.8-rt7, though they
  should apply trivially to stable as well.  Please consider for
  inclusion in the next -rt if the -stable team does not pick them up
  immediately.

Stable team,
  Please consider these patches for 25.11 (assuming the appropriate acks
  from Ingo et. al. are recieved) though I do not believe they are
  considered significant risks and are therefore not urgent.

Feedback welcome..

Thanks!
-Greg

[PATCH 1/2] sched: remove extraneous load manipulations

[Gregory Haskins]

commit 62fb185130e4d420f71a30ff59d8b16b74ef5d2b reverted some patches
in the scheduler, but it looks like it may have left a few redundant
calls to inc_load/dec_load remain in set_user_nice (since the
 dequeue_task/enqueue_task take care of the load.  This could result
in the load values being off since the load may change while dequeued.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Ingo Molnar <mingo@elte.hu>
---

 kernel/sched.c |    6 ++----
 1 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index 31f91d9..b046754 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -4679,10 +4679,8 @@ void set_user_nice(struct task_struct *p, long nice)
 		goto out_unlock;
 	}
 	on_rq = p->se.on_rq;
-	if (on_rq) {
+	if (on_rq)
 		dequeue_task(rq, p, 0);
-		dec_load(rq, p);
-	}
 
 	p->static_prio = NICE_TO_PRIO(nice);
 	set_load_weight(p);
@@ -4692,7 +4690,7 @@ void set_user_nice(struct task_struct *p, long nice)
 
 	if (on_rq) {
 		enqueue_task(rq, p, 0);
-		inc_load(rq, p);
+
 		/*
 		 * If the task increased its priority or is running and
 		 * lowered its priority, then reschedule its CPU:

Re: [PATCH 1/2] sched: remove extraneous load manipulations

[Peter Zijlstra]

On Thu, 2008-07-03 at 15:37 -0600, Gregory Haskins wrote:
> commit 62fb185130e4d420f71a30ff59d8b16b74ef5d2b reverted some patches
> in the scheduler, but it looks like it may have left a few redundant
> calls to inc_load/dec_load remain in set_user_nice (since the
>  dequeue_task/enqueue_task take care of the load.  This could result
> in the load values being off since the load may change while dequeued.

I just checked out v2.6.25.10 but cannot see dequeue_task() do it.

deactivate_task() otoh does do it.

static void dequeue_task(struct rq *rq, struct task_struct *p, int sleep)
{
	p->sched_class->dequeue_task(rq, p, sleep);
	p->se.on_rq = 0;
}

vs

static void deactivate_task(struct rq *rq, struct task_struct *p, int sleep)
{
	if (task_contributes_to_load(p))
		rq->nr_uninterruptible++;

	dequeue_task(rq, p, sleep);
	dec_nr_running(p, rq);
}

where

static void dec_nr_running(struct task_struct *p, struct rq *rq)
{
	rq->nr_running--;
	dec_load(rq, p);
}

And since set_user_nice() actually changes the load we'd better not
forget to do this dec/inc load stuff.

So I'm thinking this patch would actually break stuff.

> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
> CC: Peter Zijlstra <peterz@infradead.org>
> CC: Ingo Molnar <mingo@elte.hu>
> ---
> 
>  kernel/sched.c |    6 ++----
>  1 files changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/sched.c b/kernel/sched.c
> index 31f91d9..b046754 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -4679,10 +4679,8 @@ void set_user_nice(struct task_struct *p, long nice)
>  		goto out_unlock;
>  	}
>  	on_rq = p->se.on_rq;
> -	if (on_rq) {
> +	if (on_rq)
>  		dequeue_task(rq, p, 0);
> -		dec_load(rq, p);
> -	}
>  
>  	p->static_prio = NICE_TO_PRIO(nice);
>  	set_load_weight(p);
> @@ -4692,7 +4690,7 @@ void set_user_nice(struct task_struct *p, long nice)
>  
>  	if (on_rq) {
>  		enqueue_task(rq, p, 0);
> -		inc_load(rq, p);
> +
>  		/*
>  		 * If the task increased its priority or is running and
>  		 * lowered its priority, then reschedule its CPU:
> 

Re: [PATCH 1/2] sched: remove extraneous load manipulations

[Gregory Haskins]

>>> On Fri, Jul 18, 2008 at  8:39 AM, in message <1216384754.28405.31.camel@twins>,
Peter Zijlstra <peterz@infradead.org> wrote: 
> On Thu, 2008-07-03 at 15:37 -0600, Gregory Haskins wrote:
>> commit 62fb185130e4d420f71a30ff59d8b16b74ef5d2b reverted some patches
>> in the scheduler, but it looks like it may have left a few redundant
>> calls to inc_load/dec_load remain in set_user_nice (since the
>>  dequeue_task/enqueue_task take care of the load.  This could result
>> in the load values being off since the load may change while dequeued.
> 
> I just checked out v2.6.25.10 but cannot see dequeue_task() do it.

Perhaps I was trying to hit a moving target, or did not have enough coffee that day ;)

I will look again to see if I made a mistake.

Thanks Peter,

-Greg

> 
> deactivate_task() otoh does do it.
> 
> static void dequeue_task(struct rq *rq, struct task_struct *p, int sleep)
> {
> 	p->sched_class->dequeue_task(rq, p, sleep);
> 	p->se.on_rq = 0;
> }
> 
> vs
> 
> static void deactivate_task(struct rq *rq, struct task_struct *p, int sleep)
> {
> 	if (task_contributes_to_load(p))
> 		rq->nr_uninterruptible++;
> 
> 	dequeue_task(rq, p, sleep);
> 	dec_nr_running(p, rq);
> }
> 
> where
> 
> static void dec_nr_running(struct task_struct *p, struct rq *rq)
> {
> 	rq->nr_running--;
> 	dec_load(rq, p);
> }
> 
> And since set_user_nice() actually changes the load we'd better not
> forget to do this dec/inc load stuff.
> 
> So I'm thinking this patch would actually break stuff.
> 
>> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
>> CC: Peter Zijlstra <peterz@infradead.org>
>> CC: Ingo Molnar <mingo@elte.hu>
>> ---
>> 
>>  kernel/sched.c |    6 ++----
>>  1 files changed, 2 insertions(+), 4 deletions(-)
>> 
>> diff --git a/kernel/sched.c b/kernel/sched.c
>> index 31f91d9..b046754 100644
>> --- a/kernel/sched.c
>> +++ b/kernel/sched.c
>> @@ -4679,10 +4679,8 @@ void set_user_nice(struct task_struct *p, long nice)
>>  		goto out_unlock;
>>  	}
>>  	on_rq = p->se.on_rq;
>> -	if (on_rq) {
>> +	if (on_rq)
>>  		dequeue_task(rq, p, 0);
>> -		dec_load(rq, p);
>> -	}
>>  
>>  	p->static_prio = NICE_TO_PRIO(nice);
>>  	set_load_weight(p);
>> @@ -4692,7 +4690,7 @@ void set_user_nice(struct task_struct *p, long nice)
>>  
>>  	if (on_rq) {
>>  		enqueue_task(rq, p, 0);
>> -		inc_load(rq, p);
>> +
>>  		/*
>>  		 * If the task increased its priority or is running and
>>  		 * lowered its priority, then reschedule its CPU:
>> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] sched: remove extraneous load manipulations

[Gregory Haskins]

[-- Attachment #1: Type: text/plain, Size: 2699 bytes --]

Peter Zijlstra wrote:
> On Thu, 2008-07-03 at 15:37 -0600, Gregory Haskins wrote:
>   
>> commit 62fb185130e4d420f71a30ff59d8b16b74ef5d2b reverted some patches
>> in the scheduler, but it looks like it may have left a few redundant
>> calls to inc_load/dec_load remain in set_user_nice (since the
>>  dequeue_task/enqueue_task take care of the load.  This could result
>> in the load values being off since the load may change while dequeued.
>>     
>
> I just checked out v2.6.25.10 but cannot see dequeue_task() do it.
>   

Indeed.  I think my eyes glazed over the dequeue vs deactivate, and 
enqueue vs activate.   Good catch, and sorry for the noise.  I was wrong..

Please ignore this patch.

-Greg


> deactivate_task() otoh does do it.
>
> static void dequeue_task(struct rq *rq, struct task_struct *p, int sleep)
> {
> 	p->sched_class->dequeue_task(rq, p, sleep);
> 	p->se.on_rq = 0;
> }
>
> vs
>
> static void deactivate_task(struct rq *rq, struct task_struct *p, int sleep)
> {
> 	if (task_contributes_to_load(p))
> 		rq->nr_uninterruptible++;
>
> 	dequeue_task(rq, p, sleep);
> 	dec_nr_running(p, rq);
> }
>
> where
>
> static void dec_nr_running(struct task_struct *p, struct rq *rq)
> {
> 	rq->nr_running--;
> 	dec_load(rq, p);
> }
>
> And since set_user_nice() actually changes the load we'd better not
> forget to do this dec/inc load stuff.
>
> So I'm thinking this patch would actually break stuff.
>
>   
>> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
>> CC: Peter Zijlstra <peterz@infradead.org>
>> CC: Ingo Molnar <mingo@elte.hu>
>> ---
>>
>>  kernel/sched.c |    6 ++----
>>  1 files changed, 2 insertions(+), 4 deletions(-)
>>
>> diff --git a/kernel/sched.c b/kernel/sched.c
>> index 31f91d9..b046754 100644
>> --- a/kernel/sched.c
>> +++ b/kernel/sched.c
>> @@ -4679,10 +4679,8 @@ void set_user_nice(struct task_struct *p, long nice)
>>  		goto out_unlock;
>>  	}
>>  	on_rq = p->se.on_rq;
>> -	if (on_rq) {
>> +	if (on_rq)
>>  		dequeue_task(rq, p, 0);
>> -		dec_load(rq, p);
>> -	}
>>  
>>  	p->static_prio = NICE_TO_PRIO(nice);
>>  	set_load_weight(p);
>> @@ -4692,7 +4690,7 @@ void set_user_nice(struct task_struct *p, long nice)
>>  
>>  	if (on_rq) {
>>  		enqueue_task(rq, p, 0);
>> -		inc_load(rq, p);
>> +
>>  		/*
>>  		 * If the task increased its priority or is running and
>>  		 * lowered its priority, then reschedule its CPU:
>>
>>     
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

[PATCH 2/2] sched: readjust the load whenever task_setprio() is invoked

[Gregory Haskins]

The load may change with the priority, so be sure to recompute its value.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Ingo Molnar <mingo@elte.hu>
---

 kernel/sched.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index b046754..c3f41b9 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -4637,6 +4637,7 @@ void task_setprio(struct task_struct *p, int prio)
 		p->sched_class = &fair_sched_class;
 
 	p->prio = prio;
+	set_load_weight(p);
 
 //	trace_special_pid(p->pid, __PRIO(oldprio), PRIO(p));
 	prev_resched = _need_resched();

Re: [PATCH 2/2] sched: readjust the load whenever task_setprio() is invoked

[Peter Zijlstra]

On Thu, 2008-07-03 at 15:37 -0600, Gregory Haskins wrote:
> The load may change with the priority, so be sure to recompute its value.
> 
> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
> CC: Peter Zijlstra <peterz@infradead.org>
> CC: Ingo Molnar <mingo@elte.hu>

Right, but in this case we'd need to do the dec/inc load game again
because otherwise we'll skew stuff - see the previuos mail on how
dequeue_task() doesn't actually do that.

Also, it looks like current mainline still has this issue.

OTOH - since prio boosting is a temporal feature, not changing the load
isn't too bad, we ought to get back to where we came from pretty
quickly.



> ---
> 
>  kernel/sched.c |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/sched.c b/kernel/sched.c
> index b046754..c3f41b9 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -4637,6 +4637,7 @@ void task_setprio(struct task_struct *p, int prio)
>  		p->sched_class = &fair_sched_class;
>  
>  	p->prio = prio;
> +	set_load_weight(p);
>  
>  //	trace_special_pid(p->pid, __PRIO(oldprio), PRIO(p));
>  	prev_resched = _need_resched();
> 

Re: [PATCH 2/2] sched: readjust the load whenever task_setprio()is invoked

[Gregory Haskins]

>>> On Fri, Jul 18, 2008 at  8:43 AM, in message <1216384984.28405.36.camel@twins>,
Peter Zijlstra <peterz@infradead.org> wrote: 
> On Thu, 2008-07-03 at 15:37 -0600, Gregory Haskins wrote:
>> The load may change with the priority, so be sure to recompute its value.
>> 
>> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
>> CC: Peter Zijlstra <peterz@infradead.org>
>> CC: Ingo Molnar <mingo@elte.hu>
> 
> Right, but in this case we'd need to do the dec/inc load game again
> because otherwise we'll skew stuff - see the previuos mail on how
> dequeue_task() doesn't actually do that.
> 
> Also, it looks like current mainline still has this issue.
> 
> OTOH - since prio boosting is a temporal feature, not changing the load
> isn't too bad, we ought to get back to where we came from pretty
> quickly.

Yeah, I agree.  This issue probably didn't actually matter much in practice.

It just "looked" wrong, so I figured I'd fix it ;)

-Greg

> 
> 
> 
>> ---
>> 
>>  kernel/sched.c |    1 +
>>  1 files changed, 1 insertions(+), 0 deletions(-)
>> 
>> diff --git a/kernel/sched.c b/kernel/sched.c
>> index b046754..c3f41b9 100644
>> --- a/kernel/sched.c
>> +++ b/kernel/sched.c
>> @@ -4637,6 +4637,7 @@ void task_setprio(struct task_struct *p, int prio)
>>  		p->sched_class = &fair_sched_class;
>>  
>>  	p->prio = prio;
>> +	set_load_weight(p);
>>  
>>  //	trace_special_pid(p->pid, __PRIO(oldprio), PRIO(p));
>>  	prev_resched = _need_resched();
>>