[PATCH 1/4] sched/rt: Sum number of all children tasks in hierarhy at rt_nr_running

[PATCH 1/4] sched/rt: Sum number of all children tasks in hierarhy at rt_nr_running

[Kirill Tkhai]

{inc,dec}_rt_tasks used to count entities which are directly queued
on rt_rq. If an entity was not a task (i.e., it is some queue), its
children were not counted.

There is no problem here, but now we want to count number of all tasks
which are actually queued under the rt_rq in all the hierarhy (except
throttled rt queues).

Empty queues are not able to be queued and all of the places, which
use rt_nr_running, just compare it with zero, so we do not break
anything here.

Signed-off-by: Kirill Tkhai <tkhai@yandex.ru>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Ingo Molnar <mingo@kernel.org>
---
 kernel/sched/rt.c |   15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index d8cdf16..e4def13 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -1045,12 +1045,23 @@ void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {}
 #endif /* CONFIG_RT_GROUP_SCHED */
 
 static inline
+unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se)
+{
+	struct rt_rq *group_rq = group_rt_rq(rt_se);
+
+	if (group_rq)
+		return group_rq->rt_nr_running;
+	else
+		return 1;
+}
+
+static inline
 void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
 {
 	int prio = rt_se_prio(rt_se);
 
 	WARN_ON(!rt_prio(prio));
-	rt_rq->rt_nr_running++;
+	rt_rq->rt_nr_running += rt_se_nr_running(rt_se);
 
 	inc_rt_prio(rt_rq, prio);
 	inc_rt_migration(rt_se, rt_rq);
@@ -1062,7 +1073,7 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
 {
 	WARN_ON(!rt_prio(rt_se_prio(rt_se)));
 	WARN_ON(!rt_rq->rt_nr_running);
-	rt_rq->rt_nr_running--;
+	rt_rq->rt_nr_running -= rt_se_nr_running(rt_se);
 
 	dec_rt_prio(rt_rq, rt_se_prio(rt_se));
 	dec_rt_migration(rt_se, rt_rq);

Re: [PATCH 1/4] sched/rt: Sum number of all children tasks in hierarhy at rt_nr_running

[Preeti Murthy]

On Sat, Mar 15, 2014 at 3:44 AM, Kirill Tkhai <tkhai@yandex.ru> wrote:
> {inc,dec}_rt_tasks used to count entities which are directly queued
> on rt_rq. If an entity was not a task (i.e., it is some queue), its
> children were not counted.

Its always the case that a task is queued right, never a sched entity?
When a task is queued, the nr_running of every rt_rq in the hierarchy
of sched entities which are parents of this task is incremented by 1.
Similar is with dequeue of a sched_entity. If the sched_entity has
just 1 task on it, then its dequeued from its parent queue and its number
decremented for every rq in the hierarchy. But you would
never dequeue a sched_entity if it has more than 1 task in it. The
granularity of enqueue and dequeue of sched_entities is one task
at a time. You can extend this to enqueue and dequeue of a sched_entity
only if it has just one task in its queue.

Regards
Preeti U Murthy


>
> There is no problem here, but now we want to count number of all tasks
> which are actually queued under the rt_rq in all the hierarhy (except
> throttled rt queues).
>
> Empty queues are not able to be queued and all of the places, which
> use rt_nr_running, just compare it with zero, so we do not break
> anything here.
>
> Signed-off-by: Kirill Tkhai <tkhai@yandex.ru>
> CC: Peter Zijlstra <peterz@infradead.org>
> CC: Ingo Molnar <mingo@kernel.org>
> ---
>  kernel/sched/rt.c |   15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
> index d8cdf16..e4def13 100644
> --- a/kernel/sched/rt.c
> +++ b/kernel/sched/rt.c
> @@ -1045,12 +1045,23 @@ void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {}
>  #endif /* CONFIG_RT_GROUP_SCHED */
>
>  static inline
> +unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se)
> +{
> +       struct rt_rq *group_rq = group_rt_rq(rt_se);
> +
> +       if (group_rq)
> +               return group_rq->rt_nr_running;
> +       else
> +               return 1;
> +}
> +
> +static inline
>  void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
>  {
>         int prio = rt_se_prio(rt_se);
>
>         WARN_ON(!rt_prio(prio));
> -       rt_rq->rt_nr_running++;
> +       rt_rq->rt_nr_running += rt_se_nr_running(rt_se);
>
>         inc_rt_prio(rt_rq, prio);
>         inc_rt_migration(rt_se, rt_rq);
> @@ -1062,7 +1073,7 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
>  {
>         WARN_ON(!rt_prio(rt_se_prio(rt_se)));
>         WARN_ON(!rt_rq->rt_nr_running);
> -       rt_rq->rt_nr_running--;
> +       rt_rq->rt_nr_running -= rt_se_nr_running(rt_se);
>
>         dec_rt_prio(rt_rq, rt_se_prio(rt_se));
>         dec_rt_migration(rt_se, rt_rq);
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 1/4] sched/rt: Sum number of all children tasks in hierarhy at rt_nr_running

[Kirill Tkhai]

18.03.2014, 15:08, "Preeti Murthy" <preeti.lkml@gmail.com>:
> On Sat, Mar 15, 2014 at 3:44 AM, Kirill Tkhai <tkhai@yandex.ru> wrote:
>
>>  {inc,dec}_rt_tasks used to count entities which are directly queued
>>  on rt_rq. If an entity was not a task (i.e., it is some queue), its
>>  children were not counted.
>
> Its always the case that a task is queued right, never a sched entity?

With patch applied, when sched entity is group queue, we add number of
its child tasks instead of "1".

> When a task is queued, the nr_running of every rt_rq in the hierarchy
> of sched entities which are parents of this task is incremented by 1.

Only if they had not had a queued task before.

> Similar is with dequeue of a sched_entity. If the sched_entity has
> just 1 task on it, then its dequeued from its parent queue and its number
> decremented for every rq in the hierarchy. But you would
> never dequeue a sched_entity if it has more than 1 task in it. The
> granularity of enqueue and dequeue of sched_entities is one task
> at a time. You can extend this to enqueue and dequeue of a sched_entity
> only if it has just one task in its queue.

We do not queue entites, which are empty. In __enqueue_rt_entity():

if (group_rq && (rt_rq_throttled(group_rq) || !group_rq->rt_nr_running))
         return;

Where do you see a collision? Please explain, if so.

> Regards
> Preeti U Murthy
>
>>  There is no problem here, but now we want to count number of all tasks
>>  which are actually queued under the rt_rq in all the hierarhy (except
>>  throttled rt queues).
>>
>>  Empty queues are not able to be queued and all of the places, which
>>  use rt_nr_running, just compare it with zero, so we do not break
>>  anything here.
>>
>>  Signed-off-by: Kirill Tkhai <tkhai@yandex.ru>
>>  CC: Peter Zijlstra <peterz@infradead.org>
>>  CC: Ingo Molnar <mingo@kernel.org>
>>  ---
>>   kernel/sched/rt.c |   15 +++++++++++++--
>>   1 file changed, 13 insertions(+), 2 deletions(-)
>>
>>  diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
>>  index d8cdf16..e4def13 100644
>>  --- a/kernel/sched/rt.c
>>  +++ b/kernel/sched/rt.c
>>  @@ -1045,12 +1045,23 @@ void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {}
>>   #endif /* CONFIG_RT_GROUP_SCHED */
>>
>>   static inline
>>  +unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se)
>>  +{
>>  +       struct rt_rq *group_rq = group_rt_rq(rt_se);
>>  +
>>  +       if (group_rq)
>>  +               return group_rq->rt_nr_running;
>>  +       else
>>  +               return 1;
>>  +}
>>  +
>>  +static inline
>>   void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
>>   {
>>          int prio = rt_se_prio(rt_se);
>>
>>          WARN_ON(!rt_prio(prio));
>>  -       rt_rq->rt_nr_running++;
>>  +       rt_rq->rt_nr_running += rt_se_nr_running(rt_se);
>>
>>          inc_rt_prio(rt_rq, prio);
>>          inc_rt_migration(rt_se, rt_rq);
>>  @@ -1062,7 +1073,7 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
>>   {
>>          WARN_ON(!rt_prio(rt_se_prio(rt_se)));
>>          WARN_ON(!rt_rq->rt_nr_running);
>>  -       rt_rq->rt_nr_running--;
>>  +       rt_rq->rt_nr_running -= rt_se_nr_running(rt_se);
>>
>>          dec_rt_prio(rt_rq, rt_se_prio(rt_se));
>>          dec_rt_migration(rt_se, rt_rq);
>>
>>  --
>>  To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>  the body of a message to majordomo@vger.kernel.org
>>  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>  Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 1/4] sched/rt: Sum number of all children tasks in hierarhy at rt_nr_running

[Preeti U Murthy]

On 03/18/2014 05:14 PM, Kirill Tkhai wrote:
> 
> 
> 18.03.2014, 15:08, "Preeti Murthy" <preeti.lkml@gmail.com>:
>> On Sat, Mar 15, 2014 at 3:44 AM, Kirill Tkhai <tkhai@yandex.ru> wrote:
>>
>>>  {inc,dec}_rt_tasks used to count entities which are directly queued
>>>  on rt_rq. If an entity was not a task (i.e., it is some queue), its
>>>  children were not counted.
>>
>> Its always the case that a task is queued right, never a sched entity?
> 
> With patch applied, when sched entity is group queue, we add number of
> its child tasks instead of "1".
> 
>> When a task is queued, the nr_running of every rt_rq in the hierarchy
>> of sched entities which are parents of this task is incremented by 1.
> 
> Only if they had not had a queued task before.
> 
>> Similar is with dequeue of a sched_entity. If the sched_entity has
>> just 1 task on it, then its dequeued from its parent queue and its number
>> decremented for every rq in the hierarchy. But you would
>> never dequeue a sched_entity if it has more than 1 task in it. The
>> granularity of enqueue and dequeue of sched_entities is one task
>> at a time. You can extend this to enqueue and dequeue of a sched_entity
>> only if it has just one task in its queue.
> 
> We do not queue entites, which are empty. In __enqueue_rt_entity():
> 
> if (group_rq && (rt_rq_throttled(group_rq) || !group_rq->rt_nr_running))
>          return;
> 
> Where do you see a collision? Please explain, if so.

Ok I went through the patchset more closely and I understand it better
now. So this patchset targets the throttled rt runqueues specifically. I
am sorry I missed this focal point. So this patch looks good to me.

Reviewed-by: Preeti U Murthy <preeti@linux.vnet.ibm.com>
> 
>> Regards
>> Preeti U Murthy
>>
>>>  There is no problem here, but now we want to count number of all tasks
>>>  which are actually queued under the rt_rq in all the hierarhy (except
>>>  throttled rt queues).
>>>
>>>  Empty queues are not able to be queued and all of the places, which
>>>  use rt_nr_running, just compare it with zero, so we do not break
>>>  anything here.
>>>
>>>  Signed-off-by: Kirill Tkhai <tkhai@yandex.ru>
>>>  CC: Peter Zijlstra <peterz@infradead.org>
>>>  CC: Ingo Molnar <mingo@kernel.org>
>>>  ---
>>>   kernel/sched/rt.c |   15 +++++++++++++--
>>>   1 file changed, 13 insertions(+), 2 deletions(-)
>>>
>>>  diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
>>>  index d8cdf16..e4def13 100644
>>>  --- a/kernel/sched/rt.c
>>>  +++ b/kernel/sched/rt.c
>>>  @@ -1045,12 +1045,23 @@ void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {}
>>>   #endif /* CONFIG_RT_GROUP_SCHED */
>>>
>>>   static inline
>>>  +unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se)
>>>  +{
>>>  +       struct rt_rq *group_rq = group_rt_rq(rt_se);
>>>  +
>>>  +       if (group_rq)
>>>  +               return group_rq->rt_nr_running;
>>>  +       else
>>>  +               return 1;
>>>  +}
>>>  +
>>>  +static inline
>>>   void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
>>>   {
>>>          int prio = rt_se_prio(rt_se);
>>>
>>>          WARN_ON(!rt_prio(prio));
>>>  -       rt_rq->rt_nr_running++;
>>>  +       rt_rq->rt_nr_running += rt_se_nr_running(rt_se);
>>>
>>>          inc_rt_prio(rt_rq, prio);
>>>          inc_rt_migration(rt_se, rt_rq);
>>>  @@ -1062,7 +1073,7 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
>>>   {
>>>          WARN_ON(!rt_prio(rt_se_prio(rt_se)));
>>>          WARN_ON(!rt_rq->rt_nr_running);
>>>  -       rt_rq->rt_nr_running--;
>>>  +       rt_rq->rt_nr_running -= rt_se_nr_running(rt_se);
>>>
>>>          dec_rt_prio(rt_rq, rt_se_prio(rt_se));
>>>          dec_rt_migration(rt_se, rt_rq);
>>>
>>>  --
>>>  To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>  the body of a message to majordomo@vger.kernel.org
>>>  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>  Please read the FAQ at  http://www.tux.org/lkml/
> 

[tip:sched/core] sched/rt: Sum number of all children tasks in hierarhy at ->rt_nr_running

[tip-bot for Kirill Tkhai]

Commit-ID:  22abdef37cebcdd4933c72339401a174b7d87768
Gitweb:     http://git.kernel.org/tip/22abdef37cebcdd4933c72339401a174b7d87768
Author:     Kirill Tkhai <tkhai@yandex.ru>
AuthorDate: Sat, 15 Mar 2014 02:14:49 +0400
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 18 Apr 2014 12:07:25 +0200

sched/rt: Sum number of all children tasks in hierarhy at ->rt_nr_running

{inc,dec}_rt_tasks() used to count entities which are directly queued
on the rt_rq. If an entity was not a task (i.e., it is some queue), its
children were not counted.

There is no problem here, but now we want to count number of all tasks
which are actually queued under the rt_rq in all the hierarchy (except
throttled rt queues).

Empty queues are not able to be queued and all of the places, which
use ->rt_nr_running, just compare it with zero, so we do not break
anything here.

Signed-off-by: Kirill Tkhai <tkhai@yandex.ru>
Reviewed-by: Preeti U Murthy <preeti@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1394835289.18748.31.camel@HP-250-G1-Notebook-PC
Cc: linux-kernel@vger.kernel.org
[ Twiddled the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/sched/rt.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 1e4992e..6892ed7 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -1045,12 +1045,23 @@ void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {}
 #endif /* CONFIG_RT_GROUP_SCHED */
 
 static inline
+unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se)
+{
+	struct rt_rq *group_rq = group_rt_rq(rt_se);
+
+	if (group_rq)
+		return group_rq->rt_nr_running;
+	else
+		return 1;
+}
+
+static inline
 void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
 {
 	int prio = rt_se_prio(rt_se);
 
 	WARN_ON(!rt_prio(prio));
-	rt_rq->rt_nr_running++;
+	rt_rq->rt_nr_running += rt_se_nr_running(rt_se);
 
 	inc_rt_prio(rt_rq, prio);
 	inc_rt_migration(rt_se, rt_rq);
@@ -1062,7 +1073,7 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
 {
 	WARN_ON(!rt_prio(rt_se_prio(rt_se)));
 	WARN_ON(!rt_rq->rt_nr_running);
-	rt_rq->rt_nr_running--;
+	rt_rq->rt_nr_running -= rt_se_nr_running(rt_se);
 
 	dec_rt_prio(rt_rq, rt_se_prio(rt_se));
 	dec_rt_migration(rt_se, rt_rq);