INGO Why you remove set_user_nice() from kernel/kthread.c

INGO Why you remove set_user_nice() from kernel/kthread.c

[Pavel Vasilyev]

Next patсh -  
http://www.kernel.org/diff/diffview.cgi?file=%2Fpub%2Flinux%2Fkernel%2F%2Fv2.6%2Fsnapshots%2Fpatch-2.6.31-git2.bz2;z=548

This patch defines the core processes that are working with nice leve equal to 
zero , as in the BFS. :)

Why?

VirtualBox, Vmware, QEMU, Firefox, Azureus, and many subsystems and 
applications began working with large timeouts. In appearance similar to 
hang.

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Mike Galbraith]

If you're asking Ingo a question, maybe a CC is in order.

On Mon, 2009-09-14 at 17:42 +0400, Pavel Vasilyev wrote:
> 
> Next patсh -  
> http://www.kernel.org/diff/diffview.cgi?file=%2Fpub%2Flinux%2Fkernel%2F%2Fv2.6%2Fsnapshots%2Fpatch-2.6.31-git2.bz2;z=548
> 
> This patch defines the core processes that are working with nice leve equal to 
> zero , as in the BFS. :)
> 
> Why?

I did that, not Ingo, and did so because with kthreads that use
diddly-spit CPU (every one I see), it's just a waste of math.  What
kthreads are you seeing using so much CPU that their weight is a factor?
They _should_ be able to preempt and get their work done just fine
without a boost.

> VirtualBox, Vmware, QEMU, Firefox, Azureus, and many subsystems and 
> applications began working with large timeouts. In appearance similar to 
> hang.

In any case, if it's causing problems, reverting is easy enough, though
better would be to find out the why.

	-Mike

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Chris Friesen]

On 09/14/2009 08:05 AM, Mike Galbraith wrote:
> If you're asking Ingo a question, maybe a CC is in order.
> 
> On Mon, 2009-09-14 at 17:42 +0400, Pavel Vasilyev wrote:
>>
>> Next patсh -  
>> http://www.kernel.org/diff/diffview.cgi?file=%2Fpub%2Flinux%2Fkernel%2F%2Fv2.6%2Fsnapshots%2Fpatch-2.6.31-git2.bz2;z=548
>>
>> This patch defines the core processes that are working with nice leve equal to 
>> zero , as in the BFS. :)
>>
>> Why?
> 
> I did that, not Ingo, and did so because with kthreads that use
> diddly-spit CPU (every one I see), it's just a waste of math.  What
> kthreads are you seeing using so much CPU that their weight is a factor?
> They _should_ be able to preempt and get their work done just fine
> without a boost.

Under heavy network load ksoftirqd can use significant amounts of cpu.

Chris

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Pavel Vasilyev]

   
   Compare 

2.6.31-git2 with KTHREAD_NICE_LEVEL = 0
2.6.31-git2 with KTHREAD_NICE_LEVEL = -5

Diff.

diff --git a/kernel/kthread.c b/kernel/kthread.c
index 5fe7099..eb8751a 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -16,6 +16,8 @@
 #include <linux/mutex.h>
 #include <trace/events/sched.h>

+#define KTHREAD_NICE_LEVEL (-5)
+
 static DEFINE_SPINLOCK(kthread_create_lock);
 static LIST_HEAD(kthread_create_list);
 struct task_struct *kthreadd_task;
@@ -143,6 +145,7 @@ struct task_struct *kthread_create(int (*threadfn)(void 
*data),
                 * The kernel thread should not inherit these properties.
                 */
                sched_setscheduler_nocheck(create.result, SCHED_NORMAL, 
&param);
+         set_user_nice(create.result, KTHREAD_NICE_LEVEL);
                set_cpus_allowed_ptr(create.result, cpu_all_mask);
        }
        return create.result;
@@ -218,6 +221,7 @@ int kthreadd(void *unused)
        /* Setup a clean context for our children to inherit. */
        set_task_comm(tsk, "kthreadd");
        ignore_signals(tsk);
+ set_user_nice(tsk, KTHREAD_NICE_LEVEL);
        set_cpus_allowed_ptr(tsk, cpu_all_mask);
        set_mems_allowed(node_possible_map);
 


Used benchmarks.

# cyclictest and signaltest - 
http://www.osadl.org/Realtime-test-utilities-cyclictest-and-s.rt-test-cyclictest-signaltest.0.html

 ----------
 CYCLE TEST
 ----------

-T: 0 ( 5263) P: 0 I:1000 C:  98345 Min:      8 Act:656014 Avg:287390 Max:  
656450
-T: 1 ( 5264) P: 0 I:1500 C:  65680 Min:      7 Act:481583 Avg:236140 Max:  
482343
-T: 2 ( 5265) P: 0 I:2000 C:  49358 Min:      7 Act:286071 Avg:111300 Max:  
287453
-T: 3 ( 5266) P: 0 I:2500 C:  39453 Min:      7 Act:370028 Avg:116111 Max:  
372481
+T: 0 ( 6634) P: 0 I:1000 C:  98888 Min:      7 Act:113011 Avg:28733 Max:  
113817
+T: 1 ( 6635) P: 0 I:1500 C:  65953 Min:      8 Act:72013 Avg:25026 Max:   
73110
+T: 2 ( 6636) P: 0 I:2000 C:  49468 Min:      6 Act:66076 Avg:17455 Max:   
67486
+T: 3 ( 6637) P: 0 I:2500 C:  39580 Min:      7 Act:52514 Avg:12882 Max:   
53256

 ----------
 SIGNAL TEST
 ----------

-T: 0 ( 5285) P: 0 C: 100000 Min:     13 Act:   23 Avg:   30 Max:    9229
-T: 1 ( 5286) P: 0 C: 100000 Min:     13 Act:   99 Avg:  662 Max:   17282
-T: 2 ( 5287) P: 0 C: 100000 Min:     13 Act:  110 Avg:  662 Max:   17294
-T: 3 ( 5288) P: 0 C: 100000 Min:     13 Act:  119 Avg:  662 Max:   18645
+T: 0 ( 6698) P: 0 C: 100000 Min:     13 Act:   22 Avg:   24 Max:    7898
+T: 1 ( 6699) P: 0 C: 100000 Min:     13 Act:  104 Avg:  654 Max:   15728
+T: 2 ( 6700) P: 0 C: 100000 Min:     13 Act:  114 Avg:  654 Max:   15740
+T: 3 ( 6701) P: 0 C: 100000 Min:     13 Act:  124 Avg:  654 Max:   16102

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Thomas Gleixner]

Pavel,

On Mon, 14 Sep 2009, Pavel Vasilyev wrote:
>  ----------
>  CYCLE TEST
>  ----------
> 
> -T: 0 ( 5263) P: 0 I:1000 C:  98345 Min:      8 Act:656014 Avg:287390 Max:  
> 656450
> -T: 1 ( 5264) P: 0 I:1500 C:  65680 Min:      7 Act:481583 Avg:236140 Max:  
> 482343
> -T: 2 ( 5265) P: 0 I:2000 C:  49358 Min:      7 Act:286071 Avg:111300 Max:  
> 287453
> -T: 3 ( 5266) P: 0 I:2500 C:  39453 Min:      7 Act:370028 Avg:116111 Max:  
> 372481
> +T: 0 ( 6634) P: 0 I:1000 C:  98888 Min:      7 Act:113011 Avg:28733 Max:  
> 113817
> +T: 1 ( 6635) P: 0 I:1500 C:  65953 Min:      8 Act:72013 Avg:25026 Max:   
> 73110
> +T: 2 ( 6636) P: 0 I:2000 C:  49468 Min:      6 Act:66076 Avg:17455 Max:   
> 67486
> +T: 3 ( 6637) P: 0 I:2500 C:  39580 Min:      7 Act:52514 Avg:12882 Max:   
> 53256
> 
>  ----------
>  SIGNAL TEST
>  ----------
> 
> -T: 0 ( 5285) P: 0 C: 100000 Min:     13 Act:   23 Avg:   30 Max:    9229
> -T: 1 ( 5286) P: 0 C: 100000 Min:     13 Act:   99 Avg:  662 Max:   17282
> -T: 2 ( 5287) P: 0 C: 100000 Min:     13 Act:  110 Avg:  662 Max:   17294
> -T: 3 ( 5288) P: 0 C: 100000 Min:     13 Act:  119 Avg:  662 Max:   18645
> +T: 0 ( 6698) P: 0 C: 100000 Min:     13 Act:   22 Avg:   24 Max:    7898
> +T: 1 ( 6699) P: 0 C: 100000 Min:     13 Act:  104 Avg:  654 Max:   15728
> +T: 2 ( 6700) P: 0 C: 100000 Min:     13 Act:  114 Avg:  654 Max:   15740
> +T: 3 ( 6701) P: 0 C: 100000 Min:     13 Act:  124 Avg:  654 Max:   16102

These comparisons are not telling anything without the used command
line options and a description of the load on the machine while
running the tests.

Thanks,

	tglx

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Mike Galbraith]

On Mon, 2009-09-14 at 09:12 -0600, Chris Friesen wrote:
> On 09/14/2009 08:05 AM, Mike Galbraith wrote:
> > If you're asking Ingo a question, maybe a CC is in order.
> > 
> > On Mon, 2009-09-14 at 17:42 +0400, Pavel Vasilyev wrote:
> >>
> >> Next patсh -  
> >> http://www.kernel.org/diff/diffview.cgi?file=%2Fpub%2Flinux%2Fkernel%2F%2Fv2.6%2Fsnapshots%2Fpatch-2.6.31-git2.bz2;z=548
> >>
> >> This patch defines the core processes that are working with nice leve equal to 
> >> zero , as in the BFS. :)
> >>
> >> Why?
> > 
> > I did that, not Ingo, and did so because with kthreads that use
> > diddly-spit CPU (every one I see), it's just a waste of math.  What
> > kthreads are you seeing using so much CPU that their weight is a factor?
> > They _should_ be able to preempt and get their work done just fine
> > without a boost.
> 
> Under heavy network load ksoftirqd can use significant amounts of cpu.

OK, that may justify a weight adjustment, since it is a proxy for many.
Question is, does it really need it?

I certainly have no objection to a heavier weight for any kthread,
though I think it's a waste of cycles for the general case.

	-Mike

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Chris Friesen]

On 09/14/2009 09:45 AM, Mike Galbraith wrote:
> On Mon, 2009-09-14 at 09:12 -0600, Chris Friesen wrote:
>> On 09/14/2009 08:05 AM, Mike Galbraith wrote:

>>> I did that, not Ingo, and did so because with kthreads that use
>>> diddly-spit CPU (every one I see), it's just a waste of math.  What
>>> kthreads are you seeing using so much CPU that their weight is a factor?
>>> They _should_ be able to preempt and get their work done just fine
>>> without a boost.
>>
>> Under heavy network load ksoftirqd can use significant amounts of cpu.
> 
> OK, that may justify a weight adjustment, since it is a proxy for many.
> Question is, does it really need it?

I guess it depends how we want it to behave by default.  Likely anyone
that really cares is going to fine-tune the ksoftirqd priority level
anyways.

I've added netdev to the CC list.  Maybe some of the people there have
an opinion on what the default priority should be for ksoftirqd.

Chris

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Mike Galbraith]

On Mon, 2009-09-14 at 10:31 -0600, Chris Friesen wrote:

> I've added netdev to the CC list.  Maybe some of the people there have
> an opinion on what the default priority should be for ksoftirqd.

Good idea.

	-Mike

Re: INGO Why you remove set_user_nice() from kernel/kthread.c

[Pavel Vasilyev]

On Friday 18 September 2009 13:13:31 you wrote:
> Pavel,
>
> On Mon, 14 Sep 2009, Pavel Vasilyev wrote:
> >  ----------
> >  CYCLE TEST
> >  ----------
> >
> > -T: 0 ( 5263) P: 0 I:1000 C:  98345 Min:      8 Act:656014 Avg:287390
> > Max: 656450
> > -T: 1 ( 5264) P: 0 I:1500 C:  65680 Min:      7 Act:481583 Avg:236140
> > Max: 482343
> > -T: 2 ( 5265) P: 0 I:2000 C:  49358 Min:      7 Act:286071 Avg:111300
> > Max: 287453
> > -T: 3 ( 5266) P: 0 I:2500 C:  39453 Min:      7 Act:370028 Avg:116111
> > Max: 372481
> > +T: 0 ( 6634) P: 0 I:1000 C:  98888 Min:      7 Act:113011 Avg:28733 Max:
> > 113817
> > +T: 1 ( 6635) P: 0 I:1500 C:  65953 Min:      8 Act:72013 Avg:25026 Max:
> > 73110
> > +T: 2 ( 6636) P: 0 I:2000 C:  49468 Min:      6 Act:66076 Avg:17455 Max:
> > 67486
> > +T: 3 ( 6637) P: 0 I:2500 C:  39580 Min:      7 Act:52514 Avg:12882 Max:
> > 53256
> >
> >  ----------
> >  SIGNAL TEST
> >  ----------
> >
> > -T: 0 ( 5285) P: 0 C: 100000 Min:     13 Act:   23 Avg:   30 Max:    9229
> > -T: 1 ( 5286) P: 0 C: 100000 Min:     13 Act:   99 Avg:  662 Max:   17282
> > -T: 2 ( 5287) P: 0 C: 100000 Min:     13 Act:  110 Avg:  662 Max:   17294
> > -T: 3 ( 5288) P: 0 C: 100000 Min:     13 Act:  119 Avg:  662 Max:   18645
> > +T: 0 ( 6698) P: 0 C: 100000 Min:     13 Act:   22 Avg:   24 Max:    7898
> > +T: 1 ( 6699) P: 0 C: 100000 Min:     13 Act:  104 Avg:  654 Max:   15728
> > +T: 2 ( 6700) P: 0 C: 100000 Min:     13 Act:  114 Avg:  654 Max:   15740
> > +T: 3 ( 6701) P: 0 C: 100000 Min:     13 Act:  124 Avg:  654 Max:   16102
>
> These comparisons are not telling anything without the used command
> line options and a description of the load on the machine while
> running the tests.
>

# ./cyclictest -t -b -o -D 100s -w -q
# ./signaltest -t4 -l 100000 -q

# cat /proc/cpuinfo  | grep "model name"
model name      : Dual Core AMD Opteron(tm) Processor 265
model name      : Dual Core AMD Opteron(tm) Processor 265
model name      : Dual Core AMD Opteron(tm) Processor 265
model name      : Dual Core AMD Opteron(tm) Processor 265

# Linux amd64 2.6.31 #15 SMP PREEMPT Thu Sep 17 00:04:02 MSD 2009 x86_64 
x86_64 x86_64 GNU/Linux