From: Libo Chen <libo.chen@huawei.com>
To: <tglx@linutronix.de>, <mingo@elte.hu>,
LKML <linux-kernel@vger.kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>, Li Zefan <lizefan@huawei.com>
Subject: balance storm
Date: Mon, 26 May 2014 11:04:14 +0800 [thread overview]
Message-ID: <5382AF2E.1040407@huawei.com> (raw)
hi,
my box has 16 cpu (E5-2658,8 core, 2 thread per core), i did a test on
3.4.24stable, startup 50 same process, every process is sample:
#include <unistd.h>
int main()
{
for (;;)
{
unsigned int i = 0;
while (i< 100){
i++;
}
usleep(100);
}
return 0;
}
the result is process uses 15% cpu time, perf tool shows 70w migrations in 5 second.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4374 root 20 0 6020 332 256 S 15 0.0 0:03.73 a2.out
4371 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4373 root 20 0 6020 332 256 R 15 0.0 0:03.72 a2.out
4377 root 20 0 6020 332 256 R 15 0.0 0:03.72 a2.out
4389 root 20 0 6020 328 256 S 15 0.0 0:03.71 a2.out
4391 root 20 0 6020 332 256 S 15 0.0 0:03.72 a2.out
4394 root 20 0 6020 332 256 S 15 0.0 0:03.70 a2.out
4398 root 20 0 6020 328 256 S 15 0.0 0:03.71 a2.out
4403 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4405 root 20 0 6020 328 256 S 15 0.0 0:03.72 a2.out
4407 root 20 0 6020 332 256 S 15 0.0 0:03.73 a2.out
4369 root 20 0 6020 332 256 S 15 0.0 0:03.72 a2.out
4370 root 20 0 6020 332 256 S 15 0.0 0:03.70 a2.out
4372 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4375 root 20 0 6020 332 256 S 15 0.0 0:03.70 a2.out
4378 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4379 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4380 root 20 0 6020 332 256 S 15 0.0 0:03.72 a2.out
4381 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4383 root 20 0 6020 332 256 S 15 0.0 0:03.69 a2.out
4384 root 20 0 6020 332 256 S 15 0.0 0:03.72 a2.out
4386 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4387 root 20 0 6020 328 256 S 15 0.0 0:03.70 a2.out
4388 root 20 0 6020 332 256 R 15 0.0 0:03.72 a2.out
4390 root 20 0 6020 332 256 S 15 0.0 0:03.70 a2.out
4392 root 20 0 6020 332 256 S 15 0.0 0:03.72 a2.out
4393 root 20 0 6020 332 256 S 15 0.0 0:03.72 a2.out
4395 root 20 0 6020 332 256 S 15 0.0 0:03.70 a2.out
4396 root 20 0 6020 328 256 S 15 0.0 0:03.71 a2.out
4397 root 20 0 6020 332 256 S 15 0.0 0:03.70 a2.out
4399 root 20 0 6020 332 256 R 15 0.0 0:03.72 a2.out
4400 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4402 root 20 0 6020 332 256 S 15 0.0 0:03.70 a2.out
4404 root 20 0 6020 332 256 R 15 0.0 0:03.69 a2.out
4406 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
4408 root 20 0 6020 328 256 R 15 0.0 0:03.71 a2.out
4409 root 20 0 6020 332 256 R 15 0.0 0:03.71 a2.out
4410 root 20 0 6020 328 256 S 15 0.0 0:03.72 a2.out
4411 root 20 0 6020 332 256 S 15 0.0 0:03.71 a2.out
===========================================================================
when i reverts commit 908a3283728d92df36e0c7cd63304fd35e93a8a9:
diff --git a/kernel/sched.c b/kernel/sched.c
index 1874c74..4cdc91c 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -5138,7 +5138,20 @@ EXPORT_SYMBOL(task_nice);
*/
int idle_cpu(int cpu)
{
- return cpu_curr(cpu) == cpu_rq(cpu)->idle;
+ struct rq *rq = cpu_rq(cpu);
+
+ if (rq->curr != rq->idle)
+ return 0;
+
+ if (rq->nr_running)
+ return 0;
+
+#ifdef CONFIG_SMP
+ if (!llist_empty(&rq->wake_list))
+ return 0;
+#endif
+
+ return 1;
}
the result is process uses 3-5% cpu time, perf tool shows only 1k migrations in 5 second.
4444 root 20 0 6020 328 256 S 5 0.0 2:18.49 a2.out
4469 root 20 0 6020 328 256 S 5 0.0 2:15.93 a2.out
4423 root 20 0 6020 328 256 S 5 0.0 2:14.36 a2.out
4433 root 20 0 6020 332 256 S 5 0.0 2:15.81 a2.out
4466 root 20 0 6020 328 256 S 4 0.0 2:17.62 a2.out
4428 root 20 0 6020 332 256 S 4 0.0 2:13.92 a2.out
4457 root 20 0 6020 332 256 R 4 0.0 2:15.30 a2.out
4429 root 20 0 6020 328 256 R 4 0.0 2:17.13 a2.out
4431 root 20 0 6020 332 256 S 3 0.0 2:15.91 a2.out
4438 root 20 0 6020 332 256 S 3 0.0 2:14.04 a2.out
4439 root 20 0 6020 332 256 S 3 0.0 2:15.94 a2.out
4462 root 20 0 6020 332 256 R 3 0.0 2:16.40 a2.out
4422 root 20 0 6020 328 256 S 3 0.0 2:17.41 a2.out
4434 root 20 0 6020 332 256 R 3 0.0 2:15.67 a2.out
4440 root 20 0 6020 332 256 S 3 0.0 2:14.40 a2.out
4447 root 20 0 6020 332 256 S 3 0.0 2:16.02 a2.out
4448 root 20 0 6020 332 256 S 3 0.0 2:16.40 a2.out
4453 root 20 0 6020 332 256 R 3 0.0 2:15.75 a2.out
4459 root 20 0 6020 328 256 S 3 0.0 2:16.66 a2.out
4461 root 20 0 6020 332 256 S 3 0.0 2:15.77 a2.out
4471 root 20 0 6020 328 256 S 3 0.0 2:20.68 a2.out
4424 root 20 0 6020 328 256 S 3 0.0 2:15.90 a2.out
4427 root 20 0 6020 332 256 S 3 0.0 2:14.28 a2.out
4432 root 20 0 6020 332 256 S 3 0.0 2:14.63 a2.out
4435 root 20 0 6020 328 256 S 3 0.0 2:15.32 a2.out
4436 root 20 0 6020 328 256 S 3 0.0 2:15.40 a2.out
4437 root 20 0 6020 332 256 S 3 0.0 2:15.42 a2.out
4441 root 20 0 6020 332 256 S 3 0.0 2:18.59 a2.out
4443 root 20 0 6020 332 256 S 3 0.0 2:14.82 a2.out
4445 root 20 0 6020 332 256 R 3 0.0 2:13.12 a2.out
4449 root 20 0 6020 332 256 R 3 0.0 2:21.37 a2.out
4450 root 20 0 6020 332 256 S 3 0.0 2:15.78 a2.out
4451 root 20 0 6020 332 256 S 3 0.0 2:16.25 a2.out
4455 root 20 0 6020 332 256 S 3 0.0 2:18.58 a2.out
4456 root 20 0 6020 332 256 S 3 0.0 2:16.37 a2.out
4458 root 20 0 6020 328 256 S 3 0.0 2:18.03 a2.out
4460 root 20 0 6020 332 256 S 3 0.0 2:14.04 a2.out
4463 root 20 0 6020 328 256 S 3 0.0 2:16.74 a2.out
4464 root 20 0 6020 328 256 S 3 0.0 2:18.11 a2.out
I guess task migration takes up a lot of cpu, so i did another test. use taskset tool to bind
a task to a fixed cpu. Results in line with expectations, cpu usage is down to 5%.
other test:
- 3.15upstream has the same problem with 3.4.24.
- suse sp2 has low cpu usage about 5%.
so I think 15% cpu usage and migration event are too high, how to fixed?
next reply other threads:[~2014-05-26 3:04 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-26 3:04 Libo Chen [this message]
2014-05-26 5:11 ` balance storm Mike Galbraith
2014-05-26 12:16 ` Libo Chen
2014-05-26 14:19 ` Mike Galbraith
2014-05-27 7:56 ` Libo Chen
2014-05-27 9:55 ` Mike Galbraith
2014-05-27 12:50 ` Libo Chen
2014-05-27 13:20 ` Mike Galbraith
2014-05-28 1:04 ` Libo Chen
2014-05-28 1:53 ` Mike Galbraith
2014-05-28 6:54 ` Libo Chen
2014-05-28 8:16 ` Mike Galbraith
2014-05-28 9:08 ` Thomas Gleixner
2014-05-28 10:30 ` Peter Zijlstra
2014-05-28 10:52 ` Borislav Petkov
2014-05-28 11:43 ` Libo Chen
2014-05-28 11:55 ` Mike Galbraith
2014-05-29 7:58 ` Libo Chen
2014-05-29 7:57 ` Libo Chen
2014-05-27 20:53 ` Thomas Gleixner
2014-05-28 1:06 ` Libo Chen
2014-05-26 7:56 ` Mike Galbraith
2014-05-26 11:49 ` Libo Chen
2014-05-26 14:03 ` Mike Galbraith
2014-05-27 7:44 ` Libo Chen
2014-05-27 8:12 ` Mike Galbraith
2014-05-27 9:48 ` Peter Zijlstra
2014-05-27 10:05 ` Mike Galbraith
2014-05-27 10:43 ` Peter Zijlstra
2014-05-27 10:55 ` Mike Galbraith
2014-05-27 12:56 ` Libo Chen
2014-05-27 12:55 ` Libo Chen
2014-05-27 13:13 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5382AF2E.1040407@huawei.com \
--to=libo.chen@huawei.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=mingo@elte.hu \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox