From: Jens Axboe <jens.axboe@oracle.com>
To: Linux Kernel <linux-kernel@vger.kernel.org>
Cc: mingo@elte.hu, a.p.zijlstra@chello.nl
Subject: find_busiest_group using lots of CPU
Date: Wed, 30 Sep 2009 10:18:11 +0200 [thread overview]
Message-ID: <20090930081811.GP23126@kernel.dk> (raw)
Hi,
I stuffed a few more SSDs into my text box. Running a simple workload
that just does streaming reads from 10 processes (throughput is around
2.2GB/sec), find_busiest_group() is using > 10% of the CPU time. This is
a 64 thread box.
The top two profile entries are:
10.86% fio [kernel] [k] find_busiest_group
|
|--99.91%-- thread_return
| io_schedule
| sys_io_getevents
| system_call_fastpath
| 0x7f4b50b61604
| |
| --100.00%-- td_io_getevents
| io_u_queued_complete
| thread_main
| run_threads
| main
| __libc_start_main
--0.09%-- [...]
5.78% fio [kernel] [k] cpumask_next_and
|
|--67.21%-- thread_return
| io_schedule
| sys_io_getevents
| system_call_fastpath
| 0x7f4b50b61604
| |
| --100.00%-- td_io_getevents
| io_u_queued_complete
| thread_main
| run_threads
| main
| __libc_start_main
|
--32.79%-- find_busiest_group
thread_return
io_schedule
sys_io_getevents
system_call_fastpath
0x7f4b50b61604
|
--100.00%-- td_io_getevents
io_u_queued_complete
thread_main
run_threads
main
__libc_start_main
This is with SCHED_DEBUG=y and SCHEDSTATS=y enabled, I just tried with
both disabled but that yields the same result (well actually worse, 22%
spent in there. dunno if that's normal "fluctuation"). GROUP_SCHED is
not set. This seems way excessive!
--
Jens Axboe
next reply other threads:[~2009-09-30 8:18 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-09-30 8:18 Jens Axboe [this message]
2009-10-05 12:31 ` find_busiest_group using lots of CPU Peter Zijlstra
2009-10-06 7:51 ` Jens Axboe
2009-10-06 11:20 ` Jens Axboe
2009-10-06 11:47 ` Ingo Molnar
2009-10-06 11:56 ` Jens Axboe
2009-10-06 12:04 ` Peter Zijlstra
2009-10-06 12:14 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090930081811.GP23126@kernel.dk \
--to=jens.axboe@oracle.com \
--cc=a.p.zijlstra@chello.nl \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox