From: Tejun Heo <tj@kernel.org>
To: David Howells <dhowells@redhat.com>
Cc: torvalds@linux-foundation.org, mingo@elte.hu,
peterz@infradead.org, awalls@radix.net,
linux-kernel@vger.kernel.org, jeff@garzik.org,
akpm@linux-foundation.org, jens.axboe@oracle.com,
rusty@rustcorp.com.au, cl@linux-foundation.org,
arjan@linux.intel.com, avi@redhat.com, johannes@sipsolutions.net,
andi@firstfloor.org, oleg@redhat.com
Subject: Re: [PATCHSET] workqueue: concurrency managed workqueue, take#4
Date: Wed, 17 Mar 2010 01:03:03 +0900 [thread overview]
Message-ID: <4B9FABB7.6030906@kernel.org> (raw)
In-Reply-To: <11791.1268750298@redhat.com>
Hello,
On 03/16/2010 11:38 PM, David Howells wrote:
>> Well, you can RR queue them but in general I don't think things like
>> that would be much of a problem for IO bounded works.
>
> "RR queue"? Do you mean realtime?
I meant round-robin as the last resort but if fscache really needs
such workaround, cmwq is probably a bad fit for it.
>> If it becomes bad, scheduler will end up moving the source around
>
> "The source"? Do you mean the process that's loading the deferred
> work items onto the workqueue? Why should it get moved? Isn't it
> pinned to a CPU?
Whatever the source may be. If a cpu gets loaded heavily from fscache
workload, things which aren't pinned to the cpu will be distributed to
other cpus. But again, I have difficult time imagining cpu loading
being an actual issue for fscache even in pathological cases. It's
almost strictly IO bound and CPU intensive stuff sitting in the IO
path already have or should grow mechanisms to schedule those properly
anyway.
>> and for most common cases, those group queued works are gonna hit similar
>> code paths over and over again during their short CPU burn durations so it's
>> likely to be more efficient.
>
> True.
>
>> Are you seeing maleffects of cpu affine work scheduling during
>> fscache load tests?
>
> Hard to say. Hear are some benchmarks:
Yay, some numbers. :-) I reorganized them for easier comparison.
(*) cold/cold-ish server, cold cache:
SLOW-WORK CMWQ
real 2m0.974s 1m5.154s
user 0m0.492s 0m0.628s
sys 0m15.593s 0m14.397s
(*) hot server, cold cache:
SLOW-WORK CMWQ
real 1m31.230s 1m13.408s 1m1.240s 1m4.012s
user 0m0.612s 0m0.652s 0m0.732s 0m0.576s
sys 0m17.845s 0m15.641s 0m13.053s 0m14.133s
(*) hot server, warm cache:
SLOW-WORK CMWQ
real 3m22.108s 3m52.557s 3m10.949s 4m9.805s
user 0m0.636s 0m0.588s 0m0.636s 0m0.648s
sys 0m13.317s 0m16.101s 0m14.065s 0m13.505s
(*) hot server, hot cache:
SLOW-WORK CMWQ
real 1m54.331s 2m2.745s 1m22.511s 2m57.075s
user 0m0.596s 0m0.608s 0m0.612s 0m0.604s
sys 0m11.457s 0m12.625s 0m11.629s 0m12.509s
(*) hot server, no cache:
SLOW-WORK CMWQ
real 1m1.508s 0m54.973s
user 0m0.568s 0m0.712s
sys 0m15.457s 0m13.969s
> Note that it took me several goes to get a second result for this
> case: it kept failing in a way that suggested that the
> non-reentrancy stuff you put in there failed somehow, but it's
> difficult to say for sure.
Sure, there could be a bug in the non-reentrance implementation but
I'm leaning more towards a bug in work flushing before freeing thing
which also seems to show up in the debugfs path. I'll try to
reproduce the problem here and debug it.
That said, the numbers look generally favorable to CMWQ although the
sample size is too small to draw conclusions. I'll try to get things
fixed up so that testing can be smoother.
Thanks a lot for testing.
--
tejun
next prev parent reply other threads:[~2010-03-16 16:02 UTC|newest]
Thread overview: 86+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-02-26 12:22 [PATCHSET] workqueue: concurrency managed workqueue, take#4 Tejun Heo
2010-02-26 12:22 ` [PATCH 01/43] sched: consult online mask instead of active in select_fallback_rq() Tejun Heo
2010-02-26 12:22 ` [PATCH 02/43] sched: rename preempt_notifiers to sched_notifiers and refactor implementation Tejun Heo
2010-02-26 12:22 ` [PATCH 03/43] sched: refactor try_to_wake_up() Tejun Heo
2010-02-26 12:22 ` [PATCH 04/43] sched: implement __set_cpus_allowed() Tejun Heo
2010-02-26 12:22 ` [PATCH 05/43] sched: make sched_notifiers unconditional Tejun Heo
2010-02-26 12:22 ` [PATCH 06/43] sched: add wakeup/sleep sched_notifiers and allow NULL notifier ops Tejun Heo
2010-02-26 12:22 ` [PATCH 07/43] sched: implement try_to_wake_up_local() Tejun Heo
2010-02-28 12:33 ` Oleg Nesterov
2010-03-01 14:22 ` Tejun Heo
2010-02-26 12:22 ` [PATCH 08/43] workqueue: change cancel_work_sync() to clear work->data Tejun Heo
2010-02-26 12:22 ` [PATCH 09/43] acpi: use queue_work_on() instead of binding workqueue worker to cpu0 Tejun Heo
2010-02-26 12:22 ` [PATCH 10/43] stop_machine: reimplement without using workqueue Tejun Heo
2010-02-28 14:11 ` Oleg Nesterov
2010-03-01 15:07 ` Tejun Heo
2010-03-01 15:37 ` Oleg Nesterov
2010-03-01 16:36 ` Tejun Heo
2010-03-01 16:50 ` Oleg Nesterov
2010-03-01 18:02 ` Tejun Heo
2010-02-28 14:34 ` Oleg Nesterov
2010-03-01 15:11 ` Tejun Heo
2010-03-01 15:41 ` Oleg Nesterov
2010-02-26 12:22 ` [PATCH 11/43] workqueue: misc/cosmetic updates Tejun Heo
2010-02-26 12:22 ` [PATCH 12/43] workqueue: merge feature parameters into flags Tejun Heo
2010-02-26 12:22 ` [PATCH 13/43] workqueue: define masks for work flags and conditionalize STATIC flags Tejun Heo
2010-02-26 12:22 ` [PATCH 14/43] workqueue: separate out process_one_work() Tejun Heo
2010-02-26 12:22 ` [PATCH 15/43] workqueue: temporarily disable workqueue tracing Tejun Heo
2010-02-26 12:22 ` [PATCH 16/43] workqueue: kill cpu_populated_map Tejun Heo
2010-02-28 16:00 ` Oleg Nesterov
2010-03-01 15:32 ` Tejun Heo
2010-03-01 15:50 ` Oleg Nesterov
2010-03-01 16:19 ` Tejun Heo
2010-02-26 12:22 ` [PATCH 17/43] workqueue: update cwq alignement Tejun Heo
2010-02-28 17:12 ` Oleg Nesterov
2010-03-01 16:40 ` Tejun Heo
2010-02-26 12:22 ` [PATCH 18/43] workqueue: reimplement workqueue flushing using color coded works Tejun Heo
2010-02-28 20:31 ` Oleg Nesterov
2010-03-01 17:33 ` Tejun Heo
2010-03-01 19:47 ` Oleg Nesterov
2010-02-26 12:22 ` [PATCH 19/43] workqueue: introduce worker Tejun Heo
2010-02-26 12:22 ` [PATCH 20/43] workqueue: reimplement work flushing using linked works Tejun Heo
2010-03-01 14:53 ` Oleg Nesterov
2010-03-01 18:00 ` Tejun Heo
2010-03-01 18:51 ` Oleg Nesterov
2010-02-26 12:22 ` [PATCH 21/43] workqueue: implement per-cwq active work limit Tejun Heo
2010-02-26 12:22 ` [PATCH 22/43] workqueue: reimplement workqueue freeze using max_active Tejun Heo
2010-02-26 12:23 ` [PATCH 23/43] workqueue: introduce global cwq and unify cwq locks Tejun Heo
2010-02-26 12:23 ` [PATCH 24/43] workqueue: implement worker states Tejun Heo
2010-02-26 12:23 ` [PATCH 25/43] workqueue: reimplement CPU hotplugging support using trustee Tejun Heo
2010-02-26 12:23 ` [PATCH 26/43] workqueue: make single thread workqueue shared worker pool friendly Tejun Heo
2010-02-26 12:23 ` [PATCH 27/43] workqueue: add find_worker_executing_work() and track current_cwq Tejun Heo
2010-02-26 12:23 ` [PATCH 28/43] workqueue: carry cpu number in work data once execution starts Tejun Heo
2010-02-28 7:08 ` [PATCH UPDATED " Tejun Heo
2010-02-26 12:23 ` [PATCH 29/43] workqueue: implement WQ_NON_REENTRANT Tejun Heo
2010-02-26 12:23 ` [PATCH 30/43] workqueue: use shared worklist and pool all workers per cpu Tejun Heo
2010-02-26 12:23 ` [PATCH 31/43] workqueue: implement concurrency managed dynamic worker pool Tejun Heo
2010-02-26 12:23 ` [PATCH 32/43] workqueue: increase max_active of keventd and kill current_is_keventd() Tejun Heo
2010-02-26 12:23 ` [PATCH 33/43] workqueue: add system_wq, system_long_wq and system_nrt_wq Tejun Heo
2010-02-26 12:23 ` [PATCH 34/43] workqueue: implement DEBUGFS/workqueue Tejun Heo
2010-02-28 7:13 ` [PATCH UPDATED " Tejun Heo
2010-02-26 12:23 ` [PATCH 35/43] workqueue: implement several utility APIs Tejun Heo
2010-02-28 7:15 ` [PATCH UPDATED " Tejun Heo
2010-02-26 12:23 ` [PATCH 36/43] libata: take advantage of cmwq and remove concurrency limitations Tejun Heo
2010-02-26 12:23 ` [PATCH 37/43] async: use workqueue for worker pool Tejun Heo
2010-02-26 12:23 ` [PATCH 38/43] fscache: convert object to use workqueue instead of slow-work Tejun Heo
2010-02-26 12:23 ` [PATCH 39/43] fscache: convert operation " Tejun Heo
2010-02-26 12:23 ` [PATCH 40/43] fscache: drop references to slow-work Tejun Heo
2010-02-26 12:23 ` [PATCH 41/43] cifs: use workqueue instead of slow-work Tejun Heo
2010-02-28 7:09 ` [PATCH UPDATED " Tejun Heo
2010-02-26 12:23 ` [PATCH 42/43] gfs2: " Tejun Heo
2010-02-28 7:10 ` [PATCH UPDATED " Tejun Heo
2010-02-26 12:23 ` [PATCH 43/43] slow-work: kill it Tejun Heo
2010-02-27 22:52 ` [PATCH] workqueue: Fix build on PowerPC Anton Blanchard
2010-02-28 6:08 ` Tejun Heo
2010-02-28 1:00 ` [PATCH] workqueue: Fix a compile warning in work_busy Anton Blanchard
2010-02-28 6:18 ` Tejun Heo
2010-02-28 1:11 ` [PATCHSET] workqueue: concurrency managed workqueue, take#4 Anton Blanchard
2010-02-28 6:32 ` Tejun Heo
2010-03-10 14:52 ` David Howells
2010-03-12 5:03 ` Tejun Heo
2010-03-12 11:23 ` David Howells
2010-03-12 22:55 ` Tejun Heo
2010-03-16 14:38 ` David Howells
2010-03-16 16:03 ` Tejun Heo [this message]
2010-03-16 17:18 ` David Howells
2010-04-25 8:09 ` [PATCHSET UPDATED] " Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4B9FABB7.6030906@kernel.org \
--to=tj@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=andi@firstfloor.org \
--cc=arjan@linux.intel.com \
--cc=avi@redhat.com \
--cc=awalls@radix.net \
--cc=cl@linux-foundation.org \
--cc=dhowells@redhat.com \
--cc=jeff@garzik.org \
--cc=jens.axboe@oracle.com \
--cc=johannes@sipsolutions.net \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=rusty@rustcorp.com.au \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox