From: Jens Axboe <axboe@kernel.dk>
To: "Matias Bjørling" <m@bjorling.me>, sbradshaw@micron.com
Cc: linux-kernel@vger.kernel.org
Subject: Re: [PATCH] block: per-cpu counters for in-flight IO accounting
Date: Fri, 09 May 2014 10:41:27 -0600 [thread overview]
Message-ID: <536D0537.7010905@kernel.dk> (raw)
In-Reply-To: <536CE25C.5040107@kernel.dk>
On 05/09/2014 08:12 AM, Jens Axboe wrote:
> On 05/09/2014 03:17 AM, Matias Bjørling wrote:
>> With multi-million IOPS and multi-node workloads, the atomic_t in_flight
>> tracking becomes a bottleneck. Change the in-flight accounting to per-cpu
>> counters to elevate.
>
> The part stats are a pain in the butt, I've tried to come up with a
> great fix for them too. But I don't think the percpu conversion is
> necessarily the right one. The summing is part of the hotpath, so percpu
> counters aren't necessarily the right way to go. I don't have a better
> answer right now, otherwise it would have been fixed :-)
Actual data point - this slows my test down ~14% compared to the stock
kernel. Also, if you experiment with this, you need to watch for the
out-of-core users of the part stats (like DM).
--
Jens Axboe
next prev parent reply other threads:[~2014-05-09 16:41 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-09 9:17 [PATCH] block: small performance optimization Matias Bjørling
2014-05-09 9:17 ` [PATCH] block: per-cpu counters for in-flight IO accounting Matias Bjørling
2014-05-09 14:12 ` Jens Axboe
2014-05-09 16:41 ` Jens Axboe [this message]
2014-05-30 12:11 ` Shaohua Li
2014-05-30 13:49 ` Jens Axboe
2014-06-04 10:39 ` Shaohua Li
2014-06-04 11:29 ` Matias Bjørling
2014-06-04 20:08 ` Jens Axboe
2014-06-05 2:09 ` Shaohua Li
2014-06-05 2:16 ` Jens Axboe
2014-06-05 2:33 ` Shaohua Li
2014-06-05 2:42 ` Jens Axboe
2014-06-04 14:29 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=536D0537.7010905@kernel.dk \
--to=axboe@kernel.dk \
--cc=linux-kernel@vger.kernel.org \
--cc=m@bjorling.me \
--cc=sbradshaw@micron.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.