From: Jens Axboe <axboe@kernel.dk>
To: "Matias Bjørling" <m@bjorling.me>, sbradshaw@micron.com
Cc: linux-kernel@vger.kernel.org
Subject: Re: [PATCH] block: per-cpu counters for in-flight IO accounting
Date: Fri, 09 May 2014 10:41:27 -0600 [thread overview]
Message-ID: <536D0537.7010905@kernel.dk> (raw)
In-Reply-To: <536CE25C.5040107@kernel.dk>
On 05/09/2014 08:12 AM, Jens Axboe wrote:
> On 05/09/2014 03:17 AM, Matias Bjørling wrote:
>> With multi-million IOPS and multi-node workloads, the atomic_t in_flight
>> tracking becomes a bottleneck. Change the in-flight accounting to per-cpu
>> counters to elevate.
>
> The part stats are a pain in the butt, I've tried to come up with a
> great fix for them too. But I don't think the percpu conversion is
> necessarily the right one. The summing is part of the hotpath, so percpu
> counters aren't necessarily the right way to go. I don't have a better
> answer right now, otherwise it would have been fixed :-)
Actual data point - this slows my test down ~14% compared to the stock
kernel. Also, if you experiment with this, you need to watch for the
out-of-core users of the part stats (like DM).
--
Jens Axboe
next prev parent reply other threads:[~2014-05-09 16:41 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-09 9:17 [PATCH] block: small performance optimization Matias Bjørling
2014-05-09 9:17 ` [PATCH] block: per-cpu counters for in-flight IO accounting Matias Bjørling
2014-05-09 14:12 ` Jens Axboe
2014-05-09 16:41 ` Jens Axboe [this message]
2014-05-30 12:11 ` Shaohua Li
2014-05-30 13:49 ` Jens Axboe
2014-06-04 10:39 ` Shaohua Li
2014-06-04 11:29 ` Matias Bjørling
2014-06-04 20:08 ` Jens Axboe
2014-06-05 2:09 ` Shaohua Li
2014-06-05 2:16 ` Jens Axboe
2014-06-05 2:33 ` Shaohua Li
2014-06-05 2:42 ` Jens Axboe
2014-06-04 14:29 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=536D0537.7010905@kernel.dk \
--to=axboe@kernel.dk \
--cc=linux-kernel@vger.kernel.org \
--cc=m@bjorling.me \
--cc=sbradshaw@micron.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox