Re: [PATCH] block: per-cpu counters for in-flight IO accounting

Jens Axboe
2014-05-09T16:41:27Z

On 05/09/2014 08:12 AM, Jens Axboe wrote:
> On 05/09/2014 03:17 AM, Matias Bjørling wrote:
>> With multi-million IOPS and multi-node workloads, the atomic_t in_flight
>> tracking becomes a bottleneck. Change the in-flight accounting to per-cpu
>> counters to elevate.
>
> The part stats are a pain in the butt, I've tried to come up with a
> great fix for them too. But I don't think the percpu conversion is
> necessarily the right one. The summing is part of the hotpath, so percpu
> counters aren't necessarily the right way to go. I don't have a better
> answer right now, otherwise it would have been fixed :-)

Actual data point - this slows my test down ~14% compared to the stock
kernel. Also, if you experiment with this, you need to watch for the
out-of-core users of the part stats (like DM).

--
Jens Axboe