Re: blk-mq: bitmap tag: performance degradation?

[Jens Axboe <axboe@kernel.dk>]

On 06/05/2014 08:16 AM, Ming Lei wrote:
> On Thu, Jun 5, 2014 at 10:03 PM, Jens Axboe <axboe@kernel.dk> wrote:
>> On 2014-06-05 08:01, Alexander Gordeev wrote:
>>>
>>> On Wed, Jun 04, 2014 at 08:18:42AM -0600, Jens Axboe wrote:
>>>>
>>>> A null_blk test is the absolute best case for percpu_ida, since
>>>> there are enough tags and everything is localized. The above test is
>>>> more useful for testing blk-mq than any real world application of
>>>> the tagging.
>>>>
>>>> I've done considerable testing on both 2 and 4 socket (32 and 64
>>>> CPUs) and bitmap tagging is better in a much wider range of
>>>> applications. This includes even high tag depth devices like nvme,
>>>> and more normal ranges like mtip32xx and scsi-mq setups.
>>>
>>>
>>> Just for the record: bitmap tags on a 48 CPU box with NVMe device
>>> indeed shows almost the same performance/cache rate as the stock
>>> kernel.
>>
>>
>> Thanks for confirming. It's one of the dangers of null_blk, it's not always
>> a very accurate simulation of what a real device will do. I think it's
>> mostly a completion side thing, would be great with a small device that
>> supported msi-x and could be used as an irq trigger :-)
> 
> Maybe null_blk at IRQ_TIMER mode is more close to
> a real device, and I guess the result may be different with
> mode IRQ_NONE/IRQ_SOFTIRQ.

It'd be closer in behavior, but the results might then be skewed by
hitting the timer way too hard. And it'd be a general slowdown, again
possibly skewing it. But I haven't tried with the timer completion, to
see if that yields more accurate modelling for this test, so it might
actually be a lot better.

-- 
Jens Axboe