Re: [PATCH v2 2/2] block: use a per disk workqueue for zone write plugging

[Damien Le Moal <dlemoal@kernel.org>]

On 2024/04/22 16:25, Christoph Hellwig wrote:
> On Sat, Apr 20, 2024 at 04:58:11PM +0900, Damien Le Moal wrote:
>> A zone write plug BIO work function blk_zone_wplug_bio_work() calls
>> submit_bio_noacct_nocheck() to execute the next unplugged BIO. This
>> function may block. So executing zone plugs BIO works using the block
>> layer global kblockd workqueue can potentially lead to preformance or
>> latency issues as the number of concurrent work for a workqueue is
>> limited to WQ_DFL_ACTIVE (256).
>> 1) For a system with a large number of zoned disks, issuing write
>>    requests to otherwise unused zones may be delayed wiating for a work
>>    thread to become available.
>> 2) Requeue operations which use kblockd but are independent of zone
>>    write plugging may alsoi end up being delayed.
>>
>> To avoid these potential performance issues, create a workqueue per
>> zoned device to execute zone plugs BIO work. The workqueue max active
>> parameter is set to the maximum number of zone write plugs allocated
>> with the zone write plug mempool. This limit is equal to the maximum
>> number of open zones of the disk and defaults to 128 for disks that do
>> not have a limit on the number of open zones.
> 
> Looks good:
> 
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> 
> Should the zone write plug submission do non-blocking submissions as well
> to avoid stalling in the workqueue thread all the time?

I do not think that the stalling actually happens that often. The 2 main cases I
see are:
1) Out of tag so we block on tag allocation when preparing the request in
submit_bio_noacct_nocheck(), or
2) The device has BLK_MQ_F_BLOCKING set for its tag set (e.g. nullblk with
memory backing).

For (1), we could use RQF_NOWAIT to prevent blocking, but then we would need to
retry later on with a timer to make forward progress for the plug. And I do not
think we can actually avoid (2). So in the end, I do not see a clean way to
completely avoid blocking in all cases.

-- 
Damien Le Moal
Western Digital Research