[PATCH] brd: mark as nowait compatible

[PATCH] brd: mark as nowait compatible

[Jens Axboe]

By default, non-mq drivers do not support nowait. This causes io_uring
to use a slower path as the driver cannot be trust not to block. brd
can safely set the nowait flag, as worst case all it does is a NOIO
allocation.

For io_uring, this makes a substantial difference. Before:

submitter=0, tid=453, file=/dev/ram0, node=-1
polled=0, fixedbufs=1/0, register_files=1, buffered=0, QD=128
Engine=io_uring, sq_ring=128, cq_ring=128
IOPS=440.03K, BW=1718MiB/s, IOS/call=32/31
IOPS=428.96K, BW=1675MiB/s, IOS/call=32/32
IOPS=442.59K, BW=1728MiB/s, IOS/call=32/31
IOPS=419.65K, BW=1639MiB/s, IOS/call=32/32
IOPS=426.82K, BW=1667MiB/s, IOS/call=32/31

and after:

submitter=0, tid=354, file=/dev/ram0, node=-1
polled=0, fixedbufs=1/0, register_files=1, buffered=0, QD=128
Engine=io_uring, sq_ring=128, cq_ring=128
IOPS=3.37M, BW=13.15GiB/s, IOS/call=32/31
IOPS=3.45M, BW=13.46GiB/s, IOS/call=32/31
IOPS=3.43M, BW=13.42GiB/s, IOS/call=32/32
IOPS=3.43M, BW=13.39GiB/s, IOS/call=32/31
IOPS=3.43M, BW=13.38GiB/s, IOS/call=32/31

or about an 8x in difference.

Signed-off-by: Jens Axboe <axboe@kernel.dk>

---

diff --git a/drivers/block/brd.c b/drivers/block/brd.c
index 20acc4a1fd6d..82419e345777 100644
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@ -412,6 +412,7 @@ static int brd_alloc(int i)
 	/* Tell the block layer that this is not a rotational device */
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, disk->queue);
 	blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, disk->queue);
+	blk_queue_flag_set(QUEUE_FLAG_NOWAIT, disk->queue);
 	err = add_disk(disk);
 	if (err)
 		goto out_cleanup_disk;

-- 
Jens Axboe

Re: [PATCH] brd: mark as nowait compatible

[Christoph Hellwig]

On Wed, Feb 15, 2023 at 04:43:47PM -0700, Jens Axboe wrote:
> By default, non-mq drivers do not support nowait. This causes io_uring
> to use a slower path as the driver cannot be trust not to block. brd
> can safely set the nowait flag, as worst case all it does is a NOIO
> allocation.

But a NOIO allocation can block.  I think we need to do a
GFP_NOWAIT allocation in brd_insert_page if the NOWAIT flag is set.

Re: [PATCH] brd: mark as nowait compatible

[Jens Axboe]

On 2/15/23 11:08 PM, Christoph Hellwig wrote:
> On Wed, Feb 15, 2023 at 04:43:47PM -0700, Jens Axboe wrote:
>> By default, non-mq drivers do not support nowait. This causes io_uring
>> to use a slower path as the driver cannot be trust not to block. brd
>> can safely set the nowait flag, as worst case all it does is a NOIO
>> allocation.
> 
> But a NOIO allocation can block.  I think we need to do a
> GFP_NOWAIT allocation in brd_insert_page if the NOWAIT flag is set.

I did consider that, but we do allocations almost everywhere and
as long as we're not waiting on IO, it's mostly considered acceptable.
But I can make that change, no reason not to.

-- 
Jens Axboe

Re: [PATCH] brd: mark as nowait compatible

[Christoph Hellwig]

On Thu, Feb 16, 2023 at 06:11:41AM -0700, Jens Axboe wrote:
> I did consider that, but we do allocations almost everywhere and

Do we?  All the code I've touched for nowait goes to great length
to avoid blocking allocations.

Re: [PATCH] brd: mark as nowait compatible

[Jens Axboe]

On 2/16/23 7:40 AM, Christoph Hellwig wrote:
> On Thu, Feb 16, 2023 at 06:11:41AM -0700, Jens Axboe wrote:
>> I did consider that, but we do allocations almost everywhere and
> 
> Do we?  All the code I've touched for nowait goes to great length
> to avoid blocking allocations.

It is prudent, I'm not disagreeing. I've got a v2, will send it
out shortly.

-- 
Jens Axboe