Linux block layer
 help / color / mirror / Atom feed
* [PATCH v4 0/2] scsi: bsg: release io_uring requests from end-I/O
@ 2026-11-18 15:37 Yingjie Wang
  2026-11-18 15:37 ` [PATCH v4 1/2] block: bsg: add a per-file pool of uring_cmd completion buffers Yingjie Wang
  2026-11-18 15:37 ` [PATCH v4 2/2] scsi: bsg: release io_uring requests from end-I/O Yingjie Wang
  0 siblings, 2 replies; 3+ messages in thread
From: Yingjie Wang @ 2026-11-18 15:37 UTC (permalink / raw)
  To: Martin K. Petersen
  Cc: James.Bottomley@HansenPartnership.com, FUJITA Tomonori,
	axboe@kernel.dk, linux-scsi@vger.kernel.org,
	linux-block@vger.kernel.org, bvanassche@acm.org,
	jthoughton@google.com, yangxiuwei@kylinos.cn,
	linux-kernel@vger.kernel.org

This is the reworked bsg half of the series "scsi: release completed
passthrough requests before user copyback" (RFC v3, 2026-11-07).  The
four synchronous patches have been posted separately as a non-RFC v4
series.  These two patches apply on top of that series:

https://lore.kernel.org/r/cover.1794990780.git.x-professor@outlook.com

Original RFC v3 series:
https://lore.kernel.org/r/cover.1794045010.git.x-professor@outlook.com

Yang's suggestion:
https://lore.kernel.org/r/20261111003655.3678579-1-yangxiuwei@kylinos.cn

Compared to the RFC v3 version of this half:

- RFC v3 5/6 exposed a new io_uring helper that handed the copied-SQE
  storage of a uring command to the driver.  Yang Xiuwei pointed out
  that storage owned by bsg itself avoids coupling the BSG driver to
  the io_uring layout and lifetime, with a per-file or per-queue pool
  and -EAGAIN on exhaustion instead of sleeping.  That removes the need
  for the io_uring contract ruling that RFC v3 asked for; the io_uring
  helper is dropped, and the reworked series no longer changes io_uring
  code.

- Patch 1 adds a per-file pool of 96-byte completion buffers to bsg,
  allocated at open() and freed at release().  Every open() of a SCSI
  bsg device creates a pool, so it is charged to the opener's memory
  cgroup; transport bsg devices and kernels built without io_uring
  allocate no pool.  Patch 2 checks a buffer out at issue time,
  snapshots the sense data in end-I/O, and returns the buffer in task
  work before posting the CQE.  Only commands that pass a response
  buffer and a nonzero response length can deliver sense bytes, so only
  those consume a pool buffer.  The buffer is taken before the user
  data buffer is mapped, because unmapping a bounce-mapped READ copies
  zero-filled bounce pages back to the caller: a command rejected after
  mapping would overwrite the caller's data.

- When the pool is exhausted, submission returns -EAGAIN without
  waiting for a buffer.  An initial nonblocking attempt may be retried
  through io-wq; if the pool is still empty in the worker, the command
  completes with -EAGAIN and userspace must resubmit it to retry.  This
  introduces a per-file limit of 64 commands that pass both a response
  buffer and a nonzero response length and are in flight or waiting for
  task work.
  64 is BSG_DEFAULT_CMDS, the default bsg reports through
  SG_GET_COMMAND_Q, but the old io_uring path enforced no such limit,
  and SG_SET_COMMAND_Q does not resize the pool.  The depth can be
  raised in the code if deeper queues on one file matter; each further
  64 slots cost about another 8.6 KiB per open file.

I chose a per-file pool over a per-queue one so that one opener cannot
exhaust the buffers available to other openers and make their commands
fail with -EAGAIN, for example with many commands in flight, or with a
DEFER_TASKRUN ring whose task has not entered the kernel to wait for
completions, so its task work has not run.  It also makes the
64-command limit apply per file rather than to the whole device.  The
cost is one pool per open file, about 8.6 KiB of slab on x86-64,
including for openers that only use SG_IO, on kernels built with
io_uring.

Tested on a KASAN+lockdep kernel in QEMU with a virtio-scsi disk: the
existing regression from the synchronous series passes (960
assertions, 740 of them on the BSG io_uring path), and its 20 deferred
io_uring trace windows now retain no request, where the synchronous
series alone leaves one in each.  The canceled-task-work regression
passes.  A new pool test submits one 256-deep batch on a DEFER_TASKRUN
ring, which runs task work only when the task waits for completions:
the 192 commands beyond the pool depth complete with -EAGAIN instead of
sleeping, all of them complete on resubmission with the expected
status and sense length, and all 64 buffers are available afterwards.
With all 64 buffers held, an unaligned READ is rejected before its data
buffer is mapped, so the caller's data is untouched.  No kernel
diagnostics.  Both patches build with W=1 on x86-64; the final objects
also build on i386 and with IO_URING=n and SCSI=m.

An earlier memcg test of the same allocation code, before this rebase,
confirmed the accounting on a non-debug kernel: each bsg open was
charged 8832 bytes more than an open of /dev/null, and with memory.max
at 32 MiB, repeated opens triggered the memcg OOM killer after about
3600 opens.

To check the overhead of this series, earlier tests of the same
implementation, before this rebase, timed batches of INQUIRY commands
through a QEMU virtio-scsi disk on a non-debug x86-64 kernel (one fd,
normal and DEFER_TASKRUN rings; batch time divided by the number of
commands, not per-command latency).  Up to 64 commands per batch, the
time per command was within about 2% of the kernel without these two
patches, with or without a response buffer.  At 96 with a response
buffer, the 32 commands beyond the pool depth completed with -EAGAIN
and were resubmitted after the batch, adding about 17% per command.
These numbers come from one VM, not real hardware.

Both patches are tagged for stable.  Patch 2 fixes 7b6d3255e7f8
("scsi: bsg: add io_uring passthrough handler") and does not build
without patch 1, so a backport needs both, in order.  Both are larger
than the 100 lines the stable rules ask for; if that rules them out,
I am fine with dropping the tags.

Yingjie Wang (2):
  block: bsg: add a per-file pool of uring_cmd completion buffers
  scsi: bsg: release io_uring requests from end-I/O

 block/bsg-lib.c         |   3 +-
 block/bsg.c             | 141 +++++++++++++++++++++++++++++++++++++++-
 drivers/scsi/scsi_bsg.c | 118 ++++++++++++++++++++++++---------
 include/linux/bsg.h     |   6 +-
 4 files changed, 232 insertions(+), 36 deletions(-)


base-commit: 9f9e0b1e5399045982788a954d3461de894e40c7
prerequisite-patch-id: a314d6d602c04c1f965a95edc39f159adfabdc2f
prerequisite-patch-id: d2d516da234fb37cd463dfc6292a62fcdcc6673f
prerequisite-patch-id: ce66c68fb99ef813e5506ba7f3de512c1fd0c832
prerequisite-patch-id: 9de82eb99c62cae2c0a4a266744008e25a241fc2
-- 
2.43.0


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2026-11-18 15:37 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 15:37 [PATCH v4 0/2] scsi: bsg: release io_uring requests from end-I/O Yingjie Wang
2026-11-18 15:37 ` [PATCH v4 1/2] block: bsg: add a per-file pool of uring_cmd completion buffers Yingjie Wang
2026-11-18 15:37 ` [PATCH v4 2/2] scsi: bsg: release io_uring requests from end-I/O Yingjie Wang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox