[PATCH v4 0/2] ublk: honor rq_affinity on request completion

[Yoav Cohen <yoav@nvidia.com>]

ublk ends successfully completed requests inline on the server thread,
so rq_affinity never redirects completion work back to the submitting
CPU the way it does for NVMe, SCSI, virtio-blk, loop, nbd and rnbd.
Patch 1 fixes that and adds a feature flag so servers can detect
support. Patch 2 adds a selftest for it.

With an SPDK-style polling user-space block server at queue depth 96,
rq_affinity=2 gives 7-8% higher IOPS than today's always-inline
completion by offloading a saturated server CPU; rq_affinity=0 keeps
the old behavior for anyone who wants to opt out.

v4, addressing Ming Lei's review of v3:
- The selftest didn't reach the remote completion path, as kublk pins the
  queue 0 thread to the first CPU of its hctx mask, which is where I/O was
  submitted. Use a single queue and submit from the last CPU in the
  hctx's cpu_list, so rq_affinity=2 always takes the remote path.
- No change to the driver patch.

v3, addressing Caleb Sander Mateos's and Ming Lei's review of v2:
- Pass whether the device opted in to __ublk_complete_rq() from the
  ublk_device its callers already have, instead of reading it through
  req->mq_hctx->driver_data and ubq->flags, which are likely cache
  misses on a server thread servicing many queues.
- ublk_end_rq() may run on a remote CPU, so it no longer touches ublk_io
  or ublk_queue; it only ends the request. Only a request that completed
  in full (io->res == blk_rq_bytes()) is redirected. Short completion,
  errors and copy-back failures stay in __ublk_complete_rq() as before.
- Drop the re-derivation of the completion path in ublk_end_rq(), and
  with it ublk_rq_need_unmap() and ublk_update_and_requeue().

v2, addressing Ming Lei's review of v1:
- v1 only redirected the classic-copy-mode successful-read completion;
  every write, flush/discard, error, and zero-copy/user-copy/
  auto-buf-reg/shmem-zc I/O still completed inline as before, so
  rq_affinity had no effect for most traffic. v2 redirects every
  completion outcome.
- v1 ran the new completion path unconditionally on any patched
  kernel. v2 adds UBLK_F_SUPPORT_RQ_AFFINITY, which a server must
  request at ADD_DEV time; without it, behavior is unchanged.

Yoav Cohen (2):
  ublk: honor rq_affinity on request completion
  selftests: ublk: add rq_affinity test

 drivers/block/ublk_drv.c                      | 68 +++++++++++++++----
 include/uapi/linux/ublk_cmd.h                 |  8 +++
 tools/testing/selftests/ublk/Makefile         |  1 +
 tools/testing/selftests/ublk/kublk.c          |  6 +-
 .../testing/selftests/ublk/test_generic_19.sh | 51 ++++++++++++++
 5 files changed, 121 insertions(+), 13 deletions(-)
 create mode 100755 tools/testing/selftests/ublk/test_generic_19.sh

-- 
2.50.1 (Apple Git-155)