All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ming Lei <tom.leiming@gmail.com>
To: Qiliang Yuan <qiliang.yuan@linux.dev>
Cc: "Martin K. Petersen" <mkp@kernel.org>,
	Shuah Khan <shuah@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org,
	target-devel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	linux-doc@vger.kernel.org, Jens Axboe <axboe@kernel.dk>,
	io-uring@vger.kernel.org,
	Mike Christie <michael.christie@oracle.com>,
	Bodo Stroesser <bostroesser@gmail.com>
Subject: Re: [PATCH RFC 00/18] scsi: target: Add target_core_uring, an io_uring based user backstore
Date: Tue, 17 Nov 2026 19:40:18 -0600	[thread overview]
Message-ID: <av0CAptnbIZP-53U@fedora-laptop> (raw)
In-Reply-To: <20261117024640.3184554-1-qiliang.yuan@linux.dev>

On Mon, Nov 16, 2026 at 09:46:40PM -0500, Qiliang Yuan wrote:
> On Sun, Nov 15, 2026 at 08:32:02PM -0600, Ming Lei wrote:
> > This is ublk's uapi re-implemented inside LIO: FETCH/COMMIT_AND_FETCH
> > per tag, pread()/pwrite() with (qid, tag) in the offset, and buffer
> > registration via io_buffer_register_bvec().
> 
> Yes, the interface is modelled on ublk on purpose.
> 
> > 1) If LIO is required (vhost-scsi, FC, SRP, iSER), serve the device
> > with ublk and export it via target_core_iblock. Same per-queue
> > io_uring, same user copy and zero copy path, no new uapi. Please
> > benchmark that configuration, with UBLK_F_USER_COPY and with
> > UBLK_F_AUTO_BUF_REG, same 32 queues and threads.
> 
> Done, with kublk from selftests/ublk (loop target, -q 32 -d 128, "-u"
> and "--auto_zc") behind target_core_iblock, against this series with 32
> queues. Both serve the same 2 GiB file on tmpfs. Same VM (32 CPUs) and
> fio jobs as in the cover letter, three alternating runs, median.

Can you share your setup? It is supposed that you run fio from VM,
otherwise you can just compare ublk with scsi backed with tcmu.

But if you run fio from VM, I believe you can setup virtio-scsi backed by
ublk block device directly, which should cover your requirement, or maybe
I miss something?

...

> > 32 threads against one, over tcm_loop with data in memory, measures
> > only the hand-off. Please run a real fabric and explain the -13%
> > single job result.
> 
> Same fabrics, data in memory, TCM/USER against this series with one
> queue (one server thread) and with 32. Over iSCSI:
> 
>   workload               TCM/USER  1 queue          32 queues
>                            (MB/s)   (MB/s)  change     (MB/s)  change
>   4k randread, 1 job          138      157    +14%        155    +12%
>   4k randread, 4 jobs         538      679    +26%        685    +27%
>   4k randread, 16 jobs        720     1101    +53%       2631   +265%
>   4k randwrite, 16 jobs       859     1228    +43%       2161   +152%
>   1M read, 4 jobs             756      799     +6%        765     +1%
>   1M write, 4 jobs           2263     6075   +168%       5379   +138%
> 
> 1M reads over iSCSI are limited by the loopback TCP connection for
> every backend here (retransmits and zero window stalls), so that row
> says nothing about the backstores.
> 
> And through vhost-scsi:
> 
>   workload               TCM/USER  1 queue          32 queues
>                            (MB/s)   (MB/s)  change     (MB/s)  change
>   4k randread, 1 job          201      237    +18%        238    +18%
>   4k randread, 4 jobs        1095      902    -18%        843    -23%
>   4k randread, 16 jobs        957     1119    +17%       2865   +199%
>   4k randwrite, 16 jobs       749     1098    +47%       2599   +247%
>   1M read, 4 jobs            5373    15317   +185%      56532   +952%
>   1M write, 4 jobs           2064    10948   +430%      27074  +1212%

My question is actually why there is regression wrt. 4k randread:

 workload               TCM/USER (MB/s)   uring (MB/s)          change
  4k randread, 1 job      426 (409-463)      369 (366-384)         -13%

Maybe you can focus on single job perf only, which is easy to compare and
investigate.

> 
> On the single job over tcm_loop: one fio job submits from one CPU, so
> only one queue and one server thread are in use and it is one thread
> against one. The TCM/USER handler finds the data area in its mmap and
> takes a whole batch per wakeup, while the server here issues one
> pwrite() per read on top of the io_uring_enter(). Counted over the
> whole system during that job, that is 1.1 system calls and 0.27 context
> switches per I/O for TCM/USER against 2.2 and 0.39 for this series.
> 
> > 2) If the goal is iSCSI for userspace storage, keep the kernel out of
> > the data path entirely.
> 
> The goal is the fabrics LIO already has, vhost-scsi and tcm_loop as
> much as iSCSI, for setups that run TCM/USER today.

When userspace target becomes more efficient, I'd suggest you to think
about why these applications can't be improved or refactored for the
potential (bigger)perf requirement.



Thanks,
Ming

      reply	other threads:[~2026-11-18  1:40 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  0:17 [PATCH RFC 00/18] scsi: target: Add target_core_uring, an io_uring based user backstore Qiliang Yuan
2026-11-16  0:17 ` [PATCH RFC 01/18] scsi: target: uring: Introduce " Qiliang Yuan
2026-11-16  0:38   ` sashiko-bot
2026-11-16  0:17 ` [PATCH RFC 02/18] scsi: target: uring: Allocate per-queue command descriptors Qiliang Yuan
2026-11-16  0:17 ` [PATCH RFC 03/18] scsi: target: uring: Create a character device per backstore device Qiliang Yuan
2026-11-16  0:33   ` sashiko-bot
2026-11-16  0:17 ` [PATCH RFC 04/18] scsi: target: uring: Map the command descriptors into the server Qiliang Yuan
2026-11-16  0:17 ` [PATCH RFC 05/18] scsi: target: uring: Queue commands until a server fetches them Qiliang Yuan
2026-11-16  0:46   ` sashiko-bot
2026-11-16  0:17 ` [PATCH RFC 06/18] scsi: target: uring: Hand commands out through TCMUR_IO_FETCH Qiliang Yuan
2026-11-16  0:18 ` [PATCH RFC 07/18] scsi: target: uring: Complete commands with TCMUR_IO_COMMIT_AND_FETCH Qiliang Yuan
2026-11-16  0:18 ` [PATCH RFC 08/18] scsi: target: uring: Copy command data with pread() and pwrite() Qiliang Yuan
2026-11-16  0:18 ` [PATCH RFC 09/18] scsi: target: uring: Fail in-flight commands when the server exits Qiliang Yuan
2026-11-16  0:18 ` [PATCH RFC 10/18] scsi: target: uring: Time out commands the server does not complete Qiliang Yuan
2026-11-16  0:41   ` sashiko-bot
2026-11-16  0:18 ` [PATCH RFC 11/18] scsi: target: uring: Drop aborted commands that are still queued Qiliang Yuan
2026-11-16  0:37   ` sashiko-bot
2026-11-16  0:18 ` [PATCH RFC 12/18] scsi: target: uring: Support SYNCHRONIZE CACHE Qiliang Yuan
2026-11-16  0:40   ` sashiko-bot
2026-11-16  0:18 ` [PATCH RFC 13/18] scsi: target: uring: Implement UNMAP Qiliang Yuan
2026-11-16  0:45   ` sashiko-bot
2026-11-16  0:18 ` [PATCH RFC 14/18] scsi: target: uring: Handle WRITE SAME Qiliang Yuan
2026-11-16  0:44   ` sashiko-bot
2026-11-16  0:18 ` [PATCH RFC 15/18] scsi: target: uring: Register command data as io_uring fixed buffers Qiliang Yuan
2026-11-16  0:18 ` [PATCH RFC 16/18] selftests: target_core_uring: Add a minimal server Qiliang Yuan
2026-11-16  1:03   ` sashiko-bot
2026-11-16  0:18 ` [PATCH RFC 17/18] selftests: target_core_uring: Test a device exported through tcm_loop Qiliang Yuan
2026-11-16  0:49   ` sashiko-bot
2026-11-16  0:18 ` [PATCH RFC 18/18] Documentation: target: Describe the uring backstore Qiliang Yuan
2026-11-16  2:32 ` [PATCH RFC 00/18] scsi: target: Add target_core_uring, an io_uring based user backstore Ming Lei
2026-11-17  2:46   ` Qiliang Yuan
2026-11-18  1:40     ` Ming Lei [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=av0CAptnbIZP-53U@fedora-laptop \
    --to=tom.leiming@gmail.com \
    --cc=axboe@kernel.dk \
    --cc=bostroesser@gmail.com \
    --cc=corbet@lwn.net \
    --cc=io-uring@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=michael.christie@oracle.com \
    --cc=mkp@kernel.org \
    --cc=qiliang.yuan@linux.dev \
    --cc=rdunlap@infradead.org \
    --cc=shuah@kernel.org \
    --cc=skhan@linuxfoundation.org \
    --cc=target-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.