From: Ming Lei <ming.lei@redhat.com>
To: io-uring@vger.kernel.org, linux-block@vger.kernel.org,
linux-kernel@vger.kernel.org, nbd@other.debian.org
Cc: ming.lei@redhat.com
Subject: ublk-nbd: ublk-nbd is avaialbe
Date: Thu, 19 Jan 2023 22:23:28 +0800 [thread overview]
Message-ID: <Y8lSYBU9q5fjs7jS@T590> (raw)
Hi,
ublk-nbd[1] is available now.
Basically it is one nbd client, but totally implemented in userspace,
and wrt. current nbd-client in [2], the transmission phase is done
by linux block nbd driver.
The handshake implementation is borrowed from nbd project[2], so
basically ublk-nbd just adds new code for implementing transmission
phase, and it can be thought as moving linux block nbd driver into
userspace.
The added new code is basically in nbd/tgt_nbd.cpp, and io handling
is based on liburing[3], and implemented by c++20 coroutine, so
everything is done in single pthread totally lockless, meantime turns
out it is pretty easy to design & implement, attributed to ublk framework,
c++20 coroutine and liburing.
ublk-nbd supports both tcp and unix socket, and allows to enable io_uring
send zero copy via command line '--send_zc', see details in README[4].
No regression is found in xfstests by using ublk-nbd as both test device
and scratch device, and builtin test(make test T=nbd) runs well.
Fio test("make test T=nbd") shows that ublk-nbd performance is
basically same with nbd-client/nbd driver when running fio on real
ethernet link(1g, 10+g), but ublk-nbd IOPS is higher by ~40% than
nbd-client(nbd driver) with 512K BS, which is because linux nbd
driver sets max_sectors_kb as 64KB at default.
But when running fio over local tcp socket, it is observed in my test
machine that ublk-nbd performs better than nbd-client/nbd driver,
especially with 2 queue/2 jobs, and the gap could be 10% ~ 30%
according to different block size.
Any comments are welcome!
[1] https://github.com/ming1/ubdsrv/blob/master/nbd
[2] https://github.com/NetworkBlockDevice/nbd
[3] https://github.com/axboe/liburing
[4] https://github.com/ming1/ubdsrv/blob/master/nbd/README.rst
Thanks,
Ming
next reply other threads:[~2023-01-19 14:24 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-19 14:23 Ming Lei [this message]
2023-01-19 18:49 ` ublk-nbd: ublk-nbd is avaialbe Jens Axboe
2023-01-26 3:08 ` Ming Lei
2023-01-26 4:08 ` Willy Tarreau
2023-01-26 11:41 ` Ming Lei
2023-01-26 12:54 ` Willy Tarreau
2023-02-28 10:04 ` Pavel Machek
2023-03-02 3:11 ` Ming Lei
2023-03-11 13:21 ` Wouter Verhelst
2023-03-12 8:30 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y8lSYBU9q5fjs7jS@T590 \
--to=ming.lei@redhat.com \
--cc=io-uring@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nbd@other.debian.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.