qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Nir Soffer <nirsof@gmail.com>
To: qemu-devel@nongnu.org
Cc: "Daniel P. Berrangé" <berrange@redhat.com>,
	"Eric Blake" <eblake@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Richard Jones" <rjones@redhat.com>,
	"Vladimir Sementsov-Ogievskiy" <vsementsov@yandex-team.ru>,
	qemu-block@nongnu.org, "Nir Soffer" <nirsof@gmail.com>
Subject: [PATCH v4 0/3] nbd: Increase unix socket buffer size
Date: Sat, 17 May 2025 23:11:51 +0300	[thread overview]
Message-ID: <20250517201154.88456-1-nirsof@gmail.com> (raw)

On both macOS and Linux, the default send buffer size is too small causing poor
performance when reading and writing to qemu-nbd. A simple way to experience
this is to compare TCP and unix sockets, showing that TCP socket is much
faster. Programs like nbdcopy partly mitigate this by using multiple NBD
connections.

On macOS the default send buffer size is 8192 bytes. Increasing the send buffer
size to 2 MiB shows up to *12.6 times higher throughput* and lower cpu usage.

On Linux the default and maximum buffer size is 212992 bytes. Increasing the
send buffer size to 2 MiB shows up to *2.7 times higher throughput* and lower
cpu usage. On older machine we see very little improvement, up to 1.03 times
higher throughput.

We likely have the same issue on other platforms. It should be easy to enable
this change for more platform by defining UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE.

Changes since v3:
- Modify only NBD sockets, since not all sockets need a larger buffer size, and
  the fixed memory overhead per socker may be a significant amount of extra
  memory overhead on a host. (Daniel)
- Add qio_channel_socket_set_send_buffer() helper to set buffer size from nbd
  code.
- Remove the check for SOCK_STREAM since NBD does not support SOCK_DGRAM.
- Warn if we fail to set the socket send buffer size.

v3 was here:
https://lists.gnu.org/archive/html/qemu-devel/2025-04/msg05096.html

Nir Soffer (3):
  io: Add helper for setting socket send buffer size
  nbd: Set unix socket send buffer on macOS
  nbd: Set unix socket send buffer on Linux

 include/io/channel-socket.h | 13 +++++++++++++
 io/channel-socket.c         | 11 +++++++++++
 nbd/client-connection.c     |  3 +++
 nbd/common.c                | 26 ++++++++++++++++++++++++++
 nbd/nbd-internal.h          |  5 +++++
 nbd/server.c                |  2 ++
 6 files changed, 60 insertions(+)

-- 
2.39.5 (Apple Git-154)



             reply	other threads:[~2025-05-17 20:12 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-05-17 20:11 Nir Soffer [this message]
2025-05-17 20:11 ` [PATCH v4 1/3] io: Add helper for setting socket send buffer size Nir Soffer
2025-05-19 10:46   ` Daniel P. Berrangé
2025-05-19 11:20   ` Richard W.M. Jones
2025-05-19 20:11     ` Eric Blake
2025-05-17 20:11 ` [PATCH v4 2/3] nbd: Set unix socket send buffer on macOS Nir Soffer
2025-05-19 10:47   ` Daniel P. Berrangé
2025-05-17 20:11 ` [PATCH v4 3/3] nbd: Set unix socket send buffer on Linux Nir Soffer
2025-05-19 10:47   ` Daniel P. Berrangé
2025-05-19 20:12 ` [PATCH v4 0/3] nbd: Increase unix socket buffer size Eric Blake

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250517201154.88456-1-nirsof@gmail.com \
    --to=nirsof@gmail.com \
    --cc=berrange@redhat.com \
    --cc=eblake@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=rjones@redhat.com \
    --cc=vsementsov@yandex-team.ru \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).