From: Eric Blake <eblake@redhat.com>
To: libguestfs@redhat.com
Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, nbd@other.debian.org
Subject: [libnbd PATCH v2 00/23] libnbd 64-bit NBD extensions
Date: Mon, 14 Nov 2022 16:51:35 -0600 [thread overview]
Message-ID: <20221114225158.2186742-1-eblake@redhat.com> (raw)
In-Reply-To: <20221114224141.cm5jgyxfmvie5xb5@redhat.com>
This series is posted alongside a spec change to NBD, and
interoperable with changes posted to qemu-nbd/qemu-storage-daemon.
The RFC patch at the end is optional; ineroperability with qemu works
only when either both projects omit the RFC patch, or when both
projects include it (if only one of the two RFC projects include it,
the protocol is incompatible between the two, but at least client and
server gracefully detect the bug rather than SEGV'ing).
Eric Blake (23):
block_status: Refactor array storage
internal: Refactor layout of replies in sbuf
protocol: Add definitions for extended headers
states: Prepare to send 64-bit requests
states: Prepare to receive 64-bit replies
states: Break deadlock if server goofs on extended replies
generator: Add struct nbd_extent in prep for 64-bit extents
block_status: Track 64-bit extents internally
block_status: Accept 64-bit extents during block status
api: Add [aio_]nbd_block_status_64
api: Add several functions for controlling extended headers
copy: Update nbdcopy to use 64-bit block status
dump: Update nbddump to use 64-bit block status
info: Expose extended-headers support through nbdinfo
info: Update nbdinfo --map to use 64-bit block status
examples: Update copy-libev to use 64-bit block status
ocaml: Add example for 64-bit extents
generator: Actually request extended headers
api: Add nbd_[aio_]opt_extended_headers()
interop: Add test of 64-bit block status
api: Add nbd_can_block_status_payload()
api: Add nbd_[aio_]block_status_filter()
RFC: pread: Accept 64-bit holes
docs/libnbd.pod | 18 +-
info/nbdinfo.pod | 21 +-
sh/nbdsh.pod | 2 +-
lib/internal.h | 42 +-
lib/nbd-protocol.h | 120 ++--
generator/API.ml | 532 +++++++++++++++---
generator/API.mli | 1 +
generator/C.ml | 24 +-
generator/GoLang.ml | 24 +
generator/Makefile.am | 3 +-
generator/OCaml.ml | 18 +-
generator/Python.ml | 20 +-
generator/state_machine.ml | 50 +-
generator/states-issue-command.c | 33 +-
.../states-newstyle-opt-extended-headers.c | 110 ++++
generator/states-newstyle-opt-starttls.c | 7 +-
.../states-newstyle-opt-structured-reply.c | 3 +-
generator/states-newstyle.c | 3 +
generator/states-reply-simple.c | 4 +-
generator/states-reply-structured.c | 279 ++++++---
generator/states-reply.c | 57 +-
lib/aio.c | 7 +-
lib/flags.c | 11 +
lib/handle.c | 25 +-
lib/opt.c | 44 ++
lib/rw.c | 250 +++++++-
python/t/110-defaults.py | 1 +
python/t/120-set-non-defaults.py | 2 +
python/t/465-block-status-64.py | 56 ++
ocaml/examples/Makefile.am | 3 +-
ocaml/examples/extents64.ml | 42 ++
ocaml/helpers.c | 20 +
ocaml/nbd-c.h | 3 +-
ocaml/tests/Makefile.am | 1 +
ocaml/tests/test_110_defaults.ml | 2 +
ocaml/tests/test_120_set_non_defaults.ml | 3 +
ocaml/tests/test_465_block_status_64.ml | 58 ++
tests/Makefile.am | 4 +
tests/meta-base-allocation.c | 111 +++-
tests/pwrite-extended.c | 112 ++++
examples/copy-libev.c | 21 +-
examples/server-flags.c | 7 +-
interop/Makefile.am | 18 +
interop/block-status-payload.c | 241 ++++++++
interop/block-status-payload.sh | 80 +++
interop/large-status.c | 186 ++++++
interop/large-status.sh | 49 ++
interop/opt-extended-headers.c | 153 +++++
interop/opt-extended-headers.sh | 29 +
.gitignore | 4 +
copy/nbd-ops.c | 22 +-
dump/dump.c | 27 +-
fuzzing/libnbd-fuzz-wrapper.c | 22 +-
golang/Makefile.am | 1 +
golang/handle.go | 8 +-
golang/libnbd_110_defaults_test.go | 8 +
golang/libnbd_120_set_non_defaults_test.go | 12 +
golang/libnbd_465_block_status_64_test.go | 119 ++++
info/can.c | 14 +
info/info-can.sh | 30 +
info/info-packets.sh | 17 +-
info/main.c | 7 +-
info/map.c | 67 +--
info/show.c | 9 +-
64 files changed, 2920 insertions(+), 357 deletions(-)
create mode 100644 generator/states-newstyle-opt-extended-headers.c
create mode 100644 python/t/465-block-status-64.py
create mode 100644 ocaml/examples/extents64.ml
create mode 100644 ocaml/tests/test_465_block_status_64.ml
create mode 100644 tests/pwrite-extended.c
create mode 100644 interop/block-status-payload.c
create mode 100755 interop/block-status-payload.sh
create mode 100644 interop/large-status.c
create mode 100755 interop/large-status.sh
create mode 100644 interop/opt-extended-headers.c
create mode 100755 interop/opt-extended-headers.sh
create mode 100644 golang/libnbd_465_block_status_64_test.go
--
2.38.1
next prev parent reply other threads:[~2022-11-15 0:14 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-14 22:41 [cross-project PATCH v2] NBD 64-bit extensions Eric Blake
2022-11-14 22:46 ` [PATCH v2 0/6] NBD spec changes for " Eric Blake
2022-11-14 22:46 ` [PATCH v2 1/6] spec: Recommend cap on NBD_REPLY_TYPE_BLOCK_STATUS length Eric Blake
2022-12-16 19:32 ` Vladimir Sementsov-Ogievskiy
2023-03-03 22:17 ` Eric Blake
2023-03-05 8:41 ` Wouter Verhelst
2023-03-06 8:48 ` [Libguestfs] " Laszlo Ersek
2023-03-06 13:48 ` Nir Soffer
2022-11-14 22:46 ` [PATCH v2 2/6] spec: Tweak description of maximum block size Eric Blake
2022-12-16 20:22 ` Vladimir Sementsov-Ogievskiy
2023-03-03 22:20 ` Eric Blake
2023-02-21 15:21 ` Wouter Verhelst
2023-03-03 22:26 ` Eric Blake
2023-03-05 8:45 ` Wouter Verhelst
2022-11-14 22:46 ` [PATCH v2 3/6] spec: Add NBD_OPT_EXTENDED_HEADERS Eric Blake
2022-12-19 18:26 ` Vladimir Sementsov-Ogievskiy
2023-02-22 9:49 ` Wouter Verhelst
2023-03-03 22:36 ` Eric Blake
2023-03-05 8:49 ` Wouter Verhelst
2022-11-14 22:46 ` [PATCH v2 4/6] spec: Allow 64-bit block status results Eric Blake
2022-11-14 22:46 ` [PATCH v2 5/6] spec: Introduce NBD_FLAG_BLOCK_STATUS_PAYLOAD Eric Blake
2023-02-22 10:05 ` Wouter Verhelst
2023-03-03 22:40 ` Eric Blake
2023-03-05 8:50 ` Wouter Verhelst
2022-11-14 22:46 ` [PATCH v2 6/6] RFC: spec: Introduce NBD_REPLY_TYPE_OFFSET_HOLE_EXT Eric Blake
2022-11-14 22:48 ` [PATCH v2 00/15] qemu patches for 64-bit NBD extensions Eric Blake
2022-11-14 22:48 ` [PATCH v2 01/15] nbd/client: Add safety check on chunk payload length Eric Blake
2022-11-14 22:48 ` [PATCH v2 02/15] nbd/server: Prepare for alternate-size headers Eric Blake
2022-11-14 22:48 ` [PATCH v2 03/15] nbd: Prepare for 64-bit request effect lengths Eric Blake
2022-11-14 22:48 ` [PATCH v2 04/15] nbd: Add types for extended headers Eric Blake
2022-11-14 22:48 ` [PATCH v2 05/15] nbd/server: Refactor handling of request payload Eric Blake
2022-11-14 22:48 ` [PATCH v2 06/15] nbd/server: Refactor to pass full request around Eric Blake
2022-11-14 22:48 ` [PATCH v2 07/15] nbd/server: Initial support for extended headers Eric Blake
2022-11-14 22:48 ` [PATCH v2 08/15] nbd/server: Support 64-bit block status Eric Blake
2022-11-14 22:48 ` [PATCH v2 09/15] nbd/client: Initial support for extended headers Eric Blake
2022-11-14 22:48 ` [PATCH v2 10/15] nbd/client: Accept 64-bit block status chunks Eric Blake
2022-11-14 22:48 ` [PATCH v2 11/15] nbd/client: Request extended headers during negotiation Eric Blake
2022-11-14 22:48 ` [PATCH v2 12/15] nbd/server: Prepare for per-request filtering of BLOCK_STATUS Eric Blake
2022-11-14 22:48 ` [PATCH v2 13/15] nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS Eric Blake
2022-11-14 22:48 ` [PATCH v2 14/15] RFC: nbd/client: Accept 64-bit hole chunks Eric Blake
2022-11-14 22:48 ` [PATCH v2 15/15] RFC: nbd/server: Send 64-bit hole chunk Eric Blake
2022-11-14 22:51 ` Eric Blake [this message]
2022-11-14 22:51 ` [libnbd PATCH v2 01/23] block_status: Refactor array storage Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 02/23] internal: Refactor layout of replies in sbuf Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 03/23] protocol: Add definitions for extended headers Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 04/23] states: Prepare to send 64-bit requests Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 05/23] states: Prepare to receive 64-bit replies Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 06/23] states: Break deadlock if server goofs on extended replies Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 07/23] generator: Add struct nbd_extent in prep for 64-bit extents Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 08/23] block_status: Track 64-bit extents internally Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 09/23] block_status: Accept 64-bit extents during block status Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 10/23] api: Add [aio_]nbd_block_status_64 Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 11/23] api: Add several functions for controlling extended headers Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 12/23] copy: Update nbdcopy to use 64-bit block status Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 13/23] dump: Update nbddump " Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 14/23] info: Expose extended-headers support through nbdinfo Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 15/23] info: Update nbdinfo --map to use 64-bit block status Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 16/23] examples: Update copy-libev " Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 17/23] ocaml: Add example for 64-bit extents Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 18/23] generator: Actually request extended headers Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 19/23] api: Add nbd_[aio_]opt_extended_headers() Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 20/23] interop: Add test of 64-bit block status Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 21/23] api: Add nbd_can_block_status_payload() Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 22/23] api: Add nbd_[aio_]block_status_filter() Eric Blake
2022-11-14 22:51 ` [libnbd PATCH v2 23/23] RFC: pread: Accept 64-bit holes Eric Blake
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221114225158.2186742-1-eblake@redhat.com \
--to=eblake@redhat.com \
--cc=libguestfs@redhat.com \
--cc=nbd@other.debian.org \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.