[Qemu-devel] [PATCH v3 00/44] NBD protocol additions

[Eric Blake <eblake@redhat.com>]

This series is for qemu 2.7, and is a bit more stable this
time (upstream NBD extensions have been reaching some consensus
based on feedback I've made while implementing this series).

Included are some interoperability bug fixes, code cleanups, then
added support both client-side and server-side for:
NBD_FLAG_C_NO_ZEROES
NBD_CMD_WRITE_ZEROES
NBD_OPT_INFO
NBD_OPT_GO
NBD_OPT_BLOCK_SIZE

Still to come:
- support for NBD_OPT_STRUCTURED_REPLY
- strawman implementations to help with discussions towards
 NBD_CMD_BLOCK_STATUS

This posting is tied to this particular version of the NBD protocol:
https://github.com/yoe/nbd/blob/772b264/doc/proto.md
plus these two extensions:
https://github.com/yoe/nbd/blob/extension-info/doc/proto.md
https://github.com/yoe/nbd/blob/extension-write-zeroes/doc/proto.md

I performed testing by temporarily turning on DEBUG_NBD while
compiling, then connecting variations on:
  ./qemu-nbd -f raw -x foo [-D string] file
  ./qemu-io -t none -f raw nbd://localhost:10809/foo
and watching the traces on both screen (both for startup negotiation,
and for various 'write -z [-u]', 'write -f [-u]', 'discard', and 'q'
commands in qemu-io.  I intentionally tested all three combinations of:
  old client, new server
  new client, old server
  new client, new server
at various phases of the series, to make sure that either side
gracefully handles unknown advertisements when the other side is
newer, and correctly falls back to older usage when the other side
is too old.

I'm posting now so that others may compile my work and help with
cross-project testing (such as qemu client to Alex's NBDGO
server), which in turn will help us move experimental extensions
into final form in the NBD protocol.

Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git nbd-flags-v3

Comparison to v2:
001/44:[----] [-C] 'nbd: More debug typo fixes, use correct formats'
002/44:[down] 'nbd: Quit server after any write error'
003/44:[down] 'nbd: Improve server handling of bogus commands'
004/44:[----] [-C] 'nbd: Reject unknown request flags'
005/44:[down] 'nbd: Group all Linux-specific ioctl code in one place'
006/44:[down] 'nbd: Clean up ioctl handling of qemu-nbd -c'
007/44:[0036] [FC] 'nbd: Limit nbdflags to 16 bits'
008/44:[down] 'nbd: Add qemu-nbd -D for human-readable description'
009/44:[down] 'block: Allow BDRV_REQ_FUA through blk_pwrite()'
010/44:[down] 'fdc: Switch to byte-based block access'
011/44:[down] 'nand: Switch to byte-based block access'
012/44:[down] 'onenand: Switch to byte-based block access'
013/44:[down] 'pflash: Switch to byte-based block access'
014/44:[down] 'sd: Switch to byte-based block access'
015/44:[down] 'm25p80: Switch to byte-based block access'
016/44:[down] 'atapi: Switch to byte-based block access'
017/44:[down] 'nbd: Switch to byte-based block access'
018/44:[down] 'qemu-img: Switch to byte-based block access'
019/44:[down] 'qemu-io: Switch to byte-based block access'
020/44:[down] 'block: Switch blk_read_unthrottled() to byte interface'
021/44:[down] 'block: Switch blk_write_zeroes() to byte interface'
022/44:[down] 'block: Kill blk_write(), blk_read()'
023/44:[down] 'qemu-io: Add missing option documentation'
024/44:[down] 'qemu-io: Add 'write -f' to test FUA flag'
025/44:[down] 'qemu-io: Add 'open -u' to set BDRV_O_UNMAP after the fact'
026/44:[down] 'qemu-io: Add 'write -z -u' to test MAY_UNMAP flag'
027/44:[down] 'nbd: Use BDRV_REQ_FUA for better FUA where supported'
028/44:[----] [--] 'nbd: Detect servers that send unexpected error values'
029/44:[0008] [FC] 'nbd: Avoid magic number for NBD max name size'
030/44:[0026] [FC] 'nbd: Treat flags vs. command type as separate fields'
031/44:[0025] [FC] 'nbd: Share common reply-sending code in server'
032/44:[0002] [FC] 'nbd: Share common option-sending code in client'
033/44:[----] [-C] 'nbd: Let client skip portions of server reply'
034/44:[----] [--] 'nbd: Less allocation during NBD_OPT_LIST'
035/44:[----] [-C] 'nbd: Support shorter handshake'
036/44:[down] 'nbd: Improve handling of shutdown requests'
037/44:[down] 'nbd: Create struct for tracking export info'
038/44:[down] 'block: Add blk_get_opt_transfer_length()'
039/44:[0065] [FC] 'nbd: Implement NBD_OPT_GO on server'
040/44:[0139] [FC] 'nbd: Implement NBD_OPT_GO on client'
041/44:[0043] [FC] 'nbd: Implement NBD_CMD_WRITE_ZEROES on server'
042/44:[0005] [FC] 'nbd: Implement NBD_CMD_WRITE_ZEROES on client'
043/44:[down] 'nbd: Implement NBD_OPT_BLOCK_SIZE on server'
044/44:[down] 'nbd: Implement NBD_OPT_BLOCK_SIZE on client'

Eric Blake (44):
  nbd: More debug typo fixes, use correct formats
  nbd: Quit server after any write error
  nbd: Improve server handling of bogus commands
  nbd: Reject unknown request flags
  nbd: Group all Linux-specific ioctl code in one place
  nbd: Clean up ioctl handling of qemu-nbd -c
  nbd: Limit nbdflags to 16 bits
  nbd: Add qemu-nbd -D for human-readable description
  block: Allow BDRV_REQ_FUA through blk_pwrite()
  fdc: Switch to byte-based block access
  nand: Switch to byte-based block access
  onenand: Switch to byte-based block access
  pflash: Switch to byte-based block access
  sd: Switch to byte-based block access
  m25p80: Switch to byte-based block access
  atapi: Switch to byte-based block access
  nbd: Switch to byte-based block access
  qemu-img: Switch to byte-based block access
  qemu-io: Switch to byte-based block access
  block: Switch blk_read_unthrottled() to byte interface
  block: Switch blk_write_zeroes() to byte interface
  block: Kill blk_write(), blk_read()
  qemu-io: Add missing option documentation
  qemu-io: Add 'write -f' to test FUA flag
  qemu-io: Add 'open -u' to set BDRV_O_UNMAP after the fact
  qemu-io: Add 'write -z -u' to test MAY_UNMAP flag
  nbd: Use BDRV_REQ_FUA for better FUA where supported
  nbd: Detect servers that send unexpected error values
  nbd: Avoid magic number for NBD max name size
  nbd: Treat flags vs. command type as separate fields
  nbd: Share common reply-sending code in server
  nbd: Share common option-sending code in client
  nbd: Let client skip portions of server reply
  nbd: Less allocation during NBD_OPT_LIST
  nbd: Support shorter handshake
  nbd: Improve handling of shutdown requests
  nbd: Create struct for tracking export info
  block: Add blk_get_opt_transfer_length()
  nbd: Implement NBD_OPT_GO on server
  nbd: Implement NBD_OPT_GO on client
  nbd: Implement NBD_CMD_WRITE_ZEROES on server
  nbd: Implement NBD_CMD_WRITE_ZEROES on client
  nbd: Implement NBD_OPT_BLOCK_SIZE on server
  nbd: Implement NBD_OPT_BLOCK_SIZE on client

 block/nbd-client.h             |   5 +-
 include/block/nbd.h            | 101 ++++--
 include/sysemu/block-backend.h |  16 +-
 nbd/nbd-internal.h             |  18 +-
 block/block-backend.c          |  58 ++--
 block/crypto.c                 |   2 +-
 block/nbd-client.c             |  56 +++-
 block/nbd.c                    |  41 ++-
 block/parallels.c              |   5 +-
 block/qcow.c                   |   8 +-
 block/qcow2.c                  |   4 +-
 block/qed.c                    |   6 +-
 block/sheepdog.c               |   2 +-
 block/vdi.c                    |   4 +-
 block/vhdx.c                   |   5 +-
 block/vmdk.c                   |  10 +-
 block/vpc.c                    |  10 +-
 hw/block/fdc.c                 |  25 +-
 hw/block/hd-geometry.c         |   2 +-
 hw/block/m25p80.c              |   3 +-
 hw/block/nand.c                |  36 ++-
 hw/block/onenand.c             |  36 ++-
 hw/block/pflash_cfi01.c        |  12 +-
 hw/block/pflash_cfi02.c        |  12 +-
 hw/ide/atapi.c                 |   8 +-
 hw/nvram/spapr_nvram.c         |   4 +-
 hw/sd/sd.c                     |  46 +--
 nbd/client.c                   | 717 ++++++++++++++++++++++++++---------------
 nbd/server.c                   | 552 +++++++++++++++++++++++--------
 qemu-img.c                     |  31 +-
 qemu-io-cmds.c                 | 114 +++----
 qemu-io.c                      |  12 +-
 qemu-nbd.c                     |  35 +-
 qemu-nbd.texi                  |   5 +-
 34 files changed, 1307 insertions(+), 694 deletions(-)

-- 
2.5.5