[Qemu-devel] [PATCH v3 00/38] block: generic copy-on-read

[Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>]

The new -drive copy-on-read=on|off feature populates the image file with data
from the backing file on read.  This is useful when accessing images backed
over a slow medium (e.g. http over internet).  All read data will be stored in
the local image file so it does not need to be fetched again in the future.

This series is a prerequisite for the image streaming feature, which uses
copy-on-read to populate the image file in the background while the VM is
running.  However, the copy-on-read feature is useful on its own.

Copy-on-read is implemented by checking whether or not data is allocated in the
image file before reading it.  If data is not allocated then it needs to be
read and written back to the image file.

The tricky bit is avoiding races with other I/O requests.  These patches add
request tracking to BlockDriverState so that the list of pending requests is
available.  Copy-on-read prevents races by serializing overlapping requests.

Finally, there is a performance impact when enabling this feature since an
additional write is performed.  Serializing overlapping requests also means
that I/O patterns where multiple requests access the same cluster will see a
loss in parallelism.  Perhaps we can be smarter about preventing corruption in
the future and win back some performance.

v3:
 * Improve wait_for_overlapping_requests() comment [Kevin]

v2:
 * Based on bdrv_co_is_allocated patch series - now safe in coroutine context
 * Use QEMU_ALIGN_DOWN/UP() macros for copy-on-read cluster calculations [Zhi Yong]
 * Reset bs->copy_on_read on bdrv_close() [Kevin]
 * Refcount bs->copy_on_read so it doesn't get clobbered by multiple users [Marcelo]
 * Use bool instead of int where appropriate [Kevin]
 * Use compound literal assignment to ensure BdrvTrackedRequest fields always get zeroed [Kevin]
 * Comment rationale for copy-on-read bounce buffer [Kevin]

Dong Xu Wang (1):
  block:add coroutine_fn marker to coroutine functions

Li Zhi Hui (1):
  block: Use bdrv functions to replace file operation in cow.c

Paolo Bonzini (13):
  scsi: fix fw path
  scsi-disk: guess geometry
  atapi: kill MODE SENSE(6), fix MODE SENSE(10)
  scsi: update list of commands
  scsi: fix parsing of allocation length field
  scsi: remove block descriptors from CDs
  scsi: pass down REQUEST SENSE to the device when there is no stored
    sense
  scsi-block: always use SG_IO for MMC devices
  virtio-blk: fix cross-endian config space
  usb-msd: do not register twice in the boot order
  scsi: fix fw path
  scsi-generic: add as boot device
  xen_disk: remove dead code

Ronnie Sahlberg (1):
  Documentation: Add section about iSCSI LUNS to qemu-doc

Stefan Hajnoczi (17):
  block: use public bdrv_is_allocated() interface
  block: add .bdrv_co_is_allocated()
  qed: convert to .bdrv_co_is_allocated()
  block: convert qcow2, qcow2, and vmdk to .bdrv_co_is_allocated()
  vvfat: convert to .bdrv_co_is_allocated()
  vdi: convert to .bdrv_co_is_allocated()
  cow: convert to .bdrv_co_is_allocated()
  block: drop .bdrv_is_allocated() interface
  block: add bdrv_co_is_allocated() interface
  qemu-common: add QEMU_ALIGN_DOWN() and QEMU_ALIGN_UP() macros
  coroutine: add qemu_co_queue_restart_all()
  block: add request tracking
  block: add bdrv_set_copy_on_read()
  block: wait for overlapping requests
  block: request overlap detection
  block: core copy-on-read logic
  block: add -drive copy-on-read=on|off

Zhi Yong Wu (5):
  qed: adjust the way to get nb_sectors
  block: add the blockio limits command line support
  CoQueue: introduce qemu_co_queue_wait_insert_head
  block: add I/O throttling algorithm
  hmp/qmp: add block_set_io_throttle

 block.c               |  564 ++++++++++++++++++++++++++++++++++++++++++++++++-
 block.h               |   10 +
 block/cow.c           |   42 ++--
 block/qcow.c          |   12 +-
 block/qcow2.c         |   23 ++-
 block/qed-table.c     |    6 +-
 block/qed.c           |   15 +-
 block/sheepdog.c      |    4 +-
 block/vdi.c           |    6 +-
 block/vmdk.c          |    8 +-
 block/vvfat.c         |    4 +-
 block_int.h           |   40 ++++-
 blockdev.c            |  109 ++++++++++
 blockdev.h            |    2 +
 hmp-commands.hx       |   20 ++-
 hmp.c                 |   10 +
 hw/ide/atapi.c        |   20 +-
 hw/pci-hotplug.c      |    3 +-
 hw/scsi-bus.c         |  137 ++++++++++--
 hw/scsi-defs.h        |   10 +-
 hw/scsi-disk.c        |   37 +++-
 hw/scsi-generic.c     |    5 +
 hw/scsi.h             |    4 +-
 hw/usb-msd.c          |    4 +-
 hw/virtio-blk.c       |    7 +-
 hw/xen_disk.c         |   86 +--------
 qapi-schema.json      |   16 ++-
 qemu-common.h         |    6 +
 qemu-config.c         |   28 +++
 qemu-coroutine-lock.c |   23 ++-
 qemu-coroutine.h      |   11 +
 qemu-doc.texi         |   56 +++++
 qemu-options.hx       |   10 +-
 qerror.c              |    4 +
 qerror.h              |    3 +
 qmp-commands.hx       |   53 +++++-
 trace-events          |    1 +
 37 files changed, 1186 insertions(+), 213 deletions(-)

-- 
1.7.7.1