qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v3 00/38] block: generic copy-on-read
@ 2011-11-23 11:44 Stefan Hajnoczi
  2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 01/38] Documentation: Add section about iSCSI LUNS to qemu-doc Stefan Hajnoczi
                   ` (14 more replies)
  0 siblings, 15 replies; 16+ messages in thread
From: Stefan Hajnoczi @ 2011-11-23 11:44 UTC (permalink / raw)
  To: qemu-devel; +Cc: Kevin Wolf, Paolo Bonzini, Marcelo Tosatti, Stefan Hajnoczi

The new -drive copy-on-read=on|off feature populates the image file with data
from the backing file on read.  This is useful when accessing images backed
over a slow medium (e.g. http over internet).  All read data will be stored in
the local image file so it does not need to be fetched again in the future.

This series is a prerequisite for the image streaming feature, which uses
copy-on-read to populate the image file in the background while the VM is
running.  However, the copy-on-read feature is useful on its own.

Copy-on-read is implemented by checking whether or not data is allocated in the
image file before reading it.  If data is not allocated then it needs to be
read and written back to the image file.

The tricky bit is avoiding races with other I/O requests.  These patches add
request tracking to BlockDriverState so that the list of pending requests is
available.  Copy-on-read prevents races by serializing overlapping requests.

Finally, there is a performance impact when enabling this feature since an
additional write is performed.  Serializing overlapping requests also means
that I/O patterns where multiple requests access the same cluster will see a
loss in parallelism.  Perhaps we can be smarter about preventing corruption in
the future and win back some performance.

v3:
 * Improve wait_for_overlapping_requests() comment [Kevin]

v2:
 * Based on bdrv_co_is_allocated patch series - now safe in coroutine context
 * Use QEMU_ALIGN_DOWN/UP() macros for copy-on-read cluster calculations [Zhi Yong]
 * Reset bs->copy_on_read on bdrv_close() [Kevin]
 * Refcount bs->copy_on_read so it doesn't get clobbered by multiple users [Marcelo]
 * Use bool instead of int where appropriate [Kevin]
 * Use compound literal assignment to ensure BdrvTrackedRequest fields always get zeroed [Kevin]
 * Comment rationale for copy-on-read bounce buffer [Kevin]

Dong Xu Wang (1):
  block:add coroutine_fn marker to coroutine functions

Li Zhi Hui (1):
  block: Use bdrv functions to replace file operation in cow.c

Paolo Bonzini (13):
  scsi: fix fw path
  scsi-disk: guess geometry
  atapi: kill MODE SENSE(6), fix MODE SENSE(10)
  scsi: update list of commands
  scsi: fix parsing of allocation length field
  scsi: remove block descriptors from CDs
  scsi: pass down REQUEST SENSE to the device when there is no stored
    sense
  scsi-block: always use SG_IO for MMC devices
  virtio-blk: fix cross-endian config space
  usb-msd: do not register twice in the boot order
  scsi: fix fw path
  scsi-generic: add as boot device
  xen_disk: remove dead code

Ronnie Sahlberg (1):
  Documentation: Add section about iSCSI LUNS to qemu-doc

Stefan Hajnoczi (17):
  block: use public bdrv_is_allocated() interface
  block: add .bdrv_co_is_allocated()
  qed: convert to .bdrv_co_is_allocated()
  block: convert qcow2, qcow2, and vmdk to .bdrv_co_is_allocated()
  vvfat: convert to .bdrv_co_is_allocated()
  vdi: convert to .bdrv_co_is_allocated()
  cow: convert to .bdrv_co_is_allocated()
  block: drop .bdrv_is_allocated() interface
  block: add bdrv_co_is_allocated() interface
  qemu-common: add QEMU_ALIGN_DOWN() and QEMU_ALIGN_UP() macros
  coroutine: add qemu_co_queue_restart_all()
  block: add request tracking
  block: add bdrv_set_copy_on_read()
  block: wait for overlapping requests
  block: request overlap detection
  block: core copy-on-read logic
  block: add -drive copy-on-read=on|off

Zhi Yong Wu (5):
  qed: adjust the way to get nb_sectors
  block: add the blockio limits command line support
  CoQueue: introduce qemu_co_queue_wait_insert_head
  block: add I/O throttling algorithm
  hmp/qmp: add block_set_io_throttle

 block.c               |  564 ++++++++++++++++++++++++++++++++++++++++++++++++-
 block.h               |   10 +
 block/cow.c           |   42 ++--
 block/qcow.c          |   12 +-
 block/qcow2.c         |   23 ++-
 block/qed-table.c     |    6 +-
 block/qed.c           |   15 +-
 block/sheepdog.c      |    4 +-
 block/vdi.c           |    6 +-
 block/vmdk.c          |    8 +-
 block/vvfat.c         |    4 +-
 block_int.h           |   40 ++++-
 blockdev.c            |  109 ++++++++++
 blockdev.h            |    2 +
 hmp-commands.hx       |   20 ++-
 hmp.c                 |   10 +
 hw/ide/atapi.c        |   20 +-
 hw/pci-hotplug.c      |    3 +-
 hw/scsi-bus.c         |  137 ++++++++++--
 hw/scsi-defs.h        |   10 +-
 hw/scsi-disk.c        |   37 +++-
 hw/scsi-generic.c     |    5 +
 hw/scsi.h             |    4 +-
 hw/usb-msd.c          |    4 +-
 hw/virtio-blk.c       |    7 +-
 hw/xen_disk.c         |   86 +--------
 qapi-schema.json      |   16 ++-
 qemu-common.h         |    6 +
 qemu-config.c         |   28 +++
 qemu-coroutine-lock.c |   23 ++-
 qemu-coroutine.h      |   11 +
 qemu-doc.texi         |   56 +++++
 qemu-options.hx       |   10 +-
 qerror.c              |    4 +
 qerror.h              |    3 +
 qmp-commands.hx       |   53 +++++-
 trace-events          |    1 +
 37 files changed, 1186 insertions(+), 213 deletions(-)

-- 
1.7.7.1

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2011-11-23 12:05 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-11-23 11:44 [Qemu-devel] [PATCH v3 00/38] block: generic copy-on-read Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 01/38] Documentation: Add section about iSCSI LUNS to qemu-doc Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 02/38] scsi: fix fw path Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 03/38] scsi-disk: guess geometry Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 04/38] atapi: kill MODE SENSE(6), fix MODE SENSE(10) Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 05/38] scsi: update list of commands Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 06/38] scsi: fix parsing of allocation length field Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 07/38] scsi: remove block descriptors from CDs Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 08/38] scsi: pass down REQUEST SENSE to the device when there is no stored sense Stefan Hajnoczi
2011-11-23 11:44 ` [Qemu-devel] [PATCH v3 09/38] scsi-block: always use SG_IO for MMC devices Stefan Hajnoczi
2011-11-23 11:45 ` [Qemu-devel] [PATCH v3 10/38] virtio-blk: fix cross-endian config space Stefan Hajnoczi
2011-11-23 11:45 ` [Qemu-devel] [PATCH v3 11/38] usb-msd: do not register twice in the boot order Stefan Hajnoczi
2011-11-23 11:45 ` [Qemu-devel] [PATCH v3 12/38] scsi: fix fw path Stefan Hajnoczi
2011-11-23 11:45 ` [Qemu-devel] [PATCH v3 13/38] scsi-generic: add as boot device Stefan Hajnoczi
2011-11-23 11:45 ` [Qemu-devel] [PATCH v3 14/38] qed: adjust the way to get nb_sectors Stefan Hajnoczi
2011-11-23 11:49 ` [Qemu-devel] [PATCH v3 00/38] block: generic copy-on-read Stefan Hajnoczi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).