[Qemu-devel] [PULL 00/10] Block patches

[Qemu-devel] [PULL 00/10] Block patches

[Kevin Wolf]

The following changes since commit 02a89b219039621c940863aa5a9da4fec81a1546:

  isapc: fix segfault. (2010-08-28 08:50:40 +0000)

are available in the git repository at:
  git://repo.or.cz/qemu/kevin.git for-anthony

Andrew de Quincey (1):
      posix-aio-compat: Fix async_conmtext for ioctl

Izumi Tsutsui (1):
      sheepdog: remove unnecessary includes

Kevin Wolf (4):
      virtio: Factor virtqueue_map_sg out
      virtio-blk: Fix migration of queued requests
      block: Fix image re-open in bdrv_commit
      qemu-img rebase: Open new backing file read-only

Laurent Vivier (1):
      nbd: Introduce NBD named exports.

Loïc Minier (1):
      vvfat: fat_chksum(): fix access above array bounds

Miguel Di Ciurcio Filho (2):
      monitor: make 'info snapshots' show only fully available snapshots
      savevm: Generate a name when run without one

 block.c            |   13 ++++--
 block/nbd.c        |   68 +++++++++++++++++++++--------
 block/sheepdog.c   |   10 ----
 block/vvfat.c      |    2 +-
 hw/virtio-blk.c    |    5 ++
 hw/virtio.c        |   38 +++++++++++------
 hw/virtio.h        |    3 +
 nbd.c              |  118 ++++++++++++++++++++++++++++++++++++++++++++++-----
 nbd.h              |    5 ++-
 posix-aio-compat.c |    1 +
 qemu-doc.texi      |    7 +++
 qemu-img.c         |    2 +-
 qemu-nbd.c         |    4 +-
 savevm.c           |   88 ++++++++++++++++++++++++++++-----------
 14 files changed, 276 insertions(+), 88 deletions(-)

Re: [Qemu-devel] [PULL 00/10] Block patches

[Kevin Wolf]

Am 30.08.2010 18:32, schrieb Kevin Wolf:
> The following changes since commit 02a89b219039621c940863aa5a9da4fec81a1546:
> 
>   isapc: fix segfault. (2010-08-28 08:50:40 +0000)
> 
> are available in the git repository at:
>   git://repo.or.cz/qemu/kevin.git for-anthony
> 
> Andrew de Quincey (1):
>       posix-aio-compat: Fix async_conmtext for ioctl
> 
> Izumi Tsutsui (1):
>       sheepdog: remove unnecessary includes
> 
> Kevin Wolf (4):
>       virtio: Factor virtqueue_map_sg out
>       virtio-blk: Fix migration of queued requests
>       block: Fix image re-open in bdrv_commit
>       qemu-img rebase: Open new backing file read-only
> 
> Laurent Vivier (1):
>       nbd: Introduce NBD named exports.
> 
> Loïc Minier (1):
>       vvfat: fat_chksum(): fix access above array bounds
> 
> Miguel Di Ciurcio Filho (2):
>       monitor: make 'info snapshots' show only fully available snapshots
>       savevm: Generate a name when run without one

Anthony, I think you already pulled the patches for stable-0.13 that I
sent on the same day, but these patches for master still seem to be missing.

Kevin

[Qemu-devel] [PULL 00/10] Block patches

[Kevin Wolf]

The following changes since commit 5fc9cfedfa09199e10b5f9b67dcd286bfeae4f7a:

  Fold send_all() wrapper unix_write() into one function (2010-11-03 12:48:09 -0500)

are available in the git repository at:
  git://repo.or.cz/qemu/kevin.git for-anthony

Blue Swirl (1):
      block: avoid a warning on 64 bit hosts with long as int64_t

Kevin Wolf (9):
      scsi-disk: Implement rerror option
      block: Allow bdrv_flush to return errors
      scsi-disk: Complete failed requests in scsi_disk_emulate_command
      scsi-disk: Implement werror for flushes
      vpc: Implement bdrv_flush
      qcow2: Invalidate cache after failed read
      ide: Handle immediate bdrv_aio_flush failure
      virtio-blk: Handle immediate flush failure properly
      scsi-disk: Fix immediate failure of bdrv_aio_*

 block.c                |   21 ++++++-
 block.h                |    2 +-
 block/blkdebug.c       |    4 +-
 block/blkverify.c      |    8 +-
 block/cow.c            |    4 +-
 block/qcow.c           |    4 +-
 block/qcow2-cluster.c  |    1 +
 block/qcow2-refcount.c |    1 +
 block/qcow2.c          |    4 +-
 block/raw-posix.c      |    4 +-
 block/raw-win32.c      |    9 +++-
 block/raw.c            |    4 +-
 block/vdi.c            |    4 +-
 block/vmdk.c           |    4 +-
 block/vpc.c            |   21 ++++---
 block_int.h            |    2 +-
 blockdev.c             |    6 +-
 hw/ide/core.c          |   12 +++-
 hw/scsi-disk.c         |  147 +++++++++++++++++++++++++++++++----------------
 hw/virtio-blk.c        |    2 +-
 20 files changed, 172 insertions(+), 92 deletions(-)

[Qemu-devel] [PULL 00/10] Block patches

[Kevin Wolf]

The following changes since commit 9df38c47d01eb1fd7eb9d60ac70a4170e638b4a2:

  target-arm: Detect tininess before rounding for FP operations (2011-04-12 23:33:33 +0200)

are available in the git repository at:
  git://repo.or.cz/qemu/kevin.git for-anthony

Amit Shah (7):
      atapi: Drives can be locked without media present
      atapi: Report correct errors on guest eject request
      atapi: Allow GET_EVENT_STATUS_NOTIFICATION after media change
      atapi: Move GET_EVENT_STATUS_NOTIFICATION command handling to its own function
      atapi: GESN: Use structs for commonly-used field types
      atapi: GESN: Standardise event response handling for future additions
      atapi: GESN: implement 'media' subcommand

Anthony Liguori (1):
      qed: Add support for zero clusters

Mitnick Lyu (1):
      vpc.c: Use get_option_parameter() does the search

Stefan Hajnoczi (1):
      docs: Describe zero data clusters in QED specification

 block/qed-check.c       |    5 +-
 block/qed-cluster.c     |   31 +++++---
 block/qed.c             |   21 ++++-
 block/qed.h             |   26 ++++++
 block/vpc.c             |    8 +--
 docs/specs/qed_spec.txt |    8 ++
 hw/ide/core.c           |  204 ++++++++++++++++++++++++++++++++++++++++-------
 hw/ide/internal.h       |    6 ++
 8 files changed, 258 insertions(+), 51 deletions(-)

[Qemu-devel] [PULL 00/10] Block patches

[Kevin Wolf]

The following changes since commit 03834e22abafbc8dc4052d46a5ccd6dd135a54a3:

  Merge remote-tracking branch 'origin/master' into staging (2012-08-14 15:19:50 -0500)

are available in the git repository at:

  http://repo.or.cz/r/qemu/kevin.git for-anthony

Corey Bryant (7):
      qemu-char: Add MSG_CMSG_CLOEXEC flag to recvmsg
      qapi: Introduce add-fd, remove-fd, query-fdsets
      block: Prevent detection of /dev/fdset/ as floppy
      block: Convert open calls to qemu_open
      block: Convert close calls to qemu_close
      block: Enable qemu_open/close to work with fd sets
      monitor: Clean up fd sets on monitor disconnect

Kevin Wolf (2):
      block: Flush parent to OS with cache=unsafe
      qemu-iotests: Fix 030 after switch to GenericError

Stefan Priebe (1):
      iscsi: Fix NULL dereferences / races between task completion and abort

 Makefile               |    6 +-
 block.c                |    3 +-
 block/iscsi.c          |   55 ++++-----
 block/raw-posix.c      |   46 ++++----
 block/raw-win32.c      |    6 +-
 block/vdi.c            |    5 +-
 block/vmdk.c           |   25 ++---
 block/vpc.c            |    4 +-
 block/vvfat.c          |   16 ++--
 cutils.c               |    5 +
 monitor.c              |  294 ++++++++++++++++++++++++++++++++++++++++++++++++
 monitor.h              |    5 +
 osdep.c                |  116 +++++++++++++++++++
 qapi-schema.json       |   98 ++++++++++++++++
 qemu-char.c            |   12 ++-
 qemu-common.h          |    2 +
 qemu-tool.c            |   20 ++++
 qemu-user.c            |   20 ++++
 qmp-commands.hx        |  122 ++++++++++++++++++++
 savevm.c               |    4 +-
 tests/Makefile         |    2 +-
 tests/qemu-iotests/030 |    6 +-
 22 files changed, 776 insertions(+), 96 deletions(-)

Re: [Qemu-devel] [PULL 00/10] Block patches

[Anthony Liguori]

Kevin Wolf <kwolf@redhat.com> writes:

> The following changes since commit 03834e22abafbc8dc4052d46a5ccd6dd135a54a3:
>
>   Merge remote-tracking branch 'origin/master' into staging (2012-08-14 15:19:50 -0500)
>
> are available in the git repository at:
>
>   http://repo.or.cz/r/qemu/kevin.git for-anthony
>

Pulled. Thanks.

Regards,

Anthony Liguori

> Corey Bryant (7):
>       qemu-char: Add MSG_CMSG_CLOEXEC flag to recvmsg
>       qapi: Introduce add-fd, remove-fd, query-fdsets
>       block: Prevent detection of /dev/fdset/ as floppy
>       block: Convert open calls to qemu_open
>       block: Convert close calls to qemu_close
>       block: Enable qemu_open/close to work with fd sets
>       monitor: Clean up fd sets on monitor disconnect
>
> Kevin Wolf (2):
>       block: Flush parent to OS with cache=unsafe
>       qemu-iotests: Fix 030 after switch to GenericError
>
> Stefan Priebe (1):
>       iscsi: Fix NULL dereferences / races between task completion and abort
>
>  Makefile               |    6 +-
>  block.c                |    3 +-
>  block/iscsi.c          |   55 ++++-----
>  block/raw-posix.c      |   46 ++++----
>  block/raw-win32.c      |    6 +-
>  block/vdi.c            |    5 +-
>  block/vmdk.c           |   25 ++---
>  block/vpc.c            |    4 +-
>  block/vvfat.c          |   16 ++--
>  cutils.c               |    5 +
>  monitor.c              |  294 ++++++++++++++++++++++++++++++++++++++++++++++++
>  monitor.h              |    5 +
>  osdep.c                |  116 +++++++++++++++++++
>  qapi-schema.json       |   98 ++++++++++++++++
>  qemu-char.c            |   12 ++-
>  qemu-common.h          |    2 +
>  qemu-tool.c            |   20 ++++
>  qemu-user.c            |   20 ++++
>  qmp-commands.hx        |  122 ++++++++++++++++++++
>  savevm.c               |    4 +-
>  tests/Makefile         |    2 +-
>  tests/qemu-iotests/030 |    6 +-
>  22 files changed, 776 insertions(+), 96 deletions(-)

[Qemu-devel] [PULL 00/10] Block patches

[Kevin Wolf]

The following changes since commit 171392406d8e230d62e5ebf4805f71460854b8ec:

  gtk: don't use g_object_unref on GdkCursor (2013-06-03 16:14:05 -0500)

are available in the git repository at:

  git://repo.or.cz/qemu/kevin.git for-anthony

for you to fetch changes up to 5b91704469c0f801e0219f26458356872c4145ab:

  block: dump snapshot and image info to specified output (2013-06-04 13:56:30 +0200)

----------------------------------------------------------------
Fam Zheng (1):
      block: add block driver read only whitelist

Stefan Hajnoczi (6):
      qemu-iotests: fix 054 cluster size help output
      qemu-iotests: make assert_no_active_block_jobs() common
      qemu-iotests: make cancel_and_wait() common
      qemu-iotests: make compare_images() common
      qemu-iotests: make create_image() common
      block: drop bs_snapshots global variable

Wenchao Xia (3):
      block: move snapshot code in block.c to block/snapshot.c
      block: move qmp and info dump related code to block/qapi.c
      block: dump snapshot and image info to specified output

 block.c                       | 356 ++++------------------------------------
 block/Makefile.objs           |   1 +
 block/qapi.c                  | 366 ++++++++++++++++++++++++++++++++++++++++++
 block/snapshot.c              | 157 ++++++++++++++++++
 blockdev.c                    |   4 +-
 configure                     |  20 ++-
 hw/block/xen_disk.c           |   8 +-
 include/block/block.h         |  31 +---
 include/block/block_int.h     |   1 +
 include/block/qapi.h          |  43 +++++
 include/block/snapshot.h      |  53 ++++++
 qemu-img.c                    | 163 +------------------
 savevm.c                      |  40 ++---
 scripts/create_config         |  11 +-
 tests/qemu-iotests/030        |  99 ++++--------
 tests/qemu-iotests/041        | 181 ++++++++-------------
 tests/qemu-iotests/054.out    |   2 +-
 tests/qemu-iotests/iotests.py |  38 +++++
 18 files changed, 843 insertions(+), 731 deletions(-)
 create mode 100644 block/qapi.c
 create mode 100644 block/snapshot.c
 create mode 100644 include/block/qapi.h
 create mode 100644 include/block/snapshot.h

Re: [Qemu-devel] [PULL 00/10] Block patches

[Anthony Liguori]

Pulled.  Thanks.

Regards,

Anthony Liguori

[Qemu-devel] [PULL 00/10] Block patches

[Stefan Hajnoczi]

The following changes since commit d8e3b729cf452d2689c8669f1ec18158db29fd5a:

  Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2015-06-11 15:33:38 +0100)

are available in the git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 2db33f88d2b340c049c576ad75d442e4b6ffe768:

  qemu-iotests: expand test 093 to support group throttling (2015-06-12 14:00:00 +0100)

----------------------------------------------------------------

----------------------------------------------------------------

Alberto Garcia (7):
  throttle: Add throttle group infrastructure
  throttle: Add throttle group infrastructure tests
  throttle: Add throttle group support
  throttle: acquire the ThrottleGroup lock in bdrv_swap()
  throttle: add the name of the ThrottleGroup to BlockDeviceInfo
  throttle: Update throttle infrastructure copyright
  qemu-iotests: expand test 093 to support group throttling

Benoît Canet (1):
  throttle: Extract timers from ThrottleState into a separate structure

Kevin Wolf (1):
  raw-posix: Fix .bdrv_co_get_block_status() for unaligned image size

Stefan Hajnoczi (1):
  Revert "iothread: release iothread around aio_poll"

 async.c                         |   8 +-
 block.c                         |  38 ++-
 block/Makefile.objs             |   1 +
 block/io.c                      |  71 ++----
 block/qapi.c                    |   8 +-
 block/raw-posix.c               |   5 +-
 block/throttle-groups.c         | 496 ++++++++++++++++++++++++++++++++++++++++
 blockdev.c                      |  38 ++-
 hmp.c                           |  10 +-
 include/block/block.h           |   3 +-
 include/block/block_int.h       |   7 +-
 include/block/throttle-groups.h |  46 ++++
 include/qemu/throttle.h         |  46 ++--
 iothread.c                      |  11 +-
 qapi/block-core.json            |  29 ++-
 qemu-options.hx                 |   1 +
 qmp-commands.hx                 |   3 +-
 tests/qemu-iotests/093          |  89 ++++---
 tests/test-aio.c                |  19 +-
 tests/test-throttle.c           | 163 +++++++++----
 util/throttle.c                 |  81 ++++---
 21 files changed, 954 insertions(+), 219 deletions(-)
 create mode 100644 block/throttle-groups.c
 create mode 100644 include/block/throttle-groups.h

-- 
2.4.2

Re: [Qemu-devel] [PULL 00/10] Block patches

[Peter Maydell]

On 12 June 2015 at 15:57, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit d8e3b729cf452d2689c8669f1ec18158db29fd5a:
>
>   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2015-06-11 15:33:38 +0100)
>
> are available in the git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 2db33f88d2b340c049c576ad75d442e4b6ffe768:
>
>   qemu-iotests: expand test 093 to support group throttling (2015-06-12 14:00:00 +0100)
>

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 00/10] Block patches

[Stefan Hajnoczi]

The following changes since commit c688084506cf2cf2eba4ba9df4e91abb6e3dab83:

  Merge remote-tracking branch 'remotes/berrange/tags/pull-qcrypto-secrets-base-2015-12-18-1' into staging (2015-12-18 17:04:15 +0000)

are available in the git repository at:

  git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 723697551a7e926abe7d3c7f2966012b8075143d:

  sdhci: add optional quirk property to disable card insertion/removal interrupts (2015-12-22 16:34:26 +0800)

----------------------------------------------------------------

----------------------------------------------------------------

Andrew Baumann (2):
  sdhci: don't raise a command index error for an unexpected response
  sdhci: add optional quirk property to disable card insertion/removal
    interrupts

Gonglei (1):
  virtio-blk: trivial code optimization

Peter Crosthwaite (1):
  sd: sdhci: Delete over-zealous power check

Stefan Hajnoczi (4):
  block: add BlockLimits.max_iov field
  block-backend: add blk_get_max_iov()
  block: replace IOV_MAX with BlockLimits.max_iov
  block/mirror: replace IOV_MAX with blk_get_max_iov()

Vladimir Sementsov-Ogievskiy (1):
  parallels: add format spec

Yang Wei (1):
  scripts/gdb: Fix a python exception in mtree.py

 MAINTAINERS                    |   1 +
 block/block-backend.c          |   5 +
 block/io.c                     |  10 +-
 block/mirror.c                 |   6 +-
 docs/specs/parallels.txt       | 228 +++++++++++++++++++++++++++++++++++++++++
 hw/block/virtio-blk.c          |  28 ++---
 hw/sd/sdhci.c                  |  10 +-
 include/block/block_int.h      |   3 +
 include/hw/sd/sdhci.h          |   1 +
 include/sysemu/block-backend.h |   1 +
 scripts/qemugdb/mtree.py       |  10 +-
 11 files changed, 272 insertions(+), 31 deletions(-)
 create mode 100644 docs/specs/parallels.txt

-- 
2.5.0

Re: [Qemu-devel] [PULL 00/10] Block patches

[Peter Maydell]

On 22 December 2015 at 08:54, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit c688084506cf2cf2eba4ba9df4e91abb6e3dab83:
>
>   Merge remote-tracking branch 'remotes/berrange/tags/pull-qcrypto-secrets-base-2015-12-18-1' into staging (2015-12-18 17:04:15 +0000)
>
> are available in the git repository at:
>
>   git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 723697551a7e926abe7d3c7f2966012b8075143d:
>
>   sdhci: add optional quirk property to disable card insertion/removal interrupts (2015-12-22 16:34:26 +0800)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Andrew Baumann (2):
>   sdhci: don't raise a command index error for an unexpected response
>   sdhci: add optional quirk property to disable card insertion/removal
>     interrupts
>
> Gonglei (1):
>   virtio-blk: trivial code optimization
>
> Peter Crosthwaite (1):
>   sd: sdhci: Delete over-zealous power check
>
> Stefan Hajnoczi (4):
>   block: add BlockLimits.max_iov field
>   block-backend: add blk_get_max_iov()
>   block: replace IOV_MAX with BlockLimits.max_iov
>   block/mirror: replace IOV_MAX with blk_get_max_iov()
>
> Vladimir Sementsov-Ogievskiy (1):
>   parallels: add format spec
>
> Yang Wei (1):
>   scripts/gdb: Fix a python exception in mtree.py
>
>  MAINTAINERS                    |   1 +
>  block/block-backend.c          |   5 +
>  block/io.c                     |  10 +-
>  block/mirror.c                 |   6 +-
>  docs/specs/parallels.txt       | 228 +++++++++++++++++++++++++++++++++++++++++
>  hw/block/virtio-blk.c          |  28 ++---
>  hw/sd/sdhci.c                  |  10 +-
>  include/block/block_int.h      |   3 +
>  include/hw/sd/sdhci.h          |   1 +
>  include/sysemu/block-backend.h |   1 +
>  scripts/qemugdb/mtree.py       |  10 +-
>  11 files changed, 272 insertions(+), 31 deletions(-)
>  create mode 100644 docs/specs/parallels.txt

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 00/10] Block patches

[Jeff Cody]

The following changes since commit d7f30403576f04f1f3a5fb5a1d18cba8dfa7a6d2:

  cputlb: don't cpu_abort() if guest tries to execute outside RAM or RAM (2016-06-28 18:50:53 +0100)

are available in the git repository at:

  git@github.com:codyprime/qemu-kvm-jtc.git tags/block-pull-request

for you to fetch changes up to 15d6729850728ee49859711dd40b00d8d85d94ee:

  mirror: fix misleading comments (2016-06-28 23:08:25 -0400)

----------------------------------------------------------------

----------------------------------------------------------------

Changlong Xie (2):
  blockjob: assert(cb) when create job
  mirror: fix misleading comments

Denis V. Lunev (1):
  mirror: fix trace_mirror_yield_in_flight usage in mirror_iteration()

Jeff Cody (1):
  block/gluster: add support for selecting debug logging level

John Snow (3):
  mirror: clarify mirror_do_read return code
  mirror: limit niov to IOV_MAX elements, again
  iotests: add small-granularity mirror test

Niels de Vos (1):
  block/gluster: add support for SEEK_DATA/SEEK_HOLE

Peter Lieven (2):
  block/nfs: refuse readahead if cache.direct is on
  block/nfs: add support for libnfs pagecache

 block/backup.c             |   1 -
 block/gluster.c            | 230 +++++++++++++++++++++++++++++++++++++++++++--
 block/mirror.c             |  14 ++-
 block/nfs.c                |  55 ++++++++++-
 blockjob.c                 |   1 +
 tests/qemu-iotests/041     |  30 ++++++
 tests/qemu-iotests/041.out |   4 +-
 7 files changed, 317 insertions(+), 18 deletions(-)

-- 
1.9.3

Re: [Qemu-devel] [PULL 00/10] Block patches

[Peter Maydell]

On 29 June 2016 at 04:10, Jeff Cody <jcody@redhat.com> wrote:
> The following changes since commit d7f30403576f04f1f3a5fb5a1d18cba8dfa7a6d2:
>
>   cputlb: don't cpu_abort() if guest tries to execute outside RAM or RAM (2016-06-28 18:50:53 +0100)
>
> are available in the git repository at:
>
>   git@github.com:codyprime/qemu-kvm-jtc.git tags/block-pull-request
>
> for you to fetch changes up to 15d6729850728ee49859711dd40b00d8d85d94ee:
>
>   mirror: fix misleading comments (2016-06-28 23:08:25 -0400)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------

Applied, thanks.

-- PMM

[Qemu-devel] [PULL 00/10] Block patches

[Jeff Cody]

The following changes since commit 411ad78115ebeb3411cf4b7622784b93dfabe259:

  Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2017-12-15-1' into staging (2017-12-17 15:27:41 +0000)

are available in the git repository at:

  git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request

for you to fetch changes up to 996922de45299878cdc4c15b72b19edf2bc618a4:

  block/curl: fix minor memory leaks (2017-12-18 15:44:39 -0500)

----------------------------------------------------------------
Blockjob and protocol patches
----------------------------------------------------------------

Jeff Cody (4):
  block/sheepdog: remove spurious NULL check
  block/sheepdog: code beautification
  block/curl: check error return of curl_global_init()
  block/curl: fix minor memory leaks

John Snow (1):
  blockjob: kick jobs on set-speed

Vladimir Sementsov-Ogievskiy (5):
  hbitmap: add next_zero function
  backup: move from done_bitmap to copy_bitmap
  backup: init copy_bitmap from sync_bitmap for incremental
  backup: simplify non-dirty bits progress processing
  backup: use copy_bitmap in incremental backup

 block/backup.c               | 116 +++++++++++++++++-------------
 block/curl.c                 |  24 +++++--
 block/dirty-bitmap.c         |   5 ++
 block/sheepdog.c             | 166 +++++++++++++++++++++----------------------
 blockjob.c                   |  30 +++++++-
 include/block/dirty-bitmap.h |   1 +
 include/qemu/hbitmap.h       |   8 +++
 tests/test-hbitmap.c         |  61 ++++++++++++++++
 util/hbitmap.c               |  39 ++++++++++
 9 files changed, 309 insertions(+), 141 deletions(-)

-- 
2.9.5

[Qemu-devel] [PULL 01/10] hbitmap: add next_zero function

[Jeff Cody]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The function searches for next zero bit.
Also add interface for BdrvDirtyBitmap and unit test.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20171012135313.227864-2-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/dirty-bitmap.c         |  5 ++++
 include/block/dirty-bitmap.h |  1 +
 include/qemu/hbitmap.h       |  8 ++++++
 tests/test-hbitmap.c         | 61 ++++++++++++++++++++++++++++++++++++++++++++
 util/hbitmap.c               | 39 ++++++++++++++++++++++++++++
 5 files changed, 114 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index bd04e99..7879d13 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -715,3 +715,8 @@ char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp)
 {
     return hbitmap_sha256(bitmap->bitmap, errp);
 }
+
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset)
+{
+    return hbitmap_next_zero(bitmap->bitmap, offset);
+}
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 3579a75..a591c27 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -91,5 +91,6 @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
 BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
                                         BdrvDirtyBitmap *bitmap);
 char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start);
 
 #endif
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 81e7804..6b6490e 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -292,6 +292,14 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
  */
 unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
 
+/* hbitmap_next_zero:
+ * @hb: The HBitmap to operate on
+ * @start: The bit to start from.
+ *
+ * Find next not dirty bit.
+ */
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start);
+
 /* hbitmap_create_meta:
  * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
  * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index af41642..9091c63 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -925,6 +925,61 @@ static void test_hbitmap_iter_and_reset(TestHBitmapData *data,
     hbitmap_iter_next(&hbi);
 }
 
+static void test_hbitmap_next_zero_check(TestHBitmapData *data, int64_t start)
+{
+    int64_t ret1 = hbitmap_next_zero(data->hb, start);
+    int64_t ret2 = start;
+    for ( ; ret2 < data->size && hbitmap_get(data->hb, ret2); ret2++) {
+        ;
+    }
+    if (ret2 == data->size) {
+        ret2 = -1;
+    }
+
+    g_assert_cmpint(ret1, ==, ret2);
+}
+
+static void test_hbitmap_next_zero_do(TestHBitmapData *data, int granularity)
+{
+    hbitmap_test_init(data, L3, granularity);
+    test_hbitmap_next_zero_check(data, 0);
+    test_hbitmap_next_zero_check(data, L3 - 1);
+
+    hbitmap_set(data->hb, L2, 1);
+    test_hbitmap_next_zero_check(data, 0);
+    test_hbitmap_next_zero_check(data, L2 - 1);
+    test_hbitmap_next_zero_check(data, L2);
+    test_hbitmap_next_zero_check(data, L2 + 1);
+
+    hbitmap_set(data->hb, L2 + 5, L1);
+    test_hbitmap_next_zero_check(data, 0);
+    test_hbitmap_next_zero_check(data, L2 + 1);
+    test_hbitmap_next_zero_check(data, L2 + 2);
+    test_hbitmap_next_zero_check(data, L2 + 5);
+    test_hbitmap_next_zero_check(data, L2 + L1 - 1);
+    test_hbitmap_next_zero_check(data, L2 + L1);
+
+    hbitmap_set(data->hb, L2 * 2, L3 - L2 * 2);
+    test_hbitmap_next_zero_check(data, L2 * 2 - L1);
+    test_hbitmap_next_zero_check(data, L2 * 2 - 2);
+    test_hbitmap_next_zero_check(data, L2 * 2 - 1);
+    test_hbitmap_next_zero_check(data, L2 * 2);
+    test_hbitmap_next_zero_check(data, L3 - 1);
+
+    hbitmap_set(data->hb, 0, L3);
+    test_hbitmap_next_zero_check(data, 0);
+}
+
+static void test_hbitmap_next_zero_0(TestHBitmapData *data, const void *unused)
+{
+    test_hbitmap_next_zero_do(data, 0);
+}
+
+static void test_hbitmap_next_zero_4(TestHBitmapData *data, const void *unused)
+{
+    test_hbitmap_next_zero_do(data, 4);
+}
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
@@ -985,6 +1040,12 @@ int main(int argc, char **argv)
 
     hbitmap_test_add("/hbitmap/iter/iter_and_reset",
                      test_hbitmap_iter_and_reset);
+
+    hbitmap_test_add("/hbitmap/next_zero/next_zero_0",
+                     test_hbitmap_next_zero_0);
+    hbitmap_test_add("/hbitmap/next_zero/next_zero_4",
+                     test_hbitmap_next_zero_4);
+
     g_test_run();
 
     return 0;
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 2f9d0fd..289778a 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -188,6 +188,45 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first)
     }
 }
 
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start)
+{
+    size_t pos = (start >> hb->granularity) >> BITS_PER_LEVEL;
+    unsigned long *last_lev = hb->levels[HBITMAP_LEVELS - 1];
+    uint64_t sz = hb->sizes[HBITMAP_LEVELS - 1];
+    unsigned long cur = last_lev[pos];
+    unsigned start_bit_offset =
+            (start >> hb->granularity) & (BITS_PER_LONG - 1);
+    int64_t res;
+
+    cur |= (1UL << start_bit_offset) - 1;
+    assert((start >> hb->granularity) < hb->size);
+
+    if (cur == (unsigned long)-1) {
+        do {
+            pos++;
+        } while (pos < sz && last_lev[pos] == (unsigned long)-1);
+
+        if (pos >= sz) {
+            return -1;
+        }
+
+        cur = last_lev[pos];
+    }
+
+    res = (pos << BITS_PER_LEVEL) + ctol(cur);
+    if (res >= hb->size) {
+        return -1;
+    }
+
+    res = res << hb->granularity;
+    if (res < start) {
+        assert(((start - res) >> hb->granularity) == 0);
+        return start;
+    }
+
+    return res;
+}
+
 bool hbitmap_empty(const HBitmap *hb)
 {
     return hb->count == 0;
-- 
2.9.5

[Qemu-devel] [PULL 02/10] backup: move from done_bitmap to copy_bitmap

[Jeff Cody]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Use HBitmap copy_bitmap instead of done_bitmap. This is needed to
improve incremental backup in following patches and to unify backup
loop for full/incremental modes in future patches.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20171012135313.227864-3-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 99e6bcc..5175808 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -40,11 +40,12 @@ typedef struct BackupBlockJob {
     BlockdevOnError on_target_error;
     CoRwlock flush_rwlock;
     uint64_t bytes_read;
-    unsigned long *done_bitmap;
     int64_t cluster_size;
     bool compress;
     NotifierWithReturn before_write;
     QLIST_HEAD(, CowRequest) inflight_reqs;
+
+    HBitmap *copy_bitmap;
 } BackupBlockJob;
 
 /* See if in-flight requests overlap and wait for them to complete */
@@ -109,10 +110,11 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
     cow_request_begin(&cow_request, job, start, end);
 
     for (; start < end; start += job->cluster_size) {
-        if (test_bit(start / job->cluster_size, job->done_bitmap)) {
+        if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) {
             trace_backup_do_cow_skip(job, start);
             continue; /* already copied */
         }
+        hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1);
 
         trace_backup_do_cow_process(job, start);
 
@@ -132,6 +134,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
             if (error_is_read) {
                 *error_is_read = true;
             }
+            hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1);
             goto out;
         }
 
@@ -148,11 +151,10 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
             if (error_is_read) {
                 *error_is_read = false;
             }
+            hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1);
             goto out;
         }
 
-        set_bit(start / job->cluster_size, job->done_bitmap);
-
         /* Publish progress, guest I/O counts as progress too.  Note that the
          * offset field is an opaque progress value, it is not a disk offset.
          */
@@ -260,7 +262,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp)
     }
 
     len = DIV_ROUND_UP(backup_job->common.len, backup_job->cluster_size);
-    bitmap_zero(backup_job->done_bitmap, len);
+    hbitmap_set(backup_job->copy_bitmap, 0, len);
 }
 
 void backup_wait_for_overlapping_requests(BlockJob *job, int64_t offset,
@@ -425,19 +427,22 @@ static void coroutine_fn backup_run(void *opaque)
     BackupBlockJob *job = opaque;
     BackupCompleteData *data;
     BlockDriverState *bs = blk_bs(job->common.blk);
-    int64_t offset;
+    int64_t offset, nb_clusters;
     int ret = 0;
 
     QLIST_INIT(&job->inflight_reqs);
     qemu_co_rwlock_init(&job->flush_rwlock);
 
-    job->done_bitmap = bitmap_new(DIV_ROUND_UP(job->common.len,
-                                               job->cluster_size));
+    nb_clusters = DIV_ROUND_UP(job->common.len, job->cluster_size);
+    job->copy_bitmap = hbitmap_alloc(nb_clusters, 0);
+    hbitmap_set(job->copy_bitmap, 0, nb_clusters);
 
     job->before_write.notify = backup_before_write_notify;
     bdrv_add_before_write_notifier(bs, &job->before_write);
 
     if (job->sync_mode == MIRROR_SYNC_MODE_NONE) {
+        /* All bits are set in copy_bitmap to allow any cluster to be copied.
+         * This does not actually require them to be copied. */
         while (!block_job_is_cancelled(&job->common)) {
             /* Yield until the job is cancelled.  We just let our before_write
              * notify callback service CoW requests. */
@@ -512,7 +517,7 @@ static void coroutine_fn backup_run(void *opaque)
     /* wait until pending backup_do_cow() calls have completed */
     qemu_co_rwlock_wrlock(&job->flush_rwlock);
     qemu_co_rwlock_unlock(&job->flush_rwlock);
-    g_free(job->done_bitmap);
+    hbitmap_free(job->copy_bitmap);
 
     data = g_malloc(sizeof(*data));
     data->ret = ret;
-- 
2.9.5

[Qemu-devel] [PULL 03/10] backup: init copy_bitmap from sync_bitmap for incremental

[Jeff Cody]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We should not copy non-dirty clusters in write notifiers. So,
initialize copy_bitmap from sync_bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20171012135313.227864-4-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 44 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 43 insertions(+), 1 deletion(-)

diff --git a/block/backup.c b/block/backup.c
index 5175808..b8901ea 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -422,6 +422,43 @@ out:
     return ret;
 }
 
+/* init copy_bitmap from sync_bitmap */
+static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
+{
+    BdrvDirtyBitmapIter *dbi;
+    int64_t offset;
+    int64_t end = DIV_ROUND_UP(bdrv_dirty_bitmap_size(job->sync_bitmap),
+                               job->cluster_size);
+
+    dbi = bdrv_dirty_iter_new(job->sync_bitmap);
+    while ((offset = bdrv_dirty_iter_next(dbi)) != -1) {
+        int64_t cluster = offset / job->cluster_size;
+        int64_t next_cluster;
+
+        offset += bdrv_dirty_bitmap_granularity(job->sync_bitmap);
+        if (offset >= bdrv_dirty_bitmap_size(job->sync_bitmap)) {
+            hbitmap_set(job->copy_bitmap, cluster, end - cluster);
+            break;
+        }
+
+        offset = bdrv_dirty_bitmap_next_zero(job->sync_bitmap, offset);
+        if (offset == -1) {
+            hbitmap_set(job->copy_bitmap, cluster, end - cluster);
+            break;
+        }
+
+        next_cluster = DIV_ROUND_UP(offset, job->cluster_size);
+        hbitmap_set(job->copy_bitmap, cluster, next_cluster - cluster);
+        if (next_cluster >= end) {
+            break;
+        }
+
+        bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size);
+    }
+
+    bdrv_dirty_iter_free(dbi);
+}
+
 static void coroutine_fn backup_run(void *opaque)
 {
     BackupBlockJob *job = opaque;
@@ -435,7 +472,12 @@ static void coroutine_fn backup_run(void *opaque)
 
     nb_clusters = DIV_ROUND_UP(job->common.len, job->cluster_size);
     job->copy_bitmap = hbitmap_alloc(nb_clusters, 0);
-    hbitmap_set(job->copy_bitmap, 0, nb_clusters);
+    if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
+        backup_incremental_init_copy_bitmap(job);
+    } else {
+        hbitmap_set(job->copy_bitmap, 0, nb_clusters);
+    }
+
 
     job->before_write.notify = backup_before_write_notify;
     bdrv_add_before_write_notifier(bs, &job->before_write);
-- 
2.9.5

[Qemu-devel] [PULL 04/10] backup: simplify non-dirty bits progress processing

[Jeff Cody]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Set fake progress for non-dirty clusters in copy_bitmap initialization,
to. It simplifies code and allows further refactoring.

This patch changes user's view of backup progress, but formally it
doesn't changed: progress hops are just moved to the beginning.

Actually it's just a point of view: when do we actually skip clusters?
We can say in the very beginning, that we skip these clusters and do
not think about them later.

Of course, if go through disk sequentially, it's logical to say, that
we skip clusters between copied portions to the left and to the right
of them. But even now copying progress is not sequential because of
write notifiers. Future patches will introduce new backup architecture
which will do copying in several coroutines in parallel, so it will
make no sense to publish fake progress by parts in parallel with
other copying requests.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 20171012135313.227864-5-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index b8901ea..8ee2200 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -369,7 +369,6 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
     int64_t offset;
     int64_t cluster;
     int64_t end;
-    int64_t last_cluster = -1;
     BdrvDirtyBitmapIter *dbi;
 
     granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap);
@@ -380,12 +379,6 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
     while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
         cluster = offset / job->cluster_size;
 
-        /* Fake progress updates for any clusters we skipped */
-        if (cluster != last_cluster + 1) {
-            job->common.offset += ((cluster - last_cluster - 1) *
-                                   job->cluster_size);
-        }
-
         for (end = cluster + clusters_per_iter; cluster < end; cluster++) {
             do {
                 if (yield_and_check(job)) {
@@ -407,14 +400,6 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
         if (granularity < job->cluster_size) {
             bdrv_set_dirty_iter(dbi, cluster * job->cluster_size);
         }
-
-        last_cluster = cluster - 1;
-    }
-
-    /* Play some final catchup with the progress meter */
-    end = DIV_ROUND_UP(job->common.len, job->cluster_size);
-    if (last_cluster + 1 < end) {
-        job->common.offset += ((end - last_cluster - 1) * job->cluster_size);
     }
 
 out:
@@ -456,6 +441,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
         bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size);
     }
 
+    job->common.offset = job->common.len -
+                         hbitmap_count(job->copy_bitmap) * job->cluster_size;
+
     bdrv_dirty_iter_free(dbi);
 }
 
-- 
2.9.5

[Qemu-devel] [PULL 05/10] backup: use copy_bitmap in incremental backup

[Jeff Cody]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We can use copy_bitmap instead of sync_bitmap. copy_bitmap is
initialized from sync_bitmap and it is more informative: we will not try
to process data, that is already in progress (by write notifier).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 20171012135313.227864-6-vsementsov@virtuozzo.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/backup.c | 55 +++++++++++++++++--------------------------------------
 1 file changed, 17 insertions(+), 38 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 8ee2200..4a16a37 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -362,49 +362,28 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job)
 
 static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
 {
+    int ret;
     bool error_is_read;
-    int ret = 0;
-    int clusters_per_iter;
-    uint32_t granularity;
-    int64_t offset;
     int64_t cluster;
-    int64_t end;
-    BdrvDirtyBitmapIter *dbi;
+    HBitmapIter hbi;
 
-    granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap);
-    clusters_per_iter = MAX((granularity / job->cluster_size), 1);
-    dbi = bdrv_dirty_iter_new(job->sync_bitmap);
-
-    /* Find the next dirty sector(s) */
-    while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
-        cluster = offset / job->cluster_size;
-
-        for (end = cluster + clusters_per_iter; cluster < end; cluster++) {
-            do {
-                if (yield_and_check(job)) {
-                    goto out;
-                }
-                ret = backup_do_cow(job, cluster * job->cluster_size,
-                                    job->cluster_size, &error_is_read,
-                                    false);
-                if ((ret < 0) &&
-                    backup_error_action(job, error_is_read, -ret) ==
-                    BLOCK_ERROR_ACTION_REPORT) {
-                    goto out;
-                }
-            } while (ret < 0);
-        }
-
-        /* If the bitmap granularity is smaller than the backup granularity,
-         * we need to advance the iterator pointer to the next cluster. */
-        if (granularity < job->cluster_size) {
-            bdrv_set_dirty_iter(dbi, cluster * job->cluster_size);
-        }
+    hbitmap_iter_init(&hbi, job->copy_bitmap, 0);
+    while ((cluster = hbitmap_iter_next(&hbi)) != -1) {
+        do {
+            if (yield_and_check(job)) {
+                return 0;
+            }
+            ret = backup_do_cow(job, cluster * job->cluster_size,
+                                job->cluster_size, &error_is_read, false);
+            if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
+                           BLOCK_ERROR_ACTION_REPORT)
+            {
+                return ret;
+            }
+        } while (ret < 0);
     }
 
-out:
-    bdrv_dirty_iter_free(dbi);
-    return ret;
+    return 0;
 }
 
 /* init copy_bitmap from sync_bitmap */
-- 
2.9.5

[Qemu-devel] [PULL 06/10] blockjob: kick jobs on set-speed

[Jeff Cody]

From: John Snow <jsnow@redhat.com>

If users set an unreasonably low speed (like one byte per second), the
calculated delay may exceed many hours. While we like to punish users
for asking for stupid things, we do also like to allow users to correct
their wicked ways.

When a user provides a new speed, kick the job to allow it to recalculate
its delay.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20171213204611.26276-1-jsnow@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 blockjob.c | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/blockjob.c b/blockjob.c
index 715c2c2..6173e47 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -59,6 +59,7 @@ static void __attribute__((__constructor__)) block_job_init(void)
 
 static void block_job_event_cancelled(BlockJob *job);
 static void block_job_event_completed(BlockJob *job, const char *msg);
+static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job));
 
 /* Transactional group of block jobs */
 struct BlockJobTxn {
@@ -480,9 +481,16 @@ static void block_job_completed_txn_success(BlockJob *job)
     }
 }
 
+/* Assumes the block_job_mutex is held */
+static bool block_job_timer_pending(BlockJob *job)
+{
+    return timer_pending(&job->sleep_timer);
+}
+
 void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
 {
     Error *local_err = NULL;
+    int64_t old_speed = job->speed;
 
     if (!job->driver->set_speed) {
         error_setg(errp, QERR_UNSUPPORTED);
@@ -495,6 +503,12 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
     }
 
     job->speed = speed;
+    if (speed <= old_speed) {
+        return;
+    }
+
+    /* kick only if a timer is pending */
+    block_job_enter_cond(job, block_job_timer_pending);
 }
 
 void block_job_complete(BlockJob *job, Error **errp)
@@ -821,7 +835,11 @@ void block_job_resume_all(void)
     }
 }
 
-void block_job_enter(BlockJob *job)
+/*
+ * Conditionally enter a block_job pending a call to fn() while
+ * under the block_job_lock critical section.
+ */
+static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job))
 {
     if (!block_job_started(job)) {
         return;
@@ -836,6 +854,11 @@ void block_job_enter(BlockJob *job)
         return;
     }
 
+    if (fn && !fn(job)) {
+        block_job_unlock();
+        return;
+    }
+
     assert(!job->deferred_to_main_loop);
     timer_del(&job->sleep_timer);
     job->busy = true;
@@ -843,6 +866,11 @@ void block_job_enter(BlockJob *job)
     aio_co_wake(job->co);
 }
 
+void block_job_enter(BlockJob *job)
+{
+    block_job_enter_cond(job, NULL);
+}
+
 bool block_job_is_cancelled(BlockJob *job)
 {
     return job->cancelled;
-- 
2.9.5

[Qemu-devel] [PULL 07/10] block/sheepdog: remove spurious NULL check

[Jeff Cody]

'tag' is already checked in the lines immediately preceding this check,
and set to non-NULL if NULL.  No need to check again, it hasn't changed.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/sheepdog.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/sheepdog.c b/block/sheepdog.c
index 696a714..459d93a 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -1632,7 +1632,7 @@ static int sd_open(BlockDriverState *bs, QDict *options, int flags,
     if (!tag) {
         tag = "";
     }
-    if (tag && strlen(tag) >= SD_MAX_VDI_TAG_LEN) {
+    if (strlen(tag) >= SD_MAX_VDI_TAG_LEN) {
         error_setg(errp, "value of parameter 'tag' is too long");
         ret = -EINVAL;
         goto err_no_fd;
-- 
2.9.5

[Qemu-devel] [PULL 08/10] block/sheepdog: code beautification

[Jeff Cody]

No functional changes, just whitespace manipulation.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/sheepdog.c | 164 +++++++++++++++++++++++++++----------------------------
 1 file changed, 82 insertions(+), 82 deletions(-)

diff --git a/block/sheepdog.c b/block/sheepdog.c
index 459d93a..488bad3 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -400,7 +400,7 @@ typedef struct BDRVSheepdogReopenState {
     int cache_flags;
 } BDRVSheepdogReopenState;
 
-static const char * sd_strerror(int err)
+static const char *sd_strerror(int err)
 {
     int i;
 
@@ -3078,111 +3078,111 @@ static QemuOptsList sd_create_opts = {
 };
 
 static BlockDriver bdrv_sheepdog = {
-    .format_name    = "sheepdog",
-    .protocol_name  = "sheepdog",
-    .instance_size  = sizeof(BDRVSheepdogState),
-    .bdrv_parse_filename    = sd_parse_filename,
-    .bdrv_file_open = sd_open,
-    .bdrv_reopen_prepare    = sd_reopen_prepare,
-    .bdrv_reopen_commit     = sd_reopen_commit,
-    .bdrv_reopen_abort      = sd_reopen_abort,
-    .bdrv_close     = sd_close,
-    .bdrv_create    = sd_create,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
-    .bdrv_getlength = sd_getlength,
+    .format_name                  = "sheepdog",
+    .protocol_name                = "sheepdog",
+    .instance_size                = sizeof(BDRVSheepdogState),
+    .bdrv_parse_filename          = sd_parse_filename,
+    .bdrv_file_open               = sd_open,
+    .bdrv_reopen_prepare          = sd_reopen_prepare,
+    .bdrv_reopen_commit           = sd_reopen_commit,
+    .bdrv_reopen_abort            = sd_reopen_abort,
+    .bdrv_close                   = sd_close,
+    .bdrv_create                  = sd_create,
+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+    .bdrv_getlength               = sd_getlength,
     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
-    .bdrv_truncate  = sd_truncate,
+    .bdrv_truncate                = sd_truncate,
 
-    .bdrv_co_readv  = sd_co_readv,
-    .bdrv_co_writev = sd_co_writev,
-    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
-    .bdrv_co_pdiscard = sd_co_pdiscard,
-    .bdrv_co_get_block_status = sd_co_get_block_status,
+    .bdrv_co_readv                = sd_co_readv,
+    .bdrv_co_writev               = sd_co_writev,
+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
+    .bdrv_co_pdiscard             = sd_co_pdiscard,
+    .bdrv_co_get_block_status     = sd_co_get_block_status,
 
-    .bdrv_snapshot_create   = sd_snapshot_create,
-    .bdrv_snapshot_goto     = sd_snapshot_goto,
-    .bdrv_snapshot_delete   = sd_snapshot_delete,
-    .bdrv_snapshot_list     = sd_snapshot_list,
+    .bdrv_snapshot_create         = sd_snapshot_create,
+    .bdrv_snapshot_goto           = sd_snapshot_goto,
+    .bdrv_snapshot_delete         = sd_snapshot_delete,
+    .bdrv_snapshot_list           = sd_snapshot_list,
 
-    .bdrv_save_vmstate  = sd_save_vmstate,
-    .bdrv_load_vmstate  = sd_load_vmstate,
+    .bdrv_save_vmstate            = sd_save_vmstate,
+    .bdrv_load_vmstate            = sd_load_vmstate,
 
-    .bdrv_detach_aio_context = sd_detach_aio_context,
-    .bdrv_attach_aio_context = sd_attach_aio_context,
+    .bdrv_detach_aio_context      = sd_detach_aio_context,
+    .bdrv_attach_aio_context      = sd_attach_aio_context,
 
-    .create_opts    = &sd_create_opts,
+    .create_opts                  = &sd_create_opts,
 };
 
 static BlockDriver bdrv_sheepdog_tcp = {
-    .format_name    = "sheepdog",
-    .protocol_name  = "sheepdog+tcp",
-    .instance_size  = sizeof(BDRVSheepdogState),
-    .bdrv_parse_filename    = sd_parse_filename,
-    .bdrv_file_open = sd_open,
-    .bdrv_reopen_prepare    = sd_reopen_prepare,
-    .bdrv_reopen_commit     = sd_reopen_commit,
-    .bdrv_reopen_abort      = sd_reopen_abort,
-    .bdrv_close     = sd_close,
-    .bdrv_create    = sd_create,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
-    .bdrv_getlength = sd_getlength,
+    .format_name                  = "sheepdog",
+    .protocol_name                = "sheepdog+tcp",
+    .instance_size                = sizeof(BDRVSheepdogState),
+    .bdrv_parse_filename          = sd_parse_filename,
+    .bdrv_file_open               = sd_open,
+    .bdrv_reopen_prepare          = sd_reopen_prepare,
+    .bdrv_reopen_commit           = sd_reopen_commit,
+    .bdrv_reopen_abort            = sd_reopen_abort,
+    .bdrv_close                   = sd_close,
+    .bdrv_create                  = sd_create,
+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+    .bdrv_getlength               = sd_getlength,
     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
-    .bdrv_truncate  = sd_truncate,
+    .bdrv_truncate                = sd_truncate,
 
-    .bdrv_co_readv  = sd_co_readv,
-    .bdrv_co_writev = sd_co_writev,
-    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
-    .bdrv_co_pdiscard = sd_co_pdiscard,
-    .bdrv_co_get_block_status = sd_co_get_block_status,
+    .bdrv_co_readv                = sd_co_readv,
+    .bdrv_co_writev               = sd_co_writev,
+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
+    .bdrv_co_pdiscard             = sd_co_pdiscard,
+    .bdrv_co_get_block_status     = sd_co_get_block_status,
 
-    .bdrv_snapshot_create   = sd_snapshot_create,
-    .bdrv_snapshot_goto     = sd_snapshot_goto,
-    .bdrv_snapshot_delete   = sd_snapshot_delete,
-    .bdrv_snapshot_list     = sd_snapshot_list,
+    .bdrv_snapshot_create         = sd_snapshot_create,
+    .bdrv_snapshot_goto           = sd_snapshot_goto,
+    .bdrv_snapshot_delete         = sd_snapshot_delete,
+    .bdrv_snapshot_list           = sd_snapshot_list,
 
-    .bdrv_save_vmstate  = sd_save_vmstate,
-    .bdrv_load_vmstate  = sd_load_vmstate,
+    .bdrv_save_vmstate            = sd_save_vmstate,
+    .bdrv_load_vmstate            = sd_load_vmstate,
 
-    .bdrv_detach_aio_context = sd_detach_aio_context,
-    .bdrv_attach_aio_context = sd_attach_aio_context,
+    .bdrv_detach_aio_context      = sd_detach_aio_context,
+    .bdrv_attach_aio_context      = sd_attach_aio_context,
 
-    .create_opts    = &sd_create_opts,
+    .create_opts                  = &sd_create_opts,
 };
 
 static BlockDriver bdrv_sheepdog_unix = {
-    .format_name    = "sheepdog",
-    .protocol_name  = "sheepdog+unix",
-    .instance_size  = sizeof(BDRVSheepdogState),
-    .bdrv_parse_filename    = sd_parse_filename,
-    .bdrv_file_open = sd_open,
-    .bdrv_reopen_prepare    = sd_reopen_prepare,
-    .bdrv_reopen_commit     = sd_reopen_commit,
-    .bdrv_reopen_abort      = sd_reopen_abort,
-    .bdrv_close     = sd_close,
-    .bdrv_create    = sd_create,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
-    .bdrv_getlength = sd_getlength,
+    .format_name                  = "sheepdog",
+    .protocol_name                = "sheepdog+unix",
+    .instance_size                = sizeof(BDRVSheepdogState),
+    .bdrv_parse_filename          = sd_parse_filename,
+    .bdrv_file_open               = sd_open,
+    .bdrv_reopen_prepare          = sd_reopen_prepare,
+    .bdrv_reopen_commit           = sd_reopen_commit,
+    .bdrv_reopen_abort            = sd_reopen_abort,
+    .bdrv_close                   = sd_close,
+    .bdrv_create                  = sd_create,
+    .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+    .bdrv_getlength               = sd_getlength,
     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
-    .bdrv_truncate  = sd_truncate,
+    .bdrv_truncate                = sd_truncate,
 
-    .bdrv_co_readv  = sd_co_readv,
-    .bdrv_co_writev = sd_co_writev,
-    .bdrv_co_flush_to_disk  = sd_co_flush_to_disk,
-    .bdrv_co_pdiscard = sd_co_pdiscard,
-    .bdrv_co_get_block_status = sd_co_get_block_status,
+    .bdrv_co_readv                = sd_co_readv,
+    .bdrv_co_writev               = sd_co_writev,
+    .bdrv_co_flush_to_disk        = sd_co_flush_to_disk,
+    .bdrv_co_pdiscard             = sd_co_pdiscard,
+    .bdrv_co_get_block_status     = sd_co_get_block_status,
 
-    .bdrv_snapshot_create   = sd_snapshot_create,
-    .bdrv_snapshot_goto     = sd_snapshot_goto,
-    .bdrv_snapshot_delete   = sd_snapshot_delete,
-    .bdrv_snapshot_list     = sd_snapshot_list,
+    .bdrv_snapshot_create         = sd_snapshot_create,
+    .bdrv_snapshot_goto           = sd_snapshot_goto,
+    .bdrv_snapshot_delete         = sd_snapshot_delete,
+    .bdrv_snapshot_list           = sd_snapshot_list,
 
-    .bdrv_save_vmstate  = sd_save_vmstate,
-    .bdrv_load_vmstate  = sd_load_vmstate,
+    .bdrv_save_vmstate            = sd_save_vmstate,
+    .bdrv_load_vmstate            = sd_load_vmstate,
 
-    .bdrv_detach_aio_context = sd_detach_aio_context,
-    .bdrv_attach_aio_context = sd_attach_aio_context,
+    .bdrv_detach_aio_context      = sd_detach_aio_context,
+    .bdrv_attach_aio_context      = sd_attach_aio_context,
 
-    .create_opts    = &sd_create_opts,
+    .create_opts                  = &sd_create_opts,
 };
 
 static void bdrv_sheepdog_init(void)
-- 
2.9.5

[Qemu-devel] [PULL 09/10] block/curl: check error return of curl_global_init()

[Jeff Cody]

If curl_global_init() fails, per the documentation no other curl
functions may be called, so make sure to check the return value.

Also, some minor changes to the initialization latch variable 'inited':

- Make it static in the file, for clarity
- Change the name for clarity
- Make it a bool

Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/curl.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 2a244e2..00a9879 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -89,6 +89,8 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
 
 struct BDRVCURLState;
 
+static bool libcurl_initialized;
+
 typedef struct CURLAIOCB {
     Coroutine *co;
     QEMUIOVector *qiov;
@@ -686,14 +688,23 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
     double d;
     const char *secretid;
     const char *protocol_delimiter;
+    int ret;
 
-    static int inited = 0;
 
     if (flags & BDRV_O_RDWR) {
         error_setg(errp, "curl block device does not support writes");
         return -EROFS;
     }
 
+    if (!libcurl_initialized) {
+        ret = curl_global_init(CURL_GLOBAL_ALL);
+        if (ret) {
+            error_setg(errp, "libcurl initialization failed with %d", ret);
+            return -EIO;
+        }
+        libcurl_initialized = true;
+    }
+
     qemu_mutex_init(&s->mutex);
     opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
     qemu_opts_absorb_qdict(opts, options, &local_err);
@@ -772,11 +783,6 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
         }
     }
 
-    if (!inited) {
-        curl_global_init(CURL_GLOBAL_ALL);
-        inited = 1;
-    }
-
     DPRINTF("CURL: Opening %s\n", file);
     QSIMPLEQ_INIT(&s->free_state_waitq);
     s->aio_context = bdrv_get_aio_context(bs);
-- 
2.9.5

[Qemu-devel] [PULL 10/10] block/curl: fix minor memory leaks

[Jeff Cody]

Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
---
 block/curl.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index 00a9879..35cf417 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -857,6 +857,9 @@ out_noclean:
     qemu_mutex_destroy(&s->mutex);
     g_free(s->cookie);
     g_free(s->url);
+    g_free(s->username);
+    g_free(s->proxyusername);
+    g_free(s->proxypassword);
     qemu_opts_del(opts);
     return -EINVAL;
 }
@@ -955,6 +958,9 @@ static void curl_close(BlockDriverState *bs)
 
     g_free(s->cookie);
     g_free(s->url);
+    g_free(s->username);
+    g_free(s->proxyusername);
+    g_free(s->proxypassword);
 }
 
 static int64_t curl_getlength(BlockDriverState *bs)
-- 
2.9.5

Re: [Qemu-devel] [PULL 00/10] Block patches

[Peter Maydell]

On 18 December 2017 at 21:08, Jeff Cody <jcody@redhat.com> wrote:
> The following changes since commit 411ad78115ebeb3411cf4b7622784b93dfabe259:
>
>   Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2017-12-15-1' into staging (2017-12-17 15:27:41 +0000)
>
> are available in the git repository at:
>
>   git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request
>
> for you to fetch changes up to 996922de45299878cdc4c15b72b19edf2bc618a4:
>
>   block/curl: fix minor memory leaks (2017-12-18 15:44:39 -0500)
>
> ----------------------------------------------------------------
> Blockjob and protocol patches
> ----------------------------------------------------------------
>

Applied, thanks.

-- PMM