* [Qemu-devel] [PULL 00/17] Block patches
@ 2010-07-06 15:33 Kevin Wolf
2010-07-06 19:07 ` Anthony Liguori
0 siblings, 1 reply; 41+ messages in thread
From: Kevin Wolf @ 2010-07-06 15:33 UTC (permalink / raw)
To: anthony; +Cc: kwolf, qemu-devel
The following changes since commit 734003e6153b3552b9406ef598a1e67aac4a899e:
Anthony Liguori (1):
Merge remote branch 'kwolf/for-anthony' into staging
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git for-anthony
Kevin Wolf (2):
qemu-img check: Distinguish different kinds of errors
qcow2/vdi: Change check to distinguish error cases
MORITA Kazutaka (1):
block: add sheepdog driver for distributed storage support
Markus Armbruster (13):
blockdev: Clean up how readonly persists across virtual media change
block migration: Fix test for read-only drive
raw-posix: Fix test for host CD-ROM
fdc: Reject unimplemented error actions
qdev: Don't hw_error() in qdev_init_nofail()
scsi: Reject unimplemented error actions
error: New qemu_opts_loc_restore()
scsi: Error locations for -drive if=scsi device initialization
ide: Improve error messages
ide: Replace IDEState members is_cdrom, is_cf by drive_kind
ide: Make ide_init_drive() return success
ide: Reject readonly drives unless CD-ROM
ide: Reject invalid CHS geometry
john cooper (1):
Add virtio disk identification support
Makefile.objs | 2 +-
block-migration.c | 2 +-
block.c | 9 +-
block.h | 10 +-
block/qcow2-refcount.c | 120 ++--
block/qcow2.c | 4 +-
block/qcow2.h | 2 +-
block/raw-posix.c | 17 +-
block/sheepdog.c | 2036 ++++++++++++++++++++++++++++++++++++++++++++++++
block/vdi.c | 10 +-
block_int.h | 7 +-
blockdev.c | 2 +-
hw/fdc.c | 22 +-
hw/ide/core.c | 70 ++-
hw/ide/internal.h | 9 +-
hw/ide/macio.c | 2 +-
hw/ide/microdrive.c | 2 +-
hw/ide/qdev.c | 13 +-
hw/qdev.c | 6 +-
hw/scsi-bus.c | 4 +
hw/scsi-disk.c | 5 +
hw/scsi-generic.c | 9 +
hw/virtio-blk.c | 14 +
hw/virtio-blk.h | 3 +
qemu-img.c | 63 ++-
qemu-option.c | 5 +
qemu-option.h | 1 +
27 files changed, 2308 insertions(+), 141 deletions(-)
create mode 100644 block/sheepdog.c
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2010-07-06 15:33 Kevin Wolf
@ 2010-07-06 19:07 ` Anthony Liguori
0 siblings, 0 replies; 41+ messages in thread
From: Anthony Liguori @ 2010-07-06 19:07 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel
On 07/06/2010 10:33 AM, Kevin Wolf wrote:
> The following changes since commit 734003e6153b3552b9406ef598a1e67aac4a899e:
> Anthony Liguori (1):
> Merge remote branch 'kwolf/for-anthony' into staging
>
Pulled. Thanks.
Regards,
Anthony Liguori
> are available in the git repository at:
>
> git://repo.or.cz/qemu/kevin.git for-anthony
>
> Kevin Wolf (2):
> qemu-img check: Distinguish different kinds of errors
> qcow2/vdi: Change check to distinguish error cases
>
> MORITA Kazutaka (1):
> block: add sheepdog driver for distributed storage support
>
> Markus Armbruster (13):
> blockdev: Clean up how readonly persists across virtual media change
> block migration: Fix test for read-only drive
> raw-posix: Fix test for host CD-ROM
> fdc: Reject unimplemented error actions
> qdev: Don't hw_error() in qdev_init_nofail()
> scsi: Reject unimplemented error actions
> error: New qemu_opts_loc_restore()
> scsi: Error locations for -drive if=scsi device initialization
> ide: Improve error messages
> ide: Replace IDEState members is_cdrom, is_cf by drive_kind
> ide: Make ide_init_drive() return success
> ide: Reject readonly drives unless CD-ROM
> ide: Reject invalid CHS geometry
>
> john cooper (1):
> Add virtio disk identification support
>
> Makefile.objs | 2 +-
> block-migration.c | 2 +-
> block.c | 9 +-
> block.h | 10 +-
> block/qcow2-refcount.c | 120 ++--
> block/qcow2.c | 4 +-
> block/qcow2.h | 2 +-
> block/raw-posix.c | 17 +-
> block/sheepdog.c | 2036 ++++++++++++++++++++++++++++++++++++++++++++++++
> block/vdi.c | 10 +-
> block_int.h | 7 +-
> blockdev.c | 2 +-
> hw/fdc.c | 22 +-
> hw/ide/core.c | 70 ++-
> hw/ide/internal.h | 9 +-
> hw/ide/macio.c | 2 +-
> hw/ide/microdrive.c | 2 +-
> hw/ide/qdev.c | 13 +-
> hw/qdev.c | 6 +-
> hw/scsi-bus.c | 4 +
> hw/scsi-disk.c | 5 +
> hw/scsi-generic.c | 9 +
> hw/virtio-blk.c | 14 +
> hw/virtio-blk.h | 3 +
> qemu-img.c | 63 ++-
> qemu-option.c | 5 +
> qemu-option.h | 1 +
> 27 files changed, 2308 insertions(+), 141 deletions(-)
> create mode 100644 block/sheepdog.c
>
>
>
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2013-10-29 16:30 Kevin Wolf
0 siblings, 0 replies; 41+ messages in thread
From: Kevin Wolf @ 2013-10-29 16:30 UTC (permalink / raw)
To: anthony; +Cc: kwolf, qemu-devel
The following changes since commit fc8ead74674b7129e8f31c2595c76658e5622197:
Merge remote-tracking branch 'qemu-kvm/uq/master' into staging (2013-10-18 10:03:24 -0700)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-anthony
for you to fetch changes up to a7cf03d4e150abec88f5837461242dc8a0eee189:
qemu-iotests: Fix 051 reference output (2013-10-29 17:05:35 +0100)
----------------------------------------------------------------
Block patches for 1.7.0-rc0
----------------------------------------------------------------
Alexander Graf (1):
ahci: fix win7 hang on boot
Eric Blake (1):
qapi: fix documentation example
Kevin Wolf (5):
exec: Fix bounce buffer allocation in address_space_map()
ide-test: Check what happens with bus mastering disabled
tests: Multiboot mmap test case
block: Avoid unecessary drv->bdrv_getlength() calls
qemu-iotests: Fix 051 reference output
Liu Yuan (2):
sheepdog: explicitly set copies as type uint8_t
sheepdog: pass copy_policy in the request
Max Reitz (5):
qcow2: Restore total_sectors value in save_vmstate
qcow2: Unset zero_beyond_eof in save_vmstate
qemu-iotests: Test for loading VM state from qcow2
qcow2: Flush image after creation
block: Don't copy backing file name on error
Peter Lieven (2):
qemu-img: add special exit code if bdrv_check is not supported
block/vpc: check that the image has not been truncated
Thibaut LAURENT (1):
block: Disable BDRV_O_COPY_ON_READ for the backing file
block.c | 14 +++--
block/qcow2.c | 19 ++++++
block/raw-posix.c | 9 ++-
block/raw-win32.c | 4 +-
block/raw_bsd.c | 1 +
block/sheepdog.c | 30 ++++++----
block/vpc.c | 7 +++
docs/qapi-code-gen.txt | 2 +-
exec.c | 4 +-
hw/ide/ahci.c | 3 +-
include/block/block_int.h | 3 +
qemu-img.c | 2 +-
tests/ide-test.c | 26 +++++++++
tests/multiboot/Makefile | 18 ++++++
tests/multiboot/libc.c | 139 ++++++++++++++++++++++++++++++++++++++++++++
tests/multiboot/libc.h | 61 +++++++++++++++++++
tests/multiboot/link.ld | 19 ++++++
tests/multiboot/mmap.c | 56 ++++++++++++++++++
tests/multiboot/mmap.out | 93 +++++++++++++++++++++++++++++
tests/multiboot/multiboot.h | 66 +++++++++++++++++++++
tests/multiboot/run_test.sh | 81 ++++++++++++++++++++++++++
tests/multiboot/start.S | 51 ++++++++++++++++
tests/qemu-iotests/051.out | 2 +-
tests/qemu-iotests/068 | 65 +++++++++++++++++++++
tests/qemu-iotests/068.out | 11 ++++
tests/qemu-iotests/group | 1 +
26 files changed, 761 insertions(+), 26 deletions(-)
create mode 100644 tests/multiboot/Makefile
create mode 100644 tests/multiboot/libc.c
create mode 100644 tests/multiboot/libc.h
create mode 100644 tests/multiboot/link.ld
create mode 100644 tests/multiboot/mmap.c
create mode 100644 tests/multiboot/mmap.out
create mode 100644 tests/multiboot/multiboot.h
create mode 100755 tests/multiboot/run_test.sh
create mode 100644 tests/multiboot/start.S
create mode 100755 tests/qemu-iotests/068
create mode 100644 tests/qemu-iotests/068.out
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2014-05-09 19:03 Stefan Hajnoczi
2014-05-13 10:32 ` Peter Maydell
0 siblings, 1 reply; 41+ messages in thread
From: Stefan Hajnoczi @ 2014-05-09 19:03 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi
The following changes since commit 43cbeffb19877c62cbe0aaf08b2f235d98d71340:
Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging (2014-05-08 12:38:01 +0100)
are available in the git repository at:
git://github.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to 5a007547df76446ab891df93ebc55749716609bf:
glib: fix g_poll early timeout on windows (2014-05-09 20:57:35 +0200)
----------------------------------------------------------------
Block pull request
----------------------------------------------------------------
Fam Zheng (4):
qemu-img: Convert by cluster size if target is compressed
vmdk: Implement .bdrv_write_compressed
vmdk: Implement .bdrv_get_info()
qemu-iotests: Test converting to streamOptimized from small cluster size
Jeff Cody (3):
block: qemu-iotests - add common.qemu, for bash-controlled qemu tests
block: qemu-iotests - update 085 to use common.qemu
block: qemu-iotests - test for live migration
Kevin Wolf (1):
block: Fix open flags with BDRV_O_SNAPSHOT
Kirill Batuzov (1):
vl.c: remove init_clocks call from main
Max Reitz (4):
qcow2: Fix alloc_clusters_noref() overflow detection
iotests: Use configured python
block/nfs: Check for NULL server part
block/raw-posix: Try both FIEMAP and SEEK_HOLE
Mike Day (1):
qemu-img: sort block formats in help message
Peter Krempa (1):
gluster: Correctly propagate errors when volume isn't accessible
Peter Lieven (1):
block/iscsi: bump year in copyright notice
Sangho Park (1):
glib: fix g_poll early timeout on windows
block.c | 34 +++----
block/gluster.c | 7 +-
block/iscsi.c | 2 +-
block/nfs.c | 4 +
block/qcow2-refcount.c | 4 +-
block/raw-posix.c | 127 +++++++++++++++-----------
block/vmdk.c | 35 ++++++++
configure | 6 ++
include/block/block.h | 6 +-
include/glib-compat.h | 9 +-
qemu-img.c | 29 +++++-
qemu-timer.c | 3 +
tests/qemu-iotests/031 | 9 +-
tests/qemu-iotests/036 | 7 +-
tests/qemu-iotests/039 | 19 ++--
tests/qemu-iotests/051 | 4 +
tests/qemu-iotests/051.out | 10 +++
tests/qemu-iotests/054 | 3 +-
tests/qemu-iotests/059 | 7 ++
tests/qemu-iotests/059.out | 8 ++
tests/qemu-iotests/060 | 21 ++---
tests/qemu-iotests/061 | 25 +++---
tests/qemu-iotests/065 | 2 +-
tests/qemu-iotests/083 | 3 +-
tests/qemu-iotests/085 | 73 +++------------
tests/qemu-iotests/091 | 105 ++++++++++++++++++++++
tests/qemu-iotests/091.out | 28 ++++++
tests/qemu-iotests/check | 18 +++-
tests/qemu-iotests/common.qemu | 200 +++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/group | 1 +
util/oslib-win32.c | 112 +++++++++++++++++++++++
vl.c | 1 -
32 files changed, 744 insertions(+), 178 deletions(-)
create mode 100755 tests/qemu-iotests/091
create mode 100644 tests/qemu-iotests/091.out
create mode 100644 tests/qemu-iotests/common.qemu
--
1.9.0
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2014-05-09 19:03 Stefan Hajnoczi
@ 2014-05-13 10:32 ` Peter Maydell
0 siblings, 0 replies; 41+ messages in thread
From: Peter Maydell @ 2014-05-13 10:32 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: QEMU Developers
On 9 May 2014 20:03, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit 43cbeffb19877c62cbe0aaf08b2f235d98d71340:
>
> Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging (2014-05-08 12:38:01 +0100)
>
> are available in the git repository at:
>
>
> git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 5a007547df76446ab891df93ebc55749716609bf:
>
> glib: fix g_poll early timeout on windows (2014-05-09 20:57:35 +0200)
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2014-06-02 13:56 Kevin Wolf
2014-06-02 14:46 ` Peter Maydell
0 siblings, 1 reply; 41+ messages in thread
From: Kevin Wolf @ 2014-06-02 13:56 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf
The following changes since commit d7d3d6092cb7edc75dc49fb90c86dd5425ab4805:
Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' into staging (2014-05-28 18:38:39 +0100)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 55d492d7602c27cabb605f42e72c755de1c186c1:
qemu-img: Report error even with --oformat=json (2014-06-02 13:58:40 +0200)
----------------------------------------------------------------
Block patches
----------------------------------------------------------------
Fam Zheng (1):
vmdk: Fix local_err in vmdk_create
Markus Armbruster (14):
qemu-img: Plug memory leak on block option help error path
block/vvfat: Plug memory leak in enable_write_target()
qcow2: Plug memory leak on qcow2_invalidate_cache() error paths
block: Plug memory leak on brv_open_image() error path
qemu-io: Support multiple -o in open command
qemu-io: Plug memory leak in open command
qemu-io: Don't print NULL when open without non-option arg fails
blockdev: Plug memory leak in blockdev_init()
blockdev: Plug memory leak in drive_init()
block/qapi: Plug memory leak in dump_qobject() case QTYPE_QERROR
block/vvfat: Plug memory leak in check_directory_consistency()
block/vvfat: Plug memory leak in read_directory()
block/sheepdog: Plug memory leak in sd_snapshot_create()
qemu-img: Plug memory leak in convert command
Max Reitz (1):
qemu-img: Report error even with --oformat=json
Peter Maydell (1):
block/raw-posix.c: Avoid nonstandard LONG_LONG_MAX
block.c | 1 +
block/qapi.c | 1 +
block/qcow2.c | 3 +--
block/raw-posix.c | 2 +-
block/sheepdog.c | 4 ++--
block/vmdk.c | 8 ++++----
block/vvfat.c | 7 +++++--
blockdev.c | 7 +++++--
qemu-img.c | 7 +++----
qemu-io.c | 22 +++++++++++++++-------
10 files changed, 38 insertions(+), 24 deletions(-)
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2014-06-02 13:56 Kevin Wolf
@ 2014-06-02 14:46 ` Peter Maydell
0 siblings, 0 replies; 41+ messages in thread
From: Peter Maydell @ 2014-06-02 14:46 UTC (permalink / raw)
To: Kevin Wolf; +Cc: QEMU Developers
On 2 June 2014 14:56, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit d7d3d6092cb7edc75dc49fb90c86dd5425ab4805:
>
> Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' into staging (2014-05-28 18:38:39 +0100)
>
> are available in the git repository at:
>
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 55d492d7602c27cabb605f42e72c755de1c186c1:
>
> qemu-img: Report error even with --oformat=json (2014-06-02 13:58:40 +0200)
>
> ----------------------------------------------------------------
> Block patches
>
> ----------------------------------------------------------------
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2015-06-05 11:57 Stefan Hajnoczi
2015-06-05 13:53 ` Peter Maydell
0 siblings, 1 reply; 41+ messages in thread
From: Stefan Hajnoczi @ 2015-06-05 11:57 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Peter Maydell, Stefan Hajnoczi
The following changes since commit 3b730f570c5872ceea2137848f1d4554d4847441:
Merge remote-tracking branch 'remotes/agraf/tags/signed-ppc-for-upstream' into staging (2015-06-04 14:04:14 +0100)
are available in the git repository at:
git://github.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to 1ad69e1128519a970a9fdf0203a67ab4bc18eb68:
qemu-iotests: expand test 093 to support group throttling (2015-06-05 11:03:07 +0100)
----------------------------------------------------------------
----------------------------------------------------------------
Alberto Garcia (7):
throttle: Add throttle group infrastructure
throttle: Add throttle group infrastructure tests
throttle: Add throttle group support
throttle: acquire the ThrottleGroup lock in bdrv_swap()
throttle: add the name of the ThrottleGroup to BlockDeviceInfo
throttle: Update throttle infrastructure copyright
qemu-iotests: expand test 093 to support group throttling
Benoît Canet (1):
throttle: Extract timers from ThrottleState into a separate structure
Fam Zheng (8):
block: Add bdrv_get_block_status_above
qmp: Add optional bool "unmap" to drive-mirror
mirror: Do zero write on target if sectors not allocated
block: Fix dirty bitmap in bdrv_co_discard
block: Remove bdrv_reset_dirty
qemu-iotests: Make block job methods common
qemu-iotests: Add test case for mirror with unmap
iotests: Use event_wait in wait_ready
Stefan Hajnoczi (1):
Revert "iothread: release iothread around aio_poll"
async.c | 8 +-
block.c | 50 ++--
block/Makefile.objs | 1 +
block/io.c | 131 +++++------
block/mirror.c | 27 ++-
block/qapi.c | 8 +-
block/throttle-groups.c | 496 ++++++++++++++++++++++++++++++++++++++++
blockdev.c | 43 +++-
hmp.c | 12 +-
include/block/block.h | 7 +-
include/block/block_int.h | 11 +-
include/block/throttle-groups.h | 46 ++++
include/qemu/throttle.h | 46 ++--
iothread.c | 11 +-
qapi/block-core.json | 37 ++-
qemu-options.hx | 1 +
qmp-commands.hx | 6 +-
tests/qemu-iotests/041 | 66 ++----
tests/qemu-iotests/093 | 89 ++++---
tests/qemu-iotests/132 | 59 +++++
tests/qemu-iotests/132.out | 5 +
tests/qemu-iotests/group | 1 +
tests/qemu-iotests/iotests.py | 23 ++
tests/test-aio.c | 19 +-
tests/test-throttle.c | 161 +++++++++----
util/throttle.c | 81 ++++---
26 files changed, 1144 insertions(+), 301 deletions(-)
create mode 100644 block/throttle-groups.c
create mode 100644 include/block/throttle-groups.h
create mode 100644 tests/qemu-iotests/132
create mode 100644 tests/qemu-iotests/132.out
--
2.4.2
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2015-06-05 11:57 Stefan Hajnoczi
@ 2015-06-05 13:53 ` Peter Maydell
0 siblings, 0 replies; 41+ messages in thread
From: Peter Maydell @ 2015-06-05 13:53 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: Kevin Wolf, QEMU Developers
On 5 June 2015 at 12:57, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit 3b730f570c5872ceea2137848f1d4554d4847441:
>
> Merge remote-tracking branch 'remotes/agraf/tags/signed-ppc-for-upstream' into staging (2015-06-04 14:04:14 +0100)
>
> are available in the git repository at:
>
> git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 1ad69e1128519a970a9fdf0203a67ab4bc18eb68:
>
> qemu-iotests: expand test 093 to support group throttling (2015-06-05 11:03:07 +0100)
>
> ----------------------------------------------------------------
I'm afraid this doesn't pass 'make check' on my OSX box:
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$((RANDOM % 255 + 1))} gtester -k
--verbose -m=quick tests/test-throttle
TEST: tests/test-throttle... (pid=56152)
/throttle/leak_bucket: OK
/throttle/compute_wait:
qemu: qemu_mutex_lock: Invalid argument
OK
/throttle/init: OK
/throttle/destroy: OK
/throttle/have_timer: OK
/throttle/detach_attach: OK
/throttle/config/enabled: OK
/throttle/config/conflicting: OK
/throttle/config/is_valid: OK
/throttle/config_functions: OK
/throttle/accounting: OK
/throttle/groups: FAIL
GTester: last random seed: R02S1a38f05a43ead1a639325fc354866dd4
(pid=56154)
FAIL: tests/test-throttle
make: *** [check-tests/test-throttle] Error 1
Backtrace:
#0 0x00007fff8d25c286 in __pthread_kill ()
#1 0x00007fff9495d42f in pthread_kill ()
#2 0x00007fff9635fb53 in abort ()
#3 0x00000001000c0890 in error_exit (err=22, msg=0x1000f0042
"qemu_mutex_lock") at
/Users/pm215/src/qemu/util/qemu-thread-posix.c:48
#4 0x00000001000c0904 in qemu_mutex_lock (mutex=0x1000fa560) at
/Users/pm215/src/qemu/util/qemu-thread-posix.c:75
#5 0x00000001000719b3 in throttle_group_incref (name=0x1000e0d20
"bar") at /Users/pm215/src/qemu/block/throttle-groups.c:86
#6 0x0000000100071829 in throttle_group_register_bs (bs=0x10100c400,
groupname=0x1000e0d20 "bar") at
/Users/pm215/src/qemu/block/throttle-groups.c:399
#7 0x0000000100002c89 in test_groups () at
/Users/pm215/src/qemu/tests/test-throttle.c:519
#8 0x00000001005d891d in g_test_run_suite_internal ()
#9 0x00000001005d8ae1 in g_test_run_suite_internal ()
#10 0x00000001005d8198 in g_test_run_suite ()
#11 0x00000001000012da in main (argc=1, argv=0x7fff5fbff8e0) at
/Users/pm215/src/qemu/tests/test-throttle.c:595
OSX errno 22 is EINVAL.
As far as I can tell nothing is calling throttle_groups_init() and
so the mutex hasn't been initialized when the test code tries to
lock it.
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2015-07-02 9:19 Stefan Hajnoczi
2015-07-02 12:46 ` Peter Maydell
2015-07-07 13:47 ` Peter Maydell
0 siblings, 2 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2015-07-02 9:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Peter Maydell, Stefan Hajnoczi
The following changes since commit d2966f804d70a244f5dde395fc5d22a50ed3e74e:
Merge remote-tracking branch 'remotes/vivier/tags/pull-m68k-20150629' into staging (2015-06-29 17:03:20 +0100)
are available in the git repository at:
git://github.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to 764ba3ae511adddfa750db290ac8375d660ca5b9:
block: remove redundant check before g_slist_find() (2015-07-02 10:06:23 +0100)
----------------------------------------------------------------
----------------------------------------------------------------
Alberto Garcia (3):
timer: Move NANOSECONDS_PER_SECONDS to timer.h
timer: Use a single definition of NSEC_PER_SEC for the whole codebase
block: remove redundant check before g_slist_find()
Fam Zheng (8):
block: Add bdrv_get_block_status_above
qmp: Add optional bool "unmap" to drive-mirror
mirror: Do zero write on target if sectors not allocated
block: Fix dirty bitmap in bdrv_co_discard
block: Remove bdrv_reset_dirty
qemu-iotests: Make block job methods common
qemu-iotests: Add test case for mirror with unmap
iotests: Use event_wait in wait_ready
Jindřich Makovička (1):
qcow2: Handle EAGAIN returned from update_refcount
John Snow (1):
qapi: Rename 'dirty-bitmap' mode to 'incremental'
Paolo Bonzini (1):
blockdev: no need to drain+flush in hmp_drive_del
Peter Lieven (3):
block/iscsi: add support for request timeouts
block/iscsi: restore compatiblity with libiscsi 1.9.0
block/nfs: limit maximum readahead size to 1MB
block.c | 12 -----
block/backup.c | 10 ++--
block/io.c | 62 +++++++++++++++++------
block/iscsi.c | 111 +++++++++++++++++++++++++++++++++---------
block/mirror.c | 32 +++++++++---
block/nfs.c | 7 +++
block/qcow2-refcount.c | 22 +++++----
blockdev.c | 8 +--
docs/bitmaps.md | 8 +--
hmp.c | 2 +-
hw/ppc/ppc.c | 2 -
hw/ppc/spapr_rtc.c | 3 +-
hw/timer/mc146818rtc.c | 1 -
hw/usb/hcd-ehci.c | 2 +-
include/block/block.h | 4 ++
include/block/block_int.h | 6 +--
include/qemu/throttle.h | 2 -
include/qemu/timer.h | 2 +
qapi/block-core.json | 16 ++++--
qemu-options.hx | 5 ++
qmp-commands.hx | 9 ++--
tests/qemu-iotests/041 | 66 ++++++-------------------
tests/qemu-iotests/124 | 6 +--
tests/qemu-iotests/132 | 59 ++++++++++++++++++++++
tests/qemu-iotests/132.out | 5 ++
tests/qemu-iotests/group | 1 +
tests/qemu-iotests/iotests.py | 23 +++++++++
tests/rtl8139-test.c | 10 ++--
tests/test-throttle.c | 8 +--
tests/wdt_ib700-test.c | 15 +++---
util/throttle.c | 4 +-
31 files changed, 353 insertions(+), 170 deletions(-)
create mode 100644 tests/qemu-iotests/132
create mode 100644 tests/qemu-iotests/132.out
--
2.4.3
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2015-07-02 9:19 Stefan Hajnoczi
@ 2015-07-02 12:46 ` Peter Maydell
2015-07-07 13:47 ` Peter Maydell
1 sibling, 0 replies; 41+ messages in thread
From: Peter Maydell @ 2015-07-02 12:46 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: Kevin Wolf, QEMU Developers
On 2 July 2015 at 10:19, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit d2966f804d70a244f5dde395fc5d22a50ed3e74e:
>
> Merge remote-tracking branch 'remotes/vivier/tags/pull-m68k-20150629' into staging (2015-06-29 17:03:20 +0100)
>
> are available in the git repository at:
>
> git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 764ba3ae511adddfa750db290ac8375d660ca5b9:
>
> block: remove redundant check before g_slist_find() (2015-07-02 10:06:23 +0100)
>
> ----------------------------------------------------------------
>
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2015-07-02 9:19 Stefan Hajnoczi
2015-07-02 12:46 ` Peter Maydell
@ 2015-07-07 13:47 ` Peter Maydell
2015-07-08 13:52 ` Stefan Hajnoczi
1 sibling, 1 reply; 41+ messages in thread
From: Peter Maydell @ 2015-07-07 13:47 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: Kevin Wolf, QEMU Developers
On 2 July 2015 at 10:19, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit d2966f804d70a244f5dde395fc5d22a50ed3e74e:
>
> Merge remote-tracking branch 'remotes/vivier/tags/pull-m68k-20150629' into staging (2015-06-29 17:03:20 +0100)
>
> are available in the git repository at:
>
> git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 764ba3ae511adddfa750db290ac8375d660ca5b9:
>
> block: remove redundant check before g_slist_find() (2015-07-02 10:06:23 +0100)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Alberto Garcia (3):
> timer: Move NANOSECONDS_PER_SECONDS to timer.h
> timer: Use a single definition of NSEC_PER_SEC for the whole codebase
I've just noticed that this clashes with the OSX standard
headers:
In file included from /Users/pm215/src/qemu/ui/cocoa.m:31:
In file included from /Users/pm215/src/qemu/include/sysemu/sysemu.h:8:
/Users/pm215/src/qemu/include/qemu/timer.h:8:9: warning:
'NSEC_PER_SEC' macro redefined [-Wmacro-redefined]
#define NSEC_PER_SEC 1000000000LL
^
/usr/include/dispatch/time.h:48:9: note: previous definition is here
#define NSEC_PER_SEC 1000000000ull
Can we use some other constant name, please?
thanks
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2015-07-07 13:47 ` Peter Maydell
@ 2015-07-08 13:52 ` Stefan Hajnoczi
0 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2015-07-08 13:52 UTC (permalink / raw)
To: Peter Maydell; +Cc: Kevin Wolf, QEMU Developers, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]
On Tue, Jul 07, 2015 at 02:47:50PM +0100, Peter Maydell wrote:
> On 2 July 2015 at 10:19, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > The following changes since commit d2966f804d70a244f5dde395fc5d22a50ed3e74e:
> >
> > Merge remote-tracking branch 'remotes/vivier/tags/pull-m68k-20150629' into staging (2015-06-29 17:03:20 +0100)
> >
> > are available in the git repository at:
> >
> > git://github.com/stefanha/qemu.git tags/block-pull-request
> >
> > for you to fetch changes up to 764ba3ae511adddfa750db290ac8375d660ca5b9:
> >
> > block: remove redundant check before g_slist_find() (2015-07-02 10:06:23 +0100)
> >
> > ----------------------------------------------------------------
> >
> > ----------------------------------------------------------------
> >
> > Alberto Garcia (3):
> > timer: Move NANOSECONDS_PER_SECONDS to timer.h
> > timer: Use a single definition of NSEC_PER_SEC for the whole codebase
>
> I've just noticed that this clashes with the OSX standard
> headers:
>
> In file included from /Users/pm215/src/qemu/ui/cocoa.m:31:
> In file included from /Users/pm215/src/qemu/include/sysemu/sysemu.h:8:
> /Users/pm215/src/qemu/include/qemu/timer.h:8:9: warning:
> 'NSEC_PER_SEC' macro redefined [-Wmacro-redefined]
> #define NSEC_PER_SEC 1000000000LL
> ^
> /usr/include/dispatch/time.h:48:9: note: previous definition is here
> #define NSEC_PER_SEC 1000000000ull
>
> Can we use some other constant name, please?
Alberto is on vacation so I guess I'm on the hook to fix this.
[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2016-01-20 16:24 Kevin Wolf
2016-01-21 13:42 ` Peter Maydell
0 siblings, 1 reply; 41+ messages in thread
From: Kevin Wolf @ 2016-01-20 16:24 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The following changes since commit 3db34bf64ab4f8797565dd8750003156c32b301d:
Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' into staging (2016-01-18 17:40:50 +0000)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to e9b155501da2638e4e319af9960d35da1bc8662b:
iotests: Test that throttle values ranges (2016-01-20 13:37:57 +0100)
----------------------------------------------------------------
Block layer patches
----------------------------------------------------------------
Christian Borntraeger (1):
block/raw-posix: avoid bogus fixup for cylinders on DASD disks
Eric Blake (1):
qemu-iotests: Reduce racy output in 028
Fam Zheng (4):
qemu-img: Speed up comparing empty/zero images
vmdk: Create streamOptimized as version 3
blockdev: Error out on negative throttling option values
iotests: Test that throttle values ranges
Kevin Wolf (10):
block: Fix .bdrv_open flags
qcow2: Write feature table only for v3 images
qcow2: Write full header on image creation
block: Assert no write requests under BDRV_O_INCOMING
block: Fix error path in bdrv_invalidate_cache()
block: Rename BDRV_O_INCOMING to BDRV_O_INACTIVE
block: Inactivate BDS when migration completes
qcow2: Implement .bdrv_inactivate
qcow2: Fix BDRV_O_INACTIVE handling in qcow2_invalidate_cache()
qcow2: Make image inaccessible after failed qcow2_invalidate_cache()
Peter Maydell (1):
block: Clean up includes
block.c | 55 ++++++++++++++++----
block/accounting.c | 1 +
block/archipelago.c | 2 +-
block/backup.c | 4 +-
block/blkdebug.c | 1 +
block/blkverify.c | 2 +-
block/block-backend.c | 1 +
block/bochs.c | 1 +
block/cloop.c | 1 +
block/commit.c | 1 +
block/curl.c | 1 +
block/dmg.c | 1 +
block/gluster.c | 1 +
block/io.c | 3 ++
block/iscsi.c | 2 +-
block/linux-aio.c | 1 +
block/mirror.c | 1 +
block/nbd-client.c | 1 +
block/nbd.c | 3 +-
block/nfs.c | 2 +-
block/null.c | 1 +
block/parallels.c | 1 +
block/qapi.c | 1 +
block/qcow.c | 1 +
block/qcow2-cache.c | 3 +-
block/qcow2-cluster.c | 1 +
block/qcow2-refcount.c | 1 +
block/qcow2-snapshot.c | 1 +
block/qcow2.c | 111 ++++++++++++++++++++++++----------------
block/qed-check.c | 1 +
block/qed-cluster.c | 1 +
block/qed-gencb.c | 1 +
block/qed-l2-cache.c | 1 +
block/qed-table.c | 1 +
block/qed.c | 5 +-
block/quorum.c | 1 +
block/raw-posix.c | 10 +---
block/raw-win32.c | 1 +
block/raw_bsd.c | 1 +
block/rbd.c | 2 +-
block/sheepdog.c | 1 +
block/snapshot.c | 1 +
block/ssh.c | 4 +-
block/stream.c | 1 +
block/throttle-groups.c | 1 +
block/vdi.c | 1 +
block/vhdx-endian.c | 1 +
block/vhdx-log.c | 1 +
block/vhdx.c | 1 +
block/vmdk.c | 9 +++-
block/vpc.c | 1 +
block/vvfat.c | 2 +-
block/win32-aio.c | 1 +
block/write-threshold.c | 1 +
blockdev.c | 3 +-
hw/block/block.c | 1 +
hw/block/cdrom.c | 1 +
hw/block/dataplane/virtio-blk.c | 1 +
hw/block/ecc.c | 1 +
hw/block/fdc.c | 1 +
hw/block/hd-geometry.c | 1 +
hw/block/m25p80.c | 1 +
hw/block/nvme.c | 1 +
hw/block/onenand.c | 1 +
hw/block/pflash_cfi01.c | 1 +
hw/block/pflash_cfi02.c | 1 +
hw/block/tc58128.c | 1 +
hw/block/virtio-blk.c | 1 +
hw/block/xen_disk.c | 12 +----
include/block/block.h | 3 +-
include/block/block_int.h | 1 +
include/qemu/throttle.h | 2 +
migration/migration.c | 7 +++
nbd/server.c | 2 +-
qemu-img.c | 47 +++++++++++------
qemu-io-cmds.c | 1 +
qemu-io.c | 5 +-
qmp.c | 12 +++++
tests/qemu-iotests/028 | 6 ++-
tests/qemu-iotests/028.out | 3 --
tests/qemu-iotests/031.out | 17 +++---
tests/qemu-iotests/036 | 2 +
tests/qemu-iotests/036.out | 5 ++
tests/qemu-iotests/051 | 18 +++++++
tests/qemu-iotests/051.out | 39 ++++++++++++++
tests/qemu-iotests/051.pc.out | 39 ++++++++++++++
tests/qemu-iotests/061.out | 35 +++++++------
util/throttle.c | 16 +++---
88 files changed, 390 insertions(+), 155 deletions(-)
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2016-01-20 16:24 Kevin Wolf
@ 2016-01-21 13:42 ` Peter Maydell
0 siblings, 0 replies; 41+ messages in thread
From: Peter Maydell @ 2016-01-21 13:42 UTC (permalink / raw)
To: Kevin Wolf; +Cc: QEMU Developers, Qemu-block
On 20 January 2016 at 16:24, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit 3db34bf64ab4f8797565dd8750003156c32b301d:
>
> Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' into staging (2016-01-18 17:40:50 +0000)
>
> are available in the git repository at:
>
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to e9b155501da2638e4e319af9960d35da1bc8662b:
>
> iotests: Test that throttle values ranges (2016-01-20 13:37:57 +0100)
>
> ----------------------------------------------------------------
> Block layer patches
>
> ----------------------------------------------------------------
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2016-09-12 14:08 Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 01/17] linux-aio: consume events in userspace instead of calling io_getevents Stefan Hajnoczi
` (17 more replies)
0 siblings, 18 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi
The following changes since commit c2a57aae9a1c3dd7de77daf5478df10379aeeebf:
Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging (2016-09-09 12:49:41 +0100)
are available in the git repository at:
git://github.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to 8f1096787517c66b67cc29bab65edc0188a86326:
tests: fix qvirtqueue_kick (2016-09-12 15:06:29 +0100)
----------------------------------------------------------------
----------------------------------------------------------------
Changlong Xie (6):
virtio-blk: rename virtio_device_info to virtio_blk_info
Backup: export interfaces for extra serialization
configure: support replication
replication: Introduce new APIs to do replication operation
tests: add unit test case for replication
MAINTAINERS: add maintainer for replication
Laurent Vivier (1):
tests: fix qvirtqueue_kick
Roman Pen (3):
linux-aio: consume events in userspace instead of calling io_getevents
linux-aio: split processing events function
linux-aio: process completions from ioq_submit()
Wen Congyang (7):
block: unblock backup operations in backing file
Backup: clear all bitmap when doing block checkpoint
block: Link backup into block core
docs: block replication's description
mirror: auto complete active commit
replication: Implement new driver for block replication
support replication driver in blockdev-add
MAINTAINERS | 9 +
Makefile.objs | 1 +
block.c | 17 ++
block/Makefile.objs | 3 +-
block/backup.c | 59 +++-
block/linux-aio.c | 184 +++++++++---
block/mirror.c | 13 +-
block/replication.c | 659 +++++++++++++++++++++++++++++++++++++++++++
blockdev.c | 2 +-
configure | 11 +
docs/block-replication.txt | 239 ++++++++++++++++
hw/block/virtio-blk.c | 4 +-
include/block/block_backup.h | 39 +++
include/block/block_int.h | 3 +-
qapi/block-core.json | 36 ++-
qemu-img.c | 2 +-
replication.c | 107 +++++++
replication.h | 174 ++++++++++++
tests/.gitignore | 1 +
tests/Makefile.include | 4 +
tests/libqos/virtio.c | 6 +-
tests/test-replication.c | 575 +++++++++++++++++++++++++++++++++++++
22 files changed, 2086 insertions(+), 62 deletions(-)
create mode 100644 block/replication.c
create mode 100644 docs/block-replication.txt
create mode 100644 include/block/block_backup.h
create mode 100644 replication.c
create mode 100644 replication.h
create mode 100644 tests/test-replication.c
--
2.7.4
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 01/17] linux-aio: consume events in userspace instead of calling io_getevents
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 02/17] linux-aio: split processing events function Stefan Hajnoczi
` (16 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Roman Pen, Stefan Hajnoczi, Paolo Bonzini
From: Roman Pen <roman.penyaev@profitbricks.com>
AIO context in userspace is represented as a simple ring buffer, which
can be consumed directly without entering the kernel, which obviously
can bring some performance gain. QEMU does not use timeout value for
waiting for events completions, so we can consume all events from
userspace.
Signed-off-by: Roman Pen <roman.penyaev@profitbricks.com>
Message-id: 1468931263-32667-2-git-send-email-roman.penyaev@profitbricks.com
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: qemu-devel@nongnu.org
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/linux-aio.c | 129 ++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 101 insertions(+), 28 deletions(-)
diff --git a/block/linux-aio.c b/block/linux-aio.c
index e906abe..62ee1ea 100644
--- a/block/linux-aio.c
+++ b/block/linux-aio.c
@@ -59,7 +59,6 @@ struct LinuxAioState {
/* I/O completion processing */
QEMUBH *completion_bh;
- struct io_event events[MAX_EVENTS];
int event_idx;
int event_max;
};
@@ -102,6 +101,85 @@ static void qemu_laio_process_completion(struct qemu_laiocb *laiocb)
}
}
+/**
+ * aio_ring buffer which is shared between userspace and kernel.
+ *
+ * This copied from linux/fs/aio.c, common header does not exist
+ * but AIO exists for ages so we assume ABI is stable.
+ */
+struct aio_ring {
+ unsigned id; /* kernel internal index number */
+ unsigned nr; /* number of io_events */
+ unsigned head; /* Written to by userland or by kernel. */
+ unsigned tail;
+
+ unsigned magic;
+ unsigned compat_features;
+ unsigned incompat_features;
+ unsigned header_length; /* size of aio_ring */
+
+ struct io_event io_events[0];
+};
+
+/**
+ * io_getevents_peek:
+ * @ctx: AIO context
+ * @events: pointer on events array, output value
+
+ * Returns the number of completed events and sets a pointer
+ * on events array. This function does not update the internal
+ * ring buffer, only reads head and tail. When @events has been
+ * processed io_getevents_commit() must be called.
+ */
+static inline unsigned int io_getevents_peek(io_context_t ctx,
+ struct io_event **events)
+{
+ struct aio_ring *ring = (struct aio_ring *)ctx;
+ unsigned int head = ring->head, tail = ring->tail;
+ unsigned int nr;
+
+ nr = tail >= head ? tail - head : ring->nr - head;
+ *events = ring->io_events + head;
+ /* To avoid speculative loads of s->events[i] before observing tail.
+ Paired with smp_wmb() inside linux/fs/aio.c: aio_complete(). */
+ smp_rmb();
+
+ return nr;
+}
+
+/**
+ * io_getevents_commit:
+ * @ctx: AIO context
+ * @nr: the number of events on which head should be advanced
+ *
+ * Advances head of a ring buffer.
+ */
+static inline void io_getevents_commit(io_context_t ctx, unsigned int nr)
+{
+ struct aio_ring *ring = (struct aio_ring *)ctx;
+
+ if (nr) {
+ ring->head = (ring->head + nr) % ring->nr;
+ }
+}
+
+/**
+ * io_getevents_advance_and_peek:
+ * @ctx: AIO context
+ * @events: pointer on events array, output value
+ * @nr: the number of events on which head should be advanced
+ *
+ * Advances head of a ring buffer and returns number of elements left.
+ */
+static inline unsigned int
+io_getevents_advance_and_peek(io_context_t ctx,
+ struct io_event **events,
+ unsigned int nr)
+{
+ io_getevents_commit(ctx, nr);
+ return io_getevents_peek(ctx, events);
+}
+
/* The completion BH fetches completed I/O requests and invokes their
* callbacks.
*
@@ -116,43 +194,38 @@ static void qemu_laio_process_completion(struct qemu_laiocb *laiocb)
static void qemu_laio_completion_bh(void *opaque)
{
LinuxAioState *s = opaque;
-
- /* Fetch more completion events when empty */
- if (s->event_idx == s->event_max) {
- do {
- struct timespec ts = { 0 };
- s->event_max = io_getevents(s->ctx, MAX_EVENTS, MAX_EVENTS,
- s->events, &ts);
- } while (s->event_max == -EINTR);
-
- s->event_idx = 0;
- if (s->event_max <= 0) {
- s->event_max = 0;
- return; /* no more events */
- }
- s->io_q.in_flight -= s->event_max;
- }
+ struct io_event *events;
/* Reschedule so nested event loops see currently pending completions */
qemu_bh_schedule(s->completion_bh);
- /* Process completion events */
- while (s->event_idx < s->event_max) {
- struct iocb *iocb = s->events[s->event_idx].obj;
- struct qemu_laiocb *laiocb =
+ while ((s->event_max = io_getevents_advance_and_peek(s->ctx, &events,
+ s->event_idx))) {
+ for (s->event_idx = 0; s->event_idx < s->event_max; ) {
+ struct iocb *iocb = events[s->event_idx].obj;
+ struct qemu_laiocb *laiocb =
container_of(iocb, struct qemu_laiocb, iocb);
- laiocb->ret = io_event_ret(&s->events[s->event_idx]);
- s->event_idx++;
+ laiocb->ret = io_event_ret(&events[s->event_idx]);
- qemu_laio_process_completion(laiocb);
- }
-
- if (!s->io_q.plugged && !QSIMPLEQ_EMPTY(&s->io_q.pending)) {
- ioq_submit(s);
+ /* Change counters one-by-one because we can be nested. */
+ s->io_q.in_flight--;
+ s->event_idx++;
+ qemu_laio_process_completion(laiocb);
+ }
}
qemu_bh_cancel(s->completion_bh);
+
+ /* If we are nested we have to notify the level above that we are done
+ * by setting event_max to zero, upper level will then jump out of it's
+ * own `for` loop. If we are the last all counters droped to zero. */
+ s->event_max = 0;
+ s->event_idx = 0;
+
+ if (!s->io_q.plugged && !QSIMPLEQ_EMPTY(&s->io_q.pending)) {
+ ioq_submit(s);
+ }
}
static void qemu_laio_completion_cb(EventNotifier *e)
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 02/17] linux-aio: split processing events function
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 01/17] linux-aio: consume events in userspace instead of calling io_getevents Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 03/17] linux-aio: process completions from ioq_submit() Stefan Hajnoczi
` (15 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Roman Pen, Stefan Hajnoczi, Paolo Bonzini
From: Roman Pen <roman.penyaev@profitbricks.com>
Prepare processing events function to be called from ioq_submit(),
thus split function on two parts: the first harvests completed IO
requests, the second submits pending requests.
Signed-off-by: Roman Pen <roman.penyaev@profitbricks.com>
Message-id: 1468931263-32667-3-git-send-email-roman.penyaev@profitbricks.com
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: qemu-devel@nongnu.org
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/linux-aio.c | 31 +++++++++++++++++++++----------
1 file changed, 21 insertions(+), 10 deletions(-)
diff --git a/block/linux-aio.c b/block/linux-aio.c
index 62ee1ea..252ebef 100644
--- a/block/linux-aio.c
+++ b/block/linux-aio.c
@@ -180,20 +180,20 @@ io_getevents_advance_and_peek(io_context_t ctx,
return io_getevents_peek(ctx, events);
}
-/* The completion BH fetches completed I/O requests and invokes their
- * callbacks.
+/**
+ * qemu_laio_process_completions:
+ * @s: AIO state
+ *
+ * Fetches completed I/O requests and invokes their callbacks.
*
* The function is somewhat tricky because it supports nested event loops, for
* example when a request callback invokes aio_poll(). In order to do this,
- * the completion events array and index are kept in LinuxAioState. The BH
- * reschedules itself as long as there are completions pending so it will
- * either be called again in a nested event loop or will be called after all
- * events have been completed. When there are no events left to complete, the
- * BH returns without rescheduling.
+ * indices are kept in LinuxAioState. Function schedules BH completion so it
+ * can be called again in a nested event loop. When there are no events left
+ * to complete the BH is being canceled.
*/
-static void qemu_laio_completion_bh(void *opaque)
+static void qemu_laio_process_completions(LinuxAioState *s)
{
- LinuxAioState *s = opaque;
struct io_event *events;
/* Reschedule so nested event loops see currently pending completions */
@@ -222,18 +222,29 @@ static void qemu_laio_completion_bh(void *opaque)
* own `for` loop. If we are the last all counters droped to zero. */
s->event_max = 0;
s->event_idx = 0;
+}
+static void qemu_laio_process_completions_and_submit(LinuxAioState *s)
+{
+ qemu_laio_process_completions(s);
if (!s->io_q.plugged && !QSIMPLEQ_EMPTY(&s->io_q.pending)) {
ioq_submit(s);
}
}
+static void qemu_laio_completion_bh(void *opaque)
+{
+ LinuxAioState *s = opaque;
+
+ qemu_laio_process_completions_and_submit(s);
+}
+
static void qemu_laio_completion_cb(EventNotifier *e)
{
LinuxAioState *s = container_of(e, LinuxAioState, e);
if (event_notifier_test_and_clear(&s->e)) {
- qemu_laio_completion_bh(s);
+ qemu_laio_process_completions_and_submit(s);
}
}
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 03/17] linux-aio: process completions from ioq_submit()
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 01/17] linux-aio: consume events in userspace instead of calling io_getevents Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 02/17] linux-aio: split processing events function Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 04/17] virtio-blk: rename virtio_device_info to virtio_blk_info Stefan Hajnoczi
` (14 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Roman Pen, Stefan Hajnoczi, Paolo Bonzini
From: Roman Pen <roman.penyaev@profitbricks.com>
In order to reduce completion latency it makes sense to harvest completed
requests ASAP. Very fast backend device can complete requests just after
submission, so it is worth trying to check ring buffer in order to peek
completed requests directly after io_submit() has been called.
Indeed, this patch reduces the completions latencies and increases the
overall throughput, e.g. the following is the percentiles of number of
completed requests at once:
1th 10th 20th 30th 40th 50th 60th 70th 80th 90th 99.99th
Before 2 4 42 112 128 128 128 128 128 128 128
After 1 1 4 14 33 45 47 48 50 51 108
That means, that before the current patch is applied the ring buffer is
observed as full (128 requests were consumed at once) in 60% of calls.
After patch is applied the distribution of number of completed requests
is "smoother" and the queue (requests in-flight) is almost never full.
The fio read results are the following (write results are almost the
same and are not showed here):
Before
------
job: (groupid=0, jobs=8): err= 0: pid=2227: Tue Jul 19 11:29:50 2016
Description : [Emulation of Storage Server Access Pattern]
read : io=54681MB, bw=1822.7MB/s, iops=179779, runt= 30001msec
slat (usec): min=172, max=16883, avg=338.35, stdev=109.66
clat (usec): min=1, max=21977, avg=1051.45, stdev=299.29
lat (usec): min=317, max=22521, avg=1389.83, stdev=300.73
clat percentiles (usec):
| 1.00th=[ 346], 5.00th=[ 596], 10.00th=[ 708], 20.00th=[ 852],
| 30.00th=[ 932], 40.00th=[ 996], 50.00th=[ 1048], 60.00th=[ 1112],
| 70.00th=[ 1176], 80.00th=[ 1256], 90.00th=[ 1384], 95.00th=[ 1496],
| 99.00th=[ 1800], 99.50th=[ 1928], 99.90th=[ 2320], 99.95th=[ 2672],
| 99.99th=[ 4704]
bw (KB /s): min=205229, max=553181, per=12.50%, avg=233278.26, stdev=18383.51
After
------
job: (groupid=0, jobs=8): err= 0: pid=2220: Tue Jul 19 11:31:51 2016
Description : [Emulation of Storage Server Access Pattern]
read : io=57637MB, bw=1921.2MB/s, iops=189529, runt= 30002msec
slat (usec): min=169, max=20636, avg=329.61, stdev=124.18
clat (usec): min=2, max=19592, avg=988.78, stdev=251.04
lat (usec): min=381, max=21067, avg=1318.42, stdev=243.58
clat percentiles (usec):
| 1.00th=[ 310], 5.00th=[ 580], 10.00th=[ 748], 20.00th=[ 876],
| 30.00th=[ 908], 40.00th=[ 948], 50.00th=[ 1012], 60.00th=[ 1064],
| 70.00th=[ 1080], 80.00th=[ 1128], 90.00th=[ 1224], 95.00th=[ 1288],
| 99.00th=[ 1496], 99.50th=[ 1608], 99.90th=[ 1960], 99.95th=[ 2256],
| 99.99th=[ 5408]
bw (KB /s): min=212149, max=390160, per=12.49%, avg=245746.04, stdev=11606.75
Throughput increased from 1822MB/s to 1921MB/s, average completion latencies
decreased from 1051us to 988us.
Signed-off-by: Roman Pen <roman.penyaev@profitbricks.com>
Message-id: 1468931263-32667-4-git-send-email-roman.penyaev@profitbricks.com
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: qemu-devel@nongnu.org
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/linux-aio.c | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)
diff --git a/block/linux-aio.c b/block/linux-aio.c
index 252ebef..d4e19d4 100644
--- a/block/linux-aio.c
+++ b/block/linux-aio.c
@@ -94,7 +94,11 @@ static void qemu_laio_process_completion(struct qemu_laiocb *laiocb)
laiocb->ret = ret;
if (laiocb->co) {
- qemu_coroutine_enter(laiocb->co);
+ /* Jump and continue completion for foreign requests, don't do
+ * anything for current request, it will be completed shortly. */
+ if (laiocb->co != qemu_coroutine_self()) {
+ qemu_coroutine_enter(laiocb->co);
+ }
} else {
laiocb->common.cb(laiocb->common.opaque, ret);
qemu_aio_unref(laiocb);
@@ -320,6 +324,19 @@ static void ioq_submit(LinuxAioState *s)
QSIMPLEQ_SPLIT_AFTER(&s->io_q.pending, aiocb, next, &completed);
} while (ret == len && !QSIMPLEQ_EMPTY(&s->io_q.pending));
s->io_q.blocked = (s->io_q.in_queue > 0);
+
+ if (s->io_q.in_flight) {
+ /* We can try to complete something just right away if there are
+ * still requests in-flight. */
+ qemu_laio_process_completions(s);
+ /*
+ * Even we have completed everything (in_flight == 0), the queue can
+ * have still pended requests (in_queue > 0). We do not attempt to
+ * repeat submission to avoid IO hang. The reason is simple: s->e is
+ * still set and completion callback will be called shortly and all
+ * pended requests will be submitted from there.
+ */
+ }
}
void laio_io_plug(BlockDriverState *bs, LinuxAioState *s)
@@ -377,6 +394,7 @@ int coroutine_fn laio_co_submit(BlockDriverState *bs, LinuxAioState *s, int fd,
.co = qemu_coroutine_self(),
.nbytes = qiov->size,
.ctx = s,
+ .ret = -EINPROGRESS,
.is_read = (type == QEMU_AIO_READ),
.qiov = qiov,
};
@@ -386,7 +404,9 @@ int coroutine_fn laio_co_submit(BlockDriverState *bs, LinuxAioState *s, int fd,
return ret;
}
- qemu_coroutine_yield();
+ if (laiocb.ret == -EINPROGRESS) {
+ qemu_coroutine_yield();
+ }
return laiocb.ret;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 04/17] virtio-blk: rename virtio_device_info to virtio_blk_info
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (2 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 03/17] linux-aio: process completions from ioq_submit() Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 05/17] block: unblock backup operations in backing file Stefan Hajnoczi
` (13 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Changlong Xie, Stefan Hajnoczi
From: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
The old one is confusing with @virtio_device_info in virtio.c,
so make it more appropriate.
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Message-id: 1470214147-32560-1-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
hw/block/virtio-blk.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 331d766..3a6112f 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -992,7 +992,7 @@ static void virtio_blk_class_init(ObjectClass *klass, void *data)
vdc->load = virtio_blk_load_device;
}
-static const TypeInfo virtio_device_info = {
+static const TypeInfo virtio_blk_info = {
.name = TYPE_VIRTIO_BLK,
.parent = TYPE_VIRTIO_DEVICE,
.instance_size = sizeof(VirtIOBlock),
@@ -1002,7 +1002,7 @@ static const TypeInfo virtio_device_info = {
static void virtio_register_types(void)
{
- type_register_static(&virtio_device_info);
+ type_register_static(&virtio_blk_info);
}
type_init(virtio_register_types)
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 05/17] block: unblock backup operations in backing file
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (3 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 04/17] virtio-blk: rename virtio_device_info to virtio_blk_info Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 06/17] Backup: clear all bitmap when doing block checkpoint Stefan Hajnoczi
` (12 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Wen Congyang, Changlong Xie, Wang WeiWei,
Stefan Hajnoczi
From: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Kashyap Chamarthy <kchamart@redhat.com>
Message-id: 1469602913-20979-2-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/block.c b/block.c
index 101f8c6..66ed1c0 100644
--- a/block.c
+++ b/block.c
@@ -1312,6 +1312,23 @@ void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd)
/* Otherwise we won't be able to commit due to check in bdrv_commit */
bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_COMMIT_TARGET,
bs->backing_blocker);
+ /*
+ * We do backup in 3 ways:
+ * 1. drive backup
+ * The target bs is new opened, and the source is top BDS
+ * 2. blockdev backup
+ * Both the source and the target are top BDSes.
+ * 3. internal backup(used for block replication)
+ * Both the source and the target are backing file
+ *
+ * In case 1 and 2, neither the source nor the target is the backing file.
+ * In case 3, we will block the top BDS, so there is only one block job
+ * for the top BDS and its backing chain.
+ */
+ bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_SOURCE,
+ bs->backing_blocker);
+ bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_TARGET,
+ bs->backing_blocker);
out:
bdrv_refresh_limits(bs, NULL);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 06/17] Backup: clear all bitmap when doing block checkpoint
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (4 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 05/17] block: unblock backup operations in backing file Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 07/17] Backup: export interfaces for extra serialization Stefan Hajnoczi
` (11 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Wen Congyang, Changlong Xie, Wang WeiWei,
zhanghailiang, Gonglei, Stefan Hajnoczi
From: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1469602913-20979-3-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/backup.c | 18 ++++++++++++++++++
include/block/block_backup.h | 25 +++++++++++++++++++++++++
2 files changed, 43 insertions(+)
create mode 100644 include/block/block_backup.h
diff --git a/block/backup.c b/block/backup.c
index bb3bb9a..dbe54e2 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -17,6 +17,7 @@
#include "block/block.h"
#include "block/block_int.h"
#include "block/blockjob.h"
+#include "block/block_backup.h"
#include "qapi/error.h"
#include "qapi/qmp/qerror.h"
#include "qemu/ratelimit.h"
@@ -255,6 +256,23 @@ static void backup_attached_aio_context(BlockJob *job, AioContext *aio_context)
blk_set_aio_context(s->target, aio_context);
}
+void backup_do_checkpoint(BlockJob *job, Error **errp)
+{
+ BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
+ int64_t len;
+
+ assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP);
+
+ if (backup_job->sync_mode != MIRROR_SYNC_MODE_NONE) {
+ error_setg(errp, "The backup job only supports block checkpoint in"
+ " sync=none mode");
+ return;
+ }
+
+ len = DIV_ROUND_UP(backup_job->common.len, backup_job->cluster_size);
+ bitmap_zero(backup_job->done_bitmap, len);
+}
+
static const BlockJobDriver backup_job_driver = {
.instance_size = sizeof(BackupBlockJob),
.job_type = BLOCK_JOB_TYPE_BACKUP,
diff --git a/include/block/block_backup.h b/include/block/block_backup.h
new file mode 100644
index 0000000..157596c
--- /dev/null
+++ b/include/block/block_backup.h
@@ -0,0 +1,25 @@
+/*
+ * QEMU backup
+ *
+ * Copyright (c) 2013 Proxmox Server Solutions
+ * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ * Copyright (c) 2016 Intel Corporation
+ * Copyright (c) 2016 FUJITSU LIMITED
+ *
+ * Authors:
+ * Dietmar Maurer <dietmar@proxmox.com>
+ * Changlong Xie <xiecl.fnst@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef BLOCK_BACKUP_H
+#define BLOCK_BACKUP_H
+
+#include "block/block_int.h"
+
+void backup_do_checkpoint(BlockJob *job, Error **errp);
+
+#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 07/17] Backup: export interfaces for extra serialization
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (5 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 06/17] Backup: clear all bitmap when doing block checkpoint Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 08/17] block: Link backup into block core Stefan Hajnoczi
` (10 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Changlong Xie, Wen Congyang, Wang WeiWei,
Stefan Hajnoczi
From: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Normal backup(sync='none') workflow:
step 1. NBD peformance I/O write from client to server
qcow2_co_writev
bdrv_co_writev
...
bdrv_aligned_pwritev
notifier_with_return_list_notify -> backup_do_cow
bdrv_driver_pwritev // write new contents
step 2. drive-backup sync=none
backup_do_cow
{
wait_for_overlapping_requests
cow_request_begin
for(; start < end; start++) {
bdrv_co_readv_no_serialising //read old contents from Secondary disk
bdrv_co_writev // write old contents to hidden-disk
}
cow_request_end
}
step 3. Then roll back to "step 1" to write new contents to Secondary disk.
And for replication, we must make sure that we only read the old contents from
Secondary disk in order to keep contents consistent.
1) Replication workflow of Secondary
virtio-blk
^
-------> 1 NBD |
|| server 3 replication
|| ^ ^
|| | backing backing |
|| Secondary disk 6<-------- hidden-disk 5 <-------- active-disk 4
|| | ^
|| '-------------------------'
|| drive-backup sync=none 2
Hence, we need these interfaces to implement coarse-grained serialization between
COW of Secondary disk and the read operation of replication.
Example codes about how to use them:
*#include "block/block_backup.h"
static coroutine_fn int xxx_co_readv()
{
CowRequest req;
BlockJob *job = secondary_disk->bs->job;
if (job) {
backup_wait_for_overlapping_requests(job, start, end);
backup_cow_request_begin(&req, job, start, end);
ret = bdrv_co_readv();
backup_cow_request_end(&req);
goto out;
}
ret = bdrv_co_readv();
out:
return ret;
}
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1469602913-20979-4-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/backup.c | 41 ++++++++++++++++++++++++++++++++++-------
include/block/block_backup.h | 14 ++++++++++++++
2 files changed, 48 insertions(+), 7 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index dbe54e2..582bd0f 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -28,13 +28,6 @@
#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
#define SLICE_TIME 100000000ULL /* ns */
-typedef struct CowRequest {
- int64_t start;
- int64_t end;
- QLIST_ENTRY(CowRequest) list;
- CoQueue wait_queue; /* coroutines blocked on this request */
-} CowRequest;
-
typedef struct BackupBlockJob {
BlockJob common;
BlockBackend *target;
@@ -273,6 +266,40 @@ void backup_do_checkpoint(BlockJob *job, Error **errp)
bitmap_zero(backup_job->done_bitmap, len);
}
+void backup_wait_for_overlapping_requests(BlockJob *job, int64_t sector_num,
+ int nb_sectors)
+{
+ BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
+ int64_t sectors_per_cluster = cluster_size_sectors(backup_job);
+ int64_t start, end;
+
+ assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP);
+
+ start = sector_num / sectors_per_cluster;
+ end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster);
+ wait_for_overlapping_requests(backup_job, start, end);
+}
+
+void backup_cow_request_begin(CowRequest *req, BlockJob *job,
+ int64_t sector_num,
+ int nb_sectors)
+{
+ BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
+ int64_t sectors_per_cluster = cluster_size_sectors(backup_job);
+ int64_t start, end;
+
+ assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP);
+
+ start = sector_num / sectors_per_cluster;
+ end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster);
+ cow_request_begin(req, backup_job, start, end);
+}
+
+void backup_cow_request_end(CowRequest *req)
+{
+ cow_request_end(req);
+}
+
static const BlockJobDriver backup_job_driver = {
.instance_size = sizeof(BackupBlockJob),
.job_type = BLOCK_JOB_TYPE_BACKUP,
diff --git a/include/block/block_backup.h b/include/block/block_backup.h
index 157596c..8a75947 100644
--- a/include/block/block_backup.h
+++ b/include/block/block_backup.h
@@ -20,6 +20,20 @@
#include "block/block_int.h"
+typedef struct CowRequest {
+ int64_t start;
+ int64_t end;
+ QLIST_ENTRY(CowRequest) list;
+ CoQueue wait_queue; /* coroutines blocked on this request */
+} CowRequest;
+
+void backup_wait_for_overlapping_requests(BlockJob *job, int64_t sector_num,
+ int nb_sectors);
+void backup_cow_request_begin(CowRequest *req, BlockJob *job,
+ int64_t sector_num,
+ int nb_sectors);
+void backup_cow_request_end(CowRequest *req);
+
void backup_do_checkpoint(BlockJob *job, Error **errp);
#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 08/17] block: Link backup into block core
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (6 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 07/17] Backup: export interfaces for extra serialization Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 09/17] docs: block replication's description Stefan Hajnoczi
` (9 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Wen Congyang, Changlong Xie, Wang WeiWei,
zhanghailiang, Gonglei, Stefan Hajnoczi
From: Wen Congyang <wency@cn.fujitsu.com>
Some programs that add a dependency on it will use
the block layer directly.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 1469602913-20979-5-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/Makefile.objs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 2593a2f..8a3270b 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -22,11 +22,11 @@ block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
block-obj-$(CONFIG_LIBSSH2) += ssh.o
block-obj-y += accounting.o dirty-bitmap.o
block-obj-y += write-threshold.o
+block-obj-y += backup.o
block-obj-y += crypto.o
common-obj-y += stream.o
-common-obj-y += backup.o
iscsi.o-cflags := $(LIBISCSI_CFLAGS)
iscsi.o-libs := $(LIBISCSI_LIBS)
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 09/17] docs: block replication's description
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (7 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 08/17] block: Link backup into block core Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 10/17] mirror: auto complete active commit Stefan Hajnoczi
` (8 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Wen Congyang, Changlong Xie, Wang WeiWei,
zhanghailiang, Gonglei, Stefan Hajnoczi
From: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1469602913-20979-6-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
docs/block-replication.txt | 239 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 239 insertions(+)
create mode 100644 docs/block-replication.txt
diff --git a/docs/block-replication.txt b/docs/block-replication.txt
new file mode 100644
index 0000000..6bde673
--- /dev/null
+++ b/docs/block-replication.txt
@@ -0,0 +1,239 @@
+Block replication
+----------------------------------------
+Copyright Fujitsu, Corp. 2016
+Copyright (c) 2016 Intel Corporation
+Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+
+This work is licensed under the terms of the GNU GPL, version 2 or later.
+See the COPYING file in the top-level directory.
+
+Block replication is used for continuous checkpoints. It is designed
+for COLO (COarse-grain LOck-stepping) where the Secondary VM is running.
+It can also be applied for FT/HA (Fault-tolerance/High Assurance) scenario,
+where the Secondary VM is not running.
+
+This document gives an overview of block replication's design.
+
+== Background ==
+High availability solutions such as micro checkpoint and COLO will do
+consecutive checkpoints. The VM state of the Primary and Secondary VM is
+identical right after a VM checkpoint, but becomes different as the VM
+executes till the next checkpoint. To support disk contents checkpoint,
+the modified disk contents in the Secondary VM must be buffered, and are
+only dropped at next checkpoint time. To reduce the network transportation
+effort during a vmstate checkpoint, the disk modification operations of
+the Primary disk are asynchronously forwarded to the Secondary node.
+
+== Workflow ==
+The following is the image of block replication workflow:
+
+ +----------------------+ +------------------------+
+ |Primary Write Requests| |Secondary Write Requests|
+ +----------------------+ +------------------------+
+ | |
+ | (4)
+ | V
+ | /-------------\
+ | Copy and Forward | |
+ |---------(1)----------+ | Disk Buffer |
+ | | | |
+ | (3) \-------------/
+ | speculative ^
+ | write through (2)
+ | | |
+ V V |
+ +--------------+ +----------------+
+ | Primary Disk | | Secondary Disk |
+ +--------------+ +----------------+
+
+ 1) Primary write requests will be copied and forwarded to Secondary
+ QEMU.
+ 2) Before Primary write requests are written to Secondary disk, the
+ original sector content will be read from Secondary disk and
+ buffered in the Disk buffer, but it will not overwrite the existing
+ sector content (it could be from either "Secondary Write Requests" or
+ previous COW of "Primary Write Requests") in the Disk buffer.
+ 3) Primary write requests will be written to Secondary disk.
+ 4) Secondary write requests will be buffered in the Disk buffer and it
+ will overwrite the existing sector content in the buffer.
+
+== Architecture ==
+We are going to implement block replication from many basic
+blocks that are already in QEMU.
+
+ virtio-blk ||
+ ^ || .----------
+ | || | Secondary
+ 1 Quorum || '----------
+ / \ ||
+ / \ ||
+ Primary 2 filter
+ disk ^ virtio-blk
+ | ^
+ 3 NBD -------> 3 NBD |
+ client || server 2 filter
+ || ^ ^
+--------. || | |
+Primary | || Secondary disk <--------- hidden-disk 5 <--------- active-disk 4
+--------' || | backing ^ backing
+ || | |
+ || | |
+ || '-------------------------'
+ || drive-backup sync=none 6
+
+1) The disk on the primary is represented by a block device with two
+children, providing replication between a primary disk and the host that
+runs the secondary VM. The read pattern (fifo) for quorum can be extended
+to make the primary always read from the local disk instead of going through
+NBD.
+
+2) The new block filter (the name is replication) will control the block
+replication.
+
+3) The secondary disk receives writes from the primary VM through QEMU's
+embedded NBD server (speculative write-through).
+
+4) The disk on the secondary is represented by a custom block device
+(called active-disk). It should start as an empty disk, and the format
+should support bdrv_make_empty() and backing file.
+
+5) The hidden-disk is created automatically. It buffers the original content
+that is modified by the primary VM. It should also start as an empty disk,
+and the driver supports bdrv_make_empty() and backing file.
+
+6) The drive-backup job (sync=none) is run to allow hidden-disk to buffer
+any state that would otherwise be lost by the speculative write-through
+of the NBD server into the secondary disk. So before block replication,
+the primary disk and secondary disk should contain the same data.
+
+== Failure Handling ==
+There are 7 internal errors when block replication is running:
+1. I/O error on primary disk
+2. Forwarding primary write requests failed
+3. Backup failed
+4. I/O error on secondary disk
+5. I/O error on active disk
+6. Making active disk or hidden disk empty failed
+7. Doing failover failed
+In case 1 and 5, we just report the error to the disk layer. In case 2, 3,
+4 and 6, we just report block replication's error to FT/HA manager (which
+decides when to do a new checkpoint, when to do failover).
+In case 7, if active commit failed, we use replication failover failed state
+in Secondary's write operation (what decides which target to write).
+
+== New block driver interface ==
+We add four block driver interfaces to control block replication:
+a. replication_start_all()
+ Start block replication, called in migration/checkpoint thread.
+ We must call block_replication_start_all() in secondary QEMU before
+ calling block_replication_start_all() in primary QEMU. The caller
+ must hold the I/O mutex lock if it is in migration/checkpoint
+ thread.
+b. replication_do_checkpoint_all()
+ This interface is called after all VM state is transferred to
+ Secondary QEMU. The Disk buffer will be dropped in this interface.
+ The caller must hold the I/O mutex lock if it is in migration/checkpoint
+ thread.
+c. replication_get_error_all()
+ This interface is called to check if error happened in replication.
+ The caller must hold the I/O mutex lock if it is in migration/checkpoint
+ thread.
+d. replication_stop_all()
+ It is called on failover. We will flush the Disk buffer into
+ Secondary Disk and stop block replication. The vm should be stopped
+ before calling it if you use this API to shutdown the guest, or other
+ things except failover. The caller must hold the I/O mutex lock if it is
+ in migration/checkpoint thread.
+
+== Usage ==
+Primary:
+ -drive if=xxx,driver=quorum,read-pattern=fifo,id=colo1,vote-threshold=1,\
+ children.0.file.filename=1.raw,\
+ children.0.driver=raw
+
+ Run qmp command in primary qemu:
+ { 'execute': 'human-monitor-command',
+ 'arguments': {
+ 'command-line': 'drive_add -n buddy driver=replication,mode=primary,file.driver=nbd,file.host=xxxx,file.port=xxxx,file.export=colo1,node-name=nbd_client1'
+ }
+ }
+ { 'execute': 'x-blockdev-change',
+ 'arguments': {
+ 'parent': 'colo1',
+ 'node': 'nbd_client1'
+ }
+ }
+ Note:
+ 1. There should be only one NBD Client for each primary disk.
+ 2. host is the secondary physical machine's hostname or IP
+ 3. Each disk must have its own export name.
+ 4. It is all a single argument to -drive and you should ignore the
+ leading whitespace.
+ 5. The qmp command line must be run after running qmp command line in
+ secondary qemu.
+ 6. After failover we need remove children.1 (replication driver).
+
+Secondary:
+ -drive if=none,driver=raw,file.filename=1.raw,id=colo1 \
+ -drive if=xxx,id=topxxx,driver=replication,mode=secondary,top-id=topxxx\
+ file.file.filename=active_disk.qcow2,\
+ file.driver=qcow2,\
+ file.backing.file.filename=hidden_disk.qcow2,\
+ file.backing.driver=qcow2,\
+ file.backing.backing=colo1
+
+ Then run qmp command in secondary qemu:
+ { 'execute': 'nbd-server-start',
+ 'arguments': {
+ 'addr': {
+ 'type': 'inet',
+ 'data': {
+ 'host': 'xxx',
+ 'port': 'xxx'
+ }
+ }
+ }
+ }
+ { 'execute': 'nbd-server-add',
+ 'arguments': {
+ 'device': 'colo1',
+ 'writable': true
+ }
+ }
+
+ Note:
+ 1. The export name in secondary QEMU command line is the secondary
+ disk's id.
+ 2. The export name for the same disk must be the same
+ 3. The qmp command nbd-server-start and nbd-server-add must be run
+ before running the qmp command migrate on primary QEMU
+ 4. Active disk, hidden disk and nbd target's length should be the
+ same.
+ 5. It is better to put active disk and hidden disk in ramdisk.
+ 6. It is all a single argument to -drive, and you should ignore
+ the leading whitespace.
+
+After Failover:
+Primary:
+ The secondary host is down, so we should run the following qmp command
+ to remove the nbd child from the quorum:
+ { 'execute': 'x-blockdev-change',
+ 'arguments': {
+ 'parent': 'colo1',
+ 'child': 'children.1'
+ }
+ }
+ { 'execute': 'human-monitor-command',
+ 'arguments': {
+ 'command-line': 'drive_del xxxx'
+ }
+ }
+ Note: there is no qmp command to remove the blockdev now
+
+Secondary:
+ The primary host is down, so we should do the following thing:
+ { 'execute': 'nbd-server-stop' }
+
+TODO:
+1. Continuous block replication
+2. Shared disk
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 10/17] mirror: auto complete active commit
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (8 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 09/17] docs: block replication's description Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 11/17] configure: support replication Stefan Hajnoczi
` (7 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Wen Congyang, Changlong Xie, Wang WeiWei,
Stefan Hajnoczi
From: Wen Congyang <wency@cn.fujitsu.com>
Auto complete mirror job in background to prevent from
blocking synchronously
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Message-id: 1469602913-20979-7-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/mirror.c | 13 +++++++++----
blockdev.c | 2 +-
include/block/block_int.h | 3 ++-
qemu-img.c | 2 +-
4 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index e0b3f41..f9d1fec 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -916,7 +916,8 @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs,
BlockCompletionFunc *cb,
void *opaque, Error **errp,
const BlockJobDriver *driver,
- bool is_none_mode, BlockDriverState *base)
+ bool is_none_mode, BlockDriverState *base,
+ bool auto_complete)
{
MirrorBlockJob *s;
@@ -952,6 +953,9 @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs,
s->granularity = granularity;
s->buf_size = ROUND_UP(buf_size, granularity);
s->unmap = unmap;
+ if (auto_complete) {
+ s->should_complete = true;
+ }
s->dirty_bitmap = bdrv_create_dirty_bitmap(bs, granularity, NULL, errp);
if (!s->dirty_bitmap) {
@@ -990,14 +994,15 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
mirror_start_job(job_id, bs, target, replaces,
speed, granularity, buf_size, backing_mode,
on_source_error, on_target_error, unmap, cb, opaque, errp,
- &mirror_job_driver, is_none_mode, base);
+ &mirror_job_driver, is_none_mode, base, false);
}
void commit_active_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, int64_t speed,
BlockdevOnError on_error,
BlockCompletionFunc *cb,
- void *opaque, Error **errp)
+ void *opaque, Error **errp,
+ bool auto_complete)
{
int64_t length, base_length;
int orig_base_flags;
@@ -1038,7 +1043,7 @@ void commit_active_start(const char *job_id, BlockDriverState *bs,
mirror_start_job(job_id, bs, base, NULL, speed, 0, 0,
MIRROR_LEAVE_BACKING_CHAIN,
on_error, on_error, false, cb, opaque, &local_err,
- &commit_active_job_driver, false, base);
+ &commit_active_job_driver, false, base, auto_complete);
if (local_err) {
error_propagate(errp, local_err);
goto error_restore_flags;
diff --git a/blockdev.c b/blockdev.c
index 97062e3..1399fbd 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3090,7 +3090,7 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device,
goto out;
}
commit_active_start(has_job_id ? job_id : NULL, bs, base_bs, speed,
- on_error, block_job_cb, bs, &local_err);
+ on_error, block_job_cb, bs, &local_err, false);
} else {
commit_start(has_job_id ? job_id : NULL, bs, base_bs, top_bs, speed,
on_error, block_job_cb, bs,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 0ca6a78..713cea6 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -702,13 +702,14 @@ void commit_start(const char *job_id, BlockDriverState *bs,
* @cb: Completion function for the job.
* @opaque: Opaque pointer value passed to @cb.
* @errp: Error object.
+ * @auto_complete: Auto complete the job.
*
*/
void commit_active_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, int64_t speed,
BlockdevOnError on_error,
BlockCompletionFunc *cb,
- void *opaque, Error **errp);
+ void *opaque, Error **errp, bool auto_complete);
/*
* mirror_start:
* @job_id: The id of the newly-created job, or %NULL to use the
diff --git a/qemu-img.c b/qemu-img.c
index 1090286..ea52486 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -921,7 +921,7 @@ static int img_commit(int argc, char **argv)
};
commit_active_start("commit", bs, base_bs, 0, BLOCKDEV_ON_ERROR_REPORT,
- common_block_job_cb, &cbi, &local_err);
+ common_block_job_cb, &cbi, &local_err, false);
if (local_err) {
goto done;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 11/17] configure: support replication
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (9 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 10/17] mirror: auto complete active commit Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 12/17] replication: Introduce new APIs to do replication operation Stefan Hajnoczi
` (6 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Changlong Xie, Wen Congyang, Wang WeiWei,
Stefan Hajnoczi
From: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
configure --(enable/disable)-replication to switch replication
support on/off, and it is on by default.
We later introduce replation support.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Message-id: 1469602913-20979-8-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
configure | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/configure b/configure
index 5a9bda1..ef9ace9 100755
--- a/configure
+++ b/configure
@@ -320,6 +320,7 @@ vhdx=""
numa=""
tcmalloc="no"
jemalloc="no"
+replication="yes"
# parse CC options first
for opt do
@@ -1150,6 +1151,10 @@ for opt do
;;
--enable-jemalloc) jemalloc="yes"
;;
+ --disable-replication) replication="no"
+ ;;
+ --enable-replication) replication="yes"
+ ;;
*)
echo "ERROR: unknown option $opt"
echo "Try '$0 --help' for more information"
@@ -1380,6 +1385,7 @@ disabled with --disable-FEATURE, default is enabled if available:
numa libnuma support
tcmalloc tcmalloc support
jemalloc jemalloc support
+ replication replication support
NOTE: The object files are built at the place where configure is launched
EOF
@@ -4920,6 +4926,7 @@ echo "NUMA host support $numa"
echo "tcmalloc support $tcmalloc"
echo "jemalloc support $jemalloc"
echo "avx2 optimization $avx2_opt"
+echo "replication support $replication"
if test "$sdl_too_old" = "yes"; then
echo "-> Your SDL version is too old - please upgrade to have SDL support"
@@ -5497,6 +5504,10 @@ if test "$have_rtnetlink" = "yes" ; then
echo "CONFIG_RTNETLINK=y" >> $config_host_mak
fi
+if test "$replication" = "yes" ; then
+ echo "CONFIG_REPLICATION=y" >> $config_host_mak
+fi
+
# Hold two types of flag:
# CONFIG_THREAD_SETNAME_BYTHREAD - we've got a way of setting the name on
# a thread we have a handle to
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 12/17] replication: Introduce new APIs to do replication operation
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (10 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 11/17] configure: support replication Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 13/17] replication: Implement new driver for block replication Stefan Hajnoczi
` (5 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Changlong Xie, Wen Congyang, Wang WeiWei,
zhanghailiang, Gonglei, Stefan Hajnoczi
From: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
This commit introduces six replication interfaces(for block, network etc).
Firstly we can use replication_(new/remove) to create/destroy replication
instances, then in migration we can use replication_(start/stop/do_checkpoint
/get_error)_all to handle all replication operations. More detail please
refer to replication.h
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Message-id: 1469602913-20979-9-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
Makefile.objs | 1 +
qapi/block-core.json | 13 ++++
replication.c | 107 +++++++++++++++++++++++++++++++
replication.h | 174 +++++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 295 insertions(+)
create mode 100644 replication.c
create mode 100644 replication.h
diff --git a/Makefile.objs b/Makefile.objs
index 6d5ddcf..7301544 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -15,6 +15,7 @@ block-obj-$(CONFIG_POSIX) += aio-posix.o
block-obj-$(CONFIG_WIN32) += aio-win32.o
block-obj-y += block/
block-obj-y += qemu-io-cmds.o
+block-obj-$(CONFIG_REPLICATION) += replication.o
block-obj-m = block/
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 31f9990..4755c75 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2163,6 +2163,19 @@
'*debug-level': 'int' } }
##
+# @ReplicationMode
+#
+# An enumeration of replication modes.
+#
+# @primary: Primary mode, the vm's state will be sent to secondary QEMU.
+#
+# @secondary: Secondary mode, receive the vm's state from primary QEMU.
+#
+# Since: 2.8
+##
+{ 'enum' : 'ReplicationMode', 'data' : [ 'primary', 'secondary' ] }
+
+##
# @BlockdevOptions
#
# Options for creating a block device. Many options are available for all
diff --git a/replication.c b/replication.c
new file mode 100644
index 0000000..be3a42f
--- /dev/null
+++ b/replication.c
@@ -0,0 +1,107 @@
+/*
+ * Replication filter
+ *
+ * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ * Copyright (c) 2016 Intel Corporation
+ * Copyright (c) 2016 FUJITSU LIMITED
+ *
+ * Author:
+ * Changlong Xie <xiecl.fnst@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "replication.h"
+
+static QLIST_HEAD(, ReplicationState) replication_states;
+
+ReplicationState *replication_new(void *opaque, ReplicationOps *ops)
+{
+ ReplicationState *rs;
+
+ assert(ops != NULL);
+ rs = g_new0(ReplicationState, 1);
+ rs->opaque = opaque;
+ rs->ops = ops;
+ QLIST_INSERT_HEAD(&replication_states, rs, node);
+
+ return rs;
+}
+
+void replication_remove(ReplicationState *rs)
+{
+ if (rs) {
+ QLIST_REMOVE(rs, node);
+ g_free(rs);
+ }
+}
+
+/*
+ * The caller of the function MUST make sure vm stopped
+ */
+void replication_start_all(ReplicationMode mode, Error **errp)
+{
+ ReplicationState *rs, *next;
+ Error *local_err = NULL;
+
+ QLIST_FOREACH_SAFE(rs, &replication_states, node, next) {
+ if (rs->ops && rs->ops->start) {
+ rs->ops->start(rs, mode, &local_err);
+ }
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+ }
+}
+
+void replication_do_checkpoint_all(Error **errp)
+{
+ ReplicationState *rs, *next;
+ Error *local_err = NULL;
+
+ QLIST_FOREACH_SAFE(rs, &replication_states, node, next) {
+ if (rs->ops && rs->ops->checkpoint) {
+ rs->ops->checkpoint(rs, &local_err);
+ }
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+ }
+}
+
+void replication_get_error_all(Error **errp)
+{
+ ReplicationState *rs, *next;
+ Error *local_err = NULL;
+
+ QLIST_FOREACH_SAFE(rs, &replication_states, node, next) {
+ if (rs->ops && rs->ops->get_error) {
+ rs->ops->get_error(rs, &local_err);
+ }
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+ }
+}
+
+void replication_stop_all(bool failover, Error **errp)
+{
+ ReplicationState *rs, *next;
+ Error *local_err = NULL;
+
+ QLIST_FOREACH_SAFE(rs, &replication_states, node, next) {
+ if (rs->ops && rs->ops->stop) {
+ rs->ops->stop(rs, failover, &local_err);
+ }
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+ }
+}
diff --git a/replication.h b/replication.h
new file mode 100644
index 0000000..ece6ca6
--- /dev/null
+++ b/replication.h
@@ -0,0 +1,174 @@
+/*
+ * Replication filter
+ *
+ * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ * Copyright (c) 2016 Intel Corporation
+ * Copyright (c) 2016 FUJITSU LIMITED
+ *
+ * Author:
+ * Changlong Xie <xiecl.fnst@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef REPLICATION_H
+#define REPLICATION_H
+
+#include "qemu/queue.h"
+
+typedef struct ReplicationOps ReplicationOps;
+typedef struct ReplicationState ReplicationState;
+
+/**
+ * SECTION:replication.h
+ * @title:Base Replication System
+ * @short_description: interfaces for handling replication
+ *
+ * The Replication Model provides a framework for handling Replication
+ *
+ * <example>
+ * <title>How to use replication interfaces</title>
+ * <programlisting>
+ * #include "replication.h"
+ *
+ * typedef struct BDRVReplicationState {
+ * ReplicationState *rs;
+ * } BDRVReplicationState;
+ *
+ * static void replication_start(ReplicationState *rs, ReplicationMode mode,
+ * Error **errp);
+ * static void replication_do_checkpoint(ReplicationState *rs, Error **errp);
+ * static void replication_get_error(ReplicationState *rs, Error **errp);
+ * static void replication_stop(ReplicationState *rs, bool failover,
+ * Error **errp);
+ *
+ * static ReplicationOps replication_ops = {
+ * .start = replication_start,
+ * .checkpoint = replication_do_checkpoint,
+ * .get_error = replication_get_error,
+ * .stop = replication_stop,
+ * }
+ *
+ * static int replication_open(BlockDriverState *bs, QDict *options,
+ * int flags, Error **errp)
+ * {
+ * BDRVReplicationState *s = bs->opaque;
+ * s->rs = replication_new(bs, &replication_ops);
+ * return 0;
+ * }
+ *
+ * static void replication_close(BlockDriverState *bs)
+ * {
+ * BDRVReplicationState *s = bs->opaque;
+ * replication_remove(s->rs);
+ * }
+ *
+ * BlockDriver bdrv_replication = {
+ * .format_name = "replication",
+ * .protocol_name = "replication",
+ * .instance_size = sizeof(BDRVReplicationState),
+ *
+ * .bdrv_open = replication_open,
+ * .bdrv_close = replication_close,
+ * };
+ *
+ * static void bdrv_replication_init(void)
+ * {
+ * bdrv_register(&bdrv_replication);
+ * }
+ *
+ * block_init(bdrv_replication_init);
+ * </programlisting>
+ * </example>
+ *
+ * We create an example about how to use replication interfaces in above.
+ * Then in migration, we can use replication_(start/stop/do_checkpoint/
+ * get_error)_all to handle all replication operations.
+ */
+
+/**
+ * ReplicationState:
+ * @opaque: opaque pointer value passed to this ReplicationState
+ * @ops: replication operation of this ReplicationState
+ * @node: node that we will insert into @replication_states QLIST
+ */
+struct ReplicationState {
+ void *opaque;
+ ReplicationOps *ops;
+ QLIST_ENTRY(ReplicationState) node;
+};
+
+/**
+ * ReplicationOps:
+ * @start: callback to start replication
+ * @stop: callback to stop replication
+ * @checkpoint: callback to do checkpoint
+ * @get_error: callback to check if error occurred during replication
+ */
+struct ReplicationOps {
+ void (*start)(ReplicationState *rs, ReplicationMode mode, Error **errp);
+ void (*stop)(ReplicationState *rs, bool failover, Error **errp);
+ void (*checkpoint)(ReplicationState *rs, Error **errp);
+ void (*get_error)(ReplicationState *rs, Error **errp);
+};
+
+/**
+ * replication_new:
+ * @opaque: opaque pointer value passed to ReplicationState
+ * @ops: replication operation of the new relevant ReplicationState
+ *
+ * Called to create a new ReplicationState instance, and then insert it
+ * into @replication_states QLIST
+ *
+ * Returns: the new ReplicationState instance
+ */
+ReplicationState *replication_new(void *opaque, ReplicationOps *ops);
+
+/**
+ * replication_remove:
+ * @rs: the ReplicationState instance to remove
+ *
+ * Called to remove a ReplicationState instance, and then delete it from
+ * @replication_states QLIST
+ */
+void replication_remove(ReplicationState *rs);
+
+/**
+ * replication_start_all:
+ * @mode: replication mode that could be "primary" or "secondary"
+ * @errp: returns an error if this function fails
+ *
+ * Start replication, called in migration/checkpoint thread
+ *
+ * Note: the caller of the function MUST make sure vm stopped
+ */
+void replication_start_all(ReplicationMode mode, Error **errp);
+
+/**
+ * replication_do_checkpoint_all:
+ * @errp: returns an error if this function fails
+ *
+ * This interface is called after all VM state is transferred to Secondary QEMU
+ */
+void replication_do_checkpoint_all(Error **errp);
+
+/**
+ * replication_get_error_all:
+ * @errp: returns an error if this function fails
+ *
+ * This interface is called to check if error occurred during replication
+ */
+void replication_get_error_all(Error **errp);
+
+/**
+ * replication_stop_all:
+ * @failover: boolean value that indicates if we need do failover or not
+ * @errp: returns an error if this function fails
+ *
+ * It is called on failover. The vm should be stopped before calling it, if you
+ * use this API to shutdown the guest, or other things except failover
+ */
+void replication_stop_all(bool failover, Error **errp);
+
+#endif /* REPLICATION_H */
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 13/17] replication: Implement new driver for block replication
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (11 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 12/17] replication: Introduce new APIs to do replication operation Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 14/17] tests: add unit test case for replication Stefan Hajnoczi
` (4 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Wen Congyang, Changlong Xie, Wang WeiWei,
zhanghailiang, Gonglei, Stefan Hajnoczi
From: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Message-id: 1469602913-20979-10-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/Makefile.objs | 1 +
block/replication.c | 659 ++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 660 insertions(+)
create mode 100644 block/replication.c
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 8a3270b..55da626 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -23,6 +23,7 @@ block-obj-$(CONFIG_LIBSSH2) += ssh.o
block-obj-y += accounting.o dirty-bitmap.o
block-obj-y += write-threshold.o
block-obj-y += backup.o
+block-obj-$(CONFIG_REPLICATION) += replication.o
block-obj-y += crypto.o
diff --git a/block/replication.c b/block/replication.c
new file mode 100644
index 0000000..3bd1cf1
--- /dev/null
+++ b/block/replication.c
@@ -0,0 +1,659 @@
+/*
+ * Replication Block filter
+ *
+ * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ * Copyright (c) 2016 Intel Corporation
+ * Copyright (c) 2016 FUJITSU LIMITED
+ *
+ * Author:
+ * Wen Congyang <wency@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "block/nbd.h"
+#include "block/blockjob.h"
+#include "block/block_int.h"
+#include "block/block_backup.h"
+#include "sysemu/block-backend.h"
+#include "qapi/error.h"
+#include "replication.h"
+
+typedef struct BDRVReplicationState {
+ ReplicationMode mode;
+ int replication_state;
+ BdrvChild *active_disk;
+ BdrvChild *hidden_disk;
+ BdrvChild *secondary_disk;
+ char *top_id;
+ ReplicationState *rs;
+ Error *blocker;
+ int orig_hidden_flags;
+ int orig_secondary_flags;
+ int error;
+} BDRVReplicationState;
+
+enum {
+ BLOCK_REPLICATION_NONE, /* block replication is not started */
+ BLOCK_REPLICATION_RUNNING, /* block replication is running */
+ BLOCK_REPLICATION_FAILOVER, /* failover is running in background */
+ BLOCK_REPLICATION_FAILOVER_FAILED, /* failover failed */
+ BLOCK_REPLICATION_DONE, /* block replication is done */
+};
+
+static void replication_start(ReplicationState *rs, ReplicationMode mode,
+ Error **errp);
+static void replication_do_checkpoint(ReplicationState *rs, Error **errp);
+static void replication_get_error(ReplicationState *rs, Error **errp);
+static void replication_stop(ReplicationState *rs, bool failover,
+ Error **errp);
+
+#define REPLICATION_MODE "mode"
+#define REPLICATION_TOP_ID "top-id"
+static QemuOptsList replication_runtime_opts = {
+ .name = "replication",
+ .head = QTAILQ_HEAD_INITIALIZER(replication_runtime_opts.head),
+ .desc = {
+ {
+ .name = REPLICATION_MODE,
+ .type = QEMU_OPT_STRING,
+ },
+ {
+ .name = REPLICATION_TOP_ID,
+ .type = QEMU_OPT_STRING,
+ },
+ { /* end of list */ }
+ },
+};
+
+static ReplicationOps replication_ops = {
+ .start = replication_start,
+ .checkpoint = replication_do_checkpoint,
+ .get_error = replication_get_error,
+ .stop = replication_stop,
+};
+
+static int replication_open(BlockDriverState *bs, QDict *options,
+ int flags, Error **errp)
+{
+ int ret;
+ BDRVReplicationState *s = bs->opaque;
+ Error *local_err = NULL;
+ QemuOpts *opts = NULL;
+ const char *mode;
+ const char *top_id;
+
+ ret = -EINVAL;
+ opts = qemu_opts_create(&replication_runtime_opts, NULL, 0, &error_abort);
+ qemu_opts_absorb_qdict(opts, options, &local_err);
+ if (local_err) {
+ goto fail;
+ }
+
+ mode = qemu_opt_get(opts, REPLICATION_MODE);
+ if (!mode) {
+ error_setg(&local_err, "Missing the option mode");
+ goto fail;
+ }
+
+ if (!strcmp(mode, "primary")) {
+ s->mode = REPLICATION_MODE_PRIMARY;
+ } else if (!strcmp(mode, "secondary")) {
+ s->mode = REPLICATION_MODE_SECONDARY;
+ top_id = qemu_opt_get(opts, REPLICATION_TOP_ID);
+ s->top_id = g_strdup(top_id);
+ if (!s->top_id) {
+ error_setg(&local_err, "Missing the option top-id");
+ goto fail;
+ }
+ } else {
+ error_setg(&local_err,
+ "The option mode's value should be primary or secondary");
+ goto fail;
+ }
+
+ s->rs = replication_new(bs, &replication_ops);
+
+ ret = 0;
+
+fail:
+ qemu_opts_del(opts);
+ error_propagate(errp, local_err);
+
+ return ret;
+}
+
+static void replication_close(BlockDriverState *bs)
+{
+ BDRVReplicationState *s = bs->opaque;
+
+ if (s->replication_state == BLOCK_REPLICATION_RUNNING) {
+ replication_stop(s->rs, false, NULL);
+ }
+
+ if (s->mode == REPLICATION_MODE_SECONDARY) {
+ g_free(s->top_id);
+ }
+
+ replication_remove(s->rs);
+}
+
+static int64_t replication_getlength(BlockDriverState *bs)
+{
+ return bdrv_getlength(bs->file->bs);
+}
+
+static int replication_get_io_status(BDRVReplicationState *s)
+{
+ switch (s->replication_state) {
+ case BLOCK_REPLICATION_NONE:
+ return -EIO;
+ case BLOCK_REPLICATION_RUNNING:
+ return 0;
+ case BLOCK_REPLICATION_FAILOVER:
+ return s->mode == REPLICATION_MODE_PRIMARY ? -EIO : 0;
+ case BLOCK_REPLICATION_FAILOVER_FAILED:
+ return s->mode == REPLICATION_MODE_PRIMARY ? -EIO : 1;
+ case BLOCK_REPLICATION_DONE:
+ /*
+ * active commit job completes, and active disk and secondary_disk
+ * is swapped, so we can operate bs->file directly
+ */
+ return s->mode == REPLICATION_MODE_PRIMARY ? -EIO : 0;
+ default:
+ abort();
+ }
+}
+
+static int replication_return_value(BDRVReplicationState *s, int ret)
+{
+ if (s->mode == REPLICATION_MODE_SECONDARY) {
+ return ret;
+ }
+
+ if (ret < 0) {
+ s->error = ret;
+ ret = 0;
+ }
+
+ return ret;
+}
+
+static coroutine_fn int replication_co_readv(BlockDriverState *bs,
+ int64_t sector_num,
+ int remaining_sectors,
+ QEMUIOVector *qiov)
+{
+ BDRVReplicationState *s = bs->opaque;
+ BdrvChild *child = s->secondary_disk;
+ BlockJob *job = NULL;
+ CowRequest req;
+ int ret;
+
+ if (s->mode == REPLICATION_MODE_PRIMARY) {
+ /* We only use it to forward primary write requests */
+ return -EIO;
+ }
+
+ ret = replication_get_io_status(s);
+ if (ret < 0) {
+ return ret;
+ }
+
+ if (child && child->bs) {
+ job = child->bs->job;
+ }
+
+ if (job) {
+ backup_wait_for_overlapping_requests(child->bs->job, sector_num,
+ remaining_sectors);
+ backup_cow_request_begin(&req, child->bs->job, sector_num,
+ remaining_sectors);
+ ret = bdrv_co_readv(bs->file, sector_num, remaining_sectors,
+ qiov);
+ backup_cow_request_end(&req);
+ goto out;
+ }
+
+ ret = bdrv_co_readv(bs->file, sector_num, remaining_sectors, qiov);
+out:
+ return replication_return_value(s, ret);
+}
+
+static coroutine_fn int replication_co_writev(BlockDriverState *bs,
+ int64_t sector_num,
+ int remaining_sectors,
+ QEMUIOVector *qiov)
+{
+ BDRVReplicationState *s = bs->opaque;
+ QEMUIOVector hd_qiov;
+ uint64_t bytes_done = 0;
+ BdrvChild *top = bs->file;
+ BdrvChild *base = s->secondary_disk;
+ BdrvChild *target;
+ int ret, n;
+
+ ret = replication_get_io_status(s);
+ if (ret < 0) {
+ goto out;
+ }
+
+ if (ret == 0) {
+ ret = bdrv_co_writev(top, sector_num,
+ remaining_sectors, qiov);
+ return replication_return_value(s, ret);
+ }
+
+ /*
+ * Failover failed, only write to active disk if the sectors
+ * have already been allocated in active disk/hidden disk.
+ */
+ qemu_iovec_init(&hd_qiov, qiov->niov);
+ while (remaining_sectors > 0) {
+ ret = bdrv_is_allocated_above(top->bs, base->bs, sector_num,
+ remaining_sectors, &n);
+ if (ret < 0) {
+ goto out1;
+ }
+
+ qemu_iovec_reset(&hd_qiov);
+ qemu_iovec_concat(&hd_qiov, qiov, bytes_done, n * BDRV_SECTOR_SIZE);
+
+ target = ret ? top : base;
+ ret = bdrv_co_writev(target, sector_num, n, &hd_qiov);
+ if (ret < 0) {
+ goto out1;
+ }
+
+ remaining_sectors -= n;
+ sector_num += n;
+ bytes_done += n * BDRV_SECTOR_SIZE;
+ }
+
+out1:
+ qemu_iovec_destroy(&hd_qiov);
+out:
+ return ret;
+}
+
+static bool replication_recurse_is_first_non_filter(BlockDriverState *bs,
+ BlockDriverState *candidate)
+{
+ return bdrv_recurse_is_first_non_filter(bs->file->bs, candidate);
+}
+
+static void secondary_do_checkpoint(BDRVReplicationState *s, Error **errp)
+{
+ Error *local_err = NULL;
+ int ret;
+
+ if (!s->secondary_disk->bs->job) {
+ error_setg(errp, "Backup job was cancelled unexpectedly");
+ return;
+ }
+
+ backup_do_checkpoint(s->secondary_disk->bs->job, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ ret = s->active_disk->bs->drv->bdrv_make_empty(s->active_disk->bs);
+ if (ret < 0) {
+ error_setg(errp, "Cannot make active disk empty");
+ return;
+ }
+
+ ret = s->hidden_disk->bs->drv->bdrv_make_empty(s->hidden_disk->bs);
+ if (ret < 0) {
+ error_setg(errp, "Cannot make hidden disk empty");
+ return;
+ }
+}
+
+static void reopen_backing_file(BDRVReplicationState *s, bool writable,
+ Error **errp)
+{
+ BlockReopenQueue *reopen_queue = NULL;
+ int orig_hidden_flags, orig_secondary_flags;
+ int new_hidden_flags, new_secondary_flags;
+ Error *local_err = NULL;
+
+ if (writable) {
+ orig_hidden_flags = s->orig_hidden_flags =
+ bdrv_get_flags(s->hidden_disk->bs);
+ new_hidden_flags = (orig_hidden_flags | BDRV_O_RDWR) &
+ ~BDRV_O_INACTIVE;
+ orig_secondary_flags = s->orig_secondary_flags =
+ bdrv_get_flags(s->secondary_disk->bs);
+ new_secondary_flags = (orig_secondary_flags | BDRV_O_RDWR) &
+ ~BDRV_O_INACTIVE;
+ } else {
+ orig_hidden_flags = (s->orig_hidden_flags | BDRV_O_RDWR) &
+ ~BDRV_O_INACTIVE;
+ new_hidden_flags = s->orig_hidden_flags;
+ orig_secondary_flags = (s->orig_secondary_flags | BDRV_O_RDWR) &
+ ~BDRV_O_INACTIVE;
+ new_secondary_flags = s->orig_secondary_flags;
+ }
+
+ if (orig_hidden_flags != new_hidden_flags) {
+ reopen_queue = bdrv_reopen_queue(reopen_queue, s->hidden_disk->bs, NULL,
+ new_hidden_flags);
+ }
+
+ if (!(orig_secondary_flags & BDRV_O_RDWR)) {
+ reopen_queue = bdrv_reopen_queue(reopen_queue, s->secondary_disk->bs,
+ NULL, new_secondary_flags);
+ }
+
+ if (reopen_queue) {
+ bdrv_reopen_multiple(reopen_queue, &local_err);
+ error_propagate(errp, local_err);
+ }
+}
+
+static void backup_job_cleanup(BDRVReplicationState *s)
+{
+ BlockDriverState *top_bs;
+
+ top_bs = bdrv_lookup_bs(s->top_id, s->top_id, NULL);
+ if (!top_bs) {
+ return;
+ }
+ bdrv_op_unblock_all(top_bs, s->blocker);
+ error_free(s->blocker);
+ reopen_backing_file(s, false, NULL);
+}
+
+static void backup_job_completed(void *opaque, int ret)
+{
+ BDRVReplicationState *s = opaque;
+
+ if (s->replication_state != BLOCK_REPLICATION_FAILOVER) {
+ /* The backup job is cancelled unexpectedly */
+ s->error = -EIO;
+ }
+
+ backup_job_cleanup(s);
+}
+
+static bool check_top_bs(BlockDriverState *top_bs, BlockDriverState *bs)
+{
+ BdrvChild *child;
+
+ /* The bs itself is the top_bs */
+ if (top_bs == bs) {
+ return true;
+ }
+
+ /* Iterate over top_bs's children */
+ QLIST_FOREACH(child, &top_bs->children, next) {
+ if (child->bs == bs || check_top_bs(child->bs, bs)) {
+ return true;
+ }
+ }
+
+ return false;
+}
+
+static void replication_start(ReplicationState *rs, ReplicationMode mode,
+ Error **errp)
+{
+ BlockDriverState *bs = rs->opaque;
+ BDRVReplicationState *s;
+ BlockDriverState *top_bs;
+ int64_t active_length, hidden_length, disk_length;
+ AioContext *aio_context;
+ Error *local_err = NULL;
+
+ aio_context = bdrv_get_aio_context(bs);
+ aio_context_acquire(aio_context);
+ s = bs->opaque;
+
+ if (s->replication_state != BLOCK_REPLICATION_NONE) {
+ error_setg(errp, "Block replication is running or done");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ if (s->mode != mode) {
+ error_setg(errp, "The parameter mode's value is invalid, needs %d,"
+ " but got %d", s->mode, mode);
+ aio_context_release(aio_context);
+ return;
+ }
+
+ switch (s->mode) {
+ case REPLICATION_MODE_PRIMARY:
+ break;
+ case REPLICATION_MODE_SECONDARY:
+ s->active_disk = bs->file;
+ if (!s->active_disk || !s->active_disk->bs ||
+ !s->active_disk->bs->backing) {
+ error_setg(errp, "Active disk doesn't have backing file");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ s->hidden_disk = s->active_disk->bs->backing;
+ if (!s->hidden_disk->bs || !s->hidden_disk->bs->backing) {
+ error_setg(errp, "Hidden disk doesn't have backing file");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ s->secondary_disk = s->hidden_disk->bs->backing;
+ if (!s->secondary_disk->bs || !bdrv_has_blk(s->secondary_disk->bs)) {
+ error_setg(errp, "The secondary disk doesn't have block backend");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ /* verify the length */
+ active_length = bdrv_getlength(s->active_disk->bs);
+ hidden_length = bdrv_getlength(s->hidden_disk->bs);
+ disk_length = bdrv_getlength(s->secondary_disk->bs);
+ if (active_length < 0 || hidden_length < 0 || disk_length < 0 ||
+ active_length != hidden_length || hidden_length != disk_length) {
+ error_setg(errp, "Active disk, hidden disk, secondary disk's length"
+ " are not the same");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ if (!s->active_disk->bs->drv->bdrv_make_empty ||
+ !s->hidden_disk->bs->drv->bdrv_make_empty) {
+ error_setg(errp,
+ "Active disk or hidden disk doesn't support make_empty");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ /* reopen the backing file in r/w mode */
+ reopen_backing_file(s, true, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ aio_context_release(aio_context);
+ return;
+ }
+
+ /* start backup job now */
+ error_setg(&s->blocker,
+ "Block device is in use by internal backup job");
+
+ top_bs = bdrv_lookup_bs(s->top_id, s->top_id, NULL);
+ if (!top_bs || !bdrv_is_root_node(top_bs) ||
+ !check_top_bs(top_bs, bs)) {
+ error_setg(errp, "No top_bs or it is invalid");
+ reopen_backing_file(s, false, NULL);
+ aio_context_release(aio_context);
+ return;
+ }
+ bdrv_op_block_all(top_bs, s->blocker);
+ bdrv_op_unblock(top_bs, BLOCK_OP_TYPE_DATAPLANE, s->blocker);
+
+ backup_start("replication-backup", s->secondary_disk->bs,
+ s->hidden_disk->bs, 0, MIRROR_SYNC_MODE_NONE, NULL, false,
+ BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
+ backup_job_completed, s, NULL, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ backup_job_cleanup(s);
+ aio_context_release(aio_context);
+ return;
+ }
+ break;
+ default:
+ aio_context_release(aio_context);
+ abort();
+ }
+
+ s->replication_state = BLOCK_REPLICATION_RUNNING;
+
+ if (s->mode == REPLICATION_MODE_SECONDARY) {
+ secondary_do_checkpoint(s, errp);
+ }
+
+ s->error = 0;
+ aio_context_release(aio_context);
+}
+
+static void replication_do_checkpoint(ReplicationState *rs, Error **errp)
+{
+ BlockDriverState *bs = rs->opaque;
+ BDRVReplicationState *s;
+ AioContext *aio_context;
+
+ aio_context = bdrv_get_aio_context(bs);
+ aio_context_acquire(aio_context);
+ s = bs->opaque;
+
+ if (s->mode == REPLICATION_MODE_SECONDARY) {
+ secondary_do_checkpoint(s, errp);
+ }
+ aio_context_release(aio_context);
+}
+
+static void replication_get_error(ReplicationState *rs, Error **errp)
+{
+ BlockDriverState *bs = rs->opaque;
+ BDRVReplicationState *s;
+ AioContext *aio_context;
+
+ aio_context = bdrv_get_aio_context(bs);
+ aio_context_acquire(aio_context);
+ s = bs->opaque;
+
+ if (s->replication_state != BLOCK_REPLICATION_RUNNING) {
+ error_setg(errp, "Block replication is not running");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ if (s->error) {
+ error_setg(errp, "I/O error occurred");
+ aio_context_release(aio_context);
+ return;
+ }
+ aio_context_release(aio_context);
+}
+
+static void replication_done(void *opaque, int ret)
+{
+ BlockDriverState *bs = opaque;
+ BDRVReplicationState *s = bs->opaque;
+
+ if (ret == 0) {
+ s->replication_state = BLOCK_REPLICATION_DONE;
+
+ /* refresh top bs's filename */
+ bdrv_refresh_filename(bs);
+ s->active_disk = NULL;
+ s->secondary_disk = NULL;
+ s->hidden_disk = NULL;
+ s->error = 0;
+ } else {
+ s->replication_state = BLOCK_REPLICATION_FAILOVER_FAILED;
+ s->error = -EIO;
+ }
+}
+
+static void replication_stop(ReplicationState *rs, bool failover, Error **errp)
+{
+ BlockDriverState *bs = rs->opaque;
+ BDRVReplicationState *s;
+ AioContext *aio_context;
+
+ aio_context = bdrv_get_aio_context(bs);
+ aio_context_acquire(aio_context);
+ s = bs->opaque;
+
+ if (s->replication_state != BLOCK_REPLICATION_RUNNING) {
+ error_setg(errp, "Block replication is not running");
+ aio_context_release(aio_context);
+ return;
+ }
+
+ switch (s->mode) {
+ case REPLICATION_MODE_PRIMARY:
+ s->replication_state = BLOCK_REPLICATION_DONE;
+ s->error = 0;
+ break;
+ case REPLICATION_MODE_SECONDARY:
+ /*
+ * This BDS will be closed, and the job should be completed
+ * before the BDS is closed, because we will access hidden
+ * disk, secondary disk in backup_job_completed().
+ */
+ if (s->secondary_disk->bs->job) {
+ block_job_cancel_sync(s->secondary_disk->bs->job);
+ }
+
+ if (!failover) {
+ secondary_do_checkpoint(s, errp);
+ s->replication_state = BLOCK_REPLICATION_DONE;
+ aio_context_release(aio_context);
+ return;
+ }
+
+ s->replication_state = BLOCK_REPLICATION_FAILOVER;
+ commit_active_start("replication-commit", s->active_disk->bs,
+ s->secondary_disk->bs, 0, BLOCKDEV_ON_ERROR_REPORT,
+ replication_done,
+ bs, errp, true);
+ break;
+ default:
+ aio_context_release(aio_context);
+ abort();
+ }
+ aio_context_release(aio_context);
+}
+
+BlockDriver bdrv_replication = {
+ .format_name = "replication",
+ .protocol_name = "replication",
+ .instance_size = sizeof(BDRVReplicationState),
+
+ .bdrv_open = replication_open,
+ .bdrv_close = replication_close,
+
+ .bdrv_getlength = replication_getlength,
+ .bdrv_co_readv = replication_co_readv,
+ .bdrv_co_writev = replication_co_writev,
+
+ .is_filter = true,
+ .bdrv_recurse_is_first_non_filter = replication_recurse_is_first_non_filter,
+
+ .has_variable_length = true,
+};
+
+static void bdrv_replication_init(void)
+{
+ bdrv_register(&bdrv_replication);
+}
+
+block_init(bdrv_replication_init);
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 14/17] tests: add unit test case for replication
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (12 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 13/17] replication: Implement new driver for block replication Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 15/17] support replication driver in blockdev-add Stefan Hajnoczi
` (3 subsequent siblings)
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Changlong Xie, Wen Congyang, Wang WeiWei,
Stefan Hajnoczi
From: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Message-id: 1469602913-20979-11-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/.gitignore | 1 +
tests/Makefile.include | 4 +
tests/test-replication.c | 575 +++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 580 insertions(+)
create mode 100644 tests/test-replication.c
diff --git a/tests/.gitignore b/tests/.gitignore
index dbb5263..b4a9cfc 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -63,6 +63,7 @@ test-qmp-introspect.[ch]
test-qmp-marshal.c
test-qmp-output-visitor
test-rcu-list
+test-replication
test-rfifolock
test-string-input-visitor
test-string-output-visitor
diff --git a/tests/Makefile.include b/tests/Makefile.include
index e3a3266..e560ecb 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -112,6 +112,7 @@ check-unit-y += tests/test-crypto-xts$(EXESUF)
check-unit-y += tests/test-crypto-block$(EXESUF)
gcov-files-test-logging-y = tests/test-logging.c
check-unit-y += tests/test-logging$(EXESUF)
+check-unit-$(CONFIG_REPLICATION) += tests/test-replication$(EXESUF)
check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
@@ -502,6 +503,9 @@ tests/test-base64$(EXESUF): tests/test-base64.o \
tests/test-logging$(EXESUF): tests/test-logging.o $(test-util-obj-y)
+tests/test-replication$(EXESUF): tests/test-replication.o $(test-util-obj-y) \
+ $(test-block-obj-y)
+
tests/test-qapi-types.c tests/test-qapi-types.h :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
diff --git a/tests/test-replication.c b/tests/test-replication.c
new file mode 100644
index 0000000..b63f1ef
--- /dev/null
+++ b/tests/test-replication.c
@@ -0,0 +1,575 @@
+/*
+ * Block replication tests
+ *
+ * Copyright (c) 2016 FUJITSU LIMITED
+ * Author: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+
+#include "qapi/error.h"
+#include "replication.h"
+#include "block/block_int.h"
+#include "sysemu/block-backend.h"
+
+#define IMG_SIZE (64 * 1024 * 1024)
+
+/* primary */
+#define P_ID "primary-id"
+static char p_local_disk[] = "/tmp/p_local_disk.XXXXXX";
+
+/* secondary */
+#define S_ID "secondary-id"
+#define S_LOCAL_DISK_ID "secondary-local-disk-id"
+static char s_local_disk[] = "/tmp/s_local_disk.XXXXXX";
+static char s_active_disk[] = "/tmp/s_active_disk.XXXXXX";
+static char s_hidden_disk[] = "/tmp/s_hidden_disk.XXXXXX";
+
+/* FIXME: steal from blockdev.c */
+QemuOptsList qemu_drive_opts = {
+ .name = "drive",
+ .head = QTAILQ_HEAD_INITIALIZER(qemu_drive_opts.head),
+ .desc = {
+ { /* end of list */ }
+ },
+};
+
+#define NOT_DONE 0x7fffffff
+
+static void blk_rw_done(void *opaque, int ret)
+{
+ *(int *)opaque = ret;
+}
+
+static void test_blk_read(BlockBackend *blk, long pattern,
+ int64_t pattern_offset, int64_t pattern_count,
+ int64_t offset, int64_t count,
+ bool expect_failed)
+{
+ void *pattern_buf = NULL;
+ QEMUIOVector qiov;
+ void *cmp_buf = NULL;
+ int async_ret = NOT_DONE;
+
+ if (pattern) {
+ cmp_buf = g_malloc(pattern_count);
+ memset(cmp_buf, pattern, pattern_count);
+ }
+
+ pattern_buf = g_malloc(count);
+ if (pattern) {
+ memset(pattern_buf, pattern, count);
+ } else {
+ memset(pattern_buf, 0x00, count);
+ }
+
+ qemu_iovec_init(&qiov, 1);
+ qemu_iovec_add(&qiov, pattern_buf, count);
+
+ blk_aio_preadv(blk, offset, &qiov, 0, blk_rw_done, &async_ret);
+ while (async_ret == NOT_DONE) {
+ main_loop_wait(false);
+ }
+
+ if (expect_failed) {
+ g_assert(async_ret != 0);
+ } else {
+ g_assert(async_ret == 0);
+ if (pattern) {
+ g_assert(memcmp(pattern_buf + pattern_offset,
+ cmp_buf, pattern_count) <= 0);
+ }
+ }
+
+ g_free(pattern_buf);
+}
+
+static void test_blk_write(BlockBackend *blk, long pattern, int64_t offset,
+ int64_t count, bool expect_failed)
+{
+ void *pattern_buf = NULL;
+ QEMUIOVector qiov;
+ int async_ret = NOT_DONE;
+
+ pattern_buf = g_malloc(count);
+ if (pattern) {
+ memset(pattern_buf, pattern, count);
+ } else {
+ memset(pattern_buf, 0x00, count);
+ }
+
+ qemu_iovec_init(&qiov, 1);
+ qemu_iovec_add(&qiov, pattern_buf, count);
+
+ blk_aio_pwritev(blk, offset, &qiov, 0, blk_rw_done, &async_ret);
+ while (async_ret == NOT_DONE) {
+ main_loop_wait(false);
+ }
+
+ if (expect_failed) {
+ g_assert(async_ret != 0);
+ } else {
+ g_assert(async_ret == 0);
+ }
+
+ g_free(pattern_buf);
+}
+
+/*
+ * Create a uniquely-named empty temporary file.
+ */
+static void make_temp(char *template)
+{
+ int fd;
+
+ fd = mkstemp(template);
+ g_assert(fd >= 0);
+ close(fd);
+}
+
+static void prepare_imgs(void)
+{
+ Error *local_err = NULL;
+
+ make_temp(p_local_disk);
+ make_temp(s_local_disk);
+ make_temp(s_active_disk);
+ make_temp(s_hidden_disk);
+
+ /* Primary */
+ bdrv_img_create(p_local_disk, "qcow2", NULL, NULL, NULL, IMG_SIZE,
+ BDRV_O_RDWR, &local_err, true);
+ g_assert(!local_err);
+
+ /* Secondary */
+ bdrv_img_create(s_local_disk, "qcow2", NULL, NULL, NULL, IMG_SIZE,
+ BDRV_O_RDWR, &local_err, true);
+ g_assert(!local_err);
+ bdrv_img_create(s_active_disk, "qcow2", NULL, NULL, NULL, IMG_SIZE,
+ BDRV_O_RDWR, &local_err, true);
+ g_assert(!local_err);
+ bdrv_img_create(s_hidden_disk, "qcow2", NULL, NULL, NULL, IMG_SIZE,
+ BDRV_O_RDWR, &local_err, true);
+ g_assert(!local_err);
+}
+
+static void cleanup_imgs(void)
+{
+ /* Primary */
+ unlink(p_local_disk);
+
+ /* Secondary */
+ unlink(s_local_disk);
+ unlink(s_active_disk);
+ unlink(s_hidden_disk);
+}
+
+static BlockBackend *start_primary(void)
+{
+ BlockBackend *blk;
+ QemuOpts *opts;
+ QDict *qdict;
+ Error *local_err = NULL;
+ char *cmdline;
+
+ cmdline = g_strdup_printf("driver=replication,mode=primary,node-name=xxx,"
+ "file.driver=qcow2,file.file.filename=%s"
+ , p_local_disk);
+ opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
+ g_free(cmdline);
+
+ qdict = qemu_opts_to_qdict(opts, NULL);
+ qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
+ qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
+
+ blk = blk_new_open(NULL, NULL, qdict, BDRV_O_RDWR, &local_err);
+ g_assert(blk);
+ g_assert(!local_err);
+
+ monitor_add_blk(blk, P_ID, &local_err);
+ g_assert(!local_err);
+
+ qemu_opts_del(opts);
+
+ return blk;
+}
+
+static void teardown_primary(void)
+{
+ BlockBackend *blk;
+
+ /* remove P_ID */
+ blk = blk_by_name(P_ID);
+ assert(blk);
+
+ monitor_remove_blk(blk);
+ blk_unref(blk);
+}
+
+static void test_primary_read(void)
+{
+ BlockBackend *blk;
+
+ blk = start_primary();
+
+ /* read from 0 to IMG_SIZE */
+ test_blk_read(blk, 0, 0, IMG_SIZE, 0, IMG_SIZE, true);
+
+ teardown_primary();
+}
+
+static void test_primary_write(void)
+{
+ BlockBackend *blk;
+
+ blk = start_primary();
+
+ /* write from 0 to IMG_SIZE */
+ test_blk_write(blk, 0, 0, IMG_SIZE, true);
+
+ teardown_primary();
+}
+
+static void test_primary_start(void)
+{
+ BlockBackend *blk = NULL;
+ Error *local_err = NULL;
+
+ blk = start_primary();
+
+ replication_start_all(REPLICATION_MODE_PRIMARY, &local_err);
+ g_assert(!local_err);
+
+ /* read from 0 to IMG_SIZE */
+ test_blk_read(blk, 0, 0, IMG_SIZE, 0, IMG_SIZE, true);
+
+ /* write 0x22 from 0 to IMG_SIZE */
+ test_blk_write(blk, 0x22, 0, IMG_SIZE, false);
+
+ teardown_primary();
+}
+
+static void test_primary_stop(void)
+{
+ Error *local_err = NULL;
+ bool failover = true;
+
+ start_primary();
+
+ replication_start_all(REPLICATION_MODE_PRIMARY, &local_err);
+ g_assert(!local_err);
+
+ replication_stop_all(failover, &local_err);
+ g_assert(!local_err);
+
+ teardown_primary();
+}
+
+static void test_primary_do_checkpoint(void)
+{
+ Error *local_err = NULL;
+
+ start_primary();
+
+ replication_start_all(REPLICATION_MODE_PRIMARY, &local_err);
+ g_assert(!local_err);
+
+ replication_do_checkpoint_all(&local_err);
+ g_assert(!local_err);
+
+ teardown_primary();
+}
+
+static void test_primary_get_error(void)
+{
+ Error *local_err = NULL;
+
+ start_primary();
+
+ replication_start_all(REPLICATION_MODE_PRIMARY, &local_err);
+ g_assert(!local_err);
+
+ replication_get_error_all(&local_err);
+ g_assert(!local_err);
+
+ teardown_primary();
+}
+
+static BlockBackend *start_secondary(void)
+{
+ QemuOpts *opts;
+ QDict *qdict;
+ BlockBackend *blk;
+ char *cmdline;
+ Error *local_err = NULL;
+
+ /* add s_local_disk and forge S_LOCAL_DISK_ID */
+ cmdline = g_strdup_printf("file.filename=%s,driver=qcow2", s_local_disk);
+ opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
+ g_free(cmdline);
+
+ qdict = qemu_opts_to_qdict(opts, NULL);
+ qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
+ qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
+
+ blk = blk_new_open(NULL, NULL, qdict, BDRV_O_RDWR, &local_err);
+ assert(blk);
+ monitor_add_blk(blk, S_LOCAL_DISK_ID, &local_err);
+ g_assert(!local_err);
+
+ /* format s_local_disk with pattern "0x11" */
+ test_blk_write(blk, 0x11, 0, IMG_SIZE, false);
+
+ qemu_opts_del(opts);
+
+ /* add S_(ACTIVE/HIDDEN)_DISK and forge S_ID */
+ cmdline = g_strdup_printf("driver=replication,mode=secondary,top-id=%s,"
+ "file.driver=qcow2,file.file.filename=%s,"
+ "file.backing.driver=qcow2,"
+ "file.backing.file.filename=%s,"
+ "file.backing.backing=%s"
+ , S_ID, s_active_disk, s_hidden_disk
+ , S_LOCAL_DISK_ID);
+ opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
+ g_free(cmdline);
+
+ qdict = qemu_opts_to_qdict(opts, NULL);
+ qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
+ qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
+
+ blk = blk_new_open(NULL, NULL, qdict, BDRV_O_RDWR, &local_err);
+ assert(blk);
+ monitor_add_blk(blk, S_ID, &local_err);
+ g_assert(!local_err);
+
+ qemu_opts_del(opts);
+
+ return blk;
+}
+
+static void teardown_secondary(void)
+{
+ /* only need to destroy two BBs */
+ BlockBackend *blk;
+
+ /* remove S_LOCAL_DISK_ID */
+ blk = blk_by_name(S_LOCAL_DISK_ID);
+ assert(blk);
+
+ monitor_remove_blk(blk);
+ blk_unref(blk);
+
+ /* remove S_ID */
+ blk = blk_by_name(S_ID);
+ assert(blk);
+
+ monitor_remove_blk(blk);
+ blk_unref(blk);
+}
+
+static void test_secondary_read(void)
+{
+ BlockBackend *blk;
+
+ blk = start_secondary();
+
+ /* read from 0 to IMG_SIZE */
+ test_blk_read(blk, 0, 0, IMG_SIZE, 0, IMG_SIZE, true);
+
+ teardown_secondary();
+}
+
+static void test_secondary_write(void)
+{
+ BlockBackend *blk;
+
+ blk = start_secondary();
+
+ /* write from 0 to IMG_SIZE */
+ test_blk_write(blk, 0, 0, IMG_SIZE, true);
+
+ teardown_secondary();
+}
+
+static void test_secondary_start(void)
+{
+ BlockBackend *top_blk, *local_blk;
+ Error *local_err = NULL;
+ bool failover = true;
+
+ top_blk = start_secondary();
+ replication_start_all(REPLICATION_MODE_SECONDARY, &local_err);
+ g_assert(!local_err);
+
+ /* read from s_local_disk (0, IMG_SIZE) */
+ test_blk_read(top_blk, 0x11, 0, IMG_SIZE, 0, IMG_SIZE, false);
+
+ /* write 0x22 to s_local_disk (IMG_SIZE / 2, IMG_SIZE) */
+ local_blk = blk_by_name(S_LOCAL_DISK_ID);
+ test_blk_write(local_blk, 0x22, IMG_SIZE / 2, IMG_SIZE / 2, false);
+
+ /* replication will backup s_local_disk to s_hidden_disk */
+ test_blk_read(top_blk, 0x11, IMG_SIZE / 2,
+ IMG_SIZE / 2, 0, IMG_SIZE, false);
+
+ /* write 0x33 to s_active_disk (0, IMG_SIZE / 2) */
+ test_blk_write(top_blk, 0x33, 0, IMG_SIZE / 2, false);
+
+ /* read from s_active_disk (0, IMG_SIZE/2) */
+ test_blk_read(top_blk, 0x33, 0, IMG_SIZE / 2,
+ 0, IMG_SIZE / 2, false);
+
+ /* unblock top_bs */
+ replication_stop_all(failover, &local_err);
+ g_assert(!local_err);
+
+ teardown_secondary();
+}
+
+
+static void test_secondary_stop(void)
+{
+ BlockBackend *top_blk, *local_blk;
+ Error *local_err = NULL;
+ bool failover = true;
+
+ top_blk = start_secondary();
+ replication_start_all(REPLICATION_MODE_SECONDARY, &local_err);
+ g_assert(!local_err);
+
+ /* write 0x22 to s_local_disk (IMG_SIZE / 2, IMG_SIZE) */
+ local_blk = blk_by_name(S_LOCAL_DISK_ID);
+ test_blk_write(local_blk, 0x22, IMG_SIZE / 2, IMG_SIZE / 2, false);
+
+ /* replication will backup s_local_disk to s_hidden_disk */
+ test_blk_read(top_blk, 0x11, IMG_SIZE / 2,
+ IMG_SIZE / 2, 0, IMG_SIZE, false);
+
+ /* write 0x33 to s_active_disk (0, IMG_SIZE / 2) */
+ test_blk_write(top_blk, 0x33, 0, IMG_SIZE / 2, false);
+
+ /* do active commit */
+ replication_stop_all(failover, &local_err);
+ g_assert(!local_err);
+
+ /* read from s_local_disk (0, IMG_SIZE / 2) */
+ test_blk_read(top_blk, 0x33, 0, IMG_SIZE / 2,
+ 0, IMG_SIZE / 2, false);
+
+
+ /* read from s_local_disk (IMG_SIZE / 2, IMG_SIZE) */
+ test_blk_read(top_blk, 0x22, IMG_SIZE / 2,
+ IMG_SIZE / 2, 0, IMG_SIZE, false);
+
+ teardown_secondary();
+}
+
+static void test_secondary_do_checkpoint(void)
+{
+ BlockBackend *top_blk, *local_blk;
+ Error *local_err = NULL;
+ bool failover = true;
+
+ top_blk = start_secondary();
+ replication_start_all(REPLICATION_MODE_SECONDARY, &local_err);
+ g_assert(!local_err);
+
+ /* write 0x22 to s_local_disk (IMG_SIZE / 2, IMG_SIZE) */
+ local_blk = blk_by_name(S_LOCAL_DISK_ID);
+ test_blk_write(local_blk, 0x22, IMG_SIZE / 2,
+ IMG_SIZE / 2, false);
+
+ /* replication will backup s_local_disk to s_hidden_disk */
+ test_blk_read(top_blk, 0x11, IMG_SIZE / 2,
+ IMG_SIZE / 2, 0, IMG_SIZE, false);
+
+ replication_do_checkpoint_all(&local_err);
+ g_assert(!local_err);
+
+ /* after checkpoint, read pattern 0x22 from s_local_disk */
+ test_blk_read(top_blk, 0x22, IMG_SIZE / 2,
+ IMG_SIZE / 2, 0, IMG_SIZE, false);
+
+ /* unblock top_bs */
+ replication_stop_all(failover, &local_err);
+ g_assert(!local_err);
+
+ teardown_secondary();
+}
+
+static void test_secondary_get_error(void)
+{
+ Error *local_err = NULL;
+ bool failover = true;
+
+ start_secondary();
+ replication_start_all(REPLICATION_MODE_SECONDARY, &local_err);
+ g_assert(!local_err);
+
+ replication_get_error_all(&local_err);
+ g_assert(!local_err);
+
+ /* unblock top_bs */
+ replication_stop_all(failover, &local_err);
+ g_assert(!local_err);
+
+ teardown_secondary();
+}
+
+static void sigabrt_handler(int signo)
+{
+ cleanup_imgs();
+}
+
+static void setup_sigabrt_handler(void)
+{
+ struct sigaction sigact;
+
+ sigact = (struct sigaction) {
+ .sa_handler = sigabrt_handler,
+ .sa_flags = SA_RESETHAND,
+ };
+ sigemptyset(&sigact.sa_mask);
+ sigaction(SIGABRT, &sigact, NULL);
+}
+
+int main(int argc, char **argv)
+{
+ int ret;
+ qemu_init_main_loop(&error_fatal);
+ bdrv_init();
+
+ g_test_init(&argc, &argv, NULL);
+ setup_sigabrt_handler();
+
+ prepare_imgs();
+
+ /* Primary */
+ g_test_add_func("/replication/primary/read", test_primary_read);
+ g_test_add_func("/replication/primary/write", test_primary_write);
+ g_test_add_func("/replication/primary/start", test_primary_start);
+ g_test_add_func("/replication/primary/stop", test_primary_stop);
+ g_test_add_func("/replication/primary/do_checkpoint",
+ test_primary_do_checkpoint);
+ g_test_add_func("/replication/primary/get_error",
+ test_primary_get_error);
+
+ /* Secondary */
+ g_test_add_func("/replication/secondary/read", test_secondary_read);
+ g_test_add_func("/replication/secondary/write", test_secondary_write);
+ g_test_add_func("/replication/secondary/start", test_secondary_start);
+ g_test_add_func("/replication/secondary/stop", test_secondary_stop);
+ g_test_add_func("/replication/secondary/do_checkpoint",
+ test_secondary_do_checkpoint);
+ g_test_add_func("/replication/secondary/get_error",
+ test_secondary_get_error);
+
+ ret = g_test_run();
+
+ cleanup_imgs();
+
+ return ret;
+}
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 15/17] support replication driver in blockdev-add
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (13 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 14/17] tests: add unit test case for replication Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-10-25 21:31 ` Eric Blake
2016-09-12 14:08 ` [Qemu-devel] [PULL 16/17] MAINTAINERS: add maintainer for replication Stefan Hajnoczi
` (2 subsequent siblings)
17 siblings, 1 reply; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Wen Congyang, Changlong Xie, Wang WeiWei,
zhanghailiang, Gonglei, Stefan Hajnoczi
From: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1469602913-20979-12-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
qapi/block-core.json | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 4755c75..ada3202 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -252,6 +252,7 @@
# 2.3: 'host_floppy' deprecated
# 2.5: 'host_floppy' dropped
# 2.6: 'luks' added
+# 2.8: 'replication' added
#
# @backing_file: #optional the name of the backing file (for copy-on-write)
#
@@ -1712,8 +1713,8 @@
'data': [ 'archipelago', 'blkdebug', 'blkverify', 'bochs', 'cloop',
'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom',
'host_device', 'http', 'https', 'luks', 'null-aio', 'null-co',
- 'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'tftp',
- 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
+ 'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw',
+ 'replication', 'tftp', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
##
# @BlockdevOptionsFile
@@ -2176,6 +2177,23 @@
{ 'enum' : 'ReplicationMode', 'data' : [ 'primary', 'secondary' ] }
##
+# @BlockdevOptionsReplication
+#
+# Driver specific block device options for replication
+#
+# @mode: the replication mode
+#
+# @top-id: #optional In secondary mode, node name or device ID of the root
+# node who owns the replication node chain. Ignored in primary mode.
+#
+# Since: 2.8
+##
+{ 'struct': 'BlockdevOptionsReplication',
+ 'base': 'BlockdevOptionsGenericFormat',
+ 'data': { 'mode': 'ReplicationMode',
+ '*top-id': 'str' } }
+
+##
# @BlockdevOptions
#
# Options for creating a block device. Many options are available for all
@@ -2240,6 +2258,7 @@
'quorum': 'BlockdevOptionsQuorum',
'raw': 'BlockdevOptionsGenericFormat',
# TODO rbd: Wait for structured options
+ 'replication':'BlockdevOptionsReplication',
# TODO sheepdog: Wait for structured options
# TODO ssh: Should take InetSocketAddress for 'host'?
'tftp': 'BlockdevOptionsFile',
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 16/17] MAINTAINERS: add maintainer for replication
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (14 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 15/17] support replication driver in blockdev-add Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 17/17] tests: fix qvirtqueue_kick Stefan Hajnoczi
2016-09-12 15:12 ` [Qemu-devel] [PULL 00/17] Block patches Peter Maydell
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Changlong Xie, Stefan Hajnoczi, Wen Congyang
From: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
As per Stefan's suggestion, add Wen and I as co-maintainers
of replication.
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Message-id: 1469602913-20979-13-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
MAINTAINERS | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index b6fb84e..2ffdb36 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1624,6 +1624,15 @@ L: qemu-block@nongnu.org
S: Supported
F: tests/image-fuzzer/
+Replication
+M: Wen Congyang <wency@cn.fujitsu.com>
+M: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
+S: Supported
+F: replication*
+F: block/replication.c
+F: tests/test-replication.c
+F: docs/block-replication.txt
+
Build and test automation
-------------------------
M: Alex Bennée <alex.bennee@linaro.org>
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 17/17] tests: fix qvirtqueue_kick
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (15 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 16/17] MAINTAINERS: add maintainer for replication Stefan Hajnoczi
@ 2016-09-12 14:08 ` Stefan Hajnoczi
2016-09-12 15:12 ` [Qemu-devel] [PULL 00/17] Block patches Peter Maydell
17 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-12 14:08 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Laurent Vivier, Stefan Hajnoczi
From: Laurent Vivier <lvivier@redhat.com>
vq->avail.idx and vq->avail->ring[] are a 16bit values,
so read and write them with readw()/writew() instead of
readl()/writel().
To read/write a 16bit value with a 32bit accessor works fine
on little-endian CPU but not on big endian CPU.
[An equivalent patch for the writew() calls was also sent by
Zhang Shuai <zhangshuai13@huawei.com>.
--Stefan]
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Message-id: 1472330054-22607-1-git-send-email-lvivier@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/libqos/virtio.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/libqos/virtio.c b/tests/libqos/virtio.c
index d8c2970..37ff860 100644
--- a/tests/libqos/virtio.c
+++ b/tests/libqos/virtio.c
@@ -257,16 +257,16 @@ void qvirtqueue_kick(const QVirtioBus *bus, QVirtioDevice *d, QVirtQueue *vq,
uint32_t free_head)
{
/* vq->avail->idx */
- uint16_t idx = readl(vq->avail + 2);
+ uint16_t idx = readw(vq->avail + 2);
/* vq->used->flags */
uint16_t flags;
/* vq->used->avail_event */
uint16_t avail_event;
/* vq->avail->ring[idx % vq->size] */
- writel(vq->avail + 4 + (2 * (idx % vq->size)), free_head);
+ writew(vq->avail + 4 + (2 * (idx % vq->size)), free_head);
/* vq->avail->idx */
- writel(vq->avail + 2, idx + 1);
+ writew(vq->avail + 2, idx + 1);
/* Must read after idx is updated */
flags = readw(vq->avail);
--
2.7.4
^ permalink raw reply related [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
` (16 preceding siblings ...)
2016-09-12 14:08 ` [Qemu-devel] [PULL 17/17] tests: fix qvirtqueue_kick Stefan Hajnoczi
@ 2016-09-12 15:12 ` Peter Maydell
2016-09-12 15:56 ` Peter Maydell
17 siblings, 1 reply; 41+ messages in thread
From: Peter Maydell @ 2016-09-12 15:12 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: QEMU Developers
On 12 September 2016 at 15:08, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit c2a57aae9a1c3dd7de77daf5478df10379aeeebf:
>
> Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging (2016-09-09 12:49:41 +0100)
>
> are available in the git repository at:
>
> git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 8f1096787517c66b67cc29bab65edc0188a86326:
>
> tests: fix qvirtqueue_kick (2016-09-12 15:06:29 +0100)
/replication/primary/get_error: OK
/replication/secondary/get_error: OK
Please can you rename these tests? They create false positives in
scripts that look in the build logs for errors by grepping for
"error:" or "warning:".
thanks
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2016-09-12 15:12 ` [Qemu-devel] [PULL 00/17] Block patches Peter Maydell
@ 2016-09-12 15:56 ` Peter Maydell
2016-09-13 1:11 ` Fam Zheng
2016-09-13 8:53 ` Stefan Hajnoczi
0 siblings, 2 replies; 41+ messages in thread
From: Peter Maydell @ 2016-09-12 15:56 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: QEMU Developers
On 12 September 2016 at 16:12, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 12 September 2016 at 15:08, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>> The following changes since commit c2a57aae9a1c3dd7de77daf5478df10379aeeebf:
>>
>> Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging (2016-09-09 12:49:41 +0100)
>>
>> are available in the git repository at:
>>
>> git://github.com/stefanha/qemu.git tags/block-pull-request
>>
>> for you to fetch changes up to 8f1096787517c66b67cc29bab65edc0188a86326:
>>
>> tests: fix qvirtqueue_kick (2016-09-12 15:06:29 +0100)
>
>
> /replication/primary/get_error: OK
> /replication/secondary/get_error: OK
>
> Please can you rename these tests? They create false positives in
> scripts that look in the build logs for errors by grepping for
> "error:" or "warning:".
Also, two new sanitizer errors:
/home/petmay01/linaro/qemu-for-merges/block/qcow2.c:1807:41: runtime
error: null pointer passed as argument 2, which is declared to never
be null
/home/petmay01/linaro/qemu-for-merges/block/qcow2-cluster.c:86:26:
runtime error: null pointer passed as argument 2, which is declared to
never be null
both attempts to memcpy() from a NULL source pointer while running
the tests/test-replication test.
thanks
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2016-09-12 15:56 ` Peter Maydell
@ 2016-09-13 1:11 ` Fam Zheng
2016-09-13 8:34 ` Stefan Hajnoczi
2016-09-13 8:53 ` Stefan Hajnoczi
1 sibling, 1 reply; 41+ messages in thread
From: Fam Zheng @ 2016-09-13 1:11 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: QEMU Developers
On Mon, 09/12 16:56, Peter Maydell wrote:
> On 12 September 2016 at 16:12, Peter Maydell <peter.maydell@linaro.org> wrote:
> > On 12 September 2016 at 15:08, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >> The following changes since commit c2a57aae9a1c3dd7de77daf5478df10379aeeebf:
> >>
> >> Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging (2016-09-09 12:49:41 +0100)
> >>
> >> are available in the git repository at:
> >>
> >> git://github.com/stefanha/qemu.git tags/block-pull-request
> >>
> >> for you to fetch changes up to 8f1096787517c66b67cc29bab65edc0188a86326:
> >>
> >> tests: fix qvirtqueue_kick (2016-09-12 15:06:29 +0100)
> >
> >
> > /replication/primary/get_error: OK
> > /replication/secondary/get_error: OK
> >
> > Please can you rename these tests? They create false positives in
> > scripts that look in the build logs for errors by grepping for
> > "error:" or "warning:".
>
> Also, two new sanitizer errors:
>
> /home/petmay01/linaro/qemu-for-merges/block/qcow2.c:1807:41: runtime
> error: null pointer passed as argument 2, which is declared to never
> be null
> /home/petmay01/linaro/qemu-for-merges/block/qcow2-cluster.c:86:26:
> runtime error: null pointer passed as argument 2, which is declared to
> never be null
>
> both attempts to memcpy() from a NULL source pointer while running
> the tests/test-replication test.
>
Stefan, if you are going to do another PULL, do you mind including
https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg01717.html
as well? :)
Fam
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2016-09-13 1:11 ` Fam Zheng
@ 2016-09-13 8:34 ` Stefan Hajnoczi
0 siblings, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-13 8:34 UTC (permalink / raw)
To: Fam Zheng; +Cc: Stefan Hajnoczi, QEMU Developers
On Tue, Sep 13, 2016 at 2:11 AM, Fam Zheng <famz@redhat.com> wrote:
> On Mon, 09/12 16:56, Peter Maydell wrote:
>> On 12 September 2016 at 16:12, Peter Maydell <peter.maydell@linaro.org> wrote:
>> > On 12 September 2016 at 15:08, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>> >> The following changes since commit c2a57aae9a1c3dd7de77daf5478df10379aeeebf:
>> >>
>> >> Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging (2016-09-09 12:49:41 +0100)
>> >>
>> >> are available in the git repository at:
>> >>
>> >> git://github.com/stefanha/qemu.git tags/block-pull-request
>> >>
>> >> for you to fetch changes up to 8f1096787517c66b67cc29bab65edc0188a86326:
>> >>
>> >> tests: fix qvirtqueue_kick (2016-09-12 15:06:29 +0100)
>> >
>> >
>> > /replication/primary/get_error: OK
>> > /replication/secondary/get_error: OK
>> >
>> > Please can you rename these tests? They create false positives in
>> > scripts that look in the build logs for errors by grepping for
>> > "error:" or "warning:".
>>
>> Also, two new sanitizer errors:
>>
>> /home/petmay01/linaro/qemu-for-merges/block/qcow2.c:1807:41: runtime
>> error: null pointer passed as argument 2, which is declared to never
>> be null
>> /home/petmay01/linaro/qemu-for-merges/block/qcow2-cluster.c:86:26:
>> runtime error: null pointer passed as argument 2, which is declared to
>> never be null
>>
>> both attempts to memcpy() from a NULL source pointer while running
>> the tests/test-replication test.
>>
>
> Stefan, if you are going to do another PULL, do you mind including
>
> https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg01717.html
>
> as well? :)
Sure, I'll resend today.
Stefan
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2016-09-12 15:56 ` Peter Maydell
2016-09-13 1:11 ` Fam Zheng
@ 2016-09-13 8:53 ` Stefan Hajnoczi
1 sibling, 0 replies; 41+ messages in thread
From: Stefan Hajnoczi @ 2016-09-13 8:53 UTC (permalink / raw)
To: Peter Maydell; +Cc: Stefan Hajnoczi, QEMU Developers
On Mon, Sep 12, 2016 at 4:56 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 12 September 2016 at 16:12, Peter Maydell <peter.maydell@linaro.org> wrote:
>> On 12 September 2016 at 15:08, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>>> The following changes since commit c2a57aae9a1c3dd7de77daf5478df10379aeeebf:
>>>
>>> Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging (2016-09-09 12:49:41 +0100)
>>>
>>> are available in the git repository at:
>>>
>>> git://github.com/stefanha/qemu.git tags/block-pull-request
>>>
>>> for you to fetch changes up to 8f1096787517c66b67cc29bab65edc0188a86326:
>>>
>>> tests: fix qvirtqueue_kick (2016-09-12 15:06:29 +0100)
>>
>>
>> /replication/primary/get_error: OK
>> /replication/secondary/get_error: OK
>>
>> Please can you rename these tests? They create false positives in
>> scripts that look in the build logs for errors by grepping for
>> "error:" or "warning:".
>
> Also, two new sanitizer errors:
>
> /home/petmay01/linaro/qemu-for-merges/block/qcow2.c:1807:41: runtime
> error: null pointer passed as argument 2, which is declared to never
> be null
> /home/petmay01/linaro/qemu-for-merges/block/qcow2-cluster.c:86:26:
> runtime error: null pointer passed as argument 2, which is declared to
> never be null
>
> both attempts to memcpy() from a NULL source pointer while running
> the tests/test-replication test.
These sanitizer errors cannot be introduced by this pull request
because no patches touched block/qcow2.c or block/qcow2-cluster.c.
Strange.
I will add a patch to fix them anyway.
Stefan
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 15/17] support replication driver in blockdev-add
2016-09-12 14:08 ` [Qemu-devel] [PULL 15/17] support replication driver in blockdev-add Stefan Hajnoczi
@ 2016-10-25 21:31 ` Eric Blake
0 siblings, 0 replies; 41+ messages in thread
From: Eric Blake @ 2016-10-25 21:31 UTC (permalink / raw)
To: Stefan Hajnoczi, qemu-devel
Cc: Peter Maydell, Changlong Xie, Wang WeiWei, zhanghailiang, Gonglei
[-- Attachment #1: Type: text/plain, Size: 1898 bytes --]
On 09/12/2016 09:08 AM, Stefan Hajnoczi wrote:
> From: Wen Congyang <wency@cn.fujitsu.com>
>
> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
> Signed-off-by: Wang WeiWei <wangww.fnst@cn.fujitsu.com>
> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
> Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Message-id: 1469602913-20979-12-git-send-email-xiecl.fnst@cn.fujitsu.com
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
> qapi/block-core.json | 23 +++++++++++++++++++++--
> 1 file changed, 21 insertions(+), 2 deletions(-)
>
> +++ b/qapi/block-core.json
> @@ -252,6 +252,7 @@
> # 2.3: 'host_floppy' deprecated
> # 2.5: 'host_floppy' dropped
> # 2.6: 'luks' added
> +# 2.8: 'replication' added
You added documentation here...
> #
> # @backing_file: #optional the name of the backing file (for copy-on-write)
> #
> @@ -1712,8 +1713,8 @@
...but not for the BlockdevDriver enum.
> 'data': [ 'archipelago', 'blkdebug', 'blkverify', 'bochs', 'cloop',
> 'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom',
> 'host_device', 'http', 'https', 'luks', 'null-aio', 'null-co',
> - 'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'tftp',
> - 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
> + 'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw',
> + 'replication', 'tftp', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
Also, I failed to notice that you added TAB damage here. Other patches
are currently proposed to touch the same area (nfs, ssh, nbd), so
hopefully one of them will correct it in the process.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* [Qemu-devel] [PULL 00/17] Block patches
@ 2019-08-19 16:17 Max Reitz
2019-08-20 12:40 ` Peter Maydell
0 siblings, 1 reply; 41+ messages in thread
From: Max Reitz @ 2019-08-19 16:17 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
The following changes since commit 3fbd3405d2b0604ea530fc7a1828f19da1e95ff9:
Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-08-17' into staging (2019-08-19 14:14:09 +0100)
are available in the Git repository at:
https://github.com/XanClic/qemu.git tags/pull-block-2019-08-19
for you to fetch changes up to fa27c478102a6b5d1c6b02c005607ad9404b915f:
doc: Preallocation does not require writing zeroes (2019-08-19 17:13:26 +0200)
----------------------------------------------------------------
Block patches:
- preallocation=falloc/full support for LUKS
- Various minor fixes
----------------------------------------------------------------
Max Reitz (16):
qemu-img: Fix bdrv_has_zero_init() use in convert
mirror: Fix bdrv_has_zero_init() use
block: Add bdrv_has_zero_init_truncate()
block: Implement .bdrv_has_zero_init_truncate()
block: Use bdrv_has_zero_init_truncate()
qcow2: Fix .bdrv_has_zero_init()
vdi: Fix .bdrv_has_zero_init()
vhdx: Fix .bdrv_has_zero_init()
iotests: Convert to preallocated encrypted qcow2
iotests: Test convert -n to pre-filled image
iotests: Full mirror to existing non-zero image
vdi: Make block_status recurse for fixed images
vmdk: Make block_status recurse for flat extents
vpc: Do not return RAW from block_status
iotests: Fix 141 when run with qed
doc: Preallocation does not require writing zeroes
Maxim Levitsky (1):
LUKS: support preallocation
qapi/block-core.json | 15 +++++---
include/block/block.h | 1 +
include/block/block_int.h | 9 +++++
block.c | 21 +++++++++++
block/crypto.c | 30 ++++++++++++++--
block/file-posix.c | 1 +
block/file-win32.c | 1 +
block/gluster.c | 4 +++
block/mirror.c | 11 ++++--
block/nfs.c | 1 +
block/parallels.c | 2 +-
block/qcow2.c | 30 +++++++++++++++-
block/qed.c | 1 +
block/raw-format.c | 6 ++++
block/rbd.c | 1 +
block/sheepdog.c | 1 +
block/ssh.c | 1 +
block/vdi.c | 16 +++++++--
block/vhdx.c | 28 +++++++++++++--
block/vmdk.c | 3 ++
block/vpc.c | 2 +-
blockdev.c | 16 +++++++--
qemu-img.c | 11 ++++--
tests/test-block-iothread.c | 2 +-
docs/qemu-block-drivers.texi | 4 +--
qemu-img.texi | 4 +--
tests/qemu-iotests/041 | 62 +++++++++++++++++++++++++++++---
tests/qemu-iotests/041.out | 4 +--
tests/qemu-iotests/122 | 17 +++++++++
tests/qemu-iotests/122.out | 8 +++++
tests/qemu-iotests/141 | 9 +++--
tests/qemu-iotests/141.out | 5 ---
tests/qemu-iotests/188 | 20 ++++++++++-
tests/qemu-iotests/188.out | 4 +++
tests/qemu-iotests/common.filter | 5 +++
35 files changed, 313 insertions(+), 43 deletions(-)
--
2.21.0
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PULL 00/17] Block patches
2019-08-19 16:17 Max Reitz
@ 2019-08-20 12:40 ` Peter Maydell
0 siblings, 0 replies; 41+ messages in thread
From: Peter Maydell @ 2019-08-20 12:40 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block
On Mon, 19 Aug 2019 at 17:17, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit 3fbd3405d2b0604ea530fc7a1828f19da1e95ff9:
>
> Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-08-17' into staging (2019-08-19 14:14:09 +0100)
>
> are available in the Git repository at:
>
> https://github.com/XanClic/qemu.git tags/pull-block-2019-08-19
>
> for you to fetch changes up to fa27c478102a6b5d1c6b02c005607ad9404b915f:
>
> doc: Preallocation does not require writing zeroes (2019-08-19 17:13:26 +0200)
>
> ----------------------------------------------------------------
> Block patches:
> - preallocation=falloc/full support for LUKS
> - Various minor fixes
>
> ----------------------------------------------------------------
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2019-08-20 12:41 UTC | newest]
Thread overview: 41+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-09-12 14:08 [Qemu-devel] [PULL 00/17] Block patches Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 01/17] linux-aio: consume events in userspace instead of calling io_getevents Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 02/17] linux-aio: split processing events function Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 03/17] linux-aio: process completions from ioq_submit() Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 04/17] virtio-blk: rename virtio_device_info to virtio_blk_info Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 05/17] block: unblock backup operations in backing file Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 06/17] Backup: clear all bitmap when doing block checkpoint Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 07/17] Backup: export interfaces for extra serialization Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 08/17] block: Link backup into block core Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 09/17] docs: block replication's description Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 10/17] mirror: auto complete active commit Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 11/17] configure: support replication Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 12/17] replication: Introduce new APIs to do replication operation Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 13/17] replication: Implement new driver for block replication Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 14/17] tests: add unit test case for replication Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 15/17] support replication driver in blockdev-add Stefan Hajnoczi
2016-10-25 21:31 ` Eric Blake
2016-09-12 14:08 ` [Qemu-devel] [PULL 16/17] MAINTAINERS: add maintainer for replication Stefan Hajnoczi
2016-09-12 14:08 ` [Qemu-devel] [PULL 17/17] tests: fix qvirtqueue_kick Stefan Hajnoczi
2016-09-12 15:12 ` [Qemu-devel] [PULL 00/17] Block patches Peter Maydell
2016-09-12 15:56 ` Peter Maydell
2016-09-13 1:11 ` Fam Zheng
2016-09-13 8:34 ` Stefan Hajnoczi
2016-09-13 8:53 ` Stefan Hajnoczi
-- strict thread matches above, loose matches on Subject: below --
2019-08-19 16:17 Max Reitz
2019-08-20 12:40 ` Peter Maydell
2016-01-20 16:24 Kevin Wolf
2016-01-21 13:42 ` Peter Maydell
2015-07-02 9:19 Stefan Hajnoczi
2015-07-02 12:46 ` Peter Maydell
2015-07-07 13:47 ` Peter Maydell
2015-07-08 13:52 ` Stefan Hajnoczi
2015-06-05 11:57 Stefan Hajnoczi
2015-06-05 13:53 ` Peter Maydell
2014-06-02 13:56 Kevin Wolf
2014-06-02 14:46 ` Peter Maydell
2014-05-09 19:03 Stefan Hajnoczi
2014-05-13 10:32 ` Peter Maydell
2013-10-29 16:30 Kevin Wolf
2010-07-06 15:33 Kevin Wolf
2010-07-06 19:07 ` Anthony Liguori
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).