[Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

Block replication is a very important feature which is used for
continuous checkpoints(for example: COLO).

Usage:
Please refer to docs/block-replication.txt

You can get the patch here:
https://github.com/wencongyang/qemu-colo/commits/block-replication-v7

You can get ths patch with framework here:
https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2

TODO:
1. Continuous block replication. It will be started after basic functions
   are accepted.

Changs Log:
V7:
1. Implement adding/removing quorum child. Remove the option non-connect.
2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
V6:
1. Rebase to the newest qemu.
V5:
1. Address the comments from Gong Lei
2. Speed the failover up. The secondary vm can take over very quickly even
   if there are too many I/O requests.
V4:
1. Introduce a new driver replication to avoid touch nbd and qcow2.
V3:
1: use error_setg() instead of error_set()
2. Add a new block job API
3. Active disk, hidden disk and nbd target uses the same AioContext
4. Add a testcase to test new hbitmap API
V2:
1. Redesign the secondary qemu(use image-fleecing)
2. Use Error objects to return error message
3. Address the comments from Max Reitz and Eric Blake


Wen Congyang (17):
  Add new block driver interface to add/delete a BDS's child
  quorum: implement block driver interfaces add/delete a BDS's child
  hmp: add monitor command to add/remove a child
  introduce a new API qemu_opts_absorb_qdict_by_index()
  quorum: allow ignoring child errors
  introduce a new API to enable/disable attach device model
  introduce a new API to check if blk is attached
  block: make bdrv_put_ref_bh_schedule() as a public API
  Backup: clear all bitmap when doing block checkpoint
  allow writing to the backing file
  Allow creating backup jobs when opening BDS
  block: Allow references for backing files
  docs: block replication's description
  Add new block driver interfaces to control block replication
  skip nbd_target when starting block replication
  quorum: implement block driver interfaces for block replication
  Implement new driver for block replication

 block.c                        | 198 +++++++++++++++++-
 block/Makefile.objs            |   3 +-
 block/backup.c                 |  13 ++
 block/block-backend.c          |  33 +++
 block/quorum.c                 | 244 ++++++++++++++++++++++-
 block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
 blockdev.c                     |  90 ++++++---
 blockjob.c                     |  10 +
 docs/block-replication.txt     | 179 +++++++++++++++++
 hmp-commands.hx                |  28 +++
 include/block/block.h          |  11 +
 include/block/block_int.h      |  19 ++
 include/block/blockjob.h       |  12 ++
 include/qemu/option.h          |   2 +
 include/sysemu/block-backend.h |   3 +
 include/sysemu/blockdev.h      |   2 +
 qapi/block.json                |  16 ++
 util/qemu-option.c             |  44 ++++
 18 files changed, 1303 insertions(+), 47 deletions(-)
 create mode 100644 block/replication.c
 create mode 100644 docs/block-replication.txt

-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 01/17] Add new block driver interface to add/delete a BDS's child

[Wen Congyang]

In some cases, we want to take a quorum child offline, and take
another child online.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 block.c                   | 39 +++++++++++++++++++++++++++++++++++++++
 include/block/block.h     |  4 ++++
 include/block/block_int.h |  5 +++++
 3 files changed, 48 insertions(+)

diff --git a/block.c b/block.c
index 81233be..617e431 100644
--- a/block.c
+++ b/block.c
@@ -4207,3 +4207,42 @@ BlockAcctStats *bdrv_get_stats(BlockDriverState *bs)
 {
     return &bs->stats;
 }
+
+/*
+ * Hot add/remove a BDS's child. So the user can take a child offline when
+ * it is broken and take a new child online
+ */
+void bdrv_add_child(BlockDriverState *bs, QDict *options, Error **errp)
+{
+
+    if (!bs->drv || !bs->drv->bdrv_add_child) {
+        error_setg(errp, "this feature or command is not currently supported");
+        return;
+    }
+
+    bs->drv->bdrv_add_child(bs, options, errp);
+}
+
+void bdrv_del_child(BlockDriverState *bs, BlockDriverState *child_bs,
+                    Error **errp)
+{
+    BdrvChild *child;
+
+    if (!bs->drv || !bs->drv->bdrv_del_child) {
+        error_setg(errp, "this feature or command is not currently supported");
+        return;
+    }
+
+    QLIST_FOREACH(child, &bs->children, next) {
+        if (child->bs == child_bs) {
+            break;
+        }
+    }
+
+    if (!child) {
+        error_setg(errp, "Invalid child");
+        return;
+    }
+
+    bs->drv->bdrv_del_child(bs, child_bs, errp);
+}
diff --git a/include/block/block.h b/include/block/block.h
index 07bb724..c43160d 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -605,4 +605,8 @@ void bdrv_flush_io_queue(BlockDriverState *bs);
 
 BlockAcctStats *bdrv_get_stats(BlockDriverState *bs);
 
+void bdrv_add_child(BlockDriverState *bs, QDict *options, Error **errp);
+void bdrv_del_child(BlockDriverState *bs, BlockDriverState *child,
+                    Error **errp);
+
 #endif
diff --git a/include/block/block_int.h b/include/block/block_int.h
index b0476fc..d7af9e3 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -288,6 +288,11 @@ struct BlockDriver {
      */
     int (*bdrv_probe_geometry)(BlockDriverState *bs, HDGeometry *geo);
 
+    void (*bdrv_add_child)(BlockDriverState *bs, QDict *options,
+                           Error **errp);
+    void (*bdrv_del_child)(BlockDriverState *bs, BlockDriverState *child,
+                           Error **errp);
+
     QLIST_ENTRY(BlockDriver) list;
 };
 
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 02/17] quorum: implement block driver interfaces add/delete a BDS's child

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Cc: Alberto Garcia <berto@igalia.com>
---
 block/quorum.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 70 insertions(+), 2 deletions(-)

diff --git a/block/quorum.c b/block/quorum.c
index a7df17c..4a15493 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -67,6 +67,9 @@ typedef struct QuorumVotes {
 typedef struct BDRVQuorumState {
     BlockDriverState **bs; /* children BlockDriverStates */
     int num_children;      /* children count */
+    int max_children;      /* The maximum children count, we need to reallocate
+                            * bs if num_children will larger than maximum.
+                            */
     int threshold;         /* if less than threshold children reads gave the
                             * same result a quorum error occurs.
                             */
@@ -879,9 +882,9 @@ static int quorum_open(BlockDriverState *bs, QDict *options, int flags,
         ret = -EINVAL;
         goto exit;
     }
-    if (s->num_children < 2) {
+    if (s->num_children < 1) {
         error_setg(&local_err,
-                   "Number of provided children must be greater than 1");
+                   "Number of provided children must be 1 or more");
         ret = -EINVAL;
         goto exit;
     }
@@ -930,6 +933,7 @@ static int quorum_open(BlockDriverState *bs, QDict *options, int flags,
     /* allocate the children BlockDriverState array */
     s->bs = g_new0(BlockDriverState *, s->num_children);
     opened = g_new0(bool, s->num_children);
+    s->max_children = s->num_children;
 
     for (i = 0; i < s->num_children; i++) {
         char indexstr[32];
@@ -1000,6 +1004,67 @@ static void quorum_attach_aio_context(BlockDriverState *bs,
     }
 }
 
+static void quorum_add_child(BlockDriverState *bs, QDict *options, Error **errp)
+{
+    BDRVQuorumState *s = bs->opaque;
+    int ret;
+    Error *local_err = NULL;
+
+    bdrv_drain(bs);
+
+    if (s->num_children == s->max_children) {
+        if (s->max_children >= INT_MAX / 2) {
+            error_setg(errp, "Too many children");
+            return;
+        }
+
+        s->bs = g_renew(BlockDriverState *, s->bs, s->max_children * 2);
+        memset(&s->bs[s->max_children], 0, s->max_children * sizeof(void *));
+        s->max_children *= 2;
+    }
+
+    ret = bdrv_open_image(&s->bs[s->num_children], NULL, options, "child", bs,
+                          &child_format, false, &local_err);
+    if (ret < 0) {
+        error_propagate(errp, local_err);
+        return;
+    }
+    s->num_children++;
+
+    /* TODO: Update vote_threshold */
+}
+
+static void quorum_del_child(BlockDriverState *bs, BlockDriverState *child_bs,
+                             Error **errp)
+{
+    BDRVQuorumState *s = bs->opaque;
+    int i;
+
+    for (i = 0; i < s->num_children; i++) {
+        if (s->bs[i] == child_bs) {
+            break;
+        }
+    }
+
+    if (i == s->num_children) {
+        error_setg(errp, "Invalid child");
+        return;
+    }
+
+    if (s->num_children == 1) {
+        error_setg(errp, "Cannot remove the last child");
+        return;
+    }
+
+    bdrv_drain(bs);
+    /* We can safe remove this child now */
+    memmove(&s->bs[i], &s->bs[i+1], (s->num_children - i - 1) * sizeof(void *));
+    s->num_children--;
+    s->bs[s->num_children] = NULL;
+
+    /* TODO: update vote_threshold */
+}
+
 static void quorum_refresh_filename(BlockDriverState *bs)
 {
     BDRVQuorumState *s = bs->opaque;
@@ -1054,6 +1119,9 @@ static BlockDriver bdrv_quorum = {
     .bdrv_detach_aio_context            = quorum_detach_aio_context,
     .bdrv_attach_aio_context            = quorum_attach_aio_context,
 
+    .bdrv_add_child                     = quorum_add_child,
+    .bdrv_del_child                     = quorum_del_child,
+
     .is_filter                          = true,
     .bdrv_recurse_is_first_non_filter   = quorum_recurse_is_first_non_filter,
 };
-- 
2.4.3

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 02/17] quorum: implement block driver interfaces add/delete a BDS's child

[Alberto Garcia]

On Tue 30 Jun 2015 05:34:30 AM CEST, Wen Congyang wrote:

> +static void quorum_add_child(BlockDriverState *bs, QDict *options, Error **errp)
> +{
> +    BDRVQuorumState *s = bs->opaque;
> +    int ret;
> +    Error *local_err = NULL;
> +
> +    bdrv_drain(bs);
> +
> +    if (s->num_children == s->max_children) {
> +        if (s->max_children >= INT_MAX / 2) {
> +            error_setg(errp, "Too many children");
> +            return;
> +        }
> +
> +        s->bs = g_renew(BlockDriverState *, s->bs, s->max_children * 2);
> +        memset(&s->bs[s->max_children], 0, s->max_children * sizeof(void *));
> +        s->max_children *= 2;
> +    }
> +
> +    ret = bdrv_open_image(&s->bs[s->num_children], NULL, options, "child", bs,
> +                          &child_format, false, &local_err);
> +    if (ret < 0) {
> +        error_propagate(errp, local_err);
> +        return;
> +    }
> +    s->num_children++;
> +
> +    /* TODO: Update vote_threshold */
> +}

A few comments:

1) Is there any reason why you grow the array exponentially instead of
   adding a fixed number of elements when the limit is reached? I guess
   in practice the number of children is never going to be too high
   anyway...

2) Did you think of any API to update vote_threshold? Currently it
   cannot be higher than num_children, so it's effectively limited by
   the number of children that are attached when the quorum device is
   created.

3) I don't think it's necessary to set to NULL the pointers in s->bs[i]
   when i >= num_children. There's no way to access those pointers
   anyway. Same for the ' s->bs[s->num_children] = NULL; ' bit in
   quorum_del_child(). I also think that using memset() for setting NULL
   pointers is not portable, although QEMU is already doing this in a
   few places.

Otherwise the code looks good to me. Thanks!

Berto

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 02/17] quorum: implement block driver interfaces add/delete a BDS's child

[Wen Congyang]

At 2015/7/2 22:02, Alberto Garcia Wrote:
> On Tue 30 Jun 2015 05:34:30 AM CEST, Wen Congyang wrote:
>
>> +static void quorum_add_child(BlockDriverState *bs, QDict *options, Error **errp)
>> +{
>> +    BDRVQuorumState *s = bs->opaque;
>> +    int ret;
>> +    Error *local_err = NULL;
>> +
>> +    bdrv_drain(bs);
>> +
>> +    if (s->num_children == s->max_children) {
>> +        if (s->max_children >= INT_MAX / 2) {
>> +            error_setg(errp, "Too many children");
>> +            return;
>> +        }
>> +
>> +        s->bs = g_renew(BlockDriverState *, s->bs, s->max_children * 2);
>> +        memset(&s->bs[s->max_children], 0, s->max_children * sizeof(void *));
>> +        s->max_children *= 2;
>> +    }
>> +
>> +    ret = bdrv_open_image(&s->bs[s->num_children], NULL, options, "child", bs,
>> +                          &child_format, false, &local_err);
>> +    if (ret < 0) {
>> +        error_propagate(errp, local_err);
>> +        return;
>> +    }
>> +    s->num_children++;
>> +
>> +    /* TODO: Update vote_threshold */
>> +}
>
> A few comments:
>
> 1) Is there any reason why you grow the array exponentially instead of
>     adding a fixed number of elements when the limit is reached? I guess
>     in practice the number of children is never going to be too high
>     anyway...

Yes, will do it in the next version.

>
> 2) Did you think of any API to update vote_threshold? Currently it
>     cannot be higher than num_children, so it's effectively limited by
>     the number of children that are attached when the quorum device is
>     created.

The things I think about it is: if vote_threshold-- when deleting a 
child, vote_threshold
will be less than 0. If we don't do vote_threshold-- when it is 1, the 
vote_threshold will
be changed when all children are added again. Which behavior is better?

>
> 3) I don't think it's necessary to set to NULL the pointers in s->bs[i]
>     when i >= num_children. There's no way to access those pointers
>     anyway. Same for the ' s->bs[s->num_children] = NULL; ' bit in
>     quorum_del_child(). I also think that using memset() for setting NULL
>     pointers is not portable, although QEMU is already doing this in a
>     few places.

OK, will remove it in the next version. Just a question: why is using 
memset()
for setting NULL pointers is not prtable?

Thanks
Wen Congyang

>
> Otherwise the code looks good to me. Thanks!
>
> Berto
>
>

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 02/17] quorum: implement block driver interfaces add/delete a BDS's child

[Alberto Garcia]

On Thu 02 Jul 2015 04:30:50 PM CEST, Wen Congyang wrote:

>> 2) Did you think of any API to update vote_threshold? Currently it
>>     cannot be higher than num_children, so it's effectively limited by
>>     the number of children that are attached when the quorum device is
>>     created.
>
> The things I think about it is: if vote_threshold-- when deleting a
> child, vote_threshold will be less than 0. If we don't do
> vote_threshold-- when it is 1, the vote_threshold will be changed when
> all children are added again. Which behavior is better?

Adding/removing a child and changing vote_threshold are in principle two
unrelated operations. One user could want to modify one but not the
other, so linking them together does not seem like a good idea. A
specific API to change vote_threshold could be added when the use case
appears (currently no one seems to have missed it).

So I think I would keep these things separate, I would also remove the
"TODO" comments that mention it.

With the current patches (that do not touch vote_threshold), you can
remove as many children as you want as long as there's at least one
left. However if you end up with num_chilren < vote_threshold then you
will get read errors. I see two alternatives:

  - Allow that and expect that the user will add the necessary children
    afterwards.

  - Forbid that completely and return an error.

I think I prefer the second option.

>> 3) I don't think it's necessary to set to NULL the pointers in
>> s->bs[i] when i >= num_children. There's no way to access those
>> pointers anyway. Same for the ' s->bs[s->num_children] = NULL; ' bit
>> in quorum_del_child(). I also think that using memset() for setting
>> NULL pointers is not portable, although QEMU is already doing this in
>> a few places.
>
> OK, will remove it in the next version. Just a question: why is using
> memset() for setting NULL pointers is not prtable?

The standard allows for null pointers to be internally represented by
nonzero bit patterns. However I'm not aware of any system that we
support that does that.

   http://c-faq.com/null/confusion4.html
   http://c-faq.com/null/machexamp.html

Berto

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 02/17] quorum: implement block driver interfaces add/delete a BDS's child

[Wen Congyang]

On 07/02/2015 11:21 PM, Alberto Garcia wrote:
> On Thu 02 Jul 2015 04:30:50 PM CEST, Wen Congyang wrote:
> 
>>> 2) Did you think of any API to update vote_threshold? Currently it
>>>     cannot be higher than num_children, so it's effectively limited by
>>>     the number of children that are attached when the quorum device is
>>>     created.
>>
>> The things I think about it is: if vote_threshold-- when deleting a
>> child, vote_threshold will be less than 0. If we don't do
>> vote_threshold-- when it is 1, the vote_threshold will be changed when
>> all children are added again. Which behavior is better?
> 
> Adding/removing a child and changing vote_threshold are in principle two
> unrelated operations. One user could want to modify one but not the
> other, so linking them together does not seem like a good idea. A
> specific API to change vote_threshold could be added when the use case
> appears (currently no one seems to have missed it).
> 
> So I think I would keep these things separate, I would also remove the
> "TODO" comments that mention it.
> 
> With the current patches (that do not touch vote_threshold), you can
> remove as many children as you want as long as there's at least one
> left. However if you end up with num_chilren < vote_threshold then you
> will get read errors. I see two alternatives:
> 
>   - Allow that and expect that the user will add the necessary children
>     afterwards.
> 
>   - Forbid that completely and return an error.
> 
> I think I prefer the second option.

OK, I will do it.

> 
>>> 3) I don't think it's necessary to set to NULL the pointers in
>>> s->bs[i] when i >= num_children. There's no way to access those
>>> pointers anyway. Same for the ' s->bs[s->num_children] = NULL; ' bit
>>> in quorum_del_child(). I also think that using memset() for setting
>>> NULL pointers is not portable, although QEMU is already doing this in
>>> a few places.
>>
>> OK, will remove it in the next version. Just a question: why is using
>> memset() for setting NULL pointers is not prtable?
> 
> The standard allows for null pointers to be internally represented by
> nonzero bit patterns. However I'm not aware of any system that we
> support that does that.
> 
>    http://c-faq.com/null/confusion4.html
>    http://c-faq.com/null/machexamp.html

Thanks for your explantion.

Wen Congyang

> 
> Berto
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 02/17] quorum: implement block driver interfaces add/delete a BDS's child

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1187 bytes --]

On 07/02/2015 09:21 AM, Alberto Garcia wrote:

> 
>>> 3) I don't think it's necessary to set to NULL the pointers in
>>> s->bs[i] when i >= num_children. There's no way to access those
>>> pointers anyway. Same for the ' s->bs[s->num_children] = NULL; ' bit
>>> in quorum_del_child(). I also think that using memset() for setting
>>> NULL pointers is not portable, although QEMU is already doing this in
>>> a few places.
>>
>> OK, will remove it in the next version. Just a question: why is using
>> memset() for setting NULL pointers is not prtable?
> 
> The standard allows for null pointers to be internally represented by
> nonzero bit patterns. However I'm not aware of any system that we
> support that does that.
> 
>    http://c-faq.com/null/confusion4.html
>    http://c-faq.com/null/machexamp.html

What's more, POSIX has very recently taken the stance that memset() to 0
will work on all POSIX systems, even if someone is insane enough to use
a non-zero bit pattern for NULL on modern hardware:

http://austingroupbugs.net/view.php?id=940

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

[Qemu-devel] [PATCH COLO-BLOCK v7 03/17] hmp: add monitor command to add/remove a child

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 blockdev.c                | 53 +++++++++++++++++++++++++++++++++++++++++++++++
 hmp-commands.hx           | 28 +++++++++++++++++++++++++
 include/sysemu/blockdev.h |  2 ++
 3 files changed, 83 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index b354676..935c081 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2189,6 +2189,59 @@ void hmp_drive_del(Monitor *mon, const QDict *qdict)
     aio_context_release(aio_context);
 }
 
+void hmp_child_add(Monitor *mon, const QDict *qdict)
+{
+    const char *id = qdict_get_str(qdict, "id");
+    const char *optstr = qdict_get_str(qdict, "opts");
+    QemuOpts *opts;
+    QDict *bs_opts = qdict_new();
+    BlockDriverState *bs;
+    Error *local_err = NULL;
+
+    opts = drive_def(optstr);
+    if (!opts) {
+        /* We have reported error in drive_def */
+        return;
+    }
+    bs_opts = qemu_opts_to_qdict(opts, bs_opts);
+
+    bs = bdrv_lookup_bs(id, id, &local_err);
+    if (!bs) {
+        error_report_err(local_err);
+        return;
+    }
+
+    bdrv_add_child(bs, bs_opts, &local_err);
+    if (local_err) {
+        error_report_err(local_err);
+    }
+}
+
+void hmp_child_del(Monitor *mon, const QDict *qdict)
+{
+    const char *id = qdict_get_str(qdict, "id");
+    const char *child_id = qdict_get_str(qdict, "child");
+    BlockDriverState *bs, *child_bs;
+    Error *local_err = NULL;
+
+    bs = bdrv_lookup_bs(id, id, &local_err);
+    if (!bs) {
+        error_report_err(local_err);
+        return;
+    }
+
+    child_bs = bdrv_lookup_bs(child_id, child_id, &local_err);
+    if (!child_bs) {
+        error_report_err(local_err);
+        return;
+    }
+
+    bdrv_del_child(bs, child_bs, &local_err);
+    if (local_err) {
+        error_report_err(local_err);
+    }
+}
+
 void qmp_block_resize(bool has_device, const char *device,
                       bool has_node_name, const char *node_name,
                       int64_t size, Error **errp)
diff --git a/hmp-commands.hx b/hmp-commands.hx
index d3b7932..1d5b392 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -193,6 +193,34 @@ actions (drive options rerror, werror).
 ETEXI
 
     {
+        .name       = "child_add",
+        .args_type  = "id:B,opts:s",
+        .params     = "device child.file=file",
+        .help       = "add a child to a BDS",
+        .mhandler.cmd = hmp_child_add,
+    },
+
+STEXI
+@item child_add @var{device} @var{options}
+@findex child_add
+Add a child to the block device.
+ETEXI
+
+    {
+        .name       = "child_del",
+        .args_type  = "id:B,child:B",
+        .params     = "parent child",
+        .help       = "remove a child from a BDS",
+        .mhandler.cmd = hmp_child_del,
+    },
+
+STEXI
+@item child_del @var{parent device} @var{child device}
+@findex child_del
+Remove a child from the parent device.
+ETEXI
+
+    {
         .name       = "change",
         .args_type  = "device:B,target:F,arg:s?",
         .params     = "device filename [format]",
diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index 3104150..594bfab 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -67,4 +67,6 @@ void qmp_change_blockdev(const char *device, const char *filename,
                          const char *format, Error **errp);
 void hmp_commit(Monitor *mon, const QDict *qdict);
 void hmp_drive_del(Monitor *mon, const QDict *qdict);
+void hmp_child_add(Monitor *mon, const QDict *qdict);
+void hmp_child_del(Monitor *mon, const QDict *qdict);
 #endif
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 04/17] introduce a new API qemu_opts_absorb_qdict_by_index()

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 include/qemu/option.h |  2 ++
 util/qemu-option.c    | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/include/qemu/option.h b/include/qemu/option.h
index 57e51c9..725a781 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -129,6 +129,8 @@ QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
                                Error **errp);
 QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
 void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
+void qemu_opts_absorb_qdict_by_index(QemuOpts *opts, QDict *qdict,
+                                     const char *index, Error **errp);
 
 typedef int (*qemu_opts_loopfunc)(void *opaque, QemuOpts *opts, Error **errp);
 int qemu_opts_foreach(QemuOptsList *list, qemu_opts_loopfunc func,
diff --git a/util/qemu-option.c b/util/qemu-option.c
index efe9d27..a93a269 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -1021,6 +1021,50 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
 }
 
 /*
+ * Adds all QDict entries to the QemuOpts that can be added and removes them
+ * from the QDict. The key starts with "%index." in the %qdict. When this
+ * function returns, the QDict contains only those entries that couldn't be
+ * added to the QemuOpts.
+ */
+void qemu_opts_absorb_qdict_by_index(QemuOpts *opts, QDict *qdict,
+                                     const char *index, Error **errp)
+{
+    const QDictEntry *entry, *next;
+    const char *key;
+    int len = strlen(index);
+
+    entry = qdict_first(qdict);
+
+    while (entry != NULL) {
+        Error *local_err = NULL;
+        OptsFromQDictState state = {
+            .errp = &local_err,
+            .opts = opts,
+        };
+
+        next = qdict_next(qdict, entry);
+        if (strncmp(entry->key, index, len) || *(entry->key + len) != '.') {
+            entry = next;
+            continue;
+        }
+
+        key = entry->key + len + 1;
+
+        if (find_desc_by_name(opts->list->desc, key)) {
+            qemu_opts_from_qdict_1(key, entry->value, &state);
+            if (local_err) {
+                error_propagate(errp, local_err);
+                return;
+            } else {
+                qdict_del(qdict, entry->key);
+            }
+        }
+
+        entry = next;
+    }
+}
+
+/*
  * Convert from QemuOpts to QDict.
  * The QDict values are of type QString.
  * TODO We'll want to use types appropriate for opt->desc->type, but
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 05/17] quorum: allow ignoring child errors

[Wen Congyang]

If the child is not ready, read/write/getlength/flush will
return -errno. It is not critical error, and can be ignored:
1. read/write:
   Just not report the error event.
2. getlength:
   just ignore it. If all children's getlength return -errno,
   and be ignored, return -EIO.
3. flush:
   Just ignore it. If all children's getlength return -errno,
   and be ignored, return 0.

Usage: children.x.ignore-errors=true

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Cc: Alberto Garcia <berto@igalia.com>
---
 block/quorum.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 87 insertions(+), 7 deletions(-)

diff --git a/block/quorum.c b/block/quorum.c
index 4a15493..6774d4c 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -31,6 +31,7 @@
 #define QUORUM_OPT_BLKVERIFY      "blkverify"
 #define QUORUM_OPT_REWRITE        "rewrite-corrupted"
 #define QUORUM_OPT_READ_PATTERN   "read-pattern"
+#define QUORUM_CHILDREN_OPT_IGNORE_ERRORS   "ignore-errors"
 
 /* This union holds a vote hash value */
 typedef union QuorumVoteValue {
@@ -66,6 +67,7 @@ typedef struct QuorumVotes {
 /* the following structure holds the state of one quorum instance */
 typedef struct BDRVQuorumState {
     BlockDriverState **bs; /* children BlockDriverStates */
+    bool *ignore_errors;   /* ignore children's error? */
     int num_children;      /* children count */
     int max_children;      /* The maximum children count, we need to reallocate
                             * bs if num_children will larger than maximum.
@@ -101,6 +103,7 @@ typedef struct QuorumChildRequest {
     uint8_t *buf;
     int ret;
     QuorumAIOCB *parent;
+    int index;
 } QuorumChildRequest;
 
 /* Quorum will use the following structure to track progress of each read/write
@@ -213,6 +216,7 @@ static QuorumAIOCB *quorum_aio_get(BDRVQuorumState *s,
         acb->qcrs[i].buf = NULL;
         acb->qcrs[i].ret = 0;
         acb->qcrs[i].parent = acb;
+        acb->qcrs[i].index = i;
     }
 
     return acb;
@@ -306,7 +310,7 @@ static void quorum_aio_cb(void *opaque, int ret)
     acb->count++;
     if (ret == 0) {
         acb->success_count++;
-    } else {
+    } else if (!s->ignore_errors[sacb->index]) {
         quorum_report_bad(acb, sacb->aiocb->bs->node_name, ret);
     }
     assert(acb->count <= s->num_children);
@@ -721,19 +725,31 @@ static BlockAIOCB *quorum_aio_writev(BlockDriverState *bs,
 static int64_t quorum_getlength(BlockDriverState *bs)
 {
     BDRVQuorumState *s = bs->opaque;
-    int64_t result;
+    int64_t result = -EIO;
     int i;
 
     /* check that all file have the same length */
-    result = bdrv_getlength(s->bs[0]);
-    if (result < 0) {
-        return result;
-    }
-    for (i = 1; i < s->num_children; i++) {
+    for (i = 0; i < s->num_children; i++) {
         int64_t value = bdrv_getlength(s->bs[i]);
+
         if (value < 0) {
             return value;
         }
+
+        if (value == 0 && s->ignore_errors[i]) {
+            /*
+             * If the child is not ready, it cannot return -errno,
+             * otherwise refresh_total_sectors() will fail when
+             * we open the child.
+             */
+            continue;
+        }
+
+        if (result == -EIO) {
+            result = value;
+            continue;
+        }
+
         if (value != result) {
             return -EIO;
         }
@@ -771,6 +787,9 @@ static coroutine_fn int quorum_co_flush(BlockDriverState *bs)
 
     for (i = 0; i < s->num_children; i++) {
         result = bdrv_co_flush(s->bs[i]);
+        if (result < 0 && s->ignore_errors[i]) {
+            result = 0;
+        }
         result_value.l = result;
         quorum_count_vote(&error_votes, &result_value, i);
     }
@@ -845,6 +864,19 @@ static QemuOptsList quorum_runtime_opts = {
     },
 };
 
+static QemuOptsList quorum_children_common_opts = {
+    .name = "quorum children",
+    .head = QTAILQ_HEAD_INITIALIZER(quorum_children_common_opts.head),
+    .desc = {
+        {
+            .name = QUORUM_CHILDREN_OPT_IGNORE_ERRORS,
+            .type = QEMU_OPT_BOOL,
+            .help = "ignore child I/O error",
+        },
+        { /* end of list */ }
+    },
+};
+
 static int parse_read_pattern(const char *opt)
 {
     int i;
@@ -863,6 +895,37 @@ static int parse_read_pattern(const char *opt)
     return -EINVAL;
 }
 
+static int parse_children_options(BDRVQuorumState *s, QDict *options,
+                                  const char *indexstr, int index,
+                                  Error **errp)
+{
+    QemuOpts *children_opts = NULL;
+    Error *local_err = NULL;
+    int ret = 0;
+    bool value;
+
+    children_opts = qemu_opts_create(&quorum_children_common_opts, NULL, 0,
+                                     &error_abort);
+    qemu_opts_absorb_qdict_by_index(children_opts, options, indexstr,
+                                    &local_err);
+    if (local_err) {
+        ret = -EINVAL;
+        goto out;
+    }
+
+    value = qemu_opt_get_bool(children_opts, QUORUM_CHILDREN_OPT_IGNORE_ERRORS,
+                              false);
+    s->ignore_errors[index] = value;
+
+out:
+    qemu_opts_del(children_opts);
+    /* propagate error */
+    if (local_err) {
+        error_propagate(errp, local_err);
+    }
+    return ret;
+}
+
 static int quorum_open(BlockDriverState *bs, QDict *options, int flags,
                        Error **errp)
 {
@@ -934,12 +997,18 @@ static int quorum_open(BlockDriverState *bs, QDict *options, int flags,
     s->bs = g_new0(BlockDriverState *, s->num_children);
     opened = g_new0(bool, s->num_children);
     s->max_children = s->num_children;
+    s->ignore_errors = g_new0(bool, s->num_children);
 
     for (i = 0; i < s->num_children; i++) {
         char indexstr[32];
         ret = snprintf(indexstr, 32, "children.%d", i);
         assert(ret < 32);
 
+        ret = parse_children_options(s, options, indexstr, i, &local_err);
+        if (ret < 0) {
+            goto close_exit;
+        }
+
         ret = bdrv_open_image(&s->bs[i], NULL, options, indexstr, bs,
                               &child_format, false, &local_err);
         if (ret < 0) {
@@ -981,6 +1050,7 @@ static void quorum_close(BlockDriverState *bs)
     }
 
     g_free(s->bs);
+    g_free(s->ignore_errors);
 }
 
 static void quorum_detach_aio_context(BlockDriverState *bs)
@@ -1020,9 +1090,17 @@ static void quorum_add_child(BlockDriverState *bs, QDict *options, Error **errp)
 
         s->bs = g_renew(BlockDriverState *, s->bs, s->max_children * 2);
         memset(&s->bs[s->max_children], 0, s->max_children * sizeof(void *));
+        s->ignore_errors = g_renew(bool, s->ignore_errors, s->max_children * 2);
         s->max_children *= 2;
     }
 
+    ret = parse_children_options(s, options, "child", s->num_children,
+                                 &local_err);
+    if (ret < 0) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
     ret = bdrv_open_image(&s->bs[s->num_children], NULL, options, "child", bs,
                           &child_format, false, &local_err);
     if (ret < 0) {
@@ -1059,6 +1137,8 @@ static void quorum_del_child(BlockDriverState *bs, BlockDriverState *child_bs,
     bdrv_drain(bs);
     /* We can safe remove this child now */
     memmove(&s->bs[i], &s->bs[i+1], (s->num_children - i - 1) * sizeof(void *));
+    memmove(&s->ignore_errors[i], &s->ignore_errors[i+1],
+            (s->num_children - i - 1) * sizeof(bool));
     s->num_children--;
     s->bs[s->num_children] = NULL;
 
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 06/17] introduce a new API to enable/disable attach device model

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
---
 block/block-backend.c          | 24 ++++++++++++++++++++++++
 include/sysemu/block-backend.h |  2 ++
 2 files changed, 26 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index aee8a12..72d8b2c 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -344,6 +344,30 @@ void *blk_get_attached_dev(BlockBackend *blk)
 }
 
 /*
+ * Disable to attach a device mode to @blk.
+ * Return 0 on success, -EBUSY when a device model is attached already.
+ */
+int blk_disable_attach_dev(BlockBackend *blk)
+{
+    if (blk->dev) {
+        return blk->dev == (void *)-1 ? 0 : -EBUSY;
+    }
+
+    blk->dev = (void *)-1;
+    return 0;
+}
+
+/*
+ * Enable to attach a device mode to @blk.
+ */
+void blk_enable_attach_dev(BlockBackend *blk)
+{
+    if (blk->dev == (void *)-1) {
+        blk->dev = NULL;
+    }
+}
+
+/*
  * Set @blk's device model callbacks to @ops.
  * @opaque is the opaque argument to pass to the callbacks.
  * This is for use by device models.
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 8fc960f..7619a9f 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -80,6 +80,8 @@ int blk_attach_dev(BlockBackend *blk, void *dev);
 void blk_attach_dev_nofail(BlockBackend *blk, void *dev);
 void blk_detach_dev(BlockBackend *blk, void *dev);
 void *blk_get_attached_dev(BlockBackend *blk);
+int blk_disable_attach_dev(BlockBackend *blk);
+void blk_enable_attach_dev(BlockBackend *blk);
 void blk_set_dev_ops(BlockBackend *blk, const BlockDevOps *ops, void *opaque);
 int blk_read(BlockBackend *blk, int64_t sector_num, uint8_t *buf,
              int nb_sectors);
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 07/17] introduce a new API to check if blk is attached

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
---
 block.c                        | 4 ++--
 block/block-backend.c          | 9 +++++++++
 include/sysemu/block-backend.h | 1 +
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/block.c b/block.c
index 617e431..dbbba5d 100644
--- a/block.c
+++ b/block.c
@@ -2032,7 +2032,7 @@ void bdrv_swap(BlockDriverState *bs_new, BlockDriverState *bs_old)
     }
 
     /* bs_new must be unattached and shouldn't have anything fancy enabled */
-    assert(!bs_new->blk);
+    assert(!blk_is_attached(bs_new->blk));
     assert(QLIST_EMPTY(&bs_new->dirty_bitmaps));
     assert(bs_new->job == NULL);
     assert(bs_new->io_limits_enabled == false);
@@ -2049,7 +2049,7 @@ void bdrv_swap(BlockDriverState *bs_new, BlockDriverState *bs_old)
     bdrv_move_feature_fields(bs_new, &tmp);
 
     /* bs_new must remain unattached */
-    assert(!bs_new->blk);
+    assert(!blk_is_attached(bs_new->blk));
 
     /* Check a few fields that should remain attached to the device */
     assert(bs_new->job == NULL);
diff --git a/block/block-backend.c b/block/block-backend.c
index 72d8b2c..1463c37 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -368,6 +368,15 @@ void blk_enable_attach_dev(BlockBackend *blk)
 }
 
 /*
+ * Return true if a device model is attached to @blk already,
+ * otherwise, return false.
+ */
+bool blk_is_attached(BlockBackend *blk)
+{
+    return blk != NULL && blk->dev != NULL && blk->dev != (void *)-1;
+}
+
+/*
  * Set @blk's device model callbacks to @ops.
  * @opaque is the opaque argument to pass to the callbacks.
  * This is for use by device models.
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 7619a9f..a8c6fd2 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -82,6 +82,7 @@ void blk_detach_dev(BlockBackend *blk, void *dev);
 void *blk_get_attached_dev(BlockBackend *blk);
 int blk_disable_attach_dev(BlockBackend *blk);
 void blk_enable_attach_dev(BlockBackend *blk);
+bool blk_is_attached(BlockBackend *blk);
 void blk_set_dev_ops(BlockBackend *blk, const BlockDevOps *ops, void *opaque);
 int blk_read(BlockBackend *blk, int64_t sector_num, uint8_t *buf,
              int nb_sectors);
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 08/17] block: make bdrv_put_ref_bh_schedule() as a public API

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
---
 block.c               | 25 +++++++++++++++++++++++++
 blockdev.c            | 37 ++++++-------------------------------
 include/block/block.h |  1 +
 3 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/block.c b/block.c
index dbbba5d..7619981 100644
--- a/block.c
+++ b/block.c
@@ -3574,6 +3574,31 @@ void bdrv_unref(BlockDriverState *bs)
     }
 }
 
+typedef struct {
+    QEMUBH *bh;
+    BlockDriverState *bs;
+} BDRVPutRefBH;
+
+static void bdrv_put_ref_bh(void *opaque)
+{
+    BDRVPutRefBH *s = opaque;
+
+    bdrv_unref(s->bs);
+    qemu_bh_delete(s->bh);
+    g_free(s);
+}
+
+/* Release a BDS reference in a BH */
+void bdrv_put_ref_bh_schedule(BlockDriverState *bs)
+{
+    BDRVPutRefBH *s;
+
+    s = g_new(BDRVPutRefBH, 1);
+    s->bh = qemu_bh_new(bdrv_put_ref_bh, s);
+    s->bs = bs;
+    qemu_bh_schedule(s->bh);
+}
+
 struct BdrvOpBlocker {
     Error *reason;
     QLIST_ENTRY(BdrvOpBlocker) list;
diff --git a/blockdev.c b/blockdev.c
index 935c081..dd1022c 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -276,37 +276,6 @@ static void bdrv_format_print(void *opaque, const char *name)
     error_printf(" %s", name);
 }
 
-typedef struct {
-    QEMUBH *bh;
-    BlockDriverState *bs;
-} BDRVPutRefBH;
-
-static void bdrv_put_ref_bh(void *opaque)
-{
-    BDRVPutRefBH *s = opaque;
-
-    bdrv_unref(s->bs);
-    qemu_bh_delete(s->bh);
-    g_free(s);
-}
-
-/*
- * Release a BDS reference in a BH
- *
- * It is not safe to use bdrv_unref() from a callback function when the callers
- * still need the BlockDriverState.  In such cases we schedule a BH to release
- * the reference.
- */
-static void bdrv_put_ref_bh_schedule(BlockDriverState *bs)
-{
-    BDRVPutRefBH *s;
-
-    s = g_new(BDRVPutRefBH, 1);
-    s->bh = qemu_bh_new(bdrv_put_ref_bh, s);
-    s->bs = bs;
-    qemu_bh_schedule(s->bh);
-}
-
 static int parse_block_error_action(const char *buf, bool is_read, Error **errp)
 {
     if (!strcmp(buf, "ignore")) {
@@ -2329,6 +2298,12 @@ static void block_job_cb(void *opaque, int ret)
         block_job_event_completed(bs->job, msg);
     }
 
+
+    /*
+     * It is not safe to use bdrv_unref() from a callback function when the
+     * callers still need the BlockDriverState. In such cases we schedule
+     * a BH to release the reference.
+     */
     bdrv_put_ref_bh_schedule(bs);
 }
 
diff --git a/include/block/block.h b/include/block/block.h
index c43160d..0dec16c 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -503,6 +503,7 @@ void bdrv_disable_copy_on_read(BlockDriverState *bs);
 
 void bdrv_ref(BlockDriverState *bs);
 void bdrv_unref(BlockDriverState *bs);
+void bdrv_put_ref_bh_schedule(BlockDriverState *bs);
 
 bool bdrv_op_is_blocked(BlockDriverState *bs, BlockOpType op, Error **errp);
 void bdrv_op_block(BlockDriverState *bs, BlockOpType op, Error *reason);
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 09/17] Backup: clear all bitmap when doing block checkpoint

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Cc: Jeff Cody <jcody@redhat.com>
---
 block/backup.c           | 13 +++++++++++++
 blockjob.c               | 10 ++++++++++
 include/block/blockjob.h | 12 ++++++++++++
 3 files changed, 35 insertions(+)

diff --git a/block/backup.c b/block/backup.c
index 4a1af68..b4fea46 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -211,11 +211,24 @@ static void backup_iostatus_reset(BlockJob *job)
     bdrv_iostatus_reset(s->target);
 }
 
+static void backup_do_checkpoint(BlockJob *job, Error **errp)
+{
+    BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
+
+    if (backup_job->sync_mode != MIRROR_SYNC_MODE_NONE) {
+        error_setg(errp, "this feature or command is not currently supported");
+        return;
+    }
+
+    hbitmap_reset_all(backup_job->bitmap);
+}
+
 static const BlockJobDriver backup_job_driver = {
     .instance_size  = sizeof(BackupBlockJob),
     .job_type       = BLOCK_JOB_TYPE_BACKUP,
     .set_speed      = backup_set_speed,
     .iostatus_reset = backup_iostatus_reset,
+    .do_checkpoint  = backup_do_checkpoint,
 };
 
 static BlockErrorAction backup_error_action(BackupBlockJob *job,
diff --git a/blockjob.c b/blockjob.c
index ec46fad..cb412d1 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -400,3 +400,13 @@ void block_job_defer_to_main_loop(BlockJob *job,
 
     qemu_bh_schedule(data->bh);
 }
+
+void block_job_do_checkpoint(BlockJob *job, Error **errp)
+{
+    if (!job->driver->do_checkpoint) {
+        error_setg(errp, "this feature or command is not currently supported");
+        return;
+    }
+
+    job->driver->do_checkpoint(job, errp);
+}
diff --git a/include/block/blockjob.h b/include/block/blockjob.h
index 57d8ef1..b832dc3 100644
--- a/include/block/blockjob.h
+++ b/include/block/blockjob.h
@@ -50,6 +50,9 @@ typedef struct BlockJobDriver {
      * manually.
      */
     void (*complete)(BlockJob *job, Error **errp);
+
+    /** Optional callback for job types that support checkpoint. */
+    void (*do_checkpoint)(BlockJob *job, Error **errp);
 } BlockJobDriver;
 
 /**
@@ -348,4 +351,13 @@ void block_job_defer_to_main_loop(BlockJob *job,
                                   BlockJobDeferToMainLoopFn *fn,
                                   void *opaque);
 
+/**
+ * block_job_do_checkpoint:
+ * @job: The job.
+ * @errp: Error object.
+ *
+ * Do block checkpoint on the specified job.
+ */
+void block_job_do_checkpoint(BlockJob *job, Error **errp);
+
 #endif
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 10/17] allow writing to the backing file

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 block.c | 41 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)

diff --git a/block.c b/block.c
index 7619981..12efd21 100644
--- a/block.c
+++ b/block.c
@@ -747,6 +747,15 @@ static const BdrvChildRole child_backing = {
     .inherit_flags = bdrv_backing_flags,
 };
 
+static int bdrv_backing_rw_flags(int flags)
+{
+    return bdrv_backing_flags(flags) | BDRV_O_RDWR;
+}
+
+static const BdrvChildRole child_backing_rw = {
+    .inherit_flags = bdrv_backing_rw_flags,
+};
+
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
 {
     int open_flags = flags | BDRV_O_CACHE_WB;
@@ -1133,6 +1142,20 @@ out:
     bdrv_refresh_limits(bs, NULL);
 }
 
+#define ALLOW_WRITE_BACKING_FILE    "allow-write-backing-file"
+static QemuOptsList backing_file_opts = {
+    .name = "backing_file",
+    .head = QTAILQ_HEAD_INITIALIZER(backing_file_opts.head),
+    .desc = {
+        {
+            .name = ALLOW_WRITE_BACKING_FILE,
+            .type = QEMU_OPT_BOOL,
+            .help = "allow write to backing file",
+        },
+        { /* end of list */ }
+    },
+};
+
 /*
  * Opens the backing file for a BlockDriverState if not yet open
  *
@@ -1147,6 +1170,9 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
     int ret = 0;
     BlockDriverState *backing_hd;
     Error *local_err = NULL;
+    QemuOpts *opts = NULL;
+    bool child_rw = false;
+    const BdrvChildRole *child_role = NULL;
 
     if (bs->backing_hd != NULL) {
         QDECREF(options);
@@ -1159,6 +1185,18 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
     }
 
     bs->open_flags &= ~BDRV_O_NO_BACKING;
+
+    opts = qemu_opts_create(&backing_file_opts, NULL, 0, &error_abort);
+    qemu_opts_absorb_qdict(opts, options, &local_err);
+    if (local_err) {
+        ret = -EINVAL;
+        error_propagate(errp, local_err);
+        QDECREF(options);
+        goto free_exit;
+    }
+    child_rw = qemu_opt_get_bool(opts, ALLOW_WRITE_BACKING_FILE, false);
+    child_role = child_rw ? &child_backing_rw : &child_backing;
+
     if (qdict_haskey(options, "file.filename")) {
         backing_filename[0] = '\0';
     } else if (bs->backing_file[0] == '\0' && qdict_size(options) == 0) {
@@ -1191,7 +1229,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
     assert(bs->backing_hd == NULL);
     ret = bdrv_open_inherit(&backing_hd,
                             *backing_filename ? backing_filename : NULL,
-                            NULL, options, 0, bs, &child_backing,
+                            NULL, options, 0, bs, child_role,
                             NULL, &local_err);
     if (ret < 0) {
         bdrv_unref(backing_hd);
@@ -1205,6 +1243,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
     bdrv_set_backing_hd(bs, backing_hd);
 
 free_exit:
+    qemu_opts_del(opts);
     g_free(backing_filename);
     return ret;
 }
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 11/17] Allow creating backup jobs when opening BDS

[Wen Congyang]

When opening BDS, we need to create backup jobs for
image-fleecing.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Cc: Jeff Cody <jcody@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/Makefile.objs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/Makefile.objs b/block/Makefile.objs
index c34fd7c..f068666 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -22,10 +22,10 @@ block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o
 block-obj-y += write-threshold.o
+block-obj-y += backup.o
 
 common-obj-y += stream.o
 common-obj-y += commit.o
-common-obj-y += backup.o
 
 iscsi.o-cflags     := $(LIBISCSI_CFLAGS)
 iscsi.o-libs       := $(LIBISCSI_LIBS)
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 12/17] block: Allow references for backing files

[Wen Congyang]

Usage:
-drive file=xxx,id=Y, \
-drive file=xxxx,id=X,backing.backing_reference=Y

It will create such backing chain:
               {virtio-blk dev 'Y'}      {virtio-blk dev 'X'}
                         |                         |
                         |                         |
                         v                         v

    [base] <- [mid] <- ( Y )  <----------------- ( X )

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 block.c               | 39 +++++++++++++++++++++++++++++++++++----
 include/block/block.h |  1 +
 2 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/block.c b/block.c
index 12efd21..587990a 100644
--- a/block.c
+++ b/block.c
@@ -1143,6 +1143,7 @@ out:
 }
 
 #define ALLOW_WRITE_BACKING_FILE    "allow-write-backing-file"
+#define BACKING_REFERENCE           "backing_reference"
 static QemuOptsList backing_file_opts = {
     .name = "backing_file",
     .head = QTAILQ_HEAD_INITIALIZER(backing_file_opts.head),
@@ -1152,6 +1153,11 @@ static QemuOptsList backing_file_opts = {
             .type = QEMU_OPT_BOOL,
             .help = "allow write to backing file",
         },
+        {
+            .name = BACKING_REFERENCE,
+            .type = QEMU_OPT_STRING,
+            .help = "reference to the exsiting BDS",
+        },
         { /* end of list */ }
     },
 };
@@ -1168,11 +1174,12 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
 {
     char *backing_filename = g_malloc0(PATH_MAX);
     int ret = 0;
-    BlockDriverState *backing_hd;
+    BlockDriverState *backing_hd = NULL;
     Error *local_err = NULL;
     QemuOpts *opts = NULL;
     bool child_rw = false;
     const BdrvChildRole *child_role = NULL;
+    const char *reference = NULL;
 
     if (bs->backing_hd != NULL) {
         QDECREF(options);
@@ -1195,9 +1202,10 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
         goto free_exit;
     }
     child_rw = qemu_opt_get_bool(opts, ALLOW_WRITE_BACKING_FILE, false);
+    reference = qemu_opt_get(opts, BACKING_REFERENCE);
     child_role = child_rw ? &child_backing_rw : &child_backing;
 
-    if (qdict_haskey(options, "file.filename")) {
+    if (qdict_haskey(options, "file.filename") || reference) {
         backing_filename[0] = '\0';
     } else if (bs->backing_file[0] == '\0' && qdict_size(options) == 0) {
         QDECREF(options);
@@ -1220,7 +1228,9 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
         goto free_exit;
     }
 
-    backing_hd = bdrv_new();
+    if (!reference) {
+        backing_hd = bdrv_new();
+    }
 
     if (bs->backing_format[0] != '\0' && !qdict_haskey(options, "driver")) {
         qdict_put(options, "driver", qstring_from_str(bs->backing_format));
@@ -1229,7 +1239,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
     assert(bs->backing_hd == NULL);
     ret = bdrv_open_inherit(&backing_hd,
                             *backing_filename ? backing_filename : NULL,
-                            NULL, options, 0, bs, child_role,
+                            reference, options, 0, bs, child_role,
                             NULL, &local_err);
     if (ret < 0) {
         bdrv_unref(backing_hd);
@@ -1240,12 +1250,30 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
         error_free(local_err);
         goto free_exit;
     }
+    if (reference) {
+        if (bdrv_op_is_blocked(backing_hd, BLOCK_OP_TYPE_BACKING_REFERENCE,
+                               errp)) {
+            ret = -EBUSY;
+            goto free_reference_exit;
+        }
+        if (backing_hd->blk && blk_disable_attach_dev(backing_hd->blk)) {
+            error_setg(errp, "backing_hd %s is used by the other device model",
+                       reference);
+            ret = -EBUSY;
+            goto free_reference_exit;
+        }
+    }
     bdrv_set_backing_hd(bs, backing_hd);
 
 free_exit:
     qemu_opts_del(opts);
     g_free(backing_filename);
     return ret;
+
+free_reference_exit:
+    bdrv_unref(backing_hd);
+    bs->open_flags |= BDRV_O_NO_BACKING;
+    goto free_exit;
 }
 
 /*
@@ -1899,6 +1927,9 @@ void bdrv_close(BlockDriverState *bs)
         if (bs->backing_hd) {
             BlockDriverState *backing_hd = bs->backing_hd;
             bdrv_set_backing_hd(bs, NULL);
+            if (backing_hd->blk) {
+                blk_enable_attach_dev(backing_hd->blk);
+            }
             bdrv_unref(backing_hd);
         }
         bs->drv->bdrv_close(bs);
diff --git a/include/block/block.h b/include/block/block.h
index 0dec16c..65c9974 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -168,6 +168,7 @@ typedef enum BlockOpType {
     BLOCK_OP_TYPE_RESIZE,
     BLOCK_OP_TYPE_STREAM,
     BLOCK_OP_TYPE_REPLACE,
+    BLOCK_OP_TYPE_BACKING_REFERENCE,
     BLOCK_OP_TYPE_MAX,
 } BlockOpType;
 
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 13/17] docs: block replication's description

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 docs/block-replication.txt | 179 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 179 insertions(+)
 create mode 100644 docs/block-replication.txt

diff --git a/docs/block-replication.txt b/docs/block-replication.txt
new file mode 100644
index 0000000..1f80a80
--- /dev/null
+++ b/docs/block-replication.txt
@@ -0,0 +1,179 @@
+Block replication
+----------------------------------------
+Copyright Fujitsu, Corp. 2015
+Copyright (c) 2015 Intel Corporation
+Copyright (c) 2015 HUAWEI TECHNOLOGIES CO., LTD.
+
+This work is licensed under the terms of the GNU GPL, version 2 or later.
+See the COPYING file in the top-level directory.
+
+Block replication is used for continuous checkpoints. It is designed
+for COLO (COurse-grain LOck-stepping) where the Secondary VM is running.
+It can also be applied for FT/HA (Fault-tolerance/High Assurance) scenario,
+where the Secondary VM is not running.
+
+This document gives an overview of block replication's design.
+
+== Background ==
+High availability solutions such as micro checkpoint and COLO will do
+consecutive checkpoints. The VM state of Primary VM and Secondary VM is
+identical right after a VM checkpoint, but becomes different as the VM
+executes till the next checkpoint. To support disk contents checkpoint,
+the modified disk contents in the Secondary VM must be buffered, and are
+only dropped at next checkpoint time. To reduce the network transportation
+effort at the time of checkpoint, the disk modification operations of
+Primary disk are asynchronously forwarded to the Secondary node.
+
+== Workflow ==
+The following is the image of block replication workflow:
+
+        +----------------------+            +------------------------+
+        |Primary Write Requests|            |Secondary Write Requests|
+        +----------------------+            +------------------------+
+                  |                                       |
+                  |                                      (4)
+                  |                                       V
+                  |                              /-------------\
+                  |      Copy and Forward        |             |
+                  |---------(1)----------+       | Disk Buffer |
+                  |                      |       |             |
+                  |                     (3)      \-------------/
+                  |                 speculative      ^
+                  |                write through    (2)
+                  |                      |           |
+                  V                      V           |
+           +--------------+           +----------------+
+           | Primary Disk |           | Secondary Disk |
+           +--------------+           +----------------+
+
+    1) Primary write requests will be copied and forwarded to Secondary
+       QEMU.
+    2) Before Primary write requests are written to Secondary disk, the
+       original sector content will be read from Secondary disk and
+       buffered in the Disk buffer, but it will not overwrite the existing
+       sector content(it could be from either "Secondary Write Requests" or
+       previous COW of "Primary Write Requests") in the Disk buffer.
+    3) Primary write requests will be written to Secondary disk.
+    4) Secondary write requests will be buffered in the Disk buffer and it
+       will overwrite the existing sector content in the buffer.
+
+== Architecture ==
+We are going to implement block replication from many basic
+blocks that are already in QEMU.
+
+         virtio-blk       ||
+             ^            ||                            .----------
+             |            ||                            | Secondary
+        1 Quorum          ||                            '----------
+         /      \         ||
+        /        \        ||
+   Primary    2 filter
+     disk         ^                                                             virtio-blk
+                  |                                                                  ^
+                3 NBD  ------->  3 NBD                                               |
+                client    ||     server                                          2 filter
+                          ||        ^                                                ^
+--------.                 ||        |                                                |
+Primary |                 ||  Secondary disk <--------- hidden-disk 5 <--------- active-disk 4
+--------'                 ||        |          backing        ^       backing
+                          ||        |                         |
+                          ||        |                         |
+                          ||        '-------------------------'
+                          ||           drive-backup sync=none
+
+1) The disk on the primary is represented by a block device with two
+children, providing replication between a primary disk and the host that
+runs the secondary VM. The read pattern for quorum can be extended to
+make the primary always read from the local disk instead of going through
+NBD.
+
+2) The new block filter(the name is replication) will control the block
+replication.
+
+3) The secondary disk receives writes from the primary VM through QEMU's
+embedded NBD server (speculative write-through).
+
+4) The disk on the secondary is represented by a custom block device
+(called active-disk). It should be an empty disk, and the format should
+support bdrv_make_empty() and backing file.
+
+5) The hidden-disk is created automatically. It buffers the original content
+that is modified by the primary VM. It should also be an empty disk, and
+the driver supports bdrv_make_empty() and backing file.
+
+== Failure Handling ==
+There are 6 internal errors when block replication is running:
+1. I/O error on primary disk
+2. Forwarding primary write requests failed
+3. Backup failed
+4. I/O error on secondary disk
+5. I/O error on active disk
+6. Making active disk or hidden disk empty failed
+In case 1 and 5, we just report the error to the disk layer. In case 2, 3,
+4 and 6, we just report block replication's error to FT/HA manager(which
+decides when to do a new checkpoint, when to do failover).
+There is one internal error when doing failover:
+1. Commiting the data in active disk/hidden disk to secondary disk failed
+We just to report this error to FT/HA manager.
+
+== New block driver interface ==
+We add three block driver interfaces to control block replication:
+a. bdrv_start_replication()
+   Start block replication, called in migration/checkpoint thread.
+   We must call bdrv_start_replication() in secondary QEMU before
+   calling bdrv_start_replication() in primary QEMU. The caller
+   must hold the I/O mutex lock if it is in migration/checkpoint
+   thread.
+b. bdrv_do_checkpoint()
+   This interface is called after all VM state is transferred to
+   Secondary QEMU. The Disk buffer will be dropped in this interface.
+   The caller must hold the I/O mutex lock if it is in migration/checkpoint
+   thread.
+c. bdrv_stop_replication()
+   It is called on failover. We will flush the Disk buffer into
+   Secondary Disk and stop block replication. The vm should be stopped
+   before calling it. The caller must hold the I/O mutex lock if it is
+   in migration/checkpoint thread.
+
+== Usage ==
+Primary:
+  -drive if=xxx,driver=quorum,read-pattern=fifo,no-connect=on,\
+         children.0.file.filename=1.raw,\
+         children.0.driver=raw,\
+         children.1.file.driver=nbd,\
+         children.1.file.host=xxx,\
+         children.1.file.port=xxx,\
+         children.1.file.export=xxx,\
+         children.1.driver=replication,\
+         children.1.mode=primary,\
+         children.1.ignore-errors=on
+  Note:
+  1. NBD Client should not be the first child of quorum.
+  2. There should be only one NBD Client.
+  3. host is the secondary physical machine's hostname or IP
+  4. Each disk must have its own export name.
+  5. It is all a single argument to -drive, and you should ignore
+     the leading whitespace.
+
+Secondary:
+  -drive if=none,driver=raw,file=1.raw,id=nbd_target1 \
+  -drive if=xxx,driver=replication,mode=secondary,export=xxx,\
+         file.file.filename=active_disk.qcow2,\
+         file.driver=qcow2,\
+         file.backing_reference.drive_id=nbd_target1,\
+         file.backing_reference.hidden-disk.file.filename=hidden_disk.qcow2,\
+         file.backing_reference.hidden-disk.driver=qcow2,\
+         file.backing_reference.hidden-disk.allow-write-backing-file=on
+  Then run qmp command:
+    nbd-server-start host:port
+  Note:
+  1. The export name for the same disk must be the same in primary
+     and secondary QEMU command line
+  2. The qmp command nbd-server-start must be run before running the
+     qmp command migrate on primary QEMU
+  3. Don't use nbd-server-start's other options
+  4. Active disk, hidden disk and nbd target's length should be the
+     same.
+  5. It is better to put active disk and hidden disk in ramdisk.
+  6. It is all a single argument to -drive, and you should ignore
+     the leading whitespace.
-- 
2.4.3

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 13/17] docs: block replication's description

[Michael R. Hines]

On 06/29/2015 10:34 PM, Wen Congyang wrote:
> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
> Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
> Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> ---
>   docs/block-replication.txt | 179 +++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 179 insertions(+)
>   create mode 100644 docs/block-replication.txt
>
> diff --git a/docs/block-replication.txt b/docs/block-replication.txt
> new file mode 100644
> index 0000000..1f80a80
> --- /dev/null
> +++ b/docs/block-replication.txt
> @@ -0,0 +1,179 @@
> +Block replication
> +----------------------------------------
> +Copyright Fujitsu, Corp. 2015
> +Copyright (c) 2015 Intel Corporation
> +Copyright (c) 2015 HUAWEI TECHNOLOGIES CO., LTD.
> +
> +This work is licensed under the terms of the GNU GPL, version 2 or later.
> +See the COPYING file in the top-level directory.
> +
> +Block replication is used for continuous checkpoints. It is designed
> +for COLO (COurse-grain LOck-stepping) where the Secondary VM is running.
> +It can also be applied for FT/HA (Fault-tolerance/High Assurance) scenario,
> +where the Secondary VM is not running.
> +
> +This document gives an overview of block replication's design.
> +
> +== Background ==
> +High availability solutions such as micro checkpoint and COLO will do
> +consecutive checkpoints. The VM state of Primary VM and Secondary VM is
> +identical right after a VM checkpoint, but becomes different as the VM
> +executes till the next checkpoint. To support disk contents checkpoint,
> +the modified disk contents in the Secondary VM must be buffered, and are
> +only dropped at next checkpoint time. To reduce the network transportation
> +effort at the time of checkpoint, the disk modification operations of
> +Primary disk are asynchronously forwarded to the Secondary node.
> +
> +== Workflow ==
> +The following is the image of block replication workflow:
> +
> +        +----------------------+            +------------------------+
> +        |Primary Write Requests|            |Secondary Write Requests|
> +        +----------------------+            +------------------------+
> +                  |                                       |
> +                  |                                      (4)
> +                  |                                       V
> +                  |                              /-------------\
> +                  |      Copy and Forward        |             |
> +                  |---------(1)----------+       | Disk Buffer |
> +                  |                      |       |             |
> +                  |                     (3)      \-------------/
> +                  |                 speculative      ^
> +                  |                write through    (2)
> +                  |                      |           |
> +                  V                      V           |
> +           +--------------+           +----------------+
> +           | Primary Disk |           | Secondary Disk |
> +           +--------------+           +----------------+
> +
> +    1) Primary write requests will be copied and forwarded to Secondary
> +       QEMU.
> +    2) Before Primary write requests are written to Secondary disk, the
> +       original sector content will be read from Secondary disk and
> +       buffered in the Disk buffer, but it will not overwrite the existing
> +       sector content(it could be from either "Secondary Write Requests" or
> +       previous COW of "Primary Write Requests") in the Disk buffer.
> +    3) Primary write requests will be written to Secondary disk.
> +    4) Secondary write requests will be buffered in the Disk buffer and it
> +       will overwrite the existing sector content in the buffer.
> +
> +== Architecture ==
> +We are going to implement block replication from many basic
> +blocks that are already in QEMU.
> +
> +         virtio-blk       ||
> +             ^            ||                            .----------
> +             |            ||                            | Secondary
> +        1 Quorum          ||                            '----------
> +         /      \         ||
> +        /        \        ||
> +   Primary    2 filter
> +     disk         ^                                                             virtio-blk
> +                  |                                                                  ^
> +                3 NBD  ------->  3 NBD                                               |
> +                client    ||     server                                          2 filter
> +                          ||        ^                                                ^
> +--------.                 ||        |                                                |
> +Primary |                 ||  Secondary disk <--------- hidden-disk 5 <--------- active-disk 4
> +--------'                 ||        |          backing        ^       backing
> +                          ||        |                         |
> +                          ||        |                         |
> +                          ||        '-------------------------'
> +                          ||           drive-backup sync=none
> +
> +1) The disk on the primary is represented by a block device with two
> +children, providing replication between a primary disk and the host that
> +runs the secondary VM. The read pattern for quorum can be extended to
> +make the primary always read from the local disk instead of going through
> +NBD.
> +
> +2) The new block filter(the name is replication) will control the block
> +replication.
> +
> +3) The secondary disk receives writes from the primary VM through QEMU's
> +embedded NBD server (speculative write-through).
> +
> +4) The disk on the secondary is represented by a custom block device
> +(called active-disk). It should be an empty disk, and the format should
> +support bdrv_make_empty() and backing file.
> +
> +5) The hidden-disk is created automatically. It buffers the original content
> +that is modified by the primary VM. It should also be an empty disk, and
> +the driver supports bdrv_make_empty() and backing file.
> +
> +== Failure Handling ==
> +There are 6 internal errors when block replication is running:
> +1. I/O error on primary disk
> +2. Forwarding primary write requests failed
> +3. Backup failed
> +4. I/O error on secondary disk
> +5. I/O error on active disk
> +6. Making active disk or hidden disk empty failed
> +In case 1 and 5, we just report the error to the disk layer. In case 2, 3,
> +4 and 6, we just report block replication's error to FT/HA manager(which
> +decides when to do a new checkpoint, when to do failover).
> +There is one internal error when doing failover:
> +1. Commiting the data in active disk/hidden disk to secondary disk failed
> +We just to report this error to FT/HA manager.
> +
> +== New block driver interface ==
> +We add three block driver interfaces to control block replication:
> +a. bdrv_start_replication()
> +   Start block replication, called in migration/checkpoint thread.
> +   We must call bdrv_start_replication() in secondary QEMU before
> +   calling bdrv_start_replication() in primary QEMU. The caller
> +   must hold the I/O mutex lock if it is in migration/checkpoint
> +   thread.
> +b. bdrv_do_checkpoint()
> +   This interface is called after all VM state is transferred to
> +   Secondary QEMU. The Disk buffer will be dropped in this interface.
> +   The caller must hold the I/O mutex lock if it is in migration/checkpoint
> +   thread.
> +c. bdrv_stop_replication()
> +   It is called on failover. We will flush the Disk buffer into
> +   Secondary Disk and stop block replication. The vm should be stopped
> +   before calling it. The caller must hold the I/O mutex lock if it is
> +   in migration/checkpoint thread.
> +
> +== Usage ==
> +Primary:
> +  -drive if=xxx,driver=quorum,read-pattern=fifo,no-connect=on,\
> +         children.0.file.filename=1.raw,\
> +         children.0.driver=raw,\
> +         children.1.file.driver=nbd,\
> +         children.1.file.host=xxx,\
> +         children.1.file.port=xxx,\
> +         children.1.file.export=xxx,\
> +         children.1.driver=replication,\
> +         children.1.mode=primary,\
> +         children.1.ignore-errors=on
> +  Note:
> +  1. NBD Client should not be the first child of quorum.
> +  2. There should be only one NBD Client.
> +  3. host is the secondary physical machine's hostname or IP
> +  4. Each disk must have its own export name.
> +  5. It is all a single argument to -drive, and you should ignore
> +     the leading whitespace.
> +
> +Secondary:
> +  -drive if=none,driver=raw,file=1.raw,id=nbd_target1 \
> +  -drive if=xxx,driver=replication,mode=secondary,export=xxx,\
> +         file.file.filename=active_disk.qcow2,\
> +         file.driver=qcow2,\
> +         file.backing_reference.drive_id=nbd_target1,\
> +         file.backing_reference.hidden-disk.file.filename=hidden_disk.qcow2,\
> +         file.backing_reference.hidden-disk.driver=qcow2,\
> +         file.backing_reference.hidden-disk.allow-write-backing-file=on
> +  Then run qmp command:
> +    nbd-server-start host:port
> +  Note:
> +  1. The export name for the same disk must be the same in primary
> +     and secondary QEMU command line
> +  2. The qmp command nbd-server-start must be run before running the
> +     qmp command migrate on primary QEMU
> +  3. Don't use nbd-server-start's other options
> +  4. Active disk, hidden disk and nbd target's length should be the
> +     same.
> +  5. It is better to put active disk and hidden disk in ramdisk.
> +  6. It is all a single argument to -drive, and you should ignore
> +     the leading whitespace.

Can you make sure that the block-only patchset either has a separate 
wiki page if you are going to continue
to maintain it as a separate patchset?

Either your github documentation or your wiki documentation should match 
this patchset or your github patchset,
but they are all different.

- Michael

[Qemu-devel] [PATCH COLO-BLOCK v7 14/17] Add new block driver interfaces to control block replication

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 block.c                   | 40 ++++++++++++++++++++++++++++++++++++++++
 include/block/block.h     |  5 +++++
 include/block/block_int.h | 14 ++++++++++++++
 qapi/block.json           | 16 ++++++++++++++++
 4 files changed, 75 insertions(+)

diff --git a/block.c b/block.c
index 587990a..1cfbbf9 100644
--- a/block.c
+++ b/block.c
@@ -4341,3 +4341,43 @@ void bdrv_del_child(BlockDriverState *bs, BlockDriverState *child_bs,
 
     bs->drv->bdrv_del_child(bs, child_bs, errp);
 }
+
+void bdrv_start_replication(BlockDriverState *bs, ReplicationMode mode,
+                            Error **errp)
+{
+    BlockDriver *drv = bs->drv;
+
+    if (drv && drv->bdrv_start_replication) {
+        drv->bdrv_start_replication(bs, mode, errp);
+    } else if (bs->file) {
+        bdrv_start_replication(bs->file, mode, errp);
+    } else {
+        error_setg(errp, "this feature or command is not currently supported");
+    }
+}
+
+void bdrv_do_checkpoint(BlockDriverState *bs, Error **errp)
+{
+    BlockDriver *drv = bs->drv;
+
+    if (drv && drv->bdrv_do_checkpoint) {
+        drv->bdrv_do_checkpoint(bs, errp);
+    } else if (bs->file) {
+        bdrv_do_checkpoint(bs->file, errp);
+    } else {
+        error_setg(errp, "this feature or command is not currently supported");
+    }
+}
+
+void bdrv_stop_replication(BlockDriverState *bs, bool failover, Error **errp)
+{
+    BlockDriver *drv = bs->drv;
+
+    if (drv && drv->bdrv_stop_replication) {
+        drv->bdrv_stop_replication(bs, failover, errp);
+    } else if (bs->file) {
+        bdrv_stop_replication(bs->file, failover, errp);
+    } else {
+        error_setg(errp, "this feature or command is not currently supported");
+    }
+}
diff --git a/include/block/block.h b/include/block/block.h
index 65c9974..7657dee 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -611,4 +611,9 @@ void bdrv_add_child(BlockDriverState *bs, QDict *options, Error **errp);
 void bdrv_del_child(BlockDriverState *bs, BlockDriverState *child,
                     Error **errp);
 
+void bdrv_start_replication(BlockDriverState *bs, ReplicationMode mode,
+                            Error **errp);
+void bdrv_do_checkpoint(BlockDriverState *bs, Error **errp);
+void bdrv_stop_replication(BlockDriverState *bs, bool failover, Error **errp);
+
 #endif
diff --git a/include/block/block_int.h b/include/block/block_int.h
index d7af9e3..dac492a 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -293,6 +293,20 @@ struct BlockDriver {
     void (*bdrv_del_child)(BlockDriverState *bs, BlockDriverState *child,
                            Error **errp);
 
+    void (*bdrv_start_replication)(BlockDriverState *bs, ReplicationMode mode,
+                                   Error **errp);
+    /* Drop Disk buffer when doing checkpoint. */
+    void (*bdrv_do_checkpoint)(BlockDriverState *bs, Error **errp);
+    /*
+     * After failover, we should flush Disk buffer into secondary disk
+     * and stop block replication.
+     *
+     * If the guest is shutdown, we should drop Disk buffer and stop
+     * block representation.
+     */
+    void (*bdrv_stop_replication)(BlockDriverState *bs, bool failover,
+                                  Error **errp);
+
     QLIST_ENTRY(BlockDriver) list;
 };
 
diff --git a/qapi/block.json b/qapi/block.json
index aad645c..04dc4c2 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -40,6 +40,22 @@
   'data': ['auto', 'none', 'lba', 'large', 'rechs']}
 
 ##
+# @ReplicationMode
+#
+# An enumeration of replication modes.
+#
+# @unprotected: Replication is not started or after failover.
+#
+# @primary: Primary mode, the vm's state will be sent to secondary QEMU.
+#
+# @secondary: Secondary mode, receive the vm's state from primary QEMU.
+#
+# Since: 2.4
+##
+{ 'enum' : 'ReplicationMode',
+  'data' : ['unprotected', 'primary', 'secondary']}
+
+##
 # @BlockdevSnapshotInternal
 #
 # @device: the name of the device to generate the snapshot from
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 15/17] skip nbd_target when starting block replication

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 block.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/block.c b/block.c
index 1cfbbf9..7371c5a 100644
--- a/block.c
+++ b/block.c
@@ -4347,6 +4347,10 @@ void bdrv_start_replication(BlockDriverState *bs, ReplicationMode mode,
 {
     BlockDriver *drv = bs->drv;
 
+    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKING_REFERENCE, NULL)) {
+        return;
+    }
+
     if (drv && drv->bdrv_start_replication) {
         drv->bdrv_start_replication(bs, mode, errp);
     } else if (bs->file) {
@@ -4360,6 +4364,10 @@ void bdrv_do_checkpoint(BlockDriverState *bs, Error **errp)
 {
     BlockDriver *drv = bs->drv;
 
+    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKING_REFERENCE, NULL)) {
+        return;
+    }
+
     if (drv && drv->bdrv_do_checkpoint) {
         drv->bdrv_do_checkpoint(bs, errp);
     } else if (bs->file) {
@@ -4373,6 +4381,10 @@ void bdrv_stop_replication(BlockDriverState *bs, bool failover, Error **errp)
 {
     BlockDriver *drv = bs->drv;
 
+    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKING_REFERENCE, NULL)) {
+        return;
+    }
+
     if (drv && drv->bdrv_stop_replication) {
         drv->bdrv_stop_replication(bs, failover, errp);
     } else if (bs->file) {
-- 
2.4.3

[Qemu-devel] [PATCH COLO-BLOCK v7 16/17] quorum: implement block driver interfaces for block replication

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Cc: Alberto Garcia <berto@igalia.com>
---
 block/quorum.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)

diff --git a/block/quorum.c b/block/quorum.c
index 6774d4c..bafa774 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -88,6 +88,8 @@ typedef struct BDRVQuorumState {
                             */
 
     QuorumReadPattern read_pattern;
+
+    int replication_index; /* store which child supports block replication */
 } BDRVQuorumState;
 
 typedef struct QuorumAIOCB QuorumAIOCB;
@@ -1019,6 +1021,7 @@ static int quorum_open(BlockDriverState *bs, QDict *options, int flags,
     }
 
     g_free(opened);
+    s->replication_index = -1;
     goto exit;
 
 close_exit:
@@ -1178,6 +1181,77 @@ static void quorum_refresh_filename(BlockDriverState *bs)
     bs->full_open_options = opts;
 }
 
+static void quorum_start_replication(BlockDriverState *bs, ReplicationMode mode,
+                                     Error **errp)
+{
+    BDRVQuorumState *s = bs->opaque;
+    int count = 0, i, index;
+    Error *local_err = NULL;
+
+    /*
+     * TODO: support REPLICATION_MODE_SECONDARY if we allow secondary
+     * QEMU becoming primary QEMU.
+     */
+    if (mode != REPLICATION_MODE_PRIMARY) {
+        error_setg(errp, "Invalid parameter 'mode'");
+        return;
+    }
+
+    if (s->read_pattern != QUORUM_READ_PATTERN_FIFO) {
+        error_setg(errp, "Invalid parameter 'read pattern'");
+        return;
+    }
+
+    for (i = 0; i < s->num_children; i++) {
+        bdrv_start_replication(s->bs[i], mode, &local_err);
+        if (local_err) {
+            error_free(local_err);
+            local_err = NULL;
+        } else {
+            count++;
+            index = i;
+        }
+    }
+
+    if (count == 0) {
+        /* No child supports block replication */
+        error_setg(errp, "this feature or command is not currently supported");
+    } else if (count > 1) {
+        for (i = 0; i < s->num_children; i++) {
+            bdrv_stop_replication(s->bs[i], false, NULL);
+        }
+        error_setg(errp, "too many children support block replication");
+    } else {
+        s->replication_index = index;
+    }
+}
+
+static void quorum_do_checkpoint(BlockDriverState *bs, Error **errp)
+{
+    BDRVQuorumState *s = bs->opaque;
+
+    if (s->replication_index < 0) {
+        error_setg(errp, "Block replication is not started");
+        return;
+    }
+
+    bdrv_do_checkpoint(s->bs[s->replication_index], errp);
+}
+
+static void quorum_stop_replication(BlockDriverState *bs, bool failover,
+                                    Error **errp)
+{
+    BDRVQuorumState *s = bs->opaque;
+
+    if (s->replication_index < 0) {
+        error_setg(errp, "Block replication is not started");
+        return;
+    }
+
+    bdrv_stop_replication(s->bs[s->replication_index], failover, errp);
+    s->replication_index = -1;
+}
+
 static BlockDriver bdrv_quorum = {
     .format_name                        = "quorum",
     .protocol_name                      = "quorum",
@@ -1204,6 +1278,10 @@ static BlockDriver bdrv_quorum = {
 
     .is_filter                          = true,
     .bdrv_recurse_is_first_non_filter   = quorum_recurse_is_first_non_filter,
+
+    .bdrv_start_replication             = quorum_start_replication,
+    .bdrv_do_checkpoint                 = quorum_do_checkpoint,
+    .bdrv_stop_replication              = quorum_stop_replication,
 };
 
 static void bdrv_quorum_init(void)
-- 
2.4.3

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 16/17] quorum: implement block driver interfaces for block replication

[Alberto Garcia]

On Tue 30 Jun 2015 05:34:44 AM CEST, Wen Congyang wrote:

> +static void quorum_start_replication(BlockDriverState *bs, ReplicationMode mode,
> +                                     Error **errp)
> +{
> +    BDRVQuorumState *s = bs->opaque;
> +    int count = 0, i, index;
> +    Error *local_err = NULL;
> +
> +    /*
> +     * TODO: support REPLICATION_MODE_SECONDARY if we allow secondary
> +     * QEMU becoming primary QEMU.
> +     */
> +    if (mode != REPLICATION_MODE_PRIMARY) {
> +        error_setg(errp, "Invalid parameter 'mode'");
> +        return;
> +    }
> +
> +    if (s->read_pattern != QUORUM_READ_PATTERN_FIFO) {
> +        error_setg(errp, "Invalid parameter 'read pattern'");
> +        return;
> +    }

Those error messages seem a bit confusing. As I understand it, what's
wrong is the value of the parameter, not the parameter itself.

A more descriptive error message would be something along the lines of
"The only supported replication mode for this operation is 'primary'",
"The only supported read pattern for this operation is 'fifo'".

It doesn't need to be those exact words, but the idea is that the
message explains what's wrong.

> +    if (count == 0) {
> +        /* No child supports block replication */
> +        error_setg(errp, "this feature or command is not currently supported");
> +    } else if (count > 1) {
> +        for (i = 0; i < s->num_children; i++) {
> +            bdrv_stop_replication(s->bs[i], false, NULL);
> +        }
> +        error_setg(errp, "too many children support block replication");
> +    } else {
> +        s->replication_index = index;
> +    }
> +}

Not very important, but the previous error messages start with uppercase
and these are all lowercase.

> +        error_setg(errp, "Block replication is not started");

It sounds a bit odd to me, any native speaker can confirm?

Comments about the messages aside, the code looks good to me, hence

Reviewed-by: Alberto Garcia <berto@igalia.com>

Berto

[Qemu-devel] [PATCH COLO-BLOCK v7 17/17] Implement new driver for block replication

[Wen Congyang]

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
---
 block/Makefile.objs |   1 +
 block/replication.c | 443 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 444 insertions(+)
 create mode 100644 block/replication.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index f068666..84952b1 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -23,6 +23,7 @@ block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o
 block-obj-y += write-threshold.o
 block-obj-y += backup.o
+block-obj-y += replication.o
 
 common-obj-y += stream.o
 common-obj-y += commit.o
diff --git a/block/replication.c b/block/replication.c
new file mode 100644
index 0000000..2124e2d
--- /dev/null
+++ b/block/replication.c
@@ -0,0 +1,443 @@
+#include "qemu-common.h"
+#include "block/block_int.h"
+#include "block/blockjob.h"
+#include "block/nbd.h"
+
+typedef struct BDRVReplicationState {
+    ReplicationMode mode;
+    int replication_state;
+    BlockDriverState *active_disk;
+    BlockDriverState *hidden_disk;
+    BlockDriverState *secondary_disk; /* nbd target */
+    int error;
+} BDRVReplicationState;
+
+enum {
+    BLOCK_REPLICATION_NONE,     /* block replication is not started */
+    BLOCK_REPLICATION_RUNNING,  /* block replication is running */
+    BLOCK_REPLICATION_DONE,     /* block replication is done(failover) */
+};
+
+#define COMMIT_CLUSTER_BITS 16
+#define COMMIT_CLUSTER_SIZE (1 << COMMIT_CLUSTER_BITS)
+#define COMMIT_SECTORS_PER_CLUSTER (COMMIT_CLUSTER_SIZE / BDRV_SECTOR_SIZE)
+
+static void replication_stop(BlockDriverState *bs, bool failover, Error **errp);
+
+#define REPLICATION_MODE        "mode"
+static QemuOptsList replication_runtime_opts = {
+    .name = "replication",
+    .head = QTAILQ_HEAD_INITIALIZER(replication_runtime_opts.head),
+    .desc = {
+        {
+            .name = REPLICATION_MODE,
+            .type = QEMU_OPT_STRING,
+        },
+        { /* end of list */ }
+    },
+};
+
+static int replication_open(BlockDriverState *bs, QDict *options,
+                            int flags, Error **errp)
+{
+    int ret;
+    BDRVReplicationState *s = bs->opaque;;
+    Error *local_err = NULL;
+    QemuOpts *opts = NULL;
+    const char *mode;
+
+    ret = -EINVAL;
+    opts = qemu_opts_create(&replication_runtime_opts, NULL, 0, &error_abort);
+    qemu_opts_absorb_qdict(opts, options, &local_err);
+    if (local_err) {
+        goto fail;
+    }
+
+    mode = qemu_opt_get(opts, REPLICATION_MODE);
+    if (!mode) {
+        error_setg(&local_err, "Missing the option mode");
+        goto fail;
+    }
+
+    if (!strcmp(mode, "primary")) {
+        s->mode = REPLICATION_MODE_PRIMARY;
+    } else if (!strcmp(mode, "secondary")) {
+        s->mode = REPLICATION_MODE_SECONDARY;
+    } else {
+        error_setg(&local_err,
+                   "The option mode's value should be primary or secondary");
+        goto fail;
+    }
+
+    return 0;
+
+fail:
+    qemu_opts_del(opts);
+    /* propagate error */
+    if (local_err) {
+        error_propagate(errp, local_err);
+    }
+    return ret;
+}
+
+static void replication_close(BlockDriverState *bs)
+{
+    BDRVReplicationState *s = bs->opaque;
+
+    if (s->replication_state == BLOCK_REPLICATION_RUNNING) {
+        replication_stop(bs, false, NULL);
+    }
+}
+
+static int64_t replication_getlength(BlockDriverState *bs)
+{
+    return bdrv_getlength(bs->file);
+}
+
+static int replication_get_io_status(BDRVReplicationState *s)
+{
+    switch (s->replication_state) {
+    case BLOCK_REPLICATION_NONE:
+        return -EIO;
+    case BLOCK_REPLICATION_RUNNING:
+        return 0;
+    case BLOCK_REPLICATION_DONE:
+        return s->mode == REPLICATION_MODE_PRIMARY ? -EIO : 1;
+    default:
+        abort();
+    }
+}
+
+static int replication_return_value(BDRVReplicationState *s, int ret)
+{
+    if (s->mode == REPLICATION_MODE_SECONDARY) {
+        return ret;
+    }
+
+    if (ret < 0) {
+        s->error = ret;
+        ret = 0;
+    }
+
+    return ret;
+}
+
+static coroutine_fn int replication_co_readv(BlockDriverState *bs,
+                                             int64_t sector_num,
+                                             int remaining_sectors,
+                                             QEMUIOVector *qiov)
+{
+    BDRVReplicationState *s = bs->opaque;
+    int ret;
+
+    if (s->mode == REPLICATION_MODE_PRIMARY) {
+        /* We only use it to forward primary write requests */
+        return -EIO;
+    }
+
+    ret = replication_get_io_status(s);
+    if (ret < 0) {
+        return ret;
+    }
+
+    /*
+     * After failover, because we don't commit active disk/hidden disk
+     * to secondary disk(nbd target), so we should read from active disk
+     * directly.
+     */
+    ret = bdrv_co_readv(bs->file, sector_num, remaining_sectors, qiov);
+    return replication_return_value(s, ret);
+}
+
+static coroutine_fn int replication_co_writev(BlockDriverState *bs,
+                                              int64_t sector_num,
+                                              int remaining_sectors,
+                                              QEMUIOVector *qiov)
+{
+    BDRVReplicationState *s = bs->opaque;
+    QEMUIOVector hd_qiov;
+    uint64_t bytes_done = 0;
+    BlockDriverState *top = bs->file;
+    BlockDriverState *base = s->secondary_disk;
+    BlockDriverState *target;
+    int ret, n;
+
+    ret = replication_get_io_status(s);
+    if (ret < 0) {
+        return ret;
+    }
+
+    if (ret == 0) {
+        ret = bdrv_co_writev(bs->file, sector_num, remaining_sectors, qiov);
+        return replication_return_value(s, ret);
+    }
+
+    /*
+     * Only write to active disk if the sectors have
+     * already been allocated in active disk/hidden disk.
+     */
+    qemu_iovec_init(&hd_qiov, qiov->niov);
+    while (remaining_sectors > 0) {
+        ret = bdrv_is_allocated_above(top, base, sector_num,
+                                      remaining_sectors, &n);
+        if (ret < 0) {
+            return ret;
+        }
+
+        qemu_iovec_reset(&hd_qiov);
+        qemu_iovec_concat(&hd_qiov, qiov, bytes_done, n * 512);
+
+        target = ret ? top: base;
+        ret = bdrv_co_writev(target, sector_num, n, &hd_qiov);
+        if (ret < 0) {
+            return ret;
+        }
+
+        remaining_sectors -= n;
+        sector_num += n;
+        bytes_done += n * BDRV_SECTOR_SIZE;
+    }
+
+    return 0;
+}
+
+static coroutine_fn int replication_co_discard(BlockDriverState *bs,
+                                               int64_t sector_num,
+                                               int nb_sectors)
+{
+    BDRVReplicationState *s = bs->opaque;
+    int ret;
+
+    ret = replication_get_io_status(s);
+    if (ret < 0) {
+        return ret;
+    }
+
+    if (ret == 1) {
+        /* It is secondary qemu and we are after failover */
+        ret = bdrv_co_discard(s->secondary_disk, sector_num, nb_sectors);
+        if (ret) {
+            return ret;
+        }
+    }
+
+    ret = bdrv_co_discard(bs->file, sector_num, nb_sectors);
+    return replication_return_value(s, ret);
+}
+
+static bool replication_recurse_is_first_non_filter(BlockDriverState *bs,
+                                                    BlockDriverState *candidate)
+{
+    return bdrv_recurse_is_first_non_filter(bs->file, candidate);
+}
+
+static void secondary_do_checkpoint(BDRVReplicationState *s, Error **errp)
+{
+    Error *local_err = NULL;
+    int ret;
+
+    if (!s->secondary_disk->job) {
+        error_setg(errp, "Backup job is cancelled unexpectedly");
+        return;
+    }
+
+    block_job_do_checkpoint(s->secondary_disk->job, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    ret = s->active_disk->drv->bdrv_make_empty(s->active_disk);
+    if (ret < 0) {
+        error_setg(errp, "Cannot make active disk empty");
+        return;
+    }
+
+    ret = s->hidden_disk->drv->bdrv_make_empty(s->hidden_disk);
+    if (ret < 0) {
+        error_setg(errp, "Cannot make hidden disk empty");
+        return;
+    }
+}
+
+static void backup_job_completed(void *opaque, int ret)
+{
+    BDRVReplicationState *s = opaque;
+
+    if (s->replication_state != BLOCK_REPLICATION_DONE) {
+        /* The backup job is cancelled unexpectedly */
+        s->error = -EIO;
+    }
+
+    bdrv_put_ref_bh_schedule(s->secondary_disk);
+}
+
+static void replication_start(BlockDriverState *bs, ReplicationMode mode,
+                              Error **errp)
+{
+    BDRVReplicationState *s = bs->opaque;
+    int64_t active_length, hidden_length, nbd_length;
+    AioContext *aio_context;
+    Error *local_err = NULL;
+
+    if (s->replication_state != BLOCK_REPLICATION_NONE) {
+        error_setg(errp, "Block replication is running or done");
+        return;
+    }
+
+    if (s->mode != mode) {
+        error_setg(errp, "Invalid parameter 'mode'");
+        return;
+    }
+
+    switch (s->mode) {
+    case REPLICATION_MODE_PRIMARY:
+        break;
+    case REPLICATION_MODE_SECONDARY:
+        s->active_disk = bs->file;
+        if (!bs->file->backing_hd) {
+            error_setg(errp, "Active disk doesn't have backing file");
+            return;
+        }
+
+        s->hidden_disk = s->active_disk->backing_hd;
+        if (!s->hidden_disk->backing_hd) {
+            error_setg(errp, "Hidden disk doesn't have backing file");
+            return;
+        }
+
+        s->secondary_disk = s->hidden_disk->backing_hd;
+        if (!s->secondary_disk->blk) {
+            error_setg(errp, "The secondary disk doesn't have block backend");
+            return;
+        }
+
+        /* verify the length */
+        active_length = bdrv_getlength(s->active_disk);
+        hidden_length = bdrv_getlength(s->hidden_disk);
+        nbd_length = bdrv_getlength(s->secondary_disk);
+        if (active_length < 0 || hidden_length < 0 || nbd_length < 0 ||
+            active_length != hidden_length || hidden_length != nbd_length) {
+            error_setg(errp, "active disk, hidden disk, nbd target's length"
+                       " are not the same");
+            return;
+        }
+
+        if (!s->active_disk->drv->bdrv_make_empty ||
+            !s->hidden_disk->drv->bdrv_make_empty) {
+            error_setg(errp,
+                       "active disk or hidden disk doesn't support make_empty");
+            return;
+        }
+
+        /* start backup job now */
+        bdrv_op_unblock(s->hidden_disk, BLOCK_OP_TYPE_BACKUP_TARGET,
+                        s->active_disk->backing_blocker);
+        bdrv_op_unblock(s->secondary_disk, BLOCK_OP_TYPE_BACKUP_SOURCE,
+                        s->hidden_disk->backing_blocker);
+        bdrv_ref(s->hidden_disk);
+
+        aio_context = bdrv_get_aio_context(bs);
+        aio_context_acquire(aio_context);
+        bdrv_set_aio_context(s->secondary_disk, aio_context);
+        backup_start(s->secondary_disk, s->hidden_disk, 0,
+                     MIRROR_SYNC_MODE_NONE, NULL, BLOCKDEV_ON_ERROR_REPORT,
+                     BLOCKDEV_ON_ERROR_REPORT, backup_job_completed,
+                     s, &local_err);
+        aio_context_release(aio_context);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            bdrv_unref(s->hidden_disk);
+            return;
+        }
+        break;
+    default:
+        abort();
+    }
+
+    s->replication_state = BLOCK_REPLICATION_RUNNING;
+
+    if (s->mode == REPLICATION_MODE_SECONDARY) {
+        secondary_do_checkpoint(s, errp);
+    }
+
+    s->error = 0;
+}
+
+static void replication_do_checkpoint(BlockDriverState *bs, Error **errp)
+{
+    BDRVReplicationState *s = bs->opaque;
+
+    if (s->replication_state != BLOCK_REPLICATION_RUNNING) {
+        error_setg(errp, "Block replication is not running");
+        return;
+    }
+
+    if (s->error) {
+        error_setg(errp, "I/O error occurs");
+        return;
+    }
+
+    if (s->mode == REPLICATION_MODE_SECONDARY) {
+        secondary_do_checkpoint(s, errp);
+    }
+}
+
+static void replication_stop(BlockDriverState *bs, bool failover, Error **errp)
+{
+    BDRVReplicationState *s = bs->opaque;
+
+    if (s->replication_state != BLOCK_REPLICATION_RUNNING) {
+        error_setg(errp, "Block replication is not running");
+        return;
+    }
+
+    s->replication_state = BLOCK_REPLICATION_DONE;
+
+    switch (s->mode) {
+    case REPLICATION_MODE_PRIMARY:
+        break;
+    case REPLICATION_MODE_SECONDARY:
+        if (!failover) {
+            secondary_do_checkpoint(s, errp);
+            return;
+        }
+
+        if (s->secondary_disk->job) {
+            block_job_cancel(s->secondary_disk->job);
+        }
+        break;
+    default:
+        abort();
+    }
+}
+
+BlockDriver bdrv_replication = {
+    .format_name                = "replication",
+    .protocol_name              = "replication",
+    .instance_size              = sizeof(BDRVReplicationState),
+
+    .bdrv_open                  = replication_open,
+    .bdrv_close                 = replication_close,
+
+    .bdrv_getlength             = replication_getlength,
+    .bdrv_co_readv              = replication_co_readv,
+    .bdrv_co_writev             = replication_co_writev,
+    .bdrv_co_discard            = replication_co_discard,
+
+    .is_filter                  = true,
+    .bdrv_recurse_is_first_non_filter = replication_recurse_is_first_non_filter,
+
+    .bdrv_start_replication     = replication_start,
+    .bdrv_do_checkpoint         = replication_do_checkpoint,
+    .bdrv_stop_replication      = replication_stop,
+
+    .has_variable_length        = true,
+};
+
+static void bdrv_replication_init(void)
+{
+    bdrv_register(&bdrv_replication);
+}
+
+block_init(bdrv_replication_init);
-- 
2.4.3

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Michael R. Hines]

Is this up to date:

On 06/29/2015 10:34 PM, Wen Congyang wrote:
> Block replication is a very important feature which is used for
> continuous checkpoints(for example: COLO).
>
> Usage:
> Please refer to docs/block-replication.txt
>
> You can get the patch here:
> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>
> You can get ths patch with framework here:
> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>
> TODO:
> 1. Continuous block replication. It will be started after basic functions
>     are accepted.
>
> Changs Log:
> V7:
> 1. Implement adding/removing quorum child. Remove the option non-connect.
> 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
> V6:
> 1. Rebase to the newest qemu.
> V5:
> 1. Address the comments from Gong Lei
> 2. Speed the failover up. The secondary vm can take over very quickly even
>     if there are too many I/O requests.
> V4:
> 1. Introduce a new driver replication to avoid touch nbd and qcow2.
> V3:
> 1: use error_setg() instead of error_set()
> 2. Add a new block job API
> 3. Active disk, hidden disk and nbd target uses the same AioContext
> 4. Add a testcase to test new hbitmap API
> V2:
> 1. Redesign the secondary qemu(use image-fleecing)
> 2. Use Error objects to return error message
> 3. Address the comments from Max Reitz and Eric Blake
>
>
> Wen Congyang (17):
>    Add new block driver interface to add/delete a BDS's child
>    quorum: implement block driver interfaces add/delete a BDS's child
>    hmp: add monitor command to add/remove a child
>    introduce a new API qemu_opts_absorb_qdict_by_index()
>    quorum: allow ignoring child errors
>    introduce a new API to enable/disable attach device model
>    introduce a new API to check if blk is attached
>    block: make bdrv_put_ref_bh_schedule() as a public API
>    Backup: clear all bitmap when doing block checkpoint
>    allow writing to the backing file
>    Allow creating backup jobs when opening BDS
>    block: Allow references for backing files
>    docs: block replication's description
>    Add new block driver interfaces to control block replication
>    skip nbd_target when starting block replication
>    quorum: implement block driver interfaces for block replication
>    Implement new driver for block replication
>
>   block.c                        | 198 +++++++++++++++++-
>   block/Makefile.objs            |   3 +-
>   block/backup.c                 |  13 ++
>   block/block-backend.c          |  33 +++
>   block/quorum.c                 | 244 ++++++++++++++++++++++-
>   block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
>   blockdev.c                     |  90 ++++++---
>   blockjob.c                     |  10 +
>   docs/block-replication.txt     | 179 +++++++++++++++++
>   hmp-commands.hx                |  28 +++
>   include/block/block.h          |  11 +
>   include/block/block_int.h      |  19 ++
>   include/block/blockjob.h       |  12 ++
>   include/qemu/option.h          |   2 +
>   include/sysemu/block-backend.h |   3 +
>   include/sysemu/blockdev.h      |   2 +
>   qapi/block.json                |  16 ++
>   util/qemu-option.c             |  44 ++++
>   18 files changed, 1303 insertions(+), 47 deletions(-)
>   create mode 100644 block/replication.c
>   create mode 100644 docs/block-replication.txt
>

Is this update to date with the wiki the documentation?

https://github.com/wencongyang/qemu-colo/commits/block-replication-v7

I just want to test this patchset, rather than the whole colo patcheset.

- Michael

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/02/2015 10:47 PM, Michael R. Hines wrote:
> Is this up to date:
> 
> On 06/29/2015 10:34 PM, Wen Congyang wrote:
>> Block replication is a very important feature which is used for
>> continuous checkpoints(for example: COLO).
>>
>> Usage:
>> Please refer to docs/block-replication.txt
>>
>> You can get the patch here:
>> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>>
>> You can get ths patch with framework here:
>> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>>
>> TODO:
>> 1. Continuous block replication. It will be started after basic functions
>>     are accepted.
>>
>> Changs Log:
>> V7:
>> 1. Implement adding/removing quorum child. Remove the option non-connect.
>> 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
>> V6:
>> 1. Rebase to the newest qemu.
>> V5:
>> 1. Address the comments from Gong Lei
>> 2. Speed the failover up. The secondary vm can take over very quickly even
>>     if there are too many I/O requests.
>> V4:
>> 1. Introduce a new driver replication to avoid touch nbd and qcow2.
>> V3:
>> 1: use error_setg() instead of error_set()
>> 2. Add a new block job API
>> 3. Active disk, hidden disk and nbd target uses the same AioContext
>> 4. Add a testcase to test new hbitmap API
>> V2:
>> 1. Redesign the secondary qemu(use image-fleecing)
>> 2. Use Error objects to return error message
>> 3. Address the comments from Max Reitz and Eric Blake
>>
>>
>> Wen Congyang (17):
>>    Add new block driver interface to add/delete a BDS's child
>>    quorum: implement block driver interfaces add/delete a BDS's child
>>    hmp: add monitor command to add/remove a child
>>    introduce a new API qemu_opts_absorb_qdict_by_index()
>>    quorum: allow ignoring child errors
>>    introduce a new API to enable/disable attach device model
>>    introduce a new API to check if blk is attached
>>    block: make bdrv_put_ref_bh_schedule() as a public API
>>    Backup: clear all bitmap when doing block checkpoint
>>    allow writing to the backing file
>>    Allow creating backup jobs when opening BDS
>>    block: Allow references for backing files
>>    docs: block replication's description
>>    Add new block driver interfaces to control block replication
>>    skip nbd_target when starting block replication
>>    quorum: implement block driver interfaces for block replication
>>    Implement new driver for block replication
>>
>>   block.c                        | 198 +++++++++++++++++-
>>   block/Makefile.objs            |   3 +-
>>   block/backup.c                 |  13 ++
>>   block/block-backend.c          |  33 +++
>>   block/quorum.c                 | 244 ++++++++++++++++++++++-
>>   block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
>>   blockdev.c                     |  90 ++++++---
>>   blockjob.c                     |  10 +
>>   docs/block-replication.txt     | 179 +++++++++++++++++
>>   hmp-commands.hx                |  28 +++
>>   include/block/block.h          |  11 +
>>   include/block/block_int.h      |  19 ++
>>   include/block/blockjob.h       |  12 ++
>>   include/qemu/option.h          |   2 +
>>   include/sysemu/block-backend.h |   3 +
>>   include/sysemu/blockdev.h      |   2 +
>>   qapi/block.json                |  16 ++
>>   util/qemu-option.c             |  44 ++++
>>   18 files changed, 1303 insertions(+), 47 deletions(-)
>>   create mode 100644 block/replication.c
>>   create mode 100644 docs/block-replication.txt
>>
> 
> Is this update to date with the wiki the documentation?
> 
> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
> 
> I just want to test this patchset, rather than the whole colo patcheset.

I forgot to update the patch 13. So please setup the envrionment according to the wiki.

Thanks
Wen Congyang

> 
> - Michael
> 
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Michael R. Hines]

On 06/29/2015 10:34 PM, Wen Congyang wrote:
> Block replication is a very important feature which is used for
> continuous checkpoints(for example: COLO).
>
> Usage:
> Please refer to docs/block-replication.txt
>
> You can get the patch here:
> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>
> You can get ths patch with framework here:
> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>
> TODO:
> 1. Continuous block replication. It will be started after basic functions
>     are accepted.
>
> Changs Log:
> V7:
> 1. Implement adding/removing quorum child. Remove the option non-connect.
> 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
> V6:
> 1. Rebase to the newest qemu.
> V5:
> 1. Address the comments from Gong Lei
> 2. Speed the failover up. The secondary vm can take over very quickly even
>     if there are too many I/O requests.
> V4:
> 1. Introduce a new driver replication to avoid touch nbd and qcow2.
> V3:
> 1: use error_setg() instead of error_set()
> 2. Add a new block job API
> 3. Active disk, hidden disk and nbd target uses the same AioContext
> 4. Add a testcase to test new hbitmap API
> V2:
> 1. Redesign the secondary qemu(use image-fleecing)
> 2. Use Error objects to return error message
> 3. Address the comments from Max Reitz and Eric Blake
>
>
> Wen Congyang (17):
>    Add new block driver interface to add/delete a BDS's child
>    quorum: implement block driver interfaces add/delete a BDS's child
>    hmp: add monitor command to add/remove a child
>    introduce a new API qemu_opts_absorb_qdict_by_index()
>    quorum: allow ignoring child errors
>    introduce a new API to enable/disable attach device model
>    introduce a new API to check if blk is attached
>    block: make bdrv_put_ref_bh_schedule() as a public API
>    Backup: clear all bitmap when doing block checkpoint
>    allow writing to the backing file
>    Allow creating backup jobs when opening BDS
>    block: Allow references for backing files
>    docs: block replication's description
>    Add new block driver interfaces to control block replication
>    skip nbd_target when starting block replication
>    quorum: implement block driver interfaces for block replication
>    Implement new driver for block replication
>
>   block.c                        | 198 +++++++++++++++++-
>   block/Makefile.objs            |   3 +-
>   block/backup.c                 |  13 ++
>   block/block-backend.c          |  33 +++
>   block/quorum.c                 | 244 ++++++++++++++++++++++-
>   block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
>   blockdev.c                     |  90 ++++++---
>   blockjob.c                     |  10 +
>   docs/block-replication.txt     | 179 +++++++++++++++++
>   hmp-commands.hx                |  28 +++
>   include/block/block.h          |  11 +
>   include/block/block_int.h      |  19 ++
>   include/block/blockjob.h       |  12 ++
>   include/qemu/option.h          |   2 +
>   include/sysemu/block-backend.h |   3 +
>   include/sysemu/blockdev.h      |   2 +
>   qapi/block.json                |  16 ++
>   util/qemu-option.c             |  44 ++++
>   18 files changed, 1303 insertions(+), 47 deletions(-)
>   create mode 100644 block/replication.c
>   create mode 100644 docs/block-replication.txt
>

Can you move the interfaces for 
blk_start_replication()/blk_stop_replication/blk_do_checkpoint() to this 
patch series
instead of the COLO patch series?

I don't think those functions are specific to COLO --- they should be 
available to all users.

- Michael

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/02/2015 10:59 PM, Michael R. Hines wrote:
> On 06/29/2015 10:34 PM, Wen Congyang wrote:
>> Block replication is a very important feature which is used for
>> continuous checkpoints(for example: COLO).
>>
>> Usage:
>> Please refer to docs/block-replication.txt
>>
>> You can get the patch here:
>> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>>
>> You can get ths patch with framework here:
>> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>>
>> TODO:
>> 1. Continuous block replication. It will be started after basic functions
>>     are accepted.
>>
>> Changs Log:
>> V7:
>> 1. Implement adding/removing quorum child. Remove the option non-connect.
>> 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
>> V6:
>> 1. Rebase to the newest qemu.
>> V5:
>> 1. Address the comments from Gong Lei
>> 2. Speed the failover up. The secondary vm can take over very quickly even
>>     if there are too many I/O requests.
>> V4:
>> 1. Introduce a new driver replication to avoid touch nbd and qcow2.
>> V3:
>> 1: use error_setg() instead of error_set()
>> 2. Add a new block job API
>> 3. Active disk, hidden disk and nbd target uses the same AioContext
>> 4. Add a testcase to test new hbitmap API
>> V2:
>> 1. Redesign the secondary qemu(use image-fleecing)
>> 2. Use Error objects to return error message
>> 3. Address the comments from Max Reitz and Eric Blake
>>
>>
>> Wen Congyang (17):
>>    Add new block driver interface to add/delete a BDS's child
>>    quorum: implement block driver interfaces add/delete a BDS's child
>>    hmp: add monitor command to add/remove a child
>>    introduce a new API qemu_opts_absorb_qdict_by_index()
>>    quorum: allow ignoring child errors
>>    introduce a new API to enable/disable attach device model
>>    introduce a new API to check if blk is attached
>>    block: make bdrv_put_ref_bh_schedule() as a public API
>>    Backup: clear all bitmap when doing block checkpoint
>>    allow writing to the backing file
>>    Allow creating backup jobs when opening BDS
>>    block: Allow references for backing files
>>    docs: block replication's description
>>    Add new block driver interfaces to control block replication
>>    skip nbd_target when starting block replication
>>    quorum: implement block driver interfaces for block replication
>>    Implement new driver for block replication
>>
>>   block.c                        | 198 +++++++++++++++++-
>>   block/Makefile.objs            |   3 +-
>>   block/backup.c                 |  13 ++
>>   block/block-backend.c          |  33 +++
>>   block/quorum.c                 | 244 ++++++++++++++++++++++-
>>   block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
>>   blockdev.c                     |  90 ++++++---
>>   blockjob.c                     |  10 +
>>   docs/block-replication.txt     | 179 +++++++++++++++++
>>   hmp-commands.hx                |  28 +++
>>   include/block/block.h          |  11 +
>>   include/block/block_int.h      |  19 ++
>>   include/block/blockjob.h       |  12 ++
>>   include/qemu/option.h          |   2 +
>>   include/sysemu/block-backend.h |   3 +
>>   include/sysemu/blockdev.h      |   2 +
>>   qapi/block.json                |  16 ++
>>   util/qemu-option.c             |  44 ++++
>>   18 files changed, 1303 insertions(+), 47 deletions(-)
>>   create mode 100644 block/replication.c
>>   create mode 100644 docs/block-replication.txt
>>
> 
> Can you move the interfaces for blk_start_replication()/blk_stop_replication/blk_do_checkpoint() to this patch series
> instead of the COLO patch series?
> 
> I don't think those functions are specific to COLO --- they should be available to all users.

OK, I will do it in the next version.

Thanks
Wen Congyang

> 
> - Michael
> 
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Dr. David Alan Gilbert]

* Wen Congyang (wency@cn.fujitsu.com) wrote:
> Block replication is a very important feature which is used for
> continuous checkpoints(for example: COLO).
> 
> Usage:
> Please refer to docs/block-replication.txt
> 
> You can get the patch here:
> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
> 
> You can get ths patch with framework here:
> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2

Hi,
  I seem to be having problems with the new listed syntax on the wiki;
on the secondary I'm getting the error

 Block format 'replication' used by device 'virtio0' doesn't support the option 'export'

./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
     -boot c -m 4096 -smp 4 -S \
     -name debug-threads=on -trace events=trace-file \
     -netdev tap,id=hn0,script=$PWD/ifup-slave,\
downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
     -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
     -device virtio-rng-pci \
     -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
     -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
file.driver=qcow2,\
file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
file.backing.driver=qcow2,\
file.backing.backing.backing_reference=colo1,\
file.backing.allow-write-backing-file=on \
     -incoming tcp:0:8888

This is using 6fd6ce32 from the colo_framework_v7.2  tag.

What have I missed?

Dave
P.S. the name 'file.backing.backing.backing_reference' is not nice!

> 
> TODO:
> 1. Continuous block replication. It will be started after basic functions
>    are accepted.
> 
> Changs Log:
> V7:
> 1. Implement adding/removing quorum child. Remove the option non-connect.
> 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
> V6:
> 1. Rebase to the newest qemu.
> V5:
> 1. Address the comments from Gong Lei
> 2. Speed the failover up. The secondary vm can take over very quickly even
>    if there are too many I/O requests.
> V4:
> 1. Introduce a new driver replication to avoid touch nbd and qcow2.
> V3:
> 1: use error_setg() instead of error_set()
> 2. Add a new block job API
> 3. Active disk, hidden disk and nbd target uses the same AioContext
> 4. Add a testcase to test new hbitmap API
> V2:
> 1. Redesign the secondary qemu(use image-fleecing)
> 2. Use Error objects to return error message
> 3. Address the comments from Max Reitz and Eric Blake
> 
> 
> Wen Congyang (17):
>   Add new block driver interface to add/delete a BDS's child
>   quorum: implement block driver interfaces add/delete a BDS's child
>   hmp: add monitor command to add/remove a child
>   introduce a new API qemu_opts_absorb_qdict_by_index()
>   quorum: allow ignoring child errors
>   introduce a new API to enable/disable attach device model
>   introduce a new API to check if blk is attached
>   block: make bdrv_put_ref_bh_schedule() as a public API
>   Backup: clear all bitmap when doing block checkpoint
>   allow writing to the backing file
>   Allow creating backup jobs when opening BDS
>   block: Allow references for backing files
>   docs: block replication's description
>   Add new block driver interfaces to control block replication
>   skip nbd_target when starting block replication
>   quorum: implement block driver interfaces for block replication
>   Implement new driver for block replication
> 
>  block.c                        | 198 +++++++++++++++++-
>  block/Makefile.objs            |   3 +-
>  block/backup.c                 |  13 ++
>  block/block-backend.c          |  33 +++
>  block/quorum.c                 | 244 ++++++++++++++++++++++-
>  block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
>  blockdev.c                     |  90 ++++++---
>  blockjob.c                     |  10 +
>  docs/block-replication.txt     | 179 +++++++++++++++++
>  hmp-commands.hx                |  28 +++
>  include/block/block.h          |  11 +
>  include/block/block_int.h      |  19 ++
>  include/block/blockjob.h       |  12 ++
>  include/qemu/option.h          |   2 +
>  include/sysemu/block-backend.h |   3 +
>  include/sysemu/blockdev.h      |   2 +
>  qapi/block.json                |  16 ++
>  util/qemu-option.c             |  44 ++++
>  18 files changed, 1303 insertions(+), 47 deletions(-)
>  create mode 100644 block/replication.c
>  create mode 100644 docs/block-replication.txt
> 
> -- 
> 2.4.3
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote:
> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>> Block replication is a very important feature which is used for
>> continuous checkpoints(for example: COLO).
>>
>> Usage:
>> Please refer to docs/block-replication.txt
>>
>> You can get the patch here:
>> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>>
>> You can get ths patch with framework here:
>> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>
> Hi,
>    I seem to be having problems with the new listed syntax on the wiki;
> on the secondary I'm getting the error
>
>   Block format 'replication' used by device 'virtio0' doesn't support the option 'export'
>
> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>       -boot c -m 4096 -smp 4 -S \
>       -name debug-threads=on -trace events=trace-file \
>       -netdev tap,id=hn0,script=$PWD/ifup-slave,\
> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
>       -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>       -device virtio-rng-pci \
>       -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
>       -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
> file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
> file.driver=qcow2,\
> file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
> file.backing.driver=qcow2,\
> file.backing.backing.backing_reference=colo1,\
> file.backing.allow-write-backing-file=on \
>       -incoming tcp:0:8888

Sorry, the option export is removed, because we use the qmp command 
nbd-server-add to let a BB be NBD server.

>
> This is using 6fd6ce32 from the colo_framework_v7.2  tag.
>
> What have I missed?
>
> Dave
> P.S. the name 'file.backing.backing.backing_reference' is not nice!

Which name is better? My native language is not English, so any such 
suggestion is welcome.

Thanks
Wen Congyang

>
>>
>> TODO:
>> 1. Continuous block replication. It will be started after basic functions
>>     are accepted.
>>
>> Changs Log:
>> V7:
>> 1. Implement adding/removing quorum child. Remove the option non-connect.
>> 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
>> V6:
>> 1. Rebase to the newest qemu.
>> V5:
>> 1. Address the comments from Gong Lei
>> 2. Speed the failover up. The secondary vm can take over very quickly even
>>     if there are too many I/O requests.
>> V4:
>> 1. Introduce a new driver replication to avoid touch nbd and qcow2.
>> V3:
>> 1: use error_setg() instead of error_set()
>> 2. Add a new block job API
>> 3. Active disk, hidden disk and nbd target uses the same AioContext
>> 4. Add a testcase to test new hbitmap API
>> V2:
>> 1. Redesign the secondary qemu(use image-fleecing)
>> 2. Use Error objects to return error message
>> 3. Address the comments from Max Reitz and Eric Blake
>>
>>
>> Wen Congyang (17):
>>    Add new block driver interface to add/delete a BDS's child
>>    quorum: implement block driver interfaces add/delete a BDS's child
>>    hmp: add monitor command to add/remove a child
>>    introduce a new API qemu_opts_absorb_qdict_by_index()
>>    quorum: allow ignoring child errors
>>    introduce a new API to enable/disable attach device model
>>    introduce a new API to check if blk is attached
>>    block: make bdrv_put_ref_bh_schedule() as a public API
>>    Backup: clear all bitmap when doing block checkpoint
>>    allow writing to the backing file
>>    Allow creating backup jobs when opening BDS
>>    block: Allow references for backing files
>>    docs: block replication's description
>>    Add new block driver interfaces to control block replication
>>    skip nbd_target when starting block replication
>>    quorum: implement block driver interfaces for block replication
>>    Implement new driver for block replication
>>
>>   block.c                        | 198 +++++++++++++++++-
>>   block/Makefile.objs            |   3 +-
>>   block/backup.c                 |  13 ++
>>   block/block-backend.c          |  33 +++
>>   block/quorum.c                 | 244 ++++++++++++++++++++++-
>>   block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
>>   blockdev.c                     |  90 ++++++---
>>   blockjob.c                     |  10 +
>>   docs/block-replication.txt     | 179 +++++++++++++++++
>>   hmp-commands.hx                |  28 +++
>>   include/block/block.h          |  11 +
>>   include/block/block_int.h      |  19 ++
>>   include/block/blockjob.h       |  12 ++
>>   include/qemu/option.h          |   2 +
>>   include/sysemu/block-backend.h |   3 +
>>   include/sysemu/blockdev.h      |   2 +
>>   qapi/block.json                |  16 ++
>>   util/qemu-option.c             |  44 ++++
>>   18 files changed, 1303 insertions(+), 47 deletions(-)
>>   create mode 100644 block/replication.c
>>   create mode 100644 docs/block-replication.txt
>>
>> --
>> 2.4.3
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>
>

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Dr. David Alan Gilbert]

* Wen Congyang (ghostwcy@gmail.com) wrote:
> At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote:
> >* Wen Congyang (wency@cn.fujitsu.com) wrote:
> >>Block replication is a very important feature which is used for
> >>continuous checkpoints(for example: COLO).
> >>
> >>Usage:
> >>Please refer to docs/block-replication.txt
> >>
> >>You can get the patch here:
> >>https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
> >>
> >>You can get ths patch with framework here:
> >>https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
> >
> >Hi,
> >   I seem to be having problems with the new listed syntax on the wiki;
> >on the secondary I'm getting the error
> >
> >  Block format 'replication' used by device 'virtio0' doesn't support the option 'export'
> >
> >./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
> >      -boot c -m 4096 -smp 4 -S \
> >      -name debug-threads=on -trace events=trace-file \
> >      -netdev tap,id=hn0,script=$PWD/ifup-slave,\
> >downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
> >      -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
> >      -device virtio-rng-pci \
> >      -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
> >      -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
> >file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
> >file.driver=qcow2,\
> >file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
> >file.backing.driver=qcow2,\
> >file.backing.backing.backing_reference=colo1,\
> >file.backing.allow-write-backing-file=on \
> >      -incoming tcp:0:8888
> 
> Sorry, the option export is removed, because we use the qmp command
> nbd-server-add to let a BB be NBD server.

If I remove the export=  the secondary starts, but then when I issue the child_add
on the primary I get the error:

Server requires an export name

on the secondary I had done:

(qemu) nbd_server_start :8889
(qemu) nbd_server_add -w colo1

Can you please confirm your exact command line and sequence needed with this 7.2 set please.

> 
> >This is using 6fd6ce32 from the colo_framework_v7.2  tag.
> >
> >What have I missed?
> >
> >Dave
> >P.S. the name 'file.backing.backing.backing_reference' is not nice!
> 
> Which name is better? My native language is not English, so any such
> suggestion is welcome.

My only problem is the way that 'backing' is repeated 3 times; that seems
odd!

Dave

> 
> Thanks
> Wen Congyang
> 
> >
> >>
> >>TODO:
> >>1. Continuous block replication. It will be started after basic functions
> >>    are accepted.
> >>
> >>Changs Log:
> >>V7:
> >>1. Implement adding/removing quorum child. Remove the option non-connect.
> >>2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
> >>V6:
> >>1. Rebase to the newest qemu.
> >>V5:
> >>1. Address the comments from Gong Lei
> >>2. Speed the failover up. The secondary vm can take over very quickly even
> >>    if there are too many I/O requests.
> >>V4:
> >>1. Introduce a new driver replication to avoid touch nbd and qcow2.
> >>V3:
> >>1: use error_setg() instead of error_set()
> >>2. Add a new block job API
> >>3. Active disk, hidden disk and nbd target uses the same AioContext
> >>4. Add a testcase to test new hbitmap API
> >>V2:
> >>1. Redesign the secondary qemu(use image-fleecing)
> >>2. Use Error objects to return error message
> >>3. Address the comments from Max Reitz and Eric Blake
> >>
> >>
> >>Wen Congyang (17):
> >>   Add new block driver interface to add/delete a BDS's child
> >>   quorum: implement block driver interfaces add/delete a BDS's child
> >>   hmp: add monitor command to add/remove a child
> >>   introduce a new API qemu_opts_absorb_qdict_by_index()
> >>   quorum: allow ignoring child errors
> >>   introduce a new API to enable/disable attach device model
> >>   introduce a new API to check if blk is attached
> >>   block: make bdrv_put_ref_bh_schedule() as a public API
> >>   Backup: clear all bitmap when doing block checkpoint
> >>   allow writing to the backing file
> >>   Allow creating backup jobs when opening BDS
> >>   block: Allow references for backing files
> >>   docs: block replication's description
> >>   Add new block driver interfaces to control block replication
> >>   skip nbd_target when starting block replication
> >>   quorum: implement block driver interfaces for block replication
> >>   Implement new driver for block replication
> >>
> >>  block.c                        | 198 +++++++++++++++++-
> >>  block/Makefile.objs            |   3 +-
> >>  block/backup.c                 |  13 ++
> >>  block/block-backend.c          |  33 +++
> >>  block/quorum.c                 | 244 ++++++++++++++++++++++-
> >>  block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
> >>  blockdev.c                     |  90 ++++++---
> >>  blockjob.c                     |  10 +
> >>  docs/block-replication.txt     | 179 +++++++++++++++++
> >>  hmp-commands.hx                |  28 +++
> >>  include/block/block.h          |  11 +
> >>  include/block/block_int.h      |  19 ++
> >>  include/block/blockjob.h       |  12 ++
> >>  include/qemu/option.h          |   2 +
> >>  include/sysemu/block-backend.h |   3 +
> >>  include/sysemu/blockdev.h      |   2 +
> >>  qapi/block.json                |  16 ++
> >>  util/qemu-option.c             |  44 ++++
> >>  18 files changed, 1303 insertions(+), 47 deletions(-)
> >>  create mode 100644 block/replication.c
> >>  create mode 100644 docs/block-replication.txt
> >>
> >>--
> >>2.4.3
> >>
> >--
> >Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >
> >
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/06/2015 05:42 PM, Dr. David Alan Gilbert wrote:
> * Wen Congyang (ghostwcy@gmail.com) wrote:
>> At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote:
>>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>>>> Block replication is a very important feature which is used for
>>>> continuous checkpoints(for example: COLO).
>>>>
>>>> Usage:
>>>> Please refer to docs/block-replication.txt
>>>>
>>>> You can get the patch here:
>>>> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>>>>
>>>> You can get ths patch with framework here:
>>>> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>>>
>>> Hi,
>>>   I seem to be having problems with the new listed syntax on the wiki;
>>> on the secondary I'm getting the error
>>>
>>>  Block format 'replication' used by device 'virtio0' doesn't support the option 'export'
>>>
>>> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>>>      -boot c -m 4096 -smp 4 -S \
>>>      -name debug-threads=on -trace events=trace-file \
>>>      -netdev tap,id=hn0,script=$PWD/ifup-slave,\
>>> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
>>>      -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>>>      -device virtio-rng-pci \
>>>      -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
>>>      -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
>>> file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
>>> file.driver=qcow2,\
>>> file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
>>> file.backing.driver=qcow2,\
>>> file.backing.backing.backing_reference=colo1,\
>>> file.backing.allow-write-backing-file=on \
>>>      -incoming tcp:0:8888
>>
>> Sorry, the option export is removed, because we use the qmp command
>> nbd-server-add to let a BB be NBD server.
> 
> If I remove the export=  the secondary starts, but then when I issue the child_add
> on the primary I get the error:
> 
> Server requires an export name
> 
> on the secondary I had done:
> 
> (qemu) nbd_server_start :8889
> (qemu) nbd_server_add -w colo1
> 
> Can you please confirm your exact command line and sequence needed with this 7.2 set please.

This is the child_add command that I used:

child_add disk1 child.driver=replication,child.mode=primary,child.file.host=$nbd_host,child.file.port=$nbd_port,child.file.export=colo1,child.file.driver=nbd,child.ignore-errors=on

I guess you remove export from this command too.

Thanks
Wen Congyang

> 
>>
>>> This is using 6fd6ce32 from the colo_framework_v7.2  tag.
>>>
>>> What have I missed?
>>>
>>> Dave
>>> P.S. the name 'file.backing.backing.backing_reference' is not nice!
>>
>> Which name is better? My native language is not English, so any such
>> suggestion is welcome.
> 
> My only problem is the way that 'backing' is repeated 3 times; that seems
> odd!
> 
> Dave
> 
>>
>> Thanks
>> Wen Congyang
>>
>>>
>>>>
>>>> TODO:
>>>> 1. Continuous block replication. It will be started after basic functions
>>>>    are accepted.
>>>>
>>>> Changs Log:
>>>> V7:
>>>> 1. Implement adding/removing quorum child. Remove the option non-connect.
>>>> 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion
>>>> V6:
>>>> 1. Rebase to the newest qemu.
>>>> V5:
>>>> 1. Address the comments from Gong Lei
>>>> 2. Speed the failover up. The secondary vm can take over very quickly even
>>>>    if there are too many I/O requests.
>>>> V4:
>>>> 1. Introduce a new driver replication to avoid touch nbd and qcow2.
>>>> V3:
>>>> 1: use error_setg() instead of error_set()
>>>> 2. Add a new block job API
>>>> 3. Active disk, hidden disk and nbd target uses the same AioContext
>>>> 4. Add a testcase to test new hbitmap API
>>>> V2:
>>>> 1. Redesign the secondary qemu(use image-fleecing)
>>>> 2. Use Error objects to return error message
>>>> 3. Address the comments from Max Reitz and Eric Blake
>>>>
>>>>
>>>> Wen Congyang (17):
>>>>   Add new block driver interface to add/delete a BDS's child
>>>>   quorum: implement block driver interfaces add/delete a BDS's child
>>>>   hmp: add monitor command to add/remove a child
>>>>   introduce a new API qemu_opts_absorb_qdict_by_index()
>>>>   quorum: allow ignoring child errors
>>>>   introduce a new API to enable/disable attach device model
>>>>   introduce a new API to check if blk is attached
>>>>   block: make bdrv_put_ref_bh_schedule() as a public API
>>>>   Backup: clear all bitmap when doing block checkpoint
>>>>   allow writing to the backing file
>>>>   Allow creating backup jobs when opening BDS
>>>>   block: Allow references for backing files
>>>>   docs: block replication's description
>>>>   Add new block driver interfaces to control block replication
>>>>   skip nbd_target when starting block replication
>>>>   quorum: implement block driver interfaces for block replication
>>>>   Implement new driver for block replication
>>>>
>>>>  block.c                        | 198 +++++++++++++++++-
>>>>  block/Makefile.objs            |   3 +-
>>>>  block/backup.c                 |  13 ++
>>>>  block/block-backend.c          |  33 +++
>>>>  block/quorum.c                 | 244 ++++++++++++++++++++++-
>>>>  block/replication.c            | 443 +++++++++++++++++++++++++++++++++++++++++
>>>>  blockdev.c                     |  90 ++++++---
>>>>  blockjob.c                     |  10 +
>>>>  docs/block-replication.txt     | 179 +++++++++++++++++
>>>>  hmp-commands.hx                |  28 +++
>>>>  include/block/block.h          |  11 +
>>>>  include/block/block_int.h      |  19 ++
>>>>  include/block/blockjob.h       |  12 ++
>>>>  include/qemu/option.h          |   2 +
>>>>  include/sysemu/block-backend.h |   3 +
>>>>  include/sysemu/blockdev.h      |   2 +
>>>>  qapi/block.json                |  16 ++
>>>>  util/qemu-option.c             |  44 ++++
>>>>  18 files changed, 1303 insertions(+), 47 deletions(-)
>>>>  create mode 100644 block/replication.c
>>>>  create mode 100644 docs/block-replication.txt
>>>>
>>>> --
>>>> 2.4.3
>>>>
>>> --
>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>>
>>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Michael R. Hines]

On 07/04/2015 07:46 AM, Wen Congyang wrote:
> At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote:
>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>>> Block replication is a very important feature which is used for
>>> continuous checkpoints(for example: COLO).
>>>
>>> Usage:
>>> Please refer to docs/block-replication.txt
>>>
>>> You can get the patch here:
>>> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>>>
>>> You can get ths patch with framework here:
>>> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>>
>> Hi,
>>    I seem to be having problems with the new listed syntax on the wiki;
>> on the secondary I'm getting the error
>>
>>   Block format 'replication' used by device 'virtio0' doesn't support 
>> the option 'export'
>>
>> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>>       -boot c -m 4096 -smp 4 -S \
>>       -name debug-threads=on -trace events=trace-file \
>>       -netdev tap,id=hn0,script=$PWD/ifup-slave,\
>> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 
>> \
>>       -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>>       -device virtio-rng-pci \
>>       -drive 
>> if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native 
>> \
>>       -drive 
>> if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
>> file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
>> file.driver=qcow2,\
>> file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
>> file.backing.driver=qcow2,\
>> file.backing.backing.backing_reference=colo1,\
>> file.backing.allow-write-backing-file=on \
>>       -incoming tcp:0:8888
>
> Sorry, the option export is removed, because we use the qmp command 
> nbd-server-add to let a BB be NBD server.
>

Still doesn't work. The server says:

nbd.c:nbd_receive_options():L447: read failed
nbd.c:nbd_send_negotiate():L562: option negotiation failed

- Michael

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/07/2015 08:25 AM, Michael R. Hines wrote:
> On 07/04/2015 07:46 AM, Wen Congyang wrote:
>> At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote:
>>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>>>> Block replication is a very important feature which is used for
>>>> continuous checkpoints(for example: COLO).
>>>>
>>>> Usage:
>>>> Please refer to docs/block-replication.txt
>>>>
>>>> You can get the patch here:
>>>> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
>>>>
>>>> You can get ths patch with framework here:
>>>> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
>>>
>>> Hi,
>>>    I seem to be having problems with the new listed syntax on the wiki;
>>> on the secondary I'm getting the error
>>>
>>>   Block format 'replication' used by device 'virtio0' doesn't support the option 'export'
>>>
>>> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>>>       -boot c -m 4096 -smp 4 -S \
>>>       -name debug-threads=on -trace events=trace-file \
>>>       -netdev tap,id=hn0,script=$PWD/ifup-slave,\
>>> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
>>>       -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>>>       -device virtio-rng-pci \
>>>       -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
>>>       -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
>>> file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
>>> file.driver=qcow2,\
>>> file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
>>> file.backing.driver=qcow2,\
>>> file.backing.backing.backing_reference=colo1,\
>>> file.backing.allow-write-backing-file=on \
>>>       -incoming tcp:0:8888
>>
>> Sorry, the option export is removed, because we use the qmp command nbd-server-add to let a BB be NBD server.
>>
> 
> Still doesn't work. The server says:
> 
> nbd.c:nbd_receive_options():L447: read failed

This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
give me more log and how do you use it?

Thanks
Wen Congyang

> nbd.c:nbd_send_negotiate():L562: option negotiation failed
> 
> - Michael
> 
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Dr. David Alan Gilbert]

* Wen Congyang (wency@cn.fujitsu.com) wrote:
> On 07/07/2015 08:25 AM, Michael R. Hines wrote:
> > On 07/04/2015 07:46 AM, Wen Congyang wrote:
> >> At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote:
> >>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
> >>>> Block replication is a very important feature which is used for
> >>>> continuous checkpoints(for example: COLO).
> >>>>
> >>>> Usage:
> >>>> Please refer to docs/block-replication.txt
> >>>>
> >>>> You can get the patch here:
> >>>> https://github.com/wencongyang/qemu-colo/commits/block-replication-v7
> >>>>
> >>>> You can get ths patch with framework here:
> >>>> https://github.com/wencongyang/qemu-colo/commits/colo_framework_v7.2
> >>>
> >>> Hi,
> >>>    I seem to be having problems with the new listed syntax on the wiki;
> >>> on the secondary I'm getting the error
> >>>
> >>>   Block format 'replication' used by device 'virtio0' doesn't support the option 'export'
> >>>
> >>> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
> >>>       -boot c -m 4096 -smp 4 -S \
> >>>       -name debug-threads=on -trace events=trace-file \
> >>>       -netdev tap,id=hn0,script=$PWD/ifup-slave,\
> >>> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
> >>>       -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
> >>>       -device virtio-rng-pci \
> >>>       -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
> >>>       -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
> >>> file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
> >>> file.driver=qcow2,\
> >>> file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
> >>> file.backing.driver=qcow2,\
> >>> file.backing.backing.backing_reference=colo1,\
> >>> file.backing.allow-write-backing-file=on \
> >>>       -incoming tcp:0:8888
> >>
> >> Sorry, the option export is removed, because we use the qmp command nbd-server-add to let a BB be NBD server.
> >>
> > 
> > Still doesn't work. The server says:
> > 
> > nbd.c:nbd_receive_options():L447: read failed
> 
> This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
> from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
> give me more log and how do you use it?

That was the same failure I was getting.   I think it's that the NBD server and client are in different
modes, with one of them expecting the export.

Dave

> Thanks
> Wen Congyang
> 
> > nbd.c:nbd_send_negotiate():L562: option negotiation failed
> > 
> > - Michael
> > 
> > .
> > 
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Paolo Bonzini]

On 07/07/2015 11:13, Dr. David Alan Gilbert wrote:
>> > This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
>> > from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
>> > give me more log and how do you use it?
> That was the same failure I was getting.   I think it's that the NBD server and client are in different
> modes, with one of them expecting the export.

nbd_server_add always expects the export.

Paolo

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Michael R. Hines]

On 07/07/2015 04:23 AM, Paolo Bonzini wrote:
>
> On 07/07/2015 11:13, Dr. David Alan Gilbert wrote:
>>>> This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
>>>> from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
>>>> give me more log and how do you use it?
>> That was the same failure I was getting.   I think it's that the NBD server and client are in different
>> modes, with one of them expecting the export.
> nbd_server_add always expects the export.
>
> Paolo
>

OK, Wen, so your wiki finally does reflect this, but now we're back to 
the "export not found error".

Again, here's the exact command line:

1. First on the secondary VM:

qemu-system-x86_64 .........snip...........  -drive 
if=none,driver=qcow2,file=foo.qcow2,id=mc1,cache=none,aio=native -drive 
if=virtio,driver=replication,mode=secondary,throttling.bps-total-max=70000000,file.file.filename=active_disk.qcow2,file.driver=qcow2,file.backing.file.filename=hidden_disk.qcow2,file.backing.driver=qcow2,file.backing.allow-write-backing-file=on,file.backing.backing.backing_reference=mc1

2. Then, then HMP commands:

nbd_server_start 0:6262
nbd_server_add -w mc1

3. Then the primary VM:

qemu-system-x86_64 .........snip...........  -drive 
if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on

With the error: Server requires an export name

*but*, your wiki has no export name on the primary VM size, so I added 
the export name back which is on your old wiki:

qemu-system-x86_64 .........snip...........  -drive 
if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.export=mc1,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on: 
Failed to read export length

And server now says:

nbd.c:nbd_handle_export_name():L416: export not found
nbd.c:nbd_send_negotiate():L562: option negotiation failed

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/08/2015 12:56 AM, Michael R. Hines wrote:
> On 07/07/2015 04:23 AM, Paolo Bonzini wrote:
>>
>> On 07/07/2015 11:13, Dr. David Alan Gilbert wrote:
>>>>> This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
>>>>> from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
>>>>> give me more log and how do you use it?
>>> That was the same failure I was getting.   I think it's that the NBD server and client are in different
>>> modes, with one of them expecting the export.
>> nbd_server_add always expects the export.
>>
>> Paolo
>>
> 
> OK, Wen, so your wiki finally does reflect this, but now we're back to the "export not found error".
> 
> Again, here's the exact command line:
> 
> 1. First on the secondary VM:
> 
> qemu-system-x86_64 .........snip...........  -drive if=none,driver=qcow2,file=foo.qcow2,id=mc1,cache=none,aio=native -drive if=virtio,driver=replication,mode=secondary,throttling.bps-total-max=70000000,file.file.filename=active_disk.qcow2,file.driver=qcow2,file.backing.file.filename=hidden_disk.qcow2,file.backing.driver=qcow2,file.backing.allow-write-backing-file=on,file.backing.backing.backing_reference=mc1
> 
> 2. Then, then HMP commands:
> 
> nbd_server_start 0:6262
> nbd_server_add -w mc1
> 
> 3. Then the primary VM:
> 
> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on
> 
> With the error: Server requires an export name
> 
> *but*, your wiki has no export name on the primary VM size, so I added the export name back which is on your old wiki:
> 
> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.export=mc1,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on: Failed to read export length
> 

Hmm, I think you use v7 version. Is it right?
In this version, the correct useage for primary qemu is:
1. qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,id=disk1,
                      children.0.file.filename=bar.qcow2,children.0.driver=qcow2,

Then run hmp monitor command(It should be run after you run nbd_server_add in the secondary qemu):
child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on

Thanks
Wen Congyang

> And server now says:
> 
> nbd.c:nbd_handle_export_name():L416: export not found
> nbd.c:nbd_send_negotiate():L562: option negotiation failed
> 
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Michael R. Hines]

On 07/07/2015 08:38 PM, Wen Congyang wrote:
> On 07/08/2015 12:56 AM, Michael R. Hines wrote:
>> On 07/07/2015 04:23 AM, Paolo Bonzini wrote:
>>> On 07/07/2015 11:13, Dr. David Alan Gilbert wrote:
>>>>>> This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
>>>>>> from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
>>>>>> give me more log and how do you use it?
>>>> That was the same failure I was getting.   I think it's that the NBD server and client are in different
>>>> modes, with one of them expecting the export.
>>> nbd_server_add always expects the export.
>>>
>>> Paolo
>>>
>> OK, Wen, so your wiki finally does reflect this, but now we're back to the "export not found error".
>>
>> Again, here's the exact command line:
>>
>> 1. First on the secondary VM:
>>
>> qemu-system-x86_64 .........snip...........  -drive if=none,driver=qcow2,file=foo.qcow2,id=mc1,cache=none,aio=native -drive if=virtio,driver=replication,mode=secondary,throttling.bps-total-max=70000000,file.file.filename=active_disk.qcow2,file.driver=qcow2,file.backing.file.filename=hidden_disk.qcow2,file.backing.driver=qcow2,file.backing.allow-write-backing-file=on,file.backing.backing.backing_reference=mc1
>>
>> 2. Then, then HMP commands:
>>
>> nbd_server_start 0:6262
>> nbd_server_add -w mc1
>>
>> 3. Then the primary VM:
>>
>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on
>>
>> With the error: Server requires an export name
>>
>> *but*, your wiki has no export name on the primary VM size, so I added the export name back which is on your old wiki:
>>
>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.export=mc1,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on: Failed to read export length
>>
> Hmm, I think you use v7 version. Is it right?
> In this version, the correct useage for primary qemu is:
> 1. qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,id=disk1,
>                        children.0.file.filename=bar.qcow2,children.0.driver=qcow2,
>
> Then run hmp monitor command(It should be run after you run nbd_server_add in the secondary qemu):
> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
>
> Thanks
> Wen Congyang
>
>> And server now says:
>>
>> nbd.c:nbd_handle_export_name():L416: export not found
>> nbd.c:nbd_send_negotiate():L562: option negotiation failed
>>
>> .
>>
>

OK, I'm totally confused at this point. =)

Maybe it would be easier to just wait for your next clean patchset + 
documentation which is all consistent with each other.

- Michael

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/08/2015 11:49 PM, Michael R. Hines wrote:
> On 07/07/2015 08:38 PM, Wen Congyang wrote:
>> On 07/08/2015 12:56 AM, Michael R. Hines wrote:
>>> On 07/07/2015 04:23 AM, Paolo Bonzini wrote:
>>>> On 07/07/2015 11:13, Dr. David Alan Gilbert wrote:
>>>>>>> This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
>>>>>>> from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
>>>>>>> give me more log and how do you use it?
>>>>> That was the same failure I was getting.   I think it's that the NBD server and client are in different
>>>>> modes, with one of them expecting the export.
>>>> nbd_server_add always expects the export.
>>>>
>>>> Paolo
>>>>
>>> OK, Wen, so your wiki finally does reflect this, but now we're back to the "export not found error".
>>>
>>> Again, here's the exact command line:
>>>
>>> 1. First on the secondary VM:
>>>
>>> qemu-system-x86_64 .........snip...........  -drive if=none,driver=qcow2,file=foo.qcow2,id=mc1,cache=none,aio=native -drive if=virtio,driver=replication,mode=secondary,throttling.bps-total-max=70000000,file.file.filename=active_disk.qcow2,file.driver=qcow2,file.backing.file.filename=hidden_disk.qcow2,file.backing.driver=qcow2,file.backing.allow-write-backing-file=on,file.backing.backing.backing_reference=mc1
>>>
>>> 2. Then, then HMP commands:
>>>
>>> nbd_server_start 0:6262
>>> nbd_server_add -w mc1
>>>
>>> 3. Then the primary VM:
>>>
>>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on
>>>
>>> With the error: Server requires an export name
>>>
>>> *but*, your wiki has no export name on the primary VM size, so I added the export name back which is on your old wiki:
>>>
>>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.export=mc1,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on: Failed to read export length
>>>
>> Hmm, I think you use v7 version. Is it right?
>> In this version, the correct useage for primary qemu is:
>> 1. qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,id=disk1,
>>                        children.0.file.filename=bar.qcow2,children.0.driver=qcow2,
>>
>> Then run hmp monitor command(It should be run after you run nbd_server_add in the secondary qemu):
>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
>>
>> Thanks
>> Wen Congyang
>>
>>> And server now says:
>>>
>>> nbd.c:nbd_handle_export_name():L416: export not found
>>> nbd.c:nbd_send_negotiate():L562: option negotiation failed
>>>
>>> .
>>>
>>
> 
> OK, I'm totally confused at this point. =)
> 
> Maybe it would be easier to just wait for your next clean patchset + documentation which is all consistent with each other.

I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
as a quorum child when the nbd server is ready.

The nbd server is ready after you run the following command:
nbd_server_start 0:6262 # the secondary qemu will listen to host:port
nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
                        # -w means we allow to write to this disk.

Then you can run the following command in the primary qemu:
child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on

After this monitor command, nbd client has connected to the nbd server.

Thanks
Wen Congyang

> 
> - Michael
> 
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Dr. David Alan Gilbert]

* Wen Congyang (wency@cn.fujitsu.com) wrote:
> On 07/08/2015 11:49 PM, Michael R. Hines wrote:
> > On 07/07/2015 08:38 PM, Wen Congyang wrote:
> >> On 07/08/2015 12:56 AM, Michael R. Hines wrote:
> >>> On 07/07/2015 04:23 AM, Paolo Bonzini wrote:
> >>>> On 07/07/2015 11:13, Dr. David Alan Gilbert wrote:
> >>>>>>> This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
> >>>>>>> from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
> >>>>>>> give me more log and how do you use it?
> >>>>> That was the same failure I was getting.   I think it's that the NBD server and client are in different
> >>>>> modes, with one of them expecting the export.
> >>>> nbd_server_add always expects the export.
> >>>>
> >>>> Paolo
> >>>>
> >>> OK, Wen, so your wiki finally does reflect this, but now we're back to the "export not found error".
> >>>
> >>> Again, here's the exact command line:
> >>>
> >>> 1. First on the secondary VM:
> >>>
> >>> qemu-system-x86_64 .........snip...........  -drive if=none,driver=qcow2,file=foo.qcow2,id=mc1,cache=none,aio=native -drive if=virtio,driver=replication,mode=secondary,throttling.bps-total-max=70000000,file.file.filename=active_disk.qcow2,file.driver=qcow2,file.backing.file.filename=hidden_disk.qcow2,file.backing.driver=qcow2,file.backing.allow-write-backing-file=on,file.backing.backing.backing_reference=mc1
> >>>
> >>> 2. Then, then HMP commands:
> >>>
> >>> nbd_server_start 0:6262
> >>> nbd_server_add -w mc1
> >>>
> >>> 3. Then the primary VM:
> >>>
> >>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on
> >>>
> >>> With the error: Server requires an export name
> >>>
> >>> *but*, your wiki has no export name on the primary VM size, so I added the export name back which is on your old wiki:
> >>>
> >>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.export=mc1,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on: Failed to read export length
> >>>
> >> Hmm, I think you use v7 version. Is it right?
> >> In this version, the correct useage for primary qemu is:
> >> 1. qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,id=disk1,
> >>                        children.0.file.filename=bar.qcow2,children.0.driver=qcow2,
> >>
> >> Then run hmp monitor command(It should be run after you run nbd_server_add in the secondary qemu):
> >> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
> >>
> >> Thanks
> >> Wen Congyang
> >>
> >>> And server now says:
> >>>
> >>> nbd.c:nbd_handle_export_name():L416: export not found
> >>> nbd.c:nbd_send_negotiate():L562: option negotiation failed
> >>>
> >>> .
> >>>
> >>
> > 
> > OK, I'm totally confused at this point. =)
> > 
> > Maybe it would be easier to just wait for your next clean patchset + documentation which is all consistent with each other.
> 
> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
> as a quorum child when the nbd server is ready.
> 
> The nbd server is ready after you run the following command:
> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
>                         # -w means we allow to write to this disk.
> 
> Then you can run the following command in the primary qemu:
> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
> 
> After this monitor command, nbd client has connected to the nbd server.

Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
that probably explains the problem that we've been having.

Dave

> 
> Thanks
> Wen Congyang
> 
> > 
> > - Michael
> > 
> > .
> > 
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/09/2015 09:55 AM, Dr. David Alan Gilbert wrote:
> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>> On 07/08/2015 11:49 PM, Michael R. Hines wrote:
>>> On 07/07/2015 08:38 PM, Wen Congyang wrote:
>>>> On 07/08/2015 12:56 AM, Michael R. Hines wrote:
>>>>> On 07/07/2015 04:23 AM, Paolo Bonzini wrote:
>>>>>> On 07/07/2015 11:13, Dr. David Alan Gilbert wrote:
>>>>>>>>> This log is very stange. The NBD client connects to NBD server, and NBD server wants to read data
>>>>>>>>> from NBD client, but reading fails. It seems that the connection is closed unexpectedly. Can you
>>>>>>>>> give me more log and how do you use it?
>>>>>>> That was the same failure I was getting.   I think it's that the NBD server and client are in different
>>>>>>> modes, with one of them expecting the export.
>>>>>> nbd_server_add always expects the export.
>>>>>>
>>>>>> Paolo
>>>>>>
>>>>> OK, Wen, so your wiki finally does reflect this, but now we're back to the "export not found error".
>>>>>
>>>>> Again, here's the exact command line:
>>>>>
>>>>> 1. First on the secondary VM:
>>>>>
>>>>> qemu-system-x86_64 .........snip...........  -drive if=none,driver=qcow2,file=foo.qcow2,id=mc1,cache=none,aio=native -drive if=virtio,driver=replication,mode=secondary,throttling.bps-total-max=70000000,file.file.filename=active_disk.qcow2,file.driver=qcow2,file.backing.file.filename=hidden_disk.qcow2,file.backing.driver=qcow2,file.backing.allow-write-backing-file=on,file.backing.backing.backing_reference=mc1
>>>>>
>>>>> 2. Then, then HMP commands:
>>>>>
>>>>> nbd_server_start 0:6262
>>>>> nbd_server_add -w mc1
>>>>>
>>>>> 3. Then the primary VM:
>>>>>
>>>>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on
>>>>>
>>>>> With the error: Server requires an export name
>>>>>
>>>>> *but*, your wiki has no export name on the primary VM size, so I added the export name back which is on your old wiki:
>>>>>
>>>>> qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,no-connect=on,children.0.file.filename=bar.qcow2,children.0.driver=qcow2,children.1.file.driver=nbd,children.1.file.export=mc1,children.1.file.host=127.0.0.1,children.1.file.port=6262,children.1.driver=replication,children.1.mode=primary,children.1.ignore-errors=on: Failed to read export length
>>>>>
>>>> Hmm, I think you use v7 version. Is it right?
>>>> In this version, the correct useage for primary qemu is:
>>>> 1. qemu-system-x86_64 .........snip...........  -drive if=virtio,driver=quorum,read-pattern=fifo,id=disk1,
>>>>                        children.0.file.filename=bar.qcow2,children.0.driver=qcow2,
>>>>
>>>> Then run hmp monitor command(It should be run after you run nbd_server_add in the secondary qemu):
>>>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
>>>>
>>>> Thanks
>>>> Wen Congyang
>>>>
>>>>> And server now says:
>>>>>
>>>>> nbd.c:nbd_handle_export_name():L416: export not found
>>>>> nbd.c:nbd_send_negotiate():L562: option negotiation failed
>>>>>
>>>>> .
>>>>>
>>>>
>>>
>>> OK, I'm totally confused at this point. =)
>>>
>>> Maybe it would be easier to just wait for your next clean patchset + documentation which is all consistent with each other.
>>
>> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
>> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
>> as a quorum child when the nbd server is ready.
>>
>> The nbd server is ready after you run the following command:
>> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
>> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
>>                         # -w means we allow to write to this disk.
>>
>> Then you can run the following command in the primary qemu:
>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
>>
>> After this monitor command, nbd client has connected to the nbd server.
> 
> Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
> that probably explains the problem that we've been having.

Sorry for this mistake.

Thanks
Wen Congyang

> 
> Dave
> 
>>
>> Thanks
>> Wen Congyang
>>
>>>
>>> - Michael
>>>
>>> .
>>>
>>
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Dr. David Alan Gilbert]

* Wen Congyang (wency@cn.fujitsu.com) wrote:

> >> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
> >> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
> >> as a quorum child when the nbd server is ready.
> >>
> >> The nbd server is ready after you run the following command:
> >> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
> >> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
> >>                         # -w means we allow to write to this disk.
> >>
> >> Then you can run the following command in the primary qemu:
> >> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
> >>
> >> After this monitor command, nbd client has connected to the nbd server.
> > 
> > Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
> > that probably explains the problem that we've been having.
> 
> Sorry for this mistake.

OK, so this is working for me (with the 7.2 world).   What isn't working in this setup is migrate -b, I get:

(qemu) Receiving block device images
Error unknown block device disk1
qemu-system-x86_64: error while loading state section id 1(block)
qemu-system-x86_64: load of migration failed: Invalid argument

Can you explain the id=disk1 on the master side?

Dave

> 
> Thanks
> Wen Congyang
> 
> > 
> > Dave
> > 
> >>
> >> Thanks
> >> Wen Congyang
> >>
> >>>
> >>> - Michael
> >>>
> >>> .
> >>>
> >>
> >>
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> > .
> > 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/09/2015 05:16 PM, Dr. David Alan Gilbert wrote:
> * Wen Congyang (wency@cn.fujitsu.com) wrote:
> 
>>>> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
>>>> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
>>>> as a quorum child when the nbd server is ready.
>>>>
>>>> The nbd server is ready after you run the following command:
>>>> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
>>>> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
>>>>                         # -w means we allow to write to this disk.
>>>>
>>>> Then you can run the following command in the primary qemu:
>>>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
>>>>
>>>> After this monitor command, nbd client has connected to the nbd server.
>>>
>>> Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
>>> that probably explains the problem that we've been having.
>>
>> Sorry for this mistake.
> 
> OK, so this is working for me (with the 7.2 world).   What isn't working in this setup is migrate -b, I get:
> 
> (qemu) Receiving block device images
> Error unknown block device disk1
> qemu-system-x86_64: error while loading state section id 1(block)
> qemu-system-x86_64: load of migration failed: Invalid argument
> 
> Can you explain the id=disk1 on the master side?

Can you give me the command line? I think the id in primary and secondary qemu is not same.

Thanks
Wen Congyang

> 
> Dave
> 
>>
>> Thanks
>> Wen Congyang
>>
>>>
>>> Dave
>>>
>>>>
>>>> Thanks
>>>> Wen Congyang
>>>>
>>>>>
>>>>> - Michael
>>>>>
>>>>> .
>>>>>
>>>>
>>>>
>>> --
>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>> .
>>>
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Dr. David Alan Gilbert]

* Wen Congyang (wency@cn.fujitsu.com) wrote:
> On 07/09/2015 05:16 PM, Dr. David Alan Gilbert wrote:
> > * Wen Congyang (wency@cn.fujitsu.com) wrote:
> > 
> >>>> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
> >>>> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
> >>>> as a quorum child when the nbd server is ready.
> >>>>
> >>>> The nbd server is ready after you run the following command:
> >>>> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
> >>>> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
> >>>>                         # -w means we allow to write to this disk.
> >>>>
> >>>> Then you can run the following command in the primary qemu:
> >>>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
> >>>>
> >>>> After this monitor command, nbd client has connected to the nbd server.
> >>>
> >>> Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
> >>> that probably explains the problem that we've been having.
> >>
> >> Sorry for this mistake.
> > 
> > OK, so this is working for me (with the 7.2 world).   What isn't working in this setup is migrate -b, I get:
> > 
> > (qemu) Receiving block device images
> > Error unknown block device disk1
> > qemu-system-x86_64: error while loading state section id 1(block)
> > qemu-system-x86_64: load of migration failed: Invalid argument
> > 
> > Can you explain the id=disk1 on the master side?
> 
> Can you give me the command line? I think the id in primary and secondary qemu is not same.

Sure, primary:

./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
     -boot c -m 4096 -smp 4 -S \
     -name debug-threads=on -trace events=trace-file \
     -netdev tap,id=hn0,script=$PWD/ifup-prim,\
downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
     -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
     -device virtio-rng-pci \
     -drive if=virtio,id=disk1,driver=quorum,read-pattern=fifo,\
cache=none,aio=native,\
children.0.file.filename=./bugzilla.raw,\
children.0.driver=raw

Sure, secondary:

TMPDISKS=/run
qemu-img create -f qcow2 $TMPDISKS/colo-hidden-disk.qcow2 40G
qemu-img create -f qcow2 $TMPDISKS/colo-active-disk.qcow2 40G

./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
     -boot c -m 4096 -smp 4 -S \
     -name debug-threads=on -trace events=trace-file \
     -netdev tap,id=hn0,script=$PWD/ifup-slave,\
downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
     -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
     -device virtio-rng-pci \
     -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
     -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
file.driver=qcow2,\
file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
file.backing.driver=qcow2,\
file.backing.backing.backing_reference=colo1,\
file.backing.allow-write-backing-file=on \
     -incoming tcp:0:8888


Secondary:
   nbd_server_start :8889
   nbd_server_add -w colo1

primary:

child_add disk1 child.driver=replication,child.mode=primary,child.file.host=ibpair,child.file.port=8889,child.file.export=colo1,child.file.driver=nbd,child.ignore-errors=on
(qemu) migrate_set_capability colo on
(qemu) migrate -d -b tcp:ibpair:8888


note the 'id=disk1' in the primary is shown in your wiki.

Dave

 Thanks
> Wen Congyang
> 
> > 
> > Dave
> > 
> >>
> >> Thanks
> >> Wen Congyang
> >>
> >>>
> >>> Dave
> >>>
> >>>>
> >>>> Thanks
> >>>> Wen Congyang
> >>>>
> >>>>>
> >>>>> - Michael
> >>>>>
> >>>>> .
> >>>>>
> >>>>
> >>>>
> >>> --
> >>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >>> .
> >>>
> >>
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> > .
> > 
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

On 07/09/2015 06:37 PM, Dr. David Alan Gilbert wrote:
> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>> On 07/09/2015 05:16 PM, Dr. David Alan Gilbert wrote:
>>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>>>
>>>>>> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
>>>>>> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
>>>>>> as a quorum child when the nbd server is ready.
>>>>>>
>>>>>> The nbd server is ready after you run the following command:
>>>>>> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
>>>>>> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
>>>>>>                         # -w means we allow to write to this disk.
>>>>>>
>>>>>> Then you can run the following command in the primary qemu:
>>>>>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
>>>>>>
>>>>>> After this monitor command, nbd client has connected to the nbd server.
>>>>>
>>>>> Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
>>>>> that probably explains the problem that we've been having.
>>>>
>>>> Sorry for this mistake.
>>>
>>> OK, so this is working for me (with the 7.2 world).   What isn't working in this setup is migrate -b, I get:
>>>
>>> (qemu) Receiving block device images
>>> Error unknown block device disk1
>>> qemu-system-x86_64: error while loading state section id 1(block)
>>> qemu-system-x86_64: load of migration failed: Invalid argument
>>>
>>> Can you explain the id=disk1 on the master side?
>>
>> Can you give me the command line? I think the id in primary and secondary qemu is not same.
> 
> Sure, primary:
> 
> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>      -boot c -m 4096 -smp 4 -S \
>      -name debug-threads=on -trace events=trace-file \
>      -netdev tap,id=hn0,script=$PWD/ifup-prim,\
> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
>      -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>      -device virtio-rng-pci \
>      -drive if=virtio,id=disk1,driver=quorum,read-pattern=fifo,\

If you want to use block migration, set the id to colo1

> cache=none,aio=native,\
> children.0.file.filename=./bugzilla.raw,\
> children.0.driver=raw
> 
> Sure, secondary:
> 
> TMPDISKS=/run
> qemu-img create -f qcow2 $TMPDISKS/colo-hidden-disk.qcow2 40G
> qemu-img create -f qcow2 $TMPDISKS/colo-active-disk.qcow2 40G
> 
> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>      -boot c -m 4096 -smp 4 -S \
>      -name debug-threads=on -trace events=trace-file \
>      -netdev tap,id=hn0,script=$PWD/ifup-slave,\
> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
>      -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>      -device virtio-rng-pci \
>      -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
>      -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
> file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
> file.driver=qcow2,\
> file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
> file.backing.driver=qcow2,\
> file.backing.backing.backing_reference=colo1,\
> file.backing.allow-write-backing-file=on \
>      -incoming tcp:0:8888
> 
> 
> Secondary:
>    nbd_server_start :8889
>    nbd_server_add -w colo1
> 
> primary:
> 
> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=ibpair,child.file.port=8889,child.file.export=colo1,child.file.driver=nbd,child.ignore-errors=on

here, it is child_add colo1 ....

> (qemu) migrate_set_capability colo on
> (qemu) migrate -d -b tcp:ibpair:8888
> 
> 
> note the 'id=disk1' in the primary is shown in your wiki.

IIRC, the default id is diskx, x is 1, 2, 3...

Thanks
Wen Congyang

> 
> Dave
> 
>  Thanks
>> Wen Congyang
>>
>>>
>>> Dave
>>>
>>>>
>>>> Thanks
>>>> Wen Congyang
>>>>
>>>>>
>>>>> Dave
>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Wen Congyang
>>>>>>
>>>>>>>
>>>>>>> - Michael
>>>>>>>
>>>>>>> .
>>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>>>> .
>>>>>
>>>>
>>> --
>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>> .
>>>
>>
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> .
> 

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Dr. David Alan Gilbert]

* Wen Congyang (wency@cn.fujitsu.com) wrote:
> On 07/09/2015 06:37 PM, Dr. David Alan Gilbert wrote:
> > * Wen Congyang (wency@cn.fujitsu.com) wrote:
> >> On 07/09/2015 05:16 PM, Dr. David Alan Gilbert wrote:
> >>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
> >>>
> >>>>>> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
> >>>>>> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
> >>>>>> as a quorum child when the nbd server is ready.
> >>>>>>
> >>>>>> The nbd server is ready after you run the following command:
> >>>>>> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
> >>>>>> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
> >>>>>>                         # -w means we allow to write to this disk.
> >>>>>>
> >>>>>> Then you can run the following command in the primary qemu:
> >>>>>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
> >>>>>>
> >>>>>> After this monitor command, nbd client has connected to the nbd server.
> >>>>>
> >>>>> Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
> >>>>> that probably explains the problem that we've been having.
> >>>>
> >>>> Sorry for this mistake.
> >>>
> >>> OK, so this is working for me (with the 7.2 world).   What isn't working in this setup is migrate -b, I get:
> >>>
> >>> (qemu) Receiving block device images
> >>> Error unknown block device disk1
> >>> qemu-system-x86_64: error while loading state section id 1(block)
> >>> qemu-system-x86_64: load of migration failed: Invalid argument
> >>>
> >>> Can you explain the id=disk1 on the master side?
> >>
> >> Can you give me the command line? I think the id in primary and secondary qemu is not same.
> > 
> > Sure, primary:
> > 
> > ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
> >      -boot c -m 4096 -smp 4 -S \
> >      -name debug-threads=on -trace events=trace-file \
> >      -netdev tap,id=hn0,script=$PWD/ifup-prim,\
> > downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
> >      -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
> >      -device virtio-rng-pci \
> >      -drive if=virtio,id=disk1,driver=quorum,read-pattern=fifo,\
> 
> If you want to use block migration, set the id to colo1

OK, does it make sense to always make it colo1 rather than having the 'disk1' and 'colo1' names?

> > cache=none,aio=native,\
> > children.0.file.filename=./bugzilla.raw,\
> > children.0.driver=raw
> > 
> > Sure, secondary:
> > 
> > TMPDISKS=/run
> > qemu-img create -f qcow2 $TMPDISKS/colo-hidden-disk.qcow2 40G
> > qemu-img create -f qcow2 $TMPDISKS/colo-active-disk.qcow2 40G
> > 
> > ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
> >      -boot c -m 4096 -smp 4 -S \
> >      -name debug-threads=on -trace events=trace-file \
> >      -netdev tap,id=hn0,script=$PWD/ifup-slave,\
> > downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
> >      -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
> >      -device virtio-rng-pci \
> >      -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
> >      -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
> > file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
> > file.driver=qcow2,\
> > file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
> > file.backing.driver=qcow2,\
> > file.backing.backing.backing_reference=colo1,\
> > file.backing.allow-write-backing-file=on \
> >      -incoming tcp:0:8888
> > 
> > 
> > Secondary:
> >    nbd_server_start :8889
> >    nbd_server_add -w colo1
> > 
> > primary:
> > 
> > child_add disk1 child.driver=replication,child.mode=primary,child.file.host=ibpair,child.file.port=8889,child.file.export=colo1,child.file.driver=nbd,child.ignore-errors=on
> 
> here, it is child_add colo1 ....
> 
> > (qemu) migrate_set_capability colo on
> > (qemu) migrate -d -b tcp:ibpair:8888
> > 
> > 
> > note the 'id=disk1' in the primary is shown in your wiki.
> 
> IIRC, the default id is diskx, x is 1, 2, 3...

OK, using the colo1 in both those places that works (this is in the 7.2 world)
and the RAM disk size now doesn't grow in the -b block migrate.

Dave

> 
> Thanks
> Wen Congyang
> 
> > 
> > Dave
> > 
> >  Thanks
> >> Wen Congyang
> >>
> >>>
> >>> Dave
> >>>
> >>>>
> >>>> Thanks
> >>>> Wen Congyang
> >>>>
> >>>>>
> >>>>> Dave
> >>>>>
> >>>>>>
> >>>>>> Thanks
> >>>>>> Wen Congyang
> >>>>>>
> >>>>>>>
> >>>>>>> - Michael
> >>>>>>>
> >>>>>>> .
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>> --
> >>>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >>>>> .
> >>>>>
> >>>>
> >>> --
> >>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >>> .
> >>>
> >>
> >>
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> > .
> > 
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

[Wen Congyang]

At 2015/7/9 21:40, Dr. David Alan Gilbert Wrote:
> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>> On 07/09/2015 06:37 PM, Dr. David Alan Gilbert wrote:
>>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>>>> On 07/09/2015 05:16 PM, Dr. David Alan Gilbert wrote:
>>>>> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>>>>>
>>>>>>>> I have sent the v8. But the usage is not changed. You can setup the environment according to the wiki.
>>>>>>>> When we open nbd client, we need to connect to the nbd server. So I introduce a new command child_add to add NBD client
>>>>>>>> as a quorum child when the nbd server is ready.
>>>>>>>>
>>>>>>>> The nbd server is ready after you run the following command:
>>>>>>>> nbd_server_start 0:6262 # the secondary qemu will listen to host:port
>>>>>>>> nbd_server_add -w mc1   # the NBD server will know this disk is used as NBD server. The export name is its id wc1.
>>>>>>>>                          # -w means we allow to write to this disk.
>>>>>>>>
>>>>>>>> Then you can run the following command in the primary qemu:
>>>>>>>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=127.0.0.1,child.file.port=6262,child.file.export=mc1,child.file.driver=nbd,child.ignore-errors=on
>>>>>>>>
>>>>>>>> After this monitor command, nbd client has connected to the nbd server.
>>>>>>>
>>>>>>> Ah! The 'child.file.export=mc1' wasn't there previously; I see Yang added that to the wiki yesterday;
>>>>>>> that probably explains the problem that we've been having.
>>>>>>
>>>>>> Sorry for this mistake.
>>>>>
>>>>> OK, so this is working for me (with the 7.2 world).   What isn't working in this setup is migrate -b, I get:
>>>>>
>>>>> (qemu) Receiving block device images
>>>>> Error unknown block device disk1
>>>>> qemu-system-x86_64: error while loading state section id 1(block)
>>>>> qemu-system-x86_64: load of migration failed: Invalid argument
>>>>>
>>>>> Can you explain the id=disk1 on the master side?
>>>>
>>>> Can you give me the command line? I think the id in primary and secondary qemu is not same.
>>>
>>> Sure, primary:
>>>
>>> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>>>       -boot c -m 4096 -smp 4 -S \
>>>       -name debug-threads=on -trace events=trace-file \
>>>       -netdev tap,id=hn0,script=$PWD/ifup-prim,\
>>> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
>>>       -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>>>       -device virtio-rng-pci \
>>>       -drive if=virtio,id=disk1,driver=quorum,read-pattern=fifo,\
>>
>> If you want to use block migration, set the id to colo1
>
> OK, does it make sense to always make it colo1 rather than having the 'disk1' and 'colo1' names?

You can use anyone. The export name must be the secondary disk's id.

Thanks
Wen Congyang

>
>>> cache=none,aio=native,\
>>> children.0.file.filename=./bugzilla.raw,\
>>> children.0.driver=raw
>>>
>>> Sure, secondary:
>>>
>>> TMPDISKS=/run
>>> qemu-img create -f qcow2 $TMPDISKS/colo-hidden-disk.qcow2 40G
>>> qemu-img create -f qcow2 $TMPDISKS/colo-active-disk.qcow2 40G
>>>
>>> ./try/bin/qemu-system-x86_64 -enable-kvm -nographic \
>>>       -boot c -m 4096 -smp 4 -S \
>>>       -name debug-threads=on -trace events=trace-file \
>>>       -netdev tap,id=hn0,script=$PWD/ifup-slave,\
>>> downscript=no,colo_script=$PWD/qemu/scripts/colo-proxy-script.sh,colo_nicname=em4 \
>>>       -device e1000,mac=9c:da:4d:1c:b5:89,id=net-pci0,netdev=hn0 \
>>>       -device virtio-rng-pci \
>>>       -drive if=none,driver=raw,file=/home/localvms/bugzilla.raw,id=colo1,cache=none,aio=native \
>>>       -drive if=virtio,driver=replication,mode=secondary,export=colo1,throttling.bps-total-max=70000000,\
>>> file.file.filename=$TMPDISKS/colo-active-disk.qcow2,\
>>> file.driver=qcow2,\
>>> file.backing.file.filename=$TMPDISKS/colo-hidden-disk.qcow2,\
>>> file.backing.driver=qcow2,\
>>> file.backing.backing.backing_reference=colo1,\
>>> file.backing.allow-write-backing-file=on \
>>>       -incoming tcp:0:8888
>>>
>>>
>>> Secondary:
>>>     nbd_server_start :8889
>>>     nbd_server_add -w colo1
>>>
>>> primary:
>>>
>>> child_add disk1 child.driver=replication,child.mode=primary,child.file.host=ibpair,child.file.port=8889,child.file.export=colo1,child.file.driver=nbd,child.ignore-errors=on
>>
>> here, it is child_add colo1 ....
>>
>>> (qemu) migrate_set_capability colo on
>>> (qemu) migrate -d -b tcp:ibpair:8888
>>>
>>>
>>> note the 'id=disk1' in the primary is shown in your wiki.
>>
>> IIRC, the default id is diskx, x is 1, 2, 3...
>
> OK, using the colo1 in both those places that works (this is in the 7.2 world)
> and the RAM disk size now doesn't grow in the -b block migrate.
>
> Dave
>
>>
>> Thanks
>> Wen Congyang
>>
>>>
>>> Dave
>>>
>>>   Thanks
>>>> Wen Congyang
>>>>
>>>>>
>>>>> Dave
>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Wen Congyang
>>>>>>
>>>>>>>
>>>>>>> Dave
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Wen Congyang
>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Michael
>>>>>>>>>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>>>>>> .
>>>>>>>
>>>>>>
>>>>> --
>>>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>>>> .
>>>>>
>>>>
>>>>
>>> --
>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>> .
>>>
>>
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>