* [Qemu-devel] [PATCH 1/6] COLO-compare: Add new parameter to communicate with remote colo-frame
2019-06-02 3:42 [Qemu-devel] [PATCH 0/6] Add Xen COLO support Zhang Chen
@ 2019-06-02 3:42 ` Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 2/6] COLO-compare: Add remote notification chardev handler frame Zhang Chen
` (4 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Zhang Chen @ 2019-06-02 3:42 UTC (permalink / raw)
To: Li Zhijian, Dr. David Alan Gilbert, Juan Quintela, zhanghailiang,
Jason Wang, qemu-dev
Cc: Zhang Chen, Zhang Chen
From: Zhang Chen <chen.zhang@intel.com>
We add the "notify_dev=chardevID" parameter. After that colo-compare can connect with
remote(currently just for Xen, KVM-COLO didn't need it.) colo-frame through chardev socket,
it can notify remote(Xen) colo-frame to handle checkpoint event.
Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
net/colo-compare.c | 21 +++++++++++++++++++++
qemu-options.hx | 33 ++++++++++++++++++++++++++++++++-
2 files changed, 53 insertions(+), 1 deletion(-)
diff --git a/net/colo-compare.c b/net/colo-compare.c
index fcb491121b..21849748b6 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -83,6 +83,7 @@ typedef struct CompareState {
char *pri_indev;
char *sec_indev;
char *outdev;
+ char *notify_dev;
CharBackend chr_pri_in;
CharBackend chr_sec_in;
CharBackend chr_out;
@@ -897,6 +898,21 @@ static void compare_set_vnet_hdr(Object *obj,
s->vnet_hdr = value;
}
+static char *compare_get_notify_dev(Object *obj, Error **errp)
+{
+ CompareState *s = COLO_COMPARE(obj);
+
+ return g_strdup(s->notify_dev);
+}
+
+static void compare_set_notify_dev(Object *obj, const char *value, Error **errp)
+{
+ CompareState *s = COLO_COMPARE(obj);
+
+ g_free(s->notify_dev);
+ s->notify_dev = g_strdup(value);
+}
+
static void compare_pri_rs_finalize(SocketReadState *pri_rs)
{
CompareState *s = container_of(pri_rs, CompareState, pri_rs);
@@ -1057,6 +1073,10 @@ static void colo_compare_init(Object *obj)
(Object **)&s->iothread,
object_property_allow_set_link,
OBJ_PROP_LINK_STRONG, NULL);
+ /* This parameter just for Xen COLO */
+ object_property_add_str(obj, "notify_dev",
+ compare_get_notify_dev, compare_set_notify_dev,
+ NULL);
s->vnet_hdr = false;
object_property_add_bool(obj, "vnet_hdr_support", compare_get_vnet_hdr,
@@ -1103,6 +1123,7 @@ static void colo_compare_finalize(Object *obj)
g_free(s->pri_indev);
g_free(s->sec_indev);
g_free(s->outdev);
+ g_free(s->notify_dev);
}
static const TypeInfo colo_compare_info = {
diff --git a/qemu-options.hx b/qemu-options.hx
index 7ae3373a00..30e1ca754f 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4475,7 +4475,7 @@ Dump the network traffic on netdev @var{dev} to the file specified by
The file format is libpcap, so it can be analyzed with tools such as tcpdump
or Wireshark.
-@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},outdev=@var{chardevid},iothread=@var{id}[,vnet_hdr_support]
+@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},outdev=@var{chardevid},iothread=@var{id}[,vnet_hdr_support][,notify_dev=@var{id}]
Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with
secondary packet. If the packets are same, we will output primary
@@ -4484,11 +4484,15 @@ do checkpoint and send primary packet to outdev@var{chardevid}.
In order to improve efficiency, we need to put the task of comparison
in another thread. If it has the vnet_hdr_support flag, colo compare
will send/recv packet with vnet_hdr_len.
+If you want to use Xen COLO, will need the notify_dev to notify Xen
+colo-frame to do checkpoint.
we must use it with the help of filter-mirror and filter-redirector.
@example
+KVM COLO
+
primary:
-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
@@ -4512,6 +4516,33 @@ secondary:
-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
+
+Xen COLO
+
+primary:
+-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
+-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
+-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
+-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
+-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
+-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
+-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
+-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
+-chardev socket,id=notify_way,host=3.3.3.3,port=9009,server,nowait
+-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
+-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
+-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
+-object iothread,id=iothread1
+-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0,notify_dev=nofity_way,iothread=iothread1
+
+secondary:
+-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown
+-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
+-chardev socket,id=red0,host=3.3.3.3,port=9003
+-chardev socket,id=red1,host=3.3.3.3,port=9004
+-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
+-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
+
@end example
If you want to know the detail of above command line, you can read
--
2.17.GIT
^ permalink raw reply related [flat|nested] 9+ messages in thread* [Qemu-devel] [PATCH 2/6] COLO-compare: Add remote notification chardev handler frame
2019-06-02 3:42 [Qemu-devel] [PATCH 0/6] Add Xen COLO support Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 1/6] COLO-compare: Add new parameter to communicate with remote colo-frame Zhang Chen
@ 2019-06-02 3:42 ` Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 3/6] COLO-compare: Make the compare_chr_send() can send notification message Zhang Chen
` (3 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Zhang Chen @ 2019-06-02 3:42 UTC (permalink / raw)
To: Li Zhijian, Dr. David Alan Gilbert, Juan Quintela, zhanghailiang,
Jason Wang, qemu-dev
Cc: Zhang Chen, Zhang Chen
From: Zhang Chen <chen.zhang@intel.com>
Add chardev handler to send notification to remote(current from Xen) colo-frame.
Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
net/colo-compare.c | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 21849748b6..8b7bd9f0c7 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -87,8 +87,10 @@ typedef struct CompareState {
CharBackend chr_pri_in;
CharBackend chr_sec_in;
CharBackend chr_out;
+ CharBackend chr_notify_dev;
SocketReadState pri_rs;
SocketReadState sec_rs;
+ SocketReadState notify_rs;
bool vnet_hdr;
/*
@@ -745,6 +747,19 @@ static void compare_sec_chr_in(void *opaque, const uint8_t *buf, int size)
}
}
+static void compare_notify_chr(void *opaque, const uint8_t *buf, int size)
+{
+ CompareState *s = COLO_COMPARE(opaque);
+ int ret;
+
+ ret = net_fill_rstate(&s->notify_rs, buf, size);
+ if (ret == -1) {
+ qemu_chr_fe_set_handlers(&s->chr_notify_dev, NULL, NULL, NULL, NULL,
+ NULL, NULL, true);
+ error_report("colo-compare notify_dev error");
+ }
+}
+
/*
* Check old packet regularly so it can watch for any packets
* that the secondary hasn't produced equivalents of.
@@ -832,6 +847,11 @@ static void colo_compare_iothread(CompareState *s)
qemu_chr_fe_set_handlers(&s->chr_sec_in, compare_chr_can_read,
compare_sec_chr_in, NULL, NULL,
s, s->worker_context, true);
+ if (s->notify_dev) {
+ qemu_chr_fe_set_handlers(&s->chr_notify_dev, compare_chr_can_read,
+ compare_notify_chr, NULL, NULL,
+ s, s->worker_context, true);
+ }
colo_compare_timer_init(s);
s->event_bh = qemu_bh_new(colo_compare_handle_event, s);
@@ -943,6 +963,10 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs)
}
}
+static void compare_notify_rs_finalize(SocketReadState *notify_rs)
+{
+ /* Get Xen colo-frame's notify and handle the message */
+}
/*
* Return 0 is success.
@@ -1013,6 +1037,17 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp)
net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, s->vnet_hdr);
net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, s->vnet_hdr);
+ /* Try to enable remote notify chardev, currently just for Xen COLO */
+ if (s->notify_dev) {
+ if (find_and_check_chardev(&chr, s->notify_dev, errp) ||
+ !qemu_chr_fe_init(&s->chr_notify_dev, chr, errp)) {
+ return;
+ }
+
+ net_socket_rs_init(&s->notify_rs, compare_notify_rs_finalize,
+ s->vnet_hdr);
+ }
+
QTAILQ_INSERT_TAIL(&net_compares, s, next);
g_queue_init(&s->conn_list);
@@ -1091,6 +1126,10 @@ static void colo_compare_finalize(Object *obj)
qemu_chr_fe_deinit(&s->chr_pri_in, false);
qemu_chr_fe_deinit(&s->chr_sec_in, false);
qemu_chr_fe_deinit(&s->chr_out, false);
+ if (s->notify_dev) {
+ qemu_chr_fe_deinit(&s->chr_notify_dev, false);
+ }
+
if (s->iothread) {
colo_compare_timer_del(s);
}
--
2.17.GIT
^ permalink raw reply related [flat|nested] 9+ messages in thread* [Qemu-devel] [PATCH 3/6] COLO-compare: Make the compare_chr_send() can send notification message.
2019-06-02 3:42 [Qemu-devel] [PATCH 0/6] Add Xen COLO support Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 1/6] COLO-compare: Add new parameter to communicate with remote colo-frame Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 2/6] COLO-compare: Add remote notification chardev handler frame Zhang Chen
@ 2019-06-02 3:42 ` Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 4/6] COLO-compare: Add colo-compare remote notify support Zhang Chen
` (2 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Zhang Chen @ 2019-06-02 3:42 UTC (permalink / raw)
To: Li Zhijian, Dr. David Alan Gilbert, Juan Quintela, zhanghailiang,
Jason Wang, qemu-dev
Cc: Zhang Chen, Zhang Chen
From: Zhang Chen <chen.zhang@intel.com>
We need use this function to send notification message for remote colo-frame(Xen).
So we add new parameter for this job.
Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
net/colo-compare.c | 41 +++++++++++++++++++++++++++++++++--------
1 file changed, 33 insertions(+), 8 deletions(-)
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 8b7bd9f0c7..16285f4a96 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -129,7 +129,8 @@ static void colo_compare_inconsistency_notify(void)
static int compare_chr_send(CompareState *s,
const uint8_t *buf,
uint32_t size,
- uint32_t vnet_hdr_len);
+ uint32_t vnet_hdr_len,
+ bool notify_remote_frame);
static gint seq_sorter(Packet *a, Packet *b, gpointer data)
{
@@ -241,7 +242,8 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt)
ret = compare_chr_send(s,
pkt->data,
pkt->size,
- pkt->vnet_hdr_len);
+ pkt->vnet_hdr_len,
+ false);
if (ret < 0) {
error_report("colo send primary packet failed");
}
@@ -671,7 +673,8 @@ static void colo_compare_connection(void *opaque, void *user_data)
static int compare_chr_send(CompareState *s,
const uint8_t *buf,
uint32_t size,
- uint32_t vnet_hdr_len)
+ uint32_t vnet_hdr_len,
+ bool notify_remote_frame)
{
int ret = 0;
uint32_t len = htonl(size);
@@ -680,7 +683,14 @@ static int compare_chr_send(CompareState *s,
return 0;
}
- ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len));
+ if (notify_remote_frame) {
+ ret = qemu_chr_fe_write_all(&s->chr_notify_dev,
+ (uint8_t *)&len,
+ sizeof(len));
+ } else {
+ ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len));
+ }
+
if (ret != sizeof(len)) {
goto err;
}
@@ -691,13 +701,26 @@ static int compare_chr_send(CompareState *s,
* know how to parse net packet correctly.
*/
len = htonl(vnet_hdr_len);
- ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len));
+
+ if (!notify_remote_frame) {
+ ret = qemu_chr_fe_write_all(&s->chr_out,
+ (uint8_t *)&len,
+ sizeof(len));
+ }
+
if (ret != sizeof(len)) {
goto err;
}
}
- ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size);
+ if (notify_remote_frame) {
+ ret = qemu_chr_fe_write_all(&s->chr_notify_dev,
+ (uint8_t *)buf,
+ size);
+ } else {
+ ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size);
+ }
+
if (ret != size) {
goto err;
}
@@ -943,7 +966,8 @@ static void compare_pri_rs_finalize(SocketReadState *pri_rs)
compare_chr_send(s,
pri_rs->buf,
pri_rs->packet_len,
- pri_rs->vnet_hdr_len);
+ pri_rs->vnet_hdr_len,
+ false);
} else {
/* compare packet in the specified connection */
colo_compare_connection(conn, s);
@@ -1075,7 +1099,8 @@ static void colo_flush_packets(void *opaque, void *user_data)
compare_chr_send(s,
pkt->data,
pkt->size,
- pkt->vnet_hdr_len);
+ pkt->vnet_hdr_len,
+ false);
packet_destroy(pkt, NULL);
}
while (!g_queue_is_empty(&conn->secondary_list)) {
--
2.17.GIT
^ permalink raw reply related [flat|nested] 9+ messages in thread* [Qemu-devel] [PATCH 4/6] COLO-compare: Add colo-compare remote notify support
2019-06-02 3:42 [Qemu-devel] [PATCH 0/6] Add Xen COLO support Zhang Chen
` (2 preceding siblings ...)
2019-06-02 3:42 ` [Qemu-devel] [PATCH 3/6] COLO-compare: Make the compare_chr_send() can send notification message Zhang Chen
@ 2019-06-02 3:42 ` Zhang Chen
2019-06-03 2:33 ` Li Zhijian
2019-06-02 3:42 ` [Qemu-devel] [PATCH 5/6] COLO-compare: Rename the colo_compare_inconsistency_notify Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 6/6] migration/colo.c: Add missed filter notify for Xen COLO Zhang Chen
5 siblings, 1 reply; 9+ messages in thread
From: Zhang Chen @ 2019-06-02 3:42 UTC (permalink / raw)
To: Li Zhijian, Dr. David Alan Gilbert, Juan Quintela, zhanghailiang,
Jason Wang, qemu-dev
Cc: Zhang Chen, Zhang Chen
From: Zhang Chen <chen.zhang@intel.com>
This patch make colo-compare can send message to remote COLO frame(Xen) when occur checkpoint.
Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
net/colo-compare.c | 51 +++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 46 insertions(+), 5 deletions(-)
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 16285f4a96..19075c7a66 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -251,6 +251,17 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt)
packet_destroy(pkt, NULL);
}
+static void notify_remote_frame(CompareState *s)
+{
+ char msg[] = "DO_CHECKPOINT";
+ int ret = 0;
+
+ ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true);
+ if (ret < 0) {
+ error_report("Notify Xen COLO-frame failed");
+ }
+}
+
/*
* The IP packets sent by primary and secondary
* will be compared in here
@@ -435,7 +446,11 @@ sec:
qemu_hexdump((char *)spkt->data, stderr,
"colo-compare spkt", spkt->size);
- colo_compare_inconsistency_notify();
+ if (s->notify_dev) {
+ notify_remote_frame(s);
+ } else {
+ colo_compare_inconsistency_notify();
+ }
}
}
@@ -577,7 +592,7 @@ void colo_compare_unregister_notifier(Notifier *notify)
}
static int colo_old_packet_check_one_conn(Connection *conn,
- void *user_data)
+ CompareState *s)
{
GList *result = NULL;
int64_t check_time = REGULAR_PACKET_CHECK_MS;
@@ -588,7 +603,11 @@ static int colo_old_packet_check_one_conn(Connection *conn,
if (result) {
/* Do checkpoint will flush old packet */
- colo_compare_inconsistency_notify();
+ if (s->notify_dev) {
+ notify_remote_frame(s);
+ } else {
+ colo_compare_inconsistency_notify();
+ }
return 0;
}
@@ -608,7 +627,7 @@ static void colo_old_packet_check(void *opaque)
* If we find one old packet, stop finding job and notify
* COLO frame do checkpoint.
*/
- g_queue_find_custom(&s->conn_list, NULL,
+ g_queue_find_custom(&s->conn_list, s,
(GCompareFunc)colo_old_packet_check_one_conn);
}
@@ -637,7 +656,12 @@ static void colo_compare_packet(CompareState *s, Connection *conn,
*/
trace_colo_compare_main("packet different");
g_queue_push_head(&conn->primary_list, pkt);
- colo_compare_inconsistency_notify();
+
+ if (s->notify_dev) {
+ notify_remote_frame(s);
+ } else {
+ colo_compare_inconsistency_notify();
+ }
break;
}
}
@@ -989,7 +1013,24 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs)
static void compare_notify_rs_finalize(SocketReadState *notify_rs)
{
+ CompareState *s = container_of(notify_rs, CompareState, notify_rs);
+
/* Get Xen colo-frame's notify and handle the message */
+ char *data = g_memdup(notify_rs->buf, notify_rs->packet_len);
+ char msg[] = "COLO_COMPARE_GET_XEN_INIT";
+ int ret;
+
+ if (!strcmp(data, "COLO_USERSPACE_PROXY_INIT")) {
+ ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true);
+ if (ret < 0) {
+ error_report("Notify Xen COLO-frame INIT failed");
+ }
+ }
+
+ if (!strcmp(data, "COLO_CHECKPOINT")) {
+ /* colo-compare do checkpoint, flush pri packet and remove sec packet */
+ g_queue_foreach(&s->conn_list, colo_flush_packets, s);
+ }
}
/*
--
2.17.GIT
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [Qemu-devel] [PATCH 4/6] COLO-compare: Add colo-compare remote notify support
2019-06-02 3:42 ` [Qemu-devel] [PATCH 4/6] COLO-compare: Add colo-compare remote notify support Zhang Chen
@ 2019-06-03 2:33 ` Li Zhijian
2019-06-03 7:43 ` Zhang, Chen
0 siblings, 1 reply; 9+ messages in thread
From: Li Zhijian @ 2019-06-03 2:33 UTC (permalink / raw)
To: Zhang Chen, Dr. David Alan Gilbert, Juan Quintela, zhanghailiang,
Jason Wang, qemu-dev
Cc: Zhang Chen
how about do switch inside colo_compare_inconsistency_notify(), like:
colo_compare_inconsistency_notify(CompareState *s)
{
if (s->remote_notify)
remote_notify
else
local_notity
}
Thanks
Zhijian
On 6/2/19 11:42 AM, Zhang Chen wrote:
> From: Zhang Chen <chen.zhang@intel.com>
>
> This patch make colo-compare can send message to remote COLO frame(Xen) when occur checkpoint.
>
> Signed-off-by: Zhang Chen <chen.zhang@intel.com>
> ---
> net/colo-compare.c | 51 +++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 46 insertions(+), 5 deletions(-)
>
> diff --git a/net/colo-compare.c b/net/colo-compare.c
> index 16285f4a96..19075c7a66 100644
> --- a/net/colo-compare.c
> +++ b/net/colo-compare.c
> @@ -251,6 +251,17 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt)
> packet_destroy(pkt, NULL);
> }
>
> +static void notify_remote_frame(CompareState *s)
> +{
> + char msg[] = "DO_CHECKPOINT";
> + int ret = 0;
> +
> + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true);
> + if (ret < 0) {
> + error_report("Notify Xen COLO-frame failed");
> + }
> +}
> +
> /*
> * The IP packets sent by primary and secondary
> * will be compared in here
> @@ -435,7 +446,11 @@ sec:
> qemu_hexdump((char *)spkt->data, stderr,
> "colo-compare spkt", spkt->size);
>
> - colo_compare_inconsistency_notify();
> + if (s->notify_dev) {
> + notify_remote_frame(s);
> + } else {
> + colo_compare_inconsistency_notify();
> + }
> }
> }
>
> @@ -577,7 +592,7 @@ void colo_compare_unregister_notifier(Notifier *notify)
> }
>
> static int colo_old_packet_check_one_conn(Connection *conn,
> - void *user_data)
> + CompareState *s)
> {
> GList *result = NULL;
> int64_t check_time = REGULAR_PACKET_CHECK_MS;
> @@ -588,7 +603,11 @@ static int colo_old_packet_check_one_conn(Connection *conn,
>
> if (result) {
> /* Do checkpoint will flush old packet */
> - colo_compare_inconsistency_notify();
> + if (s->notify_dev) {
> + notify_remote_frame(s);
> + } else {
> + colo_compare_inconsistency_notify();
> + }
> return 0;
> }
>
> @@ -608,7 +627,7 @@ static void colo_old_packet_check(void *opaque)
> * If we find one old packet, stop finding job and notify
> * COLO frame do checkpoint.
> */
> - g_queue_find_custom(&s->conn_list, NULL,
> + g_queue_find_custom(&s->conn_list, s,
> (GCompareFunc)colo_old_packet_check_one_conn);
> }
>
> @@ -637,7 +656,12 @@ static void colo_compare_packet(CompareState *s, Connection *conn,
> */
> trace_colo_compare_main("packet different");
> g_queue_push_head(&conn->primary_list, pkt);
> - colo_compare_inconsistency_notify();
> +
> + if (s->notify_dev) {
> + notify_remote_frame(s);
> + } else {
> + colo_compare_inconsistency_notify();
> + }
> break;
> }
> }
> @@ -989,7 +1013,24 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs)
>
> static void compare_notify_rs_finalize(SocketReadState *notify_rs)
> {
> + CompareState *s = container_of(notify_rs, CompareState, notify_rs);
> +
> /* Get Xen colo-frame's notify and handle the message */
> + char *data = g_memdup(notify_rs->buf, notify_rs->packet_len);
> + char msg[] = "COLO_COMPARE_GET_XEN_INIT";
> + int ret;
> +
> + if (!strcmp(data, "COLO_USERSPACE_PROXY_INIT")) {
> + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true);
> + if (ret < 0) {
> + error_report("Notify Xen COLO-frame INIT failed");
> + }
> + }
> +
> + if (!strcmp(data, "COLO_CHECKPOINT")) {
> + /* colo-compare do checkpoint, flush pri packet and remove sec packet */
> + g_queue_foreach(&s->conn_list, colo_flush_packets, s);
> + }
> }
>
> /*
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [Qemu-devel] [PATCH 4/6] COLO-compare: Add colo-compare remote notify support
2019-06-03 2:33 ` Li Zhijian
@ 2019-06-03 7:43 ` Zhang, Chen
0 siblings, 0 replies; 9+ messages in thread
From: Zhang, Chen @ 2019-06-03 7:43 UTC (permalink / raw)
To: 'Li Zhijian', Dr. David Alan Gilbert, Juan Quintela,
zhanghailiang, Jason Wang, qemu-dev
Cc: Zhang Chen
From: Li Zhijian [mailto:lizhijian@cn.fujitsu.com]
Sent: Monday, June 3, 2019 10:33 AM
To: Zhang, Chen <chen.zhang@intel.com>; Dr. David Alan Gilbert <dgilbert@redhat.com>; Juan Quintela <quintela@redhat.com>; zhanghailiang <zhang.zhanghailiang@huawei.com>; Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-devel@nongnu.org>
Cc: Zhang Chen <zhangckid@gmail.com>
Subject: Re: [PATCH 4/6] COLO-compare: Add colo-compare remote notify support
how about do switch inside colo_compare_inconsistency_notify(), like:
colo_compare_inconsistency_notify(CompareState *s)
{
if (s->remote_notify)
remote_notify
else
local_notity
}
Make sense. I will update it in next version.
Thank you Zhijian~~
Thanks
Zhang Chen
Thanks
Zhijian
On 6/2/19 11:42 AM, Zhang Chen wrote:
From: Zhang Chen <chen.zhang@intel.com><mailto:chen.zhang@intel.com>
This patch make colo-compare can send message to remote COLO frame(Xen) when occur checkpoint.
Signed-off-by: Zhang Chen <chen.zhang@intel.com><mailto:chen.zhang@intel.com>
---
net/colo-compare.c | 51 +++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 46 insertions(+), 5 deletions(-)
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 16285f4a96..19075c7a66 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -251,6 +251,17 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt)
packet_destroy(pkt, NULL);
}
+static void notify_remote_frame(CompareState *s)
+{
+ char msg[] = "DO_CHECKPOINT";
+ int ret = 0;
+
+ ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true);
+ if (ret < 0) {
+ error_report("Notify Xen COLO-frame failed");
+ }
+}
+
/*
* The IP packets sent by primary and secondary
* will be compared in here
@@ -435,7 +446,11 @@ sec:
qemu_hexdump((char *)spkt->data, stderr,
"colo-compare spkt", spkt->size);
- colo_compare_inconsistency_notify();
+ if (s->notify_dev) {
+ notify_remote_frame(s);
+ } else {
+ colo_compare_inconsistency_notify();
+ }
}
}
@@ -577,7 +592,7 @@ void colo_compare_unregister_notifier(Notifier *notify)
}
static int colo_old_packet_check_one_conn(Connection *conn,
- void *user_data)
+ CompareState *s)
{
GList *result = NULL;
int64_t check_time = REGULAR_PACKET_CHECK_MS;
@@ -588,7 +603,11 @@ static int colo_old_packet_check_one_conn(Connection *conn,
if (result) {
/* Do checkpoint will flush old packet */
- colo_compare_inconsistency_notify();
+ if (s->notify_dev) {
+ notify_remote_frame(s);
+ } else {
+ colo_compare_inconsistency_notify();
+ }
return 0;
}
@@ -608,7 +627,7 @@ static void colo_old_packet_check(void *opaque)
* If we find one old packet, stop finding job and notify
* COLO frame do checkpoint.
*/
- g_queue_find_custom(&s->conn_list, NULL,
+ g_queue_find_custom(&s->conn_list, s,
(GCompareFunc)colo_old_packet_check_one_conn);
}
@@ -637,7 +656,12 @@ static void colo_compare_packet(CompareState *s, Connection *conn,
*/
trace_colo_compare_main("packet different");
g_queue_push_head(&conn->primary_list, pkt);
- colo_compare_inconsistency_notify();
+
+ if (s->notify_dev) {
+ notify_remote_frame(s);
+ } else {
+ colo_compare_inconsistency_notify();
+ }
break;
}
}
@@ -989,7 +1013,24 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs)
static void compare_notify_rs_finalize(SocketReadState *notify_rs)
{
+ CompareState *s = container_of(notify_rs, CompareState, notify_rs);
+
/* Get Xen colo-frame's notify and handle the message */
+ char *data = g_memdup(notify_rs->buf, notify_rs->packet_len);
+ char msg[] = "COLO_COMPARE_GET_XEN_INIT";
+ int ret;
+
+ if (!strcmp(data, "COLO_USERSPACE_PROXY_INIT")) {
+ ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true);
+ if (ret < 0) {
+ error_report("Notify Xen COLO-frame INIT failed");
+ }
+ }
+
+ if (!strcmp(data, "COLO_CHECKPOINT")) {
+ /* colo-compare do checkpoint, flush pri packet and remove sec packet */
+ g_queue_foreach(&s->conn_list, colo_flush_packets, s);
+ }
}
/*
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Qemu-devel] [PATCH 5/6] COLO-compare: Rename the colo_compare_inconsistency_notify
2019-06-02 3:42 [Qemu-devel] [PATCH 0/6] Add Xen COLO support Zhang Chen
` (3 preceding siblings ...)
2019-06-02 3:42 ` [Qemu-devel] [PATCH 4/6] COLO-compare: Add colo-compare remote notify support Zhang Chen
@ 2019-06-02 3:42 ` Zhang Chen
2019-06-02 3:42 ` [Qemu-devel] [PATCH 6/6] migration/colo.c: Add missed filter notify for Xen COLO Zhang Chen
5 siblings, 0 replies; 9+ messages in thread
From: Zhang Chen @ 2019-06-02 3:42 UTC (permalink / raw)
To: Li Zhijian, Dr. David Alan Gilbert, Juan Quintela, zhanghailiang,
Jason Wang, qemu-dev
Cc: Zhang Chen, Zhang Chen
From: Zhang Chen <chen.zhang@intel.com>
We have add the notify_remote_frame function for Xen,
so we rename the colo_compare_inconsistency_notify to
notify_native_frame(KVM-qemu COLO frame) looks better.
Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
net/colo-compare.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/net/colo-compare.c b/net/colo-compare.c
index 19075c7a66..101c1f477e 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -120,7 +120,7 @@ enum {
SECONDARY_IN,
};
-static void colo_compare_inconsistency_notify(void)
+static void notify_native_frame(void)
{
notifier_list_notify(&colo_compare_notifiers,
migrate_get_current());
@@ -449,7 +449,7 @@ sec:
if (s->notify_dev) {
notify_remote_frame(s);
} else {
- colo_compare_inconsistency_notify();
+ notify_native_frame();
}
}
}
@@ -606,7 +606,7 @@ static int colo_old_packet_check_one_conn(Connection *conn,
if (s->notify_dev) {
notify_remote_frame(s);
} else {
- colo_compare_inconsistency_notify();
+ notify_native_frame();
}
return 0;
}
@@ -660,7 +660,7 @@ static void colo_compare_packet(CompareState *s, Connection *conn,
if (s->notify_dev) {
notify_remote_frame(s);
} else {
- colo_compare_inconsistency_notify();
+ notify_native_frame();
}
break;
}
--
2.17.GIT
^ permalink raw reply related [flat|nested] 9+ messages in thread* [Qemu-devel] [PATCH 6/6] migration/colo.c: Add missed filter notify for Xen COLO.
2019-06-02 3:42 [Qemu-devel] [PATCH 0/6] Add Xen COLO support Zhang Chen
` (4 preceding siblings ...)
2019-06-02 3:42 ` [Qemu-devel] [PATCH 5/6] COLO-compare: Rename the colo_compare_inconsistency_notify Zhang Chen
@ 2019-06-02 3:42 ` Zhang Chen
5 siblings, 0 replies; 9+ messages in thread
From: Zhang Chen @ 2019-06-02 3:42 UTC (permalink / raw)
To: Li Zhijian, Dr. David Alan Gilbert, Juan Quintela, zhanghailiang,
Jason Wang, qemu-dev
Cc: Zhang Chen, Zhang Chen
From: Zhang Chen <chen.zhang@intel.com>
We need to notify net filter to do checkpoint for Xen COLO, like KVM side.
Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
migration/colo.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/migration/colo.c b/migration/colo.c
index 8c1644091f..9f84b1fa3c 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -259,6 +259,8 @@ ReplicationStatus *qmp_query_xen_replication_status(Error **errp)
void qmp_xen_colo_do_checkpoint(Error **errp)
{
replication_do_checkpoint_all(errp);
+ /* Notify all filters of all NIC to do checkpoint */
+ colo_notify_filters_event(COLO_EVENT_CHECKPOINT, errp);
}
#endif
--
2.17.GIT
^ permalink raw reply related [flat|nested] 9+ messages in thread