* Error migrating VM to secondary host using COLO replication @ 2016-10-27 22:56 Sadi 2016-10-28 16:56 ` Konrad Rzeszutek Wilk 2016-11-03 6:33 ` Wen Congyang 0 siblings, 2 replies; 11+ messages in thread From: Sadi @ 2016-10-27 22:56 UTC (permalink / raw) To: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 1665 bytes --] Hello, I've been trying to set COLO replication to work but i'm stuck on a problem when migrating de primary VM to secondary host. I have been following the instructions from this wiki - http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping and this mail thread - http://xen.markmail.org/search/?q=COLO#query:COLO+page:1+mid:fb7wrn62vbks4unn+state:results I'm anexing the steps i took setting the environment before facing this problem when executing 'xl remus' command: >migration target: Ready to receive domain. >Saving to migration stream new xl format (info 0x3/0x0/2840) >Loading new save file <incoming migration stream> (new xl fmt info 0x3/0x0/2840) >Savefile contains xl domain config in JSON format >Parsing config from <saved> >xc: info: Saving domain 2, type x86 HVM >xc: info: Found x86 HVM domain from Xen 4.7 >xc: info: Restoring domain >xc: Frames iteration 0 of 5: 1045504/1045504 100% >xc: Domain now suspended: 0/0 0% >libxl: error: libxl_qmp.c:702:libxl__qmp_initialize: Connection error: No such file or directory >libxl: error: libxl_colo_restore.c:817:colo_restore_setup_cds_done: COLO: failed to setup device >for guest with domid 1 >xc: error: Restore failed (38 = Function not implemented): Internal error >libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo fails >libxl: error: libxl_stream_read.c:852:libxl__xc_domain_restore_done: restoring domain: Function >not implemented >libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo fails I'm hoping that someone could provide with directions. Thanks for your time and sory for bad english (not native language). Sadi. [-- Attachment #1.2: Type: text/html, Size: 2384 bytes --] [-- Attachment #2: steps.txt --] [-- Type: text/plain, Size: 3223 bytes --] Network master: br0 : 10.20.107.30 binded with eth0 eth1: 192.168.1.30 eth2: 192.168.2.30 slave: br0 eth0: 10.20.107.33 binded with eth0 br1: no ip address binded with eth1 eth1: 192.168.1.33 eth2: 192.168.2.33 Eth1 both sides directly connected by cable Eth2 both sides directly connected by cable Repositories used: https://github.com/Pating/colo-proxy/tree/changlox https://github.com/macrosheep/iptables.git https://github.com/torvalds/linux https://github.com/wencongyang/xen Kernel build instructions followed: 2. Prepare host kernel for Dom0 colo-proxy kernel module need cooperate with linux kernel. You should patch kernel with ~/colo-proxy/colo-patch-for-kernel.patch -cd ~/colo-proxy/; git checkout 405527cbfa9f -cd ~/linux/; git checkout v4.0; git am ~/colo-proxy/colo-patch-for-kernel.patch -cp /boot/config-3.0.76-0.11-xen .config; make menuconfig to config your kernel support Dom0. Ref: http://wiki.xenproject.org/wiki/Mainline_Linux_Kernel_Configs -make -j8; make modules_install; make install -reboot COLO-Proxy: -cd ~/colo-proxy/; git checkout 405527cbfa9f; make; make install IPTables: -cd iptables; ./autogen.sh; ./configure --prefix=/usr/ --libdir=/usr/lib64; make; make install XEN: -./autogen.sh -./configure --enable-debug -touch tools/libxl/libxlu_disk_l.l -touch tools/libxl/libxlu_cfg_l.l -make dist-xen -make dist-tools -make install-xen -make install-tools *i've tried with https://github.com/wencongyang/qemu-xen but got an error with qemu when xl creating the VM as follows: >libxl: error: libxl_qmp.c:287:qmp_handle_error_response: received an error message from QMP server: Could not set password Qemu -cd ~/qemu-xen/; git checkout colo-xen-v2 Configured QEMU with script provided at: http://xen.markmail.org/message/y4jcdqxw2s2labdo?q=COLO#query:COLO+page:1+mid:3lzcuzeokqsqpu4i+state:results *path_to_xen_source updated according my directory tree. then.. -make -make install Running COLO *HVM SUSE 64bits primary: rm -f /var/log/xen/* rm -f /var/lib/xen/userdata-d.* service xencommons start modprobe nf_conntrack_ipv4 modprobe xt_PMYCOLO sec_dev=eth1 secondary: rm -f /var/log/xen/* rm -f /var/lib/xen/userdata-d.* service xencommons start modprobe xt_SECCOLO active_disk=/mnt/ramfs/active_disk.img hidden_disk=/mnt/ramfs/hidden_disk.img local_img=/root/new/SUSE/xenguest.img tmp_disk_size=`/root/new/pating/qemu-xen/qemu-img info $local_img |grep 'virtual size' |awk '{print $3}'` rm -rf /mnt/ramfs/* umount /mnt/ramfs/ rm -rf /mnt/ramfs/ mkdir /mnt/ramfs function create_image() { /root/new/pating/qemu-xen/qemu-img create -f qcow2 $1 $tmp_disk_size } function prepare_temp_images() { grep -q "^none /mnt/ramfs ramfs" /proc/mounts if [[ $? -ne 0 ]]; then mount -t ramfs none /mnt/ramfs/ -o size=2G fi if [[ ! -e $active_disk ]]; then create_image $active_disk fi if [[ ! -e $hidden_disk ]]; then create_image $hidden_disk fi } prepare_temp_images primary: xl create new/SUSE/vm-suse.cfg xl pause vm-suse xl remus -c -u vm-suse 192.168.2.33 [-- Attachment #3: Type: text/plain, Size: 127 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-10-27 22:56 Error migrating VM to secondary host using COLO replication Sadi @ 2016-10-28 16:56 ` Konrad Rzeszutek Wilk 2016-11-03 6:23 ` Zhang Chen 2016-11-03 6:33 ` Wen Congyang 1 sibling, 1 reply; 11+ messages in thread From: Konrad Rzeszutek Wilk @ 2016-10-28 16:56 UTC (permalink / raw) To: Sadi, zhangchen.fnst; +Cc: xen-devel On Thu, Oct 27, 2016 at 08:56:34PM -0200, Sadi wrote: > Hello, Hey! CC-ing relevant people. > > I've been trying to set COLO replication to work but i'm stuck on a problem > when migrating de primary VM to secondary host. > > I have been following the instructions from this wiki > > - http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping > > and this mail thread > > - > http://xen.markmail.org/search/?q=COLO#query:COLO+page:1+mid:fb7wrn62vbks4unn+state:results > > I'm anexing the steps i took setting the environment before facing this > problem when executing 'xl remus' command: > > >migration target: Ready to receive domain. > >Saving to migration stream new xl format (info 0x3/0x0/2840) > >Loading new save file <incoming migration stream> (new xl fmt info > 0x3/0x0/2840) > >Savefile contains xl domain config in JSON format > >Parsing config from <saved> > >xc: info: Saving domain 2, type x86 HVM > >xc: info: Found x86 HVM domain from Xen 4.7 > >xc: info: Restoring domain > >xc: Frames iteration 0 of 5: 1045504/1045504 100% > >xc: Domain now suspended: 0/0 0% > >libxl: error: libxl_qmp.c:702:libxl__qmp_initialize: Connection error: No > such file or directory > >libxl: error: libxl_colo_restore.c:817:colo_restore_setup_cds_done: COLO: > failed to setup device >for guest with domid 1 > >xc: error: Restore failed (38 = Function not implemented): Internal error > >libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo > fails > >libxl: error: libxl_stream_read.c:852:libxl__xc_domain_restore_done: > restoring domain: Function >not implemented > >libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo > fails > > I'm hoping that someone could provide with directions. > > Thanks for your time and sory for bad english (not native language). > > > Sadi. > Network > > master: > br0 : 10.20.107.30 binded with eth0 > eth1: 192.168.1.30 > eth2: 192.168.2.30 > > slave: > br0 eth0: 10.20.107.33 binded with eth0 > br1: no ip address binded with eth1 > eth1: 192.168.1.33 > eth2: 192.168.2.33 > > Eth1 both sides directly connected by cable > Eth2 both sides directly connected by cable > > Repositories used: > > https://github.com/Pating/colo-proxy/tree/changlox > https://github.com/macrosheep/iptables.git > https://github.com/torvalds/linux > > https://github.com/wencongyang/xen > > Kernel build instructions followed: > > 2. Prepare host kernel for Dom0 > colo-proxy kernel module need cooperate with linux kernel. You should patch kernel with ~/colo-proxy/colo-patch-for-kernel.patch > -cd ~/colo-proxy/; git checkout 405527cbfa9f > -cd ~/linux/; git checkout v4.0; git am ~/colo-proxy/colo-patch-for-kernel.patch > -cp /boot/config-3.0.76-0.11-xen .config; make menuconfig to config your kernel support Dom0. Ref: http://wiki.xenproject.org/wiki/Mainline_Linux_Kernel_Configs > -make -j8; make modules_install; make install > -reboot > > > COLO-Proxy: > > -cd ~/colo-proxy/; git checkout 405527cbfa9f; make; make install > > IPTables: > > -cd iptables; ./autogen.sh; ./configure --prefix=/usr/ --libdir=/usr/lib64; make; make install > > XEN: > > -./autogen.sh > -./configure --enable-debug > -touch tools/libxl/libxlu_disk_l.l > -touch tools/libxl/libxlu_cfg_l.l > -make dist-xen > -make dist-tools > -make install-xen > -make install-tools > > *i've tried with https://github.com/wencongyang/qemu-xen but got an error with qemu when xl creating the VM as follows: > > >libxl: error: libxl_qmp.c:287:qmp_handle_error_response: received an error message from QMP server: Could not set password > > Qemu > > -cd ~/qemu-xen/; git checkout colo-xen-v2 > > Configured QEMU with script provided at: > http://xen.markmail.org/message/y4jcdqxw2s2labdo?q=COLO#query:COLO+page:1+mid:3lzcuzeokqsqpu4i+state:results > > *path_to_xen_source updated according my directory tree. > then.. > > -make > -make install > > Running COLO > > *HVM SUSE 64bits > > primary: > rm -f /var/log/xen/* > rm -f /var/lib/xen/userdata-d.* > service xencommons start > modprobe nf_conntrack_ipv4 > modprobe xt_PMYCOLO sec_dev=eth1 > > secondary: > rm -f /var/log/xen/* > rm -f /var/lib/xen/userdata-d.* > service xencommons start > modprobe xt_SECCOLO > active_disk=/mnt/ramfs/active_disk.img > hidden_disk=/mnt/ramfs/hidden_disk.img > local_img=/root/new/SUSE/xenguest.img > tmp_disk_size=`/root/new/pating/qemu-xen/qemu-img info $local_img |grep 'virtual size' |awk '{print $3}'` > rm -rf /mnt/ramfs/* > umount /mnt/ramfs/ > rm -rf /mnt/ramfs/ > mkdir /mnt/ramfs > > function create_image() > { > /root/new/pating/qemu-xen/qemu-img create -f qcow2 $1 $tmp_disk_size > } > function prepare_temp_images() > { > grep -q "^none /mnt/ramfs ramfs" /proc/mounts > if [[ $? -ne 0 ]]; then > mount -t ramfs none /mnt/ramfs/ -o size=2G > fi > > if [[ ! -e $active_disk ]]; then > create_image $active_disk > fi > > if [[ ! -e $hidden_disk ]]; then > create_image $hidden_disk > fi > } > prepare_temp_images > > primary: > > xl create new/SUSE/vm-suse.cfg > xl pause vm-suse > xl remus -c -u vm-suse 192.168.2.33 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-10-28 16:56 ` Konrad Rzeszutek Wilk @ 2016-11-03 6:23 ` Zhang Chen 0 siblings, 0 replies; 11+ messages in thread From: Zhang Chen @ 2016-11-03 6:23 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, Sadi Cc: Changlong Xie, imhy.yang, Wen Congyang, xen-devel On 10/29/2016 12:56 AM, Konrad Rzeszutek Wilk wrote: > On Thu, Oct 27, 2016 at 08:56:34PM -0200, Sadi wrote: >> Hello, > Hey! > > CC-ing relevant people. CC Xie Changlong, Wen Congyang and Yang Hongyang on the COLO-Xen wiki for help. Thanks Zhang Chen >> I've been trying to set COLO replication to work but i'm stuck on a problem >> when migrating de primary VM to secondary host. >> >> I have been following the instructions from this wiki >> >> - http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping >> >> and this mail thread >> >> - >> http://xen.markmail.org/search/?q=COLO#query:COLO+page:1+mid:fb7wrn62vbks4unn+state:results >> >> I'm anexing the steps i took setting the environment before facing this >> problem when executing 'xl remus' command: >> >>> migration target: Ready to receive domain. >>> Saving to migration stream new xl format (info 0x3/0x0/2840) >>> Loading new save file <incoming migration stream> (new xl fmt info >> 0x3/0x0/2840) >>> Savefile contains xl domain config in JSON format >>> Parsing config from <saved> >>> xc: info: Saving domain 2, type x86 HVM >>> xc: info: Found x86 HVM domain from Xen 4.7 >>> xc: info: Restoring domain >>> xc: Frames iteration 0 of 5: 1045504/1045504 100% >>> xc: Domain now suspended: 0/0 0% >>> libxl: error: libxl_qmp.c:702:libxl__qmp_initialize: Connection error: No >> such file or directory >>> libxl: error: libxl_colo_restore.c:817:colo_restore_setup_cds_done: COLO: >> failed to setup device >for guest with domid 1 >>> xc: error: Restore failed (38 = Function not implemented): Internal error >>> libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo >> fails >>> libxl: error: libxl_stream_read.c:852:libxl__xc_domain_restore_done: >> restoring domain: Function >not implemented >>> libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo >> fails >> >> I'm hoping that someone could provide with directions. >> >> Thanks for your time and sory for bad english (not native language). >> >> >> Sadi. >> Network >> >> master: >> br0 : 10.20.107.30 binded with eth0 >> eth1: 192.168.1.30 >> eth2: 192.168.2.30 >> >> slave: >> br0 eth0: 10.20.107.33 binded with eth0 >> br1: no ip address binded with eth1 >> eth1: 192.168.1.33 >> eth2: 192.168.2.33 >> >> Eth1 both sides directly connected by cable >> Eth2 both sides directly connected by cable >> >> Repositories used: >> >> https://github.com/Pating/colo-proxy/tree/changlox >> https://github.com/macrosheep/iptables.git >> https://github.com/torvalds/linux >> >> https://github.com/wencongyang/xen >> >> Kernel build instructions followed: >> >> 2. Prepare host kernel for Dom0 >> colo-proxy kernel module need cooperate with linux kernel. You should patch kernel with ~/colo-proxy/colo-patch-for-kernel.patch >> -cd ~/colo-proxy/; git checkout 405527cbfa9f >> -cd ~/linux/; git checkout v4.0; git am ~/colo-proxy/colo-patch-for-kernel.patch >> -cp /boot/config-3.0.76-0.11-xen .config; make menuconfig to config your kernel support Dom0. Ref: http://wiki.xenproject.org/wiki/Mainline_Linux_Kernel_Configs >> -make -j8; make modules_install; make install >> -reboot >> >> >> COLO-Proxy: >> >> -cd ~/colo-proxy/; git checkout 405527cbfa9f; make; make install >> >> IPTables: >> >> -cd iptables; ./autogen.sh; ./configure --prefix=/usr/ --libdir=/usr/lib64; make; make install >> >> XEN: >> >> -./autogen.sh >> -./configure --enable-debug >> -touch tools/libxl/libxlu_disk_l.l >> -touch tools/libxl/libxlu_cfg_l.l >> -make dist-xen >> -make dist-tools >> -make install-xen >> -make install-tools >> >> *i've tried with https://github.com/wencongyang/qemu-xen but got an error with qemu when xl creating the VM as follows: >> >>> libxl: error: libxl_qmp.c:287:qmp_handle_error_response: received an error message from QMP server: Could not set password >> Qemu >> >> -cd ~/qemu-xen/; git checkout colo-xen-v2 >> >> Configured QEMU with script provided at: >> http://xen.markmail.org/message/y4jcdqxw2s2labdo?q=COLO#query:COLO+page:1+mid:3lzcuzeokqsqpu4i+state:results >> >> *path_to_xen_source updated according my directory tree. >> then.. >> >> -make >> -make install >> >> Running COLO >> >> *HVM SUSE 64bits >> >> primary: >> rm -f /var/log/xen/* >> rm -f /var/lib/xen/userdata-d.* >> service xencommons start >> modprobe nf_conntrack_ipv4 >> modprobe xt_PMYCOLO sec_dev=eth1 >> >> secondary: >> rm -f /var/log/xen/* >> rm -f /var/lib/xen/userdata-d.* >> service xencommons start >> modprobe xt_SECCOLO >> active_disk=/mnt/ramfs/active_disk.img >> hidden_disk=/mnt/ramfs/hidden_disk.img >> local_img=/root/new/SUSE/xenguest.img >> tmp_disk_size=`/root/new/pating/qemu-xen/qemu-img info $local_img |grep 'virtual size' |awk '{print $3}'` >> rm -rf /mnt/ramfs/* >> umount /mnt/ramfs/ >> rm -rf /mnt/ramfs/ >> mkdir /mnt/ramfs >> >> function create_image() >> { >> /root/new/pating/qemu-xen/qemu-img create -f qcow2 $1 $tmp_disk_size >> } >> function prepare_temp_images() >> { >> grep -q "^none /mnt/ramfs ramfs" /proc/mounts >> if [[ $? -ne 0 ]]; then >> mount -t ramfs none /mnt/ramfs/ -o size=2G >> fi >> >> if [[ ! -e $active_disk ]]; then >> create_image $active_disk >> fi >> >> if [[ ! -e $hidden_disk ]]; then >> create_image $hidden_disk >> fi >> } >> prepare_temp_images >> >> primary: >> >> xl create new/SUSE/vm-suse.cfg >> xl pause vm-suse >> xl remus -c -u vm-suse 192.168.2.33 >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> https://lists.xen.org/xen-devel > > > . > -- Thanks zhangchen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-10-27 22:56 Error migrating VM to secondary host using COLO replication Sadi 2016-10-28 16:56 ` Konrad Rzeszutek Wilk @ 2016-11-03 6:33 ` Wen Congyang [not found] ` <CAMHADrE4rXxM9K0UrSq38MZrz=dwKYyDSq0MQRiEZQyC+Qeviw@mail.gmail.com> 1 sibling, 1 reply; 11+ messages in thread From: Wen Congyang @ 2016-11-03 6:33 UTC (permalink / raw) To: Sadi, xen-devel On 10/28/2016 06:56 AM, Sadi wrote: > Hello, > > I've been trying to set COLO replication to work but i'm stuck on a problem when migrating de primary VM to secondary host. > > I have been following the instructions from this wiki > > - http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping > > and this mail thread > > - http://xen.markmail.org/search/?q=COLO#query:COLO+page:1+mid:fb7wrn62vbks4unn+state:results > > I'm anexing the steps i took setting the environment before facing this problem when executing 'xl remus' command: Sorry for the late reply. > >>migration target: Ready to receive domain. >>Saving to migration stream new xl format (info 0x3/0x0/2840) >>Loading new save file <incoming migration stream> (new xl fmt info 0x3/0x0/2840) >>Savefile contains xl domain config in JSON format >>Parsing config from <saved> >>xc: info: Saving domain 2, type x86 HVM >>xc: info: Found x86 HVM domain from Xen 4.7 >>xc: info: Restoring domain >>xc: Frames iteration 0 of 5: 1045504/1045504 100% >>xc: Domain now suspended: 0/0 0% >>libxl: error: libxl_qmp.c:702:libxl__qmp_initialize: Connection error: No such file or directory >>libxl: error: libxl_colo_restore.c:817:colo_restore_setup_cds_done: COLO: failed to setup device >for guest with domid 1 According to the log, we cannot connect to the secondary qemu. Can you give the secondry qemu log? Thanks Wen Congyang >>xc: error: Restore failed (38 = Function not implemented): Internal error >>libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo fails >>libxl: error: libxl_stream_read.c:852:libxl__xc_domain_restore_done: restoring domain: Function >not implemented >>libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo fails > > I'm hoping that someone could provide with directions. > > Thanks for your time and sory for bad english (not native language). > > > Sadi. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel > _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <CAMHADrE4rXxM9K0UrSq38MZrz=dwKYyDSq0MQRiEZQyC+Qeviw@mail.gmail.com>]
* Fwd: Re: Error migrating VM to secondary host using COLO replication [not found] ` <CAMHADrE4rXxM9K0UrSq38MZrz=dwKYyDSq0MQRiEZQyC+Qeviw@mail.gmail.com> @ 2016-11-04 17:50 ` Sadi 2016-11-07 20:37 ` Sadi 0 siblings, 1 reply; 11+ messages in thread From: Sadi @ 2016-11-04 17:50 UTC (permalink / raw) To: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 14143 bytes --] ---------- Forwarded message ---------- From: "Sadi" <sadijrp@gmail.com> Date: Nov 3, 2016 20:34 Subject: Re: [Xen-devel] Error migrating VM to secondary host using COLO replication To: "Wen Congyang" <wency@cn.fujitsu.com> Cc: > Hi, > > Thanks for replying. Here is what i got from /var/log/syslog: > > Nov 3 20:04:07 colob-HP-Compaq-6005-Pro-MT-PC systemd[1]: Started LSB: Start/stop xenstored and xenconsoled. > Nov 3 20:04:10 colob-HP-Compaq-6005-Pro-MT-PC systemd-timesyncd[631]: Timed out waiting for reply from 91.189.89.199:123 (ntp.ubuntu.com). > Nov 3 20:04:20 colob-HP-Compaq-6005-Pro-MT-PC systemd-timesyncd[631]: Timed out waiting for reply from 91.189.94.4:123 (ntp.ubuntu.com). > Nov 3 20:04:49 colob-HP-Compaq-6005-Pro-MT-PC systemd[1]: session-3.scope: Cannot determine UID from slice user-0.slice > Nov 3 20:04:49 colob-HP-Compaq-6005-Pro-MT-PC systemd[1]: Started Session 3 of user root. > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0-emu): new Tun device (carrier: OFF, driver: 'tun', ifindex: 7) > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC systemd-udevd[2063]: Could not generate persistent MAC address for vif2.0-emu: No such file or directory > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> devices added (path: /sys/devices/virtual/net/vif2.0-emu, iface: vif2.0-emu) > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> device added (path: /sys/devices/virtual/net/vif2.0-emu, iface: vif2.0-emu): no ifupdown configuration found. > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <warn> (vif2.0): failed to find device 8 'vif2.0' with udev > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0): new Ethernet device (carrier: OFF, driver: 'vif', ifindex: 8) > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> devices added (path: /sys/devices/vif-2-0/net/vif2.0, iface: vif2.0) > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> device added (path: /sys/devices/vif-2-0/net/vif2.0, iface: vif2.0): no ifupdown configuration found. > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2] > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.363596] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.363696] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/vif-bridge: online type_if=vif XENBUS_PATH=backend/vif/2/0 > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (br0): bridge port vif2.0 was attached > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.463896] device vif2.0 entered promiscuous mode > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0): enslaved to br0 > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.466360] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.529006] ip_tables: (C) 2000-2006 Netfilter Core Team > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.585769] Bridge firewalling registered > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/vif-bridge: Successful vif-bridge online for vif2.0, bridge br0. > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/vif-bridge: Writing backend/vif/2/0/hotplug-status connected to xenstore. > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/vif-bridge: add type_if=tap XENBUS_PATH=backend/vif/2/0 > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (br0): bridge port vif2.0-emu was attached > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0-emu): enslaved to br0 > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0-emu): link connected > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.693288] device vif2.0-emu entered promiscuous mode > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.695493] br0: port 3(vif2.0-emu) entered forwarding state > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.695500] br0: port 3(vif2.0-emu) entered forwarding state > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/vif-bridge: Successful vif-bridge add for vif2.0-emu, bridge br0. > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: setup XENBUS_PATH= > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (br0): bridge port vif2.0-emu was detached > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0-emu): released from master br0 > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.765881] device vif2.0-emu left promiscuous mode > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.765922] br0: port 3(vif2.0-emu) entered disabled state > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (colobr0): new Bridge device (carrier: OFF, driver: 'bridge', ifindex: 9) > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.771631] device vif2.0-emu entered promiscuous mode > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (colobr0): bridge port vif2.0-emu was attached > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0-emu): enslaved to colobr0 > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC systemd-udevd[2251]: Could not generate persistent MAC address for colobr0: No such file or directory > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl addif colobr0 eth1 failed > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.783326] colobr0: port 1(vif2.0-emu) entered forwarding state > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.783351] colobr0: port 1(vif2.0-emu) entered forwarding state > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (colobr0): link connected > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> devices added (path: /sys/devices/virtual/net/colobr0, iface: colobr0) > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> device added (path: /sys/devices/virtual/net/colobr0, iface: colobr0): no ifupdown configuration found. > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (colobr0): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2] > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (colobr0): detached bridge port vif2.0-emu > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0-emu): released from master colobr0 > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (colobr0): link disconnected > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.818240] device vif2.0-emu left promiscuous mode > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.818254] colobr0: port 1(vif2.0-emu) entered disabled state > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (colobr0): device state change: unavailable -> disconnected (reason 'none') [20 30 0] > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> Device 'colobr0' has no connection; scheduling activate_check in 0 seconds. > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.832977] ip6_tables: (C) 2000-2006 Netfilter Core Team > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: Writing /hotplug-status connected to xenstore. > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup setup for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, forwardbr: colobr0. > Nov 3 20:04:56 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: Joining mDNS multicast group on interface vif2.0-emu.IPv6 with address fe80::fcff:ffff:feff:ffff. > Nov 3 20:04:56 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: New relevant interface vif2.0-emu.IPv6 for mDNS. > Nov 3 20:04:56 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: Registering new address record for fe80::fcff:ffff:feff:ffff on vif2.0-emu.*. > Nov 3 20:04:59 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: Received packet from invalid interface. > Nov 3 20:05:00 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: message repeated 3 times: [ Received packet from invalid interface.] > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: teardown XENBUS_PATH= > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: teardown XENBUS_PATH= > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 eth1 failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 eth1 failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 vif2.0-emu failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 vif2.0-emu failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl delbr colobr0 failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl delbr colobr0 failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (br0): bridge port vif2.0-emu was attached > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: <info> (vif2.0-emu): enslaved to br0 > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 470.501624] device vif2.0-emu entered promiscuous mode > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 470.501656] br0: port 3(vif2.0-emu) entered forwarding state > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 470.501661] br0: port 3(vif2.0-emu) entered forwarding state > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: brctl addif br0 vif2.0-emu failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: rmmod xt_SECCOLO failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: iptables -t mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup teardown for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, forwardbr: colobr0. > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: ip6tables -t mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: rmmod xt_SECCOLO failed > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup teardown for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, forwardbr: colobr0. > Nov 3 20:05:02 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: Received packet from invalid interface. > Nov 3 20:05:02 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: Received packet from invalid interface. > > ... and dmesg: > > [ 464.363596] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > [ 464.363696] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > [ 464.463896] device vif2.0 entered promiscuous mode > [ 464.466360] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > [ 464.529006] ip_tables: (C) 2000-2006 Netfilter Core Team > [ 464.585769] Bridge firewalling registered > [ 464.693288] device vif2.0-emu entered promiscuous mode > [ 464.695493] br0: port 3(vif2.0-emu) entered forwarding state > [ 464.695500] br0: port 3(vif2.0-emu) entered forwarding state > [ 464.765881] device vif2.0-emu left promiscuous mode > [ 464.765922] br0: port 3(vif2.0-emu) entered disabled state > [ 464.771631] device vif2.0-emu entered promiscuous mode > [ 464.783326] colobr0: port 1(vif2.0-emu) entered forwarding state > [ 464.783351] colobr0: port 1(vif2.0-emu) entered forwarding state > [ 464.818240] device vif2.0-emu left promiscuous mode > [ 464.818254] colobr0: port 1(vif2.0-emu) entered disabled state > [ 464.832977] ip6_tables: (C) 2000-2006 Netfilter Core Team > [ 470.501624] device vif2.0-emu entered promiscuous mode > [ 470.501656] br0: port 3(vif2.0-emu) entered forwarding state > [ 470.501661] br0: port 3(vif2.0-emu) entered forwarding state > [ 485.549302] br0: port 3(vif2.0-emu) entered forwarding state > > *These logs are from secondary host. I have not found the qemu logs. > > You have mentioned Qemu, so i reviewed my steps and realized that i've set it in a different way. These were the actually steps: > > qemu: > ./configure --target-list=x86_64-softmmu --enable-gcrypt --enable-kvm > make > make install > > xen > ./autogen.sh > ./configure --enable-debug --enable-githttp --with-system-qemu > make dist > make install > > I have tried many ways to set colo up and ended up a little confused. Sory for that. > > I see a lot of 'invalid interface' in the log. So i review my network topology to. > > In this mail http://xen.markmail.org/search/?q=COLO#query:COLO+page:1+mid:zgz7wrdys5hi6c5s+state:results, the eth1 in the secondary host is not explictly binded with br1. > > The topology in https://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping suggest it. I'm using the second way, binding the interfaces. Is this correct? > > Again, Thanks for yor time, > > Sadi. > > > [-- Attachment #1.2: Type: text/html, Size: 16281 bytes --] [-- Attachment #2: Type: text/plain, Size: 127 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-11-04 17:50 ` Fwd: " Sadi @ 2016-11-07 20:37 ` Sadi 2016-11-08 19:57 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 11+ messages in thread From: Sadi @ 2016-11-07 20:37 UTC (permalink / raw) To: xen-devel, Wen Congyang [-- Attachment #1.1: Type: text/plain, Size: 26648 bytes --] Hi, I've noticed that in the /etc/xen/scripts/colo-proxy-setup the forward bridge was configured to colobr0. So, i've updated the script to use br1 and the situation seems to get a little bether. In Primary host, after executing 'xl remus' command the error is still the same: migration target: Ready to receive domain. Saving to migration stream new xl format (info 0x3/0x0/2840) Loading new save file <incoming migration stream> (new xl fmt info 0x3/0x0/2840) Savefile contains xl domain config in JSON format Parsing config from <saved> xc: info: Saving domain 2, type x86 HVM xc: info: Found x86 HVM domain from Xen 4.7 xc: info: Restoring domain xc: Frames iteration 0 of 5: 1045504/1045504 100% xc: Frames iteration 1 of 5: 1024/1024 100% xc: Domain now suspended: 0/0 0% libxl: error: libxl_qmp.c:702:libxl__qmp_initialize: Connection error: No such file or directory libxl: error: libxl_colo_restore.c:817:colo_restore_setup_cds_done: COLO: failed to setup device for guest with domid 2 xc: error: Restore failed (38 = Function not implemented): Internal error libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo fails libxl: error: libxl_stream_read.c:852:libxl__xc_domain_restore_done: restoring domain: Function not implemented libxl: info: libxl_colo_restore.c:320:libxl__colo_restore_teardown: colo fails Looking in the secondary host's syslog, it seems that the secondary host has understood that the primary one has failed. So it assumes the handle of the machine entering the network topology "after failover" especified in https://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping. Secondary host's syslog: Nov 7 18:10:13 colob systemd[1]: session-3.scope: Cannot determine UID from slice user-0.slice Nov 7 18:10:13 colob systemd[1]: Started Session 3 of user root. Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0-emu): new Tun device (carrier: OFF, driver: 'tun', ifindex: 6) Nov 7 18:10:24 colob systemd-udevd[1917]: Could not generate persistent MAC address for vif2.0-emu: No such file or directory Nov 7 18:10:24 colob NetworkManager[907]: <info> devices added (path: /sys/devices/virtual/net/vif2.0-emu, iface: vif2.0-emu) Nov 7 18:10:24 colob NetworkManager[907]: <info> device added (path: /sys/devices/virtual/net/vif2.0-emu, iface: vif2.0-emu): no ifupdown configuration found. Nov 7 18:10:24 colob NetworkManager[907]: <warn> (vif2.0): failed to find device 7 'vif2.0' with udev Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0): new Ethernet device (carrier: OFF, driver: 'vif', ifindex: 7) Nov 7 18:10:24 colob NetworkManager[907]: <info> devices added (path: /sys/devices/vif-2-0/net/vif2.0, iface: vif2.0) Nov 7 18:10:24 colob NetworkManager[907]: <info> device added (path: /sys/devices/vif-2-0/net/vif2.0, iface: vif2.0): no ifupdown configuration found. Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2] Nov 7 18:10:24 colob kernel: [ 604.313326] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready Nov 7 18:10:24 colob kernel: [ 604.313426] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready Nov 7 18:10:24 colob root: /etc/xen/scripts/vif-bridge: online type_if=vif XENBUS_PATH=backend/vif/2/0 Nov 7 18:10:24 colob NetworkManager[907]: <info> (br0): bridge port vif2.0 was attached Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0): enslaved to br0 Nov 7 18:10:24 colob kernel: [ 604.405016] device vif2.0 entered promiscuous mode Nov 7 18:10:24 colob kernel: [ 604.407207] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready Nov 7 18:10:24 colob kernel: [ 604.470129] ip_tables: (C) 2000-2006 Netfilter Core Team Nov 7 18:10:24 colob kernel: [ 604.526929] Bridge firewalling registered Nov 7 18:10:24 colob root: /etc/xen/scripts/vif-bridge: Successful vif-bridge online for vif2.0, bridge br0. Nov 7 18:10:24 colob root: /etc/xen/scripts/vif-bridge: Writing backend/vif/2/0/hotplug-status connected to xenstore. Nov 7 18:10:24 colob root: /etc/xen/scripts/vif-bridge: add type_if=tap XENBUS_PATH=backend/vif/2/0 Nov 7 18:10:24 colob NetworkManager[907]: <info> (br0): bridge port vif2.0-emu was attached Nov 7 18:10:24 colob kernel: [ 604.634201] device vif2.0-emu entered promiscuous mode Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0-emu): enslaved to br0 Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0-emu): link connected Nov 7 18:10:24 colob kernel: [ 604.636443] br0: port 3(vif2.0-emu) entered forwarding state Nov 7 18:10:24 colob kernel: [ 604.636463] br0: port 3(vif2.0-emu) entered forwarding state Nov 7 18:10:24 colob root: /etc/xen/scripts/vif-bridge: Successful vif-bridge add for vif2.0-emu, bridge br0. Nov 7 18:10:24 colob root: /etc/xen/scripts/colo-proxy-setup: setup XENBUS_PATH= Nov 7 18:10:24 colob NetworkManager[907]: <info> (br0): bridge port vif2.0-emu was detached Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0-emu): released from master br0 Nov 7 18:10:24 colob kernel: [ 604.721085] device vif2.0-emu left promiscuous mode Nov 7 18:10:24 colob kernel: [ 604.721122] br0: port 3(vif2.0-emu) entered disabled state Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): new Bridge device (carrier: OFF, driver: 'bridge', ifindex: 8) Nov 7 18:10:24 colob kernel: [ 604.726401] device vif2.0-emu entered promiscuous mode Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): bridge port vif2.0-emu was attached Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0-emu): enslaved to br1 Nov 7 18:10:24 colob systemd-udevd[2102]: Could not generate persistent MAC address for br1: No such file or directory Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): bridge port eth1 was attached Nov 7 18:10:24 colob kernel: [ 604.730047] device eth1 entered promiscuous mode Nov 7 18:10:24 colob NetworkManager[907]: <info> (eth1): enslaved to br1 Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): link connected Nov 7 18:10:24 colob kernel: [ 604.732580] br1: port 2(eth1) entered forwarding state Nov 7 18:10:24 colob kernel: [ 604.732605] br1: port 2(eth1) entered forwarding state Nov 7 18:10:24 colob kernel: [ 604.732610] br1: port 1(vif2.0-emu) entered forwarding state Nov 7 18:10:24 colob kernel: [ 604.732613] br1: port 1(vif2.0-emu) entered forwarding state Nov 7 18:10:24 colob kernel: [ 604.774105] ip6_tables: (C) 2000-2006 Netfilter Core Team Nov 7 18:10:24 colob systemd[1]: Started ifup for br1. Nov 7 18:10:24 colob root: /etc/xen/scripts/colo-proxy-setup: Writing /hotplug-status connected to xenstore. Nov 7 18:10:24 colob NetworkManager[907]: <info> devices added (path: /sys/devices/virtual/net/br1, iface: br1) Nov 7 18:10:24 colob NetworkManager[907]: <info> device added (path: /sys/devices/virtual/net/br1, iface: br1): no ifupdown configuration found. Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2] Nov 7 18:10:24 colob kernel: [ 604.785960] device vif2.0-emu left promiscuous mode Nov 7 18:10:24 colob kernel: [ 604.785973] br1: port 1(vif2.0-emu) entered disabled state Nov 7 18:10:24 colob systemd[1]: Found device /sys/subsystem/net/devices/br1. Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): detached bridge port vif2.0-emu Nov 7 18:10:24 colob NetworkManager[907]: <info> (vif2.0-emu): released from master br1 Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): detached bridge port eth1 Nov 7 18:10:24 colob NetworkManager[907]: <info> (eth1): released from master br1 Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): link disconnected Nov 7 18:10:24 colob NetworkManager[907]: <info> (br1): device state change: unavailable -> disconnected (reason 'none') [20 30 0] Nov 7 18:10:24 colob kernel: [ 604.787454] device eth1 left promiscuous mode Nov 7 18:10:24 colob kernel: [ 604.787475] br1: port 2(eth1) entered disabled state Nov 7 18:10:24 colob NetworkManager[907]: <info> Device 'br1' has no connection; scheduling activate_check in 0 seconds. Nov 7 18:10:24 colob root: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup setup for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, forwardbr: br1. Nov 7 18:10:25 colob systemd[1]: Reloading OpenBSD Secure Shell server. Nov 7 18:10:25 colob systemd[1]: Reloaded OpenBSD Secure Shell server. Nov 7 18:10:25 colob avahi-daemon[920]: Joining mDNS multicast group on interface vif2.0-emu.IPv6 with address fe80::fcff:ffff:feff:ffff. Nov 7 18:10:25 colob avahi-daemon[920]: New relevant interface vif2.0-emu.IPv6 for mDNS. Nov 7 18:10:25 colob avahi-daemon[920]: Registering new address record for fe80::fcff:ffff:feff:ffff on vif2.0-emu.*. Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: teardown XENBUS_PATH= Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: teardown XENBUS_PATH= Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl delif br1 eth1 failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl delif br1 eth1 failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl delif br1 vif2.0-emu failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl delif br1 vif2.0-emu failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl delbr br1 failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl delbr br1 failed Nov 7 18:10:30 colob NetworkManager[907]: <info> (br0): bridge port vif2.0-emu was attached Nov 7 18:10:30 colob kernel: [ 610.445938] device vif2.0-emu entered promiscuous mode Nov 7 18:10:30 colob kernel: [ 610.445972] br0: port 3(vif2.0-emu) entered forwarding state Nov 7 18:10:30 colob kernel: [ 610.445977] br0: port 3(vif2.0-emu) entered forwarding state Nov 7 18:10:30 colob NetworkManager[907]: <info> (vif2.0-emu): enslaved to br0 Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl addif br0 vif2.0-emu failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: iptables -t mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: rmmod xt_SECCOLO failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup teardown for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, forwardbr: br1. Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: ip6tables -t mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: rmmod xt_SECCOLO failed Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup teardown for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, forwardbr: br1. Nov 7 18:10:45 colob kernel: [ 625.475165] br0: port 3(vif2.0-emu) entered forwarding state When 'xl remus' hang on the error, the state of the vm in both hosts is: Primary: Name ID Mem VCPUs State Time(s) Domain-0 0 2502 4 r----- 65.5 suse-hvm 2 1023 2 --pss- 42.0 Secondary: Name ID Mem VCPUs State Time(s) Domain-0 0 2502 4 r----- 453.2 suse-hvm--incoming 2 1023 2 --p--- 0.0 At this moment, if i unpause the VM in the secondary host it keeps running from the point where the process was paused before the xl remus command. I've tried removing the bound between eth1 and br1 , but apparently nothing has changed. Thanks for your help. Sadi. On Fri, Nov 4, 2016 at 3:50 PM, Sadi <sadijrp@gmail.com> wrote: > ---------- Forwarded message ---------- > From: "Sadi" <sadijrp@gmail.com> > Date: Nov 3, 2016 20:34 > Subject: Re: [Xen-devel] Error migrating VM to secondary host using COLO > replication > To: "Wen Congyang" <wency@cn.fujitsu.com> > Cc: > > > Hi, > > > > > Thanks for replying. Here is what i got from /var/log/syslog: > > > > Nov 3 20:04:07 colob-HP-Compaq-6005-Pro-MT-PC systemd[1]: Started LSB: > Start/stop xenstored and xenconsoled. > > Nov 3 20:04:10 colob-HP-Compaq-6005-Pro-MT-PC systemd-timesyncd[631]: > Timed out waiting for reply from 91.189.89.199:123 (ntp.ubuntu.com). > > Nov 3 20:04:20 colob-HP-Compaq-6005-Pro-MT-PC systemd-timesyncd[631]: > Timed out waiting for reply from 91.189.94.4:123 (ntp.ubuntu.com). > > Nov 3 20:04:49 colob-HP-Compaq-6005-Pro-MT-PC systemd[1]: > session-3.scope: Cannot determine UID from slice user-0.slice > > Nov 3 20:04:49 colob-HP-Compaq-6005-Pro-MT-PC systemd[1]: Started > Session 3 of user root. > > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0-emu): new Tun device (carrier: OFF, driver: 'tun', ifindex: > 7) > > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC systemd-udevd[2063]: > Could not generate persistent MAC address for vif2.0-emu: No such file or > directory > > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> devices added (path: /sys/devices/virtual/net/vif2.0-emu, iface: > vif2.0-emu) > > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> device added (path: /sys/devices/virtual/net/vif2.0-emu, iface: > vif2.0-emu): no ifupdown configuration found. > > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <warn> (vif2.0): failed to find device 8 'vif2.0' with udev > > Nov 3 20:04:54 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0): new Ethernet device (carrier: OFF, driver: 'vif', > ifindex: 8) > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> devices added (path: /sys/devices/vif-2-0/net/vif2.0, iface: > vif2.0) > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> device added (path: /sys/devices/vif-2-0/net/vif2.0, iface: > vif2.0): no ifupdown configuration found. > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0): device state change: unmanaged -> unavailable (reason > 'managed') [10 20 2] > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.363596] > IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.363696] > IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/vif-bridge: online type_if=vif XENBUS_PATH=backend/vif/2/0 > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (br0): bridge port vif2.0 was attached > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.463896] > device vif2.0 entered promiscuous mode > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0): enslaved to br0 > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.466360] > IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.529006] > ip_tables: (C) 2000-2006 Netfilter Core Team > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.585769] > Bridge firewalling registered > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/vif-bridge: Successful vif-bridge online for vif2.0, > bridge br0. > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/vif-bridge: Writing backend/vif/2/0/hotplug-status > connected to xenstore. > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/vif-bridge: add type_if=tap XENBUS_PATH=backend/vif/2/0 > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (br0): bridge port vif2.0-emu was attached > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0-emu): enslaved to br0 > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0-emu): link connected > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.693288] > device vif2.0-emu entered promiscuous mode > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.695493] > br0: port 3(vif2.0-emu) entered forwarding state > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.695500] > br0: port 3(vif2.0-emu) entered forwarding state > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/vif-bridge: Successful vif-bridge add for vif2.0-emu, > bridge br0. > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: setup XENBUS_PATH= > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (br0): bridge port vif2.0-emu was detached > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0-emu): released from master br0 > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.765881] > device vif2.0-emu left promiscuous mode > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.765922] > br0: port 3(vif2.0-emu) entered disabled state > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (colobr0): new Bridge device (carrier: OFF, driver: 'bridge', > ifindex: 9) > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.771631] > device vif2.0-emu entered promiscuous mode > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (colobr0): bridge port vif2.0-emu was attached > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0-emu): enslaved to colobr0 > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC systemd-udevd[2251]: > Could not generate persistent MAC address for colobr0: No such file or > directory > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl addif colobr0 eth1 failed > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.783326] > colobr0: port 1(vif2.0-emu) entered forwarding state > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.783351] > colobr0: port 1(vif2.0-emu) entered forwarding state > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (colobr0): link connected > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> devices added (path: /sys/devices/virtual/net/colobr0, iface: > colobr0) > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> device added (path: /sys/devices/virtual/net/colobr0, iface: > colobr0): no ifupdown configuration found. > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (colobr0): device state change: unmanaged -> unavailable (reason > 'managed') [10 20 2] > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (colobr0): detached bridge port vif2.0-emu > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0-emu): released from master colobr0 > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (colobr0): link disconnected > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.818240] > device vif2.0-emu left promiscuous mode > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.818254] > colobr0: port 1(vif2.0-emu) entered disabled state > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (colobr0): device state change: unavailable -> disconnected (reason > 'none') [20 30 0] > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> Device 'colobr0' has no connection; scheduling activate_check in 0 > seconds. > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 464.832977] > ip6_tables: (C) 2000-2006 Netfilter Core Team > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: Writing /hotplug-status connected to > xenstore. > > Nov 3 20:04:55 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup setup for > vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, forwardbr: > colobr0. > > Nov 3 20:04:56 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: > Joining mDNS multicast group on interface vif2.0-emu.IPv6 with address > fe80::fcff:ffff:feff:ffff. > > Nov 3 20:04:56 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: New > relevant interface vif2.0-emu.IPv6 for mDNS. > > Nov 3 20:04:56 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: > Registering new address record for fe80::fcff:ffff:feff:ffff on > vif2.0-emu.*. > > Nov 3 20:04:59 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: > Received packet from invalid interface. > > Nov 3 20:05:00 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: > message repeated 3 times: [ Received packet from invalid interface.] > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: teardown XENBUS_PATH= > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: teardown XENBUS_PATH= > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 eth1 failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 eth1 failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 vif2.0-emu failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl delif colobr0 vif2.0-emu failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl delbr colobr0 failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl delbr colobr0 failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (br0): bridge port vif2.0-emu was attached > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC NetworkManager[924]: > <info> (vif2.0-emu): enslaved to br0 > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 470.501624] > device vif2.0-emu entered promiscuous mode > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 470.501656] > br0: port 3(vif2.0-emu) entered forwarding state > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC kernel: [ 470.501661] > br0: port 3(vif2.0-emu) entered forwarding state > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: brctl addif br0 vif2.0-emu failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: rmmod xt_SECCOLO failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: iptables -t mangle -D PREROUTING -m > physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup teardown > for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, > forwardbr: colobr0. > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: ip6tables -t mangle -D PREROUTING -m > physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: rmmod xt_SECCOLO failed > > Nov 3 20:05:01 colob-HP-Compaq-6005-Pro-MT-PC root: > /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup teardown > for vif2.0-emu. vifname: vif2.0-emu, index: 1, forwarddev: eth1, > forwardbr: colobr0. > > Nov 3 20:05:02 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: > Received packet from invalid interface. > > Nov 3 20:05:02 colob-HP-Compaq-6005-Pro-MT-PC avahi-daemon[1117]: > Received packet from invalid interface. > > > > ... and dmesg: > > > > [ 464.363596] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > > [ 464.363696] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > > [ 464.463896] device vif2.0 entered promiscuous mode > > [ 464.466360] IPv6: ADDRCONF(NETDEV_UP): vif2.0: link is not ready > > [ 464.529006] ip_tables: (C) 2000-2006 Netfilter Core Team > > [ 464.585769] Bridge firewalling registered > > [ 464.693288] device vif2.0-emu entered promiscuous mode > > [ 464.695493] br0: port 3(vif2.0-emu) entered forwarding state > > [ 464.695500] br0: port 3(vif2.0-emu) entered forwarding state > > [ 464.765881] device vif2.0-emu left promiscuous mode > > [ 464.765922] br0: port 3(vif2.0-emu) entered disabled state > > [ 464.771631] device vif2.0-emu entered promiscuous mode > > [ 464.783326] colobr0: port 1(vif2.0-emu) entered forwarding state > > [ 464.783351] colobr0: port 1(vif2.0-emu) entered forwarding state > > [ 464.818240] device vif2.0-emu left promiscuous mode > > [ 464.818254] colobr0: port 1(vif2.0-emu) entered disabled state > > [ 464.832977] ip6_tables: (C) 2000-2006 Netfilter Core Team > > [ 470.501624] device vif2.0-emu entered promiscuous mode > > [ 470.501656] br0: port 3(vif2.0-emu) entered forwarding state > > [ 470.501661] br0: port 3(vif2.0-emu) entered forwarding state > > [ 485.549302] br0: port 3(vif2.0-emu) entered forwarding state > > > > *These logs are from secondary host. I have not found the qemu logs. > > > > You have mentioned Qemu, so i reviewed my steps and realized that i've > set it in a different way. These were the actually steps: > > > > qemu: > > ./configure --target-list=x86_64-softmmu --enable-gcrypt --enable-kvm > > make > > make install > > > > xen > > ./autogen.sh > > ./configure --enable-debug --enable-githttp --with-system-qemu > > make dist > > make install > > > > I have tried many ways to set colo up and ended up a little confused. > Sory for that. > > > > I see a lot of 'invalid interface' in the log. So i review my network > topology to. > > > > In this mail http://xen.markmail.org/search/?q=COLO#query:COLO+ > page:1+mid:zgz7wrdys5hi6c5s+state:results, the eth1 in the secondary host > is not explictly binded with br1. > > > > The topology in https://wiki.xenproject.org/ > wiki/COLO_-_Coarse_Grain_Lock_Stepping suggest it. I'm using the second > way, binding the interfaces. Is this correct? > > > > Again, Thanks for yor time, > > > > Sadi. > > > > > > > -- Sadi. [-- Attachment #1.2: Type: text/html, Size: 30488 bytes --] [-- Attachment #2: Type: text/plain, Size: 127 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-11-07 20:37 ` Sadi @ 2016-11-08 19:57 ` Konrad Rzeszutek Wilk 2016-11-08 20:53 ` Sadi 0 siblings, 1 reply; 11+ messages in thread From: Konrad Rzeszutek Wilk @ 2016-11-08 19:57 UTC (permalink / raw) To: Sadi; +Cc: Wen Congyang, xen-devel > entered forwarding state > Nov 7 18:10:30 colob NetworkManager[907]: <info> (vif2.0-emu): enslaved > to br0 > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl addif > br0 vif2.0-emu failed How come this failed? > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: iptables -t > mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index > 1 failed Ah b/c of this. Are there any errors of what exactly failed? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-11-08 19:57 ` Konrad Rzeszutek Wilk @ 2016-11-08 20:53 ` Sadi 2016-11-09 17:31 ` Sadi 0 siblings, 1 reply; 11+ messages in thread From: Sadi @ 2016-11-08 20:53 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: Wen Congyang, xen-devel [-- Attachment #1.1: Type: text/plain, Size: 1933 bytes --] Hi, Apparently vif2.0-emu was already binded with br0 when "brctl addif br0 vif2.0-emu" failed. root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl addif br0 vif2.0-emu device vif2.0-emu is already a member of a bridge; can't enslave it to bridge br0. root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl show bridge name bridge id STP enabled interfaces br0 8000.001a3fc46255 no eth0 vif2.0 vif2.0-emu br1 8000.000000000000 no About the iptables, it seems like SECCOLO target can't be recognised. root@colob-HP-Compaq-6005-Pro-MT-PC:~# iptables -t mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 iptables: No chain/target/match by that name. Here is my active modules matching colo: root@colob-HP-Compaq-6005-Pro-MT-PC:~# lsmod | grep -i colo xt_SECCOLO 16384 1 nf_conntrack_colo 16384 2 xt_SECCOLO x_tables 36864 8 xt_physdev,ip6table_mangle,ip_tables,xt_SECCOLO,xt_tcpudp,iptable_filter,iptable_mangle,ip6_tables nf_conntrack 106496 4 xt_SECCOLO,nf_nat,nf_conntrack_colo,nf_conntrack_ipv4 So i was looking in the iptables and this really looks like the source of the problem. Sadi. On Tue, Nov 8, 2016 at 5:57 PM, Konrad Rzeszutek Wilk < konrad.wilk@oracle.com> wrote: > > entered forwarding state > > Nov 7 18:10:30 colob NetworkManager[907]: <info> (vif2.0-emu): enslaved > > to br0 > > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl > addif > > br0 vif2.0-emu failed > > > How come this failed? > > > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: iptables > -t > > mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO > --index > > 1 failed > > Ah b/c of this. Are there any errors of what exactly failed? > -- Sadi. [-- Attachment #1.2: Type: text/html, Size: 2962 bytes --] [-- Attachment #2: Type: text/plain, Size: 127 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-11-08 20:53 ` Sadi @ 2016-11-09 17:31 ` Sadi 2016-11-10 1:03 ` Wen Congyang 0 siblings, 1 reply; 11+ messages in thread From: Sadi @ 2016-11-09 17:31 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: Wen Congyang, xen-devel [-- Attachment #1.1: Type: text/plain, Size: 3322 bytes --] Hello again, Looking at the primary host's syslog, the arptables command from xen/etc/scripts/colo-proxy-setup has failed. Here's the log: Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: setup XENBUS_PATH= Nov 9 14:43:39 colop kernel: [ 302.825788] u32 classifier Nov 9 14:43:39 colop kernel: [ 302.825791] Actions configured Nov 9 14:43:39 colop kernel: [ 302.835407] Mirror/redirect action on Nov 9 14:43:39 colop kernel: [ 302.919605] ip6_tables: (C) 2000-2006 Netfilter Core Team Nov 9 14:43:39 colop kernel: [ 302.941511] arp_tables: (C) 2002 David S. Miller Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: arptables -I INPUT -i eth1 -j MARK --set-mark 1 failed Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: Writing /hotplug-status connected to xenstore. Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup setup for vif2.0-emu. mode = primary vifname: vif2.0-emu, index: 1, forwarddev: eth1. It's ok for the --set-mark argument to have value equal '1' , not a hex? The i got running the command is: root@colop:~# arptables -I INPUT -i eth1 -j MARK --set-mark 1 Bad argument `1' Thanks, Sadi. On Tue, Nov 8, 2016 at 6:53 PM, Sadi <sadijrp@gmail.com> wrote: > Hi, > > Apparently vif2.0-emu was already binded with br0 when "brctl addif br0 > vif2.0-emu" failed. > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl addif br0 vif2.0-emu > device vif2.0-emu is already a member of a bridge; can't enslave it to > bridge br0. > root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl show > bridge name bridge id STP enabled interfaces > br0 8000.001a3fc46255 no eth0 > vif2.0 > vif2.0-emu > br1 8000.000000000000 no > > About the iptables, it seems like SECCOLO target can't be recognised. > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# iptables -t mangle -D PREROUTING > -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 > iptables: No chain/target/match by that name. > > Here is my active modules matching colo: > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# lsmod | grep -i colo > xt_SECCOLO 16384 1 > nf_conntrack_colo 16384 2 xt_SECCOLO > x_tables 36864 8 xt_physdev,ip6table_mangle,ip_ > tables,xt_SECCOLO,xt_tcpudp,iptable_filter,iptable_mangle,ip6_tables > nf_conntrack 106496 4 xt_SECCOLO,nf_nat,nf_ > conntrack_colo,nf_conntrack_ipv4 > > So i was looking in the iptables and this really looks like the source of > the problem. > > Sadi. > > On Tue, Nov 8, 2016 at 5:57 PM, Konrad Rzeszutek Wilk < > konrad.wilk@oracle.com> wrote: > >> > entered forwarding state >> > Nov 7 18:10:30 colob NetworkManager[907]: <info> (vif2.0-emu): >> enslaved >> > to br0 >> > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl >> addif >> > br0 vif2.0-emu failed >> >> >> How come this failed? >> >> > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: >> iptables -t >> > mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO >> --index >> > 1 failed >> >> Ah b/c of this. Are there any errors of what exactly failed? >> > > > > -- > Sadi. > -- Sadi. [-- Attachment #1.2: Type: text/html, Size: 4982 bytes --] [-- Attachment #2: Type: text/plain, Size: 127 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-11-09 17:31 ` Sadi @ 2016-11-10 1:03 ` Wen Congyang 2016-11-10 21:53 ` Sadi 0 siblings, 1 reply; 11+ messages in thread From: Wen Congyang @ 2016-11-10 1:03 UTC (permalink / raw) To: Sadi, Konrad Rzeszutek Wilk; +Cc: xen-devel On 11/10/2016 01:31 AM, Sadi wrote: > Hello again, > > Looking at the primary host's syslog, the arptables command from xen/etc/scripts/colo-proxy-setup has failed. > > Here's the log: > > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: setup XENBUS_PATH= > Nov 9 14:43:39 colop kernel: [ 302.825788] u32 classifier > Nov 9 14:43:39 colop kernel: [ 302.825791] Actions configured > Nov 9 14:43:39 colop kernel: [ 302.835407] Mirror/redirect action on > Nov 9 14:43:39 colop kernel: [ 302.919605] ip6_tables: (C) 2000-2006 Netfilter Core Team > Nov 9 14:43:39 colop kernel: [ 302.941511] arp_tables: (C) 2002 David S. Miller > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: arptables -I INPUT -i eth1 -j MARK --set-mark 1 failed > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: Writing /hotplug-status connected to xenstore. > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: Successful colo-proxy-setup setup for vif2.0-emu. mode = primary vifname: vif2.0-emu, index: 1, forwarddev: eth1. > > It's ok for the --set-mark argument to have value equal '1' , not a hex? This log is very useful, we will investigate it. Thanks Wen Congyang > > The i got running the command is: > root@colop:~# arptables -I INPUT -i eth1 -j MARK --set-mark 1 > Bad argument `1' > > Thanks, Sadi. > > > > On Tue, Nov 8, 2016 at 6:53 PM, Sadi <sadijrp@gmail.com <mailto:sadijrp@gmail.com>> wrote: > > Hi, > > Apparently vif2.0-emu was already binded with br0 when "brctl addif br0 vif2.0-emu" failed. > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl addif br0 vif2.0-emu > device vif2.0-emu is already a member of a bridge; can't enslave it to bridge br0. > root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl show > bridge name bridge id STP enabled interfaces > br0 8000.001a3fc46255 no eth0 > vif2.0 > vif2.0-emu > br1 8000.000000000000 no > > About the iptables, it seems like SECCOLO target can't be recognised. > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# iptables -t mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 > iptables: No chain/target/match by that name. > > Here is my active modules matching colo: > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# lsmod | grep -i colo > xt_SECCOLO 16384 1 > nf_conntrack_colo 16384 2 xt_SECCOLO > x_tables 36864 8 xt_physdev,ip6table_mangle,ip_tables,xt_SECCOLO,xt_tcpudp,iptable_filter,iptable_mangle,ip6_tables > nf_conntrack 106496 4 xt_SECCOLO,nf_nat,nf_conntrack_colo,nf_conntrack_ipv4 > > So i was looking in the iptables and this really looks like the source of the problem. > > Sadi. > > On Tue, Nov 8, 2016 at 5:57 PM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com <mailto:konrad.wilk@oracle.com>> wrote: > > > entered forwarding state > > Nov 7 18:10:30 colob NetworkManager[907]: <info> (vif2.0-emu): enslaved > > to br0 > > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: brctl addif > > br0 vif2.0-emu failed > > > How come this failed? > > > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: iptables -t > > mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index > > 1 failed > > Ah b/c of this. Are there any errors of what exactly failed? > > > > > -- > Sadi. > > > > > -- > Sadi. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Error migrating VM to secondary host using COLO replication 2016-11-10 1:03 ` Wen Congyang @ 2016-11-10 21:53 ` Sadi 0 siblings, 0 replies; 11+ messages in thread From: Sadi @ 2016-11-10 21:53 UTC (permalink / raw) To: Wen Congyang; +Cc: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 4627 bytes --] Hi, Thanks for helping me. I'm trying lots of different things so maybe there are traces of older tries i've done conflicting right now. I'm planning on starting over again from the very beginning. I just wanna know what tutorial follow. The one from the Wiki, from the mail thread or another one. Link to tutorials: Wiki: http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping Mail Thread: http://xen.markmail.org/search/?q=COLO#query:COLO+page:1+ mid:fb7wrn62vbks4unn+state:results I had a little fight against ubuntu, so if you could suggest me an operating system to, would be of great help. Thanks, Sadi. On Wed, Nov 9, 2016 at 11:03 PM, Wen Congyang <wency@cn.fujitsu.com> wrote: > On 11/10/2016 01:31 AM, Sadi wrote: > > Hello again, > > > > Looking at the primary host's syslog, the arptables command from > xen/etc/scripts/colo-proxy-setup has failed. > > > > Here's the log: > > > > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: setup > XENBUS_PATH= > > Nov 9 14:43:39 colop kernel: [ 302.825788] u32 classifier > > Nov 9 14:43:39 colop kernel: [ 302.825791] Actions configured > > Nov 9 14:43:39 colop kernel: [ 302.835407] Mirror/redirect action on > > Nov 9 14:43:39 colop kernel: [ 302.919605] ip6_tables: (C) 2000-2006 > Netfilter Core Team > > Nov 9 14:43:39 colop kernel: [ 302.941511] arp_tables: (C) 2002 David > S. Miller > > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: > arptables -I INPUT -i eth1 -j MARK --set-mark 1 failed > > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: Writing > /hotplug-status connected to xenstore. > > Nov 9 14:43:39 colop colop: /etc/xen/scripts/colo-proxy-setup: > Successful colo-proxy-setup setup for vif2.0-emu. mode = primary vifname: > vif2.0-emu, index: 1, forwarddev: eth1. > > > > It's ok for the --set-mark argument to have value equal '1' , not a hex? > > This log is very useful, we will investigate it. > > Thanks > Wen Congyang > > > > > The i got running the command is: > > root@colop:~# arptables -I INPUT -i eth1 -j MARK --set-mark 1 > > Bad argument `1' > > > > Thanks, Sadi. > > > > > > > > On Tue, Nov 8, 2016 at 6:53 PM, Sadi <sadijrp@gmail.com <mailto: > sadijrp@gmail.com>> wrote: > > > > Hi, > > > > Apparently vif2.0-emu was already binded with br0 when "brctl addif > br0 vif2.0-emu" failed. > > > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl addif br0 vif2.0-emu > > device vif2.0-emu is already a member of a bridge; can't enslave it > to bridge br0. > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# brctl show > > bridge name bridge id STP enabled interfaces > > br0 8000.001a3fc46255 no eth0 > > vif2.0 > > vif2.0-emu > > br1 8000.000000000000 no > > > > About the iptables, it seems like SECCOLO target can't be recognised. > > > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# iptables -t mangle -D > PREROUTING -m physdev --physdev-in vif2.0-emu -j SECCOLO --index 1 > > iptables: No chain/target/match by that name. > > > > Here is my active modules matching colo: > > > > root@colob-HP-Compaq-6005-Pro-MT-PC:~# lsmod | grep -i colo > > xt_SECCOLO 16384 1 > > nf_conntrack_colo 16384 2 xt_SECCOLO > > x_tables 36864 8 xt_physdev,ip6table_mangle,ip_ > tables,xt_SECCOLO,xt_tcpudp,iptable_filter,iptable_mangle,ip6_tables > > nf_conntrack 106496 4 xt_SECCOLO,nf_nat,nf_ > conntrack_colo,nf_conntrack_ipv4 > > > > So i was looking in the iptables and this really looks like the > source of the problem. > > > > Sadi. > > > > On Tue, Nov 8, 2016 at 5:57 PM, Konrad Rzeszutek Wilk < > konrad.wilk@oracle.com <mailto:konrad.wilk@oracle.com>> wrote: > > > > > entered forwarding state > > > Nov 7 18:10:30 colob NetworkManager[907]: <info> > (vif2.0-emu): enslaved > > > to br0 > > > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: > brctl addif > > > br0 vif2.0-emu failed > > > > > > How come this failed? > > > > > Nov 7 18:10:30 colob root: /etc/xen/scripts/colo-proxy-setup: > iptables -t > > > mangle -D PREROUTING -m physdev --physdev-in vif2.0-emu -j > SECCOLO --index > > > 1 failed > > > > Ah b/c of this. Are there any errors of what exactly failed? > > > > > > > > > > -- > > Sadi. > > > > > > > > > > -- > > Sadi. > > > > -- Sadi. [-- Attachment #1.2: Type: text/html, Size: 6770 bytes --] [-- Attachment #2: Type: text/plain, Size: 127 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2016-11-10 21:53 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-10-27 22:56 Error migrating VM to secondary host using COLO replication Sadi
2016-10-28 16:56 ` Konrad Rzeszutek Wilk
2016-11-03 6:23 ` Zhang Chen
2016-11-03 6:33 ` Wen Congyang
[not found] ` <CAMHADrE4rXxM9K0UrSq38MZrz=dwKYyDSq0MQRiEZQyC+Qeviw@mail.gmail.com>
2016-11-04 17:50 ` Fwd: " Sadi
2016-11-07 20:37 ` Sadi
2016-11-08 19:57 ` Konrad Rzeszutek Wilk
2016-11-08 20:53 ` Sadi
2016-11-09 17:31 ` Sadi
2016-11-10 1:03 ` Wen Congyang
2016-11-10 21:53 ` Sadi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).