Live migration stops working...

Live migration stops working...

[Thomas Besser]

Hi,

since upgrade from qemu-kvm 0.10.5 to 0.10.6 (Debian Lenny, Packages from
http://www.corpit.ru/debian/tls/kvm/, kvm module from kernel 2.6.30)
live-migration does not work anymore.

After "migrate -d tcp:192.168.0.2:4444" info says "migration completed",
but "incoming" kvm machine dies.

Any hint how to debug this?

Regards
Thomas

Re: Live migration stops working...

[Avi Kivity]

On 08/17/2009 04:57 PM, Thomas Besser wrote:
> Hi,
>
> since upgrade from qemu-kvm 0.10.5 to 0.10.6 (Debian Lenny, Packages from
> http://www.corpit.ru/debian/tls/kvm/, kvm module from kernel 2.6.30)
> live-migration does not work anymore.
>
> After "migrate -d tcp:192.168.0.2:4444" info says "migration completed",
> but "incoming" kvm machine dies.
>
> Any hint how to debug this?
>
>    

What do you mean exactly by "dies"?

-- 
error compiling committee.c: too many arguments to function

Re: Live migration stops working...

[Stephane Bakhos]

>>  since upgrade from qemu-kvm 0.10.5 to 0.10.6 (Debian Lenny, Packages from
>>  http://www.corpit.ru/debian/tls/kvm/, kvm module from kernel 2.6.30)
>>  live-migration does not work anymore.
>>
>>  After "migrate -d tcp:192.168.0.2:4444" info says "migration completed",
>>  but "incoming" kvm machine dies.
>>
>>  Any hint how to debug this?
>>
>> 
>
> What do you mean exactly by "dies"?

Not sure if this is related but this is what happens on my system when I 
try to do a live migration with kvm-88 and the latest kernel 2.6.30 
(gentoo 2.6.30-gentoo-r5).

[   73.168871] general protection fault: 0000 [#1] SMP
[   73.169409] last sysfs file: /sys/kernel/uevent_seqnum
[   73.169733] Modules linked in:
[   73.169733]
[   73.169733] Pid: 1, comm: init Not tainted (2.6.30-gentoo-r5 #1)
[   73.169733] EIP: 0060:[<c0102f01>] EFLAGS: 00010246 CPU: 0
[   73.169733] EIP is at sysenter_exit+0xf/0x16
[   73.169733] EAX: 00000000 EBX: 0000000b ECX: bfbc2a80 EDX: b7f90424
[   73.169733] ESI: 00000000 EDI: bfbc2d74 EBP: 00000000 ESP: df82bfb4
[   73.169733]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[   73.169733] Process init (pid: 1, ti=df82a000 task=df830000 
task.ti=df82a000)
[   73.169733] Stack:
[   73.169733]  0000000b bfbc2c44 00000000 00000000 bfbc2d74 bfbc2d88 
00000000 0000007b
[   73.169733]  0000007b 00000000 00000000 0000008e b7f90424 00000073 
00000246 bfbc2a80
[   73.169733]  0000007b 00000000 00000000
[   73.169733] Call Trace:
[   73.169733] Code: 42 c0 89 44 24 18 fa 8d 80 00 00 00 00 8b 4d 08 f7 c1 
ff fe 00 08 0f 85 56 01 00 00 8b 54 24 30 8b 4c 24 3c 31 ed 8e 64 24 24 fb 
<0f> 35 8d 44 20 00 90 50 fc 6a 00 0f a0 06 1e 50 55 57 56 52 51
[   73.169733] EIP: [<c0102f01>] sysenter_exit+0xf/0x16 SS:ESP 
0068:df82bfb4
[   73.179556] ---[ end trace f1a39bb1992a39a4 ]---
[   73.180072] general protection fault: 0000 [#2] SMP
[   73.180547] last sysfs file: /sys/kernel/uevent_seqnum
[   73.181009] Modules linked in:
[   73.181038]
[   73.181038] Pid: 2620, comm: klogd Tainted: G      D 
(2.6.30-gentoo-r5 #1)
[   73.181038] EIP: 0060:[<c0102f01>] EFLAGS: 00010246 CPU: 0
[   73.181038] EIP is at sysenter_exit+0xf/0x16
[   73.181038] EAX: 00000511 EBX: 00000000 ECX: bf97b0d8 EDX: b7ef3424
[   73.181038] ESI: 0804c479 EDI: 00000001 EBP: 00000000 ESP: df3d7fb4
[   73.181038]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[   73.181038] Process klogd (pid: 2620, ti=df3d6000 task=df86a5d0 
task.ti=df3d6000)
[   73.181038] Stack:
[   73.181038]  00000000 0804e5e0 00000fff 0804c479 00000001 bf97b108 
00000511 0000007b
[   73.181038]  0000007b 00000000 00000000 00000003 b7ef3424 00000073 
00000246 bf97b0d8
[   73.181038]  0000007b 00000000 00000000
[   73.181038] Call Trace:
[   73.181038] Code: 42 c0 89 44 24 18 fa 8d 80 00 00 00 00 8b 4d 08 f7 c1 
ff fe 00 08 0f 85 56 01 00 00 8b 54 24 30 8b 4c 24 3c 31 ed 8e 64 24 24 fb 
<0f> 35 8d 44 20 00 90 50 fc 6a 00 0f a0 06 1e 50 55 57 56 52 51
[   73.181038] EIP: [<c0102f01>] sysenter_exit+0xf/0x16 SS:ESP 
0068:df3d7fb4
[   73.190938] ---[ end trace f1a39bb1992a39a5 ]---
[   73.191390] general protection fault: 0000 [#3] SMP
[   73.191865] last sysfs file: /sys/kernel/uevent_seqnum
[   73.192329] Modules linked in:
[   73.192365]
[   73.192365] Pid: 2672, comm: apache2 Tainted: G      D 
(2.6.30-gentoo-r5 #1)
[   73.192365] EIP: 0060:[<c0102f01>] EFLAGS: 00010246 CPU: 0
[   73.192365] EIP is at sysenter_exit+0xf/0x16
[   73.192365] EAX: 00000000 EBX: 00000000 ECX: bfb7aea0 EDX: b80b7424
[   73.192365] ESI: 00000000 EDI: bfb7aee4 EBP: 00000000 ESP: dc453fb4
[   73.192365]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[   73.192365] Process apache2 (pid: 2672, ti=dc452000 task=df833390 
task.ti=dc452000)
[   73.192365] Stack:
[   73.192365]  00000000 00000000 00000000 00000000 bfb7aee4 bfb7aef8 
00000000 0000007b
[   73.192365]  0000007b 00000000 00000000 0000008e b80b7424 00000073 
00000246 bfb7aea0
[   73.192365]  0000007b 00000000 00000000
[   73.192365] Call Trace:
[   73.192365] Code: 42 c0 89 44 24 18 fa 8d 80 00 00 00 00 8b 4d 08 f7 c1 
ff fe 00 08 0f 85 56 01 00 00 8b 54 24 30 8b 4c 24 3c 31 ed 8e 64 24 24 fb 
<0f> 35 8d 44 20 00 90 50 fc 6a 00 0f a0 06 1e 50 55 57 56 52 51
[   73.192365] EIP: [<c0102f01>] sysenter_exit+0xf/0x16 SS:ESP 
0068:dc453fb4
[   73.202214] ---[ end trace f1a39bb1992a39a6 ]---
[   73.202655] general protection fault: 0000 [#4] SMP
[   73.203137] last sysfs file: /sys/kernel/uevent_seqnum
[   73.203207] Modules linked in:
[   73.203207]
[   73.203207] Pid: 2598, comm: mysqld Tainted: G      D 
(2.6.30-gentoo-r5 #1)
[   73.203207] EIP: 0060:[<c0102f01>] EFLAGS: 00210246 CPU: 0
[   73.203207] EIP is at sysenter_exit+0xf/0x16
[   73.203207] EAX: 00000000 EBX: 00000000 ECX: b324832c EDX: b7f23424
[   73.203207] ESI: 00000000 EDI: b3248370 EBP: 00000000 ESP: df3c9fb4
[   73.203207]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[   73.203207] Process mysqld (pid: 2598, ti=df3c8000 task=df831810 
task.ti=df3c8000)
[   73.203207] Stack:
[   73.203207]  00000000 00000000 00000000 00000000 b3248370 b3248378 
00000000 0000007b
[   73.203207]  0000007b 00000000 00000000 0000008e b7f23424 00000073 
00200246 b324832c
[   73.203207]  0000007b 00000000 00000000
[   73.203207] Call Trace:
[   73.203207] Code: 42 c0 89 44 24 18 fa 8d 80 00 00 00 00 8b 4d 08 f7 c1 
ff fe 00 08 0f 85 56 01 00 00 8b 54 24 30 8b 4c 24 3c 31 ed 8e 64 24 24 fb 
<0f> 35 8d 44 20 00 90 50 fc 6a 00 0f a0 06 1e 50 55 57 56 52 51
[   73.203207] EIP: [<c0102f01>] sysenter_exit+0xf/0x16 SS:ESP 
0068:df3c9fb4
[   73.213415] ---[ end trace f1a39bb1992a39a7 ]---
[   73.213874] general protection fault: 0000 [#5] SMP
[   73.214366] last sysfs file: /sys/kernel/uevent_seqnum
[   73.214409] Modules linked in:
[   73.214409]
[   73.214409] Pid: 2599, comm: mysqld Tainted: G      D 
(2.6.30-gentoo-r5 #1)
[   73.214409] EIP: 0060:[<c0102f01>] EFLAGS: 00210246 CPU: 0
[   73.214409] EIP is at sysenter_exit+0xf/0x16
[   73.214409] EAX: 00000000 EBX: 00000000 ECX: b2a4732c EDX: b7f23424
[   73.214409] ESI: 00000000 EDI: b2a47370 EBP: 00000000 ESP: df3cbfb4
[   73.214409]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[   73.214409] Process mysqld (pid: 2599, ti=df3ca000 task=df8314a0 
task.ti=df3ca000)
[   73.214409] Stack:
[   73.214409]  00000000 00000000 00000000 00000000 b2a47370 b2a47378 
00000000 0000007b
[   73.214409]  0000007b 00000000 00000000 0000008e b7f23424 00000073 
00200246 b2a4732c
[   73.214409]  0000007b 00000000 00000000
[   73.214409] Call Trace:
[   73.214409] Code: 42 c0 89 44 24 18 fa 8d 80 00 00 00 00 8b 4d 08 f7 c1 
ff fe 00 08 0f 85 56 01 00 00 8b 54 24 30 8b 4c 24 3c 31 ed 8e 64 24 24 fb 
<0f> 35 8d 44 20 00 90 50 fc 6a 00 0f a0 06 1e 50 55 57 56 52 51
[   73.214409] EIP: [<c0102f01>] sysenter_exit+0xf/0x16 SS:ESP 
0068:df3cbfb4
[   73.224816] ---[ end trace f1a39bb1992a39a8 ]---
[   73.225781] Kernel panic - not syncing: Attempted to kill init!
[   73.226337] Pid: 1, comm: init Tainted: G      D    2.6.30-gentoo-r5 #1
[   73.226930] Call Trace:
[   73.227185]  [<c0428219>] ? panic+0x42/0xf0
[   73.227571]  [<c012ca9e>] ? do_exit+0x66e/0x670
[   73.227984]  [<c01037c6>] ? apic_timer_interrupt+0x2a/0x30
[   73.228491]  [<c04282de>] ? printk+0x17/0x1b
[   73.228884]  [<c0104a50>] ? do_general_protection+0x0/0x190
[   73.229406]  [<c0106491>] ? oops_end+0x91/0xa0
[   73.229815]  [<c042acc5>] ? error_code+0x6d/0x74
[   73.230253]  [<c0104a50>] ? do_general_protection+0x0/0x190
[   73.230757]  [<c0102f01>] ? sysenter_exit+0xf/0x16

Re: Live migration stops working...

[Avi Kivity]

On 08/17/2009 08:15 PM, Stephane Bakhos wrote:
>>>  since upgrade from qemu-kvm 0.10.5 to 0.10.6 (Debian Lenny, 
>>> Packages from
>>>  http://www.corpit.ru/debian/tls/kvm/, kvm module from kernel 2.6.30)
>>>  live-migration does not work anymore.
>>>
>>>  After "migrate -d tcp:192.168.0.2:4444" info says "migration 
>>> completed",
>>>  but "incoming" kvm machine dies.
>>>
>>>  Any hint how to debug this?
>>>
>>>
>>
>> What do you mean exactly by "dies"?
>
> Not sure if this is related but this is what happens on my system when 
> I try to do a live migration with kvm-88 and the latest kernel 2.6.30 
> (gentoo 2.6.30-gentoo-r5).
>
> [   73.168871] general protection fault: 0000 [#1] SMP
> [   73.169409] last sysfs file: /sys/kernel/uevent_seqnum
> [   73.169733] Modules linked in:
> [   73.169733]
> [   73.169733] Pid: 1, comm: init Not tainted (2.6.30-gentoo-r5 #1)
> [   73.169733] EIP: 0060:[<c0102f01>] EFLAGS: 00010246 CPU: 0
> [   73.169733] EIP is at sysenter_exit+0xf/0x16
> [   73.169733] EAX: 00000000 EBX: 0000000b ECX: bfbc2a80 EDX: b7f90424
> [   73.169733] ESI: 00000000 EDI: bfbc2d74 EBP: 00000000 ESP: df82bfb4
> [   73.169733]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
> [   73.169733] Process init (pid: 1, ti=df82a000 task=df830000 
> task.ti=df82a000)
> [   73.169733] Stack:
> [   73.169733]  0000000b bfbc2c44 00000000 00000000 bfbc2d74 bfbc2d88 
> 00000000 0000007b
> [   73.169733]  0000007b 00000000 00000000 0000008e b7f90424 00000073 
> 00000246 bfbc2a80
> [   73.169733]  0000007b 00000000 00000000
> [   73.169733] Call Trace:
> [   73.169733] Code: 42 c0 89 44 24 18 fa 8d 80 00 00 00 00 8b 4d 08 
> f7 c1 ff fe 00 08 0f 85 56 01 00 00 8b 54 24 30 8b 4c 24 3c 31 ed 8e 
> 64 24 24 fb <0f> 35 8d 44 20 00 90 50 fc 6a 00 0f a0 06 1e 50 55 57 56 
> 52 51

This is the sysexit instruction.

Are you doing cross-vendor migration (AMD->Intel or vice versa)?

This is supported in kvm-88, but not in previous versions.  Are you sure 
the kvm-88 modules are loaded on the target host, and not the modules 
that came with the kernel?

Try 'cat /sys/module/kvm/version'.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: Live migration stops working...

[Stephane Bakhos]

>>  Not sure if this is related but this is what happens on my system when I
>>  try to do a live migration with kvm-88 and the latest kernel 2.6.30
>>  (gentoo 2.6.30-gentoo-r5).
>>
>>  [   73.168871] general protection fault: 0000 [#1] SMP
>>  [   73.169409] last sysfs file: /sys/kernel/uevent_seqnum
>>  [   73.169733] Modules linked in:
>>  [   73.169733]
>>  [   73.169733] Pid: 1, comm: init Not tainted (2.6.30-gentoo-r5 #1)
>>  [   73.169733] EIP: 0060:[<c0102f01>] EFLAGS: 00010246 CPU: 0
>>  [   73.169733] EIP is at sysenter_exit+0xf/0x16
>>  [   73.169733] EAX: 00000000 EBX: 0000000b ECX: bfbc2a80 EDX: b7f90424
>>  [   73.169733] ESI: 00000000 EDI: bfbc2d74 EBP: 00000000 ESP: df82bfb4
>>  [   73.169733]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
>>  [   73.169733] Process init (pid: 1, ti=df82a000 task=df830000
>>  task.ti=df82a000)
>>  [   73.169733] Stack:
>>  [   73.169733]  0000000b bfbc2c44 00000000 00000000 bfbc2d74 bfbc2d88
>>  00000000 0000007b
>>  [   73.169733]  0000007b 00000000 00000000 0000008e b7f90424 00000073
>>  00000246 bfbc2a80
>>  [   73.169733]  0000007b 00000000 00000000
>>  [   73.169733] Call Trace:
>>  [   73.169733] Code: 42 c0 89 44 24 18 fa 8d 80 00 00 00 00 8b 4d 08 f7 c1
>>  ff fe 00 08 0f 85 56 01 00 00 8b 54 24 30 8b 4c 24 3c 31 ed 8e 64 24 24 fb
>>  <0f> 35 8d 44 20 00 90 50 fc 6a 00 0f a0 06 1e 50 55 57 56 52 51
>
> This is the sysexit instruction.
>
> Are you doing cross-vendor migration (AMD->Intel or vice versa)?
>
> This is supported in kvm-88, but not in previous versions.  Are you sure the 
> kvm-88 modules are loaded on the target host, and not the modules that came 
> with the kernel?
>
> Try 'cat /sys/module/kvm/version'.

Only AMD->AMD, all of them are at kvm-kmod-devel-88
All of them run the userspace with kvm-88

Migration works if I do it with a live cd, but not if I do it with a 
running machine.

Each guest only has 1 cpu, 2 nics (I tried both virtio and e1000),

I tried with both 2.6.29 and 2.6.30

2.6.29 was a known working with kvm-84.

Re: Live migration stops working...

[Andre Przywara]

Hi,

> This is the sysexit instruction.
I recently stumbled upon a similar issue.
Can you try the following patch? This is not a very nice one,
but it cured the problem I had.
Do you see anything in the guest? I had some messages on the console
about a faulting init (at boot time, when I was booting a 32bit userland
on a 64bit kernel).

Regards,
Andre.

-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632 

---
 arch/x86/kvm/svm.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index fc14bdf..bddcde5 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -2140,6 +2140,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 data)
 	case MSR_VM_IGNNE:
 		pr_unimpl(vcpu, "unimplemented wrmsr: 0x%x data 0x%llx\n", ecx, data);
 		break;
+	case MSR_EFER:
+		{
+		int enable = !(data & (1<<_EFER_LME));
+		u32 *msrpm = svm->msrpm;
+		printk(KERN_INFO "%sabling SYSENTER MSR intercepts (%llx)\n",
+			enable ? "dis" : "en", data);
+		set_msr_interception(msrpm, MSR_IA32_SYSENTER_CS, enable, enable);
+		set_msr_interception(msrpm, MSR_IA32_SYSENTER_ESP, enable, enable);
+		set_msr_interception(msrpm, MSR_IA32_SYSENTER_EIP, enable, enable);
+		}
+		return kvm_set_msr_common(vcpu, ecx, data);
 	default:
 		return kvm_set_msr_common(vcpu, ecx, data);
 	}
-- 
1.6.1.3

Re: Live migration stops working... (SEGFAULT)

[Stephane Bakhos]

> Hi,
>
>> This is the sysexit instruction.
> I recently stumbled upon a similar issue.
> Can you try the following patch? This is not a very nice one,
> but it cured the problem I had.
> Do you see anything in the guest? I had some messages on the console
> about a faulting init (at boot time, when I was booting a 32bit userland
> on a 64bit kernel).

I've tried the patch and it did not help, however I could only have it on 
the receiver of the migration. Would it need to be installed on the one 
sending the data ?

The guest kernel and userland are 32bits, the host's are 64 bits.

I do not see any message about a faulting init.

Re: Live migration stops working... (SEGFAULT)

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]

On 08/18/2009 11:11 AM, Stephane Bakhos wrote:
>
> I've tried the patch and it did not help, however I could only have it 
> on the receiver of the migration. Would it need to be installed on the 
> one sending the data ?
>
> The guest kernel and userland are 32bits, the host's are 64 bits.
>
> I do not see any message about a faulting init.

It needs to be on both.  However try this better patch (attached, needs 
to be on both hosts).

-- 
error compiling committee.c: too many arguments to function


[-- Attachment #2: fix-sysenter-on-amd.patch --]
[-- Type: text/x-patch, Size: 1429 bytes --]

diff --git a/kvm/kernel/x86/svm.c b/kvm/kernel/x86/svm.c
index e158a2f..6df3cea 100644
--- a/kvm/kernel/x86/svm.c
+++ b/kvm/kernel/x86/svm.c
@@ -101,9 +101,6 @@ struct vcpu_svm {
 	unsigned long vmcb_pa;
 	struct svm_cpu_data *svm_data;
 	uint64_t asid_generation;
-	uint64_t sysenter_cs;
-	uint64_t sysenter_esp;
-	uint64_t sysenter_eip;
 
 	u64 next_rip;
 
@@ -2087,13 +2084,13 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 *data)
 		break;
 #endif
 	case MSR_IA32_SYSENTER_CS:
-		*data = svm->sysenter_cs;
+		*data = svm->vmcb->save.sysenter_cs;
 		break;
 	case MSR_IA32_SYSENTER_EIP:
-		*data = svm->sysenter_eip;
+		*data = svm->vmcb->save.sysenter_eip;
 		break;
 	case MSR_IA32_SYSENTER_ESP:
-		*data = svm->sysenter_esp;
+		*data = svm->vmcb->save.sysenter_esp;
 		break;
 	/* Nobody will change the following 5 values in the VMCB so
 	   we can safely return them on rdmsr. They will always be 0
@@ -2176,13 +2173,13 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 data)
 		break;
 #endif
 	case MSR_IA32_SYSENTER_CS:
-		svm->sysenter_cs = data;
+		svm->vmcb->save.sysenter_cs = data;
 		break;
 	case MSR_IA32_SYSENTER_EIP:
-		svm->sysenter_eip = data;
+		svm->vmcb->save.sysenter_eip = data;
 		break;
 	case MSR_IA32_SYSENTER_ESP:
-		svm->sysenter_esp = data;
+		svm->vmcb->save.sysenter_esp = data;
 		break;
 	case MSR_IA32_DEBUGCTLMSR:
 		if (!svm_has(SVM_FEATURE_LBRV)) {

Re: Live migration stops working... (SEGFAULT)

[Stephane Bakhos]

On Tue, 18 Aug 2009, Avi Kivity wrote:
> 
> On 08/18/2009 11:11 AM, Stephane Bakhos wrote:
>>
>>  I've tried the patch and it did not help, however I could only have it on
>>  the receiver of the migration. Would it need to be installed on the one
>>  sending the data ?
>>
>>  The guest kernel and userland are 32bits, the host's are 64 bits.
>>
>>  I do not see any message about a faulting init.
>
> It needs to be on both.  However try this better patch (attached, needs to be 
> on both hosts).

That solved it, thanks!

Re: Live migration stops working...

[Thomas Besser]

Avi Kivity wrote:
> On 08/17/2009 04:57 PM, Thomas Besser wrote:
>> After "migrate -d tcp:192.168.0.2:4444" info says "migration completed",
>> but "incoming" kvm machine dies.
> 
> What do you mean exactly by "dies"?

I'm starting the 'waiting' kvm process like this:

/usr/bin/kvm -name testbench -vnc :5 -net
nic,vlan=0,model=virtio,macaddr=00:08:15:00:09:95 -net
tap,vlan=0,ifname=tap05,script=no -m 256 -drive
file=/vm/testbench/testbench.raw,if=virtio,boot=on -daemonize -vga std -k
de -tdf -incoming tcp:0:4444

The resulting kvm process which is waiting for the incoming migration isn't
there anymore after the migration command on the 'source' host.

Re: Live migration stops working...

[Avi Kivity]

On 08/18/2009 09:14 AM, Thomas Besser wrote:
> Avi Kivity wrote:
>    
>> On 08/17/2009 04:57 PM, Thomas Besser wrote:
>>      
>>> After "migrate -d tcp:192.168.0.2:4444" info says "migration completed",
>>> but "incoming" kvm machine dies.
>>>        
>> What do you mean exactly by "dies"?
>>      
> I'm starting the 'waiting' kvm process like this:
>
> /usr/bin/kvm -name testbench -vnc :5 -net
> nic,vlan=0,model=virtio,macaddr=00:08:15:00:09:95 -net
> tap,vlan=0,ifname=tap05,script=no -m 256 -drive
> file=/vm/testbench/testbench.raw,if=virtio,boot=on -daemonize -vga std -k
> de -tdf -incoming tcp:0:4444
>
> The resulting kvm process which is waiting for the incoming migration isn't
> there anymore after the migration command on the 'source' host.
>    

Please run without -daemonize and capture stdio output, monitor output, 
and/or core dump.

-- 
error compiling committee.c: too many arguments to function

Re: Live migration stops working...

[Thomas Besser]

Avi Kivity wrote:
> On 08/18/2009 09:14 AM, Thomas Besser wrote:
> 
> Please run without -daemonize and capture stdio output, monitor output,
> and/or core dump.

Output from the starting shell:
Guest moved used index from 23 to 3995

Because the whole kvm process dies no vnc output can be captured. Or did you
mean something else with "monitor output"?

Howto produce more debug informations (i.e. core dump)?

Re: Live migration stops working...

[Avi Kivity]

On 08/18/2009 03:07 PM, Thomas Besser wrote:
> Avi Kivity wrote:
>    
>> On 08/18/2009 09:14 AM, Thomas Besser wrote:
>>
>> Please run without -daemonize and capture stdio output, monitor output,
>> and/or core dump.
>>      
> Output from the starting shell:
> Guest moved used index from 23 to 3995
>
>    

virtio issue.  What's the guest kernel version?

> Because the whole kvm process dies no vnc output can be captured. Or did you
> mean something else with "monitor output"?
>    

The monitor where you issue the migrate command (unless you're using a 
management application like virt-manager or virsh).

> Howto produce more debug informations (i.e. core dump)?
>    

'ulimit -c unlimited' before running qemu.  But in this case you won't 
get a dump since qemu exited cleanly.

-- 
error compiling committee.c: too many arguments to function

Re: Live migration stops working...

[Thomas Besser]

Avi Kivity wrote:

> On 08/18/2009 03:07 PM, Thomas Besser wrote:
>> Avi Kivity wrote:
>>> On 08/18/2009 09:14 AM, Thomas Besser wrote:
>> Output from the starting shell:
>> Guest moved used index from 23 to 3995   
> 
> virtio issue.  What's the guest kernel version?

2.6.30.4 (self-compiled, reduced for qemu/kvm, no modules)

>> Because the whole kvm process dies no vnc output can be captured. Or did
>> you mean something else with "monitor output"?
> 
> The monitor where you issue the migrate command (unless you're using a
> management application like virt-manager or virsh).

There are no error messages.

Re: Live migration stops working...

[Thomas Besser]

Thomas Besser wrote:

> Avi Kivity wrote:
>> virtio issue.  What's the guest kernel version?
> 
> 2.6.30.4 (self-compiled, reduced for qemu/kvm, no modules)

For completion: not vanilla sources, debian patched kernel sources.