qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* Re: [Qemu-devel] live migration between amd fam15h-fam10h
@ 2014-01-27 14:20 Markus Kovero
  2014-02-01 17:45 ` Brian Jackson
  2014-03-26 11:27 ` Alexandre DERUMIER
  0 siblings, 2 replies; 4+ messages in thread
From: Markus Kovero @ 2014-01-27 14:20 UTC (permalink / raw)
  To: qemu-devel

> Hi,
>
> I am getting a frozen guest when migrating from an Opteron 6274 host 
> (amd
> fam15h) to
> an Opteron 6174 host (amd fam10h). The live migration completes 
> succesfully, but
> the guest is frozen: vcn screen is still there, but no input is 
> possible and
> no kernel output is seen. Trying "c" on the qemu-monitor does not 
> help.
> I am using "-cpu Opteron_G3" which I assumed would be ok for both 
> host cpus.
>
> In the opposite direction (migrating from an amd fam10h host to an 
> amdfam15h
> host) the guest continues to run on the destination. However, on most 
> of these
> successfull live migrations, I notice a "clocksource unstable" 
> message on the
> guest kernel (using the default kvm-clock clocksource) e.g.
> Clocksource tsc unstable (delta = -1500533439 ns)
> Same situation (guest runs on destination with clocksource unstable 
> message)
> happens when migrating between fam15h hosts (I have not tried between 
> fam10h
> hosts)
>
> Changing the clocksource (tsc, acpi_pm, hpet) does not solve the 
> issue.
> Also tried with "-cpu kvm64" with same result.
>
> qemu-kvm version: 0.15.1, 1.0 or qemu-kvm/master
> Host kernel: 3.0.15 (on both hosts)
> Guest kernel: 3.0.6 or 3.2
>
> this is the qemu-kvm command line used on the source host:
>
> "
> kvm -enable-kvm -m 1024 -smp 1 -cpu Opteron_G3,check -drive \
> 
> file=/opt/test.img,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,boot=on
> -device
> 
> virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1
> -monitor stdio -vnc 0.0.0.0:6 -vga std -chardev pty,id=charserial0 
> -device
> isa-serial,chardev=charserial0,id=serial0 -usb -device 
> usb-tablet,id=input0
> "
>
> The destination host has the same command line with an added 
> "-incoming
> tcp:4444". I have mainly tested this with non-shared storage (but 
> also shared
> storage has the same result). Migration is triggered with "migrate -b
> tcp:destip:4444"
>
> Do the TSC microarchitecture changes in amdfam15h (see AMD SW 
> optimiization
> guide for fam15h, 47414 Rev 3.02 Appendix E) affect pvclock stability 
> on
> migration in same family or across families?
>
> cpuid information follows in case it's helpful.
..snip..


Hi, I can confirm this problem still exists in live migrations between 
Opteron 6128HE and Opteron 6274.
Live migration from 6100-series to 6200-series work, but never from 
6200 to 6100.
Issue is reproducible and symptoms are identical with previous poster.
I have tested with 3.10.5 host-kernel and 1.7 qemu, also with 3.1.4 and 
 >1.0 qemu, guest kernel seems to be irrelevant at this point (as it 
crashes any OS).

I would say this needs attention, and I'm willing to help to get this 
sorted out.

Thanks for your thoughts.

Yours
Markus Kovero
+358 40 577 1129

^ permalink raw reply	[flat|nested] 4+ messages in thread
* [Qemu-devel] live migration between amd fam15h-fam10h
@ 2012-03-01 14:35 Vasilis Liaskovitis
  0 siblings, 0 replies; 4+ messages in thread
From: Vasilis Liaskovitis @ 2012-03-01 14:35 UTC (permalink / raw)
  To: kvm, qemu-devel; +Cc: andre.przywara

Hi,

I am getting a frozen guest when migrating from an Opteron 6274 host (amd fam15h) to
an Opteron 6174 host (amd fam10h). The live migration completes succesfully, but
the guest is frozen: vcn screen is still there, but no input is possible and
no kernel output is seen. Trying "c" on the qemu-monitor does not help.
I am using "-cpu Opteron_G3" which I assumed would be ok for both host cpus.

In the opposite direction (migrating from an amd fam10h host to an amdfam15h
host) the guest continues to run on the destination. However, on most of these
successfull live migrations, I notice a "clocksource unstable" message on the
guest kernel (using the default kvm-clock clocksource) e.g.
Clocksource tsc unstable (delta = -1500533439 ns)
Same situation (guest runs on destination with clocksource unstable message)
happens when migrating between fam15h hosts (I have not tried between fam10h
hosts)

Changing the clocksource (tsc, acpi_pm, hpet) does not solve the issue.
Also tried with "-cpu kvm64" with same result.

qemu-kvm version: 0.15.1, 1.0 or qemu-kvm/master
Host kernel: 3.0.15 (on both hosts)
Guest kernel: 3.0.6 or 3.2

this is the qemu-kvm command line used on the source host: 

"
kvm -enable-kvm -m 1024 -smp 1 -cpu Opteron_G3,check -drive \
file=/opt/test.img,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,boot=on
-device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1
-monitor stdio -vnc 0.0.0.0:6 -vga std -chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
"

The destination host has the same command line with an added "-incoming
tcp:4444". I have mainly tested this with non-shared storage (but also shared
storage has the same result). Migration is triggered with "migrate -b tcp:destip:4444"

Do the TSC microarchitecture changes in amdfam15h (see AMD SW optimiization
guide for fam15h, 47414 Rev 3.02 Appendix E) affect pvclock stability on
migration in same family or across families?

cpuid information follows in case it's helpful.

6274 host:

 eax in    eax      ebx      ecx      edx
00000000 0000000d 68747541 444d4163 69746e65
00000001 00600f12 02100800 1e98220b 178bfbff
00000002 00000000 00000000 00000000 00000000
00000003 00000000 00000000 00000000 00000000
00000004 00000000 00000000 00000000 00000000
00000005 00000040 00000040 00000003 00000000
00000006 00000000 00000000 00000001 00000000
00000007 00000000 00000000 00000000 00000000
00000008 00000000 00000000 00000000 00000000
00000009 00000000 00000000 00000000 00000000
0000000a 00000000 00000000 00000000 00000000
0000000b 00000000 00000000 00000000 00000000
0000000c 00000000 00000000 00000000 00000000
0000000d 00000000 00000000 00000000 00000000
80000000 8000001e 68747541 444d4163 69746e65
80000001 00600f12 30000000 01c9bfff 2fd3fbff
80000002 20444d41 6574704f 286e6f72 20294d54
80000003 636f7250 6f737365 32362072 20203437
80000004 20202020 20202020 20202020 00202020
80000005 ff20ff18 ff20ff30 10040140 40020140
80000006 64000000 64004200 08008140 0060e140
80000007 00000000 00000000 00000000 000003d9
80000008 00003030 00000000 0000500f 00000000
80000009 00000000 00000000 00000000 00000000
8000000a 00000001 00010000 00000000 000014ff
8000000b 00000000 00000000 00000000 00000000
8000000c 00000000 00000000 00000000 00000000
8000000d 00000000 00000000 00000000 00000000
8000000e 00000000 00000000 00000000 00000000
8000000f 00000000 00000000 00000000 00000000
80000010 00000000 00000000 00000000 00000000
80000011 00000000 00000000 00000000 00000000
80000012 00000000 00000000 00000000 00000000
80000013 00000000 00000000 00000000 00000000
80000014 00000000 00000000 00000000 00000000
80000015 00000000 00000000 00000000 00000000
80000016 00000000 00000000 00000000 00000000
80000017 00000000 00000000 00000000 00000000
80000018 00000000 00000000 00000000 00000000
80000019 f020f018 64000000 00000000 00000000
8000001a 00000003 00000000 00000000 00000000
8000001b 000000ff 00000000 00000000 00000000
8000001c 00000000 80032013 00010200 8000000f
8000001d 00000000 00000000 00000000 00000000
8000001e 00000022 00000101 00000100 00000000

Vendor ID: "AuthenticAMD"; CPUID level 13

AMD-specific functions
Version 00600f12:
Family: 15 Model: 1 []

Standard feature flags 178bfbff:
Floating Point Unit
Virtual Mode Extensions
Debugging Extensions
Page Size Extensions
Time Stamp Counter (with RDTSC and CR4 disable bit)
Model Specific Registers with RDMSR & WRMSR
PAE - Page Address Extensions
Machine Check Exception
COMPXCHG8B Instruction
APIC
SYSCALL/SYSRET or SYSENTER/SYSEXIT instructions
MTRR - Memory Type Range Registers
Global paging extension
Machine Check Architecture
Conditional Move Instruction
PAT - Page Attribute Table
PSE-36 - Page Size Extensions
19 - reserved
MMX instructions
FXSAVE/FXRSTOR
25 - reserved
26 - reserved
28 - reserved
Generation: 15 Model: 1
Extended feature flags 2fd3fbff:
Floating Point Unit
Virtual Mode Extensions
Debugging Extensions
Page Size Extensions
Time Stamp Counter (with RDTSC and CR4 disable bit)
Model Specific Registers with RDMSR & WRMSR
PAE - Page Address Extensions
Machine Check Exception
COMPXCHG8B Instruction
APIC
SYSCALL/SYSRET or SYSENTER/SYSEXIT instructions
MTRR - Memory Type Range Registers
Global paging extension
Machine Check Architecture
Conditional Move Instruction
PAT - Page Attribute Table
PSE-36 - Page Size Extensions
20 - reserved
AMD MMX Instruction Extensions
MMX instructions
FXSAVE/FXRSTOR
25 - reserved
26 - reserved
27 - reserved
29 - reserved

Processor name string: AMD Opteron(TM) Processor 6274                 
L1 Cache Information:
2/4-MB Pages:
   Data TLB: associativity 255-way #entries 32
   Instruction TLB: associativity 255-way #entries 24
4-KB Pages:
   Data TLB: associativity 255-way #entries 32
   Instruction TLB: associativity 255-way #entries 48
L1 Data cache:
   size 16 KB associativity 4-way lines per tag 1 line size 64
L1 Instruction cache:
   size 64 KB associativity 2-way lines per tag 1 line size 64

L2 Cache Information:
2/4-MB Pages:
   Data TLB: associativity 4-way #entries 0
   Instruction TLB: associativity L2 off #entries 0
4-KB Pages:
   Data TLB: associativity 4-way #entries 0
   Instruction TLB: associativity 2-way #entries 0
   size 8 KB associativity L2 off lines per tag 129 line size 64

Advanced Power Management Feature Flags
Has temperature sensing diode
Maximum linear address: 48; maximum phys address 48


6174 host:


 eax in    eax      ebx      ecx      edx
00000000 00000005 68747541 444d4163 69746e65
00000001 00100f91 110c0800 00802009 178bfbff
00000002 00000000 00000000 00000000 00000000
00000003 00000000 00000000 00000000 00000000
00000004 00000000 00000000 00000000 00000000
00000005 00000040 00000040 00000003 00000000
80000000 8000001b 68747541 444d4163 69746e65
80000001 00100f91 300004bf 000837ff efd3fbff
80000002 20444d41 6574704f 286e6f72 20296d74
80000003 636f7250 6f737365 31362072 00003437
80000004 00000000 00000000 00000000 00000000
80000005 ff30ff10 ff30ff20 40020140 40020140
80000006 20800000 42004200 02008140 0050d140
80000007 00000000 00000000 00000000 000001f9
80000008 00003030 00000000 0000400b 00000000
80000009 00000000 00000000 00000000 00000000
8000000a 00000001 00000040 00000000 0000040f
8000000b 00000000 00000000 00000000 00000000
8000000c 00000000 00000000 00000000 00000000
8000000d 00000000 00000000 00000000 00000000
8000000e 00000000 00000000 00000000 00000000
8000000f 00000000 00000000 00000000 00000000
80000010 00000000 00000000 00000000 00000000
80000011 00000000 00000000 00000000 00000000
80000012 00000000 00000000 00000000 00000000
80000013 00000000 00000000 00000000 00000000
80000014 00000000 00000000 00000000 00000000
80000015 00000000 00000000 00000000 00000000
80000016 00000000 00000000 00000000 00000000
80000017 00000000 00000000 00000000 00000000
80000018 00000000 00000000 00000000 00000000
80000019 f0300000 60100000 00000000 00000000
8000001a 00000003 00000000 00000000 00000000
8000001b 0000001f 00000000 00000000 00000000

Vendor ID: "AuthenticAMD"; CPUID level 5

AMD-specific functions
Version 00100f91:
Family: 15 Model: 9 []

Standard feature flags 178bfbff:
Floating Point Unit
Virtual Mode Extensions
Debugging Extensions
Page Size Extensions
Time Stamp Counter (with RDTSC and CR4 disable bit)
Model Specific Registers with RDMSR & WRMSR
PAE - Page Address Extensions
Machine Check Exception
COMPXCHG8B Instruction
APIC
SYSCALL/SYSRET or SYSENTER/SYSEXIT instructions
MTRR - Memory Type Range Registers
Global paging extension
Machine Check Architecture
Conditional Move Instruction
PAT - Page Attribute Table
PSE-36 - Page Size Extensions
19 - reserved
MMX instructions
FXSAVE/FXRSTOR
25 - reserved
26 - reserved
28 - reserved
Generation: 15 Model: 9
Extended feature flags efd3fbff:
Floating Point Unit
Virtual Mode Extensions
Debugging Extensions
Page Size Extensions
Time Stamp Counter (with RDTSC and CR4 disable bit)
Model Specific Registers with RDMSR & WRMSR
PAE - Page Address Extensions
Machine Check Exception
COMPXCHG8B Instruction
APIC
SYSCALL/SYSRET or SYSENTER/SYSEXIT instructions
MTRR - Memory Type Range Registers
Global paging extension
Machine Check Architecture
Conditional Move Instruction
PAT - Page Attribute Table
PSE-36 - Page Size Extensions
20 - reserved
AMD MMX Instruction Extensions
MMX instructions
FXSAVE/FXRSTOR
25 - reserved
26 - reserved
27 - reserved
29 - reserved
3DNow! Instruction Extensions
3DNow instructions

Processor name string: AMD Opteron(tm) Processor 6174
L1 Cache Information:
2/4-MB Pages:
   Data TLB: associativity 255-way #entries 48
   Instruction TLB: associativity 255-way #entries 16
4-KB Pages:
   Data TLB: associativity 255-way #entries 48
   Instruction TLB: associativity 255-way #entries 32
L1 Data cache:
   size 64 KB associativity 2-way lines per tag 1 line size 64
L1 Instruction cache:
   size 64 KB associativity 2-way lines per tag 1 line size 64

L2 Cache Information:
2/4-MB Pages:
   Data TLB: associativity L2 off #entries 128
   Instruction TLB: associativity L2 off #entries 0
4-KB Pages:
   Data TLB: associativity 2-way #entries 0
   Instruction TLB: associativity 2-way #entries 0
   size 2 KB associativity L2 off lines per tag 129 line size 64

Advanced Power Management Feature Flags
Has temperature sensing diode
Maximum linear address: 48; maximum phys address 48


VM (with -cpu Opteron_G3)

 eax in    eax      ebx      ecx      edx
00000000 00000005 68747541 444d4163 69746e65
00000001 00000f61 00000800 80802001 078bfbfd
00000002 00000001 00000000 00000000 002c307d
00000003 00000000 00000000 00000000 00000000
00000004 00000000 00000000 00000000 00000000
00000005 00000000 00000000 00000003 00000000
80000000 80000008 68747541 444d4163 69746e65
80000001 00000f61 00000000 000000e5 2191abfd
80000002 20444d41 6574704f 206e6f72 78783332
80000003 65472820 2033206e 73616c43 704f2073
80000004 6f726574 0000296e 00000000 00000000
80000005 01ff01ff 01ff01ff 40020140 40020140
80000006 00000000 42004200 02008140 00000000
80000007 00000000 00000000 00000000 00000000
80000008 00003028 00000000 00000000 00000000

Vendor ID: "AuthenticAMD"; CPUID level 5

AMD-specific functions
Version 00000f61:
Family: 15 Model: 6 []

Standard feature flags 078bfbfd:
Floating Point Unit
Debugging Extensions
Page Size Extensions
Time Stamp Counter (with RDTSC and CR4 disable bit)
Model Specific Registers with RDMSR & WRMSR
PAE - Page Address Extensions
Machine Check Exception
COMPXCHG8B Instruction
APIC
SYSCALL/SYSRET or SYSENTER/SYSEXIT instructions
MTRR - Memory Type Range Registers
Global paging extension
Machine Check Architecture
Conditional Move Instruction
PAT - Page Attribute Table
PSE-36 - Page Size Extensions
19 - reserved
MMX instructions
FXSAVE/FXRSTOR
25 - reserved
26 - reserved
Generation: 15 Model: 6
Extended feature flags 2191abfd:
Floating Point Unit
Debugging Extensions
Page Size Extensions
Time Stamp Counter (with RDTSC and CR4 disable bit)
Model Specific Registers with RDMSR & WRMSR
PAE - Page Address Extensions
Machine Check Exception
COMPXCHG8B Instruction
APIC
SYSCALL/SYSRET or SYSENTER/SYSEXIT instructions
Global paging extension
Conditional Move Instruction
PAT - Page Attribute Table
20 - reserved
MMX instructions
FXSAVE/FXRSTOR
29 - reserved

Processor name string: AMD Opteron 23xx (Gen 3 Class Opteron)
L1 Cache Information:
2/4-MB Pages:
   Data TLB: associativity 1-way #entries 255
   Instruction TLB: associativity 1-way #entries 255
4-KB Pages:
   Data TLB: associativity 1-way #entries 255
   Instruction TLB: associativity 1-way #entries 255
L1 Data cache:
   size 64 KB associativity 2-way lines per tag 1 line size 64
L1 Instruction cache:
   size 64 KB associativity 2-way lines per tag 1 line size 64

L2 Cache Information:
2/4-MB Pages:
   Data TLB: associativity L2 off #entries 0
   Instruction TLB: associativity L2 off #entries 0
4-KB Pages:
   Data TLB: associativity 2-way #entries 0
   Instruction TLB: associativity 2-way #entries 0
   size 2 KB associativity L2 off lines per tag 129 line size 64

Advanced Power Management Feature Flags
Maximum linear address: 48; maximum phys address 40

^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-03-26 11:27 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-01-27 14:20 [Qemu-devel] live migration between amd fam15h-fam10h Markus Kovero
2014-02-01 17:45 ` Brian Jackson
2014-03-26 11:27 ` Alexandre DERUMIER
  -- strict thread matches above, loose matches on Subject: below --
2012-03-01 14:35 Vasilis Liaskovitis

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).