Problem with booting 2.6.32.16 pvops DomU

Problem with booting 2.6.32.16 pvops DomU

[Carsten Schiers]

Hi,

I tried to boot a 64 Bit 2.6.32.16 pvops DomU from Jeremy's git on Xen 3.4.4-rc1-pre 
and 64 Bit 2.6.18.8 Dom0.

It will hang very quickly after 0.5 secs and produce the following output on xm dmesg:

(XEN) traps.c:2230:d25 Domain attempted WRMSR 00000000c0010004 from 
00009310:79804ace to 00000000:00000000

(XEN) traps.c:2230:d25 Domain attempted WRMSR 00000000c0010000 from 
00000009:0487e489 to 00000000:00430076


BR,
Carsten.



----- Originalnachricht -----
Von: Carsten Schiers <carsten@schiers.de>
Gesendet: Mon, 5.7.2010 11:09
An: xen-devel@lists.xensource.com
Betreff: [Xen-devel] Question on xenpm

Dear all,

after having upgraded my server from AMD 4050e to X4 640, I now use cpufreq=xen and had
to adapt a munin script (monitoring tool) to display the residency in the different P-states.
This script uses /sys/device/system/cpu/cpu0/cpufreq to read out the information, whereas
I now use xenpm get-cpufreq-state.

Before I noticed that the CPU is in highest possible P-state (lowest frequences) nearly all 
of the time, and a minimal percentage in the lowest. Now I can see a 50/50 distribution. 
Interesting enough, the xenpm get-cpuidle-state will show that the CPUs are at aprox. 90%
in C1 idle state.

Can there be a difference in how the two methods to collect the info are working? I mean 
something like xenpm will not count residency when in C1, but cpufreq driver will normaly
do?

BR,
Carsten.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Problem with booting 2.6.32.16 pvops DomU

[Jeremy Fitzhardinge]

On 07/07/2010 02:07 AM, Carsten Schiers wrote:
> Hi,
>
> I tried to boot a 64 Bit 2.6.32.16 pvops DomU from Jeremy's git on Xen 3.4.4-rc1-pre 
> and 64 Bit 2.6.18.8 Dom0.
>
> It will hang very quickly after 0.5 secs and produce the following output on xm dmesg:
>
> (XEN) traps.c:2230:d25 Domain attempted WRMSR 00000000c0010004 from 
> 00009310:79804ace to 00000000:00000000
>
> (XEN) traps.c:2230:d25 Domain attempted WRMSR 00000000c0010000 from 
> 00000009:0487e489 to 00000000:00430076
>   

Do you get any other console output?  Could you boot with
"earlyprink=xen" (and "console=hvc0" if you don't already) on the kernel
command line to see if any more comes out.

Those two MSRs relate to the K8 performance counter subsystem, but I
can't see any obviously relevent changes in 2.6.32.15->16 which might
cause this regression.

It might also be useful to use xenctx to work out where the hang is.

Thanks,
    J

>
> BR,
> Carsten.
>
>
>
> ----- Originalnachricht -----
> Von: Carsten Schiers <carsten@schiers.de>
> Gesendet: Mon, 5.7.2010 11:09
> An: xen-devel@lists.xensource.com
> Betreff: [Xen-devel] Question on xenpm
>
> Dear all,
>
> after having upgraded my server from AMD 4050e to X4 640, I now use cpufreq=xen and had
> to adapt a munin script (monitoring tool) to display the residency in the different P-states.
> This script uses /sys/device/system/cpu/cpu0/cpufreq to read out the information, whereas
> I now use xenpm get-cpufreq-state.
>
> Before I noticed that the CPU is in highest possible P-state (lowest frequences) nearly all 
> of the time, and a minimal percentage in the lowest. Now I can see a 50/50 distribution. 
> Interesting enough, the xenpm get-cpuidle-state will show that the CPUs are at aprox. 90%
> in C1 idle state.
>
> Can there be a difference in how the two methods to collect the info are working? I mean 
> something like xenpm will not count residency when in C1, but cpufreq driver will normaly
> do?
>
> BR,
> Carsten.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>   

Re: Problem with booting 2.6.32.16 pvops DomU

[GiovanniB]

Hello

same problem here.

Supermicro Board
XEON E5506

xm dmesg:
(XEN) traps.c:2230:d11 Domain attempted WRMSR 000000000000008b from
00000011:00000000 to 00000000:00000000.
(XEN) traps.c:2230:d11 Domain attempted WRMSR 000000000000008b from
00000011:00000000 to 00000000:00000000.
(XEN) traps.c:2230:d11 Domain attempted WRMSR 000000000000008b from
00000011:00000000 to 00000000:00000000.
(XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
00000011:00000000 to 00000000:00000000.
(XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
00000011:00000000 to 00000000:00000000.
(XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
00000011:00000000 to 00000000:00000000.
(XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
00000011:00000000 to 00000000:00000000


domU hangs here with 400% CPU load (4 vcpus)
(same problem with 1 vcpu)

...
[    1.807477] kjournald starting.  Commit interval 5 seconds
[    2.005607] EXT3 FS on xvda1, internal journal
[    2.005618] EXT3-fs: recovery complete.
[    2.005626] EXT3-fs: mounted filesystem with writeback data mode.
[    2.005643] VFS: Mounted root (ext3 filesystem) on device 202:1.
[    2.005727] Freeing unused kernel memory: 580k freed
[    2.005892] Write protecting the kernel read-only data: 8160k

here it hangs

2.6.32.15 there is no problem. I have used the same .config for 2.6.32.16.

I have recompiled it with no energy save option, same error.
C1 and else are disabled in BIOS.

Regards
Gio


Jeremy Fitzhardinge wrote:
> 
> On 07/07/2010 02:07 AM, Carsten Schiers wrote:
>> Hi,
>>
>> I tried to boot a 64 Bit 2.6.32.16 pvops DomU from Jeremy's git on Xen
>> 3.4.4-rc1-pre 
>> and 64 Bit 2.6.18.8 Dom0.
>>
>> It will hang very quickly after 0.5 secs and produce the following output
>> on xm dmesg:
>>
>> (XEN) traps.c:2230:d25 Domain attempted WRMSR 00000000c0010004 from 
>> 00009310:79804ace to 00000000:00000000
>>
>> (XEN) traps.c:2230:d25 Domain attempted WRMSR 00000000c0010000 from 
>> 00000009:0487e489 to 00000000:00430076
>>   
> 
> Do you get any other console output?  Could you boot with
> "earlyprink=xen" (and "console=hvc0" if you don't already) on the kernel
> command line to see if any more comes out.
> 
> Those two MSRs relate to the K8 performance counter subsystem, but I
> can't see any obviously relevent changes in 2.6.32.15->16 which might
> cause this regression.
> 
> It might also be useful to use xenctx to work out where the hang is.
> 
> Thanks,
>     J
> 
>>
>> BR,
>> Carsten.
>>
>>
>>
>> ----- Originalnachricht -----
>> Von: Carsten Schiers <carsten@schiers.de>
>> Gesendet: Mon, 5.7.2010 11:09
>> An: xen-devel@lists.xensource.com
>> Betreff: [Xen-devel] Question on xenpm
>>
>> Dear all,
>>
>> after having upgraded my server from AMD 4050e to X4 640, I now use
>> cpufreq=xen and had
>> to adapt a munin script (monitoring tool) to display the residency in the
>> different P-states.
>> This script uses /sys/device/system/cpu/cpu0/cpufreq to read out the
>> information, whereas
>> I now use xenpm get-cpufreq-state.
>>
>> Before I noticed that the CPU is in highest possible P-state (lowest
>> frequences) nearly all 
>> of the time, and a minimal percentage in the lowest. Now I can see a
>> 50/50 distribution. 
>> Interesting enough, the xenpm get-cpuidle-state will show that the CPUs
>> are at aprox. 90%
>> in C1 idle state.
>>
>> Can there be a difference in how the two methods to collect the info are
>> working? I mean 
>> something like xenpm will not count residency when in C1, but cpufreq
>> driver will normaly
>> do?
>>
>> BR,
>> Carsten.
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>   
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 

-- 
View this message in context: http://old.nabble.com/Problem-with-booting-2.6.32.16-pvops-DomU-tp29094017p29105236.html
Sent from the Xen - Dev mailing list archive at Nabble.com.

Re: Problem with booting 2.6.32.16 pvops DomU

[Jeremy Fitzhardinge]

On 07/08/2010 02:51 AM, GiovanniB wrote:
> Hello
>
> same problem here.
>
> Supermicro Board
> XEON E5506
>
> xm dmesg:
> (XEN) traps.c:2230:d11 Domain attempted WRMSR 000000000000008b from
> 00000011:00000000 to 00000000:00000000.
> (XEN) traps.c:2230:d11 Domain attempted WRMSR 000000000000008b from
> 00000011:00000000 to 00000000:00000000.
> (XEN) traps.c:2230:d11 Domain attempted WRMSR 000000000000008b from
> 00000011:00000000 to 00000000:00000000.
> (XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
> 00000011:00000000 to 00000000:00000000.
> (XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
> 00000011:00000000 to 00000000:00000000.
> (XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
> 00000011:00000000 to 00000000:00000000.
> (XEN) traps.c:2230:d12 Domain attempted WRMSR 000000000000008b from
> 00000011:00000000 to 00000000:00000000
>
>
> domU hangs here with 400% CPU load (4 vcpus)
> (same problem with 1 vcpu)
>
> ...
> [    1.807477] kjournald starting.  Commit interval 5 seconds
> [    2.005607] EXT3 FS on xvda1, internal journal
> [    2.005618] EXT3-fs: recovery complete.
> [    2.005626] EXT3-fs: mounted filesystem with writeback data mode.
> [    2.005643] VFS: Mounted root (ext3 filesystem) on device 202:1.
> [    2.005727] Freeing unused kernel memory: 580k freed
> [    2.005892] Write protecting the kernel read-only data: 8160k
>
> here it hangs
>
> 2.6.32.15 there is no problem. I have used the same .config for 2.6.32.16.
>
> I have recompiled it with no energy save option, same error.
> C1 and else are disabled in BIOS.
>   

Does this patch help?

Thanks,
    J

 arch/x86/include/asm/cmpxchg_64.h |   16 ++++++++--------
 1 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/cmpxchg_64.h b/arch/x86/include/asm/cmpxchg_64.h
index 485ae41..70ac0a4 100644
--- a/arch/x86/include/asm/cmpxchg_64.h
+++ b/arch/x86/include/asm/cmpxchg_64.h
@@ -26,26 +26,26 @@ extern void __cmpxchg_wrong_size(void);
 	switch (size) {							\
 	case 1:								\
 		asm volatile("xchgb %b0,%1"				\
-			     : "=q" (__x)				\
-			     : "m" (*__xg(ptr)), "0" (__x)		\
+			     : "=q" (__x), "+m" (*__xg(ptr))		\
+			     : "0" (__x)				\
 			     : "memory");				\
 		break;							\
 	case 2:								\
 		asm volatile("xchgw %w0,%1"				\
-			     : "=r" (__x)				\
-			     : "m" (*__xg(ptr)), "0" (__x)		\
+			     : "=r" (__x), "+m" (*__xg(ptr))		\
+			     : "0" (__x)				\
 			     : "memory");				\
 		break;							\
 	case 4:								\
 		asm volatile("xchgl %k0,%1"				\
-			     : "=r" (__x)				\
-			     : "m" (*__xg(ptr)), "0" (__x)		\
+			     : "=r" (__x), "+m" (*__xg(ptr))		\
+			     : "0" (__x)				\
 			     : "memory");				\
 		break;							\
 	case 8:								\
 		asm volatile("xchgq %0,%1"				\
-			     : "=r" (__x)				\
-			     : "m" (*__xg(ptr)), "0" (__x)		\
+			     : "=r" (__x), "+m" (*__xg(ptr))		\
+			     : "0" (__x)				\
 			     : "memory");				\
 		break;							\
 	default:							\