AW: RE: AMD P-States not recognized for Xen 3.3 and 3.4

AW: RE: AMD P-States not recognized for Xen 3.3 and 3.4

[Carsten Schiers]

So having read that, I have to summarize that working with the
Xen 3.2.1 / 2.6.18-xen-3.1-2 kernel / cpufreq=dom0-kernel team 
seems to be the best option.

Only drawback is for sure that typical tools, like powernowd is
only using Dom0 load. But I think I could possibly try to modify
powernowd or another tool to use what e.g. xentop is producing
as output, so that I tune up the system a bit when there is a
noticable bigger load than the 10% it is typically sleeping at.

Or am I wrong?

Thanks,
Carsten.

-----Ursprüngliche Nachricht-----
Von: Langsdorf, Mark [mailto:mark.langsdorf@amd.com] 
Gesendet: Mittwoch, 7. Januar 2009 21:22
An: Carsten Schiers; xen-devel@lists.xensource.com
Betreff: RE: [Xen-devel] AMD P-States not recognized for Xen 3.3 and 3.4

> > ... but was told that it is intentional that there's no 
> code to handle these.
> 
> Hm. And for what reason it does work in dom0-kernel? Because 
> it's done by powernow-k8.ko and userspace software?

Correct.  The family 0xf P-state interface is significantly
more complicated than the Family 0x10/architectural P-state
interface.  Porting it into the Xen hypervisor would possibly
introduce some more stability issues.  The architectural 
P-state interface is much stabler and smaller, and it made
sense to move it into Xen.  

Since the family 0xf P-state interface already existed in the
Linux kernel, it was easy enough to allow Linux dom0s to use
it for RevF systems.  Sadly, it's still got some problems 
due to the use of TSC as a time source.

-Mark Langsdorf
Operating System Research Center
AMD

AW: RE: AMD P-States not recognized for Xen 3.3 and 3.4

[Carsten Schiers]

Ah, when you just droped in, Mark: could I safely ignore the TSC
error messages or is there any risk included in it...

BR,
Carsten.

-----Ursprüngliche Nachricht-----
Von: Carsten Schiers 
Gesendet: Mittwoch, 7. Januar 2009 22:03
An: jbeulich; mark.langsdorf; xen-devel
Betreff: AW: RE: [Xen-devel] AMD P-States not recognized for Xen 3.3 and 
3.4

So having read that, I have to summarize that working with the
Xen 3.2.1 / 2.6.18-xen-3.1-2 kernel / cpufreq=dom0-kernel team 
seems to be the best option.

Only drawback is for sure that typical tools, like powernowd is
only using Dom0 load. But I think I could possibly try to modify
powernowd or another tool to use what e.g. xentop is producing
as output, so that I tune up the system a bit when there is a
noticable bigger load than the 10% it is typically sleeping at.

Or am I wrong?

Thanks,
Carsten.

-----Ursprüngliche Nachricht-----
Von: Langsdorf, Mark [mailto:mark.langsdorf@amd.com] 
Gesendet: Mittwoch, 7. Januar 2009 21:22
An: Carsten Schiers; xen-devel@lists.xensource.com
Betreff: RE: [Xen-devel] AMD P-States not recognized for Xen 3.3 and 3.4

> > ... but was told that it is intentional that there's no 
> code to handle these.
> 
> Hm. And for what reason it does work in dom0-kernel? Because 
> it's done by powernow-k8.ko and userspace software?

Correct.  The family 0xf P-state interface is significantly
more complicated than the Family 0x10/architectural P-state
interface.  Porting it into the Xen hypervisor would possibly
introduce some more stability issues.  The architectural 
P-state interface is much stabler and smaller, and it made
sense to move it into Xen.  

Since the family 0xf P-state interface already existed in the
Linux kernel, it was easy enough to allow Linux dom0s to use
it for RevF systems.  Sadly, it's still got some problems 
due to the use of TSC as a time source.

-Mark Langsdorf
Operating System Research Center
AMD




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

AW: RE: AMD P-States not recognized for Xen 3.3 and 3.4

[Carsten Schiers]

Oh, thanks, Niraj. 

I overlooked this initially, because this code is not implemented in waldi 3.1-2
kernel, but only in 3.3-1. I'll check it out tonight, as booting a new kernel 
per remote is a bit too dangerous.

This would be exactly what I need.

BR,
Carsten.

Von: Niraj Tolia 
Gesendet: Mit, 7.1.2009 22:53
An: Carsten Schiers 
Cc: jbeulich  ; mark.langsdorf  ; xen-devel 
Betreff: Re: RE: [Xen-devel] AMD P-States not recognized for Xen 3.3 and 3.4



On Wed, Jan 7, 2009 at 1:02 PM, Carsten Schiers  wrote:

So having read that, I have to summarize that working with the

Xen 3.2.1 / 2.6.18-xen-3.1-2 kernel / cpufreq=dom0-kernel team

seems to be the best option.



Only drawback is for sure that typical tools, like powernowd is

only using Dom0 load. But I think I could possibly try to modify

powernowd or another tool to use what e.g. xentop is producing

as output, so that I tune up the system a bit when there is a

noticable bigger load than the 10% it is typically sleeping at.



If you are using the in-kernel ondemand governor (cpufreq=dom0-kernel), it will look at complete system load and not just dom0. Look at cpufreq_ondemand.c in the 2.6.18-xen.hg tree for more info.


Cheers,
Niraj

 

Or am I wrong?



Thanks,

Carsten.



-----Ursprüngliche Nachricht-----

Von: Langsdorf, Mark [mailto:mark.langsdorf@amd.com]

Gesendet: Mittwoch, 7. Januar 2009 21:22

An: Carsten Schiers; xen-devel@lists.xensource.com

Betreff: RE: [Xen-devel] AMD P-States not recognized for Xen 3.3 and 3.4



> > ... but was told that it is intentional that there's no

> code to handle these.

>

> Hm. And for what reason it does work in dom0-kernel? Because

> it's done by powernow-k8.ko and userspace software?



Correct.  The family 0xf P-state interface is significantly

more complicated than the Family 0x10/architectural P-state

interface.  Porting it into the Xen hypervisor would possibly

introduce some more stability issues.  The architectural

P-state interface is much stabler and smaller, and it made

sense to move it into Xen.



Since the family 0xf P-state interface already existed in the

Linux kernel, it was easy enough to allow Linux dom0s to use

it for RevF systems.  Sadly, it's still got some problems

due to the use of TSC as a time source.



-Mark Langsdorf

Operating System Research Center

AMD









_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel



-- 
Niraj Tolia, Researcher, HP Labs
http://www.hpl.hp.com/personal/Niraj_Tolia/

AW: RE: AMD P-States not recognized for Xen 3.3 and 3.4

[Carsten Schiers]

OK, I have found the change and ported it into the 2.6.18-xen-3.1-2
waldi kernel. Fortunately, the Debian Xen 3.2.1 Hypervisor from Lenny,
which is used by my dsitribution (c't-Server), already implements the
xenpf_cpuidletime hypercall #53, which is needed. Some changes here 
and there where necessary, too.

Now, when setting the ondemand govenor, the system behaves as expected.

Still (as with userspace govenors), there are 

  - powernow-k8: error - out of sync, fix 0xd 0x2, vid 0xe 0x16
  - Timer ISR/0: Time went backwards: delta=

messages, when the p-states are changed. 4050e, as already mentioned,
dual-core, single socket.

Can I safely ignore them or do I have to take care somewhere...?

BR,
Carsten.

Von: Niraj Tolia 
Gesendet: Mit, 7.1.2009 22:53
An: Carsten Schiers 
Cc: jbeulich  ; mark.langsdorf  ; xen-devel 
Betreff: Re: RE: [Xen-devel] AMD P-States not recognized for Xen 3.3 and 3.4



On Wed, Jan 7, 2009 at 1:02 PM, Carsten Schiers  wrote:

So having read that, I have to summarize that working with the

Xen 3.2.1 / 2.6.18-xen-3.1-2 kernel / cpufreq=dom0-kernel team

seems to be the best option.



Only drawback is for sure that typical tools, like powernowd is

only using Dom0 load. But I think I could possibly try to modify

powernowd or another tool to use what e.g. xentop is producing

as output, so that I tune up the system a bit when there is a

noticable bigger load than the 10% it is typically sleeping at.



If you are using the in-kernel ondemand governor (cpufreq=dom0-kernel), it will look at complete system load and not just dom0. Look at cpufreq_ondemand.c in the 2.6.18-xen.hg tree for more info.


Cheers,
Niraj

 

Or am I wrong?



Thanks,

Carsten.



-----Ursprüngliche Nachricht-----

Von: Langsdorf, Mark [mailto:mark.langsdorf@amd.com]

Gesendet: Mittwoch, 7. Januar 2009 21:22

An: Carsten Schiers; xen-devel@lists.xensource.com

Betreff: RE: [Xen-devel] AMD P-States not recognized for Xen 3.3 and 3.4



> > ... but was told that it is intentional that there's no

> code to handle these.

>

> Hm. And for what reason it does work in dom0-kernel? Because

> it's done by powernow-k8.ko and userspace software?



Correct.  The family 0xf P-state interface is significantly

more complicated than the Family 0x10/architectural P-state

interface.  Porting it into the Xen hypervisor would possibly

introduce some more stability issues.  The architectural

P-state interface is much stabler and smaller, and it made

sense to move it into Xen.



Since the family 0xf P-state interface already existed in the

Linux kernel, it was easy enough to allow Linux dom0s to use

it for RevF systems.  Sadly, it's still got some problems

due to the use of TSC as a time source.



-Mark Langsdorf

Operating System Research Center

AMD









_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel



-- 
Niraj Tolia, Researcher, HP Labs
http://www.hpl.hp.com/personal/Niraj_Tolia/

RE: RE: AMD P-States not recognized for Xen 3.3 and 3.4

[Langsdorf, Mark]

> OK, I have found the change and ported it into the 2.6.18-xen-3.1-2
> waldi kernel. Fortunately, the Debian Xen 3.2.1 Hypervisor from Lenny,
> which is used by my dsitribution (c't-Server), already implements the
> xenpf_cpuidletime hypercall #53, which is needed. Some changes here 
> and there where necessary, too.
> 
> Now, when setting the ondemand govenor, the system behaves as 
> expected.
> 
> Still (as with userspace govenors), there are 
> 
>   - powernow-k8: error - out of sync, fix 0xd 0x2, vid 0xe 0x16
>   - Timer ISR/0: Time went backwards: delta=
> 
> messages, when the p-states are changed. 4050e, as already mentioned,
> dual-core, single socket.
> 
> Can I safely ignore them or do I have to take care somewhere...?

The out of sync errors are occurring because when you
perform a frequency change, the kernel's record of what
the fid and should be for that CPU are not matching
what it is reading from the physical CPU's MSRs.

I thought Keir and I had narrowed that problem down to
an issue in the implementation of VCPU pinning and fixed
it.  Do you have all of the following change sets?

http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/a37a8c474d8b
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/98de2b149423
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/7aaec9c0a213
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/8a407c41dfb0
http://xenbits.xensource.com/xen-unstable.hg?rev/2477e94450aa
http://xenbits.xensource.com/xen-unstable.hg?rev/c05ec22a9106
http://xenbits.xensource.com/xen-unstable.hg?rev/ddc9e6b2babb
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/fc406f9e9a0a
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/922fc4040264
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/ba918cb2cf75
http://xenbits.xensource.com/staging/xen-unstable.hg?rev/136f80d2195
http://xenbits.xensource.com/xen-unstable.hg?rev/21532468020b
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/26e1e96bd46a 

-Mark Langsdorf
Operating System Research Center
AMD