Re: AW: Re: AW: Re: Xen 3.4 strange behaviour as compared to Xen 3.3.1

[Carsten Schiers <carsten@schiers.de>]

Sorry, I think it could also be that the CPU mask is somehow modified in the
hypercall itself. Too much code for me to realy understand. 

Just a guess, but does Changeset 18898 take care of cpufreq=dom0-kernel case? It's
patching the hypercall 52 and the cpufreq_ondemand.c of Xen near a modfication of
a CPU mask variable, but no change in the cpufreq_ondemand.c of the Dom0 kernel. 
It also seems to fit into the time window, doesn't it?

BR,
Carsten.

----- Originalnachricht -----
Von: Carsten Schiers <carsten@schiers.de>
Gesendet: Die, 2.6.2009 17:28
An: Carsten Schiers <carsten@schiers.de> ; keir.fraser <keir.fraser@eu.citrix.com> ; xen-devel <xen-devel@lists.xensource.com> ; mark.langsdorf@amd.com
Betreff: AW: Re: AW: Re: AW: Re: [Xen-devel] Xen 3.4 strange behaviour as compared to Xen 3.3.1

I managed to narrowed the problem down to the fact that inside dbs_calculate_load, which 
submits a hypercall to receive the load of all CPUs and then loops through all of them
to calculate the combined load (which is a max of all (wall-idle)/wall scaled to 100).

It will not loop as policy->cpus, which is a bitmap of CPUs is zero. I am currently stuck 
and not able to find out, where this is set. It seems that in the drivers/cpufreq dir, no 
change was made at all, so it must be somewhere else.

I thought, maybe somebody that changed something that has to do with this bitmap might
come up with something like: uuuh, I missed that one...

I honestly have no idea what CPU hotpluging is, as I only have a simple mainstream board
with a normal dual-core CPU, but the only change that is somehow related to CPUs that 
I recently came accross was that. 

Comments, ideas, help?

BR,
Carsten.



----- Originalnachricht -----
Von: Carsten Schiers <carsten@schiers.de>
Gesendet: Sam, 30.5.2009 15:29
An: keir.fraser <keir.fraser@eu.citrix.com> ; xen-devel <xen-devel@lists.xensource.com>
Betreff: AW: Re: AW: Re: AW: Re: [Xen-devel] Xen 3.4 strange behaviour as compared to Xen 3.3.1

No it's the kernel of Xen 3.4.0 with the same config. I will try later with the 
older kernel, which is the one of 06.04.2009 (I have not realy understood the
tagging or versioning of the Xen kernel). The other combination would be Xen 3.3.1
with the latest kernel. That's how we narrow it down to hypervisor vs. kernel.

As I use cpufreq=dom0-kernel and enable cpufreq modules in drivers/cpufreq/Kconfig,
I guess it should use the hypercall for idle time. But to double check, I generated
load in dom0. No difference. It simply doesn't step upwards. The kernel with cpufreq
debugging is enabled. I will also try it, maybe it's loging something usefull.

BR,
Carsten.

-----Ursprüngliche Nachricht-----
Von: Keir Fraser [mailto:keir.fraser@eu.citrix.com] 
Gesendet: Samstag, 30. Mai 2009 11:30
An: Carsten Schiers; xen-devel
Betreff: Re: AW: Re: AW: Re: [Xen-devel] Xen 3.4 strange behaviour as compared to Xen 3.3.1

Are you using the exact same dom0 kernel as before, when it worked? One
theory would be that dom0 is looking at its own idle stats, and it probably
is pretty idle. So then it steps down the CPUs and keeps them down. When you
work the CPUs, are you working dom0 hard?

 -- Keir

On 30/05/2009 07:59, "Carsten Schiers" <carsten@schiers.de> wrote:

> Sorry, it's lowest frequency (1.0 out of 1.0, 1.8, 2.0, and 2.1 GHz).
> When booting,
> the CPU will be at 2.1 GHz, when switching the governor from performance
> to ondemand,
> this one will set it to 1.0 GHz, where it's sort of fixed. I can set it
> manually with
> cpufreq-set when switching to userspace governor, though.
> 
> So my guess is that the ondemand governor donesn't get te right
> information about idle
> time though the according hypercall. I recompiled with CPUFREQ DEBUG
> set. But as said
> earlier, I don't have that much knowledge about how to debug kernels.
> 
> BR,
> Carsten.
> 
> -----Ursprüngliche Nachricht-----
> Von: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
> Gesendet: Freitag, 29. Mai 2009 23:31
> An: Carsten Schiers; xen-devel
> Betreff: Re: AW: Re: [Xen-devel] Xen 3.4 strange behaviour as compared
> to Xen 3.3.1
> 
> Is lowest p-state lowest or highest frequency/voltage?
> 
>  -- Keir
> 
> On 29/05/2009 17:47, "Carsten Schiers" <carsten@schiers.de> wrote:
> 
>> Thanks Keir, I tried it out but there is no difference. And by the
> way:
>> it
>> is in lowest p-state and doesn't come up, even if under heavy load.
> Hmm.
>> 
>> BR,
>> Carsten.
>> 
>> -----Ursprüngliche Nachricht-----
>> Von: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
>> Gesendet: Freitag, 29. Mai 2009 17:02
>> An: Carsten Schiers; xen-devel
>> Betreff: Re: [Xen-devel] Xen 3.4 strange behaviour as compared to Xen
>> 3.3.1
>> 
>> On 29/05/2009 15:03, "Carsten Schiers" <carsten@schiers.de> wrote:
>> 
>>>   - as already reported, ondemand manager in dom0-kernel doesn't
> step,
>>> manual
>>>     setting works so it seems a bit like the communication between
>> dom0
>>> and
>>>     hypervisor regarding idle time is not working
>> 
>> Could be an interaction with C-state support, preferring deep sleep to
>> running at lower voltage/frequency? You could try no-cpuidle on Xen's
>> command line at boot time and see if that changes things.
>> 
>>  -- Keir
>> 
>>>   - all beside one domu use Xen 3.4.0 kernel, the one who uses it's
>>> customized
>>>     kernel won't start up as first domu. It simply hangs and this
>>> prevents also
>>>     all other domus (I all auto start them, no save/restore) don't
>> come
>>> up. When
>>>     I start the chain with a different one and this (with the
>> different
>>> kernel)
>>>     is started as #2 or #3, not problem
>>> 
>>>   - one domu is for vdr with three dvb pci cards passed trough. This
>>> one, when
>>>     started as the first one, will cause xentop to show 20% load.
> When
>>> restarted
>>>     or started as #2, the load is like with 3.3.1 at roughly 3-5%.
>> 
>> 
>> 
>> 
> 
> 
> 
>