Re: Performance difference between Xen versions

[Juergen Gross <juergen.gross@ts.fujitsu.com>]

On 05/11/11 08:08, Tian, Kevin wrote:
>> From: Juergen Gross [mailto:juergen.gross@ts.fujitsu.com]
>> Sent: Friday, May 06, 2011 9:49 PM
>>
>> On 05/03/11 05:06, Tian, Kevin wrote:
>>>> From: Keir Fraser
>>>> Sent: Monday, May 02, 2011 4:49 PM
>>>>
>>>> On 02/05/2011 09:23, "Juergen Gross"<juergen.gross@ts.fujitsu.com>
>> wrote:
>>>>> On 05/02/11 10:15, Jan Beulich wrote:
>>>>>>>>> On 02.05.11 at 10:00, Juergen
>>>>>>>>> Gross<juergen.gross@ts.fujitsu.com>
>>>> wrote:
>>>>>>> On the long run I'd like to make the cpufreq governor a feature of
>>>>>>> the cpupool. This would enable an administrator of a large Xen
>>>>>>> machine with a heterogeneous load to specify which domains should
>>>>>>> run at full speed and which are allowed to save energy at the cost
>>>>>>> of
>>>> latency.
>>>>>>> What do you think?
>>>>>> Certainly an interesting idea, with the question of how an
>>>>>> implementation of this would look like.
>>>>> Let me do some research work first :-) I hope to make a proposal soon.
>>>> I think it's a good idea, and it should be quite possible to implement cleanly.
>>>>
>>> yes, this is a good direction. Actually there have been several papers
>>> around this topic before. Basically it's a reasonable choice to inject
>>> higher level knowledge together with VMM heuristics, as in
>>> virtualization or cloud we usually have an intelligent stack which
>>> needs to understand many high level requirements/characteristics
>>> already. :-)
>>>
>> Okay, I think I understand the basic mechanisms of cpufreq stuff now :-) I
>> propose the following changes:
>>
>> - Cpupools get a new parameter "cpufreq" which is similar to the hypervisor
>>     boot parameter. It is valid if the hypervisor is responsible for cpufreq
>>     handling (this excludes cases cpufreq=none and cpufreq=dom0-kernel)
>> - Cpupool0 is initialized with the boot parameter settings, new cpupools are
>>     created with the cpupool0 settings, they get their new cpufreq parameters
>>     via libxl later (this avoids changing the interface for cpupool creation, I only
>>     need a new interface to set the cpufreq parameters for a cpupool, which
>>     can be used for changing the settings, too. This interface could take the
>>     cpufreq parameters as text string resulting in support of exactly the same
>>     parameters as the hypervisor).
>> - cpufreq_policy is only spanning multiple cpus of one cpupool (if at all). This
>>     requires a check for the max frequency to be set in a frequency domain
>>     if the frequency of a processor is changing. This is similar to the ondemand
>>     governor, but might cross cpufreq_policy boundaries.
>>
>> Did I miss anything? Any other suggestions?
>>
> I'm a little bit concerned whether cpupool is a good logical entity to bundle a
> cpufreq policy. Basically the question is how do you define a cpupool, socket based,
> core based, or thread based? fully controllable by the admin?

Cpupools can be either configured by the admin or you can create
one cpupool per numa node (applicable to "big" machines only).

> the reason why I ask this question is because a frequency scaling is fundamentally
> a hardware attribute, and there may have some cross-dependencies among
> different cores/threads within same package. In some implementations, e.g. you
> may only have one core entering a lower frequency when all other cores within
> same packages request to enter same or lower frequency. Such low level
> dependency may be either managed by the firmware level automatically, or
> fully coordinated by the cpufreq driver. But whatever model, the scaling dependency
> may not be the same range as what user may want to define a cpupool.

It might be a good idea to add some information about frequency domains
to e.g. "xl info" output.

> Possibly you may want to warn the user if the low level cpufreq dependency is
> broken by the user-defined pool.

That's what I want to do.
Thanks for your thoughts.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html