Re: CAP and performance problem

[George Dunlap <george.dunlap@eu.citrix.com>]

On 06/06/13 14:55, Massimo Canonico wrote:
>
> On 06/06/2013 03:02 PM, Juergen Gross wrote:
>> On 06.06.2013 14:52, Massimo Canonico wrote:
>>>
>>> On 06/06/2013 12:44 PM, George Dunlap wrote:
>>>> On 06/06/13 11:39, Juergen Gross wrote:
>>>>> On 06.06.2013 10:57, Massimo Canonico wrote:
>>>>>>
>>>>>> On 06/06/2013 10:37 AM, Dario Faggioli wrote:
>>>>>>> On mer, 2013-06-05 at 19:05 +0200, Massimo Canonico wrote:
>>>>>>>> Hi Dario,
>>>>>>>> and thanks for these test.
>>>>>>>>
>>>>>>>> I forgot to ask you which xen version has beed used for your 
>>>>>>>> experiments.
>>>>>>>>
>>>>>>> Yeah, me too! :-)
>>>>>>>
>>>>>>> I'm using xen-unstable, pulled yesterday (commit id
>>>>>>> e430510e5cbbfcdc1077739292def633e70fedea), compiled and 
>>>>>>> installed on a
>>>>>>> Debian unstable system. Dom0 kernel is a bit old, as it's a 3.6.
>>>>>>>
>>>>>>> What about you?
>>>>>> xen 4.2.2
>>>>>> kernel dom0: 3.8.11-200.fc18.x86_64
>>>>>
>>>>> Just had an idea: is there any other load on the system during 
>>>>> your test (other
>>>>> domains, dom0 load)? If not, it could be that the power management 
>>>>> is reducing
>>>>> the cpu speed during idle (when the cap applies). This could lead 
>>>>> to reduced
>>>>> performance overall.
>>>>>
>>>>> You can test this by setting the xen hypervisor boot option
>>>>>
>>>>> cpufreq=none
>>>>>
>>>>> and run your test again (with and without cap).
>>>>
>>>> Ah, genius Juergen! That would make total sense.
>>>>
>>>> -George
>>>>
>>> Unfortunately, this did not change much. I set "cpufreq=none" in the 
>>> boot line
>>
>> You added the boot parameter for the hypervisor, not dom0? 
> Fedora, after few seconds, asks you which kernel do you want to use. 
> You can add some parameter in the command line who launches the 
> kernel. So, I add "cpufreq=none" in the command line.
>> And (please forgive
>> my paranoia) you rebooted the complete system after that?
> Your paranoia is also mine, I always reboot my machines. Thanks for 
> asking.
>>> and restart my experiment.
>>> With no cap I got 298.029
>>> with cap=50% I got 910.272
>>> (average values of 3 experiments for each cap setting)
>>>
>>> dom0 load during the experiment is less than 1% (that says xentop)
>>
>> What was the load reported by xentop for your domu?
> My virtual machines is called rubis-web and during "cap=50%" 
> experiment I can see this values in xentop for this specific domain.
>>
>> Could you try:
>>
>> xl vcpu-list; sleep 10; xl vcpu-list
>>
>> when the test is running and post the output?
>>
>>
>> Juergen
>>
> here we go:
>
> [root@csitest ~]# xl vcpu-list; sleep 10; xl vcpu-list
> Name                                ID  VCPU   CPU State   Time(s) CPU 
> Affinity
> Domain-0                             0     0    0   ---      23.0 0
> Domain-0                             0     1    0   ---      10.5 0
> Domain-0                             0     2    0   ---       8.5 0
> Domain-0                             0     3    0   r--       6.8 0
> rubis-web                            1     0    2   r--    2968.5 2
> Name                                ID  VCPU   CPU State   Time(s) CPU 
> Affinity
> Domain-0                             0     0    0   ---      23.0 0
> Domain-0                             0     1    0   r--      10.6 0
> Domain-0                             0     2    0   ---       8.5 0
> Domain-0                             0     3    0   ---       6.8 0
> rubis-web                            1     0    2   r--    2973.5 2
>
> Concerning the George's question:
>> Have you checked your BIOS for performance settings?
> I'm not sure what you mean for "BIOS perfomance settings". To my best 
> knowledge, in the BIOS I have to be sure that the "hw virtualization" 
> is enabled.

Some BIOSes have settings that will automatically mess around with the 
performance settings of the processor.  This could be automatically 
slowing the core down because it's only 50% busy. This would be 
different on each BIOS.  Just take a quick look for anything that seems 
to say something about performance, and set it to "max" or "performance" 
mode (as opposed to say, power-saving or balanced mode).

You might also try playing around with the "turbo" mode if it's 
available -- maybe the "turbo" mode takes a bit of time to get going, 
and running at 50% never kicks it in.  If you disable "turbo" mode, you 
may find that with no cap you get 450s instead of 300s.

  -George