Re: [PATCH] RFC: Linux: disable APERF/MPERF feature in PV kernels

[Andre Przywara <andre.przywara@amd.com>]

On 05/23/2012 03:26 PM, Konrad Rzeszutek Wilk wrote:
> On Wed, May 23, 2012 at 12:44:07AM +0200, Andre Przywara wrote:
>> On 05/22/2012 11:00 PM, Konrad Rzeszutek Wilk wrote:
>>> On Tue, May 22, 2012 at 11:02:01PM +0200, Andre Przywara wrote:
>>>> On 05/22/2012 07:18 PM, Konrad Rzeszutek Wilk wrote:
>>>>> On Tue, May 22, 2012 at 06:07:11PM +0200, Andre Przywara wrote:
>>>>>> Hi,
>>>>>>
>>>>>> while testing some APERF/MPERF semantics I discovered that this
>>>>>> feature is enabled in Xen Dom0, but is not reliable.
>>>>>> The Linux kernel's scheduler uses this feature if it sees the CPUID
>>>>>> bit, leading to costly RDMSR traps (a few 100,000s during a kernel
>>>>>> compile) and bogus values due to VCPU migration during the
>>>>>
>>>>> Can you point me to the Linux scheduler code that does this? Thanks.
>>>>
>>>> arch/x86/kernel/cpu/sched.c contains code to read out and compute
>>>> APERF/MPERF registers. I added a Xen debug-key to dump a usage
>>>> counter added in traps.c and thus could prove that it is actually
>>>> the kernel that accesses these registers.
>>>> As far as I understood this the idea is to learn about boosting and
>>>> down-clocking (P-states) to get a fairer view on the actual
>>>> computing time a process consumed.
>>>
>>> Looks like its looking for this:
>>>
>>> X86_FEATURE_APERFMPERF
>>>
>>> Perhaps masking that should do it? Something along this in enlighten.c:
>>>
>>>           cpuid_leaf1_edx_mask =
>>>                   ~((1<<   X86_FEATURE_MCE)  |  /* disable MCE */
>>>                     (1<<   X86_FEATURE_MCA)  |  /* disable MCA */
>>>                     (1<<   X86_FEATURE_MTRR) |  /* disable MTRR */
>>>                     (1<<   X86_FEATURE_ACC));   /* thermal monitoring
>>>
>>> would be more appropiate?
>>>
>>> Or is that attribute on a different leaf?
>>
>> Right, it is bit 0 on level 6. That's why I couldn't use any of the
>> predefined masks and I didn't feel like inventing a new one just for
>> this single bit.
>> We could as well explicitly use clear_cpu_cap somewhere, but I
>> didn't find any code place in the Xen tree already doing this,
>> instead it looks like it belongs to where I put it (we handle leaf 5
>> in a special way already here)
>
> OK, can you resend the patch please, looking similar to what you sent
> earlier, but do use a #define if possible (you can have the #define
> in that file) and an comment explaining why this is neccessary -
> and point to the Linux source code that uses this.

Well, I was about to do this and wanted to see if this has any 
performance impact - only to discover that 3.4 does not trigger it 
anymore. After some debugging it turns out the guy reading APERF/MPERF 
was not arch/x86/kernel/cpu/sched.c, but drivers/cpufreq/mperf.c. So 
with disabling cpufreq the only real user is gone already.
So the patch is kind of pointless as it on 3.4 with cpufreq already 
disabled. Remains to be investigated why sched.c is not called (I added 
a usage counter, it stays at zero).
To avoid future mis-uses of APERF/MPERF by the kernel, I'd like to add 
the patch anyway. I will send it again when I have a clearer picture of 
this.

.. snip..
> Looks like a patch to cpupower should be cooked up too?

I will contact the author.

Regards,
Andre.

-- 
Andre Przywara
AMD-OSRC (Dresden)
Tel: x29712