perf, x86: only do lbr init if bts is available

perf, x86: only do lbr init if bts is available

[David Ahern]

KVM recently added support for a version 2 PMU. When passing -cpu host 
as the CPU model for the guest we get an abnormal configuration from 
perf's perspective in that the guest identifies the processor as a 
Westmere or Nehalem (etc):

[    0.013998] Performance Events: Westmere events, Intel PMU driver.


but yet the processor does not have the debug store mechanisms 
(X86_FEATURE_DTES64 is not set) meaning there is no PEBS or BTS.

Right now the LBR init functions are run based on processor model which 
leads to attempts to write to LBR MSRs generating messages like:

[ 1170.125605] kvm: 6873: cpu0 unhandled rdmsr: 0x345
[ 1170.135354] kvm_set_msr_common: 54 callbacks suppressed
[ 1170.135612] kvm: 6873: cpu0 unhandled wrmsr: 0x680 data 0
[ 1170.135900] kvm: 6873: cpu0 unhandled wrmsr: 0x6c0 data 0
[ 1170.136163] kvm: 6873: cpu0 unhandled wrmsr: 0x681 data 0
[ 1170.136437] kvm: 6873: cpu0 unhandled wrmsr: 0x6c1 data 0
[ 1170.136703] kvm: 6873: cpu0 unhandled wrmsr: 0x682 data 0
[ 1170.136968] kvm: 6873: cpu0 unhandled wrmsr: 0x6c2 data 0
[ 1170.137227] kvm: 6873: cpu0 unhandled wrmsr: 0x683 data 0
[ 1170.137502] kvm: 6873: cpu0 unhandled wrmsr: 0x6c3 data 0
[ 1170.137797] kvm: 6873: cpu0 unhandled wrmsr: 0x684 data 0
[ 1170.138061] kvm: 6873: cpu0 unhandled wrmsr: 0x6c4 data 0


MSR 0x345 is MSR_IA32_PERF_CAPABILITIES; the 0x6XX MSRs are LBR from/to.

Question: is something like this acceptable/make sense -- to only setup 
lbr values if the BTS is detected?

diff --git a/arch/x86/kernel/cpu/perf_event_intel.c 
b/arch/x86/kernel/cpu/perf_event_intel.c
index 166546e..7ba4afa 100644
--- a/arch/x86/kernel/cpu/perf_event_intel.c
+++ b/arch/x86/kernel/cpu/perf_event_intel.c
@@ -1783,7 +1784,8 @@ __init int intel_pmu_init(void)
         memcpy(hw_cache_extra_regs, nehalem_hw_cache_extra_regs,
                sizeof(hw_cache_extra_regs));

-       intel_pmu_lbr_init_nhm();
+       if (x86_pmu.bts)
+           intel_pmu_lbr_init_nhm();

         x86_pmu.event_constraints = intel_nehalem_event_constraints;
         x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints;

David

Re: perf, x86: only do lbr init if bts is available

[Stephane Eranian]

On Thu, May 24, 2012 at 6:19 PM, David Ahern <dsahern@gmail.com> wrote:
> KVM recently added support for a version 2 PMU. When passing -cpu host as
> the CPU model for the guest we get an abnormal configuration from perf's
> perspective in that the guest identifies the processor as a Westmere or
> Nehalem (etc):
>
> [    0.013998] Performance Events: Westmere events, Intel PMU driver.
>
>
> but yet the processor does not have the debug store mechanisms
> (X86_FEATURE_DTES64 is not set) meaning there is no PEBS or BTS.
>
> Right now the LBR init functions are run based on processor model which
> leads to attempts to write to LBR MSRs generating messages like:
>
> [ 1170.125605] kvm: 6873: cpu0 unhandled rdmsr: 0x345
> [ 1170.135354] kvm_set_msr_common: 54 callbacks suppressed
> [ 1170.135612] kvm: 6873: cpu0 unhandled wrmsr: 0x680 data 0
> [ 1170.135900] kvm: 6873: cpu0 unhandled wrmsr: 0x6c0 data 0
> [ 1170.136163] kvm: 6873: cpu0 unhandled wrmsr: 0x681 data 0
> [ 1170.136437] kvm: 6873: cpu0 unhandled wrmsr: 0x6c1 data 0
> [ 1170.136703] kvm: 6873: cpu0 unhandled wrmsr: 0x682 data 0
> [ 1170.136968] kvm: 6873: cpu0 unhandled wrmsr: 0x6c2 data 0
> [ 1170.137227] kvm: 6873: cpu0 unhandled wrmsr: 0x683 data 0
> [ 1170.137502] kvm: 6873: cpu0 unhandled wrmsr: 0x6c3 data 0
> [ 1170.137797] kvm: 6873: cpu0 unhandled wrmsr: 0x684 data 0
> [ 1170.138061] kvm: 6873: cpu0 unhandled wrmsr: 0x6c4 data 0
>
>
> MSR 0x345 is MSR_IA32_PERF_CAPABILITIES; the 0x6XX MSRs are LBR from/to.
>
> Question: is something like this acceptable/make sense -- to only setup lbr
> values if the BTS is detected?
>
> diff --git a/arch/x86/kernel/cpu/perf_event_intel.c
> b/arch/x86/kernel/cpu/perf_event_intel.c
> index 166546e..7ba4afa 100644
> --- a/arch/x86/kernel/cpu/perf_event_intel.c
> +++ b/arch/x86/kernel/cpu/perf_event_intel.c
> @@ -1783,7 +1784,8 @@ __init int intel_pmu_init(void)
>        memcpy(hw_cache_extra_regs, nehalem_hw_cache_extra_regs,
>               sizeof(hw_cache_extra_regs));
>
> -       intel_pmu_lbr_init_nhm();
> +       if (x86_pmu.bts)
> +           intel_pmu_lbr_init_nhm();
>
Well, no. There is no connection between BTS and LBR and you're creating one.

Where is the wrmsr coming from? What we need to do is ensure that LBR is not
touched if we don't actually use it.

>        x86_pmu.event_constraints = intel_nehalem_event_constraints;
>        x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints;
>
> David

Re: perf, x86: only do lbr init if bts is available

[David Ahern]

On 5/24/12 10:35 AM, Stephane Eranian wrote:
> Well, no. There is no connection between BTS and LBR and you're creating one.

Ok. That was not clear to me from skimming the manual.
Then should it be tied to X86_FEATURE_DTES64?

diff --git a/arch/x86/kernel/cpu/perf_event_intel_lbr.c 
b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
index 520b426..1090ae6 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_lbr.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
@@ -640,6 +640,9 @@ void intel_pmu_lbr_init_core(void)
  /* nehalem/westmere */
  void intel_pmu_lbr_init_nhm(void)
  {
+   if (!boot_cpu_has(X86_FEATURE_DTES64))
+       return;
+
     x86_pmu.lbr_nr     = 16;
     x86_pmu.lbr_tos    = MSR_LBR_TOS;
     x86_pmu.lbr_from   = MSR_LBR_NHM_FROM;


> Where is the wrmsr coming from? What we need to do is ensure that LBR is not
> touched if we don't actually use it.

e.g., intel_pmu_lbr_init_nhm sets up lbr_nr, lbr_from, lbr_to and from, 
etc. Fhat I can tell intel_pmu_lbr_reset() gets invoked some where 
during the VM boot; I haven't traced how/when yet.

David

Re: perf, x86: only do lbr init if bts is available

[David Ahern]

On 5/24/12 10:41 AM, David Ahern wrote:
> On 5/24/12 10:35 AM, Stephane Eranian wrote:
>> Where is the wrmsr coming from? What we need to do is ensure that LBR
>> is not
>> touched if we don't actually use it.
>
> e.g., intel_pmu_lbr_init_nhm sets up lbr_nr, lbr_from, lbr_to and from,
> etc. Fhat I can tell intel_pmu_lbr_reset() gets invoked some where
> during the VM boot; I haven't traced how/when yet.

[    0.012998]  [<c04220ca>] intel_pmu_lbr_reset+0x2a/0xa0
[    0.012998]  [<c0423beb>] intel_pmu_cpu_starting+0x3b/0x100
[    0.012998]  [<c0424bd0>] ? allocate_shared_regs+0x20/0x50
[    0.012998]  [<c0932714>] x86_pmu_notifier+0xb3/0xc0
[    0.012998]  [<c0bed54c>] init_hw_perf_events+0x42c/0x456
[    0.012998]  [<c0403034>] do_one_initcall+0x34/0x170
[    0.012998]  [<c0bf2388>] ? native_smp_prepare_cpus+0x2b7/0x2f2
[    0.012998]  [<c0bed120>] ? check_bugs+0xf9/0xf9
[    0.012998]  [<c0be4833>] kernel_init+0x77/0x1a4
[    0.012998]  [<c0be47bc>] ? start_kernel+0x36d/0x36d
[    0.012998]  [<c094af7e>] kernel_thread_helper+0x6/0x10

Re: perf, x86: only do lbr init if bts is available

[Stephane Eranian]

On Thu, May 24, 2012 at 6:41 PM, David Ahern <dsahern@gmail.com> wrote:
> On 5/24/12 10:35 AM, Stephane Eranian wrote:
>>
>> Well, no. There is no connection between BTS and LBR and you're creating
>> one.
>
>
> Ok. That was not clear to me from skimming the manual.
> Then should it be tied to X86_FEATURE_DTES64?
>
No, it is unrelated to the Debug Store.
There is really nothing you can use to figure that out, not event IA32_DEBUGCTL.
I wish Intel had something in PERF_CAPABILITIES, but LBR was
originally considered
as a debug feature, I think. To the point that it is described in the
debug section not in
vol 3b with performance monitoring.

> diff --git a/arch/x86/kernel/cpu/perf_event_intel_lbr.c
> b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
> index 520b426..1090ae6 100644
> --- a/arch/x86/kernel/cpu/perf_event_intel_lbr.c
> +++ b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
> @@ -640,6 +640,9 @@ void intel_pmu_lbr_init_core(void)
>  /* nehalem/westmere */
>  void intel_pmu_lbr_init_nhm(void)
>  {
> +   if (!boot_cpu_has(X86_FEATURE_DTES64))
> +       return;
> +
>    x86_pmu.lbr_nr     = 16;
>    x86_pmu.lbr_tos    = MSR_LBR_TOS;
>    x86_pmu.lbr_from   = MSR_LBR_NHM_FROM;
>
>
>
>> Where is the wrmsr coming from? What we need to do is ensure that LBR is
>> not
>> touched if we don't actually use it.
>
>
> e.g., intel_pmu_lbr_init_nhm sets up lbr_nr, lbr_from, lbr_to and from, etc.
> Fhat I can tell intel_pmu_lbr_reset() gets invoked some where during the VM
> boot; I haven't traced how/when yet.
>
> David

Re: perf, x86: only do lbr init if bts is available

[Andi Kleen]

Stephane Eranian <eranian@google.com> writes:

> On Thu, May 24, 2012 at 6:41 PM, David Ahern <dsahern@gmail.com> wrote:
>> On 5/24/12 10:35 AM, Stephane Eranian wrote:
>>>
>>> Well, no. There is no connection between BTS and LBR and you're creating
>>> one.
>>
>>
>> Ok. That was not clear to me from skimming the manual.
>> Then should it be tied to X86_FEATURE_DTES64?
>>
> No, it is unrelated to the Debug Store.
> There is really nothing you can use to figure that out, not event IA32_DEBUGCTL.

One way to do it would be to reread DEBUGCTL after writing and see if
the bit really changed to 1. If not assume LBR is not available.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only

Re: perf, x86: only do lbr init if bts is available

[Peter Zijlstra]

On Thu, 2012-05-24 at 10:19 -0600, David Ahern wrote:
> KVM recently added support for a version 2 PMU. When passing -cpu host 
> as the CPU model for the guest we get an abnormal configuration from 
> perf's perspective in that the guest identifies the processor as a 
> Westmere or Nehalem (etc):
> 
> [    0.013998] Performance Events: Westmere events, Intel PMU driver.
> 
> 
> but yet the processor does not have the debug store mechanisms 
> (X86_FEATURE_DTES64 is not set) meaning there is no PEBS or BTS.
> 
> Right now the LBR init functions are run based on processor model which 
> leads to attempts to write to LBR MSRs generating messages like:

Right, as Stephane already noted, LBR is not an CPUID enumerated
feature, hence the only thing you can do is go by model. So if you set
qemu -cpu host but fail to implement all the architectural MSRs that go
with that model you get to keep the pieces.

Not really our problem.

Re: perf, x86: only do lbr init if bts is available

[Peter Zijlstra]

On Thu, 2012-05-24 at 19:18 +0200, Peter Zijlstra wrote:

> Right, as Stephane already noted, LBR is not an CPUID enumerated
> feature, hence the only thing you can do is go by model. So if you set
> qemu -cpu host but fail to implement all the architectural MSRs that go
> with that model you get to keep the pieces.

s/architectural//

They're called Model-Specific-Registers for a reason ;-)