Re: [RFC PATCH 2/3 v3] perf: Implement Nehalem uncore pmu

[Lin Ming <ming.m.lin@intel.com>]

On Mon, 2011-01-17 at 16:44 +0800, Stephane Eranian wrote:
> On Mon, Jan 17, 2011 at 2:29 AM, Lin Ming <ming.m.lin@intel.com> wrote:
> > On Fri, 2011-01-14 at 01:14 +0800, Stephane Eranian wrote:
> >> Lin,
> >
> > Hi, Stephane,
> >
> > Sorry for late response, I'm just back from vacation.
> >
> >>
> >> On Thu, Dec 2, 2010 at 6:20 AM, Lin Ming <ming.m.lin@intel.com> wrote:
> >> > +static void uncore_pmu_enable_all(int nmi_core)
> >> > +{
> >> > +       u64 ctrl;
> >> > +
> >> > +       ctrl = ((1 << UNCORE_NUM_GENERAL_COUNTERS) - 1) | MSR_UNCORE_PERF_GLOBAL_CTRL_EN_FC0;
> >> > +
> >> > +       /* Route all interrupts to the first core that accesses uncore */
> >> > +       ctrl |= 1ULL << (48 + nmi_core);
> >> > +
> >> > +       wrmsrl(MSR_UNCORE_PERF_GLOBAL_CTRL, ctrl);
> >> > +}
> >>
> >> Are you sure nmi_core is always between 0-3 on a 4-core system and 0-5
> >> on a 6-core system?
> >> In other words, is that what topology_core_id(raw_smp_processor_id()) returns?
> >
> > I just have a look at a 6-core system, the core id is not 0-5
> >
> > $ cat /proc/cpuinfo |grep "core id"
> > core id         : 0
> > core id         : 1
> > core id         : 2
> > core id         : 8
> > core id         : 9
> > core id         : 10
> >
> > So we'd better route all the interrupts to the first core of the socket.
> >
> I recently realized the issue with 0,1,2,8,9,10. At the time I wrote the perfmon
> support for uncore, those systems did not exist. Sparse APIC id is a major pain
> for uncore PMU interrupt routine given the way UNC_GLOBAL_CTRL works.
> 
> Unfortunately, routing to core 0 (core_cpu_id=0) won't be enough in the
> presence of HOTPLUG CPU. Imagine I disable the first three 3 cores.
> Now you the cpu you have to play with are 8,9,10. You need to remap
> to a number between 0-5.

Good idea.

And I have confirmed that there are 6 interrupt target bits on my 6-core
Westmere machine, although documentation does not say this.

u64 val=0x3FULL << 48;
wrmsrl(MSR_UNCORE_PERF_GLOBAL_CTRL, val);

Thanks,
Lin Ming

> 
> 
> > Thanks for the catch.
> > Lin Ming
> >
> >>
> >> Note that, unfortunately, I have not seen documentation that says on
> >> 6-core system
> >> UNC_GLOBAL_CTRL has 6 interrupt target bits, but it would make sense.
> >>
> >>
> >> Otherwise, you will get a kernel panic when you wrmsr UNC_GLOBAL_CTRL.
> >>
> >> > +
> >> > +       if (uncore->n_events == 1) {
> >> > +               nmi_core = topology_core_id(raw_smp_processor_id());
> >> > +               uncore->nmi_core = nmi_core;
> >> > +               uncore_pmu_enable_all(nmi_core);
> >> > +       }
> >
> >
> >