Re: [RFC PATCH 2/3 v3] perf: Implement Nehalem uncore pmu

Stephane Eranian
2011-01-17T10:56:18Z

On Mon, Jan 17, 2011 at 11:51 AM, Lin Ming <ming.m.lin@intel.com> wrote:
> On Mon, 2011-01-17 at 16:44 +0800, Stephane Eranian wrote:
>> On Mon, Jan 17, 2011 at 2:29 AM, Lin Ming <ming.m.lin@intel.com> wrote:
>> > On Fri, 2011-01-14 at 01:14 +0800, Stephane Eranian wrote:
>> >> Lin,
>> >
>> > Hi, Stephane,
>> >
>> > Sorry for late response, I'm just back from vacation.
>> >
>> >>
>> >> On Thu, Dec 2, 2010 at 6:20 AM, Lin Ming <ming.m.lin@intel.com> wrote:
>> >> > +static void uncore_pmu_enable_all(int nmi_core)
>> >> > +{
>> >> > +       u64 ctrl;
>> >> > +
>> >> > +       ctrl = ((1 << UNCORE_NUM_GENERAL_COUNTERS) - 1) | MSR_UNCORE_PERF_GLOBAL_CTRL_EN_FC0;
>> >> > +
>> >> > +       /* Route all interrupts to the first core that accesses uncore */
>> >> > +       ctrl |= 1ULL << (48 + nmi_core);
>> >> > +
>> >> > +       wrmsrl(MSR_UNCORE_PERF_GLOBAL_CTRL, ctrl);
>> >> > +}
>> >>
>> >> Are you sure nmi_core is always between 0-3 on a 4-core system and 0-5
>> >> on a 6-core system?
>> >> In other words, is that what topology_core_id(raw_smp_processor_id()) returns?
>> >
>> > I just have a look at a 6-core system, the core id is not 0-5
>> >
>> > $ cat /proc/cpuinfo |grep "core id"
>> > core id         : 0
>> > core id         : 1
>> > core id         : 2
>> > core id         : 8
>> > core id         : 9
>> > core id         : 10
>> >
>> > So we'd better route all the interrupts to the first core of the socket.
>> >
>> I recently realized the issue with 0,1,2,8,9,10. At the time I wrote the perfmon
>> support for uncore, those systems did not exist. Sparse APIC id is a major pain
>> for uncore PMU interrupt routine given the way UNC_GLOBAL_CTRL works.
>>
>> Unfortunately, routing to core 0 (core_cpu_id=0) won't be enough in the
>> presence of HOTPLUG CPU. Imagine I disable the first three 3 cores.
>> Now you the cpu you have to play with are 8,9,10. You need to remap
>> to a number between 0-5.
>
> Good idea.
>
> And I have confirmed that there are 6 interrupt target bits on my 6-core
> Westmere machine, although documentation does not say this.
>
> u64 val=0x3FULL << 48;
> wrmsrl(MSR_UNCORE_PERF_GLOBAL_CTRL, val);
>
Yes, at least, this part looks good ;->

> Thanks,
> Lin Ming
>
>>
>>
>> > Thanks for the catch.
>> > Lin Ming
>> >
>> >>
>> >> Note that, unfortunately, I have not seen documentation that says on
>> >> 6-core system
>> >> UNC_GLOBAL_CTRL has 6 interrupt target bits, but it would make sense.
>> >>
>> >>
>> >> Otherwise, you will get a kernel panic when you wrmsr UNC_GLOBAL_CTRL.
>> >>
>> >> > +
>> >> > +       if (uncore->n_events == 1) {
>> >> > +               nmi_core = topology_core_id(raw_smp_processor_id());
>> >> > +               uncore->nmi_core = nmi_core;
>> >> > +               uncore_pmu_enable_all(nmi_core);
>> >> > +       }
>> >
>> >
>> >
>
>
>