Re: [RFC][PATCH 0/6] perf: x86 RDPMC and RDTSC support

[Arun Sharma <asharma@fb.com>]

On 12/5/11 12:16 PM, Arun Sharma wrote:
> On 12/2/11 2:22 PM, Stephane Eranian wrote:
>> Have you tried with inheritance disabled?
>> I don't remember if perf stat has it enabled buy default.
>
> Disabling inheritance using the -i switch as in:
>
> (for i in `seq 1 10`; do numactl --cpunodebind 1 perf stat -i -e
> instructions ./lat_ctx -P1 -s32k 4; done)
>
> shows numbers closer to baseline, since the threads used for
> benchmarking are not counting events.

I spent some more time looking into this. Running:

perf stat -e instructions  ./lat_ctx -P1 -s32k 4 -N 1000000 &
perf record -ag -- sleep 3

didn't show high cycles counts on any perf_events related functions in 
the context switch path. The only PMU related stuff that showed up had 
to do with x86_pmu_enable called from an IPI.

So just as an experiment I disabled perf_rotate_context() via:

--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -4252,8 +4252,6 @@ void scheduler_tick(void)
         curr->sched_class->task_tick(rq, curr, 0);
         raw_spin_unlock(&rq->lock);

-       perf_event_task_tick();
-

That seems to bring the lat_ctx numbers very close to baseline.

I suspect that some optimizations are possible in perf_rotate_context 
that don't involve enabling/disabling the PMU via an IPI for simple 
cases that involve one or two hardware events (eg: fixed counters).

  -Arun

Sample stack trace:

lat_ctx 29874 [012] 350729.824173: cycles:
         ffffffff8101149c intel_pmu_enable_all ([kernel.kallsyms])
         ffffffff810115f7 intel_pmu_nhm_enable_all ([kernel.kallsyms])
         ffffffff8100e877 x86_pmu_enable ([kernel.kallsyms])
         ffffffff810a99de perf_pmu_enable ([kernel.kallsyms])
         ffffffff8100d682 x86_pmu_commit_txn ([kernel.kallsyms])
         ffffffff810aa4f9 group_sched_in ([kernel.kallsyms])
         ffffffff810ab06d __perf_event_enable ([kernel.kallsyms])
         ffffffff810a8c93 remote_function ([kernel.kallsyms])
         ffffffff81065eec generic_smp_call_function_single_interrupt 
([kernel.kallsyms])
         ffffffff81018064 smp_call_function_single_interrupt 
([kernel.kallsyms])
         ffffffff81461fcb call_function_single_interrupt ([kernel.kallsyms])
                   401ec4 bread (/root/lat_ctx)