Re: perf tools miscellaneous questions

[Reid Kleckner <reid.kleckner-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

-lkml

On Mon, Nov 8, 2010 at 2:43 PM, Francis Moreau <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Vince Weaver <vweaver1-qKp7vQ+Mknf2fBVCVOL8/A@public.gmane.org> writes:
>
>> This is rapidly getting of topic, especially for linux-kernel
>
> Don't think so but feel free to remove LKML from Cc.
>
> [...]
>
>> Most events are poorly documented, if at all.  And the Linux kernel
>> predefined event list is loosely based upon the intel architectural
>> events, which not every processor has and I've heard from insiders saying
>> that you should be very careful for the results from those events.
>
> I agree, that's why I try to clarify some events.
>
> Perf tools are cool stuffs, IMHO, but it's pretty hard for me to
> interpret results. I tried to compare some numbers in my previous posts
> but I got some 'random' figures for now.
>
> Another example is given below where I'm trying to bench a 2 functions
> which do the same thing but differently.
>
>   $ perf stat -e cache-misses:u,l1d-loads-misses:u,cycles:u -p $(pgrep test)
>     C-c C-c
>    Performance counter stats for process id '30263':
>
>                406532  cache-misses
>               4986030  L1-dcache-load-misses
>             120247366  cycles
>
>           2.482196928  seconds time elapsed
>
>
>   $ perf stat -e cache-misses:u,l1d-loads-misses:u,cycles:u -p $(pgrep test)
>     C-c C-c
>    Performance counter stats for process id '30271':
>
>                459683  cache-misses
>               2513338  L1-dcache-load-misses
>             159968076  cycles
>
>           2.129021265  seconds time elapsed
>
> Which numbers are important here ? cache-misses ? L1-dcache-load-misses
> ?

Totally depends.  In this particular piece of code, you seem to have
improved your L1 hit rate, but you've hurt your hit rate somewhere
else, so the extra memory traffic has hurt you overall.  Also, it's
also helpful to look at the rate, and not just absolute numbers.  You
may be doing more L1 references in the second, so you just have more
memory traffic overall.

I don't know what level of cache the generic cache-misses and
-references refer to on your processor.  Unfortunately, you'd have to
go look up the source code and cross reference it with a manual to
know for 100%.  Having looked at the code, I can assert that it's an
event that has to do with the higher level caches, ie not L1, and
apparently it's not LLC on your machine.

Try comparing it to the numbers for L2-dcache-load-misses and
L2-dcache-store-misses.  IMO it's worth doing multiple runs to look at
*all* of the cache counters on a variety of workloads with known cache
behavior so you can get an understanding.

The reality is that these things aren't well documented either by the
manufacturer or the kernel developers, and the best way to understand
them right now is to run your own experiments.

Reid