Re: [patch 0/3] [Announcement] Performance Counters for Linux

[Rob Fowler <rjf@renci.org>]

My reaction is more from a downstream tool developer and end user perspective.

What I don't see in the new proposal is support for real end users of hardware
performance counter information.  There is a long-existing community that is using the
counters, including the hardware designers, driver writers, tool developers, and
performance tuning specialists working for both vendors and end customers.   Not
everyone is in the same camp, as each the hardware capabilities change from revision to
revision of the chips as features are added, architectures evolve, and implementations are
cleaned up.  System vendors have their own tools and developers (SpeedShop, Vtune, Tprof, Sun Studio
Code Analyst, etc). There are academic and open source efforts with long histories (PAPI,
oprofile, HPCToolkit (Rice, not IBM), etc). We've lived with proprietary drivers/APIs and with
a succession of open-source drivers (pci, perfctr, oprofile, perfmon).  (My apologies to
readers/developers whose favorite tool(s) I haven't mentioned.)  Out-and-out religious wars
have not erupted, but there are a lot of healthy disagreements. A significant part of this
community has been converging around Perfmon2/3, not because it is a thing of beauty, but
because it is a tool that exposes the full HPM capabilities (which are often ugly) in a useful
way for a community of tool developers and end users.

Before considering this new proposal seriously, I'd need to see it proven.  This means
that it needs to be developed, by the proposers, enough to be used seriously.  I've
got collaborators that measure compute resources in units of tens of TeraFLOP-years, so
my definition of "seriously" is that the HPM tool chain has to work with low overhead
on huge clusters of multi-core, multi-socket machines and it has to be able to provide
performance insights that will let us get even more performance out of applications
that already do pretty well.  Google and other large users have similar notions of "serious".

Here's my set of strawman requirements:

-- Can it support a *completely* functional PAPI?  There are a lot of tools (HPCToolkit,
    TAU, etc.) built on this layer.

-- Means to support IBS/EBS profiling and efficiently record execution contexts?  Can it
    support event-based call stack profiling?

-- Can it supplant or support oprofile by supporting the tools (Code Analyst, etc) that
    depend on it?

-- Kernel and daemon profiling capabilities?

-- Does it have sufficiently low overhead?  Six years ago DCPI/ProfileMe was capable of
    collecting around 5000 samples/second on a quad socket 1GHz Alpha EV67 system with
    about a 1.5% overhead.  That's the gold standard. Oprofile and pfmon are not far off
    that mark.

-- Does it even scale within one box?  My workhorse systems today are quad-socket Barcelonas.
    I'm reliably using multiple, cooperating (Some measure on-core, others measure off-core events.)
    instances of pfmon to collect profiles using  all 64 (4 per core x 16 cores) counters
    productively with low overhead.  Real soon now I will have similar expectations
    regarding multi-socket Nehalems where the resources will be 7 (heterogeneous) counters per
    core plus 8 "uncore" counters (I prefer "nest", Alex Mericas' terminology.) per socket.


Regards,
Rob


stephane eranian wrote:
> Hello,
> 
> I have been reading all the threads after this unexpected announcement
> of a competing proposal for an interface to access the performance counters.
> I would like to respond to some of the things I have seen.
> 

  <<<<<< Details of Stephane's comment's elided >>>>>>

> 
> In summary, although the idea of simplifying tools by moving the
> complexity elsewhere is legitimate, pushing it down to the kernel
> is the wrong approach in my opinion, perfmon has avoided that as much
> as possible for good reasons. We have shown , with libpfm,
> that a large part of complexity can easily be encapsulated into a user
> library. I also don't think the approach of managing events
> independently of each others works for all processors. As pointed out
> by others, there are other factors at stake and they may not
> even be on the same core.
> 
> S. Eranian
> 
> ------------------------------------------------------------------------------
> SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
> The future of the web can't happen without you.  Join us at MIX09 to help
> pave the way to the Next Web now. Learn more and register at
> http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
> _______________________________________________
> perfmon2-devel mailing list
> perfmon2-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/perfmon2-devel

-- 
Robert J. Fowler
Chief Domain Scientist, HPC
Renaissance Computing Institute
The University of North Carolina at Chapel Hill
100 Europa Dr, Suite 540
Chapel Hill, NC 27517
V: 919.445.9670
F: 919 445.9669
rjf@renci.org

------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you.  Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/