Re: [RFC][PATCH 5/5] perfcounter: Add support for kernel hardware breakpoints

[Frederic Weisbecker <fweisbec@gmail.com>]

On Mon, Jul 27, 2009 at 10:53:37AM +0200, Peter Zijlstra wrote:
> On Sun, 2009-07-26 at 05:27 +0530, K.Prasad wrote:
> 
> > I don't claim to have understood the requirements of perf-counters
> > fully, but I sense the persistence of a few doubts about the register
> > allocation mechanism of the hw-breakpoints API...thought it might be
> > worthwhile to briefly explain.
> > 
> > A few notes about the same:
> > 
> > Kernel-space breakpoints: They are system-wide i.e. one kernel-space
> > breakpoint request consumes one debug register on 'all' CPUs of the
> > system.
> > User-space breakpoints: Requests are stored in per-thread data
> > structures. Written onto the debug register only when scheduled-into
> > the CPU, and are removed when another task using the debug registers is
> > scheduled onto the CPU.
> > Debug register allocation mechanism: register request succeeds only when
> > the availability of a debug register is guaranteed (for both user and
> > kernel space). Allocation on a first-come-first-serve basis, no
> > priorities for register requests and hence no pre-emption of requests.
> > 
> > In short the available debug registers are divided into:
> > 
> > # of debug registers = # of kernel-space requests + MAX(# of all
> > user-space requests) + freely available debug registers.
> > 
> > Thus on an x86 system with 4 debug registers, if there exists 1
> > kernel-space request it consumes one debug register on all CPUs in the
> > system. The remaining 3 debug registers can be consumed by user-space
> > i.e. upto 3 user-space requests per-thread.
> > 
> > There is no restriction on how many debug registers can be consumed by
> > the kernel- or user-space requests and it is entirely dependent on the
> > number of free registers available for use then.
> 
> Right, this is an utter miss-match for what perf counters would want.
> 
> Firstly, you seem to have this weird split of kernel/userspace
> breakpoints. Perf counters looks at things in a per-cpu fashion, so the
> all-cpus kernel breakpoint stuff is useless. Also, from perf counters'
> POV its perfectly reasonable to have a per-task kernel breakpoint.


Hmm, indeed. The breakpoint api lacks these services for perf counter,
although targetting specific cpus seems a (somewhat) trivial add-on to me on the
bp API.

Also I forgot the kernel-only profiling things.

 
> Secondly, perf counters wants to schedule the per task breakpoints
> because we can optimize the context switch, saving lots of these MSR
> writes under some common scenarios.


Ok with that.
But if you have an efficient pmu scheduling, it may be still interesting
to make it a seperate core entity. The bp API may find some profit in it.

The breakpoint registers also require specific treatments with several registers
magic, all organized inside the arch API which also cares about cpu hotplug and
kexec things.


> Thirdly, we can multiplex perf counters beyond their hardware maximum,
> something you simply cannot do for a debug interface.


Again, I'm stuck in what you mean by multiplexing here :-)


> Like I said, please use the raw per-cpu breakpoint interface for perf
> counters and connect that with the minimally required reservation you
> need to make your other thing work.
>
> You simply cannot put perf-counter breakpoints on top of whatever virt
> layer you created going by what you say it is.


Well, indeed it seems perfcounter handles the counters to cover very specific
needs, uncovered by the debug API we have.
I guess we can create a struct pmu for the breakpoints and plug it with
some low level add-on in the breakpoint API.