conflict between tickless and perfmon2

conflict between tickless and perfmon2

[Stephane Eranian]

Hello,

We have identified a conflict between TICKLESS (CONFIG_NO_HZ) and
the current perfmon2 implementation. The problem impacts system-wide
sessions using timeout-based event set multiplexing.

Event set multiplexing allows monitoring tools to measure more events
than there are actual performance counters on the processor. Events
are grouped in sets which are then multiplexed onto the actual counters.
Switching can be triggered either by a timeout or by a counter overflow.
This is supported for per-thread and system-wide sessions.

For timeout-based switching, the duration expressed in nanoseconds is
meant to represent wall-clock time in system-wide mode, and execution
time in per-thread mode. Granularity is limited by HZ.

The current implementation for timeout is a simple hook on the timer
interrupt path in apic_*.c:smp_local_timer_interrupt(). Unfortunately,
this does not work when tickless is enabled: we get much less set
switches than expected on an idle system.

It looks like a solution would be to change the implementation of
timeout-based switching to use HR timers instead. Similar to what is
done for ITIMER_REAL and ITIMER_VIRTUAL.

Unless someone has a better proposal, I will experiment with this on
2.6.24-rc2.

Thanks.

-- 
-Stephane

Re: conflict between tickless and perfmon2

[Peter Zijlstra]

On Fri, 2007-11-09 at 02:44 -0800, Stephane Eranian wrote:
> Hello,
> 
> We have identified a conflict between TICKLESS (CONFIG_NO_HZ) and
> the current perfmon2 implementation. The problem impacts system-wide
> sessions using timeout-based event set multiplexing.
> 
> Event set multiplexing allows monitoring tools to measure more events
> than there are actual performance counters on the processor. Events
> are grouped in sets which are then multiplexed onto the actual counters.
> Switching can be triggered either by a timeout or by a counter overflow.
> This is supported for per-thread and system-wide sessions.
> 
> For timeout-based switching, the duration expressed in nanoseconds is
> meant to represent wall-clock time in system-wide mode, and execution
> time in per-thread mode. Granularity is limited by HZ.
> 
> The current implementation for timeout is a simple hook on the timer
> interrupt path in apic_*.c:smp_local_timer_interrupt(). Unfortunately,
> this does not work when tickless is enabled: we get much less set
> switches than expected on an idle system.
> 
> It looks like a solution would be to change the implementation of
> timeout-based switching to use HR timers instead. Similar to what is
> done for ITIMER_REAL and ITIMER_VIRTUAL.
> 
> Unless someone has a better proposal, I will experiment with this on
> 2.6.24-rc2.

Might help if you CC the tickless folks :-)

Re: conflict between tickless and perfmon2

[Thomas Gleixner]

On Fri, 9 Nov 2007, Peter Zijlstra wrote:

> On Fri, 2007-11-09 at 02:44 -0800, Stephane Eranian wrote:
> > Hello,
> > 
> > We have identified a conflict between TICKLESS (CONFIG_NO_HZ) and
> > the current perfmon2 implementation. The problem impacts system-wide
> > sessions using timeout-based event set multiplexing.
> > 
> > Event set multiplexing allows monitoring tools to measure more events
> > than there are actual performance counters on the processor. Events
> > are grouped in sets which are then multiplexed onto the actual counters.
> > Switching can be triggered either by a timeout or by a counter overflow.
> > This is supported for per-thread and system-wide sessions.
> > 
> > For timeout-based switching, the duration expressed in nanoseconds is
> > meant to represent wall-clock time in system-wide mode, and execution
> > time in per-thread mode. Granularity is limited by HZ.
> > 
> > The current implementation for timeout is a simple hook on the timer
> > interrupt path in apic_*.c:smp_local_timer_interrupt(). Unfortunately,
> > this does not work when tickless is enabled: we get much less set
> > switches than expected on an idle system.

What a surprise. :)

> > It looks like a solution would be to change the implementation of
> > timeout-based switching to use HR timers instead. Similar to what is
> > done for ITIMER_REAL and ITIMER_VIRTUAL.

Using a hrtimer is perfrectly fine, I'd say it's preferred over hooks in 
some code which has absoluty no guarantee of being executed periodically 
or even executed at all. OTOH it seems rather stupid to measure stuff 
while the system is idle and doing nothing.

	tglx

Re: conflict between tickless and perfmon2

[Stephane Eranian]

Thomas,

On Fri, Nov 09, 2007 at 07:40:31PM +0100, Thomas Gleixner wrote:
> 
> > > It looks like a solution would be to change the implementation of
> > > timeout-based switching to use HR timers instead. Similar to what is
> > > done for ITIMER_REAL and ITIMER_VIRTUAL.
> 
> Using a hrtimer is perfrectly fine, I'd say it's preferred over hooks in 
> some code which has absoluty no guarantee of being executed periodically 
> or even executed at all. OTOH it seems rather stupid to measure stuff 
> while the system is idle and doing nothing.
> 
I'll start looking into this soon. To answer your point about idle,
this is not because the core is idle that counters do not capture events
related to buses or caches for instance.

-- 
-Stephane

Re: [perfmon] Re: conflict between tickless and perfmon2

[Stephane Eranian]

Thomas,

On Fri, Nov 09, 2007 at 07:40:31PM +0100, Thomas Gleixner wrote:
> On Fri, 9 Nov 2007, Peter Zijlstra wrote:
> 
> 
> > > It looks like a solution would be to change the implementation of
> > > timeout-based switching to use HR timers instead. Similar to what is
> > > done for ITIMER_REAL and ITIMER_VIRTUAL.
> 
> Using a hrtimer is perfrectly fine, I'd say it's preferred over hooks in 
> some code which has absoluty no guarantee of being executed periodically 
> or even executed at all. OTOH it seems rather stupid to measure stuff 
> while the system is idle and doing nothing.
> 
I managed to switch the perfmon2 code to use hrtimer(CLOCK_MONOTONIC)
for system-wide (per-cpu) measurements. The code is simple and this allowed
me to do some more cleanups. I think this was a good suggestion and I made
the change rapidly.

Now, I must admit I don't quite understand how to make this work for per-thread
measurements where the timer would have to operate like ITIMER_VIRTUAL,i.e., only
run when the thread runs. I looked at the setitimer() code and I admit it is
not clear to me. What about CLOCK_THREAD_CPUTIME_ID, would it do what I need from
inside the kernel?

Thanks.

-- 
-Stephane