Re: perf hw in kexeced kernel broken in tip

[Peter Zijlstra <peterz@infradead.org>]

On Wed, 2010-12-01 at 11:06 -0500, Vivek Goyal wrote:
> On Wed, Dec 01, 2010 at 12:27:47PM +0100, Peter Zijlstra wrote:
> > On Wed, 2010-12-01 at 00:00 -0800, Yinghai Lu wrote:
> > > First kernel:
> > > [    1.139418] calling  init_hw_perf_events+0x0/0xb77 @ 1
> > > [    1.159111] Performance Events: PEBS fmt1+, Nehalem events, Intel PMU
> > > driver.
> > > [    1.159567] ... version:                3
> > > [    1.179121] ... bit width:              48
> > > [    1.179353] ... generic registers:      4
> > > [    1.179593] ... value mask:             0000ffffffffffff
> > > [    1.199211] ... max period:             000000007fffffff
> > > [    1.199554] ... fixed-purpose events:   3
> > > [    1.219108] ... event mask:             000000070000000f
> > > [    1.219454] initcall init_hw_perf_events+0x0/0xb77 returned 0 after
> > > 11719 usecs
> > > 
> > > .....
> > > [   20.220997] checking TSC synchronization [CPU#0 -> CPU#11]: passed.
> > > [   20.260818] NMI watchdog enabled, takes one hw-pmu counter.
> > > 
> > > kexeced kernel.
> > > 
> > > 
> > > [    1.169470] calling  init_hw_perf_events+0x0/0xb77 @ 1
> > > [    1.189265] Performance Events: PEBS fmt1+, Nehalem events, Broken
> > > PMU hardware detected, software events only.
> > > ...
> > > [   21.010407] NMI watchdog failed to create perf event on cpu14:
> > > fffffffffffffffe
> > > 
> > > caused by:
> > 
> > *sigh*, and people ask me why kexec/kdump are such bad ideas..
> > 
> > apparently kexec doesn't properly shut down the first kernel and leaves
> > a counter running, then when we write and read the counter value they
> > don't match because its still running and voila, crap happens.
> > 
> > I've CC'ed the kexec people, maybe they got clue as to how to sort this.
> 
> So we can shutdown counters while first kernel is going down. Is there a
> simple function already which I can call?

Dunno, the cpu hotplug stuff should suffice I think, but then I don't
think you actually unplug the boot cpu.

What does kexec normally do to ensure hardware is left in a sane state?