perf samples too long without perf running (?!?)

perf samples too long without perf running (?!?)

[Andy Lutomirski]

On recent (3.13) kernels, I'm frequently getting messages like:

perf samples too long (2503 > 2500), lowering
kernel.perf_event_max_sample_rate to 50000

a few seconds to minutes after bootup.  As far as I know, I'm not
using perf.  What's going on here?

--Andy

Re: perf samples too long without perf running (?!?)

[Mike Galbraith]

On Fri, 2014-03-07 at 17:57 -0800, Andy Lutomirski wrote: 
> On recent (3.13) kernels, I'm frequently getting messages like:
> 
> perf samples too long (2503 > 2500), lowering
> kernel.perf_event_max_sample_rate to 50000
> 
> a few seconds to minutes after bootup.  As far as I know, I'm not
> using perf.  What's going on here?

14c63f17b1

I get those all the time too, just building kernels.

[  598.583918] perf samples too long (2539 > 2500), lowering kernel.perf_event_max_sample_rate to 50000

Darn nmi_watchdog is an expensive employee.  Scarier are the ones from
2ab00456ea when using perf.  Firing the dog makes it all batter.

-Mike

Re: perf samples too long without perf running (?!?)

[Markus Trippelsdorf]

On 2014.03.08 at 08:51 +0100, Mike Galbraith wrote:
> On Fri, 2014-03-07 at 17:57 -0800, Andy Lutomirski wrote: 
> > On recent (3.13) kernels, I'm frequently getting messages like:
> > 
> > perf samples too long (2503 > 2500), lowering
> > kernel.perf_event_max_sample_rate to 50000
> > 
> > a few seconds to minutes after bootup.  As far as I know, I'm not
> > using perf.  What's going on here?
> 
> 14c63f17b1
> 
> I get those all the time too, just building kernels.

I have:
 echo 0 > /proc/sys/kernel/perf_cpu_time_max_percent
in my local startup script since that commit.

-- 
Markus

Re: perf samples too long without perf running (?!?)

[Andy Lutomirski]

On Sat, Mar 8, 2014 at 1:13 AM, Markus Trippelsdorf
<markus@trippelsdorf.de> wrote:
> On 2014.03.08 at 08:51 +0100, Mike Galbraith wrote:
>> On Fri, 2014-03-07 at 17:57 -0800, Andy Lutomirski wrote:
>> > On recent (3.13) kernels, I'm frequently getting messages like:
>> >
>> > perf samples too long (2503 > 2500), lowering
>> > kernel.perf_event_max_sample_rate to 50000
>> >
>> > a few seconds to minutes after bootup.  As far as I know, I'm not
>> > using perf.  What's going on here?
>>
>> 14c63f17b1
>>
>> I get those all the time too, just building kernels.
>
> I have:
>  echo 0 > /proc/sys/kernel/perf_cpu_time_max_percent
> in my local startup script since that commit.

...so the NMI watchdog (which happens IIRC once every two minutes)
takes more than 0.4ms or so, so perf flips out, changes its parameters,
and writes a log message.

Is it just me or is this whole situation screwed up for multiple reasons?

--Andy

>
> --
> Markus



-- 
Andy Lutomirski
AMA Capital Management, LLC

Re: perf samples too long without perf running (?!?)

[Paul Bolle]

On Sat, 2014-03-08 at 08:22 -0800, Andy Lutomirski wrote:
> Is it just me or is this whole situation screwed up for multiple reasons?

One minor reason would be that this message should read (say) 
    perf samples taking too long (2503 ns > 2500 ns), [...]

At least, that's what I thought after diving into the code to see what
was causing warnings like these.

And, since apparently things are just fine without administrator
intervention, I'd say another minor reason is that printing this at
KERN_INFO or KERN_NOTICE level should be enough here.


Paul Bolle