From: Maynard Johnson <maynardj@us.ibm.com>
To: linux-perf-users@vger.kernel.org
Subject: --mmap-pages option seemingly has no effect to help with LOST samples
Date: Tue, 12 Jun 2012 15:55:21 -0500 [thread overview]
Message-ID: <4FD7ACB9.70205@us.ibm.com> (raw)
Hi,
On my Intel Core 2 Duo with RHEL 6.2 with the watchdog timer disabled, I'm using perf to collect a CPI profile as follows:
perf record -e cycles 100000 -e instructions -c 50000 ./memcpyt 500000000
where 'memcpyt' is a test program that simply does a LOT of memcpy's -- takes about 20 seconds of real time to complete.
This fails roughly half the time with:
[ perf record: Woken up 11 times to write data ]
[ perf record: Captured and wrote 4.540 MB perf.data (~198348 samples) ]
Processed 0 events and LOST 872662!
Check IO/CPU overload!
I've seen some postings on this list in the past about the LOST events and the suggestion to try the --mmap-pages option. I see from the perf source that the default number of pages to use for mmap'ing the kernel's perf_events data is '8'. I tried going up to 64 pages with little noticeable effect. Additionally, sometimes when I get the LOST samples message, I'll also see the following junk pop up in all of my terminal sessions:
Message from syslogd@oc3431575272 at Jun 12 15:21:52 ...
kernel:Uhhuh. NMI received for unknown reason 00 on CPU 1.
Message from syslogd@oc3431575272 at Jun 12 15:21:52 ...
kernel:Do you have a strange power saving mode enabled?
Message from syslogd@oc3431575272 at Jun 12 15:21:52 ...
kernel:Dazed and confused, but trying to continue
(Not sure, but these syslogd messages may have occurred only when I was running as root.)
I tried decreasing my sampling rate for both events by half (200000 for cycles and 100000 for instructions), but still got LOST samples, with or without the "--mmap-pages=64" option. Decreasing sampling rate by half again finally did get rid of the LOST samples.
Questions:
1) Why doesn't the number of mmap pages seem to have the expected beneficial effect?
2) Why doesn't the kernel's throttle capabilities prevent the LOST events in the first place?
3) What's up with the weird syslogd messages? heh.
I realize none of these may be perf userspace issues, but may be perf_events kernel issues instead. But I thought I'd start out here on this list instead of wading neck-deep into LKML land.
Thanks.
-Maynard
next reply other threads:[~2012-06-12 20:56 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-12 20:55 Maynard Johnson [this message]
2012-06-12 21:05 ` --mmap-pages option seemingly has no effect to help with LOST samples David Ahern
2012-06-13 15:35 ` Maynard Johnson
2012-06-13 15:48 ` David Ahern
2012-06-22 15:59 ` Maynard Johnson
2012-06-22 16:16 ` David Ahern
2012-06-22 19:20 ` Maynard Johnson
2012-06-22 19:44 ` David Ahern
2012-06-22 20:23 ` Maynard Johnson
2012-06-22 20:38 ` David Ahern
2012-06-25 14:10 ` Maynard Johnson
2012-06-26 20:16 ` David Ahern
2012-06-26 20:32 ` Maynard Johnson
2012-06-22 19:23 ` David Ahern
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4FD7ACB9.70205@us.ibm.com \
--to=maynardj@us.ibm.com \
--cc=linux-perf-users@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).