From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jonas Wagner Subject: Re: Problems with -b, and confusing error messages Date: Thu, 6 Oct 2016 00:03:55 +0200 Message-ID: References: <20161005111131.GO7143@kernel.org> <1475681885.13740.1.camel@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Return-path: Received: from smtp4.epfl.ch ([128.178.224.219]:59419 "EHLO smtp4.epfl.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933610AbcJEWEl (ORCPT ); Wed, 5 Oct 2016 18:04:41 -0400 Received: by mail-wm0-f49.google.com with SMTP id k125so293055274wma.1 for ; Wed, 05 Oct 2016 15:04:36 -0700 (PDT) In-Reply-To: <1475681885.13740.1.camel@gmail.com> Sender: linux-perf-users-owner@vger.kernel.org List-ID: To: Venkat Cc: Jonas Wagner , Arnaldo Carvalho de Melo , linux-perf-users@vger.kernel.org Hi, > What does "dmesg|grep PEBS" return? and is that any different with prior > kernel ?(could verify old dmesg outputs) I'm seeing "Performance Events: PEBS fmt2+, 16-deep LBR, Haswell events, full-width counters, Intel PMU driver." This is the same for previous boots. Here is it with a bit more context: [ 0.172417] Performance Events: PEBS fmt2+, 16-deep LBR, Haswell events, full-width counters, Intel PMU driver. [ 0.172426] ... version: 3 [ 0.172427] ... bit width: 48 [ 0.172428] ... generic registers: 4 [ 0.172429] ... value mask: 0000ffffffffffff [ 0.172431] ... max period: 0000ffffffffffff [ 0.172432] ... fixed-purpose events: 3 [ 0.172433] ... event mask: 000000070000000f Best, Jonas > On Wed, 2016-10-05 at 14:24 +0200, Jonas Wagner wrote: >> Hi, >> >> Thanks a lot Arnaldo for your help! >> >> Unfortunately the problem persists, even with generous permissions and >> when running as root :( Here are some more examples: >> >> --- >> >> jowagner@kirk3[~] % sudo perf record -b /bin/true >> Error: 'precise' request may not be supported. Try removing 'p' modifier. >> >> jowagner@kirk3[~] % sudo perf record -b -e cycles /bin/true >> Error: No hardware sampling interrupt available. >> No APIC? If so then you can boot the kernel with the "lapic" boot >> parameter to force-enable it. >> >> jowagner@kirk3[~] % perf evlist -v >> zero-sized file (perf.data), nothing to do! >> >> jowagner@kirk3[~] % sudo perf record /bin/true >> [ perf record: Woken up 1 times to write data ] >> [ perf record: Captured and wrote 0.014 MB perf.data (9 samples) ] >> >> jowagner@kirk3[~] % perf evlist -v >> cycles:pp: size: 112, { sample_period, sample_freq }: 4000, >> sample_type: IP|TID|TIME|PERIOD, disabled: 1, inherit: 1, mmap: 1, >> comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 2, >> sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1 >> >> --- >> >> Here are my settings in /proc/sys/kernel: >> /proc/sys/kernel/perf_cpu_time_max_percent 25 >> /proc/sys/kernel/perf_event_max_sample_rate 100000 >> /proc/sys/kernel/perf_event_mlock_kb 516 >> /proc/sys/kernel/perf_event_paranoid 0 >> /proc/sys/kernel/kptr_restrict 0 >> >> And these are the CPU flags: >> model name : Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz >> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge >> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe >> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts >> rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq >> dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm >> pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave >> avx f16c rdrand lahf_lm abm epb tpr_shadow vnmi flexpriority ept vpid >> fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt >> cqm_llc cqm_occup_llc dtherm ida arat pln pts >> >> Any other idea about the problem, or hints as to how I could debug it? >> >> Best, >> Jonas >> >> On Wed, Oct 5, 2016 at 1:11 PM, Arnaldo Carvalho de Melo >> wrote: >> > Em Tue, Oct 04, 2016 at 05:02:58PM +0200, Jonas Wagner escreveu: >> >> I would like to ask for help, because perf stopped working when I >> >> enable taken branch stack sampling (with the -b option). Here are a >> >> few examples: >> >> >> >> $ perf record -b /bin/true >> >> Error: >> >> 'precise' request may not be supported. Try removing 'p' modifier. >> >> >> >> $ perf record -b -e cycles /bin/true >> >> Error: >> >> No hardware sampling interrupt available. >> >> No APIC? If so then you can boot the kernel with the "lapic" boot >> >> parameter to force-enable it. >> >> >> >> $ perf record -e cycles /bin/true >> >> [ perf record: Woken up 1 times to write data ] >> >> [ perf record: Captured and wrote 0.014 MB perf.data (8 samples) ] >> >> >> >> $ perf record -e cycles:p /bin/true >> >> [ perf record: Woken up 1 times to write data ] >> >> [ perf record: Captured and wrote 0.014 MB perf.data (9 samples) ] >> >> >> >> $ cat /proc/cmdline >> >> BOOT_IMAGE=/boot/vmlinuz-4.4.0-38-generic >> >> root=UUID=a5440a73-d559-4b61-808c-2430cd67ba25 ro quiet lapic >> >> >> >> I have a couple questions about this: >> >> >> >> 1. What's wrong? Branch stack sampling used to work on this machine... >> >> how can I make it work again? >> >> 2. Apparently, the problem is not related to the precise modifier, or >> >> the lapic boot parameter. Could we make these error messages more >> >> meaningful? >> >> >> >> Note regarding (1): I recently upgraded the kernel on this machine >> >> from 4.4.0-36-generic to 4.4.0-38-generic, on Ubuntu Xenial. Using the >> >> previous kernel, perf worked fine. It's tricky for me to undo the >> >> update though :( >> >> >> >> Help would be appreciated. Happy to provide more logs if needed. Thanks! >> > >> > So the message needs to be improved, no question about it, but can you >> > try running as root or tweaking perf_event_paranoid, as here I have: >> > >> > [root@jouet ~]# perf record -b /bin/true >> > [ perf record: Woken up 1 times to write data ] >> > [ perf record: Captured and wrote 0.024 MB perf.data (9 samples) ] >> > [root@jouet ~]# uname -a >> > Linux jouet 4.7.3-200.fc24.x86_64 #1 SMP Wed Sep 7 17:31:21 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux >> > [root@jouet ~]# perf record -b /bin/true >> > [ perf record: Woken up 1 times to write data ] >> > [ perf record: Captured and wrote 0.024 MB perf.data (8 samples) ] >> > [root@jouet ~]# perf evlist -v >> > cycles:ppp: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD|BRANCH_STACK, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, branch_sample_type: ANY >> > [root@jouet ~]# >> > >> > >> > And, as user: >> > >> > [acme@jouet linux]$ perf record -b /bin/true >> > Error: >> > You may not have permission to collect stats. >> > >> > Consider tweaking /proc/sys/kernel/perf_event_paranoid, >> > which controls use of the performance events system by >> > unprivileged users (without CAP_SYS_ADMIN). >> > >> > The current value is 2: >> > >> > -1: Allow use of (almost) all events by all users >> >>= 0: Disallow raw tracepoint access by users without CAP_IOC_LOCK >> >>= 1: Disallow CPU event access by users without CAP_SYS_ADMIN >> >>= 2: Disallow kernel profiling by users without CAP_SYS_ADMIN >> > [acme@jouet linux]$ >> > >> > If I tweak /proc/sys/kernel/perf_event_paranoid, changing it from 2 to 1, say: >> > >> > [acme@jouet linux]$ perf record -b /bin/true >> > [ perf record: Woken up 1 times to write data ] >> > [ perf record: Captured and wrote 0.024 MB perf.data (9 samples) ] >> > [acme@jouet linux]$ perf evlist -v >> > cycles:ppp: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD|BRANCH_STACK, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, branch_sample_type: ANY >> > [acme@jouet linux]$ >> > >> > It seems to work. >> > >> > IIRC there was a change in the default of /proc/sys/kernel/perf_event_paranoid >> > in Ubuntu recently, I bet this is the case. >> > >> > Probably they did that but forgot to update the tool, that would get better >> > error messages, but still not that good wrt branch stack support :-\ >> > >> > - Arnaldo >> > -- >> > To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in >> > the body of a message to majordomo@vger.kernel.org >> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >