linux-perf-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* "perf list" built from git repository Segmentation fault with 4.4.0 kernels
@ 2015-12-10 22:54 William Cohen
  2015-12-11 13:10 ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 3+ messages in thread
From: William Cohen @ 2015-12-10 22:54 UTC (permalink / raw)
  To: linux-perf-users

I have noticed that with the 4.4.0 kernels "perf list" has been segfaulting on both aarch64 and x86_64.  Below is the failure on fedora rawhide running on x86_64:

[wcohen@localhost ~]$ rpm -q perf
perf-4.4.0-0.rc4.git0.1.fc24.x86_64
[wcohen@localhost ~]$ gdb /bin/perf ./core.2110 
GNU gdb (GDB) Fedora 7.10.50.20151113-33.fc24
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /bin/perf...Reading symbols from /usr/lib/debug/usr/bin/perf.debug...done.
done.
[New LWP 2110]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `perf list'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055b1fc97fdce in print_symbol_events (type=<optimized out>, 
    syms=0x55b1fcd81580 <event_symbols_sw+160>, max=<optimized out>, 
    name_only=<optimized out>, event_glob=<optimized out>)
    at util/parse-events.c:1895
1895			if (!name_only && strlen(syms->alias))
Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-19.fc24.x86_64 nss-softokn-freebl-3.21.0-3.fc24.x86_64
(gdb) where
#0  0x000055b1fc97fdce in print_symbol_events (type=<optimized out>, 
    syms=0x55b1fcd81580 <event_symbols_sw+160>, max=<optimized out>, 
    name_only=<optimized out>, event_glob=<optimized out>)
    at util/parse-events.c:1895
#1  print_events (event_glob=event_glob@entry=0x0, name_only=false)
    at util/parse-events.c:1943
#2  0x000055b1fc90a63e in cmd_list (argc=0, argv=0x7ffde1523f80, 
    prefix=<optimized out>) at builtin-list.c:43
#3  0x000055b1fc95c413 in run_builtin (p=0x55b1fcd803d8 <commands+120>, 
    argc=1, argv=0x7ffde1523f80) at perf.c:386
#4  0x000055b1fc8fe308 in handle_internal_command (argv=0x7ffde1523f80, argc=1)
    at perf.c:447
#5  run_argv (argv=0x7ffde1523d10, argcp=0x7ffde1523d1c) at perf.c:491
#6  main (argc=1, argv=0x7ffde1523f80) at perf.c:608
(gdb) 


I did a git bisect on the kernel source code to find out when "perf list" broke and the git bisect pointed to the following commit causing the problem:

# git bisect good
a43eec304259a6c637f4014a6d4767159b6a3aa3 is the first bad commit
commit a43eec304259a6c637f4014a6d4767159b6a3aa3
Author: Alexei Starovoitov <ast@plumgrid.com>
Date:   Tue Oct 20 20:02:34 2015 -0700

    bpf: introduce bpf_perf_event_output() helper
    
    This helper is used to send raw data from eBPF program into
    special PERF_TYPE_SOFTWARE/PERF_COUNT_SW_BPF_OUTPUT perf_event.
    User space needs to perf_event_open() it (either for one or all cpus) and
    store FD into perf_event_array (similar to bpf_perf_event_read() helper)
    before eBPF program can send data into it.
    
    Today the programs triggered by kprobe collect the data and either store
    it into the maps or print it via bpf_trace_printk() where latter is the debug
    facility and not suitable to stream the data. This new helper replaces
    such bpf_trace_printk() usage and allows programs to have dedicated
    channel into user space for post-processing of the raw data collected.
    
    Signed-off-by: Alexei Starovoitov <ast@kernel.org>
    Signed-off-by: David S. Miller <davem@davemloft.net>

:040000 040000 322abd03170df11e43da73d3fa4f74f1826788e9 14f3ee14b7fb4b6c76daf18e026cf67acd3154fa M	include
:040000 040000 f84baec2884fe97ec501ae8d45bbb8a7aad3a578 ea440aeb13f1dd25aa8f2e1dadfc32dbf4b3d2b3 M	kernel


-Will

^ permalink raw reply	[flat|nested] 3+ messages in thread
* Re: "perf list" built from git repository Segmentation fault with 4.4.0 kernels
  2015-12-10 22:54 "perf list" built from git repository Segmentation fault with 4.4.0 kernels William Cohen
@ 2015-12-11 13:10 ` Arnaldo Carvalho de Melo
  2015-12-11 15:08   ` William Cohen
  0 siblings, 1 reply; 3+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-12-11 13:10 UTC (permalink / raw)
  To: William Cohen; +Cc: linux-perf-users, Ingo Molnar

Em Thu, Dec 10, 2015 at 05:54:03PM -0500, William Cohen escreveu:

> I have noticed that with the 4.4.0 kernels "perf list" has been
> segfaulting on both aarch64 and x86_64.  Below is the failure on
> fedora rawhide running on x86_64:

Ok, this was fixed, but just on the perf/core branch:

"perf list: Add support for PERF_COUNT_SW_BPF_OUT"
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/tools/perf/util/parse-events.c?h=perf/core&id=bae9cc41105b9edd74d68a9636be2ba240e74b9e

And then this one on top to avoid such things in the future:
"perf list: Robustify event printing routine"
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/tools/perf/util/parse-events.c?h=perf/core&id=e37df6c76cb19971f1228bfaff504d8a3ea6f748

I'll cherry-pick those and push to Ingo.

Thanks for the report!

- Arnaldo

> [wcohen@localhost ~]$ rpm -q perf
> perf-4.4.0-0.rc4.git0.1.fc24.x86_64
> [wcohen@localhost ~]$ gdb /bin/perf ./core.2110 
> GNU gdb (GDB) Fedora 7.10.50.20151113-33.fc24
> Copyright (C) 2015 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from /bin/perf...Reading symbols from /usr/lib/debug/usr/bin/perf.debug...done.
> done.
> [New LWP 2110]
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> Core was generated by `perf list'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  0x000055b1fc97fdce in print_symbol_events (type=<optimized out>, 
>     syms=0x55b1fcd81580 <event_symbols_sw+160>, max=<optimized out>, 
>     name_only=<optimized out>, event_glob=<optimized out>)
>     at util/parse-events.c:1895
> 1895			if (!name_only && strlen(syms->alias))
> Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-19.fc24.x86_64 nss-softokn-freebl-3.21.0-3.fc24.x86_64
> (gdb) where
> #0  0x000055b1fc97fdce in print_symbol_events (type=<optimized out>, 
>     syms=0x55b1fcd81580 <event_symbols_sw+160>, max=<optimized out>, 
>     name_only=<optimized out>, event_glob=<optimized out>)
>     at util/parse-events.c:1895
> #1  print_events (event_glob=event_glob@entry=0x0, name_only=false)
>     at util/parse-events.c:1943
> #2  0x000055b1fc90a63e in cmd_list (argc=0, argv=0x7ffde1523f80, 
>     prefix=<optimized out>) at builtin-list.c:43
> #3  0x000055b1fc95c413 in run_builtin (p=0x55b1fcd803d8 <commands+120>, 
>     argc=1, argv=0x7ffde1523f80) at perf.c:386
> #4  0x000055b1fc8fe308 in handle_internal_command (argv=0x7ffde1523f80, argc=1)
>     at perf.c:447
> #5  run_argv (argv=0x7ffde1523d10, argcp=0x7ffde1523d1c) at perf.c:491
> #6  main (argc=1, argv=0x7ffde1523f80) at perf.c:608
> (gdb) 
> 
> 
> I did a git bisect on the kernel source code to find out when "perf list" broke and the git bisect pointed to the following commit causing the problem:
> 
> # git bisect good
> a43eec304259a6c637f4014a6d4767159b6a3aa3 is the first bad commit
> commit a43eec304259a6c637f4014a6d4767159b6a3aa3
> Author: Alexei Starovoitov <ast@plumgrid.com>
> Date:   Tue Oct 20 20:02:34 2015 -0700
> 
>     bpf: introduce bpf_perf_event_output() helper
>     
>     This helper is used to send raw data from eBPF program into
>     special PERF_TYPE_SOFTWARE/PERF_COUNT_SW_BPF_OUTPUT perf_event.
>     User space needs to perf_event_open() it (either for one or all cpus) and
>     store FD into perf_event_array (similar to bpf_perf_event_read() helper)
>     before eBPF program can send data into it.
>     
>     Today the programs triggered by kprobe collect the data and either store
>     it into the maps or print it via bpf_trace_printk() where latter is the debug
>     facility and not suitable to stream the data. This new helper replaces
>     such bpf_trace_printk() usage and allows programs to have dedicated
>     channel into user space for post-processing of the raw data collected.
>     
>     Signed-off-by: Alexei Starovoitov <ast@kernel.org>
>     Signed-off-by: David S. Miller <davem@davemloft.net>
> 
> :040000 040000 322abd03170df11e43da73d3fa4f74f1826788e9 14f3ee14b7fb4b6c76daf18e026cf67acd3154fa M	include
> :040000 040000 f84baec2884fe97ec501ae8d45bbb8a7aad3a578 ea440aeb13f1dd25aa8f2e1dadfc32dbf4b3d2b3 M	kernel
> 
> 
> -Will
> --
> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 3+ messages in thread
* Re: "perf list" built from git repository Segmentation fault with 4.4.0 kernels
  2015-12-11 13:10 ` Arnaldo Carvalho de Melo
@ 2015-12-11 15:08   ` William Cohen
  0 siblings, 0 replies; 3+ messages in thread
From: William Cohen @ 2015-12-11 15:08 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo; +Cc: linux-perf-users, Ingo Molnar

On 12/11/2015 08:10 AM, Arnaldo Carvalho de Melo wrote:
> Em Thu, Dec 10, 2015 at 05:54:03PM -0500, William Cohen escreveu:
> 
>> I have noticed that with the 4.4.0 kernels "perf list" has been
>> segfaulting on both aarch64 and x86_64.  Below is the failure on
>> fedora rawhide running on x86_64:
> 
> Ok, this was fixed, but just on the perf/core branch:
> 
> "perf list: Add support for PERF_COUNT_SW_BPF_OUT"
> https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/tools/perf/util/parse-events.c?h=perf/core&id=bae9cc41105b9edd74d68a9636be2ba240e74b9e
> 
> And then this one on top to avoid such things in the future:
> "perf list: Robustify event printing routine"
> https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/tools/perf/util/parse-events.c?h=perf/core&id=e37df6c76cb19971f1228bfaff504d8a3ea6f748
> 
> I'll cherry-pick those and push to Ingo.
> 
> Thanks for the report!
> 
> - Arnaldo

Thanks Arnaldo.  Glad to hear that this is already addressed and just needed to be pulled into the appropriate git repo.

-Will


> 
>> [wcohen@localhost ~]$ rpm -q perf
>> perf-4.4.0-0.rc4.git0.1.fc24.x86_64
>> [wcohen@localhost ~]$ gdb /bin/perf ./core.2110 
>> GNU gdb (GDB) Fedora 7.10.50.20151113-33.fc24
>> Copyright (C) 2015 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-redhat-linux-gnu".
>> Type "show configuration" for configuration details.
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>.
>> Find the GDB manual and other documentation resources online at:
>> <http://www.gnu.org/software/gdb/documentation/>.
>> For help, type "help".
>> Type "apropos word" to search for commands related to "word"...
>> Reading symbols from /bin/perf...Reading symbols from /usr/lib/debug/usr/bin/perf.debug...done.
>> done.
>> [New LWP 2110]
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library "/lib64/libthread_db.so.1".
>> Core was generated by `perf list'.
>> Program terminated with signal SIGSEGV, Segmentation fault.
>> #0  0x000055b1fc97fdce in print_symbol_events (type=<optimized out>, 
>>     syms=0x55b1fcd81580 <event_symbols_sw+160>, max=<optimized out>, 
>>     name_only=<optimized out>, event_glob=<optimized out>)
>>     at util/parse-events.c:1895
>> 1895			if (!name_only && strlen(syms->alias))
>> Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-19.fc24.x86_64 nss-softokn-freebl-3.21.0-3.fc24.x86_64
>> (gdb) where
>> #0  0x000055b1fc97fdce in print_symbol_events (type=<optimized out>, 
>>     syms=0x55b1fcd81580 <event_symbols_sw+160>, max=<optimized out>, 
>>     name_only=<optimized out>, event_glob=<optimized out>)
>>     at util/parse-events.c:1895
>> #1  print_events (event_glob=event_glob@entry=0x0, name_only=false)
>>     at util/parse-events.c:1943
>> #2  0x000055b1fc90a63e in cmd_list (argc=0, argv=0x7ffde1523f80, 
>>     prefix=<optimized out>) at builtin-list.c:43
>> #3  0x000055b1fc95c413 in run_builtin (p=0x55b1fcd803d8 <commands+120>, 
>>     argc=1, argv=0x7ffde1523f80) at perf.c:386
>> #4  0x000055b1fc8fe308 in handle_internal_command (argv=0x7ffde1523f80, argc=1)
>>     at perf.c:447
>> #5  run_argv (argv=0x7ffde1523d10, argcp=0x7ffde1523d1c) at perf.c:491
>> #6  main (argc=1, argv=0x7ffde1523f80) at perf.c:608
>> (gdb) 
>>
>>
>> I did a git bisect on the kernel source code to find out when "perf list" broke and the git bisect pointed to the following commit causing the problem:
>>
>> # git bisect good
>> a43eec304259a6c637f4014a6d4767159b6a3aa3 is the first bad commit
>> commit a43eec304259a6c637f4014a6d4767159b6a3aa3
>> Author: Alexei Starovoitov <ast@plumgrid.com>
>> Date:   Tue Oct 20 20:02:34 2015 -0700
>>
>>     bpf: introduce bpf_perf_event_output() helper
>>     
>>     This helper is used to send raw data from eBPF program into
>>     special PERF_TYPE_SOFTWARE/PERF_COUNT_SW_BPF_OUTPUT perf_event.
>>     User space needs to perf_event_open() it (either for one or all cpus) and
>>     store FD into perf_event_array (similar to bpf_perf_event_read() helper)
>>     before eBPF program can send data into it.
>>     
>>     Today the programs triggered by kprobe collect the data and either store
>>     it into the maps or print it via bpf_trace_printk() where latter is the debug
>>     facility and not suitable to stream the data. This new helper replaces
>>     such bpf_trace_printk() usage and allows programs to have dedicated
>>     channel into user space for post-processing of the raw data collected.
>>     
>>     Signed-off-by: Alexei Starovoitov <ast@kernel.org>
>>     Signed-off-by: David S. Miller <davem@davemloft.net>
>>
>> :040000 040000 322abd03170df11e43da73d3fa4f74f1826788e9 14f3ee14b7fb4b6c76daf18e026cf67acd3154fa M	include
>> :040000 040000 f84baec2884fe97ec501ae8d45bbb8a7aad3a578 ea440aeb13f1dd25aa8f2e1dadfc32dbf4b3d2b3 M	kernel
>>
>>
>> -Will
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2015-12-11 15:08 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-12-10 22:54 "perf list" built from git repository Segmentation fault with 4.4.0 kernels William Cohen
2015-12-11 13:10 ` Arnaldo Carvalho de Melo
2015-12-11 15:08   ` William Cohen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).