From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754440AbcBXCBI (ORCPT ); Tue, 23 Feb 2016 21:01:08 -0500 Received: from szxga03-in.huawei.com ([119.145.14.66]:9012 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752353AbcBXCBD (ORCPT ); Tue, 23 Feb 2016 21:01:03 -0500 Subject: Re: [PATCH 10/48] perf tools: Introduce bpf-output event To: Arnaldo Carvalho de Melo References: <1456132275-98875-1-git-send-email-wangnan0@huawei.com> <1456132275-98875-11-git-send-email-wangnan0@huawei.com> <20160223174539.GG15284@kernel.org> CC: Alexei Starovoitov , Arnaldo Carvalho de Melo , Brendan Gregg , Adrian Hunter , Cody P Schafer , "David S. Miller" , He Kuang , =?UTF-8?Q?J=c3=a9r=c3=a9mie_Galarneau?= , Jiri Olsa , Kirill Smelkov , Li Zefan , Masami Hiramatsu , Namhyung Kim , Peter Zijlstra , , From: "Wangnan (F)" Message-ID: <56CD0E4A.9080804@huawei.com> Date: Wed, 24 Feb 2016 09:58:34 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <20160223174539.GG15284@kernel.org> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.111.66.109] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020202.56CD0E75.0106,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0, ip=0.0.0.0, so=2013-05-26 15:14:31, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 1c838656e0eb1466008408c0246019d9 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2016/2/24 1:45, Arnaldo Carvalho de Melo wrote: > Em Mon, Feb 22, 2016 at 09:10:37AM +0000, Wang Nan escreveu: >> Commit a43eec304259a6c637f4014a6d4767159b6a3aa3 (bpf: introduce >> bpf_perf_event_output() helper) add a helper to enable BPF program >> output data to perf ring buffer through a new type of perf event >> PERF_COUNT_SW_BPF_OUTPUT. This patch enable perf to create perf >> event of that type. Now perf user can use following cmdline to >> receive output data from BPF programs: >> >> # ./perf record -a -e bpf-output/no-inherit,name=evt/ \ >> -e ./test_bpf_output.c/map:channel.event=evt/ ls / >> # ./perf script >> perf 1560 [004] 347747.086295: evt: ffffffff811fd201 sys_write ... >> perf 1560 [004] 347747.086300: evt: ffffffff811fd201 sys_write ... >> perf 1560 [004] 347747.086315: evt: ffffffff811fd201 sys_write ... >> ... >> >> Test result: >> # cat ./test_bpf_output.c >> /************************ BEGIN **************************/ >> #include >> struct bpf_map_def { >> unsigned int type; >> unsigned int key_size; >> unsigned int value_size; >> unsigned int max_entries; >> }; >> >> #define SEC(NAME) __attribute__((section(NAME), used)) >> static u64 (*ktime_get_ns)(void) = >> (void *)BPF_FUNC_ktime_get_ns; >> static int (*trace_printk)(const char *fmt, int fmt_size, ...) = >> (void *)BPF_FUNC_trace_printk; >> static int (*get_smp_processor_id)(void) = >> (void *)BPF_FUNC_get_smp_processor_id; >> static int (*perf_event_output)(void *, struct bpf_map_def *, int, void *, unsigned long) = >> (void *)BPF_FUNC_perf_event_output; >> >> struct bpf_map_def SEC("maps") channel = { >> .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY, >> .key_size = sizeof(int), >> .value_size = sizeof(u32), >> .max_entries = __NR_CPUS__, >> }; >> >> SEC("func_write=sys_write") >> int func_write(void *ctx) >> { >> struct { >> u64 ktime; >> int cpuid; >> } __attribute__((packed)) output_data; >> char error_data[] = "Error: failed to output: %d\n"; >> >> output_data.cpuid = get_smp_processor_id(); >> output_data.ktime = ktime_get_ns(); >> int err = perf_event_output(ctx, &channel, get_smp_processor_id(), >> &output_data, sizeof(output_data)); >> if (err) >> trace_printk(error_data, sizeof(error_data), err); >> return 0; >> } >> char _license[] SEC("license") = "GPL"; >> int _version SEC("version") = LINUX_VERSION_CODE; >> /************************ END ***************************/ >> >> # ./perf record -a -e bpf-output/no-inherit,name=evt/ \ >> -e ./test_bpf_output.c/map:channel.event=evt/ ls / >> # ./perf script | grep ls >> ls 2242 [003] 347851.557563: evt: ffffffff811fd201 sys_write ... >> ls 2242 [003] 347851.557571: evt: ffffffff811fd201 sys_write ... > So, there is something strange here: > > if (unlikely(event->oncpu != smp_processor_id())) > return -EOPNOTSUPP; > > This is where I am hitting, with: > > [acme@jouet linux]$ uname -r > 4.5.0-rc4 > > int err = perf_event_output(ctx, &channel, get_smp_processor_id(), > &output_data, sizeof(output_data)); > if (err) > trace_printk(error_data, sizeof(error_data), err); > > And then: > > [root@jouet bpf]# tail /sys/kernel/debug/tracing/trace > perf-13040 [003] d... 12062.807729: : Error: failed to output: -95 > perf-13040 [003] d... 12062.807731: : Error: failed to output: -95 > perf-13040 [003] d... 12062.807732: : Error: failed to output: -95 > perf-13040 [003] d... 12062.807735: : Error: failed to output: -95 > perf-13040 [003] d... 12062.807737: : Error: failed to output: -95 > perf-13040 [003] d... 12062.807744: : Error: failed to output: -95 > gnome-terminal--3091 [001] d... 12062.807773: : Error: failed to output: -95 > gnome-terminal--3091 [001] d... 12062.807784: : Error: failed to output: -95 > gmain-2830 [002] d... 12062.811791: : Error: failed to output: -95 > gmain-2830 [002] d... 12062.811810: : Error: failed to output: -95 > [root@jouet bpf]# > > Ideas? AFK for a while, will continue investigating. I also noticed this output, but didn't digg into it because all events I concerned is okay. I'll look into this today. > This already was submitted to Ingo, BTW. > > I used, as in the changeset comment tests: > > perf record -a -e bpf-output/no-inherit,name=evt/ -e ./test_bpf_output.c/map:channel.event=evt/ ls / > > And perf script told me: > > [root@jouet bpf]# perf script | tail > perf 13040 [003] 12062.708337: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708339: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708340: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708341: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708343: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708344: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708346: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708347: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708348: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > perf 13040 [003] 12062.708350: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux) > [root@jouet bpf]# > > Wonder where that /lib/modules/4.5.0-rc4/build/vmlinux came from... > > [root@jouet bpf]# perf script | cut -d'(' -f2 | sort | uniq -c > 1141 /lib/modules/4.5.0-rc4/build/vmlinux) It's a standard directory for perf searching vmlinux. Isn't it? tools/perf/util/symbol.c: static const char * const vmlinux_paths_upd[] = { "/boot/vmlinux-%s", "/usr/lib/debug/boot/vmlinux-%s", "/lib/modules/%s/build/vmlinux", "/usr/lib/debug/lib/modules/%s/vmlinux", "/usr/lib/debug/boot/vmlinux-%s.debug" }; So what's your problem? Thank you.