[PATCH 1/2] perf probe: Improve error message when function not found

[PATCH 1/2] perf probe: Improve error message when function not found

[David Ahern]

When requesting a function from a userspace library the error message to
the user is less than helpful. e.g.,

    perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'
    no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
    Failed to load map.
       Error: Failed to add events. (-22)

In this cae the symbol really does exist but is a local symbol which is
filtered:

    nm /lib64/libpthread-2.14.90.so | grep __pthread_mutex_lock_full
    0000000000005700 t __pthread_mutex_lock_full

With this patch:
    perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'

    no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
    Failed to find function in /lib64/libpthread-2.14.90.so. Perhaps it is a local variable?
      Error: Failed to add events. (-22)

Signed-off-by: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
 tools/perf/util/probe-event.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 9c6989ca2bea..638071b7bdd2 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -2309,7 +2309,8 @@ static int convert_name_to_addr(struct perf_probe_event *pev, const char *exec)
 	}
 	available_func_filter = strfilter__new(function, NULL);
 	if (map__load(map, filter_available_functions)) {
-		pr_err("Failed to load map.\n");
+		pr_err("Failed to find requested function in %s. Perhaps it is a local variable?\n",
+		       name);
 		goto out;
 	}
 
-- 
1.8.3.4 (Apple Git-47)

[PATCH 2/2] perf probe: Allow user to specify address within executable

[David Ahern]

Allow user to specify an address within an executable. This is useful, for
example, in probing local functions. If the function name begins with 0x
then try to convert the supplied name to an address. If succuessful then
treat the function name as the address within the executable to be probed.

Signed-off-by: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
 tools/perf/Documentation/perf-probe.txt |  2 +-
 tools/perf/util/probe-event.c           | 37 +++++++++++++++++++++++++--------
 2 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/tools/perf/Documentation/perf-probe.txt b/tools/perf/Documentation/perf-probe.txt
index b715cb71592b..ed500b311176 100644
--- a/tools/perf/Documentation/perf-probe.txt
+++ b/tools/perf/Documentation/perf-probe.txt
@@ -123,7 +123,7 @@ Probe points are defined by following syntax.
 
 
 'EVENT' specifies the name of new event, if omitted, it will be set the name of the probed function. Currently, event group name is set as 'probe'.
-'FUNC' specifies a probed function name, and it may have one of the following options; '+OFFS' is the offset from function entry address in bytes, ':RLN' is the relative-line number from function entry line, and '%return' means that it probes function return. And ';PTN' means lazy matching pattern (see LAZY MATCHING). Note that ';PTN' must be the end of the probe point definition.  In addition, '@SRC' specifies a source file which has that function.
+'FUNC' specifies a probed function name, and it may have one of the following options; '+OFFS' is the offset from function entry address in bytes, ':RLN' is the relative-line number from function entry line, and '%return' means that it probes function return. And ';PTN' means lazy matching pattern (see LAZY MATCHING). Note that ';PTN' must be the end of the probe point definition.  In addition, '@SRC' specifies a source file which has that function. If 'FUNC' starts with 0x and the entire string converts to an unsigned long it is treated as an address.
 It is also possible to specify a probe point by the source line number or lazy matching by using 'SRC:ALN' or 'SRC;PTN' syntax, where 'SRC' is the source file path, ':ALN' is the line number and ';PTN' is the lazy matching pattern.
 'ARG' specifies the arguments of this probe point, (see PROBE ARGUMENT).
 
diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 638071b7bdd2..000e535b3df2 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -2208,6 +2208,9 @@ static struct strfilter *available_func_filter;
 static int filter_available_functions(struct map *map __maybe_unused,
 				      struct symbol *sym)
 {
+	if (available_func_filter == NULL)
+		return 0;
+
 	if (sym->binding == STB_GLOBAL &&
 	    strfilter__compare(available_func_filter, sym->name))
 		return 0;
@@ -2281,9 +2284,10 @@ static int convert_name_to_addr(struct perf_probe_event *pev, const char *exec)
 	struct perf_probe_point *pp = &pev->point;
 	struct symbol *sym;
 	struct map *map = NULL;
-	char *function = NULL, *name = NULL;
+	char *function = NULL, *name = NULL, *endptr;
 	int ret = -EINVAL;
 	unsigned long long vaddr = 0;
+	unsigned long fcn_addr = 0;
 
 	if (!pp->function) {
 		pr_warning("No function specified for uprobes");
@@ -2297,6 +2301,13 @@ static int convert_name_to_addr(struct perf_probe_event *pev, const char *exec)
 		goto out;
 	}
 
+	/* perhaps an address was given instead of a function name */
+	if (strncmp(function, "0x", 2) == 0) {
+		fcn_addr = strtoul(function, &endptr, 0);
+		if (endptr && *endptr != '\0')
+			fcn_addr = 0;
+	}
+
 	name = realpath(exec, NULL);
 	if (!name) {
 		pr_warning("Cannot find realpath for %s.\n", exec);
@@ -2307,22 +2318,30 @@ static int convert_name_to_addr(struct perf_probe_event *pev, const char *exec)
 		pr_warning("Cannot find appropriate DSO for %s.\n", exec);
 		goto out;
 	}
-	available_func_filter = strfilter__new(function, NULL);
+
+	if (fcn_addr == 0)
+		available_func_filter = strfilter__new(function, NULL);
+
 	if (map__load(map, filter_available_functions)) {
 		pr_err("Failed to find requested function in %s. Perhaps it is a local variable?\n",
 		       name);
 		goto out;
 	}
 
-	sym = map__find_symbol_by_name(map, function, NULL);
-	if (!sym) {
-		pr_warning("Cannot find %s in DSO %s\n", function, exec);
-		goto out;
+	if (fcn_addr) {
+		vaddr = fcn_addr + pp->offset + map->pgoff;
+	} else {
+		sym = map__find_symbol_by_name(map, function, NULL);
+		if (!sym) {
+			pr_warning("Cannot find %s in DSO %s\n", function, exec);
+			goto out;
+		}
+
+		if (map->start > sym->start)
+			vaddr = map->start;
+		vaddr += sym->start + pp->offset + map->pgoff;
 	}
 
-	if (map->start > sym->start)
-		vaddr = map->start;
-	vaddr += sym->start + pp->offset + map->pgoff;
 	pp->offset = 0;
 
 	if (!pev->event) {
-- 
1.8.3.4 (Apple Git-47)

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[Masami Hiramatsu]

(2013/12/02 9:07), David Ahern wrote:
> Allow user to specify an address within an executable. This is useful, for
> example, in probing local functions. If the function name begins with 0x
> then try to convert the supplied name to an address. If succuessful then
> treat the function name as the address within the executable to be probed.

Hmm, IMHO, this kind of functionality is only good for debugging.
In that case, you should use uprobe_events interface directly.

I recommend you to try enabing dwarf support in user space.
Perf probe is a user-friendly interface of dynamic events.

Thank you,


-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 1/2] perf probe: Improve error message when function not found

[Masami Hiramatsu]

(2013/12/02 9:07), David Ahern wrote:
> When requesting a function from a userspace library the error message to
> the user is less than helpful. e.g.,
> 
>     perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'
>     no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
>     Failed to load map.
>        Error: Failed to add events. (-22)
> 
> In this cae the symbol really does exist but is a local symbol which is
> filtered:
> 
>     nm /lib64/libpthread-2.14.90.so | grep __pthread_mutex_lock_full
>     0000000000005700 t __pthread_mutex_lock_full
> 
> With this patch:
>     perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'
> 
>     no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
>     Failed to find function in /lib64/libpthread-2.14.90.so. Perhaps it is a local variable?

Hmm, indeed current message is too less information, but your message looks
wired. I think just showing "Failed to find requested symbol in %s" is enough. :)

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[Masami Hiramatsu]

(2013/12/02 9:07), David Ahern wrote:
> Allow user to specify an address within an executable. This is useful, for
> example, in probing local functions. If the function name begins with 0x
> then try to convert the supplied name to an address. If succuessful then
> treat the function name as the address within the executable to be probed.

Hmm, IMHO, this kind of functionality is only good for debugging.
In that case, you should use uprobe_events interface directly.

I recommend you to try enabing dwarf support in user space.
Perf probe is a user-friendly interface of dynamic events.

Thank you,


-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[David Ahern]

On 12/1/13, 11:15 PM, Masami Hiramatsu wrote:
> (2013/12/02 9:07), David Ahern wrote:
>> Allow user to specify an address within an executable. This is useful, for
>> example, in probing local functions. If the function name begins with 0x
>> then try to convert the supplied name to an address. If succuessful then
>> treat the function name as the address within the executable to be probed.
> 
> Hmm, IMHO, this kind of functionality is only good for debugging.

Why? perf-probe takes the function name, looks up the address in the
executable, adds an offset and then pushes that address to the kernel.
Why can't I have an option to jump straight to the desired address?

A few use cases that come to mind:

1. Allows manual creation of return probes in kernels which do not have
that support.

2. probing static functions

3. probing versioned addresses in libc -- something the current syntax
does not allow because '@' is used as a field separator:

$ nm /lib64/libc-2.14.90.so | grep pthread_cond_timedwait
00000000000fe6c0 t __pthread_cond_timedwait
000000000012d920 t __pthread_cond_timedwait_2_0
00000000000fe6c0 T pthread_cond_timedwait@@GLIBC_2.3.2
000000000012d920 T pthread_cond_timedwait@GLIBC_2.2.5

$ perf probe -x /lib64/libc-2.14.90.so -a
'cond_timewait=pthread_cond_timedwait'
no symbols found in /lib64/libc-2.14.90.so, maybe install a debug package?
Failed to load map.
  Error: Failed to add events. (-22)

$ perf probe -x /lib64/libc-2.14.90.so -a
'cond_timewait=pthread_cond_timedwait@@GLIBC_2.3.2'
Semantic error :SRC@SRC is not allowed.


4. A workaround for any other shortcomings in the interface that require
patches to fix and backports to be done. backports which take time if
even possible.

> In that case, you should use uprobe_events interface directly.

How do I do that within the context of perf?

> 
> I recommend you to try enabing dwarf support in user space.
> Perf probe is a user-friendly interface of dynamic events.

I do have dwarf support enabled in perf. Are you referring to other
components of userspace (like system libraries)?

David

Re: [PATCH 1/2] perf probe: Improve error message when function not found

[David Ahern]

On 12/1/13, 10:59 PM, Masami Hiramatsu wrote:
> (2013/12/02 9:07), David Ahern wrote:
>> When requesting a function from a userspace library the error message to
>> the user is less than helpful. e.g.,
>>
>>      perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'
>>      no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
>>      Failed to load map.
>>         Error: Failed to add events. (-22)
>>
>> In this cae the symbol really does exist but is a local symbol which is
>> filtered:
>>
>>      nm /lib64/libpthread-2.14.90.so | grep __pthread_mutex_lock_full
>>      0000000000005700 t __pthread_mutex_lock_full
>>
>> With this patch:
>>      perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'
>>
>>      no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
>>      Failed to find function in /lib64/libpthread-2.14.90.so. Perhaps it is a local variable?
> 
> Hmm, indeed current message is too less information, but your message looks
> wired. I think just showing "Failed to find requested symbol in %s" is enough. :)

I disagree. perf-probe filters non-global variables:

    if (sym->binding == STB_GLOBAL &&
        strfilter__compare(available_func_filter, sym->name))
        return 0;

That needs to be explicitly stated in the error message -- only global
symbols may be given.

Between the 2 original error messages I wasted 2 hours strace'ing perf
looking at files -- including all the debuginfo variants -- that perf
opens. nm/objdump and every other tool sees the symbols yet perf throws
out bizarre error messages -- "no symbols found in ..." and "Failed to
load map". Let's fix those messages with something meaningful.

David

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[David Ahern]

On 12/2/13, 7:49 AM, David Ahern wrote:
> On 12/1/13, 11:15 PM, Masami Hiramatsu wrote:
>> (2013/12/02 9:07), David Ahern wrote:
>>> Allow user to specify an address within an executable. This is useful, for
>>> example, in probing local functions. If the function name begins with 0x
>>> then try to convert the supplied name to an address. If succuessful then
>>> treat the function name as the address within the executable to be probed.
>>
>> Hmm, IMHO, this kind of functionality is only good for debugging.
> 
> Why? perf-probe takes the function name, looks up the address in the
> executable, adds an offset and then pushes that address to the kernel.
> Why can't I have an option to jump straight to the desired address?
> 
> A few use cases that come to mind:
> 
> 1. Allows manual creation of return probes in kernels which do not have
> that support.
> 
> 2. probing static functions

D'oh. That is supposed to be local functions, not static functions.
Right now perf-probe does not allow probes in local functions of an
executable. Giving a specific address provides a mean to do that.

> 
> 3. probing versioned addresses in libc -- something the current syntax
> does not allow because '@' is used as a field separator:
> 
> $ nm /lib64/libc-2.14.90.so | grep pthread_cond_timedwait
> 00000000000fe6c0 t __pthread_cond_timedwait
> 000000000012d920 t __pthread_cond_timedwait_2_0
> 00000000000fe6c0 T pthread_cond_timedwait@@GLIBC_2.3.2
> 000000000012d920 T pthread_cond_timedwait@GLIBC_2.2.5
> 
> $ perf probe -x /lib64/libc-2.14.90.so -a
> 'cond_timewait=pthread_cond_timedwait'
> no symbols found in /lib64/libc-2.14.90.so, maybe install a debug package?
> Failed to load map.
>    Error: Failed to add events. (-22)
> 
> $ perf probe -x /lib64/libc-2.14.90.so -a
> 'cond_timewait=pthread_cond_timedwait@@GLIBC_2.3.2'
> Semantic error :SRC@SRC is not allowed.
> 
> 
> 4. A workaround for any other shortcomings in the interface that require
> patches to fix and backports to be done. backports which take time if
> even possible.
> 
>> In that case, you should use uprobe_events interface directly.
> 
> How do I do that within the context of perf?
> 
>>
>> I recommend you to try enabing dwarf support in user space.
>> Perf probe is a user-friendly interface of dynamic events.
> 
> I do have dwarf support enabled in perf. Are you referring to other
> components of userspace (like system libraries)?
> 
> David
> 

Re: Re: [PATCH 1/2] perf probe: Improve error message when function not found

[Masami Hiramatsu]

(2013/12/02 23:52), David Ahern wrote:
> On 12/1/13, 10:59 PM, Masami Hiramatsu wrote:
>> (2013/12/02 9:07), David Ahern wrote:
>>> When requesting a function from a userspace library the error message to
>>> the user is less than helpful. e.g.,
>>>
>>>      perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'
>>>      no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
>>>      Failed to load map.
>>>         Error: Failed to add events. (-22)
>>>
>>> In this cae the symbol really does exist but is a local symbol which is
>>> filtered:
>>>
>>>      nm /lib64/libpthread-2.14.90.so | grep __pthread_mutex_lock_full
>>>      0000000000005700 t __pthread_mutex_lock_full
>>>
>>> With this patch:
>>>      perf probe -x /lib64/libpthread-2.14.90.so -a 'lock_full=__pthread_mutex_lock_full'
>>>
>>>      no symbols found in /lib64/libpthread-2.14.90.so, maybe install a debug package?
>>>      Failed to find function in /lib64/libpthread-2.14.90.so. Perhaps it is a local variable?
>>
>> Hmm, indeed current message is too less information, but your message looks
>> wired. I think just showing "Failed to find requested symbol in %s" is enough. :)
> 
> I disagree. perf-probe filters non-global variables:
> 
>     if (sym->binding == STB_GLOBAL &&
>         strfilter__compare(available_func_filter, sym->name))
>         return 0;
> 
> That needs to be explicitly stated in the error message -- only global
> symbols may be given.

Ah, I see. In that case, I think the "variable" is not a correct word,
the "symbol" is better, because perf probe can take tracing variable
arguments after the trace point definition. :)

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[Masami Hiramatsu]

(2013/12/03 2:55), David Ahern wrote:
> On 12/2/13, 7:49 AM, David Ahern wrote:
>> On 12/1/13, 11:15 PM, Masami Hiramatsu wrote:
>>> (2013/12/02 9:07), David Ahern wrote:
>>>> Allow user to specify an address within an executable. This is useful, for
>>>> example, in probing local functions. If the function name begins with 0x
>>>> then try to convert the supplied name to an address. If succuessful then
>>>> treat the function name as the address within the executable to be probed.
>>>
>>> Hmm, IMHO, this kind of functionality is only good for debugging.
>>
>> Why? perf-probe takes the function name, looks up the address in the
>> executable, adds an offset and then pushes that address to the kernel.
>> Why can't I have an option to jump straight to the desired address?
>>
>> A few use cases that come to mind:
>>
>> 1. Allows manual creation of return probes in kernels which do not have
>> that support.
>>
>> 2. probing static functions
> 
> D'oh. That is supposed to be local functions, not static functions.
> Right now perf-probe does not allow probes in local functions of an
> executable. Giving a specific address provides a mean to do that.

No, perf probe for kprobe support to probing on local functions even
if it is inlined. The problem currently you encountered is that
perf probe uprobe support is incomplete. It should be enhanced to
support dwarf(debuginfo) to find the functions(even lines) from it.

I don't want to make perf-probe just a wrapper of the ftrace dynamic
event interface, because it doesn't add any "value" for users.
Current uprobe support state is not sufficient for me.

>>> In that case, you should use uprobe_events interface directly.
>>
>> How do I do that within the context of perf?

No way, but here, you can save this script as perf-uprobe and
can use it for that purpose. :)
---
#!/bin/sh
BIN=$1
FUNC=$2
ADDR=`nm $BIN | grep " $FUNC\$" | cut -f1 -d" "`
echo p $BIN:0x$ADDR > /sys/kernel/debug/tracing/uprobe_events
---

>>> I recommend you to try enabing dwarf support in user space.
>>> Perf probe is a user-friendly interface of dynamic events.
>>
>> I do have dwarf support enabled in perf. Are you referring to other
>> components of userspace (like system libraries)?

Ah, right. Unfortunately, perf probe for uprobes does not support
dwarf for user space yet. This should be enabled for completing
the perf probe work...

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 1/2] perf probe: Improve error message when function not found

[David Ahern]

On 12/2/13, 10:12 PM, Masami Hiramatsu wrote:
>> That needs to be explicitly stated in the error message -- only global
>> symbols may be given.
> 
> Ah, I see. In that case, I think the "variable" is not a correct word,
> the "symbol" is better, because perf probe can take tracing variable
> arguments after the trace point definition. :)

Right, variable is not the correct word. So how about this:

"Failed to find requested symbol in %s. Is it a global symbol?"

David

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[David Ahern]

On 12/3/13, 2:24 AM, Masami Hiramatsu wrote:
> I don't want to make perf-probe just a wrapper of the ftrace dynamic
> event interface, because it doesn't add any "value" for users.

Sure it does -- a consistent user experience in using a single command
(perf probe ...) for setting up userspace probes.

Specifying the address directly is really on par with perf-record having
the rNNN interface for directly passing the raw event id to the kernel.

> 
>>>> In that case, you should use uprobe_events interface directly.
>>>
>>> How do I do that within the context of perf?
> 
> No way, but here, you can save this script as perf-uprobe and
> can use it for that purpose. :)

I figured out what you meant by uprobe_events interface yesterday. If I
have to go to that interface for even 1 function I would do it for all
-- from a user perspective it is just simpler to have 1 command to setup
probes. I would prefer that 1 command be perf-probe.

David

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[Masami Hiramatsu]

(2013/12/04 0:58), David Ahern wrote:
> On 12/3/13, 2:24 AM, Masami Hiramatsu wrote:
>> I don't want to make perf-probe just a wrapper of the ftrace dynamic
>> event interface, because it doesn't add any "value" for users.
> 
> Sure it does -- a consistent user experience in using a single command
> (perf probe ...) for setting up userspace probes.
> 
> Specifying the address directly is really on par with perf-record having
> the rNNN interface for directly passing the raw event id to the kernel.

No, since the perf interface is not directly exposed to users
the perftools need to provide such raw interface. But the dynamic
event has it on debugfs.

>>>>> In that case, you should use uprobe_events interface directly.
>>>>
>>>> How do I do that within the context of perf?
>>
>> No way, but here, you can save this script as perf-uprobe and
>> can use it for that purpose. :)
> 
> I figured out what you meant by uprobe_events interface yesterday. If I
> have to go to that interface for even 1 function I would do it for all
> -- from a user perspective it is just simpler to have 1 command to setup
> probes. I would prefer that 1 command be perf-probe.

Yeah, but in that case, why you don't ask us adding sym->binding == STB_LOCAL
in filter_available_functions? :)

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 1/2] perf probe: Improve error message when function not found

[Masami Hiramatsu]

(2013/12/04 0:15), David Ahern wrote:
> On 12/2/13, 10:12 PM, Masami Hiramatsu wrote:
>>> That needs to be explicitly stated in the error message -- only global
>>> symbols may be given.
>>
>> Ah, I see. In that case, I think the "variable" is not a correct word,
>> the "symbol" is better, because perf probe can take tracing variable
>> arguments after the trace point definition. :)
> 
> Right, variable is not the correct word. So how about this:
> 
> "Failed to find requested symbol in %s. Is it a global symbol?"

Yeah, that's good for me now :)

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[David Ahern]

On 12/3/13, 6:22 PM, Masami Hiramatsu wrote:

>> I figured out what you meant by uprobe_events interface yesterday. If I
>> have to go to that interface for even 1 function I would do it for all
>> -- from a user perspective it is just simpler to have 1 command to setup
>> probes. I would prefer that 1 command be perf-probe.
> 
> Yeah, but in that case, why you don't ask us adding sym->binding == STB_LOCAL
> in filter_available_functions? :)

I did in a separate email -- you said because there can be multiple
local functions with the same name. But local functions is not the only
use case I need.

For now I will carry the patch locally. At this point I am 20 patches
deep and have probably another 20 to go. What's one more. I'll come back
to this when I have more time.

Thanks,

David

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[Masami Hiramatsu]

(2013/12/04 10:44), David Ahern wrote:
> On 12/3/13, 6:22 PM, Masami Hiramatsu wrote:
> 
>>> I figured out what you meant by uprobe_events interface yesterday. If I
>>> have to go to that interface for even 1 function I would do it for all
>>> -- from a user perspective it is just simpler to have 1 command to setup
>>> probes. I would prefer that 1 command be perf-probe.
>>
>> Yeah, but in that case, why you don't ask us adding sym->binding == STB_LOCAL
>> in filter_available_functions? :)
> 
> I did in a separate email -- you said because there can be multiple
> local functions with the same name.

Yeah, and this still seems to be a kind of workaround for me. The best way
to make you requirement enable is to support dwarf for userspace tracing.
OK I'll try it.

> But local functions is not the only
> use case I need.

What would you like to do with perf probe? Direct address accessing for
userspace is not a good way to do, because userspace is relocatable...

> For now I will carry the patch locally. At this point I am 20 patches
> deep and have probably another 20 to go. What's one more. I'll come back
> to this when I have more time.

Would you have any public git repository for that? And could you share
us what would you like to do before sending patch? We can help you to
tell the best way.

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 2/2] perf probe: Allow user to specify address within executable

[David Ahern]

On 12/4/13, 12:39 AM, Masami Hiramatsu wrote:
> What would you like to do with perf probe? Direct address accessing for
> userspace is not a good way to do, because userspace is relocatable...

Masami, I am referring to the address within the executable which is
what perf-probe pushes to the kernel.

> 
>> For now I will carry the patch locally. At this point I am 20 patches
>> deep and have probably another 20 to go. What's one more. I'll come back
>> to this when I have more time.
> 
> Would you have any public git repository for that?

https://github.com/dsahern/linux/tree/perf-full-monty


 And could you share
> us what would you like to do before sending patch? We can help you to
> tell the best way.

I want to save myself some time and effort by giving the relative
address with an executable. That allows me to probe any function, and I
want to do it from a consistent user interface -- perf-probe. With the
proposed patch I have that.

David