Re: [RFC] perf record: missing buildid for callstack modules

[Adrian Hunter <adrian.hunter@intel.com>]

On 09/01/16 12:31, Namhyung Kim wrote:
> Hi Stephane,
> 
> On Fri, Jan 08, 2016 at 10:01:24AM -0800, Stephane Eranian wrote:
>> On Thu, Jan 7, 2016 at 3:47 PM, Arnaldo Carvalho de Melo
>> <acme@kernel.org> wrote:
>>> Em Fri, Jan 08, 2016 at 07:47:03AM +0900, Namhyung Kim escreveu:
>>>> On January 8, 2016 7:00:35 AM GMT+09:00, Stephane Eranian <eranian@google.com> wrote:
>>>>> On Thu, Jan 7, 2016 at 1:59 PM, Arnaldo Carvalho de Melo
>>>>> <acme@kernel.org> wrote:
>>>>>> Em Thu, Jan 07, 2016 at 01:56:14PM -0800, Stephane Eranian escreveu:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Whenever you do:
>>>>>>>
>>>>>>>     $ perf record -g -a sleep 10
>>>>>>>
>>>>>>> Perf will collect the callstack for each sample. At the end of the
>>>>>>> run, perf record
>>>>>>> adds the buildid for all dso with at least one sample. But when it
>>>>> does this, it
>>>>>>> only looks at the sampled IP and ignore the modules traversed by the
>>>>> callstack.
>>>>>>> That means that, it is not possible to uniquely identify the modules
>>>>> executed,
>>>>>>> unless they had at least one IP sample captured. But this is not
>>>>>>> always the case.
>>>>>>>
>>>>>>> How about providing an option to perf record to force collecting
>>>>>>> buildid for all IPs
>>>>>>> captured in the callstack? I understand that would cost more at the
>>>>> end of the
>>>>>>> collection, but this would be beneficial to several monitoring
>>>>> scenarios.
>>>>>>
>>>>>> I agree, would consider applying a patch that provides the option but
>>>>>> does not do this by default.
>>>>>>
>>>>> I agree, not the default.
>>>>
>>>> Hi Stephane,
>>>>
>>>> Please see
>>>>
>>>> https://lkml.org/lkml/2015/3/22/249
>>>
>>>
>>> Oops, Stephane, please try this, so that we can finally merge it :-\
>>>
>> I will try it today. However, I am a bit worried about the performance
>> impact. Unless I am missing something in this approach we may end up
>> looking up N times the same module if it appears in N callstacks. In
>> Andi's suggested approach, there would be only one pass at the beginning
>> (or the end of the run). But you could miss some modules if they are gone
>> by the time you run the pass.
> 
> How about this then?
> 
> Adrian, is it ok to skip process_buildids() for the auxtrace?

If you don't post-process (i.e. call process_buildids), then where do the
DSOs come from? i.e. dsos__hit_all() just hits the DSOs that exist.

> 
> Thanks,
> Namhyung
> 
> 
> diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
> index 3a1a32f5479f..fbceb631387c 100644
> --- a/tools/perf/Documentation/perf-record.txt
> +++ b/tools/perf/Documentation/perf-record.txt
> @@ -338,6 +338,9 @@ Options passed to clang when compiling BPF scriptlets.
>  Specify vmlinux path which has debuginfo.
>  (enabled when BPF prologue is on)
>  
> +--buildid-all::
> +Record build-id of all DSOs regardless whether it's actually hit or not.
> +
>  SEE ALSO
>  --------
>  linkperf:perf-stat[1], linkperf:perf-list[1]
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index dc4e0adf5c5b..ab18db3153a6 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -50,6 +50,7 @@ struct record {
>  	int			realtime_prio;
>  	bool			no_buildid;
>  	bool			no_buildid_cache;
> +	bool			buildid_all;
>  	unsigned long long	samples;
>  };
>  
> @@ -755,14 +756,10 @@ out_child:
>  		file->size = lseek(perf_data_file__fd(file), 0, SEEK_CUR);
>  
>  		if (!rec->no_buildid) {
> -			process_buildids(rec);
> -			/*
> -			 * We take all buildids when the file contains
> -			 * AUX area tracing data because we do not decode the
> -			 * trace because it would take too long.
> -			 */
> -			if (rec->opts.full_auxtrace)
> +			if (rec->buildid_all)
>  				dsos__hit_all(rec->session);
> +			else
> +				process_buildids(rec);
>  		}
>  		perf_session__write_header(rec->session, rec->evlist, fd, true);
>  	}
> @@ -1138,6 +1135,8 @@ struct option __record_options[] = {
>  		   "options passed to clang when compiling BPF scriptlets"),
>  	OPT_STRING(0, "vmlinux", &symbol_conf.vmlinux_name,
>  		   "file", "vmlinux pathname"),
> +	OPT_BOOLEAN(0, "buildid-all", &record.buildid_all,
> +		    "Record build-id of all DSOs regardless of hits"),
>  	OPT_END()
>  };
>  
> @@ -1255,6 +1254,14 @@ int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
>  	if (err)
>  		goto out_symbol_exit;
>  
> +	/*
> +	 * We take all buildids when the file contains
> +	 * AUX area tracing data because we do not decode the
> +	 * trace because it would take too long.
> +	 */
> +	if (rec->opts.full_auxtrace)
> +		rec->buildid_all = true;
> +
>  	if (record_opts__config(&rec->opts)) {
>  		err = -EINVAL;
>  		goto out_symbol_exit;
>