Per thread sampling

Per thread sampling

[Manuel Selva]

Hi all,

I just tried in a 2-threaded application to call twice (one call for
each thread) perf_event_open syscall with parameters configured to do
sampling on MEM INST RETIRED.LATENCY ABOVE THRESHOLD event. Both call
succeed.

"mmaping" the file descriptor return by the first call succeed but
"mmaping" the second file descriptor results in a "Operation not
permitted" error (errno = 1).

A work around could be to sample all threads (with pid = -1) including
pid and tid in samples and filter samples at processing time. Before
switching to this solution I wanted to ask if this a known limitation
of the syscall, an error from my side, or a bug  (the man page doesn't
answer to this question) ?

Thanks,

Manu

Re: Per thread sampling

[Jiri Olsa]

On Wed, Feb 19, 2014 at 04:00:45PM +0100, Manuel Selva wrote:
> Hi all,
> 
> I just tried in a 2-threaded application to call twice (one call for
> each thread) perf_event_open syscall with parameters configured to do
> sampling on MEM INST RETIRED.LATENCY ABOVE THRESHOLD event. Both call
> succeed.
> 
> "mmaping" the file descriptor return by the first call succeed but
> "mmaping" the second file descriptor results in a "Operation not
> permitted" error (errno = 1).

seems like you might have crossed the perf mem user limit?

[jolsa@krava ~]$ sysctl kernel.perf_event_mlock_kb 
kernel.perf_event_mlock_kb = 516

you can either split the memory amount between those
threads or you can increase that limit.. or run your
app under root ;-)

jirka

>mit 
> A work around could be to sample all threads (with pid = -1) including
> pid and tid in samples and filter samples at processing time. Before
> switching to this solution I wanted to ask if this a known limitation
> of the syscall, an error from my side, or a bug  (the man page doesn't
> answer to this question) ?
> 
> Thanks,
> 
> Manu
> --
> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Per thread sampling

[Manuel Selva]

Thanks for the answer Jirka. I saw this parameter and was wondering what 
was its purpose (because the name is
perf_event_mlock_kb and not
perf_event_mmap_kb).

Manu


On 02/20/2014 09:04 AM, Jiri Olsa wrote:
> On Wed, Feb 19, 2014 at 04:00:45PM +0100, Manuel Selva wrote:
>> Hi all,
>>
>> I just tried in a 2-threaded application to call twice (one call for
>> each thread) perf_event_open syscall with parameters configured to do
>> sampling on MEM INST RETIRED.LATENCY ABOVE THRESHOLD event. Both call
>> succeed.
>>
>> "mmaping" the file descriptor return by the first call succeed but
>> "mmaping" the second file descriptor results in a "Operation not
>> permitted" error (errno = 1).
>
> seems like you might have crossed the perf mem user limit?
>
> [jolsa@krava ~]$ sysctl kernel.perf_event_mlock_kb
> kernel.perf_event_mlock_kb = 516
>
> you can either split the memory amount between those
> threads or you can increase that limit.. or run your
> app under root ;-)
>
> jirka
>
>> mit
>> A work around could be to sample all threads (with pid = -1) including
>> pid and tid in samples and filter samples at processing time. Before
>> switching to this solution I wanted to ask if this a known limitation
>> of the syscall, an error from my side, or a bug  (the man page doesn't
>> answer to this question) ?
>>
>> Thanks,
>>
>> Manu
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Per thread sampling

[Dipanjan Sengupta]

Manuel Selva <selva.manuel <at> gmail.com> writes:


Hi Manu,

I am also working on a similar program. Would it be possible for you 
to share your implementation. 

Thanks,
Dipanjan