[PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Brian Robbins]

Programs can execute code contained in files that don't conform
to the ELF standard.  This is common for cross-platform runtimes
that support ahead-of-time compilation.  In this case, perf is
unable to resolve addresses to symbolic names, as the format of
the mmap'd file is unknown to perf.

Make perf fallback to JIT support (/tmp/perf-%pid.map) when
resolving function names if the file containing the code is not
an ELF binary.

Signed-off-by: Brian Robbins <brianrob@microsoft.com>
---
 tools/perf/util/map.c    |  1 +
 tools/perf/util/map.h    |  1 +
 tools/perf/util/symbol.c | 33 +++++++++++++++++++++++++++++++--
 3 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index 4e38c39..b33d460 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -164,6 +164,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,
 		map->ino_generation = ino_gen;
 		map->prot = prot;
 		map->flags = flags;
+		map->pid = pid;
 
 		if ((anon || no_dso) && type == MAP__FUNCTION) {
 			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
diff --git a/tools/perf/util/map.h b/tools/perf/util/map.h
index 7309d64..ba80f13 100644
--- a/tools/perf/util/map.h
+++ b/tools/perf/util/map.h
@@ -52,6 +52,7 @@ struct map {
 	struct dso		*dso;
 	struct map_groups	*groups;
 	atomic_t		refcnt;
+	u32			pid;
 };
 
 struct kmap {
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index b4cc766..12bc2f5 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -1516,7 +1516,7 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter)
 	}
 
 	if (!runtime_ss && !syms_ss)
-		goto out_free;
+		goto jit_fallback;
 
 	if (runtime_ss && !syms_ss) {
 		syms_ss = runtime_ss;
@@ -1541,7 +1541,36 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter)
 
 	for (; ss_pos > 0; ss_pos--)
 		symsrc__destroy(&ss_[ss_pos - 1]);
-out_free:
+
+jit_fallback:
+	/* If we couldn't load symbols, fall back to /tmp/perf-%pid.map. */
+	if (ret < 0) {
+		struct stat st;
+		const char *origlongname;
+		snprintf(name, PATH_MAX, "/tmp/perf-%d.map", map->pid);
+
+		if (lstat(name, &st) < 0)
+			goto out;
+
+		if (st.st_uid && (st.st_uid != geteuid())) {
+			pr_warning("File %s not owned by current user or root, "
+				"ignoring it.\n", name);
+			goto out;
+		}
+
+		origlongname = dso->long_name;
+		dso->long_name = name;
+
+		ret = dso__load_perf_map(dso, map, filter);
+
+		dso->long_name = origlongname;
+		dso->symtab_type = ret > 0 ? DSO_BINARY_TYPE__JAVA_JIT :
+					     DSO_BINARY_TYPE__NOT_FOUND;
+
+		if (ret > 0)
+			map->map_ip = map->unmap_ip = identity__map_ip;
+	}
+
 	free(name);
 	if (ret < 0 && strstr(dso->name, " (deleted)") != NULL)
 		ret = 0;
-- 
1.9.1

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Peter Zijlstra]

On Thu, Nov 19, 2015 at 11:09:07AM -0800, Brian Robbins wrote:
> Programs can execute code contained in files that don't conform
> to the ELF standard.  This is common for cross-platform runtimes
> that support ahead-of-time compilation.  In this case, perf is
> unable to resolve addresses to symbolic names, as the format of
> the mmap'd file is unknown to perf.

I'm assuming you're using PECOFF? Why not teach perf to read those?

Surely there's a library that can parse that stuff, it seems to me Wine
for one needs fairly decent support for this, also GDB (and other
toolchain bits) on Windows targets.

Its just that I was hoping we could move towards getting rid of those
/tmp files now that we have much better JIT support.

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Brian Robbins]

From: Peter Zijlstra <peterz@infradead.org>
Sent: Thursday, November 19, 2015 11:31 AM
To: Brian Robbins
Cc: Ingo Molnar; Arnaldo Carvalho de Melo; linux-kernel@vger.kernel.org; Stephane Eranian
Subject: Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

> On Thu, Nov 19, 2015 at 11:09:07AM -0800, Brian Robbins wrote:
> > Programs can execute code contained in files that don't conform
> > to the ELF standard.  This is common for cross-platform runtimes
> > that support ahead-of-time compilation.  In this case, perf is
> > unable to resolve addresses to symbolic names, as the format of
> > the mmap'd file is unknown to perf.

> I'm assuming you're using PECOFF? Why not teach perf to read those?

> Surely there's a library that can parse that stuff, it seems to me Wine
> for one needs fairly decent support for this, also GDB (and other
> toolchain bits) on Windows targets.

> Its just that I was hoping we could move towards getting rid of those
> /tmp files now that we have much better JIT support.

Hi Peter,

Thank you for the feedback.  The file format is similar to PE, but is not identical.  So, we would be implementing something very scoped, which doesn't feel right to me.

I am interested in the new JIT support, however my understanding from the information that I've read is that it requires kernel support in 4.x, though I can't seem to find where I read that.  I want to make sure that this works on older kernels (3.x) as well.

The reason I went with this approach is because it is simple for runtimes to implement and has no requirement that perf understand the file format.  I am open to feedback if there is a preferred solution that would still work for older kernels as well.

Thanks.
-Brian

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Stephane Eranian]

On Thu, Nov 19, 2015 at 3:45 PM, Brian Robbins <brianrob@microsoft.com> wrote:
>
> From: Peter Zijlstra <peterz@infradead.org>
> Sent: Thursday, November 19, 2015 11:31 AM
> To: Brian Robbins
> Cc: Ingo Molnar; Arnaldo Carvalho de Melo; linux-kernel@vger.kernel.org; Stephane Eranian
> Subject: Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.
>
> > On Thu, Nov 19, 2015 at 11:09:07AM -0800, Brian Robbins wrote:
> > > Programs can execute code contained in files that don't conform
> > > to the ELF standard.  This is common for cross-platform runtimes
> > > that support ahead-of-time compilation.  In this case, perf is
> > > unable to resolve addresses to symbolic names, as the format of
> > > the mmap'd file is unknown to perf.
>
> > I'm assuming you're using PECOFF? Why not teach perf to read those?
>
> > Surely there's a library that can parse that stuff, it seems to me Wine
> > for one needs fairly decent support for this, also GDB (and other
> > toolchain bits) on Windows targets.
>
> > Its just that I was hoping we could move towards getting rid of those
> > /tmp files now that we have much better JIT support.
>
> Hi Peter,
>
> Thank you for the feedback.  The file format is similar to PE, but is not identical.  So, we would be implementing something very scoped, which doesn't feel right to me.
>
> I am interested in the new JIT support, however my understanding from the information that I've read is that it requires kernel support in 4.x, though I can't seem to find where I read that.  I want to make sure that this works on older kernels (3.x) as well.
>
You need 4.1 if you have a situation where your JIT may recompile code
multiple times. The key difference is that
with 4.1 you can use the same timestamp clock source for both the JIT
runtime and the kernel perf_event subsystem.
That makes it possible to correlate samples from the JIT with samples
from the hardware. Without that, you may not
get correct symbolization if the code is re-jitted or moved.

>
> The reason I went with this approach is because it is simple for runtimes to implement and has no requirement that perf understand the file format.  I am open to feedback if there is a preferred solution that would still work for older kernels as well.
>
> Thanks.
> -Brian

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Brian Robbins]

On Fri, 20 Nov 2015, Stephane Eranian wrote:

> On Thu, Nov 19, 2015 at 3:45 PM, Brian Robbins <brianrob@microsoft.com> wrote:
> >
> > From: Peter Zijlstra <peterz@infradead.org>
> > Sent: Thursday, November 19, 2015 11:31 AM
> > To: Brian Robbins
> > Cc: Ingo Molnar; Arnaldo Carvalho de Melo; linux-kernel@vger.kernel.org; Stephane Eranian
> > Subject: Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.
> >
> > > On Thu, Nov 19, 2015 at 11:09:07AM -0800, Brian Robbins wrote:
> > > > Programs can execute code contained in files that don't conform
> > > > to the ELF standard.  This is common for cross-platform runtimes
> > > > that support ahead-of-time compilation.  In this case, perf is
> > > > unable to resolve addresses to symbolic names, as the format of
> > > > the mmap'd file is unknown to perf.
> >
> > > I'm assuming you're using PECOFF? Why not teach perf to read those?
> >
> > > Surely there's a library that can parse that stuff, it seems to me Wine
> > > for one needs fairly decent support for this, also GDB (and other
> > > toolchain bits) on Windows targets.
> >
> > > Its just that I was hoping we could move towards getting rid of those
> > > /tmp files now that we have much better JIT support.
> >
> > Hi Peter,
> >
> > Thank you for the feedback.  The file format is similar to PE, but is not identical.  So, we would be implementing something very scoped, which doesn't feel right to me.
> >
> > I am interested in the new JIT support, however my understanding from the information that I've read is that it requires kernel support in 4.x, though I can't seem to find where I read that.  I want to make sure that this works on older kernels (3.x) as well.
> >
> You need 4.1 if you have a situation where your JIT may recompile code
> multiple times. The key difference is that
> with 4.1 you can use the same timestamp clock source for both the JIT
> runtime and the kernel perf_event subsystem.
> That makes it possible to correlate samples from the JIT with samples
> from the hardware. Without that, you may not
> get correct symbolization if the code is re-jitted or moved.
> 

Thanks Stephane.  That's good to know.

Unfortunately, because the scenario here is not actually a JIT, it 
requires a bit more control around when the data is generated.  
Specifically, the requirement is that the data be generated offline as 
opposed to when the process is running.  Rather than using virtual 
addresses, we use offsets from the beginning of the file, and then update 
the data adding in the base address of the loaded file to get the full 
virtual address after the data has been collected.

Thus, the scenario is a bit different than a JIT scenario, but can re-use 
some of the plumbing.  Knowing this information, does this scenario sound 
OK?

I'd be happy to look at seeing what it would take to make this a bit 
cleaner (and deprecate the tmp file), but am keen to enable the scenario 
sooner rather than later.

Thanks again for your consideration.

-Brian


> >
> > The reason I went with this approach is because it is simple for runtimes to implement and has no requirement that perf understand the file format.  I am open to feedback if there is a preferred solution that would still work for older kernels as well.
> >
> > Thanks.
> > -Brian
> 

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Peter Zijlstra]

On Thu, Nov 19, 2015 at 11:45:45PM +0000, Brian Robbins wrote:

> Thank you for the feedback.  The file format is similar to PE, but is
> not identical.  So, we would be implementing something very scoped,
> which doesn't feel right to me.

*groan* you just had to go and invent yet another executable format,
right? :-)

> I am interested in the new JIT support, however my understanding from
> the information that I've read is that it requires kernel support in
> 4.x, though I can't seem to find where I read that.  I want to make
> sure that this works on older kernels (3.x) as well.

As I think Stephane explained, this is only required if you need to
match up kernel and userspace timestamps, which is important for dynamic
code generation, less so for static code in a weird format.

So what the new JIT stuff does is online write 'fake' ELF files with
symbol sections and (optionally?) dwarf debug info for line numbers.

Since you don't dynamically generate code, you can offline generate
these ELF files and redirect the symbol parser bits to that (we already
look for debug ELF files in various locations), or...

> The reason I went with this approach is because it is simple for
> runtimes to implement and has no requirement that perf understand the
> file format.  I am open to feedback if there is a preferred solution
> that would still work for older kernels as well.

Since, someone somewhere needs to go parse this funny new file format
anyhow to either generate /tmp files or fake ELF files or whatever, you
might as well put that decoder in perf?

Or just ship these fake ELF files in /usr/lib/debug/ or whatever the
'right' location for the distro at hand is.

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Brian Robbins]

On Fri, 20 Nov 2015, Peter Zijlstra wrote:

> On Thu, Nov 19, 2015 at 11:45:45PM +0000, Brian Robbins wrote:
> 
> > Thank you for the feedback.  The file format is similar to PE, but is
> > not identical.  So, we would be implementing something very scoped,
> > which doesn't feel right to me.
> 
> *groan* you just had to go and invent yet another executable format,
> right? :-)
Nah, it just has some extra stuff in it that makes this less desirable - 
it's technically PE, but anyway.
> 
> > I am interested in the new JIT support, however my understanding from
> > the information that I've read is that it requires kernel support in
> > 4.x, though I can't seem to find where I read that.  I want to make
> > sure that this works on older kernels (3.x) as well.
> 
> As I think Stephane explained, this is only required if you need to
> match up kernel and userspace timestamps, which is important for dynamic
> code generation, less so for static code in a weird format.

Yes, agreed.

> 
> So what the new JIT stuff does is online write 'fake' ELF files with
> symbol sections and (optionally?) dwarf debug info for line numbers.
> 
> Since you don't dynamically generate code, you can offline generate
> these ELF files and redirect the symbol parser bits to that (we already
> look for debug ELF files in various locations), or...
> 
> > The reason I went with this approach is because it is simple for
> > runtimes to implement and has no requirement that perf understand the
> > file format.  I am open to feedback if there is a preferred solution
> > that would still work for older kernels as well.
> 
> Since, someone somewhere needs to go parse this funny new file format
> anyhow to either generate /tmp files or fake ELF files or whatever, you
> might as well put that decoder in perf?
> 
> Or just ship these fake ELF files in /usr/lib/debug/ or whatever the
> 'right' location for the distro at hand is.
>

This seems like a reasonable approach.

Stephane, are your changes available for public consumption?  Last I 
recall, the patches were still in review.

Thanks.
-Brian 

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Arnaldo Carvalho de Melo]

Em Fri, Nov 20, 2015 at 01:18:55PM -0800, Brian Robbins escreveu:
> 
> 
> On Fri, 20 Nov 2015, Peter Zijlstra wrote:
> 
> > On Thu, Nov 19, 2015 at 11:45:45PM +0000, Brian Robbins wrote:
> > 
> > > Thank you for the feedback.  The file format is similar to PE, but is
> > > not identical.  So, we would be implementing something very scoped,
> > > which doesn't feel right to me.
> > 
> > *groan* you just had to go and invent yet another executable format,
> > right? :-)
> Nah, it just has some extra stuff in it that makes this less desirable - 
> it's technically PE, but anyway.
> > 
> > > I am interested in the new JIT support, however my understanding from
> > > the information that I've read is that it requires kernel support in
> > > 4.x, though I can't seem to find where I read that.  I want to make
> > > sure that this works on older kernels (3.x) as well.
> > 
> > As I think Stephane explained, this is only required if you need to
> > match up kernel and userspace timestamps, which is important for dynamic
> > code generation, less so for static code in a weird format.
> 
> Yes, agreed.
> 
> > 
> > So what the new JIT stuff does is online write 'fake' ELF files with
> > symbol sections and (optionally?) dwarf debug info for line numbers.
> > 
> > Since you don't dynamically generate code, you can offline generate
> > these ELF files and redirect the symbol parser bits to that (we already
> > look for debug ELF files in various locations), or...
> > 
> > > The reason I went with this approach is because it is simple for
> > > runtimes to implement and has no requirement that perf understand the
> > > file format.  I am open to feedback if there is a preferred solution
> > > that would still work for older kernels as well.
> > 
> > Since, someone somewhere needs to go parse this funny new file format
> > anyhow to either generate /tmp files or fake ELF files or whatever, you
> > might as well put that decoder in perf?
> > 
> > Or just ship these fake ELF files in /usr/lib/debug/ or whatever the
> > 'right' location for the distro at hand is.
> 
> This seems like a reasonable approach.
> 
> Stephane, are your changes available for public consumption?  Last I 
> recall, the patches were still in review.

Its in my queue, AFAIK there were not objections expressed, I'll try to
get this reviewed/merged soon.

- Arnaldo

Re: [PATCH] perf: Fallback to JIT support for mmap'd non-ELF binaries.

[Stephane Eranian]

On Fri, Nov 20, 2015 at 1:18 PM, Brian Robbins <brianrob@microsoft.com> wrote:
>
>
> On Fri, 20 Nov 2015, Peter Zijlstra wrote:
>
>> On Thu, Nov 19, 2015 at 11:45:45PM +0000, Brian Robbins wrote:
>>
>> > Thank you for the feedback.  The file format is similar to PE, but is
>> > not identical.  So, we would be implementing something very scoped,
>> > which doesn't feel right to me.
>>
>> *groan* you just had to go and invent yet another executable format,
>> right? :-)
> Nah, it just has some extra stuff in it that makes this less desirable -
> it's technically PE, but anyway.
>>
>> > I am interested in the new JIT support, however my understanding from
>> > the information that I've read is that it requires kernel support in
>> > 4.x, though I can't seem to find where I read that.  I want to make
>> > sure that this works on older kernels (3.x) as well.
>>
>> As I think Stephane explained, this is only required if you need to
>> match up kernel and userspace timestamps, which is important for dynamic
>> code generation, less so for static code in a weird format.
>
> Yes, agreed.
>
>>
>> So what the new JIT stuff does is online write 'fake' ELF files with
>> symbol sections and (optionally?) dwarf debug info for line numbers.
>>
The debug info section is optional and contains the line table info. This
can be used should you want to display source code along with assembly.

>> Since you don't dynamically generate code, you can offline generate
>> these ELF files and redirect the symbol parser bits to that (we already
>> look for debug ELF files in various locations), or...
>>
>> > The reason I went with this approach is because it is simple for
>> > runtimes to implement and has no requirement that perf understand the
>> > file format.  I am open to feedback if there is a preferred solution
>> > that would still work for older kernels as well.
>>
>> Since, someone somewhere needs to go parse this funny new file format
>> anyhow to either generate /tmp files or fake ELF files or whatever, you
>> might as well put that decoder in perf?
>>
If you have a new binary format, I think you'd have to write the parser in perf.

>> Or just ship these fake ELF files in /usr/lib/debug/ or whatever the
>> 'right' location for the distro at hand is.
>>
>
> This seems like a reasonable approach.
>
> Stephane, are your changes available for public consumption?  Last I
> recall, the patches were still in review.
>
I will post V8 this week.