[PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Wang YanQing]

I meet strange behavior with below commands on my gentoo box:
1:perf kmem record
2:CTRL-C to stop 1
3:perf report
4:"Enter", "Enter", "Run scripts for all samples",
  "event_analyzing_sample".

Then perf report:
"
No kallsyms or vmlinux with build-id xxxx was found
/lib/modules/4.10.0+/build/vmlinux with build id xxxx not found,
continuing without symbols
".

It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
is right for perf.data.

After digging, I find out the reason is "perf report" generates many
open fds, then "script_browse" uses popen to run "perf script" which
run out of open files. The gentoo box has a small default value for
"max open files", 1024. Yes, "ulimit -n " with a bigger number could
fix it, but I think that using O_CLOEXEC in do_open is a better way.

Signed-off-by: Wang YanQing <udknight@gmail.com>
---
 tools/perf/util/dso.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index d5b6f7f..36ef45b 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -446,7 +446,7 @@ static int do_open(char *name)
 	char sbuf[STRERR_BUFSIZE];
 
 	do {
-		fd = open(name, O_RDONLY);
+		fd = open(name, O_RDONLY|O_CLOEXEC);
 		if (fd >= 0)
 			return fd;
 
-- 
1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Jiri Olsa]

On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> I meet strange behavior with below commands on my gentoo box:
> 1:perf kmem record
> 2:CTRL-C to stop 1
> 3:perf report
> 4:"Enter", "Enter", "Run scripts for all samples",
>   "event_analyzing_sample".
> 
> Then perf report:
> "
> No kallsyms or vmlinux with build-id xxxx was found
> /lib/modules/4.10.0+/build/vmlinux with build id xxxx not found,
> continuing without symbols
> ".
> 
> It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> is right for perf.data.
> 
> After digging, I find out the reason is "perf report" generates many
> open fds, then "script_browse" uses popen to run "perf script" which
> run out of open files. The gentoo box has a small default value for
> "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> fix it, but I think that using O_CLOEXEC in do_open is a better way.

that seems right.. so why does kmem record open those
files before calling cmd_record? should that be fixed?

thanks,
jirka

> 
> Signed-off-by: Wang YanQing <udknight@gmail.com>
> ---
>  tools/perf/util/dso.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> index d5b6f7f..36ef45b 100644
> --- a/tools/perf/util/dso.c
> +++ b/tools/perf/util/dso.c
> @@ -446,7 +446,7 @@ static int do_open(char *name)
>  	char sbuf[STRERR_BUFSIZE];
>  
>  	do {
> -		fd = open(name, O_RDONLY);
> +		fd = open(name, O_RDONLY|O_CLOEXEC);
>  		if (fd >= 0)
>  			return fd;
>  
> -- 
> 1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Wang YanQing]

On Mon, Jan 15, 2018 at 10:45:06AM +0100, Jiri Olsa wrote:
> On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> > I meet strange behavior with below commands on my gentoo box:
> > 1:perf kmem record
> > 2:CTRL-C to stop 1
> > 3:perf report
> > 4:"Enter", "Enter", "Run scripts for all samples",
> >   "event_analyzing_sample".
> > 
> > Then perf report:
> > "
> > No kallsyms or vmlinux with build-id xxxx was found
> > /lib/modules/4.10.0+/build/vmlinux with build id xxxx not found,
> > continuing without symbols
> > ".
> > 
> > It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> > is right for perf.data.
> > 
> > After digging, I find out the reason is "perf report" generates many
> > open fds, then "script_browse" uses popen to run "perf script" which
> > run out of open files. The gentoo box has a small default value for
> > "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> > fix it, but I think that using O_CLOEXEC in do_open is a better way.
> 
> that seems right.. so why does kmem record open those
> files before calling cmd_record? should that be fixed?
The problem is "perf report", isn't "perf kmem record",
see the above reproduction steps again.

The problem is "perf report" generates may open fds with
dso:do_open for symbol analysis, and "perf report" uses
popen to run another command "perf script", then the cmd
"perf script" will run out of "max open files".

Thanks.
> 
> thanks,
> jirka
> 
> > 
> > Signed-off-by: Wang YanQing <udknight@gmail.com>
> > ---
> >  tools/perf/util/dso.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> > index d5b6f7f..36ef45b 100644
> > --- a/tools/perf/util/dso.c
> > +++ b/tools/perf/util/dso.c
> > @@ -446,7 +446,7 @@ static int do_open(char *name)
> >  	char sbuf[STRERR_BUFSIZE];
> >  
> >  	do {
> > -		fd = open(name, O_RDONLY);
> > +		fd = open(name, O_RDONLY|O_CLOEXEC);
> >  		if (fd >= 0)
> >  			return fd;
> >  
> > -- 
> > 1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Jiri Olsa]

On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> I meet strange behavior with below commands on my gentoo box:
> 1:perf kmem record
> 2:CTRL-C to stop 1
> 3:perf report
> 4:"Enter", "Enter", "Run scripts for all samples",
>   "event_analyzing_sample".
> 
> Then perf report:
> "
> No kallsyms or vmlinux with build-id xxxx was found
> /lib/modules/4.10.0+/build/vmlinux with build id xxxx not found,
> continuing without symbols
> ".
> 
> It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> is right for perf.data.
> 
> After digging, I find out the reason is "perf report" generates many
> open fds, then "script_browse" uses popen to run "perf script" which
> run out of open files. The gentoo box has a small default value for
> "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> fix it, but I think that using O_CLOEXEC in do_open is a better way.
> 
> Signed-off-by: Wang YanQing <udknight@gmail.com>

Acked-by: Jiri Olsa <jolsa@kernel.org>

thanks,
jirka

> ---
>  tools/perf/util/dso.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> index d5b6f7f..36ef45b 100644
> --- a/tools/perf/util/dso.c
> +++ b/tools/perf/util/dso.c
> @@ -446,7 +446,7 @@ static int do_open(char *name)
>  	char sbuf[STRERR_BUFSIZE];
>  
>  	do {
> -		fd = open(name, O_RDONLY);
> +		fd = open(name, O_RDONLY|O_CLOEXEC);
>  		if (fd >= 0)
>  			return fd;
>  
> -- 
> 1.8.5.6.2.g3d8a54e.dirty

[tip:perf/core] perf symbols: Using O_CLOEXEC in do_open

[tip-bot for Wang YanQing]

Commit-ID:  4c0d8d27954d9efb2a02ec9fc16f39b02f248bb7
Gitweb:     https://git.kernel.org/tip/4c0d8d27954d9efb2a02ec9fc16f39b02f248bb7
Author:     Wang YanQing <udknight@gmail.com>
AuthorDate: Mon, 15 Jan 2018 13:04:48 +0800
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Tue, 23 Jan 2018 09:49:28 -0300

perf symbols: Using O_CLOEXEC in do_open

I've meet a strange behavior with these commands on my gentoo box:

1: perf kmem record
2: CTRL-C to stop 1
3: perf report
4: "Enter", "Enter", "Run scripts for all samples",
   "event_analyzing_sample".

Then 'perf report' says:

  "
  No kallsyms or vmlinux with build-id xxxx was found
  /lib/modules/4.10.0+/build/vmlinux with build id xxxx not found,
  continuing without symbols
  ".

It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux is
right for perf.data.

After digging, I found out the reason is that "perf report" generates
many open fds, then "script_browse" uses popen to run "perf script"
which run out of open files.

The gentoo box has a small default value for "max open files", 1024.
Yes, "ulimit -n " with a bigger number could fix it, but I think that
using O_CLOEXEC in do_open is a better way.

Signed-off-by: Wang YanQing <udknight@gmail.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20180115050448.GA20759@udknight
[ Make sure O_CLOEXEC is available in old systems by adding a patch
  just before this one, to keep this bisectable in such systems ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/dso.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index d5b6f7f..36ef45b 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -446,7 +446,7 @@ static int do_open(char *name)
 	char sbuf[STRERR_BUFSIZE];
 
 	do {
-		fd = open(name, O_RDONLY);
+		fd = open(name, O_RDONLY|O_CLOEXEC);
 		if (fd >= 0)
 			return fd;