All of lore.kernel.org
 help / color / mirror / Atom feed
From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Leo Yan <leo.yan@linaro.org>
Cc: James Clark <james.clark@arm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Yonghong Song <yhs@fb.com>,
	John Fastabend <john.fastabend@gmail.com>,
	KP Singh <kpsingh@kernel.org>, Ian Rogers <irogers@google.com>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1] perf synthetic-events: Return error if procfs isn't mounted for PID namespaces
Date: Sun, 6 Feb 2022 08:46:37 -0300	[thread overview]
Message-ID: <Yf+1HQ9+7TPmSSr7@kernel.org> (raw)
In-Reply-To: <20220204124849.GB2040169@leoy-ThinkPad-X240s>

Em Fri, Feb 04, 2022 at 08:48:49PM +0800, Leo Yan escreveu:
> On Fri, Jan 07, 2022 at 02:27:57PM +0000, James Clark wrote:
> > On 24/12/2021 12:40, Leo Yan wrote:
> > > For perf recording, it retrieves process info by iterating nodes in proc
> > > fs.  If we run perf in a non-root PID namespace with command:
> > > 
> > >   # unshare --fork --pid perf record -e cycles -a -- test_program
> > > 
> > > ... in this case, unshare command creates a child PID namespace and
> > > launches perf tool in it, but the issue is the proc fs is not mounted
> > > for the non-root PID namespace, this leads to the perf tool gathering
> > > process info from its parent PID namespace.
> > > 
> > 
> > I had some concerns that this would prevent use of perf in docker, but docker
> > does mount /proc so it is still working. And you've added a warning about how
> > to fix the unshare command so I think this is ok.
> > 
> > Reviewed-by: James Clark <james.clark@arm.com>
> 
> Thanks a lot for review, James.
> 
> Arnaldo, Jiri, could you take a look for this patch?  Thanks!

Looks sane, there was discussion, a detailed set of steps to reproduce
the problem was provided, thanks, applied.

- Arnaldo
 
> Leo
> 
> > > We can use below command to observe the process nodes under proc fs:
> > > 
> > >   # unshare --pid --fork ls /proc
> > > 1    137   1968  2128  3    342  48  62   78	     crypto	  kcore        net	      uptime
> > > 10   138   2	 2142  30   35	 49  63   8	     devices	  keys	       pagetypeinfo   version
> > > 11   139   20	 2143  304  36	 50  64   82	     device-tree  key-users    partitions     vmallocinfo
> > > 12   14    2011  22    305  37	 51  65   83	     diskstats	  kmsg	       self	      vmstat
> > > 128  140   2038  23    307  39	 52  656  84	     driver	  kpagecgroup  slabinfo       zoneinfo
> > > 129  15    2074  24    309  4	 53  67   9	     execdomains  kpagecount   softirqs
> > > 13   16    2094  241   31   40	 54  68   asound     fb		  kpageflags   stat
> > > 130  164   2096  242   310  41	 55  69   buddyinfo  filesystems  loadavg      swaps
> > > 131  17    2098  25    317  42	 56  70   bus	     fs		  locks        sys
> > > 132  175   21	 26    32   43	 57  71   cgroups    interrupts   meminfo      sysrq-trigger
> > > 133  179   2102  263   329  44	 58  75   cmdline    iomem	  misc	       sysvipc
> > > 134  1875  2103  27    330  45	 59  76   config.gz  ioports	  modules      thread-self
> > > 135  19    2117  29    333  46	 6   77   consoles   irq	  mounts       timer_list
> > > 136  1941  2121  298   34   47	 60  773  cpuinfo    kallsyms	  mtd	       tty
> > > 
> > > So it shows many existed tasks, since unshared command has not mounted
> > > the proc fs for the new created PID namespace, it still accesses the
> > > proc fs of the root PID namespace.  This leads to two prominent issues:
> > > 
> > > - Firstly, PID values are mismatched between thread info and samples.
> > >   The gathered thread info are coming from the proc fs of the root PID
> > >   namespace, but samples record its PID from the child PID namespace.
> > > 
> > > - The second issue is profiled program 'test_program' returns its forked
> > >   PID number from the child PID namespace, perf tool wrongly uses this
> > >   PID number to retrieve the process info via the proc fs of the root
> > >   PID namespace.
> > > 
> > > To avoid issues, we need to mount proc fs for the child PID namespace
> > > with the option '--mount-proc' when use unshare command:
> > > 
> > >   # unshare --fork --pid --mount-proc perf record -e cycles -a -- test_program
> > > 
> > > Conversely, when the proc fs of the root PID namespace is used by child
> > > namespace, perf tool can detect the multiple PID levels and
> > > nsinfo__is_in_root_namespace() returns false, this patch reports error
> > > for this case:
> > > 
> > >   # unshare --fork --pid perf record -e cycles -a -- test_program
> > >   Couldn't synthesize bpf events.
> > >   Perf runs in non-root PID namespace but it tries to gather process info from its parent PID namespace.
> > >   Please mount the proc file system properly, e.g. add the option '--mount-proc' for unshare command.
> > > 
> > > Signed-off-by: Leo Yan <leo.yan@linaro.org>
> > > ---
> > >  tools/perf/util/synthetic-events.c | 19 +++++++++++++++++++
> > >  1 file changed, 19 insertions(+)
> > > 
> > > diff --git a/tools/perf/util/synthetic-events.c b/tools/perf/util/synthetic-events.c
> > > index 198982109f0f..cf7800347f77 100644
> > > --- a/tools/perf/util/synthetic-events.c
> > > +++ b/tools/perf/util/synthetic-events.c
> > > @@ -1784,6 +1784,25 @@ int __machine__synthesize_threads(struct machine *machine, struct perf_tool *too
> > >  				  perf_event__handler_t process, bool needs_mmap,
> > >  				  bool data_mmap, unsigned int nr_threads_synthesize)
> > >  {
> > > +	/*
> > > +	 * When perf runs in non-root PID namespace, and the namespace's proc FS
> > > +	 * is not mounted, nsinfo__is_in_root_namespace() returns false.
> > > +	 * In this case, the proc FS is coming for the parent namespace, thus
> > > +	 * perf tool will wrongly gather process info from its parent PID
> > > +	 * namespace.
> > > +	 *
> > > +	 * To avoid the confusion that the perf tool runs in a child PID
> > > +	 * namespace but it synthesizes thread info from its parent PID
> > > +	 * namespace, returns failure with warning.
> > > +	 */
> > > +	if (!nsinfo__is_in_root_namespace()) {
> > > +		pr_err("Perf runs in non-root PID namespace but it tries to ");
> > > +		pr_err("gather process info from its parent PID namespace.\n");
> > > +		pr_err("Please mount the proc file system properly, e.g. ");
> > > +		pr_err("add the option '--mount-proc' for unshare command.\n");
> > > +		return -EPERM;
> > > +	}
> > > +
> > >  	if (target__has_task(target))
> > >  		return perf_event__synthesize_thread_map(tool, threads, process, machine,
> > >  							 needs_mmap, data_mmap);
> > > 

-- 

- Arnaldo

  reply	other threads:[~2022-02-06 11:52 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-24 12:40 [PATCH v1] perf synthetic-events: Return error if procfs isn't mounted for PID namespaces Leo Yan
2021-12-24 12:43 ` Leo Yan
2022-01-07 14:27 ` James Clark
2022-02-04 12:48   ` Leo Yan
2022-02-06 11:46     ` Arnaldo Carvalho de Melo [this message]
2022-02-07  3:31       ` Leo Yan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Yf+1HQ9+7TPmSSr7@kernel.org \
    --to=acme@kernel.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=irogers@google.com \
    --cc=james.clark@arm.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@redhat.com \
    --cc=kafai@fb.com \
    --cc=kpsingh@kernel.org \
    --cc=leo.yan@linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=songliubraving@fb.com \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.