perf: remount of tracefs required

perf: remount of tracefs required

[Milian Wolff]

Hey all,

on ArchLinux with Linux 4.1.3, perf 4.1.0, I get:

$ perf trace
Error:  No permissions to read 
/sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'
$ sudo mount -o remount,mode=755 /sys/kernel/debug
$ perf trace
Error:  No permissions to read 
/sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'

This is b/c Arch uses tracefs, and running

$ sudo mount -o remount,mode=755 /sys/kernel/debug/tracefs

fixes the issue for me. Could someone please amend the error message? Do I 
also need to remount debugfs, or is tracefs sufficient for using more advanced 
perf tools?

Thanks

-- 
Milian Wolff
mail@milianw.de
http://milianw.de

Re: perf: remount of tracefs required

[Arnaldo Carvalho de Melo]

Em Tue, Aug 04, 2015 at 04:44:53PM +0200, Milian Wolff escreveu:
> on ArchLinux with Linux 4.1.3, perf 4.1.0, I get:
> 
> $ perf trace
> Error:  No permissions to read 
> /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
> Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'
> $ sudo mount -o remount,mode=755 /sys/kernel/debug
> $ perf trace
> Error:  No permissions to read 
> /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
> Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'
> 
> This is b/c Arch uses tracefs, and running
> 
> $ sudo mount -o remount,mode=755 /sys/kernel/debug/tracefs
> 
> fixes the issue for me. Could someone please amend the error message? Do I 
> also need to remount debugfs, or is tracefs sufficient for using more advanced 
> perf tools?

Well:

  [acme@zoo linux]$ perf trace ls
  Error:	No permissions to read
  /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
  Hint:	Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'

  [acme@zoo linux]$ ls -la /sys/kernel/debug
  ls: cannot open directory /sys/kernel/debug: Permission denied
  [acme@zoo linux]$

So, I can't access debugfs, that is where the canonical mount point for
tracefs is, so the message can be read as well as: "hey, let me get to
where tracefs is".

Once that is done:

  [acme@zoo linux]$ sudo mount -o remount,mode=755 /sys/kernel/debug
  [sudo] password for acme: 
  [acme@zoo linux]$ ls -la /sys/kernel/debug
  total 0
  drwxr-xr-x. 32 root root 0 Aug  4 08:35 .
  drwxr-xr-x. 10 root root 0 Aug  4 08:35 ..
  drwxr-xr-x.  2 root root 0 Aug  4 08:35 acpi
  <SNIP>
  drwxr-xr-x.  2 root root 0 Aug  4 08:35 zswap
  [acme@zoo linux]$ ls -la /sys/kernel/debug/tracing/
  ls: cannot open directory /sys/kernel/debug/tracing/: Permission denied
  [acme@zoo linux]$

So, if I try:

  [acme@zoo linux]$ trace usleep 1
  Error:	No permissions to read
  /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
  Hint:	Try 'sudo mount -o remount,mode=755 /sys/kernel/debug/tracing'

  [acme@zoo linux]$

It asks for that same remount to be done, this time for tracefs, once
that is done:

  [acme@zoo linux]$ sudo mount -o remount,mode=755 /sys/kernel/debug/tracing
  [acme@zoo linux]$ trace usleep 1
     0.086 (0.001 ms): usleep/10122 brk(                                                              ) = 0x556309717000
     0.101 (0.003 ms): usleep/10122 mmap(len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1) = 0x7f6f856ab000
  <SNIP>
  [acme@zoo linux]$

It works.

I guess what fixes what you report is this cset:

  commit ab85785aa13c36440a91a8e9f7616357de411a1f
  Author: Arnaldo Carvalho de Melo <acme@redhat.com>
  Date:   Mon Jul 6 08:51:41 2015 -0300

    tools lib api debugfs: Check for tracefs when reporting errors
    
    Now that we have two mountpoints, one for debugfs and another, for
    tracefs, we end up needing to check permissions for both, so, on
    a system with default config we were always asking the user to
    check the permission of the debugfs mountpoint, even when it was
    already sufficient. Fix it.

-----------

[acme@zoo linux]$ git describe ab85785aa13c36440a91a8e9f7616357de411a1f
v4.2-rc1-17-gab85785aa13c

So, can you try running the perf tools, built from the latest 4.2-rc codebase
to see if it behaves as you expect?

- Arnaldo

Re: perf: remount of tracefs required

[Milian Wolff]

On Tuesday 04 August 2015 12:27:46 Arnaldo Carvalho de Melo wrote:
> Em Tue, Aug 04, 2015 at 04:44:53PM +0200, Milian Wolff escreveu:
> > on ArchLinux with Linux 4.1.3, perf 4.1.0, I get:
> > 
> > $ perf trace
> > Error:  No permissions to read
> > /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
> > Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'
> > $ sudo mount -o remount,mode=755 /sys/kernel/debug
> > $ perf trace
> > Error:  No permissions to read
> > /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
> > Hint:   Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'
> > 
> > This is b/c Arch uses tracefs, and running
> > 
> > $ sudo mount -o remount,mode=755 /sys/kernel/debug/tracefs
> > 
> > fixes the issue for me. Could someone please amend the error message? Do I
> > also need to remount debugfs, or is tracefs sufficient for using more
> > advanced perf tools?
> 
> Well:
> 
>   [acme@zoo linux]$ perf trace ls
>   Error:	No permissions to read
>   /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
>   Hint:	Try 'sudo mount -o remount,mode=755 /sys/kernel/debug'
> 
>   [acme@zoo linux]$ ls -la /sys/kernel/debug
>   ls: cannot open directory /sys/kernel/debug: Permission denied
>   [acme@zoo linux]$
> 
> So, I can't access debugfs, that is where the canonical mount point for
> tracefs is, so the message can be read as well as: "hey, let me get to
> where tracefs is".

Yes, at that point this message is correct.

> Once that is done:
> 
>   [acme@zoo linux]$ sudo mount -o remount,mode=755 /sys/kernel/debug
>   [sudo] password for acme:
>   [acme@zoo linux]$ ls -la /sys/kernel/debug
>   total 0
>   drwxr-xr-x. 32 root root 0 Aug  4 08:35 .
>   drwxr-xr-x. 10 root root 0 Aug  4 08:35 ..
>   drwxr-xr-x.  2 root root 0 Aug  4 08:35 acpi
>   <SNIP>
>   drwxr-xr-x.  2 root root 0 Aug  4 08:35 zswap
>   [acme@zoo linux]$ ls -la /sys/kernel/debug/tracing/
>   ls: cannot open directory /sys/kernel/debug/tracing/: Permission denied
>   [acme@zoo linux]$
> 
> So, if I try:
> 
>   [acme@zoo linux]$ trace usleep 1
>   Error:	No permissions to read
>   /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
>   Hint:	Try 'sudo mount -o remount,mode=755 /sys/kernel/debug/tracing'

This message I do not get, not even with my perf compiled from the v4.2-rc5 
tag. And it is exactly what I was asking for.

<snip>

> I guess what fixes what you report is this cset:
> 
>   commit ab85785aa13c36440a91a8e9f7616357de411a1f
>   Author: Arnaldo Carvalho de Melo <acme@redhat.com>
>   Date:   Mon Jul 6 08:51:41 2015 -0300
> 
>     tools lib api debugfs: Check for tracefs when reporting errors
> 
>     Now that we have two mountpoints, one for debugfs and another, for
>     tracefs, we end up needing to check permissions for both, so, on
>     a system with default config we were always asking the user to
>     check the permission of the debugfs mountpoint, even when it was
>     already sufficient. Fix it.
> 
> -----------
> 
> [acme@zoo linux]$ git describe ab85785aa13c36440a91a8e9f7616357de411a1f
> v4.2-rc1-17-gab85785aa13c
> 
> So, can you try running the perf tools, built from the latest 4.2-rc
> codebase to see if it behaves as you expect?

This is not included in 4.2-rc5 as far as I can see:

$ git describe
v4.2-rc5-2-ga64d3e5
$ git show ab85785aa13c36440a91a8e9f7616357de411a1f
fatal: bad object ab85785aa13c36440a91a8e9f7616357de411a1f
$ git log --grep debugfs --oneline --author=Arnaldo
f816b3c tools lib fs debugfs: Check if debugfs is mounted when handling ENOENT
...

Thanks
-- 
Milian Wolff
mail@milianw.de
http://milianw.de

Re: perf: remount of tracefs required

[Arnaldo Carvalho de Melo]

Em Wed, Aug 05, 2015 at 04:17:42PM +0200, Milian Wolff escreveu:
> On Tuesday 04 August 2015 12:27:46 Arnaldo Carvalho de Melo wrote:
> >   [acme@zoo linux]$ trace usleep 1
> >   Error:	No permissions to read
> >   /sys/kernel/debug/tracing/events/raw_syscalls/sys_(enter|exit)
> >   Hint:	Try 'sudo mount -o remount,mode=755 /sys/kernel/debug/tracing'
 
> This message I do not get, not even with my perf compiled from the v4.2-rc5 
> tag. And it is exactly what I was asking for.
 
> <snip>
 
> > I guess what fixes what you report is this cset:

> >   commit ab85785aa13c36440a91a8e9f7616357de411a1f

> >     tools lib api debugfs: Check for tracefs when reporting errors

> > [acme@zoo linux]$ git describe ab85785aa13c36440a91a8e9f7616357de411a1f
> > v4.2-rc1-17-gab85785aa13c

> > So, can you try running the perf tools, built from the latest 4.2-rc
> > codebase to see if it behaves as you expect?
 
> This is not included in 4.2-rc5 as far as I can see:
 
> $ git describe
> v4.2-rc5-2-ga64d3e5
> $ git show ab85785aa13c36440a91a8e9f7616357de411a1f
> fatal: bad object ab85785aa13c36440a91a8e9f7616357de411a1f
> $ git log --grep debugfs --oneline --author=Arnaldo
> f816b3c tools lib fs debugfs: Check if debugfs is mounted when handling ENOENT
> ...

Oops, it is just in tip/perf/core, i.e. scheduled for 4.3 :-\ Wonder how
difficult would be to cherry-pick that into perf/urgent.

I.e. can you try doing something like:

  git cherry-pick ab85785aa13c36440a91a8e9f7616357de411a1f

With a checked out v4.2-rc5, or better, torvalds/master, i.e. the top of
Linus's git tree? It is possible that one or more patches just before
this one, also in perf/core, may be needed.

If it is easy, perhaps we can push it for 4.2 final.

Ah, you may want to take a look at:

https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/core

To get from this:

  [root@zoo ~]# trace -e newfstatat,unlinkat rm /tmp/foo
     0.489 ( 0.011 ms): rm/8428 newfstatat(dfd: CWD, filename: 0x9153f8, statbuf: 0x915368, flag: 256 ) = 0
     0.508 ( 0.010 ms): rm/8428 unlinkat(dfd: CWD, pathname: 0x9140c0) = 0
  [root@zoo ~]#

After doing this:

  [root@zoo ~]# perf probe 'vfs_getname=getname_flags:72 pathname=filename:string'
  Added new event:
    probe:vfs_getname    (on getname_flags:72 with pathname=filename:string)

  You can now use it in all perf tools, such as:

	perf record -e probe:vfs_getname -aR sleep 1

To this:

  [root@zoo ~]# trace -e newfstatat,unlinkat rm /tmp/foo
  rm: cannot remove ‘/tmp/foo’: No such file or directory
     0.503 ( 0.007 ms): rm/8480 newfstatat(dfd: CWD, filename: /tmp/foo, statbuf: 0x2370368, flag: 256) = -1 ENOENT No such file or directory
     0.514 ( 0.004 ms): rm/8480 unlinkat(dfd: CWD, pathname: /tmp/foo) = -1 ENOENT No such file or directory
  [root@zoo ~]#

Auto inserting that probe remains to be done, but for now can be done just once
per boot.

- Arnaldo