Building tools/perf fails on next

Building tools/perf fails on next

[Riku Voipio]

Hi,

The commit:

commit d72da4a4d973d8a0a0d3c97e7cdebf287fbe3a99
Author: Peter Zijlstra <peterz@infradead.org>
Date:   Wed May 27 11:09:36 2015 +0930

    rbtree: Make lockless searches non-fatal

Adds <linux/rcupdate.h> to rbtree.h, which in turn is included from perf userspace
headers. Now building tools/perf will fail with hundreds of lines of gcc complaining
about kernel defines not available. Reverting the patch makes perf build again. 
This is with gcc-4.9 from debian but I don't think it's compiler specific.

Beginning of build error:

$ make -C tools/perf
make: Entering directory '/build/linux/tools/perf'
  BUILD:   Doing 'make -j4' parallel build

Auto-detecting system features:
...                         dwarf: [ on  ]
...                         glibc: [ on  ]
...                          gtk2: [ OFF ]
...                      libaudit: [ on  ]
...                        libbfd: [ OFF ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...                      libslang: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ OFF ]

config/Makefile:395: GTK2 not found, disables GTK2 support. Please install gtk2-devel or libgtk2.0-dev
config/Makefile:527: No bfd.h/libbfd found, please install binutils-dev[el]/zlib-static/libiberty-dev to gain symbol demangling
config/Makefile:556: No liblzma found, disables xz kernel module decompression, please install xz-devel/liblzma-dev
  CC       util/abspath.o
  CC       event-parse.o
  CC       fd/array.o
  LD       fd/libapi-in.o
  CC       fs/fs.o
  CC       fs/debugfs.o
  CC       event-plugin.o
  CC       fs/findfs.o
  CC       trace-seq.o
  CC       util/alias.o
  CC       parse-filter.o
  CC       fs/tracefs.o
  CC       util/annotate.o
  LD       fs/libapi-in.o
  LD       libapi-in.o
  AR       libapi.a
  CC       parse-utils.o
  GEN      common-cmds.h
  PERF_VERSION = 4.1.rc8.gca3cfa
  CC       kbuffer-parse.o
  LD       libtraceevent-in.o
  CC       arch/common.o
  LINK     libtraceevent.a
  CC       ui/setup.o
In file included from /build/linux/include/uapi/linux/kernel.h:4:0,
                 from /build/linux/include/linux/cache.h:4,
                 from /build/linux/include/linux/rcupdate.h:37,
                 from /build/linux/tools/perf/util/include/../../../../include/linux/rbtree.h:34,
                 from /build/linux/tools/perf/util/include/linux/rbtree.h:4,
                 from arch/../util/map.h:7,
                 from arch/../util/event.h:8,
                 from arch/../util/session.h:5,
                 from arch/common.h:4,
                 from arch/common.c:3:
/build/linux/include/uapi/linux/sysinfo.h:8:2: error: unknown type name ‘__kernel_long_t’
  __kernel_long_t uptime;  /* Seconds since boot */
  ^
/build/linux/include/uapi/linux/sysinfo.h:9:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t loads[3]; /* 1, 5, and 15 minute load averages */
  ^
/build/linux/include/uapi/linux/sysinfo.h:10:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t totalram; /* Total usable main memory size */
  ^
/build/linux/include/uapi/linux/sysinfo.h:11:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t freeram; /* Available memory size */
  ^
/build/linux/include/uapi/linux/sysinfo.h:12:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t sharedram; /* Amount of shared memory */
  ^
/build/linux/include/uapi/linux/sysinfo.h:13:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t bufferram; /* Memory used by buffers */
  ^
/build/linux/include/uapi/linux/sysinfo.h:14:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t totalswap; /* Total swap space size */
  ^
/build/linux/include/uapi/linux/sysinfo.h:15:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t freeswap; /* swap space still available */
  ^
/build/linux/include/uapi/linux/sysinfo.h:18:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t totalhigh; /* Total high memory size */
  ^
/build/linux/include/uapi/linux/sysinfo.h:19:2: error: unknown type name ‘__kernel_ulong_t’
  __kernel_ulong_t freehigh; /* Available high memory size */
  ^
/build/linux/include/uapi/linux/sysinfo.h:21:22: error: ‘__kernel_ulong_t’ undeclared here (not in a function)
  char _f[20-2*sizeof(__kernel_ulong_t)-sizeof(__u32)]; /* Padding: libc5 uses this.. */
....

[PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Ingo Molnar]

* Riku Voipio <riku.voipio@iki.fi> wrote:

> Hi,
> 
> The commit:
> 
> commit d72da4a4d973d8a0a0d3c97e7cdebf287fbe3a99
> Author: Peter Zijlstra <peterz@infradead.org>
> Date:   Wed May 27 11:09:36 2015 +0930
> 
>     rbtree: Make lockless searches non-fatal
> 
> Adds <linux/rcupdate.h> to rbtree.h, which in turn is included from perf userspace
> headers. Now building tools/perf will fail with hundreds of lines of gcc complaining
> about kernel defines not available. Reverting the patch makes perf build again. 
> This is with gcc-4.9 from debian but I don't think it's compiler specific.

Does the patch below make things work?

This fix could go into the modules tree, as this commit came via Rusty.

Stephen, feel free to add:

   make -C tools/perf

to the linux-next build tests. It's always supposed to build without failure, in 
pretty much whatever x86 distro you run your build tests on.

Thanks,

	Ingo


===================>
>From 62c251255f07ede8efa356d4ea9ab51827ffa0d0 Mon Sep 17 00:00:00 2001
From: Ingo Molnar <mingo@kernel.org>
Date: Wed, 17 Jun 2015 11:07:11 +0200
Subject: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

Reported-by: Riku Voipio <riku.voipio@iki.fi>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 tools/perf/util/include/linux/rcupdate.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/perf/util/include/linux/rcupdate.h b/tools/perf/util/include/linux/rcupdate.h
new file mode 100644
index 000000000000..3e022dd9a69b
--- /dev/null
+++ b/tools/perf/util/include/linux/rcupdate.h
@@ -0,0 +1,9 @@
+#ifndef PERF_LINUX_RCUPDATE_H_
+#define PERF_LINUX_RCUPDATE_H_
+
+/* Simple trivial wrappers for now, we don't use RCU in perf user-space (yet): */
+#define WRITE_ONCE(var, val)			((var) = (val))
+#define rcu_assign_pointer(ptr, val)		WRITE_ONCE(ptr, val)
+
+#endif
+

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Riku Voipio]

On Wednesday, June 17, 2015 12:17:04 PM EEST, Ingo Molnar wrote:
> * Riku Voipio <riku.voipio@iki.fi> wrote:
>
>> Hi,
>> 
>> The commit:
>> 
>> commit d72da4a4d973d8a0a0d3c97e7cdebf287fbe3a99
>> Author: Peter Zijlstra <peterz@infradead.org>
>> Date:   Wed May 27 11:09:36 2015 +0930 ...
>
> Does the patch below make things work?

It does,

Tested-by: Riku Voipio <riku.voipio@iki.fi>

> This fix could go into the modules tree, as this commit came via Rusty.
>
> Stephen, feel free to add:
>
>    make -C tools/perf
>
> to the linux-next build tests. It's always supposed to build 
> without failure, in 
> pretty much whatever x86 distro you run your build tests on.
>
> Thanks,
>
> 	Ingo
>
>
> ===================>
> From 62c251255f07ede8efa356d4ea9ab51827ffa0d0 Mon Sep 17 00:00:00 2001
> From: Ingo Molnar <mingo@kernel.org>
> Date: Wed, 17 Jun 2015 11:07:11 +0200
> Subject: [PATCH] tools/perf, rbtree: Add RCU wrappers to make 
> rbtree.h usable in user-space
>
> Reported-by: Riku Voipio <riku.voipio@iki.fi>
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> ---
>  tools/perf/util/include/linux/rcupdate.h | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/tools/perf/util/include/linux/rcupdate.h 
> b/tools/perf/util/include/linux/rcupdate.h
> new file mode 100644
> index 000000000000..3e022dd9a69b
> --- /dev/null
> +++ b/tools/perf/util/include/linux/rcupdate.h
> @@ -0,0 +1,9 @@
> +#ifndef PERF_LINUX_RCUPDATE_H_
> +#define PERF_LINUX_RCUPDATE_H_
> +
> +/* Simple trivial wrappers for now, we don't use RCU in perf 
> user-space (yet): */
> +#define WRITE_ONCE(var, val)			((var) = (val))
> +#define rcu_assign_pointer(ptr, val)		WRITE_ONCE(ptr, val)
> +
> +#endif
> +
>
>

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Tejun Heo]

(cc'ing Rusty and lkml)

Hello,

On Wed, Jul 01, 2015 at 10:18:48PM -0500, Jon Christopherson wrote:
> Hello guys,
> 
>     One last thing .. the recent commit : 02201e3f1 ("Merge tag
> 'modules-next-for-linus' of
> git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux") broke perf tools
> compilation. I know this isnt your area, but its a simple accidental
> omission of a file. Here is the needed change to correct retrieved from the
> list:
> 
> diff --git a/tools/perf/util/include/linux/rcupdate.h
> b/tools/perf/util/include/linux/rcupdate.h
> new file mode 100644
> index 0000000..51c0f45
> --- /dev/null
> +++ b/tools/perf/util/include/linux/rcupdate.h
> @@ -0,0 +1,9 @@
> +#ifndef PERF_LINUX_RCUPDATE_H_
> +#define PERF_LINUX_RCUPDATE_H_
> +
> +/* Simple trivial wrappers for now, we don't use RCU in perf user-space
> (yet): */
> +#define WRITE_ONCE(var, val) ((var) = (val))
> +#define rcu_assign_pointer(ptr, val) WRITE_ONCE(ptr, val)
> +
> +#endif
> +

Rusty?

Thanks.

-- 
tejun

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Rusty Russell]

Tejun Heo <tj@kernel.org> writes:
> (cc'ing Rusty and lkml)

Looks like Peter Zijlstra is the one to take this fix...

Cheers,
Rusty.

>
> Hello,
>
> On Wed, Jul 01, 2015 at 10:18:48PM -0500, Jon Christopherson wrote:
>> Hello guys,
>> 
>>     One last thing .. the recent commit : 02201e3f1 ("Merge tag
>> 'modules-next-for-linus' of
>> git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux") broke perf tools
>> compilation. I know this isnt your area, but its a simple accidental
>> omission of a file. Here is the needed change to correct retrieved from the
>> list:
>> 
>> diff --git a/tools/perf/util/include/linux/rcupdate.h
>> b/tools/perf/util/include/linux/rcupdate.h
>> new file mode 100644
>> index 0000000..51c0f45
>> --- /dev/null
>> +++ b/tools/perf/util/include/linux/rcupdate.h
>> @@ -0,0 +1,9 @@
>> +#ifndef PERF_LINUX_RCUPDATE_H_
>> +#define PERF_LINUX_RCUPDATE_H_
>> +
>> +/* Simple trivial wrappers for now, we don't use RCU in perf user-space
>> (yet): */
>> +#define WRITE_ONCE(var, val) ((var) = (val))
>> +#define rcu_assign_pointer(ptr, val) WRITE_ONCE(ptr, val)
>> +
>> +#endif
>> +
>
> Rusty?
>
> Thanks.
>
> -- 
> tejun

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Peter Zijlstra]

On Fri, Jul 03, 2015 at 06:21:12AM +0930, Rusty Russell wrote:
> Looks like Peter Zijlstra is the one to take this fix...

acme is the steward of tools/perf/

> >> diff --git a/tools/perf/util/include/linux/rcupdate.h
> >> b/tools/perf/util/include/linux/rcupdate.h
> >> new file mode 100644
> >> index 0000000..51c0f45
> >> --- /dev/null
> >> +++ b/tools/perf/util/include/linux/rcupdate.h
> >> @@ -0,0 +1,9 @@
> >> +#ifndef PERF_LINUX_RCUPDATE_H_
> >> +#define PERF_LINUX_RCUPDATE_H_
> >> +
> >> +/* Simple trivial wrappers for now, we don't use RCU in perf user-space
> >> (yet): */
> >> +#define WRITE_ONCE(var, val) ((var) = (val))

It looks like perf includes linux/compiler.h so it should already have this.

> >> +#define rcu_assign_pointer(ptr, val) WRITE_ONCE(ptr, val)

That's plain wrong, WRITE_ONCE(*(ptr), (val))

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Jon Christopherson]

On 07/03/2015 02:14 AM, Peter Zijlstra wrote:
> On Fri, Jul 03, 2015 at 06:21:12AM +0930, Rusty Russell wrote:
>> Looks like Peter Zijlstra is the one to take this fix...
>
> acme is the steward of tools/perf/

This is the full context of the patch mentioned:

https://lkml.org/lkml/2015/6/17/129

>
>>>> diff --git a/tools/perf/util/include/linux/rcupdate.h
>>>> b/tools/perf/util/include/linux/rcupdate.h
>>>> new file mode 100644
>>>> index 0000000..51c0f45
>>>> --- /dev/null
>>>> +++ b/tools/perf/util/include/linux/rcupdate.h
>>>> @@ -0,0 +1,9 @@
>>>> +#ifndef PERF_LINUX_RCUPDATE_H_
>>>> +#define PERF_LINUX_RCUPDATE_H_
>>>> +
>>>> +/* Simple trivial wrappers for now, we don't use RCU in perf user-space
>>>> (yet): */
>>>> +#define WRITE_ONCE(var, val) ((var) = (val))
>
> It looks like perf includes linux/compiler.h so it should already have this.
>
>>>> +#define rcu_assign_pointer(ptr, val) WRITE_ONCE(ptr, val)
>
> That's plain wrong, WRITE_ONCE(*(ptr), (val))
>

The original author of the patch appears to be Ingo. Syntax aside .. it 
solves the issue mentioned. Perhaps a corrected version could be 
included instead.

-Jon

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Arnaldo Carvalho de Melo]

Em Fri, Jul 03, 2015 at 09:14:46AM +0200, Peter Zijlstra escreveu:
> On Fri, Jul 03, 2015 at 06:21:12AM +0930, Rusty Russell wrote:
> > Looks like Peter Zijlstra is the one to take this fix...
> 
> acme is the steward of tools/perf/
> 
> > >> diff --git a/tools/perf/util/include/linux/rcupdate.h
> > >> b/tools/perf/util/include/linux/rcupdate.h
> > >> new file mode 100644
> > >> index 0000000..51c0f45
> > >> --- /dev/null
> > >> +++ b/tools/perf/util/include/linux/rcupdate.h
> > >> @@ -0,0 +1,9 @@
> > >> +#ifndef PERF_LINUX_RCUPDATE_H_
> > >> +#define PERF_LINUX_RCUPDATE_H_
> > >> +
> > >> +/* Simple trivial wrappers for now, we don't use RCU in perf user-space
> > >> (yet): */
> > >> +#define WRITE_ONCE(var, val) ((var) = (val))
> 
> It looks like perf includes linux/compiler.h so it should already have this.
> 
> > >> +#define rcu_assign_pointer(ptr, val) WRITE_ONCE(ptr, val)
> 
> That's plain wrong, WRITE_ONCE(*(ptr), (val))

Are you sure?

In the kernel, we have this sequence:

#define rcu_assign_pointer(p, v) smp_store_release(&p, RCU_INITIALIZER(v))

#define smp_store_release(p, v)			\
do {						\
        compiletime_assert_atomic_type(*p);	\
        smp_mb();				\
        ACCESS_ONCE(*p) = (v);			\
} while (0)


So, if you go shortcircuiting things you remove that & and that *, no?

I.e. end up with what Rusty suggested.

So, I am trying to keep as much as the semantics of the kernel not to
fall into thse traps...

Will post a RFC soon, if the rain continues preventing me from
running...

- Arnaldo

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 2528 bytes --]

Hi Ingo,

On Wed, 17 Jun 2015 11:17:04 +0200 Ingo Molnar <mingo@kernel.org> wrote:
>
> * Riku Voipio <riku.voipio@iki.fi> wrote:
> 
> > The commit:
> > 
> > commit d72da4a4d973d8a0a0d3c97e7cdebf287fbe3a99
> > Author: Peter Zijlstra <peterz@infradead.org>
> > Date:   Wed May 27 11:09:36 2015 +0930
> > 
> >     rbtree: Make lockless searches non-fatal
> > 
> > Adds <linux/rcupdate.h> to rbtree.h, which in turn is included from perf userspace
> > headers. Now building tools/perf will fail with hundreds of lines of gcc complaining
> > about kernel defines not available. Reverting the patch makes perf build again. 
> > This is with gcc-4.9 from debian but I don't think it's compiler specific.
> 
> Does the patch below make things work?
> 
> This fix could go into the modules tree, as this commit came via Rusty.
> 
> Stephen, feel free to add:
> 
>    make -C tools/perf
>
> to the linux-next build tests. It's always supposed to build without failure, in 
> pretty much whatever x86 distro you run your build tests on.

OK, I have started doing that, but int order to make it build at all, I
have added the patch below to my fixes tree (since the breakage is now
in Linus' tree).

This means that it will get a conflict with the tip tree tomorrow, but
I will just fix the conflict by using the version from the tip tree.
Once the tip tree fixes are sent to Linus, I will drop this patch from
my fixes tree.

> From 62c251255f07ede8efa356d4ea9ab51827ffa0d0 Mon Sep 17 00:00:00 2001
> From: Ingo Molnar <mingo@kernel.org>
> Date: Wed, 17 Jun 2015 11:07:11 +0200
> Subject: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space
> 
> Reported-by: Riku Voipio <riku.voipio@iki.fi>
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> ---
>  tools/perf/util/include/linux/rcupdate.h | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/tools/perf/util/include/linux/rcupdate.h b/tools/perf/util/include/linux/rcupdate.h
> new file mode 100644
> index 000000000000..3e022dd9a69b
> --- /dev/null
> +++ b/tools/perf/util/include/linux/rcupdate.h
> @@ -0,0 +1,9 @@
> +#ifndef PERF_LINUX_RCUPDATE_H_
> +#define PERF_LINUX_RCUPDATE_H_
> +
> +/* Simple trivial wrappers for now, we don't use RCU in perf user-space (yet): */
> +#define WRITE_ONCE(var, val)			((var) = (val))
> +#define rcu_assign_pointer(ptr, val)		WRITE_ONCE(ptr, val)
> +
> +#endif
> +

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Ingo,
> 
> On Wed, 17 Jun 2015 11:17:04 +0200 Ingo Molnar <mingo@kernel.org> wrote:
> >
> > * Riku Voipio <riku.voipio@iki.fi> wrote:
> > 
> > > The commit:
> > > 
> > > commit d72da4a4d973d8a0a0d3c97e7cdebf287fbe3a99
> > > Author: Peter Zijlstra <peterz@infradead.org>
> > > Date:   Wed May 27 11:09:36 2015 +0930
> > > 
> > >     rbtree: Make lockless searches non-fatal
> > > 
> > > Adds <linux/rcupdate.h> to rbtree.h, which in turn is included from perf userspace
> > > headers. Now building tools/perf will fail with hundreds of lines of gcc complaining
> > > about kernel defines not available. Reverting the patch makes perf build again. 
> > > This is with gcc-4.9 from debian but I don't think it's compiler specific.
> > 
> > Does the patch below make things work?
> > 
> > This fix could go into the modules tree, as this commit came via Rusty.
> > 
> > Stephen, feel free to add:
> > 
> >    make -C tools/perf
> >
> > to the linux-next build tests. It's always supposed to build without failure, in 
> > pretty much whatever x86 distro you run your build tests on.
> 
> OK, I have started doing that, but int order to make it build at all, I
> have added the patch below to my fixes tree (since the breakage is now
> in Linus' tree).
> 
> This means that it will get a conflict with the tip tree tomorrow, but
> I will just fix the conflict by using the version from the tip tree.
> Once the tip tree fixes are sent to Linus, I will drop this patch from
> my fixes tree.

On next linux-next iteration you should be able to just drop your merge conflict 
resolution, and use the -tip tree as-is with no extra fixups.

Thanks,

	Ingo

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Peter Zijlstra]

On Sat, Jul 04, 2015 at 01:15:56PM -0300, Arnaldo Carvalho de Melo wrote:
> > That's plain wrong, WRITE_ONCE(*(ptr), (val))
> 
> Are you sure?

I'm sure I was having a heat stroke,.. that original was fine.

Re: [PATCH] tools/perf, rbtree: Add RCU wrappers to make rbtree.h usable in user-space

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

Hi Ingo,

On Mon, 6 Jul 2015 11:41:17 +0200 Ingo Molnar <mingo@kernel.org> wrote:
>
> On next linux-next iteration you should be able to just drop your merge conflict 
> resolution, and use the -tip tree as-is with no extra fixups.

I dropped the patch I had yesterday and just merged
perf-urgent-for-linus from the tip tree into my fixes tree (since I
merge my fixes tree immediately on top of Linus' tree and before I try
to do any builds).  I will drop that merge when Linus takes your pull
request.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]