[PATCH] samples/bpf: Fix cross compiler error with bpf sample

[PATCH] samples/bpf: Fix cross compiler error with bpf sample

[Joel Fernandes]

When cross-compiling the bpf sample map_perf_test for aarch64, I find that
__NR_getpgrp is undefined. This causes build errors. Fix it by allowing the
deprecated syscall in the sample.

Signed-off-by: Joel Fernandes <joelaf@google.com>
---
 samples/bpf/map_perf_test_user.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/samples/bpf/map_perf_test_user.c b/samples/bpf/map_perf_test_user.c
index 1a8894b5ac51..6e6fc7121640 100644
--- a/samples/bpf/map_perf_test_user.c
+++ b/samples/bpf/map_perf_test_user.c
@@ -8,7 +8,9 @@
 #include <sched.h>
 #include <stdio.h>
 #include <sys/types.h>
+#define __ARCH_WANT_SYSCALL_DEPRECATED
 #include <asm/unistd.h>
+#undef __ARCH_WANT_SYSCALL_DEPRECATED
 #include <unistd.h>
 #include <assert.h>
 #include <sys/wait.h>
-- 
2.14.0.rc1.383.gd1ce394fe2-goog

Re: [PATCH] samples/bpf: Fix cross compiler error with bpf sample

[Daniel Borkmann]

On 08/04/2017 07:46 AM, Joel Fernandes wrote:
> When cross-compiling the bpf sample map_perf_test for aarch64, I find that
> __NR_getpgrp is undefined. This causes build errors. Fix it by allowing the
> deprecated syscall in the sample.
>
> Signed-off-by: Joel Fernandes <joelaf@google.com>
> ---
>   samples/bpf/map_perf_test_user.c | 2 ++
>   1 file changed, 2 insertions(+)
>
> diff --git a/samples/bpf/map_perf_test_user.c b/samples/bpf/map_perf_test_user.c
> index 1a8894b5ac51..6e6fc7121640 100644
> --- a/samples/bpf/map_perf_test_user.c
> +++ b/samples/bpf/map_perf_test_user.c
> @@ -8,7 +8,9 @@
>   #include <sched.h>
>   #include <stdio.h>
>   #include <sys/types.h>
> +#define __ARCH_WANT_SYSCALL_DEPRECATED
>   #include <asm/unistd.h>
> +#undef __ARCH_WANT_SYSCALL_DEPRECATED

So the only arch that sets __ARCH_WANT_SYSCALL_DEPRECATED
is score:

   $ git grep -n __ARCH_WANT_SYSCALL_DEPRECATED
   arch/score/include/uapi/asm/unistd.h:7:#define __ARCH_WANT_SYSCALL_DEPRECATED
   include/uapi/asm-generic/unistd.h:837:#ifdef __ARCH_WANT_SYSCALL_DEPRECATED
   include/uapi/asm-generic/unistd.h:899:#endif /* __ARCH_WANT_SYSCALL_DEPRECATED */

But even if this would make aarch64 compile, the syscall
numbers don't match up:

   $ git grep -n __NR_getpgrp include/uapi/asm-generic/unistd.h
   include/uapi/asm-generic/unistd.h:841:#define __NR_getpgrp 1060
   include/uapi/asm-generic/unistd.h:843:__SYSCALL(__NR_getpgrp, sys_getpgrp)

The only thing that can be found on arm64 is:

   $ git grep -n __NR_getpgrp arch/arm64/
   arch/arm64/include/asm/unistd32.h:154:#define __NR_getpgrp 65
   arch/arm64/include/asm/unistd32.h:155:__SYSCALL(__NR_getpgrp, sys_getpgrp)

In arch/arm64/include/asm/unistd.h, it does include the
uapi/asm/unistd.h when compat is not set, but without the
__ARCH_WANT_SYSCALL_DEPRECATED. That doesn't look correct
unless I'm missing something, hmm, can't we just attach the
kprobes to a different syscall, one that is not deprecated,
so that we don't run into this in the first place?

Thanks,
Daniel

Re: [PATCH] samples/bpf: Fix cross compiler error with bpf sample

[Joel Fernandes]

On Fri, Aug 4, 2017 at 6:58 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 08/04/2017 07:46 AM, Joel Fernandes wrote:
>>
>> When cross-compiling the bpf sample map_perf_test for aarch64, I find that
>> __NR_getpgrp is undefined. This causes build errors. Fix it by allowing
>> the
>> deprecated syscall in the sample.
>>
>> Signed-off-by: Joel Fernandes <joelaf@google.com>
>> ---
>>   samples/bpf/map_perf_test_user.c | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/samples/bpf/map_perf_test_user.c
>> b/samples/bpf/map_perf_test_user.c
>> index 1a8894b5ac51..6e6fc7121640 100644
>> --- a/samples/bpf/map_perf_test_user.c
>> +++ b/samples/bpf/map_perf_test_user.c
>> @@ -8,7 +8,9 @@
>>   #include <sched.h>
>>   #include <stdio.h>
>>   #include <sys/types.h>
>> +#define __ARCH_WANT_SYSCALL_DEPRECATED
>>   #include <asm/unistd.h>
>> +#undef __ARCH_WANT_SYSCALL_DEPRECATED
>
>
> So the only arch that sets __ARCH_WANT_SYSCALL_DEPRECATED
> is score:
>
>   $ git grep -n __ARCH_WANT_SYSCALL_DEPRECATED
>   arch/score/include/uapi/asm/unistd.h:7:#define
> __ARCH_WANT_SYSCALL_DEPRECATED
>   include/uapi/asm-generic/unistd.h:837:#ifdef
> __ARCH_WANT_SYSCALL_DEPRECATED
>   include/uapi/asm-generic/unistd.h:899:#endif /*
> __ARCH_WANT_SYSCALL_DEPRECATED */
>
> But even if this would make aarch64 compile, the syscall
> numbers don't match up:
>
>   $ git grep -n __NR_getpgrp include/uapi/asm-generic/unistd.h
>   include/uapi/asm-generic/unistd.h:841:#define __NR_getpgrp 1060
>   include/uapi/asm-generic/unistd.h:843:__SYSCALL(__NR_getpgrp, sys_getpgrp)
>
> The only thing that can be found on arm64 is:
>
>   $ git grep -n __NR_getpgrp arch/arm64/
>   arch/arm64/include/asm/unistd32.h:154:#define __NR_getpgrp 65
>   arch/arm64/include/asm/unistd32.h:155:__SYSCALL(__NR_getpgrp, sys_getpgrp)
>
> In arch/arm64/include/asm/unistd.h, it does include the
> uapi/asm/unistd.h when compat is not set, but without the
> __ARCH_WANT_SYSCALL_DEPRECATED. That doesn't look correct
> unless I'm missing something, hmm, can't we just attach the
> kprobes to a different syscall, one that is not deprecated,
> so that we don't run into this in the first place?

Yes, I agree that's better. I think we can use getpgid. I'll try to
whip something today and send it out.

I also wanted to fix something else, HOSTCC is set to gcc, but I want
the boostrap part of the sample to run an ARM so I have to make HOSTCC
my cross-compiler. Right now I'm hacking it to point to the arm64 gcc
however I think I'd like to add a 'cross compile mode' or something
whether HOSTCC points to CROSS_COMPILE instead. I'm happy to discuss
any ideas to get this fixed too.

thanks!

-Joel


>
> Thanks,
> Daniel

Re: [PATCH] samples/bpf: Fix cross compiler error with bpf sample

[Daniel Borkmann]

On 08/04/2017 08:33 PM, Joel Fernandes wrote:
> On Fri, Aug 4, 2017 at 6:58 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 08/04/2017 07:46 AM, Joel Fernandes wrote:
>>>
>>> When cross-compiling the bpf sample map_perf_test for aarch64, I find that
>>> __NR_getpgrp is undefined. This causes build errors. Fix it by allowing
>>> the
>>> deprecated syscall in the sample.
>>>
>>> Signed-off-by: Joel Fernandes <joelaf@google.com>
>>> ---
>>>    samples/bpf/map_perf_test_user.c | 2 ++
>>>    1 file changed, 2 insertions(+)
>>>
>>> diff --git a/samples/bpf/map_perf_test_user.c
>>> b/samples/bpf/map_perf_test_user.c
>>> index 1a8894b5ac51..6e6fc7121640 100644
>>> --- a/samples/bpf/map_perf_test_user.c
>>> +++ b/samples/bpf/map_perf_test_user.c
>>> @@ -8,7 +8,9 @@
>>>    #include <sched.h>
>>>    #include <stdio.h>
>>>    #include <sys/types.h>
>>> +#define __ARCH_WANT_SYSCALL_DEPRECATED
>>>    #include <asm/unistd.h>
>>> +#undef __ARCH_WANT_SYSCALL_DEPRECATED
>>
>>
>> So the only arch that sets __ARCH_WANT_SYSCALL_DEPRECATED
>> is score:
>>
>>    $ git grep -n __ARCH_WANT_SYSCALL_DEPRECATED
>>    arch/score/include/uapi/asm/unistd.h:7:#define
>> __ARCH_WANT_SYSCALL_DEPRECATED
>>    include/uapi/asm-generic/unistd.h:837:#ifdef
>> __ARCH_WANT_SYSCALL_DEPRECATED
>>    include/uapi/asm-generic/unistd.h:899:#endif /*
>> __ARCH_WANT_SYSCALL_DEPRECATED */
>>
>> But even if this would make aarch64 compile, the syscall
>> numbers don't match up:
>>
>>    $ git grep -n __NR_getpgrp include/uapi/asm-generic/unistd.h
>>    include/uapi/asm-generic/unistd.h:841:#define __NR_getpgrp 1060
>>    include/uapi/asm-generic/unistd.h:843:__SYSCALL(__NR_getpgrp, sys_getpgrp)
>>
>> The only thing that can be found on arm64 is:
>>
>>    $ git grep -n __NR_getpgrp arch/arm64/
>>    arch/arm64/include/asm/unistd32.h:154:#define __NR_getpgrp 65
>>    arch/arm64/include/asm/unistd32.h:155:__SYSCALL(__NR_getpgrp, sys_getpgrp)
>>
>> In arch/arm64/include/asm/unistd.h, it does include the
>> uapi/asm/unistd.h when compat is not set, but without the
>> __ARCH_WANT_SYSCALL_DEPRECATED. That doesn't look correct
>> unless I'm missing something, hmm, can't we just attach the
>> kprobes to a different syscall, one that is not deprecated,
>> so that we don't run into this in the first place?
>
> Yes, I agree that's better. I think we can use getpgid. I'll try to
> whip something today and send it out.

Ok, cool. Please make sure that this doesn't clash with anything
else attached to map_perf_test_kern.c already given the obj file
is loaded first with the attachment points.

> I also wanted to fix something else, HOSTCC is set to gcc, but I want
> the boostrap part of the sample to run an ARM so I have to make HOSTCC
> my cross-compiler. Right now I'm hacking it to point to the arm64 gcc
> however I think I'd like to add a 'cross compile mode' or something
> whether HOSTCC points to CROSS_COMPILE instead. I'm happy to discuss
> any ideas to get this fixed too.

Yeah, sounds like a good idea to add such possibility. In case of
cross compiling to a target arch with different endianess, you might
also need to specifically select bpfeb (big endian) resp. bpfel
(little endian) as clang target. (Just bpf target uses host endianess.)

Thanks, Joel!

Re: [PATCH] samples/bpf: Fix cross compiler error with bpf sample

[Joel Fernandes]

Hi Daniel,

<snip>
>>>
>>> So the only arch that sets __ARCH_WANT_SYSCALL_DEPRECATED
>>> is score:
>>>
>>>    $ git grep -n __ARCH_WANT_SYSCALL_DEPRECATED
>>>    arch/score/include/uapi/asm/unistd.h:7:#define
>>> __ARCH_WANT_SYSCALL_DEPRECATED
>>>    include/uapi/asm-generic/unistd.h:837:#ifdef
>>> __ARCH_WANT_SYSCALL_DEPRECATED
>>>    include/uapi/asm-generic/unistd.h:899:#endif /*
>>> __ARCH_WANT_SYSCALL_DEPRECATED */
>>>
>>> But even if this would make aarch64 compile, the syscall
>>> numbers don't match up:
>>>
>>>    $ git grep -n __NR_getpgrp include/uapi/asm-generic/unistd.h
>>>    include/uapi/asm-generic/unistd.h:841:#define __NR_getpgrp 1060
>>>    include/uapi/asm-generic/unistd.h:843:__SYSCALL(__NR_getpgrp,
>>> sys_getpgrp)
>>>
>>> The only thing that can be found on arm64 is:
>>>
>>>    $ git grep -n __NR_getpgrp arch/arm64/
>>>    arch/arm64/include/asm/unistd32.h:154:#define __NR_getpgrp 65
>>>    arch/arm64/include/asm/unistd32.h:155:__SYSCALL(__NR_getpgrp,
>>> sys_getpgrp)
>>>
>>> In arch/arm64/include/asm/unistd.h, it does include the
>>> uapi/asm/unistd.h when compat is not set, but without the
>>> __ARCH_WANT_SYSCALL_DEPRECATED. That doesn't look correct
>>> unless I'm missing something, hmm, can't we just attach the
>>> kprobes to a different syscall, one that is not deprecated,
>>> so that we don't run into this in the first place?
>>
>>
>> Yes, I agree that's better. I think we can use getpgid. I'll try to
>> whip something today and send it out.

So switching to getppid fixes it, however most of the _kern.o BPF
samples still don't build on aarch64 for me.

The trouble is with inline assembler statements in the ARM64 asm
include headers (which I think upsets LLVM/clang when building for the
BPF target).

Also about skipping headers to fix this, currently we pass
-D__ASM_SYSREG_H from samples/bpf/Makefile so that inline assembler
statements are skipped (this was added in 2015 to this file). However,
not including this causes build errors at the moment. So as sysreg.h
isn't included, the SYS_MIDR_EL1 macro from
arch/arm64/include/asm/sysreg.h is missing causing this build error.
This caused by an include originating from skbuff.h :

In file included from samples/bpf/map_perf_test_kern.c:7:
In file included from ./include/linux/skbuff.h:17:
In file included from ./include/linux/kernel.h:13:
In file included from ./include/linux/printk.h:8:
In file included from ./include/linux/cache.h:5:
In file included from ./arch/arm64/include/asm/cache.h:19:
./arch/arm64/include/asm/cputype.h:119:9: error: use of undeclared
identifier 'SYS_MPIDR_EL1'
        return read_cpuid(MPIDR_EL1);
               ^

There are other paths too like mm.h including asm/pgtable.h which
breaks because of the inline assembler issues.

In file included from samples/bpf/map_perf_test_kern.c:7:
In file included from ./include/linux/skbuff.h:34:
In file included from ./include/linux/dma-mapping.h:10:
In file included from ./include/linux/scatterlist.h:7:
In file included from ./include/linux/mm.h:70:
./arch/arm64/include/asm/pgtable.h:607:9: error: value
'18446744073709550591' out of range for constraint 'L'
        : "L" (~PTE_AF), "I" (ilog2(PTE_AF)));


I believe the main problem here is we are trying to build for BPF
target while including ARM64 asm headers (which I think we can't
avoid?). I was thinking of just setting ARCH to x86 when compiling the
_kern.o parts of the bpf samples. However that doesn't make sense as
we probably need to be arch aware in the samples?

I am at a loss at the moment for ideas. I would love to get any other
ideas - how does x86 solve this issue?. I tried doing something like:
skip as many asm headers using the -D trick,  passing "-include
<stubs.h>" and defining as many things as I can in stubs.h as noops.
However I got stuck with that idea when I hit the above mm build
error.

I am also CC'ing some ARM64 folks as well for any thoughts, I am
looking forward to getting the samples working on my arm64 platform.

Thanks!

Best,

-Joel




>
> Ok, cool. Please make sure that this doesn't clash with anything
> else attached to map_perf_test_kern.c already given the obj file
> is loaded first with the attachment points.
>
>> I also wanted to fix something else, HOSTCC is set to gcc, but I want
>> the boostrap part of the sample to run an ARM so I have to make HOSTCC
>> my cross-compiler. Right now I'm hacking it to point to the arm64 gcc
>> however I think I'd like to add a 'cross compile mode' or something
>> whether HOSTCC points to CROSS_COMPILE instead. I'm happy to discuss
>> any ideas to get this fixed too.
>
>
> Yeah, sounds like a good idea to add such possibility. In case of
> cross compiling to a target arch with different endianess, you might
> also need to specifically select bpfeb (big endian) resp. bpfel
> (little endian) as clang target. (Just bpf target uses host endianess.)
>
> Thanks, Joel!