netronome/nfp/bpf/jit.c cannot be build with -O3

netronome/nfp/bpf/jit.c cannot be build with -O3

[Oleksandr Natalenko]

Hi.

Obligatory disclaimer: building the kernel with -O3 is a non-standard 
thing done via this patch [1], but I've asked people in #kernelnewbies, 
and it was suggested that the issue should be still investigated.

So, with v5.1 kernel release I cannot build the kernel with -O3 anymore. 
It fails as shown below:

===
   CC      drivers/net/ethernet/netronome/nfp/bpf/jit.o
In file included from ./include/asm-generic/bug.h:5,
                  from ./arch/x86/include/asm/bug.h:83,
                  from ./include/linux/bug.h:5,
                  from drivers/net/ethernet/netronome/nfp/bpf/jit.c:6:
In function ‘__emit_shf’,
     inlined from ‘emit_shf.constprop’ at 
drivers/net/ethernet/netronome/nfp/bpf/jit.c:364:2,
     inlined from ‘shl_reg64_lt32_low’ at 
drivers/net/ethernet/netronome/nfp/bpf/jit.c:379:2,
     inlined from ‘shl_reg’ at 
drivers/net/ethernet/netronome/nfp/bpf/jit.c:2506:2:
./include/linux/compiler.h:344:38: error: call to 
‘__compiletime_assert_341’ declared with attribute error: BUILD_BUG_ON 
failed: (((0x001f0000000ULL) + (1ULL << 
(__builtin_ffsll(0x001f0000000ULL) - 1))) & (((0x001f0000000ULL) + (1ULL 
<< (__builtin_ffsll(0x001f0000000ULL) - 1))) - 1)) != 0
   _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
                                       ^
./include/linux/compiler.h:325:4: note: in definition of macro 
‘__compiletime_assert’
     prefix ## suffix();    \
     ^~~~~~
./include/linux/compiler.h:344:2: note: in expansion of macro 
‘_compiletime_assert’
   _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
   ^~~~~~~~~~~~~~~~~~~
./include/linux/build_bug.h:39:37: note: in expansion of macro 
‘compiletime_assert’
  #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
                                      ^~~~~~~~~~~~~~~~~~
./include/linux/bitfield.h:57:3: note: in expansion of macro 
‘BUILD_BUG_ON_MSG’
    BUILD_BUG_ON_MSG(__builtin_constant_p(_val) ?  \
    ^~~~~~~~~~~~~~~~
./include/linux/bitfield.h:89:3: note: in expansion of macro 
‘__BF_FIELD_CHECK’
    __BF_FIELD_CHECK(_mask, 0ULL, _val, "FIELD_PREP: "); \
    ^~~~~~~~~~~~~~~~
drivers/net/ethernet/netronome/nfp/bpf/jit.c:341:3: note: in expansion 
of macro ‘FIELD_PREP’
    FIELD_PREP(OP_SHF_SHIFT, shift) |
    ^~~~~~~~~~
make[1]: *** [scripts/Makefile.build:276: 
drivers/net/ethernet/netronome/nfp/bpf/jit.o] Error 1
make: *** [Makefile:1726: drivers/net/ethernet/netronome/nfp/bpf/jit.o] 
Error 2
===

Needless to say, with -O2 this file is built just fine. My compiler is:

===
$ gcc --version
gcc (GCC) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is 
NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR 
PURPOSE.
===

I had no issues with -O3 before, so, maybe, this deserves a peek.

I'm open to testing patches and providing more info if needed.

Thanks.

[1] 
https://gitlab.com/post-factum/pf-kernel/commit/7fef93015ff1776d08119ef3d057a9e9433954a9

-- 
   Oleksandr Natalenko (post-factum)

Re: netronome/nfp/bpf/jit.c cannot be build with -O3

[Jakub Kicinski]

On Mon, 06 May 2019 21:40:07 +0200, Oleksandr Natalenko wrote:
> Hi.
> 
> Obligatory disclaimer: building the kernel with -O3 is a non-standard 
> thing done via this patch [1], but I've asked people in #kernelnewbies, 
> and it was suggested that the issue should be still investigated.
> 
> So, with v5.1 kernel release I cannot build the kernel with -O3 anymore. 
> It fails as shown below:

Any chance you could try different compiler versions?  The code in
question does not look too unusual.  Could you try if removing
FIELD_FIT() on line 326 makes a difference?

Re: netronome/nfp/bpf/jit.c cannot be build with -O3

[Oleksandr Natalenko]

Hi.

On 06.05.2019 23:00, Jakub Kicinski wrote:
> Any chance you could try different compiler versions?  The code in
> question does not look too unusual.  Could you try if removing
> FIELD_FIT() on line 326 makes a difference?

If building with gcc from CentOS 7:

gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)

the issue is not reproducible.

Also, commenting out the whole "if" block with FIELD_FIT() prevents the 
issue from occurring too.

-- 
   Oleksandr Natalenko (post-factum)

Re: netronome/nfp/bpf/jit.c cannot be build with -O3

[Jakub Kicinski]

On Mon, 06 May 2019 23:24:39 +0200, Oleksandr Natalenko wrote:
> Hi.
> 
> On 06.05.2019 23:00, Jakub Kicinski wrote:
> > Any chance you could try different compiler versions?  The code in
> > question does not look too unusual.  Could you try if removing
> > FIELD_FIT() on line 326 makes a difference?  
> 
> If building with gcc from CentOS 7:
> 
> gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)
> 
> the issue is not reproducible.

I just did a make CC=gcc-8 CFLAGS=-O3 with GCC 8.2 here, and doesn't
seem to trigger either.

> Also, commenting out the whole "if" block with FIELD_FIT() prevents the 
> issue from occurring too.

Hm, could it be that GCC 8.3 has some constant propagation bug which
breaks BUILD_BUG_ON()?  My uneducated guess it tries to pull the mask
validation out of FIELD_FIT() and FIELD_PREP() into one place but in
doing so loses some information.

Re: netronome/nfp/bpf/jit.c cannot be build with -O3

[Oleksandr Natalenko]

Hi.

On 06.05.2019 23:35, Jakub Kicinski wrote:
> I just did a make CC=gcc-8 CFLAGS=-O3 with GCC 8.2 here, and doesn't
> seem to trigger either.

I think passing CFLAGS like this is inefficient, and you should do it 
via Makefile directly.

-- 
   Oleksandr Natalenko (post-factum)

Re: [oss-drivers] netronome/nfp/bpf/jit.c cannot be build with -O3

[Jiong Wang]

'Oleksandr Natalenko' via OSS Drivers writes:

> Hi.
>
> Obligatory disclaimer: building the kernel with -O3 is a non-standard 
> thing done via this patch [1], but I've asked people in #kernelnewbies, 
> and it was suggested that the issue should be still investigated.
>
> So, with v5.1 kernel release I cannot build the kernel with -O3 anymore. 
> It fails as shown below:

I guess it's because constant prop. Could you try the following change to
__emit_shift?

drivers/net/ethernet/netronome/nfp/bpf/jit.c
__emit_shift:331
-       if (sc == SHF_SC_L_SHF)
+       if (sc == SHF_SC_L_SHF && shift)
                shift = 32 - shift;

emit_shf_indir is passing "0" as shift to __emit_shift which will
eventually be turned into 32 and it was OK because we truncate to 5-bit,
but before truncation, it will overflow the shift mask.

Regards,
Jiong

>
> ===
>    CC      drivers/net/ethernet/netronome/nfp/bpf/jit.o
> In file included from ./include/asm-generic/bug.h:5,
>                   from ./arch/x86/include/asm/bug.h:83,
>                   from ./include/linux/bug.h:5,
>                   from drivers/net/ethernet/netronome/nfp/bpf/jit.c:6:
> In function ‘__emit_shf’,
>      inlined from ‘emit_shf.constprop’ at 
> drivers/net/ethernet/netronome/nfp/bpf/jit.c:364:2,
>      inlined from ‘shl_reg64_lt32_low’ at 
> drivers/net/ethernet/netronome/nfp/bpf/jit.c:379:2,
>      inlined from ‘shl_reg’ at 
> drivers/net/ethernet/netronome/nfp/bpf/jit.c:2506:2:
> ./include/linux/compiler.h:344:38: error: call to 
> ‘__compiletime_assert_341’ declared with attribute error: BUILD_BUG_ON 
> failed: (((0x001f0000000ULL) + (1ULL << 
> (__builtin_ffsll(0x001f0000000ULL) - 1))) & (((0x001f0000000ULL) + (1ULL 
> << (__builtin_ffsll(0x001f0000000ULL) - 1))) - 1)) != 0
>    _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
>                                        ^
> ./include/linux/compiler.h:325:4: note: in definition of macro 
> ‘__compiletime_assert’
>      prefix ## suffix();    \
>      ^~~~~~
> ./include/linux/compiler.h:344:2: note: in expansion of macro 
> ‘_compiletime_assert’
>    _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
>    ^~~~~~~~~~~~~~~~~~~
> ./include/linux/build_bug.h:39:37: note: in expansion of macro 
> ‘compiletime_assert’
>   #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
>                                       ^~~~~~~~~~~~~~~~~~
> ./include/linux/bitfield.h:57:3: note: in expansion of macro 
> ‘BUILD_BUG_ON_MSG’
>     BUILD_BUG_ON_MSG(__builtin_constant_p(_val) ?  \
>     ^~~~~~~~~~~~~~~~
> ./include/linux/bitfield.h:89:3: note: in expansion of macro 
> ‘__BF_FIELD_CHECK’
>     __BF_FIELD_CHECK(_mask, 0ULL, _val, "FIELD_PREP: "); \
>     ^~~~~~~~~~~~~~~~
> drivers/net/ethernet/netronome/nfp/bpf/jit.c:341:3: note: in expansion 
> of macro ‘FIELD_PREP’
>     FIELD_PREP(OP_SHF_SHIFT, shift) |
>     ^~~~~~~~~~
> make[1]: *** [scripts/Makefile.build:276: 
> drivers/net/ethernet/netronome/nfp/bpf/jit.o] Error 1
> make: *** [Makefile:1726: drivers/net/ethernet/netronome/nfp/bpf/jit.o] 
> Error 2
> ===
>
> Needless to say, with -O2 this file is built just fine. My compiler is:
>
> ===
> $ gcc --version
> gcc (GCC) 8.3.0
> Copyright (C) 2018 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is 
> NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR 
> PURPOSE.
> ===
>
> I had no issues with -O3 before, so, maybe, this deserves a peek.
>
> I'm open to testing patches and providing more info if needed.
>
> Thanks.
>
> [1] 
> https://gitlab.com/post-factum/pf-kernel/commit/7fef93015ff1776d08119ef3d057a9e9433954a9

Re: [oss-drivers] netronome/nfp/bpf/jit.c cannot be build with -O3

[Oleksandr Natalenko]

Hi.

On 07.05.2019 00:01, Jiong Wang wrote:
> I guess it's because constant prop. Could you try the following change 
> to
> __emit_shift?
> 
> drivers/net/ethernet/netronome/nfp/bpf/jit.c
> __emit_shift:331
> -       if (sc == SHF_SC_L_SHF)
> +       if (sc == SHF_SC_L_SHF && shift)
>                 shift = 32 - shift;
> 
> emit_shf_indir is passing "0" as shift to __emit_shift which will
> eventually be turned into 32 and it was OK because we truncate to 
> 5-bit,
> but before truncation, it will overflow the shift mask.

Yup, it silences the error for me.

-- 
   Oleksandr Natalenko (post-factum)

Re: [oss-drivers] netronome/nfp/bpf/jit.c cannot be build with -O3

[Jiong Wang]

Oleksandr Natalenko writes:

> Hi.
>
> On 07.05.2019 00:01, Jiong Wang wrote:
>> I guess it's because constant prop. Could you try the following change 
>> to
>> __emit_shift?
>> 
>> drivers/net/ethernet/netronome/nfp/bpf/jit.c
>> __emit_shift:331
>> -       if (sc == SHF_SC_L_SHF)
>> +       if (sc == SHF_SC_L_SHF && shift)
>>                 shift = 32 - shift;
>> 
>> emit_shf_indir is passing "0" as shift to __emit_shift which will
>> eventually be turned into 32 and it was OK because we truncate to 
>> 5-bit,
>> but before truncation, it will overflow the shift mask.
>
> Yup, it silences the error for me.

Thanks for the testing.

I have also reproduced this issue after switching to gcc 8.3, and confirmed
the error is triggered from "value too large for the field" check inside
__BF_FIELD_CHECK due to immediate "32" is out of range for mask 0x1f.

Will send out a fix.

Regards,
Jiong