[Buildroot] Which order: $(MAKE) $(TARGET_CONFIGURE_OPTS) or $(TARGET_CONFIGURE_OPTS) $(MAKE)

[Buildroot] Which order: $(MAKE) $(TARGET_CONFIGURE_OPTS) or $(TARGET_CONFIGURE_OPTS) $(MAKE)

[Thomas De Schampheleire]

Hi,

In package makefiles, you can find both uses:

    $(MAKE) $(TARGET_CONFIGURE_OPTS)
versus
    $(TARGET_CONFIGURE_OPTS) $(MAKE)

and likewise for host packages:

    $(MAKE) $(HOST_CONFIGURE_OPTS)
versus
    $(HOST_CONFIGURE_OPTS) $(MAKE)


The order matters because it has impact on the precedence rules of make.
In the case where $(MAKE) is present first, the variables are passed
as make variables, which have precedence over variables explicitly
defined in the makefiles.
However, if $(MAKE) is _after_ the variable list, then the variables
are treated as environment variables. In make, environment variables
are directly readable as make variables, but an explicit assignment to
such variable overwrites the value passed via the environment, unless
the assignment is done conditionally, e.g. with ?= instead of = / :=.

Given this, if you want to make sure that the value you give is taken
into account, the forms:

    $(MAKE) $(TARGET_CONFIGURE_OPTS)
    $(MAKE) $(HOST_CONFIGURE_OPTS)

should be used.

Do you agree with this analysis?
If so, there is a cleanup opportunity for motivated developers :-)

Thanks,
Thomas

[Buildroot] Which order: $(MAKE) $(TARGET_CONFIGURE_OPTS) or $(TARGET_CONFIGURE_OPTS) $(MAKE)

[Thomas Petazzoni]

Hello,

On Wed, 29 Mar 2017 19:58:18 +0200, Thomas De Schampheleire wrote:

> The order matters because it has impact on the precedence rules of make.
> In the case where $(MAKE) is present first, the variables are passed
> as make variables, which have precedence over variables explicitly
> defined in the makefiles.
> However, if $(MAKE) is _after_ the variable list, then the variables
> are treated as environment variables. In make, environment variables
> are directly readable as make variables, but an explicit assignment to
> such variable overwrites the value passed via the environment, unless
> the assignment is done conditionally, e.g. with ?= instead of = / :=.
> 
> Given this, if you want to make sure that the value you give is taken
> into account, the forms:
> 
>     $(MAKE) $(TARGET_CONFIGURE_OPTS)
>     $(MAKE) $(HOST_CONFIGURE_OPTS)

Sadly, doing this breaks a number of packages. Many packages that use
hand-written Makefiles do:

CFLAGS += -I../include

for example. So if you pass CFLAGS="$(TARGET_CFLAGS)" as a make
variable (i.e after $(MAKE)), then it overrides the CFLAGS of the
package Makefile, and it no longer builds.

It would work if those packages were doing:

override CFLAGS += -I../include

but they often don't do this.

Recent example:

  https://git.buildroot.org/buildroot/commit/?id=f4dc73568b08bd96aa659c5ef29226349dee05de

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] Which order: $(MAKE) $(TARGET_CONFIGURE_OPTS) or $(TARGET_CONFIGURE_OPTS) $(MAKE)

[Thomas De Schampheleire]

On Wed, Mar 29, 2017 at 9:35 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> On Wed, 29 Mar 2017 19:58:18 +0200, Thomas De Schampheleire wrote:
>
>> The order matters because it has impact on the precedence rules of make.
>> In the case where $(MAKE) is present first, the variables are passed
>> as make variables, which have precedence over variables explicitly
>> defined in the makefiles.
>> However, if $(MAKE) is _after_ the variable list, then the variables
>> are treated as environment variables. In make, environment variables
>> are directly readable as make variables, but an explicit assignment to
>> such variable overwrites the value passed via the environment, unless
>> the assignment is done conditionally, e.g. with ?= instead of = / :=.
>>
>> Given this, if you want to make sure that the value you give is taken
>> into account, the forms:
>>
>>     $(MAKE) $(TARGET_CONFIGURE_OPTS)
>>     $(MAKE) $(HOST_CONFIGURE_OPTS)
>
> Sadly, doing this breaks a number of packages. Many packages that use
> hand-written Makefiles do:
>
> CFLAGS += -I../include
>
> for example. So if you pass CFLAGS="$(TARGET_CFLAGS)" as a make
> variable (i.e after $(MAKE)), then it overrides the CFLAGS of the
> package Makefile, and it no longer builds.
>
> It would work if those packages were doing:
>
> override CFLAGS += -I../include
>
> but they often don't do this.
>
> Recent example:
>
>   https://git.buildroot.org/buildroot/commit/?id=f4dc73568b08bd96aa659c5ef29226349dee05de
>

But, on the other hand, if a package would do:

CFLAGS = -I../include

then we cannot use either approach because one approach will ignore
the -I../include, and the other will ignore our own settings. In that
case, a patch of the package is required, right?

Is there a recommendation for the case that the package allows either way?

Thanks,
Thomas

[Buildroot] Which order: $(MAKE) $(TARGET_CONFIGURE_OPTS) or $(TARGET_CONFIGURE_OPTS) $(MAKE)

[Thomas Petazzoni]

Hello,

On Wed, 29 Mar 2017 21:43:18 +0200, Thomas De Schampheleire wrote:

> But, on the other hand, if a package would do:
> 
> CFLAGS = -I../include
> 
> then we cannot use either approach because one approach will ignore
> the -I../include, and the other will ignore our own settings. In that
> case, a patch of the package is required, right?

Correct. But I guess in many cases we don't realize when the package
"ignores" our CFLAGS, because there is nothing in our CFLAGS that is
really mandatory for the thing to build.

Of course, the fact that our CFLAGS may be ignored means that the
package may not have the correct optimization level, debugging level,
stack smashing protection flags, etc. I remember at some Buildroot
meeting, we discussed the idea of injecting a fake CFLAGS, and then
checking in the toolchain wrapper that we have this fake flag. But I'm
sure this would cause lots and lots of false positives.

> Is there a recommendation for the case that the package allows either way?

I don't really have a good suggestion. We tend to fix those issues on a
case by case basis, with no well defined "best practice".

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] Which order: $(MAKE) $(TARGET_CONFIGURE_OPTS) or $(TARGET_CONFIGURE_OPTS) $(MAKE)

[Arnout Vandecappelle]

On 29-03-17 21:49, Thomas Petazzoni wrote:
> Hello,
> 
> On Wed, 29 Mar 2017 21:43:18 +0200, Thomas De Schampheleire wrote:
> 
>> But, on the other hand, if a package would do:
>>
>> CFLAGS = -I../include
>>
>> then we cannot use either approach because one approach will ignore
>> the -I../include, and the other will ignore our own settings. In that
>> case, a patch of the package is required, right?
> 
> Correct. But I guess in many cases we don't realize when the package
> "ignores" our CFLAGS, because there is nothing in our CFLAGS that is
> really mandatory for the thing to build.
> 
> Of course, the fact that our CFLAGS may be ignored means that the
> package may not have the correct optimization level, debugging level,
> stack smashing protection flags, etc. I remember at some Buildroot
> meeting, we discussed the idea of injecting a fake CFLAGS, and then
> checking in the toolchain wrapper that we have this fake flag. But I'm
> sure this would cause lots and lots of false positives.

 Lots and lots of positives, indeed, but they wouldn't be false. They would be
annoying to fix, however, if there are really lots and lots.

 That's why I toyed with the idea of coding the CFLAGS directly in the toolchain
wrapper. But that's not good either, because in some cases a package really may
want to remove some CFLAG, e.g. ssp or some -mfpu option or something similar.
And for some packages, we intentionally pass our CFLAGS at all, e.g. linux. So
then we'd need to find a workaround for those packages again...

 We could of course try the add a poisoning option to CFLAGS somewhere in the
beginning of a cycle and see how far we get, and revert if necessary. But I'm
afraid it's a lot of work for relatively little gain.


>> Is there a recommendation for the case that the package allows either way?
> 
> I don't really have a good suggestion. We tend to fix those issues on a
> case by case basis, with no well defined "best practice".

 I think the best practice is to pass things in the environment and make an
upstreamable patch that changes CFLAGS assignments into ?= and/or +=.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF