[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Carlos Santos]

When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
this flag requires GCC v4.9 or later. Older versions fail, complaining
about the unrecognized command line option.

The problem is not detectable in the autobuilders because they redirect
the output to files/pipes and ccache adds the offending flag only when
stderr is a terminal.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
---
Change-Id: I0574c7527b24d9e21119961adfa1c7aa17337314

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
---
 package/Makefile.in | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/package/Makefile.in b/package/Makefile.in
index 5d591e9..6a010b1 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -237,6 +237,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
 
 TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifneq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),y)
+	TARGET_MAKE_ENV += GCC_COLORS=""
+endif
+
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
 	AR="$(TARGET_AR)" \
@@ -286,6 +293,11 @@ HOST_MAKE_ENV = \
 	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
 	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
 
+# Prevent ccache from passing '-fdiagnostics-color' (see above).
+ifneq ($(BR2_HOST_GCC_AT_LEAST_4_9),y)
+	HOST_MAKE_ENV += GCC_COLORS=""
+endif
+
 HOST_CONFIGURE_OPTS = \
 	$(HOST_MAKE_ENV) \
 	AR="$(HOSTAR)" \
-- 
2.7.4

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Thomas Petazzoni]

Hello,

Adding Arnout, Yann and Gustavo in Cc.

On Fri, 10 Feb 2017 11:34:08 -0200, Carlos Santos wrote:
> When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
> this flag requires GCC v4.9 or later. Older versions fail, complaining
> about the unrecognized command line option.
> 
> The problem is not detectable in the autobuilders because they redirect
> the output to files/pipes and ccache adds the offending flag only when
> stderr is a terminal.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.ind.br>

This looks good to me in principle, so if Arnout, Yann and Gustavo
don't complain, I'm going to soon apply this to master.

Another option would be to adjust this in ccache itself at compile time
(i.e when ccache gets built) rather than having to pass this additional
GCC_COLORS variable in the environment when building things. This way
we would be absolutely sure -fdiagnostics-color will never be passed.
Hence why I'd like the feedback from other developers.

A few nits below (but I can fix while applying).

> +# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
> +# this flag requires GCC v4.9 or later. Older versions fail, complaining
> +# about unrecognized command line option.
> +ifneq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),y)

I think we tend to use:

ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)

instead.

> +	TARGET_MAKE_ENV += GCC_COLORS=""

And we don't indent such lines.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

Thomas, Carlos, All,

On 2017-02-11 16:38 +0100, Thomas Petazzoni spake thusly:
> Hello,
> 
> Adding Arnout, Yann and Gustavo in Cc.
> 
> On Fri, 10 Feb 2017 11:34:08 -0200, Carlos Santos wrote:
> > When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
> > this flag requires GCC v4.9 or later. Older versions fail, complaining
> > about the unrecognized command line option.
> > 
> > The problem is not detectable in the autobuilders because they redirect
> > the output to files/pipes and ccache adds the offending flag only when
> > stderr is a terminal.
> > 
> > Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
> 
> This looks good to me in principle, so if Arnout, Yann and Gustavo
> don't complain, I'm going to soon apply this to master.
> 
> Another option would be to adjust this in ccache itself at compile time
> (i.e when ccache gets built) rather than having to pass this additional
> GCC_COLORS variable in the environment when building things. This way
> we would be absolutely sure -fdiagnostics-color will never be passed.
> Hence why I'd like the feedback from other developers.

But there is no config option to disable that in ccache. The behaviour
is hard-coded; we would need a patch to disable that.

> A few nits below (but I can fix while applying).
> 
> > +# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
> > +# this flag requires GCC v4.9 or later. Older versions fail, complaining
> > +# about unrecognized command line option.
> > +ifneq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),y)
> 
> I think we tend to use:
> 
> ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)
> 
> instead.
> 
> > +	TARGET_MAKE_ENV += GCC_COLORS=""
> 
> And we don't indent such lines.

And we might need that for the host as well, no?

    ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
    HOST_MAKE_ENV += GCC_COLORS=""
    endif

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Carlos Santos]

Ensure that the GCC_COLORS environment variable is empty when calling
ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to
GCC but this flag requires GCC v4.9 or later. Older versions complain
about the unrecognized command line option.

That behavior is hard-coded in ccache; we would need a patch to either
make it optional or disable it at all. Notice also that the problem is
not detectable in the autobuilders because they redirect the output to
files/pipes and ccache adds the offending flag only when stderr is a
terminal.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
---
Changes v1->v2
  Fix indentation, use positive logic and improve the commit message,
  acording to comments from Thomas Petazzoni and Yann E. MORIN.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
---
 package/Makefile.in | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/package/Makefile.in b/package/Makefile.in
index 4a3eb26..e5e595a 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
 
 TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)
+TARGET_MAKE_ENV += GCC_COLORS=""
+endif
+
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
 	AR="$(TARGET_AR)" \
@@ -285,6 +292,11 @@ HOST_MAKE_ENV = \
 	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
 	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
 
+# Prevent ccache from passing '-fdiagnostics-color' (see above).
+ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
+HOST_MAKE_ENV += GCC_COLORS=""
+endif
+
 HOST_CONFIGURE_OPTS = \
 	$(HOST_MAKE_ENV) \
 	AR="$(HOSTAR)" \
-- 
2.7.4

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Thomas Petazzoni]

Hello,

On Sun, 12 Feb 2017 11:08:00 +0100, Yann E. MORIN wrote:

> > Another option would be to adjust this in ccache itself at compile time
> > (i.e when ccache gets built) rather than having to pass this additional
> > GCC_COLORS variable in the environment when building things. This way
> > we would be absolutely sure -fdiagnostics-color will never be passed.
> > Hence why I'd like the feedback from other developers.  
> 
> But there is no config option to disable that in ccache. The behaviour
> is hard-coded; we would need a patch to disable that.

We could sed it like we already do in package/ccache/ccache.mk for
other things.

> And we might need that for the host as well, no?
> 
>     ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
>     HOST_MAKE_ENV += GCC_COLORS=""
>     endif

Check Carlos original patch, it was there.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

Thomas, All,

On 2017-02-12 13:14 +0100, Thomas Petazzoni spake thusly:
> On Sun, 12 Feb 2017 11:08:00 +0100, Yann E. MORIN wrote:
> 
> > > Another option would be to adjust this in ccache itself at compile time
> > > (i.e when ccache gets built) rather than having to pass this additional
> > > GCC_COLORS variable in the environment when building things. This way
> > > we would be absolutely sure -fdiagnostics-color will never be passed.
> > > Hence why I'd like the feedback from other developers.  
> > 
> > But there is no config option to disable that in ccache. The behaviour
> > is hard-coded; we would need a patch to disable that.
> 
> We could sed it like we already do in package/ccache/ccache.mk for
> other things.

Something like, maybe:

    diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
    index f174a5d..35d7a7b 100644
    --- a/package/ccache/ccache.mk
    +++ b/package/ccache/ccache.mk
    @@ -31,6 +31,7 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
     define HOST_CCACHE_PATCH_CONFIGURATION
     	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
     	sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
    +	sed -i  '/if (getenv("GCC_COLORS")/,+4d' $(@D)/ccache.c
     endef
     
     HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION

Or would we want it to be conditional on gcc < 4.9? In anycase, I think
overridign the environment variable is much easier...

> > And we might need that for the host as well, no?
> > 
> >     ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
> >     HOST_MAKE_ENV += GCC_COLORS=""
> >     endif
> Check Carlos original patch, it was there.

I didn't look; I only saw your own reply...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

Carlos, All,

On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:
> Ensure that the GCC_COLORS environment variable is empty when calling
> ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to
> GCC but this flag requires GCC v4.9 or later. Older versions complain
> about the unrecognized command line option.
> 
> That behavior is hard-coded in ccache; we would need a patch to either
> make it optional or disable it at all. Notice also that the problem is
> not detectable in the autobuilders because they redirect the output to
> files/pipes and ccache adds the offending flag only when stderr is a
> terminal.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.ind.br>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Should go to master.

Regards,
Yann E. MORIN.

> ---
> Changes v1->v2
>   Fix indentation, use positive logic and improve the commit message,
>   acording to comments from Thomas Petazzoni and Yann E. MORIN.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
> ---
>  package/Makefile.in | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/package/Makefile.in b/package/Makefile.in
> index 4a3eb26..e5e595a 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
>  
>  TARGET_MAKE_ENV = PATH=$(BR_PATH)
>  
> +# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
> +# this flag requires GCC v4.9 or later. Older versions fail, complaining
> +# about unrecognized command line option.
> +ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)
> +TARGET_MAKE_ENV += GCC_COLORS=""
> +endif
> +
>  TARGET_CONFIGURE_OPTS = \
>  	$(TARGET_MAKE_ENV) \
>  	AR="$(TARGET_AR)" \
> @@ -285,6 +292,11 @@ HOST_MAKE_ENV = \
>  	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
>  	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
>  
> +# Prevent ccache from passing '-fdiagnostics-color' (see above).
> +ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
> +HOST_MAKE_ENV += GCC_COLORS=""
> +endif
> +
>  HOST_CONFIGURE_OPTS = \
>  	$(HOST_MAKE_ENV) \
>  	AR="$(HOSTAR)" \
> -- 
> 2.7.4
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Thomas Petazzoni]

Hello,

On Sun, 12 Feb 2017 13:33:26 +0100, Yann E. MORIN wrote:

> Something like, maybe:
> 
>     diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
>     index f174a5d..35d7a7b 100644
>     --- a/package/ccache/ccache.mk
>     +++ b/package/ccache/ccache.mk
>     @@ -31,6 +31,7 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
>      define HOST_CCACHE_PATCH_CONFIGURATION
>      	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
>      	sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
>     +	sed -i  '/if (getenv("GCC_COLORS")/,+4d' $(@D)/ccache.c
>      endef
>      
>      HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION
> 
> Or would we want it to be conditional on gcc < 4.9?

Yes, it should be conditional on gcc < 4.9.

> In anycase, I think overridign the environment variable is much
> easier...

Right, *but* if we forget to pass TARGET_MAKE_ENV or HOST_MAKE_ENV
somewhere, then we will get a build failure. As a trivial example,
package/makedevs/makedevs.mk directly calls $(HOSTCC), which uses
ccache, but does not use $(HOST_MAKE_ENV). So that one would not pass
GCC_COLORS="", and would therefore fail to build on machines with gcc <
4.9.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

Thomas, All,

On 2017-02-12 13:59 +0100, Thomas Petazzoni spake thusly:
> On Sun, 12 Feb 2017 13:33:26 +0100, Yann E. MORIN wrote:
> 
> > Something like, maybe:
> > 
> >     diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
> >     index f174a5d..35d7a7b 100644
> >     --- a/package/ccache/ccache.mk
> >     +++ b/package/ccache/ccache.mk
> >     @@ -31,6 +31,7 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
> >      define HOST_CCACHE_PATCH_CONFIGURATION
> >      	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
> >      	sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
> >     +	sed -i  '/if (getenv("GCC_COLORS")/,+4d' $(@D)/ccache.c
> >      endef
> >      
> >      HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION
> > 
> > Or would we want it to be conditional on gcc < 4.9?
> 
> Yes, it should be conditional on gcc < 4.9.
> 
> > In anycase, I think overridign the environment variable is much
> > easier...
> 
> Right, *but* if we forget to pass TARGET_MAKE_ENV or HOST_MAKE_ENV
> somewhere, then we will get a build failure. As a trivial example,
> package/makedevs/makedevs.mk directly calls $(HOSTCC), which uses
> ccache, but does not use $(HOST_MAKE_ENV). So that one would not pass
> GCC_COLORS="", and would therefore fail to build on machines with gcc <
> 4.9.

But we do a single build of ccache which gets used for both the target
and host compiler. So if we patch ccache, we have to do if any of them
if older than 4.9.

Which is not impossible, by far, but means that an older host gcc will
disable colouring of the target gcc output, and vice-versa.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Thomas Petazzoni]

Hello,

On Sun, 12 Feb 2017 14:08:03 +0100, Yann E. MORIN wrote:

> But we do a single build of ccache which gets used for both the target
> and host compiler. So if we patch ccache, we have to do if any of them
> if older than 4.9.
> 
> Which is not impossible, by far, but means that an older host gcc will
> disable colouring of the target gcc output, and vice-versa.

Correct, but it still seems better than failing completely.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

Carlos, All,

On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:
> Ensure that the GCC_COLORS environment variable is empty when calling
> ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to
> GCC but this flag requires GCC v4.9 or later. Older versions complain
> about the unrecognized command line option.

So, while fixing my own patch about the issue [0], I noticed that I was
not even able to make it fail in the first place...

I'm using this defconfig, which uses a gcc-4.7:

    BR2_x86_i686=y
    BR2_CCACHE=y
    BR2_TOOLCHAIN_EXTERNAL=y

and Thomas reported he was also unable to reproduce the failure with:

    BR2_sh=y
    BR2_sh4a=y
    BR2_CCACHE=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_INIT_NONE=y
    BR2_SYSTEM_BIN_SH_NONE=y
    # BR2_PACKAGE_BUSYBOX is not set
    BR2_PACKAGE_DEVMEM2=y
    # BR2_TARGET_ROOTFS_TAR is not set

which also uses gcc-4.7.

Can you provide a reliable way to reproduce the issue, please?

Regards,
Yann E. MORIN.

> That behavior is hard-coded in ccache; we would need a patch to either
> make it optional or disable it at all. Notice also that the problem is
> not detectable in the autobuilders because they redirect the output to
> files/pipes and ccache adds the offending flag only when stderr is a
> terminal.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
> ---
> Changes v1->v2
>   Fix indentation, use positive logic and improve the commit message,
>   acording to comments from Thomas Petazzoni and Yann E. MORIN.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
> ---
>  package/Makefile.in | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/package/Makefile.in b/package/Makefile.in
> index 4a3eb26..e5e595a 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
>  
>  TARGET_MAKE_ENV = PATH=$(BR_PATH)
>  
> +# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
> +# this flag requires GCC v4.9 or later. Older versions fail, complaining
> +# about unrecognized command line option.
> +ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)
> +TARGET_MAKE_ENV += GCC_COLORS=""
> +endif
> +
>  TARGET_CONFIGURE_OPTS = \
>  	$(TARGET_MAKE_ENV) \
>  	AR="$(TARGET_AR)" \
> @@ -285,6 +292,11 @@ HOST_MAKE_ENV = \
>  	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
>  	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
>  
> +# Prevent ccache from passing '-fdiagnostics-color' (see above).
> +ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
> +HOST_MAKE_ENV += GCC_COLORS=""
> +endif
> +
>  HOST_CONFIGURE_OPTS = \
>  	$(HOST_MAKE_ENV) \
>  	AR="$(HOSTAR)" \
> -- 
> 2.7.4
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Carlos Santos]

> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> To: "Carlos Santos" <casantos@datacom.ind.br>
> Cc: buildroot at buildroot.org, "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>, arnout at mind.be,
> gustavo at zacarias.com.ar
> Sent: Sunday, February 12, 2017 12:14:25 PM
> Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

> Carlos, All,
> 
> On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:
>> Ensure that the GCC_COLORS environment variable is empty when calling
>> ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to
>> GCC but this flag requires GCC v4.9 or later. Older versions complain
>> about the unrecognized command line option.
> 
> So, while fixing my own patch about the issue [0], I noticed that I was
> not even able to make it fail in the first place...
> 
> I'm using this defconfig, which uses a gcc-4.7:
> 
>    BR2_x86_i686=y
>    BR2_CCACHE=y
>    BR2_TOOLCHAIN_EXTERNAL=y
> 
> and Thomas reported he was also unable to reproduce the failure with:
> 
>    BR2_sh=y
>    BR2_sh4a=y
>    BR2_CCACHE=y
>    BR2_TOOLCHAIN_EXTERNAL=y
>    BR2_INIT_NONE=y
>    BR2_SYSTEM_BIN_SH_NONE=y
>    # BR2_PACKAGE_BUSYBOX is not set
>    BR2_PACKAGE_DEVMEM2=y
>    # BR2_TARGET_ROOTFS_TAR is not set
> 
> which also uses gcc-4.7.
> 
> Can you provide a reliable way to reproduce the issue, please?

What does "echo $GCC_COLORS" say? And remember that you must not
redirect the output to a file or pipe.

Carlos Santos (Casantos) - DATACOM, P&D
?Something must be done. This is something. Therefore we must do it.?
(from the BBC series ?Yes, Prime Minister?, ?Power to the people?)

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

Carlos, All,

On 2017-02-12 12:55 -0200, Carlos Santos spake thusly:
> > From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > To: "Carlos Santos" <casantos@datacom.ind.br>
> > Cc: buildroot at buildroot.org, "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>, arnout at mind.be,
> > gustavo at zacarias.com.ar
> > Sent: Sunday, February 12, 2017 12:14:25 PM
> > Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9
> 
> > Carlos, All,
> > 
> > On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:
> >> Ensure that the GCC_COLORS environment variable is empty when calling
> >> ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to
> >> GCC but this flag requires GCC v4.9 or later. Older versions complain
> >> about the unrecognized command line option.
> > 
> > So, while fixing my own patch about the issue [0], I noticed that I was
> > not even able to make it fail in the first place...
> > 
> > I'm using this defconfig, which uses a gcc-4.7:
> > 
> >    BR2_x86_i686=y
> >    BR2_CCACHE=y
> >    BR2_TOOLCHAIN_EXTERNAL=y
> > 
> > and Thomas reported he was also unable to reproduce the failure with:
> > 
> >    BR2_sh=y
> >    BR2_sh4a=y
> >    BR2_CCACHE=y
> >    BR2_TOOLCHAIN_EXTERNAL=y
> >    BR2_INIT_NONE=y
> >    BR2_SYSTEM_BIN_SH_NONE=y
> >    # BR2_PACKAGE_BUSYBOX is not set
> >    BR2_PACKAGE_DEVMEM2=y
> >    # BR2_TARGET_ROOTFS_TAR is not set
> > 
> > which also uses gcc-4.7.
> > 
> > Can you provide a reliable way to reproduce the issue, please?
> 
> What does "echo $GCC_COLORS" say? And remember that you must not
> redirect the output to a file or pipe.

The GCC_COLORS are the default as dopcumented in the man page for gcc-5.

    $ cat defconfig
    BR2_x86_i686=y
    BR2_CCACHE=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_PACKAGE_LIBCEC=y
    $ export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'
    $ make BR2_DEFCONFIG=$(pwd)/defconfig defconfig
    $ make busybox libcec

And the build does not fail... :-/

So I did a few other tentative:

    $ rm -rf ~/.buildroot-ccache/*/
    $ echo log_file = /home/ymorin/meh >> ~/.buildroot-ccache/ccache.conf
    $ touch meh.c
    $ ./host/usr/bin/i686-pc-linux-gnu-gcc -c -o meh.o meh.c
    $ ~/meh
    [2017-02-12T18:19:31.365731 7668 ] === CCACHE 3.3.3 STARTED =========================================
    [2017-02-12T18:19:31.365731 7668 ] Config: (environment) base_dir = /home/ymorin/dev/buildroot/O
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) cache_dir = /home/ymorin/.buildroot-ccache
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) cache_dir_levels = 2
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compiler = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compiler_check = mtime
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compression = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compression_level = 6
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) cpp_extension = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) direct_mode = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) disable = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) extra_files_to_hash = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) hard_link = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) hash_dir = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) ignore_headers_in_manifest = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) keep_comments_cpp = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) limit_multiple = 0.8
    [2017-02-12T18:19:31.365731 7668 ] Config: (/home/ymorin/.buildroot-ccache/ccache.conf) log_file = /home/ymorin/meh
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) max_files = 0
    [2017-02-12T18:19:31.365731 7668 ] Config: (/home/ymorin/.buildroot-ccache/ccache.conf) max_size = 5.0G
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) path = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) prefix_command = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) prefix_command_cpp = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) read_only = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) read_only_direct = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) recache = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) run_second_cpp = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) sloppiness = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) stats = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) temporary_dir = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) umask = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) unify = false
    [2017-02-12T18:19:31.365999 7668 ] Command line: /home/ymorin/dev/buildroot/O/host/usr/bin/ccache /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot /home/ymorin/dev/buildroot/O/host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -c -o meh.o meh.c
    [2017-02-12T18:19:31.366080 7668 ] Hostname: gourin
    [2017-02-12T18:19:31.366157 7668 ] Working directory: /home/ymorin/dev/buildroot/O
    [2017-02-12T18:19:31.366280 7668 ] Automatically enabling colors
    [2017-02-12T18:19:31.366312 7668 ] Source file: meh.c
    [2017-02-12T18:19:31.366331 7668 ] Object file: meh.o
    [2017-02-12T18:19:31.366367 7668 ] Trying direct lookup
    [2017-02-12T18:19:31.366441 7668 ] Looking for object file hash in /home/ymorin/.buildroot-ccache/e/2/095355fd431a7dfa30fcd2c18aa6c7-359.manifest
    [2017-02-12T18:19:31.366468 7668 ] No such manifest file
    [2017-02-12T18:19:31.366486 7668 ] Did not find object file hash in manifest
    [2017-02-12T18:19:31.366695 7668 ] Running preprocessor
    [2017-02-12T18:19:31.366720 7668 ] Executing /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -fdiagnostics-color -E meh.c
    [2017-02-12T18:19:31.368786 7668 ] Preprocessor gave exit status 1
    [2017-02-12T18:19:31.368834 7668 ] Failed; falling back to running the real compiler
    [2017-02-12T18:19:31.368847 7668 ] Executing /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot /home/ymorin/dev/buildroot/O/host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -c -o meh.o meh.c
    [2017-02-12T18:19:31.368943 7668 ] lockfile_acquire: symlink /home/ymorin/.buildroot-ccache/c/stats.lock: No such file or directory
    [2017-02-12T18:19:31.369031 7668 ] Acquired lock /home/ymorin/.buildroot-ccache/c/stats.lock
    [2017-02-12T18:19:31.369147 7668 ] Releasing lock /home/ymorin/.buildroot-ccache/c/stats.lock
    [2017-02-12T18:19:31.369163 7668 ] Unlink /home/ymorin/.buildroot-ccache/c/stats.lock
    [2017-02-12T18:19:31.369189 7668 ] Result: preprocessor error

Hope you can make something out of this.

BTW, I never trusted ccache; it only ever brought me more headaches on
which I spent more time than it ever saved me...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Thomas Petazzoni]

Hello,

On Sun, 12 Feb 2017 12:55:37 -0200 (BRST), Carlos Santos wrote:

> > Can you provide a reliable way to reproduce the issue, please?  
> 
> What does "echo $GCC_COLORS" say?

thomas at skate:~$ echo $GCC_COLORS

thomas at skate:~$

So the issue only arise when GCC_COLORS is defined non-empty in the
user's environment?

> And remember that you must not redirect the output to a file or pipe.

I just ran "make" without redirecting the output to a file or pipe.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

On 2017-02-12 18:22 +0100, Thomas Petazzoni spake thusly:
> Hello,
> 
> On Sun, 12 Feb 2017 12:55:37 -0200 (BRST), Carlos Santos wrote:
> 
> > > Can you provide a reliable way to reproduce the issue, please?  
> > 
> > What does "echo $GCC_COLORS" say?
> 
> thomas at skate:~$ echo $GCC_COLORS
> 
> thomas at skate:~$
> 
> So the issue only arise when GCC_COLORS is defined non-empty in the
> user's environment?

Well, from the point of view of ccache, any value whould have triggered
the issue, because ccache only ever checks if the variables exists in
the environment.

But see my other reply: even with a valid value, I could not trigger the
bug either...

> > And remember that you must not redirect the output to a file or pipe.
> I just ran "make" without redirecting the output to a file or pipe.

Same as I did.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Carlos Santos]

> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> To: "Carlos Santos" <casantos@datacom.ind.br>
> Cc: buildroot at buildroot.org, "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>, arnout at mind.be,
> gustavo at zacarias.com.ar
> Sent: Sunday, February 12, 2017 12:14:25 PM
> Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

> Carlos, All,
[...]
> 
> Can you provide a reliable way to reproduce the issue, please?
> 
> Regards,
> Yann E. MORIN.

Sorry, I forgot so send this yesterday. Here is a reliable reproducer:

$ cat defconfig 
BR2_powerpc=y
BR2_powerpc_8548=y
BR2_CCACHE=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng_e500v2-linux-gnuspe.tar.xz"
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="powerpc-ctng_e500v2-linux-gnuspe"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_PACKAGE_OPENSSL=y
$ 
$ echo $GCC_COLORS 
error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01
$
$ make V=1 BR2_JLEVEL=1 openssl-dirclean openssl
[...]
/work/ccache-colors/host/usr/bin/powerpc-ctng_e500v2-linux-gnuspe-gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DB_ENDIAN -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -mabi=spe -mfloat-gprs=double -Wa,-me500x2 -Os  -Wall -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DVPAES_ASM -c  -o ppccpuid.o ppccpuid.s
powerpc-ctng_e500v2-linux-gnuspe-gcc: error: unrecognized command line option '-fdiagnostics-color'
<builtin>: recipe for target 'ppccpuid.o' failed
make[3]: *** [ppccpuid.o] Error 1
Makefile:287: recipe for target 'build_crypto' failed
make[2]: *** [build_crypto] Error 1
package/pkg-generic.mk:216: recipe for target '/work/ccache-colors/build/openssl-1.0.2k/.stamp_built' failed
make[1]: *** [/work/ccache-colors/build/openssl-1.0.2k/.stamp_built] Error 2
Makefile:16: recipe for target '_all' failed
make: *** [_all] Error 2

Carlos Santos (Casantos) - DATACOM, P&D
?Something must be done. This is something. Therefore we must do it.?
(from the BBC series ?Yes, Prime Minister?, ?Power to the people?)

[Buildroot] [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

[Yann E. MORIN]

Carlos, All,

On 2017-02-13 20:41 -0200, Carlos Santos spake thusly:
> Sorry, I forgot so send this yesterday. Here is a reliable reproducer:
> 
> $ cat defconfig 
> BR2_powerpc=y
> BR2_powerpc_8548=y
> BR2_CCACHE=y
> BR2_TOOLCHAIN_EXTERNAL=y
> BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng_e500v2-linux-gnuspe.tar.xz"
> BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="powerpc-ctng_e500v2-linux-gnuspe"
> BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
> BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
> BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
> BR2_TOOLCHAIN_EXTERNAL_CXX=y
> BR2_PACKAGE_OPENSSL=y
> $ 
> $ echo $GCC_COLORS 
> error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01
> $
> $ make V=1 BR2_JLEVEL=1 openssl-dirclean openssl

Indeed, I could reproduce. Thanks.

So, I think Arnout is right in saying we should just globally unexport
GCC_COLORS and be done with that.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'