[Buildroot] [PATCH 1/3] docs/manual/ccache-support.txt: expand explanation about ccache cache location

[Buildroot] [PATCH 1/3] docs/manual/ccache-support.txt: expand explanation about ccache cache location

[Thomas Petazzoni via buildroot]

The manual explanation seemed to imply that the cache is always in
~/.buildroot-ccache/, but it's just the default value. Clarify this
point.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 docs/manual/ccache-support.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/docs/manual/ccache-support.txt b/docs/manual/ccache-support.txt
index f6746ad7d8..ea64628615 100644
--- a/docs/manual/ccache-support.txt
+++ b/docs/manual/ccache-support.txt
@@ -15,10 +15,12 @@ speed up the build process.
 +Enable compiler cache+ in +Build options+. This will automatically
 build +ccache+ and use it for every host and target compilation.
 
-The cache is located in +$HOME/.buildroot-ccache+. It is stored
-outside of Buildroot output directory so that it can be shared by
-separate Buildroot builds. If you want to get rid of the cache, simply
-remove this directory.
+The cache is located in the directory defined by the +BR2_CCACHE_DIR+
+configuration option, which defaults to
++$HOME/.buildroot-ccache+. This default location is outside of
+Buildroot output directory so that it can be shared by separate
+Buildroot builds. If you want to get rid of the cache, simply remove
+this directory.
 
 You can get statistics on the cache (its size, number of hits,
 misses, etc.) by running +make ccache-stats+.
-- 
2.37.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/3] docs/manual/ccache-support.txt: document the new BR2_USE_CCACHE variable

[Thomas Petazzoni via buildroot]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 docs/manual/ccache-support.txt | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/docs/manual/ccache-support.txt b/docs/manual/ccache-support.txt
index ea64628615..ab4f10be23 100644
--- a/docs/manual/ccache-support.txt
+++ b/docs/manual/ccache-support.txt
@@ -55,3 +55,15 @@ first.
 See https://ccache.samba.org/manual.html#_compiling_in_different_directories[the
 ccache manual's section on "Compiling in different directories"] for
 more details about this rewriting of absolute paths.
+
+When +ccache+ is enabled in Buildroot using the +BR2_CCACHE=y+ option:
+
+* +ccache+ is used during the Buildroot build itself
+
+* +ccache+ is not used when building outside of Buildroot, for example
+  when directly calling the cross-compiler or using the SDK
+
+One can override this behavior using the +BR2_USE_CCACHE+ environment
+variable: when set to +1+, usage of ccache is enabled (default during
+the Buildroot build), when unset or set to a value different from +1+,
+usage of ccache is disabled.
-- 
2.37.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 3/3] toolchain/toolchain-wrapper: don't hardcode CCACHE_BASEDIR

[Thomas Petazzoni via buildroot]

The CCACHE_BASEDIR variable was hardcoded to $(BASE_DIR), and
therefore when moving around the toolchain-wrapper as part of the SDK,
the CCACHE_BASEDIR would still point to the original location of the
Buildroot build.

Instead of doing this, leverage the logic we already have in the
toolchain wrapper, which determines based on the toolchain wrapper
location itself what is the base directory.

So now, toolchain-wrapper.mk is only passing a boolean to
toolchain-wrapper.c, that tells whether the BR2_USE_CCACHE_BASEDIR
option is enabled or not.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 toolchain/toolchain-wrapper.c  | 6 +++---
 toolchain/toolchain-wrapper.mk | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index 37b24dd24a..bed90d6292 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -538,9 +538,9 @@ int main(int argc, char **argv)
 		return 3;
 	}
 #endif
-#ifdef BR_CCACHE_BASEDIR
-	/* Allow compilercheck to be overridden through the environment */
-	if (setenv("CCACHE_BASEDIR", BR_CCACHE_BASEDIR, 0)) {
+#ifdef BR_USE_CCACHE_BASEDIR
+	/* Allow basedir to be overridden through the environment */
+	if (setenv("CCACHE_BASEDIR", absbasedir, 0)) {
 		perror(__FILE__ ": Failed to set CCACHE_BASEDIR");
 		return 3;
 	}
diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
index cbf46f15fa..c87120f0f7 100644
--- a/toolchain/toolchain-wrapper.mk
+++ b/toolchain/toolchain-wrapper.mk
@@ -70,7 +70,7 @@ endif
 endif
 
 ifeq ($(BR2_CCACHE_USE_BASEDIR),y)
-TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BASEDIR='"$(BASE_DIR)"'
+TOOLCHAIN_WRAPPER_ARGS += -DBR_USE_CCACHE_BASEDIR
 endif
 
 ifeq ($(BR2_PIC_PIE),y)
-- 
2.37.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/3] docs/manual/ccache-support.txt: expand explanation about ccache cache location

[Yann E. MORIN]

Thomas, All,

On 2022-07-27 20:48 +0200, Thomas Petazzoni via buildroot spake thusly:
> The manual explanation seemed to imply that the cache is always in
> ~/.buildroot-ccache/, but it's just the default value. Clarify this
> point.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  docs/manual/ccache-support.txt | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/docs/manual/ccache-support.txt b/docs/manual/ccache-support.txt
> index f6746ad7d8..ea64628615 100644
> --- a/docs/manual/ccache-support.txt
> +++ b/docs/manual/ccache-support.txt
> @@ -15,10 +15,12 @@ speed up the build process.
>  +Enable compiler cache+ in +Build options+. This will automatically
>  build +ccache+ and use it for every host and target compilation.
>  
> -The cache is located in +$HOME/.buildroot-ccache+. It is stored
> -outside of Buildroot output directory so that it can be shared by
> -separate Buildroot builds. If you want to get rid of the cache, simply
> -remove this directory.
> +The cache is located in the directory defined by the +BR2_CCACHE_DIR+
> +configuration option, which defaults to
> ++$HOME/.buildroot-ccache+. This default location is outside of
> +Buildroot output directory so that it can be shared by separate
> +Buildroot builds. If you want to get rid of the cache, simply remove
> +this directory.
>  
>  You can get statistics on the cache (its size, number of hits,
>  misses, etc.) by running +make ccache-stats+.
> -- 
> 2.37.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/3] docs/manual/ccache-support.txt: document the new BR2_USE_CCACHE variable

[Yann E. MORIN]

Thomas, All,

On 2022-07-27 20:48 +0200, Thomas Petazzoni via buildroot spake thusly:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  docs/manual/ccache-support.txt | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/docs/manual/ccache-support.txt b/docs/manual/ccache-support.txt
> index ea64628615..ab4f10be23 100644
> --- a/docs/manual/ccache-support.txt
> +++ b/docs/manual/ccache-support.txt
> @@ -55,3 +55,15 @@ first.
>  See https://ccache.samba.org/manual.html#_compiling_in_different_directories[the
>  ccache manual's section on "Compiling in different directories"] for
>  more details about this rewriting of absolute paths.
> +
> +When +ccache+ is enabled in Buildroot using the +BR2_CCACHE=y+ option:
> +
> +* +ccache+ is used during the Buildroot build itself
> +
> +* +ccache+ is not used when building outside of Buildroot, for example
> +  when directly calling the cross-compiler or using the SDK
> +
> +One can override this behavior using the +BR2_USE_CCACHE+ environment
> +variable: when set to +1+, usage of ccache is enabled (default during
> +the Buildroot build), when unset or set to a value different from +1+,
> +usage of ccache is disabled.
> -- 
> 2.37.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/3] toolchain/toolchain-wrapper: don't hardcode CCACHE_BASEDIR

[Yann E. MORIN]

Thomas, All,

On 2022-07-27 20:48 +0200, Thomas Petazzoni via buildroot spake thusly:
> The CCACHE_BASEDIR variable was hardcoded to $(BASE_DIR), and
> therefore when moving around the toolchain-wrapper as part of the SDK,
> the CCACHE_BASEDIR would still point to the original location of the
> Buildroot build.
> 
> Instead of doing this, leverage the logic we already have in the
> toolchain wrapper, which determines based on the toolchain wrapper
> location itself what is the base directory.
> 
> So now, toolchain-wrapper.mk is only passing a boolean to
> toolchain-wrapper.c, that tells whether the BR2_USE_CCACHE_BASEDIR
> option is enabled or not.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  toolchain/toolchain-wrapper.c  | 6 +++---
>  toolchain/toolchain-wrapper.mk | 2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
> index 37b24dd24a..bed90d6292 100644
> --- a/toolchain/toolchain-wrapper.c
> +++ b/toolchain/toolchain-wrapper.c
> @@ -538,9 +538,9 @@ int main(int argc, char **argv)
>  		return 3;
>  	}
>  #endif
> -#ifdef BR_CCACHE_BASEDIR
> -	/* Allow compilercheck to be overridden through the environment */
> -	if (setenv("CCACHE_BASEDIR", BR_CCACHE_BASEDIR, 0)) {
> +#ifdef BR_USE_CCACHE_BASEDIR
> +	/* Allow basedir to be overridden through the environment */
> +	if (setenv("CCACHE_BASEDIR", absbasedir, 0)) {

You forgot the one instance in the debug code, lines 524 and 525.

And once this is fixed, you'll notice that the old and the new
CCACHE_BASEDIR will be different:

    (before)$ BR2_DEBUG_WRAPPER=2 ./host/bin/arm-linux-gcc --help
    ...
    Toolchain wrapper executing:
        CCACHE_BASEDIR='/home/ymorin/dev/buildroot/O/master'
        '/home/ymorin/dev/buildroot/O/master/host/opt/ext-toolchain/bin/arm-linux-gcc.br_real'
        ...

    (after)$ BR2_DEBUG_WRAPPER=2 ./host/bin/arm-linux-gcc --help
    ...
    Toolchain wrapper executing:
        CCACHE_BASEDIR='/home/ymorin/dev/buildroot/O/master/host'
        '/home/ymorin/dev/buildroot/O/master/host/opt/ext-toolchain/bin/arm-linux-gcc.br_real'
        ...

So as you can see, before, CCACHE_BASEDIR was pointing to BASE_DIR, but
now it points to HOST_DIR. This is one-level too deep, as it will not
match the path where packages are actually built, which is
BASE_DIR/build, and hapenned to be CCACHE_BASEDIR/build.

Even though I had the debug code path fixed, the second issue was not
trivial enough for me to fix when applying, so I amrked the patch as
changes requested.

Regards,
Yann E. MORIN.

>  		perror(__FILE__ ": Failed to set CCACHE_BASEDIR");
>  		return 3;
>  	}
> diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
> index cbf46f15fa..c87120f0f7 100644
> --- a/toolchain/toolchain-wrapper.mk
> +++ b/toolchain/toolchain-wrapper.mk
> @@ -70,7 +70,7 @@ endif
>  endif
>  
>  ifeq ($(BR2_CCACHE_USE_BASEDIR),y)
> -TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BASEDIR='"$(BASE_DIR)"'
> +TOOLCHAIN_WRAPPER_ARGS += -DBR_USE_CCACHE_BASEDIR
>  endif
>  
>  ifeq ($(BR2_PIC_PIE),y)
> -- 
> 2.37.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/3] toolchain/toolchain-wrapper: don't hardcode CCACHE_BASEDIR

[Thomas Petazzoni via buildroot]

Howdy,

On Sat, 30 Jul 2022 14:11:03 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> And once this is fixed, you'll notice that the old and the new
> CCACHE_BASEDIR will be different:
> 
>     (before)$ BR2_DEBUG_WRAPPER=2 ./host/bin/arm-linux-gcc --help
>     ...
>     Toolchain wrapper executing:
>         CCACHE_BASEDIR='/home/ymorin/dev/buildroot/O/master'
>         '/home/ymorin/dev/buildroot/O/master/host/opt/ext-toolchain/bin/arm-linux-gcc.br_real'
>         ...
> 
>     (after)$ BR2_DEBUG_WRAPPER=2 ./host/bin/arm-linux-gcc --help
>     ...
>     Toolchain wrapper executing:
>         CCACHE_BASEDIR='/home/ymorin/dev/buildroot/O/master/host'
>         '/home/ymorin/dev/buildroot/O/master/host/opt/ext-toolchain/bin/arm-linux-gcc.br_real'
>         ...
> 
> So as you can see, before, CCACHE_BASEDIR was pointing to BASE_DIR, but
> now it points to HOST_DIR. This is one-level too deep, as it will not
> match the path where packages are actually built, which is
> BASE_DIR/build, and hapenned to be CCACHE_BASEDIR/build.
> 
> Even though I had the debug code path fixed, the second issue was not
> trivial enough for me to fix when applying, so I amrked the patch as
> changes requested.

You're totally right. And in fact, it makes the change useless. Indeed,
CCACHE_BASEDIR only makes sense when ccache is used during the
Buildroot build, because it points to $(BASE_DIR)... which once we are
outside of Buildroot, in the SDK, would no longer point to anything
sensible.

So instead, I've sent a different patch that sets the
CCACHE_COMPILERCHECK and CCACHE_BASEDIR environment variables, only if
ccache support is used (BR2_USE_CCACHE=1). Let me know what you think.

Thanks again for the good review!

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot