[Buildroot] Buildroot does not use ccache for compiling Linux kernel?

[Buildroot] Buildroot does not use ccache for compiling Linux kernel?

[Andreas Düring via buildroot]

Hello,

I am currently experimenting with building a minimal image for my
Raspberry Pi.

During this process, I realized that ccache, although enabled, is not
used for compiling the Linux kernel. I think this is because of the
recipe in linux/linux.mk:

LINUX_MAKE_FLAGS = \
        HOSTCC="$(HOSTCC) $(subst -I/,-isystem /,$(subst -I /,-isystem /,$(HOST_CFLAGS))) $(HOST_LDFLAGS)" \
        ARCH=$(KERNEL_ARCH) \
        INSTALL_MOD_PATH=$(TARGET_DIR) \
        CROSS_COMPILE="$(TARGET_CROSS)" \
        WERROR=0 \
        REGENERATE_PARSERS=1 \
        DEPMOD=$(HOST_DIR)/sbin/depmod


The linux kernel seems to assemble the compiler using TARGET_CROSS,
e.g.
CC = $(TARGET_CROSS)gcc

As a very crude solution, I changed the corresponding line to
        CROSS_COMPILE="ccache $(TARGET_CROSS)"
but this is probably not the optimal solution. (I didn't carefully
check where else CROSS_COMPILE is used).

If this is a known limitation? Should this be added to the
documentation?
https://buildroot.org/downloads/manual/manual.html#ccache
("This will automatically build ccache and use it for every host and
target compilation.")

With best regards,
Andreas
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Buildroot does not use ccache for compiling Linux kernel?

[Thomas Petazzoni via buildroot]

Hello Andreas,

On Sat, 14 Sep 2024 20:19:37 +0200
Andreas Düring via buildroot <buildroot@buildroot.org> wrote:

> During this process, I realized that ccache, although enabled, is not
> used for compiling the Linux kernel. I think this is because of the
> recipe in linux/linux.mk:
> 
> LINUX_MAKE_FLAGS = \
>         HOSTCC="$(HOSTCC) $(subst -I/,-isystem /,$(subst -I /,-isystem /,$(HOST_CFLAGS))) $(HOST_LDFLAGS)" \
>         ARCH=$(KERNEL_ARCH) \
>         INSTALL_MOD_PATH=$(TARGET_DIR) \
>         CROSS_COMPILE="$(TARGET_CROSS)" \
>         WERROR=0 \
>         REGENERATE_PARSERS=1 \
>         DEPMOD=$(HOST_DIR)/sbin/depmod
> 
> The linux kernel seems to assemble the compiler using TARGET_CROSS,
> e.g.
> CC = $(TARGET_CROSS)gcc
> 
> As a very crude solution, I changed the corresponding line to
>         CROSS_COMPILE="ccache $(TARGET_CROSS)"
> but this is probably not the optimal solution. (I didn't carefully
> check where else CROSS_COMPILE is used).
> 
> If this is a known limitation? Should this be added to the
> documentation?

The thing you're missing is that for target builds, calling ccache is
hidden inside our toolchain wrapper.

When we call $(HOST_DIR)/bin/arm-linux-gcc, it doesn't call the
compiler. It calls our toolchain wrapper
(toolchain/toolchain-wrapper.c), which itself calls the compiler. And
as part of that, it does invoke ccache.

So we are using ccache for all target code, as it's directly done by
our compiler wrapper.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Buildroot does not use ccache for compiling Linux kernel?

[Andreas Düring via buildroot]

Hello Thomas,

> The thing you're missing is that for target builds, calling ccache is
> hidden inside our toolchain wrapper.

Ohh. That would explain why I did not see any change in the ccache
statictics. I only enabled ccache *after* the initial build. (I know,
not a good idea, the docs explicitely state that for changes affecting
the build, a clean build is recommended). Of course the
toolchain-wrapper is still the "old" one without ccache.
(My assumption was that this is handled via the recipes.)

Thanks for the clarification.
Best regards,
Andreas

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Buildroot does not use ccache for compiling Linux kernel?

[Arnout Vandecappelle via buildroot]

On 14/09/2024 20:19, Andreas Düring via buildroot wrote:
> Hello,
> 
> I am currently experimenting with building a minimal image for my
> Raspberry Pi.
> 
> During this process, I realized that ccache, although enabled, is not
> used for compiling the Linux kernel.

  That is very weird, because ccache is added by the toolchain wrapper. It can 
only be disabled by setting BR2_USE_CCACHE=0 in the environment (or on the make 
command line).

  How did you determine that it doesn't use ccache?

> I think this is because of the
> recipe in linux/linux.mk:
> 
> LINUX_MAKE_FLAGS = \
>          HOSTCC="$(HOSTCC) $(subst -I/,-isystem /,$(subst -I /,-isystem /,$(HOST_CFLAGS))) $(HOST_LDFLAGS)" \

  We explicitly don't use ccache here (HOSTCC doesn't use the wrapper, so it 
needs to be added explicitly there). See commit 71a31b23 for an explanation why.

>          ARCH=$(KERNEL_ARCH) \
>          INSTALL_MOD_PATH=$(TARGET_DIR) \
>          CROSS_COMPILE="$(TARGET_CROSS)" \
>          WERROR=0 \
>          REGENERATE_PARSERS=1 \
>          DEPMOD=$(HOST_DIR)/sbin/depmod
> 
> 
> The linux kernel seems to assemble the compiler using TARGET_CROSS,
> e.g.
> CC = $(TARGET_CROSS)gcc
> 
> As a very crude solution, I changed the corresponding line to
>          CROSS_COMPILE="ccache $(TARGET_CROSS)"

  This is what we had originally, but it was removed in commit 546a69db66ab 
because it is now done by the wrapper.

  Regards,
  Arnout

> but this is probably not the optimal solution. (I didn't carefully
> check where else CROSS_COMPILE is used).
> 
> If this is a known limitation? Should this be added to the
> documentation?
> https://buildroot.org/downloads/manual/manual.html#ccache
> ("This will automatically build ccache and use it for every host and
> target compilation.")
> 
> With best regards,
> Andreas
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot