[Buildroot] [PATCH v2] Enable ccache for cmake packages

[Luca Ceresoli <luca@lucaceresoli.net>]

Samuel Martin wrote:
> [...]
>>>>                  -DCMAKE_VERBOSE_MAKEFILE=ON \
>>> Ahem... I think this belongs to some debug remains ;)
>> But it hasn't been added by this patch from Luca.
> Not this patch, but for sure, it's not on master ;)

Sorry, my mistake! :)

[...]
>>>> +               set(CMAKE_C_COMPILER $(CCACHE))\n\
>>>> +               set(CMAKE_CXX_COMPILER $(CCACHE))\n\
>>>> +               set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE) CACHE STRING \"\" FORCE)\n\
>>>> +               set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE) CACHE STRING \"\" FORCE)\n\
>>>> +       endif(DEFINED USE_CCACHE AND USE_CCACHE)\n\
>>>> +       if(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\
>>> You can replace the 2 above lines by:
>>> else ()
>> I don't think so. See the explanation in Luca's commit log. He needs to
>> differentiate three cases:
>>
>>   * USE_CCACHE is defined and set to ON
>>   * USE_CCACHE is defined and set to OFF
>>   * USE_CCACHE is not defined
> Well, I didn't really pay enough/that much attention to the tristate
> thing because I didn't really get the point...

If you add at the beginning of toolchainfile.cmake the line:
message("USE_CCACHE=${USE_CCACHE}")
and then run `make rpi-userland{-dirclean,}` you'll see:

> >>> rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Configuring
> (cd 
> /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ 
> && rm -f CMakeCache.txt && 
> /home/murray/devel/buildroot/output/host/usr/bin/cmake 
> /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ 
> -DCMAKE_TOOLCHAIN_FILE="/home/murray/devel/buildroot/output/host/usr/share/buildroot/toolchainfile.cmake" 
> -DCMAKE_INSTALL_PREFIX="/usr" -DUSE_CCACHE=ON -DVMCS_INSTALL_PREFIX=/usr )
> Re-run cmake no build system arguments
> USE_CCACHE=ON
> USE_CCACHE=ON
> -- The C compiler identification is GNU 4.7.3
> -- The CXX compiler identification is GNU 4.7.3
> -- Check for working C compiler: 
> /home/murray/devel/buildroot/output/host/usr/bin/ccache
> USE_CCACHE=
> -- Check for working C compiler: 
> /home/murray/devel/buildroot/output/host/usr/bin/ccache -- works
> -- Detecting C compiler ABI info
> USE_CCACHE=
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: 
> /home/murray/devel/buildroot/output/host/usr/bin/ccache
> USE_CCACHE=
> -- Check for working CXX compiler: 
> /home/murray/devel/buildroot/output/host/usr/bin/ccache -- works
> -- Detecting CXX compiler ABI info
> USE_CCACHE=
> -- Detecting CXX compiler ABI info - done
> USE_CCACHE=
> -- Looking for execinfo.h
> USE_CCACHE=
> -- Looking for execinfo.h - found
> -- The ASM compiler identification is GNU
> -- Found assembler: 
> /home/murray/devel/buildroot/output/host/usr/bin/arm-linux-gnueabihf-gcc

Apparently CMake interprets multiple times the toolchainfile.cmake,
and it does not always pass the original parameters, so USE_CCACHE is
sometimes undefined. Hence my idea to handle that variable as a
tristate.

> Now, I got it, but I'm really dubious about behavior in the last case.
>
> I mean: the goal of this toolchainfile.cmake is, among other thing, to
> be able to reuse the Buildroot toolchain
> to build a projects outside Buildroot because CMake is good at this.
>
> So, in that case, we want to be sure we correctly export the toolchain
> properties.
>
> If we don't set any compiler in the case USE_CCACHE is undefined, then
> we totally miss this goal,
> and IMHO, it does not make sense because the toolchainfile.cmake file
> is generated by Buildroot,
> for the Buildroot toolchain and nothing else.

I actually overlookedthe need to make the whole thing usable from
outside Buildroot. It should be made working indeed.

> Maybe the logic is a bit more complex... something like that:
>
> set(FIND_PROGRAM_PATH "$(HOST_DIR)/usr/bin")
> find_program(CCACHE "ccache")
> if (DEFINED USE_CCACHE)
>    if (USE_CCACHE AND CCACHE)
>      # set the compiler variables with ccache support
>    else ()
>      # set the compiler variables without ccache support
>    endif ()
> else ()
>    message("To enable ccache usage, add -DUSE_CACHE=ON on the command line")
>    # fallback: set only the compiler variables (with no ccache support)
> endif ()

I don't like very much automagic things, such as using a tool
transparently when it'sfound. We have a BR2_CCACHE config knob, that
should rule IMO.

Still, yours is an interesting proposal. The only alternative is
probably to put the ccachelogic in the wrappers as Thomas proposed in
reply to my v1 patch.

But I see a problem here: Buildroot exports BUILDROOT_CACHE_DIR, and
$(HOST_DIR)/usr/bin/ccacheuses that environment variable to locate its
cache directory. If the same ccache executable is executed standalone
it does won't find that environment variable and default to ~/.ccache.

Anyway, thinking more about how to implement this feature, I think
maybe Thomas' proposal can avoid the mentioned issues, as the call to
ccache (as well as the ccache directory path) can be coded inside a
(C?) wrapper if the wrapper has been built with BR2_CCACHE=y.
But we would need to regenerate the wrapper whenever the user changes
the BR2_CCACHE...

Also, the BR-built ccache would be used transparently even when
calling ${HOST_DIR}/usr/bin/*-gcc. This shows clearly an advantage,
although the user might not understand well that he's using ccache
under the hood (maybe not a problem).

But I'd like to understand better your idea, Thomas. The ccache
usage is independent from the toolchain type (Buildroot/ct-NG/
external).  Would you implement a unique wrapper to be used for
all three toolchain types, and replace the current
ext-toolchain-wrapper.c?

Luca