From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.windriver.com (mail.windriver.com [147.11.1.11]) by mail.openembedded.org (Postfix) with ESMTP id 3F0797A076 for ; Tue, 15 Jan 2019 02:06:27 +0000 (UTC) Received: from ALA-HCB.corp.ad.wrs.com ([147.11.189.41]) by mail.windriver.com (8.15.2/8.15.1) with ESMTPS id x0F26Rcm023604 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL) for ; Mon, 14 Jan 2019 18:06:27 -0800 (PST) Received: from localhost.corp.ad.wrs.com (128.224.163.177) by ALA-HCB.corp.ad.wrs.com (147.11.189.41) with Microsoft SMTP Server id 14.3.408.0; Mon, 14 Jan 2019 18:06:20 -0800 To: Randy MacLeod , References: <6777b5a2a2442988d4497678639c018f638ad86b.1545102457.git.liezhi.yang@windriver.com> From: Robert Yang Message-ID: <9e1430b7-e754-a6b3-70e5-848d4c4c250e@windriver.com> Date: Tue, 15 Jan 2019 10:11:05 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Subject: Re: [PATCH V3 03/11] ccache.bbclass: Refactor it to make it more reliable X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jan 2019 02:06:27 -0000 Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Hi Randy, I'm sorry for the types, and thanks for the correction. I found that it has been merged, I will be more careful. // Robert On 1/12/19 4:49 AM, Randy MacLeod wrote: > A couple of typos and grammar improvements, otherwise a good > improvement for iterative development of a package/image where > sstate isn't useful. > > On 12/18/18 2:18 AM, Robert Yang wrote: >> The previous ccache.bbclass has the following problems: >> - It uses host's ccache for native recipes, but this may not work on some >>    hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always >>    build failures (m4-native failed at do_configure, and others will also be >>    failed if I disable CCACHE for m4-native) >> >> - native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses > * host's ccache >>    ccache-native, this may confuse user. >> >> - The target recipes may use both host's ccache and ccache-native, this may >>    cause unexpected problems and hard to debug. This is because ccache-native is > * and be hard to debug >>    in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when >>    rebuild target recipes, and then it would use hosttools/ccache, but the >>    previous ccache files were generated by ccache-native. >> >> - Target recipes can't use ccache when no ccahe is installed on the host: > * no ccache > >>    CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}" >> >> After refactored: >> All types recipes (native, target and others) will use ccache-native except >> ccache-native itself, host's cache won't be used any more. It is more >> reliable now, which will work everywhere when ccache-native can be built. >> >> And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe > * disable ccache > > ../Randy > > >> rather than "CCACHE = ''" since we set CCACHE in anonymous function, and >> d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we >> can't check whether CCACHE is set or not in anonymous function since it is >> always set. Use CCACHE_DISABLE to disable it would be more clear. >> >> Signed-off-by: Robert Yang >> --- >>   meta/classes/ccache.bbclass | 25 ++++++++++++++++++++++--- >>   meta/conf/bitbake.conf      |  6 +----- >>   meta/conf/layer.conf        |  1 - >>   meta/lib/oe/utils.py        |  3 +++ >>   4 files changed, 26 insertions(+), 9 deletions(-) >> >> diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass >> index 9609020..59e1022 100644 >> --- a/meta/classes/ccache.bbclass >> +++ b/meta/classes/ccache.bbclass >> @@ -1,4 +1,14 @@ >> -CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}" >> +# >> +# Usage: >> +# - Enable ccache >> +#   Add the following line to a conffile such as conf/local.conf: >> +#   INHERIT += "ccache" >> +# >> +# - Disable ccache for a recipe >> +#   Add the following line to the recipe if it can't be built with ccache: >> +#   CCACHE_DISABLE = '1' >> +# >> + >>   export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}" >>   # We need to stop ccache considering the current directory or the >> @@ -7,5 +17,14 @@ export CCACHE_DIR ?= >> "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}" >>   # ${PV} or ${PR} change. >>   export CCACHE_NOHASHDIR ?= "1" >> -DEPENDS_append_class-target = " ccache-native" >> -DEPENDS[vardepvalueexclude] = " ccache-native" >> +python() { >> +    """ >> +    Enable ccache for the recipe >> +    """ >> +    pn = d.getVar('PN') >> +    # quilt-native doesn't need ccache since no c files >> +    if not (pn in ('ccache-native', 'quilt-native') or >> +            bb.utils.to_boolean(d.getVar('CCACHE_DISABLE'))): >> +        d.appendVar('DEPENDS', ' ccache-native') >> +        d.setVar('CCACHE', 'ccache ') >> +} >> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf >> index 6480062..c88d66c 100644 >> --- a/meta/conf/bitbake.conf >> +++ b/meta/conf/bitbake.conf >> @@ -492,7 +492,7 @@ HOSTTOOLS += " \ >>   HOSTTOOLS += "${@'ip ping ps scp ssh stty' if >> (bb.utils.contains_any('IMAGE_CLASSES', 'testimage testsdk', True, False, d) >> or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testimage.bbclass", >> "testsdk.bbclass"])) else ''}" >>   # Link to these if present >> -HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc pigz sftp >> socat ssh sudo" >> +HOSTTOOLS_NONFATAL += "aws gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh >> sudo" >>   # Temporary add few more detected in bitbake world >>   HOSTTOOLS_NONFATAL += "join nl size yes zcat" >> @@ -504,10 +504,6 @@ HOSTTOOLS_NONFATAL += "bzr" >>   HOSTTOOLS_NONFATAL += "scp" >>   CCACHE ??= "" >> -# ccache < 3.1.10 will create CCACHE_DIR on startup even if disabled, and >> -# autogen sets HOME=/dev/null so in certain situations builds can fail. >> -# Explicitly export CCACHE_DIR until we can assume ccache >3.1.10 on the host. >> -export CCACHE_DIR ??= "${@os.getenv('HOME')}/.ccache" >>   TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}" >> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf >> index c67dec4..4141d6c 100644 >> --- a/meta/conf/layer.conf >> +++ b/meta/conf/layer.conf >> @@ -49,7 +49,6 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \ >>     *->quilt-native \ >>     *->subversion-native \ >>     *->git-native \ >> -  *->ccache-native \ >>     *->icecc-create-env-native \ >>     gcc-cross-${TARGET_ARCH}->virtual/${TARGET_PREFIX}libc-for-gcc \ >>     gcc-cross-${TARGET_ARCH}->linux-libc-headers \ >> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py >> index 29b4115..ee6f0e6 100644 >> --- a/meta/lib/oe/utils.py >> +++ b/meta/lib/oe/utils.py >> @@ -363,6 +363,9 @@ def host_gcc_version(d, taskcontextonly=False): >>           return >>       compiler = d.getVar("BUILD_CC") >> +    # Get rid of ccache since it is not present when parsing. >> +    if compiler.startswith('ccache '): >> +        compiler = compiler[7:] >>       try: >>           env = os.environ.copy() >>           env["PATH"] = d.getVar("PATH") >> > >