What is TOOLCHAIN_NEED_CONFIGSITE_CACHE for?

What is TOOLCHAIN_NEED_CONFIGSITE_CACHE for?

[Phil Blundell]

This morning I happened to notice that some/all of my images had a
dependency on ncurses for no obviously good reason.  (Following an
earlier small crusade these are now the only remaining dependencies on
ncurses in my build, so I would quite like to get rid of them.)

It turns out that the dependency in question is due to the code in
toolchain-scripts.bbclass (which is inherited from
populate_sdk_base.bbclass, from image.bbclass) which does:

TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC} ncurses"

[...]

python __anonymous () {
    deps = ""
    for dep in (d.getVar('TOOLCHAIN_NEED_CONFIGSITE_CACHE', True) or "").split():
        deps += " %s:do_populate_sysroot" % dep
    d.appendVarFlag('do_configure', 'depends', deps)
}

The code in question was added in:

commit 9e87f1347788beed181476dc4563085db14a4729
Author: Lianhao Lu <lianhao.lu@intel.com>
Date:   Tue Aug 16 11:15:36 2011 +0800

    toolchain-script.bbclass: Collected cached site config in runtime.
    
    [YOCTO #892]
    Modify the function toolchain_create_sdk_siteconfig to collect the
    cached site config files which are specified by
    TOOLCHAIN_NEED_CONFIGSITE_CACHE in runtime.
    
    Also added task dependency to ensure the cached site config files are
    generated.
    
    Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>

... but I don't entirely understand from that commentary what exactly is
special about ncurses that means it needs to be handled this way.  Can
anybody clarify?

thanks

p.

Re: What is TOOLCHAIN_NEED_CONFIGSITE_CACHE for?

[Richard Purdie]

On Mon, 2012-10-29 at 13:33 +0000, Phil Blundell wrote:
> This morning I happened to notice that some/all of my images had a
> dependency on ncurses for no obviously good reason.  (Following an
> earlier small crusade these are now the only remaining dependencies on
> ncurses in my build, so I would quite like to get rid of them.)
> 
> It turns out that the dependency in question is due to the code in
> toolchain-scripts.bbclass (which is inherited from
> populate_sdk_base.bbclass, from image.bbclass) which does:
> 
> TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC} ncurses"
> 
> [...]
> 
> python __anonymous () {
>     deps = ""
>     for dep in (d.getVar('TOOLCHAIN_NEED_CONFIGSITE_CACHE', True) or "").split():
>         deps += " %s:do_populate_sysroot" % dep
>     d.appendVarFlag('do_configure', 'depends', deps)
> }
> 
> The code in question was added in:
> 
> commit 9e87f1347788beed181476dc4563085db14a4729
> Author: Lianhao Lu <lianhao.lu@intel.com>
> Date:   Tue Aug 16 11:15:36 2011 +0800
> 
>     toolchain-script.bbclass: Collected cached site config in runtime.
>     
>     [YOCTO #892]
>     Modify the function toolchain_create_sdk_siteconfig to collect the
>     cached site config files which are specified by
>     TOOLCHAIN_NEED_CONFIGSITE_CACHE in runtime.
>     
>     Also added task dependency to ensure the cached site config files are
>     generated.
>     
>     Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
> 
> ... but I don't entirely understand from that commentary what exactly is
> special about ncurses that means it needs to be handled this way.  Can
> anybody clarify?

We ship the "site" cache files with the toolchain. We decided to include
the generated cache files as well as the static ones. We currently
generate "site" files for libc and ncurses.

You can almost certainly just set:

TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC}"

and be happy since this just lists which generated cache site files to
include.

Cheers,

Richard

Re: What is TOOLCHAIN_NEED_CONFIGSITE_CACHE for?

[Chris Larson]

On Mon, Oct 29, 2012 at 8:43 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Mon, 2012-10-29 at 13:33 +0000, Phil Blundell wrote:
>> This morning I happened to notice that some/all of my images had a
>> dependency on ncurses for no obviously good reason.  (Following an
>> earlier small crusade these are now the only remaining dependencies on
>> ncurses in my build, so I would quite like to get rid of them.)
>>
>> It turns out that the dependency in question is due to the code in
>> toolchain-scripts.bbclass (which is inherited from
>> populate_sdk_base.bbclass, from image.bbclass) which does:
>>
>> TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC} ncurses"
>>
>> [...]
>>
>> python __anonymous () {
>>     deps = ""
>>     for dep in (d.getVar('TOOLCHAIN_NEED_CONFIGSITE_CACHE', True) or "").split():
>>         deps += " %s:do_populate_sysroot" % dep
>>     d.appendVarFlag('do_configure', 'depends', deps)
>> }
>>
>> The code in question was added in:
>>
>> commit 9e87f1347788beed181476dc4563085db14a4729
>> Author: Lianhao Lu <lianhao.lu@intel.com>
>> Date:   Tue Aug 16 11:15:36 2011 +0800
>>
>>     toolchain-script.bbclass: Collected cached site config in runtime.
>>
>>     [YOCTO #892]
>>     Modify the function toolchain_create_sdk_siteconfig to collect the
>>     cached site config files which are specified by
>>     TOOLCHAIN_NEED_CONFIGSITE_CACHE in runtime.
>>
>>     Also added task dependency to ensure the cached site config files are
>>     generated.
>>
>>     Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
>>
>> ... but I don't entirely understand from that commentary what exactly is
>> special about ncurses that means it needs to be handled this way.  Can
>> anybody clarify?
>
> We ship the "site" cache files with the toolchain. We decided to include
> the generated cache files as well as the static ones. We currently
> generate "site" files for libc and ncurses.
>
> You can almost certainly just set:
>
> TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC}"
>
> and be happy since this just lists which generated cache site files to
> include.

Shouldn't it be pulled in via depends from the populate_sdk bits,
rather than a task that's used both for image builds and toolchain
builds, though?
-- 
Christopher Larson

Re: What is TOOLCHAIN_NEED_CONFIGSITE_CACHE for?

[Mark Hatle]

On 10/29/12 12:33 PM, Chris Larson wrote:
> On Mon, Oct 29, 2012 at 8:43 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> On Mon, 2012-10-29 at 13:33 +0000, Phil Blundell wrote:
>>> This morning I happened to notice that some/all of my images had a
>>> dependency on ncurses for no obviously good reason.  (Following an
>>> earlier small crusade these are now the only remaining dependencies on
>>> ncurses in my build, so I would quite like to get rid of them.)
>>>
>>> It turns out that the dependency in question is due to the code in
>>> toolchain-scripts.bbclass (which is inherited from
>>> populate_sdk_base.bbclass, from image.bbclass) which does:
>>>
>>> TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC} ncurses"
>>>
>>> [...]
>>>
>>> python __anonymous () {
>>>      deps = ""
>>>      for dep in (d.getVar('TOOLCHAIN_NEED_CONFIGSITE_CACHE', True) or "").split():
>>>          deps += " %s:do_populate_sysroot" % dep
>>>      d.appendVarFlag('do_configure', 'depends', deps)
>>> }
>>>
>>> The code in question was added in:
>>>
>>> commit 9e87f1347788beed181476dc4563085db14a4729
>>> Author: Lianhao Lu <lianhao.lu@intel.com>
>>> Date:   Tue Aug 16 11:15:36 2011 +0800
>>>
>>>      toolchain-script.bbclass: Collected cached site config in runtime.
>>>
>>>      [YOCTO #892]
>>>      Modify the function toolchain_create_sdk_siteconfig to collect the
>>>      cached site config files which are specified by
>>>      TOOLCHAIN_NEED_CONFIGSITE_CACHE in runtime.
>>>
>>>      Also added task dependency to ensure the cached site config files are
>>>      generated.
>>>
>>>      Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
>>>
>>> ... but I don't entirely understand from that commentary what exactly is
>>> special about ncurses that means it needs to be handled this way.  Can
>>> anybody clarify?
>>
>> We ship the "site" cache files with the toolchain. We decided to include
>> the generated cache files as well as the static ones. We currently
>> generate "site" files for libc and ncurses.
>>
>> You can almost certainly just set:
>>
>> TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC}"
>>
>> and be happy since this just lists which generated cache site files to
>> include.
>
> Shouldn't it be pulled in via depends from the populate_sdk bits,
> rather than a task that's used both for image builds and toolchain
> builds, though?
>

Ya, this seems like a design issue.  The site cache files require the package to 
be built, but I was under the impression they were included with the dev items 
within that package.  If this isn't how it works, maybe this is a change we 
should consider.

--Mark

Re: What is TOOLCHAIN_NEED_CONFIGSITE_CACHE for?

[Richard Purdie]

On Mon, 2012-10-29 at 10:33 -0700, Chris Larson wrote:
> On Mon, Oct 29, 2012 at 8:43 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >
> > We ship the "site" cache files with the toolchain. We decided to include
> > the generated cache files as well as the static ones. We currently
> > generate "site" files for libc and ncurses.
> >
> > You can almost certainly just set:
> >
> > TOOLCHAIN_NEED_CONFIGSITE_CACHE = "${TCLIBC}"
> >
> > and be happy since this just lists which generated cache site files to
> > include.
> 
> Shouldn't it be pulled in via depends from the populate_sdk bits,
> rather than a task that's used both for image builds and toolchain
> builds, though?

If we turned these into packaged output then we could do this, yes. As
things stand those files aren't packaged and hence there are no
dependencies from which to infer/pull these in. Converting them to that
isn't a particularly trivial exercise.

We currently only generate them for libc/ncurses which provide most of
the configure cache speedups we'd want so it hasn't been a particularly
pressing problem to solve.

I'm all for improving this and will happily take patches but its not
something on the immediate task list as there are higher priority issues
IMO.

Cheers,

Richard