[RFC] Add something like bitbake -cmenuconfig <recipe> ?

[RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Robert Yang]

Hello,

Is is useful/possible if we add something like bitbake <recipe> -cmenuconfig,
just like kernel's make menuconfig ?

We can use the menuconfig to config the vars such as MACHINE, DL_DIR,
DISTRO_FEATURES, MACHINE_FEATURES and all the variables which are
configurable, I think that this would help the newbie a lot.

I think that we can add a menuconfig.bbclass (or other names) to do this,
and I'd like to work on it.

-- 
Thanks

Robert

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Richard Purdie]

On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
> Hello,
> 
> Is is useful/possible if we add something like bitbake <recipe> -cmenuconfig,
> just like kernel's make menuconfig ?
> 
> We can use the menuconfig to config the vars such as MACHINE, DL_DIR,
> DISTRO_FEATURES, MACHINE_FEATURES and all the variables which are
> configurable, I think that this would help the newbie a lot.
> 
> I think that we can add a menuconfig.bbclass (or other names) to do this,
> and I'd like to work on it.

Why would you want to specify a <recipe> when configuring MACHINE? I
understand why you're thinking this but it isn't well thought out and in
this form would confuse users more than help them.

I don't think the system will even parse without a valid MACHINE, let
alone execute tasks.

Cheers,

Richard

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Robert Yang]

On 05/17/2015 05:34 AM, Richard Purdie wrote:
> On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
>> Hello,
>>
>> Is is useful/possible if we add something like bitbake <recipe> -cmenuconfig,
>> just like kernel's make menuconfig ?
>>
>> We can use the menuconfig to config the vars such as MACHINE, DL_DIR,
>> DISTRO_FEATURES, MACHINE_FEATURES and all the variables which are
>> configurable, I think that this would help the newbie a lot.
>>
>> I think that we can add a menuconfig.bbclass (or other names) to do this,
>> and I'd like to work on it.
>
> Why would you want to specify a <recipe> when configuring MACHINE? I
> understand why you're thinking this but it isn't well thought out and in
> this form would confuse users more than help them.
>
> I don't think the system will even parse without a valid MACHINE, let
> alone execute tasks.

Hi RP,

I meant that we need something to help configure the build easier, it
can generate something like local.conf.append, not configure the recipe.

The example "bitbake <recipe> -cmenuconfig" wasn't right enough, it's
just a rough thought, we can use the current default local.conf
(MACHINE = qemux86) to make system parse.

The problem is that we have many bbclasses in oe-core, a lot of them
has specify configurations, and also a lot of vars in the conf file such
as bitbake.conf, it's not easy to know how and what to config, especially,
for newbies. The "bitbake -cmenuconfig" maybe not a good idea, I think that
we need something to help config the build (generate local.conf) easier,
do you have any suggestions, please ?

// Robert

>
> Cheers,
>
> Richard
>
>
>

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Paul Eggleton]

Hi Robert,

On Monday 18 May 2015 09:52:50 Robert Yang wrote:
> On 05/17/2015 05:34 AM, Richard Purdie wrote:
> > On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
> >> Is is useful/possible if we add something like bitbake <recipe>
> >> -cmenuconfig, just like kernel's make menuconfig ?
> >> 
> >> We can use the menuconfig to config the vars such as MACHINE, DL_DIR,
> >> DISTRO_FEATURES, MACHINE_FEATURES and all the variables which are
> >> configurable, I think that this would help the newbie a lot.
> >> 
> >> I think that we can add a menuconfig.bbclass (or other names) to do this,
> >> and I'd like to work on it.
> > 
> > Why would you want to specify a <recipe> when configuring MACHINE? I
> > understand why you're thinking this but it isn't well thought out and in
> > this form would confuse users more than help them.
> > 
> > I don't think the system will even parse without a valid MACHINE, let
> > alone execute tasks.
> 
> I meant that we need something to help configure the build easier, it
> can generate something like local.conf.append, not configure the recipe.
> 
> The example "bitbake <recipe> -cmenuconfig" wasn't right enough, it's
> just a rough thought, we can use the current default local.conf
> (MACHINE = qemux86) to make system parse.
> 
> The problem is that we have many bbclasses in oe-core, a lot of them
> has specify configurations, and also a lot of vars in the conf file such
> as bitbake.conf, it's not easy to know how and what to config, especially,
> for newbies. The "bitbake -cmenuconfig" maybe not a good idea, I think that
> we need something to help config the build (generate local.conf) easier,
> do you have any suggestions, please ?

This is likely the direction we will be going in with the Toaster web UI - 
with a web-based tool we can present a much friendlier interface and have the 
chance to link to other information, for example we can link to the 
appropriate manual section for individual variables (and in future error 
messages, classes, etc.), analyse the output of the build, manage multiple 
sets of configuration, etc. These are things that would be difficult to do 
practically from the command line.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Otavio Salvador]

On Mon, May 18, 2015 at 5:45 AM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> Hi Robert,
>
> On Monday 18 May 2015 09:52:50 Robert Yang wrote:
>> On 05/17/2015 05:34 AM, Richard Purdie wrote:
>> > On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
>> >> Is is useful/possible if we add something like bitbake <recipe>
>> >> -cmenuconfig, just like kernel's make menuconfig ?
>> >>
>> >> We can use the menuconfig to config the vars such as MACHINE, DL_DIR,
>> >> DISTRO_FEATURES, MACHINE_FEATURES and all the variables which are
>> >> configurable, I think that this would help the newbie a lot.
>> >>
>> >> I think that we can add a menuconfig.bbclass (or other names) to do this,
>> >> and I'd like to work on it.
>> >
>> > Why would you want to specify a <recipe> when configuring MACHINE? I
>> > understand why you're thinking this but it isn't well thought out and in
>> > this form would confuse users more than help them.
>> >
>> > I don't think the system will even parse without a valid MACHINE, let
>> > alone execute tasks.
>>
>> I meant that we need something to help configure the build easier, it
>> can generate something like local.conf.append, not configure the recipe.
>>
>> The example "bitbake <recipe> -cmenuconfig" wasn't right enough, it's
>> just a rough thought, we can use the current default local.conf
>> (MACHINE = qemux86) to make system parse.
>>
>> The problem is that we have many bbclasses in oe-core, a lot of them
>> has specify configurations, and also a lot of vars in the conf file such
>> as bitbake.conf, it's not easy to know how and what to config, especially,
>> for newbies. The "bitbake -cmenuconfig" maybe not a good idea, I think that
>> we need something to help config the build (generate local.conf) easier,
>> do you have any suggestions, please ?
>
> This is likely the direction we will be going in with the Toaster web UI -
> with a web-based tool we can present a much friendlier interface and have the
> chance to link to other information, for example we can link to the
> appropriate manual section for individual variables (and in future error
> messages, classes, etc.), analyse the output of the build, manage multiple
> sets of configuration, etc. These are things that would be difficult to do
> practically from the command line.

Toaster is nice but we shouldn't stop improving cmdline use as the
first won't work for some use-cases.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Paul Eggleton]

On Monday 18 May 2015 08:52:04 Otavio Salvador wrote:
> On Mon, May 18, 2015 at 5:45 AM, Paul Eggleton
> 
> <paul.eggleton@linux.intel.com> wrote:
> > Hi Robert,
> > 
> > On Monday 18 May 2015 09:52:50 Robert Yang wrote:
> >> On 05/17/2015 05:34 AM, Richard Purdie wrote:
> >> > On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
> >> >> Is is useful/possible if we add something like bitbake <recipe>
> >> >> -cmenuconfig, just like kernel's make menuconfig ?
> >> >> 
> >> >> We can use the menuconfig to config the vars such as MACHINE, DL_DIR,
> >> >> DISTRO_FEATURES, MACHINE_FEATURES and all the variables which are
> >> >> configurable, I think that this would help the newbie a lot.
> >> >> 
> >> >> I think that we can add a menuconfig.bbclass (or other names) to do
> >> >> this,
> >> >> and I'd like to work on it.
> >> > 
> >> > Why would you want to specify a <recipe> when configuring MACHINE? I
> >> > understand why you're thinking this but it isn't well thought out and
> >> > in
> >> > this form would confuse users more than help them.
> >> > 
> >> > I don't think the system will even parse without a valid MACHINE, let
> >> > alone execute tasks.
> >> 
> >> I meant that we need something to help configure the build easier, it
> >> can generate something like local.conf.append, not configure the recipe.
> >> 
> >> The example "bitbake <recipe> -cmenuconfig" wasn't right enough, it's
> >> just a rough thought, we can use the current default local.conf
> >> (MACHINE = qemux86) to make system parse.
> >> 
> >> The problem is that we have many bbclasses in oe-core, a lot of them
> >> has specify configurations, and also a lot of vars in the conf file such
> >> as bitbake.conf, it's not easy to know how and what to config,
> >> especially,
> >> for newbies. The "bitbake -cmenuconfig" maybe not a good idea, I think
> >> that
> >> we need something to help config the build (generate local.conf) easier,
> >> do you have any suggestions, please ?
> > 
> > This is likely the direction we will be going in with the Toaster web UI -
> > with a web-based tool we can present a much friendlier interface and have
> > the chance to link to other information, for example we can link to the
> > appropriate manual section for individual variables (and in future error
> > messages, classes, etc.), analyse the output of the build, manage
> > multiple sets of configuration, etc. These are things that would be
> > difficult to do practically from the command line.
> 
> Toaster is nice but we shouldn't stop improving cmdline use as the
> first won't work for some use-cases.

Sure, I'm pointing out that this kind of thing is being worked on in a 
slightly different context, it's not that nothing is being done in this area.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 3825 bytes --]

On Mon, May 18, 2015 at 5:04 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

> On Monday 18 May 2015 08:52:04 Otavio Salvador wrote:
> > On Mon, May 18, 2015 at 5:45 AM, Paul Eggleton
> >
> > <paul.eggleton@linux.intel.com> wrote:
> > > Hi Robert,
> > >
> > > On Monday 18 May 2015 09:52:50 Robert Yang wrote:
> > >> On 05/17/2015 05:34 AM, Richard Purdie wrote:
> > >> > On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
> > >> >> Is is useful/possible if we add something like bitbake <recipe>
> > >> >> -cmenuconfig, just like kernel's make menuconfig ?
> > >> >>
> > >> >> We can use the menuconfig to config the vars such as MACHINE,
> DL_DIR,
> > >> >> DISTRO_FEATURES, MACHINE_FEATURES and all the variables which are
> > >> >> configurable, I think that this would help the newbie a lot.
> > >> >>
> > >> >> I think that we can add a menuconfig.bbclass (or other names) to do
> > >> >> this,
> > >> >> and I'd like to work on it.
> > >> >
> > >> > Why would you want to specify a <recipe> when configuring MACHINE? I
> > >> > understand why you're thinking this but it isn't well thought out
> and
> > >> > in
> > >> > this form would confuse users more than help them.
> > >> >
> > >> > I don't think the system will even parse without a valid MACHINE,
> let
> > >> > alone execute tasks.
> > >>
> > >> I meant that we need something to help configure the build easier, it
> > >> can generate something like local.conf.append, not configure the
> recipe.
> > >>
> > >> The example "bitbake <recipe> -cmenuconfig" wasn't right enough, it's
> > >> just a rough thought, we can use the current default local.conf
> > >> (MACHINE = qemux86) to make system parse.
> > >>
> > >> The problem is that we have many bbclasses in oe-core, a lot of them
> > >> has specify configurations, and also a lot of vars in the conf file
> such
> > >> as bitbake.conf, it's not easy to know how and what to config,
> > >> especially,
> > >> for newbies. The "bitbake -cmenuconfig" maybe not a good idea, I think
> > >> that
> > >> we need something to help config the build (generate local.conf)
> easier,
> > >> do you have any suggestions, please ?
> > >
> > > This is likely the direction we will be going in with the Toaster web
> UI -
> > > with a web-based tool we can present a much friendlier interface and
> have
> > > the chance to link to other information, for example we can link to the
> > > appropriate manual section for individual variables (and in future
> error
> > > messages, classes, etc.), analyse the output of the build, manage
> > > multiple sets of configuration, etc. These are things that would be
> > > difficult to do practically from the command line.
> >
> > Toaster is nice but we shouldn't stop improving cmdline use as the
> > first won't work for some use-cases.
>
> Sure, I'm pointing out that this kind of thing is being worked on in a
> slightly different context, it's not that nothing is being done in this
> area.


A good first step for both may be to add more typing information to the
metadata, so the UI can present something more intelligent than a bunch of
text boxes for text input. I expect that would be useful for any
configuration UI, whether toaster or console.

I’d certainly find a curses / menuconfig style interface to be interesting,
I’m sure there are folks who would find it useful, as long as it’s done
well.

My question would be, how does the UI determine what variables to present?
Do we hardcode the list of variables they might want to edit, or provide
that information in the metadata in some fashion?
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 4986 bytes --]

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Paul Eggleton]

On Monday 18 May 2015 08:15:05 Christopher Larson wrote:
> On Mon, May 18, 2015 at 5:04 AM, Paul Eggleton <
> paul.eggleton@linux.intel.com> wrote:
> > On Monday 18 May 2015 08:52:04 Otavio Salvador wrote:
> > > On Mon, May 18, 2015 at 5:45 AM, Paul Eggleton
> > > <paul.eggleton@linux.intel.com> wrote:
> > > > 
> > > > On Monday 18 May 2015 09:52:50 Robert Yang wrote:
> > > >> On 05/17/2015 05:34 AM, Richard Purdie wrote:
> > > >> > On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
> > > >> >> Is is useful/possible if we add something like bitbake <recipe>
> > > >> >> -cmenuconfig, just like kernel's make menuconfig ?
> > > >> >> 
> > > >> >> We can use the menuconfig to config the vars such as MACHINE,
> > > >> >> DL_DIR, DISTRO_FEATURES, MACHINE_FEATURES and all the variables
> > > >> >> which are configurable, I think that this would help the newbie a
> > > >> >> lot.
> > > >> >> 
> > > >> >> I think that we can add a menuconfig.bbclass (or other names) to
> > > >> >> do this, and I'd like to work on it.
> > > >> > 
> > > >> > Why would you want to specify a <recipe> when configuring MACHINE?
> > > >> > I understand why you're thinking this but it isn't well thought out
> > > >> > and in this form would confuse users more than help them.
> > > >> > 
> > > >> > I don't think the system will even parse without a valid MACHINE,
> > > >> > let alone execute tasks.
> > > >> 
> > > >> I meant that we need something to help configure the build easier, it
> > > >> can generate something like local.conf.append, not configure the
> > > >> recipe.
> > > >>
> > > >> The example "bitbake <recipe> -cmenuconfig" wasn't right enough, it's
> > > >> just a rough thought, we can use the current default local.conf
> > > >> (MACHINE = qemux86) to make system parse.
> > > >> 
> > > >> The problem is that we have many bbclasses in oe-core, a lot of them
> > > >> has specify configurations, and also a lot of vars in the conf file
> > > >> such as bitbake.conf, it's not easy to know how and what to config,
> > > >> especially, for newbies. The "bitbake -cmenuconfig" maybe not a good
> > > >> idea, I think that we need something to help config the build
> > > >> (generate local.conf) easier, do you have any suggestions, please ?
> > > > 
> > > > This is likely the direction we will be going in with the Toaster web
> > > > UI - with a web-based tool we can present a much friendlier interface
> > > > and have the chance to link to other information, for example we can
> > > > link to the appropriate manual section for individual variables (and
> > > > in future error messages, classes, etc.), analyse the output of the
> > > > build, manage multiple sets of configuration, etc. These are things
> > > > that would be difficult to do practically from the command line.
> > > 
> > > Toaster is nice but we shouldn't stop improving cmdline use as the
> > > first won't work for some use-cases.
> > 
> > Sure, I'm pointing out that this kind of thing is being worked on in a
> > slightly different context, it's not that nothing is being done in this
> > area.
> 
> A good first step for both may be to add more typing information to the
> metadata, so the UI can present something more intelligent than a bunch of
> text boxes for text input. I expect that would be useful for any
> configuration UI, whether toaster or console.

Indeed, this is something I'd like to see as well. There are already 
situations where we need to know the type of various variables (e.g. in 
buildhistory, where we have some of the variable typing hardcoded in the 
absence of proper definition within the metadata).

> I’d certainly find a curses / menuconfig style interface to be interesting,
> I’m sure there are folks who would find it useful, as long as it’s done
> well.

Sure, I wouldn't object to it either, but the important thing is someone not 
only needs to create it, but also maintain it in the future. Ideally it would 
also make use of some underlying definitions / code that would be re-usable 
elsewhere (as you're suggesting).

> My question would be, how does the UI determine what variables to present?
> Do we hardcode the list of variables they might want to edit, or provide
> that information in the metadata in some fashion?

I'd certainly prefer to define as much as possible in the metadata. Truth be 
told we have defined a bit more than I'd like in the way of OE-specific 
knowledge in Toaster itself; my intent was that we'd come back and clean up a 
lot of it later where practical, but of course that does mean improving what 
we expose on the metadata side, and sometimes that's really quite difficult.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [RFC] Add something like bitbake -cmenuconfig <recipe> ?

[Robert Yang]

On 05/18/2015 11:32 PM, Paul Eggleton wrote:
> On Monday 18 May 2015 08:15:05 Christopher Larson wrote:
>> On Mon, May 18, 2015 at 5:04 AM, Paul Eggleton <
>> paul.eggleton@linux.intel.com> wrote:
>>> On Monday 18 May 2015 08:52:04 Otavio Salvador wrote:
>>>> On Mon, May 18, 2015 at 5:45 AM, Paul Eggleton
>>>> <paul.eggleton@linux.intel.com> wrote:
>>>>>
>>>>> On Monday 18 May 2015 09:52:50 Robert Yang wrote:
>>>>>> On 05/17/2015 05:34 AM, Richard Purdie wrote:
>>>>>>> On Fri, 2015-05-15 at 10:35 +0800, Robert Yang wrote:
>>>>>>>> Is is useful/possible if we add something like bitbake <recipe>
>>>>>>>> -cmenuconfig, just like kernel's make menuconfig ?
>>>>>>>>
>>>>>>>> We can use the menuconfig to config the vars such as MACHINE,
>>>>>>>> DL_DIR, DISTRO_FEATURES, MACHINE_FEATURES and all the variables
>>>>>>>> which are configurable, I think that this would help the newbie a
>>>>>>>> lot.
>>>>>>>>
>>>>>>>> I think that we can add a menuconfig.bbclass (or other names) to
>>>>>>>> do this, and I'd like to work on it.
>>>>>>>
>>>>>>> Why would you want to specify a <recipe> when configuring MACHINE?
>>>>>>> I understand why you're thinking this but it isn't well thought out
>>>>>>> and in this form would confuse users more than help them.
>>>>>>>
>>>>>>> I don't think the system will even parse without a valid MACHINE,
>>>>>>> let alone execute tasks.
>>>>>>
>>>>>> I meant that we need something to help configure the build easier, it
>>>>>> can generate something like local.conf.append, not configure the
>>>>>> recipe.
>>>>>>
>>>>>> The example "bitbake <recipe> -cmenuconfig" wasn't right enough, it's
>>>>>> just a rough thought, we can use the current default local.conf
>>>>>> (MACHINE = qemux86) to make system parse.
>>>>>>
>>>>>> The problem is that we have many bbclasses in oe-core, a lot of them
>>>>>> has specify configurations, and also a lot of vars in the conf file
>>>>>> such as bitbake.conf, it's not easy to know how and what to config,
>>>>>> especially, for newbies. The "bitbake -cmenuconfig" maybe not a good
>>>>>> idea, I think that we need something to help config the build
>>>>>> (generate local.conf) easier, do you have any suggestions, please ?
>>>>>
>>>>> This is likely the direction we will be going in with the Toaster web
>>>>> UI - with a web-based tool we can present a much friendlier interface
>>>>> and have the chance to link to other information, for example we can
>>>>> link to the appropriate manual section for individual variables (and
>>>>> in future error messages, classes, etc.), analyse the output of the
>>>>> build, manage multiple sets of configuration, etc. These are things
>>>>> that would be difficult to do practically from the command line.
>>>>
>>>> Toaster is nice but we shouldn't stop improving cmdline use as the
>>>> first won't work for some use-cases.
>>>
>>> Sure, I'm pointing out that this kind of thing is being worked on in a
>>> slightly different context, it's not that nothing is being done in this
>>> area.
>>
>> A good first step for both may be to add more typing information to the
>> metadata, so the UI can present something more intelligent than a bunch of
>> text boxes for text input. I expect that would be useful for any
>> configuration UI, whether toaster or console.
>
> Indeed, this is something I'd like to see as well. There are already
> situations where we need to know the type of various variables (e.g. in
> buildhistory, where we have some of the variable typing hardcoded in the
> absence of proper definition within the metadata).
>
>> I’d certainly find a curses / menuconfig style interface to be interesting,
>> I’m sure there are folks who would find it useful, as long as it’s done
>> well.
>
> Sure, I wouldn't object to it either, but the important thing is someone not
> only needs to create it, but also maintain it in the future. Ideally it would
> also make use of some underlying definitions / code that would be re-usable
> elsewhere (as you're suggesting).
>
>> My question would be, how does the UI determine what variables to present?
>> Do we hardcode the list of variables they might want to edit, or provide
>> that information in the metadata in some fashion?
>
> I'd certainly prefer to define as much as possible in the metadata. Truth be

Has a uniform backend for configuration sounds like a good idea, here are
some rough thoughts that we can do in metadata, please feel free to give
your comments:

1) Add a conf file, bbclass or bb which contains the configable var list
    for the build, for example, DISTRO, MACHINES, DISTRO_FEATURES,
    PACKAGE_CLASSES and so on, we need maintain this file manually, we can
    discuss what is configurable in the mailing list, it may cost us a few
    time, but it is really good for OOBE, and would make the new user's
    life more easier.

2) Other layer can extend the configable var list easily.

3) Suppose we call the file config-build (.conf, .bbclass or .bb).

4) The UIs (toaster UI, web HOB, or menuconfig) can invoke the contents in
    config-build and display them.

5) The format of config-build can be:
    CONFIG_MACHINE[keys] = "machine1, machine2, ..."
    # The machines should be got automatically from a function.
    CONFIG_MACHINE[doc] = "xxx"
    # Re-use the MACHINE[doc] in meta/conf/documentation.conf
    [snip]

    CONFIG_IMAGE_FSTYPES[keys] = "typ1, type2, ..."
    CONFIG_IMAGE_FSTYPES[doc] = "xxx"
    CONFIG_IMAGE_FSTYPES[bbclass] = "image"
    # The required bbclass.
    [snip]

6) The result will be saved to a file such as local.conf.extend, and
    local.conf will include/require it.


Any comments is appreciated.

// Robert

> told we have defined a bit more than I'd like in the way of OE-specific
> knowledge in Toaster itself; my intent was that we'd come back and clean up a
> lot of it later where practical, but of course that does mean improving what
> we expose on the metadata side, and sometimes that's really quite difficult.
>
> Cheers,
> Paul
>

Re: [RFC] Add something like bitbake -cmenuconfig <recipe>?

[Paul Eggleton]

On Tuesday 19 May 2015 15:09:31 Robert Yang wrote:
> Has a uniform backend for configuration sounds like a good idea, here are
> some rough thoughts that we can do in metadata, please feel free to give
> your comments:
> 
> 1) Add a conf file, bbclass or bb which contains the configable var list
>     for the build, for example, DISTRO, MACHINES, DISTRO_FEATURES,
>     PACKAGE_CLASSES and so on, we need maintain this file manually, we can
>     discuss what is configurable in the mailing list, it may cost us a few
>     time, but it is really good for OOBE, and would make the new user's
>     life more easier.
> 
> 2) Other layer can extend the configable var list easily.

I'd rather we end up with the configuration being defined where the variable is 
actually used - it's tidier and easier to extend.

> 3) Suppose we call the file config-build (.conf, .bbclass or .bb).
> 
> 4) The UIs (toaster UI, web HOB, or menuconfig) can invoke the contents in
>     config-build and display them.
> 
> 5) The format of config-build can be:
>     CONFIG_MACHINE[keys] = "machine1, machine2, ..."
>     # The machines should be got automatically from a function.
>     CONFIG_MACHINE[doc] = "xxx"
>     # Re-use the MACHINE[doc] in meta/conf/documentation.conf
>     [snip]

MACHINE is such that we don't need to define the valid values for it - we can 
simply search for conf/machine/*.conf along BBPATH. (This is what Hob does, 
the mechanism it uses for finding conf files probably still exists.)
 
>     CONFIG_IMAGE_FSTYPES[keys] = "typ1, type2, ..."
>     CONFIG_IMAGE_FSTYPES[doc] = "xxx"
>     CONFIG_IMAGE_FSTYPES[bbclass] = "image"
>     # The required bbclass.
>     [snip]

Similarly here we already have IMAGE_TYPES which defines the valid choices for 
IMAGE_FSTYPES. Possibly not ideal, but it does already exist. On bbclasses, 
surely the usage for a class and any variables it supports ought to be defined 
in the class itself (see bug 6059), then it's easily applicable to any class 
outside of the core.

I think we need to step back a bit and think about how this should be 
implemented properly. I would urge you to consider that we went through a lot 
of this stuff already quite some time ago with Hob; not that that ended up as a 
perfect codebase by any means, but it would be worth learning from it.

As Chris was suggesting I'd rather we look at setting up a type declaration 
mechanism (possibly at the bitbake level?) and make use of that rather than 
creating something just for this UI where possible.
 
> 6) The result will be saved to a file such as local.conf.extend, and
>     local.conf will include/require it.

We went back-and-forth on this in the Hob timeframe; eventually it was 
realised that what people really wanted is to have the settings applied in 
local.conf itself rather than some UI-specific conf file. (On the other hand, 
thinking further ahead, we ought to be thinking about distro customisation and 
whether it's really appropriate to put every setting in local.conf as opposed 
to a custom distro config that you select via DISTRO.)

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [RFC] Add something like bitbake -cmenuconfig <recipe>?

[Robert Yang]

On 05/19/2015 04:35 PM, Paul Eggleton wrote:
> On Tuesday 19 May 2015 15:09:31 Robert Yang wrote:
>> Has a uniform backend for configuration sounds like a good idea, here are
>> some rough thoughts that we can do in metadata, please feel free to give
>> your comments:
>>
>> 1) Add a conf file, bbclass or bb which contains the configable var list
>>      for the build, for example, DISTRO, MACHINES, DISTRO_FEATURES,
>>      PACKAGE_CLASSES and so on, we need maintain this file manually, we can
>>      discuss what is configurable in the mailing list, it may cost us a few
>>      time, but it is really good for OOBE, and would make the new user's
>>      life more easier.
>>
>> 2) Other layer can extend the configable var list easily.
>
> I'd rather we end up with the configuration being defined where the variable is
> actually used - it's tidier and easier to extend.
>
>> 3) Suppose we call the file config-build (.conf, .bbclass or .bb).
>>
>> 4) The UIs (toaster UI, web HOB, or menuconfig) can invoke the contents in
>>      config-build and display them.
>>
>> 5) The format of config-build can be:
>>      CONFIG_MACHINE[keys] = "machine1, machine2, ..."
>>      # The machines should be got automatically from a function.
>>      CONFIG_MACHINE[doc] = "xxx"
>>      # Re-use the MACHINE[doc] in meta/conf/documentation.conf
>>      [snip]
>
> MACHINE is such that we don't need to define the valid values for it - we can
> simply search for conf/machine/*.conf along BBPATH. (This is what Hob does,
> the mechanism it uses for finding conf files probably still exists.)

Yes, as I said in the comment, it should be got automatically.

>
>>      CONFIG_IMAGE_FSTYPES[keys] = "typ1, type2, ..."
>>      CONFIG_IMAGE_FSTYPES[doc] = "xxx"
>>      CONFIG_IMAGE_FSTYPES[bbclass] = "image"
>>      # The required bbclass.
>>      [snip]
>
> Similarly here we already have IMAGE_TYPES which defines the valid choices for
> IMAGE_FSTYPES. Possibly not ideal, but it does already exist. On bbclasses,
> surely the usage for a class and any variables it supports ought to be defined
> in the class itself (see bug 6059), then it's easily applicable to any class
> outside of the core.
>
> I think we need to step back a bit and think about how this should be
> implemented properly. I would urge you to consider that we went through a lot
> of this stuff already quite some time ago with Hob; not that that ended up as a
> perfect codebase by any means, but it would be worth learning from it.

We can learn a lot from hob when we do the implementation.

>
> As Chris was suggesting I'd rather we look at setting up a type declaration
> mechanism (possibly at the bitbake level?) and make use of that rather than
> creating something just for this UI where possible.

A type declaration mechanism also sounds good, but I'm afraid that
it would make a lot of changes embed in the bbclass or conf files,
but the build system itself doesn't need these changes, I'm not sure
whether it is worth or not, our aim is just make OE easier for newbies.

// Robert

>
>> 6) The result will be saved to a file such as local.conf.extend, and
>>      local.conf will include/require it.
>
> We went back-and-forth on this in the Hob timeframe; eventually it was
> realised that what people really wanted is to have the settings applied in
> local.conf itself rather than some UI-specific conf file. (On the other hand,
> thinking further ahead, we ought to be thinking about distro customisation and
> whether it's really appropriate to put every setting in local.conf as opposed
> to a custom distro config that you select via DISTRO.)
>
> Cheers,
> Paul
>

Re: [RFC] Add something like bitbake -cmenuconfig <recipe>?

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]

On Tue, May 19, 2015 at 2:19 AM, Robert Yang <liezhi.yang@windriver.com>
wrote:

>
>> As Chris was suggesting I'd rather we look at setting up a type
>> declaration
>> mechanism (possibly at the bitbake level?) and make use of that rather
>> than
>> creating something just for this UI where possible.
>>
>
> A type declaration mechanism also sounds good, but I'm afraid that
> it would make a lot of changes embed in the bbclass or conf files,
> but the build system itself doesn't need these changes, I'm not sure
> whether it is worth or not, our aim is just make OE easier for newbies.


That’s not correct. Typing is useful to check the validity of values as
well as to simplify the creation of objects from strings in python code.
It’s not exclusively for a UI, that just happens to be an additional
benefit of using it.

I also agree with Paul, the knowledge of what variables should be visible
to the user in user interfaces should be held next to the variable
definitions in appropriate places, not a separate config file. A flag makes
a great deal of sense there. Specification of what variables we want the
user to be able to edit in the UI is really just a way of specifying which
variables are intended to be manipulated by the user, and this is valuable,
as it’d help the user know which variables are e.g. internal artifacts of
classes, not intended for them to modify. So while it’s used by a UI, I
don’t think that knowledge is specific to a UI either, necessarily.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2232 bytes --]

Re: [RFC] Add something like bitbake -cmenuconfig <recipe>?

[Bernhard Reutner-Fischer]

On May 19, 2015 10:35:36 AM GMT+02:00, Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
>On Tuesday 19 May 2015 15:09:31 Robert Yang wrote:
>> Has a uniform backend for configuration sounds like a good idea, here
>are
>> some rough thoughts that we can do in metadata, please feel free to
>give
>> your comments:

Will read the thread but the subject reminds me of http://lists.openembedded.org/pipermail/openembedded-devel/2011-January/074285.html
maybe?

Cheers :)
>> 
>> 1) Add a conf file, bbclass or bb which contains the configable var
>list
>>     for the build, for example, DISTRO, MACHINES, DISTRO_FEATURES,
>>     PACKAGE_CLASSES and so on, we need maintain this file manually,
>we can
>>     discuss what is configurable in the mailing list, it may cost us
>a few
>>     time, but it is really good for OOBE, and would make the new
>user's
>>     life more easier.
>> 
>> 2) Other layer can extend the configable var list easily.
>
>I'd rather we end up with the configuration being defined where the
>variable is 
>actually used - it's tidier and easier to extend.
>
>> 3) Suppose we call the file config-build (.conf, .bbclass or .bb).
>> 
>> 4) The UIs (toaster UI, web HOB, or menuconfig) can invoke the
>contents in
>>     config-build and display them.
>> 
>> 5) The format of config-build can be:
>>     CONFIG_MACHINE[keys] = "machine1, machine2, ..."
>>     # The machines should be got automatically from a function.
>>     CONFIG_MACHINE[doc] = "xxx"
>>     # Re-use the MACHINE[doc] in meta/conf/documentation.conf
>>     [snip]
>
>MACHINE is such that we don't need to define the valid values for it -
>we can 
>simply search for conf/machine/*.conf along BBPATH. (This is what Hob
>does, 
>the mechanism it uses for finding conf files probably still exists.)
> 
>>     CONFIG_IMAGE_FSTYPES[keys] = "typ1, type2, ..."
>>     CONFIG_IMAGE_FSTYPES[doc] = "xxx"
>>     CONFIG_IMAGE_FSTYPES[bbclass] = "image"
>>     # The required bbclass.
>>     [snip]
>
>Similarly here we already have IMAGE_TYPES which defines the valid
>choices for 
>IMAGE_FSTYPES. Possibly not ideal, but it does already exist. On
>bbclasses, 
>surely the usage for a class and any variables it supports ought to be
>defined 
>in the class itself (see bug 6059), then it's easily applicable to any
>class 
>outside of the core.
>
>I think we need to step back a bit and think about how this should be 
>implemented properly. I would urge you to consider that we went through
>a lot 
>of this stuff already quite some time ago with Hob; not that that ended
>up as a 
>perfect codebase by any means, but it would be worth learning from it.
>
>As Chris was suggesting I'd rather we look at setting up a type
>declaration 
>mechanism (possibly at the bitbake level?) and make use of that rather
>than 
>creating something just for this UI where possible.
> 
>> 6) The result will be saved to a file such as local.conf.extend, and
>>     local.conf will include/require it.
>
>We went back-and-forth on this in the Hob timeframe; eventually it was 
>realised that what people really wanted is to have the settings applied
>in 
>local.conf itself rather than some UI-specific conf file. (On the other
>hand, 
>thinking further ahead, we ought to be thinking about distro
>customisation and 
>whether it's really appropriate to put every setting in local.conf as
>opposed 
>to a custom distro config that you select via DISTRO.)
>
>Cheers,
>Paul

Re: [RFC] Add something like bitbake -cmenuconfig <recipe>?

[Trevor Woerner]

On 05/19/15 04:35, Paul Eggleton wrote:
> On Tuesday 19 May 2015 15:09:31 Robert Yang wrote:
>> Has a uniform backend for configuration sounds like a good idea, here are
>> some rough thoughts that we can do in metadata, please feel free to give
>> your comments:
>>
>> 1) Add a conf file, bbclass or bb which contains the configable var list
>>     for the build, for example, DISTRO, MACHINES, DISTRO_FEATURES,
>>     PACKAGE_CLASSES and so on, we need maintain this file manually, we can
>>     discuss what is configurable in the mailing list, it may cost us a few
>>     time, but it is really good for OOBE, and would make the new user's
>>     life more easier.
>>
>> 2) Other layer can extend the configable var list easily.
> I'd rather we end up with the configuration being defined where the variable is 
> actually used - it's tidier and easier to extend.
>
>> 3) Suppose we call the file config-build (.conf, .bbclass or .bb).
>>
>> 4) The UIs (toaster UI, web HOB, or menuconfig) can invoke the contents in
>>     config-build and display them.
>>
>> 5) The format of config-build can be:
>>     CONFIG_MACHINE[keys] = "machine1, machine2, ..."
>>     # The machines should be got automatically from a function.
>>     CONFIG_MACHINE[doc] = "xxx"
>>     # Re-use the MACHINE[doc] in meta/conf/documentation.conf
>>     [snip]
> MACHINE is such that we don't need to define the valid values for it - we can 
> simply search for conf/machine/*.conf along BBPATH. (This is what Hob does, 
> the mechanism it uses for finding conf files probably still exists.)
>  

Without knowing that others felt the same way, I've been playing around
with something (that is currently very rudimentary) along the same lines:

    https://github.com/twoerner/layer-repos

It was inspired by Otavio's "setup-environment", which Khem and Koen
have also started using for their new project to get Angstrom to use a
repo manifest.

My repo manifest (from above) includes a "setup" script which you can
source (if you want to configure a build) or run directly (if you want
to get lists of MACHINEs, DISTROs, etc).

If sourced, my setup script prompts the user for a machine, an sdk
machine, a distro, and the download directory location. These variables
are then put into conf/auto.conf, leaving a user's conf/local.conf for
true local, user configurations. The prompting occurs using one of
select, dialog, or whiptail.

Other things that could/should be configured include (in addition to
those already discussed):
- choice of compiler (gcc, linaro, external) and version
- choice of kernel (linux-yocto, ltsi, bsp, etc)
- PACKAGE_CLASSES
- EXTRA_IMAGE_FEATURES
- USER_CLASSES

I feel somewhat strongly that the proper way forward is to make repo
manifests (or something like it) an integral part of the build process
(or, at the very least, making layers more integral to the build!). In a
way that's what the whole "meta-poky" (i.e. git clone
http://git.yoctoproject.org/git/poky) is doing, isn't it? We're just
packaging openembedded-core, meta-yocto-bsp, meta-yocto, meta-hob, etc
_manually_ into one repository. Wouldn't it be less work to just switch
to using a repo manifest to pull them together instead of endlessly
applying patches twice? I'm glad to see lots of work going on in areas
like "bitbake-layers" since that sort of thing helps make layers part of
the process, instead of a separate step outside the process.

One thing that hinders progress is the fact that some layers don't play
nicely together. Some BSP layers, for example, assume you wouldn't ever
have more than one BSP layer in your build at a time. The same is true
of distro layers. Why would anyone ever have more than one distro layer
listed in conf/bblayers.conf at a time? Well, if you wanted to give your
users the possibility of choose which distro they want to use, then they
have to be available during the "lets have the user choose what they
want" phase. Then, in order to make things work, a lot of ad-hoc
knowledge needs to be built into the setup script so that it can shuffle
layers around depending on what the user chooses (i.e. remove all BSP
layers except the one the user chooses, remove all distro layers except
the one the user chooses, etc). Maybe the proper way forward is to ask
the user a bunch of questions and then pull layers together based on
their answers? This would require some way of knowing the list of all
possible machines, distros, etc. by somehow only fetching very little
from the Internet.

Is the metadata that drives
http://layers.openembedded.org/layerindex/branch/master/layers/
available someplace (preferably as a git clone)? If it were possible to
grab that metadata, then perhaps a setup script could use this
information to ask the user all the required questions then, once the
user choices were known, it could "bitbake-layers layerindex-fetch" the
required layers and get the build going (or at least be able to then ask
better questions!).

> I think we need to step back a bit and think about how this should be 
> implemented properly. I would urge you to consider that we went through a lot 
> of this stuff already quite some time ago with Hob; not that that ended up as a 
> perfect codebase by any means, but it would be worth learning from it.
>
> As Chris was suggesting I'd rather we look at setting up a type declaration 
> mechanism (possibly at the bitbake level?) and make use of that rather than 
> creating something just for this UI where possible.
>  
>> 6) The result will be saved to a file such as local.conf.extend, and
>>     local.conf will include/require it.
> We went back-and-forth on this in the Hob timeframe; eventually it was 
> realised that what people really wanted is to have the settings applied in 
> local.conf itself rather than some UI-specific conf file. (On the other hand, 
> thinking further ahead, we ought to be thinking about distro customisation and 
> whether it's really appropriate to put every setting in local.conf as opposed 
> to a custom distro config that you select via DISTRO.)

The configuration changes that are required to build, say, Wayland are
DISTRO_FEATURES, so why then don't we have a poky-wayland, poky-fb, or
poky-x11 distros in addition to the poky, poky-lsb, poky-tiny, and
poky-bleeding distros?